ed 299 883 he 021 795 title report to the new jersey board of … · document resume ed 299 883 he...

DOCUMENT RESUME

ED 299 883 HE 021 795

TITLE Report to the New Jersey Board of Higher Educationfrom the Advisory Committee to the College, OutcomesEvaluation Program.

INSTITUTION New Jersey State Coll. Outcomes Evaluation ProgramAdvisory Committee.

PUB DATE 23 Oct 87NOTE 44p.; Paper collected as part of the American

Association for Higher Education Assessment 1".-.)rum.

Appendices cited in the table of contents as aseparate document of five subcommittee reports arenot included with this report.

PUB TYPE Reports - Evaluative/Feasibility (142)

EDRS PRICE MF01/PCO2 Plus Postage.DESCRIPTORS *Accountability; College Faculty; *Educational

Assessment; Higher Education; Institutional Role;*Outcomes of Education; State Boards of Education;Statewide Planning; Student Development; *StudentEvaluation; Undergraduate Study

IDENTIFIERS *AAHE Assessment Forum; College Outcomes Assessment;*Mew Jersey

ABSTRACTThe College Outcomes Evaluation Program (COEP),

intended to be a comprehensive assessment of higher education in NewJersey, focuses on outcomes in order to improve undergraduateeducation. After an introduction covering the national perspective,New Jersey's efforts, COEP, and rationale and principles,subcommittee reports summarize the content of five reports, using theperspectives of students, faculty, and instituti;n. A glossary ofrelated terms is provided in each section. The 10 COEPrecommendations include: a common statewide assessment of generalintellectual skills should be developed for use by each institution;each institution should assess the specific outcomes of its generaleducation program; student development should be assessed at eachinstitution using common statewide definitions for each of severalindicators; and each institution should assess the outcomes of itsefforts in the areas of research, scholarship, and creativeexpression. A report by Rodney T. Hartnett, Rutgers University,offers a different perspective to the Advisory Committee'srecommendations. A minority report by K. Kiki Konstantinos,Superintendent of the Lenape School District, is included. Anaddendum looks at the board resolution and charge. Contains 12references. (SM)

***********************************************************************

Reproductions supplied by EDRS are the best that can be madefrom the original document.

***********************************************************************

"PERMISSION TO REPRODUCE THISMATERIAL HAS BEEN GRANTED BY

New Jersey Board of

Higher Education

TO THE EDUCATIONAL RESOURCESINFORMATION CENTER (ERIC)."

U.S. DEPARTMENT OF EDUCATIONOnce ot Educatronal Research and Imotov;:nent

EDUC °NAL RESOURCES INFORMATIONCENTER (ERIC)

his document has been reproduced asreceived from the person or organizanonoriginating it

C Mmor changes have been made to Improvereproduction quality

Points of view or opirilons stated in th.s docu-ment do nol nacessanly represent official0 ERI posMon or policy

REPORTTOTHE NEW JERSEYBOARDOF HIGHEREDUCATIONFROMTHE ADVISORYCOMMITTEETOTHE COLLEGEOUTCOMESEVALUATIONPROGRAMOCTOBER 23, 1987

BEST COPY AVAILABLE

One Dupont CircleSuite 600Washington. D.C. 20036202/293.6440

y 4:; "IA\

AMERICAN ASSOCIATIONFOR HIGHER EDUCATION

Board of Directors

OW7Joseph F. KauffmanUniversity of WisconstnMadison

asturEtestAdele S. SimmonsHampshire College

KUCluzu

Reatha Clark KingMetropolitan State University

Past ChauHarriet W. SheridanBrown University

Carlos H. ArceNu Stats. Inc.

Este la M. BensimonTeachers CollegeColumbia University

Anne L. BryantAmencan Association ofUniversity Women

Donald L. FreehlingMcGraw-Hill. Inc.

Ellen V. FutterBarnard College

Jerry G. GaffHamlin Uniyerstiv

Zelda F. sazssonUniversity of Michigan

Stephen R. GraubardDaratuto

Joseph KatzState Urns ersitv of New Yorkat Stony Bruck

Arthur E. LevineBradford College

Frank NewmanEducation Commissioncf the States

Alan Pifer-Casistelit-Ccwyorationof New York

W. Ann ReynoldsThe California StateUntverray

Picdad F. RobertsonMa t&-Dade CommunityCollege

D. Wayne SUbyCroupware Systems

P. Michael TimpaneTeachers CollegeColumbia University

PresidentRussell Edgerton

The AAHE ASSESSMENT FORUM is a three-year project supportedby the Fund for the Improvement of Postsecondary Education.It entails three distinct but overlapping activities:

--an apaual conference

(the first scheduled for June 14-17, 1987, in Denver) 4.

--commissioned papers

(focused on implementation and other timely assessment

concerns; available through the Forum for a small fee)

--information services(including consultation, referrals, a national directory,and more)

This paper is part of an on-going assessment collectionmaintained by the Forum. We are pleased to make it morewidely available through the ERIC system.

For further information about ASSESSMENT FORUM activities,contact Patricia Hutchings, Director, AAHE ASSESSMENT FORUM,One Dupont Circle, Suite 600, Washington, DC 20036

COLLEGE OUTCOMES EVALUATION PROGRAM

Advisory Committee

Stanley S. Bergen, Jr., ChairUniversity of Medicine & Dentistry

of New Jersey

Robert BarringerBrookdale Community College

Thomas BridgesMontclair State College

Paolo CucchiDrew University

William DalyStockton State College

Lewis DarsN.J. Department of Higher Education

Joan GirgusPrinceton University

Arthur GreenbergNew Jersey Institute of Technology

Anna H. GrovesKean College

Jerome C. Harris, Jr.City of Plainfield

Rodney HartnettRutgers, The State University

Nina JemmottNew Jersey Institute of Technology

Isabelle L KirchnerPrudential Insurance Company

K. Kiki KonstantinosLenape School District

Frederic F. KreislerN.J. Department of Higher Education

Kerry PerrettaN.J. Department of Personnel

Albert PorterMercer County Community College

Margaret RiveraBell Communications Research

Tanuni SchaefferRutgers, The State University

Robert ShaughnessyAT&T

Greta ShepherdMercer County School District

Vivien TamerRutgers, The State University

Nathan WeissKean College

College Outcomes Evaluation Program StaffNew Jersey Department of Higher Education

Edward A. Morante, Director

Novella Z. Keith, Assistant Director

Frances L Edwards, Program Officer

Steven L Johnson, Program Officer

Cynthia I. Gola, Program Secretary

Angela D. Carlisi, Program Secretary

STATE OF NEW JERSEYTHOMAS H. KEAN, GOVERNOR

Board. of Higher Education

Dr. Deborah P. Wolfe, Chair

Mr. Albert W. Merck, Vice-Chair

Dr. William 0, Baker

Mr. Martin S. Barber

Mr. Alfred W. Biondi

Mr. Michael Bongiovanni

Milton A. Buck, Esq.

Commissioner Saul Cooperman, ex-officio

Mrs. Anne Dillman

Dr. Marion Epstein

Rabbi Martin Freedman

Thomas H. Gassert, Esq.

Milton H. Gelzer, Esq.

Dr. Paul Hardin

Mr. Donald A. Peterson1

Mrs. Eleanor Todd

Cnance nor T. Edward Hollander, ex-officio

5

Community/Society Outcomes Subcommittee

Robert Barringer, ChairBrookdale Community College

Jerome C. Harris, Jr., Co-ChairCity of Plainfield

Felix J. AquinoAtlantic Community College


Flora Mancuso EdwardsMiddlesex County College

Anna H. GrovesKean College

Briavel HolcombRutgefs, The State University

Isabelle L KirchnerPrudential Insurance Company

Rose NiniMercer County Community College

Greta ShepherdMercer County School District

Ted TannenbaumGlassboro State College

Gary ThomasNew Jersey Institute of Technology

Research, Scholarship, and Creative Expression Subcommittee

Joan Girgus, ChairPrinceton University

Edward CafrunyUniversity of Medicine & Dentistry

of New Jersey

Paolo CucchiDrew University

Kathy FedorkoMiddlesex County College

Arthur GreenbergNew Jersey Institute of Technology


Casimir KulikowskiRutgers, The State University

6

Peter LindenfeldRutgers, The State University

Margaret MarshStockton State College

Gerald M. PomperRutgers, The State University

Donald E. StokesPrinceton University

Harold WeinbergExxon Research & Engineering Company

Elsa Nunez-WormackNew York College of Staten Island

Student Development/Post-Collegiate Activities Subcommittee

Rodney Hartnett, ChairRutgers, The State University

Carol BeerCentenary College

Ken BurenRutgers, The State University

Madan CapoorMiddlesex County College

John N. CrispStevens Institute of Technology


Lion GardinerRutgers, The State University

Jo Ann GoraFairleigh Dickinson University

Thomas GritesStockton State College

Syed HagueUniversity of Medicine & Dentistry

of New Jersey

James HarrisMontclair State College

Linda LyonsJersey City State College

Kerry PerrettaN.J. Department of Personnel

Margaret RiveraBell Communications Research

Tammi SchaefferRutgers, The State University

Samina ShahidiMiddlesex County College

Student Learning Outcomes Subcommittee

William Daly, ChairStockton State College

Barbara BershakRutgers, The State University

Thomas BridgesMontclair State College

Wade CurryTrenton State College

Ann HowardAT&T

Nina JemmottNew Jersey Institute ofTechnology

K. Kild KonstantinosLenape School District


Miles MacMahonEssex County College

Carlos A. PerezWilliam Paterson College

Albert PorterMercer County Community College

Joseph RosensteinRutgers, The State University

Vivien TartterRutgers, The State University

Nathan WeissKean College

Office of the President201/456-4400

NEW JERSEYUniversity of Medicine & Dentistry of New Jersey

MEMORANDUMOctober 23, 1987

TO: Dr. T. Edward Hollander, ChancellorDepartment of Higher Education

Members, Board of Higher Education

FROM: Dr. Stanley S. Bergen, Jr., ChairCollege Outcomes Evaluation Program Advisory Committee

SUBJECT: Report of the College Outcomes Evaluation ProgramAdvisory Committee

Administration Complex30 Bergen Street

Newark, New Jersey 07107-3007

It is with pride and satisfaction that the College Outcomes Evaluation Program (COEP) AdvisoryCommittee presents this report and its recommendations for your review. We thank you for theopportunity provided the committee to explore this issue of vital importance to the continueddevelopment, future stability, and well-being of higher education in New Jersey.

Since we first accepted your charge two years ago, we have investigated, debated, and reviewed manycomplex issues. Higher education is a diverse enterprise. Different sectors and institutions withinsectors have their own missions, goals, and populations. We believe we should build on this diversity,not diminish this strength.

Our discussions led us to conclude that there were also commonalities. All institutions enroll andeducate students, are concerned with scholarship, and impact society. We want our students to think,to solve problems, to read widely, and to write welL

In light of the need to balance the diversity of our institutions with their commonalities, the chargeyou gave us to develop an assessment program was difficult. As committee members, we haveexpressed diverse opinions. We do not pretend to have all the answers. What united us was a drivingfocus on excellence; we wanted New Jersey's system of higher education to be the best possible. AsGovernor Kean said at our second statewide conference on assessment last May

The assessment system that you and the Board of Higher Education are building mustsatisfy your own highest standards.... Design an assessment that is real, that hasintegrity, that you believe in. It has got to reflect the best traditions of higher learning.And you have got to live it in your work.

This is what we have tried to do. The recommendations we are making represent a consensus of ourefforts. Not every one of us agrees with every point, but as a whole, we believe that what werecommend is needed for our colleges and universities to improve. We believe that we have put asideour biases and special interests and joined to produce an assessment program that is viable, worthwhile,and significant. Based upon careful review, we have attempted to make observations and commentupon most issues we found central to your charge to us. Lack of comment and/or dissenting opinionshould not be viewed as complete concurrence, rather as consensus based upon hours of discussionand mutual respect.

We are not yet finished; we still have to implement those recommendations approved or amended bythe Board of Higher Education. We hope you will soon permit us to continue what we have begun. Ifyou agree that implementation should take place, we hope you will provide ongoing support for our efforts.

We seek collaboration with our colleagues across the state. Working together, we can attainexcellence for New Jersey higher education in the future.

Report to the New Jersey Board of Higher Educationfrom the Advisory Committee to the

College Outcomes Evaluation Program

TABLE OF CONTENTS

EXECUTIVE SUMMARY iii

I. INTRODUCTION .1

A. The National Perspective 1

B. New Jersey's Efforts 4

C. The College Outcomes Evaluation Program 5

D. Rationale/Principles 6

IL SUMMARY OF SUBCOMMITTEE REPORTS 9

A. Students 9

B. The Faculty 14

C. The Institution 16

III. RECOMMENDATIONS

BIBLIOGRAPHY

MINORITY REPORTS

ADDENDUM Board Resolution and Charge

APPENDICES

A. Community/Society Subcommittee Report

B. Research, Scholarship, and CreativeExpression Subcommittee Report

C. Student Development/Post-CollegiateActivities Subcommittee Report

D. Student Learning Subcommittee Report

E. Minority Report

20

25

26

30

Separate Document

Appendices werenot received by

ERIC.

i

EXECUTIVE SUMMARYThe College Outcomes Evaluation Program(COEP) is intended to be a comprehensiveassessment of higher education in New Jersey.Its focus is on outcomes; its purpose is theimprovement of undergraduate education. Thisreport explains why we should develop anassessment program and what we should assess.It is not intended to be a final report, but ratherthe first of a series of annual reports to theBoard of Higher Education on the success ofour colleges' and universities' efforts.

It has been two years since the first meeting ofthe COEP Advisory Committee. All of us,including the members of our four subcommittees,have devoted many hours in numerous meetingsto reach this point We have moved carefullyand deliberately for we are convinced thatthis effort demands that kind of pace and concern.But we are far from finished. Much work needsto be done in refining definitions, developinginstruments and measures, and, most importantly,in working with faculty and administrators at allof our institutions. We must continue what wehave begun: to implement a sound system ofassessment on each campus. We seekimprovement through active involvement andcommitment. Mere compliance and data collectionwill not be sufficient to produce the kind ofpost-secondary education system we are capableof providing and that students and taxpayersdemand and deserve.

A Time for ActionThe Board of Higher Education created COEPby resolution in June, 1985. An AdvisoryCommittee was appointed and charged withdeveloping a comprehensive assessment program.Student learning and development, facultyresearch and scholarship, and the impact ofinstitutions on society comprised the focus ofthe charge. Special emphasis was placed on thequestion of developing a sophomore test in verbalskills, quantitative reasoning, and critical thinking.

The Board's action followed a long tradition ofbold statewide initiatives. Two of these, inparticular, have strong .assessment components:the Educational Opportunity Fund and theBasic Skills Assessment Program. In addition,The Joint Statewide Task Force on Pre-CollegePreparation in 1983 specifically recommendeda sophomoilevel test.

Nationally, we have seen a number of significant

reports on the condition of higher education.While each has had its own priorities, a recurringtheme has been the assessment and improvementof undergraduate education. Governors inparticular have called upon states to play aconcerted role in this effort.

In New Jersey, Governor Kean has beenforthright in his support of higher education andin his insistence that the colleges demonstratethat they are worthy of that support. Includedin his program has been the Governor'sChallenge Grant which has prov'eled millionsof dollars to upgrade our public m.. 7-s anduniversities. While assessment is an impt.,..... Acomponent of all of these grants, Kean Collegeof New Jersey in particular has taken a leadershiprole in creating a broad-based, faculty-intensiveassessment of student learning.

The COEP Advisory Committee stronglybelieves that all of these factors combine tocreate a rare and powerful opportunity to improveour institutions. Most Americans are nowconvinced of the importance of a collegeeducation for intellectual growth, for greatercareer opportunities, and for the health of oursociety. The evidence is compelling to us thatthe twenty-first century will require higher levelsof ,kills and increased flexibility. Now isclearly the time to develop systematic methodsthat ensure the success of our efforts.

What Should We Assess?Over the past two years members of the AdvisoryCommittee and its four subcommittees havestruggled with the questions of what should beassessed and how. We have reviewed numerousdocuments, reports, and articles. We have calledupon out consultants and sought theadvice of colleagues. We have listened carefullyto faculty and staff from every one of ourcolleges at the two statewide conferences wesponsored and on other occasions. But mostimportantly we have tried to understand andexpress what we expect students to learn, whatis the rationale for research and related activities,and what impacts our institutions have andshould have on our state. We have concludedthat there arc many answers to these questionsand that the diversity of our enterprise is a sourceof strength and encouragement

Without completely defining what highereducation is all about, we addressed what was

10ill

important, the priorities of our endeavors.Rationale, definitions, measures, and proceduresfollowed. This process frequently took a cyclicalpattern that allowed us to return to earlierdiscussions and models as we sought to integrateour ideas and ultimately our recommendations.This process continues and must continue throughvarious stages of implementation. But we havereached a point where we are ready to reportand to recommend.

We decided to concentrate on defining andassessing the outcomes of higher education,what students learn, for example, not what theybring to us as they enter. The inputs (e.g., entrytest scores and high school rank, number ofbooksin the library, or the size of the endowmentfund), while traditional measures of quality, areinsufficient for determining how successful highereducation is in achieving its goals.

Chapter II of this document provides a summaryof the major areas addressed by each of our foursubcommittees. It does not do their reports justiceas no summary can fully capture the consideredthought and rich detail of each report. The readeris, therefore, referred to the full subcommitteereports included as appendices to this report.

The Subcommittee on Student Learning addressedthe current and future needs of our society interms of student learning in the areas of generalintellectual skills, modes of inquiry, appreciationfor the human condition/ethical issues, and themajor field of study. How students develop ashuman beings, their involvement in learning, aswell as how many students stay in college,achieve satisfactory grades, graduate, and whatthey do after they leave college are all areasaddressed by the subcommittee on StudentDevelopment/Post-Collegiate Activities.

Similarly, we examined the activities of facultymembers in terms of research, scholarship, andcreative expression. This subcommittee developeda comprehensive matrix of faculty activities,considering both the long- and short-termoutcomes of these activities as well as theiraudiences. We concluded that the traditionalmethods of counting publications and citationsor summing the dollars received from grants areinadequate measures of the richness of facultyresearch, scholarship, and creative expression.

Finally, we explored the impacts educationalinstitutions as a whole might have on variouspopulations. We constructed matrices ofoutcomes showing these different impacts

iv 1 i

on populations -tinging from the individualto the state.

In all this, we kept in mind the diversity of ourinstitutions, their individuality, and the goal ofexcellence in undergraduate education. It becameobvious that no single measure, no simplenumber, no assessment of isolated indicatorswould suffice to describe, much less measure,the success of our efforts. What we all felta priori, that multiple met! Ids and indicatorswere needed, was increasingly driven home byour discussions and our conclusions. Highereducation is a massive effort; any real assessmentof it must be complex, longitudinal, multi-variable. Consequently, the effort will take time,dollars, patience, and perseverance.

RecommendationsHigher education can be a powerful force forindividual growth and opportunity. Alreadyvital to the state's economy, it will becomeincreasingly so in the future. A trained andeducated citizenry is also a required ingredientof a democratic society. Excellence in highereducation must be our goaL

The COEP Advisory Committee is convincedthat a broad-based, comprehensive assessmentprogram can help achieve that goaL Improvementcan be accomplished best when we work togetherto learn how we are doing and what we can dobetter. Given the millions of dollars spent tocarry out our endeavors, we can do no less thanensure that our colleges and universities are thebest that they can be.

After two years of careful deliberation, theCollege Outcomes Advisory Committee makesthe following recommendations for subsequentaction (see also Chart I).

1. A common statewide assessment of generalintellectual skills should be developed foruse by each institution.

2. Each institution should assess the specificoutcomes of its general education program.

3. Faculty in each program, department,or discipline should assess students'leaming in each major course of studyprior to graduation.

4. Student development should be assessed ateach institution using common statewidedefinitions for eachofthe following indicators:

retention ratesprogram completion (includinggraduation) rates

grade point averagescredit completion ratioslicensure/certification exam resultspost-collegiate activities including

job/career informationfurther educationcommunity /professional involvement

5. Each institution should assess both thepersonal development of its students andthe degree of their satisfaction/involvementwith their institutions.

6. Each institution should assess the outcomesof its efforts in the areas of research,scholarship, and creative expression.

7. Using common statewide definitfons,each institution should assess its successin providing access and meeting theInman resource needs of its population, aswell as appraising its economic impact onthe community.

8. Based upon its mission and goals, eachinstitution should also assess its particularimpacts on the community it serves.

9. Provide all institutions with additionalfunding, guidelines and criteria, and

technical usistance needed to carry outthese recommendations.and technical assistance needed to carryout these recommendations.

10. In all lf these matters, involve faculty andadministrators intimately in the processwith a goal of commitment, not merecompliance. In this regard, create astanding broad -based COEP Council tocontinue the development of these efforts,oversee the collection and analysis of theinformation, and report regularly to theBoard of Higher Education.

Finally, we must stress the need for patience.The prominent models of assessment at singleinstitutions in other states took ten or more yearsto develop. What we are attempting toaccomplish is a statewide effort requiring muchdiscussion, interaction, compromise, andconsensus. Our goals can be accomplished, butit will take funding, hard work, perseverance,creativity, and especially patience on the partof many individuals. We are convinced thatwhat we propose is worth the effort.

IOutcomes Clusters/Variables

I. General Intellectual Skills-students' ability tofind, use, & present information/data; skills inanalysis, problem solving, critical thinking,quantitative reasoning, verbal abilities

2. General Education defined partly as(a) ability to understand & apply modes ofinquiry, (b) appreciate & confront enduringaspects of human condition, variety ofresponses to human issues & problems, andfashion reasoned ethical responses

3. Major field of study defined in terms ofobjectives/outcomes chosen by faculty in eachprogram/department

4. Indirect indicators of student learninga. Retention ratesb. Grade point averagesc. Credit completion rates

d. Program completion ratese. Licensure examsf. Students on academic probation/dismissedg. Reasohs for withdrawal

h. Graduate/professional school exams

CHART ICOEP VARIABLES

IIType/Source Of Data

Statewide assessment; test samples ofstudents for institutional assessment

Locally developed assessment defined partlyin accordance with statewide definition

Program-level assessment

Common definitions for (a) to (e)

Local definitions for (f) to (h)

13

IIICollection/Reporting Frequency

Periodic public reporting


Part of ongoing 5-year evaluation andreporting cycle for all programs

Periodic putic reporting for (a) to (e)

Periodic internal reporting for (f) to (h)

5. Student involvement and satisfactiona. Enrolled students' involvement

& off-campus activities

b. Enrolled students' satisfaction

6. Students' personal development

a. self-awareness

b. values

c. interpersonal relationships

d. leadership

7. Community/Society Impacta. Human Resource Development training/job

relates programs offered; projections onlabor force needs; employer needs andperceptions re quality of students

b. Access. percent of target subgroup membersadmitted as students &/or receivingservices, compared to demographics ofregion/community

c. Economic Impact: e.g., expenditures;economic contribution by institutionalemployees, students; data on costs deityservices; taxes

d. Local priorities

8. Research, Scholarship, & Creative Expression(e.g., dissemination of knowledge/methods/new discoveries to students, peers, business,& industry)

Locally defined

Locally defined

Institutions collect/analyze data, with commondefinitions and designs

Admission data from SURE; institutional datare participation in programs/services; surveysof needs assessment and perceptions of access

Institutions compile & analyze data;report to COEP

Locally defined

Defined in consultation with institutions;possible combination of statewide and locallyselected outcomes

14

Periodic internal reporting

Periodic internal reporting






-,.

"The central aim of allof these programs,including NewJersey's,is the strengtheningof instructionand educationalperformance in highereducation throughself-examination andself - improvement."

L INTRODUCTIONThe Advisory Committee of the College OutcomesEvaluation Program is proposing the creation of acomprehensive assessment program aimed both atpromoting educational excellence and strengtheningpublic confidence in higher education. According toEwell (1987), New Jersey is at the forefront of a growingnational trend. Many of the country's fifty states areaddressing or have already addressed the issue ofoutcomes assessment, and more states are becominginvolved every year. Though diverse in form andsubstance, the central aim of all of these programs,including New Jersey's, is the strengthening of instructionand educational performance in higher educationthrough self-examination and self-improvement.

The National PerspectiveIn 1983, the National Commission on Excellence inEducation shocked the nation with its report entitledA Nation at Risk The report stated that if any othercountry had done to us what we have done to oureducational system, we would have considered it an actof war. It questioned the standards being used byAmerican high schools and concluded that

the educational foundations of our society arepresently being eroded by a rising tide of mediocritythat threatens our very future as a Nation andpeople. (p. 5)

After a series of similar reports, it was not long beforeseveral began to focus on higher education. In 1984, theNational Institute of Education issued Involvement inLearning, which called for increased emphasis onundergraduate teaching and learning. The report argued

that institutions should be accountable not onlyfor stating their expectations and standards but forassessing the degree to which those ends.havebeen met. In practical terms, our colleges mustvalue information far more than their currentpractices imply. They should make a conscientiouseffort to acquire and use better information aboutstudent learning, the effects of courses, and theimpact of programs. (p. 21)

Th at Same year Secretary ofE ducation William J. Bennettreleased a report entitled To Reclaim a Legacy. Thereport expressed a concern that students lack "culturalliteracy" due to the growth of professional studies at theexpense of the humanities. This situation, Bennettasserted, is due both-to students pefceptions and choicesand to poor teaching, which is frequently the result ofnarrow research specialties. He believed that overly narrowspecializations have led to a vacuum of responsibilityfor the curriculum as a whole resulting in a loss ofcoherence and meaning. Bennett concluded that knowledge

15 1

"Successful economicdevelopment,internationalcompetition, schoolreform, and teacherpreparation all dependon excellence in under-graduate education.

i

2

itself should be at the core of the curriculum, and thatassessment should focus on determining whether thestudent has the required knowledge before graduation.

Early in 1985, the Association of American Collegesreleased a report entitled Integrity in the CollegeCurriculum, which argued that there is a crisis inAmerican education revealed in the decay in the collegecourse of study. The report cited "evidence of declineand devaluation" of a widespread nature.

The report listed the abilities which students should gainIn college: I) the ability to engage in inquiry, 2) abstract,logical thinking, 3) the ability to engage in criticalanalysis, and 4) the ability to read, write, and speak atlevels of distinction. An understanding of science, history,politics, the arts, and other cultures was also cited asnecessary for the college graduate.

The report recolmended that faculty members reassumeresponsibility for the whole curriculum, and that theyshould design and monitor mechanisms to evaluatestudentprogress in all desired abilities. It also recommendeda national program of support for the development ofsophisticated and reliable instruments of assessment.

In 1986, the Education Commission of the States,chaired by Governor Kean, released a report entitledTransforming the State Role in UndergraduateEducation. The report emphasized that successfuleconomic development, international competition, schoolreform, and teacher preparation all depend on excellencein undergraduate education. It contended that thenation's ability to maintain its leadership in a competitiveworld was at stake, and that undergraduate educationmust respond to these changing demands.

The report puts forth eight challenges to the highereducation community:

1. to prepare students for the wide range of opportunitiesoffered by a changing workforce and-society;

2. to improve students' preparation for college;3. to improve overall rates of college participation

and completion;4. to meet the educational needs of an increasingly

diverse student population;5. to build greater student involvement in the

undergraduate experience;6. to improve assessment of student and institutional

performance;7. to motivate faculty and reward them for improving

undergraduate education; and8. to carry outmore sharply defined institutionalmissions.

In order to meet these challenges, a transformationof the state role in higher education was necessary,and the committee made twenty-two specificrecommendations to state leaders, including the use ofmultiple methods of assessment to improve student andinstitutional performance.

16

"It seems safe toassume that assessmentis here to stay."

The importance of outcomes assessment in promotingeducational excellence is also being emphasized in anincreasing number of articles, reports, and conferences.For example, Derek Bok (in Toward Higher Learning,1986) argues that institutions will eventually need to assessthe effects of their programs if they are to sustain progress,while the 1986 National Governor's Association reportentitled Time for Results recommends comprehensiveassessment programs developed jointly by institutions andstate governments or coordinating boards. From a 1985meeting of hundreds of educators held in Columbia,South Carolina, a report entitled Assessment in AmericanHigher Education emerged in which various authorsconcluded that assessment programs, properly designed,could only benefit the academy. As Peter Ewell (1987)argues in a recent article, it seems safe to assume thatassessment is here to stay.

Some institutions and states have taken leadership rolesin developing assessment programs. Alvemo College inWisconsin, for example, has implemented a"competency-based" approach to assessment that sets specific goalsfor students in the areas of communication skills, analyticabilities, problem solving, making judgments, socialinteraction, individual/environment interaction, andunderstanding the contemporary world and the arts.Criteria for assessing these abilities are determined bythe faculty, and multiple means (including extensiveinterviews and use of "outside" evaluators) are used tojudge student performance.

Another institution, Northeast Missouri State University(NMSU), has implemented a form of "value-added"assessment of student performance. There, standardizedtests (e.g., ACT Assessment, ACT-COMP, or GRE),administered at several points in a student's career, yieldscores that measure academic growth. These scores aresupplemented with survey data on attitudes of students,faculty, alumni, and employers to give a clearer sense ofthe impact of a NMSU education.

A similar approach to student outcomes asssessment isused at the University of Tennessee, Knoxville (UTK).There, freshmen and seniors take the ACT-COMP,seniors are evaluated in their major fields, and studentopinion surveys are administered periodically. Again,the purpose is to approach student outcomes from avalue-added perspective.

The program at UTK is designed to satisfy the State ofTennessee's desire for data on student outcomes thatare tied into a statewide performance-funding program.In that state, institutions son clexnanSTate .their impacton students through outcomes assessment, and thefindings have implications for providing additional funds.

State legislatures and boards in other states have askedinstitutions to strengthen their outcomes assessmentprograms without mandating the specific details. These

3

"There is overwhelm-ing evidence thatassessment is a majornational focus, not onlyamong educators, butgovernors, legislators,and coordinatingboards/departments."

4 JE)

include the states of Colorado, Connecticut, Illinois,Minnesota, Missouri, South Dakota, and Virginia.

Florida has mandated that students must pass a test atthe end of their sophomore year before receiving anassociate degree at a two-year college or proceeding onto the junior year at a four-year college. While Georgiahas a similar program, according to Ewell (1987), fewother states are considering such a gateway (or "risingjunior") exam.

There is overwhelming evidence that assessment is amajor national focus, not only among educators, butgovernors, legislators, and coordinating boards/departments. Whether the emphasis is on accountabilityorimprovement, many leaders want increased informationon how dollars are being spent and how well studentsare learning. The traditional methods of assessing inputsare no longer sufficient. Neither are assertions, howeversincere, of trust, autonomy, diversity, or complexity.

New Jersey's EffortsThe State of New Jersey has been at the forefront of themovement to maintain and improve academic standardsat the undergraduate leveL Programs in various areas(e.g., the Basic Skills Assessment Program, theEducational Opportunity Fund, and the Governor'sChallenge Grant Program) have been developed tobenefit higher education throughout the state.

The New Jersey Basic Skills Asssessment Program wascreated by the Board of Higher Education in 1977 toassess the basic skills proficiencies of entering freshmenand to evaluate the effectiveness of each institution'sremedial efforts. A cornerstone of this program has beenthe administration of the New Jersey College Basic SkillsPlacement Test In addition, every public college mustassess the outcomes of its remediaLprogramsfollowingspecific statewide guidelines. Essential to these effortshas been the work of the Basic Skills Council and itsseveral subcommittees, which have provided a successfulmodel of collaboration between faculty members andthe Department of Higher Education.

Begun in the late sixties to foster access to highereducation for disadvantaged students, the EducationalOpportunity Fund Program (EOF) has providededucational and support services to thousands of studentsentering New Jersey colleges. In addition to specialclasses and summer programs, financial aid and counseling,EOF has developed an elaborate statewide system ofevaluation to ensure that funds are used appropriatelyand services provided effectively.

In 1983, the Joint Statewide Task Force on Pre-CollegePreparation issued a report defining the proficiencies inreading, writing, and mathematics needed for collegeand for life. Recommendations were also made calling

"COEP ... is intendedto provide feedback onhow higher educationis performing in ourstate and as a catalystfor improvement.

for periodic testing of skills and knowledge throughoutthe educational system, including an eleventh grade highschool graduation test and a test for college sophomores.

More recently, Governor Kean spearheaded the creationof the Governor's Challenge Grant Program to"challenge the institutions of higher learning in the stateto achieve excellence." With only one exception, everypublic four-year institution has received this specialfunding; two-year institutions have begun receivingsimilar grants. Assessment is a key element in all ofthese grant programs.

One of the first to receive a grant through the Governor'sChallenge was Kean College of New Jersey, whichreceived a three-year grant totaling $3.9 million ofwhich approximately $600,000 has been designated forassessment. Kean College has used these funds todevelop a faculty-intensive, campus-wide program todefine and assess student learning in general educationand in the major.

The College Outcomes Evaluation Program(COEP)COEP grew out of these successful New Jerseyexperiences and has been spurred on by the nationalmovement to assess higher education. Created by theBoard of Higher Education in June, 1985, it is intendedto provide feedback on how higher education is performingin our state and as a catalyst for improvement. TheBoard resolution (see Addendum) called for the creationof a comprehensive statewide assessment program.While much focus was placed on the development of asophomore test in critical thinking, quantitative reasoning,and verbal skills, the Board also sought assessment ofother areas of student learning as well as the outcomesof faculty research and the impact of institutions onsociety. An Advisory Committee was appointed to makerecommendations to the Board on how best to implementsuch an assessment program.

We first met in October, 1985, and soon realized thesize of our group (twenty-three) and the complexity ofthe task were too large for one committee. We dividedour efforts into four subtasks and created a subcommitteefor each, chaired by one of our committee members.Each of the following subcommittees was charged,appointed, and began meeting late in 1985:

(A) Student Learning focusing on direct measures ofcognitive student learning

(B) Student Development/Post Collegiate Activitiesalso addressing student learning but focusing moreon indirect measures, as well as the personaldevelopment of students, and the activities ofstudents after they leave their institutions.

(C) Research, Scholarship, and Creative Expressionstudying the activities and outcomes of faculty in

105

"We believe that theAmerican system ofhigher education,combining quality withaccess and diversity,is as good as anysystem anywhere in theworld. We also believeit can improve."

k

6

these three areas.(D) Community/Society Impact determining the

outcomes and impacts that institutions have on theircommunity or the society as a whole.

Each subcommittee regularly kept us informed with oraland written progress reports culminating in a "finalreport" from each group this summer. (Appendices A,B, C, and D provide the complete reports of the COEPsubcommittees.)

Rationale/PrinciplesHigher education in the United States has a long andrich history of providing broad-based education andtraining to its citizens. This tradition has served to providemost of the leaders of our society in many occupationsand fields. While this role of educating our leaderscontinues today, higher education plays a broader role(and will undoubtedly provide-an even greater part inthe future) in educating people in order to achieve jobsand levels of skills necessary for a successful economicbase. The evidence is growing that a state (or a country)cannot prosper, cannot successfully compete in markets,without a sufficient number of people with the level ofeducation our colleges and universities can and haveprovided (see especially the ECS report, HigherEducation and the American Resurgence, 1985).

We believe that the American system of highereducation, combining quality with access and diversity,is as good as any system anywhere in the world. Wealso believe it can improve.

Diversity is clearly an important strength of highereducation both nationwide and in New Jersey. We arestrongly committed to maintaining the strengths of thatdiversity. At the same time, there are commonalitiesacross our institutions. We can and do learn from eachother, we can and do expect a number of similar outcomes.In examining and strengthening our commonalities, weneed not and must not lose that diversity which isintegral to our system.

Assessment can play an important role in achievingexcellence. This is not a new concept, but the traditionalfocus has been on the inputs of the system. Popularindicators of excellence have been the quality of thestudents who enter (e.g., SAT or ACT scores and highschool rank), the degrees held by the faculty, the numberof books in the library, the tuition charged, and the sizeof the endowment fund. These factors, we believe, areinsufficient to measure how effective we are in educatingstudents and achieving our other goals. Rather, we needto focus on the outcomes of our endeavors: how manystudents stay in the institution and complete their program;how much do students learn, what they learn, and howmuch growth has taken place; what are the results of ourresearch activities; and what impact any institution has

20

Assessment is atechnique for demon-strating that our effortshave been worthwhile,that we are makingprogress, that we canaccomplish our goals.

"It is criticallyimportant to design anoutcomes assessmentprogram that allowsfaculty members andadministrators to useinformation to improvehigher education'sservices to itsstudents and to thesociety-at-large."

on a community. A focus on outcomes permits commonmethods of assessment without undermining autonomyand diversity of programs and institutions. What we areattempting to measure is complicated; no single measurewill do it. Rather our efforts in this area must be broad-based. To focus on a single variable, a few numbers, oreven attempting to quantify all aspects of our endeavorson some simplistic scale is contrary to good practices ofassessment. A simplistic approach will tell us little ofwhat we are achieving or what needs to be impri id.We are equally convinced that careful and crew,.thinking and planning, well conceived definitions andmethods, and collection of information on multipleoutcomes variables can and will give a reasonably clearsense of our accomplishments.

Assessment cannot be a one-shot effort. Longitudinal,ongoing, even continuous assessment is required foraccuracy, to allow for change, and to demonstrate_ growth.A long-term approach is also needed because changetakes place slowly and because we want assessment,feedback, and improvement to be an integral part of everyinstitution. Mere compliance is rarely effective and toooften counterproductive. What we seek is broad-based,ongoing commitment and involvement in our educationalsystem. As sess mentcan be a catalyst for thi s commitment;it can also be a tool for identifying where our efforts andpriorities should be placed. And finally, assessment is atechnique for demonstrating that our efforts have beenworthwhile, that we are making progress, that we canaccomplish our goals.

Thus, assessment serves dual purposes: accountabilityand institutional diagnosis. The former relates to theneed to demonstrate that the funding and resourcesprovided are worthy of public support, while the latterrelates to the importance of self - knowledge as the basisfor improvement.

It is critically important to design an outcomes assessmentprogram that allows faculty members and administratorsto use information to improve higher education'sservices to its students and to the society-at-large.Individual faculty members should be able to look atoutcomes data, relate the findings to their own pedagogy,and respond accordingly if they find aspects of theirteaching that need improvement. The entire college oruniversity community (faculty, administration, students,and alumni) should use outcomes data to examinecurricula at both the program and institutional levels.Similarly, all-inclusive discussions should be held on

The institution's obligations to its surrounding communityand the wider society of which it is a part.

In strongly endorsing the concept of continuousinstitutional evaluation for academic programimprovement, we are also aware of the need for thestate to evaluate institutional performance and to play a

21

7

"We believe thatstatewide assessmentdemands that multipleoutcomes measures beidentified for potentialstudy, and thatmultiple methods ofdata collection andanalysis be availablefOr use."

1

8

role in assessment. As noted above, this role may includecollecting and analyzing institutional information aboutstudents, reporting to the public and to decision-makers,providing external guidelines on some key areas ofinstitutional assessment, and perhaps most important,serving as a catalyst to motivate institutions. In this lastrole, the state also has an obligation to provide incentives,funding, and technical assistance to the institutions.

In the final analysis, both purposes for assessment areimportant, but a delicate balance must be maintained toavoid having outcomes evaluation become simply anotherreporting requirement (Ewell, 1987). As GovernorKean stated on May 8, 1987, at COEP's second,statewide conference on assessment in New Jersey:

No, this is not to be just a reporting requirement.What we seek together is stronger undergraduateeducation in New Jersey. The only point of theassessment system is to push us along that path.That means you have to use the information youget about performance.

Notwithstanding the needs of a statewide assessment,we are fully aware that each college has a unique history,mission, faculty, student body, and relevant public. Anystatewide effort must take this uniqueness into account.Because of our sensitivity to institutional uniqueness, onthe one hand, and our awareness of an appropriate staterole, on the other, we are convinced that the emphasisof a state effort should be to encourage and aid institutionsto carry out their own programs for assessment andimprovement. Thus, requirements for collecting uniformdata statewide should be relatively few in number andgeared to meet only the top priorities of the state.

New Jersey's outcomes assessment program must becomprehensive and flexible to capture the variety ofimpacts that higher education has on the state.Consequently, we believe that statewide assessmentdemands that multiple outcomes measures beidentifiedfor potential study, and that multiple methods of datacollection and analysis be available for use. A trulycomprehensive assessment program must rely on a varietyofmethods (e.g,. survey, interview, and direct observation)to yield both qualitative and quantitative information.

Clearly the development of an outcomes assessmentprogram is an exceedingly difficult task, one thatrequires breadth of vision, sensitivity, attention todetail, and, above all, patience. But we believe it isworth the effort. If properly done, outcomes assessmentcan be a powerful tool to improve undergraduate educationin New Jersey and the nation. And it can accomplishthis end without undermining the inherent strengths ofstudent heterogeneity and institutional diversity andautonomy. At the same time, an assessment programcan provide the state with the information it needs toassess the accomplishments of the system as a whole inareas of particular importance.

Five major areasof student learning

41.

II. SUMMARY OFSUBCOMMITTEE REPORTS

This chapter summarizes the twenty months of workand the resulting final report from each of our foursubcommittees. It does not do justice to the rich detailof their efforts. The reader is referred to the full reportscontained as appendices to this report to understandbetter and appreciate the rationale and definitionsdescribed here.

We have organized their work from three perspectives:students, faculty, and institution. Within each perspectivewe have tried to describe common outcomes areas anddiscuss, possible methods of measurement.

StudentsThere are undoubtedly many ways to define and classifystudent learning. One common way is to differentiatebetween cognitive and affective learning. Cognitivelearning focuses on such intellectual aspects as thinkingand analyzing while affective learning is more related tofeelings and personal development In truth, most learninghas components of both and a differentiation is mademore on emphasis. Any understanding, and thus anymeasurement of learning must appreciate the interactionof both cognitive and affective components.

We have chosen to focus on five major areas of studentlearning basic skills, general intellectual skills, modesof inquiry, appreciation of the human condition/ethicalissues, and the major. (Glossary 1 provides a briefdefinition of each; for a more complete description, seethe report of the Student Learning Subcommittee inAppendix D.)The basic skills of reading, writing, and mathematicsare the building blocks of a college education; indeedthey are necessary for life. These are the skills that astudent should possess when entering college. In thisregard, all incoming freshmen at New Jersey's publicand participating independent institutions take the NewJersey College Basic Skills Placement Test (NJCBSPT).This is a direct, standardized, external assessment ofpre-college reading, writing, and mathematics. Studentswho do not meet institutional standards are providedwith developmental (remedial) education, includingcoursework in those areas in which they are lackingsufficient skills. When they have completed remediation,students take a post-test to determine if they haveachieved-the proficiency needed-for college -level work.

The Advisory Committee believes that this Basic SkillsAssessment Program is appropriate and effective inassessing and requiring proficiency in basic skills beforestudents begin college-level courses. The recent Boardresolution on post-testing (February, 1987) furtherensures the basic skills proficiency of all entering students.

239

GLOSSARY I: AREAS OF STUDENT LEARNING

Basic Skills

GeneralIntellectualSkills

Modes of.Inquiry

These are skills in the areas of reading, writing, and mathematics that areprerequisites for college-level work. As such, they cannot be consideredcollege-level skills, in-and-of themselves. For students enrolled in remedialcourses, however, some of these proficiencies are learned and developed in acollege setting. In addition, college should maintain, expand, and refine thebasic skills proficiencies of ali students, whether or not they are enrolledin remedial courses.

These are skills necessary to critically analyze and utilize information(sometimes referred to as "higher order" skills). Specifically, they include theskills necessary to:

a) Accumulate and Exr.L,n'fie Information including the skills necessary to:determine the kinds of information needed for a given task; construct andimplement a systematic search procedure, using both traditional andcomputerized methods; discard or retain information based on an initial screeningfor relevance and credibility; and develop abstract concepts appropriate to thetask at hand for initially ordering the information which is retained.

b) Reconfigure, Think About, and Draw Conclusions from Informationincluding the skills necessary to: evaluate the interpretations presented byothers in terms of their assumptions, logical inferences, and empiricalevidence; reconfigure information in ways that suggest a range of alternativeinterpretations and evaluate their relative merits; construct hypotheses thatlogically extend thought from areas in which information is already availableinto areas where it is not; specify the additional information which mightconfirm or disconfirm those hypotheses; and draw conclusions based on allof the above.

c) Present Information including the skills necessary to express one's ownideas in written, oral, and graphic forms which will be intelligible andpersuasive to a variety of audiences.

a) The Rational Mode as used in all aspects of inquiry requiring exploration ofthe logical implications of a priori assumptions, and, in particular, the formalmanipulation and application of abstract models.

b) The Empirical Mode, including the

Experimental Method as used in all aspects of inquiry requiringobserzationand ;measurement of quantifiable variables under controlled conditions in thenatural environment or in the laboratory.

Comparative Method as used in all aspects of inquiry requiring observationand analysis of qualitative as well as quantitative variables under conditionsthat cannot be controlled or characterized completely.

Historical Method as used in all aspects of inquiry where explanationdepends upon analysis of change over time and understanding presupposes anact of synthesis of the forces that produce change.

c) The Intuitive Mode as used in all areas of human endeavor, includingthe scientific, in which non-linear/non-sequential thought processes (e.g.speculation, cogitation, and serendipity) lead to valuable insights, the creationof new paradigms, and "discovery," and where perception and expression arenot limited to the written or spoken word.

d) The Aesthetic Mode as used in areas where ideas of "beauty" in formand function gain a place of priority in the creative and evaluative processes,and where perception and expression are not limited to the written or spoken word.

10

24

Appreciationfor theHumanConditionandEthical Issues

Applicationto Study inDepth orThe Major

Refers to the appreciation of major continuities in the human experienceacross time and cultural boundaries. Such an appreciation will require: a) abreadth of knowledge about some of the major continuities apparent in the"natural" (biological and physical) world, in organized societies, and in theinterrelationship between the two; b) an understanding of the variety ofresponses human cultures have fashioned across time and space to some ofthe issues raised by those continuities; and c) an understanding that all thoseresponses reflect underlying values, and hence have an ethical dimensionwhich may help to inform the individual choices which students will have tomake in their own li..'es. These three kinds of understandings might bedescribed in greater detail as follows:

a) An understanding of the continuities in the human condition should include:

An understanding of the physical and biological world of the enduringissues which have been raised regarding the place of the human species in thatworld, and of the implications for those issues of current transformations ofour knowledge in these areas.

An understanding of organized societies, of the enduring issues which havebeen raised regarding the relationship of individuals to organized societies,and of the implications for those issues of the growing interdependence andinterpenetration of diverse societies and cultures.

An understanding of the implications for all of the above issues of ourgrowing technological ability to manipulate the physical and biological worlds,individual members of organized societies, and information about both.

b) An understanding of the variety ofresponses human cultures have fashionedto such issues should include:

Historical consciousness and, in particular, an understanding of the varietyof ways humankind has responded to enduring individual and social issuesover time

Both intracultural and intercultural understanding and, in particular, anunderstanding of the variety of ways humankind has responded to enduringindividual and social issues within and across cultures.

Aesthetic appreciation and, in particular, an understanding of the visualarts, the performing arts, and literature as commentaries on the humancondition across time and cultures.

c) The capacity to construct a reasoned personal response to ethical issuesno doubt depends on a broad range of factors, many of which lie outside thepurview of formal education. We believe, however, that the development ofsuch a capacity will be aided by understanding in all of the above areas, andby the self-understanding derived from it.

Refers to the ability to apply the intellectual capacities described above to adetailed and discipline° study of a specific area of knowledge and set ofproblems. This is an important justification for the traditional major. Thespecific capacities which should be developed will, of course, vary from majorto major.

11

25

"Students' generalintellectual skills aredeveloped and refinedat all of New Jersey'sinstitutions of highereducation."

1

12

26

General intellectual skills include analysis, problemsol.:ing, critical thinking, quantitative reasoning, andwren and oral expression. These skills are seen in a.student's ability to find, use, and present information.

Students' general intellectual skills are developed andrefined at all of New Jersey's institutions of highereducation. It does not matter which program or departmentstudents major in, or what degree or certificate theyultimately attain. These are broad-based, common skillsthat are necessary in all disciplines and fields.

Inextricably tied to these general intellectual skills aretwo key elements of general education that we refer toas Modes of Inquiry and Appreciation for the HumanCondition and Ethical Issues. Together, these representthe processes of learning and the broad range of knowledgewe expect our college and university graduates to possess.An ability to understand and use modes of inquiry helpsstudents t3 know how they and others arrive at conclusionsusing empirical, intuitive, or aesthetic processes.

An appreciation for the human condition and ethicalissues develops through wide study of the arts andletters, history, the social and natural sciences, and theprofessions. It entails knowledge of the "natural" worldand humanity's place in it. This appreciation also includeshistorical, inter- and intra-cultural, and aestheticconsciousness, as well as understanding the valuesinherent in all human action. In short, it is an appreciationof 1) the enduring issues facing humanity, 2) thevariety of ways humans have responded to these issues,and 3) the underlying ethical dimension associated withthese various responses to enduring issues.

Finally, we believe it is important for students to learnto apply these broader skills and understandings to thedisciplined study of a specific area of knowledge or setof problems. Work within a major field, with itsconcentrated focus and in-depth study, provides studentswith the special skills and experience they will need tosuccessfully address a variety of issues and solvepractical problems. The t Act skills and experiencesthat students acquire will, of course, vary with the fieldeach has chosen as ti,,: major.

In these latter four areas of st Jent learning that wehave identified, most measurement currently takes placethrough tests, projects, and term papers designed byindividual faculty members. These assessments and theassignment of grades generally occur within the frameworkof a particular course. Neither their development northeir scoring is usually evaluated by an external source.

For some students at some institutiors, assessment inthese areas includes a departmental exam, a senior project.or some similar capstone experience, an occasionalcollege-wide measure (e.g., a writing test), or a licensure/

"We believe thatcareful analyses ofappropriate informationabout stun Int learningcan better inform policydecisions at bothinstitutional andstatewide levels."

certification exam. Many students have none of theseexperiences or have them only in a particular course.Rarely are such assessments employed systematicallyto assess institutional or even programmatic effectiveness.

We also believe that students learn in areas whichinclude more affective components. We have definedthese areas as personal development. Understandingoneself, relating to others, leadership, and values are allaspects of student learning having both affective andcognitive aspects. (See the Student DevelopmentSubcommittee Report included in Appendix C forexamples of descriptions of these four personaldevelopment areas.) Other areas could also be identifiedsuch as philosophy of life, attitudes, and beliefs. Manycolleges and universities identify several goals ofpersonal development; rarely do they assess how wellthey meet their goals.

There appears to be little agreement on what areas ofpersonal development should be stressed or how ourinstitutions play a role in their learning. In addition,definitions are not well set and assessment not clearlydefined. Nevertheless, we believe that personaldevelopment does take place at college, that institutionsaffect this development, and that, therefore, there is aneed to assess how well these goals are being met.

A number of indicators provide important informationabout student learning although they are not classifiedas learning per se. Retention, for example, is certainlyimportant for student learning at least in assessinginstitutional impact on learning. Involvement in learningand in extracurricular activities as well as studentsatisfaction are all student-oriented indicators ofinstitutional effectiveness. Whether students completetheir programs and graduate, as well as what activitiesthey engage in after they leave their institution, can providerelevant feedback for institutional improvement. (SeeGlossary II for definitions of these indicators.) Otherimportant indicators might include results of graduate andprofessional school admissions tests, number of studentsplaced on probation, or reasons for withdrawing fromcollege. A comprehensive assessment program shouldexamine most, if not all, of these variables in order toconstruct a valid picture of an institution's effectiveness.

Learning takes place for all students before, during, andafter their college attendance as well as from both on-campus and off - campus experiences. The current stateof research does not allow clear differentiation of theinternal and external sources. What is clear, however, isthat students do change and grow and that colleges affectthis process. We believe that careful analyses .ofappropriate information about student learning canbetter inform policy decisions at both institutional andstatewide levels.

2713

GLOSSARY II: AREAS OF ACADEMIC PERFORMANCERetention Rate

Dr ...61datat ..a..111111.4.111111.1 116salb,

Grade Point Average

Credit Completion Rate

Licensure/CertificationExam Results

Post-Collegiate Activities

The number and percentage of an entering cohort ofstudents who are enrolled in the college of entry orany college in the state system.

The number and percentage of an entering cohort ofstudents who complete a program.

The average of grades received for college-level courses.

The ratio of college-level credits earned tocredits attempted.

The number and percentage of students who take andpass a licensure or certification exam.

Survey results from students after they leave aninstitution in the areas of job, further education; andcommunity involvement.

"Assessment offaculty outcomes mustbe broad-basedand inclusive."

14 28

The FacultyIn choosing the faculty as one of our three perspectives,we focused on their activities in the areas of research,scholarship, and creative expression. Further, we areinterested in assessing the outcomes of faculty endeavorsin these areas in the aggregate only. Nothing COEPassesses should relate to the evaluation of individualfaculty members.

A central theme running throughout the work of thissubcommittee is that faculty activities, and the outcomesof those activities, have been viewed too narrowly bythose involved with assessment at the institutional,divisional, and even departmental levels. We endorsethe view that assessment of faculty outcomes must bebroad-based and inclusi /e.

A partial listing of tf ese activities might indicate thatfaculty are expected

1. continue to learn and develop as academics(i.e., professional developmeni);

2. prepare courses and teach both skills and subjectmatter to their students;

3. contribute to discussions on the entire collegecurriculum;

4. provide formal and informal guidance to students inmatters of academic performance and personaldevelopment (i.e., advising);

5. contribute what skills and expertise they possess topromote the smooth functioning of the institution(i.e., college service);

6. discover, generate, propose, or create new data,reinterpretations of existing data, new technologies,new models/theories/paradigms, new works of art orliterature, or reinterpretations of existing works of artand literature (i.e., research and creative expression);

7. disseminate information (using expressive behaviorwhere appropriate) on these discoveries or creationsboth within and beyond the walls of the institution(i.e., publishing, presentation of findings, displays ofwork, or performances); and

8. contribute skills and knowledge to help solve practicalproblems facing a community, or to enhance thequality of life of that community or society as awhole (i.e., public/com.nunity service).

A list of the key audiences to whom these activitieswould be directed would include current students, formerstudents, peers (inside and outside the institution),college or university administrative staff, the generalpublic, the media, non-profit and voluntary organizations,business and industry, and government agencies (local,state, or national).

The "outputs" of these faculty activities can assumemany forms, from a lecture given in a classroom, to apainting, to an article published in a refereed journal, toa book. Outputs, in and of themselves, are not outcomes.Outcomes refer to the impact that these products offaculty activity have on the audiences to whom they aredirected. Outcomes may be immediately observed, suchas when a faculty production of a comedy results inlaughter and appreciation among an audience of students,peers, and the general public, or they may be felt bylarger audiences months, years, and even generationsinto the future (e.g. between the time a vaccine isdiscovered in the laboratory, and the time it seeswidespread use, saving human lives).

We believe that, while long-range outcomes may bemore important for the society as a whole, practicalitydemands that assessment efforts be focused on what wecall intermediate outcomes of research, scholarship, andcreative expression. These are more immediate withregard to time, and more circumscribed in their scope.An example of an intermediate outcome might beimproved research capability on the part of studentswho have benefitted from outputs (e.g. iectures anddemonstrations) of faculty teaching activities.

We have divided intermediate outcomes into twocategories: 1) those resulting from a process ofgeneration or creation, and 2) those resulting from aprocess of dissemination. Other outcomes may also beidentified as institutions and faculty focus their attentionon this issue.

As difficult and complex as the measurement of studentoutcomes is, we believe that measurement of the

2 15

1

"Traditional measuresof research, scholarship,and creative expressionare, by and large,inadequate.

"New ground is beingbroken, and both timeand broad participationwill be required beforesystematic assessmentis possible in this area."

16

outcomes of research, scholarship, and creativeexpression is far more problematic and complicated,even when intermediate rather than long-rangeoutcomesare assessed, and even without consideration of theadded complexity of diverse institutional missions andgoals. We are therefore convinced that outcomesassessment in this area must be approached cautiously,with wide participation of faculty members andinstitutional representatives.

Traditional measures of research, scholarship, andcreative expression are, by and large, inadequate. Atbest, they are too narrowly focused and mechanistic,emphasizing routine counts of faculty publications,grant monies, presentations, or performances. Facultymembers do more than simply publish, and theoutcomes of these activities are not always quantifiable.At worst, these traditional measures completely missthe mark in the sense that they are focused on easilyquantifiable outputs of facultractivities, rather thanbeing focused on the outcomes achieved as a result ofthese activities. To simply count publications or grantdollars attracted to the institution is not addressingoutcomes assessment adequately. Additional measuresand methods are called for.

The literature on outcomes assessment has not beenhelpful in supplying these needed measures andmethods. New ground is being broken, and both timeand broad participation will be required before systematic,assessment is possible in this area. Institutions andfaculty will need to identify priorities, help in the selectionof a small number of common measures, and participatein the development and testing of appropriatemethodologies for assessment.

We are convinced that it will ultimately be possible toassess the intermediate-range outcomes of research,scholarship, and creative expression. In the process, theoutcomes of these important activities of the faculty willbe more easily communicated to the public, institutionswill be more able todentify the extent to which theirgoals in this area are being met, and state policy in thisarea will be better informed.

The InstitutionThe mere presence of an institution of higher educationin a particular community has immediate and tangibleconsequences for everyone living within the college's oruniversity's service area. Individuals, groups (e.g. thosedefined by occupation, ethnicity, sex, or age), and,indeed, the entire population of the region (the societalperspective) will be affected. Some institutions fffectthe lives of people far distant from their campuses.Every aspect ofpeoples' lives will betouched to one degreeor another, from the purely economic, to the political,social, and cultural dimensions of existence. Student,

30

"Institutions varywidely in their missions. . . assessment musttake this diversity

IIsuit, tneirtnial

faculty, staff, visitor, and institutional spending willincrease business volume, stimulate growth, and createjobs. Training, educational, and cultural programs forboth students and community residents can help developa region's human resources, thus improving its business,political, and cultural climate, and making it an attractivelocation for both investors and people looking for aplace to settle and live. Social justice and equity can bepromoted by granting access to services andcourses tohistorically disenfranchised groups in society. In sho-,althoue.h its primary responsibility is to US students, webelieve that each institution can and should serve boththese students and the society at large.

Institutions vary widely in their missions; a communitycollege defines its community much more narrowly, forexample, than a state university. Assessment must takethis diversity into account. Nevertheless, we believe thatthere are three indicators of community/society impactcommon to all institutions: access, economic impact,and human resource development. (See Glossary III fora more complete description of these indicators.)

Several methods of data collection currently exist thatcan provide institutions with new outcomes data oninstitutional impact. A good example would be themethods developed by Caffrey and Isaacs (1971) foreconomic impact studies. Most institutions, too, alreadycollect a great deal of information about institutionaloutcomes. Some examples of existing data include recordsof college or university expenditures, results of needsassessment surveys, results of alumni surveys thatinclude information on alumni activities in the community,attendance figures for cultural and sporting events,enrollment statistics for non-credit courses, and figureson use of facilities and services.

Additional information exists outside the institution thatcan be readily used. Some examples include censusdata, data on taxes, historical data found in newspaper(or other) archives, and data on the more quantitativemeasures of quality of life (e.g. crime rates, availabilityof housing, or transportation statistics). These sourcesof data are fairly inexpensive to tap, and permit the"unobtrusive" measurement of community outcomes.In the areas common to all institutions (i.e., access,economic impact, and human resource development), avariety of measures could be used to assess outcomes. Forexample, measures of access might include: 1) admissionto the academic program or to non-creditcourses,training programs, and workshops; 2) admission to pre-college programs designed to enhance the likelihood ofsuccessful completion of college or university programs;3) provision of services such as health clinics, referralservices, or expert advice of faculty and staff (includingnumber of participants); 4) use of facilities such as alibrary, gymnasium, or meeting rooms by the public, or

31 17

"Analysis of outcomesdata on student learningand development,faculty activities, andcommunity/societyimpact must beconducted so as toreflect an integrationof perspectives.

18 044

sponsorship of cultural and sporting events; and 5) otheroutreach activities associated with recruiting, financialaid, or other services designed to make the institutionaccessible to the people of New Jersey.

Possible measures of economic impact include: I) directinstitutional expenditures (adjusted to reflect themultiplier effect); 2) indirect institutional expendituressuch as spending by students and visitors (adjusted toreflect the multiplier effect); 3) net effec,. in acommunity's tax base; 4) changes in a community'sstore of finance capital (i.e., savings); 5) changes inlevels of technology; 6) changes in the attractiveness ofan area to outside business interests; and 7) an institution'sown activities in the area of new venture developmentand entrepreneurship.

Possible measures in the area of human resourcedevelopment include the availability and quality ofI) professional training programs for students (includingnumber of participants); 2) workshops, seminars, ornon-credit courses for citizens already in the workforcedesigned to enhance their skills; 3) placement andreferral services matching employer needs with studentqualifications; 4) faculty and staff who lend expertise tocommunity leaders; and 5) data on the perceptions ofcorporate, government, and non-profit employersregarding an institution's programs.

In summary, we have examined the work of oursubcommittees through the student, faculty, andinstitutional perspectives. We wish to emphasize,however, that in reality these are overlapping andinterrelated perspectives. Data collection in the variousareas must be coordinated to avoid duplication of effort.More important, analysis of outcomes data on studentlearning and development,' aculty activities, andcommunity/society impact must be conducted so as toreflect an integration of perspectives.

This becomes critically important as we move from datacollection and analysis into the realm of policy planningand action. If interrelationships are not clearly understood(e.g., between faculty research and a community'squality of life), then it becomes difficult to recommendchanges that will bring about the ends desired (i.e.,excellence in both undergraduate education and serviceto the wider society). Only analysis that integrates theseperspectives will allow us to address fruitfully the concernswe may have about the critical processes of learning,development, teaching, research, scholarship, creativeexpression, and service to community and society.

GLOSSARY III: AREAS OF COMMUNITY/SOCIETYIMPACT

Access

Economic Impact

Human ResourceDevelopment

An outcome of efforts made by institutions to reach all ofNew Jersey's citizens, particularly those belonging totraditionally disenfranchised groups, in order to promote socialequity and greater civic involvement, improve the quality oflife, enhance status and mobility for individuals, and increaselevels of personal satisfaction that are related to increasedlikelihood of achieving personal goals.

Access, here, refers both to normal admissions to the academicprogram and access to other institutional services that maybe provided to community residents who may not seekadmission as students.

An outcome of an institution's participation in a local economythat results in, e. &, changes in economic production andproductivity, changes in business volume, changes in the locallabor market, and changes in levels of capitalization inindustry and the service sector.

Assessment in this area requires institutions to study boththe economic costs and the economic benefits of their presencein a particular community. Some factors that bear uponeconomic impact cannot be controlled by institutions(e. &, where and how much visitors, students, or faculty spend),but it is important to know their effects notwithstanding.

An outcome of an institution's efforts to provide optimaltraining to individual students about to enter the labor market,provide continued training to those already in the workforce,provide agencies and organizations with a trained laborforce, and enhance the overall levels of skills of a community'smost valuable resource, its people.

This is an area of economic impact over which institutionsexert a great deal of control through their curricula and specialprogram offerings. Ultimately, the institutions may wish toconsider the non-economic aspects of human resourcedevelopment, either within the context of community /societyoutcomes assessment, or under the rubric of student learningor student development outcomes.

3319

III. RECOMMENDATIONSAssessment can and should be an integral partof any institution. Periodic, ongoing evaluationof effort provides feedback both on areas ofaccomplishment and on places where reform isneeded. Even where goals are achieved, noinstitution should allow itself to becomecomplacent. High standards need to be constantlyscrutinized and maintainea. Changing conditionsneed to be understood and accommodated.Assessment provides information needed toachieve and maintain excellence. We make theserecommendations with these principles in mindand with a goal to make the New Jersey highereducation system the best in the country.

In making these recommendations, we againemphasize that this program asks for commitment,not mere compliance. We are convinced thatcommitment will only take place when largenumbers of individuals at each institution areinvolved in the process. Faculty members workingwithin their own departments must also workwith colleagues across the campus. Administrativeleaders must work with faculty and staff to makeeach institution aware of its particular mission,its strengths, and its areas needing improvement

The process we are proposing must take placeon each campus as well as sector-wide andstatewide. Collaborative efforts must beencouraged if this program is to succeed.

We believe that now is the time to act. Thedirection and support needed to carry out this

currently being provided. Themassive effort is ci y

Governor and the legislature, the Board and theChancellor, even the national climate, are alignedto focus on undergraduate education. Thisopportunity will not last forever.

The College Outcomes Evaluation ProgramAdvisory Committee has been meeting for twoyears. Over that time, we have reviewed severalinterim reports as well as a fmal report fromeach of our four subcommittees. We have alsoreviewed a variety of other reports from nationalagencies and groups, individual states, consultants,and staff. Two statewide conferences attendedby hundreds of New Jersey college faculty andadministrators provided review and commenton our work. We have read and we have thought;we have listened, debated and thought again.I've now make the following recommendations:

1. A common statewide assessment of general intellectual skills should bedeveloped for use by each institution.

General intellectual skills include the ability tofind, use, and present information and data.They include skills in analysis, problem solving,critical thinking, quantitative reasoning, andwritten and oral expression. All degree-seekingstudents should learn these skills regardless ofinstitution; they are, therefore, common acrossthe state. Consequently, we believe that astatewide assessment in these general intellectualskills is feasible and desirable.

No current test or assessment device adequatelyaddresses our needs in this area. We need tocreate assessment techniques that reliably andvalidly measure general intellectual skills.These techniques should resemble as much aspossible what students are required to do in theclassroom and in the world of work. At leasttwo years should be set aside to develop and

20 34

pilot such instruments. No statewide testingshould be undertaken before the 1989-1990academic year. Further, no results should bepublished until the instruments have beenappropriately validated.

The purpose of this assessment must beinstitutional evaluation; itspurpose shouldotbe to prevent individual students from progressingtoward a degree, nor to evaluate individualfaculty members.

As part of the two years of development andpiloting of instruments, special emphasis mustalso be placed on addressing student motivationand feedback, which and when students shouldbe assessed, various sampling procedures,"value-added" growth in student learning as wellas standards, and sensitivityin interpreting.andreporting the results.

2. Each institution should assess theeducation program.

Specifically, we recommend that institutionsmeet the following minimum criteria for assessinggeneral education:

(A) a clear definition of the objectives of itsgeneral education nroaram;

(B) an operational definition of the studentlearning outcomes associated with thoseobjectives, including the treatment of somecommon areas of concern as defined byCOEP in consultation with the campuses;

(C) specification of the methods and instrumentswhich will be used to collect informationon the degree to which those outcomeshave been achieved;

(D) .specification of the mechanisms for usingassessment findings to improve studentlearning, including possible modificationsof curriculum and pedagogy;a plan for systematic and on-goingassessment so that the effects of changesin curriculum and pedagogy can beevaluated over time; and

(F) evidence that the assessment plan for generaleducation has been approved by thecorporate faculty and will be given thenecessary support by the administration.

We recommend that individual institutions beasked to address, in the general educationoutcomes they assess, some common areas of

(E)

specific outcomes of its general

concern that will be defined by consultationsbetween COEP and the campuses. We recommendthat those consultations begin with a discussionof the "modes of inquiry" and" human condition/ethical issues" categories (see the StudentLearning Subcommittee Report in Appendix D).The goal should be the development of agenuine inter-institutional consensus.

In addition, we recommend that the Departmentof Higher Education provide a competitivegrant program to individual faculty members toencourage definition and assessment of generaleducation. This effort should begin with a focuson modes of inquiry and appreciation of thehuman condition/ezhical issues as .defmed_byour Student Learning Subcommittee.

Finally we recommend the following schedule:

(A) an interim planning report for generaleducation by May, 1988;

(B) an overall assessment plan for generaleducation by May, 1989;

(C) an interim report on assessing generaleducation, including definition, method ofassessment, and plans for implementationby May, 1990;

(D) a report to the Department of HigherEducation including results of generaleducation assessment by September, 1991.

3. Faculty in each program, department, or discipline should assess students'learning in each major course of study prior to graduation.

The faculty members of each degree-grantingprogram or department in the state should definethe objectives of their program especially interms of the resultant outcomes to be assessed.They should then define an assessment planincluding methods and measures, as well as howthe results will be used to improve curriculumand instruction. Collaborative efforts should beencouraged, especially among similar programsat different institutions.

We recommend that each institution meet thefollowing minimum criteria for developingassessment plans in each "major":

(A) a clear definition of the objectives ofthe major;

(B) an operational definition of the studentlearning outcomes (i.e., the effects of

instruction on students) associated withthose objectives;

(C) specification of the methods and instrumentswhich will be used to collect informationon those objectives;

(D) specification of the mechanisms for usingassessment findings to improve studentlearning, including possible modificationof curriculum and instruction;a plan for systematic and ongoingassessment, so that the effects of changesin curriculum and pedagogy can beevaluated over time; and

(F) evidence that the assessment plan hasbeen approved by the corporate facultyand will be given the necessary support bythe administration.

(E)

5 21

Finally, we recommend that each publicinstitution submit for review and approval to theCollege Outcomes Evaluation Program Counciland a team of outside consultants:

(A) an interim planning report for assessmentin the major by May, 1988;

(B) an overall institutional plan for assessmentin the major by May, 1989;

..mmilommImioNINIO

(C) information on field testing of that plan for atleast some degree programs, by May, 1990;

(D) a follow-up report by May, 1991, onprogress toward full implementation ofthe plan; and

(E) the results and utilization of outcomesassessment, thereafter, as a regular part ofthe current five-year evaluation cycle ford.., w programs.

4. Student development should be assessed at each institution using commonstatewide definitions for each of the following indicators:

retention ratesprogram completion (including graduation) ratesgrade point averagescredit completion ratioslicensure/certification exam resultspost-collegiate activities, including:

job/career informationfurther educationcommunity/professional involvement

Building on the work of our subcommittee, theDepartment of Higher Education shouldimmediately promulgate common definitionsfor each of these indicators. Wherever feasible,the Department's Student Unit Record System

(SURE) should be utilized to facilitate datacollection. Workshops and technical assistanceshould be provided where needed. Informationon these indicators should be collected annuallyfor each institution, analyzed by COEP, andreported to the Department of Higher Education,with the first report due by Fall, 1988.

In addition, each institution should supplementthe statewide indicators with the collection andexamination of information appropriate to itsmission. Examples of these include: number ofstudents placed on probation, reasons for leavingan institution, student satisfaction, andgraduate/professional school examination results.

5. Each institution should assess both the personal development of its studentsand the degree of their satisfaction/involvement with their institutions.

Each institution should review its mission in thearea of personal development of students anddefinelpecific objectives and priorities. Broad-based involvement within and outside theinstitution is strongly encouraged in a model ofcolllboration and sharing. Specific measures and

methods should be identified at each institutionto assess its success in achieving its goals andobjectives, with the results reported periodicallyto the Department of Higher Education. Aschedule of implementation similar to that forassessing the major is proposed.

3 622

6. Each institution should assess the outcomes of its efforts in the areas ofresearch, scholarship, and creative expression.

After a review of its mission, and after goals andobjectives are defined or revised, each institutionshould select four to seven outcomes prioritiesfrom the matrix created by our subcommittee.Using consultants as appropriate, the Departmentshould attempt to identify common outcomesand assist institutions in developing assessmenttechniques for these areas.

The institutions should identify their priorityoutcomes by the Spring of 1988, including theprocess by which they arrived at their selectionof priorities. Comprehensive reports should besubmitted to the Department of Higher Educationbeginning September, 1 990, acing erirprinsimilar to those for assessing the major.

7. Using common statewide definitions, each institution shouldassess its successin providing access and meeting the human resource needs of its population,as well as appraising its economic impact on the community.

Each institution has a common need to provideeducational and other services to the citizens inits service area. Special attention needs to befocused on those groups of the populationwhich have traditionally been underserved.Understanding the training and personnel needsof industry, within each county and the entirestate, is an essential ingredient of success inhigher education. In order to assess the outcomes

of their efforts in these two areas, of course,each institution will need to determine both itspopulation and its community's needs.

All institutions have an economic impact on theircommunities, although they often have littlecontrol over that impact We believe that it isimportant to know not merely the cost of runningan institution but also the impact thoseexpenditures have on the sun ounding community.

8. Based upon its mission and goals, each institution should assess itsparticular impacts on the community it serves.

Each institution should review its mission anddefine specific goals and objectives in the areaof community impact. Using the guidelinesdeveloped by our subcommittee (see Appendix A),each institution should identify its outcomes;audiences, and geographical areas and assess

the impact of its priority activities on thecommunity. Each institution should report tothe Department of Higher Education periodicallybased on a schedule similar to thatrecommended for assessing the major.

9. Provide all institutions with additional funding, guidelines and criteria, andtechnical assistance needed to carry out these recommendations.

The Department of Higher Education shouldprovide immediate start-up funding, as well asongoing funding to carry out theserecommendations. The ultimate goal, however,should be to include assessment as an integralpart of an institution and thus its budget

The Department should immediately prepareappropriate guidelines, definitions, and manualswhich follow the reports of our subcommitteesand offer workshops and technical assistance toour institutions in implementing ourrecommendations and establishing broad-basedassessment efforts.

3 7 23

10. In all of these matters, involve faculty and administrators intimately in theprocess with a goal of commitment, not mere compliance. In this regard,create a standing broad-based CQEP Council to continue the developmentof these efforts, oversee the collection and analysis of the information, andreport regularly to the Board of Higher Education.

(A) A COEP Council should be created whichwill oversee several subcommittees andreport annually to the Board of HigherEducation. Membership should be broad-based and reflective of both requisiteexpertise and various state constituencies.Appointed by the Chancellor, the membersshould include: three from the state colleges;three from the community colleges; threefrom Rutgers University; two from theindependent colleges; two from outsideconstituencies; one each from New JerseyInstitute of Technology, University ofMedicine and Dentistry of New Jersey,and the Department of Higher Education;and the College Outcomes EvaluationProgram director.

(B) Three subcommittees should be created withthe chair of each also serving as a memberof the Council to facilitate communication.These subcommittees should include:1. Student Learning and Development with

two subgroups, one focusing on theassessment of general intellectual skills,and the other focusing on assessment ofgeneral education, the major, andstudent development;

1 2. Research, Scholarship, and CreativeExpression which will focus on assessing

24

the outcomes of faculty activities inthese areas; and

3. Community/Society Impact which willfocus on assessing the areas covered byits tide as well as the post-collegiateactivities of former students.

(C) Current COEP members should be invitedto continue under the new structure in orderto provide essential continuity.

In addition to the formation of a COEP Council,the Department should:(D) encourage inter-institutional collaboration

on assessment among members of the samedisciplines (including the use of appropriateprofessional association services);

(E) encourage each campus to develop astructure for coordinating assessmentactivities appropriate to its individualneeds that includes both faculty andadministrators; and

(F) establish a formal statewide network thatwill offer campus representatives and othersinterested in assessment the opportunity tocommunicate readily with one another, aswell as the opportunity to be significantlyinvolved in shaping the statewide assessmenteffort on an ongoing basis.

BIBLIOGRAPHYAssociation of American Colleges. (1985). Integrity in the College Curriculum: A Report to the

Academic Community. Washington, D.C.: Author.

Astin, A., Panos, J., and Creager, E. (1967). National Norms for Entering College FreshmenFall 1966. American Council on Education lieseurdi Reports.

Bennett, W. (1984). To Reclaim a Legacy. Washington, D.C.: National Endowment for the Humanities.

Bok, D. (1986). Toward Higher Learning: The Importance of Assessing Outcomes. ChangeNov/Dec: 18-27.

Caffrey, J. & Isaacs, H. (1971). Estimating the Impact ofa College or University on the LocalEconomy. Washington, D.C.: American Council on Education.

Education Commission of the States. (1986). Transforming the State Role in UndergraduateEducation: Time for a Different View. Denver: Author.

Ewell, P. (1987). Assessment Where Are We? Change Jan/Feb: 23-28.

National Commission on Excellence in Education. (1983). A Nation at Risk The imperative forEducational Reform. Washington, D.C.: Author.

National Governors' Association. (1986). Timefor Results:The Governors' 1991 Report on Education.Washington, D.C.: Center for Policy Research and Analysis, NGA.

National Institute of Education. (1984). Involvement in Learning: Realizing the Potential ofAmericanHigher Education. Washington, D.C.: Author.

Newman, F. (1985). Higher Education and the American Resurgence. Princeton: CarnegieFoundation for the Advancement of Teaching.

United States Department of Education. (1985). Assessment in American Higher Education: Issuesand Contexts. Washington, D.C.: Author.

25

A DIFFERENT PERSPECTIVERodney T. Hartnett, Rutgers University

The Advisory Committee's report contains somerecommendations that might help achieve thegoal of improving undergraduate education inour colleges and universities. The report alsoincludes recommendations that I do not agreewith, however, and it therefore seemed to methat as a member of the Committee I have anobligation to make those disagreements both clearand public. Thus, the purpose of this statementis to describe several ways in which the viewsof one Committee member differ from those ofthe Committee at large.

I am concerned principally about therecommendations that deal with assessing studentlearning. One recommendation calls for acommon test of " general intellectual skills" tobe administered to students in all publiccolleges and universities , while two otherrecommendations offer fairly specific guidelinesfor the assessment of general education andlearning in each student's major field. The reportsuggests that such test-score information wouldprovide evidence that is necessary to makejudgments about institutional effectiveness(accountability), while at the same time supplyinstitutions with information that could be usedto improve the quality of education (institutionaldiagnosis). Unfortunately, nowhere does thereport address the many technical problemsthat will make fair, reliable interpretation of suchtest information very difficult, to say the least.In short, no indication is given of how suchinformation would actually be used to achieve

, either the accountability or the institutionalrdiagnosis goal.

In the case of the institutional accountabilitypurpose, how would the effectiveness ofInstitution A be compared to the effectivenessof Institution B if there are differences ininstitutional histories, student backgrounds,faculty training, and adequacy of facilities? It isnot sufficient to suggest testing students atdifferent points during their college years in orderto calculate a "growth" or "value-added" indexwhich could then be compared, for suchprocedures arefratightwithinterpretiveproblems.Educational researchers have been examiningthe question of differential college effecta foryears, and literally hundreds of research articlesaddressing the topic have appeared in professionaljournals. Though it would not be appropriatehere to describe the complex technical problems

26 40

involved, the general conclusion to be drawnfrom this research is that changes in student testscores do not lead to a reliable assessment ofcomparative institutional performance. Whatsuch a requirement may do, on the ether hand,is read to the unintended eunbcquen.:c Viinstitutions "teaching to the test"

With regard to using test score information toimprove education once again the report ignoresthe important question of how this would beachieved. A test score in this context can bev,,,likened to a physician taking a patient'stemperature. The temperature can indicatewhether jomething is amiss, but not what or c>how to treat it_The reports described in the introductory pagesof the Advisory Committee report, as well asnumerous other recent criticisms of Americanhigher educiaton, raise questions about thecurriculum (i.e., what is being taught or nottaught?), the teaching (how well is it taught?),and the students (who is being admitted, andmore importantly, awarded degrees?). Havinr,read and thought carefully about these variouscriticisms of higher education, and havingdiscussed many of them with my colleagues, Iam baffled by the suggestion that the way to beginto deal with any of those three general criticismswould be to develop a large, multi - institutionaltesting program. On the contrary, if there aregood reasons to doubt the integrity of ourcurriculum, let's reexamine our educational goalsand the substance of our curriculum as a meansof attaining those goals; if we have indicationsthat our teaching is not as good as it should r-could be, let's examine our teaching carefully,and change the ways that we evaluate and rewardit; and if there are recurring criticisms about theabilities of those we admit and confer degreesupon, by all means let's carefully scrutinize '7othour admissions policies and standards forevaluating student performance, being mindfulas we do of the effects changes might have onthe diversity of the population we would serve.In other words, let us respond to these variouscriticisms about the educational process by havinginstitutions attend directly to those criticisms,not by constructing some elaborate centralizedassessment program that won't tell us what wereally need to know.

As a general rule, I think we should be less

concerned about educational "outcomes" andmore concerned about educational quality. Thetwo are not synonymous. Information aboutoutcomes represents only one type of indicatorabout program quality. Other indicators ofprogram quality include various resources (e.g.,such "inputs" as ability of entering students,the credentials of the faculty) and educationalprocess characteristics (e.g., how good is theteaching). While resources, process, andoutcomes are all important, I would personallyplace much more value on resources and processas valid indicators of program quality.

I should like to emphasize that I do not opposetesting or assessment in general, and that, to thecontrary, I strongly favor the notion of institutionsmaking efforts to learn about their programs

and their students. It is for this reason that I agreedto serve on the COEP Advisory Committee inthe first place, and also why I fmd much in thisreport to be laudable. The worth of any testingor assessment procedure, however, must bejudged on the basis of a cost-benefit analysis:Did we learn something we didn't know before,and, if so, was this additional information worththe costs to those being tested as well as tothe testers of obtaining it? It seems to me thatthis is the central question here. What wouldthe assessment program proposed here tell usabout our institutions that we don't already knowor that we couldn't learn through more directmethods, and would the new information beworth the cost and attendant bureaucraticmachinery? I frankly don't think so.

4127

;

A MINORITY REPORTK. Kiki KonstantinosSuperintendent, Lenape School District

Appended to this report (see Appendix E) is aminority statement that calls for an open publicdiscussion of the desirability of a specific typeof basic skills examination for collegesophomores. This examination was recommendedunanimously in 1983 by a Statewide TaskForce on Pre-College Preparation comprised ofcollegiate and public school educators appointedby the Chancellor of Higher Education and theCommissioner of Education. However, sincethen, the proposal has not received serious orsystematic consideration in a forum thatencourages or even allows open public discussion.

The Statewide Task Force recommended that astandard statewide examination of Basic Skillsbe administered to all college students prior totheir junior year. The Task Force also urged thatstudents not be admitted to upper division (junior)status unless they demonstrate through theirperformance on this test that "they have verbalskills which are significantly higher than, andmathematical skills that are at least as high as,those expected of entering students to the college."

The minority statement appended to this reportdoes not present a detailed rationale for such asophomore examination. That rationale isimplicit in the intensive year-long effort of theTask Force. Instead, the minority report focuseson the more basic issue of the need to discussthe concept, in part because it has not beendiscussed in any depth, by the College OutcomesEvaluation Program Committee. Yet, there arestrong arguments for the adoption of such anexamination requirement.

First and foremost, the examination was proposedby the Statewide Task Force on Pre-CollegePreparation in the spirit of recognition that theproblem of skills-deficient college students isshared by bo th the public education and collegiatecommunities. As colleges open their doors tohigh school graduates with wider ranges of ability,it is imperative that public schools do a better

job of providing more of their students withspecial skills-needed-to sueceed-in college.However, the Task Force was well aware thatthe goal of preparing potentially all high schoolstudents for college will not be accomplishedeasily or quickly. In recognition of the fact, andof the role that college admissions policies haveplayed in generating the problem, members ofthe Statewide Task Force all agreed that the

28 42

higher education community must itself takesteps to require every student who enters college,particularly those with acknowledged skilldeficiencies, to demonstrate essential skills in aspecific way before they are permitted to takeupper-division courses.

Such an examination requirement is not viewedas a "penalty" to students but, on the contrary,as a means of ensuring that they are capable ofbenefiting fully from a baccalaureate educationas well as a means of guaranteeing the integrityof the diplomas they are awarded. The realinjustice to students would be to permit, withthe best of intentions, those who lack essentialcapabilities to invest tuition as well as their time,energy and aspirations in an experience fromwhich they derive little benefit. Nor is theproposed examination in any sense a transgressionupon academic prerogative of faculties to decidewhat and how to teach and the degree to whichstudents master the content ofcours es. It is aimedat providing a universal, basic assurance whichis different from, but no less essential, thansuccess in introductory courses.

The proposed examination is entirely consistentwith the existing policy by which collegefreshmen are tested on the NJCBSPT. Thatpolicy recognizes the validity of a statewidestandardized basic skills test as the primarymeans of determining college students' skillsdeficiencies as well as their readiness to takecollege courses. A major advantage of theproposed sophomore test would be identical tothat of the NJCBSPT it would help publicschool districts in their efforts to share the burdenof addressing the skills problem by sending astrong message that higher education is notonly committed to open access and remediation.It would communicate that colleges are alsoserious about requiring that all students meetminimal skills standards, even though thatcommitment may produce academic consequencesfor some students and therefore economicconsequences far r alleges. Such a commitmentdefines the differences between true accountabilityand the mere Illusion of accountability.

The proposed examination would generate otherbenefits as well and these are cited briefly onpage one of the attached minority report. Thetest would be a direct, and therefore a meaningful,gauge of the effectiveness of college remedial

programs. It would also be an effective screenfor students who transfer from communitycolleges, having met these relative standards bythose institutions, into four-year colleges, whichhave their own sets of relative standards.

For these and other reasons, the sophomoreexamination requirement recommended by theStatewide Task Force on Pre-College Preparationis indispensable to a sound accountabilitypolicy for higher e..sucation.

43 29

ADDENDUM: BOARD RESOLUTION AND CHARGE

The College Outcomes Evaluation Program(COEP) was created by resolution of the Boardof Higher Education in June 1985. The Board'sexplicit intentions in creating COEP were to:1) maintain public confidence in higher education,2) ensure continued support and funding forhigher education, 3) stimulate curricularimprovements, 4) nurture institutional autonomyand individual diversity, and 5) stimulateeducational excellence. An Advisory Committeecomposed of students, faculty, ulninistrators,and members of the business community,government, and the nonprofit sector, wasappointed by the Board upon the Chancellor'snomination. The Board resolution establishingthis committee contained the following statements:

That the said advisory committee becharged to study options and report to theChancellor its recommendations on howbest to design and institute a comprehensivesystem of evaluating the outcomes ofhigher education; and be it further resolved:

That the evaluation system shall includean assessment of students' learning throughthe administration of a test battery thatmeasures proficiencies in writing,quantitative reasoning, critical thinking,and any other areas appropriate for theevaluation of general college-levelacademic proficiencies. The tests are tobe taken toward the end of the sophomoreyear by all students attending publiccolleges and universities in New Jersey,and by students attending independentcolleges and universities that choose toparticipate. The test battery shall beconstructed so as to include the capacityto measure students' proficiencies in thebasic skills after two years of college andto permit comparison with their basic skillsproficiencies at college entrance asoriginally measured by the New JerseyCollege Basic Skills Placement Test Thetest battery may include some componentscommon to all institutions and othersspecific to individual institutions or groupsof institutions; ancrbe it further resolve&

That the said advisory committee becharged further:1. To consider the feasibility, design, and

implementation of the following potentialcomponents of the outcomes evaluationsystem. In addition to the specific

3044

elements outlined below, the committeemay recommend other elements asdeemed appropriate and feasible.

Student outcomes:

institutional development of specificgraduation requirements, includingsatisfactory performance oncomprehensive examinations to betaken before graduation in eachmajor field of concentration;development of measurements ofprogress toward the degree and post-graduation activities relevant toinstitutional assessment; particularattention should be paid to minorityretention and graduation as a measureof institutional performance.

Community-based/society-wideoutcomes:

establishment, at the state level, ofa matrix and guidelines for theevaluation of these outcomes;selection, at the local level, ofappropriate outcomes anddevelopment of instruments andpractices for their evaluation;identification of models for the soundand objective evaluation of theseoutcomes and development of policiesfor their use on a system-wide basis.

Feedback and information sharing:

review of information systemspresently in use and, if necessary,redesign so as to report the resultsof outcomes assessments in ways thatcan be used constructively;implementation of a unit recordsystem with standardized definitionsand reporting formats.

2. To recommend ways of implementingthe program, with specific reference tomethods of evaluating various distinctoutcomes and identification of thoseelements that should be centralized andthose that should be designed andadministered at the institutional level.

3. To recommend ways of utilizingincentives to improve performance.

4. To recommend an appropriatedistribution of the costs of the program.

ed 299 883 he 021 795 title report to the new jersey board of … · document resume ed 299 883 he...

Documents