slo focus on results

7/21/2019 Slo Focus on Results

1/73

Learning ProcessThe Teaching-

Learning Cycle:Using Student Learning Outcome Resultsto Improve Teaching and Learning

Workshop Activities &Resource Materials


2/73

Bill ScrogginsNovember 2004

2


3/73

Table of Contents

Student Learning Outcomes at the Lesson Level ............................................................................ 1Student Learning Outcomes at the Course Level: From Course Objectives to SLOs ............. ........ 2Primary Trait Analysis: Statements o !rading Criteria .................................................................. "Selecting the Assessment #ethod: Authentic Assessment and $ee% Learning ............................... &

'orming or (nter)*ater *eliability: Assuring Consistency o !rading Among Faculty ............. .... +The Assessment *e%ort: Sharing the *esults o Student Learning Outcomes ................................ +Program Level Student Learning Outcomes .................................................................................... ,$irect and (ndirect #easures o Student Learning Outcomes ....................................................... 1-(denti ying Program Com%etencies./0ternal and (nternal Sources ........................................ .... 11Strategies or $irect Assessment o Program SLOs: #osaic and Ca%stone A%%roaches .............. 11!eneral /ducation Student Learning Outcomes ............................................................................ 1Conclusion 1"

Appendices

A%%endi0 1 !ood Practices in Assessing Student Learning Outcomes33333333333333333333333333333333A%%endi0 2 Activity : 5riting Student Learning Outcomes ..................................................... 22A%%endi0 $evelo%ing and A%%lying *ubrics.......................................................................... 2A%%endi0 " /0am%les o Scoring *ubrics .................................................................................. 2+A%%endi0 4 Activities " 6 4: 7uilding and 8sing a !rading *ubric ......................................... 2,A%%endi0 & The Case or Authentic Assessment by !rant 5iggins ......................................... -A%%endi0 9 )) State and 'ational Standards Academic 6 ;ocational Com%etencies ............... 2A%%endi0 + Assessment *e%ort /0am%les ................................................................................. &A%%endi0 , Assessment Plan /0am%les (nternet Sites ............................................................... "-A%%endi0 1- Activity 4 Program SLOs rom Com%etency Statements .................................. "1A%%endi0 11 /0am%les o Program Assessment *e%orts3333333333333333333333333333333333333333

A%%endi0 12 !eneral /ducation Student Learning Outcomes333333333333333333333333333333333333333A%%endi0 1 .*esources and *e erences or Student Learning Outcomes Assessment3333333333333342

/ndnotes33333333333333333333333333333333333333333333333333333333333333333333333333333333333

8*L or this document: htt%:


4/73

The Teaching'(earning C&cle)sing Student (earning *utcome +esults to !mprove Teaching , (earning

Since the Accrediting Commission identi ied ?measuring student learning outcomes@ as theocus o the latest revision o the 5ASC standards many o us have been struggling =ith =hat

=e are e0%ected to do di erently3 5hatever =e do to im%lement Student Learning Outcomesthis initiative must be seen to add value to the teaching and learning %rocess.value that clearlyout=eighs the tas> o constructing SLOs3 Those o us =ho have taught or years consider that =ealready measure student learning3 o=ever ( have come to believe that SLOs really do have ane= and use ul em%hasis that can be best ca%tured by one =ord: results .collecting themsharing them and using them to im%rove both learning and the o%eration o our colleges3 Thisseries o re lections are intended to address getting use ul results rom the SLO %rocess. ma0imiBing utility and minimiBing utility3 That little ? @ really ma>es a di erence doesnDt itE

Student (earning *utcomes at the (esson (evel

As =e teach each lesson and grade the related studentassignments =e ty%ically have a clear conce%t o theresults e0%ected and =e have de ined methods or assessing student =or> and assigning grades3

o=ever there are several things that =e ty%icallydonDt do that can %otentially im%rove studentlearning3 5hile many o us do give students =rittenlearning objectives or each lesson =e usually do notwrite down criteria for grading nor share them with

students .other than ho= total %oints relate to theinal grade in the course3

(n listening to %ractitioners o SLOs such as Lisa7re=ster a S%eech teacher at San $iego #iramar College and Ganet Ful>s a #icrobiology teacher at7a>ers ield College it is clear that SLOs can becomea %o=er ul pedagogical tool by:

sharing grading criteria =ith students getting students to use these criteria as a =ay

to better understand the material and having students evaluate their o=n and each

otherDs =or>3

Activity 1(n small grou%s by disci%line or cluster o related disci%lines discuss ho= you develo% grading criteria3

$o you =rite do=n your grading criteria or each assignmentE o= consistent are you in a%%lying your grading criteriaE $o you use the results o student assessment to im%rove your grading criteriaE $o you communicate your grading criteria to studentsE 7e ore or a ter the assignmentE $o you encourage students to a%%ly the grading criteria to their o=n =or>E $o you involve students in develo%ing or modi ying your grading criteriaE

*esults o SLOs at the Lesson Level#ost o us have criteria or grading studentlearning or the individual objectives o each lesson =e teach.but =e may not=rite these do=n or share them =ithstudents3 ereDs an e0am%le:

Lesson Learning Objective : $escribe anddra= the our vibrations o carbon dio0ideand sho= ho= (* light is absorbed by CO 23Sample Graded Question : 5hat t=o ty%eso motion are caused by the absorbance o (* light by CO 2E $ra= one o thesemotions3Grading Criteria :Full credit: Student names ?bending@ and?stretching@ and dra=s ball)and)stic> models =ith arro=s u%)and)do=n or

bending and side)to)side or stretching3$eductions: 24H or one name missingI4-H or bothI 24H or =rong or missingarro=sI 4-H or no dra=ing3

Results of Grading Student Work :+2H earned ull creditI 4H con usedarro=sI 1 H had no dra=ing3

ction to !mprove Learning :The greatest de iciency seems to bedra=ing so do in)class dra=ing e0ercise3


5/73

$o you share your grading criteria =ith other acultyE

Aggregating the eedbac> rom grading student assignments can %rovide valuable insight intoareas in need o im%rovement3 5ith all the demands on our time =e may not give adeJuateattention to mining this valuable source o in ormation or im%rovement o the teaching andlearning %rocess3 One o the challenges that the ne= accreditation standards %resent is creatingan assessment plan that outlines objectives grading criteria results o assessing student =or>and ho= =e use those results to im%rove student learning3

O course %art o im%roving student learning is im%roving the =ay =e teach3 This inevitableoutcome can %otentially be threatening to aculty members3 o=ever =hen these issues have

been raised in =or>sho%s =ith aculty the result has generally been a serious engagement indiscussions o teaching methods to im%rove authentic dee% learning3 (t is e0tremely im%ortant to

build environments or discussing the im%rovement o student learning =hich are %ositive andrein orcing3 Several colleges have made e0%licit commitments to this %rinci%le3 The endnotere erences include a%%roaches by Palomar College in Cali ornia College o $uPage in (llinoisand the American Association o igher /ducation3

ctivit" #*ead the ollo=ing resource documents see A%%endi0 1 and join in the grou% discussion on ?!ood Practices or Assessment o Student Learning Outcomes3@

?An Assessment #ani esto@ by College o $uPage (L?, Princi%les o !ood Practice or Assessing Student Learning@ by AA /?Palomar College Statement o Princi%les on Assessment@ rom Palomar College CA?Closing the Loo%.Seven Common #is Perce%tions About Outcomes Assessment@ by Tom Angelo?Five #yths o KAssessmentD@ by $avid Clement aculty member #onterey Peninsula College

Student (earning *utcomes at the Course (evel From Course *b-ectives to S(*s

7eyond the lesson level =e must address results o student learning at the course level3#oreover =e should do so or all sections of each course meaning collaboration among the

ull) and %art)time aculty teaching the course3 (n stating the desired student learning outcomes=e have the advantage o agreed)u%on student objectives in the course outline3

A great deal o energy has been e0%ended in discussing the di erence bet=een a course objectiveand a student learning outcome3 The di erence may be clearer =hen vie=ed in the conte0t o

%roducing assessment results that 1 %rovide use ul eedbac> to im%rove the teaching andlearning %rocess and 2 %rovide use ul in ormation to im%rove college %ractices3 SLOs moreclearly connect =ith how the instructor will evaluate student work to determine i the objective

has been met3 5hen =e =rite an assignment =e %rovide a conte$t in =hich the student =illres%ond and =e evaluate the res%onse based on criteria =e use to judge i the student has met theobjective.usually =e have at least a mental construct o minimum acce%table %er ormancestandards3 These are the t=o additional %ieces that trans orm an objective into an SLO3 ereDsho= it might =or>3

( course objectives have been =ritten =ell they =ill be com%lete measurable and rigorous3 (n %ractice as aculty loo> more closely at the criteria and methods to assess these objectives

4


6/73

changes o ten result3 To ?o%erationaliBe@ an objective or assessment %ur%oses that is totrans orm it into a statement o desired student learning outcomes ty%ically =e must address:

1 the stated objectives in terms o acJuired >no=ledge s>ill or values ho%e ully thee0isting course objectives

2 the conte0t or conditions under =hich the student =ill be e0%ected to a%%ly the>no=ledge s>ill or values and

the %rimary traits =hich =ill be used in assessing student %er ormance3

7elo= are some e0am%les o ?robust course objectives@ or ?statements o desired studentlearning outcomes3@ 'ote that this di erence is largely semantic3 Some colleges have chosen to

%ut SLO statements in course outlines as an enhancement o the objectives =hile others have built statements o desired SLOs into a de%artmental assessment %lan ty%ically related to %rogram revie=3 5hatever vehicle the college uses to o%erationaliBe course objectives to SLOs

it must be done collaboratively among aculty =ho teach the course3/amples of Course *b-ectives Transformed !nto Student (earning *utcomes

Course *b-ective Statement of esired S(*5rite =ell)organiBedaccurate and signi icantcontent3 /nglish

Conte0t: !iven an in)class =riting tas> based on an assigned readingObjective: demonstrate a%%ro%riate and com%etent =riting =hichTraits: states a thesis su%%orts assertions maintains unity o thought and

%ur%ose is organiBed and is technically correct in %aragra%hcom%osition sentence structure grammar s%elling and =ord use3

AnalyBe behavior ollo=ingthe major acce%ted theories3

Psychology

Conte0t: !iven a %articular behavior and its conte0t e3g3 %laying incessantly =ithoneDs hair =hen under %ressure in the %resence o the o%%osite se0

Objective: describe ho= the %ers%ectives o behaviorism humanistic

%sychoanalytic and biological %sychology =ould inter%ret that behavior and =hat methods might each use to alter that behavior3Traits: (nclude theoretical basis descri%tion o causality and treatment regimen3

8nderstand and a%%ly thescienti ic method3 7iology

Conte0t: !iven a hy%othesisObjective: design e0%eriments and inter%ret data according to the scienti ic method

in order to evaluate the hy%othesis3Traits: (nclude the ability to a%%roach the scienti ic method in a variety o =ays

ormulate Juestions design e0%eriments that ans=er the JuestionsI andmani%ulate and evaluate the e0%erimental data to reach conclusions3

Com%are and contrast thete0t and ilm versions o aliterary =or>3 Film

Conte0t: A ter vie=ing an assigned ilm based on a literary te0tObjective: =rite a revie= o the ilm3Traits: (nclude an a%%raisal o the directorDs selection and e ective translation

o content rom the literary te0t and the dominant tone the director seemsto be trying to achieve su%%orting each statement =ith detail rom thete0t and ilm and your %ersonal reaction to the cited scenes3

ctivit" %Per orm the ?5riting Student Learning Outcomes@ e0ercise in A%%endi0 3 *evie= the irst e0am%le3 Then or thesecond course objective com%lete the Per ormance Conte0t #easurable Objective and Primary Traits3 Finallyselect an objective rom a course in your disci%line and construct the three)%art SLO statement3

&


7/73

7uilding a *ubricStart =ith e0%ectations or satis actory=or> or each trait such asOrganiBation in the table to the le t:(deas generally related to one another

and to the ocus but may have someunrelated material

AdeJuate introduction and conclusionSome attem%t at transitionsThen stretch u% to e0cellent and do=nto unsatis actory3

"rimar& Trait Anal&sis Statements of 1rading Criteria

Primary traits are the characteristics that are evaluated in assessing student =or>3 (denti ying

%rimary traits or a given assignment involved listing those s%eci ic com%onents that ta>entogether ma>e u% a com%lete %iece o =or>3 They are the collection o things that =e as teachersloo> or =hen =e grade student =or>3

$e inition o Primary Trait AssessmentPrimary trait assessment is a method o e0%licitly stating the criteria and standards or evaluation o student %er ormance o an assignment or test3 The %ro essor identi ies thetraits that =ill be evaluated and ran>s the student s %er ormance o each trait on a scale o Mmost e ectiveM to Mleast e ectiveM realiBation o the assignment goals3 On this scale thelevel o the student s %er ormance is e0%licitly ran>ed so that the student >no=s ho= sheis being evaluated3 The instructor has created the scale or direct a%%lication to theassignment the student is %er orming so that i the entire class does %oorly on the

assignment it is clear to the instructor =hat di iculties the class may share =ith oneanother3 This recursive eedbac> o %rimary trait assessment can be used to in ormclassroom and de%artmental im%rovement3

5hile ?%rimary traits@ are the categories into =hich =e can sort com%etencies =hen =e evaluatestudent =or> =e loo> or s%eci ic levels o %er ormance in each o these areas3 For e0am%le anessay might be rated on develo%ment organiBation style and mechanics3 These %rimary traitsare then rated on some sort o a scale.as sim%le as A


8/73

"rimar& Trait 1rading of #ath "roblem SolvingTrait %oints 2 %oints 1 %oint - %oints

8nderstanding com%lete understanding o

the %roblem in the %roblem statement sectionas =ell as in thedevelo%ment o the %lanand inter%retation o thesolution

good understanding o the

%roblem in the %roblemstatement section3 Someminor %oint s o the %roblemmay be overloo>ed in the

%roblem statement thedevelo%ment o the %lan orthe inter%retation o thesolution

minimal understanding o the

%roblemI the %roblemstatement may be unclear tothe reader3 The %lan ands signi icant %arts othe %roblem

no understanding o the

%roblemI the %roblemstatement section does notaddress the %roblem ormay even be missing3 The

%lan and discussion o thesolution have nothing to do=ith the %roblem

Plan %lan is clearly articulatedA'$ =ill lead to a correctsolution

%lan is articulated reasonably=ell and correct O* maycontain a minor la= based ona correct inter%retation o the

%roblem

%lan is not clearly %resentedO* only %artially correct

based on a correct


9/73

8se the /nglish rubric in A%%endi0 47 to grade the sam%le student essay in A%%endi0 4C3Com%are your results =ith colleagues =ho graded the same %a%er3 5here =ere your assessmentsdi erentE Can you come to agreement on the overall rating o the %a%erE

To this %oint =e have discussed stating the desired student learning outcome and develo%ing agrading rubric3 These are the beginning ste%s that can lead us to=ard collecting and using theresults o measured student learning outcomes3 A road ma% o a %ossible ?SLO Assessment Plan@is sho=n in the diagram belo=3

Selecting the Assessment #ethod Authentic Assessment and eep (earning

The ne0t logical Juestion is ?5hat assessment method should be usedE@ There are certainly a=ide variety o methods or determining =hether or not a student has demonstrated learning o a

%articular objective3

Summar& of Tools for irect Assessment of Student (earningCa%stone Projectills in terms o absolute levelso mastery3 The ocus is on %er ormance o an individual as measured against a standard or criteria rather thanagainst %er ormance o others =ho ta>e the same test as =ith norm)re erenced tests3

'orm)*e erenced Test.an objective test that is standardiBed on a grou% o individuals =hose %er ormance isevaluated in relation to the %er ormance o othersI contrasted =ith criterion)re erenced test3

Port olio.a collection o student =or> organiBed around a s%eci ic goal e3g3 set o standards or benchmar>s orinstructional objectives I it can contain items such as handouts essays rough dra ts inal co%ies art=or> re%orts

%hotogra%hs gra%hs charts videota%es audiota%es notes anecdotal records and recommendations and revie=sIeach item in the %ort olio %rovides a %ortion o the evidence needed to sho= that the goal has been attained3

Per ormance Assessments.activities in =hich students are reJuired to demonstrate their level o com%etence or>no=ledge by creating a %roduct or res%onse scored so as to ca%ture not just the Mright ans=erM but also thereasonableness o the %rocedure used to carry out the tas> or solve the %roblem3

*ating Scales.subjective assessments made on %redetermined criteria in the orm o a scale3 *ating scales includenumerical scales or descri%tive scales3 Forced choice rating scales reJuire that the rater determine =hether anindividual demonstrates more o one trait than another3

Simulation.a com%etency based measure =hereby %re)o%erationaliBed abilities are measured in most direct real)=orld a%%roach3 Simulation is %rimarily utiliBed to a%%ro0imate the results o %er ormance a%%raisal but =hen dueto the target com%etency involved logistical %roblems or cost direct demonstration o the student s>ill isim%ractical3

,

Course (evel T(C lements of an Assessment "lan Statement o $esired SLO Faculty Collaboration Course N Conte0t or N Primary N Observables or /ach N Assessment N 'orm Among N /valuate N Com%ile N 8se Feedbac> Objective Conditions Traits Per ormance Level #ethod Selected (nstructors Student 5or> *esults or (m%rovement

!rading *ubric Assessment *e%ort Com iled or /ach $esired SLO


10/73

ctivit" 3 *ead the article ? The Case or Authentic Assessment@ by !rant 5iggins in A%%endi0 &3 $iscuss theassessment methods you use in your classes3 5hat methods do you useE o= e ective do you ind themE

ctivit" 4 ;ie= the ilm ?A Private 8niverse3@ $iscuss the im%lications or %roducing and assessing dee% learning3

As ( have listened to aculty discuss assessment methods at si0 state=ide Cali ornia Assessment(nstitutes eight regional *Psho%s and our o=n collegeDs summer institute on SLOs (have come to several conclusions:

School)o )/ducation level discussions o assessment instruments are not =ell received3

Faculty are eager to tal> about the challenges they e0%erience in assessing students3

$iscussions o ten turn to great stu such as authentic assessment and dee% learning3

#ost aculty use a rather narro= range o methods.but use them =ell3

Faculty =ill more o ten try another assessment techniJue i recommended by a colleague3

#any aculty use assessments that need just slight enhancement to yield SLO results3

A e= s%eci ics on the last %oint may hel%:

One vocational de%artment teaches %ort olios in its introductory course.and uses %ort olios =hen doing aculty career advising.but does not ollo= through by havingstudents add to the %ort olio as com%etencies are acJuired in subseJuent courses3 The

ca%stone course in this de%artment has students build a %ort olio as %art o %re%aring toenter the job mar>et but there is no connection =ith the %ort olio in the intro class nor isthere a grading rubric3

One de%artment has a clinical com%onent in =hich students are evaluated using a ratingsheet on their hands)on com%etencies3 The de%artment has com%lained about needing

eedbac> rom clinical to the theory courses but has not consistently used the results othe rating sheets or this %ur%ose3 The com%etencies taught in the theory course are airly=ell aligned =ith those assessed in clinical but could be im%roved3

Faculty in one o the social science de%artments have =or>ed on de%artmental standardsor term %a%ers to the %oint o a %rimary trait analysis and meet regularly to discuss

grading o term %a%ers but have not illed in the observables to establish a rubric3 The /nglish de%artment has a grading rubric or =ritten essays and ull) and %art)time

aculty have regular norming sessions to im%rove consistency o grading but the systemhas only been used or t=o courses reshman com% and its %rereJuisite3

7ased on these observations my recommendation is to start =ith these good things that acultyare doing get them engaged in tal>ing about grading &ffective Grading/ 5ool for Learningand ssessment by 7arbara 5alvoord and ;irginia Anderson has been great or this get aculty

1-


11/73

to share assessment strategies =ith one another.es%ecially across disci%lines and %rovide thesu%%ort or moving these good e0isting assessment %ractices to the ne0t level3

Norming or !nter'+ater +eliabilit& Assuring Consistenc& of 1rading Among Facult&

5hatever method is chosen to assess student learning and a%%ly the agreed)u%on grading rubricaculty =ho teach sections o the course should =or> together to assure that the results o grading

student =or> are consistent3 This %rocess is >no=n as ?norming@ or ?inter)rater reliability@ andhas been used in a variety o venues including construction o standardiBed tests evaluating

%lacement test =riting sam%les and ran>ing grant %ro%osals3 An e$plicit %rocess or establishinginter)rater reliability =ould be to have evaluators use the grading rubric on a series o studentassignments and then evaluate the e0tent o agreement using standard statistical measures3 The>a%%a statistic the chi sJuare test the Pearson correlation coe icient and %ercent agreementhave all been used under various circumstances3 Agreement can be im%roved through discussion

and training3 'orming can be %er ormed informall" by having regular discussions among acultyraters revie=ing and debating e0am%les related to the observables in the grading rubric untilconsensus is reached3

5ith the statement o the desired student learning outcome in %lace =ith the grading rubricestablished and normed the results collected can be %o=er ul in ormation or im%roving studentlearning.and may %rovide the basis or directing college resources in areas to address thelearning ga%s identi ied3

The Assessment +eport Sharing the +esults of Student (earning *utcomes

A sensitive as%ect o the discussion o Student Learning Outcomes has been ho= the in ormationis to be used3 #ost im%ortantly the signi icance o the results relates directly to improving teaching and learning 3 #ost o that im%rovement lies =ith aculty.curriculum design

%edagogy learning environment assessment methods and the li>e3 The rest is in the hands o thecollegeDs su%%ort system.%roviding acilities eJui%ment student services and so on.to theinstructional %rogram to ma>e those im%rovements identi ied by SLO results3 To the e0tent that=e can build an Assessment *e%ort that focuses on the instructional program level .hel%ing

aculty im%rove student learning and identi ying needed college resources college aculty andsta =ill buy into the %rocess3 The e0am%les belo= illustrate a e= >ey %oints:

An analysis o the results.by aculty %articularly all %rogram aculty.must

accom%any the results3 *esults can be listed com%letely or summariBed in narrative orm3

S%eci ic actions to be ta>en as a conseJuence o the results should be described3

*esults o ten contradict our assum%tions o ho= and =hat students learn3

8se o SLO results can be e ectively centered in the instructional program asthe locus of change 3

11


12/73

Sim%le %resentations o results orm elegant evidence or accreditation3

The >ey com%onents o the Assessment Plan are the student learning outcomes statements andthe assessment methods used or each3 The %lan o ten includes benchmar>s that indicate theincremental gains e0%ected in the assessment results3 The essential eatures o the Assessment

*e%ort are a summary o the results o student evaluations an analysis o those indings and asummary o the actions ta>en to im%rove the student assessment %er ormance3 The diagram belo= summariBes the elements o an e ective Assessment Plan and the resulting Assessment*e%ort3 /0am%les o Assessment *e%orts are sho=n in the A%%endi03

Course Assessment +eport$e%artment Term 6 ear Course 'ame and 'umber

Student (earning *utcome Statements Assessment #ethod escription attach rubric13 Q Ca%stone Project Q /mbedded Test Ruestion Q Port olio

Q Per ormance Assessment Q *ating Scale Q Other

23Q Ca%stone Project Q /mbedded Test Ruestion Q Port olioQ Per ormance Assessment Q *ating Scale Q Other

3 Q Ca%stone Project Q /mbedded Test Ruestion Q Port olioQ Per ormance Assessment Q *ating Scale Q Other

Assessment +esults Anal&sis , Actions Ta3en1323

3

"rogram (evel Student (earning *utcomes

The term ?%rogram@ here re ers to core reJuired courses or occu%ational %rograms and lo=er division major %re%aration or trans er %rograms3 #any %ro essional societies have standards or com%etencies that can be used as the basis or %rogram level SLOs3 Some e0am%les arere erenced in the endnotes and summariBed in the A%%endi03 O ten ho=ever thesecom%etencies are in the orm o discrete s>ills rather than more global outcomes that =ould lendthemselves to summaries o student learning by those =ho have com%leted those %rograms3 Ane0am%le o aggregating detailed standards into more com%rehensive SLO statements is thissam%le ta>en rom the American Psychological Association3

/ample of Aggregation of Specific "rogram Competencies into a "rogram Student (earning *utcome

Global Student Learning Outcome : )se critical thin3ing effectivel& 3

Specific Competencies :a3 /valuate the Juality o in ormation including di erentiating em%irical evidence rom s%eculation and the

%robable rom the im%robable3 b3 (denti y and evaluate the source conte0t and credibility o in ormation3c3 *ecogniBe and de end against common allacies in thin>ing3d3 Avoid being s=ayed by a%%eals to emotion or authority3e3 /valuate %o%ular media re%orts o %sychological research3

3 $emonstrate an attitude o critical thin>ing that includes %ersistence o%en)mindedness tolerance or

12


13/73

ambiguity and intellectual engagement3g3 #a>e lin>ages or connections bet=een diverse acts theories and observations3

From 1ndergraduate 6s"cholog" ,ajor Learning Goals nd Outcomes/ Report American Psychological Association #arch 2--2 /ii

irect and !ndirect #easures of Student (earning *utcomes

One consideration o the method is =hether it is a direct or indirect measure o student learning3$irect assessment includes using criteria that assesses or measures student learning directly suchas =riting an essay giving a s%eech solving %roblems using a ca%stone e0%erience or evaluating a

%ort olio o student)created %roducts3 (ndirect assessment e0amines student %er ormance or behavior using criteria =hich i accom%lished assume learning has ta>en %lace3 /0am%lesinclude surveys o students and em%loyers e0it intervie=s o graduates retention and trans erstudies and job %lacement data3

(ndirect measures are o ten thought o as out%uts: course com%letions degrees certi icates andtrans ers or e0am%le3 These are the institutional measures o accountability measured by theCali ornia Community CollegeDs Partnershi% or /0cellence initiative3 These measures are o ten>ey indicators o success or a %rogram as e0em%li ied belo=3

/ample of the )se of irect and !ndirect #easures of Student (earningFrom O>lahoma State 8niversity: htt%:ills %lus =ritten and verbal communication s>illsI $irect

A%%ly geologic >no=ledge and s>ills to a range o %roblems aced by business industrygovernmentI $irect

!ain em%loyment in the geology %ro ession or advance to graduate studies in geology or anallied ield3 (ndirect

!dentif&ing "rogram Competencies /ternal , !nternal Sources

One o the ne= activities that the accreditation standards reJuire is theconstruction o com%etencies or each o our degree and certi icate

%rograms3 One =ay to a%%roach this tas> is to begin =ith thecom%etencies or standards that are used by state or national

%ro essional organiBations or licensing


14/73

The American Psychological Association recently %ublished 1ndergraduate 6s"cholog" Learning Goals and Outcomes that lists both global student learning outcomes and detailed com%etencies or both the %sych major and liberal studies students3

The Cali ornia State 7oard o 7arbering and Cosmetology tests graduates or licensure based on curriculumstandards enacted in Title 1& o the Cali ornia Code o *egulations3

Lin>s to these and other com%etencies and standards are ound in the A%%endi03 5hile anindividual %rogram may not teach to all the outcomes that these grou%s s%eci y the lists are ane0cellent starting %oint3 'ot all %rograms have industry associations or %ro essional societies=ho =rite standards3 Such %rograms may need to consult local vocational advisory committeesor aculty colleagues at neighboring institutions3

Strategies for irect Assessment of "rogram S(*s #osaic and Capstone Approaches

5he ,osaic pproach- Assessment o %rogram)level student learning outcomes can bea%%roached by assessing either detailed com%etencies or more global %rogram learning goals3

Loo> again at the e0am%le in the table at the to% o %age 1- or the distinction bet=een a global

SLO statement and its detailed com%etencies3 Assessing detailed com%etencies vie=s theacJuiring o >no=ledge s>ills and attitudes as ta>ing %lace rather li>e assembling a com%le0mosaic rom individual colored tiles3 (t is a more analytical model and %rovides more targetedin ormation about student learning3 o=ever the e0tent o the e ort to ind authenticassessments or a large number o ?mosaic@ com%etencies get agreement among %rogram acultyon those assessments construct rubrics norm on sam%les o student =or> and then collect andanalyBe the data may stretch %rogram resources to the brea>ing %oint3 Furthermore theacJuisition o small discrete %ac>ets o >no=ledge may not lead the student to acJuire a moreintegrated understanding that %rovides needed a%%licability to the ne0t ste% in that studentDscareer be it trans er or directly entering the job mar>et3 ConseJuently more holistic assessmentsare o ten %re erred such as ca%stone courses or internshi%s3

5he 6rogram udit 3 /ven i an integrated assessment is used at the end o the %rogram it isuse ul to identi y =here in the curriculum each SLO or even individual com%etency isacJuired3 Furthermore learning most o ten occurs in cycles: the student =ill be e0%osed to ato%ic then later gain com%etency in that area and inally master that s>ill3 $oing a %rogramaudit o e0actly =here SLOs and


15/73

/0am%le rom ?A Program !uide or Outcomes Assessment@ by !eneva College A%ril2---: htt%:to% !ra%hics %rogram revised toinclude more e0%eriencein 5eb site gra%hics3Students designedgra%hics or current #Chome %age and lin>s3

23 Students =ill demonstratelearning the basic conce%tsnecessary to %er ormsatis actorily in #assCommunications entry)level jobs3

23 5hen surveyed using 6arklandCollege Student Occupational

:ollow;1p Surve"( graduates =illdescribe satis action =ith their#ass Communication >no=ledge torecall analyBe evaluate and utiliBe

basic conce%ts3

23 Feedbac> rom em%loyers andstudents strongly indicated that;isual Arts %rogram o%tion had

become obsoleteI %re erence isgiven to graduates =ith $es>to%Publishing s>ills3

23 ;isual Arts %rogramo%tion shelved3

3 Students in the #assCommunication A3A3

%rogram =ill have the>no=ledge to success ullycom%lete a 7achelorsdegree in #assCommunication3

3 Four)year institutions =ill re%ort o a 94 %ercent acce%tance rate into#ass Communication %rograms3

3 8 o ( Coordinator o Trans erArticulation re%orted that out o2, a%%licants rom other schoolsto !ra%hics a #ass Com student=as the only admit3

3 Continue togatherland !ra%hicsstudents a%%lied3

ctivit" < 8 Creating 6rogram SLO Statements and 6erforming a 6rogram udit 5ith a grou% o aculty rom your disci%line =rite a set o Program SLOs or a degree or

certi icate in your area3

14

Pre


16/73

Assemble the outlines o record or the reJuired courses or a degree or certi icate in your disci%line3

List the course objectives or all o these courses %re erably a ter having revised them to?robust@ objectivese that above3 ou may =ish to categoriBe each course objective by thee0tent to =hich is moves students to=ard mastery o the Program SLO3

Summar& of "rogram (evel S(* Assessment Strategies Indirect : im%lies that SLOs are achieved

Trans er Program com%letion Gob %lacement /m%loyer surveys Student e0it surveys Licensure e0ams

Direct : students assessed on SLOs =hile in %rogramCa%stone strategies:

Ca%stone course or %roject StandardiBed test: commercial or local sam%led

or com%rehensive (nternshi%%lace evaluation Port olio: student) or instructor)generated

#osaic strategies: embeddede those or %rograms in the major3 #ost colleges =rite global learning statements and then brea> those do=n

1&

:(m%lementProgram

2(denti y(ndirect

#easures

;$ecide onProgram

(m%rovements

ing

1A3 /nglish Com%osition173 Critical Thin>ing1C3 Oral Communication

73 'atural Sciences 713 Physical Science723 Li e Science7 3 Laboratory Activity7"3 #athematicsground

"3 Social 6 7ehavioral Sciences

/3 Livelong 8nderstanding 6 Sel )$evelo%ment

/3 Livelong 8nderstanding 6 Sel )$evelo%ment 4A3 Physical Science473 7iological Science

F3 istory 6 !overnment &3 Language Other Than /nglish

For more in ormation re er to CS8 /0ecutive Order 4,4 and ?(!/TC 'otes@ 1 2 and

ctivit" => 8 Writing Global SLO Statements with Specific Competencies for &ach *evie= the models o general education student learning outcomes in A%%endi0 12 assumes

>no=ledge o CCC !/ CS8 !/)7readth and (!/TC %atterns 3 For each college !/ area =rite a global student learning outcome statement3 For each college !/ area =rite s%eci ic com%etency SLO statements under each o the global

SLO statements3

ctivit" == 86erforming a General &ducation 6rogram udit Assemble the outlines o record or the courses a%%roved in each !/ area3 List the course objectives or all o these courses %re erably a ter having revised them to

?robust@ objectivese that discussed %reviously3 ou may =ish to categoriBe each course objective by the e0tent to =hich is moves students to=ard mastery o the AA Friedlander /0ecutive ;ice President o Santa 7arbara City College concluded that mostcolleges around the country are still at the %rocess level o develo%ing SLOs3 'evertheless thereare many e0am%les o e0cellent =or> on SLOs at colleges around the country summariBed inA%%endi0 1 3 These e0am%les should %rovide colleges =hich are ne= to the Student LearningOutcomes %rocess =ith the shared e0%eriences o their colleagues so that climbing the learning

19


18/73

curve can be acilitated3 The climate in education today sim%ly =ill not allo= us to e0%endvaluable time and energy on a %rocess that =ill not yield use ul results3 Such results have the

%otential to allo= aculty and others to engage in re lection about the %rocess o teaching andlearning and then use the insights they develo% to adjust the teaching)learning)assessment

%rocess to o%timiBe learning to the ull e0tent %ossible3 7y having a clear %ath to those results

=e can move ahead =ith ta>ing the irst e= ste%s3 7ut =e need to >ee% our eye on the goal as=eDre =al>ing3 *emember utility can Juic>ly become utility by adding a e= Ds

1+


19/73


20/73

Appendi/ :B > "alomar College 8CA9 Statement of "rinciples on Assessment

?h& do Assessment@

PalomarDs ;ision Statement %rojects a uture in =hich MPalomar College judges its =or> and its

%rograms and ormulates its %olicies %rimarily on the basis o learning outcomes and has acom%rehensive %rogram or assessing those outcomes and res%onding to its indings3M 5eado%ted this strategic goal even be ore our accrediting body revised its accreditation standards toM ocus on outcomes and accom%lishments embracing a model o accreditation =hich reJuiresassessment o resources %rocesses and outcomes at the institutional level3M Thus our o=ncommitment to assess student learning at the institutional level %recedes but com%lements themandates o accreditation3 To carry out that commitment Palomar =ill develo% and continuouslyre ine and im%rove an institutional rame=or> or assessing student learning and using thein ormation gained rom such assessment to serve our students better35hat is assessmentE5e mean by MassessmentM Mthe systematic collection analysis inter%retation and use o

in ormation to understand and im%rove teaching and learningM Tom Angelo 35hat is assessment orEAt Palomar =e =ill use assessment %rimarily to understand and thereby im%rove studentlearning3 #ore s%eci ically assessment can serve the ollo=ing roles in the institution:

To %rovide im%roved eedbac> guidance and mentoring to students so as to hel% them better %lan and e0ecute their educational %rograms3

To %rovide im%roved eedbac> about student learning to su%%ort aculty in their =or>3 To hel% us design and modi y %rograms to better %romote learning and student success3 To develo% common de initions and benchmar>s or im%ortant student abilities that =ill

enable us to act more coherently and e ectively to %romote student learning3 To hel% us understand ho= di erent grou%s o students e0%erience the college di erently

so as to ada%t our courses and %rograms to the needs and ca%acities o all students3

To hel% us understand ho= our di erent courses and %rograms a ect students over time so that=e can better coordinate and seJuence the studentDs e0%erience to %roduce more and dee%erlearning3

?hat is assessment not for@

$i erent institutions may o course use the tools o learning assessment di erently3 (t =ill hel%to clari y the nature o PalomarDs commitment to learning assessment to s%eci y some o the

%ossible %ur%oses o assessment that =e =ill e0clude rom our a%%roach3

5e =ill not use assessment as an end in itsel 3 Assessment that does not hel% us to %romote student learning is a =aste o time3 5e =ill not use assessment o student learning %unitively or as a means o determining

aculty or sta salaries or re=ards3 The %ur%ose o assessment is to evaluate studentlearning not to re=ard or %unish aculty or sta 3

5e =ill not use any single mode o assessment to ans=er all Juestions or strictlydetermine %rogram decisions3

2-A%%endi0 17


21/73

5e =ill not use assessment in a =ay that =ill im%inge u%on the academic reedom or %ro essional rights o aculty3 (ndividual aculty members must continue to e0ercise their best %ro essional judgment in matters o grading and disci%line3

5e =ill not assume that assessment can ans=er all Juestions about all students3 5e neednot directly assess all students in order to learn about the e ectiveness o our %rograms

and %olicies3 5e =ill not assume that assessment is Juantitative3 5hile numerical scales or rubricssuch as the our)%oint grading scale can be use ul their accuracy al=ays de%ends on the

clear understanding o the conce%ts behind the numbers3 O ten the best indicator ostudent learning can be e0%ressed better as a narrative or a %er ormance than as a number3

5e =ill not use assessment only to evaluate the end o the studentDs e0%erience or merelyto be accountable to outside %arties3 Assessment must be ongoing observation o =hat =e

believe is im%ortant3 5e =ill not assume that assessment is only grading3

?ho ill do assessment@

Palomar s aculty in consultation =ith the entire college community =ill sha%e and designinstitutional assessment activities and =ill identi y the core >no=ledge and s>ills that ourstudents need to master3 The aculty =ill li>e=ise develo% benchmar>s by =hich student %rogresscan be evaluated3 These =ill be ongoing %rocesses o%en to modi ication and im%rovement3 'otall assessment need be done in individual classes and not every aculty member need assess allo the core learning3

o ill e use assessment@

The ollo=ing guidelines =ill govern the methodology and a%%roach =e =ill em%loy at Palomar

to institutional assessment: 5e =ill al=ays see> multi%le judgments o student learning rather than a single standard3 5e =ill assess those s>ills and >no=ledge that our aculty in consultation =ith the entire

college community judges to be im%ortant and valuable3 5e =ill assess the ongoing %rogress o students throughout their e0%erience at the

college3

21A%%endi0 17


22/73

Appendi/ :C > AA Nine "rinciples of 1ood "ractice for Assessing Student (earning

AAHE ASSESSMENT !"#M $ Principles of %ood Practice for Assessing Student &earning

13 The assessment of student learning begins ith educational values. Assessment is notan end in itsel but a vehicle or educational im%rovement3 (ts e ective %ractice then

begins =ith and enacts a vision o the >inds o learning =e most value or students andstrive to hel% them achieve3 /ducational values should drive not only what =e choose toassess but also how =e do so3 5here Juestions about educational mission and values ares>i%%ed over assessment threatens to be an e0ercise in measuring =hat s easy rather thana %rocess o im%roving =hat =e really care about3

23 Assessment is most effective hen it reflects an understanding of learning asmultidimensional integrated and revealed in performance over time. Learning is a

com%le0 %rocess3 (t entails not only =hat students >no= but =hat they can do =ith =hatthey >no=I it involves not only >no=ledge and abilities but values attitudes and habitso mind that a ect both academic success and %er ormance beyond the classroom3Assessment should re lect these understandings by em%loying a diverse array o methodsincluding those that call or actual %er ormance using them over time so as to revealchange gro=th and increasing degrees o integration3 Such an a%%roach aims or a morecom%lete and accurate %icture o learning and there ore irmer bases or im%roving ourstudents educational e0%erience3

3 Assessment or3s best hen the programs it see3s to improve have clear e/plicitl&stated purposes. Assessment is a goal)oriented %rocess3 (t entails com%aring educational

%er ormance =ith educational %ur%oses and e0%ectations )) those derived rom the

institution s mission rom aculty intentions in %rogram and course design and rom>no=ledge o students o=n goals3 5here %rogram %ur%oses lac> s%eci icity oragreement assessment as a %rocess %ushes a cam%us to=ard clarity about =here to aimand =hat standards to a%%lyI assessment also %rom%ts attention to =here and ho=

%rogram goals =ill be taught and learned3 Clear shared im%lementable goals are thecornerstone or assessment that is ocused and use ul3

"3 Assessment re uires attention to outcomes but also and e uall& to the e/periencesthat lead to those outcomes. (n ormation about outcomes is o high im%ortanceI =herestudents Mend u%M matters greatly3 7ut to im%rove outcomes =e need to >no= aboutstudent e0%erience along the =ay )) about the curricula teaching and >ind o studente ort that lead to %articular outcomes3 Assessment can hel% us understand =hich students

learn best under =hat conditionsI =ith such >no=ledge comes the ca%acity to im%rove the=hole o their learning3

43 Assessment or3s best hen it is ongoing not episodic. Assessment is a %rocess =hose %o=er is cumulative3 Though isolated Mone)shotM assessment can be better than noneim%rovement is best ostered =hen assessment entails a lin>ed series o activitiesunderta>en over time3 This may mean trac>ing the %rocess o individual students or ocohorts o studentsI it may mean collecting the same e0am%les o student %er ormance orusing the same instrument semester a ter semester3 The %oint is to monitor %rogress

22A%%endi0 1C


23/73

to=ard intended goals in a s%irit o continuous im%rovement3 Along the =ay theassessment %rocess itsel should be evaluated and re ined in light o emerging insights3

&3 Assessment fosters ider improvement hen representatives from across theeducational communit& are involved. Student learning is a cam%us)=ide res%onsibilityand assessment is a =ay o enacting that res%onsibility3 Thus =hile assessment e orts

may start small the aim over time is to involve %eo%le rom across the educationalcommunity3 Faculty %lay an es%ecially im%ortant role but assessment s Juestions can t beully addressed =ithout %artici%ation by student)a airs educators librarians

administrators and students3 Assessment may also involve individuals rom beyond thecam%us alumni or small grou%s o e0%erts but a collaborative activityI its aim is =ider better)in ormedattention to student learning by all %arties =ith a sta>e in its im%rovement3

93 Assessment ma3es a difference hen it begins ith issues of use and illuminatesuestions that people reall& care about. Assessment recogniBes the value o

in ormation in the %rocess o im%rovement3 7ut to be use ul in ormation must be

connected to issues or Juestions that %eo%le really care about3 This im%lies assessmenta%%roaches that %roduce evidence that relevant %arties =ill ind credible suggestive anda%%licable to decisions that need to be made3 (t means thin>ing in advance about ho= thein ormation =ill be used and by =hom3 The %oint o assessment is not to gather data andreturn MresultsMI it is a %rocess that starts =ith the Juestions o decision)ma>ers thatinvolves them in the gathering and inter%reting o data and that in orms and hel%s guidecontinuous im%rovement3

+3 Assessment is most li3el& to lead to improvement hen it is part of a larger set ofconditions that promote change. Assessment alone changes little3 (ts greatestcontribution comes on cam%uses =here the Juality o teaching and learning is visiblyvalued and =or>ed at3 On such cam%uses the %ush to im%rove educational %er ormanceis a visible and %rimary goal o leadershi%I im%roving the Juality o undergraduateeducation is central to the institution s %lanning budgeting and %ersonnel decisions3 Onsuch cam%uses in ormation about learning outcomes is seen as an integral %art odecision ma>ing and avidly sought3

,3 Through assessment educators meet responsibilities to students and to the public. There is a com%elling %ublic sta>e in education3 As educators =e have a res%onsibility tothe %ublics that su%%ort or de%end on us to %rovide in ormation about the =ays in =hichour students meet goals and e0%ectations3 7ut that res%onsibility goes beyond there%orting o such in ormationI our dee%er obligation )) to ourselves our students andsociety )) is to im%rove3 Those to =hom educators are accountable have a corres%ondingobligation to su%%ort such attem%ts at im%rovement3

Authors Ale0ander 53 AstinI Trudy 53 7antaI U3 Patricia CrossI /laine /l)Uha=asI Peter T3 /=ellI Pat utchingsITheodore G3 #archeseI Uay #3 #cClenneyI #arcia #ent>o=s>iI #argaret A3 #illerI /3 Thomas #oranI 7arbara $35rightThis document =as develo%ed under the aus%ices o the AA / Assessment Forum =ith su%%ort rom the Fund orthe (m%rovement o Postsecondary /ducation =ith additional su%%ort or %ublication and dissemination rom the/00on /ducation Foundation3 Co%ies may be made =ithout restriction3

2A%%endi0 1C


24/73

Appendi/ : > 5Closing the (oop6 b& Tom Angelo

Seven Common #is Perce%tions About Outcomes Assessment

13 5eDre doing just ine =ithout it3 Assessment is medicine only or the sic>323 5eDre already doing it3 Assessment is just old =ine in ne= bottles3

3 5eDre ar too busy to do it3 Assessment is an ?administrivial@ burden3

"3 The most im%ortant things =e do canDt or a Trojan horse393 'o one =ill care about or use =hat =e ind out3 Assessment is a =aste o time3

Seven *easonable *es%onses to Those #is Perce%tions

13 5eDre doing just ine =ithout it3O>ay then letDs use assessment to ind out =hat =or>s and to hel% us document and build onour successes3

23 5eDre already doing it3O>ay then letDs audit all the assessments =e already do to discover =hat =e >no= and =hat =edonDt3

3 5eDre ar too busy to do it3O>ay but since =eDre already doing it letDs use assessment to see =here and ho= =e can savetime and e ort3

"3 The most im%ortant things =e do canDtely to get more resources ho= =hat and =here can =e %iggybac> embed

and substituteE&3 TheyDll use the results against us3They might3 So letDs build in strong sa eguards against misuse be ore =e agree to assess3

93 'o one =ill care about or use =hat =e ind out3To avoid that letDs agree not to do any assessments =ithout a irm commitment romsta>eholders to use the results3

Seven Trans ormative !uidelines or 8sing Assessment to (m%rove Teaching and Learning

13 7uild shared trust3 7egin by lo=ering social and inter%ersonal barriers to change323 7uild shared motivation3 Collectively determine goals =orth =or>ing to=ard and %roblems

=orth solving.and consider the li>ely costs and bene its33 7uild a shared language3 $evelo% a collective understanding o ne= conce%ts mental models

needed or trans ormation3"3 $esign bac>=ard and =or> or=ard3 $esign bac>=ard rom the shared vision and long)term

goals to develo% coherent outcomes strategies and activities343 Thin> and act systematically3 8nderstand the advantages and limitations o the larger system s

=ithin =hich =e o%erate and see> connections and a%%lications to those larger =orlds3&3 Practice =hat =e %reach3 8se =hat =e have learned about individual and organiBational learning

to in orm and e0%lain our e orts and strategies3

2"A%%endi0 1$


25/73

93 $onDt assume as>3 #a>e the im%licit e0%licit3 8se assessment to ocus on =hat matters most3

24A%%endi0 1$


26/73

Appendi/ : > Five #&ths of Assessment b& avid Clement #ontere& "eninsula College

As my campus management moves inexorablytoward Learning Outcomes, Outcomes BasedEducation (OBE), and Assessment rubrics, Irealize t at t ere are !ve ma"or #aculty

ob"ections, none o# w ic as been ade$uatelyaddressed% OBE, at one time or anot er, asappealed to bot t e &ig t and t e Le#t% Itsgenesis was during t e !rst Busadministration w ile its current ad erents aremore li'ely to be social utopians orpro#essional administrators% e &ig t sawOBE as a means to accountability,productivity, and particularizing standards% eLe#t saw OBE as an engine #or social c ange,attitude engineering, and in#using ideologyinto curriculum% at s w y t e OBEvocabulary is suc a

loopy con*ation o# edu+babble, computer "argon, and t erapy+spea'% I summarize t e!ve ob"ections below #or t e bene!t o# ot er#aculty #acing t is latest, management+driven,educational #ad% e $uotations are allassertions made by our Learning Outcomes

as' -orce%.

I% Assessment rubrics and learningoutcomes will not a/ect teac erevaluation%.

'onsense3 'o one can ma>e such a guarantee3Assessment schemes and e0%ected outcomes areeasily ada%ted or use on teacher evaluation orms3For e0am%le ? o= =ell did the teacher e0%lain your classDs learning outcomesE@ And ? o= =ell did theteacherDs learning activities acilitate class and collegelearning outcomesE@ #aybe not this year or ne0t but?learning outcomes@ are a technocratDs idea o education: lo= charts gra%h %a%er and scores3

II% Assessment does not intrude on yourclassroom%. O# course it does, in t e most#undamental way%

/very com%etent teacher has goals and gradingcriteria or his or her classesI many o us have usedsuch schemes as =riting (nstructional Objectives or setting Cognitive and A ective $omain goals3 5here?Assessment@ intrudes is by insisting that all learningis observable and measurable3 This may be true ins>ill develo%ment or %er ormance courses nursing or cello but it is clearly alse in humanities or artcourses3 There as one Gose%h Conrad character said o those =ho travel to A rica ?The changes ta>e %laceinside you >no=3@ o= does a student come torealiBe that #oBart is better than 7ritney S%ears or

#ichelangelo better than Thomas Uin>adeE 'o charto the measurable and observable =ill tell you yet itsurely involves learning3

III% Learning outcomes do not compromiseacademic #reedom%.

Academic reedom is a com%le0 issue but basically its %ractice insures that students =ill be e0%osed tovarious academically legitimate yet contradictoryideas3 That is they =ill be dra=n into ?the !reatConversation @ not sim%ly inoculated =ith a currently

%revailing orthodo0y3 8ni ormity o in%ut is anathemato academic reedomI uni ormity o outcome isinhuman3 A ter over - years teaching ( still have noidea =hat any individual student =ill ?get out@ o aclass3

I0% All students can succeed%.

This %remise is idealistic but misguided3 The only =ayto insure eJual outcomes is to =ater do=n standards3All students must have eJual o%%ortunity but eachstudent is a uniJue and com%le0 individual3 Thereasons or ?success@ or ? ailure@ cannot be teaseda%art rom the mysteries o %ersonality and talent3

0% ere s ould be unanimous learningoutcomes #or t e w ole college%.

(m%ossible as =ell as undesirable and most disturbing=hen es%ousing nebulous thera%eutic or value)charged goals3 One teacher may %riBe collaboration=hile another values sel )reliance3 One avors

?!lobalism@ =hile another avors ?!lobaliBation3@One teacher is !reen another is Libertarian3 This is asit should be3 ou sim%ly canDt have a collegecommitment both to ?diversity@ and to ?unanimity3@ThatDs hy%ocrisy3 (n college education as in scienceres%ect ul learned disagreement is an essential %art o the %rocess3 O7/ is also behaviorist S>innerianconcerned solely =ith ('P8T and O8TP8T ignoring=hat ha%%ens in bet=een3 $ee% learning is %rivateinvisible and reJuently ine able3 O ten it isdangerous u%setting and un%redictable3 ou canDt %utit on the (nternet and you canDt turn it into aPo=erPoint magic lantern sho=3 5hat ( ind is that

O7/ and Learning Outcomes and Assessment are notabout education at allI they are about control3 'othingis more seductive to ideologues and to managementthan the %ros%ect o creating a meaningless ?jargonand data storm@ to justi y or conceal =hatever they do35here does it endE As 5illiam S3 7urroughs said ? 3 33 control can never be a means to any %racticalend 3 3 33 (t can never be a means to anything but morecontrol 3 3 3 @ 1 3

5or> Cited

2&A%%endi0 1/


27/73

Burroug s, 1illiam 2% 3a'ed Lunc % 3ew 4or'56rove 7ress, 89:;% &eissue edition 899


28/73

Appendi/ 2 > Activit& D< ?riting Student (earning *utcomes*evie= the irst e0am%le3 Then or the second course objective com%lete the Per ormance Conte0t #easurable Objective and Primary Traits3

Finally select an objective rom a course in your disci%line and construct the three)%art SLO statement3

Course *b-ective "erformance Conte/t #easurable *b-ective 1rading CriteriaE "rimar&Traits

#atch the various ty%es o sheetmetal =elding methods to thea%%ro%riate a%%lication3

!iven s%eci ications andmaterials reJuiring a =eld

evaluate the %er ormance needsand match the =elding method tothe reJuired a%%lication3

5elds should have a Juality edge joint meet design s%eci icationshave an evenly %ositioned =eld

bead =ith good %enetration and

have the minimum heat)a ectedBone to ma0imiBe strength o the=eld3

$emonstrate and develo% correct>eyboarding techniJuesa%%licable to >eyboarding bytouch or s%eed and accuracy3

2+A%%endi0 2


29/73

Appendi/ < > eveloping and Appl&ing +ubricsMary Allen, CSU Institute for Teaching & Learning, [email protected]

Scoring rubrics are e0%licit schemes or classi ying %roducts or behaviors into categories thatvary along a continuum3 They can be used to classi y virtually any %roduct or behavior such as

essays research re%orts %ort olios =or>s o art recitals oral %resentations %er ormances andgrou% activities3 Gudgments can be sel )assessments by studentsI or judgments can be made byothers such as aculty other students ield=or> su%ervisors and e0ternal revie=ers3 *ubrics can

be used to %rovide ormative eedbac> to students to grade students andness3 (t may be un ocusedunderdevelo%ed or rambling: Problems =ith the use o language seriouslyinter ere =ith the reader s abilit y to understand =hat is being communicated3

$evelo%ing The essay may be some=hat un ocusedI underdevelo%ed or rambling but it doesCom%etence have some coherence3 Problems =ith the use o language occasionally inter ere

=ith the reader s ability to understand =hat is being communicated:Acce%table The essay is generally ocused and contains some develo%ment o ideasI but the

discussion may be sim%listic or re%etitive3 The language lac>s syntacticcom%le0ity and may contain occasional grammatical errors but the reader is ableto understand =hat is being communicated3

So%histicated The essay is ocused and clearly organiBed and it sho=s de%th o develo%ment3 )

The language is %recise and sho=s syntactic variety and ideas are clearlycommunicated to the reader3

Anal&tic +ubric for "eer Assessment of Team "ro-ect #embersBelo /pectation 1ood /0ce% tional

Project #ade e= substantive Contributed a M air ContributedContributions contributions to the shareM o substance to considerable

team s inal %roduct the team s inal %roduct substance to theteam s inal %roduct:3

Leadershi% *arely or never Acce%ted a M air shareM *outinely %rovided

e0ercised leadershi% o leadershi% e0cellent leadershi%res%onsibilitiesCollaboration 8ndermined grou% *es%ected other s *es%ected other s

discussions or o ten o%inions)and contributed o%inions and madeailed to %artici%ate to the grou% s discussion major contributions to

the grou% s discussion

2,A%%endi0 31


30/73

*nline +ubricsFor lin>s to online rubrics go to htt%:


31/73

Suggestions for )sing +ubrics in Courses 13 and out the grading rubric =ith the assignment so students =ill >no= your e0%ectations

and ho= they ll be graded3 This should hel% students master your learning objectives byguiding their =or> in a%%ro%riate directions3

23 8se a rubric or grading student =or> and return the rubric =ith the grading on it3 Facultysave time =riting e0tensive commentsI they just circle or highlight relevant segments o therubric3 Some aculty include room or additional comments on the rubric %ageI either=ithin3)each section or at the end3

3 $evelo% a rubric =ith your students or an assignment or grou% %roject3 Students can thenmonitor themselves and their %eers using agreed)u%on criteria that they hel%ed develo%:#any aculty ind that students =ill create higher standards or themselves than aculty=ould im%ose on them3

"3 ave students a%%ly your rubric to some sam%le %roducts be ore they create their o=n:

Faculty re%ort that students are Juite accurate =hen doing this and this %rocess should hel%them evaluate their o=n %roducts as they are being develo%ed3 The ability to evaluate editand im%rove dra t documents is an im%ortant s>ill3

43 ave students e0change %a%er dra ts and give %eer eedbac> using the rubric then givestudents a e= days be ore the inal dra ts are turned in to you3 ou might also reJuire thatthey turn in the dra t and scored rubric =ith their inal %a%er3

&3 ave students sel )assess their %roducts using the grading rubric and hand)in the sel )assessment =ith the %roductI then aculty and students can com%are sel ) and acultygenerated evaluations3

Sometimes a generic rubric can be used and it can be refined as raters become more'e/perienced or as problems emerge 3

!eneric *ubric or Assessing Port olios)nacceptable #arginal Acceptable /ceptional/vidence that the /vidence that the /vidence sho=s /videncestudent has student has that the student demonstrates thatmastered this mastered this has generally the student hasobjective is not objective is attained 3this mastered this

%rovided %rovided but it is objective3 objective at aunconvincing or =ea> or high level3very incom%lete3 incom%lete3

LearningOb jective (LearningObjective 2LearningObjective

1A%%endi0 3


32/73

Steps for Creating a +ubric13 (denti y =hat you are assessingI e3g3I critical thin>ing32: (denti y the characteristics o =hat you are assessing e3g3 a%%ro%riate use o evidence

recognition o logical allacies33 $escribe the best =or> you could e0%ect using these characteristics3 This describes the to% category3

"3 $escribe the =orst acce%table %roduct using these characteristics3 This describes the lo=estacce%table category3

43 $escribe an unacce%table %roduct3 This describes the lo=est category3&3 $evelo% descri%tions o intermediate)level %roducts and assign them to intermediate categories3 ou

might decide to develo% a scale =ith ive levels e3g3 unacce%table marginal acce%table com%etentoutstanding three levels e3g3 novice com%etente0em%lary or any other set that is meaning ul3

93 As> colleagues =ho =ere not involved in the rubric s develo%ment to a%%ly it to some %roducts or behaviors and revise as needed to eliminate ambiguities3

1roup +eadings*ubrics can be a%%lied by one %erson but grou% readings can be very e ective because they bring acultytogether to analyBe and discuss student learning3 ( data are aggregated as results come in the grou%reading can end =ith a discussion o =hat the results mean =ho needs to >no= the resultsI =hatres%onses might be reasonable e3g3 curricula %edagogy or su%%ort changes and ho= the assessment

%rocess itsel could be im%roved3?ho should be invited to group readings@Faculty and others e3g3 graduate students ield=or> su%ervisors community %ro essionalses%ecially those =ho control and o er the curriculum and =ho can ma>e valid in ormed

judgments about student learning3

#anaging 1roup +eadings13 ( the reliability o the rubric is >no=n to be high it may be reasonable to have only one reader

analyBe each document but it generally is %re erable to use t=o readers so that inter)rater

reliability can be e0amined and discre%ancies can be identi ied and resolved323 5hen t=o readers =or> inde%endently the second reader may be allo=ed to %ee> at the irst

rater s judgments3 *eaders o ten are curious about other s o%inions and no harm is done i theirst rater s scores are hidden until a ter the second o%inions have been recorded3

3 Sometimes results are monitored as they are turned in and documents are given to a third reader =hen necessary to resolve discre%ancies3 For e0am%le the acilitator may send any documentthat has a scorer di erence o more than one %oint to a third reader =ho determines =hich ratingis more accurate3

"3 Sometimes readers =or> in %airs inde%endently rating each document then jointly resolving alldisagreements3 They may be as>ed to discuss only the ratings that di er by some amount suchas at least t=o units3

43 5hen t=o rates disagree aculty must decide =hich rating =ill be used in the analysis or theymay decide to use both3 5hatever the decision the %roject re%ort should document ho= data=ere generated3

2A%%endi0 3"


33/73

Scoring +ubric 1roup *rientation and Calibration

13 $escribe the %ur%ose or the revie= stressing ho= it its into %rogram assessment %lans3 /0%lain that the %ur%ose is to assess the %rogram not individual students or acultyI and describe ethical guidelines including res%ect or con identiality and %rivacy:

23 $escribe the nature o the %roducts that =ill be revie=ed brie ly summariBing ho= they =ereobtained3

3 $escribe the scoring rubric and its categories3 /0%lain ho= it =as develo%ed3"3 /0%lain that readers should rate each dimension o an analytic rubric se%arately and they should

a%%ly the criteria =ithout concern or ho= o ten each category is used343 !ive each revie=er a co%y o several student %roducts that are e0em%lars o di erent levels o

%er ormance3 (nclude i %ossible a =ea> %roduct an intermediate)level %roduct a strong %roductand a %roduct that a%%ears to be %articularly di icult to judge3 As> each volunteer to inde%endentlya%%ly the rubric to each o these %roducts and sho= them ho= to record their ratings3

&3 Once everyone is done collect everyone s ratings and dis%lay them so everyone can see the degreeo agreement3 This is o ten done on a blac>board =ith each %erson in turnannouncing his raters to raise their hands =hen their rating category is announced ma>ing thee0tent o agreement very clear to everyone and ma>ing it very easy to identi y raters=ho routinely give unusually high or lo= ratings3

93 !uide the grou% in a discussion o their ratings3 There =ill be di erences and this discussion isim%ortant to establish standards3 Attem%t to reach consensus on the most a%%ro%riate rating or eacho the %roducts being e0amined by inviting %eo%le =ho gave di erent ratings to e0%lain their

judgments3 8sually consensus is %ossible but sometimes a s%lit decision is develo%ed e3g3 thegrou% may agree that a %roduct is a ? )"M s%lit because it has elements o both categories3 /0%ectmore discussion time i you include a hard)to)score e0am%leI but be a=are that its inclusion =illsave everyone grie later because such documents are bound to occur3 ou might allo= the grou% torevise the rubric to clari y its use but avoid allo=ing the grou% to dri t a=ay rom the learningobjective being assessed3

+3 Once the grou% is com ortable =ith the recording orm and the rubric distribute the %roducts and begin the data collection3

,3 ( you accumulate data as they come in and can easily %resent a summary to the grou% at the end othe reading you might end the meeting =ith a discussion o our Juestions:

a3 5hat do the results meanE b3 5ho needs to >no= the resultsEc3 5hat are the im%lications o the results or curriculum %edagogy or student su%%ort

servicesEd3 o= might the assessment %rocess itsel be im%rovedE

1-3 (t can be use ul to set u% a s%readsheet to calculate means reJuencies and reliability3 Then discuss=hat the scores mean or your curriculum teaching methods students etc3 *e%ort the inter)rater

reliability: H W or 1 %oint H W or 2 %oints etc3

A%%endi0 34


34/73

Appendi/ 4 > /amples of Scoring +ubrics

?#a% *ubric@ is a scoring tool or the Online #a% Creation =eb site ===3aJuarius3geomar3de or KThe Living /nvironmentD 7(OL 111@ is a rubric or =riting in7iology A)F scales =ith de initions at Southern (llinois 8niversity /d=ardsvillehtt%:


35/73

Appendi/ =A > Activit& D4 Building a +ubricSelect or =rite an SLO identi y Primary Traits and then decide on ?observables@ or each assessment level

S(* Statement

Trait /cellent Satisfactor& )nsatisfactor& Score

Total

4A%%endi0 4


36/73

Appendi/ =B > Scoring +ubric > nglish epartment > #odesto $unior College

/cellent #ar3edl& /ceptional Superior Clearl& AboveAverage

Satisfactor& Full&Competent

"oor #arginall& Acceptable Failing )nacceptable

$evelo

%ment

A com%rehensive gras% o the subjectmatter is demonstrated

7ody is develo%ed =ith originalinsight ul and creative su%%ortI the

%a%er goes beyond re%eating =hatothers have said and contributessomething ne= to our understanding o the to%ic

Focus is clear imaginative and ully

realiBed $emonstrates s%eci ic attention torelationshi% bet=een audience and

%ur%ose

A thorough gras% o the subjectmatter is demonstrated

Focus is clear and thought ul 7ody is generally su%%orted by

acts e0am%les etc3 thoughsu%%ort =ill not be as varied or vivid as in an e0cellent %a%er

$emonstrates understanding o audience and %ur%ose though

may occasionally stray rom it

A basic gras% o the subjectmatter is demonstrated

Focus is generally adeJuate but may not be immediatelyclear to all readers

*es%onse to the assignmentis generally adeJuate

7ody su%%orted by actse0am%les details but are

mainly sur ace oriented andgeneraliBed $emonstrates only some

understanding o audienceand %ur%ose

A lac> o amiliarity =ith thesubject matter is demonstrated

Focus is vague either toogeneral too narro= su%er icialor indirect

7ody su%%orted by e=e0am%les or actsI manye0am%les are unanalyBed

$emonstrates %oor

understanding o audience and %ur%ose

A basic lac> ounderstanding o thesubject matter isdemonstrated

Focus is not evident 7ody largely

unsu%%orted byrelevant acts ore0am%les

$emonstrates nounderstanding oaudienceing

#inimal organiBationina%%ro%riate or n

%aragra%hing (ne ective or missi

introduction andconclusion

#inimal or no use otransitions

Styleing a Stand: igh School /0it /0am

igh school is stress ul considering issue such as: %eer %ressure the struggle o %assing classes

and trying to maintain a high !rade Point Average3 #ost students are des%erately trying to >ee%

themselves a loat in this society o raging =aters3 They eel they cannot handle anything else3 For many

o them can hardly carry =hat they have already3 'o= students have one more burden to carry and that is

the high school e0it e0am3

Learning contains many >ey %rinci%les but the most basic o all is desire3 The students have to

have a %assion to learn3 #any argue this e0am hurts the ?under%rivileged@ such as minorities and lo=)

income amilies but is this true 7ur>e ?/0it /0am@ 74 3 The greatest hindrance that >ee%s most

students rom learning is %roblems o drugs alcohol and domestic violence3 These %roblems are ound

both in the homes o the rich the %oor and almost o any ethnic bac>ground3 There isnDt any good reason

=hy a student that doesnDt have any disabilities or language barriers should have %roblems learning3

Students that have such %roblems concerning language barriers and disabilities should be %rovided

%rograms that =ill steadily %re%are them or the e0it e0am3 igh School students should be made to ta>e

the e0it e0am to ma>e sure they are %rogressing and that they have basic s>ills to survive li e to get good

jobs or to %ursue careers3 There shouldnDt be a student le t behind not >no=ing their basic s>ills o

reading =riting and math3

$uring high school the greatest amount o %rogression should be made above any other time in

grade school and this can only be done =ith the hel% o our school system3 5hat is done bet=een grades

, th to 12 th does matter or =hatever they learn bet=een these grades they =ill %robably carry =ith them

or the rest o their lives3 The teachers should hel% the students %rogress by ully e0%laining =hat goals

,A%%endi0 &


40/73

they =ant the students to meet3 They should %ush the student to thin>: al=ays getting them involved in

every class %roject and discussion3 There needs to be an interaction bet=een teachers and students3 The

class should never loo> bored and stagnant3 There is a great need or o%en communication bet=een

teachers and students3 Students should be able to come to the teacher i they have any trouble =ith the

assignment or any other issues %ertaining to any o their educational needs3

( they are %lanning on giving an e0amI that test high school students abilitiesI the schools should

ully %re%are the teachers and the students3 Teachers should be made to teach all the materials that =ill be

on the e0ams year around or the ull our years o high school3 Students should be tested every year so

they can see =here they need to %rogress or the u%coming year3 This =ill be hel% ul to both the teacher

and the student3 C'' Student 'e=s center director Gac> Gennings said ? ou have to %rovide a system to

hel% >ids succeed\These test are a good idea i theyDre done right Gennings Jtd3 (n ?States stic>@ 3 ?5e

cannot just dro% an e0am on studentDs la%s and e0%ect that they ta>e it i =e donDt ully %re%are them3 'o

%art o the e0am should be a mystery to themI it should all be revie=3 Students on the other hand should

be made accountable or =hat they learn3 They should study o ten3 This e0am is su%%osed to test =hat

they have learned during these %ast our years o their lives3 ( =e go about this the right =ay this e0am

should be li>e any other test or the student3

This e0am should be ta>en so that the student =ill have the basic s>ills to survive li e3 /veryday

i =e realiBe it or not =e are surrounded by =riting reading and mathematics3 For e0am%le anytime =e

go to the store =e use math =hether it is or calculating -H o o item on sale or giving and receiving

money rom the cahier3 Another e0am%le is the ability to read or =rite and its im%ortant usage or the

voter in an election3 (ts im%ortance is beyond our reasoning or =e really have to >no= =hat =e are

reading =hen it has to do =ith dra ting in di erent la=s3 /veryday =e are surrounded by these

obscurities that call or basic s>ills s>ills that may loo> non use ul but one)day students =ill need3

Once students graduate rom high school thatDs =hen li e really begins3 They =ill most li>ely use

all they learned in high school in college and even a ter that in the =or> %lace3 All students =ill need"-A%%endi0 &


41/73

these basic s>ills o reading =riting and math in their jobs and also in =hatever career they decide to

%ursue3 The =hole %oint o the e0am is to encourage students to %rogress so they =onDt eel lost and

con used =hen they graduate and try to ind a job or see> a %ro ession3

The high school e0it e0am shouldnDt even be a debate i its just basic material that high school

students should already >no=3 ?$avid Coo%er director o secondary education or #odesto City Schools

said students may ta>e the test u% to eight times and most =ill eventually %ass Jtd3 (n erendeen

?Students Cheer@ A1 3@ Students shouldnDt ?eventually@ understand the materialI they should >no= the

material Jtd3 (n erendeen ?Students Cheer: A1 3@ The reason =hy ta>ing the high school e0it e0am is

an issue is because they donDt already >no= the basic material =hich =ill be sooner or later in li e be %ut

be ore them3 5e need to go bac> to the basics and ma>e sure that math reading and =riting are being

taught be ore any other materials3 These basics need to be %riority and any other e0tra curricular subject

secondary3 The only =ay =e can ma>e sure students are being taught is to test their abilities3 5e need to

strive together as a %eo%le and ma>e sure students are learning3 5e =ant students to leave high school

>no=ing they have %rogressed that they have learned something o great value3 They should eel

con ident =hen they get out o high school3 They should have the ability and o%%ortunity to survive in

li e get a good job and %ursue the career o their dreams3 (t is our res%onsibility to ma>e sure they have

their eet %lanted on solid ground ready to go out in this =orld and ma>e a di erence3

5or>s Cited

7ur>e Fran>3 Letter3 The #odesto 7ee 2+ Gune 2-- : 74

erendeen Susan3 Letter3 The #odesto 7ee 1- Guly 2-- : A1

?States stic> =ith high)school e0it e0am3@ C'' Student 'e=s 2- Aug3 2-- 3 123 Oct3

2--

]htt%:


42/73

Appendi/ ; > The Case for Authentic Assessment b& 1rant ?igginshtt%:no=ledge3 Traditional teststend to reveal only =hether the student can recogniBe recall or M%lug inM =hat =as learned out o conte0t3 Thismay be as %roblematic as in erring driving or teaching ability rom =ritten tests alone3 'ote there ore that thedebate is not Meither)orM: there may =ell be virtue in an array o local and state assessment instruments as be itsthe %ur%ose o the measurement3

Authentic assessments %resent the student =ith the ull array o tas>s that mirror the %riorities and challengesound in the best instructional activities: conducting researchI =riting revising and discussing %a%ersI %roviding

an engaging oral analysis o a recent %olitical eventI collaborating =ith others on a debate etc3 Conventionaltests are usually limited to %a%er)and)%encil one) ans=er Juestions3

Authentic assessments attend to =hether the student can cra t %olished thorough and justi iable ans=ers %er ormances or %roducts3 Conventional tests ty%ically only as> the student to select or =rite correct res%onses))irres%ective o reasons3 There is rarely an adeJuate o%%ortunity to %lan revise and substantiate res%onses onty%ical tests even =hen there are o%en)ended Juestions 3 As a result

Authentic assessment achieves validity and reliability by em%hasiBing and standardiBing the a%%ro%riate criteriaor scoring such varied %roductsI traditional testing standardiBes objective MitemsM and hence the one right

ans=er or each3 MTest validityM should de%end in %art u%on =hether the test simulates real)=orld MtestsM o ability3 ;alidity on

most multi%le)choice tests is determined merely by matching items to the curriculum content or throughso%histicated correlations =ith other test results 3

Authentic tas>s involve Mill)structuredM challenges and roles that hel% students rehearse or the com%le0ambiguities o the MgameM o adult and %ro essional li e3 Traditional tests are more li>e drills assessing staticand too)o ten arbitrarily discrete or sim%listic elements o those activities37eyond these technical considerations the move to re orm assessment is based u%on the %remise that

assessment should %rimarily su%%ort the needs o learners3 Thus secretive tests com%osed o %ro0y items andscores that have no obvious meaning or use ulness undermine teachers ability to im%rove instruction and studentsability to im%rove their %er ormance3 5e rehearse or and teach to authentic tests))thin> o music and militarytraining))=ithout com%romising validity3

The best tests al=ays teach students and teachers ali>e the >ind o =or> that most mattersI they are enabling andor=ard)loo>ing not just re lective o %rior teaching3 (n many colleges and all %ro essional settings the essential

challenges are >no=n in advance))the u%coming re%ort recital 7oard %resentation legal case boo> to =rite etc3Traditional tests by reJuiring com%lete secrecy or their validity ma>e it di icult or teachers and students torehearse and gain the con idence that comes rom >no=ing their %er ormance obligations3 A >no=n challenge alsoma>es it %ossible to hold all students to higher standards 35 $O 5/ '//$ TO (';/ST (' T /S/ LA7O*)('T/'S(;/ FO*#S OF ASS/SS#/'TE

5hile multi%le)choice tests can be valid indicators or %redictors o academic %er ormance too o ten our testsmislead students and teachers about the >inds o =or> that should be mastered3 'orms are not standardsI items arenot real %roblemsI right ans=ers are not rationales3

5hat most de enders o traditional tests ail to see is that it is the orm not the content o the test that is harm ulto learningI demonstrations o the technical validity o standardiBed tests should not be the issue in the assessmentre orm debate3 Students come to believe that learning is crammingI teachers come to believe that tests are a ter)the)

"2A%%endi0 &
http://www.ericfacility.net/databases/ERIC_Digests/ed328611.htmlhttp://www.ericfacility.net/databases/ERIC_Digests/ed328611.html


43/73

act im%osed nuisances com%osed o contrived Juestions))irrelevant to their intent and success3 7oth %arties are ledto believe that right ans=ers matter more than habits o mind and the justi ication o one s a%%roach and results3

A move to=ard more authentic tas>s and outcomes thus im%roves teaching and learning: students have greater clarity about their obligations and are as>ed to master more engaging tas>s and teachers can come to believe thatassessment results are both meaning ul and use ul or im%roving instruction3

( our aim is merely to monitor %er ormance then conventional testing is %robably adeJuate3 ( our aim is toim%rove %er ormance across the board then the tests must be com%osed o e0em%lary tas>s criteria and standards3

5O' T A8T /'T(C ASS/SS#/'T 7/ TOO /^P/'S(;/ A'$ T(#/)CO'S8#('!EThe costs are dece%tive: =hile the scoring o judgment)based tas>s seems e0%ensive =hen com%ared tomulti%le)choice tests about _2 %er student vs3 1 cent the gains to teacher %ro essional develo%ment localassessing and student learning are many3 As states li>e Cali ornia and 'e= or> have ound =ith their =riting andhands)on science tests signi icant im%rovements occur locally in the teaching and assessing o =riting and science=hen teachers become involved and invested in the scoring %rocess3

( costs %rove %rohibitive sam%ling may =ell be the a%%ro%riate res%onse))the strategy em%loyed in Cali ornia;ermont and Connecticut in their ne= %er ormance and %ort olio assessment %rojects3 5hether through a sam%lingo many =riting genres =here each student gets one %rom%t onlyI or through sam%ling a small number o allstudent %a%ers and school)=ide %ort oliosI or through assessing only a small sam%le o students valuablein ormation is gained at a minimum cost3 And =hat have =e gained by ailing to adeJuately assess all the ca%acitiesand outcomes =e %ro ess to value sim%ly because it is time) consuming e0%ensive or labor)intensiveE #ost other

countries routinely as> students to res%ond orally and in =riting on their major tests))the same countries thatout%er orm us on international com%arisons3 #oney time and training are routinely set aside to insure thatassessment is o high Juality3 They also correctly assume that high standards de%end on the Juality o day)to)daylocal assessment)) urther o setting the a%%arent high cost o training teachers to score student =or> in regional or national assessments3

5(LL T / P87L(C A;/ A' FA(T (' T / O7G/CT(;(T A'$ */L(A7(L(T OF G8$!#/'T)7AS/$SCO*/SE

5e orget that numerous state and national testing %rograms =ith a high degree o credibility and integrity haveor many years o%erated using human judges:

the 'e= or> *egents e0ams %arts o =hich have included essay Juestions since their ince%tion))and =hichare scored locally =hile audited by the state I

the Advanced Placement %rogram =hich uses o%en)ended Juestions and tas>s including not only essays onmost tests but the %er ormance)based tests in the Art Port olio and Foreign Language e0amsI

state)=ide =riting assessments in t=o doBen states =here model %a%ers training o readers %a%ers read MblindMand %rocedures to %revent bias and dri t gain adeJuate reliabilityI

the 'ational Assessment o /ducational Progress 'A/P the Congressionally)mandated assessment usesnumerous o%en)ended test Juestions and =riting %rom%ts and success ully %iloted a hands)on test o science

%er ormance I ne=ly)mandated %er ormance)based and %ort olio)based state)=ide testing in AriBona Cali ornia Connecticut

Uentuc>y #aryland and 'e= or>3Though the scoring o standardiBed tests is not subject to signi icant error the %rocedure by =hich items are

chosen and the manner i

slo focus on results

Documents