lowe-the extraordinary potential of a smart assessment ......the extraordinary potential of a smart...

!"!#"$%

$

The Extraordinary Potential of a SMART Assessment Program

Crafting and Leading a

Highly Effective Assessment Program

AgendaSection 1 - Philosophy &

Methodology of Assessment

Understand the strengths of an advanced philosophy of assessmentExplore the structure of a sound assessment methodologyWeigh the pros and cons of various approachesKnow the key components of a superior assessment program

Section 2 - Crafting and Leading

Highly Assessment Program

Building the right foundation-leadership buy in, policies, processesCrafting a sound plan for a successful launch - when and how to startReview the key components for assessment managementAvoiding common mistakes that can unravel your plans and wreck your results

!"!#"$%

!

Preview Question 1

Developing an institutional effectiveness plan with program goals and student learning outcomes is a fad that will likely be replaced by another agenda.1. Strongly Disagree2. Somewhat Disagree 3. Somewhat Agree 4. Strongly Agree

Preview Question 2

Program goals and student learning outcomes are important to institutional quality and student success. Are these important enough to evaluate the effectiveness of each?1. Strongly Disagree2. Somewhat Disagree 3. Somewhat Agree 4. Strongly Agree

!"!#"$%

&

Preview Question 3

Analyzing program goals and student learning outcomes are not an important part of my job responsibilities--primarily the responsibility of university staff.1. Strongly Disagree2. Somewhat Disagree 3. Somewhat Agree 4. Strongly Agree

Preview Question 4

Program goal and student learning outcome attainment would be strengthened by active participation by faculty members.1. Strongly Disagree2. Somewhat Disagree 3. Somewhat Agree 4. Strongly Agree

!"!#"$%

&

Building the Right Foundation -Leadership buy in, policies, and processes

Session 2.1

Buy - in……depends on seeing the impact --

…results are used to make improvements…if the improvement meets a need

expressed by leadership, faculty, staff, and students

...individuals will begin to investMust represent a value…

!"!#"$%

&

Laying a Foundation: ExpectationsLeadership…

Articulates vision…defines strategic direction (use evidence)Articulates where we are now…what will new direction accomplishArticulates how it aligns with missionArticulates how it will serve students, faculty, programs and institution

Plan, Policies, and Process

!"!#"$%

&

Place your screenshot here

11

Establish Who Will Lead the Process of Planning and Assessment

Place your screenshot herePlace your screenshot here

Crafting a Sound Plan for a successful

Launch - When and How to Start

Session 2.2

!"!#"$%

#

!"##$%%$&#'&()#'*#+,&-*#'./%+0123

!"#$%&'()#)*+#',-."-/-'%#%&0-/)&'1+23-4)5-#3-"/6&.)$7")89:);3<&'&/%"#%&7':)=#12+%>:)?%#$$:)?%23-'%/

9/%#@+&/6)%6-)*2".7/-)7$);//-//<-'%

?%#"%)A&%6)%6-)B8C)*&1%2"-)

!"##$%"&$'(")*()*+,$-.

Getting Started… 14

Provide Professional Development

! Share process and cycle.

! Provide templates for process.

! Provide training if using data system

Establish Committee Structure

! Strategic Planning ! Assessment Team! May impact

Curriculum Team

Establish Framework

! Committee sets policy and process.

! Design plan! Determine data

warehousing! Determine review

cycle and timeline

!"!#"$%

&

Most Important: Plans that Close the Loop (Act on Results)Making changes to improve…

Student Learning

ServiceOperations

15

University of Florida’s: Purpose, Mission, Vision for Institutional AssessmentExample

!"#$#%&$'()*+$, -"'$.&/.%*'$%0$12$)*$+%$*&..%/+$+"'$34$5)**)%6$7#$'*+879)*")6:;$58)6+8)6)6:;$86<$/'0)6)6:$+"'$&6)='/*)+#>*$)6*+)+&+)%689$'00'?+)='6'**$86<$8**'**5'6+$./%?'**'*@

!"8+$#%&$<%$, -"'$5)**)%6$%0$12$)*$+%$9'8<$+"'$36)='/*)+#>*$'00%/+*$)6$8??/'<)+8+)%6$86<$)6*+)+&+)%689$'00'?+)='6'**;$8**'**5'6+$*&..%/+;$86<$+%$58)6+8)6$+/86*.8/'6+$?%55&6)?8+)%6$A)+"$899$34$*+8B'"%9<'/*@

!"'/'$#%&$:%$, !'$'6=)*)%6$34$8*$86$)6*+)+&+)%6$A"'/'$899$&6)+*$C$8?8<'5)?$./%:/85*$?%6+/)7&+'$+%$+"'$0&90)995'6+$%0$+"'$5)**)%6$7#$'*+879)*")6:$:%89*;$%&+?%5'*;$/':&98/9#$8**'**)6:$A)+"$=89)<;$/'9)879'$5'8*&/'*$86<$&*'$/'*&9+*$0%/$?%6+)6&%&*$)5./%='5'6+@

!"!#"$%

%

Academic Assessment Plan Components 17

!"#$%&'"(!))%))&%*+(

,-#*

,./01)%2(3'))'1*2(4')'1*(51#-)

6+.$%*+(7%#/*'*8(9.+"1&%)

:.//'".-.&;(!))%))&%*+(

3#0)!))%))&%*+(

:<"-%(

3%+=1$)(#*$(

,/1"%$./%)

!))%))&%*+(9>%/)'8=+

Balanced Academic Assessment Begins with Data Sources

!"#$%"&' ()#$%"&'

! "#$%&'()&&'&&*'+,&! -'%.#%*/+0'(

)&&'&&*'+,&

*+%$,#

! "%'1'+,2/3('4/*! 5/,2#+/3(6'+'%/3(718(

74/*&! 9/:#%(;2'31(<'&,&(=7<>?

*+%$,#

! @+AB#$&'(>/,2&./0,2#+(>$%C'D&(=/++$/33D?

! 6%/1$/,'(742,(>$%C'D

!"-+%$,#! 5/,2#+/3(>$%C'D(#.(>,$1'+,(

7+E/E'*'+,(=5>>7?! ;/0$3,D(>$%C'D(#.(>,$1'+,(

7+E/E'*'+,(=;>>7?! )3$*+2(>$%C'D&

!"-+%$,#

*&#&./0%,$1

()#$%"&'!"#$%"&'

Balanced Academic Assessment Begins with Data Sources

!"#$%"&' ()#$%"&'

! "#$%&'()&&'&&*'+,&! -'%.#%*/+0'(

)&&'&&*'+,&

*+%$,#

! "%'1'+,2/3('4/*! 5/,2#+/3(6'+'%/3(718(

74/*&! 9/:#%(;2'31(<'&,&(=7<>?

*+%$,#

! @+AB#$&'(>/,2&./0,2#+(>$%C'D&(=/++$/33D?

! 6%/1$/,'(742,(>$%C'D

!"-+%$,#! 5/,2#+/3(>$%C'D(#.(>,$1'+,(

7+E/E'*'+,(=5>>7?! ;/0$3,D(>$%C'D(#.(>,$1'+,(

7+E/E'*'+,(=;>>7?! )3$*+2(>$%C'D&

!"-+%$,#

*&#&*&#&*&#&*&#&*&#&./0%,$1

()#$%"&'!"#$%"&'

!"!#"$%

$&

Balanced Academic General Education Data Sources

!"#$%"&' ()#$%"&'

! "#$%&'()*+,-./0! 1'/'%*-("#%'(23*+! 44"5(6$7%.8&

*+%$,#

! (-./!"#$%&'(0 12#,34$5/655$554$"#/.2+#$

! "#--'0.*9'(:'*%/./0(4&&'&&+'/9

! 2;)(<%#=.8.'/8>(<%#=.-'

*+%$,#

! ?/@A#$&'()*9.&=*89.#/()$%B'>&(C*//$*-->D

! 1%*E$*9'(23.9()$%B'>

!"7+%$,#! F*9.#/*-()$%B'>(#=()9$E'/9(

2/0*0'+'/9(CF))2D! G*8$-9>()$%B'>(#=()9$E'/9(

2/0*0'+'/9(CG))2D! 4-$+/.()$%B'>&

F*9.#/*-()$%B'>(#=()9$E'/9(

!"7+%$,#

*&#&*&#&*&#&*&#&*&#&.32%,$5

()#$%"&'!"#$%"&'

What are the data sources you use?

20

!"!#"$%

$$

IE Plans for Support Systems:

Support Services & Operations

21

!"#$%$&$%'"()*+,,-.$%/-"-##*

0)("

1%##%'"*2)%3"4-"$

5'()#

1-(#&6-#7$6($-3%-#*

8(94%"%#$6($%'":*#$&9-"$*#-6/%.-#:*6-#-(6.;<

2##-##4-"$*=/-6#%3;$

Balanced Operational Assessment Begins with Data Sources

!"#$%"&' ()#$%"&'! "#$%&$'()*+,$'-.! /'&%-*'+0-#.(.'-*1-! ,-'-*'()*! 2*#)334-*'! 5&%6-'

"#$%&$'()*+,$'-.

*+%$,#

! 7)8+03$1-4-*'! 5-6(**(*6+/$3$#(-.! 5-*194$#:(*6

*+%$,#

! /'&%-*'+/$'(.;$1'()*! <3&4*(+/&#=->.! "#$%&$'-+2?('+/&#=->

!"-+%$,#! "#$%&$'-+24@3)>-#+/&#=->.! <3&4*(+/&#=->.! 5-*194$#:! A//2+$*%+B//2

!"-+%$,#

*&#&*&#&*&#&*&#&*&#&*&#&./0%,$1

()#$%"&'!"#$%"&'

!"!#"$%

$!

What Comprises the Plan? 23

!"#$%&'(&)(*+,' -".#/%0$%&'-"0$1(2%..%&' 2%..%&'(,'3(*4/0&."(!$,$"5"'$.

6&,+.789:"#$%;". <'&=+"3>"?(!@%++.?(A"B,;%&/.(6/,34,$".(=%++(3"5&'.$/,$"(&'(>/,34,$%&'1

*/&>/,5(,'3(!$43"'$(C",/'%'>(84$#&5".(D!C8.E

*/&#"..(F G#$%;%$H(+";"+?("))%#%"'#H?(#&50+%,'#"I,/>"$(F JB,$(.$43"'$.(=%++(@'&=?(3&?(;,+4"?(9"+%";"(!,$%.),#$%&'(F !$43"'$(!,$%.),#$%&'(,)$"/(."/;%#"

G.."..5"'$(2"$B&3.

I&&+.?(3,$,(.&4/#".?(#4//"'$(5",.4/".(F G/"($B"($&&+.(0/&;%3%'>($B"(,00/&0/%,$"(%')&/5,$%&'K(8L(G/"(="(,.@%'>($B"(/%>B$(M4".$%&'.K

A"'#B5,/@. I,/>"$(,'3(!4##"..(&)(6&,+(G$$,%'5"'$(F N'$"/',+(,'3(OP$"/',+(A"'#B5,/@%'>

L".4+$. G#$4,+(L".4+$.?(N'3%#,$%&'(&)(Q%'3%'>.?(!400&/$%'>(";%3"'#"(DM4,+%$,$%;"E(&/(!$,$%.$%#.(DM4,'$%$,$%;"E(F 2""$(%'$"'3"3(&4$#&5"K

R."(&)(L".4+$. JB,$(#B,'>".(=%++(&##4/($&(%50/&;"(>&,+(#&50+"$%&'K

S"P$(TH#+"K JBH("P0"#$"3(&4$#&5"(=%++(9"(4."3(%'('"P$(H",/U.(0+,'

Review the Key Components for Assessment Management

Session 2.3

!"!#"$%

$&

!"##$%%$&#'&()#'*#+,&-*#'./%+0123

!"#$$%$&'()*"+Program/Unit

Develop Academic Assessment and IE Plans

Set up data reports

Committee Work:Submit for institutional review

Program/Unit:Implement plan

Collect data

Committee Work:Submit for ReportsAnalyze Reports

!"#$%&'()*+,-.(/0,(!1#&*#2-34(5!(6&#37

52'$(03(6&#3 8'1'&0%-34 9..'%2#+&' !"'$%&#,:;0#&7(<(53-2-#2-1'7 =0(;0#&7(>-72'? ;0#&7(>-72'?@(

>-3A#4'(B3.&'#,;0#&7(>-72'?(<(

>-3A'?C-77-03 ;'3',#&(D2#2'$'32(

0/(-32'32@(&#.A7(7%'.-/-.7(

53.&*?'7(%*,%07'E(#&-43$'32(F-2G(

?'%#,2$'32@(7.0%'(&-$-2'?

H&'#,IH03.-7'5?'32-/-'7(%,04,#$(

-$%#.2E(72#A'G0&?',7(#3?(

#&-43$'32(20(.0&&'4'($-77-03(

J+K'.2-1'7 J+K'.2-1'7(302(-3($'#7*,#+&'(2',$7@(

30(2#,4'27@(30(/0.*7(03(72*?'32(

&'#,3-34

C072(0+K'.2-1'7(#,'($'#7*,#+&'@($072(G#1'(2#,4'27@(70$'(/0.*7(03(72*?'32(

&'#,3-34

9&&(0+K'.2-1'7(-3($'#7*,#+&'(2',$7E(/0.*7(03(&'#,3-34(

0*2.0$'7(F-2G(72#2'?(2#,4'27L

!"!#"$%

$&

!"#$%&'()*+,-.(/0,(!1#&*#2-34(5!(6&#3789

52'$(03(6&#3 :'1'&0%-34 ;..'%2#+&' !"'$%&#,<!"%'.2'=(>*2.0$'7 ?03'(:'/-3'=@ :'/-3'=(+*2(302(

#&-43'=(A-2B(0+C'.2-1'D$-77-03

:'/-3'=E(#&-43'=E(#3=(7*%%0,2(

%,0.'77E(2#,4'2(0,(7#2-7/#.2-03

;77'77$'32(F'2B0=

?02(#&&(0*2.0$'7(B#1'($'#7*,'7@(

G0*,7'(4,#='7(*7'=(#7(#($'#7*,'@

53.&*='7(H($'#7*,'(%',(0*2.0$'(#3=(*7'7(=-,'.2(#3=(

-3=-,'.2($'#7*,'7@

F*&2-%&'(=-,'.2D-3=-,'.2(

$'#7*,'7(/0,(70$'(0,(#&&(0*2.0$'7I(6*,%07'/*&(8)'7*&27(/0,(5$%,01'$'32

!"#$%#&'"()#**&+("&,--"(.$/&(+&'/$0&$01&2$0$3#4#0%

!"#$"%&

!"#$"%&

!'%(

)*+,-

5#-$"%4#0%6!(//#3#

!(447%%##*,)$1#47)&$01&89--("%6:-#"$%7(0$/

8%$"%703&'(70%

;#%9"0&%(&'/$0&%(&'"(3"$4;#%9"0&;#*9/%*6'"()#**&+("&<($/&,%%$704#0%

!"!#"$%

$&

Assessmentand Data Reporting Cycle

!"#$%&'"()*'""'+,-).+$&"-)$,/)01#2+34"

5""4""34,#)6&$,,',7

839&434,#)#(4)6&$,)$,/).$#(4:);$#$

8,#4:9:4#)$,/)!<$&1$#4)#(4);$#$

*+/'=>)$,/)839:+<4

303030

!"#$%&#'()*'+),-&%.$'/0.01%&%.$'0.2'+0$0'30*%4)-#5.1

!"!#"$%

$&

Avoiding Common Mistakes that can Unravel our Plans and Wreck your Results

Session 2.4

Mistakes, Omissions, Additions…

Leadership not involvedLack of Getting People on the busLack of Faculty InvolvementLack of InfrastructureLack of Professional DevelopmentLack of Support Personnel and ResourcesDetermining Data Systems and Document Management

!"!#"$%

$#

!"#$%"#&'%()*%+,-.)&'/'+0

!

Thank You!"#$%"&#$'(()*(+"*

!"!#"$%

$&

Anatomy of Key Performance Indicators (KPIs)

Key Performance Indicators (KPIs) are elements of the plan that express what you want to achieve by when. May use 5-7 KPIs to manage and track the progress of their plan.

The anatomy of a structured KPI includes:A Measure – Every KPI must have a measure..A Target – Every KPI needs to have a target that matches your measure and the time period of your goal. A Data Source – Every KPI needs to have a clearly defined data source.Reporting Frequency – Different KPIs have different reporting needs; good rule to follow is to report at least monthly.

35

36