lowe-the extraordinary potential of a smart assessment ......the extraordinary potential of a smart...
TRANSCRIPT
!"!#"$%
$
The Extraordinary Potential of a SMART Assessment Program
Crafting and Leading a
Highly Effective Assessment Program
AgendaSection 1 - Philosophy &
Methodology of Assessment
Understand the strengths of an advanced philosophy of assessmentExplore the structure of a sound assessment methodologyWeigh the pros and cons of various approachesKnow the key components of a superior assessment program
Section 2 - Crafting and Leading
Highly Assessment Program
Building the right foundation-leadership buy in, policies, processesCrafting a sound plan for a successful launch - when and how to startReview the key components for assessment managementAvoiding common mistakes that can unravel your plans and wreck your results
!"!#"$%
!
Preview Question 1
Developing an institutional effectiveness plan with program goals and student learning outcomes is a fad that will likely be replaced by another agenda.1. Strongly Disagree2. Somewhat Disagree 3. Somewhat Agree 4. Strongly Agree
Preview Question 2
Program goals and student learning outcomes are important to institutional quality and student success. Are these important enough to evaluate the effectiveness of each?1. Strongly Disagree2. Somewhat Disagree 3. Somewhat Agree 4. Strongly Agree
!"!#"$%
&
Preview Question 3
Analyzing program goals and student learning outcomes are not an important part of my job responsibilities--primarily the responsibility of university staff.1. Strongly Disagree2. Somewhat Disagree 3. Somewhat Agree 4. Strongly Agree
Preview Question 4
Program goal and student learning outcome attainment would be strengthened by active participation by faculty members.1. Strongly Disagree2. Somewhat Disagree 3. Somewhat Agree 4. Strongly Agree
!"!#"$%
&
Building the Right Foundation -Leadership buy in, policies, and processes
Session 2.1
Buy - in……depends on seeing the impact --
…results are used to make improvements…if the improvement meets a need
expressed by leadership, faculty, staff, and students
...individuals will begin to investMust represent a value…
!"!#"$%
&
Laying a Foundation: ExpectationsLeadership…
Articulates vision…defines strategic direction (use evidence)Articulates where we are now…what will new direction accomplishArticulates how it aligns with missionArticulates how it will serve students, faculty, programs and institution
Plan, Policies, and Process
!"!#"$%
&
Place your screenshot here
11
Establish Who Will Lead the Process of Planning and Assessment
Place your screenshot herePlace your screenshot here
Crafting a Sound Plan for a successful
Launch - When and How to Start
Session 2.2
!"!#"$%
#
!"##$%%$&#'&()#'*#+,&-*#'./%+0123
!"#$%&'()#)*+#',-."-/-'%#%&0-/)&'1+23-4)5-#3-"/6&.)$7")89:);3<&'&/%"#%&7':)=#12+%>:)?%#$$:)?%23-'%/
9/%#@+&/6)%6-)*2".7/-)7$);//-//<-'%
?%#"%)A&%6)%6-)B8C)*&1%2"-)
!"##$%"&$'(")*()*+,$-.
Getting Started… 14
Provide Professional Development
! Share process and cycle.
! Provide templates for process.
! Provide training if using data system
Establish Committee Structure
! Strategic Planning ! Assessment Team! May impact
Curriculum Team
Establish Framework
! Committee sets policy and process.
! Design plan! Determine data
warehousing! Determine review
cycle and timeline
!"!#"$%
&
Most Important: Plans that Close the Loop (Act on Results)Making changes to improve…
Student Learning
ServiceOperations
15
University of Florida’s: Purpose, Mission, Vision for Institutional AssessmentExample
!"#$#%&$'()*+$, -"'$.&/.%*'$%0$12$)*$+%$*&..%/+$+"'$34$5)**)%6$7#$'*+879)*")6:;$58)6+8)6)6:;$86<$/'0)6)6:$+"'$&6)='/*)+#>*$)6*+)+&+)%689$'00'?+)='6'**$86<$8**'**5'6+$./%?'**'*@
!"8+$#%&$<%$, -"'$5)**)%6$%0$12$)*$+%$9'8<$+"'$36)='/*)+#>*$'00%/+*$)6$8??/'<)+8+)%6$86<$)6*+)+&+)%689$'00'?+)='6'**;$8**'**5'6+$*&..%/+;$86<$+%$58)6+8)6$+/86*.8/'6+$?%55&6)?8+)%6$A)+"$899$34$*+8B'"%9<'/*@
!"'/'$#%&$:%$, !'$'6=)*)%6$34$8*$86$)6*+)+&+)%6$A"'/'$899$&6)+*$C$8?8<'5)?$./%:/85*$?%6+/)7&+'$+%$+"'$0&90)995'6+$%0$+"'$5)**)%6$7#$'*+879)*")6:$:%89*;$%&+?%5'*;$/':&98/9#$8**'**)6:$A)+"$=89)<;$/'9)879'$5'8*&/'*$86<$&*'$/'*&9+*$0%/$?%6+)6&%&*$)5./%='5'6+@
!"!#"$%
%
Academic Assessment Plan Components 17
!"#$%&'"(!))%))&%*+(
,-#*
,./01)%2(3'))'1*2(4')'1*(51#-)
6+.$%*+(7%#/*'*8(9.+"1&%)
:.//'".-.&;(!))%))&%*+(
3#0)!))%))&%*+(
:<"-%(
3%+=1$)(#*$(
,/1"%$./%)
!))%))&%*+(9>%/)'8=+
Balanced Academic Assessment Begins with Data Sources
!"#$%"&' ()#$%"&'
! "#$%&'()&&'&&*'+,&! -'%.#%*/+0'(
)&&'&&*'+,&
*+%$,#
! "%'1'+,2/3('4/*! 5/,2#+/3(6'+'%/3(718(
74/*&! 9/:#%(;2'31(<'&,&(=7<>?
*+%$,#
! @+AB#$&'(>/,2&./0,2#+(>$%C'D&(=/++$/33D?
! 6%/1$/,'(742,(>$%C'D
!"-+%$,#! 5/,2#+/3(>$%C'D(#.(>,$1'+,(
7+E/E'*'+,(=5>>7?! ;/0$3,D(>$%C'D(#.(>,$1'+,(
7+E/E'*'+,(=;>>7?! )3$*+2(>$%C'D&
!"-+%$,#
*&#&./0%,$1
()#$%"&'!"#$%"&'
Balanced Academic Assessment Begins with Data Sources
!"#$%"&' ()#$%"&'
! "#$%&'()&&'&&*'+,&! -'%.#%*/+0'(
)&&'&&*'+,&
*+%$,#
! "%'1'+,2/3('4/*! 5/,2#+/3(6'+'%/3(718(
74/*&! 9/:#%(;2'31(<'&,&(=7<>?
*+%$,#
! @+AB#$&'(>/,2&./0,2#+(>$%C'D&(=/++$/33D?
! 6%/1$/,'(742,(>$%C'D
!"-+%$,#! 5/,2#+/3(>$%C'D(#.(>,$1'+,(
7+E/E'*'+,(=5>>7?! ;/0$3,D(>$%C'D(#.(>,$1'+,(
7+E/E'*'+,(=;>>7?! )3$*+2(>$%C'D&
!"-+%$,#
*&#&*&#&*&#&*&#&*&#&./0%,$1
()#$%"&'!"#$%"&'
!"!#"$%
$&
Balanced Academic General Education Data Sources
!"#$%"&' ()#$%"&'
! "#$%&'()*+,-./0! 1'/'%*-("#%'(23*+! 44"5(6$7%.8&
*+%$,#
! (-./!"#$%&'(0 12#,34$5/655$554$"#/.2+#$
! "#--'0.*9'(:'*%/./0(4&&'&&+'/9
! 2;)(<%#=.8.'/8>(<%#=.-'
*+%$,#
! ?/@A#$&'()*9.&=*89.#/()$%B'>&(C*//$*-->D
! 1%*E$*9'(23.9()$%B'>
!"7+%$,#! F*9.#/*-()$%B'>(#=()9$E'/9(
2/0*0'+'/9(CF))2D! G*8$-9>()$%B'>(#=()9$E'/9(
2/0*0'+'/9(CG))2D! 4-$+/.()$%B'>&
F*9.#/*-()$%B'>(#=()9$E'/9(
!"7+%$,#
*&#&*&#&*&#&*&#&*&#&.32%,$5
()#$%"&'!"#$%"&'
What are the data sources you use?
20
!"!#"$%
$$
IE Plans for Support Systems:
Support Services & Operations
21
!"#$%$&$%'"()*+,,-.$%/-"-##*
0)("
1%##%'"*2)%3"4-"$
5'()#
1-(#&6-#7$6($-3%-#*
8(94%"%#$6($%'":*#$&9-"$*#-6/%.-#:*6-#-(6.;<
2##-##4-"$*=/-6#%3;$
Balanced Operational Assessment Begins with Data Sources
!"#$%"&' ()#$%"&'! "#$%&$'()*+,$'-.! /'&%-*'+0-#.(.'-*1-! ,-'-*'()*! 2*#)334-*'! 5&%6-'
"#$%&$'()*+,$'-.
*+%$,#
! 7)8+03$1-4-*'! 5-6(**(*6+/$3$#(-.! 5-*194$#:(*6
*+%$,#
! /'&%-*'+/$'(.;$1'()*! <3&4*(+/&#=->.! "#$%&$'-+2?('+/&#=->
!"-+%$,#! "#$%&$'-+24@3)>-#+/&#=->.! <3&4*(+/&#=->.! 5-*194$#:! A//2+$*%+B//2
!"-+%$,#
*&#&*&#&*&#&*&#&*&#&*&#&./0%,$1
()#$%"&'!"#$%"&'
!"!#"$%
$!
What Comprises the Plan? 23
!"#$%&'(&)(*+,' -".#/%0$%&'-"0$1(2%..%&' 2%..%&'(,'3(*4/0&."(!$,$"5"'$.
6&,+.789:"#$%;". <'&=+"3>"?(!@%++.?(A"B,;%&/.(6/,34,$".(=%++(3"5&'.$/,$"(&'(>/,34,$%&'1
*/&>/,5(,'3(!$43"'$(C",/'%'>(84$#&5".(D!C8.E
*/&#"..(F G#$%;%$H(+";"+?("))%#%"'#H?(#&50+%,'#"I,/>"$(F JB,$(.$43"'$.(=%++(@'&=?(3&?(;,+4"?(9"+%";"(!,$%.),#$%&'(F !$43"'$(!,$%.),#$%&'(,)$"/(."/;%#"
G.."..5"'$(2"$B&3.
I&&+.?(3,$,(.&4/#".?(#4//"'$(5",.4/".(F G/"($B"($&&+.(0/&;%3%'>($B"(,00/&0/%,$"(%')&/5,$%&'K(8L(G/"(="(,.@%'>($B"(/%>B$(M4".$%&'.K
A"'#B5,/@. I,/>"$(,'3(!4##"..(&)(6&,+(G$$,%'5"'$(F N'$"/',+(,'3(OP$"/',+(A"'#B5,/@%'>
L".4+$. G#$4,+(L".4+$.?(N'3%#,$%&'(&)(Q%'3%'>.?(!400&/$%'>(";%3"'#"(DM4,+%$,$%;"E(&/(!$,$%.$%#.(DM4,'$%$,$%;"E(F 2""$(%'$"'3"3(&4$#&5"K
R."(&)(L".4+$. JB,$(#B,'>".(=%++(&##4/($&(%50/&;"(>&,+(#&50+"$%&'K
S"P$(TH#+"K JBH("P0"#$"3(&4$#&5"(=%++(9"(4."3(%'('"P$(H",/U.(0+,'
Review the Key Components for Assessment Management
Session 2.3
!"!#"$%
$&
!"##$%%$&#'&()#'*#+,&-*#'./%+0123
!"#$$%$&'()*"+Program/Unit
Develop Academic Assessment and IE Plans
Set up data reports
Committee Work:Submit for institutional review
Program/Unit:Implement plan
Collect data
Committee Work:Submit for ReportsAnalyze Reports
!"#$%&'()*+,-.(/0,(!1#&*#2-34(5!(6%
52'$(03(6 8'1'&0%-34 9..'%2#+&' !"'$%&#,:;0#&7(<(53-2-#2-1'7 =0(;0#&7(>-72'? ;0#&7(>-72'?@(
>-3A#4'(B3.&'#,;0#&7(>-72'?(<(
>-3A'?C-77-03 ;'3',#&(D2#2'$'32(
0/(-32'32@(&#.A7(7%'.-/-.7(
53.&*?'7(%*,%07'E(#&-43$'32(F-2G(
?'%#,2$'32@(7.0%'(&-$-2'?
H&'#,IH03.-7'5?'32-/-'7(%,04,#$(
-$%#.2E(72#A'G0&?',7(#3?(
#&-43$'32(20(.0&&'4'($-77-03(
J+K'.2-1'7 J+K'.2-1'7(302(-3($'#7*,#+&'(2',$7@(
30(2#,4'27@(30(/0.*7(03(72*?'32(
&'#,3-34
C072(0+K'.2-1'7(#,'($'#7*,#+&'@($072(G#1'(2#,4'27@(70$'(/0.*7(03(72*?'32(
&'#,3-34
9&&(0+K'.2-1'7(-3($'#7*,#+&'(2',$7E(/0.*7(03(&'#,3-34(
0*2.0$'7(F-2G(72#2'?(2#,4'27L
!"!#"$%
$&
!"#$%&'()*+,-.(/0,(!1#&*#2-34(5!(6ໍ
52'$(03(6 :'1'&0%-34 ;..'%2#+&' !"'$%&#,<!"%'.2'=(>*2.0$'7 ?03'(:'/-3'=@ :'/-3'=(+*2(302(
#&-43'=(A-2B(0+C'.2-1'D$-77-03
:'/-3'=E(#&-43'=E(#3=(7*%%0,2(
%,0.'77E(2#,4'2(0,(7#2-7/#.2-03
;77'77$'32(F'2B0=
?02(#&&(0*2.0$'7(B#1'($'#7*,'7@(
G0*,7'(4,#='7(*7'=(#7(#($'#7*,'@
53.&*='7(H($'#7*,'(%',(0*2.0$'(#3=(*7'7(=-,'.2(#3=(
-3=-,'.2($'#7*,'7@
F*&2-%&'(=-,'.2D-3=-,'.2(
$'#7*,'7(/0,(70$'(0,(#&&(0*2.0$'7I(6*,%07'/*&(8)'7*&27(/0,(5$%,01'$'32
!"#$%#&'"()#**&+("&,--"(.$/&(+&'/$0&$01&2$0$3#4#0%
!"#$"%&
!"#$"%&
!'%(
)*+,-
5#-$"%4#0%6!(//#3#
!(447%%##*,)$1#47)&$01&89--("%6:-#"$%7(0$/
8%$"%703&'(70%
;#%9"0&%(&'/$0&%(&'"(3"$4;#%9"0&;#*9/%*6'"()#**&+("&<($/&,%%$704#0%
!"!#"$%
$&
Assessmentand Data Reporting Cycle
!"#$%&'"()*'""'+,-).+$&"-)$,/)01#2+34"
5""4""34,#)6&$,,',7
839&434,#)#(4)6&$,)$,/).$#(4:);$#$
8,#4:9:4#)$,/)!<$&1$#4)#(4);$#$
*+/'=>)$,/)839:+<4
303030
!"#$%&#'()*'+),-&%.$'/0.01%&%.$'0.2'+0$0'30*%4)-#5.1
!"!#"$%
$&
Avoiding Common Mistakes that can Unravel our Plans and Wreck your Results
Session 2.4
Mistakes, Omissions, Additions…
Leadership not involvedLack of Getting People on the busLack of Faculty InvolvementLack of InfrastructureLack of Professional DevelopmentLack of Support Personnel and ResourcesDetermining Data Systems and Document Management
!"!#"$%
$#
!"#$%"#&'%()*%+,-.)&'/'+0
!
Thank You!"#$%"&#$'(()*(+"*
!"!#"$%
$&
Anatomy of Key Performance Indicators (KPIs)
Key Performance Indicators (KPIs) are elements of the plan that express what you want to achieve by when. May use 5-7 KPIs to manage and track the progress of their plan.
The anatomy of a structured KPI includes:A Measure – Every KPI must have a measure..A Target – Every KPI needs to have a target that matches your measure and the time period of your goal. A Data Source – Every KPI needs to have a clearly defined data source.Reporting Frequency – Different KPIs have different reporting needs; good rule to follow is to report at least monthly.
35
36