Missouri  Associa+on  of  School  Administrators  Show-­‐Me  Accredita+on  and  Assessment  Task  Force  

July  26,  2015  Columbia,  Missouri  

Fred  with  a  few  reminders…  

Why  do  we  assess?  How  What  

Why  

Read,  Reflect,  Respond  

WHY  do  we  assess  student  learning?  •  What  role  does  summa+ve  assessment  play  in  the  learning  process?  

•  What  role  does  forma+ve  assessment  play  in  the  learning  process?  

•  In  the  ideal  world,  how  would  you  link  forma+ve  and  summa+ve  assessment?  

•  What  type(s)  of  summa+ve  assessment  best  support  your  ideal  world?  

On  our  assessment  journey  remember…  

….children  are  our  FIRST  and  most  important  AUDIENCE.  

Assessment  should  support  a  child’s  growth  in  understanding  important  ideas  and  concepts.  

 

Some+mes  the  work  is  hard,  but…  

…together  we  can  help  children  grow,  achieve  and  become…  

…college  and  career  ready.  

 SO…REMEMBER  OUR  FOCUS  

Task  Force  Goal  Goal:  Iden%fy  and  recommend  to  the  Missouri  Commissioner  of  Educa%on  a  plan  for  accredita%on  and  assessment  that  emphasizes:  •  local  control  •  con+nuous  improvement  •  individual  student  growth  with  con+nued  a[en+on  to  subgroup  

achievement  •  right  test,  right  +me  •  adaptability  (flexible  enough  to  meet  current  and  future  federal/

state  guidelines  •  clarity  of  purpose  (can  be  explained  by  a  third  grader  to  an  adult  

audience)  •  achieving  Top  10  state  status  one  student  at  a  +me  

BUILD  ON  OUR  STRONG  FOUNDATION  

RIGOROUS  STANDARDS  

RIGOROUS  ASSESSMENTS  

STATE  WIDE  COLLABORATION  

Proficiency-­‐Based  Learning  

Recommenda@on  6:  Develop  an  assessment  system,  aligned  to  Missouri  learning  standards,  for  primary  and  intermediate  grades/ages  that  supports  Proficiency-­‐Based  Learning.      Aspects  of  this  recommenda@on  that  require  significant  considera@on  include:    •  An  EOC-­‐like  assessment  system  based  on  testlets  (i.e.  small  tests)  that  build  toward  a  

grade-­‐level  specific  composite  score.  Not  Met  (Note:  ESEA  requires  assessing  depth  and  breadth  of  grade  level  content  standards  making  this  recommenda+on  difficult  to  achieve  in  grades  3-­‐8  without  a  waiver  or  reauthoriza+on  of  ESEA  that  gives  assessment  control  back  to  the  states.)  

•  There  is  a  maximum  of  two  administra+ons  of  each  testlet  per  student,  with  the  last  score  coun+ng  for  accountability.  Not  Met  (See  ESEA  Note  Above)    

•  Define  and  develop  an  adap@ve  assessment  system  for  English  Language  Arts  and  Mathema+cs  that  not  only  assesses,  but  serves  as  a  screener  to  determine  if  current  students  are  ready  for  first  testlet  and  to  determine  grade-­‐level  equivalent  for  new  students.  Not  Met  (See  ESEA  note  above)/Local  Op/on    

•  Classify  students  by  learning  level  and  those  learning  levels  must  have  a  grade-­‐level  equivalency.  Local  Op/on  

•  Treat  @me  as  a  variable  to  demonstrate  proficiency  with  an  assessment  design  that  supports  that  concept.  Local  Op/on  

 

Proficiency-­‐Based  Learning  

Recommenda@on  7:  Develop  an  assessment  repor%ng  system  that  supports  Proficiency-­‐Based  Learning.    As  a  part  of  assessment  redesign  and  development,  the  repor+ng  system  should  provide  for  the  following:    •  MOSIS  matures  so  that  it  not  only  collects  data  from  but  provides  

informa+on  for  school  districts.  Not  Met  •  Assessment  results  are  available  quickly,  system  is  user  friendly,  and  

helps  teachers  easily  improve  learning  through  specific  diagnos+c  results  at  a  minimum  of  the  standard  level.  In  Progress  

   

MASA’S  SHOW-­‐ME  ACCREDITATION  AND  ASSESSMENT  TASK  FORCE  Progress  Report  on  April  2015  Task  Force  Sugges+ons  to  DESE  

 

•  Less  is  more  for  15-­‐16  (e.g.  give  NAEP,  use  current  item  bank  to  develop  test,  etc  to  meet  ESEA  requirements).  MET  

 •  Spend  @me  on  a  long  term  solu@on.  IN  PROCESS    •  Maintain  ACT  as  long  it  is  not  used  for  Standards  1  and  2.  MET    •  Iden@fy  pilot  sites  to  emphasize  forma+ve/interim  for  federal  compliance  using  an  approach  

that  emphasizes  helping  every  child  achieve  in  their  learning.  IN  PROCESS    •  Local  control  is  good.  IN  PROCESS    •  Amend  the  federal  waiver  to  support  proficiency  based  learning  (e.g  New  Hampshire)  

 NOT  MET    

CTB,  Missouri’s  test  provider,  was  sold  to  Data  Recogni@on  Corpora@on  (DRC).  DRC  is  a  Minneapolis  based  company  working  in  13  states,  including  Pennsylvania  where  they  have  a  7  year  $201  million  contract  to  develop  gradua@on  exams,  a  model  curriculum,  a  teacher  training  program  and  monitor  student  achievement  in  10  subject  areas.    Sources:  Bumsted  and  Roche,  TribLive,  May  21,  2009  and  DRC  web  site.  

State  Assessment  2015-­‐2016  ELA/MATH  Grades  3-­‐8  

State  Summa@ve    Required  per  ESEA  (DRC  is  vendor)    

Statewide  Classroom  Diagnos/c  Tool  Assessment  Op@on  from  DRC  

Local  Forma@ve/Interim  Op@ons  Example:  eValuate  

Fixed  Form  On-­‐Line   Computer  Adap@ve    

Computer  Adap@ve  

Item  Review/Approval  Fall  2015  by  Missouri  Educators  

Purpose  Monitor  student  progress  and  growth  (by  standard)  

Purpose  Students  self-­‐monitor  and  set  goals  regarding  progress  toward  end  of  year  expecta+ons  on  state  standards    

Plaeorm  (same  as  2015)  eDIRECT  and  INSIGHT  

Student  as  1st  Audience  Unknown  

Student  as  1st  Audience  Yes  

Training  On-­‐line  for  grade  level  and  content  area  

Predic@ve  of  State  Summa@ve  No  

Predic@ve  of  State  Summa@ve  Yes    

Tes@ng  Window  April  4,  2016  May  27  2016  

Training  On-­‐line  for  grade  band  and  content  area  

Training  On-­‐site  and  on-­‐line  

Cut  Point  Valida@on  June/July  2016  

Content  Areas  Math/Reading/Wri+ng  3-­‐5  Math/ELA  6-­‐HS  

Content  Areas  Reading  and  Math  Grades  2-­‐11  

Student  Reports  September  1,  2016  

Student  Reports  Nov.  2,  2015  May  29,  2016  

Student  Reports  On-­‐going/monthly  

               ESEA  Requirement    –  assess  depth  and  breadth  of  grades  level  standards  Contractor:  Data  Recogni+on  Corpora+on  (purchased  CTB)    Note:  End  of  Course  Exams  and  Science  Grades  5  and  8  are  the  same  for  2015-­‐2016  as  2014-­‐2015.  

FEDERAL  REQUIREMENTS  

MISSOURI  REQUIREMENTS  IN  ADDITION  TO  FEDERAL  

LOCAL  CONTROL  PRACTICES  AND  POLICIES  

ANNUAL  ASSESSMENTS  Math/ELA  (Grades  3-­‐8)  Math/ELA  (once  in  H.S)    

ANNUAL  ASSESSMENTS  Science  (5,  8,  H.S)  Social  Studies  (H.S)  ACT  (H.S.)  SAT  (H.S)  ASVAB  (H.S)  Compass  (H.S)  AB  (H.S)  IB  (H.S)  TSA  (H.S)  Dual  Credit  (H.S)    

ASSESSMENT  Forma+ve  Interim  Benchmark  Summa+ve  Etc.    

GRADUATION  RATE   GRADUATION  RATE  (Note:  status  requirement  exceeds  gradua+on  rate  of  #1  state  in  country)  

GRADUATION  REQUIREMENTS  (e.g.  number  of  credits,  community  service  requirements,  etc.)  

POST  SECONDARY  PLACEMENT   POST  SECONDARY  PARTNERSHIPS  

ATTENDANCE   ATTENDANCE  PRACTICES  AND  EXPECTATIONS  

OTHER  

Waived  from  Federal  Compliance:  Annual  Measureable  Objec+ves  to  Adequate  Yearly  Progress  (AYP)  en  route  to  100%  proficiency  by  2014;  Interven+ons  for  Title  I  schools  failing  to  make  AYP  2  consecu+ve  years;  Highly  Qualified  Teacher  (HQT)  requirements;  En+tlement  transfer  restric+ons;  All  content  to  same  students:  Double  Tes+ng  (Alg  I  in  8th  Grade).  

   

….children  are  our  FIRST  and  most  important  AUDIENCE.  

WHO  IS  THE  AUDIENCE?  

Assessment  for  Learning  

USE  OUR  STRENGTHS  TO  CREATE  LOCAL  AND  STATE-­‐WIDE  SOLUTIONS  FOR  

ASSESSMENT  ACCREDITATION   STUDENT  SUCCESS  

Accredita/on  Subcommi@ee  Update  

Prepared  for  MASA  Show  Me  Accredita@on  and  Assessment  Task  Force  

July  26,  2015    

Dave  Buck  Wright  City  

Mak  Goodman  Educa@onPlus  

Jim  Masters  Monroe  City  

Sarah  Riss  Webster  Groves  

Prepared  &  Presented  by  

21  

Accredita/on  Subcommi@ee  Task  

•  Spearhead  preliminary  research  on  accredita+on.  – The  advance  work  will  allow  the  rest  of  the  task  force  to  hit  the  ground  running  with  accredita+on  once  assessment  work  is  complete.    

22  

Subcommi@ee  Ini/al  Ques/ons  of  Interest    

•  Is  there  some  sort  of  framework  to  apply  when  thinking  about  state  accredita+on  models?  –  Specifically,  we  are  interested  in  understanding  how  many  different  models  are  out  there.  

•  Can  we  describe  where  other  states  are  at  in  applying  their  model?  –  How  the  models  work,  or  are  applied,  in  different  seqngs  (rural,  urban,  suburban),  and  –  What  the  success  rate  is  by  state  (e.g.,  how  many  districts  are  in  each  range/ra+ng  type  by  

state).  

•  What  is  the  stated  aim  for  each  model  (i.e.,  why  are  the  states  doing  what  they  are  doing)?  –  Point  and  purpose  espoused  –  Mission  to  accredit  –  Mission  to  support  high  quality  schools  

•  What  other  resources  are  out  there  that  could/should  be  reviewed  to  shape  our  understanding  around  accredita+on  and  accountability?  

23  

Resources  Reviewed  Ques@ons  posed  to  Hanover:    

•  Is  there  some  sort  of  framework  to  apply  when  thinking  about  state  accredita+on  models?  

•  Specifically,  we  were  interested  in  understanding  how  many  different  models  are  out  there.  

 

Response  from  Hanover:    •  They  were  not  able  to  iden+fy  

a  framework  for  classifying  accredita+on/accountability  models.  

•  They  could  work  to  create  an  infographic  from  the  exis+ng  review  of  models  and/or  expand  the  review  to  include  more  models.  

24  

Educa/on  Commission  of  the  States  (ECS)  From  Accredita/on  to  Accountability  

Key  Takeaways  •  26  states  (52%)  have  some  type  of  accredita+on  program;  all  50  have  

some  form  of  accountability.    •  At  least  11  states  have  folded  the  accredita+on  process  into  their  

accountability  systems:  Colorado,  Iowa,  Kansas,  Mississippi,  Missouri,  Nebraska,  New  Mexico,  Texas,  Virginia,  West  Virginia  and  Wyoming.  –  22%  of  all  states  (11/50);  42%  of  the  states  that  have  some  type  of  

accredita+on  program  (11/26).    •  Six  states  have  discon+nued  the  use  of  state  accredita+on  processes  since  

1998:  Alabama,  Delaware,  Louisiana,  Maryland,  Rhode  Island  and  Washington.  

 •  Six  states—Alaska,  Connec@cut,  Idaho,  Michigan,  North  Dakota  and  Utah

—use  regional  or  na+onal  accredi+ng  agencies,  such  as  the  New  England  Associa+on  of  Colleges  and  Schools.  

Wixom,  M.A.  (June  2014).  States  moving  from  accredita%on  to  accountability.  Educa+on  Commission  of  the  States,  Denver,  CO.  Retrieved  from  h[p://www.ecs.org/clearinghouse/01/12/72/11272.pdf   25  

State  Accredita@on  (26,  Yes  =  Yellow;  24,  No  =  Blue)  

Alaska  Arkansas  Colorado  

Connec+cut  Idaho  Illinois  Indiana  Iowa  Kansas  Maine  

Michigan  Mississippi  Missouri  Montana  Nebraska  

New  Mexico  North  Carolina  North  Dakota  Oklahoma  

South  Carolina  South  Dakota  

Texas  Utah  

Virginia  West  Virginia  Wyoming  

States  that  accredit  (Yes)  

States  that    do  not  

accredit  (No)  

Alabama  Arizona  California  Delaware  Florida  Georgia  Hawaii  Kentucky  Louisiana  Maryland  

Massachuse[s  Minnesota  Nevada  

New  Hampshire  New  Jersey  New  York  Ohio  

Oregon  Pennsylvania  Rhode  Island  Tennessee  Vermont  

Washington  Wisconsin  

26  

State  Accredita@on  (26,  Yes  =  Yellow;  24,  No  =  Blue)  

Alaska  Arkansas  Colorado  

Connec+cut  Idaho  Illinois  Indiana  Iowa  Kansas  Maine  

Michigan  Mississippi  Missouri  Montana  Nebraska  

New  Mexico  North  Carolina  North  Dakota  Oklahoma  

South  Carolina  South  Dakota  

Texas  Utah  

Virginia  West  Virginia  Wyoming  

States  that  accredit  (Yes)  

States  that    do  not  

accredit  (No)  

Alabama  Arizona  California  Delaware  Florida  Georgia  Hawaii  Kentucky  Louisiana  Maryland  

Massachuse[s  Minnesota  Nevada  

New  Hampshire  New  Jersey  New  York  Ohio  

Oregon  Pennsylvania  Rhode  Island  Tennessee  Vermont  

Washington  Wisconsin  

=  states  with  accredita+on  process  folded  into  their  accountability  system  

27  

11  States  with  Accredita/on  Process  Folded  into  their  Accountability  Systems  

28  

Drivers  For  Whole  System  Reform  (Na/on  –  State  –  District  –  School)  

Accountability  

Collabora@ve  Work  

Capacity  building  

Technology  

Individual  teacher  and  leadership  quality  

Pedagogy  

Fragmented  Strategies  

‘Systemness’  

29  

Drivers  A  policy  and  associated  set  of  strategies  that  are  designed  to  

effect  posi/vely  ‘whole  system  reform.’  

Wrong  Drivers   Right  Drivers  One  that  all  evidence  points  to  the  fact  that  it  

does  not  have  a  posi/ve  effect.  One  that  the  evidence  confirms  it  does  have  the  

desired  effect.  

Accountability   Capacity  building  

Individual  teacher  and  leadership  quality  

Collabora@ve  work  

Technology   Pedagogy  

Fragmented  Strategies   ‘Systemness’  

Fullan,  M.  (2011).  Choosing  the  wrong  drivers  for  whole  system  reform.  Centre  for  Strategic  Educa+on.  Retrieved  from  h[p://edsource.org/wp-­‐content/uploads/Fullan-­‐Wrong-­‐Drivers1.pdf    

30  

Four  Criteria  to  Judge  Likely  Effec/veness  of  a  Driver  

Do  the  drivers…  1.  Foster  intrinsic  mo+va+on  of  teachers  and  

students  2.  Engage  educators  and  students  in  con+nuous  

improvement  of  instruc+on  and  learning  3.  Inspire  collec+ve  or  team  work  4.  Affect  all  teachers  and  students—100%  

31  

Rela/onship  between  Wrong  and  Right  Drivers  

Accountability  

Fragmented  Strategies  

Individual  teacher  and  leadership  

quality  

Technology  

Capacity  Building  

Collabora@ve  work  

Pedagogy  

‘Systemness’  

Wrong   Right  

32  

Clarifica+ons  from  Fullan  (2011)  on  the  ‘wrong’  drivers.  

 “The  four  ‘wrong  drivers’  are  not  forever  wrong.  They  are  just  badly  placed  as  lead  

drivers.  The  four  ‘right  drivers’  are  the  anchors  of  whole  

system  reform.  You  don’t  have  to  give  up  your  affinity  to  accountability,  individual  quality,  technology,  and  

favored  quality  components  of  the  reform  package.  Stated  another  way,  I  am  not  talking  about  presence  or  absence  or  even  sequence,  but  rather  

dominance.  

Dominance  is  another  word  for  saying  what  system  leaders  state  and  acknowledge  as  the  anointed,  explicitly  ar+culated  lead  drivers.  The  encouraging  news  is  that  the  judicious  use  of  the  four  right  drivers  ends  up  accomplishing  be[er  the  

goals  that  those  espousing  the  wrong  drivers  are  seeking.  And  it  does  so  in  a  fundamentally  more  powerful  and  sustainable  

ma[er”  (p.5).  

Revised  Rela/onship  between  Drivers  

Accountability  

Fragmented  Strategies  

Individual  teacher  and  leadership  

quality  

Technology  

Capacity  Building  

Collabora@ve  work  

Pedagogy  

‘Systemness’  

33  

“Based  on  the  associa+on’s  whole  child  approach,  ASCD  has  been  calling  for  more  meaningful  accountability  systems  that  promote  con/nuous  support  and  

improvement  and  align  with  the  broader  outcomes  we  collec/vely  want  for  our  

students.  In  par+cular,  such  systems  should  incorporate  a  variety  of  measures  that  more  fully  reflect  a  comprehensive  defini+on  of  student  success,  accurately  measure  student  learning,  and  systema+cally  track  educators’  efforts  to  

engage  and  support  learners”  (p.3).  

34  

Understanding  Accountability  in    Five  Real-­‐World  Examples  

(Tacoma,  WA,  California’s  CORE  districts,  New  Hampshire’s  PACE  districts,  Kentucky,  Alberta,  Canada)  

Ques@on   Finding  

What  measures  are  reliable  and  valid  for  use  as  accountability  metrics?  

No  shortage  of  reliable  and  valid  measures  that  can  be  used  for  school  accountability  purposes.  

At  the  local  level:  •  Which  measures  to  use,    •  What  combina+on  of  measures  

cons+tutes  an  appropriate  mix,    •  How  the  measures  should  be  weighted  

(if  they  are  weighted  at  all),  and    •  How  to  effec+vely  communicate  about  

the  progress  toward  mee+ng  them.  

There  is  no  right  answer;  instead,  each  district,  state  or  province  needs  to  collabora+vely  determine  what  is  best  for  its  students.  

Mellor,  M.  &  Griffith,  D.  (2015).  Mul%metric  accountability  systems:  A  next-­‐genera%on  vision  of  student  success.  ASCD:  Alexandria,  VA.  

35  

Next  Steps  •  Hanover  Research  is  preparing  a  report  similar  to  the  first  (top  ten  and  

border  states)  with  the  focus  being  on  the  states  that  have  folded  the  accredita+on  process  into  their  accountability  systems.  

 –  Four  states  were  completed  in  the  first  report—Iowa,  Kansas,  Nebraska,  and  

Virginia,  leaving  six  states  remaining  to  compile—Colorado,  Mississippi,  New  Mexico,  Texas,  West  Virginia  and  Wyoming.  

–  Expected  to  have  the  report  finished  by  the  end  of  July/start  of  August.  

•  Summarize  key  elements  of  the  ASCD  Mul+metric  Accountability  Systems    

–  Iden+fy  Missouri  districts  using  approaches  similar  to  those  illustrated  by  ASCD.  

–  Research  the  role  of  lead  and  lag  measures.  

•  Iden+fy  meaningful  ways  to  share  findings.  

36