school evaluation services presentation to nga november 4, 2001
TRANSCRIPT
School Evaluation ServicesSchool Evaluation Services
Presentation toPresentation to
NGANGANovember 4, 2001November 4, 2001
School Evaluation ServicesSchool Evaluation Services
2
What Is School Evaluation Services?What Is School Evaluation Services?
Management and communication tool for administrators, Management and communication tool for administrators,
policymakers, parents and other stakeholderspolicymakers, parents and other stakeholders
Dynamic public website displays district reports, key data Dynamic public website displays district reports, key data
trends, and comparisons for each district and/or schooltrends, and comparisons for each district and/or school
Annual independent S&P written analysis of strengths and Annual independent S&P written analysis of strengths and
challengeschallenges
Annual Annual Statewide InsightsStatewide Insights report focused on state trends, report focused on state trends,
data quality, and progressdata quality, and progress
School Evaluation ServicesSchool Evaluation Services
3
“Because SES doesn't rely on rankings or ratings and instead provides a rich new data resource, we have an unprecedented opportunity to assist the 34 districts in our region.”
- Superintendent Marlene Davis Wayne RESA
Capital Journal, May 25, 2001
School Evaluation ServicesSchool Evaluation Services
4
How Is Information Organized?How Is Information Organized?Hundreds of Data ItemsHundreds of Data Items
Student Results
Spending
Return on Resources
Learning Environment
Financial Environment
Demographic Environment
What are the academic results?
Where is the money spent?
What is the comparative return?
What is the scholastic context?
What is the financial context?
What is the socioeconomic context?
School Evaluation ServicesSchool Evaluation Services
5
“This is helping to clarify the unique challenges faced by urban school districts.”
- CEO Kenneth BurnleyDetroit Public SchoolsDetroit Free Press, May 25, 2001
“It's the most comprehensive evaluation of schools that I've seen.”
- Superintendent Craig MisnerKalamazoo RESA
Kalamazoo Gazette, May 24, 2001
School Evaluation ServicesSchool Evaluation Services
6
Unique Analytical FrameworkUnique Analytical Frameworkwww.ses.standardandpoors.comwww.ses.standardandpoors.com
Not only data, but written analysisNot only data, but written analysis
““Return on Resources”—simultaneous examination of Return on Resources”—simultaneous examination of
spending, results, and contextspending, results, and context
Each district compared to four benchmarks: state, regional, Each district compared to four benchmarks: state, regional,
and socioeconomic peer averages; and district over timeand socioeconomic peer averages; and district over time
Multi-year trends measure progressMulti-year trends measure progress
Website allows users to customize comparison groupsWebsite allows users to customize comparison groups
School Evaluation ServicesSchool Evaluation Services
7
“The real power of this report will be to allow school districts to compare themselves to other similar school districts and look for exceptions. That would be very helpful to discover if there are deficiencies.”
- Superintendent David DresslarJenison School District
The Grand Rapids Press, May 25, 2001
School Evaluation ServicesSchool Evaluation Services
8
Statewide InsightsStatewide InsightsMichigan Findings
MEAP data analysis reveals success and MEAP data analysis reveals success and challenges in educational achievement but also challenges in educational achievement but also raises questions about state’s accountability raises questions about state’s accountability program.program.
- Overall pass rates are lowOverall pass rates are low- Not a “value added” systemNot a “value added” system- Snapshot versus continuumSnapshot versus continuum- Timing is problematicTiming is problematic
School Evaluation ServicesSchool Evaluation Services
9
Statewide InsightsStatewide InsightsMichigan Findings
Significant Achievement Gaps exist across the Significant Achievement Gaps exist across the state, in school districts and in school buildingsstate, in school districts and in school buildings
- Many of these gaps are more pervasive than Many of these gaps are more pervasive than places that research says should places that research says should “be expected“be expected””- Gaps in achievement may be significantly Gaps in achievement may be significantly understatedunderstated
School Evaluation ServicesSchool Evaluation Services
10
Statewide InsightsStatewide InsightsMichigan Findings
Many schools and districts are “beating the odds”Many schools and districts are “beating the odds”
School Evaluation ServicesSchool Evaluation Services
11
Statewide InsightsStatewide InsightsMichigan Findings
Major indicators of student achievement are Major indicators of student achievement are misalignedmisaligned
- Graduation and dropout rates are problematic Graduation and dropout rates are problematic due to measurement methodologydue to measurement methodology- 57% MEAP HS pass rate versus 81% graduation 57% MEAP HS pass rate versus 81% graduation raterate- e.g. 29,000 HS graduates did not pass the e.g. 29,000 HS graduates did not pass the reading testreading test- MEAP vs. NAEP 4MEAP vs. NAEP 4thth grade grade
School Evaluation ServicesSchool Evaluation Services
12
Statewide InsightsStatewide InsightsMichigan Findings
When looking at MEAP and other student results When looking at MEAP and other student results indicators, how money is spent and the context in indicators, how money is spent and the context in which it is spent is at least as important as the which it is spent is at least as important as the “amount” of money spent.“amount” of money spent.
Through examination of numerous “return Through examination of numerous “return indicators” (PCIs), it appears that increased indicators” (PCIs), it appears that increased spending alone is not likely to accomplish the goal spending alone is not likely to accomplish the goal of having all students meet state standards. The of having all students meet state standards. The return on resources must be improved.return on resources must be improved.
School Evaluation ServicesSchool Evaluation Services
13
No silver bulletsNo silver bullets
Need to align policy making with data analysisNeed to align policy making with data analysis
Without improved “return”, goals of improvement of student Without improved “return”, goals of improvement of student
achievement overall and of closing achievement gaps are achievement overall and of closing achievement gaps are
probably unaffordableprobably unaffordable
Districts and schools that are “beating the odds” are potential Districts and schools that are “beating the odds” are potential
sources of best practicesources of best practice
State’s accountability system needs to improveState’s accountability system needs to improve
What Do the Data Tell Us?
School Evaluation ServicesSchool Evaluation Services
14
What Influence Can the Data Have?What Influence Can the Data Have? Increased parent involvementIncreased parent involvement
Increased communication between schools and communityIncreased communication between schools and community
Enhanced ability to identify needs in districtsEnhanced ability to identify needs in districts
Greater accuracy of data collection and reportingGreater accuracy of data collection and reporting
More complete examination of prior assumptions about More complete examination of prior assumptions about
accreditation and resource allocation at the state levelaccreditation and resource allocation at the state level
School Evaluation ServicesSchool Evaluation Services
15
Why States Need a Third Party?Why States Need a Third Party?
Perceived political biases interfere with or “limit” analysisPerceived political biases interfere with or “limit” analysis
Economies of scale— no state alone would be willing to Economies of scale— no state alone would be willing to
expend amount necessary to have a comprehensive systemexpend amount necessary to have a comprehensive system
Technological Advantages—sophisticated technology tools Technological Advantages—sophisticated technology tools
and content are constantly and rapidly being improved and content are constantly and rapidly being improved
without requests for unplanned appropriations without requests for unplanned appropriations
Standard Analytical Framework—will allow comparisons Standard Analytical Framework—will allow comparisons
between states and across state lines for individual districtsbetween states and across state lines for individual districts
- Michigan and Pennsylvania are the first states to lead the - Michigan and Pennsylvania are the first states to lead the
nation by participating in a new standardnation by participating in a new standard
School Evaluation ServicesSchool Evaluation Services
16
Why Standard & Poor’s?Why Standard & Poor’s?
SES based on more than 5 years of research & SES based on more than 5 years of research &
developmentdevelopment
Reputation for objectivity and analytical rigorReputation for objectivity and analytical rigor
Vast experience in public sector analysisVast experience in public sector analysis
Global provider of analytical benchmarksGlobal provider of analytical benchmarks
Experienced staff and advisors, including Experienced staff and advisors, including
educators and administratorseducators and administrators
School Evaluation ServicesSchool Evaluation Services
17
“Standard & Poor's School Evaluation Services is an excellent example of providing parents and taxpayers with a more complete picture of their school's performance…The S&P system has overwhelming support…”
- Bob Chase, President National Education Association
Letter to the Editor, Wall Street Journal, July 31, 2001
School Evaluation ServicesSchool Evaluation Services
18
SES National Board of AdvisorsSES National Board of Advisors Former Governor Jim Hunt,Former Governor Jim Hunt, Chair of SES Advisory Board Chair of SES Advisory Board
Christopher Cross,Christopher Cross, President and CEO of the Council for President and CEO of the Council for
Basic EducationBasic Education
Dr. Susan Fuhrman,Dr. Susan Fuhrman, Dean of the Graduate School of Dean of the Graduate School of
Education at the University of PennsylvaniaEducation at the University of Pennsylvania
Dr. Jim Kelly,Dr. Jim Kelly, Former President and CEO of the National Former President and CEO of the National
Board for Professional Teaching StandardsBoard for Professional Teaching Standards
Dr. Ted Sanders,Dr. Ted Sanders, President of the Education Commission President of the Education Commission
of the States of the States
School Evaluation ServicesSchool Evaluation Services
19
“When you involve people in the process, you lessen resistance and maximize benefit. All of us should learn from their efforts.”
-Tom Watkins
Michigan Superintendent of Public InstructionMichigan Department of Education News Release, May 23, 2001
School Evaluation ServicesSchool Evaluation Services
20
Understanding and Applying the DataUnderstanding and Applying the Data
In many districts and states, there are not sufficient In many districts and states, there are not sufficient
resources and staff to conduct thorough analysis and resources and staff to conduct thorough analysis and
benchmarking, SES fills that void benchmarking, SES fills that void
Helps districts and policymakers avoid being “data rich Helps districts and policymakers avoid being “data rich
but information poor”but information poor”
Helps end policymakers’ reliance on anecdotal Helps end policymakers’ reliance on anecdotal
generalizationsgeneralizations
School Evaluation ServicesSchool Evaluation Services
21
What Influence Does the Data Have?What Influence Does the Data Have?
Increased parent involvementIncreased parent involvement
Increased communication between schools and Increased communication between schools and
communitycommunity
Increased ability to identify needs in districtsIncreased ability to identify needs in districts
More accurate data collection and reportingMore accurate data collection and reporting
More complete examination of prior assumptions about More complete examination of prior assumptions about
accreditation and resource allocation at the state levelaccreditation and resource allocation at the state level
School Evaluation ServicesSchool Evaluation Services
22
“For parents it’s a godsend…relevant data about how schools are performing…For the first time a nonpartisan company with considerable analytical skills and no connection to the education establishment has undertaken an objective analysis of school systems.”
-Pete duPont Former Governor of Delaware
Wall Street Journal Opinion Journal, October 17, 2001
School Evaluation ServicesSchool Evaluation Services
23
“SES is beginning to change the balance of power in America’s public-school systems, among our most insular bureaucratic structures. SES information will change the way parents perceive their schools, and that will change the way schools work, and then one day, that will change the way the world works.”
-Pete duPont Former Governor of Delaware
Wall Street Journal Opinion Journal, October 17, 2001
School Evaluation ServicesSchool Evaluation Services
24
How Does SES Assist with ESEA?How Does SES Assist with ESEA?
Increased accountability through transparencyIncreased accountability through transparency
Increased disclosure of disaggregated dataIncreased disclosure of disaggregated data
Measures of annual progress and longer term trend Measures of annual progress and longer term trend
analysesanalyses
Measures to understand and therefore help maximize Measures to understand and therefore help maximize
return on federal and state dollarsreturn on federal and state dollars
Ability to rapidly adapt and include multiple measures of Ability to rapidly adapt and include multiple measures of
performanceperformance
School Evaluation ServicesSchool Evaluation Services
25
How Will Comparability Be Achieved?How Will Comparability Be Achieved?
Use NAEP as a benchmark for state test Use NAEP as a benchmark for state test
comparisonscomparisons
Other student results measures such as SAT, Other student results measures such as SAT,
ACT and AP scores ACT and AP scores andand participation rates participation rates
Basic financial data are comparable across Basic financial data are comparable across
most states and therefore, so too are braod most states and therefore, so too are braod
measures of “return”measures of “return”
School Evaluation ServicesSchool Evaluation Services
26
"I've already heard from several superintendents who shared with me their plans to review the reports, to confer with their colleagues from districts with similar characteristics and challenges, and to identify best practices that may be replicated in their own districts. I'm happy the service is turning out to be so helpful to local educators."
-Mike Flanagan, Executive Director Michigan Association of School Administrators
Capital Journal, May 25, 2001
School Evaluation ServicesSchool Evaluation Services
27
Where To Find SESWhere To Find SES
SES website URL:SES website URL:
www.ses.standardandpoors.comwww.ses.standardandpoors.com
SES e-mail address:SES e-mail address:
[email protected]@standardandpoors.com
School Evaluation ServicesSchool Evaluation Services
28
“It’s good to have them [Standard & Poor’s] doing it. I’m very excited about it. I regret that I won’t be in a position to do something about these conclusions over the next 15 months, over the next couple of years, but I think Pennsylvania ought to take them to heart.”
-Tom RidgeFormer Governor of PennsylvaniaThe Harrisburg Patriot News, October 5, 2001
School Evaluation ServicesSchool Evaluation Services
30
Elementary Economically Disadvantaged vs. State Test % Passing
0
10
20
30
40
50
60
70
80
90
100
0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100
Students Eligible for Free or Reduced Lunch ($)
Michigan Elementary Schools
MEAP
Passing
(%)
Economically Disadvantaged Students (%)
School Evaluation ServicesSchool Evaluation Services
31
Operating Expenditures vs. State Test Passing
20
25
30
35
40
45
50
55
60
65
70
75
80
85
90
5,000 5,500 6,000 6,500 7,000 7,500 8,000 8,500 9,000 9,500 10,000 10,500 11,000 11,500
Operating Expenditures Per Student ($)
MEAP
Passing
(%)
Operating Spending Per Student ($)
Michigan K-12 Districts
School Evaluation ServicesSchool Evaluation Services
32
Michigan School Districts - 1999
0
10
20
30
40
50
60
70
80
90
100
0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100
Enrollment of Disadvantaged Students (%)
ME
AP
Co
mp
osi
te P
ass
Rat
e (%
)
37 Districts139 Districts
60 Districts 64 Districts
30.5% (avg.)
54.7%(avg.)
Sites with MEAP Participation > 84.2%
MEAP Composite Pass Rates and Proportion of Disadvantaged Students
BEATING THE ODDS 3 YEARS RUNNING
School Evaluation ServicesSchool Evaluation Services
33
Mike Addonizzio,Mike Addonizzio, Associate Professor, College of Education, Wayne State Associate Professor, College of Education, Wayne State
UniversityUniversity
Chuck AndersonChuck Anderson, Executive Director of the Michigan Education Association, Executive Director of the Michigan Education Association
Madhu AndersonMadhu Anderson, Director of the Center Education Performance and , Director of the Center Education Performance and
InformationInformation
Mike FlanaganMike Flanagan, Executive Director of the Michigan Association of School , Executive Director of the Michigan Association of School
AdministratorsAdministrators
Justin KingJustin King, Executive Director of the Michigan Association of School Boards, Executive Director of the Michigan Association of School Boards
SES Michigan Advisory GroupSES Michigan Advisory Group
School Evaluation ServicesSchool Evaluation Services
34
Doug RobertsDoug Roberts, State Treasurer, Michigan Department of Treasury, State Treasurer, Michigan Department of Treasury
Dan QuisenberryDan Quisenberry, President of the Michigan Association of Public School , President of the Michigan Association of Public School
AcademiesAcademies
Jim SandyJim Sandy, Director for the Michigan Business Leaders for Education , Director for the Michigan Business Leaders for Education
ExcellenceExcellence
Ray TelmanRay Telman, Executive Director of the Middle Cities Education Association, Executive Director of the Middle Cities Education Association
Tom WatkinsTom Watkins, Superintendent of Public Instruction for the Michigan , Superintendent of Public Instruction for the Michigan
Department of EducationDepartment of Education
Tom WhiteTom White, Executive Director of the Michigan School Business Officials, Executive Director of the Michigan School Business Officials
SES Michigan Advisory Group (cont.)SES Michigan Advisory Group (cont.)
School Evaluation ServicesSchool Evaluation Services
35
SES Complements Existing State SES Complements Existing State Accountability ProgramsAccountability Programs
Michigan and Pennsylvania have extensive school Michigan and Pennsylvania have extensive school
information systems and have supplemented them with SESinformation systems and have supplemented them with SES
SES assembles data from fragmented sources into a single SES assembles data from fragmented sources into a single
databasedatabase
SES enhances state report cards by providing additional SES enhances state report cards by providing additional
data, trends, benchmarks and comparisons. data, trends, benchmarks and comparisons.