OECD Review on Evaluation and Assessment Frameworks for Improving School Outcomes

School Evaluation in the Flemish Community of Belgium

Brussels, 7 December 2011

Claire ShewbridgeAnalyst, Education and Training Policy Divisionclaire.shewbridge@oecd.org

Structure of the presentation

• The OECD review in general and the OECD review of school evaluation in the Flemish Community

• Policy trends in the Flemish Community and the analytical approach used by the OECD review team

• The OECD review team’s assessment of strengths and challenges in the current approach to school evaluation

• The OECD review team’s policy recommendations to build on the current approach to school evaluation in the Flemish Community

• Further information and OECD review outputs

THE REVIEW IN GENERAL AND IN THE FLEMISH COMMUNITY

The OECD review of School Evaluation in the

Flemish Community of Belgium

OECD reviews of evaluation and assessment: School evaluation in the Flemish Community of Belgium (1)

• The Flemish Community is one of 12 systems being reviewed– 12 other systems providing analytical reports, i.e. total 24 systems– Presenting evidence and including stakeholder views

Country review strand

OECD review visit Country report

OECDReport

Australia June 2010 Y Y

Belgium (Fl.) January 2011 Y Y

Chile November 2011

Czech Republic March 2011 Y

Denmark October 2010 Y Y

Luxembourg June 2010

Mexico First quarter 2012

New Zealand August 2010 Y

Norway December 2010 Y Y

Portugal February 2011

Slovak Republic First quarter 2012

Sweden May 2010 Y Y

Analytical strand

Country Report

Austria

Belgium (Fr.) Y

Canada

Finland

France

Hungary

Iceland

Ireland

Korea

Netherlands

Poland

Slovenia Y

OECD reviews of evaluation and assessment: School evaluation in the Flemish Community of Belgium (2)

• Focus of international review is on how to best use evaluation and assessment to improve school outcomes– Designing a coherent framework for student assessment, teacher

appraisal, school evaluation and system evaluation– Paying attention to use of results, evaluation capacity, procedures and

implementation

• Background report developed by University of Antwerp Edubron Research Group and the Ministry of Education and Training

• 6 days of review, including interviews of all major stakeholders and visits to Brussels, Antwerp, Vilvoorde and Sint-Niklaas– Seeking a broad cross-section of information and opinions on school

evaluation policies and how to improve these• OECD review team = Claire Shewbridge and Deborah Nusche (OECD),

Marian Hulshof (Netherlands) and Louise Stoll (UK/Canada)

POLICY TRENDS IN THE FLEMISH COMMUNITY AND OECD’S ANALYTICAL APPROACH

The OECD review of School Evaluation in the

Flemish Community of Belgium

Major policy developments influencing school evaluation in the Flemish Community

• Inspection focus on “output”• Differentiated inspection to target schools/areas with

greater quality concerns (2009)• Publication of school inspection reports (2004, 2007)• Schools legally responsible for their quality (2009)• Consultation platforms within the Policy on Equal

Educational Opportunities (GOK) (2002)

Increased focus on school

quality and reducing inequities

• GOK funding evaluation requirements (2002)• School innovation projects• Feedback to schools from National assessment• Participation Decree (2004) ensures stakeholder voice• Promotion of “school communities”• Inspection judges school “policy-making capacity”

Stimulating school self-evaluation

Analytical framework for the review of school evaluation in the Flemish Community

Governance: a framework for school evaluation

Effective evaluation procedures

Competencies for evaluation /

feedback

Use of evaluation

results

STRENGTHS AND CHALLENGES

The OECD’s assessment of the current approach

to school evaluation

Governance: a framework for school evaluation

Effective evaluation procedures

Use of evaluation

results

Competencies for evaluation/

feedback

• Schools have the major responsibility for the school improvement process• The Ministry fosters a degree of common understanding of basic quality • An increasingly information rich environment for school evaluation• Availability of robust student assessment tools for primary schools• Increased focus on the importance of engaging all stakeholders in school

evaluation• The Ministry does not mandate or steer school self-evaluation• School evaluation is not well embedded in a larger vision for evaluation

and assessment• Miss potential synergies between different evaluation approaches• Insufficient emphasis on improvement/excellence in attainment targets• High degree of variation in school policy-making capacity• Lack of information flow impedes school evaluation efforts

Governance: a frameworkfor school evaluation

• Considerable use of technology to support self-evaluation• The Context-Input-Process-Output (CIPO) inspection framework is

empirically grounded and comprehensive• Collection of evidence during the inspection can stimulate school self-

evaluation activities• There is a legal basis for inspection, including the examination of

school policy-making capacity• Lack of clarity around purpose of school self-evaluation and minimum

quality• Inadequate information base for risk assessment prior to inspections• Communicating school understanding and uptake of the new

inspection methodology• Judging schools’ implementation of attainment targets

Effective evaluationprocedures

• Recognition of the importance of a school’s policy-making capacity• Network support to promote and develop school self-evaluation

capacity• Identified teacher competencies to support school evaluation• Emerging collegial relations within and between schools to support

competency development• Efforts to improve the capacity of the Inspectorate to conduct coherent

inspections• Variation among school leaders in policy-making capacity• A need to strengthen educators’ evaluation literacy• Teachers’ orientation to research and enquiry• Clarifying and making more uniform inspectors’ judgements on quality

Competencies for evaluation / feedback

• Students have a voice in school policy, including growing involvement of students in self-evaluation

• Good balance of school responsibility and external pressure to use inspection results for improvement

• Publication of inspection reports• Examples of schools using feedback from national and international

assessments in their self-evaluation activities• Examples of primary schools using network test results as part of

whole-school evaluation• Lack of strategic and consistent use of self-evaluation• There is room for improvement in the public use of inspection results• Schools only receive Inspectorate profile information when being

inspected• School self-evaluation results not necessarily shared with inspectors

Use of evaluationresults

Some challenges to implementing school evaluation for continuous improvement

Lack of objective information to assess “output” prior to inspection

Varying levels ofself-evaluation capacity among schools.Lack of strategic use of evaluation results for school improvement.

Focus on minimum only. Hard to judge how schools implement them.

Continuous

improvement

Flemish learning

objectives

Inspectionfocus onoutput /

differentiation

Schoolsresponsiblefor quality

！

！ ！

POLICY RECOMMENDATIONS

The OECD’s assessment of how to build on

the current approach to school evaluation

Suggested priorities in going forward

Governance: a framework for

school evaluation

Competencies for evaluation/

feedback

Use of evaluation

results

Effective evaluation procedures

Clarify the goals of school evaluation and how different types of evaluation fit together

Continue to invest in school leader and

teacher capacity to conduct evaluation

and use its results for improvement

Increase the objectivity of evaluation procedures and ensure they promote improvement and excellence

Increase the use of available information

(collected at school and central level) in

both internal and external

school evaluation

• Further clarify common goals and expectations with a view to encouraging excellence and continuous improvement

• Strengthen consistency and coherence of different elements of school evaluation– Articulate appropriately with teacher appraisal– Reinforce links with school leadership appraisal– Better integration between self-evaluation and inspection

• Promote the use of evaluation and assessment tools by schools for improvement– Comparative evaluation of externally developed self-evaluation tools for

schools– Promote examples of effective practice

• Continue to embed self-evaluation requirements in new policies/programs– Schools should link this to school quality assurance/improvement plan– Provide guidelines on how to map this to CIPO inspection framework

Governance: a frameworkfor school evaluation

• Evaluate potential to develop set of criteria for learning progressions– Reference to assess student progress in different subjects

• Clarify the criteria for inspection judgements on the quality of education– Define clear criteria for all components used to determine quality– School self-evaluation capacity is a core component of its quality

• Go further in improving the inter-rater reliability of inspection reports– Concrete rating scale; build on inspection of school groups

• Strengthen the commitment of inspection and schools to the implementation of Flemish attainment targets

• Extend collegial practice both within and among schools– Further develop and coordinate emerging critical friendship

• Promote further involvement of students in self-evaluation activities– Critical feedback for learning, teaching and school improvement

Effective evaluationprocedures

• Further strengthen professional development for effective school self-evaluation– Prioritise support to building school policy-making capacity– Conduct external review of PBD tools and services

• Recognise and strengthen key role of school leaders in self-evaluation– Flemish framework for leadership competencies– Develop new leadership roles– Refine leadership training– Ministry project on school evaluation competency development

• Increase teacher understanding of policy-making capacity– Strengthen links between teaching quality and self-evaluation– Build competencies in data use, research and innovation– Fund collaborative teacher research projects with cross-school peer

review

Competencies for evaluation / feedback

• Strengthen information flow on key indicators from and to schools– Provide schools with access to information in Data Warehouse– Schools to provide performance information to the Inspectorate– Improve availability and use of objective output measures

• Ensure regular feedback to schools on key CIPO inspection framework indicators– Help schools with clear goals and measurable objectives– Work with schools to build shared language of quality/ evaluation

• Promote the availability and use of appropriate self-evaluation resources– Comparative overview of tools available to schools– Assess school demand for other self-evaluation tools

• Devise ways to improve public use of inspection results– Simplified language; dedicated website; reports for school groups

Use of evaluationresults

Ensure the quality of teaching and learning is at the heart of school evaluation

• Clarify the goals of school evaluation and how different types of evaluation fit together

• Continue to invest in school leader and teacher capacity to conduct evaluation and use its results for improvement

• Increase the use of available information in both internal and external school evaluation

• Increase the objectivity of evaluation procedures and ensure they promote improvement and excellence

ImprovementTeacher

appraisal /development

Self-evaluation / inspection

Support to schools (PBD)Leadership /evaluation competenciesCollaboration

Learning progressions

Inspection criteriaTools for

improvement

School access to central dataSchool performance data to Inspectorate

Quality of teaching

and learning

Outputs available on the Review’s website:www.oecd.org/edu/evaluationpolicy

OECD Reviews of Evaluation and Assessment in Education:

AustraliaDenmarkNorwaySweden

Country Background Reports: 9 out of 24 available

Literature reviews /conceptual papers including:

formative and summative assessment; teacher evaluation; school evaluation; education standards; use of results...