Evaluation Tools & Approaches for

Engaging Stakeholders:

The AREA ApproachPresentation at the Canadian Evaluation Society

Dr. Mariam Manley, Principal EvaluatorMid-Continent Research for Education &

LearningMmanley@mcrel.org

June 1, 2003

What is the Advanced Reading Enhancement Approach (AREA)?  AREA is a statewide program designed to help ensure that, by the third grade, all South Dakota students are reading and writing at grade level. The AREA project trains and mentors teachers over an entire school year and consists of the following elements:        45 hours of direct instruction to teachers      18-22 hours of administrator training      Ongoing mentoring & coaching       Provision of resources & materials      Student Assessment 

Stakeholder Involvement … Do I REALLY want to do this?!

External, third party Objectivity Validity Reliability Time Consensus (yikes!)

Friend/Foe Useful Meaningful Ownership Cost Effective Educational

Methods of Stakeholder Involvement

       Logic Model and evaluation design     Instrument development     Data collection     Design & management of data collection system     Feedback mechanisms     Reporting

DEVELOPMENT OF A PROGRAM LOGIC MODEL

  Resources/Inputs Activities Short-term Intermediate Long-term Impact    

External conditions influencing success       Understanding & “Big Picture”      Identification of outcomes      Links between activities and outcomes      Communication      Formative      Identify evaluation questions and sources of existing information      Reporting

What types of information were collected as part of the AREA evaluation?

       Documentation and ongoing monitoring of program implementation     Pre and post teacher & administrator surveys     Classroom observations     Student assessment data      Site visits & interviews   

 

INSTRUCTIONS FOR USE   What is the sequence of events in using these instruments? 1.      Arrange the observation session(s) with the teacher2.      Ask a pre-observation question (2 minutes or so)3.      Observe the instructional area to be focused upon4.      Conduct a post-observation interview (approximately 15 minutes)5.      Complete the observation form6.       Enter the rating form into the web-based data collection system