Upload File

how to use ed360 and other data tools for year end evaluation · •decide what you want to know....

  • Home
  • Documents
  • How to Use Ed360 and Other Data Tools for Year End Evaluation · •Decide what you want to know. (PURPOSE) • Form your evaluation questions. (What do you want to know?) • Use
47
How to Use Ed360 and Other Data Tools for Year End Evaluation Presented by: Gina Fulton Module #11 May 27, 2020

Upload: others

Post on 28-Sep-2020

0 views

Category:

Documents


0 download

Report

TRANSCRIPT

Page 1: How to Use Ed360 and Other Data Tools for Year End Evaluation · •Decide what you want to know. (PURPOSE) • Form your evaluation questions. (What do you want to know?) • Use

How to Use Ed360 and Other Data Tools for Year End Evaluation

Presented by: Gina FultonModule #11

May 27, 2020

Page 2: How to Use Ed360 and Other Data Tools for Year End Evaluation · •Decide what you want to know. (PURPOSE) • Form your evaluation questions. (What do you want to know?) • Use

Learning Objectives Module #11

2

To review and understand evaluation models and frameworks

To conceptualize and gather the data needed for the evaluation

To discuss theory and questions related to program evaluations

To refresh everyone’s memory about the data available in Ed360 for supporting program evaluation

Page 3: How to Use Ed360 and Other Data Tools for Year End Evaluation · •Decide what you want to know. (PURPOSE) • Form your evaluation questions. (What do you want to know?) • Use

Program Evaluation Definition

“A common definition used to separate evaluation from research is that program evaluation is conducted for decision-making purposes, whereas research is intended to build our general understanding and knowledge of a particular topic and to inform practice.”

-Spaulding, Dean T.

Page 4: How to Use Ed360 and Other Data Tools for Year End Evaluation · •Decide what you want to know. (PURPOSE) • Form your evaluation questions. (What do you want to know?) • Use
Page 5: How to Use Ed360 and Other Data Tools for Year End Evaluation · •Decide what you want to know. (PURPOSE) • Form your evaluation questions. (What do you want to know?) • Use
Page 6: How to Use Ed360 and Other Data Tools for Year End Evaluation · •Decide what you want to know. (PURPOSE) • Form your evaluation questions. (What do you want to know?) • Use

https://www2.ed.gov/about/offices/list/oese/sst/evaluationmatters.pdf

https://www2.ed.gov/about/offices/list/oese/sst/evaluationmatters.pdf
Page 7: How to Use Ed360 and Other Data Tools for Year End Evaluation · •Decide what you want to know. (PURPOSE) • Form your evaluation questions. (What do you want to know?) • Use

https://www2.ed.gov/about/offices/list/oese/sst/evaluationmatters.pdf

https://www2.ed.gov/about/offices/list/oese/sst/evaluationmatters.pdf
Page 8: How to Use Ed360 and Other Data Tools for Year End Evaluation · •Decide what you want to know. (PURPOSE) • Form your evaluation questions. (What do you want to know?) • Use

Why Evaluate Your Program?

Let’s use the Title I Program for an example as you’re required to evaluate Title I annually.

Brainstorm and list reasons why you would evaluate your Title I program.

Besides it being the law…

Page 9: How to Use Ed360 and Other Data Tools for Year End Evaluation · •Decide what you want to know. (PURPOSE) • Form your evaluation questions. (What do you want to know?) • Use

Brainstorm Ideas--Responses

To research if Title I program is effective To determine if we’re meeting the needs of all

students To determine if our resources align to our needs To decide whether there are areas we can improve

upon To determine if some areas of the program are not

working

Page 10: How to Use Ed360 and Other Data Tools for Year End Evaluation · •Decide what you want to know. (PURPOSE) • Form your evaluation questions. (What do you want to know?) • Use

What other questions might you want to include in your program evaluation? Why did you choose these?

Page 11: How to Use Ed360 and Other Data Tools for Year End Evaluation · •Decide what you want to know. (PURPOSE) • Form your evaluation questions. (What do you want to know?) • Use

What questions would you ask?

GENERAL IDEAS FOR PROGRAM EVALUATION QUESTIONS:(1) Has the Title I program been effective?(2) What has worked well in the Title I program? (3) What has not worked well in the Title I program(4) How should the Title I program be refined?

The data are analyzed, and the results of these analyses are used as the source of evidence to determine the answers to the four questions.

Page 12: How to Use Ed360 and Other Data Tools for Year End Evaluation · •Decide what you want to know. (PURPOSE) • Form your evaluation questions. (What do you want to know?) • Use

What are the steps in an evaluation?

• Decide what you want to know. (PURPOSE) • Form your evaluation questions. (What do you want to

know?)• Use a good model, design or evaluation matrix. (Logic

model and program overview slides)• Determine what data you want collected. (Types and

methods)• Collect data throughout the year. (Action plan)• Analyze data. (frequency, % change, aligned to ?)

Page 13: How to Use Ed360 and Other Data Tools for Year End Evaluation · •Decide what you want to know. (PURPOSE) • Form your evaluation questions. (What do you want to know?) • Use

What are the steps in an evaluation?

• Make connections to what the data is saying through triangulation. (3 or more valid data points/sets)

• Write a summary of program effectiveness. (Narrative style, using a standard template—see Rockwood 5* example.)

• Share findings and recommendations.• Ask for feedback on evaluation. What can others

do to help support the necessary changes?

Page 14: How to Use Ed360 and Other Data Tools for Year End Evaluation · •Decide what you want to know. (PURPOSE) • Form your evaluation questions. (What do you want to know?) • Use

Sample Evaluation Matrix

Evaluation Objective

Stakeholder Group

Tools Used to Collect Data

When Purpose

To document depth and breadth of activities that support theTitle I program during the 2019-2020 school year.

Certified and non-certified staff members, administrators, any providing services to Title I students.

InterviewSurvey

October during Parent/Teacher Conference weekOr throughoutthe year at different times.

Summative

Page 15: How to Use Ed360 and Other Data Tools for Year End Evaluation · •Decide what you want to know. (PURPOSE) • Form your evaluation questions. (What do you want to know?) • Use

Questions?15

Page 16: How to Use Ed360 and Other Data Tools for Year End Evaluation · •Decide what you want to know. (PURPOSE) • Form your evaluation questions. (What do you want to know?) • Use

Pause & Ponder16

Please write down your reflections.What ideas stuck with you?What aspects do you disagree with?What aspects do you want to ponder further?What resonates?How does this perspective differ from previous knowledge?

Page 17: How to Use Ed360 and Other Data Tools for Year End Evaluation · •Decide what you want to know. (PURPOSE) • Form your evaluation questions. (What do you want to know?) • Use

17

Demonstration of Ed360-Dashboard

Page 18: How to Use Ed360 and Other Data Tools for Year End Evaluation · •Decide what you want to know. (PURPOSE) • Form your evaluation questions. (What do you want to know?) • Use

18

Academic Overview, Grade 3, Low Income with All Races

Page 19: How to Use Ed360 and Other Data Tools for Year End Evaluation · •Decide what you want to know. (PURPOSE) • Form your evaluation questions. (What do you want to know?) • Use

19

Academic Overview, Grade 3, Students with Disabilities and All Races

Page 20: How to Use Ed360 and Other Data Tools for Year End Evaluation · •Decide what you want to know. (PURPOSE) • Form your evaluation questions. (What do you want to know?) • Use

20

Academic Overview, Grade 3, English Learners, All Races

Page 21: How to Use Ed360 and Other Data Tools for Year End Evaluation · •Decide what you want to know. (PURPOSE) • Form your evaluation questions. (What do you want to know?) • Use

21

KIDS Survey

Page 22: How to Use Ed360 and Other Data Tools for Year End Evaluation · •Decide what you want to know. (PURPOSE) • Form your evaluation questions. (What do you want to know?) • Use

22

KIDS Results by Readiness Developmental Areas Fall 2018 - 2019 with 2019-2020 School Overview

Page 23: How to Use Ed360 and Other Data Tools for Year End Evaluation · •Decide what you want to know. (PURPOSE) • Form your evaluation questions. (What do you want to know?) • Use

23

KIDS Readiness Measures by School

Page 24: How to Use Ed360 and Other Data Tools for Year End Evaluation · •Decide what you want to know. (PURPOSE) • Form your evaluation questions. (What do you want to know?) • Use

24

KIDS, Percentage of Students Demonstrating K Readiness-Male

Page 25: How to Use Ed360 and Other Data Tools for Year End Evaluation · •Decide what you want to know. (PURPOSE) • Form your evaluation questions. (What do you want to know?) • Use

25

KIDS, Percentage of Students Demonstrating K Readiness-Female

Page 26: How to Use Ed360 and Other Data Tools for Year End Evaluation · •Decide what you want to know. (PURPOSE) • Form your evaluation questions. (What do you want to know?) • Use

26

KIDS, Percentage of Students Demonstrating K Readiness, All Students, Hispanic or Latino

Page 27: How to Use Ed360 and Other Data Tools for Year End Evaluation · •Decide what you want to know. (PURPOSE) • Form your evaluation questions. (What do you want to know?) • Use

What data could your collect? Why?

Brainstorm activity: Share and discuss the types of data and the importance of the timeline in data collection.What data have you collected in your district for program measures?Once you collected data, what occurred? Why are the types important? Why is the timeline important?

Page 28: How to Use Ed360 and Other Data Tools for Year End Evaluation · •Decide what you want to know. (PURPOSE) • Form your evaluation questions. (What do you want to know?) • Use

Types of Data

Quantitative Qualitative Mixed Measures (both) Summative—collected at end of project to measure

outcomes and how those outcomes relate to the overall judgment of program (both quant. & qual.)

Formative—collected during project to measure outcomes and report back to stakeholders to be able to make timely adjustments to program

Page 29: How to Use Ed360 and Other Data Tools for Year End Evaluation · •Decide what you want to know. (PURPOSE) • Form your evaluation questions. (What do you want to know?) • Use

Tools for Collecting Data

Surveys Interviews Questionnaires Observations Document review Achievement data Student demographic data Student behavioral data

Page 30: How to Use Ed360 and Other Data Tools for Year End Evaluation · •Decide what you want to know. (PURPOSE) • Form your evaluation questions. (What do you want to know?) • Use

NAEP DATA

https://www.nationsreportcard.gov/profiles/stateprofile?chort=2&sub=MAT&sj=AL&sfj=NP&st=MN&year=2019R3

https://www.nationsreportcard.gov/profiles/stateprofile?chort=2&sub=MAT&sj=AL&sfj=NP&st=MN&year=2019R3
Page 31: How to Use Ed360 and Other Data Tools for Year End Evaluation · •Decide what you want to know. (PURPOSE) • Form your evaluation questions. (What do you want to know?) • Use

Data Analysis

Page 32: How to Use Ed360 and Other Data Tools for Year End Evaluation · •Decide what you want to know. (PURPOSE) • Form your evaluation questions. (What do you want to know?) • Use

Table 1-A Illinois Snapshot for NAEP Mathematics Grade 4 for 2000-2019

https://www.isbe.net/Documents/naep-2019-math-state-4-8.pdf

https://www.isbe.net/Documents/naep-2019-math-state-4-8.pdf
Page 33: How to Use Ed360 and Other Data Tools for Year End Evaluation · •Decide what you want to know. (PURPOSE) • Form your evaluation questions. (What do you want to know?) • Use

Triangulate Data

Be sure to include various data sets or points of collection

Check for outliers Make sure there isn’t bias or other factors that

skew your data Make sure you use evidence, grounded in facts—

not person feelings.

Page 34: How to Use Ed360 and Other Data Tools for Year End Evaluation · •Decide what you want to know. (PURPOSE) • Form your evaluation questions. (What do you want to know?) • Use

Quick reminders from the start of the DT Roadmap with online sessions

Page 35: How to Use Ed360 and Other Data Tools for Year End Evaluation · •Decide what you want to know. (PURPOSE) • Form your evaluation questions. (What do you want to know?) • Use

District Data Team• Meets monthly• Identifies District:

– Goals– Student Outcome Indicators – District Targets – Adult Action Indicators

• Develops District Strategic Improvement Plan

• Evaluates Programs

Page 36: How to Use Ed360 and Other Data Tools for Year End Evaluation · •Decide what you want to know. (PURPOSE) • Form your evaluation questions. (What do you want to know?) • Use

District Level Data

Page 37: How to Use Ed360 and Other Data Tools for Year End Evaluation · •Decide what you want to know. (PURPOSE) • Form your evaluation questions. (What do you want to know?) • Use

Data Analysis Protocol

Page 38: How to Use Ed360 and Other Data Tools for Year End Evaluation · •Decide what you want to know. (PURPOSE) • Form your evaluation questions. (What do you want to know?) • Use

School Level Data

Page 39: How to Use Ed360 and Other Data Tools for Year End Evaluation · •Decide what you want to know. (PURPOSE) • Form your evaluation questions. (What do you want to know?) • Use

Data Analysis Protocol

Page 40: How to Use Ed360 and Other Data Tools for Year End Evaluation · •Decide what you want to know. (PURPOSE) • Form your evaluation questions. (What do you want to know?) • Use

Classroom/Student Level Data

Page 41: How to Use Ed360 and Other Data Tools for Year End Evaluation · •Decide what you want to know. (PURPOSE) • Form your evaluation questions. (What do you want to know?) • Use

Data Analysis Protocol

Page 42: How to Use Ed360 and Other Data Tools for Year End Evaluation · •Decide what you want to know. (PURPOSE) • Form your evaluation questions. (What do you want to know?) • Use

Remember, it’s all about practicing the 5-Step process that works for your DT

Reeves’ 5 Steps: Collect & chart data Analyze strengths and

obstacles Establish goals: set,

review, revise Select instructional

strategies Determine results

indicators

Discuss the Process in DT: Consider a problem you face

as a leaders Record data you can

reconstruct from memory or have with you

Set a SMART goal Brainstorm strategies for

moving toward goal List indicators of fidelity and

success

Page 43: How to Use Ed360 and Other Data Tools for Year End Evaluation · •Decide what you want to know. (PURPOSE) • Form your evaluation questions. (What do you want to know?) • Use

Just remember…

If it is not written down, it doesn’t exist!

And

Document your process, analysis and results to demonstrate program effectiveness and areas of needed improvement annually.

Page 44: How to Use Ed360 and Other Data Tools for Year End Evaluation · •Decide what you want to know. (PURPOSE) • Form your evaluation questions. (What do you want to know?) • Use

Questions?44

Page 45: How to Use Ed360 and Other Data Tools for Year End Evaluation · •Decide what you want to know. (PURPOSE) • Form your evaluation questions. (What do you want to know?) • Use

Pause & Ponder45

Please write down your reflections.What ideas stuck with you?What aspects do you disagree with?What aspects do you want to ponder further?What resonates?How does this perspective differ from previous knowledge?

Page 46: How to Use Ed360 and Other Data Tools for Year End Evaluation · •Decide what you want to know. (PURPOSE) • Form your evaluation questions. (What do you want to know?) • Use

Wrap Up46

All Modules for Building Your Data Team (DT) Roadmap Training Modules can be found on Ed360’s Resource page https://www.isbe.net/ed360

Thanks for your participation.

IMPORTANT READING SUGGESTIONS

https://www2.ed.gov/about/offices/list/oese/sst/evaluationmatters.pdf

https://www.isbe.net/ed360
https://www2.ed.gov/about/offices/list/oese/sst/evaluationmatters.pdf
Page 47: How to Use Ed360 and Other Data Tools for Year End Evaluation · •Decide what you want to know. (PURPOSE) • Form your evaluation questions. (What do you want to know?) • Use

Content questions about this presentationGina Fulton: [email protected]

47

Ed360 Outreach [email protected]

Ed360 webpageisbe.net/ed360

Ed360 Logined360.isbe.net

CONTACT INFORMATION

https://ed360.isbe.net/

The users know what they want

Before Reading: Know: Want to Know: Learned:

I want to know

Want to know more… (Further Reading)

DON’T WANT YOU TO KNOW

Information Overload: What You Want to Know (And Don’t ... · Information Overload: What You Want to Know (And Don’t Want to Know) About Your Students Chris Ashley and Anne E

Inquiring Minds Want to Know!

i Really Don't Want to Know

WANT TO KNOW MORE? - LifelineWA

5 things you want to know

So You Want To Know Your Leadership Style?bipard.bih.nic.in/Downloads/Management/So You Want To Know Your... · So You Want To Know Your Leadership Style? Created Date: 10/8/2002

The Advertising Interview. Where you should be right now: Know what job you want Know what job you want Know where you want to work (top 10-15 in order)

What you want to know…

Everything You Want To Know About Home Schoolingcurriculumconnection.net/EverythingYouWantToKnowAboutHomeSchooling.pdf · Everything You Want To Know About Home Schooling – Ultimate

I Want You to Know

We Want to Know

Want to know more about Menopause?

DO YOU WANT TO KNOW WHAT WE KNOW ABOUT OBSERVATIONS

God's People Want to Know

Inquiring Minds Want to Know

Know,Want to know,Learned

Do you really want to know

STRATEGI INQUIRING MINDS WANT TO KNOW UNTUK … · 2017. 9. 15. · pembelajaran Inquiring Minds Want To Know. Strategi pembelajaran Inquiring Minds Want To Know merupakan strategi

You Don't Want to Know

If You Want To Know More

Lady Want to Know

DO YOU WANT TO KNOW WHAT WE KNOW ABOUT OBSERVATIONS

Want to know more? Visit

So, You Want to Know Connectors You Want to Know... · So, You Want to Know Connectors... REFERENCE GUIDE to Cylindrical Connectors Learn the Amphenol Connector Language Notice: Specifications

What do they want to know?

What Patients Want To Know

Want to know me

How do organisms reproduce? Know Want to know Learned

Developing Minds Want to Know!

Everything you want to know