what works clearinghouse: educationcdn.trustedpartner.com/docs/library... · 2015. 10. 6. · wwc...
TRANSCRIPT
![Page 1: What Works Clearinghouse: Educationcdn.trustedpartner.com/docs/library... · 2015. 10. 6. · WWC Mission •To be a central and trusted source of scientific evidence for what works](https://reader033.vdocument.in/reader033/viewer/2022052015/602dcd39a3bd1b7b3f044934/html5/thumbnails/1.jpg)
What Works Clearinghouse:Education
M. C. Bradley
Mathematica Policy Research
![Page 2: What Works Clearinghouse: Educationcdn.trustedpartner.com/docs/library... · 2015. 10. 6. · WWC Mission •To be a central and trusted source of scientific evidence for what works](https://reader033.vdocument.in/reader033/viewer/2022052015/602dcd39a3bd1b7b3f044934/html5/thumbnails/2.jpg)
Building Blocks of Presentation
• Understanding the What Works Clearinghouse (WWC)• Introduction to the WWC
• Discussion of the backbone of the WWC
• Examining products released by the WWC, including relevant examples
• Identifying whether a study was reviewed, and its WWC rating
• Searching for particular evidence from Intervention Reports
![Page 3: What Works Clearinghouse: Educationcdn.trustedpartner.com/docs/library... · 2015. 10. 6. · WWC Mission •To be a central and trusted source of scientific evidence for what works](https://reader033.vdocument.in/reader033/viewer/2022052015/602dcd39a3bd1b7b3f044934/html5/thumbnails/3.jpg)
What is the WWC?
![Page 4: What Works Clearinghouse: Educationcdn.trustedpartner.com/docs/library... · 2015. 10. 6. · WWC Mission •To be a central and trusted source of scientific evidence for what works](https://reader033.vdocument.in/reader033/viewer/2022052015/602dcd39a3bd1b7b3f044934/html5/thumbnails/4.jpg)
Problem and Context
• Substantial amount of research on education interventions• Large variation in quality• No common standards• Synthesizing findings is not
simple• Studies summarized many ways
• How does that affect choices?• Often based on personal
experience and ideology • Infrequently on strong research
![Page 5: What Works Clearinghouse: Educationcdn.trustedpartner.com/docs/library... · 2015. 10. 6. · WWC Mission •To be a central and trusted source of scientific evidence for what works](https://reader033.vdocument.in/reader033/viewer/2022052015/602dcd39a3bd1b7b3f044934/html5/thumbnails/5.jpg)
WWC Mission
• To be a central and trusted source of scientific evidence for what works in education• Develops and implements standards for reviewing and
synthesizing education research
• Assesses the credibility of research evidence on the effectiveness of interventions (does not directly assess interventions)
• Transparent, objective, and high quality
![Page 6: What Works Clearinghouse: Educationcdn.trustedpartner.com/docs/library... · 2015. 10. 6. · WWC Mission •To be a central and trusted source of scientific evidence for what works](https://reader033.vdocument.in/reader033/viewer/2022052015/602dcd39a3bd1b7b3f044934/html5/thumbnails/6.jpg)
http://ies.ed.gov/ncee/wwc/pdf/reference_resources/wwc_info_what_061015.pdf
WHAT
The WWC reviews evidence of effectiveness of programs, policies, or practices by using a consistent and transparent set of standards. The WWC doesn’t rank, evaluate, or endorse interventions.
![Page 7: What Works Clearinghouse: Educationcdn.trustedpartner.com/docs/library... · 2015. 10. 6. · WWC Mission •To be a central and trusted source of scientific evidence for what works](https://reader033.vdocument.in/reader033/viewer/2022052015/602dcd39a3bd1b7b3f044934/html5/thumbnails/7.jpg)
WWC: Influencing Research and Practice
7
Producers of Information
• Universities
• Research organizations
• Developers
Consumers of Information
• State and federal govt.
• Education departments
• Superintendents
• School boards
• Principals and teachers
• Parents and the public
• Funders and consultants
Research Standards Effectiveness Information
Evidence-Based
Decisions
Improved Student
Outcomes
Research
Topics
What Works Clearinghouse
• Develop standards
• Train reviewers
• Identify research
• Review research
• Summarize evidence
• Disseminate findings
![Page 8: What Works Clearinghouse: Educationcdn.trustedpartner.com/docs/library... · 2015. 10. 6. · WWC Mission •To be a central and trusted source of scientific evidence for what works](https://reader033.vdocument.in/reader033/viewer/2022052015/602dcd39a3bd1b7b3f044934/html5/thumbnails/8.jpg)
The Review: Backbone of a WWC Product
• Review of publication to determine whether it supports causal inferences
• Three sets of design standards • Group design standards
• Pilot single-case design standards
• Pilot regression-discontinuity design standards
• Four ratings• Meets WWC design standards without reservations
• Meets WWC design standards with reservations
• Does not meet WWC design standards
• Not eligible for review
![Page 9: What Works Clearinghouse: Educationcdn.trustedpartner.com/docs/library... · 2015. 10. 6. · WWC Mission •To be a central and trusted source of scientific evidence for what works](https://reader033.vdocument.in/reader033/viewer/2022052015/602dcd39a3bd1b7b3f044934/html5/thumbnails/9.jpg)
Review Protocols: Set Parameters
http://ies.ed.gov/ncee/wwc/Publications_Reviews.aspx?f=All%20Publication%20and%20Product%20Types,5;#pubsearch
![Page 10: What Works Clearinghouse: Educationcdn.trustedpartner.com/docs/library... · 2015. 10. 6. · WWC Mission •To be a central and trusted source of scientific evidence for what works](https://reader033.vdocument.in/reader033/viewer/2022052015/602dcd39a3bd1b7b3f044934/html5/thumbnails/10.jpg)
Are the groups formed randomly?
Is sample attrition high or low?
Were groups similar before the intervention began?
Are there confounding factors or concerns with outcomes?
http://ies.ed.gov/ncee/wwc/pdf/reference_resources/wwc_info_rates_061015.pdf
Key questions for a group design review
![Page 11: What Works Clearinghouse: Educationcdn.trustedpartner.com/docs/library... · 2015. 10. 6. · WWC Mission •To be a central and trusted source of scientific evidence for what works](https://reader033.vdocument.in/reader033/viewer/2022052015/602dcd39a3bd1b7b3f044934/html5/thumbnails/11.jpg)
http://ies.ed.gov/ncee/wwc/pdf/reference_resources/wwc_info_reporting_061015.pdf
Improvement Index
Effectiveness Rating
Extent of Evidence
What does the WWC report?
![Page 12: What Works Clearinghouse: Educationcdn.trustedpartner.com/docs/library... · 2015. 10. 6. · WWC Mission •To be a central and trusted source of scientific evidence for what works](https://reader033.vdocument.in/reader033/viewer/2022052015/602dcd39a3bd1b7b3f044934/html5/thumbnails/12.jpg)
WWC Products
![Page 13: What Works Clearinghouse: Educationcdn.trustedpartner.com/docs/library... · 2015. 10. 6. · WWC Mission •To be a central and trusted source of scientific evidence for what works](https://reader033.vdocument.in/reader033/viewer/2022052015/602dcd39a3bd1b7b3f044934/html5/thumbnails/13.jpg)
WWC Website: All Resources and Products
http://whatworks.ed.gov
http://ies.ed.gov/ncee/wwc/
![Page 14: What Works Clearinghouse: Educationcdn.trustedpartner.com/docs/library... · 2015. 10. 6. · WWC Mission •To be a central and trusted source of scientific evidence for what works](https://reader033.vdocument.in/reader033/viewer/2022052015/602dcd39a3bd1b7b3f044934/html5/thumbnails/14.jpg)
Intervention Reports
An intervention report is a summary of findings of the most reliable research on a given curriculum, program, practice, or policy in education.
http://ies.ed.gov/ncee/wwc/pdf/intervention_reports/wwc_fasttrack_100714.pdfhttp://ies.ed.gov/ncee/wwc/pdf/intervention_reports/wwc_socialskills_020513.pdf
![Page 15: What Works Clearinghouse: Educationcdn.trustedpartner.com/docs/library... · 2015. 10. 6. · WWC Mission •To be a central and trusted source of scientific evidence for what works](https://reader033.vdocument.in/reader033/viewer/2022052015/602dcd39a3bd1b7b3f044934/html5/thumbnails/15.jpg)
http://ies.ed.gov/ncee/wwc/pdf/reference_resources/wwc_info_making_061015.pdf
1. Define scope of review2. Conduct a comprehensive
search3. Screen and review studies4. Combine findings5. Summarize the review
Steps in Writing an Intervention Report
![Page 16: What Works Clearinghouse: Educationcdn.trustedpartner.com/docs/library... · 2015. 10. 6. · WWC Mission •To be a central and trusted source of scientific evidence for what works](https://reader033.vdocument.in/reader033/viewer/2022052015/602dcd39a3bd1b7b3f044934/html5/thumbnails/16.jpg)
Searching for an intervention report
http://ies.ed.gov/ncee/wwc/Publications_Reviews.aspx?f=All+Publication+and+Product+Types%2c5%3b
![Page 17: What Works Clearinghouse: Educationcdn.trustedpartner.com/docs/library... · 2015. 10. 6. · WWC Mission •To be a central and trusted source of scientific evidence for what works](https://reader033.vdocument.in/reader033/viewer/2022052015/602dcd39a3bd1b7b3f044934/html5/thumbnails/17.jpg)
What can you learn from an intervention report?
http://ies.ed.gov/ncee/wwc/interventionreport.aspx?sid=628
![Page 18: What Works Clearinghouse: Educationcdn.trustedpartner.com/docs/library... · 2015. 10. 6. · WWC Mission •To be a central and trusted source of scientific evidence for what works](https://reader033.vdocument.in/reader033/viewer/2022052015/602dcd39a3bd1b7b3f044934/html5/thumbnails/18.jpg)
Parts of an Intervention Report
• Program description
• Research
• Effectiveness
http://ies.ed.gov/ncee/wwc/pdf/intervention_reports/wwc_careeracademies_092215.pdf
![Page 19: What Works Clearinghouse: Educationcdn.trustedpartner.com/docs/library... · 2015. 10. 6. · WWC Mission •To be a central and trusted source of scientific evidence for what works](https://reader033.vdocument.in/reader033/viewer/2022052015/602dcd39a3bd1b7b3f044934/html5/thumbnails/19.jpg)
Inside an Intervention Report
• Program Information
• Research Summary
• Effectiveness Summary
• References
• Appendices
http://ies.ed.gov/ncee/wwc/pdf/intervention_reports/wwc_careeracademies_092215.pdf
![Page 20: What Works Clearinghouse: Educationcdn.trustedpartner.com/docs/library... · 2015. 10. 6. · WWC Mission •To be a central and trusted source of scientific evidence for what works](https://reader033.vdocument.in/reader033/viewer/2022052015/602dcd39a3bd1b7b3f044934/html5/thumbnails/20.jpg)
Single Study Reviews
Assess one study against WWC standards• Not a systematic review on that particular intervention
First stage for some single study reviews is a “Quick Review”• Email blast giving sense of whether study will meet standards,
ideally one week after media attention
![Page 21: What Works Clearinghouse: Educationcdn.trustedpartner.com/docs/library... · 2015. 10. 6. · WWC Mission •To be a central and trusted source of scientific evidence for what works](https://reader033.vdocument.in/reader033/viewer/2022052015/602dcd39a3bd1b7b3f044934/html5/thumbnails/21.jpg)
What can you learn from a quick review?
http://ies.ed.gov/ncee/wwc/quickreview.aspx?sid=20155
![Page 22: What Works Clearinghouse: Educationcdn.trustedpartner.com/docs/library... · 2015. 10. 6. · WWC Mission •To be a central and trusted source of scientific evidence for what works](https://reader033.vdocument.in/reader033/viewer/2022052015/602dcd39a3bd1b7b3f044934/html5/thumbnails/22.jpg)
What can you learn from a single study review?
http://ies.ed.gov/ncee/wwc/SingleStudyReview.aspx?sid=10112
![Page 23: What Works Clearinghouse: Educationcdn.trustedpartner.com/docs/library... · 2015. 10. 6. · WWC Mission •To be a central and trusted source of scientific evidence for what works](https://reader033.vdocument.in/reader033/viewer/2022052015/602dcd39a3bd1b7b3f044934/html5/thumbnails/23.jpg)
http://ies.ed.gov/ncee/wwc/pdf/single_study_reviews/wwc_griffiths_070715.pdf
![Page 24: What Works Clearinghouse: Educationcdn.trustedpartner.com/docs/library... · 2015. 10. 6. · WWC Mission •To be a central and trusted source of scientific evidence for what works](https://reader033.vdocument.in/reader033/viewer/2022052015/602dcd39a3bd1b7b3f044934/html5/thumbnails/24.jpg)
WWC Practice Guides
A practice guide is a set of recommendations for educators to address challenges in their classrooms and schools. The level of research evidence supporting each recommendation is provided.
• Expert panel to develop recommendations
• Conduct a comprehensive literature search
• Review each study against WWC standards
• Summarize the review
http://ies.ed.gov/ncee/wwc/Publications_Reviews.aspx?f=All%20Publication%20and%20Product%20Types,3;#pubsearch
![Page 25: What Works Clearinghouse: Educationcdn.trustedpartner.com/docs/library... · 2015. 10. 6. · WWC Mission •To be a central and trusted source of scientific evidence for what works](https://reader033.vdocument.in/reader033/viewer/2022052015/602dcd39a3bd1b7b3f044934/html5/thumbnails/25.jpg)
What is in a practice guide?
http://ies.ed.gov/ncee/wwc/pdf/practice_guides/wwc_algebra_040715.pdf http://ies.ed.gov/ncee/wwc/pdf/practice_guides/early_math_pg_111313.pdf
![Page 26: What Works Clearinghouse: Educationcdn.trustedpartner.com/docs/library... · 2015. 10. 6. · WWC Mission •To be a central and trusted source of scientific evidence for what works](https://reader033.vdocument.in/reader033/viewer/2022052015/602dcd39a3bd1b7b3f044934/html5/thumbnails/26.jpg)
Recommendations
![Page 27: What Works Clearinghouse: Educationcdn.trustedpartner.com/docs/library... · 2015. 10. 6. · WWC Mission •To be a central and trusted source of scientific evidence for what works](https://reader033.vdocument.in/reader033/viewer/2022052015/602dcd39a3bd1b7b3f044934/html5/thumbnails/27.jpg)
How to steps
![Page 28: What Works Clearinghouse: Educationcdn.trustedpartner.com/docs/library... · 2015. 10. 6. · WWC Mission •To be a central and trusted source of scientific evidence for what works](https://reader033.vdocument.in/reader033/viewer/2022052015/602dcd39a3bd1b7b3f044934/html5/thumbnails/28.jpg)
Lots of Examples
![Page 29: What Works Clearinghouse: Educationcdn.trustedpartner.com/docs/library... · 2015. 10. 6. · WWC Mission •To be a central and trusted source of scientific evidence for what works](https://reader033.vdocument.in/reader033/viewer/2022052015/602dcd39a3bd1b7b3f044934/html5/thumbnails/29.jpg)
What can an example do for you?
![Page 30: What Works Clearinghouse: Educationcdn.trustedpartner.com/docs/library... · 2015. 10. 6. · WWC Mission •To be a central and trusted source of scientific evidence for what works](https://reader033.vdocument.in/reader033/viewer/2022052015/602dcd39a3bd1b7b3f044934/html5/thumbnails/30.jpg)
What can an example do for you?
![Page 32: What Works Clearinghouse: Educationcdn.trustedpartner.com/docs/library... · 2015. 10. 6. · WWC Mission •To be a central and trusted source of scientific evidence for what works](https://reader033.vdocument.in/reader033/viewer/2022052015/602dcd39a3bd1b7b3f044934/html5/thumbnails/32.jpg)
If you are a researcher,
http://ies.ed.gov/ncee/wwc/pdf/reference_resources/wwc_gsa_v1.pdf
![Page 33: What Works Clearinghouse: Educationcdn.trustedpartner.com/docs/library... · 2015. 10. 6. · WWC Mission •To be a central and trusted source of scientific evidence for what works](https://reader033.vdocument.in/reader033/viewer/2022052015/602dcd39a3bd1b7b3f044934/html5/thumbnails/33.jpg)
Has a study been reviewed by the WWC?
http://ies.ed.gov/ncee/wwc/ReviewedStudies.aspx
![Page 34: What Works Clearinghouse: Educationcdn.trustedpartner.com/docs/library... · 2015. 10. 6. · WWC Mission •To be a central and trusted source of scientific evidence for what works](https://reader033.vdocument.in/reader033/viewer/2022052015/602dcd39a3bd1b7b3f044934/html5/thumbnails/34.jpg)
Find What Works
http://ies.ed.gov/ncee/wwc/findwhatworks.aspx
![Page 35: What Works Clearinghouse: Educationcdn.trustedpartner.com/docs/library... · 2015. 10. 6. · WWC Mission •To be a central and trusted source of scientific evidence for what works](https://reader033.vdocument.in/reader033/viewer/2022052015/602dcd39a3bd1b7b3f044934/html5/thumbnails/35.jpg)
Contact Information
The What Works Clearinghouse
• http://whatworks.ed.gov
On Facebook at
• www.facebook.com/whatworksclearinghouse
On Twitter at
• www.twitter.com/WhatWorksED
And a widget you can add to your own pages
• http://ies.ed.gov/ncee/wwc/Widget.aspx
M. C. Bradley