Hope over Fear

Towards an evidence-based teaching profession

Fears‘The establishment considered evidence-based medicine an affront to their omniscience and authority, and dismissed it as both “old hat” (“everybody’s already doing it”) and a “dangerous innovation, perpetuated by the arrogant to serve cost-cutters and suppress clinical freedom.”’

David Sackett (the Godfather of evidence-based medicine aka medicine’s Dylan Wiliam/Rob Coe), 1996

BMJ Editorial

“Despite its widespread acceptance as a mechanism for rational decision making, evidence-based healthcare remains in many ways an ideal rather than a fully fledged reality.”

Carl Heneghan, Professor of Evidence-Based Medicine at the Department of Primary Care Health Sciences at the University of Oxford, Director of the Centre for Evidence- Based Medicine6 February 2013

Why teaching will never be a research-based profession, and

why that’s a good thing…

Beyond Guesswork?Teaching will only become an evidence-based profession when a leadership-wisdom prevails which creates structures in schools where classroom teachers: • can access good evidence easily; • feel encouraged and safe to change their practice

in the light of the evidence; • are supported by a school-based research-lead

with a Higher Education connection; • can evaluate the impact on student outcomes of

the changes to their pedagogy.

Evidence-based Teaching

The practice of evidence-based teaching means integrating

individual pedagogic expertise with the best available external pedagogic evidence from

systematic research.

What evidence-based teaching isn’t…

• Evidence-based teaching is not "cook-book" teaching.

• Evidence-based teaching is not cost-cutting teaching.

• Evidence-based teaching is not restricted to randomised trials and meta-analyses.

Dylan Wiliam

The role of teachers:–All teachers should be seeking to

improve their practice through a process of ‘disciplined inquiry’• Some may wish to share their work with others;• Some may wish to write their work up for

publication;• Some may wish to pursue research degrees;• Some may even wish to undertake research.

A school-based structure to foster disciplined enquiry

CPD is privileged

Performance Development

Research Lead

Student outcomesLesson study

Research Lead1. Help teachers and school leaders ask good questions about prospective changes and improvements they want to make to T&L. 2. Have clear channels of external support e.g. universities – to help to use the best evidence in timely fashion.3. Find and appraise the best evidence – scrutinise the evidence for the “how”, the “who” and the “why”?4. Help ensure that good evidence is channelled through regular CPD to get it to teachers in a format that they can use to take tangible actions in practice. 5. Help to better evaluate changes and interventions that are undertaken in school.

CPD is privileged

Performance Development

Lesson study

How can I observe you in a way that will best help

you improve your teaching?

‘How would you like to be observed to

help you best develop your

teaching?’

It’s all about the outcomes…

Dylan Wiliam

The role of teachers:–All teachers should be seeking to

improve their practice through a process of ‘disciplined inquiry’• Some may wish to share their work with others;• Some may wish to write their work up for

publication;• Some may wish to pursue research degrees;• Some may even wish to undertake research.

Outcomes into learning

“What the best may have, above all, is a capacity to learn and change – and to do so faster than everyone else.”- Atul Gawande

Every September

• Write commentary of your individual exam group results on proforma.

• Spend time in your department shaping a departmental objective based on department-wide examination analysis.

2014

11 studentsMock average residual: -0.3 of a grade averageFinal average residual: - 1.0 of a grade averageMock-Final average residual: -0.6 of a grade

My question…

Why can’t my students apply their conceptual understanding to a new problem, when, fundamentally, the new problem is testing exactly the

same element of their understanding as was tested by the previous

problems from the last three past papers?

How do we help…

…those students with limited short-term

memory?

• Willingham: When students are practising be explicit about the difference between deep structure and surface structure;

• Nuthall: Students need to learn a concept three times before the concept is embedded in their long-term memory;

• Berger: Aim for depth of learning not surface knowledge;

• Brown, Roediger and McDaniel: Create desirable difficulties in the classroom, and use frequent, regular and meaningful testing.

February 2015

Co-planning Lisa’s Lesson

• Year 11 mock feedback.• She modelled her thinking as she completed the

simultaneous equation question.• The class wrote down her answer verbatim as

she modelled thinking through it.• The class then attempted a similar question.• She identified Kallan as someone who

completed the question well.• Kallan modelled his thinking to the class.

September 2015

201411 studentsMock average residual: -0.3 of a grade averageFinal average residual: - 1.0 of a grade averageMock-Final average residual: -0.6 of a grade

201521 studentsMock average residual: -1.8 of a grade averageFinal average residual: - 1.0 of a grade averageMock-Final average residual: 0.8 of a grade

Exogenous Variables and Value-Added Assessments: A Fatal Flaw

I conclude that because of the effects of countless exogenous variables on student

classroom achievement, value-added assessments do not now and may never be

stable enough from class to class or year to year to be used in evaluating teachers.

David C. Berliner, 2014

Why…

…did Oliver make 4 grades improvement between the mock and final examinations?

…did Luke drop by 2 grades between the mock and final examinations?

Did…

… I teach the metacognition skills with the same effectiveness to the two different groups of AS students?

…they all attend all the lessons on metacognition?

Was…

…the metacognition intervention responsible for the 0.8 grade average rise between the mock and final examinations…or would the students’ outcomes have improved anyway?

How…

…on earth did Ben attain a U grade with me when he attained an A grade in his History AS paper?

(Beware….NOT: …on earth did Ben attain an A grade in his History AS paper when he attained a U grade with me?)

Research methods 101: causality• Does c cause e?– Given c, e happened (factual)

• Problem: post hoc ergo propter hoc

– If c had not happened, e would not have happened (counterfactual)• Problem: c did happen

– So we need to create a parallel world where c did not happen• Same group different time (baseline measurement)

– Need to assume stability over time

• Different group same time (control group)– Need to assume groups are equivalent

• Randomised controlled trial

Comparable group?

Yes - comparing 2014 & 15 group the issue with the effect size I reckon. Still, I reckon the improvement can't be ignored & given metacognition has such substantial evidence, you can argue correlational evidence. For me, I secretly don't care much for effect sizes in this type of instance (too many flaws) – but using the evidence focuses us on proper deliberate practice & that matters most for me. Hard thinking teachers get students to think hard for themselves – sounds about right for improvement!

3606. A ‘Growth Mindset’ is encouraged in all students and staff. We aim to develop proficient and independent learners, who are intrinsically motivated by the reward of achievement. We are all part of a learning community. (0-18 numeric range)• 5 - My teacher encourages and helps me to succeed

1 2 3 4 5 6• 18 - My teacher helps me work out ways of solving issues

or problems1 2 3 4 5 6

• 24 - My teacher involves me in making choices and decisions about my school activities

1 2 3 4 5 6

Professor Rob Coe

‘Our strategy should therefore be to make the best choices we can from the best evidence available, to try it out, with an open mind, and see if it works. If it does, we can keep doing it; if not, we will learn from that experience and try something else.’

Green/Amber• Be explicit about surface vs deep;• “Teach” everything 3 times to shift from

short-term to long-term memory;• Chunk things up even smaller for the

“forgetful” boys;• Make “similar question” booklets;• Begin metacognition earlier;• Plan regular testing more rigorously;• “Dictate” model 6 point answers like I did 18

markers.

Red

• Make case studies from real examples from the BBC website and ask students to carefully construct the questions.

Change

• Modify the classroom climate somehow…

Dylan Wiliam

The role of teachers:–All teachers should be seeking to

improve their practice through a process of ‘disciplined inquiry’• Some may wish to share their work with others;• Some may wish to write their work up for

publication;• Some may wish to pursue research degrees;• Some may even wish to undertake research.

CEMB

Centre for Evidence Based Medicine aims to develop, teach and promote evidence-based health care through conferences, workshops and EBM tools so that all health care professionals can maintain the highest standards of medicine.

CEBT @ Huntington School, YorkA demolition site

Better

“We always hope for the easy fix: the one easy change that will erase a problem in a stroke. But few things in life work this way. Instead, success requires making a hundred small steps go right…and making medicine go right is less often like making a difficult diagnosis than like making sure everyone washes their hands.”