geoff lindsay: university of warwick seminar
DESCRIPTION
Seminar Presentation from the Evidence Based Parenting Programmes and Social Inclusion conference held at Middlesex University, 20th September 2012TRANSCRIPT
UK evaluation of evidence based
parenting programmes: Past and
future
Geoff Lindsay
Evidence based parenting programmes and social inclusion
Middlesex University, 20 September 2012
overview
• Developing EB parenting programmes – local
level
• National policy driven – Parenting Early
Intervention Programme 2006-11
• From targeted to universal parenting
programmes: CANparent trial 2012-14
• Future?
1. Initial developments
• Development of a programme
– Key role of the developer
– Theoretical rationale and parents aimed at
– Initial studies: try–out leading to randomized
controlled efficacy trial
– More trials
– Trials by independent researchers
– Effectiveness trial(s) in real life community settings
– Scaling up
• Opting in by individual organisations
• Local, regional, national policy
Developments in UK
• England has implemented national policies of
supporting parents through EB parenting
programmes
• 3-stage process to develop EB practice
– Literature review (Moran et al. 2004)
– Parenting Early Intervention Pathfinder (Lindsay et
al. 2008, 2011a)
– Parenting Early Intervention Programme (Lindsay et
al 2011b)
• Scaling up
• Evaluation of both outcomes and processes
2. The PEIP evaluation 2006-11
• Government initiative, LAs funded
• Pathfinder 2006-8 in 18 LAs,
– 3 EB programmes
– Focus children 8-13 years with or at risk of developing
behavioural, emotional and social difficulties
• Study of effectiveness across 18 LAs?
– Are there different effects for different programmes?
• What influence successful implementation on this scale?
• Implications for policy and practice?
Pathfinder evidence 2006-08
• Positive parent outcomes
– All 3 programmes were effective
– Parents rated them highly
• Process factors also important
– LA variations in efficiency and cost effectiveness
– Importance of the organisations support to implement
• On basis of positive evidence: government
funded a national roll out across all LAs in
England
The PEIP evaluation 2008-11
• Government initiative, all LAs funded
• Can evidence-based parenting programmes be
implemented effectively when rolled out across a
whole country?
– Are there different effects for different programmes?
• What factors influence successful
implementation on this scale?
• Implications for policy and practice?
Purpose of the PEIP evaluation
• All PEIP recognised programmes were evidence-
based (in controlled conditions- efficacy trials)
• Pathfinder showed the effectiveness of the three
programmes (across 18 LAs)
• PEIP explored national roll out across England
– Impact on parent and child outcomes
– LA and organisational factors in implementation
– Differential effects by parents/programme?
• One-year follow up
• Cost effectiveness
The programmesFive main programmes
Pathfinder 2006-08 (Wave 1):
• Triple P
• Incredible Years
• Strengthening Families Strengthening Communities
PEIP 2008-11 (Waves 2 & 3) – above plus:
• Strengthening Families 10-14
• Families and Schools Together (FAST)
– Insufficient data from FAST for the main quantitative analyses
– Similarly for Parent Power, STOP and Parents Plus (added later
in the PEIP)
Quantitative data• Pre-course questionnaires
– 6,143 parents
– Attending 860 groups
– In 43 LAs
• Post-course questionnaires
– returned by 3,325 (54%) of original sample.
• Change from pre-course to post-course in
scores on key parent and child measures
• Is there improvement?
• If so, how much?
Parent mental well-being: Warwick
Edinburgh Mental Well-being Scale
•Each item rated 1-5 (none to all of the time) score range
14–70, highly reliable (alpha=0.94), national norms
•Looking for increase in score
14 items, e.g.:
- I've been feeling optimistic about the future
- I've been feeling useful
- I've been feeling relaxed
Parenting Laxnesse.g. 1 7
1. When I give a fair threat or
warning ....
I always do
what I said
I often don't
carry it out
2. If my child gets upset when I
say 'No' ....
I stick to
what I said
I back down
and give in to
my child
•Each item scored 1-7, score range 6-42,
•Good reliability (alpha=.77).
•High scores represent less effective practice so looking for a
reduction in score.
Parenting Over-reactivity
Each item rated 1-7, score range 6-42, good reliability
(alpha=.72). Looking for reduction in score.
e.g.
When my child misbehaves ....
I raise my
voice or yell
I speak
calmly to my
child
When there's a problem with
my child ....
things build
up and I do
things I don't
mean to
Things don't
get out of
hand
Did PEIP target the ‘right’ parents?-
‘disadvantaged’ Yes
• 91% biological parents to the target child, 85% female
• Wide range of minority ethnic groups (19% vs 8% nationally)
• High proportion single parents (44% vs. 24% national
statistics) two-thirds living in rented property (63% vs. 27%)
• 54% no qualifications or only some GCSEs, but 20% with
HE qualifications and 11% with degrees
• High levels of support needs: 49% had seen GP in last 6
months and 21% had seen a social worker
• Low levels of mental well-being: 75% scored below the
national median as started their programme.
• So, skewed to disadvantaged but note also the range
Parents’ mental well-being at start
Did PEIP target the right children? Yes
• Mean age 8.6 years (but wide range: 37% age 0-7;
54% age 8-13; 9% age 14+)
• 62% boys
• 49% entitled to Free School Meal (16% nationally)
• 12% - statements of SEN (3% nationally)
• 31% - additional educational support in school
• High % had behavioural, emotional and social
difficulties with a negative impact on everyday life
Strengths & Difficulties Questionnaire (SDQ)
• 25 items, each rated on 3 point scale (not true,
somewhat true, certainly true)
• E.g. My child:
– Often has temper tantrums or hot tempers
– Often fights with other children or bullies them
• UK national norms from parents of a sample of
10,000+ children aged 5-15
• About 5-6 times higher prevalence of
behavioural problems than the norm
SDQ at pre-course
(% above clinical cut-off)
SDQ Scale National% PEIP%
Emotional symptoms 11.4 39.2
Conduct problems 12.7 60.7
Hyperactivity 14.7 48.3
Peer problems 11.7 44.7
SDQ Total difficulties 9.8 56.5
Prosocial scale 2.3 18.8
Impact score 8.8 60.4
p < .001 in all cases
Did the parent and child outcomes
improve after the groups?
• All effects are
statistically highly
significant
• Large effect sizes for
parent gains
• Effect size is a standardised
way of presenting the change
in outcomes:
<0.2 small,
0.5 medium;
0.8+ large.
• Parent outcomes show the
largest effects because
directly targeted
Were there different effects by
Programme?• Four programmes had sufficiently large samples to
support comparison:
• All programmes were effective in improving
outcomes, but some (relatively small) differences in
favour of Triple P compared with SFP 10-14
– Taking into account pre-group scores to look at
change/improvement
– Controlling for a range parent and child background factors
(gender, housing, education, ethnicity, child age etc.)
– Including fixed effects for LA
e.g. Comparison of programme effects
relative to Triple P
How did parents rate the parenting group?
• Generally very positive for all programmes
• Two factors
– Group leader style
• made me feel respected (98%);
• understood me and my situation (98%);
• I could be honest about my family (98%);
• interested in what I had to say (98%) etc.
– Helpfulness of the programme
• has been helpful (98%);
• helped me personally to cope (95%);
• helped me deal with child’s behaviour (96%);
• I have fewer problems than before (84%).
Group leaders’ styleHelpfulness of
programme
• SFP 10-14 significantly lower ratings - possibly because mean pre-
course scores for SFP 10-14 were particularly low
One year follow up
• Parent mental well-being fell back somewhat, but still better
than at pre-course
• Improvements in Parenting Scale scores and child
outcomes completely maintained
30
32
34
36
38
40
42
44
46
48
Pre-course Post-course Follow-up
Me
an
sco
re
with
95
% c
on
fid
en
ce
in
terva
l
Mental well being Parenting Child difficulties
Quantitative data: conclusions
• Clear evidence of success of the PEIP
– Parent and child improvements across a range of
important dimensions
– Parents very positive about the group experience
(those who completed)
– Maintained positive outcomes one year later
• Some variation between programmes in gains
– On these particular measures
– Programmes do have different foci
– No random allocation to treatment
Qualitative data
• 429 interviews with
– LA strategic leads and/or operational leads
– other professionals involved in parenting support,
e.g., parenting experts
– parenting group facilitators
– school representatives
– parents
LA factors & quality outcomes
• Strategic leadership & operational co-
ordination combined supported roll-out of the
PEIP in LAs.
• Where these were not in place, the PEIP was
less efficient in organising groups & reaching
parents
– Strategic leadership, including the existence of a
parenting strategy, helped establish the PEIP and
support it through the roll out
– Strategic leadership meshed the PEIP with existing
priorities & infrastructure.
LA factors & quality outcomes
• Operational co-ordination helped PEIP to be
delivered across an LA.
• Models differed – some LAs had one central co-
ordinator & others divided the co-ordination role
on a geographic basis.
– Each model could be effective
Qualitative data conclusions
• PEIP roll-out across LAs was not an easy task,
but
• Possible to achieve positive outcomes with a
diverse workforce & in a range of settings
• Facilitators needed to be able to engage
parents, be well trained, to deliver with fidelity &
to have access to supervision
• Strategic & operational management &
leadership necessary for successful LA roll-out.
So,…
• Overall PEIP was effective on all our measures
– And improvements were maintained one year later
• All 4 programmes were effective
– with some differences between them
• Large variation between LAs in numbers of parents
supported - cost effectiveness varied as a result
• Strategic and operational leadership and management
were crucial
• Use of a wide range of facilitators possible
• Very successful government initiative and clear
evidence supporting use of these programmes
3. CANparent trial
• Change of government – and policy focus
• From targeted to universal parenting classes
• 3 trial areas
– Camden, High Peak Derbyshire, Middlesbrough
• 14 providers of parenting classes
– All met criteria shown by research to be
characteristics of effective parenting programmes
• But lower level of evidence than PEIP
• Aim: to examine whether a market can be
developed so
– no need to subsidise (at all? partially?)
– Providers will want to develop their provision across
England/UK
• Parents receive £100 value vouchers to
‘purchase’ a parenting programme they choose
from those available in that trial area
• 4th non-voucher trial area - Bristol
The CANparent study• Strand 1: Supply side
– How the programme providers develop over the trial
• Strand 2: Survey research and impact evaluation
– Penetration surveys: in trial and comparison areas
• parents attitudes to parenting classes, awareness and
take up.
– Participating parents
• 2000 parent sample: improvements in mental well-
being, satisfaction and sense of effectiveness as a
parents, dealing with daily hassles
• All parents: satisfaction survey
• Strand 3: cost effectiveness
4. future?
• Importance of both targeted and universal
support?
• Parental choice?
• Funding?
• ‘Evidence based’ concept
• Will providers welcome opportunities o develop
large scale?
• Infrastructure?
• Support for implementing? For facilitators?
Maintenance of fidelity?
Conclusions
More information: http://www2.warwick.ac.uk/fac/soc/cedar/
For information about the CANparent evaluation
http://warwick.ac.uk/canparent