The Big Plus Effectiveness Research

May 2008

2

Background

• Approx. 800,000 adults with low literacy and numeracy skills in Scotland

• The Big Plus was launched in January 2004• ‘to encourage adults in Scotland who need

help with reading, writing and numbers to access free local support’

• 2 campaigns: September 2006 and January 2007

• 2005/2006: 5-6000 adults went on to learn after calling helpline

• 30,000 took a different route into learning

3

Additional aims include measuring/determining:• The effectiveness of the fulfilment process• Satisfaction with the service• What motivates callers?• How many call for themselves and how many with

some one else in mind?• What action has callers to the helpline made since

calling?• How many callers have gone on to improve their

literacy/numeracy skills?

Objectives (1)

Overall objective: to assess the effectiveness of the campaign

4

Objectives (2)

Additional aims include measuring/determining:• What motivated those who have gone on to learn to

do so?• If callers have not yet gone on to learn, what might

have prevented them from doing so?• If callers have not yet gone on to learn, what might

motivate them to do so?• What motivated non BP callers to get into learning?• Are these non BP callers aware of The Big Plus

campaign?

5

Methodology• CATI – telephone interviews• Sample size

• 250 BP helpline callers (Phase 1)• 250 learners who approached the partnerships directly

(Phase 2)

• Progress to date• The Big Plus interviews completed by Progressive’s CATI

team in April 2007 • Toplines produced May 2007• Partnership Direct interviews commenced May 2007

• 82 interviews completed by end of July

• Sample provided by partnerships across the country

• This sample size has a maximum standard error of +/- 11% at the 95% confidence interval

Sample is not representative of all learners within Partnerships as the sample obtained did not include all Partnerships or all types of learners

6

Sample profileThe Big Plus

Partnership Direct

The Big Plus

Partnership Direct

The Big Plus

Partnership Direct

Age Age left FT education

SEG

18 – 24 16% 20% 15 or under 31% 32% AB 4% 2%

25 – 34 19% 23% 16 40% 35% C1 20% 26%

35 – 44 26% 28% 17 10% 11% C2 15% 11%

45 – 54 30% 17% 18 2% 6% D 27% 27%

55 – 64 6% 10% 19 0% - E 33% 34%

65+ 5% 2% 20 2% 2%

Gender 21 or over 7% 6%

Male 47% 45% Still in FT education

3% 6%

Female 53% 55% Unsure 5% 1%

Base 250 82 250 82 250 82

Partnership sample slightly skewed towards younger ages (18-34). Otherwise broadly consistent with TBP caller sample

Key Findings

AwarenessPartnership Direct

9

Spontaneous awareness of orgs who offer help with reading, writing & numbers Partnership Direct

TBP brand spontaneously mentioned by a fifth (compare with 9% gen pop from brand tracker Mar 07)

Base: Partnership Direct (82)

35%

14% 13% 10%5% 2% 1% 1% 1% 1%

17%

5% 4%

21%

0%

20%

40%

60%

80%

100%

FE Colle

ge

The B

ig P

lus

Buddies

for L

earn

ing

Learn

ing

cent

re lds

CLAN

LEA

Job C

entre

Privat

e co

.

Libra

ry

Unive

rsity

Other

None

Unsure

10Awareness of TBP advertising = very strong. (compare with 16% gen pop

ad awareness from brand tracker Mar 07)

87%

66%60% 57% 55% 54%

43%38%

29% 29%

15%11%

79%

0%

20%

40%

60%

80%

100%

lds

The B

ig P

lus

FE Colle

ge

Job C

entre

Caree

rs S

cotla

nd

Libra

ry

Open U

nive

rsity

Learn

ing

cent

re

ILA S

cotla

ndLEA

Unive

rsity

Privat

e co

RAW/B

BC

Base: Partnership Direct (82)

Prompted advertising awareness Partnership Direct

11

Have you heard of The Big Plus?Partnership Direct

Yes definitely67%

Yes I think so15%

No18%

Base: Partnership Direct – all who have not mentioned Big Plus (72)

Two-thirds of those who hadn’t already mentioned TBP are aware.

12

How did you hear about The Big Plus?Partnership Direct

22%

22%

3%

3%

6%

8%

11%

43%

0% 20% 40% 60% 80% 100%

Unsure

Other

My Doctor/GP

Radio Ad

Job Centre Plus

Friend/Family member

Poster

Leaflet

TV Ad

Base: Partnership Direct – all who have not mentioned Big Plus (72)

Most awareness driven by TV ads.

13

Impact of The Big Plus ads in decision to get helpPartnership Direct

Yes , 19%

No, 79%

Unsure, 1%

Base: Partnership Direct – all who have not mentioned Big Plus (72)

“Just basically making me think I need help”

“I saw the poster and thought I wanted to get out and do

something with my life”

“Knowing it’s there encouraged me”

“The personal experiences of other people in the advert made me think

about my own experiences”

“The fact that it’s one to one, not in a big group”

“The advert showed people who couldn’t help their kids because of their own

problems – that scared me”

14

What prompted callers to contact TBP? The Big Plus

5%

19%

2%

4%

4%

4%

6%

61%

0% 10% 20% 30% 40% 50% 60% 70%

Unsure

Other

Recommended by doctor/GP

Poster

Advice from friend/family

Radio ad

Recommended by Job Centre Plus

TV ad

Leaflets, website, wanting help with maths.

Three out of five callers said that the TV ads were the reason they called the helpline.

Base: TBP helpline (250)

15

How easy was it to get through to someone?The Big Plus

Base (BP Callers): 250

Learners: 138

Non-learners: 112

60%

73%

67%

23%

17%

20%

6%

3%

4%3%

5%

5%

5%

4%

Non-learner

Learner

Total

Very Easy Quite Easy Neither Quite Difficult Very Difficult Unsure

87% said that they found it easy to get through to someone when they called the helpline.

16

Satisfaction with the telephone service The Big Plus

7%

4%

4%

4%

15%

68%

0% 10% 20% 30% 40% 50% 60% 70%

Unsure

Very dissatisfied

Quite dissatisfied

Neither/nor

Quite satisfied

Very satisfied

Base: TBP helpline (250)

Satisfaction levels were high with 83% indicating that they were happy with the service they received. However Learners tended to be more

satisfied (94%) than Non-learners (68%)

17

Summary: ContactPartnerships Direct• Majority contacted partnerships for themselves (93%)• 83% have never contacted TBP directly – as expected• TV ads were the main source of awarenessThe Big Plus Callers• 3 out of 5 callers said the TV ads prompted them to make

the call• Leaflets, website and personal goals (e.g. want to improve maths)

also mentioned by some• 86% called for themselves (they were more likely to do so

if in lower SEGs)• 85% did get a call back • 85% went on to call contact provided • 83% satisfied with telephone service • 85% in 35-54 yr olds, but 69% in 55+ group

18

Did callers receive a pack?The Big Plus

Three quarters of callers said that they received an information pack

Yes

75%

No

17%

Unsure

8%

Base: TBP helpline (250)

19

Summary: The information packThe Big Plus

• 83% found the information pack ‘very’ or ‘quite’ useful

• 7% said it was ‘not very useful’ or ‘not useful at all’

• Main reason given for dissatisfaction: not enough detail on specific courses in pack

• Small number said they could not read the pack

20

Summary: The non-learner’s experienceThe Big Plus

Callers that did not go on to learn typically experienced one or more of the following:

• Did not find it easy to get through to a Big Plus advisor

• Did not receive the pack • Did not get a call back when promised • Were not transferred immediately to a learning centre

or specialist • Less likely to call the contact provided by the Big Plus

21

Summary: The non-learner’s experienceThe Big Plus

Subsequently callers that did not go on to learn were less likely to:

• Be satisfied with the telephone service• Be satisfied with the overall service provided • Recommend the Big Plus to a friend

Partnership ExperienceThe Big Plus & Partnership Direct

Includes: Community Centres, Learning Centres, Libraries, Colleges and other venues

23

Where learners went…

Again, largely reflective of source of sample for Partnerships stage.

12%

9%

30%

33%

16%

15%

5%

16%

21%

44%

0% 20% 40% 60% 80% 100%

Somewhere else

Library

Community Centre

Learning Centre

College

Partnership Direct

The Big Plus Helpline

Base: Partnership Direct (82); TBP helpline (250)

School (4); CLAN (1); Job Centre (1); Private tutor (1)

24

What happened when The Big Plus callers made contact with centres?The Big Plus

• Most common answers:• Started course (56%)• Attended centre to visit centre/meet tutor

/discuss needs (31%)

• Small number said they were directed to online learning

Base (TBP callers that went on to learn) : 32

They gave me time to attend and told me where to go

Had a meeting then they asked me to study

through the internet

I was very worried about going to class… but the staff were very helpful and one of the girls that works in the learning centre met me outside so I

didn’t have to go in on my own

25

Summary: Partnership experience

Partnership Direct• College and learning centres were the most common places

where those who contacted them directly, undertook learning• Everyone was satisfied with the helpfulness of the staff

(100%)• 97% felt that the service they had received was useful• 96% felt ‘very’ or ‘quite’ satisfied with the serviceThe Big Plus• Learning centres and community centres were most common

places to undertake ALN learning • Satisfaction levels with staff were very high – 94% said they

were helpful• 93% of learners said that the service they had received was

useful• 94% felt ‘very’ or ‘quite’ satisfied with the service

26

Reasons expectations met / exceededPartnership Direct

“if I need help they give it to me”

“we don’t use kids’ books! But we do go

back to basics”

“They're really friendly and willing to help anyone in difficulties. They don’t

embarrass you.”

“I do have more confidence and I’ve met with people I

wouldn’t have spoken to before”

“I now have the confidence to write a

business letter, or to the local press or an MP”

“it was just really good. I didn’t think people would be

so understanding”

“I got help”

27

Reasons expectations met / exceededThe Big Plus

Most common responses No.

They were very nice/helpful/friendly 10

Course/service was useful 4

I got the information I needed/was looking for 4

I got more (information) than I expected 2

Satisfied with service/advice 2

They provided all the information I was looking for

I got a lot more out of it than I expected

They were honest and answered any questions they

could, and they said when they didn’t know the answerThey made me feel

comfortable

28

Summary: Expectations

• Those who went though the Partnerships direct were unsure whether or not their expectations had been met as many were not sure what to expect

• Much more likely to be met when The Big Plus callers go on to learn

• Expectations met when: • Staff friendly, helpful and made learners feel at ease• Courses are correct level• Information is relevant and correct amount

• Expectations not met when: • Call back/info pack not received• Courses/service not relevant/suitable

29

End result

All of those in the partnership sample had done some learning therefore their end result is further on than some in TBP sample. 70% had improved their reading /

writing or number skills.

9%

20%

28%

6%

22%

24%

28%

22%

2%

11%

11%

39%

70%

1%

1%

0% 20% 40% 60% 80% 100%

Other

Done nothing

Had further help with reading, writing,numbers

I am now employed

Started higher/further education

Started learning

Increased confidence

Improved my reading, writing and numberskills

Partnership Direct

The Big Plus Helpline

Base: Partnership Direct (82); TBP helpline (250)

30

Have skills improved as a result of the help received?

28%22%

34%

16%

77%

21%

2%0%

20%

40%

60%

80%

100%

Yes, a lot Yes, a little No Unsure

The Big Plus Helpline

Partnership Direct

Base: Partnership Direct (82); TBP helpline (250)

Real progress evident as a result of help from Partnerships.

31

“I have better qualifications and job prospects and I’ve

made friends” “I feel I’ve developed more as a person, it’s opened up

a lot for me”

“I feel a lot happier about it all…before I was a nervous

wreck and couldn’t talk about this”

“I can now do everyday things I couldn’t do before”

“I now use a computer”

“I’ve passed my driving theory test”

Why skills have improved Partnership Direct

32

Why skills have improvedThe Big Plus

Most common responses No.

Writing/spelling/English/number skills have improved 17

Learned things I didn’t think I was able to do 3

Confidence has increased 3

Had a good tutor/teacher 2

I couldn’t read and write before the Big Plus and

now I can… but still have problems with big words

I’ve got dyslexia, and at one time I didn’t have any idea how to break words up, but now I do

Having a one to one tutor gave me a lot of confidence, than I was able to work in groups… the step

by step process really worked

The tutor helps me to concentrate

33

Overall satisfaction with the whole process The Big Plus

6%

4%

5%

2%

26%

57%

0% 10% 20% 30% 40% 50% 60% 70%

Unsure

Very dissatisfied

Quite dissatisfied

Neither/nor

Quite satisfied

Very satisfied

Base: TBP helpline (250)

Overall 83% of callers said they were satisfied with the service they received throughout the whole process

34

9%

4%

3%

4%

15%

66%

1%

1%

1%

5%

91%

0% 20% 40% 60% 80% 100%

Unsure/Don't know

Not at all likely

Not very likely

Neither/nor

Quite likely

Very likely

Partnership Direct

The Big Plus Helpline

Base: Partnership Direct (82); TBP helpline (250)

Reflective of high satisfaction levels, virtually all learners in the Partnership sample would recommend the experience to others.

Likelihood to recommend

35

Effectiveness of TBP advertising in encouraging people to get help Partnership Direct

20%

15%

4%

5%

1%

35%

21%

0% 20% 40% 60% 80% 100%

Unaware of TPBads

Unsure/Don't know

Not at all effective

Not very effective

Neither/nor

Quite effective

Very effective

Base: Partnership Direct (82)

Clearly TBP advertising plays a part in encouraging even those who go direct to the partnerships to get help.

Summary & Conclusions

37

Overall Summary – Partnerships Direct• A fifth spontaneously aware of TBP brand• 79% aware of TBP advertising

• Upon further prompting 72% definitely aware of TBP

• And 13% thought they had heard of TBP

• Most awareness driven by TV• And 19% of this sample stated that the ads

impacted on their decision to get help

TBP definitely impacting on learners who go to Partnerships direct

38

Overall Summary – The Big Plus Callers

• The TV ad was the main prompt for callers• Majority of callers contacted TBP for

themselves rather than someone else• 87% found it easy to get through to someone

• Learners (90%) found it easier than non-learners (83%)

• High levels of satisfaction with the telephone service• However Non-Learners (68%) were less satisfied

than Learners (94%)

39

Conclusions• The majority contacted a Partnership (93%) or The

Big Plus (86%) for themselves• Overall respondents very positive about the

services provided by the Partnerships• Helpfulness of staff (99% - Partnership Direct, 94% - TBP,

Very or Quite helpful)• Usefulness of service (97% - Partnership Direct, 93% - TBP,

Very or Quite useful)• Overall satisfaction with the service (96% - Partnership

Direct, 94% - TBP, Very or Quite satisfied)

• This translates into likelihood to recommend • Real progress evident in skills as a result of help

from Partnerships

Together TBP helpline and Partnerships doing a good job!

40

Learners - key strengths to maintain

Quality of teaching

Appropriate level of teaching

Helpfulness of staff

Friendliness of staff

Appropriate amount of info

41

Recommendations - Learners

More one-to-one learning options

Higher staff/learner ratio

Family friendly/ flexible hours

Call people back sooner

Very important

More detailed info in pack about what to expect

Not as important

More choice of services in local area

More publicity / advertising

42

Non learners – why?

Could be won over with improvements

Non-Learners (because…)

Found service was not suitable

Not satisfied with service

Continue to provide direction to more

appropriate services

Ensure all advertising / comms give clear

message about what to expect from TBP

43

Recommendations – Non Learners

More one-to-one learning options

Higher staff/learner ratio

Family friendly/ flexible hours

Call people back sooner

Very important

Ensure pack reaches caller

Not as importantMore choice of services in local

area More publicity / advertising

More people straight through to a centre or specialist

More detailed information in the

packTelephone staff

trained to increase confidence

44

17 Corstorphine RoadEdinburgh, EH12 6DD

t: 0131 316 1900 f: 0131 316 1901

jacqueline.meldrum@progressivepartnership.co.uklouise.fraser@progressivepartnership.co.uk