platform$3$2013715pilot$evaluation$ - communication trust ·...

The Communication Trust Platform 3 2013-‐15 pilot evaluation Dr Sam Baars

[email protected] -‐ www.lkmco.org.uk -‐ @LKMco -‐ +44(0)7793 370459

“Society should ensure that all young people receive the support they need in order to make a fulfilling transition to adulthood”

2

Sam Baars is a Research Associate at LKMco. He has particular interests in youth research, area-‐based inequalities and social science impact, and has experience using a range of quantitative and qualitative methods, from film-‐based work in schools to rapid research reviews and large-‐scale survey analysis. Sam believes that robust, innovative social research is the key to tackling the barriers that prevent some young people from making fulfilling transitions to adulthood, and he channels this belief into a range of research projects at LKMco. Sam holds a PhD in Social Change from the University of Manchester. This report was written by the education and youth development ‘think and action tank’ LKMco. LKMco is a social enterprise -‐ we believe that society has a duty to ensure children and young people receive the support they need in order to make a fulfilling transition to adulthood. We work towards this vision by helping education and youth organisations develop, evaluate and improve their work with young people. We then carry out academic and policy research and advocacy that is grounded in our experience. www.lkmco.org.uk / @LKMco / [email protected]



3

The Communication Trust Platform 3 2013-‐15 pilot evaluation

Table of contents 1 Overview ........................................................................................................................................................................... 6

2 Methodology ..................................................................................................................................................................... 6 2.1 Quality of individual practice .................................................................................................................................................... 6 2.2 Quality of language environments ........................................................................................................................................... 7 2.3 Practitioners’ knowledge and confidence ................................................................................................................................ 7 2.4 Working with parents to support their child’s speech language and communication ............................................................. 7 2.5 Early identification of children with delayed speech, language and communication .............................................................. 8 2.6 Effective use of the 2-‐year progress check ............................................................................................................................... 8 2.7 Effectiveness of the online delivery model ............................................................................................................................... 8 2.8 Differences between roles ........................................................................................................................................................ 8 2.9 A note on statistical significance .............................................................................................................................................. 8

3 The impact of Platform 3 .................................................................................................................................................... 9 3.1 The quality of early years speech, language and communication provision ............................................................................ 9

3.1.1 Number of practitioners qualified to Level 3 ..................................................................................................................... 9 3.1.2 Quality of individual practice ............................................................................................................................................. 9 3.1.3 Quality of communication environments ........................................................................................................................ 10

3.2 Practitioners’ effectiveness in working with children to support their speech, language and communication .................... 11 3.2.1 Practitioners’ knowledge ................................................................................................................................................. 11 3.2.2 Practitioners’ confidence ................................................................................................................................................. 14 3.2.3 Differences between roles ............................................................................................................................................... 15 3.2.4 Working with parents to support their child’s speech, language and communication ................................................... 15

3.3 Early identification of children with delayed speech, language and communication and effective use of the 2-‐year check 19 3.3.1 Early identification of children with delayed speech, language and communication ...................................................... 19 3.3.2 Effective use of the 2-‐year progress check ...................................................................................................................... 19

4 The effectiveness of the online delivery model ................................................................................................................ 23 4.1 Learning materials .................................................................................................................................................................. 23 4.2 The eportfolio area ................................................................................................................................................................. 24 4.3 Support from tutors ................................................................................................................................................................ 25 4.4 Benefits and challenges of completing the qualification online ............................................................................................. 26

4.4.1 Benefits of completing the qualification online ............................................................................................................... 26 4.4.2 Challenges of completing the qualification online .......................................................................................................... 27

4.5 Quality of work produced ....................................................................................................................................................... 28

5 Conclusions and recommendations .................................................................................................................................. 29 5.1 Conclusions ............................................................................................................................................................................. 29

5.1.1 The quality of early years speech, language and communication provision ................................................................... 29 5.1.2 Practitioners’ effectiveness in working with children to support their speech, language and communication .............. 29 5.1.3 Early identification of children with delayed SLC and effective use of the 2-‐year check ................................................. 30 5.1.4 The effectiveness of the online delivery model ................................................................................................................ 30

5.2 Evaluation recommendations ................................................................................................................................................. 30 5.3 Programme recommendations ............................................................................................................................................... 30



4

Executive summary This impact evaluation assesses the impact of the 2013-‐15 Platform 3 pilot on three outcomes:

1. The quality of early years speech, language and communication provision 2. Practitioners’ effectiveness in working with children to support their speech, language and communication 3. Early identification of children with delayed speech, language and communication and effective use of the 2-‐

year progress check The evaluation also considers the effectiveness of the online delivery model used for Platform 3, in order to identify the factors that have supported and hindered the programme in achieving its outcomes. The evaluation is based on qualitative and quantitative analysis of a range of data including: surveys completed by learners before, during and after the course; observations of individual practice and setting environments; phone interviews conducted with learners upon completion of the course, and focus groups with tutors. The main findings of the evaluation are: The quality of early years speech, language and communication provision

• 150 practitioners completed the course, gaining the Level 3 qualification Supporting children and young people’s speech, language and communication

• The quality of individual practice improved during the course of the programme, with significant improvements in two specific area: demonstrating/modelling rather than criticising, and using language at roughly the same level as the child

• The quality of practitioners’ language environments also showed signs of improvement during the course, with greater evidence of positive environmental features, and fewer instances where good practice could not be evidenced -‐ particularly in relation to language learning interactions

• Learners reported unanimously that they were more confident, knowledgeable and skilled in supporting the development of speech, language and communication after having completed the course. There were significant increases in learners’ overall knowledge of speech, language and communication, their own assessments of their knowledge, and also in their confidence

Practitioners’ effectiveness in working with children to support their speech, language and communication

• There were significant increases in learners’ confidence in working with parents/carers to support speech, language and communication, with their knowledge in this area also increasing during the course. Outcomes were particularly positive for learners who took the optional unit Working with parents, families and carers to support their children’s speech, language and communication development

• During the course of the programme practitioners became less likely to rely on face-‐to-‐face strategies, and more likely to rely on formal records when gathering information from parents to support progress checks

Early identification of children with delayed speech, language and communication and effective use of the 2-‐year check

• More than 98% of learners reported that they felt more knowledgeable, confident and skilled in identifying children struggling with speech, language and communication than they had been before taking the qualification. The largest increases in practitioners’ confidence related to their knowledge of typically developing speech, language and communication and their ability to support children who are struggling with their speech, language and communication

• There were significant increases in practitioners’ confidence in using the 2-‐year progress check as a basis to structure discussions with parents about children’s progress, and being able to provide a short summary of a child’s progress

• After completing the qualification practitioners were relatively more likely to use progress checks as a basis for liaising directly with other professionals rather than via parents



5

The effectiveness of the online delivery model • Over 90% of learners rated the content, presentation and flow of the Platform 3 learning materials as good

or excellent. Ratings of the ease of use and navigation of the learning materials were slightly less positive, with 71% judging them to be good or excellent

• Overall, learners rated the support they had received from their tutors highly. However, these ratings diverged significantly between learners from different assessment centres

• Learners reported that the main benefit of completing the qualification online was the flexibility this gave them to study in their own time and fit their learning around work and family commitments. Meanwhile, time pressures were cited as the biggest challenge of studying in this way

• 95% of learners stated they felt happy with the quality of work they had produced during the course



6

1 Overview The Communication Trust works to ensure that all children and young people are supported to develop their speech, language and communication skills. It has a particular focus on ensuring that those children with speech, language and communication needs (SLCN) have their needs identified at the earliest possible point and receive timely support, including targeted and specialist interventions, appropriate to their needs. In November 2012 the Trust successfully applied to the Department for Education VCS grant scheme to pilot a model of delivering the Level 3 qualification Supporting children and young people’s speech, language and communication via e-‐learning and assessment over two years from April 2013 to March 2015. The aim of the project was to develop, in cross-‐sector partnership, an accessible, cost-‐effective route for early year’s practitioners, from a wide range of backgrounds, to gain this robust Level 3 Award through an online route of learning, assessment and accreditation. This evaluation considers the extent to which Platform 3 has impacted on the following three outcomes:

1. The quality of early years speech, language and communication provision 2. Practitioners’ effectiveness in working with children to support their speech, language and communication 3. Early identification of children with delayed speech language and communication and effective use of the 2-‐

year progress check In addition the evaluation considers the effectiveness of the online delivery model used for Platform 3, in order to identify the factors that have supported and hindered the programme in achieving its outcomes. The report concludes with a set of recommendations arising from these findings.

2 Methodology This impact evaluation draws on a range of quantitative and qualitative data gathered before, during and after the 2013-‐15 Platform 3 programme, including practitioner surveys, tutor feedback, telephone interviews with learners and practice and setting observations.

2.1 Quality of individual practice Changes in the quality of individual practice during the course of the Platform 3 programme were assessed by observing practitioners in their settings before and after the course. Observations were conducted by a variety of practitioners including speech and language therapists, educational psychologists and senior setting staff, with the same person completing both pre-‐ and post-‐qualification observations to support validity. Observations were designed to record evidence-‐based interaction strategies that facilitate speech, language and communication development in children, adapted from widely available resources developed for practitioners12. The observation tool documents the frequency with which a given practitioner uses 10 specific positive interaction strategies, outlined in Figure 1.

1 ICAN (2007) Top techniques to develop children’s speech and language: a guide for early years practitioners. Available at: http://www.ican.org.uk/~/media/Ican2/Book%20Shop/Downloads/top%20techniques%20chart001%20pdf.ashx [Accessed 29th April 2015]. 2 The Communication Trust (2011) Universally Speaking 0-‐5. Available at: http://www.talkingpoint.org.uk/sites/talkingpoint.org.uk/files/FINAL_Universally_Speaking_-‐_Early_Years_Web_Version.pdf [Accessed 6th May 2015].



7

Figure 1

1 Responding to what the child says by repeating/answering 2 Following the child’s lead in play 3 Using simple, repetitive language 4 Building on and extending what the child says 5 Demonstrating/modelling rather than criticising e.g. ‘yes, you saw a cat’, instead

of ‘no, it’s called a cat, not a tat’) 6 Using language at roughly the same level as the child, i.e. not using language

that is too complex for the child 7 Giving the child time to respond 8 Giving the child time to initiate interaction, using words, eye contact or gestures 9 Being careful with questions – using more comments than questions 10 Using an interesting voice to maintain the child’s interest

Observations were made of 104 practitioners pre-‐qualification and 32 practitioners post-‐qualification, yielding a matched sample of 25 practitioners (17% of those who completed the qualification) who had both a pre-‐ and post-‐qualification observation. Our analysis is based on this matched sample.

2.2 Quality of language environments The quality of practitioners’ language environments was assessed pre-‐ and post-‐qualification using an observation schedule based on the Better Communication Research Programme’s Communication Supporting Classroom Observation Tool, which assesses the quality of language environments on three dimensions: Language Learning Environment, Language Learning Opportunities, and Language Learning Interactions and elements from the Infant/Toddler and Early Childhood Environment Rating Scales (ITERS and ECERS). Observations were conducted by a variety of practitioners including speech and language therapists, educational psychologists and senior setting staff, with the same person completing both pre-‐ and post-‐qualification observations to support validity. The quality of the language environment was assessed in 89 practice settings pre-‐qualification and 14 practice settings post-‐qualification, yielding a matched sample of 10 practice settings (9% of all practice settings) with both pre-‐ and post-‐qualification data. We used this matched sample to assess changes in the quality of the language environment during the course of the Platform 3 programme.

2.3 Practitioners’ knowledge and confidence Learners completed an online survey before and after completing the Platform 3 programme. The survey was designed and administered by The Communication Trust, and responses were analysed by LKMco. The survey included a range of questions designed to rate learners’ own assessments of their knowledge and confidence. Confidence questions consisted of 5-‐category Likert scales, while self-‐assessed knowledge questions consisted of 10-‐category Likert scales. In addition, learners completed a pre-‐qualification knowledge test consisting of 18 questions, which was repeated in identical form post-‐qualification. Matching pre-‐ and post-‐qualification surveys yielded a sample of 103 learners (69% of all completers) for whom we were able to calculate confidence, knowledge and self-‐assessed knowledge scores. These form the basis of our analysis.

2.4 Working with parents to support their child’s speech language and communication The pre-‐ and post-‐qualification survey included a number of items specifically addressing learners’ confidence and knowledge in working with parents to support their children’s speech, language and communication, as well as asking learners to give free-‐text responses on how they gain information from parents when completing the 2-‐year



8

progress check. We have conducted quantitative and qualitative analysis of these survey data3. In addition to completing a survey, seven learners also took part in one-‐to-‐one phone interviews, conducted by Communication Trust staff, which included a question relating to their confidence when speaking to parents about how to support their child’s speech, language and communication. We have drawn on the notes from these interviews to add detail to our analysis. Finally, some learners chose Working with parents, families and carers to support their children’s speech, language and communication development as their third unit and we have broken down our analysis to assess any difference in outcomes between learners who did, and did not, take this unit.

2.5 Early identification of children with delayed speech, language and communication The pre-‐ and post-‐qualification surveys contained a range of items designed to assess learners’ knowledge, confidence and skill in identifying children who are struggling with their speech, language and communication, and we conducted our analysis on a matched sample of 103 responses. We also made use of qualitative accounts from learner case studies, which were conducted by Communication Trust staff during the course of the programme.

2.6 Effective use of the 2-‐year progress check The pre-‐ and post-‐qualification surveys assessed learners’ confidence in using the 2-‐year progress check as a basis to structure discussions with parents about their child’s progress, as well as their confidence in being able to provide a short summary of a child’s progress. As well as quantitative analysis of changes in learners’ confidence scores between the start and end of the programme, using a matched sample of 103 responses, we have also conducted qualitative analysis of free-‐text responses relating to specific areas of confidence, training needs, concerns regarding the completion of progress checks and the way in which learners used progress checks when liaising with other professionals.

2.7 Effectiveness of the online delivery model In order to assess the effectiveness of the online delivery model we draw on learners’ responses to questions in the pre-‐ and post-‐qualification survey relating to: learning materials; the eportfolio area; support from tutors; the benefits and challenges of completing the qualification online, and the quality of work produced. In our analysis we use qualitative analysis of free-‐text responses to supplement quantitative analysis of fixed-‐category responses, and we also break down our analysis by assessment centre where possible, in order to assess any differences in outcomes.

2.8 Differences between roles Where possible, we have broken down our analysis to consider whether the results of the programme vary between more and less senior practitioners. We define senior practitioners based on their job title as managers, SENCOs, leaders and coordinators.

2.9 A note on statistical significance During this impact evaluation when we claim that a change is statistically significant we mean that the difference is sufficiently sizeable to confidently rule out that the change may have been due to chance, given the size of our sample. We set our significance tests at a lower limit of p<.05, which means we can be at least 95% confident that a change was not due to chance.

3 Our quantitative analysis consists of paired-‐samples t-‐tests of mean differences in raw scores on confidence, self-‐assessed knowledge and knowledge test scores, both for aggregate scores and scores on individual questions/elements. Our qualitative analysis consists of coding free-‐text responses by theme in NVivo and, where appropriate, counting the frequency with which themes arise.



9

3 The impact of Platform 3

3.1 The quality of early years speech, language and communication provision

3.1.1 Number of practitioners qualified to Level 3 In total, 230 practitioners enrolled onto Platform 3, meeting the objective set by the Department for Education. These learners were arranged into eight different groups or ‘cohorts’, overseen by three approved centres and three awarding organisations. The cohorts had staggered start dates from April to November 2014. Across all learner cohorts 150 learners (65%) completed the course, although there was wide variety within this overall figure, with completion rates ranging from 23% to 93% as shown in Figure 2. Of the learners who did not complete: 20 cited extenuating circumstances; 32 were unable to complete as they were unable to submit sufficient evidence by deadlines; 8 learners moved jobs; 6 could not complete due to ill health and 14, although enrolled, did not respond to communications and so were deemed non-‐starters.

Figure 2

Awarding organisation

Approved centre Learner cohort Learners enrolled

Learners completed

Completion rate

Apt Awards Elklan Puffins 30 7 23% Essex 20 14 70%

CACHE Central Bedfordshire

Central Beds 1 30 28 93% Central Beds 2 35 32 91% Central Beds 3 35 31 89%

City & Guilds South Thames College

Lewisham 1 30 15 50% Lewisham 2 30 11 37% Barnardos/Essex 20 12 60%

230 150 65%

3.1.2 Quality of individual practice Individual practitioner observations provide firm evidence that the quality of individual practice improved during the course of the programme. Practitioners’ use of each of the ten positive interaction strategies in the observation schedule increased between the pre-‐ and post-‐qualification observations. As shown in Figure 3, this increase was statistically significant in relation to two specific elements of individual practice: demonstrating/modelling rather than criticising, and using language at roughly the same level as the child. Increases in the other eight positive interaction strategies fell short of statistical significance.



10

Figure 3

Pre-‐ and post-‐qualification notes from the individual practitioner observations4 give a more detailed picture of some of the improvements in individual practice that were made during the programme, as illustrated in Figure 4.

Figure 4

3.1.3 Quality of communication environments Across the 10 settings where both pre-‐ and post-‐qualification environment data were available, the average number of observed items relating to the three environmental dimensions increased. None of these increases were statistically significant, although with such a small sample it is less likely that changes will be large enough to be significant. Meanwhile, the average number of ‘unseen’ items decreased, although again, these decreases were not statistically significant. Due to a small matched sample and a limited number of observations of senior practitioners’ speech, language and communication environments, it was not possible to assess the link between practitioner seniority and changes in

4 Pseudonyms used

3"

4.7"4.5"

5.7"

0"

1"

2"

3"

4"

5"

6"

Demonstra4ng/modelling"rather"than"cri4cising"

Using"language"at"roughly"the"same"level"as"the"child"

Average'nu

mbe

r'of'recorde

d'ob

serva1

ons'

'per'prac11o

ner'

Matched'sample'n=25,'p<.05'

Significant'increases'in'elements'of'individual'good'prac1ce'

PreAqualifica4on"

PostAqualifica4on"

Excerpt from pre-‐qualification observation of Amy’s practice: “Amy tended to over-‐talk, rather than giving the children time to initiate interaction. She also asked too many questions, rather than making comments.” Excerpt from post-‐qualification observation of Amy’s practice: “Amy allowed the children time to respond and gave them the opportunity to initiate their own conversations with her. Amy also made comments about the children’s mark making, and when children began discussing their home stories Amy responded and extended on what the children were telling her.”



11

the quality of speech, language and communication environments. Given that senior practitioners are more likely to be in a position to affect changes in the language environments within their practice settings, it would be valuable in future evaluations of the Platform 3 programme to ensure a larger number of pre-‐ and post-‐qualification environment observations with senior practitioners are conducted, and we reiterate this point in our concluding recommendations. In summary:

• 230 practitioners enrolled onto Platform 3, meeting the objective set by the Department for Education • 150 learners (65%) completed the course, although there was wide variety between assessment centres • The quality of individual practice improved during the course of the programme, with statistically

significant increases in two out of eight elements of individual practice: demonstrating/modelling rather than criticising, and using language at roughly the same level as the child

• There were increases in the measures of the quality of communication environments between the start and end of the qualification, but these were not statistically significant

3.2 Practitioners’ effectiveness in working with children to support their speech, language and communication

3.2.1 Practitioners’ knowledge In the end of qualification survey, learners reported unanimously that they were more confident, knowledgeable and skilled in supporting the development of speech, language and communication. Responses to the pre-‐ and post-‐qualification survey showed a statistically significant increase in learners’ overall knowledge test scores, as shown in Figure 5.

Figure 5

As shown in Figure 6, increases in participants’ knowledge test scores were particularly large in relation to particular questions:

• "Which of the following is an example of speech?" • "Which of the following is an example of a child's language?" • "Which two of the following are the most important skills when identifying children with SLCN?" • "What advice/information is most accurate for parents wishing their child to be bilingual?"

74%$83%$

0%$

20%$

40%$

60%$

80%$

100%$

Pre.qualifica6on$ Post.qualifica6on$Matched(sample(n=103,(p<.001(

Knowledge(test(scores(



12

Figure 6

As well as their knowledge test scores, learners’ own subjective judgments of their knowledge also increased, as shown in Figure 7. There were significant increases in participants’ self-‐assessed knowledge scores on each of the individual questions in the pre-‐ and post-‐qualification questionnaire. The largest increase in learners’ self-‐assessed knowledge related to Q9 “how would you rate your knowledge of formal and informal assessment approaches that can be used to assess the speech, language and communication skills of children and young people?”

Figure 7

58%$

6%$

60%$ 61%$

85%$

28%$

79%$ 79%$

0%$

20%$

40%$

60%$

80%$

100%$

"Which$of$the$following$is$an$

example$of$speech?"$

"Which$of$the$following$is$an$

example$of$a$child's$language?"$

"Which$two$of$the$following$are$the$most$important$skills$when$idenDfying$children$

with$SLCN?"$

"What$advice/informaDon$is$most$accurate$for$parents$wishing$their$child$to$

be$bilingual?"$

Prop

or%o

n'of'learne

rs'answering'correctly

'

n=103'

Knowledge'test'ques%ons'with'the'largest'score'increases'

PreNqualificaDon$

PostNqualificaDon$

57%$

86%$

0%$

20%$

40%$

60%$

80%$

100%$

Pre.qualifica6on$ Post.qualifica6on$Matched(sample(n=103,(p<.005(

Self8assessed(knowledge(scores(



13

As well as indicating significant increases in learners’ knowledge, and their own appraisals of their knowledge between the start and end of the programme, the data also suggest that participants gained a more accurate sense of their own knowledge during the course of the qualification. Before undertaking the qualification there wasn’t a strong link between learners’ self-‐assessed knowledge scores and their knowledge test scores. On completing the qualification, however, their self-‐assessed scores and test scores were significantly more aligned5, as shown in Figure 8 and Figure 9. This indicates that during the course of completing the qualification learners gained a more accurate sense of their knowledge and skills in relation to speech, language and communication.

Figure 8

5 Pre-‐qualification correlation between self-‐assessed knowledge scores and knowledge test scores r=.117, p=.237. Post-‐qualification correlation between self-‐assessed knowledge scores and knowledge test scores r=.196, p=.047.



14

Figure 9

3.2.2 Practitioners’ confidence Responses to the pre-‐ and post-‐qualification survey also showed a statistically significant increase in learners’ ratings of their confidence, as shown in Figure 10.

Figure 10

There were significant increases in participants’ confidence scores on each of the individual confidence questions in the pre-‐ and post-‐qualification questionnaire, but the largest increases in confidence related to:

42%$

67%$

0%$

20%$

40%$

60%$

80%$

100%$

Pre-qualifica5on$ Post-qualifica5on$Matched(sample(n=103,(p<.001(

Confidence(scores(



15

• Q2 “how confident are you that you have good knowledge of typically developing speech, language and communication?”

• Q4 “how confident are you in being able to support children in your setting who are struggling with their speech, language and communication?”

The smallest increase in confidence related to using 2-‐year progress checks as a basis to structure discussions with parents about their children’s progress. However, the increase in confidence in this area was still statistically significant.

3.2.3 Differences between roles There were no significant differences, either pre-‐ or post-‐qualification, between the knowledge and confidence of childminders and other types of practitioner. However, differences were evident between practitioners of different levels of seniority. Managers, SENCOs, coordinators and leaders had significantly higher self-‐assessed knowledge and confidence before the programme, although they did not score significantly higher on the knowledge test. After the programme, these differences were no longer significant, as shown in Figure 11.

Figure 11

3.2.4 Working with parents to support their child’s speech, language and communication The pre-‐ and post-‐qualification survey included a number of items specifically addressing learner’s confidence and knowledge in working with parents to support their children’s speech, language and communication. Three questions addressed learners’ confidence in this area:

• Q6: How confident do you feel in your ability to discuss a child’s strengths, weaknesses and needs with their parent/carer?

• Q37: Do you feel confident in using your 2-‐year progress check as a basis to structure discussions with parents about their child’s progress?

• Q38: Do you feel confident in being able to support parents with how to encourage their child’s development at home?

Meanwhile two questions addressed learners’ knowledge in this area:

• Q20: What advice/information is most accurate for parents wishing their child to be bilingual? • Q30: Which of the following is not true when considering working with parents, carers and families to

support their children’s speech, language and communication development? Across all learners on the Platform 3 programme, there were significant increases in their confidence scores for each of the confidence questions relating to working with parents/carers to support SLC, as shown in Figure 12.

Self%assessed)knowledge)score

Knowledge)test)score

Confidence)score

Self%assessed)knowledge)score

Knowledge)test)score

Confidence)score

Managers 67% 74% 62% 87% 81% 95% n=21Others 54% 74% 51% 85% 83% 84% n=82Childminders 54% 77% 56% 86% 86% 91% n=17Others 58% 73% 52% 86% 82% 85% n=86

Seniority

Role

Pre3qualification Post3qualification



16

Figure 12

Likewise, the proportion of learners answering the two parent-‐related knowledge questions on the survey correctly also rose. The proportion responding correctly to question 20 increased from 61% to 79% of learners – a statistically significant change from pre-‐ to post-‐qualification. Meanwhile, the proportion of learners responding correctly to question 30 rose from 78% to 84%, although this change fell short of statistical significance. Alongside two core units studied commonly by all practitioners, all learners on the programme could choose their third unit of study from a menu of four options. As shown in Figure 13, the optional unit focusing on working with parents, families and carers was chosen by a relatively small minority (11%) of learners.

Figure 13

Third unit options Number of learners Understanding the speech, language and communication needs of children and young people with behavioural, social and emotional difficulties (BESD) 73 (49%) Supporting positive practice with children and young people with speech, language and communication needs 38 (25%) Supporting the speech, language and communication development of children who are learning more than one language 22 (15%) Working with parents, families and carers to support their children’s speech, language and communication development 17 (11%)

41%$

48%$44%$

64%$68%$

62%$

0%$

20%$

40%$

60%$

80%$

Confidence$in$discussing$a$child’s$strengths,$weaknesses$and$needs$

with$their$parent/carer$

Confidence$in$using$the$2?year$progress$check$as$a$basis$to$

structure$discussions$with$parents$about$their$child’s$progress$

Confidence$in$being$able$to$support$parents$with$encouraging$

their$child’s$development$at$home$

Average'confi

dence'score'across'all'respon

dents'

Matched'sample'n=87,'n=37'and'n=46'respec;vely,'p<.001'

Significant'increases'in'confidence'in'working'with'parents/carers'to'support'children's'SLC'

Pre?qualificaFon$

Post?qualificaFon$



17

Nonetheless, we were able to analyse differences in the confidence and knowledge of learners who had opted to take the ‘working with parents’ unit, and those who had not. As shown in Figure 14, those who took this optional unit had significantly higher post-‐qualification confidence scores in relation to question 6 – “How confident do you feel in your ability to discuss a child’s strengths, weaknesses and needs with their parent/carer?” – than those who did not take the unit. Other differences in the parent-‐related knowledge and confidence of practitioners who did and did not take the ‘working with parents’ unit were not statistically significant.

Figure 14

The pre-‐ and post-‐qualification surveys also asked learners to state, if applicable, how they gather information from parents to support with their progress checks. Qualitative analysis of these open-‐text responses revealed a broad distinction between three methods of gathering information from parents:

• Face-‐to-‐face methods (such as conversations at pick-‐up time; parents’ evenings) • Informal records (such as ad-‐hoc emails; comments boxes) • Formal records (such as progress forms; All About Me forms; registration forms)

A further analysis of practitioners’ responses, using these three categories, revealed that use of informal records remained broadly similar at the start and end of the programme, as shown in Figure 15. However, the data suggest that during the course of Platform 3 practitioners made more use of their records from observations and assessments to inform their conversations with parents about the child’s progress. Practitioners relied more on formal records to support them with progress checks, as opposed to more informal, ad-‐hoc face-‐to-‐face discussions with parents, even though this remained the most widely used strategy after the qualification.

62%$

70%$

40%$

60%$

80%$

'Working$with$parents'$op8onal$unit$not$taken$ 'Working$with$parents'$op8onal$unit$taken$

Average'confi

dence'score'

Matched'sample'n=101,'p<.05'

Post<qualifica@on'confidence'in'discussing'a'child’s'strengths,'weaknesses'and'needs'with'their'parent/carer'



18

Figure 15

As well as increasing practitioners’ confidence in discussing a child’s strengths, weaknesses and needs with parents, and gathering information from parents to support progress checks, phone interviews conducted with learners at the end of the programme revealed that Platform 3 had also encouraged practitioners to share resources and advice with parents to allow them to support their child’s speech, language and communication at home, as the excerpts in Figure 16 show.

Figure 16

In summary:

• Learners reported unanimously that they were more confident, knowledgeable and skilled in supporting the development of speech, language and communication at the end of the qualification

83%$

15%$

42%$

67%$

14%$

56%$

0%$

20%$

40%$

60%$

80%$

100%$

Face0to0face$ Informal$records$ Formal$records$

Prop

or%o

n'of're

spon

dents'm

en%o

ning'each'strategy'

n=53'pre7qualifica%on,'n=36'post'qualifica%on'

Strategies'for'gathering'informa%on'from'parents'to'support'with'progress'checks'

Pre0qualifica@on$

Post$qualifica@on$

Excerpts from phone interviews with learners relating to supporting SLC in the home “I try to involve parents more by sending the children home with activities that they can do with their parents.” “I’m able to talk to parents more about language development. I have strong bonds with the parents so I’m happy to talk to them about how they can support their child at home.” “I now feel more informative and confident and can pass on resources (the same that are used in the setting) for parents to use at home.”



19

• Between the start and end of the qualification there were statistically significant increases in learners’ overall knowledge test scores, their own assessments of their knowledge, and their confidence

• Learners had a more accurate sense of their knowledge and skills in relation to speech, language and communication by the end of the qualification

• Across all learners on the Platform 3 programme, there were significant increases in their confidence and knowledge relating to working with parents/carers to support children’s speech, language and communication

• Those who took the optional unit Working with parents, families and carers to support their children’s speech, language and communication development were significantly more confident in their ability to discuss a child’s strengths, weaknesses and needs with their parent/carer than those who did not take the unit

• At the end of the qualification, practitioners were more likely to use formal records to support them with progress checks, as opposed to more informal, ad-‐hoc face to face discussions with parents

3.3 Early identification of children with delayed speech, language and communication and effective use of the 2-‐year check

3.3.1 Early identification of children with delayed speech, language and communication In the end of qualification survey, more than 98% of learners reported that they felt more knowledgeable, confident and skilled in identifying children who are struggling with speech, language and communication than they had been before embarking on the qualification. As identified in section 3.2.2 above, matched responses from the pre-‐ and post-‐qualification surveys also suggest that the largest increases in practitioners’ confidence related to:

• Having good knowledge of typically developing speech, language and communication • Being able to support children who are struggling with their speech, language and communication

The learner case studies also provided evidence that individual practitioners’ ability to identify children whose language skills are not developing typically for their age had improved as a result of taking the course, as shown in Figure 17.

Figure 17

• Together, these findings suggest that Platform 3 impacted positively on practitioners’ confidence and ability in

the early identification of children with delayed speech, language and communication.

3.3.2 Effective use of the 2-‐year progress check As outlined in section 2 above, pre-‐ and post-‐qualification survey data show a statistically significant increase in practitioners’ confidence scores in relation to using the 2-‐year progress check as a basis to structure discussions with parents about their child’s progress. The survey data also show a statistically significant increase in practitioners’ confidence in being able to provide a short summary of a child’s progress.

Excerpt from learner case study relating to early identification of children with delayed SLC “I have always thought she was quite advanced for her age but the observations have made it clearer to me that her language is still quite simple.”



20

A small minority of practitioners (12% in both the pre-‐ and post-‐qualification survey) felt there was one particular area in which they felt more confident in terms of observing/recognising skills of the children they work with. Before the qualification, the majority (7 out of 11) of these practitioners felt most confident at recognising physical development, with a minority (4 out of 11) citing communication and language. After the qualification, all twelve of the practitioners who stated they had a particular area of confidence identified this as communication and language. Moreover, a number of these practitioners linked their confidence in this area specifically to their completion of Platform 3, as shown in Figure 18.

Figure 18

This final excerpt demonstrates how some practitioners felt their increased knowledge of children’s speech, language and communication development had allowed them to approach the 2-‐year progress check with greater confidence, and Figure 19 illustrates how this sentiment was mirrored in the phone interviews conducted with learners.

Figure 19

Before and after the qualification, a small minority of learners stated that they had specific training needs in terms of being able to complete the 2-‐year progress check to the best of their ability. This fell from 10% to 4% of respondents between the pre-‐ and post-‐qualification survey. The specific training needs respondents cited included:

• Improving their observation skills • Refresher training in the case of changes to requirements • Cross-‐checking/validating against other practitioners’ progress judgments

Excerpts from survey responses relating to practitioners’ confidence in observing/recognising communication and language development “I feel most confident at recognising communication and language development, because of the knowledge and confidence I've gained from this course.” “The recent course is foremost in my mind and my practice and confidence has changed considerably.” “I now have a broader knowledge of what to look for and where a child’s expected age and stage of development should be. This particular area has always been easier for me as I have a wider interest, but now I feel I can explain how this links with other areas of development more clearly too.”

Excerpt from phone interview relating to use of the 2-‐year progress check “I feel a lot more confident when writing about SLC development and how the children are progressing in relation to their age. I look at the progress check in more detail than I used to, as I now have more understanding of the areas of development for different ages.”



21

Learners were also given an opportunity to state any concerns they had about completing progress checks. The proportion of learners with specific concerns fell from 6% before the qualification to 2% at the end of the qualification, and the concerns cited included:

• Appropriate wording • The level of detail required • Maintaining objectivity • Parents’ responses when needs are identified

A sizeable majority of practitioners reported that they used progress checks as a way to identify children who may require further support in one or more areas of their development, rising slightly from 89% pre-‐qualification to 95% post-‐qualification as shown in Figure 20. Alongside a slight increase in practitioners using the check however, there was a slight decrease in their propensity to see progress checks as a ‘key’ tool.

Figure 20

The pre-‐ and post-‐qualification survey also gathered free text responses from learners relating to the way in which they use the 2-‐year progress check to liaise with other professionals who may be working with a child. Qualitative analysis of these responses revealed one main dimension of variation: some respondents reported that they used progress checks as a basis for liaising directly with other professionals, while others reported that they passed the findings of progress checks to parents (for instance, via their red books) who then liaised with other professionals. Analysing pre-‐ and post-‐qualification responses using this distinction suggests that after having completed the programme practitioners were relatively more likely to use progress checks as a basis for liaising directly with other professionals, as opposed to doing so via the parents of the children they work with.

23%$

67%$

9%$

2%$

17%$

78%$

5%$0%$

0%$

20%$

40%$

60%$

80%$

100%$

Yes,$it$is$one$of$the$key$ways$I$use$to$iden>fy$children$who$may$need$

further$support$

Yes,$although$there$are$also$several$other$tools$I$

use$to$help$iden>fy$children$who$need$further$

support$

No,$but$I$would$like$to$be$able$to$know$more$about$how$to$use$the$progress$check$to$help$with$early$iden>fica>on$of$needs$

No,$I$don't$think$the$progress$check$is$useful$in$

this$regard$

Prop

or%o

n'of're

spon

ses'

n=58'

"Do'you'use'the'progress'checks'as'a'way'to'iden%fy'children'who'may'require'further'support'in'one'or'more'areas'of'their'development?'

PreKqualifica>on$

PostKqualifica>on$



22

Figure 21

Taken together, the data suggest that practitioners may have shifted towards liaising more directly with other professionals because they were equipped with better knowledge of speech, language and communication development, which gave them greater confidence to expose their judgments to professional scrutiny. This link was voiced by a practitioner when they expressed their concerns about completing progress checks during the pre-‐qualification survey (see Figure 22).

Figure 22

In summary: • More than 98% of learners reported that they felt more knowledgeable, confident and skilled in

identifying children who are struggling with speech, language and communication than they had been before embarking on the qualification

• The largest increases in practitioners’ confidence related to having good knowledge of typically developing speech, language and communication, and being able to support children who are struggling with their speech, language and communication

• There was a statistically significant increase in practitioners’ confidence in using the 2-‐year progress check as a basis to structure discussions with parents about their child’s progress

55%#

28%#

63%#

14%#

0%#

20%#

40%#

60%#

80%#

Liaise#directly# Liaise#via#parents#

Prop

or%o

n'of're

spon

ses'

n=41'

"How'do'you'use'the'progress'checks'to'liaise'with'other'professionals'who'may'be'working'with'a'child?"'

Pre:qualifica>on#

Post:qualifica>on#

Excerpt from pre-‐qualification survey relating to use of the 2-‐year progress check “I wonder if I am suitably qualified to [complete progress checks] in terms of being objective. I am in two minds about it: on the one hand I have the advantage of knowing the child well and can take my time with observations. On the other hand, a health visitor may spot something I missed or have a different way of assessing progress outside the child's comfort zone.”



23

• There was also a significant increase in practitioners’ confidence in being able to provide a short summary of a child’s progress

• At the end of the qualification, practitioners were less likely to have specific concerns about the 2-‐year progress check, or to see it as a specific training need

• The data suggest that after the qualification practitioners were more likely to use the 2-‐year progress check to liaise directly with other professionals, and that this was due to greater knowledge and confidence of speech, language and communication development

4 The effectiveness of the online delivery model

4.1 Learning materials As shown in Figure 23 a large majority of learners (more than 90%) rated the content, presentation and flow of the Platform 3 learning materials as good or excellent, including more than 40% of learners who rated the content of the learning materials as excellent. Ratings of the ease of use and navigation of the learning materials were slightly less positive, with 71% judging them to be good or excellent, 20% (13) judging them to be satisfactory, and 9% (6) judging them to be poor on this dimension. • As we flag up in the recommendations, while the learning materials appear to be very positively received on

the whole, their ease of use and navigation may be an area for future improvement.

Figure 23

Sixteen learners left additional feedback in their surveys on the Platform 3 learning materials. 6 comments related to miscellaneous technical issues, including: broken links; issues with printing; logouts, and the use of editable PDFs – an issue which was resolved during the course of the programme. Some issues were beyond the control of the course, such as internet connectivity problems.

51%$

59%$63%$

43%$43%$

34%$28%$ 28%$

6%$ 8%$ 8%$

20%$

0%$ 0%$ 2%$

9%$

0%$

20%$

40%$

60%$

80%$

Learning$materials:$content$

Learning$materials:$presenta;on$

Learning$materials:$flow$

Learning$materials:$ease$of$use/naviga;on$

Prop

or%o

n'of're

spon

dents'

n=65'

Prac%%oner'ra%ngs'of'Pla4orm'3'learning'materials'

Good$

Excellent$

Sa;sfactory$

Poor$



24

Four comments related to the need for the structure of the learning materials to be made clearer. This included: signposting what topics and themes were coming up in advance; better systems for learners to keep track of their progress/completed tasks, and linking units more clearly to learning outcomes, as illustrated in Figure 24. These issues are more likely to arise for learners who go straight to the end-‐of-‐unit activity/portfolio task and refer back to the learning materials as required, as opposed to those who work through the materials sequentially and then complete the end-‐of-‐unit task. The post-‐qualification survey indicates that more than three quarters (77%) of learners worked through the learning materials sequentially, while 18% tended to go straight to the portfolio task and 4% went straight to activities.

Figure 24

A quarter of those who gave feedback on the learning materials left a positive comment, including some practitioners who had not previously studied online, as shown in Figure 25.

Figure 25

4.2 The eportfolio area Learners’ ratings of the eportfolio area were slightly less positive than those for the learning materials, although the majority of eportfolio elements were rated ‘good’ or ‘excellent’ by more than two thirds of learners, as shown in Figure 26. The most critical judgements were directed at the ease of use/navigation of the eportfolio area, and the forums: these elements were rated ‘poor’ by 11% and 19% of learners respectively.

Excerpt from post-‐qualification survey relating to the Platform 3 learning materials “It wasn't clear enough what was linked to which learning outcome and what criteria you were trying to meet. I felt you had to click between things to verify constantly.”

Excerpts from post-‐qualification survey relating to the Platform 3 learning materials “I found it a bit of a challenge as I am so used to using textbooks rather than online materials. However, it was a very good introduction to me.” “Everything I needed seemed to be there and easy to find.” “Once I found my way around the lessons it all made sense.” “Loved it.”



25

Figure 26

Learners commented that the forums were not brought to their attention sufficiently early in the course, and that as a result this resource was underused. • As we state in the recommendations, the takeaway message here is therefore not that the forums were of a

poor quality, but that they were not adequately signposted. Meanwhile, learners’ critical feedback in relation to the ease of use/navigation of the eportfolio area tended to cluster around three themes:

• Difficulties uploading documents to the file store • Difficulties handling editable PDFs • Lack of notification when feedback from their tutor had been posted

4.3 Support from tutors Overall, learners seemed to rate the support they had received from their tutors highly, with around 70% rating the overall support from their tutor, their tutor’s feedback on marked work, and their tutor’s flexibility and ability to meet their individual needs as ‘good’ or ‘excellent’. However, these ratings diverged significantly between learners from different centres. On some dimensions more learners felt their tutors had been ‘satisfactory’ or ‘poor’ than ‘good’ or ‘excellent’. As we state in the recommendations, this between-‐centre variation in tutors’ performance warrants further examination. Learners’ responses to the post-‐qualification survey suggest communication issues may have been part of the explanation: while 95% of learners taking the course through one approved centre reported that their tutor had maintained good communication through the Platform 3 discussion area, this fell to 85% and 79% among those taking the course through the other two centres. Meanwhile, while 85% of learners taking the course through this centre felt their tutor had maintained good communication by phone, only one of the respondents from the other

10%$

19%$

19%$

29%$

40%$

25%$

34%$

20%$

50%$

47%$

52%$

48%$

37%$

58%$

49%$

65%$

0%$ 20%$ 40%$ 60%$ 80%$ 100%$

Forums$

Ease$of$use‎/naviga=on$

Online$help$and$user$guides$

Uploading$porDolio$evidence$

Contac=ng$your$tutor$using$the$discussions$area$

Using$the$file$store$

Accessing$feedback$on$porDolio$evidence$

Look$and$feel$

Propor%on'of'respondents'(n=65)'

Prac%%oner'ra%ngs'of'the'epor6olio'area'

Excellent$

Good$



26

two centres responded likewise. Due to a limited sample size this data cannot support firm conclusions, but does suggest that while tutors’ communication was good in many instances, this was not replicated across all centres.

4.4 Benefits and challenges of completing the qualification online

4.4.1 Benefits of completing the qualification online When asked how they had found completing the qualification online beneficial, the vast majority of learners referred to the flexibility of this mode of learning. Two types of flexibility were cited: the ability to complete the course in their own time and location, to fit around work and family commitments, and the ability to complete the course at their own pace. Learners also stated that the online nature of the course had entailed less travelling and fewer travel costs. Some excerpts from learners’ feedback relating to the flexibility of completing the qualification online are shown in Figure 27.

Figure 27

Learners also cited benefits other than flexibility. Some expressed that the course had offered them the only feasible way of gaining the qualification:

“I work long hours and have a young family and no car so it's the only way I could have done it.” Others stated that they had been able to put their learning into practice with immediate effect:

“I was able to put theory into practice straight away with the children I work with.” Finally, the experience of Platform 3 had motivated some practitioners to pursue further online training in future, as shown in Figure 28.

Excerpts from post-‐qualification survey relating to the flexibility of completing the qualification online “I was able to work at my own pace but to a clear deadline. Having not studied for many years it was a good way to reintroduce myself to study. I was able to study around my working hours and family commitments.” “As a childminder it was beneficial, as I could complete in my own time and fit around work commitments and my own family.” “It was great as I could work in my own time (after work) and when the kids go to bed etc. It worked really well as I was able to fit it around my commitments. It also gave me the opportunity to learn at my own pace and I thoroughly enjoyed the course.”



27

Figure 28

4.4.2 Challenges of completing the qualification online As well as discussing the benefits of having completed the qualification online, learners were also asked about the challenges of completing the qualification in this way. Qualitative analysis revealed three broad types of challenge, as shown in Figure 29.

Figure 29

Time pressures The most frequently-‐cited challenge was time pressures, as exemplified by one respondent: “Finding the time to do the work, it was hard to fit it in around everything else, if you were going to a college you

would put that time to having to go out and study, where as online other things could take over.”

Taken together with the excerpts from Figure 27, it appears that practitioners appreciated the ability to work flexibly, but that completing the qualification successfully relied on their ability to work independently and be well organised.

41#

29#

23#

0#

10#

20#

30#

40#

50#

Managing#.me# Technical#issues# Ge8ng#support#

Freq

uency)of)m

en-o

ns)

Total)responses=100)

Challenges)of)comple-ng)the)qualifica-on)online)

Excerpts from post-‐qualification survey relating to practitioners’ motivation to undertake further online training “I am feeling more positive to pursue more online training in the future to further my knowledge in childcare.” “It is a very good way of completing a qualification and I would do one again given the opportunity.”



28

Technical issues The next most frequently-‐cited challenge was technical issues. In some cases learners raised specific issues relating to:

• Uploading work • Switching between units • Navigating the content • Browser compatibility

In other cases, learners had broader difficulties accessing and using the content of the course due to having limited IT skills although, as the following excerpt demonstrates, some learners saw the course as an opportunity to develop these skills:

“My computer skills have really improved so for that I'm very grateful.”

Support Finally, learners cited the issue of support as a challenge to completing the qualification online. In some cases they cited delays in hearing back from their tutors, and the difficulties of adapting to a learning style where questions could not be answered immediately:

“Not being able to clarify certain questions straight away as you would in a classroom e.g. ‘am I on track here or way off the mark?’ You really have to be an independent learner.”

In other cases learners cited the absence of ‘classmates’ to discuss ideas with: “It would have been beneficial for something like a forum being set up so other learners could have discussed the course and units which would have helped us debate ideas and reflect on them in practice in different settings.”

The data suggest a link between centres’ completion rates and the proportion of their learners raising the issue of support as a challenge to completing the qualification online: learners from centres with higher completion rates were markedly less likely to raise the issue of support. Returning to the findings in section 4.2, this final challenge could be tackled by better signposting of the Platform 3 forum, which may help to establish a greater sense of learning community for those taking the course.

4.5 Quality of work produced In the post-‐qualification survey 95% of learners stated they felt happy with the quality of work they had produced during the course. There was some between-‐centre variation in this figure, but these differences were not statistically significant. When learners were asked if there was anything that would have helped them to improve the quality of their work, the most frequently cited factor was additional time, followed by more timely and frequent feedback and, finally, the provision of paper resources to reduce the reliance on screen reading. In summary:

• More than 90% of learners rated the content, presentation and flow of the Platform 3 learning materials as good or excellent. Ratings of the ease of use and navigation of the learning materials were slightly less positive, with 71% of learners judging them to be good or excellent



29

• Learners’ ratings of the eportfolio area were slightly less positive than those for the learning materials, although the majority of eportfolio elements were rated ‘good’ or ‘excellent’ by more than two thirds of learners

• The most critical judgements were directed at the ease of use/navigation of the eportfolio area, and the forums: these elements were rated ‘poor’ by 11% and 19% of learners respectively

• Learners’ feedback suggests the forums were a valuable resource, but were not adequately signposted • Around 70% of learners rated the overall support from their tutor, their tutor’s feedback on marked work,

and their tutor’s flexibility and ability to meet their individual needs as ‘good’ or ‘excellent’. However, these ratings diverged significantly between learners from different centres

• Practitioners appreciated the ability to work flexibly, but they felt that completing the qualification successfully relied on their ability to work independently and be well organised

• Technical issues and tutor support were also raised as challenges to completing the qualification online • 95% of learners stated they felt happy with the quality of work they had produced during the course

5 Conclusions and recommendations

5.1 Conclusions This evaluation found clear evidence that the 2013-‐15 Platform 3 pilot impacted on all three of its desired outcomes. Our analysis found significant increases in practitioners’ knowledge, confidence and skills during the course of the programme, alongside improvements in individual practice and language environments, practitioners’ ability to identify children with delayed speech, language and communication and to make effective use of the 2-‐year progress check. Learners felt unanimously that they were more confident, knowledgeable and skilled in supporting the development of speech, language and communication after having completed the course. The evaluation also considered the effectiveness of the online delivery model. Despite some variation in experiences between assessment centres, particularly relating to the quality of tutor support, alongside some difficulties navigating the online units, learners rated the learning materials highly and found that the online mode of delivery had enabled them to gain a qualification they may otherwise not have been able to undertake.

5.1.1 The quality of early years speech, language and communication provision • 150 practitioners completed the course, gaining the Level 3 qualification Supporting children and young

people’s speech, language and communication • The quality of individual practice improved during the course of the programme, with significant

improvements in two specific areas: demonstrating/modelling rather than criticising, and using language at roughly the same level as the child

• The quality of practitioners’ language environments also showed signs of improvement during the course, with greater evidence of positive environmental features, and fewer instances where good practice could not be evidenced -‐ particularly in relation to language learning interactions

• Learners reported unanimously that they were more confident, knowledgeable and skilled in supporting the development of speech, language and communication after having completed the course. There were significant increases in learners’ overall knowledge of speech, language and communication, their own assessments of their knowledge, and also in their confidence

5.1.2 Practitioners’ effectiveness in working with children to support their speech, language and communication • There were significant increases in learners’ confidence in working with parents/carers to support speech,

language and communication, with their knowledge in this area also increasing during the course. Outcomes were particularly positive for learners who took the optional unit Working with parents, families and carers to support their children’s speech, language and communication development



30

• During the course of the programme practitioners became less likely to rely on face-‐to-‐face strategies, and more likely to rely on formal records when gathering information from parents to support progress checks

5.1.3 Early identification of children with delayed speech, language and communication and effective use of the 2-‐year check

• More than 98% of learners reported that they felt more knowledgeable, confident and skilled in identifying children struggling with speech, language and communication than they had been before taking the qualification. The largest increases in practitioners’ confidence related to their knowledge of typically developing speech, language and communication and their ability to support children who are struggling with their speech, language and communication

• There were significant increases in practitioners’ confidence in using the 2-‐year progress check as a basis to structure discussions with parents about children’s progress, and being able to provide a short summary of a child’s progress

• After completing the programme practitioners were relatively more likely to use progress checks as a basis for liaising directly with other professionals rather than via parents

5.1.4 The effectiveness of the online delivery model • Over 90% of learners rated the content, presentation and flow of the Platform 3 learning materials as good

or excellent. Ratings of the ease of use and navigation of the learning materials were slightly less positive, with 71% judging them to be good or excellent

• Overall, learners rated the support they had received from their tutors highly. However, these ratings diverged significantly between learners from different assessment centres

• Learners reported that the main benefit of completing the qualification online was the flexibility this gave them to study in their own time and fit their learning around work and family commitments. Meanwhile, time pressures were cited as the biggest challenge of studying in this way

• 95% of learners stated they felt happy with the quality of work they had produced during the course

5.2 Evaluation recommendations • A larger sample of environment observations with senior practitioners would allow the impact of changes in

individual practice on setting environments to be assessed • Although this evaluation was able to call on a range of free-‐text response data from surveys, alongside notes

from phone interviews and learner case studies, richer qualitative data would allow a more detailed analysis of how learners’ experiences of the programme feed into top-‐level outcomes. Instruments that gather rich data on a sample of learners’ experiences throughout the programme, such as learning journals, would achieve this

• The provision of settings data (for instance from the EYFS and Ofsted) would allow the performance of the programme in different settings/against different individual child baselines, to be evaluated

• To maintain the objectivity of future evaluations we would recommend using an independent evaluator to conduct focus groups and interviews with learners and tutors

5.3 Programme recommendations • While the Platform 3 learning materials appeared to be very positively received on the whole, their ease of

use and navigation may be an area for future improvement • Feedback from learners suggests the forums could be better advertised at the start of the course. In general,

the forums appeared to be a potentially valuable but underused resource • There was marked variation in learners’ ratings of their tutors between the three assessment centres, with

tutors’ communication appearing to be a particular issue. The factors behind this variation would benefit from more detailed examination

• A number of learners appeared to be waiting longer than expected for feedback from their tutors. Although initial guidance on waiting times and tutor response times was given, this may need to be made clearer



31

• Given that a number of learners appeared to struggle with basic IT skills, it may be beneficial to signpost some initial basic training in this area before the start of the course

platform$3$2013715pilot$evaluation$ - communication trust ·...

Documents