welcome to the national flagship end-point assessment conference · 2019. 12. 20. · institute for...
TRANSCRIPT
Welcome to the national flagship
end-point assessment conference
Tom Bewick
Chief Executive
Federation of Awarding Bodies
Headline SponsorsHeadline Sponsors
Sam Taylor
Head of the Assessors Guild
Belonging to a community of professionals
Belonging to a community of professionals
Professional Growth
Expert support and seasoned professional guidance
Access to specialists
Valuable networking
Strength in numbers
Leads and sector visibility
Attention to industry news
Finally…a sense of belonging
Paul Eeles
Chief Executive
Skills and Education Group
Chair of Federation of Awarding Bodies
Nikki Christie
Deputy Director Assessment
Institute for Apprenticeships and Technical
Education Assessments and EQA
Nikki Christie
EPA 2019 - Quality &
Independence in Assessment
13 June 2019
Institute’s Role• Development and approval of new apprenticeship
standards, including making recommendations to Ministers on the appropriate funding level.
• Assurance of End Point Assessment (EPA) and Endpoint Quality Assurance (EQA).
• Development and approval of the content of new T Levels, and the procurement and contract management for T levels.
Standards vs Framework Starts
% Starts by Level Standards vs Frameworks
0%
20%
40%
60%
80%
100%
Full year 2017/18 Aug-Dec 2018/19 Full year 2017/18 Aug-Dec 2018/19
Standards Frameworks
Higher (L4+)
Advanced (L3)
Intermediate (L2)
Reviews and Revisions
Scope of Digital Pilot Review Standard Publication date
ST0114 - Cyber Intrusion Analyst (L4) 12/11/2014
ST0124 - Cyber Security Technologist (L4) 11/05/2016
ST0118 - Data Analyst (L4) 09/12/2014
ST0119 - Digital & Technology Solution Professional (L6)
12/03/2015
ST0125 - Infrastructure Technician (L3) 09/12/14, updated 09/12/16
ST0117 - IS Business Analyst (L4) 31/03/2017
ST0127 - Network Engineer (L4) 12/11/14, updated 11/02/16
ST0116 - Software Developer (L4) 12/11/2014
ST0128 - Software Development Technician (L3) 01/09/2016
ST0129 - Software Tester (L4) 12/11/2014
ST0130 - Unified Communications Technician (L3) 01/09/2016
ST0131 - Unified Communications Trouble Shooter (L4)
09/12/2014
Pilot: Digital Review launched 20 Sept 2018. Experience will inform future reviews
Risk-based approach: pre-IFA standards
Principles: • Employer Led Throughout• Consultative at Every Stage• Comprehensive and Thorough• Result in Betterment• Sensibly Paced
Content Driven: • Focus on standard content; • Revise EPA plan and funding depending on
changes and process changes through BAU approvals
12 standards in scope
Standard Recommend
IS Business Analyst (4)(Rename Digital Product analyst/ Digital Business Analyst)
Revise (broaden & reframe)
Unified Communications Trouble Shooter (4) Incorporate UCTS into a revised NE and infrastructureWithdraw UCTS
Network Engineer (4)
(Rename Network & Infrastructure Engineer)
Software Tester (4) Revise and relevel to L3
Digital & Technology Solutions Professional (6) Retain & revise (ensure options are distinct occupations).
Standard Recommend
Cyber Intrusion Analyst (4) CIA withdrawn after incorporating into a revised CST
Cyber Security Technologist (4)
Infrastructure Technician (3) UCT withdrawn after incorporatinginto a revised IT
Unified Communications Technician (3)
(Combined into one Infrastructure Technician standard)
Data Analyst (4) Retain & revise
Software developer (4) Retain & revise
Software Development Tech’n (3) Retain & revise
Recommendations
26
Quality
The Quality Alliance
Quality Assurance of Apprenticeships
Institute: Approval and publication
(through RPs and Board) Training Provider End Point
Assessment Organisation
IfATE (Open Awards) (48%)Employer Groups & Professional Bodies (29%)Ofqual (14%)QAA (9%)
External Quality Assurance (EQA)
Regulation
Institute
Institute
EQA Service Delivery Trailblazer Choice
End Point Assessment
DeliveryEmployer Choice
Standard Delivery
Employer Choice
Development of Standard with
Assessment Plan
Trailblazer
Framework Principles
Theme Description
The ‘what’
The EPA
Relevant
• Construct validity – EPA measures occupational competence
• Predictive validity – EPA predicts success in the occupation after passing the
apprenticeship
ReliableThe EPA produces the same results (i.e. the right grades & results) across different
times, cohorts, EPAOs etc
The ‘how’
The process
Efficient The system is easy to use and facilitates entry. The right activity is undertaken by the
right people, at the right time, enabled by the right digital solution
PositiveEPA is trusted and respected by employers and apprentices. The experience is open,
transparent and accessible.
LearningContinuous improvement is embedded in all areas of the framework to ensure there is
a proactive approach to evolving
Assurance that EPA is both relevant and reliable: that it is the right assessment of occupational competence, done in
the right way.
Framework built around 6 key areas
1. Setting out the right assessment approach via clear end-point assessment plans
2. Ensuring that only high quality organisations can deliver assessment via the RoEPAO
3. Ensuring that EPAOs are ready to deliver when required to, through readiness checks
4. Ensuring that EPA delivery is high quality through a programme of EQA audits and visits
5. Taking action when things go wrong
6. Feedback looped back through the system to drive improvements in:• End-point assessment plans• The EPAO market• End-point assessment delivery
Implementation
• Institute’s EQA Framework will be live from 1st July 2019
• All EQA providers will transition onto the Framework over the summer
• Started with Open Awards (during June)
• On-boarding EQA providers onto the digital system in fortnightly tranches
• We will be on-boarding UKFT, NSAR, People 1st, Skills for Care, CITB, RMISC, Academy of Healthcare Science over the summer.
• Then the remaining ‘live’ EQA providers during early Autumn.
• We will on-board EQA providers who aren’t currently ‘live’ at the appropriate time for them.
THANK YOU
Gary Tovey Head of Assessment Services Babcock Assessments
Putting the learner and quality at the
heart of end-point assessment
babcockinternational.com/assessments
Putting the learner & quality at the heart of EPA
Gary Tovey Head of Assessments Services, Babcock Assessments & Fellow, The Assessors Guild
Hello!
Babcock Assessments is an independent part of the
Babcock International Group, a FTSE 250 company with a
heritage of designing workforce development solutions that
put the customer brand at the forefront of delivery.
We operate two brands:
• Babcock Assessments
• Babcock Engineering Assessments
• An established record of delivering End Point Assessment
• Developed capability and capacity and an approved End Point Assessment Organisation for 17
standards across a range of sectors
• A trusted partner to protect the customers reputation and deliver quality EPA outcomes for
apprentices
• Multiple successful EPA mobilisation to delivery implementations already completed
• No areas for improvement and good practice identified from EQA monitoring visits
• Successful first time applications via latest REPAO questions
• Fully mobilised
• Commitment and ready to deliver
• Multiple clients and industry sectors
Working with employers and providers nationally
Some of our clients include:
Our approach
Face-to-face End Point Assessment delivery to ensure
quality experience for the apprentice
Planning meetings to support the
apprentice understand EPA
expectations
Pre-EPA & gateway
support, guidance,
mock assessment
Dedicated, full-time
End Point Assessors
Invigilation provided
by Babcock
Assessments
Dedicated
Assessment Internal
Quality Assurance
team
babcockinternational.com/assessments
Our EPA journey so far…
1 2 3 4 5
Setup Mobilisation Success Challenges Future
Set up….
Clean sheet….
• Opportunity
• New operating unit
• New team and culture
• New processes
• New policies and procedures
• New P&L
• New ways of working
• New internal understanding – independence
• New external understanding - independence
Alison Pearce
Quality Delivery Manager Apprenticeships
BCS - How to maintain the quality of assessors
BCS Approach to Recruitment
▪ BCS Professional Members
▪ Sector experience in recent working career
▪ Supporting new talent
▪ Assessment experience is desirable
▪ Independent, fair, inclusive
▪ Selection and Assessment days
bcs.org 49
BCS Approach to Training
51bcs.org
▪ Two Day Assessment Event
▪ Overview
▪ Standards v frameworks
▪ Considerations
▪ Objectivity not subjectivity
▪ Grading
▪ Feedback
▪ Mentoring and observation
BCS Approach to Quality
52bcs.org
▪ Moderation
▪ Maintaining the standard
▪ Feedback
▪ Grading
▪ Rating
▪ Standardisation
▪ Generic
▪ Cross standard
▪ Standard specific
▪ Training Providers
▪ Webinars
▪ Top Tips
▪ Guides
THANK YOU
Genevieve PotterGeneral Manager
The Bauer Academy
End-point assessment from an employer/provider’s perspective
End point assessment from an employer/provider’s perspective
Genevieve Potter, General ManagerThe Bauer Academy
WE ARE… A UNIQUE UK ENTERTAINMENT NETWORK
94 MAGAZINE BRANDS
80 RADIO STATIONS
07 DIGITAL TV CHANNELS
100+ WEBSITES
300+ SOCIAL CHANNELS
Our Apprenticeship story so far
• We didn’t deliver Apprenticeship Frameworks• Bauer Media and external clients - ITV, the7Stars, Space&Time, Sue
Ryder, Double Negative, plus levy transfer project with Arqiva• Primarily upskill programmes• Premium, learner first delivery model • Changing the language of Apprenticeships (DDT not off the job)
Our Portfolio:
• Junior Content Producer level 3 (Brilliant Story-Telling)• Digital Marketer level 3• Sales Executive level 4• Broadcast Production Assistant level 3• Junior Journalist level 3• Advertising & Media Executive level 3• Team Leader/Supervisor level 3 (Effective Leadership)
• NEW: Bauer Senior Leaders Masters Degree Level 7 Apprenticeship with De Montfort University (Sept 2019)
End Point Assessment - what we have learnt:
❑ Provider shoulders the responsibility (not employer)❑ Digital solutions are no substitute for good account management❑ Transparency is important❑ Flexibility is key ❑ Detail is essential❑ Good communication is more important than all of the above!
End Point Assessment – what good looks like?
❑ Main point of contact who is across all of the logistics ❑ Clear timeline❑ Gateway forms, matrix grids for mapping evidence/portfolios, ❑ Lots of past papers and useful resources❑ Clear quality processes - IQA❑ Work with us on how we can demonstrate learners’ KSBs and showcase evidence❑ Come and see our learners and delivery team ❑ Detailed and constructive feedback in final grading report - ‘isn’t showing enough of
the competencies’ – which ones?❑ Adapt to your audience❑ Communicating the results – who, how, when?
And finally ……. A good news story
Tom Bewick
Chief Executive
Federation of Awarding Bodies
Making EQA Work
Alex Morris, Head of Apprenticeship Quality Assurance, Institute for Apprenticeships and Technical Education
Steve Walker, Associate Director – Apprenticeships, Ofqual
Neil Cruickshank, Head of External Quality Assurance, People1st International
Barry Smith, Head of Assessment, National Skills Academy for Rail
Heather Akehurst, CEO, Open Awards
Alex Morris
Head of Apprenticeship Quality Assurance
Institute for Apprenticeships and Technical Education
Steve Walker
Associate Director - Apprenticeships
Ofqual
Neil Cruickshank
Head of External Quality Assurance
People1st International
www.people1st.co.ukwww.people1st.co.uk
External quality assurance Neil Cruickshank, Head of External Quality Assurance
www.people1st.co.uk
www.people1st.co.uk
Overview
• Approved as EQA provider since February 2018.
• Operate an employer-led quality model.
• Four sector focus – hospitality, retail, aviation and travel.
• 26 end-point assessment organisations.
• 21 apprenticeship standards (10 in development).
www.people1st.co.uk
Approach to date
• Just over half EPAOs are actively assessing.
• Various states of readiness.
• Quality model: Plan – Do – Review.
• Three phased approach:o Introductory.o Readiness to deliver.o Actively delivering.
www.people1st.co.uk
Readiness for end-point assessment
• Early dialogue with EPAOs.
• Responding to assessment plan queries and clarifications.
• Disseminate guidance to all approved EPAOs.
• Provide specific feedback and review assessment material.
• Undertake an initial monitoring activities.
www.people1st.co.uk
Communication
• Dedicated secure web based portal:o Supports EQA work flows.o Retains resources.o Points of contacts: EPAO – People 1st.
• Communication approach:o Remote.o Face to face. o Progress meetings. o Regular EPAO forums.o e mail updates.
www.people1st.co.uk
Emerging themes-readiness
• High level of engagement with EPAOs.
• Responsive to feedback and support provided.
• Open to discuss approaches and practice.
• Focus on interpretation and clarification of assessment plans.
www.people1st.co.uk
Areas of focus of readiness
• Design and development of assessment methods.
• Assessor competence and capacity.
• Internal quality assurance.
• Reporting and management systems.
www.people1st.co.uk
People 1st International
Hospitality House
11-59 High Road
London N2 8AB
United Kingdom
Tel: 020 3074 1222
www.people1st.co.uk
@p1stgroup
People 1st International
Barry Smith
Head of Assessment
National Skills Academy for Rail
STANDARD COMPLIANCE & READINESS
NSAR:DIGITAL-LOGISTICS-RAIL
Barry Smith – Head of Assessment
PROVIDER APPROVALINSIGHTS - DIGITAL
• A good news story with quality
and momentum building
• Gateway and prescriptive sets of
requirements
• Employer references in
assessment
• Grading profiles
• Comparability
https://www.nsar.co.uk/eqa/
PROVIDER APPROVALINSIGHTS - RAIL
https://www.nsar.co.uk/eqa/
• Provider awareness
• Gateway and ‘readiness’
• Increased employer role
• ‘Hard-wiring’ of professional
registration readiness
• Amount of on-programme
‘value’
PROVIDER APPROVALINSIGHTS - LOGISTICS
Grading Grading criteria are ambiguous and unclear.
Grade BoundariesThe 70% and 90% Grade Boundaries are considered to be high and could lead to high number of retakes / resits and guidance around retakes and resits is lacking and they are applied to mastery models and criterion-referenced assessment.
SAQ’s and Scenarios for Behaviours
The use of SAQ / Scenarios for Behaviours is ‘questionable’ practice. There is a suggestion that the assessment of behaviours are better suited (at least in-part) to practical assessment.
Distinction Descriptors
Some Distinction descriptors are unclear and not specific enough to support grading. Some descriptors are not appropriate for a written test such as “show detailed planning, organisation and investigatory skills in a well-structured and thorough format”. Some Distinction descriptors for Practical Tests are ambiguous and difficult to interpret e.g. “demonstrate originality in the consistent, effective application of technical processes, resources, techniques and materials”; what is meant by “originality” in this context?
SAQ’s / MCQ Interpretation
Concerns exist that some EPAOs are interpreting SAQ as MCQ (or are including an element of MCQ within the SAQs) and this will not assist in-terms of standardisation and comparability across EPAO’s.
Heather Akehurst
CEO
Open Awards
(contract end June 2021)
Changing
Lives
Through
Learning
EQA Provider
Standards
295 (at last formal count)
Completed 81 reviews
Across 27 standards
Involving 33 EPAOs
How Do We Resolve Issues With EPAOs ?
Work to build trust and relationships
Recognise it’s a ‘journey’
(From Paris down to Turkey)
The Standard and Assessment Plan must be the starting point (keep us in the loop!)
Look at evidence early
What’s Working?
EPAOs aware of; and wanting to meet their responsibilities
Commitment to quality – this means more
Better contacts for Assessment Plan queries
Better cross working – need consistency
Cross EPAO discussions
What Do We Do When There’s An Issue ?
Be clear about EPAO approach for each
Standard
Discuss/resolve any early concerns
(readiness checks)
Raise as soon as possible
Work ‘together’ to resolve it
Ensure EPAO aware of actions at all stages
Serious Concerns
Reflected in EPAO and Standard Report
(may not wait for full Standard or QAC)
“Agree” actions and timescales
Work ‘with’ EPAO to address the issues
(involve IFATE/Trailblazer/Buddy
up/alternative staff – impact on
Apprentice drives timescales)
Very Serious Concerns
Refer straight to IFATE
If no starts for an EPAO from 2 years
since register – advise ESFA
Complaints Procedure
Questions and Comments
Anstey Room - The Data Journey Richard Mole, ESFA
Ashby Room - The Quality Code is Key Katrina Cleaver, CMI & Dr Andy Smith, (QAA)
Enderby Room - Independence in Assessment Keeley Turton, Ever Tree Training
Sam Taylor
Head of the Assessors Guild
Belonging to a community of professionals
Sue ShutlerSenior Business Development Manager
Apprenticeships - Pearson
Using Apprenticeships to address UK PLC’s Productivity puzzle
June 2019
10
2
Apprenticeships
and
UK PLC’s Productivity Puzzle
Suzanne Shutler
Senior Business Development Manager
BTEC & Apprenticeships
UK ProductivityHow do we compare?
10
3
SOURCE: McKinsey Global Institute Analysis, Sept 2018
UK Productivity growth between 2010 – 2014: - 0.2% per annum
UK Productivity growth between 2014 – 2017: 0.9% per annum; still trailing behind European partners
such as France and Germany
UK plc’s Productivity puzzle
10
4
UK plc hired
labour faster and
added more hours
worked while
value-added
growth declined Employment increased, particularly among
the young and old, likely influenced by policy
changes around apprenticeships, university
fees, and pensions2013/14 2014/15 2015/16 2016/17 2017/18
Apprenticeship Starts 440,160 499,290 509,340 491,470 375,940
SOURCE: McKinsey Global Institute Analysis, Sept 2018
SOURCE: ESFA Market registration data, March 2019
The New landscape
105
Training
Providers
Employer
ProvidersFE
Colleges
Training Delivery
ESFA IFATE
Regulation/
Compliance
Awarding
Organisations
Assessment
Organisations
Professional
Bodies
EPA Organisations
Membership
bodies
Universities
External Quality
Assurance
Ofsted
Ofqual
Professional
bodies
Employer
bodies
IFA
2539
20 223P
are
nt –
Child
lega
l
en
tities
Trailblazer
Employer
Groups
Standards and
Assessment
Plans
Funding for
EPA delivery
Review of
Standards
???
LEARNEREMPLOYER
Only 13% (£268m) of the
Treasury’s £2.01 bnApprenticeship budget was spent in
2017-18
Source: Personnel Today, March 2019
Employers are grappling with a number of hiring challenges. First and foremost is
finding candidates with the needed job-specific or hard skills
10
7
Source: Pearson Global Employer Research , 2019
Base: Employers
10
8
Reputation of instructors and brand, alignment with work and flexible scheduling are what employers look for most in training programmes offered to employees
Past experience with provider
Tied to assessment
Personal recommendation
Payment options
Quality/reputation of instructor
Alignment of curriculum with real work
Brand/reputation of provider
Industry-based Accreditation
Ability to tailor program
Flexible scheduling
Cost
Among the top challenges in
providing L&D for employers is the difficulty of employees taking time off to participate (38% UK
employers said this)
Source: Pearson Global Employer Research , 2019
Is the EPA delivery model financially sustainable?
Is the narrow focus on job-role specific
Apprenticeship standards the right approach to
benefit the skills needs of the economy?
Does the inflexibility of inclusion of qualifications in
standards alongside an EPA disadvantage some
learners?
How do we work in partnership with employers to
make the Apprenticeships deliver the skills needs of
the economy?
109
Could we make the new Apprenticeships work
better?
11
0
Possibilities : EPA 2.0
Challenge Possible solutions
1
2
3
EPA Delivery model
Narrow focus of the new
standards
Inflexibility to include
qualifications – disadvantage
to learners
• Assessment centre delivery model
• Incentivising employees to become independent EPA
assessors – staff CPD points
• Hard skills and Soft skills certified separately
• ‘Skills CV’ – demonstrate evidence of transferable skills
learnt and relevant to other industries/ sectors
• Flexibility in funding rules
• ‘Stackable’ credits to allow flexible learning and credit
based uptake leading to qualifications
4 Partnership with employers• In-house EPA assessors – CPD and funding
• Dual professionalism approach
Tom Bewick and Sam Taylor
Benefits of universal partnership working for the
sector and sharing for learner success
Thank you for attending