ms. shivaun o' brien, qfi national co-ordinator quality framework initiative for youthreach and...
TRANSCRIPT
Ms. Shivaun O' Brien, QFI National Co-ordinator
Quality Framework Initiativefor Youthreach and Senior Traveller Training Centres
Embedding a Culture of EvaluationOctober 4th 2007
Ms. Shivaun O' Brien, QFI National Co-ordinator
Why a QFI for Youthreach and STTCs?
• Demand from the centres (Youthreach 2000)• Education Act (1998) - Centres were designated as
Centres for Education in 2004• YR/ STTCs both work with children, ie. under 18, also
sixteen year olds/ those not completed 3 yrs post primary school
• DES responsible for the quality of education provided in Centres for Education as for schools
• Schools have SDPI• YR/ STTC have QFI• SDPI and QFI similar in process and resources
Ms. Shivaun O' Brien, QFI National Co-ordinator
PHASE TIMEFRAME
Exploratory Phase Nov ‘00 – Feb ‘01
Consultation Phase May ’01- April ‘02
Development Phase May ’02 – July ‘03
Pilot Phase Sept ’03 – July ‘04
Roll out Phase March ’05 - present
Development of the Quality Framework Initiative- Timeframe
Ms. Shivaun O' Brien, QFI National Co-ordinator
The Quality FrameworkQUALITY FRAMEWORK
Quality Standards – 27 Quality AreasOrganisational Management
Personnel and Development
Learning Environment
Programme
1. Ethos 11. Staff Team 15. Social Environment 19. Prog. Development and Delivery
2. Planning 12. Staff Recruitment and Induction
16. Code of Behaviour 20. Recruitment of Learners and Admission
3. Evaluation 13. Staff Dev. & Train. 17. Equality 21. Initial Ass. Ind. & Review
4. Commuic. & Links with the Community
14. Staff Support 18.Interculturalism 22. Learning Ass. & Certification
5. Transparency, Acc. and PR
23. Support Structures for Learners
6. Admin. & Financial Mgt.
24. Literacy & Numeracy
7. Record Keeping 25. Soc. Personal & Health Ed.
8. Health & Safety 26. Work Exp.
9. Premises 27. Transfer & Progression
10. Equipment
Ms. Shivaun O' Brien, QFI National Co-ordinator
How ICE happens (practicalities)
• ICE session takes place over two days• Centre closed • Some preparation required in advance• QFI Facilitator booked in advance/ TES rates• Structure of session prescribed/ guidelines• Important that each activity is useful/ time well spent• Facilitator types up outcomes of the session• Centre Co-ordinator/ Director prepares annual report
for VEC following session• Check implementation of actions from year to year
Ms. Shivaun O' Brien, QFI National Co-ordinator
What can be evaluated?
• Any of the 27 quality areas – select 9 areas to start
• Specific Programmes/ activities of the centre
• Implementation of the Centre Plan
• Combination of these
Who is involved?Core people (attend 2 days)
• Centre Co-Ordinator/ Director• Centre Staff incl. admin
Key - but involved to varying degrees
• Learners (evaluation workshops & questionnaires)
• Regional Co-Ordinator• Adult Education Officer (AEO)• Education Officer (EO)• Chief Executive Officer (CEO)• Board of Management Reps
Involve where relevant (questionnaires/interview/ focus groups)
• Community Reps. e.g if evaluating links with community • Parents/ Employers e.g. if contact with parents is being evaluated
May attend part of 2 day session or complete questionnaire/ interview in advance
Ms. Shivaun O' Brien, QFI National Co-ordinator
Internal Centre Evaluation
Internal Centre Evaluation
Ms. Shivaun O' Brien, QFI National Co-ordinator
Purpose of the ICE
Primary purpose: • To examine centre practice, identify
strengths and weaknesses, identify & implement actions to improve the service.
• Accountability: annual reports to VEC, evidence for DES inspectors
But is that all?
Ms. Shivaun O' Brien, QFI National Co-ordinator
Evaluation can have process outcomes
“Process refers to and is indicated by individual changes in thinking and behaviour and programme or organisational changes in procedures and culture, that occur among those involved in evaluation as a result of the learning that occurs during the evaluation process”
Michael Quinn Patten (1993) Utilization Focused Evaluation
UFE :Concern with the usefulness of the evaluation
Ms. Shivaun O' Brien, QFI National Co-ordinator
Process Outcomes
• Enhancing shared understanding ( NOT teachers teach and managers manage), staff concerned with overall service not just the subjects they teach
• Teambuilding - regularity NB• Capacity Building – more capable more confident• Increasing staff engagement, sense of ownership and
self determination ( participatory and empowerment evaluation)
• Increased intentionality – end up with plan, a sense of direction, commitment to making progress
Ms. Shivaun O' Brien, QFI National Co-ordinator
Building in process outcomes• Bring staff together• Clear communication – facilitator clarifies and
challenges• Everyone talks/ listens – equally valued• See the service through each others’ eyes –
opportunities for personal reflection• Safety – speaking openly, evaluation of systems not
people, controlled & kept focused• Learning- new knowledge and skills: from gathering
and discussing evidence, learning to solve problems, learning to evaluate (double loop learning), becomes integral to programme
• Making decisions - deciding on actions that are actually carried out
Ms. Shivaun O' Brien, QFI National Co-ordinator
Evidence
• Evidence can be Quantitative or Qualitative
• Lists of evidence pg 23-26 ICE guidelines
Ms. Shivaun O' Brien, QFI National Co-ordinator
Evaluation by learnersResources• See QFI Guidelines for Internal Centre
Evaluation• Chapter 3 ICE guidelines- reflect
current good practice
www.youthreach.iewww.sttc.ie
• Hearing Young Voices www.youth.ie• Participation Spice it Up Dynamix UK• Explain the reasons for evaluation to
learners• Approach depends on learners age,
ability• Decide the focus of the learner
evaluation• Plan session/s
EVALUATION SHOULD BE A POSITIVE and
USEFUL EXPERIENCE
FOR LEARNERS
Ms. Shivaun O' Brien, QFI National Co-ordinator
Evaluation by learners ctd.• How will learners receive
feedback• Important to show learners
that their opinion was considered and that appropriate action was taken
• Evaluation (reviews) of programmes by learners should happen every time a programme is delivered- feed into FETAC QA requirement
• SEE HANDOUTS
Ms. Shivaun O' Brien, QFI National Co-ordinator
Follow-up to ICE Session
• Facilitator will type up outcomes of 2 day session and return it to centre Co-ordinator/Director
• Staff and VEC mgt get copy of evaluation report• Ensure that action plan is implemented • Keep action plan on agenda at staff meetings• Co-ordinator/ Director responsible for ensuring
that actions are implemented/ time allocated• ICE following year – check progress
Ms. Shivaun O' Brien, QFI National Co-ordinator
Contact: Shivaun O’Brien
Quality Framework Co-Ordinator
C/O Co Louth VEC
King St
Drogheda
Co. Louth
041-9839930
For QFI Resources
www.youthreach.ie
www.sttc.ie