p2e driving performance through evaluation richard mcmorn [email protected] 07737514762...

p2e Driving Performance through Evaluation

Richard [email protected]

07737514762

PROGRAMME IMPACT REACTION REFLECTION LEARNING ACTION

An Impact Evaluation

PROGRAMME IMPACT REACTIONPROGRAMME IMPACT REFLECTIONREACTIONPROGRAMME IMPACT LEARNINGREFLECTIONREACTIONPROGRAMME IMPACT ACTIONLEARNINGREFLECTIONREACTIONPROGRAMME IMPACT

mailto:[email protected]



07737514762

PROGRAMME IMPACT REACTION REFLECTION LEARNING ACTIONPROGRAMME IMPACT REACTIONPROGRAMME IMPACT REFLECTIONREACTIONPROGRAMME IMPACT LEARNINGREFLECTIONREACTIONPROGRAMME IMPACT ACTIONLEARNINGREFLECTIONREACTIONPROGRAMME IMPACT

The Manchester Active Voices Essential Leadership Programme ran as a residential programme between 17th and 23rd July 2010

The programme was designed for 16-25 year and consisted of three main strands

Civic Participation

Empowerment

Leadership

The programme offers young people the opportunity to operate within established strategic frameworks and develop key transferable skills, necessary to become effective leaders within their schools, homes and communities.




07737514762


AIMS AND OBJECTIVES

The aims of the programme are: To raise awareness of issues impacting on Young People within the UK

To provide for opportunities for YP to engage with the community

To help achieve these aims the programme had the following objectives:

To increase the levels of knowledge and understanding in

•Equality and Diversity

•Leadership

•Communication

•self and personality

•Social / Domestic Issues

To agree a series of actions or goals that work towards increasing community involvement, seeking education or employment, taking ownership and removal of barriers to personal growth




07737514762


A CHAIN OF IMPACT

In order to effectively evaluate the impact of this programme, the aims and objectives were aligned with a ‘need’ for each of the 5 steps in the evaluation chain as shown below.

This will ultimately lead to the programme being able to evidence a Social Return on Investment

Chain of Impact

Participants find programme relevant / important

Participants increase KU of key skills / themes

Participants complete action plan and engage with community

Participant influences community policy

Programme OutcomeFor YP to make significant impact within their commu...




07737514762


THE NEED

NEED OBJECTIVE

There is a need for YP to recognise the relevance and importance of an empowerment programme in addressing their personal growth needs as well as to reflect on their experiences during the programme

Attendees should rate the content of the programme as relevant and have indicated that their needs are fully or partially met.

YP should be able to demonstrate emotional intelligence through reflective journals

There is a need for YP to increase their levels of knowledge and understanding in the identified key issues

80 – 100% of attendees should indicate an increase in their KU of each of the key learning areas resulting in 70% of participants showing ‘some’ or ‘complete’ KU in the areas

There is a need for YP to apply and develop their skills within their own circumstances and implement changes towards achieving their aims

60-80% of attendees should show commitment to progressing and applying their skills through the development of an action plan.

40-60% of participants should evidence actions undertaking to achieve their goals up to 6 months following the programme

There is a need for achieved their aims in terms of improving their educational options, employability, community involvement and influencing community policy

30-50% of attendees should be able to evidence:•Improved attainment / employability•Increased involvement within community•Improved efficiencies within community policy / services

There is a need for the programme to demonstrate a return on investment through the measuring of YP’s aims and social benefit

To evidence a return on investment through cost to benefits ratio calculation and measure of intangible benefits




07737514762


A PROFILE OF YOUNG PEOPLE

Ten Young People took part in the empowerment Programme. Of these ten the following information was captured:

4 Young People indicated that they were not in Education, Employment or Training (NEET)

6 indicated they were in involved in some level of employment, education or a combination of both

7 suggested they had previous involvement with at least one community group, this often included groups within education settings

4 indicated involvement with the shaping and influencing of local service delivery




07737514762


Participants were asked to rate the relevance of the programme in addressing their personal growth needs using the below scale

•None of the content was appropriate / did not reflect need

•Little content was appropriate / needs were not met

•Some content was appropriate / needs partially met

•All content was appropriate / needs fully met

•100% rated the programme as ‘all content appropriate / needs fully met’

They were also asked to rate the importance (on a 4 point scale) of the programme in relation to community involvement being a key driver for change:

•100% rated the programme as ‘Very Important’




07737514762


As well as rating the programme YP also had the opportunity to provide additional comments summarising their reaction….. Below is an example…..

Could not have found a better life changing opportunity …. every piece

of information gained…. has been inspirational and 100% effective.

Without a doubt, I am beyond pleased I partook in this course.

This has been an absolutely intense and enjoyable experience. I have

learned a lot about myself and others around me…… I started this course running with the world spinning too fast for me, and now I can walk with

the world following behind

The programme has been a great eye opener for me, and I look forward to implementing the

knowledge gained to improve society for everyone




07737514762


As part of the programme Young People kept a daily reflective log to account for their thoughts and feelings as the programme progressed.

These logs have been reviewed by identifying the frequency of use of key descriptive words. Words such as ‘emotional’, ‘intense’, ‘enjoyable’, ‘great’. These were also categorised words relating to some measure of satisfaction or some measure of emotion. A proportion of frequency is demonstrated below

69%

31%

Emotion

Satisfaction

Proportion Words

Category




07737514762


Further analysis of the frequency of word use suggests an emotional journey for the participants. The chart below shows the frequency of word use per day. Notably ‘satisfaction’ or enjoyment words feature high to begin with however emotionally descriptive words soon become more prevalent.

0%

2%

4%

6%

8%

10%

12%

14%

16%

18%

Saturday Sunday Monday Tuesday Wednesday Thursday

Emotion

Satisfaction

Proportion Words

Day

Category




07737514762


…another quote

This programme has really helped me bring out my confidence and helped

me to communicate with others from a number of different backgrounds.

I feel like this programme has given me a vision of where I want to be in the future and the skills that I have

learnt will help me get to where I want to be.




07737514762


An objective of the programme was to increase the levels of knowledge and understanding in several key areas:

Equality and diversity – to be completed

Leadership

•Theories and models of leadership

•Understanding of leadership, vision and strategic thinking

Communication

•Knowledge of effective communication, presentation, giving and receiving feedback

•Understanding of diversity and prejudice, how to tackle conflict

Self and personality

•Understanding of self and emotional intelligence

•Knowledge of personality types and what they mean

Social / Domestic Issues

•Understanding the effects of domestic abuse, the tactics and tell tale signs

•Knowledge of the barriers to help and what you can do about it

•Understanding of what makes a healthy relationship




07737514762


Participants were asked to comparatively rate their levels of knowledge and understanding of each of the learning outcomes both prior to, and following the programme. This was measured on four point scale with 1 being no KU and 4 being complete KU.

100% of participants indicated an increase in their levels of knowledge and understanding in each of the outcomes

On average 78% of participants suggested they had little or no knowledge and understanding of all of the outcomes prior to the workshop

This shifted to 100% of participants indicating some or complete knowledge and understanding following the programme




07737514762


The measured change for each participant was recorded as a score. These were totalled for each learning outcome to show the area of the highest increase; the results of which are below:




07737514762


Further to this the charts below demonstrate the average shift of levels of knowledge and understanding (K/U) in the 5 key learning areas.

Leadership

78%

17% 6% 0%0% 0%39%

61%

0%

20%

40%

60%

80%

100%

No K/U Little K/U Some K/U Complete K/U

Pre Post

Self and Personality

56%

33%

11% 0%0% 6%

39%56%

0%

10%

20%

30%

40%

50%

60%


Pre Post

Communication

22%

50%28%

0%0% 0%28%

72%

0%10%20%30%40%50%60%70%80%


Pre Post




07737514762


Further to learning, participants were also asked if they committed to making personal changes and the continued development of their skills.

All participants committed to development in the following areas:

Leadership skills (9)

Personal / social skills (9)

Community Involvement (8)

Home life improvements (8)

Education / Employment (9)




07737514762


At the point of this evaluation 7* YP completed an action plan which set out a series of targets or goals for them to achieve over the coming months. These goals are designed to implement changes through the application of their skills and work towards achieving the programmes overall aims.

They generated goals in the following areas:

Involvement in community programmes / youth panels

Taking ownership over personal / community based circumstances

Seeking out educational / employment opportunities

Identifying and removing personal growth barriers

*please note that 2 YP are still expected to submit their action plans which will be incorporated into the final evaluation



07737514762


The Goals

the table below represents a paraphrase of the goals set for each of the key areas:

involvement in community programmes / youth panels

• All of the returned action plans included goals of increasing involvement with community programmes. This included continued involvement with MAVYEP

• Two YP suggested developing workshops within MAVYEP

Taking ownership over personal or community based circumstances

•Three YP identified situations where they could share their knowledge and skills within community groups and their peers

•Three YP referred to taking responsibility for finding a job and working on their vision

Seeking out employment / education opportunities

• Four YP indicated that they intended to find employment or begin training to aid future employment

• Two YP suggested that they would be continuing into further education

Identifying a barrier to personal growth and removing it

• Three YP suggested lack of confidence was a barrier to growth and were working on removing that which included improving public speaking. A fourth identified a leadership programme to attend•Two YP referred to improving their personal skills through reflection and applying their learning into daily situations



07737514762


The Goals

Each goal within the action plans were reviewed to identified running themes. The chart below demonstrates the frequency of themes for all of the key areas.



07737514762


WHAT NEXT?

Manchester Active Voices Youth Empowerment Programme has clearly been highly successful in achieving its short term aims:

The programme engaged with young people that were identified as NEET, who were seeking employment and with a drive to be active in community involvement

Every Young Person recognised the relevance and importance of the programme in relation to their growth needs and to the fact that community involvement is a key driver in change



07737514762


WHAT NEXT?

Every Young Person could evidence an increase in their levels of knowledge and Understanding of each key learning outcome

Through reflection Young People demonstrated an increase in their emotional intelligence and awareness

Through action planning Young People committed to developing their skills and reaching goals

Young People demonstrated a passion to share their skills and experience and make significant improvement s to their own lives and the community



07737514762


WHAT NEXT?

Further Evaluation

Young People’s progress towards their identified goals will be monitored by young people being given the opportunity to reflect, evidence and assess their achievements

Qualitative and intangible measures will be evaluated to show the impact the programme has had in terms of confidence, improved community engagement and increased leadership

Quantitative measures will be identified and evaluate to show a financial benefit to the programme. Areas of benefit will include:

Gaining / improving employment

Delivering community programmes / workshops

Influencing community service delivery



07737514762


WHAT NEXT?

Evaluation Timeline

Task Aug Sep Oct Nov DecEvaluating Feedback and action plans from YEP residential

Preparing and delivering evaluation presentation

MAVYEP collection of ongoing action plan progress

Evaluation of action plan feedback

Telephone interview with YP

Evaluate cost to benefit ratio

Prepare and Deliver final ROI evaluation report

p2e driving performance through evaluation richard mcmorn [email protected] 07737514762...

Documents