1

FINAL REPORT Executive Summary

EVALUATION OF PHASES III & IV OF THE PROGRAM PALESTINIAN ADOLESCENTS: AGENTS OF

POSITIVE CHANGE – TOWARDS AN ENVIRONMENT PROMOTING PEACE AND RECONCILIATION (Jordan, Lebanon, State of Palestine and Syrian Arab Republic)

Submitted by:

Dana Peebles With Contributions from: Awny Amer Morsy Samira Smirat Georgina Saria Mahmoud Abu Aisha Kartini International Consulting Inc. Tel: 416-260-2761 79 O’Hara Ave. Fax: 416-260-2791 Toronto, Ontario Email: kartini@sympatico.ca Canada M6K 2R3 www.kartiniconsulting.com

May 25, 2015

2

3

“What was really unique about this project are the relationships and friendships that we built. Also,

the facilitators set a role model for us in the way they led us with care, love and effectiveness as if we were their children or friends. Whenever we faced a problem they used to help us solve it without

being hesitant using some of the creative problem solving skills we acquired.”

Abdel 18 years old

State of Palestine

“Over time, I managed to turn my fears and confusion as obstacles in my way into a ladder to climb and grow and joining UNICEF Program that helped me to do so.”

Jumana

17 years old El Bakaa Refugee Camp, Jordan

“I’m extremely happy to join the "young researcher program" that enabled me to be active citizen who can

serve her community through the field research skills I acquired there”.

Bisan 16 years old

State of Palestine

4

ACKNOWLEDGEMENTS:

The evaluation team would like to most gratefully acknowledge the contributions and support of UNICEF’s Evaluation Manager, Liv Elin Indreiten, in navigating the complexities of this evaluation process and for her valuable insights into adolescent programming in the region. We were also assisted in data collection process by the following young researchers from the Palestinian Adolescents: Agents of Positive Change. Their contributions were also invaluable: State of Palestine East Jerusalem Hussam Hijazi Widad Saman Lina Abu Snene Lubna zarou Rasha Joulani Hadeel Naser Eldeen Mayar Idkedik West Bank Sarah Shabane Samer Shaweet Faris Jamal Lana Awad

Gaza Ahmad Ayman Daher Eyad Wael Hijazi Malak Issam Abu Aoda Rawan Zakarya Qaseem Ruaya Issam Al Zanin Bisan Atef Owda Mona Ibrahim Hamad Ahmed Jamal Abu Sada Ibrahim Ahmed Humeed Jordan Saleh Mahmoud Abu-Mahfouz Ma’az Khalil Qatnani Mohanad Mohammad Al-Tarwneh Mohammad Faisal Daras All’a Al-Dein Alsbehat Thabet Jamal Al-Sharif Maisa Hussein Elwa Afnan Khader Al-Biredat

5

LIST OF ACRONYMS ADAP Adolescent Development And Participation AFS Adolescent Friendly Spaces CBOs Community Based Organisations CDC Community Development Committee CO Country Office CSO Civil Society Organisation EJ East Jerusalem FGDs Focus Group Discussions GAPAR General Administration for Palestinian Arab Refugees (Syrian Arab Republic) GUPW General Union for Palestinian Women HCYS Higher Council of Youth and Sport KAP Knowledge, Attitudes and Practice MDGs Millennium Development Goals M&E Monitoring and Evaluation MENA Middle East & North Africa MENARO Middle East & North Africa- Regional Office MOU Memorandum Of Understanding MSC Most Significant Change Methodology NGOs Non-Governmental Organisations PCBS Palestinian Central Bureau of Statistics PPM Permanent Peace Movement RO Regional Office SIDA Swedish International Development Agency SoP State of Palestine TOR Terms of Reference WB The West Bank WPCs Women’s Program Centres UCAN Youth Change Agent Network UNHCR United Nations High Commissioner for Refugees UNDP United Nations Development Program

UNICEF UNITED NATIONS CHILDREN’S FUND UNOCHA UN Office for the Coordination of Humanitarian Affairs UNRWA United Nations Relief and Works Agency for Palestine Refugees in the Near East YRs Young Researchers

6

Executive Summary Program Background There have been four phases of the multi-country Palestinian Adolescents: Agents of Positive Change program since the initial pilot (Phase I) in 2004. Phase I focused on i) Providing safe spaces; promoting skills development; ii) Providing life skills for adolescents; iii) Undertaking action research to support adolescent-led initiatives; and iv) Developing leaders who could mobilize other adolescents for community action. This model served as the basis for the program’s subsequent three phases. Phase II (2005- 2007) added a focus on enabling adolescents to participate in the management of safe spaces; supported networking among adolescents; and coordination with other organisations working with adolescents. Phase III (2008 – 2010) placed a stronger emphasis on mainstreaming and institutionalising adolescent programming within UNICEF’s overall program priorities starting in 2009 by strengthening cooperation with the United Nations Relief and Works Agency for Palestine Refugees in the Near East (UNRWA) and other UN agencies as well as strengthening links between and with non-governmental organisations (NGOs). Phase IV (2011-2013) expanded its scope and integrated rights-based and gender equality perspectives, as well as adopted an assets-based and life cycle approach. In all four phases, the program has operated in the State of Palestine, Jordan, Lebanon and the Syrian Arab Republic. The scope of the program over the last two phases covered: Jordan: 10 refugee camps and 4 gatherings; Lebanon: 11 refugee camps, 9 gatherings1, 4 gardens; State of Palestine: 8 refugee camps, 48 youth centres both in and outside of the camps; and Syrian Arab Republic: 9 official refugee camps and 3 unofficial ones. While the four program locations share common objectives, how it has been implemented is different in each location due to the widely different contexts affecting Palestinian adolescents in each country, e.g., adolescents in State of Palestine are deeply affected by the occupation and related conflicts. In Lebanon, Palestinian refugees are under the international community’s responsibility and Palestinian refugees of any age have limited opportunities. In the Syrian Arab Republic, the program experienced considerable success only to have much of this work undone or halted due to the conflict. In Jordan, Palestinians who live in refugee camps are the most disadvantaged and vulnerable. Key common issues are how to build Palestinian adolescents’ sense of belonging and identity and help them build a future for themselves while living in restricted and traumatic circumstances. Evaluation Scope and Objectives 1. To determine the extent to which the program has achieved the overall stated goals of Phases I to IV,

with an emphasis on phases III and IV. 2. To determine the extent to which the program’s specific objective have been met with regard to: a)

Mainstreaming and institutionalising programming for and with adolescents; b) Strengthened cooperation between UNICEF and UNRWA; c) Strengthening of technical support to relevant governments and authorities to address Palestinian adolescents’ needs and rights; and d) Stronger emphasis on systems and standards for sustainable programs and policies in favour of Palestinian adolescents’ development and participation.

3. To enhance learning processes by highlighting the lessons learnt to improve the design and performance of programs, and to strengthen sustainability and replication.

Evaluation Methodology

1 Gatherings are non-official organized refugee camps, where a group of refugees live.

7

Kartini International fielded a 5-person evaluation team, supplemented by a group of 15 young researchers (YRs) who had participated in the program as beneficiaries and who the team trained to serve as Focus Group Discussion (FGD) facilitators for other program beneficiaries. The Kartini team developed an evaluation matrix based on the 40 evaluation questions outlined in the terms of reference (subsequently condensed to 20) and collected data through: i) Key informant interviews in East Jerusalem, Gaza, the West Bank and Jordan; ii) Interviews with UNICEF and UNRWA regional staff and donors; iii) Conducting FGDs with diverse groups of program beneficiaries using a simplified form of the Most Significant Change (MSC) methodology as well as specific questions related to the different aspects of the program; iv) Documentation of mini-case studies through the MSC process illustrative of the program’s key results for adolescent beneficiaries; v) Phone/skype interviews with stakeholders from the Syriaa Arab Republic and Lebanon, as well as with a small group of program beneficiaries in Lebanon and the Syrian Arab Republic; and a review of the key results and findings using a Knowledge, Attitudes and Practice framework. A total of 441 people were interviewed as a part of the evaluation process. Of these, 41.2% were male and 58.7% female, with 243 or 59.6% being adolescents (ages 10 to 19) and 20 youth ages 20 to 24 years, with an overall total of 263 adolescents and youth. The remaining 178 key informants were implementing partners (29), parents (39) or facilitators/educators (65), UNICEF/UNRWA staff (34) plus other stakeholders such as donors, social workers and other NGOs and UN agencies. Evaluation Limitations

A delay in the start of the evaluation meant that its timing coincided with student exams in June, which made it difficult for some beneficiaries to participate.

The program does not have much baseline data and UNICEF staff were not able to provide any additional quantitative data, making it necessary to focus on assessing the qualitative results which were the main focus of Phase III & IV’s program proposals.

With one exception, the UNICEF offices and implementing partners did not organise FGDs or interviews with community or religious leaders requested by the evaluation team.

The original TORs included 40+ evaluation questions. With UNICEF’s approval, these were condensed to 20 with 60 indicators. Given the scarcity of quantitative data it was not possible to find sufficient data to assess several of the quantitative indicators.

The political tensions which arose during the last two weeks of the data collection period (in June 2014), limited both the data collection process and the participation of the Young Researcher FGD facilitators in State of Palestine. According to UNICEF, the Gaza conflict in June-August left 539 children dead and injured 2,956 as of the ceasefire on 26 August, leaving many struggling with life-long disabilities and psychological trauma2.This most recent crisis also delayed reporting on the evaluation data collected from Gaza.

Positive Adolescent Development and Participation: The program’s approach was based on the principle that adolescents are an asset and resource for their communities as opposed to a liability or a group that primarily needs services. This is the core premise of the positive adolescent development and participation approach (ADAP). Thus programming needs to take this age group’s (10 to 19 years) specific developmental needs into account, and to actively foster adolescent participation at multiple levels. Participation in this context refers to participation in decision-making within their communities, at home and at school and in community-based activities and projects. This participation, in turn, will lead to

2 http://www.unicef.org/infobycountry/oPt_74620.html

8

stronger adolescent development and a more resilient human being with increased skills in diverse areas (communication, time management, non-violent conflict resolution, action research, etc.). Adolescent Friendly Spaces: AFS were another pillar of the program. It involved creating spaces where adolescents could feel safe to network, socialize and learn new skills. The premise was to provide Palestinian adolescents with somewhere they could build a sense of normalcy and hope in addition to learning specific skills in safe, structured after-school learning and recreation programs as well as venues for positive engagement in their communities. Each AFS offered diverse programming such as remedial education using interactive learning techniques; drama, theatre and art; recreational sports; life skills training (including non-violent conflict resolution, time management, stress management, study skills, decision-making, career and education planning, discussion of sensitive issues such as drug use, relationships, etc.) and action research/leadership training depending on the context and resources available. The different program components worked to build adolescents’ skills in critical thinking, creativity, collaboration and culture and as such, were also intended to help build their resilience. Action Research Approach: The adolescents learned how to conduct action research using credible research methodologies. They then selected a research topic focused at the community level and conducted the related research. This was followed by a process that helped the adolescent participants engage with their communities and develop adolescent-led community initiatives to address key issues identified through their research and needs assessment efforts. Evaluation Findings The following summarizes the main evaluation findings for each evaluation question, noting that while we have given specific country examples in some instances, that degree of specificity is outlined in more detail in the main body of the report. I. Effectiveness Individual Results: The evaluation found the program met its overall objective of the “creation of opportunities for Palestinian adolescents to grow to their full potential and participate in development processes that influence their lives”. At the individual level 90% of the program beneficiaries that participated in the evaluation FGDs3 reported positive changes related to: i) Their confidence in dealing with school, community, and families; ii) Increased ability to interact with their peers; iii) Increased participation in decision-making and sharing ideas and opinions with their families and communities; iv) Increased ability to identify problems and priorities; v) Increased sense of ambition and ability to set clear life goals; and vi) Strengthened communication skills. This was corroborated by FGDs with program facilitators/educators (65) and parents (39) and by interviews with implementing partners (29), with all groups citing multiple examples of these changes for individual adolescents as a result of their participation. Around 70% of stakeholders also observed an increased sense of direction and hope among the adolescent participants. FGDs also provided multiple examples that demonstrated participants had increased their resilience and ability to deal with living in challenging environments. The FGDs with adolescents and parents and stakeholder interviews also found evidence that the program contributed directly to Palestinian adolescents developing increased capacity and skills to finding non-violent solutions to conflict and a reduction in violence and bullying among peers, within the family and at school. Their academic performance also improved and in the Syrian Arab Republic, the Government's

3 This finding is based on feedback from FGDs with 263 adolescents and youth (ages 11 – 24) in the four countries.

9

General Administration for Palestinian Arab Refugees (GAPAR) directly attributed the program’s support to a reduction of 12.5% in the school dropout rate in in the El-Hessenia Camp where the adolescents conducted a survey on this issue. In addition to gaining access to remedial education, the adolescents also learned more effective time management and study skills as well as how to deal with school-related stress. Effectiveness of Program Approaches: The most effective program components for increasing adolescent skills and resilience were the life skills and the action research training and activities, followed by the remedial education activities. The adolescent-led initiatives generated by the action research program also directly contributed to Palestinian adolescents gaining and acting on an increased sense of community engagement and volunteerism. Institutional Results

Adoption of Positive ADAP Approach: UNICEF selected CSO partners carefully to ensure they were knowledgeable about how to work with adolescents, but the evaluation found limited evidence their association with UNICEF led these organisations to change how they operated significantly. For some this was because they were already using a positive ADAP approach. Most partners however, did make a point of including adolescents as decision makers to some degree. This was done by many CSOs systematically but only on an ad hoc basis by other partners (depending upon the country). In Jordan, for example, the Women’s Program Centers (WPCs) involve adolescents in decision making related to related needs assessments as well as contribute to the planning and implementation of program activities based on the assessment results. They also participate in the program’s management committees in each center4. UNRWA and Governmental Approaches to Positive ADAP: The program laid the foundation for the institutionalisation of positive ADAP approaches in the State of Palestine, and was able to institutionalise and implement this approach systematically in Jordan and the Syrian Arab Republic and to a somewhat lesser degree in Lebanon. In the Syrian Arab Republic, UNRWA and GAPAR adopted the AFS model, particularly the life skills and action research components and implemented it in all the refugee camps where they operated. In Jordan, UNRWA adopted the program’s action research approach as a part of school curricula and implemented this in the 172 UNRWA-run schools in Palestinian refugee camps there. In Lebanon, aspects of this approach were also picked up in 63 UNRWA schools. UNICEF’s collaboration with UNRWA in these three countries represents the program’s greatest success with regard to the institutionalisation of positive ADAP approaches. In the State of Palestine, UNICEF staff worked with the Higher Council of Youth and Sports (HCYS) to establish as a set of minimum National Standards for Youth Centres and Adolescent Friendly Spaces based on a positive ADAP approach that included adolescent participation in centre management committees. These standards were finalized in 2013 and endorsed by the authorities in 2014. However, this being such a recent approval process these standards are not yet being implemented widely or consistently enough in the centres run by the Palestinian authorities. Interface with the UNICEF area program for Palestinian children and women in Jordan, Lebanon, the Syrian Arab Republic and the State of Palestine: The Palestinian Adolescents program provided the foundation for the model on how to operationalise the positive ADAP approach within the new area program for 2015-2016. The evaluation noted that the new area program explicitly builds on lessons learned in the four countries that participated in the Palestinian Adolescents program and the last area program (2011-2014) also included many elements of the positive ADAP approach.

4 Interview. CSO #1. Jordan. June, 2014.

10

Results related to Girls’ Participation: The program used a combination of equality and equity-based strategies to increase girls’ participation from 10 to 20% in Phase 1 to 40 to 65% in Phase IV. All four countries made a concerted effort to include measures that would increase or lead to more equitable participation of girls. These included outreach to parents, holding activities in youth centres and AFS at separate times or days from those of boys, hiring more female facilitators and including activities of greater interest to girls. Inclusion of Disadvantaged Groups: All stakeholders and UNICEF staff mentioned they had not reached or addressed the programming needs of adolescents with disabilities well enough. The Syrian Arab Republic program was the main exception and worked actively to integrate this group into their regular programming. Other offices claimed they had made a concerted effort to reach disadvantaged Palestinian adolescents living in rural and remote areas or in particularly difficult circumstances. It was difficult to track this quantitatively – although AFS/youth centre locations appeared to bear out this assertion. However, there was not a consistent system for counting the number and type of program beneficiaries or a systematic disaggregation of beneficiaries by disadvantaged groups, except in the Syrian Arab Republic. The other disadvantaged groups the program did not reach well included juvenile labour, married adolescents, the Bedouin and the extremely poor. Constraints on Effectiveness: Political movements and events in the region acted as a significant constraint in the Syrian Arab Republic, and Lebanon, particularly in Phase IV. The Syrian conflict limited Palestinian adolescent programming in the country and strained resources in refugee camps in Lebanon. The unpredictable nature of the situations in the areas of program implementation also made it difficult to implement as planned in these countries at times. II. Relevance AFS Model: The use of the positive ADAP approach was appropriate for this group as it helps to build resilience among a highly vulnerable population and addresses their previously unmet need for safe places to go, where they can both live a more normal life and learn how to cope with living in the challenging environments in which they find themselves. UNICEF’s Comparative Advantage: UNICEF’s credibility in the region with governments, authorities, CSOs and parents5 gives UNICEF a comparative advantage in its adolescent programming work. This has helped UNICEF build the trust needed to establish an effective working relationship with adolescents. The adolescents themselves also feel proud to be working with UNICEF.6 UNICEF staff expertise on adolescent programming and the agency’s access to donors is also a comparative advantage7 as are UNICEF’s partnerships with government, local NGOs and CBOs and ability to link these different groups. UNICEF is also perceived to be a good partner with which it is easy to collaborate,8 despite the fact that this partnership comes with a few administrative challenges in terms of disbursement and reporting processes, particularly for the CSO partners. III - Efficiency

5 The credibility with parents is important for the adolescents’ opportunity to participate as this influence the parents’ willingness to give their adolescent daughters or sons permission to participate in the programs. 6 Interviews – Jordan: UNICEF staff #2; CSO #3; CSO #4 ; June 2014 7 Interviews – Jordan: CSO #2. June 3, 2014; CSO #3, June 2014. 8 Interview: CSO #4. Jordan. June 2014

11

The program has been efficient in the following areas: i) Use of existing local capacities and resources to support program initiatives; ii) Including adolescents as partners in the program process, especially with regard to the adolescent-led community initiatives; iii) Delivery and implementation of funding and program commitments on time most of the time by diverse implementing partners ; iv) Expending program funds fully and mostly on time; v) Sharing of lessons learned and experiences at the regional level and selected implementing partners and development of program resources that could be used in more than one country; vi) Building linkages between young people and decision-makers at the community level, particularly through the establishment of youth centres and AFS management committees in which the adolescents participate (although this latter approach was not always applied systematically in all four countries or locations within each country); and vii) Building an effective collaborative working relationship with UNRWA in Jordan and the Syrian Arab Republic, as well as in Lebanon, that capitalised on the comparative advantages of the two agencies. Where the program has been less efficient has been in terms of: i) Not adhering to planned budget lines, with funds being reallocated to other budget lines by more than 15%, particularly during Phase IV - these changes generally made sense given the shifting contexts in which the program worked but not all offices followed the donor agreements fully and coordinated with the RO and requested approval of these budget line changes before expending the funds on the new budget lines; ii) There are inconsistencies in how the program has been monitored at the country/area office level and from partner to partner, including how to count how many beneficiaries there were; iii) The program’s log frames tended to use indicators that described completion of activities as opposed to the actual results stemming from those activities and consequently progress reports were also light on the analysis of the program’s results; iv) Implementing partners were not trained on how to monitor and report on the program’s progress and results; v) UNICEF laid the groundwork for, but was not able to develop a more regional approach to, its collaboration with UNRWA due to the decentralised nature of both organisations. IV. Sustainability Sustainability for this program in this region requires a combination of three key elements: i) Skills and resilience of adolescent participants increased to a degree at which they will continue in leadership roles in the community and be able handle to deal with the stresses and decisions they confront in their lives on a daily basis while living in challenging contexts; ii) Institutionalisation of the positive adolescent development approach by key program partners such as UNRWA (through their services) and the Palestinian authorities (for example through the HYCS applying minimum standards for AFS); and iii) Identification of new donor funds to continue to build on the successes and lessons learned from Phases III and IV. Examining these three pillars one by one, the evaluation team agreed that UNICEF’s perspective that the program has been working on building the capacity of its implementing partners and of the adolescents, plus the institutionalisation of the positive ADAP approach among partners, are the main ways in which the program can and will be sustained.9 Given the political situation in the region it is highly unlikely this type of program will be financially sustainable for years to come. In Jordan, Lebanon and the Syrian Arab Republic, the program is working with refugee populations living in camps where often neither parents nor older adolescents can find legal paid employment. In the State of Palestine, tax revenues are collected by

9 Interview. UNICEF MENARO personnel #3. Apr – Aug. 2014.

12

the Israeli authorities. This can affect the predictability of the funding which the Palestinian authorities draw upon to support adolescent and youth programming. 10 Using this broad definition of sustainability, the program has been successful in building adolescent capacity to be more engaged and resilient community members as well as in strengthening their academic performance that opens new doors and opportunities for them in the long term. The program has also had some successes with regard to the institutionalisation of the positive ADAP approach with UNRWA programming at the country level and within the APD, as well as the development and adoption of Minimum Standards to Operate AFS and Youth Centers. However, the evaluation did not find evidence the program had a financial exit strategy, with the exception of Gaza. Part of this is that the program has evolved from phase to phase and the program has been uniquely privileged to benefit from funding from one donor for its duration. However, one lesson learned is that there is a need to start talking about financial and other sustainability issues at the beginning of each phase and to involve implementing partners in these discussions on an on-going basis. Scaling Up and Replicability: The program’s approaches have been replicated and scaled up in diverse ways. These include that the action research component was scaled up from work in a few AFS in Jordan to incorporation in school curricula in the 172 schools UNRWA operates in Jordan as well as having been adopted as a major approach by UNRWA in the Syrian Arab Republic, and 83 UNRWA schools and 25 health clinics are now accessible for UNICEF to implement its health, education and psychosocial programs in Lebanon. Al Nayzak, a CSO based in the State of Palestine, is also now working with UNICEF and the Ministry of Education to expand its young researcher/community initiatives approach to reach 10,000 adolescents in the school system. In 2011, UNRWA in the Syrian Arab Republic also received funds from the EU to develop a program called “Engaging Youth” which replicated the experience of the Palestinian Adolescents program.11 At the regional level, the key approaches developed in the Palestinian program served as a catalyst for the Sida-funded Adolescents: Agents of Positive Change program which operated in 9 countries in the region. The key elements replicated in the Swedish-funded program were the adoption of the AFS model and the adolescent civic engagement and community initiatives approach. Lessons Learned: The key lessons learned in this evaluation process were that:

1. AFS programming needs to be offered year round as opposed to in short three-four month stints to consolidate program results and to help build a more sustainable and qualified cadre of facilitators.

2. The life skills approach is very effective in teaching adolescents how to address conflict in a non-violent way.

3. While the action research component of the program is relatively expensive to implement compared to other more class-room or recreationally based adolescent programming, it represents resources well spent in terms of the positive impact it has on both the participants and the communities in which they live. Initiatives such as this can also serve as one form of prevention of disenfranchisement amongst youth.

4. The program worked consistently and proactively towards more equal participation of adolescent girls and was able to do this effectively through diverse measures such as special

10 The issue was raised in letters dated 5 January 2015 from the Permanent Observer of the State of Palestine to the United Nations addressed to the Secretary-General, the President of the General Assembly and the President of the Security Council, and in a statement dated 18 April 2015 by the United Nations Special Coordinator for the Middle East Peace Process. 11 Donor Report. Palestinian Adolescents: Agents Of Positive Change - Towards An Environment Promoting Peace And Reconciliation. (Phase III, 2009- 2011) June, 2011.

13

outreach to parents, holding programming at separate times or days from that of the boys in some locations, hiring female facilitators and offering program options of greater interest to girls. This approach works well.

5. The cost savings and efficiencies possible with a regional approach to Palestinian adolescent development provide a strong rationale for this to be a multi-country program. The sharing of experiences regarding how each country applied the AFS and positive ADAP approach was useful but would have been more useful if it had been set up in a way that would allow all partners to participate as opposed to selected ones each year. The program also could have benefited from development of a regional advocacy strategy to maximize impact of the regional positive ADAP approach and related regional resources.

6. Given the emphasis on activities completion in the program’s M&E systems at the country level, its log frames missed multiple opportunities to track some of the program’s most significant results. This included the numbers of participants who go on to become volunteers and who obtained jobs as a result of the skills they gained through the program; the effect of life skills training on reduced violence between adolescents, within the home and in schools; and the effect of adolescent-led community initiatives on the community members and on community perceptions of adolescents.

14

Conclusions The evaluation found the program had considerable success in both Phases III and IV in achieving its overall objective: “Opportunities are created for Palestinian adolescents to grow to their full potential and participate in development processes that influence their lives”, particularly as a result of the action research component and the life skills training. The AFS model had a positive effect overall but needs to be offered for longer periods each year to really consolidate outcomes and needed to be planned to avoid conflicts with student exam periods. What the short-term nature of this programming in the AFS did show, however, was how even a relatively small investment in adolescent programming could make a significant difference in the adolescents’ lives. For “Outcome 1 – The information knowledge management base on Palestinian adolescents is improved”, the evaluation found that for example the action research results of the adolescents were used by camp management committees in Jordan and Lebanon to help inform programming needs. There was also ongoing data collection through monitoring activities but these processes focused heavily on collecting qualitative data and some program results did not readily fit into the log frames, and so were not captured beyond anecdotal evidence. A knowledge base regarding lessons learned and shared experiences was developed through annual regional meetings among UNICEF staff and partners but not all partners were able to participate and UNICEF offices did not always systematically share this information with all their partners. The knowledge gained in each country was however, shared in UNICEF and partner advocacy efforts with camp authorities, governments and UNRWA. The biggest successes with regard to the institutionalisation of action research by partners was found in the Syrian Arab Republic with GAPAR and in the State of Palestine where this approach is now being adopted by the Ministry of Education, as well as in Jordan where UNRWA has incorporated the action research component into its refugee camp-based schools. The program also helped produce 130 knowledge products, predominantly generated by the adolescents themselves through action research and subsequently used to inform camp and government authorities about issues of importance to their communities. For “Outcome 2 – Access of adolescents to safe environments where they can develop and participate is improved and institutionalised”, there was considerable success in supporting environments such as schools, community centres, youth centres, and women’s centres, to become more adolescent friendly and despite the limitations of program length in Jordan and the State of Palestine, the existence of these AFS is making a significant difference to adolescent participants. From an institionalisation perspective, one of the program’s successes lies in the creation of minimum standards to operate youth centres that include the participation of adolescents in centre management committees. However, while these standards exist and this process has also been adopted in Jordan, their implementation is still somewhat inconsistent with regard to the inclusion of adolescents in decision-making. This latter approach has been extensive in Jordan and the Syrian Arab Republic, but to a lesser degree in the State of Palestine where there are many more youth centres. There has however, been considerable support given for the adolescent-led initiatives approach in which the adolescents themselves design, plan and implement their own projects to improve their living conditions. The related projects have served to create a much more positive view of adolescent participation among camp authorities, and among communities to some degree, as well as changing parental perspectives on their adolescent children to a more positive one. For “Outcome 3 – Partnerships and strategic alliances are strengthened to support adolescent programming”, UNICEF has been able to develop more strategic partnerships for advocating for adolescent programming with UNRWA, governments/authorities and NGOs to varying degrees, but with

15

an overall much strengthened set of institutional linkages. At the individual level, the adolescents themselves have been key to developing these relationships with key community leaders. Outreach efforts to parents by implementing partners have also been an important strategic approach. The adolescents were also given several venues for direct networking among themselves – at the AFS/youth centre level, between communities and camps through the research-action adolescent-led initiatives and to a lesser degree for Palestinian adolescents among the four countries involved in the program. The latter became more challenging since the escalation/outbreak of conflicts in the region during Phase IV. While of unquestionable value for this age group, the networking initiatives, with one exception (the UCAN network in Jordan), have been more ad hoc and occasional than systematic (i.e. restricted to either networking within youth centres and AFS or occasional meetings between communities). Therefore there were some questions raised regarding the sustainability of these networking efforts. There is also an ongoing debate about quantity versus quality in program approaches among UNICEF staff. The quality approach that focused on longer-term training of smaller numbers of adolescents in community action research appears to have had a more sustainable and profound impact. The AFS model reached much larger numbers of adolescents but has had a more sporadic impact, depending upon the quality of the programming offered at the specific centres and on the continuity and length of programming offered. The latter approach, however, could be strengthened if more of the program components were run year round. The program approach has much to offer, particularly in light of the immense challenges facing Palestinian adolescents. Programming to support Palestinian adolescents cannot be self-sustaining financially. Therefore UNICEF needs to consider finding ways of obtaining long-term funding as opposed to one year and three-year tranches of donor commitment – if this is possible in the current funding climate. While it has not been well documented by the program, there is still sufficient evidence to indicate that this program approach makes a significant difference to the lives of a highly vulnerable male and female Palestinian adolescent population. Any program that helps build the resilience and adaptability of Palestinian adolescents and helps them gain a stronger sense of belonging and of having a future is one in which it is well worthwhile for international and national actors to invest. Recommendations are listed in order of priority within each sub-heading.

Regional Approach (RO responsibility unless otherwise specified): 1. The regional approach is valuable and could be strengthened further by:

a. Based on needs assessment within each country, developing a regional strategy and oversight mechanism in collaboration with the COs regarding which particularly disadvantaged groups of adolescents the program can and will address that includes explicit targets and strategies for reaching these particular groups (with special attention to adolescents with disabilities, including mental illness). Sufficient funds need to be explicitly set aside to make the implementation of these strategies possible (including the needs assessments, and related capacity development for UNICEF and partner staff). The use of these funds should also be disaggregated so that they can be tracked and measured for cost effectiveness purposes. As a part of this process, explore the use and/or adaptation of the UNWRA referral system12 as one means of identifying particularly vulnerable adolescents.

12 UNRWA has through its services (health, education and social services) to Palestinian refugees a referral system, which could be used to identify and ensure identified vulnerable adolescents receive the services they need.

16

b. Building more formal regional partnerships (e.g., with academic institutions to assist with

documenting of longer term program results and a longitudinal study on the impact of the AFS/action research and positive adolescent development approach on Palestinian adolescents in the region, and continuing to work with UNRWA to determine how to develop a more regional approach to integrating positive adolescent development approaches in UNRWA schools and other services, etc.).

c. Work with UNWRA to develop a regional framework which outlines how the two organisations

will work together in the future, building on past successful collaboration models developed through the Palestinian Adolescents: Agents of Positive Change program, with a target date of year end for completion.

Program Approaches (Responsibility: Country/area Office and Implementing Partner level) 2. Review the current content within the AFS programming and develop a consolidated and

standardized minimum package of delivery which addresses skills development, learning support, psychosocial services, resilience building and community engagement conducted through interactive sessions and sports for development as well as theatre, the arts and action research. This minimum package also needs to be coordinated with what is offered through formal education modalities in UNWRA schools so that the two offerings are complementary. Where possible, all AFS/youth centres’ program components should be offered on a year-round basis in all countries. This would facilitate a more holistic approach to programming with multiple program options so that the adolescents can take advantage of all of the program corners as well as consolidate program results. It will also help ensure better quality facilitation as the facilitators would then have more stable employment and provide greater opportunities for internal staff training in positive ADAP approaches and interactive learning methods.

3. There needs to be a more systematic, institutionalised approach to involve the targeted

adolescents and youth in a meaningful way during the planning phase of related programs and interventions and consider their insights, ideas and recommendations through participatory reflection planning processes as well as to include them more systematically in program monitoring and evaluation.

4. The social cohesion and conflict resolution components of the life skills part of the AFS/youth

centres approach has been particularly successful and highly relevant given the context for Palestinians in the four areas of the program. This is as they give the Palestinian adolescents increased skills to handle these challenges in a more positive way. Therefore this component should be further strengthened and scaled up as much as possible and its results tracked systematically.

5. The peer learning approach employing mobile training units that was used in the Syrian Arab

Republic was an effective way to increase outreach, especially for the disadvantaged groups and those who are facing mobility and security issues in the countries and districts most affected by conflict, and to promote program ownership. The scaling up of this peer learning mobile training unit approach could be considered as one program approach in Phase V.

6. Future programming needs to take a systematic approach to building on the experience of

volunteerism generated by the action research and peer-to-peer program components and use

17

this to help ensure that there are new opportunities for the program’s adolescent participants after they turn 19 years old. This should include formal recognition of these contributions. UNICEF offices could do this in partnership with its current implementing partners and other agencies such as UNRWA, UNDP and UNFPA as well as build on the foundation created by the existence of a core group of volunteer action researchers who have already been working at the community level.

Monitoring and Evaluation 7. Future programming needs to strengthen its M&E approaches at both the RO and country/area

office level by: a. In collaboration with UNRWA, developing a unified monitoring framework for the program

implementation in all four countries which establishes clear quantitative targets and means of measuring the program’s qualitative results (e.g., amount of decreased violence, numbers of adolescents who feel better equipped to make life decisions, etc.) that are consistent across the four areas for the UNICEF offices, UNWRA and all implementing partners. This should include a definition of who is a beneficiary and how to count them each year. It could also include a comparison of participation and impact for adolescents participating in the program through UNWRA schools and through the AFS/youth centres.

b. Establishing clear and consistent qualitative indicators related to how to measure the

changes in male and female adolescents lives on multiple levels (i.e., not just academic but also in life skills, sense of self-esteem and identity, communication skills, etc.). This would also need to include baseline testing on core wellbeing, skills and knowledge indicators prior to the participants taking part in the program, the results of which would be reported regularly in all levels of progress reports (i.e., partners to UNICEF offices, UNICEF offices to the regional office, and the regional offices to donors). This would also help to strengthen the analysis included in progress reports to reflect on the results of activity completion and qualitative changes in the adolescents’ lives, particularly for girls.

Sustainability 8. As was done with both the Palestinian Adolescents: Agents of Change and the Sida-funded

Adolescents: Agents of Change programs, if at all possible within the current funding environment, UNICEF MENARO should strive to continue to set up future programming as a ten-year program (or at a minimum 5 years) and build in different aspects of sustainability into it. This would include:

a. Applying the capacity building approach to sustainability with implementing partners in a

more systematic way by developing a regional- and country/area office-level capacity-building plan accompanied by the necessary resources. This would include (but not be limited to) M&E systems, interactive learning, how to institutionalise positive adolescent approaches and how to attract alternative funding such as other donors, private sector funding and community funding.

b. The country/area offices holding frank discussions with implementing partners at the both

the planning and initial program phases to develop realistic sustainability strategies within the context in which they work, including discussion of financial sustainability.