PROVISION FOR POST PROJECT EVALUATIONS FOR THE UNITED NATIONS DEMOCRACY FUND

Contract NO.PD:C0110/10

UDF-ALB-08-264 – Empowering people through citizens’ journalism in Albania

Date: 20 February 2013

EVALUATION REPORT

Acknowledgements The evaluators would like to thank all of those who took the time to provide their expertise and insight on citizen journalism and the media in Albania and on the implementation of the Empowering People Through Citizens’ Journalism Tirana/Albania/Europe project. In particular, Elena Cherniavska and Dr. Pani Pandelli of the Institute for Democracy, Media and Cultural Exchange (IDEM) and Elda Halikaj and Klodiana Collaku of IRIOM for their institutional memory and logistical support. All errors and omissions remain the responsibility of the authors. Disclaimer The views expressed in this report are those of the evaluators. They do not represent those of UNDEF or of any of the institutions referred to in the report. Authors This report was written by Sue Nelson and Silvana Russi. Landis McKellar, the Evaluation Team Leader, provided editorial and methodological advice and quality assurance. Ms. Aurélie Ferreira was Evaluation Manager and Mr. Eric Tourres Project Director at Transtec.

Table of Contents I. EXECUTIVE SUMMARY ........................................................................................................................... 1

II. INTRODUCTION AND DEVELOPMENT CONTEXT ..................................................................................... 5

(i) The project and evaluation objectives ........................................................................................................... 5

(ii) Evaluation methodology ............................................................................................................................... 5

(iii) Development context ..................................................................................................................................... 6

III. PROJECT STRATEGY ............................................................................................................................... 7

(i) Project approach and strategy ....................................................................................................................... 7

(ii) Logical framework .......................................................................................................................................... 9

IV. EVALUATION FINDINGS ....................................................................................................................... 10

(i) Relevance ..................................................................................................................................................... 10

(ii) Effectiveness ................................................................................................................................................ 11

(iii) Efficiency ...................................................................................................................................................... 14

(iv) Impact .......................................................................................................................................................... 16

(v) Sustainability ................................................................................................................................................ 18

V. CONCLUSIONS ..................................................................................................................................... 19

VI. RECOMMENDATIONS .......................................................................................................................... 20

VII. OVERALL ASSESSMENT AND CLOSING THOUGHTS ............................................................................... 21

VIII. LIMITATIONS, CONSTRAINTS AND CAVEATS ....................................................................................... 22

IX. ANNEXES ............................................................................................................................................. 23

ANNEX 1: EVALUATION QUESTIONS: .............................................................................................................. 23

ANNEX 2: DOCUMENTS REVIEWED: ............................................................................................................... 24

ANNEX 3: PERSONS INTERVIEWED ................................................................................................................. 25

ANNEX 4 : ACRONYMS ................................................................................................................................... 26

1 | P a g e

I. Executive Summary

(i) Project Data According to the project document, the Empowering People Through Citizens’ Journalism in Albania project sought to strengthen the outreach of civil society organizations (CSOs) representing socially vulnerable groups to the community through citizen journalism. This was intended to be done by 1) equipping CSOs with public communication skills, 2) establishing a community radio station and a web portal at Tirana University, and 3) training students to report on socially relevant topics. The target groups were to be at least 12 CSOs working with vulnerable groups and 20 journalism students from the University of Tirana. This 180,000 USD project1 was implemented by the Institute for Democracy, Media and Cultural Exchange (IDEM) based in Germany, with in-country logistic support provided by IRIOM, an Albanian non-governmental organization (NGO) based in Tirana. The project ran from 1 November 2009 to 31 December 2011. This timeframe included a two month no-cost time extension. The main activities intended to be done under the project were to:

Establish a community (campus) radio station and web portal at the University of Tirana and a core team to run it;

Identify and train students and Tirana-based CSOs working on social issues through three public for a and six training courses on media production and communication skills;

Prepare and air six programmes on socially relevant themes produced by CSO representatives and the core team; and

Undertake a study tour to Germany for the radio’s core team.

(ii) Evaluation Findings The project was only partially implemented as planned. IDEM was unable to establish an on-air radio station at the University of Tirana campus due to regulatory issues and the University of Tirana did not become a full implementing partner as anticipated in the project document. Instead the project did some training at the University, but created an internet based radio at the IRIOM office in Tirana (“YouRadio”) and made arrangements with the University Aleksander Xhuvani Elbasan (referred to as the “University of Elbasan” in this report) to air the material produced by the project. It also extended its awareness raising and training to students at that university. It also bought air time on a national FM radio station (Ora) to air each of its programmes. Later in the project, IDEM found a private university in Tirana, Marlin Barleti University, with an interest in creating a campus radio to serve as the repository for the project equipment. In 2012 it transferred the radio equipment to that university’s journalism department which also intends to take over the YouRadio web portal at the start of the academic year in October 2012. The project design highlighted work with CSOs and increasing their access to community media, but in implementation the project was geared primarily towards university youth and increasing their voice and understanding of social issues. The project objectives and activities were relevant given the difficulties of youth and NGOs in Albania to access the media, voice their opinion or raise social issues. Although the Albanian media is free, mainstream media requires payment for airtime that NGOs cannot

1 Of which USD 18,000 was retained by UNDEF for the evaluation component.

2 | P a g e

afford, and it has shown a general lack of interest in covering social issues as part of the news. The CSOs that participated through their volunteers did work for the interests of marginalized groups. These included battered women, children in need, the Roma people, and those with HIV/AIDS. The CSOs themselves were extremely interested in obtaining access to media, especially one that was free of charge and could reach a national audience as was intended with the university radio in Tirana. Participating youth were interested in citizen journalism and reporting on social issues, but the number of active participants beyond initial training or awareness raising appears to have been limited, especially in light of the large number of university students in Tirana and Elbasan. Risks were accurately identified but not adequately addressed in the conceptualization of the project in regards to broadcast licensing and the continued participation of the University of Tirana. This required a reworking of the implementation plan after the project had started. The project only partially reached its objectives as stated in its results framework. In addition to the setbacks related to the licensing, the CSO strengthening element focused on strengthening the skills of youth volunteers which resulted in individual strengthening rather than institutional. The choice of the University of Elbasan as an alternative to the University of Tirana seemed to be appropriate as it had a functioning student-run radio and an interested dean who participated in some of the project’s trainings and development of material. The awareness raising efforts for the radio and its programmes seems to have been effective, as according to the project’s baseline data, awareness of You Radio increased 53 percent from March 2010 to November 2011 among the project’s target group (mainly youth under 25). However, without the grounding of the project around the running of a community/campus radio, the project’s activities seemed intermittent and scattered which limited their effectiveness and potential impact. The project did not develop synergies with other efforts in the sector, which included another UNDEF-funded media project and a youth radio funded by other donors. Project implementation responsibilities were divided between the IDEM technical expert based in Germany and two part-time staff from IRIOM who maintained the in-country activities. They appeared to have a good working relationship and implemented the project as one team. IRIOM and the technical expert appeared to do their work efficiently even though the setbacks with the University of Tirana and broadcasting license made their work more complicated and required more programmatic effort than anticipated. Outside of a few main service contracts, most of the supervision, reporting and contracting was informal.2 This resulted in the project not having readily available information on its activities, outputs, contracting or expenditures, and its financial and programmatic reports were not submitted to UNDEF in a timely manner. This delayed release of the second tranche of project funding, pushing back activities and requiring a time extension to complete the project. The final narrative report was not submitted until the post-project evaluation was scheduled eight months after the end of the project, and was incomplete. The final financial report still had not been submitted as of the end of the evaluation. This financial report is required for the release of the final tranche of funding for the project. The dearth of project reporting also meant that the achievement of project outputs and outcomes beyond the holding of events is not known. This makes it almost impossible to determine impact. IDEM did commission a survey on “YouRadio” to measure awareness of the station. Of the 76 percent of those who had heard about Youth Radio by November 2011, only 8 percent of these had actually listened to any of the radio programmes. , Thirty-three percent of these listened to the pod casts on the web portal while 20 percent heard the programs on the FM station Ora, where the project bought airtime to air each programme

2 Contracting based on verbal or e-mail agreements or invoices for services. Reporting primarily verbal or through e-mails or on

specific activities.

3 | P a g e

once. But the survey did not ask about quality of programmes or on differences in awareness/knowledge between project participants and other youth on social/community media. Anecdotal information from the interviews suggests that the project did help to increase awareness among students on citizen journalism and the use of a community radio to raise issues of interest to them. The core group of youth participants also seemed to have felt empowered by the experience. The universities of Elbasan and Marlin Barleti felt association with the project was an asset which gave them an edge over other universities. The study tour to Germany appeared to have been well prepared and was a good experience for the student participants. Holding it at the end of the project though limited its programmatic usefulness. The knowledge imparted to the students is likely to remain with them, especially for the youth that developed programmes and/or went to Germany. Some seemed to be applying their experiences in their student and volunteer activities. The participating CSOs are also likely to retain the awareness raised by the project on the value of a community radio, but very few CSOs will retain the skills as these were primarily provided to their volunteers. Both Elbasan and Marlin Barleti universities intended to adapt their media curricula to include the community journalism concept which will help sustain the training and informational aspects of the project. Marlin Barleti also appeared committed to ensuring the operation of the YouRadio internet portal and expanding it into a more dynamic site which will help to ensure the continuity of that web portal.

(iii) Conclusions The project’s focus on citizen journalism and increasing CSO and

youth access to media was needed and relevant within the Albanian context. Basing a radio within a university context was also a good approach for a

project that wanted to target youth. That would have grounded the activities around a common element and provided a purpose to the larger programme.

Without it, the training took place and the YouRadio portal was created,

but the project seemed activity driven and lacked a clear sense of purpose beyond the training.

The University of Elbasan was a good alternative under the

circumstances, but project coherence and impact are likely to have been greater had the original plan been implementable.

The CSO strengthening element of the project was not fully developed as

the project targeted youth volunteers rather than CSOs as institutions. The lack of more formal project management, monitoring and

reporting systems affected performance, and ability to provide more timely and complete reporting to its donor as well as to track its outcome and results.

Without having more specific information on the results of the activities, it is

not possible to assess impact. However, it does appear that the project far exceeded its targets in relation to numbers of students reached and made a positive difference for its participants, participating university departments and to a lesser degree for CSOs.

4 | P a g e

(iv) Recommendations

For similar projects in the future, the evaluators recommend that the formal agreements with the universities and for licensing be done before the project is submitted for funding.

Project targets should also be realistic but ambitious enough to justify the

activities and funding. The project purpose should be clearly stated in the project document and accurately reflected in the results framework and performance indicators, with all activities designed to contribute directly towards that strategic purpose.

Formal project management systems should be installed in all projects, which would normally include quarterly progress and financial reporting, use of timesheets and/or deliverables for consultants and a monitoring and evaluation plan that tracks progress towards achievement of the project purpose as well as outputs.

CSO staff should be systematically included in training opportunities with volunteers. Linkages with other media and/or social awareness projects should be made to strengthen the project and extend its outreach and impact. Tailor training to the different groups participating, as journalism students will have professional needs that the general student body will not.

5 | P a g e

II. Introduction and development context

(i) The project and evaluation objectives The Empowering People Through Citizens’ Journalism in Tirana/Albania project was a two-year USD 180,000 project implemented by the Institute for Democracy, Media and Cultural Exchange (IDEM) based in Germany, with in-country logistical support provided by IRIOM, an Albanian NGO based in Tirana. The project ran from 1 November 2009 to 31 December 2011 which included a two month no-cost time extension. Of the USD 180,000, IDEM received USD 162,000 and UNDEF retained USD 18,000 for evaluation and monitoring. The project sought to strengthen the outreach of civil society organizations (CSOs) representing socially vulnerable groups to the community through citizen journalism. This was intended to be done by 1) equipping CSOs with public communication skills, 2) establishing a community radio station and a web portal at Tirana University, and 3) training students to report on socially relevant topics. The evaluation of this project is part of the larger evaluation of the Round 3 UNDEF-funded projects. Its purpose is to contribute towards a better understanding of what constitutes a successful project which will in turn help UNDEF to develop future project strategies. Evaluations are also to “assist stakeholders to determine whether projects have been implemented in accordance with the project document and whether anticipated project outputs have been achieved.”3

(ii) Evaluation methodology The evaluation took place in August 2012 with the field work in Albania done 13-17 August, 2012. The evaluation was conducted by Sue Nelson and Silvana Russi, both experts in democratic governance and development projects. The UNDEF Round 3 evaluations are more qualitative in nature and follow a standard set of evaluation questions that focus on the project’s relevance, effectiveness, efficiency, impact, sustainability and any value added from UNDEF-funding (Annex 1). This report follows that structure. The evaluators reviewed available documentation on the project and on the issue of youth/community media in Albania (Annex 2). Interviews were held with IDEM, IRIOM, project participants from universities and CSOs, and project consultants and other stakeholders. The evaluators interviewed those in Tirana and Elbasan in person, and the remainder by phone, skype, and e-mail (Annex 3). During the preparatory work, the evaluators identified several issues which they followed up on during the field work in Albania. These included:

Project design issues and whether these extended to other areas of the project beyond under-estimating the risks related to licensing and continued participation of the University of Tirana.

Study tour and the value of doing such trainings in the last month of a project instead of in Month 18 as planned.

Use of social media which the project appeared to have leveraged to draw attention to its internet radio and broadcasting.

Project management and reporting which was late and incomplete and resulted in funding delays for tranches two and three.

3 Operational manual for the UNDEF-funded project evaluations, p. 3.

6 | P a g e

UNICEF-Funded Youth Media Survey

Sustainability and the future of the YouRadio portal. Value added of IDEM which was off-shore and provided technical assistance but

used a national NGO to implement the project in country.

In addition, the team explored the issues raised by UNDEF: Extent of project outcomes which were not reported on in the project’s Final

Narrative Report. Financial management and ascertaining if the project was accomplished within

budget and without over-expenditures in line items as these were unknown without the final financial report.

Reporting delays and ascertaining why the project was unable to meet UNDEF’s regular reporting requirements.

Value added of the IDEM management team since the reporting issues raise questions as to their role and management capacity.

(iii) Development context

Albanians have a wide range of media options available. In 2012 there were 160 print outlets, 58 radio stations (56 local, 2 national), 77 television stations (71 of these local) and 83 cable TV stations. There were 1.3 million internet users in 2012 but these are mostly concentrated in urban areas.4 The number of radio stations available on the internet is also increasing, with more than 40 stations currently streaming their programming on line. Most of the broadcasters in Albania are private commercial stations with many dominated by partisan political interests. According to the IREX Media Sustainability Index, journalist professionalism is increasing but journalists are poorly paid and tend to exercise self-censorship to avoid conflict with their editors/owners and advertisers.5 This, among other issues, has placed Albania in 96th place out of the 179 countries on the Reporters Without Borders World Press Freedom Index 2011-2012. A recent UNICEF-funded study on the use of media by youth aged 13 to 18 showed that the use of online media and alternative ways to distribute information is increasing, especially among youth. It also found increasing attention to this sector from news agencies and news aggregation services which provide a diverse range of information for general audiences.6 Although television was reported as the favorite form of media, with 70 percent of the youth surveyed saying they watched three or more hours per day, the youth could not identify what they liked about television and were uninterested in seeking programming change.7 The internet was their most commonly used medium with more than 60 percent using it daily for

4 IREX, Media Sustainability Index 2012, p 4

5 IREX, OpCit, p 9

6 UNICEF, Children and the Media in Albania, p 5

7 UNICEF, OpCit, p 7

7 | P a g e

social networking, entertainment and information (in that order). Much fewer listened to the radio, and only 7 percent of those listened to radio news. The most credible sources of information were reported to be teachers and schools, followed closely by books and the internet.8

Albania’s civil society sector is still weak and has difficulties finding space for meaningful activities in a highly politicized environment.9 Cooperation between the media and CSOs is more present on political issues than for other issues, however the Civicus study found that this cooperation politicized concerns and debates which were often counterproductive for citizen participation.10 CSOs do have access to mainstream media, however as most of the stations are commercial this requires payment for airtime as their interest in covering social issues as part of the news is low. Many CSOs are struggling to survive financially and are dependent on volunteer labour and short-term project funding. This contributes to the lack of sustainability of impact for their efforts.

III. Project strategy

(i) Project approach and strategy IDEM is an NGO based in Dusseldorf, Germany, and was created a year before this project was funded. This project was its first. It designed the project in coordination with several persons who had worked with IRIOM which was a joint project that an IDEM founder had worked with previously as an expert. It also had the input of the Dean of the Philology Department at Tirana University which was listed in the project document as an implementing partner. The university had the largest student body in Albania and transmissions from their campus in Tirana could reach most of the country. The project intended to help the University apply for and establish the campus radio and web portal, and train a core group of students to run it. This would give students in the faculty practical experience which is not normally available at Albanian universities. Through this 20 journalism students were expected to gain experience in reporting on social issues by developing programmes and working on radio. The radio was intended to provide CSOs

8 UNICEF, OpCit p 22

9 Civicus, Civil Society Index Albania 2010; Freedom House, Nations in Transition: Albania 2012

10 Civicus, OpCit, p 30

UNICEF Funded Survey Results for Importance of Media to Youth

8 | P a g e

working with vulnerable groups access to media to get their messages and information of their communities out. IDEM intended to identify and train 12 Tirana-based CSOs through three public fora and six training courses on media production and communication skills. They and the core radio team would prepare and air six programmes on socially relevant themes which would be aired by the campus radio station. Members of the core team would undertake a study tour to Germany to learn about other community media efforts. The logistical support was provided by a service contract with IRIOM for USD 32,750 to provide support for workshops, contact with participants/national experts/students/CSOs, establish the radio studio and keep records of workshops and presentations. This contract primarily funded two part-time IROM persons for 23 months and their office and communication costs for 20 months. IDEM provided a project coordinator and international experts. The project coordinator was the IDEM chairperson. The main expert was an IDEM director and professional journalist based in Germany. He was contracted to provide four workshop trainings in Tirana and six months of editorial support for the campus radio, web portal and milestone events.11 This expert provided the technical direction for the project. IDEM also contracted a web designer to develop the web portal and internet radio who was later given a contract for its maintenance. Several other university lecturers in Albania were hired to participate in the presentations and/or trainings. Management support and project reporting was done by project coordinator in Germany. The project design outlined in the project document had an overall objective of “providing CSOs representing socially vulnerable groups with outreach through citizens’ journalism.” The document also mentioned “equipping CSOs with necessary public communication skills” and on helping CSOs “create a viable communications strategy and gain access to a community radio station.”12 However, the university youth were the main focus for the project and the CSOs that participated in the project did so primarily through the participation of university students who volunteered in different NGOs and thus reflected their message in any programmes that they may have developed. As a result, the main project objective seemed to be “establish a community radio station on the basis of Tirana University and train students in reporting on socially relevant topics” which was also listed among objectives in the project document. The project was unable to establish the on-air radio but continued some training components at the University of Tirana. Activities were also extended to the University of Elbasan which had an existing campus radio station and a dean interested in community radio and citizen journalism. The project created an internet portal called “YouRadio” that it based in the IRIOM project office where it could record and upload its programming. It also branched out to broadcast each programme on a national FM radio station run by Ora, through paid airtime at a regular time each week. Towards the end of the project, it also found the private Marlin Barleti University in Tirana interested in establishing a campus radio station and provided the project equipment to its communications department. Marlin Barleti University will also take over and manage the internet portal for YouRadio starting with the academic year in October 2012.

11

Milestone events listed in the project document were: 1: Project signature Month 1; 2) First radio production and training course in Month 8; 3) Completion of training courses and programmes in Month 23; and 4) Completion of narrative and financial reports in Month 24. 12

UDF-ALB-08-264, Project Document, p 4,

9 | P a g e

(ii) Logical framework

ESTABLISH A COMMUNITY (CAMPUS) RADIO AT THE UNIVERSITY OF TIRANA

Obtain license for station Sustainable on-air campus

radio station established

Increased access for youth and CSOs to media

Increase youth and CSO voice

Purchase equipment for station

On-air campus radio station established catering to needs of urban community

Increased access for youth and CSOs to media Increase youth and CSO voice

Selection and training of core radio team

Core team selected and trained

Core team develops contacts in CSO community

Increased access for youth and CSOs to media Core team able to train further CSOs outside Tirana

Increase youth and CSO voice

Development of web portal

Web portal based on broadcast programmes

Increased access for youth and CSOs to media Increase youth and CSO voice

Awareness raising on radio station and outreach

3 public fora at U Tirana with students, CSOs, public

Posters, leaflets and ads

Expand audience for radio within and beyond university

Better informed public on social problems

TRAINING FOR CSOS/STUDENTS

Identify CSOs to participate in project

At least 12 CSOs identified

Increased awareness of these CSOs and social issues by communities

Increased #s of volunteers, and assistance for socially vulnerable groups

Provide training

6 courses for CSO/ students on media production basics and communication skills

12 CSOs with a viable communications strategy and access to a community radio station

At least 20 students trained in practical skills in digital editing and reporting on CSOs

CSOs with better public communications and outreach Improved reporting and coverage of social issues

Increased awareness and coverage of social issues

Study tour to Germany 5 participants More objective and professional journalism

More independent and objective media

DEVELOP PROGRAMMES ON SOCIAL ISSUES AND VULNERABLE GROUPS

Develop and upload programmes

6 programmes on socially relevant themes by core team and CSOs

Increased awareness and coverage of social issues by community in Tirana

Better informed public on social problems

Medium-term

impacts Long-term development

objective

Intended

outcomes

Medium Term

Impacts

Project activities

10 | P a g e

IV. Evaluation findings

(i) Relevance The programme design and anticipated outputs/outcomes were relevant. There are difficulties in Albania for youth and NGOs to access the media, voice their opinion and raise social issues. Although the Albanian media are free, mainstream media require payment for airtime that NGOs cannot afford, and have shown a general lack of interest in covering social issues as part of the news. Journalism classes are also mainly lectures with little opportunity for students to practice newly learned skills. Creating a functioning radio station and web portal there would have been a good means to provide those students with the tools to practice their craft and give a channel to university youth and participating CSOs to voice their opinions and cover issues of interest. The CSOs that participated either directly or through student volunteers did represent the interests of marginalized and socially vulnerable groups or issues. These included battered

women, children in need, the Roma people, and those with HIV/AIDS. The CSOs themselves were extremely interested in the project’s approach to media, which they saw as helping them to become an actor inside the media instead of being portrayed by the media, and at times, misinterpreted or sensationalized by it. They were also extremely interested in obtaining access to media, especially one that was free of charge and reached a national audience as anticipated with a Tirana-based radio. Reach was more limited with the switch to the provincial radio in Elbasan and an internet-based platform in Tirana. This reduced the project’s relevance for some of the CSOs. The project compensated by purchasing airtime to broadcast each of the 24 programmes

produced on a national FM radio station (Ora) which the CSOs found useful. Some CSOs also felt their opportunities to benefit from the skills training beyond the participation of their student volunteers was limited which affected relevance for their organization. The participating university youth also felt the project was relevant given the relatively new nature of citizen journalism and the lack of opportunities they felt were available to them to express their opinion and voice. Relevance of the project beyond the introductory meetings and awareness raising for other students is uncertain as the number of active participants in the project appeared limited given the potential pool of university students in Tirana and Elbasan. One of the presenters felt students were apathetic, saying the room was half empty and should have been full. One of the students suggested the programme needed to be trendier to attract more participants. However, the project only targeted 20 youth for capacity building, a target which it exceeded, and the photos for the sharing of project experiences at Elbasan showed a good turnout. Risks were not adequately addressed in the project design. Although they were accurately identified there was no alternative strategy developed as a fall back, specifically regarding

CSO Participants 1.Center for a Roma Contemporary Vision 2. Another Vision 3. Progress and Civilisation 4. STOP AIDS 5. Soroptimist international 6. ALTERNATIVA 7. CEAPAL 8. Albanian Center for Population and Development 9. ADRA 10. Democracy School (Shkolla Demokratike) 11. Shendet 2000 12. Association for Protecting and Preserving the Natural Environment

11 | P a g e

YouRadio Logo

the continued participation of the University of Tirana as an implementing partner and the ability to obtain a broadcast license for a campus station. As these were the basis for the project, the implementation plan needed to be reworked and new partners found well after the project had started. Fortunately, the project’s objectives were directly relevant to the interests of the two universities that were added (Elbasan in mid 2010 and Marlin Barleti at the end of 2011) which felt the project provided activities that would be useful for their students and something the universities themselves would not be able to do on their own due to lack of funding.

(ii) Effectiveness The strategy of strengthening CSOs at the same time as building the capacity of journalism students to report on social issues and creating an outlet for their products seemed to be an effective approach. The training provided practical information on objective reporting, the role of the media, use of a community radio and for those interested, technical training on broadcasting equipment. The establishment of the YouRadio portal gave interested students the opportunity to practice the lessons learned in training by producing a programme on topics of social interest. Twenty-four programmes were developed and 11 of these were uploaded onto the web portal and are available as podcasts. The reason for not uploading all of the programmes during the project is unclear although there appeared to be software formatting issues for some. IDEM told the evaluators that it still intended to upload the remainder. All 24 programmes were broadcasted on the University of Elbasan campus radio and through airtime bought on Ora FM 96.7, a national radio station. On Ora, each programme aired once in the morning at a regular time in the first half of 201113 and a special show was done on Ora News on World AIDS Day that repeated and discussed the programme on persons with HIV with the participating CSO. The effectiveness of the internet radio is hard to determine. It developed a distinctive name and logo reflecting its citizen journalism perspective and showing the main building of the University of Tirana wearing earphones. The logo was reportedly a joint effort of the IDEM technical expert, 2 IRIOM staff, the webmaster and the Dean from the University of Tirana. YouRadio did give the participating students and student volunteers from CSOs practice in developing a radio programme-- picking a topic, setting up interviews, discussing the issues, and then editing it into a programme.

13

Four times a month from 8:00 AM to 8:30 AM from January 2011 through March 2011 and three times a month from April 2011 through July 2011. Cost was 145USD each.

Programmes Produced for YouRadio Children's rights; Family planning; Consumer rights; Problems with the GCSE; Problems with public transport; Case study: I live in the dark (problems of the blind people); Cultural tourism; How much are vacations; Voluntary work; Social groups at risk; Integration of the Roma; Sport as a hobby; How to follow traffic rules; Environmental pollution; Blood donors; Case study: I live as a HIV positive; Young artists; The new political leadership; Youth and books; Disabled people; Youth work and unemployment; Copy right; Health protection; Case study: A family living under vendetta threat (ngujuar).

12 | P a g e

Training Manual

User statistics are only available for 2012 as records of the 2010 and 2011 data were not kept, but the 2012 data shows the number of visitors to the website ranged from 228 visitors in February 2012 to a high of 796 in May 2012. The detailed user statistics for May show that 44 percent of the users got to the web pages through an internet search engine, while 12 percent used its direct address, two percent were from links from other sites, and 41 percent were of unknown origin. Statistics also show that almost 84 percent of the visits lasted less than 30 seconds, and that only 3.9 percent of the users stayed for 15 minutes or more. The most frequently used keywords were: children and human rights (10.8 percent), drugs (7.3 percent), environment (2.8 percent), Kosovo forests (1.9 percent) and other phrases (69.4 percent). Most of the hits were on images, with 6.4 percent on the audio files. The project placed ads in newspapers and on other stations for their broadcasts to build an audience for the YouRadio portal. It also developed a Facebook page and Twitter account. Their marketing seems to have been effective. According to the awareness survey commissioned by IDEM, awareness of YouRadio increased from 23 percent from the March 2010 baseline to 75 percent in November 2011. More than a quarter of these had heard about the radio from Facebook (28.8 percent) and the internet (15.7 percent). The remainder heard about it from ads (9.8 percent), friends (8.2 percent), posters (7.7 percent) and promotional events (4.9 percent).14 The inability to implement the project as designed affected the effectiveness of the project. It required a rethinking of project partners and activities and created implementation delays. Training was still undertaken with students from the University of Tirana, but its effectiveness was reduced without the radio station to ground it and to give the training its programmatic purpose. The students who participated in YouRadio and the development of programmes did so individually rather than as part of a university activity or class. The training activities were the main focus of the project for IDEM. They appear to have been useful, providing interested students with tools as well as information. The interactive nature of the training was appreciated by the students as well as by the lecturers. The main training sessions took four full days and averaged about 14 students per session. IDEM provided a highly experienced professional journalist as its main technical expert who recruited several Albanian media professionals and professors who participated as lecturers. They developed a training manual and other presentation materials. This material included Albanian translations of excerpts from different journalism manuals and excerpts of the

14

Progress and Civilization Organizations, YouRadio- Citizen journalism evaluation study report, p 10. Survey was random but was done in areas known to have high concentrations of university students.

Programmes Available on YouRadio August 2012

First Broadcast Revenge Second Broadcast Youth Employment Third Broadcast Groups at Risk Special: Copyright For Roma: Special: Tobacco Voluntary Work Special: Sports Entertainment

13 | P a g e

Doriana Veshi from the University in Elbasan, who also volunteered for the Roma organization & produced

that radio programme

UNESCO manual on community radio. This manual was distributed to students in Elbasan and in Tirana. The project also provided a radio technician who provided almost individualized training for about 20-25 students who were interested in the technical aspects of radio broadcasting. The study tour exposed the students to university and other media in Germany as a model for professional journalism and social responsibility. The students also met with a member of the German Bundestag /Viola von Cramon-Taubadel). But holding this in the last month of the project (Month 26) instead of in Month 18 as intended limited its programmatic usefulness for the project as the activities had already been completed.

The approach of strengthening CSOs through student volunteers reflected IDEM’s approach and philosophy – that students are active as volunteers in Europe and strengthening them, strengthens the CSOs they volunteer for. IDEM also felt the students would serve as replicators for the training by informing other students. However, if the purpose was to strengthen CSOs as was stated in the project document, this was not an effective approach. It reached the student volunteers, but not in most cases to the organization, and any benefits accrued in terms of strengthening was with the individual students rather than the CSO. Adding the University of Elbasan as an alternative to the University of Tirana seems to have been a strategic choice. It was a relatively large public university with a functioning campus radio and a committed dean with demonstrated interest in citizen journalism. He was also tapped as a project lecturer and although he did not consider himself as such because of the intermittent nature of his work with the project, IDEM/IRIOM saw him as one of the core team.

Project Trainings & Workshops Jan 18 - 21, 2010: Workshop with German media expert on establishing UT radio portal May 25-28, 2010: Partnerships, stereotypes, rights/responsibilities, messaging, 14 persons Tirana May 31, 2010: Discussion July 5, 2010: Discussion June 19, 2010: YouRadio Launch event. Aug 23-26, 2010: ICT, Media & Social Participation of Youth in a Democracy . 14 persons Tirana Sept 21, 2010: Discussion Oct 15, 2010: Discussion Nov 22-25, 2010: Stereotypes, hate speech, storytelling & interview techniques. 14 persons Elbasan Nov 26-29, 2010: Features & reports vs. commentary. 14 persons Tirana Jan-July 2011: Airing of programmes on Ora FM (1/2 hour 1x week) Jun 29 - July 2 2011: New Citizens as Global Player, reporting on social issues. 12 persons Tirana. Oct 10, 2011: Presentation workshop on project activities at Marlin Barleti University Nov 2-5, 2011: Features & reports vs. commentary. 18 persons Elbasan. Nov 23, 2011: Presentation on project activities at Elbasan Dec, 2011: Study tour to Germany

14 | P a g e

The project did not work with other efforts in the sector that might have provided synergies and increased effectiveness. As an example, another UNDEF-funded media project was running in Albania in year 1 of this project.15 It had training programmes on investigatory journalism and professional standards, and produced broadcasts on issues of community and national importance with which this project could have linked to exchange programming and training opportunities. There was also Radio Aktive, an independent radio funded by the National Endowment for Democracy and the Open Society Foundation run by the NGO Mjaff. It was intended to provide citizens a voice and raise awareness on civic and social issues. It aired daily reports from youth, including journalism students from the University of Tirana who served as reporters. It also created an online radio portal “Radiostation.”16 UNICEF also funded “Speak Out” (TROC) which supported younger students in Albania (13-18) to produce a weekly show broadcast on the national public TV station. It trained youth in reporting and supplied technical equipment to 11 bureaus across Albania. They produced 150 reports giving the youth perspective on a broad range of issues.17 There were many synergies that could have been developed between these initiatives as well as with the Albanian media and elected officials as was done in the German study tour.

(iii) Efficiency The efficiency of the project is hard to judge. The final financial report for UNDEF was not done before the evaluation was over. These reports are also at a consolidated line item level which makes it difficult to identify operational and programmatic costs within them. IDEM offered their receipts and bank statements to the evaluation team, but the evaluation timeframe is not sufficient to be able to work at the level of individual receipts. The narrative reports also did not provide enough detail to be able to assess programmatic efficiencies. Project responsibilities and activities were primarily divided between the two part-time staff from IRIOM and the main IDEM technical expert based in Germany. The technical expert was originally from Albania and visited frequently during the project.18 These visits did not necessarily overlap with the training activities. He provided the technical expertise and programmatic vision for the project while the IRIOM staff provided the in-country logistics and continuity of the project on the ground. The IDEM Chair served as the project coordinator and was responsible for project management reports and interfacing with the donor. The technical expert knew IRIOM from earlier work, seemed to have a good working relationship and implemented the project as one team. Outside of a few main service contracts, most of the supervision, reporting and contracting was done informally. For example, experts were hired based on a verbal agreement and paid based on submission of an invoice. Terms of reference for those who had contracts were brief and lacked specificity. IRIOM’s contract was to provide logistical services through its two part time staff but the contract funds were to cover their salary and office/support costs and did not include programmatic operating costs. They seemed to have managed the in-country administration and finances based on guidance and cash advances provided by the main technical expert. The setbacks with broadcasting license and the University of Tirana not filling its role as the second implementing partner made their work more complicated and required more programmatic effort than anticipated in their contracts. Both IRIOM staff were trained as members of the radio’s “core staff.” These programmatic efforts were probably an efficient use of their time, although their contract was not amended to reflect the additional responsibilities and time. They had copies of the project budget for their work and told the

15

UDF-ALB-07-190: Media, Transparency and Accountability with IREX, Hapur and the Union of Albanian Journalists. 16

Open Society Foundation for Albania, Annual Report 2009, p20 17

UN Albania, 2011 One UN Programme Annual Report, p 78 18

Jan. 13-21, 2010; Feb. 16-26, 2010; Mar 20 - 31, 2010, April 7-13; May 28 - June 22, 2010, November 19 - Dec 3, 2010; June 28-July 3, 2011, Sept 13-23, 2011, Octo 27 - Nov 8, 2011; and Nov 22-29, 2011.

15 | P a g e

evaluators they had seen the project summary, but not the results framework or complete programme description which would be expected for an implementing partner. The technical expert also appeared to have put in more work than was anticipated for the same reasons. He also seemed to have taken on the responsibility for supervising and managing the in-country funds and IRIOM staff’s activities even though these were not included in his contract. He provided the two IRIOM staff with cash advances for operating costs which they reconciled on the basis of receipts. The funds for the national experts seemed to have been used more as honoraris than as experts’ fees, but according to IDEM this was in line with the project budget. IRIOM also appeared to efficiently leverage the volunteer system in Albania to get representation for CSOs among university students, and for them to volunteer their time to present their activities at seminars and other events. However, as noted the evaluation is not able to verify the programmatic costs for this project for the reasons already discussed. Reporting systems also appeared to be informal in nature with information passed orally or through e-mails. IRIOM provided 1-2 page activity reports or copies of the agenda to IDEM for the main events, but there seemed to be no system of written quarterly or annual reporting by consultants, contractors, or IRIOM, or by IDEM that captured all of the aspects of the project to that date, tracked project progress, use of funds or activity outputs. There was no performance monitoring plan with the exception of contracting an outside company to undertake the short baseline survey to have measured awareness of YouRadio, and the holding of milestone events which IDEM saw as a monitoring checklist verified by an outside auditing firm contracted by UNDEF. Required reporting to UNDEF was tardy for both the midterm and final reports. This delayed the release of the second tranche of funding and required a time extension in order to complete the project. The inefficiencies in reporting affected project performance as activities lost momentum when pushed back. IDEM still had not submitted its final financial report as of the end of the evaluation. This is required for IDEM to receive its final tranche of project funding. These issues were discussed with IDEM. It felt that it had complied with all of UNDEF’s reporting requirements, had sufficient project management systems in place and had submitted the necessary reporting on its activities with the exception of the financial report which its auditor will prepare. Despite repeated reminders from UNDEF, IDEM stated that it did not realize that they had to submit the final report to UNDEF within a given amount of time and had been busy on another project in the interim. Project efficiency was also affected by not having ironed out the issues with the licensing and participation of the University of Tirana as a project implementer in the design phase. The Dean said she had been pursuing the idea of starting a university TV and radio for students to run when she learned about the IDEM idea for a campus radio station and felt they could be complementary activities. They agreed to work together and the Dean reportedly sent a letter of commitment to IDEM stating her intention to participate in the project. She had also applied for and received a government grant from World Bank funds for 75,000€ but said when no

Sharing information on the project in Elbasan

16 | P a g e

Producers of the YouRadio programmes

company bid on the tender the funds had to be returned to the government. Their participation was also apparently dependent upon being able to set up the radio station which was not possible without a license. The University also felt it did not have reliable enough internet to host an internet portal and radio. As a result, the University of Tirana did not act as an implementing partner as stated in the project document. The project then linked with the University of Elbasan, but it did not use that university in the same implementation manner as intended in the project document with the University of Tirana. Instead it made an individual arrangement with the Dean to participate as a project lecturer and to participate in some of the project’s substantive discussions, and the Dean arranged for some of his university students to participate in the training and programme development. But the campus radio at Elbasan was not used as the hub for the project and project efforts were then split between Tirana and Elbasan and the YouRadio web portal equipment placed in the IRIOM office. Efficiency issues also arise over the low targets set by the project. Organizing training is time consuming and required the development of a manual and training materials which seemed to be a significant level of effort for targets of 20 students and 12 CSOs. IDEM explained that this was an optimum number for a genuine capacity building class. A related issue is having developed the training curriculum and materials before doing a needs assessments of the participants. The project document identified the participants as journalism students and CSO representatives. The project assumed it knew the needs and skill levels of the students from its preparatory discussions and developed a training project around that. This is likely the case for professors and their journalism students, but the training was open to those beyond journalism students and they were unlikely to have had the same level of experience and skills as the journalism students.

(iv) Impact The project is likely to have made an impact on several levels, but the extent of this is unknown due to the scarcity of information. For IDEM, the purpose of the project was to train youth and to give them a voice through the community radio, and other benefits were secondary. Looking at it from that perspective, information in interviews seemed to indicate that there was an increased awareness among participants on how to use a mechanism such as a community radio or social media to deliver a message, how to develop a radio programme and on such practical issues such as how to hold a microphone or to do an interview. For a few, it included increased technical skills on radio broadcasting, such as how to use recording and broadcast equipment. Most of the students who seemed to be part of the core group, said they felt empowered by the process and were using the skills in their school and/or volunteer work. One said it gave him a broader vision of how to address social issues. Another said it increased her awareness on how to act as a responsible citizen regarding social problems. Most of the core participants seemed articulate and active. This was probably a result of the self-selection process for students to participate in the more advanced parts of the training and in the development of the programmes.

17 | P a g e

Students developing their programme

Impact of the YouRadio station and the programmes aired is unknown. The baseline study on the station showed a 46 percent increase in listeners for YouRadio from March 2010 to November 2011, and that 33 percent of these heard podcasts and 20 percent heard them on Ora FM radio. The most preferred shows were those on the economy (14 percent of those who responded), health (13.8 percent), science (10.3 percent), environment (8 percent) and social sector (7.4 percent). The survey did not measure what the listeners had taken away from the programmes, or the differences in awareness/knowledge between project participants and other youth on social/community media or on the social issues raised in the programmes. The project did not have the 2010 or 2011 website statistics so it is not possible to draw inferences on the extent the programmes were accessed or heard during the life of the project, or which ones might have made more of an impact than others. The producers of the programmes felt they made a difference. For example, Alternative, one of the participating CSOs, felt that airing the messages was the first step to sensitize the audience and that this was accomplished by the project. The student volunteer for Youth with a Vision, an NGO focusing on Roma issues, felt that the programmes she developed helped balance the negative view of the Roma in the media by showcasing the accomplishments of Roma artists. This student herself was seen as a role model by IRIOM and IDEM. She was active in social work and promoting equality issues for the Roma people. She participated in the sharing of project experiences at the University of Elbasan and in Tirana by presenting her programmes as did some of the other students and CSO volunteers. Impact on CSOs institutionally was minor as the project targeted the student volunteers and their gains were mostly individual. The CSOs benefited from them in that they developed a programme on their issues but it did not appear to improve their ability to develop a viable communications strategy as anticipated in the project document. They do have access to the YouRadio internet portal which was still functioning from the IRIOM office, but there was no indication that any of the CSOs used the portal beyond the programmes developed for the project. The messages in the programmes might have reached some listeners and changed their attitudes or generated more volunteerism which the project also sought, but as these were not tracked, any changes for those are unknown. Some of the NGOs interviewed felt

their association with the project was beneficial as it had gotten their name out among a wider group of people and they felt they had an increased understanding on the benefits for their organization on the use of social and community media. One said he had learned how to draft a press release that would be picked up by the media by making it interesting and short instead of long and full of facts. He also created a newsletter after the training to help distribute information about his organization’s issues.

The University of Elbasan thought association with the project helped increase the value of their communications programme as it provided activities for students that they could not afford on the school’s budget. This gave their school an edge over other universities which they said already attracted students from all over Albania in addition to students from the region (such as Kosovo and Montenegro). The Marlin Barleti University hopes that being the recipient of the project radio equipment and YouRadio portal will do the same for it. Both

18 | P a g e

universities spoke of incorporating aspects of the project’s training manual and on community radio into their communications curriculum. If this is done, then the principles expressed in the training will be replicated in classes to come. As noted, the targets for the project were extremely modest especially for a training and voice programme that was to include a community radio with reach to the wider community. Aiming so low probably already limits the chances to make a catalytic effect unless the individuals targeted are opinion leaders or agents of change. The project appears to have far exceeded its targets in terms of the general training although the numbers and types of attendees were not tracked or recorded. Impact is likely to have been greater had the project gone as planned, with the programme implemented by and grounded in a university and its community radio. Without it, the activities were scattered and intermittent which limited the project’s potential impact. The audience for internet radio is also limited and users have to be pro-active to access it. The study tour to Berlin and Dusseldorf was at the end of the project. This was too late to benefit the project, but it was an enriching experience for the youth participants. The two participants from Elbasan and three from Tirana felt it gave them an opportunity to see how the media worked outside of Albania. The trip appeared to have given the youth new ideas and insights in to the meaning of independent reporting and citizen journalism and where the media worked in service for the community. Impact on the journalism students is hard to judge as it is unclear how many students reached were journalism students. Many of the participating students were not journalism students as IDEM wanted to open the idea of citizen journalism up to a wider group of students at the universities. There did seem to be an impact made on the Albanian lecturers and journalists who participated in the project. They felt they had learned new training methods from the technical experts who had a more direct and interactive approach than what was done in the universities. One said she intended to include social media in her youth news show on national television and radio.

(v) Sustainability It is difficult to assess issues of sustainability without knowing the extent of the results. However, the knowledge imparted to the students is likely to remain with them, especially for the youth that developed programmes and/or went to Germany. The participating CSOs are also likely to look for opportunities to use citizen journalism and to go beyond the traditional media, but very few will retain any skills transferred as almost all of those trained were their volunteers. The most sustainable results are probably with the University of Elbasan, where the Dean participated in the project, helped develop its messages, and is intending to amend his curriculum to reflect it. This means that future students taking that communications course will receive the messages of the project. He is also actively searching for additional funding to continue activities with his journalism students to give them practical experience, such as what was provided by the project. The YouRadio portal is also likely to remain open and functioning in the near term as it is absorbed by Marlin Barleti University. They already have a room with the radio equipment set up next to a media room where students congregate. They have received the technical training from the project’s expert on how to run the website and upload programming. IRIOM has also maintained the functioning of the YouRadio portal since the end of the project. The equipment was still sitting in the former project offices and the site is still maintained by the webmaster contracted by IDEM.

19 | P a g e

V. Conclusions Based on the evaluation findings, the team concludes:

(i) The project’s focus on increasing citizen journalism and access to media for youth and CSOs was highly relevant and needed within the Albanian context. Doing this through training and the use of a community radio station was also a good approach as it provided the means for communication as well as the skills needed to use it more effectively. This conclusion follows from findings (i) and (ii).

(ii) The project implemented focused almost exclusively on youth and

any strengthening of CSOs would have been achieved primarily as a by-product of the work with CSO volunteers. As CSO strengthening was the key objective in the project document, these elements either needed to have been included in project implementation, or the project document/results framework reworked so that the ultimate purpose for the project and its activities was clear. This conclusion follows from findings (i), (ii), (iv) and (v).

(iii) More pre-project work was needed to determine the feasibility of

creating a community radio within a university and to formalize the relationship with the intended recipient/implementer as these were the basis for the entire project design. Although the activities themselves proved to be flexible enough to be adapted to other implementation circumstances, not being able to do the project as designed affected the project’s relevance, effectiveness, efficiency and impact. This conclusion follows from findings (i), (ii), (iii) and (iv).

(iv) The choice of the provincial university with an existing radio

station provided a credible alternative to the University of Tirana, however, the project did not use it to ground the programme as it had intended to do with the University of Tirana, and instead split its efforts between the two locations. Synergies between project elements were limited and the project worked in isolation from other efforts to strengthen civil society and youth voice. This affected its efficiencies, effectiveness and impact. This conclusion follows from findings (ii), (iii), (iv) and (v).

(v) IDEM provided highly qualified experts but its targets were

extremely low which limited the project’s vision and potential impact. Many of the participants were not journalism students, so the level of expertise needed to train those students was probably at a much lower level than what was provided. Training was also provided without assessing participant skill level or on what basic/advanced information might need and to target the training accordingly. This conclusion follows from findings (i), (ii), (iii) and (iv).

(vi) The project achieved its outputs and exceeded its targets, but its

activity driven nature limited its effectiveness and potential. The outputs were seen as project outcomes and as noted in conclusion (ii) the ultimate purpose for the project beyond these outputs was unclear and differed depending who was asked and between them and the project document. This was a systemic issue that affected design, implementation and reporting. This conclusion follows from findings (i), (ii), (iii), (iv) and (v).

20 | P a g e

(vii) Project management was too informal for an international development project and only worked as well as it did because of the small scale nature of the project and the good personal relations between IDEM and IRIOM, and between them and the experts they recruited. But the lack of formal systems affected the programmatic aspects of the project as well as the administrative/reporting sides. This conclusion follows from findings (ii), (iii) and (iv).

(viii) The project made a difference for the core participants and

participating institutions. It seems to have increased their awareness of citizen journalism, was personally empowering for some students, and created a youth internet radio portal which will be available for students at Marlin Barleti to use in the coming years. However, the extent of this impact is unknown due to the lack of tracking and evaluation data. This conclusion follows from findings (ii), (iv) and (v).

VI. Recommendations To strengthen similar projects in the future, the team recommends:

(i) Similar projects be ground firmly in a university communications

department and become part of the regular university’s programming. As a community radio, the radio should remain open for access by the community as well as by students, by dedicating a certain percentage of airtime for broader community access. The university should be the implementing partner. This recommendation follows from conclusions (i) and (iv).

(ii) Ensure project objectives and outcomes are clear and are

accurately reflected in the project document and results framework. The project purpose should drive the project and guide the selection of activities, participants and locations. All of the activities should contribute directly towards achieving that project purpose. This recommendation follows conclusions (ii) (iv) and (v).

(iii) Fully discuss and negotiate the details for major partnerships

intended in a project before the proposal is finalized and submitted to the donor. The written agreements of understanding that detail the roles and responsibilities of each should be attached as part of the proposal. The feasibility of obtaining major elements required for a project, such as a broadcast license, should also be determined, and the initial steps also initiated to obtain that element before a proposal is submitted for funding. This recommendation follows from conclusion (ii). and iii.

(iv) Ensure synergistic programming within the project so that

activities build on each other and develop links between the different elements of the project. For citizen journalism and community radio, build links with the broader community and ensure synergies with other projects working on the same issues. This will help extend project outreach and impact. This recommendation follows from conclusion (iv).

(v) Ensure project targets are realistic within the means of the project

but also are ambitious enough to justify the project efforts and to make a difference. Ensure a good match between the level of expertise provided and the skills/interest levels of

21 | P a g e

the students and base skill training on the assessed needs of each group, which are likely to vary within groups. This recommendation follows from conclusion (v).

(vi) Where CSO strengthening is an objective, CSO staff should be

included in training opportunities as well as their individual volunteers. The staff adds the institutional element that will help to ensure that the knowledge and skills transferred during training remain with the organization after the end of the project. CSOs are also able to replicate the project activities and use the new skills in their daily work to improve their institutional performance and impact. This recommendation follows from conclusion (ii) and (iv).

(vii) Grantees should ensure that donor-funded projects have basic

project management systems in place that include regular written reporting requirements for all programmatic, contractual and financial/administrative elements. Contracts should be done with all experts and include detailed terms of reference and conditions of employment. Timesheets and/or deliverables should be instituted for consultants so that their contribution to the project can be tracked. These systems should be comprehensive enough that the implementer knows at any point in the process where it is in the project, and how much funds have been used for what programmatic and administrative elements. This recommendation follows from conclusions (v), (vi) and (viii).

(viii) All projects should have a performance monitoring system

put into place from the start of the project that tracks outputs and progress made towards targets. The monitoring and evaluation system should be used by project management to make sure its project is on track and by donors to ensure the grantee is implementing the project as planned. This will allow the project to demonstrate its results as it should measure its outcomes as well as outputs. For projects such as these baseline data should include level of skills and attitudes of students, level of citizen journalism and voice of youth and CSOs, communication skills of CSOs, number of volunteers, awareness on social issues, etc. This recommendation follows from conclusion (viii).

VII. Overall assessment and closing thoughts This project was seen by the implementer as a media training programme for youth interested in communications and social issues based around the development and use of a community radio station. These activities were implemented, but the ultimate purpose for doing this training programme was not clear. Had the project been grounded in a university’s journalism school as intended, the university would have provided this missing element –using the radio for student practice and to improve its journalism training, while providing some outreach capacity to others. As the grantee was offshore and only intended to provide part time technical assistance and training, it needed that engaged national partner who would have used the project activities as part of its larger programme and to contribute towards achieving its institutional vision. IRIOM was a very engaged partner, but its focus was on logistics and ensuring the activities happened. The activities did happen and the training was done, but without the university, or a similar replacement to provide the programmatic grounding for these activities, their results remained primarily as outputs.

22 | P a g e

VIII. Limitations, constraints and caveats The evaluation took place eight months after the end of the project and relied on project documents and interviews to make its assessments. Project reporting was general and the final report lacked specific information on outputs and outcomes. The project’s final financial report was also not available as of the end of the evaluation. As a result the evaluators relied on the activity reports and other pieces of information generated during the project. The evaluators also used information gathered during the interviews to fill in missing information and to base their findings. It was also summer vacation in Albania and most university students were home for the holidays. However, the evaluators were able to reach most of the key participants and stakeholders and the information they provided on the project and its activities was consistent and helped to validate the evaluation’s findings.

23 | P a g e

IX. ANNEXES

Annex 1: Evaluation questions: DAC

criterion Evaluation Question Related sub-questions

Relevance To what extent was the project, as designed and implemented, suited to context and needs at the beneficiary, local, and national levels?

Were the objectives of the project in line with the needs and priorities for democratic development, given the context?

Should another project strategy have been preferred rather than the one implemented to better reflect those needs, priorities, and context? Why?

Were risks appropriately identified by the projects? How appropriate are/were the strategies developed to deal with identified risks? Was the project overly risk-averse?

Effectiveness To what extent was the project, as implemented, able to achieve objectives and goals?

To what extent have the project’s objectives been reached? To what extent was the project implemented as envisaged

by the project document? If not, why not? Were the project activities adequate to make progress

towards the project objectives? What has the project achieved? Where it failed to meet the

outputs identified in the project document, why was this?

Efficiency To what extent was there a reasonable relationship between resources expended and project impacts?

Was there a reasonable relationship between project inputs and project outputs?

Did institutional arrangements promote cost-effectiveness and accountability?

Was the budget designed, and then implemented, in a way that enabled the project to meet its objectives?

Impact To what extent has the project put in place processes and procedures supporting the role of civil society in contributing to democratization, or to direct promotion of democracy?

To what extent has/have the realization of the project objective(s) and project outcomes had an impact on the specific problem the project aimed to address?

Have the targeted beneficiaries experienced tangible impacts? Which were positive; which were negative?

To what extent has the project caused changes and effects, positive and negative, foreseen and unforeseen, on democratization?

Is the project likely to have a catalytic effect? How? Why? Examples?

Sustainability To what extent has the project, as designed and implemented, created what is likely to be a continuing impetus towards democratic development?

To what extent has the project established processes and systems that are likely to support continued impact?

Are the involved parties willing and able to continue the project activities on their own (where applicable)?

UNDEF value added

To what extent was UNDEF able to take advantage of its unique position and comparative advantage to achieve results that could not have been achieved had support come from other donors?

What was UNDEF able to accomplish, through the project, that could not as well have been achieved by alternative projects, other donors, or other stakeholders (Government, NGOs, etc).

Did project design and implementing modalities exploit UNDEF’s comparative advantage in the form of an explicit mandate to focus on democratization issues?

24 | P a g e

Annex 2: Documents Reviewed: Civicus, IDM, UNDP: Civil Society Index for Albania, In Search of Citizens and Impact, 2010 DeliCast, Albania, Radio- Community, http://delicast.com/radio/Albania/community/t:10/3 Freedom House, Nations in Transition: Albania 2012, http://www.freedomhouse.org/report/nations-transit/2012/albania Institute for Democracy, Media and Cultural Exchange, Projects: http://www.idem-institute.org/eng/index.php?l=wwt_pr IREX, Media Sustainability Index 2012, Albania, 2012 Open Society Institute, Mapping Digital Media: Albania, January 2012 Open Society Foundation for Albania, Annual Report 2009 http://www.soros.al/2010/foto/uploads/File/Soros-Anglisht.pdf Progress and Civilization Organization, YouRadio-Citizen Journalism: Evaluation Study Report, December 2011 Radiostacioni, www.radiostacioni.com Reporters Without Borders, World Press Freedom Index 2011-2012, http://en.rsf.org/IMG/CLASSEMENT_2012/C_GENERAL_ANG.pdf UDF-ALB-08-264, Empowering people through citizens’ journalism, Final Project Narrative Report, August 2012 UDF-ALB-08-264, Empowering people through citizens’ journalism, Project Document, September 2009 UDF-ALB-08-264, Empowering people through citizens’ journalism, Activity Report- Development of the radio program II. 2 December 2010 UDF-ALB-08-264, Empowering people through citizens’ journalism, Workshop Agenda, October 2011. UDF-ALB-08-264, Empowering people through citizens’ journalism, Terms of reference/contracts for Dr. Pani, IRIOM, F. Lohmaier UDF-ALB-08-264, Empowering people through citizens’ journalism, copies of newspaper and other advertisements UDF-ALB-08-264, Empowering people through citizens’ journalism, Manual Praktick Per Gazetaret e Radios UDF-ALB-08-264, Empowering people through citizens’ journalism, Presentations for Marlin Barleti 15 October 2011; for University of Elbasan November 23, 2011, UDF-ALB-08-264, Empowering people through citizens’ journalism, Agreement between IRIOM and Marlin Barleti University, April 2012 UDF-ALB-08-264, Empowering people through citizens’ journalism, Activity Report Launch 21 June 2010 UDF-ALB-08-264, Empowering people through citizens’ journalism, copies of newspaper articles on project. United Nations Albania, 2011 One UN Programme Annual Report, http://www.un.org.al/editor-files/file/Onepercent20UNpercent20Reportpercent202011percent20-percent20web.pdf YouRadio, Enjoy your choice, internet portal: http://www.yradio.org/

25 | P a g e

Annex 3: Persons Interviewed 12 August 2012

Arrival, international consultant

13 August 2012

Elda Hallkaj Project staff (programme), IRIOM

Klodiana Collaku Project staff (administrative), IRIOM

Agim Bulku Technical advisor (radio) and trainer

Irida Agolli Coordinator, Alternative Center (NGO)

Visit to IRIOM project office and YouRadio set up

Adriana Berberi Managing Director University of Marlin Barleti

Dr. Artan Puto Director Albanian Institute for Public Affairs, University of Marlin Barleti

Erion Kristo PR Students Union Office, University of Marlin Barleti

Silio Stefani Radio Technician, University of Marlin Barleti

Visit to University of Marlin Barleti’s, Communications department and radio room equipped by project

14 August 2012

Departure for Elbasan

Dr. Hektor Ciftja Head, Department of Journalism, University of Elbasan, Project consultant/expert/trainer

Visit to University of Elbasan Journalism department and campus radio

Doriana Veshi Project participant and volunteer at “Youth for a Vision (NGO)

Aldo Dervishi Project participant and volunteer for “Different Visions” (NGO).

Return to Tirana

15 August 2012

Arian Boci Director, “Stop AIDS” (NGO)

Enisa Karaxho Project participant and volunteer at Albanian Center for Population and Development (NGO)

Ilva Vjero Project participant and student volunteer at CEPAL and Alternative (NGOs).

Dr. Pandeli Pani IDEM technical expert for project

16 August 2012

Pranvera Bekteshi Director: Protection and Preservation of National Environment in Albania and leader for the study tour to Germany

Blendi Salaj Radio journalist, Club FM, and lecturer Marlin Barleti University

Dr. Dritan Kamani Director, Action + (NGO)

Aulona Fishta Legal Advisor for IDEM for radio licensing

Helind Demaj Web Designer and maintenance web portal for YouRadio

Elda Hallkaj Project staff (programme), IRIOM

Klodian Collaku Project staff (administrative), IRIOM

17 - 24 August 2012

Elena Cherniavska Director, IDEM and Project Coordinator

Kozeta Kurti Editor, Ora News and a project trainer

Kleta Dibra Dean Department of Philology, University of Tirana

Klaudia Bani Participant, Student at University of Tirana

26 | P a g e

Annex 4 : Acronyms CSO Civil Society Organization IDEM Institute for Democracy, Media and Cultural Exchange NGO Non-Governmental Organization UNDEF United Nations Democracy Fund UNICEF United Nations International Children’s Emergency Fund