Lessons

Learned

Turkey

Intersector and Multisector Assessment for HNO 2017

October 2016

Inter/Multi-sector

Assessment

Premise

(i)

Executive Summary

Acknowledgement

(i)

Table of Contents

(ii)

Timeline & Process Management

An early start of HNO process (April)

Agreement by all hubs on major milestones

Consultative process and highly inclusive approach led to buy-in by all hubs

Idea of dedicated workstreams and taskforces helped brought special-ized resources and developed niche market.

Coordination and partners cover-age mapping was good.

Preparation for the questionnaire and the tools (translation of the materials, questionnaire, kobo form, etc.) took considerable time and as a result the data collection process was delayed by 10 days.

The kobo tool was not ready for training and kobo form was updat-ed during the data collection pro-cess which impacted some of the partners (re-entered data)

Some parts of the questionnaire were not translated to address the Syrian dialect.

The process of change of admin PCODE shortly before assessment was not managed and communi-cated well to the partners col-lecting data.

Data collection timeline was not consistent for all hubs.

More coordination is required with actors as early as possible. Timeline needs to be communicated in a better way to partners.

Tools and materials should be available before the start of train-ing

Tools to be piloted before data collection start.

Any changes in the questionnaire (questions, answers, options, etc.) and kobo collect form after the start of data collection process needs to be effectively communi-cated and followed up.

Page 1

Trainings (Face to Face—Online)

Documents and Materials

Questionnaire guidelines helped in explaining each question to the enumerators and getting right in-formation.

Material provided during the train-ing in both Arabic and English helped the participants. Good ex-planation was provided during the training.

Data cleaning training was helpful.

Some partners shared training vid-eos on youtube with their enumer-ators.

Experienced enumerators were used for inter sector / multi sector assessments and other also gained experience.

All materials are available and ac-cessible.

The ToT training was not conduct-ed according to the planned dates which resulted in a ripple effect on further trainings in the field.

Most of the enumerators only speak Arabic.

Trainings in two languages simulta-neously consumed more time.

Training time for master trainers and enumerators wasn’t enough and also because of Eid holidays not all partners could join the train-ing.

Draft questionnaire was used for training and there were delays in sharing of technical document and final tools.

Translation was not tuned to ad-dress Syrian dialect and the feed-back/correction provided multiple times was not incorporated.

Use of real examples during the trainings.

Add an explanation for each ques-tion on kobo collect form

Provide training videos

In addition to explain the question include sessions on:

How to introduce yourself to the key informant.

How to coordinate with the local authorities.

Materials should be available in both languages (Arabic and Eng-lish); however the training should be conducted in one language at a time.

Simplify the questionnaire and questions as much as possible.

Consider conducting trainings in-side Syria and use training facilities on Turkey-Syria border.

Analysis workshop to be conducted and include partners who collected data.

Mentoring and visit to the part-ners during their enumerators’ trainings is suggested.

Page 2

Data Collection, Field Management, Technical Support & Tools

Partners intensively contributed in the HNO.

Use of technology for data collec-tion (mobile tool and web browser tool).

Trainings for data collection and cleaning helped partners to per-form better.

Kobo Collect Form Tool is very use-ful.

Enumerators got more experience after this exercise at community level.

Enhanced coordination among the units in some of the organization.

Merging the data collection and data entry phases helped in saving time.

The delay in start of the data col-lection effected the coverage of partners.

Data collection started directly without proper reviewing and pi-loting the tools and many issues were reported by the partners dur-ing data collection.

Change in KoBo collect form during the data collection exercise created confusion.

Training on Kobo was very short and was not really done using the final questionnaire. Link to access the KoBo form came very late which affected further training to the field enumerators and familiar-izing the questionnaire.

Some of the enumerators faced issues with KoBo collect form dur-ing data collection (e.g. data loss)

Challenges were faced in getting clarity on questionnaire and KoBo collect tool.

The questionnaire was very long and detailed (some interviews took around 4 hours).

The Kobo collect Form on mobile was stopping most of the time.

Regular training are recommended for assessments.

Questionnaire needs to be short and better usage of secondary data is recommended .

When planning for an assessment of this scale, allocation of sufficient time for training using final ques-tionnaires and tool would help in identifying problems early.

Considering that not all enumera-tors in the field are familiar with KoBo tool for data entry, in-depth trainings are required for both master trainers and enumerators.

In many cases, master trainers pro-vided training to the field enumer-ators remotely. Use of youtube videos and documents explaining methodology and steps are recom-mended.

In logistical planning, consider in-clusion of travel time and cost for the enumerators who has to visit communities to interview KIs.

In trainings, include practical exam-ples that would help enumerators in better reconciliation of infor-mation and accurate reporting of information

Page 3

Data Collection, Field Management, Technical Support &

Tools

Problems were reported from the field about slow loading of the form in KoBo collect, validation rules and skip logic. When entering data through mobile, KoBo form crashed many time which resulted delays in data entry.

The internet connection was very slow inside Syria which delayed the process of data entry especially in the inner governorates.

Limited hints for questions provid-ed in the KoBo tool was not suffi-cient for some questions. It is rec-ommended to include elaborated hints on KoBo or on the question-naire itself.

In many cases, enumerators had to travel to the communities to inter-view KIs. Travel cost was not pro-vided to the enumerators.

In some locations, different key informants reported inconsistent figures (SARC, Local councils and NGOs) which was difficult to recon-cile and report.

Separate the humanitarian cluster questions in different forms.

List of locations where partners initially committed were reduced during the data collection phase because of delay in sharing tools.

Not enough time for data verifica-tion was available.

More attention to be paid to open ended answers.

A huge challenge remains for such a large scale assessment involving multi-hubs, many partners and multiple modalities to clearly antic-ipate the timeline, plan every steps and timely communicate to focal person in the hubs, the partners and further to the enumerators collecting data. Any changes in timeline, sharing updated tools, fixing problems had to go through chain of communication and time required for this should be consid-ered in future assessments.

The questionnaire was very long.

The Arabic version on Kobo didn’t match the English version.

Some of the organizations faced problem with the assigned codes.

Kobo design was not good in gen-eral (skip logic was not correct for some of the questions and had many validation problems).

The inherent KoBo design allowed collection of sensitive information like GSM number and SIM number of the mobiles. Security concern was raised by the partners as it was done without informing them.

Online form was not working properly entering while the name of communities.

Use of multiple versions of online tools let to some missing data and challenges in managing data entry and cleaning.

Consider creating questionnaire in Arabic from the beginning and then translating into English.

Give enough time for testing the tools in advance.

Consider adding team leader code and provide edit facility to the team leader using the mobile.

Creating iOS app for Kobo Collect.

Explore other options of data col-lection tools. Provide trainings in using various mobile data collec-tion tools: like ONA, Kobo Collect, Formhub etc.

Use matrix question type in XLS form and KoBo

Debriefing sessions at major mile-stones.

Adequate explanation of question-naire and many homogenize an-swers.

Deleting or editing the log file to collect less information.

Create a shared folder with all part-ners to put the daily problems and how they address to record.

Page 4

Data Granularity/Level

Data was collected at lowest possi-ble granular level consistently by all partners.

Some of the partners also collected data at household level.

Analysis of data collected at differ-ent granular levels was complicat-ed for the sectors.

According to some of the partners contributing to UCP, UFA was cre-ated based on criteria that didn’t take the current situation and con-trol by different actors.

Page 5

Methodology

This year methodology was more comprehensive based on different modalities and inclusion of multiple partners from all hubs.

Lack of experience for conducting Household and FGD data collection process.

Household & Focus Group Discus-sion data collection methods were not fully understood by partners.

The sample size for Household and FGD were not clear.

The questions in the Household questionnaire were not suitable for household interviews; they were more suitable for key informant questionnaire.

The concept of weighting vs sam-ple size and coverage was not clear.

Some of the participants highlight-ed that household methodology to be explored more in addition to key informant methodology.

Some UCP question were not rele-vant for example: water testing was not valid in HTR and besieged areas due to the lack of testing equipment.

More training is suggested for household data collection & focus group discussion methods.

Improve the planning for training, data collection and methodology.

In case of use of different modali-ties the type of question should match the type of methodology.

Since most of the areas can be reached physically, consider data collected through face to face only.

Page 6

Coordination

Good level of coordination with around 45 organizations in Turkey hub.

Continuous liaison with regional office.

Not enough time for coordination and communication with large number of partners (changes in tools, follow up with partners and partners following up with the field teams, etc.).

Bad internet and mobile coverage, in addition to non-availability of fuel in some of the locations inside Syria led to communication delays.

Multiple lines of communication to Turkey based partners led to con-fusion.

Some of the partners suggested leading the assessment process to ensure better coordination among partners.

Since the Turkey hub had the most of the work and performance, the next assessment should be led by Turkey hub.

Partners suggested debriefing at regular interval for major mile-stones.

Page 7

Workshop Agenda

Page 8

For feedback & clarifications: ochaturkey@un.org