2014.06.01 gwc evaluation final report d8€¦ · 8*...

27
Global WASH Cluster Evaluation of the Support Provided to National Coordination Platforms In collaboration with: DRAFT Final Report Date of submission 7 May 2014

Upload: others

Post on 01-Jun-2020

0 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: 2014.06.01 GWC Evaluation Final Report D8€¦ · 8* Table’2–’Diversity’of’Context’in’Evaluation’Case’Studies’ * * * * * * * * * * * * support support)

       

   Global  WASH  Cluster  Evaluation  of  the  Support  Provided  to  National  Coordination  Platforms    

   In  collaboration  with:    

   DRAFT  Final  Report             Date  of  submission                     7  May  2014  

Page 2: 2014.06.01 GWC Evaluation Final Report D8€¦ · 8* Table’2–’Diversity’of’Context’in’Evaluation’Case’Studies’ * * * * * * * * * * * * support support)

GWC  Evaluation  of  Support  to  National  Coordination  Platforms    Copyright  ©  Avenir  Analytics,  2014  

 

ii  

Acknowledgements  

 Avenir  Analytics  thanks  the  many  organizations  and  individuals  that  willingly  contributed  their  time  and  insights  to  make  this  evaluation  of  Global  WASH  Cluster  (GWC)  support  to  national  coordination  platforms  possible.    Particular  gratitude  goes  to  Paul  Shanahan,  the  GWC  Coordinator  and  Franck  Bouvet  the  GWC  Deputy  Coordinator,  members  of  the  GWC  Strategic  Advisory  Group,  current  and  past  members  of  the  GWC  Rapid  Response  Team,  Rapid  Assessment  Team,  Regional  Emergency  Coordination  Advisors  and  particularly  Leilia  Dore  and  Damien  Brosnan.  Their  insights,  support  and  guidance  have  been  extremely  helpful  throughout  the  process.      Ben  Allen,  the  ACF-­‐UK  Evaluations,  Learning  &  Accountability  Manager,  provided  critical  support,  collaboration  and  guidance  throughout  the  process  and  deserves  special  thanks  for  his  expert  contributions  and  sound  advice.  

Disclaimer  

 The  statements  of  findings  and  conclusions  in  this  report  represent  the  best  judgment  of  the  evaluation  team  based  on  careful  analysis  of  available  data  and  do  not  necessarily  represent  the  opinions  of  ACF  International,  the  GWC  and  its  members  or  UNICEF,  which  serves  as  the  Cluster  Lead  Agency  for  WASH.  Responsibility  for  the  content  of  this  report  rests  solely  with    Avenir  Analytics.                

                   

     

www.aveniranalytics.com  

Page 3: 2014.06.01 GWC Evaluation Final Report D8€¦ · 8* Table’2–’Diversity’of’Context’in’Evaluation’Case’Studies’ * * * * * * * * * * * * support support)

GWC  Evaluation  of  Support  to  National  Coordination  Platforms    Copyright  ©  Avenir  Analytics,  2014  

 

iii  

Table  of  Contents  

   Acknowledgements  .............................................................................................................................  ii  Disclaimer  ............................................................................................................................................  ii  Table  of  Contents  ................................................................................................................................  iii    Executive  Summary  ..............................................................................................................................  1    1.   Introduction  .................................................................................................................................  4  

1.1   Objective  ........................................................................................................................  4  1.2   Evaluation  Questions  .....................................................................................................  4  1.3   Approach  and  methodology  ..........................................................................................  5  

 2.   Findings  ........................................................................................................................................  7  

2.1   Overall  Findings  ..............................................................................................................  7  2.2   Relevance  /  Appropriateness  .........................................................................................  8  2.3   Effectiveness  ................................................................................................................  10  2.4   Efficiency  ......................................................................................................................  12  2.5   Coherence  /  Connectedness  ........................................................................................  17  2.6   Coverage  ......................................................................................................................  18  2.7   Sustainability  ................................................................................................................  18  

 3.   Conclusions  ................................................................................................................................  20    4.   Recommendations  .....................................................................................................................  21    5.   List  of  Annexes  ...........................................................................................................................  23    6.   Acronyms  Used  ..........................................................................................................................  24    

Page 4: 2014.06.01 GWC Evaluation Final Report D8€¦ · 8* Table’2–’Diversity’of’Context’in’Evaluation’Case’Studies’ * * * * * * * * * * * * support support)

GWC  Evaluation  of  Support  to  National  Coordination  Platforms    Copyright  ©  Avenir  Analytics,  2014  

 

1  

Executive  Summary      This  evaluation  was  commissioned  by  the  Global  WASH  Cluster  (GWC)  coordinator  and  ACF-­‐UK  as  lead  for  learning  and  evaluation  in  consultation  with  the  GWC  Strategic  Advisory  Group  (SAG).  It  seeks  to  generate  credible  evidence  of  results  achieved  or  not  achieved  by  the  cluster’s  Field  Support  Team  (FST)  in  support  of  national  WASH  coordination.    The  GWC  initiated  its  global  surge  capacity  systems  in  2007  to  support  coordination  efforts  during  emergencies  with  the  Rapid  Response  Team  (RRT).  With  the  support  of  different  consortia  of  members,  the  RRT  has  been  complemented  by  a  Rapid  Assessment  Team  (RAT)  and  Regional  Emergency  Cluster  (Coordination)  Advisors  (RECA).  Total  GWC  standing  surge  capacity  has  grown  from  3  to  as  many  as  18  personnel  over  the  last  7  years  and  an  effort  is  underway  to  integrate  these  mechanisms  under  the  FST.  Notwithstanding  this  dedicated  investment,  GWC  capacity  to  meet  the  support  needs  of  national  coordination  platforms  stretched  in  2013  and  2014  with  4  simultaneous  Level  3  emergencies.        This  evaluation  used  systematic  interviews,  observation,  focus  groups  and  document  review.  Inquiry  focused  on  6  criteria:  relevance  /  appropriateness,  effectiveness,  efficiency,  coherence,  coverage  and  sustainability.  The  evaluation  focused  on  4  critical  case  studies  representing  the  L3  emergencies  in  2013/2014.    Key  Findings    Overall  Overall  the  evaluation  found  the  GWC’s  support  to  national  coordination  platforms  to  be  strong.  

Across  all  28  evaluation  indicators,  26  found  average  responses  of  ‘adequate’  to  ‘good’  based  on  key  informants,  documentary  evidence  and  observation.    The  only  indicators  that  revealed  significant  weaknesses  related  to  the  ability  to  determine  the  relationship  between  FST  contributions  and  response  outputs  (assessment  speed)  and  outcomes  (delivering  relief).    The  dependency  of  the  FST  on  pre-­‐existing  coordination  structures  and  partner  capacities  explained  these  weaknesses.    While  average  responses  to  key  evaluation  questions  showed  strength,  there  was  considerable  variability  across  case  studies  and  key  informants.    This  variability  was  driven  by  significant  differences  in  country  context,  type  of  emergency,  pre-­‐existing  capacity  and  the  experience  of  FST  personnel  deployed.    Relevance/Appropriateness  GWC  efforts  to  support  national  coordination  platforms  are  well  aligned  with  IASC  standards,  including  recent  efforts  of  the  Transformative  Agenda  (TA).  88%  of  2013/2014  GWC  surge  capacity  deployments  were  focused  on  the  4  IASC  declared  L3  emergencies.  FST  mechanisms  have  also  allowed  the  GWC  to  support  preparedness  and  capacity  building,  increasing  GWC  relevance  to  national  platforms.    The  challenge  of  guiding  vs.  being  responsive  to  requests  from  the  national  level  presents  a  relevance  dilemma.  This  was  highlighted  in  efforts  of  the  FST  to  identify  risks  and  capacity  gaps,  which  some  felt  could  be  perceived  as  shifting  to  an  accountability  focus.    Effectiveness  GWC  support  has  been  broadly  effective  in  improving  coordination  and  preparedness.  

Page 5: 2014.06.01 GWC Evaluation Final Report D8€¦ · 8* Table’2–’Diversity’of’Context’in’Evaluation’Case’Studies’ * * * * * * * * * * * * support support)

GWC  Evaluation  of  Support  to  National  Coordination  Platforms    Copyright  ©  Avenir  Analytics,  2014  

 

2  

National  and  global  cluster  members  unanimously  agree  FST  support  is  equally  available  to  all  sector  actors.      The  RECA  project  has  increased  GWC  effectiveness  in  its  focus  countries  and  use  of  RECAs  in  a  few  responses  shows  cross-­‐over  value  beyond  the  project’s  scope.  RRTs  were  broadly  viewed  as  effective  in  augmenting  coordination  capacities,  with  some  variance  based  on  the  experience  and  soft  skills  of  members.    The  RAT  project’s  effectiveness  was  most  frequently  critiqued,  largely  based  on  divergent  views  on  its  purpose  and  management  structure,  though  national  key  informants  also  reported  some  highly  effective  RAT  deployments.    Information  Management  (IM)  approaches  were  also  critiqued,  though  most  often  linked  to  a  broader  dilemma  related  to  the  upstream  information  demands  compared  with  the  actionable  response  information  national  actors  value  most.    The  GWC  Cluster  Advocacy  and  Support  Team  (CAST)  also  mobilizes  significant  support  beyond  the  FST  to  support  national  coordination  during  emergencies.  From  January  2013  to  April  2014  30%  of  GWC  deployments  came  from  other  sources  (e.g.  stand-­‐by  partners).    Efficiency  The  GWC  provides  strong  value  for  money  through  its  support  to  national  coordination  platforms  based  on  analysis  of  cost,  quality  and  speed.  While  48%  of  total  FST  time  recorded  was  spent  on  emergency  deployments,  significant  time  was  also  dedicated  to  remote,  regional  and  global  support  for  preparedness  and  capacity  building.    

The  overall  cost  structure  of  the  FST  provides  good  value  for  money,  largely  because  it  relies  on  NGO  personnel,  which  makes  it  60%  less  costly  than  a  comparable  UN  staff  surge  model.  However,  the  fragmented  evolution  of  the  RRT,  RAT  and  RECA  under  different  management  structures  has  led  to  significant  variance  in  budgeted  cost  models.  Combined  with  the  days  of  work  recorded  within  FST  tracking  systems,  this  variance  in  budget  models  leads  to  a  significant  difference  in  measured  efficiency,  with  RRTs  being  most  cost-­‐efficient  and  RECAs  being  least  cost  efficient.  The  cost-­‐efficiency  of  RRTs  comes  with  a  reported  risk  –  that  of  a  high  risk  of  burnout  and  turnover.    FST  deployments  were  found  to  be  timely  as  measured  against  the  IASC  L3  declarations.  In  all  4  cases  studied  the  GWC  had  surge  capacity  staff  in  country  before  the  IASC  declaration.    Ongoing  efforts  of  the  GWC  to  further  integrate  the  components  of  the  FST  show  potential  for  further  increasing  the  efficiency  and  effectiveness  of  GWC  support  to  national  coordination  platforms.    Coherence/Connectedness  The  GWC  has  generally  demonstrated  strong  inter-­‐cluster  and  intra-­‐cluster  coordination  to  achieve  coherence  and  connectedness.  FST  members  mostly  showed  an  understanding  and  respect  for  pre-­‐existing  coordination  structures  where  they  were  present,  including  national  government  authorities  where  they  were  able  to  work  with  the  humanitarian  system.        Coordination  within  national  platforms  was  enhanced  by  the  experience  and  diversity  of  FST  personnel,  which  come  from  various  major  WASH  actors  not  just  the  UN  system.      

Page 6: 2014.06.01 GWC Evaluation Final Report D8€¦ · 8* Table’2–’Diversity’of’Context’in’Evaluation’Case’Studies’ * * * * * * * * * * * * support support)

GWC  Evaluation  of  Support  to  National  Coordination  Platforms    Copyright  ©  Avenir  Analytics,  2014  

 

3  

Coordination  with  other  clusters/sectors  was  enhanced  by  the  experience  and  relationships  of  FST  personnel,  but  weaknesses  still  exist  in  the  broader  effectiveness  of  country  level  inter-­‐cluster/sector  coordination  mechanisms,  which  is  beyond  the  control  of  the  FST.    Connectedness  between  the  GWC  and  UNICEF  as  the  CLA  is  strong  at  global  levels  but  linkages  with  UNICEF  country  offices  is  variable  and  can  lead  to  challenges  in  deploying  FST  members.    Coverage  GWC  coverage  of  the  needs  of  national  coordination  platforms  was  found  to  be  strong  in  case  study  countries  during  emergencies.    FST  deployments  strengthened  the  capacity  of  national  and  sub-­‐national  coordination  structures  through  augmentation,  coaching  and  training.  Coverage  of  technical  advisory  needs  was  more  variable,  though  this  is  at  least  partly  driven  by  different  perspectives  on  the  relative  importance  of  coordination  competencies  vs.  technical  competencies.    One  dilemma  regarding  coverage  relates  to  country  level  awareness  and  acceptance  of  GWC  support.  Key  informants  noted  that  many  existing  coordination  platforms  and  key  WASH  actors  are  not  familiar  with  the  support  available  from  the  FST.  This  relates  to  the  challenges  for  the  GWC  in  establishing  a  balance  between  a  pro-­‐active  vs.  reactive  stance  and  coordination  between  the  GWC,  national  coordination  platforms  and  various  levels  of  the  Cluster  Lead  Agency  (CLA).    Sustainability  FST  deployments  were  found  to  include  strategies  for  transition  and  sustainability,  though  these  plans  can  be  undermined  by  

difficulties  in  recruitment,  emergency  context  and  funding.    The  lasting  impacts  of  FST  deployments  were  difficult  to  assess  in  most  cases  given  the  ongoing  nature  of  the  emergencies  and  depend  heavily  on  partners  that  remain  in  a  country.    The  sustainability  of  support  for  the  FST  structure  was  identified  as  a  potential  concern.  While  ongoing  efforts  to  further  integrate  the  FST  are  seen  as  having  likely  positive  effects  on  efficiency  and  quality  of  support,  a  number  of  key  risks  were  cited  related  to  maintaining  partner,  donor  and  CLA  support  for  the  FST.    Alternative  funding  models,  such  as  a  partial  cost-­‐recovery  system,  have  been  suggested  but  not  fully  analyzed  to  date.    Recommendations      1. Proceed  with  the  proposed  further  

integration  of  the  FST  under  unified  management  of  the  CAST.  

2. Analyze  the  potential  for  implementing  a  partial  cost  recovery  model  and  other  mechanisms  to  increase  financial  sustainability  of  the  FST.  

3. Strengthen  adherence  to  a  minimum  staff  reporting  structure  to  enable  unified  management  and  value  for  money  analysis.  

4. Continue  to  closely  collaborate  with  UNICEF  as  CLA  to  facilitate  coordination  preparedness,  capacity  building  and  other  approaches  to  enhance  continuity.  

5. Develop  a  Theory  of  Change  and  monitoring  and  evaluation  plan  to  accompany  the  next  GWC  strategy,  including  potential  use  of  social  network  analysis.    

Page 7: 2014.06.01 GWC Evaluation Final Report D8€¦ · 8* Table’2–’Diversity’of’Context’in’Evaluation’Case’Studies’ * * * * * * * * * * * * support support)

GWC  Evaluation  of  Support  to  National  Coordination  Platforms    Copyright  ©  Avenir  Analytics,  2014  

 

4  

1. Introduction    In  2007,  the  GWC  established  the  Rapid  Response  Team  (RRT)  with  three  WASH  Cluster  Coordinators  (WCC)  to  provide  initial  support  to  National  Coordination  Platforms  during  emergencies.      This  team  was  supplemented  in  2009  by  a  Rapid  Assessment  Team  (RAT)  of  3  persons  that  support  partners  by  carrying  out  entire  emergency  assessments  in  the  early  days  of  an  emergency.  RAT  members  are  hosted  by  3  partners  (CARE  USA,  Oxfam  GB  and  IFRC)  grouped  in  a  consortium,  and  until  recently  were  directly  managed  by  the  consortium  itself.      Additionally,  in  2009  6  Regional  Emergency  Cluster  (Coordinator)  Adviser  positions  were  created  by  a  consortium  (of  ACF,  IFRC,  NCA,  CARE  Germany/Luxemburg  and  TearFund)  in  order  to  operationalize  the  capacity  building  component  of  the  GWC  Strategic  Plan.      In  2013,  the  GWC  partners  consolidated  these  3  teams  (RRT,  RAT  and  RECAs)  under  one  Field  Support  Team  (FST),  and  by  the  end  of  2013  these  units  had  provided  the  following  support  to  national  coordination  platforms  for  emergencies,  preparedness  and  capacity  building:  • RRT:  support  to  16  countries  for  an  

equivalent  of  2,708  person/days.  • RAT:  support  to  10  countries  for  an  

equivalent  of  333  person/days.  • RECAs:  support  to  12  countries  for  an  

equivalent  of  908  person/days.      

The  GWC  has  been  under  extreme  pressure  in  providing  surge  support  to  National  WASH  Coordination  Platforms  during  2013  and  2014  because  of  the  simultaneous  triggering  of  4  Level  3  (L3)  emergencies  in  2013  (Syria,  the  Philippines,  South  Sudan  and  Central  African  Republic  (CAR))  in  addition  to  2  on-­‐going  L2  emergencies  (Somalia,  Mali).      This  context  of  geographic  diversity,  strained  resources,  and  diversity  of  emergency  type  is  an  excellent  opportunity  for  the  GWC  to  evaluate  the  strategy  and  structure  of  the  FST  as  it  supports  National  WASH  Coordination  Platforms  in  the  framework  of  the  Transformative  Agenda  (TA),  the  IASC  operational  guidance  on  cluster  coordination,  and  UNICEF’s  Cluster  Lead  Agency  (CLA)  mandate.    1.1 Objective    The  broad  evaluation  objective  was  to  generate  credible  evidence  of  results  either  achieved  or  not  achieved  by  the  FST  in  providing  support  to  National  WASH  Coordination  Platforms.  Within  this  context,  the  primary  intended  use  for  this  evaluation  is  to  inform  the  new  GWC  Strategic  Plan  for  2016-­‐2020  and  guide  the  GWC  to  provide  more  effective  support  to  National  WASH  Coordination  Platforms.      1.2  Evaluation  Questions    The  primary  analytic  framework  for  this  evaluation  is  the  OECD/DAC  criteria,  operationalized  in  the  Terms  of  Reference  by  a  series  of  eight  focal  questions  as  follows:      

   

Page 8: 2014.06.01 GWC Evaluation Final Report D8€¦ · 8* Table’2–’Diversity’of’Context’in’Evaluation’Case’Studies’ * * * * * * * * * * * * support support)

GWC  Evaluation  of  Support  to  National  Coordination  Platforms    Copyright  ©  Avenir  Analytics,  2014  

 

5  

   Table  1  –  Evaluation  Questions    

     

OECD/DAC  Criteria:     Focal  Question(s):    Relevance/  Appropriateness:    

1. How  closely  aligned  is  the  support  from  CAST  and  the  FST  to  National  WASH  Coordination  Platforms  with  the  principles  and  standards  found  in  the  IASC  Cluster  Coordination  Reference  Modules?    

    2. How  closely  is  FST  support  aligned  with  in-­‐country  coordination  needs?    Effectiveness:     3. To  what  degree  has  CAST  and  FST  contributed  to  improved  coordination,  

preparedness  and  response  through  the  support  provided  to  countries?  Efficiency:     4. To  what  extent  can  the  GWC  show  value-­‐for-­‐money  for  their  surge  capacity  

systems  within  the  ‘balanced  scorecard’  approach  using  the  dimensions  of  cost,  quality  and  speed?  

    5. To  what  extent  has  the  FST  proactively  identified  the  most  cost-­‐effective  means  of  achieving  human  resources  and  training  needs  of  WASH  Coordination  Platforms?  

Coherence/  Coordination:    

6. How  clearly  have  the  FST  support  mechanisms  demonstrated  inter-­‐cluster  and  intra-­‐cluster  coordination?  

Coverage:     7. How  well  do  the  FST  mechanisms  enable  coverage  of  country  coordinator  positions  and  functions  as  well  as  short-­‐term  technical  needs?    

Sustainability:     8. How  well  have  FST  members  strengthened  in-­‐country  coordination  and  technical  capacity  and  ensured  a  smooth  transition  to  ongoing  leadership?    

 

1.3 Approach  and  methodology    Design  The  evaluation  design  followed  the  general  approach  outlined  in  the  terms  of  reference,  primarily  involving  qualitative  methods  of  interviewing  and  document  review,  supplemented  by  a  ‘balanced  scorecard’  for  cost  analysis  to  address  questions  related  to  efficiency.  The  approach  balanced  both  global-­‐level  and  country-­‐level  data  collection  and  inquiry  in  order  to  identify  credible  evidence  of  operational  results  and  maximize  the  usefulness  of  those  findings  within  the  context  of  a  global  mandate.  The  four  level  three  emergencies  of  2013  (South  Sudan,  Philippines,  Central  African  Republic,  Syria)  were  selected  as  the  focus  for  the  country-­‐level  data  for  several  reasons:  since  they  are  recent  (and  in  3  cases  on-­‐going)  the  feedback  from  key  informants  is  more  reliable  and  less  subject  to  memory  loss,  they  include  both  natural  disasters  and  complex  emergency  contexts,  represent  relatively  broad  geographic  

contexts,  and  comprise  a  natural  ‘stress  test’  scenario  of  responding  to  four  disasters  simultaneously.      Data  Collection  The  primary  data  collected  for  this  evaluation  included  documents  and  key  informant  interviews,  supplemented  by  feedback  provided  during  small  group  discussions  during  the  GWC  annual  meeting  in  Oslo  in  April  2014.  As  listed  in  Annex  3,  a  total  of  187  documents  were  reviewed,  which  included  both  global  and  country-­‐level  data  intentionally  sampled  to  provide  both  complimentary  and  supplementary  perspectives  to  the  key  informant  interviews.  A  total  of  38  key  informants  were  interviewed,  including  23  from  the  global  level  (CAST,  SAG,  RRT,  RAT,  members,  donors  and  CLA)  and  15  from  the  country  and  regional  levels.          

Page 9: 2014.06.01 GWC Evaluation Final Report D8€¦ · 8* Table’2–’Diversity’of’Context’in’Evaluation’Case’Studies’ * * * * * * * * * * * * support support)

GWC  Evaluation  of  Support  to  National  Coordination  Platforms    Copyright  ©  Avenir  Analytics,  2014  

 

6  

Preliminary  findings  from  this  evaluation  were  presented  at  the  annual  meeting  of  the  GWC  in  Oslo  in  April  2014.  Involvement  in  the  meeting  served  three  goals:    i)  the  initial  findings  contributed  to  the  ongoing  discussion  about  refining  the  GWC  strategy,  ii)  sharing  initial  findings  facilitated  a  ‘member  check’  of  the  findings  as  part  of  an  ongoing  analysis  process,  and  iii)  conducting  small  group  discussions  allowed  a  wider  group  of  participants  to  provide  feedback  than  would  otherwise  be  possible.      The  evaluators  also  observed  the  full  GWC  meeting  in  Oslo,  led  a  short  group  discussion  during  a  follow-­‐on  FST  meeting  in  Oslo  and  observed  the  remainder  of  this  meeting.    Analysis    Data  workbooks  were  developed  in  Excel  to  systematically  compile  the  notes  from  the  key  informant  interviews  and  excerpts  from  the  document  review.  A  total  of  363  excerpts  were  collected  from  the  document  review  and  coded  to  analyze  the  data  within  the  evaluation  criteria.      Notes  from  the  key  informant  interviews  were  recorded  by  indicator,  and  analyzed  within  a  level  (country  and  global)  and  across  levels  to  identify  overall  themes  and  situate  the  findings  within  a  relevant  and  useful  context.  These  interview  notes  were  scored  on  a  0-­‐4  scale  in  order  to  systematically  condense  the  data  from  the  28  indicators  into  the  6  criteria  (see  Annex  7  Evaluation  Matrix),  and  the  coding  and  overall  themes  identified  were  checked  and  discussed  by  both  members  of  the  evaluation  team  in  order  to  maximize  the  rigor  of  the  analysis.      The  initial  findings  were  shared  and  generally  affirmed  during  the  GWC  annual  meeting  in  Oslo  in  April;  the  only  controversial  finding  was  the  

cost  analysis  comparison  of  the  RECA,  RRT  and  RAT,  therefore  additional  analysis  was  conducted  to  confirm  and  clarify  this  finding.    Reporting    This  report,  and  an  accompanying  presentation,  seeks  to  present  the  findings  and  analysis  from  the  evaluation  and  make  specific  recommendations  for  improvement  of  GWC  support  to  national  coordination  platforms  through  its  surge  capacity  systems.      The  findings  and  analysis  are  organized  based  on  the  criteria  and  present  answers  to  the  key  questions  agreed  during  the  inception  phase.  Where  applicable,  unexpected  findings  that  go  beyond  the  key  questions  are  also  highlighted  to  represent  critical  information  that  was  discovered  during  the  evaluation.    Quotes  from  key  informant  interviews  are  provided  throughout  the  text  of  the  findings  section  to  illustrate  key  points  and  give  key  stakeholders  an  understanding  of  the  diverse  views  encountered  during  data  collection.  However,  all  findings  are  the  result  of  thorough  triangulation  across  data  sources  and  represent  the  balance  of  data  and  best  judgment.    

Page 10: 2014.06.01 GWC Evaluation Final Report D8€¦ · 8* Table’2–’Diversity’of’Context’in’Evaluation’Case’Studies’ * * * * * * * * * * * * support support)

GWC  Evaluation  of  Support  to  National  Coordination  Platforms    Copyright  ©  Avenir  Analytics,  2014  

 

7  

2. Findings      2.1 Overall  Findings    There  was  extensive  evidence  of  the  overall  strength  of  the  Global  Wash  Cluster,  both  from  the  documents  reviewed  and  in  a  diverse  range  of  key  informants.  Several  informants  with  broad  experience  within  the  cluster  system  noted  that  the  GWC  is  the  overall  strongest,  and  pointed  to  evidence  of  other  clusters  following  the  GWC  lead  (e.g.  the  Nutrition  Cluster  recently  implementing  a  similar  field  support  team  structure.)    

 Additionally,  only  two  of  the  overall  average  responses  within  the  28  indicators  revealed  areas  of  overall  weakness;  on  average,  responses  ranged  from  ‘adequate’  to  ‘good.’      The  two  indicators  with  the  weakest  response  were  not  related  to  FST  activities  directly,  but  included  the  indirect  involvement  of  partners  providing  the  response:  the  speed  of  completing  the  initial  assessment  and  speed  of  providing  response  to  SPHERE  standards.  While  this  is  a  relevant  finding  in  itself,  it  underscores  a  broad  theme  highlighted  by  many  participants;  the  lack  of  an  overall  theory  of  change  for  the  FST  and  practical  method  to  adequately  link  the  activities  of  the  FST  to  impacts  received  by  beneficiaries.  Despite  the  fact  that  “just  brokering  

relationships  and  getting  people  together  has  had  clear  impact,”  a  rigorous  evidence  base  for  the  impact  on  response  is  widely  perceived  as  a  missing  piece  for  not  just  the  GWC  but  the  broader  cluster  system  itself.    

 While  the  average  responses  were  generally  positive,  there  was  considerable  variation  of  response  within  most  of  the  criteria.  (The  two  exceptions  were  relevance  and  coherence  /connectedness,  both  of  which  were  different  in  that  they  were  more  uniformly  positive.)      This  variability  of  response  was  such  that  depending  on  participant  and/or  context,  the  indicators  would  be  reported  across  the  spectrum  from  extremely  poor  to  extremely  strong.  Therefore,  while  the  overall  average  strength  of  the  GWC  is  the  first  primary  finding,  the  second  highlights  a  concerning  caveat;  that  this  strength  is  inconsistent,  and  the  inconsistency  is  related  to  differences  in  country  context,  type  of  emergency,  pre-­‐existing  capacity,  and  differences  in  the  experience  and  training  of  the  personnel  deployed.    Across  most  evaluation  criteria,  the  unique  context  of  the  emergency  was  stressed  as  a  determining  factor  for  how  the  FST  could  and  did  support  national  coordination  platforms.  The  diversity  of  the  four  case  studies  is  illustrated  in  Table  2  below.

         

“There  is  no  theory  of  change  for  the  cluster  system…no  framework,  so  everyone  sets  their  own  benchmarks  for  success  and  dialog  becomes  very  difficult  because  people  can  move  the  goalposts  as  they  

“There  is  much  work  still  to  do  and  problems  to  address,  but  we  should  keep  it  in  perspective  of  the  strength  and  positive  that  has  been  achieved.”  

Page 11: 2014.06.01 GWC Evaluation Final Report D8€¦ · 8* Table’2–’Diversity’of’Context’in’Evaluation’Case’Studies’ * * * * * * * * * * * * support support)

GWC  Evaluation  of  Support  to  National  Coordination  Platforms    Copyright  ©  Avenir  Analytics,  2014  

 

8  

Table  2  –  Diversity  of  Context  in  Evaluation  Case  Studies    

   2.2 Relevance/Appropriateness    There  was  clear  evidence  of  strength  within  the  relevance/appropriateness  criteria,  and  the  findings  demonstrated  less  variance  of  response  as  views  within  the  other  criteria.      Both  the  documents  reviewed  and  feedback  from  key  informants  showed  that  the  goals,  tools  and  approach  of  the  CAST/FST  are  very  well  aligned  with  the  IASC  principles  and  standards.      The  four  cases  in  this  evaluation,  each  of  which  was  designated  as  a  Level  3  (L3)  emergency  by  the  IASC  during  the  period  of  analysis,  demonstrated  broad  GWC  alignment  with  the  IASC  Transformative  Agenda  and  Cluster  

Coordination  Reference  Modules.  L3  emergencies  are  declared  in  situations  where  the  urgency,  scale  or  complexity  overwhelms  regional  and  national  capacity  to  respond  and  extraordinary  resources  are  temporarily  required.  The  Transformative  Agenda  calls  on  clusters  to  deploy  on  a  ‘no-­‐regrets’  basis  once  L3  emergencies  are  declared  to  ensure  national  coordination  platforms  are  able  to  function.  The  scale  of  this  response  is  meant  to  be  determined  based  on  the  pre-­‐existing  resources  and  capacities  of  national  coordination  structures.  Figure  1  below  shows  that  during  2013  and  early  2014,  GWC  surge  capacity  deployments  were  primarily  focused  (88%)  on  the  four  countries  that  became  L3  emergencies.

           

South&Sudan Central&African&Republic Syria Philippines

Emergency&Type Complex(humanitarian(emergency

Complex(humanitarian(emergency

Complex(humanitarian(emergency Natural(disaster

Initial&WASH&responseWater(and(sanitation(to(large(numbers(of(IDP's,(refugees(and(the(general(population

Water(and(sanitation(provided(to(IDP(camps

Water(and(sanitation(provided(to(refugees(and(IDP(camps

Restored(electricity(and(functionality(of(municipal(water(systems,(emergency(water(to(rural(areas(

Geographic&ScopeWASH(response(across(entire(country;(remote(with(difficult(accessibility

Primary(focus(on(IDP(camps(followed(by(efforts(to(reach(rural(areas

Very(constrained(response(opportunity(within(Syria;(Response(included(Jordan,(Southern(Turkey(and(Lebanon(

Large(area(with(urban,(periJurban(and(rural(contexts,(all(with(total(destruction(of(infrastructure

Pre@Existing&Coordination&Networks

Longstanding(cluster(activities(in(country(but(high(turnover(greatly(diminished(capacity(

Very(weak(preJexisting(networks(were(completely(destroyed(by(turnover,(violence(and(looting

Varied(levels(of(capacity(in(surrounding(country(contexts;(none(in(Syria(itself

Uniquely(strong(national(structure;(primary(need(was(expanding(local/subJnational(support

Status&of&National&Partners Very(few(partners(/(high(turnover(because(of(violence

Very(few(partners(/(high(turnover(because(of(violence

Adequate(number(of(partners(in(surrounding(countries

Large(number(of(very(capable(partners;(perhaps(too(many(

Primary&Challenges&Cited&by&Key&Informants

Adequately(scaling(response(within(fastJchanging(context(related(to(patterns(of(violence(and(weather

Extreme(lack(of(resources((i.e.(finding(office(supplies)(and(ongoing(violence(make(any(progress(extremely(difficult

Identifying(practical(opportunities(to(help(given(extreme(violence(and(political(constraints

Managing(the(chaos(of(a(huge(international(response(with(all(infrastructure(destroyed

Number&of&Globally&Deployed&Individuals&(includes&CAST,&FST&and&other&augmentation&support)

6(Individuals 9(Individuals 16(Individuals 13(Individuals

Page 12: 2014.06.01 GWC Evaluation Final Report D8€¦ · 8* Table’2–’Diversity’of’Context’in’Evaluation’Case’Studies’ * * * * * * * * * * * * support support)

GWC  Evaluation  of  Support  to  National  Coordination  Platforms    Copyright  ©  Avenir  Analytics,  2014  

 

9  

Figure  1  –  GWC  Surge  Staff  Deployments    

   The  three  components  of  the  FST  provide  a  combined  capacity  that  allows  the  GWC  to  provide  ongoing  support  to  national  coordination  platforms  beyond  deployments  to  large  scale  emergencies  through  remote,  regional  and  short-­‐term  country  level  assistance.  There  was  broad  evidence  of  the  flexibility  of  the  combined  FST  mechanisms  to  facilitate  context-­‐appropriate  support  for  national  coordination  mechanisms.      For  example,  support  is  targeted  to  the  specific  national  needs  across  many  non-­‐deployment  dimensions,  including  capacity  building  and  training  for  assessment,  coordination,  and  information  management;  and  assistance  with  setting  up  and  facilitating  extraordinary  partnership,  monitoring  and  learning  exercises.  RRT,  RAT  and  RECA  staff  have  served  both  gap-­‐filling  and  capacity  building  roles.  Additionally,  

the  structure  utilizing  NGO  partners  provides  greater  flexibility  and  ability  to  customize  the  support  than  is  otherwise  possible  within  the  more  rigid  UN  structure.      To  prioritize  GWC  support  to  national  coordination  platforms,  the  FST  began  using  a  capacity  assessment  tool  in  2013  to  map  the  strengths  and  gaps  for  coordination  per  country  for  all  regions,  based  on  an  early  version  of  the  IASC  cluster  coordination  performance  assessment  tool.    

While  this  initiative  is  oriented  towards  enabling  the  GWC  to  best  prioritize  its  assistance  before  

0"

2"

4"

6"

8"

10"

12"

14"

16"

18"

Dec+12" Jan+13" Feb+13" Mar+13" Apr+13" May+13" Jun+13" Jul+13" Aug+13" Sep+13" Oct+13" Nov+13" Dec+13" Jan+14" Feb+14" Mar+14" Apr+14"

GWC$Surge$Staff$Deployed$by$Week$and$Emergency$

South"Sudan" CAR" Philippines" Syria" Other"

Philip-pines

Typhoon Pablo

Philip-pines

Typhoon Haiyan,

L3 Declared

South Sudan

Violence

Syria L3 Declared

CAR Coup d’Etat

CAR L3 Declared

South Sudan

L3 Declared

Includes RRT, RAT, RECA, CAST and other globally mobilized surge personnel

“There  are  questionnaires  developed  that  are  very  linked  to  IASC  global  tools...but  RECA  struggles  with  how  to  present  them  to  national  partners  because  they  don’t  want  to  be  perceived  as  evaluators  but  rather  capacity  builders…”    

Page 13: 2014.06.01 GWC Evaluation Final Report D8€¦ · 8* Table’2–’Diversity’of’Context’in’Evaluation’Case’Studies’ * * * * * * * * * * * * support support)

GWC  Evaluation  of  Support  to  National  Coordination  Platforms    Copyright  ©  Avenir  Analytics,  2014  

 

10  

and  following  emergencies,  it  has  raised  a  tension  among  FST  members  related  to  being  seen  as  an  accountability  mechanism  rather  than  a  resource.  The  long  term  relevance  of  these  efforts  to  national  coordination  structures  needs  to  be  well  communicated.    The  responses  to  the  two  sub-­‐questions  about  relevance  and  appropriateness  highlighted  a  dynamic  tension  within  the  overall  cluster  system  itself  which  contributes  to  strain  within  the  GWC:  the  tension  between  maintaining  global  standards  (i.e.  IASC  cluster  coordination  reference  modules  and/or  SPHERE  standards)  and  the  need  to  work  in  support  of  country/local  organizations  and  processes.    

 While  these  are  not  mutually  exclusive  in  concept,  this  evaluation  found  that  the  uniquely  strong  government,  NGO  and  UN  community  in  the  Philippines  was  the  only  context  that  approximated  the  needed  capacity  to  implement  effective  coordinated  disaster  response.    Yet  even  in  the  Philippines  there  were  significant  gaps  and  needs,  especially  at  the  sub-­‐national  level.      In  complex  emergency  contexts  involving  violence  (e.g.  CAR,  Syria  and  South  Sudan)  most  or  all  of  the  pre-­‐existing  national  structure  and  NGO  capacity  was  quickly  overwhelmed  when  the  complexity  and  scale  of  needs  rapidly  escalated.      

In  CAR  the  pre-­‐existing  coordination  capacity  was  judged  very  weak  by  key  informants.  Prior  to  the  conflict  in  Syria,  the  countries  affected  by  this  regional  emergency  had  little  or  no  coordination  structures  in  place.  During  the  conflict,  access  issues  made  scaling  up  coordination  in  Syria  highly  difficult.  While  the  humanitarian  community  had  significantly  invested  in  the  cluster  system  in  South  Sudan  prior  to  the  outbreak  of  violence  in  December  2013,  a  combination  of  annual  turnover  in  sub-­‐national  coordinators  (covered  by  cluster  member  agencies)  and  unexpected  personnel  gaps  at  the  national  level  coincided  with  the  escalation  of  emergency  needs.  This  led  to  a  significant  deterioration  of  WASH  cluster  capacity  that  was  exacerbated  by  evacuations  and  access  limitations  due  to  the  violence.    

 Because  of  these  realities,  in  many  situations  the  FST  ‘supported’  national  systems  by  significantly  augmenting  them  or  building  or  re-­‐building  them  from  scratch.  While  such  contributions  are  clearly  relevant  and  needed,  they  are  substantially  different  than  the  more  supplemental  ‘support’  anticipated  in  theory.      2.3 Effectiveness    The  CAST  and  FST  contribution  to  improved  coordination  and  preparedness  of  national  structures  was  generally  considered  strong.      One  indicator  within  this  criteria  related  to  the  FST  support  being  available  to  all  WASH  sector  partners  equally.  This  was  the  only  indicator  receiving  unanimous  and  strong  affirmation,  

“They  want  to  enable  local  structures,  but  when  the  circus  comes  to  town  you  need  lion  trainers…it's  such  an  arrogant  system…not  enough  examples  like  the  Philippines  where  there  is  a  credible  local  platform  that  can  be  built  on.”  

“Unfortunately,  at  the  onset  of  the  recent  emergency,  the  structures  we  had  established  before  just  collapsed  like  a  house  of  cards….”    

Page 14: 2014.06.01 GWC Evaluation Final Report D8€¦ · 8* Table’2–’Diversity’of’Context’in’Evaluation’Case’Studies’ * * * * * * * * * * * * support support)

GWC  Evaluation  of  Support  to  National  Coordination  Platforms    Copyright  ©  Avenir  Analytics,  2014  

 

11  

such  that  there  is  very  strong  evidence  that  the  FST  is  appropriately  serving  the  sector.      RECA  contributions  to  preparedness  through  capacity  building  were  generally  seen  as  positive  for  the  8  focus  countries  that  have  received  intense  attention.  In  particular,  the  relationships  formed  and  knowledge  of  local  structures  and  capacities  gained  through  on-­‐going  capacity  building  assistance  were  credited,  with  increasing  country  level  acceptance  of  GWC  resources  during  emergencies  (e.g.  the  Philippines).  Additionally,  RECA  efforts  provided  key  mentoring  and  skill  development  opportunities  for  national  coordination  platforms  in  “peace  time.”  However,  many  key  informants  noted  that  the  discrete  mandate  and  project  structure  of  the  RECA  limited  its  potential  contribution  to  overall  GWC  effectiveness  and  sometimes  created  tension  with  other  regional  and  FST  actors.  

 The  strengths  within  this  criteria  contrast  with  clear  perceived  weakness  in  the  effects  of  the  FST  on  the  speed  of  total  sector  response,  both  in  completing  initial  assessments  and  providing  initial  WASH  relief  up  to  SPHERE  standards.  While  FST  members  generally  deployed  quickly  (see  timeline  analysis  in  Section  2.4),  the  ability  to  influence  achievement  of  operational  objectives  was  highly  dependent  on  government  capacity  and  the  capacity  and  ability  of  member  NGOs  to  deliver  in  a  consistent  way,  and  was  therefore  highly  variable  across  contexts.      

Many  participants  further  underscored  that  point  by  noting  that  there  were  no  clear  indicators  of  success  for  FST  activities  vs.  national  coordination  platforms  and  the  ability  to  attribute  FST  contributions  to  beneficiary  impacts  was  difficult  or  impossible.    

 There  was  also  clear  variability  of  effectiveness  between  different  facets  of  GWC  surge  capacity.  The  RRTs  were  reported  as  effective  in  their  role  to  augment,  support  and  develop  coordination  mechanisms  during  emergencies.  However,  the  RAT  and  RRT-­‐IM  efforts  were  reported  as  less  effective.  It  should  be  noted  that  there  was  substantial  variability  within  these  responses  based  on  the  experience  and  ‘fit’  of  individual  FST  members  to  the  needs  of  the  context.    

 Key  informants  cited  attributes  such  as  adaptability  to  context,  ability  to  handle  ambiguity  and  appreciation  for  pre-­‐existing  coordination  systems  and  initiatives  when  critiquing  the  effectiveness  of  particular  deployed  FST  personnel.  Documentation  review  showed  that  these  and  other  coordination  competencies  are  being  addressed  within  a  broader  cluster  coordination  competency  development  initiative  of  UNICEF  as  Cluster  Lead  Agency  (CLA)  for  WASH  and  other  sectors,  which  should  help  further  increase  the  effectiveness  of  FST  members.  

“What  is  quite  frustrating  is  that  there  are  few  indicators  of  impact…a  very  difficult  situation”    

“…The  weakness  in  last  6  years  isn't  the  coordination  side  but  the  information  management  side…I  have  strongly  pushed  to  have  dual  deployments  of  IM/RRT.”  

“In  countries  with  proper  structure  (i.e.  Philippines)  the  FST  has  a  strong  added  value;  in  countries  without  the  structure,  the  contribution  is  questionable.”    

Page 15: 2014.06.01 GWC Evaluation Final Report D8€¦ · 8* Table’2–’Diversity’of’Context’in’Evaluation’Case’Studies’ * * * * * * * * * * * * support support)

GWC  Evaluation  of  Support  to  National  Coordination  Platforms    Copyright  ©  Avenir  Analytics,  2014  

 

12  

The  weaknesses  noted  regarding  the  RAT  and  IM  functions  were  presented  by  key  informants  within  the  context  of  substantial  demands  (often  upstream  Headquarters  or  inter-­‐agency)  related  to  information  management,  compared  to  actionable  and  operationally  oriented  information  and  assessment  systems  preferred  by  national  coordination  platform  members.      Similarly  to  the  RRT,  some  key  informants  found  RAT  members  to  be  highly  effective,  based  on  their  technical  competence  and  adaptability  to  context  driven  needs  and  capacities.    

 Limitations  to  the  effectiveness  of  the  RAT  is  that  they  were  found  to  be  largely  driven  by  the  project  design  and  project  management.  RAT,  like  RECA,  has  been  managed  as  a  discreet  project  by  GWC  member  agencies.  Key  informants  noted  highly  variable  expectations  of  the  purpose  and  expectations  of  the  RAT  members,  especially  in  comparison  to  broader  and  increasingly  established  cross-­‐sectoral  assessment  tools  and  initiatives  such  as  REACH,  ACAPS  and  the  assessment  mechanisms  tied  to  the  Transformative  Agenda  (e.g.  MIRA,  IARRM).  Until  2013,  the  RAT  project  did  not  have  a  dedicated  project  manager,  with  the  three  members  instead  reporting  to  their  own  agencies.  Beginning  in  2013  the  RECA  project  manager  assumed  this  function,  though  the  expectations  of  this  member-­‐led  role  are  evolving  with  ongoing  integration  efforts  across  the  FST.    

During  the  course  of  the  evaluation  the  role  of  the  CAST  in  managing  and  mobilizing  surge  capacity  became  clear  in  expected  and  unexpected  ways.  The  Global  WASH  Cluster  Coordinator  and  Deputy  Cluster  Coordinator  play  an  active  and  effective  role  in  facilitating  the  recruitment,  selection  and  placement  of  RRTs  in  coordination  with  NGO  members.  These  GWC  leaders  also  deploy  in  L3  emergencies  to  help  initiate  operations,  coach  national  coordinators  and  facilitate  acceptance  of  GWC  support  by  governments  and  the  Cluster  Lead  Agency.  Together  the  global  coordinator  and  deputy  spent  approximately  14  weeks  in  the  four  L3  countries  during  the  period  covered  by  this  evaluation  in  addition  to  2  joint  monitoring  missions  with  donors  and  members.    The  analysis  of  operational  timelines  and  documentation  showed  that  the  global  coordinator  and  deputy  play  an  effective  role  in  mobilizing  additional  support  beyond  the  resources  of  the  FST.  From  January  2013  to  April  2014  approximately  30%  of  GWC  globally  directed  surge  capacity  support  for  national  coordination  was  mobilized  from  non-­‐FST  sources  with  direct  involvement  of  the  global  coordinator  and  deputy.  The  sources  of  this  additional  surge  capacity  help  to  augment  the  FST  and  include  stand-­‐by  partner  staff,  national  cluster  coordinators  that  are  temporarily  reassigned  to  an  acute  emergency,  former  FST  members,  GWC  member  staff  and  other  individuals  with  on-­‐going  relationships  with  the  GWC.    2.4 Efficiency  

“The  RAT  member  provided  very  good  work  and  had  substantial  knowledge  of  his  work.  After  his  deployment  finished  he  provided  remote  support  that  was  very  valuable…”  

“For  a  national  cluster,  calling  on  the  FST  is  heads  and  shoulders  above  the  other  options  for  support.”  

Page 16: 2014.06.01 GWC Evaluation Final Report D8€¦ · 8* Table’2–’Diversity’of’Context’in’Evaluation’Case’Studies’ * * * * * * * * * * * * support support)

GWC  Evaluation  of  Support  to  National  Coordination  Platforms    Copyright  ©  Avenir  Analytics,  2014  

 

13  

The  ‘value  for  money’  of  GWC  FST  support  to  national  coordination  was  considered  based  on  the  combination  of  three  elements  –  cost,  quality  and  speed.  Overall,  the  evaluation  found  that  partners  and  national  coordination  staff  believe  the  FST  provides  strong  value  for  money.    First,  the  overall  value  provided  to  national  partners  and  national  coordinators  was  viewed  positively  at  country  level.  This  is  partly  because  partners  and  national  coordination  platforms  do  not  pay  for  surge  support.  Participants  therefore  typically  expressed  either  gratitude  for  the  ‘free’  support  they  received  or  were  unaware  of  the  costs  entirely  (i.e.  working  in  roles  removed  from  administrative  hubs.).  

However,  there  are  real  costs  associated  with  this  support  and  global  GWC  partners  have  helped  to  mobilize  funding  for  all  three  elements  of  the  FST,  through  stand-­‐by  partnerships  for  RRTs  and  other  surge  augmentation  deployments  and  resource  mobilization  for  the  RAT  and  RECA  projects.  Table  3  shows  the  overall  recorded  time  for  all  FST  members  and  the  estimated  actual  costs  of  this  support.  Costs  were  estimated  based  on  the  budget  analysis  presented  in  Table  4  and  the  time  allocations  captured  in  the  FST  monitoring  dashboard.      

 Table  3  –  Recorded  allocation  of  FST  time  and  estimated  costs    

   The  second  aspect  was  considerably  more  difficult,  and  included  an  effort  to  provide  a  ‘balanced  scorecard’  cost  analysis.  As  noted  above,  the  speed  and  quality  of  services  (the  other  factors  within  this  analysis)  were  either  unclear  or  unfavorable,  such  that  a  clear  measure  of  efficiency  was  extremely  difficult  to  determine  within  the  scope  of  this  evaluation.      

Within  those  limitations,  the  FST  was  broadly  perceived  as  a  good  value  in  terms  of  the  cost  structure,  which  because  of  its  reliance  on  NGO  members  is  regarded  as  less  expensive  than  similar  deployments  within  the  UN  structure.  While  the  cost  structure  of  RRTs  was  found  to  be  60%  less  than  a  comparable  UN  staff  surge  model  there  is  significant  variance  in  the  budgeted  cost  models  for  each  component  of  the  FST  as  shown  in  Table  4  below.  

       

Days%Deployed%

Remote%Support

Regional Global%Cluster%

Agency%%Support

Other%(Admin,(

medical,(etc)

Time%Off Total%

Days%Recorded 1,587 254 142 610 220 327 172 3,311

%%of%Total%Time%Recorded

48% 8% 4% 18% 7% 10% 5%

Estimated%Total%Cost

$1,099,289 $271,479 $172,201 $499,406 $161,077 $290,765 $155,351 $2,649,569

Full%FST%L%January%2013%to%midLFebruary%2014

Page 17: 2014.06.01 GWC Evaluation Final Report D8€¦ · 8* Table’2–’Diversity’of’Context’in’Evaluation’Case’Studies’ * * * * * * * * * * * * support support)

GWC  Evaluation  of  Support  to  National  Coordination  Platforms    Copyright  ©  Avenir  Analytics,  2014  

 

14  

Table  4  –  Budgeted  annual  costs  per  FST  member/type  and  UN  staff  equivalent  costs    

   

While  FST  costs  are  generally  viewed  as  well  managed  by  knowledgeable  key  informants,  the  fragmented  evolution  of  the  RRT  (under  management  of  the  CAST)  and  the  RAT  and  RECA  (under  project  funding  led  by  consortia  of  members)  has  led  to  significant  cost  differences  for  each  component.  Many  key  informants  believe  the  differences  are  related  to  the  amount  of  time  deployed  or  regionally  based  vs.  home  based,  but  analysis  of  the  budgets  shows  

that  the  drivers  of  the  cost  differences  are  actually  based  on  salary,  management  and  coordination  costs.  The  variable  cost  effectiveness  of  each  component  of  the  FST  was  further  analyzed  based  on  recorded  utilization  rates.  Using  the  GWC  FST  monitoring  dashboard  data  collected  by  the  CAST,  the  recorded  utilization  rates  and  cost  per  day  recorded  were  calculated  as  shown  in  Table  5.

   Table  5  –  FST  component  recorded  utilization  and  cost  efficiency  

   

RRT  members  provide  the  greatest  overall  cost  efficiency  based  on  the  lower  budgeted  cost  basis  and  the  higher  amount  of  time  they  are  deployed  in  the  field  (68%  for  RRTs  vs.  29%  for  RAT  and  20%  for  RECA)  where  they  often  work  7  days  a  week.  Key  informants  noted  that  this  cost  

efficiency  comes  with  a  significant  downside  risk  of  burnout  and  potential  for  turnover  among  RRT  members,  which  could  lessen  overall  effectiveness  if  the  most  experience  staff  leave  without  suitable  replacements.    

Budget'Categories RRT'Members*

RAT'Members**

RECA'Members***

UN'Staff****'comparison'(P4/L4'level)

Annual'Salary'(including'insurance,'benefits) $119,727 $98,640 $152,673 $236,340Communications'Annual $2,159 $8,333 $4,467 $2,159Deployments'Annual $44,407 $45,387 $47,526 $50,232Direct'Mgmt'and'Coord'Costs $4,737 $4,000 $13,397Indirect'Costs $11,972 $15,264 $20,211Total&Annual&Costs $183,002 $156,360 $233,328 $308,942*'Based'on'RRT'budgets'for'PCA'funded'posts.

**'Based'on'RAT'budget.'Does'not'include'$113,487'annual'cost'in'RAT'budget'for'local/national'acessor'costs.

***'Based'on'RECA'project'budget.'Does'not'include'RECA'project'manager,'training','visibility'costs,'evaluation'and'joint'monitoring'mission'costs.

****'Salary'based'on'fully'loaded'standard'position'cost'rates.'Travel'based'on'average'DSA'rates'and'UN'travel'rules.'Indirect'estimated'at'7%.

Recorded'Utilization*'(January'2013'to'February'2014)

RRT'Members

RAT'Members

RECA'Members

Days'Recorded'(including'deployed'and'nonFdeployed'work'and'time'off)

1,814 635 862

%'Recorded'Utilization'(days'recorded'vs.'nonFweekend'days'on'contract)**

116% 74% 73%

Cost'per'Day'Recorded'(based'on'total'annual'cost,'adjusted'for'days'on'contract)

$603 $799 $1,217

**'Vacant'positions'not'included'in'analysis.*Recorded'days'are'based'on'GWC'Field'Support'Team'monitoring'dashboard'data.

Page 18: 2014.06.01 GWC Evaluation Final Report D8€¦ · 8* Table’2–’Diversity’of’Context’in’Evaluation’Case’Studies’ * * * * * * * * * * * * support support)

GWC  Evaluation  of  Support  to  National  Coordination  Platforms    Copyright  ©  Avenir  Analytics,  2014  

 

15  

 A  few  key  informants  cited  weaknesses  in  this  analysis  when  presented  with  preliminary  findings  based  on  concerns  with  the  quality  of  the  data  reported  by  RAT  and  RECA  members  and  variability  in  understanding  of  the  categories  for  time  allocation.  This  suggests  a  potential  for  improving  the  accuracy  of  future  cost  analysis  by  strengthening  adherence  to  time  based  reporting  systems.      However,  as  the  budgeted  cost  basis  for  each  component  is  also  quite  variable,  improved  

accuracy  of  time  reporting  would  not  be  likely  to  significantly  change  the  outcome  of  this  analysis  unless  components  of  the  FST  are  managed  differently  in  the  future.    While  effects  of  the  FST  on  the  speed  of  response  to  beneficiary  needs  is  difficult  to  determine  as  noted  above,  the  speed  of  deployments  was  analyzed  based  on  timelines  reconstructed  from  emails,  the  FST  dashboard  deployment  data  and  other  documentary  evidence.  In  all  four  cases  from  2013  and  2014,  GWC  FST  members  or  other  globally  mobilized  staff  (facilitated  directly  by  CAST)  were  present  before  the  declaration  of  an  L3  emergency  by  the  IASC  as  shown  in  Figure  2  below.

   Figures  2  -­‐  Timeline  analysis  for  Syria,  Philippines,  South  Sudan  and  CAR    

   

   

Dec$13 Jan$14 Feb$14Jan$13 Feb$13 Mar$13 Apr$13 May$13 Mar$14 Apr$14Jun$13 Jul$13 Aug$13 Sep$13 Oct$13 Nov$13Trigger UNICEFAL3ADeclaration

IASCALA3ADeclarationCASTRRT

RRTA$AIM

RATRECAOtherAAugmentation

Syria

Deploy$ments

Dec$13 Jan$14 Feb$14Nov$13Trigger Typhoon7Haiyan7Landfall

UNICEF7L37DeclarationIASC7L37DeclarationIASC7L37Deactivation

Deployments CAST

Philipp

ines RRT

RRT7$7IM

Philipp

ines

RATRECAOther7Augmentation

Philipp

ines

Monitoring JMM

Philipp

ines

“When  you  see  number  of  days  provided  and  extent  of  support  RRT  provides  as  compared  to  price  of  project,  it's  cheap…the  project  is  efficient.”    

Page 19: 2014.06.01 GWC Evaluation Final Report D8€¦ · 8* Table’2–’Diversity’of’Context’in’Evaluation’Case’Studies’ * * * * * * * * * * * * support support)

GWC  Evaluation  of  Support  to  National  Coordination  Platforms    Copyright  ©  Avenir  Analytics,  2014  

 

16  

   

   To  further  enhance  the  efficiency  and  effectiveness  of  the  FST,  the  SAG  and  CAST  initiated  an  effort  in  2013  to  better  integrate  the  three  components  of  the  FST.  Donors  and  NGO  consortia  that  have  supported  the  RAT  and  RECA  projects  agreed  to  experiment  with  more  flexibly  allocating  their  project  staff  to  serve  the  most  critical  current  surge  capacity  needs  and  broadening  the  capacity  assessment  and  building  initiatives  somewhat  beyond  the  8  RECA  focus  countries.    Based  on  the  initial  positive  learning  from  these  initiatives,  the  CAST  provided  the  SAG  and  other  key  stakeholders  with  a  discussion  paper  in  early  2014  that  envisions  a  completely  integrated  FST  under  the  direct  management  of  the  CAST.  Under  this  proposal,  FST  members  would  be  more  flexibly  assigned  to  cover  7  functional  areas  including:  assessment,  information  management,  quality  assurance,  coordination,  continuity,  capacity  building  and  preparedness.  The  discussion  paper  envisions  loosely  organizing  

FST  staff  into  two  functional  sub-­‐teams:  i)  assessment,  information  management  and  quality  assurance,  and  ii)  coordination,  continuity,  capacity  building  and  preparedness.      To  facilitate  better  contact  with  and  support  to  national  coordination  platforms  the  discussion  paper  suggests  organizing  most  staff  into  teams  based  on  a  decentralized  hub  system,  while  maintaining  flexibility  by  stressing  the  expectation  that  hub-­‐based  staff  will  be  routinely  redeployed  to  support  needs  outside  of  their  region  based  on  risk  and  demand.              

Apr$14Dec$13 Jan$14 Feb$14 Mar$14Trigger Outbreak8of8Violence

IASC8L38DeclarationDeployments CAST

RRT

RRT8$8IMRATRECAOther8Augmentation

South8Sudan

Dec$13 Jan$14 Feb$14 Mar$14 Apr$14Trigger UNICEF9L3

IASC9L3Deployments CAST

CAR

RRT

CAR

RRT9$9IM

RATRECAOther9Augmentation

CAR

Monitoring JMM94

CAR

Page 20: 2014.06.01 GWC Evaluation Final Report D8€¦ · 8* Table’2–’Diversity’of’Context’in’Evaluation’Case’Studies’ * * * * * * * * * * * * support support)

GWC  Evaluation  of  Support  to  National  Coordination  Platforms    Copyright  ©  Avenir  Analytics,  2014  

 

17  

2.5 Coherence/Connectedness    Broadly  speaking,  the  FST  surge  support  demonstrated  clear  strength  in  both  inter-­‐cluster  and  intra-­‐cluster  coordination  which  was  the  focal  question  related  to  this  criteria.  In  particular,  the  diversity  and  depth  of  the  previous  experience  brought  by  RRT  members  was  noted  as  contributing  to  this  strength  and  speeding  their  connection  with  other  clusters.    

 The  RAT  and  IM  members  were  occasionally  noted  as  weaker  in  coordination,  some  of  which  was  related  to  their  TOR  and  need  for  more  structures.  The  inclusiveness  of  NGO  participation  and  leadership  was  also  noted  as  a  clear  strength  supporting  connectedness,  which  is  certainly  appropriate  given  the  NGO  leadership  within  the  GWC  SAG.  Similarly,  there  was  broad  perception  that  the  FST  activities  effectively  understood  and  supported  the  unique  role  of  cluster  coordinators,  including  addressing  role  ambiguity  and  double-­‐hatting  issues.      The  primary  concerns  noted  within  this  criteria  related  to  communication  and  collaboration  between  the  WASH  cluster  and  other  clusters,  though  part  of  this  weakness  was  mitigated  by  stating  that  in  some  situations  other  clusters  were  not  functioning  as  quickly  or  as  strongly  to  be  able  to  adequately  engage.    

General  weaknesses  in  inter-­‐cluster  coordination  systems  were  cited  as  one  reason  for  these  challenges,  with  many  stakeholders  noting  that  the  processes  for  such  cross-­‐sectoral  coordination  seem  primarily  oriented  to  up-­‐stream  information  demands.      Similarly,  several  participants  expressed  the  inherent  difficulties  of  engaging  with  the  cluster  system  itself,  the  mandate  and  “turf”  struggles  within  the  UN  System  and  the  dynamic  tension  between  maintaining  optimal  coordination  and  accomplishing  tangible  relief  objectives  -­‐  all  of  which  underscored  the  complex  dynamics  related  to  connectedness  that  must  be  balanced  in  relief  operations.    FST  members  mostly  demonstrated  a  strong  appreciation  and  respect  for  their  roles  in  supporting  pre-­‐existing  coordination  structures  where  they  exist.  In  emergency  and  non-­‐emergency  support,  FST  members  understood  the  importance  of  working  with  national  governments  where  appropriate  and  possible  and  strengthening  rather  national  coordination  rather  than  building  parallel  systems.  Coordination  with  national  platforms  and  the  credibility  of  the  FST  appears  to  be  enhanced  by  the  origin  of  FST  members,  who  come  from  a  diverse  range  of  WASH  agencies  outside  the  UN  system.    Linkages  between  the  GWC  and  UNICEF  as  CLA  are  strong  at  the  global  level,  with  clear  participation  of  UNICEF  in  GWC  initiatives  and  increasing  collaboration  on  efforts  to  strengthen  sectoral  coordination.    At  the  country  level  some  weaknesses  were  identified  in  the  connectedness  of  the  GWC  to  UNICEF  country  offices  in  places  where  the  FST  has  not  recently  deployed.  Lack  of  understanding  about  the  value  added  of  the  FST,  or  country  office  preference  to  

“One  RRT  was  very  keen  on  inter-­‐cluster  coordination,  and  went  to  each  cluster  meeting  to  get  oriented  in  first  days.”  

“They  were  trying,  but  there  is  an  underlying  coordination  problem  between  [clusters];  OCHA  coordination  was  very  poor,  especially  for  level  3  emergencies.  Things  were  thrown  on  the  table  and  discussed  a  little  but  no  real  decisions.”  

Page 21: 2014.06.01 GWC Evaluation Final Report D8€¦ · 8* Table’2–’Diversity’of’Context’in’Evaluation’Case’Studies’ * * * * * * * * * * * * support support)

GWC  Evaluation  of  Support  to  National  Coordination  Platforms    Copyright  ©  Avenir  Analytics,  2014  

 

18  

handle  CLA  responsibilities  without  outside  support  can  sometimes  complicate  and  delay  deployment  of  the  FST  where  its  assistance  is  needed.    2.6 Coverage  

The  expanded  coverage  of  coordination  needs  was  widely  considered  strong,  including  filling  staffing  gaps,  helping  establish  sub-­‐national  systems,  and  starting  new  coordination  efforts  from  scratch.  To  support  these  points,  many  participants  noted  the  strength  of  experience  and  overall  dedication  of  the  FST  staff,  particularly  the  RRT’s  responsible  for  coordination  related  efforts.      Expanding  the  coverage  of  technical  advisory  needs  was  highly  variable  and  largely  related  to  the  experience  levels  of  individual  FST  members  and  key  informant  perspectives.  Views  were  split  between  those  that  value  coordination  competencies  most  vs.  those  that  believe  coordinators  need  to  have  paramount  technical  expertise  to  “lead”  coordination.    

 A  key  theme  from  the  analysis  underscored  a  mitigating  factor  for  coverage  with  complex  strategic  implications;  the  tension  between  a  reactive  or  demand-­‐responsive  posture  of  the  FST  and  a  more  proactive  or  promotional  stance.  This  theme  was  especially  apparent  in  analyzing  the  divergence  of  response  between  participants.    

On  the  one  hand,  some  explicitly  said  that  the  FST  needs  to  do  more  to  ‘promote  their  services’  or  ‘build  awareness’  within  the  broad  WASH  community  and  with  key  national  coordination  structures  and  the  CLA  more  specifically.  At  the  same  time,  other  participants  referenced  the  need  to  be  responsive  to  expressed  local  needs.  While  both  goals  can  be  pursued  simultaneously,  the  strategic  tension  between  a  proactive  and  reactive  stance  is  a  mitigating  factor  related  to  coverage.  In  so  much  as  the  FST  is  reactive,  the  identification  and  extent  of  the  gaps  covered  is  limited  by  the  requests  of  national  organizations,  and  in  so  much  as  the  FST  is  proactive,  they  risk  disempowering  the  very  national  structures  they  ultimately  seek  to  serve.    2.7 Sustainability    There  was  broad  evidence  in  the  documents  reviewed  and  from  the  key  informants  that  the  FST  surge  deployments  included  strategies  and  plans  for  transition  and  sustainability  that  were  either  pre-­‐existing  or  developed  during  the  deployment.  In  particular  it  was  noted  that  the  RRT  mandate  is  to  start  planning  the  transition  from  ‘day  one’  and  many  deployments  are  explicitly  gap  filling  where  transition  is  at  least  implicit,  if  not  formally  structured.    

However,  despite  the  clarity  of  intent  and  planning,  there  was  high  variability  in  the  actual  implementation  of  the  transition  plans  due  to  difficulties  in  recruitment,  emergency  context,  and  funding.  Similarly,  the  lack  of  cluster  system  clarity  related  to  deactivation  and  milestones  for  signaling  the  transition  from  emergency  to  

[The  FST  was  a]  “Big  help  with  filling  staffing  gaps,  helping  get  new  state  focal  points  up  and  running”  

“Yes,  with  focus  countries  and  intentional  relationship  development  at  country  levels  there  are  clear  strategies  to  provide  continuity  of  impact…but  there  are  many  variables  and  struggles.”  

“One  (FST  member)  was  excellent,  one  was  good,  and  the  third  didn't  have  any  technical  ideas...so  uneven,  and  this  should  be  considered  in  forming  TOR.”    

Page 22: 2014.06.01 GWC Evaluation Final Report D8€¦ · 8* Table’2–’Diversity’of’Context’in’Evaluation’Case’Studies’ * * * * * * * * * * * * support support)

GWC  Evaluation  of  Support  to  National  Coordination  Platforms    Copyright  ©  Avenir  Analytics,  2014  

 

19  

development  was  noted  as  a  reason  for  this  weakness,  as  previous  evaluations  of  the  cluster  system  have  also  found.  While  the  Transformative  Agenda  has  initiated  a  process  to  reassess  cluster  activation  on  an  annual  basis,  key  informants  did  not  mention  significant  changes  in  clarity  regarding  deactivation.  However,  in  the  Philippines,  the  IASC  formally  deactivated  the  L3  designation  after  3  months.      The  lasting  impact  of  FST  deployments  and  overall  quality  of  transitions  was  highly  variable  based  on  the  partners  involved,  emergency  context  and  funding.  Similar  to  the  effectiveness  findings,  sustaining    FST  efforts  is  depends  on  partners  that  remain  in  the  local  context.    

There  are  no  formal  commitments  beyond  specific  deployments  and  projects  related  to  FST’s  ongoing  activities,  and  the  path  toward  sustained  impact  is  unclear.    Sustainability  of  the  FST  structure  was  identified  as  a  potential  concern  across  many  global  key  informants,  despite  the  clear  value  added  by  the  FST,  due  to  concerns  about  funding  and  burnout.  Donor  support  for  the  RAT  and  RECA  projects  has  been  strong  for  the  past  few  years  and  key  donors  have  given  positive,  if  vague,  indications  that  they  continue  to  support  the  direction  of  the  GWC.  However,  funding  delays  have  complicated  implementation  of  the  RECA  project  in  2013  and  further  delays  or  reductions  could  significantly  affect  FST  capacity,  and  thus  the  predictability  and  quality  of  GWC  surge  capacity.    Although  the  GWC  SAG,  CAST  and  members  all  acknowledge  these  risks,  alternative  financing  

models  do  not  appear  to  have  been  strongly  considered.  In  meetings  observed  during  the  evaluation  most  focus  on  funding  appears  oriented  towards  showing  value  for  money  to  current  donors  and  expanding  the  donor  base  by  reaching  out  to  other  potential  donor  agencies.      These  risks  may  increase  with  the  integration  of  FST  components  under  the  CAST  if  GWC  members  do  not  continue  to  actively  participate  in  mobilizing  resources  and  staff  and  UNICEF  as  the  CLA  is  expected  to  ensure  all  funding  and  recruitment.  As  UNICEF  is  the  CLA  or  co-­‐CLA  for  3  clusters  and  two  areas  of  responsibility,  the  potential  for  seeking  a  common  funding  level  across  its  clusters  could  negatively  affect  the  capacity  built  within  the  GWC  thanks  to  its  strong  partnership  model.    One  possibility  raised  by  a  few  key  informants  is  the  introduction  of  a  partial  cost-­‐recovery  system  to  reimburse  the  GWC  for  FST  deployments  using  funds  raised  for  emergencies.  While  this  possibility  does  not  seem  to  have  been  thoroughly  analyzed  due  to  potential  complications  with  the  CLA’s  finance  systems,  such  a  system  has  successfully  been  established  in  other  international  humanitarian  agencies  for  their  surge  capacity.      For  example,  within  the  IFRC,  the  cost  of  Emergency  Response  Unit  (ERU)  deployments  and  the  costs  of  the  Field  Assessment  and  Coordination  Teams  (FACT)  are  regularly  reimbursed  to  the  organizational  unit  that  regularly  employs  deployed  staff  from  the  funds  globally  raised  for  an  operation.  While  potentially  complicated  for  the  GWC  with  its  diversity  of  members  that  provide  FST  staff,  such  a  model  could  significantly  increase  the  sustainability  of  the  FST  funding  model.

“The  key  to  sustainability  is  reinforcing  the  partners…but  there  is  a  massive  rotation  of  the  people  in  those  contexts,  so  difficult  to  sustain  anything.”    

Page 23: 2014.06.01 GWC Evaluation Final Report D8€¦ · 8* Table’2–’Diversity’of’Context’in’Evaluation’Case’Studies’ * * * * * * * * * * * * support support)

GWC  Evaluation  of  Support  to  National  Coordination  Platforms    Copyright  ©  Avenir  Analytics,  2014  

 

20  

3. Conclusions      The    FST  is  broadly  relevant  and  appropriate  within  IASC  standards  and  vis  a  vis  the  needs  of  national  coordination  structures.  GWC  efforts  have  been  broadly  effective  and  efficient  and  show  overall  value  for  money.  However  variance  in  effectiveness  is  linked  to  the  experience  and  soft  skills  of  FST  members,  and  variance  in  cost-­‐efficiency  is  driven  by  the  fragmented  evolution  of  FST  components  and  their  budget  models.    The  support  provided  shows  coherence  through  inter-­‐cluster  coordination  within  the  limits  of  larger  structures  beyond  the  GWC’s  control,  and  intra-­‐cluster  coordination  is  strong.  Connectedness  between  the  GWC  and  the  CLA  at  national  levels  is  not  as  strong  in  places  where  the  FST  has  not  been  active.      The  surge  capacity  and  support  functions  of  the  FST  have  had  a  positive  effect  on  coverage  of  coordination  needs,  though  perspectives  on  coverage  of  technical  needs  varies.        FST  support  has  demonstrated  an  effort  to  plan  for  transition  and  sustainability,  but  realizing  these  plans  depends  on  effective  recruitment  and  resource  mobilization.  Sustainability  of  the  FST  funding  and  structure  is  merited  by  the  value  it  adds  but  may  be  at  risk  without  addressing  burnout  and  further  consideration  of  alternative  funding  models.  

 The  lack  of  a  clear  theory  of  change  tied  to  the  GWC  strategy,  and  related  monitoring  and  evaluation  mechanisms  makes  it  more  difficult  to  assess  total  performance  and  show  value  for  money.    Agreed  metrics  and  new  methods  for  monitoring  collective  performance  are  merited  to  ensure  a  common  understanding  of  success.    The  findings  of  this  evaluation  support  the  ongoing  and  proposed  integration  of  the  three  components  of  the  FST  under  a  unified  CAST  management  to  improve  flexibility,  effectiveness  and  efficiency.  This  includes  proceeding  with  some  form  of  the  decentralized  operational  hubs  to  build  on  the  learned  benefits  of  the  RECA  system  and  refocusing  of  roles  within  the  FST  on  the  seven  functions  outlined  in  the  discussion  paper  for  the  GWC  SAG.      The  operational  hubs  or  greater  regional  focus  would  address  several  needs  highlighted  in  this  evaluation:  the  need  to  expand  the  base  of  local  relationships  and  knowledge  of  specific  country  contexts,  broaden  teams  involved  with  regions  to  allow  more  realistic  deployments  (reducing  turnover/burnout)  and  be  able  to  provide  better  response  through  coordination  and  assessment/IM  members  working  together.      

       

Page 24: 2014.06.01 GWC Evaluation Final Report D8€¦ · 8* Table’2–’Diversity’of’Context’in’Evaluation’Case’Studies’ * * * * * * * * * * * * support support)

GWC  Evaluation  of  Support  to  National  Coordination  Platforms    Copyright  ©  Avenir  Analytics,  2014  

 

21  

4. Recommendations      1. Proceed  with  the  proposed  further  

integration  of  FST  components  under  unified  management  of  the  CAST,  organized  into  functional  teams  and  assigned  to  decentralized  hubs.  Emergency  deployments  should  typically  include  a  minimum  of  two  members  each,  including  one  focused  on  coordination  efforts  (i.e.  FST  -­‐  coordination,  continuity,  capacity  building  and  preparedness)  and  one  focused  on  assessment/IM  (i.e.  FST  –  Assessment,  Information  Management  and  Quality  Assurance.)1      

2. Analyze  the  potential  for  implementing  a  partial  cost  recovery  model  and  other  mechanisms  to  increase  the  financial  sustainability  of  the  FST  and  develop  agreements  with  GWC  members  to  continue  helping  with  resource  mobilization  and  staff  recruitment  efforts.    

3. Strengthen  the  minimum  staff  reporting  structure  building  on  the  existing  dashboard  data  collection  to  facilitate  consistent  reporting,  contribute  to  value  for  money  analysis  and  support  integrated  management  of  FST  members.      

4. Continue  to  closely  collaborate  with  UNICEF’s  global  and  regional  WASH  sections  to  facilitate  coordination  preparedness,  

                                                                                                               1  Note  that  the  decentralized  hubs  do  not  necessarily  require  physical  location  in  a  regional  office,  as  it  is  recognized  that  this  would  involve  tradeoffs  (i.e.  increased  costs  vs.  proximity).  The  office  location  of  teams  should  be  determined  based  on  cost/need  specifics,  and  could  differ  by  region.      

national  capacity  building  and  other  tools  and  approaches  to  enhance  continuity.    Jointly  develop  country-­‐specific  preparation/preparedness  initiatives  in  support  of  national  coordination  platform  strategies  for  high  risk/lower  capacity  contexts.  This  type  of  coordination  should  enhance  the  continuity  from  preparedness  to  response  and  development  coordination  efforts  over  time  and  increase  awareness  of  the  potential  value  added,  when  needed,  of  GWC  FST  support.    

 5. Develop  a  theory  of  change  and  monitoring  

and  reporting  plan  linked  to  the  next  GWC  strategy  based  on  a  participatory  dialogue  among  SAG  members.  The  M&E  plan  should  consider  a  pilot  test  of  social  network  analysis.  As  more  fully  outlined  in  Appendix  8,  the  data  needed  for  this  analysis  can  be  gathered  through  web-­‐based  surveys  and  analyzed  with  free  software  that  provides  several  statistical  metrics  and  visualizations.  This  established  methodology  could  address  the  frequent  references  to  the  need  for  improved  M&E  of  FST  activities,  and  serve  the  GWC  in  two  primary  ways:      a. Provide  useful  data  related  to  the  key  

output  of  WASH  cluster  efforts:  the  quantity  and  quality  of  relationships  between  WASH  participants.  This  data  will  help  FST  leaders  and  the  SAG  identify  strengths/weaknesses  within  the  FST  and  help  FST  members  focus  their  efforts  on  key  needs  at  country  level.    

b. Provide  a  basis  for  attributing  FST  activities  to  beneficiary  impacts.    The  data  and  visualizations  of  this  analysis  links  the  relationships  formed  or  

Page 25: 2014.06.01 GWC Evaluation Final Report D8€¦ · 8* Table’2–’Diversity’of’Context’in’Evaluation’Case’Studies’ * * * * * * * * * * * * support support)

GWC  Evaluation  of  Support  to  National  Coordination  Platforms    Copyright  ©  Avenir  Analytics,  2014  

 

22  

strengthened  during  cluster  activities  to  the  resources  or  technical  knowledge  ultimately  impacting  beneficiaries.      

6. Explore  FST  use  of  a  standardized  mobile  electronic  data  collection  tool  (i.e.  Magpi)  for  assessment  and  information  management  and  pilot  the  training  of  national  coordination  platforms.  Such  tools  would  allow  coordination  platforms  to  develop  a  common  living  database  regarding  risks,  

vulnerabilities  and  capacities  to  serve  as  a  baseline  for  future  emergencies,  help  focus  readiness  efforts,  bring  WASH  agencies  into  more  efficient  synchronization  and  potentially  provide  data  to  eventually  measure  collective  impact.  Such  data  could  also  provide  a  platform  of  information  to  enhance  continuity  as  members  of  national  coordination  platforms  and  the  FST  change  over  time.  

     

Page 26: 2014.06.01 GWC Evaluation Final Report D8€¦ · 8* Table’2–’Diversity’of’Context’in’Evaluation’Case’Studies’ * * * * * * * * * * * * support support)

GWC  Evaluation  of  Support  to  National  Coordination  Platforms    Copyright  ©  Avenir  Analytics,  2014  

 

23  

5. List  of  Annexes      1) Terms  of  reference  2) Evaluation  methodology  description  3) List  of  documentary  data  sources    4) List  of  stakeholders  consulted  5) Interview  Guide  &  Scales  –  Country  Level  6) Interview  Guide  &  Scales  –  FST  Level  7) Evaluation  Matrix    8) Social  Network  Analysis  for  the  FST    

Page 27: 2014.06.01 GWC Evaluation Final Report D8€¦ · 8* Table’2–’Diversity’of’Context’in’Evaluation’Case’Studies’ * * * * * * * * * * * * support support)

GWC  Evaluation  of  Support  to  National  Coordination  Platforms    Copyright  ©  Avenir  Analytics,  2014  

 

24  

6. Acronyms  Used      ACAPS     Assessment  Capacities  Project  ACF     Action  Contre  la  Faim  CAR     Central  African  Republic  CAST     Cluster  Advocacy  and  Support  Team  CLA     Cluster  Lead  Agency  DAC     Development  Assistance  Committee  (of  the  OECD)  DSA     Daily  Subsistence  Allowance  ERU     Emergency  Response  Unit  FACT     Field  Assessment  and  Coordination  Team    FST     Field  Support  Team  GB     Great  Britain  GWC     Global  WASH  Cluster  IARRM     Inter-­‐Agency  Rapid  Response  Mechanism  IASC     Inter-­‐Agency  Standing  Committee  IDP     Internally  Displaced  People  IFRC     International  Federation  of  Red  Cross  and  Red  Crescent  Societies  IM     Information  Management  JMM     Joint  Monitoring  Mission  L3     Level  3  Emergency  M&E     Monitoring  and  Evaluation  MIRA     Multi-­‐Cluster  (Sector)  Initial  Rapid  Assessment  NCA     Norwegian  Church  Aid  NGO     Non-­‐Governmental  Organization  OCHA     Office  for  the  Coordination  of  Humanitarian  Assistance  OECD     Organization  for  Economic  Co-­‐operation  and  Development  PCA     Programme  Cooperation  Agreement  RAT     Rapid  Assessment  Team  RECA     Regional  Emergency  Cluster  (Coordination)  Advisers    RRT     Rapid  Response  Team  SAG     Strategic  Advisory  Group  TA     IASC  Transformative  Agenda  TOR     Terms  of  Reference  UK     United  Kingdom  UNICEF     United  Nations  Children’s  Fund  USA     United  States  of  America  UN     United  Nations  WASH     Water,  Sanitation  and  Hygiene