ri monitoring
DESCRIPTION
by arindam RayTRANSCRIPT
Sca ing-up RI programme
monitoring
Dr Arindam Ray
M&E Focal Person, WHO NPSP
SEPIO Meeting: 20th May, 2011
Presentation outline
Strategy
Generating evidence
Feedback and action
Expansion of the system
Way forward
Strategy
Recommendations on M&E for RI
GoI has circulated RI monitoring formats & SOPs to all states in July, 2009
IEAG, 2010 recommended that RI monitoring should be conducted in high priority, low coverage areas
Programme monitoring is a key component in national MO Training Handbook
RI monitoring Strategy
Strategic monitoring of HRAs Recognized HRAs (polio SIA microplan) Areas missed in RI microplan Villages with vacant sub-centres Peri-urban underserved areas ANM with large catchment population Migrant locations
Session monitoring Availability of vaccines & logistics Safe injection practices Tracking & mobilisation efforts
H-t-H (Community) monitoring Immunization status in community Reason for Left out & Drop out
Coordinate between Govt counterparts and partner organizations
Timely feedback (Block, District and State) for programme decisions
Consulted RI and SIA Microplan
Plan for monitoring based on Priority
Session Site
Session held
Session Monitoring
Session not held
Area for House to House Monitoring
Ten Households with 12-35 m children
Information from RI card & family members
Feedback to ANM
No Session Monitoring
1. Listed HRA (SIA Microplan)
2. Areas missed in Microplan
3. Villages with vacant Subcentres
4. Peri-Urban Underserved areas
5. ANM with large catchment population
6. MOB/ WPV / VDPV area
7. Migrant/ mobile
RI Monitoring Process
30 min ~1 hour
1~2 hours
Visit Block for monitoring & feedback
xkWao dk uke%
i zFke ?kj o fpUg%
ok; k ¼gksrs gq,½
vafre ?kj o fpUg%
xkWao dk uke%
i zFke ?kj o fpUg%
ok; k ¼gksrs gq,½
vafre ?kj o fpUg%
jkt s'k egrks] Hkou jk; ]uUnkl ko] jkt sUæBkdqj]jkt dqekj fo'odekZ]y{e.k fo'odekZ] HkksykfeL=h] t hokuUn mi k/; k;
vl QhZjk; ds ?kj l sjk; unqKku fudsru] cka; seqM+dj QqVdqu jk; ]t eknkj jk; ] vksei zdk'kfo'odekZ nka; s l s ck; sa
pkSFkk fnu
fo'odekZ Vksyh
fcUn] gfjt u ,oavU;
i kapok fnu
Lo0 l R; k egrks] i jekegrks] l quhy fxfj] mÙkejk; ] eqds'k jk; ] f'kodqekj egrks] l qxzho egrks
ckys'oj jke] xks/kuegrks] l gohj egrks] mekjk; ] nsokyh egrks] l okZegrks
fcUn Vksyh ,oa vU;
;kno Vksyk
nwl jk fnu rhl jk fnui gyk fnu
; kno ,oa vU;l qjs'k jk; ds ?kj l s Lo0l afpr jk; ] vf{krkuUn jk; ]fl i kgh Bkdqj] eaMy MªkbZoj]nqxkZjk; nksuksarjQ] eksgujk; ] fHk[kkjh jk;
vl jQh jk; ds ?kj l sjk"Vªh; Kku fudsru cka; sy[ku jk; t ehankj jk;vksei zdk'kfo'odekZnk; sal scka; segkohj jk; ] jkek/kkjjk; ] jkenso jk;
cPpq jk; ds ?kj l sl q[kkM+h jk; ] uckc jk; ]Hkksyk jk; ] jfoUæ jk; ]ukxsUæ Bkdqj] dkaxzsl jk;
; kno ,oa vU;jkeckcqjk; ds?kj t qxyjk; ] jke dqekj jk; dsJ hHkksyk jk; ] j?kqoj Bkdqj]d"̀.kk jk;
i zFke ,oanwl jsfnu dkX?kjksa dk i quHkZze.k ,oai zfrj{k.k LFky l sdk; Zdjuk gSA
uskan
eqckjdiqj QkeZ inkol gom
t ol i ioyrk
gfjt u ,oa vU;ujs'kjk; ds?kj l spqUuqjk; ]l rsUæjk; ] yky l kgsc jk; ]Hkksyk l ko] mt su jk; ]f'koi wt u jk;
1st & 3rd
Wed-
2nd Sat -
Wed- 2nd
eqckjdiqj QkeZ
eqckjdiqj QkeZ
Uskantoli inkol gom;kno Vksy
k
ioyrk
3rd Sat -4th Wed-
4th Sat -1st Sat -
SIA Microplan
RI Microplan
? ?
?
Plan for Mubarakpur S/C (Session) and Bind tola(HtH)12
Generating evidence
Indicators derivable from RI session monitoring
Sessions held or not? If not, reasons why
ANM/vaccinator was present as per microplan (%)
Utilization of AVD Availability of vaccine and
logistics: Vaccines (BCG,DPT,OPV, MCV,TT,
Hep B, JE, PV) Diluents (for BCG, MCV, JE) Syringes (AD: >0.1ml & 0.5ml,
Disposable). Unusable vaccines
Mobilisation of beneficiaries
Injection safety
AEFI management & reporting
Waste disposal New born tracking Vaccine safety (frozen,
unusable VVM, expired vaccine etc.)
Key RI messages conveyed to parents of beneficiaries
Which message is missed
Availability of other logistics: Vit A, PCM, ORS Hub cutter, RI cards, R&B bags
Stock-outs Supervisory visits
Key Indicators Additional Indicators
Planned sessions not conducted: Jan10-Dec10: Bihar (with reasons)
Conducted,
86%
Not
conducted,
14%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Bihar (n=30,604)
Source: RI monitoring data (Jan10-Dec10)
Reason for session not held (n=4356)
64%5%
8%
16%
7%
ANM absent, vaccine/ logistics not deliveredANM absent, vaccine/ logistics available ANM present, vaccine/ logistics not availableOthersUnknown
% Availability of all vaccines at sessions sites (Year – 2010 & 1st Quarter 2011)
JBSA – Jachha Bachha Suraksha Abhiyan -UP
80% 80% 80% 80%
70%
80% 80%
70%
50%
80%
90%
80% 81% 80%
88%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
JBSA started form August’10
Visited1
Up.shp
BCG_0M0 - 4040.1 - 6060.1 - 8080.1 - 100
Visited1
Up.shp
BCG_0M0 - 4040.1 - 6060.1 - 8080.1 - 100
Data not available
<= 40%
40% to 60%
60% to 80%>= 80%Not monitored
UP districts Cumulative Jan’10 - Dec’10 : 80%Sessions held: 59,811
UP districts CumulativeJan’11 – Mar’11: State Average-
83%Sessions held: 17,573
KRI
SBD
STP
LLP
HDO
JAL
JNS
AHB
BJN
BAD
BRC
PIL
RBL
SHA
SUL
MZP
BBK
BRL
FTP
BNA
JNP
UNN
AZG
MZN
AGR
ALG
SHP
HMP
BLS
GND
PTG
KSN
MRD
BRP
GZP
BAL
GRPBST
KPN
CKT
MTR
MAI
KPD
MHB
FAI
ETA
SDN
MRT
LNO
DOR
JPN
FKB
CND
FER
ABN
RMP
MHG
ETW
KNA
KAN
AUR
GZA
HTR
KSM
SRW
MAU
SKN
VRN
BGT
GBN
BDH
AEFI of Mohanlal Ganj (Lucknow)
KRI
SBD
STP
LLP
HDO
JAL
JNS
AHB
BJN
BAD
BRC
PIL
SHA
MZP
BRL
FTP
BNA
JNP
UNN
AZG
MZN
AGR
ALG
SHP
BBK
HMP
BLS
GND
PTG
KSN
MRD
BRP
RBL
FAI
GZP
BAL
GRPBST
KPN
CKT
MTR
MAI
KPDSULCSN
MHB
ETA
SDN
MRT
LNO
DOR
JPN
FKB
CND
FER
ABN
RMP
MHG
ETW
KNA
KAN
AUR
GZA
HTR
KSM
SRW
MAU
SKN
VRN
BGT
GBN
BDH
KRI
SBD
STP
LLP
HDO
JAL
JNS
AHB
BJN
BAD
BRC
PIL
SHA
MZP
BRL
FTP
BNA
JNP
UNN
AZG
MZN
AGR
ALG
SHP
BBK
HMP
BLS
GND
PTG
KSN
MRD
BRP
RBL
FAI
GZP
BAL
GRPBST
KPN
CKT
MTR
MAI
KPDSULCSN
MHB
ETA
SDN
MRT
LNO
DOR
JPN
FKB
CND
FER
ABN
RMP
MHG
ETW
KNA
KAN
AUR
GZA
HTR
KSM
SRW
MAU
SKN
VRN
BGT
GBN
BDH
<Empty Picture><Empty Picture><Empty Picture><Empty Picture>
Mobilisation of Children Bihar, 2010
47%
53%
YES NO
41%
59%
YES NO
Due List PreparedCounterfoils Updated
N=21,706
N=26248
N=26,248
N=24511
Mobilisers Delivery of 4 key messages
71%
29%
YES NO
43%17%
3%10%
27%
0%
ICDS, ASHA & Others ICDS & ASHAOnly ICDS Only ASHA
Others None
% Any Mobilizer (ICDS/ASHA/Others) present at the session sites, Jan-Mar
2011, BiharState average
Jan-Mar2011: 91 %
Nil1 - 30 %31 - 59 %60 - 79 %>= 80 %Data Not Available
Col No.
KAIMURKATIHARKHAGARIAKISHANGANJLAKHISARAIMADHEPURAMADHUBANIMUNGERMUZAFFARPURNALANDANAWADAPATNAPURNIAROHTASSAHARSASAMASTIPURSARANSHEIKHPURA
Ga ya
Patna
Jamui
Rohtas
Purnia
Kaimur
Banka
Ara ria
Saran
Katihar
Siwan
Supaul
Madhubani
Nawada
Champaran We st
BhojpurBuxar
Nala nda
Muzaffarpur
Bhagalpu r
Aurang aabad
Si tamarhi
Vaishali
Darbhanga
Champaran East
Sam astipur
Gopalga nj
Saharsa
Beg usa ra i
Munger
Kha garia
Madhepura
Kis han ganj
ArwalLakhisaraiJe hanabad
She ikhpura
She ohar
88 89 90 9188
91 89 90 91 91 91 90 90 9188 89 90 90 92
0
10
20
30
40
50
60
70
80
90
100
Se
p'0
9
Oct
'09
No
v'0
9
De
c'0
9
Jan
'10
Fe
b'1
0
Ma
r'10
Ap
r'10
Ma
y'1
0
Jun
'10
Jul'1
0
Au
g'1
0
Se
p'1
0
Oct
'10
No
v'1
0
De
c'1
0
Jan
'11
Fe
b'1
1
Ma
r'11
Source: RI session monitoring data by WHO NPSP and Govt. of BiharN = 7840 RI session found held
Nil
1 – 30 %
31 – 60%
61 – 80 %
> 80 %Data Not Available
Source: RI session monitoring data N = 5,426 RI sessions found held
% Due list availability at the session site,
Jan-Dec 2010, JharkhandState average 2010:
65 %
GUMLA
GIRIDIH
RANCHI
PALAMU
DUMKA
LATEHAR
CHATRAGARHWA
SIMDEGA
SINGHBHUM WEST
HAZARIBAGH
KHUNTI
GODDA
BOKARO
PAKUR
DEOGHAR
SARAIKELLA
DHANBAD
JAMTARA
SINGHBHUM EAST
SAHIBGANJ
RAMGARH
KODERMA
LOHARDAGA
Safe Injection Practices Bihar, Jan – Dec 2010
4%
96%
Yes No
DPT Given at Mid-Thigh
ANM Touching the needle
95%
5%
Yes No
92%
8%
Yes No
65%
35%
Yes No
n=24,706
n=21,909
n=23,607
n=1078
Time of Reconstitution Mentioned on Vial
Hub of syringe is cut in hub cutter immediately
n = number of RI sessions monitored
Source: RI session monitoring data by WHO NPSP and Govt. of BiharN = 1,151 sites where Vaccination seen
% Syringes being cut by hubcutter immediately after use, Jan-Mar 2011, Bihar
State average Jan-Mar2011: 85 %
Nil1 - 30 %31 - 59 %60 - 79 %>= 80 %Data Not Available
Col No.
KAIMURKATIHARKHAGARIAKISHANGANJLAKHISARAIMADHEPURAMADHUBANIMUNGERMUZAFFARPURNALANDANAWADAPATNAPURNIAROHTASSAHARSASAMASTIPURSARANSHEIKHPURA
Ga ya
Patna
Jamui
Rohtas
Purnia
Kaimur
Banka
Ara ria
Saran
Katihar
Siwan
Supaul
Madhubani
Nawada
Champaran We st
BhojpurBuxar
Nala nda
Muzaffarpur
Bhagalpu r
Aurang aabad
Si tamarhi
Vaishali
Darbhanga
Champaran East
Sam astipur
Gopalga nj
Saharsa
Beg usa ra i
Munger
Kha garia
Madhepura
Kis han ganj
ArwalLakhisaraiJe hanabad
She ikhpura
She ohar
54
42
57 5955 53
6359
63 62
71 71
79
55
69 71
80
86 87
0
10
20
30
40
50
60
70
80
90
100
Se
p'0
9
Oct
'09
No
v'0
9
De
c'0
9
Jan
'10
Fe
b'1
0
Ma
r'10
Ap
r'10
Ma
y'1
0
Jun
'10
Jul'1
0
Au
g'1
0
Se
p'1
0
Oct
'10
No
v'1
0
De
c'1
0
Jan
'11
Fe
b'1
1
Ma
r'11
Monitored sessions with safe injection practices, Uttar Pradesh
100%
84%
92% 91%
30%
100%
87%93% 95%
30%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
AD Syringe used forinjectable vaccines
(n=56,896 / 16,961)
DPT givenanterolaterally
(n=52,994 / 16,111)
ANM not touching anypart of needle while
giving injection(n=55,149 / 16,555)
Reconstitution timewritten on vials
(n=49,133 / 14,855)
Syringe cutimmediately after use(n=41,686 / 11,509)
2010 2011 (1st Quarter)
n=monitored sessions where corresponding activities were observed ( 2010 / 2011- 1st Quarter)
Indicators derivable from household monitoring
No of Children in 12-35 months fully immunized (%)
Children receiving age specific antigens including
Hep B, MCV2, DPT Booster
Reasons for not getting the due vaccine
Drop out and left out rate Reasons for drop out & Left
out Areas where RI sessions are
not held in last 3 months
Mobilisation of beneficiaries Presence of ASHA/ AWW Participation by PRI/ SHG/
NGO Vaccination status in
community: By migration By religion By caste By gender By occupation By education
No of beneficiaries provided with RI/ MCP card (%)
Retention of RI / MCP card (%)
Key Indicators Additional Indicators
39% 39% 41% 45% 46% 47% 46% 50% 50% 50% 49% 50%
42% 43% 42%40% 40% 39% 40%
37% 37% 38% 39% 37%
18% 17% 17% 15% 14% 14% 14% 13% 13% 13% 12% 13%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Jan
(n=4087)
Feb
(n=7142)
Mar
(n=15129)
Apr
(n=16189)
May
(n=11048)
Jun
(n=14176)
Jul
(n=22352)
Aug
(n=20403)
Sep
(n=12239)
Oct
(n=18920)
Nov
(n=10404)
Dec
(n=24545)
Fully immunized Partially immunized Unimmunized
n=Number of children 12 to 23 month of age (176,634)
Immunization status of monitored children 12-23 months, UP, Jan-Dec
2010
Fully immunized: 47.1%
Partially immunized: 39.1%
Left-outs: 13.9%
Full immunization status and BCG-measles drop out rates, Bihar, Jan-Dec
2010
BCG-Measles Drop-outBelow 1010 % to 30 %30 % to 50 %50% to 60 %
Up.shpFIC (Fully immunized coverage)
BCG-measles drop out rate
n = 72,162 children 12 to 23 month of age
Col No.
KAIMURKATIHARKHAGARIAKISHANGANJLAKHISARAIMADHEPURAMADHUBANIMUNGERMUZAFFARPURNALANDANAWADAPATNAPURNIAROHTASSAHARSASAMASTIPURSARANSHEIKHPURA
Gaya
Patna
Jamui
Rohtas
Purnia
KaimurBanka
Araria
Saran
Katihar
Siwan
Supaul
Madhubani
Nawada
Champaran West
BhojpurBuxar
Nalanda
Muzaffarpur
Bhagalpur
Aurangaabad
Sitamarhi
Vaishali
Darbhanga
Champaran East
Samastipur
Gopalganj
Saharsa
Begusarai
Munger
Khagaria
Madhepura
Kishanganj
ArwalLakhisaraiJehanabad
Sheikhpura
Sheohar
Col No.
KAIMURKATIHARKHAGARIAKISHANGANJLAKHISARAIMADHEPURAMADHUBANIMUNGERMUZAFFARPURNALANDANAWADAPATNAPURNIAROHTASSAHARSASAMASTIPURSARANSHEIKHPURA
Gaya
Patna
Jamui
Rohtas
Purnia
KaimurBanka
Araria
Saran
Katihar
Siwan
Supaul
Madhubani
Nawada
Champaran West
BhojpurBuxar
Nalanda
Muzaffarpur
Bhagalpur
Aurangaabad
Sitamarhi
Vaishali
Darbhanga
Champaran East
Samastipur
Gopalganj
Saharsa
Begusarai
Munger
Khagaria
Madhepura
Kishanganj
ArwalLakhisaraiJehanabad
Sheikhpura
Sheohar
20 - 39%40 - 49%50 - 59%60 - 90%
33%8%
7%
16%
5%6%
25%
Immunization status of monitored children, Jan-Dec, 2010, Bihar
61%
33%
6%
0.0%
10.0%
20.0%
30.0%
40.0%
50.0%
60.0%
70.0%
80.0%
90.0%
100.0%
BIHAR (n=72,162)
Reason for children not being fully immunized
Fully Immunized
PartiallyImmunized
No Immunization
n=children 12 to 23 months of age
19%
4%
30%
9%
5%
21%12%
N=23,783
N=4,234
Indicators derivable from Block/ District monitoring
Linkage with session monitoring
ANM/vaccinator vacancy status
Utilization of AVD Availability of vaccine
and logistics (stock outs)
Cold Chain Report-returns
Training status Supportive
supervision Review mechanism AEFI reporting VPD surveillance Functioning of
committees: Task Force ERT/ RRT AEFI committee
Key Indicators Additional Indicators
Feedback and action
November 2010:8 of 19 districts (39%)
Districts with HR blocks where RI monitoring data reviewed during the DTF meeting at least once during 2 months
January 2011:19 of 19 districts (100%)
Districts with HR blocks where RI monitoring data not reviewed during the DTF meeting at least once during 2 months
Districts with no HR blocks
Data not available
RI monitoring feedback in high risk areas: Use of District Task Force (DTF)
Information feedback process
Core indicators: Immediate verbal and/or written feedback to Block MO
and District Immunization Officer Written feedback of core indicators forwarded to State
Immunization Officer
Detailed analysis: Data is compiled, entered and cleaned at state level Exhaustive analysis (by district and blocks) shared with
districts by states
RI Monitoring feedback is regularly discussed and shared in RI Cell meetings
in Jharkhand, Bihar, UP, Orissa….
Expansion of the system
Scaling up RI monitoring
Following a state review meeting, Karnataka has started monitoring with special emphasis on northern districts
Rajasthan has also initiated RI monitoring since end of 2010
Govt officials in Punjab are spearheading monitoring of RI sessions for last six months
West Bengal has also initiated RI monitoring with more emphasis on HR districts like Murshidabad, Howrah etc
Medical colleges in Orissa are monitoring RI sessions in selected districts
Govt of Delhi has also launched RI monitoring in last 1 year
RI monitoring is planned to be introduced in the coming months to: Maharashtra, Madhya Pradesh, Kerala, Tamilnadu, Chhattisgarh, Assam…..
Every state to operationalise RI
monitoring with GoI formats
Formats to be used by all States:Session and household monitoring
Formats to be used by all States:Block and District level
Easy to use
Structured MCQ
KEYs given
One pager
Way forward…
All states are expected to initiate monitoring Adapt strategy and tools to local situations Ensure feedback and follow-up interventions Technical assistance is available from
partners New technologies: OMR/ PDA Oversight is critical to ensure quality of data Monitoring to be dynamic to capture
changing practices and priorities
Thank You
Strengthening Monitoring
Problem in the content: Multiple variables clubbed Some essential components missing No data on determinants No monitoring for mid-level operations at the Block and District
levels
Change in the Context: Evolving practices and priorities since 2009 MCV2 and Pentavalent are being introduced Hep B to be rolled out to all remaining states Lessons from session & RCA monitoring in MCUP Need for universalisation of the monitoring tools
Data tools also need further updating and revision