ASER INDIA

Strengthening the process through innovations over time

Introduced “Recheck”:•2-4 villages to be backchecked post survey in every district

Substantial changes to:•training model•survey model•monitoring & recheck model•Created repository of information to measure data quality

Independent process audit

Collective Survey in MP:•Entire ASER team posted in one state for 3 weeks•Many valuable lessons learnt

Consolidation & growth•Building on innovations of 2011

Changes to the process over time

Lets discuss some of these innovations..

Many substantial and successful measures were introduced in ASER 2011

a) Mock trainings• All team members at national and state level practiced training• Helped them revise their content and delivery

1. Trainings

b) Rigorous field practice• In all 4 day state trainings, 1 day was spent practicing the survey and 1 day was

spent practicing recheck• Helped merge theory with practice for the participants

c)Evaluation & Master trainer (MT) profiling• All MTs and surveyors were given a quiz to assess their understanding of the survey

process• Only the ones who passed satisfactorily were allowed to train/do the survey• MT information and performance (in field, in class, and mock trainings) from every state

training was documented and results were analyzed. The goal is to create a pool of good MTs who can be relied on each year for ASER and other studies as well

d) Better training material• Multimedia content used for training• Flex banners – durable; low-cost substitute for projectors in remote districts; allow

practice of filling survey formats

1. Trainings

e) District level training duration now 3 days instead of 2

ASER 2011 - Innovations

a) “2 weekend” surveys instead of the usual “one weekend” ones• In most districts, 30 villages were surveyed over two weekends in batches of

15 each weekend• Only 30 volunteers needed instead of 60 – better PTR and hence improved training

quality• Increased amounts of monitoring and recheck between the two weekends

2. Survey Model

Cost of ASER increased a fair bit, but so did the quality of survey

ASER 2011 - Innovations

b) “Purposive” Nature• For the first time in ASER, the districts and villages for monitoring and recheck were

selected on the basis of certain pre-defined criteria (for e.g. volunteers surveying that village scored low on quiz etc

• Feedback from the state teams• In previous years, villages to be rechecked were selected randomly – this purposive re-

check was in addition

3. Monitoring & Recheck

c) Call Centre at state level• An ASER team member was responsible for regularly telephoning Master Trainers in

every district to monitor the progress of the survey• ensured instant troubleshooting of problems and prompt support to remote or

problematic districts

a) Increased magnitude• In ASER 2011, at least 4-8 villages were monitored in every district• Another 4-8 villages were also rechecked in every district

ASER 2011 - Innovations

d) Employing mobile technology – the SMS Recheck• Successfully employed in Rajasthan, Gujarat, Chhattisgarh and Himachal Pradesh• MTs texted village related data to a designated phone number• All SMSes were downloaded from a website and data analyzed• Villages for recheck were assigned• Also, provided a sense of the quality of survey at an early stage

e) New recheck method• An extremely detailed recheck method was devised which stopped just short of re-

testing the child • New comprehensive recheck format – piloted by entire ASER National team• 25% of time in all state trainings devoted to understanding and practice of new

methods

f) ASER Centre Recheck• Entire ASER national team made cross-state visits to conduct rechecks• 14 states and 43 districts were covered in this second round of recheck

3. Monitoring & Recheck

ASER 2011 - Innovations

e) “District Compilation Sheet” (DCS)• recorded contact numbers, attendance information and quiz performance of all

25,000 surveyors• Data was digitized and information analyzed; this enabled:

i) purposive recheck ii) Research on data quality – revision to the operational models for ASER 2012iii) The data will also be used for dissemination purposes

4.Repository of Information

In ASER 2011, approximately 41% of all villages surveyed were either monitored or rechecked by Master Trainers

ASER 2011 - Innovations

ASER 2012 – consolidation and continuing to innovate

• All the successful measures from ASER 2011 have been retained and are being strengthened

• Based on the research on data and operations, processes have been tweaked if required

Some new features have also been introduced in ASER 2012:

a) Increased duration of state trainings• State trainings are now 5 days instead of the 4 in previous years• More class work, more practice in the field, more effective mock trainings

b) Novel use of the District Compilation (DCS) sheet• In addition to all the applications last year, the DCS data will serve as a proxy for the

ASER 2012 data• Inferences shall be drawn 2 months earlier than usual; time to investigate learning

level trends

UP- DCS UP- ASER BH- DCS BH- ASER CHH- DCS CHH- ASER

MP- DCS MP- ASER

MH- DCS MH- ASER

0%

20%

40%

60%

80%

100%

Math Levels(2011):District Compilation Sheet Vs ASER HH Data

Nothing Num (1-9) Num (10-99) Subtraction Division

Perfect matches in case of other states too!; holds true for Reading levels also! We’re getting these numbers through SMSes in many states to ensure early receipt of data

ASER 2012 – consolidation and continuing to innovate

THANKS!

All feedback is most welcome

We’re committed to improving quality through continuous innovation