ms. trina vithayathil, public affairs foundation, bangalore citizen report card methodology the...

Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore

Citizen Report Card Citizen Report Card MethodologyMethodology

THE BUILDING BLOCKS


Overview of CRC Overview of CRC MethodologyMethodology

Defining Research Objectives

Development of Instrument

Data collection

Analysis

Interpretation

Sampling Design


The CRC JourneyThe CRC Journey



Data collection

Analysis

Interpretation

Sampling Design


Examples of Research Examples of Research ObjectivesObjectives

General-Getting feedback from people on quality of

public servicesSpecific - Finding out satisfaction with quality of

service, satisfaction with behaviour of staff, extent of corruption, problem resolving capabilities of the agency


The CRC JourneyThe CRC Journey



Data collection

Analysis

Interpretation

Sampling Design


Designing the InstrumentDesigning the Instrument

Basic structure of a schedule/ questionnaire

Investigator informationIntroductionFilter questionsDemographicsThe body of the schedule - sections if

needed - sub- schedules if required


Thumb Rules for a Schedule Thumb Rules for a Schedule

All research objectives to be coveredShortest length possibleLanguage - simple and convenient

based on respondents’ and investigator’s level of understanding

Clear instructions for the investigator


Basic Tools in Instrument Basic Tools in Instrument DesignDesign

Different types of questions Open-ended and closed questions /

combinations yes/ no questions

Coding - pre and post codingUse of show cardsScaling - 5 pt or 2 ptSkipsGrids


CRC Process ChartCRC Process Chart



Data collection

Analysis

Interpretation

Sampling


Sampling DesignSampling Design

Why sampling?Importance of representativenessHow to assure representativeness

Fixing quotas by area or category or both Choosing appropriate sample size

• time constraints and cost constraints

• standard error

• level of sub classification required for analysis and conclusions


SAMPLING DESIGNSAMPLING DESIGN

BMP: 606 general households 812 slum households

General Households: 10 wards from each of 3 zones selected Number of households allocated according to population of ward Mains and crosses selected from each ward Proportionate number of households selected from each street

MAIN SAMPLE:2000 households from city and suburban areas: 800 general households 1200 slum households

Slum Households: Slums selected from each region using PPS Proportionate allocation according to slum type & size within regions Households selected using right-hand rule in each slum through systematic sampling


The Research Journey...The Research Journey...



Data collection

Analysis

Interpretation

Sampling


Field Issues...Field Issues...

Method of data collection Right field teamTraining/Briefing of investigatorsRegular reportingSpot checks and back checksTime scheduleAdherence to budget


The Right TeamThe Right TeamHow do we decide?How do we decide?

Familiarity with locality, languageAble to relate to target audienceIntelligentAvailable for the entire duration of

fieldwork And, INTEGRITY


The Right TeamThe Right TeamHow many?How many?

f {Sample size, Timeline, Productivity }

Sample size : 2000Timeline : 20 daysProductivity per person/ day : 5Team size = [(2000/20) / 5]

= 20 Investigators


The Final Team...The Final Team...

Field Supervisor

Field Supervisor

5 Interviewers5 Interviewers

Field ManagerField Manager

5 Interviewers5 Interviewers5 Interviewers5 Interviewers5 Interviewers5 Interviewers

Field Coordinator

Field Coordinator

Field Supervisor

Field Supervisor

Field Supervisor

Field Supervisor

Field Supervisor

Field Supervisor


Data Collection Data Collection

Random sampling

Starting points selection Sample Size : 1000

100 starting points

10 listings around each starting point

Right hand rule

Boosters-- phone call, exit interviews


Quality ChecksQuality Checks

At all stages

Before launching fieldwork

During fieldwork

After fieldwork



Before fieldwork Pilot exercise10-15 interviewsCheck for

LengthComprehensionFlowSensitiveness



During fieldwork

Accompanied interviews

Sample of each interviewer’s work spot checked by field supervisor

10% back checks by the field coordinator

100% scrutiny of schedules



After fieldwork

Sample sizes

Quotas

Representation


Cost Heads in FieldworkCost Heads in Fieldwork

Cost is a function of Nature of projectType of respondentSelection criteriaSample size, sampling

Two broad types

OverheadExpenses


Cost Heads in FieldworkCost Heads in Fieldwork

OverheadsTime cost, personnel

Communication: telephone, computers

Courier / mailings


Cost Heads in FieldworkCost Heads in FieldworkExpenses

Printing of questionnaires Briefing and piloting Interviewer fees Supervisor fees Local conveyance Outstation travel : Field officer, interviewers Others: Venue, equipment, moderation,

gifts, transcriptions Translation





Data collection

Analysis

Interpretation

Sampling


Data AnalysisData Analysis

Preliminary analysis by research agencyFurther analysis by PAC


Framework of AnalysisFramework of Analysis

Select computer package - Foxpro, SPSS

Run basic demographics

Check if this is in line with population

If yes, proceed with further analysis

If no, weight data to represent population


Framework of AnalysisFramework of AnalysisWhat is the depth of analysis required?What are the relevant analysis heads?

By demographics By agency By responses to other questions :

• satisfied vs. dissatisfied

• had problem vs. did not have problem

• paid bribe vs. did not pay bribeSimple measures used- percentages and mean

scores. Recently regression models





Data collection

Analysis

Interpretation

Sampling


Compare across agencies - rank them on satisfaction, responsiveness and corruption- using percentages and means

Within agencies - specific information - timings of water supply, quality of water - again using percentages and means

Two ways of interpreting same information - eg - number of visits to agency

Interpretation of DataInterpretation of Data


Interpretation of DataInterpretation of Data

Assimilation of scattered findings into cohesive whole

Translation of findings into diagnostic statements

Conversion of diagnosis into conclusions

Formulation of recommendations based on conclusions


From Findings to From Findings to RecommendationsRecommendations

Satisfaction with behaviour of staffObjective statement of finding -

percentage satisfied, dissatisfied, ambivalent.

Conclusion - general / high/low satisfaction/dissatisfaction

Recommendation - training for staff to improve their behaviour

ms. trina vithayathil, public affairs foundation, bangalore citizen report card methodology the...

Documents