citizen report card methodology
DESCRIPTION
Citizen Report Card Methodology. THE BUILDING BLOCKS. Overview of CRC Methodology. Defining Research Objectives. Development of Instrument. Sampling Design. Data collection. Analysis. Interpretation. The CRC Journey. Defining Research Objectives. Development of Instrument. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Citizen Report Card Methodology](https://reader036.vdocument.in/reader036/viewer/2022062517/56813b1d550346895da3cfed/html5/thumbnails/1.jpg)
Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore
Citizen Report Card Citizen Report Card MethodologyMethodology
THE BUILDING BLOCKS
![Page 2: Citizen Report Card Methodology](https://reader036.vdocument.in/reader036/viewer/2022062517/56813b1d550346895da3cfed/html5/thumbnails/2.jpg)
Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore
Overview of CRC Overview of CRC MethodologyMethodology
Defining Research Objectives
Development of Instrument
Data collection
Analysis
Interpretation
Sampling Design
![Page 3: Citizen Report Card Methodology](https://reader036.vdocument.in/reader036/viewer/2022062517/56813b1d550346895da3cfed/html5/thumbnails/3.jpg)
Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore
The CRC JourneyThe CRC Journey
Defining Research Objectives
Development of Instrument
Data collection
Analysis
Interpretation
Sampling Design
![Page 4: Citizen Report Card Methodology](https://reader036.vdocument.in/reader036/viewer/2022062517/56813b1d550346895da3cfed/html5/thumbnails/4.jpg)
Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore
Examples of Research Examples of Research ObjectivesObjectives
General-Getting feedback from people on quality of
public servicesSpecific - Finding out satisfaction with quality of
service, satisfaction with behaviour of staff, extent of corruption, problem resolving capabilities of the agency
![Page 5: Citizen Report Card Methodology](https://reader036.vdocument.in/reader036/viewer/2022062517/56813b1d550346895da3cfed/html5/thumbnails/5.jpg)
Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore
The CRC JourneyThe CRC Journey
Defining Research Objectives
Development of Instrument
Data collection
Analysis
Interpretation
Sampling Design
![Page 6: Citizen Report Card Methodology](https://reader036.vdocument.in/reader036/viewer/2022062517/56813b1d550346895da3cfed/html5/thumbnails/6.jpg)
Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore
Designing the InstrumentDesigning the Instrument
Basic structure of a schedule/ questionnaire
Investigator informationIntroductionFilter questionsDemographicsThe body of the schedule - sections if
needed - sub- schedules if required
![Page 7: Citizen Report Card Methodology](https://reader036.vdocument.in/reader036/viewer/2022062517/56813b1d550346895da3cfed/html5/thumbnails/7.jpg)
Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore
Thumb Rules for a Schedule Thumb Rules for a Schedule
All research objectives to be coveredShortest length possibleLanguage - simple and convenient
based on respondents’ and investigator’s level of understanding
Clear instructions for the investigator
![Page 8: Citizen Report Card Methodology](https://reader036.vdocument.in/reader036/viewer/2022062517/56813b1d550346895da3cfed/html5/thumbnails/8.jpg)
Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore
Basic Tools in Instrument Basic Tools in Instrument DesignDesign
Different types of questions Open-ended and closed questions /
combinations yes/ no questions
Coding - pre and post codingUse of show cardsScaling - 5 pt or 2 ptSkipsGrids
![Page 9: Citizen Report Card Methodology](https://reader036.vdocument.in/reader036/viewer/2022062517/56813b1d550346895da3cfed/html5/thumbnails/9.jpg)
Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore
CRC Process ChartCRC Process Chart
Defining Research Objectives
Development of Instrument
Data collection
Analysis
Interpretation
Sampling
![Page 10: Citizen Report Card Methodology](https://reader036.vdocument.in/reader036/viewer/2022062517/56813b1d550346895da3cfed/html5/thumbnails/10.jpg)
Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore
Sampling DesignSampling Design
Why sampling?Importance of representativenessHow to assure representativeness
Fixing quotas by area or category or both Choosing appropriate sample size
• time constraints and cost constraints
• standard error
• level of sub classification required for analysis and conclusions
![Page 11: Citizen Report Card Methodology](https://reader036.vdocument.in/reader036/viewer/2022062517/56813b1d550346895da3cfed/html5/thumbnails/11.jpg)
Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore
SAMPLING DESIGNSAMPLING DESIGN
BMP: 606 general households 812 slum households
General Households: 10 wards from each of 3 zones selected Number of households allocated according to population of ward Mains and crosses selected from each ward Proportionate number of households selected from each street
MAIN SAMPLE:2000 households from city and suburban areas: 800 general households 1200 slum households
Slum Households: Slums selected from each region using PPS Proportionate allocation according to slum type & size within regions Households selected using right-hand rule in each slum through systematic sampling
![Page 12: Citizen Report Card Methodology](https://reader036.vdocument.in/reader036/viewer/2022062517/56813b1d550346895da3cfed/html5/thumbnails/12.jpg)
Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore
The Research Journey...The Research Journey...
Defining Research Objectives
Development of Instrument
Data collection
Analysis
Interpretation
Sampling
![Page 13: Citizen Report Card Methodology](https://reader036.vdocument.in/reader036/viewer/2022062517/56813b1d550346895da3cfed/html5/thumbnails/13.jpg)
Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore
Field Issues...Field Issues...
Method of data collection Right field teamTraining/Briefing of investigatorsRegular reportingSpot checks and back checksTime scheduleAdherence to budget
![Page 14: Citizen Report Card Methodology](https://reader036.vdocument.in/reader036/viewer/2022062517/56813b1d550346895da3cfed/html5/thumbnails/14.jpg)
Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore
The Right TeamThe Right TeamHow do we decide?How do we decide?
Familiarity with locality, languageAble to relate to target audienceIntelligentAvailable for the entire duration of
fieldwork And, INTEGRITY
![Page 15: Citizen Report Card Methodology](https://reader036.vdocument.in/reader036/viewer/2022062517/56813b1d550346895da3cfed/html5/thumbnails/15.jpg)
Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore
The Right TeamThe Right TeamHow many?How many?
f {Sample size, Timeline, Productivity }
Sample size : 2000Timeline : 20 daysProductivity per person/ day : 5Team size = [(2000/20) / 5]
= 20 Investigators
![Page 16: Citizen Report Card Methodology](https://reader036.vdocument.in/reader036/viewer/2022062517/56813b1d550346895da3cfed/html5/thumbnails/16.jpg)
Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore
The Final Team...The Final Team...
Field Supervisor
Field Supervisor
5 Interviewers5 Interviewers
Field ManagerField Manager
5 Interviewers5 Interviewers5 Interviewers5 Interviewers5 Interviewers5 Interviewers
Field Coordinator
Field Coordinator
Field Supervisor
Field Supervisor
Field Supervisor
Field Supervisor
Field Supervisor
Field Supervisor
![Page 17: Citizen Report Card Methodology](https://reader036.vdocument.in/reader036/viewer/2022062517/56813b1d550346895da3cfed/html5/thumbnails/17.jpg)
Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore
Data Collection Data Collection
Random sampling
Starting points selection Sample Size : 1000
100 starting points
10 listings around each starting point
Right hand rule
Boosters-- phone call, exit interviews
![Page 18: Citizen Report Card Methodology](https://reader036.vdocument.in/reader036/viewer/2022062517/56813b1d550346895da3cfed/html5/thumbnails/18.jpg)
Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore
Quality ChecksQuality Checks
At all stages
Before launching fieldwork
During fieldwork
After fieldwork
![Page 19: Citizen Report Card Methodology](https://reader036.vdocument.in/reader036/viewer/2022062517/56813b1d550346895da3cfed/html5/thumbnails/19.jpg)
Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore
Quality ChecksQuality Checks
Before fieldwork Pilot exercise10-15 interviewsCheck for
LengthComprehensionFlowSensitiveness
![Page 20: Citizen Report Card Methodology](https://reader036.vdocument.in/reader036/viewer/2022062517/56813b1d550346895da3cfed/html5/thumbnails/20.jpg)
Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore
Quality ChecksQuality Checks
During fieldwork
Accompanied interviews
Sample of each interviewer’s work spot checked by field supervisor
10% back checks by the field coordinator
100% scrutiny of schedules
![Page 21: Citizen Report Card Methodology](https://reader036.vdocument.in/reader036/viewer/2022062517/56813b1d550346895da3cfed/html5/thumbnails/21.jpg)
Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore
Quality ChecksQuality Checks
After fieldwork
Sample sizes
Quotas
Representation
![Page 22: Citizen Report Card Methodology](https://reader036.vdocument.in/reader036/viewer/2022062517/56813b1d550346895da3cfed/html5/thumbnails/22.jpg)
Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore
Cost Heads in FieldworkCost Heads in Fieldwork
Cost is a function of Nature of projectType of respondentSelection criteriaSample size, sampling
Two broad types
OverheadExpenses
![Page 23: Citizen Report Card Methodology](https://reader036.vdocument.in/reader036/viewer/2022062517/56813b1d550346895da3cfed/html5/thumbnails/23.jpg)
Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore
Cost Heads in FieldworkCost Heads in Fieldwork
OverheadsTime cost, personnel
Communication: telephone, computers
Courier / mailings
![Page 24: Citizen Report Card Methodology](https://reader036.vdocument.in/reader036/viewer/2022062517/56813b1d550346895da3cfed/html5/thumbnails/24.jpg)
Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore
Cost Heads in FieldworkCost Heads in FieldworkExpenses
Printing of questionnaires Briefing and piloting Interviewer fees Supervisor fees Local conveyance Outstation travel : Field officer, interviewers Others: Venue, equipment, moderation,
gifts, transcriptions Translation
![Page 25: Citizen Report Card Methodology](https://reader036.vdocument.in/reader036/viewer/2022062517/56813b1d550346895da3cfed/html5/thumbnails/25.jpg)
Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore
The Research Journey...The Research Journey...
Defining Research Objectives
Development of Instrument
Data collection
Analysis
Interpretation
Sampling
![Page 26: Citizen Report Card Methodology](https://reader036.vdocument.in/reader036/viewer/2022062517/56813b1d550346895da3cfed/html5/thumbnails/26.jpg)
Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore
Data AnalysisData Analysis
Preliminary analysis by research agencyFurther analysis by PAC
![Page 27: Citizen Report Card Methodology](https://reader036.vdocument.in/reader036/viewer/2022062517/56813b1d550346895da3cfed/html5/thumbnails/27.jpg)
Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore
Framework of AnalysisFramework of Analysis
Select computer package - Foxpro, SPSS
Run basic demographics
Check if this is in line with population
If yes, proceed with further analysis
If no, weight data to represent population
![Page 28: Citizen Report Card Methodology](https://reader036.vdocument.in/reader036/viewer/2022062517/56813b1d550346895da3cfed/html5/thumbnails/28.jpg)
Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore
Framework of AnalysisFramework of AnalysisWhat is the depth of analysis required?What are the relevant analysis heads?
By demographics By agency By responses to other questions :
• satisfied vs. dissatisfied
• had problem vs. did not have problem
• paid bribe vs. did not pay bribeSimple measures used- percentages and mean
scores. Recently regression models
![Page 29: Citizen Report Card Methodology](https://reader036.vdocument.in/reader036/viewer/2022062517/56813b1d550346895da3cfed/html5/thumbnails/29.jpg)
Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore
The Research Journey...The Research Journey...
Defining Research Objectives
Development of Instrument
Data collection
Analysis
Interpretation
Sampling
![Page 30: Citizen Report Card Methodology](https://reader036.vdocument.in/reader036/viewer/2022062517/56813b1d550346895da3cfed/html5/thumbnails/30.jpg)
Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore
Compare across agencies - rank them on satisfaction, responsiveness and corruption- using percentages and means
Within agencies - specific information - timings of water supply, quality of water - again using percentages and means
Two ways of interpreting same information - eg - number of visits to agency
Interpretation of DataInterpretation of Data
![Page 31: Citizen Report Card Methodology](https://reader036.vdocument.in/reader036/viewer/2022062517/56813b1d550346895da3cfed/html5/thumbnails/31.jpg)
Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore
Interpretation of DataInterpretation of Data
Assimilation of scattered findings into cohesive whole
Translation of findings into diagnostic statements
Conversion of diagnosis into conclusions
Formulation of recommendations based on conclusions
![Page 32: Citizen Report Card Methodology](https://reader036.vdocument.in/reader036/viewer/2022062517/56813b1d550346895da3cfed/html5/thumbnails/32.jpg)
Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore
From Findings to From Findings to RecommendationsRecommendations
Satisfaction with behaviour of staffObjective statement of finding -
percentage satisfied, dissatisfied, ambivalent.
Conclusion - general / high/low satisfaction/dissatisfaction
Recommendation - training for staff to improve their behaviour