teacher survey design national research coordinators meeting amsterdam, 16 - 19 october 2006
DESCRIPTION
NRC Meeting Amsterdam Oct 2006 CIVED teacher survey Population: Teachers teaching in civic- related subjects Identification of “civic subjects” using cognitive test content One to three teachers per student Student-teacher link Use of student weights for reportingTRANSCRIPT
Teacher Survey Design
National Research Coordinators Meeting Amsterdam, 16 - 19 October 2006
NRCMeeting
Amsterdam16-19 Oct 2006
Issues to be discussed• Purpose of teacher survey• Target population and exclusions• Sampling and administration
procedures• Sample sizes• Response rate requirements
NRCMeeting
Amsterdam16-19 Oct 2006
CIVED teacher survey• Population: Teachers teaching in civic-
related subjects• Identification of “civic subjects” using
cognitive test content• One to three teachers per student• Student-teacher link• Use of student weights for reporting
NRCMeeting
Amsterdam16-19 Oct 2006
New direction for ICCES• Focus on “whole school approach” in
C&C education and school climate• Inclusion of teacher from all subjects• Possibility of reporting results from
teacher survey (with teacher weights)• Greater emphasis on development of
teacher instrument
NRCMeeting
Amsterdam16-19 Oct 2006
Uses of teacher data• Reporting at country level: General
context of C&C education• Reporting of school level aggregates:
School implementation of C&C education
• Reporting of school level aggregates in student level or multi-level analyses
NRCMeeting
Amsterdam16-19 Oct 2006
Target population• Four potential options
– Teachers teaching any subject in the international target grade 8.
– Teachers teaching any subject in the international target grade 8 plus one adjacent grade.
– Teachers teaching any subject in lower secondary study programmes.
– Teachers teaching in any subject and any grade at the sampled school.
NRCMeeting
Amsterdam16-19 Oct 2006
Issues regarding target population
• Comparability of populations– Best for teachers only teaching in grade 8– Worst for ALL teachers at school
• Achieving representative sample of teaching staff at school– Grade 8 restricted to one particular grade– Lower Secondary teacher sample appropriate– Sample of ALL teachers may not reflect those
influencing tested students (combined schools with different ISCED levels)
NRCMeeting
Amsterdam16-19 Oct 2006
Possible teacher exclusion criteria
• Minimum teaching load?• Status of teachers (specific subjects,
casual staff)• Temporary absence (replacement
teachers instead?)
NRCMeeting
Amsterdam16-19 Oct 2006
Sampling and Administration• PPS sampling at school level (already
done)• List of teachers in target population
provided by school coordinator• Random selection from list• Administration of teacher
questionnaire through school coordinator
NRCMeeting
Amsterdam16-19 Oct 2006
Teacher sample sizes• Smaller sample sizes (< 5) only viable
when aim is to report teacher sample at country level only
• School level aggregates should be based on at least 10 teachers per school
• Non-response needs to be taken into account when determining sample size
NRCMeeting
Amsterdam16-19 Oct 2006
Response rate requirements• The minimum response rate for IEA
studies is 85 percent• For CIVED, the response rates ranged
between 70 and 100 percent• Standards for reporting will have to be
developed• Further discussions with the IEA TEG
about reporting of data from countries with response rates below 85 percent.
NRCMeeting
Amsterdam16-19 Oct 2006
Conclusions• Preference for target population
definition that maximises comparability (grade 8 teachers)
• Teacher exclusion criteria to be established with NRCs
• Tentative within-school sample size set at 15 (to ensure an achieved size of 10 per school)
NRCMeeting
Amsterdam16-19 Oct 2006
Questions or comments?