![Page 1: Proposed student questionnaire for the main survey](https://reader034.vdocument.in/reader034/viewer/2022051419/56815884550346895dc5e680/html5/thumbnails/1.jpg)
Proposed student questionnaire for the main survey
National Research Coordinators Meeting Windsor, June 2008
![Page 2: Proposed student questionnaire for the main survey](https://reader034.vdocument.in/reader034/viewer/2022051419/56815884550346895dc5e680/html5/thumbnails/2.jpg)
NRCMeetingWindsor
June 2008
Contents of presentation
• Criteria used for selection of questionnaire material
• Overview of field trial questions and recommendations
• Length of proposed instrument
• Proposed changes
![Page 3: Proposed student questionnaire for the main survey](https://reader034.vdocument.in/reader034/viewer/2022051419/56815884550346895dc5e680/html5/thumbnails/3.jpg)
NRCMeetingWindsor
June 2008
Criteria for selection
• Field trial analysis results– Scaling properties (for scaled items)– Item dimensionality– Face validity– Review of relationship between related
variables/constructs
• Coverage of assessment framework• Feasibility of adaptations to national
contexts
![Page 4: Proposed student questionnaire for the main survey](https://reader034.vdocument.in/reader034/viewer/2022051419/56815884550346895dc5e680/html5/thumbnails/4.jpg)
NRCMeetingWindsor
June 2008
General proposals
• Only one (combined) student questionnaire (with both background and perceptions questions)
• Only one form
• No “don’t know” categories
![Page 5: Proposed student questionnaire for the main survey](https://reader034.vdocument.in/reader034/viewer/2022051419/56815884550346895dc5e680/html5/thumbnails/5.jpg)
NRCMeetingWindsor
June 2008
Review of field trial instruments
• Three categories of questions– To be retained without changes (in green)– To be modified (in yellow)– To be deleted (in red)
• Some questions are flagged as having “medium priority”
• Recent advice from the PAC meeting 7-8 June appears in orange
![Page 6: Proposed student questionnaire for the main survey](https://reader034.vdocument.in/reader034/viewer/2022051419/56815884550346895dc5e680/html5/thumbnails/6.jpg)
NRCMeetingWindsor
June 2008
Student characteristics
• Age (A01)
• Gender (A02)
• Ethnicity (optional) (A03)
• Expected educational level (A04)– without “don’t know” category
![Page 7: Proposed student questionnaire for the main survey](https://reader034.vdocument.in/reader034/viewer/2022051419/56815884550346895dc5e680/html5/thumbnails/7.jpg)
NRCMeetingWindsor
June 2008
Home environment
• Household composition (B01)– Medium priority only
• Country of birth (B02)
• Language use at home (B03)
![Page 8: Proposed student questionnaire for the main survey](https://reader034.vdocument.in/reader034/viewer/2022051419/56815884550346895dc5e680/html5/thumbnails/8.jpg)
NRCMeetingWindsor
June 2008
Parent background
• Parental occupation (B04, B06)– Good measure of socio-economic background– Somewhat larger percentages of missing values
in some countries
• Parental education (B05, B07)– Without “don’t know” category– Still considerable percentages of missing values
• Parental interest in political/social issues (B08)– Interesting results!
![Page 9: Proposed student questionnaire for the main survey](https://reader034.vdocument.in/reader034/viewer/2022051419/56815884550346895dc5e680/html5/thumbnails/9.jpg)
NRCMeetingWindsor
June 2008
Household possessions
• Household items (B09)– low reliabilities (especially in more
developed countries)– low correlations with other socio-
economic indicators)– PAC advice: Consider as an
international option
• Books in the home (B10)– Good indicator of socio-economic
background
![Page 10: Proposed student questionnaire for the main survey](https://reader034.vdocument.in/reader034/viewer/2022051419/56815884550346895dc5e680/html5/thumbnails/10.jpg)
NRCMeetingWindsor
June 2008
Measuring socio-economic background in the main survey
• It is proposed to have three indicators– Parental occupation (highest of both
parents)– Parental education (highest of both
parents)– Number of books at home
• Analysis will be undertaken to explore computation of composite index
![Page 11: Proposed student questionnaire for the main survey](https://reader034.vdocument.in/reader034/viewer/2022051419/56815884550346895dc5e680/html5/thumbnails/11.jpg)
NRCMeetingWindsor
June 2008
Student activities
• Out-of-school leisure time activities (C01)– Keep only items on reading and hanging out– Add: Homework and study time– PAC advice: Consider keeping also (at least)
computer and television use items• Activities out of school (C02)
– Discussion item form scale– Media information items do not scale but are of
interest as single items (wording change for TV news)
– Social participation items do not scale and should not be retained
– PAC advice: Consider combining items on organised social activities (boy scouts, youth clubs etc.)
![Page 12: Proposed student questionnaire for the main survey](https://reader034.vdocument.in/reader034/viewer/2022051419/56815884550346895dc5e680/html5/thumbnails/12.jpg)
NRCMeetingWindsor
June 2008
Students’ civic participation
• Civic participation in the community (C03)– Worked as scale and interesting associations
with other variables – PAC advice: An additional note to include
religious services was initially considered but should rather not be included
• Civic participation at school (C04)– Two items reflect different dimension (tutoring,
school newspaper) and are proposed to be deleted
– There were adaptation problems with the item on school protest, which is proposed as another deletion
– PAC suggestion: Consider same categories as for previous question
![Page 13: Proposed student questionnaire for the main survey](https://reader034.vdocument.in/reader034/viewer/2022051419/56815884550346895dc5e680/html5/thumbnails/13.jpg)
NRCMeetingWindsor
June 2008
School-related perceptions - 1
• Open climate for classroom discussion (D01)– Good scaling properties and interesting
correlations– PAC advice: Use rather same categories as in
CIVED• Student influence at school (D02)
– Good scaling characteristics but negative correlations with civic knowledge
– ISC sees this as question with medium priority– PAC advice: Additional item about
extracurricular activities
![Page 14: Proposed student questionnaire for the main survey](https://reader034.vdocument.in/reader034/viewer/2022051419/56815884550346895dc5e680/html5/thumbnails/14.jpg)
NRCMeetingWindsor
June 2008
School-related perceptions - 2
• School climate items (D03)– Good scaling properties for student-teacher
relation items (medium priority)– Students’ sense of belonging has not always
satisfactory scaling properties and should be deleted
• Students’ confidence in school participation (D04)– Negatively worded items did not load on the
same dimensions and should not be retained
![Page 15: Proposed student questionnaire for the main survey](https://reader034.vdocument.in/reader034/viewer/2022051419/56815884550346895dc5e680/html5/thumbnails/15.jpg)
NRCMeetingWindsor
June 2008
Perceptions of citizens and society - 1
• Democratic value beliefs (E01)– Agreement item format leads to highly skewed
responses– Alternative format proposed asked about
characteristics of “ideal society”– PAC advice: Keep original format
• Good citizenship beliefs (E02)– Two-dimensional structure confirmed– Proposed to delete three items that did not scale
well with either of the two scales– PAC advice: Consider keeping “obeying the
law” and “working hard” for single-item reporting
![Page 16: Proposed student questionnaire for the main survey](https://reader034.vdocument.in/reader034/viewer/2022051419/56815884550346895dc5e680/html5/thumbnails/16.jpg)
NRCMeetingWindsor
June 2008
Perceptions of citizens and society - 2
• General trust (E03)– Few interesting associations– Only item about trust in “people in
general” should be added to question about trust in institutions
• Acceptance of socially undesirable actions (E04)– Some problems with item scaling
properties and generally not too interesting results
![Page 17: Proposed student questionnaire for the main survey](https://reader034.vdocument.in/reader034/viewer/2022051419/56815884550346895dc5e680/html5/thumbnails/17.jpg)
NRCMeetingWindsor
June 2008
Self-perceptions - 1
• Interest in political and social issue (F01)– Generally good scaling properties– Five items to be retained and one modified
• Students’ self-concept (political internal efficacy) (F02)– Six items have good scaling properties– One (negatively phrased) item did not scale well
and should be deleted
![Page 18: Proposed student questionnaire for the main survey](https://reader034.vdocument.in/reader034/viewer/2022051419/56815884550346895dc5e680/html5/thumbnails/18.jpg)
NRCMeetingWindsor
June 2008
Self-perceptions - 2
• Feelings of distinctiveness (F03)– Doubts that items measure what is
intended
• Attachment to community levels (F04)– Results not very interesting– May be retained for regional instruments
![Page 19: Proposed student questionnaire for the main survey](https://reader034.vdocument.in/reader034/viewer/2022051419/56815884550346895dc5e680/html5/thumbnails/19.jpg)
NRCMeetingWindsor
June 2008
Rights and responsibilities - 1
• Gender rights items (G01)– One item did not scale well and should be
deleted
• Rights for ethnic/racial groups (G02)– Good scaling properties– Correct adaptation need to be ensured
(not minorities!)
![Page 20: Proposed student questionnaire for the main survey](https://reader034.vdocument.in/reader034/viewer/2022051419/56815884550346895dc5e680/html5/thumbnails/20.jpg)
NRCMeetingWindsor
June 2008
Rights and responsibilities - 2
• Rights for immigrants (G03)– Negatively phrased items reflect different
dimension and should not be retained– PAC suggestion: Keep item h (“When there
are not many [instead of enough]many [instead of enough] jobs available the number of <immigrants> should be limited“) for single-item reporting
• Reaction to threats to democracy (G04)– Items did not scale and only single-item reporting
might be considered– Proposed to delete this question– PAC advice: Include three items (b, d and f)
in democracy belief question (E01)
![Page 21: Proposed student questionnaire for the main survey](https://reader034.vdocument.in/reader034/viewer/2022051419/56815884550346895dc5e680/html5/thumbnails/21.jpg)
NRCMeetingWindsor
June 2008
Perceptions of institutions and society - 1
• Trust in groups and institutions (H01)– Good scaling properties for CIVED items and
interest in single-item reporting for not scaled items
– Addition of “people in general”– “Standardisation” of optional items– PAC suggestion: Consider asking about
media separately (Newspaper, TV, radio, internet) and ask about “people in general” first
• Attitudes toward country (H02)– Generally, good scaling properties except for
three items (two of them should be deleted)
![Page 22: Proposed student questionnaire for the main survey](https://reader034.vdocument.in/reader034/viewer/2022051419/56815884550346895dc5e680/html5/thumbnails/22.jpg)
NRCMeetingWindsor
June 2008
Perceptions of institutions and society - 2
• Government responsiveness (external political efficacy) (H03)– Analysis revealed two dimensions
(“responsiveness” and “cynicism”)– Both scales have weak reliabilities
• Satisfaction with country’s achievements (H04)– Generally, good scaling properties– Relatively high correlations with attitudes toward
country– Deletion proposed on content grounds– Mixed PAC advice
![Page 23: Proposed student questionnaire for the main survey](https://reader034.vdocument.in/reader034/viewer/2022051419/56815884550346895dc5e680/html5/thumbnails/23.jpg)
NRCMeetingWindsor
June 2008
Perceptions of institutions and society - 3
• Support for political parties (H05)– Interesting results– Sufficient proportions in each category
![Page 24: Proposed student questionnaire for the main survey](https://reader034.vdocument.in/reader034/viewer/2022051419/56815884550346895dc5e680/html5/thumbnails/24.jpg)
NRCMeetingWindsor
June 2008
Perceptions of participation in society - 1
• Citizenship self-efficacy (I01)– Good scaling properties for all but one
item (“raising money for people in need”)
• Protest activities (I02)– Two dimensional-structure (legal and
illegal protest)– Shortening of legal protest scale (two
items proposed for deletion)
![Page 25: Proposed student questionnaire for the main survey](https://reader034.vdocument.in/reader034/viewer/2022051419/56815884550346895dc5e680/html5/thumbnails/25.jpg)
NRCMeetingWindsor
June 2008
Perceptions of participation in society - 2
• Participation as an adult (I03)– Two dimensions (expected electoral
participation and expected active political participation)
– Good scaling properties for all items
• Participation as a young person (I04)– First two items reflect different dimension
(community-based participation)– Good scaling properties for item
measuring informal participation
![Page 26: Proposed student questionnaire for the main survey](https://reader034.vdocument.in/reader034/viewer/2022051419/56815884550346895dc5e680/html5/thumbnails/26.jpg)
NRCMeetingWindsor
June 2008
International option regarding religion
• Religious denomination (J01)
• Participation in religious services (J02)
• Attitudes toward religion (J03)– Good scaling properties for five items– Three items do not really reflect attitudes
and should be deleted
![Page 27: Proposed student questionnaire for the main survey](https://reader034.vdocument.in/reader034/viewer/2022051419/56815884550346895dc5e680/html5/thumbnails/27.jpg)
NRCMeetingWindsor
June 2008
Proposed student questionnaire for main survey
• With 5 items per minute the estimated length is 34 minutes
• There are about 170 to 180 items in the instrument
• The CIVED questionnaire had 162 items in the perceptions part (for 30 minutes)
![Page 28: Proposed student questionnaire for the main survey](https://reader034.vdocument.in/reader034/viewer/2022051419/56815884550346895dc5e680/html5/thumbnails/28.jpg)
NRCMeetingWindsor
June 2008
Proposed changes
• Reduction in questionnaire length of about 10 minutes
• Generally, only slight modifications or deletions are proposed
• The only more fundamental change for democratic value beliefs a format change has been dropped following PAC advice– Risk of using an un-trialled question format in
main survey!
![Page 29: Proposed student questionnaire for the main survey](https://reader034.vdocument.in/reader034/viewer/2022051419/56815884550346895dc5e680/html5/thumbnails/29.jpg)
NRCMeetingWindsor
June 2008
Questions or comments?
![Page 30: Proposed student questionnaire for the main survey](https://reader034.vdocument.in/reader034/viewer/2022051419/56815884550346895dc5e680/html5/thumbnails/30.jpg)
NRCMeetingWindsor
June 2008
Group discussions
• No complete re-writing of questions possible• No additional questions
– unless previously used in surveys of the same age group...
• Go through the original field trial material (tables in document 6) and discuss– Agreement with proposed selection– Prioritisation of questions – Feedback on proposed – Suggestions of (minor) changes to wording
![Page 31: Proposed student questionnaire for the main survey](https://reader034.vdocument.in/reader034/viewer/2022051419/56815884550346895dc5e680/html5/thumbnails/31.jpg)
NRCMeetingWindsor
June 2008
Group allocations
• From six groups
• Three groups start from the beginning of questionnaire material (those sitting in front)
• Three groups start from the end (those sitting towards the back)
• Try to form mixed groups (from different countries and regions)