ATTITUDINAL SURVEYS AND SELF EVALUATION

JUNE 2011 London Conference

Geoff Davies

What are schools about?

• Effective Learning• Opening windows on a world• Choices• Realising potential• Achievement and Standards• Qualifications

BUT DO YOU RECOGNISE THIS???

You arrive at school one morning and you receive the following information

•Two pupils have misbehaved on the bus upsetting a third child

•A parent has phoned saying her child is being bullied in school and refuses to come to school

•A pupil has told her LSA that her mother has slapped her hard across the face the previous evening and there are finger marks on her face

•Two girls have been fighting in the canteen before school

•A member of staff is very upset after a disagreement with a colleague

•The gym roof is leaking and you have examinations scheduled there for the day

•You receive a serious complaint about a teacher’s conduct

•You are due to attend a meeting in an hour with the Chair of Governors

What are schools about?

• Preventive Detention• For some pupils just keeping them off

the streets is a major achievement

• Social function• Nobody else may take on this function

• Providing security and a safe environment

• Pupils not at ease will not learn

• Effective Learning• Opening windows on a world• Choices• Realising potential• Achievement and Standards• Qualifications

Schools are complex organisations

Far too complex to make decisions by hunch!

Can you really know your school without detailed research and surveys?

SELF EVALUATION

What do you do already?Why do you do it?What might you do to develop your practice?

Think about

Can the CEM centre help?

OFSTED REPORT JULY 2006

Best practice in self-evaluation

A survey of schools, colleges and local authorities

Best practice

Leaders gave priority to and led self-evaluation personally.

They had a clear overview of their institutions, based on an accurate understanding of strengths and weaknesses

Self-evaluation was integral to the culture of the organisations. People at all levels were committed to it and fully involved. It was a continuous process, governed by the needs of the institution rather than the requirements of external bodies. It was clearly built into management systems.

External inspection supported but did not replace internal review. It provided, however, the main external source of validation. An increasingly sophisticated use of a widening range of performance indicators enhanced the quality of self-evaluation. Rigorous analysis of strengths and weaknesses, particularly of teaching and learning, led to the clear identification of priorities and strategies for improvement.

Sharply focused monitoring, based on clear indicators, helped institutions to measure the extent to which their work improved outcomes for pupils and young people.

The views of those who received services, particularly learners but also parents and carers, were actively sought and influenced decision making.

WIDE RANGING PUPIL ANALYSIS

PROMOTING ACTION RESEARCH

EFFECTIVE QA SYSTEMS

PERCEPTION SURVEYS

DEVELOPING LEADERSHIP AND MANAGEMENT SKILLS

EFFECTIVELY TARGETING AND MEASURING SUCCESS

COLLABORATIVE REVIEWS

BENCHMARKING TO HELP JUDGEMENTS

SCHOOL SELF EVALUATION

Schools put the process of self evaluation at the heart of their culture

It is better to identify a problem yourself than have others do it for you

Carol Fitz-Gibbon 2001 British Psychological Society

It gradually dawned on me that providing the data to schools was the most important outcome of the effort, far more important than writing research papers…..

The provision of data to practitioners meant that they participated in the research. Indeed they were the only ones who knew the surrounding circumstances for their classrooms, their department, each pupil, each family, etc. They were the major players: the ones who could interpret and learn from the detailed data.

25 YEARS OF CEM CENTRE DATA

I keep six honest serving men

Who taught me all I know

Their names are What and Why and When

And How and Where and Who

Rudyard Kipling The Elephant’s Child

Can school leaders find the time?

Important but not urgent

Urgent but not important

Is it more like? 65%-80% or 15%

Is it more like? 15% or 60%

PRIORITISING

DOING IT ALL!

WHAT information do I want?WHY do I want it?WHEN do I need it?HOW do I collect it?WHERE can I find it?From WHO do I get it?

The CEM centre ticks many of these boxes for schools

Paper based surveys:

•Alis and Yellis have their ‘Extended Questionnaires’

•Yellis probes attitudes to school and student perceptions of the school and also their subjects

• Alis focuses extensively on their subjects

•A wealth of historical data

On-line attitudinal surveys

• MidYIS, SOSCA, INSIGHT Yellis and Alis

• Developed to meet new needs

• Can be more easily adapted to meet new developments

• The Parental Questionnaire

• The Event mapper

Why use our surveys when you can write your own?

• Questions re-analysed for reliability and validity

• No data to process

• ‘background’ of other schools enables you to see differences ,if they are significant and monitors attitudes over time

Yellis Attitudinal Feedback

Attitudinal Comparison Graphs

The Yellis Baseline questionnaire looks briefly at students' attitudes towards education and their aspirations for the near future. Feedback from this questionnaire takes the form of five graphs that look at students' job aspirations, cultural capital, likelihood of staying in education and feelings about their current school. 

Basic Comparison Graphs2010.pdf. 

Some examples

Basic Comparison by Gender Graphs.pdf

Attitudinal Trend Graphs These 35 graphs illustrate how your student's attitudes change year-by-year - based on the responses given during the Yellis Baseline Test session.

Year 10 Basic Comparison Trends.pdf

Further Attitudinal Comparison Graphs Feedback from this questionnaire takes the form of 23 graphs that illustrate how the responses of your students to the Attitudinal Questionaire differ when compared to all Yellis students in the same cohort.

.  Year 11 Extended Comparison Graphs.pdf

The following list gives the titles of Comparison Graphs currently provided by the Extended Yellis questionnaire:

•Alienation Indicators•Attitudes to Design & Technology•Attitudes to English•Attitudes to a Foreign Language•Attitudes to Mathematics •Attitudes to Science•Career-Relevant Activities Experienced•Sources of Careers Information Found Useful•Preferences for Kinds of Work•Items in the Freedom from Free Scale•Design and Technology Homework•English Homework•Foreign Language Homework•Mathematics Homework•Science Homework•Career Choice Motivators•Parental Involvement: Mothers•Parental Involvement: Fathers•Influences on Staying-On•Traumatic Events Experienced by Pupils•Places Where Pupils Feel Unsafe•Work Place Preferences•Cigarettes, Alcohol and Drugs

Your data is above the Yellis average

Your data is below the Yellis average

Your data is about the same as the Yellis average

Your data is about the same as the Yellis average

Extended Comparison Graphs

These 197 graphs illustrate how your student's attitudes change year-by-year - based on the responses given on the Yellis Attitudinal Questionnaire

Further attitudinal trends

The Extended MidYIS Attitudinal Questionnaires will help you to answer these questions and meet the relevant requirements set out in the Framework for inspecting schools.Topics CoveredThree questionnaires are included in the registration fee; the General, Induction and Parental questionnaires.General QuestionnaireThe General questionnaire includes a series of questions relating specifically to the completion of your Self Evaluation Form (SEF) ie bullying, racism, sectarianism, healthy lifestyles.Other sections include:Attitudes to school and study: do your students enjoy and feel happy in school?Attitudes and motivation toward subjects: what do your students feel about the classes they take? How does their motivation vary with subject? Bullying in school: do your students ever feel threatened or bullied? If so, where? Extra-curricular participation: do your students participate in additional clubs, sports or school trips? Drug use: how prevalent are cigarettes, alcohol and illegal drugs amongst your students? Career aspirations: what expectations do your students have for their future after school?  Peer pressure: do your students feel pressure from their friends to behave in anti-social or anti-school ways? Socio-economic status: what are the educational levels and occupational status of the parents of your students? Cultural capital: what support for learning (books, visits to museums, libraries) is there in your students' homes?   We recommend that the general questionnaire is taken in term 2 or 3. The general questionnaire takes about 45 minutes to complete.

Enhance your feedback by providing your teacher groups! - If you want to enhance your subject data please provide us with your pupil teacher groups

Online Attitudinal Questionnaires

Healthy Lifestyles : I take part in PE and Games lessons.I take part regularly in a sporting or outdoor activity provided by the school either at lunchtime, after school or at weekends.I take part in playground games at lunchtimes or breaktimes.I make use of the sporting facilities at school.The school encourages me to be physically healthy.The school encourages everyone to take part in sports no matter how good or how bad they are.The sporting and recreational facilities at school are excellent.I am unlikely to smoke because the school has told me about the dangers of smoking.How many cigarettes per day can someone safely smoke ? I eat less sweets and crisps now because the school has told me about the health risks involved.I eat more healthily because the school has warned me not to eat too much fried food.I am more likely to drink responsibly because the school has told me about the dangers of drinking alcohol.How many measures of alcohol (glasses of wine, half pints of beer) can someone drink before it affects their driving ? How often do you see bullying in the corridors at the change of lessons ?

How often do you see bullying in the toilets ?How often do you see bullying in outdoor lessons ?How often do you see bullying at lunchtimes ?How often is there bullying via e-mail in school ?How often is there bullying by text messaging in school ?I understand the school’s policy on bullying.I know who to tell if I am bullied.The school would be able to stop me being bullied if it happened.I would tell someone in school if I saw someone else being bullied.There is a lot of bullying in school.

ALIS

The Extended ALIS Questionnaire, generally takes about an hour to complete, covers many areas - including attitudes to school (or college), courses, Teaching and Learning and career plans.Students are also asked to comment on their responses to some open-ended questions.The questionnaire can either be taken on paper (in Term 2 of Year 2) or online (in Term 2 of Year 1).

PEDAGOGY….ALIS surveys

How has this been used?

SUBJECT A 2008

Why the improvement?

SUBJECT A 2005

SUBJECT A 2007

SUBJECT A 2008

DO NOT GET TOO EXCITED!

We have made a comparison of perceived teaching methods as analysed by ALIS in 2004-5 with those in 2007-8. Some subject areas have appeared to change their methods radically. Others have not. Though the samples are small it is an interesting exercise to try to correlate it with the departments statistical process charts over that period. One would like to say that changes in the variety of teaching methods result in improvement but the evidence is a little tenuous so far.

SUBJECT

Exercises

exampapers

essays

Reading

discussions

notes dictated

own notes

handouts

Practical work

Using AV

IT

Researching

Presenting

helping students

original work 2005 to 2008

SIGNIFICANTCHANGE

ART YES L L M L LBIOLOGY NO L LBUSINESS YES M L M LCHEMISTRY NO LCOMPUTING YES L L M M L MDRAMA NO M L MFOOD YES L L M M M MPRODUCT DESIGN YES L L L M LENGLISH NO LFRENCH YES M M M L MGEOGRAPHY YES M M M MHISTORY NO L LICT YES L L L L M M MMATHS NO M M MMUSIC YES M M M M M M MPE YES M M M M M M MPHYSICS YES M M M M M M LRE NO L MWELSH YES M M M M M

YES 11NO 7MORE (M) 54 5 4 1 3 4 5 6 5 2 3 2 3 2 6 3LESS (L) 29 3 3 2 6 3 1 2 1 2 1 0 0 2 3 0

A USEFUL STAFF DEVELOPMENT EXERCISE

SOSCA Attitudes to Science Survey

The Science Assessment includes a survey of pupils' attitudes to science.  The feedback from the survey shows you each pupil's average score on each sub-section, on a scale of 1 to 5 from negative to positive attitudes, and their overall score.

  How your school's pupils scored on each sub-section is also shown against a background of all the pupils in the SOSCA sample. 

Lastly, pupils are asked to reply to three questions about science and their free-text comments are reported anonymously and in random order for your information.

Perception of Science

This area covers all aspects of science outside the classroom and in the wider world. A typical question is:

Science causes many environmental problems.

Relevance of Science to Self

This section explores pupils' perceptions of how science relates to them, in particular when studying science at school. A typical question is:

I need to do well in science to get the job I want

Enjoyment of Science

This area explores pupils' enjoyment of the science they are taught in school. A typical question is:

I look forward to doing experiments

Environmental Empathy

This section includes pertinent questions about waste and the environment to assess what the pupils have done recently with regard to actions generally considered good or bad for the environment. A typical question is:

My empty cans always get recycled.

.

Whole School Summary

THE MOST IMPORTANT WORD IN EDUCATION???

FINALLY

CULTURE

Changing attitudes and the culture of the school is a long game so you need reliable data over a period of time

 Take action, with meaningful consequences for success or failure.

CHALLENGE

ATTITUDINAL SURVEYS AND SELF EVALUATION

JUNE 2011 London Conference

Geoff Davies