2012

A guide to using PISA as a learning context

Programme for International Student Assessment (PISA) in Wales

Ref: CAD/GM/0208 WG15932ISBN 978 0 7504 7717 8 © Crown copyright June 2012

A guide to using PISA as a learning context

Audience Mainstream secondary school headteachers, school leadership teams, school managers and classroom teachers; local authorities and national bodies with an interest in education.

Overview This document provides a guide to using PISA and PISA-style questions as activities to help support and improve learning and teaching.

Action School leadership teams and classroom teachers are encouraged to required use this resource as part of everyday learning and teaching, to better prepare learners for these types of assessments and support a more expansive pedagogy.

Further Enquiries about this document should be directed to: information Curriculum Division Department for Education and Skills Welsh Government Cathays Park Cardiff CF10 3NQ

Tel: 029 2082 5751 e-mail: assessment@wales.gsi.gov.uk

Additional This document can be accessed from the Welsh Government’s copies website at www.wales.gov.uk/educationandskills

Related PISA Take the Test: Sample Questions from the OECD’s PISA documents Assessments (OECD, 2009)

A guide to using PISA as a learning context 1

ContentsIntroduction 2

Key definitions 4

Suggestions for teachers 8

Skills assessed by the tasks 10

Tasks 11

Appendix: PISA sample questions 33

2 A guide to using PISA as a learning context

What is PISA?

PISA is the world’s biggest international education survey, involving schools and students in over 60 countries. It looks at the way in which 15-year-olds can apply knowledge and skills in mathematical literacy, reading and scientific literacy. PISA was developed jointly by member countries of the OECD (Organisation for Economic Co-operation and Development). PISA has been running since 2000, Wales has participated since 2006.

What is involved?

Schools are selected by the international organisers of the study, 30 learners in the age group are then randomly selected to take part. At least 150 schools and 4,500 students from each of the 65 countries took part in PISA 2009.

The survey takes place in school at a convenient time for the school. The selected sample of 15-year-olds take pencil and paper tests which last around two hours. The tests include multiple choice and open-ended questions on mathematical literacy, scientific literacy and reading. There is also a questionnaire for learners to answer on their attitudes and values and experiences of learning, and a questionnaire to be completed by the school.

PISA assessments are undertaken every three years. In each round the main focus is on one domain, the other two domains are assessed but using a smaller number of questions with a sub-set of the full sample. In 2006 the main domain was scientific literacy, in 2009 it was reading and in 2012 it will be mathematical literacy.

Strict standards are applied to ensure equivalence in question translation and countries are invited to comment on the difficulty and cultural appropriateness of the questions. Schools can select which language test each individual student takes but there is no bilingual option and students cannot ‘mix and match’ taking for example reading questions in one language and mathematics and science in another.

Significance of PISA

PISA seeks to answer questions like:

• Are students well prepared for future challenges?

• Can they analyse, reason and communicate effectively?

• How well do they transfer ideas and solve problems?

• Do they have the capacity to continue learning throughout life?

Introduction

A guide to using PISA as a learning context 3

PISA provides internationally comparable evidence that can help drive improvement of learner outcomes and potential life-chances and support policy development for governments.

Wales’ 2009 PISA results were poor in terms of comparison to prior performance, to other UK nations and internationally. The Minister for Education and Skills responded to the evidence from PISA and other indicators of educational performance by setting out a detailed plan of action to bring about improved performance for learners and schools in Wales.

About this resource

This resource is intended to help demonstrate how PISA, and PISA-style questions, can support pedagogy and help improve learning and teaching.

The questions used in this booklet are drawn from released PISA materials published on the OECD website.

Learners who perform best are those who can apply, analyse, synthesise and evaluate in a variety of contexts. PISA assessments require learners to utilise these processes within mathematical, language and scientific contexts.

The PISA questions are rich contexts requiring a high degree of literacy to access and understand them. Many of the questions require learners to apply their problem-solving skills to new contexts. Learners therefore need to have well-developed skills, especially literacy, numeracy and problem solving, and the confidence to take on the challenge of questions that are outside their ‘comfort zone’.

The main objective in producing this resource, and of the accompanying workshops and development programme, is to ensure that there is an understanding in the educational community in Wales as to how PISA assessments work in terms of contexts, demand and structure and how they can be used to support improved learning and teaching. Using PISA and PISA-style materials as part of everyday learning and teaching will better prepare learners for these types of assessments and support a more expansive pedagogy for teachers, which develop deeper levels of questioning, problem solving and use of literacy and numeracy in cross-curricular contexts.

Exposing learners to more PISA-style assessment is not about ‘practising’ or ‘teaching to the test’ it is about checking if learners understand how to access information and apply skills and knowledge. It is about creating and using an environment where they feel safe to take risks, collaborate with others to solve problems and receive formative feedback to help them improve. It is fundamentally about improving our young people’s life skills.

4 A guide to using PISA as a learning context

Key definitions

Developing thinking enables learners to:

• gain a deeper understanding of concepts

• understand how they arrived at an idea and the strategy they develop to do it

• be more critical about evidence

• think flexibly

• make reasoned judgements and decisions rather than jumping to conclusions.

These qualities in thinking are needed both in school and in the wider world and clearly reflect what is assessed by PISA.

PISA focuses squarely on the ability of learners to use information from texts in ‘real life’ scenarios. As a result, questions are highly contextualised and therefore require learners to engage with the scenario and decide how best to solve the problem. This necessitates a high degree of skills and strategy to be shown to access and interpret the information, extract relevant information and use and apply knowledge and skills to creatively solve problems. The key crucial process which drives this is metacognition. If learners are not aware of how they have solved problems previously or what strategies and skills were useful in doing this, then an independent transfer and application of skills and ideas – the capability for which is measured by PISA and required for lifelong learning and the world of work – becomes impossible.

Reading

Reading is defined in PISA as understanding, using, reflecting on and engaging with written texts, in order to achieve one’s goals, to develop one’s knowledge and potential, and to participate in society.

Mathematical literacy

Mathematical literacy is defined in PISA as an individual’s capacity to identify and understand the role that mathematics plays in the world, to make well-founded judgements and to use and engage with mathematics in ways that meet the needs of that individual’s life as a constructive, concerned and reflective citizen.

Scientific literacy

Scientific literacy is defined in PISA as the capacity to identify questions, acquire new knowledge, explain scientific phenomena, and draw evidence-based conclusions about science-related issues.

A guide to using PISA as a learning context 5

Metacognition and effective learning

Metacognition is defined as ‘knowledge of one’s own cognitive processes’. It spans several areas including:

• making sense of the task

• knowledge of strategies and methods, and how and when to use them

• knowledge and understanding of thinking processes

• monitoring and evaluating learning from the success (or otherwise) of chosen strategies or methods

• making connections across contexts.

As metacognition has a critical role to play in successful learning it is important that it is demonstrated by both learners and teachers. There is much research which shows that learners who demonstrate a wide range of metacognitive skills perform better in examinations and complete work more efficiently. They are self-regulated learners who utilise the ‘right tool for the job’ and modify learning strategies and skills based on their awareness of effectiveness. Individuals with a high level of metacognitive knowledge and skill identify blocks to learning as early as possible and change ‘tools’ or strategies to ensure goal attainment. Learners who demonstrate metacognition are aware of their own strengths and weaknesses, the nature of the task at hand, and available ‘tools’ or skills. In short, developing metacognition as a central process in learning creates greater learner autonomy and independence in decision-making, more effective self-regulation and increased capability.

Learners need to be exposed to and use a broad repertoire of tools/strategies as part of everyday learning. Many examples of tools/strategies to support thinking skills, decision-making and problem solving are outlined in the web-based booklet How to develop thinking and assessment for learning in the classroom (www.ngfl-cymru.org.uk/dtaafl/). Teachers need to be mindful of promoting metacognition across the five areas highlighted above by guiding deconstruction of the task and processes, tools and strategies used throughout. Questions such as the following should be used routinely with learners so that they start to internalise the question prompts for themselves.

• What is this task about?

• How do you know?

• Where have you used a strategy like this before?

• How did you extract the information you needed?

• How will you identify and summarise the key ideas to answer this question?

6 A guide to using PISA as a learning context

Specific metacognitive tools which support stepwise deconstruction include:

• reflection triangles

• flow charts/stepping stones

• who-what-when-why-where-how

• reflection starters

• reflection mobiles.

These are best used with the ‘language of learning and thinking’ to develop a shared vocabulary to enable clear expression of their thinking process. Teaching metacognition is arguably the most difficult aspect of developing a learner’s thinking. It is, however, one of the key aspects to promoting deeper understanding and transfer of ideas and skills to all areas of learning.

A guide to using PISA as a learning context 7

Assessment-taking strategies for learners

Some strategies for learners to consider.

• Always read the information for each task carefully.

• Don’t be put off by the length of the text to read – break it down into sections to read and try to pick out the key points of what it is about before you jump to the questions which follow.

• Read all accompanying text first for each question to get an idea of what it is about, then read the questions you are required to answer. Look for clues in any graphs, tables and diagrams also.

• Reread the text and use strategies such as underlining, skimming, scanning, colour coding and other tools/strategies you have to identify relevant parts of the text.

• Give each question a try, even when you are not sure. Remember partial value is given for partially correct answers.

• Remember credit is given for how you arrive at your answers so try and clearly show your thinking processes. This is especially important in terms of showing your methods of working using number.

• Think about the types of questions asked, take clues from the question starters and think about sentence starters for your answers, e.g. interpretive, reflective and evaluative questions are questions that begin with:

Why . . . ?

Why do you think . . . ?

How do you know . . . ?

One or two word answers won’t be sufficient. Reasons, usually with reference to the task, are required; often the word ‘because’ is used in the response.

• Non-continuous texts such as graphs, charts, diagrams and tables provide supporting information for any text. Study the axes and determine the purpose of the text before answering the questions.

• For multiple-choice questions dealing with mathematical literacy, remember you will be expected to carry out a calculation to be able to choose an answer.

• For multiple-choice questions, develop a methodical process of elimination of the alternative answers. When the list is narrowed to the best possibilities, choose one; there is no extra penalty for wrong choices.

8 A guide to using PISA as a learning context

Included in this booklet are four sample tasks. It is strongly suggested that the tasks are used formatively with learners in a collaborative manner. Teachers may choose to begin a detailed discussion of each task, what it involves and the scoring criteria or they may ask learners to work collaboratively (pairs or groups of three would work best) to attempt each task as an activity (as opposed to individually attempting questions) and then discuss the processes used afterward. Tasks are best used as activities rather than simply as questions to be answered – text can be used as source material to be interrogated initially with the questions removed, if so desired, to emphasise the importance of engagement with the material and support development of strategies to deal with this.

The intention in using these tasks in the classroom as part of everyday learning and teaching is not to ‘practice the test’ but to expose learners to a variety of learning resources to broaden their learning experience. The long-term aim of education is to support the learner in developing a wide repertoire of sustainable strategies to access relevant information and solve problems in preparation for the adult world of work. This can only be achieved by supporting a more expansive pedagogy for teachers to develop deeper levels of questioning, problem solving and use of literacy and numeracy in cross-curricular contexts.

To aid teachers in planning use of the tasks each has been deconstructed to identify the most likely range of skills to be developed and to show relevant links to the curriculum. The appendix gives an overview of the deconstruction for each of the released PISA questions. A full deconstruction can be found at www.wales.gov.uk/educationandskills

Sample tasks

There are four sample tasks in this booklet which have been drawn from all three PISA test areas.

Task 1 – Mobile Phone Safety (Reading)

Task 2 – Exports (Mathematical literacy)

Task 3 – Greenhouse (Scientific literacy)

Task 4 – The Best Car (Mathematical literacy)

For each task, information about the task (including level of difficulty, national curriculum contexts and links, text types, question types), Skills framework links, suggested tools/strategies and approaches to using the tasks as activities, the task itself and PISA scoring information are included.

These are intended to provide background support for teachers in planning the use of these tasks as activities within the normal range of learning and teaching activities.

Suggestions for teachers

A guide to using PISA as a learning context 9

How to use this resource

• These sample tasks can be used when planning a unit of work on a topic. It is best to use them where they link appropriately to a curriculum context that you are exploring. Relevant curriculum contexts are outlined in the overview information for each task. However, be aware that they are inherently cross-curricular in nature and not limited to English/Welsh, mathematics and science only. They are best used to bring out elements of literacy (in a language sense) and mathematical and scientific literacy naturally within other subject contexts also. The pedagogy that you use contributes to the transferability of the skills and strategies developed across all areas of learning.

• To engage your learners, the sample tasks should be used as the basis for activities where appropriate in the curriculum context. It is recommended that teachers use a variety of tools and strategies which exemplify effective pedagogy and stimulate developing thinking and assessment for learning to explore the tasks (see the booklet How to develop thinking and assessment for learning in the classroom, www.ngfl-cymru.org.uk/dtaafl/ for further support).

• Tasks should not be graded or ‘scores’ attributed to questions. The intention is that learners should be involved in formative feedback and collaboration with others in order to discuss and improve ideas and methods. It will certainly be useful to share and discuss the scoring criteria with learners. The criteria are the same as those used by PISA markers to mark the actual assessment.

• Although learners will need to work with tasks in their entirety, it is suggested that a preliminary focus is made on engaging learners with strategies to extract

relevant information from different forms of text. Background text (including graphical and mathematical representations) can be used as ‘source’ material for

learners (without the subsequent questions initially) to engage and support learners in developing strategic approaches to developing higher-order

literacy skills.

• As metacognition is a vital element in effective transfer of skills and strategies for learning, it is essential that learners are supported and encouraged to deconstruct, verbalise and share the processes they go through in solving a problem. Suggested reflection tools are discussed in the previous section. More ideas can be found at www.ngfl-cymru.org.uk/dtaafl/

10 A guide to using PISA as a learning context

Skills assessed by the tasksPISA tasks cover a wide range of skills; different tasks will draw on different combinations.

Thinking skills Literacy and communication skills

Plan • Asking questions• Activating prior skills, knowledge and understanding• Gathering information• Determining the process/method and strategy• Determining success criteria

Develop• Generating and developing ideas • Valuing errors and unexpected outcomes • Entrepreneurial thinking • Thinking about cause and effect and making inferences • Thinking logically and seeking patterns • Considering evidence, information and ideas • Forming opinions and making decisions • Monitoring progress

Reflect • Reviewing outcomes and success criteria • Reviewing the process/method • Evaluate own learning and thinking • Linking and lateral thinking

Oracy• Developing information and ideas• Presenting information and ideas

Reading• Locating, selecting and using information using reading strategies• Responding to what has been read

Writing• Organising ideas and information• Writing accurately

Wider communication skills• Communicating ideas and emotions• Communicating information

Numeracy skills

Use mathematical information• Using numbers• Measuring• Gathering information

Calculate• Using the number system• Using a variety of methods

Interpret and present findings• Talking about and explaining work• Comparing data• Recording and interpreting data and presenting findings

A guide to using PISA as a learning context 11

TasksTask 1 – Mobile Phone SafetyInformation about the task

Question Level of difficulty

PISA identified skills

Text type Question types

Subject matter and cross-curricular links

Mobile Phone Safety

Medium (Q2)High (Q11) Medium (Q6) Medium (Q9)

InterpretReflect and evaluateReflect and evaluateInterpret

Short information text presented in point format.

Multiple choice open ended. Using points to integrate, interpret, reflect and evaluate information.

Pros and cons of phone safety. General knowledge cross-curricular links to language (English and Welsh), history, geography, PSE and science.

Skills assessed by the task

Thinking skills Literacy and communication skills

Plan• Activating prior skills, knowledge and understanding• Gathering information

Develop• Generating and developing ideas

Reflect• Evaluate own learning and thinking

Writing• Organising ideas and information• Writing accurately

Wider communication skills• Communicating information

Suggested tools/strategies to use question as an activity

• PMI diagram to explore and summarise points for and against evidence for ‘danger’ (P – positive, i.e. ‘for’, M – minus, i.e. ‘against’, I – interesting, i.e. neither for/against).

• Fishbone diagram to explore cause and effect of issues as a means of summarising and linking the two tables shown.

• Just-a-minute to summarise 1. The purpose of the two tables and 2. What conclusions can be drawn.

12 A guide to using PISA as a learning context

Task 1 – Mobile Phone Safety

A guide to using PISA as a learning context 13

14 A guide to using PISA as a learning context

13 | P a g e

A guide to using PISA as a learning context 15

Scoring information

Question 2

Question intent Integrate and interpret: Form a broad understanding.

Recognise the purpose of a section (a table) in an expository text.

Full credit: B. To suggest that debate about mobile phone safety is ongoing.

Question 11

Question intent Reflect and evaluate: Reflect on and evaluate the content of a text.

Recognise the relationship between a generalised statement external to the text and a pair of statements in a table.

Full credit: C. It supports the No argument but does not approve it.

Question 6

Question intent Reflect and evaluate: Reflect on and evaluate the content of a text.

Use prior knowledge to reflect on information presented in a text.

Full credit: Identifies a factor in modern lifestyles that could be related to fatigue, headaches, or loss of concentration. The explanation may be self-evident, or explicitly stated.

• Not getting enough sleep. If you don’t you will be fatigued.

• Being too busy. Makes you tired.

• Too much homework, that makes you tired and gives you headaches.

• Noise – that gives you headaches.

• Stress.

• Working late.

• Exams.

• The world is just too loud.

16 A guide to using PISA as a learning context

• People don’t take time to relax anymore.

• People don’t prioritise the things that matter, so they get grumpy and sick.

• Computers.

• Pollution.

• Watching too much TV.

• Drugs.

• Microwave ovens.

• Too much emailing.

No credit: Gives an insufficient or vague response.

• Fatigue (Repeats information in the text).

• Tiredness (Repeats information in the text).

• Loss of concentration (Repeats information in the text).

• Headaches (Repeats information in the text).

• Lifestyle (Vague).

Shows inaccurate comprehension of the material or gives an implausible or irrelevant response.

• Sore ears.

• Egg cups.

Question 9

Question intent Integrate and interpret: Develop and interpretation.

Recognise an assumption in part of an expository text.

Full credit: C. There may or may not be danger involved in using mobile phones, but it is worth taking precautions.

A guide to using PISA as a learning context 17

Task 2 – ExportsInformation about the task

Question Level of difficulty

National curriculum mathematicalcontext

Text type Question types

Subject matter and cross-curricular links

Exports Low (Q13.1) Medium (Q13.2)

Handling data, interpret and analyse data.

Short continuous text with pie chart and bar chart to interpret.

Graphical interpretation and calculation. Short answer calculation and multiple choice.

Graphical representation involving exports per year.

Links to mathematics, geography, PSE

Skills assessed by the task

Thinking skills Literacy and communication skills

Plan • Activating prior skills, knowledge and understanding• Gathering information• Determining the process/method and strategy

Develop• Thinking logically and seeking patterns

Reading• Locating, selecting and using information using reading strategies• Responding to what has been read

Wider communication skills• Communicating information

Numeracy skills

Use mathematical information• Using numbers• Measuring• Gathering information

Calculate• Using the number system• Using a variety of methods

Interpret and present findings• Comparing data• Recording and interpreting data and presenting findings

18 A guide to using PISA as a learning context

Suggested tools/strategies to use question as an activity

• Graphs can be used in a source square and learners extract as much information as possible from the graphs before being given the questions.

• Most likely/What is the question? – learners are presented with values of exports calculated from the pie chart and total export graph (either they can generate these or they can be given them by the teacher) and their challenge is to justify which type of export does it describes and the year it was exported.

A guide to using PISA as a learning context 19

Task 2 – Exports

20 A guide to using PISA as a learning context

Scoring information

Question 13.1

Question intentHandling data: analysing and interpreting data in graphical form.

Full credit: 27.1 million zeds or 27 100 000 zeds or 27.1 (unit not required).

No credit:Other responses and missing.

Question 13.2

Question intentHandling data: analysing, interpreting and extracting data in graphical form and calculating using percentages.

Full credit: E. 3.8 million zeds.

No credit:Other responses and missing.

A guide to using PISA as a learning context 21

Task 3 – GreenhouseInformation about the task

Thinking skills Literacy and communication skills

Plan • Activating prior skills, knowledge and understanding• Gathering information

Develop• Generating and developing ideas • Thinking about cause and effect and making inferences • Thinking logically and seeking patterns • Considering evidence, information and ideas • Forming opinions and making decisions

Reflect • Evaluate own learning and thinking • Linking and lateral thinking

Reading• Locating, selecting and using information using reading strategies• Responding to what has been read

Writing• Organising ideas and information• Writing accurately

Wider communication skills • Communicating information

Numeracy skills

Use mathematical information• Gathering information

Interpret and present findings• Talking about and explaining work• Comparing data• Recording and interpreting data and presenting findings

Skills assessed by the task

Question Level of difficulty

National curriculum mathematicalcontext

Text type Question types

Subject matter and cross-curricular links

Greenhouse Medium (Q5.1) High (Q5.2) High (Q5.3)

Greenhouse effect – considering evidence.

Longer length continuous text incorporating line graph to interpret.

Interpret graph trends, draw comparisons, short written responses required to justify ideasIdentification of anomaly in data.

Text-based information and graphical data looking at evidence of global warming.

Links to science, geography, PSE, English/Welsh, mathematics.

22 A guide to using PISA as a learning context

Suggested tools/strategies to use question as an activity

• Graphs can be used in a source square and learners extract as much information as possible from the graphs to draw a conclusion which links both graphs. Then compare against Andre’s conclusion and discuss extent of agreement and evidence from the graphs (Question 5.1).

• Alternative outcome – using an aspect of thinking hats, learners develop graphical evidence which would produce the opposite conclusion reached by Andre. They must justify the processes used and how they developed their evidence.

• Without showing any of the task, give Andre’s conclusion and challenge learners to draw a graph/graphs which would support his conclusion. Compare with actual graphs given.

• ‘Golden sentence’ – Divide the task into sections (by paragraphs if preferred). Working in small groups, each group must present a ‘golden sentence’ summary of their given paragraph. Learners should be encouraged to describe how they developed their summary (e.g. use of keywords, skimming, scanning, etc.). They can then consider the effect of the overall conclusion drawn if certain pieces of evidence (certain ‘golden sentence summaries’) were missed out or purposely not included. This challenges learners’ ideas and helps develop strategies in pulling together vast quantities of information and making decisions regarding their relevance.

• From the ‘golden sentence’ summaries and/or source square, present questions to learners and ask them to use the information they have gathered to provide group answers to the questions. Deconstruct with learners afterwards which answers originated from the actual task/text/graphs (and how these were located) and which answers drew on previous scientific knowledge and understanding and how these are linked. Using think-pair-share, invite learners to show what clues in question stems would point towards using the information given and what requires them to draw on previous experience/knowledge.

A guide to using PISA as a learning context 23

Task 3 – Greenhouse

22 | P a g e

Task – Greenhouse

24 A guide to using PISA as a learning context

A guide to using PISA as a learning context 25

Scoring information

Question 5.1

Question intention Analysing and interpreting data to draw scientific conclusions consistent with evidence.

Full credit: • Responses that refer to the increase of both (average) temperature and carbon dioxide emission.

• As the emissions increased the temperature increased.

• Both graphs are increasing.

• Because in 1910 both the graphs began to increase.

• Temperature is rising as CO2 is emitted.

• The information lines on the graphs rise together.

• Everything is increasing.

• The more CO2 emission, the higher the temperature.

• Responses that refer (in general terms) to a positive relationship between temperature and carbon dioxide emission.

• The amount of CO2 and average temperature of the Earth is directly proportional.

• They have a similar shape indicating a relationship.

No credit: • Responses that refer to the increase of either the (average) temperature or the carbon dioxide emission.

• The temperature has gone up.

• CO2 is increasing.

• It shows the dramatic change in temperatures.

• Responses that refer to temperature and carbon dioxide emission without being clear about the nature of the relationship.

• The carbon dioxide emission (graph 1) has an effect on the earth’s rising temperature (graph 2).

• The carbon dioxide is the main cause of the increase in the earth’s temperature.

• Missing.

26 A guide to using PISA as a learning context

• The carbon dioxide emission is greatly rising more than the average Earth’s temperature. (Note: this answer is incorrect because the extent to which the CO2 emission and the temperature are rising is seen as the answer, rather than that they are both increasing).

• The rise of the CO2 over the years is due to the rise of the temperature of the Earth’s atmosphere.

• The way the graph goes up.

• There is a rise.

• Missing.

Question 5.2

Question intention Analysing and interpreting data to draw scientific conclusions consistent with evidence.

Full credit: Responses that refer to one particular part of the graphs in which the curves are not both descending or both climbing and gives the corresponding explanation, such as:

• In 1900–1910 (about) CO2 was increasing, whilst the temperature was going down.

• In 1980–1983 carbon dioxide went down and the temperature rose.

• The temperature in the 1800s is much the same but the first graph keeps climbing.

• Between 1950 and 1980 the temperature didn’t increase but the CO2 did.

• From 1940 until 1975 the temperature stays about the same but the carbon dioxide emission shows a sharp rise.

• In 1910 the temperature is a lot higher than in 1920 and they have similar carbon dioxide emissions.

Partial credit: • Responses that mention a correct period, without any explanation.

• Responses that mention only one particular year (not a period of time), with an acceptable explanation.

– In 1980 the emissions were down but the temperature still rose.

• Responses that give an example that doesn’t support Andre’s conclusion but makes a mistake in mentioning the period. (Note: there should be

A guide to using PISA as a learning context 27

evidence of this mistake – e.g. an area clearly illustrating a correct answer is marked on the graph and then a mistake made in transferring this information to the text.) – Between 1950 and 1960 the temperature decreased and the carbon dioxide emission increased.

• Responses that refer to differences between the two curves, without mentioning a specific period. – At some places the temperature rises even if the emission decreases. – Earlier there was little emission but nevertheless high temperature. – When there is a steady increase in graph 1, there isn’t an increase in graph 2. It stays constant. (Note: it stays constant “overall”.) – Because at the start the temperature is still high where the carbon dioxide was very low.

• Responses that refer to an irregularity in one of the graphs. – It is about 1910 when the temperature had dropped and went on for a certain period of time. – In the second graph there is a decrease in temperature of the Earth’s atmosphere just before 1910.

• Responses that indicate difference in the graphs, but explanation is poor. – In the 1940s the heat was very high but the carbon dioxide very low.

No credit: • Responses that refer to an irregularity in a curve without referring specifically to the two graphs. – It goes a little up and down. – It went down in 1930.

• Responses that refer to a poorly defined period or year without any explanation. – The middle part – 1910.

• Missing. – In 1940 the average temperature has increased, but not the carbon dioxide emission. – Around 1910 the temperature has increased but not the emission.

• Missing.

28 A guide to using PISA as a learning context

Question 5.3

Question intention Explaining phenomena scientifically.

Full credit: • Responses that give a factor referring to the energy/radiation coming from the Sun. – The sun heating and maybe the earth changing position. – Energy reflected back from the Earth. (Assuming that by ‘Earth’ the student means ‘the ground’).

• Responses that give a factor referring to a natural component or a potential pollutant. – Water vapour in the air. – Clouds. – The things such as volcanic eruptions. – Atmospheric pollution (gas, fuel). – The amount of exhaust gas. – CFCs. – The number of cars. – Ozone (as a component of air).

No credit: • Responses that refer to a cause that influences the carbon dioxide concentration. – Clearing of rain forest. – The amount of CO

2 being let off. – Fossil fuels.

• Responses that refer to a non-specific factor. – Fertilisers. – Sprays. – How the weather has been.

• Other incorrect factors or Missing. – The amount of oxygen.

– Nitrogen. – The hole in the ozone layer is also getting bigger.

• Missing.

A guide to using PISA as a learning context 29

Task 4 – The Best CarInformation about the task

Thinking skills Literacy and communication skills

Plan • Activating prior skills, knowledge and understanding• Gathering information• Determining the process/method and strategy

Develop• Generating and developing ideas • Thinking about cause and effect and making inferences • Thinking logically and seeking patterns

Reflect • Linking and lateral thinking

Reading• Locating, selecting and using information using reading strategies• Responding to what has been read

Wider communication skills• Communicating information

Numeracy skills

Use mathematical information• Using numbers• Gathering information

Calculate• Using the number system• Using a variety of methods

Interpret and present findings• Talking about and explaining work• Comparing data• Recording and interpreting data and presenting findings

Skills assessed by the task

Question Level of difficulty

National curriculum mathematicalcontext

Text type Question types

Subject matter and cross-curricular links

The Best Car Low (Q25.1)High (Q25.2)

Algebraic relationships and functions, patterns and relationships.

Short continuous text with tabulated data to interpret.

Closed short answer calculation and interpretation of data as formulae.

Looking at different criteria for cars to develop a conclusion.

Links to mathematics science, design and technology.

30 A guide to using PISA as a learning context

Suggested tools/strategies to use question as an activity

• The information table (minus the subsequent questions) can be used within a source square to allow learners to interrogate the data, extract information and make inferences on its use and meaning.

• A concept cartoon of ‘correct’ and ‘incorrect’ answers for both questions can be given and learners challenged to select the answer they most agree with and justify their decision. By providing ‘answers’ in this manner, learners have to deconstruct the processes used to arrive at the answer as part of their justification – this is a critical element of metacognition.

• Equations/formulae can be represented using a flowchart/stepping stones (see How to develop thinking and assessment for learning booklet for support). This allows learners to deconstruct calculations and make visual to peers their decision processes in developing a formula for Question 25.2.

• Learners can look at ‘what would happen if’ scenarios by using the ratings criteria as a whole and part activity. This would allow them to consider the purpose and construction of the formula used in Question 25.1 and to consider the effect each rating has (i.e. a form of ‘weighting’) to the overall outcome, as in Question 25.2.

A guide to using PISA as a learning context 31

Task 4 – The Best Car

32 A guide to using PISA as a learning context

Scoring information

Question 25.1

Question intention Interpretation and extraction of data, substitution into and use of given formula to make calculation.

Full credit: 15 points.

No credit: Other responses or missing.

Question 25.2

Question intention Extraction and use of data, development and correct representation of a formula to make calculation to a given criterion.

Full credit: Correct rule that will make ‘Ca’ the winner.

No credit: Other resonses or missing.

A guide to using PISA as a learning context 33

App

endi

x: P

ISA

sam

ple

ques

tion

s

Que

stio

nLe

vel o

f di

fficu

lty

PISA

id

enti

fied

skill

sTe

xt t

ype

Que

stio

n ty

pes

Subj

ect

mat

ter

and

cros

s-cu

rric

ular

link

s

1. L

ake

Chad

Low

(Q1.

1)M

ediu

m (Q

1.2)

High

(Q1.

3)Lo

w (Q

1.4)

Med

ium

(Q1.

5)

Retri

eve

info

Retri

eve

info

Refle

ct a

nd e

valu

ate

text

Inte

rpre

tIn

terp

ret

Shor

t con

tinuo

us in

form

atio

n te

xt w

ith b

ar a

nd li

ne g

raph

s.M

ultip

le c

hoice

.O

pen

ende

d sh

ort w

ritte

n re

spon

se.

Requ

ire in

fere

nce

and

calcu

latio

ns.

Geo

grap

hica

l inf

orm

atio

n.N

umer

acy

skill

s.Li

nks

with

geo

grap

hy,

num

erac

y, sc

ienc

e.

2. F

luLo

w (Q

2.1)

Med

ium

(Q2.

2)

Med

ium

(Q2.

3)Hi

gh (Q

2.4)

Med

ium

(Q2.

5)

Retri

eve

info

Re

flect

and

eva

luat

e te

xtIn

terp

ret

Refle

ct a

nd e

valu

ate

Inte

rpre

t

Med

ium

leng

th in

form

atio

n te

xt in

sec

tions

with

ill

ustra

tions

.

Mul

tiple

cho

ice.

Ope

n en

ded

long

er w

ritte

n re

spon

se.

Iden

tifyin

g an

d ex

plai

ning

fe

atur

es o

f te

xt/s

tyle

.

Med

ical p

ublic

ity m

ater

ial.

Link

s w

ith P

SE, s

cienc

e.

3. G

raffi

tiLo

w (Q

3.1)

Med

ium

(Q3.

2)Lo

w (Q

3.3)

Med

ium

(Q3.

4)

Inte

rpre

tIn

terp

ret

Refle

ct a

nd e

valu

ate

Refle

ct a

nd e

valu

ate

Med

ium

leng

th te

xt.

Two

lette

rs e

xpre

ssin

g co

ntra

stin

g po

ints

of v

iew

.

Mul

tiple

cho

ice.

Ope

n en

ded

shor

t writ

ten

resp

onse

.Ex

pres

sing

and

just

ifyin

g ow

n op

inio

ns.

For a

nd a

gain

st g

raffi

ti.Li

nks

with

art,

PSE

.

4. L

abou

rLo

w (Q

4.1)

High

(Q4.

2)Hi

gh (Q

4.3)

Low

(Q4.

4)Lo

w (Q

4.5)

Inte

rpre

tRe

triev

e in

foIn

terp

ret

Refle

ct a

nd e

valu

ate

Refle

ct a

nd e

valu

ate

Shor

t non

-con

tinuo

us

info

rmat

ion

text

in a

tree

di

agra

m in

cludi

ng fi

gure

s.

Mul

tiple

cho

ice.

Ope

n en

ded

fact

ual.

Cate

goris

ing.

Empl

oym

ent i

nfor

mat

ion

pres

ente

d in

figu

res

and

visu

ally.

Li

nks

with

num

erac

y an

d PS

E.

5. P

lan

Inte

rnat

iona

lHi

gh (Q

5.2)

Refle

ct a

nd e

valu

ate

Shor

t non

-con

tinuo

us

info

rmat

ion

text

pre

sent

ed in

ta

ble

form

at.

Mul

tiple

cho

ice.

Ope

n en

ded

shor

t writ

ten

resp

onse

.In

terp

retin

g da

ta e

xpre

ssin

g op

inio

n w

ith ju

stifi

catio

n.

Inte

rnat

iona

l aid

or

gani

satio

n an

d its

act

iviti

es

in a

sin

gle

year

.Li

nks

with

art,

geo

grap

hy,

num

erac

y.

Read

ing

34 A guide to using PISA as a learning context

Que

stio

nLe

vel o

f di

fficu

lty

PISA

id

enti

fied

skill

sTe

xt t

ype

Que

stio

n ty

pes

Subj

ect

mat

ter

and

cros

s-cu

rric

ular

link

s

6. P

olice

Med

ium

(Q6.

1)M

ediu

m (Q

6.2)

Low

(Q6.

3)Lo

w (Q

6.4)

Retri

eve

info

Inte

rpre

tIn

terp

ret

Inte

rpre

t

Long

info

rmat

ion

text

pr

esen

ted

in s

ectio

ns w

ith

head

ings

.

Mul

tiple

cho

ice re

latin

g to

fa

ctua

l inf

orm

atio

n, o

pini

on

and

style

of t

ext.

Artic

le o

n po

lice

inve

stig

atio

n m

etho

ds u

sing

DNA

evid

ence

. Li

nks

to s

cienc

e.

7. R

unne

rsLo

w (Q

7.1)

Low

(Q7.

2)Lo

w (Q

7.3)

Low

(Q7.

4)

Inte

rpre

tRe

triev

e in

foRe

triev

e in

foRe

flect

and

eva

luat

e

Med

ium

info

rmat

ion

text

in

sect

ions

with

hea

ding

s.M

ultip

le c

hoice

Ope

n en

ded

fact

ual r

equi

ring

shor

t and

long

er re

spon

ses

Artic

le a

bout

spo

rts s

hoes

(‘r

unne

rs’).

Li

nks

to P

E an

d sc

ienc

e.

8. G

iftLo

w (Q

8.1)

High

(Q8.

2)Hi

gh (Q

8.3)

Low

(Q8.

4)Hi

gh (Q

8.5)

M

ediu

m (Q

8.6)

Med

ium

(Q8.

7)

Inte

rpre

tIn

terp

ret

Refle

ct a

nd e

valu

ate

Retri

eve

info

Inte

rpre

tIn

terp

ret

Refle

ct a

nd e

valu

ate

Very

long

des

crip

tive

st

ory/

narra

tive.

Mul

tiple

cho

ice.

Ope

n en

ded

shor

t and

long

er

resp

onse

s.Us

ing

text

to s

uppo

rt an

swer

s an

d ju

stify

ow

n po

int o

f vie

w.

Stor

y ab

out a

chi

ldho

od

expe

rienc

e in

Am

erica

. Co

ntai

ns s

ome

less

fam

iliar

vo

cabu

lary

for l

earn

ers

in

Wal

es.

Link

s to

Eng

lish

and

Wel

sh.

9. A

man

da

and

the

Du

ches

s

Low

(Q9.

1)M

ediu

m (Q

9.2)

Med

ium

(Q9.

3)Hi

gh (Q

9.4)

Low

(Q9.

5)

Inte

rpre

tRe

flect

and

eva

luat

eIn

terp

ret

Retri

eve

info

Inte

rpre

t

Very

long

ext

ract

from

pl

ay s

crip

t inc

ludi

ng s

tage

di

rect

ions

and

sec

ond

text

w

ith d

efini

tions

of t

heat

rical

oc

cupa

tions

.

Mul

tiple

cho

ice.

Iden

tifyin

g fe

atur

es.

Cate

goris

ing.

Visu

al in

terp

reta

tion

follo

win

g st

age

dire

ctio

ns.

Play

abo

ut a

Prin

ce, a

Du

ches

s an

d sh

op

assis

tant

– e

xtra

ct fr

om a

pl

ay b

y Je

an A

noui

lh.

Lang

uage

ver

y lit

erar

y an

d so

me

unfa

mili

ar v

ocab

ular

y.

Link

s to

dra

ma,

Eng

lish

and

Wel

sh.

10. P

erso

nnel

Low

(Q10

.1)

High

(Q10

.2)

Retri

eve

info

Retri

eve

info

Med

ium

leng

th in

form

atio

n te

xt, b

roke

n in

to s

ectio

ns b

y qu

estio

n he

adin

gs.

Ope

n en

ded

shor

t and

long

er

resp

onse

s.W

ork-

rela

ted

cont

ext a

bout

su

ppor

t for

peo

ple

chan

ging

jo

bs.

Link

s w

ith P

SE, w

ork-

rela

ted

educ

atio

n.

A guide to using PISA as a learning context 35

11. N

ew R

ules

Med

ium

(Q11

.1)

High

(Q11

.2)

Inte

rpre

tIn

terp

ret

Med

ium

leng

th n

ewsp

aper

ed

itoria

l.Un

derli

ning

text

.Li

stin

g ex

ampl

es.

Ethi

cs re

late

d to

froz

en

embr

yos.

Li

nks

with

PSE

, scie

nce.

12. M

orel

and

Tria

l:Lo

w (Q

12.1

)Lo

w (Q

12.2

)

Retri

eve

info

Retri

eve

info

Shor

t not

ice w

ith

non-

cont

inuo

us te

xt –

mai

nly

times

and

day

s.

Iden

tifica

tion

of ti

mes

.Li

brar

y op

enin

g ho

urs.

Li

nks

with

num

erac

y.

13. W

arra

nty

Tria

l:M

ediu

m (Q

13.1

)Lo

w (Q

13.2

)Lo

w (Q

13.3

)M

ediu

m (Q

13.4

)

Retri

eve

info

Retri

eve

info

Retri

eve

info

Refle

ct a

nd e

valu

ate

Shor

t tex

t – re

ceip

t and

w

arra

nty

for a

pur

chas

e.Co

mpl

etio

n of

war

rant

y fo

rm

from

info

rmat

ion

in re

ceip

t.O

pen

ende

d sh

ort a

nd lo

nger

re

spon

ses.

Inclu

ding

opi

nion

and

ju

stifi

catio

n.

Cam

era

purc

hase

and

rela

ted

docu

men

tatio

n.

Link

s w

ith n

umer

acy

and

finan

cial e

duca

tion.

14. A

Just

Judg

eTr

ial:

High

(Q14

.1)

Med

ium

(Q14

.2)

High

(Q14

.3)

Med

ium

(Q14

.4)

High

(Q14

.5)

High

(Q14

.6)

Inte

rpre

tRe

triev

e in

foRe

flect

and

eva

luat

eIn

terp

ret

Refle

ct a

nd e

valu

ate

Refle

ct a

nd e

valu

ate

Very

long

trad

ition

al s

tory

.M

ultip

le c

hoice

.Ex

tend

ed re

spon

ses

givi

ng

opin

ions

with

reas

ons.

Com

paris

on o

f law

in s

tory

w

ith o

wn

lega

l sys

tem

.

Trad

ition

al A

lger

ian

stor

y ab

out c

rime

and

puni

shm

ent

and

a re

quire

men

t to

have

so

me

know

ledg

e of

ow

n la

ws.

Li

nks

with

PSE

, hist

ory,

RE.

15. I

n Po

or Ta

ste

Tria

l:Lo

w (Q

15.1

)M

ediu

m (Q

15.2

)

Inte

rpre

tRe

flect

and

eva

luat

eSh

ort l

ette

r to

new

spap

er.

Mul

tiple

cho

ice.

Ope

n en

ded

shor

t res

pons

e.W

este

rn c

onsu

mpt

ion

and

over

seas

aid

. Li

nks

with

PSE

and

ge

ogra

phy.

16. B

ullyi

ngTr

ial:

High

(Q16

.1)

High

(Q16

.2)

Inte

rpre

tRe

triev

e in

foM

ediu

m le

ngth

new

spap

er

artic

le.

Mul

tiple

cho

ice in

terp

reta

tion

of b

ar g

raph

s.Sc

hool

bul

lying

in Ja

pan.

Li

nks

with

PSE

.

36 A guide to using PISA as a learning context

Que

stio

nLe

vel o

f di

fficu

lty

PISA

id

enti

fied

skill

sTe

xt t

ype

Que

stio

n ty

pes

Subj

ect

mat

ter

and

cros

s-cu

rric

ular

link

s

17. B

ees

Tria

l:M

ediu

m (Q

17.1

)M

ediu

m (Q

17.2

)Hi

gh (Q

17.3

)M

ediu

m (Q

17.4

)

Inte

rpre

tRe

triev

e in

foIn

terp

ret

Inte

rpre

t

Med

ium

leng

th in

form

atio

n te

xt w

ith il

lust

ratio

ns/

diag

ram

.

Mul

tiple

cho

ice.

Ope

n en

ded

fact

ual

resp

onse

s.

Bee

keep

ing

and

mak

ing

hone

y.

Link

s w

ith s

cienc

e.

18. M

obile

P

hone

Safe

ty

Tria

l:M

ediu

m (Q

2)Hi

gh (Q

11)

Med

ium

(Q6)

Med

ium

(Q9)

Inte

rpre

tRe

flect

and

eva

luat

eRe

flect

and

eva

luat

eIn

terp

ret

Shor

t inf

orm

atio

n te

xt

pres

ente

d in

poi

nt fo

rmat

.M

ultip

le c

hoice

ope

n en

ded.

Us

ing

poin

ts to

inte

grat

e,

inte

rpre

t, re

flect

and

eva

luat

e in

form

atio

n.

Pros

and

con

s of

pho

ne

safe

ty.

Gen

eral

kno

wle

dge

cr

oss-

curri

cula

r lin

ks to

la

ngua

ge (E

nglis

h an

d W

elsh

), hi

stor

y, ge

ogra

phy,

PSE

and

scie

nce.

19.

Tele

com

mut

ing

Tria

l:Hi

gh (Q

1)Lo

w (Q

7)M

ediu

m (Q

4)

Inte

rpre

tRe

flect

and

eva

luat

eIn

terp

ret

Two

shor

t arg

umen

tativ

e te

xts.

Mul

tiple

cho

ice o

pen

ende

d.

Two

para

grap

hs to

inte

grat

e,

inte

rpre

t, re

flect

and

eva

luat

e pr

ior k

now

ledg

e.

For a

nd a

gain

st

tele

com

mut

ing.

Li

nks

with

lang

uage

(Eng

lish

and

Wel

sh),

wor

k-re

late

d ed

ucat

ion,

PSE

and

scie

nce.

20. T

he P

lay’s

the

Thin

gTr

ial:

Low

(Q3)

Med

ium

(Q4)

High

(Q6)

High

(Q7)

Retri

eve

info

Inte

rpre

t/inf

erIn

terp

ret

Inte

rpre

t

Med

ium

leng

th te

xt in

the

form

of i

nfor

mat

ion

and

dial

ogue

.

Mul

tiple

cho

ice o

pen

ende

d.

Acce

ss a

nd re

triev

e in

form

atio

n.

Inte

grat

e, in

terp

ret a

nd in

fer

refe

renc

es.

Actio

n ta

king

pla

ce b

efor

e th

e ev

ents

of t

he p

lay.

Li

nks

with

lang

uage

(Eng

lish

and

Wel

sh),

dram

a an

d hi

stor

y.

A guide to using PISA as a learning context 37

Que

stio

nLe

vel o

f di

fficu

lty

PISA

id

enti

fied

skill

sTe

xt t

ype

Que

stio

n ty

pes

Subj

ect

mat

ter

and

cros

s-cu

rric

ular

link

s

17. B

ees

Tria

l:M

ediu

m (Q

17.1

)M

ediu

m (Q

17.2

)Hi

gh (Q

17.3

)M

ediu

m (Q

17.4

)

Inte

rpre

tRe

triev

e in

foIn

terp

ret

Inte

rpre

t

Med

ium

leng

th in

form

atio

n te

xt w

ith il

lust

ratio

ns/

diag

ram

.

Mul

tiple

cho

ice.

Ope

n en

ded

fact

ual

resp

onse

s.

Bee

keep

ing

and

mak

ing

hone

y.

Link

s w

ith s

cienc

e.

18. M

obile

P

hone

Safe

ty

Tria

l:M

ediu

m (Q

2)Hi

gh (Q

11)

Med

ium

(Q6)

Med

ium

(Q9)

Inte

rpre

tRe

flect

and

eva

luat

eRe

flect

and

eva

luat

eIn

terp

ret

Shor

t inf

orm

atio

n te

xt

pres

ente

d in

poi

nt fo

rmat

.M

ultip

le c

hoice

ope

n en

ded.

Us

ing

poin

ts to

inte

grat

e,

inte

rpre

t, re

flect

and

eva

luat

e in

form

atio

n.

Pros

and

con

s of

pho

ne

safe

ty.

Gen

eral

kno

wle

dge

cr

oss-

curri

cula

r lin

ks to

la

ngua

ge (E

nglis

h an

d W

elsh

), hi

stor

y, ge

ogra

phy,

PSE

and

scie

nce.

19.

Tele

com

mut

ing

Tria

l:Hi

gh (Q

1)Lo

w (Q

7)M

ediu

m (Q

4)

Inte

rpre

tRe

flect

and

eva

luat

eIn

terp

ret

Two

shor

t arg

umen

tativ

e te

xts.

Mul

tiple

cho

ice o

pen

ende

d.

Two

para

grap

hs to

inte

grat

e,

inte

rpre

t, re

flect

and

eva

luat

e pr

ior k

now

ledg

e.

For a

nd a

gain

st

tele

com

mut

ing.

Li

nks

with

lang

uage

(Eng

lish

and

Wel

sh),

wor

k-re

late

d ed

ucat

ion,

PSE

and

scie

nce.

20. T

he P

lay’s

the

Thin

gTr

ial:

Low

(Q3)

Med

ium

(Q4)

High

(Q6)

High

(Q7)

Retri

eve

info

Inte

rpre

t/inf

erIn

terp

ret

Inte

rpre

t

Med

ium

leng

th te

xt in

the

form

of i

nfor

mat

ion

and

dial

ogue

.

Mul

tiple

cho

ice o

pen

ende

d.

Acce

ss a

nd re

triev

e in

form

atio

n.

Inte

grat

e, in

terp

ret a

nd in

fer

refe

renc

es.

Actio

n ta

king

pla

ce b

efor

e th

e ev

ents

of t

he p

lay.

Li

nks

with

lang

uage

(Eng

lish

and

Wel

sh),

dram

a an

d hi

stor

y.

Que

stio

nLe

vel o

f di

fficu

lty

Nat

iona

l cur

ricu

lum

m

athe

mat

ical

Con

text

Text

typ

eQ

uest

ion

type

sSu

bjec

t m

atte

r an

d

cros

s-cu

rric

ular

link

s

1. F

arm

sLo

w (Q

1.1)

Med

ium

(Q1.

2)Un

ders

tand

and

use

m

easu

res,

area

, sha

pe a

nd

unde

rsta

nd n

umbe

r and

no

tatio

n.

Shor

t con

tinuo

us te

xt

info

rmat

ion

with

pict

ures

an

d di

agra

ms

to in

terp

ret.

Calcu

latio

n.Us

ing

a di

agra

m to

look

at

area

s of

a ro

of.

Link

s to

mat

hem

atics

, des

ign

and

tech

nolo

gy.

2. W

alki

ngHi

gh (Q

2.1)

High

(Q2.

2)Al

gebr

aic

rela

tions

hips

, fu

nctio

ns, u

nder

stan

d nu

mbe

r and

not

atio

n,

calcu

late

in v

arie

ty o

f way

s.

Shor

t con

tinuo

us te

xt

info

rmat

ion

with

pict

ure

to

inte

rpre

t.

Calcu

latio

n w

ith m

etho

d to

ju

stify

ans

wer

.Us

ing

form

ulae

to c

alcu

late

pa

cele

ngth

from

a d

iagr

am.

Link

s to

mat

hem

atics

, sc

ienc

e.

3. A

pple

sM

ediu

m (Q

3.1)

High

(Q3.

2)Hi

gh (Q

3.4)

Alge

brai

c re

latio

nshi

ps,

func

tions

, pat

tern

s an

d re

latio

nshi

ps.

Shor

t con

tinuo

us te

xt

info

rmat

ion

with

dia

gram

s to

in

terp

ret.

Calcu

latio

ns s

how

ing

met

hods

in o

rder

to ju

stify

an

swer

s.

Ope

n en

ded

shor

t writ

ten

expl

anat

ions

requ

ired.

Patte

rn-s

eeki

ng fr

om

diag

ram

on

plan

ting

appl

e tre

es.

Link

s to

mat

hem

atics

, ge

ogra

phy.

4. C

ubes

Med

ium

(Q4.

1)Pa

ttern

s an

d re

latio

nshi

ps,

unde

rsta

nd n

umbe

r and

no

tatio

n.

Shor

t con

tinuo

us te

xt

info

rmat

ion

with

pict

ure

to

inte

rpre

t

Calcu

latio

n re

quire

d (m

enta

l or

writ

ten

as p

refe

rred)

.Pa

ttern

-see

king

and

usin

g se

t crit

eria

. Li

nks

to m

athe

mat

ics.

5. C

ontin

ent A

rea

High

(Q5.

1)Un

ders

tand

num

ber a

nd

nota

tion,

cal

cula

te in

va

riety

of w

ays,

shap

e, a

rea,

pe

rimet

er, i

nter

pret

and

use

sc

ales

, est

imat

ion.

Inte

rpre

tatio

n of

dia

gram

.Ca

lcula

tion

requ

ired,

sh

owin

g w

orki

ngs

and

expl

aini

ng h

ow a

nsw

er

arriv

ed a

t.

Usin

g an

d in

terp

retin

g a

scal

e gr

aph

of A

ntar

ctica

.Li

nks

to m

athe

mat

ics,

geog

raph

y.

6. G

row

ing

UpM

ediu

m (Q

6.1)

Med

ium

(Q6.

2)M

ediu

m (Q

6.3)

Inte

rpre

ting

and

anal

ysin

g da

ta, p

atte

rns

and

rela

tions

hips

, und

erst

and

and

use

mea

sure

s.

Shor

t con

tinuo

us te

xt

info

rmat

ion

with

pict

ure

to

inte

rpre

t.

Sim

ple

calcu

latio

n an

d gr

aphi

cal i

nter

pret

atio

n.

Ope

n en

ded

shor

t writ

ten

just

ifica

tion

usin

g gr

aph.

Gra

phica

l rel

atio

nshi

p be

twee

n he

ight

and

age

.Li

nks

to m

athe

mat

ics,

scie

nce,

PSE

.

Mat

hem

atic

al li

tera

cy*

deno

tes

poss

ible

cro

ss-c

urri

cula

r lin

ks w

ith

scie

nce

38 A guide to using PISA as a learning context

Que

stio

nLe

vel o

f di

fficu

lty

Nat

iona

l cur

ricu

lum

m

athe

mat

ical

Con

text

Text

typ

eQ

uest

ion

type

sSu

bjec

t m

atte

r an

d

cros

s-cu

rric

ular

link

s

7. S

peed

of

Racin

g Ca

r*Lo

w (Q

7.1)

Low

(Q7.

2)Lo

w (Q

7.3)

High

(Q7.

4)

Unde

rsta

nd a

nd u

se

mea

sure

s, in

terp

ret a

nd

anal

yse

data

, pat

tern

s an

d re

latio

nshi

ps.

Shor

t con

tinuo

us te

xt

info

rmat

ion

with

pict

ure

to

inte

rpre

t.

Mul

tiple

cho

ice, g

raph

ical

inte

rpre

tatio

n.G

raph

ical r

elat

ions

hip

betw

een

spee

d an

d di

stan

ce

for a

car

.Li

nks

to m

athe

mat

ics,

scie

nce.

8. T

riang

les

Med

ium

(Q8.

1)Sh

ape.

Shor

t con

tinuo

us te

xt

info

rmat

ion

with

dia

gram

s to

in

terp

ret.

Mul

tiple

cho

ice,

diag

ram

mat

ical

inte

rpre

tatio

n an

d tra

nsla

tion

of w

ritte

n de

scrip

tion

into

di

agra

m.

Use

of d

iagr

ams

and

te

xt-b

ased

des

crip

tions

of

trian

gles

.Li

nks

to m

athe

mat

ics, a

rt an

d de

sign.

9. R

obbe

ries

High

(Q9.

1)Pa

ttern

s an

d re

latio

nshi

ps,

inte

rpre

t and

ana

lyse

data

.Sh

ort c

ontin

uous

text

in

form

atio

n w

ith b

ar c

hart

to

inte

rpre

t.

Ope

n en

ded

long

er w

ritte

n re

spon

se ju

stify

ing

opin

ions

.Da

ta o

n ro

bber

ies

in d

iffer

ent

year

s.Li

nks

to m

athe

mat

ics, P

SE,

Engl

ish/W

elsh

.

10. C

arpe

nter

High

(Q10

.1)

Unde

rsta

nd a

nd u

se

mea

sure

s, pe

rimet

er, a

rea,

sh

ape,

und

erst

and

num

ber

and

nota

tion,

cal

cula

te in

a

varie

ty o

f way

s.

Shor

t con

tinuo

us te

xt

info

rmat

ion

with

dia

gram

s to

in

terp

ret.

Mul

tiple

cho

ice.

Inte

rpre

tatio

n of

dia

gram

s an

d ca

lcula

tion

(m

enta

l/writ

ten

as p

refe

rred)

re

quire

d.

Diag

ram

mat

ical

repr

esen

tatio

n of

gar

den

desig

n.Li

nks

to m

athe

mat

ics, a

rt an

d de

sign,

des

ign

and

tech

nolo

gy.

11. I

nter

net R

elay

Cha

tM

ediu

m (Q

11.1

)Hi

gh (Q

11.2

)Un

ders

tand

and

use

m

easu

res,

time,

und

erst

and

num

ber a

nd n

otat

ion,

ca

lcula

te in

a v

arie

ty o

f way

s.

Shor

t con

tinuo

us te

xt

info

rmat

ion

with

dia

gram

s to

in

terp

ret.

Calcu

latio

ns.

Text

-bas

ed a

nd

diag

ram

mat

ical i

nfor

mat

ion

rela

ted

to ti

me

zone

s.Li

nks

to m

athe

mat

ics,

geog

raph

y, PS

E, s

cienc

e.

12. E

xcha

nge

R

ate

Low

(Q12

.1)

Low

(Q12

.2)

Med

ium

(Q12

.3)

Unde

rsta

nd a

nd u

se m

oney

, al

gebr

aic

rela

tions

hips

and

fu

nctio

ns, c

alcu

late

in a

va

riety

of w

ays,

use

num

ber

and

nota

tion.

Shor

t con

tinuo

us te

xt.

Inte

rpre

tatio

n of

fo

rmul

ae a

nd te

xt

equi

vale

nce.

Ca

lcula

tions

. O

pen

ende

d lo

nger

writ

ten

resp

onse

to ju

stify

arg

umen

t.

Text

-bas

ed in

form

atio

n an

d ca

lcula

tions

invo

lvin

g ex

chan

ge ra

tes.

Link

s to

mat

hem

atics

, ge

ogra

phy,

PSE.

A guide to using PISA as a learning context 39

13. E

xpor

tsLo

w (Q

13.1

)M

ediu

m (Q

13.2

)Ha

ndlin

g da

ta, i

nter

pret

and

an

alys

e da

ta.

Shor

t con

tinuo

us te

xt w

ith

pie

char

t and

bar

cha

rt to

in

terp

ret.

Gra

phica

l int

erpr

etat

ion

and

calcu

latio

n.

Shor

t ans

wer

cal

cula

tion

and

mul

tiple

cho

ice.

Gra

phica

l rep

rese

ntat

ion

invo

lvin

g ex

ports

per

yea

r.Li

nks

to m

athe

mat

ics,

geog

raph

y, PS

E.

14. C

olou

red

C

andi

esM

ediu

m (Q

14.1

)Pr

obab

ility

, dat

a ha

ndlin

g.Sh

ort c

ontin

uous

text

with

ba

r cha

rt to

inte

rpre

t.M

ultip

le c

hoice

. Ca

lcula

tion

nece

ssar

y to

se

lect

ans

wer

.

Gra

phica

l inf

orm

atio

n to

look

at

pro

babi

lity.

Link

s to

mat

hem

atics

.

15. S

cienc

e Te

sts

Med

ium

(Q15

.1)

Hand

ling

data

, und

erst

and

num

ber a

nd n

otat

ion,

ca

lcula

te in

var

iety

of w

ays,

fract

ions

, per

cent

ages

, mea

n.

Shor

t con

tinuo

us te

xt.

Clos

ed s

hort

answ

er

calcu

latio

n.Te

xt-b

ased

info

rmat

ion

look

ing

at m

athe

mat

ical

reas

onin

g.Li

nks

to m

athe

mat

ics.

16. B

ooks

helv

esLo

w (Q

16.1

)Un

ders

tand

num

ber a

nd

nota

tion,

alg

ebra

, cal

cula

te

in v

arie

ty o

f way

s.

Shor

t con

tinuo

us te

xt w

ith

diag

ram

to in

terp

ret.

Inte

rpre

tatio

n of

dia

gram

an

d ca

lcula

tion

requ

ired

for

close

d sh

ort a

nsw

er.

Text

-bas

ed c

riter

ia to

bui

ld a

bo

okca

se d

escr

ibed

.Li

nks

to m

athe

mat

ics, d

esig

n an

d te

chno

logy

.

17. L

itter

*M

ediu

m (Q

17.1

)Ha

ndlin

g da

ta, i

nter

pret

and

an

alys

e da

ta.

Shor

t con

tinuo

us te

xt w

ith

tabl

e of

dat

a to

inte

rpre

t.O

pen

ende

d sh

ort w

ritte

n re

spon

se to

just

ify o

pini

on.

Tabu

late

d da

ta a

nd

text

-bas

ed in

form

atio

n on

ty

pes

of li

tter c

olle

cted

.Li

nks

to m

athe

mat

ics, P

SE,

scie

nce,

geo

grap

hy.

18. E

arth

quak

eM

ediu

m (Q

18.1

)Pr

obab

ility

, pat

tern

s an

d re

latio

nshi

ps, u

nder

stan

d nu

mbe

r and

not

atio

n,

calcu

late

in v

arie

ty o

f way

s.

Shor

t con

tinuo

us te

xt.

Mul

tiple

cho

ice.

Inte

rpre

tatio

n of

text

and

tra

nsla

tion

to m

athe

mat

ical

reas

onin

g, in

cludi

ng s

ome

calcu

latio

n.

Text

-bas

ed in

form

atio

n on

ea

rthqu

akes

to p

roce

ss a

nd

mak

e de

cisio

ns re

gard

ing

stat

emen

ts m

ade.

Link

s to

mat

hem

atics

, En

glish

/Wel

sh, s

cienc

e,

geog

raph

y.

40 A guide to using PISA as a learning context

Que

stio

nLe

vel o

f di

fficu

lty

Nat

iona

l cur

ricu

lum

m

athe

mat

ical

Con

text

Text

typ

eQ

uest

ion

type

sSu

bjec

t m

atte

r an

d

cros

s-cu

rric

ular

link

s

19. C

hoice

sM

ediu

m (Q

19.1

)Co

mbi

natio

ns, p

atte

rns

and

rela

tions

hips

, und

erst

and

num

ber a

nd n

otat

ion.

Shor

t con

tinuo

us te

xt.

Shor

t ans

wer

cal

cula

tion.

Text

-bas

ed in

form

atio

n re

gard

ing

com

bina

tions

of

pizz

a to

ppin

gs to

be

used

.Li

nks

to m

athe

mat

ics,

scie

nce,

des

ign

and

tech

nolo

gy, P

SE.

20. T

est S

core

sHi

gh (Q

20.1

)Ha

ndlin

g da

ta, p

roba

bilit

y, un

ders

tand

and

use

m

easu

res.

Shor

t con

tinuo

us te

xt w

ith

bar c

hart

to in

terp

ret.

Ope

n en

ded

long

er w

ritte

n re

spon

se to

just

ify o

pini

on

and

set o

ut m

athe

mat

ical

argu

men

t.

Text

-bas

ed a

nd g

raph

ical

info

rmat

ion

on te

st s

core

s to

inte

rpre

t and

eva

luat

e st

atem

ents

mad

e.Li

nks

to m

athe

mat

ics,

Engl

ish/W

elsh

, PSE

.

21. S

kate

boar

dLo

w (Q

21.1

)M

ediu

m (Q

21.2

)M

ediu

m (Q

21.3

)

Com

bina

tions

, pat

tern

s an

d re

latio

nshi

ps, u

nder

stan

d an

d us

e m

oney

, und

erst

and

num

ber a

nd n

otat

ion,

ca

lcula

te in

var

iety

of w

ays.

Shor

t con

tinuo

us te

xt w

ith

pict

ures

and

tabu

late

d da

ta

to in

terp

ret.

Mul

tiple

cho

ice a

nd s

ort

answ

er c

alcu

latio

n.Da

ta o

n co

nstru

ctio

n of

a

skat

eboa

rd a

nd p

ossib

le

price

s.Li

nks

to m

athe

mat

ics, P

SE,

desig

n an

d te

chno

logy

.

22. S

tairc

ase

Low

(Q22

.1)

Shap

e, p

erim

eter

, are

a, u

se

mea

sure

s, ca

lcula

te in

var

iety

of

way

s.

Shor

t con

tinuo

us te

xt w

ith

diag

ram

to in

terp

ret.

Shor

t ans

wer

cal

cula

tion.

Inte

rpre

ting

diag

ram

to

calcu

late

info

rmat

ion

abou

t a

stai

rcas

e.Li

nks

to m

athe

mat

ics,

scie

nce,

des

ign

and

tech

nolo

gy.

23. N

umbe

r

C

ubes

Med

ium

(Q23

.1)

Shap

e, p

atte

rns

and

rela

tions

hips

.Sh

ort c

ontin

uous

text

with

pi

ctur

es a

nd d

iagr

ams

to

inte

rpre

t.

Mul

tiple

cho

ice.

Com

parin

g te

xt-b

ased

cr

iteria

with

‘net

s’ to

pro

duce

a

dice

.Li

nks

to m

athe

mat

ics, d

esig

n an

d te

chno

logy

.

A guide to using PISA as a learning context 41

24. S

uppo

rt fo

r

the

P

resid

ent

High

(Q24

.1)

Hand

ling

data

, pat

tern

s an

d re

latio

nshi

ps, u

nder

stan

d nu

mbe

r and

not

atio

n,

calcu

late

in v

arie

ty o

f way

s.

Med

ium

leng

th c

ontin

uous

te

xt w

ith m

athe

mat

ical d

ata

inclu

ded.

Ope

n en

ded

long

er le

ngth

w

ritte

n re

spon

se to

just

ify

opin

ions

and

set

out

re

ason

ed m

athe

mat

ical

argu

men

t.

Cons

ider

ing

evid

ence

from

op

inio

n po

lls.

Link

s to

mat

hem

atics

, En

glish

/Wel

sh, P

SE.

25. T

he B

est C

arLo

w (Q

25.1

)Hi

gh (Q

25.2

)Al

gebr

aic

rela

tions

hips

and

fu

nctio

ns, p

atte

rns

and

rela

tions

hips

.

Shor

t con

tinuo

us te

xt w

ith

tabu

late

d da

ta to

inte

rpre

t.Cl

osed

sho

rt an

swer

ca

lcula

tion

and

inte

rpre

tatio

n of

dat

a as

form

ulae

.

Look

ing

at d

iffer

ent c

riter

ia

for c

ars

to d

evel

op a

co

nclu

sion.

Link

s to

mat

hem

atics

, sc

ienc

e, d

esig

n an

d te

chno

logy

.

26. S

tep

Patte

rnLo

w (Q

26.1

)Pa

ttern

s an

d re

latio

nshi

ps.

Shor

t con

tinuo

us te

xt w

ith

diag

ram

s to

inte

rpre

t.Cl

osed

sho

rt an

swer

ca

lcula

tion.

Patte

rn s

eeki

ng u

sing

squa

res

to b

uild

a s

et

patte

rn.

Link

s to

mat

hem

atics

, art

and

desig

n.

27. L

ichen

Tria

l:M

ediu

m (Q

27.1

)M

ediu

m (Q

27.2

)

Alge

brai

c re

latio

nshi

ps a

nd

func

tions

, pat

tern

s an

d re

latio

nshi

ps, u

nder

stan

d an

d us

e nu

mbe

r and

not

atio

n,

calcu

late

in v

arie

ty o

f way

s.

Med

ium

leng

th c

ontin

uous

te

xt w

ith m

athe

mat

ical

form

ulae

to in

terp

ret.

Calcu

latio

ns w

ith w

orki

ngs

to ju

stify

ans

wer

s.Te

xt-b

ased

info

rmat

ion

on

liche

n an

d us

ing

a fo

rmul

a to

des

crib

e th

e re

latio

nshi

p be

twee

n di

amet

er o

f lich

en

and

age.

Link

s to

mat

hem

atics

, sc

ienc

e, g

eogr

aphy

.

28. C

oins

Tria

l:Hi

gh (Q

28.1

)Pa

ttern

s an

d re

latio

nshi

ps,

unde

rsta

nd n

umbe

r and

no

tatio

n, u

nder

stan

d an

d us

e m

oney

, cal

cula

te in

var

iety

of

way

s, un

ders

tand

and

use

m

easu

res.

Med

ium

leng

th te

xt w

ith

mat

hem

atica

l dat

a an

d pi

ctur

es to

inte

rpre

t.

Calcu

latio

ns re

quire

d.De

signi

ng a

set

of c

oins

to a

se

t crit

eria

.Li

nks

to m

athe

mat

ics, d

esig

n an

d te

chno

logy

.

42 A guide to using PISA as a learning context

Que

stio

nLe

vel o

f di

fficu

lty

Nat

iona

l cur

ricu

lum

m

athe

mat

ical

Con

text

Text

typ

eQ

uest

ion

type

sSu

bjec

t m

atte

r an

d

cros

s-cu

rric

ular

link

s

29. P

izzas

Tria

l:Lo

w (Q

29.1

)

Com

bina

tions

, pat

tern

s an

d re

latio

nshi

ps, u

nder

stan

d an

d us

e m

oney

, und

erst

and

num

ber a

nd n

otat

ion,

ca

lcula

te in

var

iety

of w

ays.

Shor

t con

tinuo

us te

xt

cont

aini

ng m

athe

mat

ical

info

rmat

ion.

Shor

t ans

wer

writ

ten

resp

onse

to ju

stify

opi

nion

s.Re

ason

ing

valu

e fo

r mon

ey

for d

iffer

ent p

izzas

.Li

nks

to m

athe

mat

ics, P

SE.

30. S

hape

sTr

ial:

Med

ium

(Q30

.1)

Low

(Q30

.2)

Low

(Q30

.3)

Shap

e, u

nder

stan

d an

d us

e m

easu

res,

area

, per

imet

er,

estim

atio

n, c

alcu

late

in a

va

riety

of w

ays.

Shor

t tex

t and

pict

ures

to

inte

rpre

t.O

pen

ende

d lo

nger

leng

th

writ

ten

resp

onse

to e

xpla

in

reas

onin

g.

Estim

atin

g th

e ar

ea a

nd

perim

eter

of r

egul

ar a

nd

irreg

ular

sha

pes.

Link

s to

mat

hem

atics

, des

ign

and

tech

nolo

gy, a

rt.

31. B

raki

ng*

Tria

l:Hi

gh (Q

31.1

)Hi

gh (Q

31.2

)Hi

gh (Q

31.3

)Hi

gh (Q

31.4

)Hi

gh (Q

31.5

)

Alge

brai

c re

latio

nshi

ps a

nd

func

tions

, han

dlin

g da

ta,

anal

yse

and

inte

rpre

t dat

a,

unde

rsta

nd a

nd u

se n

umbe

r an

d no

tatio

n, c

alcu

late

in a

va

riety

of w

ays.

Med

ium

leng

th c

ontin

uous

te

xt w

ith d

etai

led

diag

ram

to

inte

rpre

t.

Long

er le

ngth

writ

ten

resp

onse

s in

volv

ing

calcu

latio

ns a

nd in

terp

retin

g an

d ex

tract

ing

data

from

di

agra

m.

Text

-bas

ed in

form

atio

n an

d co

mpl

ex d

iagr

am a

nd

grap

hica

l dat

a fo

r sto

ppin

g an

d br

eaki

ng d

istan

ces.

Link

s to

mat

hem

atics

, sc

ienc

e, P

SE.

32. P

atio

Tria

l:Lo

w (Q

32.1

)Sh

ape,

und

erst

and

and

use

mea

sure

s, ar

ea, u

nder

stan

d nu

mbe

r and

not

atio

n,

calcu

late

in a

var

iety

of w

ays.

Shor

t con

tinuo

us te

xt

cont

aini

ng m

athe

mat

ical

data

.

Shor

t ans

wer

cal

cula

tion.

Desig

n of

pat

io u

sing

set

crite

ria.

Link

s to

mat

hem

atics

, des

ign

and

tech

nolo

gy.

33. D

rug

Conc

entra

tions

*Tr

ial:

Med

ium

(Q33

.1)

Low

(Q33

.2)

Low

(Q33

.3)

Patte

rns

and

rela

tions

hips

, un

ders

tand

num

ber a

nd

nota

tion,

cal

cula

te in

a

varie

ty o

f way

s, ha

ndlin

g da

ta, a

nalys

e an

d in

terp

ret

data

.

Med

ium

leng

th c

ontin

uous

te

xt c

onta

inin

g m

athe

mat

ical

data

and

tabu

late

d da

ta, a

nd

line

grap

h to

inte

rpre

t.

Calcu

latio

n an

d m

ultip

le

choi

ce.

Inte

rpre

tatio

n fro

m g

raph

ical

info

rmat

ion

give

n.

Text

-bas

ed in

form

atio

n an

d gr

aphi

cal r

elat

ions

hip

betw

een

amou

nt o

f dru

g an

d tim

e.Li

nks

to m

athe

mat

ics,

scie

nce,

PSE

.

34. B

uild

ing

B

lock

sTr

ial:

Low

(Q34

.1)

Low

(Q34

.2)

Low

(Q34

.3)

Low

(Q34

.4)

Patte

rns

and

rela

tions

hips

, sh

ape,

und

erst

and

and

use

mea

sure

s, ca

lcula

te in

a

varie

ty o

f way

s.

Med

ium

leng

th c

ontin

uous

da

ta a

nd d

iagr

ams

to

inte

rpre

t.

Shor

t ans

wer

cal

cula

tions

.Us

ing

two-

dim

ensio

nal

diag

ram

s of

cub

es to

bui

ld

larg

er s

truct

ures

.Li

nks

to m

athe

mat

ics, d

esig

n an

d te

chno

logy

.

A guide to using PISA as a learning context 43

Que

stio

nLe

vel o

f di

fficu

lty

Nat

iona

l cur

ricu

lum

m

athe

mat

ical

Con

text

Text

typ

eQ

uest

ion

type

sSu

bjec

t m

atte

r an

d

cros

s-cu

rric

ular

link

s

29. P

izzas

Tria

l:Lo

w (Q

29.1

)

Com

bina

tions

, pat

tern

s an

d re

latio

nshi

ps, u

nder

stan

d an

d us

e m

oney

, und

erst

and

num

ber a

nd n

otat

ion,

ca

lcula

te in

var

iety

of w

ays.

Shor

t con

tinuo

us te

xt

cont

aini

ng m

athe

mat

ical

info

rmat

ion.

Shor

t ans

wer

writ

ten

resp

onse

to ju

stify

opi

nion

s.Re

ason

ing

valu

e fo

r mon

ey

for d

iffer

ent p

izzas

.Li

nks

to m

athe

mat

ics, P

SE.

30. S

hape

sTr

ial:

Med

ium

(Q30

.1)

Low

(Q30

.2)

Low

(Q30

.3)

Shap

e, u

nder

stan

d an

d us

e m

easu

res,

area

, per

imet

er,

estim

atio

n, c

alcu

late

in a

va

riety

of w

ays.

Shor

t tex

t and

pict

ures

to

inte

rpre

t.O

pen

ende

d lo

nger

leng

th

writ

ten

resp

onse

to e

xpla

in

reas

onin

g.

Estim

atin

g th

e ar

ea a

nd

perim

eter

of r

egul

ar a

nd

irreg

ular

sha

pes.

Link

s to

mat

hem

atics

, des

ign

and

tech

nolo

gy, a

rt.

31. B

raki

ng*

Tria

l:Hi

gh (Q

31.1

)Hi

gh (Q

31.2

)Hi

gh (Q

31.3

)Hi

gh (Q

31.4

)Hi

gh (Q

31.5

)

Alge

brai

c re

latio

nshi

ps a

nd

func

tions

, han

dlin

g da

ta,

anal

yse

and

inte

rpre

t dat

a,

unde

rsta

nd a

nd u

se n

umbe

r an

d no

tatio

n, c

alcu

late

in a

va

riety

of w

ays.

Med

ium

leng

th c

ontin

uous

te

xt w

ith d

etai

led

diag

ram

to

inte

rpre

t.

Long

er le

ngth

writ

ten

resp

onse

s in

volv

ing

calcu

latio

ns a

nd in

terp

retin

g an

d ex

tract

ing

data

from

di

agra

m.

Text

-bas

ed in

form

atio

n an

d co

mpl

ex d

iagr

am a

nd

grap

hica

l dat

a fo

r sto

ppin

g an

d br

eaki

ng d

istan

ces.

Link

s to

mat

hem

atics

, sc

ienc

e, P

SE.

32. P

atio

Tria

l:Lo

w (Q

32.1

)Sh

ape,

und

erst

and

and

use

mea

sure

s, ar

ea, u

nder

stan

d nu

mbe

r and

not

atio

n,

calcu

late

in a

var

iety

of w

ays.

Shor

t con

tinuo

us te

xt

cont

aini

ng m

athe

mat

ical

data

.

Shor

t ans

wer

cal

cula

tion.

Desig

n of

pat

io u

sing

set

crite

ria.

Link

s to

mat

hem

atics

, des

ign

and

tech

nolo

gy.

33. D

rug

Conc

entra

tions

*Tr

ial:

Med

ium

(Q33

.1)

Low

(Q33

.2)

Low

(Q33

.3)

Patte

rns

and

rela

tions

hips

, un

ders

tand

num

ber a

nd

nota

tion,

cal

cula

te in

a

varie

ty o

f way

s, ha

ndlin

g da

ta, a

nalys

e an

d in

terp

ret

data

.

Med

ium

leng

th c

ontin

uous

te

xt c

onta

inin

g m

athe

mat

ical

data

and

tabu

late

d da

ta, a

nd

line

grap

h to

inte

rpre

t.

Calcu

latio

n an

d m

ultip

le

choi

ce.

Inte

rpre

tatio

n fro

m g

raph

ical

info

rmat

ion

give

n.

Text

-bas

ed in

form

atio

n an

d gr

aphi

cal r

elat

ions

hip

betw

een

amou

nt o

f dru

g an

d tim

e.Li

nks

to m

athe

mat

ics,

scie

nce,

PSE

.

34. B

uild

ing

B

lock

sTr

ial:

Low

(Q34

.1)

Low

(Q34

.2)

Low

(Q34

.3)

Low

(Q34

.4)

Patte

rns

and

rela

tions

hips

, sh

ape,

und

erst

and

and

use

mea

sure

s, ca

lcula

te in

a

varie

ty o

f way

s.

Med

ium

leng

th c

ontin

uous

da

ta a

nd d

iagr

ams

to

inte

rpre

t.

Shor

t ans

wer

cal

cula

tions

.Us

ing

two-

dim

ensio

nal

diag

ram

s of

cub

es to

bui

ld

larg

er s

truct

ures

.Li

nks

to m

athe

mat

ics, d

esig

n an

d te

chno

logy

.

35. R

eact

ion

Ti

me*

Tria

l:Lo

w (Q

35.1

)M

ediu

m (Q

35.2

)

Patte

rns

and

rela

tions

hips

, un

ders

tand

and

use

num

ber

and

nota

tion,

cal

cula

te in

a

varie

ty o

f way

s, un

ders

tand

an

d us

e m

easu

res,

time,

ha

ndlin

g da

ta, i

nter

pret

and

an

alys

e da

ta.

Long

er le

ngth

con

tinuo

us

data

with

tabu

late

d da

ta to

in

terp

ret.

Calcu

latio

n an

d op

en

ende

d lo

nger

leng

th w

ritte

n re

spon

se to

just

ify o

pini

ons

and

show

mat

hem

atica

l re

ason

ing.

Inte

rpre

tatio

n an

d in

terro

gatio

n of

tabu

late

d da

ta o

f rea

ctio

n tim

es fo

r sp

rinte

rs.

Link

s to

mat

hem

atics

, sc

ienc

e, P

E.

36. W

ater

Tank

*Tr

ial:

Low

(Q36

.1)

Shap

e, u

nder

stan

d an

d us

e m

easu

res,

patte

rns

and

rela

tions

hips

, han

dlin

g da

ta,

anal

yse

and

inte

rpre

t dat

a.

Shor

t len

gth

cont

inuo

us d

ata

with

dia

gram

and

line

gra

phs

to in

terp

ret.

Mul

tiple

cho

ice.

Usin

g di

agra

m o

f wat

er ta

nk

to p

redi

ct h

ow h

eigh

t of

wat

er s

urfa

ce c

hang

es o

ver

time.

Link

s to

mat

hem

atics

, sc

ienc

e, d

esig

n an

d te

chno

logy

.

37. S

prin

g Fa

irTr

ial:

Low

(Q37

.1)

Prob

abili

ty.

Shor

t con

tinuo

us te

xt a

nd

pict

ures

to in

terp

ret.

Mul

tiple

cho

ice.

Mod

ellin

g pr

obab

ility

in th

e co

ntex

t of a

gam

e.Li

nks

to m

athe

mat

ics, P

SE.

38. S

win

g*Tr

ial:

Low

(Q38

.1)

Hand

ling

data

, ana

lyse

and

inte

rpre

t dat

a, p

atte

rns

and

rela

tions

hips

.

Shor

t con

tinuo

us te

xt a

nd

line

grap

hs to

inte

rpre

t.M

ultip

le c

hoice

.In

terp

retin

g gr

aphi

cal

repr

esen

tatio

n of

rela

tions

hip

betw

een

heig

ht o

f fee

t and

tim

e fo

r a p

erso

n on

a s

win

g.Li

nks

to m

athe

mat

ics,

scie

nce.

39. S

tude

nt

Hei

ghts

Tria

l:Lo

w (Q

39.1

)Ha

ndlin

g da

ta, a

nalys

e an

d in

terp

ret d

ata,

pat

tern

s an

d re

latio

nshi

ps.

Med

ium

leng

th c

ontin

uous

te

xt in

corp

orat

ing

mat

hem

atica

l dat

a.

Mul

tiple

cho

ice.

Text

-bas

ed in

form

atio

n de

scrib

ing

the

rang

e of

he

ight

s of

pup

ils in

a

class

and

look

ing

at w

hat

conc

lusio

ns c

an b

e re

liabl

y dr

awn.

Link

s to

mat

hem

atics

, En

glish

/Wel

sh, s

cienc

e.

44 A guide to using PISA as a learning context

Que

stio

nLe

vel o

f di

fficu

lty

Nat

iona

l cur

ricu

lum

m

athe

mat

ical

Con

text

Text

typ

eQ

uest

ion

type

sSu

bjec

t m

atte

r an

d

cros

s-cu

rric

ular

link

s

40. P

aym

ents

by

Are

aTr

ial:

Low

(Q40

.1)

Med

ium

(Q40

.2)

Patte

rns

and

rela

tions

hips

, un

ders

tand

and

use

num

ber

and

nota

tion,

und

erst

and

and

use

mea

sure

s, ar

ea,

mon

ey, c

alcu

late

in a

var

iety

of

way

s.

Med

ium

leng

th c

ontin

uous

te

xt in

corp

orat

ing

mat

hem

atica

l dat

a.

Mul

tiple

cho

ice a

nd

calcu

latio

n, s

how

ing

wor

king

s to

pre

sent

m

athe

mat

ical a

rgum

ent.

Look

ing

at s

tate

men

ts a

nd

calcu

latin

g co

sts

per a

rea.

Link

s to

mat

hem

atics

, En

glish

/Wel

sh, P

SE.

41. S

hoes

for K

ids

Tria

l:Lo

w (Q

41.1

)Ha

ndlin

g da

ta, a

nalys

e an

d in

terp

ret d

ata.

Shor

t con

tinuo

us te

xt w

ith

tabu

late

d da

ta to

inte

rpre

t.Sh

ort a

nsw

er –

inte

rpre

tatio

n fro

m ta

ble.

Inte

rpre

ting

conv

ersio

n da

ta

for s

hoes

.Li

nks

to m

athe

mat

ics, P

SE.

42. T

able

Tenn

is

T

ourn

amen

tTr

ial:

Med

ium

(Q42

.2)

Patte

rns

and

rela

tions

hips

, co

mbi

natio

ns, s

eque

nces

, un

ders

tand

and

use

num

ber

and

nota

tion.

Shor

t con

tinuo

us te

xt.

Shor

t writ

ten

an

swer

s –

com

plet

ion

of ta

ble,

alth

ough

muc

h re

ason

ing

and

calcu

latio

n re

quire

d.

Look

ing

at c

ombi

natio

ns in

th

e co

ntex

t of a

tabl

e te

nnis

tour

nam

ent.

Link

s to

mat

hem

atics

.

43. L

ight

hous

eTr

ial:

Med

ium

(Q43

.1)

Med

ium

(Q43

.2)

Med

ium

(Q43

.3)

Patte

rns

and

rela

tions

hips

, se

quen

ces,

unde

rsta

nd a

nd

use

mea

sure

s, tim

e, h

andl

ing

data

, ana

lyse

and

inte

rpre

t da

ta.

Med

ium

leng

th c

ontin

uous

te

xt w

ith g

raph

to in

terp

ret.

Mul

tiple

cho

ice a

nd g

raph

pl

ottin

g. C

alcu

latio

n ne

eded

.In

terp

retin

g te

xt-b

ased

in

form

atio

n an

d gr

aphi

cal

data

for l

ight

pat

tern

s fro

m a

lig

htho

use.

Link

s to

mat

hem

atics

, sc

ienc

e.

44. D

ecre

asin

g

C

O2 l

evel

s*Tr

ial:

High

(Q44

.1)

High

(Q44

.2)

High

(Q44

.3)

Patte

rns

and

rela

tions

hips

, un

ders

tand

and

use

num

ber

and

nota

tion,

cal

cula

te in

a

varie

ty o

f way

s, ha

ndlin

g da

ta, a

nalys

e an

d in

terp

ret

data

.

Long

er le

ngth

con

tinuo

us

text

with

bar

cha

rt an

d m

athe

mat

ical

data

inco

rpor

ated

for

inte

rpre

tatio

n.

Long

er le

ngth

ans

wer

s –

calcu

latio

n w

ith w

orki

ngs

and

open

end

ed w

ritte

n re

spon

se to

just

ify o

pini

ons.

Inte

rpre

ting

grap

hica

l dat

a on

car

bon

diox

ide

leve

ls in

th

e at

mos

pher

e an

d dr

awin

g ap

prop

riate

con

clusio

ns.

Link

s to

mat

hem

atics

, scie

nce,

ge

ogra

phy,

Engl

ish/W

elsh

, PS

E.

A guide to using PISA as a learning context 45

45. T

wist

ed

Bui

ldin

gTr

ial:

High

(Q45

.1)

High

(Q45

.2)

High

(Q45

.3)

High

(Q45

.4)

Shap

e, u

nder

stan

d an

d us

e po

sitio

n an

d m

ovem

ent,

unde

rsta

nd a

nd u

se

mea

sure

s, ar

ea, e

stim

atio

n,

anal

yse

and

inte

rpre

t dat

a,

scal

e.

Long

er le

ngth

con

tinuo

us te

xt

with

pict

ures

and

dia

gram

s to

in

terp

ret.

Mul

tiple

cho

ice, c

alcu

latio

n w

ith lo

nger

leng

th

writ

ten

just

ifica

tion

and

expl

anat

ion,

dia

gram

mat

ical

repr

esen

tatio

n of

idea

s.

Usin

g ar

chite

ctur

e as

a

cont

ext t

o es

timat

e di

men

sions

and

orie

ntat

ion

of

a bu

ildin

g.Li

nks

to m

athe

mat

ics, d

esig

n an

d te

chno

logy

, scie

nce.

46. H

eartb

eat*

Tria

l:Hi

gh (Q

46.1

)Hi

gh (Q

46.2

)

Alge

brai

c re

latio

nshi

ps a

nd

func

tions

, pat

tern

s an

d re

latio

nshi

ps, i

nter

pret

and

us

e fo

rmul

ae, u

nder

stan

d an

d us

e nu

mbe

r and

not

atio

n,

calcu

late

in a

var

iety

of w

ays.

Long

er le

ngth

con

tinuo

us te

xt

with

mat

hem

atica

l for

mul

ae

to in

terp

ret.

Long

er le

ngth

writ

ten

resp

onse

to s

how

cal

cula

tion

and

just

ify a

nsw

er.

Tran

slatio

n of

text

to

mat

hem

atica

l for

mul

ae.

Usin

g fit

ness

and

cal

cula

ting

pulse

rate

from

giv

en

form

ulae

and

com

parin

g fo

rmul

ae.

Link

s to

mat

hem

atics

, scie

nce,

PE

, PSE

.

47. S

pace

Flig

ht*

Tria

l:M

ediu

m

(Q47

.1)

Alge

brai

c re

latio

nshi

ps a

nd

func

tions

, und

erst

and

and

use

num

ber a

nd n

otat

ion,

ca

lcula

te in

a v

arie

ty o

f way

s.

Shor

t con

tinuo

us te

xt

cont

aini

ng m

athe

mat

ical

info

rmat

ion.

Mul

tiple

cho

ice.

Info

rmat

ion

abou

t the

orb

it of

sp

ace

stat

ion

Mir.

Link

s to

mat

hem

atics

, scie

nce.

48. R

ock

Conc

ert

Tria

l:Lo

w (Q

48.1

)Sh

ape,

und

erst

and

and

use

num

ber a

nd n

otat

ion,

un

ders

tand

and

use

m

easu

res,

estim

atio

n,

calcu

late

in a

var

iety

of w

ays.

Shor

t con

tinuo

us te

xt

cont

aini

ng m

athe

mat

ical

info

rmat

ion.

Mul

tiple

cho

ice.

Estim

atin

g th

e nu

mbe

r of

peop

le a

ttend

ing

a ro

ck

conc

ert.

Link

s to

mat

hem

atics

.

49. M

ovin

g

Wal

kway

s*Tr

ial:

Low

(Q49

.1)

Patte

rns

and

rela

tions

hips

, un

ders

tand

and

use

m

easu

res,

hand

ling

data

, an

alys

e an

d in

terp

ret d

ata.

Med

ium

leng

th c

ontin

uous

te

xt w

ith p

ictur

e an

d lin

e gr

aph

to in

terp

ret.

Gra

phica

l rep

rese

ntat

ion

of

outc

ome

of re

ason

ing.

Inte

rpre

ting

dist

ance

tim

e gr

aph

for a

mov

ing

wal

kway

an

d m

akin

g pr

edict

ions

.Li

nks

to m

athe

mat

ics, s

cienc

e.

50. P

osta

l

C

harg

esTr

ial:

Med

ium

(Q

50.1

)

Patte

rns

and

rela

tions

hips

, us

e m

oney

, han

dlin

g da

ta,

anal

yse

and

inte

rpre

t dat

a.

Long

er le

ngth

text

invo

lvin

g ta

bula

ted

data

and

line

gr

aphs

to in

terp

ret.

Mul

tiple

cho

ice a

nd

calcu

latio

n sh

owin

g w

orki

ngs

to ju

stify

mat

hem

atica

l ar

gum

ent.

Usin

g da

ta o

n po

stal

cha

rges

to

pre

dict

the

rela

tions

hip

betw

een

wei

ght a

nd c

harg

e of

an

item

.Li

nks

to m

athe

mat

ics, s

cienc

e.

46 A guide to using PISA as a learning context

Que

stio

nLe

vel o

f di

fficu

lty

Nat

iona

l cur

ricu

lum

sc

ienc

e co

ntex

tTe

xt t

ype

Que

stio

n ty

pes

Subj

ect

mat

ter

and

cr

oss-

curr

icul

ar li

nks

1. S

emm

elw

eis’

Diar

yHi

gh (Q

1.1)

Low

(Q1.

2)Lo

w (Q

1.3)

Med

ium

(Q1.

4)

Heal

th, d

iseas

e, d

rugs

to

treat

dise

ase.

Long

er le

ngth

con

tinuo

us

text

with

gra

ph to

inte

rpre

t.In

fere

nce

from

text

, mul

tiple

ch

oice

.Sh

ort w

ritte

n re

spon

ses

to

expl

ain

and

just

ify o

pini

ons.

Text

and

gra

phica

l-bas

ed

info

rmat

ion

on o

utbr

eak

of d

iseas

e an

d w

hat

conc

lusio

ns c

an b

e dr

awn.

Link

s to

scie

nce,

PSE

, En

glish

/Wel

sh.

2. O

zone

High

(Q2.

1)Hi

gh (Q

2.2)

Med

ium

(Q2.

3)

Atom

s an

d m

olec

ules

, bo

ndin

g.Lo

nger

leng

th c

ontin

uous

te

xt w

ith p

ictur

e st

rip to

in

terp

ret.

Acce

ss/re

triev

e in

form

atio

n fro

m te

xt, i

nter

pret

car

toon

st

imul

us in

form

atio

n.

Ope

n en

ded

shor

t writ

ten

resp

onse

to e

xpla

in id

eas.

Mul

tiple

cho

ice to

retri

eve

info

rmat

ion

from

text

.

Mod

ellin

g at

oms

and

mol

ecul

es a

nd th

e oz

one

laye

r fro

m te

xt-b

ased

in

form

atio

n.Li

nks

to s

cienc

e.

3. D

aylig

htM

ediu

m (Q

3.1)

High

(Q3.

2)Ea

rth in

spa

ce –

day

and

ni

ght.

Shor

t con

tinuo

us te

xt

info

rmat

ion

with

dia

gram

to

inte

rpre

t.

Mul

tiple

cho

ice.

Acce

ss/u

se in

form

atio

n fro

m te

xt, p

rese

nt/d

raw

on

diag

ram

.

Extra

ct fr

om n

ewsp

aper

ar

ticle

on

day

leng

th.

Link

s to

scie

nce,

geo

grap

hy,

Engl

ish/W

elsh

.

4. C

loni

ngLo

w (Q

4.1)

Med

ium

(Q4.

2)M

ediu

m (Q

4.3)

Gen

etics

and

clo

ning

.M

ediu

m le

ngth

con

tinuo

us

text

for i

nfor

mat

ion.

Mul

tiple

cho

ice to

ac

cess

/retri

eve

info

rmat

ion

and

infe

r fro

m te

xt.

Extra

ct fr

om n

ewsp

aper

ar

ticle

on

cloni

ng.

Link

s to

scie

nce,

PSE

, En

glish

/Wel

sh.

5. G

reen

hous

eM

ediu

m (Q

5.1)

High

(Q5.

2)Hi

gh (Q

5.3)

Gre

enho

use

effe

ct –

co

nsid

erin

g ev

iden

ce.

Long

er le

ngth

con

tinuo

us

text

inco

rpor

atin

g lin

e gr

aph

to in

terp

ret.

Inte

rpre

t gra

ph tr

ends

, dra

w

com

paris

ons,

shor

t writ

ten

resp

onse

s re

quire

d to

just

ify

idea

s.Id

entifi

catio

n of

ano

mal

y in

da

ta.

Text

-bas

ed in

form

atio

n an

d gr

aphi

cal d

ata

look

ing

at

evid

ence

of g

loba

l war

min

g.Li

nks

to s

cienc

e, g

eogr

aphy

, PS

E, E

nglis

h/W

elsh

, m

athe

mat

ics.

Scie

ntifi

c lit

erac

y

A guide to using PISA as a learning context 47

Que

stio

nLe

vel o

f di

fficu

lty

Nat

iona

l cur

ricu

lum

sc

ienc

e co

ntex

tTe

xt t

ype

Que

stio

n ty

pes

Subj

ect

mat

ter

and

cr

oss-

curr

icul

ar li

nks

1. S

emm

elw

eis’

Diar

yHi

gh (Q

1.1)

Low

(Q1.

2)Lo

w (Q

1.3)

Med

ium

(Q1.

4)

Heal

th, d

iseas

e, d

rugs

to

treat

dise

ase.

Long

er le

ngth

con

tinuo

us

text

with

gra

ph to

inte

rpre

t.In

fere

nce

from

text

, mul

tiple

ch

oice

.Sh

ort w

ritte

n re

spon

ses

to

expl

ain

and

just

ify o

pini

ons.

Text

and

gra

phica

l-bas

ed

info

rmat

ion

on o

utbr

eak

of d

iseas

e an

d w

hat

conc

lusio

ns c

an b

e dr

awn.

Link

s to

scie

nce,

PSE

, En

glish

/Wel

sh.

2. O

zone

High

(Q2.

1)Hi

gh (Q

2.2)

Med

ium

(Q2.

3)

Atom

s an

d m

olec

ules

, bo

ndin

g.Lo

nger

leng

th c

ontin

uous

te

xt w

ith p

ictur

e st

rip to

in

terp

ret.

Acce

ss/re

triev

e in

form

atio

n fro

m te

xt, i

nter

pret

car

toon

st

imul

us in

form

atio

n.

Ope

n en

ded

shor

t writ

ten

resp

onse

to e

xpla

in id

eas.

Mul

tiple

cho

ice to

retri

eve

info

rmat

ion

from

text

.

Mod

ellin

g at

oms

and

mol

ecul

es a

nd th

e oz

one

laye

r fro

m te

xt-b

ased

in

form

atio

n.Li

nks

to s

cienc

e.

3. D

aylig

htM

ediu

m (Q

3.1)

High

(Q3.

2)Ea

rth in

spa

ce –

day

and

ni

ght.

Shor

t con

tinuo

us te

xt

info

rmat

ion

with

dia

gram

to

inte

rpre

t.

Mul

tiple

cho

ice.

Acce

ss/u

se in

form

atio

n fro

m te

xt, p

rese

nt/d

raw

on

diag

ram

.

Extra

ct fr

om n

ewsp

aper

ar

ticle

on

day

leng

th.

Link

s to

scie

nce,

geo

grap

hy,

Engl

ish/W

elsh

.

4. C

loni

ngLo

w (Q

4.1)

Med

ium

(Q4.

2)M

ediu

m (Q

4.3)

Gen

etics

and

clo

ning

.M

ediu

m le

ngth

con

tinuo

us

text

for i

nfor

mat

ion.

Mul

tiple

cho

ice to

ac

cess

/retri

eve

info

rmat

ion

and

infe

r fro

m te

xt.

Extra

ct fr

om n

ewsp

aper

ar

ticle

on

cloni

ng.

Link

s to

scie

nce,

PSE

, En

glish

/Wel

sh.

5. G

reen

hous

eM

ediu

m (Q

5.1)

High

(Q5.

2)Hi

gh (Q

5.3)

Gre

enho

use

effe

ct –

co

nsid

erin

g ev

iden

ce.

Long

er le

ngth

con

tinuo

us

text

inco

rpor

atin

g lin

e gr

aph

to in

terp

ret.

Inte

rpre

t gra

ph tr

ends

, dra

w

com

paris

ons,

shor

t writ

ten

resp

onse

s re

quire

d to

just

ify

idea

s.Id

entifi

catio

n of

ano

mal

y in

da

ta.

Text

-bas

ed in

form

atio

n an

d gr

aphi

cal d

ata

look

ing

at

evid

ence

of g

loba

l war

min

g.Li

nks

to s

cienc

e, g

eogr

aphy

, PS

E, E

nglis

h/W

elsh

, m

athe

mat

ics.

6. C

loth

esM

ediu

m (Q

6.1)

Low

(Q6.

2)Pr

oper

ties

of m

ater

ials.

Med

ium

leng

th c

ontin

uous

te

xt fo

r inf

orm

atio

n.M

ultip

le c

hoice

ans

wer

s.

Acce

ss/re

triev

e in

form

atio

n fro

m te

xt.

Text

-bas

ed e

xtra

ct o

n fa

brics

an

d lo

okin

g at

usin

g th

em in

sc

ient

ific

test

ing.

Link

s to

scie

nce,

En

glish

/Wel

sh, d

esig

n an

d te

chno

logy

.

7. T

he G

rand

Ca

nyon

Low

(Q7.

1)Lo

w (Q

7.2)

Low

(Q7.

3)

Rock

s an

d ro

ck c

ycle

.Sh

ort c

ontin

uous

text

with

ph

otog

raph

to in

terp

ret.

Mul

tiple

cho

ice to

exp

lain

id

eas

and

expr

ess

view

poin

t.Te

xt a

nd d

iagr

am o

f Gra

nd

Cany

on a

nd ro

cks

whi

ch

cons

titut

e it.

Link

s to

scie

nce,

geo

grap

hy.

8. S

unsc

reen

sM

ediu

m (Q

8.1)

Low

(Q8.

2)

Med

ium

(Q8.

3)Hi

gh (Q

8.4)

Scie

ntifi

c m

etho

d –

test

ing

suns

cree

ns.

Med

ium

leng

th te

xt fo

r in

form

atio

n w

ith d

iagr

ams.

Acce

ss/in

terp

ret i

nfor

mat

ion

from

text

and

dia

gram

s, in

terp

ret d

ata

and

draw

co

nclu

sions

. M

ultip

le c

hoice

and

ope

n en

ded

long

er a

nsw

er

resp

onse

to ju

stify

opi

nion

s an

d id

eas.

Desc

riptio

n of

test

s pe

rform

ed o

n su

nscr

een

prot

ectio

n.Li

nks

to s

cienc

e, P

SE.

9. M

ary

Mon

tagu

Low

(Q9.

1)Lo

w (Q

9.2)

Med

ium

(Q9.

3)

Spre

ad a

nd p

reve

ntio

n of

di

seas

e.Sh

ort c

ontin

uous

text

for

info

rmat

ion.

Mul

tiple

cho

ice a

nd s

hort

writ

ten

resp

onse

to g

ive

reas

ons

for a

nsw

er.

New

spap

er a

rticle

on

vacc

inat

ions

.Li

nks

to s

cienc

e, P

SE.

10. A

cid R

ain

Med

ium

(Q10

.1)

Low

(Q10

.2)

High

(Q10

.3)

Envi

ronm

enta

l im

pact

of a

cid

rain

, pol

lutio

n.Sh

ort c

ontin

uous

text

for

info

rmat

ion

with

num

erica

l in

form

atio

n in

clude

d.

Mul

tiple

cho

ice to

re

ason

usin

g nu

mer

ical

info

rmat

ion.

Shor

t writ

ten

resp

onse

s re

quire

d to

exp

lain

idea

s.

Info

rmat

ion

on th

e ef

fect

of

acid

rain

on

stat

ues

in th

e Ac

ropo

lis.

Link

s to

scie

nce,

geo

grap

hy,

PSE.

11. P

hysic

al

Exe

rcise

Med

ium

(Q11

.1)

Low

(Q11

.2)

Med

ium

(Q11

.3)

Fact

ors

rela

ted

to h

uman

he

alth

.Sh

ort c

ontin

uous

text

.M

ultip

le c

hoice

, clo

sed

ques

tions

. Sh

ort w

ritte

n re

spon

se.

Effe

cts

of e

xerc

ise o

n th

e bo

dy.

Link

s to

scie

nce,

PSE

, PE.

48 A guide to using PISA as a learning context

Que

stio

nLe

vel o

f di

fficu

lty

Nat

iona

l cur

ricu

lum

sc

ienc

e co

ntex

tTe

xt t

ype

Que

stio

n ty

pes

Subj

ect

mat

ter

and

cr

oss-

curr

icul

ar li

nks

12. G

enet

ically

M

odifi

ed

Cro

ps

Low

(Q12

.1)

Low

(Q12

.2)

Gen

etic

engi

neer

ing.

Med

ium

leng

th c

ontin

uous

te

xt fo

r inf

orm

atio

n.Cl

osed

resp

onse

s as

mul

tiple

ch

oice

. Ac

cess

/retri

eve

info

rmat

ion

and

reas

onin

g.

Artic

le o

n G

M c

rops

to lo

ok

at v

aria

bles

invo

lved

in s

tudy

an

d al

so s

cient

ific

met

hod.

Link

s to

scie

nce,

PSE

, En

glish

/Wel

sh.

13. B

iodi

vers

ityTr

ial:

Low

(Q13

.1)

Low

(Q13

.2)

Food

web

s an

d ch

ains

an

d in

terd

epen

denc

e of

or

gani

sms.

Med

ium

leng

th c

ontin

uous

te

xt a

nd d

iagr

am (f

ood

web

) to

inte

rpre

t.

Mul

tiple

cho

ice.

Acce

ss/re

triev

e an

d in

terp

retin

g in

form

atio

n fro

m

diag

ram

.

New

spap

er a

rticle

on

ecos

yste

ms

with

food

cha

ins

and

web

s to

stu

dy.

Link

s to

scie

nce,

PSE

.

14. B

uses

Tria

l:Lo

w (Q

14.1

)M

ediu

m (Q

14.2

)

Forc

es, b

urni

ng fo

ssil

fuel

s, en

viro

nmen

tal i

ssue

s.M

ediu

m le

ngth

con

tinuo

us

text

with

dia

gram

to

inte

rpre

t.

Mul

tiple

cho

ice a

nd lo

nger

w

ritte

n op

en re

spon

se to

ju

stify

arg

umen

t.

Forc

es a

pplie

d to

bra

king

an

d lo

okin

g at

opi

nion

s on

po

llutio

n an

d fo

ssil

fuel

use

.Li

nks

to s

cienc

e, g

eogr

aphy

, PS

E.

15. C

limat

e

Cha

nge

Tria

l:M

ediu

m (Q

15.1

)Ev

iden

ce fo

r glo

bal w

arm

ing.

Long

er le

ngth

con

tinuo

us

text

with

bar

cha

rt to

in

terp

ret.

Long

er w

ritte

n op

en

resp

onse

, usin

g

grap

h/in

form

atio

n to

sup

port

argu

men

t.

Text

-bas

ed a

nd g

raph

ical

info

rmat

ion

look

ing

at

cont

ribut

ions

to g

loba

l w

arm

ing.

Link

s to

scie

nce,

PSE

, ge

ogra

phy.

16. F

lies

Tria

l:M

ediu

m (Q

16.1

)Hi

gh (Q

16.2

)

Inse

ctici

de u

se –

con

sider

ing

evid

ence

.M

ediu

m le

ngth

con

tinuo

us

text

.Lo

nger

writ

ten

open

re

spon

ses

to c

onsid

er th

e in

form

atio

n an

d ex

plai

n id

eas

and

just

ify.

Text

-bas

ed e

xtra

ct lo

okin

g at

evi

denc

e fo

r use

of

inse

ctici

des.

Link

s to

scie

nce,

PSE

, En

glish

/Wel

sh.

17. C

alf C

lone

sTr

ial:

Med

ium

(Q17

.1)

Low

(Q17

.2)

Gen

etics

and

clo

ning

.Lo

nger

leng

th c

ontin

uous

te

xt.

Shor

t writ

ten

resp

onse

and

m

ultip

le c

hoice

to e

xtra

ct

info

rmat

ion

from

text

.

New

spap

er a

rticle

abo

ut

cloni

ng o

f cow

s an

d m

etho

ds

used

.Li

nks

to s

cienc

e,

Engl

ish/W

elsh

, PSE

.

A guide to using PISA as a learning context 49

18. C

orn

Tria

l:Lo

w (Q

18.1

)Lo

w (Q

18.2

)M

ediu

m (Q

18.3

)

Com

bust

ion

of fu

els,

envi

ronm

enta

l iss

ues.

Long

er le

ngth

con

tinuo

us

text

for i

nfor

mat

ion

with

w

ord

equa

tion.

Mul

tiple

cho

ice re

spon

ses

to c

onsid

er in

form

atio

n an

d dr

aw c

onclu

sions

. Co

mpl

etio

n of

wor

d eq

uatio

n.

New

spap

er a

rticle

abo

ut

usin

g co

rn a

s a

fuel

.Li

nks

to s

cienc

e, P

SE,

geog

raph

y, En

glish

/Wel

sh.

19. F

it fo

r

d

rinki

ngTr

ial:

Low

(Q19

.1)

Low

(Q19

.2)

Low

(Q19

.3)

Low

(Q19

.4)

Low

(Q19

.5)

Dise

ase

prev

entio

n –

clean

ing

wat

er.

Shor

t con

tinuo

us te

xt a

nd

diag

ram

to in

terp

ret.

Shor

t writ

ten

resp

onse

s to

ex

plai

n an

d ju

stify

idea

s. M

ultip

le c

hoice

also

.

Diag

ram

sho

win

g ho

w w

ater

su

pplie

d to

hou

ses

is m

ade

fit fo

r drin

king

.Li

nks

to s

cienc

e, P

SE,

geog

raph

y.

20. T

ooth

Dec

ayTr

ial:

Low

(Q20

.1)

Med

ium

(Q20

.2)

Med

ium

(Q20

.3)

Dise

ase

prev

entio

n, b

acte

ria.

Shor

t con

tinuo

us te

xt w

ith

diag

ram

and

sca

tter g

raph

to

inte

rpre

t.

Mul

tiple

cho

ice re

spon

ses

to re

triev

e in

form

atio

n fro

m

text

and

dra

w c

onclu

sions

fro

m g

raph

and

text

.

Diag

ram

and

text

look

ing

at c

ause

and

effe

ct o

f too

th

deca

y.Li

nks

to s

cienc

e, P

SE,

mat

hem

atics

, geo

grap

hy.

21. H

ot W

ork

Tria

l:Lo

w (Q

21.1

)M

ediu

m (Q

21.2

)

Heat

loss

from

con

tain

ers.

Shor

t con

tinuo

us te

xt w

ith

num

erica

l inf

orm

atio

n to

in

terp

ret.

Mul

tiple

cho

ice re

spon

ses

to

infe

r and

reas

on.

Text

-bas

ed in

form

atio

n an

d da

ta lo

okin

g at

hea

ting

and

cool

ing

of m

ater

ials.

Link

s to

scie

nce,

des

ign

and

tech

nolo

gy, m

athe

mat

ics.

22. M

ouse

pox

Tria

l:Hi

gh (Q

22.1

)M

ediu

m (Q

22.2

)M

ediu

m (Q

22.3

)

Gen

etic

engi

neer

ing

and

viru

ses.

Shor

t con

tinuo

us te

xt.

Mul

tiple

cho

ice to

infe

r fro

m

text

and

exp

lore

opi

nion

s.Te

xt-b

ased

info

rmat

ion

on th

e sp

read

of d

iseas

e an

d ap

plica

tion

of g

enet

ic en

gine

erin

g.Li

nks

to s

cienc

e, P

SE,

Engl

ish/W

elsh

.

50 A guide to using PISA as a learning context

Que

stio

nLe

vel o

f di

fficu

lty

Nat

iona

l cur

ricu

lum

sc

ienc

e co

ntex

tTe

xt t

ype

Que

stio

n ty

pes

Subj

ect

mat

ter

and

cr

oss-

curr

icul

ar li

nks

23. S

tickl

ebac

k

B

ehav

iour

Tria

l:Hi

gh (Q

23.1

)Hi

gh (Q

23.2

)Hi

gh (Q

23.3

)

Scie

ntifi

c m

etho

d –

pl

anni

ng e

xper

imen

ts.

Long

er le

ngth

con

tinuo

us

text

with

dia

gram

s an

d ba

r ch

arts

to in

terp

ret.

Mul

tiple

cho

ice a

nd

long

er o

pen

writ

ten

resp

onse

to

test

infe

renc

e an

d in

terp

reta

tion

of re

sults

.

Text

-bas

ed a

nd

diag

ram

mat

ical/g

raph

ical

evid

ence

for b

ehav

iour

of

stick

leba

cks

in a

n aq

uariu

m

– lo

okin

g at

scie

ntifi

c pr

oces

s an

d m

etho

d.Li

nks

to s

cienc

e,

Engl

ish/W

elsh

, mat

hem

atics

.

24. T

obac

co

Sm

okin

gTr

ial:

Low

(Q24

.1)

Low

(Q24

.2)

Med

ium

(Q24

.3)

Med

ium

(Q24

.4)

Dise

ase

prev

entio

n, u

se o

f dr

ugs,

smok

ing

and

lung

s.M

ediu

m le

ngth

con

tinuo

us

text

.M

ultip

le c

hoice

resp

onse

s to

test

und

erst

andi

ng o

f sc

ient

ific

idea

s an

d pr

oces

s.

Look

ing

at c

ause

and

effe

ct

of s

mok

ing

on th

e bo

dy.

Link

s to

scie

nce,

PSE

, En

glish

/Wel

sh, P

E.

25. S

tarli

ght

Tria

l:Lo

w (Q

25.1

)M

ediu

m (Q

25.2

)

Astro

nom

y.Sh

ort c

ontin

uous

dat

a.M

ultip

le c

hoice

. St

atem

ents

to te

st

inte

rpre

tatio

n an

d in

fere

nce.

Look

ing

at o

bser

vatio

n of

th

e ni

ght s

ky a

nd u

se o

f te

lesc

opes

.Li

nks

to s

cienc

e.

26. U

ltras

ound

Tria

l:Lo

w (Q

26.1

)Lo

w (Q

26.2

)Lo

w (Q

26.3

)

EM s

pect

rum

, ultr

asou

nd.

Shor

t con

tinuo

us te

xt w

ith

phot

ogra

ph.

Long

er le

ngth

writ

ten

resp

onse

s re

quire

d to

ex

plai

n id

eas

and

show

un

ders

tand

ing.

M

ultip

le c

hoice

to te

st

infe

renc

e.

Disc

ussio

n as

to h

ow

ultra

soun

d is

used

in

preg

nanc

y, w

hat i

t sho

ws

and

its s

afet

y. Li

nks

to s

cienc

e,

mat

hem

atics

, PSE

.

27. L

ip G

loss

Tria

l:Lo

w (Q

27.1

)M

ediu

m (Q

27.2

)M

ediu

m (Q

27.3

)

Sepa

ratin

g te

chni

ques

, pr

oper

ties

of m

ater

ials.

Shor

t con

tinuo

us te

xt w

ith

num

erica

l dat

a in

form

of

recip

e to

inte

rpre

t.

Long

er le

ngth

writ

ten

resp

onse

and

mul

tiple

cho

ice

to te

st in

terp

reta

tion

of

info

rmat

ion,

infe

renc

e an

d un

ders

tand

ing.

Look

ing

at re

cipes

for

lip g

loss

and

pre

dict

ing

prop

ertie

s of

the

mat

eria

ls fo

r diff

eren

t rat

ios

and

prop

ortio

ns o

f ing

redi

ents

.Li

nks

to s

cienc

e,

mat

hem

atics

, des

ign

and

tech

nolo

gy.

A guide to using PISA as a learning context 51

28. E

volu

tion

Tria

l:M

ediu

m (Q

28.1

)Lo

w (Q

28.2

)M

ediu

m (Q

28.3

)

Evid

ence

for e

volu

tion.

Shor

t con

tinuo

us te

xt w

ith

tabu

late

d da

ta a

nd d

iagr

ams

to in

terp

ret.

Shor

t writ

ten

resp

onse

an

d m

ultip

le c

hoice

to te

st

acce

ss/re

triev

e in

form

atio

n fro

m te

xt, i

nfer

ence

and

in

terp

reta

tion.

Text

-bas

ed a

nd

diag

ram

mat

ical e

vide

nce

for

evol

utio

n of

the

hors

e.Li

nks

to s

cienc

e, P

SE,

Engl

ish/W

elsh

.

29. B

read

Dou

ghTr

ial:

Low

(Q29

.1)

Med

ium

(Q29

.2)

Med

ium

(Q29

.3)

Low

(Q29

.4)

Chem

ical r

eact

ions

, fe

rmen

tatio

n.Sh

ort c

ontin

uous

text

with

di

agra

ms

to in

terp

ret.

Mul

tiple

cho

ice re

spon

ses

to te

st a

cces

s/re

triev

e in

form

atio

n fro

m te

xt,

inte

rpre

tatio

n, u

nder

stan

ding

of

scie

ntifi

c m

etho

d an

d m

odel

ling.

Look

ing

at c

hem

ical

proc

esse

s in

mak

ing

brea

d.

Link

s to

scie

nce,

des

ign

and

tech

nolo

gy.

30. T

rans

it of

V

enus

Tria

l:Lo

w (Q

30.1

)M

ediu

m (Q

30.2

)Hi

gh (Q

30.3

)

Astro

nom

y, pl

anet

ary

mot

ion.

Shor

t con

tinuo

us te

xt w

ith

phot

ogra

ph.

Mul

tiple

cho

ice to

test

un

ders

tand

ing

of id

eas

and

appl

icatio

n.

Look

ing

at o

bser

vatio

ns o

f th

e tra

nsit

of V

enus

acr

oss

the

sun.

Link

s to

scie

nce,

En

glish

/Wel

sh.

31. H

ealth

Risk

?Tr

ial:

High

(Q31

.1)

High

(Q31

.2)

Envi

ronm

enta

l effe

cts

on

heal

th.

Med

ium

leng

th c

ontin

uous

te

xt.

Long

er le

ngth

writ

ten

resp

onse

s to

exp

lain

reas

ons

for i

deas

.

Cons

ider

ing

diffe

rent

vi

ewpo

ints

of t

he e

ffect

of

usin

g fe

rtilis

ers

in a

gricu

lture

on

hum

an h

ealth

Link

s to

scie

nce,

geo

grap

hy,

PSE,

Eng

lish/

Wel

sh.

32. C

atal

ytic

C

onve

rter

Tria

l:M

ediu

m (Q

32.1

)Hi

gh (Q

32.2

)M

ediu

m (Q

32.3

)

Chem

ical r

eact

ions

, ato

ms

and

mol

ecul

es, p

ollu

tant

s.Sh

ort c

ontin

uous

text

w

ith d

iagr

am c

onta

inin

g nu

mer

ical d

ata

to in

terp

ret.

Long

er le

ngth

writ

ten

resp

onse

s to

exp

lain

idea

s.Lo

okin

g at

how

cat

alyt

ic co

nver

ters

wor

k an

d de

scrib

ing

in te

rms

of

chem

ical r

eact

ions

.Li

nks

to s

cienc

e, P

SE,

geog

raph

y.

52 A guide to using PISA as a learning context

Que

stio

nLe

vel o

f di

fficu

lty

Nat

iona

l cur

ricu

lum

sc

ienc

e co

ntex

tTe

xt t

ype

Que

stio

n ty

pes

Subj

ect

mat

ter

and

cr

oss-

curr

icul

ar li

nks

33. M

ajor

Sur

gery

Tria

l:Lo

w (Q

33.1

)Lo

w (Q

33.2

)Lo

w (Q

33.3

)M

ediu

m (Q

33.4

)

Trea

tmen

t of d

iseas

e an

d dr

ug u

se.

Shor

t con

tinuo

us te

xt w

ith

phot

ogra

ph a

nd b

ar c

hart

to

inte

rpre

t.

Mos

tly m

ultip

le c

hoice

to te

st

inte

rpre

tatio

n an

d id

eas

and

draw

ing

conc

lusio

ns.

Shor

t writ

ten

resp

onse

also

to

set

out

exp

lana

tion.

Cons

ider

ing

proc

esse

s in

volv

ed in

sur

gery

–

ster

ilisa

tion

of in

stru

men

ts,

anae

sthe

tics,

drug

use

–

and

draw

ing

appr

opria

te

conc

lusio

ns.

Link

s to

scie

nce,

PSE

, En

glish

/Wel

sh.

34. W

ind

Farm

sTr

ial:

Med

ium

(Q34

.1)

High

(Q34

.2)

High

(Q34

.3)

Low

(Q34

.4)

Alte

rnat

ive

ener

gy re

sour

ces,

gene

ratin

g el

ectri

city.

Shor

t con

tinuo

us te

xt w

ith

line

grap

hs to

inte

rpre

t.M

ostly

mul

tiple

cho

ice to

an

alys

e an

d in

terp

ret g

raph

s an

d dr

aw c

onclu

sions

. Sh

ort w

ritte

n re

spon

se to

co

nsid

er a

dvan

tage

s an

d di

sadv

anta

ges.

Cons

ider

atio

n of

arg

umen

ts

in s

iting

a w

ind

farm

.Li

nks

to s

cienc

e, g

eogr

aphy

, PS

E, m

athe

mat

ics.