advancing assessment literacy data analysis iii: extending the assessment

Advancing Assessment Literacy

Data Analysis III:

Extending the Assessment

Advancing Assessment Literacy Modules: Data Analysis III (February 2008) 2

Revisiting Hypotheses

• You have been provided with the hypotheses generated following the initial examination of the data.

• This session is going to apply some of the principles from earlier modules to clarify the hypotheses created and expand the analysis to other data sources in order to confirm or deny what the data seems to be indicating.

3

Effect of the Scaleof the Data Source

From Understanding the numbers. Saskatchewan Learning

Definitive Indicative

Individual Classroom School Division Provincial National International

Student Evaluations System Evaluations

The analysis to this point has used provincial level data both in terms of general achievement and was followed by an item analysis which demonstrated how students performed relative to specific objectives at a school wide level.


Different Lenses

• What other types of data might be required to gain a clearer picture of how specific groups of students are doing? You may need to discuss how students might be grouped.

• How might this data shed a different light on what has been discovered so far?

• To do this, Bernhardt’s triangulation process will be used.


Applying Data Collectionand Triangulation

• Using the hypotheses created from the initial examination of the data, complete a more detailed analysis thinking about the four categories of data available – demographics, perceptions, student learning and school processes.

• You will be furnished with a data intersections template.


Refining Questions

• Take each hypothesis and write it at the top of a template.

• Write questions of the hypothesis in the questions column.

• In the left column, use the diagram to identify the intersections or triangulations implied within the question.

– What other intersections would increase the specificity of this question?

– If necessary, rewrite the question to reflect the new intersections or triangulations.

– Including an “over time” element to a question expands the data that might be accessed to answer the question.

• For each question identify the existing data source available or what tool will be required to collect it.

• Create new questions using a variety of the intersections identified earlier.

7

What types of strategies are used in reading?

What data is available or needed to answer

the question?

What data collection tools will be required?

Questions

Intersections

(Colour in and name the

intersection)

D

P

SL

SP

Adapted from: Bernhardt, V. L. (2004). Data analysis for continuous school improvement, 2nd Edition. Larchmont, NY: Eye on Education.

Hypothesis: Similar to process lists in math, providing detailed strategy lists for student reading will increase the number of strategies used and therefore increase comprehension.

Place the question in the questions column

8

Available Data:• Processes from the

curriculum guide.

• Processes outlined in the AFL documents.

What types of strategies are used in reading?

(School Processes (SP))


the question?


Questions

Intersections


intersection)

D

P

SL

SP


Identify the intersection within the question


9

Available Data:

• Student reading journal responses

Data Needed:

• Student opinions gathered via

questionnaire.

What reading strategies do students seem to favour? (School Processes by Perceptions)


the question?


Questions

Intersections


intersection)

D

P

SL

SP


What other intersections and data sources might enrich the analysis?


10

Available Data:• Student reading journal

responses

Data Needed:• Pre- and post-test data

for all reading strategies

Which reading strategies from the curriculum produce the greatest gains in comprehension for students? Which of those were favoured by students? (School Processes by Perceptions by Student Learning)


the question?


Questions

Intersections


intersection)

D

P

SL

SP




11

Available Data:• Student reading journal

responses by gender.

Data Needed:• Pre- and post test data

for favoured reading strategies by gender.

Which of the reading strategies from the curriculum produce the greatest gains in comprehension for students? Which of those were favoured? Are certain strategies more powerful at different grade levels? (School Processes by Perceptions by Student Learning by Demographics)


the question?


Questions

Intersections


intersection)

D

P

SL

SP





Refining and Finalizing Questions

• Complete the process for each question created.• When finished, evaluate the quality of the

questions then decide which should go forward and which should be abandoned.

• On the sheet provided, write the hypothesis and the refined questions that your group has decided to keep.

• These questions will be used to gather data as more in-depth goal statements are created.


Reflection

Individually, complete the following stems:

• Key concepts I have learned . . .

• Implications for me . . .

• Questions I have . . .

advancing assessment literacy data analysis iii: extending the assessment

Documents

available data

types of data

data sources

data intersections template

provincial level data

data collection tools

existing data source

assessment slide