advancing assessment literacy data gathering iv: collecting and collating data

Advancing Assessment Literacy

Data Gathering IV:

Collecting and Collating Data

Advancing Assessment Literacy Modules: Data Gathering IV (February 2008) 2

Data Sources

• It is important to think about data sources as questions are being created. For some questions data may already be available, for others data may need to be collected.

• Data can be collated in a variety of ways.• Some of the most accessible data collation

and display tools are quality control tools such as run charts, scatter plot diagrams, and histograms.

3

Run Charts

• Run charts are simple tables to gather and display data in one area over time. For example, data might be gathered on quizzes, number of students on time, number of students who participated in the breakfast program each week day, etc.

• Run charts enable analysis of trends over time.

Assignment #

Num

ber

Cor

rect

1

2

3

4

5

6

7

8

1 2 3 4 5

4

Scatter Plot Diagrams

• Scatter plot diagrams are used to collect and display data on performance, attainment, or usage by number of subjects.

• This is useful for charting the progress or actions of a group of students.

Assignment #

Num

ber

Cor

rect

1

2

3

4

5

6

7

8

1 2 3 4 5

5

Histogram

• A histogram is a means of collecting and displaying detailed data regarding the number of people who have attained a certain level of achievement or the frequency of an action.

• A histogram provides a clear representation of the distribution of data across a population group.

Number of Questions Correct

Num

ber

of P

eopl

e

1

2

3

4

5

6

7

8

1 2 3 4 5


Four MajorCategories of Data

• Demographics– Descriptive information such

as enrollment, attendance, ethnicity, grade level, etc.

– Can disaggregate other data by demographic variables.

• Perceptions– Provides information regarding

what students, parents, staff, and community think about school programs and processes.

– This data is important because people act in congruence with what they believe.

• Student Learning– Describes outcomes in terms

of standardized test results, grade averages, etc.

• School Processes– What the system and teachers

are doing to get the results they are getting.

– Includes programs, assessments, instructional strategies, and classroom practices.

Bernhardt, V. L. (2004). Data analysis for continuous school improvement, 2nd Edition. Larchmont, NY: Eye on Education.


Selecting Data Sources

• On the supplied template, indicate which data sources would best match each of the four categories of data.

Demographics Perceptions

Student Learning

School Processes


Other Data Sources

• Questionnaires• Student Profiles• CAT3• Assessment for

Learning• PISA, TIMSS• Interviews• Surveys

• Portfolios• Classroom

Assessments• Archival Data

– Previous Marks– Course Selection

• Demographic Data

Bernhardt, V. L. (2004). Data analysis for continuous school improvement, 2nd Edition. Larchmont, NY: Eye on Education.

DemographicsEnrollment, Attendance, Drop out Rate, Gender, Grade

PerceptionsPerceptions of

Learning EnvironmentValues & Beliefs

AttitudesObservations

Student LearningStandardized Tests, Norm/Criterion Referenced Tests

Teacher Observations, Authentic Assessment

SchoolProcessesSchool Programs

AndProcesses


Triangulations

• Triangulation is the use of multiple data sources and types (quantitative & qualitative) to increase the validity of results.

• If two or more different data sources or types are giving the same information, it is more likely that what is being witnessed is true.

Wellman, B. & Lipton, L. (2004). Data-driven dialogue. Mira Via, LLC.Bernhardt, V. L. (2004). Data analysis for continuous school improvement,

2nd Edition. Larchmont, NY: Eye on Education.

The shaded area is the most valid.


Applying Data Collectionand Triangulation

• Using the goal statements and initial questions previously created, complete a more detailed analysis thinking about the four categories of data available – demographics, perceptions, student learning, and school processes.

• You will be furnished with a data intersections template.


Refining Questions

• Take the initial questions and write them in the questions column.

• In the left column, use the diagram to identify the intersections or triangulations implied within the question.– What other intersections would increase the specificity of

this question?– If necessary, rewrite the question to reflect the new

intersections or triangulations.– Including an “over time” element to a question expands the

data that might be accessed to answer the question.• For each question identify the existing data source available

or what tool will be required to collect it. • Create new questions using a variety of the intersections or

identified earlier.

13

Original: In what classes is representation best taught?

What data is available or needed to answer the question?

What data collection tools will be required?

Questions

Intersections

(Colour in and name the

intersection)

D

P

SL

SP

Adapted from: Bernhardt, V. L. (2004). Data analysis for continuous school improvement, 2nd Edition. Larchmont, NY: Eye on Education.

Goal Statement: Teachers will better actualize the representation strand of the curriculum.

Place the question inthe questions column

14

Original: In what classes is representation best taught? (Demographics (D) by Perception (P) )

What data is available or needed to answer

the question?


Questions

Intersections


intersection)

D

P

SL

SP



Identify the intersection within the question

15


(D by P)

Intersecting or triangulating with School Processes (SP) will add clarity


the question?


Questions

Intersections


intersection)

D

P

SL

SP



What other intersectionswould increase specificity?

16


(D by P)

Modified: In what classes is representation best taught and what methods are being used?

(D by P by SP)


the question?


Questions

Intersections


intersection)

D

P

SL

SP



Rewrite the question to reflect the new triangulation

17

Anecdotal data

AFL data

Lesson Plans

Anecdotal Data

AFL data

Original:

In what classes is representation best taught? (D by P)

Modified:

In what classes is representation best taught and what methods are being used? (D by P by SP)


the question?


Questions

Intersections


intersection)

D

P

SL

SP



Identify Data Sources


Refining and Finalizing Questions

• Complete the process with existing questions and create new ones.

• When finished, evaluate the quality of the questions then decide which should go forward and which should be abandoned.

• On the sheet provided, write the goal statement and the refined questions that your group has decided to keep.

• These questions will be used to gather data as more in-depth goal statements are created.


Reflection

• In what ways did this process refine the questions?

• Were any initial questions eliminated? Why?

advancing assessment literacy data gathering iv: collecting and collating data

Documents

display data

data analysis

questions data

distribution of data

data sourcesit

data sourceson

detailed data

different data sources