seasons final curriculum analysis - the george …scale-up/documents/final cur… · ·...

Seasons: Curriculum Analysis

Faubert, Geer, Skelton, Slater, Pyke, and Lynch ©SCALE-uP 2003

1

Running head: SEASONS: CURRICULUM ANALYSIS

Using the Project 2061 Curriculum Analysis to Rate a Middle School Science

Curriculum Unit: LHS GEMS: The Real Reasons for Seasons. SCALE-uP Report No. 11

Scaling-up Curriculum for Achievement, Learning, and Equity Project

JULY, 2003

Report Prepared by: Ryan Faubert, The George Washington University

Jaqueline Geer, Montgomery County Public Schools Dr. Earl Skelton, The George Washington University

Dr. Timothy Slater, The University of Arizona Dr. Curtis Pyke, The George Washington University

Dr. Sharon Lynch, The George Washington University



2



3

Executive Summary This report presents the results of the Curriculum Analysis for a middle school astronomy/earth science curriculum unit, The Real Reasons for Seasons (Lawrence Hall of Science, Great Explorations in Math and Science, 2000). A team of researchers, scientists, and science educators used the Project 2061 Curriculum Analysis procedure to guide this work (AAAS, 2001b). The analysis work is part of the Scaling up Curriculum for Achievement, Learning, and Equity Project (SCALE-uP), a project in which middle school science curriculum units, such as Seasons, are being implemented in a large suburban school district to examine the effects of highly rated curriculum units on student achievement.

The Project 2061 Curriculum Analysis provides a comprehensive, research-based tool for determining the quality of instructional materials. We have used the Project 2061 Curriculum Analysis to systematically examine, rate, and report on the alignment and instructional qualities of Seasons. The Lawrence Hall of Science at the University of California Berkeley developed the unit for the Great Explorations in Math and Science (GEMS) curriculum program. The curriculum unit is designed for grades 5-8 and is comprised of 8 activities that can be completed in 30-90 minute sessions. The Curriculum Analysis of Seasons was completed by an Evaluation Team, which included a scientist, a science educator, a middle school science teacher, and a member of the SCALE-uP research team. Project 2061

was not involved in the analysis but the SCALE-uP staff attempted to follow the Curriculum Analysis procedure as closely as possible. Several of the SCALE-uP staff members have had experience with the process.



4

The Curriculum Analysis consists of two distinct phases: the Content Analysis and the Instructional Analysis. The Content Analysis entails reviewing the unit’s content to determine whether it aligns with specific learning goals selected by the SCALE-uP research staff. The Instructional Analysis determines how well the material’s instructional strategies support student learning of the criteria goals selected by Project 2061. The results of the Content Analysis found Seasons to align with the following five target ideas:

1) Like all planets and stars, the Earth is approximately spherical in shape. 2) The rotation of the Earth on its axis every 24 hours produces the night-and-day cycle 3) Planets move around the Sun in nearly circular orbits 4) Because the Earth turns daily on an axis that is tilted relative to the plane of the

Earth’s yearly orbit around the Sun, sunlight falls more intensely on different parts of the Earth during the year

5) The difference in heating of the Earth’s surface produces the planet’s seasons and weather patterns (American Association for the Advancement of Science, 1993, p. 64 and 68-69).

Seasons makes strong explicit connections among these target ideas, but less explicit connections between the target ideas and the related and prerequisite ideas. The unit also develops an evidence-based argument for these target ideas. Students are able to model their experiences and validate their everyday observations. They are also given a chance to reconsider their observations and the evidence given for the main ideas in the unit. There is a natural progression of the ideas in Seasons from more elementary to more complex. This allows students to arrive at the scientifically correct conceptual knowledge within each activity and link together each concept in order to understand the reasons for seasons. This unit is grade-appropriate and does a very good job of limiting material that is too advanced for middle school students. In addition, nearly all of Seasons has accurate information, containing few errors or misleading statements. The Instructional Analysis for Seasons reveals relative strengths in the areas of providing a sense of purpose, engaging students with relevant phenomena, developing and using scientific ideas, and enhancing the science learning environment. The process revealed relative weaknesses in the areas of taking account of student ideas; promoting student thinking about phenomena, experiences, and knowledge; and, assessing progress. The technical report, “Using the Project 2061 Curriculum Analysis to Rate a Middle School Science Curriculum Unit: LHS GEMS: The Real Reasons for Seasons,” presents the results of the Curriculum Analysis conducted for the SCALE-uP research study.



5

Curriculum Analysis Background

The need to do a curriculum analysis of The Real Reasons for Seasons, (LHS GEMS, 2000) was precipitated by receiving a NSF/IERI research grant entitled, Scaling Up Highly Rated Science Curricula in Diverse Student Populations: Using Evidence to Close Achievement Gaps (Lynch, Kuipers, Pyke, & Szesze, 2002). A goal of this research was to locate “reform” or “standards-based” middle school science curriculum units, and scale them up in a large, diverse, public school system, over six years. It was understood that the units would likely have in common a well-defined set of instructional attributes that would lead to superior outcomes when compared to “standard” curriculum materials already in use in the school system. However, this study also seeks to learn if the scaled-up units might result in better outcomes for various subpopulations of students (defined by gender, socioeconomic status, ethnicity, and status as a learner of English or as having an identified disability).

In the preliminary phases of our study, we implemented an eighth grade curriculum unit, Chemistry That Applies (State of Michigan, 1993). Students in five highly diverse middle schools had statistically significant higher outcome scores than their counterparts in five comparison schools who studied the same concepts with the standard curriculum materials (Lynch, Kuipers, Pyke, & Szesze, 2003). Chemistry That Applies (CTA) had received a high rating according to the Project 2061 Curriculum Analysis (AAAS, 2001b), and was one of the few existing middle school science curriculum units to have done so. This process of rating a curriculum unit requires that it focuses on particular standards/benchmarks and that it has instructional characteristics well defined by the analysis procedures (Kesidou & Roseman, 2003).

Thus, as the Scaling up Curriculum for Achievement, Learning, and Equity Project (SCALE-uP) research proceeded, we sought to identify two additional curriculum units that also would earn a high rating on the Project 2061 criteria, (one suitable for sixth grade science and one for seventh). By “suitable,” we meant that the new units had to align with the local school district’s curriculum framework, be cost and time efficient, and have the potential for rating highly when subjected to the Project 2061 Curriculum Analysis. We examined a number of “reform-oriented” middle school science curriculum units. Many of these had their provenance with NSF’s curriculum development efforts. In the final analysis, the research/science education staffs at the local district and university were able to find two curriculum units that seemed suitable: ARIES: Exploring Motion and Forces (Harvard-Smithsonian Center for Astrophysics, 2001) and Great Explorations in Math and Science (GEMS): The Real Reasons for Seasons (Lawrence Hall of Science, GEMS, 2000). It was our hope that both of these units would rate highly on the rigorous Project 2061 Curriculum Analysis. The intent of this report is to present the results of the analysis for the GEMS unit, The Real Reasons for Seasons.

The Curriculum Unit

The Real Reasons for Seasons (LHS GEMS, 2000) was developed at the Lawrence Hall

of Science (LHS) of the University of California at Berkeley. It is an astronomy/earth science curriculum unit designed for grades six through eight that consists of eight activities focusing on Sun-Earth connections. Each activity requires about 30-90 minutes of class time. Seasons’ activities are hands-on in their approach to student learning and are written with instructions and background information that allow teachers without much background in science to implement



6

the unit. The Seasons Teacher Guide includes a manual, a CD-ROM that provides a variety of supporting resources, and a student Lab Book that can be duplicated for students to use.

Overview of the Project 2061 Curriculum Analysis

The Curriculum Analysis was developed by the American Association for the

Advancement of Science (AAAS), Project 2061. The analysis consists of two distinct phases. The first phase, called the Content Analysis, requires reviewers to look closely at the curriculum material to determine whether its content aligns with specific learning goals. The content should address the substance of the goal (such as a benchmark, standard, or local school district’s indicators) rather than be a superficial topic match (Kesidou & Roseman, 2002). If there is a good content match, then the Curriculum Analysis can enter the second phase, called the Instructional Analysis (see Appendix A), which determines how well the material’s instructional strategies support student learning of the selected learning goal. Finally, both the Content and Instructional Analyses are finalized in a report (this document for The Real Reasons for Seasons), which includes the development of a criterion-by-criterion profile that shows how the learning goal is met by the unit’s instructional attributes (Kesidou & Roseman, 2002). The resulting profile allows for comparison of various curriculum units rated by the Project 2061 Curriculum Analysis. (It should be noted that the Project 2061 Curriculum Analysis does not include the universe of all desirable characteristics of curriculum materials, but rather a number of attributes with rationales supported by research literature. See Kesidou & Roseman, 2002; Holliday, 2003; and Kesidou & Roseman, 2003 for a more extensive discussion.)

The Rating Process for The Real Reasons for Seasons

The Curriculum Analysis for The Real Reasons for Seasons unit was done during the

spring of 2003. Project 2061 was not involved with the analysis. However, the SCALE-uP staff followed the Project 2061 Curriculum Analysis procedures closely. Curtis Pyke, Assistant Professor of Secondary Education at George Washington University (GWU), has been trained by Project 2061 and participated in the Project 2061 Algebra textbook analysis. Sharon Lynch, Professor of Secondary Education at GWU, was on the Project 2061 High School Biology Textbook Advisory Board and has used a modified version of the Curriculum Analysis in her classes at GWU for six consecutive years. The research team attempted to be as faithful to the Project 2061 Curriculum Analysis as possible, but this analysis should not be construed as sponsored by Project 2061.

The initial groundwork for the Content Analysis (the location of various “sightings” of evidence within the curriculum unit for the target benchmark) was completed by the GWU SCALE-uP research staff. Then an outside evaluation team of science/science education specialists traveled to the GWU campus to analyze Seasons, using the Project 2061 procedures (see Appendix B for a list of evaluation team members). The Instructional Analysis took place May 14-17, 2003. The evaluation team worked on the Instructional Analysis for about eight hours per day, for four consecutive days. The first day was mostly devoted to familiarizing the team with the goals of the SCALE-uP research and teaching the Curriculum Analysis process. The team also studied the Seasons curriculum materials, and reviewed a map of the learning goals for the curriculum unit. In the next two days, the evaluation team broke into two pairs of partners who worked methodically through the Project 2061 Instructional Analysis Categories



7

(Appendix A). On the end of the third day, the two pairs of partners met to reconcile the ratings they had made for the unit. The team used the final day primarily for writing its report on The Real Reasons for Seasons Instructional Analysis. Finally, the GWU research team proofread and edited the evaluation team’s work, and added supplementary information to create this final report. This report presents a description of the Content Analysis process and reasoning, and a criterion-by-criterion profile of the Instructional Analysis for Seasons.

Content Analysis

The goal of Content Analysis (of the Project 2061 Curriculum Analysis) is to identify a

target benchmark or standard (or a discrete set of targeted benchmarks/standards) aligned with the curriculum unit. The Project 2061 Content Analysis has three steps: 1) benchmark/standard selection; 2) the sighting phase (determining the locations or “sightings” of activities in the curriculum material that address specific benchmarks/standards); and, 3) determining quality of alignment (consisting of five components: alignment, building a case, coherence, beyond literacy, and accuracy) (AAAS, 1999). Each step is described in more detail below.

Benchmark Selection

This Content Analysis follows the procedures used by Project 2061 in the Middle Grades

Science Textbooks evaluation (AAAS, 1999). The Project 2061 Content Analysis focuses on a set of ideas that comprise either an entire Project 2061 benchmark, (see Benchmarks for Science Literacy, 1993), selected parts of related benchmarks, or statements from the supporting text in Benchmarks for Science Literacy (1993). These benchmarks or standards can be considered as either a single “target idea” or an “idea set,” composed of related science concepts that students need to understand to learn the target idea/idea set. The development of a Target Concept Map (Figure 1) was essential to determine the alignment of the Seasons content to the target ideas. The map was constructed using the convention established in Atlas of Science Literacy (AAAS, 2001a); a collection of strand maps that correlate to particular Benchmark ideas. Project 2061 has defined a strand map as “…identifiable concepts or stories that are developed in groups of benchmarks across different grade levels”(AAAS, 2001a, p. 6). Such a map is intended to show how students’ understanding of the ideas and skills that lead to literacy in science develop over time and help to display the explicit connections that exist among the individual benchmarks.

The particular target ideas that serve as the focus of this analysis are presented in Figure 1. This map was initially constructed by the SCALE-uP research team and then refined and approved in a consensus process by the evaluation team of scientists, science educators, and teachers. The team agreed that Figure 1’s ideas are essential for mastery of the concept that explains why there are seasons. Figure 1 shows the five target ideas, as well as prerequisite ideas, related ideas, and more advanced ideas for the concept of seasons. The map is presented by grade bands (K-2, 3-5, 6-8, and 9-12) and arranged logically such that connections between benchmarks/ideas means that one “‘contributes to achieving’” the other (AAAS, 2001a, p. 7).



8

The difference in heating of the earth's surface produces the planet's seasons and

weather patterns (4B, 6-8, #4).

Because the earth turns daily on an axis that is tilted relative to the plane of the earth's yearly

orbit around the sun, sunlight falls more intensely on different parts of the earth during

the year (4B, 6-8, #4).

...planets...move around the sun in nearly circular orbits (4A, 6-8, #3).

(Part of much larger benchmark, see Benchmarks, p. 64)

The earth is one of several planets that orbit the sun

and the moon orbits around the earth (4A, 3-5, #4).

-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

... To people on earth, this turning of the planet makes it seem as though the sun, moon, planets,

and stars are orbiting the earth once a day

(4B, 3-5, #2).

The sun warms the land, air and water (4E, K-2, #1)

Some events in nature have a repeating pattern. The weather changes from day to day, but

things such as temperature tend to be high, low or medium in the same months every year (4B, K-2,

#1)

Like all planets and stars, the earth is

approximately spherical in shape

(4B, 3-5, #2).

The rotation of the earth on its axis every 24 hour.produces the

night-and-day cycles... (4B, 3-5, #2).

--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Seasons Target Ideas

K-2

3-5

6-8

Things that give off light often also give off heat. Heat is

produced by mechanical and electrical machines, and any time one thing rubs against something

else(4E, 3-5, #1).

Things change in steady, repetitive, or irregular ways--or sometimes in more than

one way at the same time. Often the best way to tell which kinds of change are happening is to make a table or graph of measurements.

(11C, 3-5, #2).

Cycles, such as the seasons or body temperature, can be described by their cycle length or

frequency, what their highest and lowest values are, and when these values occur. Different cycles range from many thousands of years

down to less than a billionth of a second (11C, 6-8, #6).

Weather (in the short run) and climate (in the long run) involve the transfer of energy in and out of the atmosphere. Solar radiation heats the land masses, oceans, and air. Transfer of heat energy at the boundaries between the atmosphere, the land masses and oceans results in layers of different temperatures and densities in both the ocean and atmosphere. The action of gravitational force on regions of different densities causes them to rise or fall-- and such circulation, influenced by the rotation of the earth, produces winds and ocean currents (4B, 9-12, #2).

-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Potential 9-12 Benchmark

Cycles of Planets and StarsTemperature Change Sun, Light, and Heat

Heat can be transmitted through materials by the collision of atoms or across space by radiation. If the

material is fluid, currents will be set up in it that aid the transfer of heat

(4E, 6-8, #3).

Some changes are so slow or so fast that they are hard to see (11C, K-2,

#4).

Geometric figures, number sequences, graphs, diagrams, sketches, number lines, maps, and

stories can be used to represent objects, events, and processes in the real world, although such

representations can never be exact in every detail. (11B, 3-5, #2).

Pre-requisite Ideas

More Advanced

Ideas

Related Ideas

1 2

34

5

Figure 1. Map showing the five target ideas, prerequisite ideas, related ideas, and more advanced ideas by grade level.

The Sighting Phase

After constructing the map, the next step in the Content Analysis was to locate areas

within the curriculum material that correspond reasonably well to the idea set displayed in Figure 1. This was accomplished by a meticulous, page-by-page survey of the unit’s activities, investigations, reading selections, and other learning opportunities that address the target ideas within the idea set. This process is referred to as “sighting,” and included the specification of page number and quotes from Seasons that link to a target idea. Two SCALE-uP research



9

assistants eventually arrived at a complete set of sightings. Appendix C presents the sightings where The Real Reasons for Seasons explicitly addresses each target idea.

Quality of Alignment

Quality of alignment explores and describes (a) how the curriculum materials align to each target idea, (b) the evidence provided by the material that builds a case for the ideas, (c) the connections the material makes among ideas in the idea set and to other ideas, (c) level of sophistication (that is, to determine if the coverage is appropriately aligned with the literacy expectations of the target grade level), and (e) the accuracy of the materials related to the ideas. Each of these aspects is described in the following sections. ALIGNMENT

The alignment part of the Content Analysis requires evaluators to determine if the curriculum material’s content aligns with the specific target ideas that have been selected for the analysis. Appendix C shows how the Seasons’ “sightings” explicitly address each target idea. The following is a summary of how the Seasons’ curriculum materials align with each target idea. Description and Rationale for Target Ideas

Target ideas 1 and 2 are part of benchmark 4B, 3-5, #2 (AAAS, 1993) (see Figure 1). This benchmark was divided into two target ideas because the ideas appear to be separate, yet still closely related concepts. Specific explorations in the curriculum address distinct parts of the benchmark, but not others. The entire AAAS (1993) benchmark is:

Like all planets and stars the Earth is approximately spherical in shape. The rotation of the Earth on its axis every 24 hours produces the night- and-day cycle. To the people on Earth, this turning of the planet makes it seem as though the Sun, moon, planets, and stars are orbiting the Earth once a day (p. 68).

Target ideas 4 and 5 are part of a related benchmark at a higher grade level (4B, 6-8, #4)

(AAAS, 1993) (see Figure 1), which is: Because the Earth turns daily on an axis that is tilted relative to the plane of the Earth’s yearly orbit around the Sun, sunlight falls more intensely on different parts of the Earth during the year. The difference in heating the Earth’s surface produces the planet’s seasons and weather patterns (p. 69).

Target idea 3 is part of a different, complex benchmark (4A, 6-8, #3) (AAAS, 1993) (see

Figure 1). We have isolated the idea of “orbit” from other ideas in this benchmark. The remaining ideas in the benchmark are not present in Seasons. This AAAS (1993) benchmark reads:

Nine planets of very different size, composition, and surface features move around the Sun in nearly circular orbits. Some planets have a great variety of moons and even flat rings of rock and ice particles orbiting around them. Some of these planets and moons show evidence of geologic activity. The



10

Earth is orbited by one moon, many artificial satellites, and debris (p. 64). All of these target ideas are integrated in an important component of the Seasons unit, the

Sun-Earth Survey (p.26 of the Seasons Teacher Guide). In this survey, students are asked to respond to a series of questions about the Sun-Earth system, and explore how other people and scientists view these target ideas. The introduction to the Seasons Teacher Guide begins by establishing these ideas as key components to understanding the real reasons for the seasons. Ideas 1-3 are explicitly identified as key ideas contained within the Seasons unit; however, ideas 4 and 5 are not explicitly identified as key ideas. Students are expected to understand Ideas 4 and 5 after participating in the unit.

Alignment to Target Ideas Target idea 1. Like all planets and stars, the Earth is approximately spherical in shape

(4B, 3-5, #2).

There is content alignment to target idea 1 in Seasons. In Activity 2 of the Seasons Teacher Guide (p.22), the overview states, “Before they can understand what causes seasons, students need to know that the Earth is spherical, …” Activity 3 takes students on an imaginary trip to the Sun with a tangential departure from the Earth’s surface as opposed to a perpendicular departure. Students find out that as their distance from the Earth increases, the altitude from Earth is not equal to the distance they have traveled. Students are to infer from this relationship that the Earth’s surface is curved and not flat. The overview states:

In the first part of the journey, as they speed along in a straight line over the United States, their altitude above the Earth increases, due to the spherical shape of the Earth. A main goal of the activity is for students to reinforce their conception of the Earth’s spherical shape, which is key to understanding the cause of the seasons (p. 29).

A slide show on the accompanying Seasons CD-ROM provides a visual representation of the Earth as a sphere. The section “Reflecting on the Trip to the Sun” (pp. 45-6) provides an in-class activity and diagram to show how students might travel to the Sun.

The Earth as a sphere, Idea 1, is referred to at various times (pp. 46 and 81) throughout the remaining Seasons’ activities and is prerequisite to understanding seasons.

Target idea 2. The rotation of the Earth on its axis every 24 hours produces the night-and-day cycle…(4B, 3-5, #2). There is content alignment to target idea 2 in Seasons. In Activity 2 of the Seasons Teacher Guide (p.22), the overview states, “Before they can understand what causes seasons, students need to know that the Earth… spins daily on its axis (rotates)… They also need to know what causes day and night.” The “Night and Day on ‘Mount Nose’” activity asks students to imagine their nose as a mountain and that a person lives on the tip of their nose. Students learn that as they turn relative to a stationary light bulb (Sun), the Sun would rise and set, causing day and night. The idea of Earth turning on an axis is reinforced in this activity. These ideas are extended throughout the next series of activities and are prerequisite to more difficult concepts regarding seasons that will come later in the unit.



11

Activity 6 strengthens students’ understanding of the day night cycle and states: By graphing the number of daylight hours per day in cities around the world, students find a very symmetrical pattern of daylight hours that is exactly opposite for the Southern and Northern Hemispheres. They also discover the meaning of the equinoxes, as they find day and night hours are equal in September and March everywhere on Earth (p. 65). The idea of axis and rotation in this activity is prerequisite to the idea of tilt. This activity

uses an inductive model of teaching that leads students into the idea of tilt as a critical component for explaining the latitudinal variations in hours of daylight, and ultimately, the seasons. Additional Seasons’ activities focus on critical concepts that help explain seasons, in combination with the Earth’s rotation on its axis.

Target idea 3. …planets…move around the Sun in nearly circular orbits…(4A, 6-8, #3). There is content alignment to target idea 3 in Seasons. The Seasons Teacher Guide begins by addressing common misconceptions students may have relating to seasons. In particular, it addresses the Earth’s orbit as an ellipse, noting that the orbit is so nearly circular that the Earth-Sun distance remains very nearly constant throughout the year. This begins to dispel the common student misconception that seasons are caused by variation in Sun-Earth distance. In Activity 2 of the Seasons Teacher Guide (p. 22), the overview states, “Before they can understand what causes seasons, students need to know that the Earth… orbits the Sun (revolves).” The Sun-Earth Survey (p. 26) asks in Question #1, “Which of the four drawings do you think best shows the shape of the Earth’s orbit around the Sun?” Students are asked repeatedly to return to their initial responses on the Sun-Earth Survey and revisit them, based upon their new experiences in the Seasons unit. Activity 4, “What Shape is the Earth’s Orbit?” (pp. 49-56), is an exercise designed to show the shape of the Earth’s orbit around the Sun, eliminate any misconceptions about its shape, and also reinforce the idea that orbit is a cycle completed in one calendar year. Orbit is also used to explain the effects of tilt in Activity 7 and contributes to understanding how different locations on the Earth’s surface experience different degrees of temperature and sunlight intensity at different times of the year in Activities 5 and 8.

Target idea 4. Because the Earth turns daily on an axis that is tilted relative to the plane of the Earth’s yearly orbit around the Sun, sunlight falls more intensely on different parts of the Earth during the year (4B, 6-8, #4). There is content alignment to target idea 4 in Seasons. This target idea is represented throughout the Seasons Teacher Guide. Ideas 1-3 dealt with axis, rotation, and orbit as they relate to tilt. Activities 7 and 8 are the culmination of those ideas and require students to understand how the Earth’s tilt affects the amount of sunlight that strikes its surface at various latitudes. The overview in Activity 7 states: Having explored the distance to the Sun, the shape of the Earth’s orbit, and the differing temperatures and day lengths around the world, your class is ready to gain a deeper and more scientifically accurate and



12

complete understanding of what causes the seasons. …students create a model that shows how the tilt of the Earth’s spin axis causes seasons (p. 73). Activity 7 uses a polystyrene globe on a pencil to model the Earth revolving around a

light source. Students mark points on the globe indicating different latitudes. Students tilt the globe and keep it pointed toward a reference point in the classroom (representing the North Star) as their bodies revolve around a bright light bulb in a darkened room. As they do this, they are able to see not only the difference in night and day but also the difference in intensity of light for different latitudes on Earth. Students are asked to discuss question #2 on page 78, “Do you think a planet whose axis was not tilted at all would have seasons?”, and #5 “If the tilt doesn’t make much of a difference in our distance from the Sun, why does it get hotter in summer?” Discussion should elicit student responses that show their understanding of how the angle of sunlight makes light more concentrated in summer.

The overview for Activity 8 states:

Students have now seen how the tilt of the Earth causes day length to change with the seasons. In this closing activity and discussion, they are able to see more clearly how the Earth’s tilt also changes the angle at which the sunlight hits the ground. In winter, rays of sunlight strike the ground at a slant and are less concentrated than the more perpendicular rays of the summer months. Using a “Sun Angle Analyzer” students explore and model this aspect (p. 81). The closing discussion brings everything together in this unit, focusing on the ideas of tilt and the intensity of light as the two biggest factors that cause seasons.

Target idea 5. The difference in heating of the Earth’s surface produces the planet’s seasons and weather patterns (4B, 6-8, #4). There is content alignment to target idea 5 in Seasons. The ideas of tilt and sunlight intensity contribute to understanding the differential heating of the Earth’s surface. Initially, the unit treats this as a separate idea, but later integrates it into the overall scheme of understanding seasons in Activity 8, the last lesson in the unit. In Activity 5, students analyze temperatures taken around the world from data gathered by the GLOBE project (a network of students in schools around the world who take weather readings and record them on the GLOBE internet database). In the Seasons Teacher Guide, students examine GLOBE data provided in the CD-ROM (indicated in the margin on p. 57) and account for the variation of average monthly temperatures in different latitudes throughout the world. “By graphing temperatures around the world, they can find out if seasons differ in various locations” (p. 61 #5). In Activity 8, “teachers help students understand how the idea of sunlight angle is related to increased sunlight concentration, and how a more concentrated light will produce more heat” (p. 86 #4).

In “Behind the Scenes” located in the Seasons Teacher Guide (pp. 89-92), background information and correct responses to the Sun-Earth Survey (p. 26) are given in more detail. Each of these five key ideas is represented here and discussed in relation to how the five ideas taken together contribute to an understanding of the changing seasons. Building a Case



13

To build a case, the curriculum material is examined to determine if it develops an evidence-based argument in support of the target ideas, including whether the case presented is valid, comprehensible, and convincing. The Seasons Teacher Guide develops an evidence-based argument in support of the target ideas. The unit starts by addressing misconceptions students may have about seasons. These identified misconceptions are based on the relevant research done in the field of astronomy and earth science. The Seasons Teacher Guide includes references for teachers in the margins and introduction sections of certain activities (i.e. Children’s Ideas in Science and A Private Universe).

Students’ experiences with seasons are elicited explicitly in Activity 1 of the Seasons Teacher Guide (pp.17-21) and further Seasons’ activities are directed toward correcting common problems students experience when learning about the seasons. Likely problems for students contained in the Sun-Earth Survey (Activity 2, p.26) are constantly revisited, and students are expected to gradually change their initial ideas as they learn from the Seasons unit. This process is intended to be illuminating to students. Questions throughout the Seasons Teacher Guide provide students with a chance to reconsider the evidence given for the main ideas in the unit and represented by Figure 1. Multiple questions throughout each activity give students a chance to reconsider their initial conceptions about seasons or validate the correct responses given in the Sun-Earth Survey.

In the classroom, students are able to model their experiences and validate their everyday observations. Working with physical models and concrete data will allow students to increasingly build an evidence-based argument in support of the target ideas in Figure 1. However, due to the enormous scale in which the students must observe these phenomena related to seasons, it is unclear whether there are enough contexts contained in the Seasons Teacher Guide to make each concept comprehensible and convincing to students. Extensions for lessons are suggested at the end of each activity, nevertheless each activity sequence only allows for a single context from which the students will work. A single context seems insufficient to master a particular concept or aspect of seasonal variation. If students are unable to understand the point of a particular activity, then it is likely subsequent activities may not be successful in helping students to construct increasingly complex ideas about the seasons. The Seasons Teacher Guide relies on teachers to “go further” with related lessons suggested within the Seasons unit. Coherence

In examining the coherence of curriculum materials, one looks for connections that the

curriculum material makes (1) among the target ideas, (2) between the target ideas and their prerequisites, and (3) between the target ideas and other, related ideas.

The Seasons Teacher Guide is coherent, based on this criterion. There is a natural progression of the ideas in the Seasons Teacher Guide from more elementary to more complex. Prerequisite ideas are provided first (i.e., scale, the Earth as a sphere). The shape of the Earth’s orbit is then developed along with rotation as it relates to the day and night cycle and the differential heating of the Earth’s surface (Activities 4-6). The tilt (Activity 7) relative to the



14

plane of the Earth’s orbit is the key to understanding the causes of phenomena addressed in previous activities. For example, tilt helps to explain the differential heating of the Earth’s surface and changes in day length at different latitudes on the Earth’s surface. Activity 8 in the Seasons Teacher Guide shows how differences in heat can also be explained by both the tilt of the Earth and the angle at which sunlight hits the Earth, resulting in more or less intense sunlight and the differential in heating of the Earth’s surface. The unit makes strong connections among the targeted ideas. In Activity 6 the number of daylight hours around the world is examined. Question #10 of the Seasons Teacher Guide reads, “Tell the class the next activity will make clear for them why the day length changes with the seasons the way it does” (p.70). Activity 7 addresses the link between rotation axis, orbit, and Earth as a sphere to the idea of tilt. Question #5 of the Seasons Teacher Guide reads:

If the tilt doesn’t make much difference in our distance from the Sun, why does it get hotter in summer? [There are more hours of daylight. It’s the angle of the sunlight that makes the light more concentrated in summer, but don’t reveal this yet unless a student mentions it.] Tell students that in the next session, the class will explore that question (p. 78).

Activity 7 provides a conceptual bridge to Activity 8, which goes further in depth to show how the tilt of the Earth spinning on its axis causes seasons.

Seasons’ final activity links the Earth’s tilt to the changes in angle at which the sunlight hits the ground (intensity and sunlight concentration). This Activity also connects all the key ideas for a final discussion: The closing discussion brings home the fact that the Earth

is a spinning globe whose axis tilts with respect to its orbit around the Sun, and this gives rise to: (a) a varying number of daylight hours in different seasons, and (b) variations in concentration of sunlight on the ground related to the angle the light strikes the ground. These are the main factors that cause the seasons and since they stem from the relation between the Sun and the Earth, they are considered a “Sun-Earth connection.” (p. 81)

In Seasons, prerequisite ideas are linked to the targeted ideas (see Figure 1). However, the links seem inconsistent and inadequate. The teacher is rarely asked to probe students’ prerequisite knowledge, and the students’ essential terms and skills needed for understanding the content of the activities. Although these skills and terms are not well defined in the unit, they are necessary for the understanding of the target ideas in the activity sequence. An exception is Activity 3, where the concept of scale is learned through the activity “Trip to the Sun” and linked to the prerequisite idea of Earth as a sphere. This allows students to visually experience the Sun and Earth as spherical bodies and their relative sizes based on distance traveled away from Earth. Other prerequisite and related ideas within this curriculum unit are not investigated during formal activities. The “Going Further” section in each activity gives a preliminary introduction to related ideas, but does not develop them to a great extent. There are data sheets and related information for the “Going Further” activities on Seasonal Changes contained in the Seasons Teacher Guide. This information is optional, and the Seasons Teacher Guide does not adequately tell teachers how to use this information.



15

Beyond Literacy

When determining if curriculum materials are beyond literacy, evaluators look for any information presented in the curriculum material that is more advanced than the set of target ideas. They are particularly wary of “beyond literacy” information that may interrupt the presentation of the grade-appropriate information in the unit. The Seasons Teacher Guide is grade-appropriate and does a very good job of limiting material that is too advanced for middle school students. Although not directly used in this unit, “Going Further” on page 70, examines seasonal changes in the Sun’s apparent position in the sky. Question 2 asks students to explore and graph azimuth and elevation angle of the Sun relative to a particular position on Earth. Azimuth and altitude justify angle of incidence, which contributes to the Sun’s ray’s intensity and concentration. Activity 4 introduces the term foci with no explanation in the Seasons Teacher Guide of how to explain the concept to students. The term foci can be simplified for students, however in this case it has been largely ignored.

In addition, on page 11, it states, “The Earth’s axis is tilted, there are more hours of daylight, and sunlight strikes the Northern Hemisphere at a high angle (more perpendicular), making the sunlight more concentrated resulting in more heating.” Noticeably absent from the Seasons Teacher Guide is the idea that energy is transferred from the Sun to the Earth. There is no explanation detailing how the Sun’s rays are transmitted to the Earth through the Earth’s atmosphere to create heat. This is a critical component of astronomy related to seasons, but was left out presumably because of its advanced nature. Accuracy

In judging the accuracy of curriculum materials, evaluators locate any information presented in the curriculum material that contains errors, misleading statements, or statements that may reinforce commonly held student misconceptions.

Nearly all of the Seasons Teacher Guide has accurate information. There are, however, a

few exceptions. For example, in Activity 4 “What Shape is Earth’s Orbit?” students are instructed to use a length of string and two push-pins (representing the foci) to generate three different ellipses representing the orbit of Earth, Pluto, and a comet. The Seasons Teacher Guide advises teachers that students should only look at the shape of the ellipses produced. If done properly, students will see that Earth’s orbit is very nearly circular while the comet’s orbit and Pluto’s orbit are more elongated ellipses. Unfortunately, because the string the students are using is the same size for each ellipse, Earth’s orbit looks much larger in size compared to Pluto or a comet. Students may know that Pluto and comets are further away from the Sun and as such their orbits should be larger than Earth’s orbit. However, in this activity, Earth’s orbit looks larger. The inaccuracy of this activity reinforces a misconception of scale. When using the polystyrene globe in Activity 7 the Seasons Teacher Guide tells teachers to model the Earth’s tilt inaccurately. Question #4 states: “Instruct everyone to tilt the Earth towards the Sun, but tilted roughly halfway down (45º angle). The real angle is 23.5º, but let’s exaggerate for now.” This exaggerated tilt seems like a reasonable thing to do with students for the purpose of a demonstration, but it is not an accurate representation of the day and night cycle and the sunlight intensity different parts of the Earth would be receiving throughout the day. For



16

this reason, it seems reasonable that if this is a student’s first exposure to this system the exaggeration of tilt may reinforce or create student misconceptions regardless of the information presented to them in prior activities. If a more accurate representation is initially explored, there is less of a chance students will develop representations that are inaccurate. Furthermore, in Activity 7 of the Seasons Teacher Guide dealing specifically with tilt, Question #2, pg. 78, asks, “Do you think a planet whose axis was not tilted at all would have seasons?” Teachers are instructed to say “No” without any further explanation. This answer is incomplete. Seasons as we experience them on Earth would not be experienced on that planet; however, seasons could exist in different forms. The temperature would vary greatly at different latitudes and temperatures for any specific location could vary seasonally depending on its location on the planet and the shape and plane of the planet’s orbit. Activity 8 “The Angle of Sunlight and Seasons Unraveled” provides an opportunity for students to examine how the angle of sunlight can cause differences in heating the Earth’s surface. On page 85 #4b, the instructions read:

“Partner #2 Moves the “ground” up and down. Explain that the part of the analyzer with the grid on it represents the ground. Ignoring the protractor for the moment, have the second partner practice moving the ground up and down, while partner #1 tries to keep the analyzer aligned with the light bulb.”

There is a diagram and further goes on to note: “It is important for students to understand that, in this model, the ground really does represent the ground of the Earth – changing the position of the ground represents the way the Earth’s position actually does change in relation to the sunlight, due to the Earth’s tilt. That is why students should move the ground up and down, rather than the window. Although this statement is true, it could lead students to think that the ground is moving closer and further away from the Sun, or that the Earth’s axis tilts (or moves) back and forth (instead of understanding that the tilt is fixed in relation to Earth’s orbit).

Instructional Analysis

The Instructional Analysis Team

The Instructional Analysis process was led by Dr. Curtis Pyke, and facilitated by Dr. Sharon Lynch, professors of secondary education at The George Washington University. For a complete list of the evaluation team members for The Real Reasons for Seasons Instructional Analysis, see Appendix B. For a list of the Project 2061 Instructional Analysis categories, criteria, indicators, and criterion rating scheme, see Appendix A. Appendix D summarizes the results from the Instructional Analysis for Seasons.



17

Instructional Analysis Team Report For ease of reference, each criterion in the Instructional Analysis is listed below with the corresponding section from Appendix D, followed by the evaluation team’s comments in the written analysis. The tables provide the justification for a rating of each criterion to the indicator level. The rating for each criterion was determined by using the criterion rating scheme designed by AAAS Project 2061 (AAAS, 2001b). It should be noted that a particular rating is not necessarily reflective of the number of indicators met in each criterion. Rather for each criterion, a different rating scheme was developed by Project 2061 that weights the indicators differentially. Category I: Providing a Sense of Purpose

Criterion A. Conveying unit purpose. Does the material convey an overall sense of purpose and direction that is understandable and motivating to students? Indicators of meeting the criterion: Met Not Met

1. A problem, question, representation (or otherwise identified purpose) is presented to students.

● 2. The problem, question, representation (or otherwise identified purpose) is likely to be comprehensible to students.

●

3. The problem, question, representation (or otherwise identified purpose) is likely to be interesting and/or motivating to students.

●

4. Students are given an opportunity to think about and discuss the problem, question, representation (or otherwise identified purpose).

●

5. Most lessons are consistent with the stated purpose and those that are not are explicitly labeled as digressions.

●

6. The material returns to the stated purpose at the end of the unit. ● Rating Scheme Excellent: The material meets indicators 1–6. Satisfactory: The material meets indicators 1–3 and 5. Poor: The material meets indicator 1 at best.

Rating =No Rating

The stated purpose in the Seasons Teacher Guide (LHS, GEMS, 2000, reprinted 2001) is to highlight the Sun-Earth relationship that influences the Earth’s seasons. In the first sentence of the Introducing Seasons section (p. 9), the purpose is clearly stated, but this is to the teacher, and not to the students. instructs the teacher to “tell the class they will be studying a number of relationships between the Earth and the Sun, and in particular what causes the seasons.” There is nothing at the beginning of the Seasons unit directed toward the students that indicates which variables are important, such as the orientation of the Earth’s axis relative to the plane of its orbit around the Sun, as being critical to understanding the reasons for seasons. The evaluation team inferred this to be intentional.

The unit begins with phenomena within the students’ ranges of perception and comprehension by eliciting students’ experiences and knowledge of seasonal impact on human events. Students’ experiences with seasons are elicited explicitly (pp. 17-21) in Activity 1. Activity 2 provides an opportunity for the problem and misconceptions to be presented, but will not be addressed by the teacher and students until future lessons (viz. pp. 47,52,63,70,78,87 are



18

explanations of issues where teachers are instructed to revisit the survey from page 26, Activity 2). The problem is likely to be comprehensible to students.

The key concept has the potential of being interesting or motivating to students and opportunities are provided to think about and discuss problems and questions related to seasons understanding. Students are asked to compare what many people think about the reasons for the seasons with what scientists think (p. 28). Students are also asked to think about and discuss the stated purpose. It is also possible that seventh graders could be highly motivated by learning that, after completing the unit, they could end up knowing more about seasons than some Harvard graduates (suggested as a motivator on p. 9 & 22). The first exercise (p. 26) should be motivational to the students in that it engages their everyday experiences with seasonal changes and thus provides something to which they can readily relate.

Throughout the Seasons Teacher Guide (pp. 27, 28, 46, 47, 52, 63, 70, 78, 87), students revisit a conceptual survey, found on page 26, and in particular Question 3 and consider seven possible explanations provided in the survey (items A through G). Further, on page 88, item 7, “Some of these questions may provide a chance to reconsider the main ideas of the unit.” This is judged to be an insufficient amount of time to be given to students’ cognitive processes. The Real Reasons for Seasons curriculum unit is a teacher-centered instructional approach, whereby students are not truly given time to debate and discuss ideas. There is a limited forum for discussion at the end of each activity, not often integrating both the survey and specific opportunities for the teacher to clarify the ideas and purpose (i.e., p. 88). Students are given an opportunity to discuss the seasons in Activity 2, but not in the specific context of what causes seasons until Activities 7 & 8.

Most of the lessons are highly aligned with understanding the reason for the seasons and consistent with Benchmark 4A, 6-8, #4. Each activity supports the stated purpose and there are no major diversions. The Seasons Unraveled section at the end of the unit on page 87 serves as the culminating session and summarizes the stated purpose of the entire 8-part learning activity. For this Criterion, Seasons meets five out of six indicators but was given No Rating because the rating scheme is dependent on meeting indicator 1, which the evaluation team inferred to be intentionally not met. In other words, the evaluation team interpreted the Seasons materials to instruct the teacher to withhold the unit purpose from the students. The Seasons Teacher Guide does pose a question to the students at the beginning of Activity 1, instructing the teacher to “tell the class they will be studying a number of relationships between the Earth and the Sun, and in particular what causes the seasons.” This sentence was deemed insufficient by the evaluation team in order to meet Indicator 1. The understanding of seasons is a broad and complex system of relationships between concepts that needs to be qualified to students explicitly, to convey unit purpose.

Criterion B. Conveying lesson/activity purpose. Does the material convey the purpose of each lesson/activity and its relationship to others?

Indicators of meeting the criterion: Met Not Met 1. The material conveys or prompts teachers to convey the purpose of the activity to students.

●

2. The purpose is expressed in a way that is likely to be comprehensible to students. ● 3. The material encourages each student to think about the purpose of the activity. ● 4. The material conveys or prompts teachers to convey to students how the activity relates to the unit purpose.

●



19

5. The material engages students in thinking about what they have learned so far and what they need to learn/do next at appropriate points. ●

Rating Scheme Excellent: The material meets all indicators. Satisfactory: The material meets any three out of the five indicators. Poor: The material meets no more than one out of the five indicators.

Rating =Excellent

Throughout the Seasons Teacher Guide, the teacher is prompted to remind students of the purpose of each activity (examples are given on pages 17, #1-2; 50, #1; 60, #2; 61, #5; 70, #10; 75, #4; and 88, #7-8). The purpose of each activity is articulated to the teacher through notes in the margins of the Seasons Teacher Guide and through procedural items that suggest responses to questions. Explicit instructions to the teacher also tie each lesson to the next lesson. • Activity 1: p. 17; #2 – “What kinds of changes occur with the seasons?” • Activity 2: spherical p. 23 & #2; p. 23 #4 – tilt of Earth’s axis • Activity 3: p. 45; item #5 – tangential departure from Earth’s surface vs. perpendicular

departure (see figure on p. 45). • Activity 4: p. 49-Overview & p. 50; Item #1 – shape of Earth’s orbit • Activity 5: p. 57-Overview & p. 60, Item #2 – latitudinal temperature variation • Activity 6: p. 65-Overview – latitudinal variation in hours of daylight • Activity 7: p. 73-Overview – tilt of the Earth’s axis relative to the plane of its orbit. • Activity 8: p. 81-Overview – angle between the Sun’s rays and the surface of the Earth.

The purpose of each individual activity is likely to be comprehensible to students in the targeted grade level. Initially, students are unlikely to connect each activity to the overall goal even though that goal is closely aligned to the activity. It is not always clear that students know exactly why they are doing each activity. For example, they may not realize why they are conducting an activity of making scale models of the solar system (Activities 3 & 4) until much later in unit. Overall, the purposes are inferred to be comprehensible.

Examples of this are in Activity 3 (starting on p. 43), where students conduct an imaginary trip to the Sun with the stated goal of understanding the spherical nature of Earth, but it is not explained why this is important to studying the causes of seasons. Similarly, on page 61, #5 it is not explained why the question, “Does the temperature have the same pattern as ours all over the world?” is closely related to understanding the real reasons for seasons. The Seasons Teacher Guide does not make the connection explicit.

Throughout the Seasons Teacher Guide (pp. 27, 28, 46, 47, 52, 63, 70, 78, 87), students revisit a conceptual survey, found on page 26, and in particular question #3, considering multiple possible explanations in A through G. The survey helps to encourage students to think about the ultimate goal of the activities.

The goal of conveying to students how the activity relates to the unit purpose is frequently met, but not always. For example, on page 46, #8 it states, “Refer back to question 2 on the Sun-Earth Survey. Which drawing most accurately represents the Sun-Earth distance?” This is followed by page 47, #9 in which the students are asked how it could be that Earth is not closer in June, even though it is warmer in the northern hemisphere at that time. The main goal of the activity was to reveal that Earth’s orbit is nearly circular, and consequently distance from the Sun is not an important component in the reasons for the seasons.



20

The teacher is explicitly prompted throughout the Seasons Teacher Guide to ask students to summarize what they have learned and where they are going next. For example, the instructions to the teacher on page 70, #9 state, “Remind students of question 3 in the Sun-Earth Survey.” Ask, “Do your observations about the number of daylight hours help you rule out any of the answers?”, and #10 “Tell the class the next activity will make clear for them why the day length changes with the seasons the way it does.” Page 78, item #5, directs students attention to what they have just learned, “It’s the angle of the sunlight that makes the light more concentrated in summer…” but further states: “…but don’t reveal this [correct reasoning] yet unless a student mentions it.” Most activities are structured to lead students in this way and there are instructions to tell the students what will be covered in the next activity.

Criterion C. Justifying lesson/activity sequence. Does the material involve students in a logical or strategic sequence of activities (versus just a collection of activities)?

Indicators of meeting the criterion: Met Not Met 1.The material includes a logical or strategic sequence of activities. ● 2.The material conveys the rationale for this sequence. ● Rating Scheme Excellent: The material meets both indicators. Satisfactory: The material meets the first indicator. Poor: The reviewer can infer a logical rationale for the sequence of only a few activities.

Rating =Satisfactory

The Summary Outlines (pp. 113-123) cover the sequence and rationale of activities. They convey the concepts needed to understand seasonal variation; address misconceptions; familiarize students with scales, the curvature of the Earth, and the concept of latitude; the shape of Earth’s orbit; latitudinal variations in both temperature and hours of daylight; and, the tilt of Earth’s axis relative to the plane of its orbit. Activity 8 pulls all these concepts together and also addresses possible lingering misconceptions.

On page 12 in the first paragraph it is stated, “For example, the idea that Earth’s distance from the Sun causes the seasons is first thrown into doubt when students prove to themselves in Activity 4 that the Earth’s orbit is (nearly) circular, and later when they discover, in Activities 6 and 7…opposite seasons happen simultaneously …” Although inferred by the reader that the sequence is logical and strategic, there is no clear place or map in the Seasons Teacher Guide where it has been made explicit to the teacher or student why this is the appropriate sequence of activities. This sequence is critical in developing an understanding of the real reasons for seasons. It is briefly described in the section “How This Unit Addresses Common Misconceptions” that common student misconceptions are elicited then dispelled using evidence (pp. 12-13). The Seasons Teacher Guide does not argue why the activities should not be presented out of sequence. Category II: Taking Account of Student Ideas

Criterion A. Attending to prerequisite knowledge and skills. Does the material specify prerequisite knowledge/skills that are necessary to the learning of the key idea(s)?



21

Indicators of meeting the criterion: Met Not Met 1. The material alerts the teacher to specific prerequisite ideas or skills (versus stating only prerequisite topics or terms).

● 2. The material alerts teachers to the specific ideas for which the prerequisites are needed.

● 3. The material alerts students to prerequisite ideas or experiences that are being assumed.

● 4. The material adequately addresses (provides instructional support for) prerequisites in the same unit or in earlier units (in the same or other grades). (The material should not be held accountable for addressing prerequisites from an earlier grade range. However, if a material does address such prerequisites they should count as evidence for this indicator.)

●

5. The material makes adequate connections (provides instructional support for connections) between ideas treated in a particular unit and their prerequisites (even if the prerequisites are addressed elsewhere).

●

Rating Scheme Excellent: The material meets indicators 1, 2, 3 or 4, and 5 for all or most prerequisites. Satisfactory: The material meets indicators 1, 2, 5, and either 3 or 4 for some prerequisites. Fair: The material meets indicators 5 and either 3 or 4 for some prerequisites. Poor: The material meets no more than one indicator.

Rating =Poor

The materials inconsistently alert the teacher to specific prerequisite ideas or skills. When alerts are given, they are embedded in the activity rather than presented at the beginning. The ideas that “seasons exist” and what is “tilt” and an understanding the latitude/longitude system are core prerequisite ideas or skills, as is graphing, which are not routinely identified as alerts to the teacher as necessary prerequisite ideas. Examples of these include: • Page 11: The Earth’s axis is tilted, there are more hours of daylight, and sunlight strikes the

Northern Hemisphere at a high angle (more perpendicular), making the sunlight more concentrated resulting in more heating.

• Page 17: “What kinds of changes occur with the seasons?” (but no description of seasons as prerequisite knowledge).

• Page 43: In number 2, “Spend a little time verifying that your students understand what altitude means.”

• Page 58: “Check to see if your students understand how and are proficient at finding things on Earth by latitude and longitude” (alerts the teacher to a prerequisite skill and knowledge).

• Page 74: “It may have something to do with the direction the Earth is titled” (but no alert that students need to understand what tilt is).

• Page 75: “Ask students to find the Equator and the North Pole…hemispheres … mid-latitude …high-latitude…”, is, dependent on understanding latitude and longitude” (prompted for on p. 58, but not alerted sufficiently on p. 75).

Similarly, the reader can only infer that the students need to know the definitions of a

sphere (such as Earth, p. 23, #2) and a circle (Earth’s orbit, p. 51, #2), as there are no alerts to the teacher. Finally, there is an inference not represented anywhere in the materials, that some mechanism exists allowing energy to travel from the Sun to the Earth.

The three core underlying ideas (spherical Earth, spinning Earth, and orbiting Earth) are consistently repeated and reinforced as supporting ideas throughout the entire Seasons Teacher Guide. The teacher is not alerted at the beginning of the lesson, but rather somewhere in the



22

middle of the lesson as to which prerequisites are necessary. The specific key ideas for the unit are summarized on page 9 and repeated on pages 22-24. These are: Earth is spherical (p. 29), the Earth spins which causes day and night (Mt. Nose activity on p. 23), and Earth orbits around the Sun once a year. In addition, student’s need to have the underlying ideas from:

• Page 45 (d) “Earth’s surface is curved - Earth is a ball” • Page 29: A main goal of Activity 3 is for students to reinforce their [existing] conception of

Earth’s spherical shape, which is a key to understanding the cause of the seasons. • Page 29: The scale of the Earth-Sun system is also crucial to understanding the Earth’s

seasons. • Page 81: If Earth is tilted toward the Sun, then the Sun would appear high in the sky from

Earth’s surface (espoused in Activity 8 with the purpose of “…Earth’s tilt also changes the angle at which the sunlight hits the ground … Earth is a spinning globe whose axis tilts with respect to its orbit around the Sun, and this gives rise to: (a) a varying number of daylight hours in different seasons, and (b) variations in concentration of sunlight on the ground related to the angle the light strikes the ground.

Seemingly absent is a vocabulary list at the beginning of each activity indicating words

with a description of the context needed to be understood in order to successfully accomplish the activity.

It is inferred that students recognize that hours of day and night and changing seasons have an impact on human events, but there is scant acknowledgement of student experiences. The materials also assume that students believe that getting close to an energy (light, heat) source often results in the energy being more concentrated. Examples include:

• Page 31 and 53: The prerequisite idea needed is that the farther you get away from

something, the smaller it appears which shows up in the Trip to the Sun (pages 31-45) and measuring the diameter of the Sun (pp. 53-55);

• Page 69: “Ask if any of your students have ever visited or lived in a place with very different day lengths than those in your school’s region.”

No evidence exists that the material provides instructional support for prerequisites. Page

44, items #7 & 8 reinforce that the students are aware of the prerequisites ideas like knowing the diameter of the Earth (#7) and why the Moon appears larger than the Sun (#9); and page 45, #4d, pointing out that Earth is a ball, which is prerequisite to the next activity.

No apparent evidence exists that the material makes adequate connections (provides instructional support for connections) between ideas treated in a particular unit and their prerequisites (even if the prerequisites are addressed elsewhere).

Criterion B. Alerting teacher to commonly held student ideas. Does the material alert teachers to commonly held student ideas (both troublesome and helpful) such as those described in Benchmarks Chapter 15: The Research Base?

Indicators of meeting the criterion: Met Not Met 1. The material accurately presents specific commonly held ideas that are relevant to the key ideas and have appeared in scholarly publications (rather than just stating that

●



23

students have difficulties with particular ideas or topics). 2. The material clarifies/explains commonly held ideas (rather than just listing them). ● Rating Scheme Excellent: The material meets indicators 1 and 2 for a considerable proportion of commonly held ideas that have appeared in scholarly publications. Satisfactory: The material meets indicators 1 and 2 for some commonly held ideas that have appeared in scholarly publications. Poor: The material meets indicator 1 at best.

Rating =No Rating

With the exception to the Private Universe video (Schneps, 1987), there is no reference to misconceptions reported in scholarly literature (viz. recent articles in the Astronomy Education Review and the older references therein). There are, however, frequent references to common student difficulties and beliefs that are part of the anecdotal knowledge base of experienced astronomy/earth science teachers. A margin box on page 10 makes reference to “Children’s Ideas in Science” by Driver et al. Professional publications are not required to back the misconceptions, just an explanation and the clarification. Pages 10 and 11 present commonly held misconceptions and the only reference to scholarly publications is the box in the margin of page 10.

Although not always part of the scholarly literature, there are frequent references to common student difficulties and beliefs that are part of the anecdotal knowledge base of experienced astronomy teachers (compare in contrast limited work by Novak and, separately, Vosniadou). However, there are a few obvious places where common student misconceptions reported in the scholarly literature are not highlighted (shape of Earth and direction of gravity on p. 23 #2 & #3), as well as the fact that the Earth goes around the Sun once each year (p.23 #4). One additional common misconception known from experienced teachers but not frequently identified in the literature is that students think that Earth wobbles back and forth, rather than being constantly pointing toward the North Star (p. 76, #2).

Overall, references to commonly held ideas in Seasons are deemed insufficient by the evaluation team in order to meet Indicator 1. Criterion B was given No Rating because the rating scheme is dependent on meeting Indicator 1.

Criterion C. Assisting teacher in identifying own students' ideas. Does the material include suggestions for teachers to find out what their students think about familiar phenomena related to key ideas before they are introduced?

Indicators of meeting the criterion: Met Not Met 1. The material includes specific questions or tasks that could be used by teachers to identify students' ideas.

●

2. The questions/tasks are likely to be comprehensible to students who have not studied the topic and are not familiar with the scientific vocabulary.

●

3. The questions/tasks are identified as serving the purpose of identifying students' ideas. ● 4. The material includes questions/tasks that ask students to make predictions and/or give explanations of phenomena (rather than focus primarily on identifying students' meanings for terms).

●

5. The material suggests how teachers can probe beneath students' initial responses to questions or interpret student responses (e.g., by providing annotated samples of student work).

●

Rating Scheme



24

Excellent: The material provides a sufficient number and variety of questions/tasks that meet indicators 1 and 2 and meet indicators 3–5. Satisfactory: The material provides some questions/tasks that meet indicators 1–4. Poor: The material provides some questions/tasks that meet indicator 1 or indicators 1 and 2.

Rating =Poor

Seasons’ teacher-centered discourse is highly structured, using Socratic questioning and, as clearly such, (e.g., p. 26 #2-what have we found out, what patterns do you see in the data) student ideas are not routinely elicited prior to instruction (with the clear exception being the Survey on p. 26). On page 22, it states, “Students then answer questions on a short, written survey [found on page 26] that focuses them on the following additional Sun-Earth concepts…” Activity 2 addresses this issue for the fundamental concept being addressed. • So its rays are more concentrated on the ground (also see p. 87, 90) • There are more hours of daylight in June than in December (also see p. 87, 90)

Mastering scientific vocabulary is clearly not the focus of the materials. The most complex vocabulary is frequently mathematical (e.g., ellipse, focus, foci, etc.) and geographical (e.g., hemisphere, latitude, equator, etc.). Three examples, of many, include:

• Page 51: “Which orbit is more circular, Pluto’s or Earth’s?” which clearly avoids ellipse and

eccentricity as vocabulary words. • Page 85 of the Seasons Teacher Guide says, “Tell students that their job is to find the angle

of the light when the light spreads over 4 grid squares [rather than use terms like zenith angle, area (square centimeters), and flux etc].”

• Page 86, rather than use intensity, the materials state, “at which angle is the light most concentrated or brightest looking on the ground.”

Further, on page 45, the geometrical concepts are presented graphically and technical geometric terms, such as chord, tangent, perpendicular, are not used. Students are likely to grasp these concepts through the sketch presented.

Teachers are directed to guide the students’ thinking rather than eliciting students’ misconceptions. Teachers are rarely directed to have students make predictions; still teachers are directed to frequently ask students to give explanations of phenomena. Specific examples sighted include:

• Page 61: “What do you think the graph would like in our area over the time period from

January to June?” … “Does the temperature have the same pattern as ours all over the world?”

• Page 62: “What have we found out? What patterns do you see in your graphs?” • Page 74, #3; Benchmark 4A, 6-8, #3

It seems that the text guides the teachers to elicit the correct answers from the students but does not probe significantly beyond this.



25

Criterion D. Addressing commonly held ideas. Does the material attempt to address

commonly held student ideas?

Indicators of meeting the criterion: Met Not Met 1. The material explicitly addresses commonly held ideas. ● 2. The material includes questions, tasks, or activities that are likely to help students progress from their initial ideas, for example, by a. explicitly challenging students' ideas, for example, by comparing their predictions about a phenomenon to what actually happens b. prompting students to contrast commonly held ideas with the scientifically correct ideas, and resolve differences between them c. extending correct commonly held ideas that have limited scope.

●

3. The material includes suggestions to teachers about how to take into account their own students' ideas.

● Rating Scheme If there is research on commonly held student ideas: Excellent: The material meets indicators 1 and 2 for a considerable proportion of commonly held ideas that are documented in the literature. Satisfactory: The material meets indicators 1 and 2 for some commonly held ideas that are documented in the literature. Poor: The material meets the first indicator at best.


The Seasons’ materials explicitly describe some of the commonly held ideas that are documented in research literature, such as the shape of orbits, and Sun-Earth distance as it relates to seasons. However, several ideas are not adequately addressed: the transfer of energy from sunlight to Earth, gravity, and wobble issues; which, although are not core to the topic, are in fact a part of the activities. The degree to which these exist in the scholarly literature is mixed. Relevant examples include:

• Page 10 and 11: “What Are Some Common Misconceptions About Seasons?” • Page 23 #2: Ideas surrounding the question “Why does the Earth look flat.” • Page 23 #3: Ideas surrounding that gravity always points toward the center of a sphere. • Page 23 #4: Ideas surrounding how long it takes Earth to orbit the Sun. • Page 51: Dispelling a Common Misconception About Earth’s Orbit. • Page 76 #2: …the [Earth’s] North Pole always points at the North Star.

Activity 7 addresses the commonly held ideas of the spherical Earth, the concept of Earth orbiting the Sun, and the concept of light traveling from the Sun to the Earth. (viz. content analysis sightings #54 & #57.) The commonly held idea of long days and short nights in summer is developed on page 75, #5.

Using the student survey, students are frequently prompted to revisit their answers to the survey in the form, “What can you change about your survey responses in light of what you just learned?” However, this approach does not sufficiently allow for resolution until the very end of the Seasons unit, nor are many other opportunities presented. These ideas are most closely related to Content Analysis sightings #34, Benchmark 4B, 3-5, #2; and Benchmark 4A, 6-8, #3 (AAAS, 1993). Further page 52, #8 explains the commonly held misconception that the reason



26

for the seasons is that the Earth is closer to the Sun in the summer, but further away in the winter. Throughout the Seasons Teacher Guide (pages 27, 28, 46, 47, 52, 63, 70, 78, 87), students revisit the conceptual survey, found on page 26, and in particular Question 3 and consider multiple possible explanations in items A-G.

The misconception that is identified in Activity 2 is that the Earth’s orbit around the Sun, although elliptical, is actually much closer to a circle. The relative sizes of the Earth, Moon, and Sun are addressed in Activities 4 and 3, respectively. There is not enough explicit evidence for suggestions commonly provided to teachers about how to take into account their own students’ ideas beyond that of the survey on page 26. Category III. Engaging Students with Relevant Phenomena

Criterion A. Providing a variety of phenomena. Does the material provide multiple and varied phenomena to support key ideas?

Indicators of meeting the criterion: Met Not Met 1. Phenomena are useful in making the key ideas real. ● 2. Phenomena are explicitly linked to the relevant key ideas. ● Rating Scheme Excellent: The material provides a sufficient number and variety of phenomena that meet indicators 1 and 2. Satisfactory: The material provides some phenomena that meet indicators 1 and 2. Poor: The material provides, at best, one phenomenon that meets indicators 1 and 2. If the benchmark for which the material is analyzed includes several ideas, the reviewers should proceed as follows: a. Identify the ideas for which there is a content match. b. Score the treatment of each idea that results from step a. The overall score for this criterion will be the average of the scores for each idea.

Rating =Poor

The Seasons Teacher Guide states that the three key ideas are: (i) Impact of a spherical Earth on seasons; (ii) Impact of a tilted, spinning Earth on seasons; and (iii) Impact of Earth’s orbit on seasons. These are directly related to the core benchmark 4B, 6-8, #4 (AAAS, 1993), as supported by 4A, 6-8, #3 and 4B, grades 3-5, #2 (AAAS, 1993). These benchmarks suggest that the main idea is “because the Earth turns daily on an axis that is tilted relative to the plane of the Earth’s yearly orbit around the Sun, sunlight falls more intensely on different parts of the Earth during the year. The difference in heating of the Earth’s surface produces the planet’s season and weather patterns.”

There is a dramatic conflict with the approach of the Seasons Teacher Guide for this particular Instructional Analysis criterion. For example, in Activity 1, “Name the Season” does not directly make the key ideas real and relevant. Activity 1 gives students a social contextual background to seasons but does not offer any conceptual knowledge toward understanding the reasons for seasons. However, the reviewers agree that this activity is important instructionally to make “seasons real,” particularly to students in geographic regions that do not experience dramatic seasonal changes. Similarly, in Activity 2, using the Sun-Earth Survey does not support this criterion. The “Mount Nose Activity” on pages 23 and 24 increases students’ supporting knowledge by focusing on night and day, but it does not explicitly make the impact of a rotating Earth real and relevant to the seasons. Similar arguments can be made for each of the following activities until Activity 7, where students begin assimilating their learning experiences



27

from the first six Activities. In other words, the activity sequence is seen as building learning blocks and challenging misconceptions about the Universe, but not necessarily consistent with the ideas of making the phenomena real and relevant to the students. Phenomena used as Sightings include: • Activity 2: Mt. Nose for day/night (+) (p. 23) • Activity 3: Trip to the Sun for spherical Earth (+) (p. 29) • Activity 4: Plotting of circular shape of Earth’s orbit (-) (p. 49) • Activity 5: Graphing temperatures around the world (+) (p. 61 #1) • Activity 6: Graphing hours of daylight (+) (p. 65) • Activity 7: Kinesthetic modeling of Earth’s orbit (+) (p. 74 #3) • Activity 8: Review (-) and assimilation (pp. 87-88)

At the individual activity level, there is a clear opportunity to make each phenomenon real, but there is only one experience per activity. There are not often multiple experiences of the same phenomena in different settings for the same idea. On the other hand, at the individual activity level, there is little explicit linking of phenomena to the key ideas until the final two activities. For example, the phenomena and student experiences such as the day/night on “Mt. Nose” activity makes the concept of “day and night” real to the student, but, it is not explicitly linked to the relevant key ideas and overarching goal until the very end of the activity. The poor rating is due primarily to the lack of multiple phenomena for each key idea.

Criterion B. Providing vivid experiences. Does the material include activities that provide firsthand experiences with phenomena when practical or provide students with a vicarious sense of the phenomena when not practical?

Indicators of meeting the criterion: Met Not Met 1. Each firsthand experience is efficient (when compared to other firsthand experiences) and, if several firsthand experiences target the same idea, the set of firsthand experiences is efficient. (The efficiency of an experience equals the cost of the experience [in time and money] in relation to its value.)

●

2. The experiences that are not firsthand (e.g., text, pictures, video) provide students with a vicarious sense of the phenomena. (Please note that if the material provides only firsthand experiences, this indicator is not applicable.)

n/a n/a

3. The set of firsthand and vicarious experiences is sufficient. ● Rating Scheme Excellent: The material meets all indicators or just indicators 1 and 3 or indicators 2 and 3, if firsthand experiences are not possible. Satisfactory: The material includes some efficient firsthand experiences and, if several firsthand experiences target the same idea, the set of firsthand experiences is sufficient. When firsthand experiences are not practical, the material provides students with a vicarious sense of the phenomena for some of the experiences that are not firsthand. Poor: The material includes at best only one efficient firsthand experience or provides students with a vicarious sense of one phenomenon that is not firsthand. Reviewers should proceed as follows: a. Identify the key ideas for which there is a content match. b. Score the treatment of each idea that results from step a. The overall score for this criterion will be the average of the scores for each idea.



28


Each first hand experience is efficient and listed at the beginning of each activity. The required manipulative materials are relatively easy to acquire and none of the activities appear to be problematic material-wise. In terms of time, each activity arrives at the core idea quickly and only requires one to two class periods, which seems reasonable given the constraints of many classroom environments. The experiences such as the “Mt. Nose” activity (p. 23), graphing temperatures around the world (p. 57), and day length graphing (pp. 65-67) for example, although not first hand, do provide students with a vicarious sense of the phenomena which they are intended to address. Most experiences are isolated and not explicitly tied to one another until the end of the Seasons Teacher Guide. Similarly, the experiences often are not repeated in other contexts. Together, this is seen as insufficient. Category IV. Developing and Using Scientific Ideas

Criterion A. Introducing terms meaningfully. Does the material introduce technical terms only in conjunction with experience with the idea or process and only as needed to facilitate thinking and promote effective communication?

Indicators of meeting the criterion: Met Not Met 1. The material links technical terms to relevant experiences that develop the idea as the term is used (rather than just having students learn definitions of terms).

●

2. The material restricts the use of technical terms to those needed to communicate intelligibly about key ideas. ●

Rating Scheme Excellent: The material meets both indicators. Satisfactory: The material fully meets one indicator and partially meets the other. Poor: The material marginally meets both indicators at best.


Although the Seasons text does a good job of limiting technical terms, it does not do a satisfactory job of linking the existing technical terms to students’ relevant experiences. The text introduces the terms “revolves,” “rotates,” and “axis” on page 25, #4 as prerequisite ideas that will be explored throughout the unit. Although these terms will be explored further, it is necessary to communicate these terms to efficiently understand the phenomena. Conversely, the Seasons unit uses “ellipse” and appropriately avoids using terms like “eccentricity,” “aphelion,” and “perihelion” that may encourage students to focus on the wrong ideas and engender misunderstanding. The box in the margin of Activity 4 (p. 51) mathematically defines an ellipse, but the terms “focus” and “foci” are at hand in order for students to properly identify and communicate the shape of the orbit’s of a comet, Pluto, and Earth.

The term “altitude” is used in Activity 3 on pages 44-46 as a means of distinguishing between the distance traveled and how high above ground level the students are on an imaginary trip toward the Sun. The trip starts from San Francisco at sunrise and travels in a straight line toward the Sun. This trip is used to reinforce the idea that Earth is indeed a sphere, whereby students note the curvature of the Earth and the path their trip takes out of Earth’s atmosphere. The term “diameter,” from page 44, #7, is necessary to use as a term to define how big or small the Earth and Sun seem to the viewer as distance increases toward the Sun.



29

The term “tilt” from page 47, #9 is a core term that is necessary to describe the orientation of Earth’s rotational axis relative to the plane of its orbit. Tilt becomes an important term to understand the key ideas surrounding the differences in lengths of day in Activity 6, sunlight intensity in Activity 8, and the misconception related to tilt in relation to the scale of Earth’s distance around the Sun in Activity 3.

“Latitude” and “longitude” are used as appropriate terms to verify locations around the world where students participating in the separate GLOBE project recorded temperatures and lengths of day. Page 6, #1 and #5 gives instruction toward purpose and use of terms for completing Activity 5.

Example sightings, among many, include: • Activity 2, p. 23, #4 directs teachers to discuss using the terms revolves, rotates, and axis. • Activity 3, p. 44, #6 & #7 directs to teachers use the terms altitude and diameter. • Activity 3, p. 47, #9 refers appropriately to the Earth’s tilt. • Activity 4, p. 50, uses ellipse and appropriately avoids eccentricity. • Activity 4, p. 51, #5 defines focus and foci. • Activity 5, p. 60, #1 & #5 defines to longitude and latitude at the appropriate level.

Criterion B. Representing ideas effectively. Does the material include accurate and comprehensible representations of the key ideas?

Indicators of meeting the criterion: Met Not Met 1. Representation is accurate (or, if not accurate, then students are asked to critique the representation).

●

2. Representation is likely to be comprehensible to students. ● 3. Representation is explicitly linked to the real thing. ● Rating Scheme Excellent: The material includes a sufficient number and variety of representations that meet indicators 1–3 and none of the representations included in the material are inaccurate. (In order to judge whether there is a sufficient number and variety of representations, reviewers should first consider which key ideas require representations and then decide whether these are adequately represented. However, reviewers are not expected to evaluate and rate each idea separately and average the scores.) Satisfactory: The material includes some representations that meet indicators 1–3 and few (if any) of the representations included in the material are inaccurate. (In some cases, including one accurate and comprehensible representation for a specific idea may be sufficient for a material to receive a "satisfactory" rating. However, most of the key ideas must be adequately represented.) Poor: Even though the material includes a few representations that meet indicators 1–3, few or none of the key ideas are adequately represented.

Rating =Excellent

In the context of this grade level presentation, the technical terms are appropriately limited and linked to the relevant experiences. Each representation is accurate, likely to be comprehensible to students, and explicitly linked to the real thing (Activity 7 is a clear example). In particular, the included CD-ROM provides numerous examples to help students link ideas to real world experiences. Terms used are limited to those fundamentally necessary.

Criterion C. Demonstrating use of knowledge. Does the material demonstrate/model or include suggestions for teachers on how to demonstrate/ model skills or the use of knowledge?



30

Indicators of meeting the criterion: Met Not Met 1. The material consistently carries out (or instructs teachers to carry out) the expected performance (e.g., the student text explains a particular phenomenon using the kinetic molecular theory). (Teacher's guides often include responses to questions posed in the student text. If the material does not instruct the teacher to use the answers to model the use of knowledge, such responses do not count as instances of modeling.)

●

2. The performance is step-by-step. ● 3. The performance is explicitly identified as a demonstration of the use of knowledge or skill.

● 4.The material provides running commentary that point to particular aspects of the demonstration and/or criteria for judging the quality of a performance.

● Rating Scheme Excellent: The material meets all 4 indicators. Satisfactory: The material meets indicators 1 and 2. Poor: The material meets indicator 1 at best.


Many of the student activities are similar to demonstrations that could be done by the teacher, but give students smaller versions of the equipment. Earth’s orbit activity on page 50 instructs the teacher to draw on the board or use transparencies and question students. On page 50 the guide instructs the teacher how to draw the ellipse as a demonstration/model for students. Another example is on page 69, whereby the teacher is prompted to ask students about the seasons in other geographic regions. This is based on the graphing students have done during the course of Activity 6 and the “think aloud” modeling that directs teachers in how to answer the text’s questions by showing the students how to use the graph. In each case, clear step-by-step instructions and questions are provided in the teacher’s notes. The Seasons Teacher Guide, however, does not explicitly tell teachers why they are being instructed to think aloud (model thinking). Boxes in the margins of the teacher’s notes do provide a running commentary that supports implementation. Page 76 is an example of this, where the margin box explains why the step-by-step instructions are important when orienting students to the Earth’s axis such that the south to north polar direction always points to the North Star (Polaris).

Criterion D. Providing practice. Does the material provide tasks/questions for students to practice skills or use knowledge in a variety of situations?

Indicators of meeting the criterion: Met Not Met 1. The material provides a sufficient number of tasks in a variety of contexts, including everyday contexts. (In order to determine whether the task/question addresses the actual substance of the key idea, reviewers will need to consider both the question and the expected response in the teacher's guide.)

●

2. The material includes novel tasks. ● 3. The material provides a sequence of questions or tasks in which the complexity is progressively increased.

● 4. The material provides students first with opportunities for guided practice with feedback and then with practice in which the amount of support is gradually decreased.

● Rating Scheme Excellent: The material meets indicators 1, 2, and either 3 or 4. Satisfactory: The material provides some tasks/questions, including novel tasks.



31

Poor: The material provides at best some tasks/questions, but no novel tasks. Reviewers should proceed as follows: a. Identify the key ideas for which there is a content match. b. Score the treatment of each idea that results from step a. The overall score for this criterion will be the average of the scores for each idea.

Rating =Poor

The material in The Real Reasons for Seasons curriculum unit does not provide a sufficient number of novel tasks for students to experience. Novel tasks that could be included are those that would frequently question students about modeling tasks and answer questions in new contexts about how things would be different in different situations relating to seasonal variations. For example, students could be presented with scenarios such as, “If Earth were moved to Mars’ more distant orbital position, or if Earth’s rotation rate were changed, how would seasonal conditions on Earth be different?” Or, students could be asked to design an imaginary planet’s spin and orbit around a different star to match some given hypothetical seasonal criteria. Students could even be asked to consider how seasons would change if Earth’s orbit were caused to be much more elliptical and less circular (for example, if Earth collided with a large asteroid). The CD-ROM and “Going Further” section of certain activities allude to novel tasks that would compliment the prescribed activity, however this is optional. Category V. Promoting Student Thinking about Phenomena, Experiences, and Knowledge

Criterion A. Encouraging students to explain their ideas. Does the material routinely include suggestions for having each student express, clarify, justify, and represent his or her ideas? Are suggestions made for when and how students will get feedback from peers and the teacher?

Indicators of meeting the criterion: Met Not Met 1. Material routinely encourages students to express their ideas. ● 2. Material encourages students not only to express but also to clarify, justify, and represent their ideas (a material is not expected to encourage students to clarify, justify, and represent ideas each time they are asked to express their ideas; however, in the course of teaching a particular key idea the material should provide students with opportunities to clarify, justify, and represent ideas).

●

3. Material provides opportunities for each student (rather than just some students) to express ideas.

●

4. Material includes specific suggestions on how to help the teacher provide explicit feedback to students or includes text that directly provides students with feedback.

● 5. Material includes suggestions on how to diagnose student errors, explanations about how these errors may be corrected, and recommendations for how students' ideas may be further developed.

●

Rating Scheme Excellent: Material meets all indicators. Satisfactory: Material meets 3 out of 5 indicators. Poor: Material meets no more than 1 out of 5 indicators.

Rating =Poor

The Seasons Teacher Guide does not explicitly prompt teachers to routinely get students to express, clarify, or justify their ideas in a variety of contexts, with the clear exception of



32

students revisiting the conceptual survey found on page 26 (sightings in activities where students are to revisit their answers are found on page 27, 28, 46, 47, 52, 63, 70, 78, 87), and in particular Question #3 that asks, “Why do you think it is hotter in the United States in June than in December?” in which they consider multiple possible explanations A-G. The survey does, however, allow students to reconsider their ideas individually, as opposed to relying on group discussions to promote student thinking.

The materials represent a highly structured, teacher-centered approach where the teacher rigidly guides the students through instruction. One of the only clear examples where material includes specific suggestions on how to help the teacher provide explicit feedback to students is on page 87, which is the culminating section of Activity 8 that ties all the students’ experiences together. There are fewer clear examples on pages 45 and 78 in which the teacher directs students to write down their ideas and then share them with the class by having the students revisit the survey. The teacher is directed to have the students take the survey again. This exercise allows the students to synthesize the scientific explanations they have acquired in the activities. In the Behind the Scenes notes on pages 89 through 92, teachers are presented with the scientific background to support the correct answers to the survey.

The teacher is provided with “closure” questions at the end of each activity, but rarely does the guide present “probing” questions. There are few suggestions on how to diagnose student errors or how to correct them. Further, there are limited scenarios in which students are encouraged to think about their experiences in any way that is not structured by the Seasons Teacher Guide. For these reasons, there is insufficient evidence to meet this criterion.

Criterion B. Guiding student interpretation and reasoning. Does the material include tasks and/or question sequences to guide student interpretation and reasoning about experiences with phenomena, representations, and ideas?

Indicators of meeting the criterion: Met Not Met 1. The material includes specific and relevant tasks and/or questions for the experience or reading.

●

2. The questions or tasks have helpful characteristics such as a. framing important issues b. helping students to relate their experiences with phenomena or representations to presented scientific ideas c. helping students to make connections between their own ideas and the phenomena or representations observed d. helping students to make connections between their own ideas and the presented scientific ideas e. anticipating common student misconceptions f. focusing on contrasts between student misconceptions and scientific alternatives. Please note that while a single high quality task or question sequence might have only one of these characteristics, the set of sightings should exhibit several of them.

●

3. There are scaffolded sequences of questions or tasks (as opposed to separate questions or tasks).

●

Rating Scheme Excellent: Material consistently meets all three indicators. Satisfactory: Material consistently meets indicators 1 and 2. Poor: Material meets indicator 1 at best.

Rating =Excellent



33

The activities provided in the book promote an environment where students are carefully guided through specific sequences of questions that lead them to make relevant observations and draw the scientifically accurate relationships and inferences from those questions. However, significant time and opportunity is not built into the prescribed schedule and sequence for students to make sense of ideas and practice in novel contexts.

Overall, the material includes specific and relevant tasks guided by questions from the teacher. Examples of this include the pattern analysis of worldwide temperature data graphs (p. 61, #4) and pattern analysis of length of day graphs (p. 69, #6). Throughout the book, questions are carefully structured to lead the students from one insight to another without giving the whole story away by asserting the end results.

Questions and tasks are carefully sequenced and scaffolded. They often have helpful characteristics including: • framing important issues (i.e., p. 45, “Give students a few minutes to write in their lab books

why we have gained altitude in the first six steps of the trip to the Sun) and solidifying important underlying concepts, such as the spherical nature of Earth activity during a trip to the Sun on page 45;

• helping students relate their experiences with seasons (i.e., p. 17 states, “Your students focus on their own experiences with seasonal change.”);

• helping students to make connections between their own ideas and presented phenomena is not often done prior to Activity 7;

• helping students to make connections between their own ideas and scientific ideas (i.e., p. 52, #8 which asks teachers to emphasize the difference between commonly held ideas of distance and seasons in contrast with scientifically accepted ideas and when students analyze ideas from people outside the class as revealed on the survey on p. 26 as directed on p. 87);

• anticipating common misconceptions such as described on page 10 • focusing on contrasts between student misconceptions and scientific alternatives by having

students analyze evidence as described on page 11.

Criterion C. Encouraging students to think about what they’ve learned. Does the material suggest ways to have students check and reflect on their own progress?

Indicators of meeting the criterion: Met Not Met 1. The material gives students an opportunity to revise their initial ideas based on what they have learned (without asking them explicitly to think about how their ideas have changed).

●

2. The material engages (or provides specific suggestions for teachers to engage) students in monitoring how their ideas have changed, but does so infrequently in the unit.

● 3. The material engages (or provides specific suggestions for teachers to engage) students in monitoring how their ideas have changed and does so periodically in the unit.

●

Rating Scheme Excellent: The material meets indicator 3. Satisfactory: The material meets indicator 2. Poor: The material meets indicator 1 at best.

Rating =Poor Students are frequently questioned in a Socratic-style leading directly to the scientifically correct result. Only when students revisit the conceptual survey found on page 26,



34

(pp. 27, 28, 46, 47, 52, 63, 70, 78, 87), and in particular Question #3, in which they consider multiple possible explanations A-G, do they revise their own thinking. As a single tool, it is likely insufficient, in and of itself, to encourage student metacognition. Category VI. Assessing Progress Criterion A. Aligning assessment to goals. Assuming a content match between the curriculum material and the key ideas, are assessment items included that match the key ideas?

Indicators of meeting the criterion: Met Not Met 1. The specific ideas in the key ideas are necessary in order to respond to the assessment items.

●

2. The specific ideas in the key ideas are sufficient to respond to the assessment items (or, if other ideas are needed, they are not more sophisticated than key ideas and have been taught earlier).

●

Rating Scheme Excellent: The material provides a sufficient number [9] of assessment items that meet indicators 1 and 2. Satisfactory: The material provides some assessment items that meet indicators 1 and 2. Poor: The material provides no more than one/a few assessment items that meet indicators 1 and 2.

Rating =Poor Many of the assessment suggestions are formative assessments to be used at the end of each activity. Suggested assessments call for the key ideas to be used in a way that incorporates the specific target idea for each activity. These assessments are creative authentic assessment items that do not require more sophisticated ideas than the idea taught in each activity. Overall, there are too few assessment items to recommend a rating better than poor. Criterion B. Testing for understanding. Does the material include assessment tasks that require application of ideas and avoid allowing students a trivial way out, like using a formula or repeating a memorized term from the text without understanding it?

Indicators of meeting the criterion: Met Not Met 1. Assessment items focus on understanding of key ideas. ● 2. Assessment items include both familiar and novel tasks. ● Rating Scheme Excellent: The material provides a sufficient number of assessment items that meet indicators 1 and 2. Satisfactory: The material provides some assessment items that meet indicators 1 and 2 or sufficient assessment items that meet indicator 1. Poor: The material provides no more than one/a few assessment items that meet indicator 1.

Rating =Poor Each assessment suggestion provided in the Seasons Teacher Guide is an authentic assessment and as such, requires a meaningful response in order to convey understanding of key ideas. The tasks concepts, due to their abstract nature, are unlikely to be familiar to students. Most assessment items are typical of middle school classrooms and lack novel experiences.



35

Criterion C. Using assessment to inform instruction. Are some assessments embedded in the curriculum along the way, with advice to teachers as to how they might use the results to choose or modify activities?

Indicators of meeting the criterion: Met Not Met 1. The material uses embedded assessment as a routine strategy (rather than just including occasional questions).

●

2. The material assists teachers in interpreting student responses to diagnose what learning difficulties remain.

● 3. The material provides specific suggestions to teachers about how to use the information from the embedded assessments to make instructional decisions about what ideas need to be addressed by further activities.

●

Rating Scheme Excellent: The material meets all indicators. Satisfactory: The material meets indicators 1 and 2 or indicators 1 and 3. Poor: The material meets indicator 1 at best.

Rating =Poor Pages 105-107 list four selected student outcomes and nine assessment suggestions aligned with most of the activities, but primarily focused on the final activity. Suggestions are made for how teachers might approach assessment of each of the stated outcomes. However, each idea uses an informal, formative approach that would likely be insufficient to find out what students really know and understand about the ideas. For example, on page 107, it is suggested that students could write a “letter to a fourth-grader” or a “letter to a friend in the southern hemisphere.” As presented, these assessment suggestions are insufficiently described and structured, particularly with respect to scope and sequence, to guide the teacher’s instruction and evaluation. Category VII. Enhancing the Science Learning Environment Criterion A. Providing teacher content support. Would the material help teachers improve their understanding of science, mathematics, and technology necessary for teaching the material?

Indicators of meeting the criterion: Met Not Met 1. The material provides content information for teachers that are relevant to the learning goals they are teaching.

●

2. The material alerts teachers to how ideas have been simplified for student and what the more sophisticated versions are (even though students are not expected to understand the more sophisticated ideas).

●

3. The material provides sufficiently detailed answers to questions in the student book for teachers to understand and interpret typical student responses.

● 4. The material makes suggestions for how to use recommended resources for improving teachers understanding of the learning goals.

● Rating Scheme Excellent: The material meets all indicators. Satisfactory: The material meets indicator 2 out of the 4 indicators. Poor: The material meets indicator 1 at best.




36

Succinct and helpful background information is provided at the beginning of the text on page 11 – “What DOES cause the seasons” provides the teacher with the key scientific concepts to explain what causes seasons. Extensive content support is found in the margins (e.g., p. 51) and at the end of the guide on pages 89-98 – Going Behind the Scenes. There exists an annotated bibliography of resources on pages 108-110 and literature connections on pages 111-112. Most importantly, the included CD-ROM has movies, web links, pictures, videos of students doing activities, GLOBE program data, all of which are highlighted and/or referred to in the text margins (i.e., pp. 25, 29, 46, 47, 48, 5 8, 63, 71). The scientifically accurate answers to teacher-centered questions are provided in the teacher’s notes (for an example, see p. 23, #1-#4). However, there is insufficiently detailed information for how to interpret student responses, mostly because of the highly structured nature of questions posed to students. There do not appear to be places where the material has been overly simplified for students in such a way that teachers need to be alerted. Criterion B. Encouraging curiosity and questioning. Does the material help teachers to create a classroom environment that welcomes student curiosity, rewards creativity, encourages a spirit of healthy questioning, and avoids dogmatism?

Indicators of meeting the criterion: Met Not Met 1. The material provides opportunities for students to express their curiosity and creativity.

●

2. The material provides occasion for students to ask questions and guides their search for answers.

●

3. The material provides opportunities to respect and value students’ ideas (e.g. by suggesting how each student’s ideas can be heard and considered and held up to evidence).

●

4. The material sends the message that science is more than rules or facts to be learned and that problems may have more than one solution.

●

5. The material provides examples of classroom interactions—e.g. dialogue boxes, vignettes, or video clips—that illustrate appropriate ways to respond to student questions or ideas, etc.

●

Rating Scheme Excellent: The material meets all indicators. Satisfactory: The material meets indicator 3 out of the 5 indicators. Poor: The material meets indicator 1 at best.


The material in the Seasons unit carefully guide students to the scientifically correct answer and does not allow for them to conduct their own investigations and rarely allows students to pursue the answers to their own questions. Furthermore, the Seasons Teacher Guide infrequently asks the teachers to engage students with the ideas of other students. The Seasons unit is teacher-centered, and as such, does not provide ample opportunities to value students’ ideas. However, the hands-on material covered in the Seasons Teacher Guide generally provides opportunity for students to express curiosity and creativity and also provides a platform to engage in discovery. Engaged students in these activities are afforded opportunities to question the tasks they are performing and guides their search for answers. The instructional approach is to lead students to the single correct solution, but does so in a manner that sufficiently alerts students to the idea that problems may have more than one solution. This aspect of the criterion is generally provided in the “Going Further” section of the Seasons Teacher Guide, but is mainly



37

left up to the teacher to convey to students. Video clips of students are not included on the Seasons CD-ROM, which are very helpful for showing teachers how to conduct the demonstrations and activities. Also, there are few dialogue boxes that help illustrate appropriate ways of answering student questions. Criterion C. Supporting all students. Does the material help teachers to create a classroom community that encourages high expectations for all students, that enables all students to experience success, and that provides all students a feeling of belonging in the science classroom?

Indicators of meeting the criterion: Met Not Met 1. The material avoids stereotypes or language that might be offensive to a particular group.

●

2. The material illustrates the contribution or participation of under-represented groups to science-related fields.

●

3. The material suggests alternative formats for students to express or develop their ideas during instruction and assessment.

●

4. The material includes specific suggestions about modifying activities for students with special needs, interests, or abilities.

● 5. The material provides strategies to validate students’ personal and social experiences as being relevant to mathematical or scientific ideas. ●

Rating Scheme Excellent: The material meets all indicators. Satisfactory: The material meets indicator 3 out of the 5 indicators. Poor: The material meets indicator 1 at best.


The material successfully avoids stereotypes or language that might be offensive to a particular group. The material also illustrates the contribution or participation of under-represented groups (viz. p. 93). Teachers are provided with alternative formats to the main activity to be achieved. For example, page 78 describes a game on seasons and orbits, where students model the different seasons and reinforce the position of the North Star as related to Earth’s orbit. As another example, on page 86, students are given pencils to model light rays. This also includes looking at videos and images as well as simulation software sampled on the included CD-ROM. However, there are few strategies provided to the teacher to validate students’ personal and social experiences as being relevant to mathematical or scientific ideas. The only example, page 88, #10 directs teachers to “praise their students for their excellent scientific work through the unit and the leaps they’ve made in [their] understanding.” In addition, there are no specific suggestions for modification of activities for students with special needs provided to the teacher.



38

References

American Association for the Advancement of Science (AAAS). (1993). Benchmarks

for science literacy. New York: Oxford University Press.

AAAS. (2001a). Atlas of Science Literacy. New York: Oxford University Press.

AAAS. (2001b). AAAS Science Textbooks Conference. [Computer software].

Washington, DC: AAAS.

AAAS. (1999). Middle Grades Science Textbooks. Retrieved October 27, 2003 from

the AAAS Project 2061 Web site: http://www.project2061.org/research/textbook/default.htm.

Harvard-Smithsonian Center for Astrophysics. (2001). ARIES—Exploring Motion and

Forces: Speed, Acceleration, and Friction. Watertown, MA: Charlesbridge Publishing.

Holliday, W.G. (2003). Methodological concerns about AAAS’s Project 2061 study of

science textbooks. Journal of Research in Science Teaching, 40(5), p. 529-534.

Kesidou, S., & Roseman, J.E. (2002). How well do middle school science programs

measure up? Findings from Project 2061’s curriculum review. Journal of Research in Science

Teaching, 39(6), p. 522-549.

Kesidou, S., & Roseman, J.E. (2003). Project 2061 analyses of middle-school science

textbooks: A response to Holliday. Journal of Research in Science Teaching, 40(5), p. 535-543.

Lawrence Hall of Science (LHS) & Great Explorations in Math and Science (GEMS).

(2000). The real reasons for seasons: Sun-Earth connections. Berkeley, CA: The Regents of

the University of California at Berkeley.

Lynch, S., Kuipers, J., Pyke, C., & Szesze, M. (2003, April). Examining the effects of a

highly rated science curriculum unit on diverse student populations: Results from a planning



39

grant. Paper presented at the annual meeting of the American Educational Research

Association, Chicago, IL.

Lynch, S. (1997). Novice teachers' encounters with national science education reform:

Entanglements or intelligent interconnections? Journal for Research in Science Teaching, 34

(1), 3-17.

Lynch, S., Kuipers, J.C., Pyke, C. & Szesze, M. (2002). NSF/IERI proposal, Scaling up

highly rated science curricula in diverse student populations: Using evidence to close

achievement gaps. Washington, DC: The George Washington University.

Lynch, S., Pyke, C., & Jansen, J. (2003). Deepening understanding of science and

mathematics education reform principles: Novice teachers design web-based units using Project

2061’s curriculum analysis. Journal of Science Teacher Education, 14(3), p. 193-216.

McDermott, L.C., & Redish, E.F. (1999). Resource Letter on Physics Education

Research. Educational Resources Information Center (ERIC). ED 439011.



40

Appendix A

AAAS Project 2061 Instructional Analysis

The following criteria for evaluating the quality of instructional support can be accessed at:

http://www.project2061.org/newsinfo/research/default.htm.

Category I. Providing a Sense of Purpose

Criterion A. Conveying unit purpose. Does the material convey an overall sense of purpose and

direction that is understandable and motivating to students?

Indicators of meeting the criterion

1. A problem, question, representation (or otherwise identified purpose) is presented to students.

2. The problem, question, representation (or otherwise identified purpose) is likely to be

comprehensible to students.

3. The problem, question, representation (or otherwise identified purpose) is likely to be

interesting and/or motivating to students.

4. Students are given an opportunity to think about and discuss the problem, question,

representation (or otherwise identified purpose).

5. Most lessons are consistent with the stated purpose and those that are not are explicitly labeled

as digressions.

6. The material returns to the stated purpose at the end of the unit.

Rating Scheme

Excellent: The material meets indicators 1–6.

Satisfactory: The material meets indicators 1–3 and 5.

Poor: The material meets indicator 1 at best.

Criterion B. Conveying lesson/activity purpose. Does the material convey the purpose of each

lesson/activity and its relationship to others?


1. The material conveys or prompts teachers to convey the purpose of the activity to students.

2. The purpose is expressed in a way that is likely to be comprehensible to students.

3. The material encourages each student to think about the purpose of the activity.

4. The material conveys or prompts teachers to convey to students how the activity relates to the

unit purpose.



41

5. The material engages students in thinking about what they have learned so far and what they

need to learn/do next at appropriate points.

Rating Scheme

Excellent: The material meets all indicators.

Satisfactory: The material meets any three out of the five indicators.

Poor: The material meets no more than one out of the five indicators.

Criterion C. Justifying lesson/activity sequence. Does the material involve students in a logical

or strategic sequence of activities (versus just a collection of activities)?


1.The material includes a logical or strategic sequence of activities.

2.The material conveys the rationale for this sequence.

Rating Scheme

Excellent: The material meets both indicators.

Satisfactory: The material meets the first indicator.

Poor: The reviewer can infer a logical rationale for the sequence of only a few activities.

Category II. Taking Account of Student Ideas

Criterion A. Attending to prerequisite knowledge and skills. Does the material specify

prerequisite knowledge/skills that are necessary to the learning of the key idea(s)?


1. The material alerts the teacher to specific prerequisite ideas or skills (versus stating only

prerequisite topics or terms).

2. The material alerts teachers to the specific ideas for which the prerequisites are needed.

3. The material alerts students to prerequisite ideas or experiences that are being assumed.

4. The material adequately addresses (provides instructional support for) prerequisites in the

same unit or in earlier units (in the same or other grades). (The material should not be held

accountable for addressing prerequisites from an earlier grade range. However, if a material does

address such prerequisites they should count as evidence for this indicator.)

5. The material makes adequate connections (provides instructional support for connections)

between ideas treated in a particular unit and their prerequisites (even if the prerequisites are

addressed elsewhere).

Rating Scheme



42

Excellent: The material meets indicators 1, 2, 3 or 4, and 5 for all or most prerequisites.

Satisfactory: The material meets indicators 1, 2, 5, and either 3 or 4 for some prerequisites.

Fair: The material meets indicators 5 and either 3 or 4 for some prerequisites.

Poor: The material meets no more than one indicator.

Criterion B. Alerting teacher to commonly held student ideas. Does the material alert teachers to

commonly held student ideas (both troublesome and helpful) such as those described in

Benchmarks Chapter 15: The Research Base?


1. The material accurately presents specific commonly held ideas that are relevant to the key

ideas and have appeared in scholarly publications (rather than just stating that students have

difficulties with particular ideas or topics).

2. The material clarifies/explains commonly held ideas (rather than just listing them).

Rating Scheme

Excellent: The material meets indicators 1 and 2 for a considerable proportion of commonly held

ideas that have appeared in scholarly publications.

Satisfactory: The material meets indicators 1 and 2 for some commonly held ideas that have

appeared in scholarly publications.


Criterion C. Assisting teacher in identifying own students' ideas. Does the material include

suggestions for teachers to find out what their students think about familiar phenomena related to

key ideas before they are introduced?


1. The material includes specific questions or tasks that could be used by teachers to identify

students' ideas.

2. The questions/tasks are likely to be comprehensible to students who have not studied the topic

and are not familiar with the scientific vocabulary.

3. The questions/tasks are identified as serving the purpose of identifying students' ideas.

4. The material includes questions/tasks that ask students to make predictions and/or give

explanations of phenomena (rather than focus primarily on identifying students' meanings for

terms).



43

5. The material suggests how teachers can probe beneath students' initial responses to questions

or interpret student responses (e.g., by providing annotated samples of student work).

Rating Scheme

Excellent: The material provides a sufficient number and variety of questions/tasks that meet

indicators 1 and 2 and meet indicators 3–5.

Satisfactory: The material provides some questions/tasks that meet indicators 1–4.

Poor: The material provides some questions/tasks that meet indicator 1 or indicators 1 and 2.

Criterion D. Addressing commonly held ideas. Does the material attempt to address commonly

held student ideas?


1. The material explicitly addresses commonly held ideas.

2. The material includes questions, tasks, or activities that are

likely to help students progress from their initial ideas, for example, by

a. explicitly challenging students' ideas, for example, by comparing their predictions about a

phenomenon to what actually happens

b. prompting students to contrast commonly held ideas with the scientifically correct ideas, and

resolve differences between them

c. extending correct commonly held ideas that have limited scope.

3. The material includes suggestions to teachers about how to take into account their own

students' ideas.

Rating Scheme

If there is research on commonly held student ideas:

Excellent: The material meets indicators 1 and 2 for a considerable proportion of commonly held

ideas that are documented in the literature.

Satisfactory: The material meets indicators 1 and 2 for some commonly held ideas that are

documented in the literature.

Poor: The material meets the first indicator at best.

Category III: Engaging Students with Relevant Phenomena

Criterion A. Providing variety of phenomena. Does the material provide multiple and varied

phenomena to support the key ideas?




44

1. Phenomena are useful in making the key ideas real.

2. Phenomena are explicitly linked to the relevant key ideas.

Rating Scheme

Excellent: The material provides a sufficient number and variety of phenomena that meet

indicators 1 and 2.

Satisfactory: The material provides some phenomena that meet indicators 1 and 2.

Poor: The material provides, at best, one phenomenon that meets indicators 1 and 2.

If the benchmark for which the material is analyzed includes several ideas, the reviewers should

proceed as follows:

a. Identify the ideas for which there is a content match.

b. Score the treatment of each idea that results from step a. The overall score for this criterion

will be the average of the scores for each idea.

Criterion B. Providing vivid experiences. Does the material include activities that provide

firsthand experiences with phenomena when practical or provide students with a vicarious sense

of the phenomena when not practical?


1. Each firsthand experience is efficient (when compared to other firsthand experiences) and, if

several firsthand experiences target the same idea, the set of firsthand experiences is efficient.

(The efficiency of an experience equals the cost of the experience [in time and money] in relation

to its value.)

2. The experiences that are not firsthand (e.g., text, pictures, video) provide students with a

vicarious sense of the phenomena. (Please note that if the material provides only firsthand

experiences, this indicator is not applicable.)

3. The set of firsthand and vicarious experiences is sufficient.

Rating Scheme

Excellent: The material meets all indicators or just indicators 1 and 3 or indicators 2 and 3, if

firsthand experiences are not possible.

Satisfactory: The material includes some efficient firsthand experiences and, if several firsthand

experiences target the same idea, the set of firsthand experiences is sufficient. When firsthand

experiences are not practical, the material provides students with a vicarious sense of the

phenomena for some of the experiences that are not firsthand.



45

Poor: The material includes at best only one efficient firsthand experience or provides students

with a vicarious sense of one phenomenon that is not firsthand.

Reviewers should proceed as follows:

a. Identify the key ideas for which there is a content match.



Category IV. Developing and Using Scientific Ideas

Criterion A. Introducing terms meaningfully. Does the material introduce technical terms only

in conjunction with experience with the idea or process and only as needed to facilitate thinking

and promote effective communication?


1. The material links technical terms to relevant experiences that develop the idea as the term is

used (rather than just having students learn definitions of terms).

2. The material restricts the use of technical terms to those needed to communicate intelligibly

about key ideas.

Rating Scheme

Excellent: The material meets both indicators.

Satisfactory: The material fully meets one indicator and partially meets the other.

Poor: The material marginally meets both indicators at best.

Criterion B. Representing ideas effectively. Does the material include accurate and

comprehensible representations of the key ideas?


1. Representation is accurate (or, if not accurate, then students are asked to critique the

representation).

2. Representation is likely to be comprehensible to students.

3. Representation is explicitly linked to the real thing.

Rating Scheme

Excellent: The material includes a sufficient number and variety of representations that meet

indicators 1–3 and none of the representations included in the material are inaccurate. (In order

to judge whether there is a sufficient number and variety of representations, reviewers should

first consider which key ideas require representations and then decide whether these are



46

adequately represented. However, reviewers are not expected to evaluate and rate each idea

separately and average the scores.)

Satisfactory: The material includes some representations that meet indicators 1–3 and few (if

any) of the representations included in the material are inaccurate. (In some cases, including one

accurate and comprehensible representation for a specific idea may be sufficient for a material to

receive a "satisfactory" rating. However, most of the key ideas must be adequately represented.)

Poor: Even though the material includes a few representations that meet indicators 1–3, few or

none of the key ideas are adequately represented.

Criterion C. Demonstrating use of knowledge. Does the material demonstrate/model or include

suggestions for teachers on how to demonstrate/model skills or the use of knowledge?


1. The material consistently carries out (or instructs teachers to carry out) the expected

performance (e.g., the student text explains a particular phenomenon using the kinetic molecular

theory). (Teacher's guides often include responses to questions posed in the student text. If the

material does not instruct the teacher to use the answers to model the use of knowledge, such

responses do not count as instances of modeling.)

2. The performance is step-by-step.

3. The performance is explicitly identified as a demonstration of the use of knowledge or skill.

4.The material provides running commentary that points to particular aspects of the

demonstration and/or criteria for judging the quality of a performance.

Rating Scheme

Excellent: The material meets all 4 indicators.

Satisfactory: The material meets indicators 1 and 2.


Criterion D. Providing practice. Does the material provide tasks/questions for students to

practice skills or use knowledge in a variety of situations?


1. The material provides a sufficient number of tasks in a variety of contexts, including everyday

contexts. (In order to determine whether the task/question addresses the actual substance of the

key idea, reviewers will need to consider both the question and the expected response in the

teacher's guide.)



47

2. The material includes novel tasks.

3. The material provides a sequence of questions or tasks in which the complexity is

progressively increased.

4. The material provides students first with opportunities for guided practice with feedback and

then with practice in which the amount of support is gradually decreased.

Rating Scheme

Excellent: The material meets indicators 1, 2, and either 3 or 4.

Satisfactory: The material provides some tasks/questions, including novel tasks.

Poor: The material provides at best some tasks/questions, but no novel tasks.

Reviewers should proceed as follows:

a. Identify the key ideas for which there is a content match.



Category V: Promoting Student Thinking about Phenomena, Experiences, and Knowledge

Criterion A: Encouraging students to explain their ideas. Does the material routinely include

suggestions for having each student express, clarify, justify, and represent his or her ideas? Are

suggestions made for when and how students will get feedback from peers and the teacher?


1. Material routinely encourages students to express their ideas.

2. Material encourages students not only to express but also to clarify, justify, and represent their

ideas (a material is not expected to encourage students to clarify, justify, and represent ideas each

time they are asked to express their ideas; however, in the course of teaching a particular key

idea the material should provide students with opportunities to clarify, justify, and represent

ideas).

3. Material provides opportunities for each student (rather than just some students) to express

ideas.

4. Material includes specific suggestions on how to help the teacher provide explicit feedback to

students or includes text that directly provides students with feedback.

5. Material includes suggestions on how to diagnose student errors, explanations about how these

errors may be corrected, and recommendations for how students' ideas may be further developed.

Rating Scheme



48

Excellent: Material meets all indicators.

Satisfactory: Material meets 3 out of 5 indicators.

Poor: Material meets no more than 1 out of 5 indicators.

Criterion B: Guiding student interpretation and reasoning. Does the material include tasks and/or

question sequences to guide student interpretation and reasoning about experiences with

phenomena, representations, and ideas?


1. The material includes specific and relevant tasks and/or questions for the experience or

reading.

2. The questions or tasks have helpful characteristics such as

a. framing important issues

b. helping students to relate their experiences with phenomena or representations to presented

scientific ideas

c. helping students to make connections between their own ideas and the phenomena or

representations observed

d. helping students to make connections between their own ideas and the presented scientific

ideas

e. anticipating common student misconceptions

f. focusing on contrasts between student misconceptions and scientific alternatives.

Please note that while a single high quality task or question sequence might have only one of

these characteristics, the set of sightings should exhibit several of them.

3. There are scaffolded sequences of questions or tasks (as opposed to separate questions or

tasks).

Rating Scheme

Excellent: Material consistently meets all three indicators.

Satisfactory: Material consistently meets indicators 1 and 2.

Poor: Material meets indicator 1 at best.

Criterion C: Encouraging students to think about what they've learned. Does the material

suggest ways to have students check and reflect on their own progress?




49

1. The material gives students an opportunity to revise their initial ideas based on what they have

learned (without asking them explicitly to think about how their ideas have changed).

2. The material engages (or provides specific suggestions for teachers to engage) students in

monitoring how their ideas have changed, but does so infrequently in the unit.

3. The material engages (or provides specific suggestions for teachers to engage) students in

monitoring how their ideas have changed and does so periodically in the unit.

Rating Scheme

Excellent: The material meets indicator 3.

Satisfactory: The material meets indicator 2.


Category VI: Assessing Progress

Criterion A: Aligning assessment to goals. Assuming a content match between the curriculum

material and the key ideas, are assessment items included that match the key ideas?


1. The specific ideas in the key ideas are necessary in order to respond to the assessment items.

2. The specific ideas in the key ideas are sufficient to respond to the assessment items (or, if

other ideas are needed, they are not more sophisticated than key ideas and have been taught

earlier).

Rating Scheme

Excellent: The material provides a sufficient number [9] of assessment items that meet indicators

1 and 2.

Satisfactory: The material provides some assessment items that meet indicators 1 and 2.

Poor: The material provides no more than one/a few assessment items that meet indicators 1 and

2.

Criterion B: Testing for understanding. Does the material include assessment tasks that require

application of ideas and avoid allowing students a trivial way out, like using a formula or

repeating a memorized term from the text without understanding it?


1. Assessment items focus on understanding of key ideas.

2. Assessment items include both familiar and novel tasks.

Rating Scheme



50

Excellent: The material provides a sufficient number of assessment items that meet indicators 1

and 2.

Satisfactory: The material provides some assessment items that meet indicators 1 and 2 or

sufficient assessment items that meet indicator 1.

Poor: The material provides no more than one/a few assessment items that meet indicator 1.

Criterion C: Using assessment to inform instruction. Are some assessments embedded in the

curriculum along the way, with advice to teachers as to how they might use the results to choose

or modify activities?


1. The material uses embedded assessment as a routine strategy (rather than just including

occasional questions).

2. The material assists teachers in interpreting student responses to diagnose what learning

difficulties remain.

3. The material provides specific suggestions to teachers about how to use the information from

the embedded assessments to make instructional decisions about what ideas need to be addressed

by further activities.

Rating Scheme


Satisfactory: The material meets indicators 1 and 2 or indicators 1 and 3.


Category VII. Enhancing the Science Learning Environment

Criterion A. Providing teacher content support. Would the material help teachers improve their

understanding of science, mathematics, and technology necessary for teaching the material?

Indicators of meeting the criterion.

1. The material provides content information for teachers that is relevant to the learning goals

they are teaching.

2. The material alerts teachers to how ideas have been simplified for student and what the more

sophisticated versions are (even though students are not expected to understand the more

sophisticated ideas).

3. The material provides sufficiently detailed answers to questions in the student book for

teachers to understand and interpret typical student responses.



51

4. The material makes suggestions for how to use recommended resources for improving

teachers understanding of the learning goals.

Rating Scheme


Satisfactory: The material meets indicator 2 out of the 4 indicators.


Criterion B. Encouraging curiosity and questioning. Does the material help teachers to create a

classroom environment that welcomes student curiosity, rewards creativity, encourages a spirit

of healthy questioning, and avoids dogmatism?


1. The material provides opportunities for students to express their curiosity and creativity.

2. The material provides occasion for students to ask questions and guides their search for

answers.

3. The material provides opportunities to respect and value students’ ideas (e.g. by suggesting

how each student’s ideas can be heard and considered and held up to evidence).

4. The material sends the message that science is more than rules or facts to be learned and that

problems may have more than one solution.

5. The material provides examples of classroom interactions—e.g. dialogue boxes, vignettes, or

video clips—that illustrate appropriate ways to respond to student questions or ideas, etc.

Rating Scheme




Criterion C. Supporting all students. Does the material help teachers to create a classroom

community that encourages high expectations for all students, that enables all students to

experience success, and that provides all students a feeling of belonging in the science

classroom?


1. The material avoids stereotypes or language that might be offensive to a particular group.

2. The material illustrates the contribution or participation of under-represented groups to

science-related fields.



52

3. The material suggests alternative formats for students to express or develop their ideas during

instruction and assessment.

4. The material includes specific suggestions about modifying activities for students with special

needs, interests, or abilities.

5. The material provides strategies to validate students’ personal and social experiences as being

relevant to mathematical or scientific ideas.

Rating Scheme






53

Appendix B

List of Participants for Instructional Analysis (May 14-17, 2003)

Ryan Faubert Research Assistant SCALE-uP Project Graduate School of Education and Human Development George Washington University 2134 G Street Washington, DC 20052 Email: [email protected] Jaqueline Geer Teacher Cabin John Middle School Montgomery County Public Schools 10701 Gainsborough Road Potomac, MD 20854 Email: [email protected] Dr. Earl Skelton Adjunct Professor of Physics The George Washington University Samson Hall, Room 210 2036 H Street, N.W. Washington, DC 20052 Email: [email protected] Dr. Timothy Slater Professor, Department of Astronomy Conceptual Astronomy and Physics Education Research (CAPER) Team The University of Arizona Steward Observatory 933 N. Cherry Avenue Tucson, AZ 85721 Email: [email protected]



54

Appendix C

Sightings for "The Real Reasons for

Seasons"

Benchmark Target Idea # Page Sighting

1 4B, 3-5, #2 1 p. 9 the Earth is spherical 2 4B, 3-5, #2 2 p. 9 the Earth spins (rotates every 24 hours) 3 4B, 3-5, #2 2 p. 9 which causes day and night 4 4A, 6-8, #3 3 p. 9 The Earth orbits (revolves) around the Sun once a year 5 4A, 6-8, #3 3 p. 11 The Earth's orbit…it is very nearly a perfect circle 6 4A, 6-8, #3 3 p. 11 but the orbit is so nearly circular that the Earth-Sun distance remains very nearly constant throughout the year. 7 4B, 6-8, #4 4 p. 11 The Earth spins on its north-south axis and the axis is tilted…. 8 4B, 6-8, #4 4 p. 11 Sunlight strikes the Northern Hemisphere at a higher angle (more perpendicular), making the sunlight more concentrated, and resulting in more heating. 9 4B, 3-5, #2 1 p. 12 because of the round shape of the Earth. 11 4B, 3-5, #2 1 p. 12 understanding of the spherical Earth 12 4A, 6-8, #3 3 p. 12 the true shape of the Earth's orbit around the Sun is very nearly a perfect circle 13 4B, 6-8, #4 4 p. 13 concentration of sunlight striking the ground is affected by the angle at which sunlight hits the ground. 14 4B, 3-5, #2 1 p. 22 that the Earth is spherical, 15 4B, 3-5, #2 2 p.22 spins daily on its axis (rotates) 16 4A, 6-8, #3 3 p. 22 and orbits the Sun (revolves) 17 4B, 3-5, #2 2 p. 22 ..know what causes day and night 18 4B, 3-5, #2 1 p. 23 If the Earth is shaped like a ball 19 4A, 6-8, #3 3 p. 23 The Earth revolves around the Sun in an orbit 20 4B, 3-5, #2 2 p. 23 It also spins or rotates on its axis 21 4A, 6-8, #3 3 p. 23 how long one revolution around the Sun takes [one year] 22 4B, 3-5, #2 2 p. 23 how long Earth takes to spin on its axis [24 hours] 23 4B, 3-5, #2 2 p. 24 the students rotate through one more day, observing as the Sun seems to set and rise 24 4A, 6-8, #3 3 p. 24 Earth revolves around the Sun once a year 25 4B, 3-5, #2 2 p. 24 How the spinning of the Earth 26 4B, 3-5, #2 2 p. 24 Causes day and night 27 4A, 6-8, #3 3 p. 26 the shape of Earth's orbit around the Sun (also see p. 52, 87, 90) 28 4B, 6-8, #4 4 p. 26 is facing more toward the Sun in June and away from the Sun in December (also see p. 87, 90) 29 4B, 6-8, #4 4 p. 26 so its rays are more concentrated on the ground (also see p. 87, 90) 30 4B, 6-8, #4 4 p. 26 there are more hours of daylight in June than in December (also see p. 87, 90) 31 4B, 3-5, #2 1 p. 29 due to the spherical shape of the Earth…of the Earth's spherical shape. 32 4B, 3-5, #2 1 p. 43 understanding of the Earth's shape 33 4B, 3-5, #2 2 p. 45 spin the globe (repeated) 34 4B, 3-5, #2 1 p. 45 because the globe is curved 35 4B, 3-5, #2 1 p. 45 but the Earth's surface is curved--Earth is a ball. 36 4B, 3-5, #2 1 p. 46 the curvature of the Earth limits the distance one can see to the horizon 37 4A, 6-8, #3 3 p. 49 true shape of the Earth's orbit around the Sun is very nearly a perfect circle 38 4A, 6-8, #3 3 p. 50 will learn about the shape of the Earth's orbit around the Sun 39 4A, 6-8, #3 3 p. 50 couple of ellipses representing real orbits of Earth and Pluto, which both revolve around the Sun in the solar system



55

40 4A, 6-8, #3 3 p. 50 the shape of each orbit, and how much Earth's orbit deviates from a perfect circle. 41 4A, 6-8, #3 3 p. 51 Earth's orbit….it is very nearly a circular ellipse 42 4A, 6-8, #3 3 p. 52 We've just found that the Earth's orbit is very close to a circle 43 4A, 6-8, #3 3 p. 52 not only is the Earth's orbit almost circular, but the Sun is in the center of the orbit 44 4B, 6-8, #4 5 p. 57 analyze temperature data from around the world…valid data on environmental characteristics related to weather…. 45 4B, 6-8, #4 5 p. 57 the changing seasons account for variation in weather conditions and for extremes in temperature 46 4B, 6-8, #4 5 p. 60 see how those temperatures changed throughout the seasons, and how the patterns of change differ depending on where in the world you are 47 4B, 6-8, #4 5 p. 61 graphing temperatures from several places around the world, they can find out if seasons differ in various locations 48 4B, 3-5, #2 2 p. 65 a very symmetrical pattern of daylight hours that is exactly opposite for the Southern and Northern Hemispheres 49 4B, 3-5, #2 2 p. 68 the number of hours of daylight as "length of day" 50 4B, 3-5, #2 2 p. 68 a day is the time it takes Earth to spin once, which is approximately 24 hours 51 4B, 3-5, #2 2 p. 69 very different day lengths than those in your school's region 52 4B, 6-8, #4 4 p. 73 that shows how the tilt of the Earth's spin axis causes seasons. 53 4B, 3-5, #2 2 p. 73 seasonal variation in day length 54 4A, 6-8, #3 3 p. 74 shape of the Earth's orbit [almost circular] 55 4B, 3-5, #2 2 p. 75 how the Earth rotates on a "spin axis" that runs roughly from north to the south pole 56 4B, 3-5, #2 2 p. 75 their Earth models with the spin axis (pencil) vertical 57 4B, 3-5, #2 2 p. 75 watch their dot cities move from daylight into night and back again 58 4B, 6-8, #4 4 p. 75 not with the pole pointing directly towards the Sun, but tilted 59 4B, 3-5, #2 2 p. 75 students should spin their "Earths" in a counter-clockwise direction…that the spin of the Earth on its axis 60 4A, 6-8, #3 3 p. 75 the same direction as its revolution around the Sun 61 4B, 6-8, #4 4 p. 75 the spin axis should be tilted 62 4B, 3-5, #2 2 p. 75 has long days and short nights…there are long nights and short days 63 4B, 6-8, #4 4 p. 76 The Earth's north spin axis points almost exactly towards the North Star 64 4B, 3-5, #2 2 p. 77 longer days at the south pole……longer days at the north pole….day length is about the same in both Hemispheres 65 4B, 6-8, #4 4 p. 78 do you think a planet whose axis was not tilted at all would have seasons? 66 4B, 6-8, #4 4 p. 78 It's the angle of the sunlight that makes the light more concentrated in summer 67 4B, 6-8, #4 4 p. 81 tilt of the Earth 68 4B, 3-5, #2 2 p. 81 causes day length to change with the seasons 69 4B, 6-8, #4 4 p. 81 the angle at which the sunlight hits the ground 70 4B, 3-5, #2 1 p. 81 the fact that the Earth is a spinning globe 71 4B, 6-8, #4 4 p. 81 whose axis tilts with respect to its orbit around the Sun 72 4B, 3-5, #2 2 p. 81 number of daylight hours in different seasons 73 4B, 6-8, #4 4 p. 81 variations in concentration of sunlight on the ground related to the angle the light strikes the ground 74 4B, 6-8, #4 4 p. 83 students should be able to see the change in the angle of sunlight 75 4B, 6-8, #4 4 p. 85 as the "ground" is tilted closer to the window, the light hits more directly, as it would in summer, when the Sun appears high in the sky 76 4B, 6-8, #4 4 p. 85 the Earth's position actually does change in relation to the sunlight, due to Earth's tilt 77 4B, 6-8, #4 4 p. 86 angle of sunlight is higher in summer, lower in winter 78 4B, 6-8, #4 4 p. 86 which angle is the light most concentrated, or brightest looking on the 'ground'?…which angle is the light least concentrated (least bright) on the 'ground'? 79 4B, 6-8, #4 4 p. 86 the light is more concentrated



56

80 4B, 6-8, #4 5 p. 86 the ground gets hotter 81 4B, 6-8, #4 4 p. 86 in summer, the angle of sunlight hitting the ground is higher. The light is more concentrated 82 4B, 6-8, #4 5 p. 86 so the ground gets hotter. 83 4B, 6-8, #4 5 p. 88 increased heat 84 4B, 3-5, #2 2 p. 88 more hours of daylight 85 4A, 6-8, #3 3 p. 89 The Earth's orbit is close to circular--only very slightly elliptical 86 4A, 6-8, #3 3 p. 91 that the orbit of the Earth is very nearly circular 87 4B, 6-8, #4 4 p. 91 knows that the Earth is tilted on its axis 88 4B, 6-8, #4 4 p. 91 the tilt of the Earth means that… 89 4B, 3-5, #2 2 p. 91 making for more hours of daylight….there are more hours of daylight in June. 90 4B, 6-8, #4 4 p. 91 Sun's position in the sky is higher, increasing the angle of incidence of the sunlight 91 4B, 6-8, #4 4 p. 91 this increases the concentration of the light on the ground 92 4B, 6-8, #4 5 p. 91 so the ground gets warmer 93 4B, 6-8, #4 4 p. 92 any tilt of Earth's axis is with reference to Earth's orbit around the Sun. Earth's orbit defines a plane in space



57

Appendix D: Complete ratings of Instructional Analysis for GEMS: The Real Reasons for Seasons Instructional Analysis Ratings Gems – The Real Reasons for Seasons

Not met = ○ Met = ● Instructional Categories

Indi

cato

r 1

Indi

cato

r 2

Indi

cato

r 3

Indi

cato

r 4

Indi

cato

r 5

Indi

cato

r 6

E

xcel

lent

S

atis

fact

ory

P

oor

N

o R

atin

g

I. Providing a Sense of Purpose Conveying unit purpose ○ ● ● ● ● ● ● Conveying lesson/activity purpose ● ● ● ● ● ● Justifying lesson/activity sequence ● ○ ● II. Taking Account of Student Ideas

Attending to prerequisite knowledge and skills ○ ○ ○ ○ ○ ● Alerting teacher to commonly held student ideas ○ ● ● Assisting teacher in identifying own students’ ideas

● ● ○ ○ ○ ●

Addressing commonly held ideas ● ● ○ ● III. Engaging Students with Relevant Phenomena

Providing variety of phenomena ● ● ● Providing vivid experiences ● n/a ● ● IV. Developing and Using Scientific Ideas

Introducing terms meaningfully ● ● ● Representing ideas effectively ● ● ● ● Demonstrating use of knowledge ● ● ○ ○ ● Providing practice ○ ○ ○ ○ ● V. Promoting Student Thinking about Phenomena, Experiences, and Knowledge.

Encouraging students to explain their ideas ○ ○ ● ○ ○ ● Guiding student interpretation and reasoning ● ● ● ● Encouraging students to think about what they’ve learned ● ○ ○ ● VI. Assessing Progress

Aligning assessment to goals ● ● ● Testing for understanding ● ○ ● Using assessment to inform instruction ● ○ ○ ● VII. Enhancing the Science Learning Environment

Providing teacher content support ● ● ○ ○ ● Encouraging curiosity and questioning ● ● ● ● ○ ● Supporting all students ● ● ● ○ ● ●



58

seasons final curriculum analysis - the george …scale-up/documents/final cur… · ·...

Documents