stephen m. ventura senior professional development

Data Team Certification Study Guide/Key Points

Stephen M. VenturaSenior Professional Development Associate

The Leadership and Learning Center

email: [email protected]

a) Data should be assembled in discussion format prior to start of meeting

b) Results should include number, percentage and names of students at multiple performance levels. (i.e. Exemplary, Proficient, Close to Proficient, Far to Go, etc.)

c) Data is disaggregated by standard. Multiple standards are included on the assessment in order to support specific analysis.

d) Data should be disaggregated by teacher

e) The Data supports timely, specific, and relevant feedback to teachers and students to improve performance

f) Data should include student work samples from the assessment being reviewed

a) Results are disaggregated according to specific subgroups present in the school

b) All team members have results including support personnel who may not be able to attend meeting

c) Data is triangulated (multiple sources of data included that further illuminate students' knowledge and skill in the area being examined)

Dear Certification Candidates-as you prepare for your final presentation, you may refer to these key points to assist you when presenting your specific step from the definitive five-step Data Team process.

Step One: Collecting and Charting and Data

Exemplary Step One Behaviors:

Data Team Certification-Key Points for Final Presentation Steve Ventura-The Leadership and Learning Center

a) The inferring of strengths and needs is based on a direct analysis of student work/ assessment

b) Strengths and needs identified are within the direct influence of teachers

c) Team goes beyond labeling the need or the "what" to infer the "why" or root cause - hierarchy of prerequisite skills

d) Strengths and needs are identified for each "performance group" (i.e. Strengths and Needs for Commended, Met Standard, Close, Intervention, etc.)

e) Needs are prioritized to reflect those areas that will have largest impact within subject areas (if 3 or more needs are identified; otherwise prioritization may be implied)

a) Prioritized needs reflect areas that will have impact within multiple skill areas

b)Needs inferred for intervention group are categorized according to a hierarchy of prerequisite skills

Step Two: Analyze Strengths and Obstacles

Exemplary Step Two Behaviors:


a) Establish, review, or revise a goal

b) Specific targeted subject area, grade level, and students

c) Measurable area of need is established and the assessment to be used is identified

d) Achievable gains in student learning based on the consideration of current performance of all students

e)Relevant goal addresses needs of students and supports school improvement plan

f) Time established for the administration of the next assessment

g) Specific, Measurable, Achievable, Relevant, and Timely

a) Targeted needs have impact in multiple skill areas - ex. "identifying supporting details"

b) Intervention students have a goal related to prerequisite skills necessary for proficiency

c) Goals have a .56 effect size (Hattie)

Step Three: SMART Goals

Exemplary Step Three Behaviors:


a) Strategies directly target the prioritized needs identified during the analysis

b) Strategies chosen will modify teachers' instructional practice

c) Strategies describe actions of the adults that change the thinking of students

d) Team describes strategies (for each performance group)

e) Team agrees on 1-3 prioritized research-based strategies that will have greatest impact

f) Descriptions of strategies are specific enough to allow for replication (i.e. implementation, frequency, duration, resources)

a) Strategies selected impact multiple skill areas

b) Includes modeling of how selected strategies would be implemented

c) Anticipate/discuss acceptable, ongoing adaptations to strategy implementation ("if… then…")

d) Team evaluates their capacity to use the selected instructional strategy and identifies needed resources, training, coaching, etc.

Step Four: Selecting Instructional Strategies

Exemplary Step Four Behaviors:


a) Establishes interim time-frame to monitor the implementation of the strategy

b) Specific enough to allow teachers to predict student performance on next assessment

Step Five: Selecting Results Indicators

a Results Indicators are created for each selected strategy

b) Describes what the teacher (or adult, if others are working with students) will be doing if the strategy is being implemented

c) Describes what the students will be doing if the strategy is being implemented

d) Describes the anticipated change in student performance if the strategy is having the desired impact on the prioritized need

e)Clear and detailed descriptions that allow others to replicate the described practices

Exemplary Step Five Behaviors:


Monitoring Effectiveness: Determine result indicators Results indicators complete the statement: When this strategy is implemented, we expect to see the following evidence of its effectiveness.” Results indicators serve as a monitoring tool for the team. Result indicators detail both what adults do and how students perform. The data team determines what results they expect from the implementation of instructional strategies. They determine how to monitor their efforts and when and how to administer the post-assessment. Once the post-assessment data is analyzed, the data team collaborates and the cycle continues. The team reviews the SMART goals and determines whether to set new goal(s) or to revise the current goal(s) based on the result indicators. If the team decides to move to a new focus area and a new goal because the earlier goal was achieved, they must decide what the next pre-assessment will be, and when it will be administered, and the cycle continues. If the current goal has not been achieved, the teachers analyze their use of the agreed-upon instructional strategies. This analysis leads to the refinement of the current strategies or the replacement of those strategies with new ones that the team predicts will be more effective. Guiding Questions for Defining Evaluation Strategies

x How will we know if our strategies are successful? x What evidence will we have to show the success of our actions?

Listed below are documents that provide additional information to assist data teams with the inquiry process to begin their data collection. Data Team Meetings A data team needs scheduled planning times. It is important for the principal to be involved in the scheduling of planning times for the data team. Principals need to make certain that teachers have time within the school day to conduct their data team meetings. The data teams have a greater opportunity to be successful if the team members have common planning times built into the school day. The data team’s meetings require formal scheduled meetings.

According to the Leadership and Learn Center, formerly the Center of Performance, schools that realize the greatest shift to a data culture scheduled meetings once a week. The frequency of meetings often depends on the type of Data Team. District Data Teams tend to meet on average once a month. School Data Teams may meet once a month or every two weeks. Instructional Data Teams normally meet more frequently, sometimes meeting weekly. Data Team meetings may vary in length from 45 minutes to an hour and a half. It will be the job of the timekeeper to make certain that the meetings do not run long. All minutes from the Instructional Data Teams should be submitted to the School Data Team for review and monitoring.

What%will%I%do%toengage%students’%interest%in%specificlearning%goals?

Ranking(2)Piage,an-programs-=-1.28)(8)Teacher-clarity-=.75)(26)Direct-instruc,on-=.59)(27)Tac,le-=.58)(34)Goals-=.56)(44)Interac,ve-video-methods-=.52)(46)Play-programs-=.50)(56)Quality-of-teaching-=.44)(61)Behavioral-objec,ves-=.41)

What%will%I%do%to%help%students%

interact%with%new%knowledge?

Ranking(15)Vocabulary-programs-=.67)(20)Problem-solving-teaching-=.61)(22)Phonics-instruc,on-=.60)(33)Concept-mapping-=.57)(35)Visual-percep,on-programs-=.55)(57)Wri,ng-programs-=.44)(64)Science-=.40)

What%will%I%do%to%help%students%prac8ce?

Ranking(12)Spaced-vs.-mass-prac,ce-=.71)(30)Worked-examples-=.57)(53)Ques,oning-=.46)

%%%%%%%%%LESSON%DESIGN%TEMPLATE

This%lesson%design%template%contains%suggested%areas%where%meta4analysis%strategies%with%high%to%medium%effect%sizes%would%best%be%applied.

-----------------------------------------------------------------------------------------------

---------------------------------

!

!!!

-----------------------------------------

Steve-Ventura)Stephen-Ventura-Consul,ng

What%will%I%do%next%with%the%

data%I%now%have?

Ranking(3)Forma,ve-evalua,on-=.90)(10)Feedback-=.73)(29)Mastery-learning-=.58)(34)Goals-=.56)(54)Mathema,cs-=.45)

(58)Expecta,ons-=-.43)!

What%will%I%do%to%deepen%students’%understanding%at%their%level%of%learning?

Ranking(9)Reciprocal-teaching-=.74)(17)Crea,vity-programs-=.65)(20)Problem-solving-teaching-=.61)(25)Study-skills-=.59)(36)Peer-tutoring-=.55)(48)Small-group-learning-=.49)-(62)Matching-style-of-learning-=.41)(67)Integrated-curriculum-programs-=.39)(70)Time-on-task-=.38)

What%will%I%do%if%they%s8ll%don’t%

“get%it”?

Ranking

(5)Accelerated-for-giYed-students-=.88)(7)Comprehensive-interven,on-for-LD-students-=.77)(18)Self-verbaliza,on/self-ques,oning-=.64)(21)Not-labeling-students-=.61)(23)Teaching-strategies-=.60)

(68)Enrichment-for-giYed-students-=.39)!

Steve-Ventura)Stephen-Ventura-Consul,ng

Compiled by Instructional Consultants/Brenda Hatfield Sorensen Page 1

Visible Learning: A Synthesis of over 800 Meta-Analyses Relating to Achievement

By John Hattie

Meta-analyses by rank order purple = high d = ≥ .6 (as identified by the author) green = medium d = .4

yellow = low d = .2 red = negative d = < .2 Rank Effect

size Domain Influence Additional information

1 1.44 student Self-report grades Students have reasonably accurate understandings of their levels of achievement, typically formed from past experiences in learning. High school students had very accurate understandings of their achievement levels across all subjects, and are very knowledgeable about their chance of success. These expectations of success may become a barrier for some students as they may only perform to whatever expectations they already have of their ability.

2 1.28 student Piagetian programs The relationship between Piagetian stage and achievement is very high. Knowing the ways in which students think and how this thinking may be constrained by their stages of development may be most important to how teachers choose materials and tasks, how the concept of difficulty and challenge can be realized in different tasks, and the importance of developing successive and simultaneous thinking.

3 .90 teaching approaches

Providing formative evaluation of programs

Feedback that provides teachers with formative information on attention to the purposes of innovations, the willingness to seek evidence on where students are not doing well, the keenness to see the effects on all students, and openness to new experiences make a difference. The major issue is for teachers to pay attention to the formative effects of their teaching.

4 .88 teacher Micro teaching Microteaching typically involves student-teachers conducting mini-lessons to a small group of students, often in a laboratory setting, and then engaging in post-discussions about the lesson.

5 .88 school Acceleration for gifted students

Acceleration involves progress through an educational program at rates faster or ages younger than is conventional, and may include curriculum compacting, telescoping, and advanced placement. Acceleration is successful, but one of the least used methods for gifted students. There may be a negative social impact on gifted students if they are not accelerated.

6 .80 school Classroom behavioral See classroom management, climate, decreasing disruptive behavior, and peer influences on behavior for specifics.


Comprehensive intervention for LD students

A combined direct instruction and strategy instruction model was an effective procedure for remediating learning disabilities. These two approaches are somewhat


independent, so it’s important to use both to maximize the effect on achievement. Important instructional components included attention to sequencing, drill-repetition-practice, segmenting information into parts or units for later synthesis, controlling task difficulty through prompts and cues, use of technology, systematically modeling problem solving steps, and making use of small interactive groups.

8 .75 teacher Teacher clarity Teacher clarity is defined as organization, explanation, examples and guided practice, and assessment of student learning.


Reciprocal teaching The emphasis of reciprocal teaching is on enabling students to learn and use cognitive strategies such as summarizing, questioning, clarifying, and predicting when supported through teacher-student dialogue. The effect size is very high, and this high effect was evident regardless of who delivered the intervention. Effects were highest when there was explicit teaching of cognitive strategies before beginning reciprocal teaching dialogue, showing the importance of modeling and practice as well as giving instruction in use of the strategies close to the time students used them.


Feedback Feedback is most powerful when it is from the student to the teacher. When teachers seek feedback from students as to what students know, what they understand, where they make errors, when they have misconceptions, when they are not engaged- then teaching and learning can be synchronized and powerful. The most effective feedback forms provide cues or reinforcement to the learner, are in the form of video, audio, or computer-assisted instruction feedback, or relate feedback to learning goals. The key is feedback that is received and acted upon by students. At best, each student receives moments of feedback in a single day. Programmed instruction, praise, punishment, and extrinsic awards were the least effective forms of feedback for enhancing achievement. Providing feedback is not about giving rewards, but rather providing information about the task. Feedback is more effective when it provides information on correct rather than incorrect responses. Impact is highest when goals are specific and challenging but when task complexity is low. The art is to provide the right form of feedback at, or just above, the level where the student is learning. Feedback at the self or personal level, such as praise, is rarely effective. When feedback is combined with effective instruction in classrooms, it can be very powerful in enhancing learning.


Teacher-student relationships

In classes with more person-centered teachers, there is more engagement, more respect of self and others, fewer resistant behaviors, greater non-directivity (student initiated and student regulated activities), and higher achievement outcomes. From high to low in effect size, 9 teacher-student relationship variables are non-directivity, empathy, warmth, encouragement of higher order thinking, encouraging


learning, adapting to differences, genuineness, and learner-centered beliefs.


Spaced vs. mass practice

It is the frequency of different opportunities, rather than merely spending more time on task that makes the difference in learning. This is not drill and practice, but rather deliberative practice involving specific skills, complex performance, and attainment of success criteria. Students often need 3-4 exposures to the learning- usually over several days- before there is reasonable probability they will learn. Students in spaced practice perform higher than in mass practice. The effectiveness of the length of pacing is related to the challenge/complexity of the task: stronger effects were found for simple tasks with brief rest periods, and longer rest periods (at least 24 hours) for more complex tasks.


Meta-cognitive strategies

Meta-cognitive activities can include planning how to approach a given learning task, evaluating progress, and monitoring comprehension. Programs are effective when implemented using small group instruction, with students in higher grades, with remedial students, and in less intensive programs. The most effective meta-cognitive strategies were awareness of text inconsistency and use of self-questioning.

14 .67 student Prior achievement 15 .67 curricula Vocabulary programs Students who experienced vocabulary instruction had major

improvements in reading comprehension of passages containing taught words. The most effective teaching methods include providing definitional and contextual information, involving students in deeper processing, and giving students more than one or two exposures to the words.

16 .67 curricula Repeated reading programs

Repeated reading consists of re-reading a short and meaningful passage until a satisfactory level of fluency is reached. The skills of automaticity in word recognition and decoding generally develop in grades 2 and 3. These skills need to be specifically assessed and taught, especially to learning disabled students. Repeated reading has marked positive effects on reading comprehension as well as fluency.

17 .65 curricula Creativity programs An emphasis on instructional strategies and direct instruction makes a major difference in the effectiveness of creativity programs. Creativity programs with a high level of structuring, questioning, and responding to student questioning had high constant effects across all subject areas. Creativity programs that include explicit instruction were most successful, as were those based on developing thinking strategies.


Self-verbalization/ Self-questioning

Of the cognitive mediation strategies taught to students, self-verbalization was among the most effective and worked better for task oriented skills such as writing or mathematics. The effects of student self-questioning are higher with low ability students, because high-ability students are generally


using a variety of self-regulation strategies already. The effects are higher for pre- and post-lesson questioning, compared to questioning interspersed during the lesson; when questionings were delayed rather than immediate; and where there was teacher modeling compared to none.

19 .62 teacher Professional development

Four types of instruction found to be most effective on teacher knowledge and behavior are: observation of actual classroom methods, microteaching, video/audio feedback, and practice. Lowest effects are from discussion, lectures, games/simulations, and guided field trips, as were coaching, modeling, and production of printed or instructional materials.


Problem-solving teaching

Significant direct links were found between problem solving and various measures of basic performance, in particular basic skills in mathematics. For surface knowledge, problem-based learning can have limited and even negative effects. For deeper learning, when students already have the surface level knowledge, problem-based learning can have positive effects. Application of knowledge, not development of knowledge, is the heart of the success of problem-based learning.

21 .61 teacher Not labeling students Labels often help “classify” students and can lead to extra funding, but rarely does it make a difference as to what works best.

22 .60 curricula Phonics instruction Though the findings of the NRP on phonemic awareness have been hotly contested, other studies have confirmed their conclusions. Phonological awareness training should be seen as a causal factor in learning to read. Also, phonological training in groups is more effective, and training that combines phonological and letter training combined is more effective. Phonics with additional language and literacy activities were the most effective.


Teaching strategies For students with learning disabilities, there are higher effect sizes for models of instruction that include direct and strategy instruction, with the most successful being sequencing, drill repetition, and strategy cues. Marzano found that when the instructional technique was designed for the student, the effect was higher than when the technique was designed for the teacher.

24 .59 teaching strategies

Cooperative vs. individualistic learning

Both cooperative and competitive learning are more effective than individualistic methods- pointing to the power of peers in the learning equation. Under cooperative conditions interpersonal relations have the strongest effect on achievement, clearly pointing to the value of friendship in the achievement equation.


Study skills Courses in study skills alone can have an effect on surface level information, but it is necessary to combine study skills with the content to have an effect on the deeper levels of understanding. The highest ranked study skill is organizing and transforming (rearranging of instructional materials to


improve learning, such as making an outline). Teaching note taking has higher effects when students are given instructor’s notes to work from, as the reviewing of notes is more effective than the taking of notes.


Direct instruction In a nutshell, the teacher decides the learning intentions and success criteria, demonstrates them by modeling, evaluates if students understood what they have been told by checking for understanding, and re-telling them what they have told by tying it all together with closure. The effects of direct instruction are similar for regular and special ed/lower ability students, higher for reading than mathematics, similar for low level word attack and also for high level comprehension, and similar for elementary and high school students. The messages of the meta-analyses underline the power of stating the learning intentions and success criteria, then engaging students in moving toward these.

27 .58 curricula Tactile stimulation programs

The evidence of the effectiveness of tactile stimulation used with infants and young children as a form of early intervention to stimulate the senses points to its effectiveness.

28 .58 curricula Comprehension programs

Reading comprehension programs have a greater effect on vocabulary than on reading comprehension outcomes. A higher effect results from programs that focus on processing strategies such as inferential reasoning, rules for summarizing, and chunking texts. Guthrie’s concept-oriented reading program intervention has had positive effects on multiple text comprehension, fluency, story comprehension, and motivation.


Mastery learning Features of mastery learning include providing students with a clear explanation of what is to be mastered, high levels of cooperation among classmates, high levels of frequent and specific teacher feedback, use of diagnostic formative tests, and regular corrections of mistakes. The notion is that learning should be held constant and time should be allowed to vary. It is the most successful of several innovative systems, with highest effects at the elementary level, followed by high school and college.


Worked examples Worked examples typically consist of a problem statement and the appropriate steps to the solution. Instruction using worked examples generally helps to reduce cognitive load.

31 .57 home Home environment 32 .57 home Socioeconomic status 33 .57 teaching

strategies Concept mapping Concept mapping involves developing graphical

representations of the conceptual structure of content to be learned. The importance of it relates to its emphasis on summarizing the main ideas in what is to be learned. It can assist in synthesizing and identifying major ideas, themes, and interrelationships- particularly for learners who do not have organizing and synthesizing skills. Effects were greater when done after initial exposure to the material to be mapped, and


when the emphasis was on understanding central rather than detailed ideas of the topic to be mapped.


Goals Goals are critical for enhancing performance. Achievement is enhanced to the effect that students and teachers set challenging, rather than “do your best” goals. More difficult goals are more effective in that they lead to a clearer notion of success and direct students’ attention to relevant behaviors or outcomes. Any school with a “do your best” goal should change it to “face your challenges” or “strive to the highest.” Effective teachers set appropriately challenging goals and then structure situations so students can reach them. Encouraging students to share commitment to the goals and providing feedback on how successful learning has been increases the likelihood of attainment. Challenging goals are also effective with special ed students.

35 .55 curricula Visual-perception programs

Visual perception refers to the process of organizing and interpreting letters on the page. Both auditory and visual perception were important predictors of reading for both average students and students with disabilities.


Peer tutoring The overall effect of the use of peers as co-teachers is quite powerful. It is an excellent method to teach students to become their own teachers. Peer tutoring has many academic and social benefits for both those tutoring and those being tutored. If the peer intervention is more student controlled, such as peers being involved in setting goals, monitoring performance, evaluating performance, and selecting rewards, the effects are greater than when the teacher controls the situation. When students become teachers of others, they learn as much as those they are teaching.


Cooperative vs. competitive learning

Both cooperative and competitive learning are more effective than individualistic. For rote decoding and correcting tasks, cooperation does not seem to be superior. Cooperative learning is more effective in reading than in mathematics. The positive effects of cooperative learning increased as students moved through elementary to junior high and high school, particularly when there was individual accountability and group rewards.

38 .54 student Pre-term birth weight

39 .53 school Climate of the classroom: Classroom cohesion

A key factor in positive classroom climate is classroom cohesion- the sense that all are working towards positive learning gains. Common attributes that optimize student learning include goal directedness, positive interpersonal relations, and social support. In situations with greater cohesiveness, it is more likely there are co-peer learning, tolerance and welcoming of error, increased feedback, and more discussions of goals, success criteria, and positive relationships.


Keller’s Personalized System of Instruction

Students using PSI had higher grades and higher satisfaction rates than students in conventional classes. Key features of


the program include students proceed through the course at their own pace, students demonstrate mastery of each component before proceeding to the next, materials and communication are largely text based, and teachers primarily provide tutorial support and student motivation.

41 .53 school Peer influences Low classroom peer acceptance can be consistently linked with student disengagement. Friendships can play an important part in the classroom environment, as they often involve higher levels of caring, support, and help, can ease conflict resolution, and thus lead to more learning opportunities and enhanced academic achievement, particularly from early adolescence on.

42 .52 school Classroom management

The attributes of teachers that had the greatest influence on ensuring well-managed classrooms and reducing disruption came from having an appropriate mental set or “with-it-ness” by the teacher; the teacher had the ability to identify and quickly act on potential behavioral problems and retain emotional objectivity.

43 .52 curricula Outdoor/adventure programs

The most successful programs were Outward Bound or Adventure programs. Those run by teachers were least successful, and those that were more cognitively based than emotionally or affective based were more successful. Major reason for a program’s success include the way activities are structured to emphasize very challenging learning intentions, clear success criteria, optimized peer support, and feedback that is actively sought by participants.


Interactive video methods

Cognitive achievement from interactive video instruction is influenced by a wide range of variables such as the nature of instructional content, environmental factors, instructional methods, and the learning materials. Hypermedia (multiple representations of information and interactivities between users and this information) has positive effects over traditional instruction, with effects greater in regular classrooms, in elementary school, and when used to supplement rather than substitute regular instruction.

45 .51 home Parental involvement 46 .50 curricula Play programs For younger children, play makes a difference. It promotes

improved performance outcomes in cognitive-linguistic and affective-social domains. Socio-dramatic play had the most striking effect and the smallest effect was with imaginative play.

47 .50 curricula Second/third chance programs

Well-designed, reliably implemented, one-to-one interventions (e.g., Reading Recovery) can make a significant contribution to improved reading outcomes for many students whose poor reading skills place them at risk of academic failure.

48 .49 school Small group learning Small group learning typically involves assigning a task to a small group and then expecting them to complete the task. Small group learning has significantly more positive effects than individual learning with computer technology on student


achievement, group task performance, and several process and affective outcomes. The effects of small group learning were significantly enhanced when students had group work experience or instruction, where specific learning strategies were employed, and when group size was small. Simply placing students in small or more homogeneous groups is not enough. For grouping to be maximally effective, materials and teaching must be varied and maximally effective materials and teaching must be varied and made appropriately challenging to accommodate student needs.

49 .48 student Concentration/ Persistence/ Engagement

50 .48 school School effects Classrooms are far more important than schools in determining how children perform at school. Effective schools are only as effective to the extent they have effective teachers. Many of the influences that really make a difference to student learning in developed nations are within schools, from the influence of specific teachers, specific curriculum, and strategies teachers use to teach.

51 .48 student Motivation 52 .47 student Early intervention 53 .46 teaching

practices Questioning The overall effects of questioning vary, depending on the type

of question. Higher cognitive questioning strategies have a small positive effect on learning measures. Factual pre-questions can facilitate learning if they are directly related to the materials to be learned. Teacher training in questioning matters, especially when attention is paid to teachers monitoring their own actions. Of more importance than teacher questioning is analyzing the questions students ask.

54 .45 curricula Mathematics Highest effects accrued when teachers provided feedback data or recommendations to students, peer-assisted learning, explicit teacher-led instruction, direct instruction, and concrete feedback to parents. Providing teachers and students with specific information on how each student was performing seemed to consistently enhance mathematics achievement. Overall, the use of manipulative materials does not detract from, but does little to support the learning of mathematics.

55 .45 student Preschool programs 56 .44 teacher Quality of teaching The highest correlations involved items relating to:

1. Teachers challenging students, encouraging them to think through and solve problems, either by themselves or together as a group. 2. High expectations. 3. Monitoring and evaluation, getting students to think about the nature and quality of their work; encouraging them to test mathematical ideas and discover mathematical principals. 4. Teaching the language, love, and details of mathematics. Having poor teachers can be devastating. The residual effects of relatively ineffectual teachers from prior years can be


measured in subsequent student achievement scores. The effect of poor teaching quality tends to persist for years after a student has had such a teacher.

57 .44 curricula Writing programs It is powerful to teach strategies for planning, revising, and editing compositions, particularly with struggling writers. There is power in teaching students the processes and strategies involved in writing, structuring teaching of writing by having students work together in an organized fashion, and setting of clear and specific goals as to the purpose of each piece of writing. Workshop instructional treatments have a greater effect size than computer support or teaching of inquiry skills.

58 .43 teacher Expectations Teachers must stop overemphasizing ability, and start emphasizing progress, seek evidence to surprise themselves regarding student performance, find ways to raise the achievement of all, and be evidence informed about the talents and growth of all students, by welcoming diversity and being accountable for all.

59 .43 school School size Some research indicates optimal size of a high school to be about 800. The curriculum advantages of larger schools start to reduce in their effectiveness as they grow much beyond 800. Achievement gains in mathematics and reading over the course of high school were largest in middle-sized high schools (600-800). Teachers and students at smaller schools are more likely to have positive perceptions of their school environment.

60 .43 student Self-concept 61 .41 teaching

practices Behavioral objectives/ advance organizers

Using advance organizers to introduce new materials, by providing a bridge from previous knowledge, did facilitate long-term learning, but effects were lower for written advance organizers compared to non-written ones, and had no effect with low-ability or low-knowledge learners.

62 .41 teaching practices

Matching style of learning

Authors noted it was hard to discern the meaning of some of the meta-analyses related to this topic. One conclusion was that learning style is somewhat important. One possible explanation is that when students enjoy learning, then achievement is higher.


Cooperative learning Cooperative learning is effective, especially when contrasted with competitive and individualistic learning. Cooperation was superior to competition in promoting achievement across all subjects, for all age groups, and for tasks involving concept attainment, verbal problem solving, categorizing, spatial problem solving, retention, memory, motor performance, and guessing-judging-predicting. It is more effective in reading than in math. The effects increased as students moved through elementary through high school. Under cooperative conditions, interpersonal relations have the strongest influence on achievement, and this clearly points to the value of friendship in the achievement equation.

64 .40 curricula Science Meta-analyses in science investigated skills based programs


vs. traditional programs. Skills based programs emphasize analytic and process skills, integrated lab activities, higher cognitive skills, and science appreciation. Traditional curricula emphasize knowledge of facts, laws, theories, and applications. Students in skills based programs generally outperformed those in traditional ones on attitudes, process skills, analytic skills, and achievement.

65 .40 curricula Social skills programs The effects of social skills programs are stronger on enhancing peer relations and social outcomes; lower when students are initially identified as social problems; and lowest when academic achievement is the outcome.

66 .40 student Reducing anxiety 67 .39 curricula Integrated curriculum

programs The most important elements in integrated programs were thematic instruction and an emphasis on process skills. They were more successful in elementary and middle school than in secondary; for lower achieving and ethnically diverse students; and when more experienced teachers implemented the program.

68 .39 school Enrichment for gifted students

Enrichment has less effect than acceleration. Programs in which students mastered more mature ideas had higher effects than those with a broader investigation of the regular curriculum.

69 .38 curricula Career education interventions

Career counseling has generally positive effects, with class interventions the most effective, but also requiring the most hours.


Time on task Time on learning can involve longer school days or years, procedural time, time off or on task, and so on. At best, half of student time in class involves engagement in the class activity, largely because so much time is spent listening to teachers talking. Engagement is highest when students are working in groups or laboratories. Also, time spent on practice needs to be deliberate, with a focus on effort to improve performance, requiring students to stretch themselves to higher levels of performance, and requiring concentration and effort over extended periods.


Computer assisted instruction

Computers are used effectively when there is a diversity of teaching strategies; when there is pre-training in the use of computers as a teaching/learning tool; when there are multiple opportunities for learning; when the student is in control of learning, when peer learning is optimized; and when feedback is optimized.


Adjunct aids Adjunct aids such as animation and illustrations can assist learning when they function to attract and direct attention, when they highlight main ideas and comprehension, and when the text assists readers to see details in the pictures.

73 .37 curricula Bilingual programs Bilingual programs are programs where two languages are used as a medium for instruction. A meta-analysis of a Santa Fe Bilingual Education program found the initial impact was greater in the early grades, and had a significant effect on mathematics achievement, while reading gains were small.


74 .36 school Principals/school leaders

Two major forms of leadership include instructional and transformational leadership. Evidence supports the power of the former over the latter in terms of student outcomes. School leaders who promote challenging goals, and then establish safe environments for teachers to critique, question, and support other teachers to reach these goals together that have most effect on student outcomes. Leaders who focus on student achievement and instructional strategies are the most effective. Leaders who place more attention on teaching and focused achievement domains have higher effects.

75 .36 student Attitude to mathematics/science

76 .36 curricula Exposure to reading Parent-preschooler joint book reading experiences were shown to be related to positive outcomes in language growth, emergent literacy, and reading achievement. Reading to children, however, is not sufficient to lead to competent readers- instruction is also needed.

77 .35 curricula Drama/arts programs Creative drama activities have positive effects on children’s achievement at elementary grade levels in oral language skills, self-esteem, moral reasoning, role-taking abilities, and drama skills.

78 .35 student Creativity 79 .34 teaching

practices Frequent testing/ effects of testing

Repeated testing is only effective if there is feedback from the tests to teachers such that they modify their instruction to attend to the strengths and gaps of student performance. Although performance is increased with more frequent testing, the amount of improvement in achievement diminishes as the number of tests increase.

80 .34 school Decreasing disruptive behavior

Teachers need skills to ensure no student unnecessarily disrupts their own or the learning of any other students in class. Targeting classroom disruptions via a behavioral approach is the most efficacious. The most successful programs included social or token reinforcement, cooperation, behavioral consultation, and cognitive behavior modification; the least successful involved social skills training.

81 .33 student Drugs 82 .33 teaching

practices Simulations Simulations and gaming had a small positive effect over

alternative instructional techniques, although somewhat larger effects when compared with lecture only. Simulations are more effective with lower rather than high ability students. Simulation and gaming had higher effect sizes in achievement when used with students in higher grade levels.


Inductive teaching Induction is usually described as moving from the specific to the general. In science teaching, results indicated it makes no difference if educational experiences are provided to students prior to formalizing generalizations, or if generalizations are formulated prior to any illustrative examples.


84 .32 student Positive view of own ethnicity

85 .32 teacher Teacher effects See individual teacher items for more description. 86 .31 teaching

practices Inquiry based teaching Effect sizes of inquiry teaching are greater on process than on

content, and higher for biology and physics than for chemistry. Effects were greatest at elementary level and decreased as students progressed through their school years. Results indicate inquiry based instruction can foster critical thinking which was transferable, improved achievement, and improved attitude towards the subject.

87 .30 school Ability grouping for gifted students

Gifted students, when placed in special, homogeneous classes with challenging curricula, achieved more than gifted counterparts in regular classes.


Homework The positive effects of homework are higher in secondary than junior high and higher in junior high than in elementary. The positive effects of homework were negatively related to the duration of the homework. Shorter is better, but for elementary students, the correlation is near zero between time spent on homework and achievement. Effects were higher when the material was not complex or if it was novel. Homework involving higher level conceptual thinking and project based was the least effective. Effects are greater for higher than for lower ability students and for older rather than younger students.

89 .29 home Home visits 90 .28 student Exercise/relaxation 91 .28 school Desegregation Some studies found students in desegregated schools

performed at higher achievement levels than students in control groups. Others found virtually no effect from desegregation on the achievement of African American students in reading and mathematics.

92 .28 school Mainstreaming Mainstreaming is the concept that students with disabilities should be integrated with their non-disabled peers to the maximum extent possible, and certainly placed in the least restrictive environment. It is often argued more on equity and social justice reasons than in terms of optimal effects on learning. There are small but positive advantages for mainstream over special classes, for achievement, and for social/personality outcomes.

93 .27 curricula Use of calculators The calculator is neither a help nor a hindrance to students’ overall mathematics achievement, and at minimum their use helps reduce cognitive load and enhance students’ attitudes towards the study of mathematics. Across all grades and ability levels, students using calculators led to greater effects in students’ basic skills in operations and particularly in problem solving.

94 .24 curricula Values/moral education programs

The effects of social skills programs are stronger on enhancing peer relations and social outcomes; lower when students are identified as social problems; and lowest when academic achievement is the outcome of the social skills


programs. Most programs resulted in short term gains, indicating a need to provide social skills training on a regular and sustained basis.


Programmed instruction

There are very low effects from programmed instruction. When compared to other teaching methods, programmed instruction often comes near the bottom.


Special college programs

Remediation programs for college students had higher effects for new programs and weaker effects in institutionalized programs. Thus, colleges seem more proficient at setting up programs for high-risk students than they were at keeping these programs going.


Competitive vs. individualistic learning

Competitiveness can be toward beating a standard- either a personal best standard or a standard of the curriculum. Cooperative learning has higher effects than competitive, and both are superior to individualistic learning.

98 .23 school Summer school Going to summer school does not make much of a difference, but it is difficult to ignore the small gains if they are critical to students who may be already marginal. In studies, higher achievers gained more than lower achievers, and middle and higher SES students gained more than lower SES students.

99 .23 school Finances One claim is that there is no consistent statistical relation between educational expenditure and measures of student performance. The emphasis of research may need to be not on the notion of “Does money make a difference?” but on “How does money make a difference, particularly beyond the fixed costs of running a school?”


Individualized instruction

An individualized teaching system had only a small effect on student achievement in high school courses. There was limited contribution to student self-esteem, critical thinking ability, or attitude toward subject matter taught when using individualized programs.

101 .23 school Religious schools The dividend of attending religious schools is supposedly greater for those from lower socioeconomic background, and effects are slightly stronger for high schools and middle schools.

102 .23 student Lack of illness 103 .22 teaching

practices Teaching test taking Coaching is instruction given or practice undertaken in

preparation for taking a test. While coaching did raise SAT scores, the length of the training seems important. Programs continuing for 5 weeks or more produced better effects than shorter ones, the effects were higher with students in upper rather than lower grades, and higher for students from lower socioeconomic backgrounds.


Visual/audio-visual methods

Very small effects resulted from use of television, film, slides, and tapes. Providing audio tapes of lessons had a small overall effect on student achievement in college courses.


Comprehensive teaching reforms

Comprehensive teaching reform programs, such as Success for All, Core Knowledge, and Accelerated Schools, are often “imposed” on schools in an effort to reduce the achievement gap. Their low effects provide little confidence these


programs alone will reduce the achievement gap between at-risk students and their more advantaged peers.

106 .21 school Class size It appears the effects of reducing class size may be higher on teacher and student work-related conditions, which then may or may not translate into effects on student learning. Merely reducing the number of students in front of teachers appears to change little- in teaching and in outcomes.

107 .20 school Charter schools It is striking how little we currently know about the effect of charter schools on student achievement. There is close to zero effect when comparing achievement in charter and regular schools.


Aptitude/treatment interactions

This refers to instruction being altered for different types of students. Researchers noted existing studies cannot be used with any confidence to construct general principles of instructional design.

109 .19 student Personality 110 .19 teaching

practices Learning hierarchies A different form of learning intention is to structure learning

in some form of hierarchy, such that it is more effective to acquire first a set of skills that will support later learning. Overall effects are very low, but it is more effective in promoting learning at the elementary level than at high school.

111 .19 teaching Co-teaching/ team teaching

Little research is available on the effects of team teaching.


Web-based learning Use of the world wide web is a fairly recent phenomenon in our classrooms, so little research is available to analyze its merits at this time.

113 .17 home Family structure 114 .17 curricula Extra-curricular

programs If we wish students to enhance achievement, extra-curricular activities relating to academic types of skills is optimal, and sport has the least effect on most outcomes. The greatest effects on achievement came from participation in school-based extracurricular activities, then pro-social activities such as scouting, with performing arts having the least effect on all outcomes.


Teacher immediacy The immediacy and closeness of responses to students shows them that teachers are listening and responding. The effects of teacher immediacy were much stronger on affective learning such as attitudes toward the teacher or course, or engaging in the learning experience, than on achievement.

116 .16 school Within class grouping There are two major forms of in class grouping: ability/achievement grouping and small group learning which is more spontaneous and usually for specific tasks over a shorter time period. The mean effects size of within class grouping is low overall, but slightly more advantageous for high ability students and less for low ability students. In classes with over 35 students, the mean effect of grouping is .35.


Home-school programs Use of technology to develop home-school connections in student learning has limited effects on reading, slightly higher


for mathematics, and is highest for writing. 118 .15 teaching

practices Problem-based learning

With this topic, it is important to separate the effects on surface and deep knowledge and understanding. For surface knowledge, problem-based learning can have limited and even negative effects; for deeper learning, when students have the surface level knowledge, it can have positive effects.

119 .15 curricula Sentence combining programs

Sentence combining is an instructional activity that requires students to combine sentences to create compound, complex, or compound-complex sentence. The effects are small, but more effective at the elementary than high school levels.


Mentoring Mentoring usually involves older persons providing academic or social assistance to younger people. It has close to zero effect on performance outcomes, but does have higher effects on attitudes, motivation, and involvement.

121 .12 school Ability grouping Results show that tracking had minimal effects on learning outcomes and profound negative equity effects. Many low track classes are deadening, non-educational environments. Low track classes are more fragmented, less engaging, and taught by fewer well-trained teachers.

122 .12 student Gender 123 .12 student Diet 124 .11 teacher Teacher training The effect size of teacher education on subsequent student

outcomes is negligible, although the effect on specific skills is quite high.

125 .09 teacher Teacher subject matter knowledge

There is not a large corpus of evidence to defend the thought that teaching begins with a teacher’s understanding of what is to be learned and how it is to be taught.


Distance education Results indicate similar effects between distance programs and traditional face-to-face classroom programs.

127 .09 school Out of school curriculum experiences

The more successful out of school programs were shorter rather than longer, involved one-on-one tutoring, and were for students from lower elementary and high school. The overall effects are negligible compared to what effective teachers can attain in regular classrooms using many other methods of instruction.

128 .08 curricula Perceptual-motor programs

Perceptual motor training is an intervention more often used with learning-difficulty students. Overall, they are not effective in improving academic or cognitive learning and there were no major improvements associated with perceptual/sensory outcomes.

129 .06 curricula Whole language Whole language programs have negligible effects on learning to read. Low socioeconomic status children receiving basal readers did consistently better than their counterparts receiving whole language instruction.

130 .05 school College halls of residence

It did not matter whether a student lived in a college, at home, in a fraternity or sorority house, or in off-campus housing.

131 .04 school Multi-grade classes Overall, the effects from multi-grade classes compared to single-age classes are not compelling enough to argue for the


effectiveness of one over the other. It is likely teachers teach in a similar way regardless of the distribution of age range in the class.


Student control over learning

The effect of student choice and control over learning is somewhat higher on motivation outcomes than on subsequent student learning.

133 .01 school Open vs. traditional Too often, classroom architecture may be open, but that is no guarantee that the principles of open teaching are present. Students performed slightly better on achievement tests in traditional compared to open teaching, but did worse on tests of creativity and had slightly less positive attitudes and self-concepts.

134 -.09 School Summer vacation Students lost some achievement gains over the summer, and negative effects were slightly larger in mathematics than in reading and language.

135 -.12 home Welfare policies 136 -.16 school Retention Retention has a negative effect on academic achievement in

language arts, reading, mathematics, work-study skills, social studies, social studies, and GPA. There are also negative equity effects- it is four times more likely students of color will be retained. Also, the threat of non-promotion is not a motivating force for students; retention does not generally improve achievement or adjustment for developmentally immature students, and increases the risk of dropping out of school twofold.

137 -.18 home Television 138 -.34 school Mobility The effect of student mobility between schools is quite

marked. The effects on reading and mathematics are negative. The most important reason for these declines relates to peer effects. Whenever there is a major transition in schools, the key success factor is whether a child makes a friend in the first month. It is incumbent for schools to attend to student friendships and ensure the class makes newcomers welcomed.

stephen m. ventura senior professional development

Documents