Jamila Jones KennedyEDUC 872

April 7, 2011

Schools are intended to provide distinct learning opportunities in multiple academic disciplines. Yet, the opportunities to learn the academic content are not equally distributed among districts, schools, classrooms or students.

Related to this, students’ performance varies across the country in ways that suggest certain regions and demographics lead to higher performance than others (achievement gap).

Typically lost within the achievement gap debate is the “opportunity to learn” (OTL) concept.

OTL as a concept: What students learn in school is related to what is

taught in school Whether students are provided the opportunity to

learn what is expected of them, especially the information for which they will be held accountable

The capacity of schools to provide adequate learning opportunities for all students

OTL used to encompass various ideas, including: 1)equal/equitable per-pupil funding, 2)adequate school organization, 3)equal access to qualified teachers, challenging curricula, and resources

As a research concept, OTL was first introduced in the 1960sCarroll’s model (OTL as a function of time)FIMS (OTL as content coverage w/o regard

to time) By the mid-1980s, the notion of OTL had

gone through revisionsSIMS (OTL in terms of curriculum)

More current research (in the 1990s and 2000s) focuses on OTL in terms of teacher qualifications, access to resources, funding, teaching method, etc.

Author: Marta ElliottSource: Sociology of Education, Vol. 71, No. 3 (July 1998), pp. 223-245

Research Questions:1. Do educational expenditures affect students’

achievement?2. What components of OTL affect math and

science achievement?3. If funds are allocated for the most critical

components of OTL, do students learn more?

Focused on effects of expenditures on:1. Teacher effectiveness2. Classroom resources

Data: U.S. Census Survey of Local Government

Finances for School Systems and NELS:88 Dependent variable – 10th grade

math/science item response theory (IRT) theta score

Independent variables: Student-level controls – SES, race, gender School expenditures – cost of instruction, staff and

student support services School-level controls – student composition,

school size, urban or not OTL – teacher qualifications, class size, pedagogy,

classroom resources

Sample:N=6,318 for math analysis; N=5,343 for

scienceTreatment of missing data with mean

values and dummy variables Method:

Hierarchical linear modeling (HLM)1. 8th-grade achievement and student-level

controls2. Expenditures and school-level controls3. Teacher qualifications and class size4. Teaching emphasis and classroom resources

Results – Math Achievement:Expenditures are significantly related to

higher achievement in math (p=.018)Students who achieve higher math scores

have more educated(p=.000) and experienced (p=.007) teachers and tend to be in larger math classes (p=.000).

Math scores were higher when teachers emphasized higher-order thinking (p=.000).

The use of calculators appeared to improve math achievement (p=.000), whereas the use of computers tended to decrease math achievement (p=.034).

Results – Science Achievement:Expenditures are significantly related to higher

achievement in science(p=.018)Only teacher’s education is related to student’s

science achievement (p=.000)Teachers’ emphasis on inquiry skills (p=.000)

and the condition of science equipment are significantly related to science achievement (p=.038)

No interactions between expenditures and student controls (SES, gender, etc) on math or science achievement were significant

Good Apples Bad ApplesSeeks to specify the path through which dollars translate into learning

Although significant, coefficients were very small

Thoughtful lit review; narrowly focused on OTL and spending

Uses survey data (NELS); creates teacher self-report issues

Relevant data sources No explanation of item response theory (IRT) theta score

Nice (novel?) treatment of missing data

Replicable; includes appendix w/ all variable measures

Good description of data, methods

Implications and further research ideas in conclusion

Discussion of limitations and how they were addressed

Policy Implications:A primary concern is to identify the

mechanisms through which money can effectively enhance learning.

Financial data collected at the school level is needed to further study the effects of expenditures on achievement. District level data are not the best estimate since it is differentially allocated across schools.

Author: Lisa Scherff, Carolyn PiazzaSource: Journal of Adolescent & Adult

Literacy, 52(4), Dec 2008/Jan 2009, pp. 343-352

Purpose: To examine OTL from the perspective of

students, the one, group of voices seldom heard from in discussions of quality literacy education.

Sample:More than 3,000 students in grades 9-12 in

four public high schools throughout Florida. Method:

Surveys administered by English teachers during last 3 weeks of the school year

56% response rate, n = 1,801

Instrument:Based on Standards for the English

Language Arts, co-written by IRA and NCTE Items measured:

1. Perceived Access – the right to participate2. Exposure (the number of opportunities to

participate) to: a) Content (reading, writing, speaking)b) Curricular tasks (collaborative work)c) Materials (novels, technology)

Results:Survey uncovered 3 problems:

1. Systems – OTL is largely an individual student phenomenon rather than a collective and systematic one

2. Offerings – OTL is often constrained by course and experiential offerings for students & the amount of time spent on certain literacy activities

3. Acknowledgment - If opportunity is something defined for students, and not with them, then decisions are made and goals are set without them having a say in the matter

Good Apples Bad ApplesConcluding remarks were on point! Arguments were convincing

Survey results are not used as support for the “Acknowledgments” finding

Included some limitations Bad use of citations (used to support findings)

Included reference to full survey

No context for findings (X% of students)

Lacks basic survey statistics (no data on item (non)response, frequencies, means, etc)

Used examples as “findings”

No analysis of survey data. Coding? Inter-rater reliability?

Lack of info calls accuracy and validity of findings into question

Policy Implications:Student opinions and perceptions can

contribute relevant and necessary information

For curricular standards to be accurately measured by mandated assessments, educators must ensure that students have more choice, ownership, and commitment to education

Authors: Christy Kim Boscardin, Zenaida Aguirre-Munoz, Ginger Stoker, Jinok Kim,

Mikyung Kim, and Janet LeeSource: Educational Assessment, 2005,

10(4), 307-332

Purpose: To examine whether and how various OTL

variables significantly impact outcomes and whether these effects are consistent across different content areas

Researchable Questions:1.What are various OTL variables that impact

student achievement? How are they differentially impacting student performance?

2.Is the impact of OTL variables consistent across different subject areas?

3.What student background characteristics are related to student achievement?

Sample:Tests administered to all students enrolled

in an English and/or Algebra course in schools within a large urban school district in the Midwest during the spring semester of the 2000-01 school year

Data used: Scores from the 4,715 students taught by 118

English teachers who completed the survey Scores from the 4,724 students taught by 124

Algebra teachers who completed the survey

Instrument: Teacher OTL Survey, developed by the

UCLA Natl Center for Research on Evaluation, Standards, and Student Testing (CRESST) & content experts

5 major sections related to aspects of OTL1. Teaching experience2. Teaching expertise in content topics3. Topic coverage4. Classroom activities5. Assessment strategies and preparation

Method: Two-level Hierarchical Linear Model

Student level – explores relationships between test scores and course grades, gender within teachersVars: course grade, gender, minority status,

lunch status Teacher level – explores relationships between avg

test scores from each class, teacher expertise, time spent covering content, and free/reduced lunch status of the classVars: teacher expertise, time spent on subject

area-specfic topics, average class free/reduced lunch

The other three OTL variables – teacher experience, classroom activities, assessment strategies – were not significant predictors and not considered in the final model

Results:Gender and course grade were significant

predictors of Algebra and English test scores, whereas ethnicity and free/reduced lunch status were not. Course grade positively related to Algebra and

English test scores (p<.00)Students earning As in Algebra and English

were expected to answer 7 and 4 more items correctly, respectively

Males outperformed females in Algebra; females outperformed males in English (p<.00)Average differences in Algebra scores - 0.42

points; English scores - 0.77 points

Results (cont.):Teacher expertise, time spent on content

areas, & avg free/reduced lunch status were significant predictors of Algebra and English test scores Students taught by “expert” teachers answered

3 and 4 more test items correctly, respectively Students of teachers who spent more time on

relevant content scored 0.85 points higher on Algebra and 1.59 points higher on English

Teachers with 10% more students on free/reduced lunch expected to have 1.14 point lower Algebra score and 1.20 point lower English score

Good Apples Bad Apples

Nice summary of importance of the study

Data in tables and text do not align. Which is correct?

Lit review focused on more recent work

Limitations of survey data (self-reporting)

Acknowledges limitations Participating district helped develop the end-of-course exam

Included relevant statistics in easy to read tables

No information on the teachers who took the OTL survey

Appendices describe models, variables, and effects

No information on exam or survey questions

Policy Implications:Before teachers are held accountable, they

need appropriate training and resources to ensure they are prepared to teach and see themselves as experts in their content areas

Recruit more qualified teachers. Local districts can require higher standards for teachers.

Content standards aligned with assessment, instructional goals, and professional development

Author: Aubrey H. Wang

Source: Early Childhood Education Journal , 2010, 37, 295-302

Research Questions: 1. Do African-American and Caucasian

kindergartners from low-income families have differential opportunity to learn mathematics?

2. To what extent do opportunities to learn mathematics predict math achievement for African-American and Caucasian children from low-income families?

Data: Early Childhood Longitudinal Study,

Kindergarten cohort (ECLS-K) Survey data from 48 items on math OTL

Participants:1,721 first time kindergartners in fall 1998

who lived below the poverty line Method of Analysis:

Factor analyses instructional time, instructional method,

instructional emphasis Independent t-tests

differences at entry and exitSimultaneous multiple regression

Dependent var. = math achievement Independent var. = instructional time, method,

emphasis (or OTL math)

Results of t-tests:Significant differences in math achievement

at kindergarten entry (p<.001) and exit (p<.001) with AA scoring lower than Caucasians

Significant differences in instructional time (p<.05) with low-income AA students having more than Caucasians

Significant differences in 3 types of instructional methods Use of manipulatives higher for AA (p<.001) Use of math worksheets and textbooks more

frequent for AA (p<.01) Use of explanation and real-life math more

frequent for AA (p<.001)

Results of t-tests (cont.): Significant differences in 2 aspects of

instructional emphasis Telling time and using measurement tools

accurately higher for AA (p<.05) Reading graphs and doing simple data collection

lower for AA (p<.05)

Regression Results: OTL predicts math achievement for:

Both races of students (p<.01, R2 = .62) Low-income AA kindergartners (p<.001, R2 =.59) Low-income Caucasian kindergartners (p<.001,

R2=.60)

Good Apples Bad Apples

Discusses limitations No info on 48-item teacher survey

Effective use of data tables Limitations of survey data (self-reporting)

Appendix defines models used, description of variables and relevant statistics

Nothing new or novel about the study; what’s its contribution?

Thorough lit review

Importance of topic is apparent

Policy Implications:Kindergarten teachers of low-income

students need to balance the math curriculum with more emphasis on higher order math content and less on manipulatives and math games

More current data collection efforts are needed; ECLS-K data are more than a decade old

Results make a case for increasing funding, but funding alone doesn’t explain why some students have higher achievement than others.

Given high stakes, need to ensure equitable access to educational opportunities before holding teachers, schools, and students accountable

OTL is difficult to measure. It requires information on the day to day activities of teachers and students, which is costly.

Need a national data set that includes measures of school finance, OTL, and student achievement. We also need data on the proportion of funds allocated to various needs so we can determine directly the effects of how money is spent.

Use OTL as part of the accountability system (rather than just student performance as an indicator of progress); may lead to the types of instruction and reform efforts that will ultimately yield desirable outcomes for all students