high-stake testing as a barrier to technology …

HIGH-STAKE TESTING AS A BARRIER TO TECHNOLOGY INTEGRATION

___________________________________

By

JEFFREY ALAN MATTY

___________________________________

A DISSERTATION

Submitted to the faculty of the Graduate School of Creighton University in Partial

Fulfillment of the Requirements for the degree of Doctor of Education in

Interdisciplinary Leadership

_________________________________

Omaha, NE

July 30, 2015

Copyright 2015, Jeffrey Alan Matty

This document is copyrighted material. Under copyright law, no part of this document may be reproduced without the expressed permission of the author.

iii

Abstract

The purpose of this study was to analyze the lesson plans of high school teachers for

technology integration in high-stake tested and non-tested contexts. The aim of this

research was to provide information and recommendations to educators of the district

concerning the planning of lessons and integration of technology in high-stake subject

contexts. The data collected provided information regarding a teacher’s planning of

lessons that integrated technology in high-stake tested and non-tested subjects. A

TPACK-Based Technology Integration Assessment Rubric was used to evaluate the

lesson planning of 435 teachers in English and Science subjects in either a high-stake

tested or non-tested context. ANOVA testing was completed to measure statistically the

differences among the lesson planning within the same subject area and context while t-

tests were completed for comparison between high-stake tested and non-tested subjects

for Science and English. The results of the study indicated that technology integration

was influenced by context when comparing high-stake tested Biology with non-tested

Science subjects. In contrast, results between high-stake tested and non-tested English

subjects did not support the hypothesis that a high-stake tested context was a barrier to

technology integration. Based on these results, a Six-Step Growth Design Process was

developed to further investigate the influence of subject and individual teacher planning

habitus upon the high-stake context barrier to technology integration. The Six-Step

Growth Design Process will be implemented to increase technology integration in the

classroom and improve its use in different contexts. The process will allow educators to

examine the application of technology and reflect upon instruction.

Keywords: TPACK, Six-Step Growth Design Process

HIGH-STAKE TESTING

iv

Acknowledgements

First, I would like to express my gratitude to my committee chair, Dr. Barbara

Brock, for her guidance, patience and support during the completion of this

dissertation. Also, I would like to thank Dr. Peggy Hawkins and Dr. John Hudson, II

for their time, efforts, and support of this research project.

Second, I would like to thank Dr. Judi Harris, Dr. Neal Grandgenett, and Dr.

Mark Hofer for their permission to use their rubric in this research. In addition, I

would like to thank Dr. Karen Polkabla for her permission to use lesson plans.

Third, I would like to thank my family, colleagues, and friends for their

constant support during my academic career and pursuit of a doctoral degree. Above

all, I salute Albert and Margaret Matty, my parents, who have always taught the values

of grit, compassion, and doing your best. Lastly, thank you Junko and Belle for putting

up with my research and allowing time for the journey to the doctoral degree.

HIGH-STAKE TESTING

v

Table of Contents

Abstract .............................................................................................................................. iii

Acknowledgements ............................................................................................................ iv

List of Tables ..................................................................................................................... ix

Table of Figures .................................................................................................................. x

CHAPTER ONE INTRODUCTION .................................................................................. 1

Background of the Problem ................................................................................................ 1

Statement of the Problem .................................................................................................... 6

Purpose of the Study ........................................................................................................... 8

Aim the Study ..................................................................................................................... 9

Significance of the Study .................................................................................................... 9

Research Questions and Hypotheses ................................................................................ 10

Methodology Overview .................................................................................................... 13

Definition of Terms........................................................................................................... 13

Assumptions ...................................................................................................................... 14

Delimitations and Limitations ........................................................................................... 15

Summary ........................................................................................................................... 16

CHAPTER TWO: LITERATURE REVIEW ................................................................... 18

Introduction ....................................................................................................................... 18

Purpose of the Study ......................................................................................................... 18

Aim the Study ................................................................................................................... 19

Educational Accountability and High-Stake Testing ........................................................ 19

The TPACK Framework and Barriers to Integration ....................................................... 24

Teacher Context and Style ................................................................................................ 31

Conceptual Frameworks ................................................................................................... 33

Assessing TPACK and Lesson Planning .......................................................................... 37

Summary ........................................................................................................................... 43

CHAPTER THREE: METHODOLOGY ......................................................................... 44

Introduction ....................................................................................................................... 44


Aim of the Study ............................................................................................................... 44

Research Questions and Hypotheses ................................................................................ 45

HIGH-STAKE TESTING

vi

Research Design................................................................................................................ 47

Samples and Participants .................................................................................................. 48

Instrument Rubric ............................................................................................................. 49

Materials ........................................................................................................................... 51

Lesson Plan Format........................................................................................................... 51

Quantitative Variables ...................................................................................................... 53

Data Collection Procedure ................................................................................................ 53

Data Analysis Plan ............................................................................................................ 53

Ethical Considerations ...................................................................................................... 54

Summary ........................................................................................................................... 55

CHAPTER FOUR: FINDINGS AND THE EVIDENCE-BASED SOLUTION ............. 56

Introduction ....................................................................................................................... 56


Aim the Study ................................................................................................................... 57

Data Analysis Procedures ................................................................................................. 57

Research Questions ........................................................................................................... 58

Analysis of Data ................................................................................................................ 60

Results for Research Questions ........................................................................................ 61 ANOVA Testing for High-Stake Tested English Lesson Plans ....................................61 ANOVA Testing for Non-tested English Lesson Plans .................................................61 ANOVA Testing for Tested Science Lesson Plans .......................................................62 ANOVA Testing for Non-tested Science Lesson Plans ................................................62 t-Test English .................................................................................................................62 t-Test Science .................................................................................................................63

Intra-rater Reliability Measure .......................................................................................... 67

Analysis and Synthesis of Findings .................................................................................. 67 Technology Integration Among High-Stake Tested English .........................................68 Technology Integration Among Non-Tested English ....................................................69 Technology Integration Between High-Stake Tested and Non-Tested English ............69 Technology Integration Among High-Stake Tested Biology ........................................70 Technology Integration Among Non-Tested Science ....................................................70 Technology Integration Between High-Stake Tested Biology and Non-Tested Science........................................................................................................................................71

Summary ........................................................................................................................... 74

CHAPTER FIVE: CONCLUSIONS AND RECOMMENDATIONS ............................. 76

Introduction ....................................................................................................................... 76

HIGH-STAKE TESTING

vii

Summary of the Study ...................................................................................................... 78


Aim the Study ................................................................................................................... 79

Proposed Six-Step Growth Design Process Solution........................................................ 79

Support for the Six-Step Process from Data Collected ..................................................... 83

Existing Support Structure and Resources ........................................................................ 85

Policies Influencing the Six-Step Process......................................................................... 86

Potential Barriers to the Six-Step Growth Design Process ............................................... 87

Budget and Legal Issues Related to the Six-Step Growth Design Process ....................... 87

Change Theory .................................................................................................................. 88

Internal/External Issues Related to the Six-Step Growth Design Process ........................ 89

Implementation of the Six-Step Growth Design Process and Considerations .................. 89 Step One: Introducing a New Lesson Plan ....................................................................90 Step Two: Best Practice Teaching .................................................................................90 Step Three: Lesson Plan Creation ..................................................................................90 Step Four: Reflection .....................................................................................................90 Step Five: TPACK Rubric ............................................................................................91 Step Six: Collaboration and Rubric ...............................................................................91

Roles and Responsibilities of Key Players in Implementation ......................................... 91

Leader’s Role in Implementing the Six-Step Growth Design Process ............................. 91

Evaluation and Timeline for Implementation and Assessment ........................................ 92

Convincing Others to Support the Six-Step Growth Design Process ............................... 92

Critical Pieces Needed for Implementation and Assessment ........................................... 93

Internal and External Implications for the District ........................................................... 93

Considerations for Leaders Facing Implementation ......................................................... 93

Evaluation Cycle ............................................................................................................... 94 Step 1: Effectiveness of New Lesson Plan Presentation ................................................94 Step 2: Best Practice Implementation ............................................................................94 Step 3: Lesson Plan Creation .........................................................................................94 Step 4: Teacher Reflections ...........................................................................................94 Step 5: TPACK Rubric ..................................................................................................94 Step 6: Teamwork Discussions ......................................................................................95

Implications for Action and Recommendations for Further Research ............................. 95

Summary of Chapter Five ................................................................................................. 97

References ......................................................................................................................... 98

Appendix A ..................................................................................................................... 110

HIGH-STAKE TESTING

viii

Appendix B ..................................................................................................................... 112

Appendix C ..................................................................................................................... 116

HIGH-STAKE TESTING

ix

List of Tables TABLE 1 MEANS AND STANDARD DEVIATION FOR TECHNOLOGY INTEGRATION AMONG

ENGLISH TEACHERS ............................................................................................................... 64 TABLE 2 MEANS AND STANDARD DEVIATION FOR TECHNOLOGY INTEGRATION AMONG

SCIENCE TEACHERS ................................................................................................................ 65 TABLE 3 INTRA-RATER RELIABILITY .............................................................................................. 66

HIGH-STAKE TESTING

x

Table of Figures

FIGURE 1 TPACK FRAMEWORK ........................................................................................ 26 FIGURE 2 SIX-STEP GROWTH DESIGN PROCESS ................................................................. 85 FIGURE 3 EVALUATION CYCLE TIMELINE .......................................................................... 95

HIGH-STAKE TESTING

1

CHAPTER ONE INTRODUCTION

Background of the Problem

Twenty-first-century teachers confront the task of educating all types of students

for economic global competition and academic success in a world of new technologies

and public oversight of education. Accordingly, Dufour and Marzano (2011) reported

that teachers are expected to raise the academic success of all students by using core

curriculum to compete globally at the highest level in history. Additionally, teachers and

schools are subject to public accountability based on their students’ test scores and their

abilities to prepare all students for college and career readiness. For example, on

November 7th, 2013, the Washington Post reported the findings of the Nations Report

Card for fourth and eighth-grade students in reading and mathematics. According to

Layton (2013), students scored higher than ever on the National Assessment of

Educational Progress (NAEP) with an incremental increase from the previous year;

however, a large gap still existed between the achievement of white students and they’re

fellow black and Latino classmates despite many years of legislation aimed at narrowing

the gap in student achievement.

In addition to improving student achievement in standardized testing, teachers are

to incorporate new technologies providing skills for future college and career readiness.

Specifically, students need new technology introduced and integrated into their K-12

education for preparation of career fields such as health care, business, engineering, and

manufacturing. Nonetheless, some observers of technology use in education believe

technology is not being integrated into teaching. The New York Times reported that the

Center for American Progress had developed a report from NAEP data questioning the

HIGH-STAKE TESTING

2

value of technology investment in schools (Rich, 2013). Boser (2013) concluded through

2009 and 2011 NAEP survey data that the use of technology by students in our schools

for basic skill acquisition is lacking for many, especially students in poverty.

Consequently, teachers face the tasks of integrating technology into their teaching while

preparing students for high-stake test success. Moreover, teachers face these tasks for all

despite differences in student learning abilities and teacher experience with technology in

the classroom.

As a result of student achievement expectations, teachers are now subject to new

systems of professional evaluation based partly on student test scores. According to

Ravitch (2013), bipartisan political support exists for the use of student test scores as a

basis for professional teacher evaluation and job retention. Assessing student

achievement through standardized testing has gathered momentum since the inception of

the No Child Left Behind legislation enacted in 2001. As a result, teachers must prepare

students for success on standardized tests and be subject to public and professional

scrutiny over the results. By contrast, many educators believe that focusing on

standardized testing will not prepare students for the college and career preparation

needed for the twenty-first century. For example, technology knowledge is foundational

for the students of this century in order to synthesize digital information and

understanding (Kereluik, Mishra, Fahnoe, & Terry, 2013); as a result, preparing students

for technology acquisition and application to real-world problems is paramount for

educators.

As technology use has increased in schools, traditional teacher-centered ways of

teaching and learning are becoming less used in many respects. For example, students

HIGH-STAKE TESTING

3

have access to content knowledge and general facts through the Internet and not just the

teacher. As a result, teachers provide their students with information through classroom

blogs, Google searches and flipped classrooms. Furthermore, some teachers are focusing

on student-centered approaches to instruction such as project-based learning that move

beyond content acquisition toward critical thinking and twenty-first-century skill

application. According to Gunn and Hollingsworth (2013), students are expected to think

critically while applying and evaluating learned knowledge in different situations that

differ greatly from traditional memorization and repetition thinking. One of the ways

teachers can incorporate these twenty-first-century skills into their teaching is by using

technology.

As part of the process of enhancing student learning, some educators have

realized the importance of technology in classrooms and the importance of technology to

students. For example, technology may be integrated into the learning process through

projects. According to Bell (2010), project-based learning is an approach to learning that

utilizes technology for the research and presentation phases of student work while

promoting the twenty-first-century skills of collaboration and problem solving.

Additionally, student engagement is enhanced when utilizing technology. According to

Taylor and Parsons (2011), students want a choice in their learning that includes

technology used for exploration of events and collaboration with experts. Moreover,

Sheehan and Nillas (2010) explained that students of mathematics find technology a

major contributor to interesting lessons and a bridge to real-world connections.

Comparatively, student interest in learning is enhanced through technology use because it

is a major part of their lives outside of school. For example, students aged 15-18 spend

HIGH-STAKE TESTING

4

one and a half hours sending and receiving texts each day as part of their seven and one-

half hours per day of media consumption (Rideout, Foehr, & Roberts, 2010).

Accordingly, teachers can apply technology to further student motivation for learning.

As the importance of technology in society and work has increased, schools

throughout the United States have purchased computers, iPads, interactive whiteboards,

and implemented BYOD (Bring Your Own Device) programs, online courses, and

flipped classrooms into the educational curriculum and classroom. Although investment

in technology has increased across school districts, technology use by teachers in the

classroom has faced many barriers. Extrinsic or first-order barriers to technology use

include access to technology, time for planning, and technical support, while intrinsic or

second-order barriers are teacher beliefs, unwillingness to change, and classroom

practices (Ertmer, 1999). Furthermore, Lim, Zhao, Tondeur, Chai, and Tsai (2013)

emphasized that school organizations must spend a tremendous amount of money on

technology by maintaining software and hardware while facing taxpayer expectations for

increased student achievement commensurate with the money invested. Overall,

educators must tackle a multitude of internal and external barriers to integrating

technology successfully in the classroom. As a result, educators need standards or

frameworks to assist in the pursuit of overcoming barriers to technology integration.

The International Society for Technology in Education (ISTE) has developed

standards that school administrators and teachers can follow to support the use of

technology and technology integration. One standard entitled, “digital age learning

culture such as” is applies to the integration of technology by teachers from an

administrative perspective. Specifically, administrators are encouraged to ensure

HIGH-STAKE TESTING

5

instructional innovation, promote the frequent use of technology, provide learner-

centered environments, participate in global communities and most importantly, ensure

the effective practice of technology with the curriculum (ISTE, 2009). In comparison, the

first standard from ISTE developed for teachers uses their content knowledge, teaching

and learning, and technology to promote creativity, innovation, collaboration, and

reflection in solving real-world issues in face-to-face and virtual environments (ISTE,

2009). Overall, these standards assist educators in focusing upon the general importance

of technology use in the modern classroom. However, standards are not always flexible

or adaptable to the pedagogical or content needs of teachers. Consequently, an

alternative approach is merited.

A framework that integrates technology, pedagogy, and content knowledge

(TPACK) was designed to measure teacher understanding of technology integration

(Mishra & Kohler, 2006). TPACK assists teachers in understanding their integration of

content, pedagogy, and technology to create effective teaching with technology (Koehler

& Mishra, 2009). Classroom teachers must incorporate technology into their lessons

keeping in mind the importance of twenty-first-century skills and the academic

significance of standardized tests. To meet this difficult challenge, teachers must be

willing to incorporate technology in new ways of teaching that are very different from

traditional methods within a context of high-stakes testing in a new era of accountability.

The use of technology in one western Pennsylvania school district has increased

during the last decade. Superintendents and school boards have allocated resources to

purchase interactive whiteboards, iPads, and computers for classrooms and labs. Despite

the investment in technology, a direct correlation to innovative instruction has not been

HIGH-STAKE TESTING

6

studied or evaluated. According to Halverson and Smith (2009), technology has opened

up opportunities for revolutionary teaching and learning in schools, but schools continue

to use technology to measure and guide learning for existing pedagogy, curriculum, and

assessments. A need for an understanding of the effectiveness of technology use in

creating new teacher pedagogy, curriculum delivery, and assessment exists. Moreover,

technology integration is an area of lesson development that administrators need to

address if the technology is considered to be as important as pedagogy and curriculum in

teaching.

Statement of the Problem

Technology integration and high-stake testing have become important issues to

educators. Technology has become more prevalent in schools as educators realize the

importance of using technology for engaging students, teaching applicable skills, and

college and career readiness. For example, Heafner (2004) asserted that technology use

in social studies classes enhanced student engagement and motivated students in their

learning. Furthermore, educators have encouraged the utilization of technology as

essential for contributing to the learning of problem-solving and critical thinking in

different contexts (Saavedra & Opfer, 2012). Although technology use can be beneficial

to students, teachers face barriers to integrating technology in classrooms (Hew & Brush,

2007). In particular, the context of high-stake testing has made the test scores of students

most important while steering teachers’ pedagogy toward one of repetitious instruction

on isolated pieces of information and away from project-based inquiry (Blazer, 2011).

Teachers focus on improving test scores instead of fully integrating pedagogy and

technology with content.

HIGH-STAKE TESTING

7

As high-stake testing has been emphasized as most important for measuring

student achievement, teachers must prepare students with concepts and content that will

be evaluated by state standardized tests. Subsequently, teachers face the daunting task of

preparing lessons that assimilate curriculum standards tested on high-stake state tests

with needed technology skills for the twenty-first-century learner. As a result, the

context of high-stake testing is a possible barrier to technology integration.

Barriers to technology integration implementation were defined by Ertmer (1999)

as being extrinsically first-order or intrinsically second-order categorized. Extrinsic

barriers include access to technology and training while intrinsic barriers include teacher

beliefs and practices. As extrinsic barriers are beyond the direct control of the teacher,

intrinsic barriers are directly associated with the teacher. Su (2009) explained that many

schools are equipped with technology, leaving second-order barriers such as teacher

pedagogical and psychological beliefs as fundamental barriers to technology integration.

In order to overcome intrinsic barriers, such as teacher planning for technology

integration, and extrinsic barriers such as a testing context, a theoretical framework called

TPACK (technology, pedagogy, and content knowledge) can provide an avenue for

analyzing the integration of technology in different classroom contexts. TPACK

combines teachers’ knowledge of technology, knowledge of their teaching content, and

knowledge of their pedagogy or instruction for an understanding of how all this

knowledge interacts (Koehler & Mishra, 2009). Archambault and Crippen (2009)

reported in their online teacher study that the TPACK framework organizes high-quality

instruction with technology and the relationships between technology, content, and

pedagogical knowledge.

HIGH-STAKE TESTING

8

Koehler and Mishra (2009) emphasized that teaching with technology is a

complex and ill-structured problem across contexts. Teachers must become curriculum

designers skilled at incorporating technology into their subject matter in complex

educational contexts (Koehler & Mishra, 2009). For example, many Pennsylvania

secondary teachers must design lessons that incorporate technology into contexts of high-

stake testing. Secondary teachers of Algebra, Biology, and English must prepare students

for high-stake, course ending, state mandated Pennsylvania Keystone exams (Keystone

Exams, 2015) while trying to fit technology into their teaching. Keystone exams measure

student performance in these subject areas and are mandated by the state. By contrast,

teachers of non-tested subjects, such as Chemistry or History, may incorporate

technology without facing the added accountability of Pennsylvania Keystone state tests.

An analysis of the lesson plans of teachers can provide data regarding the technology,

pedagogy, and content knowledge of teachers in different teaching contexts.

Consequently, an analysis of the context of high-stake testing as a barrier to technology is

warranted.

Purpose of the Study

The purpose of this study was to analyze the lesson plans of high school teachers

for technology integration in high-stake tested and non-tested subject contexts.

Technology integration was examined through the use of a technology integration rubric

based on the TPACK (Technology, Pedagogy, and Content Knowledge) framework. The

study determined quantitative differences in technology integration of teachers’ lesson

plans in different contexts. A better understanding of the issues and barriers to technology

integration lesson planning can assist teachers and administrators to improve the use of

HIGH-STAKE TESTING

9

technology in classroom instruction. A better comprehension of lesson planning provides

teachers and administrators information to improve the design of lessons while

integrating technology, pedagogy, and content knowledge. Furthermore, analyzing lesson

plans can provide data regarding teachers’ habits and decisions concerning technology

utilization in different contexts.

Aim the Study

The aim of this research was to provide information and recommendations to

educators of the district concerning the planning of lessons and integration of technology

in high-stake subject contexts. The data collected provided information regarding

teachers’ planning of lessons that integrated technology in high-stake tested and non-

tested subjects. As a result, educators can reflect upon technology, pedagogy, and

content knowledge (TPACK) in various contexts, examine instruction, and plan lessons

in the future accordingly.

Significance of the Study

Teachers must contend with a variety of barriers when integrating technology into

their courses. Extrinsic or first-order barriers to technology use include access to

technology, time for planning, and technical support, while intrinsic or second-order

barriers are teacher beliefs, unwillingness to change, and classroom practices (Ertmer,

1999). Another possible barrier to the successful integration of technology is the context

of high-stake testing and the planning for technology integration. In this study, lesson

plans developed by high school teachers in one western Pennsylvania school district for

high-stake test and non-test subjects were examined. In an era of educational

accountability, data collected from high-stake testing contexts and planning are essential

HIGH-STAKE TESTING

10

for solving technology integration barriers. Moreover, educators can examine the effects

of context on technology integration with respect to technology, content, and instruction.

As a means of organizing and collecting information about technology integration,

TPACK is a framework available to educators for understanding their knowledge of

technology integration in the classroom. In the technology age, teachers need to learn

skills that allow for innovative, flexible, and creative teaching. Thus, teachers need new

teaching strategies that promote the integration of technology and an understanding of

how pedagogy or instruction and relevant content, relate to each other in various teaching

contexts. Consequently, conscientious educators are looking for ways to improve their

use of technology while meeting the demands of curriculum standards, assessments,

diverse students, and career readiness. Measuring TPACK through archived lesson plans

gives the teachers, administrators, and researchers insight into areas of strength and need

concerning technology. Also, TPACK data can contribute to teachers’ professional

development and practice benefitting students.

Research Questions and Hypotheses

This study analyzed the technology, pedagogy, and content knowledge (TPACK)

of teachers’ lesson plans in high-stake tested and non-tested subjects for the 2012-2013

school year. The research questions guiding the study were based on hypotheses that a

difference exists among and between high-stake test subjects and non-test subjects in

regards to technology integration planning. Research questions one through four guided

the study among subject lesson plans. Specifically, these questions were based on the

hypotheses that teacher lesson plans within the same subject and test context would not

differ in technology integration.

HIGH-STAKE TESTING

11

Research Question #1:

In regards to teacher lesson plans, was there a significant statistical difference in

technology integration for the high-stake tested subjects of English for 10th and 11th-grade

students among these teachers?

Hypothesis #1:

Lesson plans for the high-stake subjects of English for 10th and 11th-grade students did

not differ in regards to technology integration by teachers because of the high-stake test

context.



technology integration for the non-tested subject of English for 12th graders among these

teachers?

Hypothesis #2:

Lesson plans for the non-tested subject of English for 12th graders did not differ in

regards to technology integration by teachers because of the non-test context.



technology integration for the high-stake tested science subject of Biology among these

teachers?

Hypothesis #3:

Lesson plans for the high-stake science subject of Biology did not differ in regards to

technology integration by teachers because of the high-stake test context.

HIGH-STAKE TESTING

12



technology integration for the non-tested science subjects of Chemistry, Accelerated

Chemistry, and AP Biology among these teachers?

Hypothesis #4:

Lesson plans for the non-tested science subjects of Chemistry, Accelerated Chemistry,

and AP Biology did not differ in regards to technology integration by teachers because of

the non-stake test context.

Second, research questions five and six guided the comparison between high-

stake tested subjects and non-tested subjects for technology integration in teacher

planning. The objective of these research questions was to compare the technology

integration planning of teachers that occurred in state high-stake tested and non-tested

subjects and determine if the context was a barrier to technology integration.


In regards to lesson plans, was there a significant statistical difference in technology

integration for English between high-stake tested English for 10th and 11th-grade teachers

and non-tested English for 12th-grade teachers?

Hypothesis #5:

There was a difference between the high-stake test subject of English for 10th and 11th

graders and the non-test subjects of English for 12th graders in regards to teacher

technology integration based on context.



HIGH-STAKE TESTING

13

integration for science between high-stake tested Biology teachers and non-tested

Chemistry, Accelerated Chemistry, and AP Biology teachers?

Hypothesis #6:

In regards to lesson plans, there was a statistical difference in technology integration for

science between high-stake tested Biology teachers and non-tested Chemistry,

Accelerated Chemistry, and AP Biology teachers.

Methodology Overview

The data collected were obtained through archived teacher lesson plans from an

available school district computer drive in 2014. The data were subject to quantitative

statistical testing utilizing ANOVA and t-tests. Descriptive statistics for subjects taught

were also calculated. The results were obtained and recorded using Excel spreadsheets

and Stat Plus software.

Definition of Terms

The following terms were used operationally throughout this study.

Advanced Placement (AP): A designation for courses in high school that carry

college credit upon successful completion of the year-end exam.

English courses for 10th and 11th graders: Any course including English and

literature standards concluding with a state standardized year-end exam.

English course for 12th graders: An English course for 12th graders that concludes

without a state standardized exam.

` Flipped classrooms: Classes that provide content and lessons about a subject

digitally that students review and learn the night before allowing classroom time

for projects, enhanced lessons, and skill development.

HIGH-STAKE TESTING

14

High-stake Test: A standardized exam used for making major decisions about

curriculum and accountability for students, teachers, and school districts.

In-service teacher: An individual currently teaching in a K-12 school.

Keystone Tests: End of the year standardized tests administered in the state of

Pennsylvania measuring student assessment for the specific subject areas of

Biology, Algebra, and Literature.

Lesson plans: A written or digital document that encompasses teachers’ strategies

for a particular time period of one or more days. Course objectives, curriculum

and student goals, assignments, and assessments are targeted.

Non-tested subjects: School subjects that are not tested through a Keystone state

tests.

Pre-service teacher: An individual who is currently training to be a teacher and

enrolled in a teaching program.

Technology integration: The use of technology tools in teaching and learning.

TPACK framework: A theoretical structure that organizes the knowledge base of

technology, pedagogy, and content of teachers for effective technology

integration (Koehler & Mishra, 2006).

Assumptions

The researcher assumed the lesson plans utilized were developed and completed

by the named certified teachers. Furthermore, it was assumed the information provided in

the teacher plans was accurate and inclusive to the high school staff within the district.

Also, it was assumed the teachers had access to technology. Lastly, it was assumed the

samples of the lesson plans were representative of the high school faculty who taught

HIGH-STAKE TESTING

15

these subjects.

Delimitations and Limitations

This study was delimited to high school teachers who were certified in the subject

areas of Biology, English, and Chemistry and taught in one western Pennsylvania public

school district. The technology integration of teachers’ lesson plans was measured for

teachers teaching the high-stake test subjects of English for 10th and 11th-grade students,

including English, College English, Honors English, and Biology. In comparison, the

non-test subjects of English for 12th grade students, including English, College English,

Honors English, AP English, Learning Support English, along with Chemistry, AP

Biology and Accelerated Chemistry lesson plans were measured. Teachers who teach

other subject areas were not included in the study. Technology integration was measured

using a valid and reliable assessment instrument that provided data used for comparative

purposes. Only data collected using the TPACK-Based Technology Integration

Assessment Rubric were utilized. Results were generalized toward teachers who teach

grades 9 thru 12, were certified in the subject areas of English, Biology, and Chemistry

teaching in one western Pennsylvania public school district. Results were not generalized

toward teachers of other subjects or grade levels in the school district.

Limitations of the study began with the fact that the sample was taken from a

purposeful sample of public high school teachers in a western Pennsylvania school

district. As a result, the research gathered and conclusions made from the study may not

be representative of elementary and middle school teachers within the district, or

determined to represent the technology integration of other teachers from different

schools. The results are suggested as a possible representation of technology integration

HIGH-STAKE TESTING

16

in a high school classroom if the same study was undertaken in a different setting.

Furthermore, the samples were taken from the 2012 – 2013 school year specifically, and

not representative of a different time or influence. This period is significant because the

state Keystone tests for Biology and Literature were initiated during this time. All 11th-

grade students were required to take these exams instead of the PSSA (Pennsylvania

System of School Assessment) exams. Furthermore, any student in grades 7th thru 11th

completing the proper standards and coursework for the subjects of Biology, English, and

Algebra participated in the Keystone exams.

Summary

The importance of technology in learning and teaching cannot be overestimated if

students are to be successful in the competitive global workplace. School districts have

purchased and leased computers and interactive technologies to provide teachers and

students with tools for acquiring twenty-first century skills. Unfortunately, many

teachers, administrators, and school districts have little understanding of the intricate

relationships between technology, pedagogy, and content. Consequently, the best use of

technology for integrating learning may be inhibited.

This quantitative study was an action plan targeted at teachers’ lesson plans to

understand the extent of technology integration in classes. Specifically, evaluating the

TPACK of teachers through an analysis of lesson plans for high-stake tested subjects and

non-tested subjects was the goal. Information can provide educators an extensive

understanding of technology, pedagogy, and content knowledge of teachers in various

contexts. As a result, educators can determine how technology integration is occurring in

different classrooms and how to improve their planned use of technology. Moreover,

HIGH-STAKE TESTING

17

educators can increase their use of technology while organizing lessons that integrate

their TPACK knowledge and engage students. Lastly, this study provides data as to high-

stake testing being a possible barrier to technology integration. Educators of tested

subjects can determine if they are planning for technology integration or limiting their

instruction.

The remainder of this research study is organized into the following chapters:

Chapter Two will review applicable literature; Chapter Three will explain the research

methodology used for data collection; Chapter Four will present an analysis of data and

overall results; Chapter Five will present an applicable action plan based on research

conclusions and recommendations for future studies.

HIGH-STAKE TESTING

18

CHAPTER TWO: LITERATURE REVIEW

Introduction

The chapter includes topics as background for the study of high-stake testing as a

barrier to technology integration. Topics include educational accountability and high-

stake testing, the TPACK framework and barriers to technology integration, as well as

teaching context and style. The conceptual framework of Bourdieu’s concept of habitus

and the relation to technology integration along with the influence of the TPACK

framework are reviewed. Lastly, assessing TPACK and lesson planning using the

TPACK-Based Technology Integration Assessment Rubric is considered followed by a

chapter summary.






focus of the study was to determine quantitative differences in technology integration of

teachers’ lesson plans in different contexts.

A better understanding of the issues and barriers to technology integration

planning can assist teachers and administrators to improve the use of technology in

classroom instruction. A better comprehension of lesson planning provides teachers and

administrators information to improve the design of lessons while integrating technology,

pedagogy, and content knowledge. Furthermore, analyzing lesson plans can provide data

regarding teachers’ habits and decisions concerning technology utilization in different

HIGH-STAKE TESTING

19

contexts.

Aim the Study



in high-stake subject contexts. The data collected provided information regarding a



content knowledge (TPACK) in various contexts, examine their instruction, and plan

lessons in the future accordingly.

Educational Accountability and High-Stake Testing

Teachers face the difficult task of educating students in an era of educational

accountability. School classrooms have become a place to measure the academic success

of students, the effectiveness of teachers, and the quality of schools through comparative

standardized testing results. Additionally, teachers have the multifarious task of educating

students for career and college readiness by engaging students with innovative pedagogy,

in-depth content, and cutting-edge technologies. The pressures for educating all students

for the 21st century have intensified while the responsibility on teachers for individual

student achievement on exams has increased. As a result, some teachers have had to

decrease their pedagogical approaches. For example, Grant and Hill (2006) indicated that

the context of focus upon high standards through testing has led to increased pressures on

teachers and a regression in the types of pedagogy used by teachers. As teachers have

limited pedagogical approaches, many confront the added pressure of using new

technology in the classroom. Consequently, teachers must prepare students for rigorous

HIGH-STAKE TESTING

20

standardized tests while integrating technology into their teaching using limited

pedagogical methods. Moreover, the ways of using technology in the classroom are

unfamiliar to teachers. As a result, a confounding problem for teachers is balancing the

use of technologies with limited pedagogical approaches in a high-stake testing context.

High-stake testing is a major part of the accountability for schools since the

inception of NCLB legislation in 2001. The purpose of this law was too narrow the

achievement gap in schools by providing student achievement data. Since the inception

of the law, testing has held a prominent role in school culture. As Gunzenhauser (2003)

indicated, the climate of high-stake testing has created a default philosophy of education

where tests drive the curriculum and limit teacher autonomy and creativity. Although a

testing culture may be a driving force in schools, advocates believe that testing improves

accountability and in turn the performance of students and teachers. The intention of the

high-stake testing reform is to motivate teachers and students to increase achievement

through preparation and performance measured by tests (Moses & Nanna, 2007). As a

result, the high-stake testing context adds pressure to the classroom teacher emphasizing

the accountability of student achievement.

Jonathan Supovitz (2010) declared the four theories of motivation, alignment,

information, and symbolism as being reasons for the conviction of high-stake tests.

Specifically, teachers will become motivated, the curriculum will be aligned, data or

information will be used for improvement, and society will be satisfied with test-based

accountability (Supovitz, 2010). This analysis gives credence to the changing classroom

that emphasizes student and teacher measurable achievement.

As high-stake testing has become part of the educational process, it is imperative

HIGH-STAKE TESTING

21

that administrators, school boards, and parents appreciate that testing has created a school

culture of testing that has changed teacher practices. For example, Berliner (2011)

emphasized that high-stake testing has contributed to the narrowing of curriculum; while

Barksdale-Ladd and Thomas (2000) exclaimed that high-stake testing results are the

focus for teachers concerning instruction. Furthermore, Crocco and Costigan (2007)

reported that teachers in many New York City middle and high schools find the culture of

testing limits their creativity and independence based on the mandated and narrowed

curriculum.

While a school culture of testing has been created since the NCLB legislation, an

examination of the effects on teaching is warranted. Nichols and Berliner (2007)

explained that high-stake testing has created increased pressure and anxiety in a

profession that is undervalued, underpaid, and under supported. In fact, teachers’ work

environment now includes the possibility of professional dismissal being determined by

student test scores (Nichols & Berliner, 2007). Additionally, Clarke et al. (2003)

determined through their research that teachers of tested subjects rush their teaching pace

to cover an overloaded curriculum while teachers of non-tested subjects alter their

curriculum for testing. Furthermore, Au (2011) stated that high-stake testing policies

have pressured teachers to teach to the test consistently and fostered the use of scripted

curriculum. Consequently, schools and teachers confront a context and culture of

teaching that focuses on standardized tests and results.

The culture of accountability in education is evidence as to the impact of high-

stake testing in education. Although the benefits to testing are relevant, such as an aligned

curriculum and state standards for relevant content (Yeh, 2005), testing is not a panacea

HIGH-STAKE TESTING

22

for closing the achievement gap or learning twenty-century skills needed for college and

career. Moreover, high-stake testing provides teachers with data concerning student

achievement but limits teaching toward narrow test preparation (Madaus & Russell,

2010). As a result of focusing on test preparation, the high-stake testing context may be a

barrier to technology integration practices.

The social science concept of Campbell’s law supports that high-stake testing

contexts are a barrier to technology integration. Researcher Donald T. Campbell (1976)

concluded that achievement tests are valuable for general purposes but change the

educational process when they are the focus of teaching. Campbell’s emphasis on the

change of the educational process brings to the forefront the challenges of a high-stake

testing context for the teacher. Furthermore, Campbell (1976) emphasized that a

quantitative social indicator, such as testing, used for social decision-making, educational

decisions, will be more susceptible to increased pressures and distortions in the social or

education process. As Nichols and Berliner (2007) explained, Campbell’s law indicates

that an increase in high-stake testing leads schools and teachers to extreme measures,

often compromising educators who have been the moral leaders of our country. Based on

Campbell’s research, teachers are focusing on teaching to the test and the narrowing of

the curriculum (Nichols & Berliner, 2007). As a result of teaching to the test, educators

are more concerned with overall test scores than the important changes needed to

improve the educational system for their students (Cawelti, 2006), such as using

technology in the classroom.

A second concept that has contributed to high-stake testing being a barrier to

technology integration is that of teachers’ teaching habits influenced by the narrowing of

HIGH-STAKE TESTING

23

the curriculum. Blazer (2011) and Yeh (2005) explained that high-stake testing has

narrowed the curriculum through the exclusion of non-tested subjects, eliminated non-

tested topics in subject areas, increased test preparation, and adapted teaching to fit test

formats. Moreover, research indicates that teachers are employing more teacher-centered

approaches in teaching content resulting in less time for outside the classroom activities

(Au, 2008). Consequently, high-stake testing context increasing technology integration is

suspect if technology use does not fit teacher practice and the targeted curriculum.

Furthermore, teachers’ instructional planning may be influenced by this context of

narrowing of the curriculum. Thomas (2005) supported a lack of innovated instruction by

reporting that teachers had less time for instruction focusing on a quick mention of

content, limited instructional resources, conventional curriculum sources, and narrowed

assessments due to high-stake tests. Overall, technology integration is influenced by the

high-stake testing context.

The high-stake testing context has pushed educators toward an inspection of the

subject content taught by teachers, pedagogical approaches, and the consequences for

student achievement. Simultaneously, many teachers, administrators, and school districts

are exploring the best way of using technology to improve teaching, classroom

engagement, and student achievement. Subsequently, examining the integration of

content, pedagogy, and technology knowledge of teachers is important if the intent is to

use technology optimally in all teaching contexts for students.

As technology availability in schools has increased, an understanding of its role in

pedagogy and content must be appreciated. As Yurdakal et al., (2012) emphasized, the

focus of technology integration has changed from a techno-centric approach toward a

HIGH-STAKE TESTING

24

techno-pedagogical one in which technology and pedagogy are equally important. Thus,

educators must find ways of examining the integration of technology, pedagogy, and

content employed in the classroom environment. One convenient and efficient method to

gain information as to a teacher’s aim of integrating technology, pedagogy, and content

in context is to examine teacher-developed lesson plans (Harris, Grandgenett, & Hofer,

2010). Examination of lesson plans allows for an understanding of the teachers’ intent to

integrate technology within the context of the subject while being easily accessible to

both teachers and administrators. Lesson plans provide a view into the thought processes

of decisions made by teachers and their strategies concerning pedagogy (Harris,

Grandgenett, & Hofer, 2010). Thus, analyzing teachers’ lesson plan will assist the

educator in understanding the use of technology integration in classrooms consistently

over a time span and the possible barriers that may hinder its implementation in context.

Furthermore, an examination of teacher planning provides data as to teachers’ habit of

using technology constructively in their classes.

The TPACK Framework and Barriers to Integration

Comparable data results from the examination of lesson plans can be completed

through the use of a measuring instrument for technology integration. Specifically, using

an instrument to measure the technology integration of teachers in the context of high-

stake tested subjects and non-tested subjects provides data for educators as to differences

and similarities in context. Furthermore, barriers to technology integration in context can

be explored and remedied based on the data gathered for specific subjects.

Accordingly, teachers and administrators can assess technology integration in

classrooms by utilizing a TPACK (technology, pedagogy, and content knowledge)

HIGH-STAKE TESTING

25

framework. Koehler and Mishra designed this framework originally entitled TPCK

(technology, pedagogy, content knowledge) for teachers to become curriculum designers

of technology integration in their classrooms (Koehler & Mishra, 2008).

Understanding of the TPACK framework begins with a brief explanation of

Shulman’s pedagogical content knowledge (PCK). Shulman (1987) explained that

pedagogy and content must be understood by the teacher to distinguish them from an

individual content specialist or expert in pedagogy. In other words, pedagogy content

knowledge is the understanding of what makes a subject easy or difficult to comprehend

for the diverse learner based on the examples, wisdom, and demonstrations of the teacher

(Shulman, 1986). Based on Shulman’s integration of pedagogy and content, Koehler and

Mishra designed the TPACK framework by adding technology to pedagogy and content

(Pamuk, 2012).

TPACK is a framework that explains how teacher knowledge of technology,

pedagogy, and content relate to create effective teaching with technology (Koehler &

Mishra, 2009). More specifically, Mishra and Koehler (2006) provided a definition that

explained the TPCK (TPACK) framework from an expansive perspective that includes

teacher knowledge of technology, knowledge of pedagogy that utilizes technology for

conveying content, and knowledge of how technology can address student knowledge.

The TPACK framework allows for a specific analysis of the relationships

between technology, pedagogy, and content knowledge. Technical knowledge (TK) is

that which enables an individual to utilize informational technology to accomplish a

variety of tasks while developing and evolving one’s skills as technology changes

(Koehler & Mishra, 2008). Content knowledge (CK) is the subject matter that includes

HIGH-STAKE TESTING

26

knowledge of concepts, theories, and practices to be taught or learned (Koehler &

Mishra, 2008; Shulman, 1986). Pedagogical knowledge (PK) is an understanding of the

practices of teaching needed for student learning that include but not limited to classroom

management, lesson planning, and student assessment (Koehler & Mishra, 2009).

In the context of practice, the TPACK framework allows for a specific analysis of

the relationships or complex interactions between the three areas of technology,

pedagogy, and content knowledge (Koehler & Mishra, 2008). Surrounding the interaction

of the three bodies of knowledge is context, as shown in Figure 1.

Figure 1 TPACK Framework

Reproduced by permission of the publisher, © 2012 by tpack.org

HIGH-STAKE TESTING

27

Harris and Hofer (2011) reflected that culture, socioeconomic status, and school

organization impact the complex teacher understanding of the relationships of

pedagogical content knowledge (PCK), technology content knowledge (TCK),

technology pedagogical knowledge (TPK), and technology pedagogical content

knowledge (TPACK). A brief description of each is relevant to understanding the

relationships and challenges faced by teachers when integrating technology into the

school culture.

According to Koehler and Mishra (2008), PCK is the core of teaching that

includes the ability to understand content and appropriate teaching strategies along with

curriculum and assessment while promoting learning based on a student’s prior

knowledge. TCK is the selection of technology and an understanding of how this

influences the content (Koehler & Mishra, 2008; Harris and Hofer, 2011). TPK is defined

as understanding how to utilize technology and the effect on learning and teaching

(Koehler & Mishra, 2008; Harris and Hofer, 2011). Lastly, TPACK according to Harris

and Hofer (2011) is defined as using technology effectively to teach subject content and

support students with their learning needs and interests.

Technology use in classrooms has changed as technology use in society has

become prevalent and access to technology has increased. For example, many teachers in

science classrooms are including technology in their instruction realizing the positive

benefits of technology use in learning (Guzey & Roehrig, 2012). Additionally, Advanced

Placement (AP) and National Writing Project (NWP) teachers when surveyed expressed

a significant use of digital technologies including the Internet for research and online

submission of assignments (Purcell, Heaps, Buckanan, & Friedrich, 2013). As technology

HIGH-STAKE TESTING

28

becomes widely accepted for learning, technology integration barriers in classrooms still

exist, but the traditional barrier of access to computers is no longer an obstacle in many

cases (Ertmer, Ottenbreit-Leftwich, Sadik, Sendurur, & Sendurur, 2012). For example, a

teacher survey from the National Center for Educational Statistics (Gray, Thomas, Lewis,

& Tice, 2010) indicated that teachers and students often use computers 40% of

instructional time while internet access to these computers was available at least 93% of

the time. As traditional access barriers to technology integration have declined, further

barriers remain.

The focus of global researchers is concentrated upon educators and the barriers

they confront when supporting or preventing technology integration in the classroom

(Mueller, et.al 2008). For example, Sherman and Howard (2012) revealed in a South

African study that teachers’ beliefs about their capacity to teach effectively with

technology are important factors for integrating technology. Additionally, Tondeur,

Valcke, and Van Braak (2008) determined in a study of Belgium preschools that teacher

factors and a school vision are variables in the use of technology in the classroom.

As K-12 teachers and administrators face the difficult task of preparing students

for the academic and work demands of the twenty-first century, the problem of barriers to

technology integration in the classroom is relevant. To tackle this problem, educators

must comprehend the types of technology integration barriers that exist and determine

their obstacles. Peggy A. Ertmer categorized technology integration barriers into extrinsic

first-order barriers and intrinsic second-order barriers (Ertmer, 1999). First-order barriers

are defined as types of resources such as hardware and software, training, and time

(Means & Olson, 1997; Ertmer, 1999). For example, Zhao, Pugh, Sheldon, and Byers

HIGH-STAKE TESTING

29

(2002) explained that teachers compete in schools for computer lab time. Furthermore,

Hew and Brush (2007) emphasized that a lack of resources such as limited technology

support personnel, time for learning by teachers, and technology availability as being

contributors to a lack of technology integration in the classroom.

In comparison, second-order barriers include teacher beliefs about teaching

methods and computers, an unwillingness to change, and traditional classroom practices

(Ertmer, 1999). Hew and Brush (2007) emphasized that a lack of technology knowledge

and skills related to pedagogy and classroom management are obstacles to technology

integration. Furthermore, internal barriers are exemplified in classrooms that have

computers, interactive whiteboards, and technology for teachers but little technology

integration (Hammonds, Matherson, Wilson & Wright 2013).

Administrators and teachers face the challenge of identifying the barriers that

impede technology integration when technology is available. Accordingly, educators

need to examine first-order, second-order, or a combination of both when determining the

level of technology use in the classroom. More importantly, barriers that exist because of

a specific teaching context must be researched so as to understand technology integration

in various teaching circumstances. Specifically, the second-order barrier of teacher

knowledge of technology, pedagogy, and content within a specific context is an important

focus. As educators face the task of blending new technologies with personal teaching

styles and mandated content, an examination of a teachers’ technology integration

knowledge within a high-stake tested context is reasonable. Moreover, to better

understand technology integration barriers among and between teachers in various

contexts, the TPACK framework can guide teachers and administrators in this

HIGH-STAKE TESTING

30

investigation.

Many articles by researchers have supported the importance of the TPACK

framework for improving technology integration understanding. The research of Chai,

Hoh, and Tsai (2013) indicated that 74 journal articles were published on TPACK from

2003 through 2011 with 55 of these articles being data driven and 19 being categorized as

theoretical, worked examples, or editorials. For purposes of introduction, a brief review

of three articles that demonstrate the versatility of the TPACK framework and the

measurement of teachers’ aptitudes about TPACK in different contexts is applicable.

Archambault and Crippen (2009) investigated the TPACK of 596 online teachers

using a teacher survey. The researchers found that the teachers rated their knowledge of

pedagogy, content, and pedagogical content to be higher than technology. Although these

teachers utilize technology when teaching, Archambault and Crippen (2009) concluded

that the confidence in pedagogy and content knowledge of teachers was associated with

their training in these areas as pre-service teachers.

In a second study, researchers surveyed 399 Chinese pre-service and 394 in-

service teachers concerning TPACK, teachers’ beliefs about constructivist teaching, and

design disposition or personality. The survey resulted in pre-service teachers having

significant less knowledge in all factors of TPACK (Dong, Chai, Sang, Koh, Tsai, 2015).

Furthermore, the survey indicated that in-service teachers believed in constructivist

learning but need more training in this approach (Dong et al., 2015).

Lastly, Altun (2013) conducted a quantitative study of 322 primary classroom

teachers in the city of Trabzon, Turkey concerning demographic variables and TPACK.

The hypothesis of the study was that demographic variables, such as gender, teaching

HIGH-STAKE TESTING

31

level, and use of the Internet, would influence teachers’ TPACK of Turkish classroom

teachers. The results of the research indicated that teachers’ technology integration is

significantly increased when the teacher has use of the Internet, a computer lab, or

educational software for the primary classroom (Altun, 2013). The previous studies

establish two specific points concerning the TPACK framework. First, the TPACK

framework is applicable for use across educational course levels. Second, technology

integration analysis is applicable for online and traditional classroom teachers regardless

of teaching context. Consequently, an instrument for measuring technology integration or

teachers’ TPACK is essential for an understanding of barriers in context.

As research has increased since the inception of the TPACK framework,

determining what to measure for technology integration has been a focus for researchers.

Harris, Grandgenett and Hofer (2010) suggested that teachers’ lesson plans or artifacts

provide insight for educators into ways of teachers’ thinking about instruction. Thus, an

analysis of lesson plans can provide data as to how a teacher intends to integrate

technology within the teaching context and assist the educator in understanding their

TPACK. Before reviewing a valid measuring instrument to use when analyzing lesson

plans, an examination of teaching context and style is required to understand better the

overall teaching process.

Teacher Context and Style

A major goal of the TPACK framework was to help teachers become designers of

the curriculum in various teaching situations and contexts (Koehler & Mishra, 2008). In

turn, context is a major factor in instructional planning that must be considered when

reviewing teacher effectiveness. In regards to TPACK and technology integration,

HIGH-STAKE TESTING

32

context is an important area of concern for researchers and administrators. Chai, Koh, and

Tsai (2013) emphasized the four interdependent contextual factors of intrapersonal

teacher pedagogical beliefs, interpersonal group design, cultural/institutional, and

physical/technological factors that should be recognized as influential in teacher

instructional decisions. Kelly (2008) elaborated on the importance of the TPACK context

noting that physical, cognitive, linguistic, social, psychological, and cultural factors can

be looked upon as enhancing or obstructing instruction. Furthermore, Koehler and Mishra

(2008) emphasized that context matters to teachers when understanding the knowledge of

content, pedagogy, and technology while handling school social networks or parental

concerns.

As context is a major contributor to the problem of teaching with technology

(Koehler & Mishra, 2008; Kelly, 2008), the teaching style of a teacher is a contextual

factor that must be examined. To better understand the influences of teaching style upon

context, a brief overview of two teaching styles is important. The emphasis on

educational accountability, global competitiveness, student differentiation, and

technology integration has led to a conversation about traditional teaching versus learner-

centered or student-centered teaching styles.

According to Novak (2011), traditional classrooms in most schools consist of

teachers standing in front of the room lecturing and providing information to be

memorized and regurgitated on a multiple-choice test. In fact, traditional teaching

promotes a focus on the teacher based on the physical makeup of the room, classroom

rules, and the passivity of the learner (Garrett, 2008). In contrast, student-centered

teaching is focused on the individual learner taking into consideration each student’s

HIGH-STAKE TESTING

33

individual and group needs as well as the encouragement to participate (Jones, 2007).

Furthermore, teachers realize that students learn in various ways that often involve social

relationships and communication as a major part of the learning style (Napoli, 2004). As

a result of comparing these two teaching styles, a closer look at the student-centered or

learning-centered teaching style is justified based on the integration of technology into

learning.

According to Keengwe, Onchwari, and Onchwari (2009) the goal of a learner-

centered education is to equip learners to move away from passive learning by being

encouraged to work with new information, new meanings and understandings, and

construct knowledge based on experiences. If a learner-centered classroom and

technology integration are the goals of a teacher or a school system, than Keengwe,

Onchwari, and Onchwari (2009) believed that an active learning environment that

emphasizes each learner’s needs while using technology is sound pedagogical practice.

Moreover, technology can provide multiple means of representing material, multiple

means of motivating students, and multiple means of assessing for all student needs as

promoted by the Universal Design for Learning framework (Rose et. al, 2006). However,

the ability of teachers to integrate technology in various ways and to assist individual

students may be dependent on their teaching habits.

Conceptual Frameworks

Two conceptual frameworks guided the thought processes and observations of the

action research. The two frameworks are TPACK and Pierre Bourdieu’s Cultural

Reproduction Theory (Bourdieu & Passeron, 1990) and the associated concepts of field,

habitus (Grenfell & James, 1998), and cultural capital (Bourdieu, 1986). The two

HIGH-STAKE TESTING

34

frameworks formulated the ideas for the action research of high-stake testing as a barrier

to technology integration.

Public schools are places of work where the teachers employ their trade in a place

that they were quite familiar with as a child. As a result, teachers often employ traditional

styles of teaching that reflect what they experienced. Traditional teaching styles

emphasize a teacher-centered approach where information is presented from the front of

the room or assigned through text chapters and assessed through exams (Brooks &

Brooks, 1993). These traditional experiences and practices can be justified through an

examination of Bourdieu’s cultural reproduction theory (Bourdieu & Passeron, 1990). In

short, Bourdieu’s cultural reproduction theory as it relates to education states that

inequalities exist in educational systems brought about by educational credentials that

reproduce the inequalities (Sullivan, 2002). Moreover, Dumais (2005) explained that

families possessing cultural knowledge valued by teachers are privileged in the

educational system. Consequently, many teachers use traditional teaching styles that

reinforce their success as students. As a result of this system, teachers who have learned

to perform at school well succeed and reproduce the systems of inequality again (Nolan,

2012). Hence, traditional teaching styles worked for the teacher.

In short, many teachers are comfortable teaching the way they learned. As

Belland (2009) expressed, teacher practices are generated from one’s habits or habitus,

and Bullock and Russell (2010) contended that routines and practices are embedded in

schools at a young age and understood by all who have spent thousand of hours in

schools. As a result, many teachers’ perception of teaching and learning or ways of

teaching is traditionally based because of their habitus of practice in the field and their

HIGH-STAKE TESTING

35

acquired capital. A brief definition of the terms habitus, field, and capital is warranted.

Sullivan (2002) defined habitus as a set of attitudes and values that are learned at

home. Comparatively, Bourdieu’s concept of habitus is explained by James (2011) as a

way of discussing how individuals obtain thinking and doing from the past into the

present.

The field is explained by Warde (2004) as an independent setting of constant

struggle where stakes are valued and fought for by participants who come with different

habitus and capital. In turn, capital is resources that an individual possesses that can be

utilized for success or a change in habitus to a higher class (Gaddis, 2013).

In order to understand habitus, field, and cultural capital concepts, O’Hara (2000)

explained the complexities of the relationship as individuals possessing capital, such as

skills or knowledge, that allow the individuals to act in defined ways or habitus, that is

developed from one’s class or upbringing, and this habitus affects an individual’s

attitudes or choices in any field.

Nolan (2012) explained in her analysis of pre-service teacher field experiences for

mathematics the importance of the concepts of field, habitus, and cultural credit in

determining pedagogical teacher practice. Most importantly, Nolan (2012) explained the

complex interaction of these concepts through the Bourdieu analogy of playing a game.

In summary, Nolan (2012) confided that games or fields have certain rules that players

must abide by in order for the game to run smoothly, eventually becoming rules that

become second nature to the players and unquestioned or thought about.

As for education in the accountability era, habitus, field, and capital can be

regarded as important concepts toward an understanding of the impact of a high-stake

HIGH-STAKE TESTING

36

testing on teaching context. In particular, a high-stake testing context is generated by the

state issuing the exams. Based on Bourdieu’s game analogy, the social field or high-stake

testing context comes with specific structures and rules that teachers follow blindly for a

smooth game. In turn, a teacher’s capital or resources such as testing knowledge, beliefs

in teaching, experiences as a test taker, and teaching experiences are valued in the game

as they contribute to the smoothness of the game. Lastly, the ‘feel for the game’ or

habitus encourages little thinking before acting, and the game continues without

substantial changes from the teacher. As a result, the status quo is encouraged and

reinforced by the teachers or players. The high-stake tested context promotes traditional

teaching and learning most familiar to the teacher.

In comparison, a teacher or player who encourages an acceptance of the rules but

believes in utilizing and obtaining new capital to improve one’s “feel for the game” or

habitus encourages change to the traditional game. A habitus change is exemplified in a

constructivist approach to learning that integrates technology, pedagogy, and content

knowledge into a context that decreases traditional instruction. Gilakjani, Leong, and

Ismail (2013) supported the concept that using technology with a constructivist approach

leads to a student-centered focus. As a result, an increase in technology use through a

constructivist classroom could be considered a change from a traditionalist to a learning-

centered approach.

Technology, pedagogy, and content knowledge (TPACK) is a framework that

allows educators to understand the level of technology integration in the classroom

through an analysis of the implementation and motivations in practice of technology-

related knowledge (Mishra & Kohler, 2009). Assessing the level of technology

HIGH-STAKE TESTING

37

integration in various contexts allows for an understanding of barriers to implementation

and teaching in practice. Furthermore, the use of TPACK as a theoretical framework

allows both teachers and administrators to examine possible solutions to technology

integration within the context of overall teaching. The adequate use of technology is

measured as to its relevance to both instruction and content, not measured exclusively by

it. For example, interactive whiteboards or mobile phones can be used in whole group

instruction, delivering valuable content, and allowing for student inquiry and interaction.

Their existence as tools outside of pedagogy and content is irrelevant to the technology

integration process.

Belland (2008) supported the concept of habitus and the relationship to teacher

technology integration. In brief, the solution to a lack of technology integration in schools

may be found in the experiences or dispositions of teachers, beyond a focus on traditional

barriers (Belland, 2008). The habitus of teachers can be examined through the assessment

of technology integration in context. Specifically, a teacher’s planning for technology

integration can reveal an insight into the practices of specific teachers, subjects, or

contexts that perpetuate a habitus of technology integration planning and implementation

or lack of application.

Assessing TPACK and Lesson Planning

Anyone involved in teaching or education understands the importance of teacher

planning to ensure quality instruction and academic success. According to He and

Hartley (2010) teachers write lesson plans to create activities concerning what they want

to accomplish each class. Additionally, lesson planning can provide valuable insight into

a teacher’s understanding of the curriculum. Consequently, teachers and administrators

HIGH-STAKE TESTING

38

can review and reflect upon a variety of factors in teaching. Clark and Yinger (1979),

described the reasons to study teacher planning as providing research into a teacher’s

thinking and action, the importance of planning to teacher practitioners, insight into a

teachers’ theories of teaching and learning, and a connection between curriculum and

teacher’s behavior. Lesson plans provide data as to teacher thinking and action,

connections with curriculum, and most importantly insight into teachers’ theories of

teaching and action, content understanding, and use of technology (Clark & Yinger,

1979). Essentially, lesson plans provide a window into the world of teaching that an

administrator or teacher can readily view.

According to Britten and Cassady (2005), a review of lesson plans by

administrators can be used to examine many teachers’ technology integration while

analyzing the context in a standard classroom setting. Similarly, Harris, Grandgenett, and

Hofer (2010) stated that instructional plans provide information about a teacher's

decision-making and pedagogical reasoning. Consequently, Britten and Cassidy (2005),

and Harris, Grandgenett, and Hofer (2010) developed valid measuring instruments for

technology integration analysis of lesson plans. Essentially, a review of both evaluation

tools is relevant to analyzing instructional plans of teachers for technology integration in

a high-stake tested subject or non-tested subject area. Notwithstanding, the two

instruments are useful but differ in use. The TPACK-Based Technology Integration

Assessment Rubric developed by Harris, Grandgenett and Hofer addresses the evaluation

of lesson plans for technology integration. This instrument was developed to be more

pedagogically inclusive and includes the concepts of TPK, TCK, and TPACK (Harris,

Grandgenett, & Hofer, 2010). In comparison, the Technology Integration Assessment

HIGH-STAKE TESTING

39

Instrument (TIAI) developed by Britten and Cassidy (2005), is an instrument that

measures seven dimensions across four classifications but does not include the TPACK

framework. Furthermore, the TIAI rubric is the forerunner to the TPACK-Based

Technology Integration Assessment Rubric. An explanation of the TIAI rubric is

important to understand the development of the TPACK-Based Technology Integration

Assessment Rubric.

The seven dimensions of the TIAI rubric includes (1) Planning, (2) Content

standards, (3) National Educational Technology Standards (NETS) (4) Student Needs, (5)

Implementation (use of technology in learning), (6) Implementation (use of technology in

teaching), and (7) Assessment, of a lesson plan. There are four levels of classifications

within each dimension listed as follows (a) Technology not present (b) Non-essential

technology component (c) Supportive technology component (d) Essential technology

component (Cassady & Britten, 2005). The TIAI instrument was based on the basic

framework of Maddux (1986), that classified technology into Type I teacher-centered

activities and Type II learner-centered applications. Maddux and Cummings (1986)

defined Type I activities as passive for the learner and include rote memory and

assessment activities that promote traditional teaching methods and tasks predetermined

from the developer. In contrast, Type II applications are driven by the user and involve

problem-solving and other cognitive skills that empower the learner to manage the

learning. Concerning the TIAI instrument, Britten and Cassady expressed a need for an

evaluation instrument that allows teachers to collect data and make decisions about their

technology integration with respect to assessments, student needs, and educational

standards (Britten & Cassady, 2005)

HIGH-STAKE TESTING

40

The TIAI instrument utilized archived lesson plans that were based on NETS-T

standards developed in 2000. NETS-T standards are developed by the International

Society for Technology Education (ISTE) and composed of standards and performance

indicators that deal with technology operations, planning, curriculum development,

assessment, professional practice, and human issues (ISTE, 2000). Planning and

designing learning environments and experiences supported by technologies are

emphasized through instructional strategies, current research, location of resources, plan

management, and the management of students in a technology environment (ISTE, 2000).

To understand the TIAI thoroughly, a brief description of two of the seven

dimensions is valid to give a sample of the instruments rating system. As the seven

dimensions are utilized across four levels of classification, a comparative description of

the top two classifications across the dimensions of planning and implementation in

learning will provide a look at the difference in levels of technology integration presented

in the lesson plans. The classification of technology not present is obvious and recorded

when technology is not mentioned in all dimensions (Britten & Cassady, 2005).

Additionally, the non-essential technology component is when technology may be

mentioned but not directly impactful to learning (Britten & Cassady, 2005).

According to Britten and Cassady (2005), when technology is essential to the

lesson such as data probes used for a statistical biology package, a top rating of the

essential technology component, Type II, is applicable because the lesson could not exist

without technology. In contrast, a supportive technology component is when the

computer is used for planning and replication purposes, such as the use of a PowerPoint,

a Type I application (Britten & Cassady, 2005).

HIGH-STAKE TESTING

41

When implementing technology for learning, the TIAI instrument emphasizes the

essential technology component as a learning process when the student is provided

content that can only be available through technology or digital media interaction (Britten

& Cassady, 2005). In contrast, word processing for editing or viewing expert opinions

online are supportive technology that are defined as not essential for learning the lesson

(Britten & Cassady, 2005).

Overall, the TIAI instrument has provided a basic assessment for measuring

technology integration. Specifically, the designed technology integration rubric gives

educators a framework for evaluating how technology is integrated with assessment,

student needs, and NET-S standards (Britten and Cassady, 2005). Furthermore, Britten

and Cassady (2005) emphasized that the instrument allows an evaluator to measure the

progress of educators without having to schedule observations or rely on self-report

surveys. Nonetheless, the rubric is limited in two specific ways. First, the TIAI

instrument does not address the TPACK framework from a perspective of the

interrelationships between content, pedagogy, and technology (Harris, Grandgenett, and

Hofer, 2010). Second, the instrument utilized is based on an examination of lesson plans

that are specifically centered on national technology standards. While these standards are

very important, a tool that allows for more flexibility can be advantageous for educators

unfamiliar with standards.

An instrument that addresses the evaluation of lesson plans by utilizing the

TPACK framework is the TPACK-Based Technology Integration Assessment Rubric

developed by Harris, Grandgenett, and Hofer (2010). A review of the rubric begins with

an analysis of the origin of the instrument followed by a description of the rubric.

HIGH-STAKE TESTING

42

The origin of the TPACK-Based Technology Integration Assessment Rubric

begins with a look at the development of the rubric based upon a need for a valid and

reliable TPACK instrument. According to Harris, Grandgenett, Hofer (2010), surveys,

observed behavior, and teaching artifacts are three types of data that may be used to

evaluate teachers’ TPACK. Nonetheless, the authors chose to analyze a teacher’s artifacts

instead of utilizing observed behavior or teacher surveys for two reasons. First, observed

behavior alone does not provide a look at the decision-making processes or TPACK

knowledge that dictates the observed instruction (Harris, Grandgenett, Hofer, 2010).

Second, Harris, Grandgenett, and Hofer (2010) emphasized that two teacher surveys, one

for in-service online teachers designed by Archambault and Crippen (2009) and a survey

developed for pre-service teachers by Schmidt, Baran, Thompson, Kohler, Shin, and

Mishra (2009), provide data but do not measure external performance. As a result, the

TPACK-Based Technology Integration Assessment Rubric was developed from the only

available valid and reliable measuring instrument for teacher lesson plans, the TIAI

instrument (Harris et al., 2010).

The application of a rubric for evaluating lessons or teaching is part of the climate

of the new accountability of education. Various states, including Pennsylvania have

adopted a teacher evaluation system that utilizes the Charlotte Danielson framework. This

constructivist-based framework consists of dividing teaching responsibilities into the four

domains of planning and preparation, classroom environment, instruction, and

professional responsibility that enables teachers and administrators to determine how to

improve teaching skills through coaching, mentoring, professional development, and a

teacher evaluation system, as well as student engagement (Danielsongroup.org, 2013).

http://danielsongroup.org/

HIGH-STAKE TESTING

43

Accordingly, the use of the TPACK-Based Technology Integration Assessment Rubric

for evaluating lesson plans is very helpful and applicable to the educator who is held

accountable. Specifically, the rubric is useful to the educator for four reasons.

First, the TPACK- Based Technology Integration Assessment Rubric provides the

teacher and administrator with a baseline of information concerning teaching practices

that generate analysis similar to that of the Danielson framework. Second, the rubric

provides a guide for the teacher to understand the amount of technology they use in the

class and whether there is room for improvement. Third, available data from using the

rubric provide administrators with information that can improve professional

development for technology integration. Fourth, the rubric provides information to both

the teacher and administrator as to possible barriers to technology integration.

Summary

This objective of this chapter was to review the relationships among teaching

context, teacher planning, the TPACK framework, and barriers to technology integration.

In 2015, teaching context is complex, and the integration of technology is difficult. An

understanding of a teacher’s TPACK through an analysis of lesson plans allows an

educator to evaluate technology use in context. Moreover, an analysis of lesson plans

provides a useful method for educators to assess possible barriers to technology

integration. TPACK data can provide information as to a teacher’s planning habits as it

pertains to technology integration. Lastly, Chapter Three will introduce the methodology

of the study including the research design, research questions, and data analysis used.

HIGH-STAKE TESTING

44

CHAPTER THREE: METHODOLOGY

Introduction

In this chapter, the sample and participants, instrument rubric, lesson plan format,

research design, materials, procedures, research questions, data analysis and ethical

considerations are explained. A TPACK-Based Technology Integration Assessment

Rubric was utilized to assess high school teacher lesson plans.














contexts.

Aim of the Study



HIGH-STAKE TESTING

45






Research Questions and Hypotheses

This study analyzed the technology, pedagogy, and content knowledge (TPACK)

of teachers’ lesson plans in high-stake test and non-test subjects for the 2012-2013 school

year. The research questions that guided the study were based on hypotheses that a

difference exists among and between high-stake test subjects and non-test subjects in

regards to technology integration planning. Research questions one through four guided

the study among subject lesson plans. Specifically, these questions were based on the

hypotheses that teacher lesson plans within the same subject and test context would not

differ in technology integration.


In regards to teacher lesson plans, is there a significant statistical difference in technology

integration for the high-stake tested subjects of English for 10th and 11th graders among

these teachers?

Hypothesis #1:

Lesson plans for the high-stake subjects of English for 10th and 11th graders do not differ

in regards to technology integration by teachers because of the high-stake test context.


HIGH-STAKE TESTING

46


integration for the non-tested subject of English for 12th graders among these teachers?

Hypothesis #2:

Lesson plans for the non-tested subject of English for 12th graders do not differ in regards

to technology integration by teachers because of the context.



integration for the high-stake tested science subject of biology among these teachers?

Hypothesis #3:

Lesson plans for the high-stake science subject of biology do not differ in regards to

technology integration by teachers because of the high-stake test context.



integration for the non-tested science subjects of Chemistry, Accelerated Chemistry, and

AP Biology among these teachers?

Hypothesis #4:

Lesson plans for the non-tested science subjects of Chemistry, Accelerated Chemistry,

and AP Biology does not differ in regards to technology integration by teachers because

of the non-stake test context.

Second, research questions five and six guided the comparison between high-

stake tested subjects and non-tested subjects for technology integration in teacher

planning. The objective of these research questions was to compare the technology

HIGH-STAKE TESTING

47

integration planning of teachers that occurred in state high-stake tested and non-tested

subjects and determine if the context was a barrier to technology integration.


In regards to lesson plans, is there a significant statistical difference in technology

integration for English between high-stake tested English for 10th and 11th-grade teachers

and non-tested English for 12th-grade teachers?

Hypothesis #5:

There is a difference between the high-stake test subject of English for 10th and 11th

graders and the non-test subjects of English for 12th graders in regards to teacher

technology integration based on context.




Chemistry, Accelerate Chemistry, and AP Biology teachers?

Hypothesis #6:

In regards to lesson plans, there is a statistical significant difference in technology


Chemistry, Accelerated Chemistry, and AP Biology teachers.

Research Design

A causal-comparative research design was chosen for this study. This design

allows for an examination of the cause of differences among or between groups (Brewer

& Kuhn, 2010). In this comparative research methodology, the independent variables of

context and subject have already occurred and are not manipulated. The dependent

HIGH-STAKE TESTING

48

variable was technology integration measured by examining TPACK. The comparison

method was chosen to find the influence of the subject taught and the context upon

teachers’ technology, pedagogy, and content knowledge (TPACK) when lesson planning.

A comparison of data allowed for an analysis of the subject taught and whether the

context was a high-stake tested subject or not. For example, the TPACK score for high-

stake tested English teachers were compared to other English teachers. In turn, high-test

subjects were compared to non-test subjects.

Additionally, when designing lessons teachers used the everyday lesson plan

format. The effect of context, technology understanding, pedagogy practice, and subject

expertise can be considered possible influences upon a teachers’ TPACK.

Samples and Participants

Lesson plans for the 2012-2013 school year were selected from two groups of

western Pennsylvania high school teachers analyzed using a TPACK-Based Technology

Integration Assessment Rubric. The archived lesson plans were chosen because of the

purposeful samples and the relevance to the action research chosen. A total of 435 lesson

plans were selected for the research. The research was completed in November and

December of 2014. Two months later, randomly 15 % of the lesson plans were analyzed

again to test intra-rater reliability.

High-stake tested subject (Biology and English courses for 10th and 11th graders)

teacher lesson plans and non-tested subject (English courses for 12th graders, Chemistry,

Accelerated Chemistry, and AP Biology) teacher plans were selected because of the

relevance or lack of relevance of these subjects to the state mandated Pennsylvania

Keystone Examinations. The Keystone Examinations are course-ending evaluations

HIGH-STAKE TESTING

49

developed by the state of Pennsylvania for students completing algebra, biology, and

English literature courses. School districts determine when students are to take the test

based on the completion of the designated course standards. Comparatively, non-tested

subject teacher lesson plans were analyzed for technology integration. At the time of the

lesson plan development, the non-tested subjects selected course standards, or subject

areas were not tested by a state developed Keystone Examination.

The 435 lesson plans included high-stake tested and non-tested samples: 170

high-stake tested English lesson plans; 50 non-tested English lesson plans; 143 high-stake

tested science lesson plans; and 72 non-tested science lesson plans.

Instrument Rubric

The instrument chosen to evaluate lesson plan samples was the TPACK-Based

Technology Integration Assessment Rubric (Harris et al., 2010). After receiving

permission from the authors, the rubric was utilized to evaluate in-service teacher lesson

plans.

The validity of the TPACK-Based Technology Integration Assessment Rubric

was examined by using construct validity and face validity (Harris, Grandgenett, &

Hofer, 2008). Construct validity is defined as how well theories are converted into actual

measures while face validity is how well the construct appears to be measured

(socialresearchmethods.net, 2006). The authors validated the rubric by utilizing TPACK

experts to interpret and comment on the rubrics ability to reflect technology integration, a

construct validity strategy (Harris et al., 2010). Additionally, a face validity strategy was

completed and validated by classroom teachers’ analysis and comments concerning the

utility of the rubric and technology integration knowledge acquisition (Harris et al.,

HIGH-STAKE TESTING

50

2010).

Reliability was tested through two university studies that utilized 15 classroom

teachers to evaluate 15 pre-service teachers lesson plans utilizing the rubric (Harris et al.,

2010). The lesson plans represented various content areas and grade levels of pre-service

teachers that use technology in their lesson planning (Harris et al., 2010). Lastly, the

reliability of the rubric was verified by using the four statistical strategies of intraclass

correlation coefficient, second percent agreement procedure, Cronbach’s Alpha, and test-

retest reliability as represented by percent agreement based on scoring completed one

month apart by the same teachers (Harris et al., 2010). The computed internal consistency

of the rubric was .902 for a Southwestern trial and .911 for the Midwestern trial (Harris et

al., 2010).

The TPACK-Based Technology Integration Assessment Rubric consists of the

four criteria of curriculum goals and technologies, instructional strategies and

technologies, technology selection, and fit. Each criterion is categorized by a numerical

rating of 4, 3, 2, or 1 (see Appendix A). A description of each criterion and a breakdown

of the corresponding rating are needed to understand how technology integration is

determined. Curriculum goals and technologies are criteria that measures the TCK

(technology content knowledge) of lesson plans from a rating of 4, technologies are

strongly aligned with curriculum goals, to a range of 1, technologies are not aligned with

any curriculum goals (Harris et al., 2008). Instructional strategies and technologies

measures the TPK (technology pedagogical knowledge) of prepared lesson plans by

rating technology that optimally supports instruction as a 4, and technology that does not

support instruction as a 1 (Harris et al., 2008). The third criteria measure the selection of

HIGH-STAKE TESTING

51

technology and how it is compatible with curriculum goals and instruction (Harris et al.,

2008). Technology selection is rated as a four exemplary, three appropriate but not

exemplary, two marginally appropriate, or one inappropriate (Harris et al. 2008). Lastly,

the final criterion of the rubric measures the content, teaching strategies, and technology

compatibility fit differing from that of the other criteria that evaluate technology use and

technology selection (Harris, Grandgenett, and Hofer, 2008). This category measures the

compatibility as a four when the technology, pedagogy, and content fit together strongly

as opposed to a one that exemplifies no compatibility (Harris et al. 2008).

According to Harris, Grandgenett, and Hofer (2008), the rubric has not been

tested using the lesson plans of in-service teachers because of the lack of details that

many in-service daily lesson plans provide. However, the authors of the instrument

believe that the rubric would be viable if the lesson or project plans are written in enough

detail to be evaluated by educators (Harris, Grandgenett, & Hofer, 2008). As a result, the

rubric was used with detailed lesson plans for in-service teachers in selected tested and

non-tested subjects.

Materials

Each lesson plan was located on the high school computer drive accessible to

administrators and teachers. The TPACK-Based Technology Integration Assessment

Rubric was available online at http://activitytypes.wm.edu.

Lesson Plan Format

All teachers used a common lesson plan template format (see Appendix B). The

lesson plan format allowed for projects or units and was set up for multiple days of

instruction. For example, the template was entitled, “Rigor and Relevance Lesson Plan,”

http://activitytypes.wm.edu/

HIGH-STAKE TESTING

52

that included a timeline and an activity title. Based on these areas, an educator

determined that the lesson was scheduled for multiple days and that a project or unit

activity would be completed. According to Markham (2003), project-based learning

encompasses projects that last from one or two weeks to possibly all year. Moreover, the

lesson plan format used included Pennsylvania and common core standards that created a

more detailed planning process.

The school issued the common lesson plan format template for teacher use that

included sections labeled as student learning, performance tasks, strategies to

differentiate, essential skills, formative assessments, summative assessments, and a

scoring guide for assessment that permits the use of self-reflecting rubrics. The rigor and

relevance lesson plan template did not allocate a specific section for technology

integration or media use. However, teachers were expected to plan for the integration of

technology when applicable. Teachers were expected by the school district to include

state technology standards in their planning. English teachers were expected to follow the

academic standards for reading, writing, speaking and listening developed in 2009 by the

state. The technology was addressed in the standard 1.8 for Research, and standard 1.9

Information, Communication, and Technology Literacy (Pennsylvania Department of

Education, 2015). Science teachers were expected to follow the academic standards for

science and technology and engineering developed in 2010. Science standards include

Biology standards 3.1 and Technology and Engineering Education Standards 3.4

(Pennsylvania Department of Education, 2015). Moreover, all subject teachers are

expected to utilize reading, writing, speaking, and listening standards when planning.

HIGH-STAKE TESTING

53

Quantitative Variables

The independent variables in this study were the subject taught and whether the

subject was a high-stake tested subject or a non-tested subject. The dependent variable

was technology integration as measured through the use of the TPACK-Based

Technology Integration Assessment Rubric.

Data Collection Procedure

Each lesson plan was downloaded, separated by subject, and analyzed for

technology integration. Before utilizing the TPACK-Based Technology Integration

Assessment Rubric, each teacher plan was downloaded and printed for analysis by

utilizing copies of the rubric. Following the first scoring, 15 % of the scored plans for

English and science were randomly selected for a second scoring. First and second

scores were compared for intra-rater agreement.

Data Analysis Plan

Quantitative data analysis was performed through the use of descriptive statistics,

ANOVA and t-tests. Initially, data collected from the high-stake subjects were analyzed

utilizing an ANOVA test to find any statistical difference among the lesson plans

measured results. Subsequently, an ANOVA test was utilized for the non-test subjects

looking for differences in the mean. Following the ANOVA tests, a t-test was conducted

comparing data between high-stake subjects and non-test subjects data.

The first assumption in regards to the data was that there was not a difference

statistically among high-stake subjects. This assumption was based on the premise that

teachers utilize technology similarly in high-stake contexts. The similarity was believed

to be dependent on teaching habits and not the integrating of technology, pedagogy, and

HIGH-STAKE TESTING

54

content.

The second assumption in regards to the data was that there would not be a

difference statistically among non-tested subjects. Although the subjects differ in content,

the assumption was that the context of a non-tested subject would allow for more

technology integration. As a result, teachers would engage in technology integration.

The third assumption was that non-test subjects would provide data that differs

from high-stake tested subjects. Specifically, the TPACK data of non-tested subject

teachers would differ because teachers’ teaching would not be guided by testing. As a

result, more autonomy from testing would allow for more integrating of technology. The

habits of non-tested subject teachers would not be a barrier to technology integration.

Traditional teacher-centered habits of teaching would be replaced with the more

constructivist use of technology. Non-tested subject teachers would take risks and engage

in technology integration more readily. High-stake test subject teachers would focus on

covering content and utilize teacher-centered pedagogy that limits technology integration

and supports their habitus.

Ethical Considerations

The ethical considerations of the study are threefold. First, the intent of the

research was to assist the school district with the integration of technology being

improved or recognized by my research. Most importantly, the research intent was to

assist teachers and administrators in their pursuit of improving student learning. Second,

the anonymity of the participants is very important based on the factor of working

together in the same school district. Moreover, as professionals it is imperative that we

respect each other’s work product and realize that all teacher lesson plans have validity

HIGH-STAKE TESTING

55

and merit. During the research process, it was essential to remain unbiased toward any

data or results collected. Consequently, each teacher lesson plan was scored randomly to

completion of all of the plans, blindly evaluated with names covered and identified by

subject, and scored with letter identification. Lastly, the archived lesson plans were

reviewed in the study after permission was granted by the superintendent of schools and

permission from the Creighton University IRB.

Summary

The methodology utilized in this study was a comparative approach to the

technology, pedagogy, and content knowledge of teachers in different contexts. Data

were obtained through an analysis of teacher lesson plans utilizing the TPACK-Based

Technology Integration Assessment Rubric. The lesson plans selected for evaluation

were based on the subject taught and whether a state standardized test assessment was

required or not. Descriptive statistics and an ANOVA test were completed for both high-

stake test subjects and non-test subjects. Next, a comparison based on t-test results were

completed to statistically determine if differences exist based on teaching context. A

second scoring of randomly selected lesson plans was completed for intra-rater reliability.

Lastly, Chapter Four presents the data analysis procedures, overall results, and data

analysis and synthesis for the study.

HIGH-STAKE TESTING

56

CHAPTER FOUR: FINDINGS AND THE EVIDENCE-BASED SOLUTION

Introduction

This chapter includes a description of the proposal and aim of the study followed

by a review of the data analysis, research questions that guided the study, and an analysis

of the data. The results of ANOVA and t-Test indicated that a difference in results was

evident by subject. The data for Science supported the research hypothesis that high-stake

testing was a barrier to technology integration. In contrast, results for English did not

support the research hypothesis that high-stake testing context was a barrier to technology

integration.














contexts.

HIGH-STAKE TESTING

57

Aim the Study








Data Analysis Procedures

The context of high-stake testing lesson planning was examined for the

specifically tested areas of English and Science. Comparatively, the context of non-

tested English and Science lesson plans were investigated for comparative purposes.

High-stake tested subjects were mandated by the state of Pennsylvania based on

curriculum standards for the subjects of Biology, Literature and English, and Algebra.

The research questions were examined through the acquisition of lesson plans and

evaluated through the use of a TPACK-Based Technology Integration Assessment

Rubric. A total of 435 lesson plans were evaluated: 170 high-stake tested English; 50

non-tested English; 143 high-stake Biology; and 72 non-tested Science. Data were

recorded on an Excel spreadsheet and analyzed using Stat Plus for Mac computers.

Approximately two months after the first scoring, a second TPACK score was calculated

upon 33 English, and 33 Science randomly selected lesson plans or 15% of the total

lesson plans for intra-rater reliability. Intra-rater reliability is the amount of reliability of

the test results based on a comparison of two or more occasions by a single researcher

HIGH-STAKE TESTING

58

(Intra-rater reliability, 2015). An exact percentage and an exact and adjacent percentage

were completed for both scores (Table 4.3). The adjacent percentage was determined

when the first and second scores fell within one level or one scoring point.

The tests used to compare means and determine if independent samples were

statistically different were the ANOVA and t-test respectively. The significance level for

each test was .05. ANOVA testing was used to compare the means among sets of data

within specific tested or non-tested subjects. Specifically, four ANOVA tests were used

to compare means for technology integration among English tested subject lesson plans,

English non-tested subject lesson plans, Science tested subject lesson plans, and Science

non-tested subject lesson plans independently.

Following the ANOVA tests, t-Tests were utilized to compare the means for

technology integration between subject specific tested and non-tested lesson plans.

Specifically, the first t-test compared the means between high-stake tested English

subject lesson plans and non-tested English subject lesson plans. The second t-test

compared the means between high-stake tested Science subject lesson plans and non-

tested Science subject lesson plans. Prior to the ANOVA testing, the mean and standard

deviation was collected through the use of Stat Plus for each subject and specific teacher.

(See tables 4.1 and 4.2).

Research Questions

The research questions focused on two specific areas of comparison. Research

questions one through four guided the study as to the differences among teachers’

TPACK score within the same subject and context. Comparing the TPACK score within

a specific subject and context provided data as to statistical similarities and differences

HIGH-STAKE TESTING

59

among analogous teachers. ANOVA testing was used to determine significant statistical

differences among the following specific subject and context teachers: high-stake tested

English; non-tested English; high-stake tested Biology; and non-tested Science.



technology integration for the high-stake tested subjects of English for 10th and 11th

graders among these teachers?



technology integration for the non-tested subjects of English for 12th graders among these

teachers?



technology integration for the high-stake tested science subject of Biology among these

teachers?





Research questions five and six guided the comparison between high-stake tested

subjects and non-tested subjects for technology integration in teacher planning. The

objective of these research questions was to compare the technology integration planning

of teachers between high-stake tested and non-tested subjects, determining if the context

HIGH-STAKE TESTING

60

was a barrier to technology integration. Statistical comparisons were made between the

following high-stake tested and non-tested subject lesson plans: high-stake tested English

and non-tested English; and high-stake tested Biology and non-tested Science.



integration for English between high-stake tested subject teachers and non-tested subject

teachers?



integration for Science between high-stake tested subject teachers and non-tested subject

teachers?

Analysis of Data

Each lesson plan was analyzed and scored using the TPACK-Based Technology

Integration Assessment Rubric. TPACK data was inputted into Excel to determine the

mean and standard deviation descriptive statistics for each subject teacher and is

presented in table format (See Table 4.1 and Table 4.2). A second score using the

TPACK-Based Technology Integration Assessment Rubric was performed on 15% of the

lesson plans to compare for intra-rater reliability (See Table 4.3). Intra-rater reliability

was determined by calculating the percentage of exact and adjacent first and second

scores.

Following the input of data, ANOVA and t-Tests were performed using Stat Plus

for Mac. The inferential statistic results of the ANOVA test are F (between groups

degrees of freedom and within groups degrees of freedom), the value of F and p

HIGH-STAKE TESTING

61

(probability) for each group. The comparative t-tests are reported in the findings and

include t (degrees of freedom), the value of t and p (probability).

Results for Research Questions

ANOVA Testing for High-Stake Tested English Lesson Plans

A one-way subjects analysis of variance (ANOVA) was conducted to compare the

relationship among English lesson plans of tested classes for technology integration. 170

lesson plans for eight teachers from the entire school year were analyzed and scored for

technology integration. The independent variable was the lesson plan of English teachers

of tested subjects. The dependent variable was the technology integration score. The

ANOVA result was not significant F (7,162) = 2.01, p = .057 (See Table 4.1). As per the

results, there was not a significant statistical difference in technology integration for the

high-stake subject of English among high-stake English teachers. However, as p = .057,

a larger sample size could have resulted in a significant statistical difference.

ANOVA Testing for Non-tested English Lesson Plans


relationship among English lesson plans of non-tested classes for technology integration.

50 lesson plans for five teachers from the entire school year were analyzed and scored for

technology integration. The independent variable was the lesson plans of English teachers

of non-tested subjects. The dependent variable was the technology integration score. The

ANOVA was not significant F (4,45) = 1.81, p = .144 (See table 4.1). As per the results,

there was not a significant statistical difference in technology integration for the non-

tested subject of English among non-tested subject English teachers.

HIGH-STAKE TESTING

62

ANOVA Testing for Tested Science Lesson Plans


relationship among science lesson plans of tested Biology classes for technology

integration. 143 lesson plans for four teachers from the entire school year were analyzed

and scored for technology integration. The independent variable was the lesson plans of

biology teachers of tested subjects. The dependent variable was the technology

integration score. The ANOVA was significant F (3,139) = 4.47, p = .005 (See Table

4.2). As per the results, there was a significant statistical difference in technology

integration for the high-stake subject of science among high-stake tested science teachers.

ANOVA Testing for Non-tested Science Lesson Plans


relationship among Science lesson plans of non-tested classes for technology integration.

72 lesson plans for four teachers from the entire school year were analyzed and scored for

technology integration. The independent variable was the lesson plans of Science

teachers of non-tested subjects. The dependent variable was the technology integration

score. The ANOVA was significant F (3,68) = 6.47, p = .0006 (See Table 4.2). As per

the results, there was a significant statistical difference in technology integration for the

non-tested subject of Science among non-tested science teachers.

t-Test English

An independent t-test was administered to compare technology integration of

lesson planning between high-stake tested and non-tested English subjects. There was not

a significant difference in the scores for high-stake tested English (M = 7.18, SD = 3.40)

and non-tested English (M=7.7, SD=3.74). The t-Test results were t (216) = 0.919, p =

HIGH-STAKE TESTING

63

0.359. The results indicate that tested English and non-tested English technology

integration was not significantly different.

t-Test Science

An independent t-test was administered to compare technology integration of

lesson planning between high-stake tested Biology and non-tested Science subjects.

There was a significant difference in the scores for high-stake tested Science (M=7.43,

SD=2.90) and non-tested Science (M=8.53, SD=3.67). The t-test results are t (213) =

2.38, p = 0.018. The results indicate that tested Biology and non-tested Science

technology integration scores were significantly different. As per the results, non-tested

Science technology integration scores were higher and significantly different than tested

Biology technology integration scores.

HIGH-STAKE TESTING

64

Table 1 Means and standard deviation for technology integration among English

teachers

Teacher Lesson Plans English Technology Integration

Tested English N M SD

Teacher AA

34 8.09 3.46

Teacher BB

14 7.64 2.82

Teacher CC

21 6.90 2.79

Teacher DD

38 6.47 3.13

Teacher EE

25 6.64 3.73

Teacher FF

19 7.26 3.36

Teacher GG

10 9.80 4.13

Teacher HH

9 5.44 2.96

Non-Tested English N M SD

Teacher II

10 10.40 5.10

Teacher JJ

11 6.91 3.91

Teacher KK

8 7.38 2.83

Teacher LL

9 7.33 3.32

Teacher MM 12 6.67 2.31

HIGH-STAKE TESTING

65

Table 2 Means and standard deviation for technology integration among Science teachers

Teacher Lesson Plans Science Technology Integration

Tested Science N M SD

Teacher NN

21 9.48 2.89

Teacher OO

43 6.93 7.07

Teacher PP

42 7.24 2.89

Teacher QQ

37 7.08 2.78

Non-Tested Science N M SD

Teacher RR

12 9.83 4.82

Teacher SS

21 5.86 2.63

Teacher TT

19 9.63 3.02

Teacher UU 20 9.50 3.12

HIGH-STAKE TESTING

66

Table 3 Intra-rater reliability

English

Science

Random Plan First Score

Second Score Random Plan First Score

Second Score

1 4.00 4.00 1 5.00 6.00 2 4.00 4.00 2 16.00 13.00 3 8.00 6.00 3 8.00 8.00 4 4.00 4.00 4 12.00 11.00 5 4.00 4.00 5 4.00 4.00 6 4.00 4.00 6 12.00 12.00 7 12.00 9.00 7 4.00 4.00 8 4.00 4.00 8 5.00 4.00 9 4.00 4.00 9 4.00 4.00 10 9.00 8.00 10 9.00 8.00 11 4.00 4.00 11 4.00 4.00 12 4.00 4.00 12 5.00 8.00 13 4.00 4.00 13 5.00 6.00 14 4.00 4.00 14 12.00 10.00 15 8.00 7.00 15 5.00 5.00 16 5.00 4.00 16 9.00 9.00 17 4.00 4.00 17 13.00 12.00 18 12.00 11.00 18 12.00 11.00 19 4.00 4.00 19 14.00 12.00 20 8.00 6.00 20 5.00 5.00 21 10.00 12.00 21 8.00 8.00 22 9.00 8.00 22 8.00 8.00 23 4.00 4.00 23 16.00 15.00 24 4.00 4.00 24 9.00 10.00 25 12.00 14.00 25 5.00 6.00 26 12.00 12.00 26 12.00 11.00 27 8.00 7.00 27 8.00 8.00 28 10.00 8.00 28 4.00 4.00 29 4.00 4.00 29 4.00 4.00 30 5.00 5.00 30 8.00 8.00 31 10.00 9.00 31 12.00 9.00 32 4.00 4.00 32 5.00 5.00 33 4.00 4.00 33 5.00 5.00

Exact % 51.51 % 63.63 % Exact and Adjacent % 81.81 % 84.84 %

HIGH-STAKE TESTING

67

Intra-rater Reliability Measure

Using the TPACK-Based Technology Integration Assessment Rubric a second

score was calculated from random lesson plans two months after the first score

calculations. First scores were compared with second scores to determine a percentage of

intra-rater reliability or agreement. Comparisons were made for 33 English, and 33

Science lesson plans equaling 15% of the total lesson plans scored. Results indicate that

51.51% of English and 63.63% of Science were exact in agreement. A second calculation

based on exact scores and adjacent scores, scores one point different from the first scores,

indicated an 81.81% for English and 84.84% for science supporting a substantial intra-

rater agreement.

Analysis and Synthesis of Findings

This research study was conducted to determine if high-stake testing was a barrier

to technology integration in high school classrooms. Based on a literature review of

technology integration, high-stake testing, and teaching context, the investigator

identified a need to examine the teaching context of high-stake testing being a barrier to

technology integration. The study examined the quantitative differences between high-

stake tested and non-tested contexts for the subjects of English and Science.

Teacher archived lesson plans were analyzed and scored using a Technology-

Based Integration Assessment Rubric for measuring teacher technology integration. The

findings of the scoring were divided into two major types of data. The first set was results

from ANOVA testing, comparing the technology integration scores from the lesson plans

of English and Science teachers among the same high-stake tested and the same non-

tested subjects. Specifically, technology integration was compared among teachers of the

HIGH-STAKE TESTING

68

high-stake tested subject of English for 10th and 11th-grade students. Subsequently, results

were collected and compared among non-tested subject teachers of English for 12th-grade

students.

Following the results for English, the technology integration among science

teachers of high-stake tested Biology was collected and compared. Subsequently, non-

tested Science subject data were collected, scored, and compared for technology

integration.

The second set of data were results of a comparison of technology integration

scores from the lesson plans of teachers between the high-stake subject and non-tested

subjects using a t-test. Specifically, English high-stake tested subject results were

compared to English non-tested subject results. Subsequently, high-stake tested Biology

results were compared with non-tested Science subject results.

Analyzing and synthesizing the data from the perspectives of testing context and

subject can begin with a review of the research question, the tested null hypothesis, and

the results.

Technology Integration Among High-Stake Tested English



technology integration for the high-stake tested subjects of English for 10th and 11th-grade

students among these teachers?

H01: In regards to teacher lesson plans, there was not a significant statistical

difference in technology integration for the high-stake tested subjects of English

for 10th and 11th grade students among these teachers.

HIGH-STAKE TESTING

69

The results among English teachers of 10th and 11th-grade students, a high-stake testing

context, indicated that there was not a significant statistical difference in technology

integration for the high-stake subject of English among high-stake English teachers.

Therefore, the null hypothesis was retained.

Technology Integration Among Non-Tested English


In regards to teacher lesson plans, was there a significant statistical significant difference

in technology integration for the non-tested subject of English for 12th-grade students

among these teachers?


difference in technology integration for the non-tested subject of English for 12th-

grade students among these teachers.

The results among English teachers of non-tested English for 12th-grade students also

indicated that there was not a significant statistical difference in technology integration

among the non-tested subject data. Therefore, the null hypothesis was retained.

Technology Integration Between High-Stake Tested and Non-Tested English



integration for teachers of English between the high-stake tested subjects of English for

10th and 11th-grade students and for the non- tested subject of English for 12th-grade

students.

H05: In regards to lesson plans, there was not a significant statistical difference in

technology integration for teachers of English between the high-stake tested

HIGH-STAKE TESTING

70

subjects of English for 10th and 11th-grade students and for the non-tested subject

of English for 12th-grade students.

Comparatively, the result of teacher technology integration between high-stake tested

English for 10th and 11th-grade students and non-tested English for 12th-grade students

was not a significant statistical difference. The null hypothesis was retained.

In contrast to the English results, the data for the subjects of Biology and non-

tested Science indicated statistical differences.

Technology Integration Among High-Stake Tested Biology


In regards to teacher lesson plans, was there a statistical difference in technology

integration for the high-stake tested science subject of Biology among these teachers?


difference in technology integration for the high-stake tested subject of Biology

among these teachers?

The results among Biology teachers indicated that there was a significant statistical

difference in technology integration for the high-stake subject of Biology among high-

stake English teachers. Therefore, the null hypothesis was rejected.

Technology Integration Among Non-Tested Science

Research Question 4:





HIGH-STAKE TESTING

71

difference in technology integration for the non-tested Science subjects of

Chemistry, Accelerated Chemistry, and AP Biology among these teachers.

The results among Science teachers indicated that there was a significant statistical

difference in technology integration for the non-tested Science subjects of Chemistry,

Accelerated Chemistry, and AP Biology among these teachers. Therefore, the null

hypothesis was rejected.

Technology Integration Between High-Stake Tested Biology and Non-Tested Science




Chemistry, Accelerated Chemistry, and AP Biology teachers?

H06: In regards to lesson plans, there was not a significant statistical difference in

technology integration for teachers of Science between the high-stake tested

subject of Biology and for the non-tested Science subjects of Chemistry,

Accelerated Chemistry, and AP Biology.

Comparatively, the result of teacher technology integration between high-stake tested

Biology and non-tested Science was a significant statistical difference. The null

hypothesis was rejected.

Based on the results of the data, an examination of the three assumptions made in

the data analysis plan for this research was essential in order to understand the

relationships between subject, testing context and technology integration barriers. The

first assumption in regards to the data for both English and Biology was that there would

not be a significant statistical difference in technology integration among high-stake

HIGH-STAKE TESTING

72

subject teachers because of teaching habits. Based on the results of the data, the results

for this assumption are inconclusive and subject dependent. Specifically, results for the

subject of English indicated that technology integration did not significantly differ among

high-stake tested English teachers. The lack of a significant difference in technology

integration for high-stake tested English teachers supported the assumption that teacher

habits in technology integration planning are similar within the same context.

In contrast, the Biology data revealed that a significant statistical difference

among teachers’ planning for technology integration occurred. Consequently, the

different result for Science did not support the assumption. As a result, the assumption is

supported for English but not for Biology. These results indicate that technology

integration barriers for high-stake subjects are subject dependent for English, but teacher

dependent for Biology.

The second assumption for the data was that there would not be a significant

difference statistically among non-tested subjects. Although the subjects differ in content,

the assumption was that the context of a non-tested subject would allow for more

technology integration. As a result, many non-tested subject teachers would engage in

technology integration.

Once again, the results differed by subject. English results revealed that the

technology integration data was not significantly different statistically. In contrast, the

Science data revealed significant differences statistically concerning teachers’ technology

integration. Once again, the results revealed that technology integration barriers are

subject dependent based on the data for English but teacher dependent for Science.

The third assumption was that non-test subjects would provide data that differs

HIGH-STAKE TESTING

73

statistically from high-stake tested subjects. Specifically, the TPACK data of non-tested

subject teachers would differ because teachers’ technology integration would not be

influenced by a testing context. Autonomy from a testing context would increase the

integrating of technology with content and pedagogy. As a result, the teaching habits of

non-tested subject teachers would not be a barrier to technology integration.

Results for English again revealed that a significant statistical difference did not

exist between high-stake subject teachers and non-tested subject teachers. In contrast,

high-stake Biology teachers’ technology integration differed statistically from non-tested

Science teachers. As a result, the assumption that data would differ based on testing

context was not conclusive for both subjects.

The analysis of the data reveals two important factors concerning technology

integration and the barrier of high-stake testing. First, the results indicate a teacher

dependent or subject specific influenced result. Accordingly, Science teachers differed

significantly among and between contexts in their integration of technology while

English teachers did not significantly differ statistically in their technology integration

among or between contexts. Second, high-stake subject teachers integrated technology

less than or equal to non-tested subject teachers. Specifically, the results indicated that

statistically non-tested Science teachers integrated technology more than high-stake

tested Biology teachers. This result indicated a support for the hypothesis that a high-

stake tested context is a barrier to technology integration. Comparatively, both high-

stake and non-tested English teachers integrated technology an equal amount.

Consequently, this result indicated that a high-stake tested context did not specifically

determine technology integration barriers or successes. As a result of the study, a

HIGH-STAKE TESTING

74

proposed solution for determining the effect of high-stake testing upon technology

integration is required.

Summary

This chapter began with a review of the data analysis and research questions that

guided the study followed by analysis of the data, results for research questions, intra-

rater agreement, and analysis and synthesis of findings.

The presentation and results of the study were categorized into descriptive

statistics for teacher lesson plans (Table 4.1 and Table 4.2), ANOVA results, and t-test

results. ANOVA testing was completed among high-stake tested English, non-tested

English, high-stake tested science and non-tested science lesson plans. The results for

high-stake tested English indicated that no significant difference existed in technology

integration among teacher lesson plans. The p = .057 value signified that the means

among the high-stake tested samples supported the null hypothesis that quantitatively the

means are not significantly different.

The results for non-tested English indicated that no significant difference existed

in technology integration among teacher lesson plans. The p = .144 value signified that

the means among the non-tested samples supported the null hypothesis that quantitatively

the means are not significantly different.

The results for high-stake tested science indicated that a significant difference

existed in technology integration among teacher lesson plans. The p = .005 value

signified that the means among the high-stake tested samples did not support the null

hypothesis. The alternative hypothesis was supported that the means are significantly

different.

HIGH-STAKE TESTING

75

The results for non-tested science indicated that a significant difference existed in

technology integration among teacher lesson plans. The p = .0006 value signified that the

means among the non-tested samples did not support the null hypothesis. The alternative

hypothesis was supported that the means are significantly different.

The results of the t-test between high-stake tested English and non-tested English

revealed that a significant quantitative difference did not exist for technology integration.

The p = .359 value signified that the means between the samples supported the null

hypothesis.

The results of the t-test between high-stake tested science and non-tested science

revealed that a significant quantitative difference did exist for technology integration. The

p = .018 value signified that the means between the samples did not support the null

hypothesis. Consequently, non-tested science technology integration scores were greater

than high-stake tested science scores.

The data indicated that a difference in results was evident by subject. ANOVA

and t-tests for high-stake tested English and non-tested English indicated no significant

differences among and between lesson planning for technology integration. In contrast,

ANOVA and t-tests for high-stake tested science and non-tested science resulted in a

significant statistical difference among and between lesson planning for science. Lastly,

the data for science supported the research hypothesis that high-stake testing was a

barrier to technology integration. Data for English did not support the research

hypothesis.

Chapter five includes a Six-Step Growth Design Process designed to investigate

the context of high-stake testing being a barrier to technology integration. Sections that

HIGH-STAKE TESTING

76

support and influence the process, leadership sections that incorporate hurdles and

solutions, and recommendations for further research follow an explanation of the process.

HIGH-STAKE TESTING

77

CHAPTER FIVE: CONCLUSIONS AND RECOMMENDATIONS

Introduction

This chapter will review the summary, purpose, and aim of the study that focused

on the lesson plans for high school teachers, technology integration knowledge of

teachers, and the context of high-stake tested subjects of English and Biology being a

barrier to technology integration. Based on the statistical evaluation of lesson plans, it

was determined that the technology integration planning of teachers was influenced by

the high-stake context of Biology as compared to non-tested Science subjects, but the

English technology integration planning of teachers was not influenced by the high-stake

tested context. As a result, a Six-Step Growth Design Process is needed to investigate the

influence of subject and individual teacher planning upon technology integration

knowledge for high-stake subjects.

Based on an analysis of the data, the Six-Step Growth Design Process was

developed to examine the planning process of teachers for possible barriers to technology

integration. Following an explanation of the process, existing support structures and

resources, influential policies, potential barriers, budget and legal issues, change theory

related to the Six-Step Growth Design Process, and internal and external issues related to

implementation are introduced.

Included in this chapter are leadership sections that incorporate possible hurdles

and solutions to the implementation of the Six-Step Growth Design Process. Following

these explanations, recommendations for further research that investigates elementary

and middle level educator technology integration as well as a study of the lesson planning

HIGH-STAKE TESTING

78

and teaching habitus of specific high-stake tested subject teachers related to student

achievement is necessary.

Summary of the Study


for technology integration in high-stake tested and non-tested contexts. The aim of this

research was to compare the amount of technology integrated into high-stake tested and

non-tested subjects, determining if high-stake testing was a barrier to technology

integration in high school classrooms. A TPACK-Based Technology Integration

Assessment Rubric was used to evaluate the lesson planning of 435 teachers in English

and Science subjects in either a high-stake tested or non-tested contexts. ANOVA testing

was completed to measure statistically the differences among the lesson planning within

the same subject area and context while t-tests were completed for comparison of high-

stake tested and non-tested subjects for Science or English. The results of the study

indicated that technology integration was influenced by context when comparing high-

stake tested Biology with non-tested Science subjects. In contrast, results between

English subjects did not support the hypothesis that high-stake tested context was a

barrier to technology integration. Based on these results, a Six-Step Growth Design

Process was developed to further investigate the influence of subject and individual

teacher planning habitus upon the high-stake context barrier to technology integration.





HIGH-STAKE TESTING

79






classrooms. A better comprehension of lesson planning provides teachers and




contexts.

Aim the Study






content knowledge (TPACK) in various contexts, examine their instruction, and plan

lessons in the future accordingly.

Proposed Six-Step Growth Design Process Solution

The results of the data indicated that a high-stake testing context was a barrier to

technology integration between Biology and Science, but not statistically significantly

between English contexts. Consequently, high-stake testing as a barrier to technology

integration needs additional research or a solution for three reasons. First, the results for

HIGH-STAKE TESTING

80

English did not indicate a significant statistical difference in technology integration

planning among teachers of high-stake English or when comparing high-stake English

and non-tested English teachers. Consequently, additional data as to the planning of

lessons before, during, and after a lesson of a high-stake lesson would provide insight

into teachers’ TPACK.

Second, a study of high-stake Algebra would provide additional data as to the

planning habits and TPACK comprehension of other high-stake subject teachers. The

additional subject research will provide insight into the influence of the subject upon

technology integration and allow for a comparison of technology integration between

high-stake English, high-stake Biology, and high-stake Algebra.

Third, a study of lesson plans for high-stake testing before, during, and after

lesson planning will provide the designer of the lesson with information concerning their

instructional strategies. More importantly, the teacher will be able to measure and

examine their instructional habits with technology integration. Furthermore, a Six-Step

Design Process will examine the technology integration cultural habits of the school

building.

The Six-Step Growth Design Process was created for educators that provides

additional subject data in various contexts, provide educators with an opportunity to

reflect on their teaching with technology, gain data during all phases of planning, and

provide a collaboration piece applicable for teacher and administrator interaction (Figure

2). Furthermore, the Six-Step Growth Design Process will be implemented to increase

technology integration in the classroom and improve its use in different contexts. The

HIGH-STAKE TESTING

81

process will allow educators to examine the application of technology and reflect on

successes and areas of weakness in high school instruction.

As lesson plans provided the medium for examination of technology integration,

the first step in the process is to change the format of the lesson plan for two reasons.

First, the lesson plan format studied did not specifically reference technology integration

or TPACK. Second, the lesson plan format examined did not provide specific sections

for teacher reflection about the lesson taught, or technology integrated specifically.

Furthermore, a reflection section will provide teachers and administrators with a better

understanding of what technology was integrated, implemented, and insight into a

teacher’s teaching habitus. The new design of the lesson plan is displayed in Appendix 2.

Teachers will be provided the new lesson plan format during a professional development

day.

Step two will encompass professional development for teachers of high-stake

taught subjects concerning the integration of technology, pedagogy, and content.

Specifically, teachers will be provided training as to best practices in technology

integration that can be used for specific subjects. Expert teachers and administrators will

collaborate on the plan for a training day of best practices.

Step three will incorporate teachers designing lesson plans based on newly

acquired best practices and the new lesson plan format. Particular attention will be to

recognize best practices that enhance the integration of technology with high-stake

content and relevant instruction. Teachers will be instructed to meet with their peers and

create their technology integration lesson plans during planning time. Following the

HIGH-STAKE TESTING

82

completion of this step, administrators will examine the plans for best practice inclusion

weekly.

Step four will comprise the reflection piece of the examination of technology

integration for high-stake subject high school teachers. Teachers will be provided time to

reflect on their planning of technology integration and implementation of best practices

with their teaching peers and administrators. Teachers will meet with same subject

teachers during planning time once a week to discuss their reflections.

Step five will be an explanation to faculty and staff of the TPACK-Based

Technology Integration Assessment Rubric during professional development time. Expert

teachers and administrators will explain the categories of the rubric through samples.

Teachers will be encouraged to ask questions and practice scoring a sample lesson plan

for technology integration. The training will occur during professional development

time.

Step six will consist of teacher and administrative analysis of a teacher’s lessons

through the use of the TPACK-Based Technology Integration Assessment Rubric.

Administrators and teachers will analyze and score commonly selected lessons plans

separately and discuss their findings. The intention of this step is to build collaboration

and a common vocabulary between the administration and teaching staff. Moreover, data

collected can be used to compare and resolve barriers to technology integration for high-

stake testing subjects in high school instruction. The teacher and the administrator will

arrange the meeting time.

Following the Six-Step Process, administrators will analyze the process and

overall teacher technology integration knowledge. Particular emphasis will be to

HIGH-STAKE TESTING

83

examine the influence of the high-stake context on technology integration, possible

barriers, and the professional habitus of teachers. Lastly, the opportunities for teachers to

revisit the entire growth process, or specific steps are possible if applicable.

Support for the Six-Step Process from Data Collected

Based on the data collected, support for the implementation of the Six-Step

Growth Design Process was evident in three specific areas of the research study. First,

technology integration among high-stake subject teachers varied for English and Biology.

The data indicated that subject was a major factor in determining possible barriers to

technology integration. The results for Biology indicated scored levels of technology

integration less than Science while English resulted in equally scored levels. As a result,

any new review of technology integration for high school classrooms needs to consist of

examining the influence of the subject upon teacher technology integration planning and

implementation. The review can be understood by examining the TPACK of teachers by

subject and their subsequent reflection and self-scoring.

Second, the data revealed that testing context was an influential factor in

technology as well for the Biology and Science comparison. The results for English were

inconclusive. The Six-Step Growth Design Process will help educators understand

through planning, best practice acquisition, reflection, and collaboration whether the

context is a barrier or benefit to technology integration. Furthermore, teachers and

administrators will be able to understand that lesson planning review will indicate

significant areas of strength and weakness for technology integration and content

pedagogy integration.

HIGH-STAKE TESTING

84

Third, the Six-Step Growth Design Process will provide information as to the

habitus of teachers when incorporating technology. Specifically, the design process will

provide both the individual teacher and administrator an understanding of teachers’

abilities to plan technology, perceptions of implementation, and overall understanding of

their TPACK.

Lastly, the Six-Step Design Growth Process provides an avenue for collecting

new data that reveals barriers to technology integration in context. Furthermore, the

design process will determine if technology integration barriers are context related or

rooted in a teacher’s professional habitus.

HIGH-STAKE TESTING

85

Figure 2 Six-Step Growth Design Process

Existing Support Structure and Resources

The Six-Step Growth Design Process will be implemented through the current

teacher lesson plan development and evaluation process. Specifically, teachers provide a

weekly or bi-monthly lesson plan that is part of their professional responsibility. The

planning process is already part of the professional work responsibilities of the teacher.

Lesson plans are reviewed weekly by the school principal and evaluated for completeness

STEP 1 Introducing a New

Lesson Plan

Introduce new lesson format

Questions by teachers 2 hour training

STEP 2 Best Practice

Training

Expert teacher presented

Collaborative approach required

5 hour training

STEP 3 Lesson Plan

Creation

Provide constructive

feedback

Emphasize teacher

expertise

45 Minute planning per

week

STEP 4 Reflection

Culture of instructional

reflection Peer feedback

encouraged 45 Minute

planning per week

STEP 5 TPACK Rubric

Categories explained

Practice samples 3 hour training

STEP 6 Collaboration about Rubric

Positive areas of

improvement Common

vocabulary Arranged

meeting time

HIGH-STAKE TESTING

86

and quality. As part of the design process, the lesson plans can be reviewed for TPACK

development as well.

Best practices will be implemented through professional development that is part

of the requirements of being a teacher within the district and in the state of Pennsylvania.

Teachers in Pennsylvania are required to acquire so many hours of professional

development, Act 48 hours, over a certain period of time to maintain active certification

status (Pennsylvania Department, 2015).

The reflective and teamwork steps of the process will provide the teacher with

data as to their classroom responsibilities as reviewed through the evaluation process.

Teachers of the district are required to provide evidence of professional development,

narrative reflections on teaching and proof of peer collaboration. Hence, the Six-Step

Design Growth Process will provide the teacher with tools to improve their technology

integration and data to demonstrate many of their professional responsibilities required

for their formal job performance evaluation.

Policies Influencing the Six-Step Process

The Six-Step Growth Design Process for technology integration will be

influenced by one major policy. The school district professional evaluation process

influences the teacher evaluation procedures that teachers and principals participate in

each year. Tenured teachers are evaluated yearly and non-tenured teachers twice a year

through a differentiated evaluation process. Teachers are evaluated through clinical

observation, walkthrough evaluation, or peer collaboration evaluation. All of the

evaluations include teacher and principal interaction grounded in the Charlotte Danielson

Framework for teaching evaluation instrument. Furthermore, teachers are rated on

HIGH-STAKE TESTING

87

planning and preparation, instruction, classroom environment, and professional

responsibilities through observation, teacher portfolios, and documented support for

performance. As a result, the Six-Step Growth Design Process may contribute to a

teacher’s professional evaluation in many or all of the rated areas.

Potential Barriers to the Six-Step Growth Design Process

Adopting and implementing the Six-Step Growth Design Process will be met with

potential resistance. Distinctly, individual teachers will oppose the design process

completely. Others will resist parts of the process dependent upon their fear or resistance

to change. Overall, the implementation and success of the Six-Step Growth Design

Process will depend on overcoming resistance from individual teachers, gaining buy-in

by the majority of teachers, and a commitment from the administration to the process. In

particular, individual principal participation and diligence toward teacher improvement in

planning, instruction, and technology integration are paramount.

Budget and Legal Issues Related to the Six-Step Growth Design Process

Budgetary issues will be related to professional development costs. Any costs

would increase if consultants were added for instructional support or substitute teachers

employed for meeting times during the school day. Legal issues related to the Six-Step

Growth Design Process may transpire if teachers are unwilling to participate or cooperate

in the process. Specifically, a lack of participation by teachers would influence their

professional responsibility ratings influencing their overall performance. Subsequently,

assigned improvement plans or termination of employment could result depending upon

the severity of resistance to change.

HIGH-STAKE TESTING

88

Change Theory

The aim of this research was to determine if high-stake testing was a barrier to

technology integration. As examined, high-stake testing was a barrier in the Science

lesson plans but not for the English results. Consequently, the Six-Step Growth Design

Process was created to investigate the barrier of high-stake testing context upon

technology integration planning. As a result of this investigation, some important aspects

of change theory are important to review. Fullan (2006) expressed that seven premises

drive change theory or change knowledge for educational organizations wanting school

improvement. In particular, the premises of building capacity, learning in context, and

bias for reflective action apply to the success of the Six-Step Growth Design Process.

Building capacity is defined by Fullan (2006) as strategies employed to improve

student achievement while building teacher competencies, resources, and motivation. As

for the Six-Step Growth Design Process, building capacity is employed through best

practices acquisition, lesson planning reflection, and collaborative technology integration

discussions. In particular, the final step of teacher-administrator teamwork is paramount

to building capacity for positive change that develops educator competencies and

motivation.

Learning in context is the building of cultures where learning is pervasive and

part of the organization (Fullan, 2006). In particular, professional development that

introduces and teaches the best practices of technology integration develops learning in

context. Moreover, the continued use of best practices will establish a new culture of

technology integration use based on peer development.

HIGH-STAKE TESTING

89

According to Fullan (2006), a bias for reflective action powers many of the

premises that steer change. As for this study and practice, reflection is important to the

needed changes for improving teachers’ technology integration planning. Most

importantly, reflecting on technology integration successes and failures following

instruction will permit growth in future planning and overall instruction.

Internal/External Issues Related to the Six-Step Growth Design Process

The implementation of the Six-Step Growth Design Process may be inhibited if

leaders are not aware of potential internal or external issues. Internally, uncooperative

teachers or administration apathy could complicate implementation of the process.

Specifically, a lack of teacher motivation toward technology integration may become a

barrier if principals are not connecting with their staff concerning academic planning.

Furthermore, principal apathy towards the process will lead to teacher disinterest and

subsequent failure.

Externally, the implementation may be inhibited by professional development

time limitations and the testing culture. For example, new Pennsylvania State mandates

for teacher evaluation training, common core requirements, and other mandated training

could limit professional development days for technology integration improvements.

Implementation of the Six-Step Growth Design Process and Considerations

Leaders need to gauge the cultural climate of change within their schools when

implementing any new concept, remedy, or organizational change. Implementing a Six-

Step Growth Design Process will create a need for building a foundation of change

knowledge for all stakeholders within the organizational climate. Consequently, a

HIGH-STAKE TESTING

90

simplistic approach to changes in lesson planning, best practices, reflective actions, and

teamwork related to technology integration is imperative.

A simplistic approach toward implementation of the Six-Step Growth Design

Process can only be achieved through the diligent efforts of leaders, principals or

administrators. In particular, principals and administrators within the school of

implementation must present the changing knowledge in an organizational manner that is

non-threatening and workable for all of the teachers involved. Accordingly, the

following must be considered during implementation of the specific Six-Steps:

Step One: Introducing a New Lesson Plan

Explain the lesson plan format through examples. Answer questions provided by

teachers explaining similarities and differences in the new format.

Step Two: Best Practice Teaching

Teachers skilled in technology integration should lead the presentation of best practices.

A collaborative approach between technology experts, administrators and respected

teachers is required.

Step Three: Lesson Plan Creation

Provide constructive feedback concerning the planning of best practices for technology

integration. The teacher presented practices should be emphasized during creation.

Step Four: Reflection

Provide ample time and feedback that creates a culture of learning based on individual

reflection. Utilizing peer feedback is optional based on the individual teacher’s comfort

level with technology integration reflections.

HIGH-STAKE TESTING

91

Step Five: TPACK Rubric

Provide feedback that explains the incorporation of best practices with individual

reflections of technology integration planning. Previous plans should be examined and

provided feedback. Subsequent lesson planning should be discussed, and positive

suggestions offered.

Step Six: Collaboration and Rubric

Collaborate with teachers discussing their measuring of technology integration. An

emphasis should be placed upon positive areas of improvement and how a teacher’s

TPACK has increased.

Roles and Responsibilities of Key Players in Implementation

The key players in the implementation of the Six-Step Growth Design Process are

teachers and administrators. The development of technology integration knowledge will

be determined through the efforts of administrators spearheading the change while

implementation will be heavily dependent on teachers that initiate change for example.

The importance of these change agents is significant and necessary for changing the

culture one teacher at a time. Also, the administrator must pursue change agents, foster a

connection with them, and provide ample reward through respect or responsibility.

Leader’s Role in Implementing the Six-Step Growth Design Process

A leader must understand the process thoroughly to provide appropriate

suggestions and guidance. The Six-Step Growth Design Process is collaborative and

dependent upon teacher-administrator respect. Implementation and sustainability of the

process can only be achieved through trust and understanding that pitfalls and successes

will happen. Most importantly, the rapid change of technology will influence technology

HIGH-STAKE TESTING

92

integration in the classroom of the future; therefore, a collaborative approach is most

likely to sustain a culture of technology integration in any context.

Evaluation and Timeline for Implementation and Assessment

The timeline of the Six-Step Growth Design Process will be based on the abilities

and technology integration understanding of the participants. In general, the process will

begin with professional development and end with collaborative input from teachers and

administrative assessment. The assessment will be achieved through an examination of

calculated scores from the TPACK-Based Technology Integration Assessment Rubric.

Comparing subject data for teachers through the use of data analysis based on ANOVA

testing will be completed. Training for administrators will be provided if needed.

Following a collection of the data, administrators will be able to determine if

high-stake testing subjects are integrating technology differently or at a significant level

compared to other subjects. Furthermore, a determination as to specific teacher needs

and strengths can be measured. Lastly, a teacher survey reflecting on the Six-Step

Growth Design Process will be beneficial for understanding the administrative

effectiveness and teacher professional development for the future.

Convincing Others to Support the Six-Step Growth Design Process

The Six-Step Growth Design Process investigates the context of high-stake testing

as a barrier to technology integration. Important to the success of this process is the

motivating of teachers to participate fully. Consequently, convincing teachers of the

importance of participation can be achieved from two perspectives. First, teachers must

be convinced that technology integration is important for student success in their

classroom. Second, teachers must be persuaded that their efforts will be measurable, and

HIGH-STAKE TESTING

93

evidence of their success will be recognized. Both perspectives are achievable through

the current evaluation system of teachers and students based on test scores. As a result,

teachers are vested in the overall process because the results are measurable.

Critical Pieces Needed for Implementation and Assessment

The critical pieces needed for the implementation and assessment of the Six-Step

Growth Design Process are time for professional development, time for reflection and

peer meetings, and time for administrators to meet with teachers. As time is significant

for success, additional time will cost money in employing substitute teachers or paying

teachers to come in the summer or stay after school.

Internal and External Implications for the District

When implementing the Six-Step Growth Design Process, a leader must be aware

that internally, change is difficult. Specific to this initiative is the possibility of a

negative reaction to the focus upon technology. Uncooperative teachers may create

problems with parents or school board members to divert attention away from their

technology integration abilities. Consequently, administrators must be aware of this

possibility and promote the positive aspects of improving technology integration for

teachers and students. Externally, this plan could be adopted by other schools or used at

different educational levels.

Considerations for Leaders Facing Implementation

Principals face the difficulty of implementing change in a very traditional

organization. As a result, the principal must consider their role in the process and the

benefits to teachers and students. Overall, the principal must study the entire process to

discuss each step, anticipate teacher reactions to the process, and plan for resistance to

HIGH-STAKE TESTING

94

change. Furthermore, an understanding of the timing of steps and the collection and

interpretation of results is vital to the success of the process.

Evaluation Cycle

Each principal will evaluate the Six-Step Growth Design Process following the

implementation and completion of each step. The evaluation will be achieved through

the use of a reflective narrative based on the Evaluation Cycle Timeline (Figure 3).

Step 1: Effectiveness of New Lesson Plan Presentation

Principals must determine if the new lesson plan format is understood by teachers and

functional for use based on the professional development day presentation and practice

by teachers.

Step 2: Best Practice Implementation

Principals must evaluate if the best practice session was productive. Specifically, teacher

understanding can be understood from informal discussions as to their lesson planning

intentions based on best practices in the subject.

Step 3: Lesson Plan Creation

Evaluate lessons for teacher efforts at integrating technology through best practices.

Reexamine their overall understanding of the process to this point.

Step 4: Teacher Reflections

Discuss and evaluate the reflections of teachers concerning their use of best practices and


Step 5: TPACK Rubric

Assess teacher understanding of the TPACK rubric through rating of a lesson plan for

practice.

HIGH-STAKE TESTING

95

Step 6: Teamwork Discussions

Evaluate each teacher’s plan with the TPACK rubric and compare the teacher’s self-

evaluation score. Compare strengths and weaknesses of technology integration.

Yearly, principals are evaluated based on their interactions with teachers, the

establishment of school achievement goals, and school performance. Evaluation

narratives will provide the principal with data to solidify their position as a principal that

is goal oriented toward improving technology integration.

Figure 3. Evaluation Cycle Timeline

Implications for Action and Recommendations for Further Research

Teachers must deal with a variety of factors when planning for subject lessons.

In particular, planning for the integration of technology, content, and pedagogy can be

challenging in an era of high-stake testing. As a result, understanding a teacher’s

TPACK (technology, pedagogy, and content knowledge) can provide information as to

barriers to technology integration and lesson planning that may exist for a particular

teacher or group of teachers. For this study, English and Biology teachers’ TPACK of

high-stake tested subjects were compared with non-tested teachers. Furthermore,

teachers’ TPACK was evaluated among like subjects and testing contexts.

Lesson Plan Format Practice

Understood

Informal Discussions

of Best Practices

Evaluate Lesson Plans

Discuss and Evaluate

Reflections

TPACK Rubric

Practice Evaluation

Evaluate Using

TPACK Rubric

HIGH-STAKE TESTING

96

Based on the results of this study a Six-Step Growth Design Process was

developed. This action plan was created because the results were different and dependent

on the subject analyzed. Through ANOVA and t-test analysis, high-stake testing was

found to be a barrier to technology integration for the subject of Science. In contrast,

significant differences between high-stake tested and non-tested English did not occur.

Subsequently, the researcher found that a further investigation into differences

among subjects and similar contexts could provide data that can discover barriers to


The Six-Step Growth Design Process will provide data that teachers and

administrators may utilize to solve three technology integration concerns. First, the data

will provide evidence as to the integration of technology, pedagogy, and content

knowledge of teachers in specific subjects and contexts. Through completion of each

step of the process educators can gain a better understanding of how particular subject

teachers plan for integration based on their use of best practices, self-reflection of

technology integration, and evaluation of their planning by using a TPACK rubric.

Second, administrators can gain a better understanding of teachers’ habitus based

upon their planning, learning of best practices, self-reflection, and teamwork. Habitus of

teachers’ technology integration may be reflected in their traditional or constructive

methods of teaching that may inhibit or enhance their instruction.

Third, teachers can learn to adjust their planning and teaching based on their

experiences of using newly learned technology integration best practices. Furthermore,

the Six-Step Growth Design Process allows for successes and failures, reflection, peer

assistance, and teacher-administrator teamwork related to technology integration.

HIGH-STAKE TESTING

97

Two important areas of future research, as discovered in this study, apply to the

needs of most school districts and educators. First, a study of technology integration

barriers in the context of testing for elementary and middle-level teachers needs to be

addressed. The increased use of technology by students inside and outside of the

classroom has led to a major need for an understanding of technology integration and

planning for elementary teachers. In particular, teachers of 3rd through 8th graders need

strategies to engage diverse students while utilizing best instructional practices and lesson

planning.

Second, a study of the lesson planning and teaching habitus of specific high-stake

tested subject teachers related to student achievement is necessary. Specifically, a study

of the lesson planning for technology integration, observed teaching strategies, and

resulting student achievement on state testing for high school students will provide

educators with a better understanding of the relationship between student success and

teacher technology integration knowledge.

Summary of Chapter Five

This chapter included a review of the purpose, aim, and summary of the study

followed by considerations for leaders implementing the Six-Step Growth Design

Process. Educators wanting change in technology integration must consider problems

and solutions addressed in this chapter. Lastly, a recommendation for elementary and

middle-level teacher research, as it relates to technology integration, is warranted.

Furthermore, an investigation of teacher habitus in high-stake tested contexts is suggested

to contribute to the need for understanding barriers to technology integration in

classrooms.

HIGH-STAKE TESTING

98

References

Altun, T. (2013). Examination of classroom teachers’ technological, pedagogical, and

content knowledge on the basis of different variables. Croatian Journal of

Education, 15(2). Retrieved from http://hrcak.srce.hr/index.php

show=toc&id_broj=8571

Archambault, L. M., & Crippen, K. J. (2009). Examining TPACK among K-12 online

distance educators in the United State. Contemporary Issues in Technology and

Teacher Education, 9(1). Retrieved from

http://www.citejournal.org/vol9/iss1/general/article2.cfm

Au, W. W. (2008). Devising inequality: a Bernsteinian analysis of high‐ stakes testing

and social reproduction in education. British Journal of Sociology of Education.

29:6,639-651. doi:10.1080/01425690802423312

Au, W. (2011). Teaching under the new Taylorism: High‐ stakes testing and the

standardization of the 21 century curriculum. Journal of Curriculum Studies. 43:

1, 25 – 45. doi:10.1080/00220272.2010.521261

Barksdale-Ladd, M. A., & Thomas, K. F. (2000). What's at Stake in High-Stakes

Testing: Teachers and Parents Speak Out. Journal of Teacher Education. 51: 384 -

397. doi:10.1177/0022487100051005006

Bell, S. (2010). Project-Based Learning for the 21st Century: Skills for the Future. The

Clearing House. 83(2), 39-43 doi:10.1080/00098650903505415

Belland, B. R. (2009). Using the theory of habitus to move beyond the study of barriers

to technology integration. Computers & Education. 52(2), 353-364.

HIGH-STAKE TESTING

99

doi:10.1016/j.compedu.2008.09.004

Berliner, D. C. (2011). Rational responses to high-stakes testing: The case of curriculum

narrowing and the harm that follows. Cambridge Journal of Education, 41(3),

287-302. Retrieved from http://dx.doi.org/10.1080/0305764X.2011.607151

Blazer, C. (2011). Unintended consequences of high-stakes testing (volume 1008).

Retrieved from http://eric.ed.gov/?id=ED536512

Boser, U. (2013, June 14). Are schools getting a big enough bang for their education

technology buck? Retrieved from http://www.americanprogress.org

Bourdieu, Pierre (1986). The Forms of Capital. In John Richardson, Ed. Handbook of

Theory and Research for the Sociology of Education. New York: Greenwood

Press, pp. 241-258.

Bourdieu, P., & Passeron, J. C. (1990). Reproduction in education, society and culture

(Vol. 4). Sage.

Brewer, E., & Kuhn, J. (2010). Causal-comparative design. In N. Salkind (Ed.),

Encyclopedia of research design. (pp. 125-132). Thousand Oaks, CA: SAGE

Publications, Inc. doi: http://dx.doi.org/10.4135/9781412961288.n42

Britten, J. S., & Cassady, J. C. (2005). The technology integration assessment instrument:

Understanding planned use of technology by classroom teachers. In Classroom

Integration of Type II Uses of Technology in Education (pp. 49-61). England: The

Haworth Press.

Brooks, J. G., & Brooks, M. G. (1999). In search of understanding: The case for

constructivist classrooms. Alexandria, VA: Association for Supervision and

HIGH-STAKE TESTING

100

Curriculum Development.

Bullock, S. M., & Russell, T. (2010). Does teacher education expect too much from field

experience? In field experiences in the context of reform of Canadian teacher

education programs (pp. 91-100). Winnepeg, Canada: Faculty of Education of the

University of Manitoba.

Campbell, D. T. (1976). Assessing the impact of planned social change (Paper #8).

Retrieved from The Public Affairs Center, Dartmouth College website:

https://www.globalhivmeinfo.org

Cawelti, G. (2006, November). The side effects of NCLB. Educational Leadership,

64(3), 64-68.

Chai, C. S., Koh, J. K., & Tsai, C. C. (2013). A Review of technological pedagogical

content knowledge. Educational Technology & Society, 16(2), 31-51.

Clark, C. M., & Yinger, R. J. (1979). Three studies of teacher planning. Retrieved from

The Institute for Research on Teaching website:

http://http://education.msu.edu/irt/PDFs/ResearchSeries/rs055

Clarke, M., Shore, A., Rhodes, K., Abrams, L., Miao, J., & Li, J. (2003). Perceived

effects of state- mandated testing programs on teaching and learning: findings

from interviews with educators in low-, medium- and high-stakes states. Retrieved

from National Board on Educational Testing and Public Policy website:

http://www.bc.edu/research/nbetpp/statements/nbr1.pdf

Crocco, M. S., & Costigan, A. T. (2007). The Narrowing of Curriculum and Pedagogy

in the Age of Accountability Urban Educators Speak Out. Urban Education.

doi:10.1177/0042085907304964

HIGH-STAKE TESTING

101

Dong, Y., Chai, C. S., Sang, G.-Y., Koh, H. L., & Tsai, C.-C. (2015). Exploring the

Profiles and Interplays of Pre-service and In-service Teachers’ Technological

Pedagogical Content Knowledge (TPACK) in China. Educational Technology &

Society, 18 (1), 158–169.

Dufour, R., & Marzano, R. J. (2011). Leaders of learning: How district, school, and

classroom leaders improve student achievement. Bloomington, IN: Solution Tree

Press.

Dumais, S. (2005). Children's cultural capital and teacher's assessments of effort and

ability: The influence of school sector. Catholic Education: A Journal of Inquiry

and Practice, 8(4), and 418-439. Retrieved from http://digitalcommons.lmu.edu

Ertmer, P. A., Ottenbreit-Leftwich, A.T., Sadik, O., Sendurur, E., & Sendurur, P. (2012).

Teacher beliefs and technology integration practices: A critical relationship.

Computers & Education, 59(2), 423-435.

Ertmer, P. A. (1999). Addressing first- and second-order barriers to change: Strategies

for technology integration. Educational Technology Research and Development.

doi:10.1007/BF02299597

Fullan, M. (2006). Change theory: A force for school improvement. Centre for Strategic

Education, 1-14.

The Framework. (2013). Retrieved from http://danielsongroup.org/framework/

Gaddis, S. M. (2013). The influence of habitus in the relationship between cultural

capital and academic success. Social Science Research, 42(1), 1-13. doi:

10.1016/j.ssresearch. 2012.08.002

Garrett, T. (2008). Student-centered and teacher-centered classroom management: A

HIGH-STAKE TESTING

102

case study of three elementary teachers. Journal of Classroom Interaction, 43(1),

34-37. Retrieved from files.eric.ed.gov/fulltext/EJ829018

Gilakjani, A. P., Leong, L. M., & Ismail, H. N. (2013). Teachers’ Use of Technology

and Constructivism. I.J.Modern Education and Computer Science, 4, 49-63, 4, 49-

63. Retrieved from DOI: 10.5815/ijmecs.2013.04.07

Grant, M. M., & Hill, J. R. (2006). Weighing the rewards with the risks? Implementing

student-centered pedagogy within high-stakes testing. In Understanding teacher

stress in the age of accountability (pp. 19-42). Greenwich, CT: Information Age

Publishing.

Gray, L., Thomas, N., Lewis, L., & Tice, P. (2010). Technology in U.S. public schools:

2009 first look (2010-040). Retrieved from National Center for Educational

Statistics website: http://nces.ed.gov

Grenfell, M. and James, D. eds. (1998) Bourdieu and Education: Acts of Practical

Theory, Falmer Press

Gunn, T., & Hollingsworth, M. (2013). The implementation and assessment of a shared

21st century learning vision: A district-based approach. Journal Of Research On

Technology In Education, 45(3), 201-228. Retrieved from www.iste.org

Gunzenhauser, M. G. (2003). High-Stakes Testing and the Default Philosophy of

Education. Theory Into Practice. doi:10.1353/tip.2003.0007

Guzey, S. S., & Roehrig, G. H. (2012). Integrating educational technology into the

secondary science teaching. Contemporary Issues in Technology and Teacher

Education, 12(2), 162-183. AACE. , 12(2), 162-183. Retrieved from

http://www.editlib.org

HIGH-STAKE TESTING

103

Halverson, R., & Smith, A. (2009). How new technologies have (and have not) changed

teaching and learning in schools. Journal of Computing in Teacher Education,

26(2), 49-54.

Hammonds, L., Matherson, L. H., Wilson, E. K., & Wright, V. H. (2013). Gateway

tools: Five tools to allow teachers to overcome barriers to technology integration.

Delta Kappa Gamma Bulletin, 80(1), 36-40.

Harris, J. B., & Hofer, M. J. (2011). Technological pedagogical content knowledge

(TPACK) in action: A descriptive study of secondary teachers’ curriculum-based,

technology-related instructional planning. Journal of Research on Technology in

Education, 43(3), 211-229. Retrieved from

http://files.eric.ed.gov/fulltext/EJ918905.pdf

Harris, J., Grandgenett, N., & Hofer, M. (2010). Testing a TPACK-based technology

integration assessment rubric. In Research highlights in technology and teacher

education (pp. 323-321). Chesapeake, VA: SITE.

He, W., & Hartley, K. (2010). A Supporting Framework of Online Technology

Resources for Lesson Planning. Journal of Educational Multimedia and

Hypermedia, 19(1), 23-37. Retrieved from http://www.editlib.org

Heafner, T. (2004). Using Technology to Motivate Students to Learn Social Studies.

Contemporary Issues in Technology and Teacher Education [Online serial], 4(1).

Retrieved from http://www.citejournal.org/vol4/iss1/socialstudies/article1.cfm

Hew, K. F., & Brush, T. (2007). Integrating technology into K-12 teaching and learning:

current knowledge gaps and recommendations for future research. Educational

Technology Research and Development. doi:10.1007/s11423-006-9022-5

HIGH-STAKE TESTING

104

Intra-rater reliability (2015). In Stroke Engine Glossary. Retrieved from

http://www.strokengine.ca/glossary/intra-rater-reliability

ISTE national educational standards (NETS) and Performance indicators for teachers.

(2000). Retrieved from

http://http://www.iste.org/docs/pdfs/nets_for_teachers_2000

ISTE standards for administrators resources. (2009). Retrieved from

www.iste.org/standards/standards-for-administrators

ISTE standards for teachers resources. (2009). Retrieved from

https://www.iste.org/standards/standards-for-teachers

Jones, L. (2007). The student-centered classroom. Retrieved from

http://www.cambridge.org/other_files/downloads/esl/booklets/Jones-Student-

Centered

Keengwe, J., Onchwari, G., & Onchwari, J. (2009). Technology and student learning:

Toward a learner-centered teaching model. AACE Journal, 17(1), 11-22.

Retrieved from http://www.editlib.org

Kelly, M. (2008). Incorporating context into TPCK-based instructional design. Paper

presented at the Society for Information Technology and Teacher Education

International Conference Chesapeake, VA.

Keystone Exams. (2015, July 14). Retrieved from

http://www.portal.state.pa.us/portal/server.pt/community/keystone_exams/20436

Kereluik, K., Mishra, P., Fahnoe, C., & Terry, L. (2013). What knowledge is of most

worth: Teacher knowledge for 21st century learning. Journal of Digital Learning

in Teacher Education, 29(4), 127-140. Retrieved from www.iste.org

HIGH-STAKE TESTING

105

Koehler, M. J., & Mishra, P. (2008). Bridging digital and cultural divides TPCK for

equity of access to technology. In Handbook of technological pedagogical content

knowledge (TPCK) for educators (pp. 31-58). New York, NY: Routledge.

Koehler, M. J., & Mishra, P. (2008). Introducing TPCK. In Handbook of technological

pedagogical content knowledge (TPCK) for educators (pp. 1-30). New York, NY:

Routledge.

Koehler, M. J., & Mishra, P. (2009). What is technological pedagogical content

knowledge? Contemporary Issues in Technology and Teacher Education, 9(1).

Retrieved from http://www.citejournal.org/vol9/iss1/general/article1.cfm

Layton, L. (2013, November 7). U.S. students show incremental progress on national

test. The Washington Post. Retrieved from www.washingtonpost.com

Lim, C. P., Zhao, Y., Tondeur, J., Chai, C. S., & Tsai, C. C. (2013). Bridging the gap:

Technology trends and use of technology in schools. Educational Technology &

Society, 16(2), 59-68.

Madaus, G., & Russell, M. (2010). Paradoxes of high – stakes testing. Journal of

Education, 190(1/2), 21-30.

Maddux, C., & Cummings, R. (1986). Educational computing at the crossroads: Type I

or Type II uses to predominate? Educational Technology, 26(7), 34-38. Retrieved

from http://www.editlib.org

Maddux, C. (1986). Microcomputers in education and counseling: Problems and

cautions. Techniques, 2(1), 9-14. Retrieved from http://eric.ed.gov/?id=EJ329840

Means, B., & Olson, K. (1997). Technology and education reform: Studies of education

reform. U. S. Government Printing Office.

HIGH-STAKE TESTING

106

Mishra, P., & Koehler, M. J. (2006). Technological pedagogical content knowledge: A

new framework for teacher knowledge. Teachers College Record, 108(6), 1017-

1054. Retrieved from punya.educ.msu.edu

Moses, M. S., & Nanna, M. J. (2007). The testing culture and the persistence of high

stakes testing reforms. Education and Culture. doi:10.1353/eac.2007.0010

Mueller, J., Wood, E., Willoughby, T., Ross, C., & Specht, J. (2008). Identifying

discriminating variables between teachers who fully integrate computers and

teachers with limited integration. Computers & Education, 51(4), 1523-1537.

doi:10.1016/j.compedu.2008.02.003

Napoli, R. (2004). What is student-centered learning? Retrieved from Educational

Initiative Centre, University of Westminster website:

http://https://www.westminster.ac.uk

Nichols, S. L., & Berliner, D. C. (2007). Collateral damage: How high-stakes testing

corrupts America's schools. Cambridge, MA: Harvard Education Press.

Nolan, K. (2011). Dispositions in the field: viewing mathematics teacher education

through the lens of Bourdieu’s social field theory. Educational Studies in

Mathematics May 2012, Volume 80, Issue 1-2, pp. 201-215, 80(1-2), 201-215.

doi:10.1007/s10649-011-9355-9

Novak, J. D. (2011). A theory of education: Meaningful learning underlies the

constructive integration of thinking, feeling, and acting leading to empowerment

for commitment and responsibility. Meaningful Learning Review, 1(2), 1-14.

Retrieved from http://www.if.ufrgs.br/asr/artigos/Artigo_ID7/v1_n2_a2011

O'Hara, D. (2000). Capitalism and Culture: Bourdieu's Field Theory.

HIGH-STAKE TESTING

107

Amerikastudien/American Studies, 43(53). Retrieved from http://www.jstor.org

Pamuk, S. (2012). Understanding pre-service teachers' technology use through TPACK

framework. Journal of Computer Assisted Learning, 28, 425-439. doi:

10.1111/j.1365-2729.2011.00447

Pennsylvania Department of Education (2015). Retrieved from

http://www.pdesas.org/standard/StandardsDownloads

Purcell, K., Heaps, A., Buckanan, J., & Friedrich, L. (2013). How teachers are using

technology at home and in their classrooms. Retrieved from Pew Research

Internet Project website: http://www.pewinternet.org

Ravitch, D. (2013). Reign of error: The hoax of the privatization movement and the

danger to America's public schools.

Rich, M. (2013, June 13). Study gauges value of technology in schools. The New York

Times. Retrieved from www.nytimes.com

Rideout, V. J., Foehr, U. G., & Roberts, D. F. (2010). Generation M2: Media in the lives

of 8-18 year-olds. Retrieved from kff.org

Rose, D. H., Harbour, W. S., Johnston, C. S., Daley, S. G., & Abarbanell, L. (2006).

Universal design for learning in postsecondary education: Reflections on

principles and their application. Journal of postsecondary education and disability,

19(2), 135-151.

Saavedra, A. R., & Opfer, V. D. (2012). Learning 21st-century skills requires 21st-

century teaching. Phi Delta Kappan, 92(2). Retrieved from http://pdkintl.org

Schmidt, D. A., Baran, E., Thompson, A. D., Mishra, P., & Shin, T. S. (2009).

Technological pedagogical content knowledge (tpack): The development and

HIGH-STAKE TESTING

108

validation of an assessment instrument for pre-service teachers. Journal of

Research on Technology in Education, 42(2), 123-149. Retrieved from

http://www.iste.org

Sheehan, M., & Nillas, L. A. (2010). Technology Integration in Secondary Mathematics

Classrooms: Effect on Students’ Understanding.

Sherman, K., & Howard, S. K. (2012). Teachers' beliefs about first-and second-order

barriers to ICT integration: preliminary findings from a South African study. In

Proceedings of society for information technology & teacher education

international conference 2012 (pp. 2098-2105). Retrieved from

http://www.editlib.org/p/39897

Shulman, L. (1986). Those who understand: knowledge growth in teaching. Educational

Researcher, 15(2), 4.

Shulman, L. S. (1987). Knowledge and teaching: foundations of the new reform.

Harvard Educational Review, 57(1), 1-22.

Su, B. (2009). Effective technology integration: Old topic, new thoughts. International

Journal of Education & Development using Information, 5(2), 161-171. Retrieved

from http://ijedict.dec.uwi.edu

Sullivan, A. (2002). Bourdieu and education: how useful is Bourdieu's theory for

researchers? The Netherlands Journal of Social Sciences -, 38(2), 144-166.

Retrieved from http://www.cls.ioe.ac.uk

Supovitz, J. (2010). Is high-stakes testing working? A Review of Research, 7(2), 1-8.

Retrieved from www.gse.upenn.edu/review/feature/supovitz

Taylor, L., & Parsons, J. (2011). Improving student achievement. Current Issues in

HIGH-STAKE TESTING

109

Education, 14(1). Retrieved from http://cie.asu.edu

Thomas, R. Murray (Robert Murray), 1921-. (2005). High stakes testing: Coping with

collateral damage. Mahwah, NJ: L. Erlbaum Associates.

Tondeur, J., Valcke, M., & Braak, J. V. (2008). A multidimensional approach to

determinants of computer use in primary education: teacher and school

characteristics. Journal of Computer Assisted Learning, 24, 494-506.

doi:10.1111/j.1365-2729.2008.00285.x

Warde, A. (2004). Practice and Field: revising Bourdieusian concepts (CRDC

Discussion paper 65). Retrieved from Center for Research on Innovation &

Competition website: http://http://www.cric.ac.uk

Napoli, R. (2004). What is student-centered learning? Retrieved from Educational

Initiative Centre, University of Westminster website:

http://https://www.westminster.ac.uk

Yeh, S. S. (2005). Limiting the unintended consequences of high-stakes testing.

Education Policy Analysis Archives, 13(43). Retrieved from

http://epaa.asu.edu/epaa/v13n43/

Yurdakal, I. K., Odabasi, H. F., Kilicer, K., Coklar, A. N., Birinci, G., & Kurt, A. A.

(2012). The development, validity and reliability of TPACK-deep: A

technological pedagogical content knowledge scale. Computers & Education,

58(3), 964-977. doi: 10.1016/j.compedu.2011.10.012

Zhao, Y., Pugh, K., Sheldon, S., & Byers, J. L. (2002). Conditions for Classroom

Technology Innovations. Teachers College Record, 104(3), 482-515.

doi:10.1111/1467-9620.00170

HIGH-STAKE TESTING

110

Appendix A

Technology Integration Assessment Rubric123 1 Harris, J., Grandgenett, N., & Hofer, M. (2010). Testing a TPACK-based technology integration assessment instrument. In C. D. Maddux, D. Gibson, & B. Dodge (Eds.). Research highlights in technology and teacher education 2010 (pp. 323-331). Chesapeake, VA: Society for Information Technology and Teacher Education (SITE).

Criteria 4 3 2 1

Curriculum Goals Technologies Technologies Technologies Technologies & Technologies selected for use in selected for use in selected for use in selected for use in

the instructional the instructional the instructional the instructional (Curriculum-based technology use)

plan are strongly aligned with one or

plan are aligned with one or more curriculum goals.

plan are partially aligned with one or

plan are not aligned with any curriculum goals.

goals. goals.

Instructional Technology use Technology use Technology use Technology use Strategies & optimally supports Supports minimally supports does not support Technologies instructional Instructional instructional Instructional

strategies. strategies. strategies. strategies. (Using technology in teaching/ learning)

Technology Technology Technology Technology Technology Selection(s) selection(s) are selection(s) are selection(s) are selection(s) are

exemplary, given appropriate, but not marginally inappropriate, given (Compatibility with curriculum goals & instructional strategies)

curriculum goal(s) and instructional strategies.

exemplary, given curriculum goal(s) and instructional strategies.

appropriate, given curriculum goal(s) and instructional strategies.


“Fit” Content, Content, Content, Content, instructional Instructional instructional Instructional (Content, pedagogy and technology together)

strategies and technology fit together strongly within

strategies and technology fit together within the instructional plan.

strategies and technology fit together somewhat within the

strategies and technology do not fit together within

instructional plan. instructional plan. plan.

HIGH-STAKE TESTING

111

2 Adapted from: Britten, J. S., & Cassady, J. C. (2005). The Technology Integration Assessment Instrument: Understanding planned use of technology by classroom teachers. Computers in the Schools, 22(3), 49-61.

3 “Technology Integration Assessment Rubric” by Judi Harris, Neal Grandgenett & Mark Hofer is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 United States License.

HIGH-STAKE TESTING

112

Appendix B

RIGOR AND RELEVANCE LESSON PLAN (2012 – 2013)

SCORE: ________ TEACHER: TIMELINE:

COURSE / GRADE / SUBJECT: ACTIVITY TITLE:

Student Learning: - As a result of this lesson, students will be able to: Performance Task: – How students will demonstrate their skills Resources: Staff Development Good Instruction Prezi Hands on Learning Multi Modal Instruction Strategies to Differentiate – How students’ needs will be met Essential Skills - List PA Anchors/Eligible Content, Common Core that are addressed Resources: What is Differentiated Instruction? (Content, Process, Learning Environment, & Products) Staff Development Good Instruction Prezi

HIGH-STAKE TESTING

113

Video: What is D.I. Video: Examples of Differentiated Instruction Formative Assessments: - Examples of how the teacher checks for understanding during a lesson Resources: Staff Development Good Instruction Prezi Bell Ringers Checking for Understanding Formative Assessment Examples WV DoE Formative Assessment Examples CMU: Formative and Summative Assessment and assessment reasoning Summative Assessments- How students will demonstrate mastery of the targeted skills Resources: Summative Assessment Reasoning & Examples Video: Formative vs. Summative Assessment Scoring Guide: – How performance task will be assessed Resources: Rubic Maker Staff Development Good Instruction Prezi Rubric Definition and Examples Video: Rubistar part 1 Video: Rubistar part 2 Video: Rubistar part 3 Video: Creating Rubric

HIGH-STAKE TESTING

114

LESSON ASSESSMENT GUIDE

SCORE FOR THIS LESSON: ________

Category 4 3 2 1 Level of Rigor How are students asked to think about the content?

Students are required to think in creative ways and to devise solutions to confront perplexing, unknown situations in unique ways.

Students are required to extend and refine their acquired knowledge to routinely solve problems in predictable situation.

Students are asked to acquire knowledge to solve problems, design solutions, and complete work.

Students are asked to gather and store bits of knowledge and information.

Level of Relevance What will students produce to show mastery of the content?

Students are required to develop creative solutions and to devise products that demonstrate their knowledge and skill to confront the complex situations.

Students are required to apply knowledge across disciplines and to solve problems in real world predictable situations.

Students are primarily expected to apply knowledge in one or more disciplines or extend their learning beyond the classroom.

Students are primarily expected to apply knowledge in one discipline.

Level of Student Participation How will students be engaged in the lesson?

Students are required to be actively engaged throughout the lesson by working collaboratively with partners or groups. Students called upon to provide leadership in class.

Students are required to be actively engaged throughout the lesson by working collaboratively with partners or groups.

Students are primarily expected to demonstrate on task behaviors and to work collaboratively with partners to complete assigned work.

Students are primarily expected to demonstrate on task behaviors and to function effectively as independent learners to complete assigned work.

Use of Formative and Summative Assessments Strategies How will

Integration of multiple formative and summative assessments throughout the

Ongoing use of two or three formative assessments as a diagnostic tool to understand

Periodic use of both formative and summative assessment to determine who mastered the

Employs summative assessments at the end of learning to determine who

HIGH-STAKE TESTING

115

teachers determine what students know and are able to do?

lesson used to measure student progress and how to make instruction better. Multiple summative assessment options provided.

who is learning and how to make instruction better. One or two summative assessment options provided.

objectives. Single summative assessment option provided.

mastered the objectives.

Use of Strategies to Differentiate the lesson How will the lesson match the varied learning styles and the individual needs of students?

Instruction includes consistent variations of content, and process and assessment strategies to provide students of different abilities, interests, or learning needs appropriate ways to absorb, use, develop and present skills and knowledge as a part of their daily learning process.

Instruction includes multiple variations of content, and process and assessment strategies to provide students frequent opportunities to use multi-modal learning practices.

Instruction includes some variation of content and process and assessment strategies, but is a more teacher directed lesson.

Whole class instruction dominates and coverage of curriculum guides and texts shape instruction.

Connection to PDE Anchors and Standards

Lesson consistently matches the PDE standards and anchors.

Lesson generally matches the PDE standards and anchors.

Lesson somewhat matches the PDE standards and anchors

Lesson does not match the PDE standards and anchors.

24-20 pts. = D, 19-16 pts. = C, 15-10 pts. = B, 9-0 pts. = A quadrant activities

HIGH-STAKE TESTING

116

Appendix C

NEW RIGOR AND RELEVANCE LESSON PLAN (2015 – 2016)

TEACHER: COURSE/GRADE: LESSON TITLE: Student Learning: - As a result of this lesson, students will be able to: Performance Task: – How students will demonstrate their skills: TPACK in Use: - How students and teachers use technology in learning: Strategies to Differentiate – How students’ learning needs will be met: Essential Skills - List PA Anchors/Eligible Content, Common Core addressed: Formative Assessments: - Specific examples of how the teacher checks for understanding during a lesson: Summative Assessments: - How students demonstrate mastery of the targeted skills:

HIGH-STAKE TESTING

117

Teacher TPACK (Technology, Pedagogy, and Content Knowledge) Narrative Reflection - How the teacher integrated technology for this lesson by using best practices:

HIGH-STAKE TESTING

118

Technology Integration Assessment Rubric123 TPACK SCORE ________

Criteria 4 3 2 1

Curriculum Goals Technologies Technologies Technologies Technologies & Technologies selected for use in selected for use in selected for use in selected for use in

the instructional the instructional the instructional the instructional (Curriculum-based technology use)

plan are strongly aligned with one or

plan are aligned with one or more curriculum goals.

plan are partially aligned with one or

plan are not aligned with any curriculum goals.

goals. goals.

Instructional Technology use Technology use Technology use Technology use Strategies & optimally supports Supports minimally supports does not support Technologies instructional Instructional instructional Instructional

strategies. strategies. strategies. strategies. (Using technology in teaching/ learning)

Technology Technology Technology Technology Technology Selection(s) selection(s) are selection(s) are selection(s) are selection(s) are

exemplary, given appropriate, but not Marginally inappropriate, given (Compatibility with curriculum goals & instructional strategies)


exemplary, given curriculum goal(s) and instructional strategies.

appropriate, given curriculum goal(s) and instructional strategies.


“Fit” Content, Content, Content, Content, instructional Instructional instructional Instructional (Content, pedagogy and technology together)

strategies and technology fit together strongly within

strategies and technology fit together within the instructional plan.

strategies and technology fit together somewhat within the

strategies and technology do not fit together within

instructional plan. instructional plan. plan.

1 Harris, J., Grandgenett, N., & Hofer, M. (2010). Testing a TPACK-based technology integration assessment instrument. In C. D. Maddux, D. Gibson, & B. Dodge (Eds.). Research highlights in technology and teacher education 2010 (pp. 323-331). Chesapeake, VA: Society for Information Technology and Teacher Education (SITE).

2 Adapted from: Britten, J. S., & Cassady, J. C. (2005). The Technology Integration Assessment Instrument: Understanding planned use of technology by classroom teachers. Computers in the Schools, 22(3), 49-61.

HIGH-STAKE TESTING

119

3 “Technology Integration Assessment Rubric” by Judi Harris, Neal Grandgenett & Mark Hofer is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 United States License.

high-stake testing as a barrier to technology …

Documents