measuring preservice teacher self-efficacy of technology integration

Measuring preservice teacher self-efficacy of technology

integrationJeremy Browne

Department of Instructional Psychology & TechnologyBrigham Young University

United Statesbrowne@byu.edu

IP&T 286 / 287

• Technology Integration

• Not a computer course

• Required for all preservice teachers– 286: Secondary education– 287: Elementary, Early Childhood, Special

Education

• Aligned with ISTE’s NETS-T

Fostering Technology Integration

Skills & Knowledge

National EducationalTechnology Standards

Dispositions

Confidence

Perceived Value

EffectiveIn-PracticeTechnologyIntegration

Can / Can’t

Will / Won’t

Why Self-Efficacy?

1. More clearly defined than “Confidence”

2. Well established measurement methodology

3. Significant predictor of many in-practice behaviors

1. Self-Efficacy Defined

• Self-efficacy is a personal belief about one’s own ability to perform a given action. (Bandura, 1997; Denzine et al., 2005)

• Not to be confused with “Teacher Efficacy” (Tschannen-Moran et al., 1998)

2. Self-Efficacy Measures

• Bandura (2006):

3. Predictive Power

• Job-search “self-efficacy was a significant predictor of interviews, offers, employment status, and PJ [Person-Job] fit perceptions” (Saks, 2006).

• Perceived math self-efficacy predicted interest in the subject (Özyürek, 2005).

• “Data analysis indicated that perceived self-efficacy was a significant predictor of [new in-practice teacher] performance” (Jablonski, 1995).

3. Predictive Power

• “Among the six subscales of empowerment, professional growth, status and self-efficacy were significant predictors of organizational and PC [professional commitment]” (Bogler & Somech, 2004).

• The perceived self-efficacy and context beliefs of teachers regarding the use of computer technology correlated significantly with reported hours of in-class use of technology (Whitehead, 2002).

Self-efficacy Mediated

• It does mediate distressing events.

• Chwalisz et al., 1992– High self-efficacy = Problem-focused coping – Low self-efficacy = Emotion-focused coping – “EFC, not PFC, was associated with higher

levels of burnout [of in-practice teachers].”

Literature Review

• Don’t reinvent the wheel.– (Find an existing measure.)

• Don’t reuse a flat tire.

• MUTEBI (Enoch et al., 1993)

• Findings: We needed to create our own measure.– The Technology Integration Confidence Scale

(TICS).

TICS Item Development

1. Begin with NETS-T

2. Write 4-7 tasks for each

3. Review by faculty & students• Pen & paper comments

4. Return to step 2

Important Deviations

TICS v1

• 28-item TICS

• Web-based

• 52 Spring-term preservice teachers

• Administered at end of term

• Described in proceedings

TICS v2

• 33 Items• Expanded coverage of specific NETS-T• Targeted item revision (e.g. Item 13)

• Larger sample (200+)Pre- and post-course administration

• “New General Self-efficacy Scale” (NGSE; Chen et al., 2001) administered concurrently

Results: Item Analysis (pretest)

• Improvement from TICS v1

• Lower means (10 items > 4.0)

• Higher variances (0 items < .5)

• Well represented NETS-T

Results: Reliability Analysis (Pretest)

Projected number of items required for

NETS-T N # of items Alpha α = .80 α = .90

I.A 233 6 .84 5 11

I.B 238 2 .80 2 5

II 235 7 .90 4 7

III 231 5 .88 3 7

IV 234 4 .82 4 9

V 234 5 .83 5 10

VI 234 4 .86 3 6

Results: Factor Analysis (pretest)

% of variance explained by

NETS-T # of items Factor 1 Factor 2

I.A 6 57.7 --

I.B 2 84.1 --

II 7 63.8 --

III 5 68.7 --

IV 4 65.6 --

V 5 61.0 --

VI 4 70.7 --

RSM (Functional)

Strongly

disagree

Strongly

agree

Disagree Agree

Neutral

RSM (Functional)

Strongly

disagree

Strongly

agree

Disagree Agree

Neutral

RSM (Functioning)

Strongly

disagree

Strongly

agree

Disagree Agree

Neutral

RSM (Malfunctioning)

Strongly

disagree

Strongly

agree

Disagree

Agree

Neutral

NGSE

NETS-T I.A (pre & post)

NETS-T I.B (pre & post)

NETS-T II (pre & post)

NETS-T III (pre & post)

NETS-T IV (pre & post)

NETS-T V (pre & post)

NETS-T VI (pre & post)

Evidence of Validity

TICS v1: Construct ValidityResults of Item-Domain Congruence Survey.

Number and percent of judges who classified each item on the intended subscale

NETS-T Item number Number Percent

II 11 2 40

II 15 2 40

II 25 3 60

II 26 0 0

II 28 1 20

III 9 1 20

III 10 2 40

V 13 2 40

V 16 1 20

TICS v1: Content ValidityItem Relevancy Sores (Aiken’s V index).

Number of judges that classified this item as…

SubscaleItem

number RelevantSomewhat

relevantSomewhat irrelevant Irrelevant

Aiken’s V index

II 28 3 0 2 0 .730

III 10 2 3 0 0 .800

IV 27 2 1 2 0 .670

VI 20 2 2 1 0 .730

Anachronistic View of Validity

• “The Holy Trinity” (Guion, 1980)– Content Validity– Construct Validity– Criterion Validity

• Convergent Validity• Discriminate Validity

• Others– Consequential Validities– Face Validity– Etc.

Modern View of Validity

• There is no validity but construct validity.

– Messick 1995; AERA, APA, NCME, 1999

• “Validities” reassigned as “sources of validity-supporting evidence.”

Validity…

• …is a property of your interpretation of the test data (not of the test or the data).– …is an evaluative judgment of the

“soundness of your interpretations and uses of students’ assessment results”(Nitko & Brookhart, 2006)

• … changes based on purpose.

Applying Modern Validity Theoryto the TICS

• Intended Purposes1. Establish a baseline preservice teacher

profile

2. Monitor the effects of curricular adjustments

3. Identify preservice teachers in most need of intervention

4. Predict in-practice technology integration

1. Establish a baseline preservice teacher profile

Assumes the TICS functions well psychometrically.

Internal structure analysis

• Expert reviewsLow of correlation with NGSE

( < .28 or 8% variance explained)

2. Monitor the effects of curricular adjustments

Assumes the TICS is sensitive to changes in self-efficacy.

• Pre-Post analysis

• Comparisons of scores between IP&T 286 and 287

3. Identify preservice teachers in most need of intervention

Assumes TICS can predict in-classperformance.

RSM information analysis• Regression analysis

– XPre-course TICS scores Relevant demographics

– YIn-class performance indicators

(assignment / assessment scores)

4. Predict in-practice technology integration

• 5-year longitudinal, mixed methods study

4. Predict in-practice technology integration

• Review of self-efficacy literature

Future Directions

• TICS v2 showing promise

• Expanded use

• Inform NETS-T “refreshing”

• Modern validity theory can be applied systematically.