document resume author witta, e. lea; mathews, bruce … · document resume. ed 419 541 ir 057 043....
TRANSCRIPT
DOCUMENT RESUME
ED 419 541 IR 057 043
AUTHOR Witta, E. Lea; Mathews, BruceTITLE Comparison of 1995-1996 High School Students Attitude Survey
Results.PUB DATE 1997-00-00NOTE 47p.; Paper presented at the Annual Conference of the
American Evaluation Association (San Diego, CA, November4-8, 1997).
PUB TYPE Reports Research (143) Speeches/Meeting Papers (150)EDRS PRICE MF01/PCO2 Plus Postage.DESCRIPTORS Change; *Computer Assisted Instruction; *High School
Students; High Schools; Instructional Effectiveness;Instructional Improvement; Instructional MaterialEvaluation; *Interactive Video; *Student Attitudes; StudentReaction; Surveys
ABSTRACTPrior evaluation has been concerned with the attitude of
high school students toward the educational interactive video program for theyear in which they were participants. This analysis assessed whether therehad been changes in attitude toward the program from one year (1995) to thenext (1996) and if improvements made based on the 1995 evaluation had beeneffective. Three hundred forty-four (106 from 1995) high school studentsenrolled in an interactive video class participated in this evaluation.Change was assessed on seven constructs and by open-ended questionsconcerning strengths, weaknesses, and suggested improvements. This analysisrevealed that there has been a significant improvement in equipment and itsmaintenance. Strengths of the interactive video program have remainedrelatively constant over this two-year period. There were no significantdifferences in students' evaluation of the interactive video program over thetwo years, but there was concern over scheduling and behavior weaknesses.Several charts illustrate results. (AEF)
********************************************************************************* Reproductions supplied by EDRS are the best that can be made *
* from the original document. *
********************************************************************************
Comparison of 1995 - 1996 High School Students
Attitude Survey Results
E. Lea Witta
The University of Southern Mississippi
Bruce Mathews
Southwest Virginia Education & Training Network
Paper presented at the annual conference of the American Evaluation Association. San Diego,November 1997.
U.S. DEPARTMENT OF EDUCATIONOffice of Educational Research and Improvement
EDUCATIONAL RESOURCES INFORMATIONCENTER (ERIC)
This document has been reproduced asreceived from the person or organizationoriginating it.Minor changes have been made toimprove reproduction quality.
Points of view or opinions stated in thisdocument do not necessarily representofficial OERI position or policy.
2
"PERMISSION TO REPRODUCE THISMATERIAL HAS BEEN GRANTED BY
E. Lea Witta
TO THE EDUCATIONAL RESOURCESINFORMATION CENTER (ERIC)."
Comparison 95-962
Abstract
Prior evaluation has been concerned with the attitude of high school students toward the
educational interactive video program for the year in which they were participants. This analysis
assessed whether there had been changes in attitude toward the program from one year (1995) to
the next (1996) and if improvements made based on the 1995 evaluation had been effective.
Three hundred forty four (106 from 1995) high school students participated in this evaluation.
Change was assessed on seven constructs and by open-ended questions concerning strengths,
weaknesses, and suggested improvements. This initiates a trend analysis to be continued yearly.
3
Comparison 95-963
Prior evaluation with the data used in this study has been concerned with the attitude of
high school students toward the interactive video program (ITV) for the year in which they were
participants. The purpose of this analysis is to determine if there have been changes in attitude
toward the program from one year (1995) to the next (1996). This is the beginning of a trend
analysis to be continued each year. By comparing attitude changes from year to year, the impact
of improvements and changes to the program/facilities can be detected. If a supposed
improvement has had a substantial effect, it is hoped that effect will be detected. If the effect
reflects improvement in attitude, then it will be assumed the improvement caused this change.
This would suggest continuation of the improvement. If the effect seems to be detrimental, this
would suggest the problem should be approached by another method. In addition, this approach
permits facility administrators to anticipate or detect problem areas while they are still in a
manageable condition.
Subjects
All students enrolled in an interactive video class at this facility during the Spring semester,
1995 and Spring semester, 1996 were surveyed. Surveys were administered during the regularly
scheduled class time by the class instructor or remote facilitator. For analysis, respondents were
categorized by site (home or remote), by year of program existence (year 1 or year 2), and by
survey year (1995 or 1996). Of the 184 returned student surveys during Spring 1995, 106 were
completed by high school students. Twenty-one of the 66 home site respondents were
participating in a first year program. Fifteen of the 40 remote site respondents were participating
in a first year program (see Figure 1). Surveys were returned from respondents at nine different
schools in nine subject areas (see Figure 2).
4
Figu
re 1
Rem
Y2
(25
)
Rem
Y1
(15
Rem
Y1
(47
)
5
Res
pond
ents
By
Site
and
Yea
r19
95 N
=10
6
Hom
eY1
(21
)
Rem
Y2
1996
N=
238
Hom
eY1
(78
)
Hom
eY2
(45
)
Hom
eY2
(83
)
6
Figu
re 2 >,60
(,c7
;50
g40
11-
30a1 ?
20T
2.1
0c LL
I0
7
Cla
sses
Rep
rese
nted
by Y
ear
(199
5/19
96)
Ana
tom
y
Fin
Mgm
tC
reat
Wr
Spa
nish
Phy
sics
Eco
nom
ics
Eng
lish
SA
T
Yea
r Cal
culu
s
Cur
rent
Af
Fre
nch
Stu
dyS
K
Com
pute
r
Psy
chol
Latin
8
Comparison 95-966
There were 238 returned surveys by high school students during Spring 1996. Seventy-
eight of the 161 home site respondents were in a first year program. Forty-seven of the 77
remote site respondents were in a first year program. Surveys were returned for 12 subject areas
(all levels of Spanish were coded Spanish) from 14 high schools.
All student surveys (344) were entered for analysis. Three hundred nine responses were
coded as non-applicable. This was less than 5% (7392 responses). It was assumed that those
who marked non-applicable could not be ranked as undecided since that option was offered and
was not chosen. Since any numeric value assigned would bias the results (1=strongly agree, ergo
0 would be very strongly agree) and the proportion was relatively small, these were used as
missing values. In addition, 91 responses were not marked. These were also used as missing
values yielding a grand total of 400 missing values (<6%). Although the proportion of missing
values is relatively small, if listwise deletion were used many cases would be excluded from
analysis. To prevent this, mean substitution was used for factor analysis.
Measurement and Analysis
The survey instrument consisted of demographic information (school, gender, grade, etc.)
and 34 questions. Question 1 requested reason for taking the ITV class. Questions 2-31 were 5-
point Likert style questions. Questions 32-34 were open-ended requesting strengths, weaknesses,
and suggestions for improvement for the ITV program. Questions 21-26 were to be answered by
remote site participants only. Six questions were negatively stated in the questionnaire. These
were reverse coded for this analysis. Previous exploratory analysis has provided a seven-factor
model for the questionnaire as well as reliabilities of the factors for the Likert type questions in
common for both remote and home site students. While these models (1995, 1996) were
9
Comparison 95-967
similar, there were some discrepancies between them This analysis uses one model to conduct
multivariate analyses.
Responses to open-ended questions (Q32-34) were initially coded based on the response.
After determining similarities of the responses, these were placed in categories. For example, the
suggested improvements response "have schools on the same schedule" was coded as '38'. It and
the responses "synchronize time schedule" and "establish snow schedule" were then summed to a
major category "scheduling'. If a specific response was infrequent (<10) and could not
reasonably be included within a category, it was included in other. For this analysis, if either
survey year (1995, 1996) had sufficient responses to create a category, that category was included
for contrast even though one year may have zero responses. (Raw responses and their codes are
listed in Appendix A, Tables 1-3). Categories were then summed to major categories and the
proportion of categorical responses was contrasted by survey year (1995-1996) for all categories.
The assumption was made that there would be similar proportions of responses for each category
for each year. It must be remembered that these are not independent responses. A respondent
could submit three responses for strengths. All three responses may be variations of items coded
as Availability.
10
Comparison 95-968
Results
Strengths
There were 142 responses concerning strengths (106 respondents) in 1995 and 352 (238
respondents) in 1996. The proportion of responses citing classes offered at remote sites as a
strength had decreased (8% vs 1%) from 1995 to 1996. When, however, categories were
summed (ie, remote and availability became access - meeting people and open-minded became
interaction - technology and instruction became learning), no practical differences were detected
(see Figure 3). For example, in 1995, 39% of the respondents cited interaction as a strength. In
1996, 38% of the respondents cited interaction as a strength. This finding indicates that those
areas perceived as strengths in 1995 are still perceived as strengths in 1996.
These categories are further depicted by site (Figure 4) and by existence year (Figure 5).
Typically remote site students perceive availability or remote as more important (assuming
number of responses reflects importance) than home site students, while home site students
perceive meeting people or interaction aspects as more important than remote site respondents
(see Figure 4). When contrasted by year of program existence, students in the first year of the
program in 1995 did not differ from their counterparts in 1996 nor did those in the second year
(see Figure 5).
Figu
re 3
Acc
ess
(28.
17%
)
Str
engt
hs:
1995
/199
6H
igh
Sch
ool S
tude
nts
Oth
er (
2.11
%)
Acc
ess
(22.
16%
)
Oth
er (
5.97
%)
12
1995
N=
142
Beh
avio
r (5
.63%
)
Lear
ning
(24
.65
%)-J
Beh
avio
r (5
.11%
)
1996
N=
352
Lear
ning
(28
.69%
)
Inte
ract
ion
(39.
44%
)
Inte
ract
ion
(38.
07%
)
13
Figu
re 4
Beh
avio
r (7
.45%
)
Ava
il (1
1.70
%)
emot
e (7
.45%
)
Str
engt
hs:
1995
/199
6S
ite b
y S
urve
y Y
ear
Hom
e'95
Oth
er (
2.13
%)-
Inst
ruct
(17
.02%
)
Beh
avio
r (4
.56%
)
Ava
il (1
7.43
%)
emot
e (0
.00%
)
Oth
er (
6.22
%)
(Ava
il (3
5.42
%)
Mee
tPeo
p (4
2.55
Ope
nmin
(2.
13%
)T
echn
o (9
.57%
)
Hom
e'96
Inst
ruct
(14
.52%
)
1 4
Tec
hno
(13.
69%
)
Rem
ote'
95
Beh
avio
r
(2.0
8%)1
Rem
ote
(10.
42%
)O
ther
(2.
08 %
)
Beh
avio
r (6
.31%
)
Mee
tPeo
p (3
9.83
°Aai
l(3
0.63
%)
Ope
nmin
(3.
73%
)
Rem
ote'
96
Rem
ote
(1.8
0%)
Oth
er (
3.60
%)-
--d
Inst
ruct
(12
.61%
)
Mee
tPeo
p (2
9.17
%)
Ope
nmin
(0.
00
Tec
hno
(14.
58%
)
Inst
ruct
(6.
25%
)
Mee
tPeo
p (2
6.13
%)
--O
penm
in (
0.00
Tec
hno
(18.
92%
)
15
Figu
re 5
Str
engt
hs: 1
995/
1996
Pro
gram
Yea
r by
Sur
vey
Yea
rY
earl'
95
Beh
avio
r (1
.96%
)A
vail
(13
73%
)
mot
e (1
7.65
%)-
-
Oth
er (
0.00
%)-
Inst
ruct
(11
.76%
)
Beh
avio
r (5
.82%
)
Ava
il (2
4.34
%)-
Yea
r2'9
5
Beh
avio
r (7
.69%
) ID_
Mee
tPeo
p (3
7.25
%)k
vail
(23.
08%
)-
"-O
penm
in (
0.00
%)
Tec
hno
(17.
65%
)
1101
Rem
ote
(3.3
0%)-
---
Oth
er (
3.30
%)-
'
Mee
tPeo
p (3
8.4
-\41
Ope
nmin
(2.
20%
)In
stru
ct (
14.2
9%)
Tec
hno
(7.6
9%)
Yea
r1'9
6Y
ear2
'96
Rem
ote
(0.0
0%)-
-11
1111
1
Oth
er (
5.29
%)-
-
Inst
ruct
(12
.70%
)-
Beh
avio
r (4
.29%
)
Ava
il (1
8.40
%)
Mee
tPeo
p (3
3.33
%)
Rem
ote
(1.2
3%)
Oth
er (
5.52
%)-
' Ope
nmin
(3.
17%
)
Tec
hno
(15.
34%
)
Inst
ruct
(15
.34%
)-
Mee
tPeo
p (3
8.0
Ope
nmin
(1.
84%
)T
echn
o (1
5.34
%)
1617
Comparison 95-9612
Weaknesses
There were 128 responses to the weakness question in 1995 and 323 in 1996. The
proportion of responses citing audio problems had decreased from 35% in 1995 to 16% in 1996.
The proportion citing video problems had also been reduced from 19% to 2%. Minor equipment
problems, on the other hand, were not a category in 1995, but were 16% of the responses in 1996.
The scheduling category had also increased from 2% in 1995 to 12% in 1996.
When categories were summed (camera, sound, severe equipment problems, and minor
equipment problems formed equipment - remote and instruction formed instruction) differences
between 1995 and 1996 were still detected. In 1995, 61% of the weakness responses concerned
equipment failure or problems. This was reduced to 43% in 1996. The scheduling response
remained an increase in proportion from 1995 to 1996 (see Figure 6). These responses are
further depicted by site (see Figure 7) and by year of program existence (see Figure 8). Home
site responses (Figure 7) and responses by students in an established program (Figure 8) cite
behavior (cheating, lack of discipline) as weaknesses of the program more frequently than remote
site or those participating in a new program.
These results indicate that although there may still be equipment problems that need to be
resolved, there has been a significant improvement. More troubling is the increased incidence of
scheduling problems and the non-significant increase in behavior cited as a weakness. While a
proportional decrease in responses related to equipment requires an increase in another category
(proportions must sum to 1), 62 weakness responses were related to behavior (cheating, lack of
discipline, see Table A-2) in 1996 compared to 15 in 1995. In addition, 39 weakness responses
referred to scheduling problems in 1996 but only 2 in 1995.
18
Figu
re 6
Wea
knes
ses:
199
5/19
96
Inst
ruct
ion
(25.
00%
)
Oth
er (
0.78
%)
1995
N=
128
Sch
edul
ing
(1.5
6%)
r-B
ehav
ior
(11.
72%
)
Sch
edul
ing
(12.
07%
)
Inst
ruct
ion
(22.
60%
)
Oth
er (
3.41
%)
19
1996
N=
323
\--E
quip
men
t (60
.94%
)
Beh
avio
r (1
9.20
%)
- -
Equ
ipm
ent (
42.7
2%)
20
Figu
re 7
Sch
edul
ing
(2.7
4%)
Rem
ote
(2.7
4%)
Inst
ruct
(15
.07%
)-\
Oth
er (
1.37
%)-
-\ui
pMin
(0.
00%
)-
ipm
ent (
8.22
%)-
Cam
era
(23.
29%
)
Sch
edul
ing
(11.
48%
) -
Rem
ote
(7.1
8%)
Inst
ruct
(6.
70%
) -
Oth
er (
2.39
%)-
Equ
ip M
in (
20.5
7%)
Wea
knes
s: 1
995/
1996
Site
by
Sur
vey
Yea
rH
ome'
95
Beh
avio
r (9
.59%
)
Hom
e'96
Rem
ote
(10.
91%
)
Inst
ruct
(23.
64%
)--7
Sou
nd (
36.9
9%)
Beh
avio
r (2
6.32
%)
Sou
nd (
14.8
3%)
Cam
era
(2.3
9%)
Equ
ipm
ent (
8.13
%)
Rem
ote'
95 rSch
edul
ing
(0.0
0%)
Beh
avio
r (1
4.55
%)
Oth
er (
0.00
%)
Equ
ip M
in (
0.00
%)
Equ
ipm
ent (
5.45
%)
Cam
era
(12.
73%
)
Sch
edul
ing
(12.
39%
)
Rem
ote
(19.
47%
)
Inst
ruct
(19
.47%
)-
Rem
ote'
96
Sou
nd (
32.7
3%)
Beh
avio
r (6
.19%
)
Sou
nd (
18.5
8%)
Cam
era
(2.6
5
Equ
ipm
ent (
7.08
Equ
ip M
in (
8.85
%)
Oth
er (
5.31
%)
2122
Figu
re 8
Wea
knes
s: 1
995/
1996
Pro
gram
Yea
r by
Sur
vey
Yea
rY
earl'
95S
ched
ulin
g (1
.96%
)R
emot
e (0
.00%
)In
stru
ct (
5.88
%)
Oth
er (
0.00
%)
Equ
ip M
in (
0 00
%)
Equ
ipm
ent (
7.84
%)-
amer
a (2
7.45
%)
- -
Yea
rl '9
6
Beh
avio
r (1
1.76
%)
Sou
nd (
45.1
0%)
Sch
edul
ing
(9.2
7%)
Beh
avio
r (1
1.26
%)
Rem
ote
(15.
89%
)
Inst
ruct
(14
.57%
)
23
Oth
er (
5.30
%)
Yea
r2'
95
Sch
edul
ing
(1.3
0%)
Rem
ote
(10.
39%
)
Inst
ruct
(27
.27%
)-
Oth
er (
1.30
%)
Equ
ipM
in (
0.00
%)
d
Equ
ipm
ent (
6.49
%)-
'
Sch
edul
ing
(14.
04%
)
Sou
nd (
15.2
3%)
Rem
ote
(7.6
0%)-
Inst
ruct
(8.
19%
)C
amer
a (3
.31%
)
Equ
ipm
ent (
5.30
%)
Oth
er (
1.75
%)-
I--
Equ
ipM
in (
19.8
7%)
Equ
ipM
in (
13.4
5%)-
Yea
r2'
96
Beh
avio
r (1
1.69
%)
-Sou
nd (
28.5
7%
Cam
era
(12.
99%
)
Beh
avio
r (2
6.32
%)
Sou
nd (
16.9
6%)
Cam
era
(1.7
5%)
Equ
ipm
ent (
9.94
%) 24
Comparison 95-9616
Suggested Improvements
There were 93 suggested improvements made by high school students in 1995 and 230
made in 1996. The larger proportion of suggested improvements in 1995 were concerned with
audio (20% vs 11%), camera (9% vs 1%), and remote (21% vs 12%) than in 1996. Suggested
improvements in scheduling (1% vs 12%) and general equipment (2% vs 14%), on the other
hand, had increased in 1996. When audio, camera, and equipment categories were summed to
equipment (see Figure 9), although a larger percentage of the responses occurred in 1995 (31%),
it was very similar to 1996 (26%). The nature of the suggested equipment responses, however,
had changed. In 1995, the most frequent equipment suggested improvements were 'fix audio'
and 'fix camera'. In 1996, suggested improvements centered around new equipment (ie,
chalkboards, individual microphones).
When summed, suggested improvements in remote and scheduling remained the same.
No other categories had changed significantly. The suggested improvements remote category
included responses in 1995 indicating more remote classes should be offered. Obviously, more
have been made available since that suggested improvement has decreased.
These responses are further depicted by site (Figure 10) and program existence year
(Figure 11). These responses and the weakness responses suggest that while all equipment
problems have not been resolved, there has significant improvement.
25
Figu
re 9
26
Sug
gest
ed Im
prov
emen
ts:
1995
-199
6
IA/V
ar (
29.0
9%)
Env
ironm
ent (
8.18
%)
Oth
er (
1.82
%)
1995
N=
93
Rem
ote
(20.
91%
)I
Env
ironn
ient
(6.
09%
)O
ther
(9.
13%
)
IAN
ar (
19.5
7%)
Rem
ote
(12.
17%
)
1996
N=
230
Equ
ipm
ent (
30.9
1%)
Sch
edul
ing
(0.9
1%)
earn
ing
(8.1
8%)
Equ
ipm
ent (
26.5
2%)
Sch
edul
ing
(12.
17%
)
Lear
ning
(14
.35%
)
27
Figu
re 1
0S
ugge
sted
Impr
ovem
ents
:19
95-1
996:
Site
by
Sur
vey
Yea
r
Env
ironm
ent (
5.08
%)
Oth
er (
1.69
%)
Var
iety
(10
.17%
)
ctio
n (1
5.25
%)
Hom
e'95
Rem
ote
(13.
56%
)In
stru
ctio
n (1
.69%
)D
isci
plin
e (0
.00%
)
Env
ironm
ent (
8.45
%)
Oth
er (
11.2
7%)
arie
ty (
10.5
6%)-
Inte
ract
ion
(9.8
6%)
28
Env
ironm
ent (
14.7
1%)
Oth
er (
2.94
%)
Var
iety
(2.
94%
)In
tera
ctio
n (0
.00%
)
Cam
era
(15.
25%
)
Equ
ipm
ent (
3.39
%)
Sch
edul
ing
(1.6
9%)
Cla
ss (
6.78
%)
Hom
e'96
Aud
io (
14.0
8%)
Cam
era
(1.4
1%)
Rem
ote
(4.2
3%)
Inst
ruct
ion
(7.7
5%)
1---
Equ
ipm
ent (
19.0
1%)
Rem
ot'9
5
Aud
io (
20.5
9%)
Cam
era
(2.9
4%E
quip
men
t (0.
0S
ched
ulin
g (0
.0C
lass
(2.
94%
)
Rem
ote
(44.
12%
)
Rem
ot'9
6
Env
ironm
ent (
2.27
%)
Oth
er (
5.68
%)
Var
iety
(7.
95%
)
Inte
ract
ion
(10.
23%
)
Sch
edul
ing
(5.6
3%)
Cla
ss (
2.82
%)
Dis
cipl
ine
(4.9
3%)
Rem
ote
(25.
00%
)-
b.,
-Dis
cipl
ine
(2.9
4In
stru
ctio
n (5
.88
Aud
io (
5.68
%)
/-C
amer
a (0
.00%
)E
quip
men
t (7.
95%
)
-Sch
edul
ing
(22.
Cla
ss (
3.41
%)
Dis
cipl
ine
(4.5
5%)
Inst
ruct
ion
(4.5
5%) 29
Figu
re 1
1S
ugge
sted
Impr
ovem
ents
:19
95-1
996:
Yea
r by
Pro
gram
Yea
r
Env
ironm
ent (
8.82
%)
Oth
er (
0.00
%)-
\V
arie
ty (
8.82
%)
erac
tion
(2.9
4%)
Yea
rl'95
emot
e (1
4.71
%)-
Inst
ruct
ion
(0.0
0%)
Dis
cipl
ine
(2.9
4%)
Cla
ss (
2.94
%)
Sch
edul
ing
(0.0
0%)
Equ
ipm
ent (
5.88
%)
Env
ironm
ent (
4.17
%)
Oth
er (
12.5
0%)
arie
ty (
14.5
8%)
Inte
ract
ion
(13.
54 %
) -
33
Yea
rl'96
Aud
io (
35.2
9%)
Cam
era
(17.
65%
)
Env
ironm
ent (
8.47
%)
Oth
er (
3.39
%)
Var
iety
(6.
78%
)
Inte
ract
ion
(13.
56%
)
Rem
ote
(30.
51%
)
- -A
udio
(5.
21%
)C
amer
a (0
.00%
)E
nviro
nmen
t (7.
52%
)E
quip
men
t (11
.46%
)O
ther
(6.
02%
)
Sch
edul
ing
(2.0
8%)
Var
iety
(6.
02 %
)-.
Cla
ss (
1.04
%)
Dis
cipl
ine
(3.1
3%)
Inte
ract
ion
(7.5
2%)
Inst
ruct
ion
(11.
401
0.t
m(6
.02%
)
Inst
ruct
ion
(3.0
1%)-
--'
Dis
cipl
ine
(6.0
2%)
Cla
ss (
4.51
%)
Rem
ote
(20.
83%
)
Yea
r2'9
5
Yea
r2'9
6
Aud
io (
16.9
5%)
Cam
era
(6.7
8%)
z-E
quip
men
t (0.
0S
ched
ulin
g (1
.6
Cla
ss (
6.78
%)
Dis
cipl
ine
(0.0
0%'-I
nstr
uctio
n (5
.08%
)
Aud
io (
15.0
4%)
-Equ
ipm
ent (
17.
Sch
edul
ing
(19.
55%
)
31
Comparison 95-9620
Factor Comparison
Factors did not load on the same questions consistently from 1995 to 1996. The Audio
and Environment factors loaded on the same questions, and Materials Support and ITV
evaluation factors loaded on similar questions (one question was added to each in 1996). There
were several discrepancies, however, in the Student Behavior, Class Evaluation, and Interaction
factors. When the 1996 data was forced to load by the 1995 model, reliabilities did not differ
appreciably between survey years. When the 1995 data was forced to load by the 1996 model,
reliabilities again did not differ appreciably between survey years. Some questions, however, did
not fit either model substantively. For this contrast questions were placed on the factor which
they appeared to fit logically. Reliability for both groups in this model was then determined (see
Table 1). Although the reliability of the student behavior factor could have been increased to
0.62 by combining it with the interaction factor, this was not done. Exploratory analyses from
both survey years have yielded a seven factor model. To combine two factors would alter that
model significantly. The questions would also suggest that a separate factor could be established
distinguishing teacher from class. This also was not done. The new model was an adaptation of
the two previous models with as little change as possible while still providing a logical fit. Since
this model fit reliability analyses almost as well as either of the models developed from the
survey year data and it provided a logical explanation of the factors, it was used to contrast the
survey years.
Initially the mean for each respondent on each factor was determined and imputed into
the data. Multivariate analysis of these factors was then conducted using survey year (1995-
1996),
32
Comparison 95-9621
site (remote, home), and year of program existence (yearl, year2) as independent contrasts. Due
to the relatively small sample size in 1995 (one group would only have 15 members if site and
year were contrasted together), two analyses were run: one using site with survey year and one
using year of program existence with survey year. The purpose of these analyses were primarily
to determine if there had been changes in the interactive video program as students perceived it
from 1995 to 1996. There were no significant main effects of survey year. Since this program
had been rated so highly in 1995, any significant difference by survey year would most likely
indicate a change for the worse (see Figure 12). There was, however, a statistically significant
interaction effect of survey year with year of program existence (F=3.79', df=6,276, p<.001). The
effect size was very small (0.08). Audio was the only significant contributor (F=5.75, df 1, 281,
p<.02). This interaction is depicted in Figure 13 . The range of values for audio varied from a
2.11 (agree) to a 2.52 (agree). Although statistical significance was detected, there appears to be
little practical importance.
In addition, there were main effects of both year of program existence and site. These are
are not reported in this study.
'All statistics are reported using the square root of the raw scores. Homogeneity ofvariance (BoxM) was not achieved using raw scores but was established with the square rootfunction. Means are reported as raw scores.
33
Comparison 95-9622
Table 4
Factor Model for Contrasting Survey Years 1995. 1996
Factor LoadingQuestionNumber Label
Materials Support .73 Q4 Returned WorkQ28 Talk to Teach as neededQ29 Class materials timelyQ30 See Materials on System
ITV Evaluation. .79 Q14 ITV Good Addition CurricQ15 R-Hestitate Take Anothr ITVQ16 Choice ITV ClassQ17 ITV Good Way Offer ClassQ18 Take Another ITVQ19 R-ITV More Difficult
Audio .78 Q27 Hear Students other sitesQ31 Hear Quest other Sites
Environment .55 Q2 Amt Desk SpaceQ3 Clear sight TV
Student Behavior .53 Q8 Behav better ITVQ11 Better Listener
Class Evaluation .67 Q5 R- Limit ITV GradeQ7 R Most Talk by HomesiteQ10 R More Cheating ITVQ12 Study same ITVQ13 Tchr Attn Same Home/RemotQ20 R-More Study/Prep ITV
Interaction .47 Q6 Know Stud Other SchlQ9 Meet Other Schl Stu mre ofte
Figu
re 1
2
5 -
4.5
-4
-
c3.
5 -
co.)
CD
-
2.5
-
2-
1.5
-
1-
35
Fac
tor
Mea
ns19
95/1
996
Mat
Sup
tC
las
Eva
l
Yea
r
ITV
Eva
lS
tu B
ehav
Env
iron
Inte
ract
ion
Aud
io
36
Figu
re 1
3In
tera
ctio
n E
ffect
: Aud
ioS
urve
y Y
ear
by P
rogr
am E
xist
ence
Surv
ey Y
ear
- Y
ear
1Y
ear
2
3738
Comparison 95-9625
Summary
Strengths of the interactive video program have remained relatively constant over this
two-year period. Interaction with others, access to previously unavailable classes, and learning
(including technological exposure were most frequently cite in both years.
Responses to weaknesses of the program have, however, changed indicating significant
improvement in equipment. This, however, has been replaced with increased citation of
scheduling and behavior weaknesses. The scheduling weakness is also reflected as an increase in
suggested improvements.
There were no significant differences in students evaluation of the interactive video
program between those participating in 1995 and those participating in 1996. A significant
interaction effect detected between survey year and year of program existence showed very little
practical importance. It was, therefore, concluded there was no evidence of change in attitude
toward or evaluation of the interactive video program by high school students from 1995 to 1996
on this portion of the questionnaire. There are differences, however, in student evaluations by
site and by year of program existence.
Recommendations
This evaluation indicated there has been a significant improvement in equipment and its
maintenance. There are, however, other problems that have occurred. Due to the increased
frequency of response concerning problems with school schedules, this needs to be investigated
to determine if this problem can be resolved. Students need to be on similar schedules if at all
possible. In addition, there may be some problem developing in student behavior. Home site
students have indicated a lack of discipline and/or cheating at the remote site. This may or may
Comparison 95-9626
not become a problem, but does merit further investigation.
In addition, the questionnaire needs to be revised. Three factors (Class Evaluation,
Student Behavior, and Interaction) are not measuring consistently across years. This indicated
that the questions within these factors are not clear, or the factor needs to be strengthened with
additional questions. The cautions included concerning student behavior and class evalution my
also reflect this problem.
Appendix A
Table 1: Responses to Strengths
Table 2: Responses to Weaknesses
Table 3: Responses to Suggested Improvements
41
Comparison 95-9627
0
Comparison 95-96
Table 1
Codes for Answers given in response to the open-ended Strength
28
Factor Sub-factor Code Label
Interaction Meeting People 0 'Meet more people'2 'Interaction /Oth Schl'10 'Meeting different Schools'
Interaction Open-minded 21 'More Open-minded'30 'More Opinions'
Learning Technology 4 'Different - Rerun Lecture'6 'Picture'7 'Elmo'9 'Less intimidating'15 'Technology Exposure'17 'More Interesting'24 'Something New'25 'More I/A Cony'
Learning Instruction 5 'Instruction'19 'Better Understanding'20 'More Hands On'22 'Prepare for College'23 'Better Classes'26 'Quality Teacher'33 'Easier'34 'Question as needed'36 'More Learning'
Access Remote 1 'Remote'
Access Available Classes 3 'More can take'8 'Class Variety'11 'Wider Access'18 'Take Prey. Unavail Class'
Table 1 (Continued)
Comparison 95-9629
Factor Sub-factor Code Label
Behavior 12 'Greater Student Responsibility'13 'Less Cheating'14 'Increase Listening'29 'More Participation'35 'Better Study Habits'
Other 16 'None'27 'Saves Money'28 'More students/teacher'31 'Material Timely'32 'Better Environment'
Comparison 95-9630
Table 2
Weakness Response Codes By Factors and Factor Sub-divided
Factor Sub-factor Code Label
Behavior 0 'Increased Absence'4 'Easy to Cheat'5 'Noisy (Talking)'10 'Lack Participation'21 'Lack Discipline'29 'Pay less Attention'36 'Boring'
Equipment Sound 1 'Equipment - Sound'30 'High Voice not Heard'
Equipment Camera 2 'Equipment - Camera'14 'Video Intimidating'38 'Video Small'39 'TV Glare'
Equipment 6 'Time - Iniating Equipment'19 'Lack Communication'24 'Equip Fail - Severe'42 'Slow - Delays'
Equipment Minor 28 'Equip Fail - Occas'
Instruction 7 'Attention to Remote Students'8 '1-1 diff with remote'
12 'Teacher Attn to HomeSite'13 'Instruction Light'15 'Teacher attn divided'25 'Little Learning'35 'Difficult Learning'40 No Extra Help'
Table 2 (Continued)
Comparison 95-9631
Factor Sub-factor Code Label
Instruction Remote 9 'Teachers Need to be There'11 'Personal Contact Other Student'22 'No teacher at Remote'27 'Personal Contact Techer'31 'Impersonal - slight'34 'Remote less impt'
Scheduling 16 'Different School Holidays'23 'Schedule Conflicts'26 'Class Cancelled'
Other 3 '3 hour meeting too long'17 'Mail Late'18 'None'20 'Too many students/teacher'32 'Due date materials'33 'More space'37 'Large Classes'41 'No Field Trip'
45
Comparison 95-9632
Table 3
Suggested Improvement Response Codes By Factors and Factor Sub-divided
Factor Sub-factor Code Label
Equipment Sound 1 'Fix Sound'
Equipment Camera 2 'Fix Camera'
Equipment 16 'User-Friendly Equipment'17 'Facilitator initiate equipment pri'19 'Class on Equipment Use'29 'Fix equipment'30 'Trained Eqip/fix standby'42 'Need Chalkboard'47 'Better Communication'
Learning Class 3 'More class meetings - less lngth'14 'Require # Students/Site'21 'Reduce lecture time'23 'Longer Class Period'24 'Smaller Classes'41 'Rotate Home/remote'
Learning Discipline 8 'Better Discipline'28 'Better discipline - remote'44 'Screen students'
Learning Instruction 10 'No Learning'22 'Better Teacher Interaction'25 'Better Teachers'35 'Syllabus'37 'Experienced Teachers'38 'Remove facilitators'46 'Tutors'
Remote 15 'Facilitator at Remote'18 'Teacher needs to visit'
Table 3 (Continued)
Comparison 95-9633
Factor Sub-factor Code Label
Interaction 9 'More Interaction'13 'Some Total Class Meetings'20 'Invite More Schools'33 'More people'
Scheduling 26 'Synchronize time frame'27 'Same schedule'43 'Snowday Schedule'
Variety 4 'Advertise'5 'More locations'
12 'Offer More Classes'
Other 11 'Quit Using'31 'Internet use'32 'More exposure class content'34 'none'45 'Return work'
Environment 6 'Larger Monitor'7 'Better Setting'
36 'More seating'39 'Overhead Speakers'40 'Bigger Desks'
U.S. Department of EducationOffice of Educational Research and Improvement (OEM)
National Library of Education (NLE)Educational Resources Information Center (ERIC)
REPRODUCTION RELEASE(Specific Document)
I. DOCUMENT IDENTIFICATION:
ERIC
Title: Q,antioueltson A \C'n 1Clq le \N Nrk. e-VC/01 \-t,L41e-14
Author(s): , IADCorporate Source: INE1/4.42_,Noll uvr's On 49-9r
e 0
() its I° s-vr,
cam+ELP-c6.0swx Jc
Publication Date:
tr0
II. REPRODUCTION RELEASE:
In order to disseminate as widely as possible timely and significant materials of interest to the educational community, documents announced in themonthly abstract journal of the ERIC system, Resources in Education (RIE), are usually made available to users in microfiche, reproduced paper copy,and electronic media, and sold through the ERIC Document Reproduction Service (EDRS). Credit is given to the source of each document, and, ifreproduction release is granted, one of the following notices is affixed to the document
If permission is granted to reproduce and disseminate the identified document, please CHECK ONE of the following three options and sign at the bottomof the page.
The sample sticker shown below will beaffixed to all Level 1 documents
PERMISSION TO REPRODUCE ANDDISSEMINATE THIS MATERIAL HAS
BEEN GRANTED BY
\e
TO THE EDUCATIONAL RESOURCESINFORMATION CENTER (ERIC)
Level 1
Check here for Level 1 release, permitting reproductionand dissemination in microfiche or other ERIC archival
media (e.g., electronic) and paper copy.
The sample sticker shown below will be The sample sticker shown below will beaffixed to all Level 2A documents affixed to ell Level 29 documents
PERMISSION TO REPRODUCE ANDDISSEMINATE THIS MATERIAL IN
MICROFICHE. AND IN ELECTRONIC MEDIAFOR ERIC COLLECTION SUBSCRIBERS ONLY,
HAS BEEN GRANTED BY
TO THE EDUCATIONAL RESOURCESINFORMATION CENTER (ERIC)
2A
Level 2A
ElCheck here for Level 2A release. permitting reproducgonand dissemination in microfiche and in electronic media
for ERIC archival collection subscribers only
PERMISSION TO REPRODUCE ANDDISSEMINATE THIS MATERIAL IN
MICROFICHE ONLY HAS BEEN GRANTED BY
2B
\e
TO THE EDUCATIONAL RESOURCESINFORMATION CENTER (ERIC)
Level 28
Check here for Level 2B release, permittingreproduction and dissemination in microfiche only
Documents will be processed as indicated provided reproduction quality permits.If permission to reproduce Is granted, but no box is checked, documents will be processed at Level 1.
I hereby grant to the Educational Resources Information Center (ERIC) nonexclusive permission to reproduce and disseminate this documentas indicated above. Reproductith from the ERIC microfiche or electronic media by persons other than ERIC employees and its systemcontractors requires permission from the copyright holder. Exception is made for non-profit reproduction by Wanes and other service agenciesto satisfy information needs of educators in response to discrete inquiries.
Sign s'gn.lure
here,-* \17)2:'please °manizaummaress.
t--6 z .9cksu 4y jrIki1L-les
-__-.
A _
Printed Name/Position/Ms
. Lp,x_vi,-44 6311,
TelePtAll //) _)FAA:
E-Mai Address:
US in itZin°ate
(over,