the uiuc scale experiments by dan stone (with help from lanny arvan & burks oakley) univ. of...
Post on 26-Dec-2015
217 Views
Preview:
TRANSCRIPT
The UIUC SCALE Experiments
By Dan Stone (with help from Lanny Arvan & Burks Oakley)
Univ. of Illinois at Urbana-Champaign, 1988 – 2000Univ. of Kentucky, beginning 8/2000
Presentation Outline
Thanks to …!Motivation Introduction to SCALE & ALNSCALE Research Design / EvaluationSCALE ResultsThe Bottom Line: What
creates successful ALN?
Thanks to:Bob Jensen, Lanny Arvan (UIUC), Burks Oakley (UIUC) for their enthusiasm, risk-taking, leadership & innovation
More thanks & copyright disclosure: portions of this presentation
are taken from presentations by Lanny Arvan & Burks Oakley
portions of this presentation are taken from the SCALE & CET websites
See citations at end of presentation
Presentation Outline
Thanks to …!Motivation Introduction to SCALE & ALNSCALE Research Design / EvaluationSCALE ResultsThe Bottom Line: What
creates successful ALN?
Motivation
Huge investments in asynchronous learning network (ALN) programs (e.g., Penn State World Campus, Stanford-Online, Duke Global Exec MBA)
SCALE project: over 80 on-line courses
Motivation
Do the SCALE results suggest that ALNs can improve: educational effectiveness? efficiency? access to higher education?
What makes for successful
on-line courses & programs?
Motivation
My interest in SCALE & ALN: Teach accounting systems &technology classes Use pedagogy (technology) to
teach some course content (technology)
Adopted computer aided instruction module in 1991, adopted Powerpoint in 1992In the first group of adopters of First-Class at UIUC (1994)On-line grading & email software for 6 yearsOn-line, web-based course materials for 3 years
Motivation
My interest in SCALE & ALN:Lingering doubts … if I weren’t teaching “technology as content” would “technology as pedagogy” be worth the trouble? “Recent surveys of the instructional use of
information technology in higher education clearly indicate that there have been no significant gains in either productivity improvement or pedagogical enhancement” (Kenneth Green, 1997, “The Campus Computing Project”).
Clark 1991, “Media do not influence learning.” “ALN teaching is arduous.”
Presentation Outline
Thanks to …!Motivation Introduction to SCALE & ALNSCALE Research Design / EvaluationSCALE ResultsThe Bottom Line: What
creates successful ALN?
Introduction to SCALE & ALN
Sloan Center for Asynchronous Learning Environments (SCALE) (1995-1999) Origin: $2.1 million grant from Alfred P. Sloan
Foundation + UI matching Goal: Restructure undergraduate courses to
integrate asynchronous learning network (ALN) methods
20 major projects in 1995, now > 80 projects 9 “efficiency” projects – designed to lower instructional costs per student
Introduction to SCALE & ALN
SCALE replaced by UIUC funded:
Center for Educational Technologies (CET) Lanny Arvan, Director
UI-Online Led by Sylvia Manning
(Chancellor)
Introduction to SCALE & ALN
What is ALN? In-class learning is “synchronous.” Contrast synchronous in-class learning
with “Site independent learning” (distance
education) –courses delivered largely or entirely online
Adapted from Lanny Arvan presentation (the Scale Evaluation)
ALN Goals & Technologies
Enhanced learning opportunities Interactive course materials,
e.g., PCs & WWW Simulations, multimedia,
visualization, e.g., streaming video, Java, Virtual reality (e.g., digital video cases – “WWHP A CD Case study” (http://www.cba.uiuc.edu/lab/BrownBags/WWHP.pps)
Real-time homework, quizzes & grade information, e.g., on-line, interactive testing, on-line grades Adapted from Burks Oakley
presentation (“E-Learning”)
ALN Goals & Technologies
Improve access to people & information Subject matter experts (faculty, TA’s) Other students (peer-peer & group
interactions) E.g., Conferencing, email E.g., Course management
software (i.e., CourseInfo,
WebCT, Mallard). Adapted from Burks Oakley presentation (“E-Learning”)
ALN Example Applications
Econ 300 (Intermediate Microecon) – Lanny Arvan SCALE efficiency project A required (disliked) course for all
UIUC business majors ALN goals:
More feedback & learning opportunities (Mallard on-line testing)
Increase interactions (on-line) with faculty & peers (First Class)
Disincentives for cram-for-test-&-nothing-else mentality (homework & quizzes count)
Adapted from Burks Oakley presentation (“E-Learning”)
ALN Example – Econ 300 First Class Menu
Presentation Outline
Thanks to …!Motivation Introduction to SCALE & ALNSCALE Research Design / EvaluationSCALE ResultsThe Bottom Line: What
creates successful ALN?
SCALE Research Design / Evaluation
Separate University-sponsored evaluation team & unitResearch design: Case & field studiesEven when available, ALN vs. nonALN control groups are problematic due to: Student self-selection into courses & programs Instructor self-selection into using technology Differing evaluative methods & processes (e.g.,
differing rules for on-line vs. traditional exams, quizzes) (I.e., mixing “media” & “method” – Clark)
SCALE Research Design / Evaluation
Most extensive evaluation is of 1st year classes (19 courses)
An impression evaluation team sought evidence supporting increased efficiency & effectiveness (after spending $3M – what else could UIUC do?)
SCALE Research Design / Evaluation
Data Source Sample Size
Pre-course Student Surveys 1,438 Students in 16 Courses
Student Group Interviews 10 Groups
Teaching Assistant Group Interviews 2 Groups
End-of-Semester Student Surveys 1,118 Students in 17 Courses
Pre-Course Faculty Interviews 26
End-of-Semester Faculty Surveys 23
End-of-Semester Faculty Interviews 6
Computer Conference Monitoring 5 Courses
Adapted f from SCALE website
Presentation Outline
Thanks to …!Motivation Introduction to SCALE & ALNSCALE Research Design / EvaluationSCALE ResultsThe Bottom Line: What
creates successful ALN?
SCALE Evaluation Results
ALN...Increased communications Surveys: 51% of students reported an
increase in communication with instructor & 43% with other students.
40% of students reported an increase in the quality of their interaction with the instructor.
Improved access to information – "Information when you want/need it." "Personal control of information," "quick-response from peers & students."
Adapted f from SCALE website
SCALE Evaluation Results
ALN... Enhanced learning environment “A new and exciting way to learn," “More prepared
for class," “Time to learn out of class," "work at own pace."
"On-line homework was a great experience," “On-line quizzes were "a good way to study for exams."
70% would take another course using computer conferencing.
75% rated overall experience with computer conferencing good, very good, or excellent.
60% reported increase in learning due to computer conferencing.
Adapted f from SCALE website
SCALE Evaluation Results
ALN...Improved computing & technology knowledge "Increased my knowledge of and confidence
with computers." “I’m less apprehensive about computers"
70% indicated increased familiarity with computers.
Adapted from SCALE website
SCALE Evaluation Results
Success of ALN depended on...Student training 75% found conferencing software
easy or somewhat easy to use. Interviews: Many students wanted better
training. E.g., "Require a mandatory class on system use."
Student effort & commitment 60% used computer conferencing 1 time week. Females used conferencing slightly more than did
males (not stat significant) “Class member comments helped me learn."
Adapted from SCALE website
SCALE Evaluation Results
Success of ALN depended on...Incentives, I.e., what’s in it for the student? Students: "Must give incentives to use the software." “Don’t just tack ALN on to an existing course."
Accessible computers Lack of computer access = most frequent complaint "It was hard to find an available computer." "The [modem] lines were always busy." "Get more computers or get rid of [ALN component]."
Adapted from SCALE website
SCALE Evaluation Results
Success of ALN depended on...System availability & support "I couldn't login,
so I stopped using it."
"Too many system crashes"
Adapted from SCALE website
SCALE Efficiency Project Results
Did ALN improve course efficiency?From whose perspective? (instructor vs. university) Instructor time omitted as a cost … despite
acknowledgement that “ALN teaching is arduous.” (instructor interview). Increased instructor-student contact time in ALN classes.
Never for small or writing-intensive classesFor each of the 3 examples that I use: No difference in exam performance for ALN vs.
traditional sectionsAdapted from SCALE website
SCALE Efficiency Project Results
Did ALN improve course efficiency?Introductory Statistics (100) Development cost funded by grant On-going cost of ALN initiatives = $1,000
peer tutor per semester ALN class sizes 50 to 100% larger
than traditional sections
(e.g., 154 vs. 101 students)Adapted from SCALE website
SCALE Efficiency Project Results
Did ALN improve course efficiency?Econ 300 Section sizes increased from 60 to 180 Development cost = ~ $56,224
Introductory Spanish (210) 1 time development cost =
~ $15,000 Class section sizes doubled with ALN (19 to 38)
Adapted from SCALE website
An ALN success story (LEEP)
Adapted from Burks Oakley presentation (“E-Learning”)
UIUC School of Library & Info. ScienceOnline version of on-campus MS degree (not a SCALE project) – Library Education Experimental
Program (LEEP) - accredited by the ALA
Began with 31 students in Fall 1996 (including students in Alaska & Vermont)
Currently 125 students enrolled (~ 1/2 in-state)
An ALN success story (LEEP)
As of 1999, 96 students completed MS degree (from 37 states, rural Illinois, Virgin Islands, France, Colombia, Japan, & Thailand)All faculty teach 1 online course each academic yearTo serve demand for LEEP, GSLS College: added 4.5 FTE faculty budget has grown > $750,000 New building in progress!
Adapted from Burks Oakley presentation (“E-Learning”)
SCALE Efficiency Project Results
Do the SCALE results suggest that ALNs can improve access to higher education? Yes!!!! LEEP program & SCALE efficiency
projects
Presentation Outline
Thanks to …!Motivation Introduction to SCALE & ALNSCALE Research Design / EvaluationResultsThe Bottom Line: What
creates successful ALN?
The Bottom Line: What makes for a successful ALN?
Support from the top for technological innovation (e.g., LEEP) Incentives for participation among
faculty & departments Pedagogically sound content Commitment to quality pedagogy & content
Enthusiastic & committed faculty & support staff Are faculty early (15%), middle (70%), or late (15%)
adopters? - Incentives differ by group Helpful: wired, computer literate students & faculty
Adapted from Burks Oakley presentation (“E-Learning”)
The Bottom Line: What makes for a successful ALN?
Thoughtful consideration of copyright and ownership issues (faculty vs. University, publishers vs. University, faculty vs. students) Who owns what? Accessibility vs. ownership (see
recent Chronicle of Education articles)
Support & maintenance Development / programming Access to student computing is essential (& often problematic) Supported, mature products
Adapted from Burks Oakley presentation (“E-Learning”)
The Bottom Line: What makes for a successful ALN?
Marketing to current & potential students, current & potential faculty, University administrators What’s in it for the students?
faculty? administrators?
What’s the goal? Improved class availability? Technology as
content? Richer content? Adapted from Burks Oakley presentation (“E-Learning”)
References & resources
Clark, Richard E., 1991, “When researchers swim upstream: reflections on an unpopular argument about learning from media,” Educational Technology (February), 34-40.
Scale website: http://w3.scale.uiuc.edu/scale/index.html
UI on-line: http://www.online.uillinois.edu/
Web-board tutorial: http://w3.scale.uiuc.edu/scale/docs/using_webboard/index.html
LEEP program: http://alexia.lis.uiuc.edu/gslis/degrees/leep.html
UIUC Center for Educational Technologies (CET): http://www.cet.uiuc.edu/
Webboard: http://webboard.oreilly.com/
WebCT: http://www.cet.uiuc.edu/webct/index.asp
Dan Stone
Email: dstone@pop.uky.edu
WWW: http://www.cba.uiuc.edu/d-stone1/d-stone1.html
Phone: 257-3788
top related