Download - Logging on to Improve Achievement
![Page 1: Logging on to Improve Achievement](https://reader035.vdocument.in/reader035/viewer/2022081516/558e89ac1a28ab574e8b467a/html5/thumbnails/1.jpg)
Logging On To Improve AchievementEvaluating the relationship between use of the Learning Management System, student characteristics, and academic achievement in a hybrid large enrollment undergraduate course
Research Highlights: Presentation to SoLAR Storm
November 15, 2012John C Whitmer ([email protected])
Committee Chair: Dr. Paul Porter, Sonoma State University
Slides: http://slidesha.re/sFKjcm
![Page 2: Logging on to Improve Achievement](https://reader035.vdocument.in/reader035/viewer/2022081516/558e89ac1a28ab574e8b467a/html5/thumbnails/2.jpg)
Introduction
• Educational Doctorate Degree (EdD) candidate (University of California Davis & Sonoma State University)
• Advanced to candidacy, defending ~ January 14
• Associate Director, California State University LMSS Project, Chancellor’s Office
Me
![Page 3: Logging on to Improve Achievement](https://reader035.vdocument.in/reader035/viewer/2022081516/558e89ac1a28ab574e8b467a/html5/thumbnails/3.jpg)
Presentation Outline
1. Study Case & Context
2. Results for Instructional Practices
3. Results for LMS Data Analysis
4. Conclusions
![Page 4: Logging on to Improve Achievement](https://reader035.vdocument.in/reader035/viewer/2022081516/558e89ac1a28ab574e8b467a/html5/thumbnails/4.jpg)
STUDY CASE & CONTEXT
![Page 5: Logging on to Improve Achievement](https://reader035.vdocument.in/reader035/viewer/2022081516/558e89ac1a28ab574e8b467a/html5/thumbnails/5.jpg)
Problem: Student Graduation• Less than 50% of college/university students graduate
within 6 years• California State University: 52.4%
(first-time freshman, 2000 cohort) (CSU Analytic Studies, 2011)
• Students from under-represented minority racial/ethnic groups graduate at much lower rates • California State University: 38.3%
(African American students, first-time freshman, 2000 cohort) (CSU Analytic Studies, 2011)
• Contributing factor: mega-enrollment intro courses• Infrequent interaction, prevent faculty/student relationships
![Page 6: Logging on to Improve Achievement](https://reader035.vdocument.in/reader035/viewer/2022081516/558e89ac1a28ab574e8b467a/html5/thumbnails/6.jpg)
Case: Introduction to Religious Studies
• Redesigned to hybrid delivery through Academy eLearning
• Highest LMS usage entire campus Fall 2010 (>250k hits)
• 373 students (54% increase)
• Bimodal results• 10% increased SLO mastery• 7-11% increase in DWF
54 F’s
![Page 7: Logging on to Improve Achievement](https://reader035.vdocument.in/reader035/viewer/2022081516/558e89ac1a28ab574e8b467a/html5/thumbnails/7.jpg)
Research Questions
1) Is there a relationship between student LMS usage and academic performance? Does this relationship vary by the pedagogical purpose underlying LMS usage? (correlation)
2) Is there a relationship between student background characteristics or current enrollment information and academic performance? (correlation)
3) Does analyzing combined student characteristics and current enrollment information increase the predictive relationship between combined LMS usage data and student success? (multivariate regression)
4) Does a student’s economic status and student of color status vary the predictive relationship between combined LMS usage, combined background characteristics and current enrollment information? (multivariate regression, restricted model)
![Page 8: Logging on to Improve Achievement](https://reader035.vdocument.in/reader035/viewer/2022081516/558e89ac1a28ab574e8b467a/html5/thumbnails/8.jpg)
Independent Variables: Student Characteristics
![Page 9: Logging on to Improve Achievement](https://reader035.vdocument.in/reader035/viewer/2022081516/558e89ac1a28ab574e8b467a/html5/thumbnails/9.jpg)
Independent Variables: LMS Usage
![Page 10: Logging on to Improve Achievement](https://reader035.vdocument.in/reader035/viewer/2022081516/558e89ac1a28ab574e8b467a/html5/thumbnails/10.jpg)
Research Methods (Cliff’s notes version)
1. Extract data, validate with appropriate “owner”
2. Transform variables • measures of interest (e.g. “URM”, not race/ethnicity)• analysis methods (categorical into numeric)
3. Examine data for • outliers, missing data, data distributions, etc.• colinearity between variables (e.g. independence)
4. Join data into single data file, collapse to one record/student
5. Run analysis
![Page 11: Logging on to Improve Achievement](https://reader035.vdocument.in/reader035/viewer/2022081516/558e89ac1a28ab574e8b467a/html5/thumbnails/11.jpg)
Results for Instructional Practices
![Page 12: Logging on to Improve Achievement](https://reader035.vdocument.in/reader035/viewer/2022081516/558e89ac1a28ab574e8b467a/html5/thumbnails/12.jpg)
Correlation: LMS Usage w/Final Grade
Scatterplot of Assessment Activity Hits v.
Course Grade
![Page 13: Logging on to Improve Achievement](https://reader035.vdocument.in/reader035/viewer/2022081516/558e89ac1a28ab574e8b467a/html5/thumbnails/13.jpg)
Correlation: Student Char. w/Final Grade
![Page 14: Logging on to Improve Achievement](https://reader035.vdocument.in/reader035/viewer/2022081516/558e89ac1a28ab574e8b467a/html5/thumbnails/14.jpg)
Most interesting finding (so far):
Smallest LMS Use Variable
(Administrative Activities)
r=0.3459
Largest Student Characteristic
(HS GPA)
r=0.3055>
![Page 15: Logging on to Improve Achievement](https://reader035.vdocument.in/reader035/viewer/2022081516/558e89ac1a28ab574e8b467a/html5/thumbnails/15.jpg)
Regression R2 Results Comparison
![Page 16: Logging on to Improve Achievement](https://reader035.vdocument.in/reader035/viewer/2022081516/558e89ac1a28ab574e8b467a/html5/thumbnails/16.jpg)
RESULTS FOR LMS DATA ANALYSIS
![Page 17: Logging on to Improve Achievement](https://reader035.vdocument.in/reader035/viewer/2022081516/558e89ac1a28ab574e8b467a/html5/thumbnails/17.jpg)
Lms Logfiles: “Data Exhaust”
1. Logfile tracks server actions (not educationally relevant activity)
2. Duplicate logfile hits for single student action
3. To remedy, filtered logfiles by:• Time (> 5sec, <3600 sec)• Actions (no “index views”, more)
![Page 18: Logging on to Improve Achievement](https://reader035.vdocument.in/reader035/viewer/2022081516/558e89ac1a28ab574e8b467a/html5/thumbnails/18.jpg)
Logfile Data Filtering Results
Discus
sion
Activi
ty H
its
Conte
nt A
ctivi
ty H
its
Asses
smen
t Act
ivity
Hits
Mai
l Act
ivity
Hits
Admin
istra
tive
Activi
ty ..
.0
50
100
150
200
250
300
350
400
450
382
151
58 4926
54 5123 36
16
Final data set: 72,000 records (from 250K+)
![Page 19: Logging on to Improve Achievement](https://reader035.vdocument.in/reader035/viewer/2022081516/558e89ac1a28ab574e8b467a/html5/thumbnails/19.jpg)
LMS Use Consistent across Categories
Factor Analysis of LMS Use Categories
![Page 20: Logging on to Improve Achievement](https://reader035.vdocument.in/reader035/viewer/2022081516/558e89ac1a28ab574e8b467a/html5/thumbnails/20.jpg)
Missing Data On Critical Indicators
![Page 21: Logging on to Improve Achievement](https://reader035.vdocument.in/reader035/viewer/2022081516/558e89ac1a28ab574e8b467a/html5/thumbnails/21.jpg)
Conclusions
1. At the course level, LMS use better predictor of academic achievement than any student characteristic variable. Behavioral data appears to supercede demographic information (what do, not who are).
2. Moderate strength magnitude of complete model demonstrates relevance of data, but suggests that refinement of methods could produce stronger results.
3. LMS data requires extensive filtering to be useful; student variables need pre-screening for missing data.
![Page 22: Logging on to Improve Achievement](https://reader035.vdocument.in/reader035/viewer/2022081516/558e89ac1a28ab574e8b467a/html5/thumbnails/22.jpg)
Ideas & Feedback
Potential for improved LMS analysis methods:• social learning • activity patterns • discourse content analysis• time series analysis
Group students by broader identity, with unique variables:• Continuing student (Current college GPA, URM, etc.• First-time freshman (HS GPA, SAT/Act, etc)
![Page 23: Logging on to Improve Achievement](https://reader035.vdocument.in/reader035/viewer/2022081516/558e89ac1a28ab574e8b467a/html5/thumbnails/23.jpg)
Contact Info
John Whitmer
Skype: john.whitmer
USA Phone: 530.554.1528
By WingedWolfDamián Navas