Online vs Face to Face: Is There a Significant Difference in Learning Outcomes?

Elaine Gerber & AJ Kelton, Montclair State University

& Tracy Chu,Brooklyn College, CUNY

AbstractOur aim is to better understand whether and how instructional modality (online vs face-to-face) impacts student learning. More specifically, this presentation compares survey results and final grades from an in-person face-to-face class with those from an online class, where the professor, course, and semester are the same. While much has been written about online learning, it is rare for the same course to be taught by the same professor in the same semester at the same institution. Thus, these circumstances provided a unique opportunity for comparison. Consistent with extant research, our preliminary findings suggest that there is no significant difference in learning outcomes, according to modality. What did appear significant, however, is that the online students worked more hours and experienced more life obstacles. Despite working more hours and having more obstacles, the online students performed as well in the course overall, as students who didn’t experience as many challenges or spend as many hours in paid employment. This suggests that certain attributes of online learners, such as being highly organized, a “self-starter,” or previously performing well in school, may be better predictors of success than the modality used to deliver course content. This research is ongoing in order to gather a larger sample with greater predictive power. Our presentation at the RAUL Showcase 2014 will highlight findings from the first round of data collection (Spring 2013).

Research DesignA “natural experiment”

• Same course• Same professor• Same school• Same semester

RQ: Is there a significant difference in learning outcomes by modality?

Same Course?!Similar / shared the same:• course objectives, readings• enrollment size (35 students each)• Instructor, semester, university

Differences: • “lecture” content • nature, amount, and quality of peer-based discussion• presence of the instructor• amount of writing involved• weights attributed to various assignments• type of student taking each modality (below)

Other limitations?

• How similar is the course really? (see above)• Planning / Intentional course structure for

research design vs retro-fitting to work with existing course offerings

• Intellectual property issues • Self-reports, and the problems that go with it• Small sample size• Student characteristics / attributes

Sample Bias?

Data Collection

• Subjects were recruited from the full class roster of all students who were enrolled in ANTH 110-01 (face-to-face) and ANTH 110-03 (online) at Montclair State University, Spring 2013

• Students were given 2 points on their final exam for participating; they also had the option to complete a short assignment in lieu of participating, in order to receive the same extra credit.

• The professor did not know which students had completed the survey and which had done the optional assignment, only which students were to receive the extra two points on their final exam, until after grades were submitted to the Registrar.

Data Analysis

Data were then exported from the Limesurvey database into SPSS for analysis. Most questions were close-ended and could therefore be analyzed using this program to measure frequencies and other descriptive statistics. We used T-tests for correlations and a multiple regression analysis, which I’ll talk more about below.

The few open-ended questions were analyzed “by hand,” by reading and re-reading the data in either MS Word or Excel. There was not sufficient open-ended content to warrant using a qualitative software analysis program, such as The Ethnograph.

Preliminary Findings

• Still preliminary, ongoing• Collecting “round 2” data Spring 2014

Nonetheless……..

What did we find?

Statistical Findings:

Only significant difference between the online and F2F samples is that:

online students worked more hours, and experienced more obstacles!

But, they did just as well in the course overall…

Grades by modality

Final Exam Course Grade

F2F 85.23 84.5

Online 84.13 85.5

Obstacles

Are there any obstacles to learning that you faced this semester? (check all that apply):

Working too many hours Commute to class and/or parking often caused me to miss classI had family emergenciesI had personal issuesI had housing problemsI was too busyAccess to computers & Internet were not reliableCourse content too intellectually difficult or presented too poorly to learnOther (please explain): ______________________________

Predictors of Learning Outcome

When looking at the entire sample overall (n=41) , the only variables that correlate with FINAL GRADE are RACE and GPA.

When looking at each modality separately, in the online class (n=17), RACE was correlated to FINAL GRADE, but GPA was not. However in the F2F class (n=24), GPA was correlated to Final Grade, and RACE was not.

When GPA is controlled for, race is no longer a significant predictor of course grade (Model 1 & Model 2).

Moreover, modality does NOT predict course grade, even when controlling for race and GPA (Model 3).

Controlling for modality and race, only significant predictor of course grade is previous GPA (Model 3).

Discussion

What is going on here?

Other Predictors of Outcome?

Do you consider yourself… (select all that apply):

very organizedhighly motivated to learnan independent workerable to multi-taskgood with time managementfocused & goal-orientedcomfortable with technology other: _________________

Implications

Not all students are the same, no universal student

Implications for academic advising:• Retention• Time to graduation

Questions? Suggestions?

…And, thanks!