Evaluation of a Flipped Classroom Format for an Introductory Level

Marketing Class

George KruegerAssistant Professor

University of Wisconsin--Platteville

UW-Platteville

Flipped Class Concept

The research is mixed on learning outcome results. Some studies show a positive impact on quantitative students results and qualitative student perceptions. Others show a negative impact or no difference.

Marketing is an application-based subject and largely qualitative in evaluating student results. Would a flipped class result in a more positive student outcomes?

Purpose

The purpose of this study was to evaluate the flipped classroom methodology in an introductory level course.

The professor has taught the course for several years in a conventional lecture format.

What is “Flipped”

There are numerous ways to define flipped as a teaching methodology. • Video presentations• Pre-assigned readings• Assignments to be completed outside of class and

then reviewed during class• Outside of class online discussions • Preparation for student presentations.• Online quizzes

Online Quizzes

• Flipped classroom 10 quizzes at 50 points & due the Monday night BEFORE the Tuesday/Thursday classes

• Control: 10 quizzes at 20 points due the Thursday before Friday class

Allocation of Class Time

Flipped: • Student presentations• In-depth discussions

Control• More lecture, less projects and discussion

Let’s See What Happens at UWP

The Classes

• BSAD 2630 Introduction to Marketing• Section 02: Tuesdays and Thursdays from 9:30 to 11:45 • Section 03: Monday, Wednesday, Friday from 9:00 to 9:52. • The courses used the same textbook, in-class instruction

materials and followed the same course content. • Students in each class was 35 at the start of the semester. • The students were predominantly business majors along

with business minors. Very few were marketing emphasis.

Online Quizzes

• The quizzes were administered online in D2L.• Multiple choice questions were loaded

directly from the textbook testbank.• Quiz questions were randomized for each

student. • Open book/open notes. The quizzes covered

the chapters that were assigned for each week.

Flipped Group Projects

Each group focused on one company and applied the subject matter in discussion and presentation formats. – Oakley Sunglasses– Gatorade– Canon Digital Photography– Monster Beverages– Blackberry– UnderArmour

Control Group Projects

• Sunday Ticket—Direct TV• Netflix• Kwik Trip Convenience Stores• Mountain Dew KickStart• Domino’s

Discussions…what next?

Exams

• 3 exams administered during the semester. • Study guides were provided and the professor

emphasized key items that would be on the exam.

• Exams were similar but not exactly the same for the two classes.

• Approximately 60% multiple choice and 40% short answer/essay. The exams were returned to the students.

Final Exam

• Comprehensive exam covering the entire semester.

• Exams were exactly the same for the 2 groups.• 40% multiple choice and 60% application of

terms related to a case on Subway Restaurant’s marketing.

• The case subject was given to the students along with a study guide before the exam.

Final Individual Project

• An individual marketing plan project was assigned to the students.

• The student to selects a product or service to market and write a marketing plan for their selection.

• Outline and grading rubric was provided to each class.

• Intended to be a capstone application of the marketing concepts.

Summary of Grading Control Class Flipped Class

Online Quizzes 200 500

Group Projects 100 100

Exams 300 300

Final Exam 100 100

Final Individual Project 100 100

Class Participation 100 100

Total Points 900 1200

Results

Online Quizzes

• Conventional class took the quizzes after the course material had been covered in class and averaged 77.4%.

• Flipped class took the quizzes before the lecture they only trailed the conventional class by .7 at 76.7%.

• No statistical difference between the two measurements.

Exams

The results for the exams were not statistically significant. There were no trends apparent in the data. Exam 1 Exam 2 Exam 3 Final ExamFlipped 71.6% 79.0% 77.8% 68.2%Conventional 73.5% 76.6% 75.0% 72.5%• The observations of the Final Exam answers for

the Flipped group was deemed poor by the professor.

Final Individual Project

• The flipped class scored 80.8% on the individual marketing project while the conventional class scored 83.2%. There was no statistical difference.

Overall Class Grades

• The final class percentage achievement for the flipped classroom was 78.9% and the conventional class was 77.7%. No statistical difference.

Pre and Post Survey Results

  Flipped Flipped Control ControlQuestion Average

PreAverage Post

Average Pre Average Post

This class has met my expectations so far for content delivery

4.1 4.1 4.3 4.1

This class is taught like most classes that I have taken

2.7 3.2

3.1 3.2

I am satisfied with the way this course is being taught

4.2 4.0

4.0 3.7

I believe that I understand the content that has been covered so far in the class

3.8 3.8 3.8 3.7

I believe that I can apply the content that has been covered so far in the class

4.0 4.0

4.0 3.9

I believe that the learning objectives are clear

4.0 3.9 4.0 3.9

I am satisfied with my learning so far in this class

3.8 3.9 3.8 3.8

I believe that the amount of homework in the course is

2.1 1.9 2.0 1.9

I believe the difficulty of the homework is

1.9 1.9 2.0

1.8

Student Evaluations

Question Flipped Fall 13

Control F 13

Sp 13 Fall 12 Fall 12 Fall 11 Fall 11

3. Uses examples and illustrations that clarify the material.

4.857 4.524 4.367 4.308 4.519 4.548 4.31

4. Explains course material clearly.

4.5 4.05 3.967 3.8 3.926 4.097 4.1797. Incorporates current material into the course.

4.821 4.619 4.533 4.692 4.519 4.452 4.6219. Encourages classroom participation from students.

4.857 4.476 4.4 4.28 4.481 4.677 4.48310. Presentations/ discussions enhance the text.

4.571 4.238 4.367 4.038 4.296 4.258 4.138

15. Uses class time for student discussions, presentations or questions.

4.821 4.238 4.3 4.292 4.296 4.613 4.448

Mean 4.738 4.394 4.361 4.278 4.51 4.498 4.42Responses 28 21 30 26 27 31 29

Student CommentsFlippedPositives

14 Entertaining/enthusiasm/engaging6 Good knowledge6 Liked group presentations/discussions

Improvements 9 More help on projects/expectations/assignments2 Dislike online quizzes4 Go through study guides & reviews1 Less Powerpoint by Professor1 Some student groups were not good presenters

ControlPositives

14 Entertaining/enthusiasm/engaging5 Good knowledge/examples1 Liked group presentations/discussions1 Professor is concerned about students (cares)

Improvements7 More help on projects/expectations/assignments6 Dislike online quizzes1 Go through study guides & reviews3 Lectures can be confusing at times1 Stay on syllabus1 More activities

Professor Observations

They Are Students!

• Several times during the semester the professor had to intervene in the group presentations to correct mistakes and add in omitted subject matter. In addition, some of the presentations were very poor and students attention was lacking.

Unique Challenge

• The flipped format was a challenge with 35 students. Individual attention and a large number of groups can be difficult to manage in a classroom environment. When the professor is focusing on a group of 6 students the other 29 students can be disengaged from the class.

Soaring With Eagles?

Another Observation

• As noted in the research, not having a base of knowledge led to visible frustration from some students as observed by the professor. Often when the subject matter was enhanced in the presentations by the professor the response from the students was something like, “how are we supposed to know that?”

Interesting….

• The flipped format required more work outside of class by the students and this was perceived by the professor to be an issue during the semester. However, this perception did not manifest itself in the evaluations.

Interesting….

• Some students expressed their preference to be presented the information in lecture format first with explanation by the professor. They were then more comfortable and effective with the group work and projects.

Conclusions

Conclusions

• The flipped classroom methodology utilizing the key tools of online quizzes taken before classroom coverage of the subject matter, student preparation for presentations outside of class, student presentations and a much higher level of group work in class did not yield any statistical differences in student outcomes as measured by grades when compared the control class of lecture and class discussion format.

Conclusions

• The students did not perceive a significant level of difference in the way the classes were taught from their previous experiences.

Conclusions

• The flipped classroom did create a higher level of satisfaction for the students numerically and did appear to engage the students more actively as measured by student evaluation scores.

As a Result of This Study…

• Attempted to use the “best of both worlds” in the Spring 2014 for Introduction to Marketing and it worked very well.

• More focused lecture covering key areas. • More focus was put on class presentations

and projects applying the subject matter to their group projects.

Follow-Up Study

• Lessons Learned

– 1. To take the best of both worlds and utilize highly focused lectures on key marketing concepts during the Tuesday class session and then have the students apply the concepts in a weekly discussion format during the Thursday class session.

– 2. Utilize the student’s projects throughout the semester in the context of lecture and application of marketing concepts.

Group Project Discussions

Projects:• Gatorade• NorthFace• Signal Snowboards• Apple I-Phone• Samsung Galaxy Watch

Grading Summary of Grading

Lessons Learned Class Spring 2014

Flipped Class Fall 2013

Online Quizzes 200 500

Group Projects 100 100

Exams 400 300

Final Exam 100 100

Final Individual Project—Marketing Plan

100 100

Class Participation 100 100

Total Points 1000 1200

Table 2. Average scores for the fall 2013 Flipped class and spring 2014 Lessons Learned class.

Fall 2013 Flipped n = 33

Lessons Learned n = 31

t-test results

Mean Std Dev Mean Std Dev t p

Online Quizzes 77% 8.6 78% 16.5 0.344 .37

Total Exams 76% 9.2 76% 9.8 0.035 .49

Final Exam 68% 9.5 77% 16.1 2.62 .005**

Marketing Plan Project 81% 13.6 88% 13.5 2.02 .024*

*Significant at the .05 level, **Significant at the .01 level

Table 3. Student evaluations from fall 2011to spring 2014.

Question Lessons Learned Sp 2014

Flipped Fall 13

Control F 13

Sp 13 Fall 12 Fall 12 Fall 11 Fall 11

3. Uses examples and illustrations that clarify the material.

4.45 4.857 4.524 4.367 4.308 4.519 4.548 4.31

4. Explains course material clearly.

3.69 4.5 4.05 3.967 3.8 3.926 4.097 4.179

7. Incorporates current material into the course.

4.61 4.821 4.619 4.533 4.692 4.519 4.452 4.621

9. Encourages classroom participation from students.

4.66 4.857 4.476 4.4 4.28 4.481 4.677 4.483

10. Presentations/ discussions enhance the text.

4.03 4.571 4.238 4.367 4.038 4.296 4.258 4.138

15. Uses class time for student discussions, presentations or questions.

4.64 4.821 4.238 4.3 4.292 4.296 4.613 4.448

Mean (all responses) 4.39 4.738 4.394 4.361 4.278 4.51 4.498 4.42 Responses 29 28 21 30 26 27 31 29

Table 4. Categorical assessment of student comments including number of responses (n).

Done Well

n 2013 Flipped n Lessons Learned

14 Entertaining/enthusiasm/engaging 20 Entertaining/enthusiasm/engaging

6 Good knowledge 9 Good knowledge/examples

6 Liked group presentations/discussions 4 Liked group presentations/discussions

Improvements

n 2013 Flipped n Lessons Learned

9 More help on projects/expectations/assignments

8 More help on projects/expectations/ assignments

2 Dislike online quizzes 5 Dislike online quizzes

4 Go through study guides & reviews 4 Go through study guides & reviews

1 Less PowerPoint by professor

1 Some student groups were not good presenters

Group Work

• Discussions in the context of the weekly subject matter rather than presentations

Observations

• Improved engagement of students• More consistent engagement• Basic knowledge improvement—fewer

mistakes in subject matter concepts• Improved insight from the student—”why”• Samsung/Apple—excellent compare and

contrast examples• They really hate the online quizzes

Conclusions

• Improved learning outcomes as measured by final exam and individual marketing project scores

• Reverted back to the mean on student evaluations

Thank You!!