Organizational Change:An Unnatural Act

Gary GladdingUniversity of

IllinoisNovember 2, 2003

Introductory Calculus-Based Physics Course ConferenceArlington, VA

Introduction• Ask Google

First impression: Lots of Language Play e.g.,

Most relevant book?

The Guru Guide

Discussion of ideas of 79 management gurus

• Who says what?• Link ideas• Sense and nonsense

ConclusionsNot science (no all-encompassing theory)

Storytelling is important … On to our story!!

Seven Tips for Managing Organizational Change

• You have to establish a darn good reason to change• You have to create a compelling vision• You need results – fast• Communicate, communicate, and communicate some more• Build a strong committed, top management coalition• Don’t KISS (Keep it Simple, Stupid) – Keep it Complex• People Don’t Resist Their Own Ideas

• “Cliff Notes with an Attitude”

Road Map

• Overview– the OLD (all courses prior to Fall 96)– The NEW (all courses after Fall 99)

• The Revision– Faculty Participation– The Pieces (Lectures, Discussions, Labs, Exams,

Homework)

• Concluding Thoughts– Why did it work?– Questions??

The OLD • Introductory Physics at Illinois prior to Fall 1996

– We do “Educate in Bulk”• Calculus-based sequence FALL SPRING

– Physics 106 (Mechanics) 500 1000– Physics 107 (E&M) 800 450– Physics 108 (Waves) 400 750

• Algebra-based sequence– Physics 101 (Mechanics, thermo) 300 200– Physics 102 (E&M, modern) 200 300

2200 2700– Tradition, Tradition, Tradition

• Large (200-300) Lectures with Small (24) Sections for Discussions and Labs (6-7 hrs/week)

• Lecturers free to “reinvent the flat tire” , Discussion TAs pretty much on their own, Labs intellectually disconnected from rest of course.

• Exams: Quantitative Problems• RESULTS: NOBODY IS HAPPY !!

The NEW • Introductory Physics at Illinois as of Spring 2000

ALL COURSES TOTALLY REVISED !

The Big Idea: Integrate all aspects of a course using interactive engagement methods based on physics education research in

a team teaching environment

• ONE COURSE !!– All pieces of the course (lecture, discussion, labs, homework) must be

made of the same cloth.– The student should see a coherent plan at work.

• Emphasize Concepts– Traditionally, there is a large gap between what we think we are

teaching (physics) and what is being learned (equation manipulation)– Introduce explicit instruction on concepts (and test for it!)

• Use Interactive Engagement Methods– The learning of physics is NOT a spectator sport– Engage the student in all aspects of the course (including lecture)– Make use of the products of Physics Education Research (materials

and knowledge). There is a research base here and faculty (especially at a research university) should use it!!

Faculty Participation • Overriding Rule:

• Key Ideas• Sustainability cannot be built on heroism.• Faculty assignment must be seen as an ordinary assignment• Infrastructure lowers the bar for participation

• How to Do It? • 16-17 Faculty assigned for these courses (2500 students) • Responsibilities: Lecturer,Discussion Coordinator,Lab Coordinator• Faculty team meets weekly to keep course on track.• Faculty team creates exams• Support Infrastructure developed (computing, secretarial, …)

57 Faculty have taught in these revised courses!

NO HEROES!

The Pieces• Lectures, Discussions, Labs, Homework, Exams

– IMPORTANT RULE:

STEAL FIRST!

Well, maybe ADAPT is a better word?• Local conditions often dictate some modifications• Don’t invent anything until you discover a real problem that has not yet been solved.

Lectures• Calculus-Based Courses (111-114)

• PowerPoint Lectures with Video Projection • Students can buy hard copies of slides & bring them to lecture • Lectures are also available for viewing on the Web• Lectures punctuated by ACTs (interactive segments)

• Algebra-Based Courses (101-102)• “Just In Time Teaching”

• Students complete Web-based “preflights” (questions based on readings) BEFORE 8am on day of lecture.• Lecturer reads the responses of students and prepares “Lecture” between 8am and noon.• Lecture (at 1pm and 2pm) consists of explanations and ACTs that are designed to address the student difficulties seen in the preflights.

Discussion Sections

TA to the rescue?A Question!!

NO LECTURING HERE

• Key Idea: Collaborative Learning– Students work in groups of 4 on problems prepared by the senior staff.

TAs act as facilitators, not lecturers.– TA preparation very important (credit to Tim Stelzer)

• Orientation, Weekly Meetings, Mentor TAs, Observation– Content of prepared materials very important (Tutorials, Context-Rich,

and our own)

Labs• PREDICT• OBSERVE• EXPLAIN

– Adopt the approach of Thornton & Sokoloff to actively engage the students in the learning process and to promote mastery of concepts by manipulation of experimental apparatus.

– Prelab assignments; Lab reports finished within class period.

Student Satisfaction with Discussions and Labs

• How do students rate their TAs?– University-wide ranking of

“excellent” top 30% of peers

THE OLD Spring 95Total Physics TAs = 77# “Excellent” = 15 19 ± 5 %

THE NEW Spring 01Total Physics TAs = 75# “Excellent” = 58 77 ± 6 %

Exams

• What we used to do:– Exam composed of 4 multi-part calculational problems.– Exam graded by faculty + TAs immediately afterward. Subjective partial credit

given based on student’s approach.– Problems?

• Students can learn to do these problems without understanding what they are doing.

• Whining, cheating on regrades, questionable application of partial credit.• What we do now:

– Exam composed of Multiple Choice questions, both qualitative and quantitative, often using the same physical situation. We have always believed in the importance of conceptual understanding, but students didn’t believe us because we never explicitly asked these questions before!

– Partial credit scheme for quantitative (5 possible answers) questions. Students can choose to get reduced credit if they can successfully eliminate unphysical answers.

Reliability Study

4 courses32 course-sem51 profs128 exams>4000 questions>12000 students 0

200

400

600

800

1000

1200

1400

1600

1800

-50 -45 -40 -35 -30 -25 -20 -15 -10 -5 0 5 10 15 20 25 30 35 40 45 50

σ = 3 .1%

% Difference in Even and Odd Tests

# o

f st

ud

ents

σ = ~ .25 G P

0

200

400

600

800

1000

1200

1400

1600

1800

-50 -45 -40 -35 -30 -25 -20 -15 -10 -5 0 5 10 15 20 25 30 35 40 45 50

σ = 3 .1%

% Difference in Even and Odd Tests

# o

f st

ud

ents

σ = ~ .25 G P

Validity Study

0

20

40

60

80

100

120

0 20 40 60 80 100 120

MC

Sco

re

Committee Score

Rawr = 0.78

Correctedr = 0.98

0

20

40

60

80

100

120

0 20 40 60 80 100 120

MC

Sco

re

Committee Score

Rawr = 0.78

Correctedr = 0.98

Homework Assignments• Weekly Homework Assignments are done on the Web, using

the TYCHO system developed by Denny Kane at Illinois.– Preflights

• Students submit answers (usually multiple choice plus text box for explanation)

• Analysis tools available for faculty– Homework A

• Quantitative and symbolic questions (enter number or expression), unlimited submissions.. Fixed “help” statements available.

• Immediate response.

– Homework B• Multiple choice qualitative and quantitative (as in exams)• Delayed response (i.e., an online quiz)

– Interactive Examples• Web-based “Socratic dialogues” designed to promote

concept-based problem solving– More on this now as these exercises become our

contribution to the research base!!

Why Interactive Examples?• In all other aspects of our revision, we have “borrowed freely”

from the work of others!• We created Interactive Examples (IEs) to address a specific

problem for which we could find no existing solution.– The Problem:

• It’s been our experience that too many students see “concepts” and “calculations” as two totally separate and unrelated activities.

– When given a quantitative question, most students will NOT think about the CONCEPTS that are involved.

– When given a qualitative question, most students will never consider writing down an appropriate equation… math is NOT seen as a TOOL

– Our Solution:• Create web-based exercises that engage the student in the solution

of difficult quantitative problems using a “concept-based” method.• Implementation status:

– We have created about 125 IEs for five courses at Illinois– These IEs are being used for credit at 10 other institutions

What is an Interactive Example?

• Base question is a quantitative problem (multi-step).• Students can request help which comes in the form of more

questions.– These questions are designed to guide the student along a path

suggested by the UMASS PERG:• Conceptual Analysis: What concepts and/or principles determine

what will happen in the physical situation? Graphical representations? Qualitative Behavior?

• Strategic Analysis: What general approach to take? Develop a plan for applying the principles identified in conceptual analysis.

• Quantitative Analysis: What are the appropriate equations for this problem. Work out the mathematical solution.

• Meta Analysis: What have we done? Reflect on, make sense of the previous analyses. IEs use the optional follow-up questions to do this task.

• Students can opt to answer the base question at any time.• Eventually, enough help is given to solve the problem (it is an

“example”!)• Once the base question is answered correctly:

– A Recap is given (Conceptual, Strategic and Quantitative Analyses).– Follow-Up Questions (optional, i.e. no credit) are asked.

http://wug.physics.uiuc.edu/courses/ie.html

Assessment of Interactive Examples• What do students think of them?

– They love them!• “I really liked this web based exercise. It emphasizes the concepts as well as the mathematical approaches to physics problems. The help button is a great idea because if you are stuck on a problem, there is an option to

get some help without someone just telling you the answers.”• “…It's like having a personal TA to assist you with every problem when you get stuck.”• … Sometimes I don't know how to start a problem and end up asking friends how to do it. Thanks to a little help from this system, I can figure the questions out on my own. I think I learned more and it was more

satisfying to solve a problem…”– Why?? Some conjectures…

• Students like IEs because THE STUDENT IS IN CONTROL– Students can choose to ask for help and can abort the help sequence whenever they think they can answer the base question

• Students like IEs because IEs are GOAL-DRIVEN (answer the base question)– Base question -> Help questions provides hierarchy

• Do students learn more from IEs?

YES !!

Measuring Zero

0

10

20

30

40

50

60

70

80

δ

Co

un

t

Pre IE differences

Model

0 3-3

Effect of IE's

0

10

20

30

40

50

60

70

δ

Co

un

t

Post IE - Pre IE

Model

0 3-3

Post IEPre IEMeasuring Zero

0

10

20

30

40

50

60

70

80

δ

Co

un

t

Pre IE differences

Model

0 3-3

Effect of IE's

0

10

20

30

40

50

60

70

δ

Co

un

t

Post IE - Pre IE

Model

0 3-3

Post IEPre IE

Look at , the normalized difference between the average score on Homework B questions in different semesters

Concluding Thoughts• Most Important Consideration to Always Keep in Mind:

– Character issue: The Arrogance of Physicists• What makes effective instruction is largely

an empirical question.• Listen to students and Learn from others

– Cultural issue: “My” Course• Course is NOT just lectures • Progress comes from contributions of many

Main Obstacle to Change is the Faculty !!

Concluding Thoughts• Seven Tips for Managing Organizational Change

– You have to establish a darn good reason to change• Pressure (Engineering College, Dept Head)

– You have to create a compelling vision• Lemons to Lemonade.. Existing models, knowledge

– You need results – fast (???) Communicate, communicate, and communicate some more Build a strong committed, top management coalition People don’t resist their own ideas

• Committee of 8 met regularly for a year to generate the design• Respected regular faculty that became the core of original

implemention– Don’t KISS (Keep it Simple, Stupid) – Keep it Complex

• Large-scale complex change may be easier to accomplish than small-scale incremental change

– Why? Interconnections (largely cultural) of the system make incremental change difficult.. Need to break connections..

• Changed all aspects of course at once.. a leap of faith.. Just Do It !!• Energizing and Liberating Experience

– Provides environment that supports the huge amount of work necessary for initial implementation to get done!!

• New culture: Teaching Intro Physics can be enjoyable and does not have to be a big deal!!

How to Sustain Reform?• Very Important Question

– People who create the reform are usually not the same kind of people who enjoy making the trains run on time.

• Some Possible Answers– Establish Infrastructure

• People (veteran faculty, computing help, lecture, lab & secretarial support, new Assoc Head position)

• Computing (all materials on central server, easily accessed by all)• Welcome to 1XX, here’s how we do things….

– Establish Physics Education Research Group• Basis for continuing interest (not based on making trains run on

time)• Assessment of reforms• Allows for “continuous change”

That’s All Folks !

That’s All Folks!

Sample Physics 102 ACT

1v

v

3

Which loop has the greatest induced current at the instantshown above? 1 or 2 or 3 or all the same?

L

W

2

v

Three swimmers can swim equally fast relative to the water. They have a race to see who can swim across a river in the least time. Relative to the water, Beth (B) swims perpendicular to the flow, Ann (A) swims upstream, and Carly (C) swims downstream. Which swimmer wins the race?

Sample Preflight from Physics 101

A B C

A) Ann16%

B) Beth30%

C) Carly53%

While Carly is moving forward she will also be moving along with the current. two positive(+) direction motions = faster velocity.

Beth will reach the shore first because the vertical component of her velocity is greater than that of the other swimmers.

The shortest distant across is a straight line. Beth starts off straight but the current is taking her to the right so she has to swim longer to get across. Carly is already going to the right and plus the current so she would have to travel the farthest. Ann is swimming to the left and because the current is goin to the right it would push her into a straight line. So Ann would get there the fastest.