teaching a traditional optimization course in a non-traditional manner james b. orlin

1

Teaching a Traditional Optimization Course in a non-Traditional Manner

James B. OrlinMike Metzger

MIT

November 6, 2006

2

Part I: Introduction

Quote of the day.

If you treat individuals as they are they will remain as they are, but if you treat them as if they were what they ought to be and could be, they will become what they ought to be and could be. – Johann Wolfgang von Goethe.

3

Optimization Methods in Management Science

Models, applications, and algorithms for optimization within Management Science. – finance, manufacturing,

transportation, timetabling, games, operations, e-business, and more.

Linear programming, integer programming, network optimization, and decision analysis. Mike Trick

I’ve scheduled league gamesfor a major basketball

conference. There are an incredible number ofconstraints, and it is very hard to solve.

4

Topic

Introduction

Applications of Linear and Non-Linear Programming

Geometry of LP

Simplex Method 1

Simplex Method 2

Sensitivity Analysis

LP Duality 1

LP Duality 2

Game Theory

Introduction to Networks

Maximum Flows.

Min Cost Flows

The Network Simplex Algorithm

IP Models

More IP Models

Branch and Bound

Cutting Planes

Decision Trees 1

Decision Trees 2.

Syllabus

5

Our Goals Today

Lots of pedagogical advances over the years.– lot of description at INFORMS meetings

Focus on a range of techniques– visual learning, – active learning– interactive learning– use of technology– and more

End result: – students like the material better– more interested in follow on subjects– superior mastery of the material over students in

previous years

6

Overview of the Structure of the Class

2 Lectures per week per section (1.5 hours, required)– 1 small morning section, one large afternoon

section– MIT students are used to 1 hour per lecture

Required recitation – weekly quizzes

Nearly weekly homework

2 midterms and a final (and a project)

1 professor, 3 TAs. 150 students

7

Structure of a Lecture

Intro– quotes

– announcements

First half of class

Mental break

Second half of class

game to review the class material

8

Time line for 15.053

1995: Algorithm based

1996: taught 15.053 as an

Excel-based class

1999: taught 15.053 as an

algorithms based class

2001: introduced

PowerPoint into classes;

Mike Metzger was a

student in the class

9

Time line continued

2004: taught both sections of 15.053 in the spring. Mike Metzger was one of my TAs. Hired undergrad graders. TAs helped develop educational materials

2005: introduced required recitations.

2007. dropped required text and replaced with our own notes

CLASS

15.053 class notes

10

Part II: Pedagogy

Imagination is more important than knowledge...• Albert Einstein

You don't understand anything until you learn it more than one way.

• Marvin Minsky

11

Overview of Pedagogy

Pedagogy has evolved from straight lecturing

Techniques adopted– Engaging Model

Formulations

– Algorithm animation using PowerPoint

– Use of games in class

– Active learning (talk to your neighbor)

The Jetson’s diet problem.

Who wants a piece ofpiece of candy?

TV’s hit quiz show comes to 15.053

12

A diet problem and its dual

Jane Jetson is trying to design a diet for George on her home food dispenser so as to satisfy nutritional requirements and minimize cost.

Space Burgers

Saturn Shakes

Venus Flies

Requirements

Carbs 1 4 0 9

Fat 4 2 0 8

Protein 4 1 2 12

Cost (in $100s)

6 6 2

Theme music

13

Spacely Sprockets has decided to make food pellets. Mr. Spacely wants to win over Jane Jetson by making, carbohydrate, fat, and protein pellets. How should he price them to win over Jane and maximize profit?

Upper bound

Carbs Fat Protein on Cost

Space Burgers 1 4 4 6

Saturn Shakes 4 2 1 5.5

Venus Flies 0 0 2 2

Choose Prices

14

Discuss with partners

Usual:– left of the class solves Jane’s primal problem as

best as possible by inspection

– right half of the class solves Spacely’s dual problem by inspection

– we collect answers from both, and observe that the primal opt is >= dual opt.

Do you have any other ways of getting students to understand the intuition behind weak duality?– (2 minutes)

15

Algorithm Animation

Powerpoint has lots of animation tools that are useful for illustrating concepts.

Boxes can appear, and disappear.

Colors and transitions can simulate movement.

And one can use real movement.

And pictures can help.

16

2 Variable Simplex Illustrated

1 2 3 4 5 6

1

2

3

4

5

K

SStart at any feasible corner point.

Move to an adjacent corner point with better objective value.

Continue until no adjacent corner point has a better objective value.

Maximize z = 3 K + 5 S

17

An Alternative Representation

1 2 3 4 5 6

1

2

3

4

5

K

S Introducing:

LPac Man

18

Algorithm Animation: Simplex Pivoting

2 001

3 1 0

2 0 1

0

0

3 1 0

2 0 1

0

0

-2 001

=

= 2

6

x1 x2 x4x3z

= 0

1 0

0 1

00

x4x3

1

0

0

z

-3

-3

-4

-3

-4

3

x1

-3

-4

3

z = 0, x1 = 0, x2 = 0, x3 = 6, x4 = 2.

Non-basic variable x2 becomes basic.

Basic variable x4 becomes non-basic. It was the basic variable in Constraint 2.

Choose column 2.

Choose constraint 2.Pivot on the 2.

2

19

2

x1

-3 3 1 0

-4 0 1

Pivoting to obtain a better solution

=

= 2

6

2

x2 x4x3

3 -2 00

-z

0

0

1 = 0

-2 1 0 .5 1

-1 0 0 1 2

3 0 1 -1.5 3

New Solution: basic variables z, x2 and x3. Nonbasics: x1 and x4.

1

0

0

z

20

What is the maximum number of diamonds that can be packed on a Chinese checkerboard.

Each diamond covers 4 dots.

class exercise: how many can you pack?

21

Here is a best possible

22

23 4

1

5

76

89

10

11

12

For each circle, you can select at most one diamond incident to the circle.

5 slides later

23

An Excellent Integer Program

IP

( )

z = Max

s.t. 1 for each circle

{0,1} for all

dd D

dd S i

d

x

x i

x d D

Let S(i) be the diamonds that are incident with circle i of the Chinese Checkerboard.

zLP = 27.5

And we found a solution by hand with zi = 27

24

Representing the Dual Problem

The dual linear program: choose weights for each circle so that w(d) ≥ 1 for each diamond d. Minimize the weights of the circles.

Assign each circle (or node) a “weight.”

The weight of a diamond d is w(d) = the sum of its node weights.

Class exercise: assigning ¼ to each circle gives a total weight of 30.25. Can you do better?

25

Node Weight

1

1/2

1/3

1/6

0

Each diamond has weight at least 1.

18

1

24

6

27.5jjw

The number of diamonds is at most 27.

26

Who wants a piece of

CANDY?

TV’s hit quiz show comes to 15.053

2727

Who wants another piece of candy

50-50 15.053

A B

C D

What is the advantage of using algorithm animation?

It keeps students engaged

It helps students focuson what is important

It is effective for students who are visual thinkers

A, B, and C.

2828


50-50 15.053

A B

C D


It is effective for students who are visual thinkers

A, B, and C.

2929


50-50 15.053

A B

C D


Next question

A, B, and C.

3030


50-50 15.053

A B

C D


Next question

A, B, and C.

3131


50-50 15.053

A B

C D


Both A and B Neither A nor B

3 6

1 42iji j

x

3 9

1 42iji j

x

Which valid constraint should be added to the integer programming formulation of the TSP to eliminate this solution.

1

2 3

4

5 6

7

8 9

3232


50-50 15.053

A B

C D



1

2 3

4

5 6

7

8 9

3 9

1 42iji j

x

Both A and B

3333


50-50 15.053

A B

C D


Last Slide

3 9

1 42iji j

x


1

2 3

4

5 6

7

8 9

3434


50-50 15.053

A B

C D


Last Slide

3 9

1 42iji j

x


1

2 3

4

5 6

7

8 9

35

Mental Break: Impossibilities

Fooling around with alternating current is a waste of time. Nobody will use it, ever.

Thomas Edison

Rail travel at high speed is not possible because passengers, unable to breathe, would die of asphyxia.

Dr. Dionysus Lardner, 1793-1859

Inventions have long since reached their limit, and I see no hope for future improvements.

Julius Frontenus, 10 A. D.

36

Recitations, Tutorials, Homework, Exams, Etc.The role of the Head TA

37

Structure

Quiz Statistical Break Lesson Summary

39

Quiz Problem 1 (4 points total; 2 points each) If exactly one of these points is optimal, which

point(s) could it be? If point B is optimal, must any other of these

point(s) be optimal too? If so, which one(s)?

0 0.5 1 1.5 2 2.5 3 3.5 4 4.50

0.5

1

1.5

2

2.5

3

3.5

4

4.5

A

B

C

D

E

F

G

x1

x 2

40

Statistical Break

Break Between Quiz and Lesson– It takes 5 minutes for them refocus

• Interesting statistics!

Examples:– Favorite Airline

– Exam Scores

– Have You Heard of Welcome Back Kotter

41

Quiz Grades

13

18

65

00

2

4

6

8

10

12

14

16

18

20

10 9 8 7 <7

42

Have you heard of Welcome Back Kotter?

No, 38

Yes, 10

No. 90

Yes. 10

Have you heard of Welcome Back Kotter

43

Recitation-”The Dual” Consider the Following Statements Try to Determine if they Are True or False

– It is possible for an LP to have exactly 2 optimal solutions

– Two different bases (and thus two different tableaus) can have the same corresponding basic feasible solution

– We can have negative z-row coefficients and still be at an optimal solution.

44

Problem 1: More Simplex Tableau Suppose while solving a maximization problem we obtain the following tableau, with the basic variables highlighted in blue:

z x1 x2 x3 x4 x5 rhs 1 a c 0 3 0 10 0 0 d 0 4 1 5 0 1 -5 0 -1 0 b 0 0 e 1 2 0 3

Give conditions on the missing values a, b, c, d, e and required to make the each of the following statements true:

a) The current tableau represents a basic feasible solution in canonical form. b) The current basic solution is an optimal solution, but the set of optimal solutions

is unbounded. c) The current basic solution is a degenerate basic feasible solution. d) The current basic solution is feasible, but the objective function value can be

improved by bringing x2 into the basis and pivoting x3 out.

45

Pivot- Geometry

46

Applications

(Before)

Ebel Mining Company owns two different mines that produce a given kind of ore. After crushing, ore is one of three grades: high, medium or low. Ebel must provide the parent company with at least 12 tons of high grade, 8 tons of medium grade, and 24 tons of low-grade ore per week. It costs $20,000 a day to run the first mine and $16,000 a day to run the second mine. In one day each mine produces the following tonnage:

High Medium Low Mine 1 6 2 4 Mine 2 2 2 12

47

ApplicationsAfter.

Dear 15.053 Class, My name is Jessica Simpson. I have been going through some tough times recently and am having a real problem with one of my cosmetic lines. The info for the line is on the next page. Recently though costs are changing based on market demand in addition to highly fluctuating resource costs. My problem is this we currently have an LP that we solve to find the optimal amount to produce of each product. However, every time a parameter changes, I am always forced to resolve the LP and this takes too long. I was hoping you guys could find a better way. Lately I have just been out of it. For example, Nick and I decided to split our Hummer in half, and now I need to buy a new one. Ohh yea, about the LP it seems to have been misplaced when I was moving out of my Malibu house. Please Help!

- Jessica

48

Interaction

Video clip More about video and educational technology

later

49

Initial Concerns

Students bored in recitation Rebellion to required recitation

– Against MIT Grain

50

What We Found

Main Lesson– By putting out a good product, students didn’t

complain.

– If they are learning, they will be happy

– The Fun Factor helped out

– You can’t please everyone• But you can please 95%

51

Office Hours

Structure– Twice per week for two hours each

– Attendance can be as high as 60 Students

– One TA leads each session• Team teaching is a new approach

– Small vs. large session

– Goal: To help students understand the material on the homework sets

– Management style

52

ExampleAccording to recent reports in US Weekly, after

12 consecutive nights of partying, Jessica has

decided to leave Nick. Currently they are trying to

determine how to optimally divide some of their

possessions. They hire “you” to try and help

them accomplish this task. To start, you ask them

to assign “value points” to all of the items under

consideration such that they total 100. Their

allocation is shown in the table below.

53

The Use of Excel

The main question– How to balance Excel Vs. Theory

Students:– Have disliked Excel

– However, after completing our course often say it is the most useful tool we taught.

54

The Use of Excel

Staff– Hard to grade Excel

– Very important for Management students A new approach

– Take Excel and separate from problem sets and Use as a case study

– To be tried in spring 2007

55

Our Idea

The Diet Problem as an ongoing example and case study– Formulate your problem

– Solve using simplex

– Take dual and interpret as Grocer’s Problem

– Study distribution costs using networks

– Look into decision modeling with stochastic supply

56

Modeling and Abstract modeling

More emphasis on modeling abstract scenarios

Building managerial skills while building on theory

Framing the problem

Min cij xij

s.t. aij xj <= bi for all i

x >= 0

57

Exams-Testing the “Why” not “How”

JB was performing the simplex method; however, at exactly one intermediary step, he made the following error: instead of pivoting in a variable with a negative objective row coefficient, he instead pivoted in a variable a with a positive objective row coefficient. He chooses the exiting variable using the usual minimum ratio test. Assume after this initial error JB continues to solve the problem using the simplex method and does not make any additional errors. Also assume that no basis is degenerate.

58

Testing the “why” not “how”

It is possible that the pivot that JB took will result in a basic solution that is not feasible.

It is possible that the pivot JB took will result in an optimal basis.

It is possible that there are fewer simplex pivots with the error than if the error had not been made.

59

Exam Management

Testing depth of knowledge without penalizing

students who understand the basics

Our Solution: Short Answer Questions

– Say we have an LP with only one variable.

Which of the following could be the set of all

feasible solutions?

• i) A single point, e.g. 2

• ii) A line segment, e.g. [2, 4]

• iii) Two disjoint line segments, e.g, [2, 3] [6, 7].

• iv) A ray or infinite half line, e.g. [2, ∞)

60

Suppose we have a primal-dual pair where

each has a finite optimal solution. Say we

remove the ith constraint for the primal.

What can you say about the dual

corresponding to this new problem?

– i)The dual could be infeasible.

– ii)The dual could still have an optimum solution.

– iii)The dual could be unbounded.

61

Use of Technology

Three recent additions to the subject

– Online PowerPoint reviews

– Homework solution podcasts

– Online videos of recitations and reviews

62

Online PowerPoint Reviews

An alternative to standard textbook readings– Solution: Online Tutorials

Topics:– Abstraction of LP models– Solving systems of equations– Converting an LP to standard form– Degeneracy– Sensitivity analysis using Excel– Integer programming formulations

63

With the Assistance of

Some Friends

64

Degeneracy in Linear Programming

Tim, the turkey

I heard that today’s tutorial is all about Ellen DeGeneres

Ellen DeGeneres

Sorry, Tim. But the topic is just as interesting. It’s about degeneracy in Linear Programming.

Degeneracy? Students at MIT shouldn’t learn about degeneracy. And I heard that 15.053 students have already studied convicts’ sex.

Reverend Jerry Falwell

Actually, they studied convex sets.

65

What is degeneracy? As you know, the simplex algorithm starts at a corner point and moves

to an adjacent corner point by increasing the value of a non-basic variable xs with a negative value in the z-row (objective function).

Typically, the entering variable xs does increase in value, and the objective value z improves. But it is possible that that xs does not increase at all. It will happen when one of the RHS coefficients is 0.

In this case, the objective value and solution does not change, but there is an exiting variable. This situation is called degeneracy.

x1

-3 3 1 0

-4 2 0 1

=

= 2

6

x2 x4x3

3 -2 00

z

0

0

1 = 2

1

00

A basic feasible solution is called degenerate if one of its RHS coefficients (excluding the objective value) is 0.This bfs is degenerate.

66

Nooz, the most trusted name in fox.

Great. I now know what degeneracy is. Now we can move on to other matters.

Ollie, the computationally

wise owl.

No, Nooz**. This tutorial has many more slides. Degeneracy adds complications to the simplex algorithm. And if you understand what occurs under degeneracy, you really understand what is going on with the simplex algorithm.

** As you know, “No, Nooz” is good news.”

Cleaver

Professor Orlin apologizes for this bad pun, but feels that he could not resist the temptation.

67

PodCast’s

MIT Project Launched in Spring 2006– 15.053 selected as launch course for homework

solutions via podcasts

– Over 85% reported using videos on regular basis

– Turned HW Solutions into tutorials

– Review Session 2 Set the current MIT PodCast download record in a single night

• Crashed Server!

68

PodCast’s

Show Example

teaching a traditional optimization course in a non-traditional manner james b. orlin

Documents