Download - Linear Optimization – Spring 2021
Linear Optimization – Spring 2021
Course:
Information:
Class Time:
Instructor:
Office Hours:
Math 464 (CAPS)
see Blackboard
T 1:30–2:45 and Th 1:30–2:30 via Zoom Tom Asaki ([email protected])
T,Th 10–11 & W 3–4 via ZoomHomework Session: TBD:
Text: Introduction to Linear Optimization (1997)
by Dimitris Bertsimas and John Tsitsiklis
Contents
Course Description 2
Course Materials 2
Course Goals 2
Course Elements 3Participation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
Homework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
Project . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
Exams . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
Evaluation Scale 4
Accomodation, Integrity, Safety 4
Schedule and Due Dates 6
Project Details 7
Examples of Homework Solutions 14
Homework Problem Set 19
1
DescriptionThis course explores the theory and application of linear programming – a very important and
practical subfield of mathematical optimization. We will learn how mathematicians express opti-
mization problems, model real-world decision-making problems, understand and practice solution
methods, employ software for linear programs. This course is a mathematics capstone course
which is accessible to students from a variety of majors, provided that their mathematical back-
ground is solid. The course prerequisite is Math 273 or Math 283. However, completion of Math
301, Math 364 and/or Math 220/230/420 will be very helpful.
Course MaterialsIn this course, we will work through several chapters in the textbook Introduction to Linear Pro-gramming by Dimitris Bertsimas and John Tsitsiklis. This (1997) edition can be rented from
Amazon for about $40 or purchased for about $50–$90. Some homework assignments will be
completed using Matlab (or Octave) – this software is freely available to WSU students and can
also be accessed online at no cost (details will be provided when needed). You will also complete
the writeup of the course project using the online service Overleaf for typesetting documents
using LATEX. So, you will need productive internet access.
Course GoalsMath 464 is a Mathematics Capstone course. As such it is “Intended to be taken in the final year
of a student’s degree...” and “...serve as a culminating experience for students to demonstrate
achievement of the university’s undergraduate learning goals.” Some aspects of Capstone course
requirements include
• “demonstrate a depth of knowledge within their chosen academic field of study that inte-
grates its history, core methods, techniques, vocabulary, and unsolved problems.”
• “apply concepts from their general and specialized studies to personal, academic, service
learning, professional, and/or community activities.”
• “demonstrate how the methods and concepts of a chosen discipline relate to those of other
disciplines through engaging in cross-disciplinary activities.”
In this course, you will realize some or all of these aspects through homework, classwork, and
a project based on your modeling and assessment of a decision-making process from an actual
business of your choosing. In addition, this course serves as a venue for developing your proof
construction and writing skills. In particular, the successful student will demonstrate and/or gain
the following general skills.
1. Compose well-written and concise solutions to mathematical questions.
2. Compose a mathematical project report with length and content appropriate for your mathe-
matical peers.
3. Complete a (pedagogical) mathematical project based on your own initiative and interaction
with a business or industry representative.
2
This course also incorporates some specific mathematical knowledge and practice goals – some of
which are intended to assist your project completion. The content goals are as follows.
1. Model realistic decision-making problems as linear optimization problems.
2. Use software for solving linear programs.
3. Understand convexity of functions and sets and the usefulness of this property in mathemat-
ical optimization.
4. Understand the properties of polygons and the implications for linear optimization.
5. Understand the Simplex Method for solving linear programs.
6. Understand key concepts in interior point methods for solving optimization problems.
7. Understand key concepts in solving integer optimization problems.
Course ElementsParticipation. Class attendance and participation in group activities is expected. We will meet
synchronously through Zoom. Classtime on Thursdays will be shortened by 15 minutes. Because
of this lost time together, I am offering an (optional) problem session for discussing homework
problems as a group. The time for this session will be determined based on student interest and
time constraints.
Homework. Regular homework is intended to assist your study of various topics and extend
your knowledge. Homework solutions are not scratch-work nor do they consist only of answers
to assigned problems. Homework solutions should be examples of self-contained mathematical
documents exhibiting the following properties.
1. Good document construction and English grammar.
2. Conciseness and completeness.
3. Appropriate mathematical language, notation and discussion.
4. Explanations of the key ideas addressed by the question.
Your attention to these requirements is essential to your experience and can strongly affect your
grade. Collaboration among your fellow students is encouraged, you should not seek assistance
from online tutoring and problem solving sources (e.g. Chegg and CourseHero). You are very
much encouraged to seek homework assistance from me. Examples of good homework solutions
are included later as part of this syllabus.
All homework assignments are to be submitted electronically through Blackboard. Each sub-
mission must conform to the following criteria.
1. Each assignment should be submitted separately, even if due on the same day.
2. The format is to be PDF only.
3. Submitted files should not be larger than about 500 kB. There are good online and app
tools for reducing the size of PDF documents (e.g. https://pdfcompressor.com and the
CamScanner App).
3
4. Submitted work can be derived from pictures of hand-written notes or typeset documents
(i.e. LATEX or Word). There are good online tools for collecting pictures into PDF documents
(e.g. https://jpg2pdf.com).
Homework problems will be graded soon after the due date. No late homework will be ac-cepted. The lowest score of the first 20 homework problems will be dropped. The lowest score ofthe last four homework problems will be dropped.
Project. The course project involves several aspects of real-world implementation of mathemat-
ical optimization. Individual projects are expected, though a strong proposal for a small group
project will be considered. Any such group proposal should be submitted (or presented) by Tues-
day, January 26.
A project will include several elements (details are provided later in the syllabus): initiating
contact with a company or institution, conversing with company personnel regarding potential and
relevant mathematical optimization problems, modeling their question or task, demonstrating a
solution to the problem, and writing a report of the process and results. The goal is to have a
consulting-like experience in which your mathematical skills play an integral part.
Project elements will be graded soon after they are submitted. The late submission penalty is50%.
Exams. There will be no exams for this course. The knowledge and skill typically probed by
exams will be manifested and measured in homework and the project.
Evaluation ScaleA course percentage will be determined by a weighted combination of the course requirement
elements: participation (10%), homework (50%), project (40%). The course grade will then be no
lower than provided by the following percentage scale:
Course Percent 90 87 80 77 70 60 <60
Course Grade A B+ B C+ C D F
WSU Reasonable AccommodationStudents with Disabilities: Reasonable accommodations are available for students with a docu-
mented disability. If you have a disability and need accommodations to fully participate in this
class, please either visit or call the Access Center to schedule an appointment with an Access
Advisor. All accommodations MUST be approved through the Access Center.
Academic IntegrityAcademic integrity is the cornerstone of higher education. As such, all members of the university
community share responsibility for maintaining and promoting the principles of integrity in all
activities, including academic integrity and honest scholarship. Academic integrity will be strongly
enforced in this course. Students who violate WSU’s Academic Integrity Policy (identified in
4
Washington Administrative Code (WAC) 504-26-010(3) and -404) will receive a failing grade on
any relevant assignment (and possbily for the course), will not have the option to withdraw from
the course pending an appeal, and will be reported to the Office of Student Conduct.
Cheating includes, but is not limited to, plagiarism and unauthorized collaboration as defined
in the Standards of Conduct for Students, WAC 504-26-010(3). You need to read and understand
all of the definitions of cheating. If you have any questions about what is and is not allowed in this
course, you should ask course instructor before proceeding.
If you wish to appeal a faculty member’s decision relating to academic integrity, please use the
form available at conduct.wsu.edu.
Safety and EmergencyClassroom and campus safety are of paramount importance at Washington State University, and
are the shared responsibility of the entire campus population. WSU urges students to follow the
“Alert, Assess, Act,” protocol for all types of emergencies and the “Run, Hide, Fight” response for
an active shooter incident. Remain ALERT (through direct observation or emergency notification),
ASSESS your specific situation, and ACT in the most appropriate way to assure your own safety
(and the safety of others if you are able).
Please sign up for emergency alerts on your account at MyWSU. For more information on this
subject, campus safety, and related topics, please view the FBI’s Run, Hide, Fight video and visit
the WSU safety portal.
5
ScheduleThe following table shows the classroom topics, homework due dates and project status due dates.
Classroom topics and days are tentative and subject to some change. Homework assignments and
project updates are due at the beginning of class on the stated day.
Date Class Topics Homework Project
Tu Jan 19 Introduction/Notation
Th Jan 21 LP Examples Overleaf Document Link
Tu Jan 26 LP Examples 1,2
Th Jan 28 LP Examples 3 Initial Contact List
Tu Feb 2 LP Solution Concepts 4
Th Feb 4 Convexity/ Polyhedra 5 Initial Contact Plan
Tu Feb 9 Basic Feasible Solutions 6
Th Feb 11 Standard Form Polyhedra 7,8
Tu Feb 16 9
Th Feb 18 Geometric Theorems 10
Tu Feb 23 Optimality and Simplex Method 11
Th Feb 25 NO CLASS 12
Tu Mar 2 Simplex Method Examples 13
Th Mar 4 Simplex Method Examples 14,15
Tu Mar 9 Big-M and Two-Step Methods 16 Initial Meeting Synopsis
Th Mar 11 Cycling and Degeneracy 17
Tu Mar 16 Duality 18 Problem Statement
Th Mar 18 Duality 19
Tu Mar 23 LP Solution Efficiency 20
Th Mar 25 Ellipsoid Method Concept
Tu Mar 30 Ellipsoid Method Optimization Model
Th Apr 1 Interior Point Method Concepts
Tu Apr 6 Affine Scaling 21
Th Apr 8 Barrier Methods Initial Solution
Tu Apr 13 NO CLASS 22
Th Apr 15 Primal-Dual Path Following
Tu Apr 20 Integer Programming Concepts
Th Apr 22 Cutting Planes Methods 23
Tu Apr 27 Branch and Bound Methods
Th Apr 29 24
Tu May 4 NO CLASS (finals week) Full Report
6
Project DetailsYour course project does involve interaction with an outside company or other professional insti-
tution. At every juncture, you should be very transparent with your contact about the following
points:
1. The major goal of this project is to provide you (the student) with a consulting-like experi-
ence.
2. This project is not to be construed as any form of job interview, internship or employment of
any kind.
3. Your project must not be mission-critical to the contact. You are not to promise, nor can you
guarantee, any particular result.
4. Be sure and express your gratitude to the contact at the close of the project. They receive no
particular benefit from this interaction, but they can serve as a key part of your education.
All project updates are to be recorded in a single document created using the online LATEX type-
setting service at overleaf.com. If you are not familiar with basic LATEX, then please let me know.
Each element has a specific due date given in the schedule.
Overleaf Document Creation [5%]
The specific procedures for creating the report in Overleaf are given below. You should complete
these steps. No additional action is needed on your part.
1. Download the Project template file (OptimizationProject.tex) and the figure speed.png
the Project information folder in Blackboard.
2. Navigate to https://www.overleaf.com.
3. Set up a free account and log in.
4. At left, click on “New Project.”
5. Select “Blank Project” and name your project.
6. Right click the file main.tex and delete it.
7. Use the “upload” icon in the upper left to upload OptimizationProject.tex and speed.png
from your computer.
8. Click on the uploaded file in the file list at left.
9. Click on the “Recompile” button at top-center.
After following these instructions, the formatted project document should appear in the window at
right. You do not need to supply a paper or electronic copy of this document to me at any time.
Instead you will give me read/write access to the document so that I can supply feedback online.
To give me access, do the following.
1. Click the “Share” icon at upper right.
2. Turn on “Link Share.”
7
3. Copy the edit link to the document.
4. Paste this link as your submission to the Project Assignment in Blackboard.
I will now be able to use this link to provide comments and feedback within your project document.
I will not reformat or formally edit your project document.
Contact List [5%]
Provide an initial list of several potential contacts for mathematical optimization problems. Do notinitiate contact yet! Examples of good contacts are businesses, government agencies, non-profit
organizations, academic offices. Business contacts for this project are typically not individuals.
Keep the following general criteria in mind.
1. Smaller businesses and offices are more likely to be willing to meet quickly.
2. Having several options is important because not each of your choices will be willing to assist
in your project.
3. Focus your choices according to your interests.
4. Local businesses and offices are easier to work with in terms of personal communication.
In your report briefly discuss the reasons for your choices and conjecture some types of optimiza-
tion problems which the client may have. Do not initiate contact yet!
Initial Contact Plan [5%]
Begin this step only after your Contact List has been approved.
Devise an initial contact plan based on your contact list. Your plan should include a strategy for
working through your list and examples of email introductions. Some considerations:
1. Initial contact by phone or in person is not prefered in this case because of the anticipated
length of conversation.
2. Contacting one or two at a time to gauge interest and willingness.
3. Begin with your top choices.
4. Attempt to direct your email/call appropriately.
5. Be clear about your project needs.
6. Do not offer mathematical help.
7. Do not ask to see data.
8. Be clear about the course and instructor.
9. Ask for 30 minutes of time or less.
10. Give specific potential meeting times.
8
Special Note: When formulating your contact plan, please follow all state, local and WSU
COVID-19 restrictions. This may mean contacting the client entirely through remote con-
ferencing and email.
Here is an example initial contact email:
Dear Dr. Sarah Blastik,
I understand that you are the scheduling manager for Opticorp. I am a
student in Dr. Tom Asaki’s Mathematical Optimization Course at
Washington State University. As part of my class project, I am
researching how mathematics can be used to assist decision-making
processes in business situations. One of my interests is in scheduling
events and personnel.
It would very helpful if we could have a short meeting (20-30 minutes)
to discuss scheduling decisions related to companies such as yours and
what challenges you face. I could meet next week any day after 3:00 or
on Monday or Wednesday mornings before 10:00.
Please let me know. If you have any question, please feel free to
contact my professor at [email protected].
Sincerely,
I.M. Student
Here is another example email:
Sam Sneed,
I a student in a Mathematical Optimization course at WSU. As part of
my course project, I am considering modeling how product ordering is
handled based on inventory, suppliers, consumer demand, etc. I was
hoping that you, as the inventory manager, would have some time to help
me understand more about the actual process details. I could learn much
in a short 20-30 minute meeting.
I could meet with you any time Monday or Friday. I am also free some
afternoons.
Please let me know if you can help, and please feel free to contact my
professor at [email protected] if you have any questions.
I.M. Student
9
Initial Meeting Synopsis [15%]
Meet with your contacts as soon as possible after your contact plan has been approved. Here are
some things to keep in mind:
1. You can meet by phone conversation, but you can lose context and detail. Some times there
is no substitute for experiencing the work place. (See special COVID-19 note above.)
2. Try and be flexible with scheduling. But, don’t compromise your other obligations unneces-
sarily.
3. Aim for a first meeting as soon as possible. Remember that the meeting synopsis also has a
due date.
4. Do not be late for a meeting.
5. Be prepared with questions. Take notes. Listen.
6. As you meet, think about potential optimization problems.
7. Be willing to change the expected direction of your conversation.
8. Request that you can ask follow-up questions at a later date.
Being prepared with good questions is very important. Keep in mind that as you formulate an
optimization problem you will need to understand your decision variables, your objective function,
and your constraints. You should not use this language in your meeting. Instead, consider questions
such as the following (You will need to tailor these to your specific needs).
1. How many different products do you manufacture?
2. What specific choices do you make? How often?
3. Can be manufactured in any quantity or in specific batch quantities?
4. How do costs change with specific strategies?
5. How is profit affected by ?
6. How do you decide which is a better strategy, or ?
7. Do you consider it better to or ?
8. How does one choice affect another?
9. Are there limits on the number of you can make?
10. Can you have both and ?
11. Are some limitations soft or are they all very strict?
It can be difficult to predict what questions you should ask, but you should come as prepared as
possible. Feel free to ask for additional guidance as you think about questions.
Your synopsis should summarize your experiences and conversations with people with whom you
actually met. You can briefly mention failed contacts if you wish. You can outline some initial
thoughts on potential optimization problems.
10
Problem Statement [10%]
Now that your initial meeting(s) is complete, you can write up a formal problem statement. This
statement is from the perspective of the client. It should resemble a (possibly complex) word
problem, similar to our class examples. It should exhibit good grammar, complete details and a
specific optimization goal.
You may or may not have specific data to work with – but in either case, it should not be
included here. Instead, given data should be represented as parameter vector quantities. For ex-
ample, the manufacturing cost vector for n production items can be represented as d ∈ Rn with
entries d1, d2, · · · , dn. Specific values are not needed in the problem statement, except to say that dis given. Consider the following small examples.
A well-posed (project) problem statement:Pots-R-Us obtains its outdoor decorative pots from r different suppliers, each of which sells
pots in batches containing a distribution of s different sizes. Supplier i sells pots in batches
that contain bi j pots of size j at a cost of pi dollars per batch. Each quarter, Pots-R-Us
places an order with each supplier. dj pots of size j must be purchased in total in order to
meet consumer demand. At most, mi batches can be purchased from supplier i. Determine a
cost-minimizing strategy for obtaining pots that meet consumer demand in one quarter.
A poorly-posed (project) problem statement:Pots-R-Us obtains its stock of outdoor decorative pots from three suppliers. Each supplier
sells pots in four sizes (large, medium, small and tiny) in batches of 20 pots. The table shows
the distrubution of pots in each batch.
Supplier Batch Cost Large Medium Small Tiny
1 $80 7 8 3 2
2 $65 6 5 7 2
3 $45 2 2 10 6
Each quarter, Pots-R-Us places and order with each supplier. At least 100 large, 60 medium,
50 small and 40 tiny pots must be purchased each quarter in order to meet demand. Because
the supplier facilities have limited production, at most 9 batches can be purchased from any
supplier in any quarter. Formulate an integer program that can be used to minimize the cost
of acquiring the needed pots for one quarter.
Optimization Model [10%]
Now that your problem statement is well-formulated, you can provide an optimization model
whose solution answers the problem statement. You should explain all aspects of your modeling
process and justify every model item. Your model should be expressed as compactly as possible –
probably in matrix form. Specific values for problem variables (e.g. c, A, b) are not needed, but
should be expressed in terms of parameter variables. In this sense, the model is not specific to any
particular detailed scenario you have discussed with your contact. Instead, your contact’s question
is one example of a problem your model can solve. The Pots-R-Us problem given in the previous
11
subsection has an optimization model as follows (where xi is the number of batches purchased
from supplier i).
min z =r∑
i=1
pixi
s.t
r∑
i=1
bi jxi ≥ dj j = 1, 2, . . . s
0 ≤ xi ≤ mi i = 1, 2, . . . , rx ∈ Zr
Notice that specific values are not given, but remain as parameters of the problem.
Initial Solution [15%]
Now that your model is complete, you can work on answering specific questions. If you have data
provided by your contact, then you can answer this specific question. If you do not have data,
then you can create “synthetic data” (data which is realistic and tests the ability of your analysis to
provide a clear answer). As an example of a specific model, consider the given scenario data for
the Pots-R-Us example. This leads to the optimization problem (where r = 3 and s = 4):
min z = 80x1 + 65x2 + 45x3
s.t 7x1 + 6x2 + 2x3 ≥ 100
8x1 + 5x2 + 2x3 ≥ 60
3x1 + 7x2 + 10x3 ≥ 50
2x1 + 2x2 + 6x3 ≥ 40
0 ≤ xi ≤ 9 i = 1, 2, 3
x ∈ Z3
The solution for this scenario is x∗ =[6 9 2
]and z∗ = 1155.
As part of your project, you will likely have several possible scenarios. It may be necessary to ask
clarifying questions of your contact and modify your model accordingly. This is quite typical in
applied optimization. If such changes are necessary, feel free to return to previous sections and
modify accordingly. However, this portion of the project is complete when you demonstrate a first
solution.
A brief discussion should be included which demonstrates that you understand the results and some
of the implications for the contact. That is, relate your numerical results to the Problem Statement
and give some discussion from both a mathematical and business/industry perspective.
Report [40%]
The final report will include any additional model modifications and solutions based on your initial
findings and any other discussions with your contact – also with appropriate discussion.
12
The final aspect of the Project Report should also include a personal reflection statement. This
is a key piece of the report and of the course. Your response should be significant, truthful and not
superficial. You should write in your own style, following any reasonable format. As you write
your reflection keep in mind the following questions which I will be asking myself as I read.
1. Does this student use correct mathematical and English grammar?
2. What went well for this student? What not so well?
3. What might this student do differently in a similar future situation?
4. Does the student fully understand their model and why it will answer the problem statement?
5. Can the student explain any unusual/unexpected solution results?
6. Was this student effective at communicating with the contact?
7. Was this experience valuable for the student?
8. How comfortable was the student in completing the assigned tasks?
13
Homework Assignment Example SolutionTom AsakiMath464
Problem. Find the maximal value of the continuous function f (x) = e−x2
on the interval [−1, 1].
Solution. The maximal value of a continuous function on a closed interval occurs either at an
end point or at a stationary point. At the end points,
f (−1) = f (1) = e−1.
The stationary points are solutions of f ′(x) = 0. If, in addition, f ′′(x) < 0, then the point is a local
maximal point. We find,
f ′(x) = −2xe−x2
and f ′′(x) = (4x2 − 2)e−x2
.
The only stationary point is x∗ = 0 and f ′′(0) < 0, so x∗ = 0 is a local maximal point with
f (x∗) = 1. As f (x∗) > f (−1) = f (1), the maximal value of f (x) is 1.
Note the following characteristics of the solution.
1. The problem statement is included.
2. The text is not mathematical scratchwork. Instead it flows with proper grammar.
3. The solution is concise, without unnecessary algebra, explanations or repetition.
4. The approach to the problem is included early in the solution.
5. Explanations are given for key problem concepts (e.g. stationary points).
6. Explanations are excluded for pre-requisite concepts (e.g. derivatives, algebra).
7. The conclusion responds to the problem statement.
14
Homework Assignment Example SolutionTom AsakiMath464
Problem. Sketch the set Ω ={(x, y) ∈ R2 | x + y ≤ 4, x ≥ 1, y ≥ 0
}.
Solution. The set Ω is the set of all points in R2 satisfying all three inequalities. Each inequality
defines a half-space and the intersection is Ω. The following sketch shows the half-space boundary
lines and Ω as the triangular region.
y
1
2
3
4
x1 2 3 4
x=
1
y = 0
x+
y=
4Ω
Note the following characteristics of the solution.
1. The problem statement is included.
2. The text is not mathematical scratchwork. Instead it flows with proper grammar.
3. The solution is concise, without unnecessary explanations or repetition.
4. The approach to the problem is included.
5. Clarifying explanations and graph features are included.
15
Homework Assignment Example SolutionTom AsakiMath464
Problem. You decide to travel to Seattle to sell your remaining inventory of car parts. The
weight, number of items in stock, and profit per item are listed in the table below. Airline regula-
tions limit the shipment weight to a maximum of 214 pounds. Formulate an optimization problem
that can be used to determine the shipment details that will maximize profit.
Item # description weight (lbs) stock profit ($)
1 engine 104 2 151
2 transmission 62 4 68
3 seat 28 4 44
4 water pump 21 7 29
Solution. Let xk (x ∈ Z4) be the number of items of type k to ship. The objective is to maximize
the profit function
z = f (x) = 151x1 + 68x2 + 44x3 + 29x4.
There is a constraint on shipment weight that it be no greater than 214 pounds:
104x1 + 62x2 + 28x3 + 21x4 ≤ 214.
There are also stock constraints (limits on the quantity of each item that can be shipped):
x ≥ 0, x1 ≤ 2, x2 ≤ 4, x3 ≤ 4, x4 ≤ 7.
We have the optimization model:
max z = 151x1 + 68x2 + 44x3 + 29x4
s.t. 104x1 + 62x2 + 28x3 + 21x4 ≤ 214
0 ≤ x1 ≤ 2
0 ≤ x2 ≤ 4
0 ≤ x3 ≤ 4
0 ≤ x4 ≤ 7
x ∈ Z4
Note the following characteristics of the solution.
1. The problem statement is included.
2. The text is not mathematical scratchwork. Instead it flows with proper grammar.
3. The solution is concise, without unnecessary explanations or repetition.
4. The solution is consolidated into a clear answer.
5. Note: for more complex constraints, clarifying explanations may be needed.
16
Homework Assignment Example SolutionTom AsakiMath464
Definition. A function f : R → R is said to be convex if a f (x) + (1 − a) f (y) ≥ f (ax + (1 − a)y)
for all x, y ∈ R and for all a ∈ [0, 1].
Problem. Suppose f : R → R is continuously differentiable. Prove that f is convex if and only
if f (y) ≥ f (x) + (y − x) f ′(x) for all y, z ∈ R.
Solution.
Proof. (⇒) Suppose f : R → R is continuously differentiable and convex. For any x, y ∈ R and
a ∈ [0, 1],
f (x + a(y − x)) = f ((1 − a)x + ay) ≤ a f (y) + (1 − a) f (x),
or equivalently,f (x + a(y − x)) − f (x)
a≤ f (y) − f (x).
Taking the limit of the above inequality as a→ 0, we have
(y − x) f ′(x) ≤ f (y) − f (x), or f (y) ≥ f (x) + (y − x) f ′(x).
(⇐) Suppose f : R → R is continuously differentiable and f (y) ≥ f (x) + (y − x) f ′(x) for all
x, y ∈ R. Let x = ar + (1 − a)s for r, s ∈ R and a ∈ [0, 1]. We have
f (r) ≥ f (x) + (r − x) f ′(x) and f (s) ≥ f (x) + (s − x) f ′(x)
from which we find
a f (r) + (1 − a) f (s) ≥ a f (x) + a(r − x) f ′(x) + (1 − a) f (x) + (1 − a)(s − x) f ′(x)
= f (x) + [a(r − x) + (1 − a)(s − x)] f ′(x)
= f (x) + [ar + (1 − a)s − x] f ′(x)
= f (x)
= f (ar + (1 − a)s).
That is, f is convex. �
Note the following characteristics of the solution.
1. The problem statement is included.
2. The text is not mathematical scratchwork. Instead it flows with proper grammar.
3. The solution is concise, without unnecessary explanations or repetition.
4. The proof is formally composed.
5. Explanations are minimal and sufficient.
17
Homework Assignment Example SolutionTom AsakiMath464
Problem. Suppose u, v ∈ Rn with ‖v‖ = 1 and t ∈ [0,∞). Show that the ray R = {x ∈ Rn | x = u + vt}is a polygon.
Solution. A set in Rn is a polygon if it can be written as the intersection of a finite number of
halfspaces (non-strict linear inequalities). Let {a1, a2, . . . an−1} be any set of linearly independent
unit vectors in Rn which are orthogonal to v. Then
L = {x ∈ Rn | x · ak ≥ ak · u, x · ak ≤ ak · u, k = 1, 2, . . . , n − 1}is the (polygon) line containing R. We need only add one additional inequality to limit the set to
the ray. For example,
R = {x ∈ Rn | x · v ≥ v · u, x · ak ≥ ak · u, x · ak ≤ ak · u, k = 1, 2, . . . , n − 1} .Notice that the ray R is represented as the intersection of 2n − 1 halfspaces and is thus a polygon.
Note the following characteristics of the solution.
1. The problem statement is included.
2. The text is not mathematical scratchwork. Instead it flows with proper grammar.
3. The solution is concise, without unnecessary explanations or repetition.
4. A formal proof is not appropriate when asked to show/demonstrate/justify a claim.
18
Homework Problems1. A chemical company manufactures two products from four raw materials. The company can
use three different batch processes in any combination to meet consumer demand. The table
below shows, for each process, the required raw materials, the amount of output products
and the manufacturing cost. The table also includes the consumer demand for each product.
Formulate an integer program that can be used to find the processes to employ that minimizes
cost to the company while meeting consumer demand. The available raw material quantities
are 125, 175, 225 and 150, respectively.
Process 1 Process 2 Process 3 Consumer Demand
Raw Material 1 1 2 0
Raw Material 2 2 2 1
Raw Material 3 4 1 2
Raw Material 4 0 3 4
Product 1 3 2 1 217
Product 2 1 3 1 167
Cost $44 $32 $29
2. A company produces two different products, say item #1 and item #2. Each of item #1
requires 14
hours of assembly labor, 18
hour of testing labor and $1.20 worth of raw materials.
Each of item #2 requires 13
hours of assembly labor, 13
hour of testing labor and $0.90 worth
of raw materials. The company currently has 90 hours of assembly labor and 80 hours of
testing labor available each day. The products have market values of $9 and $8, respectively.
Formulate an integer program that can be used to maximize the daily profit of the company.
Next, suppose that the company can schedule up to 50 hours of overtime assembly labor at
an additional cost of $7 per hour. Provide a modified integer program.
3. (From “The Basics of Practical Optimization” by Adam B. Levy) A violin maker expects the
following demands for violins: 5 violins in the first quarter, 4 violins in the second quarter, 4
violins in the third quarter, 3 violins in the fourth quarter. There is an inventory of 6 violins at
the beginning of the first quarter. During each quarter, the violin maker can produce 2 violins
at a cost of $200 per violin, and can use overtime employees to make additional violins if
necessary, but at a cost of $300 per violin.
Write an optimization model that can be used to answer the following question: If, at the end
of each quarter, a holding cost of $25 per violin is incurred for violins not sold, how should
the violin maker schedule production in order to minimize the sum of the production and
inventory costs during the four quarters while meeting the expected demand?
4. Consider the polyhedron P = {x ∈ Rn | Ax ≤ b}. Construct a linear program that can be used
to find the center z ∈ Rn and radius r ∈ R of the largest ball contained in P.
5. Marble Construction is remodeling an office building. There are 12 tasks which cannot be
completed simultaneously as some tasks can not be started until other tasks have been com-
pleted. The table shows each task, the time required to complete the task, and the tasks which
19
must be completed before starting the task. Suppose that tasks can be worked concurrently
as long as tasks dependencies are met. Formulate a linear or integer program that can be used
to find the minimum duration of the remodel and the earliest time each task can be started.
task # task time (days) prior tasks
1 Remove old furniture 12 (none)
2 Wall remodel 8 1
3 New duct-work 14 2,4
4 New elevators 21 1
5 Upgrade fire escapes 6 (none)
6 Wiring 15 3
7 Lighting 9 2,3,6
8 Windows 6 2,5
9 Landscaping 12 5,8,11,12
10 Entryway 7 12
11 Exterior painting 8 5,8
12 Install new furniture 23 4,6,7
6. In this exercise, you will solve the Marble Construction problem using a software solver.
(a) Obtain access to Matlab or Octave software. There are many options:
• WSU students have access to a student version of Matlab which can be down-
loaded onto a personal computer.
• Octave can be downloaded on a personal computer (no cost).
• Sign up for a free account at octave-online.net and complete exercises online.
(b) Complete the tutorial exercises given in the document MatlabOctaveTutorial.pdf
for either Matlab or Octave.
(c) Use either Matlab or Octave to solve the Marble Construction problem.
7. Let f : Rn → R be a convex function and let c ∈ R.
(a) Prove that the set X = {x ∈ Rn | f (x) ≤ c} is convex.
(b) Show using two counterexamples, that if f is not convex, then X may or may not be
convex.
8. Prove that the intersection of a finite number of polyhedra in Rn is a polyhedron in Rn.
9. For each of the following sets, determine whether it is a polyhedron. Carefully justify your
answers.
(a) U = ∅ (empty set).
(b) V ={(x, y) ∈ R2 | x ≥ 1, y ≥ 2
}.
(c) W ={x ∈ R2 | x2
1 + x22 ≤ 1, |x| + |y| ≤ 1
}.
(d) The set of all optimal solutions to a linear program.
20
(e) X ={x ∈ R2 | x2
1 + x22 ≤ 1
}.
10. Consider the set
{x ∈ Rn | x1 = x2 = · · · = xm = 1, 0 ≤ xm+1, · · · , xn ≤ 1}.Could this set be the feasible region of a standard form linear program?
11. Consider the standard form polyhedron P = {x ∈ Rn | Ax = b, x ≥ 0}, where A is m × n with
linearly independent rows. Provide a clear counterexample for each of the following false
claims.
(a) The set of all optimal solutions is bounded.
(b) At every optimal solution, no more than m variables can be positive.
(c) Consider the problem of minimizing f (x) = max{cTx, dTx} over the set P. If this prob-
lem has an optimal solution, then it must have an optimal solution which is an extreme
point of P.
12. (Exercise 2.9) Consider the standard form polyhedron P = {x ∈ Rn | Ax = b, x ≥ 0}, where
the rows of A are linearly independent.
(a) Suppose that two different bases lead to the same basic solution. Show that the basic
solution is degenerate.
(b) Consider a degenerate basic solution. Is it true that it corresponds to two distinct bases?
Prove or give a counterexample.
(c) Suppose that a basic solution is degenerate. Is it true that there exists an adjacent basic
solution which is degenerate? Prove or give a counterexample.
13. Suppose x∗ is an optimal solution to a given linear program
min cTx s.t. Ax = b, x ≥ 0, x ∈ Rn.
Prove that if Cx∗ = d, then x∗ is an optimal solution to the linear program
min cTx s.t. Ax = b, Cx = d, x ≥ 0, x ∈ Rn.
14. Consider the standard form polyhedron P = {x ∈ Rn | Ax = b, x ≥ 0} and y ∈ P. Prove that
d ∈ Rn is a feasible direction at y if and only if Ad = 0 and di ≥ 0 whenever yi = 0.
15. Let P ={x ∈ R3 | 4x1 + 2x2 + 3x3 = 12, x ≥ 0
}. Parametrically describe the set of feasible
directions at w = (3, 0, 0)T, at x = (2, 2, 0)T, and at y = (1, 1, 2)T.
16. Consider the linear program
maxx
z = 2x1 + x2
s.t. x1 − x2 ≤ 2
x1 + x2 ≤ 6
x ≥ 0
x ∈ R2
21
(a) Convert the LP into standard form and construct the basic feasible solution for which
(x1, x2) = (0, 0).
(b) Carry out the full tableau implementation of the Simplex Method, starting with the
basic feasible solution of part (a).
(c) Sketch a graphical representation of the problem in terms of the original variables x1
and x2, and indicate the path taken by the Simplex Algorithm.
17. Solve the following optimization problem using the Simplex Method.
maxx
z = x1 + x2 + x3
s.t. x1 + x2 − 2x3 ≤ 2
x1 + x2 − x3 ≤ 3
− x1 + 2x2 + x3 ≤ 4
x ≥ 0
x ∈ R3
18. Solve the following optimization problem using the Simplex Method.
maxx
z = 2x1 + 3x2
s.t. 3x1 + 4x2 ≤ 12
4x1 + 3x2 ≤ 12
− x1 + x2 ≤ 3
x1 − x2 ≤ 3
x ≥ 0
x ∈ R2
19. Solve the following optimization problem using the Two-Phase Simplex Method.
maxx
x1 − x2 + x3
s.t x1 + x2 + x3 ≤ 2
2x1 + 2x2 − 6x3 ≥ 6
x1 − x2 − x3 ≥ 0
x ≥ 0
x ∈ R3
22
20. Solve the following linear program using the Two-Phase Simplex Method.
maxx
z = 2x1 + 3x2 + 3x3 + x4 − 2x5
s.t. x1 + 3x2 + 4x4 + x5 = 2
x1 + 2x2 − 3x4 + x5 = 2
− x1 − 4x2 + 3x3 = 1
x ≥ 0
x ∈ R5
21. Consider the Ellipsoid Interior Point Method for solving linear programs. It would seem that
a sequence of interior spheroids would simplify the computational task of finding the optimal
point on the boundary of the spheroid. Carefully explain why such a sequence cannot satisfy
all of the key requirements of this type of interior point method.
22. In this exercise you will need access to Matlab or Octave software. You will run provided
code to solve linear programs using the Sliding Objective Ellipsoid Method.
(a) Revist your Octave/Matlab situation and make sure that you can run code. You can use
the WSU-supplied Matlab or Octave, octave-online, or any other method which you
have available.
(b) Download the following files from the course website: EM.m, SOEM.m, SOEMtest.m
into a working folder for this assignment. The function EM.m performs the Ellipsoid
Method for determining if a given polyhedron is empty or not. If not, it returns a
feasible point. The function SOEM.m performs the Sliding Objective Ellipsoid Method
for solving a given LP; it makes use of repeated calls to function EM.m. The function
SOEMtest.m is a script file that illustrates how to make a call to function SOEM.m.
(c) At the Matlab/Octave prompt, type SOEMtest to run the script file and solve the test
problem. Read the script file and make sure that you understand the details. Sketch the
feasible region. Compare your solution to the true solution and discuss your findings.
(d) Using the example script as a model, write a script to solve the (LP relaxation) A-1 Fur-
niture Problem (see ModelingExamples.pdf found on the course website). Compare
your solution to the IP solution and discuss your findings.
(e) Create a linear program in two variables that has multiple optimal points and for which
the feasible region is bounded. Write a script file to solve your LP using the Ellipsoid
Method. Solve your LP using your script file and discuss your result.
23. Respond to any two of the following questions/exercises. You may turn in your solutions as
a single document.
(a) Consider the Affine Scaling Algorithm discusses in class and in Section 9.1 of the
textbook. This algorithm relies on solving a sequence of subproblems which are easier
to solve than the original LP. Each subproblem considers the same linear objective
function, but restricts the feasible region to that encompassed by an ellipse which is
23
interior to the LP polyhedron. The solutions to the subproblems provide a sequence of
feasible points whose limit point is an optimal solution to the given LP. Now, consider
the simple modification of having an interior spheroid as the feasible region of each
subproblem. With this modification, show that, in general, the sequence of optimal
points will no longer have a limit point which is an optimal solution to the given LP.
(b) Consider the optimization problem of minimizing a function f (x) over a standard from
polyhedron. When f (x) is linear, one method of solution is to employ the Primal Path
Following Algorithm (discussed in class and in Section 9.4 of the textbook). Now, sup-
pose f (x) is not linear, but is twice continuously differentiable. Can the same algorithm
still be used to solve this new problem? Carefully discuss some possible outcomes.
(c) In this exercise, you will use the Primal Path Following Algorithm to solve the A-1 Fur-
niture Problem. First, download the relevant code from the course website (Ppath.m,
showsol.m, PpathScript.m, sffr.m). Using the problem model presented in the
modeling examples found in ModelingExamples.pdf, convert the LP into Standard
Form. Next, find an interior point x0 which can be used to start the algorithm – describe
your procedure for finding such a point. Finally, solve the problem by modifying and
running PpathScript.m.
24. Consider the integer program
max z = 6x1 + 5x2
s.t − x1 + 3x2 ≤ 9
7x1 + 5x2 ≤ 28
4x1 + x2 ≤ 12
x ≥ 0, x ∈ Z2
Solve this IP and the corresponding LP relaxation problem (graphically by inspection). Com-
pare and discuss the solutions. Complete one of the following additional exercises.
(a) Solve the given IP using the Branch and Bound Method. Solve subproblems graphi-
cally and illustrate the procedure.
(b) Solve the given IP using the Gomory Cutting Planes Algorithm. Solve the subproblems
graphically and illustrate the procedure.
(c) Solve the given IP by first constructing an equivalent binary IP and then using the
method of Implicit Enumeration.
24