ECE 6397, Fall, 2012

Selected Topic in OptimizationSelected Topic in Optimization

Zhu Han

Department of Electrical and Computer Engineering

Class 1

Aug. 27nd, 2012

                                                           

                                                           

OutlineOutline

Instructor information Motivation to study optimization Course descriptions and textbooks What you will study from this course

Objectives Coverage and schedule Homework, projects, and exams

Other policies Reasons to be my students Background and Preview

                                                           

Instructor InformationInstructor Information

Office location: Engineering 2 W302 Office hours: M 10am-2pm or by appointment Email: zhan2@mail.uh.edu or hanzhu22@gmail.com Phone: 713-743-4437(o), 301-996-2011(c) Course website:

http://www2.egr.uh.edu/~zhan2/ECE6397/ Research interests:

http://www2.egr.uh.edu/~zhan2

Wireless Networking, Signal Processing, and Security

http://wireless.egr.uh.edu/

                                                           

MotivationsMotivations

Optimization is the mathematical discipline which is concerned with finding the maxima and minima of functions, possibly subject to constraints.

Interdisciplinary• Architecture• Nutrition• Electrical circuits• Economics• Transportation

• Examples:• Determining which ingredients and in what quantities to add to a mixture being made so that it will meet specifications on its composition• Allocating available funds among various competing agencies• Deciding which route to take to go to a new location in the city

                                                           

Course DescriptionsCourse Descriptions

What is the optimization framework? What are the major types?

Convex vs. non-convex Continuous vs. discrete Centralized vs. distributed Deterministic vs. stochatic

What are the theorems? What are the applications? What are the state-of-art research? Can I find research topics? How to conduct research and write technique paper

                                                           

Textbook and SoftwareTextbook and Software

Require textbook:

1. Zhu Han, Dusit Niyato, Walid Saad, Tamer Basar, and Are Hjorungnes, Game Theory in Wireless and Communication Networks: Theory, Models and Applications, Cambridge University Press, UK, 2011.

2. Steven Boyd’s videos for convex optimization3. Handout for parts of book, Zhu Han and K. J. Ray Liu,

Resource Allocation for Wireless Networks: Basics, Techniques, and Applications, Cambridge University Press, 2008.

4. Other handouts Require Software: MATLAB

http://www.mathworks.com/ or type helpwin in Matlab environment

                                                           

ScheduleSchedule

• Introduction to optimization• Convex optimization

• Steven Boyd’s classhttp://www.stanford.edu/~boyd/cvxbook/• 30% of the class• Need to watch videos as homework (17 videos for 1 hour 15 min each) • Watch the video before the class!!!• Class is just review

• Integer/Combinatorial optimization• Might based on Georgia tech class• 15%

• Stochastic optimization• Might based on UIUC class• 15%

• Game Theory• based on my book• 40%

                                                           

Homework, Project, and ExamHomework, Project, and Exam

Homework Watch videos for convex optimization Some other homework

Projects: simple MATLAB programs Based on the simulation at the end of each chapter

Exams Two independent exams Grading policy

Participations Attendance and Feedback Quiz if the attendance is low

                                                           

Teaching StylesTeaching Styles

Slides plus black board Slides can convey more information in an organized way Blackboard is better for equations and prevents you from

not coming. Course Website

Print handouts with 3 slides per page before you come Homework assignment and solutions Project descriptions and preliminary codes

Feedback Too fast, too slow Presentation, Writing, English, …

                                                           

Other PoliciesOther Policies

Any violation of academic integrity will receive academic and possibly disciplinary sanctions, including the possible awarding of an XF grade which is recorded on the transcript and states that failure of the course was due to an act of academic dishonesty. All acts of academic dishonesty are recorded so repeat offenders can be sanctioned accordingly.• CHEATING• COPYING ON A TEST• PLAGIARISM • ACTS OF AIDING OR ABETTING • UNAUTHORIZED POSSESSION • SUBMITTING PREVIOUS WORK • TAMPERING WITH WORK • GHOSTING or MISREPRESENTATION • ALTERING EXAMS• COMPUTER THEFT

                                                           

Reasons to be my studentsReasons to be my students

Wireless Communication and Networking have great market Usually highly paid and have potential to retire overnight Highly interdisciplinary Do not need to find research topics which are the most

difficult part. Research Assistant Free trips to conferences in Alaska, Hawaii, Europe, Asia… A kind of nice (at least looks like) Work with hope and happiness Graduate fast

                                                           

Different Kinds of OptimizationDifferent Kinds of Optimization

                                                           

Optimization Formulation and AnalysisOptimization Formulation and Analysis

We discuss how to formulate the problem as an optimization issue.

Specifically, we study what the objectives are, what the parameters are, what the practical constraints are, and what the optimized performances across the different layers are.

The tradeoffs between the different optimization goals and different users' interests are also investigated.

The goal is to provide the students a new perspective from the optimization point of view for variety of problems in engineering fields.

                                                           

Mathematical ProgrammingMathematical Programming If the optimization problem is to find the best objective function

within a constrained feasible region, such a formulation is sometimes called a mathematical program.

Many real-world and theoretical problems can be modeled in this general framework.

We discuss the four major subfields of the mathematical programming: – linear programming,

– convex programming,

http://www.stanford.edu/~boyd/cvxbook/

– nonlinear programming,

– dynamic programming.

                                                           

What do we optimize?What do we optimize?

A real function of n variables

with or without constrains– Without constraint

– With constraint

),,,(21 n

xxxf

22 2),(min yxyxf

2

2),(min

1,52

2),(min

0

2),(min

22

22

22

or

or

yx

yxyxf

yx

yxyxf

x

yxyxf

                                                           

Lets OptimizeLets Optimize

Suppose we want to find the minimum of the function

What is special about a local max or a local min of a function f (x)?

at local max or local min f’(x)=0

f”(x) > 0 if local min

f”(x) < 0 if local max

                                                           

Review max-min for Review max-min for 33

                                                           

Integer/Combinatorial Optimization Integer/Combinatorial Optimization The discrete optimization is the problem in which the decision variables

assume discrete values from a specified set.

The combinatorial optimization problems, on the other hand, are problems of choosing the best combination out of all possible combinations.

Most combinatorial problems can be formulated as integer programs.

Integer optimization is the process of finding one or more best (optimal) solutions in a well defined discrete problem space.

The major difficulty with these problems is that we do not have any optimality conditions to check if a given (feasible) solution is optimal or not.

We listed several possible solutions such as – relaxation and decomposition,

– enumeration,

– knapsack problem

– cutting planes.

                                                           

Example of Integer ProgramExample of Integer Program(Production Planning-Furniture (Production Planning-Furniture

Manufacturer)Manufacturer) Technological data:

Production of 1 table requires 5 ft pine, 2 ft oak, 3 hrs labor

1 chair requires 1 ft pine, 3 ft oak, 2 hrs labor

1 desk requires 9 ft pine, 4 ft oak, 5 hrs labor

Capacities for 1 week: 1500 ft pine, 1000 ft oak,

20 employees (each works 40 hrs).

Market data:

Goal: Find a production schedule for 1 week tomaximize the profit.

profit demand

table $12/unit 40

chair $5/unit 130

desk $15/unit 30

                                                           

Production Planning-Furniture Production Planning-Furniture Manufacturer: modeling the problem Manufacturer: modeling the problem

as integer programas integer program

The goal can be achieved

by making appropriate decisions.

First define decision variables:

Let xt be the number of tables to be produced;

xc be the number of chairs to be produced;

xd be the number of desks to be produced.

(Always define decision variables properly!)

                                                           

Production Planning-Furniture Production Planning-Furniture Manufacturer: modeling the problem Manufacturer: modeling the problem

as integer programas integer program Objective is to maximize profit:

max 12xt + 5xc + 15xd

Functional Constraints

capacity constraints:

pine: 5xt + 1xc + 9xd 1500

oak: 2xt + 3xc + 4xd 1000

labor: 3xt + 2xc + 5xd 800

market demand constraints:

tables: xt ≥ 40

chairs: xc ≥ 130

desks: xd ≥ 30

Set Constraints

xt , xc , xd Z+

                                                           

Solutions to integer Solutions to integer programsprograms

A solution is an assignment of values to variables.

A feasible solution is an assignment of values to variables such that all the constraints are satisfied.

The objective function value of a solution is obtained by evaluating the objective function at the given point.

An optimal solution (assuming maximization) is one whose objective function value is greater than or equal to that of all other feasible solutions.

There are efficient algorithms for finding the optimal solutions of an integer program.

                                                           

Game Theory Game Theory Game theory is a branch of applied mathematics that uses models to study

interactions with formalized incentive structures (“games").

It studies the mathematical models of conflict and cooperation among intelligent and rational decision makers.

Rational means that each individual's decision-making behavior is consistent with the maximization of subjective expected utility.

Intelligent means that each individual understands everything about the structure of the situation, including the fact that others are intelligent rational decision makers.

We have discussed four different types of games, namely, the non-cooperative game, repeated game, cooperative game, and auction theory.

Slideshttp://wireless.egr.uh.edu/research.htm

The basic concepts are listed and simple examples are illustrated.

                                                           

Game Theory OverviewGame Theory Overview What is game theory?

– The formal study of conflict or cooperation– Modeling mutual interaction among rational decision makers– Widely used in economics

Components of a “game”– Rational players with conflicting interests or mutual benefit– Strategies or actions– Utility as a payoff of player’s and other players’ actions– Outcome: Nash Equilibrium

Many types– Non-cooperative game theory– Cooperative game theory– Dynamic game theory– Stochastic game– Auction theory

                                                           

Rich Game Theoretical ApproachesRich Game Theoretical Approaches Non-cooperative static

game: play once

– Mandayam and Goodman (2001)– Virginia tech

Repeated game: play multiple times– Threat of punishment by repeated game. MAD: Nobel prize 2005. – Tit-for-Tat (infocom 2003):

Dynamic game: (Basar’s book)– ODE for state– Optimization utility over time – HJB and dynamic programming– Evolutional game (Hossain and Dusit’s work)

Stochastic game (Altman’s work)

Prisoner Dilemma Payoff: (user1, user2)

                                                           

Auction TheoryAuction Theory

Book of Myerson (Nobel Prize 2007), J. Huang, H. Zheng, X. Li

                                                           

Term ProjectTerm Project Forming the group, 2-3 people per group

– Similar research background

Formulation of problems– Is that a problem?

– What is the objective and constraints

– What is best optimization techniques

Simulation– Matlab

– Victim algorithm

Analysis

Writing– It will be a headache for everybody

                                                           

HomeworkHomework Convex optimization I

– http://www.youtube.com/watch?v=McLq1hEq3UY

– Watch before Wed. class!!!

Form Term project group– 2-3 people per group

– Let me know in the next class for grouping and basic interests