march 1956! - juliaopt · 2020. 11. 8. · code generation using the osqp solver. ieee conference...

JuMP Developers Meetup, 13 Jun 2017

Bartolomeo Stellato

OSQP.jl A Julia wrapper for the Operator Splitting QP solver

joint work with Goran Banjac,

Nicholas Moehle, Paul Goulart, Alberto Bemporad, Stephen Boyd

Why quadratic programming?

3

March 1956!

5

First-order methods

Handle large-scale problems

Warm starting

Pros Cons

Embeddable

Low accuracy solutions

Don’t detect infeasibility

Problem data dependent

6

General Purpose QP Solver

Detects InfeasibilityRobust Accurate

Based on first-order methods

The OSQP Solver

8

The problem

1

2xT Px + qT x

minimize 12xT Px + qT x

subject to l � Ax � u

Quadratic Program

l � Ax � u

9

ADMM

minimize f(x̃) + g(x)subject to x̃ = x

minimize f(x) + g(x)

9

ADMM

minimize f(x̃) + g(x)subject to x̃ = x

x̃k+1 � argminx̃

�f(x̃) +

�

2

��x̃ � xk +yk

�

��2�

xk+1 � argminx

�g(x) +

�

2

��x � x̃k+1 � yk

�

��2�

yk+1 � yk + ��x̃k+1 � xk+1

�

minimize f(x) + g(x)

1

2

3

10

How to split the QP?

minimize 12 x̃T Px̃ + qT x̃ + IAx=z(x̃, z̃) + I[l,u](z)

subject to (x̃, z̃) = (x, z)


subject to Ax = zl � z � u

10


f

f





10


f

f

g

g





11

OSQP

(xk+1, �k+1) � solve�P + �I AT

A � 1�I

� �xk+1

�k+1

�=

��xk � qzk � 1

�yk

�

z̃k+1 � zk + 1� (�k+1 � yk)

zk+1 � ��z̃k+1 + 1

�yk�

yk+1 � yk + ��z̃k+1 � zk+1

�



Problem

Algorithm

1

23

{{{

11

OSQP


A � 1�I

� �xk+1

�k+1

�=

��xk � qzk � 1

�yk

�

z̃k+1 � zk + 1� (�k+1 � yk)

zk+1 � ��z̃k+1 + 1

�yk�

yk+1 � yk + ��z̃k+1 � zk+1

�

Linear systemsolve



Problem

Algorithm

11

OSQP


A � 1�I

� �xk+1

�k+1

�=

��xk � qzk � 1

�yk

�

z̃k+1 � zk + 1� (�k+1 � yk)

zk+1 � ��z̃k+1 + 1

�yk�

yk+1 � yk + ��z̃k+1 � zk+1

�

Linear systemsolve

Easyoperations



Problem

Algorithm

11

OSQP


A � 1�I

� �xk+1

�k+1

�=

��xk � qzk � 1

�yk

�

z̃k+1 � zk + 1� (�k+1 � yk)

zk+1 � ��z̃k+1 + 1

�yk�

yk+1 � yk + ��z̃k+1 � zk+1

�

Linear systemsolve

Easyoperations

Factorization caching



Problem

Algorithm

11

OSQP


A � 1�I

� �xk+1

�k+1

�=

��xk � qzk � 1

�yk

�

z̃k+1 � zk + 1� (�k+1 � yk)

zk+1 � ��z̃k+1 + 1

�yk�

yk+1 � yk + ��z̃k+1 � zk+1

�

Linear systemsolve

Easyoperations

Warm starting




Problem

Algorithm

11

OSQP


A � 1�I

� �xk+1

�k+1

�=

��xk � qzk � 1

�yk

�

z̃k+1 � zk + 1� (�k+1 � yk)

zk+1 � ��z̃k+1 + 1

�yk�

yk+1 � yk + ��z̃k+1 � zk+1

�

Linear systemsolve

Easyoperations

Warm starting




Problem

Algorithm

Solution polishing

12

OSQPosqp.readthedocs.io

EmbeddableDetects Infeasibility

Library free

http://osqp.readthedocs.io

13

Compiled code size ~80kb

300x Reduction!

CPLEX

GUROBI

OSQP

14

InterfacesLanguages Parsers

JuMP

CVXPY

YALMIP

15

OSQP interface

% Create OSQP objec tm = osqp ( ) ;

% I n i t i a l i z e so l ve rm. setup (P , q , A, l , u ,

se t t i ngs ) ;

% Solver e s u l t s = m. so lve ( ) ;

% Update cost w i th q_newm. update ( ’q ’ , q_new ) ;

% Solve againresu l ts_new = m. so lve ( ) ;

# Create OSQP objec tm = osqp .OSQP( )

# I n i t i a l i z e so l ve rm. setup (P , q , A, l , u ,

se t t i ngs )

# Solver e s u l t s = m. so lve ( )

# Update cost w i th q_newm. update (q=q_new )

# Solve againresu l ts_new = m. so lve ( )

16

Code generation

# Create OSQP objec tm = osqp .OSQP( )

# I n i t i a l i z e so l ve rm. setup (P , q , A, l , u ,

se t t i ngs )

# Generate C codem. codegen ( ’ folder_name ’ )

Optimized C code

Embedded hardware

Numerical Example

18

Lasso

minimize ∥Ax− b∥22 + λ∥x∥1

Features Data pointsn m = 100n

18

Lasso

minimize ∥Ax− b∥22 + λ∥x∥1

Weighting parameter

Features Data pointsn m = 100n

19

Lasso timingsTi

me

[s]

0.001

0.01

0.1

1

10

100

1000

Data points

1,000 5,000 10,000 50,000 100,000

OSQPGUROBIMOSEK

10x

6x

OSQP in

“meta-algorithms”

21

MIQP


subject to l � Ax � uxi � Z �i � I

21

MIQP

Integer constraints



21

MIQP

Integer constraints



QP(��, +�)

QP(3, +�)QP(��, 3)

QP(2, 3)QP(��, 2)

22

Inner QPs


subject to l � Ax � uxi � xi � xi �i � I

QP(x, x)

22

Inner QPs


subject to l � Ax � uxi � xi � xi �i � I Changing

bounds

Reusing factorization

QP(x, x)

23

Saving computations


Warm starting

ADMM

+

QP(x, x) Repeated MIQPs

24

MIQP TimingsTi

me

[s]

0.001

0.01

1 2 3 4 5

miOSQP GUROBI

4x

2.8xSimple Python

Implementation!

Hybrid Model Predictive Control

NHorizon Length

Conclusions

26

Acknowledgements

Goran Banjac

NicholasMoehle

PaulGoulart

AlbertoBemporad

StephenBoyd

Oxford Stanford Oxford IMT Lucca Stanford

27

Final remarksOSQP

Robust Embeddable

New high-level algorithms

Julia interface

C code generation

Exploit Initialization

Simple

28

References

G. Banjac, P. Goulart, B. Stellato, and S. Boyd. Infeasibility detection in the alternating direction method of multipliers for convex optimization. optimization-online.org, 2017

B. Stellato, V. Naik, A. Bemporad, P. Goulart, and S. Boyd. Embedded mixed-integer quadratic optimization using the OSQP solver. IEEE Conference on Decision and Control (CDC) (submitted), 2017

G. Banjac, B.Stellato, N. Moehle, P. Goulart, A. Bemporad and S. Boyd. Embedded code generation using the OSQP solver. IEEE Conference on Decision and Control (CDC) (submitted), 2017

B. Stellato, G. Banjac, P. Goulart, A. Bemporad and S. Boyd. OSQP: An Operator Splitting Solver for Quadratic Programs. (Coming soon!)

march 1956! - juliaopt · 2020. 11. 8. · code generation using the osqp solver. ieee conference...

Documents