![Page 1: Privacy-Preserving Linear Programming Olvi Mangasarian UW Madison & UCSD La Jolla UCSD – Center for Computational Mathematics Seminar January 11, 2011](https://reader035.vdocument.in/reader035/viewer/2022070401/56649f1d5503460f94c33646/html5/thumbnails/1.jpg)
Privacy-Preserving Linear Programming
Olvi MangasarianUW Madison & UCSD La Jolla
UCSD – Center for Computational Mathematics SeminarJanuary 11, 2011
![Page 2: Privacy-Preserving Linear Programming Olvi Mangasarian UW Madison & UCSD La Jolla UCSD – Center for Computational Mathematics Seminar January 11, 2011](https://reader035.vdocument.in/reader035/viewer/2022070401/56649f1d5503460f94c33646/html5/thumbnails/2.jpg)
Problem Statement
• Entities with related data wish to solve a linear program based on all the data
• The entities are unwilling to reveal their data to each other– If each entity holds a different set of variables for all constraints,
then the data is said to be vertically partitioned– If each entity holds a different set of constraints with all variables,
then the data is said to be horizontally partitioned
• Our approach: privacy-preserving linear programming (PPLP) using random matrix transformations– Provides exact solution to the total linear program– Does not reveal any private information
![Page 3: Privacy-Preserving Linear Programming Olvi Mangasarian UW Madison & UCSD La Jolla UCSD – Center for Computational Mathematics Seminar January 11, 2011](https://reader035.vdocument.in/reader035/viewer/2022070401/56649f1d5503460f94c33646/html5/thumbnails/3.jpg)
Vertically Partitioned MatrixHorizontally Partitioned Matrix
A
A1
A2
A3
A¢1 A¢2 A¢3
Linear Programming Constraint Matrix
Variables1 2 ..………….…………. n
Constraints
12........m
![Page 4: Privacy-Preserving Linear Programming Olvi Mangasarian UW Madison & UCSD La Jolla UCSD – Center for Computational Mathematics Seminar January 11, 2011](https://reader035.vdocument.in/reader035/viewer/2022070401/56649f1d5503460f94c33646/html5/thumbnails/4.jpg)
Outline
• Vertically (horizontally) partitioned linear program
• Secure transformation via a random matrix
• Privacy-preserving linear program solution
• Computational results
• Summary
![Page 5: Privacy-Preserving Linear Programming Olvi Mangasarian UW Madison & UCSD La Jolla UCSD – Center for Computational Mathematics Seminar January 11, 2011](https://reader035.vdocument.in/reader035/viewer/2022070401/56649f1d5503460f94c33646/html5/thumbnails/5.jpg)
Vertically Partitioned Data:Each entity holds different variables for the
same constraints
A
¢1
A
¢3
A
¢2
A
¢1
A
¢2
A
¢3
![Page 6: Privacy-Preserving Linear Programming Olvi Mangasarian UW Madison & UCSD La Jolla UCSD – Center for Computational Mathematics Seminar January 11, 2011](https://reader035.vdocument.in/reader035/viewer/2022070401/56649f1d5503460f94c33646/html5/thumbnails/6.jpg)
LP with Vertically Partitioned Data
minx2X
c0x where X = fx j Ax ¸ bg
We consider the linear program:
whereA 2 Rm£ n , c 2 Rn
Thematrix£c0A
¤is divided into p vertical blocks
Each block is (m+1) £ nj , j = 1;2;: : : ;p,such that n1+n2+:::+np =n,is \ owned" by a distinct entity unwilling tomake it public.
Objective: Solve this linear programwithout revealing anyprivately held data.
![Page 7: Privacy-Preserving Linear Programming Olvi Mangasarian UW Madison & UCSD La Jolla UCSD – Center for Computational Mathematics Seminar January 11, 2011](https://reader035.vdocument.in/reader035/viewer/2022070401/56649f1d5503460f94c33646/html5/thumbnails/7.jpg)
Secure Linear Program Generation
Each of thep entities chooses its own privately heldrandommatrixB¢j 2 Rk£ n j , j = 1;::: ;p,wherek ¸ n. De ne: B = [B¢1 B¢2 : : : : : :B¢p] 2 Rk£ n.Wenote immediately that the rank of the randommatrixB 2 Rk£ n with k ¸ n is n, which is the reasonfor choosing k ¸ n. Utilizing this fact wede nethe invertible transformation: x = B0u, andits least 2-norm inverse: u = B(B0B)¡ 1x:Wenow transformour original linear program intothe following \ secure" linear program:minu2U
c0B0u where U = fu j AB0u ¸ bg:
![Page 8: Privacy-Preserving Linear Programming Olvi Mangasarian UW Madison & UCSD La Jolla UCSD – Center for Computational Mathematics Seminar January 11, 2011](https://reader035.vdocument.in/reader035/viewer/2022070401/56649f1d5503460f94c33646/html5/thumbnails/8.jpg)
Why Secure Linear Program?
Weuse the term \ secure" to describe the transformedlinear programbecause it does not reveal any oftheprivately held data
£c0jA ¢j
¤, j = 1;: : : ;p. This is so
because for each entity di®erent fromentity j , it isimpossible to computeeither cj from the revealed productcj 0B¢j
0, or A¢j from the revealed product A¢j B¢j0
without knowing the randommatrix B¢j chosen byentity j and known to itself only.
![Page 9: Privacy-Preserving Linear Programming Olvi Mangasarian UW Madison & UCSD La Jolla UCSD – Center for Computational Mathematics Seminar January 11, 2011](https://reader035.vdocument.in/reader035/viewer/2022070401/56649f1d5503460f94c33646/html5/thumbnails/9.jpg)
Original & Secure LPs Are Equivalent
Let k ¸ n for the randommatrix B 2 Rk£ n .The secure linear programis solvable if and only iftheoriginal linear program is solvable in which casetheextrema of both linear programs areequal.
For onepageproof see:ftp:/ / ftp.cs.wisc.edu/ pub/ dmi/ tech-reports/ 10-01.pdfOptimization Letters, to appear.
![Page 10: Privacy-Preserving Linear Programming Olvi Mangasarian UW Madison & UCSD La Jolla UCSD – Center for Computational Mathematics Seminar January 11, 2011](https://reader035.vdocument.in/reader035/viewer/2022070401/56649f1d5503460f94c33646/html5/thumbnails/10.jpg)
PPLP Algorithm
1. All p entities agreeon a k ¸ n, thenumber of rowsof the randommatrix B 2 Rk£ n as de ned earlier.
2. Each entity generates its own privately held randommatrix B¢j 2 Rk£ n j , j = 1;: : :: : : ;p, wherenjis thenumber of features held by entity j . This resultsin B = [B¢1 B¢2 : : : : : :B¢p] 2 Rk£ n :
![Page 11: Privacy-Preserving Linear Programming Olvi Mangasarian UW Madison & UCSD La Jolla UCSD – Center for Computational Mathematics Seminar January 11, 2011](https://reader035.vdocument.in/reader035/viewer/2022070401/56649f1d5503460f94c33646/html5/thumbnails/11.jpg)
PPLP Algorithm (Continued)
3. Each entity j makes public only itsmatrix productA¢j B¢j
0as well as its cost coe±cient product B¢j cj .Theseproducts do not reveal either A¢j or cj butallow thepublic computation of the full constraintmatrix needed for thesecure linear program:AB0=A¢1B¢1
0+A¢2B¢20+:::: : :+A¢pB¢p
0;as well as thecost coe±cient:c0B0= c01B¢1
0+c02B¢20+:::: : :+c0pB¢p
0:
![Page 12: Privacy-Preserving Linear Programming Olvi Mangasarian UW Madison & UCSD La Jolla UCSD – Center for Computational Mathematics Seminar January 11, 2011](https://reader035.vdocument.in/reader035/viewer/2022070401/56649f1d5503460f94c33646/html5/thumbnails/12.jpg)
PPLP Algorithm (Continued)
4. A public optimal solution vector u to thesecurelinear programand a public optimal objective functionvaluec0B0u arecomputed. This optimal valueequalstheoptimal objective function of theoriginal linearprogram.
5. Each entity computes its optimal xj component groupas follows:xj =B¢j
0u; j = 1;:: : ;p:
![Page 13: Privacy-Preserving Linear Programming Olvi Mangasarian UW Madison & UCSD La Jolla UCSD – Center for Computational Mathematics Seminar January 11, 2011](https://reader035.vdocument.in/reader035/viewer/2022070401/56649f1d5503460f94c33646/html5/thumbnails/13.jpg)
PPLP Algorithm (Continued)
6. Thesolution component vectors xj , j = 1;:: : ;p,are revealed by its owners if a public solution vector totheoriginal linear program is agreed upon. Else,the component vectorsmay bekept private if onlytheminimumvalue is needed, in which case thatminimumvalueequals thepublicly availableminimumvaluemin
u2Uc0B0u of the secure linear program.
![Page 14: Privacy-Preserving Linear Programming Olvi Mangasarian UW Madison & UCSD La Jolla UCSD – Center for Computational Mathematics Seminar January 11, 2011](https://reader035.vdocument.in/reader035/viewer/2022070401/56649f1d5503460f94c33646/html5/thumbnails/14.jpg)
Computatinal ResultsExample 1 (k=n=1000)
² Generated a randomsolvable linear programwith m= 100 and n = 1000
² Partitioned thecolumns of A as well as thecost vector cinto threegroups with n1 =500, n2 =300 and n3 =200
² Generated three randommatrices, with coe±cientsuniformly distributed in the interval [0,1] withB¢1 2 Rn£ n1 , B¢2 2 Rn£ n2 andB¢3 2 Rn£ n3
² Solved secure linear programand compared its optimalobjectivevaluewith that of theoriginal linear program.
² The two optimal objectives agreed to 14 signi¯cant¯gures attained at two distinct optimal solution points.
![Page 15: Privacy-Preserving Linear Programming Olvi Mangasarian UW Madison & UCSD La Jolla UCSD – Center for Computational Mathematics Seminar January 11, 2011](https://reader035.vdocument.in/reader035/viewer/2022070401/56649f1d5503460f94c33646/html5/thumbnails/15.jpg)
Computatinal ResultsExample 2 (k=n=100)
² Generated a randomsolvable linear programwith m= 1000 and n = 100
² Partitioned thecolumns of A as well as thecost vector cinto threegroups with n1 =50, n2 =30 and n3 =20
² Generated three randommatrices, with coe±cientsuniformly distributed in the interval [0,1] withB¢1 2 Rn£ n1 , B¢2 2 Rn£ n2 andB¢3 2 Rn£ n3
² Solved secure linear programand compared its optimalobjectivevaluewith that of theoriginal linear program.
² The two optimal objectives agreed to 13 signi¯cant¯gures attained at points that wereessentially thesame, that isthe1 -normof their di®erencewas less than 2:3e¡ 13.
![Page 16: Privacy-Preserving Linear Programming Olvi Mangasarian UW Madison & UCSD La Jolla UCSD – Center for Computational Mathematics Seminar January 11, 2011](https://reader035.vdocument.in/reader035/viewer/2022070401/56649f1d5503460f94c33646/html5/thumbnails/16.jpg)
Horizontally Partitioned Constraint Matrix:Entities hold different constraints with the same variables
A 1 A 2 A 3
A 3
A 2
A 1
![Page 17: Privacy-Preserving Linear Programming Olvi Mangasarian UW Madison & UCSD La Jolla UCSD – Center for Computational Mathematics Seminar January 11, 2011](https://reader035.vdocument.in/reader035/viewer/2022070401/56649f1d5503460f94c33646/html5/thumbnails/17.jpg)
LP with Horizontally Partitioned Data
minx2X
c0x where X = fx j Ax = b; x ¸ 0gWe consider the linear program:
whereA 2 Rm£ n , c 2 Rn
Here, [A b], is divided into p horizontal blocks ofm1;m2; : : : : : : and mp, (n+1)-dimensionalrows with m1+m2+:::+mp =m.Each block of rows of [A b] is \ owned" by a distinctentity that is unwilling tomake its block of data public.Objective: Solve this linear programwithout revealing anyprivately held data.
![Page 18: Privacy-Preserving Linear Programming Olvi Mangasarian UW Madison & UCSD La Jolla UCSD – Center for Computational Mathematics Seminar January 11, 2011](https://reader035.vdocument.in/reader035/viewer/2022070401/56649f1d5503460f94c33646/html5/thumbnails/18.jpg)
Secure Linear Program Generation
Each of thep entities chooses its own privately heldrandommatrix B¢i 2 Rk£ mi , i = 1;: : :;p, wherek ¸ m,which de nes: B = [B¢1 B¢2 : : : : : :B¢p] 2 Rk£ m:Wenote immediately that the rank of the randommatrixB 2 Rk£ m with k ¸ m ism, which is the reasonfor choosing k ¸ m. Utilizing this fact wede nethe transformations: BA and Bb.Wenow transformour original linear program intothe following \ secure" linear program:miny2Y
c0y where Y = fy j BAy= Bb; y ¸ 0g:
![Page 19: Privacy-Preserving Linear Programming Olvi Mangasarian UW Madison & UCSD La Jolla UCSD – Center for Computational Mathematics Seminar January 11, 2011](https://reader035.vdocument.in/reader035/viewer/2022070401/56649f1d5503460f94c33646/html5/thumbnails/19.jpg)
Why Secure Linear Program?
Weuse the term \ secure" to describe the transformedlinear programbecause it does not reveal any of theprivatelyheld data: thehorizontal partitions of [A b].This is so because for each entity di®erent fromentity i,it is impossible to computeeither A i from the revealedproduct B¢iA i , or bi from the revealed product B¢ibiwithout knowing the randommatrix B¢i
chosen by entity i and known to itself only.
![Page 20: Privacy-Preserving Linear Programming Olvi Mangasarian UW Madison & UCSD La Jolla UCSD – Center for Computational Mathematics Seminar January 11, 2011](https://reader035.vdocument.in/reader035/viewer/2022070401/56649f1d5503460f94c33646/html5/thumbnails/20.jpg)
Original & Secure LPs Are Equivalent
Let k ¸ m for the randommatrix B 2 Rk£ m.The secure linear programis solvable if and only iftheoriginal linear program is solvable in which casetheextrema of both linear programs areequal.
Theproof follows from theobvious fact that:Ax = b ( ) BAx = Bb:For moredetails see:ftp:/ / ftp.cs.wisc.edu/ pub/ dmi/ tech-reports/ 10-02.pdf
![Page 21: Privacy-Preserving Linear Programming Olvi Mangasarian UW Madison & UCSD La Jolla UCSD – Center for Computational Mathematics Seminar January 11, 2011](https://reader035.vdocument.in/reader035/viewer/2022070401/56649f1d5503460f94c33646/html5/thumbnails/21.jpg)
PPHPLP Algorithm
1. All p entities agreeon a value for k ¸ m, wherekis thenumber of rows of the randommatrix B 2 Rk£ m
2. Each entity generates its own privately held randommatrix B¢i 2 Rk£ mi , i = 1;:: : : : : ;p,wheremi isthenumber of rows held by entity i which results in:B = [B¢1 B¢2 : : : : : :B¢p] 2 Rk£ m:
![Page 22: Privacy-Preserving Linear Programming Olvi Mangasarian UW Madison & UCSD La Jolla UCSD – Center for Computational Mathematics Seminar January 11, 2011](https://reader035.vdocument.in/reader035/viewer/2022070401/56649f1d5503460f94c33646/html5/thumbnails/22.jpg)
PPHPLP Algorithm (Continued)
3. Each entity i makes public only itsmatrix productB¢iA i as well as its right hand sideproduct B¢ibi .Theseproducts do not reveal either A i or bi but allowthepublic computation of the full constraint matrixneeded for thesecure linear program:BA = [B¢1A1+B¢2A2+:::+B¢pAp] 2 Rk£ n ;as well as the right hand side :Bb= [B¢1b1+B¢2b2+:::+B¢pbp] 2 Rk:
4. A public optimal solution vector y to thesecure linearprogram is obtained which also solves theoriginal linearprogram
![Page 23: Privacy-Preserving Linear Programming Olvi Mangasarian UW Madison & UCSD La Jolla UCSD – Center for Computational Mathematics Seminar January 11, 2011](https://reader035.vdocument.in/reader035/viewer/2022070401/56649f1d5503460f94c33646/html5/thumbnails/23.jpg)
Computatinal ResultsExample 1 (k=1000)
² Generated a randomsolvable linear programwith m= 600 and n = 1000
² Partitioned the rows of A as well as the right handsidevector b into threegroups with m1 = 100, m2 =200and m3 =300.
² Generated three randommatrices, with coe±cientsuniformly distributed in the interval [0,1] withB1 2 Rk£ m1 , B2 2 Rk£ m2 and B3 2 Rk£ m3 .
² Solved secure linear programand compared its optimalsolution with that of theoriginal linear program.
² The two optimal solutions were identical.
![Page 24: Privacy-Preserving Linear Programming Olvi Mangasarian UW Madison & UCSD La Jolla UCSD – Center for Computational Mathematics Seminar January 11, 2011](https://reader035.vdocument.in/reader035/viewer/2022070401/56649f1d5503460f94c33646/html5/thumbnails/24.jpg)
Computatinal ResultsExample 2 (k=1000)
² Generated a randomsolvable linear programwith m= 1000 and n = 1000
² Partitioned the rows of A as well as the right handsidevector b into threegroups with m1 = 200, m2 =300and m3 =500.
² Generated three randommatrices, with coe±cientsuniformly distributed in the interval [0,1] withB1 2 Rk£ m1 , B2 2 Rk£ m2 and B3 2 Rk£ m3 .
² Solved secure linear programand compared its optimalsolution with that of theoriginal linear program.
² The two optimal solutions were identical.
![Page 25: Privacy-Preserving Linear Programming Olvi Mangasarian UW Madison & UCSD La Jolla UCSD – Center for Computational Mathematics Seminar January 11, 2011](https://reader035.vdocument.in/reader035/viewer/2022070401/56649f1d5503460f94c33646/html5/thumbnails/25.jpg)
Summary & Outlook
– Based on a transformation using a random matrix B
– Get exact solution to the original linear program without revealing privately held data
Possible extensions to: horizontally partitioned inequality constraints, complementarity problems and nonlinear programs
Privacy preserving linear programmingfor vertically or horizontally partitioned data
![Page 26: Privacy-Preserving Linear Programming Olvi Mangasarian UW Madison & UCSD La Jolla UCSD – Center for Computational Mathematics Seminar January 11, 2011](https://reader035.vdocument.in/reader035/viewer/2022070401/56649f1d5503460f94c33646/html5/thumbnails/26.jpg)
References
ftp://ftp.cs.wisc.edu/pub/dmi/tech-reports/10-01.pdfftp://ftp.cs.wisc.edu/pub/dmi/tech-reports/10-02.pdf
Optimization Letters, to appear