Upload File

µ ^ µ u } µ o d Æ ] u ] Ì ] } v ] v d ] À , µ Ç> x e p µ Ç...

  • Home
  • Documents
  • µ ^ µ u } µ o D Æ ] u ] Ì ] } v ] v D ] À , µ Ç> X E P µ Ç vvova/sublinear2016/slides/Nguyen.pdf · 2018-01-28 · d Z v l z } µ J Y µ ] } v M. Title: Microsoft PowerPoint
30
Distributed Submodular Maximization in Massive Datasets Joint work with Rafael Barbosa, Alina Ene, Justin Ward Huy L. Nguyen

Upload: others

Post on 09-Jul-2020

6 views

Category:

Documents


0 download

Report

TRANSCRIPT

Page 1: µ ^ µ u } µ o D Æ ] u ] Ì ] } v ] v D ] À , µ Ç> X E P µ Ç vvova/sublinear2016/slides/Nguyen.pdf · 2018-01-28 · d Z v l z } µ J Y µ ] } v M. Title: Microsoft PowerPoint

Distributed SubmodularMaximization in Massive Datasets

Joint work with Rafael Barbosa, Alina Ene, Justin Ward

Huy L. Nguyen

Page 2: µ ^ µ u } µ o D Æ ] u ] Ì ] } v ] v D ] À , µ Ç> X E P µ Ç vvova/sublinear2016/slides/Nguyen.pdf · 2018-01-28 · d Z v l z } µ J Y µ ] } v M. Title: Microsoft PowerPoint

Combinatorial Optimization• Given

– A set of objects V– A function f on subsets of V– A collection of feasible subsets I

• Find– A feasible subset of I that maximizes f

• Goal– Abstract/general f and I– Capture many interesting problems– Allow for efficient algorithms

Page 3: µ ^ µ u } µ o D Æ ] u ] Ì ] } v ] v D ] À , µ Ç> X E P µ Ç vvova/sublinear2016/slides/Nguyen.pdf · 2018-01-28 · d Z v l z } µ J Y µ ] } v M. Title: Microsoft PowerPoint

SubmodularityWe say that a function is submodular if:

We say that is monotone if:

Alternatively, f is submodular if:

for all and Submodularity captures diminishing returns.

Page 4: µ ^ µ u } µ o D Æ ] u ] Ì ] } v ] v D ] À , µ Ç> X E P µ Ç vvova/sublinear2016/slides/Nguyen.pdf · 2018-01-28 · d Z v l z } µ J Y µ ] } v M. Title: Microsoft PowerPoint

SubmodularityExamples of submodular functions:

– The number of elements covered by a collection of sets– Entropy of a set of random variables– The capacity of a cut in a directed or undirected graph– Rank of a set of columns of a matrix– Matroid rank functions– Log determinant of a submatrix of a psd matrix

Page 5: µ ^ µ u } µ o D Æ ] u ] Ì ] } v ] v D ] À , µ Ç> X E P µ Ç vvova/sublinear2016/slides/Nguyen.pdf · 2018-01-28 · d Z v l z } µ J Y µ ] } v M. Title: Microsoft PowerPoint

Example: Multimode Sensor Coverage• We have distinct locations where we can place sensors• Each sensor can operate in different modes, each with a distinct coverage profile• Find sensor locations, each with a single mode to maximize coverage

Page 6: µ ^ µ u } µ o D Æ ] u ] Ì ] } v ] v D ] À , µ Ç> X E P µ Ç vvova/sublinear2016/slides/Nguyen.pdf · 2018-01-28 · d Z v l z } µ J Y µ ] } v M. Title: Microsoft PowerPoint

Example: Identifying Representatives In Massive Data

Page 7: µ ^ µ u } µ o D Æ ] u ] Ì ] } v ] v D ] À , µ Ç> X E P µ Ç vvova/sublinear2016/slides/Nguyen.pdf · 2018-01-28 · d Z v l z } µ J Y µ ] } v M. Title: Microsoft PowerPoint

Example: Identifying Representative Images• We are given a huge set X of images.• Each image is stored multidimensional vector.• We have a function d giving the difference between two images.• We want to pick a set S of at most k images to minimize the loss function:

• Suppose we choose a distinguished vector e0 (e.g. 0 vector), and set:

• The function f is submodular. Our problem is then equivalent to maximizing f under a single cardinality constraint.

Page 8: µ ^ µ u } µ o D Æ ] u ] Ì ] } v ] v D ] À , µ Ç> X E P µ Ç vvova/sublinear2016/slides/Nguyen.pdf · 2018-01-28 · d Z v l z } µ J Y µ ] } v M. Title: Microsoft PowerPoint

Need for Parallelization• Datasets grow very large

– TinyImages has 80M images– Kosarak has 990K sets

• Need multiple machines to fit the dataset• Use parallel frameworks such as MapReduce

Page 9: µ ^ µ u } µ o D Æ ] u ] Ì ] } v ] v D ] À , µ Ç> X E P µ Ç vvova/sublinear2016/slides/Nguyen.pdf · 2018-01-28 · d Z v l z } µ J Y µ ] } v M. Title: Microsoft PowerPoint

Problem Definition• Given set V and submodular function f• Hereditary constraint I (cardinality at most k, matroid constraint of rank k, … )• Find a subset that satisfies I and maximizes f• Parameters

– n = |V|– k = max size of feasible solutions– m = number of machines

Page 10: µ ^ µ u } µ o D Æ ] u ] Ì ] } v ] v D ] À , µ Ç> X E P µ Ç vvova/sublinear2016/slides/Nguyen.pdf · 2018-01-28 · d Z v l z } µ J Y µ ] } v M. Title: Microsoft PowerPoint

Greedy AlgorithmInitialize S = {}While there is some element x that can be added to S:

Add to S the element x that maximizes the marginal gainReturn S

Page 11: µ ^ µ u } µ o D Æ ] u ] Ì ] } v ] v D ] À , µ Ç> X E P µ Ç vvova/sublinear2016/slides/Nguyen.pdf · 2018-01-28 · d Z v l z } µ J Y µ ] } v M. Title: Microsoft PowerPoint

Greedy Algorithm• Approximation Guarantee

• 1 - 1/e for a cardinality constraint• 1/2 for a matroid constraint

• Inherently sequential• Not suitable for large datasets

Page 12: µ ^ µ u } µ o D Æ ] u ] Ì ] } v ] v D ] À , µ Ç> X E P µ Ç vvova/sublinear2016/slides/Nguyen.pdf · 2018-01-28 · d Z v l z } µ J Y µ ] } v M. Title: Microsoft PowerPoint

Distributed GreedyMirzasoleiman, Karbasi, Sarkar, Krause '13

Page 13: µ ^ µ u } µ o D Æ ] u ] Ì ] } v ] v D ] À , µ Ç> X E P µ Ç vvova/sublinear2016/slides/Nguyen.pdf · 2018-01-28 · d Z v l z } µ J Y µ ] } v M. Title: Microsoft PowerPoint

Performance of Distributed Greedy• Only requires 2 rounds of communication• Approximation ratio is:

(where m is number of machines)• Can construct bad examples• Lower bounds for the distributed setting

(Indyk et al. ’14)

Page 14: µ ^ µ u } µ o D Æ ] u ] Ì ] } v ] v D ] À , µ Ç> X E P µ Ç vvova/sublinear2016/slides/Nguyen.pdf · 2018-01-28 · d Z v l z } µ J Y µ ] } v M. Title: Microsoft PowerPoint

Power of Randomness

Page 15: µ ^ µ u } µ o D Æ ] u ] Ì ] } v ] v D ] À , µ Ç> X E P µ Ç vvova/sublinear2016/slides/Nguyen.pdf · 2018-01-28 · d Z v l z } µ J Y µ ] } v M. Title: Microsoft PowerPoint

Power of Randomness• Randomized distributed Greedy

– Distribute the elements of V randomly in round 1– Select the best solution found in rounds 1 & 2

• Theorem: If Greedy achieves a C approximation, randomized distributed Greedy achieves a C/2 approximation in expectation.• Related results: [Mirrokni, Zadimoghaddam ’15]

Page 16: µ ^ µ u } µ o D Æ ] u ] Ì ] } v ] v D ] À , µ Ç> X E P µ Ç vvova/sublinear2016/slides/Nguyen.pdf · 2018-01-28 · d Z v l z } µ J Y µ ] } v M. Title: Microsoft PowerPoint

Intuition• If elements in OPT are selected in round 1 with high probability

– Most of OPT is present in round 2 so solution in round 2 is good• If elements in OPT are selected in round 1 with low probability

– OPT is not very different from typical solution so solution in round 1 is good

Page 17: µ ^ µ u } µ o D Æ ] u ] Ì ] } v ] v D ] À , µ Ç> X E P µ Ç vvova/sublinear2016/slides/Nguyen.pdf · 2018-01-28 · d Z v l z } µ J Y µ ] } v M. Title: Microsoft PowerPoint

Power of Randomness• Randomized distributed Greedy

– Distribute the elements of V randomly in round 1– Select the best solution found in rounds 1 & 2

• Provable guarantees– Constant factor approx for several constraints

• Generality– Same approach to parallelize a class of algorithms– Only need a natural consistency property– Extends to non-monotone functions

Page 18: µ ^ µ u } µ o D Æ ] u ] Ì ] } v ] v D ] À , µ Ç> X E P µ Ç vvova/sublinear2016/slides/Nguyen.pdf · 2018-01-28 · d Z v l z } µ J Y µ ] } v M. Title: Microsoft PowerPoint

Optimal Algorithms?• Near-optimal algorithms? • Framework to parallelize algorithms with almost no loss?

YES, using a few more rounds

Page 19: µ ^ µ u } µ o D Æ ] u ] Ì ] } v ] v D ] À , µ Ç> X E P µ Ç vvova/sublinear2016/slides/Nguyen.pdf · 2018-01-28 · d Z v l z } µ J Y µ ] } v M. Title: Microsoft PowerPoint
Page 20: µ ^ µ u } µ o D Æ ] u ] Ì ] } v ] v D ] À , µ Ç> X E P µ Ç vvova/sublinear2016/slides/Nguyen.pdf · 2018-01-28 · d Z v l z } µ J Y µ ] } v M. Title: Microsoft PowerPoint
Page 21: µ ^ µ u } µ o D Æ ] u ] Ì ] } v ] v D ] À , µ Ç> X E P µ Ç vvova/sublinear2016/slides/Nguyen.pdf · 2018-01-28 · d Z v l z } µ J Y µ ] } v M. Title: Microsoft PowerPoint

Core Set

Page 22: µ ^ µ u } µ o D Æ ] u ] Ì ] } v ] v D ] À , µ Ç> X E P µ Ç vvova/sublinear2016/slides/Nguyen.pdf · 2018-01-28 · d Z v l z } µ J Y µ ] } v M. Title: Microsoft PowerPoint

Core SetSend Core Setto every machine

Page 23: µ ^ µ u } µ o D Æ ] u ] Ì ] } v ] v D ] À , µ Ç> X E P µ Ç vvova/sublinear2016/slides/Nguyen.pdf · 2018-01-28 · d Z v l z } µ J Y µ ] } v M. Title: Microsoft PowerPoint

Core Set

Page 24: µ ^ µ u } µ o D Æ ] u ] Ì ] } v ] v D ] À , µ Ç> X E P µ Ç vvova/sublinear2016/slides/Nguyen.pdf · 2018-01-28 · d Z v l z } µ J Y µ ] } v M. Title: Microsoft PowerPoint

Core Set

Page 25: µ ^ µ u } µ o D Æ ] u ] Ì ] } v ] v D ] À , µ Ç> X E P µ Ç vvova/sublinear2016/slides/Nguyen.pdf · 2018-01-28 · d Z v l z } µ J Y µ ] } v M. Title: Microsoft PowerPoint

Core SetGrow Core Setover 1/ rounds

Page 26: µ ^ µ u } µ o D Æ ] u ] Ì ] } v ] v D ] À , µ Ç> X E P µ Ç vvova/sublinear2016/slides/Nguyen.pdf · 2018-01-28 · d Z v l z } µ J Y µ ] } v M. Title: Microsoft PowerPoint

Core SetGrow Core Setover 1/ rounds

Page 27: µ ^ µ u } µ o D Æ ] u ] Ì ] } v ] v D ] À , µ Ç> X E P µ Ç vvova/sublinear2016/slides/Nguyen.pdf · 2018-01-28 · d Z v l z } µ J Y µ ] } v M. Title: Microsoft PowerPoint

Core SetGrow Core Setover 1/ rounds

Page 28: µ ^ µ u } µ o D Æ ] u ] Ì ] } v ] v D ] À , µ Ç> X E P µ Ç vvova/sublinear2016/slides/Nguyen.pdf · 2018-01-28 · d Z v l z } µ J Y µ ] } v M. Title: Microsoft PowerPoint

Core SetGrow Core Setover 1/ rounds

Leads to only an lossin the approximation IntuitionEach round adds an fraction of OPT to the Core Set

Page 29: µ ^ µ u } µ o D Æ ] u ] Ì ] } v ] v D ] À , µ Ç> X E P µ Ç vvova/sublinear2016/slides/Nguyen.pdf · 2018-01-28 · d Z v l z } µ J Y µ ] } v M. Title: Microsoft PowerPoint

Matroid Coverage (n=900, r=5) Matroid Coverage (n=100, r=100)

It's better to distribute ellipses from each location across several machines!

Matroid Coverage Experiments

Page 30: µ ^ µ u } µ o D Æ ] u ] Ì ] } v ] v D ] À , µ Ç> X E P µ Ç vvova/sublinear2016/slides/Nguyen.pdf · 2018-01-28 · d Z v l z } µ J Y µ ] } v M. Title: Microsoft PowerPoint

Thank You!Questions?


SETAC Technical Workshop SOP^ d d Z v ] o t } l Z } Æ v ( } u í r } ñ r Ç v ] v o µ µ µ o o Ç ï ì r ñ ì ] ] v X } À Ç ( } u ( µ o o Ç } ] o o Ç ( µ v v } µ µ o

Z ^ ( ( v o Ç ] } ( W } v ] o u ] ] } v Z µ ] } v ^ P ] Ç ... · r · } ( s ] ] Ç & µ v µ } Z } rZ } s o ] v î ì í ó A E µ u } ( À ] ] u Ç ( µ v µ } l } r } À o

v µ o ] Ç U ^ } ] o / v o µ ] } v v ] À ] o ^ } ] Ç v P ...kiat.or.id/resx/upload/Update_FINAL_GESI_and_CSE_Strate.pdf · ' v µ o ] Ç U ^ } ] o / v o µ ] } v v ] À ] o

d u À ] o i o Ç Ç v U o ] v v ] l v u ] v i o ] v v À Ç v ... · r À } v v µ t v µ u ] v µ o À µ µ v v v µ o ] v v ] l v u ] v v < Ç W

^ u ï r í ñ Z - YMCA · ^ u ï r í ñ Z D } v Ç d µ Ç t v Ç d Z µ Ç & ] Ç ^ µ Ç ^ µ v Ç

zE u ^ ] } µ v Ç ^ ] d Ç d v } v } µ v Ç , ^€¦ · o l & } Æ o u v Ç o Ç } µ v Ç rZ ] l ( Z } } o v & ] À ] Á } Ç v ' ] o o µ ^ } } µ v Ç rZ ] l ( Z } } o v

BTSG Green Bond at Towards the Sustaibable …...^ µ ] v ] o ] Ç d^ ' } µ K µ Z } Á µ ] v ] o ] Ç d^ ' } µ ' v } v W v ] } v ñ ^ } } ^ µ ] v ] o ] Ç W } o ] Ç _ ] v o (

^ u ] ] > µ v Z W u ] µ u µ Ç v

^ ð ñ í W K ] v P ^ Ç u µ µ u v î ì î ì D } µ o ô ^ Ç v Z ...•/ u P ] v Ç } µ [ ] v P Z r ( ] u o u v ] } v } ( } u µ µ •d Z ] v ( ] ~ l Ç v µ ~ l Ç U À o

Ç v ^ v µ Z } ] Ç } ( / v ] ^ µ } Ç µ Z } ] Ç µ v D ] v ] Ç } ( , o Z & u ] o ... · 2020. 7. 31. · W P î } ( ï ï } v } ( } P } u v ] } v &^^ / t Z r ì í l î ì î

D Ç ï í U î ì î ì ^ } o u v ] Ç } ( W v } ^ µ v Ç ^ X Z } W Z ] o ] ] v µ Z ... · 2020-05-28 · v µ u } Á Z v Ç } µ Ç } µ î ì î ì Æ µ v X > v u } Z } o ]

µ ] v E u } ] v P µ ] v } µ v Ç ] Ç ] } Á u } µ v

Ç v ^ v µ Z } ] Ç } ( / v ] ^ µ } Ç µ Z } ] Ç µ v D ] v ... · Title: Microsoft Word - FSSAI Advt 04.01.2019 Author: Sunil Created Date: 1/25/2019 4:28:33 PM

µ Ç / v µ } v W , } Á d } ^ µ } · 2020-04-28 · v µ u Ç } µ o Á ] o o Ç } µ ] X / ( Ç } µ µ ¨ í U ì ì ì Z Á ] o o Z ] v Z v Æ ] Á ] o o v } P Z v

^ Ç v Ç t } u v [ , } l Ç > P µ / v X î ì í õ v v µ o Z } · í ^ Ç v Ç t } u v [ , } l Ç > P µ / v X î ì í õ v v µ o Z } î î ì í õ ^ Ç v Ç t } u v [ , }

µ ] } ' } } / v µ Ç W ( } >W ' Ç o ] v D v P u v

^ Z } ( · 2019-07-05 · Z v µ u } ( v ] µ ] v î ì í ð X K µ o u ] o o ] À Ç } } } µ v Z µ v l v } Á v ] ] Ì v ] v À Ç v } } l v } v } ( } µ } µ v Ç ( o Z

] À v t ] µ ] µ o µ u } v [ ( } P } Z l } µ } µ Á Z } o Z ...€¦ · t Z ] Z } o Á } µ o Ç } µ o ] l } o Ç ] v v ] v P Ç M t Z u ] P Z Ç } µ o ] ( Z À v o ] l ] (

Rwanda W µ Ç ] v ] v µ } v o µ } v ] o ] v€¦ · RES-Research Studies Z ] o ] v ] v µ } v ^ Ç u W Z Á v ^ µ Ç Rwanda W µ Ç ] v ] v µ } v o µ } v ] o ] v

*ROGPDQ ,V * : } µ v Ç } ( Z ^ } µ o X...i } ] v ( } Á ] Z ^ v X t Z Ç } , o o } Á o o Z ] o u ] ( } µ v } ( o o µ M v Ç U Á l v } Á Z Z o o Ç v µ o Ç } o } À µ X

, À Ç } µ ~ } Z Ç ] Ç } µ ~ } Æ ] v , Z Z v P } Z , Z v ......, o Z } Ç } µ ( ] o o ] v P Z ] µ À Ç ] v } µ , o Z } Ç } µ ( ] o o ] v P Z ] µ À Ç ] v } µ

v µ ] } v ( } ^ µ v u o } Ç u v Z µ & } u ] v ^ u µ Ç W o µ d Z ^ µ v u o ... · 2020. 4. 22. · P v v u ] o } Ç } µ Z } µ P Z ^ u µ Ç W o µ X d Z µ v v v } P ]

µ Ç } v Ç v u ] } v } ( u ] r µ u ] o ( o } ] v P Á ] v µ ...€¦ · µ Ç } v Ç v u ] } v } ( u ] r µ u ] o ( o } ] v P Á ] v µ ] v } v ] ] v P } u ] v Á À v µ v o

W Ç u } v ] } v W sKd z · 2013. 3. 25. · sKd z ^ ( } } v } ( v Ç } µ Ç v } v ] } v J / ( Ç } µ Z À ] P v Z v Á } v sKd z ^ ( } } v } ] u } À Ç } µ Ç v } v ] } v J hZ

µ µ u v î ì î ì K ] v P ^ Ç u } u } v v v ^ µ µ...W } P u l } } l } E } Z } P u ] } o o Ç ] À i µ Ç } v ] l Z v } ] v µ ] } v } µ v } ] v ] v v } ( } P u ] v P Æ µ

u o , ] P Z À ] o ] o ] Ç ( } v Ç o ] ] } v ] v o } µ } K ... · w z Ç ] o u s ] µ o u o } µ u } v Ç } u ] v ] } v o ] ] } v z } À Ç < ] ~ z< z } À Ç k z ] } v z µ v

] v µ P Z Z ] o v [ , } ] o Z ] Ç d µ [ Z } v } µ v z ... · Z ] o v v Ç } µ v P } o [ o ] À o ] v µ Ç ] o o v X

u µ o Ì ] v ( } u i ] Ì Á ] v Ç Z Ì Ç µ ] P v ] µ ... · s Ì v ] l v í & } u µ o Ì ] v ( } u i ] Ì Á ] v Ç Z Ì Ç µ ] P v ] µ ] ' } } u } ] v v v ] Ï } u } Á

Brochure€¦ · /(Ç}µ vv} Z µ i Ç}µÁ] Z } µ Ç]v Z] o] U o } }v µ XdZ ] v Æ oo v Zv Z Á Á]oo o } vP }µ (} Ç}µ]v Ç}µ Z} v. o } ( µ ÇX u] W }P uu t Z}À Á v Ç ]+

P µ ] v P Z ] o v µ ] v P > } l } Á v€¦ · ^ ( P µ ] v P Z ] o v µ ] v P > } l } Á v d Z o } } ( ] + v Á Ç Ç } µ v P Z o ] ( Ç } µ Á } ] Title: C:UserscscfsgsmAppDataLocalTempmso9EAB.tmp

Whats New in 2019 Env Excellence Awards…9 µ v Ç Á ~ } o ] U Ç o ] v P U ZDt U Z Ì } µ 9 µ v Ç v P Ç ~ o ] ] Ç 9 µ v Ç Á µ

E µ v P D ] o ] Ç } u u µ v ] Ç ] } Ç o v P v

h^W µ u v Ç v K P v ] / u µ ] ] Ç > rhs µ ] v P Z < ] v Æ

^ Ç v Z ] v ] } Ç v Z ] ^ µ ] } v Ç } u ] ] v - Thieme^ µ } Ç ^ Ç v Z ] v ] } Ç v Z ] ^ µ ] } v Ç } u ] ] v ^ µ ] ] Á X D Z U E ] Z } o Z X > U D Z Á : X Ç v U > ] r

DQG IULHQGV Ç } ( < } l µ } } µ } u u µ v ] Ç ] v v€¦ · 6wv 3hwhu dqg 3dxo ,qylwhv