adversarial stas-cal relaonal ailowd/adversarial-starai-lowd... · 2017. 1. 18. · representation...

AdversarialSta-s-calRela-onalAI

DanielLowdUniversityofOregon

Outline

•  Whydoweneedadversarialmodeling?–  BecauseofthedreamofAI–  Becauseofcurrentreality–  Becauseofpossibledangers

•  Ourini-alapproachandresults–  Background:adversariallearning+collec-veclassifica-on

–  Robustnessthroughadversarialsimula-on[Torkamani&Lowd,ICML’13]

–  Robustnessthroughregulariza-on[Torkamani&Lowd,ICML’14]

2

WhatisStarAI?“Theore-cally,combininglogicandprobabilityinaunifiedrepresenta-onandbuildinggeneral-purposereasoningtoolsforithasbeenthedreamofAI,da-ngbacktothelate1980s.Prac-cally,successfulStarAItoolswillenablenewapplica-onsinseverallarge,complexreal-worlddomainsincludingthoseinvolvingbigdata,socialnetworks,naturallanguageprocessing,bioinforma-cs,theweb,robo-csandcomputervision.Suchdomainsareo\encharacterizedbyrichrela-onalstructureandlargeamountsofuncertainty.Logichelpstoeffec-velyhandletheformerwhileprobabilityhelpshereffec-velymanagethela^er.WeseektoinviteresearchersinallsubfieldsofAItoa^endtheworkshopandtoexploretogetherhowtoreachthegoalsimaginedbytheearlyAIpioneers.”[www.starai.org]

3

ThedreamofAI:Unifyinglogicandprobability!

WhoisStarAI?“Specifically,theworkshopwillencourageac-vepar-cipa-onfromresearchersinthefollowingcommuni-es:•  sa-sfiability(SAT)•  knowledgerepresenta-on(KR)•  constraintsa-sfac-onandprogramming(CP)•  (induc-ve)logicprogramming(LPandILP)•  graphicalmodelsandprobabilis-creasoning(UAI)•  sta-s-callearning(NIPS,ICML,andAISTATS)•  graphmining(KDDandECMLPKDD)•  probabilis-cdatabases(VLDBandSIGMOD).”[www.starai.org]

4

WhoisStarAI?“Itwillalsoac-velyinvolveresearchersfrommoreappliedcommuni-es,suchas:•  naturallanguageprocessing(ACLandEMNLP)•  informa-onretrieval(SIGIR,WWWandWSDM)•  vision(CVPRandICCV)•  seman-cweb(ISWCandESWC)•  robo-cs(RSSandICRA).”[www.starai.org]

5

AlmosteveryonedoingAIresearch!

Sta-s-calRela-onalAI

•  Therealworldiscomplexanduncertain•  Logichandlescomplexity•  Probabilityhandlesuncertainty

6

AdversarialSta-s-calRela-onalAI

•  Therealworldiscomplex,uncertain,andadversarial

•  Logichandlescomplexity•  Probabilityhandlesuncertainty•  Gametheoryhandlesadversarialinterac-on

7

• Includeresearchersinmul--agentsystems(AAMAS)andsecurity(CCS)

IfyouwanttounifyAI,whystopwithlogicandprobability?

Outline

•  Whydoweneedadversarialmodeling?–  BecauseofthedreamofAI–  Becauseofcurrentreality–  Becauseofpossibledangers

•  Ourini-alapproachandresults–  Background:adversariallearning+collec-veclassifica-on

–  Robustnessthroughadversarialsimula-on[Torkamani&Lowd,ICML’13]

–  Robustnessthroughregulariza-on[Torkamani&Lowd,ICML’14]

8

Example:SocialNetworkSpamWhichusersarespammers?

Imagecredit:[Fakhraeietal.,2015]9

t(1)

t(3) t(2)

t(4)

t(5)

t(6)

t(7)

t(8)

t(9)

t(10)

t(8)t(3)

Figure 1: A time-stamped multi-relation social network withlegitimate users and spammers. Each link hv

1

, v

2

i in thenetwork represents an action (e.g. profile view, message, orpoke) performed by v

1

towards v2

at specific time t.

late. Furthermore, many social networks can not monitorall the generated contents due to privacy and resources con-cerns. Content-independent frameworks, such as the oneproposed in this paper, can be applied to systems that pro-vide maximum user privacy with end-to-end encryption.

Perhaps the most important di↵erence between social net-works and email or web graphs is that social networks havea multi-relational nature, where users have relationships ofdi↵erent types with other users and entities in the networks.For example, they can send messages to each other, addeach other as friends, “like”each other’s posts, and send non-verbal signals such as “winks” or “pokes.” Figure 1 shows arepresentation of a social network as a time-stamped multi-relation graph. The multi-relational nature provides morechoices for spammers, but it also empowers detection sys-tems to monitor patterns across activity types, and time.In this paper, we propose a content-independent frameworkwhich is based on the multi-relational graph structure ofdi↵erent activities between users, and their sequences.

Our proposed framework is motivated by Tagged.com, asocial network for meeting new people which was foundedin 2004 and has over 300 million registered members. Moregenerally, the framework is applicable to any multi-relationalsocial network. Our goal is to identify sophisticated spam-mers that require manual or semi-automated interventionby the administrative security team. These spammers havealready passed initial classifiers and know how to manipu-late their accounts and contents to avoid being caught byautomatic filters. We show that our framework significantlyreduces the need for manual administration to control spam.

Our framework consists of three components. First, weextract graph structure features for each of the relationsand show that considering the multi-relational nature of thegraphs improves the performance. Second, we consider theactivity sequence of each user across these relations and ex-tract k -gram features and employ mixtures of Markov mod-els to label spammers. Third, we propose a statistical re-lational model based on hinge-loss Markov random fields toperform collective reasoning using signals from an abuse re-porting system in the social network.

The following sections formally define the problem and oursolution framework along with an experimental validation ofour approach on internet-scale data from Tagged.com.

2. PROBLEM STATEMENTWe represent a social network as a directed time-stamped

dynamic multi-relational graph G = hV, Ei, where V is theset of vertices of the form v = hf

1

, . . . , f

n

i representingusers and their demographic features f

i

, and E is the setof directed edges of the form e = hv

src

, v

dst

, r

i

, t

i

i represent-ing their interactions, relation type r

i

, and a discrete time-stamp t

i

. The social spam detection problem is to predictwhether v

i

with an unobserved label is a spammer or not,based on the given network G and a set of observed labelsfor already identified spammers. Since the deployed securitysystem could employ di↵erent measures based on the clas-sification confidence, we are interested in (un-normalized)probabilities or ranking scores of the likelihood that eachuser is a spammer. In other words, the problem is assign-ment of a score (e.g., a probability) to user accounts to rankthem from the most to the least probable spammer in thesystem: c : v

i

! [0, 1].

3. OUR METHODIn our framework, we focus on three di↵erent mechanisms

to identify spammers and malicious activities. We first cre-ate networks from the user interactions and compute net-work structure features from them. As these are evolvingnetworks, each user generates a sequence of actions with thepassage of time. Mining these sequences can provide valu-able insights into the intentions of the user. We use twomethods to study these sequences and extract features fromthem. We use the output of these methods as features toclassify spammers. We then employ a collective model toidentify spammer accounts only based on the signals fromthe abuse reporting system (G

report

) as a secondary sourceto reassure predictions. The following sections discuss ourframework and extracted features in more details.

3.1 Graph Structure Features (XG)We create a directed graph G

r

= hV, Er

i for each relation r

in the social network, where vertices V consist of users, andedges E

r

represent interactions of type r between users, e.g.if user

1

sends a message to user2

then Gmessage

will contain v

1

and v

2

representing the two users, and e

1,2

representing therelation between them. We have ten di↵erent graphs eachcontaining the same users as vertices but di↵erent actionsas edges.We use Graphlab CreateTM3 to generate features based

on each of these graphs for each user. We use six graphanalytics methods m

i

to compute the features. Using eachm

i

we create a set of features for each relation graph Gr

asfollowing:

XmiGr

=hXmi

Gr1. . . Xmi

Grn

i

where m

i

is one of the graph analytics methods describedbelow, and r

i

is one of the relationships considered in thestudy.We then use these features together to get a complete

multi-relational graph feature-set, as the following:

XmGr =

⇥Xm1

Gr. . . X

mkGr

⇤

The graph analytics methods m

i

we use to extract thefeatures from each relation network are described in the fol-3

http://dato.com/products/create

2

Example:FraudDetec-oninOnlineAuc-ons

Whichpeoplearefraudstersoraccomplices?

Imagecredit:[Chau&al06]

104 D.H. Chau, S. Pandit, and C. Faloutsos

(a) Initial (b) Labeled (c) Manually labeled

Fig. 1. 2LFS in action: (a) given graph (b) after labeling by 2LFS: fraud (red triangles), honest (green circles), “accomplices” (yellow diamonds) (c) after manual rearrangement, to highlight the “bipartite cores”. The nodes in the two black rectangles are confirmed fraudsters.

The goal of our work is to treat the auction fraud problem systematically, using data mining and machine learning techniques to spot unnatural patterns in auctions. We propose the 2LFS algorithm, and illustrate its effectiveness on real, public data from a large auction site. Figure 1(a) illustrates a small graph from the large auction site, in which it is difficult to spot any suspicious patterns. The result of labeling by 2LFS is shown in Figure 1(b). Fraudsters are the red triangles and honest users are the green circles. The yellow diamonds correspond to accomplices, which we will discuss in detail later in the paper. Figure 1(c) shows the same graph after manual rearrange-ment so that nodes with the same label are grouped together. Now we can clearly observe the existence of a bipartite core between the fraudsters and accomplices. As we will explain later, such bipartite cores are a tell-tale sign of a popular fraud scheme. In fact, the nodes in the two rectangles in Figure 1(c) are confirmed fraud-sters, who have received many negative feedbacks from buyers who had paid for items that never got delivered.

The rest of the paper is organized as follows. Section 2 provides an overview of re-lated work. Section 3 describes the auction fraud detection problem. Section 4 de-scribes in detail the 2LFS algorithm. Section 5 provides empirical evidence for the effectiveness, robustness and scalability of our method. Section 6 discusses some observations on how easily we can generalize our method to other fraud detection problems. Finally, we present a brief summary of our results in Section 7 with point-ers for future work.

2 Related Work

To the best of our knowledge, this is the first work that uses a systematic approach to analyze and detect electronic auction frauds. We survey earlier attempts to detect such frauds, as well as literature related to trust propagation.

Auction Frauds and Reputation Systems. Reputation systems are extensively used by electronic auctions to prevent frauds. Although helpful, these systems are very sim-ple and can be easily foiled. To study the effectiveness of today’s reputation systems,

10

Example:Securi-esDealers

Imagecredit:[Neville&Jensen07]

NEVILLE AND JENSEN

Disclosure

!"#$

%&'"

Broker (Bk)

()

*#+,-./0+

102$34."5

2$34."51)62#0+

7#0

890/)"00

Firm

:/;"

()

*#+,-./0+

<#&3==0

Branch (Bn)

>"?/3)

@$"#

8A

8A

8A

8)

8)

8)

8)

BelongsTo

LocatedAt

BelongsToBelongsTo

LocatedAt

8)

BelongsTo

LocatedAt

8)

LocatedAt

LocatedAt

LocatedAt LocatedAt

LocatedAtLocatedAt

FiledOn

FiledOn

FiledOn

FiledOn

FiledOn

8A

FiledOn

FiledOn

FiledOn

FiledOn

FiledOn

FiledOn

FiledOn

FiledOn

FiledOn

8AFiledOnLocatedAt

BelongsTo

Figure 13: RDN for the NASD data set (1999).

Page

!"#

$%&'

!"#

(%)*%)

(+,--. #/0%. LinkedFrom

LinkedToLinkedTo

LinkedToLinkedFrom

Figure 14: RDN for the WebKB data set.

RDNRPT models. Recall that the RDNRBC uses the entire set of attributes in the resulting subgraphsand the RDNRPT performs feature selection over the attribute set.

Figure 15 shows AUC results for the first three model types on the five prediction tasks. (Wediscuss RMN results below.) Figure 15a graphs the results of the RDNRPT , compared to the RPTconditional model. Figure 15b graphs the results of the RDNRBC, compared to the RBC conditionalmodel.

The graphs show AUC for the most prevalent class, averaged over a number of training/testsplits. For Cora, IMDb, and NASD, we used temporal sampling where we learned models on oneyear of data and applied the models to the subsequent year. There were four temporal samples forIMDb and NASD, and five for Cora. For WebKB we used cross-validation by department, learningon three departments and testing on pages from the fourth, held-out department. For Gene therewas no clear sampling choice, so we used ten-fold cross validation on random samples of genes.When there were links between the test and training sets, the class labels of the training set weremade available to the RDNs and RMNs for use during inference. We used two-tailed, paired t-teststo assess the significance of the AUC results obtained from the trials. The t-tests compare the RDNresults to the conditional and ceiling models, with a null hypothesis of no difference in the AUC.

680

Whichbrokersarelikelytoreceivecomplaintsinthefuture?

11

MoreExamples

•  Webspam•  Wormdetec-on•  Fakereviews•  Counterterrorism Commonthemes:

1.  Adversariescanbedetectedbytheirrela-onshipsaswellastheira^ributes.

2.  Adversariesmaychangetheirbehaviortoavoiddetec-on.

12

Outline

•  Whydoweneedadversarialmodeling?–  BecauseofthedreamofAI–  Becauseofcurrentreality–  Becauseofpossibledangers

•  Ourini-alapproachandresults–  Background:adversariallearning+collec-veclassifica-on

–  Robustnessthroughadversarialsimula-on[Torkamani&Lowd,ICML’13]

–  Robustnessthroughregulariza-on[Torkamani&Lowd,ICML’14]

13

RobustnessandSafetyinAI

•  ManyAIsystemsinteractwithpeople–thisisavulnerabilityandaliability.

•  HowcanweknowthatanAIsystemiscorrect,safe,orrobust?

•  Adversarialreasoningandmodelingcanhelpbuildmorerobustsystemsbyop-mizingpessimis-cally.

14

RelatedWorkonMul--AgentStarAI

•  Poole,1997:IndependentChoiceLogic•  Repngeretal.,2008:ASta-s-calRela-onalModelforTrustLearning

•  Lippi,2015:Sta-s-calRela-onalLearningforGameTheory

15

Outline

•  Whydoweneedadversarialmodeling?–  BecauseofthedreamofAI–  Becauseofcurrentreality–  Becauseofpossibledangers

•  Ourini-alapproachandresults–  Background:adversariallearning+collec-veclassifica-on

–  Robustnessthroughadversarialsimula-on[Torkamani&Lowd,ICML’13]

–  Robustnessthroughregulariza-on[Torkamani&Lowd,ICML’14]

16

SpecialCase:Sta-cPredic-onGames

•  Ingeneral,wemayhavearbitraryagents,u-lityfunc-ons,andgamestructures.Hard.

•  Predic-ongames– Firstplayerchoosesthemodel(e.g.,spamfilter)– Secondplayerchoosesthetestdata(e.g.,spam)

•  Domains:Socialnetworkspam,onlineauc-onfraud,badsecuri-esdealers,webspam,fakereviews,andmore!

17

Example:SpamFiltering

cheap=1.0mortgage=1.5

Totalscore=2.5

From: spammer@example.com Cheap mortgage now!!!

FeatureWeights

>1.0(threshold)

1.

2.

3.

Spam18

Example:SpammersAdapt

cheap=1.0mortgage=1.5Eugene=-1.0Oregon=-1.0

Totalscore=0.5

From: spammer@example.com Cheap mortgage now!!! Eugene Oregon

FeatureWeights

<1.0(threshold)

1.

2.

3.

OK19

Example:ClassifierAdapts

cheap=1.5mortgage=2.0Eugene=-0.5Oregon=-0.5

Totalscore=2.5

FeatureWeights

>1.0(threshold)

1.

2.

3.

OKSpam

From: spammer@example.com Cheap mortgage now!!! Eugene Oregon

20

AdversarialClassifica-onasaGame

•  Learnerselectsaclassifierc.•  Adversaryselectsmodifiedevidencex.•  Eachreceivesarewardbasedonhowcorrecttheclassifierwasandhowcorrupttheevidencewas.

21

Previouswork:Assumesinstancesareindependent!(e.g.,Dalvi&al04;Globerson&Roweis06;Teo&al08;Dekel&Shamir08;Xu&al09;Brückner&Scheffer09;Brückner&Scheffer11)

Collec-veClassifica-on

Labelasetofobjectsusingtherela-onshipsamongthemaswellastheira^ributes.

22

MarkovLogicNetworksAMarkovLogicNetwork(MLN)isalog-linearmodelwherethefeaturesarecountsofsa-sfiedformulas.Givenafinitesetofconstants,thisdefinesaprobabilitydistribu-onoverpossibleworlds:

Weight of formula i No. of true groundings of formula i in x

logP(x) = wini (x)i∑ − A

logP(y | x) = wTφ(x, y)− A(x)Condi-onaldistribu-onofqueryatoms(y)givenevidence(x):

= wTφ(x)− A

= score(w, x, y)− A(x)23

MarkovLogicNetworksforCollec-veClassifica-on

1.Thelabelforanobjectodependsonitsa^ributes:

HasAttribute(o,+a) ⇒ Label(o,+c) 2.Relatedobjectsaremorelikelytohavesimilarlabels:

Related(o,o’) ∧ Label(o,+c) ⇒ Label(o’,+c)

Createcopiesoftheserulesforeachclassanda^ribute,andthenlearnaweightforeachrule.

24

Goal:Selectwtomaximizethemarginbetweentruelabelingandanyalternatelabeling.

Max-MarginWeightLearning

y0

25

Maximizethemargin+weightedslackvariable

y

Scoreofthetruelabeling

Scoreofanalternatelabeling

Differencesbetweenthelabelings

minw

1

2w

Tw + C⇠

s.t. score(w, x, y) � score(w, x, y0) +�(y, y0)� ⇠ 8y0

Goal:Selectwtomaximizethemarginbetweentruelabelingandanyalternatelabeling.

Learner’slossfromthebestalternatelabeling

(biggestmarginviola-on)

L2regularizerontheweights.

Max-MarginWeightLearning

y y0

26

min

w

1

2

w

Tw + C

✓max

y0[score(w, x, y

0)� score(w, x, y) +�(y, y

0)]

◆

SpecialCase:Associa-veMarkovNetworks

•  Iftheweightsofthesecondformulaareposi-ve,thenlinkednodesaremorelikelytohavethesamelabel:

•  Inferencecanbedoneinpolynomial-mewithgraphcutsorasalinearprogram.[Kolmogorov&Zabin04]

•  Learningcanbedoneinpolynomial-mewithaconvexquadra-cprogram.[Taskar&al04]

27

Outline

•  Whydoweneedadversarialmodeling?–  BecauseofthedreamofAI–  Becauseofcurrentreality–  Becauseofpossibledangers

•  Ourini-alapproachandresults–  Background:adversariallearning+collec-veclassifica-on

–  Robustnessthroughadversarialsimula-on[Torkamani&Lowd,ICML’13]

–  Robustnessthroughregulariza-on[Torkamani&Lowd,ICML’14]

28

AdversarialCollec-veClassifica-onWewanttorobustlylabelrelateden--eswhoareac-velyworkingtoavoiddetec-on.

Assume:AdversarycanmodifyuptoDa^ributes.(e.g.,add/removewordsfromspamwebpages)

29

ConvexAdversarialCollec-veClassifica-onModifyourassocia-veMarkovnetworkbyassumingaworst-caseadversary:

Enforceamarginbetweentruelabelingandalternatelabelinggivenworst-caseadversariallymodifieddata.

�(x, x0) D

30

s.t.

min

w

1

2

w

T

w + C

✓max

x

0,y

0[score(w, x

0, y

0)� score(w, x

0, y) +�(y, y

0)]

◆

Reformula-ngasaquadra-cprogram:1.  Removebilineari-esinthescorefunc-onbyintroducing

auxiliaryvariables.2.  Replacetheinnermaximiza-onwithitsdualminimiza-on

problem.Theorem:Forbinary-valuedlabelsandfeatures,theadversary’smaximiza-onhasanintegralsolu-on.Thus,therelaxedlearningproblemisexact.

(Canbeextendedwithmul-pletypesofrela-ons,aslongasallareassocia-ve.)

ConvexAdversarialCollec-veClassifica-on

31

Datasets•  Poli-calblogs

–  2004blogdatacollectedbyAdamic(2005)– WerecrawledinFebruary2012andMay2012toaddwords,removedeadblogs.

–  Selected100wordswithmutualinforma-on•  Reuters

–  4classesfromtheModAptesplit:crude,grain,trade,money-fx

–  Splitinto7-meperiods,eachwith300-400ar-cles– Addedlinksto2mostsimilarar-cles(TF-IDF)–  Selected200wordswithmutualinforma-on

32

ExperimentalMethods

Tuning:Selectparameterstomaximizeperformanceonvalida-ondataagainstadversarywhocouldmodify10%ofthea^ributes.

Evalua-on:Measuredaccuracyontestdataagainstsimulatedadversarieswithbudgetsfrom0%to25%.

33

Method Rela@onal? Adversarial?LinearSVM No NoSVM-Invar[Teo&al08] No YesAMN[Taskar&al04] Yes NoCACC[Torkamani&Lowd13] Yes Yes

Results:Poli-calBlogs,Tunedfor10%adversary

0 5 10 15 20 250

10

20

30

40

50

60

70

80

Strength of adversary (%)

Cla

ssifi

catio

n Er

ror (

%)

SVMSVMINVAMNCACC

34

Results:Reuters,Tunedfor10%adversary

0 5 10 15 20 250

10

20

30

40

50

60

70

80

Strength of adversary (%)

Cla

ssifi

catio

n Er

ror (

%)

SVMSVMINVAMNCACC

35

WeightDistribu-onIntui-vely,ifanadversarycanchangesomeofthea^ributes

thenwewanttoavoidplacinghighweightsonanya^ributes.CACCdoesthisautoma-cally:

0 20 40 60 80 1000

0.05

0.1

0.15

0.2

0.25

Sorted Indices

Weig

ht

Valu

e

AMN

CACC

Maximumweight?

36

AdversarialRegulariza-on

•  Empirically,op-mizingperformanceagainstasimulatedadversarycanleadtoboundedweights.

•  Whatifweavoidsimula-ngtheadversaryandinsteadjustboundtheweights?

•  Wecanshowthatthetwoareequivalent!(Underaslightlydifferentadversarialmodelthanweusedbefore.)

•  Moregenerally,wecanachieveadversarialrobustnessonanystructuredpredic-onproblembyaddingaregularizer.

37

AdversarialModel

•  Previously,weassumedtheadversarycouldmodifytheevidence,x,byasmallnumberofchanges.

•  Nowweassumethattheadversarycanmodifythefeaturevector,φ(x,y),byasmallvectorδ/2.–  Thus,theycanmodifythedifferencebetweentwofeaturevectors,φ(x,y’)–φ(x,y),byδ.

–  Thus,theycanmodifythedifferencebetweentwoscores,score(w,x,y’)–score(w,x,y),bywTδ.

38

Op-miza-onProblem

Whichisequivalentto:

39

min

w

1

2

w

Tw + C max

�2S,y0[score(w, x, y)� score(w, x, y)

+ w

T� +�(y, y

0)]

min

w

1

2

w

Tw +max

�2Sw

T�

+ Cmax

y0[score(w, x, y)� score(w, x, y) +�(y, y

0)]

‘

‘

EllipsoidalUncertainty

Supposetheadversaryisconstrainedbyanorm:

Theorem:RobustnessoverSisequivalenttoaddingthedualnormasaregularizer:

Specialcase:ForL1ball,thedualnormis(max).

S = {�| ||M�|| 1}

L1

(c.f.[Xuetal.,2009]forrobustnessofregularSVMs.)

40

min

w

1

2

w

Tw + kM�1

wk

+ Cmax

y0[score(w, x, y)� score(w, x, y) +�(y, y

0)]

‘

PolyhedralUncertaintySupposetheadversaryisconstrainedtoapolyhedron:Theorem:RobustnessoverSisequivalenttoaddingalinearregularizerinatransformedweightspace:WecanalsoletSbetheintersec-onofapolyhedronandanellipsoidandobtainageneraliza-onofbothresults.

S = {�|A� b}

41

min

�

1

2

�

T(AA

T)�+ b

T�

+ Cmax

y0[�

T(A

T�(x, y)�A

T�(x, y

0)) +�(y, y

0)]

RobustlyClassifying11yearsofPoli-calBlogs

201220112010200920082007200620052003 2004 2013

IraqWar

•  Goal:Labeleachblogasliberalorconserva-ve•  Poli-calblogsdataset(AdamicandGlance,2005)

+bag-of-wordsfeaturesfromeachyear•  Train/tuneon2004andtestoneveryyear.•  Robustmodel:Assumeadversarycanmodifyuptokwordsandklinks.

42

43

OngoingWork:Large-ScaleApplica-ons

•  CommentspamonYouTube•  AbuseandspamonSoundCloud•  SocialnetworkspammersonTagged.com•  Fraudulentimages(DARPAMediForprogram)

44

• Mul-pletypesofrela-ons•  Complexadversaries• Millionsofobjectstolabel

Challenges:

Detec-ngSpammyComments

45

hTp://…/spammy1.htmlURL1

hTp://…/cats.html

URL2

hTp://…/spammy2.htmlURL3

SimilarityScore:0.9

Detec-ngSpammyComments

46

hTp://…/spammy1.htmlURL1

hTp://…/cats.html

URL2

hTp://…/spammy2.htmlURL3

SimilarityScore:0.9

Detec-ngSpammyComments

47

hTp://…/spammy1.htmlURL1

hTp://…/cats.html

URL2

hTp://…/spammy2.htmlURL3

SimilarityScore:0.9

Detec-ngSpammyComments

48

hTp://…/spammy1.htmlURL1

hTp://…/cats.html

URL2

hTp://…/spammy2.htmlURL3

SimilarityScore:0.9

Detec-ngSpammyComments

49

hTp://…/spammy1.htmlURL1

hTp://…/cats.html

URL2

hTp://…/spammy2.htmlURL3

SimilarityScore:0.9

AdversaryResponse

50

hTp://…/spammy1.htmlURL1

hTp://…/cats.html

URL2

hTp://…/spammy2.htmlURL3

SimilarityScore:0.9

OpenQues-ons

•  Non-zero-sumgames•  Represen-ngstrategies:Weights,decisionnodes,distribu-ons?

•  Integratewithplanning,reinforcementlearning

•  Whenisadversarialmodelingunnecessary?•  Bestmethodsforvalida-ngadversarialmodels(outsideofindustry)

51

Conclusion•  StarAIneedsadversarialmodeling

–  Tofulfilllong-termAIvision–  Tosolvecurrentapplica-ons–  Toimproverobustness/safety

•  Twowaystolearnrobustrela-onalclassifiers:–  Embedtheadversaryinsidetheop-miza-onproblem–  Constructanequivalentregularizer(Specialcase:setamaximumweight!)

–  Empirically,thesemodelsarerobusttomaliciousadversariesandnon-maliciousconceptdri\.

•  Manyopenques-onsandchallenges!

52