epic.org epic-19-09-11-nscai-foia-20200723-odni-production

epic.org EPIC-19-09-11-NSCAI-FOIA-20200723-ODNI-Production 000001


UNCLASSIFIED,ii----.5

IARPA . ;"•

Doing Bad Things ...

•With Al

•To Al ■ Because (of) Al

- -- ~ - ~ - --- -- ·- -

. , ' . . ~ - - - - -- - -- --

UNCLASSIFIEDhlfiiOUO


UNCLASSIFIED ,; , 686


■With Al ■ Autonomous weapons

--- - - ---- - - -~ l"'JT· ,l 1 (,;:_~J( :_ .\D1.,\I\Jl 1 -, ,'_\· :.,ri( l ~1 t;(1, t ·"' .\( 1 \ ', , ,·.•·1 :\ 1

~ - - - ~ -

UNCLASSIFIED,;Fet10


UNCLASSIFIE['lf'EQI 10

UNCLASSI-FIED/!F_O_I_IQ


UNCLASSIFIED c


•With Al ■ Autonomous weapons ■ Psyops

• J ; ; L , ( ~ ~ ~ j l - ~\ ) \ t, r ~ ( ) •' ~ , .- ,\ ~ ( r 1 P ·\ ( 1 I ( 1 'i ;..1 l r 1_ l 1 1 .\ 1~ • ' /, •

-- - -

-- --~ - ----UNCLASSIFIED.CW8 ~ 8


UNCLASSIFIFD 11ff¥-if¼:

thispersondoesnotexist.com

---• • ------ · ~----- -◄•----·- - - ----- -- -- ---- ~- -- -~-- ~----

' • ,I 1 i r r • ,•, ~' t \ • '< ' ) r~ • } 1•, l ( • < P- \ - ' r •~ \ • • r • • \ r' ~

- -- ---- -- ----- --• - ~---UNCLASSIFIED/' FA' 'A


UNCLASSIFIEn ttrP¥:lfr

Do Electric Ne ural Nets Dream Of A11i111e ShoH·s?

_Bisho unen Kaigan_: This is t he story of a temporar y manga • staged in a near future, to determine the fate of himself as

the world rests in a state of absolute chaos. There is a world that has been destroyed by weapons of mass destruction

and no one can escape ali ve. There are only a handful of people that can communicate with it. Kai is one of the

t wel ve shi kigami called a ::M ushi Kinga::. This is the stor y of Kai, a young man with a tragic past . (Source: MU) In the first episode of this anime fThe Last Episode J is a young girl named Kiba to herself, who meets a ghost called Tifa

and is the onl y one who can hear her as she goes about things in Tokyo. From there, Kiba's relationship with her

ghost becomes a serious and intense problem. · A series of events begin to occur after the second episode ends, in which the ghost ( s ) begins to loo k at Kiba from various

angles. From all this, Kiba becomes the ghost of Kai, and he

thiswaifudoesnotexist.com

- --- ---- ----- -- ------ ------- --- ---• j f ;:-~ \ • j 1 :.. /, 1 ', i\ ' . \ ' l I •

1 "', t ~ ' 1

( r j ~) ;- ( I j - l r "' /~ ( l , ' \. I • I. ' ' /\

- - - - - ~-UNCLASSIFIEDM5°1 IQ


UNCLASSIFIED~


■With Al Autonomous weapons

■ Psyops • • •

~ - -·---- - --------- ---~ -- --- --- --- - ---• ; ► • \ , • • I I - r ~ - • , 11

, I , , , ,

-- - - --- ~ -- - - -- - - - ----- - - - - - -- - -UNCLASS IFIEDhlffH~9



■With Al ■To Al

■ Reveal the wrong thing

-- -- - . ---- -- - - - - -- - -IJ ~ l I lit IJ 1 ~\ 1 ~/.i-.( [ u ,:: (,~,.\Rl ,. fJ::_.l() I ~( 1--i /\1 •• 1 - \ .... ~,;-,\;

UNCLASSIFIED#f-81!!@-


Let's say you're making a face identifier ...

AI "John Smith"

"John Smith"

Adversaries can invert Al models to learn about the training data

Model inversion attacks work even with black box access (e.g. if the model executable is encrypted)

--~- -~ --- - ---- -- --- ~·-- - -- --- - -. ' : l • • '\ ,~... '- l • - I ~ ·, I - ' :, • ,.

-- ~ -- ---- - -- - - --- --- - ~ -- - - -

Fredrikson et al., "Model Inversion Attacks That Exploit Confidence Information and Basic Countermeasures," (ACM, 2015)


UNCLASSIFIED.: C


■With Al ■ To Al

■ Reveal the wrong thing ■ Do the wrong thing

------- - - ~-- ---

.. r . \ ~ ~" l . ~ J ~ 1\ \ i :-- --' r,_ ; ) - :.... ) ( 1 • ' l 1 -- ( 1 -- :, { • • 1 T 1 , 1. •• 1 ,\ - - - -

UNCLASSIFIED~t!:/6


Let's say you're making an image classifier ...

Like humans, Als are subject to camouflage, illusions, etc.

Existing Al techniques are full

of adversarial examples ,

which adversaries can find and

exploit

Cl Stf t Outpur

I - ·•

Oass.1 t r Output

-- - - ----- ~----- --- ----- -- ---- ---------~------' 1 J • , • , • • , '~ • , , I • , t , • \ '

1

• •, , I

--- - -- - - ~ ----- - - - - - - -- -- -- -Brown, Tom B., Dandelion Mane, Aurko Roy, Martin Abadi , and Justin Gilmer."Adversarial Patch."ArXiv abs/1712.09665 (201J...,).-


Let's say you're making an image classifier ...

--- - - -- --- - -- -- -• J ~ I ( 1 ' ' ✓ l I .\. ) '\ •• ( . ) . i - ) ;- .: ' ( f i ,.) ' ( I ~ l \ '\ ( I \ 1 . I I ;, • I ' .\ l )

- - - - ~ ~. - -- - - -----Anish Athalye et al. "Synthesizing Robust Adversarial Examples." ArXiv abs/1707.07397 (2018).


UNCLASSIFIE.JM. a 4 !J=-.


■With Al •To Al

■ Reveal the wrong thing ■ Do the wrong thing ■ Learn the wrong thing

- - - - ----~ - -- - ---~ -- -- - -------. . .

• • • • , , I • I , • • • • ,

------ - - ------- - - ~- ---- ---------- - ------ - - - ---- - ----- ---UNCLASSIF IEDiW"@l!8


Let's say you're making a self-driving car ...

Label: ' Stop sign

Label: Speed limit sign

i -

Adversaries can insert Trojans into Als through small manipulations to training data

Al can remain infected with Trojan even after transfer learning

- -~ -- -- ----- - -- - ~ -

: • I 7 l \ l •• J ( [ ":. J 1, ,\ \ ( ~ J 1

- ') L .\ ' j l j ~ ( } I '. \ I ", ,\ ( I • ' I ~\ : ' ; ... ' l : -- --- ~ -- - -- - - - - -- --

Tianyu Gu et al., "BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain," ArXiv :1708.06733 (2017i--


UNCLASSIF IED : I a Cs


■ With Al •To Al

• Reveal the wrong thing ■ Do the wrong thing • Learn the wrong thing ■ Value the wrong thing • • • •

- - - -- -·--- ~--- -- - --- ---- - ---. .

. ~ ' . ' '

- --- ------- --------------------------------- - - - - -------- --~---- - - ~~-- - - -- - -UNCLASSIF IEDNfi@"" ~


UNCLASSIFIED~iill


■ With Al

■ To Al

• Because ( of) Al ■ OODA Loop Tightening ■ Online learning ■

■ ■ ■

-- - ---- -- -• • T 1 ( • • • • ; • ~ 1 , .\ • • t 1

1 [ , , -. ·~ , f l l > • • ( 1 . ( • , \ c I . 1 1 1 • \ ~ i .\

- --- - -- - ~ - - - -

UNCLASSIFIEDHF~~(}


UNCLASSIFIE LM Cl£ ►


•With Al • Autonomous weapons • Psyops

■ To Al ■ Reveal the wrong thing 11 Do the wrong thing ■ Learn the wrong thing ■ Value the wrong thing

■ Because ( of) Al • OODA Loop Tightening ■ Online learning

• ? ••••

-------~~---- ----- -----~~ -~- - - - -- - ---- --· ------' • ' ) • I

--- --- ----- ---- - ------- ------- - --- - - ~----- - --- -- -UNCLASSIFIEDh' FH 15 8

epic.org epic-19-09-11-nscai-foia-20200723-odni-production

Documents