![Page 1: Social Engineering at Scale - Pwn2Own Engineering at... · 2018-03-29 · Social Engineering at Scale SJ, CanSecWest 2018 @bodaceacat “The views and opinions expressed in this presentation](https://reader033.vdocument.in/reader033/viewer/2022050306/5f6f160341febc4d366efdc6/html5/thumbnails/1.jpg)
Social Engineering at
ScaleSJ, CanSecWest 2018
@bodaceacat
“The views and opinions expressed in this presentation are mine and do not necessarily reflect the
official policy or position of AppNexus. Assumptions made and conclusions drawn in this presentation
are not reflective of the position of AppNexus.”
1
![Page 2: Social Engineering at Scale - Pwn2Own Engineering at... · 2018-03-29 · Social Engineering at Scale SJ, CanSecWest 2018 @bodaceacat “The views and opinions expressed in this presentation](https://reader033.vdocument.in/reader033/viewer/2022050306/5f6f160341febc4d366efdc6/html5/thumbnails/2.jpg)
OUTLINE
❑ Social engineering at scale
❑ Responses
❑ Useful earlier work
❑ This changes internet?
❑ Now what do we do?2
![Page 3: Social Engineering at Scale - Pwn2Own Engineering at... · 2018-03-29 · Social Engineering at Scale SJ, CanSecWest 2018 @bodaceacat “The views and opinions expressed in this presentation](https://reader033.vdocument.in/reader033/viewer/2022050306/5f6f160341febc4d366efdc6/html5/thumbnails/3.jpg)
SOCIAL ENGINEERING
AT SCALEFacebook group total_shares interactions number
of postswordcount
Facebook.com/Blacktivists 103,767,792 6,182,835 500+ 15349
Facebook.com/Txrebels 102,950,151 3,453,143 500+
Facebook.Com/MuslimAmerica 71,355,895 2,128,875 500+ 6892
Facebook.Com/Patriototus 51,139,860 4,438,745 500+ 53782
Facebook.Com/Secured.Borders 5,600,136 1,592,771 500+ 53850
Facebook.Com/Lgtbun 5,187,494 1,262,386 500+ 15020
3
![Page 4: Social Engineering at Scale - Pwn2Own Engineering at... · 2018-03-29 · Social Engineering at Scale SJ, CanSecWest 2018 @bodaceacat “The views and opinions expressed in this presentation](https://reader033.vdocument.in/reader033/viewer/2022050306/5f6f160341febc4d366efdc6/html5/thumbnails/4.jpg)
ECOSYSTEM: INTERNET
4
![Page 5: Social Engineering at Scale - Pwn2Own Engineering at... · 2018-03-29 · Social Engineering at Scale SJ, CanSecWest 2018 @bodaceacat “The views and opinions expressed in this presentation](https://reader033.vdocument.in/reader033/viewer/2022050306/5f6f160341febc4d366efdc6/html5/thumbnails/5.jpg)
ENDPOINTS: WETWARE
5
![Page 6: Social Engineering at Scale - Pwn2Own Engineering at... · 2018-03-29 · Social Engineering at Scale SJ, CanSecWest 2018 @bodaceacat “The views and opinions expressed in this presentation](https://reader033.vdocument.in/reader033/viewer/2022050306/5f6f160341febc4d366efdc6/html5/thumbnails/6.jpg)
VULNERABILITIES
6
![Page 7: Social Engineering at Scale - Pwn2Own Engineering at... · 2018-03-29 · Social Engineering at Scale SJ, CanSecWest 2018 @bodaceacat “The views and opinions expressed in this presentation](https://reader033.vdocument.in/reader033/viewer/2022050306/5f6f160341febc4d366efdc6/html5/thumbnails/7.jpg)
COMMONLY USED
• Imperfect recall
•Unconscious bias
•Confirmation bias
•Mental immune systems
•Familiarity backfire effect
•Memory traces
•Emotions = stronger traces
7
![Page 8: Social Engineering at Scale - Pwn2Own Engineering at... · 2018-03-29 · Social Engineering at Scale SJ, CanSecWest 2018 @bodaceacat “The views and opinions expressed in this presentation](https://reader033.vdocument.in/reader033/viewer/2022050306/5f6f160341febc4d366efdc6/html5/thumbnails/8.jpg)
PAYLOADS
Misinformation Disinformation Abuse “Normal”
Accidental / sloppy
untruths
Deliberate /
focussed
untruths
Focussed opinion
/ facts Hate
speechBullying
Doxxing
Everything
else
8
![Page 9: Social Engineering at Scale - Pwn2Own Engineering at... · 2018-03-29 · Social Engineering at Scale SJ, CanSecWest 2018 @bodaceacat “The views and opinions expressed in this presentation](https://reader033.vdocument.in/reader033/viewer/2022050306/5f6f160341febc4d366efdc6/html5/thumbnails/9.jpg)
CHANNELS
9
![Page 10: Social Engineering at Scale - Pwn2Own Engineering at... · 2018-03-29 · Social Engineering at Scale SJ, CanSecWest 2018 @bodaceacat “The views and opinions expressed in this presentation](https://reader033.vdocument.in/reader033/viewer/2022050306/5f6f160341febc4d366efdc6/html5/thumbnails/10.jpg)
ACTORS AND AIMS
10
![Page 11: Social Engineering at Scale - Pwn2Own Engineering at... · 2018-03-29 · Social Engineering at Scale SJ, CanSecWest 2018 @bodaceacat “The views and opinions expressed in this presentation](https://reader033.vdocument.in/reader033/viewer/2022050306/5f6f160341febc4d366efdc6/html5/thumbnails/11.jpg)
Let’s make that concrete...
11
![Page 12: Social Engineering at Scale - Pwn2Own Engineering at... · 2018-03-29 · Social Engineering at Scale SJ, CanSecWest 2018 @bodaceacat “The views and opinions expressed in this presentation](https://reader033.vdocument.in/reader033/viewer/2022050306/5f6f160341febc4d366efdc6/html5/thumbnails/12.jpg)
RESPONSES
12
![Page 13: Social Engineering at Scale - Pwn2Own Engineering at... · 2018-03-29 · Social Engineering at Scale SJ, CanSecWest 2018 @bodaceacat “The views and opinions expressed in this presentation](https://reader033.vdocument.in/reader033/viewer/2022050306/5f6f160341febc4d366efdc6/html5/thumbnails/13.jpg)
CRISISMAPPERS
13
![Page 14: Social Engineering at Scale - Pwn2Own Engineering at... · 2018-03-29 · Social Engineering at Scale SJ, CanSecWest 2018 @bodaceacat “The views and opinions expressed in this presentation](https://reader033.vdocument.in/reader033/viewer/2022050306/5f6f160341febc4d366efdc6/html5/thumbnails/14.jpg)
ADTECH
14
![Page 15: Social Engineering at Scale - Pwn2Own Engineering at... · 2018-03-29 · Social Engineering at Scale SJ, CanSecWest 2018 @bodaceacat “The views and opinions expressed in this presentation](https://reader033.vdocument.in/reader033/viewer/2022050306/5f6f160341febc4d366efdc6/html5/thumbnails/15.jpg)
MEDIA / JOURNALISTS
15
![Page 16: Social Engineering at Scale - Pwn2Own Engineering at... · 2018-03-29 · Social Engineering at Scale SJ, CanSecWest 2018 @bodaceacat “The views and opinions expressed in this presentation](https://reader033.vdocument.in/reader033/viewer/2022050306/5f6f160341febc4d366efdc6/html5/thumbnails/16.jpg)
POLITICS I
16
![Page 17: Social Engineering at Scale - Pwn2Own Engineering at... · 2018-03-29 · Social Engineering at Scale SJ, CanSecWest 2018 @bodaceacat “The views and opinions expressed in this presentation](https://reader033.vdocument.in/reader033/viewer/2022050306/5f6f160341febc4d366efdc6/html5/thumbnails/17.jpg)
POLITICS II
17
![Page 18: Social Engineering at Scale - Pwn2Own Engineering at... · 2018-03-29 · Social Engineering at Scale SJ, CanSecWest 2018 @bodaceacat “The views and opinions expressed in this presentation](https://reader033.vdocument.in/reader033/viewer/2022050306/5f6f160341febc4d366efdc6/html5/thumbnails/18.jpg)
POLITICS III
18
![Page 19: Social Engineering at Scale - Pwn2Own Engineering at... · 2018-03-29 · Social Engineering at Scale SJ, CanSecWest 2018 @bodaceacat “The views and opinions expressed in this presentation](https://reader033.vdocument.in/reader033/viewer/2022050306/5f6f160341febc4d366efdc6/html5/thumbnails/19.jpg)
SOCIAL NETWORKS
19
![Page 20: Social Engineering at Scale - Pwn2Own Engineering at... · 2018-03-29 · Social Engineering at Scale SJ, CanSecWest 2018 @bodaceacat “The views and opinions expressed in this presentation](https://reader033.vdocument.in/reader033/viewer/2022050306/5f6f160341febc4d366efdc6/html5/thumbnails/20.jpg)
HACKERS
Build stuff! Add extra machine learning!
20
![Page 21: Social Engineering at Scale - Pwn2Own Engineering at... · 2018-03-29 · Social Engineering at Scale SJ, CanSecWest 2018 @bodaceacat “The views and opinions expressed in this presentation](https://reader033.vdocument.in/reader033/viewer/2022050306/5f6f160341febc4d366efdc6/html5/thumbnails/21.jpg)
USEFUL EARLIER WORK
(Those Darned Telegraph
Operators…)
21
![Page 22: Social Engineering at Scale - Pwn2Own Engineering at... · 2018-03-29 · Social Engineering at Scale SJ, CanSecWest 2018 @bodaceacat “The views and opinions expressed in this presentation](https://reader033.vdocument.in/reader033/viewer/2022050306/5f6f160341febc4d366efdc6/html5/thumbnails/22.jpg)
BIG DATA’S FOURTH V
22
![Page 23: Social Engineering at Scale - Pwn2Own Engineering at... · 2018-03-29 · Social Engineering at Scale SJ, CanSecWest 2018 @bodaceacat “The views and opinions expressed in this presentation](https://reader033.vdocument.in/reader033/viewer/2022050306/5f6f160341febc4d366efdc6/html5/thumbnails/23.jpg)
WIKIPEDIA
23
![Page 24: Social Engineering at Scale - Pwn2Own Engineering at... · 2018-03-29 · Social Engineering at Scale SJ, CanSecWest 2018 @bodaceacat “The views and opinions expressed in this presentation](https://reader033.vdocument.in/reader033/viewer/2022050306/5f6f160341febc4d366efdc6/html5/thumbnails/24.jpg)
HACKERS, OSINT
24
![Page 25: Social Engineering at Scale - Pwn2Own Engineering at... · 2018-03-29 · Social Engineering at Scale SJ, CanSecWest 2018 @bodaceacat “The views and opinions expressed in this presentation](https://reader033.vdocument.in/reader033/viewer/2022050306/5f6f160341febc4d366efdc6/html5/thumbnails/25.jpg)
AI MODEL POISONING
● Bad inputs
○ Biased classifications
○ Missing demographics
● Bad models
○ Unclean inputs, assumptions etc
○ Lazy interpretations (eg. clicks == interest)
○ Trained once in a changing world
● Willful abuse
○ gaming with ‘wrong’ data (propaganda etc)
○ gaming with adversarial data
25
![Page 26: Social Engineering at Scale - Pwn2Own Engineering at... · 2018-03-29 · Social Engineering at Scale SJ, CanSecWest 2018 @bodaceacat “The views and opinions expressed in this presentation](https://reader033.vdocument.in/reader033/viewer/2022050306/5f6f160341febc4d366efdc6/html5/thumbnails/26.jpg)
AI MODELLING HUMANS
26
![Page 27: Social Engineering at Scale - Pwn2Own Engineering at... · 2018-03-29 · Social Engineering at Scale SJ, CanSecWest 2018 @bodaceacat “The views and opinions expressed in this presentation](https://reader033.vdocument.in/reader033/viewer/2022050306/5f6f160341febc4d366efdc6/html5/thumbnails/27.jpg)
THIS CHANGES INTERNET?
27
![Page 28: Social Engineering at Scale - Pwn2Own Engineering at... · 2018-03-29 · Social Engineering at Scale SJ, CanSecWest 2018 @bodaceacat “The views and opinions expressed in this presentation](https://reader033.vdocument.in/reader033/viewer/2022050306/5f6f160341febc4d366efdc6/html5/thumbnails/28.jpg)
PEOPLE SHIFTS
28
![Page 29: Social Engineering at Scale - Pwn2Own Engineering at... · 2018-03-29 · Social Engineering at Scale SJ, CanSecWest 2018 @bodaceacat “The views and opinions expressed in this presentation](https://reader033.vdocument.in/reader033/viewer/2022050306/5f6f160341febc4d366efdc6/html5/thumbnails/29.jpg)
BOTS AREN’T GOING
AWAY
29
![Page 30: Social Engineering at Scale - Pwn2Own Engineering at... · 2018-03-29 · Social Engineering at Scale SJ, CanSecWest 2018 @bodaceacat “The views and opinions expressed in this presentation](https://reader033.vdocument.in/reader033/viewer/2022050306/5f6f160341febc4d366efdc6/html5/thumbnails/30.jpg)
FUNDING SHIFTS
30
![Page 31: Social Engineering at Scale - Pwn2Own Engineering at... · 2018-03-29 · Social Engineering at Scale SJ, CanSecWest 2018 @bodaceacat “The views and opinions expressed in this presentation](https://reader033.vdocument.in/reader033/viewer/2022050306/5f6f160341febc4d366efdc6/html5/thumbnails/31.jpg)
NEW HACK LAYER/
INDUSTRY FORMING
31
![Page 32: Social Engineering at Scale - Pwn2Own Engineering at... · 2018-03-29 · Social Engineering at Scale SJ, CanSecWest 2018 @bodaceacat “The views and opinions expressed in this presentation](https://reader033.vdocument.in/reader033/viewer/2022050306/5f6f160341febc4d366efdc6/html5/thumbnails/32.jpg)
TRUST AS A COMMODITY
32
![Page 33: Social Engineering at Scale - Pwn2Own Engineering at... · 2018-03-29 · Social Engineering at Scale SJ, CanSecWest 2018 @bodaceacat “The views and opinions expressed in this presentation](https://reader033.vdocument.in/reader033/viewer/2022050306/5f6f160341febc4d366efdc6/html5/thumbnails/33.jpg)
NOW WHAT DO WE DO?
33
![Page 34: Social Engineering at Scale - Pwn2Own Engineering at... · 2018-03-29 · Social Engineering at Scale SJ, CanSecWest 2018 @bodaceacat “The views and opinions expressed in this presentation](https://reader033.vdocument.in/reader033/viewer/2022050306/5f6f160341febc4d366efdc6/html5/thumbnails/34.jpg)
COEXIST WITH BOTS
34
![Page 35: Social Engineering at Scale - Pwn2Own Engineering at... · 2018-03-29 · Social Engineering at Scale SJ, CanSecWest 2018 @bodaceacat “The views and opinions expressed in this presentation](https://reader033.vdocument.in/reader033/viewer/2022050306/5f6f160341febc4d366efdc6/html5/thumbnails/35.jpg)
USE CRISIS MANAGEMENT
• Prevention - e.g. change structures
• Protection - e.g. set boundaries
• Mitigation - e.g. set alerts and responses
• Response - e.g. remove bots
• Recovery - e.g. rebuild community, trust
35
![Page 36: Social Engineering at Scale - Pwn2Own Engineering at... · 2018-03-29 · Social Engineering at Scale SJ, CanSecWest 2018 @bodaceacat “The views and opinions expressed in this presentation](https://reader033.vdocument.in/reader033/viewer/2022050306/5f6f160341febc4d366efdc6/html5/thumbnails/36.jpg)
STOP JUST REACTING
36
![Page 37: Social Engineering at Scale - Pwn2Own Engineering at... · 2018-03-29 · Social Engineering at Scale SJ, CanSecWest 2018 @bodaceacat “The views and opinions expressed in this presentation](https://reader033.vdocument.in/reader033/viewer/2022050306/5f6f160341febc4d366efdc6/html5/thumbnails/37.jpg)
DESIGN FOR BELIEF
HACKING
Set policies (cf spam and hate speech
policies)
Remove incentives (attention, money, effects)
Design for communities
Build protections into systems
37
![Page 38: Social Engineering at Scale - Pwn2Own Engineering at... · 2018-03-29 · Social Engineering at Scale SJ, CanSecWest 2018 @bodaceacat “The views and opinions expressed in this presentation](https://reader033.vdocument.in/reader033/viewer/2022050306/5f6f160341febc4d366efdc6/html5/thumbnails/38.jpg)
RESPOND AT
APPROPRIATE SCALE
• Nationstate
• Platform
• Organisation
• Community
• Individual
38
![Page 39: Social Engineering at Scale - Pwn2Own Engineering at... · 2018-03-29 · Social Engineering at Scale SJ, CanSecWest 2018 @bodaceacat “The views and opinions expressed in this presentation](https://reader033.vdocument.in/reader033/viewer/2022050306/5f6f160341febc4d366efdc6/html5/thumbnails/39.jpg)
SOME PRACTICAL STUFF
39
![Page 40: Social Engineering at Scale - Pwn2Own Engineering at... · 2018-03-29 · Social Engineering at Scale SJ, CanSecWest 2018 @bodaceacat “The views and opinions expressed in this presentation](https://reader033.vdocument.in/reader033/viewer/2022050306/5f6f160341febc4d366efdc6/html5/thumbnails/40.jpg)
DO DO THIS AT HOME
● https://twitter.com/probabot_ - BotOrNot
● https://data.world/d1gi/ - Troll tweets, facebook, youtube
● http://library.sewanee.edu/fakenews - ‘fakenews’ sites list
40
![Page 41: Social Engineering at Scale - Pwn2Own Engineering at... · 2018-03-29 · Social Engineering at Scale SJ, CanSecWest 2018 @bodaceacat “The views and opinions expressed in this presentation](https://reader033.vdocument.in/reader033/viewer/2022050306/5f6f160341febc4d366efdc6/html5/thumbnails/41.jpg)
BOT/TROLL HUNTING
41
![Page 42: Social Engineering at Scale - Pwn2Own Engineering at... · 2018-03-29 · Social Engineering at Scale SJ, CanSecWest 2018 @bodaceacat “The views and opinions expressed in this presentation](https://reader033.vdocument.in/reader033/viewer/2022050306/5f6f160341febc4d366efdc6/html5/thumbnails/42.jpg)
RED BOT/ TROLL
• Evasion:
• stagger creation dates
• buy or take over old dead accounts
• use dictionary or NLP to vary output text
• use api to push tweets out in e.g. Russian nighttime
• test variants on e.g. botornot
• Manipulation:
• Find high-volume, high-sentiment topics
• Microtarget (e.g. adint), and wipe artefacts immediately
42
![Page 43: Social Engineering at Scale - Pwn2Own Engineering at... · 2018-03-29 · Social Engineering at Scale SJ, CanSecWest 2018 @bodaceacat “The views and opinions expressed in this presentation](https://reader033.vdocument.in/reader033/viewer/2022050306/5f6f160341febc4d366efdc6/html5/thumbnails/43.jpg)
DISINFORMATION
HUNTING
• “Content” signals (cf “context” signals)
• Repeated information (cut n pastes)
• Bots repeating or pointing at it!
• No (or very little) original text
• Large numbers of ad slots
• Clickbait words
43
![Page 44: Social Engineering at Scale - Pwn2Own Engineering at... · 2018-03-29 · Social Engineering at Scale SJ, CanSecWest 2018 @bodaceacat “The views and opinions expressed in this presentation](https://reader033.vdocument.in/reader033/viewer/2022050306/5f6f160341febc4d366efdc6/html5/thumbnails/44.jpg)
RED DISINFORMATION
• Vary text (enough to defeat shingling)
• Mix nudges with truths
• Context matters. Format matters
• Talk about something else goddammit
• Microtarget: use network analysis and probes
• Keep using current vulnerabilities
44
![Page 45: Social Engineering at Scale - Pwn2Own Engineering at... · 2018-03-29 · Social Engineering at Scale SJ, CanSecWest 2018 @bodaceacat “The views and opinions expressed in this presentation](https://reader033.vdocument.in/reader033/viewer/2022050306/5f6f160341febc4d366efdc6/html5/thumbnails/45.jpg)
Social Engineering at
ScaleSJ, CanSecWest 2018
@bodaceacat
45