race against the machine - mcubed ai london...13 © hisolutions 2018 infosec in a nutshell confi-...

1

Race Against the Machine

M³ London October 16th, 2018

Will AI Help Or Harm Security?

David Fuhr

22 © HiSolutions 2018

• Maths

• Crypto(graphy)

• InfoSec

• Gestalt/Coaching

David FuhrHead of Research, HiSolutions AG


DATA SCIENTIST / AI RESEARCHER

99 © HiSolutions 2018blogs.balbix.com

1010 © HiSolutions 2018blogs.balbix.com


www.datasciencecentral.com


Man vs. Machine

…threatens Human Machine

HumanCivil/Military

SecurityInfoSec ……

Cyberwar

Machine Safety War of Machines

[Liggesmeyer 2015]


InfoSec in a Nutshell Confi-dentia

-lity

Inte-grity

InfoSec

Avai-lability

Goal (Why): Protect CIA triad

How?

(Risk) Management System

(PDCA Cycle, saturation curve, dynamics)

Looong list of controls (preventive, detective, corrective)

Lots of folklore,

drinking,

bragging, and

crystal balling


AI Security?

AI for Security

Security of/for AI

Security from/against AI

Security because of / thanks to AI

AI against Security / Security in spite of AI

…?


Man vs. AI vs. Machine

…threatens Human AI Machine

Human Civil/Military Security AI-Sec InfoSec

AI AI Safety Adversarial Sec AI

Machine Safety (e.g. Safety AI) (AI-Sec) War of Machines


Adversarial: AI vs. AI

Sparring: GANs (Generative Adversarial Networks, 2014)

Fight: CGC (DARPA Cyber Grand Challenge 2016)


AI-Sec: Humans vs. AI

Humans (or nature) trying to harm a piece of software

(on purpose or bad luck (e.g., fat finger))

This we know!

See InfoSec


AI-Sec: Humans vs. AI

Availability: Depending on (Cloud) resources, model parameters, data

Confidentiality: Trade secrets in models

Integrity:

Manipulation of evaluation

Manipulation of models

Manipulation of data

Manipulation of AI stacks (source code, binaries)

Manipulation of supply chain

Let “us” tell you: It’s all going to happen.


Sec-AI: AI vs. Machines

Offensive AI

Defensive AI

https://xkcd.com/


Incorrect View of InfoSec (Dullien 2017)

Thomas Dullien 2017, https://doc.dustri.org/keynotes/Machine%20Learning,%20Offense,%20and%20the%20future%20of%20Automation%20-%20Halvar%20Flake%20-%20ZeroNights%202017.pdf

https://doc.dustri.org/keynotes/Machine Learning, Offense, and the future of Automation - Halvar Flake - ZeroNights 2017.pdf


More Realistic View of InfoSec (Dullien 2017)

Thomas Dullien 2017, https://doc.dustri.org/keynotes/Machine%20Learning,%20Offense,%20and%20the%20future%20of%20Automation%20-%20Halvar%20Flake%20-%20ZeroNights%202017.pdf

https://doc.dustri.org/keynotes/Machine Learning, Offense, and the future of Automation - Halvar Flake - ZeroNights 2017.pdf


The Good, the Bad & the Ugly

Task Today Future Action

SPAM detection Near perfect SPAM evasion might win Learn about useful/fruitful content

Virus detection Mostly non-AI Not to change that fast (what is

„evil“ behavior?)

Wrong idea anyways ;-)

Whitelisting, hardening,

true software engineering

„Anomaly

detection“

AI marketing hype Will work in simple/strict

environments

Ditto

Vuln scanning Some AI hype Mostly useless (hacking is

about exploiting minor glitches)

Can work on a macro level

Attribution „It was the

Russinese!“

Please don‘t. Forget about it!

Config

Management

Non-AI Promising („most servers that

were not hacked did X“)

Start doing Config management

Use AI to make it cooler

Other What is AI? Lots of hidden wins Start researching


AI Safety: AI vs. Humans

- Opacity (vs. Transparency)

- Bias

- Singularity


AI Safety: AI vs. Humans

- Opacity (vs. Transparency)

- Transparency as crucial for democracy: Trust, Accountability

- Also a chance?

- Bias

- Cannot be avoided (part of culture), but:

- We need to stay fluid vs. power

- Stakeholder problem (bias in professional field)

- Always ask and invite those discriminated against

- Singularity

- Actually a scale

- Start researching and mitigating early(!!!)


Who Will Win?

Attacker or Defender?

In (pre AI) InfoSec:

It depends.

Used to say: attacker

New insight:

locally: attacker

globally: defender

but: cyberwar

https://xkcd.com/


Who Will Win with AI / Post-AI?

Defenders need to keep wining (statistically, without black swans)

New type of defenders and defenses needed

More research necessary

https://xkcd.com/


Man vs. AI vs. Machine

…threatens Human AI Machine

Human Civil/Military SecurityAI-Sec

- New Attack VectorsInfoSec

AI

AI Safety

- Opacity

- Bias

- Singularity

Adversarial:

- GANs

- CGC

Sec AI

- Offensive AI

- Defensive AI

MachineSafety

(e.g. Safety AI)(AI-Sec) War of Machines


Lessons To Be (Deeply) Learned

We (AI & InfoSec communities) need to talk.

Now.

Learn about

Threat Modeling

Attacks/Attack vectors

Risk Analysis and Risk Management

Security by Design, Security by Default

Accountability

Transparency

And have fun doing it!


Bouchéstraße 12 | 12435 Berlin

[email protected] | +49 30 533 289 0

www.hisolutions.com

Thanks! Questions?

David [email protected]

race against the machine - mcubed ai london...13 © hisolutions 2018 infosec in a nutshell confi-...

Documents