digital forensics as a service - dfrws€¦ · digital forensics as a service (since q4 2010) xiraf...

62
Digital Forensics as a Service: an update Harm van Beek PhD [email protected]

Upload: others

Post on 18-Jun-2020

3 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Digital Forensics as a Service - DFRWS€¦ · Digital Forensics as a Service (since Q4 2010) XIRAF / Hansken > 600 cases > 10,000 devices > 1.5 PB data > 2,500 investigators . all

Digital Forensics as a Service:

an update

Harm van Beek PhD [email protected]

Page 2: Digital Forensics as a Service - DFRWS€¦ · Digital Forensics as a Service (since Q4 2010) XIRAF / Hansken > 600 cases > 10,000 devices > 1.5 PB data > 2,500 investigators . all

Traditional digital investigation

Tactical investigator

Analyst

Seized material

Data to examine

Digital investigator

Page 3: Digital Forensics as a Service - DFRWS€¦ · Digital Forensics as a Service (since Q4 2010) XIRAF / Hansken > 600 cases > 10,000 devices > 1.5 PB data > 2,500 investigators . all

right information at the

right time to the

right people

Page 4: Digital Forensics as a Service - DFRWS€¦ · Digital Forensics as a Service (since Q4 2010) XIRAF / Hansken > 600 cases > 10,000 devices > 1.5 PB data > 2,500 investigators . all

Digital Forensics as a Service (since Q4 2010)

XIRAF / Hansken

Page 5: Digital Forensics as a Service - DFRWS€¦ · Digital Forensics as a Service (since Q4 2010) XIRAF / Hansken > 600 cases > 10,000 devices > 1.5 PB data > 2,500 investigators . all

> 600 cases

> 10,000 devices

> 1.5 PB data

> 2,500 investigators all (regional) Dutch police forces National High Tech Crime Unit

RST Former Dutch Antilles Toronto Police

Page 6: Digital Forensics as a Service - DFRWS€¦ · Digital Forensics as a Service (since Q4 2010) XIRAF / Hansken > 600 cases > 10,000 devices > 1.5 PB data > 2,500 investigators . all

The six main lessons learned

“Experience is a hard teacher because she gives the test first, the lesson afterwards.”

-- Vernon Sanders Law

Page 7: Digital Forensics as a Service - DFRWS€¦ · Digital Forensics as a Service (since Q4 2010) XIRAF / Hansken > 600 cases > 10,000 devices > 1.5 PB data > 2,500 investigators . all

Lesson one How to process a lot of data?

Bring computing power to the data

Page 8: Digital Forensics as a Service - DFRWS€¦ · Digital Forensics as a Service (since Q4 2010) XIRAF / Hansken > 600 cases > 10,000 devices > 1.5 PB data > 2,500 investigators . all
Page 9: Digital Forensics as a Service - DFRWS€¦ · Digital Forensics as a Service (since Q4 2010) XIRAF / Hansken > 600 cases > 10,000 devices > 1.5 PB data > 2,500 investigators . all

CPU

CP

U

CP

U

CP

U

CP

U

CP

U

CPU

CPU

Page 10: Digital Forensics as a Service - DFRWS€¦ · Digital Forensics as a Service (since Q4 2010) XIRAF / Hansken > 600 cases > 10,000 devices > 1.5 PB data > 2,500 investigators . all

DATA

DATA

DATA

DATA

DATA

DATA

DATA

DATA CPU

CP

U

CP

U

CP

U

CP

U

CP

U

CPU

CPU

Page 11: Digital Forensics as a Service - DFRWS€¦ · Digital Forensics as a Service (since Q4 2010) XIRAF / Hansken > 600 cases > 10,000 devices > 1.5 PB data > 2,500 investigators . all

DATA

DATA

DATA

DATA

DATA

DATA

DATA

DATA

DATA

CPU

CP

U

CP

U

CP

U

CP

U

CP

U

CPU

CPU

Page 12: Digital Forensics as a Service - DFRWS€¦ · Digital Forensics as a Service (since Q4 2010) XIRAF / Hansken > 600 cases > 10,000 devices > 1.5 PB data > 2,500 investigators . all

DATA

DATA

DATA

DATA

DATA

DATA

DATA

DATA

DATA

CPU

CP

U

CP

U

CP

U

CP

U

CP

U

CPU

CPU

Page 13: Digital Forensics as a Service - DFRWS€¦ · Digital Forensics as a Service (since Q4 2010) XIRAF / Hansken > 600 cases > 10,000 devices > 1.5 PB data > 2,500 investigators . all

DATA +

CPUs CPU

CP

U

CP

U

CP

U

CP

U

CP

U

CPU

CPU

Page 14: Digital Forensics as a Service - DFRWS€¦ · Digital Forensics as a Service (since Q4 2010) XIRAF / Hansken > 600 cases > 10,000 devices > 1.5 PB data > 2,500 investigators . all

CPU

CP

U

CP

U

CP

U

CP

U

CP

U

CPU

CPU

DATA +

CPUs

Page 15: Digital Forensics as a Service - DFRWS€¦ · Digital Forensics as a Service (since Q4 2010) XIRAF / Hansken > 600 cases > 10,000 devices > 1.5 PB data > 2,500 investigators . all

DATA +

CPUs

Page 16: Digital Forensics as a Service - DFRWS€¦ · Digital Forensics as a Service (since Q4 2010) XIRAF / Hansken > 600 cases > 10,000 devices > 1.5 PB data > 2,500 investigators . all

Logging

Google

MapReduce extraction process

LinkedIn

Kafka queue

Twitter

Storm processing

Elastic

Elastic search

search

Elastic

Elastic search

search

Hadoop

HDFS storage

RESTful web API

Facebook Cassandra anonimisation

Facebook Cassandra administration

Websites digital tactical

Python API digital

Page 17: Digital Forensics as a Service - DFRWS€¦ · Digital Forensics as a Service (since Q4 2010) XIRAF / Hansken > 600 cases > 10,000 devices > 1.5 PB data > 2,500 investigators . all

Logging

Google

MapReduce extraction process

LinkedIn

Kafka queue

Twitter

Storm processing

Elastic

Elastic search

search

Elastic

Elastic search

search

Hadoop

HDFS storage

RESTful web API

Facebook Cassandra anonimisation

Facebook Cassandra administration

Websites digital tactical

Python API digital

Google

MapReduce extraction process

Hadoop

HDFS storage

Page 18: Digital Forensics as a Service - DFRWS€¦ · Digital Forensics as a Service (since Q4 2010) XIRAF / Hansken > 600 cases > 10,000 devices > 1.5 PB data > 2,500 investigators . all

Lesson two How to provide this service?

We’re not all digital investigators

Do what you are good at!

Page 19: Digital Forensics as a Service - DFRWS€¦ · Digital Forensics as a Service (since Q4 2010) XIRAF / Hansken > 600 cases > 10,000 devices > 1.5 PB data > 2,500 investigators . all

forensic software developers

platform developers

front-end developers

python developer

system administrators

software architect

quality engineers

operators

operational support

project leader

forensic scientists

Team of specialists

Page 20: Digital Forensics as a Service - DFRWS€¦ · Digital Forensics as a Service (since Q4 2010) XIRAF / Hansken > 600 cases > 10,000 devices > 1.5 PB data > 2,500 investigators . all

forensic software developers

platform developers

front-end developers

python developer

system administrators

quality engineers

operators

operational support

project leader

forensic scientists

Team of specialists

Page 21: Digital Forensics as a Service - DFRWS€¦ · Digital Forensics as a Service (since Q4 2010) XIRAF / Hansken > 600 cases > 10,000 devices > 1.5 PB data > 2,500 investigators . all

Lesson three Can we trust the service?

Test, test… and test!

Page 22: Digital Forensics as a Service - DFRWS€¦ · Digital Forensics as a Service (since Q4 2010) XIRAF / Hansken > 600 cases > 10,000 devices > 1.5 PB data > 2,500 investigators . all

Current test set > 7,700 unit tests

> 12,500 integration tests

if 1 test fails,

the code is not accepted (by the development platform)

Page 23: Digital Forensics as a Service - DFRWS€¦ · Digital Forensics as a Service (since Q4 2010) XIRAF / Hansken > 600 cases > 10,000 devices > 1.5 PB data > 2,500 investigators . all

Lesson four How to represent the results?

Use a uniform data model

Page 24: Digital Forensics as a Service - DFRWS€¦ · Digital Forensics as a Service (since Q4 2010) XIRAF / Hansken > 600 cases > 10,000 devices > 1.5 PB data > 2,500 investigators . all
Page 25: Digital Forensics as a Service - DFRWS€¦ · Digital Forensics as a Service (since Q4 2010) XIRAF / Hansken > 600 cases > 10,000 devices > 1.5 PB data > 2,500 investigators . all
Page 26: Digital Forensics as a Service - DFRWS€¦ · Digital Forensics as a Service (since Q4 2010) XIRAF / Hansken > 600 cases > 10,000 devices > 1.5 PB data > 2,500 investigators . all

Lesson five How to present the results?

Listen to your colleagues

Page 27: Digital Forensics as a Service - DFRWS€¦ · Digital Forensics as a Service (since Q4 2010) XIRAF / Hansken > 600 cases > 10,000 devices > 1.5 PB data > 2,500 investigators . all
Page 28: Digital Forensics as a Service - DFRWS€¦ · Digital Forensics as a Service (since Q4 2010) XIRAF / Hansken > 600 cases > 10,000 devices > 1.5 PB data > 2,500 investigators . all
Page 29: Digital Forensics as a Service - DFRWS€¦ · Digital Forensics as a Service (since Q4 2010) XIRAF / Hansken > 600 cases > 10,000 devices > 1.5 PB data > 2,500 investigators . all
Page 30: Digital Forensics as a Service - DFRWS€¦ · Digital Forensics as a Service (since Q4 2010) XIRAF / Hansken > 600 cases > 10,000 devices > 1.5 PB data > 2,500 investigators . all
Page 31: Digital Forensics as a Service - DFRWS€¦ · Digital Forensics as a Service (since Q4 2010) XIRAF / Hansken > 600 cases > 10,000 devices > 1.5 PB data > 2,500 investigators . all
Page 32: Digital Forensics as a Service - DFRWS€¦ · Digital Forensics as a Service (since Q4 2010) XIRAF / Hansken > 600 cases > 10,000 devices > 1.5 PB data > 2,500 investigators . all
Page 33: Digital Forensics as a Service - DFRWS€¦ · Digital Forensics as a Service (since Q4 2010) XIRAF / Hansken > 600 cases > 10,000 devices > 1.5 PB data > 2,500 investigators . all
Page 34: Digital Forensics as a Service - DFRWS€¦ · Digital Forensics as a Service (since Q4 2010) XIRAF / Hansken > 600 cases > 10,000 devices > 1.5 PB data > 2,500 investigators . all
Page 35: Digital Forensics as a Service - DFRWS€¦ · Digital Forensics as a Service (since Q4 2010) XIRAF / Hansken > 600 cases > 10,000 devices > 1.5 PB data > 2,500 investigators . all
Page 36: Digital Forensics as a Service - DFRWS€¦ · Digital Forensics as a Service (since Q4 2010) XIRAF / Hansken > 600 cases > 10,000 devices > 1.5 PB data > 2,500 investigators . all
Page 37: Digital Forensics as a Service - DFRWS€¦ · Digital Forensics as a Service (since Q4 2010) XIRAF / Hansken > 600 cases > 10,000 devices > 1.5 PB data > 2,500 investigators . all
Page 38: Digital Forensics as a Service - DFRWS€¦ · Digital Forensics as a Service (since Q4 2010) XIRAF / Hansken > 600 cases > 10,000 devices > 1.5 PB data > 2,500 investigators . all
Page 39: Digital Forensics as a Service - DFRWS€¦ · Digital Forensics as a Service (since Q4 2010) XIRAF / Hansken > 600 cases > 10,000 devices > 1.5 PB data > 2,500 investigators . all
Page 40: Digital Forensics as a Service - DFRWS€¦ · Digital Forensics as a Service (since Q4 2010) XIRAF / Hansken > 600 cases > 10,000 devices > 1.5 PB data > 2,500 investigators . all
Page 41: Digital Forensics as a Service - DFRWS€¦ · Digital Forensics as a Service (since Q4 2010) XIRAF / Hansken > 600 cases > 10,000 devices > 1.5 PB data > 2,500 investigators . all
Page 42: Digital Forensics as a Service - DFRWS€¦ · Digital Forensics as a Service (since Q4 2010) XIRAF / Hansken > 600 cases > 10,000 devices > 1.5 PB data > 2,500 investigators . all
Page 43: Digital Forensics as a Service - DFRWS€¦ · Digital Forensics as a Service (since Q4 2010) XIRAF / Hansken > 600 cases > 10,000 devices > 1.5 PB data > 2,500 investigators . all
Page 44: Digital Forensics as a Service - DFRWS€¦ · Digital Forensics as a Service (since Q4 2010) XIRAF / Hansken > 600 cases > 10,000 devices > 1.5 PB data > 2,500 investigators . all
Page 45: Digital Forensics as a Service - DFRWS€¦ · Digital Forensics as a Service (since Q4 2010) XIRAF / Hansken > 600 cases > 10,000 devices > 1.5 PB data > 2,500 investigators . all
Page 46: Digital Forensics as a Service - DFRWS€¦ · Digital Forensics as a Service (since Q4 2010) XIRAF / Hansken > 600 cases > 10,000 devices > 1.5 PB data > 2,500 investigators . all
Page 47: Digital Forensics as a Service - DFRWS€¦ · Digital Forensics as a Service (since Q4 2010) XIRAF / Hansken > 600 cases > 10,000 devices > 1.5 PB data > 2,500 investigators . all
Page 48: Digital Forensics as a Service - DFRWS€¦ · Digital Forensics as a Service (since Q4 2010) XIRAF / Hansken > 600 cases > 10,000 devices > 1.5 PB data > 2,500 investigators . all
Page 49: Digital Forensics as a Service - DFRWS€¦ · Digital Forensics as a Service (since Q4 2010) XIRAF / Hansken > 600 cases > 10,000 devices > 1.5 PB data > 2,500 investigators . all
Page 50: Digital Forensics as a Service - DFRWS€¦ · Digital Forensics as a Service (since Q4 2010) XIRAF / Hansken > 600 cases > 10,000 devices > 1.5 PB data > 2,500 investigators . all
Page 51: Digital Forensics as a Service - DFRWS€¦ · Digital Forensics as a Service (since Q4 2010) XIRAF / Hansken > 600 cases > 10,000 devices > 1.5 PB data > 2,500 investigators . all

Lesson six What to add next?

Follow the data

Page 52: Digital Forensics as a Service - DFRWS€¦ · Digital Forensics as a Service (since Q4 2010) XIRAF / Hansken > 600 cases > 10,000 devices > 1.5 PB data > 2,500 investigators . all

knowledge

data

traces

Page 53: Digital Forensics as a Service - DFRWS€¦ · Digital Forensics as a Service (since Q4 2010) XIRAF / Hansken > 600 cases > 10,000 devices > 1.5 PB data > 2,500 investigators . all

know- ledge

data

traces

Page 54: Digital Forensics as a Service - DFRWS€¦ · Digital Forensics as a Service (since Q4 2010) XIRAF / Hansken > 600 cases > 10,000 devices > 1.5 PB data > 2,500 investigators . all

data

applicable knowledge

Page 55: Digital Forensics as a Service - DFRWS€¦ · Digital Forensics as a Service (since Q4 2010) XIRAF / Hansken > 600 cases > 10,000 devices > 1.5 PB data > 2,500 investigators . all

data

applicable knowledge

Page 56: Digital Forensics as a Service - DFRWS€¦ · Digital Forensics as a Service (since Q4 2010) XIRAF / Hansken > 600 cases > 10,000 devices > 1.5 PB data > 2,500 investigators . all

data

applicable

knowledge

Page 57: Digital Forensics as a Service - DFRWS€¦ · Digital Forensics as a Service (since Q4 2010) XIRAF / Hansken > 600 cases > 10,000 devices > 1.5 PB data > 2,500 investigators . all

Recently added

L01 AD1 Lx01

Page 58: Digital Forensics as a Service - DFRWS€¦ · Digital Forensics as a Service (since Q4 2010) XIRAF / Hansken > 600 cases > 10,000 devices > 1.5 PB data > 2,500 investigators . all
Page 59: Digital Forensics as a Service - DFRWS€¦ · Digital Forensics as a Service (since Q4 2010) XIRAF / Hansken > 600 cases > 10,000 devices > 1.5 PB data > 2,500 investigators . all
Page 60: Digital Forensics as a Service - DFRWS€¦ · Digital Forensics as a Service (since Q4 2010) XIRAF / Hansken > 600 cases > 10,000 devices > 1.5 PB data > 2,500 investigators . all
Page 61: Digital Forensics as a Service - DFRWS€¦ · Digital Forensics as a Service (since Q4 2010) XIRAF / Hansken > 600 cases > 10,000 devices > 1.5 PB data > 2,500 investigators . all

Work in progress

Call Detail Records

Language detection

Entity extraction Network tap data

Volume shadow copies

Drone data

Page 62: Digital Forensics as a Service - DFRWS€¦ · Digital Forensics as a Service (since Q4 2010) XIRAF / Hansken > 600 cases > 10,000 devices > 1.5 PB data > 2,500 investigators . all

Hansken Netherlands Forensic Institute

Digital Forensics as a Service

A game changer Game on

Harm van Beek PhD [email protected]

dx.doi.org/10.1016/j.diin.2014.03.007 dx.doi.org/10.1016/j.diin.2015.07.004