big data, pirates and dr seuss
DESCRIPTION
Big Data seems to be all the rage these days and sooner or later business professionals will need to either use it or review its usage within the organisation. This introductory presentation for non-technical users covers Big Data fundamentals and some current and potential future applications of Big Data. The presentation provides an overview of what Big Data is, what technologies can be used to process Big Data(sets) and discuss some specific examples. The structure of the presentation is: - what is big data - big data technologies - what do all these mysterious names mean - big data examples - fighting pirates (on the high seas) and graffiti (in the city) - big data examples in audit - fraud detection, payroll recalculation - what has Dr. Seuss got to say about big dataTRANSCRIPT
Mario Bojilov Meta Business Systems
20 February 2013
Copyright (c) 2012, Meta Business Systems
What is Big Data
Volume 90% of data in the world was created in the last 2
years
Copyright (c) 2012, Meta Business Systems
What is Big Data
Volume 90% of data in the world was created in the last 2
years
2.5EB are created every day
Gigabyte (GB) - 1,024MB
Terabyte (TB) - 1,024GB
Petabyte (PB) - 1,024TB
Exabyte (EB) - 1,024PB
Copyright (c) 2012, Meta Business Systems
What is Big Data
Velocity Customer Satisfaction/Churn
Social Media
Fraud
Copyright (c) 2012, Meta Business Systems
What is Big Data
Variety Structured
Semi-structured
Unstructured
Copyright (c) 2012, Meta Business Systems
Big Data Technologies Hadoop/MapReduce
Copyright (c) 2012, Meta Business Systems
Big Data Technologies Hadoop/MapReduce
Copyright (c) 2012, Meta Business Systems
Big Data Technologies
R - free software environment for statistical computing and graphics http://www.r-project.org/
Copyright (c) 2012, Meta Business Systems
Big Data Technologies
Craigslist – Archiving
Disney – Gaming
Foursquare – Social Analytics
MTV Networks – Content Management
MongoDB (from "humongous") - a scalable, high-performance, open source, NoSQL database.
Copyright (c) 2012, Meta Business Systems
Big Data Technologies Cassandra - highly scalable, open-source, distributed, structured key-value database (developed by Facebook) eBay Netflix Twitter Reddit Digg
300TB/400
Copyright (c) 2012, Meta Business Systems
Big Data Architecture
Copyright (c) 2012, Meta Business Systems
Big Data Architecture
Copyright (c) 2012, Meta Business Systems
Big Data Examples
fighting pirates (on the high seas)
$7,000,000,000 (Source: Oceans Beyond Piracy)
Copyright (c) 2012, Meta Business Systems
Big Data Examples
fighting pirates (on the high seas)
ArcGIS
http://www.businessinsider.com/how-to-fight-pirates-with-big-data-2012-12?0=sai
Copyright (c) 2012, Meta Business Systems
Big Data Examples
fighting pirates (on the high seas)
ArcGIS
54% (International Maritime Bureau)
Copyright (c) 2012, Meta Business Systems
Big Data Examples
… and graffiti (in the city)
Status
Protest
High value “marks”
Patterns
Low mobility
Copyright (c) 2012, Meta Business Systems
Name: PW Salary: $200 Overtime: $0 Allowance: $X Leave: $Y Deduction: $Z
Name: JS Salary: $200 Overtime: $0 Allowance: $X Leave: $Y Deduction: $Z
Big Data Examples in Audit Payroll Recalculation
Copyright (c) 2012, Meta Business Systems
Name: PW Salary: $200 Overtime: $0 Allowance: $X Leave: $Y Deduction: $Z
Name: JS Salary: $200 Overtime: $0 Allowance: $X Leave: $Y Deduction: $Z
Name: MB-S Amount: $200
Name: MB-O Amount : $100
Name: PW-S Amount : $250
Name: PW-O Amount : $50
Name: JS-S Amount : $300
Name: JS-O Amount : $25
Map
Name: MB Salary: $200 Overtime: $0 Allowance: $X Leave: $Y Deduction: $Z
Name: MB Salary: $200 Overtime: $0 Allowance: $X Leave: $Y Deduction: $Z
Name: PW Salary: $200 Overtime: $0 Allowance: $X Leave: $Y Deduction: $Z
Name: JS Salary: $200 Overtime: $0 Allowance: $X Leave: $Y Deduction: $Z
Big Data Examples in Audit Payroll Recalculation
Copyright (c) 2012, Meta Business Systems
Name: PW Salary: $200 Overtime: $0 Allowance: $X Leave: $Y Deduction: $Z
Name: JS Salary: $200 Overtime: $0 Allowance: $X Leave: $Y Deduction: $Z
Name: JS Total: $3,200
Name: PW Total: $2,850
Name: MB Total: $2,500
Map
Reduce
Name: MB Salary: $200 Overtime: $50 Allowance: $X Leave: $Y Deduction: $Z
Name: MB Salary: $200 Overtime: $0 Allowance: $X Leave: $Y Deduction: $Z
Name: MB-S Amount: $200
Name: MB-O Amount : $100
Name: PW-S Amount : $250
Name: PW-O Amount : $50
Name: JS-S Amount : $300
Name: JS-O Amount : $25
Big Data Examples in Audit Payroll Recalculation
Copyright (c) 2012, Meta Business Systems
MB $2500 P1
Mr. X $5500
JS $3200
Ms. Y $1500
PW $2850
Mrs. Z $8500
Big Data Examples in Audit Payroll Recalculation
Copyright (c) 2012, Meta Business Systems
MB $2600 P1
P2
Mr. X $5500
JS $3200
Ms. Y $1500
PW $2850
Mrs. Z $8500
MB $2300
Mr. X $5500
JS $3200
Ms. Y $1500
PW $2500
Mrs. Z $8500
Big Data Examples in Audit Payroll Recalculation
Copyright (c) 2012, Meta Business Systems
MB $2600 P1
P2
P26
Mr. X $5500
JS $3200
Ms. Y $1500
PW $2850
Mrs. Z $8500
MB $2300
Mr. X $5500
JS $3400
Ms. Y $1500
PW $2500
Mrs. Z $8500
MB $3500
Mr. X $6500
JS $4000
Ms. Y $2000
PW $3000
Mrs. Z $9500
Big Data Examples in Audit Fraud Detection
Copyright (c) 2012, Meta Business Systems
Big Data Examples in Audit Fraud Detection
Copyright (c) 2012, Meta Business Systems
Name: PW Salary: $200 Overtime: $0 Allowance: $X Leave: $Y Deduction: $Z
Name: JS Salary: $200 Overtime: $0 Allowance: $X Leave: $Y Deduction: $Z
Name: PW Salary: $200 Overtime: $0 Allowance: $X Leave: $Y Deduction: $Z
Name: JS Salary: $200 Overtime: $0 Allowance: $X Leave: $Y Deduction: $Z
Name: MB-1-1 Amount: $200
Name: MB-15000-8 Amount : $100
Name: PW-3-1 Amount : $250
Name: PW-2200-5 Amount : $50
Name: JS-2-1 Amount : $300
Name: JS-14000-3 Amount : $25
Map
Name: MB Line Item 1: $200 Line Item 2 : $500 Line Item 3 : $X Line Item 4 : $Y Line Item 5 : $Z
Name: MB Line Item 1: $200 Line Item 2 : $0 Line Item 3 : $X Line Item 4 : $Y Line Item 5 : $Z
Big Data Examples in Audit Fraud Detection
Copyright (c) 2012, Meta Business Systems
Name: PW Salary: $200 Overtime: $0 Allowance: $X Leave: $Y Deduction: $Z
Name: JS Salary: $200 Overtime: $0 Allowance: $X Leave: $Y Deduction: $Z
Name: PW Salary: $200 Overtime: $0 Allowance: $X Leave: $Y Deduction: $Z
Name: JS Salary: $200 Overtime: $0 Allowance: $X Leave: $Y Deduction: $Z
Name: JS-1 Benford 1: 0.32
Name: PW-1 Benford : 0.295
Name: MB-1 Benford 1: 0.31
Map Reduce
Name: MB Line Item 1: $200 Line Item 2 : $500 Line Item 3 : $X Line Item 4 : $Y Line Item 5 : $Z
Name: MB Line Item 1: $200 Line Item 2 : $0 Line Item 3 : $X Line Item 4 : $Y Line Item 5 : $Z
Name: MB-9 Benford 9: 0.05
Name: JS-9 Benford 9: 0.051
Name: PW-9 Benford 9: 0.05
Name: JS-4 Benford 4: 0.15
Name: MB-1-1 Amount: $200
Name: MB-15000-8 Amount : $100
Name: PW-3-1 Amount : $250
Name: PW-2200-5 Amount : $50
Name: JS-2-1 Amount : $300
Name: JS-14000-3 Amount : $25
… And, Dr. Seuss ….
If you read with your eyes shut
you’re likely to find
that the place where you’re going
is far, far behind.
Copyright (c) 2012, Meta Business Systems
… And, Dr. Seuss ….
If you read with your eyes shut
you’re likely to find
that the place where you’re going
is far, far behind.
SO …
that’s why I tell you
to keep you eyes wide.
Keep them wide open …
at least on one side.
Copyright (c) 2012, Meta Business Systems
Thank You! [email protected]
@mariogb
www.linkedin.com/in/mariobojilov
big-data-for-organisational-leaders.blogspot.com.au/
www.scoop.it/t/big-data-for-organisational-leaders
Copyright (c) 2012, Meta Business Systems