big data, the end of optimisation?
DESCRIPTION
Until a few years ago the term Big Data was only known to a limited group of experts. Now it is nearly impossible to visit a website or read an article without stumbling across Big Data. A Google query results in over 1.5 billion hits in less than half a second, two years ago this was only one fifth of that number. Many times Big Data is positioned as the answer to everything. It is the end of theory as former Wired editor Chris Anderson wants us to belief. The promise of Big Data seems to imply that in the sheer size of data sets there lurks some kind of magic. Will it make Operations Research obsolete?TRANSCRIPT
![Page 1: Big data, the end of optimisation?](https://reader034.vdocument.in/reader034/viewer/2022051512/53edd9dc8d7f7289708b5ef5/html5/thumbnails/1.jpg)
Big Data The end of Optimisation
John Poppelaars Vesting Conference 2014
![Page 2: Big data, the end of optimisation?](https://reader034.vdocument.in/reader034/viewer/2022051512/53edd9dc8d7f7289708b5ef5/html5/thumbnails/2.jpg)
Big Data Analytics produces uncanny
accurate predictions
http://www.forbes.com/sites/kashmirhill/2012/02/16/how-target-figured-out-a-teen-girl-was-pregnant-before-her-father-did/
![Page 3: Big data, the end of optimisation?](https://reader034.vdocument.in/reader034/viewer/2022051512/53edd9dc8d7f7289708b5ef5/html5/thumbnails/3.jpg)
Big Data correlations tell us what we need to know
http://www.nature.com/nature/journal/v457/n7232/full/nature07634.html
![Page 4: Big data, the end of optimisation?](https://reader034.vdocument.in/reader034/viewer/2022051512/53edd9dc8d7f7289708b5ef5/html5/thumbnails/4.jpg)
No need to sample, when we have all data
http://www.bbc.com/news/business-18367213
![Page 5: Big data, the end of optimisation?](https://reader034.vdocument.in/reader034/viewer/2022051512/53edd9dc8d7f7289708b5ef5/html5/thumbnails/5.jpg)
“All models are wrong, and increasingly you can succeed without them”
Peter Norvig, Google
http://archive.wired.com/science/discoveries/magazine/16-07/pb_theory
![Page 6: Big data, the end of optimisation?](https://reader034.vdocument.in/reader034/viewer/2022051512/53edd9dc8d7f7289708b5ef5/html5/thumbnails/6.jpg)
Accurate predictions
Correlations suffice
No samples
No models
The Big Data Promise
![Page 7: Big data, the end of optimisation?](https://reader034.vdocument.in/reader034/viewer/2022051512/53edd9dc8d7f7289708b5ef5/html5/thumbnails/7.jpg)
90% of the data stored today did not exist two
years ago
![Page 8: Big data, the end of optimisation?](https://reader034.vdocument.in/reader034/viewer/2022051512/53edd9dc8d7f7289708b5ef5/html5/thumbnails/8.jpg)
Generally speaking there
are two types of data
![Page 9: Big data, the end of optimisation?](https://reader034.vdocument.in/reader034/viewer/2022051512/53edd9dc8d7f7289708b5ef5/html5/thumbnails/9.jpg)
Growing up to unlock the Big Data potential
![Page 10: Big data, the end of optimisation?](https://reader034.vdocument.in/reader034/viewer/2022051512/53edd9dc8d7f7289708b5ef5/html5/thumbnails/10.jpg)
Framing Data gathering &
Analyses Modelling & Calibration
Scenarios & Sensitivity
analysis
Communication &
Implementation
Analytics is not just a tool, it requires skill ,
competency & experience
![Page 11: Big data, the end of optimisation?](https://reader034.vdocument.in/reader034/viewer/2022051512/53edd9dc8d7f7289708b5ef5/html5/thumbnails/11.jpg)
What I learned in class, there is always an error
http://suicideandsappiness.files.wordpress.com/2010/10/cartoon-6.png
![Page 12: Big data, the end of optimisation?](https://reader034.vdocument.in/reader034/viewer/2022051512/53edd9dc8d7f7289708b5ef5/html5/thumbnails/12.jpg)
How many Type I Errors does Target make?
![Page 13: Big data, the end of optimisation?](https://reader034.vdocument.in/reader034/viewer/2022051512/53edd9dc8d7f7289708b5ef5/html5/thumbnails/13.jpg)
Are Correlations all that matter?
![Page 14: Big data, the end of optimisation?](https://reader034.vdocument.in/reader034/viewer/2022051512/53edd9dc8d7f7289708b5ef5/html5/thumbnails/14.jpg)
Many more examples
Caution Correlation ≠ Causality
![Page 15: Big data, the end of optimisation?](https://reader034.vdocument.in/reader034/viewer/2022051512/53edd9dc8d7f7289708b5ef5/html5/thumbnails/15.jpg)
Indeed, correlations
are not enough
http://www.nature.com/news/when-google-got-flu-wrong-1.12413 http://www.sciencemag.org/content/343/6176/1203/F2.large.jpg
![Page 16: Big data, the end of optimisation?](https://reader034.vdocument.in/reader034/viewer/2022051512/53edd9dc8d7f7289708b5ef5/html5/thumbnails/16.jpg)
Do all observations tell the whole story?
![Page 17: Big data, the end of optimisation?](https://reader034.vdocument.in/reader034/viewer/2022051512/53edd9dc8d7f7289708b5ef5/html5/thumbnails/17.jpg)
Big Data implies N=All, but often is
a seductive illusion
http://www.bostonglobe.com/metro/2014/03/22/starts-stops/z04CiSdSdwbACUYqnvTPSL/story.html/
![Page 18: Big data, the end of optimisation?](https://reader034.vdocument.in/reader034/viewer/2022051512/53edd9dc8d7f7289708b5ef5/html5/thumbnails/18.jpg)
“ĉiuj modeloj estas erara sed iuj estas utilaj”
George Box
![Page 19: Big data, the end of optimisation?](https://reader034.vdocument.in/reader034/viewer/2022051512/53edd9dc8d7f7289708b5ef5/html5/thumbnails/19.jpg)
From
Data-rich
to
Decision-smart
The future is bright, it’s full of data
![Page 20: Big data, the end of optimisation?](https://reader034.vdocument.in/reader034/viewer/2022051512/53edd9dc8d7f7289708b5ef5/html5/thumbnails/20.jpg)
[email protected] @ORatWork http://nl.linkedin.com/in/johnpoppelaars www.john-poppelaars.blogspot.nl