statistics: the grammar of data science
TRANSCRIPT
![Page 1: Statistics: the grammar of Data Science](https://reader035.vdocument.in/reader035/viewer/2022062412/58ec73131a28abe36b8b466b/html5/thumbnails/1.jpg)
STATISTICS THE GRAMMAR OF DATA SCIENCE
ÍCARO MEDEIROS
Big Data WeekSão Paulo - SP, 23/11/2015
![Page 2: Statistics: the grammar of Data Science](https://reader035.vdocument.in/reader035/viewer/2022062412/58ec73131a28abe36b8b466b/html5/thumbnails/2.jpg)
WHY TO PURSUE SOLID BACKGROUND ON STATISTICS?
https://twitter.com/josh_wills/status/198093512149958656
![Page 3: Statistics: the grammar of Data Science](https://reader035.vdocument.in/reader035/viewer/2022062412/58ec73131a28abe36b8b466b/html5/thumbnails/3.jpg)
STATISTICS TO HELP NOT GETTING INTO THE DANGER ZONE
http://berkeleysciencereview.com/scientific-collaborations-uc-berkeley-data-driven-cover/
![Page 4: Statistics: the grammar of Data Science](https://reader035.vdocument.in/reader035/viewer/2022062412/58ec73131a28abe36b8b466b/html5/thumbnails/4.jpg)
INSPIRATIONS FOR THIS KEYNOTE
https://speakerdeck.com/jakevdp/statistics-for-hackershttps://www.goodreads.com/book/show/17986418-naked-statistics
![Page 5: Statistics: the grammar of Data Science](https://reader035.vdocument.in/reader035/viewer/2022062412/58ec73131a28abe36b8b466b/html5/thumbnails/5.jpg)
METRICS
MEANINGMETHODS
MISUSES-+++
![Page 6: Statistics: the grammar of Data Science](https://reader035.vdocument.in/reader035/viewer/2022062412/58ec73131a28abe36b8b466b/html5/thumbnails/6.jpg)
HOW TO LIE: MOVIE REVIEWS
http://fivethirtyeight.com/features/fandango-movies-ratings
BECAUSE IT LOOKS LIKE MATH, WE [THINK] IT’S SOMEHOW OBJECTIVELY TRUE, BUT IT’S ALL BASED ON SUBJECTIVE EXPERIENCE
![Page 7: Statistics: the grammar of Data Science](https://reader035.vdocument.in/reader035/viewer/2022062412/58ec73131a28abe36b8b466b/html5/thumbnails/7.jpg)
FANDANGO LOVES MOVIES
http://fivethirtyeight.com/features/fandango-movies-ratings
(AND SELLS MOVIE TICKETS)
![Page 8: Statistics: the grammar of Data Science](https://reader035.vdocument.in/reader035/viewer/2022062412/58ec73131a28abe36b8b466b/html5/thumbnails/8.jpg)
ATTENTION TO PROVENANCE
![Page 9: Statistics: the grammar of Data Science](https://reader035.vdocument.in/reader035/viewer/2022062412/58ec73131a28abe36b8b466b/html5/thumbnails/9.jpg)
HOW TO LIE: ROUNDING
http://fivethirtyeight.com/features/fandango-movies-ratings
![Page 10: Statistics: the grammar of Data Science](https://reader035.vdocument.in/reader035/viewer/2022062412/58ec73131a28abe36b8b466b/html5/thumbnails/10.jpg)
HOW TO LIE: ROUNDING
http://fivethirtyeight.com/features/fandango-movies-ratings
![Page 11: Statistics: the grammar of Data Science](https://reader035.vdocument.in/reader035/viewer/2022062412/58ec73131a28abe36b8b466b/html5/thumbnails/11.jpg)
CORRELATION IS NOT CAUSATION
![Page 12: Statistics: the grammar of Data Science](https://reader035.vdocument.in/reader035/viewer/2022062412/58ec73131a28abe36b8b466b/html5/thumbnails/12.jpg)
SHORT BREAKS AT WORK "CAUSE" CANCER
Example from ‘Naked Statistics'
![Page 13: Statistics: the grammar of Data Science](https://reader035.vdocument.in/reader035/viewer/2022062412/58ec73131a28abe36b8b466b/html5/thumbnails/13.jpg)
SMOKINGCAUSES CANCER
BREAKS AT WORK
CORRELATED WITH
LEAD TO
![Page 14: Statistics: the grammar of Data Science](https://reader035.vdocument.in/reader035/viewer/2022062412/58ec73131a28abe36b8b466b/html5/thumbnails/14.jpg)
SPURIOUS CORRELATIONS
http://www.tylervigen.com/spurious-correlations
![Page 15: Statistics: the grammar of Data Science](https://reader035.vdocument.in/reader035/viewer/2022062412/58ec73131a28abe36b8b466b/html5/thumbnails/15.jpg)
A/B TESTING
https://vwo.com/ab-testing/
![Page 16: Statistics: the grammar of Data Science](https://reader035.vdocument.in/reader035/viewer/2022062412/58ec73131a28abe36b8b466b/html5/thumbnails/16.jpg)
A/B TESTING CAN BE BAD
https://www.quora.com/When-should-A-B-testing-not-be-trusted-to-make-decisions/answer/Edwin-Chen-1
▸Feedback loops
▸Novelty effect
▸Seasonality
▸Wrong metrics
http://www.evanmiller.org/how-not-to-run-an-ab-test.html
![Page 17: Statistics: the grammar of Data Science](https://reader035.vdocument.in/reader035/viewer/2022062412/58ec73131a28abe36b8b466b/html5/thumbnails/17.jpg)
CHOOSE THE RIGHT METRICS: CLICKS VS DWELL TIME
http://yahoolabs.tumblr.com/post/99405569711/science-powering-product-and-personalization
![Page 18: Statistics: the grammar of Data Science](https://reader035.vdocument.in/reader035/viewer/2022062412/58ec73131a28abe36b8b466b/html5/thumbnails/18.jpg)
CHOOSE THE RIGHT METRICS: SHARING IS NOT NECESSARILY CARING
http://time.com/12933/what-you-think-you-know-about-the-web-is-wrong/
![Page 19: Statistics: the grammar of Data Science](https://reader035.vdocument.in/reader035/viewer/2022062412/58ec73131a28abe36b8b466b/html5/thumbnails/19.jpg)
https://xkcd.com/882 (Significant)
SIGNIFICANCE
![Page 20: Statistics: the grammar of Data Science](https://reader035.vdocument.in/reader035/viewer/2022062412/58ec73131a28abe36b8b466b/html5/thumbnails/20.jpg)
![Page 21: Statistics: the grammar of Data Science](https://reader035.vdocument.in/reader035/viewer/2022062412/58ec73131a28abe36b8b466b/html5/thumbnails/21.jpg)
![Page 22: Statistics: the grammar of Data Science](https://reader035.vdocument.in/reader035/viewer/2022062412/58ec73131a28abe36b8b466b/html5/thumbnails/22.jpg)
THE RED CARD PROBLEM
http://fivethirtyeight.com/features/science-isnt-broken/http://www.nature.com/news/crowdsourced-research-many-hands-make-tight-work-1.18508
![Page 23: Statistics: the grammar of Data Science](https://reader035.vdocument.in/reader035/viewer/2022062412/58ec73131a28abe36b8b466b/html5/thumbnails/23.jpg)
61 RESEARCHERS: SAME PROBLEM, DIFFERENT METHODS
http://fivethirtyeight.com/features/science-isnt-broken/
![Page 24: Statistics: the grammar of Data Science](https://reader035.vdocument.in/reader035/viewer/2022062412/58ec73131a28abe36b8b466b/html5/thumbnails/24.jpg)
THE BACON CONTROVERSY
![Page 25: Statistics: the grammar of Data Science](https://reader035.vdocument.in/reader035/viewer/2022062412/58ec73131a28abe36b8b466b/html5/thumbnails/25.jpg)
MORE ABOUT P-VALUES
https://twitter.com/Ted_Underwood/status/658983555008040960
![Page 26: Statistics: the grammar of Data Science](https://reader035.vdocument.in/reader035/viewer/2022062412/58ec73131a28abe36b8b466b/html5/thumbnails/26.jpg)
IS THIS A GOOD CLASSIFICATION?
http://www.wired.com/2015/10/who-does-bacon-cause-cancer-sort-of-but-not-really/
1 CARCINOGENIC
2A PROBABLY
2B POSSIBLY
…
![Page 27: Statistics: the grammar of Data Science](https://reader035.vdocument.in/reader035/viewer/2022062412/58ec73131a28abe36b8b466b/html5/thumbnails/27.jpg)
EFFECT SIZE: BACON VS CIGARETTES (SAME CATEGORY)
This is bacon
18%
Cigarette
![Page 28: Statistics: the grammar of Data Science](https://reader035.vdocument.in/reader035/viewer/2022062412/58ec73131a28abe36b8b466b/html5/thumbnails/28.jpg)
WAIT FOR IT…
![Page 29: Statistics: the grammar of Data Science](https://reader035.vdocument.in/reader035/viewer/2022062412/58ec73131a28abe36b8b466b/html5/thumbnails/29.jpg)
Cigarette
2500%
http://www.wired.com/2015/10/who-does-bacon-cause-cancer-sort-of-but-not-really/
![Page 30: Statistics: the grammar of Data Science](https://reader035.vdocument.in/reader035/viewer/2022062412/58ec73131a28abe36b8b466b/html5/thumbnails/30.jpg)
http://www.theguardian.com/society/2015/oct/26/bacon-ham-sausages-processed-meats-cancer-risk-smoking-says-who
https://catracalivre.com.br/geral/sustentavel/indicacao/muito-alem-do-bacon-agrotoxicos-tambem-podem-causar-cancer/
![Page 31: Statistics: the grammar of Data Science](https://reader035.vdocument.in/reader035/viewer/2022062412/58ec73131a28abe36b8b466b/html5/thumbnails/31.jpg)
THE SCHRODINGER’S DIET
http://www.vox.com/2015/5/20/8621527/health-tips-reporter
![Page 32: Statistics: the grammar of Data Science](https://reader035.vdocument.in/reader035/viewer/2022062412/58ec73131a28abe36b8b466b/html5/thumbnails/32.jpg)
”
http://nerds.airbnb.com/scaling-data-science
DATA SCIENCE IS AN ACT OF INTERPRETATION OF CUSTOMER'S VOICE
![Page 33: Statistics: the grammar of Data Science](https://reader035.vdocument.in/reader035/viewer/2022062412/58ec73131a28abe36b8b466b/html5/thumbnails/33.jpg)
GOOD DATA VISUALIZATION: TIPS FOR SCATTER PLOTS
http://content.visage.co/hs-fs/hub/424038/file-2094950163-pdf/Data_Visualization_101_How_to_Design_Charts_and_Graphs.pdf
![Page 34: Statistics: the grammar of Data Science](https://reader035.vdocument.in/reader035/viewer/2022062412/58ec73131a28abe36b8b466b/html5/thumbnails/34.jpg)
DAVID MCCANDLESS: INFORMATION IS BEAUTIFUL
http://www.informationisbeautiful.net/visualizations/diversity-in-tech/
![Page 35: Statistics: the grammar of Data Science](https://reader035.vdocument.in/reader035/viewer/2022062412/58ec73131a28abe36b8b466b/html5/thumbnails/35.jpg)
IT’S EASY TO LIE WITH STATISTICS, BUT IT’S HARD TO TELL THE TRUTH WITHOUT THEM
Andrejs Dunkels, as mentioned on "Naked Statistics"
TAKEAWAY MESSAGE
![Page 36: Statistics: the grammar of Data Science](https://reader035.vdocument.in/reader035/viewer/2022062412/58ec73131a28abe36b8b466b/html5/thumbnails/36.jpg)
WHY PYTHON IS BETTER FOR DATA SCIENCE
MY NEXT TALK
São Paulo Big Data MeetupSão Paulo - SP, 25/11/2015VivaReal Portal Imobiliário. Rua Bela Cintra, 539 - Consolação
http://www.meetup.com/pt/Sao-Paulo-Big-Data-Meetup
![Page 37: Statistics: the grammar of Data Science](https://reader035.vdocument.in/reader035/viewer/2022062412/58ec73131a28abe36b8b466b/html5/thumbnails/37.jpg)
slides icaromedeiros.com.br
slideshare.net/icaromedeiros
@icaromedeiros