qmeeting 2015 big data
TRANSCRIPT
![Page 1: Qmeeting 2015 Big Data](https://reader034.vdocument.in/reader034/viewer/2022042906/58ae037e1a28aba3178b4623/html5/thumbnails/1.jpg)
QMeeting 2015
BIG DATA: Data ScienceDiogenes Justo
26/set/2015
![Page 2: Qmeeting 2015 Big Data](https://reader034.vdocument.in/reader034/viewer/2022042906/58ae037e1a28aba3178b4623/html5/thumbnails/2.jpg)
SHORT BIODiógenes Justo
Mestre em Economia Aplicada (UFRGS)MBE Economia (UFRGS).
Especialista em Banking (FGV).Especialista em Data Science (John Hopkins University / Coursera).
Bacharel em Matemática Aplicada e Computacional (UFRGS).
Cursos de especialização em Big Data, Machine Learning e Data Mining no MIT, Washington University, University of Illinois e Stanford
PMO Manager da BMF&Bovespa - Profissional certificado PMP.20 anos de experiência na área de TI, tendo atuado em desenvolvimento, infraestrutura, banco de dados
e B.I., além de projetos.
![Page 3: Qmeeting 2015 Big Data](https://reader034.vdocument.in/reader034/viewer/2022042906/58ae037e1a28aba3178b4623/html5/thumbnails/3.jpg)
In the German military machines, the total number of possible combinations for message encryptions comes to a staggering figure in the quadrillions. (The exact number? 158,962,555,217,826,360,000).
![Page 4: Qmeeting 2015 Big Data](https://reader034.vdocument.in/reader034/viewer/2022042906/58ae037e1a28aba3178b4623/html5/thumbnails/4.jpg)
RECENTE HISTÓRIA
![Page 5: Qmeeting 2015 Big Data](https://reader034.vdocument.in/reader034/viewer/2022042906/58ae037e1a28aba3178b4623/html5/thumbnails/5.jpg)
BIG DATA
![Page 6: Qmeeting 2015 Big Data](https://reader034.vdocument.in/reader034/viewer/2022042906/58ae037e1a28aba3178b4623/html5/thumbnails/6.jpg)
91 95 98 99 01 02 03 04 05 12
NSF - Core Technologies for
Advancing Big Data Science
![Page 7: Qmeeting 2015 Big Data](https://reader034.vdocument.in/reader034/viewer/2022042906/58ae037e1a28aba3178b4623/html5/thumbnails/7.jpg)
TENDÊNCIA
![Page 8: Qmeeting 2015 Big Data](https://reader034.vdocument.in/reader034/viewer/2022042906/58ae037e1a28aba3178b4623/html5/thumbnails/8.jpg)
![Page 9: Qmeeting 2015 Big Data](https://reader034.vdocument.in/reader034/viewer/2022042906/58ae037e1a28aba3178b4623/html5/thumbnails/9.jpg)
CARREIRAS MAIS DEMANDADAS
![Page 10: Qmeeting 2015 Big Data](https://reader034.vdocument.in/reader034/viewer/2022042906/58ae037e1a28aba3178b4623/html5/thumbnails/10.jpg)
![Page 11: Qmeeting 2015 Big Data](https://reader034.vdocument.in/reader034/viewer/2022042906/58ae037e1a28aba3178b4623/html5/thumbnails/11.jpg)
CLUSTER COMPUTING
![Page 12: Qmeeting 2015 Big Data](https://reader034.vdocument.in/reader034/viewer/2022042906/58ae037e1a28aba3178b4623/html5/thumbnails/12.jpg)
CLUSTER COMPUTING
Min(CPU-CPU)?
Multi-Core
![Page 13: Qmeeting 2015 Big Data](https://reader034.vdocument.in/reader034/viewer/2022042906/58ae037e1a28aba3178b4623/html5/thumbnails/13.jpg)
RASPBERRY PI CLUSTER
WSO2 Conference 2013, London UK
![Page 14: Qmeeting 2015 Big Data](https://reader034.vdocument.in/reader034/viewer/2022042906/58ae037e1a28aba3178b4623/html5/thumbnails/14.jpg)
HADOOP + MAP REDUCE
Logistic regression in Hadoop and Sparkspark.apache.org
![Page 15: Qmeeting 2015 Big Data](https://reader034.vdocument.in/reader034/viewer/2022042906/58ae037e1a28aba3178b4623/html5/thumbnails/15.jpg)
DATA MINING, BIG DATA, DATA SCIENCE...
Data Mining ≈ Big Data ≈Predictive Analytics ≈
Data Science
J. Leskovec, A.Rajaraman, J.Ullman: Mining of Massive Datasetshttp://www.mmds.org
![Page 16: Qmeeting 2015 Big Data](https://reader034.vdocument.in/reader034/viewer/2022042906/58ae037e1a28aba3178b4623/html5/thumbnails/16.jpg)
WHAT IS BIG DATA?
thefinancialbrand.com (IBM Research)
![Page 17: Qmeeting 2015 Big Data](https://reader034.vdocument.in/reader034/viewer/2022042906/58ae037e1a28aba3178b4623/html5/thumbnails/17.jpg)
[DEAN, 14]
DATA MINING, MACHINE LEARNING...
![Page 18: Qmeeting 2015 Big Data](https://reader034.vdocument.in/reader034/viewer/2022042906/58ae037e1a28aba3178b4623/html5/thumbnails/18.jpg)
O QUE NÃO É O QUE É
● Algo novo ou moda
● Algo totalmente avesso a Business
Intelligence ou DatawareHouse
● Quem tem Hadoop/NoSQL tem Big
Data
● Big Data significa muitos dados
espalhados por aí (web)
● Um problema que, mais hora, menos hora,
apareceria; uns tiveram o problema primeiro
● Abrange e se utiliza de técnicas já desenvolvidas
de BI, DW, Data Mining, Machine Learning (e usa
novas)...
● Hadoop/NoSQL facilitam a solução de problemas
● Big Data significa muitos dados (in ou out),
complexos (estruturados e não), de forma
crescente
![Page 19: Qmeeting 2015 Big Data](https://reader034.vdocument.in/reader034/viewer/2022042906/58ae037e1a28aba3178b4623/html5/thumbnails/19.jpg)
APLICAÇÕES
![Page 20: Qmeeting 2015 Big Data](https://reader034.vdocument.in/reader034/viewer/2022042906/58ae037e1a28aba3178b4623/html5/thumbnails/20.jpg)
![Page 21: Qmeeting 2015 Big Data](https://reader034.vdocument.in/reader034/viewer/2022042906/58ae037e1a28aba3178b4623/html5/thumbnails/21.jpg)
![Page 22: Qmeeting 2015 Big Data](https://reader034.vdocument.in/reader034/viewer/2022042906/58ae037e1a28aba3178b4623/html5/thumbnails/22.jpg)
![Page 23: Qmeeting 2015 Big Data](https://reader034.vdocument.in/reader034/viewer/2022042906/58ae037e1a28aba3178b4623/html5/thumbnails/23.jpg)
![Page 24: Qmeeting 2015 Big Data](https://reader034.vdocument.in/reader034/viewer/2022042906/58ae037e1a28aba3178b4623/html5/thumbnails/24.jpg)
![Page 25: Qmeeting 2015 Big Data](https://reader034.vdocument.in/reader034/viewer/2022042906/58ae037e1a28aba3178b4623/html5/thumbnails/25.jpg)
![Page 26: Qmeeting 2015 Big Data](https://reader034.vdocument.in/reader034/viewer/2022042906/58ae037e1a28aba3178b4623/html5/thumbnails/26.jpg)
QUEM FAZ ACONTECER?
![Page 27: Qmeeting 2015 Big Data](https://reader034.vdocument.in/reader034/viewer/2022042906/58ae037e1a28aba3178b4623/html5/thumbnails/27.jpg)
![Page 28: Qmeeting 2015 Big Data](https://reader034.vdocument.in/reader034/viewer/2022042906/58ae037e1a28aba3178b4623/html5/thumbnails/28.jpg)
Pesquisa Científica
Machine Learning
Zona de Perigo
![Page 29: Qmeeting 2015 Big Data](https://reader034.vdocument.in/reader034/viewer/2022042906/58ae037e1a28aba3178b4623/html5/thumbnails/29.jpg)
DATA SCIENCE - PROCESSO
![Page 30: Qmeeting 2015 Big Data](https://reader034.vdocument.in/reader034/viewer/2022042906/58ae037e1a28aba3178b4623/html5/thumbnails/30.jpg)
![Page 31: Qmeeting 2015 Big Data](https://reader034.vdocument.in/reader034/viewer/2022042906/58ae037e1a28aba3178b4623/html5/thumbnails/31.jpg)
DESAFIOS...
- Demanda deve crescer muito nos próximos anos- Faltam profissionais para iniciarem as demandas ou identificar
aplicações- Preparar mão de obra
- Nas empresas: qual é mais difícil? Mat/Est, Programação ou Negócios- Nas universidades e escolas: adaptação de ementas, professores
qualificados- Trazer valor de soluções
- Foco na solução, benefícios e não nas tecnologias