advanced data analytics -- puc cce março 2017

1

Data Analytics 3.0PUC CCE

Março, 2017

Dan Reznik, PhD

Principal Consultant, Microsoft Consulting Services

[email protected]

[email protected](c) 2016 Upper West Soluções

3

Plano• Big Data• Internet of Things (IoT)• Data Science / Machine Learning• Lab• Azure ML• R

• Practical Aspects in Data Projects

(c) 2016 Upper West Soluções

4

Big Data



SPARK!

Do DW para o BD

8

Multi-structured


16

Data Storage, Access, Management

Analytics, Visualization, Machine Learning

Applications, Businesses DA

TA S

TACK


23

Internet of Things (IoT)


27

IoT: Business Drivers• Increased efficiency (38.3%)• Pursuit of a competitive market advantage (28.4%)• Market share growth (15.9%)• Cost savings (14.9%).• Expertise areas:• Hardware engineering (30.4%)• Sensors development (57.7%)• Systems design and integration (32.3%)• Data analytics (28.9%)• Machine learning (27.4%)• Security (20.9%)


33

Data Science


39

CRISP-DM


41

Machine Learning


42

Típos de Analítica


47

Apache Spark’s Mllib(9x faster than Mahout)

1) Descriptive: Summary statistics, correlations, stratified sampling, hypothesis testing, random data generation

2) Classification: support vector machines, logistic regression, linear regression, decision trees, naive Bayes classification

3) Collaborative filtering: alternating least squares (ALS)4) Clustering: k-means, and Latent Dirichlet Allocation (LDA)5) Dimensionality reduction: SVD and PCA6) Feature extraction and transformation functions7) Optimization: stochastic gradient descent, L-BFGS


48

Azure ML


49

Data Project Planning


54

Practical Aspects: Projects


56

Sources of business value

•Reduce operational costs•Increase revenues•Reduce risk


58

Ecossistema de Dados


59

Fluxo de Trabalho


60

Pirâmide de Maslow

- Qualidade- Integração- Governança

• 360 Cust. View• Anti-Fraud• Anti Churn

ESTRATÉGIA, ROI+


Maturidade

61

Ambiente HeterogêneoProcessos InformaisDados Caóticos

Ambiente IntegradoProcessos controladosDados Coerentes

SimplificaçãoEncurtar tempo de EntregaRedução de Custos


62

Barreiras


63

Dan S. [email protected]@upperwestsolucoes.com


64

Diagrama de Dependencias R (2015)


65

Crescimento Pacotes R (2016)


66

Microsoft Data Science Virtual Machine Ecosystem (DSVM)


advanced data analytics -- puc cce março 2017

Data & Analytics