pro tooling and integration [presented] · example: bitcoin artifacts database webserver email rest...

37
ANALYTIC ADMINISTRATION FOR R PROFESSIONAL R TOOLING AND INTEGRATION

Upload: dinhanh

Post on 29-Aug-2019

218 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Pro Tooling and Integration [Presented] · EXAMPLE: BITCOIN ARTIFACTS Database Webserver Email REST API Integration RStudio Server Pro Build Artifacts RStudio Connect Run Artifacts

ANALYTIC ADMINISTRATION FOR R

PROFESSIONAL R TOOLING AND INTEGRATION

Page 2: Pro Tooling and Integration [Presented] · EXAMPLE: BITCOIN ARTIFACTS Database Webserver Email REST API Integration RStudio Server Pro Build Artifacts RStudio Connect Run Artifacts

Program

DATA SCIENCE WORKFLOW

Reports Dashboards

Presentations Apps API’s

Data Products

Artifacts

Page 3: Pro Tooling and Integration [Presented] · EXAMPLE: BITCOIN ARTIFACTS Database Webserver Email REST API Integration RStudio Server Pro Build Artifacts RStudio Connect Run Artifacts

DATA SCIENCE IN THE ENTERPRISEEnterprise Environment

Programming Languages Packages and dependencies

Operating Systems

Data Sources

Deploy

Security

Reports Dashboards

Presentations Apps API’s

Data Products

Program Artifacts

Integration

ValidationControl

Manage

Scale

BI

In-Database

Reporting

Web

Workflows

API’sAccess

Page 4: Pro Tooling and Integration [Presented] · EXAMPLE: BITCOIN ARTIFACTS Database Webserver Email REST API Integration RStudio Server Pro Build Artifacts RStudio Connect Run Artifacts

ENTERPRISE TOOLING

Server based

Cloud or on-premises (or both)

IT managed

Problem: IT doesn’t know R

Page 5: Pro Tooling and Integration [Presented] · EXAMPLE: BITCOIN ARTIFACTS Database Webserver Email REST API Integration RStudio Server Pro Build Artifacts RStudio Connect Run Artifacts

R IN THE ENTERPRISE

Legitimacy Recognize R as an analytic standard

Competencies Understand and manage R tooling

Adoption Rely on integrated R based solutions

Page 6: Pro Tooling and Integration [Presented] · EXAMPLE: BITCOIN ARTIFACTS Database Webserver Email REST API Integration RStudio Server Pro Build Artifacts RStudio Connect Run Artifacts

Legitimacy Recognize R an Analytic Standard

Page 7: Pro Tooling and Integration [Presented] · EXAMPLE: BITCOIN ARTIFACTS Database Webserver Email REST API Integration RStudio Server Pro Build Artifacts RStudio Connect Run Artifacts

BRING R THROUGH THE FRONT DOOR

Show the value of R R Markdown reports, dashboards, and presentations

Shiny apps Plumber API’s

Other data products

Get official recognition Get resources

Page 8: Pro Tooling and Integration [Presented] · EXAMPLE: BITCOIN ARTIFACTS Database Webserver Email REST API Integration RStudio Server Pro Build Artifacts RStudio Connect Run Artifacts

Consumer Research (c. 2004)

Advertising Network (c. 2008)

Client Services (c. 2011)

R Users Me Team Global

R Tools Desktop Server Cluster

Relationship with IT None Support Partnership

Approved By Manager Owner Corporate Process

Groups Influenced Department Organization Customers

SOME PERSONAL HISTORY

Page 9: Pro Tooling and Integration [Presented] · EXAMPLE: BITCOIN ARTIFACTS Database Webserver Email REST API Integration RStudio Server Pro Build Artifacts RStudio Connect Run Artifacts

GETTING STARTED

Build a sandbox / lab box Proof of concept

Show the value of R R Markdown reports, dashboards, and

presentations Shiny apps

Plumber API’s Other data products

Page 10: Pro Tooling and Integration [Presented] · EXAMPLE: BITCOIN ARTIFACTS Database Webserver Email REST API Integration RStudio Server Pro Build Artifacts RStudio Connect Run Artifacts

Competencies Understand and Manage R tooling

Page 11: Pro Tooling and Integration [Presented] · EXAMPLE: BITCOIN ARTIFACTS Database Webserver Email REST API Integration RStudio Server Pro Build Artifacts RStudio Connect Run Artifacts

RSTUDIO

We build tools for doing data science

Committed to open source

We hire great people

We sell professional products

Page 12: Pro Tooling and Integration [Presented] · EXAMPLE: BITCOIN ARTIFACTS Database Webserver Email REST API Integration RStudio Server Pro Build Artifacts RStudio Connect Run Artifacts

RSTUDIO PRO PRODUCTS HELP YOU…

Access Spark from R

Connect you to your databases

Deploy your Shiny applications

Automate and distribute your reports

Secure, scale, and support your environment

Page 13: Pro Tooling and Integration [Presented] · EXAMPLE: BITCOIN ARTIFACTS Database Webserver Email REST API Integration RStudio Server Pro Build Artifacts RStudio Connect Run Artifacts

RSTUDIO SERVER PRODevelop at scale

Runs securely in your environment

Horizontally scalable

Connects to your databases

Easy to administration

Supported

Page 14: Pro Tooling and Integration [Presented] · EXAMPLE: BITCOIN ARTIFACTS Database Webserver Email REST API Integration RStudio Server Pro Build Artifacts RStudio Connect Run Artifacts

RSTUDIO CONNECTRun your artifacts at scale

On premises solution

Push button publishing

Self managed content

Scheduled updates & distribution

Horizontally scalable

Secure and supported

Page 15: Pro Tooling and Integration [Presented] · EXAMPLE: BITCOIN ARTIFACTS Database Webserver Email REST API Integration RStudio Server Pro Build Artifacts RStudio Connect Run Artifacts

DATA SCIENCE LAB IN THE ENTERPRISEEnterprise Environment

Programming Languages Packages and dependencies

Operating Systems

Data Sources

Data Science Lab BI

In-Database

Reporting

Web

Workflows

API’s

RStudio Server ProBuild Artifacts

RStudio ConnectRun Artifacts

Page 16: Pro Tooling and Integration [Presented] · EXAMPLE: BITCOIN ARTIFACTS Database Webserver Email REST API Integration RStudio Server Pro Build Artifacts RStudio Connect Run Artifacts

Deploying Artifacts

Page 17: Pro Tooling and Integration [Presented] · EXAMPLE: BITCOIN ARTIFACTS Database Webserver Email REST API Integration RStudio Server Pro Build Artifacts RStudio Connect Run Artifacts

DEPLOYMENT APPROACH1. Build the bundle

Source Code Data Files Manifest

(Metadata and environment)

2. Push the bundle

3. Deploy the bundle Parse the manifest

Packrat restore

RStudio Server ProBuild Artifacts

RStudio ConnectRun Artifacts

SourceCode

Data Files

Manifest

Bundle

Page 18: Pro Tooling and Integration [Presented] · EXAMPLE: BITCOIN ARTIFACTS Database Webserver Email REST API Integration RStudio Server Pro Build Artifacts RStudio Connect Run Artifacts

DEPLOYMENT GOALS

1. Reproduce the development environment 2. Productize the run environment

R Packages

R Version

Repositories

EnvironmentVariables

Secrets

File paths

Permissions

Operating Systems

OS Dependencies

Build Environment

Metadata

Sandbox

SecurityScalable Controls

RStudio Server Pro RStudio Connect

R Packages

R Version

Repositories

EnvironmentVariables

Secrets

File paths

Permissions

Operating Systems

OS Dependencies

Run Environment

Metadata

Page 19: Pro Tooling and Integration [Presented] · EXAMPLE: BITCOIN ARTIFACTS Database Webserver Email REST API Integration RStudio Server Pro Build Artifacts RStudio Connect Run Artifacts

EXAMPLE: STOCK PORTFOLIO

Page 20: Pro Tooling and Integration [Presented] · EXAMPLE: BITCOIN ARTIFACTS Database Webserver Email REST API Integration RStudio Server Pro Build Artifacts RStudio Connect Run Artifacts

Adoption Rely on integrated R based solutions

Page 21: Pro Tooling and Integration [Presented] · EXAMPLE: BITCOIN ARTIFACTS Database Webserver Email REST API Integration RStudio Server Pro Build Artifacts RStudio Connect Run Artifacts

CONNECT TO YOUR DATARStudio Data Connectors

Work with all pro products

Fully supported

Easy to use

Easy to set up

Adheres to enterprise standards

Page 22: Pro Tooling and Integration [Presented] · EXAMPLE: BITCOIN ARTIFACTS Database Webserver Email REST API Integration RStudio Server Pro Build Artifacts RStudio Connect Run Artifacts

COMMUNICATE INSIGHTS

Shiny Apps Interactive

Delivered via the web

R Markdown Reports Snapshot in time

Deliver via email or web

Shiny Apps

R MarkdownReports

UserExperience

Interactive(Dynamic)

Snapshot(Batch)

AudienceDelivery

Web (Pull audience)

Email or Web (Push to audience)

Page 23: Pro Tooling and Integration [Presented] · EXAMPLE: BITCOIN ARTIFACTS Database Webserver Email REST API Integration RStudio Server Pro Build Artifacts RStudio Connect Run Artifacts

AUTOMATE DATA PRODUCTS

Schedule execution

Monitor logs

Track render history

Access source versions

Page 24: Pro Tooling and Integration [Presented] · EXAMPLE: BITCOIN ARTIFACTS Database Webserver Email REST API Integration RStudio Server Pro Build Artifacts RStudio Connect Run Artifacts

CALL R AS A WEB SERVICE

What is an REST based API? So!ware that lets computer systems communicate

Humans use GUI’s Machines use API’s

Why are they important? Solve the handoff problem Integrate R into workflows

Page 25: Pro Tooling and Integration [Presented] · EXAMPLE: BITCOIN ARTIFACTS Database Webserver Email REST API Integration RStudio Server Pro Build Artifacts RStudio Connect Run Artifacts

Data Science Lab

EXAMPLE: BITCOIN ARTIFACTS

Database

Webserver

Email

REST API

IntegrationRStudio Server ProBuild Artifacts

RStudio ConnectRun Artifacts

Page 26: Pro Tooling and Integration [Presented] · EXAMPLE: BITCOIN ARTIFACTS Database Webserver Email REST API Integration RStudio Server Pro Build Artifacts RStudio Connect Run Artifacts

SPARKLYR

Use the full power of R with Apache Spark

Integrated with the RStudio IDE dplyr backend

Works with Hadoop

Page 27: Pro Tooling and Integration [Presented] · EXAMPLE: BITCOIN ARTIFACTS Database Webserver Email REST API Integration RStudio Server Pro Build Artifacts RStudio Connect Run Artifacts

PYTHON INTEROPERABILITYReticulate

Call Python from R Translate between R and Python objects

Flexible binding to different version of Python

TensorFlow Open source so!ware library for Machine Intelligence

Keras High level interface for neural networks

Page 28: Pro Tooling and Integration [Presented] · EXAMPLE: BITCOIN ARTIFACTS Database Webserver Email REST API Integration RStudio Server Pro Build Artifacts RStudio Connect Run Artifacts

EXAMPLE: RETICULATED SHINY APP

Page 29: Pro Tooling and Integration [Presented] · EXAMPLE: BITCOIN ARTIFACTS Database Webserver Email REST API Integration RStudio Server Pro Build Artifacts RStudio Connect Run Artifacts

SHARED INFRASTRUCTURE FOR DATA SCIENCE

A new data science runtime Powered by Apache Arrow

Page 30: Pro Tooling and Integration [Presented] · EXAMPLE: BITCOIN ARTIFACTS Database Webserver Email REST API Integration RStudio Server Pro Build Artifacts RStudio Connect Run Artifacts

The R Admin

Page 31: Pro Tooling and Integration [Presented] · EXAMPLE: BITCOIN ARTIFACTS Database Webserver Email REST API Integration RStudio Server Pro Build Artifacts RStudio Connect Run Artifacts

THE R ADMIN

A data scientist (who knows R) and works with IT/Ops to:

1. Establish R as an analytic standard 2. Promote best practices for R tooling

3. Make R indispensable to the organization

Page 32: Pro Tooling and Integration [Presented] · EXAMPLE: BITCOIN ARTIFACTS Database Webserver Email REST API Integration RStudio Server Pro Build Artifacts RStudio Connect Run Artifacts

BEST PRACTICES AND RECOMMENDATIONS

Use a Modern OS and Browser

Upgrade R yearly

Build R from source

Size instance with plenty of RAM

Page 33: Pro Tooling and Integration [Presented] · EXAMPLE: BITCOIN ARTIFACTS Database Webserver Email REST API Integration RStudio Server Pro Build Artifacts RStudio Connect Run Artifacts

THE R ADMIN HAS TO BE RAD!

They have a multiplier effect in their organization

Page 34: Pro Tooling and Integration [Presented] · EXAMPLE: BITCOIN ARTIFACTS Database Webserver Email REST API Integration RStudio Server Pro Build Artifacts RStudio Connect Run Artifacts

DOCUMENTATION

Page 35: Pro Tooling and Integration [Presented] · EXAMPLE: BITCOIN ARTIFACTS Database Webserver Email REST API Integration RStudio Server Pro Build Artifacts RStudio Connect Run Artifacts

OTHER RESOURCES

community.rstudio.com

Developer Blog R Views Blog

rstudio::conf Jan 15-18, 2019

Austin, TX

Page 36: Pro Tooling and Integration [Presented] · EXAMPLE: BITCOIN ARTIFACTS Database Webserver Email REST API Integration RStudio Server Pro Build Artifacts RStudio Connect Run Artifacts

R for the Enterprise Roadmap

R Admin Call to action

Legitimacy Recognize R as an analytic standard

Demonstrate the value of R Build a sandbox / lab box

Get resources

Competencies Understand and manage R processes

Build a data science lab in production Promote best practices

Adoption Rely on R based solutions

Make R indispensable Integrate R artifacts into other systems

SUMMARY

Page 37: Pro Tooling and Integration [Presented] · EXAMPLE: BITCOIN ARTIFACTS Database Webserver Email REST API Integration RStudio Server Pro Build Artifacts RStudio Connect Run Artifacts

THANK [email protected]

@nwstephens

https://github.com/sol-eng/data-science-lab