reporting at motorola - predictive analytics & business insights 2014
DESCRIPTION
In this presentation, Patrick Deglon will share his learnings and provide best practices when using open Google tools & API. He will present his daily email report that hundreds of key Motorola stakeholders are receiving to drive the business, as well as a mobile solution based on the latest web technologies, including Google Visualization, Bootstrap CSS and many of the Google APIs (Gmail, BigQuery, Analytics, Drive, App Engine, Users authentication, etc.).TRANSCRIPT
motorola confidential
Reporting at MotorolaPredictive Analytics & Business Insights 2014
http://www.gatewayanalyticsnetwork.com/b122-home
Dr Patrick DeglonDirector of Engineering, Analytics Area Tech LeadMotorola Mobility [email protected] pdeglon
motorola confidential
...
Overview of Motorola Business
About UsMotorola exists to invent, build and deliver the best mobile devices on the planet, improving the lives of millions of people.
motorola confidential
Motorola
1928
“Galvin Manufacturing Corporation” created, producing battery eliminators
1940
First walkie-talkie which was vital to Allied communication
“one small step for man, one giant leap for mankind” through a Motorola transceiver
1969 1973 2011 20141986
Invented six sigma quality improvement process
first private handheld mobile phone call; world's first commercial cellular device
1983
world's first handset to combine a Linux operating system and Java technology
2003
Source: Wikipedia
Motorola splits into Motorola Mobility & Motorola Solutions
motorola confidential
Key Performance Indicators
Motorola Factory# Shipments
Distribution Channels# Sales
First Usage# Activations
Simplified Business Flow
motorola confidential
Data Flow
Motorola Factory# Shipments
Distribution Channels# Sales
First Usage# Activations
Google BigQuery
MotorolaCloud
Insights
...
motorola confidential
• Business overview
• Demos• Daily Activations Report
• Moto Insights
• Drive Insights
Agenda
motorola confidential
Daily Activations Report
How to provide a global source of truth and start
to provide insights on data points?
Existing Situation
- Numerous (conflicting) sources of truth- Too many variations of same data cube- “Table in your face” approach- No global business definition- No curation of manually entered data points - Report accessible on an internal portal (through VPN)- No mobile form factor
motorola confidential
Demo Daily Report: Final Email
motorola confidential
Google: A galaxy of open APIs and Tools… the good, the bad and the ugly
App EngineBig Query
Compute Engine
Users Cloud Messaging
Analytics
Maps
Visualization
Data Store
Cloud Storage
Cloud SQL
Drive
Docs
Translate
Predict
MailData Store
Task QueueMemcache
URL FetchCloud End Point
Channel
JavaPython
Go
Mail and many more ...
BA
CK
UP
motorola confidential
Demo Daily Report
● Get data (pivot) from BigQuery
● Spreadsheet magic
● Insights: WoW trends with statistics test, Key driver for growth, Key
milestone, internal QA tests
● Embedded Chart
● Scheduler
https://docs.google.com/a/motorola.com/spreadsheet/ccc?key=0AgldkCMfisBTdF83VVJqdFVEZFZiZjgyTWJNdzRNblE&usp=drive_web#gid=21
motorola confidential
• Assume sales follow a diffusion S-shape, i.e.
Description of the illustrative simulation
Marketing Word of mouth
ΔN
Nmax
N
• Add random noise to theoretical daily activations (Poisson)
• Simulated daily activations (sales) for United States, Canada, Brazil, India, Russia, China, Germany and United Kingdom with various launch date per region
ΔN = a (Nmax - N) + b N (Nmax - N)
motorola confidential
Step 1: Create a backbone table
SELECTCAL_DT,Country
FROM
ON A.Dummy=B.DummyWHERE
B.CAL_DT>=A.Launch_Date
motorola.com:sandbox:demo.backbone:INNER JOIN (SELECT Country, CASE WHEN Country IN ('United States','Canada') THEN '2013-08-01' WHEN Country IN ('Brazil','Russia','India','China') THEN '2013-10-01' ELSE '2013-12-01' END AS Launch_Date, GDP_USD/1e7 AS Scale, 1 AS DummyFROM [motorola.com:sandbox:pdeglon.countries]WHERE Country IN ('United States','Canada','Brazil','Russia','India', 'China','Germany','United Kingdom')) AS B
(SELECT CAL_DT, 1 AS Dummy FROM [motorola.com:sandbox:pdeglon.calendar]) AS A
X
BA
CK
UP
motorola confidential
Step 2: Calculate KPI value over time
SELECTCAL_DT,Country,
‘Phone 123’ AS Model,INTEGER(Scale*
EXP(-POW( -150,2)/2/POW(75,2))/(75*SQRT(2*PI()))
) AS Daily_ActivationsFROM
[motorola.com:sandbox:demo.backbone]
motorola.com:sandbox:demo.baseline:
DATEDIFF(TIMESTAMP(CAL_DT),TIMESTAMP(Launch_Date))
...
Normal Distribution:
BA
CK
UP
motorola confidential
Step 3: Add Random Noise
SELECTCAL_DT,Model,
Country,INTEGER(
Daily_Activations + SQRT(Daily_Activations) *SQRT(-2*LN(RAND()))*COS(2*PI()*RAND())
) AS Daily_ActivationsFROM
[motorola.com:sandbox:demo.baseline]
motorola.com:sandbox:demo.simulation:
Normal (Gaussian) Random Number (mu=0, sigma=1)
(pseudo) Poisson distribution for N=Daily_activation
BA
CK
UP
motorola confidential
Step 4: Final Pivot for report
SELECTCAL_DT,SUM(Daily_Activations) AS Total,SUM(CASE WHEN Country IN ('United States','Canada') THEN Daily_Activations ELSE 0 END) AS NA,SUM(CASE WHEN Country IN ('Brazil','Russia','India','China') THEN Daily_Activations ELSE 0 END) AS BRIC,SUM(CASE WHEN Country IN ('Germany','United Kingdom') THEN Daily_Activations ELSE 0 END) AS EU,SUM(CASE WHEN Country='United States' THEN Daily_Activations ELSE 0 END) AS UnitedStates,SUM(CASE WHEN Country='Canada' THEN Daily_Activations ELSE 0 END) AS Canada,SUM(CASE WHEN Country='Brazil' THEN Daily_Activations ELSE 0 END) AS Brazil,SUM(CASE WHEN Country='Russia' THEN Daily_Activations ELSE 0 END) AS Russia,SUM(CASE WHEN Country='India' THEN Daily_Activations ELSE 0 END) AS India,SUM(CASE WHEN Country='China' THEN Daily_Activations ELSE 0 END) AS China,SUM(CASE WHEN Country='Germany' THEN Daily_Activations ELSE 0 END) AS Germany,SUM(CASE WHEN Country='United Kingdom' THEN Daily_Activations ELSE 0 END) AS UnitedKingdom
FROM[motorola.com:sandbox:demo.simulation]
WHERECAL_DT<CURRENT_DATE()
GROUP BY 1ORDER BY 1 DESC
BA
CK
UP
motorola confidential
Demo Daily Report: New Menu Item
motorola confidential
Demo Daily Report: Edit Code (App Script)
motorola confidential
Demo Daily Report: Example for adding a new menu item
motorola confidential
Demo Daily Report: Key Query (pivot)
motorola confidential
Demo Daily Report: Running the query
motorola confidential
Demo Daily Report: Parsing the results
motorola confidential
Demo Daily Report: Data Sheet
motorola confidential
Demo Daily Report: Summary sheet
motorola confidential
Demo Daily Report: Preparing email
motorola confidential
Demo Daily Report: Email template sheet
motorola confidential
Demo Daily Report: Preparing email
motorola confidential
Demo Daily Report: Sending email
motorola confidential
Demo Daily Report: Signal vs Noise and Main Drivers
motorola confidential
Demo Daily Report: Key Milestones
motorola confidential
Demo Daily Report: Final Email
motorola confidential
Demo Daily Report: Scheduling
motorola confidential
• Business overview
• Demos• Daily Activations Report
• Moto Insights
• Drive Insights
Agenda
motorola confidential
Moto Insights
How to provide insights to executive-on-the-go with a
robust system where new report take seconds to create?
Existing Situation
- Require VPN- New Report take weeks- New features take months- Issue Tickets come weekly
motorola confidential
Demo Moto Insights
Moto InsightsApp Engine
Data Source:Big Query
ReportMeta Data:Datastore
Tracking:Datastore
Users Access:Google Users
(email) +Datastore (role)
Moto InsightsWeb portal
Moto Insights Android App
motorola confidential
Demo Moto Insights: Main menu & Responsive Design
responsive design
motorola confidential
Demo Moto Insights: Report Details
motorola confidential
Demo Moto Insights: 1st Chart Details
motorola confidential
Demo Moto Insights: Chart Types
motorola confidential
Demo Moto Insights: Dummy Example
motorola confidential
• Business overview
• Demos• Daily Activations Report
• Moto Insights
• Drive Insights
Agenda
motorola confidential
Drive Insights
How to democratize Analytics within the company while
maintaining quality (data & insights) as well as maintain Big
Data usage under control?
Existing Situation
- Reports are produce by a centralized team- Role management is becoming out of control- Product teams are complaining to have to run SQL repeatedly on BigQuery console
motorola confidential
Demo Drive Insights (v2)
Drive InsightsApp Engine
Data Source:Big Query
Data Source:Google Analytics
iFrame Source:Tableau Server
iFrame Source:Google Documents
Data Source:Spreadsheet & CSV
ReportMeta Data:
Google Drive
(Text file with JSON)
Report Meta Data:Datastore
(Report copy & usage tracking)
Users Access Control:
Google Users + Drive Sharing
Google Drive
Moto Insights Android App
Drive InsightsWeb portal
motorola confidential
Demo Drive Insights: New document type
motorola confidential
Demo Drive Insights: Report creation
motorola confidential
Demo Drive Insights: 1st Chart Details
motorola confidential
Demo Drive Insights: 2nd Chart Details
motorola confidential
Demo Drive Insights: Chart Types
motorola confidential
Demo Drive Insights: SQL Editor (CodeMirror)
motorola confidential
Demo Drive Insights: Report document in your Google Drive
motorola confidential
Demo Drive Insights: Dummy Example
motorola confidential
Demo Drive Insights: Drive Sharing
motorola confidential
Q&A
?Dr Patrick DeglonDirector of Engineering, Analytics Area Tech LeadMotorola Mobility [email protected] pdeglon
motorola confidential
In this presentation, Patrick Deglon will share his learnings and provide best practices when using open Google tools & API. He will present his daily email report that hundreds of key Motorola stakeholders are receiving to drive the business, as well as a mobile solution based on the latest web technologies, including Google Visualization, Bootstrap CSS and many of the Google APIs (Gmail, BigQuery, Analytics, Drive, App Engine, Users authentication, etc.).
Abstract
motorola confidential
Dr Patrick DeglonDirector of Engineering, Analytics Area Tech LeadMotorola Mobility
With a PhD in Particle Physics, Patrick Deglon spent the last decade driving business insights at eBay and at Motorola Mobility, a Google company.At eBay, he led significant improvements in marketing effectiveness by developing methods to measure incremental sales, and by running large scale experiments on Internet marketing channels.
He joined Motorola Mobility in 2013 to help raise the bar in Analytics and on-board open Google tools and technologies. He is now the Area Tech Lead for Analytics within the Cloud Services organization.
He is married with two kids and enjoys his town of Campbell, CA.
Bio