using salesforce, erp, tableau & r in sales forecasting
TRANSCRIPT
SALES FORECASTING: USING SALESFORCE, ERP, TABLEAU& R
• Introduction• Predictive Modeling Methodology• Data Preparation• Accuracy vs Complexity• Choosing a Model• Evaluating and Deploying the Model• Senturus Overview• Additional Resources
Agenda
Copyright 2016 Senturus, Inc. All Rights Reserved.
Presenters
Copyright 2016 Senturus, Inc. All Rights Reserved.
Asa LeviSenior Consultant
Senturus, Inc.
Greg HerreraPresident and Co-
FounderSenturus, Inc.
Hundreds of Free Resourceshttp://www.senturus.com/resources/
RESOURCE LIBRARYAn extensive, free library of past webinars, demonstrations, whitepapers, presentations, helpful hints, and more.
Copyright 2016 Senturus, Inc. All Rights Reserved.
This slide deck is from the webinar: A Pragmatic Approach to Sales Forecasting, Using Salesforce, ERPs, Tableau & R To view the FREE video recording of the presentation or download this deck, go to:
http://www.senturus.com/resources/a-pragmatic-approach-to-sales-forecasting
/
Hear the Recording
Copyright 2016 Senturus, Inc. All Rights Reserved.
A PRAGMATIC APPROACH TO SALES FORECASTING USING SALESFORCE, ERPS, TABLEAU & R
Predictive Modeling Methodology
• Define business goals • Specify model objective
function • Design/build modeling
database• Partition modeling data• Derive potential predictors• Analyze predictor strength• Perform sub-population
analysis• Build model algorithms• Evaluate model
performance • Deploy model
Copyright 2016 Senturus, Inc. All Rights Reserved.
Sales Forecast Algorithms RequireHistorical Snapshots
Opportunity Amount
Stage Probability
Close Date
Northwind Traders
$45k Early 20% 2016-08-25
Rugged Bikes $50k Middle 40% 2016-08-31
Grand Cycle Store
$65k Favored 60% 2016-07-25
Six months ago, the pipeline said this:
Account Revenue Last 12 Months
Order Frequency
Northwind Traders
$450k 4 in last 6 months
Rugged Bikes $0 UnknownGrand Cycle Store
$650k 12 in last 6 months
Six months ago, the ERP System said this:
Copyright 2016 Senturus, Inc. All Rights Reserved.
Sales Pipelines are Current Snapshots
Opportunity Amount
Stage Probability
Close Date
Bike Universe $50k Early 20% 2016-12-25
The Cracker Box $40k Middle 40% 2017-02-25
Racing Bike Outlet
$35k Favored 60% 2016-09-01
Northwind Traders
$65k Verbal 70% 2016-11-25
Rugged Bikes $60k Closing 90% 2016-08-31
Grand Cycle Store
$55k Closed Won
100% 2016-07-25
As of now, the pipeline says this:
In Salesforce and most other CRM systems, the sales pipeline is tracked in an opportunity table, which maintains only the current values.
Copyright 2016 Senturus, Inc. All Rights Reserved.
• Configure the CRM system to capture each pipeline update and organize these records in the data warehouse, or
• Use the data warehouse for everything– Store periodic snapshots of the entire pipeline (e.g.,
Daily, Weekly, Monthly)– Capture each pipeline update by comparing new
data warehouse records with the existing records from the prior warehouse load
Techniques for Maintaining Historical Pipeline
Copyright 2016 Senturus, Inc. All Rights Reserved.
Salesforce has built-in functionality that tracks changes to a specific group of five pipeline fields:
1. Amount2. Close Date3. Forecast Category4. Probability5. Stage
Salesforce Opportunity History Table
Copyright 2016 Senturus, Inc. All Rights Reserved.
Salesforce Tracks History Functionality
• Salesforce can be configured to track changes to any/all fields in the opportunity table
• The records will appear in opportunity field history
Predictive Modeling Methodology
• Define business goals • Specify model objective
function • Design/build modeling
database• Partition modeling data• Derive potential predictors• Analyze predictor strength• Perform sub-population
analysis• Build model algorithms• Evaluate model
performance • Deploy model
Copyright 2016 Senturus, Inc. All Rights Reserved.
Goal: Merge Predictions for Actionable Insights
Opportunity Amount
Stage
Bike Universe $50k EarlyThe Cracker Box $40k MiddleRacing Bike Outlet $35k FavoredNorthwind Traders $65k VerbalRugged Bikes $60k ClosingGrand Cycle Store $55k Closed Won
Add prediction information
• Prediction data should be stored in another table, along with the date of the prediction
• Certain data points are identified for prediction
Opportunity
Prediction Date
Amount
Probability
Bike Universe
5/1/16 $30k Low
Bike Universe
6/1/16 $40k Low
Bike Universe
7/1/16 $60k High
Bike Universe
8/1/16 $50k Medium
Quantitative vs. Qualitative
Copyright 2016 Senturus, Inc. All Rights Reserved.
• Quantitative- How much (do we expect to sell)?
• Qualitative- Which (probability group is this lead in)?
• Different methods• Different measurement
Opportunity Prediction Date
Amount
Probability
Bike Universe 5/1/16 $30k Low
Bike Universe 6/1/16 $40k Low
Bike Universe 7/1/16 $60k High
Bike Universe 8/1/16 $50k Medium
How to Measure Accuracy
Copyright 2016 Senturus, Inc. All Rights Reserved.
• Separate data into training and test data sets• Models built on training data• Performance is measured on the test data• Measure by a core set of KPIs• Results fed back for further model iteration
Copyright 2016 Senturus, Inc. All Rights Reserved.
Regression KPIs• Absolute Error (AE)
• Mean Absolute Percentage Error (MAPE)
• Bias
• Accuracy = 1 - MAPE
Actual
ActualForecastOpportunity
Prediction
Actual
AE MAPE
Bias
Bike Universe
$30k $50k $20k 40% -40%
Rugged Bikes $40k $30k $10k 33% 33%Northwind Traders
$60k $55k $5k 9% 9%
Total $130k $135k
$35k
26% 4%
Copyright 2016 Senturus, Inc. All Rights Reserved.
Custom KPIs• Standard KPIs assign equal penalty to over and
under forecasting – this is usually unrealistic• Custom KPIs can come to the rescue• For example, MAPE is calculated like this:
• If a bike costs $40 to manufacture and sellsfor $400 there may be a higher cost for missed sales than over production
Actual
ActualForecastOpportunity
Prediction
Actual
MAPE
Bias
Bike Universe
$30k $50k 40% -40%
Rugged Bikes $40k $30k 33% 33%Northwind Traders
$60k $55k 9% 9%
Total $130k $135k
26% 4%
The Common Refrain
Copyright 2016 Senturus, Inc. All Rights Reserved.
Why can I predict that our net sales will be within 5%of last year, but we are 30% off at the product level?!?
• The answer is hidden in the total bias calculation• A simple analogy is predicting the sum of dice rolls• Suppose you have 100 dice • Try to predict a single roll – what is the best guess• If you guess 3 or 4 your expected error is 50%• Now try to predict the sum of 100 rolls• Guess 350 rolls and your expected error would be around
5%
The Central Limit Theorem
Copyright 2016 Senturus, Inc. All Rights Reserved.
To view the FREE video recording of the presentation and download this deck, go to:
http://www.senturus.com/resources/a-pragmatic-approach-to-sales-forecasting
/
The Senturus comprehensive library of recorded webinars, demos, white papers, presentations, and case studies is available on our website:
http://www.senturus.com/resources/
Hear the Recording
Copyright 2016 Senturus, Inc. All Rights Reserved.
Granularity , Accuracy
Copyright 2016 Senturus, Inc. All Rights Reserved.
• The more granular a prediction, the less accurate it is• A prediction is like a random choice in the area under the
curve, the area is more centralized as the data gets more accurate
Copyright 2016 Senturus, Inc. All Rights Reserved.
Accuracy By Industry
• No singular goal works for all industries• Important: understand the business model when
setting accuracy goals
Copyright 2016 Senturus, Inc. All Rights Reserved.
Choosing a Model• With a measurement framework, begin
model selection• Choose the predictors
- Data elements fed into our model• This step is not prescriptive
- Use business knowledge and analysis to choose the right predictors
• From our ERP and CRM data we create a set of possible predictors:
YTD RevenueRemaining Revenue
PPY Remaining Revenue
PY Revenue
PYTD Revenue
PY Remaining Revenue
PPY Revenue
PPYTD Revenue
Low Likelihood Revenue
Med. Likelihood Revenue
High Likelihood Revenue
Copyright 2016 Senturus, Inc. All Rights Reserved.
Choosing a Model
YTD RevenueRemaining Revenue
PPY Remaining Revenue
PY Revenue
PYTD Revenue
PY Remaining Revenue
PPY Revenue
PPYTD Revenue
Low Likelihood Revenue
Med. Likelihood Revenue
High Likelihood Revenue
• Figure out which predictors to use in our modeling
• Look at the relationship between the predictors
Copyright 2016 Senturus, Inc. All Rights Reserved.
Investigating Predictor RelationshipsYTD Revenue
Remaining Revenue
PPY Remaining Revenue
PY Revenue
PYTD Revenue
PY Remaining Revenue
PPY Revenue
PPYTD Revenue
Low Likelihood Revenue
Med. Likelihood Revenue
High Likelihood Revenue
YTD Reve
nue
Remain
ing Revenue
PPY Rem
aining
Revenue
PY Reve
nue
PYTD Reve
nue
PY Rem
aining Reve
nue
PPY Reve
nue
PPYTD
Revenue
Low Lik
elihood
Revenue
Med. Like
lihood R
evenu
e
High Like
lihood Rev
enue
Copyright 2016 Senturus, Inc. All Rights Reserved.
Removing Outliers• Unusual data can skew results during model
creation• Outliers can be automatically cleaned using
the mvoutlier package in R• Remove outliers in a consistent and
documented manner
Copyright 2016 Senturus, Inc. All Rights Reserved.
Investigating Predictor RelationshipsYTD Revenue
Remaining Revenue
PPY Remaining Revenue
PY Revenue
PYTD Revenue
PY Remaining Revenue
PPY Revenue
PPYTD Revenue
Low Likelihood Revenue
Med. Likelihood Revenue
High Likelihood Revenue
YTD Reve
nue
Remain
ing Revenue
PPY Rem
aining
Revenue
PY Reve
nue
PYTD Reve
nue
PY Rem
aining Reve
nue
PPY Reve
nue
PPYTD
Revenue
Low Lik
elihood
Revenue
Med. Like
lihood R
evenu
e
High Like
lihood Rev
enue
Copyright 2016 Senturus, Inc. All Rights Reserved.
Setting Up A RegressionUse best subset selection to find the best set of predictors given a number of predictors
Separate models need to be compared by plotting RSS and adjusted R2
Copyright 2016 Senturus, Inc. All Rights Reserved.
Occam’s Razor Applies!
• Improve accuracy on the training data by choosing a more complex method
• More flexible methods produce better results on training – not necessarily on test data though
Results
Copyright 2016 Senturus, Inc. All Rights Reserved.
• Using best subset selection we were able to generate a linear model with these coefficients:
• Model attained 24.4% MAPE at the customer level, predicting 2015 sales on the test data set
• Model can be stored and applied over and over in the future
Error Histogram
Learn and Iterate
Copyright 2016 Senturus, Inc. All Rights Reserved.
Each time predictions are ran, the MAPE is stored, and it should be periodically evaluated for accuracy and possible predictor changes
Error Histogram
Insights
Copyright 2016 Senturus, Inc. All Rights Reserved.
• Use the predictions to take action• Some of the predictions:
YTD Revenue
Low Likelihood
Medium Likelihood
High Likelihood Pipeline Prediction
Last week’s Prediction
Bike Universe $1,419,255 $732,754 $375,930 $26,350 $1,135,035 $1,988,451 $1,987,455
Racing Bike Outlet $1,109,188 $1,075,916 $487,885 $247,269 $1,811,071 $1,731,399 $1,730,443
Northwind Traders $1,446,140 $112,982 $243,254 $104,520 $460,756 $1,560,967 $1,560,565
Rugged Bikes $1,140,681 $1,136,466 $253,616 $770,128 $2,160,210 $1,917,966 $1,714,933
Grand Cycle Store $1,606,350 $1,685,855 $552,993 $296,978 $2,535,825 $2,820,317 $2,820,317
• Rugged Bikes is down year over year, and far below pipeline
Copyright 2016 Senturus, Inc. All Rights Reserved.
Clustering to Find InsightLook at customer ordering patters and pipeline to see similar behaviors among groups of customers
Opportunity Avg Order Size Orders Pipeline PY Sales
Bike Universe110 10 $30,000 $40,000
The Cracker Box 140 5 $20,000 $30,000Racing Bike Outlet 150 6 $10,000 $15,000Northwind Traders 140 7 $15,000 $15,000Rugged Bikes
150 3 $12,000 $10,000
Copyright 2016 Senturus, Inc. All Rights Reserved.
Clustering to Find Insight• Pattern may be unknown – look to see if there are
“similar” customers• Clustering algorithms can help• Predictor selection is key• Need to normalize your data before processing
Copyright 2016 Senturus, Inc. All Rights Reserved.
ConclusionPredictive methods can help:
• Give insight into future values- “How much sales can we expect for the rest of
the year at Bike Universe?”• Classify data
- “Which of our customers behave similarly, and how do we target our actions to serve each group best?”
• Quickly identify changes in large data sets- “Why are sales predictions down so much at
Rugged Bikes?”
To view the FREE video recording of the presentation and download this deck, go to: http://www.senturus.com/resources/a-pragmatic-approach-to-sales-forecasting
/
The Senturus comprehensive library of recorded webinars, demos, white papers, presentations, and case studies is available on our website:
http://www.senturus.com/resources/
Hear the Recording
Copyright 2016 Senturus, Inc. All Rights Reserved.
Business Analytics ConsultantsWHO WE ARE
Bridging the Gap Between Data & Decision Making
DECISIONS &
ACTIONS
Business Needs
Analysis Ready
Data
Analysis Ready Data
.
• Dashboards, Reporting & Visualizations• Data Preparation & Modern Data
Warehousing • Self-Service Business Analytics • Big Data & Advanced Analytics• Planning & Forecasting Systems
Business Analytics Architects
41Copyright 2016 Senturus, Inc. All Rights Reserved.
900+ Clients, 2000+ Projects, 16+ Years
ADDITIONAL RESOURCES
www.senturus.com/events Upcoming Free Events
Copyright 2016 Senturus, Inc. All Rights Reserved.
Upgrading to Cognos Analytics: What You Need to Know
Presented by:Todd SchumanSenturus Practice LeadInstallations, Upgrades, and Optimization
Upcoming Event
Copyright 2016 Senturus, Inc. All Rights Reserved.
More Free Resources http://www.senturus.com/resources/
Copyright 2016 Senturus, Inc. All Rights Reserved.
Copyright 2016 Senturus, Inc. All Rights Reserved.
Thank You!
www.senturus.com [email protected]
888 601 6010
Copyright 2016 by Senturus, Inc. This entire presentation is copyrighted and may not be reused or distributed without the written
consent of Senturus, Inc.