session 8: conclusions

Session 8: Conclusions

Demand Forecasting and Planning in Crisis30-31 July, Shanghai

Joseph Ogrodowczyk, Ph.D.

Session 8 Joseph Ogrodowczyk, Ph.D.

Demand Forecasting and Planning in Crisis 30-31 July, Shanghai 2

Conclusions Session agenda

Forecasting changes Evaluating forecasting capabilities of software A word on Microsoft Excel Creating a forecaster training program Resources


Demand Forecasting and Planning in Crisis, 30-31 July, Shanghai 3

Conclusions Forecasting changes

Very difficult to forecast sharp or hard-to-reverse changes No magic bullet for forecasting quantities Need tools that will guide a forecasting process

Linear models estimate levels (quantities) of the dependent variable

Non-linear models can estimate the probability of the state of the dependent variable

Linear model

-6

-4

-2

0

2

4

6

0 2 4 6 8 10Value of Independent

Value of Dependent

Probability model

0.0

0.3

0.5

0.8

1.0

0 2 4 6 8 10 12

Good

Bad

Probability of Good

Value of Independent




Two model types1. Modeling probabilities for future events

Statistical models called probit and logit estimate the probability that an event will occur

Independent variables are correlated with an event ISM purchasing manager’s index, price of oil, stock price

index, etc. Event is the state of the macroeconomy (Good, Bad)

As the independent variables move, the probability of the dependent variable being either Good or Bad adjusts




Two model types1. Modeling probabilities for future events

Choosing the “right” independent variables Variables can differ in effect on various occurrences

(simple models are structurally more stable than complex models)

Try to include recent history as a basis (past events are not always an indication of future events)

Historical macroeconomic data are subject to change (re-test models for explanatory robustness)

Key independent variables can change (test the relevance of the independent variables)




Two model types2. Decision-directed forecasting

Forecasting tool using after a major disruption Manager experience and statistical models may not be

able produce reliable forecasts Goal is to identify probable scenarios as guidelines for

making operational decisions Steps to the process

Manager identifies decision options and scenarios Management assigns probabilities to future outcomes Forecasters calculate new probabilities as new data

become available (Bayesian calculations) When one scenario becomes more likely, manager

selects the appropriate course of action




Two model types2. Decision-directed forecasting: Example

Casino actions in Las Vegas following 9/11 Following 9/11, casinos would need to incorporate a

sharp change in macroeconomic environment Decision actions included: Business as usual (revenue

decreases are short term), Plan for short term disruption (mandatory vacations, hold on capital projects, etc.), Recovery might never fully materialize (short and long term cutbacks), New business strategy (new ventures in other industries)





Scenario Description Business optionManager's first

probabilityA Revenues return within 3 months Business as usual 15%

BRevenues fall flat for 3 months but return by 9 months

Plan for short term disruption 35%

CRevenues fall flat for 6 months and increase at prior rate

Recovery might never fully materialize 35%

D Revenues continue to decline New business strategy 15%

Probabilities in mid-September 2001








Updated probabilities in mid-October 2001





• Probabilities are updated each month.• By January 2002, Scenarios A and D are highly unlikely• By May 2002, Scenario B is the most likely








Updated probabilities in May 2002




Two model types2. Decision-directed forecasting

• As the probabilities are updated each month, a manager can adjust the business plan

• Approach can also be used for new product introductions, responses to competitors, or new pricing strategies

• Other methods for choosing among scenarios include: Payoff tables and Decision Trees

• Techniques are available for assisting in choosing the starting probabilities



Conclusions Evaluating forecasting abilities of software

Background Companies make IT choices based on broad goals such as

supply chain management (SCM) and enterprise resource planning (ERP) needs

Demand planning software is often included with SCM or ERP systems Stand-alone forecasting software requires additional IT

knowledge, time, and funding Decision makers must balance the sophistication of the

forecasting techniques against the ease of interconnectivity of the software

What is the best way for evaluating the forecasting capabilities of the demand planning packages?




Steps in evaluation Designing a test drive

Need to test the software forecast accuracy on the organization’s data Work with vendors on a pilot study

Sample data should include a broad range of products and include all levels of the hierarchies Keep in mind any model requirements with respect to the

length of the dataset Include various demand patterns (intermittent, new product,

stable demand, etc.)




Steps in evaluation Evaluating results

Choose an evaluation period (ex post) Choose an accuracy calculation

Preference should be given to the current calculation to ensure that the software provides a reduction in error

Comparing performance Accuracy improvement may not necessarily be due to the

forecasting algorithms Other benefits from the software can include

Incorporation of point-of-sale data Ability to support Collaborative Planning, Forecasting, and

Replenishment (CPFR)



Forecasting software options

Modules in Broad Scope Statistical Software:SAS (ETS) STATA SPSS Insightful (S+FinMetrics) R

Business Forecasting Software:Autobox Stamp Forecast Pro SmartForecasts Decision Time

Demand Planning:McConnell-Chase (Forecasting for Demand)Demand Works (Smoothie)Oracle (Peoplesoft Enterprise Demand Planning)John Galt (Atlas Planning Suite)Demand Management (Demand Solutions)Delphus (Peer Planner)Modules in SAP, I2, and JDA (formerly Manugistics)

Econometrics Packages: E-Views RATS and CATS PC Give

See http://www.oswego.edu/~economic/econsoftware.htm



Conclusions A word on Excel

Overview Previous sessions of workshop used Excel

Excel can and is used for forecasting Good for quick estimates and general guideline However, Excel is not a robust forecasting tool

Specialized forecasting software (or forecasting capabilities within a demand planning software) is recommended in support of a forecasting process




Tips for using of Excel Advantages of Excel

Allows data to be visible Formulas are accessible and can be edited Calculations can be saved Scenarios can be planned using parametric analysis

Caution of Excel Excel is not a statistical software Statistical procedures do not yield accurate or precise solutions




Tips for using of Excel Guidelines

Match the tool to the job Excel is a good tool to estimate a range

Understand how to use the tool to accomplish the job Excel makes it easy for users to think they are properly

applying the wrong model to a data set Users can program statistical and forecasting equations into

Excel to obtain correct calculations Increasing the capabilities of the tool can increase the quantity

and quality of the jobs finished by the tool Consider add-ons for Excel Education for users




Tips for using of Excel Useful resources for analysis on Excel

http://www.daheiser.info/excel/frontpage.html Website documents the error and some potential solutions to

those errors Several studies documenting inconsistencies of statistical

software (McCullough 1999) SAS, SPSS (McCullough and Wilson 2005) Excel 2003 Studies also show a willingness to correct errors by SAS and

SPSS but not Microsoft (McCullough and Wilson 2002)




Nonlinear curves example In calculating coefficients of nonlinear functions (such as

exponential functions), Excel transforms the data into a line Transformation leads to incorrect calculations Optimal coefficients of a function are found by a model error

metric (such as the root mean square error or RMSE) Exponential function defined as

a: intercept (assumed to be 0)b: growth ratec: lower limitT: time (year for this example)e: mathematical constant of 2.718growth rate as a percent per year = eb-1




Nonlinear curves example Sample data

GraphSales

$0

$500

$1,000

$1,500

$2,000

$2,500

$3,000

0 5 10 15Year

Year 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Sales $301 $320 $372 $423 $500 $608 $721 $826 $978 $1,135 $1,315 $1,530 $1,800 $2,152 $2,491




Nonlinear curves example Choosing an exponential trend line in Excel Excel will find the optimal coefficient of the followingtransformed data

ln(Sales)

$5

$6

$7

$8

$9

$10

0 5 10 15Year

Year ln(Sales)1 $5.7072 $5.7683 $5.9194 $6.0475 $6.2156 $6.4107 $6.5818 $6.7179 $6.88610 $7.03411 $7.18212 $7.33313 $7.49614 $7.67415 $7.820




Nonlinear curves example Excel will convert the result from the linear function back into

the exponential function for the fit line of

where growth rate as a percent per year = e.1561-1 = 16.89%This is the annual growth rate of salesNotice that Excel assumes an intercept of zero



A word on Excel Nonlinear curves example

A statistical software will optimize the exponential function directly for a fit line of

where growth rate as a percent per year = e.1608-1 = 17.44%This is the annual growth rate of salesNotice that there is a non-zero intercept

Conclusions




Nonlinear curves example Comparing the results

Statistical Excel Parameters software Non-linear fit

a= 221.7886 237.5583b= 0.1608 0.1561c= 19.9793 0

Year Sales Statistical Statistical Excel Excelforecast error forecast error

1 $301 $280.46 -$20.54 $277.69 -$23.312 $320 $325.90 $5.90 $324.61 $4.613 $372 $379.27 $7.27 $379.45 $7.454 $423 $441.95 $18.95 $443.55 $20.555 $500 $515.56 $15.56 $518.49 $18.496 $608 $602.01 -$5.99 $606.08 -$1.927 $721 $703.55 -$17.45 $708.47 -$12.538 $826 $822.79 -$3.21 $828.17 $2.179 $978 $962.85 -$15.15 $968.08 -$9.92

10 $1,135 $1,127.33 -$7.67 $1,131.63 -$3.3711 $1,315 $1,320.51 $5.51 $1,322.81 $7.8112 $1,530 $1,547.38 $17.38 $1,546.29 $16.2913 $1,800 $1,813.84 $13.84 $1,807.52 $7.5214 $2,152 $2,126.78 -$25.22 $2,112.89 -$39.1115 $2,491 $2,494.31 $3.31 $2,469.85 -$21.15 Percent

Difference DifferenceSum squared errors $2,920.90 $4,018.70 $1,097.81 37.6%Root 54.05 63.39 $9.35 17.3%

16 $2,925.96 $2,887.12 -$38.84 -1.3%17 $3,432.90 $3,374.87 -$58.03 -1.7%18 $4,028.29 $3,945.03 -$83.25 -2.1%

Sum $10,387.15 $10,207.02 -$180.12 -1.7%



Conclusions Creating a forecaster training program

1. Program outline Program mission statement Clearly defined objectives based on ideal forecaster

characteristics Resource requirements

2. Create a measurement baseline Classify objectives into core learning areas

Forecasting and supply chain concepts Technical and software skills Process management and product knowledge Interpersonal skills

Collect data on the current ability level of the forecasters




3. Create development plan through gap analysis Define critical skills corresponding to core learning areas Compare current forecaster knowledge with critical skills Prioritize the gaps based upon criteria such as importance

of core area, size of gap, quantity of forecasters, and estimated resources needed to close the gap

4. Implementation of education program Define the content of core learning areas Outline methods used for education

Separate sessions, week long workshops, etc. Consultants or internal resources Quantity of IT support needed Use of mentoring, coaching, on-the-job training




5. Evaluation and areas of improvement for program Maintenance of forecaster education

Continuing education internally, plans for using external resources or outside education

Lessons learned from implementation Changing the course schedule or material covered Incorporating participant feedback

Defining additional supporting infrastructure Guiding principles for demand management education Roles and responsibilities of the instructors Future expectations of additions to core areas



Conclusions Resources

Forecasting portals www.appliedforecasting.com

Latest news on forecasting events, tool and papers www.forecastingeducation.com

Listing of and links to software reviews and special reports on forecasting software

www.demandplanning.net S&OP and demand management consulting and courses



Conclusions Resources

Forecasting organizations International Institute of Forecasting

www.forecastingprinciples.com Institute of Business Forecasting

www.ibf.org Both offer

Courses, seminars, and publications



Special FeaturesSpecial Features

Overcoming Challenges in Operational Overcoming Challenges in Operational Forecasting ProjectsForecasting Projects

The Organizational Politics of The Organizational Politics of Forecasting:6 Steps to Overcome BiasForecasting:6 Steps to Overcome Bias

Forecast Accuracy Metrics for Inventory Forecast Accuracy Metrics for Inventory ControlControl

The What, Why, and How of Futuring The What, Why, and How of Futuring for Forecastersfor Forecasters

Benchmarking of Forecast AccuracyBenchmarking of Forecast Accuracy plus Software Reviews, Book Reviews plus Software Reviews, Book Reviews

and briefs on Hot New Research and briefs on Hot New Research

www.forecasters.org/foresight

FORESIGHT: ConciseConcise, , objectiveobjective and and readable readable articles on issues essential to the practicing forecasterarticles on issues essential to the practicing forecaster. .



Conclusions References (Forecasting changes)

Batchelor, Roy. 2009. Forecasting sharp changes. Foresight. Spring: 7-12.

Custer, Stephen and Don Miller. 2007. Decision-directed forecasting for major disruptions: The impact of 9/11 on Las Vegas gaming revenues. Foresight. Summer: 29-35.

Jain, Chaman L. and Jack Malehorn. 2005. Practical Guide to Business Forecasting (2nd Ed.). Flushing, New York: Graceway Publishing Inc.

Sephton, Peter. 2009. Predicting recessions: A regression (probit) model approach. Foresight. Winter: 26-32.



Conclusions References (Forecasting software)

Fields, Paul. 2006. On the use and abuse of Microsoft Excel. Foresight. February: 46-47.

Hesse, Rick. 2006. Incorrect nonlinear trend curves in Excel. Foresight. February: 39-43.

Hoover, Jim. 2005. How to evaluate the forecasting ability of demand-planning software. Foresight. June: 47-49.

McCullough, Bruce D. 1999. Assessing the reliability of statistical software: Part 2. The American Statistician. 53(2): 149-159.

McCullough, Bruce D. and B. Wilson. 2005. On the accuracy of statistical procedures in Microsoft Excel 2003. Computational Statistics and Data Analysis. 49(4): 1244-1252.

McCullough, Bruce D. and B. Wilson. 2002. On the accuracy of statistical procedures in Microsoft Excel 2000 and Excel XP. Computational Statistics and Data Analysis. 40(4): 713-721.

McCullough, Bruce D. 2006. The unreliability of Excel’s statistical procedures. Foresight. February: 44-45.

session 8: conclusions

Documents

conclusionsdemand forecasting

bad adjustsdemand forecasting

shanghaijoseph ogrodowczyk

new probabilities

retest models

future outcomesforecasters

future eventschoosing

new data