integrating judgmental and quantitative forecasts stephen macdonald, ers/usda research center on...
TRANSCRIPT
Integrating Judgmental and Quantitative Forecasts
Stephen MacDonald, ERS/USDA
Research Center on Forecasting Seminar
January 17, 2007
Introduction
• Futures markets often find USDA’s forecasts crucial
• Resource constraints have reduced the staff-years USDA has for forecasting
• Quantitative methods can be used to supplement USDA’s traditionally judgmental forecasts– Example: international commodity trade
USDA forecasts: Two perspectives
1 2
Overview of USDA Forecasts
• National Agricultural Statistics Service (NASS) estimates U.S. production of more than 100 commodities
• 7 of these commodities have been legislatively deemed “market sensitive”– Wheat, corn, soybeans, cotton, citrus, cattle, and
hogs
• Since 1973, USDA has published demand forecasts as well– Interagency Commodity Estimates Committees
Interagency Commodity Estimates Committee (ICEC)
• ICEC comprised of: Economic Research Service (ERS)
Foreign Agricultural Service (FAS) Farm Service Agency (FSA) Agricultural Marketing Service (AMS)
World Agricultural Outlook Board (WAOB) ,chair
• Methodology of the ICEC:“A consensus…approach is used to arrive at supply and
demand estimates. Consensus forecasts employ ‘models’ of all types, formal and informal.”
February 2007 example: India 2006/07 cotton exports
• Forecasts available from several sources:• 4.2 million bales (mb): U.S. embassy (Delhi)• 3.9 mb: India Cotton Advisory Board• 4.1 mb: International Cotton Advisory Committee• “USDA forecast too high”: personal
communication from industry analysts
• No actual data was available—Indian official trade data is significantly lagged
• USDA’s forecast: 5 mb
January 2008: India exports
• 10 months of marketing year data published
• Averaged 437,000 bales per month
• Compared to 2006, Aug-May trade is:
• 1.3 m. bales higher• 44 % higher
• During previous 4 years:• Aug-May was 84% of year
0
200
400
600
800
1000
1200
Aug Oct Dec Feb Apr Jun
Thousand bales
20052006
Data available through May 2007
Declining resources: FAS & ERS
0200400600800
100012001400
1995 1998 2001 2004 2007
U.S. Embassy reports: foreign grains,
oilseeds, and cotton
0100200300400500600700
1995 1998 2001 2004 2007
ERS staff-years
Changing Forecasting Environment
• A consensus (Delphi) approach is resource-intensive: expertise– or labor–intensive
• Falling cost of data-processing and acquisition can offset reduced staffing
• Timely international commodity trade data commercially available– replacing embassy reports
Import forecasts based on data through month x
cxm
mmy
xm
mmy
YY
b
xm
m
xm
mmymyYY
a
xm
mmy
Y
I
III
IIII
x
II
,1
,
1
,1,1
,
:C
:B
12:A
d
i
i iY
xm
mmiy
xm
mmy
Y
I
I
IID
3
:
3 ,
,
Empirical confidence intervals
• Assume future errors distributed same as past
• Assume errors are normally distributed, with mean of zero
• Calculate a 90 percent confidence interval for each forecast using estimated variance and t distribution– Variance estimated with past forecasts errors
Alternative forecasts for India exports
• Weight forecasts by inverse of confidence interval
• Analysis of trade data corroborates USDA
• But: international organization has forecast outside of 90% confidence interval
Forecast LevelConfidence Limit
Million bales
USDA 5.1 --
ICAC 4.4 --
A 5.2 0.1
B 4.8 0.1
C 5.0 0.4
D 5.3 0.1
Average 5.0 0.1
Forecasting after structural change
0
1000
2000
3000
4000
5000
6000
1995 1997 1999 2001 2003 2005 2007
Thousand bales• Past error variances may be poor guide
• Genetically modified cotton increases exports
• Convert confidence limits to percentages of past Indian exports:
Forecasts
• Example– 100,000 / 837,000 = 12 %– 0.12 * 5.0 mb = 0.6 mb, alternative confidence limit
837,000 bales = 99-05 average
Forecasts: adjusted confidence limits
• Proportional confidence limit suggests ICAC forecast is not incompatible with published trade data
• However, actual exports totaled 4.4 mb already
• Alternative adjustments may be more appropriate
Forecast LevelConfidence Limit
Million bales
USDA 5.1 --
ICAC 4.4 --
A 5.2 0.5
B 4.8 0.5
C 5.0 2.8
D 5.3 0.5
Average 5.0 0.7
Integration with judgmental forecasts
• Confidence intervals expand compatibility of quantitative estimates with market intelligence from embassies and industry
• Also provide weights for combining forecasts—add intuitive appeal
• Can be incorporated into rules of thumb to guide judgmental decision-making
Integration with judgmental forecasts12/31/2007 Argentina Australia Belgium Brazil Canada
Data ends: Oct-07 Oct-07 Sep-07 Nov-07 Oct-07*(See note)
Imports
USDA (December 07) 225 0 130 400 120
Change USDA1-- -- -- -- --
Proposed forecast2 165 0 145 225 145
Advisability of change3-- -- 103% 294% 220%
Forecast range4
Minimum 80 0 130 165 135 Maximimum 255 0 155 285 160
Exports
USDA (December 07) 150 1,400 30 2,800 0
Change USDA1Reduce -- -- Reduce --
Proposed forecast2 40 1,400 45 1,950 0
Advisability of change3339% -- -- 209% --
Forecast range4
Minimum 11 1,250 29 1,500 0 Maximimum 75 1,550 60 2,350 0Source: ERS calculations using data from Global Trade Information Service (GTIS).
Conclusion
• USDA forecasting is increasingly substituting “capital” for labor
• We are exploring how to most efficiently exploit the growing availability of data
• We are determining how best to integrate these quantitative forecasts into USDA’s judgment-based system