module 6 forecasting policy delphi analy

8/8/2019 Module 6 Forecasting Policy Delphi Analy

1/16

Module 6Module 6

Forecasting Expected PolicyForecasting Expected PolicyOutcomes With ExpertOutcomes With Expert

JudgmentJudgmentPolicyPolicy

Delphi AnalysisDelphi Analysis


2/16

POLICYPERFORMANCE

EXPECTEDOUTCOMES

PREFERREDPOLICIES

OBSERVEDOUTCOMES

ForecastingEvaluation

Monitoring Recommendation

problem

structuring

problem

structuring

POLICYPROBLEMS


3/16

Learning ObjectivesLearning Objectives

Distinguish projections, predictions, andDistinguish projections, predictions, andconjecturesconjectures

Understand the effects of institutionalUnderstand the effects of institutionalcontexts on forecast accuracycontexts on forecast accuracy

Contrast potential, plausible, and normativeContrast potential, plausible, and normativefuturesfutures

Evaluate methods of crossEvaluate methods of cross--impact analysisimpact analysisand feasibility assessmentand feasibility assessment

Use policy Delphi analysis to make point andUse policy Delphi analysis to make point andinterval forecastsinterval forecasts

Create a Delphi forecast of MacedoniasCreate a Delphi forecast of Macedoniasaccession to the European Unionaccession to the European Union


4/16


5/16

The Logic of Theoretical PredictionThe Logic of Theoretical Prediction

I C

R

B

So far, Macedonia

is being isolatedfrom the European

Union

EU members will

realize that the costsare unacceptably

great, and stop the

isolation.

Thus

Probably

Unless: What is seen as a

cost by one party is seen as a

benefit by another. Actually,

EU countries are not isolating

Macedonia.

The greater the cost of an alternative,

the less likely it will be pursued.

The isolation of Macedonia

from the EU will destabilize

the region, which will be

very costly to Europe.

Probably not

Q

W


6/16

The Logic of Expert JudgmentThe Logic of Expert Judgment

I C

R

B

The research on EU

accession indicates thatMacedonia will become

a member sometime

between 2010 and

2020.

Macedonia will

become an EUmember between

2010 and 2020.

Thus

This is 95%

probable.

Unless: The experts are not

credible. They ignore intervening

events that make accession more(less) likely. Expert consensus

produces unreliable results.

The experts are credible because they have

the experience and insight to predict the future

of EU accession.

Probably not

Q

W

Experts on EU accession are

conducting credible research

in well-known think tanks.


7/16

Principles of Delphi AnalysisPrinciples of Delphi Analysis

AnonymityAnonymityParticipants are relatively equalParticipants are relatively equalin their freedom to make judgements andin their freedom to make judgements andprovide reasonsprovide reasons

IterationIterationRepeated rounds permitRepeated rounds permit

participants to change their judgementsparticipants to change their judgements Controlled feedbackControlled feedbackQuestionnairesQuestionnaires

revised after 2revised after 2ndnd-- and subsequent roundsand subsequent rounds

Statistical group responseStatistical group responseMedian, range,Median, range,interinter--quartile range, semiquartile range, semi--interquartile rangeinterquartile range

ExpertconsensusExpertconsensusTraditional DelphiTraditional Delphi

Stakeholder disagreementStakeholder disagreementPolicy DelphiPolicy Delphi


8/16

Steps in a Policy Delphi AnalysisSteps in a Policy Delphi Analysis

Issue selectionIssue selectionWhen will Macedonia become a member of theWhen will Macedonia become a member of theEuropean Union?European Union?

Selection of diverse stakeholdersSelection of diverse stakeholders((advocatesadvocates))Use snowballUse snowball

sampling, starting with stakeholders known to disagree.sampling, starting with stakeholders known to disagree.

Design questionnaireDesign questionnaireInclude questions (items) about issues,Include questions (items) about issues,

forecasts, goals, constraints, and options.forecasts, goals, constraints, and options.

Analyze and provide anonymous group feedback on 1Analyze and provide anonymous group feedback on 1stst--roundround

resultsresultsUse measures of variability as well as averages.Use measures of variability as well as averages.

Design 2Design 2ndnd--round questionnaireround questionnaireEliminate items on which thereEliminate items on which there

is greatestis greatest agreementagreement.. Organize group meetingOrganize group meetingStakeholders (advocates) discussStakeholders (advocates) discuss

reasons for disagreement.reasons for disagreement.

Prepare reportPrepare reportSeek new and creative ideas (goals, options,Seek new and creative ideas (goals, options,

constraints) as well as consensus.constraints) as well as consensus.


9/16

Forms of Providing FeedbackForms of Providing Feedback

Difference graphDifference graph

Histogram or bar chartHistogram or bar chart

StemStem--andand--leaf plotleaf plot

BoxBox--andand--whisker plotwhisker plot

Lists of measures of central tendencyLists of measures of central tendencyand dispersionand dispersion


10/16

Difference GraphDifference Graph

When will Macedonia become a

member ofthe European Union?

2005 2020 2035

NEVER


11/16

Histogram F04Histogram F04

When will Macedonia become a member of the EU?

2025.02022.52020.02017.52015.02012.52010.02007.52005.0

NumberofStakehol

ders

8

6

4

2

0


12/16

Histogram F05Histogram F05

In what year MK will become EU member?

2020.02018.02016.02014.02012.02010.02008.0

7

6

5

4

3

2

1

0

Std. Dev = 3.46

Mean = 2012.5

N = 21.00


13/16

Bar ChartBar Chart

When will Macedonia becme a member of the EU?

2025

2019

2018

2017

2015

2014

2012

2011

2010

2009

2006

2005

NumberofStakeholders

5

4

3

2

1

0


14/16

StemStem--andand--Leaf PlotLeaf Plot

Frequency

.00 200 .

5.00 200 . 56699

8.00 201 . 00001224

4.00 201 . 5789

1.00 Extremes (>=2025)

Stem width: 10Each leaf: 1 case(s)


15/16

BoxBox--andand--Whisker PlotWhisker Plot

18N =

When will Macedonia become a member of the EU?

2030

2020

2010

2000

18


16/16

module 6 forecasting policy delphi analy

Documents