prioritization using logic models and mira october 17. 2007 instituto nacional de ecologia mexico...
TRANSCRIPT
![Page 1: Prioritization using Logic Models and MIRA October 17. 2007 Instituto Nacional de Ecologia Mexico City, Mexico](https://reader035.vdocument.in/reader035/viewer/2022062423/56649e145503460f94afe70d/html5/thumbnails/1.jpg)
Prioritization using Logic Models and
MIRA
October 17. 2007
Instituto Nacional de Ecologia
Mexico City, Mexico
![Page 2: Prioritization using Logic Models and MIRA October 17. 2007 Instituto Nacional de Ecologia Mexico City, Mexico](https://reader035.vdocument.in/reader035/viewer/2022062423/56649e145503460f94afe70d/html5/thumbnails/2.jpg)
Part I: Logic Models
Connecting Program Activities to Environmental Outcomes
![Page 3: Prioritization using Logic Models and MIRA October 17. 2007 Instituto Nacional de Ecologia Mexico City, Mexico](https://reader035.vdocument.in/reader035/viewer/2022062423/56649e145503460f94afe70d/html5/thumbnails/3.jpg)
What is a Logic Model?
Tool to help understand how program activities affect environmental outcomes.
Kellogg Foundation template to start.Foundation Home Page:
http://www.wkkf.org/Default.aspx?LanguageID=0Logic Model Guidance Document:
http://www.wkkf.org/Pubs/Tools/Evaluation/Pub3669.pdf
Modification for environmental programs.
![Page 4: Prioritization using Logic Models and MIRA October 17. 2007 Instituto Nacional de Ecologia Mexico City, Mexico](https://reader035.vdocument.in/reader035/viewer/2022062423/56649e145503460f94afe70d/html5/thumbnails/4.jpg)
BaselineWhat is the condition of the things we care about?
ResourcesIn order to accomplish our set of activities we will need the following:
StressorsIn order of importance what are the stressors and where are they most prevalent? These will be used to target activities.
ActivitiesIn order to address our problems or asset we will accomplish the following activities.
OutputsWe expect that once accomplished these activities will produce the following evidence or service delivery. (bold = perf stds)
Short and Long term outcomesWe expect that if accomplished the se activities will lead to the following changes in 1-3 then 4-6 years.
ImpactWe expect that if accomplished these activities will lead to the following changes in 7-10 years.
Logic Model Template
![Page 5: Prioritization using Logic Models and MIRA October 17. 2007 Instituto Nacional de Ecologia Mexico City, Mexico](https://reader035.vdocument.in/reader035/viewer/2022062423/56649e145503460f94afe70d/html5/thumbnails/5.jpg)
How to build a logic model
Brainstorm activities (“individual logic model”)Program or site activities
For each activity, ask: Why do I do this activity?What is(are) the intended outcome(s) of doing that
activity?What is(are) the actual outcome(s)?What is the impact (ultimate goal) of this outcome?
![Page 6: Prioritization using Logic Models and MIRA October 17. 2007 Instituto Nacional de Ecologia Mexico City, Mexico](https://reader035.vdocument.in/reader035/viewer/2022062423/56649e145503460f94afe70d/html5/thumbnails/6.jpg)
How to build a logic model – cont’d.
Baseline = outcome/impactsNeed to measure the same thing at the baseline
as at the end.
Stressor – Distinguish between:Pollutant stressors
E.g., population growth, vehicle emissions
Program stressorsE.g., conflicting statute, no regulatory authority
Different stressors compel different activities/outcomes.
![Page 7: Prioritization using Logic Models and MIRA October 17. 2007 Instituto Nacional de Ecologia Mexico City, Mexico](https://reader035.vdocument.in/reader035/viewer/2022062423/56649e145503460f94afe70d/html5/thumbnails/7.jpg)
Logic Models good for:
Linking activities to outcomes/impactsHelps to identify dependent activities. If linking site activities, are different outcomes
expected/desired from different sites?
Describing indicators needed to measure programs.Defining indicators is necessary for program evaluation;
Not always easy.
Learning about your programs.Examine why you do your activities.
![Page 8: Prioritization using Logic Models and MIRA October 17. 2007 Instituto Nacional de Ecologia Mexico City, Mexico](https://reader035.vdocument.in/reader035/viewer/2022062423/56649e145503460f94afe70d/html5/thumbnails/8.jpg)
What Logic Models are missing:
No indicator data contained in LMs.No way to prioritize program activities.
Use MIRA to get these…
![Page 9: Prioritization using Logic Models and MIRA October 17. 2007 Instituto Nacional de Ecologia Mexico City, Mexico](https://reader035.vdocument.in/reader035/viewer/2022062423/56649e145503460f94afe70d/html5/thumbnails/9.jpg)
Part II: MIRA
Analyzing Information for Decision Making: Prioritizing Environmental Outcomes and
Managing Risk
![Page 10: Prioritization using Logic Models and MIRA October 17. 2007 Instituto Nacional de Ecologia Mexico City, Mexico](https://reader035.vdocument.in/reader035/viewer/2022062423/56649e145503460f94afe70d/html5/thumbnails/10.jpg)
What’s involved in Decision analysis?
Criteria/DataScience – exposure, fate/transport models,
otherProgram implementation (logic models).Social science – environmental justice, different
demographic impacts.ValuesIntegrative, contextual approach for
decision analysis.
![Page 11: Prioritization using Logic Models and MIRA October 17. 2007 Instituto Nacional de Ecologia Mexico City, Mexico](https://reader035.vdocument.in/reader035/viewer/2022062423/56649e145503460f94afe70d/html5/thumbnails/11.jpg)
MIRA
Multi-criteria Integrated Resource Assessment
MIRA Approach:Multi-criteriaTransparentData driven; relative analysisIterative/learning-based
![Page 12: Prioritization using Logic Models and MIRA October 17. 2007 Instituto Nacional de Ecologia Mexico City, Mexico](https://reader035.vdocument.in/reader035/viewer/2022062423/56649e145503460f94afe70d/html5/thumbnails/12.jpg)
PRIMARY LEVEL SECONDARY LEVEL THIRD LEVEL FOURTH LEVEL
Area Wide
Data Fit Population weighted
Design Value weighted
Attn. Threshold weighted
Area Wide
Region III Data Scatter Population weighted
Design Value weighted
Attn. Threshold weighted
Worst Outlier
Area Wide
Data Fit Population weighted
Design Value weighted
Attn. Threshold weighted
Area Wide
1-Hr O3 Non-Attainment Areas Data Scatter Population weighted
Design Value weighted
Attn. Threshold weighted
Ozone Air Quality Worst Outlier
Area WideData Fit Population weighted
Design Value weighted
Attn. Threshold weighted
Area Wide
1-Hr O3 Attainment Areas Data Scatter Population weighted
Design Value weighted
Attn. Threshold weighted
Worst Outlier
Area Wide
Data Fit Population weighted
Design Value weighted
Attn. Threshold weighted
Area Wide
Class I Areas Data Scatter Population weighted
Design Value weighted
Attn. Threshold weighted
Worst Outlier
Personnel Impact Monitor Servicing Distance
Work Load
Costs
Trends Impact
Data Collection Manager
Geostatistical Indicators Module
Fate and Transport Models
Programmatic and Budget Decision Analysis Module
MIRA
![Page 13: Prioritization using Logic Models and MIRA October 17. 2007 Instituto Nacional de Ecologia Mexico City, Mexico](https://reader035.vdocument.in/reader035/viewer/2022062423/56649e145503460f94afe70d/html5/thumbnails/13.jpg)
What is MIRA designed to do?
Policy DevelopmentAssist in multi-criteria analyses for the
development/implementation of policy.
Understanding alternativesImprove understanding of the relationship
between the data and the decision alternatives.
Address stakeholder concernsProvide an analytical framework for reflecting
stakeholder ideas (Inclusive)
![Page 14: Prioritization using Logic Models and MIRA October 17. 2007 Instituto Nacional de Ecologia Mexico City, Mexico](https://reader035.vdocument.in/reader035/viewer/2022062423/56649e145503460f94afe70d/html5/thumbnails/14.jpg)
Steps in the MIRA approach
Determine the decision question.Brainstorm initial criteria.Gather data for those criteria.Construct the analytical hierarchy for the
decision question.Index data (expert input).Preference criteria (stakeholder value sets).Iterate; Learn.
![Page 15: Prioritization using Logic Models and MIRA October 17. 2007 Instituto Nacional de Ecologia Mexico City, Mexico](https://reader035.vdocument.in/reader035/viewer/2022062423/56649e145503460f94afe70d/html5/thumbnails/15.jpg)
Hazard Ranking System (HRS) ExamplePossible to use HRS score in different
ways with MIRA:Option 1: Use HRS as a decision criterion.Option 2: Use HRS criteria and allow for
flexibility for expert input and decision maker judgment.Appropriate when you don’t have or can’t get type of
data required by HRS; i.e., need to use surrogate indicators.
![Page 16: Prioritization using Logic Models and MIRA October 17. 2007 Instituto Nacional de Ecologia Mexico City, Mexico](https://reader035.vdocument.in/reader035/viewer/2022062423/56649e145503460f94afe70d/html5/thumbnails/16.jpg)
Option 1: HRS as Criterion
Suppose you want to evaluate both the condition of the region and program effectiveness within the region to include:Public health impactsEcological impactsBalance condition with program
(in)effectiveness.
Possible to set up a decision hierarchy something like this…
![Page 17: Prioritization using Logic Models and MIRA October 17. 2007 Instituto Nacional de Ecologia Mexico City, Mexico](https://reader035.vdocument.in/reader035/viewer/2022062423/56649e145503460f94afe70d/html5/thumbnails/17.jpg)
Condition
Program
Public Health
Ecosystem Health
Public Health
Ecosystem Health
Risk
Source
Admin.
Habitat Condition
Stressors
Risk
Source
Admin.
Habitat Condition
Stressors
Admin.
Stressors
Option 1: Sample MIRA Decision Hierarchy
HRS ?
![Page 18: Prioritization using Logic Models and MIRA October 17. 2007 Instituto Nacional de Ecologia Mexico City, Mexico](https://reader035.vdocument.in/reader035/viewer/2022062423/56649e145503460f94afe70d/html5/thumbnails/18.jpg)
Option 1: Indicator Examples
Condition HRS score Economic/social costs Ozone concentration, Nutrient load Cancer risk, Exposure
Program # permits/regulations approved; % impaired streams % regulations that include evaluation of alternative control
technologies. Amount of time between submittal and approval
of…regulation/permit/plan. “x” type of Hazardous Waste implementation plan
producing change/improvement in “y” type of risk parameter by “z” amount.
![Page 19: Prioritization using Logic Models and MIRA October 17. 2007 Instituto Nacional de Ecologia Mexico City, Mexico](https://reader035.vdocument.in/reader035/viewer/2022062423/56649e145503460f94afe70d/html5/thumbnails/19.jpg)
Option 1: How to use HRS with other criteriaNeed to consider the relative
environmental significance of HRS with other criteria.Expert discussionWhat does HRS indicate? Is it a more decision
significant indicator than economic cost (for example)?
If you believe no other criterion than HRS needs to be considered, you don’t need MIRA.
![Page 20: Prioritization using Logic Models and MIRA October 17. 2007 Instituto Nacional de Ecologia Mexico City, Mexico](https://reader035.vdocument.in/reader035/viewer/2022062423/56649e145503460f94afe70d/html5/thumbnails/20.jpg)
Option 2: Using HRS criteria as the analysis
Suppose you only want to consider hazardous waste criteria as currently used in calculating HRS…
OR: You are unable to get data required/ expected by HRS and must use surrogate indicators…
Possible to set up decision hierarchy as follows…
![Page 21: Prioritization using Logic Models and MIRA October 17. 2007 Instituto Nacional de Ecologia Mexico City, Mexico](https://reader035.vdocument.in/reader035/viewer/2022062423/56649e145503460f94afe70d/html5/thumbnails/21.jpg)
SGW
SSW
SS
SA
Likelihood of Release
Waste Characteristics
Targets
Observed Release
Potential of Release
Toxicity/ Mobility
Haz. Waste Quantity
Nearest Indiv.
Population
Resources
Sensitive Ecosystems
Cancer
NonCancer Chronic
NonCancer Acute
Source
Constituent
Waste StreamHRS
Option 2: MIRA Hierarchy for Hazard Ranking System (HRS)
. . .
. . .
. . .
![Page 22: Prioritization using Logic Models and MIRA October 17. 2007 Instituto Nacional de Ecologia Mexico City, Mexico](https://reader035.vdocument.in/reader035/viewer/2022062423/56649e145503460f94afe70d/html5/thumbnails/22.jpg)
HRS Calculation Example 1
HRS: Likelihood of Release = greater of observed release or potential to releaseTo replicate in MIRA: one of these criterion will have a
weight of zero in the calculation (Other = 1.0).MIRA alternative (if not regulatory): weight these
criteria in any way that adds up to 1.0 (or 100%).
![Page 23: Prioritization using Logic Models and MIRA October 17. 2007 Instituto Nacional de Ecologia Mexico City, Mexico](https://reader035.vdocument.in/reader035/viewer/2022062423/56649e145503460f94afe70d/html5/thumbnails/23.jpg)
HRS Calculation Example 2
HRS Calculation MethodologyPathway Score, S = (Likelihood of
Release x Waste Characteristics x Targets)/82,500Max values for LR = 550, Waste = 100,
Targets = 150.
![Page 24: Prioritization using Logic Models and MIRA October 17. 2007 Instituto Nacional de Ecologia Mexico City, Mexico](https://reader035.vdocument.in/reader035/viewer/2022062423/56649e145503460f94afe70d/html5/thumbnails/24.jpg)
Cont’d Example 2To replicate in MIRA:
Calculate relative weights for each of 3 factors.E.g. LR weight = (550/82,500)/(550/82,500 +
100/82,500 + 150/82,500) = 0.691(LR) x 0.691 x (waste) x 0.124 x (targets) x 0.185
(Fixed weights via HRS method)Likelihood of Release is designed to be the most
important criterion in the HRS calculation method (69% vs. 12% vs. 18%).
With MIRA, you can change weights if desired (and allowed by law).
![Page 25: Prioritization using Logic Models and MIRA October 17. 2007 Instituto Nacional de Ecologia Mexico City, Mexico](https://reader035.vdocument.in/reader035/viewer/2022062423/56649e145503460f94afe70d/html5/thumbnails/25.jpg)
HRS Calculation Example 3
HRS =
Max pathway score (S) = 100.HRS equation appears to weight all pathways
equally BUT actually weights the pathway score that is highest most heavily (due to squaring).
In MIRA: possible to replicate weights via above equation or use other weights.
4
S S S S 2A
2S
2SWGW
2
![Page 26: Prioritization using Logic Models and MIRA October 17. 2007 Instituto Nacional de Ecologia Mexico City, Mexico](https://reader035.vdocument.in/reader035/viewer/2022062423/56649e145503460f94afe70d/html5/thumbnails/26.jpg)
Option 2: HRS Component analysis with MIRAWhat’s different about using HRS criteria in MIRA vs.
just calculating HRS?Allows for transparency in seeing relative importance
(weights) of all the criteria composing the HRS.Possible to use additional criteria (economic/ social) if
desired.Possible to use surrogate criteria if data required by
HRS is not available. If law requires HRS method, using MIRA is not an
option.BUT could use MIRA to inform other decisions.
![Page 27: Prioritization using Logic Models and MIRA October 17. 2007 Instituto Nacional de Ecologia Mexico City, Mexico](https://reader035.vdocument.in/reader035/viewer/2022062423/56649e145503460f94afe70d/html5/thumbnails/27.jpg)
MIRA ApproachStep 1: Determine the decision question.Step 2: Brainstorm initial criteria.Step 3: Construct the analytical hierarchy for the
decision question.Step 4: Address missing data.Step 5: Decide on decision’s unit of measure.Step 6: Index data (expert input).Step 7: Preference criteria (stakeholder value sets).Step 8: Iterate; Learn.
![Page 28: Prioritization using Logic Models and MIRA October 17. 2007 Instituto Nacional de Ecologia Mexico City, Mexico](https://reader035.vdocument.in/reader035/viewer/2022062423/56649e145503460f94afe70d/html5/thumbnails/28.jpg)
Step 1: Formulating the Decision QuestionDecision makers/stakeholders formulate
the question that they want to answer and the criteria they think they need to answer it.
What are the problem set elements that you are analyzing/ranking?e.g. watersheds?, counties?, emission control
strategies?
![Page 29: Prioritization using Logic Models and MIRA October 17. 2007 Instituto Nacional de Ecologia Mexico City, Mexico](https://reader035.vdocument.in/reader035/viewer/2022062423/56649e145503460f94afe70d/html5/thumbnails/29.jpg)
Step 2: Brainstorm Initial Criteria
Are data available for these criteria?Are data available on the scale that you want?
States?, Counties?, watersheds?, stream segments? Other?
If not: Is another scale possible?Can surrogate data be used?
Should this be identified for future data collection?
![Page 30: Prioritization using Logic Models and MIRA October 17. 2007 Instituto Nacional de Ecologia Mexico City, Mexico](https://reader035.vdocument.in/reader035/viewer/2022062423/56649e145503460f94afe70d/html5/thumbnails/30.jpg)
Step 3a: Construct the Decision HierarchyProvides decision context.Forces stakeholders to assess whether
they agree on the decision question that they want to answer.
![Page 31: Prioritization using Logic Models and MIRA October 17. 2007 Instituto Nacional de Ecologia Mexico City, Mexico](https://reader035.vdocument.in/reader035/viewer/2022062423/56649e145503460f94afe70d/html5/thumbnails/31.jpg)
Step 3b: Methodological thinking for constructing the hierarchyShould each criterion currently organized
at each level of the hierarchy be directly comparable?E.g., Would you compare Arsenic in ground
water with Ozone air quality? OR would a better comparison be Water (with groundwater under it) with Air (with Ozone under it)?
![Page 32: Prioritization using Logic Models and MIRA October 17. 2007 Instituto Nacional de Ecologia Mexico City, Mexico](https://reader035.vdocument.in/reader035/viewer/2022062423/56649e145503460f94afe70d/html5/thumbnails/32.jpg)
Step 4: Determine which criteria have no/missing dataPossible alternatives to no data
Health impact data – pollutant concentration – source emissions – number of sources?
Data collected by volunteers/other organizations.Using similar data (from another program, etc.).
Possible alternatives to missing dataStatistical analyses – e.g., multivariate analyses Data collected by volunteers/other organizations.Modeling.
![Page 33: Prioritization using Logic Models and MIRA October 17. 2007 Instituto Nacional de Ecologia Mexico City, Mexico](https://reader035.vdocument.in/reader035/viewer/2022062423/56649e145503460f94afe70d/html5/thumbnails/33.jpg)
Note about previously constructed indicatorsWhat do these indicators indicate?Is this meaningful in your current
analysis?Can better indicators for your analysis be
constructed with currently available data?
![Page 34: Prioritization using Logic Models and MIRA October 17. 2007 Instituto Nacional de Ecologia Mexico City, Mexico](https://reader035.vdocument.in/reader035/viewer/2022062423/56649e145503460f94afe70d/html5/thumbnails/34.jpg)
Step 5: Deciding on the Decision’s unit of measureDepends on the decision question
What is the condition of the watersheds in the region?Degree of degradation
Which watersheds should be restored?Degree of restorability
![Page 35: Prioritization using Logic Models and MIRA October 17. 2007 Instituto Nacional de Ecologia Mexico City, Mexico](https://reader035.vdocument.in/reader035/viewer/2022062423/56649e145503460f94afe70d/html5/thumbnails/35.jpg)
Continued…
OR combine questions: Based on the condition of the watersheds and the
restorability of the watersheds, which should we restore?
Motivation to restore
![Page 36: Prioritization using Logic Models and MIRA October 17. 2007 Instituto Nacional de Ecologia Mexico City, Mexico](https://reader035.vdocument.in/reader035/viewer/2022062423/56649e145503460f94afe70d/html5/thumbnails/36.jpg)
Step 6a: Indexing the data
Convert all criteria metrics to the decision unit.Indexing = Relative comparison among the range of metric
values on a decision scale; = unit converter (converts units of each criterion metric to the decision unit).
Expert Input hereWhat is the decision significance of the indicator values?Same indicator can have different signficance for another
decision question.
![Page 37: Prioritization using Logic Models and MIRA October 17. 2007 Instituto Nacional de Ecologia Mexico City, Mexico](https://reader035.vdocument.in/reader035/viewer/2022062423/56649e145503460f94afe70d/html5/thumbnails/37.jpg)
Step 6b: Approach to Indexing the DataUse a decision scale of 1 to 8. Assumption: Each criterion is of equal value or
importance.BUT Metrics are not looked at independently.Task in indexing is to define what value of each
criterion elicits the same response. Set these values to the same index. E.g., $1 million is a lot of money and 95 ppb of ozone is
a high ozone level (on par with $1 mil) (they both elicit a “that’s a lot” response), so set them both to the same index.
![Page 38: Prioritization using Logic Models and MIRA October 17. 2007 Instituto Nacional de Ecologia Mexico City, Mexico](https://reader035.vdocument.in/reader035/viewer/2022062423/56649e145503460f94afe70d/html5/thumbnails/38.jpg)
Step 6c: Thinking about Indexing
Range of metric/indicator values? Type of distribution?
Double check: Compare values for criteria pairs – same significance?
Initialize; Change later if needed.
![Page 39: Prioritization using Logic Models and MIRA October 17. 2007 Instituto Nacional de Ecologia Mexico City, Mexico](https://reader035.vdocument.in/reader035/viewer/2022062423/56649e145503460f94afe70d/html5/thumbnails/39.jpg)
Step 7: Preferencing
All criteria are not equally important to the decision makers/stakeholders.Preferencing = Relative comparison of the
importance of one criterion to other criteria.
![Page 40: Prioritization using Logic Models and MIRA October 17. 2007 Instituto Nacional de Ecologia Mexico City, Mexico](https://reader035.vdocument.in/reader035/viewer/2022062423/56649e145503460f94afe70d/html5/thumbnails/40.jpg)
Step 7b: Thinking about PreferencingInitialize by setting all criteria preferences to
equal weights (i.e., all criteria equally important to the decision question within each level of the hierarchy). = Equal preference value set.
IterateTest different value setsExamine indexingExamine data – including quality assurance of data.
![Page 41: Prioritization using Logic Models and MIRA October 17. 2007 Instituto Nacional de Ecologia Mexico City, Mexico](https://reader035.vdocument.in/reader035/viewer/2022062423/56649e145503460f94afe70d/html5/thumbnails/41.jpg)
Step 8: Iteration
Test different value setsExamine indexingExamine dataExamine data uncertaintyRe-run analysis with different “what if”
scenarios.
![Page 42: Prioritization using Logic Models and MIRA October 17. 2007 Instituto Nacional de Ecologia Mexico City, Mexico](https://reader035.vdocument.in/reader035/viewer/2022062423/56649e145503460f94afe70d/html5/thumbnails/42.jpg)
Phosphorus Loading Raw Data P_Load Indexed
Sulfur Deposition Raw Data S_Dep Indexed
![Page 43: Prioritization using Logic Models and MIRA October 17. 2007 Instituto Nacional de Ecologia Mexico City, Mexico](https://reader035.vdocument.in/reader035/viewer/2022062423/56649e145503460f94afe70d/html5/thumbnails/43.jpg)
P_Load and S_Dep Combined
(equally important)
80% P_Load, 20% S_Dep
S_Dep hot spot (NW PA) determined to be more scientifically significant than P_Load hot spot (Delmarva Peninsula).
Science significance stays the same.
Decision maker judgments alter priorities but decision process is transparent.
![Page 44: Prioritization using Logic Models and MIRA October 17. 2007 Instituto Nacional de Ecologia Mexico City, Mexico](https://reader035.vdocument.in/reader035/viewer/2022062423/56649e145503460f94afe70d/html5/thumbnails/44.jpg)
Role of experts in MIRA
Experts in all fields of study to discuss issues:Indicator Types; construction of appropriate
indicators?Data* for indicators (existing, new)Missing data issues Scale of indicators/dataCombining public health and ecological informationIndexing data (determine relative significance of data)
![Page 45: Prioritization using Logic Models and MIRA October 17. 2007 Instituto Nacional de Ecologia Mexico City, Mexico](https://reader035.vdocument.in/reader035/viewer/2022062423/56649e145503460f94afe70d/html5/thumbnails/45.jpg)
Role of decision makers in MIRA
Learn the impact of different value sets (i.e., relative preference weights among decision criteria) on the decision options.Science remains constant.
Examine/compare the results of different value sets.
Make a decision after being informed about the impacts of all the options examined.Build decision confidence.Provide documentation and rationale for decision.
![Page 46: Prioritization using Logic Models and MIRA October 17. 2007 Instituto Nacional de Ecologia Mexico City, Mexico](https://reader035.vdocument.in/reader035/viewer/2022062423/56649e145503460f94afe70d/html5/thumbnails/46.jpg)
MIRA different from other decision support approaches…Hierarchy: represents decision questionIndexing: Expert input = relative decision
significance of the indicatorsPreferencing: Decision maker/stakeholder
judgments = relative importance of the decision criteria for this decision.
Relative contextual analysis.Illustrates what/where the tradeoffs are – as
constrained by the data. – Learning.
![Page 47: Prioritization using Logic Models and MIRA October 17. 2007 Instituto Nacional de Ecologia Mexico City, Mexico](https://reader035.vdocument.in/reader035/viewer/2022062423/56649e145503460f94afe70d/html5/thumbnails/47.jpg)
MIRA References
http://www.epa.gov/reg3artd/airquality/mira_descr.htmCimorelli, A. and Stahl, C. (2005), BSTS 25(3): 1, “Tackling
the Dilemma of the Science-Policy Interface in Environmental Policy Analysis.”
Stahl, C.H. (2003), “Multi-criteria Integrated Resource Assessment (MIRA): A New Decision Analytic Approach to Inform Environmental Policy Analysis.” For the Degree of Doctor of Philosophy, University of Delaware.
Stahl, C. H. and Cimorelli, A. J. (2005), Risk Analysis 25(5): 1109, “How Much Uncertainty is Too Much and How Do We Know? A Case Example of Ozone Monitor Network Options.”
![Page 48: Prioritization using Logic Models and MIRA October 17. 2007 Instituto Nacional de Ecologia Mexico City, Mexico](https://reader035.vdocument.in/reader035/viewer/2022062423/56649e145503460f94afe70d/html5/thumbnails/48.jpg)
Part III: Logic Model Outputs as MIRA Inputs
Program prioritization
![Page 49: Prioritization using Logic Models and MIRA October 17. 2007 Instituto Nacional de Ecologia Mexico City, Mexico](https://reader035.vdocument.in/reader035/viewer/2022062423/56649e145503460f94afe70d/html5/thumbnails/49.jpg)
What do we get with LMs and MIRA?Integration of Data and Program Activities.
Are we doing the right activities? – based on where the “worst” conditions are.
Which activities have the greatest effect on the outcomes/impacts we seek? – based on which outcomes/impacts we value most highly AND the condition data.
Which activities are dependent on which other activities?Capability to prioritize program outcomes using
data.TransparencyLearning
![Page 50: Prioritization using Logic Models and MIRA October 17. 2007 Instituto Nacional de Ecologia Mexico City, Mexico](https://reader035.vdocument.in/reader035/viewer/2022062423/56649e145503460f94afe70d/html5/thumbnails/50.jpg)
Example Logic Models
Air Quality Monitoring Logic ModelOzone Program Logic ModelTrace monitoring activity (certification of
ozone air quality data) through Monitoring Logic Model outputs/outcome/impacts See Red text in following figure.Follow black boxes within Monitoring logic
model in following figure.
![Page 51: Prioritization using Logic Models and MIRA October 17. 2007 Instituto Nacional de Ecologia Mexico City, Mexico](https://reader035.vdocument.in/reader035/viewer/2022062423/56649e145503460f94afe70d/html5/thumbnails/51.jpg)
Baseline Stressors Activities Outputs Outcomes ImpactsO3 SIP Program:
O3 SIP Program Baseline 1= 03 DV weighted by sensitive population (children, elderly, etc.)
O3 SIP Program Stressors
O3 SIP Program
Stressor 4 = # upwind areas designated attainment for O3.
O3 Nonattainment Area Designations
Based on O3 monitor design values, concur with HQ on O3 design monitor for each area.
O3 Nonattainment Area Designations
List of DVs for each NA area
O3 SIP Program
Outcome 2= 03 DV weighted by sensitive population (children, elderly, etc.)
MIRA Indicator
O3 SIP Program (Human Health) Impact 1 =
Human health impacts from O3 pollution
O3 Monitoring Program:
O3 Monitoring Baseline 1 = need a metric for the accuracy of the monitoring network
O3 Monitoring Program Stressors
O3 Monitoring program Stressor 1 = regulatory requirement to certify O3 data and calculate O3 DV.
Data Review for official O3 DV for O3 monitor:
Review states’ certification of O3 data (AQS data prior to official use).
Data Review for official O3 monitor DV Output:
Complete and certified O3 AQ data (no missing years, etc).
Official O3 monitoring data: DV
Data usable for AQ planning. A) selection of O3 design monitor for R3 areas.
Monitored O3 levels accurately represent true O3 levels for AQ planning areas. – Correlation coefficient between monitored and other estimation methods of O3?
![Page 52: Prioritization using Logic Models and MIRA October 17. 2007 Instituto Nacional de Ecologia Mexico City, Mexico](https://reader035.vdocument.in/reader035/viewer/2022062423/56649e145503460f94afe70d/html5/thumbnails/52.jpg)
Cont’d Ozone Logic Model
Show dependency of Ozone Program activity on Monitoring certification of data.See red underlined text in previous figure.
Trace Ozone program activity through to its outputs/outcomes/impacts.Follow red boxes in previous figure.
How does this connect to MIRA?
![Page 53: Prioritization using Logic Models and MIRA October 17. 2007 Instituto Nacional de Ecologia Mexico City, Mexico](https://reader035.vdocument.in/reader035/viewer/2022062423/56649e145503460f94afe70d/html5/thumbnails/53.jpg)
MIRA Indicator
Health indicator preferred but currently no data/science.
Use Ozone concentration weighted by population as surrogate for now.
![Page 54: Prioritization using Logic Models and MIRA October 17. 2007 Instituto Nacional de Ecologia Mexico City, Mexico](https://reader035.vdocument.in/reader035/viewer/2022062423/56649e145503460f94afe70d/html5/thumbnails/54.jpg)
SummaryLogic models improve program understanding.
Logic models provide connection between program activities and outcomes/impacts.
If prioritization is desired, use as MIRA input.
MIRA approach is compatible with use of many environmental, economic and social criteria.Use of HRS criteria possible in 2 different ways.Supports the use of surrogate data (using data that is
readily available).MIRA allows transparency, learning, stakeholder
inclusiveness.