session 8: risk/benefit assessment

8/10/2019 Session 8: Risk/Benefit Assessment

1/51


2/51

Defining risk and benefit

Assessing risks and expected benefitsCommunicating risks and expected benefitsOrganizing to reduce risks and increase

expected benefits

Overview


3/51

For the research in question, the expected

benefits are potentially reduced risks. As a result, the same methodologies apply

to assessing risks and expected benefits.

Risks and Expected Benefits


4/51



expected benefits

Overview


5/51


6/51

Defining Risk

The terms of all analyses embody values

that favor some interests.When transparent, those assumptions can

be controversial. An analytical-deliberative process is needed

to create socially acceptable definitions.


7/51

Defining Risk of Death

probability of premature death

vs.expected life-years lost


8/51

Defining Risk of Death

probability of premature death

vs.expected life-years lost

The choice of metric depends on whether adeath is a death or one values deaths ofyoung people more.


9/51

Other Possible Bases for

Distinguishing among Deaths Are the risks?

distributed equitablyassumed voluntarilycatastrophicwell understood

controllabledreadborne by future generations


10/51

Fischhoff, B., & Kadvany, J. (2011). Risk: A Very Short Introduction . Oxford: Oxford University Press.

Risk: A Very Short Introduction


11/51



expected benefits

Overview


12/51

12

Define outcomes in socially acceptable way.

Identify factors believed to affect outcomes. Assess factors and interdependencies, based

on observation and expert judgment.

Assess the quality of the evidence.

Risk (and expected benefit) Assessment


13/51

RoutineTestingResults

UtilityAwareness

HealthDepartmentAwareness

MediaCoverage

ConsumerAwareness forPublic Systems

Tap Test

MedicalAwareness

TriggerEvent

Well Test

ConsumerAwareness forPrivate Wells

UtilityCommunique

UtilityTreatment

Options

Consumption ofWell Water

Consumption ofTreated Water

AvertingBehavior for

Public

Systems

AvertingBehavior forPrivate Wells

SpecialStudies

Joint TaskForce

Heal th Effec tsontamination ofD r i n k i n g Wa t e r

Info Sources

MiscellaneousAnnouncement

Casman, E., Fischhoff, B., Palmgren, C., Small, M., & Wu, F. (2000). Integrated risk model

of a drinking waterborne Cryptosporidiosis outbreak. Risk Analysis, 20 , 493-509


14/51


15/51


16/51

Design:identify better options, by understandingcomplex systems better.

Decision:determine acceptability of risks (givenexpected benefits), by predicting outcomes.

Analysis Must Fit Its Purpose


17/51


18/51


19/51


20/51

We find that WASH-1400 was a conscientious and honesteffort to apply the methods of fault-tree/event-tree analysis to

an extremely complex system in order to determine theoverall probability and consequences of an accident

We have found a number of sources of both conservativismand nonconservatism in the probability calculations of

WASH-1400 Among the former are inability to quantifyhuman adaptability during the course of an accident ..., whileamong the latter are nagging issues about completeness, andan inadequate treatment of common cause failure.

We are unable to define whether the overall probability of acore melt given in WASH-1400 is high or low, but we arecertain that the error bands are understated. We cannot sayby how much. (p. vi)


21/51

Our resulting increased understanding of the full spectrum ofreactor accident sequences has implications for nuclearpower plant design, siting, and planning for mitigation ofconsequences. (p. ix)

There have been instances in which WASH-1400 has beenmisused as a vehicle to judge the acceptability of reactorrisks. In other cases, it may have been used prematurely asan estimate of the absolute risk of reactor accidents withoutfull realization of the wide band of uncertainties involved.Such use should be discouraged. (p. x)


22/51

OBrien, M. (2000). Making better environmental decisions ; Cambridge, MA: MIT Press


23/51

as sources of vulnerability and resilience

Analyses Must Include Human Behavior


24/51


25/51

London: George Routledge & Sons, 1921


26/51


27/51

http://www.nap.edu/catalog.php?record_id=18512


28/51

28

"$

Variability in observations

Internal validity (how good were studies)

External validity (how well do studies generalize)

Pedigree (how good is underlying science)

Analysis Must Recognize Limits


29/51

Representing Uncertainty

Campbell, P. (2011). Understanding the receivers and the receptions of sciences uncertainmessages. Philosophical Transactions of the Royal Society, 369 , 4891-4912.


30/51

Morgan, M.G. & Keith, D. (1995). Environmental Science and Technology, 2 9, 468-476.

Uncertain Climate Knowledge


31/51

Uncertain Economic Knowledge

Aikman, D,, Barrett, P., et al. (2011). Uncertainty in macroeconomic policy-making: art orscience. Philosophical Transactions of the Royal Society, 369 , 4798-4817.


32/51



expected benefits

Overview


33/51

Communication RequiresBehavioral Research


34/51

Communication RequiresBehavioral Research

Because our intuitions are often faulty.


35/51

Some Faulty Intuitions

common knowledge effectfalse consensus effectfundamental attribution errorself-serving biasesmyths (panic, adolescents unique sense

of invulnerability )


36/51

http://www.pnas.org/content/110/Supplement_3


37/51

http://www.pnas.org/content/111/Supplement_4


38/51

http://www.fda.gov/AboutFDA/ReportsManualsForms/Reports/ucm268078.htm

FDA Risk Communication Advisory Committee


39/51


40/51



expected benefits

Overview


41/51

Framework for Environmental Health Risk Management. Presidential/CongressionalCommission on Risk Assessment and Risk Management (I997).


42/51

Source: Canadian Standards Association. (1997). Risk Management (CSA-850). Ottawa: Author.


43/51


44/51

FDA. (2013). Structured approach to benefit-risk assessment for drug regulatorydecision making . Draft PDUFA V implementation plan (2/13). FY2013-2017.


45/51

Recognizes scientific and policy judgmentin all analyses

Quantifies the quantifiable, without ignoringother concerns

Highlights ethical and political tradeoffs,rather than burying them in a metric

Supports risk management

Benefit-Risk Framework


46/51

http://www.nap.edu/catalog.php?record_id=18870


47/51


48/51

FDA Risk CommunicationAdvisory Committee (RCAC)

http://www.fda.gov/oc/advisory/OCRCACACpg.html


49/51

RCAC Recommendations

Communication for Emerging EventsHave a consistent policy in all domainsProvide useful, timely information

Address: risks and benefits, uncertainty,personal actions, FDA actions

Audience needs should drive agencyanalyses

Use standard formats; evaluate routinelyConsider needs of diverse populations

http://www.fda.gov/oc/advisory/OCRCACACpg.html


50/51

socially acceptable, technically sounddefinitions of risk (and benefit)

strategic focus on design or decision, withproper disciplinary breadth andtreatment of uncertainty

ongoing, scientifically sound two-waycommunication with stakeholders

organization for transparency and learning

Summary

Risk/Benefit Analysis Requires


51/51

BooksFischhoff, B., Brewer, N., & Downs, J.S. (eds.). (2011). Communicating risks and benefits: An

evidence-based users guide . Washington, DC: Food and Drug Administration.http://www.fda.gov/AboutFDA/ReportsManualsForms/Reports/ucm268078.htm

Fischhoff, B., & Chauvin, C. (eds.). (2011). Intelligence analysis: Behavioral and social sciencefoundations . Washington, DC: National Academy Presshttp://www.nap.edu/catalog.php?record_id=13062

Fischhoff, B., & Kadvany, J. (2011). Risk: A very short introduction . Oxford: Oxford University Press.Fischhoff, B., Lichtenstein, S., Slovic, P., Derby, S. L. & Keeney, R. L. (1981). Acceptable risk . New

York: Cambridge University Press. (NUREG/CR-1614).Kahneman, D. (2011). Thinking, fast and slow . New York: Farrar Giroux & Strauss.Morgan, M.G., Henrion, M. (1990). Uncertainty . New York: Cambridge University Press.Slovic, P. (ed.) (2000). Perception of risk . London: Earthscan.

Research Articles Bruine de Bruin, W., Parker, A., & Fischhoff, B. (2007) Individual differences in adult decision-making

competence (A-DMC). Journal of Personality and Social Psychology. 92 , 938-956.Fischhoff, B. (1992). Giving advice: Decision theory perspectives on sexual assault. American

Psychologist , 47, 577-588.Fischhoff, B. (2011). Communicating the risks of terrorism (and anything else). American Psychologist,

66 , 520-531.Fischhoff, B. (2012, Summer). Communicating uncertainty: Fulfilling the duty to inform. Issues in

Science and Technology , 29 , 63-70 ,

Fischhoff, B., Bruine de Bruin, W., Guvenc, U., Caruso, D., & Brilliant, L. (2006). Analyzing disasterrisks and plans: An avian flu example. Journal of Risk and Uncertainty , 33, 133-151.

http://www.hss.cmu.edu/departments/sds/src/faculty/fischhoff.php Carnegie Mellon Electricity Center: http://wpweb2.tepper.cmu.edu/ceic/ Center for Climate and Environmental Decision Making: http://cedm.epp.cmu.edu/index.php Center for Risk Perception and Communication: http://sds.hss.cmu.edu/risk/ Center for Human Rights Science: http://www.cmu.edu/chrs/

session 8: risk/benefit assessment

Documents