designing risk communications (implications from comprehension tests) louis a. morris, ph.d. drug...
TRANSCRIPT
Designing Risk Communications (implications from
Comprehension Tests)
Louis A. Morris, Ph.D.
Drug Information Association
June 15, 2004
Objectives• Review Information Processing Models
– Patients (novices)– Physicians (experts)
• Describe Models for Risk Communications– Pharmacokinetics of Information Processing
• Discuss Learnings from Comprehension Tests– Cognitive Load Problems– Signaling Problems
• Discuss Role of Readability & Comprehension Testing
General Conclusions, Not Specific Findings
Information Processing
InformationProcessing/
MemoryStimuli/Messages
Perceptual Buffer:
AttentionProcesses
DecisionsEnvironmental
Cues/Usage Needs
Implications
• What captures attention has best opportunity to be processed– Placement – top of document, beginning of
sentences– Graphics – Headers, bold, italics
• Weber’s Law
• Limited Capacity– We remember what is important --- depends on
how we will use data
Cognitive Processes• Decoding and interpreting words
– Lexical Processing
• Extracting meaning within and among sentences – Syntactical Processing
• Discerning overall theme– Discourse Processing
• Meaning is Constructed – Like a house is built– More resources needed to build foundation, less
available for other sections
Constructing a Schema
Tell Me What this Says?
• The black and white cow walked over to the purple esuoh and smiled broadly to give the namow a liap of milk that she made into a nollag of French vanilla ice cream to sell to the srotisiv.
Implications
• Simple words • Short Sentences
• However:– Short words may not be simple
• Concept is difficult and may need explanation
– Some large words help organization– Sing-Song sentences are a turn-off
Readability Formula
Types of Processing
• Top-Down – experts (physicians)– Start with Existing schema and modify
– More efficient – preferred mode of processing
– Information stresses differences
• Bottom-Up – novices (patients)– Provide building blocks, simplify information
– Reduce cognitive load, provide signals
– Provide “intellectual scaffold” (organization) for new information
Principle of “Cognitive Conservation”
• People have limited cognitive resources– Limited Capacity Working Memory
• We can think about 7 +/- 2 bits
– Distribute “Cognitive Resources"• Based on our goals
• What is the “Cognitive Load” – How much information to process– How difficult is the information to process
Interaction of Materials’ “load and structure” with Patient’s Information Processing Skills and Motivations
Cognitive Load
Signals Ability
Opportunity
Motivation InvolvementGoals
SituationalConstraints
Literacy Self-Efficacy
PatientDocument
Willingness to Process
Actual Processing
Pharmacokinetic Communications Model
Morris & Aiken
Ability
• Literacy– NALS: five literacy levels, 40% of US @ 1 or 2– Level 1:
• 25% immigrants• 33% elderly (65 yrs. or older)• 25% physical or mental problem
• Self-efficacy– If people do not expect to be able to perform, they
won’t try– Self-efficacy judgments may not be based on sufficient
information
Risk Communication Options• HCPs
– PI, Label Changes (black box), Dear Doctor letters, Advertisements (affirmative disclosure, separate campaign)
• Patients – PPI, Medication Guides, Informed Consent, Agreement– Educational Campaigns
• Public (PR)– FDA public announcements (talk papers, press
releases), website posting, advisory committee meetings
Vary in Format and Cognitive Load
Comprehension Tests
• Started with OTC Drugs– Advil/Nuprin – “LL” vs. “CS” Label– Nicotine Products– Common for Switches
• Applied to Medication Guides– Other risk management documents
• Applied to Physician Labels
General Procedure
• Recruit (n= 400 to 1,200)– Use Shopping Malls– Screen for at-risk population
• Disease characteristics
• Low Literacy (pronunciation tests)
• Design– One Cell Survey– Multi-Cell Comparisons
General Procedure (2)• Procedure
– Screening– Document Exposure – read as normally would
• Interviewer Leaves Room
• Questionnaire– Develop Communication Objectives– Funnel Approach
– Open ends– Specific Communication Objectives– Follow-up Questions
– Document usually present (may be taken away for initial open ends)
MGs vs. OTC Labels• Longer
– Cognitive constraints on information processing– Limited “take away,” “time to interpret”
• More difficult words/concepts– Needs explanation to understand consequences
• More Complex Directions – Application to variety of usage situations
• Risk Topics – warnings, contraindications, side effects list
• Associated Documents– Agreements, wallet cards, consent, audiovisual
Comprehension Test Findings
• Cognitive Load - Longer documents– Primary Points fully communicated (extensive
repetition) (over 90% correct)– Secondary Points poorly communicated (66%-75%
correct)– Evidence Participants adopt a “harm prevention bias ”
– consult physician in response to any issue, regardless on information in the material.
• Do not attempt to consult document to answer questions – Questions requiring multiple mental operations are very
difficult (more than three items to consider leads to very poor results)
Tradeoffs- Explanation vs. Length
Lessons from Comp. Tests (1)
• Simplification– Avoid Extensive Repetition –
• Use brief headers in “most important section”
– Avoid “Seductive Details”• “Interesting to know” information detracts from key
messages
– Use “Communications Objectives” to focus messages
– Cut out extra words
Lessons form Comp. Tests (2)
• Signals– Less likely to notice information in:
• second half of a long sentence,
• middle of a paragraph,
• not graphically emphasized
– Graphic emphasis works surprisingly well for OTC, not for MGs
• too much “background noise” – reduce number of sub-heads
– Simplifying language and concepts • very often necessary but often requires much background
Lessons from Comp. Tests (3)
• Context Matters– Vague words decrease comprehension
• “do not drink alcohol”• Health, sickness, etc., has unclear meaning
• Credibility and Persuasion Needed– To influence behavior, need to go beyond
comprehension– Rationale for advocated behaviors may be
needed
Is Readability Testing Enough?
• Can help simplify information– Does not address overall length and cognitive
burdens due to overall flow– Does not address simple words that are difficult
to interpret (red meat, healthy, sick)
• Question of Predictive Validity– Do readability tests accurately predict reading
level– Tests developed in late 1940’s
Conclusions
• MGs (and other documents) are here– FDA required for drugs with risk management problems.
• Affirmative Approach (ie, write your own)– Bring FDA own designed/tested
• Readability can help simplify, Comprehension Testing is defensible– Comprehension tests for liability defense
• Plan Testing Prior to Implementation– Changing document after use can imply weaknesses in
prior document