implementing genomics in a direct-to-consumer world ... · • no specific genetics training beyond...
TRANSCRIPT
![Page 1: Implementing Genomics in a Direct-To-Consumer World ... · • No specific genetics training beyond medical school ... Satisfaction & usability Kirkpatrick 1994. Learning assessment](https://reader033.vdocument.in/reader033/viewer/2022050212/5f5e74ec8399af08fa4fc702/html5/thumbnails/1.jpg)
Implementing Genomics in a
Direct-To-Consumer World:
Opportunities for Education and
Communication
Emily Edelman, MS, CGC
Kate Reed, MPH, ScM, CGC
Clinical Education Program, The Jackson Laboratory
CDC Public Health Genomics Seminar Series
August 13, 2018
![Page 2: Implementing Genomics in a Direct-To-Consumer World ... · • No specific genetics training beyond medical school ... Satisfaction & usability Kirkpatrick 1994. Learning assessment](https://reader033.vdocument.in/reader033/viewer/2022050212/5f5e74ec8399af08fa4fc702/html5/thumbnails/2.jpg)
Disclosures
No conflicts to disclose
JAX Education does not aim to increase use of JAX (or
non-JAX) commercial products or services
JAX Clinical Education empowers healthcare
professionals to integrate genomics into clinical practice
through community collaboration
![Page 3: Implementing Genomics in a Direct-To-Consumer World ... · • No specific genetics training beyond medical school ... Satisfaction & usability Kirkpatrick 1994. Learning assessment](https://reader033.vdocument.in/reader033/viewer/2022050212/5f5e74ec8399af08fa4fc702/html5/thumbnails/3.jpg)
Plan for today
Discuss evidence-based approaches to designing and
developing adult education
Identify strategies to engage learners effectively
Discuss lessons learned in disseminating genomics
education
![Page 4: Implementing Genomics in a Direct-To-Consumer World ... · • No specific genetics training beyond medical school ... Satisfaction & usability Kirkpatrick 1994. Learning assessment](https://reader033.vdocument.in/reader033/viewer/2022050212/5f5e74ec8399af08fa4fc702/html5/thumbnails/4.jpg)
Direct-To-Consumer (DTC):
Greater access to health information
![Page 5: Implementing Genomics in a Direct-To-Consumer World ... · • No specific genetics training beyond medical school ... Satisfaction & usability Kirkpatrick 1994. Learning assessment](https://reader033.vdocument.in/reader033/viewer/2022050212/5f5e74ec8399af08fa4fc702/html5/thumbnails/5.jpg)
Clinical Scenario: Antonio
30 years old
Works in IT
Married, 2 children
Family history of cancer and heart
disease
Presents in clinic with DTC genetic testing
results for over 50 health conditions
“Can you help me understand this?”
![Page 6: Implementing Genomics in a Direct-To-Consumer World ... · • No specific genetics training beyond medical school ... Satisfaction & usability Kirkpatrick 1994. Learning assessment](https://reader033.vdocument.in/reader033/viewer/2022050212/5f5e74ec8399af08fa4fc702/html5/thumbnails/6.jpg)
What are the opportunities for
Antonio in having this information?
![Page 7: Implementing Genomics in a Direct-To-Consumer World ... · • No specific genetics training beyond medical school ... Satisfaction & usability Kirkpatrick 1994. Learning assessment](https://reader033.vdocument.in/reader033/viewer/2022050212/5f5e74ec8399af08fa4fc702/html5/thumbnails/7.jpg)
Patient opportunities and challenges
Increased
engagement
in health
Assessing
information
quality
Prioritizing
information
![Page 8: Implementing Genomics in a Direct-To-Consumer World ... · • No specific genetics training beyond medical school ... Satisfaction & usability Kirkpatrick 1994. Learning assessment](https://reader033.vdocument.in/reader033/viewer/2022050212/5f5e74ec8399af08fa4fc702/html5/thumbnails/8.jpg)
What are the provider’s greatest
challenges in this scenario?
![Page 9: Implementing Genomics in a Direct-To-Consumer World ... · • No specific genetics training beyond medical school ... Satisfaction & usability Kirkpatrick 1994. Learning assessment](https://reader033.vdocument.in/reader033/viewer/2022050212/5f5e74ec8399af08fa4fc702/html5/thumbnails/9.jpg)
Provider opportunities and challenges
Opportunity for
preventive
medicine
Patient questions
motivate learning
Competing priorities
Limited time
Lack of knowledge,
confidence, skills
![Page 10: Implementing Genomics in a Direct-To-Consumer World ... · • No specific genetics training beyond medical school ... Satisfaction & usability Kirkpatrick 1994. Learning assessment](https://reader033.vdocument.in/reader033/viewer/2022050212/5f5e74ec8399af08fa4fc702/html5/thumbnails/10.jpg)
Who has the greatest educational need?
![Page 11: Implementing Genomics in a Direct-To-Consumer World ... · • No specific genetics training beyond medical school ... Satisfaction & usability Kirkpatrick 1994. Learning assessment](https://reader033.vdocument.in/reader033/viewer/2022050212/5f5e74ec8399af08fa4fc702/html5/thumbnails/11.jpg)
There are plenty of needs to go around
![Page 12: Implementing Genomics in a Direct-To-Consumer World ... · • No specific genetics training beyond medical school ... Satisfaction & usability Kirkpatrick 1994. Learning assessment](https://reader033.vdocument.in/reader033/viewer/2022050212/5f5e74ec8399af08fa4fc702/html5/thumbnails/12.jpg)
Education can be part of the solution
Opportunity for preventive
medicine
Assess information quality
Questions motivates
learning
Knowledge, skills, attitudes
Limited time
Competing priorities
Engagement in health
Prioritize information
![Page 13: Implementing Genomics in a Direct-To-Consumer World ... · • No specific genetics training beyond medical school ... Satisfaction & usability Kirkpatrick 1994. Learning assessment](https://reader033.vdocument.in/reader033/viewer/2022050212/5f5e74ec8399af08fa4fc702/html5/thumbnails/13.jpg)
Education can be part of the solution
cont.
Education
Knowledge, skills, attitudes
Prioritize information
Assess information quality
System
Limited time
Competing priorities
ETC…
![Page 14: Implementing Genomics in a Direct-To-Consumer World ... · • No specific genetics training beyond medical school ... Satisfaction & usability Kirkpatrick 1994. Learning assessment](https://reader033.vdocument.in/reader033/viewer/2022050212/5f5e74ec8399af08fa4fc702/html5/thumbnails/14.jpg)
Maximize effectiveness of education
Determine appropriate goals
Use evidence-based approaches
Plan for dissemination
![Page 15: Implementing Genomics in a Direct-To-Consumer World ... · • No specific genetics training beyond medical school ... Satisfaction & usability Kirkpatrick 1994. Learning assessment](https://reader033.vdocument.in/reader033/viewer/2022050212/5f5e74ec8399af08fa4fc702/html5/thumbnails/15.jpg)
Analysis Content Delivery Evaluation Dissemination
![Page 16: Implementing Genomics in a Direct-To-Consumer World ... · • No specific genetics training beyond medical school ... Satisfaction & usability Kirkpatrick 1994. Learning assessment](https://reader033.vdocument.in/reader033/viewer/2022050212/5f5e74ec8399af08fa4fc702/html5/thumbnails/16.jpg)
Analysis: Needs Assessment
Who is this for?
What is the goal?
What gap does this fill?
![Page 17: Implementing Genomics in a Direct-To-Consumer World ... · • No specific genetics training beyond medical school ... Satisfaction & usability Kirkpatrick 1994. Learning assessment](https://reader033.vdocument.in/reader033/viewer/2022050212/5f5e74ec8399af08fa4fc702/html5/thumbnails/17.jpg)
Scenario: Develop an education program
for PCPs about DTC test results
“ We need help with patients
who come in with DTC
results.
We’d like you to provide an
education program to help
our providers.
”
![Page 18: Implementing Genomics in a Direct-To-Consumer World ... · • No specific genetics training beyond medical school ... Satisfaction & usability Kirkpatrick 1994. Learning assessment](https://reader033.vdocument.in/reader033/viewer/2022050212/5f5e74ec8399af08fa4fc702/html5/thumbnails/18.jpg)
There are likely a number of “problems”
that could be addressed
Awareness
Knowledge
Confidence
Systems issues
![Page 19: Implementing Genomics in a Direct-To-Consumer World ... · • No specific genetics training beyond medical school ... Satisfaction & usability Kirkpatrick 1994. Learning assessment](https://reader033.vdocument.in/reader033/viewer/2022050212/5f5e74ec8399af08fa4fc702/html5/thumbnails/19.jpg)
Define problem and overall goal
for the learner
Problem: Clinicians say they aren’t sure about which DTC
results are reliable and how to incorporate results into
medical care.
Goal: Provide appropriate follow-up based on DTC
results
![Page 20: Implementing Genomics in a Direct-To-Consumer World ... · • No specific genetics training beyond medical school ... Satisfaction & usability Kirkpatrick 1994. Learning assessment](https://reader033.vdocument.in/reader033/viewer/2022050212/5f5e74ec8399af08fa4fc702/html5/thumbnails/20.jpg)
Target education to audience’s
readiness to learn
awareness agreement adoption adherence
Pathman-PRECEED. Davis et
al. 2003 BMJ 327:33.
![Page 21: Implementing Genomics in a Direct-To-Consumer World ... · • No specific genetics training beyond medical school ... Satisfaction & usability Kirkpatrick 1994. Learning assessment](https://reader033.vdocument.in/reader033/viewer/2022050212/5f5e74ec8399af08fa4fc702/html5/thumbnails/21.jpg)
Target education to audience’s
readiness to learn cont.Digital channels
E-Newsletter
Social media
Webinars
Moderated forums
Patient education
EHR reminders
awareness agreement adoption adherence
Print materials
Lectures
Facilitated workshops
Consulting
Academic Detailing
Audit and Feedback
Physical channels Pathman-PRECEED. Davis et
al. 2003 BMJ 327:33.
![Page 22: Implementing Genomics in a Direct-To-Consumer World ... · • No specific genetics training beyond medical school ... Satisfaction & usability Kirkpatrick 1994. Learning assessment](https://reader033.vdocument.in/reader033/viewer/2022050212/5f5e74ec8399af08fa4fc702/html5/thumbnails/22.jpg)
Practice
Joyce
• Family medicine physician in suburban private practice
• No specific genetics training beyond medical school
• Refers ~5 pts/year to cancer genetics
• Two patients last month came in with DTC results
• Uncertain if her interpretation was correct
![Page 23: Implementing Genomics in a Direct-To-Consumer World ... · • No specific genetics training beyond medical school ... Satisfaction & usability Kirkpatrick 1994. Learning assessment](https://reader033.vdocument.in/reader033/viewer/2022050212/5f5e74ec8399af08fa4fc702/html5/thumbnails/23.jpg)
Q. Where is Joyce on the awareness-
adherence path?
awareness agreement adoption adherence
![Page 24: Implementing Genomics in a Direct-To-Consumer World ... · • No specific genetics training beyond medical school ... Satisfaction & usability Kirkpatrick 1994. Learning assessment](https://reader033.vdocument.in/reader033/viewer/2022050212/5f5e74ec8399af08fa4fc702/html5/thumbnails/24.jpg)
Q. What delivery mechanism would
you consider?
A. Social media campaign
B. Lecture
C. Workshop
D. EHR alerts
![Page 25: Implementing Genomics in a Direct-To-Consumer World ... · • No specific genetics training beyond medical school ... Satisfaction & usability Kirkpatrick 1994. Learning assessment](https://reader033.vdocument.in/reader033/viewer/2022050212/5f5e74ec8399af08fa4fc702/html5/thumbnails/25.jpg)
Analysis Considerations
Audience characteristics influence uptake
Educational goal is applicable to job
Remember that education is not always the solution
![Page 26: Implementing Genomics in a Direct-To-Consumer World ... · • No specific genetics training beyond medical school ... Satisfaction & usability Kirkpatrick 1994. Learning assessment](https://reader033.vdocument.in/reader033/viewer/2022050212/5f5e74ec8399af08fa4fc702/html5/thumbnails/26.jpg)
Content
What is most important to
include and prioritize?
![Page 27: Implementing Genomics in a Direct-To-Consumer World ... · • No specific genetics training beyond medical school ... Satisfaction & usability Kirkpatrick 1994. Learning assessment](https://reader033.vdocument.in/reader033/viewer/2022050212/5f5e74ec8399af08fa4fc702/html5/thumbnails/27.jpg)
Scenario: Improve identification of
patients for genetics referral
“ We need to improve our
identification of patients
who are at increased risk
of cancer and need to be
referred. We’d like you to
provide an education
program to help do this. ”
![Page 28: Implementing Genomics in a Direct-To-Consumer World ... · • No specific genetics training beyond medical school ... Satisfaction & usability Kirkpatrick 1994. Learning assessment](https://reader033.vdocument.in/reader033/viewer/2022050212/5f5e74ec8399af08fa4fc702/html5/thumbnails/28.jpg)
Q: What topics would you cover?
“ We need to improve our
identification of patients
who are at increased risk
of cancer and need to be
referred. We’d like you to
provide an education
program to help do this. ”
![Page 29: Implementing Genomics in a Direct-To-Consumer World ... · • No specific genetics training beyond medical school ... Satisfaction & usability Kirkpatrick 1994. Learning assessment](https://reader033.vdocument.in/reader033/viewer/2022050212/5f5e74ec8399af08fa4fc702/html5/thumbnails/29.jpg)
Break the overarching goal into
specific tasks
Identify patients at
increased cancer risk
based on family history
Collect a sufficient
family history
Assess family history
for cancer risk
Identify red flags
Identify patterns
Stratify risk
![Page 30: Implementing Genomics in a Direct-To-Consumer World ... · • No specific genetics training beyond medical school ... Satisfaction & usability Kirkpatrick 1994. Learning assessment](https://reader033.vdocument.in/reader033/viewer/2022050212/5f5e74ec8399af08fa4fc702/html5/thumbnails/30.jpg)
GRACE by JAXGenetics Risk Assessment Cancer Education
Collecting sufficient family history
Identifying red flags and patterns
Categorizing cancer risk
Using family history to inform management
Cancer pretest decisions and counseling
Genetic testing technology
Genetic testing process
Interpreting cancer genetic testing
results
Genetic testing for breast cancer
risk
Genetic testing for colorectal
cancer risk
Identifying and managing Lynch
syndromewww.jax.org/grace
![Page 31: Implementing Genomics in a Direct-To-Consumer World ... · • No specific genetics training beyond medical school ... Satisfaction & usability Kirkpatrick 1994. Learning assessment](https://reader033.vdocument.in/reader033/viewer/2022050212/5f5e74ec8399af08fa4fc702/html5/thumbnails/31.jpg)
Use learning objectives to focus
content
Collect a family history with sufficient detail
Analyze a family history to identify patients at
increased or high risk of cancer
o Identify red flags
o Identify patterns
o Stratify risk
![Page 32: Implementing Genomics in a Direct-To-Consumer World ... · • No specific genetics training beyond medical school ... Satisfaction & usability Kirkpatrick 1994. Learning assessment](https://reader033.vdocument.in/reader033/viewer/2022050212/5f5e74ec8399af08fa4fc702/html5/thumbnails/32.jpg)
Need to know vs. Nice to know
Actionable
Accurate, not complete
Vs.
Expert
Accurate, complete
![Page 33: Implementing Genomics in a Direct-To-Consumer World ... · • No specific genetics training beyond medical school ... Satisfaction & usability Kirkpatrick 1994. Learning assessment](https://reader033.vdocument.in/reader033/viewer/2022050212/5f5e74ec8399af08fa4fc702/html5/thumbnails/33.jpg)
Teach to the clinical tasks
![Page 34: Implementing Genomics in a Direct-To-Consumer World ... · • No specific genetics training beyond medical school ... Satisfaction & usability Kirkpatrick 1994. Learning assessment](https://reader033.vdocument.in/reader033/viewer/2022050212/5f5e74ec8399af08fa4fc702/html5/thumbnails/34.jpg)
Effective education is situated in a
relevant clinical context
![Page 35: Implementing Genomics in a Direct-To-Consumer World ... · • No specific genetics training beyond medical school ... Satisfaction & usability Kirkpatrick 1994. Learning assessment](https://reader033.vdocument.in/reader033/viewer/2022050212/5f5e74ec8399af08fa4fc702/html5/thumbnails/35.jpg)
Q. What is NEED to know content?
Learning objective
Collect a family history
with sufficient detail for
risk assessment in
primary care
A. Pedigree development
B. What questions to ask
C. What tools are available
D. Factors that increase risk
![Page 36: Implementing Genomics in a Direct-To-Consumer World ... · • No specific genetics training beyond medical school ... Satisfaction & usability Kirkpatrick 1994. Learning assessment](https://reader033.vdocument.in/reader033/viewer/2022050212/5f5e74ec8399af08fa4fc702/html5/thumbnails/36.jpg)
Content Considerations
Real-life tasks define what topics are needed
Focus on the need to know vs. nice to know
Provide relevant context
![Page 37: Implementing Genomics in a Direct-To-Consumer World ... · • No specific genetics training beyond medical school ... Satisfaction & usability Kirkpatrick 1994. Learning assessment](https://reader033.vdocument.in/reader033/viewer/2022050212/5f5e74ec8399af08fa4fc702/html5/thumbnails/37.jpg)
Delivery
What is the best way to present
my content?
What is the ideal delivery
mechanism?
or
How can I maximize the
effectiveness of a set format?
![Page 38: Implementing Genomics in a Direct-To-Consumer World ... · • No specific genetics training beyond medical school ... Satisfaction & usability Kirkpatrick 1994. Learning assessment](https://reader033.vdocument.in/reader033/viewer/2022050212/5f5e74ec8399af08fa4fc702/html5/thumbnails/38.jpg)
Delivery & Instruction
Instructional strategy
Method of teaching
educational content
Examples
o Case study
o Feedback
o Self-reflection
Delivery mechanism
Means of overall delivery
of education program
Examples
o Grand rounds
o Online course
o Video
![Page 39: Implementing Genomics in a Direct-To-Consumer World ... · • No specific genetics training beyond medical school ... Satisfaction & usability Kirkpatrick 1994. Learning assessment](https://reader033.vdocument.in/reader033/viewer/2022050212/5f5e74ec8399af08fa4fc702/html5/thumbnails/39.jpg)
Scenario: Implement family history best
practices
“ Family history screening for
colon cancer can save lives, but
it is not being done in a
systematic way in most clinics.
We need a resource to help
PCPs collect and interpret family
history and detect early on
colorectal cancer. ”
![Page 40: Implementing Genomics in a Direct-To-Consumer World ... · • No specific genetics training beyond medical school ... Satisfaction & usability Kirkpatrick 1994. Learning assessment](https://reader033.vdocument.in/reader033/viewer/2022050212/5f5e74ec8399af08fa4fc702/html5/thumbnails/40.jpg)
Toolkit Goal
Develop a system that helps primary care
practices:
Implement a structured family history
collection process
Identify and manage patients at
increased/high risk of CRC
Recognize and rapidly diagnose patients
with a presenting CRC
![Page 41: Implementing Genomics in a Direct-To-Consumer World ... · • No specific genetics training beyond medical school ... Satisfaction & usability Kirkpatrick 1994. Learning assessment](https://reader033.vdocument.in/reader033/viewer/2022050212/5f5e74ec8399af08fa4fc702/html5/thumbnails/41.jpg)
Educational goal should drive
instructional strategy
GoalImplement a structured family history collection process
Identify and manage patients at increased/high risk of CRC
StrategyStepwise instruction: Implementation & clinical skills
Tools and worksheets
Curated resources and education
![Page 42: Implementing Genomics in a Direct-To-Consumer World ... · • No specific genetics training beyond medical school ... Satisfaction & usability Kirkpatrick 1994. Learning assessment](https://reader033.vdocument.in/reader033/viewer/2022050212/5f5e74ec8399af08fa4fc702/html5/thumbnails/42.jpg)
Using different strategies reinforces
learning
• Standardized patient
• Case studies
• Audience response
• Small groups
![Page 43: Implementing Genomics in a Direct-To-Consumer World ... · • No specific genetics training beyond medical school ... Satisfaction & usability Kirkpatrick 1994. Learning assessment](https://reader033.vdocument.in/reader033/viewer/2022050212/5f5e74ec8399af08fa4fc702/html5/thumbnails/43.jpg)
Reinforce learning by touching
audience multiple times Month
1Genomics and Risk
Assessment
2Genetic Testing
Decision-Making
3Implementing
Genetic Testing
4Genetics of
Complex Disease
5Genomics of
Therapeutics
6Cancer Genomics
7Prenatal & Pediatric
Genomics
8Neurogenomics
9Cardiovascular
Genomics
1
0
Putting It All
Together
![Page 44: Implementing Genomics in a Direct-To-Consumer World ... · • No specific genetics training beyond medical school ... Satisfaction & usability Kirkpatrick 1994. Learning assessment](https://reader033.vdocument.in/reader033/viewer/2022050212/5f5e74ec8399af08fa4fc702/html5/thumbnails/44.jpg)
Q. What delivery would you use?
Goal
Recognize the benefits
of cancer family history
risk assessment.
A. Checklist
B. Online course
C. Newsletter
D. Social media
E. Workshop
![Page 45: Implementing Genomics in a Direct-To-Consumer World ... · • No specific genetics training beyond medical school ... Satisfaction & usability Kirkpatrick 1994. Learning assessment](https://reader033.vdocument.in/reader033/viewer/2022050212/5f5e74ec8399af08fa4fc702/html5/thumbnails/45.jpg)
Q. What delivery would you use?
Goal
Analyze a family
history to identify
patients at increased or
high risk of cancer
A. Checklist
B. Online course
C. Newsletter
D. Social media
E. Workshop
![Page 46: Implementing Genomics in a Direct-To-Consumer World ... · • No specific genetics training beyond medical school ... Satisfaction & usability Kirkpatrick 1994. Learning assessment](https://reader033.vdocument.in/reader033/viewer/2022050212/5f5e74ec8399af08fa4fc702/html5/thumbnails/46.jpg)
Match delivery to goal
Digital channels
awareness agreement adoption adherence
E-Newsletter
Social media
Print materials
Lectures
Facilitated workshops
Consulting
Webinars
Moderated forums
Academic Detailing
Patient education
EHR reminders
Audit and Feedback
Physical channels Pathman-PRECEED. Davis et
al. 2003 BMJ 327:33.
![Page 47: Implementing Genomics in a Direct-To-Consumer World ... · • No specific genetics training beyond medical school ... Satisfaction & usability Kirkpatrick 1994. Learning assessment](https://reader033.vdocument.in/reader033/viewer/2022050212/5f5e74ec8399af08fa4fc702/html5/thumbnails/47.jpg)
Delivery Considerations
Informed by educational goal
Multiple, sequenced exposures
Multiple modalities
![Page 48: Implementing Genomics in a Direct-To-Consumer World ... · • No specific genetics training beyond medical school ... Satisfaction & usability Kirkpatrick 1994. Learning assessment](https://reader033.vdocument.in/reader033/viewer/2022050212/5f5e74ec8399af08fa4fc702/html5/thumbnails/48.jpg)
Evaluation
What is my goal for evaluation?
What outcomes are most
important?
How do I assess them?
![Page 49: Implementing Genomics in a Direct-To-Consumer World ... · • No specific genetics training beyond medical school ... Satisfaction & usability Kirkpatrick 1994. Learning assessment](https://reader033.vdocument.in/reader033/viewer/2022050212/5f5e74ec8399af08fa4fc702/html5/thumbnails/49.jpg)
Scenario: Evaluate diverse program and
learners
“ We have enduring modules
that are taken by diverse
learners. We want to assess
educational outcomes as
well as the impact of the
program as a whole.
”
![Page 50: Implementing Genomics in a Direct-To-Consumer World ... · • No specific genetics training beyond medical school ... Satisfaction & usability Kirkpatrick 1994. Learning assessment](https://reader033.vdocument.in/reader033/viewer/2022050212/5f5e74ec8399af08fa4fc702/html5/thumbnails/50.jpg)
Precision Medicine for Your Practice
Topics
• Exome testing for diagnosis
• Cardiogenomics
• Pharmacogenomics
• Non-invasive prenatal screening
• Tumor genomics
• Neurogenomics
• Genomic testing in the healthy
![Page 51: Implementing Genomics in a Direct-To-Consumer World ... · • No specific genetics training beyond medical school ... Satisfaction & usability Kirkpatrick 1994. Learning assessment](https://reader033.vdocument.in/reader033/viewer/2022050212/5f5e74ec8399af08fa4fc702/html5/thumbnails/51.jpg)
Precision Medicine for Your Practice:
Evaluation PlanShort term
o Individual learner
o Assess learning gains, satisfaction
o Short pre/post surveys
Long term
o Practice or program (with clinical implementation?)
o Assess changes in knowledge, confidence, behaviors, patient
outcomes?
o Longitudinal surveys
![Page 52: Implementing Genomics in a Direct-To-Consumer World ... · • No specific genetics training beyond medical school ... Satisfaction & usability Kirkpatrick 1994. Learning assessment](https://reader033.vdocument.in/reader033/viewer/2022050212/5f5e74ec8399af08fa4fc702/html5/thumbnails/52.jpg)
Evaluate different levels of
outcomes
Results
Behavior
Learning
Reaction
Patient outcomes
Practice change
Knowledge, skills, attitudes
Satisfaction & usability
Kirkpatrick 1994
![Page 53: Implementing Genomics in a Direct-To-Consumer World ... · • No specific genetics training beyond medical school ... Satisfaction & usability Kirkpatrick 1994. Learning assessment](https://reader033.vdocument.in/reader033/viewer/2022050212/5f5e74ec8399af08fa4fc702/html5/thumbnails/53.jpg)
Learning assessment should match
education
Assess outcomes of learning objectives
Fidelity between instructional strategies and assessment items
Learning objective Instructional Strategy Assessment
Stratify cancer risk
to average,
increased, and high
risk
![Page 54: Implementing Genomics in a Direct-To-Consumer World ... · • No specific genetics training beyond medical school ... Satisfaction & usability Kirkpatrick 1994. Learning assessment](https://reader033.vdocument.in/reader033/viewer/2022050212/5f5e74ec8399af08fa4fc702/html5/thumbnails/54.jpg)
Consider feasibility of evaluation plan
Evaluation goal: Determine the impact of a blended learning program on
clinician knowledge, skills, attitudes, confidence, and practice longitudinally.
2014 2018
Freq. Every 3 mos. Every 6 mos.
# items Many Fewer
Item type Closed and
open ended
Closed ended
Incentive No Yes
![Page 55: Implementing Genomics in a Direct-To-Consumer World ... · • No specific genetics training beyond medical school ... Satisfaction & usability Kirkpatrick 1994. Learning assessment](https://reader033.vdocument.in/reader033/viewer/2022050212/5f5e74ec8399af08fa4fc702/html5/thumbnails/55.jpg)
Q. How would you assess?
Learning objective
Communicate with
patients about DTC
test results
Delivery
Small group role plays
with feedback
A. T/F items
B. Multiple choice items
C. Likert items
D. Standardized patient
![Page 56: Implementing Genomics in a Direct-To-Consumer World ... · • No specific genetics training beyond medical school ... Satisfaction & usability Kirkpatrick 1994. Learning assessment](https://reader033.vdocument.in/reader033/viewer/2022050212/5f5e74ec8399af08fa4fc702/html5/thumbnails/56.jpg)
Evaluation Considerations
Methods informed by education
Consider different kinds of outcomes
Balance evaluation goals with feasibility
![Page 57: Implementing Genomics in a Direct-To-Consumer World ... · • No specific genetics training beyond medical school ... Satisfaction & usability Kirkpatrick 1994. Learning assessment](https://reader033.vdocument.in/reader033/viewer/2022050212/5f5e74ec8399af08fa4fc702/html5/thumbnails/57.jpg)
Putting it all together
Analysis Content Delivery Evaluation
![Page 58: Implementing Genomics in a Direct-To-Consumer World ... · • No specific genetics training beyond medical school ... Satisfaction & usability Kirkpatrick 1994. Learning assessment](https://reader033.vdocument.in/reader033/viewer/2022050212/5f5e74ec8399af08fa4fc702/html5/thumbnails/58.jpg)
Putting it all together
Analysis Content Delivery Evaluation
Set parameters:
• Cancer genetics topic
• Online including video
Audience: PCPs, early-adopters
Goals: Assess risk, manage, and
communicate with patients about cancer
genetic risk
![Page 59: Implementing Genomics in a Direct-To-Consumer World ... · • No specific genetics training beyond medical school ... Satisfaction & usability Kirkpatrick 1994. Learning assessment](https://reader033.vdocument.in/reader033/viewer/2022050212/5f5e74ec8399af08fa4fc702/html5/thumbnails/59.jpg)
Putting it all together
Analysis Content Delivery Evaluation
• Tasks
• Learning objectives
• Prioritization
• Realistic context
![Page 60: Implementing Genomics in a Direct-To-Consumer World ... · • No specific genetics training beyond medical school ... Satisfaction & usability Kirkpatrick 1994. Learning assessment](https://reader033.vdocument.in/reader033/viewer/2022050212/5f5e74ec8399af08fa4fc702/html5/thumbnails/60.jpg)
Putting it all together
Analysis Content Delivery Evaluation
• Online course
• SHORT modules
• Multiple instructional
strategies
• Video
• Case studies
• Topic summaries
• Tools
![Page 61: Implementing Genomics in a Direct-To-Consumer World ... · • No specific genetics training beyond medical school ... Satisfaction & usability Kirkpatrick 1994. Learning assessment](https://reader033.vdocument.in/reader033/viewer/2022050212/5f5e74ec8399af08fa4fc702/html5/thumbnails/61.jpg)
Putting it all together
Analysis Content Delivery Evaluation
• Identified outcomes
• Identified assessment approach
• Program evaluation every 2 years
![Page 62: Implementing Genomics in a Direct-To-Consumer World ... · • No specific genetics training beyond medical school ... Satisfaction & usability Kirkpatrick 1994. Learning assessment](https://reader033.vdocument.in/reader033/viewer/2022050212/5f5e74ec8399af08fa4fc702/html5/thumbnails/62.jpg)
www.jax.org/grace
![Page 63: Implementing Genomics in a Direct-To-Consumer World ... · • No specific genetics training beyond medical school ... Satisfaction & usability Kirkpatrick 1994. Learning assessment](https://reader033.vdocument.in/reader033/viewer/2022050212/5f5e74ec8399af08fa4fc702/html5/thumbnails/63.jpg)
We built it; will they come?
Analysis Content Delivery EvaluationAnalysis Content Delivery Evaluation Dissemination
![Page 64: Implementing Genomics in a Direct-To-Consumer World ... · • No specific genetics training beyond medical school ... Satisfaction & usability Kirkpatrick 1994. Learning assessment](https://reader033.vdocument.in/reader033/viewer/2022050212/5f5e74ec8399af08fa4fc702/html5/thumbnails/64.jpg)
GOAL: Increase enrollment
![Page 65: Implementing Genomics in a Direct-To-Consumer World ... · • No specific genetics training beyond medical school ... Satisfaction & usability Kirkpatrick 1994. Learning assessment](https://reader033.vdocument.in/reader033/viewer/2022050212/5f5e74ec8399af08fa4fc702/html5/thumbnails/65.jpg)
There’s no recipe to follow
Experts don’t agree
Literature is only helpful so far
Landscape is changing rapidly
![Page 66: Implementing Genomics in a Direct-To-Consumer World ... · • No specific genetics training beyond medical school ... Satisfaction & usability Kirkpatrick 1994. Learning assessment](https://reader033.vdocument.in/reader033/viewer/2022050212/5f5e74ec8399af08fa4fc702/html5/thumbnails/66.jpg)
Assess resources
Email lists
Subject matter expertise
Familiarity with target audience
Extensive experience in developing education
![Page 67: Implementing Genomics in a Direct-To-Consumer World ... · • No specific genetics training beyond medical school ... Satisfaction & usability Kirkpatrick 1994. Learning assessment](https://reader033.vdocument.in/reader033/viewer/2022050212/5f5e74ec8399af08fa4fc702/html5/thumbnails/67.jpg)
Recognize challenges
Target audience awareness of JAX
Limited access to existing marketing infrastructure
Internal bandwidth and expertise
$$$
![Page 68: Implementing Genomics in a Direct-To-Consumer World ... · • No specific genetics training beyond medical school ... Satisfaction & usability Kirkpatrick 1994. Learning assessment](https://reader033.vdocument.in/reader033/viewer/2022050212/5f5e74ec8399af08fa4fc702/html5/thumbnails/68.jpg)
There are common processes
and approaches
Analyze/ Goal
Develop
Deliver
Evaluate
![Page 69: Implementing Genomics in a Direct-To-Consumer World ... · • No specific genetics training beyond medical school ... Satisfaction & usability Kirkpatrick 1994. Learning assessment](https://reader033.vdocument.in/reader033/viewer/2022050212/5f5e74ec8399af08fa4fc702/html5/thumbnails/69.jpg)
Develop plan
Increase clinician engagement
# exposed
# enrolled
![Page 70: Implementing Genomics in a Direct-To-Consumer World ... · • No specific genetics training beyond medical school ... Satisfaction & usability Kirkpatrick 1994. Learning assessment](https://reader033.vdocument.in/reader033/viewer/2022050212/5f5e74ec8399af08fa4fc702/html5/thumbnails/70.jpg)
Delivery must be feasible
✓ Email TV✓ Facebook
Radio
Etc…✓ In-person
✗ LinkedIn? Paid Ads ✗ Google+
? SEO Direct mail ?Boosted tweets
![Page 71: Implementing Genomics in a Direct-To-Consumer World ... · • No specific genetics training beyond medical school ... Satisfaction & usability Kirkpatrick 1994. Learning assessment](https://reader033.vdocument.in/reader033/viewer/2022050212/5f5e74ec8399af08fa4fc702/html5/thumbnails/71.jpg)
Define the problem the education will
help solve
![Page 72: Implementing Genomics in a Direct-To-Consumer World ... · • No specific genetics training beyond medical school ... Satisfaction & usability Kirkpatrick 1994. Learning assessment](https://reader033.vdocument.in/reader033/viewer/2022050212/5f5e74ec8399af08fa4fc702/html5/thumbnails/72.jpg)
Messaging
FREE
CME/CNE
Don’t miss at-risk patients
Can you identify who’s at-risk?
This state has one of the highest cancer rates
![Page 73: Implementing Genomics in a Direct-To-Consumer World ... · • No specific genetics training beyond medical school ... Satisfaction & usability Kirkpatrick 1994. Learning assessment](https://reader033.vdocument.in/reader033/viewer/2022050212/5f5e74ec8399af08fa4fc702/html5/thumbnails/73.jpg)
The approach that works is the one you
can implement
Available resources
Access to channels
Knowledge about the target audience
Expertise to evaluate impact
Ability to sustain efforts
![Page 74: Implementing Genomics in a Direct-To-Consumer World ... · • No specific genetics training beyond medical school ... Satisfaction & usability Kirkpatrick 1994. Learning assessment](https://reader033.vdocument.in/reader033/viewer/2022050212/5f5e74ec8399af08fa4fc702/html5/thumbnails/74.jpg)
Multimodal approach
Feb Mar Apr May June July
Email 1 1 1 1 2
Twitter posts 31 25 21 36 22 15
Facebook posts 11 5 7 9 8 7
Grand rounds ✓ ✓ ✓ ✓
Personal outreach ✓ ✓
• Started @JAXClinicalEd twitter feed Feb 2018
![Page 75: Implementing Genomics in a Direct-To-Consumer World ... · • No specific genetics training beyond medical school ... Satisfaction & usability Kirkpatrick 1994. Learning assessment](https://reader033.vdocument.in/reader033/viewer/2022050212/5f5e74ec8399af08fa4fc702/html5/thumbnails/75.jpg)
Status at 6 months
>10,000 PCPs
+>104 attendees
+144 followers
122 registrants
![Page 76: Implementing Genomics in a Direct-To-Consumer World ... · • No specific genetics training beyond medical school ... Satisfaction & usability Kirkpatrick 1994. Learning assessment](https://reader033.vdocument.in/reader033/viewer/2022050212/5f5e74ec8399af08fa4fc702/html5/thumbnails/76.jpg)
Find your audience
Know your audience
Identify who they trust
Determine needed assets
Personal outreach
![Page 77: Implementing Genomics in a Direct-To-Consumer World ... · • No specific genetics training beyond medical school ... Satisfaction & usability Kirkpatrick 1994. Learning assessment](https://reader033.vdocument.in/reader033/viewer/2022050212/5f5e74ec8399af08fa4fc702/html5/thumbnails/77.jpg)
Next steps
Delivery
o Outreach to professional organization local/regional
o Personal outreach
o Boosted tweets
o Blog
Value proposition/Content messaging
o Assess trends to maximize impact
Evaluation: Complex!
![Page 78: Implementing Genomics in a Direct-To-Consumer World ... · • No specific genetics training beyond medical school ... Satisfaction & usability Kirkpatrick 1994. Learning assessment](https://reader033.vdocument.in/reader033/viewer/2022050212/5f5e74ec8399af08fa4fc702/html5/thumbnails/78.jpg)
Dissemination Considerations
Define your goals
Assess your resources
Focus on the value proposition
Go to your audience
![Page 79: Implementing Genomics in a Direct-To-Consumer World ... · • No specific genetics training beyond medical school ... Satisfaction & usability Kirkpatrick 1994. Learning assessment](https://reader033.vdocument.in/reader033/viewer/2022050212/5f5e74ec8399af08fa4fc702/html5/thumbnails/79.jpg)
So…how can we best help Antonio?
“Can you help me understand this?”
![Page 80: Implementing Genomics in a Direct-To-Consumer World ... · • No specific genetics training beyond medical school ... Satisfaction & usability Kirkpatrick 1994. Learning assessment](https://reader033.vdocument.in/reader033/viewer/2022050212/5f5e74ec8399af08fa4fc702/html5/thumbnails/80.jpg)
Best practices still apply in a DTC world
Know your audience
Focus on the skill rather than knowledge gain alone
Separate need to know from nice to know
Provide learning in relevant context
Match delivery to educational goals
Match evaluation to desired outcome
Prioritize dissemination