evaluating online disaster preparedness training for ...€¦ · design training for adults to...

l lEvaluating online disaster preparedness training for family caregivers of senior citizens

Grant ChartrandLearning Design & TechnologyUniversity of Hawaii at ManoaUniversity of Hawaii at Manoa

Presentation OutlinePresentation Outline

• BackgroundBackground• Project Design

h d l• Methodology• Findings• Conclusion

PollPoll• Do you live with grandparents and/or elderly family members? 

• Do you live with family members who will soon become senior citizens?

PollPoll• Do you live with grandparents and/or elderly family members? 

• Do you live with family members who will soon become senior citizens?

Background: How did I select my topic?Background: How did I select my topic?

• Experience in creating coursesExperience in creating courses– Developed for government (state & county)

• Natural Hazard Preparedness for Caregivers of• Natural Hazard Preparedness for Caregivers of Senior Citizens

• Subject matter beneficial to all, especially for family members

• 3 generations lived together

Project PurposeProject PurposeDesign training for adults to increase theirawareness of the special disaster preparedness needs of elderly family members with whom they reside or providethey reside or provide care.

Background: Frequency of Hazards in U.S.Background: Frequency of Hazards in U.S.

300

250

ds H d T

256 251

150

200

es of H

azard

GeophysicalClimatological

Hazard Type

126

100

Instan

ce HydrologicalMeteorological

126

0

50

1940s 1950s 1960s 1970s 1980s 1990s 2000s

2040 40 38

1940s 1950s 1960s 1970s 1980s 1990s 2000s

Center for Research on the Epidemiology of Disasters (CRED)

Background: ProblemBackground: Problem

Disproportionate numberDisproportionate number of seniors died from Hurricanes Katrina andHurricanes Katrina and Sandy. Senior citizens are vulnerable Many deathsvulnerable. Many deaths were preventable.

Keller, J. (2012, November 17). Mapping Hurricane Sandy’s deadly toll. The New York Times. Retrieved from http://www.nytimes.com/interactive/2012/11/17/nyregion/hurricane‐sandy‐map.html.

Gibson, M. J., & Hayunga, M. (2006). We can do better: lessons learned for protecting older persons in disasters. Washington, D.C.: American Association of Retired Persons.

Background: U.S. Population GrowthBackground: U.S. Population Growth

109 1

9.5

8

ns) 6.8

7.78.5

9.1

6

ion (Billion

65+15‐64

Ages

2

4

Popu

lati 15 64

>15

0

2

2010 2020 2030 2040 20502010 2020 2030 2040 2050

Department of Economic and Social Affairs, United Nations

Background: Multigenerational FamiliesBackground: Multigenerational Families

In 2009, one in six Americans lived in homes with at least two adult generations.

Taylor, P., Kochhar, R., Cohn, D., Passel, J., Velasco, G., Motel, S., & Patten, E. (2011). Fighting Poverty in a Tough Economy, Americans Move in with Their Relatives. Pew Social & Demographic Trends.

Project Purpose: FocusProject Purpose: Focus

• Formal Caregiving vs Informal CaregivingFormal Caregiving vs. Informal Caregiving– Family members, friends

• Have little/no time for or access to training• Have little/no time for or access to training• Increase knowledge  Increase resilience

Project Design: TheoryProject Design: Theory

• Baldwin and Ford’s Transfer of Training theoryBaldwin and Ford s Transfer of Training theory– Knowledge gained from training is transferred to performanceperformance

• Keller’s ARCS Model of Motivational DesignPerceived utility/relevance increases motivation– Perceived utility/relevance increases motivation and acquisitionContent redesigned to build confidence– Content redesigned to build confidence

Baldwin, T. T., Ford, J. K., & Blume, B. D. (2009). Transfer of training 1988–2008: an updated review and agenda for future research. International review of industrial and organizational psychology, 24, 41‐70.

Grossman, R., & Salas, E. (2011). The transfer of training: what really matters. International Journal of Training and Development, 15(2), 103‐120.

Keller, J. M. (2010). Motivational design for learning and performance. New York, NY: Springer.

Project DesignProject Design• Course developed by the National Disasterthe National Disaster Preparedness Training Center (NDPTC) at the 

i i f iiUniversity of Hawaii at Manoa

• Articulate Storyline• Articulate Storyline (E‐Learning Authoring Software)

• Canvas (Learning Management System)

Project Design: Repackage & Redesign j g p g gof Instructional Content

Project Design: Repackage & Redesign j g p g gof Instructional Content

Project Design: Repackage & Redesign j g p g gof Instructional Content

NDPTC version (8 hr)  Project version (1‐2 hrs)

Project DesignProject Design

• Natural Hazard IdentificationNatural Hazard Identification

Project DesignProject Design

• Assessing CapabilitiesAssessing Capabilities

Project DesignProject Design

• Assessing NeedsAssessing Needs

Project DesignProject Design

• Risk and VulnerabilityRisk and Vulnerability

MethodologyMethodology

• 36 Participants (35 completed)p ( p )• Pre‐Survey

– Demographic and Confidence Level• Instructional Content• Assessment

M t hi M lti l Ch i d S i– Matching, Multiple Choice, and Scenario• Post‐Survey

– Confidence LevelConfidence Level– Ease of Use/Engagement/Quality/Satisfaction– Feedback 

Findings: DemographicsFindings: Demographics

• More than half (69%) were under 40 years of ageMore than half (69%) were under 40 years of age• About half (54%) did not live with or provide care for seniors or elderly family membersfor seniors or elderly family members

• An overwhelming majority (90%) had at least a 2 year degree2‐year degree

• All were employed: 80% full‐time, 20% part‐time

PollPoll

• I am confident in my knowledge of• I am confident in my knowledge of disaster preparedness for senior citizens.A. Strongly agreeB. AgreeC. Unsure/NeutralD DisagreeD. DisagreeE. Strongly disagree

PollPoll

• I am confident in my knowledge of• I am confident in my knowledge of disaster preparedness for senior citizens.A. Strongly agree (5)B. Agree (4)C. Unsure/Neutral (3)D Disagree (2)D. Disagree (2)E. Strongly disagree (1)

Findings: Average Confidence LevelFindings: Average Confidence Level5.00

3.29

4.34 4.23

3.00

4.00

evel

Pre‐2.83

2.00

nfiden

ce Le Pre

Survey

Post‐Survey

1.00

Con Survey

0.00

n=35General Knowledge Senior Citizen Knowledge

Findings: Overall Confidence Level by AgeFindings: Overall Confidence Level by Age5.00

4.29

3.64

4.27

3 00

4.00

Level

Ages

2.792.00

3.00

onfid

ence L <40

40+

g

1.00

Co

0.00

n=35Pre‐Survey Post‐Survey

Note: Overall = General Knowledge + Senior Citizen Knowledge

Findings: Overall Confidence Level by g yCaregiver Status

5.00

4.31 4.234.00

Level

CaregiverAges

2.943.16

2.00

3.00

onfid

ence L Caregiver

Non‐caregiver

1.00

Co

0.00

n=35Pre‐Survey Post‐Survey

Note: Caregiver=living with/providing care for 1 or more seniors, Non‐caregiver=0 (none)

Findings: Assessment ScoresFindings: Assessment Scores

1716

18 49%

17

12

14

16

nts

9

6

8

10

Participan 25%

1 1 1

4

22

4

6

6%11%

3% 3%3%1 1 1 2

06 7 8 9 10 11 12

Number of Correct Answers Out of 12 (%) n=35

Findings: Assessment ScoresFindings: Assessment Scores

1716

18 49%

17

12

14

16

nts

9

6

8

10

Participan 25%

1 1 1

4

22

4

6

6%11%

3% 3%3%1 1 1 2

06 7 8 9 10 11 12

Number of Correct Answers Out of 12 (%) n=35

Findings: Assessment ScoresFindings: Assessment Scores

1716

18 49%

17

12

14

16

nts

9

6

8

10

Participan 25%

1 1 1

4

22

4

6

6%11%

3% 3%3%1 1 1 2

06 7 8 9 10 11 12

Number of Correct Answers Out of 12 (%) n=35

Findings: Assessment ScoresFindings: Assessment Scores

1716

18 49%

17

12

14

16

nts

9

6

8

10

Participan 25%

1 1 1

4

22

4

6

6%11%

3% 3%3%1 1 1 2

06 7 8 9 10 11 12

Number of Correct Answers Out of 12 (%) n=35

Findings: Percent Answering CorrectlyFindings: Percent Answering Correctly125%

94% 97%91%

100%91%

100% 97% 94%89%

77%

100%

46% 46%50%

75%

Score

25%

0%Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Q10 Q11 Q12

Question Number n=35Q

Findings: Percent Answering CorrectlyFindings: Percent Answering Correctly125%

94% 97%91%

100%91%

100% 97% 94%89%

77%

100%

46% 46%50%

75%

Score

25%

0%Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Q10 Q11 Q12

Question Number n=35Q

Findings: Average Participant RatingFindings: Average Participant Rating

4 47

4.47

nstruct Ease of Use

E

4.41

4.47

ematic Con Engagement

Quality

4.27

The

Satisfaction

1.00 2.00 3.00 4.00 5.00

Rating Level n=35g n 35

Findings: Highly Ratedg g yThematic Construct n=35 RatingEase of Use 4.47

Module was easy to use 4.51odu e as easy o use 5Directions are easy to understand 4.54Questions are easy to complete 4.34

Engagement 4.47Length of module is manageable 4.46Use of website makes module more engaging 4.40Videos and images used are engaging 4.54

Quality 4 41Quality 4.41Information presented is useful for me 4.29Information presented is useful for others 4.40Information presented is appropriate 4.56Information presented is appropriate 4.56

Satisfaction 4.27Consider using online training again in the future 4.31Would recommend others use online training 4.23

Findings: Section FeedbackFindings: Section FeedbackSection n=35 Confusing UsefulNatural Hazards 2 15

Caregiver 1 8Non-caregiver 1 7

Assessing Capabilities 3 20Assessing Capabilities 3 20Caregiver 0 9Non-caregiver 3 11

Assessing Needs 3 24gCaregiver 1 10Non-caregiver 2 14

Assessing Risk and Vulnerability 9 29Caregiver 6 16Non-caregiver 3 13

Note: Open‐ended question, could respond to both Confusing and Useful

Findings: FeedbackFindings: Feedback

• Want more information on preparednessWant more information on preparedness organizations that they can contact

• Instructional content:• Instructional content:– More scenario questionsP id d fi i i f i i l /j– Provide definitions for unique terminology/jargon

– More examples of/more information on risk and l bilitvulnerability

• Feedback on test (assessment) questions

ConclusionsConclusions

• Module addressed time and access concernsModule addressed time and access concerns– Majority (29 of 35) reported that they took less than an hour to completethan an hour to complete

– Content made available online for informal caregiverscaregivers

• Positive response32 of 35 would recommend the module (3 unsure)– 32 of 35 would recommend the module (3 unsure)

ConclusionsConclusions

• Future iterationsFuture iterations– Track individual progressIncrease number of participants– Increase number of participants

– Increase diversity of participantsAdd dditi l t t t d b ti i t– Add additional content requested by participants

– Work with additional subject matter experts

Thank youThank you

• Critical friendsCritical friends• Colleagues and classmates

i d h h C f l• Dr. Irvine and the other LTEC faculty• Participants

http://ndptc.hawaii.edu

For More InformationFor More Information

• Ready.gov (FEMA) Guide • Red Cross Guideeady go ( ) u dehttp://1.usa.gov/1FMbTyI

ed oss u dehttp://rdcrss.org/1bBunA1

Questions?Questions?