an evaluation of the efficacy of a remotely delivered ... evaluation of the... · hold promise for...

1Starkey Hearing Technologies’ An Evaluation of the Efficacy of a Remotely Delivered Auditory Training Program

BackgroundDespite impressive advances in hearing aid technology, after purchase, many users are either not fully satisfied, return the devices for credit, or take them off and “store them in the drawer.”

The percentage of hearing aid users that are less than satisfied with their devices has been reported to be as high as 26 percent in recent surveys (Kochkin, 2010).

One of the top reported reasons for this lack of satisfaction is a perceived lack of benefit, especially when attempting to carry on conversations in background noise (Kochkin, 2007). Listening in noise is challenging, both for those with and without hearing loss. Competing talkers and background noise can mask portions of the desired speech signal, making communication difficult. Though amplification combined with directional microphones and digital noise reduction may help, it is often not a satisfactory solution, and many hearing aid users, particularly those who are experiencing a loss of cognitive functioning in areas such as speed of processing, attention switching and working memory, are unable to communicate effectively regardless of the level of technology or quality of the fitting. An approach that appears to hold promise for improving communication function in noise and subsequent satisfaction with hearing aids is post-fitting rehabilitation services.

Aural rehabilitation (AR), specifically the use of auditory training, may be a way to bridge the gap between the technology in amplification and patient satisfaction. Auditory training includes specific listening and speech training designed to utilize any residual hearing the individual may have and to improve communication in challenging listening situations (Sweetow & Henderson Sabes, 2006). Post-fitting AR programs have been shown to be effective in improving patient performance and self-perceived benefit (Tye-Murray et al., 2012; Chisolm & Arnold, 2013).

Despite the potential benefit of AR, only a small percentage of clinicians offer these services to their patients (Schow, Balsara, Smedley & Whitcomb, 1993). The purported reasons for this are the lack of reimbursement, time or physical resources. Even in the absence of reimbursement, the provision of post-fitting AR can still be beneficial from both a financial and clinical perspective, as such services have been shown to decrease hearing aid return rates (Northern & Beyer, 1999).

One way to reduce the time burden and perceived financial burden associated with providing auditory training is by offering such programs online. As availability and comfort with technology increase, Internet-based AR programs are becoming increasingly accessible. The fact that such programs are available for at-home use doesn’t necessarily mean patients will use them. In order for a computerized program to be effective, it must be

Kirsten Bock, M.A., & Harvey Abrams, Ph.D.

AN EVALUATION OF THE EFFICACY OF A REMOTELY DELIVERED AUDITORY TRAINING PROGRAM

2 Starkey Hearing Technologies’ An Evaluation of the Efficacy of a Remotely Delivered Auditory Training Program

entertaining, engaging, interactive, accessible, and allow for self-paced progress. The content of the training must be at least as effective as that provided in an in-office environment.

ReadMyQuips™ (RMQ) is a computer-based auditory training program created by Dr. Harry Levitt that provides a focused approach to improving auditory-visual speech perception in the hopes of equipping the user with skills to better communicate in noise. The auditory-visual speech perception training is implemented through the use of games, puzzles and videos that advance with user skill improvement (Figures 1 & 2). Results of a pilot study by Levitt demonstrated improvement in sentence perception after completion of the RMQ program. The purpose of this study was to evaluate the efficacy of RMQ as a remotely delivered (through the Internet) auditory training program, as measured by improvements in speech-in-noise performance and self-perceived communication improvement among new hearing aid users. A secondary objective was to evaluate the usability and comfort with this Internet-based approach to AR among a typical hearing aid use population.

MethodsThirty new hearing aid users between the ages of 46 and 77 were recruited based on hearing loss, lack of experience with hearing aids and self-reported Internet usage. Participants were randomized into one of two groups: hearing aids alone (control) or hearing aids plus RMQ training. All participants were fit bilaterally with Wi Series® i110 receiver-in-canal (RIC) hearing aids. All fittings were prescribed to NAL-NL2 targets and verified with an Audioscan Verifit. At the time of fitting, all participants completed the Abbreviated Profile of Hearing Aid Benefit (APHAB) and were asked to reference their unaided experience (APHAB) (Cox & Alexander, 1995). One week post-fitting, participants returned for hearing aid fine-tuning, as well as to complete a pre-treatment Hearing in Noise Test (HINT) (Nilsson, Soli & Sullivan, 1994) and Words in Noise Test (WIN) (Wilson, Carnell & Cleghorn, 2007). The HINT is a standardized test of sentence reception in speech-shaped noise and was administered in three conditions: unaided, hearing aids set in an omnidirectional mode, and hearing aids set in a fixed directional microphone

Figure 2: The patient views the video clip with audio stimuli at varying levels of signal to noise in an attempt to solve the puzzle. One reward is solving the clever quip that is part of the activity.

Figure 1: An example of a ReadMyQuips™ puzzle. The patient selects a blank or partially blank line in an attempt to fill in the individual words after viewing a video.

1 2


mode. The WIN is a test of monosyllabic word reception in multitalker babble and was administered in two conditions: unaided and aided with a fixed directional microphone. The participants in the RMQ group were trained on how to access and “play” RMQ online through the MyStarkey website and instructed to complete the training on their home computers for 30 minutes a day, seven days a week, for three weeks. Each participant was given a practice log to record his or her use time. Following four weeks of hearing aid use, both groups completed post-treatment APHAB, HINT, WIN and the Device-Oriented Subjective Outcome scale (DOSO) (Cox, Alexander & Xu, 2009). The DOSO (Form B) is a 25-item questionnaire designed to measure hearing aid outcome in a way that is relatively independent of wearer personality. An example of a DOSO item is, “How good are your hearing aids at keeping background noise to a minimum?”

To assess the usability of training, the RMQ group completed the System Usability Scale (SUS), a 10-question, technology-independent questionnaire that provides a subjective assessment of usability (Sauro, 2011). The SUS is a widely administered measure of technology usability that has been used for applications such as hardware, consumer software, web pages and smartphones. Examples of SUS questions are, “I thought the system was easy to use” and “I needed to learn a lot of things before I could get going with this system.” We adapted the SUS to substitute “ReadMyQuips” for “this system” throughout the questionnaire.

ResultsOf the 30 participants recruited, 29 completed the study — 14 in the RMQ group and 15 in the control group. The mean audiograms for the participants in the RMQ and control groups are shown in Figures 3a and 3b, respectively. The mean age for the RMQ group was slightly older than that of the control

group (65.6 vs. 61.8 years) and the mean high-frequency thresholds (4,000, 6,000 and 8,000 Hz) were slightly worse in both ears for the RMQ group than for the control group, but none of these differences reached statistical significance.

Figure 3b: Mean thresholds for the participants randomized to the control hearing aid only group.

Figure 3a: Mean thresholds for the participants randomized to the RMQ.

50

60

70

80

90

100

30

40

10

20

-10

250 750500 15001000 2K 4K3K 8K6K

0

Hea

ring

Leve

l (dB

)

Frequency (Hz)

Mean Audiogram (RQM)

50

60

70

80

90

100

30

40

10

20

-10

250 750500 15001000 2K 4K3K 8K6K

0

Frequency (Hz)

Hea

ring

Leve

l (dB

)

Mean Audiogram (Ctrl)

3a

3b


Speech in Noise: Figures 4 and 5 illustrate the improvements in speech-in-noise performance on the HINT and WIN (respectively) following four weeks of hearing aid use for the RMQ group in light blue and the control group in dark blue as a function of listening condition. As can be seen, the mean improvement in the signal-to-noise ratio (SNR) required for 50 percent performance improved significantly for the RMQ group on the HINT in the static microphone condition and on the WIN in both the aided and unaided conditions. There were no statistically significant improvements on either speech-in-noise test for any condition for the control group.

Subjective Benefit: Figures 6 and 7 illustrate the APHAB and DOSO results, respectively, following four weeks of hearing aid use. Both the RMQ group (light blue) and the control group (dark blue) participants reported benefit scores of 20 to 30 percent as a function of the APHAB subscale, but there were no differences between the groups. Likewise, participants in both groups reported moderate to high levels of satisfaction with their devices on the DOSO, but again there were no between-group differences.

System Usability Scale: We were particularly interested in evaluating how comfortable our participants were accessing and using the RMQ program. Figure 8 illustrates the SUS score for each of the 14 participants in the RMQ group. The orange line indicates the 68th percentile, representing the average score for the SUS across 500 studies and approximately 5,000 individuals. Eleven of 14 participants in our study exceeded the 68th percentile, suggesting that their perceived usability of RMQ was above average.

Time on Task: Figure 9 illustrates the amount of self-reported total time, in minutes, spent engaging in the RMQ program for the 14 participants who completed the study. The orange line indicates the minimum number of minutes (630) that a participant would need to spend with RMQ if the protocol was strictly followed (i.e., 30 minutes per day for seven days per week for three weeks). Only four of the 14 participants met this minimum criterion as reported on their practice logs.

3a 3bFigure 3b: Mean thresholds for the participants randomized to the control hearing aid only group.

Figure 3a: Mean thresholds for the participants randomized to the RMQ.

50

60

70

80

90

100

30

40

10

20

-10

250 750500 15001000 2K 4K3K 8K6K

0

Hea

ring

Leve

l (dB

)

Frequency (Hz)

Mean Audiogram (RQM)

50

60

70

80

90

100

30

40

10

20

-10

250 750500 15001000 2K 4K3K 8K6K

0

Frequency (Hz)

Hea

ring

Leve

l (dB

)

Mean Audiogram (Ctrl)


4

86

5

9 7

Figure 4: Difference scores for the HINT from baseline to final visit.

Figure 5: Difference scores for the WIN test from baseline to the final visit.

Figure 6: APHAB difference scores from baseline to final visit. GBL, global; EC, ease of communication; RV, reverberant environments; BN, background noise; AV, aversiveness.

Figure 7: DOSO results. Maximum score is 7. SC, speech clarity; LE, listening effort; PL, pleasantness; QU, quietness; CON, convenience; USE, ease of use.

Figure 9: Self-reported time on task for 14 RMQ participants who completed the study. The orange line indicates the minimum number of minutes (630) required based on the study protocol.

Figure 8: System Usability Scale scores for 14 participants in the RMQ group. The orange line represents the 68th percentile — the historical average score based on 500 studies.

-3

-2.5

-2

-1.5

-1

-.5

0

.5

1Unaided

dB S

NR

Diff

eren

t

Omni Static

HINT Differences

*

*p<.05

RMQ CTRL

-3

-2.5

-2

-1.5

-1

-.5

0Unaided

dB S

NR-

50 Im

prov

emen

t

Aided

WIN Differences

*p<.05

RMQ CTRL

*

*

-60

-40

-20

0

20

40

60

80

100

GBL EC RV BN AV

Bene

fit S

core

APHAB Benefit

RMQ CTRL

0

1

2

3

4

5

6

7

8

SC LC PL QU CON USE

Ave

rage

Rat

ing

1-7

DOSO

RMQ CTRL

100

90

0

10

1 2 3 4 5 6 7 8 9 10 11 12 13 14

20

30

40

50

60

70

80

SUS

Scor

e

Participant

System Usability Scale

1000

1200

01 2 3 4 5 6 7 8 9 10 11 12 13 14

200

400

600

800

Tota

l Min

utes

Participant

Self-Reported RMQ Time


Data logging: Table 1 provides the average data logging information as a function of treatment group at the final visit. There were no between-group differences as a function of the percentage of time the hearing aids were in the directional, noise or speech-in-noise programs; however, the mean number of hours the RMQ participants wore their hearing aids was significantly higher than the control group participants. It should be noted that both groups wore their hearing aids for essentially the same number of hours following the first week of use (9.8 hours for the RMQ group and 9.9 hours for the control group).

DiscussionFollowing three weeks of engaging in an Internet-based auditory training program, the RMQ participants demonstrated significant improvements on the HINT and WIN tests, while the hearing aid-only group did not. As noted earlier, there were no differences between the groups on self-perceived benefits as measured by the APHAB and DOSO. The lack of statistically significant differences between the two groups on these measures could be related to several factors. First, there is a possibility that the items that constitute the APHAB and DOSO may not be sensitive to the effects of auditory training. A second possibility is that the benefits achieved by the hearing aids themselves resulted in a ceiling effect beyond which little additional benefit from auditory training, as measured by the APHAB and DOSO, would likely occur. A third possibility is that the RMQ participants were not given enough time to acclimatize to their new hearing aids before being asked to negotiate and engage in a new process. Finally, as suggested by their practice logs, a majority of the RMQ

participants did not spend the minimum number of hours engaged with RMQ as defined by the protocol. In terms of the usability of the RMQ, however, it was encouraging to see that most of the participants found it easy to use and felt confident accessing and using the program. Importantly, hearing aid wear time remained relatively stable at 9.6 hours for RMQ group but decreased from 9.9 hours to 8.4 hours in the control group.

ConclusionsIn a real-world application of remotely delivered AR, RMQ appears to hold the potential to improve performance in those environments in which patients experience their greatest difficulties and which threaten hearing aid success. While our patients will likely feel comfortable with and accept remotely delivered AR, compliance will continue to be a challenge requiring careful follow-up on the part of the provider.

Acknowledgments: We would like to thank Sharon Kaare and Laura Dundas for their assistance in recruiting and testing prospective participants, Ryan Irey and Edward Carney for their statistical support, and Michelle Hicks for her expertise during the design of the study.

For more information about RMQ and other AR programs available

through Starkey Hearing Technologies, contact your sales representative.

Table 1: Data logging results at final visit for average daily use and percentage of time hearing aid spent in three programs.

Daily Use % Directional % Noise % Speech in Noise

RMQ 9.6 hours 19.16% 1.4% 41.3%

CTRL 8.4 hours 17.6% 1.2% 41.2%


ReferencesChisolm, T. & Arnold, M. (2013). Evidence about the effectiveness of aural rehabilitation programs for adults. In L. Wong & L. Hickson (Eds.) Evidence-Based Practice in Audiology (pp. 237-266). San Diego, CA: Plural.

Cox, R.M. & Alexander, G.C. (1995). The Abbreviated Profile of Hearing Aid Benefit (APHAB). Ear and Hearing, 16, 176-186.

Cox, R.M., Alexander, G.C., & Xu, J. (2009). Development of the Device Oriented Subjective Outcome Scale (DOSO). Refereed poster presented at the Annual Meeting of the American Auditory Society, Scottsdale, Ariz., March 2009.

Kochkin, S. (2007). MarkeTrak VII: Obstacles to adult non-user adoption of hearing aids. The Hearing Journal, 60(4), 27-43.

Kochkin, S. (2010). MarkeTrak VIII: Customer satisfaction with hearing aids is slowly increasing. The Hearing Journal, 63(1), 11-19.

Nilsson, M., Soli, S.D., & Sullivan, J.A. (1994). Development of the Hearing In Noise Test for the measurement of speech reception thresholds in quiet and in noise. Journal of the Acoustical Society of America, 95(2), 1085-1099.

Northern, J. & Beyer, C. (1999). Reducing hearing aid returns through patient education. Audiology Today, 11(2), 10-11.

Sauro, J. (2011). Measuring Usability with the System Usability (SUS). Retrieved July 2013 from http://www.measuringusability.com/sus.php.

Schow, R.L., Balsara, N.R., Smedley, T.C., & Whitcomb, C.J. (1993). Aural rehabilitation by ASHA audiologists: 1980–1990. American Journal of Audiology, 2, 28–37.

Sweetow, R. & Henderson Sabes, J. (2006). The need for and development of an adaptive listening and communication enhancement (LACE) program. Journal of the American Academy of Audiology, 17, 538-558.

Tye-Murray, N., Sommers, M.S., Mauze, E., Schroy, C., Barcroft, J., & Spehar, B. (2012). Using patient perceptions of relative benefit and enjoyment to assess auditory training. Journal of the American Academy of Audiology, 23, 623–634.

Wilson, R.H., Carnell, C.S., & Cleghorn, A.L. (2007). The words-in-noise (WIN) test with multitalker babble and speech-spectrum noise maskers. Journal of the American Academy of Audiology, 18(6), 522-529.

an evaluation of the efficacy of a remotely delivered ... evaluation of the... · hold promise for...

Documents