FineHearing™ Technology*

A Step Closer to Natural Hearing

FocusOn

Sound perception in normal hearingOur sense of hearing allows us to discern the loudness, pitch and timbre of a sound, as well as to localise it. What we perceive as loudness, is the amplitude of a sound. Pitch indicates the frequency of a sound. Timbre represents the complexity of the sound.

The mathematician David Hilbert demonstrated that a (band-pass) signal can be decomposed into a slowly varying envelope (i.e. amplitude modulation) and a high frequency carrier of constant amplitude referred to as the fine structure of the signal (Fig. 1) (Hilbert, 1912). To investigate the role that the envelope and fine structure play in the perception of sounds, Smith et al. (2002) broke down sounds into their envelope and fine structure. They then paired the fine structure of one sound with the envelope of a second, and vice versa, to create so-called auditory chimeras. These chimeras were presented to normal hearing subjects. Results showed that for the number of independent information channels available in cochlear implants today, the envelope is the main information carrier for (western) speech. The fine structure is the main information carrier for music and for sound localisation (via inter aural time differences). Consequently, it was also found that when the two features are inconsistent in the information they provide, then the fine structure determines from where the sound is heard, and the envelope determines what is heard.

Sound coding in normal hearingWhen sound waves reach the cochlea via the outer and middle ear, they propagate from the base of the basilar membrane toward the apex, eliciting maximal vibration for high-frequency sounds at the base of the cochlea. With decreasing frequency, the point of maximal oscillation moves apically so that low-frequency sounds elicit maximal vibration at the apex of the cochlea. Thus, by this mechanism, frequency is translated into place in the cochlea so that each location along the basilar membrane is characteristic for a certain frequency (Fig. 2). This is called the tonotopical principle of the cochlea, and is one of the two fundamental mechanisms for frequency coding in the cochlea.

Envelope Fine Structure Sound Signal

=x

x =

A sound can be broken down into its envelope and fine structure. The envelope is most important for speech. The fine structure is most important for music and sound localisation via inter aural time differences.

Fig. 1: A signal can be decomposed into its envelope and its fine structure. The envelope is the main information carrier for (western) speech and sound localisation via inter aural level differences. The fine structure is the main source of information for music and sound localisation via inter aural time differences.

The cochlea codes an incoming signal in place (tonotopically) and in time (phase locking). Low frequency sounds are coded in time and place, whereas high frequency sounds are only coded in place.

32

Transduction between mechanical oscillation of the basilar membrane and electrical activity in the auditory nerve is performed by the inner hair cells. Inner hair cells act as the sound receptors by picking up the basilar membrane oscillation and translating it into neural activity. The response of the inner hair cells increases with an increase in sound intensity.

Signal transduction in the hair cells is mediated by the stereocillia and their associated ion channels. When the stereocillia are bent, the cells depolarise or hyperpolarise, depending on the direction of the displacement of the basilar membrane. The inner hair cells detect movements at their specific place on the membrane. These cells release neurotransmitters at a specific phase of the sound waveform. This synchronisation between vibrations in the air, the basilar membrane, and the activity of neurons in the resonant region is called phase-locking. It is the second fundamental mechanism for frequency coding in the cochlea. It means that the temporal response pattern of the neurons shows sound frequency and thus reflects the fine structure of the sound signal (Fig. 3). The amplitude pattern of the neural response reflects the envelope of the signal.

The question whether the frequency of a sound is coded using the tonotopical principle (also referred to as place coding), or using phase-locking (also referred to as time coding), or both, depends on the frequency of the sound. Low frequencies are coded in both dimensions, i.e. time and place, as schematically represented in Fig. 4, meaning that the neural response is created at a certain place in the cochlea (apical, since the frequency is low) and that the temporal pattern reflects sound frequency and amplitude. Thus, the neural response codes both the envelope and the fine structure of the signal.

With increasing frequency, phase-locking vanishes (Fig. 3) so that high frequencies are only coded by place (Fig. 4). The temporal response pattern of the neurons no longer reflects sound frequency; the fine structure is no longer represented in the neural response. The neuronal response thus only reflects the envelope of the sound signal.

Sound coding in cochlear implants

Processing based on envelope onlyAs described above, the fine structure of a sound is critical for music perception and sound localisation. However, all CIS and n-of-m based coding strategies that have been used in cochlear implants for the last 15–20 years rely on envelope information (Wilson, 2000) and largely disregard the fine structure. These strategies mainly provide place coding (as schematically shown in Fig. 4) by presenting envelope information across all frequencies (Fig. 5). Thus, in contrast to normal hearing, with increasing frequency there is no transition from time and place coding to place coding only (Fig. 4).

In general, users of these coding strategies show good to very good speech perception in quiet, moderate speech perception in noise, poor to moderate music appreciation (Zeng, 2004), and – if implanted bilaterally – moderate sound localisation (in the frontal horizontal plane) (Nopp et al., 2004). Specifically, the transmission of tonal speech information such as prosodic contour, as well as music perception and appraisal is poor in CI users compared to normal hearing listeners (Wong and Wong, 2004; Fu et al., 2004; McDermott, 2004).

Thus, the principal performance characteristics of these coding strategies are in agreement with the results obtained by Smith et al. (2002). That is, envelope information is suitable for providing moderate to very good degrees of speech perception but is unsuitable for providing the average user a satisfactory degree of music appreciation and better sound localisation. In short, these strategies lack fine structure information for improved speech in noise, music appreciation and sound localisation.

spik

es/b

in

time [ms] time [ms] time [ms]0

010

10

5 15 20

20

30

40

50

60

70

0.3 kHz 1.0 kHz 10 kHz

00

10

10

5 15 20

20

30

40

50

60

00

10

5

10

5 15 20

20

30

35

25

15

Fig. 2: Each place along the basilar membrane corresponds to a particular frequency. This is called the tonotopical principle of the cochlea. This place code provides funda mental information about

Fig. 3: The neural response to low frequencies (e.g. 0.3 kHz, 1 kHz) is phase locked to the fine structure of the signal. This temporal code provides fundamental information about frequency. At high frequencies (e.g. 10 kHz), the neural response is not phase locked so that the temporal code does not provide information about frequency.

Fig. 4: This schematic representation demonstrates the dimensions of sound coding. Depending on sound frequency, normal hearing uses time and place coding. Until now, cochlear implants have (mainly) used place coding only. FineHearing goes beyond this by providing the time code in the low to mid frequencies.

CIS and n-of-m based coding strategies provide envelope information only. They allow good speech perception but lack fine structure coding for satisfactory music perception.

54

Experience with EAS suggests that performance with cochlear implants is significantly improved with fine structure coding in the low to mid frequencies.

MED-EL’s FineHearing technology is designed to better model normal hearing than purely envelope-based coding strategies. Similar to frequency coding in normal hearing, the Fine Structure Processing (FSP) strategy codes frequency both in time (via CSSS) and place (via virtual channels).

What we have learned from EAS™As discussed above, the fine structure of a sound is critical for music perception and sound localisation, and phase-locking is a natural mechanism critical for coding the fine structure. With the introduction of the I100 electronics platform and the OPUS speech processors, MED-EL has developed an entirely new concept in coding strategies designed to overcome the limitations of envelope-based coding strategies and to appropriately code the fine structure. This concept is based on MED-EL‘s experience with users of Electric-Acoustic Stimulation (EAS). The idea behind EAS is to treat individuals with a ski-slope type hearing loss by combining a cochlear implant and a hearing aid in the same ear. In EAS, users are implanted with a cochlear implant to stimulate the mid to high frequency range. The low frequencies are amplified with a hearing aid. In individuals with no or a mild low-frequency hearing loss, amplification may not be required at all. Thus, in contrast to regular cochlear implant users, these users have acoustic stimulation in the low frequencies (Fig. 5) and are provided with envelope and fine structure information.

Results in users of EAS demonstrate improved speech perception in noise and improved music appreciation (Kiefer et al., 2005). Therefore, EAS users perform better in exactly those conditions where standard cochlear implant users experience their most serious problems. These results are in line with Smith et al. (2002) in that they further demonstrate how essential the fine structure is for sound perception. They further suggest that cochlear implant performance could be enhanced by improving sound coding in the low frequencies.

FineHearing™ – processing based on envelope and fine structureFineHearing technology is based on the essential findings gained from EAS users, namely that by focusing on fine structure coding in the lower frequencies, speech perception in noise and music appreciation can be enhanced. FineHearing technology aims at improving both the temporal and the tonotopic coding of sounds in cochlear implants with particular emphasis on temporal coding in the low to mid frequencies. FineHearing technology thus better models normal hearing in that there is a transition from time and place coding to place coding only (Fig. 4).

In contrast to fixed-rate envelope-based coding strategies, FineHearing uses the timing of stimulation to code the temporal structure of the sound signal.Based on EAS, Channel-Specific Sampling Sequences (CSSS) (Zierhofer, 2001) are used in the low to mid frequencies (Fig. 5) in an attempt to produce better temporal coding through improved phase locking. CSSS are series of stimulation pulses which are triggered by zero-crossings in a channel‘s band-pass filter output (Fig. 6). Thus the instantaneous repetition rate of these sequences equals the instantaneous fine structure frequency of the signal in the frequency range represented on that channel. For example, at times where the fine structure frequency is lower, zero crossings occur more rarely in time and sequences will be created less frequently (Fig. 6). Similarly, at times where the fine structure frequency is higher, zero crossings occur more often in time and sequences will be created more frequently (Fig. 6). Using these sequences, better frequency information can thus be attained in the low to mid frequencies, where EAS indicates that fine structure information may be particularly helpful.

Envelope-basedprocessing

Envelope

Envelope

Envelope

Acoustic

low Frequency

Position

high

basalapical

EAS

FineHearing CSSS

Place coding is achieved by creating pitch percepts which are intermediate to the pitch percepts created by stimulating single electrodes in isolation using so-called virtual channels (Wilson et al., 1992). FineHearing technology uses band-pass filters with a bell-shaped frequency response allowing a smooth transition of stimulation from one electrode to the adjacent electrode. For example, as input frequency increases (Fig. 7, 1 ), stimulation amplitude on the more apical electrode will decrease (Fig. 7, 2 ) and stimulation amplitude on the more basal electrode will increase (Fig. 7, 3 ) thus producing increasing pitches (Fig. 7, 4 ).

Previous studies (McDermott and McKay, 1994) found that even when these channels are stimulated sequentially, the perceived pitch is intermediate to the single-electrode pitches. Although the originally published concept of virtual channels used parallel stimulation, virtual channels can be created using both parallel (parallel virtual channels) and sequential stimulation (sequential virtual channels).

MED-EL’s Fine Structure Processing (FSP) is the first strategy to apply FineHearing technology. In FSP, CSSS is typically used on the lower (i.e. apical) 2–3 channels, which means that depending on the band-pass filters arrangement, CSSS is used for frequencies up to 300–500 Hz (Fig. 8). These channels provide the time and place code for the fundamental frequency (pitch) as well as for the lower harmonics of the sound signal. For example, for a male voice, channel 1 will code the fundamental frequency, i.e. pitch, channel 2 will code the first harmonic, and channel 3 will code the second harmonic. On the remaining channels, place coding is achieved by using sequential virtual channels. These channels will thus provide predominantly place code for the higher harmonics of the sound signal. Similar to normal hearing, envelope modulations will code pitch frequency in this frequency range.

BP out

CSSS

Envelope

FrequencyPosition

Electrode

Stimulation

Pitch

highbasal

lowapical

4

3 2

1

Fig. 5: Cochlear implants today use envelope-based sound processing providing mainly place coding. EAS and FineHearing go beyond envelope-based processing by additionally providing the time code in the low to mid frequencies.

Fig. 6: In FineHearing, time coding is achieved using Channel-Specific Sampling Sequences (CSSS). CSSS are series of stimulation pulses which are triggered by zero-crossings in a channel‘s band-pass filter output.

Fig. 7: In FineHearing, place coding is achieved using virtual channels. FineHearing technology uses band-pass filters with a bell-shaped frequency response (schematically represented here using triangles) allowing a smooth transition of stimulation from one electrode to the adjacent electrode, thereby creating pitch percepts which are intermediate to the pitch percepts created by stimulating single electrodes in isolation.

76

ResultsResults with the FSP coding strategy have been very positive. A multi-centre study recently conducted in Germany included 45 adult PulsarCI100 who were switched-over from the CIS+ strategy to the FSP strategy. Speech and pitch perception testing, and a sound quality questionnaire were administered immediately after switch-over, and after three months of FSP use. A retrospective questionnaire comparing everyday performance with FSP and CIS+ was conducted after one month of FSP use.

The results of this study demonstrate a statistically significant improvement in vowel perception in noise (+4.8 %) as well as monosyllable perception in noise (+6.1 %) (Brill et al., 2007). Sentence understanding in noise was on average better with FSP, although the resulting improvement in speech reception threshold was not statistically significant. Results also demonstrate improved pitch perception, particularly in the low frequencies. Here, users could better perceive low-frequency tones with the FSP strategy than with the C40+ strategy. These results are in line with those from the questionnaires. Both questionnaires indicate a strong preference for the FSP strategy. Results from the retrospective questionnaire are shown in Fig. 9. Across all questions, after only one month of FSP use, more than 80 % of the users rated the FSP strategy being superior to the CIS+ strategy they used before.

These results correspond to other studies comparing FSP and CIS+. For the FSP strategy, better perception of prosody information was demonstrated by Harnisch et al. (2007), and better instrument identification and melody discrimination was shown by Mühler et al. (2007).

The results of the above-discussed studies are in line with other studies investigating the effect of FineHearing technology on pitch perception in more depth. Mitterbacher et al. (2005a) found better pitch discrimination with the FSP strategy when using synthetic signals such as sawtooth and triangle waves, particularly in subjects who were poor performers. On average, just noticeable differences in pitch were ten percentage points smaller with FSP than with CIS+ in these subjects.

For pure-tone signals, Mitterbacher et al. (2005b) found improved low-frequency pitch perception ith the FSP strategy than with the CIS+ strategy. Their results indicate that at least for the most apical channel in a cochlear implant, place coding is impeded by the fact that this electrode does not have a more apical neighbor producing lower pitch – a problem which is inherent to place coding via virtual channels in general. In contrast, temporal coding in FSP succeeds in producing distinct pitch percepts. For complex tones (wide-band signals), similar results for FSP and CIS+ were found across the whole frequency range tested. This suggests that providing additional temporal information in FSP does not compromise the transmission of spectral cues when compared to CIS+.

Similar results were found by Schatzer et al. (2006). With the FSP strategy, they found large improvements in pitch discrimination for low frequencies and a constant increase in pitch over a logarithmic frequency axis compared to CIS+. Subjects again demonstrated better pitch perception in the low frequencies for the FSP strategy.

These results show that the FSP strategy allows better pitch perception than a traditional envelope-based CIS-type strategy, at least for narrow-band signals like sinusoids (Mitterbacher et al., 2005b). The results also indicate that both time (via CSSS) and place (via virtual channels) coding complement each other in that if one of the two codes fails, e.g. the place code for the low frequencies in CIS+, then patients can effectively use the other code to extract pitch information. Further, as discussed above, the results suggest that providing additional temporal information in FSP does not compromise the transmission of spectral information.

Providing both the time and the place code should thus make frequency coding more robust in cochlear implants. The literature suggests that each code should have its own specific advantages and weaknesses. In a large body of literature, the time code was shown to be very reliable; however, it currently seems to be restricted to frequencies below 300–1000 Hz, depending on the subject (Wilson et al., 1997; Zeng, 2002; Mitterbacher, 2005a; Mitterbacher, 2005b). In contrast, the place code works across a wider frequency range (basically the complete cochlear region covered by the electrode), however, the number of intermediate pitches in a certain cochlear region should depend on subject-specific parameters, e.g. neural survival, and was shown to vary largely across subjects, and across the cochlea even within subjects (Donaldson and Kreft, 2005).

In summary, the results indicate that by combining time and place coding, FineHearing technology improves pitch perception with cochlear implants. Further, the combination of both modalities seems to allow the user to make the best use of both worlds without adverse interference between the two.

All in all, the results of these studies demonstrate that the FineHearing technology as realised in the FSP strategy meets its design goals – improved low-frequency sound coding allowing better speech perception, better pitch perception, and better sound quality

Summary

The new FSP strategy allows better pitch perception than the standard envelope-based CIS+ strategy. MED-EL‘s I100 electronics platform, featured in the PULSARCI100 and SONATATI100 cochlear implants and supported by the OPUS speech processors, implements FSP, providing cochlear implant users additional temporal acoustic information that has previously been omitted in purely envelope-based coding strategies.

0%

FSP is equal or better than CIS+

FSP is worse than CIS+

Q1 Q2 Q3 Q4 Q10 Q11 Q12 Q13 Q15Q14 Q16 Q17 Q18 Q19

20%

40%

60%

80%

100%

91.1

91.1

93.3

91.193

.3

93.9

86.790

.9

82.6

82.6

82.286

.4

84.8 90

.9

Fig. 8: The FSP strategy is the first strategy to apply FineHearing technology. CSSS is typically used on the lower (i.e. apical) 2-3 channels. Thus, these channels provide the time and place code for the fundamental frequency (pitch) as well as the lower harmonics of the sound signal. On the remaining channels, place coding is achieved by using sequential virtual channels. These channels provide predominantly place code for the higher harmonics of the sound signal. Similar to normal hearing, envelope modulations will code pitch frequency in this frequency range.

First results with the FSP strategy show better pitch perception than the standard envelope-based CIS strategy.

Fig. 9: Results of a retrospective questionnaire after one month of FSP use. Questions addressed everyday-live hearing experiences with FSP and CIS+.

References

Brill et al. (2007b) Fine structure processing in cochlear implants: Results from a prospective clinical study. 11th International Conference on Cochlear Implants in Children. Charlotte, NC.

Donaldson et al. (2005) Place pitch discrimination of single versus dual electrode stimuli by cochlear implant users. J Acoust Soc Am. 118: 623 626.

Fu et al. (2004) The role of spectral and temporal cues in voice gender discrimination by normal-hearing listeners and cochlear implant users. J Assoc Res Otolaryngol. 5: 253-60.

Harnisch et al. (2007b). Prosody information and speaker recognition in fine structure processing. 11th International Conference on Cochlear Implants in Children. Charlotte, NC.

Hilbert D. (1912). Grundzüge einer allgemeinen Theorie der linearen Integralgleichungen, Leipzig: Teubner.

Kiefer et al. (2005) Combined Electric and Acoustic Stimulation of the Auditory System: Results of a Clinical Study. Audiol Neurotol 2005;10:134-144.

McDermott et al. (1994). Pitch ranking with nonsimultaneous dual-electrode electrical stimulation of the cochlea. J Acoust Soc Am. 96: 155 162.

McDermott HJ. (2004). Music perception with cochlear implants: a review. Trends Amplif. 8:49-82.

Mitterbacher et al. (2005b) Pitch, fine structure and CSSS - Results from patient tests. British Cochlear Implant Group Academic Meeting. Birmingham, UK.

Mitterbacher et al. (2005a) Encoding fine time structure with channel specific sampling sequences. Conference on Implantable Auditory Prosthesis, Pacific Grove, CA.

Mühler et al. (2007) Discrimination of musical pitch by cochlear implant users. 11th International Conference on Cochlear Implants in Children. Charlotte, NC.

Nopp et al. (2004) Sound localization in bilateral users of MED-EL COMBI 40/40+ cochlear implants. Ear Hear. 25: 205-214.

Schatzer et al. (2006) Encoding fine time structure with CSSS: concept and first results. Wien Med Wochenschr. 156 (Suppl 119): 93 94.

Smith et al. (2002). Chimaeric sounds reveal dichotomies in auditory perception. Nature. 416: 87 90.

Wilson et al. (1997) Relationships between temporal patterns of nerve activity and pitch judgments for cochlear implant patients. Eighth Quarterly Progress Report. NIH Contract N01-DC-5-2103, Neural Prosthesis Program, National Institutes of Health, Bethesda, MD.

Wilson et al. (1992) Virtual channels interleaved sampling (VCIS) processor. First Quarterly Progress Report, NIH Contract N01-DC-2-2401, Neural Prosthesis Program, National Institutes of Health, Bethesda, MD.

Wilson BS. (2000). Strategies for representing speech information with cochlear implants. In: Niparko JK, Kirk KI, Mellon NK, et al (Eds). Cochlear Implants: Principles & Practices. Philadelphia: Lippincott Williams & Wilkins, 5 46.

Wong et al. (2004). Tone perception of Cantonese-speaking prelingually hearing-impaired children with cochlear implants. Otolaryngol Head Neck Surg. 130: 751-758.

Zeng FG. (2002) Temporal pitch in electric hearing. Hear Res. 174: 101 106.

Zeng FG. (2004). Trends in Cochlear Implants. Trends Amplif, 8(1): 1-34.

Zierhofer C. (2001). Electrical nerve stimulation based on channel-specific sequences. World Patent WO 01/13991 A1.

www.medel.com

2028

7 r2

.0

*For demonstration purposes only – Some of the applications, products, features and performance characteristics described in this brochure/article are not approved for use in the USA and are not offered for sale in interstate commerce or represented for use within the USA