digital audio signal processing...
Post on 08-Aug-2018
230 Views
Preview:
TRANSCRIPT
1
Speech & Audio Processing / Part-II
Digital Audio Signal Processing DASP
Marc Moonen Dept. E.E./ESAT-STADIUS, KU Leuven
marc.moonen@esat.kuleuven.be homes.esat.kuleuven.be/~moonen/
Digital Audio Signal Processing: Introduction Version 2017-2018 Lecture-1: Introduction 2 / 40
Overview
• Aims/scope • Case study: Hearing instruments • Overview • Prerequisites • Lectures/course material/literature • Exercise sessions/project • Exam
2
Digital Audio Signal Processing: Introduction Version 2017-2018 Lecture-1: Introduction 3 / 40
Aims/Scope
Aim is 2-fold :
• Speech & audio per se S & A industry in Belgium/Europe/…
• Develop basic signal processing tools/principles Spatial filter design Adaptive filter algorithms (e.g. filtered-X LMS,..) Kalman filters Time-frequency analysis/processing Etc. …which are also used in many other application fields
Digital Audio Signal Processing: Introduction Version 2017-2018 Lecture-1: Introduction 4 / 40
• Course covers a number of (‘typical’) DASP topics, e.g. • Noise reduction • Microphone array processing/beamforming • Dereverberation • Acoustic feedback cancellation • Active noise control
• All of these (*) appear in the design of modern digital hearing instruments
• Hence very relevant DASP case…
(*) and many more…: • Dynamic range compression • Auditory scene analysis • Etc.
Case Study: Hearing Instruments 1/20
= remarkable/spectacular
= remarkable/spectacular
3
Digital Audio Signal Processing: Introduction Version 2017-2018 Lecture-1: Introduction 5 / 40
Hearing • Outer ear/middle ear/inner ear • Tonotopy of inner ear: spatial arrangement of where sounds of
different frequency are processed
= Cochlea
Low-freq tone
High-freq tone
Neural activity for low-freq tone
Neural activitity for high-freq tone
© w
ww
.cm
.be
Case Study: Hearing Instruments 2/20
Digital Audio Signal Processing: Introduction Version 2017-2018 Lecture-1: Introduction 6 / 40
Case Study: Hearing Instruments 3/20
[Source: Lapperre]
Hearing loss types: • Conductive (~outer/middle ear) • Sensorineural (~inner ear) • Mixed
One in six adults (Europe) suffers from hearing loss …and still increasing Typical causes:
• Aging • Exposure to loud sounds • …
4
Digital Audio Signal Processing: Introduction Version 2017-2018 Lecture-1: Introduction 7 / 40
1921
20
07 (O
ticon
)
Case Study: Hearing Instruments 4/20 èHearing Aids (HAs) • Audio input/audio output (`microphone-processing-loudspeaker’) • ‘Amplifier’, but so much more than an amplifier!! • History:
• Horns/trumpets/… • `Desktop’ HAs (1900) • Wearable HAs (1930) • Digital HAs (1980)
• State-of-the-art: Digital HAs with.. • MHz’s clock speed • Millions of arithmetic operations/sec, … • Multiple microphones
Digital Audio Signal Processing: Introduction Version 2017-2018 Lecture-1: Introduction 8 / 40
Ale
ssan
dro
Volta
174
5-18
27
© C
ochl
ear L
td
Case Study: Hearing Instruments 5/20
Electrical stimulation for low frequency
Electrical stimulation for high frequency
èCochlear Implants (CIs)
• Audio input/electrode stimulation output • Preprocessing similar to HAs + stimulation strategy +
• History: • Volta’s experiment… • First implants (1960) • Commercial CIs (1970-1980) • Digital CIs (1980)
• State-of-the-art: • MHz’s clock speed, Mops/sec, … • Multiple microphones
èOther: Bone anchored HAs, middle ear implants, …
Intra-cochlear electrode
5
Digital Audio Signal Processing: Introduction Version 2017-2018 Lecture-1: Introduction 9 / 40
èCochlear Implants (CIs)
• External Processor Analog/digital-conversion Digital processing & filterbank Etc.. • Coil Inductive/magnetic coupling • Implant Electrode array
PS: Number of CI-implantees worldwide approx. 200.000 PS: 3 Main companies (Cochlear LtD, Med-El, Advanced Bionics)
© C
ochl
ear L
td
Case Study: Hearing Instruments 6/20
Digital Audio Signal Processing: Introduction Version 2017-2018 Lecture-1: Introduction 10 / 40
DASP Challenges in hearing instruments (next slides) • Dynamic range compression • Noise reduction • Dereverberation • Acoustic feedback cancellation • Active noise control • Etc.
Technology Challenges in hearing instruments • Small form factor (cfr. user acceptance) • Low power: 1…5mW (cfr. battery lifetime ≈ 1 week) • Low processing delay: 10msec (cfr. synchronization with lip
reading)
Case Study: Hearing Instruments 7/20
6
Digital Audio Signal Processing: Introduction Version 2017-2018 Lecture-1: Introduction 11 / 40
DASP Challenges: Dynamic range compression Dynamic range & audibility Normal hearing Hearing impaired subjects subjects
Case Study: Hearing Instruments 8/20
Level
100dB
0dB
Digital Audio Signal Processing: Introduction Version 2017-2018 Lecture-1: Introduction 12 / 40
DASP Challenges: Dynamic range compression Dynamic range & audibility Need `signal dependent amplification’
Case Study: Hearing Instruments 9/20
Level
100dB
0dB Input Level (dB)
Out
put L
evel
(dB
)
0dB 100dB
0dB
100dB
Design: multiband DRC, attack time, release time, …
7
Digital Audio Signal Processing: Introduction Version 2017-2018 Lecture-1: Introduction 13 / 40
Why multiband DRC? …requires analysis/synthesis filter bank
Case Study: Hearing Instruments 10/20
Digital Audio Signal Processing: Introduction Version 2017-2018 Lecture-1: Introduction 14 / 40
• However: Audibility does not imply intelligibility
• Hearing impaired subjects need 5..10dB larger signal-to-noise ratio (SNR) for speech understanding in noisy environments
• Need for noise reduction (=speech enhancement) algorithms: • State-of-the-art: monaural 2-microphone adaptive noise reduction • Near future: binaural noise reduction (see below) • Not-so-near future: multi-node noise reduction (see below)
Case Study: Hearing Instruments 11/20
SNR
20dB
0dB 30 50 70 90
Hearing loss (dB, 3-freq-average)
8
Digital Audio Signal Processing: Introduction Version 2017-2018 Lecture-1: Introduction 15 / 40
DASP Challenges: Noise reduction & beamforming Multimicrophone ‘beamforming’, typically with 2 microphones
e.g. ‘directional’ front microphone and ‘omnidirectional’ back microphone
Case Study: Hearing Instruments 12/20
“filter-and-sum” the
microphone signals
Digital Audio Signal Processing: Introduction Version 2017-2018 Lecture-1: Introduction 16 / 40
Binaural hearing based on binaural auditory cues • ITD (interaural time difference) • ILD (interaural level difference)
• Binaural cues (ITD: f < 1500Hz, ILD: f > 2000Hz) used for
• Sound localization • Noise reduction =`Binaural unmasking’ (‘cocktail party’ effect) 0-5dB
Case Study: Hearing Instruments 13/20
ITD
ILD signal
9
Digital Audio Signal Processing: Introduction Version 2017-2018 Lecture-1: Introduction 17 / 40
Binaural hearing aids • Two hearing aids (L&R) with wireless link & cooperation • Opportunities:
• More signals (e.g. 2*2 microphones) • Better sensor spacing (17cm i.o. 1cm)
• Constraints: power/bandwith/delay of wireless link • ..10kBit/s: coordinate program settings, parameters,… • ..300kBits/s: exchange 1 or more (compressed) audio signals
• Challenges: • Improved localization through cue preservation • Improved noise reduction + benefit from binaural unmasking • Signal selection/filtering, audio coding, synchronisation, …
Case Study: Hearing Instruments 14/20
Digital Audio Signal Processing: Introduction Version 2017-2018 Lecture-1: Introduction 18 / 40
Future: Multi-node noise reduction in sensor networks/Internet-of-Things
Case Study: Hearing Instruments 15/20
10
Digital Audio Signal Processing: Introduction Version 2017-2018 Lecture-1: Introduction 19 / 40
DASP Challenges: Dereverberation • Reverb = filtering effect (‘echo-ing’) of acoustic channel in
between speaker and microphone(s)
• Reverb has an impact on speech understanding
• Dereverberation = undo filtering by acoustic channel (e.g. ‘inverse filtering’)
Case Study: Hearing Instruments 18/20
Digital Audio Signal Processing: Introduction Version 2017-2018 Lecture-1: Introduction 20 / 40
DASP Challenges: Acoustic feedback cancellation • Problem statement: Loudspeaker signal is fed back into microphone,
then amplified and played back again • Closed loop system may become unstable (howling) • Similar to feedback problem in public address systems (for the
musicians amongst you)
Case Study: Hearing Instruments 16/20
Model
F
-
Similar to echo cancellation in GSM handsets, Skype,… but more difficult due to signal correlation
11
Digital Audio Signal Processing: Introduction Version 2017-2018 Lecture-1: Introduction 21 / 40
DASP Challenges: Active noise control (ANC) ANC (see p.27) to counteract noise leakage & occlusion effect, exploiting additional ‘internal’ reference microphone
Case Study: Hearing Instruments 17/20
© S
.Doc
lo/U
niv.
Old
enbu
rg
Digital Audio Signal Processing: Introduction Version 2017-2018 Lecture-1: Introduction 22 / 40
Case Study: Hearing Instruments 20/20
DASP Challenges: …piecing things together
12
Digital Audio Signal Processing: Introduction Version 2017-2018 Lecture-1: Introduction 23 / 40
Overview : Lecture-2
Noise Reduction-I Single-channel noise reduction
`microphone_signal[k] = speech[k] + noise[k]’
• Spectral subtraction methods (spectral filtering) • Iterative methods based on speech modeling (Wiener & Kalman Filters) Applications: smartphones, conferencing, hearing aids, …
Digital Audio Signal Processing: Introduction Version 2017-2018 Lecture-1: Introduction 24 / 40
Overview : Lecture-3
Noise Reduction-II Microphone Array Processing/Fixed Beamforming Referred to as ‘spatial filtering’ (similar to ‘spectral filtering’)
Applications: smartphones, conferencing, hearing aids, …
),(1 θωY
),(2 θωY
),(1 θωY
)(1 ωF
)(2 ωF
)(ωmF),( θωmY
)(ωMF),( θωMY
md),(1 θωY
)(ωS
θ
Σ),( θωZ
θcosmd
Filter-and-sum beamformer
13
Digital Audio Signal Processing: Introduction Version 2017-2018 Lecture-1: Introduction 25 / 40
Overview : Lecture-4
Noise Reduction-III Adaptive Beamforming/Multi-channel noise reduction
• Adaptive Beamforming
– Known (fixed) speaker position – Unknown (time-varying) noise field
• Multi-channel noise reduction – Wiener filtering approach : spectral+spatial filtering
Applications: smartphones, conferencing, hearing aids, …
Digital Audio Signal Processing: Introduction Version 2017-2018 Lecture-1: Introduction 26 / 40
Overview : Lecture-5
Reverberation & Dereverberation ` microphone_signal[k] = filter*speech[k] (+ noise[k]) ’
• Reverb = effect of acoustic channel in between speaker and microphone(s)
• Reverb has an impact on coding, speech recognition, etc.
• Single-microphone de-reverberation – Cepstrum techniques
• Multi-microphone de-reverberation: – Estimation of acoustic impulse responses – Inverse-filtering method – Matched filtering
14
Digital Audio Signal Processing: Introduction Version 2017-2018 Lecture-1: Introduction 27 / 40
Acoustic Echo- and Feedback Cancellation Adaptive filtering problem: • Non-stationary/wideband/… speech signals • Non-stationary/long/… acoustic channels
Adaptive filtering algorithms AEC Control AEC Post-processing Stereo AEC
Overview : Lecture-6
Digital Audio Signal Processing: Introduction Version 2017-2018 Lecture-1: Introduction 28 / 40
Overview : Lecture-6 Acoustic Echo- en Feedback Cancellation • Hearing aids, public address (PA) systems • Correlation between filter input (`x ’) and near-end signal (‘ n ’) • Fixes : noise injection, pitch shifting, notch filtering, …
amplifier
15
Digital Audio Signal Processing: Introduction Version 2017-2018 Lecture-1: Introduction 29 / 40
Overview : Lecture-7
Active Noise Control • Solution based on `filtered-X LMS’ • Application : active headsets/ear defenders
Digital Audio Signal Processing: Introduction Version 2017-2018 Lecture-1: Introduction 30 / 40
Guest Lecture: Dr. Enzo De Sena, University of Surrey, UK
‘Sound Field Recording and Reproduction’
Overview : Lecture-8
16
Digital Audio Signal Processing: Introduction Version 2017-2018 Lecture-1: Introduction 31 / 40
Lectures
Lectures: 1 Intro (counts as half) + 7 Lectures PS: Time budget = 7.5*(2hrs)*4 = 60 hrs
Course Material: Slides
– Use version 2017-2018 ! – Download from DASP webpage
homes.esat.kuleuven.be/~dspuser/dasp/
Digital Audio Signal Processing: Introduction Version 2017-2018 Lecture-1: Introduction 32 / 40
Prerequisites
• Signals & Systems (JVDW) • Digital Signal Processing (I) (PW) signal transforms, sampling, multi-rate, DFT, …
• DSP-CIS (MM) filter design, filter banks, optimal & adaptive filters
17
Digital Audio Signal Processing: Introduction Version 2017-2018 Lecture-1: Introduction 33 / 40
Literature
Literature (general) (available in DSP-CIS library) • Simon Haykin `Adaptive Filter Theory’ (Prentice Hall 1996) • P.P. Vaidyanathan `Multirate Systems and Filter Banks’ (Prentice Hall 1993) Literature (specialized) (available in DSP-CIS library) • S.L. Gay & J. Benesty `Acoustic Signal Processing for Telecommunication’ (Kluwer 2000) • M. Kahrs & K. Brandenburg (Eds) `Applications of Digital Signal Processing to Audio and Acoustics’ (Kluwer1998) • B. Gold & N. Morgan `Speech and Audio Signal Processing’ (Wiley 2000)
Digital Audio Signal Processing: Introduction Version 2017-2018 Lecture-1: Introduction 34 / 40
Exercise Sessions/Project Acoustic source localization
– Direction-of-arrival estimation – Noise reduction – Synthesis – Simulated set-up
Direction-of-arrival θ
18
Digital Audio Signal Processing: Introduction Version 2017-2018 Lecture-1: Introduction 35 / 40
PS: groups of 2
Acoustic Source Localization Project
• Runs over 4 weeks (non-consecutive) • Each week
– 1 PC/Matlab session (supervised, 2.5hrs) – 2 ‘Homework’ sesions (unsupervised, 2*2.5hrs)
PS: Time budget = 4*(2.5hrs+5hrs) = 30 hrs • ‘Deliverables’ after week 2 & 4 • Grading: based on deliverables, evaluated during sessions
• TAs: guiliano.bernardi@esat (English+Italian)
randall.ali@esat (English+Spanish)
Digital Audio Signal Processing: Introduction Version 2017-2018 Lecture-1: Introduction 36 / 40
Work Plan – Week 1: Matlab acoustic simulation environment – Week 2: Direction-of-arrival (DoA) estimation based on the
‘MUSIC’ algorithm *deliverable*
– Week 3: DoA estimation + noise reduction (‘DOA informed beamforming’)
– Week 4: Binaural synthesis and 3D audio *deliverable*
Acoustic Source Localization Project
..be there !
19
Digital Audio Signal Processing: Introduction Version 2017-2018 Lecture-1: Introduction 37 / 40
• Oral exam, with preparation time • Open book • Grading
7 for question-1 7 for question-2 +6 for project ___ = 20
Exam
Digital Audio Signal Processing: Introduction Version 2017-2018 Lecture-1: Introduction 38 / 40
• Oral exam, with preparation time • Open book • Grading
7 for question-1 7 for question-2 +6 for question-3 (related to project work) ___ = 20
September Retake Exam
20
Digital Audio Signal Processing: Introduction Version 2017-2018 Lecture-1: Introduction 39 / 40
Website
1) TOLEDO 2) http://homes.esat.kuleuven.be/~dspuser/dasp/ • Contact: guiliano.bernardi@esat • Slides (use `version 2017-2018’ !!) • Schedule • DSP-library • FAQs (send questions to marc.moonen@esat)
Digital Audio Signal Processing: Introduction Version 2017-2018 Lecture-1: Introduction 40 / 40
Questions?
1) Ask teaching assistant (during exercises sessions)
2) E-mail questions to teaching assistant or marc.moonen@esat 3) Make appointment marc.moonen@esat ESAT Room B.00.14
top related