methods for low bitrate coding enhancement part i ... · audio decoder car head unit (degraded) pcm...

© Fraunhofer IIS 1 08.09.2017

Methods for Low Bitrate Coding Enhancement Part I: Spectral Restoration

Patrick Gampp1, Christian Uhle1,2,Oliver Hellmuth1, Peter Prokein1,

Jürgen Herre2,1, Sascha Disch1,2, Julia Havenstein1, Antonios Karampourniotis1

1 Fraunhofer IIS, Erlangen, Germany2 International Audio Laboratories Erlangen, Germany


Methods for Low Bitrate Coding Enhancement Part I: Spectral Restoration

1. Introduction

2. System Overview

3. Bandwidth Extension

4. Birdie Suppression

5. Evaluation

6. Conclusion


1. IntroductionMotivation

Perceptual audio coders such as MP3 and AAC are used for efficient Storage Transmission

With good encoder and sufficiently high bitrates Uncoded and coded signals are perceptually indistinguishable!

Perceptual transparency At very low bitrates, sound quality is degraded Today`s car audio systems: Lots of possible audio sources with varying audio quality, e.g. Streaming, satellite radio Personal audio collection, etc.

High quality car audio systems unsparingly reveal coding artifacts


1. IntroductionAim

Post‐processing of decoded signal without using meta data Easy, versatile integration Can handle transcoding Repeated encoding of already encoded signals E.g. MP3 at 32 kbps is decoded and then encoded with

AAC at 256 kbps Only meta data of the last encoder will be shown!

Enhancement algorithms shall increase perceptual pleasantness In contrast to perceptual transparency

Remove of perceptual audio coding artifacts at low bitrates


2. System Overview

Audio Decoder

Car Head Unit

(Degraded)PCM Audio

SignalAudio Source Manager

(Enhanced) PCM Audio

Signal

)))

)))

)))

))))))

(Compressed) Audio

Bitstream

PCM Loudspeaker

Signal

Car Amplifier

Car Sound Processing

Low Bitrate Coding Enhancement Suite

Spectral Restoration

Spatial Enhancement

...

Audio Decoder

Car Head Unit

(Degraded)PCM Audio

SignalX’Audio

Source Manager

Low Bitrate Coding Enhancement Suite

Spectral Restoration

Spatial Enhancement

(Enhanced) PCM Audio

Signal

)))

)))

)))

))))))

(Compressed) Audio

Bitstream

PCM Loudspeaker

Signal

...

Car Amplifier

Car Sound Processing

Operation in the Head Unit

Operation in the Amplifier


3. Bandwidth ExtensionBandwidth Limitation

Occurs at low bitrates Static lowpass filtering: dull and muffled sound


3. Bandwidth ExtensionOverview

Processing frequency domain (STFT) Transient‐sustained decomposition for separate processing of

Transient path Sustained path

STFT

Delay LPF

Transient Suppression

Copy Up

Envelope Shaping Whitening ISTFT Anti‐Roughness

Filter

Copy Up

Envelope Shaping Whitening ISTFT Anti‐Roughness

FilterHPF

x[n] y[n]gt

gs

+

+

+


3. Bandwidth ExtensionWhy Transient‐Sustained Decomposition?

Transient signal portions in the uncoded signal usually have Higher bandwidth Higher correlation within one spectral frame

Advantage of different parameter tunings, e.g. Create higher bandwidth for transient path More anti‐roughness filtering on sustained path


3. Bandwidth ExtensionCopy Up and Spectral Envelope Shaping

Copy up of complex spectral coefficients Source patch: [fc, fc‐w] Destination patches: [fc+(k‐1)w, fc+kw] with k = 1,2,3,…

In order to avoid discontinuities between patches Attenuate destination patches by factor T Factor T is based on spectral tilt computed on source patch

|X|

ffcw w w

TT

TT


3. Bandwidth ExtensionSpectral Whitening

Reduce spectral variance Compute and apply spectral

weights Attenuate spectral peaks Raise spectral valleys

PG11

PG11 Andere Farbe für gainsPatrick Gampp, 9/7/2017


3. Bandwidth ExtensionAnti‐Roughness Filter

Aim: suppress roughness which is caused by copy up This is achieved by filtering of destination patches Two nested allpass filters in parallel with short impulse responses Mainly beneficial for sustained signal part. Therefore the transient‐sustained

decomposition is required

PG3

PG3 Bild von Allpässen?Impulsantwort?Patrick Gampp, 9/5/2017


4. Birdies SuppressionBirdies Artifact

Can occur at low bitrates Groups of spectral bins are very coarsely quantized to Very low spectral magnitudes (spectral gap) Very high spectral magnitudes (spectral island)

Over time spectral gaps and islands may toggle or can occur isolated Introduction of temporal modulation Artifact is reminiscent of a bird`s twitter

Separate processing of Spectral islands Spectral gaps


4. Birdies SuppressionExample for Islands and Gaps


4. Birdie SuppressionOverview

Processing in frequency domain (STFT) Signals are processed using spectral weighting Scaling of spectral coefficients with real‐valued weights

XSTFTx[n] ISTFT y[n]

Spectral Gap Weight Calculation

Fill Source Signal Generation

Spectral Island Weight Calculation

G WS

F

Z Y


5. EvaluationListening Test

Performed in quiet listening room using closed studio headphones 11 listeners Listening test with multiple stimuli Seven‐grade comparison test scale (ITU‐R BS.1284)

Condition Description

Reference Coded signal

Hidden reference Coded signal

BS Birdie suppression

BWE Bandwidth extension

BWE + BS Combination of BWE and BS

NLD BWE algorithm (Larsen&Aarts, 2005)


5. EvaluationListening Test (2)

10 test signals of length between 8 s and 20 s each Loudness normalization (ITU‐R BS.1770)

Three different codec implementations1. MP3 encoder, 64 kbps, bandwidth 10 kHz (less Birdies than codec 2,3)2. MP3 encoder, 96 kbps, bandwidth 12 kHz (more Birdies than codec 1)3. MP3 encoder, 64 kbps, bandwidth 12 kHz (more Birdies than codec 1) Listeners should rate according to1. Quality of bandwidth‐extension2. Quality of concealment of spectral islands and gaps3. Overall sound quality


Results per item, for all items and groups of codecs Proposed algorithms improve sound quality over all items Speech item (Spe3) also benefits of enhancement

5. EvaluationListening Test (3)

All 1Lower BWLess Birdies

All 2,3Higher BWMore Birdies


6. Conclusion

Methods for Restoration of spectral islands and gaps Bandwidth extension

Methods do not use meta data Methods work in real‐time Evaluation in a listening test yielded Proposed algorithms improve sound quality over all items Speech also benefits of proposed algorithms


Listen to Sonamic Enhancement Right NowWe Invite You to Get an Impression

Regency Ballroom

Listen also to

Symphoria 3D

Sonamic Loudness


Thank you for your attention!

methods for low bitrate coding enhancement part i ... · audio decoder car head unit (degraded) pcm...

Documents