1 ip cablecom and mediacom 2004 eurescom p905 (aquavit) audio and audiovisual quality for mobile...

1

IP Cablecom and MEDIACOM 2004IP Cablecom and MEDIACOM 2004

EURESCOM P905 (AQUAVIT)EURESCOM P905 (AQUAVIT)Audio and audiovisual quality Audio and audiovisual quality

for mobile servicesfor mobile services

Harald Klaus – T-SystemsRapporteur of Q9/12

[email protected]


OutlineOutline

Personal ProfileFramework and Objectives of AQUAVITTest beds of AQUAVIT Test methods developed in AQUAVITResults of InvestigationsConclusions


Personal ProfilePersonal Profile

Harald KlausStudied electrotechnical engineering

at the Technical University of BerlinResearch assistant for speech recognition

applications and speech quality assessmentHead of quality assessment of tele-services

at T-Systems Nova Berkom in BerlinRapporteur in ITU-T Study Group 12 for instrumental

speech quality assessment methodsRapporteur in ETSI Technical Committee STQ

4

The EURESCOM Project P905 The EURESCOM Project P905 (AQUAVIT)(AQUAVIT)


Framework of EURESCOM P905Framework of EURESCOM P905

AQUAVIT = Assessment of Quality for Audio-Visual Signals over IP and UMTSStart: January 1999End: December 20003 Partners...


Objectives of AQUAVITObjectives of AQUAVIT

Development of test beds for IP and UMTSDevelopment of subjective and objective test

methodsQuality assessment of typical audio-video

transmission scenarios for IP and UMTSCorrelation between subjective and objective

quality evaluations


Transm

ission errors

Access Unit Access UnitAccess Unit

Sync. Layer

SL-packetized stream

AU AU AU AU AU

Elementary Stream

Compression Layer

Elementary Stream

Encoders

MP

EG

-4

TestbedsTestbeds

Coding of signals for UMTS on send side


Executive

Dem

ux

Application

MediaStream BIFSDecoder

Root SceneObject

MediaStream

MediaObject

DecoderMediaStream

T

Presenter

VisualRenderer

AudioRenderer

T

Testbeds (2)Testbeds (2)Decoding of signals for UMTS on receive

side


Testbeds (3)Testbeds (3)

IP TestbedRunning on standard PCs

in realtimeAdjustable set of

parameters forCoding/DecodingBit rateBit and Frame Errors Error distribution

Windows user interface


Sequence I

MO

S

Time

just one vote for the whole sequence

a) ordinary way of voting

b) simulation of continuos voting

SI Part a SI Part b SI Part c SI Part d SI Part e SI Part f

Six different votes for the sequence

MO

S

Time

Subjective test methodsSubjective test methodsSpeech quality evaluationSpeech quality evaluation


Subjective test methodsSubjective test methodsSpeech quality evaluationSpeech quality evaluation


Subjective test methodsSubjective test methodsAudio quality evaluationAudio quality evaluation


Subjective test methodsSubjective test methodsVideo & A/V quality evaluationVideo & A/V quality evaluation


• Questionnaire

• Conjoint analysis

• Focus groups

Subjective test methodsSubjective test methodsAcceptability testsAcceptability tests


Objective test methodsObjective test methods

TOSQA (Telecommunication objective speech quality assessment)

PEAQ (Perceptual evaluation of audio quality)

A new video quality assessment model developed by British Telecom


Experiment 1narrow band & wideband

speech

= 96.6%

1

2

3

4

5

1 2 3 4 5MOS

TM

OS

1

2

3

4

5

1 2 3 4 5MOS

TM

OS

Experiment 2aOnly narrow band

speech

= 95.4%

1

2

3

4

5

1 2 3 4 5MOS

TM

OS

1

2

3

4

5

1 2 3 4 5MOS

TM

OS

Experiment 2bOnly wideband

speech

= 98.2%

1

2

3

4

5

1 2 3 4 5MOS

TM

OS

1

2

3

4

5

1 2 3 4 5MOS

TM

OS

Results of investigations Results of investigations Correlation of subjective and objective speech Correlation of subjective and objective speech quality (TOSQA)quality (TOSQA)


010

2030

4050

6070

8090

100

Mushra

Peaq

010

2030

4050

6070

8090

100

Mushra

Peaq

Results of investigations Results of investigations Subjective test results - audio qualitySubjective test results - audio quality

Low quality conditions (average of all sequences)


0

20

40

60

80

100

120

Mushra

Peaq

0

20

40

60

80

100

120

Mushra

Peaq

Results of investigations Results of investigations Subjective test results - audio qualitySubjective test results - audio quality

High quality conditions (average of all sequences)


Results of investigations Results of investigations Acceptability test resultsAcceptability test results

Quality and cost are the equally important main factors to the population questioned.

Benefit of mobility does not compensate for any deficit in quality.


ConclusionsConclusions

AQUAVIT developed a bundle of improved subjective quality methods for assessment ofSpeech quality (narrowband and wideband)Audio quality (intermediate and high quality)Video and A/V quality (intermediate and high q.)

All tests use the same set of materialResults of all tests can be directly used for

comparisons


T-Systems Nova BerkomHarald KlausHead of Quality Assessment ===!"§==Systems=

User-centred Quality Engineering.Quality assessmentat T-Systems Nova.Consulting

Test conceptsand test design Quality Monitoringand Reporting Process Optimization

Speech, audioand A/V quality measures According to international

recommendationsandstandards Optimized, efficienttestprocedures Tests withrepresentative user groups Intrusiveand non-intrusive

measurement methods Measurements with Head-and-Torso

simulators (HATS)

Our Labs High quality accordingto ITU

requirements for auditoryand instrumental

quality assessment for recordings usingHATS for speechand audioproduction

New test New test services services of T-System of T-System Nova are Nova are based on based on AQUAVIT AQUAVIT methodsmethodsand resultsand results

1 ip cablecom and mediacom 2004 eurescom p905 (aquavit) audio and audiovisual quality for mobile...

Documents

assessment of quality

audiovisual quality

av quality intermediate

umtsdevelopment of subjective

ip testbedrunning

ip cablecom

aquavit methods

tsystems nova berkom