quality of service for…. mobile, fixed, roaming,...
TRANSCRIPT
![Page 1: Quality of Service for…. mobile, fixed, roaming, billingportal.etsi.org/stq/presentations/ascom.pdf · 2007. 3. 19. · Nokia Pannafon Chungwa Ascom speech evaluation round the](https://reader035.vdocument.in/reader035/viewer/2022071415/6110d26908b1080007095ddf/html5/thumbnails/1.jpg)
@RBNL�
4XDOLW\�RI�6HUYLFH��IRU�ZLUHG��ZLUHOHVV�QHWZRUNV
Quality of Service for….
mobile, fixed, roaming, billing
5D\PRQG�:X
$VFRP
7HO������������������
)D[������������������
UD\PRQG�ZX#LQIUDV\V�DVFRP�FK
![Page 2: Quality of Service for…. mobile, fixed, roaming, billingportal.etsi.org/stq/presentations/ascom.pdf · 2007. 3. 19. · Nokia Pannafon Chungwa Ascom speech evaluation round the](https://reader035.vdocument.in/reader035/viewer/2022071415/6110d26908b1080007095ddf/html5/thumbnails/2.jpg)
@RBNL�@RBNL�
4XDOLW\�RI�6HUYLFH��IRU�ZLUHG��ZLUHOHVV�QHWZRUNV
Speech evaluation :
- where did we start from
- the product / applications today
- the speech quality evaluation method
- ITU - T activities
- calibrations
![Page 3: Quality of Service for…. mobile, fixed, roaming, billingportal.etsi.org/stq/presentations/ascom.pdf · 2007. 3. 19. · Nokia Pannafon Chungwa Ascom speech evaluation round the](https://reader035.vdocument.in/reader035/viewer/2022071415/6110d26908b1080007095ddf/html5/thumbnails/3.jpg)
@RBNL�
4XDOLW\�RI�6HUYLFH��IRU�ZLUHG��ZLUHOHVV�QHWZRUNV
In 1993, Ascom delivered the …..
first system in the world to use real speech for
speech quality evaluation
![Page 4: Quality of Service for…. mobile, fixed, roaming, billingportal.etsi.org/stq/presentations/ascom.pdf · 2007. 3. 19. · Nokia Pannafon Chungwa Ascom speech evaluation round the](https://reader035.vdocument.in/reader035/viewer/2022071415/6110d26908b1080007095ddf/html5/thumbnails/4.jpg)
@RBNL�
4XDOLW\�RI�6HUYLFH��IRU�ZLUHG��ZLUHOHVV�QHWZRUNV
1
1.5
2
2.5
3
3.5
4
4.5
5
0 1 2 3 4 5 6 7
AvgQualIF
Motivation : RxQual = speech qualitysp
eech
qua
lity
![Page 5: Quality of Service for…. mobile, fixed, roaming, billingportal.etsi.org/stq/presentations/ascom.pdf · 2007. 3. 19. · Nokia Pannafon Chungwa Ascom speech evaluation round the](https://reader035.vdocument.in/reader035/viewer/2022071415/6110d26908b1080007095ddf/html5/thumbnails/5.jpg)
@RBNL�
4XDOLW\�RI�6HUYLFH��IRU�ZLUHG��ZLUHOHVV�QHWZRUNV
MobilkomMax.mobil
Mobistar
Radiomobil
Tele Danmark MobileSonofonMobilix
France Telecom MobileNortel Matra Cellular
T-MobilLucentMannesmann
Hutchison
Westel
P.T. SatelindoTelkomsel
Esat Digifone
Omnitel
ONPT Marocco
Netcom
Isla Communic.
era GSM
TMNTelecelMain Road
MTSKB Impuls
Telefonica MovilesAirtel
Telia
MOPT Lucent
Swisscom
Etisalat
Cellnet
Airtouch Intl.
Nokia
Pannafon
Chungwa
Ascom speech evaluation round the world
![Page 6: Quality of Service for…. mobile, fixed, roaming, billingportal.etsi.org/stq/presentations/ascom.pdf · 2007. 3. 19. · Nokia Pannafon Chungwa Ascom speech evaluation round the](https://reader035.vdocument.in/reader035/viewer/2022071415/6110d26908b1080007095ddf/html5/thumbnails/6.jpg)
@RBNL�
4XDOLW\�RI�6HUYLFH��IRU�ZLUHG��ZLUHOHVV�QHWZRUNV
End to end measurements :End to end measurements :
Speech quality (excellent, good, medium, poor, bad )
Ping Pong Robotic voice
EchoEcho
Signal level
Silence
Speech samples in use : English, French, German, Mandarin,Cantonese
![Page 7: Quality of Service for…. mobile, fixed, roaming, billingportal.etsi.org/stq/presentations/ascom.pdf · 2007. 3. 19. · Nokia Pannafon Chungwa Ascom speech evaluation round the](https://reader035.vdocument.in/reader035/viewer/2022071415/6110d26908b1080007095ddf/html5/thumbnails/7.jpg)
@RBNL�
4XDOLW\�RI�6HUYLFH��IRU�ZLUHG��ZLUHOHVV�QHWZRUNV
Products on the market :
mobile network measurements --- QVoice
fixed network measurements ------ QNet
![Page 8: Quality of Service for…. mobile, fixed, roaming, billingportal.etsi.org/stq/presentations/ascom.pdf · 2007. 3. 19. · Nokia Pannafon Chungwa Ascom speech evaluation round the](https://reader035.vdocument.in/reader035/viewer/2022071415/6110d26908b1080007095ddf/html5/thumbnails/8.jpg)
@RBNL�
4XDOLW\�RI�6HUYLFH��IRU�ZLUHG��ZLUHOHVV�QHWZRUNV
Q V M I n d o o rQ V M O u t d o o r
Q V P
Q V o i c e
1 . . . 4
G P S A n t e n n a
Q V S
Q V o i c e
M e a s u r e m e n tD a t a
Q V S
Q V MC e l l u l a rN e t w o r k 1 . . . 4
I S D N / P S T N
C a l l s , S p e e c h S a m p l e s
QVoice Principles of operation
![Page 9: Quality of Service for…. mobile, fixed, roaming, billingportal.etsi.org/stq/presentations/ascom.pdf · 2007. 3. 19. · Nokia Pannafon Chungwa Ascom speech evaluation round the](https://reader035.vdocument.in/reader035/viewer/2022071415/6110d26908b1080007095ddf/html5/thumbnails/9.jpg)
@RBNL�
4XDOLW\�RI�6HUYLFH��IRU�ZLUHG��ZLUHOHVV�QHWZRUNV
Purposely designed QVoice hardware for :
Indoor measurements Drive tests
![Page 10: Quality of Service for…. mobile, fixed, roaming, billingportal.etsi.org/stq/presentations/ascom.pdf · 2007. 3. 19. · Nokia Pannafon Chungwa Ascom speech evaluation round the](https://reader035.vdocument.in/reader035/viewer/2022071415/6110d26908b1080007095ddf/html5/thumbnails/10.jpg)
@RBNL�
4XDOLW\�RI�6HUYLFH��IRU�ZLUHG��ZLUHOHVV�QHWZRUNV
QVoice Results analysis :
![Page 11: Quality of Service for…. mobile, fixed, roaming, billingportal.etsi.org/stq/presentations/ascom.pdf · 2007. 3. 19. · Nokia Pannafon Chungwa Ascom speech evaluation round the](https://reader035.vdocument.in/reader035/viewer/2022071415/6110d26908b1080007095ddf/html5/thumbnails/11.jpg)
@RBNL�
4XDOLW\�RI�6HUYLFH��IRU�ZLUHG��ZLUHOHVV�QHWZRUNV
QNetend-to-end network
performance measurementfor PSTN/ISDN/GSM
switching quality
transmission quality
charging quality
from a customer’s point ofview
![Page 12: Quality of Service for…. mobile, fixed, roaming, billingportal.etsi.org/stq/presentations/ascom.pdf · 2007. 3. 19. · Nokia Pannafon Chungwa Ascom speech evaluation round the](https://reader035.vdocument.in/reader035/viewer/2022071415/6110d26908b1080007095ddf/html5/thumbnails/12.jpg)
@RBNL�
4XDOLW\�RI�6HUYLFH��IRU�ZLUHG��ZLUHOHVV�QHWZRUNV
QNet: different applications simultaneously
Q-Statistics Q-Monitoring
Q-Analysis
![Page 13: Quality of Service for…. mobile, fixed, roaming, billingportal.etsi.org/stq/presentations/ascom.pdf · 2007. 3. 19. · Nokia Pannafon Chungwa Ascom speech evaluation round the](https://reader035.vdocument.in/reader035/viewer/2022071415/6110d26908b1080007095ddf/html5/thumbnails/13.jpg)
@RBNL�
4XDOLW\�RI�6HUYLFH��IRU�ZLUHG��ZLUHOHVV�QHWZRUNV
LAN
QNet PUQNet PU
S0
QNet CUQNet PU
S0, a/bClient 1
Client n
ISDN / PSTN
Server
TSU
S0, a/b S0, a/bRS232 RS232
RS232
QNet system architecture
![Page 14: Quality of Service for…. mobile, fixed, roaming, billingportal.etsi.org/stq/presentations/ascom.pdf · 2007. 3. 19. · Nokia Pannafon Chungwa Ascom speech evaluation round the](https://reader035.vdocument.in/reader035/viewer/2022071415/6110d26908b1080007095ddf/html5/thumbnails/14.jpg)
@RBNL�
4XDOLW\�RI�6HUYLFH��IRU�ZLUHG��ZLUHOHVV�QHWZRUNV
QNet supports complex test scenarois
![Page 15: Quality of Service for…. mobile, fixed, roaming, billingportal.etsi.org/stq/presentations/ascom.pdf · 2007. 3. 19. · Nokia Pannafon Chungwa Ascom speech evaluation round the](https://reader035.vdocument.in/reader035/viewer/2022071415/6110d26908b1080007095ddf/html5/thumbnails/15.jpg)
@RBNL�
4XDOLW\�RI�6HUYLFH��IRU�ZLUHG��ZLUHOHVV�QHWZRUNV
Parameter ISDN PSTNteleservice, bearer capability Y Y
time, date Y Y
duration Y Y
call progress indicator Y
signaling tone measurement Y
d-channel protocol analysis Y
cause/location value Y
unsuccessfull call attempt Y Y
connection setup delay Y Y
connection release delay Y
unsuccessfull release attempt Y Y
premature release Y Y
Switching parameter
![Page 16: Quality of Service for…. mobile, fixed, roaming, billingportal.etsi.org/stq/presentations/ascom.pdf · 2007. 3. 19. · Nokia Pannafon Chungwa Ascom speech evaluation round the](https://reader035.vdocument.in/reader035/viewer/2022071415/6110d26908b1080007095ddf/html5/thumbnails/16.jpg)
@RBNL�
4XDOLW\�RI�6HUYLFH��IRU�ZLUHG��ZLUHOHVV�QHWZRUNV
Parameter ISDN PSTNloop transfer delay Y
BERT (G.821: ES, SES, DM) Y
unweighted noise (O.41) Y Y
weighted noise (O.41) Y Y
impulsive noise (O.71) Y Y
notch noise Y Y
envelope delay distortion Y Y
interruptions (O.61, O.62) Y Y
echo: level/delay (G.122, sine-burst) Y Y
attenuation (on 4 frequencies) Y Y
speech quality measurement (Ascom PACE) Y Y
Transmission parameter
![Page 17: Quality of Service for…. mobile, fixed, roaming, billingportal.etsi.org/stq/presentations/ascom.pdf · 2007. 3. 19. · Nokia Pannafon Chungwa Ascom speech evaluation round the](https://reader035.vdocument.in/reader035/viewer/2022071415/6110d26908b1080007095ddf/html5/thumbnails/17.jpg)
@RBNL�
4XDOLW\�RI�6HUYLFH��IRU�ZLUHG��ZLUHOHVV�QHWZRUNV
Parameter ISDN PSTNcharging out of rating Y Y
initial charge out of rating Y Y
no charging Y Y
CDR verification Y Y
Charging parameter
![Page 18: Quality of Service for…. mobile, fixed, roaming, billingportal.etsi.org/stq/presentations/ascom.pdf · 2007. 3. 19. · Nokia Pannafon Chungwa Ascom speech evaluation round the](https://reader035.vdocument.in/reader035/viewer/2022071415/6110d26908b1080007095ddf/html5/thumbnails/18.jpg)
@RBNL�
4XDOLW\�RI�6HUYLFH��IRU�ZLUHG��ZLUHOHVV�QHWZRUNV
PACE :
a robust objective speech quality assessment
testing wireless & wired networks in service
![Page 19: Quality of Service for…. mobile, fixed, roaming, billingportal.etsi.org/stq/presentations/ascom.pdf · 2007. 3. 19. · Nokia Pannafon Chungwa Ascom speech evaluation round the](https://reader035.vdocument.in/reader035/viewer/2022071415/6110d26908b1080007095ddf/html5/thumbnails/19.jpg)
@RBNL�
4XDOLW\�RI�6HUYLFH��IRU�ZLUHG��ZLUHOHVV�QHWZRUNV
3 types of objective methods :
Comparison of transmitted signal and a reference
Absolute Estimation without known reference e.g. INMD
Transmission Rating from knowledge of network e.g. E model
PACE is a comparison method
![Page 20: Quality of Service for…. mobile, fixed, roaming, billingportal.etsi.org/stq/presentations/ascom.pdf · 2007. 3. 19. · Nokia Pannafon Chungwa Ascom speech evaluation round the](https://reader035.vdocument.in/reader035/viewer/2022071415/6110d26908b1080007095ddf/html5/thumbnails/20.jpg)
@RBNL
�
4XDOLW\�RI�6
HUYLFH��IRU�ZLUHG�
�ZLUHOHVV�Q
HWZRUNV
Spe
ech
Coder
Source
Signal
Coded
Signal
Transm
ission
Channel
Decoder
Source Coding
Sound PressureProduced by Talker
Room Noise
Loss BetweenMicrophone and Earphone
Loss BetweenEarphone and Microphone
Transmission Path BetweenEarphone and Eardrum
Attenuation
Delay / Echo
Distortions
Time-Variation
Noise
Room Noise
Tra
nsm
ission Param
eters
![Page 21: Quality of Service for…. mobile, fixed, roaming, billingportal.etsi.org/stq/presentations/ascom.pdf · 2007. 3. 19. · Nokia Pannafon Chungwa Ascom speech evaluation round the](https://reader035.vdocument.in/reader035/viewer/2022071415/6110d26908b1080007095ddf/html5/thumbnails/21.jpg)
@RBNL�
4XDOLW\�RI�6HUYLFH��IRU�ZLUHG��ZLUHOHVV�QHWZRUNV
PACE Principle
Time align.Level equal.
SourceSignal
SpeechQuality
AssessmentPost-
processing
CodedSignal
Pre-processing
![Page 22: Quality of Service for…. mobile, fixed, roaming, billingportal.etsi.org/stq/presentations/ascom.pdf · 2007. 3. 19. · Nokia Pannafon Chungwa Ascom speech evaluation round the](https://reader035.vdocument.in/reader035/viewer/2022071415/6110d26908b1080007095ddf/html5/thumbnails/22.jpg)
@RBNL�
4XDOLW\�RI�6HUYLFH��IRU�ZLUHG��ZLUHOHVV�QHWZRUNV
PACE Preprocessing
HammingWindow DFT
TimeSpreading
Critical BandFiltering
| |
FrequencySpreading
CodedSignal
SourceSignal
ToAssessmentToAssessmentToAssessment
![Page 23: Quality of Service for…. mobile, fixed, roaming, billingportal.etsi.org/stq/presentations/ascom.pdf · 2007. 3. 19. · Nokia Pannafon Chungwa Ascom speech evaluation round the](https://reader035.vdocument.in/reader035/viewer/2022071415/6110d26908b1080007095ddf/html5/thumbnails/23.jpg)
@RBNL�
4XDOLW\�RI�6HUYLFH��IRU�ZLUHG��ZLUHOHVV�QHWZRUNV
Similaritycalculation
forfourth
energylevel
Similaritycalculation
forthird
energylevel
Similaritycalculation
forsecondenergylevel
Similaritycalculation
forfirst
energylevel
SimilarityWeightingdependent
onenergylevel
FromPreprocessingFromPreprocessingFromPreprocessing
////
To PostprocessingTo PostprocessingTo Postprocessing
PACE Assessment
![Page 24: Quality of Service for…. mobile, fixed, roaming, billingportal.etsi.org/stq/presentations/ascom.pdf · 2007. 3. 19. · Nokia Pannafon Chungwa Ascom speech evaluation round the](https://reader035.vdocument.in/reader035/viewer/2022071415/6110d26908b1080007095ddf/html5/thumbnails/24.jpg)
@RBNL�
4XDOLW\�RI�6HUYLFH��IRU�ZLUHG��ZLUHOHVV�QHWZRUNV
dcxbxaxMOS +++= 23
xxx MOSpMOSpMOSp
Maping Auditory distance to the MOS
X
MO
S
PACE Postprocessing
![Page 25: Quality of Service for…. mobile, fixed, roaming, billingportal.etsi.org/stq/presentations/ascom.pdf · 2007. 3. 19. · Nokia Pannafon Chungwa Ascom speech evaluation round the](https://reader035.vdocument.in/reader035/viewer/2022071415/6110d26908b1080007095ddf/html5/thumbnails/25.jpg)
@RBNL�
4XDOLW\�RI�6HUYLFH��IRU�ZLUHG��ZLUHOHVV�QHWZRUNV
Codecs and Systems used to calibrate PACE
G.711 64 kbit/s PCM
G.726 32 kbit/s ADPCM
IS-54 8 kbit/s American Cellular
GSM-FR GSM Full Rate
G.729 8 kbit/s CS-ACELP
G.728 16 kbit/s Low-Delay Code Excited Linear Prediction
JDC 3.45 kbit/s Japanese Digital Cellular (Half Rate)
DCME Digital Circuit Multiplication Equipment: Quad-Coder, DTX-240 (24 kbit/s variable)
IS-9x CDMA 8 kbit/s Code Division Multiple Access
Multiple Transcodings, Random -Burst frame erasure, Random bit errors
Circuit noise, Comfort noise, etc.
![Page 26: Quality of Service for…. mobile, fixed, roaming, billingportal.etsi.org/stq/presentations/ascom.pdf · 2007. 3. 19. · Nokia Pannafon Chungwa Ascom speech evaluation round the](https://reader035.vdocument.in/reader035/viewer/2022071415/6110d26908b1080007095ddf/html5/thumbnails/26.jpg)
@RBNL�
4XDOLW\�RI�6HUYLFH��IRU�ZLUHG��ZLUHOHVV�QHWZRUNV
Calibration results presented to ….
ITU-T SG 12 Q 13
Speech Quality Expert Group (SQEG)
2nd international workshop on
Advances in Objective Speech Quality Assesment
September 98 , UK
![Page 27: Quality of Service for…. mobile, fixed, roaming, billingportal.etsi.org/stq/presentations/ascom.pdf · 2007. 3. 19. · Nokia Pannafon Chungwa Ascom speech evaluation round the](https://reader035.vdocument.in/reader035/viewer/2022071415/6110d26908b1080007095ddf/html5/thumbnails/27.jpg)
@RBNL�
4XDOLW\�RI�6HUYLFH��IRU�ZLUHG��ZLUHOHVV�QHWZRUNV
Results of ITU-T Supplem. 23 test data, Exp.1
1
1.5
2
2.5
3
3.5
4
4.5
5
1 1.5 2 2.5 3 3.5 4 4.5 5
PACE
MO
S
ρ ρ = 0.955
Experiment 1: Interworking with fixed and mobile codecs (G.726, G.728 …).Multiple encodings by tandeming with other coding standards.
![Page 28: Quality of Service for…. mobile, fixed, roaming, billingportal.etsi.org/stq/presentations/ascom.pdf · 2007. 3. 19. · Nokia Pannafon Chungwa Ascom speech evaluation round the](https://reader035.vdocument.in/reader035/viewer/2022071415/6110d26908b1080007095ddf/html5/thumbnails/28.jpg)
@RBNL�
4XDOLW\�RI�6HUYLFH��IRU�ZLUHG��ZLUHOHVV�QHWZRUNV
Correlations between different Algorithms and MOS
Experime Languag eCand. 1 Cand. 2 Cand. 3 Cand. 4 Cand. 5 PACEExp1 French 0.94 0.92 0.82 0.93 0.96 0.94Exp1 Japanese 0.94 0.86 0.87 0.94 0.93 0.94Exp1 English 0.96 0.83 0.89 0.92 0.94 0.97Exp2 French 0.95 0.37 0.94 0.97Exp2 Japanese 0.93 0.96 0.41 0.91 0.95Exp2 English 0.94 0.96 0.47 0.9 0.97Exp3 French 0.74 0.74 0.74 0.75 0.94Exp3 Italian 0.95 0.71 0.86 0.46 0.94Exp3 Japanese 0.91 0.85 0.88 0.55 0.92Exp3 English 0.82 0.85 0.82 0.65 0.93
![Page 29: Quality of Service for…. mobile, fixed, roaming, billingportal.etsi.org/stq/presentations/ascom.pdf · 2007. 3. 19. · Nokia Pannafon Chungwa Ascom speech evaluation round the](https://reader035.vdocument.in/reader035/viewer/2022071415/6110d26908b1080007095ddf/html5/thumbnails/29.jpg)
@RBNL�
4XDOLW\�RI�6HUYLFH��IRU�ZLUHG��ZLUHOHVV�QHWZRUNV
QVoice (mobile) QNet (fixed)
PACE speech evaluation
testing of Interconnected networks
![Page 30: Quality of Service for…. mobile, fixed, roaming, billingportal.etsi.org/stq/presentations/ascom.pdf · 2007. 3. 19. · Nokia Pannafon Chungwa Ascom speech evaluation round the](https://reader035.vdocument.in/reader035/viewer/2022071415/6110d26908b1080007095ddf/html5/thumbnails/30.jpg)
@RBNL�
4XDOLW\�RI�6HUYLFH��IRU�ZLUHG��ZLUHOHVV�QHWZRUNV
trtrtzrtzrtttttttttttttt
BRA
LAN
BRA, PRA, a /b
Client 1 Client nClient 2 Applicat ions
QNet
Server
QNet
GSMMSC BSC BTS
BRA, a /b
PSTN / ISDN
PU
QVoice
Customer
QVoice
GSM
PU
BRA, a /b
PSTN / ISDN
PU
GSM
QoS in an Interconnected Environment