Ouagadougou, Burkina Faso, 18 July 20131
Content and presentation of Recommendation E.MQoS
Joachim Pomy, SG 12 RapporteurConsultant, Opticom GmbH
ITU Workshop on “Benchmarking QoS Evaluation of Multimedia
Networks”
(Ouagadougou, Burkina Faso, 18 July 2013)
Ouagadougou, Burkina Faso, 18 July 2013
Ouagadougou, Burkina Faso, 18 July 20132
Draft New Rec. E.MQoS
E-series:Overall network operation, telephone service, service operation and human factors
E.800-series:Quality of telecommunication services: concepts, models, objectives and dependability planning
Title:QoS Aspects for Popular Services in Mobile Networks
Ouagadougou, Burkina Faso, 18 July 20133
Scope of E.MQoS (1/2)
Assessment of Quality of ServiceSummary of basics of QoS, always seen from the user's perspectiveDifferences to Quality of Experience (QoE)Guidance to assure QoS assessments conducted in a meaningful way
Definition of Quality of Service parameters and their computationQoS parameters and their computation for popular services
abstract definition = generic description of parametersabstract equationuser and technical trigger points
Harmonized definitions are prerequisites for the comparison of QoS measurements and measurement results
Typical procedures for Quality of Service measurement equipmentMeasurement procedures needed to perform the measurements of QoS parameters
Ouagadougou, Burkina Faso, 18 July 20134
Scope of E.MQoS (2/2)
Requirements for QoS measurement equipmentDefines minimum requirements of QoS measurement equipment
values and trigger points needed to compute the QoS parameterallow performing measurements in a reliable and reproducible way
Definition of typical measurement profilesSpecifies typical measurement profiles
required to enable benchmarking of different mobile networks both within and outside national boundaries.
Post processing and statistical methodsDescribes procedures for statistical calculations
for QoS measurement of mobile networks using probing systems.
Network based Quality of Service measurementsE2E QoS measurements inside the network
without direct access to the end point terminal.
Ouagadougou, Burkina Faso, 18 July 20135
Popular Services
Services are not standardizedFocus on popular services
Widely used in the marketTest results are meaningfull to customers
ExamplesWeb radioE-mail clientsWeb portals
Ouagadougou, Burkina Faso, 18 July 20136
QoS aspects of Mobile
Netw orkAccessibility
c ircuitsw itched
packetsw itched
ServiceAccessibility
ServiceIntegrity
ServiceRetainability
TelephonySMS
PingFile
TransferMMS
Netw orkAvailability
VideoTelephony
WebBrow sing
StreamingPoC
MobileBroadcast
Layer 2
Layer 3
Layer 4
Layer 1
Ouagadougou, Burkina Faso, 18 July 20137
Main structure of E.MQoS
6 Assessment of Quality of Service7 Definition of Quality of Service parameters
and their computation8 Typical procedures for Quality of Service measurement equipment9 Requirements for Quality of Service measurement equipment10 Definition of typical measurement profiles11 Post processing and statistical methods12 Network based Quality of Service measurements Appendices (informative)
Ouagadougou, Burkina Faso, 18 July 20138
6 Assessment of Quality of Service
6.1 QoS Background6.2 QoS Assessment Process6.3 Basic Settings for QoS
Assessments6.4 Service Independent QoS
Criteria6.5 Service dependent QoS
Criteria
Ouagadougou, Burkina Faso, 18 July 20139
7 Definition of Quality of Service parameters and their computation
7.1 QoS Parameter Basics7.2 Service independent QoS
Parameters7.3 Direct Services QoS
Parameters7.4 Store‑and‑forward (S&F)
Services QoS Parameters
Ouagadougou, Burkina Faso, 18 July 201310
7.2 Service independent QoS Parameters
7.2.1 Radio Network Unavailability [%]7.2.2 Network Non Accessibility [%]7.2.3 Attach Failure Ratio [%]7.2.4 Attach Setup Time [s]7.2.5 PDP Context Activation Failure Ratio [%]7.2.6 PDP Context Activation Time [s]7.2.7 PDP Context Cut‑off Ratio [%]7.2.8 Data Call Access Failure Ratio [%]7.2.9 Data Call Access Time [s]7.2.10 DNS Host Name Resolution Failure Ratio [%]7.2.11 DNS Host Name Resolution Time [s]
Ouagadougou, Burkina Faso, 18 July 201311
7.3 Direct Services QoS Parameters
7.3.1 File Transfer (FTP)7.3.2 Mobile Broadcast7.3.3 Ping7.3.4 Push to Talk over Cellular (PoC)7.3.5 Streaming Video7.3.6 Telephony7.3.7 Video Telephony7.3.8 Web Browsing (HTTP)7.3.9 Web Radio7.3.10 WLAN service provisioning with HTTP based
authentication7.3.11 Wireless Application Protocol (WAP)7.3.12 IMS Multimedia Telephony7.3.14 Group Call7.3.13 E-mail
Ouagadougou, Burkina Faso, 18 July 201312
7.4 Store‑and‑forward (S&F) Services QoS Parameters
7.4.1 Generic Store‑and‑forward Parameters
7.4.2 E‑mail7.4.3 Multimedia Messaging Service (MMS)7.4.4 Short Message Service (SMS) Short Data Service (SDS)
Ouagadougou, Burkina Faso, 18 July 201313
8 Typical procedures for Quality of Service measurement equipment
8.1 Aim of measurement8.2 Classification of services8.3 General aspects for all types
of services8.4 Telephony measurements8.5 Store-and-forward services
measurement8.6 Data measurements
Ouagadougou, Burkina Faso, 18 July 201314
8.4 Telephony measurements
8.4.1 General aspects
8.4.2 Speech telephony
8.4.4 Group Call
8.4.3 Video telephony
Ouagadougou, Burkina Faso, 18 July 201315
8.5 Store-and-forward services measurement
8.5.1General aspects
8.5.2SMS measurements
8.5.3MMS
8.5.4E-Mail
8.5.5SDS
Ouagadougou, Burkina Faso, 18 July 201316
8.6 Data measurements
8.6.1 Common aspects8.6.2 FTP8.6.3 HTTP8.6.4 E-mail8.6.5 WAP8.6.6 Streaming Video8.6.7 Media Download
Ouagadougou, Burkina Faso, 18 July 201317
9 Requirements for Quality of Service measurement equipment
9.1 Overview9.2 General requirements9.3 Fixed QoS Test-equipment
(FQT)9.5 Mobile based measurement
equipment9.4 Mobile QoS Test-equipment
(MQT)
Ouagadougou, Burkina Faso, 18 July 201318
10 Definition of typical measurement profiles
10.1Measurement profiles 10.1.3 Usage Profiles for Data
Sessions10.1.1 Classification of
measurement environments10.1.2 Service profiles
Ouagadougou, Burkina Faso, 18 July 201319
11 Post processing and statistical methods
11.1Important measurement data types in mobile communications
11.2Distributions and moments11.3Visualization techniques11.4Time series modelling11.5Data aggregation11.6Assessment of performance
indices
Ouagadougou, Burkina Faso, 18 July 201320
12 Network based QoS measurements
12.1Network Measurement Basics
12.2Measuring QoS Parameters in the Network
12.3Comparing Network and End-point Test Measurements
Ouagadougou, Burkina Faso, 18 July 201321
Appendices (informative)
I Examples for measuring trigger pointsII Streaming explanationsIII Push to Talk over Cellular Information IV QoS parameter exportV Which parameters have an impact on the effect of
blocking ?VI Reference {SMS | SDS}VII Content integrity checkingVIII Transfer times versus used data rate and content sizeIX Examples of statistical calculationsX The concept of QoE reporting XI Examples of Network Based QoS MeasurementsXII 3GPP SA5 "UE Management"
Ouagadougou, Burkina Faso, 18 July 201322
Confidence Intervalls for Different Sample Sizes (1)
The following examples show the effect of different sample sizes in a measurement campaign. It is also based on the Pearson-Clopper formulas for the calculation of confidence intervals. Therefore, the examples are valid in a generic way and even for small sample sizes. For higher sample numbers, the calculation of confidence intervals based on the approximation of a normal distribution can be applied.
Three different graphs are depicted: Sample sizes in the range:between 100 and 1 100 samples;between 1 100 and 2 100 samples; andbetween 1 000 and 11 000 samples.
Ouagadougou, Burkina Faso, 18 July 201323
Confidence Intervalls for Different Sample Sizes (2)
0 20 40 60 80 100
05
10
15
20
25
30
Width of confidence interval for fixed sample size (Pearson-Clopper)
Estimated rate in percent
Wid
th o
f co
nfid
enc
e in
terv
al i
n pe
rce
nt
x
xx
xx
xxx xxxx xxx xxx xxx xxx xxx xxx xxx xxx xxx xxxx xxxx
xx
x
x+
++
++++++++++++++++++++++++++++++++++++++++++++
++
+
+o
oo
oooooooooooooooooooooooooooooooooooooooooooooo
o
ox
xx
xx xxx xxxx xxx xxx xxx xxx xxx xxx xxx xxx xxx xxxx xxx xxx
x
x++
++++++++++++++++++++++++++++++++++++++++++++++++
+
ox+ox+
100 Samples300 Samples500 Samples700 Samples900 Samples1100 Samples
Ouagadougou, Burkina Faso, 18 July 201324
Confidence Intervalls for Different Sample Sizes (3)
0 20 40 60 80 100
01
23
45
67
Width of confidence interval for fixed sample size (Pearson-Clopper)
Estimated rate in percent
Wid
th o
f co
nfid
en
ce in
terv
al i
n p
erc
en
t
x
x
x
x
xx
xx
xx
xx
xxx xxx xxx xxx xxx xxx xxx xxx xxxx
xx
xx
xx
x
x
x
x
x+
+
+
++
++
++
++
++++++++++++++++++++++++++++
++
++
++
++
+
+
+
+o
o
o
oo
oo
oo
oo
oooooooooooooooooooooooooooooo
oo
oo
oo
o
o
o
ox
x
x
xx
xx
xx
xxx xxx xxx xxx xxx xxx xxx xxx xxx xxx xx
xx
xx
xx
x
x
x
x+
+
+
++
++
++
++++++++++++++++++++++++++++++++
++
++
++
+
+
+
+
ox+ox+
1100 Samples1300 Samples1500 Samples1700 Samples1900 Samples2100 Samples
Ouagadougou, Burkina Faso, 18 July 201325
Confidence Intervalls for Different Sample Sizes (4)
0 20 40 60 80 100
02
46
81
0
Width of confidence interval for fixed sample size (Pearson-Clopper)
Estimated rate in percent
Wid
th o
f co
nfid
en
ce in
terv
al i
n p
erc
en
t
x
xx
xx
xxx xxxx xxx xxx xxx xxx xxx xxx xxx xxx xxx xxxx xxxx
xx
x
x+
++
++++++++++++++++++++++++++++++++++++++++++++
++
+
+0
00
0000000000000000000000000000000000000000000000
0
0x
xxxx xxx xxxx xxx xxx xxx xxx xxx xxx xxx xxx xxx xxxx xxx xxx
x
x+
++++++++++++++++++++++++++++++++++++++++++++++++
+
+
ox+ox+
1000 Samples3000 Samples5000 Samples7000 Samples9000 Samples11000 Samples
Ouagadougou, Burkina Faso, 18 July 201326
Schedule for draft new Rec. E.MQoS
31 August 2013 Final timeline for submitting comments directly to the editors
7 November 2013Submission as Contribution to SG 12
20 November 2013Deadline for Contributions submitted to TSB
12 December 2013Planned Consent of SG12 Plenary
January 2014Start of Alternate Approval Procedure (AAP)
March 2014Pre-Publication
??? 2014Publication
Ouagadougou, Burkina Faso, 18 July 201327
Any questions ?
Joachim PomyTelecommunications & Int'l StandardsGermanyTel.: +49 177 78 71958Email: [email protected]