Cotonou, Benin, 16-17 July 2012
Country example of QoS measurement of voice serviceCase of Morocco
Hassan TALIB,Vice-Chair ITU-T SG 12,
[email protected] // [email protected]
ITU Workshop on “Workshop on Practical measurement of QoS/QoE
Parameters for Regulatory Compliance”
(Cotonou, Benin, 16-17 July 2012)
Cotonou, Benin, 16-17 July 2012
Presentation Scope
The QoS for Morocco telecommunications A protocol for voice QoS measurements:
Choice of a subjective method,List and definitions of indicators,Measurements Methodology:
Definition of voice QoS measurement,Sample and Statistic considerations,
Organization and Operation,Process of Outcomes.
Recommendations to regulators/Op.
Cotonou, Benin, 16-17 July 2012
Every operators are responsible for QoS
The QoS for Morocco Telecommunications
VSATOperators
VSATOperators
GMPCS Operators
GMPCS Operators
Radio Trunk Operators
Radio Trunk Operators ISPISP Fixed and mobile
Operators Fixed and mobile
Operators Fixed Operator with
narrow mobility Fixed Operator with
narrow mobility
Fixed and mobile: oligopoly
SPACECOM
GULFSAT
CIMECOM
SPACECOM
GULFSAT
CIMECOM
MAROC TELECOM
MEDITELECOM
MAROC TELECOM
MEDITELECOM EUROPEAN DATACOM
GLOBALSTAR
ORBCOMM
SOREMAR
THURAYA
EUROPEAN DATACOM
GLOBALSTAR
ORBCOMM
SOREMAR
THURAYA
INQUAM MORATELINQUAM MORATEL
DuopolyOligopolyoligopoly Open
Competition
MAROC TELECOM
MAROC CONNECT(WANA)
MEDITELECOM
ISPs
MAROC TELECOM
MAROC CONNECT(WANA)
MEDITELECOM
ISPs
MAROC CONNECT(WANA)
MAROC CONNECT(WANA)
Cotonou, Benin, 16-17 July 2012
The QoS for telecommunications in Morocco
The ANRT regularly
monitors the QoS
A national regulatory Framework:-Allocations -QoS Obligations for operators
International regulatory framework (standards) :-IUT-T : series E, G, P, Y, QoS handbook, ...-Regional : ETSI (series EG), IEEE,…
A benchmark of best practices and technological monitoring
Use feedback :-Complaints,-Press,-End Users Associations,-…
Analysis of data received from the operators in terms of performances, KPI and QoE measurements
Ground Measurements (campaigns) by
ANRT
Cotonou, Benin, 16-17 July 2012
Development of voice mobile telephony service in Morocco
The QoS for telecommunications in Morocco
Cotonou, Benin, 16-17 July 2012
Presentation Scope
The QoS for Morocco telecommunications A protocol for voice QoS measurements:
Choice of a subjective method,List and definitions of indicators,Measurements Methodology:
Definition of voice QoS measurement,Sample and Statistic considerations,
Organization and Operation,Process of Outcomes.
Recommendations to regulators/Op.
Cotonou, Benin, 16-17 July 2012
The quality of service sets a premium on models for reference (IUT-T standards : E, I, G, P, Y… ou ETSI : ETSI TS 102 250-x), on the levels of performance for service provided and the measurement methods. The models and methods are supposed to be coherent with the users expectations.
For voice service, QoS requirements could be related to many aspects sumed up in its availability, accessibility and continuity and reliability.
Protocol for voice QoS measurements
Choice of a subjective method
Cotonou, Benin, 16-17 July 2012
Protocol for voice QoS measurements Choice of a subjective method
Operator’s vision Regulator’s Vision
Motivation Economic
Competitive
Rules and Regulations
Center of interest (users)
Users loyalty (churn) and attraction of newcomers
(Concept SLA)Protection for users
Measurement Local (various components of the network)
KPI
External (end to end: service as daily
understood by users)
Measurement Tools
- Local machines meters in the network (objectives methods)
- DTS
(objectives methods)
- Measures and investigations by subjective methods
- DTS
(Subjective Methods)Necessity to harmonize the definitions of indicators between operators and the regulator.
Cotonou, Benin, 16-17 July 2012
Presentation Scope
The QoS for Morocco telecommunications A protocol for voice QoS measurements:
Choice of a subjective method,List and definitions of indicators,Measurements Methodology:
Definition of voice QoS measurement,Sample and Statistic considerations,
Organization and Operation,Process of Outcomes.
Recommendations to regulators/Op.
Cotonou, Benin, 16-17 July 2012
Protocol for Voice QoS measurements List and definitions of indicators
Rate Indicator Definition
TR Rate of successful communications and maintained during 2 minutes
A communication is considered as successful if the call launched succeeds at the first attempt and if the communication is maintained 2 minutes without being cut off. The
rate (TR) is related to the number of successful communications and the total number of call attempts made.
TR
Perfect Rate of successful communications and maintained during 2 minutes and quality of perfect audio
A communication is considered as successful and the perfect quality if it is succeeded at the first indicator and the audio quality considered by both parties are
perfect (compared to the quality of direct communication).
Acceptable
Rate of successful communications and maintained during 2 minutes and quality of acceptable audio
A communication is considered as successful and the acceptable quality if it is succeeded at the first indicator and if audio quality considered by both parties is
slightly disrupted without interferring the conversation.
Poor Rate of successful communications and maintained during 2 minutes and quality of poor audio
A communication is considered as successful and poor quality if it is succeeded at the first indicator and if the audio quality considered by both parties is frequently
disrupted in the receiving channel by numerous hit on the line, but it is possible again to hear one another.
Bad Rate of successful communications and maintained during 2 minutes and quality of bad audio
A communication is considered as successful and the quality degenerated if it is successful at the first indicator and if it is very difficult to hear one another, the
conversation is impossible.
RC Rate of cut off communications
A communication is considered as cut off if at the first attempt, it is established and maintained more than 5 seconds, but cut off before 2 mins. The rate (TC) is the ratio
between the number of cut off communications and the total number of call attempts made.
RF Rate of failed communications
A communication is considered as failed if at the first attempt, it is not established or has not been maintained, more than 5 seconds. The Rate (TE) is the ratio between
the number of failed communications and the total of call attempts made.
TR + TE + TC = 100% and TNR = TE + TC
Cotonou, Benin, 16-17 July 2012
Protocol for voice QoS measurements List and definitions of indicators
Evaluation subjective de la qualité auditive : définitions des défauts de communication vocale
Default Definition
Low Voice Level received more and less very low than the normal level
Echo Return of its transmission distinctive from when it’s simultaneous
Metallic Noise Chopping Noises to be compared to metallic one
Frying noise Interferences
Micro-defaults Small cut-offs of sound giving an impression of chopping
Silence Blank enough long
Distorsion Degradation of voice distant without other noise
Cotonou, Benin, 16-17 July 2012
Presentation Scope
The QoS for Morocco telecommunications A protocol for voice QoS measurements:
Choice of a subjective method,List and definitions of indicators,Measurements Methodology:
Definition of voice QoS measurement,Sample and Statistic considerations,
Organization and Operation,Process of Outcomes.
Recommendations to regulators/Op.
Cotonou, Benin, 16-17 July 2012
Protocol for voice QoS measurements Measurements Methodology
A QoS measurement shall necessarily be done within the coverage area by all concerned mobile networks. The QoS measurement is different from the coverage measurement.
A voice QoS measurement consists in trying to establish a communication from the mobile investigator to the fixed investigator to verify the reception of the call (no failure) and the continuation of the communication for two minutes (no cut off) and to assess the audio quality of the established communication.
Each network is tested by a “couple” of investigators, on mobile and the other fixed.
Cotonou, Benin, 16-17 July 2012
Protocol for voice QoS measurements Measurements Methodology
Cotonou, Benin, 16-17 July 2012
Protocol for voice QoS measurements Measurements Methodology
A communication is considered cut off if, at the first attempt:
it is established and maintained more than 5 seconds, but cut off before 2 minutes.
A communication is considered failed if:
at the first attempt (maximum delay time is about 15 seconds), it is not established, or has not been maintained more than 5 seconds.
Cotonou, Benin, 16-17 July 2012
Protocol for voice QoS measurements Measurements Methodology
Precautions to be taken:Outdoor : avoid measuring under trees.
Indoor : very inside buildings and till 8th floor.
Incar : use terminals without additional antenna and drive at a speed less than 80km/h.
Force terminals on 2G in case of use of bi-mode posts.
Every points for measurements shall be covered.
Cotonou, Benin, 16-17 July 2012
Protocol for voice QoS measurements Measurements Methodology
Sample and statistic considerations:
Average duration of a measurement: 7 mn,Average duration for measurements takes place everyday: 8:00 to 10:00 (Example : 09:00-13:00 and 14:00 to 20:00),Everyday (per team) : 85 measurements x Number of operators; For 3 operators : 255 measurements per day,
Cotonou, Benin, 16-17 July 2012
Protocol for voice QoS measurements Measurements Methodology
Sample and statistic considerations:
Breakdown Element: Per operator, Per site (cities, motorways, national highways, railways), per configuration (Indoor, Outdoor, Incar), per traffic destination (OnNet and OffNet).Important numbers to provide a significant sample (statistic error about 2%) : elementary values after any breakdown shall be higher than 33.
Cotonou, Benin, 16-17 July 2012
Presentation Scope
The QoS for Morocco telecommunications A protocol for voice QoS measurements:
Choice of a subjective method,List and definitions of indicators,Measurements Methodology:
Definition of voice QoS measurement,Sample and Statistic considerations,
Organization and Operation,Process of Outcomes.
Recommendations to regulators/Op.
Cotonou, Benin, 16-17 July 2012
Protocol for voice QoS measurements Measurements Methodology
Organization and Operation:
Decider to achieve based on own resources or controlled subcontracting,Select effectively investigators (mobiles and Fixed) and conduct a targeted training,Consult with operators on the methodology but not the sample and the planning of each campaign,
Cotonou, Benin, 16-17 July 2012
Protocol for voice QoS measurements
Measurements Methodology
Organization and operation:
Develop a guide for the investigator with some templates to be used by the investigator: Ref., delay, GPS Information, recipient, outcomes,…
Use the principle for codificationprincipe de codification : opérateur, site, configuration, trafic,…
Cotonou, Benin, 16-17 July 2012
Protocol for voice QoS measurements Measurements Methodology
Organization and operation:
Guarantee any conditions so that each measurement is exactly conducted in the same conditions for every operators: delay, space, method,…
Proceed to a number of important blank tests calibrate any test system.
Cotonou, Benin, 16-17 July 2012
Presentation Scope
The QoS for Morocco telecommunications A protocol for voice QoS measurements:
Choice of a subjective method,List and definitions of indicators,Measurements Methodology:
Definition of voice QoS measurement,Sample and Statistic considerations,
Organization and Operation,Process of Outcomes.
Recommendations to regulators/Op.
Cotonou, Benin, 16-17 July 2012
Protocol for voice QoS measurements Measurements Methodology
Outcomes Processing: (Scheduled to be Delivered)
Trend in indicator “positive” : successful rate.Trend in indicator “negative” : unsuccessful rate = TE+TC.Reports:
Global Report: reference,Specific Report to each operator (request for
actions plan for corrections),Report for publication.
Cotonou, Benin, 16-17 July 2012
Presentation Scope
The QoS for Morocco telecommunications A protocol for voice QoS measurements:
Choice of a subjective method,List and definitions of indicators,Measurements Methodology:
Definition of voice QoS measurement,Sample and Statistic considerations,
Organization and Operation,Process of Outcomes.
Recommendations to regulators/Op.
Cotonou, Benin, 16-17 July 2012
Conclusions and Recommendations
Recommendation n°1Permanent Consultation with the upstream operators upon the methodology: financing of campaigns. Adoption of an agreed protocol of references for measurements.
Recommendation n°2Positive use of outcomes by the operators (consider sanctions as last resort).
Cotonou, Benin, 16-17 July 2012
Conclusions and Recommendations
Recommendation n°3Publication of measurements outcomes,Proposal of a scenario :
1st voice campaign on based upon a large sample (thirty cities, highways, national motorways and railways).1st quarter of the year
2nd similar campaign practically based on the first upon the same sample.4th quarter of the year(Publication of results)
Communication of operators’ results
Delay of 5 to 6 months for correction of discrepancies
identified by the ANRT
Cotonou, Benin, 16-17 July 2012
Conclusions and Recommendations Recommendation n°3 (bis)
Publication of outcome measurements
Taux de réussite global (On-Net et Off-Net) par site de mesures et par opérateur
Site for measurement
sTMR Operator 1 Operator 2 Operator 3)
All cities 95,32% = = =
Cities 96,78% = = =
Highways 95,68% = - - +
National Motorways 96,03%
+ = =
Railways 87,00%= = =
TMR
- - - - - - = = + + + + + +
TMR×0,95 TMR×0,98 TMR×0,99 TMR×1,01 TMR×1,02 TMR×1,05
The TMR is equal to the successfuk average rates (TR) recorded by each of the three operators:
TMR
Cotonou, Benin, 16-17 July 2012
Thanks for your attentionQuestions/Answers