bës tni test. - umlaut.com€¦ · mtn vodacom cell c telkom voice max. 340 288 246 158 74 cities...

For adding something on top in South Africa and reaching an overall score of 829 dots in mobile network benchmarking survey we proudly award this certificate to

MTN South Africa

Score 829 out of 1000 in TotalScore 288 out of 340 in Voice ServicesScore 428 out of 510 in Data ServicesScore 113 out of 150 in Crowdsourced Quality

bëst in test.

Hakan EkmenChief Executive Officer Telecommunication

Mobile BenchmarkSouth Africa

02 19

Measurement setupMeasurement Overview

Drivetest Voice Data

Device Samsung Galaxy S8 cat16 Samsung Galaxy S8 cat16

Test Cases Mobile-to-Mobile (M2M)Side1 (VoLTE) to Side2 (VoLTE) 115 sec call window70 sec call duration15 sec call setup timeoutMulti-RAB traff ic injection on both sides

Data 4G preferred CAHTTP DL datastream 7sHTTP UL datastream 7sHTTP 3MB DL fi xed fi le transferHTTP 1MB UL fi xed fi le transferWeb Browsing - Kepler ETSI Ref. Page8 Live web pages (50% http / 50% https)4 YouTube HD video ~ 45s

Mobility andRoute Types

Drive test 100%50% in Cities, 10 % in Towns and 40% on Roads

Samples ~ 5200 per Operator ~ 4500 per Operator

Dates 24 measurement days24.01.2019 – 11.02.2019

Crowd DataAssessment

Crowd Coverage and User Download Speed Nov’18 till Jan’19Crowd Data Service Availability May’18 till Jan’19

2

umlaut certifi cate umlaut certifi cate

3

umlaut tested and measured the performance of its voice

and data services on smartphones in comparison to other

LTE/UMTS/GSM mobile radio networks in metropolitan

and rural areas of Country South Africa.

The audit was done as a performance benchmark

performed by umlaut between 24.01.2019 and 11.02.2019

in cities and towns as well as on connection roads.

Dedicated measurements have been executed as drive

tests outdoors using a Samsung Galaxy S8 cat16 Smart-

phone 1000 Mbit/s Download / 105 Mbit/s Upload.

All data measurements have been performed in 4G

preferred mode. Voice measurements have been done in

4G/4G preferred mode on both sides, while call origin has

been alternated.

In addition crowdsourced performance data has been

collected and evaluated between November 2018 and

January 2019.

The following pages provide a comparative overview about

the performance results observed for the different tested

service types.

Testing areasDrive route

Testing areasCrowdsourcing based test area

40.6 % of population measured

50.4 % of populated areas

14.342 kmmeasuring distance

50.000 km2

size of tested area

Cape TownPort Elizabeth

East London

Durban

BloemfonteinKimberley

JohannesburgEvaton

PretoriaWitbank

Polokwane

Rustenburg

4


5

Claims Methodology

The leader in mobile benchmarking, umlaut, has analy-

zed the mobile networks of South Africa with regards to

mobile network performance. We measure smartphone

voice and data performance based on extensive drive-

tests – from major metropolitan areas to smaller cities and

connection roads.

We objectively define the routes and test methodology

and publish the results through certificates or public

benchmark reports.

As the de-facto industry standard, our benchmarking

methodology focuses on customer-perceived network

quality and covers a wide range of mobile services.

Today, more than 200 mobile networks in more than

120 countries are being evaluated by our unique scoring

methodology. It allows a technical analysis that is unpre-

cedented in its level of detail and enables comparisons

between the network performance and capability of each

mobile network. Our public benchmarks as well as the

certificate benchmarks help network operators to demon-

strate how well they are delivering wireless connections to

consumers, business users and enterprises and reveals the

areas of improvement.

Best in crowd-sourced qualityMTN

Best in voiceMTN

Best in dataMTN

Best in testMTN

6


7

Score and breakdown

Shown scores are rounded.

MTN achieved the highest overall score among competitors with 829 dots out of 1000.

Total score

MTN Vodacom Cell C Telkom

Voice max. 340 288 246 158 74

Cities Drivetest 204 95% 81% 53% 26%

Towns Drivetest 68 96% 80% 56% 17%

Roads Drivetest 68 44% 37% 19% 14%

Data max. 510 428 406 357 243

Cities Drivetest 306 89% 85% 77% 62%

Towns Drivetest 102 85% 82% 72% 27%

Roads Drivetest 102 67% 61% 46% 26%

Crowdsourced Quality max. 150 113 109 105 67

Crowd 150 75% 73% 70% 45%

Total 1000 829 761 620 384

Overall score considering Voice, Data and Crowdsourcing.

Total Score1000 Dots

MTN

428

288

113

Vodacom

406

246

109

Cell C

357

158

105

Telkom

243

74

67

- S

how

n to

tal s

core

s ar

e ro

und

ed.

963

829

761

620

384

Voicemax. 340

Datamax. 510

Crowdmax. 150

Operator

8


9

Achieved percentage of the maximum score in each of the diff erent data services.

Data Service Group max MTN Vodacom Cell C Telkom

Cities

Web Browsing 68,0 88% 79% 77% 66%

File Download 85,0 85% 81% 77% 69%

File Upload 85,0 90% 87% 74% 51%

YouTube 68,0 95% 95% 83% 61%

Towns

Web Browsing 22,7 85% 79% 66% 30%

File Download 28,3 79% 76% 75% 23%

File Upload 28,3 86% 80% 69% 20%

YouTube 22,7 89% 92% 80% 39%

Roads

Web Browsing 22,7 69% 56% 45% 27%

File Download 28,3 70% 64% 53% 25%

File Upload 28,3 58% 54% 35% 19%

YouTube 22,7 72% 69% 52% 36%

Voice KPI overviewAchieved values of all networks under test in each of the relevant Voice Key Performance Indicators (KPIs) for the

geographical categories “Cities”, “Towns” and “Roads”.

Voice Service Group Unit MTN Vodacom Cell C Telkom

Cities

Call Setup Time [s] 99,1 96,9 94,4 90,1

Call Setup Time (P90) [s] 2,0 2,1 6,1 6,3

Speech Quality (AVG) [MOS-LQO] 2,6 2,6 8,3 9,0

Qualifi er [%] 4,2 3,9 3,2 2,9

Towns

Call Setup Time [s] 99,3 96,7 95,0 79,6



Qualifi er [%] 4,2 3,9 3,0 3,0

Call Setup Time [s] 85,1 84,3 80,1 55,8



Qualifi er [%] 4,0 3,8 2,9 2,8

10


11

Data Services KPI overviewAchieved values of all networks under test in each of the relevant Data Key Performance Indicators (KPIs) for the

geographical category “Cities”.

Data Services KPI OverviewAchieved values of all networks under test in each of the relevant Data Key Performance Indicators (KPIs) for the geo-

graphical category “Towns”.

Data Cities KPI Name Unit MTN Vodacom Cell C Telkom

HTTP Live Page DL

Qualifi er [%] 99,3 97,6 97,8 95,4

Time to fi rst Byte [ms] 494 514 541 577

KBytes fi rst second [KB] 586 452 427 442

HTTP Static Page DLQualifi er [%] 99,4 99,7 99,2 97,0

Overall Session Time [s] 1,1 1,2 1,5 1,6

HTTP fi le DL 3MB

Qualifi er [%] 99,9 99,5 99,9 98,2

Average Session Time [s] 1,4 1,8 2,9 3,4

90% faster than [kbit/s] 10027 7294 4383 3550


HTTP fi le UL 1MB

Qualifi er [%] 99,6 99,8 98,5 95,9




HTTP DL FDTT

Qualifi er [%] 99,3 99,6 99,6 97,5

Average Data Rate [kbit/s] 43796 33423 22201 31521



HTTP UL FDTT

Qualifi er [%] 99,7 99,5 99,0 95,1




YouTube

Start Time [s] 1,8 2,0 2,4 2,5

Playouts w/o interruptions [%] 99,7 99,7 98,4 98,4

AVG Resolution [p] 1077 1076 1060 1040

Qualifi er [%] 99,6 99,8 98,0 93,8

YouTube Live

Start Time [s] 3,8 4,1 4,3 4,3


AVG Resolution [p] 1076 1072 1041 1041

Qualifi er [%] 99,6 99,6 97,8 92,4

Data Towns KPI Name Unit MTN Vodacom Cell C Telkom

HTTP Live Page DL

Qualifi er [%] 99,3 98,2 95,6 70,6





HTTP fi le DL 3MB

Qualifi er [%] 99,8 99,5 99,7 72,2




HTTP fi le UL 1MB

Qualifi er [%] 99,8 99,0 98,1 69,7




HTTP DL FDTT

Qualifi er [%] 98,7 98,8 99,3 72,1




HTTP UL FDTT

Qualifi er [%] 99,8 98,2 98,7 69,6




YouTube

Start Time [s] 2,0 2,2 2,6 2,4


AVG Resolution [p] 1075 1072 1042 1011

Qualifi er [%] 99,0 99,0 97,4 70,9

YouTube Live

Start Time [s] 4,2 4,2 4,6 4,2


AVG Resolution [p] 1065 1057 1015 999

Qualifi er [%] 97,7 99,6 96,4 67,1

12


13

Achieved values of all networks under test in each of the relevant Data Key Performance Indicators (KPIs) for the geo-

graphical category “Roads”.

Data Cities KPI Name Unit MTN Vodacom Cell C Telkom

HTTP Live Page DL

Qualifi er [%] 93,8 89,9 86,6 27,4





HTTP fi le DL 3MB

Qualifi er [%] 94,3 92,2 90,6 27,5




HTTP fi le UL 1MB

Qualifi er [%] 92,1 91,2 85,1 26,7




HTTP DL FDTT

Qualifi er [%] 93,3 92,1 88,4 28,2




HTTP UL FDTT

Qualifi er [%] 90,7 89,5 85,5 27,0




YouTube

Start Time [s] 2,2 2,2 2,6 2,9


AVG Resolution [p] 995 922 943 935

Qualifi er [%] 92,3 91,9 86,8 26,7

YouTube Live

Start Time [s] 4,1 4,3 4,4 4,4


AVG Resolution [p] 993 894 913 966

Qualifi er [%] 91,0 90,8 83,4 24,6

14

umlaut certifi cate

umlaut AG

Am Kraftversorgungsturm 3 · 52070 Aachen · Germany

Hakan Ekmen · Chief Executive Officer Telecommunication

cell +49 151 571 33 235 · [email protected]

www.umlaut.com

bës tni test. - umlaut.com€¦ · mtn vodacom cell c telkom voice max. 340 288 246 158 74 cities...

Documents