straightforward | trusted | intelligenta2.mndcdn.com/image/upload/oy3yvacoqjbf6bzvr2wk.pdf ·...

33
STRAIGHTFORWARD | TRUSTED | INTELLIGENT Competitive Network Benchmark for Telenor Sverige AB Final Report, Data Mikko Valtonen, Sampsa Laamanen | 27 February 2013

Upload: ngokhanh

Post on 08-Mar-2018

218 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: STRAIGHTFORWARD | TRUSTED | INTELLIGENTa2.mndcdn.com/image/upload/oy3yvacoqjbf6bzvr2wk.pdf · Straightforward | Trusted | Intelligent ... optimisation, quality benchmarking ... Apple

STRAIGHTFORWARD | TRUSTED | INTELLIGENT

Competitive Network Benchmark for Telenor Sverige AB Final Report, Data

Mikko Valtonen, Sampsa Laamanen | 27 February 2013

Page 2: STRAIGHTFORWARD | TRUSTED | INTELLIGENTa2.mndcdn.com/image/upload/oy3yvacoqjbf6bzvr2wk.pdf · Straightforward | Trusted | Intelligent ... optimisation, quality benchmarking ... Apple

©Omnitele Ltd. 2012 2 Competitive Network Benchmark for Telenor Sverige AB

table of contents

1. Executive summary

2. Omnitele in brief

3. Introduction

4. Methodology

5. Results

6. Annexes

Page 3: STRAIGHTFORWARD | TRUSTED | INTELLIGENTa2.mndcdn.com/image/upload/oy3yvacoqjbf6bzvr2wk.pdf · Straightforward | Trusted | Intelligent ... optimisation, quality benchmarking ... Apple

©Omnitele Ltd. 2012 3 Competitive Network Benchmark for Telenor Sverige AB

executive summary 1/2

Scope: Telenor, Telia

Tests executed in three different scenarios:

− Data 3G: iPhone 5

− Data 3G/4G: Samsung Galaxy Note 2

− Data 3G/4G: Huawei E3276 USB Modem

Test campaign was extensive:

− 79 cities

− 550 test locations

− 5 000+ test cases per operator

Telenor contracted Omnitele to benchmark the end-user QoS of data services in Sweden in Q1/2013

Page 4: STRAIGHTFORWARD | TRUSTED | INTELLIGENTa2.mndcdn.com/image/upload/oy3yvacoqjbf6bzvr2wk.pdf · Straightforward | Trusted | Intelligent ... optimisation, quality benchmarking ... Apple

©Omnitele Ltd. 2012 4 Competitive Network Benchmark for Telenor Sverige AB

Telenor has higher average bitrate across all scenarios

Telia scores higher in Total File Transfer Success Rate

Combining both factors (bitrate and file transfer success rate), Telenor performs better in 67% of the measured locations

executive summary 2/2

Page 5: STRAIGHTFORWARD | TRUSTED | INTELLIGENTa2.mndcdn.com/image/upload/oy3yvacoqjbf6bzvr2wk.pdf · Straightforward | Trusted | Intelligent ... optimisation, quality benchmarking ... Apple

©Omnitele Ltd. 2012 5 Competitive Network Benchmark for Telenor Sverige AB

table of contents

1. Executive summary

2. Omnitele in brief

3. Introduction

4. Methodology

5. Results

6. Annexes

Page 6: STRAIGHTFORWARD | TRUSTED | INTELLIGENTa2.mndcdn.com/image/upload/oy3yvacoqjbf6bzvr2wk.pdf · Straightforward | Trusted | Intelligent ... optimisation, quality benchmarking ... Apple

Straightforward | Trusted | Intelligent

Omnitele in brief

Omnitele was founded in 1988 to set up

the first GSM operator and network in

the world.

Omnitele has assisted operators in

1,000+ projects in 80+ countries.

Our services range from Technology

Strategy to Network Design and

Network Quality Improvement for

Telecom operations

“Since the beginning of DNA, Omnitele has played a critical role in developing an efficient model for radio network planning and its outsourcing.”

Riitta Tiuraniemi, Managing Director, DNA

The headquarter situates in Helsinki,

Finland. We have local presence in the

Netherlands and the Hashemite Kingdom

of Jordan.

The company is owned by Finnish telecom

operator groups and is independent of all

international operators and telecom

equipment vendors.

Page 7: STRAIGHTFORWARD | TRUSTED | INTELLIGENTa2.mndcdn.com/image/upload/oy3yvacoqjbf6bzvr2wk.pdf · Straightforward | Trusted | Intelligent ... optimisation, quality benchmarking ... Apple

Straightforward | Trusted | Intelligent

Bridging the gap between technology & business

Technology Strategy

Evaluating, selecting and managing licenses,

technologies and vendors

Network Design

Turnkey network planning services for

mobile and broadcasting networks

Network Quality

Network auditing, optimisation, quality

benchmarking and performance

management services

“Due to excellent work by Omnitele we achieved our goal to be best in quality”

Dave Newbold Chief Operations & Technology Officer at JT

Page 8: STRAIGHTFORWARD | TRUSTED | INTELLIGENTa2.mndcdn.com/image/upload/oy3yvacoqjbf6bzvr2wk.pdf · Straightforward | Trusted | Intelligent ... optimisation, quality benchmarking ... Apple

Straightforward | Trusted | Intelligent

1,000+ projects in 80+ countries

Page 9: STRAIGHTFORWARD | TRUSTED | INTELLIGENTa2.mndcdn.com/image/upload/oy3yvacoqjbf6bzvr2wk.pdf · Straightforward | Trusted | Intelligent ... optimisation, quality benchmarking ... Apple

©Omnitele Ltd. 2012 9 Competitive Network Benchmark for Telenor Sverige AB

table of contents

1. Executive summary

2. Omnitele in brief

3. Introduction

4. Methodology

5. Results

6. Annexes

Page 10: STRAIGHTFORWARD | TRUSTED | INTELLIGENTa2.mndcdn.com/image/upload/oy3yvacoqjbf6bzvr2wk.pdf · Straightforward | Trusted | Intelligent ... optimisation, quality benchmarking ... Apple

©Omnitele Ltd. 2012 10 Competitive Network Benchmark for Telenor Sverige AB

introduction | scope of work

OBJECTIVE

To evaluate and benchmark the end-user QoS of Swedish cellular networks Telenor and Telia under different data scenarios

SCOPE COVERAGE

Measurements in all cities with population > 15 000

Minimum 2 locations per city, 50 test samples per city, 5 test samples per location

Locations chosen independently by Omnitele to cover places with high subscriber density (airports, hotels, restaurants, shopping malls, railway/bus terminals, business districts...)

Alltogether 79 cities, 550 test locations, 5000+ test samples per scenario for each operator

Data Scenarios

FTP download with iPhone5 (3G)

FTP download with Galaxy Note2 LTE (3G/4G)

FTP download with Huawei E3276 LTE USB Modem (3G/4G)

Page 11: STRAIGHTFORWARD | TRUSTED | INTELLIGENTa2.mndcdn.com/image/upload/oy3yvacoqjbf6bzvr2wk.pdf · Straightforward | Trusted | Intelligent ... optimisation, quality benchmarking ... Apple

©Omnitele Ltd. 2012 11 Competitive Network Benchmark for Telenor Sverige AB

introduction | project tasks in practice

VAST TEST CAMPAIGN

79 cities

550 locations

250+ testing hours

30 000+ FTP transfers

Tests executed Jan 15 – Feb 7

Test Scope in Practice

Test Execution Totals

Benchmarked Operators 2

Number of cities 79

Number of test locations 550

Calendar days of testing 23

Aggregates Totals

Successful file transfer tests 30 188

File transfer hours 252

Total GBytes transfered 2 271

Average bitrate Mbit/s 20.1

Page 12: STRAIGHTFORWARD | TRUSTED | INTELLIGENTa2.mndcdn.com/image/upload/oy3yvacoqjbf6bzvr2wk.pdf · Straightforward | Trusted | Intelligent ... optimisation, quality benchmarking ... Apple

©Omnitele Ltd. 2012 12 Competitive Network Benchmark for Telenor Sverige AB

table of contents

1. Executive summary

2. Omnitele in brief

3. Introduction

4. Methodology

5. Results

6. Annexes

Page 13: STRAIGHTFORWARD | TRUSTED | INTELLIGENTa2.mndcdn.com/image/upload/oy3yvacoqjbf6bzvr2wk.pdf · Straightforward | Trusted | Intelligent ... optimisation, quality benchmarking ... Apple

©Omnitele Ltd. 2012 13 Competitive Network Benchmark for Telenor Sverige AB

Nemo Outdoor Measurement PC

methodology | system overview

GPS

Data Test Terminals*

ftp.sunet.se

DATA

DC to AC Inverter

Apple iPhone5

Samsung Galaxy

Note2 LTE

Huawei E3276 USB Modems

*Terminal model details in annex

Nemo Outdoor is an industry standard laptop-based drive test tool supporting over 200 terminals and scanning receivers from various vendors and all major network technologies. More information from www.anite.com

Page 14: STRAIGHTFORWARD | TRUSTED | INTELLIGENTa2.mndcdn.com/image/upload/oy3yvacoqjbf6bzvr2wk.pdf · Straightforward | Trusted | Intelligent ... optimisation, quality benchmarking ... Apple

©Omnitele Ltd. 2012 14 Competitive Network Benchmark for Telenor Sverige AB

All data scenarios in series, one scenario at time

Data test case: N x 30 sec FTP download*

methodology | test cycle overview

Telenor

Telenor

SCENARIO 1.3

Huawei E3276 USB Modem

3G/4G

Telenor

SCENARIO 1.1

Apple iPhone5

3G

SCENARIO 1.2

Samsung Galaxy Note2 LTE

3G/4G

Telia

Telia

Telia

*N = location specific value ranging from 5…25, so that in each city minimum 50 samples are collected.

Page 15: STRAIGHTFORWARD | TRUSTED | INTELLIGENTa2.mndcdn.com/image/upload/oy3yvacoqjbf6bzvr2wk.pdf · Straightforward | Trusted | Intelligent ... optimisation, quality benchmarking ... Apple

©Omnitele Ltd. 2012 15 Competitive Network Benchmark for Telenor Sverige AB

methodology | test procedures

TESTING TIMES REPRESENT TYPICAL END-USER BEHAVIOUR:

• Test days: Testing only between Monday and Saturday.

• Test hours: Testing not conducted during night hours: 00.00 – 06.00

• Saturdays: Measurements only between 10:00 – 00:00. No Saturday measurements in business or university areas.

• Sundays: No testing on Sundays.

METHODOLOGY

• Commercial state of the art smartphones and modems used to benchmark best available end-user QoS offered by the operators.

• Locations chosen independently in blind test fashion by Omnitele to cover places with high population density (airports, hotels, restaurants, shopping malls, railway/bus terminals, business districts...)

• All tests executed in stationary state inside a car. No mobility testing.

• Measurement devices always in same place, on the car dashboard

• Detailed results saved for each test separately. Failures recorded as well.

Coherent test procedure strictly followed. Methodology defined to capture true end-user behaviour. Measurements conducted where and when real subscribers are using data services.

Page 16: STRAIGHTFORWARD | TRUSTED | INTELLIGENTa2.mndcdn.com/image/upload/oy3yvacoqjbf6bzvr2wk.pdf · Straightforward | Trusted | Intelligent ... optimisation, quality benchmarking ... Apple

©Omnitele Ltd. 2012 16 Competitive Network Benchmark for Telenor Sverige AB

table of contents

1. Executive summary

2. Omnitele in brief

3. Introduction

4. Methodology

5. Results

6. Annexes

Page 17: STRAIGHTFORWARD | TRUSTED | INTELLIGENTa2.mndcdn.com/image/upload/oy3yvacoqjbf6bzvr2wk.pdf · Straightforward | Trusted | Intelligent ... optimisation, quality benchmarking ... Apple

©Omnitele Ltd. 2012 17 Competitive Network Benchmark for Telenor Sverige AB

Per Location Telenor Telia Total Locations

Total 1104 546 1650

Apple iPhone5 338 212 550

Samsung Galaxy Note2 LTE 436 114 550

Huawei E3276 330 220 550

66.9 %

61.5 %

79.3 %

60.0 %

33.1 %

38.5 %

20.7 %

40.0 %

0% 20% 40% 60% 80% 100%

Total Average

Apple iPhone5

Samsung Galaxy Note2 LTE

Huawei E3276

Score Board: Telenor vs Telia (Bitrate*TFTSR)

Telenor

Telia

data | scoreboard per location

Scoreboard metric: Average bitrate per file transfer * Total File Transfer Success Rate

- - - -

Total Average

Taking into account Average Bitrate and Total File Transfer Success Rate, Telenor scores higher in 67% of locations.

Page 18: STRAIGHTFORWARD | TRUSTED | INTELLIGENTa2.mndcdn.com/image/upload/oy3yvacoqjbf6bzvr2wk.pdf · Straightforward | Trusted | Intelligent ... optimisation, quality benchmarking ... Apple

©Omnitele Ltd. 2012 18 Competitive Network Benchmark for Telenor Sverige AB

data | scoreboard per area

Telenor 68 %

Telia 32 %

Central-Eastern

Telenor 71 %

Telia 29 %

Western

Telenor 72 %

Telia 28 %

Stockholm Metropolitan

Telenor 65 %

Telia 35 %

Southern

Telenor 49 %

Telia 51 %

Northern

Notes: - Scoreboard metric: Average bitrate per file

transfer * Total File Transfer Success Rate - The total number of tests is 1650. In each

of the 550 test sites 3 different data scenarios were evaluated.

- List of cities per area provided as annex

Area Telenor Telia Total tests

Western Sweden 215 88 303

Southern Sweden 198 108 306

Central-Eastern Sweden 208 98 306

Stockholm Metropolitan Area 388 152 540

Northern Sweden 95 100 195

Total 1104 546 1650

Considering the different geographical areas of Sweden, Telenor scores higher in 4 out of 5 areas.

Page 19: STRAIGHTFORWARD | TRUSTED | INTELLIGENTa2.mndcdn.com/image/upload/oy3yvacoqjbf6bzvr2wk.pdf · Straightforward | Trusted | Intelligent ... optimisation, quality benchmarking ... Apple

©Omnitele Ltd. 2012 19 Competitive Network Benchmark for Telenor Sverige AB

Telenor bitrate is higher than Telia across all measured scenarios

From international perspective the bitrates are very high and enable superior end-user experience

data | average bitrate per file transfer

10.4 28.2 32.4 8.6 15.2 25.5 0

5

10

15

20

25

30

35

40

Apple iPhone5 Samsung Galaxy Note2 LTE Huawei E3276

Mb

it/

s

Average Bitrate per File Transfer

Telenor

Telia

Page 20: STRAIGHTFORWARD | TRUSTED | INTELLIGENTa2.mndcdn.com/image/upload/oy3yvacoqjbf6bzvr2wk.pdf · Straightforward | Trusted | Intelligent ... optimisation, quality benchmarking ... Apple

©Omnitele Ltd. 2012 20 Competitive Network Benchmark for Telenor Sverige AB

data | peak bitrates

PEAK RATES GENERALLY VERY HIGH,

Peak bitrate is the practical maximum bitrate an end-user can experience.

In 3G scenario both operators have almost equally high peak bitrates, Telenor scoring highest with 31.3 Mbit/s – very close to DC-HSDPA practical maximum.

In 4G scenarios Telenor is also better than Telia, following the same trend observed in average bitrates.

Although Note2 and Modem are both 4G devices, the latter one has higher performance as it has better signal processing and it is optimised for data.

31.3 81.7 103.1 26.3 66.3 71.3 0

20

40

60

80

100

120

Apple iPhone5 Samsung Galaxy Note2 LTE Huawei E3276

Mb

it/

s

Peak Bitrate

Telenor

Telia

Page 21: STRAIGHTFORWARD | TRUSTED | INTELLIGENTa2.mndcdn.com/image/upload/oy3yvacoqjbf6bzvr2wk.pdf · Straightforward | Trusted | Intelligent ... optimisation, quality benchmarking ... Apple

©Omnitele Ltd. 2012 21 Competitive Network Benchmark for Telenor Sverige AB

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Den

sit

y

Mbit/s

Bitrate per File Transfer, Apple iPhone5 (Cumulative Distribution)

Telenor

Telia

data | iPhone5 bitrate distribution (3G)

The cumulative distribution shows that Telenor has more samples in the higher end of the scale.

Samples above 10 Mbit/s: Telenor 49% Telia 43%

Page 22: STRAIGHTFORWARD | TRUSTED | INTELLIGENTa2.mndcdn.com/image/upload/oy3yvacoqjbf6bzvr2wk.pdf · Straightforward | Trusted | Intelligent ... optimisation, quality benchmarking ... Apple

©Omnitele Ltd. 2012 22 Competitive Network Benchmark for Telenor Sverige AB

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Den

sit

y

Mbit/s

Bitrate per File Transfer, Samsung Galaxy Note2 LTE (Cumulative Distribution)

Telenor

Telia

data | note2 bitrate distribution (3G/4G)

The cumulative distribution shows that Telenor has more samples in the higher end of the scale.

Samples above 30 Mbit/s: Telenor 48% Telia 12%

Page 23: STRAIGHTFORWARD | TRUSTED | INTELLIGENTa2.mndcdn.com/image/upload/oy3yvacoqjbf6bzvr2wk.pdf · Straightforward | Trusted | Intelligent ... optimisation, quality benchmarking ... Apple

©Omnitele Ltd. 2012 23 Competitive Network Benchmark for Telenor Sverige AB

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Den

sit

y

Mbit/s

Bitrate per File Transfer, USB Modem (Cumulative Distribution)

Telenor

Telia

data | usb modem bitrate distribution (3G/4G)

The cumulative distribution shows that Telenor has more samples in the higher end of the scale.

Samples above 30 Mbit/s: Telenor 54% Telia 37%

Bitrate per File Transfer, Huawei E3276 (Cumulative Distribution)

Page 24: STRAIGHTFORWARD | TRUSTED | INTELLIGENTa2.mndcdn.com/image/upload/oy3yvacoqjbf6bzvr2wk.pdf · Straightforward | Trusted | Intelligent ... optimisation, quality benchmarking ... Apple

©Omnitele Ltd. 2012 24 Competitive Network Benchmark for Telenor Sverige AB

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Total File Transfer Success Rate

Telenor

Telia

data | total file transfer success rate

From end-user perspective both operators perform well.

Telia is better with 99.6%, Telenor follows close behind with 98.9%

Total File Transfer Success Rate takes into account all file transfer errors in end-to-end fashion: “No network access”, “FTP server connect failed”, “File transfer setup failed”, “FTP connection dropped”. Anything above 99% threshold value can be considered extremely good.

Significantly better than Telenor

No statistically significant difference

Significantly worse than Telenor

KPI Telenor Tele2 Telia Tre

Total File Transfer Success Rate 98.9 % 99.6 % 99.6 % 98.5 %

Difference to telenor

Ranking 3 1 2 4

KPI Telenor Tele2 Telia Tre

Total File Transfer Success Rate 98.9 % 99.6 % 99.6 % 98.5 %

Difference to telenor

Ranking 3 1 2 4

Page 25: STRAIGHTFORWARD | TRUSTED | INTELLIGENTa2.mndcdn.com/image/upload/oy3yvacoqjbf6bzvr2wk.pdf · Straightforward | Trusted | Intelligent ... optimisation, quality benchmarking ... Apple

©Omnitele Ltd. 2012 25 Competitive Network Benchmark for Telenor Sverige AB

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Apple iPhone5 Samsung Galaxy

Note2 LTE

Huawei E3276

Total File Transfer Success Rate

Telenor

Telia

Significantly better than Telenor

No statistically significant difference

Significantly worse than Telenor

data | total file transfer success rate per scenario

Telia has the highest total file transfer success rate in all scenarios

The difference is statistically significant only in USB modem scenario

Both operators score very high in all scenarios meeting industry standards and typical end-user expectations

Telenor Telia

Apple iPhone5 98.6 % 99.4 %

Samsung Galaxy Note2 LTE 99.2 % 99.6 %

Huawei E3276 99.0 % 99.9 %

Total File Transfer Success Rate

Page 26: STRAIGHTFORWARD | TRUSTED | INTELLIGENTa2.mndcdn.com/image/upload/oy3yvacoqjbf6bzvr2wk.pdf · Straightforward | Trusted | Intelligent ... optimisation, quality benchmarking ... Apple

©Omnitele Ltd. 2012 26 Competitive Network Benchmark for Telenor Sverige AB

table of contents

List of Annexes

1. Key Performance Indicators, Data

2. Confidence interval calculation

3. Test terminal details

4. Measurement scripts

5. 79 cities, 550 locations

6. Cities and Areas

Page 27: STRAIGHTFORWARD | TRUSTED | INTELLIGENTa2.mndcdn.com/image/upload/oy3yvacoqjbf6bzvr2wk.pdf · Straightforward | Trusted | Intelligent ... optimisation, quality benchmarking ... Apple

©Omnitele Ltd. 2012 27 Competitive Network Benchmark for Telenor Sverige AB

annexes | key performance indicators, data

KPI Description Unit

Average Bitrate per File Transfer

Average application layer throughput per each downlink file transfer according to ETSI TS 102 250-2 specification: Total_Bytes_Received * 8 / (T2 – T1), Where T2 = time of closing the FTP connection T1 = time of sending the file download request to FTP server

Bit/s

Average Bitrate per City

Average over all Average Bitrate per File Transfer –samples within a city: SUM[Average_Bitrate_per_File_Transfer] / COUNT[Average_Bitrateper_File_Transfer]

Bit/s

Average Bitrate per Location

Average over all Average Bitrate per File Transfer –samples within a measurement location: SUM[Average_Bitrate_per_File_Transfer] / COUNT[Average_Bitrate_per_File_Transfer]

Bit/s

Network Accessibility

According to ETSI TS 102 250-2 specification: The probability that an end-user can access the network and gets IP connectivity. Formula: [#Packet_Attach_Success / #Packet_Attach_attempts]*

[#PDP_context_activation_success / #PDP_context_activation_attempts]

%

FTP Server Connect Success Rate

The probability that an end-user can setup the connection to the FTP server, provided that Network Access exists. Formula: #Successfull_FTP_server_connects / #FTP_server_connection_attempts

Pre-conditions: PDP context activated

%

File Transfer Setup Success Rate

The probability that an end-user can start file transfer, provided that FTP server connectivity exists. Formula: #Successfull_File_Transfer_Setups / #File_Transfer_Attempts

Pre-conditions: FTP server connectivity established

%

File Transfer Completion Rate

The probability that an end-user can complete transfer, provided that file transfer setup was successfull. Formula: #Successfully_Completed_File_Transfers / #Successfull_File_Transfer_Setups

Pre-conditions: Successful FTP file transfer setup

%

Total File Transfer Success Rate

The overall probability that an end-user can successfully make a file transfer taking into account all possible failures in end-to-end fashion. Formula: Network_Accessibility * FTP_Server_Connect_Success_Rate *

File_Transfer_Setup_Success_Rate * File_Transfer_Completion_Rate

%

Page 28: STRAIGHTFORWARD | TRUSTED | INTELLIGENTa2.mndcdn.com/image/upload/oy3yvacoqjbf6bzvr2wk.pdf · Straightforward | Trusted | Intelligent ... optimisation, quality benchmarking ... Apple

©Omnitele Ltd. 2012 28 Competitive Network Benchmark for Telenor Sverige AB

annexes | confidence interval calculation

RATIONALE

In order to analyse the statistical reliability of the results, Omnitele applies confidence interval analysis on the calculated KPIs. Confidence Interval can be considered as the error margin of the reported results.

The error margins are visible in the graphs of this report in the error bars, see example figure below. In case two competing operators have differing mean values, but overlapping error bars, the observed difference in average value is not statistically significant. If the error bars don’t overlap, the difference is statistically significant.

CALCULATION

The calculated confidence interval CI is based on (two-tailed) confidence level of 95%. That is, with 95% probability the true population mean is within the sample mean +/- CI.

The CI is calculated as 1.96 x SE, where

SE is equal to the standard error for the sample mean, and

1.96 is the .975 quantile of the normal distribution (CL = 0.95 α = 0.05 1- α/2 = 0.975, Norm.Inv[0.975] = 1.96)

Standard Error SE of the sample mean, is defined as s / 𝒏, where

s is the sample standard deviation (i.e., the sample-based estimate of the standard deviation of the population), and

n is the size (number of observations) of the sample.

95%

96%

97%

98%

99%

100%

KPI1 KPI2

Operator 1

Operator 2

Statistically significant difference

Statistically insignificant difference

Page 29: STRAIGHTFORWARD | TRUSTED | INTELLIGENTa2.mndcdn.com/image/upload/oy3yvacoqjbf6bzvr2wk.pdf · Straightforward | Trusted | Intelligent ... optimisation, quality benchmarking ... Apple

©Omnitele Ltd. 2012 29 Competitive Network Benchmark for Telenor Sverige AB

annexes | test terminal details

Small screen 3G iPhone5

OS Version IOS 6.0.2 (10A551)

Modem Firmware 1.01.00

Sourced from Finland (Gigantti, Stockman)

Small screen 3G/4G Samsung Galaxy Note 2 LTE

Model GT-N7105

OS Version Android 4.1.1 Jelly Bean

Sourced from Sweden (ElGiganten, Phone House)

Large screen 3G/4G, Telenor Huawei E3276

Firmware version 21.260.03.00.00

Connection manager 23.009.05.00.03

Sourced from Sweden (Telenor)

Large screen 3G/4G, Telia Huawei E3276

Firmware version 21.192.03.01.07

Connection manager 21.005.18.00.03

Sourced from Sweden (Telia store)

Small Screen 3G, Small Screen 3G/4G:

Same individual terminal used for all operators

Devices sourced with blind test approach

Large Screen 3G/4G

Each operator tested with operator specific terminal

Flagship USB modems used

Latest firmware update applied

Page 30: STRAIGHTFORWARD | TRUSTED | INTELLIGENTa2.mndcdn.com/image/upload/oy3yvacoqjbf6bzvr2wk.pdf · Straightforward | Trusted | Intelligent ... optimisation, quality benchmarking ... Apple

©Omnitele Ltd. 2012 30 Competitive Network Benchmark for Telenor Sverige AB

annexes | measurement scripts

1. Connect to FTP server: ftp.sunet.se

2. Start file download

3. Wait 30 seconds

4. Stop file download

5. Disconnect FTP

6. Wait 10 seconds

7. Go to 1

Page 31: STRAIGHTFORWARD | TRUSTED | INTELLIGENTa2.mndcdn.com/image/upload/oy3yvacoqjbf6bzvr2wk.pdf · Straightforward | Trusted | Intelligent ... optimisation, quality benchmarking ... Apple

©Omnitele Ltd. 2012 31 Competitive Network Benchmark for Telenor Sverige AB

annexes | 79 cities, 550 locations

City Locations Tests/OP City Locations Tests/OP City Locations Tests/OP

Åkersberga 3 51 Kiruna 2 50 Piteå 3 51

Alingsås 3 51 Köpings 2 50 Sandviken 3 51

Ängelholm 3 51 Kristianstad 4 52 Skellefteå 4 52

Boden 2 50 Kristinehamns 2 50 Skövde 4 52

Boo 3 51 Kungälvs 3 51 Södertälje 7 56

Borås 7 56 Kungsbacka 2 50 Stockholm 138 690

Borlänge 5 50 Landskrona 4 52 Sundsvall 6 54

Enköping 3 51 Lerum 2 50 Täby 7 56

Eskilstuna 7 56 Lidingo 4 52 Trelleborg 3 51

Eslov 2 50 Lidköping 3 51 Trollhättan 5 50

Falkenberg 3 51 Linköping 11 55 Tumba 4 52

Falköping 2 50 Ljungby 2 50 Uddevalla 4 52

Falun 4 52 Luleå 5 50 Umeå 8 56

Gävle 8 56 Lund 9 54 Upplands 4 52

Göteborg 55 275 Malmo 29 145 Uppsala 15 75

Halmstad 6 54 Mariestad 2 50 Vallentuna 3 51

Härnösand 2 50 Märsta 3 51 Vänersborgs 3 51

Hassleholm 2 50 Mölnlycke 2 50 Varberg 3 51

Helsingborg 10 50 Motala 3 51 Värnamo 2 50

Hudiksvall 2 50 Nässjö 2 50 Västerås 12 60

Jönköping 9 54 Norrköping 9 54 Västerhaninge 2 50

Kalmar 4 52 Norrtälje 2 50 Vastervik 3 51

Karlshamn 2 50 Nyköping 3 51 Vaxjo 7 56

Karlskoga 3 51 Örebro 11 55 Visby 3 51

Karlskrona 4 52 Örnsköldsvik 3 51 Ystad 2 50

Karlstad 7 56 Oskarshamn 2 50

Katrineholm 3 51 Östersund 5 50

Aggregates

Total Towns/Cities 79

Total test locations 550

Total tests/operator/scenario 5067

Page 32: STRAIGHTFORWARD | TRUSTED | INTELLIGENTa2.mndcdn.com/image/upload/oy3yvacoqjbf6bzvr2wk.pdf · Straightforward | Trusted | Intelligent ... optimisation, quality benchmarking ... Apple

©Omnitele Ltd. 2012 32 Competitive Network Benchmark for Telenor Sverige AB

annexes | cities and areas

City per Area division of scoreboard calculation is summarised below

Western Sweden Southern Sweden Central-Eastern

Sweden Stockholm

Metropolitan Area Northern Sweden

Alingsås Ängelholm Eskilstuna Åkersberga Boden

Borås Eslöv Falköping Boo Borlänge

Göteborg Falkenberg Karlskoga Lidingö Enköping

Jönköping Halmstad Karlstad Märsta Falun

Kungälv Hässleholm Katrineholm Norrtälje Gävle

Kungsbacka Helsingborg Köping Södertälje Härnösand

Lerum Kalmar Kristinehamn Stockholm Hudiksvall

Ljungby Karlshamn Lidköping Täby Kiruna

Mölnlycke Karlskrona Linköping Tumba Luleå

Nässjö Kristianstad Mariestad UpplandsVäsby Örnsköldsvik

Trollhättan Landskrona Motala Vallentuna Östersund

Uddevalla Lund Norrköping Västerhaninge Piteå

Vänersborg Malmö Nyköping Sandviken

Värnamo Oskarshamn Örebro Skellefteå

Trelleborg Skövde Sundsvall

Varberg Uppsala Umeå

Västervik Västerås

Växjö Visby

Ystad

Page 33: STRAIGHTFORWARD | TRUSTED | INTELLIGENTa2.mndcdn.com/image/upload/oy3yvacoqjbf6bzvr2wk.pdf · Straightforward | Trusted | Intelligent ... optimisation, quality benchmarking ... Apple

Straightforward | Trusted | Intelligent

Want to know more?

Visit www.omnitele.com for news,

references, and events and check out our

blog for the latest Omnitele information:

Read our Blog

Find out about events

Check the latest news

View customer references

Read market commentaries