rfp - network benchmarking services for 36 towns

7/28/2019 RFP - Network Benchmarking Services for 36 Towns

1/12

CONFIDENTIAL Page 1 of 12

Request for Proposal

Network Benchmarking Services


2/12


2 AIRTELs Network Benchmarking RFP Scope

Scope of Service

To define End to End requirement for conducting Competitive NetworkBenchmarking and Report generation for 36 Towns of Airtel.

Detailed Methodology:

Airtel aims to get the 36 Towns benchmarked once in a Quarter.The lists of 36 towns are:

SI No. List of 36 Towns

1 Durgapur

2 Bhuwneshwar

3 Patna

4 Ranchi

5 Guwahati

6 Aizwal

7 Nagpur

8 Nasik

9 Surat

10 Baroda

11 Indore

12 Bhopal

13 Jodhpur

14 Udaipur


3/12


15 Kota

16 Mysore

17 Hubli

18 Manglore

19 Trichy

20 Madurai

21 Pondy

22 Trivendrum

23 Vijayawada

24 Vizag

25 Warangal

26 Agra

27 Dehradun

28 Meerut

29 Kanpur

30 Allahabad

31 Amritsar

32 Jalandhar

33 Patiala

34 Shimla

35 Srinagar

36 Jammu

The Benchmarking partner will have to cover both Voice and Data Services

for 2G and 2.5G. Also, if Airtel launches any new technology (like: 3G, 3.5Getc) in these town then Benchmarking Partner should be able to capture the


4/12


data as per the Airtel requirements. The drive test tool should be able tocapture the above requirements.

2.1 Benchmarking Spread

The competitive Network Benchmarking should cover minimum 04 GSMoperators and 02 CDMA operators.

As per the requirement of the Airtel the operator (GSM as well as CDMA)can be increased.

The list of operators to be benchmarked in each city will be given by Bharti.

2.2 TOOL Requirement

The Competitive Network Benchmarking Drive Test tool should be state ofthe art tool with minimum 12 Mobile handsets measurement capability. The12 handsets are needed to do the Voice and the Data measurementssimultaneously in any given city. Benchmarking Drive test tool should beable to test the EVDO services of at-least 2 CDMA operators along withother tests. Benchmarking Drive test tool should be able to test the latesttechnology available in the market for all the operators. Also the tool shouldbe updated time to time with the new technologies introduced by theoperators and Airtel will be able to get the comparable data.

Apart from the 12 Handsets, the Receive level information of all the

operators to be taken with the help of scanner integrated as part of theBenchmarking tool.

The tools should be able to give the Voice Quality in MOS values usingPESQ or SQUAD LQ. If the algorithm is changed by ITU from PESQ andnew ITU standards introduced in the market than The Partner shouldupgrade the tool accordingly.

The tool should be able to perform the entire data test including emails,WAP test, FTP, HTTP, PDP, GPRS attach/detach, SMS, RTT and MMStests.

The tool should be able to perform all the Voice and Data tests withBlackberry handsets.

The Benchmarking Partner needs to arrange their own FTP servers for alldata tests.

The Landlines required for measuring individual operator performance ifneed be can be provided by Airtel or Partner has the flexibility to do this fromhis own. The partner while submitting the proposal should clearly mentionthe scope of Airtel and the partners for each of the activity.


5/12


2.3 SIM CARDS

The Partner is responsible to procure all the operators SIM/EVDO cards.The Partner has the flexibility of giving quote for either arranging the AIRTELNetwork SIM cards of its own or telling Airtel to procure and give. The final

price quote can be given with and without the AIRTEL SIM Card options.Airtel will decide that which SIM cards with which scheme (Pre paid or Postpaid) needs to be used for the drive. The Partner needs to submit all theoptions to Airtel with all possible schemes available in the market for all theoperators.

2.4 Drive Test Routes

The Drive Test routes will be given by the Airtel team at each city. The

minimum drive test area to be covered in any town out of 36 Towns shouldbe 600 Kms.

The kilometers to be driven in each of the city in accordance to the rangegiven above will be defined by Airtel.

The Drive Test routes will cover Dense Urban, Urban, Rural, VIP areas /routes, important roads / express way etc.

The Drive Test routes will be divided in to different clusters / region in orderfor the partner to report the KPIs cluster / region wise and also as a whole

city.

The same set of tests (used in benchmarking drive) needs to be done fromAirport (International & Domestic) to City center and vice versa also withdifferent citys roaming SIMs of Airtel. Airtel will guide to the BenchmarkingPartner that which Circles Roaming SIMs need to be used during thistesting.

2.5 In-building Tests

Since there are important commercial complexes, Hotels, Shopping mall etc,

depending upon the cities, minimum 25 to 75 buildings to be visited andbenchmarked. In each building individual floor to be walk tested and resultsto be reported floor wise and building wise.

The list of building will be given by the Airtel Team. The partner to arrangefor all permissions to do the walk test and Airtel will fully support for takingpermissions.

The pricing of this In-building test to be quoted separately.

2.6 Measurement Period


6/12


The Drive test measurement period should be from 9 am to 7pm. If theNetwork Busy Hour of the city extends till 9 pm and then the Drive test teamwill be allowed to drive till 9 pm. On a city to city basis, the Drive time to beagreed with the Bharti Team.

If any Operators network test behaviour is abnormal then it is the

Benchmarking Partner responsibility to check the reason for the same. Ifthat is because of temporary network issues (Like: outage of sites, etc) thandrive will not be valid and The Partner needs to do drive again of that route.

2.7 Measurement Test

Airtel requires a comprehensive list of measurements to be provided.

The entire town will be divided in to different clusters as mentioned in thetable in section 2.7.2. For each cluster and at town level the KPI Test resultsto be furnished.

The detailed list of test and the KPIs are mentioned in the following sections.

2.7.1 Voice test:

The voice test should be based on Long calls and short calls. The long callsto be used for obtaining Voice Quality (MOS) and the short calls todetermine Voice Accessibility.

In a normal drive test mode, 2 mts to 3 mts call sequence to be used todetermine the Voice KPIs along the drive test route. If the call drops or failed

then with a minimum pause immediately the call to be initiated.

At strategic locations where the call volumes are expected to be huge, shortcalls to be made to determine how well the accessibility of the networkwhere the customer demand is maximum.

The list of Voice test to be reported cluster wise is given below:

1. Call Setup time

2. Call Setup Success Rate

3. Call Completion Rate (Drop Call Rate)

4. Voice Quality in MOS (Uplink and Downlink separately)

5. RX Level Sample distribution in 04 levels:

RX level samples >-70dbm (GSM 1800 band) and >-75dbm (GSM900 band)

RX level samples -85dbm (GSM 1800 band) and -85dbm

RX level samples -92dbm


7/12


2.7.2 Data Test

The following data test to be done continually in sequential manner (Oneafter another):

1. Attach

2. PDP Activation

3. FTP Upload (100 kbps file)

4. FTP Download (500 Kbps file)

5. HTTP Browsing

6. EMAIL Send and Receive

Apart from the above tests during the drive, at strategic locations across thecity, additional test to be carried out:

1. WAP Test (Operator Home Page download)

2. WAP Test (Downloading picture or ringtone etc from the operator homepage)

3. EMAIL send and receive

4. FTP Download (1 Mb file)

5. SMS

6. MMS

7. RTT (for all AIRTEL APN Servers)

Minimum 5 Strategic locations need to be captured in each town and thelocation will be provided by AIRTEL.

Any time as per the requirement new Data test can be added by AIRTEL

Based on the drive and static test, the final report will be generated whichwill have all the following KPIs:

1. Attach

- Attach Time

- Success Rate2. PDP Activation

- Activation Time

- Success Rate

3. FTP UP link Test

- Average Throughput speed

- Success Rate

4. FTP Downlink Test


- Success Rate

5. HTTP Browsing Test


8/12



- Success Rate

6. Email Send and Receive (with and without blackberry handsets)

- Average time to send

- Average time to receive

- Success rate for sending and receiving

7. WAP Browsing (Operator Home Page)

- Average time taken to download the page

- Success Rate

8. SMS

- Average Time taken to send

- Average time taken to receive

- Success Rate

9. MMS

- Average Time taken to send- Average time taken to receive

- Success Rate

10. RTT (For all AIRTEL APN Servers)

- Minimum RTT

- Average RTT

- Maximum RTT

- Ping Success Rate

2.8 Benchmarks

For each of the mentioned Voice and Data Test, Domestic and Internationalbenchmarks to be provided. The international benchmarks to be based uponthe Drive Test conducted in various international networks. Minimum samplesize should be 50 operators world wide.

Similar to the International benchmarks, every test to be benchmarked withDomestic Benchmarks. The Domestic benchmarks can be based on thecurrent drive test with all the competitors value plotted.

In addition to the individual test Benchmarks, the partner should provide anoverall indicator for Voice and Data separately and benchmarked withInternational and Domestic operators.

2.9 Final Benchmarking Report

The final Benchmarking report will have broadly the following sections:

1. Executive Summary

- The overall snap shot of the town in Voice and Data Test


9/12


- Detailed description of the overall assessment of the Network

Quality viz viz with the competitors

- Clearly mention whether the Network Quality degradation

was observed across the city or only in specific pockets.

2. Business Strengths and Weaknesses

- Airtel stand over the competition

- Airtel Strengths and weakness over the competition

- Reasons for weaknesses

- Improvement recommendations

- Spectrum efficiency

3. Benchmarking methodology

- Detailed explanation on the Benchmarking methodology

- Type of equipment used- Type of test conducted, volumes of calls made and tested etc

4. Routing details

- Route definitions

- Plots indicating clearly the route drive tested

- Total kilometers driven cluster wise and at town level

5. KPI Benchmarks definitions and representation

- International Benchmarks (Best, Median, Worst)

- Domestic Benchmarks ((Best, Median, Worst)=

- Individual KPI Definitions and detailed measurement

methodology

6. Voice and Data KPIs results town level and cluster level

The city divided in to many clusters. The KPI / Test to be presented at Cityand individual cluster / Zone level.

- Detailed breakup of each KPI at town and cluster level with alldomestic (All competitive networks) and internationalbenchmarks.

- One consolidated cluster level report putting together all clustersfor each of Voice and Data KPI.

In the case of Data Test, the strategic locations test results to be published interms of individual location level and overall along with other Drive test.

7. In-building Test Results


10/12


Voice and Data test results per building to be submitted. The report shouldhave floor wise Network Quality KPIs (Preferably all the floors of the buildingshown separately in one single table with all Voice and Data KPIs) and thenconsolidated at the Building level.

8. RCA for poor performance and further improvement

Recommendations

Every KPI to be analyzed at Cluster and Town level and the reasons forthe poor performance of the KPIs, to be mentioned very clearly.

e.g. a) Individual drop call reasons with serving cell id to be mentionedclearly in the backup sheet

b) If the call accessibility failed then mention clearly for each of the call

the reasons for not able to access the networkc) If the FTP Download failed then mention clearly the reasons

for the failed download.

d) Layer 3 message analysis is also required for all the KPI degradations.

e) If there is any difference observed in the drive results of the kit (used atthe time of testing) and the other kits results than vendor is responsible todo the detailed audit and submit the audit report to Airtel with properjustification.

f) Detailed cell level analysis/comparison (for all the operators) is alsorequired to identify that how many cells are covering the derived routeand where new site requirement or optimization in existing sites are

required to improve the network quality.

g) As per the guidelines from Airtel, Benchmarking tracker needs to becirculated by the benchmarking Partner on daily basis to theconcern/Authorized persons of Airtel. Airtel will provide the list of emailIds of the authorized persons.

9. Charts / Graphs and Tables (Cluster and Town Level)

Every Test results to be mentioned / displayed against the localcompetitors and the international benchmarks in terms of charts, graphs

and tables.

Rx lev and MOS plots with details of calls failed and dropped to beclearly mentioned

The 04 levels of RX lev samples mentioned in the Voice Test inSec.2.7.1 to be clearly mentioned in all the cluster and town level plots.

Spectrum efficiency calculations cluster wise and Town wise.

Cluster level and Town level Cell density plots.


11/12


Separate MCQI (Messaging connection quality index) to be including inthe benchmarking report with the weightage of SMS (80%) & MMS(20%). The weightage can be changed by AIRTEL as per therequirement.

Apart from the detailed charts and table, the GPRS and EDGE Data

Usage to be clearly given in terms of % of samples during the Drive test.

The GPRS and EDGE codec usage to be given in terms of graphs /charts for individual cluster level and Town level along with detailedobservations and recommendations.

10. Consolidated Report:

One Consolidated report of all the 36 Towns to be provided for every

circle comparing all the Towns. This should give a snapshot of AirtelTowns standing against each other (In all the KPIs / Test) against itsown and the competition. After finalization of the reports structure, itcan be changed by AIRTEL as per the new requirements.

All the towns should be derived in breakup of clusters and after finishingof every cluster the benchmarking partner should be responsible tosubmit the detail report of the derived cluster with in 2 days.

11. KPI Trend per cluster and Town

The Benchmarking is planned every once in a Quarter for all the 36

Towns. From 2nd report onwards, KPI trending should be given in theform of Graphs / Plots to see how the KPI has performed from the lastsubmitted report. This should continue for all the subsequent reports.

From 3rd report onwards, 1st Benchmarking report will become thebaseline and other will be used for trending.

2.10 Log file of Benchmarking

The log file generated during Drive Test to be submitted to Bharti along with

details of all the Static locations measurements and In-building measurements.

2.11 Pricing

The Pricing to be given for doing the Benchmarking 04 times in a year as wellas twice a year in all the 36 Towns.

The Pricing of the Drive Test and the Indoor Measurements to

be quoted in 02 separate sections namely:

1. Drive Test + Indoor measurements cost


12/12


2. Drive Test cost and Indoor Measurement cost separately

2.12 Penalty for Non Compliance and not meeting the Timelines

The contract is for completing all the 36 Towns every Quarter and reportsubmission to be done within 05 days of completion of each of the city.

For Non Compliance on the Timelines, the partner will pay the followingpenalties:

a. Benchmarking start time delayed by 03 to 05 days

i. 5% of the Total contract valueb. Benchmarking start time delayed by >05 09 days

- 15% of the Total contract value

d. Benchmarking start time delayed by >15 days

- 20% of the Total contract value

The same is applicable for delayed reports also.

If there is any problem (like: any data or voice related services are

deactivated in any operator) occurred during the drive test, than also allthe penalty clause will be applicable.

After completion of drive, if it is found that the report submitted by thebenchmarking partner is not giving the right picture than as a penalty thebenchmarking re-drive needs to be done by the benchmarking Partner.

2.13 RFP Response

A sample benchmarking report to be submitted with all the mentioned

requirements while giving a response to the proposal.

Detailed activity list with clear responsibilities and timelines to be mentioned.

Detailed methodology with project charter to be presented on how theBenchmarking will be done covering all the 36 Towns every Quarter.

Capabilities and references on carrying out this kind of comprehensivebenchmarking.

rfp - network benchmarking services for 36 towns

Documents