machine learning based handover management for lte ...€¦ · most likely o er the user the...

UNIVERSIDADE DO RIO GRANDE DO NORTEFEDERAL

Universidade Federal do Rio Grande do NorteCentro de Tecnologia

Programa de Pos-Graduacao em Engenharia Eletrica e deComputacao

Machine Learning Based HandoverManagement for LTE Networks with Coverage

Holes

Tarciana Cabral de Brito Guerra

Advisor: Vicente A. de Sousa Jr.

Master’s Degree Dissertation presented tothe Electrical and Computer EngineeringPost-Graduation Program (PPgEEC)(concentration area: TelecommunicationsSystems) as part of the requirements toobtain the title of Master in Sciences.

Número de ordem PPgEEC: M542Natal, RN, December 2018

Guerra, Tarciana Cabral de Brito. Machine learning based handover management for LTE Networkswith coverage Holes / Tarciana Cabral de Brito Guerra. - 2019. 78 f.: il.

Dissertação (mestrado) - Universidade Federal do Rio Grandedo Norte, Centro de Tecnologia, Programa de Pós-Graduação emEngenharia Elétrica e de Computação. Natal, RN, 2018. Orientador: Sousa Junior, Vicente Angelo de.

1. Handover - Dissertação. 2. LTE - Dissertação. 3. CoverageHoles - Dissertação. 4. Machine Learning - Dissertação. 5. ns-3- Dissertação. I. Sousa Junior, Vicente Angelo de. II. Título.

RN/UF/BCZM CDU 621.395

Universidade Federal do Rio Grande do Norte - UFRNSistema de Bibliotecas - SISBI

Catalogação de Publicação na Fonte. UFRN - Biblioteca Central Zila Mamede

Elaborado por Ana Cristina Cavalcanti Tinôco - CRB-15/262

Resumo

Antigos paradigmas têm sido adaptados para atender à crescente demanda por acessosem fio à internet de banda larga. Um deles, o Hierarchical Cell Structure (HCS), quejá está sendo aplicado no LTE-A e é considerado imprescindível no 5G, consiste nainstalação de diversos tipos de small cells, criando áreas de cobertura sobrepostas com astradicionais macrocélulas. Devido à sua baixa potência de transmissão e a suas estaçõesrádio base estarem instaladas em uma altura menor do que algumas construções, estandoocasionalmente internas a elas, as small cells são severamente afetadas pelos obstáculospróximos, tornando a Qualidade de Serviço (QoS) percebida pelos usuários sujeita avariações bruscas. Dado que os algoritmos clássicos de gerenciamento de mobilidadenão conseguem prever essas flutuações na QoS, os mesmos se tornam ineficientes em taiscenários. Considerando a quantidade de informação sobre o desempenho das redes queestá atualmente disponível e a evolução da capacidade de processamento em tempo real,um aperfeiçoamento das funcionalidades do LTE por meio da utilização de algoritmosbaseados em aprendizado de máquina faz-se possível. Este trabalho propõe e avaliao desempenho de uma abordagem de handover baseada em aprendizado de máquinaem um ambiente com a presença de obstáculos à propagação, simulado por meio dosoftware ns-3. As técnicas de aprendizado aqui apresentadas conseguem aprender pormeio de experiências passadas, sendo capazes de escolher qual eNB mais provavelmenteoferecerá ao usuário a melhor QoS a longo prazo, mesmo em condições de propagaçãoseveras. A avaliação do desempenho constata que os esquemas propostos beneficiamsubstancialmente a QoS dos usuários em determinadas circunstâncias.

Palavras-chave: Handover, LTE, falhas de cobertura, aprendizado de máquina, ns-3.

Abstract

Legacy strategies have been adapted to fulfill the increasing demand for wirelessbroadband internet access. One of them, the Hierarchical Cell Structure (HCS), thatis already in use in LTE-A and it is considered essential for the 5G, consists in thedeployment of several types small cells under the umbrella of macrocells, creatingoverlaid coverages. Due to their low power and bellow-rooftop-level, sometimes indoorbase stations, the small cells are severely affected by the surrounding obstacles, makingthe perceived Quality of Service (QoS) of the users subject to fast variations, thusrendering ineffective the classical approaches to mobility management, that are unableto predict those severe fading situations (coverage holes). Considering the amountof available information on the network performance and the evolution of real-timeprocessing capabilities, the enhancement of LTE functionalities such as the handover,by means of machine learning algorithms became possible. This work proposes andevaluates the performance of a machine learning based approach to handover in scenarioswith the presence of signal-blocking obstacles, simulated with the software ns-3. Ourmachines learn from experience and they are, therefore, able to choose the eNB that willmost likely offer the user the highest long term QoS after the handover procedure, evenin severe propagation conditions. The performance evaluation shows that the proposedschemes substantially improve the users’ QoS in certain circumstances.

Keywords: Handover, LTE, coverage holes, machine learning, ns-3.

Contents

Contents i

List of Figures iii

List of Tables v

List of Acronyms vi

1 Introduction 11.1 Handover Procedure in LTE . . . . . . . . . . . . . . . . . . . . . . . . 3

1.1.1 Long Term Evolution (LTE) . . . . . . . . . . . . . . . . . . . . 31.1.2 X2-Based Handover . . . . . . . . . . . . . . . . . . . . . . . . 41.1.3 Handover Strategies . . . . . . . . . . . . . . . . . . . . . . . . 5

1.2 Related Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81.3 Dissertation’s Organization and Research Questions . . . . . . . . . . . . 10

2 System Model 112.1 Simulation Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112.2 Deterministic Handover . . . . . . . . . . . . . . . . . . . . . . . . . . . 142.3 Random Handover . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162.4 Handover Parameters and Simulation Campaigns . . . . . . . . . . . . . 162.5 QoS metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

3 Proposed Handover Strategies 193.1 Handover Frameworks’ Schemes . . . . . . . . . . . . . . . . . . . . . . 193.2 Real Life Applicability of the Handover Frameworks . . . . . . . . . . . 24

4 Machine Learning Techniques 264.1 Theoretical Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

4.1.1 Artificial Neural Networks (ANNs) . . . . . . . . . . . . . . . . 284.1.2 K-Nearest Neighbors (KNNs) . . . . . . . . . . . . . . . . . . . 324.1.3 Support Vector Machines (SVMs) . . . . . . . . . . . . . . . . . 344.1.4 Decision Trees (DTs) . . . . . . . . . . . . . . . . . . . . . . . . 384.1.5 Random Forests (RFs) . . . . . . . . . . . . . . . . . . . . . . . 39

4.2 Machine Learning Implementation . . . . . . . . . . . . . . . . . . . . . 394.2.1 ANNs Implementation . . . . . . . . . . . . . . . . . . . . . . . 414.2.2 KNN Implementation . . . . . . . . . . . . . . . . . . . . . . . . 41

i

4.2.3 SVM Implementation . . . . . . . . . . . . . . . . . . . . . . . 444.2.4 RF Implementation . . . . . . . . . . . . . . . . . . . . . . . . . 464.2.5 Training Configuration . . . . . . . . . . . . . . . . . . . . . . . 47

5 Evaluation of Handover Frameworks 495.1 Results for Scenario 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 495.2 Results for Scenario 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 535.3 Preliminary Conclusions and Chosen Frameworks . . . . . . . . . . . . . 56

6 Handover with Machine Learning Techniques 576.1 Results for Scenario 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 576.2 Results for Scenario 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 576.3 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

7 Conclusions and Future Perspectives 617.1 Discussion about the Research Questions . . . . . . . . . . . . . . . . . 617.2 Future Perspectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 627.3 Academic Productions . . . . . . . . . . . . . . . . . . . . . . . . . . . 627.4 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

Bibliography 64

List of Figures

1.1 Hierarchical Cell Structure (HCS). . . . . . . . . . . . . . . . . . . . . . 21.2 Overall E-UTRAN architecture. . . . . . . . . . . . . . . . . . . . . . . 51.3 X2-based handover (HO) procedure. . . . . . . . . . . . . . . . . . . . . 61.4 Event A3 triggered report condition. Adapted from [Sesia et al. 2011]. . . 71.5 State machine of the ns3::A2A4RsrqHandoverAlgorithm. Adapted

from (ns-3, 2017). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

2.1 The simulation setup. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122.2 REM of eNB2 in Scenario 1. . . . . . . . . . . . . . . . . . . . . . . . . 132.3 REM of eNB2 in Scenario 2. . . . . . . . . . . . . . . . . . . . . . . . . 132.4 Logic to trigger deterministic handover. Adapted from (Ali et al., 2015). . 152.5 Completed downloads in Scenario 1 for deterministic handover. . . . . . 172.6 Completed downloads in Scenario 2 (Scenario 1 with shadowing). . . . . 182.7 Percentage of downloads possible to complete. . . . . . . . . . . . . . . 18

3.1 Block diagram of the Frameworks 1 and 2. . . . . . . . . . . . . . . . . . 203.2 Scheme of the Framework 3. . . . . . . . . . . . . . . . . . . . . . . . . 213.3 Scheme of the Framework 4. . . . . . . . . . . . . . . . . . . . . . . . . 21

4.1 Correlations between inputs and the best target for Scenario 1. . . . . . . 284.2 Correlation between inputs and the best target for Scenario 2. . . . . . . . 294.3 The mathematical model of a neuron in a feed-forward MLP, where φ

represents the neuron’s activation function, wi, j is the weight on the linkfrom unit i to this unit and ai is the output of unit i, that belongs to theprevious layer. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

4.4 A MLP network with 3 hidden layers of 2 neurons each. For simplicity,the bias and their associated weights were omitted. . . . . . . . . . . . . 30

4.5 Example of application of KNN. . . . . . . . . . . . . . . . . . . . . . . 334.6 Example of support vectors and hyperplane for a linearly separable binary

classification. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 344.7 Example of a non-linearly separable problem in one dimension that is

linearly separable in two dimensions. . . . . . . . . . . . . . . . . . . . . 364.8 Example of a DT for a classification problem. . . . . . . . . . . . . . . . 384.9 k-fold cross-validation for k = 5 and a training set of 4 000 examples. . . 404.10 Learning curves for the network of the Framework 3 in Scenario 1. . . . . 474.11 Learning curves for the network of the Framework 3 in Scenario 2. . . . . 48

iii

5.1 Score of the handover frameworks in Scenario 1. . . . . . . . . . . . . . 505.2 Percentage of completed downloads in Scenario 1. . . . . . . . . . . . . 515.3 Average download time in Scenario 1. . . . . . . . . . . . . . . . . . . . 525.4 Average throughput in Scenario 1. . . . . . . . . . . . . . . . . . . . . . 525.5 Score of the handover frameworks in Scenario 2. . . . . . . . . . . . . . 545.6 Percentage of completed downloads in Scenario 2. . . . . . . . . . . . . 555.7 Average download time in Scenario 2. . . . . . . . . . . . . . . . . . . . 555.8 Average throughput in Scenario 2. . . . . . . . . . . . . . . . . . . . . . 56

6.1 Percentage of completed downloads in Scenario 1. . . . . . . . . . . . . 586.2 Average throughput in Scenario 1. . . . . . . . . . . . . . . . . . . . . . 586.3 Percentage of completed downloads in Scenario 2. . . . . . . . . . . . . 596.4 Average throughput in Scenario 2. . . . . . . . . . . . . . . . . . . . . . 60

List of Tables

1.1 LTE handover events between cells of the same RAT. . . . . . . . . . . . 6

2.1 Simulation parameters based on (Ali et al., 2016). . . . . . . . . . . . . . 152.2 Handover algorithms parameters. . . . . . . . . . . . . . . . . . . . . . . 16

3.1 Throughput levels. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

4.1 Parameters of the ANNs. . . . . . . . . . . . . . . . . . . . . . . . . . . 414.2 ANN parameters chosen through grid search for Scenario 1. . . . . . . . 424.3 ANN parameters chosen through grid search for Scenario 2. . . . . . . . 424.5 K-Nearest Neighbors (KNN) parameters chosen through grid search for

Scenario 1. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 434.4 Parameters of the KNNs . . . . . . . . . . . . . . . . . . . . . . . . . . 434.6 KNN parameters chosen through grid search for Scenario 2. . . . . . . . 444.7 Parameters of the SVMs . . . . . . . . . . . . . . . . . . . . . . . . . . 444.8 Support Vector Machine (SVM) parameters chosen through grid search

for Scenario 1. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 454.9 SVM parameters chosen through grid search for Scenario 2. . . . . . . . 454.10 Parameters of the RFs. . . . . . . . . . . . . . . . . . . . . . . . . . . . 464.11 Random Forest (RF) parameters chosen through grid search for

Scenario 1. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 464.12 RF parameters chosen through grid search for Scenario 2. . . . . . . . . 47

v

List of Acronyms

3GPP 3rd Generation Partnership Project

ANN Artificial Neural Network

DT Decision Tree

eNB Evolved Node B

E-UTRAN Evolved Universal Terrestrial Radio Access Network

HCS Hierarchical Cell Structure

HO Handover

KNN K-Nearest Neighbors

L-BFGS Limited-memory Broyden–Fletcher–Goldfarb–Shanno

LENA LTE-EPC Network Simulator

LTE Long Term Evolution

MIMO Multiple-Input Multiple-Output

MLP Multi-Layer Perceptron

NR New Radio

OFDM Orthogonal Frequency Division Multiplexing

OFDMA Orthogonal Frequency Division Multiple Access

PRB Physical Resource Block

QoE Quality of Experience

QoS Quality of Service

RAT Radio Access Technology

RB Resource Block

REM Radio Environment Map

RF Random Forest

RS Reference Signal

RSRP Reference Signal Reference Power

RSRQ Reference Signal Reference Quality

RSSI Received Signal Strength Indicator

SINR Sinal to Interference plus Noise Ratio

SVM Support Vector Machine

TTI Transmission Time Interval

UDN Ultra Dense Networks

UE User Equipment

Chapter 1

Introduction

The expansion of consumer demand for wireless broadband, driven by the use ofsmart devices, imposes a new challenge on the current telecommunications systems.From 2007, when the first iPhone was introduced, a growing number of users starteddemanding quality data services wherever they are and, usually, at all times. Datavolumes have increased more than 1000-fold from 2006 to 2016 according to (Holmaet al., 2016; Ihalainen et al., 2013). Nevertheless, they are still expected to grow even morewith the augmenting number of price-accessible devices that now use internet connection,creating the need for a paradigm change in order to better attend the consumers’ demands,despite the limited radio resources.

Moreover, this increasing number of users are typically not homogeneously disposedin the coverage area. In urban scenarios, for example, where there is a verticalizationtendency in the construction industry, the massive agglomeration of users in buildingsis common, which leads to the existence of several hotspots (areas with densecellular network usage) throughout a city, challenging the networking planning anddesign (Guerra et al., 2017). Furthermore, users inside constructions are more likelyto access the network than those on the street. Statistically, more than 50% of the voicecalls and more than 70% of the data traffic originate indoors (Chandrasekhar et al., 2008),where the signal from the conventional outdoor macrocells is further degraded.

The Hierarchical Cell Structures (HCSs) is a classical solution to these challenges.An HCS network consists of a multilayer deployment in which there are several types ofaccess nodes, each with different transmission power and coverage area, with the smallercells being under the umbrella of the larger ones, generating an overlaid coverage (Sesiaet al., 2011), such as shown in Figure 1.1. In this network deployment strategy, hotspotssuch as stadiums, train stations or residential buildings, receive a dedicated low-powernode, improving the quality of service as it serves only that specific area (Guerra et al.,2017).

The idea of such type of deployment is itself not new, it has been used since themid-1990s, through the use of different frequencies for small and macrocells (3GPP,2018). Also, it was already supported by the very first release of the Long Term Evolution(LTE) (Dahlman et al., 2016), with the possibility of using a frequency reuse of one tomaximize the utilization of the licensed bandwidth. However, since it can provide bettercapacity and end-user data rates, despite the complexities of nowadays scenarios, it hasbecome increasingly popular in the last years. As such, the HCS has received additional

CHAPTER 1. INTRODUCTION 2

Figure 1.1: Hierarchical Cell Structure (HCS).

features in LTE releases 10 and 11, improving its support (Dahlman et al., 2016).In the fifth generation (5G) radio interface specification of the 3rd Generation

Partnership Project (3GPP), also called New Radio (NR), small cells are considered a keyaspect (Gupta and Jha, 2015). 5G will support, from its start, operation from below 1 GHzto 52.6 GHz (Parkvall et al., 2017). As higher frequencies are more subject to obstaclesattenuation than lower ones, it will be even more necessary to have access nodes as closeas possible to end-users, creating a dense deployment of small cell nodes where there aremore cells than active users (Kamel et al., 2016). This paradigm shift goes beyond HCSand it is called Ultra Dense Networks (UDN).

Despite its benefits, the deployment of small cells also has its complexities, forinstance, the largely unpredictable propagation environment (Ali et al., 2015). Any givenuser might be in close proximity of many cells, therefore handover events are muchmore likely to occur, even for devices that are not moving. This could generate highhandover signaling overhead and the so-called ping-pong effect (Li et al., 2016), thatis the excessive number of handovers, between the same two base stations, that userspositioned in the cells’ borders may have. Additionally, base stations are installed at shortheights, bellow-rooftop-level, and sometimes indoor, making their signal susceptible tobe severely attenuated by high buildings, in the case of microcells, or even by walls andfurniture, for pico and femtocells.

Along with low transmission power and short footprint, those signal obstructionscause uncertainty on whether any given cell might continue to provide satisfactorycoverage to a moving user, even if its current signal strength and quality are good (Aliet al., 2015). For example, a pedestrian user might suddenly move behind an obstacle,suffering from an unpredictable signal outage.

Considering the storage and the processing capacity of the modern networks, thecontextual information that they currently generate (that is expected to increase in the5G) can be useful in order to address this radio management challenge (Ali et al.,


2016; Ihalainen et al., 2013; Gupta and Jha, 2015). Instead of using classical handovertechniques, that only consider the current measurements of the reference signals, it ispossible to use that context information to foresee the mobility pattern of the user. Themain idea is to learn from similar past experiences, in order to choose the cell that is morelikely to offer the user the higher long term Quality of Service (QoS).

Machine Learning, a subset of artificial intelligence, is a viable way to employ thistype of handover management, as its techniques are able to progressively improve theirperformance on a task without being explicitly programmed to do so (Samuel, 1959). Thisis done by using statistical methods to analyze previous data on the same phenomenaand adjust to get better results. Machine learning is currently used on many popularapplications, such as Facebook’s face recognition, virtual personal assistants (like Siriand Alexa), search engines results refining, email spam and malware filtering.

In order to prototype and analyze solutions to the above-mentioned mobilitymanagement problem, this work uses the LTE-EPC Network Simulator (LENA), theLTE module in the ns-3, a free open-source discrete-event network simulator for internetsystems. This module is based on the small cell forum LTE MAC Scheduler interfacespecification, an industrial API, which makes the protocol stack model very similar toactual protocol implementations found in commercial products (Ali et al., 2015). Also, itincludes some essential aspects to this work, such as handover, fractional frequency reuse,support for simulating the buildings in a scenario and coverage holes (Ali et al., 2015).

A scenario based on (Ali et al., 2015) is used to collect new data for training, testingand comparing LTE handover strategies. We propose four hybrid handover frameworksbased on classical handover events, and implement four machine learning techniques todeal with handover decisions.

1.1 Handover Procedure in LTEThis section presents a brief review on LTE architecture and its X2 interface, along

with a detailed explanation of the X2-handover procedure, the events that are responsiblefor triggering that operation, and their tunable parameters.

1.1.1 Long Term Evolution (LTE)The LTE technology is the first generation of cellular systems to work only with

packet-switched applications (Dahlman et al., 2016), and the system that completes thetrend of expansion of service provision beyond voice calls towards a multi-service airinterface (Sesia et al., 2011). First included in Release 8 of the 3GPP specifications,the LTE, acronym that refers to the evolution of 3G radio access technology, is alsoaccompanied by an upgrade on the non-radio aspects, the System Architecture Evolution,forming the Evolved Packet System (Sesia et al., 2011). Together, they represent a radicalstep forward for the wireless industry, aiming to provide highly efficient, low-latency,packet-optimized, and more secure services (Akyildiz et al., 2010).

In order to avoid inter-symbol interference, that typically limits the performance ofhigh-speed systems, to increase the spectral efficiency and to boost the data rates (up


to multi-Gbit/s peak throughput in the latest releases (Dahlman et al., 2016)), the LTEincludes Orthogonal Frequency Division Multiple Access (OFDMA) and Multiple-InputMultiple-Output (MIMO) techniques. At the network layer, it is constituted by an all-IPflat architecture that supports different QoS demands.

Resource allocation in LTE occurs both in time and frequency. On time domain, thereare subframes that contain 14 consecutive Orthogonal Frequency Division Multiplexing(OFDM) symbol times (12, if long preamble is used) each. On frequency domain,multiples of 12 subcarriers of 15 kHz are grouped into a 180 kHz Physical ResourceBlocks (PRBs). The number of PRBs depends on the bandwidth, for example, thereare 100 and 75 PRBs available for bandwidths of 20 MHz and 15 MHz, respectively. Theminimal resource allocation unit is one Resource Block (RB), which corresponds to a halfsubframe in time domain by 180 kHz in frequency domain. On each Transmission TimeInterval (TTI) of 1 ms, the Evolved Node B (eNB) schedules RBs to User Equipments(UEs), and multiple access is achieved by assigning subsets of RBs to individual UEs,therefore, allowing simultaneous transmissions.

1.1.2 X2-Based HandoverThe access network of LTE, called Evolved Universal Terrestrial Radio Access

Network (E-UTRAN), consists of a network of eNBs connected with each other (bymeans of the X2 interface) and to the Core Network, known as Evolved Packet Core(by means of the S1 interface), as illustrated in Figure 1.2. The E-UTRAN is responsiblefor all radio-related functions in the entire system, including radio resource managementand security. It works with a distributed control for regular user traffic (Sesia et al., 2011),which means that the nodes do not need the intervention of a centralized unit to performcertain tasks, such as handovers. Such autonomy is an important aspect of LTE becauseit saves backhaul bandwidth and reduces the delay of procedures.

Although the S1 interface is also capable of performing handovers, for intra-LTEmobility, whether we are dealing with small or macrocells, the handover through theX2 interface (X2-handover) is triggered by default. The whole procedure is directlyperformed between the two eNBs, making the preparation phase quicker. The corenetwork is only informed about it to trigger the path switch, after that the handoveroperation is successfully finished (Sesia et al., 2011).

Since LTE only works with hard handovers, the connection between the UE and thesource eNB is released by the command of the target eNB, before a new connectionhas been established. The data meant for the UE, that was received by the source nodeduring the process, is forwarded to the target node in order to minimize the packet loss.There are two categories of mobility over X2: the seamless handover, that minimizes theinterruption time during the mobility; and the lossless handover, that does not toleratepackage loss at all, but can suffer a brief interruption due to data buffering. The sourceeNB may decide which type of handover to use depending on the service QoS (Sesiaet al., 2011). The X2-handover procedure is illustrated in Figure 1.3.


Figure 1.2: Overall E-UTRAN architecture.

1.1.3 Handover StrategiesAlthough the source eNB can make the handover decision without information

measurements (blind handover) (Sesia et al., 2011), it normally bases itself on channelmeasurements performed by the UE (Herman et al., 2013). The E-UTRAN configures theUE to perform such measurements and send reports when certain conditions are met. Thefollowing parameters can be configure in the UEs: The measurement objects, definingwhich eNBs signals are going to be measured; and the reporting configurations, settingthe criteria (events or periodic) that triggers the reports along with what the UE is expectedto report, i.e. the chosen Reference Signal (RS) (Sesia et al., 2011).

Additionally, the handover decision shown in Figure 1.3 usually follows a certainpre-configured strategy. Each strategy, or algorithm, is composed by one or more eventson the level of a RS of the serving eNB’s signal and its neighbors’. LTE Release 8 definesfive events related to the handover procedure between cells with the same Radio AccessTechnology (RAT), as enumerated in Table 1.1 (3GPP, 2009).

A handover algorithm also needs to define a RS to be used by the events. ARS might either estimate the power of the link between UE and eNB or its quality,for which they are called Reference Signal Reference Power (RSRP) and ReferenceSignal Reference Quality (RSRQ), respectively. While the RSRP is calculated as theaverage received power of a single resource element, the RSRQ is calculated accordingto Equation 1.1 (3GPP, 2010):


Figure 1.3: X2-based handover (HO) procedure.

Table 1.1: LTE handover events between cells of the same RAT.Event Description

A1 Serving cell becomes better than a ThresholdA2 Serving cell becomes worst than a ThresholdA3 Neighbor cell becomes better than primary cell by an OffsetA4 Neighbor cell becomes better than a Threshold

A5Primary cell becomes worse than Threshold1

and neighbor cell becomes better than Threshold2


RS RQ = NPRBRS RPRS S I

, (1.1)

in which NPRB represents the number of PRBs used, and the Received Signal StrengthIndicator (RSSI) is the linear average of the total wide band power, including noise andinterference, measured in the RS symbol.

As indicated in Table 1.1, the main parameters of the events are the Threshold and theOffset. Other parameters to be highlighted are:

• The Time-to-Trigger, a configurable amount of time during which the stipulatedcriteria of an event must be met for the first report to be sent;

• The Hysteresis, that prevents the start or the unconcluded end of the handoverprocedure by adding or subtracting a value from the measured RS;

• The Report Interval, a time period that the UE must wait before sending again thereport if it does not receive any answers from its eNB.

Figure 1.4 illustrates the triggering of the event A3 when a Time-to-Trigger and anOffset are configured.

Figure 1.4: Event A3 triggered report condition. Adapted from [Sesia et al. 2011].

Two classical handover strategies are:

1. The A3RSRP, also called “the strongest cell handover algorithm”, as the namesuggests is based on the event A3 and the RSRP;

2. The A2A4RSRQ, based on the events A2 and A4, and on the RSRQ. Handoveris only triggered if the events A2 and A4 are both activated. This meansthat the serving cell’s RSRQ must be worst than a Threshold1 during at leastTime-to-Trigger1 (event A2), and the neighbor cell’s RSRQ must be better thana Threshold2 during at least Time-to-Trigger2 (event A4). Once the events have


been activated, if more than one neighbor cell meet the conditions of the event A4,the source eNB will choose the one with the best RSRQ to be the target of thehandover procedure. Then, the source eNB will verify if the difference between theneighbor’s RSRQ level and its own is higher than an Offset, provided that such valueis configured. In the positive case, the source eNB will start the handover procedure.Figure 1.5 illustrates the implementation of this procedure on the ns-3 (ns-3, 2017).

Figure 1.5: State machine of the ns3::A2A4RsrqHandoverAlgorithm. Adapted from (ns-3,2017).

1.2 Related WorksThe implementation of the classical LTE handover algorithms in ns-3 is presented

in (Baldo et al., 2013). Their performance in an HCS scenario is tested and compared inthe works (Guerra et al., 2017) and (Chaparro-Marroquín, 2014), being the latest centeredon both rural and urban scenarios, while the first focused on an urban environment withseveral hotspots.

Most of researches on handover algorithms focus on perceiving the impact andoptimizing the aforementioned parameters of the classical algorithms. This is the caseof the article (Kazi and Wainer, 2017), that tried to reduce the number of handoversand handover failures through parameter optimization of the A3RSRP algorithm, in anmacro-small cell scenario. The works (Cardoso et al., 2017; Saeed et al., 2017) also focuson parameter tuning on HCS networks, and both do so by using fuzzy logic.

New handover algorithms based on the events defined in (3GPP, 2009) are introducedin the work (Thakkar et al., 2017) to reduce the excessive change of eNBs that mighthappen to users located in a cell border (the ping-pong effect). The article (Priyadharshini


and Bhuvaneswari, 2016) investigates the influence of the parameters Time-to-Trigger,Hysteresis and Offset of the A3RSRP algorithm in the performance of an LTE network.The work (Malekzadeh and Rezaiee, 2018) also analyzes the impact of those threeparameters (plus the Threshold) in the network throughput using A3RSRP. Moreover,the authors compare both A3RSRP and A2A4RSRQ algorithms for different user speeds.

Machine Learning has been vastly used to improve the networks’ performance,especially in the last few years. The paper (Khunteta and Chavva, 2017) proposes amethod base on Deep Neural Networks to mitigate the link failure caused by unsuccessfulhandovers and congested cells, among other reasons. Aiming to detect network intrusion,the article (Chang et al., 2017) develops a technique based on RF and SVM, while thework (Yin et al., 2017) uses an approach based on Deep Learning.

Moreover, focusing on QoS and Quality of Experience (QoE) prediction by meansof machine learning, we also found a few researches. The article (Abar et al., 2017)uses KNN, Decision Tree (DT), RF and Artificial Neural Network (ANN) to predict theQoE of Software Defined Networks, comparing their performances. The work (Begluket al., 2018) proposes to foresee the users’ QoE for an LTE video streaming using ANNs.In its turn, the paper (Casas et al., 2017) conceives a system based on DTs to predict theQoE of end users of popular smartphone applications.

When considering specifically the mobility management area, the techniques basedon Machine Learning have recently been the object of some contributions. In (Yang, Huand Wang, 2017), a handover mechanism for unmanned aerial vehicles is developed. Thepaper (Yang, Dai and Ding, 2017) proposes a scheme based on SVM to predict the mobileequipment location in an UDN within 5 seconds.

None of the previously mentioned works has proposed machine learning handovermanagement strategies focused on LTE networks in an HCS network. However, thepaper (Ferng and Huang, 2016) proposes a Self-Organizing Networks (SON) to makea handover scheme consisting of preselecting the eNB according to user speed anddemanded QoS. Nevertheless, their solution do not delegate the handover decision toa machine learning technique. On the other hand, the article (Ali et al., 2016) proposesan ANN framework to make such decisions in an LTE network with a coverage hole,scenario presented by (Ali et al., 2015).

In order to deal with the coverage hole scenario (Ali et al., 2015), we proposeda modified version of algorithm introduced in (Ali et al., 2016), and compare itsperformance to the A2A4RSRP algorithm’s. Additionally, we also propose differentlystructured handover frameworks based on ANNs, KNNs, SVMs, and RFs. Those machinelearning frameworks vary in processing demands and scalability, allowing us to evaluatethe cost-effectiveness trade-off of the proposed schemes. Furthermore, another scenario,with more severe propagation conditions than the first one (due to shadowing), is used inthe analysis to better evidence the effects of the proposed schemes in the complexities ofthe current urban environments.


1.3 Dissertation’s Organization and Research QuestionsWith the experiments proposed on this dissertation, we ought to discuss the following

questions:

• Are the traditional handover algorithms suited to deal with a scenario with acoverage hole?

• Can a machine learning based handover algorithm work better than the traditionalone in such scenario?

• Does the shadowing affects the performance of a traditional handover algorithm?How about the performance of a machine learning based algorithm?

• How good is the performance of a machine learning based algorithm in a situationwith both shadowing and coverage holes?

Finally, this dissertation is organized as follows:

• Chapter 2 describes the two target scenarios and the main simulation parameters;• Chapter 3 presents the structure of the proposed handover frameworks;• Chapter 4 discusses the theory and implementation of the machine learning

techniques explored in this work;• Chapter 5 makes a preliminary analysis of the frameworks’ performance

using ANNs;• Chapter 6 uses all the machine learning techniques to make a complete performance

evaluation of the proposed schemes, comparing their performances to the ones of aclassical and a random handover strategy;

• Chapter 7 shows the conclusions and future perspectives of this work.

Chapter 2

System Model

This chapter presents the two simulation scenarios and the challenges they impose onmobility management. Additionally, handover algorithms are presented, along with thesimulation parameters.

2.1 Simulation SetupThe simulation environment, implemented in ns-3 and based on (Ali et al., 2015),

consists of an outdoor environment with 3 eNBs, 3 UEs and an obstacle partiallyobstructing the coverage area of eNB2, as shown in Figure 2.1. All UEs are downloading aTCP file from the remote host and are, initially, attached to the eNB with its correspondingnumber. A TCP file is chosen because of the popularity of this protocol among internetapplications.

We model the hard frequency reuse to overcome the effects of inter-cell interference,i.e., each cell transmits on different sub-bands in a reuse 3 fashion. The algorithm dividesthe bandwidth in a way that RBs from 1 to 8 are assigned to eNB1, from 8 to 15 to eNB2and from 16 to 25 to eNB3.

Moreover, in order to allow the offline machine learning analysis shown in Chapter 3,the UEs are configured by the ns-3 function ReportUeMeasurementsCallback toperiodically report both RSRP and RSRQ measurements of the 3 eNBs every 200 ms,even though the handover procedure is still event-trigged.

With respect to propagation losses, two scenarios were implemented. The firstone, based in (Ali et al., 2015), only considers path loss as its channel model and itis implemented by the ns-3 class OkumuraHataPropagationLossModel. The secondone is a contribution of this work and, in addition to the path loss, it also includesshadowing, being closer to real complex urban environments, that usually have a morehostile propagation environment. Such channel model is implemented by the ns-3 classOhBuildingsPropagationLossModel and the parameter ShadowSigmaOutdoor.

The Radio Environment Maps (REMs) of the eNB2 in Scenario 1 and Scenario 2 aredepicted in Figures 2.2 and 2.3, respectively. By analyzing the Sinal to Interference plusNoise Ratio (SINR) shown on the maps, we can see that the shadowing (Figure 2.3) givesa random aspect to propagation, making it harder to predict.

Each UE starts the simulation in a fixed point, close to its eNB. While UE2 and

CHAPTER 2. SYSTEM MODEL 12

Figure 2.1: The simulation setup.


Figure 2.2: REM of eNB2 in Scenario 1.

Figure 2.3: REM of eNB2 in Scenario 2.


UE3 are stationary, UE1 moves around at 60 km/h in a straight line with a random anglebetween -60◦ and 30◦, according to the function RandomWalk2dMobilityModel of ns-3.Since it is moving away from its serving eNB, UE1 will eventually need to be reallocatedto either eNB2 or eNB3. Each new simulation run picks a different angle, according toa random seed run. This variable controls the pseudo-random number generator in ns-3.In order to work with a large number of mobility patterns, we perform simulations with20 000 seed runs.

As pictured in Figure 2.1, the random angles are divided in three regions. The superiorregion is in the footprint (coverage area) of eNB3, without a coverage hole. Similarly,the inferior region (eNB2 coverage) does not have any signal obstructions. The centralregion should be served by eNB2, however, it is affected by the presence of a buildingwhose dimensions are wide enough to cause an unpredicted outage if the user moves toa location where the line between him and the eNB intercepts the block. Such outageregion can be clearly seen in Figures 2.2 and 2.3.

This situation illustrates the coverage hole problem introduced in Chapter 1, as theUEs in the central region will experience better RS levels from eNB2 than eNB3, but arelikely to enter the outage area moments later. Therefore, it would be more advantageousto connect the users in the coverage hole path directly to eNB3, that could offer a morestable connection during the simulation, in spite of having initially a weaker signal.

With respect to the data traffic, each user downloads a single TCP file (as previouslymentioned) of 15 MB. The file is divided into TCP segments of 1448 bytes and themaximum size of the transmission buffer is 60 KB. After the download, no more datais exchanged between the UE and the eNBs. Since the simulation time is 100 seconds,the users’ main goal is to finish the download within this time.

Other system model parameters are presented in Table 2.1.

2.2 Deterministic HandoverAiming to allow the qualitative evaluation of the performance of different

handover algorithms for each possible target eNB, we have used thens3::A2A4RSRPHandoverAlgorithm, also called the "A2 event triggered deterministichandover algorithm" (Ali et al., 2015). As the name suggests, it uses the events A2 andA4, and the RSRP (see Section 1.1.3). However, the Threshold for event A4 is set to 1(in a scale of quantized levels from 0 to 97 (3GPP, 2009)), so that any possible targeteNB can reach it (unless its signal is fully blocked), therefore making A2 the event thateffectively triggers the handover in this strategy.

Despite of being event-trigged, this algorithm allows for the target eNB to be chosenbeforehand, in a deterministic way, with the parameter targetCellId. When this parameteris set, the handover algorithm ignores the current RSRP levels, and connect directly tothe selected eNB. When targetCellId is set to -1, the algorithm works in the traditionalnon-deterministic way, choosing the strongest neighbor as its target cell.

The logic to trigger the deterministic handover algorithm is shown on Figure 2.4.When the RSRP level of the source eNB falls bellow the configured value for theThreshold (event A2 is triggered), the algorithm verifies if there is any neighbor with


Table 2.1: Simulation parameters based on (Ali et al., 2016).Parameter Value

System bandwidth 5 MHzInter-site distance 500 m

Link adaptation and error model MiErrorModelSimulation area 2000x2000m2

Number of eNBs 3eNBs transmission power 46 dBm

Number of UEs 3Velocity of UE1 60 km/hPath loss model Okumura-Hata

Shadowing Scenario 1 No shadowingShadowing Scenario 2 Lognormal with std. deviation of 8 dBeNB Antenna Height 30 m

Obstacle Height 35 mTraffic Bulk File Transfer

File Size 15 MBSimulation time 100 sec

RSRP level equal or higher than 1. Then, in case there is, it checks if the parametertargetCellId is assigned by the user. If yes, the algorithm will trigger the handover tothis target eNB. If no, the algorithm will trigger the handover for the target eNB with thehighest RSRP level.

Figure 2.4: Logic to trigger deterministic handover. Adapted from (Ali et al., 2015).


2.3 Random HandoverIn order to enrich the performance evaluation of Chapter 6, we have also implemented

a Random Strategy to decide the eNB that will be the target of the handover procedure.This strategy is triggered by the handover strategy A2A4RSRP, just like the proposedframeworks on Chapter 3. However, the choice for the target eNB is delegated to thefunction choices() from the random module of the Python language. This functionchooses randomly between the elements of a predefined set. In our case this set is{eNB2,eNB3} and both elements have equal chance of being selected (the selection ismade according to an uniform distribution).

2.4 Handover Parameters and Simulation CampaignsWe present in Table 2.2 all handover-related parameters, defined according to the

assumptions of (Ali et al., 2015), our benchmark reference.

Table 2.2: Handover algorithms parameters.A2A4RSRP

Event A2 A4RS RSRP RSRP

Threshold 50* 1*Offset - -

Hysteresis 0 dB 0 dBTime-to-trigger 0 ms 0 msReport interval 240 ms 240 ms*Quantized according to (3GPP, 2009)

As each sorted path needs to be tested by all the algorithms, each seed run is called 3times:

• Two for deterministic handover (for eNB 2 and eNB 3);• One for non-deterministic ns3::A2A4RSRPHandoverAlgorithm (the classical

handover algorithm).

This results in a total of 120 000, 60 000 runs for each scenario. All generated dataare feed into an offline simulator to train and evaluate the proposed machine learninghandover strategies (as well as the Random Handover).

2.5 QoS metricsWe choose three QoS metrics to be analyzed:

• QoS1: The percentage of completed downloads;


• QoS2: The download time;• QoS3: The throughput.

In order to better understand the dynamic of the simulations and to verify theassumptions made in Section 2.1, we used the A2 deterministic handover algorithm toillustrate the first QoS metrics in the whole data set. The results are depicted in Figures 2.5to 2.7.

Figure 2.5 shows the percentage of completed downloads in Scenario 1. Both inferiorand superior regions have a very well defined target cell (eNB2 and eNB3, respectively).However, in the central region, there is a high percentage of uncompleted downloads forboth eNBs, showing the effects of the coverage hole in the system. Nevertheless, due tothe position of the obstacle, handover for eNB3 has slightly better results.

Figure 2.5: Completed downloads in Scenario 1 for deterministic handover.

Scenario 2 results, pictured on Figure 2.6, show how the first QoS metric is degradedby the random effects of shadowing. While in the central region of Scenario 1 we haveover 70% of completed downloads for both eNBs, in this new scenario the percentagebarely reaches 40%.

Figure 2.7 shows the percentage of downloads that are completed by one or both targeteNBs. In Scenario 2, there are cases in which the signals from both eNBs are so degradedthat, regardless of the target cell choice, it is not possible to complete the download.Nevertheless, due to its aleatory aspect, the shadowing also acts constructively, as canbe verified by comparing, on Figures 2.5 and 2.6, the results of eNB3 and eNB2 on theinferior and superior regions, respectively. In the previous scenario (Figure 2.5), theseeNBs were not able to finish a single download in those regions. However, in Scenario 2(Figure 2.6) all the eNBs are able to complete at least 7% of the downloads in each region.


Figure 2.6: Completed downloads in Scenario 2 (Scenario 1 with shadowing).

Figure 2.7: Percentage of downloads possible to complete.

Chapter 3

Proposed Handover Strategies

As classified by (Li et al., 2016), we propose a hybrid handover scheme that istriggered by the handover algorithm A2A4RSRP (see Section 2.2), but delegates thehandover decision to a machine learning algorithm. Such type of scheme has theadvantage of being simpler and having less computational cost than a full machinelearning one, while still making more long term QoS-oriented decisions than the classicalalgorithms, that are fully RS-based.

Before detailing our proposed scheme, it is important to define how to choose themore suitable eNB for the handover process. Based on the QoS metrics presentedon Section 2.5, we created three rules that define the best choice of eNB for each sampleon the dataset:

Rule A: The best target is the eNB that is able to complete the user’s download;Rule B: If both eNBs complete the download, the best target is the one that do soin less time;Rule C: If none of the eNBs completes the download, the best target is the one thatoffers the highest throughput during the simulation time.

The rules A and B are based on the work (Ali et al., 2016). The rule C, however, isa contribution of this work and is necessary for a more hostile propagation environment.Since our simulations only last 100 seconds, we will not be able to gather the downloadtime (QoS2) information from the downloads still in progress by then. Therefore, wedecided to use the throughput (QoS3) to compare the services provided by two eNBs thatare unable to finish the download. As presented in the Figure 2.7, there is the possibilitythat none of the eNBs completes the download in Scenario 2, hence, the rule C will beessential to such scenario.

3.1 Handover Frameworks’ SchemesFour handover frameworks, that differ with respect to how the handover decision

is made, are compared. Framework 1, based on (Ali et al., 2016), is illustrated inthe Figure 3.1. It consists of an integrated system of six machines, each one dedicatedto predict a QoS metric for one of the two possible target eNBs (eNB2 and eNB3). Thissystem follows the approach of dividing a complex problem into simpler ones, giving

CHAPTER 3. PROPOSED HANDOVER STRATEGIES 20

each machine a set of tasks that match their capabilities (Haykin, 2009). In order to makeits predictions, each machine on Framework 1 receives from the serving eNB (eNB1) thevalues of the current and immediately previous RSs of all the eNBs, i.e. the RSRP andRSRQ from eNB1, eNB2 and eNB3 at the present time and 200 ms earlier.

Figure 3.1: Block diagram of the Frameworks 1 and 2.

There are three prediction levels in the system. Level A is dedicated to predict if theeNBs will be able to finish the download within the simulation time. Level B predictsthe amount of time that each eNB will take to download the file. Level C (a contributionof this work) has the task of predicting the throughput offered to the UE. After all thepredictions are made, the QoS results are compared and the best target is chosen accordingto the pre-established rules.

We also propose the Framework 2 following the same structure shown in Figure 3.1.The sole distinction between the two models lies on the system inputs, since the machinesin the second framework receive only the RSs of the eNBs whose QoS they are tryingto predict. For example, the machine MA2’s inputs only contain information about theeNB2 signal, while the MA3’s inputs are focused entirely on eNB3.

Both Frameworks 1 and 2, due to their structure, give priority to the decision made bythe machines in the Level A, since they skip the other levels when the Level A indicatesthat only one of the target eNBs finishes the download. This results in QoS1 beingprioritized over QoS2 and QoS3 (for the QoS definitions, see Section 2.5). Thus, theseschemes do not always use all of its machines’ outputs to predict the decisions, sincethey always at least ignore one level, which may cause some irrelevant mistakes when theperformances of the two target eNBs are not very different, as we show in the Chapter 5.

Unlike the first models, the Framework 3, whose structure is shown in Figure 3.2, does


not divide the problem into simple tasks. It uses only one machine that directly predictsthe best target and, to do so, it receives the RSRP and RSRQ from all eNBs, similarly tothe Framework 1.

Figure 3.2: Scheme of the Framework 3.

The last proposed strategy, Framework 4, has the structure shown in Figure 3.3. Thisframework is proposed as a modification of the Framework 2 aiming at reducing itscomputational cost. For that reason, instead of using 6 machines, it uses 4, suppressing theLevel B. Moreover, the Level C outputs are quantized into 6 throughput levels, in orderto make the prediction task less complex (this will be further explained in Chapter 4). Tomark this difference, Level C is called Level Q in this framework.

Figure 3.3: Scheme of the Framework 4.

Since the goal of the simulation is to download a file, each throughput level’sminimum or maximum values corresponds to a portion of the file’s size (Fs = 120Mb).The minimum and maximum throughputs are defined, respectively, as qFs/2 and (q +

1)Fs/2, in which q is the level and goes from 0 to 5. The choice for the number of levelsis based on preliminary experimental observations of our scenarios. The 6 throughputlevels are shown in Table 3.1.


Table 3.1: Throughput levels.Level (q) Minimum throughput (Mbps) Maximum throughput (Mbps)

0 0 0.601 0.60 1.202 1.20 1.803 1.80 2.404 2.40 3.005 3.00 -

Aiming to reduce the lost of performance that might be caused by the quantization,we decided to use the probabilities of the predictions instead of the predicted valuesthemselves. So, the machines in Level A provide us with the probability of having acompleted (event A=1) or uncompleted download (event A=0), that we call PA(a) =

P(A = a). On the other hand, the machines at Level Q provide us the throughput levelprobability, assuming each of the 6 quantized levels shown in Table 3.1. This probabilityis written as PQ(q) = P(Q = q), in which Q is the set of possible levels.

Those probabilities are a valuable tool to verify if the machines are certain about theirdecision or not. For example, a value of PA(1) = 0.55 does not show confidence that theeNB will finish the download. Also, since PA and PQ are entirely correlated, we cancompare the values of these two probabilities in order to know whether they agree or not.For an eNB to finish the file download in 100 s, it needs to have a throughput of at least1.20 Mbps, that is the minimum throughput of the category q = 2. Hence, if both machinesfor a certain eNB are accurate, PA(1) should be equal to P(Q >= 2). When the differencebetween those two values is too high, the Framework 4 knows that at least one of themmust be wrong, therefore the predictions cannot be trusted.

Before the handover decision, in order to verify the reliability of PA and PQ, weestablished the three following rules, that must be followed simultaneously:

• PA(a) > 0.7 or PA(a) < 0.3;• P(Q >= 2) > 0.7 or P(Q >= 2) < 0.3;• |PA(1)−P(Q >= 2)| < 0.2.

The first two rules have the purpose of ensuring that the machine is either certain thatthe eNB will finish the download (PA(a) > 0.7 and P(Q >= 2) > 0.7) or certain it will not(PA(a) < 0.3 and P(Q >= 2) < 0.3). The third rule ensures that the absolute differencebetween PA(1) and P(Q >= 2) is not too elevated, thus, the predictions have some levelof agreement. All of the thresholds were defined in a way to identify the really doubtfulpredictions and were based on experimental observations, but they can be modified tobetter fit on different scenarios without affecting the dynamics of the framework. As canbe seen in Figure 3.3 and we further explain bellow, this identification allows the doubtfulpredictions to receive a different treatment due to their unreliability.

After the predictions are made, and we know which ones are reliable and which arenot, we can follow the structure presented on the bottom half of Figure 3.3. If all theoutputs of the machines are reliable (situation 1), we can calculate an estimate on the


value of the throughput (T Ppred) by multiplying the probabilities of each level with itsrespective minimum value of throughput (T Pmin(q)), as shows Equation (3.1). Then, theFramework 4 chooses the eNB with the highest T Ppred.

T Ppred =

5∑q=0

PQ(q)∗T Pmin(q) (3.1)

If only one of the eNBs’ predictions is reliable (situations 3 and 4), the choice of theFramework 4 is based entirely on the valid prediction. So, if only eNB2’s outputs arereliable for an specific test point, the Framework 4 checks if eNB is likely to finish thedownload. In the positive case, the framework chooses eNB2 because it knows this eNBwill finish the download. In the negative case, the framework chooses eNB3 because itis certain that eNB2 will not be able to fulfill QoS1, but uncertain on whether eNB3 iscapable of doing so.

When none of the predictions are reliable (situation 2), the solution is more complexbecause, since we could not trust any of the predictions individually, they need to be usedtogether. Since the events A and Q are not independent, we can not simply multiply itsprobabilities. So, we propose to use the probability of the intersection between A and Q,with a slightly modification.

Defining PAQ(a,q) = P(A = a ∩ Q = q) as the probability of the intersectionbetween events, we could write two possible formulations, named PAQ1 and PAQ2, asin Equations (3.2) and (3.3).

PAQ1(a,q) = P(A = a)P(Q = q|A = a) (3.2)

PAQ2(a,q) = P(Q = q)P(A = a|Q = q) (3.3)

Since we know that there is only one possible value of a for each q, we canrewrite Equation (3.3) as Equation (3.4).

PAQ1(a,q) =

P(Q = q), for (a = 1 and q >= 2) or (if a = 0 and q < 2);0, otherwise.

(3.4)

After that, we rewrite the Equation (3.2) using the Bayes’ Theorem to find aformulation for P(Q = q|A = a) (the posterior probability of Q = q, given that A = a),which results in Equation (3.5).

PAQ2(a,q) = P(A = a)P(Q = q)P(A = a|Q = q)

P(A = a)(3.5)

Assuming that both machines are 100% accurate, P(A = 1) = P(Q >= 2) and P(A =

0) = P(Q< 2), we replace P(A = a) in the denominator of Equation (3.5) by the appropriateP(Q = q), yielding in the Equation (3.6).


PAQ2(a,q) =

P(A = 1)∗ P(Q=q)

P(Q>=2) , if a = 1 and q >= 2;P(A = 0)∗ P(Q=q)

P(Q<2) , if a = 0 and q < 2;0,otherwise.

(3.6)

Finally, we define PAQ(a,q) as the minimum value between PAQ1(a,q) and PAQ2(a,q).

PAQ(a,q) = min(PAQ1(a,q),PAQ2(a,q)) (3.7)

This formulation is made as a way to penalize the differences between the twoprobabilities. When P(Q >= 2) is higher than P(A = 1), PAQ(1,q) is penalized by afactor P(A = 1)/P(Q >= 2). Finally, we now choose the eNB that is more likely tofinish the download according to PAQ(a,q), as shown in Figure 3.3. This decision makesFramework 4 prioritize the QoS1 (like Frameworks 1 and 2), but only in this branch(situation 4) of the scheme.

3.2 Real Life Applicability of the Handover FrameworksAll the frameworks here introduced have the potential to be applied in real life.

Frameworks 1, 2 and 4 would require a data storage phase, performed offline, in whichthe RSRP and RSRQ (the inputs) of many users would be stored always that a handoveris requested. Moreover, the system would need to make the handover users download asmall file right after the handover to evaluate the QoS1 and QoS2 (only for Frameworks 1and 2) metrics resulting from the eNBs choice. With the same purpose, the users’throughput (QoS3) would also need to be registered for a few seconds after the handover.

After this offline stage, the frameworks would be ready to work online within theeNBs in the control of the handover decisions. Furthermore, if necessary, data for anothertraining phase could be gathered whilst the previously trained frameworks are online,without harming their performance.

Framework 3, likewise, would also need an offline storage phase before being appliedat the eNBs. However, its structure requires the knowledge of the best outputs for all thetraining data, and therefore the resulting QoS metrics for all the possible choices that anUE would have in certain conditions. Since it would be impracticable to have a real userto go to several times through same path and in the exact same propagation conditions,this phase would need to be performed in a software simulation. Such simulation wouldrequire a very careful design to get as close as possible to the real scenario.

Furthermore, the handover performance indicators could be defined for a givensituation. For example, the objective of QoS1 is to evaluate if the goal of the systemis being reached. In our case, that goal is to download a file. However, in other cases, itmight be not to lose the connection during a video call or to receive packages with a delayinferior to maximum delay. QoS2 and QoS3 evaluate how well this objective is reached(or not), hence they can also mean different indicators according to the scenario.

With respect to the real life scalability, Frameworks 2 and 4 are the most suitedframeworks. In their schemes, if a new eNB is added to the system, one machine at each


level must be added and trained, but all the "old machines" will not need to be retrained asthey receive only inputs concerning their eNBs. However, Framework 1 would require allthe machines to be retrained along with the new ones, since they would have new inputs.Framework 3, as the least scalable framework, would require not only another softwaresimulation to train the machine, but also a change in the number of inputs and outputs ofthe machines, changing completely its design.

Nevertheless, Framework 3 is very advantageous in terms of computationalrequirements, since it is composed by only one machine that has the task to decidebetween only two options of an output, i.e., a simple binary classification task.Framework 4 is also very beneficial in this matter, since it has less inputs thanFramework 1 and has less machines than Framework 2, not to mention the fact that it onlyuses classification machines. Finally, the worst framework in this aspect is Framework 1that uses more machines and inputs than the other frameworks.

Chapter 4

Machine Learning Techniques

In this chapter, we briefly present the machine learning techniques ANN, KNN, SVMand RF. We also give details about the implementation of each proposed handoverframework.

4.1 Theoretical ReviewMachine learning techniques are algorithms that have the ability to acquire their own

knowledge by extracting patterns from raw data (Goodfellow et al., 2016). This capabilityallows them to perform tasks that are not complicated to humans, but require a moresubject and intuitive knowledge and, therefore, are not easily described using a set oflogical rules. It can be extremely hard, for example, to define in logical sentences howto recognize a human face or a handwriting in a photo, as it might require an immenseamount of rules that will need to be modified if the objects’ position changes.

The first computer learning program is proposed by Arthur Samuel in (Samuel, 1959).It controls an autonomous player of checkers and is able to improve the game the more itplayed, by studying the human player’s moves and its results in the match, incorporatingthe winning strategies into its program. Despite the early start, it has not been untilrecently that the technology advancements in processing power allowed machine learningalgorithms to became popular.

In this study, we work with machines that use supervised learning. This type oflearning trains the machine with a set of examples composed of input-output pairs (labeledsamples), allowing the algorithm to know the error of its predictions and use it to refine itsprocess. With the so called training set, the machine is expected to extract the knowledgenecessary to map the input into the output (Lima et al., 2016). Then, once the programis adjusted to the training set, the learning process is finished, and the algorithm willno longer modify itself. So, we test the program on novel data examples, the test set,to check the algorithm’s ability to generalize the input-to-output mapping. To be ableto give satisfactory answers on unseen data is considered to be the main challenge ofmachine learning (de Pádua Braga, 2007).

When the machine have good results on the training set and bad results on the testset, we have a problem called overfitting. This problem usually happens when there isnoisy data on the training set or when it is not diverse enough to fully represent the whole

CHAPTER 4. MACHINE LEARNING TECHNIQUES 27

dataset (Coppin, 2010). For instance, in Scenario 1, consider that all the examples ofthe training set belong to the superior region. The machine from Framework 3 wouldlearn that the optimum output is always eNB3, incorrectly predicting all the other cases.According to (Russell and Norvig, 2009), this phenomena becomes more likely as thehypothesis space and the number of input attributes grows, and less likely as the numberof training examples is increased.

With respect to the nature of the output, the learning problems can be classified in twotypes: classification and regression. When the output is one of a finite set of values, theproblem is a classification. If there are only two possible values, we called it a boolean orbinary classification. However, if the output can take infinite values, the learning problemis called a regression. According to (Russell and Norvig, 2009), solving a regressionproblem is finding the conditional expectation or average value of the output, because theprobability of finding exactly the right real-valued number is zero. Therefore, regressionproblems tend to be more challenging than classification ones.

In our frameworks, we have both classifications and regressions. In the level A ofFrameworks 1, 2 and 4, there are only two possible values for the output, as the downloadwill either be completed or not completed, therefore, it is a binary classification. Similarly,in the machine of Framework 3, the output is either eNB2 ou eNB3, consequently, it isalso a binary classification. The level Q of Framework 4 can produce 6 different types ofoutputs, therefore it is what we call a multi-class or categorical classification. The levelsB and C of Frameworks 1 and 2, however, have the download time and the throughput,respectively, as outputs, both of which can assume a continuous range of values, hence,their machines deal with a regression problem.

The datasets can be expressed as shown in the Equation (4.1), where n stands forthe number of samples, and Xn and Yn represent, respectively, the vectors of inputs andoutputs of the sample n.

D = {(X1,Y1), (X2,Y2), ..., (Xn,Yn)} (4.1)

For our case, the inputs, also called features, are the RSRP and RSRQ of the eNBs.In Frameworks 1 and 3, the line vector of inputs has the RS measures of all eNBs, aspresented in the Equation (4.2):

X j = [x1(t), x2(t), x3(t)] (4.2)

where xi refers to the RSRP and RSRQ of the eNB i measured on times t and t− 1, aspresented in the Equation (4.3).

xi(t) = [RS RPi(t),RS RPi(t−1),RS RQi(t),RS RQi(t−1)] (4.3)

The machines from Frameworks 2 and 4, however, use only the inputs of the eNBwhose QoS is been predicted, hence, their inputs can be described as in the Equation (4.4),where j is the target eNB studied.

X j = [x j(t)] (4.4)


The outputs for each machine were specified in the Chapter 3. However, it is importantto emphasize the fact that we are only working with single-output machines, therefore,from now on we will use the notation yn instead of Yn. Moreover, to distinguish the valuepredicted by the system from the real one, the predictions are going to be called pn.

The correlations between inputs and the best target on Scenarios 1 and 2 are picturedin Figures 4.1 and 4.2, respectively. As we can see from the Best Target column, inScenario 1 most inputs are strongly correlated (the correlation value is either close to 1 or-1) to the Best Target, which will make it easier to predict. In Scenario 2, however, all theinputs are only weakly correlated to the output, making us visualize the randomness andindefinition that the shadowing brings to the system, adding, therefore, an extra challengeto the machines.

Figure 4.1: Correlations between inputs and the best target for Scenario 1.

4.1.1 Artificial Neural Networks (ANNs)As defined in (Haykin, 2009),“a neural network is a massively parallel distributed

processor made up of simple processing units that has a natural propensity for storingexperiential knowledge and making it available for use”. Based on the brain, this systemacquires knowledge from the environment through a learning process and stores thisknowledge in the strength of the connections between the units.

A model of one of these units, also called artificial neurons, is depicted inthe Figure 4.3. Each input ai, that can either be some external information passed tothe network or the output of other neuron, is multiplied by a synaptic weight wi, j. Then,the neuron j applies the so called activation function φ on the weighted sum of inputs,producing the output a j, as presented in the Equation (4.5). It will then serve as input toother neurons. The dummy input a0 is called bias and usually is set to 1.


Figure 4.2: Correlation between inputs and the best target for Scenario 2.

a j = φ

n∑i=0

wi, jai

(4.5)

Figure 4.3: The mathematical model of a neuron in a feed-forward MLP, where φrepresents the neuron’s activation function, wi, j is the weight on the link from unit i tothis unit and ai is the output of unit i, that belongs to the previous layer.

The activation function is usually either a hard threshold or a logistic function. Whenthe first one is used, the neuron is called perceptron, when the second one is applied, theneuron receives the name sigmoid perceptron. Both of these nonlinear functions bringan important advantage to the ANN: with them, the entire network is able to represent anonlinear system (Russell and Norvig, 2009).

Furthermore, with respect to its topology, an ANN is generally structured in inputlayer, hidden layers and output layer. While the hidden layers are not mandatory, theymight give an extra processing power to the network, since they act as feature detectors,discovering salient features that characterize the training data (Haykin, 2009).


In this work, we will only use feed-forward Multi-Layer Perceptrons (MLPs), that area kind of neural network that contains one or more hidden layers and have connectionsonly in one direction, forming an acyclic graph. This means that every unit in each layercan only have as inputs the output signals from the immediately preceding layer, hence,this system has no state other than the weights themselves (Russell and Norvig, 2009).

Moreover, according to (Russell and Norvig, 2009), a feed-forward MLP, with a largeenough hidden layer, can represent any continuous function with arbitrary accuracy. If asecond layer is added, even discontinuous functions can be represented. In the Figure 4.4,we can see an example of MLP, this particular structure is chosen to be the network MA2in Framework 2 for Scenario 1.

Figure 4.4: A MLP network with 3 hidden layers of 2 neurons each. For simplicity, thebias and their associated weights were omitted.

However, this type of topology also has disadvantages and the main one resideson a more complicated learning process. As previously stated, the network keeps itsknowledge in the synaptic weights, as a result, its learning phase deals with the task tofind the optimum set of weights that minimize the error between the predicted outputand the real one. Therefore, the learning process can also be viewed as an optimizationproblem.

The weights are usually adjusted iteratively through an algorithm calledback-propagation. First, we initialize them with small random values. Then, we computethe error in the prediction. Let e(k) be the error signal produced at output neuron whenthe signal Xk is applied to the input layer, we could write:

e(k) = p(k)− y(k). (4.6)

The instantaneous error energy of the network (E(k)) will be defined as inthe Equation (4.7).

E(k) =12

[e(k)]2 (4.7)


By using the gradient descent optimization technique, we seek a direction of theweight space that reduces the value of E(k). Therefore, we need to correct the valueof the weight w j,i (that connects the neurons i and j) in the next iteration (k +1) accordingto the Equations (4.8) and (4.9).

w j,i(k + 1) = w(k) +∆w j,i(k) (4.8)

∆w j,i(k) = −η∂E(k)∂wi, j

(4.9)

The η, in the Equation (4.9), stands for the learning step or rate. If j is the outputneuron, this process is very straight forward, since the error is clear. So, ∆w j,i(k) could berewritten as:

∆w j,i(k) = ηδ j(k)p(k), (4.10)

3 j(k) =∑i∈J

w j,i(k)ai(k), (4.11)

φ′(3 j(k)) =∂a j(k)∂3 j(k)

, (4.12)

δ j(k) = e(k)φ′(3 j(k)), (4.13)

where J represents the set of inputs to neuron j and p(k) is the predicted output.The error for hidden neurons, however, is not as evident, since we do not have its

desired response. If neuron j belongs to one of the hidden layers, we need to calculate itscontribution δ j(k) recursively, by making the assumption that the error is divided betweenhidden neurons according to the strength of their connections to the next layer. Hence,we calculate δ j(k) according to Equation (4.14), where I represents the set of units thatreceive the output of neuron j as their inputs.

δ j(k) = φ′(3 j(k))

∑i∈I

δi(k)wi, j(k)

(4.14)

The learning process can last many training epochs. An epoch finishes when the wholetraining set serves as input to the network. Then, the examples are randomly shuffledbefore the new epoch starts. The process will only be interrupted when some stop criterionis achieved. The criterion might be an specified error level (on a subset of the trainingexamples called validation set) or a maximum number of training epochs.

Moreover, the network might be programmed to update its weights only after an epochhas passed. This can be done by using the average of the energy function with respectto the whole weight vector instead of E(k) and wi, j in the Equation (4.9). Accordingto (Haykin, 2009), this method, called batch learning, has the advantage of a moreaccurate estimation, however it requires much more storage than the previous one.

Furthermore, other optimization methods, such Newton and quasi-Newton methodscan be used. In this case, they use the hessian matrix (or an approximation of it, in the


case of a quasi-Newton method) along with the gradient of the error, getting the secondand the first-order information about the error surface and, therefore, usually convergingfaster at the cost of higher computational requirements.

Then, five steps can summarize the training process:

1. Initialization of the weights with small random numbers, usually in the interval(0,1);

2. Propagation of the input signals through the network;3. Back-propagation of the error signals through the network, calculating the

individual contribution of each neuron;4. Adjustment of the weights according to the calculated error for each neuron;5. Go back to step 2, unless the stop criterion is achieved.

4.1.2 K-Nearest Neighbors (KNNs)The KNN is one of the simplest machine learning algorithms (Ledolter, 2013).

It consists of a non-parametric method, which means that instead of extracting abounded number of parameters from the training set, like the ANNs extract theirsynaptic weights, it simply uses the whole set to make its predictions (Russell andNorvig, 2009). Hence, its training phase consists only of storing the training set examplesfor latter use (Ledolter, 2013), which is the reason why this approach can also be calledmemory-based learning (Russell and Norvig, 2009).

The non-parametric approach avoids one of the main disadvantages of the parametricmodels: the lack of flexibility to represent complex functions (Russell and Norvig, 2009).Sometimes, the fixed number of parameters defined by the parametric techniques is notenough to represent properly the output as a function of the input. One example to thissituation is an ANN with less neurons and hidden layers than necessary.

Furthermore, in order to predict the output of a X j, the KNN uses the outputs of thetraining set data points nearest to X j. In the case of a classification problem, the mostpopular class among the neighbors will be chosen as y j. For a regression problem, the y jis chosen to be the mean of the neighbors’ outputs (Russell and Norvig, 2009). The idealnumber of neighbors (parameter k) to be selected for each problem is usually determinedthrough a process called grid search, that we will see in further detail in the Section 4.2.

Figure 4.5 illustrates how the KNN algorithm works. The illustration represents theproblem of classifying a certain data point X j in one of two categories, here representedby the colors green and red. Such classification example (whose structure will be usedas an example for other types of machines yet to be explained) uses only two inputs(x1 and x2), in order to make the visualization simpler, and the parameter k is set to 3.Moreover, the already colored circles represent the training set passed to the machine. Wesee in Figure 4.5, that the KNN find the 3 circles nearest to X j and, based on their color(2 greens and 1 red), the algorithm classifies X j as green.

In order to measure the distance between two points to select the nearest neighbors,several metrics can be applied. Typically, for continuous features, the Euclidean distanceis applied (Ledolter, 2013). However, many other metrics can be used, such as theManhattan and the Chebyshev distances. Euclidean, Manhattan and Chebychev distances


Figure 4.5: Example of application of KNN.

are presented respectively in Equations (4.15) to (4.17), in which S is the resultingdistance, X j is the input vector j and Xi is the input vector i.

S euclidean =

√∑(X j−Xi)2 (4.15)

S manhattan =∑|X j−Xi| (4.16)

S chebychev = max |X j−Xi| (4.17)

Despite its simplicity, the KNN technique tends to work really well in problems withnot a large number of features and plenty of training examples (Russell and Norvig, 2009).Moreover, it demands very small processing power and time during the training phase,


making it a very attractive solution to simple problems. However, it may take slightlymore time than other algorithms to predict a the output of the data point, since almost100% of its processing resides on the predictions and it increases withe the number ofdata points in the training set (Russell and Norvig, 2009). Furthermore, its performanceis sensitive irrelevant features.

4.1.3 Support Vector Machines (SVMs)The SVM is currently the most popular method for an “off-the-shelf” supervised

learning, which means that, when we do not possess any specialized prior knowledgeabout a problem’s domain, the SVM is usually the first supervised learning techniquethat we test (Russell and Norvig, 2009). Because its concept is more complex than theprevious ones, and would probably require a full chapter to be completely detailed, weare going to explain how it works on the simplest of the cases and focus mainly on theidea behind the complex ones.

This technique works based on the understanding that some examples, the so called“support vectors”, are more important than others, therefore paying attention at them canlead to a better generalization (Russell and Norvig, 2009). Let us suppose we have alinearly separable binary classification, like the Figure 4.6 shows. To the SVM method,the examples that are truly relevant (the support vectors) are the ones that are most difficultto classify (Haykin, 2009), i.e. the ones that lie closer to the other class area, as indicatedin the Figure 4.6.

Figure 4.6: Example of support vectors and hyperplane for a linearly separable binaryclassification.

Based on those support vectors, the method constructs an hyperplane as its decisionsurface. The position of this hyperplane is chosen to be the one that maximizes the marginof separation between the examples of different classes (Haykin, 2009). Considering thegreen class as y =−1 and the red class as y = +1, the optimum hyperplane can be described


as in the Equation (4.18), in which, similarly to the ANN formulation, X is an input vector,W is an adjustable weight vector and b is the bias.

WT X + b = 0 (4.18)

Since every circle above the hyperplane will be considered red and the ones bellowit will be designated as green, we can use the function g(Xi), on Equation (4.19), as the“discriminant” vector (the one whose value determines the output).

g(Xi) = WT Xi + b (4.19)

This vector represents an algebraic measure of distance (Haykin, 2009) of an input Xito the hyperplane, as shown in Equations (4.19) and (4.20). In the last equation, yi standsfor the the corresponding output of the input vector Xi.

g(Xi) ≥ 0, for yi = +1g(Xi) < 0, for yi = −1

(4.20)

We can rescale W and b on Equations (4.19) and (4.20) to obtain the Equation (4.21).

g(Xi) ≥ 1, for yi = +1g(Xi) ≤ −1, for yi = −1

(4.21)

Then, we express g(Xi) in terms of: the straight line distance, r, from the hyperplaneto the point Xi; and the euclidean norm of the weight vector(‖W‖), as presented inthe Equation (4.22).

g(Xi) = r‖W‖ (4.22)

Finally, as we know that the support vectors are at the points closest to the margin (i.e.g(Xi) = ±1 at Equation (4.21)), the margin of separation (ρ) between reds and greens canbe described as in Equation (4.23). Therefore, the minimization of ‖W‖ is the conditionto be attained so that ρ reaches its maximum value.

ρ = r− (−r) =2‖W‖

(4.23)

Hence, the optimum values for W and b can be found through theconstrained-optimization problem described in the Equation (4.24).

minimizeW

12

WT W

subject to yi(WT Xi + b) ≥ 1, for i = 1, . . . ,N.(4.24)

For non-linearly separable problems, the SVM uses the so-called kernel trick. Thekernel is a function that changes the feature space to map the data into a higherdimensional space by using of dot products, that require less processing than a regular


high dimensional mapping would (Russell and Norvig, 2009). Often, non-linearlyseparable data can be easily separable in a space with high enough dimensions (Russelland Norvig, 2009), as Figure 4.7 shows. In this figure, a classification problem ofone dimension (X = [x1]) is solved after the features are mapped into two dimensions(Φ(X) = [x1, (x1−2)2]).

Figure 4.7: Example of a non-linearly separable problem in one dimension that is linearlyseparable in two dimensions.

Additionally, it is important to note that all the previous calculations are valid to anon-linearly separable problem after the kernel function is applied. However, instead ofusing X in the previous equations, we use Φ(X), that is the image of the input vector Xinduced into the new feature space (Haykin, 2009).

In this work, we are going to use only one type of kernel that is the radial-basisfunction. This choice is due to the Gaussian nature of the shadowing, that is the mainobstacle to the predictions, as discussed in the Chapter 5. The used kernel is representedby k(Xa,Xb) in the Equation (4.25), in which both Xa and Xb are vectors drawn from theinput space.

k(Xa,Xb) = e−1

2σ2 ‖Xa−Xi‖2

(4.25)

In the case of noisy data, it might of interest to allow some points to violate theconstraint of the optimization problem (Equation (4.24)), instead of using very highdimensions to cleanly separate the classes (Russell and Norvig, 2009). This “soft-margin”allows some examples to fall on the wrong side of the hyperplane, but it assignsthem a penalty proportional to the distance required to move them back to the correctsize (Russell and Norvig, 2009). Therefore, the optimization problem can be rewritten asshown on Equation (4.26).


minimizeW,ε

12

WT W +CN∑

i=1

εi

subject to yi(WT Xi + b) ≥ 1− εi, for i = 1, . . . ,N.εi ≥ 0, for i = 1, . . . ,N.

(4.26)

The parameter C controls the trade-off between complexity and non-separable points.When C is assigned a large value, it indicates that the designer is very confident aboutthe quality of the data. When it is assigned a value very small (i.e. much inferior to 1), itmeans that the data is considered to be too noisy, therefore less emphasis should be placedon it to avoid overfitting (Haykin, 2009). The parameter εi, on the other hand, is a "slackvariable", that measures the deviation of a data point i from the ideal condition of patternseparability (Haykin, 2009).

Finally, in the regression problems, the SVM uses the kernel trick to take the datainto a high dimensional feature space to, then, perform a linear regression. However, isinstead of trying to minimize the error between prediction and data, the SVM goal is tolimit the violations on an error threshold. Therefore, the problem can be formulated asin Equations (4.27) to (4.29), in which pi is the estimator output, Φ(X) is the image ofthe input in the higher dimensional feature space, |yi − pi|ζ is a loss function with ζ asthe predefined chosen error threshold and ξ is a non-negative value that determines themaximum violation of this threshold.

pi = WT Φ(X) (4.27)

minimizeW

12‖W‖2 +C

N∑i=1

|yi− pi|ζ

subject to

pi− yi ≤ ζ + ξ, for i = 1, . . . ,N.yi− pi ≤ ζ + ξ, for i = 1, . . . ,N.ξ ≥ 0

(4.28)

|yi− pi|ζ =

|yi− pi| − ζ, f or|yi− pi| ≥ ζ

0, otherwise(4.29)

The SVM, similarly to the KNN, is a non-parametric model. Nevertheless, it onlystores the support vectors and not the whole set of examples, not demanding muchmemory or time after the training process is over. Despite that, it has the flexibility torepresent complex functions inherent of non-parametric models. Therefore, this techniquehas both non-parametric and parametric models advantages (Russell and Norvig, 2009).

Moreover, the SVM has the advantage of working really well in problems with largenumber of features and small training sets, but its computational cost grows with thenumber of training samples (Haykin, 2009). Furthermore, although this technique can beused for both regressions and classification problems, it is in the latter that it made theirmost significant impact (Haykin, 2009). This indicates that it is not the most efficient


machine to use when making a regression.

4.1.4 Decision Trees (DTs)Despite not being used at any framework, the concept of Decision Trees (DTs)

will be explained in this section because it is essential for the understanding of thetechnique Random Forest. The DT method is one of the simplest and yet most successfulmachine learning algorithms (Russell and Norvig, 2009). Its structure is based on asequence of tests on the input attributes, similarly to a “HowTo” manual. Those tests,also called internal nodes, lead to a finite discrete number of answers (the branches of thetree) that lead to more internal nodes until the final branches are reached, determining theoutput of the DT.

Figure 4.8 shows a very simple DT trained to solve a binary classification problem.The examples (training set), illustrated on the left side of the picture, are essential to buildthe tree (i.e., to train the machine), but they can be discarded once the training process isover. Therefore, the DT method is a parametric machine learning algorithm.

Figure 4.8: Example of a DT for a classification problem.

Furthermore, based on a single training set, one can create many DTs. With thepurpose of creating the most concise tree as possible, the algorithm tends to choose teststhat provide the biggest information gain on the problem, i.e., the ones that reduce theinformation entropy the most. The split 1, in the Figure 4.8, does so by dividing thetraining set in a way that most of the green circles are below the line and most of the redones are above it. Then, the second split reduces the entropy to 0 on the training dataabove the first split by putting all the greens on the left size and all the reds on the rightside. On the data bellow the split 1, the split 3 isolates all of the reds on the left size,leaving most of the greens on the right size of the line. Finally, the split 4 completelyseparates reds and greens on the training set by defining that all data on the left size ofsplit 3 is: red, if x1 > 20; or green, if x1 ≤ 20.

Moreover, for regression problems, the DT has at each leaf (set of branches that leadto an output) a linear function of some of the inputs, rather than a single value (Russell


and Norvig, 2009). Therefore, the learning algorithm must decide when to stop splittingand begin applying linear regression based on the information gain that the split willprovide (Russell and Norvig, 2009).

The DT is usually the first classification method tried for applications in commerceand industry (Russell and Norvig, 2009). The main reason is that this algorithm allowsfor a human to understand the reasoning behind the output choice, which can be veryuseful and sometimes required for some specific applications. For example, somecountries legally require this property for algorithms that take financial decisions subjectto anti-discrimination laws (Russell and Norvig, 2009).

4.1.5 Random Forests (RFs)RF is a technique of parametric supervised learning that uses an approach that

we call ensemble learning. This approach consists of selecting an ensemble ofmachines and combining their predictions with the purpose of reducing the chances ofmispredictions (Russell and Norvig, 2009).

The RFs, as the name suggests, uses a set of DTs, each one based on a randomlyportion of the training data and a subset of the inputs (Richards, 2017). After each treemakes its prediction, the output of the RF will either be: that class that was predictedthe most by the DTs, for a classification problem; or the average of the predictions,for a regression problem. Usually, the more trees the forest has, the more robust andaccurate it is (Richards, 2017). One of the greatest advantages of RFs consists of itslow sensibility to overfitting in classification problems with a large enough number oftrees (Richards, 2017). However, a large number of trees makes the model slower topredict its outputs (Richards, 2017). Another advantage of this model is that it can beused to identify the most important features in a problems, doing what we call featureengineering (Richards, 2017).

4.2 Machine Learning ImplementationThe implementation of all machines in the frameworks starts by pre-processing the

output data of ns-3 simulations (input data of machine learning strategies). We use thelibrary Scikit-Learn (version 0.19.1) of Python 3.6 for all parts of the implementation.Using a module named sklearn.preprocessing, we apply the function train_test_split tobest divide the inputs and outputs into training and test sets, ensuring that both setsrepresent well the data and, hence, reducing the chances of overfitting. The sizes of thesets will be clarified further down.

We apply the function StandardScaler to the divided data. The goal is to standardizeall the inputs and non-binary outputs of the dataset, giving them a Gaussian distributionwith zero mean and unity variance. This is important to ensure that the features have thesame scale, improving the convergence of the algorithms.

Furthermore, considering all the non-binary outputs have a large range of values,scaling them, while not mandatory, might improve the performance of machines.Additionally, since they belong to the levels B and C of the first two frameworks, and their


values are not the direct answer to the optimum target, the machine learning algorithmmerely compares them to other non-binary outputs in the same scale. Thus, it is notnecessary to undo the scaling process at the end of each prediction.

Then, we need to define the architecture of each machine at each framework. Theusual approach to this is try several types of architectures, and choose the best oneaccording to some score method that quantifies the quality of the prediction (Russelland Norvig, 2009). As we do not want to invalidate the results by peeking on all theavailable data, we use a method called k-fold cross validation (Haykin, 2009), as depictedin the Figure 4.9.

Figure 4.9: k-fold cross-validation for k = 5 and a training set of 4 000 examples.

In this method, we divide a training set into k parts. Then, k trials (or rounds) oflearning are performed and, on each trial, one of the k parts is held out as a test set,while the other k− 1 are used in the training process. The chosen part at each round isnamed the validation set, while the others are called the estimation set, according to thenomenclature used by (Haykin, 2009). At last, the structure with the best average scoreon the validation set of the k trials is chosen.

To perform this model selection, also named grid search, we decided to work with4000 examples (obtained with the function train_test_split and the random seed set to 0)as the training set. This decision is made due to the computational and time-related costsinvolved in the process. Additionally, we choose the k = 5, as it is considered enough togive an estimate that is likely to be accurate (Russell and Norvig, 2009).

To implement this grid search, we use the class GridSearchCV from themodel_selection module, also a part the library Scikit-Learn. The choice for the scoremethod is dependent on the nature of the problem, but in GridSearchCV all of them followthe convention that the higher the score, the better. Therefore, we chose the accuracy scorefor the classification machines and the neg_mean_squared_error (mean squared error inits negative form) for the regression ones.


4.2.1 ANNs ImplementationFor the ANNs, implemented in the module sklearn.neural_network, the variable

parameters are:

• The activation function for the hidden layers that can assume the followingvalues: identity (φ(z) = z), relu (φ(z) = max(0,z)), tanh (φ(z) = tanh(z)) and logistic(φ(z) = 1

1+e−z );• The number of hidden layers that can be varied from 1 to 3; and• The number of neurons in each hidden layer, that is set to assume any even number

between 2 and 40.

The other parameters are fixed in their default values. Among themis the solver, i.e., optimizer, whose chosen method is the Limited-memoryBroyden–Fletcher–Goldfarb–Shanno (L-BFGS). According to (Haykin, 2009), it is alimited-memory version of the best form of a quasi-Newton method. In the libraryScikit-Learn, this solver only works with batch learning and, hence, that is the type ofsupervised learning that we use. The fixed parameters also include the regularizationtherm alpha, used to avoid overfitting, and whose value is set to 0.0005. Table 4.1 showsall the fixed and variable parameters of our evaluation study.

Table 4.1: Parameters of the ANNs.Parameters ANN Classifiers ANN Regressors

Estimator MLPClassifier MLPRegressorScoring accuracy neg_mean_squared_errorSolver L-BFGS

Number of epochs 500alpha 0.0005

Activation function identity, relu, tanh or logisticNumber of hidden layers 1, 2 or 3

Number of neuronsfor each hidden layer

Nhl ∈ {2i | i ∈ [1,20]}

The variable parameters chosen by grid search are depicted on Tables 4.2 and 4.3, forthe first and second scenario, respectively. As we can see, the architectures chosen arevery diverse, but there is a clear preference for the topologies with three hidden layers,specially in Scenario 2, showing the increased processing power brought by the additionallayers, as mentioned in Subsection 4.1.1.

4.2.2 KNN ImplementationThe KNN implementation, made by the module sklearn.neighbors, uses several

variables and some fixed parameters. The main variable parameters are:

• The number of neighbors, that goes from 1 to 100;


Table 4.2: ANN parameters chosen through grid search for Scenario 1.Activation Number of neurons in the hidden layersfunction Hidden layer 1 Hidden layer 2 Hidden layer 3

Framework 1

MA2 identity 2 18 30MA3 logistic 28 - -MB2 relu 10 6 28MB3 tanh 18 16 2MC2 relu 14 8 10MC3 relu 36 28 10

Framework 2

MA2 logistic 2 2 2MA3 identity 26 - -MB2 relu 12 6 4MB3 tanh 16 30 2MC2 relu 40 8 18MC3 relu 32 32 20

Framework 3 - logistic 28 - -

Framework 4

MA2 logistic 2 2 2MA3 identity 26 - -MQ2 identity 38 6 34MQ3 tanh 6 6 34

Table 4.3: ANN parameters chosen through grid search for Scenario 2.Activation Number of neurons in the hidden layersfunction Hidden layer 1 Hidden layer 2 Hidden layer 3

Framework 1

MA2 tanh 6 22 40MA3 relu 4 4 34MB2 tanh 6 36 18MB3 relu 4 14 14MC2 tanh 8 10 12MC3 relu 4 14 12

Framework 2

MA2 tanh 8 8 12MA3 relu 12 14 10MB2 relu 12 2 36MB3 tanh 10 12 8MC2 tanh 10 32 6MC3 tanh 14 12 4

Framework 3 - tanh 2 28 22

Framework 4

MA2 tanh 8 8 12MA3 relu 12 14 10MQ2 tanh 4 38 22MQ3 tanh 40 32 34


Table 4.5: KNN parameters chosen through grid search for Scenario 1.Number ofneighbors Metric Weights Algorithm Leaf

size

Framework 2

MA2 4 euclidean uniform brute -MA3 1 euclidean uniform brute -MB2 4 chebyshev uniform kd_tree 20MB3 1 chebyshev uniform ball_tree 30MC2 16 chebyshev uniform ball_tree 100MC3 12 chebyshev uniform kd_tree 20

Framework 4

MA2 4 euclidean uniform brute -MA3 1 euclidean uniform brute -MQ2 7 euclidean uniform brute -MQ3 9 euclidean uniform brute -

• The distance metric, that can be one of the following: euclidean, chebychev,manhattan;

• The weights, that can be either uniform (all the neighbors in the neighborhood havethe same influence on the output) or distance (the influence of a neighbor in theoutput of a data point is the inverse of the distance to this point);

• The algorithm, that determines how the neighbors are found; and• the leaf size, that is a parameter used in the search for the neighbors.

The Table 4.4 shows all the fixed and variable parameters of the KNNs.

Table 4.4: Parameters of the KNNsParameter KNN Classifiers KNN RegressorsEstimator KNeighborsClassifier KNeighborsRegressorScoring accuracy neg_mean_squared_error

Number of neighbors Nneig ∈ {i | i ∈ [1,100]}Distance metric euclidean, chebychev, manhattan

Weights uniform or distanceAlgorithm ball_tree, kd_tree, auto or bruteLeaf size Lsize ∈ {10∗ i | i ∈ [1,10]}

The Tables 4.5 and 4.6 show the parameters chosen through grid search to Scenarios 1and 2, respectively. All the other parameters were kept with their default value.


Table 4.6: KNN parameters chosen through grid search for Scenario 2.Number ofneighbors Metric Weights Algorithm Leaf

size

Framework 2

MA2 24 chebyshev distance brute -MA3 69 chebyshev distance brute -MB2 27 chebyshev distance ball_tree 20MB3 48 chebyshev distance ball_tree 50MC2 22 chebyshev distance brute -MC3 54 chebyshev distance brute -

Framework 4

MA2 24 chebyshev distance brute -MA3 69 chebyshev distance brute -MQ2 89 chebyshev distance brute -MQ3 54 chebyshev uniform brute -

4.2.3 SVM ImplementationThe SVM implementation, implemented in the module sklearn.svm, has all its fixed

and variable parameters shown on Table 4.7. The most important parameter is the kernel,that, as previously discussed, is chosen to be the radial basis function (rbf ) for all thesimulations. Another fixed parameter is the hard limit on training iterations, chosen to be106. By default, the number of iterations on sklearn’s SVM is unlimited. However, sinceit could take a very long time for the regression problems to be trained with the defaultvalue of this parameter, we choose to set a limit to it.

Table 4.7: Parameters of the SVMsParameter SVM Classifiers SVM RegressorsEstimator SVC SVRScoring accuracy neg_mean_squared_errorKernel rbf

Hard limit on iterations 106

C C ∈ {10i|i ∈ [0,10]}gamma 0.25 or ∈ {10−i|i ∈ [0,10]}

class_weight balanced or None -

The main variable parameters that we searched through grid search are:

• The parameter C, called the penalty parameter of the error term by sklearn, isdiscussed in the Section 4.1.3. In our algorithm, the C value assumes any power of10 between 1 and 1010;

• The parameter gamma stands for the inverse of the standard deviation (σ) presentedin the Equation (4.25). It is able to assume any power of 10 between 1 and 10−10 orits the default value, that is the inverse of the number of inputs (0.25 in our case);


• The last parameter is the class_weight, that is exclusive for classification problemsand gives weights to each class. If this parameter takes the value balanced, it giveseach class a weight that is inversely proportional to the class frequency in the traindata. When this parameter takes the option None, all the classes are weighted by 1.

The parameters chosen through grid search to the SVMs are depicted in Tables 4.8and 4.9 for Scenarios 1 and 2, respectively.

Table 4.8: SVM parameters chosen through grid search for Scenario 1.C gamma class_weight shrinking

Framework 2

MA2 10 1 balanced TrueMA3 103 0.25 balanced TrueMB2 104 1 - FalseMB3 104 1 - FalseMC2 102 1 - TrueMC3 104 0.1 - False

Framework 4

MA2 10 1 balanced TrueMA3 103 0.25 balanced TrueMQ2 1010 1 balanced FalseMQ3 107 0.01 None False

Table 4.9: SVM parameters chosen through grid search for Scenario 2.C gamma class_weight shrinking

Framework 2

MA2 105 0.25 None FalseMA3 105 0.1 None FalseMB2 106 10−6 - TrueMB3 103 0.1 - FalseMC2 106 10−6 - FalseMC3 106 10−4 - False

Framework 4

MA2 105 0.25 None FalseMA3 105 0.1 None FalseMQ2 105 1 None FalseMQ3 105 0.25 None False


4.2.4 RF ImplementationThe RFs are implemented by the module sklearn.ensemble. Among the most

important parameters, we have the number of estimators, that defines the number of DTsin the forest and in our algorithm can assume any even number between 2 and 40. We alsohave the max_features, that stands for the maximum number of features used in each DTand in this work can be any number from 1 to 4. Moreover, we also have the criterion, thatis a function that measures the quality of a split in a DT; and the min_impurity_decreasethat imposes a minimum impurity decrease for a split to happen. All the parameters ofthe RFs and their values are exposed in the Table 4.10.

Table 4.10: Parameters of the RFs.Parameter RF Classifiers RF RegressorsEstimator RandomForestClassifier RandomForestRegressorScoring accuracy neg_mean_squared_error

Criterion gini or entropy friedman_mse, mse or maeNumber of estimators Ndt ∈ {2i | i ∈ [1,20]}

max_features 1, 2, 3 or 4min_impurity_decrease 0 or {10−i|i ∈ [0,10]}

bootstrap True or Falseoob_score True or False

class_weight balanced or None -

The values for variable parameters chosen through grid search are shown inthe Tables 4.11 and 4.12, for the Scenarios 1 and 2, respectively.

Table 4.11: RF parameters chosen through grid search for Scenario 1.Criterion class_ min_impurity_ max_ Number of bootstrap oob_scoreweight decrease features estimators

Framework 2

MA2 gini balanced 0.1 1 2 False FalseMA3 gini balanced 0.1 1 2 False FalseMB2 mae - 0.000001 4 3 True TrueMB3 mae - 0.00001 1 8 False FalseMC2 mae - 0.0001 1 20 True TrueMC3 mae - 0.00001 1 18 True True

Framework 4

MA2 gini balanced 0.1 1 2 False FalseMA3 gini balanced 0.1 1 2 False FalseMQ2 entropy None 0.1 2 12 True TrueMQ3 gini None 0.001 1 2 False False


Table 4.12: RF parameters chosen through grid search for Scenario 2.Criterion class_ min_impurity_ max_ Number of bootstrap oob_scoreweight decrease features estimators

Framework 2

MA2 entropy balanced 0.001 4 15 True TrueMA3 gini None 0.001 3 14 True TrueMB2 mae - 0.001 4 19 True TrueMB3 mae - 0.001 1 11 False FalseMC2 mae - 0.001 4 19 True TrueMC3 mae - 0.001 1 20 True True

Framework 4

MA2 entropy balanced 0.001 4 15 True TrueMA3 gini None 0.001 3 14 True TrueMQ2 entropy None 0.01 4 5 True TrueMQ3 gini None 0.001 1 8 False False

4.2.5 Training ConfigurationAfter having defined the parameters of each machine, now we need to decide the

number of training examples used in the training phase. For that, we use a tool called thelearning curves, that show the training score and the cross-validation score for differenttraining set sizes. Figures 4.10 and 4.11 present the learning curves for the Framework 3with ANNs) in Scenarios 1 and 2, respectively.

Figure 4.10: Learning curves for the network of the Framework 3 in Scenario 1.

For Scenario 1, a less hostile propagation environment, a small training set of about100 examples would be enough to reach a cross-validation score higher than 0.99.However, for Scenario 2, not even a training set size of 4000 examples can reach suchscore. In fact, the learning curve stabilizes around 88% since approximately the size of3000 examples. For this reason, we decided to remain using the training size of 4000 forthe performance tests phase. The test set size is chosen to be 25% of the training set size,therefore, it contains 1000 examples.


Figure 4.11: Learning curves for the network of the Framework 3 in Scenario 2.

Moreover, it is important to take into account the randomness of the machines. TheMLPs, for example, have varying performances according to the initial values theirweights take. Furthermore, the possible different partitions of the data between trainingand test sets could also lead to different performance outcomes. Thus, with the purposeof attaining a statistically significant evaluation, the analysis presented in the Chapters 5and 6 are based on the average of the results obtained using the Python random seedsin the interval [0,99]. The number of seeds was chosen to be the same one used in thework (Ali et al., 2016).

Chapter 5

Evaluation of Handover Frameworks

This chapter presents a performance evaluation of the four handover frameworkspresented in the Chapter 3. Using only ANNs, we are going to compare those frameworksto the classical handover strategy A2A4RSRP in order to decide which frameworks we aregoing to use in the complete analysis of the Chapter 6, that considers all the machinelearning techniques presented on Chapter 4. Our choice is based on a tradeoff betweenthe network’s performance and two other essential factors to the real life applicability ofthe frameworks: computational cost and scalability.

Moreover, with the purpose of detailing our analysis, all the results are detailed byregion. In that way, we are able to see the effects of the coverage hole and the shadowing,individually and combined, on the performance of the system (QoS metrics).

5.1 Results for Scenario 1The performance results of the Scenario 1 are depicted in Figures 5.1 to 5.4. They

show the scores and the three QoS metrics of the schemes, respectively. The score ofa proposed framework, or of a RS-based handover strategy, is the percentage of testsimulations for which the scheme is able to correctly identify the best target, according tothe rules proposed in Chapter 3. Although this statistic is not a QoS metric, it shows howaccurate the decision of our frameworks and the RS-based strategies can be.

Analyzing the Figure 5.1, all strategies and frameworks are able to find the best targetson approximately 100% of the simulations in the inferior and superior regions. The same,however, cannot be said about the central region. On account of the coverage hole, theRS-based strategy is deceived in more than 70% of the cases, illustrating the problemshown in the Chapter 2. The frameworks, however, due to the machines’ ability to foreseethe signal obstruction (explained in the Chapter 4), are only mistaken in less than 5% ofthe test simulations.

The QoS1, QoS2 and QoS3 results, respectively depicted in Figures 5.2 to 5.4,likewise show that the frameworks have better results than the classical strategy in theregion with the coverage hole. However, a few things diverge from Figure 5.1. First andforemost, all of the central region statistics for QoS1 are equal or higher than the scores.That is due to the fact that, in this region of Scenario 1, there is a considerable amount ofsimulations for which the download can be completed by both choices of target eNB, as

CHAPTER 5. EVALUATION OF HANDOVER FRAMEWORKS 50

Figure 5.1: Score of the handover frameworks in Scenario 1.

it also presented in the Figure 2.5.Secondly, even if both differences are very small, it is interesting to note that the

framework with the highest score (Figure 5.1), Framework 3, has the lowest QoS1(Figure 5.2) among the proposed frameworks. Similarly, Framework 4 that has the worstscore among the frameworks, has the highest QoS1. That apparent incongruence is aconsequence of the different structures of the frameworks. Due to its quantized Level Q(Figure 3.3), Framework 4 cannot differentiate well the performance of two eNBs withsimilar levels of throughput. Hence, it often makes mistakes that do not significantlyalter the QoS metrics, but do influence the score. Nevertheless, when the two eNBs’performance are significantly different in this scenario, such as the case when one eNBfinishes the download and the other does not (the most important situation in this case),Framework 4 is even slightly more efficient than the others.


Figure 5.2: Percentage of completed downloads in Scenario 1.

With respect to QoS2, displayed on Figure 5.3, the Framework 3 is slightly better thanthe others. That happens for the reason that this QoS only takes into account the downloadtime of the concluded file transfers, because, due the setup proposed in Chapter 2, it isnot possible to know the exact duration of the other downloads. Since Framework 3has a percentage of completed downloads lightly inferior than the other frameworks, itavoids accounting, possibly, some long downloads, which affects the results. An evidencethat corroborates with this inference is that the QoS3 for the general case, presentedon Figure 5.4, is approximately the same for all of the frameworks. This QoS is measuredin all the simulations and, therefore, presents a more complete result than the QoS2 does.


Figure 5.3: Average download time in Scenario 1.

Figure 5.4: Average throughput in Scenario 1.


5.2 Results for Scenario 2The Scenario 2 score, QoS1, QoS2 and QoS3 results are presented in Figures 5.5

to 5.8, respectively. As expected, the shadowing affects negatively the performance of theproposed schemes. Also, the frameworks now have more diverse scores, which allow usto analyze their particularities.

Initially, let us consider only the performance of the frameworks. For this scenario,the aforementioned incongruence between score and QoS1 is much more significant.Framework 3, that has a score of over 90% in the regions without obstacles and nearly75% in the central one, is, once again, the best scheme in this aspect (see Figure 5.5).The less efficient score of Frameworks 1 and 2 is a direct consequence of the preferencethat the scheme, presented on Figure 3.1, gives to the Level A (QoS1) decisions. Whenthe machines in such level indicate that only one target eNB finishes the download, thescheme automatically chooses this eNB and skips the other levels. If this prediction iswrong, the whole scheme will make a mistake. Also, due to the challenging nature ofthe regression task, it would be very difficult for the machines in Levels B and C to soaccurately predict the values of QoS2 and QoS3 that a small difference in them could becorrectly identified. On the other hand, Framework 4 does use all the machines to makeits choice. However, for the reasons mentioned in Section 5.1, it has once more the lowestscore.

Nevertheless, Framework 3 does not show prominent results when comparing to theother proposed frameworks in the QoS metrics (Figures 5.6 to 5.8). In fact, this frameworkis the worst hybrid scheme in terms of QoS1 (Figure 5.6). Hence, we can infer that, inthe extra times it finds the best target cell, there are no significant differences betweenthese two options, i.e., the eNB signals do behave similarly throughout the simulation.Therefore, the less efficient scores of Frameworks 1, 2 and 4 do not meaningfully damagethe system’s performance.

The same inference can be made when comparing the performances ofFrameworks 1, 2 and 4. Framework 1 has the second best score (Figure 5.5), beingsignificantly better than Frameworks 2 and 4, but only in this criteria. Which means thatthe simpler schemes of Frameworks 2 and 4 make them more susceptible to errors thanFramework 1, but those mistakes do not notably influence the QoS metrics, as presentedin Figures 5.6 to 5.8.


Figure 5.5: Score of the handover frameworks in Scenario 2.

Furthermore, now considering all the schemes, we can see from Figures 5.5 and 5.6that, contrarily to what happened in Scenario 1, all QoS1 results are lower than the scores.As we can verify in Figure 2.7, that is a consequence of the high percentage of downloadsthat cannot be complete by neither eNBs in all the regions of Scenario 2.

Moreover, we perceive that the all the hybrid schemes have distinctly superior scores(Figure 5.5) and considerably better QoS1 and QoS3 metrics (Figures 5.6 and 5.8) thanthe RS-based strategy in all regions, especially in the ones without a coverage hole.



Figure 5.7: Average download time in Scenario 2.



The QoS2 metric, presented in Figure 5.7, does not offer reliable results in thisscenario. That is a consequence of the fact that, as previously stated, given the setupproposed in Chapter 2, it is only possible to collect this metric for the downloads thatwere completed. Since this scenario has a high percentage of uncompleted downloads,this statistic is compromised. Therefore, it will not be analyzed.

5.3 Preliminary Conclusions and Chosen FrameworksIn this chapter, we have analyzed the performance of the four handover frameworks

and the classical strategy A2A4RSRP. As it is exposed by the results, all the proposedschemes have superior performances when compared to the classical strategy. Also,despite their different structures, the proposed frameworks provide similar QoS indicatorsto the system. Since Frameworks 2 and 4 are the most scalable frameworks and requireless computational cost than Framework 1, we have decided to use them for the analysisperformed in Chapter 6.

Moreover, considering that the QoS2 analysis is not reliable for Scenario 2 and thatthe score of the schemes is not a very good indicator of the system performance, as wediscussed in this chapter, we are going to base the performance evaluation of Chapter 6 inQoS1 and QoS3.

Chapter 6

Handover with Machine LearningTechniques

This chapter presents the performance evaluation of the Frameworks 2 and 4 with thefour machine learning techniques presented on Chapter 4. For better understanding theimpact of these hybrid handover strategies on the download success and throughput, theresults are compared to the classical strategy A2A4RSRP, similarly to Chapter 5, and tothe Random Strategy presented on Chapter 2.

6.1 Results for Scenario 1The performance evaluation of the handover schemes in Scenario 1 are displayed

in Figures 6.1 and 6.2. Figure 6.1, that shows the percentage of complete downloadsfor Scenario 1, attests that all the machine learning techniques are able to completealmost 100% of the downloads in all regions. Furthermore, the difference in performancebetween the hybrid techniques in this aspect is not significant.

The average throughput for this scenario (Figure 6.2) also shows a very goodhandover frameworks’ performance, with all of them being above 2.8 Mbps in all theregions. An interesting fact to observe is that, due to Framework 4’s quantized structure(see Figure 3.3), the strategies that use it tend to have tenths more in the downloadpercentage (Figure 6.1), but almost the same throughput than Framework 2’ ones.

Furthermore, as we have seen in Chapter 5, the classical strategy A2A4RSRP offers anacceptable performance in the inferior and superior regions, but a very degraded QoS1 andQoS3 in the central region, being even the worst than the Random Strategy. Meanwhile,all the frameworks, even the the simplest ones, such as Framework 2 and 4 with KNN,have an almost perfect score in all the regions, which shows the power of the learningmachines to this kind of solution.

6.2 Results for Scenario 2In Scenario 2, the situation is more challenging. Figures 6.3 and 6.4 show,

respectively, the QoS1 and QoS3 for this scenario. The hybrid strategies based on the

CHAPTER 6. HANDOVER WITH MACHINE LEARNING TECHNIQUES 58




less complex machine learning algorithms in this work (KNN and RF) are unable todeliver higher QoS1 than the classical strategy in the inferior and central regions. This isa consequence of the fact that they are not being able to handle properly the random effectbrought to the system by the shadowing.


The strategies that used ANNs and SVMs, however, are both able to deliver asignificantly higher performance than the A2A4RSRP in all the regions, with the SVMs’Framework 4 having a slightly higher percentage of complete downloads and throughputthan the others.

Another important thing to note from Figures 6.3 and 6.4 is that, for the ANNs,Frameworks 2 and 4 show similar results, although for QoS1 Framework 4 works a littlebetter, which indicates that the quantization made by this framework does not damage theperformance, on the contrary. The same thing can be observed for the SVMs. However,for this technique the differences between the two frameworks is more noticeable, whichis an effect of its comparatively low performance in the regression problems (nonexistenton Framework 4).

6.3 ConclusionsIn this chapter we have evaluated the performance of Frameworks 2 and 4 with

the machine learning techniques ANN, KNN, SVM and RF, comparing them with the



traditional handover strategy A2A4RSRP in the two scenarios proposed in Chapter 2. ForScenario 1, both frameworks (with all the tested techniques, even the simplest ones)delivered superior throughputs to the other schemes and almost 100% of completeddownloads, notwithstanding the coverage hole. In Scenario 2, however, the shadowingbrought complexity to the propagation environment, which demanded more complexmachines (in our case, ANN and SVM) in either framework in other to reach acceptableQoS levels. Additionally, in spite of the throughput quantization and the use ofless machines, Framework 4 continues to offer notably good results even in a harderpropagation environment, as long as it is composed of ANNs or SVMs.

Chapter 7

Conclusions and Future Perspectives

This work propose four machine learning frameworks to perform handover in twoscenarios with a coverage hole. Some machine learning techniques are used in ourschemes that, in the scenarios proposed, proved to be more efficient than the classicalapproach to handover in terms of percentage of downloads completed, download timeand throughput. This chapter is going to discuss the research questions addressed inthe Chapter 1, shows our further investigations and exposes the academic productionsresulting from this research.

7.1 Discussion about the Research QuestionsWith the analysis made by this research, we are able to discuss the following questions

defined in the Chapter 1 and here repeated by convenience:

• Are the traditional handover algorithms suited to deal with a scenario with acoverage hole?

No, the traditional handover algorithms proved to be inefficient in the presenceof a coverage hole. As shown in the Chapter 6, in a region with a coverage hole theA2A4RSRP is even more inefficient than the Random Strategy.

• Can a machine learning based handover algorithm work better than thetraditional one in such scenario?

Yes, the machine learning based handover algorithms with the configurationdepicted in the Chapter 3 show better results than the classical one. This analysis ispresented in the Chapters 5 and 6.

• Does the shadowing affects the performance of a traditional handoveralgorithm? How about the performance of a machine learning basedalgorithm?

The shadowing affects both the performances of traditional and machinelearning based handover algorithms. However, the classical approach is much moreaffected.

• How good is the performance of a machine learning based algorithm in asituation with both shadowing and coverage holes?

Despite being superior to A2A4RSRP’s performance, the proposed schemes arestill suboptimal, as attested by comparing the Figures 2.7 and 6.3.

CHAPTER 7. CONCLUSIONS AND FUTURE PERSPECTIVES 62

7.2 Future PerspectivesSeveral improvements can be made in our analysis. In order to enhance the

performance of our frameworks, we plan to investigate the following possibilities:

• Try different solutions to the situation 2 in Framework 4 (Figure 3.3);• Change the structure of Frameworks 1 and 2 to allow the use of all machines in

each decision;• Trigger handover frameworks with A2A4RSRQ instead of A2A4RSRP;• Make a different grid search with k = 10;• Use Fuzzy logic in the Frameworks.

With the purpose of investigating other configurations of scenarios, we intend to dothe following modifications:

• Use correlated shadowing;• Change the users’ data traffic;• Increase the number of users performing handovers;• Increase the number of blocks in the system;• Make the simulation last longer in order to better evaluate QoS2;• Add co-channel interference to the system.

7.3 Academic ProductionsAs part of the research activities that resulted on this work, the author wrote the

following articles for national and international conferences.

• "Performance Analysis of Handover Strategies in the 3GPP Small Cell Scenario",accepted and published in the annals of CSCI 2017 (The 2017 InternationalConference on Computational Science and Computational Intelligence, EUA). Thisarticle analyzes the impact of a macro-pico deployment in a urban environment,focusing on user throughput and number of handovers. The scenarios areimplemented in ns-3, and both A2A4RSRQ and A3RSRP algorithms are analized.This is the first time the author performed simulations with small cells and analyzedthe importance of an efficient handover management in an LTE network;

• "Handover Baseado em Aprendizado de Máquina para Redes LTE com Falhasde Cobertura", submitted to SBrT 2018 (XXXVI Simpósio Brasileiro deTelecomunicações e Processamento de Sinais). This article has the same scope ofthis dissertation, but has no grid search to define the machines’ parameters, using afixed configuration instead, and the Framework 4 was not proposed yet. Despite thepaper’s rejection, the reviewers suggestions were very useful and helped the authorto improve the performance analysis.

Furthermore, a journal article named “Machine Learning Based HandoverManagement for LTE Networks with Coverage Holes”, is currently in production. It

CHAPTER 7. CONCLUSIONS AND FUTURE PERSPECTIVES 63

contains the full scope of this dissertation, including the new machines, and it is going tobe submitted to the International Journal of Communication Systems, from Wiley OnlineLibrary.

Moreover, the knowledge acquired during research involved on the production of thisdissertation also allowed the author to be a part of the III Workshop do GppCom, as one ofthe main lecturers of the course "Conhecendo e praticando técnicas de Machine learning:descobrindo seu potencial para soluções de problemas em telecom".

7.4 ConclusionsIn this work, we have proposed four machine learning frameworks for the handover

of an LTE small cell network with a coverage hole. Using ANNs, the frameworks weretested, and compared to each other and to one classical handover strategy, in scenarioswith and without shadowing. All the strategies delivered a superior performance than theclassical one in the presence of coverage holes or shadowing (or both). However, due totheir scalability, we decided to use the Frameworks 2 and 4 to make a deeper analysis byusing three other machine learning techniques, along with the already analyzed ANN. ForScenario 1, that has a simpler propagation environment, the Frameworks 2 and 4 deliveredan almost perfect performance, whichever technique was being used. For Scenario 2,nevertheless, the complexity of the propagation environment justifies the use of morecomplex machines, such as ANNs and SVMs, to reach acceptable performance results.Despite the use of less machines, whenever one of the two aforementioned machinelearning algorithms is used, Framework 4 continues to offer satisfactory results even in aharder propagation environment.

Bibliography

3GPP (2009), ‘3GPP TS 36.133: Evolved Universal Terrestrial Radio Access(E-UTRAN); Requirements for Support of Radio Resource Management’.

3GPP (2010), ‘LTE; Evolved Universal Terrestrial Radio Access (E-UTRA); Physicallayer - Measurements (Release 9)’.

3GPP (2018), ‘Hetnet/small cells’.URL: http://www.3gpp.org/hetnet

Abar, T., A. Ben Letaifa and S. El Asmi (2017), Machine Learning Based QoE Predictionin SDN Networks, em ‘2017 13th International Wireless Communications andMobile Computing Conference (IWCMC)’, pp. 1395–1400.

Akyildiz, Ian F., David M. Gutierrez-Estevez and Elias Chavarria Reyes (2010), ‘TheEvolution to 4G Cellular Systems: LTE-Advanced’, Phys. Commun. 3(4), 217–244.

Ali, Z., N. Baldo, J. Mangues-Bafalluy and L. Giupponio (2015), ‘Simulating LTEMobility Management in Presence of Coverage Holes with ns-3’, Simulation Toolsand Techniques (SIMUTOOLS) .

Ali, Z., N. Baldo, J. Mangues-Bafalluy and L. Giupponio (2016), ‘Machine LearningBased Handover Management for Improved QoE in LTE’, Network Operations andManagement Symposium (NOMS) .

Baldo, N., G. Coskun, I. Hokelek and H. A. Cirpan (2013), ‘An Open Source Model forthe Simulation of LTE Handover Scenarios and Algorithms in ns-3’, ACM .

Begluk, T., J. B. Husic and S. Barakovic (2018), Machine Learning Based QoE Predictionfor Video Streaming Over LTE Network, em ‘2018 17th International SymposiumINFOTEH-JAHORINA (INFOTEH)’, pp. 1–5.

Cardoso, E., K. Silva and R. Francês (2017), ‘Intelligent Handover Procedure forHeterogeneous LTE Networks Using Fuzzy logic’, 13th International WirelessCommunications and Mobile Computing Conference (IWCMC) .

Casas, P., A. D’Alconzo, F. Wamser, M. Seufert, B. Gardlo, A. Schwind, P. Tran-Gia andR. Schatz (2017), Predicting QoE in Cellular Networks Using Machine Learning andin-Smartphone Measurements, em ‘2017 Ninth International Conference on Qualityof Multimedia Experience (QoMEX)’, pp. 1–6.

64

BIBLIOGRAPHY 65

Chandrasekhar, V., J. G. Andrews and A. Gatherer (2008), ‘Femtocell Networks: aSurvey’, IEEE Communications Magazine 46(9), 59–67.

Chang, Y., W. Li and Z. Yang (2017), Network Intrusion Detection Based on RandomForest and Support Vector Machine, em ‘2017 IEEE International Conference onComputational Science and Engineering (CSE) and IEEE International Conferenceon Embedded and Ubiquitous Computing (EUC)’, Vol. 1, pp. 635–638.

Chaparro-Marroquín, J. C. (2014), ‘Comparison Between Measurement Events for LTEHandover in Rural and Urban Scenarios Involving Femto-Cell Deployment’, LatestTrends on Communications .

Coppin, B. (2010), Inteligência Artificial, LTC.

Dahlman, E., S. Parkvall and J. Skold (2016), 4G, LTE-Advanced Pro and The Road to5G, Elsevier Science.

de Pádua Braga, A. (2007), Redes neurais artificiais: teoria e aplicações, LTC Editora.

Ferng, H. and Y. Huang (2016), Handover Scheme with eNode-B Pre-Selectionand Parameter Self-Optimization for LTE-A Heterogeneous Networks, em ‘2016International Conference on Machine Learning and Cybernetics (ICMLC)’, Vol. 2,pp. 594–599.

Goodfellow, Ian, Yoshua Bengio and Aaron Courville (2016), Deep Learning, MIT Press.

Guerra, T. C. B., Y. R. Dantas and V. A. Sousa Jr. (2017), ‘PerformanceAnalysis of Handover Strategies in the 3GPP Small Cell Scenario’, The2017 International Conference On Computational Science And ComputationalIntelligence (CSCI-ISMC) .

Gupta, A. and R. K. Jha (2015), ‘A Survey of 5G Network: Architecture and EmergingTechnologies’, IEEE Access 3, 1206–1232.

Haykin, S. (2009), Neural Networks and Learning Machines, número v. 10 em ‘Neuralnetworks and learning machines’, Prentice Hall.

Herman, B., D. Petrov, J. Puttonen and J. Kurjenniemi (2013), ‘A3-Based Measurementsand Handover Model for NS-3 LTE’, MOBILITY 2013 : The Third InternationalConference on Mobile Services, Resources, and Users .

Holma, H., A. Toskala and J. Reunanen (2016), LTE Small Cell Optimization: 3GPPEvolution to Release 13, Wiley.

Ihalainen, T., P Janis, Z. Li, M Moisio, V Nurmela, M Uusitalo, C Wijting and O Yilmaz(2013), Flexible scalable solutions for dense small cell networks, em ‘WirelessWorld Research Forum (WWRF), Oulu, Finland’.

BIBLIOGRAPHY 66

Kamel, M., W. Hamouda and A. Youssef (2016), ‘Ultra-Dense Networks: A Survey’,IEEE Communications Surveys Tutorials 18(4), 2522–2545.

Kazi, B. U. and G. Wainer (2017), Handover Enhancement for LTE-Advanced andBeyond Heterogeneous Cellular Networks, em ‘2017 International Symposium onPerformance Evaluation of Computer and Telecommunication Systems (SPECTS)’,pp. 1–8.

Khunteta, S. and A. K. R. Chavva (2017), Deep Learning Based Link FailureMitigation, em ‘2017 16th IEEE International Conference on Machine Learning andApplications (ICMLA)’, pp. 806–811.

Ledolter, J. (2013), Data Mining and Business Analytics with R, Wiley.

LENA (2018).URL: http://networks.cttc.es/mobile-networks/software-tools/lena/

Li, Y., B. Cao and C. Wang (2016), ‘Handover Schemes in Heterogeneous LTE Networks:Challenges and Ppportunities’, IEEE Wireless Communications 23(2), 112–117.

Lima, I., C.A.M. Pinheiro and F.A.O. Santos (2016), Inteligência Artificial, ElsevierEditora Ltda.

Malekzadeh, M. and F. Rezaiee (2018), ‘Impact of Inter-eNodeB Handover Parameterson Performance Optimization of LTE Networks’.

ns-3 (2017), ‘ns3::A2A4RsrqHandoverAlgorithm algorithm documentation’.URL: https://www.nsnam.org/doxygen/classns3_1_1_a2_a4_rsrq_handover_algorithm.html

Parkvall, S., E. Dahlman, A. Furuskar and M. Frenne (2017), ‘NR: The New 5G RadioAccess Technology’, IEEE Communications Standards Magazine 1(4), 24–30.

Priyadharshini, A. S. and P. T. V. Bhuvaneswari (2016), A Study on HandoverParameter Optimization in LTE-A Networks, em ‘2016 International Conferenceon Microelectronics, Computing and Communications (MicroCom)’, pp. 1–5.

Richards, K. (2017), Machine Learning: For Beginners – Your Comprehensive GuideFor Markov Models, Reinforced Learning, Model Evaluation, SVM, Naïves BayesClassifier, Ken Richards.

Russell, Stuart and Peter Norvig (2009), Artificial Intelligence: A Modern Approach, 3rda

edição, Prentice Hall Press, Upper Saddle River, NJ, USA.

Saeed, M., M. El-Ghoneimy and H. Kamal (2017), ‘An Enhanced Fuzzy LogicOptimization Technique Based on User Mobility for LTE Handover’, 34th NationalRadio Science Conference (NRSC) .

Samuel, A. L. (1959), ‘Some Studies in Machine Learning Using the Game of Checkers’,IBM Journal of Research and Development 3(3), 210–229.

BIBLIOGRAPHY 67

Sesia, S., I. Toufik and M. Baker (2011), LTE - The UMTS Long Term Evolution: FromTheory to Practice, Wiley.

Thakkar, M., L. Agrawal, A. Rangisetti and B. Tamma (2017), ‘Reducing Ping-PongHandovers in LTE by Using A1-Based Measurements’, Twenty-third NationalConference on Communications (NCC) .

Yang, H., B. Hu and L. Wang (2017), A Deep Learning Based Handover Mechanismfor UAV Networks, em ‘2017 20th International Symposium on Wireless PersonalMultimedia Communications (WPMC)’, pp. 380–384.

Yang, J., C. Dai and Z. Ding (2017), A Scheme of Terminal Mobility Prediction of UltraDense Network Based on SVM, em ‘2017 IEEE 2nd International Conference onBig Data Analysis (ICBDA)(’, pp. 837–842.

Yin, C., Y. Zhu, J. Fei and X. He (2017), ‘A Deep Learning Approach for IntrusionDetection Using Recurrent Neural Networks’, IEEE Access 5, 21954–21961.

machine learning based handover management for lte ...€¦ · most likely o er the user the...

Documents