5g-miedge · the early stage of the 5g-miedge project, and in this report it is reminded as...

Deliverable Horizon2020 EUJ-01-2016 723171 5G-MiEdge D3.2

Date : February 2019 Public Deliverable

5G-MiEdge

5G-MiEdge Millimeter-wave Edge Cloud as an Enabler for 5G Ecosystem

EU Contract No. EUJ-01-2016-723171

Contractual date: M32

Actual date: M32

Authors: See list

Work Package: D3.2 - Integration of mmWave edge cloud into 5G mobile networks

Security: Public

Nature: Report

Version: 1.0

Number of pages: 62

Abstract

This deliverable is the final report of Task 3.1 which studied on integration of mmWave

edge cloud into 5G mobile networks. It reports on an architecture for integrating edge

computing into 5G mobile networks, interworking between 3GPP and non-3GPP RAN,

control signaling, service handover behavior to enable 5G liquid edge cloud to support

multi-layer, multi-RAT and ultra-dense small cell deployment.

Keywords

Network architecture, 5G mobile, edge computing, system integration, resource management, control

signaling, user mobility, service handover, workload placement, hierarchical edge cloud

All rights reserved.

2

5G-MiEdge

The document is proprietary of the 5G-MiEdge consortium members. No copy or distribution, in any

form or by any means, is allowed without the prior written agreement of the owner of the property

rights.

This document reflects only the authors’ view. The European Community is not liable for any use hat

may be made of the information contained herein.

Authors

KDDI Research, Inc. Katsuo Yunoki [email protected]

Hiroyuki Shinbo [email protected]

Intel Deutschland GmbH Valerio Frascolla [email protected]

Robert Zaus [email protected]


Date : February 2019 Public Deliverable

5G-MiEdge

Table of contents

Abbreviations and acronyms ......................................................................................... 5

Executive Summary ........................................................................................................ 7

1 Introduction ............................................................................................................. 8

2 Architecture ........................................................................................................... 10

2.1 High level architecture ................................................................................... 10

2.1.1 MiEdge RAN ...................................................................................... 10

2.1.2 MEC Service Function (MSF) ............................................................ 11

2.2 5G architecture discussed in 3GPP ................................................................ 12

2.2.1 5G network architecture ..................................................................... 12

2.2.2 RAN architecture ................................................................................ 13

2.2.3 Interworking with non-3GPP access .................................................. 15

2.3 MEC architecture ........................................................................................... 20

3 Integration of mmWave edge cloud into 5G mobile networks with inter

operable control plane .......................................................................................... 22

3.1 Control signaling for 5G MiEdge registration and radio/ computational

resource map .................................................................................................. 22

3.1.1 Radio and computational resource managements .............................. 22

3.1.2 Radio and computational resource mapping ...................................... 22

3.2 Control plane delegation to support multi-connectivity in signaling for

integrated heterogeneous networks ................................................................ 24

3.2.1 Access from 3GPP RAN to MEC host ............................................... 24

3.2.2 Access from non-3GPP access to MEC host ...................................... 27

3.3 Multi-layer radio/computation resource management for enhanced mobility

support ............................................................................................................ 28

3.3.1 Mobility support for MEC service ..................................................... 28

3.3.2 Problems of conventional mobility support ....................................... 31

3.3.3 Service continuity via carry-on state service handover ...................... 33

3.3.4 VM instantiation exploiting mobility prediction ................................ 39

3.3.5 Appropriate workload placement in edge cloud ................................. 48

3.3.6 Combined result with the approach in Task 3.2 ................................. 56

3.3.7 Patent applications .............................................................................. 57

4

5G-MiEdge

4 Application to 5G MiEdge use cases ................................................................... 58

4.1 Omotenashi services ....................................................................................... 58

4.2 Moving hotspot .............................................................................................. 58

4.3 2020 Tokyo Olympic (stadium) ..................................................................... 59

4.4 Dynamic crowd .............................................................................................. 59

4.5 Automated driving .......................................................................................... 59

5 Summary ................................................................................................................ 60

6 References .............................................................................................................. 61

5

5G-MiEdge

Abbreviations and acronyms

Acronym Description

3GPP The 3rd Generation

Partnership Project

5G The 5th (fifth) Generation

5GC 5G Core network

5G-MiEdge

Millimeter-wave Edge

Cloud as an Enabler for 5G

Ecosystem

AAA Authentication, Authoriza-

tion and Accounting

AMF Access and Mobility mana-

gement Function

ANDSF Access Network Discovery

and Selection Function

AP Access Point

APC Access Point Controller

API Application Interface

App Application

AR Augmented Reality

BS Base Station

CaSH Carry-on state Service

Handover

CDF Cumulative Distribution

Function

C-Plane Control Plane

C-RAN Centralized RAN

C/U split Control/User-plane split

DC Dual Connectivity

D-RAN Distributed RAN

DN Data Network

DNN Data Network Name

ETSI European Telecommunica-

tions Standards Institute

FMC Follow Me Cloud

gNB gNode B

IEEE The Institute of Electrical

and Electronic Engineers

IPSec Internet Protocol Security

Architecture

ISG Industry Specification

Group

LADN Local Area Data Network

LAN Local Area Network

LTE Long Term Evolution

ME Mobile Edge or Multi-

access Edge

MEC

Mobile Edge Computing or

Multi-access Edge

Computing

MEH Mobile Edge Host

MEO Mobile Edge Orchestrator

MEP Mobile Edge Platform

MEPM Mobile Edge Platform

Manager

MgNB Master gNodeB

MiEdge mmWave Edge cloud

mmWave Millimeter Wave

MSF MEC Service Function

N3IWF Non-3GPP Interwork

Function

NEF Network Exposure Function

NSSAI Network Slice Selection

Assistance Information

OAM Operation and Maintenance

PCC Policy and Charging Control

PCF Policy Control Function

PDU Packet Data Unit

P-GW Packet Gateway

QoS Quality of Service

RAN Radio Access Network

6

5G-MiEdge

RAT Radio Access Technologies

REST Representational State

Transfer

RNI Radio Network Information

RRC Radio Resource Control

RSU Road Side Unit

RTT Round Trip Time

SDT Service Disruption Time

SgNB Secondary gNodeB

SM Session Management

SMF Session Management

Function

S-NSSAI Single Network Slice Sele-

ction Assistance Information

UDM Unified Data Management

UE User Equipment

UPF User Plane Function

U-Plane User Plane

V2V Vehicle to Vehicle

VM Virtual Machine

WP Work Package


Date: February 2019

Public Deliverable

5G-MiEdge

Executive Summary

The EU-Japan funded research project 5G-MiEdge (Millimeter-wave Edge cloud as

an enabler for 5G ecosystem) envisions to design, develop and demonstrate an

innovative 5G architecture, which manages to smartly and smoothly combine

millimeter-wave (mmWave) access/backhauling with mobile edge cloud (computing)/

multi access edge computing (MEC). This deliverable belongs to WP3, which runs

from month 4 to 32 of the 36-month lifetime of the project.

WP3 is in charge of mmWave access and MEC integration, control signaling and

radio/computation resource optimization in 5G mobile networks. This deliverable of

Task 3.1 especially focuses on the integration of mmWave edge cloud into 5G mobile

networks.

The highlights of this deliverable are:

• The baseline architectures for Task 3.1 studies in Chapter 2

• Integration of mmWave edge cloud into 5G mobile networks with inter operable

control plane in Chapter 3, split into:

o Control signaling for 5G MiEdge registration and radio/ computational

resource map

o Control plane delegation to support multi-connectivity in signaling for

integrated heterogeneous networks

o Multi-layer radio/computation resource management for enhanced

mobility support

• Application to 5G MiEdge use cases in Chapter 4


Date: February 2019

Public Deliverable

5G-MiEdge

1 Introduction

The EU-Japan funded research project 5G-MiEdge (Millimeter-wave Edge cloud as

an enabler for the fifth generation (5G) ecosystem) envisions to design, develop and

demonstrate an innovative 5G architecture, which manages to smartly and smoothly

combine mmWave access/backhauling with the multi-access edge computing (MEC).

This joint work of new key 5G and beyond enabling technologies will be focusing

mainly on the challenging scenarios and use cases previously reported in [D1.1].

WP3 focuses on mmWave access and MEC integration, control signaling and

radio/computation resource optimization in 5G mobile networks.

It will be possible to realize a new service paradigm that can guarantee ultra-high speed

and low latency using together mmWave access and MEC deployments. However, in

order to make that happen, it is necessary to implement intelligent service deployments

and distributions based on demands, due to the limited availability in a real deployment

of on-site MEC resources.

To this end, WP3 aims to develop

(i) control signaling to integrate edge cloud into 5G mobile networks,

(ii) learning methods to forecast future traffic,

(iii) joint radio and computation resource orchestration.

In order to carry out technology development efficiently, WP3 comprises of three tasks

(Task 3.1, Task 3.2 and Task 3.3), which have been jointly worked on by European

partners (CEA-LETI, Sapienza University of Rome, Intel) and Japanese partners

(KDDI Research, Tokyo Institute of Technology).

The deliverables of WP3 are summarized in Fig. 1. This deliverable is the final report

of Task 3.1, which focuses on the integration of mmWave edge cloud into a 5G mobile

network. Control signaling is also studied to serve contents or services, which are

deployed by utilizing edge cloud in the network in a distributed manner. It includes

signaling for accessing contents/services and optimal radio/computation/storage

resource utilization.

The rest of this deliverable is structured as follows:

• Chapter 2 provides the architectures proposed by the project, which is the

baseline for Task 3.1 studies,

• Chapter 3 provides some aspects for the integration of mmWave edge cloud into

5G mobile networks including:

Basic registration ways of radio and communication resources (subtask

3.1.1),

Signaling to access an appropriate MEC host for a requested service

(subtask 3.1.2),

Potential procedures for MEC service continuity and workload placement

considering UE’s mobility (subtask 3.1.3),

• Chapter 4 evaluates how the proposed approaches can be applied to 5G-MiEdge

use cases, and, finally,

• Chapter 5 summarizes this report.


Date: February 2019

Public Deliverable

5G-MiEdge

Fig. 1 WP3 deliverables

D3.1: (M18) Architecture of mmWave edge cloud and requirement for

control signaling

Deliver basic concepts and mechanisms of each task for harmonization among

tasks.

D3.2: (M32) Integration of mmWave edge cloud into 5G

mobile networks

Report on new C-plane to enable 5G liquid edge cloud to

support multi-layer, multi-RAT, and ultra-dense small cell

deployment.

D3.3: (M32) Context information management to create

traffic map for mmWave edge cloud

Report on architecture and protocol of context information

exchange for traffic map prediction.

D3.4: (M32) User/application centric orchestration to

realize 5G liquid edge cloud

Report on orchestration methods for optimal resource allocation,

load distribution and clustering to realize 5G liquid edge cloud.

Task 3.1

Task 3.2

Task 3.3

This deliverable


Date: February 2019

Public Deliverable

5G-MiEdge

2 Architecture

This chapter analyzes the architectures that can fulfil the requirements of the project

and surveys the results of the discussions on 5G-MiEdge relevant topics in the 3rd

Generation Partnership Project (3GPP) working groups and in the Multi-access Edge

Computing Industry Specification Group (MEC ISG), within the European

Telecommunications Standards Institute (ETSI). Such information was dealt with in

the early stage of the 5G-MiEdge project, and in this report it is reminded as baseline

for studies of the work done in work package 3 (WP 3) in the following subchapters.

2.1 High level architecture

WP1 defined a high-level system architecture for the 5G-MiEdge project, as depicted

in Fig. 2-1 (See [D1.3] for more details).

Fig. 2-1 High level system architecture for the 5G-MiEdge project

This high-level architecture is based on the architecture of the 5th generation (5G)

mobile network, as defined in [TS23.501], and on the MEC framework and

architecture as defined in [MEC003]. The architecture proposed in [D1.1] was a

preliminary version and has been subject to refinements and improvements coming out

of the work of the technical work packages, including WP3.

In the following, only some details are provided on the proposed blocks of the

architecture. A full-fledged description can be found in [D1.1].

2.1.1 MiEdge RAN

The proposed MiEdge RAN (Radio Access Network) entity is composed of one or more

instances of RAN, AP (Access Point), UPF (User Plane Function) and mobile edge

host (MEH) functional blocks. A short description of the main functions that each

block delivers to the overall entity is summarized here below:

(1) RAN


Date: February 2019

Public Deliverable

5G-MiEdge

• Provision of the user-plane (U-plane) for UE (User Equipment) using 3GPP

access,

• Provision of the control-plane (C-plane) for UE,

(2) UPF

• Routing to the requested service on MEH,

• Quality of Service (QoS) control for data flow of the requested service,

(3) AP (in case of non-3GPP access)

• U-plane provision for UE using non-3GPP access,

• Routing to the requested service on MEH,

• QoS control for data flow of the requested service, when applicable,

(4) MEH

• Hosting services/contents,

• Activating/adding/deleting/transferring/resuming services/contents, under the

control of the MEC Service Function (MSF).

Note: We assume in the MiEdge architecture a multi-connectivity scenario, where the

C-plane is always handled by the 3GPP access, as clarified in Section 2.2.3 on the

interworking with non-3GPP access.

Note: Local area data network (LADN) provides local area specific services by

infrastructure deployment in the local area such as stadium, airport and shopping mall.

The infrastructure can be configured by components of MiEdge RAN.

2.1.2 MEC Service Function (MSF)

The required features for the MSF block are summarized as follows:

• Policy control: It provides network configuration policy based on user

subscriptions, service requirement and service deployment in the network,

• Resource management: It enables resource optimizations and traffic prediction

by providing resources/statistics information of radio and computation

resources,

• Resource information sharing: It acts as an interface for sharing

radio/computation resource information in non-3GPP access network based on

business agreements between network operators. It may be implemented as

the access network discovery and selection function (ANDSF)-like function.

It is preferable that such information sharing happens in near real time manner,

it means all the time that there’s a network connection, rather than based on

fixed periodic updates,

• Data repository: It stores data including subscription data, policy data,

structured data for exposure, application information, etc. Service provider

may access a part of this data via NEF (Network Exposure Function) in an

authorized way,

• Context management: It collects various context information from user,

network, edge cloud, service-related policy, etc.,

• Host for optimization algorithms: Holding and processing all the mentioned

above information, the MSF is capable of providing optimal resource

allocations for the MiEdge architecture.


Date: February 2019

Public Deliverable

5G-MiEdge

2.2 5G architecture discussed in 3GPP

The two most active standardization bodies currently working on MEC-related topics

are the ETSI and the 3GPP groups. It is very important to be able to map the required

MSF functionalities mentioned in the previous section to equivalent functions as

defined by the ETSI and the 3GPP specifications. This section elaborates exactly on

that mapping.

In order to fit and make work our novel architectural concepts into the planned

forthcoming 5G mobile network, it is necessary to monitor and survey the discussions,

completed and ongoing, in the relevant 3GPP standardization groups. Our results

should not be in conflict with decisions taken in 3GPP. Therefore, throughout the

lifetime of the project we have been constantly surveying the discussions taking place

in 3GPP groups.

2.2.1 5G network architecture

3GPP defines architectures for 5G mobile networks in [TS 23.501]. Fig. 2-2 shows an

architecture applicable for 5G-MiEdge to access two data networks (DN), e.g. the local

and the central one. A MEH deployed closer to UE is preferable in order to deliver

services or contents to a UE with low latency in communication. So, closer deployment

of UPF to UE is necessary to bring closer deployment of MEH to UE.

However, it is worth noting that according to the latest available specifications, no

direct interface is defined between RAN and DN. As a consequence, when one

considers MEH deployments, it is necessary to have UPF together with MEH at the

same physical location of a radio base station.

Fig. 2-2 5G architecture taken as baseline for the MiEdge architecture (source:

TS 23.501)

AMF SMF

UE (R)AN UPF DN

N8

N11

N3 N6

N2 N4N1

AFN7

DN

N6

UPF

N4

N9

PCF N5

AUSF N13 UDM

N10N12

N15

NSSF

N22


Date: February 2019

Public Deliverable

5G-MiEdge

2.2.2 RAN architecture

This section explains important aspects of RAN architecture that have been discussed

in 3GPP to configure multi-Radio Access Technologies (RAT) environments in 5G

deployment.

2.2.2.1 Small cell deployment and C/U split

MmWave access will serve very high-speed radio communication, but it is also known

for the disadvantages of small area coverage and frequent link interruptions due to

blocking by obstacles present in the line-of-sight (LOS) propagation. Therefore, in real

deployments, mmWave access will be utilized in combination with non mmWave radio

access operating in lower frequency bands.

We consider a scenario in which a macro cell, operating in the sub-6GHz domain, can

cover a wider area and can provide a stable C-plane functionality. In addition to it,

small cells, operating in mmWave frequency bands, are deployed within the macro cell

coverage in an overlaid manner, in order to provide U-plane functions with broader

bandwidth. This concept is called C/U (C-plane / U-plane) split. An image of this

deployment is described in Fig. 2-3.

Fig. 2-3 C-/U-plane split in a macro-/small-cell setup

2.2.2.2 Dual Connectivity

Along with C/U split, it is fundamental to discuss the dual usage of the U-Plane at the

macro and at the small cell levels. This concept is called Dual Connectivity (DC), as

defined in [TS38.300]. The interworking between macro and small cells is very

important in DC. Fig. 2-4 represents the U-plane protocol stack for interworking

between macro cell and small cell in DC. Macro and small cells are called Master

gNodeB (MgNB) and Secondary gNodeB (SgNB) respectively from the standpoint of

the C-plane.

In Fig. 2-4(a), it is impossible to decode U-plane data on the SgNB (mmWave small

cell) side. Therefore, in this case, the UPF will be located behind the MgNB (macro


Date: February 2019

Public Deliverable

5G-MiEdge

cell). In Fig. 2-4(b), it seems possible to locate MEH resource behind the SgNB.

However, it should be noted that there is a connection (Xn interface) between MgNB

and SgNB. Multiple small cells will be deployed inside a macro cell coverage.

Multiple Xn connections will be needed between macro and small cells so to guarantee

an effective interworking.

(a) MgNB bearers for dual connectivity (b) SgNB bearers for Dual Connectivity

Fig. 2-4 Protocol stack for dual connectivity

2.2.2.3 C-RAN and D-RAN

Centralized RAN (C-RAN) is one of the key aspects in 5G mobile networks

deployments. In order to minimize infrastructure size at cell site, function split is

deployed in the RAN part. In fact, just antennas and the RF part of a RAN, the so

called Remote Radio Heads (RRH), will be deployed at the site and the baseband and

the control part of a RAN will be deployed in a centralized manner. This set-up is what

is called C-RAN. When the baseband and control part of RAN will be virtualized, they

might be realized on general purpose computer. Thanks to this centralized deployment

of RAN, it will be easier to make signaling and data paths between cells for

interworking. In this case, MEC computation resources will be deployed with

centralized part of RAN at the same location as depicted in Fig. 2-5.

Fig. 2-5 Example of MEC deployment with C-RAN

The radio base station will be downsized in small physical form, especially in higher

frequency bands like mmWave to enable outside installation in a simple and easy way,

for example on light poles along the street. It may be possible to deploy MEC functions

MgNB

PDCP

RLC

SgNB

PDCP

RLC

Xn

RLC

MAC MAC

MCG

Bearer

MCG

Split

Bearer

SDAP SDAP

SgNB

PDCP

RLC

MgNB

PDCP

RLC

Xn

RLC

MACMAC

SCG

Split

Bearer

SCG

Bearer

SDAP SDAP

Baseband & control

RRH

RRH

RRH

RAN

UPF

MEC

At the same location


Date: February 2019

Public Deliverable

5G-MiEdge

inside such equipment of the radio base station. However, U-plane data flow is

ciphered on radio link. Therefore, in order to transfer U-plane data flow to that MEC,

it is necessary to install entire functions of radio base station at that same site, so to

decode that ciphered flow. This is a conventional way of radio base station deployment

called D-RAN (Distributed RAN).

As a consequence of the constraints given by the chosen architecture, in the D-RAN

case it will be more difficult than in the C-RAN case to make signaling and data path

between cells for interworking to realize C/U split and DC.

2.2.3 Interworking with non-3GPP access

Fig. 2-6 depicts a network architecture for 5G mobile network interworking with non-

3GPP access network, as defined for Release-15 in [TS23.501]. 3GPP defined N3IWF

(Non-3GPP Interwork Function) for interworking with non-3GPP access network for

5G core network. It is considered for Release-15 only on untrusted non-3GPP access.

Note: The trust relationship of a non-3GPP access network is made known to the UE

with one of the following options [TS23.402]:

1) If the non-3GPP access supports 3GPP-based access authentication, the UE

discovers the trust relationship during the 3GPP-based access authentication,

2) The UE operates on the basis of pre-configured policy in the UE.

For this reason, 3GPP considers secure tunneling protocol between N3IWF and UE

for Release-15, which is depicted in the protocol stack layers as in Fig. 2-7, defined in

[TS23.501].

Fig. 2-6 Architecture for 5G core network with non-3GPP access

Untrusted Non-

3GPP AccessUE

N3IWF

3GPP

Access

Data Network

HPLMN

Non-3GPP

Networks

UPF

N3 N6

Y1

Y2

AMF SMF

N2

N2N4

N3

NWu

N11

N1

N1


Date: February 2019

Public Deliverable

5G-MiEdge

Fig. 2-7 User plane for untrusted non-3GPP access

Due to the fact that there is an IPSec (Internet protocol security architecture) tunnel

between N3IWF and UE, there is no insertion point for a MEH between them, because

they communicate in a ciphered way. As a consequence, it is not possible to deploy

MEC on the non-3GPP access network side with interworking with this configuration

of 5G mobile network.

In order to deploy MEH resources on the non-3GPP access network side with

interworking between 5G mobile network, the architecture needs to be extended. Some

sort of business agreement will be concluded for service provisioning from MEC

resources on the non-3GPP network side when a 5G mobile operator will provide

services from the non-3GPP network. In such a case, the non-3GPP access can be

considered as the trusted one.

Considering non-3GPP access network, currently 3GPP is not considering MEH

deployment on the non-3GPP access network side in the configuration for the

interworking between 5G mobile and non-3GPP access network. 3GPP is considering

non-3GPP access as an alternative for access technology and suggesting the same core

network facilities to be utilized as 5G mobile for the interworking. In that sense, it is

aiming at service/content provisioning from the same facilities to both 5G mobile and

non-3GPP access network. However, the MEH installation point tends to be far from

an AP, and it is impossible to insert a MEH on the non-3GPP side. To solve this issue,

the architecture needs to be extended as shown in Fig. 2-8.

IP

UPF

N3

N3IWF UE

NWu

IPSec

IP

L2

L1

UDP/IP

L2

L1

UDP/IP

L2

IPSec GTP-U

L2

L1 L1

PDU Layer PDU Layer

GRE GRE Relay

GTP-U


Date: February 2019

Public Deliverable

5G-MiEdge

Fig. 2-8 Architecture extension for MEH access on the non-3GPP access side

In this architecture, it is necessary to implement a function for local breakout to access

a local MEH or data network (the Internet) directly via the AP. Routing for that local

breakout will be provided by a gateway (GW) router. The routing control may be

realized by control from 5G mobile network. But the function for local breakout may

be out of 3GPP scope.

Speaking of combination between 5G mobile and non-3GPP access networks,

interworking on user plane will not be tight because of the configuration depicted in

Fig. 2-8. However, one of the objectives of the 5G-MiEdge project is the support of

“control signaling” for multi-connectivity. In order to make that happen, it is necessary

to identify what is the potential interworking that is relevant for the project scope.

Considering interworking between two networks, it is necessary to have a business

agreement between the operators of those two different networks. We therefore

propose three novel potential business agreement levels between 5G mobile operators

and non-3GPP access network operators, as described here below.

1) User authentication

In this case we propose an interworking between user authentication servers (AAA:

authentication, authorization and accounting). User subscription of the 5G mobile

network is used for non-3GPP access too. That will allow UEs to benefit of seamless

connectivity between two operators’ networks. Fig. 2-9 shows user authentication

agreement between 5G mobile and non-3GPP operators. A user (UE) subscribing

the 5G mobile network has been registered to access the non-3GPP network in

advance. When the UE requests to access to the non-3GPP access network, that

request is transferred to the AAA of the 5G mobile network, and then will be granted.


Date: February 2019

Public Deliverable

5G-MiEdge

Fig. 2-9 Potential agreement levels between 5G mobile operators and non-3GPP

access network operators – User authentication case

2) AP discovery and trigger for connection

In this agreement level, assist information is provided by the 5G mobile operator

for AP discovery, policy management and triggers for connection. As shown in Fig.

2-10, the policy control function (PCF) will be the source entity of this information,

as identified in 3GPP discussions for the 5G system definition. So far, in the 5G-

MiEdge discussions, PCF has not yet been mentioned, before the introduction of

this architecture, but it can be realized as a part of function in MSF. In legacy

networks, e.g. LTE networks, this functionality is defined as the ANDSF.

Fig. 2-10 Potential agreement levels between 5G mobile operators and non-

3GPP access network operators – AP discovery and trigger for connection case

3) MEC service partnering

This business agreement enables MEC service provisions from MEH on the non-

3GPP operator side by the 5G mobile operator. The 5G mobile operator rents for its

services a part of MEH resources of the non-3GPP operator. Such MEH resources


Date: February 2019

Public Deliverable

5G-MiEdge

will be seen as a part of its own MEC resources for the 5G mobile operator. In this

configuration, a control interface is required among MEH resources via secured

connection, e.g., Virtual Private Network (VPN). The MSF in the 5G mobile side

will orchestrate MEC resources on the non-3GPP network side, but will of course

be limited on its rented parts of resources. Fig. 2-11 depicts the architecture for the

MEC server partnering case.

In this scheme, it is necessary to have seamless connectivity for UE by a 5G mobile

subscription. Information of MEC service deployment may be available via 5G

mobile access. However, procedure for that information provisioning will depend

on implementation and might not be ever standardized. It is also available for UE

in this scheme to receive information about which APs provide subscribed MEC

services by using ANDSF-like function.

Fig. 2-11 Potential agreement levels between 5G mobile operators and non-

3GPP access network operators – MEC server partnering case

Based on the considerations elaborated above on potential business agreements, an

architecture that combines 5G mobile networks with and non-3GPP access networks

is shown in Fig. 2-12. As seen in the figure, interworking of user plane is not seen

between two networks. Only the interworking among MEHs (Mp3) and their

orchestration (NL2) will be considered.


Date: February 2019

Public Deliverable

5G-MiEdge

Fig. 2-12 Architecture for interworking between 5G mobile and non-3GPP

access network as proposed by the 5G-MiEdge project

2.3 MEC architecture

Multi-access Edge Computing (MEC) aspects represent also an important part of the

work on which the 5G-MiEdge project focused. Throughout the project lifetime we

have been constantly surveying the ongoing discussions in ETSI MEC, to monitor and

follow closely the latest decisions taken on the MEC architecture.

Fig. 2-13 is the reference architecture for MEC as defined in [MEC003].

Fig. 2-13 Mobile Edge system reference architecture

The mobile edge system consists of the mobile edge hosts and the mobile edge

management necessary to run mobile edge applications within an operator network or


Date: February 2019

Public Deliverable

5G-MiEdge

a subset of an operator network. In the following a short description of the main entities

involved is provided:

• The mobile edge host (MEH) is an entity that contains a mobile edge platform and

a virtualisation infrastructure which provides compute, storage, and network

resources, for the purpose of running mobile edge applications.

• The mobile edge platform (MEP) is the collection of essential functionalities

required to run mobile edge applications on a particular virtualisation

infrastructure and to enable them to provide and consume mobile edge services.

The MEP can also provide services.

• The Mobile edge applications are instantiated on the virtualisation infrastructure

of the mobile edge host, based on configuration or requests validated by the

mobile edge management.

The mobile edge management comprises the mobile edge system level management

and the mobile edge host level management.

• The mobile edge system level management includes the mobile edge orchestrator

(MEO) as its core component, which has an overview of the complete mobile edge

system.

• The mobile edge host level management comprises the mobile edge platform

manager (MEPM) and the virtualisation infrastructure manager, and handles the

management of the mobile edge specific functionality of a particular MEH and

the applications running on it.

Finally it is worth mentioning that ETSI has issued in January 2019 a very recent

update to the ETSI GS MEC 003 specification “Multi-access Edge Computing (MEC);

Framework and Reference Architecture” [MEC003_19]. It is important to notice that

the ETSI architecture has remained pretty stable, and that the unique enhancement

compared to the previous version is the addition of a different kind of architecture for

MEC in NVF scenarios, nothing really in focus of 5G-MiEdge scope.

We can therefore claim that the work the 5G-MiEdge consortium has been focusing

on till now on the architectural side, notwithstanding it was devised four years ago and

the project started three years ago, is capable of being still relevant, against the latest

updated from the ETSI standardization body.


Date: February 2019

Public Deliverable

5G-MiEdge

3 Integration of mmWave edge cloud into 5G mobile

networks with inter operable control plane

This chapter describes mainly the work done in WP3 Task 3.1. The main novel

contributions are related to new mechanisms to integrate the newly proposed mmWave

edge cloud into a 5G cellular network enhanced with an innovative control plane.

The chapter is divided in three main sections.

3.1 Control signaling for 5G MiEdge registration and radio/

computational resource map

This chapter describes basic registration ways of radio and communication resources.

Basically, registration ways will depend on vendor’s implementation.

3.1.1 Radio and computational resource managements

Radio and computational resources are managed by their own management systems

individually. Registrations of those resources are performed manually or automatically,

e.g., in a Plug and Play fashion. The way such registrations are performed depend on

how the vendors implemented the management systems.

Radio (RAN) resources are managed by access and mobility management function

(AMF) directly in the 5G mobile networks, as depicted in Fig. 2-2. Then, multiple

AMFs are connected to an operation and management (OAM) system. Radio resources

can be supervised by the OAM system.

In non-3GPP radio access cases, radio resource management is performed by access

point controllers (APC), generally.

Computational resources are managed by the mobile edge orchestrator (MEO), as

depicted in Fig. 2-13.

3.1.2 Radio and computational resource mapping

Actually, MEHs are considered as a part of the data network (DN) from a standpoint

of 5G mobile networks. However, the 5G mobile networks require to maintain policies

for setting a data communication path between a UE and an appropriate MEH for a

requested service. Those routing policies are provided by PCF. Then, routing policies

are set on a UPF to access the appropriate MEC host.

In order to serve a requested service with shorter communication latency, it is

necessary to consider network distance between a UE and a serving MEC host.

Therefore, it is required to have geographical mapping between UPF and MEC host in

addition to logical mapping. That resource mapping may be provided by the MEO. A

proper management system of access networks requires to provide logical and

geographical resource information of RANs, APs and UPFs on OAM system to the

MEO. Provisioning interfaces of these mapping information will be implemented in

real-life scenarios based on vendor-specific implementations. Fig. 3.1-1 shows an

architecture image of these resource mapping.


Date: February 2019

Public Deliverable

5G-MiEdge

Fig. 3.1-1 Image of resource mapping architecture [based on ETSI and 3GPP

specs]


Date: February 2019

Public Deliverable

5G-MiEdge

3.2 Control plane delegation to support multi-connectivity in

signaling for integrated heterogeneous networks

This section describes the signaling needed to access an appropriate MEC host for a

requested service. Basically, control signaling has been discussed in 3GPP to support

multi-connectivity in a heterogeneous networks environment. As mentioned in Section

2.2.3, the interworking with non-3GPP access has also been discussed in 3GPP.

However, those discussions didn’t touch on non-3GPP access on control signaling,

which are applicable to the novel 5G-MiEdge architecture.

The following sections identify procedures for accessing a service on MEC host from

3GPP RAN and non-3GPP access.

3.2.1 Access from 3GPP RAN to MEC host

Fig. 3.2-1 shows a procedure for PDU (packet data unit) session establishment, as

defined in [TS23.502]. Based on that, the procedure for accessing to a requested

service on a MEC host is identified as follows (for a more detailed explanation one

can refer to the provided references):

Step 1: The UE initiates the PDU Session Establishment procedure by transmitting a

PDU Session Establishment Request message. The PDU Session Establishment

Request message may include the single network slice selection assistance

information (S-NSSAI) or the data network name (DNN). Here, the S-NSSAI means

the network slice and the DNN means the name of the data network, which are

allowed to access for the requested service at the UE’s registered area. The AMF

receives from the RAN the message together with user location information (e.g.

cell Id).

Step 2: The AMF selects a SMF based on the user location, the S-NSSAI and the DNN.

Step 3: The AMF invokes the Nsmf_PDUSession_CreateSMContext Request to

associate with the selected SMF.

Step 4: The SMF registers with the unified data management function (UDM) using

Nudm_UECM_Registration message for a given PDU Session.

Step 5: the SMF creates a session management (SM) context and responds to the AMF

by providing an SM Context Identifier.

Step 6: If the SMF needs to perform authorization or authentication during the PDU

Session establishment by a DN-AAA (Authentication, Authorization and

Accounting) server, the SMF triggers the PDU Session establishment authentication

or authorization procedure.


Date: February 2019

Public Deliverable

5G-MiEdge

AMF PCF UDM (R)AN UE

7b. Session Management Policy Establishment or Modification

10a. N4 Session Establishment/Modification Request

1. PDU Session Establishment Request

UPF SMF

10b. N4 Session Establishment/Modification Response

9. Session Management Policy Modification

11. Namf_Communication_N1N2MessageTransfer

13. AN-specific resource setup (PDU Session Establishment Accept)

12. N2 PDU Session Request (NAS msg)

14. N2 PDU Session Request Ack

15. Nsmf_PDUSession_UpdateSMContext Request

16a. N4 Session Modification Request

16b. N4 Session Modification Response

19. IPv6 Address Configuration

First Uplink Data

First Downlink Data

8. UPF selection

2. SMF selection

17. Nsmf_PDUSession_UpdateSMContext Response

7a. PCF selection

DN

6. PDU Session authentication/authorization

3a. Namf_PDUSession_CreateSMContext Request

4a-4b. Registration/ Subscription retrieval/

Subscription for updates

20. Unsubscription / Deregistration

5. Namf_PDUSession_CreateSMContext Response

18. Nsmf_PDUSession_SMContextStatusNotify

Fig. 3.2-1 Procedure for PDU session establishment [TS23.502]

Step 7a: If dynamic policy and charging control (PCC) is deployed, the SMF performs

PCF selection.


Date: February 2019

Public Deliverable

5G-MiEdge

Step 7b: The SMF may perform a session management policy establishment procedure

to establish a PDU session with the PCF and get the default PCC rules for the PDU

session.

Step 8: The SMF selects one or more UPFs for the PDU session. The SMF also

allocates an IP address for the PDU session.

Step 9: The SMF may perform a session management policy modification procedure

to report some events to the PCF that has subscribed. If dynamic PCC is deployed,

the SMF notifies the PCF with the allocated UE IP address.

Step 10: The SMF initiates an N4 session establishment procedure with the selected

UPF.

10a. The SMF sends an N4 Session Establishment/Modification Request to the UPF

and provides Packet detection, enforcement and reporting rules to be installed on

the UPF for this PDU Session.

10b. The UPF acknowledges by sending an N4 Session Establishment/Modification

Response.

Step 11: The Namf_Communication_N1N2MessageTransfer message (with various

configuration information) is sent to AMF from SMF.

Step 12: N2 PDU Session Request (with various configuration information and PDU

Session Establishment Accept) is sent to RAN from AMF.

Step 13: The RAN may issue access network specific signalling exchange with the UE

that is related with the information received from SMF. For example, a radio

resource control (RRC) connection reconfiguration may take place with the UE

establishing the necessary RAN resources related to the QoS rules for the PDU

Session request received in Step 12.

RAN forwards the message (with various configuration information and PDU

Session Establishment Accept) provided in Step 12 to the UE.

Step 14: N2 PDU Session Response is sent to AMF by RAN.

Step 15: The AMF forwards the N2 SM information received from RAN to the SMF

in Nsmf_PDUSession_UpdateSMContext Request message.

Step 16a: The SMF initiates an N4 Session Modification procedure with the UPF. The

SMF provides Tunnel Info to the UPF as well as the corresponding forwarding rules.

Step 16b: The UPF provides an N4 Session Modification Response to the SMF.

Step 17: The SMF sends Nsmf_PDUSession_UpdateSMContext Response to the AMF.

Step 18: [Conditional] The SMF sends Nsmf_PDUSession_SMContextStatusNotify

(Release) message to the AMF if the PDU session establishment is not successful

any time after Step 5 during the procedure.

Step 19: In case of PDU Type IPv6, the SMF generates an IPv6 Router Advertisement

and sends it to the UE via N4 and the UPF.

Step 20: If the PDU Session cannot be established, then the SMF shall deregister for

the given PDU Session.


Date: February 2019

Public Deliverable

5G-MiEdge

To summarize these procedures above in terms of access to a service on MEC host, the

PCF provides routing rules to UPF via SMF to configure communication path between

a serving MEC host and UE.

Dual connectivity is basically provided with C/U split in case of using mmWave link

for accessing the service on the MEC host as described in Section 2.2.2. A part of

configuring radio resources may be performed based on a policy provided by PCF. In

terms of routing for accessing a MEC host, the policy affects routing rules on UPF but

should not affect selection of radio links (radio resources). Because quality of radio

links should not be deteriorated due to influences from a MEC service. The radio

resources will be allocated independently by AMF based on received signal strengths

and interferences. The QoS requirements are considered for this radio resource

allocation. Such controls for radio resource allocation are the same as existing radio

resource controls.

Control for handover to a different MEC host, due to UE’s movement or resource

shortage on a MEC host, is mentioned in Section 3.3.

3.2.2 Access from non-3GPP access to MEC host

As described in Section 2.2.3 (Fig. 2-9 and 2-10), assist information can be provided

to UE for access to a MEC service on the side of non-3GPP access by the PCF via

3GPP RAN. When the UE discovers an AP which matches the provided information,

the UE connects to that AP to access a MEC service. It depends on the policy possessed

by UE whether the UE connects to the AP.

Regardless of the assist information from the PCF, it is preferably necessary for UEs

to obtain the information of MEC services which can be served through the discovered

AP before connecting to the AP. As a potential measure for that information provision,

the standard IEEE802.11aq [11aq] defined in the IEEE (The Institute of Electrical and

Electronic Engineers) 802.11 working group may be used. This standard provides a

way for service discovery before association with an AP. But it may need extension for

transmitting information of MEC services. Broadcasting information of available

MEC services is also a potential way to notify the UE. A new task group (TGbc) has

just started discussions to standardize broadcast service in wireless local area network

(LAN) within the IEEE802.11 working group since January 2019. [11bc]


Date: February 2019

Public Deliverable

5G-MiEdge

3.3 Multi-layer radio/computation resource management for

enhanced mobility support

This chapter provides novel procedures that are intended to improve MEC service

continuity and enhance workload placement, considering UE’s mobility and MEC

resource limitation.

3.3.1 Mobility support for MEC service

The “Follow Me Cloud” (FMC) concept is proposed as a scheme to maintain service

provision environment for a user, during the user’s movements [TK13-1]. FMC

consists in migrating all or a portion of an ongoing IP service from a data center to

another one, which is more appropriate according to some Key Performance Indicators

(KPI). At a very recent (December 2018) ETSI MEC ISG meeting the contribution

[SMN18] identified some notable patterns of application (service provision

environment) for a more effective mobility support, as shown in Fig. 3.3-1.

Fig. 3.3-1 Selected use cases of UPF selection [SMN18]

In the figure above ‘App’ denotes an application running meanwhile the user is moving,

and ‘5GC’ means 5G Core network. It is desirable for App to follow the UE (a car in

the figure) as the UE moves across the network. It is necessary for the mobility support

to have controls to migrate App to another MEH as the case of normal flow. However,

App may stay on the same MEH even if the UE moves to the coverage of another

MEH when the target MEH is not available, e.g., in case of resource shortage. In that

case, the service of App is provided by the MEH in which the App runs. Two types of

data flow are stated in the figure as the abnormal flow. The first one is maintaining

service continuity by data flow between two MEHs. The second one is maintaining

service continuity by the data flow routed from a UPF to another UPF. The authors say


Date: February 2019

Public Deliverable

5G-MiEdge

this routing can be realized by the 5G specifications. Moreover, App migration

consumes backhaul resources to transfer service provisioning environment between

MEHs. Therefore, the authors of [TK13-1] propose suspension of App migration to

reduce the consumptions of backhaul resources while comparing balance between cost

and gain of the App migration.

In order to realize an effective service continuity as depicted above, the application

requires a notification when an inter-UPF handover is performed.

In case an FMC is available, reassignment of IP address on the UE is considered as the

event of the inter-UPF handover. Fig. 3.3-2 shows the flow chart of initial FMC session

establishment and FMC session migration. In the figure, an IP address is assigned from

the packet gateway (P-GW) to the UE and PDN connection is established between the

UE and the P-GW. This P-GW is an entity name in the LTE network, equivalent to the

UPF in the 5G network. When the inter-UPF handover is recognized on the UE (Step

(7): a new PDN connectivity @IP2), the UE queries the FMC manager regarding the

necessity of the migration of the ongoing service from one data center to another one,

which is more fitting to maintain the running service provided by the application (Step

(8)). Then the FMC manager determines to which data center the service should be

migrated (Step (9)). Finally the service migration is performed (Step (10) and (11)).

It is in principle possible to make use of different ways for the handover event

notification, as described in the following.

Fig. 3.3-2 Flow chart for initial FMC session establishment and FMC session

migration [TK13-1]


Date: February 2019

Public Deliverable

5G-MiEdge

3.3.1.1 Network exposure function (NEF)

The NEF introduced by the 5GC can expose the event of the inter-UPF handover

[TS23.501] to MEO, as illustrated in Fig 3.3-3. When the SMF detects the inter-UPF

handover, that event is notified to the MEO through the NEF. Then, the MEO

determines whether a service migration is required between MEC hosts or not.

Fig. 3.3-3 Handover event notification via NEF

3.3.1.2 Radio network information (RNI) application interface (API)

The ETSI MEC ISG defined a specific RNI API [MEC012] to obtain a relevant set of

information from radio base stations. Fig. 3.3-4 shows a flow example of event

notification following a cell change event. In this figure, the service consumer is the

mobile edge (ME) service on the ME platform of a MEH, and the RNIS is the RNI

Service that is embedded in the radio base station. Procedures are described based on

the representational state transfer (REST) protocol.

The cell handover event is notified when the cell handover occurs for a specific user,

if the cell handover is registered as a subscribed event. It is obviously required for this

procedure to take place in a radio base station that implements the RNIS functionality.

However, no discussion on the details and assumptions related to such an

implementation has happened until now in the standardization activities. More studies

are needed to identify the necessity and the benefits of this implementation.


Date: February 2019

Public Deliverable

5G-MiEdge

Fig. 3.3-4 Flow of receiving RNI event notifications on cell changes [MEC012]

3.3.2 Problems of conventional mobility support

The FMC concept assumes that a migration of Virtual Machines (VMs) is required

between MEHs. During the VM migration, service provision is disrupted. It also

consumes backhaul bandwidths to migrate the object between MEHs. An analytical

model of the VM migration cost in the FMC is proposed in [TK13-2]. The VM

migration cost (CostM) is based on two main factors: the time required for the migration

and the backhaul resources spent for transferring the VM between the computing nodes.

On the other hand, there is a gain (GainM) if the VM migration is initiated as soon as

the UE starts experiencing lower latency (for instance when computational tasks are

executed in UE proximity), and backhaul resources do not have to be allocated for

transferring the computation results back to the UE.

Reference [KTC14] is aimed at designing a decision policy that would determine

whether the VM migration needed to be performed by using the analytical model in

[TK13-2]. As mentioned above, a trade-off between migration cost (CostM) and

migration gain (GainM) is the key to an effective decision policy in the FMC. An

example of this decision is shown in Fig. 3.3-5.

Fig. 3.3-5 VM migration principle [KTC14]

UE UE

UE movement

BS1 BS2 BS3 BS4

MEH1 MEH4MEH2

VM VM

VM migration

HO HO HO

CostM>GainM

CostM>GainM

CostM<GainM

MEH3


Date: February 2019

Public Deliverable

5G-MiEdge

In this figure, a UE that is using a service run by MEH1 moves from Base Station 1

(BS1) to BS4. When the UE crosses the coverage boundary of a BS, an inter-cell

handover is performed between two adjacent BSs, but the related VM migration is

initiated only after a series of inter-cell handover, exactly when the UE reaches the

BS4 coverage, i.e. when the overall cost and gain calculation for the whole operation

is convenient, i.e. when the overall CostM < GainM. Numerical results in [KTC14]

show that the proposed migration decision always achieves the maximum expected

gain when compared to no VM migration and VM migration at every inter-cell

handover.

Another paper [SA16] proposes a more advantageous trade-off between VM migration cost and VM migration gain by selectively performing VM migration to reduce the VM migration cost. Moreover, UE mobility prediction [NHB16] and adaptive compression rate change [HAC15] are also proposed to minimize the VM migration time when VM migration is going to be performed.

Various efforts have been made to optimize VM migration as described above. We identified the following factors as potential problems for the VM migration.

1) Service Disruption Time

A live VM migration is accompanied by a service disruption time because it takes time to transfer the VM to another MEH, even when the overall size of the objects to be migrated is very small. This most probably is a problem for latency constrained applications.

2) Backhaul Resource Consumption

The main focus of related work on VM migration was on how to reduce backhaul resource consumption while shortening the required VM migration time. To reduce resource consumption means, in other words, to delay the VM migration to the optimal MEH until the UE has moved a certain distance in the network away from the currently serving MEH. This may not be acceptable for latency constrained applications. Therefore, for latency constrained applications, probably the best solution is to always perform the VM migration. In such a case, resource consumption in backhaul is one of the problems that needs to be resolved.

3) Backhaul and Access Resource Availability

There are situations in which a migration, even if overall beneficial for service

continuity, simply cannot take place due to lack of available resources in the backhaul,

in the access network or in both. For instance, during network congestions in very

densely populated areas or, at the opposite, in poorly served areas in the country side,

where equipment have not been upgraded since a while due to lack of economic returns,

and therefore MEC technologies might have never been deployed. In such cases

operators and network providers have to balance cost vs. potential obtained benefits of

instantiating more capacity, or of upgrading old equipment. Usually this cost/benefit

assessments are very difficult to run and are strongly dependent on volatile and non-

steady or random parameters, like the change of the number of citizens in a selected

area (in case of arrival of an important number of new citizens in a relatively short

time) or at the opposite in case of people leaving the area due to natural catastrophic

events or heavy economic problems.

4) Control complexity


Date: February 2019

Public Deliverable

5G-MiEdge

There are many aspects to be considered for VM migration such as the service

disruption time, the backhaul resource consumption, pause/resume of the service for a

user, delay requirements for the service. It is necessary to design control measure

considering these aspects. Sometimes, there may be requirement conflicts between a

network operator and service providers. It will be very complicated to organize VM

migration policy in easily understandable control algorithm. Such a complex algorithm

may require many signaling and can potentially heavily affect the system scalability.

3.3.3 Service continuity via carry-on state service handover

We propose a different way to continue the service without VM migration in the

backhaul when the optimal MEH changes [YS18]. Our approach resumes the ongoing

service at the optimal MEH by the UE sending the service state parameters held in the

UE itself as progress information of the service to the target MEH. We refer to this

proposed method as “carry-on state service handover”, (hereinafter referred to as

CaSH).

Fig. 3.3-6 illustrates our proposed CaSH concept. MEH1 and MEH2 represent the

current serving MEH and the new optimal MEH, respectively. The cell handover is

performed from BS1 to BS2 for UE1. Then the optimal MEH is changed from MEH1

to MEH2 to maintain the service. The important point is that the provisioning

environment for that service is running on these MEHs constantly regardless of the

service request from a UE, or is pre-fetched, without any object migrations between

corresponding MEHs, based on UE’s mobility prediction. When service provisioning

is requested from a UE, a service instance for that UE is created on the service

provisioning environment on an MEH. Next, the MEH (the service provisioning

environment) provides the requested service to the UE. When cell handover is

performed while the data communication session is being established for UE1, the SMF

of the mobile network, not depicted in the figure, identifies the necessity of changing

the serving MEH from the current MEH (MEH1) to the new optimal MEH (MEH2) for

a service requiring low latency communication. We assume the service handover is

always performed between two MEHs whenever a cell handover with the optimal MEH

change is performed, because the service requires low latency communication. Then

the SMF notifies the optimal MEH change to UE1 via BS2. The notification message

includes the IP address of MEH2. When UE1 receives the notification message, UE1

sends a request message to resume the service with state information of the service held

inside UE1. Upon receiving the service resume request, MEH2 (service provision

environment) creates a service instance for UE1 based on the state information and

starts service provision for UE1. Summarizing the steps described above, CaSH

maintains service provision by re-creating the service instance for the UE on the new

optimal MEH based on the service state information provided by the UE, not by VM

migration.


Date: February 2019

Public Deliverable

5G-MiEdge

Fig. 3.3-6 Proposed CaSH

Fig. 3.3-7 depicts the novel signaling for CaSH integrated into the inter-UPF handover

procedure defined in [TS23.502]. In the figure, S-xxx and T-xxx are Source-xxx and

Target-xxx respectively, representing origin and destination of the handover. gNB

(gNodeB) is a base station in the 5G mobile network. The UPF is the anchor point of

the data communication session. An MEH is always located behind a UPF. The inter-

UPF handover represents handover between BSs when a user session is transferred to

another UPF. This necessitates reconfiguration of the connection between the UPF and

the MEH currently serving the UE at that time. Dashed arrows in steps 6, 7 and 8 in the

figure are novel signaling that has not been defined in the original procedure.

Considering a procedure without steps 6 and 7, i.e. the procedure “as-is” defined in

[TS23.502], the UE does not recognize its movement across UPFs. Because the UE

does not receive any information on the optimal UPF change, i.e. the optimal MEH

change, the UE cannot trigger the VM migration process to the MEC management

system.

Thanks to the CaSH concept, we propose to add a message notifying the optimal MEH

change (steps 6 and 7) to the original procedure. The message is sent to the UE when

the optimal MEH is identified based on the service deployment information shared by

the MEC management system. The message may include the IP address of the new

optimal MEH in addition to the notification. If the IP address of the new optimal MEH

is not included in the message, the UE may perform the IP address resolution process

for the new optimal MEH. Then, the UE requests resumption of the ongoing service

to the new optimal MEH (step 8). The format of the message in step 8 depends on the

kind of implementation of an application for resuming the ongoing service. However,

we believe that the size of our proposed message is relatively small compared to the

overall payload of the whole procedure for new kinds of applications with strict latency

requirements, such as safety messaging for connected cars, control messages for

drones, quick positioning used for eyeglasses projecting augmented reality (AR) and

so on.

MEH1

HO

Service

UE

1

MEH2

Service

UE

1

UE1 UE1

Trigger

State info.

State State

BS1 BS2


Date: February 2019

Public Deliverable

5G-MiEdge

Fig. 3.3-7 Proposed signaling to notify the optimal MEH change

3.3.3.1 CaSH evaluation results

In this section we present some significant results obtained w.r.t. the CaSH concept.

Basically the analysis models in [TK13-2] are used for the evaluations.

(1) Service Disruption Time (SDT)

We are going to compare the SDT values obtained in a standard set-up with the ones

obtained using our CaSH approach.

To derive the SDT value for the VM migration, we use the same latency model of TCP

as modeled in [TK13-2], assuming data transfer by FTP or similar applications. Based

on the empirical model of TCP validated in [SKV01], the SDT value is derived as

follows:

SDT = [log1.57N + {f(ploss, RTT) N + 4plosslog1.57N + 20ploss} +

(10+3𝑅𝑇𝑇)

4(1−𝑝𝑙𝑜𝑠𝑠𝑊𝑚𝑎𝑥√𝑊𝑚𝑎𝑥 ] RTT + TVMconvert (3.3-1)

where ploss denotes the packet loss, N denotes the number of packets to transfer, Wmax

is the maximum size of the congestion window, RTT is the round trip time of the TCP

connection between the two MEHs and f(ploss, RTT) is equal to:

= 2.32(2𝑝𝑙𝑜𝑠𝑠+4𝑝𝑙𝑜𝑠𝑠

2 +16𝑝𝑙𝑜𝑠𝑠3 )

(1+𝑅𝑇𝑇)3N +

(1+𝑝𝑙𝑜𝑠𝑠)

𝑅𝑇𝑇103

Note that N is equal to ⌈𝑂𝑏𝑗𝑒𝑐𝑡 𝑠𝑖𝑧𝑒

𝑀𝑆𝑆⌉, where MSS is the maximum segment size used by

the TCP connection. Parameters for SDT calculations are in Table 3.3-1, the same as

the analytical model in [TK13-2].

A live migration of a VM can take a long time because of the VM size, which includes

the operating system (OS) and the application software. Containers are a valid

UE S-gNB T-gNB S-UPF T-UPFS-MEH 5GCT-MEH

DL/UL data

1. Handover execution

2. Path switch request

3. T-UPF selection

4. Session establishment request

5. Session modification request

6. Path switch request ack

(with notification of the optimal MEH change)7. Notification of

the optimal MEH change

8. Request of resuming the ongoing service

DL/UL data


Date: February 2019

Public Deliverable

5G-MiEdge

alternative to VM, because they occupy less resources and have lower virtualization

overhead. However, containers are less adaptable than VM's. Furthermore, security

may represent a serious concern because containers, unlike their VM counterparts, are

not fully isolated from their host system, and can therefore be more susceptible to

attacks from a compromised host. To limit the amount of data to be transferred during

a live VM migration, it is convenient to exploit the multi-layer approach proposed in

[MACHEN17]. Using that approach, a VM is split into a base layer (containing the

OS), an application layer (containing the software associated to the specific

application the UE is asking for), and an instance layer, which is specific of the

program executing the UE requests. Exploiting this split, assuming that the base and

application layers are already present (or have been pre-fetched) in the target MEH, it

is only necessary to transfer the instance layer. To further diminish the amount of data

to be transmitted, it is useful to use a file transfer protocol that uses incremental file

synchronization, so that the files to be transferred are only the ones that are different

from those located in the destination.

In the following, we provide some numerical results considering, for simplicity, VM's

whose instance layer occupies a limited amount of memory.

TABLE 3.3-1 Parameter Settings

Parameter Value

Object size 5, 10, 20, 50 kbits

MSS 1460 Bytes

Wmax 300

Ploss 0

TVMconvert 0

Fig. 3.3-8 shows the SDT calculated by Eq. (3.3-1) based on the RTT between the two

MEHs and the VM size to be transferred, with different values ranging from 5 to 50

kbits.


Date: February 2019

Public Deliverable

5G-MiEdge

Fig. 3.3-8 Service Disruption Time (SDT)

As one can see from the results, the SDT is about 30 msec when the RTT is 5 msec

between the two MEHs, even if the VM size is as small as 5 kbits. Some applications

with a packet delay budget of 10 msec are listed in the standardized QoS characteristics

mapping table for 5G in [TS23.501]. This means that the VM migration time cannot

comply with the set requirements, at least for some applications. Moreover, the time

required for transferring a VM object of 5 kbits is almost equal to send nothing for VM

migration. Actually, a VM size could be in the order of 1GB. A VM includes operating

system inside. Even in the case of a container migration not including the operating

system, its size could be in the order of 100MB. Therefore, the results show the

difficulty of realizing an effective VM migration for applications with strict latency

requirements.

On the other hand, if we apply our CaSH concept, very short SDT can be achieved,

and that is obtained regardless of the RTT. That is due to the fact that the RTT value

between the two MEHs plays no role, as the context information are brought by the

UE itself moving along its path and are not exchanged between the MEHs.

The length of SDT for CaSH depends on the chosen implementation, for example 6

msec are assumed in [YS18]. That figure comes out as a sum of 3 msec for signaling

between the UE and the new optimal MEH for resuming the service, plus 3 msec for

process of service resumption on the MEH. However, we have not studied an exact

process for resuming service at the new optimal MEH yet. Therefore, it is necessary

to verify the certainty of these values as the future work.

(2) Backhaul resource consumption

Fig. 3.3-9 shows the amount of resources consumed for VM migration and CaSH in

each 10 sec interval based on UE’s cell handovers. 50 UEs download the file of 5

kBytes in random timing while moving in the simulation range. The VM objects with

a size of 20 and 10 kbits are evaluated. Resource consumption is treated as a cost as

expressed in Eq. (3.3-2) and represents consumed resources (bandwidth) for

transferring the VM object and required signaling (CostVM) as modeled in [TK13-2].

Resource consumption for CaSH (CostCaSH) is assumed as Eq. (3.3-3) and represents

signaling for notification of the optimal MEH change and service resume request.

CostVM = Objectsize + 3 * SIGSIZE (3.3-2)

0

30

60

90

120

150

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

SD

T [

mse

c]

RTT [msec]

VM=50 kbits

VM=20 kbits

VM=10 kbits

VM=5 kbits

CaSH


Date: February 2019

Public Deliverable

5G-MiEdge

CostCaSH = 2 * SIGSIZE (3.3-3)

Obviously, CaSH consumes fewer resources in backhaul than VM migration simply

because it does not transfer data objects between MEHs to support service continuation.

Neither is it necessary to control the relevant MEHs to transfer the VM objects. That

control is handled in the central MEC management system. In this evaluation, the

simulation range was very limited and only 50 UEs were used. Considering a

nationwide service range, tremendous control signaling will be generated frequently

for VM migration. Also, the cost will increase readily as the VM size to be transferred

becomes bigger. As stated in the SDT part, VM size is in the order of 1GB actually,

not 10kbits. But CaSH is not affected by the VM size. Therefore, it is very beneficial

when considering system scalability.

Fig. 3.3-9 Backhaul resource consumption

(3) Service delivery time

Service delivery time is compared among the following 3 cases:

• VM migration (3 cells): VM migration is performed when the UE is 3 cells

away from the serving MEC host to reduce backhaul resource consumption.

• VM migration (1 cell): VM migration is performed at every cell handover.

• CaSH (our proposed way): CaSH is performed instead of VM migration.

The results are shown using a cumulative distribution function (CDF) in Fig. 3.3-10.

CaSH shows the shortest latency distribution for the service delivery time. Indeed it is

the best solution for service continuation between MEC hosts for latency constrained

application.

0

500

1000

1500

2000

2500

3000

3500

4000

0 100 200 300 400 500 600 700 800 900

Cost

[kbits

/10se

c]

Simulation Time [sec]

VM=20kbits VM=10kbits CaSH


Date: February 2019

Public Deliverable

5G-MiEdge

Fig. 3.3-10 Service delivery time

It is worth noting that our analysis on CaSH has been limited to the use case of data

download, and has not taken into consideration data upload [YS18]. That aspect is left

for future works, as well as the analysis that takes into consideration a variety of

different service types that use the CaSH approach.

3.3.4 VM instantiation exploiting mobility prediction

In this section we elaborate on a different approach than the one presented in the

previous section. This second approach intends to save storage resources avoiding to

replicate VMs in too many nodes, while still guaranteeing service continuity during

handovers.

3.3.4.1 Mobility prediction for proactive VM instantiation concept

The proposed idea is based on the following concept: by predicting the trajectory of

the UE during its movement, it is possible to proactively instantiate the VM that is

running the service in the next target MEH. This concept is introduced in [MEC018],

in which it is shown that an application relocation failure most of the times stems from

the following three causes:

• Too late relocation;

• Too early relocation;

• Relocation to a wrong MEH.

For this reason, an accurate prediction of a handover would help avoiding, or reducing

the effects of the mentioned three drawbacks. Indeed, an effective pre-relocation of the

running application can reduce ME service’s end-to-end delay, thus lessening the

perceived performance degradation of the QoS. On the other hand, having VMs

running in all nodes for a given service can be inefficient in terms of storage and

memory resource utilization.

Obviously, the more accurate the prediction is, the more confident the system is on the

next target MEH. The inaccuracy of the prediction can lead to the relocation to a wrong

MEH. To avoid this situation, ETSI introduced the concept of relocation group in

[MEC018], i.e., a set of MEHs where to relocate the application in order to guarantee

high probability of finding it in the actual target MEH. In Fig. 3.3-11, an example of a

0%

20%

40%

60%

80%

100%

0 20 40 60 80 100 120 140 160 180 200

CD

F

Latency [msec]

VM migration (3 cells)

VM migration (1 cell)

CaSH


Date: February 2019

Public Deliverable

5G-MiEdge

relocation group is depicted, considering a scenario where a car moves within an area

where MEC servers are deployed.

Fig. 3.3-11 Relocation group as defined by ETSI [MEC018]

The size of the relocation group, i.e., the number of MEHs belonging to it, depends on

both the wanted accuracy level of the prediction and the application requirements. For

instance, in order to avoid a relocation to a wrong MEH, an inaccurate prediction

would lead to the need of relocating the application in many MEHs, operation that

implies a new instantiation of VMs. At the same time, this large relocation group would

consume a lot of system memory and storage. Moreover, the more stringent is the

latency constraint, the more accurate should be the prediction on the target MEH.

In this section, the service continuity probability and the storage resource savings are

used as performance indicators.

3.3.4.2 Publications related to the mobility prediction problem

Several works investigate the problem of mobility prediction in wireless networks for

optimizing resource utilization, also in the mobile edge computing framework

[ABA17], [OUY18], [PLA16], [OJI18]. In [ABA17]. Mobility prediction is used for

proactive content caching at next nodes in the user trajectory in Information-Centric

networks. Concerning service placement, in [OUY18] mobility prediction is used to

assist the dynamic service placement with a Markov Approximation Based Placement

Policy Search, with the aim of minimizing the long-term time-average service latency

under the constraints of long-term cost budget incorporating a service migration cost.

In [PLA16], the authors exploit users’ mobility prediction to enable flexible selection

of the communication path, together with VM placement. In [OJI18], authors use a

Kalman filter for mobility prediction and select the closest MEH at prediction point

for the requested service. In [ZHANG18], users’ mobility prediction is used to


Date: February 2019

Public Deliverable

5G-MiEdge

minimize the network overhead due to frequent VM migrations, in case of both certain

and uncertain user mobility models.

3.3.4.3 Kalman filtering-based approach to mobility prediction

Our approach is based on Kalman filtering [KAL11], coupled with the concept of

relocation group. Kalman filtering, also known as linear quadratic estimation, is a

procedure that, thanks to a set of measurements taken in an extended time period,

creates an estimate of the unknown variables. Such estimates are supposed to be more

accurate than the ones obtained on a single measurement alone, and that is due to the

introduction of a joint probability distribution over the set of chosen variables in each

observed time frame.

In particular, we use a Kalman filtering algorithm for the prediction of the user position,

based on the work [THEO15], together with a parameter used to shape the area in

which to relocate the application. At every predicted position, the running application

will be relocated in all MEHs that fall in a circle with an area that increases as the error

prediction increases. This concept is depicted in Fig. 3.3-12, where a UE moves along

a trajectory within a scenario with several APs and MEHs. At time 𝑡0 the UE starts

being monitored and its trajectory is being tracked. At time 𝑡1 the predicted position is

being calculated. Finally at time 𝑡2 the actual and the predicted positions are close to

each other, and the UE creates a circle around the predicted position and the application

is relocated in all MEHs falling within that circle. A parameter is tuned to choose the

radius of the circle. Two different circles are shown in the figure: the red one is smaller,

so that the application in relocated only in MEH 7, thus leading to service disruption,

since the actual next target MEH is MEH 9. On the other hand, the green circle leads

to service continuity, but it consumes more storage and memory resources, since the

application is also uselessly relocated in MEH 5, 7 and 8.

Fig. 3.3-12 Scenario in focus for the mobility prediction


Date: February 2019

Public Deliverable

5G-MiEdge

In the next sections the novel proposed algorithm is described in details and then

numerical results are provided, to assess the performance of our new algorithm.

3.3.4.4 Mobility model and Kalman filtering

Given 𝐾 UEs, to model the mobility of the 𝑘 -th UE, a vector 𝒁𝒌[𝑡] is introduced,

defined as follows

𝒛𝒌[𝑡] = (𝑥𝑘[𝑡], 𝑣𝑥,𝑘[𝑡], 𝑦𝑘[𝑡], 𝑣𝑦,𝑘[𝑡])𝑇

where 𝑥𝑘[𝑡] and 𝑦𝑘[𝑡] are the coordinates of the UE position, while 𝑣𝑥,𝑘[𝑡] and

𝑣𝑦,𝑘[𝑡] are the velocity components over the two directions, during the time slot 𝑡. A

transition matrix 𝑨𝑘 for a uniform motion can be defined as follows:

𝑨𝑘 = (

1 Δ𝑡0 1

0 00 0

0 00 0

1 Δ𝑡0 1

)

where Δ𝑡 is the time that passes from one observation to the next prediction. Making

use of the Matrix 𝑨𝑘, the process equation can be written as follows

𝒛𝒌[𝑡 + 1] = 𝑨𝑘𝒛𝑘[𝑡] + 𝒘𝒌[𝑡]

where 𝒘𝒌[𝑡] is the process noise (system error vector), which depends on how erratic

is the motion of the user. For simplicity, we assume this noise to be modeled as

Gaussian with zero mean and covariance matrix 𝑸𝑘 ∈ ℝ4×4 . Moreover, to get the

position of the UE at time 𝑡, we define a measurement matrix as follows:

𝑯𝑘 = (1 0 0 00 0 1 0

)

Thus, we can also define a measurement equation as follows:

𝒎𝒌[𝑡] = 𝑯𝑘𝒛𝑘[𝑡] + 𝒖𝒌[𝑡]

where 𝒖𝒌[𝑡] is the measurement noise, which depends on how accurate is the user

localization. Again, we assume this noise to be Gaussian with zero mean and

covariance matrix 𝑹 ∈ ℝ2×2. At time 𝑡, we have a prior estimate �̂�𝑘[𝑡|𝑡 − 1], and we

can define an a posteriori estimate


Date: February 2019

Public Deliverable

5G-MiEdge

�̂�𝑘[𝑡|𝑡] = �̂�𝑘[𝑡|𝑡 − 1] + 𝐾𝑔[𝑡](𝒛𝒌[𝑡] − 𝑯𝑘�̂�𝑘[𝑡|𝑡 − 1])

where 𝐾𝑔[𝑡] is the so-called Kalman gain. Defining a prior error covariance matrix

𝑷𝑘[𝑡|𝑡 − 1], the Kalman gain is given by

𝐾𝑔[𝑡] = 𝑷𝑘[𝑡|𝑡 − 1]𝑯𝑘𝑇(𝑯𝑘𝑷𝑘[𝑡|𝑡 − 1]𝑯𝑘

𝑇 + 𝑹)−1

The posterior error covariance matrix is given by

𝑷𝑘[𝑡|𝑡] = (𝑰 − 𝐾𝑔[𝑡]𝑯𝑘)𝑷𝑘[𝑡|𝑡 − 1]

and the predicted error covariance matrix is

𝑷𝑘[𝑡 + 1|𝑡] = 𝑨𝑘𝑷𝑘[𝑡|𝑡]𝑨𝑘𝑇 + 𝑸𝑘

The one step-ahead estimate is

�̂�𝑘[𝑡 + 1|𝑡] = 𝑨𝑘�̂�𝑘[𝑡|𝑡]

Once the prediction is available, it is necessary to choose the radius of the circle for

the VM instantiation and application relocation. Given 𝑷𝑘[𝑡 + 1|𝑡] , we choose the

radius to be proportional to the standard deviation of the localization prediction error,

using the following rule

𝑟 = 𝑐 ⋅ √𝑷𝑘(1,1) + 𝑷𝑘(3,3)

where 𝑐 is the shape parameter.

The overall algorithm for mobility prediction and proactive VM instantiation is

synthetized in Table 3.3-2.


Date: February 2019

Public Deliverable

5G-MiEdge

TABLE 3.3-2 Novel algorithm for the mobility prediction problem

Table X. Initialize �̂�𝒌[𝟎| − 𝟏], 𝑷𝒌[𝟎| − 𝟏]; define 𝑨𝒌, 𝑯𝒌, 𝑸𝒌, 𝑹𝒌, 𝒄 1. Enter measurement 𝒎𝒌[𝑡] 2. Update estimate

�̂�𝑘[𝑡|𝑡] = �̂�𝑘[𝑡|𝑡 − 1] + 𝐾𝑔[𝑡](𝒛𝒌[𝑡] − 𝑯𝑘�̂�𝑘[𝑡|𝑡 − 1])

3. Compute error covariance matrix

𝑷𝑘[𝑡|𝑡] = (𝑰 − 𝐾𝑔[𝑡]𝑯𝑘)𝑷𝑘[𝑡|𝑡 − 1]

4. One step-ahead prediction

{�̂�𝑘[𝑡 + 1|𝑡] = 𝑨𝑘�̂�𝑘[𝑡|𝑡]

𝑷𝑘[𝑡 + 1|𝑡] = 𝑨𝑘𝑷𝑘[𝑡|𝑡]𝑨𝑘𝑇 + 𝑸𝑘

5. Compute the radius

𝑟 = 𝑐 ⋅ √𝑷𝑘(1,1) + 𝑷𝑘(3,3)

6. Relocate the application (migrate the VM) to all MEHs

associated to the APs closer than 𝑟

3.3.4.5 Numerical analysis

In this section, we provide some numerical results in a simple scenario with 50 UEs

moving in a squared area of size 3x3 km2. We also consider 300 APs and associated

MEHs randomly deployed with a uniform distribution in the same area. The UE

mobility is modeled as explained in the previous section, with initial velocity

components 𝑣𝑥,𝑘[0] and 𝑣𝑦,𝑘[0] randomly distributed between – 1 and +1 m/s, for 50

slots of Δ𝑡 = 5 s each. This means that the predicted UE positions are calculated 5

seconds earlier than the actual next position. The measurement noise is assumed to

have a covariance matrix

𝑹 = (0.05 00 0.05

)


Date: February 2019

Public Deliverable

5G-MiEdge

whereas the covariance matrix of the process noise is

𝑸 =

(

𝜎𝑤,12 0

0 𝜎𝑤,22

0 00 0

0 00 0

𝜎𝑤,32 0

0 𝜎𝑤,42)

where 𝜎𝑥,𝑗2 , 𝑗 = 1, . . ,4 , are used as parameters to evaluate the performance of the

algorithm. In particular, as shown in the next figures, they vary from 0.01 to 0.1. A

higher value of 𝜎𝑥,𝑗2 , denotes more uncertainty on the user trajectory, while lower 𝜎𝑥,𝑗

2

indicate a more deterministic trajectory, thus easily predictable. For simplicity, the

parameters are assumed to be equal to each other, so that we denote all of them as 𝜎𝑤2

in the figures below.

Fig. 3.3-13 shows how the service continuity probability value changes according to

different values of the circle shape parameter 𝑐. The former is the probability that the

ongoing service is resumed at target MEH without service disruption, the latter shapes

the radius where to proactively instantiate the VMs.

Fig. 3.3-13 Service continuity probability vs. shape parameter c

We can notice how a higher 𝜎𝑤2 leads to a probability equal to one for lower values of

the parameter 𝑐. This happens because for higher 𝜎𝑤2 , the trajectories of the UEs are

more uncertain, so that the prediction error is higher. As a consequence, using the

algorithm of Table 3.3-2, the radius 𝑟 is larger and more VMs are instantiated, and thus

migrated. The price to be paid is in terms of memory, because more copies of the same

VM are created. This behavior can be seen in Fig. 3.3-14 and Fig. 3.3-15, where we

show the average number of instantiated VMs and migrated VMs, respectively. In

50 100 150 200 250 300

c

0

0.2

0.4

0.6

0.8

1

Se

rvic

e c

on

tin

uity p

roba

bili

ty

w

2 = 0.1

w

2 = 0.05

w

2 = 0.01


Date: February 2019

Public Deliverable

5G-MiEdge

particular, we can notice how a higher 𝜎𝑤2 leads to a proactive instantiation and

migration to many nodes with respect to lower values of 𝜎𝑤2 .

Fig. 3.3-14 Average number of instantiated VMs vs. shape parameter c

Fig. 3.3-15 Average number of migrations vs. shape parameter c

However, as a performance indicator, we are interested in showing the necessary

number of instantiated VMs vs. the service continuity probability. Indeed, if we were

50 100 150 200 250 300

c

0

500

1000

1500

2000

2500

Avera

ge

num

be

r o

f in

sta

ntia

ted V

Ms

w

2 = 0.1

w

2 = 0.02

w

2 = 0.01

50 100 150 200 250 300

c

0

5

10

15

20

25

Avera

ge

num

be

r o

f m

igra

ted V

Ms

w

2 = 0.1

w

2 = 0.05

w

2 = 0.01


Date: February 2019

Public Deliverable

5G-MiEdge

able to obtain a probability close to 1 (or even 1), we could determine the minimum

average number of VMs to be instantiated (and migrated) to guarantee this constraint.

We show this tradeoff in Fig. 3.3-16 and Fig. 3.3-17 for the values of probabilities

greater than 0.99. Here, we see how a more deterministic trajectory (low 𝜎𝑤2 ), can

afford a service continuity probability equal to the unity with less instantiations and

migrations than the other two cases, since the prediction algorithm is more confident

on the next position of the UE. On the contrary, a very noisy trajectory needs more

instantiations and migrations to guarantee the constraint.

Fig. 3.3-16 Tradeoff between VMs instantiated (storage) and service continuity

probability

Fig. 3.3-17 Tradeoff between VMs migrations and service continuity probability

0.99 0.992 0.994 0.996 0.998 1

Service continuity probability

200

250

300

350

400

Avera

ge

num

be

r o

f in

sta

ntia

ted V

Ms

w

2 = 0.1

w

2 = 0.02

w

2 = 0.01

0.99 0.992 0.994 0.996 0.998 1

Service continuity probability

2

4

6

8

10

Avera

ge n

um

ber

of

mig

rate

d V

Ms

w

2 = 0.1

w

2 = 0.05

w

2 = 0.01


Date: February 2019

Public Deliverable

5G-MiEdge

3.3.4.6 Conclusion

To conclude, in this section we proposed a proactive VM migration strategy for service

continuity based on UE mobility prediction. The prediction is computed through a

Kalman filtering algorithm, and the MEH relocation group is chosen with a rule that

is dependent on the error prediction. The algorithm helps in avoiding service disruption

due to application migration delays and, at the same time, in avoiding useless copies

of the service running in many nodes in the network. This leads to a solution that

preserves storage and memory resources of the MEHs and guarantees service

continuity, while reducing the number of VM migrations. The provided numerical

results assess the performance of the algorithm, showing a better behavior when the

trajectories of the UE are more deterministic. However, even for higher uncertainty on

the UE paths, the proactive solution shows good performance in terms of storage,

migration, and service disruption.

3.3.5 Appropriate workload placement in edge cloud

The MEC architecture brings undoubted benefits w.r.t. the communication latency.

However, the computation capabilities of each MEH are necessarily much more

limited than a cloud, because of an economy of scale in the deployment of MEHs.

Several papers have been published on the topic of enhanced workload placement. For

instance, a cooperative scheduling scheme of local cloud and Internet cloud is

proposed in [ZZG15], which analyzes the execution of delay-sensitive applications in

the local cloud, while delay-tolerant tasks can be offloaded to the Internet cloud when

traffic load is heavy. Reasonable computational resources need to be deployed in order

to execute all task requests in local cloud, so to satisfy the delay requirements of the

applications in focus. Provisioning more capacities in local cloud will increase the

capability to meet delay requirements, but unfortunately will also impair the efficiency

of local cloud resource utilization during off-peak hours [AFG10].

To improve the efficiency of cloud resource utilization, it is proposed in [TLG16] to

organize edge servers into a hierarchical architecture. The architecture called

hierarchical edge cloud is illustrated in Fig. 3.3-18. Instead of serving UEs from a flat

collection of edge servers, the new concept proposes to configure tiers of edge servers

in the hierarchy topology and opportunistically aggregate and serve the peak loads that

exceed the capacities of edge servers in lower tiers to other edge servers in higher tiers

of the hierarchy. The authors propose to serve larger amounts of peak loads with the

same amount of computational capacities being provisioned at the edge of the network.

But, they also stress that it is difficult to find a good balance between the computation

and communication delays in the edge cloud. Therefore, the authors just locate

workloads at edge servers in higher tier when loads excess computational capacity at

edge servers in lower tier, and they do not consider communication delay in their

decision of workload placement. Executing computational task at high-tier edge

servers reduces contention on computation resources and improves the task execution


Date: February 2019

Public Deliverable

5G-MiEdge

performance, but a longer communication time is needed to transfer task states

between edge servers of each tiers.

The MEC concept is to decrease the response delay for a service, but its proposed

architecture increases the communication delay which is included in the response delay.

To address the mentioned delay problem, we propose a method of workload placement

considering both computation and communication delay in the hierarchical

architecture. The following sections explain our proposed approach and its advantages.

Fig. 3.3-18 Hierarchical edge cloud architecture

Our proposed workload placement is based on some delay assumptions for each

service. The following subsections provide more details on our proposed approach.

3.3.5.1 Service delay assumptions

In order to realize dynamic workload placement in the hierarchical edge cloud, it is

necessary to predict delay for executing a task and its related communications for

determining the best place of executing the task. In our assumption, the required CPU

cycles to execute a task at an edge server is assumed to be the same as the data size of

upload for the task, for simplicity. The computational delay will be decreased based

on allocated CPU cycles for the task. Generally, an edge server at higher-tier has bigger

computing capacity and also less contention of computational tasks in the concept of

the hierarchical edge cloud. Therefore, the delay needed to execute a computational

task will be shorter when the task will be placed at a higher-tier edge server. An

additional communication delay should be considered in such a case. It will be derived

from data size to be transferred for the task and bandwidth in backhaul link. It is

necessary to formulate the balance between time reduction of computation delay and

time increase of communication delay.

With respect to the above mentioned descriptions, the expected delay, 𝐸𝐷𝑛𝑒𝑤, on an

edge server 𝛾𝑖 for a new incoming task i can be represented as:

𝐸𝐷𝑛𝑒𝑤𝛾𝑖 = [(𝑎𝑖𝑈𝐿 + 𝑎𝑖𝐷𝐿) 𝐵⁄ ]𝛾𝑖 +

𝑤𝑖

𝑐𝛾𝑖 (𝑛𝑢𝑚𝛾𝑖+1)⁄

(3.3-2)

UEs

Hierarchicaledge cloud

Tier-1

Tier-2

Tier-3

Wireless links

Backhaul links


Date: February 2019

Public Deliverable

5G-MiEdge

where 𝑎𝑖𝑈𝐿 and 𝑎𝑖𝐷𝐿 are the data sizes of upload and download for task i, B is the

expected bandwidth to communicate for the task i between the edge server 𝛾𝑖 and the

UE, 𝑤𝑖 is the amount of computations for task i, 𝑐𝛾𝑖 is the computation capacity (i.e.,

CPU cycles per second) dedicated to an application of the edge server 𝛾𝑖, and 𝑛𝑢𝑚𝛾𝑖

is the average number of executing tasks simultaneously running on the edge server 𝛾𝑖 in the previous measurement period. In Eq. (3.3-2), the [𝑋]𝛾𝑖 and the remaining parts

represent communication and computation delay, respectively, for the task i executed

at the edge server 𝛾𝑖.

Actually, data sizes 𝑎𝑖𝑈𝐿, 𝑎𝑖𝐷𝐿 and computation amount 𝑤𝑖 are similar among tasks for

an application. Therefore, these values are known approximately at the edge manager.

Just bandwidth B and the current number of task execution 𝑛𝑢𝑚𝛾𝑖 are variables to

determine the expected delay 𝐸𝐷𝑛𝑒𝑤𝛾𝑖 for the incoming task i at the edge server 𝛾𝑖.

Based on this delay assumption, the workload placement problem is defined as finding

which edge server within the hierarchical edge cloud should handle a coming task.

3.3.5.2 Signaling proposal

We propose a distributed signaling scheme between any two edge servers. It is

composed of a state query message and a state query response message. Format

examples of these messages are shown in Fig. 3.3-19. In fact, application management

signaling is defined for MEC in [MEC010-2]. Our proposed signaling can be

applicable to this standardized message format. An edge server at lower-tier sends a

“state query message” to an edge server at higher-tier with a service identifier (SERV.

ID) and indications of requesting status information, identified in information filter

field (Info. filter). Then the queried server responds by “state query response message”

with requested state information. Generation timing of these query messages depends

on a decision algorithm implementation on workload placement.

Fig. 3.3-19 Examples of messages format

In order to estimate delay for a coming task at a higher tier server by Eq. (3.3-2),

backhaul bandwidth and computational capacity for the task need to be obtained from

the server. Workloads on each server dynamically change in a second. But workload

trend changes slowly, probably in a minute time windows, unless unexpected task

requests are received, thus creating spikes. Therefore, periodic queries with a certain

Msg. type

Sequence no.

Source server ID

Destination server ID

SERV.ID

Info. fi l ter

State info. #1

. . . . .State

info. #n

Bits: 4 12 16 16 16 8 8 8

(a) State query message

(b) State query response message

Msg.

typeSequence

no.Source

server IDDestination

server IDSERV.

IDInfo.

fi l ter

Bits: 4 12 16 16 16 8


Date: February 2019

Public Deliverable

5G-MiEdge

interval will still help the workload placement for tasks among MEC servers. Based

on this state information of other MEC servers, edge manager on a server can

determine appropriate workload placement for a coming task by comparing estimated

delays for the task. Estimated delays are not to be recomputed at each time, because

state calculations are based on periodic values, but they can capture delay trends on

servers. Exact local computational loads can be monitored internally at a tier-1 server,

then delay can be calculated for a coming task if the task is placed at the tier-1 server.

By comparing the calculated delay to the estimated delays above, it is possible to find

the best server to place the coming task finally.

In order to evaluate the benefits of our proposed workload placement, we carried out

a number of simulations. The following subsections describe simulation models and

evaluation results.

3.3.5.3 Simulation models

(1) Simulation range

Cell deployment in the simulation range is assumed as shown in Fig. 3.3-20. In the

figure, hexagons represent macro cells operating at 2 GHz. Small cells operating at 30

GHz are distributed at the center and at the vertices of each hexagon. 8x8 macro cells

are deployed. The total number of macro and small cells is 64 and 224 respectively.

The function split of the control and user plane (C/U split) is considered, i.e. control

signaling is provided by the macro cell and data communication is provided by the

small cell. The user plane for service provision between an edge server and a UE are

always provided via the small cell. And the control plane is linked to a macro cell

because of its higher reliability. Other simulation parameters on wireless

communications are provided in TABLE 3.3-3.

(2) Edge server deployment

A tier-1 edge server is located and connected with each macrocell. The area of 64

macro cells is divided into 4 4x4 macro cell areas as depicted by bold lines in Fig. 3.3-

20. Each divided area has a tier-2 server above 16 tier-1 servers. Moreover, a tier-3

server is deployed above 4 tier-2 servers. In the end, 3 tiers of edge servers are

considered totally for the hierarchical edge cloud. We allocate an amount of

computational capacity for each edge server, based on what [TLG16] proposes, as

shown in Fig. 3.3-21. The number associated to each server indicates the amount of

computational power (CPU cycles [GHz]) being provisioned. For the tier-1 servers, 1

GHz is assumed for the computational power. As also proposed in [TLG16],

computational capacity of concurrent threads is introduced on an edge server. It means

that the amount of computational output per minute increases linearly when the

number of concurrent threads is less than a certain number. And the amount of

computational output per minute is saturated when the number of concurrent threads

exceeds that number. 4 is assumed as the capacity of concurrent threads at tier-1 servers.

CPU cycles to be allocated is degraded as shown in Fig. 3.3-22 when the number of

concurrent threads exceeds 4. We enlarge the capacity of servers for the other tiers,

according to the values of the tier-1 servers.


Date: February 2019

Public Deliverable

5G-MiEdge

Fig. 3.3-20 Simulation range

Fig. 3.3-21 Edge cloud topology for

simulations

TABLE 3.3-3 Simulation Parameters

Parameter Value

Frequency band

(macro / small) 2 GHz / 30 GHz

Inter-cell distance

(macro / small) 480 m / 240 m

Cell antenna height

(macro / small) 32 m / 10 m

UE antenna height 1.5 m

Tx power

(macro / small) 43 dBm / 30 dBm

Channel model [10] 3GPP TR 38.901 UMi model

Fig. 3.3-22 CPU cycles for a task (tier-1)

(3) UE behavior

A number of UEs generate task requests while moving in the simulation range. As

shown in Fig. 3.3-23, the number of UEs is increased as simulation time passes. At the

Macro cel ls Small cells

8

8

.... .... .... ....

16

64

1x16 1x16 1x16 1x16

161616

Tier-1

Tier-2

Tier-3


Date: February 2019

Public Deliverable

5G-MiEdge

initial, 64UEs (a UE per macro cell) are assumed. Then 192UEs are added every 30

seconds. Data sizes and amounts of computations among tasks are similar for an

application. However, a variety of cases should be analyzed for the evaluation to keep

generality. Therefore, we assume that the application has the following characteristics.

Assuming 500 different types of tasks, data size is randomly chosen from the range

between 104 and 107 bits each for upload and download of a task. A UE selects one of

those tasks randomly when it generates a task request. Then data sizes of upload and

download are determined for that task request. Computation amount for a task is

assumed to be proportional to the data size of the upload for the task. UEs generate

task requests in a random timing once in five seconds, which is selected as threads are

overlapped at the same tier-1 server when the number of UEs increases.

Fig. 3.3-23 Number of UEs for simulation

(4) Workload placement

Conventional hierarchical edge cloud

Whenever a new task arrives at a tier-1 server, it is checked if the tier-1 server has a

spare computing thread available. If yes, the task is executed at the tier-1 server.

Otherwise, the tier-1 server will offload the task to the tier-2 server.

Our proposed hierarchical edge cloud

The place for executing a task is determined whenever a new task arrives at a tier-1

server. Determination process is as follows. First, average number of tasks executed

and bandwidth used for a task on each server at tier-2 and tier-3 is obtained every 5

seconds. Second, when a new task arrives at a tier-1 server, delay estimations are

performed for relevant servers at tier-1, tier-2 and tier-3 for the task. Obtained values

at the previous step are used to derive delays for tier-2 and tier-3. And the current

number of tasks executed at the server is used for deriving delay at the tier-1 server. In

the derivations of delay, data sizes are taken into account for the task. Data size of

upload is used for deriving computational amount for a task. And finally, a server with

the shortest delay is determined as the executing server of the task, the workload

placement.

3.3.5.4 Evaluation results

Average task completion times are compared among different backhaul bandwidth of

100Mbps, 1Gbps and 10Gbps between MEC hosts. In Fig. 3.3-24b, results are similar


Date: February 2019

Public Deliverable

5G-MiEdge

around 70msec between the 2 approaches with backhaul links of 1Gbps. Fig. 3.3-24a

shows the result of 100Mbps backhaul between conventional and our proposed

hierarchical edge cloud. In this results, completion time is increased gradually in the

conventional way when the number of UE is increased (simulation time passes by).

That results is obtained because the conventional way locates tasks at higher tier

servers even if the communication delay is increased. But our proposed method keeps

task executions at lower tier servers by considering communication delay even if

computational capacity was fully occupied at the lower tier servers as allocated CPU

cycles were deteriorated for task execution, as shown in Fig. 3.3-22.

In Fig. 3.3-24c, big advantages are seen in the proposed approach. That is because the

proposed approach aggressively uses higher tier servers which have less contentions

of executing tasks under the condition with backhaul links of 10Gbps. Therefore, the

average completion time of tasks are shortened in the proposed approach.

The results show that backhaul bandwidth is very important for workload placement

in the hierarchical edge cloud. And when applying the hierarchical edge cloud, it is

necessary to deploy backhaul links of adequate bandwidth for connecting edge servers.

More tasks are executed at tier-1 servers when bandwidth of backhaul link is 100Mbps

in our proposed approach. Because it takes longer time to transfer tasks to upper tier

servers. In that case, it is less effective to configure the edge cloud in a hierarchical

structure while satisfying delay requirement of service.

Incidentally, some performance disadvantage points (indicated as (*) in Fig. 3.3-24)

are seen in the proposed approach between 0 and 30 seconds of the simulation time

lapse in Fig. 3.3-24a and Fig. 3.3-24b. In that time interval, the number of UEs are less

than in other time slots. Therefore, there’s a tendency to locate tasks at upper layers,

where computing capacity is greatly available. All MEC host servers at tier-1 are aware

that any upper tier server can process task in a shorter time. But sometimes multiple

tasks may be overlapping at the upper tier MEC host server. As a consequence, it is

likely that abnormal values affect the average value when there are few samples.


Date: February 2019

Public Deliverable

5G-MiEdge

Fig. 3.3-24 Average completion time of computational tasks

The CDF of task completion times are also shown in Fig. 3.3-25 for each backhaul

bandwidth. In the cases of 100Mbps and 10Gbps in backhaul bandwidth (Fig. 3.3-25a

and c), our approach shows shorter distributions of task completion times. Significant

differences are not seen in the case of 1Gbps in backhaul bandwidth (Fig. 3.3-25b).

a. Backhaul = 100Mbps b. Backhaul = 1Gbps c. Backhaul = 10Gbps

Fig. 3.3-25 CDF of task completion times

Offloading rates to higher tier servers are shown in TABLE 3.3-4 for all the cases taken

into consideration. As this table shows, offloading rates are almost the same for the

conventional approach. Because the conventional approach does not change offloading

policy in terms of backhaul bandwidth. But offloading rates vary very much in our

proposed approach. This behaviour is the cause of the improved task completion time,

as seen in Fig. 3.3-24.

0.5

0.6

0.7

0.8

0.9

1

0 50 100 150 200

CD

F

Task completion time [msec]

Conventional(64ue)

Proposal(64ue)

Conventional(640ue)

Proposal(640ue)0.5

0.6

0.7

0.8

0.9

1

0 50 100 150 200

CD

F


Conventional(64ue)Proposal(64ue)Conventional(640ue)Proposal(640ue)0.9

0.92

0.94

0.96

0.98

1

0 200 400 600 800 1000

CD

F


Conventional(64ue)Proposal(64ue)Conventional(640ue)Proposal(640ue)

a. Backhaul = 100Mbps b. Backhaul = 1Gbps

c. Backhaul = 10Gbps

30

60

90

120

0 30 60 90 120

Ave

. co

mp

leti

on

tim

e [m

sec]

Simulation time lapse [second]

ConventionalProposed

(*)

30

60

90

120

0 30 60 90 120

Ave

. co

mp

leti

on

tim

e [m

sec]



(*)

30

60

90

120

0 30 60 90 120

Ave

. co

mp

leti

on

tim

e [m

sec]




Date: February 2019

Public Deliverable

5G-MiEdge

TABLE 3.3-4 Offloading Rates to Higher Tier Servers

Backhaul

bandwidth Conventional Proposed

100Mbps 5.9% 0.4%

1Gbps 6.2% 1.3%

10Gbps 6.3% 38.2%

3.3.5.5 Frequency of query procedure for workload placement

In our evaluations, the number of UEs is fixed for the duration of 30 seconds. Under

such a condition, query procedures are assumed every 5 seconds to obtain status from

higher tier servers. Considering a real system, it is necessary to perform the query

procedure 24 hours a day. However, traffic trends are not subject to drastic changes in

just a few seconds. Therefore it is reasonable out claim that it is sufficient to estimate

the total delay trends at a higher tier server just every couple of minutes. Moreover,

these signal exchanges among edge servers are confined in a specific edge server group.

Thus, amount of signaling in this form will put just light loads on the system. This

approach will not affect the system scalability. However, it is necessary as our future

work to study a method of workload placement for unexpected short-term demand

spike not meeting long-term trend.

3.3.6 Combined result with the approach in Task 3.2

Task 3.2 studies the optimization of computation offloading based on caching of

computational results. The approach reduces the costs of tasks offloading by

preventing from:

i) the cost of uploading the computation inputs from the UE to the MEH,

ii) the cost of running the computation at the MEH.

when the result of the requested computation task is cached in the MEH (see D3.3 for

details). Fig. 3.3-26 shows an example of combined result of the hierarchical edge

cloud with caching of computational results. To obtaining this result, task results from

the most requested task to the top 3 are cached at each MEHs. Other conditions are the

same as described in Section 3.3.5.3. The result is obtained only for 1Gbps of backhaul

bandwidth this time. As seen in the figure, shorter distributions of task completion

times are obtained by caching of computational results.

As stated in D3.3, computation caching helps in reducing by considerable percentages

the uplink traffic from UEs to the edge cloud. Moreover, more offloading requests can

be treated per time unit by the same MEH, hence allowing the same MEH to serve

more UEs. This is the reason why the average computational delay required by

offloading tasks is shortened.

Because of reserving more computational capacity at each MEH, workload placements

can be different from ones without the computational caching. Workload placement

with caching techniques is expected as the future works.


Date: February 2019

Public Deliverable

5G-MiEdge

Fig. 3.3-26 CDF of task completion times by combination with content cache

[Task 3.2]

3.3.7 Patent applications

On the topics discussed in this chapter the KDDI research team submitted the

following two patent applications.

TABLE 3.3-5 Patent applications

No. Date Country Inventor App. No.

(1) Jul. 2018 Japan KDDI Research JP. 2018-130844

(2) Feb. 2019 Japan KDDI Research JP. 2019-031941

Titles of patent applications

(1) Communication service system, user terminal equipment, control equipment,

service management equipment, edge host server, communication control method

and computer program

(2) Edge computing system, edge host computer, control equipment, computer

program and decision method of edge host computer

0

0.2

0.4

0.6

0.8

1

0 50 100 150C

DF


Proposal

Proposal +cache

Improved


Date: February 2019

Public Deliverable

5G-MiEdge

4 Application to 5G MiEdge use cases

In the 5G-MiEdge project we assumed 5 use cases. This section evaluates the

applicability of our proposed approaches to those use cases and summarizes the

obtained results in TABLE 4. Some explanations are added for each use case in the

following subsections.

TABLE 4 Applicability summary to use cases

Use case CaSH Hierarchical

edge cloud

Omotenashi O O

Moving hotspot Y N/A

2020 Tokyo

Olympic (stadium) O O

Dynamic crowd Y Y

Automated driving Y Y

Legend: Y = Yes

O = Optional

N/A = Not applicable

CaSH = Carry-on state Service Handover (see Subsection 3.3.3)

4.1 Omotenashi services

Omotenashi service is considered as a venue specific service via LADN in airport,

train station and shopping mall. Therefore, it is not necessary to consider service

mobility generally. Required computational resources should also be prepared based

on the demand estimation. Because of these reasons, our proposed approaches are not

necessary for this use case, but they might be applicable as options.

4.2 Moving hotspot

Generally, in this use case services are provided for users from MEC server on board

in a train car, a bus or an airplane. As defined in D1.1, use cases and scenario definition

for the project, it is not necessary to consider mobility between a UE and an AP in the

car, as users stay most of the time under the coverage of the AP in the car (we assume

that most recent cars are equipped with AP on board). The hierarchical edge cloud

architecture is not applicable due to the limited backhaul bandwidth for moving cars.

However, it is necessary to consider mobility of the moving hotspots themselves. In

order to cache contents appropriately, synchronization of contents is performed for the

vehicle of moving hotspot at some places, e.g. train stations, airplane parking apron.

When the vehicle moves to another synchronization point, handover of synchronizing

contents for cache may need to be performed between MEC servers at these

synchronization points. In that case, our proposed CaSH method may be applicable.


Date: February 2019

Public Deliverable

5G-MiEdge

Fig. 4 Handover of synchronizing contents for moving hotspot

4.3 2020 Tokyo Olympic (stadium)

For this use case, MEC services can be served in the same way as “Omotenashi”

service use case as a venue specific service. Service mobility is not necessary to be

served outside the stadium. Therefore, it is not necessary to consider service mobility.

Required computational resources should also be prepared based on the demand

estimation. Because of these reasons, our proposed approaches are not necessary for

this use case fundamentally, but they can be applicable as options.

4.4 Dynamic crowd

When wide area deployment is considered with user mobility, MEC service provision

may need to be considered from MEC servers at multiple installation sites. In that case,

it is necessary to apply the proposed approaches for service handover and workload

placement among MEC servers.

4.5 Automated driving

It is fundamentally necessary to consider wide area deployment of the service for

vehicles. Our proposed approaches for service handover and workload placement

among MEC servers are applicable for vehicles communicating with a MEC server via

wireless links from road side.

However, further studies on applicability for services performed via wireless link of

vehicle to vehicle (V2V) communication are needed. It may not be necessary to

consider our proposed approach if handover and workload placement are not

considered among MEC servers for such a service.

MEC server on board

MEC server for sync.

MEC server on board

MEC server for sync.

Vehicle moves

Sync. Sync.


Date: February 2019

Public Deliverable

5G-MiEdge

5 Summary

This deliverable provides the final report of Task 3.1, which focuses on integration of

mmWave edge cloud into a 5G mobile network. Control signaling is studied to serve

contents and services, which are deployed by utilizing edge cloud in the network in a

distributed manner. Our work covers as well signaling for accessing contents/services

and optimal resource utilization.

Basic control methods for MEC services are presented in Chapter 2 in terms of

discussed directions in several standardization bodies. Then, our proposed approaches

are presented especially on service handover and workload placement among MEC

servers in Chapter 3. Chapter 4 finally maps the proposed novel methods to the existing

5G-miEdge use cases.

We are convinced that the outcomes of our work, as explained in this deliverable, will

help to direct how to implement MEC services in network systems in a more realistic

way.


Date: February 2019

Public Deliverable

5G-MiEdge

6 References

[5G-MiEdge] 5G-MiEdge website. Available online at: http://5g-miedge.eu.

[D1.1] 5G-MiEdge deliverable D1.1, “Use Cases and Scenario Definition”, Available online at:

http://5g-miedge.eu.

[D1.3] 5G-MiEdge deliverable D1.3, “System Architecture and Requirements”, Project confidential

deliberable

[TK13-1] T. Taleb and A. Ksentini, "Follow Me Cloud: Interworking Federated Clouds and Distributed

Mobile Networks", IEEE Network, vol. 27, no. 5, Oct. 2013, pp. 12-19.

[SMN18] M. Suzuki, T. Miyasaka and T. Niwa, “Discussion on DN/UPF selection”, ETSI MEC #16

F2F meeting, Plano, TX, USA, Dec. 2018. (not public)

[TK13-2] T. Taleb and A. Ksentini, "An analytical model for follow me cloud," in Proc. IEEE

GLOBECOM 2013, Atlanta, GA, USA, 2013, pp. 1291-1296.

[KTC14] A. Ksentini, T. Taleb and M. Chen, "A Markov decision process-based service migration

procedure for follow me cloud," in Proc. IEEE Int. Conf. Commun. (ICC), Sydney, NSW,

Australia, 2014, pp. 1350-1354.

[SA16] X. Sun and N. Ansari, "PRIMAL: PRfIt Maximization Avatar pLacement for Mobile edge

computing," in Proc. IEEE Int. Conf. Commun. (ICC), Kuala Lumpur, Malaysia, 2016, pp. 1-

6.

[NHB16] A. Nadembega, A. S. Hafid and R. Brisebois, "Mobility prediction model-based service

migration procedure for follow me cloud to support QoS and QoE," in Proc. IEEE Int. Conf.

Commun. (ICC), Kuala Lumpur, Malaysia, 2016, pp. 1-6.

[HAC15] K. Ha, Y. Abe, Z. Chen, W. Hu, B. Amos and P. Pillai, "Adaptive VM handoff across

cloudlets," Dept. Comput. Sci., Carnegie Mellon Univ., Pittsburgh, PA, USA, Tech. Rep.

CMU-CS-15-113, Jun. 2015.

[YS18] K. Yunoki and H. Shinbo, “Carry-on state service handover between edge hosts for latency

strict applications in mobile networks,” in Proc. 20th Int. Symposium on Wireless Personal

Multimedia Commun. (WPMC), Chiang Rai, Thailand, Nov. 2018, pp. 472-477.

[SKV01] B. Sikdar, S. Kalyanaraman and K. S. Vastola, “An Integrated Model for the Latency and

Steady State Throughput of TCP Connections”, Elsevier Performance Evaluation Journal, Oct.

2001.

[ZZG15] T. Zhao, S. Zhou, X. Guo, Y. Zhao, and Z. Niu, “A cooperative scheduling scheme of local

cloud and Internet cloud for delay-aware mobile cloud computing,” in Proc. IEEE Globecom

Workshops (GC Wkshps), San Diego, CA, USA, 2015, pp. 1–6.

[AFG10] M. Armbrust, A. Fox, R. Griffith, A. D. Joseph, R. Katz, A. Konwinski, G. Lee, D. Patterson,

A. Rabkin, I. Stoica, M. Zaharia, “A View of Cloud Computing,” in Communications of the

ACM, Apr. 2010, vol. 53 No. 4, pp. 50-58.

[TLG16] L. Tong, Y. Li and W. Gao, “A hierarchical edge cloud architecture for mobile computing,”

in Proc. 35th annual IEEE international conference on computer communications

(INFOCOM), San Francisco, CA, USA, Apr. 2016, pp. 1-9.

[JFF13] L. Jiao, R. Friedman, X. Fu, S. Secci, A. Smoreda and H. Tschofenig, "Cloud-based

computation offloading for mobile devices: State of the art, challenges and opportunities," in

Proc. Future Network Mobile Summit, Lisbin, Portugal, 2013, pp. 1-22.

[TS23.501] 3GPP TS 23.501, System Architecture for the 5G System, Rel. 15, Dec. 2017.

[TS23.502] 3GPP TS 23.502, Procedures for the 5G System, Rel. 15, Dec. 2017.

[MEC003] ETSI GS MEC 003, Mobile Edge Computing (MEC); Framework and Reference Architecture,

v1.1.1, Mar. 2016.

[MEC003_19] ETSI GS MEC 003, Mobile Edge Computing (MEC); Framework and Reference Architecture,

v2.1.1, Jan. 2019.

[MEC010-2] ETSI GS MEC 010-2, Mobile Edge Computing (MEC); Mobile edge management; Part 2:

Application lifecycle, rules and requirements management, v1.1.1, Jul. 2017.


Date: February 2019

Public Deliverable

5G-MiEdge

[MEC012] ETSI GS MEC 012, Mobile Edge Computing (MEC); Radio Network Information API,

v1.1.1, Jul. 2017.

[11aq] IEEE 802.11aq-2018 - IEEE Standard for Information technology--Telecommunications and

information exchange between systems Local and metropolitan area networks--Specific

requirements Part 11: Wireless LAN Medium Access Control (MAC) and Physical Layer

(PHY) Specifications Amendment 5: Preassociation Discovery, Sep. 2018.

[11bc] IEEE P802.11 TGbc - ENHANCED BROADCAST SERVICES (eBCS) TASK GROUP (TG)

- MEETING UPDATE. Available online at:

http://www.ieee802.org/11/Reports/tgbc_update.htm

[KAL11] Moriya, N. (2011). Primer to Kalman Filtering: A Physicist Perspective. New York: Nova

Science Publishers, Inc. ISBN 978-1-61668-311-5.

[MEC018] ETSI GR MEC 018, Mobile Edge Computing (MEC); End to End Mobility Aspects, October 2017

[PLA16] J. Plachy, Z. Becvar and E. Calvanese Strinati, "Dynamic resource allocation exploiting mobility prediction in mobile edge computing," 2016 IEEE 27th Annual International Symposium on Personal, Indoor, and Mobile Radio Communications (PIMRC), Valencia, 2016, pp. 1-6.

[OJI18] T. Ojima and T. Fujii, "Resource management for mobile edge computing using user mobility prediction," 2018 International Conference on Information Networking (ICOIN), Chiang Mai, 2018, pp. 718-720.

[ZHANG18] Zhang F, Liu G, Zhao B, Fu X, Yahyapour R., “Reducing the network overhead of user mobility–induced virtual machine migration in mobile edge computing.” Softw Pract Exper. 2018;1–21.

[THEO15] S. Theodoridis, Machine Learning, Chapter 4 - Mean-Square Error Linear Estimation, Academic Press, 2015, Pages 105-160.

[MACHEN17

]

A. Machen, S. Wang, K.K. Leung, B. J. Ko, T. Salonidis, "Live service migration in mobile edge clouds", IEEE Wireless Communications, Feb. 2018, pp. 140 - 147

https://en.wikipedia.org/wiki/Nova_Science_Publishers,_Inc.

https://en.wikipedia.org/wiki/Nova_Science_Publishers,_Inc.

https://en.wikipedia.org/wiki/International_Standard_Book_Number

https://en.wikipedia.org/wiki/Special:BookSources/978-1-61668-311-5

5g-miedge · the early stage of the 5g-miedge project, and in this report it is reminded as...

Documents