1

Abstract

Fog computing brings cloud computing capabilitiescloser to the end-device and users, while enablinglocation-dependent resource allocation, low latencyservices, and extending significantly the IoT servicesportfolioaswellasmarketandbusinessopportunitiesin the cloud sector. With the number of devicesexponentially growing globally, new cloud and fogmodels are expected to emerge, paving theway forshared,collaborative,extensible,mobile,volatileanddynamic compute, storage and networkinfrastructure. When put together, cloud and fog-computingcreateanewstackofresources,whichwerefertoasFog-to-Cloud(F2C),creatingtheneedforanew,openandcoordinatedmanagementecosystem.The mF2C proposal sets the goal of designing anopen, secure, decentralized, multi-stakeholdermanagement framework, including novelprogramming models, privacy and security, datastorage techniques, service creation, brokeragesolutions, SLA policies, and resource orchestrationmethods.Thisdocumentoutlinesthearchitectureandmain functionalities of the management frameworkdesignedintheH2020mF2Cprojecttocoordinatetheexecution of services in this heterogeneous anddistributedsetofresources.

Introduction TheemergenceofIoT–thenetworkedconnectionofpeople, process, data and things – is expected tosignificantly increase the number of connecteddevicesworldwide,fromthebillionsofunitswehavetoday, to tens of billions of units expected to bedeployed in the coming years. According to HIS [1]

there were 27 billion of interconnected devices in2017,whileCisco[2]predicts50billionby2020.ThesameHIS reportexpectsanannualgrowthofa12%until2030toreach125billionofconnecteddevices.At the same time, cloud service providers (AmazonAWS,Google Compute Engine,Microsoft Azure) aretodayenablingcustomerstoquicklydeployamyriadof private and corporate services at comparablylower prices than buying andmaintaining their owninfrastructure. When combined, fog and cloudcomputing are undeniably setting standards inflexibility,cost,economyofscale,butalsoinnovationinnewservices,devicesandapplications.There is no doubt therefore that any future,enriched, smart scenario, is likely to deploy andbenefitfromthecombinedcomputinginfrastructure,be it cloud, fog, or a combination of both. Figure 1showshow today’s ecosystem integrates centralizedcloudinfrastructure,withvariouslevels(orlayers)ofdispersedelementsstartingwithsmallerscaleclouds,somefogcomputingcapabilitieswithvariousdegreesof decision making and data processing capabilities(the stack of resources). We refer to the scenarioshowninFigure1asaFog-to-Cloud(F2C)system[3].In this novel computing paradigm, users will see anoptimizedserviceperformancewhentheservicecandecide on-the-fly the best suited set of fog/cloudresources, enabling enriched service executionfeatures to upscale performance, such as parallelexecution of tasks and computational offloading onthecloud.

Towards an Open, Secure, Decentralized and Coordinated Fog-to-Cloud Management Ecosystem (mF2C)

2

Themainobjectiveof themF2Cproject is todesignand develop a hierarchical, open, secure,decentralizedandcoordinatedmanagementplatformfacilitating the efficient usage of resources, takinginto consideration service requirements and userdemands, in a paradigm-shifting scenario combiningcloudandfogcomputing.The F2C collaborative and coordinated computingecosystem has been conceived to: i) efficiently andtransparently use available distributed andheterogeneous resources at the edge; ii) supportapplicationsandservicesthatdonotfitwellintotheparadigm of the traditional centralized cloud (e.g.,lowlatency,fasthandoverandconnectivityofmobileapplications), and; iii)pave theway tonewbusinessmodels in both cloud and smart devices sectors.Security and privacy are also addressed in atransversal fashion in F2C systems. The highlynecessary features of security and privacy,throughoutbothcloudandfog,maketheF2Csystema highly interesting subject of future research andinnovation.Lastbutnotleast,despitefog’spotential,many devices are likely to suffer from resourceconstraints (limited storage, battery, computepower),which can lead to inability to satisfy servicelevel agreements. Hence, a critical question here ishow collaborative scenarios, based on resourcesharing and clustering, can extend the concept ofcloudprovidertoanunknownfrontier.In this whitepaper, we outline the mainfunctionalities of themF2Cmanagement frameworkto being developed in the mF2C project. Thisframeworkwillmanagetheexecutionofapplicationsand services in a coordinated way in this newcomputingecosystem.

Challenges for an open and coordinated management of fog and cloud computing systems Themainobjectiveof themF2Cproject is todesignand develop a hierarchical, open, secure,decentralizedandcoordinatedmanagementplatformfacilitating the efficient usage of resources, takinginto consideration service requirements and userdemands, in a paradigm-shifting scenario thatcombines cloud and fog computing. This ambitiousobjectivemay be divided into the following generalchallenges:

• Manage a large, decentralized,heterogeneous,open,volatile,dynamicandnon-trustable set of resources (from cloudto the edge of the network), boosting andallowing an efficient and transparentutilization of the available distributed andheterogeneousresourcesattheedge.

• Cloud/fogsidentification:Anaddress,alabelor a name must be linked to the resource(cloud and fog) in a secure and verifiablefashion. This is especially important whenconsideringdynamicresources(especiallyinfog), whose time in the market is not pre-determined,constantorevenguaranteed.

• Transparently and optimally offloadcomputations, between fog and cloudcomputing systems, reallocating bothresourcesandservices,aswellasexecuting

Figure1:Fog-to-cloud(F2C)layeredstructure:Thestackofresources

3

services in parallel, addressing a solutionbased on a programming model thathandles the distribution, parallelism andheterogeneityintheresourcestransparentlyto the application programmer. Theframeworkisabletohandledataregardlessof itspersistencybysupportingasingleandunified data model (e.g. data can bereplicated “up” towards the cloud orsideways, with lower latency replicas beingpreferred).

• Resources discovery and allocation:Executing a service requiring differentresources (fogs and/or cloud) to interactwitheachotherwill firstrequiretheproperselection, and in some cases discovery, oftheseresources.Amanagemententitymustbe responsible for discovering the set ofavailable resources (in the fog(s) that it isresponsible for) and then choosing thosethatcanbestmeettherequirementsoftheservice.

• Incentivize users and devices to participatein theparadigm througheaseof operation,services customization, optimizedperformance aswell as features of securityand privacy. That is, how can collaborativescenarios, based on resource sharing andclustering, extend the concept of cloudprovider to an unknown frontier, creatinginnovative resource-rich proximateinfrastructures near to the user, whileremaining profitable? The concept is basedon the contributory/volunteer computing,involving volunteered resources from users(device owners) willingness to do so, butgiving users incentives (not necessarilyfinancial) and transparency (they can seeandcontrolwhattheycontribute).

• Coordinated layer orchestration: Acoordinated orchestration is required to: i)generate an individual service workflow; ii)map the service workflow into the fog andcloud resources best suited for the servicerequested, and iii) coordinate theinteractions among the different layersinvolvedintheserviceexecution.

• Services execution scheduling: Servicescheduling is required to decide how aservice’s individual functions are split intothe different fog layers and mapped ondifferent physical resources, and even

dynamically managing schedules based onruntimeconditions.

• Semanticadaptation:Thesemanticmappingbetweentheattributesofaserviceandthecapacities offered by cloud and fog layersinclude attributes such as static/dynamicinfrastructure (whether the infrastructure ispersistent in time or not), reliability (howreliableistheresource),time-to-leave(timetoexpectedteardown),securityandprivacyproperties,connectivity,tonameafew.

• Themanagement of security andprivacy inF2C systems. The envisioned scenario isundoubtedly inheriting most of the issuescoming from the uncertainty of edgedevices:theycouldbecompromised,hackedand malicious, malfunctioning, etc.Particular challenges include preventingbotnet attacks and implementing privacycontrols for personally identifiableinformation.

• Develop a dynamic business and marketmodelthatcantriggernewbusinessgrowthopportunities.Newplayers inthecloudandservices sectors are expected to emerge inthenear future, leveraging IoTdeployment.Itisclearthatcoordinationisrequiredwhenservices are executed on resources hostedby different providers. Hence, new modelsof collaboration must be sought at thebusinesslevel.

Withtheseambitiouschallengestheprojecthasbeenorganizedintwoiterations,thefirst,iteration1(IT-1)frommonth1tomonth18,andthesecond,iteration2(IT-2)frommonth18tomonth36.Afunctionalandworking platform is expected at the end of bothiterations. The advantage of this approach is that itgives the project enough time to design andimplement a model for IT-1 which is sufficientlyrealistic that we can learn from it and reusecomponents,anduseIT-2toaddressanyopenissues.Additionally, it is expected that the IT-2 will furtherexpandthefunctionalitiesofIT-1.

4

mF2C view mF2C hierarchical approach: Agents, leaders and IoT This section presents the global design of themF2Carchitecture. The mF2C system proposes acoordinated management solution leveraging allexistingandpotentiallyavailableresources,fromtheedge up to the cloud, when executing a service.Figure 2a) shows examples of possibleheterogeneous devices participating in the mF2Csystem;thesedevicesareheterogeneousyetabletoexecutetheservicesinadistributedandcoordinatedfashion. In order to manage this execution, wepropose to organize these devices in a hierarchicalarchitecture, as shown in Figure 2b), where theresources are categorized according to a certainpolicy and an mF2C agent entity deploys themanagement functionalities in every componentwithin the system.We see how different layers areset (from layer 0 at cloud to Layer N+2 at the levelcloser to the edge) and the fact that an agent isdeployed on all components. In fact, the agentsoftware must in principle be installed all devicesparticipating in the system; all devices thus becomemF2Ccapabledevices.However,inpractice,onlynlydevices with sufficient capacity will have the mF2Cagent installed. All information from other IoTdeviceswithoutenoughcapacity,suchassensorsandactuators (red balls in the figure) is gathered,processed and distributed by the agent connectingtheseIoTdevicestothesystem.

Hierarchically, several devices are clustered underthe control of one device that is defined as theleader. The policy to identify the clustering strategyand the leadership role is to be defined during theproject,althoughcharacteristicssuchasdistanceandconnectivitymaybeconsideredinafirstapproach.Weassumethat:

• Fog area or cluster is comprised of set ofnodes,managedbyaleader.

• Only one node acts as a leader in each fogarea.

• Initially, only one leader backup node ((forrobustness,actingwhenthe leader fails), ineachfogarea,isconsidered.

• IoTdevices canbeconnected toanyof theagentsinthemF2Csystem.

The whole set of agent management and controlfunctionalities has been divided into two big blocks,thePlatformManager(PM),andtheAgentController(AC), roughlyas foreseen in thedescriptionofwork,but with their names and roles changed slightly. Inshort, the PM provides high-level functionalities,responsible for inter-agent communications (agentscommunicate through theirPMs)and thus,with thecapacity to take decisions with a more global view.On the other hand, the Agent Controller (AC)functionalitieshaveamore local scope,dealingwithlocal resources and services. It isworth noting, thatwhen themF2C device acts as a normal agent (notbeing a leader) “local resource”means only its ownresources(suchassensorsconnectedtoit,oritsownmemory),butwhenthemF2Cdeviceactsasaleader,“local resources”mean its own local resources plusthesetoftheresourcesofthewholefogareathatitmanages.

Figure2mF2Cresourcesandarchitecture

a)Setofavailableresources b)LayeredandhierachicalmF2Carchitecture

5

It isalso important toexplainwhich is theapproachto managing services/tasks, and in particular whichservice/task is considered local.Whena service/taskisrequestedfromanyofthemF2Cagents,alwaystheresponsibilityofdecidingifthistaskcanbeexecutedin that agentordelegateddownward (toanyof theagentsintheareaiftheagentisaleader)orupward(to the higher hierarchical layer) is taken by thePlatform Manager. If the task is delegated up ordown,thecommunicationisalsodonebythePMsofthe agents. Only when the task is decided to beexecuted in the specific agent is the request passedto theAgentController,whichwillexecute that taskin the agent’s local (own) resources. Thus, inconclusion,allthesmartnesstodecidewhenandwhyforwarding tasks/services to higher/lower layersresidesinthePM.A final important remark refers to the databasescontaining the system information (resources,servicesandusers).This informationisdistributedinthedatabasesofallthemF2Cagentsinthesystem.

• A normal agent will contain informationaboutitselfanditsconnectedIoTdevices

• A leader will contain information aboutitself, the set of nodes in its fog area(“children”), and its connected IoT devices.Information about its “children” may be inanaggregated form.Theaggregationpolicyis one of the topics under study in theproject.

• The cloud agent will contain information(maybeinsomecasesaggregated)aboutallthedevicesinthemF2Csystem.

• EachmF2Cagentwill haveadatabasewithinformation about resources, services andusers; and this database is shared by thePlatform Manager (PM) and the AgentController(AC).

Finally, for IT-1 we made a set of assumptions, inorder to simplify the set of functionalities to beimplementedinIT-1.

• Onlyonecloudisconsidered• Only three layers are considered (Layer 0:

cloud, Layer1: leaders and Layer2: normalagents). IoT devices such as sensors,actuators, etc. are not per se a layer, buttheyareattached(andmanaged/controlled)bytheanyofthemF2Cagents

• There is no horizontal communicationamongthemF2Cagents,soservicerequestsare only communicated vertically, i.e.between a leader and its children.

Horizontal communication could still beimplemented by relaying messages via theleader.

• Onlyoneleaderbackupnodeisconsidered• Only one node acts as leader at any given

time• Resourcevirtualizationisnotconsidered• Mobility is only considered at the lowest

layer(normalagents,e.g.mobilephone)

The mF2C agent: Platform Manager and Agent Controller In this section, we detail the mF2C agentfunctionalities, divided into the set of functionalitiesof the Platform Manager (PM) and the set offunctionalities of the Agent Controller (AC) asoutlinedabove.

The Platform Manager (PM) ThePlatformManageristheblockresponsiblefortheorchestration of services based on the compute,storage,networkand IoT resourcesandusinga full-stack monitoring system, which receives telemetrydata fromdifferent sources. Thisblock also includesthedistributedexecutionruntime,whichcoordinatesthe execution of end-user applications within themF2C infrastructure. The Platform Manager can beseen as a global entity that works as a controllerwhen it ismanagingagents in lower layers,andasareceiverofcontroldatawhenitisbeingmanagedbyagents from upper layers. The PlatformManager isdivided into three main components according tothese responsibilities: Service Orchestration,DistributedExecutionRuntime(DER),andTelemetry.TheServiceOrchestrationisresponsibleforallocatingthe services to the most suitable resources. It is

Figure3.Platformmanagerfunctionalarchitecture

6

composedbythefollowingcomponents:1. Lifecycle management: The Lifecycle

Management component is responsible formanagingthelifecycleoftheapplicationstobe executed by the mF2C infrastructure.This includes the initialization, thesubmission and the termination of theseapplications,amongotheroperations.

2. Landscaper: The landscaper is intended toobtain a view of the whole mF2Cinfrastructure for fog/cloud infrastructures.The Landscaper must be able to query allphysical elements of the mF2CinfrastructureforagivenmF2Carea,that isledbyaleader.Thisincludesallthephysicalmachines andpartsof, e.g., CPU resources,storage,memory,etc.Metadataabouttheseelements are also required, e.g., CPU corecount, speed, cache size, etc. Each nodestored in the Landscaper should have amapping to any telemetry probes that aremeasuringperformanceonthatmachine;aswellasasubscriptiontoaneventsystemtoensure any changes in physical or servicelayers are updated accordingly (servicestop/start, machines added/removed tocluster)

3. SLA management: The SLA Managementcomponent is responsible formanaging theSLAs between the parties involved in aserviceon themF2Cplatform: theplatformandtheplatformusers.Thecomponentisinchargeof generating, storingandobservingthe electronic documents that describe theexpected service level of a service. Theagreements contain functional and non-functional terms that describe the servicebeingdelivered.

4. Recommender:Prior todeployingaservice,the Lifecycle Manager module will checkwith the Recommender for an appropriaterecipeof suitable typeof resources for thisservice (based on previous analysis). Therecommender should match thecharacteristic of the service (obtained bymeans of the Service Categorizationmodule) as well as the analytics fromprevious executions. To that end, theRecommender module must store theheuristics and models derived from theoutput of the Analytics module (inTelemetrycomponent).

TheDistributedExecutionRuntime(DER)componentreceives the requests for the execution of theservices/tasks and optimizing their execution on theavailableresources.Asmentionedearlier,theselocalresources could be those of the agent where theapplication is started, or resources managed byagentsofotherlevelsofthearchitecture.TheDERiscomposedoffourdifferentsub-components:

1. Task management: The DER considersapplicationscomposedbypiecesofsoftwareencapsulated as methods called CoreElements (CE). The main purpose of theruntime toolkit is to orchestrate theexecutionofCEinvocations(tasks),andthusto fully exploit the available computingresources. The Task Managementcomponent should receive the request forthe execution of tasks from the LifecycleManager

2. Task Scheduling: The Task Schedulingcomponent is in charge of distributing thetasks generated by the execution of theapplicationson the local resources selectedby the Lifecycle Manager. In the case ofresources belonging to another agent, theDER has to interact with the samecomponent in other levels. It also needs toprovide ameans to get information on thefinished tasks, and eventually get theirresults.

3. Policies: The Policies component is neededto support the runtime in the selection ofthe resources for the scheduling of tasks.The Policies component provides the list ofresources to the Task Schedulingcomponent. The current implementation ofthe runtime only supports the usage ofresources (or resource providers) whosedescription is provided before theinstantiation of the application. A basicrequirement for the project is to allow theregistration of additional resourcesdynamically,allowingtheruntimetohaveanupdated list of agents that can providenodesfortaskexecution.

4. Data management: The main responsibilityof the Data Management in the PM is tokeep the metadata of the objects that arestoredbytheDataManagementcomponentintheAC.TheDataManagementmustreactto the corresponding requests to get orupdate metadata. In case an agent is aleader, itmustalsobeawareof itschildren

7

and the data they contain, so that therequiredreplicascanbemanaged insuchaway that data is accessible from the agentthatwillneedit,whichisnotnecessarilytheone that created or updated it. Thisrequirement is derived from the nature ofthemF2Cplatform,wheredevicesmayloseconnectivity, but they should be able toperform their functions even though theyareisolated.

Finally, the Telemetry and Monitoring componentanalysesserviceperformanceontheinfrastructureitisdeployedon.Thethreemaincomponentsare:

1. Intelligent Instrumentation:Thiscomponentprovides the telemetry collectors andaggregators of the metrics. A telemetryframeworkwillperformtheInstrumentationon the nodes of the mF2C cluster,measuring performance of key variables.The Intelligent Instrumentationmodule willmonitor the telemetry metrics, e.g. settingthe collection parameters of probesaccordingtodeviceconstraints.

2. Distributed Query Engine. The DistributedQueryEngineshouldprovideasingleAPItofacilitate the querying of all telemetry datacaptured. This abstraction layer reducesaccessing telemetry data from multiplelocationstoasinglesource.Eachprobewillregisterwiththequeryengine,notifyingitofidentity, metrics captured and publishinglocation; and also, it will register with theIntelligentInstrumentationmodulesothatitcan receive a notification to throttle itsmeasurementsandpublishingfrequency.

3. The Analytics module characterises serviceexecution by mapping the service'sdeploymentconfigurationagainst telemetrycapturedforthosesamenodes:

a. Deployment configuration: theAnalyticsmodule needs to be ableto query the deploymentconfiguration for any given servicethat has been deployed by theLifecycle Manager. This may be acurrently executing service or ahistorical deployment. It isenvisionedthattheLandscaperwilltrack service deployments overtime, so should be able to takebothaserviceidentifierandatimewindowasparameters.

b. Telemetry: For each node in thesubgraph of the landscapereturned, the Analytics moduleneeds to query all Telemetrymetricsrelatingtoeachnodeofthesubgraph from the DistributedQueryEngine.Eachindividualqueryshould includeamachine identifierandatimewindow.

c. Analysis: A number of analyticsalgorithmswillrunagainstthisdatatoderiveheuristicsandmodelswillbe stored in the Recommendersystem.

The Agent Controller (AC) Inthedistributedandcoordinatedstrategyproposedin mF2C, where services are executed in differentdevices, the PM includes the logic of the system,takingdecisionsbasedonamoreglobalview.Ontheotherhand, theAChasamore local scope, focusingonlocalresources,servicesandusers.Regardingtheservices, the AC only controls and manages theservicesbeingexecutedinitsowndevice.Finally,theACalsomanagesthepreferences,roles,profile,etc.,oftheuser(owner)ofthedevice.Figure4showstheAC and its set of functionalities. The set of ACfunctionalities is split into three main blocks,Resources,Services,andUsersManagement.The ResourceManagement component collects andmanages local resources. However, due to thehierarchical nature of the proposed mF2Carchitecture, resources are grouped into clusters ofdevices,withoneof thesedevicesbeing the leader.ForthisreasonanddespitethelocalscopeoftheAC,in the case of a device being leader, its ‘local view’includes its own resources and the resources of the

Figure4AgentControllerfunctionalarchitecture

8

devices forming part of this cluster. It is also worthmentioning that, although the PM includes all thesmartness of the system, in the agent entity, thedatabaseisuniqueandboththePMandtheACshareit. For these two reasons, the hierarchicalarchitecture and the shared database, one of themain responsabilities of the AC regarding theresourcescanbesummarizedas:

• Filling in the resourcedatabase, tobeusedbybothPMandAC,withinformationabout:

o Ownresourcesifthedevicesispartoftheclusterbutitisnotaleader

o Own resources and the resourcesofthe“children”ifthedeviceistheleaderofthecluster.

ThesixsubcomponentsoftheResourceManagementare:

1. Discovery. This subcomponent is in chargeof discovering resources in a fog areamanaged by a leader. The discoverycomponent should allow the leader toadvertise its presence, as well as to allowtheagentstodetectaleaderintheirvicinity.Inthecaseofanewagentinafogarea,andafter the success of discovery process, thenewagentwill join themF2Csystem,beingpartofthatfogarea.

2. Policies. The policies block will be a set ofrules to be applied and used by differentblocks in the Agent Controller. This set ofrulescouldbechangedbythePM,accordingto a high-level policy managed by the PM.Examples of these policies are: theclustering, the discovery (related to thefrequency leader’s advertisement), theleader selection, the backup selection, theprotection and the resource aggregationpolicies.

3. Identification.Theobjectivesofthismoduleare to provide every device participating inthemF2CnetworkwithagloballyuniqueID,aimed at facilitating an unambiguousresource identity and to establishmechanism to update and/or revoke theresourceID.Theidentificationcomponentisdivided into two sub-components, theregistration and the identity management.Whiletheregistrationtakesplaceinacloudserver,theidentitymanagementisexecutedby the agent in the resource itself. Duringthe registration phase the IDKey isgenerated and delivered to the useraccording with the chosen registration

method.e.g.tothemF2Cappontheuser’sphone).Thus, Identificationcancover threedifferent aspects: the unique identificationof the device, the association of the devicewith the user (ownership) through theIDKey,andthecloud-issuedcredentialwhichis issued to the agent controller. The latterallows the agent to establish trust acrossfogs, because the issuer of the credentialresides in the cloud and is shared by thefogs.

4. Categorization. The basic objective of theResource Categorization module is toprovide the information about theresources. When running this module, thesystem is able to determine the hardware(storage, RAM, processor, etc.), power,software (operating system), securityrequirements (data, device, and network),attached components (webcam, Printer,scanner etc.), attached IoT information(sensors& actuators), and also informationabout resourcebehaviour and features (i.e.mobility). This information is stored in theresource’s local database and, at a laterstage, this information may be aggregatedtobesharedwiththeleaderinhigherlayers.

5. Monitoring. The Monitoring module isresponsible for instrumentation of eachcompute resource. A number of telemetryprobeswill capture performancemetrics ofthe hardware/software that services aredeployed onto. Each probe is required toperform3mainfunctions:

a. Collect metrics: Software probesmust be able to capture metricsfrom hardware (both in-band andout-of-band), from any softwaresource: host O/S, middleware andhosted application. Collectorsshouldbeabletoquerytheoutputof other collectors as this couldimpactcontinuedoperation.

b. Process metrics: Captured datashould be passed throughcustomised filters to perform adefined action on the data, e.g.,generate average, standarddeviation,etc.

c. Publish metrics: Processed datashould be published to defineddestinations, e.g., file, database,message queue. The publish

9

module should be able to analysethisdatapriortopublishingsoastodecide howmuch data to publish,e.g., publish averages, anomaliesonly,etc.

6. Data management. The data managementfunctionality of the AC is in charge ofallowingapplicationsorotherfunctionalitiesto store, retrieve,anddeletedata inmF2C.This data can be either the informationneeded tomanage the platform resources,services and users, or the data needed orgenerated by the services themselves. Forthe data managed by the mF2C platform,this component is also in charge ofmanaging the appropriate replicas of eachpieceofinformation,insuchawaythatdataisaccessiblefromtheagentthatwillneedit.Also, this component must react to therequests coming from the DataManagement component in the PMregarding the data locality requirementsfrom the Task Scheduling component, andthe deployment of classes so that objectscan be accessed. As mentioned earlier, aparticular data challenge is privacy ofpersonalidentifiableinformation,sothefirststep isdatacategorisation,asmentioned inthesecurity sectionbelow (note that this isdifferent from the data resourcecategorisationof theServiceManagement).A more thorough implementation of dataprivacy(e.g.addressinganygapsinmeetingthe requirements of the General DataProtection Regulation GDPR [4]) should beavailableinIT-2.

The Service Management component is responsiblefor the orchestration of local services. The mainfunctionalities of this block are: categorization,mapping, allocation and QoS provisioning. Theservice requests are decomposed into tasks in theTask Management block of the Platform Manager.After that, the Task Scheduler block decides whereeach individual task will be executed, and tasks arethendeployedtothecorrespondingagentcontrollerofeachselectedagent.TheSMfunctionalitiesare:

1. Categorization: The categorizationcomponent receives a service request, andcategorizes this service according to somedefinedattributes.Theattributesdefinedina firstapproachareCPU,Storage,Network,Memory,Priority,TimelimitandLocation.

2. Mapping: The mF2C control architectureleverages a distributed and hierarchicalcontrol topology intended to map servicerequestsintothemostsuitableresourcesfora successful service execution. The servicerequests can be decomposed into tasks intheTaskManagementblockofthePlatformManager, and the Task Scheduler blockdecides where each individual task will beexecuted, that is, the set of agents wherethe tasks will be deployed. Once theresource (Agent) is selected, the mappingconsistsofselectingtheresourcesmatchingthe requested task in theown resources ofthe agent (be they own agent resources ordevicesnon-mF2Ccapableattachedtoit).

3. Allocation: This subcomponent isresponsible for the optimal allocation ofavailable resources in the agent to thevarious tasks requests, during the mappingprocessandintheruntimeexecutionphase.In any case, all communication will gothrough the mapping component, and itsunique functionality is to allocate availableresources to the various requests, trying tomeetsecurityandprivacyrules,costmodels,while also guaranteeing overall optimalresourcesusage.

4. QoS provisioning. The QoS provisioningsubcomponentis,unsurprisinglyresponsiblefor guaranteeing the required quality ofservice, according to the SLAs. For eachservice, itwillcontacttheSLAManagementblock in the Platform Manager to getinformation about the expected servicerequirements. The QoS provisioningsubcomponent will be contacted by theLifecyclemanagement component for eachservice query to get the characteristics ofthe candidate devices to execute theservice. This information allows the QoSprovisioningblock todiscard theunsuitablecandidates. The parameters that should bedefinedwilldependonthedifferentservicescharacteristics, but in a first approach thefocuswillbeontheserviceexecutiontime.

Finally, theUserManagementmodule is responsiblefor managing the profiling and the sharing modelpropertiesoftheuserswhohaveaccesstothemF2Csystemandtheapplicationsrunningontopofit.Thismodule is composed by three subcomponents,Profiling, Assessment and Sharing model, describedasfollows:

10

1. Profiling: A user profile is the collection ofpersonaldatarelatedtoaspecificuser(beitboth, the user/owner of the device or themF2C consumer client executing a service)and thedigital representationof aperson’sidentity. It can be also considered as alogical representation of a usermodel. Theuser profile information can be extendedaccording to the user’s preferences andbehaviour. In a first approach, thisinformationincludestheuser’sidentificationkey (if available), the user’s email (if theychoose to provide it), how the user joinsmF2C (as a contributor, as a consumer, orboth), and the parameters such as servicesthe user is allowed to use, the services theuser will allow to run in his/her devices,maximumnumberofservicesallowedtorunin their devices, security levels defined fordifferentdataflowsrelatedtotheuser,etc.

2. Assessment: The User ManagementAssessment component is responsible forchecking that the mF2C applications meetthesharingmodelandtheprofilepropertiesdefined by the device's user. The mainfunctionalitiesare:

a. Double check that the profilingpropertiesaremet

b. Double check that the shareableresourcesconstraintsaremet

c. Send a warning to the PlatformManager if some constraint isviolated.

3. Sharing model. The Sharing Modelcomponent is responsible for defining theresources that the device’s user wants toshare through (or with) the mF2C system.The functionalities included in this modulearethefollowing:

a. Definitionoftheresourcesthatwillbe available to the mF2Capplications, such as memory,storage,etc.

b. Definition of the rules orconstraintsinordertoestablishnotonlytheamountofresourcestobeshared, but also the conditionsunder which these resourcesshould be increased, decreased ornotsharedatall,suchasmaximumCPU, memory usage, batteryor/andbandwidthlimits,etc.

c. Definition of reward mechanismsfor these resource contributions,likesomekindofserviceexecutioncredits,economicrewards,etc.

11

mF2C security approach Security Policy Whenever a system is designed it must considersecurityasa key feature, addressingaspects relatedtodata,connectivityandhardwareinteroperation.Tothat end, it is essential to define a security policy,whichmustguide implementerstowardsthinkingonhowsecuritymustbedeployed.

mF2C system security Table 1 lists the security requirements andfunctionalitieswehaveidentifiedasnecessaryinthedifferent layers, mapped into each one of the

functionalitiesexpectedfromthemF2Cagents.After identifying the mapping of the agentfunctionalities into the security requirements, wemayconcludethatagentsarenotnecessarilyisolatedentities and they, and their security functionalities,will be implemented/supported and deployedthroughanagentcontroller.The security features can be implemented in theagentcontroller’sfunctionalblocksdirectly,orintheagent controller itself. The advantage is that eachcontroller will have all the functionalities it mightneed, at the cost of increasing the size andpossiblythe computational requirements of the agentcontroller(allcalculationsaredonelocally).

Table1SecurityrequirementsinfunctionalblocksinmF2Cagents

12

Use cases validated in mF2C The mF2C project proposes a business innovativeapproach based on three different butcomplementary and incremental use cases tovalidate the concepts developed within the project,to show a wide spectrum of scenarios mF2C maysubstantiallyimpacton.

Emergency Situation Management in Smart Cities (ESM) Continuingintothiscentury,societyhassupportedamovement of people from rural areas to cities.Nowadays,more people live in urban environmentsthan in rural ones. It is estimated that this processwill not stop and within 20 years the urbanpopulationwillbearound5billionofpeople.Thebigchallenges for the whole society will be related toresourcemanagementandmobilitythroughouttheseovercrowded environments. mF2C works in thedevelopmentofamodel forsmartcities to facilitatethe deployment of innovative services in highlyvaluable sectors (health, traffic control,entertainment,etc.)whilealsomaking themostoutofthesetofIoTdevicesdeployed.However,forthatevolutiontooccuranefficientcoordinationbetweenall the actions is the key to have the maximumpositive effect in the fulfilment of our objective,which is actually themainmF2C rationale. Thus,weproposetouseaFogtoCloudmanagement(mf2c)tohandle an emergency situation in smart cities, sincemF2C allows an immediate, safe and reliableresponsetosuchanevent.One of the proposed services in this emergencymanagement for smart cities is thedetectionof the

collapseofacityconstruction,seeFigure5.Thereareareas in the world where seismic movements arevery frequent. These natural phenomena can causeseriousaccidentsduetothecollapseofbuildings.Theseverity of the accident can range from a simplemovementoflandwithoutanynotableconsequenceto a collapse with fatalities. Among this range ofvalues, the performance of emergency services isvital to reduce the negative consequences thatmayoccurasmuchaspossible.Itwillbenecessaryatalltimestohavedifferenttypesofsensorsthatallowustoknowtherealstateoftheconstruction. Inaddition, itmustbeguaranteedthatsensordataisbeingreceivedatalltimes.Tothisend,mechanisms must be used to verify and restorecommunication between the sensors and the datacollection center. Based on the values provided bythe sensors, an interpretation system must decidethe degree of action to be taken and activate theservicescorrespondingtothesaidaction.To know what kind of services are the mostappropriate to be activated, the mF2C system willprovide the management between all the elementsof the city. While the fog layer provides a rapidresponse to an emergency, the connectionwith thecloud allows optimizing the resources to be usedbased on the historical knowledge of similarsituations.

Enriched Navigation Service Sentinel is an IoT device consisting in differentsensorscurrentlyappliedinthenavigationsectorforvessel monitoring. There are 1,000 Sentinel devicesmainlyintheAdriaticSea,aimingatimprovingqualityof the sailor’s journey. As of today, Sentinel deviceswork in an isolated way, hence losing the potential

Figure5CollapsebuildingdetectioninaSmartCity

13

benefits brought by correlated data processing. ThemF2Cprojectcanhelpondevelopingandeasingtheimplementation of novel services enriching theSentinel port-folio. However, most of the ideas forinnovative services require the missing datacorrelation but also additional capacities such asinteractionwithexternaldata,etc.Oneof theproposedservices, is illustrated inFigure6, where an agent located on land or in the sea,wantstoknowtheaveragetemperatureonsea:

1. It sends the service request to theagentatcloud.

2. ThePMoftheagentatclouddecideswhicharetheneededresourcesandallocatestasksindifferentships(asexplainedinFigure6)

3. Oneof the selecteddeviceshas the taskofaggregatingthedatafromitselfandthedatacomingfromtheotherdevices,andsendingittotheagentatcloud.

4. The agent at cloud replies to the requestwiththeaveragetemperatureonthesea.

5. One of the main characteristics of theproposed services is the use of Wi-Fi orLORAforconnectingbetweenships,andtheuseof4G/5Gforconnectingtotheagentincloud.

Smart Fog-Hub Service The main idea is setting up hubs in publicenvironments (e.g. airports, train stations, hospitals,mallsand relatedparkingareas), capableof trackingthepresenceofpeopleandotherobjectsinthefield,and developing value added services on top for

proximitymarketing,predictionofpath/behaviourofconsumers,andtakingrealtimedecisions.Let us consider an airport as a small city inwhich alargenumberofpeoplemustspenda longperiodoftime (more than3 hours)walking through it. In thisspace, many services are offered to users such asshops, restaurants, relaxation areas, etc., which aredistributed throughout the airport. Users candedicate their waiting time to make use of theseservices but they must always be attentive to thetimeofboardingfortheirflightandtothelocationofthe boarding gate. The uncertainty caused by notknowingthetimeauserneedstogetfromanypointin the airport to the boarding gate means that, inlargeairports, theuseof these services is limited totheir physical proximity to the boarding gate and atthetimeofthesaidboarding.Wemustalsoaddthepossible incidents caused by delays in flights orchanges in theboardinggate. It is true that insomeairportstheestimatedtimestogetfromoneareatoanotherare indicated intheairportsignage.Butthisinformation does not provide any reliableinformation thatallows theuser tospendtheir timewaitingtomakeuseofairportservicesinacalmandrelaxed way, without having to constantly worryabout their physical situation inside the airport andthe time remaining before the boarding time. Inaddition, these times are affected by the differentagglomerations of people that occur irregularly(departure of passengers from a flight, queues atboardinggates,luggagecontrol,etc.)throughouttheentireairport'spassablespace.Thismeansthatmostusersdonotmoveoutofaradiusneartheboardinggate during the last hour of boarding their flight.

Figure6ExampleofEnrichedNavigationService

14

Thus,manyoftheservicesofferedatairportsdependontheareawheretheboardinggate is locatedand,therefore, are limited to users from other areas. Asolutionthathasbeenimplementedisthereplicationof services in different areas to give maximumcoverage to users. However, this solution is totallyinefficientanddoesnofulfilitsobjective.Inaddition,onlylargecommercialcompaniescanaffordtomakesuchareplica.Figure 7 shows the architecture overview of theproposed Smart Fog-Hub Service,with the followinglayers:

• L-0 Cloud: OpenStack engine for heavyprocessing,e.g.,ML

• L-1 Fog Leaders: Fog aggregator forunderlyinglayer,andinterfacewithexternalsystems for relevant events gathering, e.g.,Admin Dashboard; Flight gate opening, lastcall,etc.

• L-2 Fog workers: Keep devices position,session-oriented information,dispatchingofevents, delivering notification of Points ofInterest(PoI)inproximity

• L-3 user devices: Without an mF2C agent,usingaspecificandroidapp,theyconnecttoL-2nodesusingsecurityprotocol,accordingtoacertainprivacypolicy.Theymustagreeontermsofcondition

Themainfeaturesofthedevelopedserviceswillbe:• able to track the presence of people and

objectsinthefield

• develop value added services for proximitymarketing

• suggestionsonbestuseofairportservices• predictiononpath/behaviourofconsumers

With these main features some of the specificservicesmaybe:

• Locationoftheboardinggate• Timeofboarding/departureoftheflight• Servicesofferedbytheairport• Physical situation of available airport

services• Serviceselection:

o Calculation of the actual time ofaccesstotheairportservice

o Timeandplaceofoffers/activities/ performances offered by theservicewithintheairport.

o Guided path through the airportfacilities to get to the preferredservices.

Figure7SmartFog-HubServicearchitecture

15

Who mF2C benefits? Following recommendations provided by ISOstandards related to Cloud Computing [5], theseactors and roles are classified in three categories:Customer,ProviderandIntermediate/PartnerSide.CustomerSide

• Application Developer, a technical personcapableofdevelopingasoftwareapplicationto be operated in an mF2C-poweredinstallation.

• IoT Platform / Solution Provider, Theproviderofanend2endIoTspecificsolutionor Platform candidate to consume as finalusermF2Csystemandservices.

ProviderSide• Fog Service Provider, the provider of a the

provider of a single or multi-fog set ofinstances and services which comprise andinclude software, platform and/orinfrastructure (compute, storage, andnetworking services) operated fromgeographically dispersed close to the edgeclustersofresources.

• Cloud Service Provider, the provider of asingle or multi-cloud set of services whichcomprise and include software, platformand/or infrastructure (compute, storage,and networking services) operated fromtraditionaldata-centers inprivate/publicorhybridformsinan“as-a-ServiceModel”.

• Resource Contributor, the provider of aresource or a set of them which does notofferitinas-a-Service”butinacontributorymodel.

Intermediate/PartnerSide• Fog Equipment Provider, an equipment

provider which sells Fog and Edge specificHardware.

• Cloud Equipment Provider, an equipmentprovider which sells Cloud capablehardware,traditionallyservers.

• Sensor Provider, Wireless Sensor Networkshardwaremanufacturers.

• Network provider, Network servicesprovideroroperatorprovidedinisolationtoadditional services commonly tagged asCloud-services.

IthastobenotedthatcurrentlyintheIoT,Fog(Edge)andCloudmarkets,manyvariationsandaggregationsoftheseActorscanbefound.

Conclusions Thispaper is intendedtobring light to thekey issueof making the best of the scenario set by puttingtogether cloud and fog resources. As widelyaccepted, cloudand fogareexpected tocollaborateso a key challenge comes up when setting acoordinated resources framework where servicesmaybeexecutedatcloudorfogonlyleveragingrealresourcescharacteristicsandavailability.In this paper, we describe the mF2C initiative thatpresents an ongoing work intended to design andimplement a coordinatedmanagement architecture,pushing for innovativesolutionsdealingwith thesetof foreseen challenges. These challenges are alsoincluded in the paper aswell as tentative directionsandtrends,asactivelinesofworkwithintheproject,foraclearandcomprehensiveunderstanding.Finally,some use cases are also presented to illustrate thebenefits that such a coordinated resourcesmanagement may bring to real-world scenarios,supported by flagship companies in their individualsectors.Certainly, being mF2C an active and ongoing effortsupported by industrial and academicmembers,weareopentoanycollaborationthatmayhelpsolvethealready identified open challenges as well as anysuggestion for testing, validation, etc. Moreinformation about the project as well as contactdirectionsmay be found in the links included in thenextpage.

16

References [1] “TheInternetofThings:Amovement,nota

market”HISMarkit,https://ihsmarkit.com/Info/1017/internet-of-things.html

[2] “HowManyInternetConnectionsareintheWorld?Right.Now”CiscoBlogsbyKarenTilman.https://blogs.cisco.com/news/cisco-connections-counter

[3] X.Masip-Bruin,E.Marín-Tordera,A.Jukan,G.J.Ren,G.Tashakor,"Foggycloudsandcloudyfogs:arealneedforcoordinatedmanagementoffog-to-cloud(F2C)computingsystems",IEEEWirelessCommunicationsMagazine,Vol.23,Issue5,October2016.

[4] EUGeneralDataProtectionRegulationhttps://www.eugdpr.org/

[5] http://standards.iso.org/ittf/PubliclyAvailableStandards/c060545_ISO_IEC_17789_2014.zip

For more information Webpage:http://www.mf2c-project.eu/

Deliverables:http://www.mf2c-project.eu/blog/press-room/deliverables/

Twitter:https://twitter.com/mF2C_project?ref_src=twsrc%5Etfw&ref_url=http%3A%2F%2Fwww.mf2c-project.eu%2F

LinkedIn:https://www.linkedin.com/in/mf2c-project-22b4ba139/

This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 730929. Any dissemination of results here presented reflects only the consortium view. The Research Executive Agency is not responsible for any use that may be

made of the information it contains.