real-time iot stream processing and large-scale...

D2.2 Smart City Framework– Dissemination Level: Confidential

GRANT AGREEMENT No 609035 FP7-SMARTCITIES-2013

Real-Time IoT Stream Processing and Large-scale Data Analytics for Smart City Applications

Collaborative Project

Smart City Framework

Document Ref. D2.2

Document Type Report

Workpackage WP2

Lead Contractor ERIC

Author(s) Vlasios Tsiatsis (Editor, ERIC), Pramod Anantharam (WSU), Payam Barnaghi (UNIS), Marten Fischer (UASO), Frieder Ganz (UNIS), Muhammad Intizar Ali (NUIG), Sefki Kolozali (UNIS), Daniel Kuemper (UASO), Alessandra Mileo(NUIG), Nechifor, Cosmin-Septimiu (SIE), Dan Puiu (SIE), Ralf Tonjes (UASO), Thorben Iggena (UASO)

Contributing Partners ERIC, NUIG, SIE, UASO, UNIS, WSU

Delivery of Version 1.0 M12 (31st August 2015)

Delivery of Updated Version 11th February 2016

Dissemination Level Public

Status Final

Version V2.3

Reviewed by Michelle Bak Mikkelsen (AA), Piraba Navaratnam (UNIS)

Ref. Ares(2016)1092239 - 03/03/2016


Executive Summary This report presents the Smart City Framework (SCF), a high level architecture of a platform thatdescribes the scopeof the innovations expected from theCityPulseproject and theway they areintegratedtogetherinacoherentconceptualsystem.Thepurposeofthisdocumentistoserveasareferencemodelandarchitecture(ARM)tobeusedbysmartcitystakeholders,projectpartnersandanyotherinterestedpartieswhenengagedintechnicaldiscussionsaboutsmartcitiesservicesbasedonreal-timeinformationstreams.TheSmartCityFrameworkisexpectedtobeaninitialarchitecturetosetthemainconcepts,commonlanguageandtheboundariesforthewholeprojectwhilethedetailsareexpectedtochangewithinthecourseoftheprojectexecution.Thereportpresentsthearchitectureindifferentviews,functional,interfaceandinformation,securityandprivacyviewandthusexplainsrespectivelywhattheframeworkdoes,howcomponentsinteractwitheachother,thegenerationandflowofinformation,andthenecessarymechanismstoaddresssecurityandprivacyconcernsaboutcityandcitizenrelevantdata.InthepresentationoftheinterfaceandinformationvieweachCityPulseworkpackage(WP)hasprovidedmoredetaileddescriptionsoftheirindividualarchitecturesasofthewritingofthisreport.

Figure 1 – High-level view of Smart City Framework

OnaveryhighlevelSmartCityFramework(orsimplytheframework)canbeshowninFigure1.Theboundariesoftheframeworkarethedifferentinterfaces(I/Finthefigure)towardstheapplicationsandtowardstheInformationsources/sinks.Theframeworkcanbeusedasatooltoaddresssmartcityissuesandthereforeitshouldbeutilisedbysmartcityrelatedapplications.Similarly,theframeworkneedstouseinformationsourcesandsinksrelatedtoasmartcity.ThefigureshowsthatInternetofThings (IoT) Sensors deployed in a city environment can be one type of source of real-timeinformation. Moreover, IoT Actuators can be the sinks of information coming from either theframeworkortheapplications.Anothertypeofinformationsourceisthelegacyinformationsourcesprovidedalreadybyacitye.g.OpenDataportals,cityGIS(GeographicalInformationSystem)dataetc.A City Information Sink represents the city infrastructure that the framework can use to storeprocessedinformationproducedbytheframework.ApartfrominformationcomingfromsensorsortheCityInformationSources,informationaboutthesituationinacitycanoriginatefromthecitizensthemselves through the use of socialmedia (e.g. twitter feeds). Therefore these sources are alsoincludedasrelevantinputssourcesfortheframework.Thedifferencebetweenthesesourcesandthe


IoTorCityInformationSourcesisthattheformerarepotentiallysourcesofunstructuredinformationexpressedinnaturallanguage(e.g.atweetisashorttextwrittenbyahumanwithpotentiallyalinkto a website potentially written by a human) while the IoT or City Information Sources producestructured data although not standardized. The Social information Sinks represent social mediachannelsthroughwhichcitiescouldpotentiallypushinformationtotheircitizens,e.g.achannelforcitizenstoreceiveupdatesonthetrafficsituationincertainpartsofthecity.TheSCFinternallyconsistsofafewfunctionalgroups(FG)namelytheLargeScaleDataAnalysis,theReasoning and Decision Support, the Reliable Information Processing, the Knowledge Base, theActuation, the FrameworkManagement, the Exposure and the Information sources/sinks FG. TheLarge-Scale Data Analysis addresses issues that are related to the integration of a large scale ofheterogeneoussourcesproducingreal-timestreamsandtheirsemanticenrichment.TheReasoningandDecisionSupportfunctionalitytacklesissuesthatarerelatedtotheabilityoftheSCFtoadapttochangingsituationsbasedonthereal-timeinformationstreams.Itisresponsibleformonitoringthesemanticallyenrichedstreamsandadaptingthecollectionofstreaminformation.ItisalsoresponsibleforprovidinganAPItowardstheSmartCityApplications.TheLargeScaleAnalysisandReasoningandDecisionSupportfunctionalitiesaresupportedbya-prioriknowledgeintheformoftheKnowledgeBaseandreliabilityandqualityofservicecontrolmechanismsbytheReliableInformationProcessingFG. The Actuation FG covers any functionality that allows the SCF to push control commands orinformation to the IoT actuators, social media sinks and city information sinks. The FrameworkManagement FG includes functionality for the management of the SCF itself such as fault,configuration,securitymanagementetc.TheExposureFGcoversthemediationofaccessbetweenCityPulseandManagementapplicationandtheSCF.TheSCFspecificationdescribesmorefunctionalitythanintendedfromthedescriptionofworkandthereforethisfunctionalitywillnotbeexploredordetailedfurtherwithinthetechnicalworkpackages(WP3-WP6).Forexamplealargepartoftheframeworkmanagementandmanagementapplicationsarenotexpectedtobedevelopedfurtherinresearchbecausetheymayalreadybecoveredbystateoftheart techniques.Nevertheless, theframeworkasspecified inthisreportcoversthepromiseddescriptionofworkand ithasalreadybeenusedby individualworkpackagesotherthanWP2,fortheirinternalsystemarchitecturedefinitions.After the individual WPs have performed in depth research on each part of the SCF and afterintegratingconcretesoftwareimplementations,theyproducedanupdateontheSCFwhichisincludedasaseparatesectioninthisdocument.TheupdatepresentsmainlythefunctionalandinterfaceviewsasasetoffunctionalcomponentsandasasetofhighlevelAPIs(ApplicationProgrammingInterface)methodsdescribedinnaturallanguagerespectively.Theupdatealsoincludesadescriptionofeachfunctional component and some information about the corresponding exposed API as well asimplementationdetails.


1. Contents 1. Introduction...................................................................................................................................7

2. DefinitionoftheSmartCityFramework........................................................................................8

2.1 Smart City Framework stakeholders...............................................................................9

2.2 Stakeholder concerns........................................................................................................92.2.1 Discussion .................................................................................................................... 10

2.3 Baseline assumptions, scope and boundary conditions.............................................10

2.4 Contribution of Work Packages and stakeholder concerns.......................................12

3. DescriptionofWorkHighLevelarchitecture...............................................................................14

4. SmartCityFrameworkSpecification............................................................................................15

4.1 Functional View................................................................................................................164.1.1 Information Sources Functional Group ......................................................................... 164.1.2 Information Sinks Functional Group ............................................................................. 174.1.3 Large Scale Data Analysis Functional Group ............................................................... 174.1.4 Knowledge Base Functional Group .............................................................................. 194.1.5 Reliable Information Processing Functional Group ...................................................... 204.1.6 Reasoning and Decision Support Functional Group ..................................................... 214.1.7 Actuation Functional Group .......................................................................................... 214.1.8 Exposure Functional Group .......................................................................................... 224.1.9 Framework Management Functional Group ................................................................. 22

4.2 Interface and Information View......................................................................................234.2.1 Large Scale Data Analysis FG internal interfaces ........................................................ 244.2.2 Reliable Information Processing FG internal interfaces ................................................ 274.2.3 Reasoning and Decision Support FG internal interfaces .............................................. 304.2.4 Actuation FG interfaces and information flow ............................................................... 31

4.3 Security and Privacy View..............................................................................................324.3.1 General Security and Privacy considerations ............................................................... 324.3.2 IoT Stream Security and Privacy .................................................................................. 334.3.3 Social Media Stream Privacy ........................................................................................ 344.3.4 Discussion .................................................................................................................... 36

4.4 Smart City Framework and the relationship to IoT-A..................................................36

4.5 Smart City Framework Update.......................................................................................38

5. Designconceptsfortechnicalwork.............................................................................................41


5.1 Large Scale Data Analysis..............................................................................................41

5.2 Reasoning, Decision Support.........................................................................................42

5.3 Reliable Information Processing....................................................................................43

6. StateoftheArt.............................................................................................................................43

6.1 KAT.....................................................................................................................................44

6.2 GSN....................................................................................................................................44

6.3 iCity.....................................................................................................................................44

6.4 iCore...................................................................................................................................45

6.5 IoT.est................................................................................................................................45

6.6 OpenIoT.............................................................................................................................45

6.7 S-Cube...............................................................................................................................45

6.8 PLAY..................................................................................................................................45

6.9 Smart Santander..............................................................................................................46

6.10 SPITFIRE..........................................................................................................................46

7. Conclusion....................................................................................................................................46

8. Abbreviations...............................................................................................................................47

9. References....................................................................................................................................49

Figures

Figure 1 – High-level view of Smart City Framework.....................................................................2Figure 2 – Smart City Framework scope.......................................................................................11Figure 3 – Smart City Framework high level architecture............................................................15Figure 4 – Smart City Framework functional view........................................................................16Figure 5 – SCF Knowledge Base....................................................................................................19Figure 6 – Information flow of the Large Scale Data Analysis FG.............................................25Figure 7 – Reliable Information Processing architecture.............................................................28Figure 8 – Reasoning and Decision Support FG interfaces and Information Flow..................31Figure 9 – Actuation FG interface and information view..............................................................32Figure 10 – Tweets reporting various concerns about a city spanning power supply, water quality, traffic jams, and public transport delays...........................................................................34Figure 11 – A sample twitter text with annotation regarding the topic and location.................35Figure 12 – Reporting location and using Geo-hashing for defining boundary of an event...35Figure 13 – Event concepts in social data analysis......................................................................36Figure 14 – A set of events in a social data analysis scenario...................................................36


Figure 15 – Mapping a subset of the CityPulse functional components to the IoT-A reference architecture.........................................................................................................................................37Figure 16 – Update of the Smart City Framework at the end of year two (2) of the project...39


1. Introduction Acityisaformofhumanhabitatextremelydiverseintermsofinfrastructureandpeople.SmartnessisoneofthequalifiersforacitythathasmainlyemergedfromtheInformationandCommunicationTechnology(ICT)community.Thetermsmartcityisnotwelldefinedintheliteratureandsubjecttointerpretationaccording to the scopeof activity that the term isused in. For thepurposesof theCityPulseprojectwechoosetoscopethetermSmartCityfrom[Holler14].Insummaryaccordingto[Holler14]aSmartCityisacitythatutilizesICTinordertoa)efficientlymanageexistinginfrastructureandassist indevelopmentofnew infrastructure,b)provideefficientservices tocitizens,c)enableefficient organization of city authorities in order to deliver these efficient services to citizens in afashionthatcompliestotheenvironmentalmandatesandd)enabletheprivateandpublicsectortodevelop new and innovative businesses that serve the citizens. A city contains a large number ofphysical infrastructure and human capital that interacts with each other and motivates theintroductionofICT:

a) Acitycontainssemi-staticphysicalorhardinfrastructureforexampleroads,parks,buildings,trainstracks,etc.Thisinfrastructureneedstobemonitoredforefficiencyandfaultdetectionreasons

b) A city contains energy, water and waste infrastructure in order to support living thatpotentiallyneedmonitoringandcontrol

c) Acityinvolveshumans,animalsandthingsinvolvedinthedailyhumanactivitieswhichcouldbesupportedbyacity-wiseICTsystem

Acity-wiseICTsystemsupportingthecityauthorities,businessesandthecitizensneedtoaddressafewfundamentalchallenges:

1) Alargenumberofheterogeneousanddistributedinformationsourcessuchasinfrastructuretomonitortheenvironmentandhumanactivity(e.g.roadtraffic)

2) A smart city is a dynamic interaction environment and therefore the large number ofinformationsourcesproducevastamountsofreal-timecapturingthecurrentstateofthecity

3) Humans through social interactions also generate city related information that is typicallydisseminatedthroughsocialmediaortheInternetofPeople(IoP)asopposedtoInternetofThings(IoT)

4) Novel servicesoffered to citizensneed to combineall the IoT and IoP sources inorder toconstructacompletepictureofalivingcity

InorderforasmartcitytobecomearealityandaddresstheabovechallengesthegeneralstructureofanICTsystemsupportingasmartcityaswellasanumberofgeneralguidelinesaboutthesmartcity ICT infrastructure should be provided. This structure and guidelines is what a “Framework”encompassesinthecontextofthisdeliverable.Themainreasonforprovidingaframeworkasopposedtospecificarchitectureandtechnologyrecommendationsisbasicallytwofold:a)Thediversityofthecities intermsof theirhard infrastructureandhumancapitalandb) theeverchangingnatureand


capabilities of technology. A framework can be adjusted to a small or a big city aswell as to thetechnologiesoftodayandtomorrowtorealisetheconcreteICTsupportplatformforasmartcity.

Thedefinitionofaproblemisthesinglemostimportantstepformovingtowardsthesolution.HencetheproblemofthespecificationoftheSmartCityFramework(SCF)muststartwiththedefinitionofthe framework itself. The definition contains information about the technical objectives of theframeworkas stated in theCityPulseDescriptionofWork (DoW), thedescriptionof thegroupsofindividualsthathaveastakeontheframeworkandasetofconcernsmanifested inrequirements,designgoalsandconstraints.Inaddition,theframeworkshouldincludethedescriptionofthescopeandbaselineassumptionsguidingtheformationoftheframeworkaswellasthesetofcomponentsthatcomprisetheSCFtogetherwiththerelationshipbetweenthecomponents. ItshouldbenotedthatnotallpartsoftheSCFarefinalizedinthisdeliverable.Ratherthisdeliverableaimsatsettingtheinitialframeinwhichotherworkpackages(WP)willdevelopthroughoutthecourseoftheproject.

TheSCF reuses concepts fromexistingplatforms, ICTnetworksand Internetof Thingsenablers todevelopandtestadistributed framework for thesemanticdiscoveryandprocessingof large-scalereal-timeIoTandrelevantsocialdatastreams.TheSmartCityFrameworkaimsatbridgingthegapbetween the related technologies and platforms (e.g. iCity platform[iCity]) and the higher-levelknowledgethat isrequiredfromintelligentdecision-makingbycitizensandcityoperationservices.While the existing solutions focus on publishing the data and creating service enablers tointeract/accesstoIoTinfrastructuresinthecities,theSCFfocusesonintegrationandfederationofIoT(andsocial)datastreamsandprovidesprocessingandanalysismechanismstoaggregate,summarizeand create higher-level abstractions from myriad of multimodal streaming data. This provides amechanismforrespondingtoreal-timequeries,andeventdetectionandknowledgeextractionanddiscovery processes that are required by end users and city operators to make intelligent andsituation-awaredecisions indifferentdomains fromdaily-life tasks (e.g. commuting in the city) tooperationplanningandmaintenance.

Thedeliverableisstructuredasfollows.Section2providesahighleveldefinitionoftheFrameworkcovering the objectives of framework, the stakeholders and their concerns and the boundaryconditions for a smart city framework. Section 3 presents a high level view of the frameworkoriginatingfromthedescriptionofworkoftheCityPulseproject,whichmainlycoversworkpackageresponsibilities. Section 4 describes in detail the Smart City Framework from a number of viewsaccordingtothecurrentpresentationstyleofmodernarchitectures.Section5includesadescriptionofthemaindesignconcepts,whichguidedtheSmartCityFramework.Section6outlinesthestateoftheartinexistingframeworksandfinallySection7concludesthereport.

2. Definition of the Smart City Framework TheintentionbehindtheSCFistobeusedbycertaingroupsofpeoplewhoactindifferentroles(alsoreferredtoasstakeholders)inordertoaddresstheirconcernswhentheframeworkistobeappliedasatoolforaddressingsmartcityusecasessuchascurrenttrafficconditionsinacertaincityroad,utilizationofbustransport,safeeventservicesetc.ThereadercanrefertotheDoWortheDeliverableD2.1[CityPulse-D2.1]formoreusecaseswheretheSCFisapplicable.Thereforethefirststepintothedirectionofdefiningaframeworkistodefinetherolesofpeopleinterestedinsuchaframework.


2.1 Smart City Framework stakeholders TheSCFstakeholderscanbefollowing:

1) CityStakeholders.Thesecanbegroupedintothreesub-categoriesa. CityITservices:Sincetheprojectcreatestheblueprintsaswellasservicesforsmart

cities,theseshouldbetypicallyretrofittedintheexistingITinfrastructureofacity.TheresponsibleindividualsforsmartcityinformationinfrastructurearethemembersofthecityITpersonnel.

b. City departments: The city departments that solve specific problems such astransportation,wastemanagement,lightingetc.

c. Citydecisionmakers:Thedecisionmakersofacityaretheoneswhodecidehowtoallocatefundsforthedevelopmentofanewsmartcityservice

2) Thirdpartyproviders:Enterprisesthatprovideservicestothecityby,forexample,developingphonemobileapplicationswhichuseopencityinformationsources.

3) Citizens:Citizensarethemainusersandalsoendusersofthesmartcityframeworkservices,and requirement providers for applications. The assumption is that the citizens use cityauthorities as a means to voice their concerns. Therefore, citizen concerns are indirectlyexpressedastheconcernsofthecityauthorities.

2.2 Stakeholder concerns Deliverable D2.1 [CityPulse-D2.1] focused on the collection and ranking of smart city scenariosaccordingtothefollowingevaluationcriteria:

• Userdifferentiation:Thiscriterionmeasurestheimpact(positiveornegativeofascenario)tocitizen’sdailylifeaswellasthedegreeofacceptancebythesociety.

• CityRelevance:Thiscriterionmeasuresthedegreethatthescenarioisrelevanttoacity.• DataStreaming:Thiscriterionmeasureswhetherornottheavailabledataarestreamingor

not• DecisionSupport:Thiscriterionindicatesthelevelofcomplexityofthescenario• Big Data: This criterionmeasures the existence of big data volumes and large number of

sources

DeliverableD2.1[CityPulse-D2.1]alsoproducedalistofrequirementsrelatingtheabovecriteriaandusecases.TherelevanttechnicalrequirementsfortheSmartCityFrameworkaretheDataStreaming,Decision Support and Big Data. Herewe provide a consolidated list of the relevant requirementswrittenintechnicallanguage.Pleasenotethatthewords“SHALL”,“SHOULD”,“SHOULDNOT”havesimilar semantics as in the Internet Engineering task Force (IETF) Request For Comments (RFC)documents[RFC2119]:

1) RequirementsonDataStreaminga. TheSCFSHOULDexposethedatabyAPItoapplicationsb. TheSCFSHOULDsupport(near)real-timestreamingofdatac. TheSCFprocessingpowerSHOULDbesufficienttodeliverthedatatotheapplications


d. The SCF SHOULD support secure mechanisms of collecting, processing anddisseminatinginputandprocessedinformation

e. THESCFSHOULDsupportreliabilityofdataandinformationtransport2) DecisionSupport

a. TheSCFSHOULDsupportmultipleanddifferenttypesofinputdatab. TheSCFSHOULDsupportactuationandcontrolloopsandautomationc. TheSCFSHOULDsupportautomation

3) BigDataa. TheSCFSHOULDbescalableintermsofusersanddatastreamsb. TheSCFSHOULDhaveprivacypreservingmechanisms

2.2.1 Discussion TheabovetechnicalrequirementlistaswellastheselectedscenariosfromdeliverableD2.1wereusedasabasisforthedevelopmentoftheSCFandtoignitefurtherstudyforthedifferenttechnicalworkpackages.HerewesummarisesomeofthemotivationofthechoicesoftheSCFcomponentsbasedontheseindividualWPstudies.

With some of the scenarios requiring complex inferences to be performed on computationallyintensive reasoning tasks, on the technical side the decision support components needs to beexpressive enough to deal with missing knowledge, conflicts and optimization problems. Thecomplexityofthereasoningisnottheonlytechnicalrequirement.InfactDecisionSupportcapabilitiesneedtocaterforadaptabilityandcontinuousevaluationthroughincrementalreasoning,sothatonlythemostrecentandpertinentknowledgeabout therealworld is taken intoaccount.Theneedtoreasonaboutmissinginformationviacommonsenseanddefaultknowledgeledustowardstheuseofnon-monotonicreasoningframeworksthatcanalsocaterforconflictresolutionandconstraints,andthatareextendedwithnativestreamingoperatorsforincrementalevaluationoftheinferencerulesoverslidingwindows.

Some of the selected scenarios deal with data and information created by people using mobileapplicationsorwebinterfaces.Somescenariosmakeanextensiveuseofsimplephysicalsensors,e.g.temperatureorpollutionsensors.Duetofaultysensorsorintentionallyprovidedmisinformationthedatasetshavetobeevaluatedbeforeusage.Toachievethis,alldatasetsareannotatedwithqualityvaluesdescribingthegradeoftheprovidedinformation.Theexpectednumberofdatastreamsontheonesideandlimitedmemoryandcomputationtimeontheothersiderequiretheuseofincrementalalgorithmstoannotatedatasets.Todisregardincorrectinformationsourcesanadditionalreputationhastobeintegratedintotheframeworkthatratesthetrustworthinessofdatasourcesovertimeandallowapplicationstoselectthemostproperandtrustydatastream.Aggregationmechanismshavetobe applied to efficiently monitor a sensor’s past behaviour. Additionally, the integration of testpossibilitiesfornewapplicationsensuresthattheyareworkingcorrectlybeforetheyareusedintherealworldenvironment.

2.3 Baseline assumptions, scope and boundary conditions On a very high level the realization of the framework can be presented as in the Figure 2,whichprovidesafirstorderapproximationforthescopeoftheframework.Theframeworkcanbeusedasa


tooltoaddresssmartcityissuesandthereforeitshouldbeutilisedbysmartcityrelatedapplications.Similarly,theframeworkneedstouseinformationsourcesandsinksrelatedtoasmartcity.ThefigureshowsthatInternetofThings(IoT)Sensorsdeployedinacityenvironmentcanbeonetypeofsourceofreal-timeinformation.MoreoverIoTActuatorscanbethesinksofinformationcomingfromeitherthe framework or the applications. Another type of information source is the legacy informationsourcesprovidedalreadybyacitye.g.OpenDataportals,cityGIS(GeographicalInformationSystem)dataetc.ACityInformationSinkrepresentsthecityinfrastructurethattheframeworkcanusetostoreprocessedinformationproducedbytheframework.ApartfrominformationcomingfromsensorsortheCityInformationSources,informationaboutthesituationinacitycanoriginatefromthecitizensthemselves through the use of socialmedia (e.g. twitter feeds). Therefore these sources are alsoincludedasrelevantinputssourcesfortheframework.ThedifferencebetweenthesesourcesandtheIoTorCityInformationSourcesisthattheformerarepotentiallysourcesofunstructuredinformationexpressedinnaturallanguage(e.g.atweetisashorttextwrittenbyahumanwithpotentiallyalinkto a website potentially written by a human) while the IoT or City Information Sources producestructured data although not standardized. The Social Information Sinks represent social mediachannelsthroughwhichcitiescouldpotentiallypushinformationtotheircitizens,e.g.achannelforcitizenstoreceiveupdatesonthetrafficsituationincertainpartsofthecity.

Figure 2 – Smart City Framework scope

Fromthehighlevelviewoftheframeworkwecanmakethefollowingobservations.Pleasenotethatthewords“SHALL”,“SHOULD”,“SHOULDNOT”havesimilarsemanticsasintheInternetEngineeringtaskForce(IETF)RequestForComments(RFC)documents[RFC2119].Forcompletenesswereiterateherethemaintermsusedbelow."SHALL",meansthatthedefinitionisanabsoluterequirementofthe specification. “SHOULD” means that mean that there may exist valid reasons in particularcircumstancestoignoreaparticularitem,butthefullimplicationsmustbeunderstoodandcarefullyweighedbeforechoosingadifferentcourse.“SHOULDNOT”meansthattheremayexistvalidreasonsinparticularcircumstanceswhentheparticularbehaviour isacceptableorevenuseful,butthefull


implicationsshouldbeunderstoodandthecasecarefullyweighedbeforeimplementinganybehaviourdescribedwiththislabel.

1. TheSCFSHOULDintegrateonlineIoTDataSources,onlineSocialMediaSourcesaswellasCityInformationSources,whicharetypicallysemi-staticsources(forthedefinitionofthesemi-static as opposed to real-time please see below). The interfaces (IoT, Social Media, CityInformation interfaces) supportedby theSCF foraccessing thesedata sourcesSHOULDbespecifiedorrecommendationsSHOULDbeprovidedbytheproject

2. TheSCFSHALLsupportaccesstoSmartCityInformationSinksandSHALLspecifytheinterfaces(IoTActuator,SocialMediaSink,CityInformationSinkinterfaces)towardsthesesinks.

3. TheSCFSHOULDsupportSmartCityApplicationsaccessingthevalueaddedservicesthattheSCF provides by using the above-mentioned data sources. The interfaces for applicationdevelopers (Smart City Application interfaces) SHOULD be specified or recommendationsSHOULDbeprovidedbytheproject

4. TheSCFSHALLsupportSmartFrameworkmanagementportalsasaspecialtypeofapplicationaccessing a special type of interfaces (Management Interfaces). The interfaces formanagementapplicationdevelopersSHOULDbespecifiedorrecommendationsSHOULDbeprovidedbytheproject

5. TheSCFSHOULDproviderecommendationsorSHALLspecifytheinternalSCFarchitecture6. TheSCFSHOULDproviderecommendationsorSHALLspecifytheinternalSCFcomponents7. TheSCFSHOULDNOTstretchinthedomainofIoTSources/Sinks,SocialMediaSources/Sinks

ortheCityInformationSources/Sinksinotherwords,thesetechnicalareasarenotconcernsofanySCFstakeholder. Inotherwords,theSCFdoesnotprovideguidelinesforwhichCityinformationsourcesshouldexist.

8. The SCF SHOULD NOT stretch to the domain of application frameworks for creating SCFapplications.Inotherwords,theSCFdoesnotprovideguidelinesforwhichCityapplicationshouldexist.

Thethreedifferenttypesofinformationsources(IoT,SocialMediaandCityInformation)aretypicalrepresentationsofbothreal-timeandsemi-staticinformation.Byreal-timeinformationsourceswemeanthesourcesthatreportrelativelyfast, i.e. fromafewsamplesperdaytoafewsamplespersecond while by semi-static information sources we mean sources that report relatively slow orcontainnon-changinginformation(e.g.cityrealestatepropertycoordinatesthatmaychangeontheorderofonceperafewtensofyears).

2.4 Contribution of Work Packages and stakeholder concerns TheSCFdefinitionandspecificationascapturedinthisdocumentaimsatprovidingrecommendationsabout certain parts of the framework. Since the framework contains functional components,interfaces etc. the specification ofwhich is subject to further research fromotherwork packages(WP3,WP4,WP5), the framework details shall be finalised after the delivery of this document inmonth 12 (M12). This means that this specification is inherently a first attempt to capture anintegratedsystemarchitectureandcertaindetailswillchangethroughoutthecourseoftheproject.Thelistbelowdescribesthetypesofresultseitherinpaperspecificationand/orsoftwarecomponents


that CityPulse aims at delivering throughout its duration. A work package (WP) name inside theparenthesesindicatesthattheparticularWPhasprovidedinputforthisdeliverableuntilthedeliverydateandthatthespecificWPisresponsibleforthefurtherspecificationofthecorrespondingpartoftheframeworkevenafterthedeliveryofthisdocument.

1) Overallarchitecture(WP2/D2.1,D2.2)2) Informationmodels(WP3,WP4,WP5)3) Softwarecomponents(WP3,WP4,WP5,WP6)4) Interfaces(WP2/D2.2,WP3,WP4,WP5)5) Tools(WP3,WP4,WP5,WP6)6) Applications(WP2/D2.1,WP3,WP4,WP5,WP6)

Aninitialmappingbetweenthesepartsoftheframeworkspecificationandthestakeholders,isgiveninthetablebelow.An“X”indicatesthatastakeholdermaybeinterestedinthespecificpartoftheSCFbecausethispartpotentiallyaddressesthespecificstakeholderconcerns.TheCityITservicesareinterestedineveryfacetoftheframeworkandspecificcomponentssincetheneedtoknowhowthesystemworksinordertosupportothercitydepartments,thecityauthoritiesorthecitizens.TheCityDepartmentsasmainlyusersof the frameworkaremainly interested in thetoolsandapplicationsusingtheframeworkandtheinformationmodelthatthesetoolsandapplicationuse.TheCityDecisionMakersaswellasthecitizensneedtounderstandtheframeworkmainlyfromtheservicesitprovidestothecitizensandthereforetheyareinterestedonlytheintoolsandapplicationsthatthecitizenscoulduse.TheThirdPartyprovidersthatcouldpotentiallycontributetothecitywithsoftwareintheframework, tools and applications need to know the respective details aswell as the informationmodels,interfacesandoverallarchitectureinordertodotheappropriatesystemintegration.

Deliverable/Stakeholder

CityStakeholdersThirdpartyprovidersCityITservices

CityDepartments

CityDecisionmakers

Recommendationdocuments

Overallarchitecture X X

Informationmodels X X X

Softwarecomponents X

Interfaces X X

Tools X X X X

Applications X X X X


TheevaluationoftheSCFaswellasindividualresultsfromeachWPisaresponsibilityofWP6andtheresultswillberecordedtoWP6deliverables.

3. Description of Work High Level architecture The high level architecture presented in the DoW is shown in Figure 3. The figure presents an“architecture” picture that combines functional groups/components and interfaces with WPresponsibilities.ThishighlevelarchitecturewasusedasastartingpointforthespecificationoftheSCF.TheLarge-ScaleDataAnalysisfunctionalityaddressesissuesthatarerelatedtotheintegrationofalargescaleofheterogeneoussourcesproducingreal-timestreamsandtheirsemanticenrichment.TheReal-TimeIntelligencefunctionalitytacklesissuesthatarerelatedtotheabilityoftheSCFtoadapttochangingsituationsbasedonthereal-timeinformationstreams.Itisresponsibleformonitoringthesemanticallyenrichedstreamsandadaptingthecollectionofstreaminformation.Itisalsoresponsiblefor providing an API towards the Smart City Applications. The Large Scale Analysis and Real-timeIntelligencefunctionalitiesaresupportedbya-prioriknowledgeintheformofaknowledgebaseandreliabilitycontrolmechanismsfortheincomingrawaswellassemanticinformation.

OneobservationaboutthehighlevelarchitecturefromtheDoWisthatitdoesnotaddresstheissueofactuation,informationdisseminationorstorage.TheDoWpresentsaCityPulsesystemasmainlyadataanalyticssystemwhileaSmartCityFrameworkisbothananalyticsandactionframeworkatleastfromaconceptualpointofview.Therefore,thepresentationoftheSCFalsoincludestheactuation,informationdisseminationandstoragefunctionalcomponentsforthesakeofcompleteness.Howeverthedescriptionofthesecomponentsisratherhighlevelasatthetimeofthewritingofthisreportthereisnocorrespondingworkplanfortheprojecttoaddressthesecomponents.


Figure 3 – Smart City Framework high level architecture

4. Smart City Framework Specification ThespecificationoftheSCFfollowsthebestpractiseswithrespecttothespecificationofasystemarchitecture.WefollowtheapproachofRozanski&Woods[Rozanski11]ofpresentinganarchitectureusingmultipleviewsandperspectiveswiththeexceptionthatwelimitthepresentedviewsforSCFsince it ismainly a framework and not a concrete systemarchitecture. The difference between aframeworkandaconcretesystemarchitectureisthefollowing:Aframeworkisdescribedwithonlyalimitednumberofviews(e.g.functional,interfaceetc)sinceitprovidesrecommendationsabouthowthesystemshouldfunctionshoulditbeimplemented;aconcretesystemarchitectureisadescriptionof a functioning systemwhich in turnmeans thatmore details could be described such aswhichmachineshostwhichsoftwarethatimplementscertainfunctionality.ForthedescriptionoftheSCFwehavechosentoincludethefollowingviews:

• FunctionalView:Thisisadescriptionofwhatthesystemdoeswithoutprovidingdetailsonhowandwherethefunctionsarerealized.

• Interface and Information view: This is a description of how the different functionalcomponentscommunicatewitheachotherandhowinformationisgenerated,howitflowsandhow it is transformed in the framework.Typically inasystemarchitecturedescriptioneachoftheseviewsiscoveredinaseparatesectionbuttheseviewsarequitesimilarforthe

WP6SmartCityApplication

Reliability

Control

SmartC

ontrol

ofSem

antics

WP3Large-ScaleDataAnalysis

ReliableInform

ationProcessing

Datastream

Virtualization

Federation(SensorFusion)

Aggregation(DataFusion)

IoT Socialmedia

API

SemanticIoTstream

Configuration

APrioriKn

owledge

WP3,5

WP4WP5Real-TimeIntelligence

User-CentricDecisionSupport

SmartAdaptation


CityPulseSCFsincetheSCFismainlyadataflowframework.InotherwordstheCityPulseSCFdescribesstreamsofdata flowing through theSCFcomponentsand thereforemostof theinterfacesbetweencomponentsaredataflowinterfaces.

• Security,PrivacyView:ThisspecialviewdescribesthesecurityandprivacyfunctionsfortheSmartCityFramework.

4.1 Functional View StartingfromFigure3andelaboratingthefunctionalityoftheSCFbasedoninputfromWP3,WP4,WP5,Figure4presentsamoredetailedarchitecturewithblueboxesrepresentingfunctionalgroups(FG) and black boxes/ellipses/cylinders showing functional components (FC) that comprise thefunctional groups. The blue lines represent the interface between the SCF and external systems(Sources, Sinks, Applications). The functional view elaborates on the functional groups andcomponents.

Figure 4 – Smart City Framework functional view

4.1.1 Information Sources Functional Group TheInformationSourcesgroupcontainsthesourcesofcityrelatedinformation.Theseareofthreekinds:

a) InternetofThingsdeploymentswhichrepresentsensorandidentificationtaginfrastructuredeployedinthecityenvironment;thesesourcescanbestreamingasynchronouslyortheycanbepolled

b) Social media sources that represent streaming text messages from social media such asTwitter[Twitter].


c) City information sources that represent both static and dynamic information repositoriesaboutthecitysuchascityplans,periodictrafficreportsfromcityentrypointsetc.

4.1.2 Information Sinks Functional Group TheInformationSinksgroupcontainsthepossiblesinksforinformationorcontrol.By“sinks”wemeanendpointsthatcanacceptinformationandcauseachangeinthephysicalordigitalworld.Theseareofthreekinds:

d) InternetofThingsdeploymentswhichrepresentactuatorandidentificationtaginfrastructuredeployedinthecityenvironmenttowhichcontrolcommandscanbedispatched(actuators)oridentificationinformationcouldbewritten(tags).Anexampleofanactuatorisacitysignthatthecityauthoritiesusetopushpublicannouncementstothecitizens.

e) Socialmediasinksthatrepresente.g.socialmediachannelstowhich informationcouldbesend(e.g.“mainstreetisheavilycongested”).

f) CityinformationsinksthatrepresentbothstaticanddynamicinformationrepositoriesaboutthecitythatcouldpotentiallybeupdatedbytheSCF.Anexampleofacityinformationsinkisatrafficcongestionrepositoryoperatedbythecityauthorities.

4.1.3 Large Scale Data Analysis Functional Group

Virtualization Function

The virtualisation function (not necessarily a single software component) provides open APIs andcommon services to publish real world data streams from IoT, social media and city informationsources,andcreatevirtualrepresentationsofthem.Itusestheselectedinterfacestowardsthedatasources and converts them into streams if applicable and if they are not already in this form.Anexampleofasourcethatcouldbeconvertedintoastreambutnotoriginallyofferedassuchisasensorthat can only be polled. For the sources which cannot be converted into streams such as staticdemographicdata this functionallowsaccess to the relateddatawith a request-response typeofinterface. This function is responsible fortheconnectivityaspectsofon-boardingnewsources forexamplereachability information(e.g. IPaddress), interfacedescription,accesscontrolcredentials,etc.ThisfunctionisalsoresponsibleforadaptationbetweenaproprietarysourceinterfaceandtheinternalSCFinterfacethatothercomponentsuse.

Semantic Annotation Function

Thisfunctionperformssemanticannotationtothevirtualizedstreamsortheresponsesofsemi-staticinformationsources.Inordertoperformthesemanticannotationthisfunctionutilisesaknowledgebaseofstaticandinferredfactsabouttheinformationsource(e.g. location),hencethisfunctionisrelatedtotheKnowledgeBasefunctionalgroup.Theoutputofthesemanticannotationfunctionisasemanticallyannotatedstreamorsemanticallyannotatedresponsefromthevirtualizedsemi-staticinformationsource.


Federation, Aggregation and Mash up Management Function

This functionalcomponent is responsible forthe integrationandabstractionof thefederateddatastreams. Integration of relevant streams is performed by automatic discovery and binding ofstreamingsourcesby thediscoverycomponent (notshown in thepicturetoavoidcluttering).TheDiscovery component utilises a stream metadata store to automatically discover relevant datastreamsand filters thediscoveredstreamsbasedonQoS/QoIconstraintsandpreferencesdefinedwithin the application request. This component is also responsible for the integration ofheterogeneousdatastreamsand(semi)staticdatasourcestogeneratemashedupstreamsaccordingtotheapplicationrequirement.

Event Detection

Theoutputofthedataaggregationmodelcanvaryovertime.Especially insensornetworksor IoTdeployments the state of an observation is constantly changing and most likely following somepatterns. To perceive a concept or phenomenaboth the current and past states are required. Tomodel and include this time-dependent aspect of higher-level concept creation, we combine theprevious staticmodel withmachine learning techniques. Thus, it enables to use the abstractionsobtainedfromthedataaggregationcomponenttoacquireeventsandprocessesthatoccuroveracertainamountoftime.Forinstance,thecongestionleveloftrafficchangesduringadayfromnormaloverbusyandbacktonormalthatrepresentsadailytrafficpattern.Eachofthesepatternscanbemodelledas anew state thateventually canbeperceivedasoutliers. Finally, theeventdetectioncomponent transforms observation andmeasurement data (originated from sensory devices) andrelationsintohigher-levelabstractionstoformaliseconceptsandknowledgefromtheunderlyingrawdata[Ganz13].

Resource/Stream Management Function

Thisfunctioninterfacestherequestsfromotherfunctionalgroups,performsrun-timemanagementofsemanticannotation,aggregationandmashupandeventfiltering.ItincludesaResource/StreamDirectory which manages metadata information about the information sources currently feedinginformation to the framework, the virtualized streams or the virtualized semi-static informationsources,theaggregateandmashedupstreamsandtheirconfiguration(e.g.stream#5isanaggregateof streams #1 and #3 with the aggregation function #56), the event specifications and theirconfigurationsetc.


4.1.4 Knowledge Base Functional Group

Figure 5 – SCF Knowledge Base

Knowledge Management Function

ThisfunctioninterfacesalltheothercomponentsandperformsaccesstothestoresintheKnowledgebaseFG.Themanagementcomponentprovidesaunifiedaccess(end)-pointtotheinformationandoffersthemeanstoupdateandalterresourcesduringtheworkflowtoallowcomponentstoupdateandrectifyparametersthroughouttheirevaluation.Theknowledgebase(KB)containsallinformationthat is acquired through the SCF, which is deemed worthy to be stored. The knowledge baserepresentstheMetainformationabouttheparticularstreamsincludingprovenance,QoIparametersand the streaming data. The KB is represented as a semanticmodel that follows the linked dataapproachtomodelandtracerelationsbetweenparameters,streamsourcesandthedatapayload.ThedatapayloadispartlystoredintheStreamDatastoreuponrequest.TheKnowledgemanagementcomponentservesasasingleentryendpointforallinformationrelatedqueriestoavoidredundanciesandkeepanintegrateddatamodel.

User Profiles Function

Thisisastoreoftheuserprofilesusedfortheuser-centricdecisionsupportdescribedlaterinthetext.The user storagemodels the relations between created andmaintained streams and provides anaccesscontrolmechanismtoproviderightsmanagement.Theuser-centricdecisionsupportreferstodecisionsupportmechanismsthattakeintoaccountthetypesofusers(e.g.elderlypeopleinacityenvironment)makingtherequeststotheSCF.

Semantic Stream Datastore Function

ThisisastoreofselectedsemanticallyannotatedstreamsgeneratedbytheLargeScaleDataAnalysisFGi.e.semanticallyannotatedvirtualizedstreams,aggregated/mashedupstreamsoreventstreams.Pleasenotethatnotallthepossiblesemanticallyannotatedstreamsarestoredhere,onlyaselectedset.Theconfigurationsaboutwhichstreamsarestoredexist intheResource/StreamManagementfunctionoftheLargeScaleDataAnalysisFG.

Virtual Stream Datastore Function

ThisisastoreofrawvirtualizedstreamsgeneratedbytheLargeScaleDataAnalysisFG.Pleasenotethatnotallthepossiblerawstreamsarestoredhere,onlyaselectedset.Theconfigurationsabout


whichstreamsarestoredexistintheResource/StreamManagementfunctionoftheLargeScaleDataAnalysisFG.Forefficiencypurposesthestorageofsemanticallyannotatedstreamsdescribedabovemaybesplitintotwoparts:a)thiscomponent/functionwhichstorestherawstreamsandb)anothercomponentwhichonlystoresthesemanticannotationofthestoredrawstreamswithreferencestothecorrespondingrawstreams.ThecomponentwhichstoresonlythesemanticannotationswiththereferencescouldbetheSemanticStreamDatastoredescribedabove.

Domain Knowledge Function

This contains static/factual as well as inferred knowledge objects and rules. For example, it maycontain city map information annotated with road addresses as well as rules operating on thisknowledgetogroupcertainpartsofacityboundedcertainroadsintoonecityregionwithaspecificname.

QoI Datastore

This store maintains the Quality of Information (QoI) and reputation results produced by theReputationandQoIEvaluationsystemFunctiondescribedbelow.

4.1.5 Reliable Information Processing Functional Group

Reputation & QoI Evaluation System Function

ThisfunctionistriggeredbyneweventsdetectedintheLargeScaleDataAnalysisFG.ItupdatestheQoIdataforoneormorecorrespondingdatastreamsandalsoinformstheReasoningandDecisionSupportFGaboutsignificantchangesoftheQoIinthedatastreams.TheresultingQoIandreputationdescribingadatastreamisstoredintheQoIDatastore.OthercomponentsareabletoquerytheQoIattheQoIDatastore.Also,newdatastreamscanberegisteredwithinitialQoIvalues.

Test Management and Execution Function

This function is responsible to test the reliability, robustness and performance of the Smart Cityapplications. It utilises (a subset of) the CityPulse Reference Data Set (will be described in D2.3)injected as test data into the Virtualization component through the SCF interface. The TestManagementFunctioncontrolstheTestExecutionandalsoevaluatestheavailabilityandreliabilityofinformationprovidedforapplicationsaswellastheapplication’soutput.

Monitoring Function

This function observes data streams and detects outliers, violations against the QoI and otherinconsistencies,e.g.thebreakdownofadatasource.Ifnecessary,e.g.becauseofmismatchingQoIrequirements,thefaultrecoveryfunctionisinformed.Unlikethetestingthisisdoneduringruntimeand hence has to copewith large amounts of data. To avoid performance issues themonitoringfunctionthereforedirectlyinteractswiththestreamvirtualisation.


4.1.6 Reasoning and Decision Support Functional Group ThisFGcatersfortheadaptivecapabilityoftheframework.Thisadaptabilityisrealizedintermsofflexible control for configuration of relevant data sources and context-driven filtering of relevantevents for decision support considering usage patterns and profiles of classes of users to suggestoptimalconfigurationofsmartcityapplications.

Real-time Adaptive Reasoning Function

Theadaptivefunctionalityofthiscomponentlaysintheabilityofidentifyingandreactingtochangesintwodifferentways:

1) monitoringanddetectingchangesinNon-FunctionalProperties(NFP)attheservicestreamlevel,tosuggestchangesinthediscoveryandbindingprocessifduetosuchchange,someapplication-specificconstraintsonthosepropertiesareviolated

2) monitoringeventsthatmightberelevantforthereasoningtasktobeperformed,matchingthoseeventsagainstcontextualrequirementsanduserprofilestodeterminewhethertheyrefertoarelevantchange intherealworldortheyneedtobefilteredout (thisoperationmightneedfeedbackfromtheuser)

Decision Support Function

This functionalblock is inchargeof reasoningaboutevents thatare relevant foraparticular task.Theseeventsaredetectedbytheadaptivereasoningfunctionalblock.TheDecisionSupportfunctionalblock has different reasoning modules that are application dependent (such as a module thatcomputes the optimal path from one location to another, according to some constraints andpreferencesspecifiedbytheuserexplicitlyorimplicitlyderivedbyusers’profiles),andwillalsoprovidea genericmodule to incorporate (implicit andexplicit)usagepatterns, contextual informationandconfigurationpreferencestoprovidesupporttodecisions.TheDecisionSupportfunctionalblockisalsoincludingthefunctionalitiesthatallowderivingandupdatinguserprofiles.

Visualization Function

Thevisualizationfunctionwillhavetwomaincapabilities:

- anapplication-dependentuserinterfacethatwouldenableeasyconfigurationofpreferencesandrequirementstobeusedinthedecisionsupportphase

- asetofmodalitiesforgeneratingdiagrams,intensitygraphsandmapoverlaytoprovidetheuserwiththeresultsofDecisionSupportandAdaptiveReasoning

User feedback will be considered and specificmodalities to personalize users’ experiencewill bedesigned.

4.1.7 Actuation Functional Group The actuation FG includes the functions that keep track of the potential actuators that could becontrolled or the information sinks that could be updated as a result of the decisions from theframeworke.g.streetlightstobeturnedoffwhennopeoplearearound.


Virtualization Function

The virtualisation function (not necessarily a single software component) provides open APIs andcommonservicestoallowaccesstoheterogeneouscontrolcommandsorsinksofinformation.Thisfunctionisresponsiblefortheconnectivityaspectsofon-boardingnewsinksforexamplereachabilityinformation(e.g.IPaddress),interfacedescription,accesscontrolcredentials,etc.Thisfunctionisalsoresponsibleforadaptationbetweenaproprietarysink interfaceandtheinternalSCFinterfacethatothercomponentsuse.

Actuation Management Function

Theactuationmanagement functionprovides theexecutionenvironmentof thedecompositionofcomplexactuationtaskstosimpleactuationcommandsfortheactuatorsortheinformationsinks.Forexample a complex actuation task could be to turn off the streetlights for a specific street. Thiscomplexactuationtaskisdecomposedintosinglecommandstoindividualstreetlightsonthespecificstreet. This function also includes concurrency control ofmultiple actuation requests to the sameactuator/sink. This function includes metadata information/descriptions about the actuators orinformation sinks as well as description of complex actuation tasks and their decomposition intosimpleractions.

4.1.8 Exposure Functional Group Theexposurefunctionalgroupincludesonesinglefunctiontomediateexternalrequestsperformedby Applications (Smart City Applications orManagement Applications). Themediation includes a)request routing functions such as redirecting Smart City Application interface calls to the e.g. theReasoning,DecisionSupportcomponentwhileredirectingManagementApplicationinterfacecallstoFramework Management functions, b) request load balancing in the case of a function beingdistributed in multiple physical machines, c) enforcing access control rules (e.g. allowing onlyanonymous and aggregated data access to applications or allowing administrator user to accessmanagementfunctions)andd)formatadaptationofresponsestorequestse.g.theReasoningandDecisionSupportreturnsaresponseinformatAandtheapplicationexpectstheresponseinformatBwithAandBdifferent.

4.1.9 Framework Management Functional Group The Framework management FG covers operational aspects of the framework ensuring that theframeworkdeliverstheservicestotheCitystakeholdersthroughtheCityPulseApplications.Forthispurpose the management FG includes the typical FCAPS (Fault, Configuration, AccountingPerformance and Security) functions. Typically all these functions expose special managementinterfaces for other framework functions to send or retrieve respective information as well asinterfacesformanagementapplication,whicharetypicallyrunbyhumanoperators.

Fault Management Function

The fault management function is responsible for the detection and resolution of faults in theframework. It isassumedthatthisfunctionhandlestheescalationoffaultsandexceptionscomingfromotherframeworkfunctionsandittypicallyinvolvesahumanoperatorforthefaultresolution.


Configuration Management Function

Theconfigurationmanagement functionmaintains the configurationof the frameworkandallowsotherfunctionstoreadandwritethisconfiguration.Thisfunctionalsocontainsasoftwarerepositoryfortheframeworksoftwarecomponents.

Accounting Management Function

Theaccountingmanagementmaintains statistics andaccounting figures for theoperationofeachframeworkcomponentaswelltheinteractionsbetweenframeworkcomponentsandtheinteractionsacross the smart city framework. Thepurposeof this function is toassist any chargingandbillingsystembasedonusagestatistics.Thisfunctionmayberedundantwhennochargingoftheframeworkusageisexpected.

Performance Management

Theperformancemanagementfunctionisresponsibleforthecollectionandanalysisofperformancestatisticsfortheoperationoftheframework.Forexampleasmartcityapplicationrequestmaytaketoolongtobeprocessedintheframeworkandthereforeitsend-to-enddelayperformancewillbepoor.Performancemanagementistypicallyusedalongwithconfigurationmanagementinordertoimprovepoorperformingsystems.

Security and Privacy Management Function

TheSecurityandPrivacymanagementfunctionisresponsibleforthesecureoperationofthewholeframeworkaswellasthenecessaryprivacypreservationfunctions.ThisfunctionincludescomponentssuchasanAuthentication(whichinturnincludesIdentityManagement),Authorization(accesscontrolrulesandenforcementthereof),Keymanagement(foralltheauthentication/encryptionkeysinvolvedintheframework),TrustandReputation,PrivacyPreservation,etc.ThisfunctionispresentedinmoredetailintheSecurityandPrivacyviewbelow.

4.2 Interface and Information View Thisviewdescribestheinterfacesbetweenthefunctionalcomponentsinthearchitectureaswellastheinformationflowbetweenthefunctionalcomponents.ThechoicetomergethesetwoviewswasbasedonthefactthattheSCFdevelopedinCityPulseismainlyadataflowframeworkwithfunctionalcomponentsoperatingindataflowsorreal-timestreams.Incertainviewsthedescriptionsalsoincludeimplementationdetails(e.g.aPublish-Subscribeengine),whichshouldbepartofthedeploymentviewbutwechosetoleavethesedetailsinthisviewforkeepingabalancedpresentationoftheframework.Otherwisethedeploymentviewwouldbeverylimitedcomparedtotheothertwoviewssincereal-lifeimplementationoftheSCFfunctionsareininitialstageinthisphaseoftheproject.TheinterfaceandinformationviewtakeseachofthemainfunctionalgroupsfortheSCFandprovidesadetailedexplanationoftheinterfacesandinformationflowwithintheFGs.


4.2.1 Large Scale Data Analysis FG internal interfaces TheLargeScaleDataAnalysisFGinternalinterfacesaimatsemanticannotationandanalysisofIoTand social network data streams by taking into account dimensionality reduction and reliabilityprocessing.Theframeworkinvolvessixmaincomponents:a)virtualisation,b)middleware,c)semanticannotationc)datafederationande)dataaggregationf)eventdetection.Figure6depictsthemaincomponentsandbasicworkflowoftheframework.Initially,thesensornodestransmiteitherrawdataoraggregateddatatothevirtualisationcomponent.ThevirtualisationcomponentincludesdifferentAPIsandinterfacesandisabletocollectandstoredatafromheterogeneousnodes.Aftercollectingthedata,itforwardsthedatatothreecomponentsofthesystem,namely,semanticannotation,datafederation, and reliable information processing. The data federation component discovers andmashes up to obtain enriched information regarding data stream. In case of the data was notaggregatedanddiscretisedbyanymodule,thepatterncreationanddiscretisationprocessisappliedinthedataaggregationcomponent.Afterwards,theeventdetectioncomponentperformsabstractionprocess using machine learning techniques. The abstracted data is finally accessible through themiddlewarewheredifferentlayerssuchasreal-timeintelligencelayerorgraphicaluserinterfacecanbeusedtoaccessthedata.

Virtualisation

Thevirtualisationcomponentisintroducedtofacilitateseamlessaccessandmanagementofsensorobservationandmeasurementdatabasedonsemanticwebtechnologies.Itdefinesaninterfaceinwhich sensor services can collaborate and cooperate to support data access and integration fromdifferent sensor networks and application services that provides the real world information forintelligent decision making. With the semantic descriptions and service interfaces, it enablesunconstrained access to large-scale and distributed sensor services across heterogeneous sensornetworks


Figure 6 – Information flow of the Large Scale Data Analysis FG

Middleware

Themiddleware component is representedwith double line arrows in the information flow view,whichenablesdeliveryoflargevolumeofdatathatcaninfluencetheperformanceofthesmartcitysystemsthatuseIoTdata.Thedatawrappersmanagethecommunicationprotocolwiththesensorsand provide sensor observations into the system in an effort to annotate them using ontologies.However,whileIoTapplicationsaredistributedsystems,there'stwocrucialpointsthatneedstobeaccomplishedforthecommunicationbetweenthecomponentsoftheframework.Firstly, thedatacomingfromthedatawrappershastobepassedthroughtothedataprocessingcomponentssuchasthe Reliable Information Processing and the Semantic Annotation. Since the components can bedeveloped on different platforms there is a need for a unified, platform neutral format for the


messages to enable the communication. Secondly, the components need a possibility to passsignallingmessagestoeachothere.g.toinitiateprocessesorkeeptheothercomponentsinformedaboutrecognisedevents. Inordertomakesurethat thesepossiblycriticalsignallingmessagesaredelivered in a timely fashion and are not blocked by the continuously published data from thewrappers,differentchannelsareusedforthetasks.

Semantic Annotation

Describingtheobtaineddatastreamforinteroperabilityorfacilitatedsearchisthecoreobjectiveofsemanticannotationcomponent,asmanyinformationmanagementtools.However,theamountoftrafficgeneratedbySmartCityapplicationscanbevoluminous,particularlyforrealtimeapplicationsin environments with resource constrains devices, for example sensors with limited bandwidth,memoryorpower.Therefore,theinformationmodelthatisbeingusedbythesystemnotonlyneedsto explicitly represent themeaning and relationships of terms in vocabularies but also should belightweightinordertoreducethetrafficandprocessingtime.Inthiscomponent,weusealightweightinformationmodeltoannotatesensorydata,whichisbasedonwell-knownmodelssuchasSSN[SSN]andIoT.est[Wang12].

Data Federation and Sensor Fusion

This component enables automated discovery and federation of the heterogeneous data bydetermining relevant sensor data sources and their processing techniques. An event requestcontaining functional requirements (e.g. type of sensors) and non-functional requirements (e.g.QoI/QoSconstraintsandpreferences)isreceivedbythiscomponent.Discoverycomponentprocessesthe event request and discovers the relevant data source by utilising knowledge base containingstreamdescriptionand theirQoI/QoSvalues.Once the relevantdata sourcesaredetermined, thefederation component integrates the semantically annotated data streams according to the userrequests.Variousoptimisationtechniques(e.g. indexingandcaching)arealsoappliedtoefficientlyintegraterelevantstreamaswellasstaticdatasources.Federationcomponentgeneratesmashedupdatastreamsasanoutput.

Data Aggregation

Data aggregation and compression are the most important remedies utilized to decreasecommunicationtrafficinIoT.Dataaggregationordatafusioncanhelptoextractmeaningfulfeaturesfromcollectedsensorstreams.Anothermethodtoreducethecommunicationtrafficforthesensorydataistoreducethesizeofthemessages.Thiscanbeappliedusingdatacompressionalgorithms.However,compressingthedataitselfcouldleadtoalossofinformation(inlossycompression)andthe compression techniques can require higher power consumption as it might require dataprocessing before transmission, and in long term observations (e.g. environmental monitoringapplications)compressiontechniquescanstillproducelargeamountsofdata[Kimura05].Chenetal.[Chen09]giveanoverviewaboutapproachesthatcreateasummariseddatastreamofasetofsensorydatastreamsandusetheaggregateddatafortransmission.Theaggregationofthedatausuallyrelieson the mathematical sum, max, min, average and count aggregate functions [Jesus11]. In large


distributedWSN this usually happens via clustering algorithms; however this can lead to loss ofimportantdatathathasbeenmaskedduetotheaggregationinlowerlayers.

Inthiscomponent,weperformlocaldataprocessingandcreatehigher-leveldataabstractionsthatcan be communicated globally. We process the stored raw sensory data using the KnowledgeAcquisition Toolkit ([KAT]), which is designed to extract and represent human understandableinformationfromrawdata.Thetoolkitincludesacollectionofalgorithmsrangingfromdataandsignalpre-processingalgorithmssuchasFrequencyFilters,dimensionalityreductiontechniquessuchasPCA,Wavelets,FFT,SAX,andFeatureExtractionandAbstractionandInferencemethodssuchasClustering,ClassificationandLogicalReasoning.

Event detection

While the state of sensor observations are continually changing and most likely following somepatterns,weusetheeventdetectioncomponentinordertoperceiveaconceptorphenomenausingboththecurrentandpaststates.Tomodeland includethistime-dependentaspectofhigher-levelconceptcreation,weusemachinelearningmethodstoacquireeventsandprocessesthatoccuroveracertainamountoftime.For instance,thestateofaroadtrafficorcongestion(e.g.busy,normal,low), thatoccurs invarious regions, canbemodelled in theeventdetectioncomponentbasedonseveralabstractionsinferredduringtheday,andderivationsfromthispatterncanleadtoaneweventobservationthateventuallycanbeprovidedtothereal-timeintelligencelayer.

4.2.2 Reliable Information Processing FG internal interfaces The reliable information processing aims at the annotation of information sources with QoI andreputation.ThisannotationandaconflictresolutioncomponentshouldsupportothercomponentsoftheCityPulseframeworkwithhandlingdifferentdatasources.Anadditionallyaddedtestcomponentisresponsiblefortestingnewapplicationatthetimetheyaredesigned.


Figure 7 – Reliable Information Processing architecture

ThereliableinformationprocessingconsistsofthefollowingcomponentswhichareshowninFigure7anddescribedindetailinthefollowing:

• Reputation&QoIEvaluationSystemandQoIDatastore• Atomic&CorrelationMonitoring• ConflictResolution&FaultRecoveryandTechnicalAdaption• TestExecutionandTestManagement

Reputation&QoIEvaluationSystemandQoIDatastore

The Reputation & QoI Evaluation System is the main component of the reliable informationprocessing.ItisresponsiblefortheactiveevaluationofQoIfordatasourcesandtheirsteadyadaptiontriggeredbyeventsthatcouldbesentbythemonitoringcomponentsandtheeventmanagementcomponents of other CityPulse framework modules. In contrast to the simpler monitoringcomponentstheEvaluationSystemhasacompleteviewaboutalldatasourcesandcanfindadditionalrelationshipsbetweenthedatasources.WithadditionalhistoricaldataispossibletoadapttheQoIand save them to the QoI Datastore. In addition the reputation of data sources is calculated tomaintainthetrustworthinessofdifferentdatasources.BoththeQoIandthereputationofasource

Reputation & QoI Evaluation System

QoIDatastore

Larg

e S

cale

Dat

a A

nal

ysis

De

cisi

on

Su

pp

ort

an

d R

easo

nin

g

notifyQoIChange

getStreamQuality

Ato

mic

M

on

ito

rin

g

fetchTestData

injectTestDatacontrol

Smar

t C

ity

Ap

plic

ati

on

evaluate

Co

rre

lati

on

M

on

ito

rin

g

analysecorrelations

updateStreamInformation

register/getStreamQuality

Conflict Resolution & Fault Recovery

User ProfilesDomain

Knowledge

Tech

nic

al A

dap

tati

on

requestFaultRecoverygenerateEstimatedEvents

getRecoveryStrategy

Test Execution

Test Management

Reference Data Set

Knowledge Base

get/updateStreamQuality


aresavedintheQoIDatastore.Thisstoredeliversthisadditionalinformationtoothercomponentsthatneedthemtoselectthebeststreamforaspecificusecase.Ontheflytranslationtosemanticannotationformatsensurescompatibilitywithvariousrepresentationsystems.

Atomic&CorrelationMonitoring

The monitoring module consists of two components. The Atomic Monitoring is responsible forwatching one single data stream for inconsistencies. As an example a data stream for an indoortemperaturesensorshouldonlydelivertemperaturesbetween10and40degrees.Otherwisetheremightbesomethingstrangehappenorthesensorhasadefect.AdditionallytheAtomicMonitoringcouldapplyanincrementaltimeseriesanalysistothedatastreamstomodelanexpectedbehaviourofthedeliveredinformation.Becauseofthefactthatthemonitoringneedsdirectaccesstothedatastreamsitisplacednearthevirtualisationofthestream.

In contrast to the AtomicMonitoring that watches the RAW data of one stream the CorrelationMonitoringcombinessomeAtomicMonitoringcomponentsandwatchestheabstractvaluesthataredelivered. Bases on entity-, time- and geospatial- relationships the different Atomic Monitoringcomponentsareaggregatedandcheckedforplausibility.WiththeCorrelationMonitoringitispossibletodetectfaultyinformationsourceswithinagroupofdatasourcesbycomparisonwithothergroupmembers.

ConflictResolution&FaultRecoveryandTechnicalAdaption

Theconflictresolutionandfaultrecoverycomponentistriggeredwhenoneorseveralstreams,usedby an application, are not able to ensure the sameQoI parameters (as were specified when theapplication was instantiated). Its role is to temporary generated estimated events for the abovementioned streams. Different interpolation methods and prediction models will be used forgenerating theestimatedevents.The interpolationmethodsareusedwhentherearesimilardatasources in the surrounding area of the “faulty” data stream. If there are no other streams in thesurroundingareaapredictionmodelwillbeusedinstead.Inthiscasefaultrecoverycomponentwillcachetheeventsgeneratedbythedatasourceinthepreviousperiod(definedbythedomainexpert,e.g.onehour)andusingthisdatawillgeneratethepredictions.

Thefaultrecoverycomponentgeneratestheestimatedeventsonlyforashortperiodoftime.Iftheproblem persists (the QoI of the stream was not restored to the initial value) the adaptationcomponenttriggerstheresourcediscoverycomponenttofindanalternativedatasource.

TestExecutionandTestManagement

The additional test module adds the possibility to test new CityPulse applications. Therefore areference data set is used. Driven by the use of the TestManagement component the data set isinjectedviatheTestExecution intotheVirtualisation layerof theCityPulseframework.Thecorrectfunction of the new application is then evaluated through some evaluation interfaces. Thewholetesting component is designed to get used within the design time of a new application prior todeploymentforpublicusage.


4.2.3 Reasoning and Decision Support FG internal interfaces Figure8illustratesthecommunicationandinformationflowbetweenthedifferentcomponentsintheReasoningandDecisionSupportFG.

Theflowis initiatedbyauserrequest,comingfromthesmartcityapplicationviathevisualizationcomponentoftheReasoningandDecisionSupportFG.

The implicit user profile and the explicit application request and user requirements as well asbackgroundknowledgearemappedtoamachine-readable format (XML-like)andpassedonto theDecisionSupportcomponent.TheDecisionSupportcomponenttriggersarequestforrelevantup-to-datedatatothestreamdiscoveryservice,whichcomplywithuserpreferencesandinitialapplicationrequirementsbasedonNFP(e.g.accuracy,timeofresponse,andotherQoIparameters).TheresponsetosuchrequestispassedbacktotheDecisionSupportcomponentthatprovidesaninitialresponsetothevisualizationcomponentfortheuser.

Atthispointtwoadaptivemechanismsaretriggeredtilltheusercompletesthetask(e.g.getstoadestinationalongapath):

1. TechnicalAdaptation:thiscomponentreceivesnotificationsfromtheQoIinterfaceinWP4whenevertherearesomechangesintheQoIofthestreamingsourcesthathavebeenselectedbytheDataFederationandMash-upcomponenttoprovidedata;whensuch notified changes are in contrast with the QoI of sources selected for thecompositionplanthatisprovidinganswerstotheapplicationrequest,arequestforadaptation and re-discovery of new more suitable sources is requested to theFederationcomponent.

2. Unexpectedeventdetection(andcontextualfiltering):assoonastheinitialanswerisprovidedtotheuser,theDecisionSupportcomponentsubscribestoasetofeventsthatarerelatedtothereasoningtask(e.g.fortravelplannertask,accidentsorroadworks might be relevant events).When any of such event is detected, the EventManagersendsanotificationtotheContextualFilteringcomponent;theContextualFilteringcomponentconsiderstheuser-dependentqueryresponsegeneratedinitiallytodetermineiftherelevanteventispotentiallycriticaland,ifso,generatesatriggerforanewdatarequesttotheDataRetriever,andcommunicatesalistofcriticaleventsto the Decision Support component to perform a new computation and provideupdatedresultstotheuser.


Figure 8 – Reasoning and Decision Support FG interfaces and Information Flow

4.2.4 Actuation FG interfaces and information flow Figure 9 shows the information flow for the actuation functional group. The information flow istypically unidirectional from a CityPulse application down to an information sink. A CityPulseapplication issues a complex actuation task request and the request is checked in the ExposurefunctionforapplicationauthenticationandauthorizationrightstoaccesstheActuationManagementfunction.TheActuationManagementfunctiondecomposesthecomplexactuationtask intosimpleactuationtasksanddiscoversthesimple informationsinks,whichcanreceivethesimpleactuationtasks. The simple actuation tasks are also checked for the proper authorization rights (e.g. if theCityPulseapplicationhastherighttosendanactuationtasktoaspecificactuator)andthesimpletasksaredispatchedtotheVirtualizationfunction.TheVirtualizationfunctionadaptsthesimpleactuation

Stream Reasoning

User-centric Decision Support

Real-Time Adaptive Urban Reasoning

Reliable Information Processing

Large-scale IoT Stream Processing

Event ManagementData Federation and Mash-up

Users’ profiles

Domain knowledge

QoI & QoSQoI

notification interface

Event N

otification

Adaptation Request

Technical Adaptation

Contextual filtering

Decision support

Profile Learning

User

feedback

Smart City Application

Data Retriever

Visualization

Mapper

User

Query

implicit profile

Composition Plan Event

Subscription

Data Request

Data Response

Critical Events

profile update

Query Response

Query Response


task language to the specific formats/language that the specific information sink accepts anddispatchestheactuationrequest.

Figure 9 – Actuation FG interface and information view

4.3 Security and Privacy View The CityPulse SCF integrates real-time information emanating fromdifferent types of informationsources(IoT,socialmediaandcitysources)andeitherexposesprocessedinformationtousersthroughtheCityPulseapplicationsorconsumesinformationviatheactuationtasks.ThereforetheSCFsecurityissues touch upon IoT, social media as well as streaming & static information security concerns.MoreovertheSCFprocessingofphysicalworldinformationmaytriggerprivacyconcernsthatneedtobeaddressed.

SincethefocusoftheCityPulseprojectisontheanalyticsfunctionalityratherthanthesecurityandprivacyconcerns,wechoosetorelyonthestateofthearttechniquesforaddressingtheseconcernsratherthanre-inventexistingwork.

4.3.1 General Security and Privacy considerations The Security Management function in the Framework Management FG is expected to addresspotentialsecurity issueswiththeSCF. ItconsistsofAuthentication(which inturn includes IdentityManagement),Authorization(accesscontrolrulesandenforcementthereof),Keymanagement(foralltheauthentication/encryptionkeysinvolvedintheframework)andTrust&Reputation.OnepartoftheTrustandReputationfunctionisimplementedwithintheReliableInformationProcessingFGanditcoversthereliabilityandtrustofinformationsources.WithrespecttotrustoftheSCFusersitisassumedthatthereisatrustrelationshipbetweentheusersandtheSCFinthesensethattheSCFincludes in the Authentication function the list of users allowed to access the SCF and in theAuthorization function a list of access rights for each user. With respect to Authentication,


Authorization state of the art solutions such as OpenID[OpenID],WebID[WebID], XACML[XACML]couldalsobeusedintherealizationoftheSCF.

With respect to privacy there are only a fewpotential entry and exposurepoints of private user-relatedinformationthattheSCFneedstoaddress.Privateuser-relatedinformationmayentertheSCFthrough:a)thoughuserregistrationsthatresult intheuserprofilesbeingstoredintheKnowledgeBase,b)theinformationsources(IoT,socialmedia,cityinformationsources).Moreoverpotentiallyprivate information (e.g. user profile information or stream data) may be exposed to CityPulseapplicationsthoughtheCityPulseAPI.UserprofilesasalreadydescribedindeliverableD5.1[CityPulse-D5.1]arestoredforthesolepurposetoassistthedecisionsupportmechanismsandthroughaccesscontrol mechanisms (Authorization function) they are shared only to their owners. Aggregationtechniques are used to create the profiles of groups of users (aggregate user profiles),which aresharedtoanyCityPulseapplicationthroughtheCityPulseAPIiftheapplicationneedsthisinformation(e.g.“usersbetweentheagesof20-40preferspecifickindsofinformation”).

WithrespecttotheinformationsourcesthecityinformationsourcesareassumedandexpectedtobevoidofpersonaluserinformationandthustheIoTandsocialmediasourcesmightbethesourceforanyprivacyconcern.BelowwepresentadiscussiononthespecificsecurityandprivacyaspectsoftheIoTandsocialmediastreamsandhowthestateoftheartcanbeusedtoaddresstheseconcerns.

IngeneralwecanassumethattheSCFdoesnotexpose(viatheExposureFG)individualrawstreamdata(eitherIoTorsocialmediadata)sincetheLargeScaleDataAnalysisFGandReasoning&DecisionSupportFGareexpectedtoaggregateandfusethestreamsthusremovinganypersonalinformation.Examples of enabling technologies for aggregation are summarization techniques (via machinelearning)aredatacubes[DataCubes].

4.3.2 IoT Stream Security and Privacy FortheIoTsecurityandprivacyissuestheSCFcouldreusetherelevantconceptsfromthestateoftheartcomingfromtheIoT-ATrust,Security,Privacymodel[IoT-A-ARM][ioT-A-D4.2]inadditiontothetrustandreputationconceptscoveredintheReliableInformationProcessingFG,whicharestillworkinprogress.TheIoT-AprivacymechanismsarebasedonAuthentication,IdentityManagementandAuthorization in the sense that they allow for derivation of pseudo-identifiers for users (Identitymanagement,Authentication)andaccesscontrolforstreaminformationandSCFservicesenforcedbytheAuthorizationfunction.

Moreoversemanticallyannotated IoTstreams (in theLargeScaleDataAnalysisFG)canre-use thestandardisationeffortsbyboththeW3C[W3C]andOASIS[OASIS]withrespecttoprivacyforwebdatasinceannotatedIoTstreamsdonotdifferradicallyfromannotatedwebdata.Therelevantstandards(including WebID [WebID] as well as Access control languages like XACML [XACML]) have beenenhancedoradaptedthroughsemantictechnologiesandtheyfulfiltherequirementforasecureandprivacy-preservingSCF.


4.3.3 Social Media Stream Privacy Thissectionaddressesonlytheprivacyissuesfromthesocialmediasourceswhilethesecurityaspects(e.g.encryptionofsocialmediastreams)isoutofscopesincebydefinitionsocialmediastreamsarepublic.

ThesocialdatawithinthecontextoftheCityPulseprojectareexpectedtobecollectedfromsocialmediasourcesviaopenAPIs(e.g.TwitterAPI)and/orusersubmittedinformationviasmartdevices(e.g.anapprunningonusersmobiledevices).TypicallythiskindofsocialinformationcollectedfromsourcessuchasTwitterisofteninnaturaltextformasshowninFigure10.ThemotivationbehindtheuseofsocialmediastreamsinthecontextofSmartCitiesistoprovideacomprehensiveviewofeventsinacity(complementingothermodalitiessuchasobservationsprovidedbytheIoTresources).Twitter(a microblogging platform) has developed into a near real-time source of information spanningheterogeneous topics of varying importance. With over 500 million users world-wide, Twittergenerates500milliontweetsaday.

Figure 10 – Tweets reporting various concerns about a city spanning power supply, water quality, traffic jams, and public transport delays

Increasingly,tweetsdoprovide interestingandvital informationsuchasstatusofpublictransport,trafficandenvironmentalconditions,publicsafety,andgeneraleventsinacity.TheCityPulseprojectattemptstoaddressthefollowingresearchquestions:a)howcityinfrastructurerelatedeventscanbeextracted from tweets, b) how event and location knowledge bases can be exploited for eventextractionandc)howaccuratetheextractionofthecityeventsisfromtweets.

HoweveruserswhohavepublishedTwitterdataalsohaveprofilesontheSCFandsometimeswiththeirrealnameandevenwiththeirlocation.EventhoughtheTwitterdataispublicthispublicdatacanstillbepersonal.Privacyinthiscontextisanimportantissue.Theprojectwillconsiderdifferentapproaches to address this issue. We will not develop newmethods and do research in privacypreservationbutwillusethestate-of-the-artsolutionstoaddresstheprivacyissues.

A few key steps are considered in using social data from in CityPulse. For city event analysis it isexpectedthattweetswillbestrippedoffuseridentificationinformationresultinginastreamofdatawhich includes only the text, time, location and other relevantmetadata.Moreover the resultingstreamoftextistaggedviaanautomatedprocess.AnexampleoftheresultoftheremovalorpersonalinformationandthetaggingofatweetisshowninFigure11.


Figure 11 – A sample twitter text with annotation regarding the topic and location

However,tweetscanstillcontaininformationaboutpersons,locationandotherdatathatcanmakeapersonidentifiableorrefertoaneventthathassomeprivacyconcerns.Toaddressthisissue,asocialdataanalysismethodisexpectedtouseapre-definedtaxonomyoftheevents(i.e.aneventontology)inwhichonlytheeventsdefinedinthistaxonomywillbeidentifiedandreported.Theseeventswillbegenericforexampleaccident,fire,trafficjam,airpollution,slowmovingtrafficandtheeventreportwillonlycontain theevent“concept”, startandend timeand the locationof theevent.Figure12shows how location of an event is expected to be reported without any personal identificationinformation.Forinternaltrustandreliabilityissuestheremightbeapre-analysisofthecollecteddatabeforesubmittingittooureventanalysisbutthisanalysiswilltakeplaceclosetothesocialmediasourceandwillnotstorethepersonalinformation.

Figure 12 – Reporting location and using Geo-hashing for defining boundary of an event

Figure13showsasetofsampleeventconcepts,whichweareusingforsocialdataanalysis.TheresultsofthelatterworkwillbereportedinWP3andhereweonlyshowtheeventconceptstoclarifytheprivacyissues.Figure14showsasetofeventsthatwillbereportedafterthesocialmediaanalysis.


Figure 13 – Event concepts in social data analysis

Figure 14 – A set of events in a social data analysis scenario

Intheabovewediscussedanapproachthatwehavetakenintheprojectforprivacypreservationinreporting the city incidents using the social data; however there still could be other security andprivacyissuesandconcerndependingontheuse-caseandhowthedataisgoingtobecollectedandused.

4.3.4 Discussion Overall for both IoT and social data source, there are several state-of-the-art techniques foraggregation,anonymisationandprivacypreservationthatcanbeusedintheCityPulseSCF.

Duringimplementationoftheuse-caseswewillanalysetheprivacyandsecurityrequirements(theprivacyissuesinnon-technicaltermsarealreadyincludedintheuse-casescenariosreportedinD2.1[CityPulse-D2.1])andwewillprovideanapplication levelprivacyandsecuritysolutionfortheuse-casesandwillreportonthebestpractices.

4.4 Smart City Framework and the relationship to IoT-A TheEUFP7IoT-Aproject[IoT-A]providesanArchitectureReferenceModel(ARM)[IoT-A-ARM]forIoTconcretearchitectures.SincetheCityPulseSmartCityFrameworkincludessomeIoTcomponents,asubsetofitsfunctionalarchitecturecouldbemappedtotheIoT-Areferencearchitecture.OnewaytovisuallyrelatethetworeferencearchitecturesisshowintheFigure15below.


Figure 15 – Mapping a subset of the CityPulse functional components to the IoT-A reference architecture

TheimplicationsofmappingtheSmartCityFrameworkcomponentstotheIoT-Aare:

1) IoT-AonlycoversIoTrelatedfunctionalcomponents.OtherCityPulseSCFcomponentsthatarenotIoT-specificaremappedtotheApplicationFunctionalGroupoftheIoT-Aarchitecture.

2) The SCF components mapped to IoT-A functional groups IoT-Service, Virtual Entity (VE),ServiceOrganizationandIoTProcessManagementareassumedtofollowaServiceOrientedArchitectureprinciple. Inturnthismeansthatthesecomponentsexposeservice interfacesthat canbe invokedbyanyother service in the SCFand the requirement that the serviceinterfacesareregisteredintoaserviceregistrysothattheyarediscoverable.

3) TheSCFcomponentsmappedtotheIoT-AfunctionalgroupsIoT-Service,VirtualEntity(VE),ServiceOrganizationandIoTProcessManagementareassumedtoexposetheirinterfacestoanyotherSCFcomponentsintheSCF.ThemotivationisthattheFGsIoT-ServiceVEService,ServiceOrganizationandIoTProcessManagementareassumedintheIoT-Aarchitecturetoexposetheirservicestotheapplicationsi.e.highlevelcomponents.

TheCityPulseSCFcomponentsthatrelatetotheIoT-Afunctionalarchitectureviewarethefollowing:

1) IoT Sources and IoT Actuators map to the IoT-A Device functional group since these aretypicallythesensorsandactuatorsdeployedinacityenvironment.MoreoverSocialMediaSources that report IoT-related information can alsobe conceptuallymapped in the IoT-ADevice FG in the sense that e.g the mobile phone used for reporting e.g. a fire can beconsideredanIoT“sensor”forfires.


2) TheVirtualizationandSemanticAnnotationfunctionalcomponentsproduceeithervirtualrawstreamsorsemanticallyannotatedstreamsofreal-timecityinformation.Thesestreamsfromthe point of view of the IoT-A functional view are IoT Services that expose the real-timestreams through standardized interfaces. Moreover repeating the first implication statedabove, these functional components are assumed to produce IoT-related information. Incontrast a social media stream source that goes through the Virtualization and SemanticAnnotationfunctionsmaynotalwaysproduceIoTrelatedinformation.

3) The Virtual and Semantic Stream Datastores can also be viewed as IoT Services exposingaccesstothehistoricalIoT-relatedinformationcollectedinacityenvironment.

4) The City Information Sources that are IoT- related (e.g. maps, locations of public places,parkingspotavailability,etc)canbeconsideredaVirtualEntityServicesfromanIoT-Apointof view. The reason is that typically theCity Information Servicesprovide the informationannotatedwithaVirtualEntity informationattachedto it,e.g.“theparkingspaceonMainStreethas5freespots”.Here“MainStreet”istheVirtualEntityand“5freespots”istheIoTservicethatprovidesthenumberoffreeparkingspotsonMainStreet.TheCityInformationSources also expose a Virtual Entity type of access interface i.e. an interface that allowsqueriesbasedonVirtualEntityidentifiersasbasicpartsofthequery,e.g.“WhatarethefreeparkingspotsonMainStreet?”.

5) TheEventDetectionFCandtheFederation,Aggregation&MashupManagementFCcanbemappedtotheIoT-AServiceOrganizationandIoTProcessManagementsincetheoperationsthat these FCs represent are typically compositions of semantically annotated streams orworkflowsthatinvolvesemanticallyannotatedstreamsofcityinformation.

6) TheSecurity&PrivacyFCfromtheSCFcanbemappedtotheSecurityFGoftheIoT-AsincethelattercoversTrust,ReputationandPrivacyapartfromthetypicalsecurityfunctionssuchasconfidentialityprotection.

4.5 Smart City Framework Update

WhiletherestofthedocumentincludestheSmartCityFrameworkdesignattheendofthefirstyearoftheCityPulseproject,thissectiondescribesanupdateoftheframeworkattheendofthesecondyear.ThisupdateisnotrequiredaccordingtotheDoW,neverthelesstheprojectdecidedtopublishitasacourtesytotheresearchcommunity.TheupdatereflectstheimpactoftheresultsoftheindividualworkpackagesonthesystemarchitectureandtheupdateshouldbeconsideredastheCityPulsefinalarchitecture. Figure 16 below shows the updated functional architecture annotated with theApplication Programming Interfaces (APIs) for each functional component. The description of thefunctionsandtheAPIsisprovidedbelow.PleasenotethattheAPIsaredescribedinnaturallanguageand not in a specific programming language although some information may exist about theimplementationdetailsofselectedfunctions.AsubsetoftheseAPIsareeitheralreadyimplementedorwillbeimplementedinintheCityPulseprototypeplatform.


Resource Management

TheResourcemanagementcomponentisresponsibleformanagingallDataWrappers.Duringruntimean application developer or the CityPulse framework operator can deploy newDataWrappers toincludedata fromnewdatastreams.FurthermoredeployedDataWrapperscanbedeactivatedorremovedfromthesystem.ForadefinitionoftheDataWrapperpleaseseebelow.

Figure 16 – Update of the Smart City Framework at the end of year two (2) of the project

Data Wrappers and Semantic Annotation

ADataWrapperisthegatewayofadatastreamintotheCityPulseframework.Dependingonthetypeofstream,transporttechnologyorthestreamdataformatmanydifferentDataWrappersmayexist.Basic information about the stream is provided via a SensorDescription. A SensorDescription alsoholds all required information for the semantic annotation of the stream and observations.OperationaldetailsareimplementedextendingasetofabstractclassesintheprogramminglanguagePython.Oftenrequiredfeatures,suchasperiodicpullingofdatafromaHTTPresource,havebeendevelopedwithintheCityPulseprojectandcanbeuseddirectly.The implementationalongwithadeploymentdescriptorcanbebundledandtransferredtotheResourceManagementviatheResourceManagementAPI.


Data Aggregation

TheDataAggregationcomponentdealswithlargevolumesofdatausingtimeseriesanalysisanddatacompression techniques to reduce the size of raw sensor observations that are delivered by datawrappers.ThisallowsreducingthecommunicationoverheadintheCityPulseframeworkandhelpsperformingmoreadvancedtasksinlargescale,suchasclustering,outlierdetectionoreventdetection.To effectively access and use sensor data, the semantic representation of the aggregations andabstractionsarecrucialtoprovidemachine-interpretableobservationsforhigher-levelinterpretationsoftherealworldcontext.Todate,mostofthesmartcityframeworkstransmitrawsensordataandlackenergyefficienttimeseriesanalysisaswellasgranularsemanticrepresentationofthetemporalandspatialinformationfortheaggregateddata.

Data Federation

ThemainobjectiveoftheDataFederationcomponentistocomposesensorstreamsandrespondtoexplicituser/applicationqueriesoverthestreams.Thesensorstreamsaremodelledasprimitiveeventservicesandthequeryismodelledasacomplexeventpattern.Users/Applicationscanalsospecifytheirnon-functionalrequirementsasQoSconstraintsandpreferences.

Event Detection

TheEventDetectioncomponentisgenericandcanbeusedtodetectrelevanteventsforthecitybyprocessingthesensorstreams.ThecomponenthasaJavaAPI,whichallowstheusertodefinecustommadeeventdetectionpatterns.

Contextual Filtering

TheContextualFilteringcomponentaimsatprovidingadaptivefeedbacktotheusersbytriggeringnewconfigurationsthattakeintoaccountthepotentialeffectofdetectedeventsanduser'scontext.Theapplicationdeveloperscantriggerthefilteringmechanisminordertoobtainthecriticalevents,whichaffectauserbasedonher/hiscontext.Theycanalsorankthesecriticaleventsbyprovidingthelogic of ranking factors (eg. the closer (temporal) events are, themore critical they are).In detail,developersneedtoprovideUserActivity,InterestArea,Effects,andRankFactorswiththeirtotalorderasinputofthefunctionstartCF(UserActivity,InterestArea,Effects,RankFactors)inordertoobtaintheContextualEventwiththecriticality.

Decision Support

TheDecisionSupportcomponentsupportsdecision-makingthattakesintoaccounttheuser-centricfactorsandusagepatterns.User-centricfactorssuchashandlinguserrequirements,preferencesandpreviousapplicationusagepatternswillenablegoal-drivencustomisationofsmartcityapplicationsandprovidededicateddecision-makingsupporttousers.


Fault Recovery

TheFaultRecoverycomponentisstronglyintegratedintheDataWrappercomponent.WhentheFaultRecovery is turned on, an instance of FaultRecovery component is triggered for estimating theobservationswhenthestreamqualityislow.

Atomic Monitoring

TheAtomicMonitoringcomponentisstrongly integratedintotheDataWrapper.Aftersomeinitialprocessingstepsonthereceiveddatabythewrapper,aninstanceoftheAtomicMonitoringiscalledtocalculatethequality.

Composite Monitoring

Thecompositemonitoringprovidesthevalidationofeventsbycomparingthemtocorrelatedstreamsthatcouldaffectorbeaffectedbytheevent.ThisdataevaluationwillbetriggeredautomaticallyifresourcesareavailablebutcanalsobetriggeredmanuallyviaanAPI,whichallowsevaluatingacertainevent.TheAPIcanalsobeusedtoupdatethemodelsofpotentiallyaffectedservicecategories.

Geo-spatial Data Infrastructure

Thiscomponentallowsspatiotemporalsearchesforrelatedeventsandstreams.Itallowsaccesstothegeodata infrastructure,which isbasedonOpenstreetmapandgivesaccess to themultimodalroutingtool.

Technical Adaptation

TheTechnicalAdaptationcomponentmonitorstheQoSperformanceofthequeriesregisteredattheDataFederationcomponentandmakeadjustmentswhennecessary.Therecoveryandadjustmentscan be transparent to the user/application. Currently the technical adaptation module is tightlycoupledwiththeACEISengineandprovidesanAPItospecifythemodeoftheadaptationasfollows:

• createTechnicalAdaptationManager(EventRequest,AdaptationMode).Thismethodinitializesanadaptationmanager instanceforaneventrequest,whichcontrols theqos-awareeventservice adaptation. The AdaptationMode can be local, global or incremental, indicatingdifferentadaptationstrategiestobeused.

5. Design concepts for technical work In this section we present themain initial design concepts which guided the design of themainfunctionalgroupsofthesmartcityframework:a)LargeScaleDataAnalysis,b)ReasoningandDecisionSupportandc)ReliableInformationProcessing

5.1 Large Scale Data Analysis The design of the Large Scale Data Analysis functional group is guided by the following designconcepts:


i. Unifiedaccesstoheterogeneousdatasources.Whilesmartcitydataismulti-source,multi-modalandvariesindataformat,havingaunifiedaccessallowsmodellingtheresourcesinamannertoaccesstheseresourcessystematicallyininteroperableway.

ii. Semantic Interoperability. Using semantic descriptions for sensor data enablesrepresentation, formalisation and enhanced interoperability of sensor data. We useinformation models as a design concept to store semantic concepts that represent aphenomenaandattributesfromtherealworldthatareunderstandableforthehumanuserandalsointerpretableforduemachinestothestandardiseddatarepresentation.

iii. Efficientdataaggregationandsummarisation.Tocopewiththelargeamountofdatathathastobeprocessedandstored,dimensionalityreductiontechniquescanbeappliedtoreducethesizeandlengthofthedatabyapplyingdifferentmethodsonthedatawhilekeepingthekeyfeaturesandpatterns.Theaim is to reduce theamountofdatabyhavingahighgranularrepresentationofthesensormeasurementsattimeswhenthereishighactivityandalowergranularrepresentationintimesoflowactivity.

iv. Real-timeeventdetection.Toperceiveaconceptorphenomenaboththecurrentandpaststatesarerequired.Tomodelandincludethistime-dependentaspectofhigher-levelconceptcreation, we aim to use datamining techniques, whichwill enable to predict events andprocessesthatoccuroveracertainamountoftime,basedonavailableIoTdataorextractingknowledgefromhumansensorydata(onsocialmedia).

5.2 Reasoning, Decision Support Knowledge-basedautomationforstreamdiscoveryandconfigurationreliesonthefollowingconceptsthatwillbedevelopedanddocumentedinfuturedeliverablesbyWP5andthathelpscalability:

i. Uniformrepresentationofstreammetadata.Thisconcept is relatedtothe interoperabilityrequirementandisachievedbythetechnicalworkinWP3throughvirtualizationandsemanticannotation

ii. Re-usabilityofeventpatterns. Ineventdetection, reusabilityofeventpatterncanmake itmore efficient to detect complex events.With the reuse (partial) results of similar eventquerieswherepartialpatternshavebeenalreadyevaluated,thereisnoneedtore-evaluatethewholequery,savingintimeandresourcesthatmightgetcriticalinreal-timeapplications;ThisiswhyefficientdiscoveryandcompositionofstreamingeventservicesinCityPulsehaveabigadvantagewhenreusabilityisconsidered

iii. Real-timemonitoring.ThekeyforconfigurationcontrolistheabilitytomonitorthequalityofthestreaminginformationthatisgoingtobeprocessedinourSmartCityFramework:Real-timemonitoringenablestocontinuouslyverifythequalitymetricsoftheIoTstreamsinvolvedin stream federation and discovery and to take action when the quality drops (adaptivecontrolloop)

iv. Critical assessment. Inorder todeterminewhetheranyparticularupdate in thequalityofincomingstreams iscritical requires to identifyapplication-driven threshold foracceptabledataquality(atthelevelofdatastreams)andquantifytheimpactofhigh-leveleventsontheparticulardecisionprocessthat isusingthem(atthelevelofeventsanddecisionsupport).Thefirstaspectisspecifiedatdesigntimebytheapplicationandcanbeadaptedbytheuserbefore the application is run, while the second aspect requires to assess the contextualrelevanceofaneventanditisdonebybuildingacontextgraphasdocumentedindeliverableD5.1[CityPulse-D5.1].

v. Adaptability:Once the criticality level (be it forqualityupdatesor forhigh-levelevents) isdefined and assessed, the Smart city Framework is expected to have the ability to triggeractions that change the way configuration and processing are performed. For stream


discovery, thismeansswitchingonestreamingsource foranotherwhen thequalitydrops,while for event detection and decision support, it means changing the way reasoning isperformed to take into account changes in the realworld thatmight affect the reasoningoutcome(suchasanaccidentmight invalidateapreviouslycomputednavigationpaththatmightnolongerbeoptimal).

5.3 Reliable Information Processing TheReliable InformationProcessing relieson the followingconcepts thatwillbedefined in futuredeliverablesofWP4toensureefficientdataprocessingandscalability:

i. Uniform quality information representation: To achieve full interoperability a clearlystructuredinformationrepresentationforthedistributionofstreamquality isdefined. It isbasedonsemanticannotationandextensivelydescribedinterfaces.

ii. EventBasedNotification:Decreasingqualityofsmartcitydatastreamscanleadtoimmediateneedof changing information sources for applications. Therefore the reliability processinguses event based publish/subscribemechanisms to inform the Real-Time Adaptive UrbanReasoningandtheDataFederationandMash-up.

iii. Event Based Information Processing and Real-Time Atomic Monitoring: The utilisation ofeventdetectionallowsanefficientanalysisofaggregateddatawithoutanenormousincreaseofneedednetworkinfrastructure.ItisusedtoenablevirtualisationmodulestoinformtheReputation & QoI Evaluation System about events that could lead to quality annotationadaptation.

iv. CompositeMonitoringofAggregatedData:Correlationsofvariousdatastreamscanleadtovalidationordisproofofthecorrectnessofadatastream.Thereforeadaptedknowledgeiscomparedtoinformationprovidedbysimilarstreamswithoutcomparingtherawdata.E.g.theaveragespeedoftrafficinanareacanbecomparedtoareasanddatesoftrafficincidentsandtherebyvalidatetheexpectedbehaviour.

v. IncrementalTimeSeriesAnalysis:Topreventrecurrenttimeseriesanalysisofolddatasets,incremental approaches that use knowledge gathered by previous algorithmic operationsenableanefficientdataprocessing.

vi. Reusing Stream and Application Descriptions to Generate Tests: A model-based testingapproachutilisesinformationofinterfacesandstreamstosemi-automaticallygeneratetestcasesfornovelsmartcityapplicationsandconflictresolutionalgorithms.AReferenceDataSetisusedtoastestdatainputaswellastocomputeatestverdict.

6. State of the Art Thissectionpresentsthestateoftheartonsmartcityframeworksasdefinedforthisdeliverable.TheexistingworkdoesnothaveexactlythesameaimsastheCityPulseprojectandthereforecoverspartsofthetargetedfunctionalitybyCityPulse.

TheSmartCityFrameworkaimsatbridgingthegapbetweentherelatedtechnologiesandplatforms(e.g.iCityplatform[iCity])andthehigher-levelknowledgethatisrequiredfromintelligentdecision-makingbycitizensandcityoperationservices.Whiletheexistingsolutionsfocusonpublishingthedata and creating service enablers to interact/access to IoT infrastructures in the cities, the SCFfocusesonintegrationandfederationofIoT(andsocial)datastreamsandprovidesprocessingand


analysismechanisms toaggregate, summarizeandcreatehigher-levelabstractions frommyriadofmultimodalstreamingdata.Thisprovidesamechanismforrespondingtoreal-timequeries,andeventdetectionandknowledgeextractionanddiscoveryprocessesthatarerequiredbyendusersandcityoperatorstomakeintelligentandsituation-awaredecisionsindifferentdomainsfromdaily-lifetasks(e.g.commutinginthecity)tooperationplanningandmaintenance.

6.1 KAT TheKnowledgeAcquisitionToolkit[KAT]isdesignedtosupporttheprocessofknowledgeacquisitionfromnumericalsensorydata.Itaimstoprovideatoolkitthatisabletoextractandrepresenthumanunderstandableand/ormachineinterpretableinformationfromrawdata.

Thetoolkitincludesacollectionofalgorithmsoneachstepoftheacquisitionworkflowrangingfromdata and signal pre-processing algorithms such as Frequency Filters, dimensionality reductiontechniquessuchasPCA,Wavelets,FFT,SAX,andFeatureExtractionandAbstractionandInferencemethods such as Clustering, Classification and Logical Reasoning. KAT can be used to design andevaluatealgorithmsforsensordatathataimtoextractandfindnewinsightsfromthedata.

6.2 GSN GlobalSensorNetworks(GSN)[GSN]providesagenericplatformfordeployingsensornetworksandprocessingdataproducedbythesensornetworkinadistributedfashion.

GSNachievesthisgoalbysupportingrapidandsimpledeploymentofawiderangeofsensornetworktechnologies,facilitatingtheflexibleintegrationanddiscoveryofsensornetworksandsensordata,enablingfastdeploymentandadditionofnewplatforms,providingdistributedquerying,filtering,andcombinationofsensordata,andsupportingthedynamicadaptionofthesystemconfigurationduringoperation.

ThedevelopmentofGSN isbasedon theobservation thatmostof the requirementsbetweenthesoftwareplatformsdevelopedforsensornetworksaresimilar,anditoffersvirtualsensorsasasimpleand powerful abstraction which enables the user to declaratively specify XML-based deploymentdescriptors incombinationwiththepossibilityto integratesensornetworkdatathroughplainSQLqueriesoverlocalandremotesensordatasources.

6.3 iCity TheEuropeanprojectiCity[iCity]providesaframeworkforsmartcitiestocreate,deployandoperateservices that utilise publically available information from digital assets and infrastructures. iCityextendstheideaofOpenDataandapproachestodevelopOpenInfrastructureswherepublicuserscanaccessandusetheinformationfromICTnetworksthataredeployedinthecities. Theprojectprovidestoolsandmechanismsforthethirdpartyservicedeveloperstoenablesharingandac-cessingtheinfrastructureandthedatafromthesmartcitiestotheend-usersandcityauthorities.InoveralliCityprovidesaplatform for smart cities to share their informationandprovideaccess to thecityinfrastructures. Theplatformwill enable thirdparties todevelop services thataccessanduse thisinformation.Theprojectisexpectedtocreateasetofservicesforpublicinterest(e.g.mobileapps,webservices)createdbythirdpartiesorend-usersthatwillbeofferedthroughtheiCityPlatform.iCityalsoaimstomaketheusersinvolvedincreationandutilisationoftheservicesontheiCityplatform.


6.4 iCore TheEuropeanprojectiCore[iCore]aimstoempowertheIoTthroughcognitivetechnologiessoastoaddresstwokeyissues:(i)howtoabstractthetechnologicalheterogeneityofobjects/devicesintheIoT,whileenhancingreliabilityand(ii)howtoconsidertheviewsofdifferentstakeholdersforensuringproperapplicationprovision,business integrityand, thusmaximizeexploitationopportunities.Thecognitive framework [iCoreArhRef] comprises 3 levels of functionality, reusable for diverseapplications:VirtualObjects(VOs),CompositeVirtualObjects(CVOs)andServicelevel.VOsarevirtualrepresentationsofdevices(e.g.sensors,actuators,smartphones,etc)associatedtoeverydayobjectsorpeople(e.g.atable,aroom,anelderly,etc)thathidetheunderlyingtechnologicalheterogeneity.CVOsarecognitivemash-upsofsemanticallyinteroperableVOs,deliveringservicesinaccordancewithuser/stakeholder requirements. The latter are derived at the Service level, which also includescapabilitiesforbuilding,updatingandexploitingRealWorldKnowledge.Cognitiveentitiesatalllevelsprovidethemeansforself-management(configuration,healing,optimization).

6.5 IoT.est TheEuropeanprojectIoT.est[IoT.est]aimsatdevelopinganIoTservicecreationenvironmentwhilstbridging the gap between various business services and the heterogeneity of networked sensors,actuatorsandobjects.TheapproachemployssemanticservicedescriptionstocomposeIoTservicesand derive corresponding functional conformance tests, semi-automatically. To enable IoTintegration,aconsistentserviceconceptisspecified.Theimplementedplatformenablesre-usageofatomicIoTservicesbycomposingthemwiththeuseofBPMN(BusinessProcessModelandNotation).

6.6 OpenIoT TheEuropeanprojectOpenIoT[OpenIoT]aimsatprovidinganOpenSourceBlueprintforlargescaleself-organizingcloudenvironmentsforIoTApplications.Besidesenablingtheconceptof“Sensing-as-a-service”andprovidingefficientwaystomanagecloudenvironmentsinIoTcontextviautility-basedIoTservices,OpenIoTalsoincludesamiddlewareplatformandtoolsfordevelopmentofcloud-basedIoTapplications.Suchplatformblends IoTwithsemantictechnologies (basedonSSNontology) forbetterinteroperability,andsupportsvirtuallyanysensortypeavailable,includingphysicalandvirtualsensors.Furthermore,theplatformprovidesaccesstoanenhancedversionofGSN(X-GSN),includesseveral visual tools and supports the implementation of on-demand IoT services. The OpenIoTarchitectureisapracticalinstantiationoftheIOT-A[IoT-A]/IERC[IERC]ARM.

6.7 S-Cube TheEuropeanNetworkofExcellenceS-Cube[S-Cube]exploitedthesynergyandlearningeffectsacrosstraditionalresearchboundaries,anddevisedanintegratedsetofprinciples,techniquesandmethodsforengineering,adaptingandmonitoringhybridservicebasedapplications,whileguaranteeingend-to-endqualityprovisionandSLAconformance.

6.8 PLAY TheEuropeanprojectPLAY[PLAY]aimedatrevolutionisingthewaythatPeople,ThingsandServicescooperate in the Future Internet by enabling ubiquitous exchange of information betweenheterogeneousservicesinlargescaledistributedenvironments.Itproposedtheideaofsituational-


drivenprocessadaptabilityandstrivestodevelopanelasticandreliablearchitecturefordynamicandcomplexeventhandling.

6.9 Smart Santander The European project SmartSantander [SmartSantander] is a unique in the world city-scaleexperimental research facility in support of typical applications and services for a smart city. Thetestbed that has been deployed has a dual purpose. On the one hand it allows real-worldexperimentation on Internet-of-Things related technologies (protocols, middleware, applications,etc.). On the other hand it is currently supporting the provision of smart city services aimed atenhancingthequalityoflifeinthecityofSantander.

The framework provides interconnected IoT infrastructure and physical deployment of IoT in thefollowing cities: Santander,Guildford, LubeckandBelgrade. It supports secure IoT communicationprovidingsupportforvariousexperiments.TheSmartSantandertestbedconsistsoffoursubsystems,whichoperateacrossa setofdifferentdevices (IoT,GatewayandTestbed servernodes)providingdifferentcharacteristicsandcapabilities,as follows:Authentication,AuthorizationandAccounting;Testbedmanagement;Experimentalsupport;Applicationsupport.

6.10 SPITFIRE The European project SPITFIRE [SPITFIRE] project aims to facilitate the efficient development ofrobust,interoperableandscalableapplicationsintheInternetofThings.TheSPITFIREkeyobjectivesinclude:enablingsearch,interpretationandtransformationoflowleveldataonthebasisofitsexplicitsemantics, anddeveloping a common semantic abstractionof the realworld entities. The Servicemodel and event-handling in SmartCity may benefit from the approaches developed in SPITFIREproject. However, the focus of the CityPulse project goes beyond providing IoT abstraction layer.CityPulseplanstoprovideanopenservicemarketmatchmakingplatformmakingbothIoTandvariousdata streamsdiscoverable taking into the account environmental dynamics. CityPulsewill providesupportforreactive,smartapplicationsallowingbuildingsmarterandreal-worlddrivenapplications.

7. Conclusion ThisreportpresentedtheCityPulseSmartCityFramework,ahighlevelarchitectureoftheCityPulsetechnicalcomponents,howtheyarerelatedandhowthey interactwitheachother.TheSCF isaninitialstepforotherCityPulseworkpackagestosetacommonlanguageandcommonboundariesoftheindividualinnovationsaswellascitystakeholdersasageneralmapoftheinnovationsthattheprojectwillcontributewith.HowevertheSCFspecificationdescribesmorefunctionalitythanintendedfromthedescriptionofworkandthereforethisfunctionalitywillnotbeexploredordetailedfurtherwithin the technical work packages (WP3-WP6). For example a large part of the frameworkmanagementandmanagementapplicationsarenotexpected tobedeveloped further in researchbecausetheymayalreadybecoveredbystateofthearttechniques.Neverthelesstheframeworkasspecified in this report covers the promised description ofwork and it has already been used byindividualworkpackagesotherthanWP2,fortheirinternalsystemarchitecturedefinitions.


8. Abbreviations ARM ArchitectureReferenceModel

BPMN BusinessProcessModelandNotation

CVO CompositeVirtualObject

DoW DescriptionofWork

FC FunctionalComponent

FCAPS Fault,Configuration,Accounting,Performance,Security

FFT FastFourierTransform

FG FunctionalGroup

FP FunctionalProperties

FP7 SeventhFrameworkProgramme(Europeanresearchprojectfundingframework)

GIS GeographicalInformationSystem

GSN GlobalSensorNetworks

I/F Interface

ICT InformationandCommunicationTechnology

IERC EuropeanResearchClusterontheInternetofThings

IETF InternetEngineeringTaskForce

IoP InternetofPeople

IoT InternetofThings

IoT-A InternetofThingsArchitecture(ECFP7project)

KAT KnowledgeAcquisitionToolkit

KB KnowledgeBase

NFP Non-FunctionalProperties

OASIS OrganizationfortheAdvancementofStructuredInformationStandards

PCA PrincipalComponentAnalysis


QoI QualityofInformation

RFC RequestForComments

SAX SymbolicAggregateApproximation

SCF SmartCityFramework

SLA ServiceLayerAgreement

SSN SemanticSensorNetworks

VE VirtualEntity

VO VirtualObject

W3C WorldWideWebConsortium

WP Workpackage

WSN WirelessSensorNetworks

XACML OASISeXtensibleAccessControlMarkupLanguage

XML eXtensibleMarkupLanguage


9. References [Chen09]Chen,Y.,Shu,J.,Zhang,S.,Liu,L.,andSun,L.,“Datafusioninwirelesssensornetworks,”inProc.2ndISECS,vol.2.May2009,pp.504–509.

[CityPulse-D2.1] Presser, M. (editor), Vestergaard, L., Ganea S., “Smart City Use Cases andRequirements”,EUFP7CityPulseDeliverableD2.1,April2014.

[CityPulse-D5.1]Mileo,A. (editor), et.al., “Real-timeAdaptiveUrbanReasoning”, EU FP7CityPulseDeliverableD5.1,July2014.

[DataCubes]Han,J.,Kamber,M.,Pei,J.,“DataMining:ConceptsandTechniques”,MorganKaufmannPublishers,July2011,3rded,chapter5.

[Ganz13]Ganz,F.,Barnaghi,P.,Carrez,F.,"InformationAbstractionforHeterogeneousRealWorldInternetData",IEEESensorsJournal,vol.13,no.10,pp.3793-3805,2013.

[GSN]GlobalSensorNetworks,http://info.iet.unipi.it/~marco/teaching/AmI/GSN.pdf

[Holler14]Holler,J.,Tsiatsis,V.,MulliganC.,Karnouskos,S.,AvesandS.,Boyle,D.,“FromMachine-to-Machinetothe InternetofThings: IntroductiontoaNewAgeof Intelligence”,Elsevier,2014,DOI:http://dx.doi.org/10.1016/B978-0-12-407684-6.00001-2

[iCity] Linked Open Apps Ecosystem To Open Up Innovation In Smart Cities (iCity),http://www.icityproject.com/

[iCore]CognitiveVirtualObjects(iCore)projecthttp://www.iot-icore.eu/

[iCoreArchRef]MinervaR.,(editor)et.al,“D2.3iCoreArchitectureReferenceModel”,EUFP7project,iCoreDeliverable,http://www.iot-icore.eu/public-deliverables.

[IERC] European Research Cluster on the Internet of Things: http://www.internet-of-things-research.eu/

[IoT-A]InternetofThingsArchitectureproject,http://www.iot-a.eu

[IoT-A-ARM] Carrez, F., Bauer, M., Boussard, M., Bui, N., Jardak, C., Loof, J.D., Magerkurth C., ,Meissner,S.,Nettsträter,A.,Olivereau,A.,Thoma,M.,Walewski,J.W.,Stefan,J.,Salinas,A.(2013).‘Deliverable D1.5 – Final architectural reference model for the IoT v3.0’, Internet of Things –Architecture IoT-A EC Project deliverable D1.5. [online]. Available at: http://www.iot-a.eu/public/public-documents/d1.5/at_download/file

[IoT-A-D4.2]Gruschka,N.,Gessner,D.(editors),et.al.,“ConceptsandSolutionsforthePrivacyandSecurity in the Resolution Architecture”, IoT-A Deliverable D4.2, Feb, 2012, http://www.iot-a.eu/public/public-documents/d4.2/at_download/file

[IoT.est]InternetofThingsEnvironmentforServiceCreationandTestingproject,http://ict-iotest.eu/


[Jesus11] Jesus, P, Baquero, C., and Almeida, P. S., “A Survey of Distributed Data AggregationAlgorithms”,Cambridge,U.K.:CambridgeUniv.Press,Oct.2011.

[Kimura05]KimuraN.andS.LatifiS.,“Asurveyondatacompressioninwirelesssensornetworks,”inProc.ITCC,vol.2.Apr.2005,pp.8–13.

[KAT]KnowledgeAcquisitionToolkit:http://info.ee.surrey.ac.uk/CCSR/KAT/

[OASIS]OrganizationfortheAdvancementofStructuredInformationStandards,https://www.oasis-open.org/

[OpenID]OpenID,TheInternetIdentifyLayer,http://openid.net/

[OpenIoT]OpenIoTproject:http://openiot.eu/

[PLAY]PushingdynamicandubiquitousinteractionbetweenservicesLeveragedintheFutureInternetbyApplYingcomplexeventprocessing(PLAY),http://www.play-project.eu/

[RFC2119]Bradner,S.,“Keywordsforuse inRFCstoIndicateRequirementLevels”, IETFRFC2119,http://www.ietf.org/rfc/rfc2119.txt

[Rozanski11] Rozanski, N., Woods, E. (2011). Software systems architecture: Working withstakeholdersusingviewpointsandperspectives(2nded.)Addison-Wesley

[S-Cube]TheSoftware,ServicesandSystemsNetwork(S-Cube),http://www.s-cube-network.eu/

[SmartSantander]SmartSantanderproject,http://www.smartsantander.eu/

[SPITFIRE]SemanticServiceProvisioningfortheInternetofThingsusingFutureInternetResearchbyExperiment(SPITFIRE),http://www.spitfire-project.eu/

[SSN]Compton,M.,Barnaghi, P., Bermudez, L.,García-Castro,R., Corcho,O., Cox, S.,Graybeal, J.,Hauswirth,M.,Henson,C.,Herzog,A.,etal.“TheSSNontologyoftheW3Csemanticsensornetworkincubatorgroup”,WebSemantics:Science,ServicesandAgentsontheWorldWideWeb,17:25–32,2012.

[Twitter]http://twitter.com

[W3C]WorldWideWedConsortium,http://www.w3.org/

[Wang12]Wang,W.,De,S.,Toenjes,R.,Reetz,E.,andMoessner,K.,“Acomprehensiveontologyforknowledge representation in the internet of things”, in 11th International Conference on Trust,SecurityandPrivacyinComputingandCommunications(TrustCom),pages1793–1798.IEEE,2012.

[WebID]WebID-UniversalLoginandIdentityfortheWeb,http://webid.info

[XACML] OASIS eXtensible Access Control Markup Language (XACML) TC https://www.oasis-open.org/committees/tc_home.php?wg_abbrev=xacml

real-time iot stream processing and large-scale...

Documents