h2020 ict9 project deliverabled3 - chorevolution.eu · particular its three core enablers, the...

H2020 ICT9 Project

Deliverable D3.3

CHOReVOLUTION Service Bus,Security and Cloud - Final out-comes

http://www.chorevolution.eu

LATEX template v. 1.0

Project Number : H2020-644178Project Title : CHOReVOLUTION

Automated Synthesis of Dynamic and Secured Choreographiesfor the Future Internet

Deliverable Number : D3.3Title of Deliverable : CHOReVOLUTION Service Bus, Security and Cloud - Fi-

nal outcomesNature of Deliverable : ReportDissemination level : PublicLicence : –Version : 1.0Contractual Delivery Date : 1 July 2017Actual Delivery Date : 1 October 2017Contributing WP : WP3Editor(s) : Frederic Motte (Thales)Author(s) : Fabio Martelli (Tirasa), Nikolaos Georgantas (Inria), Geor-

gios Bouloukakis (Inria), Patient Ntumba (Inria), RadhaPallavali (Inria), Frederic Motte (THA), Alessio Carenini(CEFRIEL)

Reviewer(s) : Michele Masnata (Softeco), Cristofer Englund (Viktoria)

AbstractThis deliverable describes the third and last version of the CHOReVOLUTION Middleware, and inparticular its three core enablers, the CHOReVOLUTION Service Bus, CHOReVOLUTION Security,and CHOReVOLUTION Cloud, elaborated in WP3.

Keyword ListMiddleware, Service Bus, Security, Cloud, Protocol Interoperability, Binding Component, Timed Automata, Ver-ification, Simulation, Queueing Network Models, Quality of Service, Performance Evaluation, Authentication,Authorization, Identity Management, Security Filter, Provisioning, Horizontal Scaling, Vertical Scaling, LoadBalancer, Dynamic and Secured Choreographies

CHOReVOLUTIONH2020-644178 III

CHOReVOLUTIONH2020-644178 IV

Document History

Version Changes Author(s)0.1 Outline Draft Frederic Motte (Thales)1.0 First preliminary draft Frederic Motte (Thales)1.1 First preliminary draft containing all chapters All authors2.0 Final version ready for internal review All authors

2.1 Comments from internal reviewers addressed

All authors, MicheleMasnata (Softeco),Nikolaos Georgantas(Inria), Cristofer Englund(Viktoria)

2.0 Final version ready for PMC Frederic Motte (Thales)

A.0 Final version Frederic Motte (Thales),Sebastien Keller (Thales)

Document Reviews

Review Date Ver. Reviewers Comments

Outline 07 August2017

1.0Frederic Motte(Thales)

Final outline

Draft 25 Septem-ber 2017

2.0Frederic Motte(Thales)

Final draft released for internal re-view

QA 01 October2017

2.0

Michele Masnata(Softeco), Niko-laos Georgantas(Inria), CristoferEnglund (Vik-toria), FredericMotte (Thales),Sebastien Keller(Thales)

Final version ready for PMC

PMC 01 October2017

A.0 PMC Final version accepted by the PMC

DraftQA

PMC

CHOReVOLUTIONH2020-644178 V

CHOReVOLUTIONH2020-644178 VI

Glossary, acronyms & abbreviations

Item DescriptionA Adapter

API Application Programming InterfaceBC Binding ComponentCD Coordination Delegate

CRUD Create, Read, Update and DeleteCS Client-Server

ChorSpec Choreography SpecificationCoAP Constrained Application Protocol

EE Enactment EngineESB Enterprise Service BusGBC Generic Binding ComponentGM Generic Middleware

GM-IDL Generic Middleware Interface Description LanguageHA High Availability

IaaS Infrastructure as a ServiceIdM Identity ManagerJMS Java Messaging ServiceLB Load BalancerOS Open SourcePS Publish/Subscribe

QNM Queueing Network ModelsPaaS Platform as a ServicePAP Policy Administration Point

PASTA Poisson Arrivals See Time AveragesPDP Policy Decision PointPEP Policy Enforcement Point

RBAC Role Based ACcessREST REpresentational State Transfer

SF Security FilterSOAP Simple Object Access Protocol

FS Federation ServerSTS Security Token ServiceTS Tuple SpaceVM Virtual MachineVSB eVolution Service BusUTC Urban Traffic CoordinationWS Web Service

XACML eXtensible Access Control Language

CHOReVOLUTIONH2020-644178 VII

CHOReVOLUTIONH2020-644178 VIII

Table Of Contents

List Of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . XI

List Of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . XIV

1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

2 CHOReVOLUTION Service Bus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52.1 VSB End-to-End Performance Evaluation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

2.1.1 Test Scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

2.1.2 Test Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

2.1.3 VSB Monitoring System. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

2.1.4 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

2.2 Design Time Analysis of Choreography QoS. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

2.2.1 SEADA choreography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

2.2.2 Interactions within the SEADA choreography. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

2.2.3 Case Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

2.2.4 Experiments. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

2.3 Run Time Analysis of Choreography QoS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

2.3.1 Self-adaptation Framework. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

2.3.2 Monitoring Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

2.3.3 Analysis. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

2.3.4 Experiments. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

3 CHOReVOLUTION Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253.1 Security Filter evolution. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

3.1.1 Security Filter functionalities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

3.1.2 Security Filter architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

3.2 Security applied to the WP5 use case . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

3.3 Improving Performance & Scalability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

3.3.1 Performance of the Security Filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

3.3.2 Performance of the Identity Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

4 CHOReVOLUTION Cloud . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 354.1 Logging tool for developers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

4.2 Automatic adaptation in the cloud layer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

4.3 Assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

4.3.1 Enactment tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

4.3.2 End-to-End Performance Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

5 Conclusions and Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

CHOReVOLUTIONH2020-644178 IX

Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

CHOReVOLUTIONH2020-644178 X

List Of Tables

Table 2.1: Results for one-way interaction in the three scenarios with 300 concurrent senders. . . . . 7

Table 2.2: End-to-End Interactions of SEADA Choreography. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

Table 2.3: Monitoring Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

CHOReVOLUTIONH2020-644178 XI

CHOReVOLUTIONH2020-644178 XII

List Of Figures

Figure 1.1: CHOReVOLUTION components overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

Figure 2.1: Components of the mock environment for the VSB runtime capacity testing. . . . . . . . . . . 6

Figure 2.2: Throughput for one-way interactions through REST, CoAP and DPWS bus protocolsin scenarios 1, 2 and 3. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

Figure 2.3: Response times for one-way interactions through REST, CoAP and DPWS bus proto-cols in scenarios 1, 2 and 3.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

Figure 2.4: SEADA Choreography Components. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

Figure 2.5: Interactions within the SEADA choreography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

Figure 2.6: System Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

Figure 2.7: Queueing network for evaluating CS/DS one-way interactions.. . . . . . . . . . . . . . . . . . . . . . . . 12

Figure 2.8: Queueing network for evaluating one-way interconnections in the SEADA choreogra-phy. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

Figure 2.9: Success rates for the reliable end-to-end interconnection with varying connection/dis-connection and lifetime periods. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

Figure 2.10: Cumulative distributions of response times for the reliable end-to-end interconnectionwith varying connection/disconnection and lifetime periods. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

Figure 2.11: Self-Adaptation Framework. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

Figure 2.12: Queueing Network Model for Client-Server Interactions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

Figure 2.13: Queue Length based on ODEs and Simulations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

Figure 2.14: Response Time based on ODEs and Simulations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

Figure 2.15: Queue Length based on ODEs and Simulations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

Figure 2.16: Response Time based on ODEs and Simulations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

Figure 3.1: Security Filter - architecture. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

Figure 3.2: Security Performance Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

Figure 4.1: Kibana user interface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

Figure 4.2: WP4 choreography model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

CHOReVOLUTIONH2020-644178 XIII

Figure 4.3: WP5 choreography model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

CHOReVOLUTIONH2020-644178 XIV

1 Introduction

The present deliverable reports on the final outcomes of our work on the CHOReVOLUTION Middle-ware, elaborated in WP3, by highlighting the last evolution of the different elements with a particularfocus on performance. The work reported in this deliverable has been driven by the requirements givenin the deliverable D6.1 [14] and by the inputs retrieved from the use cases in WP4 and WP5. These, inparticular, have been determinant in orienting our design and development.

To resume the main architecture already described in the previous WP3 documentes, CHOReVOLU-TION aims at an integrated platform providing full support to dynamic and secure services and thingschoreographies, from their design up to their runtime and evolution.

Figure 1.1: CHOReVOLUTION components overview

Figure 1.1 gives an overview of all the CHOReVOLUTION components distributed on three differ-ent layers: the Frontend, concerning all the front-end components including administration consoles,monitor tools, wizards and so on; the Backend, mainly concerning deployment and security compo-nents; and the third one, Execution, concerning the software entities that come into play in a running

CHOReVOLUTIONH2020-644178 1

CHOReVOLUTION choreography.The CHOReVOLUTION Middleware, vertically distributed on all these three layers but mainly repre-

sentative in the Backend, provides the deployment and runtime infrastructure, on which choreographiesdesigned and synthesized by means of the CHOReVOLUTION Synthesis Processor (see DeliverableD2.2 [12]) are deployed and executed. It also participates in the synthesis process with regard to thesynthesis of Binding Components (BCs) and Security Filters (SFs). More specifically, the CHOReVOLU-TION Middleware is structured in three main enablers: the CHOReVOLUTION Service Bus, CHOReV-OLUTION Security, and CHOReVOLUTION Cloud.

The Service Bus, named eVolution Service Bus (VSB), applies the well-known Enterprise ServiceBus (ESB) paradigm [9] to enable the interconnection of choreography peers at the middleware layer,despite their potentially diverse communication protocols. In D3.1 and D3.2, we described the VSBarchitecture and method for synthesizing BCs, including a common interface description language forheterogeneous services and Things (GIDL). We also provided a number of initial QoS models andrelated analyses for VSB interactions, meaning end-to-end middleware interactions among servicesand Things interconnected via the VSB. We concretize our common interface description language forservices and Things and develop related tools, as part of the CHOReVOLUTION Studio, to automate thesynthesis of BCs from interface descriptions. Additionally, we have QoS models and related analysesfor VSB interactions, particularly considering the potentially intermittent connectivity of choreographypeers running on mobile devices. Our QoS models concern the middleware layer of every service,Thing and artifact contained into the choreography.

The CHOReVOLUTION Security responds to the need for end-to-end security in choreography in-teractions, despite the fact that each participant may join the choreography with different security re-quirements, features and native support mechanisms. Additionally, security expectations by and on achoreography peer will also, and most importantly, depend on the specific role that this peer will haveinside the choreography, as well as on the security policies that will be enforced on the choreography byits stakeholders. In CHOReVOLUTION, we deal with the above challenges with two core components:the Identity Manager and the Federation Server. The Identity Manager is mainly used to implement theService Inventory features, manages the life-cycle of concrete service and end-user profiles and pro-vides all the required features to allow service providers to enroll their concrete services into a serviceinventory. The Federation Server, which is the guarantor of user authentication and interoperability ofauthentication means within the choreography. The outcome of the combined action of the two compo-nents above will have to be applied/enforced on the choreography and its peers. This task is undertakenby the Security Filter (SF) fully generated using the CHOReVOLUTION studio which offer a dedicatedstep for the security enforcement workflow. This filter is fully customized to offer flexibility and efficiencyin a relative transparent manner for the protection and the adaptation of the security into the choreog-raphy by providing custom security plugins (more information the security part). This new feature offeran unlimited possibility for authentication methods both in protection (protection of choreography) oradaptation (for the integration of legacy services)

The CHOReVOLUTION Cloud implements the deployment and runtime infrastructure for CHOReVO-LUTION choreographies, providing the essential support for dynamic, on-demand and elastic provisionof resources. The CHOReVOLUTION Cloud architecture offers a unifying API that allows leveragingseveral cloud underlays, depending on the availability of underlays as well as on the choreographytopology. The latter may integrate both services/Things developed specifically for a CHOReVOLUTIONchoreography (which we call CHOReVOLUTION-enabled or deployable services/Things) and existing(or equivalently third-party or legacy) services/Things, hence deployed and managed independently.Besides the deployment of services and Things, the CHOReVOLUTION Cloud undertakes the deploy-ment and provisioning of all the CHOReVOLUTION entities that participate in supporting choreographycomposition and execution, such as Binding Components, Security Filters, Adapters and CoordinationDelegates. For this, the CHOReVOLUTION Cloud works in full collaboration with the Identity Managerwhich adds a security layer on top of the Enactment Engine API.

This delivery describes the work done during this last period (year 3), to finalize the different platform


elements in order to provide a powerful platform that can support real-life scenarios to end-users. Inparticular, main focus will be given to the new features that will be offered by the different componentsand to the performance result of each. Evolutions are mainly user oriented to provide facilities forcomponent generation, remove functionality limitations and to provide a powerful easy-to-use platform.The WP4 and WP5 use cases are used as reference to illustrate new features, for exemplifying them.

This document is organized as follow.

• Chapter 2 focuses on the last performance evaluation of the VSB including additional middlewareprotocol. Then, the result of the application of QoS model and methodology (presented into theD3.2 [15]) is presented to analyze end-to-end QoS in service/Thing choreographies including theapplication layer. Finally, the presentation of the the initial study for QoS modeling and analysis inservice/Thing choreographies at run time is presented.

• Chapter 3 focuses on the last feature implemented into the security filter offering the possibility, fora designer, to implement it own security mechanisms by providing security plugins (for adaptationand protection). This chapter describes also the performance results of the different securityelements.

• Chapter 4 focuses on new functionalities like the centralized logging features to help the debugof a choreography and the experimental support of automatic computing for autoscaling and selfhealing. Next, performances is discuss to apprehend the benefits of using dedicated and auto-mated solution for enacting choreographies

• Chapter 5 concludes the document and discusses the future directions to be undertaken up tothe end of the project.


2 CHOReVOLUTION Service Bus

The CHOReVOLUTION Service Bus (named eVolution Service Bus, VSB) is the CHOReVOLUTIONMiddleware Enabler responsible for the interconnection of heterogeneous services and Things insidechoreographies. To this end, VSB deploys Binding Components (BCs), which resolve the middlewareprotocol incompatibilities of services and Things by converting these protocols to the common middle-ware protocol designated for the choreography. While VSB BCs can generically work with potentiallyany combination of protocols, we have opted for SOAP as the common protocol by default for CHOReV-OLUTION choreographies as already defined in both D3.1 [13] and D3.2 [15].

In the previous D3.2 [15] deliverable of this series, we: i) introduced a common interface descriptionlanguage for services and Things (under the name of Generic Interface Description Language, GIDL)and we developed related tools to automate the synthesis of BCs from GIDL interfaces; ii) introducedQoS modeling patterns for VSB interactions by considering the potentially intermittent connectivity ofchoreography mobile peers; using these models we provided a methodology for evaluating the end-to-end QoS inside choreographies at the middleware layer; iii) discussed VSB’s readiness for developingreal life scenarios; finally iv) provided an assessment of the VSB development and runtime environmentfor evaluating the support to developers and the performance of the synthesized BCs.

In the present deliverable, we rely on the work done in D3.1 [13] and D3.2 [15] to develop further thefollowing aspects: i) we complete the performance evaluation of the VSB runtime environment by usingadditional middleware protocols (Section 2.1); ii) we apply the models and methodology of D3.2 [15] toanalyze end-to-end QoS in service/Thing choreographies including the application layer (Section 2.2);finally iii) we present the initial study for QoS modeling and analysis in service/Thing choreographies atrun time by introducing a self-adaptation framework (Section 2.3).

2.1. VSB End-to-End Performance Evaluation

In D3.2 [15] we evaluated the performance of the synthesized BCs, which introduce runtime transforma-tions and data conversion among heterogeneous middleware protocols. However, the presented resultswere not satisfactory in terms of the number of tests as well as the utilized middleware protocols. In thissection, we run and demonstrate additional tests for additional middleware protocols using the samemethodology as in D3.2 [15] by relying on [46] and [20].

In particular, we evaluate the performance of the VSB runtime with the following experimental setup.We interconnect heterogeneous Things through a specific bus protocol and measure end-to-end re-sponse times and throughput, under both low traffic and stress conditions. At the same time, we mea-sure the introduced latency inside the BCs. Then, we substitute the bus protocol with other middlewareprotocols aiming to observe trade-offs in the resulting end-to-end performance. Our approach enablessetting a lightweight testing environment around the VSB BCs with practical hardware resources andmaking sure that BCs employ their maximum capacity. In particular, to evaluate the performance ca-pacity of VSB, we have to saturate each BC system to determine its maximum performance. To thisend, Things have to take the role of senders and receivers and need to send and receive high messagerate through BCs.


App i

App 2

App 1

(M1)

(M4)

MQTT

Subscriber

(M2) (M3)

Bottleneck ? Bottleneck ?

Bottleneck ?

Bottleneck ?

WebSocket Producers

BC 1:

WebSocket -> REST or CoAP or DPWS

BC 2:

REST or CoAP or DPWS -> MQTT

Figure 2.1: Components of the mock environment for the VSB runtime capacity testing.

2.1.1. Test Scenario

We set up our test environment with heterogeneous mock Things (senders and receivers) and we syn-thesize corresponding BCs. We utilize the supported middleware protocols of the VSB Framework(WebSockets, REST, CoAP, MQTT and DPWS). Our purpose is to remove any bottlenecks from theThings (senders/receivers) and create potential bottlenecks in the BCs for testing their maximum per-formance. Regarding performance, we measure throughput and one-way end-to-end response times.More specifically, we develop a sender (using WebSockets) and threads for creating many mock pro-ducer applications. Then, the synthesized BCs interconnect the producers with a single receiver andhandle the traffic sent through the bus protocol. As bus protocol, we leverage and test REST, CoAPand DPWS middleware protocols. In order to overload the BCs, our receiver must be able to receivethousands of messages per second. The CPU usage of the machines hosting the BCs should be closeto 100% to reach the maximum performance of the BCs. On the other hand, it is important that thesenders and the receiver are not highly loaded.

2.1.2. Test Setup

We used the following software and hardware for our experiments. The setup consisted of five ma-chines, connected via a local switch (GS900/8, Allied Telesis) creating a private 1000 Mb/s Ethernetlocal network. The first machine (M1) has as Intel Core i7-3520M CPU 2.90Ghz x 4 (8 GB RAM), thesecond (M2) an Intel Xeon(R) CPU W3540 2.93GHz x 4 (4 GB RAM), the third (M3) an Intel Corei7-4600U CPU 2.10GHz x 4 (8 GB), the forth (M4) an Intel(R) Xeon(R) CPU W3550 3.07GHz (8GBRAM), and the last machine (M5) has an Intel Core i7-4790 CPU 3.60GHz x 8 (8GB RAM). M5 is usedas monitor that collects information related to the exact arrival time of messages at each machine, forestimating the end-to-end response times and throughput. Running tests on powerful machines allowssimulating a large number of senders more accurately, as opposed to simple core machines.As depicted in Fig. 2.1, we provide three test scenarios:

1) Producers - BCs (REST) - Subscriber: Using the WebSockets middleware we create mock pro-ducers running on M1; BCs are synthesized employing the REST protocol running on M2 andM3; and finally we create a subscriber using the MQTT (with the “fire-and-forget” QoS mode [5])middleware running on M4.

2) Producers - BCs (CoAP) - Subscriber: This is the same scenario as the previous one with onlyone difference: BCs are synthesized employing the CoAP (with the “non-confirmable” QoSmode [43]) protocol running on M2 and M3;

3) Producers - BCs (DPWS) - Subscriber: This is also the same scenario with BCs employing theDPWS [50] protocol running on M2 and M3;

The first scenario leverages BCs performing a transformation from WebSockets to REST and then to


Scenario (busprotocol)

BC 1 Latency(ms)

BC 2 Latency(ms)

End-to-end Re-sponse Time (ms)

Throughput(msg/sec)

Avg.Senders

Scenario 1(REST)

0.1 0.1 2.44 299 300

Scenario 2(CoAP)

0.16 0.11 453 300 300

Scenario 3(DPWS)

0.16 0.05 1.6 299 300

Table 2.1: Results for one-way interaction in the three scenarios with 300 concurrent senders.

MQTT primitives by relying on GM. Then, in the second and third scenario we substitute the REST busprotocol with CoAP and DPWS, respectively. Thus, BCs perform a transformation from WebSocketsto CoAP/DPWS and from CoAP/DPWS to MQTT primitives. Note that, in all cases senders producemessages every second. In our measurements, we discard the first 4000 messages allowing machinesto reach a steady state. Things generally do not exchange very large quantities of data, so messagesusually tend to be of average size. Hence, we set the size of messages to 284 bytes (such messagepayloads are usually encountered in IoT applications). To evaluate the performance of the above sce-narios, we have implemented a monitoring system which is described in the next subsection.

2.1.3. VSB Monitoring System

The VSB Monitoring System (VSB-MS) is an open source framework designed to facilitate system de-signers to perform performance evaluation of middleware interactions. VSB-MS has been implementedusing Java 8 and it contains the following components.

Agent: the component responsible for the instrumentation of the system to be monitored by injectingprobes. Probes are positioned at the input/output of an operation and respectively collect input/outputtimestamps. It is developed using the Managed Bean technique in Java and it sends the information tothe Collector component by employing the RMI1 protocol.

Collector: the component responsible for collecting data received from the Agent component. Thesedata are stored in the memory of the java virtual machine. In a distributed system the Collector is ableto collect data from multiple Agent components.

Controller: is the central component that activates each Agent to publish data which are stored by theCollector. Then, it analyzes these data to compute end-to-end response times. The Controller is alsoresponsible for providing a local server time to each instance of the Agent for ensuring accurate timesynchronization. Finally, the Controller provides a console for printing the results.

To evaluate the end-to-end performance of the VSB runtime environment, we have created 4 Agentsfor each component of the above scenarios: i) the agent at the Producer ’s component collects thesending timestamp of each message; ii) one agent at each BC collects the input/output timestampsof each message; finally ii) the agent at the Subscriber ’s component collects input timestamp of eachmessage. Each agent pushes the timestamp information to the Collector. Finally, this information isavailable for the Controller to compute the end-to-end response time of each message.

2.1.4. Results

Table 2.1 presents some detailed data for a test run with 300 concurrent senders. To run a test, wecreate and send a sufficient number of messages by running each experiment for at least 2 hours. Ac-cordingly, for 300 concurrent senders we send approximately 2160000 messages and then we calculatethe average response times and throughput. In total we have performed 49 tests for REST and DPWS

1https://docs.oracle.com/javase/7/docs/platform/rmi/spec/rmiTOC.html


Senders (low traffic)20 40 60 80 100 120 140 160 180 200

mes

sage

s/se

c

0

50

100

150

200Bus protocol: CoAPBus protocol: RESTBus protocol: DPWS

Senders (high traffic)200 400 600 800 1000 1200 1400 1600 1800 2000 2200

mes

sage

s/se

c

200

400

600

800

1000

1200

1400

1600

1800

2000

2200

Bus protocol: CoAPBus protocol: RESTBus protocol: DPWS

Figure 2.2: Throughput for one-way interactions through REST, CoAP and DPWS bus protocolsin scenarios 1, 2 and 3.

and 25 for CoAP. Showing detailed results for each performed test it would be impractical, as it wouldrequire several pages; nevertheless, we plot these tests in the following figures. Based on the Table 2.1,the end-to-end response time is 2.44 ms when employing REST as the bus protocol, 453 ms for CoAPand 1.6 ms for DPWS. In case of CoAP, the throughput is 300 messages per second, which corre-sponds to the maximum limit of this protocol (based on its RFC2, it enables up to about 250 messagesper second from one endpoint to another with default protocol parameters). Thus, as the incoming loadof messages (more then 250 msg/sec) increases, this results to high end-to-end response time wherethe synthesized BCs have reached their maximum resources (CPU, Memory) since they employ CoAPas the bus protocol. It is worth noting that the latency inside the BCs is negligible (0.05 - 0.16 ms, forthis particular message size), with regard to the end-to-end response time.

Fig. 2.2 shows the measured throughput when sending one-way messages to the MQTT subscriber,in function of the number of concurrent senders for each of the above scenarios (employing differentbus protocols). The procedure we applied to execute this experiment is the following: i) in all cases,after BCs reach the steady state, the MQTT subscriber counts incoming messages for a duration oftime; and ii) we repeat the same experiment by increasing the WebSocket application senders. Thus,the MQTT subscriber receives an increasing number of messages according to the concurrent senders.We observed that the number of messages passing via BCs per second (throughput) for high input loadsdepend on the employed bus protocol. In scenario 1, (REST bus protocol), the maximum throughputis 1865 messages per second, in scenario 2 (CoAP bus protocol) is 324 messages per second and inscenario 3 (DPWS bus protocol) is 1801 messages per second. We verified at the same time that theCPU usage of the machine hosting BC 1 had reached its maximum, while the CPU for senders/receiverswas about 30% - 50%. To achieve this, we have deployed BCs to less powerful machines (M2, M3), incomparison to the ones that host the senders and the receiver (M1, M4). In this way we are able to:i) reach the maximum resources of machines hosting BCs; and ii) compare the three bus protocols withthe same criteria.

Fig. 2.3 shows the measured one-way response times when sending messages to the MQTT sub-2https://tools.ietf.org/html/rfc7252


Senders (low traffic)0 20 40 60 80 100 120 140 160 180 200

mes

sage

s/se

c

0

2

4

6

8

10

12


Senders (high traffic)200 400 600 800 1000 1200 1400 1600 1800 2000 2200

mes

sage

s/se

c

0

200

400

600

800

1000

1200

1400

1600

1800


200 400 600 800 1000 1200 1400 1600 18000

10

20

Figure 2.3: Response times for one-way interactions through REST, CoAP and DPWS bus pro-tocols in scenarios 1, 2 and 3.

scriber, in function of the number of concurrent senders for each of the above scenarios. The procedurewe applied to execute this experiment is the same as the previous one for the throughput. We observedthat for low traffic (up to 200 senders) DPWS presents the lowest end-to-end response times (∼1.6 ms),while REST presents quite low values (∼2.5 ms). Regarding CoAP, for low traffic it presents similarvalues in comparison to DPWS; nevertheless, after having 60 senders end-to-end response times itreaches quite high values. For high traffic, DPWS maintains its low response time values (∼1.6 ms)up to 400 senders; while afterwards it presents quite low values (∼10 ms) up to 1300 senders. DPWSis scalable enough, since it presents high values of response times and low values of throughput (seeFig. 2.2) after having 1700 senders. Regarding REST, it presents the lowest end-to-end responsetimes (∼4.5 ms) for high traffic and is much more scalable than both CoAP and DPWS protocols. It isworth noting that after having 1600 senders, REST maintains its end-to-end response times, howeverits throughput values become lower (see Fig. 2.2) due to a considerable number of losses. Hence,for heavy traffic load REST presents lower response times and higher message losses while DPWSpresents higher response times and lower message losses. For all the above cases we verify that thebottleneck is present at the machines hosting BCs.

2.2. Design Time Analysis of Choreography QoS

In this section we provide an approach to system designers that can be leveraged to evaluate end-to-end QoS in service/Thing choreographies. As a first step, we provide the description of the studiedchoreography, derived by the Urban Traffic Coordination (UTC) use case (see D4.2 [16]). Then, weidentify the various interactions and interaction types between services/Things and CHOReVOLUTIONartifacts that are implemented inside our choreography. Subsequently, we select particular end-to-end interactions to apply our QoS analysis (introduced in D3.2 [15]) by relying on simulation-basedexperiments.


��

��

��

��

��

��

��

��

��

��

��

��

Figure 2.4: SEADA Choreography Components

��

��

��

��

��

��

��

��

��

��

��

��

��!��

��

��

��

��

��

��

��

��

��

��

��

Figure 2.5: Interactions within the SEADA choreography

2.2.1. SEADA choreography

The Situation and Eco-Aware Driving Application (SEADA) choreography has been designed in theCHOReVOLUTION platform as part of the UTC use case. The SEADA choreography supports ex-isting external services, newly developed services, prosumer services and related coordination dele-gates as described in the deliverable D4.2 [16]. The following Fig. 2.4 illustrates the components ofSEADA choreography, which are able to coordinate all their activities. The navigation device (ND),a front-end application handles all the user interaction. The external provider services: DTS-Google,DTS-Here, DTS-WEATHER, DTS-ACCIDENTS, DTS-CONGESTION and DTS-Bridge associate theiractivities with prosumer services (SEADA-SEARP, SEADA-SEATSA and SEADA-TRAFFIC). Which canprovide real-time traffic and Eco-friendly route information to its end-users.

2.2.2. Interactions within the SEADA choreography

From the description of SEADA choreography [16], we attempted to identify interactions between theND, external provider services and the prosumer services. The identified interactions are describedbelow as depicted in Fig. 2.5.

An end-user uses the ND to trigger the choreography by selecting the origin and destination points.The ND sends a ReqEcoRoute request to SEADA-SEARP component to get the eco-routes. After-


service/Thing or artifact interaction type service/Thing or artifact

ND two-way-sync SEADA-SEARP

ND two-way-streaming SEADA-SEATSA

SEADA-SEARP two-way-sync DTS-Google, DTS-Here

SEADA-SEARP two-way-sync SEADA-SEATSA

SEADA-SEATSA two-way-sync SEADA-TRAFFIC

SEADA-TRAFFIC two-way-sync DTS-Accidents

SEADA-TRAFFIC two-way-sync DTS-Weather

SEADA-TRAFFIC two-way-sync DTS-Bridge

SEADA-TRAFFIC two-way-sync DTS-Congestion

Table 2.2: End-to-End Interactions of SEADA Choreography

wards, SEADA-SEARP receives multiple routes (RoutesResp) as a response from DTS-Google andDTS-Here for the request RoutesReq they received from SEADA-SEARP. Then, the SEADA-SEARPsends RespEcoRute response to the ND. Based on these request-response patterns we have iden-tified the interaction type between the ND & SEADA-SEARP and SEADA-SEARP & the digital maps(DTS-Google and DTS-Here) as two-way-synchronous.

When the SEADA-SEARP has multiple routes, a request ReqEcoRoute has been made to SEADA-SEATSA component that will analyze the eco-routes and sends back a response RespEcoRoute.Clearly, this is also a two-way-synchronous interaction.

The SEADA-SEATSA analyzes the eco-route by requesting ReqEcoFriendlyRoute the SEADA-Traffic component, which further sends a request ReqOtherRelatedInfo to the external services (DTS-ACCIDENTS, DTS-WEATHER, DTS-CONGESTION and DTS-BRIDGE). Once, the SEADA-TRAFFICgets a response from external services, It will pass (RespEcoFriendlyRouteInfo) this informationto the SEADA-SEATSA. The above interactions clearly shows the request-response is a two-way-synchronous.

Finally, when an end-user opts an Eco-route, the ND will receive EcoSpeedRouteInfo periodicallyfrom the SEADA-SEATSA component until the user reaches his destination. To do so, the SEADA-TRAFFIC periodically gathers information from external services and updates with SEADA-SEATSA.In conclusion, we can see the above interaction as two-way-streaming since, ND receives multipleresponses to a single request.

We include all the SEADA choreography interactions analyzed above in the below Table 2.2.

2.2.3. Case Study

In the previous subsection we identified the available interactions and their types (Table 2.2) withinthe SEADA choreography. These interactions are between the external services/Things and prosumerservices and thay are part of the overall end-to-end interactions between services and Things of theSEADA choreography. Our main purpose is leverage the QoS modeling patterns presented in D3.2 [15],for evaluating end-to-end QoS inside the SEADA choreography.

System Model

Based on the description of the SEADA choreography, once the requested path is provided to theND, it can receive additional notifications from DTS-Accidents, DTS-Bridge, DTS-Congestion and DTS-Weather services/Things. We demonstrate our QoS analysis by selecting and evaluating end-to-endinteractions between the above DTS-services and the navigation device. The resulting end-to-endpush-based notification system is composed by the following artifacts:

DTS-service→ BC 1→ SEADA-TRAFFIC→ SEADA-SEATSA→ BC 2→ Navigation Device


��

� ��

��

��

��

��

��

��

��

Figure 2.6: System Model

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

Figure 2.7: Queueing network for evaluating CS/DS one-way interactions.

Each DTS-service employs the REST protocol to push notifications. On the other hand, ND employsREST to receive notifications from DTS-Bridge, DTS-Congestion and DTS-Accidents; and CoAP forreceiving notifications from DTS-Weather. Coordination issues are tackled by SEADA-TRAFFIC andSEADA-SEATSA (see D4.2 [16]) which employ SOAP to send/receive notifications. Hence, BC 1 andBC 2 are synthesized for the conversion of primitives and data from REST/CoAP to SOAP and viceversa. As depicted in Fig. 2.6, to evaluate the above end-to-end interaction it is essential to considerthe following QoS parameters: i) the notification arrival rates (λ); ii) service demands of DTS-servicesand the Navigation Device; (DDTS , DND) iii) service demands for the conversion of primitives/data inBCs 1 & 2 (DBC1, DBC2); iv) service demands to coordinate notifications in SEADA-TRAFFIC andSEADA-SEATSA (DCD1, DCD2); v) ND’s disconnections (TOFF ) for energy saving purposes or dueto the wireless network coverage and connections (TON ) for receiving notifications; and vi) the validi-ty/availability of traffic-related notifications for a lifetime period.

To evaluate our end-to-end interactions we design a queueing model that incorporates the aboveparameters. The next subsection provides an overview for the composition of this model.

Queueing Model

Within the end-to-end interaction of Fig. 2.6, services/Things interact with BCs by employing the RESTor CoAP protocol and SEADA-TRAFFIC/SEADA-SEATSA using the SOAP protocol. In D3.2 [15], wehave introduced performance modeling patterns using queueing network models (QNMs) for both reli-


��

��

� ��

��

��

��

��

��

��

��

��

��

��

��

� ��

��

��

��

��

��

��

��

��

��

� ��

��

��

��

��

��

��

��

��

��

� ��

��

��

��

��

��

��

��

��

Figure 2.8: Queueing network for evaluating one-way interconnections in the SEADA choreog-raphy.

able and unreliable middleware protocols. Based on D3.2 [15], REST, CoAP or SOAP services/Thingsare classified to the Client/Server (CS) communication style and thus, they can be evaluated through thequeueing network of Fig. 2.8 (Pattern 1 in D3.2 [15]). In particular, REST and SOAP protocols rely onTCP’s delivery mechanisms and their performance can be evaluated using the reliable model of Fig. 2.8where the overall end-to-end connectivity follows TCP’s sessions. On the other hand, CoAP transmitsmessages over the unreliable UDP protocol. As already defined in D3.2 [15] the “non-confirmable” QoSfeature can be modeled using the unreliable pattern of Fig. 2.7.

Evaluating the performance using the pattern of Fig. 2.7, requires that both server and client employthe same protocol (REST, CoAP or SOAP). However, our end-to-end interactions (Fig. 2.6) containservices/Things and CHOReVOLUTION artifacts that employ REST, CoAP and SOAP protocols withboth reliable and unreliable interactions. In D3.2 [15], we introduced a methodology to compose per-formance models of heterogeneous services/Things at the middleware layer. In our end-to-end inter-actions, ND receives notifications from DTS-Bridge, DTS-Congestion and DTS-Accidents by employingthe REST protocol (reliable way) and from DTS-Weather by employing the CoAP protocol with the “non-confirmable” QoS level [43] (unreliable way). Hence, by following the methodology of D3.2 [15], we se-lect the corresponding pattern (reliable or unreliable) that matches the employed protocol of ND and theone of each DTS-service. Then, we connect these two patterns in sequence by using in the middle thecorresponding pattern of the bus protocol (SOAP in our case). As already defined, SEADA-TRAFFICand SEADA-SEATSA interact with each other and with the two BCs by employing SOAP. Hence, werepresent the application layer service demand of SEADA-TRAFFIC and SEADA-SEATSA by using ad-ditional queueing centers inside the CS performance pattern of the bus protocol. Based on the abovedescription, Fig. 2.8 depicts the end-to-end queueing network for each one of these interactions.

In the next subsections, we utilize the reliable end-to-end queueing networks of Fig. 2.8 to evaluatetheir end-to-end QoS in terms of response time and delivery success rate.

2.2.4. Experiments

In this subsection we select the end-to-end queueing pattern (see Fig. 2.8) that models the perfor-mance of notifications sent from DTS-Bridge to the navigation device. As defined in D3.2 [15], thismodel is reliable where losses occur only due to message expirations – i.e., there are no losses dueto disconnections. At the input of the pattern’s queueing network, notifications arrive with rate λinapp = 1notifications/sec (publishing rate). Each notification is valid for a deterministic lifetime period andthen discarded by the queueing network. We alternate between values of 10, 30 and 60 sec for thelifetime parameter.

We parameterize the queueing network as follows: as already defined, ND connects and disconnects


TON

(sec)10 20 30 40 50 60 70

Suc

cess

Rat

e

0

0.2

0.4

0.6

0.8

1Lifetime 10 T

OFF = 60

Lifetime 10 TOFF

= 30

Lifetime 10 TOFF

= 10

Lifetime 30 TOFF

= 60

Lifetime 30 TOFF

= 30

Lifetime 30 TOFF

= 10

Lifetime 60 TOFF

= 60

Lifetime 60 TOFF

= 30

Lifetime 60 TOFF

= 10

Figure 2.9: Success rates for the reliable end-to-end interconnection with varying connec-tion/disconnection and lifetime periods.

to receive notifications. We set the total average connected + disconnected period to be TON + TOFF

= 120 sec. Experiments are performed by varying the TON and TOFF periods inside the 120 sec inter-val. To process the produced notifications in ND and DTS-services, we apply a service rate of µpr =32 notifications/sec. We apply the same service rate to process notifications in SEADA-TRAFFIC andSEADA-SEATSA artifacts. It is worth noting that system designers are able to parameterize such ser-vice demands based on processing demands of real applications. For instance, based on the Table 2.1,BCs convert data and primitives between middleware protocols (REST, CoAP, DPWS) in an averagedelay of 0.13 ms. Hence, to parameterize the application-layer service demand in BCs, we apply a ser-vice rate of µpr = 7.7 notifications/ms. To transmit notifications between services/Things and artifacts,we apply a service rate of µtr = 16 messages/sec. The applied service demand can vary, depending onthe bandwidth of the connection between the software artifacts.

Delivery Success Rates

In order to evaluate the effect of varying lifetime and connection/disconnection periods on deliverysuccess rates, we perform simulations after applying the above parameters to the 1st reliable queueingnetwork of Fig. 2.8. Based on the navigation device’s connectivity behavior, the TON period variesfrom 10 to 60 sec, increased by 10 sec at each experiment. Furthermore, TOFF period varies betweenthe values of 10, 30 and 60. The rates of successful interactions are shown in Fig. 2.9 for variousvalues of lifetime, and TON/TOFF periods for the navigation device. Using MobileJINQS, we performaround 700000 interactions for each experiment. As expected, increasing TON periods for individuallifetime values improves the success rate. On the other hand, the success rate is severely boundedby lifetime periods, especially for lower values. Hence, increasing lifetime periods from 10 sec to 30 secis necessary to have a success rate of more than 60% for a connectivity of TON = 50 sec and TOFF =60 sec.

Response Time vs. Success Rate

In order to study the trade-off between end-to-end response times and delivery success rates, wepresent cumulative response time distributions in Fig. 2.10. In comparison to the previous set of exper-iments, we keep the same intervals for disconnections (TOFF), while connections (TON) occur for 10, 30and 60 sec. Fig. 2.10, shows response times for successful interactions (i.e., we plot only interactions


Figure 2.10: Cumulative distributions of response times for the reliable end-to-end interconnec-tion with varying connection/disconnection and lifetime periods.

having response times lower than the lifetime period). From Fig. 2.10, lower lifetime periods producemarkedly improved response time. For instance, with lifetime = 10 sec and equal TON/TOFF periods,90% of the interactions complete within 1.5 sec. Comparing this to Fig. 2.9, with lifetime 10 sec, TON =30 sec and TOFF = 30 sec, the probability of response time to be less than 10 sec is 1 while the successrate is 0.50. By increasing the lifetime to 30 sec, the probability of response time to be less than 10 secis 0.71 and the success rate is 0.7. Generally, these tradeoffs confirm that higher lifetimes give bettersuccess rates but with higher response times.

2.3. Run Time Analysis of Choreography QoS

In the previous section, we analyzed the QoS of a choreography at design time to ensure accurateruntime behavior. However, unpredicted changes may occur at runtime that affect the choreographyQoS. Therefore, the choreography components may have to reconfigure themselves to support thesechanges and to provide appropriate levels of performance. This cannot be done by relying on thedesign time configurations because of their static nature and uncertainty in the operating environmentof the running system [10]. Therefore, there is a crucial need to investigate runtime reconfigurations. Inother terms we need a dynamic adaptation mechanism also called as self-adaptation that can solve theproblem mentioned above.

This section provides the proposed MAPE-K [18] (Monitor, Analyze, Plan, Execute - Knowledge)based self-adaptation framework which relies on choreography QoS analysis at runtime. Currently, weare concentrated on the monitoring and the analysis components of MAPE-K loop. In 2.3.2 we providestate-of-the-art tools available for monitoring and compare them with our own monitoring tool (VSB-MS)as described in 2.1.3. Afterwards, we present the runtime choreography QoS analysis in 2.3.3.

2.3.1. Self-adaptation Framework

Our proposed self-adaptation framework relies on the MAPE-K loop [18], which has been widely usedfor adaptations, as shown in Fig. 2.11. In this, M stands for monitoring, A is analysis, P stands forplanning, E is execution, and finally the K stands for the knowledge obtained from the entire MAPEloop. The monitoring component (M) provides the data (a symptom) gathered from the running systemand its environment by using the sensors (or probes) and saves it in the knowledge component. Thenthe analysis component (A) performs the specified analysis operations on symptoms provided by the


��

��

� ��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

Figure 2.11: Self-Adaptation Framework

monitoring component; and checks whether an adaptation is required. If so, it triggers the planning com-ponent (P) that takes appropriate decision and composes adaptation actions to fulfill the required goalthat is providing the required level of QoS in our case. Finally, the execution component (E) performsthe adaptation actions planned in the planning component on the running system by using effectors (oractuators). The knowledge component (K) consists the data relating to monitoring (monitored systemand its environment), analysis (previous performance patterns), planning (reconfiguration decisions),and execution (execution plans). These M, A, P, and E components are decentralized and can commu-nicate either explicitly or implicitly by sharing the information through the knowledge component. In thisdeliverable, we concentrate on the first two components of MAPE-K which are the Monitoring in 2.3.2and the Analysis in 2.3.3.

2.3.2. Monitoring Systems

A choreography of heterogeneous services/Things results into complex middleware interactions (e.g.,end-to-end interactions using both reliable and unreliable middleware protocols). At runtime, these in-teractions may lead to unexpected behavior and performance degradation. To detect such QoS issues,it is essential to monitor the choreography (or system) at runtime. To monitor a system, as a first step, itis essential to instrument it. Instrumentation, is either a manual or an automatic process, where probesshould be positioned at the input/output of an operation and respectively collect input/output informa-tion of interest (eg: timestamps). Manual instrumentation requires human intervention in order to injectprobes in the source code of the target system at the design time, while automatic instrumentation doesnot require human intervention and probes injection is done automatically at runtime by the MonitoringSystem.

In the following paragraphs we provide a brief survey of four existing monitoring systems. Our mainpurpose is to compare them with our monitoring system (VSB-MS, described in Section 2.1).


Kieker

The Kieker3 system is an open source framework designed for continuous monitoring of a softwaresystem. It provides an extensible architecture that supports both centralized and distributed systems.Kieker is a java-based framework designed using aspect-oriented programming to support non-intrusiveinstrumentation, composed by the following components:

– Monitoring: is the component that collects data from probes and push them for recording.

– Monitoring Analysis: is the component responsible for data provision and processing. It is alsoa control panel for the kieker framework and comes along with multiple plugins for visualizationusing different representations such as graph, Markov chains and 3D.

– Monitoring Log: is the component responsible for data recording. Recording with Kieker can beapplied to any persistent storage repository such as a file system, database, or a JMS queue.

– Monitoring Controller : is the component that manages the activation of probes and the commu-nication between the Monitoring and the Monitoring Log components. In addition, it providesinformation about the Monitoring Log to the Monitoring Analysis component. In this way the lattercan analyze and process the recorded data.

Kieker framework supports manual instrumentation so far, it can be classified as a tool based on manualinstrumentation. However, some work is still underway to enable automatic instrumentation.

Compas Monitoring Framework

Compas [38] is an extensible framework designed for Java Enterprise Application and based on de-coupled communication mechanisms for providing a common monitoring system across distributedapplications. To minimize the overhead of the monitoring infrastructure, Compas uses asynchronouscommunication and enables non-intrusive instrumentation on the system to be monitored. The probecomponent in Compas is implemented as a proxy layer surrounding the system to be monitored tointercept all the method invocations and life cycle events. There is no need to instrument the systemwith manual probe injection since Compas uses metadata to derive the internal structure of the targetsystem. The metadata is placed in deployment descriptors that contain structural as well as behavioralinformation about the systeme to be monitored.

The information needed for probe insertion, is obtained from the metadata thanks to the non-intrusiveinstrumentation.

Performance metrics and life cycle are collected by probes and sent to the Monitoring Dispatcher.The dispatcher component is the client-side that has the central role for data processing and mediat-ing the client acces to the Compas probes. To ensure the extensibility of its architecture, extensionsare built on the Framework Extension Points (FEP) component based on a decoupled asynchronouscommunication. For this purpose Compas provide two types of framework extensibility :

– server-side FEP: it facilitates creation of new probes. In addition it facilitates the extension of thefunctionality that is provided by the probe component.

– client-side FEP: it facilitates the extension of the functionality available by the dispatcher compo-nent.

Finally, Compas supports automatic instrumentation and it can be classified as a tool for dynamicanalysis and discovery. On the other hand, it supports only Java-based applications.

3http://eprints.uni-kiel.de/14459/1/vanhoorn tr0921.pdf


Esper

Esper4 is an open source framework dedicated to Complex Event Processing. Complex event pro-cessing, is event processing that combines data from multiple sources to infer events or patterns thatsuggest more complicated circumstances. The goal of complex event processing is to identify mean-ingful events (such as opportunities or threats) and respond to them as quickly as possible.

Esper can be used to instrument a system based on the Complex Event Processing approach. Esperenables a manual insertion of probes (manual instrumentation) in a standalone Java application, anapplication server or a Tomcat web container for monitoring instrumentation.

InfraRED

InfraRED5 is a tool for monitoring performance of Java Enterprise applications and diagnosing perfor-mance problems. It contains three main components: the Agent, the Collector and the Graphical UserInterface. The Agent is embedded in a JVM application server and instrumentation is achieved manuallyusing aspect-oriented programming. The Agent collects metrics regarding aspects of an application’sperformance and sends notifications over the network to the Collector. The latter aggregates these dataand stores them in a database. The Graphical User Interface queries the Collector for displaying thecollected information to the user.

Comparison

In the previous subsections we presented: i) tools that are designed based on automatic architecturediscovery in order to support automatic instrumentation; and ii) other tools designed based on manualinstrumentation. In addition, we noticed that these different tools support different protocols for sendingthe collected data (by injected probes) over the network to the component responsible for data process-ing or analysis. Table 2.3 provides a summary of these tools and the VSB Monitoring System (VSB-MS)which we presented in subsection 2.1.3.

Kieker Compas Esper InfraRED VSB-MS

Category 2 1 2 2 2

Extensibility yes yes no no no

Scalability yes yes no tested yes yes

Instrumentation manual automatic manual manual manual

Technology Java JEE, .Net JEE, .Net Java Java

Protocol RMI, REST, HTTP, SOAP, MQTT RMI, REST, HTTP, SOAP, MQTT REST,MQTT not defined HTTP

Data persistence yes yes yes yes no

Table 2.3: Monitoring Systems

Based on the above comparison and the requirements of our runtime adaptation approach, our futureworks includes the experimentation with each one of these tools and the selection of the appropriateone or the extension of the VSB-MS.

2.3.3. Analysis

As depicted in Fig. 2.11, the analysis component analyzes the symptoms received from the monitoringcomponent by performing complex data analysis and reasoning. The task of the analysis component isto alert the planning component if there are any fluctuations in the system performance and thus, the

4http://www.espertech.com/esper5http://infrared.sourceforge.net


planning component can takes appropriate adaptation decisions. Moreover, this analysis componentcan be influenced by previously stored data in the knowledge component. As explained in the Fig. 2.11,the analysis should be done on the gathered data from monitoring component. Since, our monitoringsystem is not yet completely ready, we are using predefined values. We will use QN models to performour choreography QoS analysis at runtime, similarly to our design time analysis.

There are three well-known ways to analyze system performance based on QN models [25]: analyt-ical, numerical and simulations. At runtime the available time for model analysis is limited and crucial,therefore, we need faster and closed-form solutions. Based on, the computational complexity and ac-curacy levels, a broad set of techniques can solve QN models as explained in [7]. From [7], analyticalsolutions based on Markov chains can provide faster and accurate results; however, it is also clear thatthey often suffer from a state-space explosion problem at runtime[44, 49]. Moreover, for hybrid systems(continuous behavior and discrete aspects) [27], it is hard to find analytical solutions. Generally, analyt-ical solutions are learned at design and testing phases of a system and applied at runtime. However,these models are rather limited in terms of configurations to homogeneous servers and static structures[44]. To perform the analysis we can use discrete-simulations, which however take longer time to pro-vide the performance fluctuations and has higher computational complexity [44]. Moreover, interactivesimulations require performance trace files for post-processing, which is not possible due to the lackof observability on the performance data during simulations [34]. Therefore, we can not use them atruntime for making adaptation decisions. However, simulations can be useful to predict the systemsperformance in near future so that the performance fluctuations can be anticipated.

Regarding numerical analysis, there are three approaches based on QN models: continuous-timeMarkov chain (CTMCs) models, fluid approximation and diffusion approximation. From [4, 23, 40, 39],Markov models are vital for performance evaluation; however, the sojourn time is exponentially dis-tributed at each state and the number of states increase rapidly with the complexity. To describe thequeue-length probability distribution, diffusion approximation [7, 48, 8], uses a second order partialdifferential equation (i.e. diffusion equation) that defines the position of a particle in diffusion motion.This approach need much less computation cost when compared with the Markov models. Since, thisapproach merges the states of QN model. However, this approach is semi-numerical, does not sup-port efficient runtime performance evaluation. A simplified version of this diffusion approximation is thefluid-flow approximation [26, 3, 2, 29, 35], which considers the mean values of queue-lengths, servicetimes, etc. First order differential equations used to represent the fluid-approximations are simpler thanthe diffusion equations and in a reasonable time the computation can be done. Therefore, to performchoreography QoS analysis we rely on queuing network models, in which a system can be representedas a set of differential equations. We provide in the following more information about numerical analysisto support our needs to perform runtime choreography QoS analysis.

Numerical Analysis

In the above, we discussed several performance evaluation methodologies. That concludes the numeri-cal analysis is well suited to perform runtime QoS analysis. In this section we provide further discussionon numerical methods particularly with regard to i) steady-state analysis and ii) transient analysis. Inboth the methods CTMCs are used to describe the running system. Then ODEs generated from theCTMCs represent a system for performance evaluation.

When the system time approaches to infinity the system provide the steady-state performance. Thesteady-state analysis provide the performance behavior of the system when it reach it’s steady-state.This kind of behavior depends on both the quantity of inputs and the dynamic nature of the system.From literature [36, 31, 45], it is clear that the majority of work on queuing models designed to analyzesteady-state, such as, long-run system utilization, throughputs and average response times. Our designtime choreography QoS analysis is also performed by relying on steady-state measures as describedin the section 2.2. However, the steady-state analysis have limitations for providing quality and queuingmodels applicability, as it cannot capture real traffic which always varies [19]. Moreover, the steady-state


Figure 2.12: Queueing Network Model for Client-Server Interactions

analysis does not describe the self-similarity behavior where the system loses its Poisson probabilityand influences the queue-length [33]. In [1], author’s claimed that the steady-state measures can beinsufficient to describe the behavior of a single queue. The paper [37], reveals that steady-state anal-ysis is not sufficient for characterizing queues which are not Poisson. Moreover, the modern softwaresystems never reach a steady-state frequently due to their complex, distributed and dynamic nature [1].In [30], Krishna kumar et. al. explained the inefficiency of steady-state analysis for providing service tothe random catastrophic failures or the impatience of customers. Furthermore, the authors mentionedthe steady-state measures produce poor data to predict the performance of a system with load fluc-tuations. Based on these load fluctuations the software components have to reconfigure dynamically.Therefore, to solve the problems mentioned above with steady-state analysis on queuing models thetransient analysis can be a possible solution.

The transient analysis provides a systems temporary performance behavior. Analyzing the systemperformance at unsteady state provide the transient analysis which is highly dependent on time. More-over, transient analysis results can be used to investigate the finite-time properties which is helpful tounderstand the systems intermediate behavior before it reaches the steady-state. Transient behaviorreacts not only for immediate events but also any event that influences the steady-state behavior. Thetransient behavior is initiated from zero to a certain period of time until it reaches the steady-state.For analyzing a complete system behavior we need both the steady-state behavior and the transientbehavior. Transient analysis can be used to understand a system that operates in an uncertain envi-ronment [41]. By relying on fluid approximations, which will be represented by a system of ODEs thetransient analysis explores the performance bottlenecks [11]. We can see more interesting works whichuses transient performance analysis for instance WinPEPSY-QNS, a performance evaluation and pre-diction tool based on queuing networks [6]. The authors in [47], describe the usage of transient analysisfor performance evaluation through fluid approximations by solving the state-space explosion problem.Therefore, we argue to use the transient analysis and fluid approximations to perform runtime chore-ography QoS analysis. In which, end-to-end response time and queue-lengths will be evaluated aschoreography QoS parameters.

Transient Analysis Formulation for M/M/1 Queues

To perform the transient analysis we use simple M/M/1 queues, which represent a single server system.The M/M/1 queue is the most elementary model which describes many interested metrics as closed-form expressions can be derived. Therefore, the following are a set of ODEs to perform transientanalysis on simple M/M/1 queues. The queue length qj at a station j can be calculated by:

dqj(t)

dt=

∞∑(n=1)

(n− 1)pn =

∞∑(n=1)

npn −∞∑

(n=1)

pn = Lsρ

Where, n is the number of jobs in the system means the queue length is n-1, Ls is the averagenumber of jobs in the system i.e. ρ

1−ρ .In [42], authors provide the detailed information how to formulate the ODEs for single queuing sys-

tems i.e. M/M/1 queues. The following represent a set of differential equations describe M/M/1 queuesin terms of the evolution in time and state probabilities.


dPn(t)

dt= −(λ+ µ)Pn + λP(n−1) + µP(n+1)

dP0(t)

dt= −λP0(t) + µP1(t)

The above are the set of differential equations for M/M/1 queues describes the evolution of the stateprobabilities as a function of time. This is useful for transient analysis of queuing system for a givenperiod of time. Furthermore, these differential equations also provide steady-state from transient statewhen the queuing system has been running for a long time. Meaning that the probabilities of jobs inthe system will be constant. So that the derivatives for constant functions is equal to zero. This can berepresented as dPn(t)

dt = 0. Therefore, the above differential equations can be written as follows sincethe probabilities do not change with the time where n = 1.

0 = −(λ+ µ)Pn(t) + λP(n−1)(t) + µP(n+1)(t)

0 = −λP0 + µP1

Hence, these equations are no more differential equations, which represents a system of linear equa-tions. Based on the above set of algebraic equations we can calculate steady-state for any queuingsystem.

To perform the transient analysis with the above mentioned differential equations we used the queue-ing network modeling language (QNML) tool which is developed by [28], and then adapted it to supportour needs. The QNML tool takes the queuing network model with associated message arrival rates,nodes service rates and routing probabilities as inputs as shown in the Fig. 2.12. Internally the queuingnetwork model converted into an equivalent CTMCs. In the sense Kurtz [32], the fluid analysis on thequeuing network model gives a system of ODEs [26]. In this, one equation associates to each queue inthe model. The solution to this equation gives an estimate to the average queue length as a function oftime. Finally, we also performed simulations based on Gillespie algorithm. Gillespie et al. proposed astochastic simulation algorithm in [22], which has been implemented alongside with the fluid approxima-tion in the QNML tool. The Gillespie algorithm generates a possible solution for a stochastic equation,we can find more about Gillespie algorithm in [21]. In the experiments, we performed simulations alon-side with the numerical approximations to compare the achieved results whether they are coincide ornot. Based on these comparisons we can investigate: uncertainty in the assumed coefficients/param-eters, missing information for experimental settings, wrong assumptions, etc. Therefore, if a flaw isdetected we can refine or improve the model solution. Afterwards, the generated ODEs are analyzedby using Octave tool [24], a powerful mathematics-oriented scientific programming language. Throughthe plotting and visualization tools of Octave generates the graphs for end-to-end response times andqueue-lengths. Based on the achieved results we must analyze/decide, if there is any performanceviolation for the present situation. In practice, we assume that the monitoring tool will send monitoredvalues (job arrival rates, service rates and routing probabilities) to the analysis component, where theQN model within the QNML tool needs to be updated automatically with the newly received values.Therefore, the QNML tool analyze and produce the queue-legths and end-to-end response times. Thenthe reasoning mechanism needs to be implemented to check whether the end-to-end response timeand queue-lengths are in their appropriate levels or not. If the reasoning mechanism detects any fluc-tuation an alert can be send to the planning component. Therefore, the planning component can makean appropriate reconfiguration to reach the desired level of QoS. However, the QNML tool does notsupport more advanced queue structures i.e. ON/OFF queues for example.

2.3.4. Experiments

In this subsection, we used a modified end-to-end QN model (Fig. 2.12) taken from design time CS/DSpattern (see Fig. 2.7), for the chosen choreography interaction (DTS-Bridge to ND) as described in the


0 50 100 150 2000

50

100

150

200

Queue length

Server Ode(t)

Ms Ode(t)

Mc Ode(t)

Server Sim(t)

Ms Sim(t)

Mc Sim(t)

Figure 2.13: Queue Length based on ODEs and Simulations

section 2.2.3. Where, the DTS-Bridge serves as a server while the navigation device as a client. In theQN model we used four queues to represent four nodes namely: a server (S), a server-side middleware(Ms), a client-side middleware (Mc) and a client (C); where client and server queues are representedwith the clipart to distinguish with other queue types. These four queues are connected by using threerouters R1, R2 and R3 as shown in Fig. 2.12. The QN model with its associated service rates, routingprobabilities, job arrival rates are given as inputs to the QNML tool. Subsequently, the QNML toolfollows series of steps as described in the previous section and provides the results: queue-lengthsand end-to-end response times. Finally, we present both the numerical approximations and Gillespiealgorithm based simulation results; where the legends with Ode(t) represents the results based onODEs and Sim(t) represents simulation results with respect to time. We analyzed the performancetransient behavior: when the system is steady λ/µ < 1 and when the system is saturated λ/µ > 1.Here, we say the queuing model is stable if λ < µ, meaning that the average arrivals are slower thanthe service completions. Where ρ can be used to represent the buffer utilization.

Experiment 1

This experiment is performed based on the system saturated state meaning that ρ > 1. Achieved queue-length results are represented in the Fig. 2.13. The X-axis represents the time window of 200*stepsizeseconds (i.e. 200*0.01 = 2 seconds) and the queue-length on Y-axis. Afterwards, the numerical approx-imations and simulations shows their saturation behavior: client-side middleware (queue Mc) reachesits saturation state much earlier than the middleware queue (Ms) at server-side.

End-to-end response times are represented in the Fig. 2.14 which are based on both the numericalapproximations and simulations. Time window of 200 P-slots are represented on X-axis while end-to-end response time on Y-axis. Both the numerical and simulation results shows the saturation behavior.Therefore, we can use these numerical methods for runtime QoS analysis while the simulations can beused for predictions.

Experiment 2

This experiment is performed based on the condition ρ < 1 and the achieved results are shown in Fig.2.15. The X-axis represents the time window of 200 P-slots and queue-length in Y-axis. From the graphit is clear that both the numerical approximations and simulations shows the same behavior until thequeue-length is 1; then the numerical approximation reaches its steady-state while simulations showthe stationary behavior.


Figure 2.14: Response Time based on ODEs and Simulations

0 50 100 150 2000

0.2

0.4

0.6

0.8

1

1.2

1.4

Queue length

Server Ode(t)

Ms Ode(t)

Mc Ode(t)

Server Sim(t)

Ms Sim(t)

Mc Sim(t)

Figure 2.15: Queue Length based on ODEs and Simulations

The end-to-end response times of the experiment ρ < 1 is shown in Fig. 2.16. Where the time windowof 200 P-slots shown on X-axis and end-to-end response times on Y-axis. We compare both the nu-merical approximations and simulations. Clearly, numerical approximations and simulations present thesame behavior in their transient phase. Aftwrwards, the numerical approximations reach their steady-state while the simulations represent the stationary behavior. Therefore, these results can be utializedfor possible reconfigurations to improve the QoS.


Figure 2.16: Response Time based on ODEs and Simulations


3 CHOReVOLUTION Security

In D3.1 [13] and D3.2 [15], we discussed how different security aspects must be taken into accountto secure choreographies and their deployments. The main architecture features were described.. Inthe following sections, we present new architecture elements and performance results of the differentsecurity elements.

The Security Filter component undertakes the application of security interoperability mechanisms byacting as a proxy of the service/Thing provider in its interaction with the service/Thing consumer (Sec-tion 3.1). In this section we focus on the last evolution of this component driven by the use cases. Toopen new perspectives in term of authentication mechanisms supported for protection and adaptation,new features are implemented to offer the possibility for the choreography designer to customize theprotection and adaptation phase by providing plugin elements which implement the specific securitymechanisms(It’s not possible to implement all the possible web authentication method in the CHOReV-OLUTION project). A specific section describes the impacts of the last reflections conducted in thecontext of WP5 (Section 3.2).

At the end, new performance tests are described for Security filter and Identity Manager. For Securityfilter, this section describes different tests validating the optimizations performed on their implementa-tions. For Identity Manager, this section describes performance/stress test scenarios and their results.

3.1. Security Filter evolution

The security Filter is the element enforcing the security inside the choreography. It is closely linked tothe Federation Server for the Authentication part and the Identity Manager for his management. Thiscomponent has undergone some modification in order to guarantee the best performance but alsothe addition of new functionalities allowing the designer to be able to customize its use in terms ofauthentication methods.

3.1.1. Security Filter functionalities

The Security filter offers different functionalities like:

• The Authentication Filter, in charge of validating the credentials provided by the service consumerand preparing the credentials required to call the service provider

• The Authorization Filter, in charge of authorization or not the access depending of the securitypolicy defined during th synthesis phase.

In the previous version, this component offers two working modes: the protection or the adaptationbased on the login/password authentication forcing the end user to be register into the IdM before usingthe choreography.

• Protection: when the Security Filter is configured in ”Protection Mode”, it checks for the authen-tication credentials provided by the user, checks for the authorizations and, if all is OK, forwardsthe request with the same user credentials received.


• Adaptation: when the Security Filter is configured in ”Adaptation Mode”, it checks for the authen-tication credentials provided by the user, checks for the authorizations and, if all is OK, find thecredentials expected by the service provider following the specification of this service. These cre-dentials could be about an account that the user has on the service attempting to access, or abouta specific given account that the choreography itself has on the service.

The mode applied to the SF is dependent on its position into the choreography. In general, theprotection mode is applied to the Security Filter present at the entry point to the choreography and theadaptation model is more for the Security Filter in front of the existing services.

These mechanisms are useful for a lot of cases but could be not adapted for others. An approachcould be to offer other kinds of authentication mechanism (for protection and adaptation) based onexisting one like the Google of Facebook authentication. With that, we don’t cover enough the au-thentication possibilities and what about custom authentication? Consequently, the Security Filter wasupdated in order to provide plugin mechanism offering to the choreography designer (supported bya security expert), the possibility to implement its own security mechanism (for protection or adapta-tion) by just providing a zip archive during the synthesis phase containing the security implementation(based on a java application). Consequently, two new working modes were implemented to offer thesenew functionalities.

• Custom Protection mode: When the security filter is configured in ”Custom Protection Mode”, thisSF loads the plugin containing the protection authentication mechanism provided by the choreog-raphy designer and calls the method. Following the result provided by the plugin, the authentica-tion is validated or not.

• Custom Adaptation mode: When the security filter is configured in ”Custom Adaptation Mode”,this SF loads the plugin containing the adaptation authentication mechanism provided by thechoreography designer and calls the method. The result of the request is the request forwardedto the legacy service containing the security element required for the authentication set by theplugin.

Using this plugin approach, new authentication mechanisms could be developed (standard or custom)and shared between choreographies which could increase quickly the authentication possibilities.

The technology used to manage plugin is the service mechanism provided by the JDK. It imposes tothe service provider to implement a specific interface and to specify into the META-INF/serices repos-itory of the jar, which interface is implemented. The whole being packed in a zip file containing theimplementation, all the dependencies and possibly, some configuration files.

In the case of the cusom protection, the interface eu.chorevolution.securityfilter.api.ProtectionAuthNInterfacemust be implemented

Listing 3.1: Security filter configuration filepublic boolean checkProtectionAuthN(ServletRequest request) {

boolean result = false;/: or truereturn result;

}

And for the adaptation, the following interface, eu.chorevolution.securityfilter.api.AdaptationAuthNInterface,must be implemented

Listing 3.2: Security filter configuration filepublic SFServletRequestWrapper checkAdaptationAuthN(SFServletRequestWrapper request) {

// TODO Auto-generated method stubSFServletRequestWrapper ReturnRequest = null;return ReturnRequest;

}


3.1.2. Security Filter architecture

This architecture is an update version to support new functionalities like the custom authentication. It iscomposed of different components (see Figure 3.1):

• The Authentication Filter (AF),

• The Policy Decision Point (PDP),

• The Policy Enforcement Point (PEP),

• The Context Manager,

• The Configuration Manager,

• The Plugin Manager

Figure 3.1: Security Filter - architecture

The Context Manager

The Security Filter could modify its behavior based on the execution context of the choreography. Thiscontext is coming from the Identity Manager by an administrator which is aware of the choreographystatus and could decide or not to change the security constraints applied on this choreography.

The Configuration Manager

The Security Filter is a generic element whose the behavior is customized by the configuration filegenerated during the synthesis process of the choreography.

The Authentication Filter

The role of the Authentication Filter is to :


• Validate the credentials provided by the user that wants to access the protected resource.

• Retrieve the credentials required by the protected resource.

• Extract user group information used by the authorization part.

This part is also responsible for the adaptation of the security element depending of the configurationand the requirement of the choreography. For more information regarding the authentication adaptation,please refer to the D3.2 [15]

The PEP/PDP

The authorization part is realized directly into the security filter component and based on the two follow-ing sub-components

• The PEP protects the resource from unauthorized accesses.

• The PDP provides authorization decisions by evaluation of policies based on the security usergroup created by the IdM.

The audit mechanism

All the elements of the security filter (the authentication filter and the Policy Enforcement Point) mustlog access information, access evaluation response and the reason) and send this information to theFederation Server using a log mechanism compliant with syslog.

The Plugin Manager

This element is a new one in charge of managing the plugins provided by the choreography designerand to wrap the call to them. Following the configuration created during the synthesis, the pluginmanager calls the right one and manages the output by enforcing the authentication result return by theplugin or by propagating the adaptation provided.

3.2. Security applied to the WP5 use case

This part illustrates the different security aspects previously described and applied to the WP5 scenario.For more information about this use case, refer to the deliverable D5.2 [17].

Firstly, since the previous version of the use case, some reflections were conducted around the userexperience regarding the security applied to the use case. The fact that a user wanting use the touristapplication must be register into the IdM before could slow down its use. In addition, in this use case,the protection required is more a protection against DOS attacks than an need of user authentication.For that, different possibilities are available but only two tracks will be considered:

• The usage of a third part for the authentication of the user. In general, people have Google orFacebook accounts. It’s possible using the plugin mechanism to implement one of them in orderto identify the end-user using a third part. Note that with this method, the choreography knowsthat the user is known by a third part (as Google, for instance) and doesn’t know who is this enduser.

• Another possibility is to use the Google reCAPTCHA mechanism in order to validate that only”humans” could access the choreography.


For the moment, the choice has not yet been made but in both cases, a plugin implementation aspresented previously must be realized implementing the mechanism chosen.

Secondly, it’s appear that some legacy systems connected to the choreography-based system pre-sented in this use case use an authentication based on a Time-Variant Session Token. This token iscreated with an algorithm mixing the current time and a secret shared between the client and the server.This token is used to authenticate the client using the shared secret and prevent DOS attack using thetime notion. In order to contact the different legacy services using this mechanism, a custom adaptationplugin will be developed implementing the algorithm and putting the token generated into the context ofthe request sent to the legacy service. This plugin will be configurable in order to configure the pass-word that generates the token, and position in front of the corresponding service during the synthesisphase.

3.3. Improving Performance & Scalability

This section present the performance results of the different security elements. In the last D3.2 [15],some results were also presented consequently, this section only focus on new result elements.

3.3.1. Performance of the Security Filter

This part presents the performances results of the Security Filter. Since the last performance testspresented in D3.2 [15], no evolution is realized into the Federation Server but many investigations wererealized in order to improve the Security Filter performance.

Test scenario

The investigation to improve Security performances is around:

• the source code himself in order to streamline the object creations and the interactions betweenthe different components.

• the configuration of webapp container (the tomcat server and the JVM). In the previous document,3 different configurations was tested.

• -Xms1024m -Xmx1024m -server

• -Xms1024m -Xmx1024m -XX:+UseParallelGC -server

• -Xms2048m -Xmx2048m -server

Now, two more configurations are added.

• -Xms1024m -Xmx1024m -XX:+UseConcMarkSweepGC -server

• -Xms2048m -Xmx2048m -server -XX:+AggressiveHeap -XX:+AggressiveOpts

In addition, two new steps are realized based on the new plugin features implemented in this lastversion of the SF (named step 5 for the custom protection and step 6 for the custom adaptation. Theother steps are described into the D3.2 [15]). The steps 5 and 6 are closed to the step 3 in term ofinteractions with the other CHOReVOLUTION components (same workflow and interactions with theother components). The performance of these steps is very dependent of the implementation and theinteraction realized. For instance, if the custom protection plugin is based on a Google authentication,interactions with the Google server must be realized and could take time. Consequently, to understandthe minimal impact of this approach, empty plugins are used to realize the tests.

The result are presented in figure 3.2


Figure 3.2: Security Performance Summary

Conclusions

To summarize, the impact of the different servers and JVM configurations are negligible and the bestconfiguration is ”-Xms2048m -Xmx2048m -server”. Reading the impact of the improvement realizedinto the implementation of the security filter, a big gap was realized before the first performance testspresented in the D3.2 [15] (with a performance gain of about 60 %) but between the previous perfor-mance tests and now, only a gain of 10 % is realized. Again, the security is an overhead and thisoverhead is necessary to guarantee the security level for a system.

3.3.2. Performance of the Identity Manager

CHOReVOLUTION Identity Manager permits different entities life-cycle management. In particular, it isin charge of the services, choreographies, users and user groups life-cycle management. However, themost expensive operations are all that about the user management.

The number of user accounts is usually greater than the number of all the other managed entities.Moreover, generally a user profile concerns a lot of attributes and this can heavely affect the perfor-mance of the user profile management operations.

Furthermore, for the nature of CHOReVOLUTION Identity Manager, a relevant concurrency in man-aging user profiles is expected. Conversely, although it is well supported, we don’t expect frequentconcurrent operations in managing services and choreographies.

For these reasons we focused out performance/stress tests on the user profile management oper-ations which involve the most expensive activities and for which it is expected a relevant concurrencydegree.

All the tests are executed off-line (no choreography was running during test execution), on a pre-loaded Identity Manager instance. Pre-loading times have been collected as well and reported below.Considering that user profiles will be stored just for choreographies requiring user identification, weestimate that CHOReVOLUTION target applications will be based on a set of thousands registeredusers: a choreographed service with 10,000 registered users could be considered flat; one with 50,000


could be considered a growing service; from 100,000 onwards we can speak of a consolidated service.On these thresholds performance/stress tests have been executed.

Test scenario

First of all we pre-loaded the system with 9,000 user profiles by running 30 threads executing 300different creates per thread. Each user profile concerns 20 specific attributes more or less. On thispreliminar set of users, we concurrently executed 10 heterogeneous operations a time for 5 minutes.

Afterwards, we increased our set of pre-loaded users from 9,000 to 45,000 (same number of profileattributes). On this second set of users, we concurrently executed 10 heterogeneous operations a timefor 5 minutes.

On the same set, we concurrently executed 10 heterogeneous operations a time for 40 minutes inorder to be sure that this kind of load does not affect the performance of the system by extending thestress period.

On the same set, we concurrently executed 80 heterogeneous operations a time for 10 minutes inorder to be sure that the system is able to work fine in case of more concurrent operations.

Finally we increased our set of pre-loaded users from 45,000 to 105,000 (same number of profileattributes). On this third set of users, we concurrently executed 10 heterogeneous operations a time for20 minutes.

Test results are reported below.

Test setup

Test environment is configured as reported below.

• Docker container for the Identity Manager environment

• Identity Manager deployed on Apache Tomcat

• Java 8

• JVM arguments for Tomcat JVM are

• -server

• -Xms1536m -Xmx1536m

• -XX:NewSize=256m -XX:MaxNewSize=256m -XX:+DisableExplicitGC

• The internal storage for the Identity Manager is PostgreSQL 9.6

• Identity Manager connection pool configured as following

• Maximum active connection number 100

• Minimum number of connection in idle 10

• Host machine

• Ubuntu 16.04 LTS

• CPU Intel Core i7-7500U CPU @ 2.70GHz 4

• 64-bit

• RAM 16 GB

Tests are executed through Apache JMeter 3.2 1.1http://jmeter.apache.org/


Results

All the results about the scenarios discussed above are reported below.Pre-loading of 9,000 user profiles:

• 30 threads

• 300 create operations per thread

• 13 minutes to complete (11.5 users per second)

10 concurrent heterogeneous operations for 5 minutesOperation Average in msCreate 759Search 1183Read 11Update 430Delete 95

Pre-loading up to 45,000 user profiles:

• 30 threads

• 1200 create operations per thread






Pre-loading up to 105,000 user profiles:

• 30 threads

• 2,000 create operations per thread



As we can immediately consider, by increasing the number of existing users, the search operationsare affected in terms of performance: from 50,000 users onwards the completion time of a searchoperation (executed concurrently with other operations) could not be acceptable anymore. Fortunately,CHOReVOLUTION Identity Manager can benefit of an improvement implemented on Apache Syncope


2.0.X. In fact, it is possible to configure Elasticsearch2 as search back-end. This extension provides analternate internal search engine for Users, Groups and Any Objects, requiring an external Elasticsearchcluster. More specifically, it is a standalone database server, written in Java, that stores data in asophisticated format optimized for language based searches.

As search operations are central for different aspects of the provisioning process, the globalperformances are expected to be improved when using this extension. In Fact, with this configurationthe results for 10 concurrent operations executed for 20 minutes on a set of 100,000 user profiles arethe following.

10 concurrent heterogeneous operations for 20 minutes (Elastic)Operation Average in msCreate 1323Search 41Read 33Update 891Delete 271

Conclusions

The tests carried out show that CHOReVOLUTION Identity Manager works fine in managing the ex-pected loads easily.

However, a deterioration of the search operations’ performance can be noticed at the increases of thenumber of existing users. Even more when the the number of concurrent operations grows. Fortunately,as already said above, Elasticsearch can be configured as search back-end for Apache Syncope. Withthis configuration it has been possible a great improvement of the search operation completion time. SoElasticsearch is recommended whenever more than 50,000 users/entities have to be managed.

Further, no deterioration of the performance is noticed by increasing the load period: operations timesseem to remain constants during the whole test (same instance for all tests with no restarts).

Furthermore, it is important to notice that when increasing the number of concurrent operations, theIdentity Manager is able to reply, proportionally, in a good range of time. In fact, when the load has beenincreased of 8 times, operations completion times have been just doubled. Moreover, if we considerthat performance tests have been executed on a single instance hosted by a single machine, we canbe quite sure that by scaling the system horizontally and vertically we are able to easily handle a loadgreater than which ones reported above.

Finally, last but not least consideration is about the user profile defined for tests. It presents a largenumber of attributes (more frequent scenarios would require half of them): we know very well how muchthe number of attributes can heavily affect all the operations with particular reference to the searchone. For this reasons and for all the considerations done till now, we can assert that the executedperformance tests well highlight what we can see as a maximum average cost for typical scenarios.

2https://www.elastic.co/guide/en/elasticsearch/reference/current/index.html


4 CHOReVOLUTION Cloud

In this chapter we describe the final developments related to the Enactment Engine. In details, wewill document the centralised logging features offered to help developers in debugging choreographies.Moreover, we will describe the experimental support for autonomic computing that enables implement-ing autoscaling and self-healing.

In the last section, we will then analyse the results of the tests which has been carried out to measurethe impact and the benefits of using a dedicated and automated solution for enacting choreographies.

4.1. Logging tool for developers

One of the challenges while developing a choreography is to check for errors in a distributed environ-ment. An error can be caused by multiple issues, like networking, bad configuration, security errors ora buggy implementation of a prosumer service. The initial version of a choreography can be tested ona single node, therefore all the application and server logs will be available on a single virtual machine.When scaling up to a production environment, checking the logs manually can become cumbersome,since it may involve finding out which replica of the same service hosted on different virtual machinesproduced the error. As a possible solution to this problem, we decided to integrate the open source,off-the-shelf ELK solution composed by ElasticSearch, LogStash and Kibana. LogStash is an agentwhich is installed on each deployed Tomcat node, and forwards all the produced logs to a running Elas-ticSearch server (one per choreography). Kibana can then be used as a web-based graphical userinterface to access the log messages, view the error frequency over time and to build dashboards tosummarise the log data in a easily accessible fashion.

Figure 4.1: Kibana user interface


4.2. Automatic adaptation in the cloud layer

To explore the limits of the features offered by Brooklyn, we decided to examine the autonomic comput-ing support to be able to support the use cases in showing the benefit of a self-managing choreographyinfrastructure. Below the application layer, a choreography is indeed a set of services hosted on differ-ent Tomcat servers running on multiple virtual machines. Managing such clusters requires the ability tomonitor virtual machine health and load conditions, and to promptly react when a failure is detected.

The ability to continuously monitoring different runtime parameters enables the definition of rulesto drive the automatic management of a cluster. Virtual machines failing or not responding softwarecan be detected and substituted with clean and working ones. Moreover, since the available sensorsinclude the number of requests per second and the response times, we can decide to shrink or enlargea cloud-based cluster according to load.

− type : org . apache . brook lyn . p o l i c y . ha . Serv iceRes ta r te rbrook lyn . con f i g :

f a i lOnRecu r r i ngFa i l u res InTh i sDu ra t i on : 5mbrook lyn . en r i che rs :− type : org . apache . brook lyn . p o l i c y . ha . Se rv i ceFa i l u reDe tec to r

brook lyn . con f i g :e n t i t y F a i l e d . s t a b i l i z a t i o n D e l a y : 30s

In the listing above, a sensor detects whether a service is failing (the “stabilizationDelay” optionmeaning that only one error in 30 seconds can be reported in order to avoid over-reacting in responseto temporary failures). The service is restarted if multiple failures have been reported during the past5 minutes. Server restarts can be configured to be hard or soft. A soft restart implies a server beingsimply killed, whereas a complete reinstall and reconfiguration of the software takes place in case of ahard restart.

− type : org . apache . brook lyn . p o l i c y . au tosca l ing . AutoSca lerPo l icybrook lyn . con f i g :

met r i c : webapp . reqs . perSec . perNodemetricUpperBound : 10metricLowerBound : 1res i zeUpS tab i l i za t i onDe lay : 2mres izeDownStab i l i za t ionDe lay : 1mmaxPoolSize : 10minPoolSize : 4

In the listing above, we define a policy governing the automatic scaling of a cluster. The policyanalyses the average number of requests per second, and mandates an up-scaling action if the valueexceeds 10 and a down-scaling action if the value drops below one. It also states that a grace period oftwo minutes must be observed after an up-scaling (one minute for the down-scaling) to avoid the clustersize continuously bouncing between an increase and a resize. The maximum number of nodes in theexample will be 10, whereas the minimum number will be 4. By tuning such values we can define aself-managing cluster of Tomcat servers hosting CHOReVOLUTION services which is able to avoid aprolonged denial of service state while facing load spikes, and which is able to shrink back to a definedminimum when the processing power exceeds the actual load.

In the table below, we can see that the number of requests per second is not the only sensor thatcan be checked while defining a policy. Moreover, new sensors can be defined to better detect faultconditions or heavy load.


Sensor name Meaningwebapp.reqs.perSec.last Reqs/sec (last datapoint)webapp.reqs.perSec.windowed Reqs/sec (over time window)webapp.reqs.processingTime.fraction.last Fraction of time spent processing, reported by

webserver (percentage, last datapoint)webapp.reqs.processingTime.fraction.windowed Fraction of time spent processing, reported by

webserver (percentage, over time window)webapp.reqs.processingTime.max Max processing time for any single request, re-

ported by webserver (millis)webapp.reqs.processingTime.total Total processing time, reported by webserver

(millis)

4.3. Assessment

To assess how much choreography developers and companies benefit from the Enactment Engine,we repeated the tests described in D3.2 with a more realistic deployment scenario. We focused onassessing the business response time for evaluating the benefit for companies, by comparing the timerequired to build a production-like cluster for hosting the services supporting our use cases. Production-like means either a real production environment ready to be accessed by users, or a testing system usedto find bugs related to a distributed usage.

Exploiting the autoscaling policies allows for an even faster response time. Common monitoringtools like nagios usually ship with triggers for warning system administrators about abnormal high loadconditions. Such events happen during a spike in the number of requests per second and usually causean up-scaling of the cloud environment sustaining the web application. The capabilities of continuouslymonitoring virtual machines parameters described in the previous section and to act according to pre-defined rules enable both to up-scale and to down-scale the environment by setting an upper and alower threshold. That means that according to the scaling rule, we can adapt promptly to the load, andprovide new virtual machines in less than 10 minutes.

Use case deployment scenarios

We decided to run the tests with production-like requirements. We therefore grouped choreographyservices and deployed each group on a different set of redundant virtual machines. The grouping wasdecided on the basis of the number of inbound and outbound exchanges in the choreography model.The Enactment Engine is able to treat each group as a dynamic cluster, meaning that we are able toshrink or enlarge the cluster size according to the expected load.

Figure 4.2 shows the selected deployment scenario for the Urban Traffic Coordination use case. Wedecided to deploy its 13 services (green, gray and red boxes in the diagram) in three different groups.Each group had its dedicated load balancer and its number of virtual machines could be scaled up anddown independently. Each cluster was initially composed by 3 virtual machines, therefore the initialnumber of nodes supporting the choreography was 13 (9 cluster nodes plus 3 load balancer nodes,and a log collection node with ElasticSearch and Kibana).

Figure 4.3 depicts the deployment scenario for the Smart Mobility and Tourism Scenario use case.We decided to deploy its services in two different groups. Each group had its dedicated load balancerand its number of virtual machines could be scaled up and down independently. Each cluster wasinitially composed by 4 virtual machines, plus 2 load balancers, and a log collection node with Elastic-Search and Kibana.


Figure 4.2: WP4 choreography model

4.3.1. Enactment tests

In this section we report on our evaluation of the support that the CHOReVOLUTION platform providesto the developers in dealing with deployment and scaling aspects.

WP4 Deployment time

In this test we measured the performances of the Enactment Engine compared to a human operatorwhile performing the deployment of WP4 choreography. The list of the implied tasks was:

1) create 13 virtual machines VM1 - VM13;

2) install Apache Tomcat on VM1 - VM9;

3) install development environment and compile nginx on VM10, VM11 and VM12;

4) install Apache ODE on VM1 - VM9;

5) install and configure Logstash on VM1 - VM9;

6) install ElasticSearch on VM13;


Figure 4.3: WP5 choreography model

7) install Kibana on VM13;

8) copy choreography services to VM1 - VM9;

9) deploy choreography services on Tomcat in VM1 - VM9;

10) configure nginx as a load balancer on VM10, VM11 and VM12;

11) configure the services according to choreography dependencies using the setInvocationAddressSOAP call.

The Enactment Engine completed the test both on OpenStack and Amazon Web Services EC2 in about12 minutes, with a mean of 4 minutes from issuing the command to the cloud infrastructure to obtainingshell access to the newly created virtual machine. The operator took 4 working days to perform thesame set of tasks. The operator used the cloning features found in both AWS and OpenStack to avoidrepeating the basic configuration of the virtual machines, therefore saving time.

To mimic a much more realistic case, we ran a second round of tests, this time providing the operatorthree base virtual machine images: the first one contained an updated version of the operating systemtogether with a preinstalled Apache Tomcat server with Apache ODE, the second one contained apreconfigured installation of Kibana and ElasticSearch (used to collect and show logs), and the thirdimage contained a preinstalled load balancer ready to be configured. This time the operator was able toperform the tasks list in 2.5 working days. The most time consuming and error-prone task for the humanoperator was to configure the dependencies between services, since it required manually creating andsending message requests using SoapUI. This task alone required around 6 hours.


WP5 Deployment time

In this test we measured the performances of the Enactment Engine compared to a human operatorwhile performing the deployment of WP5 choreography. The list of the implied tasks was:

1) create 11 virtual machines VM1 - VM11;

2) install Apache Tomcat on VM1 through VM8;

3) install development environment and compile nginx on VM9 and VM10;

4) install Apache ODE on VM1 - VM8;

5) install and configure Logstash on VM1 - VM8;

6) install ElasticSearch on VM11;

7) install Kibana on VM11;

8) copy choreography services to VM1 - VM8;

9) deploy choreography services on Tomcat in VM1 - VM8;

10) configure nginx as a load balancer on VM9 and VM10;

11) configure the services according to choreography dependencies using the setInvocationAddressSOAP call.

The Enactment Engine completed the test both on OpenStack and Amazon Web Services EC2 inabout 11 minutes, with a mean of 4 minutes from issuing the command to the cloud infrastructureto obtaining shell access to the newly created virtual machine. The operator took 4 working daysto perform the same set of tasks. The operator used the cloning features found in both AWS andOpenStack to avoid repeating the basic configuration of the virtual machines, therefore saving time.

As for the WP4 case, we repeated the test providing the operator the set of base images to be usedwhile deploying. This time the operator was able to perform the tasks list in 2 working days.

WP4 Upscaling due to high load

In this test we measured the performances of the Enactment Engine compared to a human operator af-ter detecting an abnormally high traffic condition requiring the deployment of additional virtual machinesto the server pool. The list of the implied tasks was:

1) create one virtual machine VMx;

2) install Apache Tomcat on VMx;

3) install Apache ODE on VMx;

4) install and configure Logstash on VMx;

5) copy choreography services to VMx;

6) deploy choreography services on Tomcat in VMx;

7) configure the load balancer on to route requests to the newly created virtual machine;


The Enactment Engine completed the WP4 upscaling test (both on OpenStack and Amazon WebServices EC2) in 6 minutes. The operator took 4 hours to perform the same set of tasks.

As for the first deployment test, we ran a second round of tests, this time providing the operatora base virtual machine image containing an updated version of the operating system together with apreinstalled Apache Tomcat server with Apache ODE. The task list was therefore reduced from theinitial one to its last three elements. This time the operator was able to perform the tasks list in about30 minutes.

WP5 Upscaling due to high load

In this test we measured the performances of the Enactment Engine compared to a human operator af-ter detecting an abnormally high traffic condition requiring the deployment of additional virtual machinesto the server pool. The list of the implied tasks was:

1) create one virtual machine VMx;

2) install Apache Tomcat on VMx;

3) install Apache ODE on VMx;

4) install and configure Logstash on VMx;

5) copy choreography services to VMx;

6) deploy choreography services on Tomcat in VMx;

7) configure the load balancer on to route requests to the newly created virtual machine;

The Enactment Engine completed the WP5 upscaling test (both on OpenStack and Amazon WebServices EC2) in 6 minutes. The operator took around 4 hours to perform the same set of tasks.

As for the first deployment test, we ran a second round of tests, this time providing the operatora base virtual machine image containing an updated version of the operating system together with apreinstalled Apache Tomcat server with Apache ODE. The task list was therefore reduced from theinitial one to the last four elements. This time the operator was able to perform the tasks list in about 30minutes.

Downscaling due to very low load

In this test we measured the performances of the Enactment Engine compared to a human operatorafter detecting an abnormally low traffic condition requiring to downsize the number of virtual machinesin the server pool. For both use cases, the list of the implied tasks was:

1) reconfigure the load balancer on VM2 to route requests to less virtual machines;

2) delete virtual machine VM3;

The Enactment Engine (both on OpenStack and Amazon Web Services EC2) was able to resize theserver pool and to reconfigure the load balancer in about 5 minutes, whereas our operator took around10 minutes to perform the same task. In this case the benefit of the Enactment Engine is less evident.

Test results analysis

In deployment test the Enactment Engine was considerably faster than the human operator. This meansthat besides companies having to deploy choreographies in production environments, also developerscan benefit from the usage of a fully automated solution, since they can test faster new versions of


the choreographies. The upscaling test demonstrated that the Enactment Engine is also able to adaptquickly, since most of the time is spent in waiting the provisioning of the virtual machines by the under-lying cloud infrastructure. The only test where the human operator was nearly on par to the automatedsolution was the last test, as it required really trivial operations.

4.3.2. End-to-End Performance Evaluation

The Enactment Engine features are intended to be exploited by developers and system administrators todevelop or provision choreographies. In the CHOReVOLUTION architecture it is therefore not meant tobe a runtime component. Once the choreography services are deployed onto the cloud infrastructures,the EE monitors the virtual machines and the running servers with a negligible impact on the overallchoreography performances.


5 Conclusions and Future Work

This deliverable described the final version of the CHOReVOLUTION Service Bus, Security and Cloudelements. All the work done during this project was lead by the different use cases requirementsand the necessity to provide components with a high level of performance, flexibility and quality. Newmechanisms were implemented in these different components in relation to the first reflections to offerevolutivity eg the definition and the usage of a new language (GIDL) to describe interfaces facilitatingthe protocol mapping or the usage of security plugins to offer a flexibility regarding the possible securitymechanisms (protection and adaptation).

Regarding the CHOReVOLUTION Service Bus, we provided a complete performance evaluation ofthe VSB runtime environment. Then, we introduced a methodology for performing end-to-end QoSanalysis (at design time) in service/Thing choreographies for both application and middleware layers.However, the design time methodologies do not support dynamic reconfiguration of software compo-nents (services, middleware protocols, etc) to provide appropriate levels of QoS at runtime. To solvethe dynamic reconfiguration problem at runtime we proposed a self-adaptation framework by relyingon the MAPE-K architecture. Then, we detailed the monitoring and analysis components, which arethe first two components of the MAPE-K architecture. As an initial step, we used predefined values forperforming runtime choreography QoS analysis by relying on the QNML tool. Finally, we analyzed theachieved results: queue-lengths and end-to-end response times, which can be used to drive runtimeadaptation..

Regarding the CHOReVOLUTION Enactment Engine, we performed a comparison to evaluate thebenefits of our automated solution with respect to a human operator based on the two use case scenar-ios. We then explored the possibility to adapt the size of the infrastructure sustaining a choreographyby exploiting monitoring data. Moreover, we introduced new features to help developers in dealing withapplication logs produced in a distributed environment.

Regarding the CHOReVOLUTION Security elements, we provided a complete performance evalu-ation of the different security elements. When the performance was not acceptable, some analysisand modification were realized to improve them. New reflections were conducted with the use caseto bring security in the choreography and new mechanism without forcing the end user to provide per-sonal information. Hence the appearance of new functionality offering the possibility of configuring newauthentication mechanisms easily for the designer.

As future work for the next period, the ongoing activities will be continued up to the end of the projectto support use cases and to have a final release of the CHOReVOLUTION platform to be further takenup by the Open Source community.

In particular, concerning VSB runtime adaptation until the end of the 3rd year of the project, weintend to update the QNML tool for supporting additional types of queues (e.g., ON/OFF queue) andfeatures in order to represent the main characteristics of IoT-based choreographies. Afterwards, it isessential to investigate a reasoning mechanism for identifying performance fluctuations by examiningthe analysis results. Then, such QoS-related issues can be communicated to the planning component.Subsequently, the planning component can make an appropriate reconfiguration decision for providingappropriate levels of QoS. Finally, we intend to develop an execution mechanism for implementing theplanned reconfigurations.

For the security part, the last period will be dedicated to the support of the use cases in order to


integrate specific authentication mechanism and the bug correction.Regarding the Enactment Engine, the last period will be dedicated to help the use cases in deploy-

ing the choreographies, and to assist them in the tuning of the infrastructure to reach the expectedperformance levels.


Bibliography

[1] Alessio Angius, Andras Horvath, and Verena Wolf. Approximate transient analysis of queuing net-works by quasi product forms. In International Conference on Analytical and Stochastic ModelingTechniques and Applications, pages 22–36. Springer, 2013.

[2] Uri M Ascher and Linda R Petzold. Computer methods for ordinary differential equations anddifferential-algebraic equations, volume 61. Siam, 1998.

[3] Kendall Atkinson and Weimin Han. Numerical solution of fredholm integral equations of the secondkind. Theoretical Numerical Analysis, pages 473–549, 2009.

[4] Christel Baier, Boudewijn R Haverkort, Holger Hermanns, and Joost-Pieter Katoen. Model check-ing continuous-time markov chains by transient analysis. In CAV, volume 1855, pages 358–372.Springer, 2000.

[5] Andrew Banks and Rahul Gupta. Mqtt version 3.1. 1. OASIS standard, 2014.

[6] Peter Bazan and Reinhard German. Approximate transient analysis of large stochastic modelswith winpepsy-qns. Computer Networks, 53(8):1289–1301, 2009.

[7] Gunter Bolch, Stefan Greiner, Hermann de Meer, and Kishor S Trivedi. Queueing networks andMarkov chains: modeling and performance evaluation with computer science applications. JohnWiley & Sons, 2006.

[8] Richard J Boucherie and Nico M van Dijk. Queueing networks: a fundamental approach, volume154. Springer Science & Business Media, 2010.

[9] David A. Chappell. Enterprise Service Bus. O’Reilly Media, 2004.

[10] Bihuan Chen, Xin Peng, Yijun Yu, Bashar Nuseibeh, and Wenyun Zhao. Self-adaptation throughincremental generative model transformations at runtime. In Proceedings of the 36th InternationalConference on Software Engineering, pages 676–687. ACM, 2014.

[11] Hong Chen and Avi Mandelbaum. Discrete flow networks: bottleneck analysis and fluid approxi-mations. Mathematics of operations research, 16(2):408–446, 1991.

[12] CHOReVOLUTION Project Team. Deliverable 2.2 - CHOReVOLUTION Synthesis - First outcomes,November 2015.

[13] CHOReVOLUTION Project Team. Deliverable 3.1 - CHOReVOLUTION Service Bus, Security andCloud - First outcomes, November 2015.

[14] CHOReVOLUTION Project Team. Deliverable 6.1 - CHOReVOLUTION Platform Requirementsand Integration Requirements, July 2015.

[15] CHOReVOLUTION Project Team. Deliverable 3.2 - CHOReVOLUTION Service Bus, Security andCloud - Intermediate outcomes, November 2016.


[16] CHOReVOLUTION Project Team. Deliverable 4.2 - Urban Traffic Coordination Initial Prototyping,November 2016.

[17] CHOReVOLUTION Project Team. Deliverable 5.2 - Smart Mobility and Tourism Scenario Simula-tion, July 2016.

[18] Autonomic Computing et al. An architectural blueprint for autonomic computing. IBM White Paper,31, 2006.

[19] Tadeusz Czachorski. Queueing models for performance evaluation of computer networkstransientstate analysis. In Analytic Methods in Interdisciplinary Applications, pages 51–80. Springer, 2015.

[20] Stein Desmet, Bruno Volckaert, Steven Van Assche, Bart Dhoedt, Filip De Turck, et al. Throughputevaluation of different enterprise service bus approaches. 2007.

[21] Daniel T Gillespie. A general method for numerically simulating the stochastic time evolution ofcoupled chemical reactions. Journal of computational physics, 22(4):403–434, 1976.

[22] Daniel T Gillespie. Exact stochastic simulation of coupled chemical reactions. The journal ofphysical chemistry, 81(25):2340–2361, 1977.

[23] E Moritz Hahn, Holger Hermanns, Bjorn Wachter, and Lijun Zhang. Time-bounded model checkingof infinite-state continuous-time markov chains. Fundamenta Informaticae, 95(1):129–155, 2009.

[24] Jesper Schmidt Hansen. GNU Octave: Beginner’s Guide: Become a Proficient Octave User byLearning this High-level Scientific Numerical Tool from the Ground Up. Packt Publishing Ltd, 2011.

[25] Mor Harchol-Balter. Performance modeling and design of computer systems: queueing theory inaction. Cambridge University Press, 2013.

[26] Richard A Hayden and Jeremy T Bradley. A fluid analysis framework for a markovian processalgebra. Theoretical Computer Science, 411(22-24):2260–2297, 2010.

[27] Thomas A Henzinger. The theory of hybrid automata. In Verification of Digital and Hybrid Systems,pages 265–292. Springer, 2000.

[28] Emilio Incerto, Mirco Tribastone, and Catia Trubiani. A proactive approach for runtime self-adaptation based on queueing network fluid analysis. In Proceedings of the 1st InternationalWorkshop on Quality-Aware DevOps, QUDOS 2015, pages 19–24, New York, NY, USA, 2015.ACM.

[29] Xiaoyu Ji and Jian Zhou. Solving high-order uncertain differential equations via runge-kuttamethod. IEEE Transactions on Fuzzy Systems, 2017.

[30] B Krishna Kumar, SR Anantha Lakshmi, S Anbarasu, and S Pavai Madheswari. Transient andsteady-state analysis of queueing systems with catastrophes and impatient customers. Interna-tional Journal of Mathematics in Operational Research, 6(5):523–549, 2014.

[31] Paul Kuehn. Approximate analysis of general queuing networks by decomposition. IEEE Transac-tions on Communications, 27(1):113–126, 1979.

[32] Thomas G Kurtz. Solutions of ordinary differential equations as limits of pure jump markov pro-cesses. Journal of applied Probability, 7(1):49–58, 1970.

[33] Will E Leland, Murad S Taqqu, Walter Willinger, and Daniel V Wilson. On the self-similar natureof ethernet traffic. In ACM SIGCOMM Computer Communication Review, volume 23, pages 183–193. ACM, 1993.


[34] Thomas Lenart. Design of reconfigurable hardware architectures for real-time applications, vol-ume 5. Thomas Lenart, 2008.

[35] Yunan Liu and Ward Whitt. A network of time-varying many-server fluid queues with customerabandonment. Operations Research, 59(4):835–846, 2011.

[36] TI Matis, RM Feldman, et al. Correction: Transient analysis of state-dependent queueing networksvia cumulant functions. Journal of Applied Probability, 42(1):302–302, 2005.

[37] Benjamin Melamed. Characterizations of poisson traffic streams in jackson queueing networks.Advances in Applied Probability, 11(2):422–438, 1979.

[38] Trevor Parsons, Adrian Mos, and John Murphy. Non-intrusive end-to-end runtime path tracing forj2ee systems. IEE Proceedings-Software, 153(4):149–161, 2006.

[39] C. Paterson and R. Calinescu. Accurate analysis of quality properties of software with observation-based markov chain refinement. In 2017 IEEE International Conference on Software Architecture(ICSA), pages 121–130, April 2017.

[40] Andrew Reibman, Roger Smith, and Kishor Trivedi. Markov and markov reward model transientanalysis: An overview of numerical approaches. European Journal of Operational Research,40(2):257 – 267, 1989.

[41] Andrew Reibman and Kishor Trivedi. Numerical transient analysis of markov models. Computers& Operations Research, 15(1):19–36, 1988.

[42] Thomas G Robertazzi. Computer networks and systems: queueing theory and performance eval-uation. Springer Science & Business Media, 2012.

[43] Z. Shelby et al. The constrained application protocol (coap). Technical report, 2014.

[44] Simon Spinner, Antonio Filieri, Samuel Kounev, Martina Maggio, and Anders Robertsson. Run-time models for online performance and resource management in data centers. In Self-AwareComputing Systems, pages 485–505. Springer, 2017.

[45] Williams J Stewart. Introduction to the numerical solutions of Markov chains. Princeton Univ.Press, 1994.

[46] Ken Ueno and Michiaki Tatsubori. Early capacity testing of an enterprise service bus. In Web Ser-vices, 2006. ICWS’06. International Conference on Web Services, pages 709–716. IEEE, 2006.

[47] Ward Whitt. Decomposition approximations for time-dependent markovian queueing networks.Operations Research Letters, 24(3):97–103, 1999.

[48] Ward Whitt. Sensitivity of performance in the erlang-a queueing model to changes in the modelparameters. Operations Research, 54(2):247–260, 2006.

[49] Y. Xia, M. Zhou, X. Luo, Q. Zhu, J. Li, and Y. Huang. Stochastic modeling and quality evaluationof infrastructure-as-a-service clouds. IEEE Transactions on Automation Science and Engineering,12(1):162–170, Jan 2015.

[50] E. Zeeb, A. Bobek, H. Bohn, and F. Golatowski. Service-oriented architectures for embeddedsystems using devices profile for web services. In AINA Workshops, Niagara Falls, Canada, May2007.


h2020 ict9 project deliverabled3 - chorevolution.eu · particular its three core enablers, the...

Documents