arxiv:2110.09371v1 [cs.se] 18 oct 2021

Proceedings of the 19th International OvertureWorkshop

Hugo Daniel Macedo, Casper Thule, and Ken Pierce (Editors)

arX

iv:2

110.

0937

1v1

[cs

.SE

] 1

8 O

ct 2

021

Preface

The 19th in the ”Overture” series of workshops on the Vienna Development Method(VDM), associated tools and applications was held as a hybrid event both online atin person at Aarhus University on October 22, 2021. VDM is one of the longest es-tablished formal methods, and yet has a lively community of researchers and practi-tioners in academia and industry grown around the modelling languages (VDM-SL,VDM++, VDM-RT) and tools (VDM VSCode, VDMTools, VDMJ, ViennaTalk, Over-ture, Crescendo, Symphony, and the INTO-CPS chain). Together, these provide a plat-form for work on modelling and analysis technology that includes static and dynamicanalysis, test generation, execution support, and model checking.

Research in VDM is driven by the need to precisely describe systems. In orderto do so, it is also necessary for the associated tooling to serve the current needs ofresearchers and practitioners and therefore remain up to date. The 19th Workshop re-flected the breadth and depth of work supporting and applying VDM. This technicalreport includes first a paper on industrial usage. This is followed by a session on VDMrelated papers concerning techniques for developing models and applying VDM formodelling systems. The last sessions is related to Cyber-Physical Systems and simula-tion.

As the pandemic is still in effect across the world this workshop was held bothonline and in person. We applaud the possibility of meeting some collegues in personand still remaining in contact with others online. It is our sincere hope that the nextOverture Workshop will see an increase in personal attendance.

We would like to thank the authors, PC members, reviewers and participants fortheir help in making this a valuable and successful workshop, and we look forwardtogether to meeting once more in 2022.

Hugo Daniel Macedo, Aarhus Casper Thule, AarhusKen Pierce, Newcastle

Table of Contents

Proceedings of the 19th International Overture Workshop . . . . . . . . . . . . . . . . . . . 1Hugo Daniel Macedo, Casper Thule, and Ken Pierce (Editors)

Behaviour driven specification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5Simon Fraser, Alessandro Pezzoni

Refactoring for Exploratory Specification in VDM-SL . . . . . . . . . . . . . . . . . . . . . 21Tomohiro Oda, Keijiro Araki, Shin Sahara, Han-Myung Chang, and PeterGorm Larsen

Extending the Formal Security Analysis of the HUBCAP sandbox . . . . . . . . . . . . 36Tomas Kulik, Prasad Talasila, Pietro Greco, Giuseppe Veneziano, AngeloMarguglio, Lorenzo Franco Sutton, Peter Gorm Larsen, and Hugo DanielMacedo

Modelling an Injection Moulding Machine using the Vienna DevelopmentMethod . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

Till Böttjer, Michael Sandberg, Peter Gorm Larsen, and Hugo DanielMacedo

RMQFMU: Bridging the Real World with Co-simulation For Practitioners . . 66Mirgita Frasheri, Henrik Ejersbo, Casper Thule, and Lukas Esterle

Genetic Algorithms for Design Space Exploration of Cyber-Physical Systems:an Implementation in INTO-CPS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81

Maximilian Rose and John Fitzgerald

Multi-Objective Optimisation Support for Co-Simulation . . . . . . . . . . . . . . . . . . 96Aiden Stanley and Ken Pierce

Organization

Programme Committee

Tomo Oda Software Research Associate Incorporated, Japan

Marcel Verhoef European Space Agency, The Netherlands

Paolo Masci National Institute of Aerospace (NIA), USA

Peter Gorm Larsen Aarhus University, Denmark

Nick Battle Newcastle University, UK

Fuyuki Ishikawa National Institute of Informatics, Japan

Keijiro Araki National Institute of Technology, Kumamoto College, Japan

Sam Hall Newcastle University, UK

Hugo Daniel Macedo Aarhus University, Denmark

Ken Pierce Newcastle University, UK

Marcel Verhoef European Space Agency, The Netherlands

Kenneth G. Lausdahl AGROCorp International and Aarhus University, Denmark

Behaviour driven specificationA case study in combining formal and agile methods

Simon Fraser, Alessandro Pezzoni

Anaplan Limited, York, UK{simon.fraser, alessandro.pezzoni}@anaplan.com

http://www.anaplan.com

Abstract. The use of formal methods in industry may be resisted by developerswho have embraced agile methods. There is often a perception that formal meth-ods require much upfront work and are unable to adapt quickly to a changinglandscape. This leads to a view that formal methods are a barrier to embracingchange and are unsuitable for projects involving the iterative evolution of require-ments. In this paper, we use a project at Anaplan – a company where agile meth-ods are the norm – to illustrate how formal methods can be integrated into an agiledevelopment process. Specifically, we introduce the concept of behaviour drivenspecification as an agile-compatible method of constructing a formal specificationand show how the acceptance criteria produced by this process not only providean accessible form of requirements capture, but provide a means for validation ofboth the specification and a target implementation.

Keywords: Software Validation, Behavior Driven Development (BDD), AgileMethods, Formal Methods

1 Introduction

Agile methods are becoming ubiquitous in modern software engineering companies.Meanwhile, formal methods are seen as having limited use, with the perception beingthat they can only provide value in niche industries, such as safety-critical systems orchip development. In our experience, many software developers maintain a view thatformal methods are incomprehensible, are tied to linear development practices, and areincompatible with agile methods.

In this paper, we seek to show that formal methods are not mutually exclusive withagile methods, but can can be used alongside them to provide additional rigour in suit-able projects. We begin by introducing Anaplan, a SaaS company delivering an en-terprise planning platform using an agile development process. Having described thefundamentals of the platform, we will outline how its calculation engine is optimizedfor modelling the most common business scenarios. We will then describe a scenariothat could be more efficiently handled using a different approach, how that has led tothe need for an alternative calculation engine, and how formal methods could help usmaintain consistency between the two engines.

We then introduce the concept of behaviour driven specification (BDS), an agile-compatible methodology that we have developed to drive specification from stakeholder

http://www.anaplan.com

6-2ex S. Fraser, A Pezzoni

requirements, and which we have used to help deliver the above calculation engine.BDS is designed to enable specifiers to operate with agility, but we will also describehow it can be integrated into the wider agile process of a software project.

Finally, we present some metrics from our project that exemplify the impact that us-ing BDS has had on our project, and conclude that it is feasible to successfully combineagile and formal methods in an industrial setting.

2 Background

Anaplan is a software company that produces an eponymous, cloud-based platform forbusiness planning. The Anaplan platform provides a flexible model-building environ-ment which customers use to build their own applications. These can model a simplebusiness scenario or manage multiple, complex, planning use cases across a global en-terprise, covering functions such as finance, sales, supply chain, workforce and market-ing.

At the heart of Anaplan is a calculation engine that enables consistent managementof OLAP (online analytical processing) cubes [6]. A cube (or hypercube) is a multi-dimensional array of data such that each cell represents a measure of the business. Anexample cube might represent unit sales and have dimensions: products, shops, weeks,and versions (for example, actual vs forecast). Here the value of a cell would indicateeither a forecast or actual number of units sold of a given product, at a given shop, for agiven week.

The dimensions of a cube are frequently hierarchical structures that model the struc-tures inherent to the business. For instance, shops might have parents that are cities,which in turn have parents that are countries, and similarly weeks may be grouped toalign with the business’s accounting periods. These hierarchical structures can be usedwith an aggregation function to automatically generate additional cubes that summarizethe business’s data. From our examples above, we might have a cube where each cellrepresents either a forecast or actual total of units sold for all products in a country,for a quarter-year period. In Anaplan, the model builder chooses a default aggregationfunction, and the engine automatically calculates all possible summary cubes from theCartesian product of hierarchical layers.

Not all of a business’s data is a direct quantification of some measure, as much ofit may be calculated from other data. For instance, if we had a cube as above whichrepresents units sold, we may have another one with the same dimensions that repre-sents price. Given these two cubes, we could calculate a third cube where each cellrepresents either a forecast or actual revenue from selling a product in a week, obtainedby multiplying the values of the cells with the same coordinates from the given cubes.Anaplan has a rich formula language, providing many inbuilt calculation functions, en-abling model builders to describe all the activities of their business. Within a singlemodel, Anaplan immediately recalculates all necessary figures whenever the value of acell is changed. This makes it possible, for example, for a CEO to visualize the impacton the business’s strategic plan everytime a sale is made at an integrated EPOS.

Behaviour driven specification 7

2.1 A matter of scale

A customer’s Anaplan estate will typically consist of a number of connected models.Each model will usually focus on a specific business problem, allowing detailed anal-ysis, but will also export summarized data to be used in other models. As mentionedpreviously, when any change is made within a model the impact of that change willimmediately be reflected throughout the model – akin to making a change in a spread-sheet. Furthermore, unlike spreadsheets – which are still commonly used for businessplanning – Anaplan models will frequently have billions of cells.

To efficiently maintain the consistency of the Anaplan model, all cube data is heldin memory to enable rapid retrieval and calculation. Larger instances of models canconsume more than 700GB of memory to store the cube data alone. When the memoryrequired to process the data, operate an API, and run the OS is added, it can take aserver with 1TB of memory to operate a model. Although such amounts of memory canbe installed in your own data centres, it is at or near the limit of instances offered bycloud hosting solutions. This is a concern as it can limit options for disaster failover.

Despite the scale of model offered by Anaplan, we are sometimes asked to providefor cubes with trillions of addressable cells. Often this is due to practices carried overfrom less flexible, legacy OLAP tools, but sometimes they are due to inherent propertiesof the scenario being modelled and, upon closer inspection, we find that in the vastmajority of these scenarios not all of these addressable cells are ‘valid’. For example,an aviation company may wish to measure the time taken to fly between airports andcompare it with their competitors. According to IATA [1], there are approximately 9100airports and 1100 airlines. To track the duration of flights between any two airportsby day and airline for the period of a year would require 3.3× 1013 cells. IATA alsoreports [2] that there are less than 40 million flights made a year1. In practice, therefore,less than a thousandth of one percent of the addressable cells will ever be populated.

2.2 An alternative calculation engine

The Anaplan calculation engine was optimised for densely populated models. Usinglarge chunks of contiguous memory for cube data enables efficient calculation, aggrega-tion, and dependency management through mechanical sympathy with the CPU caches[9]. Although memory may not always be assigned for every addressable cell, the re-quirement to use contiguous memory would be inefficient for cubes like those in ourexample.

We decided to begin work on an alternative calculation engine that was optimisedfor the sparse population of a model. Here we would only assign memory for popu-lated cells – enabling larger models – and make use of sparse computation strategies toretain efficiency. This calculation engine will be offered alongside the existing engine,enabling a user to choose appropriately for the business case they are modelling. Wewould generally expect the user to be able to create the same models with either engine,and for both engines to give the same calculation results.

The calculation engine is an integral component of Anaplan, and in order to intro-duce another one we needed to refine the interface between the engine and the rest of

1 In fact, many of these flights will be between the same two airports on the same day.


the platform. This was a task with risk, as it was critical to neither impact the func-tionality nor performance of the existing calculation engine. The approach taken wasto completely fork the platform, quickly make changes to the fork to determine bestpractice and, once established, apply production-quality changes to the mainline. Thenew sparse calculation engine would then be developed in parallel and continuouslyintegrated. This led to having four alternative platforms available for the duration of theproject: the fork and mainline platforms, each paired with either the dense or sparseengine.

2.3 The need for formal methods

The aim of the project was to have all four platforms eventually exhibit the same be-haviour. However, there would naturally be some variation throughout the lifetime ofthe project. We would be working using the agile methodology [18] and the featuresavailable in all four alternatives would vary over time and would not be equivalent atmost points.

Furthermore, in some instances we wanted to alter the existing behaviour since,like most organically evolving software products, Anaplan is the culmination of designdecisions made over many years. If we were to revisit some of these decisions today, wemight choose other options, as now we have more or better data that has invalidated ourprevious assumptions. This is particularly true for design decisions that were influencedby the implementation method at the time or which would be made differently for asparse paradigm. Typically, these decisions impact performance or efficiency.

For example, consider the value of 00: in Mathematics this is either left undefined orset by convention to 00 = 1, and indeed the original definition of the POWER functionin Anaplan follows this convention, giving POWER(0, 0) = 1. This makes sense in adense context, where we can expect that the vast majority of points will have a meaning-ful user-defined value. On the other hand, in a sparse context most points will not havea user-defined value and will instead have a default value, which for numbers is 0. Asa result, the memory consumption would be dramatically increased if POWER(0, 0)returned 1, leading to performance degradation, so for our sparse engine we chose adifferent convention: POWER(0, 0) = 0.

We needed the means to not only ensure that each system exhibited the same basebehaviour, but also to rigorously capture any differences between them. We had a dif-ficulty here as the platform has been developed using agile principles, and one of thetenets of this methodology is to prefer ‘working software over comprehensive docu-mentation’ [5]. Like most teams [20], we still possess and value documentation, but theworking software is the ‘source of truth’, and we did not have an independent means ofbenchmarking its correctness.

The decision was made that a specification was needed. It was necessary that wewere able to capture both the existing behaviour and any differences between the alter-native platforms. It was essential that we could do so rigorously and without ambiguity,making the use of formal methods an obvious choice. However, we also needed thespecification to fit into our agile way or working; it would not be possible to completethe specification before starting work on the new calculation engine, and the platformswould change as the specification was written. Furthermore, we needed to make use of


the specification as soon as possible to enable quick identification and resolution of anydiscrepancy between the platforms’ behaviour. In summary, we needed to combine therigour of formal methods and the flexibility of agile methods.

2.4 Existing work

In the same way that none of the leading relational database systems satisfies Codd’stwelve rules [7], commercial OLAP systems are inconsistent in their mathematicalfoundations. Anaplan encourages the use of many small, dense, and distinct cubes thatare connected through formulae to form a model. Some of the cubes explicitly repre-sent inputs to the model and others its outputs, such that dashboards, charts and reportsare constructed automatically from those outputs. Thus, a user does not use the typi-cal OLAP operations, such as roll-up, drill-down, or slice and dice to interact with thecube. Instead once a model has been constructed, the user simply makes a change andobserves its immediate impact on the output artifacts.

Formalisms have been proposed for a ‘pure’ OLAP approach [15,14], but these havefocused on the use of operations – particularly roll-up – within a single cube; our speci-fication needed to guarantee consistency across a directed graph of cubes. Furthermore,Anaplan’s connected planning methodology enables the use of multiple models – whereeach typically corresponds to an area of a business – and our specification was also re-quired to define the asynchronous relationships between models. These differences, anda desire to use the same methods throughout, discouraged us from pursuing the use ofan existing OLAP specification.

3 Behaviour driven specification

When using agile methods, work is broken into small chunks that each provide somevalue to a stakeholder in the system. These chunks are often framed as user stories [8]which describe the work to be done from the perspective of the stakeholder. Part ofthe user story is a set of acceptance criteria (AC) that give a checklist of what must beachieved in order to consider the work complete.

Test driven development (TDD) [4] is a process frequently used by teams employingagile methods. Whenever the developer wishes to change the code, failing tests arecreated before writing just enough code to make those tests pass. The developer thenrepeats the process until they have made all the changes required. TDD does not alignneatly with user stories as the granularity of the changes tested is usually much finerthan that of the user story.

Behaviour driven development (BDD) [21] is a process that can be used in additionto TDD and which operates at the level of a user story. BDD involves the creation ofscenarios written from the point of view of the stakeholder, and which exemplify theirrequirements. Each scenario is a form of Hoare Triple [11] where we capture that:

– given an initial context (precondition)– when the stakeholder performs an action (command)– then the outcome is as expected (postcondition)


These scenarios are written using a domain-specific language (DSL) for which the de-veloper creates an adapter that enables the scenario to be executed and verified.

We believed that we could adapt BDD to form a behaviour driven specification pro-cess which would enable us to create a formal specification following agile principles.Our principal aim was to be able to comprehensively specify small chunks of the sys-tem on a regular cadence, which would enable developers to leverage the specificationto validate the corresponding chunks of implementation, providing constant feedbackon its correctness and quickly identifying where differences had arisen – whether in-tentional or not. We also wished to provide the developers with a set of requirementsfor the system in a form with which they were familiar – acceptance criteria. By doingthis, we also aimed to significantly reduce the need for natural language commentaryon the specification, replacing it with scenarios written in a DSL which would be bothless ambiguous and more maintainable.

3.1 What does BDS involve?

The BDS process is much like BDD, and it too requires a DSL in which to write sce-narios. However, unlike BDD, where the inputs are ACs and the output is code, withBDS we look to establish ACs and use those to drive the creation of the specification,with both being valuable artifacts of the process.

For each behaviour, the first step is to write a natural language description of theAC. A scenario is then created to exemplify the AC; this may involve the extension ofthe DSL if new language is needed to express the criterion. This scenario now formsan executable acceptance criterion (EAC) – we shall discuss how these are run in sec-tion 3.2 – that we can use to validate the behaviour of both the specification and thetarget implementation. When first run this EAC should fail, otherwise it is likely thatthe AC is redundant or that the scenario does not correctly capture the unicity of theAC. The specifier then makes just enough changes to the specification to satisfy theEAC without invalidating any others. This process is repeated until all of the ACs ofthe behaviour have been established and agreed with the stakeholder. At this point theEACs can be handed to a developer to implement the behaviour. The developer cannow start their work with a set of ACs, including examples scenarios that fail againstthe target implementation — an ideal place to start BDD.

We can rapidly create scenarios to exemplify behaviour questions that arise dur-ing implementation, and we can run them against the specification to determine theexpected outcome. If the developer is not satisfied, we can take that example and thespecified results, and present to the stakeholder. If the stakeholder believes this scenariois distinct from our existing EACs and wants a change of behaviour, we make the sce-nario a new EAC and change just enough of the specification to satisfy it. At this point,the specifier may discover that it is not possible to make a change that is consistentwith the existing EACs. If so, the conflicting criteria are presented to the stakeholderand a resolution agreed. The use of the DSL allows us to do this in the language of thestakeholder, but without the ambiguity of natural language.

Similarly, the developer may discover efficiency or performance issues with imple-menting a particular EAC, or in fact, any member of the team may spot some behaviour


they disagree with. Once again, these can be discussed with the stakeholder, the out-come of the EAC changed, and the requirements of the behaviour, the specification,and the implementation iteratively refined.

The use of EACs not only encourages a more agile approach to specification, butenables its continuous validation through DSL-based requirements which are accessibleto everyone involved in the project.

3.2 Running executable acceptance criteria

Running scenarios requires a DSL in which to express the actions and checks of the sys-tem in the stakeholder’s language, and means by which to translate that language intocommands and queries that the implementation can execute. Cucumber [21] is synony-mous with BDD, but we found it unsuitable for use with BDS. Although Cucumberaccepts scenarios in the given-when-then format, the distinction between the blocks islost at runtime and the scenario is reduced to a single sequence of steps. When execut-ing a program it is perhaps unimportant that certain steps are declarative and others areimperative2, but when animating a specification we wanted to transform the declara-tive steps into a single declaration rather than calling a series of functions. After sometime trying to adapt Cucumber to our needs, we found that we were better able to makeprogress with a custom Kotlin DSL [19].

In practice, we found the majority of our effort in this area was in creating theDSL and adapting to the specification and the various implementations, rather than inthe mechanics of execution. We briefly describe our approach to these mechanics forcompleteness, and it is likely that another team employing BDS would benefit fromreplicating the approach rather than using our tooling directly.

Example of some trivial EACs are given in listings 1.1 and 1.23.

@Eac("A created list is empty")fun create() {

whenever {createAList("list")

}then {

listContains("list")}

}

Listing 1.1. EAC for creating a list

@Eac("When an entity is added,it is contained by the list")

fun addToAList() {given {thereIsAList("list", "a")

}whenever {addEntityToList("list", "b")

}then {listContains("list", "a", "b")

}}

Listing 1.2. EAC for adding to a list

A single EAC is run using a custom JUnit runner; this means that all the usual JUnittools are available for running from an IDE, a build tool, or a continuous integrationenvironment. To run an EAC, the runner first identifies what implementations are avail-able on the classpath and whether any constraints have been placed on the EAC. For

2 We found it useful to make the distinction with the implementation too. Anaplan is a transac-tional system and it was useful for us to be able to create the declarative, initial context in asingle transaction.

3 Note that the keyword whenever is used in the scenario as when is a Kotlin keyword.


example, we might want to skip a particular implementation due to a known bug anduse an annotation to indicate this. Once the list of available implementations has beendetermined, the runner will attempt to run the EAC against each of them.

The DSL is backed by an API that is implemented by an adapter for each imple-mentation. The JUnit runner will use this API to convert the DSL into a series of im-plementation specific commands. There is not a one-to-one correspondence here as,particularly with the declaration, an adapter may choose to maintain a state and createa single command corresponding to several calls to the API.

The adapter is also allowed to return an ‘unsupported command’ from any APImethod – indicating that there is piece of functionality that has not yet been imple-mented, or perhaps which may never be implemented. For this reason, during conver-sion, the runner does not instantiate an implementation and none of the commands areexecuted. Only if all commands are supported does the runner actually execute themagainst the implementation, otherwise it can quickly skip the EAC without having topartially execute the scenario. This is extremely valuable for BDS where – unlike BDD– we are usually working with more than one adapter and development may not imme-diately follow specification. When the specification of a behaviour is complete, we canmark that behaviour as unsupported in the implementation’s adapter until developmentcan begin.

This capability is even more useful when working with multiple implementations— as in our Anaplan project. Consider the example in 2.3: we want to be able to specifyboth the dense and sparse behaviour of calculating 00, but clearly each implementationwill only support one outcome. We can extend our DSL to be able to disambiguatethe two methodologies4 and write two EACs as shown in listings 1.3 and 1.4. Our twosparse platforms would return an ‘unsupported command’ for the EAC requiring densearithmetic and would be skipped, but would be able to run the EAC requiring sparsearithmetic (and vice versa for the two dense platforms). Our specification would beable to support both arithmetic methodologies and evaluate both EACs. From a docu-mentation perspective, this is ideal as we can place the EACs side by side, to compareand contrast the two behaviours.

given {thereIsAFormula("f",

"POWER(0,0)",arithmetic = DENSE)

}then {

formulaEvaluatesTo("f", 1)}

Listing 1.3. Dense EAC

given {thereIsAFormula("f","POWER(0,0)",arithmetic = SPARSE)

}then {formulaEvaluatesTo("f", 0)

}

Listing 1.4. Sparse EAC

3.3 Adapting to a specification

A requirement of BDS is the capability to create an adapter from the DSL to the specifi-cation, which requires a specification framework where animation is feasible. We con-sidered a number of formal specification systems with suitable tooling, but – to reaffirm

4 To prevent cluttering, we would only specify the arithmetic used in the EAC when it matteredand we would expect identical behaviour otherwise.


the arguments presented in section 3 of [13] – as engineers in industry we preferred theflexibility and familiarity of VDM [12] to more alien, proof-oriented methods such asEvent-B [3].

VDM’s Overture ecosystem provides a choice of IDEs that can animate a speci-fication from the UI, while the published JARs enabled us to perform animation pro-grammatically. Specifically, the capability to animate any function or operation fromthe specification was a significant boon as it enabled us to construct focused EACsrather than requiring a full system definition in every instance. This flexibility, and thematurity of the tooling, enabled us to build the specification adapter iteratively, doing‘just enough’ to run new EACs. Having this capability enabled us to be fully agile inour process, and we believe that it made VDM an ideal choice for our project, and forBDS generally.

In [16], Oda illustrates how animation in VDM can be used to bridge the languagegap between that of a project’s stakeholders and that of a formal specification. TheEACs used in BDS provide an extra layer of abstraction to provide a single languagethat can also be used by developers. Sections 4, 5, and 6 of [16] demonstrate howanimation can be driven interactively through various interfaces, but for our EACs wetook an even simpler approach. Our VDM adapter creates a module that represents theEAC, animates it, and then checks the final returned value to determine success. Themodule consists of a single operation that begins with all the declarations made in thegiven section of the EAC, followed by the commands corresponding to the wheneversection. Each check in the then section is translated to a block that returns false if thecheck is failed. If the animation proceeds through all checks successfully then the EAChas passed and true is returned.�AnimateCheck: () ==> boolAnimateCheck() ==((

dcl formulaA: Formula‘Formula := mk_Formula‘UnsafeFormula([DataTypeLiterals‘CreateNumberLiteral(0.0),DataTypeLiterals‘CreateNumberLiteral(0.0),AnaplanFormulaFunctions‘POWER_SPARSE_2

], {1 |-> [], 2 |-> [], 3 |-> [1, 2]});(

dcl f: Formula‘Formula := formulaA;letexpected = ValueOption‘Create[Number‘Number](0)

inif not TestEquality‘Equals(FormulaModelling‘Evaluate(f, { |-> }), expected)then (

IO‘println("* Check ’FormulaEvaluationCheck’ failed (tolerant = true)");IO‘print(" Expected: ");IO‘println(expected);IO‘print(" Actual: ");IO‘println(FormulaModelling‘Evaluate(f, { |-> }));return false;

)else IO‘println("* Check ’FormulaEvaluationCheck’ passed");

););return true

)� �Listing 1.5. A translated EAC specification


For example, the EAC in listing 1.4 generates the VDM operation shown in listing 1.5.If an EAC fails, it is trivial to open the workspace in Overture and debug the anima-

tion of the specification using its rich tooling.

4 Integrating into the Agile Process

An agile project typically starts with the creation of user stories, which are then prior-itized and placed into an initial backlog. This is usually led by a product owner (PO)who collaborates with customers to get a general feel for what a new feature should doand to gauge the importance of each of its aspects. As work progresses, the team thengets feedback from the customer and new user stories are created to refine and extendthe behaviours of the features.

For our project we had a good idea of what the calculation engine should do —generally the same as the existing engine. The initialization of the backlog was thereforea little different as the PO was able to quickly identify the features required and rankthem according to customer need5. Specification and development teams worked withthe same prioritized backlog, but independently.

There was no need for specification to complete before implementation, if the speci-fication team found bugs or missing behaviours they would simply feed into the backlogof the development teams. Similarly, the specification was not untouchable, if differentchoices made sense for the implementation, they could be discussed with the PO, andif approved the required change would be added to the specification team’s backlog.In either case the first step would be to create an EAC that exemplified the expectedbehaviour, which would be referenced in the ticket created in the backlog and couldbe immediately committed with an appropriate annotation indicating that it was notsupported by relevant implementations. The fluidity of the specification and the im-plementations meant that the EACs generated during BDS were considered to be the‘source of truth’. The specification was ‘complete’ when it satisfied all EACs in thesystem, and an implementation was ‘complete’ when it satisfied the subset of EACsthat described its full behaviour6.

As a project progresses, the PO and team groom the backlog on a regular basis andrefine the features at the top of the backlog. For development work this usually involvesestablishing the set of AC required for the PO to mark the story as done. This was alittle different for specification work as the ACs were an output of the BDS process.Instead, grooming involved identifying the functional elements required for the featureand establishing a broad set of behaviours for those elements. A trivial example, mightbe a list functional element – a data structure which can be used as a dimension of acube – with behaviours such as creating a list, adding an item to a list, or moving anitem within a list.

5 Note that, although our intent was to provide feature parity, like all agile projects we wished toship when we had a minimum viable product and incrementally broaden the feature set withregular releases thereafter.

6 For this reason it was important to distinguish an action that was not yet supported from onethat would never be supported


The initial set of behaviours would be established from a cursory inspection of theexisting system and was created to give an idea of the work involved but, as normal in anagile environment, we would expect change as more detailed analysis and specificationwere completed.

It was not considered feasible to operate a methodology where sizing was required,as these size estimates are usually guided by the ACs. Instead, a lean methodology [17]was used, with an intent that a team member complete the specification of a singlebehaviour without interruption. We would imagine that this would be a choice made byall agile specification teams.

4.1 Specifying a behaviour

The general procedure of BDS has been detailed in section 3.1, while here we elaboratea little more on the process followed in our project.

The first step of BDS is to identify an AC. For example, the behaviour that allowsadding an entity to the end of a list could be described as ‘when an entity is added, itis contained by the list’. We would then write an EAC for this – as previously seen inlisting 1.2. Here, we had the advantage that we could immediately run the EAC againstthe dense platforms to validate that we had captured the behaviour correctly.

We would then try to run the EAC against the sparse platforms. If the requiredfunctionality had not yet been implemented, we could ensure that the adapter returnedthe unsupported command. If the EAC ran and succeeded we could conclude that it hasbeen implemented correctly, but if it ran and failed our next step would be to determinewhether the difference was believed to be correct or incorrect. If the difference wasacknowledged to be incorrect, we could raise a bug referencing the EAC and feed it intothe appropriate team’s backlog. On the other hand, if the development team believedthat they had implemented the correct behaviour, we would raise a ticket in a formaldifference backlog. At this point we would mark the EAC as ‘under review’ until thedifference was discussed by a working group at a fortnightly meeting, where all thestakeholders would discuss differences and make an informed decision on whether theproposals should be accepted or rejected7. Once the difference was resolved, it wouldbe necessary to return to the EAC and ensure if reflected the decision made.

The specifier then runs the EAC against the specification to ensure it fails, the EACis marked as unsupported by the specification, and the author pushes the work to abranch and raises a pull request (PR) for review by another team member. Once thereview of the PR is complete – and all necessary builds have passed – the EAC ismerged to the mainline, and specification can proceed.

There is not too much to say about writing the specification itself, other than to re-inforce that the only changes should be those required to make the EAC pass, and thatthe presence of EACs does not divest the author of the responsibility to write units testswithin the VDM, which should focus on testing the correctness of each individual func-tion, as well as its pre and post conditions. Having the EACs in place should discourage

7 Note that, as this was an agile process, there was always a possibility that a future discoverycould invalidate the decision, and members of the team were free to reopen the discussion andpresent new evidence to the working group before a new decision was made.


the specifier from writing end-to-end tests directly in the VDM, and instead ensure thefocus is on good coverage of the specification.

4.2 Continuous validation

An important part of agility is continually integrating the pieces of work from all partsof the team to ensure that changes do not cause problems elsewhere. The Gradle pluginintroduced in [10] enables us to build the specification, run all the VDM unit tests inJenkins, and publish the specification to Artifactory.

Change to the DSL and its associated adapters are managed like any other agiledevelopment work – with frequent commits, PRs, and adoption upstream. The imple-mentation adapters leverage Java’s service loading mechanism to use whichever versionof the implementation they find on their classpath, while the VDM adapter was neces-sarily tied to a specific version of the specification, since part of its function was topackage the text-based specification into a Java library that could be ‘run’.

The EACs are all placed into a single validation repository, and whenever the main-line of this project builds successfully, it also checks for newer versions of the DSL, theadapters and the implementations. If a new version is found, the system automaticallyraises a PR to update the version numbers of the dependencies, and if this PR buildssuccessfully, it is considered a team priority to merge it quickly. On occasion, a versionupdate PR will fail, typically due to either a regression of behaviour in an implemen-tation, or because some previously unsupported action has been implemented, causingan EAC that was previously being skipped to fail. At this point we follow the sameprocess that we use when a new EAC fails, as described previously. Again, resolvingthis difference is considered a team priority. As with any agile development team, teammembers will work on a failing mainline or version update build – rather than regularassigned work – until it is fixed.

5 Results

Behaviour driven specification was not a fully-formed concept when we began, but –like all agile processes – has evolved as the project progressed. We believe that the useof BDS not only has enabled us to deliver more efficiently, but has produced a moremaintainable specification where behaviours are encapsulated and decoupled. In total,we have used BDS to create more than 60,000 lines of VDM-SL, of which 26,000lines are the 231 modules that form the specification. There are also 115 modules ofVDM-based unit tests, containing more than 3,000 tests and comprising 34,000 lines.

Our specification is split into multiple projects and we use the Gradle plugin pre-sented in [10] to manage dependencies between them. This enables us to apply a greaterdegree of componentization than VDM-SL provides out-of-the-box and prevents spec-ifiers accidentally using information that they should not know about. For example, wehave a ‘hierarchical data’ project that has a dependency declared on an ‘ordered data’project. When working in the hierarchical data project, the specifier can use modulesfrom the ordered data project, but when working in the ordered data project the spec-ifier cannot use modules from the hierarchy data project. The underlying dependency


mechanism is the same one Gradle uses for Java; dependencies are acquired transitivelywithout redeclaration and cyclic dependencies are forbidden.

Our specification is split into 54 VDM-SL projects that are all depended upon –directly or transitively – by a ‘system’ project, which is published to the company’sArtifactory instance and used by the VDM adapter for running the EACs.

We additionally have seven library projects which provide Java implementations forVDM modules. For the most part, these provide speedy implementations for genericfunctionality that is not specific to our problem. For instance, we have libraries forIEEE-754 floating point, ISO-8601 dates, and for standard graph algorithms. We alsoused JNI to create a Rust-based library to overcome Java’s limitations with Unicodecharacters outside the Basic Multilingual Plane.

In hindsight, we should have used libraries more heavily to provide the executionmechanisms for our functions, and focussed on solely writing VDM-SL pre and post-conditions. As we added more and more EACs, and particularly as we generated largerscenarios, the speed of animating the specification became restrictive, and we had tomake compromises in our approach; this was particularly true when a scenario had alarge memory footprint and garbage collection became a significant proportion of theruntime.

One reason that we did not completely embrace library modules earlier in the projectwas that we already had fully-tested VDM from a previous project, which we were notkeen to throw away. What we would like to have seen is the capability to use a librarywith a module where function definitions were provided with a runtime flag used toselect whether to use the VDM definition or the library definition. If that were available,we would likely use the VDM definition in local work and the library definitions in theCI environment.

At time of writing we have approximately 3,000 EACs that are run against eachof the four alternative platforms whenever a commit is made to the validation reposi-tory, giving us confidence that each continually meets the defined system requirements.Once EACs are published they are also run against each of the platform implemen-tations when commits are made to their respective repositories. As alluded to above,running the same set of EACs against the specification on every commit has proved tobe too slow. However, we run these in a perpetual loop on a dedicated cloud instanceand get immediate feedback if and when an EAC fails. We have found that regressionsare very rare in the specification, but this process is useful for capturing unintended con-sequences when requirements change in a related area. We have also tended to tightenpreconditions, postconditions and invariants as the project has progressed, and this con-tinuous validation process has enabled us to quickly spot issues that low-level VDMunit tests did not catch.

As with behaviour driven development, creating a DSL and relevant adapters isa necessary part of BDS. Although we had five adapters to maintain, it was not asonerous as it might seem: much of the complexity in Anaplan is in the point transfor-mations that are possible within the geometry of the cubes. These transformations areexpressed in Anaplan’s formula syntax, and early on in the project we established con-ventions within the adapters for automatically mapping keywords into functions withinthe specification. We introduced a special module – AnaplanFormulaFunctions


– into our specification which served as an ‘interface’ for these keywords. For in-stance, in listing 1.5, we can see that the formula keyword from listing 1.4 has beenconverted by convention to POWER_SPARSE_28. The specifier creates a value in theAnaplanFormulaFunctions module that is an alias to the actual definition thatthey can specify in the most appropriate location. Similarly, the VDM adapter runs apre-build process to generate stubs for all exported parts of the specification, so thatwe can ensure compile-time consistency between the specification and adapter. Thefour implementation adapaters were all based on the same abstract adapter and onlyvaried where they supported different capabilities or where the code of the alternativeplatforms was not consistent.

One of the most important outputs of the specification process has been the iden-tification of differences between the sparse and dense calculation engines. Nearly 200differences have been captured by the specification team, and it is highly unlikely thatthey would have been discovered by other means. Each of these has been fully discussedand an informed, justified decision has been made. These differences can now be fullydocumented before a customer encounters them, so when they do they will not only beable to see that the difference is intentional, but they will be able to view the reasoningbehind it.

For the members of the specification team, BDS has had another benefit. We havegrown the size of the team from two to seven during the course of the project, withnew members having varied backgrounds. BDS enforces the breakdown of work intosmall, distinct chunks, which we have found to enable new members of the team tomake meaningful contributions early in their tenure. It is reasonably easy for anyone tolearn to analyse a behaviour and write EACs quickly, and doing so gives new membersconfidence to alter the specification – they really understand the change needed even ifthey do not understand the whole specification. Similarly, it allows a more experiencedspecifier to move more easily to an area of the specification they have not worked withbefore – they can select some simple EACs and run through how they are handled bythe specification, then add some basic EACs for their new behaviour. The EAC-drivenworkflow really helps comprehension of the specification, even for those who alreadyhave VDM expertise.

6 Future work

We have begun to use the tools developed for BDS to assist with verifying our imple-mentations. We are constructing a verification framework that generates semi-randomscenarios from the instructions available in the DSL and runs them against target sys-tems. These scenarios are slightly more expressive than those used for validation. Forexample, it is possible to use a query to determine the valid points in a cube and thencheck that the value at each point in the target system satisfies the specification.

Briefly, the verification process proceeds as follows:

8 As it has two arguments and uses the sparse arithmetic. The number of arguments is not impor-tant in this instance, but is needed as Anaplan formula syntax allows overloading, but VDM-SLdoes not.


1. Generate a scenario.2. Check that the given and whenever blocks satisfy the specification – that is, that the

scenario satisfies the precondition and that the command make sense.3. Run the scenario against the implementation in a mode that queries rather than

checks.4. Use the results of the query to generate an equivalent scenario containing a then

block, and run that scenario against the specification.

At the time of writing, we are continuing to develop the verification framework, andwe are also scaling up its use in checking the correctness of the alternative calcula-tion engine. Currently, we have only preliminary results, which are not yet ready forpublication.

7 Concluding remarks

We started using VDM at Anaplan before beginning the project described in this paper,but the focus had been on investigating new modelling concepts rather than the deliveryof working software. With the switch to a delivery-focus, we knew we needed a processthat could align with the workflow of the development teams assigned to the project.Behaviour driven specification proved to be such a process, enabling us to create aformal specification with the development teams rather than for them.

In the initial phases of the project, there was a suspicion that formal methods wereintrinsically linked to waterfall development. This worried developers who believedthey would lose the ability to influence the system’s requirements. There was a beliefthat the specification team would form a barrier between the product owner and thedevelopers, and at early meetings we needed to remind the wider group that the role ofthe specification team was to record decisions, not to make them. We believe that theuse of BDS helped to overcome this: not only because it has enabled the specificationprocess to be more flexible, but also because it has made the specifiers feel like a col-laborative partner in the implementation effort rather than a siloed team issuing diktatsto developers. The use of executable acceptance criteria has reinforced this; having a‘source of truth’ expressed in a language that requires no special understanding enablesall members of the project team to engage on an equal footing.

As we approach our first customer-facing release, the feeling across the wholeproject team is that the integration of formal specification into the agile process hasbeen a success, with specifiers, developers, and product owners all highlighting thevalue it has provided.

References

1. Airline and location code search. https://www.iata.org/en/publications/directories/code-search/, accessed: 2020-06-11

2. Economic performance of the airline industry. https://www.iata.org/en/iata-repository/publications/economic-reports/airline-industry-economic-performance-june-2020-report, ac-cessed: 2020-06-11

https://www.iata.org/en/publications/directories/code-search/

https://www.iata.org/en/publications/directories/code-search/

https://www.iata.org/en/iata-repository/publications/economic-reports/airline-industry-economic-performance-june-2020-report




3. Abrial, J.R.: Modeling in Event-B: system and software engineering. Cambridge UniversityPress (2010)

4. Beck, K.: Test-driven development: By example. Addison-Wesley Professional (2003)5. Beck, K., Beedle, M., van Bennekum, A., Cockburn, A., Cunningham, W., Fowler, M., Gren-

ning, J., Highsmith, J., Hunt, A., Jeffries, R., Kern, J., Marick, B., Martin, R.C., Mellor, S.,Schwaber, K., Sutherland, J., Thomas, D.: Manifesto for agile software development (2001),http://www.agilemanifesto.org/

6. Codd, E.F., Codd, S.B., Salley, C.T.: Providing OLAP (on-line analytical processing) to user-analysts: An IT mandate. Tech. rep. (1993)

7. Codd, E.F.: The relational model for database management: version 2. Addison-WesleyLongman Publishing Co., Inc. (1990)

8. Cohn, M.: User stories applied: For agile software development. Addison-Wesley Profes-sional (2004)

9. Drepper, U.: What every programmer should know about memory. Red Hat, Inc 11, 2007(2007)

10. Fraser, S.: Integrating VDM-SL into the continuous delivery pipelines of cloud-based soft-ware. In: The 16th Overture Workshop. pp. 123–138 (2018)

11. Hoare, C.A.R.: An axiomatic basis for computer programming. Communications of theACM 12(10), 576–580 (1969)

12. Larsen, P.G., Battle, N., Ferreira, M., Fitzgerald, J., Lausdahl, K., Verhoef, M.: The Overtureinitiative integrating tools for VDM. ACM SIGSOFT Software Engineering Notes 35(1),1–6 (2010)

13. Larsen, P.G., Fitzgerald, J.S., Wolff, S.: Are formal methods ready for agility? A realitycheck. In: FM+AM 2010: Second International Workshop on Formal Methods and AgileMethods,. Newcastle University (2011)

14. Lenz, H.J., Thalheim, B.: A formal framework of aggregation for the OLAP-OLTP model. J.Univers. Comput. Sci. 15(1), 273–303 (2009)

15. Macedo, H.D., Oliveira, J.N.: A linear algebra approach to OLAP. Formal Aspects of Com-puting 27(2), 283–307 (2015)

16. Oda, T., Yamomoto, Y., Nakakoji, K., Araki, K., Larsen, P.G.: VDM animation for a widerrange of stakeholders. In: Proceedings of the 13th Overture Workshop. pp. 18–32 (2015)

17. Shalloway, A., Beaver, G., Trott, J.R.: Lean-agile software development: achieving enterpriseagility. Pearson Education (2009)

18. Shore, J., et al.: The Art of Agile Development: Pragmatic guide to agile software develop-ment. O’Reilly Media, Inc. (2007)

19. Subramaniam, V.: Programming DSLs in Kotlin. Pragmatic Bookshelf (2021)20. Wagenaar, G., Overbeek, S., Lucassen, G., Brinkkemper, S., Schneider, K.: Working soft-

ware over comprehensive documentation–Rationales of agile teams for artefacts usage. Jour-nal of Software Engineering Research and Development 6(1), 1–23 (2018)

21. Wynne, M., Hellesoy, A., Tooke, S.: The Cucumber book: Behaviour-driven developmentfor testers and developers. Pragmatic Bookshelf (2017)

http://www.agilemanifesto.org/

Refactoring for Exploratory Specification in VDM-SL

Tomohiro Oda1, Keijiro Araki2, Shin Sahara3, Han-Myung Chang4, and Peter GormLarsen5

1 Software Research Associates, Inc. ([email protected])2 National Institute of Technology, Kumamoto College ([email protected])

3 Hosei University ([email protected])4 Nanzan University ([email protected])

5 Aarhus University, DIGIT, Department of Electrical and ComputerEngineering, ([email protected])

Abstract. This paper discusses refactoring as a tool for identifying concepts inthe problem domain and defining them with appropriate language elements in ap-propriate modules through abstraction and generalisation. Refactoring operationsrequired for exploratory specification are identified, and their implementation ona tool called ViennaTalk is described and discussed.

1 Introduction

Formal specifications have been reported to increase productivity and reliability by de-scribing system functionality with rigour, enabling early detection and correction ofspecification-induced problems in software development. VDM-SL is a formal specifi-cation language with an executable subset [1]. Interpreted executions of the specifica-tion, or specification animation, can be used to simulate and try out the behaviour of thesystem before implementation takes place.

Specification animation is a powerful tool for deepening understanding of the prob-lem domain. Software development is typically an ill-defined problem where the prob-lem to be solved is not defined in advance. Although a development project starts withobjectives and a tentative goal, the developers do not always have a full understandingof the problem domain. During the development, the developers learn and understandthe problems to be solved, and specify, design, and implement the software as a meansto solve them. Especially in the early stages, it is important to learn about the prob-lem domain by defining and animating the formal specification, and also feedback fromstakeholders. Exploration involves frequent modification to the specification to identifyassociated concepts and find their definition with appropriate abstraction and concisepresentation.

Refactoring is a technique to modify the program source while keeping the be-haviour of the program code [4]. Refactoring has mainly been used to improve themaintainability of code. In particular, in agile development, refactoring plays an impor-tant role in removing design distortions caused by added features and in preparing forfurther feature additions. The tool support for refactoring originated from the program-ming community of Smalltalk [8]. Smalltalk is by its nature bundled with powerfulIDEs featuring class browsers that strongly support prototyping. It is not a coincidence

22-2ex Oda, T.; Araki, K.; Sahara, S.; Chang, H.M.; Larsen, P.G.

that Smalltalk also delivered a unit testing framework to the programming communitywhich later derived JUnit [3].

Both refactoring tools and unit testing framework support rapid modification to theprogram code in prototyping. Exploratory specification and prototyping share rapidmodification to the source as well as learning aspects through trial and error. Refac-toring in agile development support the maintenance aspect of prototyping. Rapid trialsand errors followed by modifications often make the source code less readable and thusmake further modifications harder. On the other hand, the exploratory specification val-ues its learning aspect. We expect refactoring techniques to help the engineers widenand deepen their understandings of the problem domain.

Section 2 will discuss the role of refactoring in the exploratory specification phaseand show families of semi-automated refactoring operations for exploratory specifica-tion. In Section 3, refactoring functionalities implemented in ViennaTalk will be de-scribed with an example. Afterwards, Section 4 explains existing research on refactor-ing of VDM-SL. Finally, Section 5 discusses the result of implementation and con-cludes the paper.

2 Refactoring for exploration

Refactoring is technically a collection of transformations on abstract syntax trees (ASTs)that preserve the behaviour of the existing code. Refactoring of program code oftenbenefits the maintainability of the code including readability and extensibility by reor-ganising the structure of the code. Refactoring is often applied to code fragments thattypically contain magic numbers, frequent expressions that can be abstracted, defini-tions located in inappropriate modules, misleading identifiers, and so on. In the contextof software maintenance, such code fragments are often called code smell.

The exploratory specification is the stage of writing a specification with a partialunderstanding of the application domain. Software development typically goes parallelwith acquiring new concepts in the target domain and exploring how they are associatedwith other known concepts. Since a specification engineer who writes a specificationdoes not have an accurate grasp of the entire domain, many parts of the specificationremain tentative. The specification at this stage is not always comprehensive and con-cise due to limited understanding of the domain and thus contains code smell. In theexploratory specification, they are signs of learning opportunities. While code smell insoftware maintenance is considered a threat to the maintainability of the code, it is anopportunity for learning in the exploratory specification. Refactoring on a formal spec-ification does not add new functionality to the system to be developed, but adds moresenses to the model.

Our approach to refactoring operations focuses on names. In programming, refac-toring is sometimes carried out to improve performance by modifying to faster codeand data structures without changing the functionality. Although VDM-SL has an ex-ecutable subset, performance is less emphasised in general. We value the readabilityand maintainability brought by appropriate naming and abstraction. In this research, weidentified and implemented a set of semi-automated refactoring operations for naming,renaming, and unnaming a fragment of the specification source. The goal is to provide

Refactoring for Exploratory Specification in VDM-SL 23

a safe way to modify the specification without unintended change of the behaviour andalso to reduce the workload of the editing task.

2.1 Refactoring for naming

The first group of refactoring operations is to give a name to a fragment of the speci-fication. In VDM-SL, there are three kinds of scopes of names: (1) global names, e.g.module name, (2) module-wide names defined in types sections, values sections, statesections, functions sections and operations sections, and (3) local names such as localdefinitions in let expressions/statements, and pattern identifiers for the pattern matchingin parameters of functions/operations, local binding in set/sequence/map comprehen-sions, lambdas, quantifiers, loop constructs and so on.

The Extract family of refactoring operations generates the language constructs enu-merated above from a fragment of the specification. The Use family of refactoring op-erations replaces fragments in the same pattern with another that uses names.

Extracting module-wide names In VDM-SL, type definitions, value definitions, statevariables, function definitions, and operation definitions have the module-wide scope.Extract type is a refactoring operation to define a new type definition for the specifiedtype expression. The newly defined type name will be used In the original place of thespecified expression.

Extract value defines a new value definition for the specified expression. Please notethat a refactoring operation should be safe; the resulting specification should not breakthe well-formedness of the specification. If the specified expression contains a referenceto a state variable or an operation call, the tool should not apply the operation.

�functionspriceIncludingTax : real -> realpriceIncludingTax(displayPrice) ==displayPrice + displayPrice * 0.1� �

⇓ Extract value on 0.1�valuesTAX_RATE = 0.1

functionspriceIncludingTax : real -> realpriceIncludingTax(displayPrice) ==displayPrice + displayPrice * TAX_RATE� �

Fig. 1. An example application of the Extract value refactoring

Figure 1 illustrates Extract value operation that transforms the definition above thearrow into the one below by adding a value definition. The original definition specifies


the computation of price with 10% sales tax using a magic number 0.1. By extracting0.1 as a constant value named TAX_RATE, the definition becomes more expressive tothe readers.

Extract state variable adds a new state variable. The expression specified by theuser will be used in the initialiser of the state definition. The specified expression will beused as the right-hand side of the type definition. If the specified expression contains astateful subexpression, the tool must block the refactoring. Also, the tool must guaranteethat all the record constructor expressions with the state must be given an appropriateelement for the newly added field. A conservative approach is to prohibit the Extractstate variable refactoring when there are one or more record constructors with the state.

Extract function and Extract operation define a new definition of function and oper-ation accordingly. To define a new function, a let expression is a language construct ofVDM-SL that has corresponding functionality with function application: parameters,arguments and a body. If there is no dependency among local definitions in a let ex-pression, pairs of a local name and its value can be interpreted as pairs of a parameterand an argument as the local bindings to evaluate the body expression. The tool mustcheck whether the body expression contains a stateful subexpression and also a refer-ence to other local names that are invisible in the top-level scope. The let statement cansimilarly be converted into an operation definition.

�functionspriceIncludingTax : real -> realpriceIncludingTax(displayPrice) ==displayPrice +let rate:real = 0.1, taxable = displayPrice in taxable * rate� �

⇓ Extract function�functionspriceIncludingTax : real -> realpriceIncludingTax(displayPrice) ==displayPrice + tax(0.1, displayPrice);

tax : real * ? -> ?tax(rate, taxable) == taxable * rate� �

Fig. 2. An example application of Extract function refactoring

Figure 2 shows an example application of Extract function. In the original specifi-cation, a 10% sales tax computation is specified using let expression. Extract functiongenerates the definition of the function tax from the let expression. Please note that thetype name ?means so-called any type. Any-type is not a legitimate type in the languagespecification of VDM-SL[2], but is a feature supported by many VDM-SL interpretersincluding VDMJ, Overture tool, VDMTools and ViennaTalk. To turn the specificationinto a legitimate one, a concrete type should be given for the any-type, i.e. real in this


particular case. It is in general difficult to infer a static type of an arbitrary expressionthough this case can be trivially inferred. This issue will be discussed in Section 5.

Extracting local names The let expression is a language construct to make local defi-nitions. Parameters of function or operation also have local scopes. Defining those localnames for expressions add contextual meaning to the expressions and thus are expectedto gain readability. Those names also help abstraction.

Extract let expression turns an expression exp into let localname = exp in local-name. Although the resulting form is trivial, this refactoring can be effectively appliedalong with refactoring operations for scoping described later. Extract local definitionadds a local definition with the specified subexpression in a let expression.

Extract parameter in function adds a parameter to the function definition and alsoadds the specified subexpression as an argument to the function for all applications ofthe function. The subexpression ported as the argument must be valid in the functionapplications. Also, the subexpression must be side-effect free because the argument isevaluated before the body of the let expression and the change of order of evaluationmay produce different results than the original. Extract parameter in operation similarlyadds a parameter to an operation definition and the extra argument to the callers.

�functionstax : real -> realtax(taxable) == taxable * 0.1� �

⇓ Extract parameter in function�functionstax : real * ? -> realtax(taxable, rate) == taxable * rate� �

Fig. 3. An example application of Extract parameter in function refactoring

Figure 3 illustrates an example application of Extract parameter in function. Themagic number 0.1 that appears in the original specification is turned into a parameterof the function so that the function gains it generality. Please note that the any-typeappears also in this refactoring. Although the literal value 0.1 is apparently of thereal type, it is not trivial in general to find an appropriate name of the type amongpossibly many alias names of the real type.

Using names Once a name is given to an expression, the occurrences of the same ex-pression may better use the name. The Use family of refactoring operations supportsthe user to deal with fragments that frequently appear in the specification. Use typereplaces an occurrence of the right-hand side of a type definition with the type name.


Use value does the same on a value definition. Use function replaces an expression thatmatches with the definition body of a function with a function application that resultsin the same expression. If a function has a precondition, Use function is not applica-ble to the function because the resulting function application may possibly violate theprecondition.

2.2 Refactoring for renaming

In the exploratory stage of specification, the specifier has limited knowledge of thesystem and the domain, often resulting in misleading names. Modification to the name,its location and its scope is also expected to be frequently performed to improve thequality of the specification as the knowledge is updated.

Changing the name Renaming is frequently applied among refactoring operationsin programming languages [5]. Renaming is not mere string replacement because thelexically same name can be semantically different names. A semi-automated renamingoperation that takes account of syntax and semantics of the language is desirable to con-duct safe and efficient renaming. The Rename family of refactorings supports renametypes, quote types, values, functions, state variables, operations, local definitions andparameters.

�functionspriceIncludingTax : real -> realpriceIncludingTax(displayPrice) ==let rate = 0.1, taxable = displayPricein displayPrice + taxable * rate� �

Narrow let expression ⇓ ⇑ Widen let expression�functionspriceIncludingTax : real -> realpriceIncludingTax(displayPrice) ==displayPrice +let rate = 0.1, taxable = displayPrice in taxable * rate� �

Fig. 4. An example application of Widen/Narrow let expression refactoring

Changing the scope Every name has its scope and choosing the right scope of a localname is considered generally good for readability and maintainability. The Widen fam-ily of refactorings widens the body of the in-clause in a let expression or a let statementinto its enclosing expression or statement. The Split family of refactoring separate one


local definition from an inner let expression to the outer let expression. If the let expres-sion is not nested, the Split refactoring will create another let expression that enclosesthe original let expression. The Narrow family of refactorings narrows the body into itssubexpression if and only if the locally defined names appear only in the subexpression.Figure 4 shows an example of widening and narrowing a let expression.

Changing the location VDM-SL supports a modular construction of a model. Thechoice of the module to define a type, value or function is important but sometimesnot trivial. The Move family of refactoring operation is to move a definition from onemodule to another and update the import/export declarations of each module.

2.3 Refactoring for unnaming

In the exploration to a better presentation of the model, a function definition or a valuedefinition may have little use or be found confusing. A let expression may introduceunnecessary syntactical complexity to the presentation. Removal of a name needs notonly just deleting its definition but also replace the name with its flat presentation. Semi-automated refactoring operation for these tasks is expected to save the user’s cognitiveworkload.

The Inline family of refactoring operations supports inlining both local and module-wide names. Inline type, Inline value and Inline function application are inverse of theircounterparts in the Use family, which replace the expression with the definition body.Inline let expression and Inline let statement replace the occurrences of the locally de-fined names with their definition bodies and remove the local definition in the let expres-sion or statement. The Remove family of refactorings deletes the declaration of nameswhen no reference to the names is left in the specification. A Remove refactoring can beeither automatically performed after an Inline refactoring or triggered independently.

3 Refactoring Support in ViennaTalk

ViennaTalk[6] is an IDE for the exploratory stage of VDM-SL specification built on topof the Pharo Smalltalk environment. ViennaTalk has a browser called VDM Browserdesigned to empower the use of animation in writing a specification, evaluating expres-sions, unit testing, UI prototyping, web API prototyping, and executable documenta-tion. While text-based source editors in conventional IDEs provide text-editing opera-tion at the source file tree, VDM Browser holds an interpreting process that containsthe source text and the binding information of state variables. We chose to implementanother browser named Refactoring Browser to implement semi-automated refactoringoperations. The design of the new Refactoring Browser is revised to hold an abstractsyntax tree (AST) because refactoring can more naturally be defined on AST rather thanthe source text. The source text displayed on the Refactoring Browser is promptly gen-erated from the AST when the user selects the definition to edit. The source text editedby the user is parsed and merged into the AST, and the source text will be disposed.

It is widely advised that refactoring should be performed with unit testing to ensurethat the functionality is not changed. Unit testing functionality was ported from VDM


Browser to Refactoring Browser with a modification to UI. While VDM Browser has aUnit Testing tab to display the result of unit tests behind the source pane, RefactoringBrowser places the unit testing pane aside the source pane so that the result of unittesting is always visible and has more chance to catch the user’s eyes.

History functions to record versions of the specification are added to Refactor-ing Browser. Semi-automated refactoring may cause unexpected modifications to thespecification, and the ability to roll back to previous versions is desirable. RefactoringBrowser records a mirror copy of AST into history at every execution of refactoringoperations and manual edits.

Fig. 5. Screenshot of Refactoring Browser

Figure 5 is a screenshot of the Refactoring Browser. The left top dropbox that reads“load” is a history selector that lists all versions of the AST with the names of refac-toring operations, and let the user choose a version to roll back. The list widget on theleft is called module list that shows all modules defined in the specification, e.g. Luhn,LuhnTest and UnitTesting in the Figure 5. The up centre list widget shows sec-tions in the selected module, and the right list enumerates top-level definitions in thesections. The source text of the selected top-level definition is generated from the ASTand shown in the lower text pane. In the screenshot, the function total in the functionssection of the Luhn module is edited.

A refactoring operation is a semi-automated manipulation of AST. Each refactoringis defined as a subclass of ViennaRefactoring class. When a user opens a context menu,each subclass of ViennaRefactoring is created and tested executability to the specified


AST node. All refactoring instances that returned true to the executability test will belisted on the context menu. In Figure 5, the keyword let highlighted in the source isselected by the user, and the context menu shows all applicable refactoring operationson the function.

3.1 Example: LUHN

�module Luhnexports alldefinitionstypesDigit = nat inv d == d < 10;

functionsluhn : seq1 of Digit -> Digitluhn(data) == total(data) * 9 mod 10;

total : seq of Digit -> nattotal(data) ==if data = []then 0else

(letmultipler = len data mod 2 + 1,product = hd data * multipler

intotal(tl data) +(if product < 10then productelse product mod 10 + 1))

measure slen;

slen : seq of Digit -> natslen(data) == len data;

end Luhn� �Fig. 6. The original source of the Luhn module

Luhn algorithm, also known as mod-10, is a checksum algorithm mainly used toverify a series of numbers and/or letters against accidental errors such as typos. Figure6 shows a part of the specification of the Luhn algorithm published at the Overture


website6 with modification. The lune function computes the check digit for the givensequence of digits. Although the algorithm is well-known and specification is alreadymature, the specification can be modified more informative by applying a series of semi-automated refactoring operations and a few manual edits on ViennaTalk’s RefactoringBrowser. Two improvements will be illustrated and explained in the rest of this section.

Improve the definition of total The first improvement is on the function total. Alet expression appears in the definition body of total. The subexpression total(tldata) is included in the let expression, but does not refer to any local definition of thelet expression. The Narrow let expression refactoring can be applied to adjust the scopeof the local definitions. The let expression will be refactored to the below.�total(tl data) +(letmultipler = len data mod 2 + 1,product = hd data * multipler

in(if product < 10 then product else product mod 10 + 1))� �The expression is now separated into the recursion part and the computation on each

element of data. Extracting a parameter datum from hd data in the let expressionand extract a function definition from the let expression will simplify the definition ofthe total function. The result of Extract local definition in let expression and Split letin order is shown below.�total(tl data) +(let datum = hd data inletmultipler = len data mod 2 + 1,product = datum * multipler

in(if product < 10 then product else product mod 10 + 1))� �

This time, the Refactoring Browser does not enable Extract function from let ex-pression because the reference to the local parameter data is left in the body of thelet expression. The local definition of multipler should be moved to the outer letexpression by Split let.�total(tl data) +(letdatum = hd data,multipler = len data mod 2 + 1,

6 https://www.overturetool.org/download/examples/VDMSL/LUHNSL/index.html

https://www.overturetool.org/download/examples/VDMSL/LUHNSL/index.html

https://www.overturetool.org/download/examples/VDMSL/LUHNSL/index.html


inlet product = datum * multipler in(if product < 10 then product else product mod 10 + 1))� �

The Refactoring Browser enables the user to apply Extract function from let ex-pression to the outer let expression. The following shows the resulting definition of thetotal function and the newly defined single function.�total : seq of Digit -> nattotal(data) ==if data = []then 0else(total(tl data) + single(hd data, len data mod 2 + 1));

single : ? * ? -> ?single(datum, multipler) ==let product = datum * multipler in(if product < 10 then product else product mod 10 + 1)� �

The type of the single function is not specified well, and the parameter multiplershould be constrained by adding a precondition. The definition of singlewill be man-ually edited into the below.�single : Digit * nat -> Digitsingle(datum, multipler) ==let product = datum * multipler in(if product < 10 then product else product mod 10 + 1)

pre multipler in set {1, 2}� �The precondition of single is important. Many explanation of this formula writes

that each digit in the 1st, 3rd, 5th, ... place from the right should be doubled. multiplershould be either 1, meaning not doubled, or 2 meaning doubled. The extraction of thefunction single created a place to formally document the constraint while the con-straint can be inferred from len data mod 2 + 1 in the original specification.

The refactored specification works the same as the original one, and asserts twoconstraints explicit: (1) the resulting value to be added is in the range of Digit type,and (2) the multipler should be either 1 or 2. The refactoring involved five semi-automated operations and one manual edit. The automated operations do not only savethe keyboard typing efforts but also prevent the user from breaking the semantics of theoriginal specification.

Generalising 10 as a base A literal value 10 appears in multiple places in the speci-fication. A constant value can be defined by Extract value refactoring.


�valuesbase = 10;

typesDigit = nat inv d == d < base;

functionsluhn : seq1 of Digit -> Digitluhn(data) == total(data) * 9 mod base;


total(tl data) + single(hd data, len data mod 2 + 1)measure slen;


single : Digit * nat -> Digitsingle(datum, multipler) ==let product = datum * multiplerin

(if product < basethen productelse product mod base + 1)

pre multipler in set {1, 2};end Luhn� �

A literal number 9 appears in the definition of luhn. The original expressiontotal(data) * 9 mod 10 is a short form of more explanatory expression (10- total(data) mod 10) mod 10, intending a single-digit value that will besummed up with the last digit of total(data) to make a natural number whose lastdigit is 0. The below is the result of manual editing to (10 - total(data) mod10) mod 10 and apply the Use value in all occurrences operation.�luhn : seq1 of Digit -> Digitluhn(data) == (base - total(data) mod base) mod base;� �

The resulting specification is shown in Figure 7. The specification gained generalityso that it can be easily extended to other variations of the Luhn algorithm to validate asequence of letters other than just digits.


�module Luhnexports alldefinitionsvaluesbase = 10;

typesDigit = nat inv d == d < base;

functionsluhn : seq1 of Digit -> Digitluhn(data) == (base - total(data) mod base) mod base;


total(tl data) + single(hd data, len data mod 2 + 1)measure slen;


single : Digit * nat -> Digitsingle(datum, multipler) ==let product = datum * multiplerin

(if product < basethen productelse product mod base + 1)

pre multipler in set {1, 2};

end Luhn� �Fig. 7. The refactored source of the Luhn module


4 Related Work

Pedersen and Mathiesen pioneered the application of refactoring to VDM-SL [7]. Theyidentified refactoring operations applicable to VDM-SL, including rename, add param-eter, convert function to operation, extract definition and remove operations, and builta Proof-of-Concept implementation on the Overture tool. Refactoring is discussed asmeans to address model smells and improve the maintainability of the model while itscounterpart in the programming languages is to address code smell in the field of soft-ware maintenance. This paper pays attention to the same technique, refactoring, fora different purpose. This paper shed light on refactoring as guidance for the specifierto learn the system and domain, which is more performed at the earlier phase of thespecification than the phase to improve the rigour and conciseness of the model.

5 Discussion and Concluding Remarks

Refactoring, in general, is often performed to improve readability and maintainability inboth programming languages and specification languages. Refactoring in programmingis widely practised in the maintenance of program sources to keep them readable andextendable. The objective of this research is to design a set of semi-automated refactor-ing operations that support the learning process in the exploratory stage of the specifi-cation rather than the maintenance phase of the specification. The question is whetherthe automated refactoring techniques widely used in programming is also useful inthe exploratory stage of the specification. The authors used the Refactoring Browserto modify several existing models including the Luhn algorithm and found the semi-automated refactoring operations safe and efficient. The semi-automated refactoringoperations make coarse-grained interaction with the user comparing the conventionaltext editing operations.

Each refactoring has a different scope and makes a different amount of changes tothe source text. The user’s expectation of the changes to be made by a refactoring oper-ation may differ from the actual one. The automated history mechanism implementedin the Refactoring Browser encourages the use of refactoring operations.

While the use of refactoring tools in the exploratory stage look promising, we ob-served difficulties to apply the semi-automated refactoring operations to VDM-SL. Onedifficulty is assertions, especially invariants and preconditions. Judging the applicabilityof a refactoring operation that replaces two expressions often involves the satisfiabilityof assertions. Even though an expression matches with the definition body of a function,it is not trivial to confirm that the arguments in the function application always satisfythe precondition of the function and the type invariants of the parameter types.

Automated inference of types on extracted identifiers is also problematic. For ex-ample, the type of constant value in the values section can be omitted while the exportsignatures of values require explicit typing. The refactoring operations implemented inthis research avoid this issue by using the any-type (?) which is not supported by thelanguage specification. To generate a formally valid fragment of specification, the user’smanual edit is required.

Multiple refactoring operations are often required to make an apparent improvementas seen in the Luhn example. Support for planning a series of refactoring operations


to achieve an expected achievement is required. History is a promising tool to showthe strategic view of the modification to the specification with a series of fine-grainedrefactoring operations applied so far. Further study to design a better interaction modelwith the combination of refactoring operations and history is needed.

Acknowledgments

The authors thank Nick Battle for providing the original specification of Luhn algorithmand supplementary documentation. We would thank the anonymous reviewers for theirvaluable comments and suggestions.

References

1. Fitzgerald, J., Larsen, P.G.: Modelling Systems – Practical Tools and Techniques in SoftwareDevelopment. Cambridge University Press, The Edinburgh Building, Cambridge CB2 2RU,UK, Second edn. (2009), ISBN 0-521-62348-0

2. Larsen, P.G., Lausdahl, K., Battle, N., Fitzgerald, J., Wolff, S., Sahara, S., Verhoef, M., Tran-Jørgensen, P.W.V., Oda, T.: VDM-10 Language Manual. Tech. Rep. TR-001, The OvertureInitiative, www.overturetool.org (April 2013)

3. Louridas, P.: Junit: unit testing and coiling in tandem. IEEE Software 22(4), 12–15 (2005)4. Mens, T., Tourwe, T.: A survey of software refactoring. IEEE Transactions on Software Engi-

neering 30(2), 126–139 (2004)5. Negara, S., Chen, N., Vakilian, M., Johnson, R.E., Dig, D.: A comparative study of manual

and automated refactorings. In: European Conference on Object-Oriented Programming. pp.552–576. Springer (2013)

6. Oda, T., Araki, K., Larsen, P.G.: A formal modeling tool for exploratory modeling in soft-ware development. IEICE Transactions on Information and Systems 100(6), 1210–1217(June 2017), https://www.jstage.jst.go.jp/article/transinf/E100.D/6/E100.D_2016FOP0003/_article

7. Pedersen, M.L., Mathiesen, P.K.: Refactoring Support for VDM-SL. Master’s thesis, AarhusUniversity, Department of Engineering (January 2017)

8. Roberts, D., Brant, J., Johnson, R.: A refactoring tool for smalltalk. Theory and Practice ofObject systems 3(4), 253–263 (1997)

https://www.jstage.jst.go.jp/article/transinf/E100.D/6/E100.D_2016FOP0003/_article

https://www.jstage.jst.go.jp/article/transinf/E100.D/6/E100.D_2016FOP0003/_article

Extending the Formal Security Analysis of the HUBCAPsandbox

Tomas Kulik1, Prasad Talasila1, Pietro Greco2, Giuseppe Veneziano2, AngeloMarguglio2, Lorenzo Franco Sutton2, Peter Gorm Larsen1, and Hugo Daniel Macedo1

1 DIGIT, Department of Engineering, Aarhus University, Denmark2 ENGIT, Italy

Abstract. Cloud computing thrives in the modern distributed and model-baseddesign area, where many models and tools are domain specific and seldom in-teroperable. As shown during the HUBCAP project, clouds thrive because theyprovides several virtual resources on demand, thus enabling the development ofcustomized workspaces of tools that operate on diverse hardware and softwareexecution environments. Yet cloud resources are public and shared, thus restrict-ing access to private tools and achieving proper isolation of the user. Customizedworkspaces require the implementation of a sandbox, which ensures the protec-tion of intellectual property rights, flexible licensing/revenue models, and clearsharing policies for the platform assets. The security of a cloud platform likeHUBCAP relies on the properties of its sandbox implementation, and we appliedformal methods techniques to establish theoretical approximations of such prop-erties. In this paper, we report on the extension of a previous study to analyze useraccess and sharing policies of sandboxes provided by the HUBCAP SandboxingMiddleware. We provide an account of the extensions/refinements to the previousformal specification and specify the semantics and properties of the “private tool”feature recently added to the middleware.

Keywords: Formal analysis, Security, Sandbox, Model-based design.

1 Introduction

The HUBCAP project provides a collaboration platform facilitating model-based de-velopment of Cyber-Physical Systems [14]. The platform provides resources to virtu-alization as a vehicle to deliver tools [1,21] and models to its users via a typical webinterface [12,13]. Tools are understood as software necessary to develop/explore/exe-cute a model. A tool is installed on top of an operating system, utilizing the dependen-cies (runtime, libraries,...) and functionality (scripts, services, connections,...) requiredto run it.

The platform features a social collaboration web portal and a so called sandboxfeature: an isolated temporary set of virtual machines (a customized workspace) wherepotential users can try out (or provide) HUBCAP assets (models and tools), using a setof tools and models without any financial charges or the need to install new software inthe user’s machine. The sandbox approach brings several implementation challenges,especially in terms of security. For example, protecting the platform from malware [15],

Extending the Formal Security Analysis of the HUBCAP sandbox 37

and ensuring the protection of intellectual property of tool and model owners requirestandard access rights policies. Access rights including the user roles and profiles havebeen implemented in the HUBCAP Sandboxing Middleware (HSM) and have beenanalyzed in a previous study [8]. In this work we extend the security analysis to coverthe new features of the HSM such as updates for tools located within the platform, theability to make a tool private to restrict access to only its owner, who can share it withina Sandbox with users authorized by himself and increasing isolation of the sandboxfrom other parts of the platform.

To study the security of the updated HSM we have utilized Vienna DevelopmentMethod (VDM) to specify the interactions between the platform users and a formalspecification of the HSM functionality. Furthermore, we specify the intra-sandbox in-teractions between the tools and models and represent the ability of the sandbox to actas a separate and isolated entity from the rest of the platform. VDM has been selectedas a tool for analysis as it provides a formal specification system with the ability toformally analyze specific scenarios and as a formal method it has been used in assertingsecurity in the midst of other tools [7]. In concrete terms, we have utilized the ISO stan-dardized VDM-SL dialect, that has been certified by the International StandardizationOrganization [9,16], providing contract based modeling. The VDM modeling systemuses logic, specifically use of pre-conditions, post-conditions and invariants to expressproperties that need to be satisfied within the model for specific scenarios. The VDMmodeling system provides several tools to assist with the modeling and analysis efforts,specifically the Overture/VDM tool support [10,11] as well as plugins for popular texteditors such as Visual Studio Code [18] and Emacs [20]. The VDM models have pre-cise semantics and provide a good basis for communication of findings to the HUBCAPplatform implementation team.

Our work is an ongoing security analysis of an evolving platform, hence we areutilizing close collaboration with the implementation team to ensure that newly imple-mented and planned features are analyzed for potential security issues. As the platformis evolving rapidly we are not able to obtain full coverage of security properties withinthe platform during the development phase, however it is our consideration that our ap-proach would provide a significant coverage once the platform enters more stable phasein its lifecycle. So far the analysis has provided feedback points to the implementationteam, what led to the refinement of one feature description.

The rest of the paper is organized as follows: Section 2 first describes the overallHUBCAP sandboxing middleware architecture. Section 3 contains the access rights fordifferent actors to the sandbox environment. Afterwards, Section 4 presents a VDMmodel of the security constraints for the sandbox followed by an analysis of this modelin Section 5. Finally, Section 6 provides pointers to related work while Section 7 pro-vides the concluding remarks.

2 Architecture

In addition to a social collaboration web portal [12], the HUBCAP Sandboxing Mid-dleware (HSM) provides repositories of Virtual Machines (VMs), also addressed to asservers within this paper (a combination of a Tool and an OS), from which users can

38-2ex Kulik et al.

Fig. 1. The HUBCAP Sandboxing Middleware architecture (adapted from [12])

draw to compose their sandboxes, fully accessible via a browser web page to users,who experience MBD tools and models in a sandbox. A sandbox is implemented asan isolated set of VMs (each one running a CPS tool) that interact with each othersharing a virtual dedicated subnet and a dedicated Network File System (NFS) storageservice. No interaction is permitted between the VMs belonging to different sandboxes.The sandbox capability integrated with the web-platform is therefore a sort of privatecloud service provider plus the middleware to manage and mediate the access to thosecloud services. In addition, as many cloud service providers offer the capability to se-lect a combination of hardware and operating systems, the HSM allows users to selecta combination of OS environments, tools, and models to run an experiment using theHUBCAP sandbox feature.

The sandbox service is outlined in Figure 1. The web-platform is enhanced with abroker component (labeled as Sandboxes Broker in the figure), which hosts a web ap-plication mediating the access of different users (Client 1 and Client 2) to the sandboxesthey created (Sandbox 1 and Sandbox 2 respectively). All the users will use an Internetbrowser to access the tools in the sandbox and all the interactions are mediated by thebroker.

The Sandbox Broker has access to the catalogs of different models, tools, and pre-configured OSs that are available, so an end user can simply pick a valid combinationto request a sandbox. In addition to those catalogs, the Sandbox Broker keeps all theuser information necessary to allow the creation of new sandboxes.

The operation of user requests and the sandboxing logic is provided by the Sandbox-ing Kernel, which is a component that interacts with the system Hypervisor to launchthe different constituents of a sandbox, namely:

– NFS - Network File System providing storage in the form of shared folders wheremodel files and tool outputs are placed.

– VLANs - Virtual networks restricting the communications of the VMs inside a sand-box to the set of VMs composing it and those only.

– VOS - Virtual machines running the OSs supporting a tool.


– Tools - In the HUBCAP Sandboxing Middleware, a Tool is a VM obtained byinstalling CPS software on a base OS VM offered by the HSM itself.

– Models - A mathematical/formal description of a component, stored as compressedarchives containing a set of folders and files.

The operation relies on a database of metadata about the different sandboxes (theSandboxes Metadata component in the figure). This component stores and keeps trackof the sandboxes’ states (running, suspended, . . . ) and user ownership of the resources.It is worth highlighting that the Kernel has direct network connections to the Sand-boxes’ VLANs. We base our model on the building blocks of the sandboxing platformpresented in this section, focusing on the components necessary for analysis of theclient access based on roles and profiles, while abstracting away details such as VLANaddresses, specifics of the operating systems, file storage and tool properties.

3 Access rights and hierarchy

The HSM has evolved with features such as tool versioning and privacy settings fortools, therefore new versions of existing tools may be uploaded by the users that haveinitially provided these tools as well as the possibility to set these tools as private. Thetool providers can always mark their tool(s) as public and hence make them availableto all users of the sandbox platform. It is important to note that the HSM could containmore tools than those available within the sandbox, however for the scope of this paperwe only consider the tools available for sandboxing.

Table 1. Overview of HSM user profiles and roles

Feature Provider Provider Consumer ConsumerOwner Guest Owner Guest

Access to remote viewer X X X XUpload archive X X

Download archive X XInvite guests X X

Destroy sandbox X XSelect tool X X

Select model X XSelect operating system X

Save tool XUpload new model X

Delete repository item X (own)

Different features of the platform (sandbox and the underlying platform) are gov-erned by a table of access roles and profiles as shown in Table 1. As can be seen in thetable, the provider profile represents users that could provide specific tools / models tothe HSM. Both the providers and consumers can utilize the operating systems, public

40-2ex Kulik et al.

tools and models. In addition, providers can also utilize the private tools they own onthe HSM.

In the context of sandbox instantiation and usage, the HSM users have two roles. Auser creating a sandbox dons the role of owner and any user invited to use that sandboxdons the role of guest. The owner, if a provider, selects meaningful subset of availableoperating systems, tools, models and then creates a sandbox. In addition, owner can alsoinvite other users to become guests in a sandbox. This invitation process is an integralpart of sandbox instantiation. Once a tool — whether public or private — is instantiatedin a sandbox, it’s available to all users who have access to that sandbox. Thus both thesandbox owner and guest have access to all the tools available in a sandbox. One usercan be the owner of many of sandboxes and can also be a guest in sandboxes owned bythe other users, yet one user can only own a single Sandbox at a time.

4 Formal access model

The model of the HUBCAP Sandboxing Middleware (HSM) consists of several sub-components, specifically (1) Client, (2) Broker, (3) Gateway, (4) Sandbox, (5) Server,(6) Tool and (7) System. Every subcomponent represents a part of the HSM and ismodeled as a VDM-SL file with different amount of detail. The model utilizes anotherVDM-SL file capturing the test traces needed for the analysis, namely the Test VDM-SL file. The model is based upon a flat structure where the a single default module isutilized for tracking of the state of all of the subcomponents. In this study the model isutilized to determine the correctness of the modeled system according to the updatedtable of roles and profiles. This sections provides a detailed overview of the modeledsubcomponents with the goal to specifically pinpoint the updated parts of the system.

4.1 Client

The client is an entity responsible for making calls to the sandbox. It could be un-derstood as an application that acts upon a behalf of a sandbox user. The HSM is amulti-user system, hence multiple clients can access the different sandboxes and sys-tem features. Since the system can handle multiple clients, each client carries a uniqueidentity as ClientId=nat. In a current iteration of the HSM, the client can selectmultiple tools and models from a repository, i.e. a library of preexisting items hostedon the system and launch a new sandbox that distributes these tools across as manynewly spawned virtual machines as there are selected tools. In the current HSM anal-ysis model, the models are not associated with a specific tool, what is sufficient as thefunctionality expressed within the table of roles and profiles does not define interactionbetween a model and a tool. The selection of tools and models and launching of a newsandbox with these entities is shown in Listing 1.1. The client can further destroy anexisting sandbox, connect and disconnect from a sandbox and upload repository items.

SelectToolsFromRepository: ClientId * SelectedTools ==> ()SelectToolsFromRepository(cId, tIds) ==clientst.selectedTools := SelectTools(tIds, cId)


pre cId in set validClients;

SelectModelsFromRepository: ClientId * SelectedModels ==> ()SelectModelsFromRepository(cId, mIds) ==clientst.selectedModels := SelectModels(mIds, cId)


LaunchNewSandbox: ClientId ==> ()LaunchNewSandbox(cId) ==StartNewSandbox(cId, clientst.selectedTools, clientst.

selectedModels)pre cId in set validClients;

Listing 1.1. Selection of sandbox entities and launching of a sandbox

In this iteration of the HSM the client can not only upload a tool once to the reposi-tory, but can keep the tool up to date by updating the tool with potential new versions.Another option is to upload an archive of files to a specific sandbox and download datagenerated by a tool running on a specific virtual machine within a sandbox that theclient has access to. The uploading of a tool, updating of a tool, uploading of an archiveand downloading of an archive is shown in Listing 1.2. It is important to note thatthe pre-condition pre cId in set validClients checks that the client is recog-nized as valid by the system. After the data download the post-condition post card

clientst.downloadedData = card clientst~.downloadedData + 1 checksthat the client has received the downloaded data. While this post-condition would notallow for a repeated download of the same data, it is sufficient within our analysis.

UploadTool : ClientId * OSId * Version * Private * OsOnly ==> ()UploadTool(cId, oId, v, p, oo) == SaveTool(cId, oId, v, p, oo)pre cId in set validClients and oId in set brokerst.validOSs;

UpdateTool : ClientId * ToolId ==> ()UpdateTool(cId, tId) == UploadNewToolVersion(cId, tId)pre cId in set validClients;

UploadArchive : ClientId * SandboxId * token ==> ()UploadArchive(cId, sbId, arch) ==UploadArchiveToSandbox(cId, sbId, arch)


DownloadArchive : ClientId * ServerId * SandboxId ==> ()DownloadArchive(cId, sId, sbId) ==let d = DownloadArchiveFromServer(cId, sId, sbId) inclientst.downloadedData := clientst.downloadedData union {d}

pre cId in set validClientspost card clientst.downloadedData =

card clientst~.downloadedData + 1;

42-2ex Kulik et al.

Listing 1.2. Upload, update and download of items

4.2 Broker

The broker is the part of the HSM that handles the calls from the client and providesreplies from the system to the client. The broker is a single entity hence it does notneed a specific identity. The broker state is responsible for handling of client/sandboxownership relation, for example a relation Guests = map ClientId to set ofSandboxId specifies that the specific clients are guests to specific sandboxes. The bro-ker further specifies other relations such as ownership of tools, assignments of tools tosandboxes and others. One very important function of the broker is to start new sand-boxes. This requires starting of new virtual machines under the sandbox. The operationfor launching of new sandboxes is shown in Listing 1.3. This operation also calls Build-NewServers operation responsible for spinning out new virtual machines under the sys-tem. Another operation responsible for getting the operating systems out of a specifictool used when launching a sandbox is GetToolOSs. This is necessary in order to keeptrack of which operating systems are used under a specific sandbox. These operationsare described in more detail in Section 4.7. The pre-condition on the operation statesthat the sandbox can only contain valid tools and valid models, i.e. items that are recog-nized by the sandboxing system. The post-condition states that new sandbox is addedupon completion of the operation, the calling client is added as an owner of the newlyspawned sandbox.

StartNewSandbox : ClientId * set of ToolId * set of ModelId ==>SandboxId

StartNewSandbox(cId, tId, mId) ==let sId = GenerateNewSandboxId()in(brokerst.sandboxOSs := brokerst.sandboxOSs munion {sId |->

GetToolOSs(GetToolsByToolIds(tId, brokerst.validTools))};brokerst.sandboxModels := brokerst.sandboxModels munion {

sId |-> mId};(brokerst.sandboxTools := brokerst.sandboxTools munion {

sId |-> tId};systemSandboxes := systemSandboxes munion {sId |->

mk_Sandbox(sId, BuildNewServers(tId,mk_token(nil)),{})});

if cId in set dom brokerst.owners thenbrokerst.owners(cId) := brokerst.owners(cId) union {sId}

elsebrokerst.owners := brokerst.owners munion {cId |-> {sId}};

return sId)pre tId <> {} and tId subset dom brokerst.validTools

and (mId <> {} => mId subset brokerst.validModels)post card dom systemSandboxes = card dom systemSandboxes~ + 1


and cId in set dom brokerst.ownersand RESULT in set dom systemSandboxes;

Listing 1.3. Launching of a new sandbox

The broker is an integral part of the system often responsible for handling operationcalls according to the table of roles and profiles, encoded as pre and post conditions.

4.3 Gateway

The gateway is a connectivity component. It provides connections between clients andvirtual machines within the sandbox. The gateway holds a state of connections, whileit is the broker that assigns these connections. The gateway is defined as shown inListing 1.4 with the connections handling as shown in Listing 1.5. All of the sandboxconnections and disconnections are recorded within the gateway state. The operationcalled during connections and disconnections is the UpdateConnections that is definedwithin system and is described in detail in Section 4.7. Within the Listing 1.5 the pre-condition checks that the client actually has rights as defined within the table of rolesand profiles and the client is either an owner of the sandbox it is connecting to, or aguest that was invited towards this sandbox.

GateWaySt ::connectedClients : ConnectedClientsconnectedServers : ConnectedServers;

ConnectedClients = set of ClientId;ConnectedServers = set of ServerId;

Listing 1.4. Gateway state

BrokerInitiateSandboxAccess: ClientId * SandboxId ==> boolBrokerInitiateSandboxAccess(cId, sId) ==let sandboxes = GetSystemSandboxes(),

servers = dunion {dom s.sandboxServers| s in set rng sandboxes& s.sandboxId = sId}

in(for all x in set servers do

UpdateConnections(cId, x, sId, true);return true)-- used to simulate error handling

pre not ClientIsNull(cId,brokerst.providers,brokerst.consumers,brokerst.owners, brokerst.guests)

and ((cId in set dom brokerst.owners and sId in set brokerst.owners(cId))

or (cId in set dom brokerst.guests and sId in set brokerst.guests(cId)));

Listing 1.5. Sandbox connection initiation

44-2ex Kulik et al.

4.4 Sandbox

The sandbox is considered to be an isolated entity, a container that contains severalvirtual machines hosting tools and models that user can interact with remotely. Eachsandbox isolates its data from other sandboxes and access to it is governed by the tableof roles and profiles. The sandbox itself can receive data remotely via a client once itis launched, this data is only present within a selected sandbox and cannot be shared toother sandboxes. An important notion within the sandboxing system is the ownershipmodel, i.e. the user could be an owner of a sandbox and be able to invite other as guestsand also destroy the sandbox. The sandbox is defined as shown in Listing 1.6.

SandboxId = nat;SandboxServers = map ServerId to Server;UploadedData = set of token;Sandbox::sandboxId : SandboxIdsandboxServers : SandboxServersuploadedData : UploadedData

Listing 1.6. Sandbox definition

Each of the sandboxes has a unique identity, defines virtual machines that belong tothis sandbox and has a set of data that has been uploaded from a client (starting as anempty set). Since each sandbox must have an unique identity it is necessary to generatenew identity each time a new sandbox is launched. This is handled by functions and op-erations shown in Listing 1.7. This shows that the identity of sandboxes is incremental.

Max: SystemSandboxes -> natMax(ss) ==if ss = {|->} then 0else let max in set dom ss be st

forall d in set dom ss & d <= maxin

max;

GenerateNewSandboxId: () ==> natGenerateNewSandboxId() == return Max(systemSandboxes) + 1;

Listing 1.7. Sandbox identity

The current iteration of the model allows for sandboxes consisting of multiple vir-tual machines, increasing the complexity from all previous attempts to model the sys-tem.

4.5 Server

The server is considered to be a virtual machine within the system used to host a specifictool. Each server within the system has a unique identity where the identity is generated


similar to the sandbox identities, i.e. they are incremental. Within the system each serverneeds to be a part of a sandbox in order for a connection to be established via a client.The server definition is shown in Listing 1.8. Each server defines what tool it carriesand potentially there could be a piece of abstract data that has been generated by theinteraction of a tool and a model. This data can then be obtained by a client and whenthe server is started is starts empty as BuildNewServers(tId,mk_token(nil)).Within the model this is used for analysis of clients ability to download data from aserver according to the table of roles and profiles.

ServerId = nat;Data = token;Server::serverId : ServerIdtoolId : ToolIddata : Data

Listing 1.8. Server definition

4.6 Tool

The tool is the newest addition to the modeling effort. The tool is considered to bean application that is used to interact with specific models and is deployed within aspecific virtual machine under a sandbox. In some cases the tool does not contain anapplication but only an operating system. The tool can further be set as private, meaningthat only the user that provided it to the repository of tools could select it and deployit within a sandbox. The tool further defines a version as in this iteration of the systemit is possible to update existing tools rather than keep all of the different tool versionswithin the platform. The tool is defined as shown in Listing 1.9. The operating systemof the tool is specified by the identity of the operating system.

Version = nat;Private = bool;OsOnly = bool;Tool::osId : OSIdversion : Versionprivate : PrivateosOnly : OsOnly

Listing 1.9. Tool definition

4.7 System

Finally the system could be understood as an abstraction encompassing the core systemfunctionality. For example the system defines operations for generation of new iden-

46-2ex Kulik et al.

tities, starting of new virtual machines and assigning connections to the gateway. Theoperation utilized to create new servers is shown in Listing 1.10. Here a new virtual ma-chine is created with its properties when required due to a new sandbox being launched.This operation creates a new server for every tool selected by the client.

BuildNewServers: set of ToolId * Data ==> map ServerId to ServerBuildNewServers(tId, d) ==(dcl servers : map ServerId to Server := {|->};for all t in set tId dolet newServerId = GenerateNewServerId()in(systemServers:= systemServers union {newServerId};servers(newServerId) := mk_Server(newServerId, t, d);

);return servers;);

Listing 1.10. New server (virtual machine) creation

The system furthermore reads out the operating system identities out of the spe-cific tools, information that is used to setup relation between sandboxes and operatingsystems. This function is shown in Listing 1.11.

GetToolOSs: set of Tool -> set of OSIdGetToolOSs(tools) == {t.osId | t in set tools};

Listing 1.11. Get operating system identities

Finally an important operation is an operation responsible for changes in connec-tions, updating the state of the gateway. This operation is shown in Listing 1.12. Bothconnections and disconnections are handled by this operation.

UpdateConnections: ClientId * ServerId * SandboxId * bool ==> ()UpdateConnections(cId, sId, sbId, connect)==if connectthen (if cId in set dom gatewayConnections

then atomic(gatewayConnections(cId):= gatewayConnections(cId)

union {sId};gatewayConnectionsSandbox(cId) :=

gatewayConnectionsSandbox(cId) union {sbId};brokerst.activeSandboxes := brokerst.

activeSandboxes union {sbId})else atomic(gatewayConnections := gatewayConnections munion {

cId|-> {sId}};


gatewayConnectionsSandbox :=gatewayConnectionsSandbox munion {cId |-> {sbId}};

brokerst.activeSandboxes := brokerst.activeSandboxes union {sbId}))

else atomic(gatewayConnections(cId) := gatewayConnections(cId) \ {

sId};gatewayConnectionsSandbox(cId) :=

gatewayConnectionsSandbox(cId) \ {sbId};brokerst.activeSandboxes := brokerst.activeSandboxes \ {

sbId})pre (connect=false) => cId in set dom gatewayConnections;

Listing 1.12. Update the gateway connections

The file System.vdmsl also holds the initialization of the model building up initialstates of different entities and holds an invariant dom ss.gatewayConnections =

dom ss.gatewayConnectionsSandbox, specifying that all of the sandbox connec-tions and server connections are established using the same clients.

5 Formal analysis

This section specifies the different traces that have been utilized within the analysis ofthe system functionality. Several new features have been analyzed such as launchingof sandboxes containing multiple servers, updating of an existing tool and download ofdata from a specific virtual machine. The traces have been analyzed using the combi-natorial testing feature of the VDM as it allowed for expansion of tests but utilizationof simple parameters. The goal of the analysis is to determine the consistency of thetable of roles and profiles as well as compare the results with interaction with the actualsandboxing system.

The scenarios considered within this paper have been selected to analyze the newfeatures of the sandboxing system as well as updates to the model.

The first trace shown in this paper has been used to analyze the private tool feature.This trace is shown in Listing 1.13. In this trace a client uploads a tool that is set toprivate. The trace uses the combinatorial testing to an extent where it tries to select thetool from a repository on behalf of two clients. The first client is the tool owner, hencethe result of the trace is a success, while the second client is not the owner of the privatetool and the result of this trace is inconclusive as it violates the outermost pre-conditionon the SelectToolsFromRepository operation of the client.

CheckPrivateToolAccess:SetupClients(1);SetupClients(2);SetupProviders(1);SetupProviders(2);

48-2ex Kulik et al.

SetupOSs(1);UploadTool(1, 1, 1, true, false);let clientId in set {1,2}in(SelectToolsFromRepository(clientId, {1}););

Listing 1.13. Trace for checking of the private tool functionality

Another interesting action shown within this paper is the ability of the client toupload an archive (i.e. a compressed archive of files) to a sandbox. The trace used toanalyze this situation is shown in Listing 1.14. First the client selects tools and models,then launches a new sandbox and finally uploads an archive to the sandbox. Within thetest the upload of an archive happens without any issue, what is reflected within theimplemented platform.

UploadArchiveTrace:SetupClients(1);SetupProviders(1);SetupTools(1,1,1,false,false);SetupModels(1);let clientId in set {1}in(SelectToolsFromRepository(clientId, {1});SelectModelsFromRepository(clientId, {1});LaunchNewSandbox(clientId);UploadArchive(clientId, 1, mk_token(1)););

Listing 1.14. Trace of upload of archive to a sandbox

The analysis had utilized several more traces, in order to provide high feature cov-erage, these have been omitted from this paper due to space considerations. These arehowever publicly accessible via [19].

6 Related work

The Access Control (AC) defines the limits of users actions and transactions of an infor-mation system [3,5,17]. The broad guidelines of access control are applicable to singlenode centralized information systems like the current implementation of HSM. Theprovider / consumer user profiles and the sandbox owner / guest roles must have well-defined access control policies to the HSM functionality. The formal model and thesandbox implementation consider the standard-compliant access control policies. Thelatest NIST SP 800-210 standard provides recommendations on access control guide-lines for cloud computing services [4]. The formal model and HUBCAP implementa-tion of the future sandbox environments will incorporate some of the recommendationsgiven in the NIST standard.

The Infrastructure as a Service (IaaS), Platform as a Service (PaaS) and Softwareas a Service (SaaS) are three popular cloud computing service models [6]. The HSM


provides PaaS to its users. The HSM platform manager provides some functionalities tosupport providers in protecting the intellectual property (IP) of their tools. For example,the HSM prevents a provider from (re)saving a tool owned by another provider. As hasbeen previously observed, software licensing and IP protection are a crucial stumblingblock in the migration of desktop software to native cloud computing software [2].Consequently, as in other PaaS cloud environments, the HSM users currently face thesame licensing issues.

7 Conclusion

This paper reports on the extension of the security analysis of a sandboxing platformdeveloped and operated as part of a project aimed at collaborative use of model baseddesign. We created a model of the system, considering features provided within a tableof roles and profiles by the implementation team. To model this feature set we haveconsidered the mentioned actions and modeled the roles and profiles as VDM pre andpost-conditions. We have then analysed several specific scenarios within the model byuse of the combinatorial testing. The analysis has in total considered 19 tests and couldbe carried out in about 10 seconds.

Extensions of our results should be made to allow the integration of the modelchecks with the implementation. The different teams involved in the security and im-plementation tasks closely cooperate and share access to the formal specification andthe implementation, thus a potential improvement is the addition of automation to thedevelopment cycle. This work has proven valuable to the platform implementation andhence we intend to continue analyzing future updates to the platform using VDM.

Acknowledgements. The work presented here is partially supported by the HUBCAPInnovation Action funded by the European Commission’s Horizon 2020 Programmeunder Grant Agreement 872698. We would also like to express our thanks to the anony-mous reviewers.

References

1. Badicu, A., Iordache, G., Suciu, G., Macedo, H.D., Sassanelli, C., Terzi, S., Larsen, P.G.:Deploying the Smart Energy Tool for Investment Simulation inside the HUBCAP Sandbox.9th International Workshop on Simulation for Energy, Sustainable Development and Envi-ronment pp. 18–26. https://doi.org/https://doi.org/10.46354/i3m.2021.sesde.003

2. Fox, A., Griffith, R., Joseph, A., Katz, R., Konwinski, A., Lee, G., Patterson, D., Rabkin,A., Stoica, I., et al.: Above the clouds: A berkeley view of cloud computing. Dept. ElectricalEng. and Comput. Sciences, University of California, Berkeley, Rep. UCB/EECS 28(13),2009 (2009)

3. Hu, V., Ferraiolo, D., Kuhn, R., Schnitzer, A., Sandlin, K., Scarfone, K.: Sp 800-162. guideto attribute based access control (abac) definitions and considerations. Nat’l Inst. Standardsand Technology pp. 800–162 (2014)

4. Hu, V.C., Iorga, M., Bao, W., Li, A., Li, Q., Gouglidis, A., et al.: General access controlguidance for cloud systems. NIST Special Publication 800, 210 (2020)

https://doi.org/https://doi.org/10.46354/i3m.2021.sesde.003

50-2ex Kulik et al.

5. Hu, V.C., Kuhn, D.R., Ferraiolo, D.F., Voas, J.: Attribute-based access control. Computer48(2), 85–88 (2015)

6. Hwang, K., Dongarra, J., Fox, G.C.: Distributed and cloud computing: from parallel process-ing to the internet of things. Morgan kaufmann (2013)

7. Kulik, T., Dongol, B., Larsen, P.G., Macedo, H.D., Schneider, S., Tran-Jørgensen, P.W.V.,Woodcock, J.: A survey of practical formal methods for security (2021)

8. Kulik, T., Macedo, H.D., Talasila, P., Larsen, P.G.: Modelling the HUBCAP Sandbox Ar-chitecture In VDM – a Study In Security. In: Fitzgerald, J.S., Oda, T., Macedo, H.D. (eds.)Proceedings of the 18th International Overture Workshop. pp. 20–34 (December 2020)

9. Larsen, P.G., Hansen, B.S., Brunn, H., Plat, N., Toetenel, H., Andrews, D.J., Dawes, J.,Parkin, G., et al.: Information technology – Programming languages, their environments andsystem software interfaces – Vienna Development Method – Specification Language – Part1: Base language (December 1996)

10. Larsen, P.G.: Ten Years of Historical Development: “Bootstrapping” VDMTools. Journal ofUniversal Computer Science 7(8), 692–709 (2001)

11. Larsen, P.G., Battle, N., Ferreira, M., Fitzgerald, J., Lausdahl, K., Verhoef, M.: The Over-ture Initiative – Integrating Tools for VDM. SIGSOFT Softw. Eng. Notes 35(1), 1–6(January 2010). https://doi.org/10.1145/1668862.1668864, http://doi.acm.org/10.1145/1668862.1668864

12. Larsen, P.G., Macedo, H.D., Fitzgerald, J., Pfeifer, H., Benedikt, M., Tonetta, S., Marguglio,A., Gusmeroli, S., Jr., G.S.: A Cloud-based Collaboration Platform for Model-based Designof Cyber-Physical Systems. pp. 263–270. INSTICC, Proceedings of the 10th InternationalConference on Simulation and Modeling Methodologies, Technologies and Applications -Volume 1: SIMULTECH (July 2020). https://doi.org/10.5220/0009892802630270

13. Larsen, P.G., Soulioti, G., Macedo, H.D., Alifragkis, V., Fitzgerald, J., Livanos, N., Pfeifer,H., Pasquinelli, M., Benedict, M., Thule, C., Tonetta, S., Stritzelberger, B., Marguglio, A.,Sutton, L.F., Obstbaum, M., Gusmeroli, S., Beutenmüller, F., Jr., G.S., Wijnands, Q., Talasila,P.: Enabling combining models and tools in an online mbse collaboration platform. In: ModelBased Space Systems and Software Engineering (MBSE2020). ESA-ESTEC, Noordwijk,The Netherlands (September 2020)

14. Macedo, H.D., Sassanelli, C., Larsen, P.G., Terzi, S.: Facilitating model-based design ofcyber-manufacturing systems. To appear in the 54th CIRP Conference on ManufacturingSystems (2021)

15. Macedo, H.D., Touili, T.: Mining malware specifications through static reachability anal-ysis. In: European Symposium on Research in Computer Security. pp. 517–535. Springer,Springer Berlin Heidelberg, Berlin, Heidelberg (2013)

16. Plat, N., Larsen, P.G.: An Overview of the ISO/VDM-SL Standard. Sigplan Notices 27(8),76–82 (August 1992)

17. Radack, S.M.: Minimum security requirements for federal information and information sys-tems: Federal information processing standard (fips) 200 approved by the secretary of com-merce (2006)

18. Rask, J.K., Madsen, F.P., Battle, N., Macedo, H.D., Larsen, P.G.: Visual Studio Code VDMSupport. John Fitzgerald, Tomohiro Oda, and Hugo Daniel Macedo (Editors) p. 35 (2021)

19. Tomas Kulik, Prasad Talasila, Pietro Greco, Giuseppe Veneziano, Angelo Marguglio,Lorenzo Franco Sutton, Peter Gorm Larsen, Hugo Daniel Macedo: VDM model for HUB-CAP security analysis. https://github.com/kuliktomas/VDM-secAnalysis(2021)

20. Tran-Jørgensen, P.W., Kulik, T.: Migrating Overture to a different IDE. In: Proceedings ofthe 17th Overture Workshop. p. 32 (2019)

21. Weiß, G.B., Pietraroia, D., Sassanelli, C., Macedo, H.D.: Manufacturing process simulationin a hybrid cloud setup. Submitted to IN4PL 2021

https://doi.org/10.1145/1668862.1668864

http://doi.acm.org/10.1145/1668862.1668864

http://doi.acm.org/10.1145/1668862.1668864

https://doi.org/10.5220/0009892802630270

https://github.com/kuliktomas/VDM-secAnalysis

Modelling an Injection Moulding Machine using theVienna Development Method

Till Böttjer, Michael Sandberg, Peter Gorm Larsen, and Hugo Daniel Macedo

DIGIT, Aarhus University, Department of Electrical and Computer Engineering, Åbogade 34,8200 Aarhus N, Denmark

{till.boettjer,ms,pgl,hdm}@ece.au.dk

Abstract. The injection moulding machine has changed the plastics manufac-turing industry. Its design and control evolved from a mechatronics artefact or-chestrating solenoids to a smart cyber-physical system, where the control relieson process modelling and computation. This evolution took place in an industrialcontext, thus designs, implementations and models of the machine and its con-troller are usually not available to the academic and public domains. In this paper,we report on the first model of a controller for an injection moulding machine re-sulting from the application of the Vienna Development Method. The work isdone in the context of a larger research project on Digital Twins in the manu-facturing domain, and our model, being executable, supplements the academicliterature with an open design of the machine process, quality and safety control.Our preliminary findings confirm that the model fits its purpose, it adequately or-chestrates a continuous time model of the process developed in MATLAB, and itmay benefit both the community developing injection moulding machine models,the community interested in formal modelling, and researchers in the area of Dig-ital Twins in manufacturing. We expect this model to be extended in the futureand used in the training of future engineers.

Keywords: model-based design, injection moulding machine, Digital Twins, co-simulation.

1 Introduction

Since its invention in the late 19th century, the Injection Moulding Machine (IMM)has changed the plastics manufacturing industry. At the same time, the IMM design hasevolved from a purely mechanical process to a full cyber-physical system [1] that utilisea high degree of computerisation to add smart features to the process. The developmentof digital models is an essential step in the process of adding smart features to a cyber-physical system. The models may serve various purposes, e.g., models may be exercisedin sync with the machine operations to detect operation anomalies, a typical DigitalTwin setting or used to specify the behaviour of its components and estimate the systemresponse by simulation. One of the first modelling tasks in a new project is to elicit therequirements and properties about the physical behaviour of the machine componentsand the computational processes controlling them.

52-2ex T. Böttjer et al.

In this article, we report on a new model of an idealised IMM derived from ourfirst-hand experience obtained while working with a leading company within the sec-tor of automated plastic manufacture. Our work has so far materialised in a model ofa controller for an injection moulding machine based on the application of the ViennaDevelopment Method (VDM) [2]. The model intends to serve as a specification of theIMM safety, process, and quality control requirements for a typical machine that canbe found on a factory floor. Given that a subset of the VDM models are executable, itis possible to validate the requirements and understanding expressed in the specifica-tion by executing tests, running the controller in parallel with a model of the machinecomponents behaviours, or generating implementations of the controller to deploy inthe physical machine using co-simulation [5]. In addition, it is possible to use the exe-cutable model as a basis for discussion, expertise sharing, and use it as a test bench inan academic setting.

Our work achieved a first-principles model (i.e., a model based on the governingphysics) of the discrete event controller using VDM-RT [16] of the IMM and validatedit against scenarios obtained from a continuous time model of the IMM behaviour asdescribed in [17]. We implemented the continuous time model in MATLAB and used itto generate tests for the VDM model. The next validation step is to connect the VDMmodel with the continuous time model by use of co-simulation. In addition, it is possibleto use the executable model as a basis for discussion, expertise sharing, and use it as atest bench in an academic setting.

Both the VDM model and the co-simulation model are to be made publicly avail-able 1, so they can be used in the future by engineers that are interested in modellingand simulation of IMM as well as the application of Digital Twins in a manufacturingsetting. We plan to add the VDM model to the Overture examples repository and theco-simulation in the INTO-CPS Association repository [8].

Continuous Time Model Discrete Event Model

Hydraulic pressure

Ram Position

Phase Change

Target Ram Velocity

Auto Scrap

Plastification speed

Monitored Variables

Controlled Variables

Contract

Fig. 1. The connection between the discrete event (DE) model and the continuous time (CT)model. The DE models change between a number of states at events occurring in time, while inthe CT models the system state variables change continuously over time.

1 The VDM model is available at: https://www.overturetool.org/download/examples/VDM++/IMMPP.

https://www.overturetool.org/download/examples/VDM++/IMMPP

https://www.overturetool.org/download/examples/VDM++/IMMPP

Modelling an Injection Moulding Machine using the Vienna Development Method 53

The remainder of the paper is organised as follows. Section 2 provides an overviewof related work, and a general overview of the injection moulding machine in Section 3.Then, Section 4 contains an overview of the VDM model. In Section 5, we provide anaccount of the continuous time model and current validation of the VDM model. Lastly,Section 6 provides a conclusion, discussion of the results, and an outline of the futurework.

2 Related Work

In this section, we present a brief review of related work. We follow a top-down ap-proach, where we list the works about Digital Twins followed by models of IMM. Fi-nally, we delve into modelling examples in VDM and cover relevant works.

The authors of [1] use a Domain Specific Language for specifying events, con-necting to an injection moulding machine, and automate the otherwise time-consumingsetup of new production jobs. Our work uses standard VDM to specify the behaviourof a controller for an IMM and validates it with a continuous-time model developed inMATLAB. Although our work towards a Digital Twin setup is in progress, we are stillfar from Hardware In the Loop (HIL) development.

Autonomous vehicles are also an area where Digital Twins bring added value. Thework in [9] presents a framework resourcing to some common technologies that we use,as the standard VDM supporting IDEs [14] are part of the INTO-CPS Application [12]ecosystem. Our work has more emphasis on the specification of the controller and isless oriented towards HIL, as both a desktop model of a commercial IMM or the usageof one is less feasible than the kind of research platform we are comparing ourselveswith.

In the following, we provide a small account of scientific works on models of in-jection moulding and IMMs. The works are listed chronologically. The only executablemodel we found is a MATLAB2 example based on a machine design from 1989. Con-trary to ours, the model relies on purely hydraulics actuation designs. New machinesare now full Cyber-Physical Systems with several computation units inside and the ac-tuation is now done using electric pumps and drives.

The model in [17] is used as a key reference in our work. In their work, the authorslay a precise mathematical formulation of the behaviour of an IMM, yet the authors donot model the controller logic, nor repeated iterations. In our work, we have developed aMATLAB implementation and use it to derive test cases/scenarios to our VDM model.

A step towards a co-simulation model of parts of an injection moulding machineis taken in [6]. The dynamic model of the clamping unit combines a multi-body sim-ulation of the mechanical structure (ADAMS/ANSYS), a hydraulic model (DSHplus),and a control model (MATLAB/Simulink). The model allows testing of interactions be-tween sub-components and analysis with varying design parameters, but overlooks theinjection unit and interactions of machine and process parameters. Our work focuses onthe injection unit and the influence of machine parameters on the process flow.

2 https://se.mathworks.com/help/physmod/hydro/ug/injection-molding-actuation-system.html

https://se.mathworks.com/help/physmod/hydro/ug/injection-molding-actuation-system.html

https://se.mathworks.com/help/physmod/hydro/ug/injection-molding-actuation-system.html


The authors of [15] present standardised (according to IEC 61499) function blocksfor typical faults in manufacturing systems that decouple fault handling and controlcode and reduce the complexity of the control architecture. The methodology is evalu-ated on an injection moulding machine.

The usage of VDM modelling to develop incremental models of CPS systems spaninceptional cases such as [10] or [11], to extensive guidelines in the form of completebooks [3].

VDM has been enabled with recent tooling support in Visual Studio Code [13,14]and together with the INTO-CPS application and the Functional Mock-up Interface-based co-simulation, it may become a valuable tool to build Digital Twins [4].

3 Injection Moulding Process

The typical aspect of a horizontal IMM used in a manufacturing setting is shown inFig. 2. The machine consists of a workbench enclosed by panels and a gate used byoperators to access the clamping unit, which alongside the injection unit sits at the topof the workbench. The two units are the main components involved in the productionof plastic parts that are ejected from the mould inside the clamping unit to a side bucketbelow the gate. Modern machines may use a conveyor belt situated under the clampingunit to collect and route the piece to a quality control check and automatically scrap non-compliant pieces. The machine controller and other components are packaged under the

Fig. 2. 3D representation of an injection moulding machine with the gate highlighted in blue.Below the gate, the part is ejected from the machine and passed onto the belt.

workbench with sensors and actuators connected to different parts of the machine. For


example, given that the gate near the clamping unit must be locked during operationto avoid damage and safety hazards to the operators, it is common to have a sensorthat gauges the state of the gate (open/closed) and feed it to the controller among otherinterlocking mechanisms. This is one of the first properties (a safety property) that anyIMM controller must satisfy:

Property 1 The machine must be idle when the gate is open.

Fig. 3. Schematics of the components of an IMM

In Fig. 3 we consider the main components of a single screw, hybrid IMM com-posed of a clamping unit and an injection unit. The units are further split into actuators.The clamping unit consists of an actuator and a control system to close the mould thatprovides the clamping pressure and ejects the part. The injection unit consists of variouscomponents - the barrel, the screw, the hopper, heat bands, the nozzle - and an electricaland hydraulic actuator, for plastification and forcing the polymer into the mould cavitiesrespectively.

3.1 The Injection Moulding Process

The IM process is cyclic and consists of four states which are associated with the activa-tion of the different machine tool components and amenable to an automaton represen-tation as shown in Figure 4. At the beginning of the process, the screw tip is assumed tobe at a position of x = 0. At this point, the barrel is filled with molten polymer material,and the mould is closed in the clamping unit. At the start of the process, the machine isin the Filling state and is controlled based on the screw velocity. Hydraulic pressure isapplied to the screw to achieve the set screw velocity. After the screw reaches a certainposition, x3, or the expected time, (tfill), the filling phase is complete. Subsequently,the machine transitions to the Packing phase and switches from velocity to pressure


control. For the Packing, Cooling/Classification, and Ejection phases, the state transi-tion is time controlled. After the Ejection phase, the machine becomes Idle and a newcycle may start. In the following paragraphs, we describe the different phases of the IMprocess and some of the properties that one can check or assure in the development ofa VDM model.

Fillingstart Packing

Idle

CoolingPlastification

Eject

x > x3

t > tfill

t > tpack t > tcool

Fig. 4. States and phase transitions of an IM process depicted as an automaton

Injection/Filling Phase: The material is filled into the mould cavity (form) under highpressure and speed. During injection, the material starts to solidify. In this phase, thescrew is position/velocity controlled, and deviations from the final position reached bythe screw are used as an input to the quality control. For example, an undershoot of thescrew position means a lack of material in the mould cavity, thus a defect in the partbeing produced. This is the second property (the first on quality control) we illustrateabout our IMM controller model:

Property 2 The screw must reach EndPositionFill. For the scenario in this paper, End-PositionFill equals 20.3 mm.

Packing Phase: Based on a pre-defined position, the machine control switches to thepacking phase. The material inside the cavity is still flowable along the cross sectionand it is possible to influence the final part weight e.g. to compensate for shrinkage.During packing, the remaining material is further compressed into the cavity at constantpressure. At the end of the packing phase, the gate freezes off such that no more materialcan flow into or out of the mould cavity. Excessive and insufficient packing pressureleads to various defects of the moulded part such as mould sticking and empty holes,respectively. Therefore, we monitor the values achieved and check if they are inside atarget interval. This is the third property (the second on quality control) we illustrateabout our IMM controller model:

Property 3 The hydraulic pressure peak must occur at PeakTimeIntervall and be Peak-PressureBand. For the scenario this paper, these are set to 0.9 < t < 1.1 s and 9 ± 1MPa.


Cooling Phase & Plastification Phase: After the packing and holding phases, coolingis applied through the coolant channels of the mould to lower the temperature and com-pletely harden the material. In parallel, a new charge of material for the next shot isheated, mixed (homogenised) and transported by screw backward rotation. The transi-tion to the ejection phase is specified by a defined cooling time.

Ejection phase: The mould opens, the solidified part is ejected, and the mould is closed.The final part is transferred via a belt to a storage box. A quality controller automaticallyrejects parts that do not meet defined standards based on monitoring relevant processvariables.

This concludes our brief presentation of an IMM and the IM process. For moredetails about this application domain, we refer the readers to the work of [7,17]. Interms of the process, we will provide more details on the continuous behaviour of themachine in Section 5, where we introduce the continuous time model of screw motionand injection process.

4 An Injection Moulding Machine Model in VDM-RT

In our idealised IMM, we conceive a computer/controller receiving readings from anopen/close sensor on the state of the gate, a position sensor receiving the x position ofthe screw tip, and a pressure sensor receiving the nozzle pressure readings. The con-troller affects the system by issuing screw tip velocity target setpoints to the injectionunit local controller, issuing commands to the belt controller on whether to accept orscrap a part, and performs PD control of an electric drive returning the screw tip to the0-position during the Cooling phase.

Fig. 5 visualises the design of the IMM which includes (1) Actuator, (2) Belt,(3) Controller, (4) Environment, (5) GateSensor, (6) IMM_System, (7)InjectionMachine, (8) MotorActuator, (9) PositionSensor,(10) PressureSensor and (11) Timer. All components are represented as VDM-RT files. Besides, the model receives a scenario input and returns the simulation outputas CSV files. Below, we highlight various relevant aspects of the model components.

4.1 World, InjectionMachine and Environment Class

Today, injection moulding machines integrate complex control and software systemsincluding closed-loop process, open-loop phase, safety and quality control. We devel-oped an abstract model of the control architecture of an injection moulding machine.This model was developed in Overture version 3.0.0. We selected the VDM-RT dialectto implement the process/machine controller, quality controller and data acquisitionsystem as a distributed control architecture. The World is the top-level entry point tothe model. A scenario is input from a CSV file containing time steps, screw position,hydraulic pressure and the order to start production of a new plastic element. The loggeroperation returns a CSV file after each step of the simulation.

In our current model, the InjectionMachine models the continuous behaviourof the injection moulding machine. The World controls the simulation. Whenever


World

Belt

Controller

Environment

GateSensor

IMM_System

MotorActuatorPressureSensor PostionSensor

Timer

Sensor ActuatorInjection MouldingMachine

Fig. 5. UML class diagram of the VDM-RT model for an Injection moulding machine controller.The red dotted rectangle indicates physical components of the machine tool (sensors, actuatorsand environment). In addition, the VDM model includes World, System and Controller classes.

time advances by calling the corresponding world functionality, the current screw po-sition and pressure are set in the InjectionMachine. The Step functionality in theInjectionMachine sets the sensor readings for both the Position and Pressure sen-sor. In turn, the Controller can only access the sensor readings to update the currentbelieved state of the system. This separation allows investigating problems arising froma controller being out of synchronization, i.e. sampling slower than needed.

The Environment describes the interaction of the injection moulding machinewith the surrounding environment by receiving the element’s output from the injectionmoulding process.

4.2 Example of Safety Analysis

One feature of our controller is a safety switch coupled to the gate. Our model uses aninvariant to check whether the gate is closed during production. Inside the Controllerclass, we defined the StateType and LockState reflecting the state of the machine andstate of the gate respectively.

typesStateType = <Filling> | <Packing> | <Cooling> | <Ejecting> |

<Idle>;LockState = <Open> | <Close>;

Listing 1.1. Highlights from the Controller class


The current state of the machine operation is kept track of by updating the ctl_statevariable, and the state of the gate is fed to the lock variable. Accordingly, safety isverified by invoking a function that checks at no point the door is open and the machineis not idle.

instance variables

ctl_state : StateType;lock : LockState;inv InterlockInvariant(lock,ctl_state)

functions

InterlockInvariant : LockState * StateType -> boolInterlockInvariant (ls,s) == (ls = <Open>) => (s = <Idle>)

Listing 1.2. Instance variables used to assert safety

4.3 Example of Process Control

The execution of the sequential open-loop process control in injection moulding ismodelled by the Step function in the Controller. The ctl_state instance variablerepresents the current process stage. For each phase, the controller logic is unique andspecified in the relevant operation - FillStep, PackStep, CoolStep and EjectStep.

public Step: () ==> ()Step() == (cases ctl_state:

<Idle> -> return,<Filling> -> FillStep(),<Packing> -> PackStep(),<Cooling> -> CoolStep(),<Ejecting> -> EjectStep() end;);

Listing 1.3. Different IMM operation stages

The FillStep operation implements the behaviour of the controller during the fillingphase. With the molten plastic in the barrel and the screw tip at position 0, the screwstarts rotating to push the melt into the mould. The rotation is controlled accordingto the desired velocity (setpoint) of the horizontal displacement of the screw tip. Thevelocity setpoints, v1, v2 and v3, were obtained from the work of [17] and they areset according to positions also defined in that work for x1, x2 and x3. In Listing 1.4,we depict the specification of the setpoint control switching based on a reading of thescrew tip position sensor, and after achieving the desired position, the machine switchesto PackStep by using a variable assignment function Delta.


public FillStep : () ==> ()FillStep() == (-- Fetch sensor readingctl_x := IMM_System‘ps.getReading();-- Run control flowif (ctl_x < x1)then (IMM_System‘imm.v_t := v1)elseif (ctl_x < x2)then (IMM_System‘imm.v_t := v2)elseif (ctl_x < x3)then (IMM_System‘imm.v_t := v3)-- Transition to Packingelse Delta() )pre ctl_state = <Filling>;

Listing 1.4. Screw velocity setpoint assignment based on current position

The CoolStep operation implements the behaviour of the controller during the cool-ing phase. With the molten plastic solidified, the screw starts rotating backwards toprepare for the next injection cycle. The screw should reach a specified position to pre-pare exactly the amount of plastic required to fill the mould completely. This behaviouris modelled as a PD controller using the position error to set the target velocity of thescrew.

public CoolStep : () ==> ()CoolStep() == (dcl t : real := IMM_System‘timer.GetTime();dcl dt : real := (t - last_time);last_time := t;

ctl_x_last := ctl_x;ctl_x := IMM_System‘ps.getReading();

e_ti_last := e_ti;e_ti := 0 - ctl_x_last;e’_ti := (e_ti - e_ti_last)/dt;

IMM_System‘imm.v_t := K_p * e_ti + K_d * e’_ti;

if (IMM_System‘timer.GetTime()-tphase >= t_cool)then Delta(); )pre ctl_state = <Cooling>

post if (ctl_state = <Ejecting>) then abs (e_ti - 0) < 0.1;

Listing 1.5. Return to initial state of the screw using PD control


4.4 Example of Quality Control

During the filling phase, constant screw velocity and reaching the screw setpoints arekey to achieving a stable process. During the packing phase, the peak injection pres-sure dictates quality defects of the injection moulded part. Quality control monitorsrelevant machine parameters and automatically differentiates between good and badquality parts.

public EjectStep : () ==> ()EjectStep() == (let success : bool = PartIsOK()in if (success) then

ctl_bin := <OK>else

ctl_bin := <Scrap>;pre ctl_state = <Ejecting>;

Listing 1.6. Return to initial state of the screw using PD control

In VDM it is easy to define the quality control by expressing a logical formula overthe historical values of the x position of the screw and the hydraulic peak value insidethe interval. The Boolean variable, hyd_peak_ok, is set to false in cases where thepressure is above the threshold outside the right interval.

pure PartIsOK : () ==> boolPartIsOK () == return (ctl_x_max >= 20.3) and hyd_peak_ok and

(hyd_peak_max >= 9.1);

Listing 1.7. Part of the quality control logic defined in the VDM specification

5 Model Validation

In this section, we describe the steps that were followed to perform a first check of thesuitability of the model. We cross-check the output behaviour of the model executionwhile running it against scenarios simulating the events/inputs to the system sensors ateach time instant. To generate the scenarios, we have implemented a continuous modelof the expected behaviour of an IMM in MATLAB and exported the events to a CSVfile that is in turn read by the VDM model.

5.1 Continuous Time Model

We follow the work by [17], where the authors develop a first-principles model of anIMM linking the controllable machine parameters (e.g., the motion of the screw) tomeasurable process variables (e.g. injection pressure). Accordingly, we only introducethe equations containing the parameters used to generate the scenarios for validating


our VDM model. For a comprehensive overview of the first-principles model, we referthe reader to [17].

The model describes the injection process based on a set of selected equations forthe Filling, Packing and Cooling phases. The set of equations consists of Newtons Sec-ond Law and thermodynamic relationships such as the equations of continuity, momen-tum and energy. In each phase the equations are solved for hydraulic pressure, nozzlepressure, ram velocity, polymer flow, and cavity pressure.

During the Filling phase, Newton’s Second Law is used for describing the velocityof the screw using first order dynamic equations. As shown in Fig. 6b, the screw velocityis changed at positions x1, x2, x3 to the associated velocity setpoints v1, v2, v3. Thescrew velocity v is described by

τ1dv

dt+ v = K1vf (1)

where vf describes the velocity trajectory, and τ1, K1 are constants. Using Newton’sSecond Law the hydraulic pressure is found (compare Fig. 6a)

dv

dt=

∑F

M=PhydAhyd − PbrlAbrl − Cfsign(v)−Bfv

M(2)

where hydraulic pressure is applied to the screw, and barrel pressure,PbrlAbrl, Coulombfriction, Cfsign(v), and viscous damping, Bfv, resist the motion of the screw. M isthe combined mass of the actuators.

During the Packing phase, the process switches to pressure-based control to reachholding pressure, Phold. Eq. 1 is replaced by

τ2dPhyddt

+ Phyd = K2Phold (3)

describing the transition of hydraulic pressure to the holding pressure, Phold. Eq. 2 issolved for describing the deceleration of the screw. Fig. 6 shows the pressure responseand screw velocity as outputted from the continuous model.

5.2 Extracting Scenarios from the Continuous Time Model

We validated the model using three scenarios generated using the physical model of aninjection moulding machine developed in MATLAB. Fig. 7 shows the hydraulic pres-sure during actuation of the injection ram for three different scenarios. Scenario A) usesthe original parameters introduced in [17], while Scenario B) and Scenario C) simulatean injection process where the position setpoint, x2, is not reached or overshoot, respec-tively.

Slight variations in the process conditions during the injection cycle can lead to badquality parts. Variation in hydraulic pressure can cause quality issues with the poly-mer part. Hydraulic pressure above the pre-set value lead excessive polymer flow intothe mould. This causes a lid surrounding the injection part at the parting line of themould halves. Lower pressure than the pre-set value leads to underfilling of the mouldcavities and causes incomplete parts. We varied the hydraulic pressure, a controllablevariable actuating the injection screw during the Filling phase, for validating the mainfunctionality of our VDM model that are:


(a) Combined nozzle pressure response(MPa) for Packing, Filling, and CoolingPhase.

(b) Ram velocity (mm/s) with the three ve-locity set points v1, v2, v3 for the transitionpositions x1, x2, x3.

Fig. 6. Output generated by the continuous time model. These plots were obtained from our MAT-LAB implementation of [17]’s model

1. switching of the injection phases at defined conditions

2. setting of the velocity set points during filling

3. returning of the screw in the cooling phase, i.e. plasticising new polymer for thenext injection cycle

4. the auto scrap control responsible for detecting quality issues from process data andautomatically reject corresponding elements

Fig. 7. Three inputs to the VDM model that were generated using a deterministic injection mould-ing machine model. The hydraulic pressure is shown for A) optimal operating conditions (x2 =15.2 mm, the same behaviour as in Fig. 6), B) the second position set point is not reached (x2 =9 mm), and C) second position is overshoot (x2 = 20 mm).


6 Conclusion and Future Work

As of today, various models of selected IMM components, associated processes, andcontrollers have been developed. Nevertheless, none of the models covers all aspects ofan injection moulding process. Also, models are seldom open-source or available in thepublic domain. This paper develops a high-level description of the control architectureof an injection moulding machine in VDM-RT. We selected VDM as it is traditionallyused to specify the discrete part of a CPS. The developed functionality includes aspectsof the process, quality and safety control. We also set up a machine-process model basedon [17] and conduct a first validation of the control functionality.

To validate the model, we generated three scenarios that were used as CSV inputsto the model. Our validation of the model shows the discrete event controller modelbehaves as expected in the scenarios. The model intends to provide controls to the CTmodel and not match its behaviour, so we decided to not use the results immediatelyas a measure of compliance. Our goal is to test the VDM model using data generatedfrom the CT model. We cross-check the results by observing the response as a plot ofpressure and position/velocity and the outcome is following our expectation.

The resulting model is a first step towards a co-simulation model that is applicable tocontrol an injection moulding machine tool in a Digital Twin setting. Currently, we areunable to accurately point towards the effective benefits beyond the theoretical model-based design advantages. The modelling activity is in its initial stages and more researchis needed to couple the model with a physical system. Next steps include enriching themodels, refining the VDM model and adding testing (e.g., combinatorial tests) and at afuture stage towards a Digital Twin for an IMM, we plan to interface the VDM modelin a co-simulation environment.

Acknowledgements. The work presented here is partially supported by the the PoulDue Jensen Foundation for funding the project Digital Twins for Cyber-Physical Sys-tems (DiT4CPS). This work is also supported by the Manufacturing Academy of Den-mark (MADE) in the MADE FAST project (see http://www.made.dk/). We wouldalso like to thank Daniella Tola for feedback on the first draft of this paper.

References

1. Bibow, P., Dalibor, M., Hopmann, C., Mainz, B., Rumpe, B., Schmalzing, D., Schmitz, M.,Wortmann, A.: Model-driven development of a digital twin for injection molding. In: Dust-dar, S., Yu, E., Salinesi, C., Rieu, D., Pant, V. (eds.) Advanced Information Systems Engi-neering. pp. 85–100. Springer International Publishing, Cham (2020)

2. Fitzgerald, J., Larsen, P.G., Mukherjee, P., Plat, N., Verhoef, M.: Validated Designs forObject–oriented Systems. Springer, New York (2005). https://doi.org/10.1007/b138800,http://overturetool.org/publications/books/vdoos/

3. Fitzgerald, J., Larsen, P.G., Verhoef, M. (eds.): Collaborative Design for Embedded Systems– Co-modelling and Co-simulation. Springer (2013)

4. Fitzgerald, J.S., Larsen, P.G., Pierce, K.G.: Multi-modelling and co-simulation in the engi-neering of cyber-physical systems: Towards the digital twin. In: ter Beek, M.H., Fantechi,A., Semini, L. (eds.) From Software Engineering to Formal Methods and Tools, and Back -

http://www.made.dk/

https://doi.org/10.1007/b138800

http://overturetool.org/publications/books/vdoos/


Essays Dedicated to Stefania Gnesi on the Occasion of Her 65th Birthday. Lecture Notes inComputer Science, vol. 11865, pp. 40–55. Springer (2019)

5. Gomes, C., Thule, C., Broman, D., Larsen, P.G., Vangheluwe, H.: Co-simulation: a Survey.ACM Comput. Surv. 51(3), 49:1–49:33 (May 2018)

6. Hostert, C., Maas, S., Nordmann, R., Zürbes, A.: Dynamic simulation of an injection mold-ing machine. VDI BERICHTE 1887, 165–180 (2005)

7. Kulkarni, S.: Robust process development and scientific molding. In: Kulkarni, S. (ed.)Robust Process Development and Scientific Molding (Second Edition), pp. I–XXII. Hanser,second edition edn. (2017). https://doi.org/https://doi.org/10.3139/9781569905876.fm,https://www.sciencedirect.com/science/article/pii/B9781569905869500013

8. Larsen, P.G., Fitzgerald, J., Woodcock, J., Fritzson, P., Brauer, J., Kleijn, C., Lecomte, T.,Pfeil, M., Green, O., Basagiannis, S., Sadovykh, A.: Integrated Tool Chain for Model-basedDesign of Cyber-Physical Systems: The INTO-CPS Project. In: CPS Data Workshop. Vi-enna, Austria (April 2016)

9. Lumer-Klabbers, G., Hausted, J.O., Kvistgaard, J.L., Macedo, H.D., Frasheri, M., Larsen,P.G.: Towards a digital twin framework for autonomous robots. In: SESS: The 5th IEEE In-ternational Workshop on Software Engineering for Smart Systems. COMPSAC 2021, IEEE(July 2021)

10. Macedo, H.D., Larsen, P.G., Fitzgerald, J.: Incremental Development of a Distributed Real-Time Model of a Cardiac Pacing System using VDM. In: Cuellar, J., Maibaum, T., Sere,K. (eds.) FM 2008: Formal Methods, 15th International Symposium on Formal Methods.Lecture Notes in Computer Science, vol. 5014, pp. 181–197. Springer-Verlag (2008)

11. Macedo, H.D., Nilsson, R.S., Larsen, P.G.: The Harvest Coach architecture: Embeddingdeviation-tolerance in a harvest logistic solution. Computers 8(2), 31 (2019)

12. Macedo, H.D., Rasmussen, M.B., Thule, C., Larsen, P.G.: Migrating the into-cps applicationto the cloud. In: International Symposium on Formal Methods. pp. 254–271. Springer (2019)

13. Rask, J.K., Madsen, F.P., Battle, N., Macedo, H.D., Larsen, P.G.: Visual Studio Code VDMSupport. In: Fitzgerald, J.S., Oda, T. (eds.) Proceedings of the 18th International OvertureWorkshop. pp. 35–49. Overture (December 2020), https://arxiv.org/abs/2101.07261

14. Rask, J.K., Madsen, F.P., Battle, N., Macedo, H.D., Larsen, P.G.: The SpecificationLanguage Server Protocol: A Proposal for Standardised LSP Extensions. In: Proceed-ings of the 6th Workshop on Formal Integrated Development Environment. F-IDE (May2021), https://cister-labs.pt/f-ide2021/images/preprints/F-IDE_2021_paper_3.pdf

15. Steinegger, M., Zoitl, A., Fein, M., Schitter, G.: Design patterns for separating fault han-dling from control code in discrete manufacturing systems. In: IECON 2013-39th AnnualConference of the IEEE Industrial Electronics Society. pp. 4368–4373. IEEE (2013)

16. Verhoef, M., Larsen, P.G., Hooman, J.: Modeling and Validating Distributed Embedded Real-Time Systems with VDM++. In: Misra, J., Nipkow, T., Sekerinski, E. (eds.) FM 2006: FormalMethods. pp. 147–162. Lecture Notes in Computer Science 4085, Springer-Verlag (2006).https://doi.org/10.1007/11813040_11

17. Woll, S.L., Cooper, D.J.: A dynamic injection-molding process model for simulating moldcavity pressure patterns. Polymer—Plastics Technology and Engineering 36(5), 809–840(1997)

https://doi.org/https://doi.org/10.3139/9781569905876.fm

https://www.sciencedirect.com/science/article/pii/B9781569905869500013


https://arxiv.org/abs/2101.07261

https://arxiv.org/abs/2101.07261

https://cister-labs.pt/f-ide2021/images/preprints/F-IDE_2021_paper_3.pdf

https://cister-labs.pt/f-ide2021/images/preprints/F-IDE_2021_paper_3.pdf

https://doi.org/10.1007/11813040_11

RMQFMU: Bridging the Real World withCo-simulation

For Practitioners?

Mirgita Frasheri1[0000−0001−7852−4582], Henrik Ejersbo1[0000−0003−4753−3800],Casper Thule1[0000−0001−6606−9236], and Lukas Esterle1[0000−0002−0248−1552]

Centre for Digital Twins, DIGIT, Aarhus UniversityFinlandsgade 22, 8200 Aarhus N, Denmark

{mirgita.frasheri,hejersbo,casper.thule,lukas.esterle}@ece.au.dkhttps://digit.au.dk/centre-for-digital-twins/

Abstract. In this paper we present an experience report for the RMQ-FMU, a plug and play tool, that enables feeding data to/from an FMI2-based co-simulation environment based on the AMQP protocol. Bridgingthe co-simulation to an external environment allows on one side to feedhistorical data to the co-simulation, serving different purposes, such as vi-sualisation and/or data analysis. On the other side, such a tool facilitatesthe realisation of the digital twin concept by coupling co-simulation andhardware/robots close to real-time. In the paper we present limitationsof the initial version of the RMQFMU with respect to the capabilityof bridging co-simulation with the real world. To provide the desiredfunctionality of the tool, we present in a step-by-step fashion how theselimitations, and subsequent limitations, are alleviated. We perform var-ious experiments in order to give reason to the modifications carriedout. Finally, we report on two case-studies where we have adopted theRMQFMU, and provide guidelines meant to aid practitioners in its use.

1 Introduction

Cyber-physical systems (CPSs) refer to systems that combine computationaland physical processes, and play an important role in the development of intel-ligent systems. Using real world operation data could possibly facilitate smartdecision-making [1]. Harnessing this potential is by no means trivial, with chal-lenges including safety [2], reliability, and security [3], among others, that need tobe tackled. Additionally, CPS development is a multi-disciplinary process, wheredifferent components will be modelled and validated by different tools used inthe different involved disciplines. The evaluation of such complex systems canbe performed through co-simulation [4]. This is made possible by developingthe components according to some standard, e.g. the Functional Mock-up In-terface (FMI) adopted in our work, with components referred to as functional? We are grateful to the Poul Due Jensen Foundation, in supporting the establishment of a new

Centre for Digital Twin Technology at Aarhus University, and Innovation Foundation Denmarkfor funding the AgroRobottiFleet and UPSIM projects.

https://digit.au.dk/centre-for-digital-twins/

RMQFMU: For Practitioners 67

mock-up units (FMUs). However, this is not sufficient, as CPSs together withtheir environment(s) are subject to continuous change, and evolve through time,possibly diverging significantly from the initial co-simulated results [5]. Digitaltwins (DTs), defined as digital replicas of the physical components, also knownas physical twins (PTs), can be used to follow the behaviour of the PTs andCPSs during operation, potentially through learning [6], by adapting the mod-els, as well as performing different tasks such as monitoring or predictions. TheDT and the aforementioned (or more) operations can be developed via the co-simulation of a modelled system that corresponds to the deployed CPS. In orderto connect the deployed CPS to the DT, the implementation of data brokeringbetween them becomes a necessity. Such data brokering can be useful in differentscenarios; in this paper we identify four such:

1. For data analysis and visualisation of system behaviour, where the user isinterested in feeding back to the co-simulation data recorded from the oper-ation of the hardware, i.e. historical data, to perform some analysis on thedata, potentially supporting the visualisation of said data.

2. For co-simulation, where the user is interested in coupling systems that aresimulated in different environments, in those cases for which it is easier touse data-brokering than to implement the FMI interface for a non FMI-compliant component, e.g. connecting to a Gazebo simulation 1.

3. For the realisation of the digital shadow (DS), where the digital models ofthe physical twin are contained within the co-simulation environment. Theuser is interested in getting the live data from the PT into the co-simulationenvironment in order to estimate the difference between actual and simulateddata along with visualising the operation of the physical twin. This is similarto the first usage, however the digital shadow would operate with live data.

4. For the realisation of the digital twin, where the user is interested in en-abling the communication link from the DS to the PT, thus fully realisingthe implementation of the DT concept. Indeed, components within the co-simulation environment, based on the live data, could send live feedback tothe PT. This can be useful for monitoring the behaviour of the PT, or duringCPS development, to lower the cost of testing with Hardware in the Loop.

In this publication we extend an existing Data-Broker called RabbitMQ FMU(RMQFMU 2) [9], initially suitable for the first scenario, to enable data brokeringboth to and from an FMI2-enabled co-simulation, applicable for all the aforemen-tioned scenarios. We present an experience report on applying the RMQFMUto two actual cases undergoing development to become digital twin systems: atabletop robot arm and an autonomous agriculture robot. As part of this experi-ence report we present guidelines on how to configure the RMQFMU parameters

1 In other cases, the generation of an FMU might be preferable. Tools like UNIFMU [7] forquick FMU prototyping, or PVSio-web based tool-kits [8], are designed to help users prototypeFMUs for their co-simulations.

2 https://github.com/INTO-CPS-Association/fmu-rabbitmq.git

https://github.com/INTO-CPS-Association/fmu-rabbitmq.git

68-2ex M. Frasheri et al.

by presenting various experiments that show their effects and relation to one an-other. Through the application of RMQFMU to the aforementioned cases andexperiments several needs were discovered such as:

1. Data Platform - Get all available data instead of minimally-needed data toenable decision making, possibly allowing to jump ahead to future data.

2. Performance - RMQFMU shall be as fast as possible.3. Data Delay - RMQFMU shall output the newest data if available.

To address need (1) and (2), the existing RMQFMU, henceforth referred toas RMQFMU0, was realised with multi-threading instead of single-threading.This new multi-threaded version is referred to as RMQFMU1. Experimentationrelated to detailing the configuration of RMQFMU1 and its related effects ledto the discovery of need (3). In order to mitigate (3), yet another version ofRMQFMU was realised referred to as RMQFMU2, which represents the latestversion of the RMQFMU.

The rest of this paper is organised as follows. The next section provides anoverview of the FMI standard, the first version of the RMQFMU (RMQFMU0),as well as some of the technologies employed in the realisation of the case-studies.Afterwards, we informally derive the requirements for RMQFMU, based on theusage scenarios. Section 3 presents RMQFMU1,2, how it mitigates some inad-equacies of RMQFMU0, and how it compares to RMQFMU0. Thereafter, inSection 4 the individual case-studies are described, and the results gained withRMQFMU1,2 are presented. Moreover, a detailed comparison between RMQFMU1and RMQFMU2 is made. Section 5 provides a set of guidelines meant to aid theuse of RMQFMU2, whereas Section 6 concludes the paper.

2 Background

In this Section we provide a brief summary on the relevant concepts for thiswork, such as co-simulation, FMI standard, master algorithms, and the tools weuse for their realisation. Thereafter, we present the first released version of theRMQFMU (RMQFMU0), which we have extended as presented in this paper.

2.1 Concepts and Tools

The realisation of CPSs and constituent systems is a cross-disciplinary process,where the different components are modelled using different formalisms and mod-elling tools [10]. In order to evaluate the behaviour of such systems as a whole,co-simulation techniques are used, which require the integration of the separatemodels into what are called multi-models [11]. For the latter to be possible, thetools used to produce the individual models need to adhere to some standard.One such standard is the Functional Mock-up Interface 2.0 for Co-simulation(FMI) [12] that defines the C-interfaces to be exposed by each model, as well asinteraction constraints, packaging, and a static description format. An individ-ual component that implements the FMI standard is called Functional Mock-up


Unit (FMU). A co-simulation is executed by an orchestration engine that em-ploys a given master algorithm. The master algorithm defines the progression ofa co-simulation in terms of getting outputs via getXXX function calls, settinginputs via setXXX function calls, and stepping the individual FMUs in time viathe doStep function calls. Common master algorithms are the Gauss-Seidel andJacobi (the curious reader is referred to [10] for an overview of these algorithmsand a survey on co-simulation). In this work, we use the open-source INTO-CPStool-chain [13], for the design and execution of co-simulation multi-models, withMaestro [14] as orchestration engine employing an FMI-based Jacobi masteralgorithm.

2.2 Overview of the RMQFMU0

RMQFMU0 was implemented to enable getting external/historical data intothe co-simulation environment, for either replaying such data in simulation, orfor performing different kinds of analysis, e.g. checking its difference from ex-pected data (the reader is referred to the Water-Tank Case-Study for an exam-ple3). Essentially, RMQFMU0 subscribes to messages with a specified routingkey and outputs messages as regular FMU outputs at specific points in timebased on the timestamp of the messages. At every call of the doStep function,the RMQFMU0 attempts to consume a message from the server. A retrievedmessage is placed in an incoming queue, from which it is thereafter processed,according to the quality constraints (maxage and lookahead), described shortly.In case there is no data available, the RMQFMU0 will wait for a configurabletimeout before exiting. Note that, the doStep is tightly bound with the consum-ing operation, i.e. they are both contained within the main thread of execution.If on the other hand there is data, its validity with respect to time is checked.The maxage, main parameter of the RMQFMU0, allows to configure the age ofdata within which it is considered valid at a given time-step. Additionally, thelookahead parameter specifies how many messages will be processed at a stepof RMQFMU and thereby at the given point in time4. The behaviour of theRMQFMU0 has been formally verified in previous work [9].

3 The RMQFMU

The design of the RMQFMU is based on a set of informally derived requirements,covering those functionalities needed for the adoption of the RMQFMU in thescenarios described in Section 1. These requirements are given as follows:

1. The RMQFMU is able to get the data published by an external system tothe RabbitMQ server. This is relevant for all scenarios.

3 Available at https://github.com/INTO-CPS-Association/example-single_watertank_rabbitmq, visited May 6, 2021

4 In this version, the functionality based on the lookahead is not fully implemented.

https://github.com/INTO-CPS-Association/example-single_watertank_rabbitmq

https://github.com/INTO-CPS-Association/example-single_watertank_rabbitmq


2. The RMQFMU is able to publish data to the RabbitMQ server, thus closingthe communication link. This is relevant for the second and fourth scenario,i.e. for co-simulation and DT realisation.

3. The FMU steps as fast as possible. Indeed the role of the RMQFMU is thatof a data broker, as such it is not part of the system being simulated, rathera facilitating entity. Therefore, the delays it causes should be minimal andimpact the overall co-simulation environment as little as possible.

4. Provide quality constraints, e.g. with respect to age of data (maxage).5. Provide performance constraints, e.g. number of messages processed per step

(lookahead).

RMQFMU0 already fulfils requirements 1 and 4. Requirement 5 is partiallyimplemented in this version as well, in that while the lookahead determines howmany messages to retrieve from the incoming queue at every time-step, therewill not be more than one message at a time in this queue, given that a consumecall is performed at the doStep call until a valid message is retrieved.

In order to tackle requirement 2, we enable the configuration of the inputs ofthe RMQFMU1,2 as needed, i.e., a user can define as many inputs as desired, ofone the following types: integer, double, boolean, and string. These inputs willbe sent to the RabbitMQ server, on change; in other words, the values of theinputs will be forwarded if they have changed as compared to the previous steptaken by the RMQFMU1,2. This check is performed within the doStep.

In order to tackle requirement 3, we investigate the potential benefit of athreaded configuration of the RMQFMU1,2. Indeed, the FMU allows for a build-time option to enable a multi-threaded implementation, with a separate threadto interface and consume data from the RabbitMQ server, parsing incoming dataand placing it in the FMU incoming queue. The main doStep function of theFMU still executes in the main thread context of the calling simulation orches-tration application, and it reads and processes data from the FMU incomingqueue and produces outputs. If the multi-threaded option is disabled (default),the doStep function will consume data directly from the RabbitMQ server inthe context of the calling main thread.

In general the potential performance benefits of the threaded implementationwill depend on the amount of data being consumed, parsed, and provided inthe internal incoming FMU queue when the doStep function is called. Hence,the benefits will depend largely on the co-simulation environment. If the co-simulation is fast, in vague terms if the delay between each call of the FMUdoStep function is short compared to the rate of the incoming data, then theFMU will be mostly blocking for I/O. In this situation, the separate consumerthread may not provide much benefit as the doStep function will still need towait/block for incoming data. On the other hand, if the co-simulation is slowercompared to the rate of data, then the separate thread may be able to consume,parse, and provide data to the internal incoming FMU queue, in parallel to themain co-simulation thread running its orchestration engine and executing otherpotential FMUs or monitors. In addition, any performance benefits of a threaded


implementation also depends largely on the execution platform having a multi-core processor and the OS/thread environment to be able to take advantage.

The threaded implementation allows for the full realisation of the lookaheadfunctionality, thus tackling requirement 5. The consumer thread is continuouslyretrieving data – when available – and placing these messages in the incomingqueue, from where said data is processed in chunks of lookahead size.

4 Experiments

The RMQFMU1,2 has been evaluated with a series of experiments, across dif-ferent combinations of parameters such as the maxage and lookahead, and in twodifferent case-studies. The purpose of these setups is to provide an understandingon the performance of the RMQFMU1,2, its parameter tuning, as well as providesome insight into the effect of external factors. The first case-study presents ascenario based on data from a deployed industrial robot, thus adequately repre-senting an industrial case. As a result, we perform the performance evaluation inthis case-study. Additionally, we add mockup components to the co-simulationstructure, to mimic a more realistic co-simulation with different components fordifferent purposes, such as data analysis, prediction among others. We also con-sider the impact of external factors, such as the frequency of sending data tothe RMQFMU1,2. Whereas, the second case-study presents a setup with datafrom the Gazebo simulation of the Robotti agricultural robot [15], rather simpleboth in terms of the size of the messages being sent and the internal structureof the co-simulation. The parameters of interest are the lookahead, maxage, andthe frequency with which we send data into the RMQFMU1,2.

The experimentation is divided in two phases. In the first, we evaluate the im-pact of the multi-threaded implementation on the performance of the RMQFMU.Specifically, we compare RMQFMU1 to RMQFMU0, and argue for the suitabilityof adopting the multi-threaded configuration as the primary implementation forthe RMQFMU. Additionally we consider the effects of the maxage and lookaheadfor RMQFMU1. In the second phase, we evaluate the alterations to RMQFMU1,i.e. RMQFMU2, in order to tackle the side effects observed in phase 1, providingboth performance and behaviour results.

4.1 Case-study 1: Universal robot 5e (UR)

This case study concerns an industrial robot called UR5e5 (Figure 1). The robothas a reach of 85 mm, payload of 5kg and 6 rotating joints. It is chosen becausethe UR5e is in production, thus represents a realistic component in a digitaltwin setting, and thereby a realistic amount of data. The data set is generatedfrom a series of movements used for calibrating the digital twin to a relatedphysical twin. Therefore, all joints are exercised. The test data from the UR robot

5 Available at https://www.universal-robots.com/products/ur5-robot/,visited April 7, 2021

https://www.universal-robots.com/products/ur5-robot/


used is present in csv format (35123 lines including the header) and consists ofmessages (excluding the header) with 107 float values (one being time) and 10integer values. The messages have a sample rate (frequency fdata) of 2ms and willbe replayed/produced into the RabbitMQ broker with this rate. The test datacontains a substantial number of time gaps, i.e. places where two consecutivemessages in the input data are spaced with more than 2ms. Data is replayedat a constant frequency of 2ms and as such some messages are produced fasterthan their timestamps indicate. These effects are discussed in the test results.Tests are also performed on a cleaned test data with a constant message spacingof 2ms between all pairs of messages - i.e. no additional time gaps.

To investigate the behaviour of the FMU, we perform tests with differentvariations and relations between the simulation step size and the delay imposedby the simulation environment. The tests are run on a 2.6GHz Intel Core i7 (6cores), with 32GB RAM. We perform the tests with and without the thread-ing option to investigate its effect, ton and toff respectively. We first considersimulation step duration and environment delays that simulates a setup wheredata is expected to be consumed as fast as its produced - i.e. every 2ms andthe overhead of the simulation delay is minimal. In this case we consider boththe simulation step and the simulation delay to match the input data rate of2ms. Then we consider situations with larger simulation step duration and largersimulation delays - to mimic co-simulation environments with additional FMUsor monitors. The full list of test data used is provided in Table 1. In short, theresults will provide an insight into how these different parameters influence theperformance of the RMQFMU. Note, that for these tests we use a fixed looka-head of 1 and a fixed maxage of 300ms. The max age is chosen to cover for timegaps in the input data up to this range.

Fig. 1. UR5e robot

Table 1. Tests for Case-study 1 6

Sim step Sim delay Thread

Case 1 2ms 2ms toff

Case 2 2ms 2ms ton

Case 3∗ 100ms 100ms toff

Case 4∗ 100ms 100ms ton

Case 5 100ms 113ms ton

Case 6 100ms 120ms ton

6 Due to space constraints, we present in the paper results for some selected cases marked with∗. The results as a whole can be found in the technical report [16].


4.2 Case-study 2: Gazebo Simulation of Robotti

The purpose of this case-study is to provide a basic example of a co-simulationreceiving data recorded 7 from the Gazebo simulation of the Robotti agriculturalrobot [15], through the RMQFMU, in order to gain insight into how the con-figurable parameters of the RMQFMU affect its behaviour. Tests are run on a2GHz Quad-Core Intel Core i5, with 16GB RAM. In this case the co-simulationenvironment consists of the RMQFMU and a monitor FMU. The data of inter-est in this scenario, i.e. the data sent through the RMQFMU consists of: the xand y positions of the robot, and the x and y position of the nearest obstacle.Additionally, a sequence number is attached to each message to keep track ofthe outputted messages during the processing of the results. The monitor takesas input such data, and computes the distance between the robot and obstaclefor every time-step. In case such distance is below a predefined safety threshold,the monitor will issue an emergency stop to the Gazebo simulation of the robot.

Table 2. Overview of experiments for Case-study (CS) 2

CS 2 Parameters CS 2 Parameters CS 2 Parametersla ma ts fdata la ma ts fdata la ma ts fdata

Case 1 1 0.2s 0.1 100ms Case 5 50 0.2s 0.1 2ms Case 9∗ 2 0.4s 0.1 200ms

Case 2 50 0.2s 0.1 100ms Case 6 200 2s 0.1 2ms Case 10∗ 5 0.4s 0.1 200ms

Case 3 1 2s 0.1 100ms Case 7 50 0.2s 0.1 2ms Case 11∗ 2 2s 0.1 200ms

Case 4∗ 50 2s 0.1 100ms Case 8 200 2s 0.1 2ms Case 12∗ 5 2s 0.1 200ms

For this case-study we are interested in the following parameters (see Ta-ble 2), the maxage ma of the messages, and the lookahead la, and how theyaffect the behaviour of the RMQFMU, i.e. in terms of the sequence of messagesoutputted at every time-step. Note that for all the cases the threaded option isused (ton). Parameters such as the time-step size ts, the frequency of sendingdata fdata to the co-simulation are fixed for cases 1 − 4, such that they align.We argue such decision with the simplicity of the case-study, which could allowa user in reality to align these values. The time-step of the simulation is fixedto 0.1s for adequate granularity. Whereas for cases 5− 12 we consider fdata thatdoesn’t match the size of the simulation step, and observe the effects of themaxage and lookahead.

4.3 Phase 1 Results

Case-study 1 In this paper we will present only the results from cases 3 and4 (Table 1) due to space constraints, and refer the reader to the technical re-port [16] that contains the results over all cases. The results not included in this

7 We record the data to ease the testing process across different runs.


0 5 10 15 200

5

10

15

20·103

sim-time (s)

Stepduration(µs)

toff ton

0 5 10 15 200

2

4

6

8·102

sim-time (s)

Queu

esize

Fig. 2. Cases 3 and 4: step=100ms, delay=100ms, thread=toff /ton

paper are coherent with the ones included. In the included scenarios, the sim-ulation step and the simulation delay are set each to 100ms, in order to mimica co-simulation environment that imposes a step size of 100ms and has a simu-lation delay also of 100ms, caused e.g. by other FMUs or monitors. The resultsof these two tests are shown in Figure 2. The number of messages to process ineach step is 50, given the input frequency of 2ms. Additionally, given the simula-tion delay equal to 100ms, for the threaded configuration all 50 messages will beavailable in the RMQFMU1 queue when the step function is called. The artificialsimulation delay of 100ms covers enough time for the separate consumer threadto have consumed approximately 50 messages. While for the non-threaded con-figuration, the step function itself needs to consume approximately 50 messagesoff the socket interface inside a single step. From Figure 2 left-hand side graphit can be observed that for this test, the threaded configuration (red) shows animprovement in average step duration of approximately 10ms compared to theunthreaded configuration (blue). This difference corresponds to the overhead ofconsuming a single message from the socket interface via the rabbitmq clientlibrary, parsing the message, and finally adding it to the internal RMQFMU1queue. As approximately 50 messages need to be consumed in a single step,this accounts to an overhead of approximately 200µs per message (10ms / 50 =200µs). The right-hand side graph in Figure 2 shows the internal FMU queuesize at the exit of each simulation time step. The increase of queue size in thethreaded configuration (red) is mostly an effect of gaps occurring in the inputdata. Large gaps in the input will cause the RMQFMU1 to stay at its currentoutput for a predefined maxage time period ideally covering the input gap. Thiseffect can also be observed in the left-hand side graph, when the step durationoccasionally lowers to around 0 when the RMQFMU1 stays at its current out-put to cover a gap. For this period, data will still be consumed by the separateRMQFMU1 consumer thread and added to the internal queue. The RMQFMU1implementation must therefore include guards to respect internal queue sizelimitations. The internal RMQFMU1 queue size in the unthreaded configuration(blue) is always 0 at simulation step exit, since that configuration consumes onlya single message off the socket interface per step. In this configuration, a queuesize build up will occur in the socket layer rather than the FMU layer.


Case-study 2 Similarly to Case-study 1, we will present only the results fromcases 9 − 12 (Table 2) due to space constraints, and refer the reader to thetechnical report [16] that contains the results over all cases. Note that the resultsnot included remain coherent with the ones presented in the current paper. Theresults for cases 9− 12 are displayed in Figure 3. For a maxage equal to 400ms,we can see that there is not much an effect of the value set for the lookahead.This is due to the fact that with lower maxage, the data becomes invalid soonerthan for higher maxage, thus triggering the RMQFMU1 to fetch newer datafrom the incoming queue. Nonetheless, for a ma = 2s, it is possible to note adifference for the two lookahead values, where for la = 5, there is a bigger jumpin the sequences of data for specific time-steps, when a new value is needed.Additionally, sequence numbers remain fixed for consecutive number of time-steps, whereas for la = 2 more in-between value are outputted, as expected. Inthe latter however, there is more delay in the output values. Beside the maxagevalue, the frequency of sending data also impacts the effect of the lookahead, asit directly affects the number of messages found in the incoming queue.

0 5 100

20

40

sim-time (s)

Sequen

cenumbers ma=400ms

la = 2

la = 5

0 5 100

20

40

sim-time (s)

ma=2000ms

Fig. 3. Messages outputted by the RMQFMU1, for varying ma, la, and fdata = 200ms.

The number of skipped messages is not exactly the same from run to run,as observed in Figure 4, with a lookahead equal to 50 and maxage fixed to2000ms, over 5 independent runs. One factor that impacts the state of the queueis the speed of the execution of a simulation step. In Figure 4, one can observethe wall-clock time for two of the simulation runs. The first run is executedfaster than real-time, at least until time step 2.0, whereas the second is executedslower than real-time. This depends on the underlying execution environment,as well as the load on said environment. As such, a specific lookahead will notnecessarily output the same sequence numbers at each time step over differentindependent runs. This effect is also visible in Figure 3, where near the end,for both lookahead values, the sequence numbers that are outputted are similar.This means that, there were no newer messages in the queue at those particulartimes for lookahead= 5, as the simulation was running faster than real-time.

4.4 Phase 2 ResultsAs discussed in the previous results, the RMQFMU0,1 has a rather conservativeinterpretation of the maxage parameter. The FMU will always stay at its current


0 5 100

20406080

sim-time (s)

Sequen

cenumbers Runs 1− 5

run 1run 2run 3run 4run 5

0 5 100

20406080

sim-time (s)

Runs 1− 2

0 3.021 8.092

Run 1: real-time (s)

0 3.021 8.092


0 5.433 8.837


0 5.433 8.837


Fig. 4. Messages outputted by RMQFMU1, over 5 runs, for ma = 2s and la = 50.

output, if the timestamp of the output is within maxage range from the currentsimulation step timestamp. This means that, the timestamp of the output plusthe maxage is greater than or equal to the current simulation timestamp. How-ever, this interpretation is rather strict, in the sense that it doesn’t take intoaccount if newer data is present in the incoming FMU queue. Thus, we want torelax this condition such that the FMU will only stay at its current output, ifthe output is within the maxage and in addition there is no newer data available.As such, the FMU should always move ahead to the latest data valid within thecurrent step, and the maxage is only considered when there is no new input.

We have implemented this change of maxage semantics in the RMQFMU2version of the FMU. The change itself had some side-effects impacting the per-formance of the FMU step function. In short, the change means that the FMUhas to consider if data is present before considering the maxage. However, situ-ations may occur where data is present in the incoming queue, but it may havetime-stamps in the future of the current simulation time. This can occur e.g.when replaying with a fixed interval historical data including gaps. If this situa-tion occurs, the RMQFMU0,1 versions of the FMU would move the future datafrom the incoming queue into an internal processing queue, that would grad-ually keep increasing in size and thereby impacting the step performance. Toremedy this issue, RMQFMU2 has an additional change to the internal queuestructure. In the following paragraphs we present results of experiments withour two case-studies using RMQFMU2.

Case-study 1 The effect of the RMQFMU2 changes to the case-study 1 can beobserved in Figure 5. The test configuration for this test is identical to Case 4of Table 1. The results of this case with RMQFMU1 are present in Figure 2.To demonstrate the effect of moving to the latest input data available, we have


specified a lookahead size of 100 in this test. This covers more messages thanbeing produced on average within a simulation time step of 100ms and a datafrequency of 2ms, which is approximately 100/2 = 50 messages. Practically, theRMQFMU2 consumes as many messages as possible within a time step.

It is possible to observe from Figure 5, graph on the left, that in contrast toFigure 2, the step duration now has decreased from around 6000us to around1000us. This is due to the fact that we now use the larger (100 = move asclose to step time as possible) lookahead size as opposed to the original size 1,additionally to the improvement in the internal queue implementation. Also, thelarger lookahead size has the effect, that the queue size (right-hand sideL graph)stays low for a longer simulation time compared to earlier test results. However,the queue size still increases due to both data being produced more frequentlythan consumed and to gaps in the input data. Thus, as mentioned before, theFMU needs to guard any internal queue size limitations.

0 10 200

0.5

1

1.5

2·103

sim-time (s)

Stepduration(µs)

0 10 200

1

2

3

·102

sim-time (s)

Queu

esize

Fig. 5. Step duration and sequence numbers outputted by RMQFMU2 at every step, for step-sizeof 100ms, and delay of 100ms.

Case-study 2 The new version of the RMQFMU2 removes the initial delay inthe outputted sequence numbers noticeable for large values of the maxage, e.g.2000ms (Figure 6, for Cases 1−4), as compared to RMQFMU1 (Figure 3) 8. Theutility of the maxage parameter becomes evident, once gaps are introduced inthe data, in these experiments equal to 500ms, Figure 7, left graph, and 1000ms,right graph. It can be observed how when there is no input, the sequence valuethat is outputted is the last value that is valid from a time perspective. Naturally,for larger gaps, the sequence numbers outputted are more delayed, e.g. for t = 5s,the RMQFMU2 with 500ms gaps is at number 10, whereas for 1000ms is at 5.Note that, with RMQFMU1, given the initial delay, we would expect a lowersequence number outputted at t = 5s.

In order to show the utility of the lookahead, we ran the experiments withsimulation step-size equal to 100ms, for values of maxage in {200ms, 2000ms},with frequency of sending data equal to 2ms (Figure 8). For smaller values of themaxage, the delay of the RMQFMU2 for lookahead equal to 1, while consistently

8 Note, the initial delay is present in all cases for RMQFMU1.


0 5 100

50

100

sim-time (s)

Sequen

cenumbers ma=200ms

la = 1

la = 50

0 5 100

50

100

sim-time (s)

ma=2000ms

Fig. 6. Messages outputted by the RMQFMU2 at every step, for fdata = 100ms.

0 5 100

5

10

15

20

sim-time (s)

Sequen

cenumbers ma=2000ms

0 5 100

5

10

sim-time (s)

ma=2000ms

Fig. 7. Messages outputted by the RMQFMU2 at every step, for fdata = 100ms and la = 1, withgaps of 500ms (left), and 1000ms (right).

present, is rather low. However, for larger values of the maxage, the delay for lowlookahead values is non-negligible. It is possible to observe that for the durationof the maxage the sequence numbers gradually increase by 1, as expected forlookahead equal to 1. After the initial maxage duration has passed, there is ajump in the sequence numbers, that continue to follow the results for lookaheadequal to 50, albeit with a more or less constant delay until the end. A similareffect was observed in Figure 3, with the difference that in the current version,the RMQFMU2 checks if there are newer messages in the queue. This is due tothe fact that after 2s, older in-between values become invalid, and thus are notoutputted, with RMQFMU2 going back to the queue for more recent messages.

5 RMQFMU2 Configuration Guidelines

The parameters of the RMQFMU2 should be configured based on the use-case inorder to result in desired behaviour. If possible, the simulation step-size shouldmatch the frequency of sending data fdata. However, depending on the timegranularity needed by different FMUs, achieving such alignment might not bepossible. The maxage value has to be big enough to make up for any time gapbetween data, but also low enough to ensure consistency in domain of use. E.g.for some applications, it might the case that data should not be older than200ms to ensure correct operation. The lookahead can be set to counter-balance


0 5 10

0

1

2

3

4·103

sim-time (s)

Sequen

cenumbers

la = 1

la = 50

0 5 10

0

1

2

3

4·103

sim-time (s)

0 0.1 0.2

1

2

3

0 1 20

10

20

Fig. 8. RMQFMU2 output at every step, for fdata = 2ms, over 2s.

the effect of the maxage, as it allows to jump to the latest data that is availablein the queue (within the limits of the lookahead). Note however, larger values ofthe lookahead will result in bigger time jumps between the messages outputted.If more intermittent values are required, then a lower value should be adopted.The frequency of sending data influences the meaningful range of values for thelookahead, e.g. sending data every 2ms, will result in 500 messages per second.For a simulation step-size of 100ms, a reasonable value of the lookahead wouldbe circa 50. Finally, the speed of the simulation also affects what is present inthe queue at any time, as a result impacting RMQFMU2 outputs.

6 Concluding Remarks

In this paper we have described an extended RMQFMU data-broker that en-ables coupling an FMI-based co-simulation environment to a non-FMI externalcomponent, via the AMQP protocol. The RMQFMU supports communicationin both directions, and can be used in different contexts ranging from replayinghistorical data into a co-simulation, to enabling the realisation of the DT conceptby connecting the DT to its physical counterpart. We evaluate this componentin terms of its performance, i.e. the real-time duration of a simulation step. Ourresults show the benefit of implementing a threaded solution, that effectivelydecouples the doStep logic from the consumption from the rabbitMQ server.Moreover, we explore different values of its configurable parameters, and provideusage guidelines for practitioners.

There are five potential directions for future work. First, we are interested inenabling the RMQFMU to take bigger step-sizes if necessary. In some monitoringapplications, where only the latest data-points are relevant, such jump could beof use to mitigation components that deal with out of synchronisation situations.Second, we plan to formally verify the presented RMQFMU, thus extendingprevious work that tackled its initial version [9]. Third, we intend to profile theoverhead of RMQFMU as data-broker compared to an FMU that directly queriesa database, for example in the context of large amounts of data. Fourth, wewill enable the master-algorithm to learn the optimal step-sizes automaticallyand deal with recurring variation at run-time. Fifth, we are interested in the


implementation of a more general solution, that can be configured to use differentlibraries and protocols, e.g. ZeroMQ.

References

1. A. Banerjee, K. K. Venkatasubramanian, T. Mukherjee, and S. K. S. Gupta. Ensur-ing safety, security, and sustainability of mission-critical cyber–physical systems.Proc. of the IEEE, 100(1), 2012.

2. R. Baheti and H. Gill. Cyber-physical systems. The impact of control technology,12(1):161–166, 2011.

3. PMB ECSEL. Multi annual strategic research and innovation agenda for ecseljoint undertaking, 2016.

4. C. Gomes, C. Thule, D. Broman, P.G. Larsen, and H. Vangheluwe. Co-simulation:a Survey. ACM Comput. Surv., 51(3), 2018.

5. J Fitzgerald, P.G. Larsen, and K. Pierce. Multi-modelling and co-simulation inthe engineering of cyber-physical systems: towards the digital twin. In From Soft.Eng. to formal methods and tools, and back. Springer, 2019.

6. J. Fitzgerald, K. Pierce, and P.G. Larsen. Co-modelling and co-simulation in theengineering of systems of cyber-physical systems. In Proc. of the Int. Conf. onSystem of Systems Engineering, 2011.

7. C.M. Legaard, C. Gomes, P.G. Larsen, and F. F. Foldager. Rapid prototyping ofself-adaptive-systems using python functional mockup units. In Proc. of the 2020Summer Sim. Conf., 2020.

8. M. Palmieri, C. Bernardeschi, and P. Masci. A flexible framework for fmi-basedco-simulation of human-centred cyber-physical systems. In Fed. of Int. Conf. onSoft. Tech.: Applications and Foundations, 2018.

9. C. Thule, C. Gomes, and K. G. Lausdahl. Formally verified fmi enabled externaldata broker: Rabbitmq fmu. In Proc. of the 2020 Summer Simulation Conference,San Diego, CA, USA, 2020. Soc. for Comp. Sim. Inter.

10. C. Gomes, C. Thule, D. Broman, P.G. Larsen, and H. Vangheluwe. Co-simulation:a survey. ACM Computing Surveys, 51(3), 2018.

11. P.G. Larsen, J. Fitzgerald, J. Woodcock, P. Fritzson, J. Brauer, C. Kleijn,T. Lecomte, M. Pfeil, O. Green, S. Basagiannis, and A. Sadovykh. Integratedtool chain for model-based design of cyber-physical systems: The into-cps project.In Proc. of the Int. Workshop on Modelling, Analysis, and Control of ComplexCPS, 2016.

12. Modelica Association. Functional Mock-up Interface for Model Exchange and Co-Simulation. https://www.fmi-standard.org/downloads, October 2019.

13. J. Fitzgerald, C. Gamble, P.G. Larsen, K. Pierce, and J. Woodcock. Cyber-PhysicalSystems design: Formal Foundations, Methods and Integrated Tool Chains. InProc. of the FME Workshop on Formal Methods in Soft. Eng., Italy, May 2015.

14. C. Thule, M. Palmieri, C. Gomes, K. Lausdahl, H.D. Macedo, N Battle, and P.G.Larsen. Towards reuse of synchronization algorithms in co-simulation frameworks.In Proc. of the Int. Conf. on Soft. Eng. and Formal Methods. Springer, 2019.

15. F. Foldager, O. Balling, C. Gamble, P.G Larsen, M. Boel, and O. Green. Designspace exploration in the development of agricultural robots. In AgEng Conference.Wageningen, volume 54, 2018.

16. M. Frasheri, H. Ejersbo, C. Thule, and L. Esterle. Rmqfmu: Bridging the realworld with co-simulation. Technical Report arXiv:2107.01010 [cs.RO], 2021.

https://www.fmi-standard.org/downloads

Genetic Algorithms for Design Space Exploration ofCyber-Physical Systems: an Implementation in

INTO-CPS

Maximilian Rose and John Fitzgerald

School of Computing, Newcastle University, United Kingdom{M.Rose, John.Fitzgerald}@newcastle.ac.uk

Abstract. We describe the initial implementation and evaluation of a facility tosupport Design Space Exploration (DSE) over multi-paradigm models of cyber-physical systems, using Genetic Algorithms (GAs). Existing features of the INTO-CPS toolchain have been updated to allow concurrent exploration of DSE param-eters and a new GA-based approach to DSE. Based on a comparison with thepreviously reported DSE of the multi-model of an agricultural robot, the devel-oped solution appears to reduce the time required to perform a DSE by providinguser-configurable parameters for the number of concurrent DSE simulations. Italso provides a modular, user-extendable, GA-based DSE system that can be usedwith the INTO-CPS toolchain.

Keywords: digital twin, FMI, co-simulation, Design Space Exploration, Genetic Al-gorithms

1 Introduction

In the model-based design of Cyber-Physical Systems (CPSs), developers work withsemantically heterogeneous collections of models (which we term multi-models) de-scribing computational and physical processes. Co-simulation is thus of particular in-terest as methods for evaluating design alternatives in order to find those that best meetcombinations of requirements (e.g., for performance or cost).

In this context, a design space is a collection of multi-models that represent al-ternative designs for a proposed CPS. These designs differ from one another in termsof the values of design parameters. The process of Design Space Exploration (DSE)aims to identify the combinations of values for design parameters that specify designswhich are optimal with respect to a defined criterion. In the DESTECS project DSE isseen as a way of managing the complex embedded systems design process [7]. Morerecently, DSE has been implemented in the Integrated Toolchain for Cyber-PhysicalSystems (INTO-CPS1) [10].

Exhaustive evaluation of all designs in a design space is infeasibly costly for allbut the simplest multi-models. For even moderately complex multi-models, it is neces-sary to hunt for optimal designs by iteratively selecting subsets of designs, evaluating

1 https://into-cps.org/

https://into-cps.org/

82-2ex Maximilian Rose and John Fitzgerald

these and using the results to suggest where an optimal design might lie. Genetic Algo-rithms (GAs), which implement optimisation methods inspired by Darwinian principlesof natural selection, may provide significant performance improvements over exhaus-tive methods. There are, for example, suggestions that GAs are feasible for DSE in thehardware/software codesign domain [13]. However, an implementation of GA-basedDSE has not yet been integrated in the INTO-CPS toolchain to permit the evaluation ofthis approach.

This paper reports an implementation and initial evaluation of GAs in DSE overmulti-models in the INTO-CPS toolchain2. In order to provide this implementation, itwas necessary to lay groundwork by updating the INTO-CPS DSE infrastructure, iden-tify GAs relevant to DSE, implement such algorithms in a general setting and integratethem with the toolchain. An evaluation was undertaken, comparing a GA-based ap-proach to the exhaustive DSE for the Robotti agricultural robot previously described byBogomolov et al. [2].

We assume that the reader is familiar with the principle of collaborative modellingand co-simulation based on the Functional Mockup Interface (FMI) standard employedin INTO-CPS. Section 2 provides further background on DSE and GAs in INTO-CPS.Section 3 briefly describes technical updates made to INTO-CPS in order to accommo-date our GA-based approach. We then are in a position to describe the GA approachesimplemented in INTO-CPS (Section 4) and the evaluation on the Robotti data (Sec-tion 5). We discuss the study so far and potential directions for future work in Section 6.

2 Background

2.1 DSE support in the INTO-CPS Toolchain

For a brief account of related work on DSE, we direct the reader to [6] and more re-cently [2]. In the DESTECS project on embedded systems, which was largely focussedon binary multi-models (co-models) the need for a systematic approach to DSE wasidentified, using the Automated Co-model Analysis features of the DESTECS tool. Arange of techniques for design parameter selection were considered, including the useof screenign experiments, fractional experiments using orthogonal matrices (Taguchimethods), space-filling searches as well as exhaustive parameter sweeps [7].

INTO-CPS emerged as an open toolchain permitting the co-simulation of multi-models of CPSs that conform to the Functional Mockup Interface (FMI) standard3. Thetoolchain supports a workflow in which a multi-model architecture represented in aSysML profile supported by the Modelio tool which can then be used to generate theFMI specifications for the interfaces of constituent models. Co-simulation is managedthrough a Co-simulation Orchestration Engine (COE) known as Maestro [17]. A web-based application can be used to launch the COE.

The purpose of DSE is to evaluate multi-models under a series of co-simulations,in order to arrive at a ranking against an objective (or cost) function which expresses

2 The implementation code is available at https://github.com/INTO-CPS-Association/dse_scripts

3 https://fmi-standard.org/

https://github.com/INTO-CPS-Association/dse_scripts

https://github.com/INTO-CPS-Association/dse_scripts

https://fmi-standard.org/

Genetic Algorithms for Design Space Exploration of CPS 83

the selection criteria that are important to the developer [6]. One or more multi-modelsselected on the basis of a ranking can then be taken forward for further design. In thisway, DSE can be used to reduce the range of prototypes that may have to be producedand on which expensive tests may have to be performed.

INTO-CPS supports DSE by running co-simulations on selected points in the designspace. Scripts implemented in a combination of Java and Python allow the analysis to beperformed from the INTO-CPS Application, and the ranking to be displayed. Fitzgeraldet al. [6] offer a systematic approach to DSE in INTO-CPS through a specific SysMLprofile and demonstrate feasibility of a simple GA on a line follower robot as a firstclosed-loop algorithm for INTO-CPS. In this paper, we describe a recent and morecomprehensive integration of GA techniques in INTO-CPS.

2.2 Genetic Algorithms

GAs are proposed by Holland [8] as an adaptive heuristic approach analogous to thebiological process of natural selection [3] whereby the selection of genes occurs throughinternal competition leading to unfavourable genes eventually being removed from thegene pool and convergence to an optimum with some variation [15]. Fig. 1 shows themain loop of a typical GA that aims to select a set of optimal candidate solutions. Eachiteration produces a new generation of candidate solutions. These are evaluated and aselection is made by a process that typically ensures the fitter candidates are have ahigher chance of selection. In order to create the next generation, these are mated, andgenetic operators such as crossover and mutation are applied. Elimination may removeinfeasible solutions before a decision is made on termination. This may be based ona range of criteria such as having reached specified number of generations or whenimprovements between iterations are below a threshold.

Fig. 1. Outline of a Genetic Algorithm.

In our DSE context, a solution is a candidate design represented in terms of thevalues of the design parameters of interest. The genetic operators that have been imple-mented are described in Section 4 below.

3 Updating INTO-CPS to Support GA-based DSE

Prior to the work reported here, the main means of performing DSE in INTO-CPSwas by using outdated Python 2 scripts and an exhaustive search method. Indeed, theauthors of [2] have noted the overhead caused by the reliance on Python 2. In addition,Python 2 support ceased in 2020. The goal of the work reported here was to update the


DSE scripts to use modern Python 3, to improve usability, and to provide INTO-CPSusers with more options based on GAs when performing DSE.

The code that is provided with INTO-CPS for DSE consists of 8 scripts that interactwith each other as shown in Fig. 2. Algorithm Selector is used to select the desired DSEalgorithm to use. The selected algorithm script then calls helper functions from theCommon script to run and evaluate a specific configuration using the COE. To producea human readable output in both CSV and HTML the Output HTML script is run.

Fig. 2. Outline of DSE Scripts in INTO-CPS.

Conversion to Python 3. It was decided that, to maintain compatibility, the overallstructure of the scripts would remain the same with only minor changes to improveflexibility. Any new options that would be added would also default to the originalbehaviour of the scripts. The largest change in respect to Python 2 and 3 was the re-quirement for brackets on print statements and also the http library used to connect tothe COE had also changed its name. As such the upgrade process was not difficult andis fully compatible with anything that may be reliant on the way the original Python 2scripts executed.

Multi-Threading The theoretical speed improvements associated with multi-threadingare exponential with increasing thread counts to a threshold, specifically the time takenfor a single simulation. Although the original scripts and COE were not designed withmulti-threading in mind, they are mostly thread safe, and those parts that were not(mainly to do with writing simulation results to a file ready for ranking) required someupdates to properly implement the multi-threading.

From Fig. 3 indicates that the threading performs as expected with a test controllerthat simply echoes its input and with the case study described in Section 5.1.

More options and flexility were also added to the upgraded DSE scripts:

Splitting Results Generation. Previously there was no option to prevent the genera-tion of the HTML & CSV results files. This has been changed so that the user mayspecify how they would like their results to be output.

Arguments. As it is possible to run the DSE scripts easily via a command line, it isdesirable that they produce useful errors. With the old Python 2 scripts the erroroutputs were simply from Python telling the user that a command was not found.With the updated scripts this has been changed so that the script will notify the usernot only that a parameter is missing but also which parameter is missing. A -helpcommand has also been added so that the user can see all the options available tothem.


Fig. 3. Actual Multi-Threaded Performance

Progress Bar. A progress bar was added to the console output of the scripts to give theuser some sense that the scripts are running and things are happening.

4 Implementation of Genetic Algorithms in INTO-CPS DSE

Several operations have been implemented in order to allow the straightforward con-struction of GAs for DSE. We here describe their functionality in terms of the GAelements described above in Section 2. These have been selected mainly on the groundsof their extensive use in the literature. We do not yet have sufficient experience withDSE in INTO-CPS to have developed heuristics on which combinations of operatorsmight be used in different design spaces.

4.1 Selection Operators

To simulate the selection of mating pairs in a meaningful way a fitness function mustbe created and then applied to each organism in the generation. In the case of DSE withINTO-CPS, the fitness function is expressed in terms of the results obtained from eachco-simulation.

Random This is the simplest operation and does not actually require the ranking formthe previous generation. This algorithm is included for completeness; it may be usefulto start the algorithm with in situations in which it may not be practical to create aranking for organisms in generation 0 in some scenarios.

Roulette Wheel Initially proposed by Holland [8], this operation can be thought of interms of a pie chart with the more fit individuals taking up a larger portion of the pie.From this the probability of a specific organism being chosen is controlled by Eqn. 1where pi is the probability of the ith organism being selected relative to its fitness, fi.

Eqn. 1 does assume that a higher fitness value is always better. This may not be thecase for the diverse applications of INTO-CPS. Thus a reversed version of this was also


implemented that offers a higher probability of selection for organisms with a lowerfitness value.

pi =fi∑nj=0 fj

(1)

Ranked This is a variation of Roulette Wheel that, instead of using the raw fitnessvalue for ranking, will instead normalise the values first [12]. This solves an issue thatwhen using the raw fitness value it is possible for a single organism to dominate theprocess by having an extremely high fitness.

The ranking is a total ordering from 1 to n where n is the organism with the worstraw fitness value. Organisms having the same fitness are ranked in the order they areprocessed. As can be seen in Fig. 4, ‘a’ has an extremely high fitness value compared tothe other organisms and thus dominates the selection being selected 74.2% of the time.When normalised, ‘a’ is still chosen 33.6% of the time.

Organism Fitness Normalised FitnessTimes Selected

Roulette Wheel Rankeda 300 1 742 (74.2%) 336 (33.6%)b 50 2 118 (11.8%) 257 (25.7%)c 30 3 52 (5.2%) 196 (19.6%)d 35 4 78 (7.8%) 127 (12.7%)e 5 5 10 (1%) 84 (8.4%)

Fig. 4. Difference in selection bias between using raw fitness values and normalised values.

Tournament This operator is based on the bracketing systems that are seen in sportswhere x individuals are chosen to compete between each other from all n individualsin the population.

The size, x, of each tournament should be a small number else the fittest individualin the population will dominate, leading to premature convergence. This value will beleft to be user defined in the implementation and taken as an argument but with a defaultvalue of 2.

To simplify implementation, this operator will accept only the normalised rankings.This is because using the normalised ranking, an organism with a ranking of 1 willalways be more fit than an organism with ranking 2, ranking 2 is always more fit than 3,etc. Thus, we do not need to account for if the DSE is looking to minimise or maximisethe fitness in the implementation of the operator as it has already been dealt with by thenormalisation operation.

4.2 Mating Operators

A traditional GA would select from the gene pool created by the previous selectionoperators to create mating pairs. However, due to the implementation in this paper, thisstep is combined into the selection operation.


4.3 Crossover OperatorsIn our DSE setting, we think of the parameter values that characterise a given designas a sequence, analogous to chromosomes. Each parameter value in the sequence isa key to a specific design characteristic. Crossover operators emulate the recombina-tion stage in natural reproduction. During this stage in natural systems two homologouschromosomes come together touching chromatids. Where these chromatids touch, theybreak and re-join exchanging DNA between the two chromatids [11]. In computationalcrossing the process does not need to simulate chromosomes moving around and touch-ing each other. Instead we can select contiguous keys (which in our setting are designparameter values) from each parent to add into the child. Note that this process doesnot change the values of the inherited parameters: each inherited value comes from aparent.

n-Point One of the simplest crossing functions that mimics biology is n-point crossover.The simple case, 1-point crossover, is the traditional operation for a GA to imple-ment [14]. To give the user more options when using the GA library, it was decidedto implement the more general n-point version.

This operator selects n keys from the parent, takes contiguous keys from the parentand adds them to the child. This process is illustrated in Fig. 5. In the diagram eachcoloured rectangle represents a key in the parent. In this example 3-point crossing isselected so three points to cross are selected, shown in yellow, blue and purple. Thechild is created with contiguous keys from A until the first crossing point is reached,when it is reached, we switch to taking keys from B, this process repeats until there areno keys left.

Fig. 5. Example of n-point crossover.

Uniform This crossover strategy differs from n-point in that it uses a bitmask to de-cide which genes are inherited by the child. Fig. 6 illustrates this strategy. Again, thecoloured rectangles represent genes. The crossing mask would be a randomly gener-ated bit string with the value 0 (white rectangles) meaning take a gene value from Aand value 1 (black rectangles) meaning take a gene from B.

BLX Blend Crossover [5,16] is a more advanced operator that also incorporates ele-ments of mutation into the crossover phase. This incorporation of changing gene val-ues might argue for this being a mutation operation rather than a crossover operation.


Fig. 6. Example of uniform crossover.

However, due to the need for two separate individuals, we will treat it as a crossoveroperation (the same argument can also be made for the SBX operator discussed below).

This operation takes the two parent gene values and uses them as a range from whichthe child’s value can be selected (See Fig. 7). The extents of the ranges past the parentvalues are controlled by α. The child value, g, is selected from this range as shown byEqn. 2. Common values of α for this equation are 0.5 [5] and 0.366 [16], 0.5 is chosenas the default value.

Fig. 7. BLX crossover.

g ∈ (min (p1g, p2g)− α|p1g − p2g|,max (p1g, p2g) + α|p1g + p2g|) (2)

SBX This is similar to BLX crossover however biases the child gene value to be closerto the parent gene values as depicted in Fig. 8. This operation is designed for real valuegenes proposed in [4]. The distribution is achieved using Eqns. 3 and 4. The value β isused to control the spread of the resulting gene with η called the distribution index, withlarger values meaning the resulting offspring values should be closer to the parents.

Fig. 8. SBX crossover.

β =

{µ ≤ 0.5, (2µ)µ > 0.5, 1

2(1−µ)

} 1η+1

(3)

β = 0.5×{(1 + β)p1g + (1− β)p2g(1− β)p1g + (1 + β)p2g

(4)


4.4 Mutation Operators

Aside from the BLX and SBX operations that some may consider mutation operations,the only realistic option is to select values from a normal distribution in the case of areal value algorithm, as a bit flipping operation would effectively result in a randomsearch. Selection from a normal distribution can be done in two ways: either absoluteselection as in Eqn. 5 or a relative distribution as in Eqn. 6.

It is recommended that small values of σ are normally used, as larger values willeffectively turn the algorithm into a random search. Larger values of σ are do not ac-curately represent the reality of evolution since large changes do not happen suddenly.Using a larger σ value can be used to prime the search space in the initial generation tohelp ensure that the full search space is explored.

As the fitness of organisms converges, it is possible to decrease the value of σ slowlyto yield a process like simulated annealing.

g′ = g + Φ(0, σ) (5)

g′ = g × (1 + Φ(0, σ)) (6)

4.5 Elitism

This operator ensures that a certain percentage of the best individuals from the previ-ous generation will be carried over into the next generation. This is to ensure that thebest individual will always be produced by the algorithm no matter what stage it wasgenerated at. Organisms can be chosen at random for removal as they have not yet beenranked. Whilst this does introduce the possibility of removing the new best organism itis probable that it will be recreated in the generation after and the probability of it beingremoved a second time is low especially with large generation sizes and a low numberof organisms being carried over from the previous generation.

4.6 Diversity Control

Diversity control is the mechanism that attempts to prevent premature convergence byapplying penalties to organisms based on how similar they are to the rest of the pop-ulation. This paper implements the first operator from the GOSET manual [1]. Thediversity operator is based on calculating a mean distance between all organisms, us-ing that to calculate a threshold and then using that threshold to count the number ofneighbours an organism has. Although itsO(n2) complexity makes it possibly the mostexpensive part of the entire GA implementation, it is highly accurate.

4.7 Controller

There are several possible solutions to controlling the iterative process. We can stopafter a set number of generations, or after a fitness threshold is reached by n organisms,or after there has been little change is fitness for n generations (convergence stopping),


Fig. 9. Diversity control.

or after the algorithm has run for a certain amount of time. The final option does notappear to be realistic in the context of DSE for INTO-CPS, but the other options havebeen implemented:

n Generation. This is the simplest of the controllers to implement is stopping after ngenerations. This approach whilst guaranteed to stop at some point is not guaran-teed to produce the fittest individuals.

Fitness Threshold. This requires a certain percentage of organisms to pass a thresholdbefore stopping. This controller is guaranteed to produce organisms of at least someminimum quality, however, this alone does not guarantee the algorithm to stop. Toguarantee that the algorithm will eventually stop we have implemented a limit tothe number of generations that is configurable by the user.

Convergence. To find if the search has GA has converged, we need two things: firstly,the threshold at which we consider things to have converged, and secondly the num-ber of previous generations to look at to check for convergence. As convergence iswhen the average fitness of n sequential generations is relatively similar we keepa list of the previous n generations in the algorithm and then check the standarddeviation of that list to check for convergence.

5 Evaluation on the Robotti Case

5.1 Existing DSE Algorithms against Upgraded Versions

Fig. 10 shows the results of speed testing done using Robotti data, which suggest thatthe upgrades described in Section 3 have had a positive effect on runtime. Each scenariowas run 5 times to find the average runtime. As no functionality was changed the resultsof the ranking, HTML and CSV output are also consistent with the original version,apart from a change in the order of the columns in the outputs, which we suggest maybe caused by a change in how sort operates between Python 2 and 3, although we cannotfind evidence to support this.

5.2 GA-based DSE of Robotti

Now we consider the use of GA-based DSE on Robotti. The GA setup was initially thesame as for the test functions used to validate the implementations of the GA operations.


Scenario Original Python 3 Python 3 ThreadedAverage Time / s Average Time / s Diff / % Threads Average Time / s Diff / %

Sin 2 2977.611 2988.454 0 10 496.001 −83Speed Ramp 1 2054.087 1756.910 −14 10 329.819 −84Speed Step 1 1960.502 1360.295 −31 5 356.744 −82Turn Ramp 1 2389.692 1604.571 −33 8 359.412 −85

Fig. 10. Average Run Time Comparisons

However, as can be seen in 11 this did not produce the expected results. The issue hereis that the diversity control operation was too aggressive, preventing convergence by es-sentially turning the algorithm into an exhaustive search. This is suggested by the meanline in Fig. 11c. At around 35 generations the mean fitness begins to rise and maximumfitness remains relatively constant, suggesting that the search is not converging. Thishighlights the need for parameter tuning to ensure GAs run effectively. To resolve this,the diversity control was disabled as it was found it was not needed for this simulation.

(a) Gene values across all gen-erations.

(b) Fitness against gene values. (c) Fitness trends.

Fig. 11. Initial Speed Ramp results.

Fig. 12 summarises the results of DSE in each of the Robotti scenarios comparedagainst the best found by the exhaustive search. As can be seen in Fig. 12 the GA-basedDSE performs as expected, finding the best parameter combinations within acceptablemargins. As we can see, although in the order of ×10−6, the genetic algorithm-basedsearch is able to find slightly better parameter combinations for both the Sin and TurnRamp scenarios. This is a consequence of the ability to select from a continuous rangeof values, so the GA found a combination that could not have been evaluated in theexhaustive DSE. The results also highlight that DSE scenarios must be selected withcare, as if DSE was only performed on Speed Step then we would come to the falseconclusion that the chosen parameters do not affect the running of the robot.

6 Discussion and Future Work

We have provided an implementation of GAs as a basis for DSE in INTO-CPS, andevaluated this using an established case study. Some groundwork had to be laid in orderto enable GA-based DSE, including the updating of existing scripts and the introductionof threading. However, the implementation appears to be viable.


ScenarioExhaustive Best Combination GA Best Combination

Cross Track Error DifferencecAlphaF µ Mass cAlphaF µ Mass

Sin 2 20000 0.7 3000 20000 0.70 3000 −1.46× 10−6

Speed Ramp 1 38000 0.4 1000 37019 0.45 1038 8.40× 10−2

Speed Step 1 20000 0.3 1000 34725 0.45 1027 0.00Turn Ramp 1 20000 0.3 3000 20000 0.30 3000 −9.21× 10−7

Fig. 12. Exhaustive Best Combination vs GA Best Combination. Results are rounded appropri-ately. Cross track error difference calculated as Exhaustive−GA.



Fig. 13. Sin Results.

Aside from increasing the sophistication of the GA operators now implementedfor INTO-CPS DSE, this opens up a range of other areas of future work. First, CPSssuch as Robotti may develop faults over time and it may be useful for the INTO-CPStoolchain to integrate Fault Space Exploration (FSE) in addition to DSE. This could beused to find the level of fault tolerance that a system can withstand before resulting inundefined, possibly unsafe, behaviour. It is reasonable to think of this in the contextof resilience modelling [9]. Second, GAs are part of the broader class of EvolutionaryAlgorithms (EAs), suggesting that a wider examination of the orle of EAs in DSE wouldbe worthwhile. Third, as a CPS may be deployed in many different environments it isnot unreasonable to find that a set of parameters may be optimal for one scenario but notanother, as shown in the DSE results for Robotti. Thus, it may be useful to implement



Fig. 14. Speed Ramp results




Fig. 15. Speed Step results.



Fig. 16. Turn Ramp results.


an approach that can automatically produce a ‘compromise’ set of parameter values thatare optimal across all scenarios for a given CPS. This could take the results of DSEs asinput and produce an ‘average organism’ from this data.

Acknowledgements We acknowledge the European Union’s support for the INTO-CPSand HUBCAP projects (Grant Agreements 644047 and 872698). Among many col-leagues we are particularly grateful to Ken Pierce at Newcastle University, CasperHansen at Aarhus University, and to AgroIntelli for permission to use Robotti data.Finally, we are grateful to the reviewers for their questions and helpful suggestions.

References

1. Academy, U.S.N., of Electrical, P.U.S., Engineering, C.: 5.2 Diversity Control, pp. 36–37.Purdue University School of Electrical and Computer Engineering (08 2007)

2. Bogomolov, S., Fitzgerald, J., Foldager, F.F., Gamble, C., Larsen, P.G., Pierce, K., Stankaitis,P., Wooding, B.: Tuning robotti: the machine-assisted exploration of parameter spaces inmulti-models of a cyber-physical system. In: Fitzgerald, J., Oda, T., Macedo, H.D. (eds.)Proc. 18th International Overture Workshop. pp. 50–64. arXiv ePrint 2101.07261

3. Darwin, C., Kebler, L.: On the Origin of Species by Means of Natural Selection, or, thePreservation of Favoured Races in the Struggle for Life. J. Murray (1859)

4. Deb, K., Agrawal, R.B.: Simulated binary crossover for continuous search space. ComplexSystems 9 (06 2000)

5. Eshelman, L.J., Schaffer, J.D.: Real-coded genetic algorithms and interval-schemata. In:WHITLEY, L.D. (ed.) Foundations of Genetic Algorithms, Foundations of Genetic Algo-rithms, vol. 2, pp. 187–202. Elsevier (1993), https://www.sciencedirect.com/science/article/pii/B9780080948324500180

6. Fitzgerald, J., Gamble, C., Payne, R., Lam, B.: Exploring the Cyber-Physical Design Space.In: Proc. INCOSE Intl. Symp. on Systems Engineering. vol. 27, pp. 371–385. Adelaide,Australia (2017), http://dx.doi.org/10.1002/j.2334-5837.2017.00366.x

7. Gamble, C., Pierce, K.: Design space exploration for embedded systems using co-simulation.In: Fitzgerald, J., Larsen, P.G., Verhoef, M. (eds.) Collaborative Design for Embedded Sys-tems, pp. 199–222. Springer Berlin Heidelberg (2014)

8. Holland, J.H.: Adaptations in Natural and Artificial Intelligence. MIT Press (1992)9. Jackson, M., Fitzgerald, J.: Towards Resilience-Explicit Modelling and Co-simulation of

Cyber-Physical Systems. In: The 1st Workshop on Formal Co-Simulation of Cyber-PhysicalSystems (CoSim-CPS). Trento, Italy (September 2017)

10. Larsen, P.G., Fitzgerald, J., Woodcock, J., Gamble, C., Payne, R., Pierce, K.: Features of in-tegrated model-based co-modelling and co-simulation technology. In: Bernardeschi, Masci,Larsen (eds.) 1st Workshop on Formal Co-Simulation of Cyber-Physical Systems. LNCS,Springer-Verlag, Trento, Italy (September 2017)

11. Ltd, P.E.: Crossing Over. Pearson Education Ltd (2015)12. Obikto, M.: Ix. selection (1998), https://www.obitko.com/tutorials/

genetic-algorithms/selection.php13. Panerati, J., Sciuto, D., Beltrame, G.: Optimization Strategies in Design Space Exploration,

pp. 189–216. Springer Science+Business Media Dordrecht (2017)14. Sivanandam, S.N., Deepa, S.N.: Crossover (recombination). In: Introduction to Genetic Al-

gorithms. Spinger (2008)



http://dx.doi.org/10.1002/j.2334-5837.2017.00366.x

http://dx.doi.org/10.1002/j.2334-5837.2017.00366.x

https://www.obitko.com/tutorials/genetic-algorithms/selection.php

https://www.obitko.com/tutorials/genetic-algorithms/selection.php


15. Sivanandam, S.N., Deepa, S.N.: Genetic algorithms. In: Introduction to Genetic Algorithms.Spinger (2008)

16. Takahashi, M., Kita, H.: A crossover operator using independent component analysis forreal-coded genetic algorithms. In: Proceedings of the 2001 Congress on Evolutionary Com-putation (IEEE Cat. No.01TH8546). vol. 1, pp. 643–649 vol. 1 (2001)

17. Thule, C., Lausdahl, K., Gomes, C., Meisl, G., Larsen, P.G.: Maestro: Theinto-cps co-simulation framework. Simulation Modelling Practice and Theory 92,45 – 61 (2019), http://www.sciencedirect.com/science/article/pii/S1569190X1830193X

http://www.sciencedirect.com/science/article/pii/S1569190X1830193X


Multi-Objective Optimisation Support forCo-Simulation

Aiden Stanley1 and Ken Pierce1

School of Computing, Newcastle University, United [email protected],[email protected]

Abstract. When designing Cyber-Physical Systems (CPSs) using Model-BasedDesign (MBD) techniques, Design Space Exploration (DSE) can be used to dis-cover optimal configurations. However, due to the computational complexity ofco-simulations, this process can take a considerable amount of time, perhaps mak-ing it an unviable tool for some time-constrained projects. This paper reportson the integration of a Multi-Objective Optimisation (MOO) library to enhancethe DSE features of INTO-CPS, a tool chain for design of CPSs based on co-simulation. The paper demonstrates that the new feature functions correctly byreplicating results of a previous study, however the true benefits of MOO are notdemonstrated and this paper suggests new MOO-specific case studies are neededfor the INTO-CPS example repository to demonstrate these benefits.

1 Introduction

Modelling and Simulation are important and powerful tools when developing systemsof any kind, especially within the field of engineering, and increasingly so in the field ofsoftware development. Simulating models of the systems we design allows us to gain asolid understanding of how they might behave once deployed in the real world. This isof particular importance for Cyber-Physical Systems (CPSs) [7], where software mustinteract with hardware and it is difficult to test exhaustively and in a non-destructiveway.

One of the key benefits of using simulation in this way is the ability to rapidly testdifferent design parameters of a system without having to first make the financial andtime commitments of creating or changing a physical prototype. This places simula-tion as an invaluable tool for performing Design Space Exploration (DSE). The ‘designspace’ of a system can be defined as the set of all configurations of its ‘design parame-ters’, and therefore DSE is the act of exploring and assessing the range of configurationspresent in this design space, with the goal of discovering ones in which the system meetscertain performance criteria most optimally.

DSE is one feature of INTO-CPS [6], a tool chain for model-based design of CPS,built around a co-simulation engine called Maestro [9]. INTO-CPS currently offers ex-haustive search (i.e. all combinations of a given set of design parameters), and geneticsearch (solving through natural selection of the fittest designs). Depending on the com-plexity of the model in question, the number of simulation runs needed to perform thesesearches accurately enough can make this process increasingly time consuming.

Multi-Objective Optimisation Support for Co-Simulation 97

In industry, it is often paramount that tasks take the minimum amount of timeneeded to be performed properly, to keep costs down and to ensure that crucial dead-lines are met. There are a wide variety of Multi-Objective Optimisation (MOO) thatattempt to efficiently search for optimal solutions in complex design spaces. Many ofthese are implemented in existing, well-maintained libraries (see Section 2.2). This pa-per describes work in extending the DSE capabilities of INTO-CPS with features thatattempt to widen the range of DSE options for users of INTO-CPS.

By incorporating state-of-the-art MOO library to INTO-CPS, the user can nowchoose from the most appropriate and up-to-date optimisation algorithm for their CPSmodel, allowing them to discover optimal parameter configurations in the most efficientway possible. Furthermore, integrating an open-source, third-party library would allowthe application to be easily updated to stay up to date with the most cutting-edge al-gorithms, eliminating the need to manually implement newly released ones from theground up.

This paper also applies the MOO support to the Robotti optimisation case study,which was reported as a previous workshop [1]. This case study demonstrates that theMOO integration works, but does not demonstrate the full benefits that MOO couldhave on co-simulations with larger design spaces and more competing objectives thatrequire trading-off.

This remainder of the paper is structured as follows. Section 2 provides some back-ground detail on INTO-CPS and MOO. Section 3 describes integration of a MOO li-brary into the INTO-CPS framework. Section 4 provides a refresher on the Robottioptimisation case study. Section 5 compare the results from the previous Robotti DSEwith those form the new MOO feature. Finally, Section 6 presents from conclusions andfuture work.

2 Background

This section provides background information on the INTO-CPS toll chain and its cur-rent Design Space Exploration (DSE) support, and introduces the concept of Multi-Objective Optimisation (MOO) and existing MOO libraries.

2.1 INTO-CPS and Design Space Exploration (DSE) Support

INTO-CPS is an integrated tool chain for comprehensive Model-Based Design (MBD)of Cyber-Physical Systems (CPSs)” [6]. INTO-CPS is based around the FunctionalMock-up Interface (FMI)1 that allows collaborative models of CPS to be defined, con-figured, and co-simulated. INTO-CPS is comprised of two main components:

1. A front-end called the INTO-CPS Application that allows for configuration andexecution of co-simulations.

2. A back-end program called the Co-simulation Orchestration Engine (COE) calledMaestro [10], which provides functionality for, and oversees, the actual executionof co-simulations.

1 https://fmi-standard.org/

https://fmi-standard.org/

98-2ex Aiden Stanley and Ken Pierce

Around the core of co-simulation, INTO-CPS provides other useful functionalitiessuch as automatic Design Space Exploration (DSE), in which multiple co-simulationsare executes automatically and the results compared. This functionality is provided by aset of Python scripts which can be downloaded from within the INTO-CPS Application.The INTO-CPS Application and DSE scripts provide the ability to configure DSE usingthe following:

Search Algorithm The ability to choose between which optimisation algorithm shouldbe used to perform the search. The currently available choices are ‘ExhaustiveSearch’ and ‘Genetic’.

Experiment Parameters The ability to define which parameters of the model shouldbe considered part of the design space and their bounds.

Parameter Constraints The ability to define constraints on the varying parameterse.g., “parameter A should always have a value greater than parameter B, to beconsidered a feasible solution.”

External Script Objectives The ability to point to a script that uses the results of asimulation run to calculate the model’s fitness according to a custom objective.

Results Output Once a run of DSE has completed, the current implementation outputsa ‘Pareto front’ plot showing the optimal solutions regarding the objectives defined,and the set of input parameters required to recreate these optimal configurations.

The work reported in this paper aims to replicate these features using a MOO library,with the aim of providing the user with a seamless option to apply further algorithms intheir exploration of design space.

2.2 Multi-Objective Optimisation (MOO)

Multi-Objective Optimisation (MOO) is the act of optimising a system as accordingto multiple specific objective criteria [4]. A solution is considered Pareto optimal if noaspect of that solution can be improved without making another aspect of the solutionworse off. Due to the nature of this, any optimisation problem that has more than oneobjective will most likely have more than one Pareto-optimum solution. MOO algo-rithms may attempt to find a representative set of optimisations, to quantify the tradeoff, or pick one solution based on the preferences of the Decision Maker (DM).

The work reported in this paper links a MOO library to the INTO-CPS tool chain,thus we do not go into details regarding how various MOO algorithms work. The coreconcept is the same as the existing genetic DSE offered by INTO-CPS — there is a setof possible parameters and a method which proposes solutions, and selects and refinesthese based on the results of co-simulation. Some MOO algorithms are based on geneticalgorithms, such as the popular Non-dominated Sorting Genetic Algorithm-II (NSGA-II) [2] and Strength Pareto Evolutionary Algorithm 2 (SPEA-2) [11] algorithms. OtherMOO algorithms are not genetic, such as Particle Swarm Optimization and SimulatedAnnealing [8].

When selecting a library for the implementation, a number of options were consid-ered. The three main options were MOEA, jMetal and pymoo. MOEA (Multi-Objective


Fig. 1. Class diagram of the inheritance hierarchy of Algorithm

Evolutionary Algorithms) is a free and open-source framework for Java2. It has an ex-tensive list of algorithms and high-quality documentation, however is only two-yearsold and many of the guides are behind a paywall. jMetal3 stands for Metaheuristic Al-gorithms in Java and is similar to MOEA. It has been established for much longer,with its fifth major release in 2015. Documentation is also freely available, althoughsomewhat lacking for the newest version. The pymoo4 framework offers state of the artsingle- and multi-objective optimization algorithms in Python. Although it has excel-lent documentation, it offers fewer algorithms and has not yet reached a major release.Based on this search, jMetal was selected as the most sensible option.

3 Multi-Objective Optimisation (MOO) Support

The support for Multi-Objective Optimisation (MOO) involved creating an interfacebetween the jMetal [3] library and the Co-Simulation Orchestration Engine (COE).Figure 1 shows a diagram of the INTO-CPS Application interacting with the new MOOsupport tool and the COE. Here the frontend can configure a MOO problem, selectingthe algorithm, parameter space, objective scripts and maximum iterations. The jMetallibrary then executes the selected algorithm. When testing solutions proposed by thealgorithm, a co-simulation configuration is generated that configures the COE to runone or more co-simulations. The selected objective scripts are used to feed back intothe jMetal library for the algorithm to generate a new set of solutions. This processrepeats until a given number of iterations is reached.

The jMetal library is object-oriented. The key classes called Problem, represent-ing the MOO problem to be solved, and Solution, representing a possible solutionto the optimisation. In this case, the values of parameters of the model. The Problemclass has a primarily method called evaluate which determines the fitness of a pro-posed Solution. Hence the primary effort of this implementation is to generate co-simulation configurations for Solution instances proposed by the MOO algorithm(s)and evaluate the Problem using the COE and objective scripts.

2 http://moeaframework.org/3 http://jmetal.sourceforge.net/4 https://pymoo.org/

http://moeaframework.org/

http://jmetal.sourceforge.net/

https://pymoo.org/


Fig. 2. Class diagram to showing the role of RunHandler

Each algorithm in JMetal comes with its own builder, which allows you to config-ure and initialise an algorithm. The MOO functionality was built in an object-orientedapproach to wrap these algorithms in an interface. Each algorithm to be imported fromjMetal is represented as its own class, which inherits from an abstract superclass calledAlgorithm. The superclass also contain an abstract method runAlgorithm thatsubclasses are required to implement. Calling this function should run the algorithmand output the results. This superclass contains empty fields for:

– Each operator kind of operator, i.e.:• Crossover• Mutation• Selection

– Every variant of algorithm parameter, e.g.• Crossover probability and distribution index• Mutation probability and distribution index• Mutation probability and distribution index• Population size• Maximum number of evaluations• Etc.

To allow the user or INTO-CPS Application to select an algorithm by name, allpossible algorithms are enumerated in a type called AlgorithmVariant. A classcalled RunHandler takes a string input and attempts to resolve which subclass ofAlgorithm should construct and run. This is shown in the class diagram in Figure 2.

To make the integration with the INTO-CPS Application as simple as possible, theMOO implementation takes a single JSON (Javascript Object Notation) file as its onlycommand line argument. This replicates the style of both the COE and existing DSEscripts. This means that tool integration only requires the INTO-CPS Application tofetch the required information from the user, store that information in a JSON file, thenexecute the MOO solution with this JSON as an argument.


Fig. 3. An example of the input JSON used to configure an MOO search

The input JSON used should contain the following fields (based on the existing DSEJSON file), with an example is presented in Figure 3:

– The path of the COE– The path of the JSON configuring the base co-simulation– The length of the simulation, in seconds– The algorithm to use– An array of the model’s parameters that make up the design space, each with the

following attributes:• Parameter identifier• Upper bound• Lower bound

– An array of the objectives, each with the following attributes:• Type (“SCRIPT” or “PARAMETER”)• Identifier (path to script, or parameter identifier)• A Boolean value for whether the objective should be maximised (false means

minimised)

In addition to basic functionality of running each algorithm offered by jMetal, thesolution also includes concurrent execution of solution evaluation and custom constrainthandling through Python scripts. At the time of writing, the INTO-CPS Application iscurrently being updated to provide a MOO option in the existing DSE interface. Ascreenshot showing this is given in Figure 4. It is expected that this functionality willbe available in the next release of the INTO-CPS Application expected in Q3 2021.

4 Case Study

To demonstrate the work reported in this paper against existing functionality, we usethe same case study as Bogomolov et al. [1] presented at a previous workshop. The


Fig. 4. A screenshot of the INTO-CPS Application showing MOO algorithm choices

paper reports on a pilot study to assess the viability of DSE as a tool in improving thefidelity of a co-simulation model, based on real-world data observed in field trials of thereal system. The study is based on the Robotti unmanned platform developed by AgroIntelligence (Agrointelli) for (see Figure 5). In the remainder of this section, the casestudy and previous results are discussed. Section 5.2 discusses the performance of thecurrent solution and the selection of the MOO algorithms.

The authors note that this study does not exhibit true multi-objective behaviours thatrequire trade off to best demonstrate . We suggest in Section 6 that new case studies arerequired in the INTO-CPS examples repository that do show multi-objective charac-teristics and would better demonstrate the strengths and weaknesses of both the DSEand MOO approaches. We selected this case study because it was readily available andwas previously reported at this venue, which meant that the results could be directlycompared to demonstrate the functionality of the new MOO add on, if not demonstrateits full utility.

In this study, the real Robotti system carried out field trials under four scenarioswith a total of 27 runs. Each scenario is a pattern of control outputs designed to as-sess how Robotti moves under different conditions. The control outputs and responsesof the Robotti are then recorded for use in model optimisation. This data is then usedto optimise the co-simulation by finding which physical parameters in the model mostclosely replicate those in the real Robotti. In this way, the optimisation should delivera co-simulation that better models the actual system. The four scenarios were as fol-lows [1]:


Fig. 5. The Robotti unmanned platform

Speed step: The speed of each wheel is increased incrementally in lock step, resultingin the Robotti driving straight.

Speed ramp: The speed of each wheel is increased smoothly. This also results in astraight path as in the speed step tests.

Turn ramp: The speed of the right wheel is increased smoothly, while the left wheelis kept constant. This results in the Robotti turning left.

Sin: The speed of the left and right wheel is increased and decreased to produce thesinusoidal motion.

The measure of accuracy in this study is the mean cross-track error. Given the knownpath and the simulated path under a given set of parameters, the mean difference be-tween then provides a measure of how well the parameters match the (unknown) at-tributes of the real Robotti. A lower mean difference therefore means a closer represen-tation of reality.

The design space involves three parameters: m_robot, the mass of the Robotti; cAl-phaF, the steering stiffness; and mu, the friction between the wheel and ground. In thestudy, exhaustive search is used for the DSE, with five values for each parameter. Thisresults in 125 co-simulation per scenario, and 1250 for the ten scenarios used. Resultsfor one each of the four scenarios are shown in Figure 6. These show that the Sine andTurn ramp scenarios strongly suggest the same specific parameter values (in the lowerright of the plots). Speed ramp shows a region of optimal solutions, and a secondaryweaker region of good solutions, while Speed step suggests no overall optimal solution.

5 Results

This section presents results of applying the MOO support to the Robotti case studydescribed above, using the four scenarios presented in Figure 6 (sin1, turn_ramp1,speed_ramp1, speed_step1). In order to ensure comparisons with exhaustive search arevalid, and any comparisons of speed were made on the same machine, the exhaustivesearch was re-run for the four scenarios. These results are shown in Figure 7.

Note that the trends found are the same except for turn_ramp1 which found acompletely different trend. To try and account for this discrepancy, two manual co-simulations were run with values of 3000 for m_robot and 20000 for cAlphaF, and muvalues of 0.3 and 0.7, respectively. After running these two co-simulations and evaluat-ing their cross-track error using the objective script, the configuration with a mu valueof 0.3 returned a considerably lower error value (in line with Figure 7b). It is therefore


(a) sin1 Scenario (b) turn_ramp1 Scenario

(c) speed_ramp1 Scenario (d) speed_step1 Scenario

Fig. 6. Mean cross track error of four scenarios against different parameter values [1]

possible that Bogomolov et al. [1] used some different setup that affected these results.In any case, the results from the re-run are used here to ensure comparisons are madeon a consistent setup.

5.1 Optimisation of Robotti using MOO Algorithms

The MOO functionality was then used to optimise these scenarios using two differentalgorithms, and compare against exhaustive search. Given its popularity, the NSGA-II(Non-dominated Sorting Genetic Algorithm-II) algorithm was chosen, as well as theSMPSO (Speed-constrained Multi-objective Particle-Swarm Optimisation) since it isthe most different from the evolutionary approach of NSGA-II. To ensure fairness, theMOO algorithms were initially limited to the same 125 co-simulations as the exhaustivesearch. Table 1 shows the mean cross track error found by the two algorithms and theexhaustive search.

As can be seen, it appears that the MOO algorithms are unable to find optimalsolutions as well as the previous study’s exhaustive search, when completing only 125evaluations. In the sin1 scenario, both algorithms came somewhat close to the error




Fig. 7. Results of a re-run exhaustive search of the four scenarios

values found in that study, but in every other scenario, both algorithms calculated amuch worse optimal amount of error. Plots of the intermediate solutions are given inFigure 8. By inspection, it appears as though the trends are the same as those in Figure 7;sin1 is in the opposite corner to turn_ramp1 and speed_ramp1, and speed_step1 showsno overall trend.

To investigate whether further co-simulations would find as optimal a solution as theexhaustive search, the optimisations for sin1 were repeated using NSGA-II and incre-mentally increasing the maximum number of evaluations. These are shown in Figure 9.It was not until around 750 evaluations that the NSGA-II consistently produces resultsas well as the exhaustive search.

This was somewhat surprising, as we expected that using MOO algorithms wouldfind the most optimal configuration much faster in terms of the number of evaluations.Or failing that, in the same number of evaluations, the MOO algorithm would find amore optimal solution. Since such a small set of values on the range of each parameterwas tested during the exhaustive study, values between those specified would not betested. Because of this, we assumed more optimal solutions would be missed, and thatthe MOO algorithms would find them since they would have the ability to test the fullrange of values possible.


Table 1. Mean cross track error found by algorithm and scenarioScenario Algorithm Mean Cross Track Error

sin1NSGA-II 1.226SMPSO 1.225Exhaustive 1.188

turn_ramp1NSGA-II 5.348SMPSO 5.348Exhaustive 3.640

speed_ramp1NSGA-II 1.508SMPSO 1.498Exhaustive 1.491

speed_step1NSGA-II 3.653SMPSO 3.655Exhaustive 5.348

These findings suggested that, in terms of number of evaluations, the MOO solu-tion was much slower at finding optimal solutions. However, we hypothesise that formodels with many more dimensions, MOO algorithms would converge on optimal con-figurations in fewer evaluations than in an exhaustive search. Due to the nature of theexhaustive search, as the number of dimensions increase, the number of evaluationsincreases exponentially. This is because the number of evaluations is given by:

|D1| · |D2| · |D3| . . . · |Dn|

where N and Dn is the set of values that makes up the nth dimension. If the cardinalityof each set tested is uniform, this would give the exhaustive search a time complexityof O(cn) in terms of the number of evaluations, where n is the number of dimensions.The time complexity of NSGA-II on the other hand is not affected by the number ofdimensions directly, since the time complexity of NSGA-II is given by O(GMN2),where G is the number of generations, M is the number of objectives and N is thepopulation size [5]. While some or all of these three variables would likely need toincrease with the number of dimensions to continue providing accurate optimisationconfigurations, this would not correspond to as much as an exponential increase inevaluations.

Using equation derived above, doubling the number of dimensions of the Robottidesign space to six dimensions would increase the number of evaluations performed bythe exhaustive search from 125 to 31,250. Even if this increase in dimensions requiredby NSGA-II to double the number of generations completed, this search would stillonly comprise of 1,500 evaluations. Because of this, we believe that problems withlarger numbers of dimensions, the MOO solution could be much more efficient at DSEwhen compared with exhaustive search. This however requires further study throughdevelopment of co-simulation examples that exhibit multi-objective behaviour.

5.2 Evaluation of Execution Time

Table 2 compares the execution speed of the exhaustive DSE, along with the NSGA-IIalgorithm running on one and six threads. It shows that when running each method for




Fig. 8. Results of a NSGA-II results for the four scenarios limited to 125 co-simulations

the same number of evaluations, using NSGA-II is slightly slower than using an exhaus-tive search, likely due to the overhead of using the jMetal library. However, because theMOO solution supports parallel execution, the ability to run multiple simulations atonce decreases optimisation time significantly. In this case, utilising six parallel coresdecreased optimisation by a factor of three when compared to single thread evaluation.Note that the most recent version of the existing DSE scripts now support parallel exe-cution, so further benchmarks are required to truly compare approaches, using a rangeof case studies to show the strengths and weaknesses of the two approaches.

6 Conclusions and Future Work

This paper presented work on integrating support for a library of Multi-Objective Op-timisations (MOO) algorithms into the INTO-CPS tool chain to provide additional op-tions for Design Space Exploration (DSE) of models through co-simulation. It demon-strated a working solution and initial integration into the INTO-CPS Application. Thesolution was demonstrated by using two common MOO algorithms (NSGA-II and


(a) sin1 (125 co-simulations) (b) sin1 (250 co-simulations)

(c) sin1 (500 co-simulations) (d) sin1 (750 co-simulations)

Fig. 9. Results of NSGA-II on sin1 with increasing numbers of co-simulations

SMPSO) to explore the design space of the Robotti case study, comparing against re-sults presented at a previous workshop [1].

The results show that the MOO algorithms were able to find similar trends to the ex-haustive search presented previously, however when limited to the same 125 evaluationsby co-simulation, performance was worse overall. The MOO library also introduces anoverhead in execution time, which can be mitigated through parallel execution support.The case study chosen was selected given the previous work on using DSE to explorethe parameter space, and provided a useful benchmark to check the correct functioningof the new MOO features.

The results show that MOO is not suitable for every use case, thus would not replaceexisting DSE solutions for INTO-CPS, but rather complement them. It is likely thatthe MOO solutions would offer an improvement to the time needed to perform DSEon CPS models when the design space in question has a high number of dimensions,in particular where multiple objectives compete and require trading off. In addition,of particular importance is to also apply this approach to case studies with a largerparameter spaces (both in the range of each parameter and the number of parameters).


Table 2. Execution time for exhaustive and NSGA-II algorithmsScenario Algorithm Mean Time / Evaluation (s) Mean Total (s)

sin1NSGA-II (6 threads) 2.2 280NSGA-II (1 thread) 6.1 771Exhaustive (1 thread) 5.3 664

turn_ramp1NSGA-II (6 threads) 3.8 480NSGA-II (1 thread) 10.9 1365Exhaustive (1 thread) 9.1 1141

speed_ramp1NSGA-II (6 threads) 3.9 488NSGA-II (1 thread) 11.8 1485Exhaustive (1 thread) 10.4 1296

speed_step1NSGA-II (6 threads) 4.6 584NSGA-II (1 thread) 12.3 1540Exhaustive (1 thread) 10.8 1350

One key area of future work is to develop case studies within the INTO-CPS exam-ples repository that truly demonstrate multi-objective characteristics. This will allow abetter comparison of existing DSE and MOO performance, particularly given the newmulti-threading support in the existing DSE scripts. In this way, we can begin to identifyguidelines for users about when and where each approach is likely to be most benefi-cial.

Acknowledgements. We are grateful to AgroIntelli for supplying Robotti data. Thiswork was partially funded under the European Union’s H2020 programme through theINTO-CPS and HUBCAP projects (Grant Agreements 644047 and 872698).

References

1. Bogomolov, S., Fitzgerald, J., Foldager, F.F., Gamble, C., Larsen, P.G., Pierce, K., Stankaitis,P., Wooding, B.: Tuning robotti: the machine-assisted exploration of parameter spaces inmulti-models of a cyber-physical system. John Fitzgerald, Tomohiro Oda, and Hugo DanielMacedo (Editors) p. 50 (2021)

2. Deb, K., Pratap, A., Agarwal, S., Meyarivan, T.: A fast and elitist multiobjective geneticalgorithm: Nsga-ii. IEEE transactions on evolutionary computation 6(2), 182–197 (2002)

3. Durillo, J.J., Nebro, A.J., Luna, F., Dorronsoro, B., Alba, E.: jmetal: A java framework fordeveloping multi-objective optimization metaheuristics. Departamento de Lenguajes y Cien-cias de la Computación, University of Málaga, ETSI Informática, Campus de Teatinos, Tech.Rep. ITI-2006-10 (2006)

4. Gunantara, N.: A review of multi-objective optimization: Methods and its applications. Co-gent Engineering 5(1), 1502242 (2018)

5. Jensen, M.T.: Reducing the run-time complexity of multiobjective eas: The nsga-ii and otheralgorithms. IEEE Transactions on Evolutionary Computation 7(5), 503–515 (2003)

6. Larsen, P.G., Fitzgerald, J., Woodcock, J., Gamble, C., Payne, R., Pierce, K.: Features of in-tegrated model-based co-modelling and co-simulation technology. In: Bernardeschi, Masci,Larsen (eds.) 1st Workshop on Formal Co-Simulation of Cyber-Physical Systems. LNCS,Springer-Verlag, Trento, Italy (September 2017)

7. Lee, E.A.: CPS foundations. In: Proceedings of the 47th Design Automation Conference. pp.737–742. DAC ’10, ACM, New York, NY, USA (2010)


8. Suman, B., Kumar, P.: A survey of simulated annealing as a tool for single and multiobjectiveoptimization. Journal of the operational research society 57(10), 1143–1160 (2006)

9. Thule, C., Lausdahl, K., Gomes, C., Meisl, G., Larsen, P.G.: Maestro: TheINTO-CPS co-simulation framework. Simulation Modelling Practice and Theory 92,45 – 61 (2019), http://www.sciencedirect.com/science/article/pii/S1569190X1830193X

10. Thule, C., Lausdahl, K., Larsen, P.G., Meisl, G.: Maestro: The INTO-CPS Co-SimulationOrchestration Engine (2018), submitted to Simulation Modelling Practice and Theory

11. Zitzler, E.: Spea2: Improving the performance of the strength pareto evolutionary algorithm.Computer Engineering and Communication Networks Lab (TIK) (2001)



arxiv:2110.09371v1 [cs.se] 18 oct 2021

Documents