validation and performance analysis of embedded...

Validation and

Performance Analysis

of CPS in UPPAAL

Kim G. Larsen

Aalborg University, DENMARK

Cyber Physical Systems

TuToR 2017 Kim Larsen [2]

DEPENDABLE:

the ability of a controller to

function (correct) under

stated conditions for a specified period of time.

OPTIMAL:

finding a controller for a

given system such that a

certain optimality criterion is achieved.

Stochasticity

Real Time

Hybrid

Discrete

Resources

UPPAAL Tool Suite

TRON

CLASSIC

TIGA

CORA

ECDAR

SMC

Optimization

Synthesis

Component

Testing

PerformanceAnalysis

Verification

STRATEGOOptimal Synthesis

1995

2001

2005

2011

2014

2010

2004


Timed Automata

Timed Automata

ADD a clock x

Synchronizing

action

Clock Guard

Conjunctions of

x~n

x: real-valued

clock

Reset

[Alur & Dill’89]


Semantics


Semantics in UPPAAL

Train Crossing

Time

River

Bridge

tracks

Safe Approaching Crossing Safe

03 – 5

20

TuToR 2017 [7]

Train Crossing

Time

River

Bridge

tracks



Stop the train while it still stoppable!

1003 – 5

20

TuToR 2017 [8]

Train Crossing

Time

River

Bridge

tracks



1003 – 5

20

Stopped

Crossing Safe

RestartedStopped

Crossing Safe

7 – 15

Crossing

Restarted

TuToR 2017 [9]

Train Crossing


Stopped Restarted

Add timing+ synchronization

TuToR 2017 [10]

Editor


GUI

• Unlimited undo and redo

• Syntax and bracket highlighting

• Rectangular selection

• Customization of colors

• Tooltip

• Hiding of information

• Improved help menu with search component

Language

• User defined functions (C-like)

• New types (records, type declarations, meta variables, scalars)

• Partial instantiation of templates

• Select clauses on edges

• Forall and exist quantifiers

Concrete Simulator


Graphical Simulator

• visualization

and recording

• inexpensive fault detection

• inspection of error traces

• Message Sequence Charts

• Gannt Charts

Symbolic Simulator


Graphical Simulator

• visualization

and recording

• inexpensive fault detection

• inspection of error traces

• Message Sequence Charts

• Gannt Charts

Verifier


Verifier

• Exhaustive & automatic

checking of requirements

• .. including validating, safety, liveness,

bounded liveness and

response properties

•.. performance properties,

e.g probabilistic and expectation.

• .. generation of debugging information

for visualisation in simulator.

• .. plot composer

Demo 1


UPPAAL ExamplesSSFT15/UPPAAL SMC/

Verification Queries

Validation Properties

Possibly: E<> P

Safety Properties

Invariant: A[] P

Pos. Inv.: E[] P

Liveness Properties

Eventually: A<> P

Leadsto: P --> Q

Bounded Liveness

Leads to within: P -->· t Q

The expressions P and Q must be type safe, side effect free, and evaluate to a boolean.

Only references to integer variables, constants, clocks, are allowed (and arrays of these).

16TuToR 2017

Demo 2


UPPAAL ExamplesSSFT15/UPPAAL SMC/

THE ”secret” of UPPAAL


Zones - Operations

x

y

x

y

x

y

x

y

x

y

x

y

(n, 2·x·4

1·y·3 y-x·0 )(n, 2·x

1·y -3· y-x·0 )

(n, 2·x

1·y·3 y-x·0 )

Delay Delay (stopwatch)

Reset

(n, x=0 1·y·3 )

Extrapolation

2

Convex Hull

(n, 2·x·4 1·y )


Verification Options

State Space Reduction None

Conservative

Aggressive

Extreme

State Space Representation DBM

Compact Form

Under Approximation

Over Approximation

Diagnostic Trace

Extrapolation Automatic

None

Difference

Local

Lower/Upper

Search Order

Depth First

Breadth Frist

Random Depth First


Stochastic Timed Automata

WithPeter Bulychev, Alexandre David,

Marius Mikucionis

Dehui Du, Axel Legay, Guangyuan Li,

Danny B. Poulsen, Amalie Stainer, Zheng Wang

FORMATS11+12, CAV11, RV12, HSB12,QAPL12,

NaSA12+13,SCIENCE CH13, STTT15

UPPAAL

A[] forall (i : id_t) forall (j : id_t)

Train(i).Cross && Train(j).Cross imply i == j

Safety

E<> Train(0).Cross and Train(1).StopReachability

Train(0).Appr --> Train(0).CrossLiveness

A<> .. E[] ..

sup: .. inf: ..Limited quantitative analysis

Performance properties

State-space explosion


UPPAAL SMC



Pr[ <= 200](<> Train(5).Cross)



Pr[ <= 100](<> Train(0).Cross) >= 0.8

Pr[ <= 100](<> Train(5).Cross) >=

Pr[ <= 100](<> Train(1).Cross)

Generate runs


Stochastic Semantics of TA


Uniform DistributionExponential Distribution

Input enabledComposition =Repeated races between components for outputting

1

2 3 4 5

0.5

1

Composed Distributions


Includes all Phase-Type

Distributions.

Can encode any distribution with

arbitrary

precision.

𝜎-algebra with prob. measure from cylinders 𝑪(𝑰𝟎 ℓ𝟎 𝑰𝟏 ℓ𝟏 𝑰𝟐… 𝑰𝒏 ℓ𝒏+𝟏)

Pr ⟨⟩≤9 END = ½

Pr ⟨⟩≤7 END ≥ ½

Composition of STA

Composition = Race between components

for outputting

TuToR 2017

Pr[time<=2](<> T.T3) ?

Pr[C<=6](<> T.T3) ?

= න𝑡𝑎=0

1

1 ⋅ න𝑡𝑏=𝑡𝑎

2

½ 𝑑𝑡𝑏 𝑑𝑡𝑎 = 3/4

Kim Larsen [26]

Statistical Model Checking

M

𝝓

Generate random run π

Validate𝝅 ⊨ 𝝓 ?

Core StatisticalAlgorithm

Inconclu

siv

e

PrM(𝝓) 2 [a-𝝐,a+𝝐] with confidence µ𝝁

p, 𝜶

PrM(𝝓)≥ ¸pat significance level 𝜶

}<T p

[FORMATS11,LPAR12, RV12]


ConfidenceInterval

Hypothesistesting

𝝁, 𝝐

Performance Queries

EvaluationPr[<=100](<> expr) Pr(𝚽):𝚽 ∈ 𝑴𝑰𝑻𝑳

Hypothesis testingPr[<=100](<> expr) >= 0.1

c<=100 #<=50 [] expr <=0.5

ComparisonPr[<=20](<> e1) >= Pr[<=10](<> e2)

Expected valueE[<=10;1000](min: expr)

Explicit number of runs. Min or max.

Simulationssimulate 10 [<=100]{expr1,expr2}


DEMO

http://www.uppaal.com/index.php

Stochastic Hybrid Automata


Stochastic Hybrid Systems

TuToR 2017

on/off

on/off

Room 1

Room 2Heater

simulate 1 [<=100]{Temp(0).T, Temp(1).T}

simulate 10 [<=100]{Temp(0).T, Temp(1).T}

Pr[<=100](<> Temp(0).T >= 10)

Pr[<=100](<> Temp(1).T<=5 and time>30) >= 0.2

Kim Larsen [31]


TuToR 2017

A Bouncing Ball

Kim Larsen [32]

simulate 1 [<=4]{Ball.p, Ball.v}


TuToR 2017

A Bouncing BallPlayer 1

Player 2

simulate 1 [<=20]{Ball1.p, Ball2.p}

Pr[<=20](<>(time>=12 && Ball1.p>4))

SSFT15/UPPAAL SMC/Hybrid

Kim Larsen [33]

Lightweight Media Access Control

Problem domain: communication

scheduling

Targeted for: self-configuring

networks,

collision avoidance,

low power consumption

Application domain: wireless sensor

networks

Initialization (listen until a neighbor is heard)

Waiting (delay a random amount of time frames)

Discovery (wait for entire frame and note used slots)

Active choose free slot, use it to transmit, including

info about detected collisions listen on other slots fallback to Discovery if

collision is detected

Only neighbors can detect collision and tell the user-node that its slot is used by others


Kim Larsen [36]TuToR 2017

adopted from A.Fehnker, L.v.Hoesel, A.Mader

added

power

discovery

random wait

active usage

initialization

..used UPPAAL to explore 4- and 5-node

topologies and found cases with

perpetual collisions

(8.000 MC problems)

Statistical MC offers an insight by

calculating the probability over the

number of collisions.

+ estimated cost in terms of energy.

http://academic.research.microsoft.com/Author/2311816/ansgar-fehnker

http://academic.research.microsoft.com/Author/1428994/lodewijk-van-hoesel

http://academic.research.microsoft.com/Author/636374/angelika-mader

Kim Larsen [37]

SMC of LMAC with 4 Nodes

Wait distribution:

geometric

uniform

Network topology:

chain

ring

Collision probability

Collision count

Power consumption Pr[<=160] (<> col_count>0)

Pr[collisions<=50000] (<> time>=1000)

no collisions

<12 collisions

zero

Pr[energy <= 50000] (<> time>=1000)

TuToR 2017

10-Node Star


The first collision:

happens before 500tu

Collision counts after 1000tu

Collision counts after 2000tu:

the numbers are doubled –

perpetual collisions

• The first collisions happen before 500tu.

• It is unlikely (8.2%) that

there will be 0 collisions.

• And if they happen, they are perpetual.

0 0 0

000

Schedulability& Performance Analysis

??

Task Scheduling


T2 is running{ T4 , T1 , T3 } readyordered according to somegiven priority:(e.g. Fixed Priority, Earliest Deadline,..)

T1

T2

Tn

Scheduler

2 14 3

readydone

stoprun

P(i), UNI[E(i), L(i)], .. : period or earliest/latest arrival or .. for Ti

C(i), UNI[BC(i),WC(i)] : execution time for Ti

D(i): deadline for Ti

utilization of CPU

Modeling Task


T1

T2

Tn

Scheduler

2 14 3

readydone

stoprun

Modeling Scheduler


T1

T2

Tn

Scheduler

2 14 3

readydone

stoprun

Modeling Queue


T1

T2

Tn

Scheduler

2 14 3

readydone

stoprun

……

Schedulability Analysis

TuToR 2017

Kim Larsen [44]

const int E[N] = { 200, 200, 100, 100 };

const int L[N] = { 400, 200, 100, 100 }; // Ready interval

const int D[N] = { 400, 200, 100, 100 }; // Deadlines

const int WC[N] = { 60, 40, 20, 10 }; // Worst Computation Times

const int BC[N] = { 20, 20, 10, 5 }; // Best Computation Times

const int P[N] = { 1, 2, 3, 4 }; // Priorities

simulate 1 [<=400] { Task0.Ready + 2*Task0.Running +3*Task0.Blocked,

Task1.Ready + 2*Task1.Running +3*Task1.Blocked + 4, Task2.Ready + 2*Task2.Running + 3*Task2.Blocked + 8, Task3.Ready + 2*Task3.Running + 3*Task3.Blocked +12 }

A[] not (Task0.Error or Task1.Error or Task2.Error or Task3.Error)

Schedulability Analysis

TuToR 2017

Kim Larsen [45]

const int E[N] = { 200, 200, 100, 100 };

const int L[N] = { 400, 200, 100, 100 }; // Ready interval

const int D[N] = { 400, 200, 100, 100 }; // Deadlines

const int WC[N] = { 60, 40, 20, 60 }; // Worst Computation Times

const int BC[N] = { 20, 20, 10, 5 }; // Best Computation Times

const int P[N] = { 1, 2, 3, 4 }; // Priorities

A[] (not Taski.Error) i : 0,1,2,3

Pr[<=4000] ( <> Task0.Error or Task1.Error

or Task2.Error or Task3.Error)

simulate 10000 [<=400] { Task0.Ready + 2*Task0.Running +3*Task0.Blocked,

Task1.Ready + 2*Task1.Running +3*Task1.Blocked + 4, Task2.Ready + 2*Task2.Running + 3*Task2.Blocked + 8, Task3.Ready + 2*Task3.Running + 3*Task3.Blocked +12 }: 1 : (Task0.Error or Task1.Error or Task2.Error or Task3.Error)


TuToR 2017

Kim Larsen [46]

sup : Task2.r, Task3.r


TuToR 2017

Kim Larsen [47]

E[<=800; 5000] (max: Task0.r)E[<=800; 5000] (max: Task0.r)E[<=800; 5000] (max: Task0.r)E[<=800; 5000] (max: Task0.r)

D=400

D=200

D=100

D=100

Herschel-Planck Scientific Mission at ESA

TuToR 2017

Kim Larsen [48]

Attitude and Orbit Control SoftwareTERMA A/S Steen Ulrik Palm, Jan Storbank Pedersen, Poul Hougaard

Herschel & Planck Satelites

Application software (ASW) built and tested by Terma:

does attitude and orbit control, tele-commanding, fault detection isolation and recovery.

Basic software (BSW) low level communication and scheduling

periodic events.

Real-time operating system (RTEMS) Priority Ceiling for ASW,

Priority Inheritance for BSW

Hardware single processor, a few communication

buses, sensors and actuators.

Kim Larsen [49]TuToR 2017

Requirements:Software tasks should be schedulable.CPU utilization should not exceed 50% load

Modeling in UPPAAL


UPPAAL 4.1 FrameworkISoLA 2010

Blocking & WCRT

TuToR 2017

Marius Micusionis

thanks and a few qeustions.eml

TERMA Case Follow-Up


[ f*WCET, WCET]

1 Day

6 Days

f=100% f=95%

f=90% f=86%

ISOLA 2012

TERMA Case - Statistical MC


TERMA Case – Conclusion


Reading

Frits Vaandrager: A first introduction to UPPAAL Alexandre David, Kim G Larsen: More features in UPPAAL Alexandre David, Kim G. Larsen, Axel Legay, Marius

Mikucionis, Zheng Wang: Time for Statistical Model Checking of Real-Time Systems. CAV 2011: 349-355.

Alexandre David, Kim G. Larsen, Axel Legay, Marius Mikucionis, Danny Bøgsted Poulsen: Uppaal SMC tutorial. STTT 17(4): 397-415 (2015)

Kim Guldstrand Larsen: Validation, Synthesis and Optimization for Cyber-Physical Systems. TACAS (1) 2017: 3-20

Alexandre David, Peter Gjøl Jensen, Kim Guldstrand Larsen, Marius Mikucionis, Jakob Haahr Taankvist: UppaalStratego. TACAS 2015: 206-211


Applications(some)


Bang & Olufsen IR-Link

Bug known to exist for 10 years

Ill-described: 2.800 lines of

assembler code + 3 flowchart + 1 B&O eng.

3 months for modeling.

UPPAAL detects error with 1.998 transition steps (shortest)

Error trace was confirmed in B&O laboratory.

Error corrected and verified in UPPAAL.

Arne Skou, Klaus Havelund

1st RTSS’97 talk, Klaus HavelundTuToR 2017 Kim G. Larsen 57

Bang & Olufsen IR-Link

Bug known to exist for 10 years

Ill-described: 2.800 lines of

assembler code + 3 flowchart + 1 B&O eng.

3 months for modeling.

UPPAAL detects error with 1.998 transition steps (shortest)

Error trace was confirmed in B&O laboratory.

Error corrected and verified in UPPAAL.

Arne Skou, Klaus Havelund

1st RTSS’97 talk, Klaus Havelund

Reliable systems & Uppaal Arne Skou 37March 25, 1999

Message

Collision

Radio Silence

Jam

1562 ms 1562 ms2*i*1562 ms

M::=T5{T1,T2,T3}>=15T4

M1

M2

M

50.000 ms

50.000 ms

Sampling:each 781 ms

TuToR 2017 Kim G. Larsen 58

Philips Bounded Retransmission Protocol

Pedro D’Argenio

Joost-Pieter Katoen

Theo Ruys

Jan Tretmans


FlexRay


Fault-tolerance

Timed hardware model

Parameterized error models

(glitches, jitter)

Voting & bit-clock alignment

BMW, Bosch, Daimler, Freescale,

General Motors, NXP

Semiconductors, and

Volkswagen

transmission

of message

byte

[Gerke, Ehlers, Finkbeiner, Peters, 2010]

Gear Controllerwith MECEL AB

Flowgraph

Magnus Lindahl

Paul Pettersson

Wang Yi

2001

TuToR 201761


Timed Automata

Models

Magnus Lindahl

Paul Pettersson

Wang Yi

2001

TuToR 201762


Requirements

Magnus Lindahl

Paul Pettersson

Wang Yi

2001

TuToR 201763

UPPAAL Model Checking – Demo

TuToR 201764

UPPAAL Model Checking – Demo

TuToR 201765

TERMA A/S (2004)Memory Management for Radars

Radar Video Processing SubsystemAdvanced Noise Reduction Techniques

e1,2

e0,5

e0,4

e0,3

e0,2e2,4

e2,3

e2,2

e1,5

e1,4

e1,3

e3,2

e3,4e3,3

e3,5

e2,5

Air

po

rt S

urv

eilla

nce

Costal Surveillance

echo

9.170 GHz

9.438 GHz

Combiner(VP3) F

req

uen

cy D

ivers

ity

combiner


TERMA A/S (2011)Herschel-Planck Scientific Mission at ESA

TuToR 2017

Kim Larsen [67]

Attitude and Orbit Control SoftwareTERMA A/S Steen Ulrik Palm, Jan Storbank Pedersen, Poul Hougaard

METAMOC


Modular Execution Time Analysis using

MOdel Checking

with

Andreas Dalsgaard

Mads Christian Olesen

Martin Toft

René Rydhof Hansen

Controllers in UPPAAL

Gearbox Controller [TACAS’98] Bang & Olufsen Power Controller [RTPS’99,FTRTFT’2k] SIDMAR Steel Production Plant [RTCSA’99, DSVV’2k] Real-Time RCX Control-Programs [ECRTS’2k] Terma, Verification of Memory Management for Radar (2001) Scheduling Lacquer Production (2005) Memory Arbiter Synthesis and Verification for a Radar Memory

Interface Card [NJC’05] Adapting the UPPAAL Model of a Distributed Lift System, 2007 Analyzing a χ model of a turntable system using Spin, CADP

and Uppaal, 2006 Designing, Modelling and Verifying a Container Terminal

System Using UPPAAL, 2008 Model-based system analysis using Chi and Uppaal: An

industrial case study, 2008 Climate Controller for Pig Stables, 2008 Optimal and Robust Controller for Hydralic Pump, 2009


(Wireless) Protocols in UPPAAL

Bang & Olufsen IR Link Philips Audio Protocol Collision-Avoidance Protocol Bounded Retransmission Protocol TDMA Protocol Multimedia Streams ATM ABR Protocol Lamport’s Leader Election Protocol ABB Fieldbus Protocol IEEE 1394 Firewire Root Contention Bluetooth Protocol Distributed Agreement Protocol FlexRay CHESS MAC Protocol Proprietary WSN, Other Big Danish Company MESH Protocol (MAC & Routing), NEOCORTEC


UPPAAL as a Back-End

Vooduu: verification of object-oriented designs using Uppaal, 2004

Moby/RT: A Tool for Specification and Verification of Real-Time Systems, 2000

Formalising the ARTS MPSOC Model in UPPAAL, 2007

Timed automata translator for Uppaal to PVS Component-Based Design and Analysis of

Embedded Systems with UPPAAL PORT, 2008 Verification of COMDES-II Systems Using UPPAAL

with Model Transformation, 2008 METAMOC: Modular WCET Analysis Using UPPAAL,

2010.


www.uppaal.org


More Applications

TuToR 2017

FIREWIRE BLUETOOTH

10 node LMAC

Battery

Scheduling

Energy Aware

Buildings

Genetic Oscilator

Mesh Network

Smart Grid

Kim Larsen [73]

validation and performance analysis of embedded...

Documents