validation of a logic controller : using conformance test...

Anaïs Guignard PhD advisor : Jean-Marc Faure

LURPA, ENS Cachan

Validation of a logic controller : Using conformance test or observation of the closed

loop system behavior

VACSIM 17/04/13

Plan

•  I / Introduction •  I.1 / Validation of a PLC •  I.2 / Objectives

•  II / Contribution to conformance test •  II.1 / Context •  II.2 / Contribution •  II.3 / With or without stability implementation

•  III / Using observation of closed-loop system for validation •  III.1 / Context •  III.2 / Enforcement •  III.3 / Application to validation

•  IV / Conclusions et perspectives

VACSIM 17/04/13 2

Validation of a PLC

Validation of an implementation [Boehm,79]: •  Are we doing the right product?

Two considered methods:

•  Conformance test on the controller alone.

•  Observation of the closed loop system behavior, then comparison with the specified behavior.

VACSIM 17/04/13

Introduction

3

Objectives

•  Conformance test on the controller alone. •  Based on Provost works [Provost, 10]. •  Grants a validation on the whole behavior accepted by the controller, without

considering the restriction due to the plant. •  Makes an hypothesis on the implementation algorithm. •  → We extend the method to the other implementation algorithm and illustrate

the influence of the choice of algorithm.

•  Observation of the closed loop system behavior, then comparison with the specified behavior.

•  Considers the closed loop system controller-plant: only the behavior possible by both elements is validated.

•  Adds phenomena due to physical PLC •  → We propose a method to differentiate deviating behaviors due to PLC

features and to nonconformity based on enforcement approaches.

VACSIM 17/04/13

Introduction

4

Plan

•  I / Introduction •  I.1 / ANR VACSIM project •  I.2 / Validation of a PLC •  I.3 / Objectives




VACSIM 17/04/13 5

Context – Conformance test

Conformance test:

Based on: •  Knowledge of the formalized specification •  Generation of a test sequence depending on a given coverage rate •  Verdict for the whole test sequence •  No knowledge about the plant

VACSIM 17/04/13

Contribution to conformance test

6

Context – Conformance test of Grafcet implementation J. Provost : Test de conformité de contrôleurs logiques spécifiés en Grafcet (2011)

Hypothesis :

•  Implementation with search of stability: Transient evolutions are performed in a single PLC cycle.

VACSIM 17/04/13


7

Contribution

VACSIM 17/04/13

Location automaton

With stability search

Without stability search

{█■𝑺↓𝑨𝒄𝒕 ={𝑺∈ 𝑺↓𝑮  | 𝑿↓𝑺 =𝑻𝒓𝒖𝒆 } 𝑬(𝑰↓𝑮 )=∏█■𝒕∈ 𝑻↓𝒆𝒏𝒂𝒃  𝒕∈𝑺𝑭𝑻 ↑▒𝒕.𝑭(𝑰↓𝑮 ) . ∏█■𝒕∈ 𝑻↓𝒆𝒏𝒂𝒃  𝒕∉𝑺𝑭𝑻 ↑▒𝒕.𝑭(𝑰↓𝑮 )   𝐶(𝑰↓𝑮 )= ∑𝒆𝒗𝒐𝒍↓𝒌 . 𝒍↓𝒖 =𝒍↑▒𝒆𝒗𝒐𝒍↓𝒌 .𝑬(𝑰↓𝑮 )     


8

With or without stability implementation – Algorithms

VACSIM 17/04/13

Execution with stability search : •  Stable = False •  While Stable = False do

•  Determine the set of firable transitions •  Determine the set of active steps •  Compute the values of outputs

controlled by stored action associated to active steps

•  Determine the set of firable transitions •  If the set of firable transitions is empty

•  Stable = True •  Compute the values of outputs controlled by

continuous actions associated to active steps

Execution without stability search : •  Determine the set of firable transitions •  Determine the set of active steps •  Compute the values of outputs controlled by

stored action associated to active steps •  Compute the values of outputs controlled by

continuous actions associated to active steps

2

3

𝑏

𝑎 

𝑂↓1  𝑂↓2 

1

𝑆↓1 =1

𝑎

𝑎=𝑏=1

𝜎↓𝑂𝑏𝑠 =(█■𝑎 𝑏�� 𝑜↓1  𝑜↓2  𝑆↓1  )=(█■11000 )(█■11011 )

𝑋↓1  𝑋↓3 

𝜎↓𝑂𝑏𝑠 =(█■𝑎 𝑏�� 𝑜↓1  𝑜↓2  𝑆↓1  )=(█■11000 )(█■11101 )(█■11011 )

𝑋↓1  𝑋↓2  𝑋↓3 


9

With or without stability implementation – Location automaton

VACSIM 17/04/13

With stability search Without stability search


10

With or without stability implementation – Conformance test Assuming that:

•  Test sequence built with stability search

•  I m p l e m e n t a t i o n conform but without stability search

Conclusion :

Nonconformity

VACSIM 17/04/13

Test step

Step 1 Step 2 Step 3

Input sequence(█■𝑠ℎ↓1  ℎ↓2  𝑙↓1  𝑙↓2  )

(█■00011 )

(█■11011 )

(█■00111 )

Expected output sequence (█■𝑉↓1  𝑉↓2  𝑊↓1  𝑊↓2  )

(█■0000 )

(█■0110 )

(█■0011 )

Observed output sequence (█■𝑉↓1  𝑉↓2  𝑊↓1  𝑊↓2  )

(█■0000 )

(█■1100 ) or

(█■1001 )

(█■1100 )

(█■0110 )

(█■0011 )


11

Plan





VACSIM 17/04/13 12

Context – Acquisition method

VACSIM 17/04/13

Using observation of closed-loop system for validation

13

Sequence 𝜎↓𝑜𝑏𝑠 =((█■𝐼↓𝑀  𝑂↓𝑀  )↑𝑜𝑏𝑠 , …)

Context – Phenomena due to PLC features

VACSIM 17/04/13

•  Outputs emission delay

•  Synchronization of asynchronous events

•  Desynchronization of synchronous events


14

Context – Illustrative example

Specification under the form of a Mealy machine

•  2 Inputs : 𝐈↓𝐌 ={𝒊↓𝟏 , 𝒊↓𝟐 }

•  2 outputs : 𝐎↓𝐌 ={𝒐↓𝟏 , 𝒐↓𝟐 }

VACSIM 17/04/13

1

2

𝑖↓1  . 𝑖↓2   / 𝑜↓1  . 𝑜↓2  

𝑖↓1 . 𝑖↓2  / 𝑜↓1 . 𝑜↓2 

𝑖↓1 . 𝑖↓2  / 𝑜↓1 . 𝑜↓2 

3

𝑖↓1 . 𝑖↓2   / 𝑜↓1  . 𝑜↓2  

𝑖↓1  . 𝑖↓2  / 𝑜↓1 . 𝑜↓2  

𝑖↓1 . 𝑖↓2   / 𝑜↓1 . 𝑜↓2 

𝑖↓1  . 𝑖↓2  / 𝑜↓1 . 𝑜↓2  

𝑖↓1  . 𝑖↓2   / 𝑜↓1 . 𝑜↓2 

𝑖↓1  . 𝑖↓2  / 𝑜↓1 . 𝑜↓2  

𝑖↓1  . 𝑖↓2  / 𝑜↓1 . 𝑜↓2  

𝑖↓1 . 𝑖↓2   / 𝑜↓1 . 𝑜↓2  

𝑖↓1 . 𝑖↓2  / 𝑜↓1 . 𝑜↓2  


15

Context – Focus on desynchronization

Limitation to desynchronization:

•  Synchronous events seen as asynchronous

•  Three consequences : •  No change on emitted outputs:

•  Delay of some cycles for all expected outputs to change: -  𝑖↓1  read before 𝑖↓2 

•  Non-expected output changes (non-conform behavior): -  𝑖↓2  read before 𝑖↓1 

VACSIM 17/04/13

𝜎↓𝑒𝑥𝑝 =(█■0000 )(█■1111 )(█■0110 )with(█■𝑖↓1  𝑖↓2  𝑜↓1  𝑜↓2  )

𝜎↓𝑜𝑏𝑠  =(█■0000 )(█■1111 )(█■0110 )

𝜎↓𝑜𝑏𝑠 =(█■0000 )(█■1100 )(█■1111 )(█■0110 )

𝜎↓𝑜𝑏𝑠 =(█■0000 )(█■1110 )(█■0110 )


16

Enforcement - Principle

VACSIM 17/04/13

Sequence of events 𝝈

Property to verify 𝜑

Enforcement monitor

Memory

Events sequence 𝜎 𝜎|= 𝜑 ?

Events sequence 𝜊 𝜊|= 𝜑


17

[Ligatti, 05]

An Edit automaton is a 6-tuple (𝚺, 𝐐, 𝐪↓𝟎 , 𝜹, 𝝎, 𝜸) where :

•  𝚺 is a finite nonempty set of observable actions, •  𝑄 is a finite state of states •  𝑞↓0  is the initial state •  𝛿 :𝑄×Σ→𝑄 is a labeled partial transition function •  𝛾↓𝑜 :Σ×𝑄→Σ↑∗  is a function which indicates the emitted actions •  𝛾↓𝑘  :𝑄×Σ→ Σ↑∗  is a function which defines the actions kept in memory

VACSIM 17/04/13

Enforcement – Enforcement automaton


18

Enforcement – Enforcement automaton

VACSIM 17/04/13

A transition (𝝈,𝒒)→𝝉┴(𝝈↑′ , 𝒒↑′ ) may have the form with 𝝈=𝒂;𝝈′: •  Validation : (𝝈,𝒒)→𝒂┴(𝝈↑′ , 𝒒↑′ )

•  Suppression : (𝝈,𝒒)→.┴(𝝈↑′ , 𝒒↑′ )

•  Insertion : (𝝈,𝒒)→𝝉┴(𝝈, 𝒒↑′ )

•  Stop : (𝝈,𝒒)→ .┴(∙, 𝒒↑′ )

1 2𝜎=𝑎;𝜎′ 𝜎=𝜎′ 𝑎 𝑎

1 2𝜎=𝑎;𝜎′ 𝜎=𝜎′ 𝑎

1 2𝜎=𝑎;𝜎′ 𝜎=𝑎;𝜎′ 𝑎 𝜏

1 2𝜎=𝑎;𝜎′ 𝜎=∅ 𝑎

Upper part : action treated Lower part : action emitted


19

Application to validation – Method

VACSIM 17/04/13

Mealy machine

Observation

Controller

Simulated plant I’ O’

I O

Enforcement

Valid I/O

Verdict

Consequences of PLC I/O scanning cycle

Edit automaton

Validation

Automaton construction


20

Application to validation – Construction principle

Two steps construction :

•  Transformation of the Mealy machine which describes the specification to an enforcement automaton,

•  Addition of behaviors due to PLC features to the enforcement automaton.

VACSIM 17/04/13


21

Application to validation – Algorithm

VACSIM 17/04/13

1

2

(█■0000 )

(█■1111 )

(█■1111 )

3

(█■1010 )

(█■0110 )

(█■1010 )

(█■0110 )

(█■0011 )

First step: specified behavior

•  For each state of the MM do: •  Create an equivalent state

•  For each evolution and self-loop of the MM do:

•  Create an equivalent arc •  Create the action as a concatenation

of input and output vectors •  Emit the action •  Free the action from the sequence

memory


22

Application to validation – Algorithm

VACSIM 17/04/13

3 (█■1101 )

1

2

(█■0000 )

(█■1111 )

(█■1111 )

(█■1010 )

(█■0110 )

(█■1010 )

(█■0110 )

(█■0011 )

4 5

3

(█■1111 )

(█■1111 ) (█■1111 )

(█■1101 ) (█■1100 )

(█■1101 ) (█■1110 )

(█■1110 ) (█■1100 )

(█■1110 )

Second step: desynchronization phenomenon

•  For Combination of input variables as at least two values are different between evolution and self-loop, and output values change do:

•  N = Number of output changes •  Create an intermediate state corresponding to no

output change •  Create the transition between initial state to the

created one, with no emission of actions

•  While 𝑁≤0 do:

-  If 𝑁>1 then: •  Create intermediate states corresponding to

N-1 outputs to change •  Create all transitions from previous states to

the created ones, with no emission of actions

•  𝑁=𝑁−1

-  If 𝑁=1 •  Create transitions from all intermediate states

to the final state, with emitted actions •  Create missing self-loops


23

Application to validation – Remark

VACSIM 17/04/13

1

2

(█■0000 )

(█■1111 )

(█■1111 )

(█■1010 )

(█■0110 )

(█■1010 )

(█■0110 )

(█■0011 )

4 5

3

(█■1111 )

(█■1111 ) (█■1111 )

(█■1101 ) (█■1100 )

(█■1101 ) (█■1110 )

(█■1110 ) (█■1100 )

(█■1110 )

This automaton is not complete.

All the evolutions not defined here correspond to forbidden evolutions.

In our case, they correspond to non-conform behavior.


24

(█■1101 )

Application to validation – Example

VACSIM 17/04/13 25


1

2

(█■0000 )

(█■1111 )

(█■1111 )

(█■1010 )

(█■0110 )

(█■1010 )

(█■0110 )

(█■0011 )

4 5

3

(█■1111 )

(█■1111 ) (█■1111 )

(█■1101 ) (█■1100 )

(█■1101 ) (█■1110 )

(█■1110 ) (█■1100 )

(█■1110 )

𝜎↓𝑜𝑏𝑠  =(█■0000 )(█■1111 )(█■0110 )→𝜎↓𝑒𝑛𝑓  =(█■0000 )(█■1111 )(█■0110 )

𝜎↓𝑜𝑏𝑠 =(█■0000 )(█■1100 )(█■1110 )(█■0110 )→𝜎↓𝑒𝑛𝑓  =(█■0000 )(█■1111 )(█■0110 )

𝜎↓𝑜𝑏𝑠 =(█■0000 )(█■1110 )(█■0110 )→𝜎↓𝑒𝑛𝑓  =(█■0000 )

Plan





VACSIM 17/04/13 26

Conclusions and perspectives

Conclusions: •  Implementation choices and PLC features should not be neglected •  Coupling controller and plant reduces the possibility of evolutions;

Not all the evolutions should be tested •  Enforcement methods are and interesting way to make the

difference between behavior due to PLC I/O scanning cycle and nonconformity

Perspectives: •  Extend the use of enforcement methods to all behaviors due to

physical PLC •  Rather than building enforcement automata, use general laws that

describe the allowed behaviors.

VACSIM 17/04/13 27

Publications

Accepted:

•  Construction de modèles formels de contrôleurs logiques pour le test de conformité

Journée des doctorants 2013

Submitted:

•  Formal models for confomance test of programmable logic controllers

CASE 2013

•  Enforcing I/O sequences for PLC validation purposes

ETFA 2013

VACSIM 17/04/13 28

Merci pour votre attention

Avez-vous des questions ?

VACSIM 17/04/13 29

References

[Boehm,79]: B. Boehm, Software engineering: R&D trends and defense needs, Research directions in software technology, 1979 [Provost, 11]: J. Provost, J.M. Roussel, J.M. Faure, Translating Grafcet specifications into Mealy machines for conformance test purposes, Control Ingineering Practices, 2011, vol.19, pp.947-957 [Ligatti, 05]: J. Ligatti, L. Bauer, D. Walker, Edit automata: Enforcement mechanisms for run-time security policies, International Journal of Information Security, 2005, vol.4, pp.2-16

VACSIM 17/04/13 30

validation of a logic controller : using conformance test...

Documents