safety-critical systems
TRANSCRIPT
Safety
-Critica
lSystem
s
Pro
f.D
r.Jan
Pelesk
a
Univ
ersitat
Brem
en—
TZI
Dr.
Ing.C
orn
eliaZahlten
Verifi
edSystem
sIn
ternatio
nalG
mbH
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Overv
iew
1.T
he
Notion
ofD
epen
dab
ility
2.Safety
-Related
Stan
dard
san
dV
-Models
3.M
odellin
gSafety
-Critical
System
s
4.H
azardA
naly
sisan
dR
iskA
ssessmen
t
5.D
esignC
riteriafor
Safety
-Critical
System
s
6.V
alidation
,V
erification
and
Test
ofSafety
-Critical
System
s
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Overv
iew
1.T
he
Notion
ofD
epen
dab
ility
2.Safety
-Related
Stan
dard
san
dV
-Models
3.M
odellin
gSafety
-Critical
System
s
4.H
azardA
naly
sisan
dR
iskA
ssessmen
t
5.D
esignC
riteriafor
Safety
-Critical
System
s
6.V
alidation
,V
erification
and
Test
ofSafety
-Critical
System
s
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Dependability
–G
enera
lD
efinitio
n
Our
obje
ctiv
e:
Develo
pm
ent
and
verifi
catio
nof
safe
ty-
critic
al
reactiv
esy
stem
sth
at
contin
uously
inte
ract
with
their
enviro
nm
ent
and
contro
l(p
arts
of)
the
enviro
nm
ent
inord
er
toperfo
rmth
euse
r-require
dse
rvic
es
and
enfo
rce
the
require
dsa
fety
pro
pertie
s.
•D
ependability
:T
he
trustw
orthin
essof
acom
puter
system
such
that
reliance
canbe
justifi
ably
placed
onth
eserv
iceit
delivers.
•Serv
ice:
System
beh
aviou
ras
itis
perceived
by
itsusers.
•U
ser:
Anoth
ersy
stem(h
um
anor
physical)
interactin
gw
ithth
etarget
system
.
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Dependability
Attrib
ute
s
•A
vaila
bility
:read
iness
forusage
•R
elia
bility
:con
tinuity
ofserv
ice
•Safe
ty:
avoidan
ceof
catastrophic
conseq
uen
ceson
the
environ
men
t
•Security
:preven
tionof
unau
thorised
accessan
d/or
han
dlin
gof
inform
ation
Secu
rityattrib
utes:
–con
fiden
tiality
–in
tegrity
–availab
ility
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Dependability
–Faults
–Erro
rs–
Failu
res
•Failu
re:
delivered
service
no
longer
complies
with
specifi
cation
•Erro
r:part
ofth
esy
stemstate
which
islikely
tolead
tosu
bseq
uen
tfailu
re
•Fault:
cause
ofan
error
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Dependability
–Fault-T
ole
rance
versu
sFault-A
void
ance
Classifi
cationof
meth
ods
achiev
ing
dep
endab
ility:
•Fault-T
ole
rance:
prov
ide
aserv
icecom
ply
ing
with
the
specifi
cationin
spite
offau
lts
•Fault-A
void
ance
(Fault-P
reventio
n):
a-priori
preven
tionof
fault
occu
rrence
orin
troduction
•Fault-R
em
oval:
reduce
the
presen
ce(n
um
ber,
seriousn
ess)of
faults
•Fault-F
ore
castin
g:
estimate
the
presen
tnum
ber,
the
futu
rein
ciden
ce,an
dth
econ
sequen
ceof
faults
Our
Favourite
Appro
ach
:Fault-T
ole
rance
on
HW
level—
Fault-A
void
ance
on
specifi
catio
nand
SW
level!
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Dependability
—Safe
tyversu
sLiv
eness
Pro
pertie
s
Incom
puter
science,
specifi
cations
areclassifi
edas
contain
ing
•Safe
tyP
ropertie
sS:A
ny
sequen
ceof
events
ortran
sitions
etc.violatin
gS
contain
sa
prefi
xall
ofw
hose
infinite
exten
sions
violate
S.
(Salw
ays
hold
s)
•Liv
eness
Pro
pertie
sL:A
ny
arbitrary
finite
sequen
ceof
events
canbe
exten
ded
toan
infinite
sequen
cesatisfy
ing
L.
(Lfinally
hold
s)
For
safe
ty-c
riticalre
al-tim
esy
stem
s,all
dependability
re-
quire
ments
should
be
specifi
ed
as
safe
typro
pertie
s:Liv
e-
ness
pro
pertie
scan
only
guara
nte
eth
at
ase
rvic
ew
illFIN
ALLY
be
deliv
ere
d,w
hich
isnot
suffi
cie
nt
inth
econ-
text
ofhard
real-tim
esy
stem
s.
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Dependability
—C
lassifi
catio
nofSyste
mB
ehavio
ur
System
statesp
aceis
partition
edin
toth
reeareas.
System
execu
tionis
regarded
asseq
uen
ceof
statetran
sitions,
possib
lyaccom
pan
iedby
visib
leaction
s(in
put/ou
tput).
o
ooo
oo
oo
o
o
o
trace s
ho
win
g n
orm
al b
eh
avio
ur
o
o
o
o
o
o
trace s
ho
win
g a
ccep
tab
le b
eh
avio
ur
No
rmal B
eh
avio
ur
Excep
tion
al B
eh
avio
ur
Cata
stro
ph
ic B
eh
avio
ur
trace s
ho
win
g n
orm
al, e
xcep
tion
al
an
d c
ata
stro
ph
ic b
eh
avio
ur:
- transitio
n to
exceptio
nal b
ehavio
ur
- transitio
n to
cata
stro
phic
behavio
ur
(AL
AR
P R
eg
ion
)
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Overv
iew
1.T
he
Notion
ofD
epen
dab
ility
2.Safety
-Related
Stan
dard
san
dV
-Models
3.M
odellin
gSafety
-Critical
System
s
4.H
azardA
naly
sisan
dR
iskA
ssessmen
t
5.D
esignC
riteriafor
Safety
-Critical
System
s
6.V
alidation
,V
erification
and
Test
ofSafety
-Critical
System
s
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Safe
ty-R
ela
ted
Term
inolo
gy
•A
ccid
ent:
An
undesired
and
unplan
ned
event
that
results
ina
specifi
edlevel
ofloss.
•Severity
ofan
Accid
ent:
Specifi
cationof
the
levelof
losscau
sedby
the
acciden
t,e.
g.,neglectab
le–
min
or–
critical–
catastrophic
•H
azard
:Som
ethin
gth
athas
the
poten
tialto
do
harm
orcan
leadto
anaccid
ent.
Hazard
s“in
herit”
their
severityattrib
utes
fromth
em
ostharm
fulaccid
ents
they
may
cause.
•(S
yste
m)
Safe
tyR
equire
ments:
Asp
ecification
stating
the
acceptab
lerelation
s
hazard
severity
↔pro
bability
ofhazard
occurre
nce
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Safe
ty-R
ela
ted
Sta
ndard
s–
Com
mon
Understa
ndin
g
Safety
requirem
ents
shou
ldbe
derived
froma
Risk
Analy
sis,con
sisting
ofH
azard
Analy
sisan
dR
iskA
ssessm
ent.
•H
azard
Analy
sis:list
ofpossib
lehazard
s,th
eirim
pact
onth
een
viron
men
t,th
eirpossib
lecau
ses(e.
g.,seq
uen
cesof
faults
leadin
gto
ahazard
).T
ypically,
itcon
sistsof
the
followin
gitem
s:
–H
azard
List:
collectionof
the
iden
tified
hazard
s
–H
azard
-Severity
Matrix
:relates
hazard
sto
severity
–H
azard
-Pro
bability
Matrix
:relates
hazard
sto
the
prob
ability
ofth
eiroccu
rrence
–H
azard
Model:
descrip
tionof
the
possib
lecau
ses–
that
is,seq
uen
cesof
events
–lead
ing
toa
hazard
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
•R
iskA
ssessm
ent:
quan
titativeor
qualitative
estimates
forth
eprob
ability
that
ahazard
will
occu
r
Accord
ing
toto
day’s
state
of
the
art,
ahazard
model
should
at
least
be
sem
i-form
al,
for
exam
ple
usin
gfa
ult-
trees,
event
tree
analy
sisofcause
-conse
quence
analy
sis.
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Safe
ty-R
ela
ted
Sta
ndard
s–
Com
mon
Understa
ndin
g
•relate
severityof
hazard
,prob
ability
ofoccu
rrence
and
system
beh
aviou
rin
Risk
Dia
gra
mor
Hazard
Risk
Index
•derive
required
effort
forvalid
ation,verifi
cationan
dtest
(VV
T)
ofsy
stemcom
pon
ents
fromth
erisk
diagram
and
the
impact
ofcom
pon
ent
failure
onhazard
occu
rrence
•V
VT
activities
forcom
pon
ents
with
high
estcriticality
shou
ldbe
perform
edby
Independent
Partie
s
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Safe
ty-R
ela
ted
Sta
ndard
s–
Risk
Dia
gra
m
no
rmal b
eh
avio
ur
excep
tion
al b
eh
avio
ur
(AL
AR
P re
gio
n)
cata
stro
ph
ic b
eh
avio
ur
(un
accep
tab
le re
gio
n)
(accep
tab
le a
rea)
pro
bab
ility o
f hazard
occu
rren
ce
severity
of h
azard
occasio
nally
low
pro
bability
impro
bable
impossib
le
pro
bable
frequently
critic
al
negle
cta
ble
min
or
cata
stro
phic
syste
m c
om
ponent
VV
T e
ffort
low
very
hig
h
hazard
caused
by c
om
ponent fa
ilure
/VV
T e
ffort re
quire
d
...
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Safe
ty-R
ela
ted
Sta
ndard
s–
Com
mon
Understa
ndin
g
If–
insp
iteof
allsafety
-relatedprecau
tions
–a
hazard
ous
situation
results
inan
acciden
t,th
endetailed
investigation
sare
perform
edw
ithth
eob
jectiveto
preven
tre-o
ccurren
ceof
similar
acciden
ts:
•R
oot
Cause
Analy
sisden
otesth
etask
toid
entify
the
crucial
causes
ofan
acciden
t.T
he
term“ro
otcau
se”in
dicates
that
the
crucial
causes
tolo
okfor
areth
oseat
the
begin
nin
gof
the
causal
chain
sfinally
resultin
gin
the
acciden
t.
•O
fsp
ecialin
terestis
the
subset
ofro
otcau
sesw
hose
occu
rrence
canbe
contro
lled
(that
is,preven
ted)
by
technical
ororgan
isational
measu
res.
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Safe
ty-R
ela
ted
Sta
ndard
s–
Com
mon
Understa
ndin
g
•R
oot
cause
analy
sispro
ceeds
accordin
gto
the
followin
gstep
s:
1.D
atacollection
2.C
ausal
factor(“cau
salch
ain”)
chartin
g
3.R
oot
cause
iden
tification
4.R
ecomm
endation
elaboration
5.R
ecomm
endation
implem
entation
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Safe
ty-R
ela
ted
Sta
ndard
s
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Safe
ty-R
ela
ted
Sta
ndard
s
Importan
tstan
dard
sfor
the
develop
men
tof
safety-critical
system
s:
•IE
C61508:
Sich
erheit
elektron
ischer
System
e
•IE
C61511:
Funktion
aleSich
erheit
–Sich
erheitstech
nisch
eSystem
efu
rdie
Prozessin
dustrie
•IE
C61513:
Kern
kraftw
erke–
Leittech
nik
fur
System
em
itsich
erheitstech
nisch
erB
edeu
tung
–A
llgemein
eSystem
anford
erungen
•E
N50129:
Bah
nan
wen
dungen
–Telekom
munikation
stechnik
,Sign
altechnik
und
Daten
verarbeitu
ngssy
steme
–Sich
erheitsrelevan
teelek
tronisch
eSystem
efu
rSign
altechnik
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Safe
ty-R
ela
ted
Sta
ndard
s
Importan
tstan
dard
sfor
the
develop
men
tof
safety-critical
system
s:
•E
N62061:
Sich
erheit
vonM
aschin
en–
Funktion
aleSich
erheit
sicherh
eitsbezogen
erelek
trischer,
elektron
ischer
und
program
mierb
arerelek
tronisch
erSteu
erungssy
steme
•IS
OC
D26262:
Road
vehicles
–Function
alsafety
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Safe
ty-R
ela
ted
Sta
ndard
sfo
rC
ivil
Airc
raft
Syste
ms
The
overalltest
pro
cessis
driven
by
the
followin
ghigh
-levelstan
dard
s
•A
irTra
nsp
ort
Asso
cia
tion
(ATA
)C
hapte
rs.A
system
aticdecom
position
ofa
concep
tual
aircraftin
toaircraft
system
s.System
descrip
tions
induce
the
fundam
ental
function
sreq
uired
inan
aircraft
Exam
ple
s.
–A
TA
-Chap
ter21.
Air
Con
dition
ing
–A
TA
-Chap
ter30.
Icean
dR
ainP
rotection
–A
TA
-Chap
ter32.
Lan
din
gG
ear
Note
.T
he
ATA
descrip
tionis
“slightly
outd
ated”
fromto
day
’spoin
tof
view
since
italread
ysu
ggestsa
function
↔sy
stemasso
ciationV
ERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Safe
ty-R
ela
ted
Sta
ndard
sfo
rC
ivil
Airc
raft
Syste
ms
Tech
nical
standard
ssu
chas
Radio
Tech
nic
alC
om
missio
nfo
rA
ero
nautic
s(R
TC
A)
and
Aero
nautic
alR
adio
Incorp
ora
ted
(AR
INC
)stan
dard
ssp
ecifyth
e(m
inim
al)req
uirem
ents
foreq
uip
men
tim
plem
entin
gaircraft
function
s
Exam
ple
s.
•A
RIN
C653:
Avion
icsA
pplication
Softw
areStan
dard
Interface
•A
RIN
C664:
Aircraft
Data
Netw
ork(A
vion
icsFull
Duplex
Sw
itched
Eth
ernet
(AFD
X))
•RT
CA
DO
-200A:Stan
dard
sfor
Pro
cessing
Aeron
autical
Data
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Safe
ty-R
ela
ted
Sta
ndard
sfo
rC
ivil
Airc
raft
Syste
ms
Tech
nical
standard
ssu
chas
Radio
Tech
nic
alC
om
missio
nfo
rA
ero
nautic
s(R
TC
A)
and
Aero
nautic
alR
adio
Incorp
ora
ted
(AR
INC
)stan
dard
ssp
ecifyth
e(m
inim
al)req
uirem
ents
foreq
uip
men
tim
plem
entin
gaircraft
function
s
Exam
ple
s.
•RT
CA
DO
-185B:M
inim
um
Operation
alPerform
ance
Stan
dard
sfor
Traffi
cA
lertan
dC
ollisionA
voidan
ceSystem
II(T
CA
SII)
•RT
CA
DO
-178B:Softw
areC
onsid
erations
inA
irborn
eSystem
san
dE
quip
men
tC
ertification
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Overv
iew
1.T
he
Notion
ofD
epen
dab
ility
2.Safety
-Related
Stan
dard
san
dV
-Models
3.M
odellin
gSafety
-Critical
System
s
4.H
azardA
naly
sisan
dR
iskA
ssessmen
t
5.D
esignC
riteriafor
Safety
-Critical
System
s
6.V
alidation
,V
erification
and
Test
ofSafety
-Critical
System
s
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Modellin
gSafe
ty-C
riticalSyste
ms
Co
ntro
ller
sen
so
r ob
serv
ab
les
actu
ato
r ob
serv
ab
les
ob
serv
ab
les
Co
ntro
l (EU
C)
Eq
uip
men
t un
der
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Modellin
gSafe
ty-C
riticalSyste
ms
•P
hysic
alM
odelsp
ecifies
how
the
Equip
ment
Under
Contro
l(E
UC
)beh
aves.T
his
model
shou
ldbe
indep
enden
tof
the
presen
ce/absen
ceof
acon
troller.
•(S
yste
m)
Hazard
Modeldescrib
esth
epossib
lecau
sesth
atm
aylead
toth
eid
entifi
edsy
stemhazard
s.
•C
ontro
ller
Model:
specifi
esreq
uirem
ents
fora
control
system
such
that
–E
UC
system
hazard
sw
illnot
occu
r(c
ontro
ller
safe
tyre
quire
ments)
–ad
dition
alnon
safety-related
EU
Cbeh
aviou
rw
illbe
ensu
red(u
ser
require
ments)
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Modellin
gSafe
ty-C
riticalSyste
ms
Obse
rvatio
ns:
•System
safetyreq
uirem
ents
areoften
specifi
edby
separate
auth
orities,not
by
the
custom
er.
•C
ontroller
safetyreq
uirem
ents
have
tobe
specifi
edby
the
teamresp
onsib
lefor
the
controller.
•U
serreq
uirem
ents
specifi
edby
the
custom
erm
aybe
incon
flict
with
safetyreq
uirem
ents.
•It
has
tobe
verified
that
the
controller
safetyreq
uirem
ents
will
fulfi
lth
esy
stemsafety
requirem
ents.
The
specifi
catio
nofcontro
ller
safe
tyre
quire
ments
should
alw
ays
be
separa
ted
from
use
rre
quire
ments.
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Modellin
gSafe
ty-C
riticalC
ontro
llers
–Specifi
catio
nM
eth
ods
Specifi
cationM
ethods
suitab
lefor
physical
model
and
controller
model
canbe
classified
accordin
gto
Method
DM
FM
UB
MT
BM
VV
TC
G
SA
/RT
/IM
++
·-
··
STAT
EC
HA
RT
S·
++
··
·
SD
L·
++
··
+
CSP
··
++
++
CC
S·
·+
++
+
LO
TO
S·
·+
-+
+
Z+
+·
-·
·
VD
M+
+·
-·
+
UM
L2.0
++
+·
++
Hybrid
UM
L+
++
++
+
DM
=D
ata
Model,
FM
=Functio
nalM
odel,U
BM
=U
ntim
ed
Behavio
uralM
odel,
TBM
=
Tim
ed
Behavio
uralM
odel,
VV
T=
support
for
Valid
atio
nVerific
atio
nand
Test,C
G=
support
for
code
generatio
nfr
om
specific
atio
ns
,+
=good
support
,·=
weak
support
,-=
no
support
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Modellin
gSafe
ty-C
riticalC
ontro
llers
–Specifi
catio
nM
eth
ods
Exam
ple
1:
Physical
Model
and
Con
trollerM
odel
specifi
cationw
ithC
SP
–door
lock
ing
mech
anism
forlab
oratory:
When
lasersy
stemis
active,door
shall
be
locked
and
laboratory
shall
be
empty.
Physical
model
(EU
C):
EUC
=LASER
|||
(DOOR
[|
open,
close
|]
PERSON)
LASER
=switchOn
->
laserActive
->
switchOff
->
laserPassive
->
LASER
DOOR
=open
->
close
->
DOOR
PERSON
=open
->
enter
->
close
->
stayInLaboratory
->
open
->
leave
->
close
->
PERSON
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Modellin
gSafe
ty-C
riticalC
ontro
llers
–Specifi
catio
nM
eth
ods
Exam
ple
1(c
ontin
ued):
Hazard
ous
tracesare
the
ones
contain
ing
event
sequen
ces
<...,open,switchOn,
...
>
<...,open,enter,switchOn,
...
>
<...,open,enter,close,switchOn,
...
>
<...,open,enter,close,stayInLaboratory,switchOn,
...
>
<...,open,enter,close,stayInLaboratory,open,switchOn,
...
>
<...,open,enter,close,stayInLaboratory,open,leave,switchOn,
...
<...,switchOn,open,...>
<...,switchOn,laserActive,open,...>
<...,switchOn,laserActive,switchOff,open,...>
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Modellin
gSafe
ty-C
riticalC
ontro
llers
–Specifi
catio
nM
eth
ods
Exam
ple
1(c
ontin
ued):
Hazard
ous
tracesare
formally
specifi
edusin
gtra
ce
specifi
catio
ns
HA
ZA
RD
(tr)≡
∃u1,u2,u3
:tr
=u1
⌢〈o
pen〉
⌢u2
⌢〈o
pen〉
⌢u3
⌢〈c
lose〉
∧#
(u1
↾{open})
mod
2=
0∧
u2
↾{open}
=〈〉∧
((u1
↾{sw
itchO
n,la
serA
ctiv
e,sw
itchO
ff,la
serP
assiv
e}6=
〈〉∧
last(u
1↾{sw
itchO
n,la
serA
ctiv
e,sw
itchO
ff,la
serP
assiv
e})6=
laserP
assiv
e)∨
(u2
↾{sw
itchO
n,la
serA
ctiv
e,sw
itchO
ff}6=
〈〉)
∨(u
3↾{sw
itchO
n,la
serA
ctiv
e,sw
itchO
ff}6=
〈〉))
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Modellin
gSafe
ty-C
riticalC
ontro
llers
–Specifi
catio
nM
eth
ods
Exam
ple
1(c
ontin
ued):
The
develop
men
tob
jectivefor
the
controller
isto
ensu
refor
system
SYSTEM
=EUC
[|
I|]
CONTROLLER
with
some
suitab
lein
terfaceI
the
safetysp
ecification
SY
ST
EM
sat¬
HA
ZA
RD
(tr)
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Modellin
gSafe
ty-C
riticalC
ontro
llers
–Specifi
catio
nM
eth
ods
Exam
ple
1(c
ontin
ued):
Asu
itable
interface
is
I=
{open
,close,switch
On,la
serPassiv
e}
and
asu
itable
controller
canbe
specifi
edas
CONTROLLER
=open
->
C1
[]
switchOn
->
C2
C1
=close
->
open
->
close
->
CONTROLLER
C2
=laserPassive
->
CONTROLLER
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Modellin
gSafe
ty-C
riticalC
ontro
llers
–Specifi
catio
nM
eth
ods
Exam
ple
1(c
ontin
ued):
The
safetyverifi
cationis
typically
perform
edby
usin
ga
watch
dog
pro
cess
which
mon
itorsall
events
fromth
esy
stemalp
hab
etA
and
induces
adead
lock
asso
onas
asafety
violation
occu
rs:
A=
{switchOn,laserActive,switchOff,laserPassive,
open,close,enter,stayInLaboratory,leave}
WATCHDOG
=W(<>)
W(tr)
=if
(\text{HAZARD}(tr)
)then
STOP
else
([]
e:A
@e->
W(tr^<e>))
IfVERIFY
=SYSTEM
[|
A|]
WATCHDOG
isfree
ofdead
lock
s,th
isproves
the
safetyof
SYSTEM
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Modellin
gSafe
ty-C
riticalC
ontro
llers
–Specifi
catio
nM
eth
ods
Exam
ple
1–
state
-base
ddesc
riptio
nw
ithsp
ecifi
catio
nin
tem
pora
llo
gic
:
Sta
te-T
ransitio
nSyste
mS
TS
=(S
,s0 ,V
,T):
•V
:Set
ofvariab
lesy
mbols
•S
:Set
ofstates,
eachstate
s∈
Sa
valuation
s:V
→D
ofvariab
les(D
the
associated
variable
dom
ain)
•T
⊆S×
S:
The
transition
relation
Run
(executio
n)
of
ST
S:
State
sequen
ce〈s
0 ,s1 ,s
2 ,...〉w
ith
∀i≥
0:(s
i ,si+
1 )∈
T
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Modellin
gSafe
ty-C
riticalC
ontro
llers
–Specifi
catio
nM
eth
ods
Exam
ple
1–
modelle
das
ST
S:
•V
ariable
sym
bols
V=
{door,la
ser,dcn
t}
•A
uxiliary
variable
dcn
t(“d
oor-op
en/closed
counter”)
counts
how
oftenth
edoor
has
been
open
edor
closed
•D
omain
sD
=D
(door)
∪D
(laser)
∪D
(dcn
t),D
(door)
={op
en,closed
},D
(laser)
={p
assiv
e,on,a
ctive,o
ff},
D(d
cnt)
=N
0
•V
aluation
canbe
written
likes
={d
oor7→
open
,laser
7→on
,dcn
t7→
5}
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Modellin
gSafe
ty-C
riticalC
ontro
llers
–Specifi
catio
nM
eth
ods
Exam
ple
1–
modelle
das
ST
S–
transitio
nre
latio
n:
T⊆
S×
S:
with
out
control,
Tallow
san
y(p
ossibly
unsafe)
transition
s(s,s
′)satisfy
ing
1.s0 (d
oor)=
closed∧
s0 (d
cnt)
=0∧
s0 (la
ser)=
passiv
e
2.s(la
ser)=
passiv
e⇒
s′(la
ser)∈{p
assiv
e,on}
3.s(la
ser)=
on⇒
s′(la
ser)∈{on
,activ
e}
4.s(la
ser)=
activ
e⇒
s′(la
ser)∈{a
ctive,o
ff}
5.s(la
ser)=
off
⇒s′(la
ser)∈{off
,passiv
e}
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Modellin
gSafe
ty-C
riticalC
ontro
llers
–Specifi
catio
nM
eth
ods
Exam
ple
1–
modelle
das
ST
S–
transitio
nre
latio
n:
(s(door)
=d∧
s(dcn
t)=
n)⇒
((s′(d
oor)=
d∧
s′(d
cnt)
=n)∨
(s′(d
oor)6=
d∧
s′(d
cnt)
=n
+1))
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Modellin
gSafe
ty-C
riticalC
ontro
llers
–Specifi
catio
nM
eth
ods
Exam
ple
1–
specifi
catio
nofsa
feru
ns
usin
gte
mpora
llo
gic
:
Recall
operators
ofLin
ear
Tem
pora
lLogic
(LT
L),
defi
ned
onru
ns
r=
〈s0 ,s
1 ,s2 ,...〉
ofS
TS
:
•Sta
tefo
rmula
sp
over
V:
logicform
ulas
with
freevariab
lesin
V,in
volvin
g∃,∀
,¬,∧
,∨,⇒
,⇐⇒
Inte
rpre
tatio
n:
phold
sin
states∈
Sif
itsvalu
ations(p
)is
true.
Exam
ple
:door
=op
en⇒
laser
6=activ
eis
interp
retedin
states
ass(d
oor)=
open
⇒s(la
ser)6=
activ
e
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Modellin
gSafe
ty-C
riticalC
ontro
llers
–Specifi
catio
nM
eth
ods
Exam
ple
1–
specifi
catio
nofsa
feru
ns
usin
gte
mpora
llo
gic
–Tem
pora
lopera
tors:
•G
lobally
p:
�p
(orG
p)
hold
sfor
run
riff
∀i≥
0:s
i (p)
•N
ext
p:©
p(or
Xp)
hold
sin
states
iof
run
riff
si+
1 (p)
istru
e
•Eventu
ally
(finally
)p:
♦p
(orF
p)
hold
sfor
run
riff
∃i≥
0:s
i (p)
•p
Until
q:pU
qhold
sfor
run
riff
(∃i≥
0:s
i (q))∧
(∀j
<i:s
j (p))
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Modellin
gSafe
ty-C
riticalC
ontro
llers
–Specifi
catio
nM
eth
ods
Exam
ple
1–
physic
alm
odelsp
ecifi
catio
nusin
gte
mpora
llo
gic
:
The
restrictions
abou
tth
ephysical
model
canbe
specifi
edas
follows:
1.s0 (d
oor)=
closed∧
s0 (d
cnt)
=0∧
s0 (la
ser)=
passiv
e
2.�
(laser
=passiv
e⇒
©(la
ser∈{p
assiv
e,on}))
3.�
(laser
=on
⇒©
(laser
∈{on
,activ
e}))
4.�
(laser
=activ
e⇒
©(la
ser∈{a
ctive,o
ff}))
5.�
(laser
=off
⇒©
(laser
∈{off
,passiv
e}))
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Modellin
gSafe
ty-C
riticalC
ontro
llers
–Specifi
catio
nM
eth
ods
Exam
ple
1–
physic
alm
odelsp
ecifi
catio
nusin
gte
mpora
llo
gic
:
∀d∈{op
en,closed
},n∈
N0
:�
(door
=d∧
dcn
t=
n⇒
©((d
oor=
d∧
dcn
t=
n)∨
(door
6=d∧
dcn
t=
n+
1)))
Obse
rve:
The
laboratory
isem
pty
ifan
don
lyif
dcn
tm
od
4=
0
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Modellin
gSafe
ty-C
riticalC
ontro
llers
–Specifi
catio
nM
eth
ods
Exam
ple
1–
hazard
specifi
catio
nusin
gte
mpora
llo
gic
:
Hazard
scan
be
characterised
by
formula
HA
ZA
RD
≡♦
((door
=op
en∨
dcn
tm
od
46=
0)∧la
ser6=
passiv
e)
Innatu
rallan
guage,
the
hazard
formula
expresses
•W
hen
everth
elaser
isnot
inpassive
state,it
ishazard
ous
ifth
edoor
isop
en.
•W
hen
everth
elaser
isnot
inpassive
state,it
ishazard
ous
ifsom
ebody
isin
side
the
laboratory
(thou
ghth
edoor
may
be
closed).V
ERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Modellin
gSafe
ty-C
riticalC
ontro
llers
–Specifi
catio
nM
eth
ods
Exam
ple
1–
safe
tysp
ecifi
catio
nusin
gte
mpora
llo
gic
:
The
associated
safetycon
dition
isth
enegation
ofth
ehazard
characterisation
:
¬H
AZA
RD
≡�
((door
=op
en∨
dcn
tm
od
46=
0)⇒
laser
=passiv
e)
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Modellin
gSafe
ty-C
riticalC
ontro
llers
–Specifi
catio
nM
eth
ods
Exam
ple
1–
state
-base
ddesc
riptio
n–
Form
alm
odelfo
rim
ple
menta
tions:
Tim
e-Discrete
Input-O
utp
ut
Hybrid
System
s
TD
IOH
SH
=(L
oc,Init,V
,I,O
,Tra
ns):
•L
oc:
Location
s
•V
:variab
lesy
mbols,
I,O
⊂V
,I∩
O=
∅
•G
uard
:quan
tifier-free
pred
icatesover
V
•Init
:L
oc→
Guard
:in
itialisationcon
straints
•A
ssign
the
setof
allpairs
(~x:=
~t)w
ith
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
–~x
=(x
1 ,x2 ,...)
vectorof
allvariab
lesin
V−
I
–~t
=(t
1 ,t2 ,...)
∈T
|V−
I|
–T
terms
overV
•T
rans⊆
Loc
×G
uard
×A
ssign×
Loc
:Tran
sitions
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Modellin
gSafe
ty-C
riticalC
ontro
llers
–Specifi
catio
nM
eth
ods
Exam
ple
1–
I/O
-safe
TD
IOH
S:P
rograms
adhere
toth
efollow
ing
pro
gra
mm
ing
model:
•P
rocessin
gis
perform
edin
aseq
uen
tialtask
operatin
gin
am
ainlo
opw
ithth
efollow
ing
pro
cessing
phases:
–In
put
phase
:In
puts
areread
and
copied
to(glob
al,static,
heap
orstack
)variab
les–
these
variables
arecalled
pro
cessin
gvaria
ble
s
–P
rocessin
gphase
:T
he
control
decision
sare
computed
,op
erating
onpro
cessing
variables
only
–O
utp
ut
phase
:T
he
pro
cessing
variables
contain
ing
pre-com
puted
outp
ut
values
arecop
iedto
their
correspon
din
gou
tputs
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Modellin
gSafe
ty-C
riticalC
ontro
llers
Exam
ple
1–
EU
Cand
Contro
ller:
EQ
UIP
ME
NT
UN
DE
R C
ON
TR
OL
DO
OR
SY
ST
EM
LA
SE
R
CO
NT
RO
LL
ER
do
orO
pen
:b
oo
l
do
or:
{op
en
,clo
sed
}
laserU
nlo
cked
:b
oo
lla
ser:
{passiv
e,o
n,a
ctiv
e,o
ff}d
oo
rReq
uest:
bo
ol
doorO
pen =
= tru
e
opened w
hen u
nlo
cked v
iaD
oor c
an o
nly
be
pre
ss a
request b
utto
n.
To o
pen d
oor, u
sers
US
ER
laserS
witc
h:
{on
,off}
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Exam
ple
1–
Contro
ller
Code:
1//Processingvariables(initialisedbyfalse/passive/closed)
2dop:bool;dx:{open,closed};lu:bool;l:{passive,on,active,off};
3dcnt:int;doorOld:bool;dr:bool;
4while(true)
{
5//Inputphase------------------------------------------------
6dx=
door;l
=laser;dr=
doorRequest;
7//Processingphase-------------------------------------------
8if(
dx!=doorOld){
doorOld=
dx;dcnt++}
9dop=dx
or(dcnt%4!=
0)or
(drandl==
passive);
10
lu=
notdop;
11
//Outputphase-----------------------------------------------
12
doorOpen=dop;laserUnlocked=
lu;
13
//Safetymonitor---------------------------------------------
14
if(
(dcnt%4!=0
ordoorOpenor
door==open)andlaser!=passive)
15
EMERGENCY_SHUTDOWN();//
Switchoffpowersupplytolaser
16
}
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Exam
ple
1–
Contro
ller
Code
Safe
tyP
roof:
First
prove
auxiliary
prop
ertyth
atdcn
tis
updated
asreq
uired
forth
ephysical
model,
i.e.,
accordin
gto
formula
∀d∈{op
en,closed
},n∈
N0
:�
(door
=d∧
dcn
t=
n⇒
©((d
oor=
d∧
dcn
t=
n)∨
(door
6=d∧
dcn
t=
n+
1)))(∗)
This
pro
offollow
sfrom
the
program
prop
ertyw
hich
hold
sat
line
8
∀d∈{op
en,closed
},n∈
N0
:�
(door
=d∧
dcn
t=
n∧©
(door
6=d⇔
doorO
ld6=
dx))
soth
ein
cremen
tdcnt++
inlin
e8
establish
es(*).
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Exam
ple
1–
Contro
ller
Code
Safe
tyP
roof:
Verifi
cationob
jective:Show
that
¬H
AZA
RD
hold
sat
the
end
ofeach
outp
ut
phase.
Pro
ofrelies
onphysic
alpro
perty
ofele
ctro
-mech
anic
al
door/
lase
rsa
fety
syste
m:
�(la
ser6=
passiv
e⇒
laserU
nlo
cked)
(1)
We
will
prove
that
pro
gra
mensu
res
�(d
oorO
pen
⇒¬
laserU
nlo
cked)
(2)�
(dcn
t%46=
0⇒
¬la
serUnlo
cked)
(3)�
(door
=op
en⇒
doorO
pen
)(4)
Prop
erties(1),(2),(3),(4)
obviou
slyim
ply
¬H
AZA
RD
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Exam
ple
1–
Contro
ller
Code
Safe
tyP
roof:
For
I/O-safe
TD
IOH
Sth
epro
ofof
aprop
erty�
φis
equivalen
tto
show
ing
that
φis
anin
variant
ofth
eprogram
’sm
ainw
hile-lo
op.
We
therefore
have
toprove
that
the
followin
gprop
ertiesare
invarian
ts:
doorO
pen
⇒¬
laserU
nlo
cked(2
′)dcn
t%46=
0⇒
¬la
serUnlo
cked(3
′)door
=op
en⇒
doorO
pen
(4′)
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Exam
ple
1–
Contro
ller
Code
Safe
tyP
roof:
Pro
ofis
based
onop
erational
seman
ticsof
while
langu
ages.
Prop
erties(2’),
(3’),(4’)
hold
when
initially
enterin
gth
ew
hile
loop
atlin
e4,
due
tovariab
lein
itialisations.
Now
suppose
that
(2’),(3’),
(4’)hold
inlin
e5
(sℓ
den
otesth
evariab
lestate
before
execu
tionof
line
ℓ):
s5|=
doorO
pen
⇒¬
laserU
nlo
cked
s5|=
dcn
t%46=
0⇒
¬la
serUnlo
cked
s5|=
door
=op
en⇒
doorO
pen
Ithas
tobe
show
nth
atth
enth
eseprop
ertiesalso
hold
atlin
e13
inprogram
states13 .
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Exam
ple
1–
Contro
ller
Code
Safe
tyP
roof:
Prop
erty(2’)
follows
fromth
eassign
men
tan
dseq
uen
ceru
lesof
the
operation
alsem
antics,
applied
tolin
es10
—12:
s12
=s10⊕
{lu
7→¬
s10 (d
op)}
s13
=s12⊕
{doorO
pen
7→s12 (d
op),la
serUnlo
cked7→
s12 (lu
)}
which
implies
s13 (la
serUnlo
cked)
=¬
s10 (d
op)
and,sin
ces12 (d
op)=
s10 (d
op),
s13|=
doorO
pen
⇒¬
laserU
nlo
cked
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Exam
ple
1–
Contro
ller
Code
Safe
tyP
roof:
Prop
erty(3’)
follows
fromth
eassign
men
tan
dseq
uen
ceru
lesof
the
operation
alsem
antics,
applied
tolin
es9
—12:
s10
=s9⊕{dop7→
(s9 (d
x)=
open
)∨
s9 (d
cnt)%
46=
0∨
(s9 (d
r)∧
s9 (l)
=passiv
e)}s12
=s10⊕
{lu
7→¬
s10 (d
op)}
s13
=s12⊕
{doorO
pen
7→s12 (d
op),la
serUnlo
cked7→
s12 (lu
)}
which
implies
s13|=
dcn
t%46=
0⇒
doorO
pen
and
therefore
s13|=
dcn
t%46=
0⇒
¬la
serUnlo
cked
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Exam
ple
1–
Contro
ller
Code
Safe
tyP
roof:
Prop
erty(4’)
follows
fromth
eassign
men
tan
dseq
uen
ceru
lesof
the
operation
alsem
antics,
applied
tolin
es6
—12:
s10
=s9⊕{dop7→
(s6 (d
oor)
=op
en)∨
s9 (d
cnt)%
46=
0∨
(s9 (d
r)∧
s9 (l)
=passiv
e)}s12
=s10⊕
{lu
7→¬
s10 (d
op)}
s13
=s12⊕
{doorO
pen
7→s12 (d
op),la
serUnlo
cked7→
s10 (lu
)}
which
impliess
13|=
door
=op
en⇒
dop
and
therefore
s13|=
door
=op
en⇒
doorO
pen
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Overv
iew
1.T
he
Notion
ofD
epen
dab
ility
2.Safety
-Related
Stan
dard
san
dV
-Models
3.M
odellin
gSafety
-Critical
System
s
4.H
azardA
naly
sisan
dR
iskA
ssessmen
t
5.D
esignC
riteriafor
Safety
-Critical
System
s
6.V
alidation
,V
erification
and
Test
ofSafety
-Critical
System
s
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Risk
Analy
sis
Risk
Analy
sis:con
sistsof
two
steps:
Hazard
Analy
sisan
dR
iskA
ssessm
ent
Hazard
Analy
sis:W
hich
condition
scau
sea
hazard
?
Risk
Asse
ssment:
What
areth
eprob
abilities
that
ahazard
occu
rs,prov
ided
the
hazard
analy
sisis
consisten
tan
dcom
plete?
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Modellin
gTech
niq
ues
for
Haza
rdA
naly
sis
Overv
iew
ofte
chniq
ues:
•FM
EA
:Failu
rem
odes
and
effects
analy
sis
•FM
EC
A:Failu
rem
odes,
effects
and
criticalityan
alysis
•H
AZO
P:H
azardan
dop
erability
studies
•ETA
:E
vent
treean
alysis
•FTA
:Fau
lttree
analy
sis
•FSM
:Fin
itestate
mach
ines
with
reachab
ilityan
alysis
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Haza
rdA
naly
sis–
FM
EC
A
Obje
ctiv
e:
Investigate
the
possib
lew
ays
(=m
odes)
acom
pon
ent
may
failan
dch
eckw
heth
erth
iscan
leadto
ahazard
.
Advanta
ges:
System
atican
alysis
ofeach
possib
lecom
pon
ent
failure
and
itshypoth
eticrelation
ship
toa
hazard
Disa
dvanta
ges:
•In
vestigationof
man
ycom
pon
ents
which
have
no
effect
onhazard
occu
rrence
•N
oan
alysis
ofsim
ulta
neous
faults
inseveral
compon
ents
Recom
mendatio
n:
Use
tocom
plem
ent
FTA
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Haza
rdA
naly
sis–
FM
EC
A
Item
Failu
re
Mode
Cause
of
failu
re
Possib
le
Effects
Prob.
Crit
icalit
yPossib
leA
ctio
ns
to
Reduce
Fail-
ure
Rate
or
Ef-
fects
Air
craft
Sm
oke
Detector
Sm
oke
threshold
too
hig
h
(1)
De-
fect
hum
idity
sensor
(2)
Arit
h-
metic
error
inthreshold
calc
u-
latio
nsoftware
Sm
oke
incom
-partm
ent
rem
ain
sunde-
tected
10−
6/1000·
flight
hours
B(1)
Use
redundant
sm
oke
detectors
(2)
Perfo
rm
detec-
tor
tests
(3)
Let
detector
issue
pre
-thre
shold
warnin
g:
This
in-
dic
ates
that
the
detector
becom
es
“blin
d”
so
that
hig
her
sm
oke
intensit
ies
are
requir
ed
to
lead
to
asm
oke
ala
rm
Air
craft
Sm
oke
Detector
Sm
oke
threshold
too
low
–see
above
–Ille
gal
sm
oke
ala
rm
s
10−
6/1000·
flight
hours
C(A
ir-
craft
cannot
start
or
contin
ue
flight)
–see
above
–
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Haza
rdA
naly
sis–
Fault
Tre
es
inte
rme
dia
te e
ven
t
exte
rna
l even
t
un
de
ve
lop
ed
eve
nt
co
nd
ition
ing
even
t
ba
sic
eve
nt
AN
D g
ate
OR
gate
INH
IBIT
ga
te
PR
IOR
ITY
AN
D
ga
te
EX
CL
US
IVE
OR
ga
te
E1
E2
1E
22
E2
3
CE
E3
1E
33
E3
2
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Deriv
ing
Safe
tyR
equire
ments
from
Hazard
Analy
sis
Ifhazard
analy
sishasbeen
perfo
rmed
usin
gfa
ult
trees,
ev-
ery
setofconditio
nsensu
ring
thatth
ero
otofth
efa
ult
tree
will
never
be
reach
ed
repre
senta
suffi
cie
ntse
tofsa
fety
re-
quire
ments.
•ro
ot-hazard
E0
givesrise
toin
itialreq
uirem
entnot(E
0)
tobe
refined
by
the
requirem
ents
derived
fromlow
er-levelnodes
inth
etree
•O
R-gates
E1,...,E
ngive
riseto
requirem
ent
not(E
1)
AN
D...A
ND
not(E
n)
•A
ND
-gatesE
1,...,En
giverise
toreq
uirem
ent
not(E
1)
OR
...O
Rnot(E
n)
•refi
nem
ent
ofreq
uirem
ents
stops
when
the
leavesof
the
fault
treehave
been
reached
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Deriv
ing
Safe
tyR
equire
ments
from
Hazard
Analy
sis
Exam
ple
for
fault-tre
esh
ow
nabove.
Ste
p1:
term
repla
cem
ent
accord
ing
tofa
ult-tre
e
E1
⇐⇒
E21
and
E22
and
E23
⇐⇒
(E31
and
not(C
E))
and
E22
and
(E32
or
E33)
Ste
p2:
transfo
rmin
todisju
nctiv
enorm
alfo
rm
⇐⇒
(E31
and
not(C
E)
and
E22
and
E32)
or
(E31
and
not(C
E)
and
E22
and
E33)
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Deriv
ing
Safe
tyR
equire
ments
from
Hazard
Analy
sis
Exam
ple
for
fault-tre
esh
ow
nabove.
Ste
p3:
dete
rmin
ere
sultin
gsa
fety
require
ments
by
negatio
nofdisju
nctiv
enorm
alfo
rm
Safe
tyR
equire
ment
1:
not(E
31and
not(C
E)
and
E22
and
E32)
Safe
tyR
equire
ment
2:
not(E
31and
not(C
E)
and
E22
and
E33)
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Deriv
ing
Safe
tyR
equire
ments
from
Hazard
Analy
sis
Definitio
nofC
ut
Sets:
Each
(min
imal)
collectionof
elemen
tarycon
dition
sjoin
tlylead
ing
toa
hazard
iscalled
acu
tset.
Definitio
nofSin
gle
-Poin
tFailu
re:
Cut
setcon
tainin
gon
lyon
eelem
ent
Cut
sets
deriv
ed
from
fault-tre
esh
ow
nabove:
givenby
disju
nctive
norm
alform
as
{E31,
not(C
E),E
22,E32}
{E31,
not(C
E),E
22,E33}
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Deriv
ing
Safe
tyR
equire
ments
from
Hazard
Analy
sis
Exam
ple
1(c
ontin
ued):
FTA
forlab
oratoryw
ithlaser
Ha
za
rd
lase
r != p
assiv
eP
ers
on
in la
se
r ran
ge
do
or =
= o
pe
nP
ers
on
in la
bo
rato
ry
dcn
t%4
!= 0
lase
rUn
locke
dla
se
rSw
itch
==
on
do
orO
pe
nD
oo
r op
en
d b
y u
se
r
co
ntro
llab
le e
ve
nt
co
ntro
llab
le e
ve
nt
un
co
ntro
llab
le e
ve
nt
un
co
ntro
llab
le e
ve
nt
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Deriv
ing
Safe
tyR
equire
ments
from
Hazard
Analy
sis
Exam
ple
1(c
ontin
ued):
Cut
setsfor
laboratory
with
laserderived
fromFTA
above:
C1
={doorO
pen
,laserU
nlo
cked,D
oor
open
edby
user,
laserS
witch
==
on}
C2
={dcn
t%46=
0,laserU
nlo
cked,la
serSw
itch=
=on}
All
uncon
trollable
events
areassu
med
toalw
ays
hap
pen
inord
erto
contrib
ute
toa
hazard
situation
.T
herefore
the
controller
has
toen
sure
that
event
combin
ations
C′1=
{doorO
pen
,laserU
nlo
cked}
C′2=
{dcn
t%46=
0,laserU
nlo
cked}
cannever
occu
r:
Safety
Req
uirem
ent
1≡
�(¬
(doorO
pen
∧la
serUnlo
cked))
Safety
Req
uirem
ent
2≡
�(¬
(dcn
t%46=
0∧
laserU
nlo
cked))
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Deriv
ing
Safe
tyR
equire
ments
from
Hazard
Analy
sis
Exam
ple
1(c
ontin
ued):
Inord
erto
prove
that
the
control
program
forth
elab
oratoryin
troduced
above
reallyfu
lfills
safetyreq
uirem
ents
1an
d2
above
we
have
tosh
owth
at
INV≡
¬(d
oorO
pen
∧la
serUnlo
cked)∧
¬(d
cnt%
46=
0∧
laserU
nlo
cked)
isan
invarian
tof
the
program
’sw
hile
loop
.O
bserve
that
the
fault
treean
alysis
above
relieson
the
addition
alfact
that
door
=op
en⇒
doorO
pen
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Contro
ller
Hazard
Analy
sis
Safety
mech
anism
shave
tobe
ensu
redeven
inpresen
ceof
intern
alsy
stemfau
lts.
Ase
cond
inte
rnalhazard
analy
sisis
necessa
ryto
show
that
the
syste
mdesig
nconta
ins
the
pro
per
mech
anism
sto
tol-
era
tefa
ults
with
out
vio
latin
gth
eesse
ntia
lsa
fety
require
-m
ents.
Intern
alhazard
analy
sissh
ould
takein
toaccou
nt:
•fau
lts/errors/failures
ofhard
ware
compon
ents
•erron
eous
beh
aviou
rof
SW
compon
ents
•corru
pted
data
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Risk
Asse
ssment
-G
enera
lD
efinitio
ns
Unre
liability
:
Defi
ne
the
Unre
liability
Functio
nQ
(t),th
atis,
the
prob
ability
fora
compon
ent
tofail
inin
terval[0,t],
as
Q(t)
=
∫
t
0
f(u
)du
where
f(t)
isa
Pro
bability
Density
Functio
n(P
DF),
the
failu
redensity
.M
athem
aticallysp
eakin
g,Q
(t)is
aC
um
ula
tive
Density
Functio
n(C
DF).
Obviou
sly
Q(∞
)=
∫
∞
0
f(u
)du
=1
since
f(t)
isa
PD
F.
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Risk
Asse
ssment
-G
enera
lD
efinitio
ns
Defi
ne
the
Relia
bility
Functio
nR
(t)as
the
prob
ability
fora
compon
entnot
tofail
inin
terval[0,t],
that
is,
R(t)
=1−
Q(t)
IfQ
(t)is
defi
ned
asab
ovew
ithden
sityfu
nction
f(t),
then
R(t)
=1−
∫
t
0
f(u
)du
=
∫
∞
0
f(u
)du−
∫
t
0
f(u
)du
=
∫
∞
t
f(u
)du
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Risk
Asse
ssment
-G
enera
lD
efinitio
ns
Q(t)
may
be
determ
ined
approx
imately
by
testing
alarge
num
ber
Nof
iden
ticalcom
pon
ents
and
setting
Q(t)
=n
f(t)
N,
where
nf(t)
≤N
isth
enum
ber
ofcom
pon
ents
which
failedin
interval
[0,t].
Con
versely,R
(t)m
aybe
determ
ined
approx
imately
by
testing
alarge
num
ber
Nof
iden
ticalcom
pon
ents
and
setting
R(t)
=n(t)
N,
where
n(t)
≤N
isth
enum
ber
ofcom
pon
ents
stillfu
nction
ing
correctlyat
aftera
time
interval
oflen
gtht
has
passed
.
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Risk
Asse
ssment
-G
enera
lD
efinitio
ns
Failu
reD
ensity
:
Indicates
tenden
cy(“con
stant”,
“strongly
/weak
lyin
creasing/d
ecreasing”)
ofunreliab
ilityat
time
poin
tt:
f(t)
=dQ
(u)
du
(t)
fordiff
erentiab
leunreliab
ilityfu
nction
Q.
This
implies
dR
(u)
du
(t)=
1−
Q(u
)
du
(t)
=−
dQ
(u)
du
(t)
=−
f(t)
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Risk
Asse
ssment
-G
enera
lD
efinitio
ns
Failu
reR
ate
:
Prob
ability
forsy
stemto
failin
interval
[t,t+
∆t],
prov
ided
that
no
failure
occu
rredbefore
t:
Failu
reR
ate=
F(t
+∆
t)−
F(t)
∆tR
(t)
Hazard
Rate
:
Lim
itof
failure
ratefor
∆t→
0:
z(t)
=lim
∆t→
0
F(t
+∆
t)−
F(t)
∆tR
(t)=
f(t)
R(t)
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Risk
Asse
ssment
-G
enera
lD
efinitio
ns
Mean
Tim
eTo
Failu
re:
Expected
value
ofth
etim
eth
esy
stemw
illop
eratebefore
the
occu
rrence
ofth
efirst
failure.
MT
TF
=
∫
∞
0
R(u
)du
Mean
Tim
eTo
Repair:
Average
repair
time.
Mean
Tim
eB
etw
een
Failu
res:
MT
BF
=M
TT
R+
MT
TF
prov
ided
that
the
system
sis
“asgo
od
asnew
”after
eachrep
air(i.e.
MT
TF
stays
the
same).
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Risk
Asse
ssment
-G
enera
lD
efinitio
ns
Availa
bility
:
Prob
ability
forth
esy
stemto
function
correctlyat
agiven
time.
Availab
ility=
MT
TF
MT
TF
+M
TT
R=
MT
TF
MT
BF
Unavaila
bility
:Unavailab
ility=
1−
Availab
ility
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Relia
bility
Modellin
g-
Com
bin
atio
nalM
odels
Ase
ries
com
bin
atio
nofcom
ponents
12
3n
inp
ut
ou
tpu
t
IfR
i (t)is
the
reliability
ofcom
pon
ent
i,an
dth
ecom
pon
ents
arein
dep
enden
t,th
esy
stemreliab
ilitycom
putes
to
R(t)
=
n∏i=
1
Ri (t)
Exam
ple
:A
seriescom
bin
ationof
100in
dep
enden
tcom
pon
ents
with
Ri (t)
=0.999
foreach
i,yield
sa
lowsy
stemreliab
ilityof
R(t)
=0.999
100
=0.905.
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Relia
bility
Modellin
g-
Com
bin
atio
nalM
odels
Apara
llelcom
bin
atio
nofcom
ponents
12n
inp
ut
ou
tpu
t IfR
i (t)is
the
reliability
ofcom
pon
ent
i,an
dth
ecom
pon
ents
arein
dep
enden
t,th
esy
stemreliab
ilitycom
putes
to
R(t)
=1−
n∏i=
1 (1−
Ri (t))
Exam
ple
:A
parallel
combin
ationof
3in
dep
enden
tcom
pon
ents
with
Ri (t)
=0.999
foreach
i,yield
sa
system
reliability
of
R(t)
=1−
(1−
0.999)3
=0.999
999999
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Relia
bility
Modellin
g-
Com
bin
atio
nalM
odels
Serie
s-para
llelcom
bin
atio
ns
234
1
57
68
9
inp
ut
ou
tpu
t
From
the
above
formulas,
the
system
reliability
iseasily
computed
intw
ostep
s,usin
gth
efollow
ing
abstraction
:
1in
pu
t1
01
1
9
ou
tpu
t
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
N-m
odula
rR
edundancy
Use
Nred
undan
tcom
pon
ents.
The
system
will
function
correctlyif
atleast
m<
Nm
odules
areop
erational.
Exam
ple
:N
=3,m
=2
indep
enden
t,voter
correct.
123
vo
ter
mu
ltica
st
relia
ble
2 / 3
R(t)
=R
1 (t)R2 (t)R
3 (t)+
(1−
R1 (t))R
2 (t)R3 (t)
+R
1 (t)(1−
R2 (t))R
3 (t)+
R1 (t)R
2 (t)(1−
R3 (t))
For
R1 (t)
=R
2 (t)=
R3 (t)
=0.95
hold
sR
(t)=
0,993,
but
R1 (t)
=R
2 (t)=
R3 (t)
=0.40
yields
R(t)
=0,352
.
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Failu
reD
istributio
ns
Exponentia
lpro
bability
density
functio
n
f(u
)=
λe−
λu
Sin
ced(−
e−
λu)
du
(t)=
f(t),
this
distrib
ution
leads
toa
failure
prob
ability
(the
so-calledexponentia
lfa
ilure
law
)
Q(t)
=
∫
t
0
f(u
)du
=−
e−
λt−
(−e−
λ0)
=1−
e−
λt
and
areliab
iltyR
(t)=
e−
λt
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Failu
reD
istributio
ns
Exponentia
lpro
bability
density
functio
nand
hazard
rate
The
expon
ential
PD
Flead
sto
ahazard
rateof
z(t)
=f(t)
R(t)
=λe−
λt
e−
λt
=λ
Inte
rpre
tatio
n:
The
expon
ential
PD
Fis
approp
riatefor
system
compon
ents
durin
gth
eirm
ostreliab
lephase
oflife
(the
use
ful
lifesta
ge),
where
the
hazard
ratedoes
not
dep
end
ontim
e,but
iscon
stant.
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Failu
reD
istributio
ns
Exponentia
lpro
bability
density
functio
nand
MT
TF
The
expon
ential
PD
Flead
sto
anM
TT
F
MT
TF
=
∫
∞
0
R(u
)du
=
∫
∞
0
e−
λudu
=lim
t→∞
(−1λe−
λt)−
(−1λe−
λ0)
=1λ
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Failu
reD
istributio
ns
Experim
enta
ldete
rmin
atio
nofexponentia
lP
DF
Sin
ceR
(t)=
e−
λt
we
candeterm
ine
anap
prox
imate
value
ofλ,if
testsof
the
form
R(t
i )=
n(t
i )
N,i
=1,...,m
(Na
largenum
ber
ofid
entical
compon
ents,
n(t
i )th
enum
ber
ofcom
pon
ents
stillop
erable
attim
eti )
canbe
perform
ed:
Solve
optim
isationprob
lem
min
imise
Φ(λ
)=
def
m∑i=
1 (R(t
i )−
e−
λti)
2
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Failu
reD
istributio
ns
Experim
enta
ldete
rmin
atio
nofexponentia
lP
DF
Alo
calm
inim
um
ofΦ
(λ)
canbe
found
by
solvin
geq
uation
dΦ
/dλ(u
)=
0,th
atis,
m∑i=
1
ti e
−λ
ti(R
(ti )−
e−
λti)
=0
The
root
λ0∈
Rw
ithdΦ
/dλ(λ
0 )=
0can
be
approx
imated
,for
exam
ple,
by
usin
gN
ewton
’sm
ethod.
Φis
am
axim
um
likelih
ood
estim
ato
rif
the
errorsoccu
rring
inth
em
easurem
ent
oftihave
norm
aldistrib
ution
.
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Failu
reD
istributio
ns
Experim
enta
ldete
rmin
atio
nofexponentia
lP
DF
If,how
ever,th
eti -m
easurem
ents
arerath
eruniform
elydistrib
uted
and/or
contain
freakvalu
es,oth
erestim
atorsare
preferab
le,e.
g.
Ψ(λ
)=
def
m∑i=
1
|R(t
i )−
e−
λti|
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Failu
reD
istributio
ns
The
Weib
ull
PD
Fis
used
forsitu
ations
where
time-d
epen
den
thazard
rateshave
tobe
consid
ered.
The
general
formof
the
Weib
ull
distrib
ution
has
three
freeparam
eters:
f(u
)=
βη
(
u−
γ
η
)
β−
1
e−
u−
γ
η
!
β
which
isdefi
ned
foru−
γ≥
0,β>
0,η>
0,γ∈
R
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Failu
reD
istributio
ns
The
failure
prob
ability
F(t)
forth
egen
eralW
eibull
PD
Fis
F(t)
=
∫
t
0
βη
(
u−
γ
η
)
β−
1
e−
u−
γ
η
!
β
du
=1−
e−
t−
γ
η
!
β
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Failu
reD
istributio
ns
The
param
etersare
called
•sc
ale
para
mete
rη
•sh
ape
para
mete
r(slo
pe)
β
•lo
catio
npara
mete
rγ:
γ6=
0is
used
ifth
eex
perim
ent
does
not
startat
u=
0,but
atan
ytim
eγ∈
R
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Failu
reD
istributio
ns
•For
experim
ents
starting
attim
eu
=0
we
canuse
the
two-p
ara
mete
rW
eib
ull
PD
Fby
setting
γ=
0:
f(u
)=
βη
(
uη
)
β−
1
e−
uη
!
β
•For
experim
ents
starting
attim
eu
=0
with
know
nparam
eterβ
=C
we
getth
eone-p
ara
mete
rW
eib
ull
PD
F
f(u
)=
Cη
(
uη
)
C−
1
e−
uη
!
C
•For
experim
ents
starting
attim
eu
=0
with
know
nβ
=1
we
getth
eex
pon
ential
PD
Fw
ithλ
=1η
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Failu
reD
istributio
ns
The
MT
TF
forth
eth
ree-param
eterW
eibull
PD
Fis
MT
TF
=
∫
∞
0
βη
(
u−
γ
η
)
β−
1
e−
u−
γ
η
!
β
du
=γ
+η·Γ
(
1β+
1
)
with
the
gam
ma
functio
nΓ(n
)defi
ned
as
Γ(n
)=
∫
∞
0
e−
uu
n−
1du
Observe
that
forγ
=0,η
=1λ,β
=1
this
coincid
esw
ithth
eM
TT
Fof
the
expon
ential
PD
F,sin
ceΓ(1)
=1
and
Γ(x
+1)
=x·Γ
(x),
soΓ(2)
=1.
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Failu
reD
istributio
ns
The
Weib
ull
hazard
rate
isgiven
by
z(t)
=βη
(
t−
γ
η
)
β−
1
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Overv
iew
1.T
he
Notion
ofD
epen
dab
ility
2.Safety
-Related
Stan
dard
san
dV
-Models
3.M
odellin
gSafety
-Critical
System
s
4.H
azardA
naly
sisan
dR
iskA
ssessmen
t
5.D
esignC
riteriafor
Safety
-Critical
System
s
6.V
alidation
,V
erification
and
Test
ofSafety
-Critical
System
s
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Desig
nC
riteria
for
Safe
ty-C
ritical
Syste
ms
•Partitio
nin
g:
Fau
ltsan
derrors
caused
by
non
-criticalpro
cessessh
ould
not
be
able
toaff
ectth
ebeh
aviou
rof
criticalpro
cesses.T
his
isach
ievedby
–m
emory
protection
forreach
able
address
ranges
–rob
ust
interfaces
tocritical
pro
cesses(ab
stractdata
types
usin
gpartition
edsh
aredm
emory,
validity
check
sfor
data)
–lim
itedresou
rces(C
PU
,IO
-chan
nels,
mem
ory)
fornon
-criticalpro
cesses
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Desig
nC
riteria
for
Safe
ty-C
riticalSyste
ms
•Sta
te-B
ase
dB
ehavio
ur:
Critical
system
beh
aviou
rsh
ould
not
dep
end
oneven
ts(i.
e.sh
ortsign
alsor
pulses
which
may
be
lost)but
onstate:
–in
put
messages
arestored
instate
space
–rep
etitionof
the
same
input
shou
ldnot
leadto
statech
anges
(e.g.,
send
absolu
tevalu
esin
steadof
delta-valu
esreferrin
gto
prev
ious
state/message)
–after
system
crash,last
statesh
ould
be
recoverable
fromen
viron
men
tan
d/or
stable
storage
Afte
rsy
stem
initia
lisatio
n,
there
should
be
no
more
dy-
nam
icm
em
ory
allo
catio
nat
run-tim
e.
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Desig
nC
riteria
for
Safe
ty-C
riticalSyste
ms
•H
ard
Real-T
ime
Behavio
ur:
Real-tim
ebeh
aviou
rsh
ould
be
based
ondisc
rete
time
sem
antic
s:
–late
messages
areeq
uivalen
tto
lostm
essages
–fixed
-cycle/fi
xed
transm
issionlen
gthfram
eproto
colsfor
comm
unication
–ap
plication
ssh
ould
operate
infixed
time
frames
–fixed
hou
sekeep
ing
phase
reservedfor
I/Opro
cessing
and
safetycon
trolm
echan
isms
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Desig
nC
riteria
for
Safe
ty-C
riticalSyste
ms
•U
seofO
pera
ting
Syste
m:
Preferab
ly,th
eop
erating
system
shou
ldon
lybe
used
toserv
iceH
Win
terrupts
forI/O
pro
cessing.
–ap
plication
ssh
ould
operate
accordin
gto
the
statem
achin
eprogram
min
gparad
igm,w
ithrou
nd
robin
sched
ulin
gor
coop
erativem
ulti
taskin
g
–use
shared
mem
ory(p
rotectedby
abstract
data
types
and
partition
ing)
forpro
cesscom
munication
instead
ofsign
als,m
essagequeu
es,pip
esetc
–use
tightly
coupled
multi-p
rocessor
system
sw
ithsim
ple
pro
cess-to-CP
Uallo
cationin
steadof
complex
sched
ulin
gpolicies
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
–on
multi-p
rocessor
system
suse
activew
aitan
dpollin
gm
echan
isms
instead
ofsem
aphores,
signals
and
alarms
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Desig
nC
riteria
for
Safe
ty-C
riticalSyste
ms
•Fault-T
ole
rance:
For
safety-critical
system
s,at
leastsafety
requirem
ents
must
hold
evenin
presen
ceof
faults
caused
by
environ
men
tor
by
the
system
itself.
–sp
ecifysta
ble
safe
state
s
–design
robust
mech
anism
s(H
W/S
W)
which
guaran
teetran
sitionin
tostab
lesafe
statein
caseof
errorsth
atm
ight
otherw
iselead
tocatastrop
hic
beh
aviou
r
–use
activ
ere
plic
atio
ntech
niq
ues
toen
sure
safetyreq
uirem
ents,
ifstab
lesafe
statecan
not
be
found
(forex
ample
inth
econ
trolof
aircraften
gines)
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
–ex
ploit
data,
code
and
HW
redundan
cyto
iden
tifyfau
ltycom
pon
ents
–use
fail-stopcom
pon
ents
oroth
erwise
use
Byzan
tine
agreemen
tproto
colsto
reachcon
sent
betw
eencorrectly
operatin
gcom
pon
ents
toisolate
faulty
ones
–use
watch
dog
mech
anism
sw
hich
guaran
teeth
aterro
rsare
reveale
d“as
soon
as
possib
le”
Obse
rve
thateven
fully
verifi
ed
SW
may
lead
toerro
neous
syste
mbehavio
ur
due
totra
nsie
nt
HW
erro
rs.A
sa
con-
sequence,
SW
div
ersity
(code
and
data
)is
mandato
ryif
HW
redundancy
isnot
availa
ble
.
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Desig
nC
riteria
for
Safe
ty-C
riticalSyste
ms
Inste
ad
ofre
-inventin
gevery
new
safe
ty-c
riticalcontro
ller
from
scra
tch,
use
Colla
bora
tions,
Desig
nPatte
rns
and
Fra
mew
ork
sw
ithpro
ven
generic
corre
ctn
ess
pro
pertie
s .
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Desig
nC
riteria
for
Safe
ty-C
riticalSyste
ms
Colla
bora
tion:
Descrip
tionof
acollection
ofco
operatin
gob
jectsby
specifi
cationof
their
•sta
ticpro
pertie
s:
–arch
itecture
ofth
eco
operatin
gob
jects
–roles
ofth
eob
jects
–relation
ship
sbetw
eenob
jects
•dynam
icpro
pertie
sre
gard
ing
the
messa
ge
flow
(interaction
s)betw
eenob
jects:
–seq
uen
cing
–sy
nch
ronisation
–tim
ing
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Desig
nC
riteria
for
Safe
ty-C
riticalSyste
ms
Desig
nPatte
rn:
•gen
eric“sm
all”collab
oration(a
“mech
anism
”)
•usab
lein
diff
erent
application
contex
ts
•gen
ericparam
etersare
Nam
e,D
ataT
ype,
Num
ber
ofO
bjects,
Meth
ods
etc.
Exam
ple
sfo
rD
esig
nPatte
rns:
•R
eader-W
riter-R
ingbuffer
Patte
rn:
generic
model
forth
easy
nch
ronou
sFIF
Ocom
munication
betw
eenread
eran
dw
ritertask
sw
ithou
tsem
aphore
utilisation
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
•M
odel-V
iew
-Contro
ller
Patte
rn:
generic
model
forth
ein
teractionbetw
een
–ap
plication
objects
(“models”)
–visu
alisationob
jects(“v
iews”)
–in
teractioncon
trolob
jects(“con
troller”)
•In
dex-S
tack
/D
ata
-Arra
yPatte
rn:
dynam
icdata
man
agemen
tw
ithou
tutilisation
ofop
erating
system
data
allocation
facilities
•M
ultip
lexer/
Concentra
tor
Patte
rn:
generic
pro
cesscom
munication
model
guaran
teeing
dead
lock
-freecom
munication
netw
orks
under
bou
ndary
condition
sw
hich
aretriv
ialto
check
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Desig
nC
riteria
for
Safe
ty-C
riticalSyste
ms
Fra
mew
ork
:
•gen
eric“large”
collaboration
•tay
loredfor
specifi
cfield
sap
plication
Exam
ple
sfo
rFra
mew
ork
s:
•Funds
Tra
nsfe
rFra
mew
ork
:gen
ericm
odel
fortran
sactionm
anagem
ent
ofban
kaccou
nts
•Safe
tyA
rchite
ctu
reFra
mew
ork
:gen
ericarch
itecture
forsafety
-criticalfail-stop
controllers
with
out
controller
redundan
cy(ex
plain
edin
more
detail
below
)
•M
aste
r-Sta
ndby
Fra
mew
ork
:gen
ericarch
itecture
forfau
lt-tolerant
2-redundan
tm
aster/slavesy
stem
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
•N
-Modula
rR
edundancy/V
ote
rFra
mew
ork
(n≥
3):n-red
undan
tfau
lt-tolerant
system
sw
ithactive
replication
–su
itable
forhard
real-time
application
s
•N
-Modula
rR
edundancy/B
yzantin
eA
gre
em
ent
Fra
mew
ork
(n≥
4):n-red
undan
tfau
lt-tolerant
system
sw
ithactive
replication
–su
itable
forhard
real-time
application
s–
protects
against
Byzan
tine
compon
ent
failures
–gu
arantees
agreemen
tbetw
eennon
-faulty
compon
ents
•T
ime-F
ram
eC
om
munic
atio
n/Sch
edulin
gFra
mew
ork
:gen
ericm
odel
forsch
edulin
gan
dcom
munication
ina
netw
orkof
controllers
with
hard
real-time
requirem
ents
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Exam
ple
:Safe
tyA
rchite
ctu
reFra
mew
ork
. . . . . . . . . . . .
US
ER
-INT
ER
FA
CE
LA
YE
R
SA
FE
TY
LA
YE
R
SY
ST
EM
INT
ER
FA
CE
LA
YE
R
WA
TC
HD
OG
SO
FT
WA
RE
SH
AR
ED
ME
MO
RY
INT
ER
FA
CE
(PE
RS
IST
EN
T)
SH
AR
ED
ME
MO
RY
INT
ER
FA
CE
(PE
RS
IST
EN
T)
CO
NF
IG-
DA
TA
SA
FE
TY
-CR
ITIC
AL
CO
NT
RO
LL
ER
US
ER
S
SY
ST
EM
AU
DIT
ING
EX
TE
RN
AL
SY
ST
EM
S
EX
TE
RN
AL
AU
DIT
ING
SY
ST
EM
EX
TE
RN
AL
HW
WA
TC
HD
OG
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Exam
ple
:Safe
tyA
rchite
ctu
reFra
mew
ork
SA
FET
YLA
YER
•takes
safety-critical
control
decision
s
•im
plem
ents
(parallel
netw
orkof)
real-time
statem
achin
es–
this
canbe
generated
autom
aticallyfrom
formal
specifi
cations
!
•in
puts
forstate
mach
ines:
abstracted
events
and
states
•ou
tputs
ofstate
mach
ines:
abstracted
control
comm
ands
and
statech
anges
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Exam
ple
:Safe
tyA
rchite
ctu
reFra
mew
ork
SY
ST
EM
INT
ER
FA
CE
LA
YER
•con
tains
hard
ware-sp
ecific
drivers
foreach
extern
alin
terface
•im
plem
ents
interface-sp
ecific
proto
cols
•tran
sforms
concrete
inputs
fromex
ternal
system
sin
toab
stracteven
ts
•tran
sforms
abstract
control
comm
ands
fromSA
FE
TY
LA
YE
Rin
tocon
cretein
terfacedata
forasso
ciatedex
ternal
system
s
•updates
statein
formation
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Exam
ple
:Safe
tyA
rchite
ctu
reFra
mew
ork
USER
INT
ER
FA
CE
LA
YER
•in
terprets
statein
formation
storedin
shared
mem
oryan
ddisp
lays
user
data
•in
terprets
user
inputs,
generates
associated
abstract
events
forSA
FE
TY
LA
YE
Ran
dupdates
statein
formation
storedin
shared
mem
ory
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Exam
ple
:Safe
tyA
rchite
ctu
reFra
mew
ork
SH
AR
ED
MEM
ORY
INT
ER
FA
CES
•prov
ides
persisten
tstorage
ofeven
tlists
and
statein
formation
=⇒
possib
ilityto
restarta
software
layerin
the
prop
ersy
stemstate
aslon
gas
the
operatin
gsy
stemw
orks
correctly
•partition
sstate
space
by
usin
gseveral
separate
shared
mem
orysegm
ents
CO
NFIG
-DA
TA
•con
tains
pro
ject-specifi
ccon
figu
rationdata
=⇒
statem
achin
esof
the
SA
FE
TY
LA
YE
RS
canbe
develop
edas
generic
specifi
cations
and
instan
tiatedw
ithsp
ecific
configu
rationdata
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Exam
ple
:Safe
tyA
rchite
ctu
reFra
mew
ork
SO
FT
WA
RE
WA
TC
HD
OG
•ch
ecks
lifeflags
ofeach
software
layer
•ch
ecks
integrity
constrain
tsof
statein
formation
and
code,
soth
atin
ternal
errorsare
revealedas
soon
aspossib
le
EX
TER
NA
LH
AR
DW
AR
EW
AT
CH
DO
G
•ch
ecks
basic
safetycon
straints
onhard
ware
interface
level
•uses
hard
ware
(and
possib
lysim
ple
software)
which
isin
dep
enden
tof
possib
lycorru
pt
controller
beh
aviou
r
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Exam
ple
:Safe
tyA
rchite
ctu
reFra
mew
ork
AU
DIT
ING
SY
ST
EM
•record
ssafety
-relateduser
interaction
s,even
ts,action
san
din
ternal
statetran
sitions
•usefu
lm
ainly
fordeb
uggin
gfau
ltycon
trollerbeh
aviou
r
EX
TER
NA
LA
UD
ITIN
GSY
ST
EM
•record
ssafety
-relatedI/O
onhard
ware
interface
level
•uses
hard
ware
(and
possib
lysim
ple
software)
which
isin
dep
enden
tof
possib
lycorru
pt
controller
beh
aviou
r–
e.g.
netw
orksn
oop
ing
mech
anism
•m
andatory
forlegally
validpro
ofof
correctcon
trollerbeh
aviou
r
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Overv
iew
1.T
he
Notion
ofD
epen
dab
ility
2.Safety
-Related
Stan
dard
san
dV
-Models
3.M
odellin
gSafety
-Critical
System
s
4.H
azardA
naly
sisan
dR
iskA
ssessmen
t
5.D
esignC
riteriafor
Safety
-Critical
System
s
6.V
alidation
,V
erification
and
Test
ofSafety
-Critical
System
s
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Valid
atio
n,V
erifi
catio
nand
Test
(VV
T)
ofSafe
tyP
ropertie
s
•V
alid
atio
n:
determ
ine
that
the
requirem
ents
areth
erigh
treq
uirem
ents
and
that
they
arecom
plete
•V
erifi
catio
n:
evaluate
develop
men
tpro
ducts
toen
sure
their
consisten
cyw
ithresp
ectto
applicab
lereferen
cedocu
men
ts
•Test:
execu
teim
plem
ented
system
compon
ents
by
prov
idin
gsp
ecific
data
atth
eir(in
put)
interfaces,
while
mon
itoring
the
compon
ent
beh
aviou
r.
Obse
rve
that
the
test
ofliv
eness
pro
pertie
sw
ould
be
im-
possib
le,
since
you
never
know
when
tosto
pw
aitin
gfo
rth
erig
ht
resu
lt!V
ERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Haza
rd-A
naly
sis-Driv
en
Sele
ction
of
VV
T-M
eth
ods
The
fundam
enta
lV
VT
pro
ble
m...
•tim
ean
dbudget
restrictions
do
not
allowfor
exhau
stiveverifi
cation,valid
ationan
dtest
coveragew
ithresp
ectto
every
system
requirem
ent
and
itspra
ctic
also
lutio
n:
•use
conventio
nalaccepta
nce
testin
gto
make
sure
that
requirem
ents
were
reallyim
plem
ented
•use
maxim
um
covera
ge
insp
ectio
n/m
odel
check
ing/te
sting
forth
osereq
uirem
ents
whose
violation
wou
ldlead
toid
entifi
edhazard
s
•in
vesteffort
pro
portio
nalto
hazard
severity
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Haza
rd-A
naly
sis-Driv
en
Sele
ction
of
VV
T-M
eth
ods
Recall:
Syste
mH
azard
Analy
sis
•R
oot
hazard
srefer
toth
ephysical
model
(EU
C)
•H
azardan
alysis
implies
safetyreq
uirem
ents
forth
esafety
controller
Contro
ller
Hazard
Analy
sis
•R
oot
hazard
is“C
ontroller
violates
itssafety
requirem
ents”
•Softw
arem
odules
asleaves
ofth
ehazard
analy
sis(e.g.
fault
treean
alysis)
•Tree
structu
reis
induced
by
the
controller
design
structu
re
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Haza
rd-A
naly
sis-Driv
en
Sele
ction
of
VV
T-M
eth
ods
VV
T-A
ppro
ach
:Sele
ct
VV
Tm
eth
ods
(insp
ectio
ns,
revie
ws,
tests
and
form
alverifi
catio
n)
on
the
basis
ofth
econtro
ller
hazard
analy
sis!
Today
’sm
ostprom
ising
VV
Tm
ethods:
•in
teractive,stru
ctured
insp
ectio
ns
ofreq
uirem
ents,
design
and
code
•form
alverifi
cationof
requirem
ents,
design
and
code
by
modelch
eck
ing
•au
tomated
hard
ware
-in-th
e-lo
op
test
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Haza
rd-A
naly
sis-Driv
en
Sele
ction
of
VV
T-M
eth
odsRe
vie
ws
&
Ins
pe
ctio
ns
Th
rea
t An
aly
sis
Fo
rma
l Ve
rifica
tion
SW
-Te
st
Ha
rdw
are
-in-th
e-
Lo
op
-Te
st
- Re
qu
irem
en
ts
- De
sig
n
- Co
de
- Fa
ult-T
ree
An
aly
sis
- FM
EA
Fa
ilure
Mo
de
s
an
d E
ffects
An
aly
sis
- Co
de
Ab
stra
ctio
n
- Mo
de
l Ch
eckin
g
- Co
de
Ve
rifica
tion
- Mo
du
le T
est
-SW
-Inte
gra
tion
Te
sts
on
Pro
ce
ss/T
hre
ad
Le
ve
l
- Su
b-S
yste
m T
est
on
Co
ntro
ller
Le
ve
l
- Syste
m T
est
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Haza
rd-A
naly
sis-Driv
en
Sele
ction
of
VV
T-M
eth
ods
Exam
ple
:D
eriv
ete
stconfigura
tion
and
test
case
from
fault-tre
edisc
usse
din
Sectio
n4.
Assu
me
that
events
CE
and
E32
aregen
eratedby
the
environ
men
tan
dth
ereforecan
not
be
preven
tedby
the
controller.
Revise
dre
quire
ments:
Safe
tyR
equire
ment
1’:
(not(C
E)
and
E32)
⇒not(E
31and
E22)
Safe
tyR
equire
ment
2’:
(not(C
E)
and
not(E
32))⇒
not(E
31and
E22
and
E33)
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Haza
rd-A
naly
sis-Driv
en
Sele
ction
of
VV
T-M
eth
ods
Exam
ple
–contin
ued.
Rev
isedreq
uirem
ents
leadto
atest
configu
rationw
here
•C
Ean
dE
32can
be
contro
lled
by
the
testsy
stemsim
ulatin
gth
eop
erational
environ
men
tof
the
system
under
test
•req
uirem
ents
1’an
d2’
canbe
check
ed
by
the
testsy
stem
•test
coverageto
be
achieved
canbe
setpro
portio
nalto
severity
ofhazard
E1
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Rem
ark
:W
hen
touse
...
•in
spectio
ns:
logicalch
ecks
ofseq
uen
tialalgorith
ms
localised
inisolated
function
s
•m
odelch
eck
ing:
logicalch
eckof
synch
ronisation
mech
anism
s,distrib
uted
algorithm
san
dcom
munication
proto
colsin
volvin
gseveral
parallel
pro
cesses
•hard
ware
-in-th
e-lo
op
test:
check
ofth
eprop
erin
tegrationof
logicallycorrect
software
onth
etarget
system
hard
ware
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
ModelC
heck
ing
•sp
ecification
SP
EC
isex
pressed
inon
eof
various
formalism
sallow
ing
todescrib
esy
stems
ofparallel
comm
unicatin
g(tim
ed)
statem
achin
es
•correctn
esscon
dition
tobe
checked
isex
pressed
by
logicalform
ula
F
•m
odel
checker
perform
sex
hau
stivestate
analy
sisto
investigate
wheth
erF
hold
sin
everystate
ofS
PE
C
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Com
ponents
ofTest
Auto
matio
nSyste
mfo
rR
eactiv
eRT
-Syste
ms
Sp
ecific
atio
ns
. . . .additio
nal
syste
m
Test M
on
itor
Test D
river
Test G
en
era
tor
Test E
valu
ato
r
inte
rface
monito
ring c
hannels
Syste
m U
nd
er T
est
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Conclu
sion
The
followin
gm
easures
will
consid
erably
improve
the
costeff
ectiveness
and
the
trustw
orthin
essof
the
develop
men
tpro
cessfor
safety-critical
control
system
s:
Develo
pm
ent
pro
cess
rela
ted
measu
res:
•derivation
ofsafety
requirem
ents
fromform
alhazard
analy
sis
•elab
orationan
dsy
stematic
use
ofsafety
-relateddesign
pattern
san
dfram
ework
s
•“d
esignfor
testability
”
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN
Conclu
sion
Quality
assu
rance
rela
ted
measu
res:
•design
ofV
VT
measu
resdriven
by
hazard
analy
sis
•com
bin
ed/com
plem
entary
use
of
–in
teractive,stru
ctured
insp
ections
–form
alverifi
cationby
model
check
ing
–au
tomated
testing
•use
ofap
plication
-dep
enden
tquality
measu
res
Managem
ent
pro
cess
rela
ted
measu
res:
•use
ofsafety
-centered
V-m
odels
•total
quality
man
agemen
tcam
paign
sto
increase
safetyaw
areness
VERIF
IEDS
YST
EM
SIN
TERN
AT
ION
AL
GM
BH
,BREM
EN