upgrade, vol. xii, issue no. 5, december 2011...4 presentation. trends and advances in risk...

3 Editorial. CEPIS UPGRADE: A Proud Farewell— Nello Scarabottolo, President of CEPIS

ATI, Novática and CEPIS UPGRADE— Dídac López-Viñas, President of ATI

4 Presentation. Trends and Advances in Risk Management— Darren Dalcher

10 The Use of Bayes and Causal Modelling in Decision Making,Uncertainty and Risk — Norman Fenton and Martin Neil

22 Event Chain Methodology in Project Management — MichaelTrumper and Lev Virine

34 Revisiting Managing and Modelling of Project Risk Dynamics -A System Dynamics-based Framework — Alexandre Rodrigues

41 Towards a New Perspective: Balancing Risk, Safety and Danger— Darren Dalcher

45 Managing Risk in Projects: What’s New? — David Hillson

48 Our Uncertain Future — David Cleden

55 The application of the ‘New Sciences’ to Risk and ProjectManagement — David Hancock

59 Communicative Project Risk Management in IT Projects— Karel de Bakker

67 Decision-Making: A Dialogue between Project and ProgrammeEnvironments — Manon Deguire

75 Decisions in an Uncertain World: Strategic Project RiskAppraisal — Elaine Harris

82 Selection of Project Alternatives while Considering Risks— Marta Fernández-Diego and Nolberto Munier

87 Project Governance — Ralf Müller

91 Five Steps to Enterprise Risk Management — Val Jonas

Vol. XII, issue No. 5, December 2011

CEPIS UPGRADE is the European Journalfor the Informatics Professional, published bi-monthly at <http://cepis.org/upgrade>

PublisherCEPIS UPGRADE is published by CEPIS (Council of Euro-pean Professional Informatics Societies, <http://www.cepis.org/>), in cooperation with the Spanish CEPIS societyATI (Asociación de Técnicos de Informática, <http://www.ati.es/>) and its journal Novática

CEPIS UPGRADE monographs are published jointly withNovática, that publishes them in Spanish (full version printed;summary, abstracts and some articles online)

CEPIS UPGRADE was created in October 2000 by CEPIS and wasfirst published by Novática and INFORMATIK/INFORMATIQUE,bimonthly journal of SVI/FSI (Swiss Federation of ProfessionalInformatics Societies)

CEPIS UPGRADE is the anchor point for UPENET (UPGRADE Euro-pean NETwork), the network of CEPIS member societies’ publications,that currently includes the following ones:• inforewiew, magazine from the Serbian CEPIS society JISA• Informatica, journal from the Slovenian CEPIS society SDI• Informatik-Spektrum, journal published by Springer Verlag on behalf

of the CEPIS societies GI, Germany, and SI, Switzerland• ITNOW, magazine published by Oxford University Press on behalf of

the British CEPIS society BCS• Mondo Digitale, digital journal from the Italian CEPIS society AICA• Novática, journal from the Spanish CEPIS society ATI• OCG Journal, journal from the Austrian CEPIS society OCG• Pliroforiki, journal from the Cyprus CEPIS society CCS• Tölvumál, journal from the Icelandic CEPIS society ISIP

Editorial TeamEditorial TeamChief Editor: Llorenç Pagés-CasasDeputy Chief Editor: Rafael Fernández CalvoAssociate Editor: Fiona Fanning

Editorial BoardProf. Nello Scarabottolo, CEPIS PresidentProf. Wolffried Stucky, CEPIS Former PresidentProf. Vasile Baltac, CEPIS Former PresidentProf. Luis Fernández-Sanz, ATI (Spain)Llorenç Pagés-Casas, ATI (Spain)François Louis Nicolet, SI (Switzerland)Roberto Carniel, ALSI – Tecnoteca (Italy)

UPENET Advisory BoardDubravka Dukic (inforeview, Serbia)Matjaz Gams (Informatica, Slovenia)Hermann Engesser (Informatik-Spektrum, Germany and Switzerland)Brian Runciman (ITNOW, United Kingdom)Franco Filippazzi (Mondo Digitale, Italy)Llorenç Pagés-Casas (Novática, Spain)Veith Risak (OCG Journal, Austria)Panicos Masouras (Pliroforiki, Cyprus)Thorvardur Kári Ólafsson (Tölvumál, Iceland)Rafael Fernández Calvo (Coordination)

English Language Editors: Mike Andersson, David Cash, ArthurCook, Tracey Darch, Laura Davies, Nick Dunn, Rodney Fennemore,Hilary Green, Roger Harris, Jim Holder, Pat Moody.

Cover page designed by Concha Arias-Pérez"Liberty with Risk" / © ATI 2011Layout Design: François Louis NicoletComposition: Jorge Llácer-Gil de Ramales

Editorial correspondence: Llorenç Pagés-Casas <[email protected]>Advertising correspondence: <[email protected]>SubscriptionsIf you wish to subscribe to CEPIS UPGRADE please send anemail to [email protected] with ‘Subscribe to UPGRADE’ as thesubject of the email or follow the link ‘Subscribe to UPGRADE’at <http://www.cepis.org/upgrade>

Copyright© Novática 2011 (for the monograph)© CEPIS 2011 (for the sections Editorial, UPENET and CEPIS News)All rights reserved under otherwise stated. Abstracting is permittedwith credit to the source. For copying, reprint, or republication per-mission, contact the Editorial Team

The opinions expressed by the authors are their exclusive responsibility

ISSN 1684-5285

MonographRisk Management(published jointly with Novática*)Guest Editor: Darren Dalcher

./..

Farewell Edition

* This monograph will be also published in Spanish (full version printed; summary, abstracts, and somearticles online) by Novática, journal of the Spanish CEPIS society ATI (Asociación de Técnicos deInformática) at <http://www.ati.es/novatica/>.

99 From inforeview (JISA, Serbia)Information SocietySteve Jobs — Dragana Stojkovic

101 From Informatica (SDI, Slovenia)Surveillance SystemsAn Intelligent Indoor Surveillance System — Rok Piltaver,Erik Dovgan, and Matjaz Gams

111 From Informatik Spektrum (GI, Germany, and SI, Switzerland)Knowledge RepresentationWhat’s New in Description Logics — Franz Baader

121 From ITNOW (BCS, United Kingdom)Computer ScienceThe Future of Computer Science in Schools — Brian Runciman

124 From Mondo Digitale (AICA, Italy)IT for HealthNeuroscience and ICT: Current and Future Scenarios— Gianluca Zaffiro and Fabio Babiloni

135 From Novática (ATI, Spain)IT for MusicKatmus: Specific Application to support Assisted MusicTranscription — Orlando García-Feal, Silvana Gómez-Meire,and David Olivieri

145 From Pliroforiki (CCS, Cyprus)IT SecurityPractical IT Security Education with Tele-Lab — ChristianWillems, Orestis Tringides, and Christoph Meinel

153 Selected CEPIS News — Fiona Fanning

Vol. XII, issue No. 5, December 2011

CEPIS NEWS

UPENET (UPGRADE European NETwork)

Farewell Edition

Cont.

CEPIS UPGRADE Vol. XII, No. 5, December 2011 101© CEPIS

UPENET

Farewell Edition

Surveillance Systems

An Intelligent Indoor Surveillance SystemRok Piltaver, Erik Dovgan, and Matjaz Gams

© Informatica, 2011

This paper was first published, in English, by Informatica Informatica Informatica Informatica Informatica (Vol. 35, issue no. 3, 2011, pp. 383-390). InformaticaInformaticaInformaticaInformaticaInformatica, <http://www.informatica.si/> is a quarterly journal published, in English, by the Slovenian CEPIS society SDI (Slovensko DrustvoInformatika – Slovenian Society Informatika, <http://www.drustvo-informatika.si/>).

The development of commercial real-time location system (RTLS) enables new ICT solutions. This paper presents anintelligent surveillance system for indoor high-security environments based on RTLS and artificial intelligence methods.The system consists of several software modules each specialized for detection of specific security risks. The validationshows that the system is capable of detecting a broad range of security risks with high accuracy.

Authors

Rok Piltaver received his B.Sc. degreein Computer Science from the Universityof Ljubljana, Slovenia, in 2008. He is aresearch assistant at the Department ofIntelligent Systems of the Jozef StefanInstitute, Ljubljana, and a Ph.D. studentof New media and e-science at the JozefStefan International PostgraduateSchool where he is working on hisdissertation on combining accurate andunderstandable classifiers. His researchinterests are in artificial intelligence andmachine learning with applications inambient intelligence and ambient assistedliving. He published two papers ininternational scientific journals and eightpapers in international conferences andwas awarded for the best innovation inSlovenia in 2009 and for the best jointproject between business and academiain 2011. <[email protected]>

Erik Dovgan received his B.Sc. degreein Computer Science from the Universityof Ljubljana, Slovenia, in 2008. He is aresearch assistant at the Department ofIntelligent Systems of the Jozef StefanInstitute, Ljubljana, and a Ph.D. studentof New media and e-science at the JozefStefan International Postgraduate Schoolwhere he is working on his dissertationon multiobjective optimization of vehiclecontrol strategies. His research interestsare in evolutionary algorithms, stochasticmultiobjective optimization, classificationalgorithms, clustering and application ofthese techniques in energy efficiency,

Keywords: Expert System, FuzzyLogic, Intelligent System, Real-TimeLocating System, Surveillance.

1 IntroductionSecurity of people, property, and

data is becoming increasingly impor-tant in today’s world. Security is en-sured by physical protection and tech-nology, such as movement detection,biometric sensors, surveillance cam-eras, and smart cards. However, thecrucial factor of most security systemsis still a human [7], providing the in-telligence to the system. The securitypersonnel has to be trustworthy, trainedand motivated, and in good psychicallyand physical shape. Nevertheless, theyare still human and as such tend tomake mistakes, are subjective and bi-ased, get tired, and can be bribed. Forexample, it is well known that a per-son watching live surveillance videooften becomes tired and may thereforeoverlook a security risk. Another prob-lem is finding trustworthy security per-sonnel in foreign countries where lo-cals are the only candidates for the job.

With that in mind there is an op-portunity of using the modern infor-mation-communication technology inconjunction with methods of artificialintelligence to mitigate or even elimi-nate the human shortcomings and in-crease the level of security while low-ering the overall security costs. Our

transportation, security systems andambient assisted living. <[email protected]>

Matjaz Gams is Head of Department ofIntelligent Systems at the Jozef StefanInstitute and professor of computerscience at the University of Ljubljana andMPS, Slovenia. He received his degreesat the University of Ljubljana and MPS.He is or was teaching at 10 Faculties inSlovenia and Germany. His professionalinterest includes intelligent systems, arti-ficial intelligence, cognitive science,intelligent agents, business intelligenceand information society. He is member ofnumerous international programcommittees of scientific meetings,national strategic boards and institutions,editorial boards of 11 journals and ismanaging director of the Informaticajournal. He was co-founder of varioussocieties in Slovenia, e.g. the EngineeringAcademy, AI Society, Cognitive Society,and was president and/or secretary ofvarious societies including ACMSlovenia. He is president of institute andfaculty union members in Slovenia. Heheaded several national and internationalprojects including the major nationalemployment agent on the Internet first topresent over 90% of all available jobs ina country. His major scientificachievement is the discovery of theprinciple of multiple knowledge. In 2009his team was awarded for the bestinnovation in Slovenia and in 2011 for thebest joint project between business andacademia. <[email protected]>

102 CEPIS UPGRADE Vol. XII, No. 5, December 2011 © CEPIS

UPENET

Farewell Edition

The rest of the paper is structuredas follows. Section 2 summarizes therelated work. An overview of softwaremodules and a brief description of usedsensors are given in Section 3. Section4 describes the five PDR modules, in-cluding the Expert System Module andFuzzy Logic Module in more detail.Section 5 presents system verificationwhile Section 6 provides conclusions.

2 Related WorkThere has been a lot of research in

the field of automatic surveillancebased on video recordings. The re-search ranges from extracting low levelfeatures and modelling of the usualoptical flow to methods for optimalcamera positioning and evaluating ofautomatic video surveillance systems[8]. There are many operational imple-mentations of such system increasingthe security in public places (subwaystations, airports, parking lots).

On the other hand, there has notbeen much research in the field of au-tomatic surveillance systems based onreal-time locating systems (RTLS), dueto the novelty of sensory equipment.Nevertheless, there are already somesimple commercial systems with socalled room accuracy RTLS [20] thatenable tracking of objects and basicalarms based on if-then rules [18].Some of them work outdoors usingGPS (e.g., for tracking vehicles [21])while others use radio systems for in-door tracking (e.g., in hospitals andwarehouses). Some systems allowvideo monitoring in combination withRTLS tracking [19].

Our work is novel as it uses severalcomplex artificial intelligence methodsto extract models of the usual behaviourand detect the unusual behaviour basedon an indoor RTLS. In addition, our workalso presents the benefits of combiningvideo and RTLS surveillance.

3 Overview of the PDR SystemThis section presents a short over-

This paper presents an intelligent surveillance systemfor indoor high-security environments

first intelligent security system that isfocused on the entry control is de-scribed in [5]. In this paper we presenta prototype of an intelligent indoor-surveillance system (i.e. it works in thewhole indoor area and not only at theentry control) that automatically de-tects security risks.

The prototype of an intelligent se-curity system, called "Poveljnikovadesna roka" (PDR, eng. commander’sright hand), is specialized for surveil-lance of personnel, data containers, andimportant equipment in indoor high-security areas (e.g., an archive of clas-sified data with several rooms). Thesystem is focused on the internalthreats; nevertheless it also detects ex-ternal security threats. It detects anyunusual behaviour based on user-de-fined rules and automatically extractedmodels of the usual behaviour. The ar-tificial intelligence methods enable thePDR system to model usual and to rec-ognize unusual behaviour. The systemis capable of autonomous learning, rea-soning and adaptation. The PDR sys-tem alarms the supervisor about unu-sual and forbidden activities, enablesan overview of the monitored environ-ment, and offers simple and effectiveanalysis of the past events. Tagging allpersonnel, data containers, and impor-tant equipment is required as it enablesreal-time localization and integrationwith automatic video surveillance. ThePDR system notifies the supervisorwith an alarm of appropriate level andan easily comprehensible explanationin the form of natural language sen-tences, tagged video recordings andgraphical animations. The PDR systemdetects intrusions of unidentified per-sons, forbidden actions of known andunknown persons and unusual activi-ties of tagged entities. The concretescenarios detected by the system in-clude thefts, sabotages, staff negli-gence and insubordination, unauthor-ised entry, unusual employee behav-iour and similar incidents.

“ ”view of the PDR system. The first sub-section presents the sensors and hard-ware used by the system. The secondsubsection introduces software mod-ules. Subsection 3.3 describes RTLSdata pre-processing and primitive rou-tines.

3.1Sensors and other HardwareThe PDR system hardware includes

a real-time locating system (RTLS),several IP video cameras (Figure 1), aprocessing server, network infrastruc-ture, and optionally one or moreworkstations, such as personal comput-ers, handheld devices, and mobilephones with internet access, which areused for alerting the security personnel.

RTLS provides the PDR systemwith information about locations of allpersonnel and important objects (e.g.container with classified documents) inthe monitored area. RTLS consists ofsensors, tags, and a processing unit(Figure 1). The sensors detect the dis-tance and the angle at which the tagsare positioned. The processing unituses these measurements to calculatethe 3D coordinates of the tags. Com-mercially available RTLS use varioustechnologies: infrared, optical, ultra-sound, inertial sensors, Wi-Fi, or ultra-wideband radio. The technology deter-mines RTLS accuracy (1 mm – 10 m),update frequency (0.1 Hz – 120 Hz),covered area (6 – 2500 m2), size andweight of tags and sensors, variouslimitations (e.g., required line of sightbetween sensors and tags), reliability,and price (2.000 – 150.000 •) [13].PDR uses Ubisense RTLS [15] that isbased on the ultra-wide band technol-ogy and is among the more affordableRTLSs. It uses relatively small andenergy efficient active tags, has an up-date rate of up to 9 Hz and accuracy of±20 cm in 3D space given good condi-tions. It covers areas of up to 900 m2

and does not require line of sight.The advantages of a RTLS are that

people feel more comfortable being


UPENET

Farewell Edition

tracked by it than being filmed by videocameras and that localization with aRTLS is simpler, more accurate, andmore robust than localization fromvideo streams. On the other hand,RTLS is not able to locate objects thatare not marked with tags. Therefore,the most vital areas need to be moni-tored by video cameras also in orderto detect intruders that do not wearRTLS tags. However, only one PDRmodule requires video cameras, whilethe other four depend on RTLS alone.Moreover, the cameras enable on-cam-era processing, therefore only extractedfeatures are sent over the network.

3.2 Software StructureThe PDR software is divided into

five modules. Each of them is special-ized for detecting a certain kind of ab-normal behaviour (i.e., a possible se-

curity risk) and uses an appropriateartificial intelligence method for de-tecting it. The modules reason in realtime independently of each other andasynchronically trigger alarms aboutdetected anomalies. Three of the PDRmodules are able to learn automaticallywhile the other two use predefinedknowledge and knowledge entered bythe supervisor. The Video Module de-tects persons without tags and is theonly module that needs video cameras.The Expert System Module iscustomisable by the supervisor, whoenters information about forbiddenevents and actions in the form of sim-ple rules, thus enabling automatic rulechecking. The three learning modulesthat automatically extract models of theusual behaviour for each monitoredentity and compare current behaviourwith it in order to detect abnormalities

Figure 1: Overview of the PDR System.

The system is capable of detecting a broad rangeof security risks with high accuracy“ ”

are Statistic, Macro and Fuzzy LogicModules. The Statistic Module collectsstatistic information about entity move-ment such as time spent walking, sit-ting, lying etc. The Macro model isbased on macroscopic properties suchas the usual time of entry in certainroom, day of the week etc. Both mod-ules analyse relatively long time inter-vals while the Fuzzy Logic Moduleanalyses short intervals. It uses fuzzydiscretization to represent short actionsand fuzzy logic to infer whether theyare usual or not.

3.3 RTLS Data Pre-processingand Primitive Routines

Since the used RTLS has relativelylow accuracy and relatively high up-date rate, a two-stage data filtering isused to increase the reliability and tomitigate the negative effect of the noisylocation measurements. In the firststage, median filter [1] with windowsize 20 is used to filter sequences of x,y, and z coordinates of tags. Equation(1) gives the median filter equation fordirection x. The median filter is usedto correct the RTLS measurements thatdiffer from the true locations by morethan ~1.5 m and occur in up to 2.5 %of measurements. Such false measure-ments are relatively rear and occur onlyin short sequences (e.g., probability ofmore than 5 consecutive measurementshaving a high error is very low) there-fore the median filter corrects theseerrors well.

{ 910 ,,~−−= nnn xxmedx

}98,,..., ++ nn xx (1)

The second stage uses a Kalmanfilter [6] that performs the followingthree tasks: smoothing of the RTLSmeasurements, estimating the veloci-ties of tags, and predicting the missingmeasurements. Kalman filter state is asix dimensional vector that includespositions and velocities in each of thethree dimensions. The new state is cal-


UPENET

Farewell Edition

culated as a sum of the previous posi-tion (e.g. xn) and a product between theprevious velocity (e.g. vx,n) and the timebetween the consecutive measurementsÄt for each direction separately. Thevelocities remain constant. Equation(2) gives the exact vector formula usedto calculate the next state of theKalman filter. The measurement noisecovariance matrix was set based onRTLS system specification, while theprocess noise covariance matrix wasfine-tuned experimentally.

Once the measurements are fil-tered, primitive routines can be ap-plied. They are a set of basic pre-processing methods used by all thePDR modules and are robust to noisein 3D location measurements. Theytake short intervals of RTLS data asinput and output a symbolic represen-tation of the processed RTLS data.

⎥⎥⎥⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢⎢⎢⎢

⎣

⎡

⎥⎥⎥⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢⎢⎢⎢

⎣

⎡

∆∆

∆

=

⎥⎥⎥⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢⎢⎢⎢

⎣

⎡

+

+

+

+

+

+

nz

ny

nx

n

n

n

t

t

t

nz

ny

nx

n

n

n

vvvzyx

vvvzyx

,

,

,

1,

1,

1,

1

1

1

100000010000001000

001000001000001

(2)

The first primitive routine detectsin which area (e.g., a room or a user-defined area) a given tag is located,when it has entered, and when it hasexited from the area. The routine takesinto account the positions of walls anddoors. A special method is used to han-dle the situations when a tag movesalong the boundary between two areasthat are not separated by a wall.

The second primitive routine clas-sifies the posture of a person wearinga tag into: standing, sitting, or lying. Aparameterized classifier, trained on

pre-recorded and hand-labelled train-ing data, is used to classify the se-quences of tag heights into the threepostures. The algorithm has three pa-rameters: the first two are thresholdstlo and thi dividing the height of a taginto the three states, while the third pa-rameter is tolerance d. The algorithmstores the previous posture and adjuststhe boundaries between the posturesaccording to it (Figure 2). If the cur-rent state is below the threshold ti, it isincreased by d, otherwise it is de-creased by d. The new posture is set tothe posture that occurs most often inthe window of consecutive tag heightsaccording to the dynamically setthresholds. The thresholds tlo and thi wereobtained from the classification tree thatclassifies the posture of a person basedon the height of a tag. It was trained onhalf an hour long manually labelled re-

Figure 2: Dynamic Thresholds.

The crucial factor of most security systems is stilla human providing the intelligence to the system“ ”

cording of lying, sitting and standing.The third group of primitive rou-

tines is a set of routines that detectwhether a tag is moving or not. This isnot a trivial task due to the consider-able amount of noise in the 3D loca-tion data. There are separate routinesfor detecting movement of persons,movable objects (e.g., a laptop) andobjects that are considered stationary.The routines include hardcoded,handcrafted, common sense algorithmsand a classifier trained on extensive,pre-recorded, hand labelled trainingset. The classifier uses the followingattributes calculated in a sliding win-dow with size 20: the average speed,the approximate distance travelled,sum of consecutive position distances,and the standard deviation of movingdirection. The classifier was trained onmore than two hours long hand-la-belled recording of consecutive mov-ing and standing still. Despite the noisein the RTLS measurements the classi-fication accuracy of 95 % per singleclassification was achieved. [12] de-scribes the classifier in more detail.

The final group of routines detectsif two tags (or a tag and a given 3Dposition) are close together by compar-ing the short sequences of tags’ posi-tions. There are separate methods usedfor detecting distances between twopersons (e.g., used to detect if a visitoris too far away from its host), between


UPENET

Farewell Edition

a person and an object, and between aperson and a given 3D location (e.g.,used to assign tags of moving personsto locations of moving objects detectedby video processing).

All of the described primitive rou-tines are robust to the noise in RTLSmeasurements and are specialized forthe PDR’s RTLS. Primitive routines’parameters were tuned according to thenoise of the RTLS and using data min-ing tools Orange [4] and Weka [16].In case of more accurate RTLS, theprimitive routines could be simpler andmore accurate. Nevertheless, the pre-sented primitive routines perform welldespite the considerable amount ofnoise. This is possible because of therelatively high update rate. If it was sig-nificantly lower, the primitive routineswould not work as well. Therefore, theaccuracy, reliability and update rate ofRTLS are crucial for the performanceof the entire PDR system.

4 PDR Modules4.1Expert System ModuleThe Expert System Module enables

the supervisor to customize the PDRsystem according to his/her needs bysetting simple rules that must not beviolated. It is the simplest and the mostreliable module of the PDR system[11]. It is capable of detecting a vastmajority of the predictable securityrisks, enables simple customization, isreliable, robust to noise, raises almostno false alarms, and offers comprehen-sible explanation for the raised alarms.In addition, it does not suffer from thetypical problems common to the learn-ing modules/algorithms, such as longlearning curve, difficulty to learn fromunlabeled data, relatively high prob-ability of false alarms, and the elusivebalance between false negative andfalse positive classifications. The ex-pert system consists of three parts de-scribed in the following subsections.

4.1.1 Knowledge BaseKnowledge base of an expert sys-

tem contains the currently availableknowledge about the state of the world.The knowledge base of PDR expertsystem consists of RTLS data,predefined rules, and user-definedrules. The first type of knowledge is inform of data stream, while the lattertwo are in form of if-then rules.

The expert system gets the knowl-edge about objects’ positions from theRTLS data stream. Each unit of the datastream is a filtered RTLS measurementthat contains a 3D location with a timestamp and a RTLS tag ID.

User-defined rules enable simplecustomization of the expert system ac-cording to specific supervisor’s needsby specifying prohibited and obliga-tory behaviour. Supervisor can add,edit, view, and delete the rules at anytime using an intuitive graphic userinterface. There are several rule tem-plates available. The supervisor has tospecify only the missing parameters ofthe rules, such as for which entities(tags), in which room(s) or user-de-fined areas(s), and at which time therules apply.

For instance, a supervisor canchoose to add a rule based on the fol-lowing template: "Person P must be inthe room R from time Tmin to time Tmax."and set P to John Smith, R to the hall-way H, Tmin to 7 am, and Tmax to 11 am.Now the expert system knows that Johnmust be in the hallway from 7 am to11 am. If he leaves the hallway duringthat period or if he does not enter itbefore 7 am, the PDR supervisor willbe notified.

Some of the most often used ruletemplates are listed below:

Object Oi is not allowed to en-ter area Ai.

Object Oi can only be moved byobject Oj.

Object Oi must always be closeto object Oj.

The PDR system detects intrusions of unidentified persons,forbidden actions of known and unknown persons

and unusual activities of tagged entities

“”

The predefined rules are a set ofrules that are valid in any applicationwhere PDR might be used. Neverthe-less, the supervisor has an option toturn them on or off. Predefined rulesdefine when alarms about hardwarefailures should be triggered.

4.1.2 Inference EngineThe inference engine is the part of

the PDR expert system that deducesconclusions about security risks fromthe knowledge stored in the knowledgebase. The inference process is done inreal-time. First, the RTLS data streamis processed using the primitive rou-tines. Second, all the rules related to agiven object (e.g., a person) arechecked. If a rule fires, an alarm israised and an explanation for the raisedalarm is generated. An example is pre-sented in the next paragraph.

Suppose that the most recent 3Dlocation of John Smith’s tag (from theprevious example) has just been re-ceived at 8:32 am. The inference en-gine checks all the rules concerningJohn Smith. Among them is the rule Rithat says: "John Smith must be in thehallway H from 7 am to 11 am." Theinference engine calls the primitiveroutine that checks whether John is inthe hallway H. There are two possibleoutcomes. In the first outcome, he isin the hallway H, therefore, the rule Riis not violated. If John was not in thehallway H in the previous instant, thereis an ongoing alarm that is now endedby the inference engine. In the secondoutcome, John is not in the hallway H;hence the rule Ri is violated at this mo-ment. In this case the inference enginechecks if there is an ongoing alarmabout John not being in the hallway H.If there is no such ongoing alarm theinference engine triggers a new alarm.On the other hand, if there is such analarm, the inference engine knows thatthe PDR supervisor was already noti-fied about it.


UPENET

Farewell Edition

If an alarm was raised every time arule was violated, the supervisorswould get flooded with alarm mes-sages. Therefore, the inference engineautomatically decreases the number ofalarm messages and groups alarm mes-sages about the same incident togetherso that they are easier to handle by thePDR supervisor. The method will beillustrated with an example. Becauseof the noise in 3D location measure-ments the inference engine does nottrigger or end an alarm immediatelyafter the status of rule Ri (violated/notviolated) changes. Instead it waits formore RTLS measurements and checksthe trend in the given time window: ifthere are only few instances when therule was violated they are consideredas noise. On the other hand, if thereare many (over the global threshold setby the supervisor) such instances, thenthe instances when rule was not vio-lated are treated as noise. Two consecu-tive alarms that are interrupted by ashort period of time will therefore re-sult in a single alarm message. A short

period in which a rule seems to be vio-lated because of the noise in RTLSdata, however, will not trigger analarm. The grouping of alarms worksin the following way: the inferenceengine groups the alarm messagesbased on the two rules Ri and Rj to-gether if at the time when rule Ri is vio-lated another rule Rj concerning JohnSmith or hallway H is violated too. Asa result, the supervisor has to deal withfewer alarm messages.

4.1.3 Generating Alarm ExplanationsThe Expert System Module also

provides the supervisor with an expla-nation of the alarm. It consists of threeparts: explanation in natural language,graphical explanation, and video re-cording of the event.

Each alarm is a result of a particu-lar rule violation. Since each rule is aninstance of a certain rule template, ex-planations are partially prepared inadvance. Each rule template has anassigned pattern in the form of a sen-tence in natural language with some

Figure 3: Video Explanation of an Alarm.

Our work is novel as it uses several complex artificialintelligence methods to extract models of the usual behaviourand detect the unusual behaviour based on an indoor RTLS

“”

objects and subjects missing. In orderto generate the full explanation, theinference engine fills in the missingparts of the sentence with details aboutthe objects (e.g., person names, areas,times, etc.) related to the alarm.

Graphical explanation is given inform of a ground plan animation andcan be played upon supervisors’ re-quest. The inference engine determinesthe start and the end times of an alarmand sets the animation to begin slightlybefore the alarm was caused and to endslightly after the causes for the alarmare no longer present. The animationis generated from the recorded RTLSdata and the ground plan of the build-ing under surveillance. The animatedobjects (e.g., persons, objects, areas)that are relevant to the alarm are high-lighted with red colour.

If a video recording of the incidentthat caused an alarm is available it isadded to the alarm explanation. Basedon the location of the person thatcaused the alarm, the person in thevideo recording is marked with abounding rectangle (Figure 3). Thevideo explanation is especially impor-tant if an alarm is caused by a personor object without a tag.

The natural language explanation,ground plan animation, and video re-cordings with embedded bounding rec-tangles produced by the PDR expertsystem efficiently indicate when andto which events the security personnelshould pay attention.

4.2 Video ModuleThe video Module periodically

checks if the movement detected by thevideo cameras is caused by peoplemarked with tags. If it detects move-ment in an area where no authorisedhumans are located, it triggers analarm. It combines the data about taglocations and visible movement to rea-son about unauthorised entry.


UPENET

Farewell Edition

Data about visible moving objects(with or without tags) is available asthe output of video pre-processing.Moving objects are described withtheir 3D locations in the same coordi-nate system as RTLS data, sizes of theirbounding boxes, similarity of the mov-ing object with a human, and a timestamp. The detailed description of thealgorithm that processes the video data(developed at the Faculty of ElectricalEngineering, University of Ljubljana,Slovenia) can be found in [9] and [10].

The Video Module determines thepairing between the locations of taggedpersonnel and the detected movementlocations. If it determines that there ismovement in a location that is farenough from all the tagged personnel,it raises an alarm. In this case the mod-ule reports moving of an unauthorisedperson or an unknown object (e.g., arobot) based on the similarity betweenthe moving object and a person. Theprobability of false alarms can be re-duced if several cameras are used tomonitor the area from various angles.It also enables more accurate localiza-tion of moving objects.

Whenever the Video Module trig-gers an alarm it also offers an explana-tion for it in form of video recordings

with embedded bounding boxes high-lighting the critical areas (Figure 3).The supervisor of the PDR system canquickly determine whether the alarmis true or false by checking the sup-plied video recording.

The video pre-processing algorithmis also capable of detecting if a certaincamera is blocked (e.g. covered with apiece of fabric). Such information isforwarded to the Video Module thattriggers an alarm.

4.3Fuzzy Logic ModuleThe Fuzzy Logic Module is based

on the following presumption: frequentbehaviour is usual and therefore unin-teresting while rare behaviour is inter-esting as it is highly possible that it isunwanted or at least unusual. There-fore the module counts the number ofactions done by the object under sur-veillance and reasons about oddity ofthe observed behaviour based on thecounters. If it detects a high number ofodd events (i.e., events that rarely tookplace in the past) in a short period oftime, it triggers an alarm.

The knowledge of the module isstored in two four- dimensional arraysof counters for each object under sur-veillance (implemented as red-black

frequency of events

oddi

ty o

f eve

nts

flow fhi

> 20 %

< 80 %

low

er cu

mul

ativ

epr

obab

ility

of e

vent

Figure 4: Calculating the Oddity of Events.

The advantages of a RTLS are that people feel morecomfortable being tracked by it than being filmed by video cameras“ ”

trees [2]). Events are split into two cat-egories, hence the two arrays: eventscaused by movement and stationaryevents. A moving event is character-ised by its location, direction, and thespeed of movement. A stationary event,on the other hand, is characterised bylocation, duration and posture (lying,sitting, or standing). When an event ischaracterised, fuzzy discretization [17]is used, hence the name of the module.The location of an event in the floorplane is determined using the RTLSsystem and discretized in classes withsize 50 cm, therefore the module con-siders the area under surveillance as agrid of 50 by 50 cm squares. The speedof movement is estimated by theKalman filter. It is used to calculate thedirection which is discretized in the 8classes (N, NE, E, SE, S, SW, W, andNW). The scalar velocity is discretizedin the following four classes: veryslow, slow, normal, and fast. The pos-ture is determined by a primitive rou-tine (see Section 3.3). The duration ofan event is discretized in the follow-ing classes: 1, 2, 4, 8, 15, 30, seconds,minutes or hours.

The fuzzy discretization has fourmajor advantages. The first is a smalleramount of memory needed to store thecounters, as there is only one counterfor a whole group of similar events.Note that the accuracy of the storedknowledge is not significantly de-creased because the discrete classes arerelatively small. The second advantageis the time complexity of counting theevents that are similar to a given event,which is constant instead of being de-pendent on the number of events seenin the past. The third advantage is thelinear interpolation implicitly intro-duced by fuzzy discretization, whichenables a more accurate estimation ofthe rare events’ frequencies. The fourthadvantage is the low time complexityof updating the counters’ values com-pared to the time complexity of add-ing a new counter with value 1 for each


UPENET

Farewell Edition

new event.The oddity of the observed behav-

iour is calculated using a sliding win-dow over which the average oddity ofevents is calculated. Averaging theoddity over time intervals prevents thefalse alarms that would be triggered ifthe oddity of single events was usedwhenever RTLS data noise or shortsequences of uncommon events wouldoccur. The oddity of a single event iscalculated by comparing the frequencyof events similar to the given eventwith the frequencies of the otherevents. For this purpose the supervi-sor sets the two relative frequencies flowand fhi. The threshold flow determines theshare of the rarest events that aretreated as completely unusual andtherefore they get assigned the maxi-mum level of oddity. On the other hand,fhi determines the share of the most fre-quent events that are treated as com-pletely usual and therefore they getassigned 0 as the level of oddity. Theoddity of an event whose frequency isbetween the thresholds flow and fhi is lin-early decreasing with the increasingshare of the events that are rarer thanthe given event (Figure 4).

The drawback of the describedmethod is a relatively long learningperiod which is needed before the mod-ule starts to perform well. On the otherhand, the module discards the outdatedknowledge and emphasizes the newdata, which enables adapting to thegradual changes in observed person’sbehaviour. The module is also highlyresponsive: it takes only about 3 sec-onds to detect the unusual behaviour.The module autonomously learns themodel of usual behaviour which ena-bles the detection of the unusual be-haviour. It can detect events such asan unconscious person lying on thefloor, running in a room where peopleusually do not run, a person sitting atthe table at which he usually does notsit etc. The module also triggers analarm when a long sequence of events

happens for the first time. If such falsealarm is triggered, the supervisor canmark it as false. Consequently, themodule will increase the appropriatecounters and will not raise an alarm forthat kind of behaviour in the future.

When the Fuzzy Logic Moduletriggers an alarm, it also provides agraphical explanation for it. It draws atarget-like graph in each square of themesh dividing the observed area. Thecolour of a sector of the target repre-sents the frequency of a given groupof similar events. The concentric cir-cles represent the speed of movement,e.g., a small radius represents a lowspeed. The triangles, on the other hand,represent the direction of movement.The location of a target on the meshrepresents the location in the physicalarea. White colour depicts the lowestfrequency, black colour depicts thehighest frequency while the shades ofgrey depict the frequencies in between.The events that caused an alarm arehighlighted with a scale ranging fromgreen to red. For stationary events, ta-bles are used instead of the targets. Therow of the table represents the posturewhile the column represents the dura-tion. A supervisor can read the graphi-cal explanations quickly and effec-tively. The visualization is also used forthe general analysis of the behaviourin the observed area.

4.4 Macro and Statistic ModulesMacro and Statistic modules ana-

lyse persons’ behaviour and triggeralarms if it significantly deviates fromthe usual behaviour. In order to do that,several statistics about the movementof each tagged person are collected,calculated, and averaged over varioustime periods. Afterwards, these statis-tics are compared to the previouslystored statistics of the same person andthe deviation factor is calculated. If itexceeds the predefined bound, themodules trigger an alarm.

The Statistic Module collects data

The PDR software is divided into five modules:Video, Expert System, Statistic, Macro and Fuzzy Logic“ ”

over time periods from one minute toseveral hours regardless of person’slocation or context. On the other hand,the Macro Module collects data regard-ing behaviour in certain areas (e.g.room), i.e. the behaviour collectionstarts when a person enters the area andends when he/she leaves it.

Both modules use behaviour at-tributes such as: the percentage of thetime the person spent lying, sitting,standing, or walking during the ob-served time period, the average walk-ing speed. Additionally, Macro mod-ule uses the following attributes: areaid, day of the week, length of stay, en-trance time, and exit time.

The behaviours are classified withthe LOF algorithm [3], a density-basedkNN algorithm, which calculates thelocal outlier factor of the tested in-stance with respect to the learning in-stances. The LOF algorithm was cho-sen based on the study [14]. Bias to-wards false positives or false negativescan be adjusted by setting the alarmthreshold.

The modules show a graphical ex-planation for each alarm in form ofparallel coordinates plot. Each attributeis represented with one of the parallelvertical axes, while statistics aboutgiven time periods are represented bya zigzag line connecting values of eachattribute from the leftmost to therightmost one. Past behaviour is rep-resented with green zigzag lines, whilethe zigzag line portending to the be-haviour that triggered the alarm is col-lared red. The visualisation offers aquick and simple way of establishingthe cause of alarm and often indicatesmore specific reason for it.

5 VerificationDue to the complexity of the PDR

system and the diverse tasks that it per-forms it is difficult to verify its qualitywith a single test or to summarize it ina single number such as true positiverate. Therefore, validation was done on


UPENET

Farewell Edition

more subjective and qualitative levelwith several scenarios for each of theindividual modules. Four demonstra-tion videos of the PDR tests are avail-able at http://www.youtube.com/user/ijsdis. A single test case or a scenariois a sequence of actions and events in-cluding a security risk that should bedetected by the system. "A person en-ters a room without the permission" isan example of scenario. Each scenariohas a complement pair: a similar se-quence of actions which, on the con-trary, must not trigger an alarm. "Aperson with permission enters theroom" is the complement scenario forthe above example. The scenarios andtheir complements were carefully de-fined in cooperation and under super-vision of security experts from theSlovenian Ministry of Defence .

The Expert System Module wastested with two to three scenarios perexpert rule template. Each scenario wasperformed ten times with various per-sons and objects. The module has per-fect accuracy (no false positives andno false negatives) in cases when theRTLS noise was within the normal lim-its. When the noise was extremelylarge, the system occasionally triggeredfalse alarms or overlooked securityrisks. However, in those cases evenhuman experts were not able to tell ifthe observed behaviour should trigger

an alarm or not based on the noisyRTLS measurements alone. Further-more, the extreme RTLS noise oc-curred in less than 2 % of the scenariorepetitions and the system made an er-ror in less than 50 % of those cases.

The Video Module was tested us-ing the following three scenarios: "aperson enters the area under surveil-lance without the RTLS tag", "a robotis moving without authorised person’spresence", and "a security camera isintentionally obscured". Scenarioswere repeated ten times with differentpeople as actors. The module detectedthe security risks in all of the scenariorepetitions with movement and distin-guished between a human and a robotperfectly. It failed to detect the ob-scured camera in one out of 10 repeti-tions. The module also did not triggerany false alarms.

The Fuzzy Logic Module wastested with several scenarios while thefuzzy knowledge was gathered overtwo weeks. The module successfullydetected a person lying on the floor,sitting on colleagues chair for a while,running in a room, walking on a table,crawling under a table, squeezing be-hind a wardrobe, standing on the samespot for extended period of time, andsimilar unusual events. However, theexperts’ opinion was that some of thealarms should not have been triggered.

Table 1: Evaluation of PDR System.

Module TP TN FP FN N Expert Sys. 197 199 2 2 400

Video 30 30 0 0 60 Fuzzy Logic 47 42 8 3 100

Macro 9 10 1 0 20 Statistic 9 10 1 0 20

Total 292 291 12 5 600 Percentage

(%) 48.7 48.5 2 0.8

The system is customizable and can be used in a range ofsecurity applications such as confidential data archives and banks“

”

Indeed we expect that in more exten-sive tests the modules supervised learn-ing capabilities would prevent furtherrepetitions of unnecessary alarms.

The test of the Macro and StatisticModules included the simulation of ausual day at work condensed into onehour. The statistic time periods were 2minutes long. Since the modules re-quire a collection of persons’ past be-haviour, two usual days of work wererecorded by a person constituting oftwo hours of past behaviour data. Af-terwards, the following activities wereperformed 10 times by the same per-son and classified: performing a nor-mal day of work, stealing a containerwith classified data, acting agitated asunder the effect of drugs and running.The classification accuracy was 90 %.This was due to the low amount of pastbehaviour data. Therefore, the modulesdid not learn the usual behaviour of thetest person but only a condensed (simu-lated) behaviour in a limited learningtime. We expect that the classificationaccuracy would be even higher, if thelearning time was extended and if theperson would act as usual instead ofsimulating the condensed day of work.

The overall system performancewas tested on a single scenario: "steal-ing a container with classified docu-ments". In the test five persons tried tosteal the container from a cabinet in asmall room under surveillance. Eachperson tried to steal the container fivetimes with and without a tag. All theattempts were successfully detected bythe system that reported the alarm andprovided an explanation for it.

The validation test data is summa-rized in Table 1. It gives the number oftrue positive (TP), true negative (TN),false positive (FP), and false negativealarms (FN), and total number (N) ofscenario repetitions. Each row givesthe results for one of the five modules.The bottom two rows give the total sumfor each column and the relative per-centage.


UPENET

Farewell Edition

The system received the award forthe best innovation among researchgroups in Slovenia for 2009 at the FourthSlovenian Forum of Innovations.

6 ConclusionThis paper presents an intelligent

surveillance system utilizing a real-time location system (RTLS), videocameras, and artificial intelligencemethods. It is designed for surveillanceof high security indoor environmentsand is focused on internal securitythreats. The data about movement ofpersonnel and important equipment isgathered by RTLS and video cameras.After basic pre-processing with filtersand primitive routines the data is sentto the five independent software mod-ules. Each of them is specialized fordetecting specific security risk. TheExpert System Module detects suspi-cious situations that can be describedby location of a person or other taggedobjects in space and time. It detectsmany different scenarios with high ac-curacy. The Video Module automati-cally detects movement of persons andobjects without tags, which is not al-lowed inside the surveillance area.Fuzzy Logic, Macro, and StatisticsModules automatically extract theusual movement patterns of personneland equipment and detect deviationsfrom the usual behaviour. Fuzzy Logicis focused on short-term anomalous be-haviour such as entering an area for thefirst time, lying on the ground or walk-ing on the table. Macro and StatisticModules, on the other hand, are focusedon mid- and long-term behaviour suchas deviations in daily work routine.

The validation of the system showsthat it is able to detect all the securityscenarios it was designed for and thatit does not raise too many false alarmseven in more challenging situations. Inaddition, the system is customizableand can be used in a range of securityapplications such as confidential dataarchives and banks.

AcknowledgementResearch presented in this paper was

financed by the Republic of Slovenia, Min-istry of Defence. We would like to thankthe colleges from the Machine Vision

Laboratory, Faculty of Electrical Engineer-ing, University of Ljubljana, Slovenia andSpica International, d.o.o. for fruitful co-operation on the project. Thanks also toBoštjan Kaluza, Mitja Lustrek, and BogdanPogorelc for help regarding the RTLS anddiscussions, and Anze Rode for discussionsabout security systems, expert system rulestemplates and specification of scenarios.

References[1] G. R. Arce. "Nonlinear Signal

Processing: A Statistical Approach",Wiley: New Jersey, USA, 2005.

[2] R. Bayer. "Symmetric Binary B-Trees: Data Structures and Mainte-nance Algorithms", Acta Informá-tica, 1, pp. 290–306, 1972.

[3] M. M. Breunig, H. P. Kriegel, R. T.Ng, J. Sander. "LOF: Identifyingdensitybased local outliers," Pro-ceedings of the International Con-ference on Management of Data –SIGMOD ’00, pp. 93–104, Dallas,Texas, 2000.

[4] J. Demsar, B. Zupan, G. Leban. "Or-ange: From Experimental MachineLearning to Interactive Data Min-ing," White Paper (www. ailab.si/orange), Faculty of computer andinformation science, University ofLjubljana, Slovenia, 2004.

[5] M. Gams, T. Tusar. (2007), "Intel-ligent High-Security Access Con-trol", Informatica, vol 31(4), pp.469-477.

[6] R.E. Kalman. "A new approach tolinear filtering and prediction prob-lems". Journal of Basic Engineer-ing, 82 (1), pp. 35–45, 1960.

[7] M. Kolbe, M. Gams. "Towards anintelligent biometric system for ac-cess control," Proceedings of the9th International MulticonferenceInformation Society - IS 2006,Ljubljana, Slovenia, 2006, pp. 118-122.

[8] B. Krausz, R. Herpers. ‘Event de-tection for video surveillance usingan expert system’, Proceedings ofthe 1st ACM Workshop on Analy-sis and Retrieval of Events/Actionsand Workflows in Video Streams -AREA 2008, Vancouver, Canada,pp. 49-56.

[9] M. Kristan, J. Pers, M. Perse, S.Kovaèiè. "Closed-world tracking of

multiple interacting targets for in-door-sports applications", Compu-ter Vision and Image Understand-ing, vol 113, 5, pp. 598-611, 2009.

[10] M. Perše, M. Kristan, S. Kovaèiè,G. Vuèkoviæ, J. Pers. "A trajectory-based analysis of coordinated teamactivity in a basketball game", Com-puter Vision and Image Under-standing, vol 113, 5, pp. 612-621,2009.

[11] R. Piltaver, G. Matjas. "Expert sys-tem as a part of intelligent surveil-lance system", Proceedings of the18th International Electrotechnicaland Computer Science Conference- ERK 2009, vol. B, pp. 191–194,2009.

[12] R. Piltaver. "Strojno uèenje prinaèrtovanju algoritmov zarazpoznavanje tipov gibanja", Pro-ceedings of the 11th InternationalMulticonference Information Soci-ety - IS 2008, str. 13–17, 2008.

[13] V. Schwarz, A. Huber, M. Tüchler."Accuracy of a Commercial UWB3D Location Tracking System andits Impact on LT Application Sce-narios," Proceedings of the IEEEInternational Conference on Ultra-Wideband, Zürich, Switzerland,2005.

[14] T. Tusar, M. Gams. "Odkrivanjeizjem na primeru inteligentnegasistema za kontrolo pristopa," Pro-ceedings of the 9th InternationalMulticonference Information Soci-ety - IS 2006, Ljubljana, Slovenia,2006, pp. 136-139.

[15] Ubisense: awailable at: http://www.ubisense.net/

[16] H. Witten, E. Frank. Data Mining."Practical Machine Learning Toolsand Techniques" (2nd edition),Morgan Kaufmann, 2005.

[17] L. A. Zadeh. "Fuzzy sets", Informa-tion and Control 8 (3), pp. 338–353,1965.

[18] http://www.pervcomconsulting.com/secure.html

[19] http://www.visonictech.com/Ac-tive-RFID-RTLS-Tracking-and-Mangement-Software-Eiris.html

[20] http://www.aeroscout.com/content/healthcare

[21] http://www.telargo.com/solutions/track_trace.asp

upgrade, vol. xii, issue no. 5, december 2011...4 presentation. trends and advances in risk...

Documents