Attack Resilient State Estimation for Autonomous Robotic Systems

Nicola Bezzo, James Weimer, Miroslav Pajic, Oleg Sokolsky, George J. Pappas, Insup Lee

Abstract— In this paper we present a methodology to controlground robots under malicious attack on sensors. Within theterm attack we intend any malicious disturbance injection onsensors, actuators, and controller that would compromise thesafety of a robot. In order to guarantee resilience againstattacks, we use a control-level technique implemented within arecursive algorithm that takes advantage of redundancy in theinformation received by the controller. We use the case studyof a vehicle cruise-control, however, the strategy we present inthis work is general for several applications. Our methodologyrelays on redundancy in the sensor measurements: specificallywe consider N velocity measurements and use a recursivefiltering technique that estimates the state of the system whilebeing resilient against sensor attacks by acting on the varianceof the measurements noise. Finally, we move our focus onhardware validation demonstrating our algorithm throughextensive outdoor experiments conducted on two unmannedground robots.

I. INTRODUCTION

Modern vehicular and robotic systems are equipped withseveral sensors and Electronic Control Units (ECUs) thatinteract with each other over a complex network. Thisavailability of technology and especially networking has ledto an overall higher comfort of driving, an increase of thesafety of the driver and passengers, and the introduction ofnew services such as remote diagnosis and vehicle-to-vehiclecommunication. However, this increase in functionality andcommunication may introduce security vulnerability andcompromise the integrity of the system. For instance anattacker who is able to spoof the GPS could mislead thevehicle to unsafe regions [1]; similarly if a vehicle is incruise control and an attacker compromises the speedometerreading, the vehicle speed could change drastically inducingan higher probability of collisions and accidents. These risksincrease even more with the new generation of unmannedground vehicles (UGVs) and self driving cars.

To address these issues, we have introduced a designframework for development of high-confidence vehicularcontrol systems that can be used in adversarial environments[2]. The framework employs system design techniques thatguarantee that the vehicle will maintain control, possibly ata reduced efficiency, under several classes of attacks. In thispaper we focus primarily on estimation design schemes andaddress attacks on sensors for autonomous ground vehicles(Fig. 1). We utilize a security-aware attack-resilient estimatorthat identifies an attack and allows the controller to pursuea mitigation strategy.

The contribution of this paper is threefold: i) we developan estimator that is easy to implement by using a recursiveapproach, ii) we compare it with other techniques, and iii) werun extensive hardware evaluations to validate the proposed

N. Bezzo, James Weimer, Miroslav Pajic, Oleg Sokolsky, GeorgeJ. Pappas, and Insup Lee are with the PRECISE Center, Departmentof Computer and Information Science, University of Pennsylvania,Philadelphia, PA 19104, USA {nicbezzo, weimerj,pajic, pappasg}@seas.upenn.edu {sokolsky,lee}@cis.upenn.edu

technique. Specifically our framework is inspired by theLinear Quadratic Estimator in which together with the updateand predict steps we add a shield procedure to cancel theeffects due to possible attacks on sensors. Our techniqueadds an extra weighted variance to the measurement errorwhenever there is a mismatch between the updated state andthe measurement from each of the sensors. By using thistechnique all sensors are always considered, however the onethat contains a corrupted measurement will have a large errorvariance and thus will count less when estimating the desiredstate.

Fig. 1. The LandShark robot [3]: one of the platforms used to studymalicious attacks on sensors.

A. Related WorkThe study of high assurance vehicular systems is a recent

topic that is attracting several researchers in both the controland computer science communities. Malicious attacks aredefined as adversarial actions conducted against a systemor part of it and with the intent of compromising theperformance, operability, integrity, and safety of the system.The main difference between failure and malicious attack isthat the former is usually not coordinated while an attack isusually camouflaged or stealthy and behaves and producesresults similar or expectable by the dynamics of the systems.Attack vectors can be classified in the following groups: i)attacks on the vehicle’s sensors and actuators; ii) attacks onRF communication and over the local network (i.e., sharedbus); iii) attacks on the system’s maintenance mechanismsand physical interfaces. The focus of this paper is on control-level defenses (i.e., the first group of attacks). These attacksinclude attacks on sensors measurements transmitted overa common bus (network) to other system components, andinjection of malformed data from corrupted system compo-nents with access to the bus or a faulty sensor. The ability toattack sensors in vehicles was demonstrated in [1] in which aGPS was spoofed misguiding a yacht off route. Similarly in[4], the authors presented the steps and equipment necessaryto spoof a GPS. A more general assessment on car securitywas recently performed by authors in [5] in which under

2014 IEEE/RSJ International Conference onIntelligent Robots and Systems (IROS 2014)September 14-18, 2014, Chicago, IL, USA

978-1-4799-6934-0/14/$31.00 ©2014 IEEE 3692

some circumstances it was demonstrated how to attacksteering, braking, acceleration, and display on two vehicles.

Even though this area of study is still at an early stage,some preliminary work on vehicular security was performedin [6] in which the authors showed through intensive ex-periments on common cars, that an attacker could takeover a vehicle and compromise its safety. Specifically itwas shown that the CAN bus system is unprotected andseveral functionality of a car can be controller and accessedby different devices in the vehicle. The main attacks onsensors in ground vehicles generally involve the speedometerand GPS. In our work we will present experimental resultsdealing with attacks on these sensors.

Standing from a control perspective, authors in [7] proposea resilient consensus algorithm based on receding-horizoncontrol to deal with replay attacks between an operator anda remotely controlled unmanned ground vehicle. In [8] theauthors use plant models for attack detection and monitor incyber-physical systems. In [9] the authors consider wirelesssensor networks and the problem of attacks on state estima-tion performed by a Kalman Filter. An ellipsoidal algorithmis proposed to estimate the resilience of the system againstsuch attacks. Our work in this paper is motivated by theprevious results in [10] in which a strategy for resilient attackdetection is formulated based on redundancy in the sensormeasurements. During the experimental implementation weuse the Robot Operating System (ROS) by Willow Garage[11] on different UGVs. It is worth mentioning that apreliminary study and assessment about the security of ROSwas performed in [12]. The authors showed that a keysecurity issue is that messages are not authenticated and canbe easily decipher and spoofed. In our work we do not solvethis problem and use it as a motivation and to create attacksthat compromise the sensors.

The remainder of this paper is organized as follows.In Section II we open our discussion with the problemformulation. In Section III we present the recursive estimatoralgorithm. In Section IV we show the architecture and modelused for the vehicles under analysis followed by simulationsin Section V. Hardware/software implementations on twoground robots are presented in VI and finally we drawconclusions in Section VII.

II. PROBLEM STATEMENT & PRELIMINARIES

Within this work we are interested in finding a strategyto guarantee that a robot under malicious attack can reach adesired state without being hijacked.

We assume that the robotic system is a discrete-time lineartime-invariant (LTI) of the following form

xk+1 = Axk + wk +Buk,zk = Cxk + νk,yk = zk + λk,

(1)

with xk ∈ Rn and uk ∈ Rm are the state vector andcontrol inputs at time k respectively. zk is the set ofmeasurements without the effects of attack while yk =[yk,1, yk,2 . . . yk,N ] ∈ RN is the sensor measurements vectorwith attack where yk,i ∈ R is the measurement taken by theith sensor at time k. λk is the attack vector in which each termrepresents attack of magnitude ‖λk,i‖. The attack vector isassumed to be arbitrary, i.e., we assume no prior knowledgeof statistical properties of bounds on the values of the attackvector. Finally, both the process noise wk = N (0,W ) and

the sensor measurement noise νk = N (0, V ) are expressedas Gaussian random variables.

Specifically we are interested in designing a linear re-cursive estimator consisting of the following prediction andupdate steps:

xk|k−1 = Axk−1|k−1 +Buk−1,xk|k = xk|k−1 +Kk

(yk − Cxk|k−1

).

(2)

We define yk|k−1 = Cxk|k−1 + λk and such that yk|k−1 =zk+sk where zk is an unknown minimum mean square error(MMSE) estimate of zk corresponding to a known covariancematrix Σk

1. sk = C(xk − xk) + λk is the deviation of theactual measurement estimate from the MMSE estimate andit is caused by potential attacks, previous attacks residuals,and counter measurements. Ideally we would choose Kk

such that KkCsk = 0. However this require mixed integerprogramming as demonstrated in [13]. Thus as a relaxation,in this work we use a recursive implementation and addressthe following problem

Problem 2.1: Bounding the Attack Innovation Given skthe effect of the attack on the system and ζ a constant smallpositive value, find the optimal gain value Kk such that thefollowing inequality holds:

E[‖Kksk‖2

]≤ ζ, (3)

where E[·] is the expected value and Kksk is the attackinnovation.

Through out this work we use the following assumptionon the maximum number of sensors under attack.

Assumption 2.2: Attack Feasibility The maximum toler-able number of sensors under malicious attack is less thanN/2.

III. DESIGN OF A RESILIENT CONTROL SCHEME

A. Resilient Recursive Adaptive Estimator (RAE)

Most of the techniques available in the current literatureare very efficient in detecting and removing sensors underattacks if we consider deterministic systems with boundedsmall noise or time invariant sensor models [7], [10]. How-ever, in real world applications the sensor noise profile is notalways fixed and variations due to non-attack effects such asbiases and environmental effects, can occur. Secondly, oftensome sensors, even if under attack or compromised, can bestill used and fused to obtain useful information and improvethe state estimation. Instead of using a binary selectioncriterium, we propose a strategy that assigns weights to eachsensor based on how close each measurement is from theestimated state.

Before showing our algorithm we introduce an oracleestimation to consider the optimal solution in the case thateverything was known a priori, including the attack vector.

1) Oracle Estimator: In the literature [14], [15] we findthat the Linear Quadratic Estimator has been widely usedin engineering applications because it combines measure-ments of the same variable but from different sensors andit combines inexact forecast of a system’s state with aninexact measurement of the state. However, an adversarialattack could destabilize the system and drive the vehicle toundesired states. If we are always able to predict a priori

1Σk is known despite zk being unknown since it only depends on thenumber of sensors

3693

when and where an attack will happen (e.g., an oracle), theprediction step takes the following form:

xok|k−1 = Axok−1|k−1 +Buk−1, (4)

where we have use the upper script o to represent the oracleestimation parameters. The predicted estimate covariancefollows as

P ok|k−1 = AP ok−1|k−1AT +W. (5)

The oracle update will take then the following form

xok|k = xk|k−1 +Kok

(yk − Cxk|k−1

), (6)

P ok|k = (I −KokC)Pk|k−1, (7)

with

Kok = Pk|k−1CQ

(QT

(CPk|k−1C

T + V)Q)−1

QT , (8)

where Q ∈ {Q|QQT = I,Qi,j ∈ {0, 1}} is the selectionmatrix given by the oracle, where I is the identity matrixand Qyk are the non-attacked measurements.

Clearly in a realistic scenario we will not have an oracle;thus, next we propose a strategy that attempts to solve thisproblem by implementing a recursive filter in which, togetherwith updating and predicting the state of the vehicle, weintroduce a resilience step (called Shield) to consider andisolate attacks on sensors. The oracle presented above willbe used later on to prove property of the developed recursiveresilient state estimator.

2) Resilient Adaptive Estimator: Our recursive algo-rithm is motivated by the well established results found inthe Linear Quadratic Estimator implementation with somemodifications to accommodate the possible presence of anattack in one of the sensors.

The generalized form of our resilient filter is

• PREDICT. The predicted state estimate becomes

xk|k−1 = Axk−1|k−1 +Buk−1, (9)

and the predicted estimate covariance follows

Pk|k−1 = APk−1|k−1AT +W. (10)

• UPDATE. In the update phase, we introduce the follow-ing modified gain

Kk = Pk|k−1CT(CPk|k−1C

T + V +Dk

)−1, (11)

where D is the Shielding Gain matrix (described below)introduced to consider and remove attacks on the sensormeasurements.The updated state and covariance become

xk|k = xk|k−1 +Kk(yk − Cxk|k−1), (12)

Pk|k = (I −KkC)Pk|k−1. (13)

• SHIELD. If an attack is present and such that one ofthe measurements is corrupted, the goal is to removeit or mitigate its effect. Since the attack vector isgenerally unknown, one strategy we can implement isto change the covariance matrix associated with themeasurement error in order to increase the uncertaintywhere the measurement is different from the predictedstate estimate. To this end, let us define the ShieldingGain Dk = diag{dk,1, dk,2, . . . , dk,N} as a positive

semidefinite diagonal matrix, then we can write

dk,j = dk−1,j + ηk,j

(∣∣∣∣∣∣∣∣yk,j − Cjxk|k−1

σk,j

∣∣∣∣∣∣∣∣2 − 1

),

(14)with σk,j = CjPk|k−1C

Tj + Vjj ,

where ηk,j is a gain factor of the measurement error.We call dk,j the shielding factor since its purpose isto increase the covariance of the measurement noiseassociated with the sensor under attack, thus “shielding”the malicious effects on the system. Notice that weimpose (14) to be always positive semidefinite. In fact, anegative value would imply that we are able to improvethe performance of a sensor (which is not possible).

We are now interested to show that the proposed strategyis guaranteed to satisfy Problem 2.1 if we choose correctlyη in (14). Before showing this, we introduce the attack-to-noise ratio, a new measure which relates the sensor noise tothe attack effects on the robotic agent, as follow:

Definition 3.1: Attack-to-noise Ratio (ANR) We definethe attack-to-noise ratio τ as a measure that compares theattack effects to the noise level of a sensor. In formula:

τ = ANR = ((Σy +Dk)−1sk)2, (15)

where Σy = CPk|k−1CT +V (see (11)) and sk is as defined

in Section II.

Theorem 3.2: The ANR is bounded For some ζ ≥ 0,there exists a η in (14) independent of sk such that theexpected value of ANR E[‖τ‖] ≤ ζ.

Proof: To prove Theorem 3.2 we start by consideringthe average ANR for the worst case scenario, and thencalculate its boundaries with respect to η. Let’s consider theexpression for ANR in (15):

‖τ‖ =∣∣∣∣((Σy +Dk)−1sk)2

∣∣∣∣ ≤ ∣∣∣∣((σ2minI +Dk)−1sk)2

∣∣∣∣=

N∑j=1

s2k,j

(σ2min + dk,j)2

. (16)

Thus, bounding E

[(sk,j

(σ2min+dk,j)

)2]

implies E[‖τ‖] is

bounded.

To this end, by Jensen’s inequality, we have that

E

[(sk,j

(σmin + dk,j)

)2]≤[

sk,jE [σ2

min + dk,j ]

]2

. (17)

Now we observe that[sk,j

E [σ2min + dk,j ]

]2

=

=

sk,j

E[σ2

min + dk−1,j + ηk,j

((yk,j−Cj xk|k−1)2

σ2k,j

)− 1]

2

=

sk,j

E[σ2

min + dk−1,j + ηk,j

((zk,j−zk+sk,j)2

σ2k,j

)− 1]

2

3694

=

sk,j

(σ2min + dk−1,j + ηk,j

(σ2k,j

σ2k,j

+s2k,j

σ2k,j− 1)

2

= f(s),

(18)where σ2

k,j = E[zk,j − zk].If we now take the derivative of f(s) with respect to s(ddsf(s) = 0

)we obtain a maximum at

s2k,j =

σ2k,j

ηj

(σ2

min + dk−1,j + ηk,j

(σ2k,j

σ2k,j

− 1

)). (19)

Now substituting s2k,j in (17) we obtain

E

[s2k,j

(σ2min + dk,j)2

]≤

σ2k,j

4ηk,j

(σ2

min + dk−1,j + ηk,j

(σ2k,j

σ2k,j− 1))

(20)which does not depend on sk,j , concluding the proof.Thus, Theorem 3.2 and the proof above show that there existsan upper bound on the ANR that depends on ηk,j in therecursive adaptive estimator defined in (9) - (14).

We are now ready to show that Problem 2.1 holds, giventhe bounds calculated on the ANR in Theorem 3.2.

Lemma 3.3: Bounded Attack Innovation Given the re-cursive adaptive estimator outlined in (9) - (14) there existsa η ≥ 0 such that

E[‖Kksk‖2

]≤ ζ (21)

Proof: Given Σy = (CPk|k−1CT + V ) and

Σxy = Pk|k−1CT ,

E[‖Kksk‖2

]=E

[‖Σxy(Σy +Dk)−1sk‖2

]≤‖Σxy‖2E

[‖(Σy +Dk)−1sk‖2

]=‖Σxy‖2E[τ ], (22)

where we notice that the second part of equation (22) is theANR, τ = ((Σy +Dk)−1sk)2.

Given thatηj =

σ2min + dk−1,j

1− σk,j

σ2k,j

(23)

is substituted into (20) and summed over all the sensors, then(21) is satisfied when

ζ ≥ Tr[(σmin2I +Dk−1)

−1(Σk − Σk) (σmin2I +Dk−1)

−1].

(24)

If (24) is not satisfied, then we need to reset the estimatorby assigning Σk = Σk.

B. Properties of the Shielding Gain

In the following lines we present some other properties ofthe proposed recursive algorithm focusing primarily on theexistence, expectation, and evolution of the shielding factors.

Lemma 3.4: Existence of the Shielding Gain Given therecursive implementation composed of the prediction, shield,and update steps of (9) - (14), there exists a shielding gainsuch that the estimated state is equivalent to the oracle one.In formula

∃Dk s.t. ∀Pk|k−1, Kk = Kok . (25)

Proof: Given K defined in (11) and assuming Ko in(8) is the optimal gain, we would like to show that there is

a Dk such that the following equality is satisfied

Q(QT

(CPk|k−1C

T + V)Q)−1

QT =

=(CPk|k−1C

T + V +Dk

)−1. (26)

Let

Py = (CPk|k−1CT + V ) =

(Pn PndPTnd Pd

), (27)

then the left hand side of (26) becomes

Q(QTPyQ)−1QT =

(P−1n 00 0

). (28)

Now for the right hand side, let us select

Dk =

(0 00 rI

), (29)

with r a constant. Then it follows that

(Py +D)−1 =

(Pn PndPTnd Pd + rI

)−1

=

=

(P−1n − P−1

n PndΨ−1PTndP

−1n −Ψ−1PndP

−1n

−P−1n PTndΨ

−1 Ψ−1,

)(30)

where Ψ = Pd + rI .Choosing r very large is equivalent to unselecting the

sensors that are under attack, thus leaving us with only thecorrect measurements. Therefore, as r goes to infinity weobtain

limr→∞

(Py +D)−1 =

(P−1n 00 0

), (31)

which is equivalent to the left hand side in (28).Lemma 3.5: Expectation of the Shielding Gain Given

the recursive algorithm described in (9) - (14), if the mag-nitude of the attacks is always non-zero, then the expectedvalue of the shield factor is always non-negative and non-decreasing. In formula, given Nλ the number of sensorsunder attack and j = 1, . . . Nλ

∀ Nλ ≤ N/2, if ‖λk,j‖ > 0 then E[Dk] ≥ Dk−1. (32)

Proof: The proof is straightforward. In the presence ofan attack:

E[Dk|attack] = Dk−1 + η

(1 +

∣∣∣∣∣∣∣∣ skσk,j∣∣∣∣∣∣∣∣2 − 1

)=

= Dk−1 + η

(∣∣∣∣∣∣∣∣ skσk,j∣∣∣∣∣∣∣∣2)≥ Dk−1, (33)

proving the statement of the lemma.Lemma 3.6: Probabilistic Evolution of the Shielding

Factors Given an attack λj , by using the recursive algorithm(9)-(14), the probability P that dk ≥ dk−1 increases byincreasing η.

Proof: To prove Lemma 3.6 we show that the proba-bility that dk ≤ dk−1 under attack depends on η in (14).

P [dk−1 ≥ dk|attack] ≤ E[dk−1

dk

]= dk−1E [1/dk] ≤

≤ dk−1

E [dk]=

1

1 + ηdk−1‖ skσk,j‖2. (34)

3695

(34) demonstrates that the probability that dk−1 ≥ dkunder attack is inversely proportional to η. In particular, wenotice that for η → ∞ the probability in (34) goes to 0,which, in turns, means that the probability of increasing dkat every step converges to 1.

In the following sections we discuss simulation results andexperimental implementations on ground wheeled robots.First we give an overview of the architecture of our controlsystem and the robots dynamical model used during thesimulations and hardware implementations.

IV. SYSTEM ARCHITECTURE

Our formulation is hierarchical and use feedback to controlthe motion of the vehicle and achieve the desired state.Fig. 2 shows a block diagram representing all the controlcomponents used in our framework.

Fig. 2. Block diagram representing the control system architecture employedfor secure cruise control.

A. Dynamical ModelWe illustrate the development framework on a design

of secure cruise control of two fully electric unmannedground vehicles (UGV): the Black-I LandShark [3] shownin Fig. 1, and a custom made robot built at the University ofPennsylvania (UPenn) which we call MiniShrak, shown inFig. 3(a). Both vehicles are equipped with encoders, IMUs,GPS, and vision sensors. From a computation perspective,both UGVs are equipped with quad-core processors runningUbuntu with ROS.

To obtain a dynamical model of the vehicle we have usedthe standard differential drive vehicle model (Fig. 3(b)) [16].Here, Fl and Fr denote forces on the left and right set ofwheels respectively and Br is the mechanical resistance ofthe wheels to rolling. The vehicle position is specified byits px and py coordinates, θ denotes the heading angle ofthe vehicle measured from the x axis, while v is the speedof the vehicle in this direction. Both the vehicles in ourtestbed employs skid steering, meaning that in order to makea turn it is necessary to generate enough torque to overcomethe sticking force Sl. Consequently, if we assume that thewheels do not slip, the dynamical model of the vehicle canbe specified as:

v =

{1m (Fl + Fr − (Bs +Br)v, if turning1m (Fl + Fr −Brv, if not turning

ω =

{1Jt

(B2 (Fl − Fr)−Blω, if turning0, if not turning

θ = ω

px = v sin(θ), py = v cos(θ)(35)

Also, w = 0 if the vehicle is not turning. Finally, to estimatethe state of the vehicle for cruise control (i.e., its speed) weuse three sensors: typically the wheel encoders on both setsof wheel, inertial sensors such as the IMU, and the GPS.We have also derived a 6-state linear model of the low-levelelectromechanical system, which is then used to derive alocal controller that provides the desired Fl, Fr levels.

(a) (b)

Fig. 3. Skid-steering ground vehicle; (a) The MiniShark unmanned groundvehicle; (b) Coordinate system and variables used to derive the model.

V. SIMULATION RESULTS

In this section we show simulation results for the resilientadaptive estimator discussed in Section III in comparisonwith a well established resilient state estimator presented in[17], [18].

A. Overview of the Resilient State Estimator (RSE)To illustrate our development framework we compare our

technique with the work from [17], [18], where recent resultson error correction over the reals and compressed sensing areused to derive a technique to develop secure state estimatorswhen system sensors or actuators are under attack.

In [17] it is shown that for linear systems the state estimatecan be obtained from the previous N sensor measurementsand actuator inputs as the minimization argument of thefollowing optimization problem:

minx∈Rn

‖Y N − ΦNx‖l1/l2 . (36)

Here, for a matrix M ∈ Rp×N , ‖M‖l1/l2 denotesthe sum of l2 norms of the rows of the matrix, andΦN =

[Cx|CAx| . . . |CAxN−1

]. Furthermore, Y N =

[yk−N+1|yk−N+2| . . . |yk] aggregates the sensor measure-ments while taking into account applied inputs – i.e., yk =yk −

∑ki=k−N+1 CA

iBuk−1−i.

B. Software ImplementationsFor ease of discussion we abbreviate the resilient adaptive

estimator as RAE and the resilient state estimator in [17],[18] as RSE. In all simulations presented in this section therobot is set to first maintain a cruise control speed of 4 m/sand after 50 seconds to switch to 10 m/s. The state spacerepresentation of the vehicle has been deduced from carefulmeasurements on the real robot.

Following the architecture and the dynamical model forskid-steering vehicles described in Section IV we developeda ROS based simulator that emulates the same electro-mechanical and dynamical behavior of the real robot. The topsubfigure in Fig. 4 displays the normalized voltage suppliedto the motors. The middle plot shows the true velocity ofthe simulated robot, and finally on the bottom of Fig. 4the three sensor measurements with applied an attack on theGPS measurement are displayed. The attack in this case ismodeled as a 10 Hz pulse with pick amplitude of 5 m/s.

3696

Fig. 4. Rqt-plot of the results from the ROS Simulator with GPS datacorrupted.

The developed ROS simulator was used in debuggingphase before using the real robot. Notice that both thesimulator and the real robots use exactly the same ROS code.

We now show a comparison between the RAE and theRSE, with simulations run in Matlab/Simulink. The samesimulation of Fig. 4 was run with the RSE. In Fig. 5 we reportthe error between reference and true velocity values for bothstrategies. Each strategy behaves correctly converging to 0every time we select a new reference velocity.

Fig. 5. Comparison between RSE and RAE state estimation errors for lownoise situation.

Fig. 6 displays the state estimation error comparisonbetween RSE and RAE when the sensor noise is increased(σ = 1 m/s). The attack on the GPS is modeled as a pulsewith amplitude alternating between 3 m/s and 0 m/s every10 s. Again we notice that both estimators errors convergeto 0 within the noise profile of the sensors. The RAE hasslightly less oscillations than the RSE.

Fig. 6. Comparison between RSE and RAE state estimation errors for noisymeasurements with large attacks

Finally, in the simulation displayed in Fig. 7 we increasethe noise of each sensor to σ = 2 m/s. In this case we hidethe attack within the noise profile of the GPS measurementbut keep it around the boundary of the noise, that is, theattack is a pulse with magnitude alternating between 2 m/s

and 0 m/s every 10 s. Because the noise is large and theattack is within the noise profile and on its boundary, we seethat the error grows when an attack is inserted and decreaseswhen it is removed creating oscillation in the velocity.

Fig. 7. Comparison between RSE and RAE state estimation errors for largenoisy measurements with attacks within the noise profile.

Table I shows numerical results that compare four differentstrategies: an Oracle estimator (Oracle), a Kalman Filter(KF), the Resilent State Estimator (RSE) of [17], [18], andthe Resilient Adaptive Estimator (RAE) proposed in thispaper. We use the same three case studies run above withinFigs. 5-7. Each entry on the table contains two values: theupper number is the average error between estimated andtrue state, while the lower number is the ratio between theestimated and the oracle errors. As expected, the KalmanFilter approach is not able to deal with attacks whose valueexceeds the noise profile of the sensor measurements, while itcan be used in the case that the attack is camouflaged withinthe error noise of the sensor (last column of the table). Asalready discussed, the RAE behaves slightly better than theRSE with a maximum recorded error of 18% against the 21%of the RSE in the case of noisy measurements with attackvector outside the noise profile (second column of the table).

TABLE ISTATE ESTIMATE ERROR UNDER ATTACK

σ = 0.01 σ = 1 σ = 2Approachλ = 5 λ = 3 λ = 2

0.005 0.13 0.18Oracle1.00 1.00 1.001.67 0.99 0.35KF334 7.62 1.94

0.005 0.21 0.32RSE1.00 1.62 1.78

0.005 0.18 0.28RAE1.00 1.38 1.56

VI. HARDWARE/SOFTWARE IMPLEMENTATION

The resilient state estimator and the recursive adaptiveestimator strategies have been implemented through severalexperiments on the two robots described in Section IV-A andon different types of surfaces. To facilitate the experimentalevaluation, we have built a remote User Interface (UI) (seesubfigures inside Figs. 8 and 9) which allows us to start/stopprocesses, attack each sensor, read/save data, and visualizeplots of important information such as the speed calculatedfrom the sensor measurements and the input sent to the

3697

actuators. For these hardware implementations we decidedto use GPS and the left and right encoders to obtain threeindependent speed measurements.

In Fig. 8 it is shown a snapshot of a cruise controlexperiment run on the LandShark on a tiled pathway insidethe UPenn campus. As noted from the UI, the robot canreach and maintain the desired reference speed even whenone of the sensors is under attack. Next experiment inFig. 9(a) displays the MiniShark running with the resilientstate estimator described in Section V-A. Finally in Fig. 9(b)we present a snapshot of the implementation of the recursiveestimator derived in Section III-A, showing similar results aswith the LandShark in Fig. 8. Once an attack is injected inone of the sensors, a weight is added to the noise varianceof the corrupted measurement decreasing its trustworthiness.All experimental results show that the two methods can beused to efficiently estimate the state of the system, althoughthe RAE is computationally more efficient and does notrequire a history of measurements and control inputs.

Fig. 8. Experimental result. Snapshot of the deployment of the LandSharkon a tiled pathway inside the University of Pennsylvania. The picture in thepicture displays the user interface used during the experiments.

(a) (b)

Fig. 9. Snapshots of the deployment of the MiniShark UGV on a grass fieldinside the University of Pennsylvania. The robot is in cruise control andone by one each sensor is compromised with a constant attack. Figure (a)shows the implementation of the RSE, while (b) depicts the RAE strategy.

VII. CONCLUSION & FUTURE WORK

In this paper we have presented a method to estimate thestate of a system under malicious attack on the sensors ona vehicle. The proposed recursive state estimator comparesthe estimated state with redundant measurements comingfrom different sources and returns a higher variance ofthe measurement noise if a sensor is under attack. Thisstrategy allows to consider noisy measurements to estimatethe correct state of the system and can be used for attacks thatact outside the noise profile of the sensors. However thereare few limitations: the algorithm needs an accurate selectionof the noise profile and weights in order to converge to the

correct state. A too small bound on the error noise impliesthat the estimator may reject most of the measurements whilea too large bound on the error can lead more attacks goingthrough the system because within the error noise profile. Inreal applications these boundaries on the noise profiles areusually given or can be calculated through hardware testing.

Future work will be centered on: i) implementing theproposed strategies on unstable systems (e.g., quadrotors)and more evolved experiments involving obstacle avoidanceand way-point navigation; ii) running more complex coordi-nated attacks, and iii) developing supervisory capabilities toalternate between different strategies.

ACKNOWLEDGMENTS

This work is based on research sponsored by DARPAunder agreement number FA8750-12-2-0247.The authors would like to thank Prof. Paulo Tabuada foruseful discussions about the resilient state estimator, PeterGebhard for creating the UI used in the experiments, andProf. Daniel Lee and his students for their help in buildingthe MiniShark robot.

REFERENCES

[1] “Spoofers Use Fake GPS Signals to Knock a Yacht OffCourse. http://www.technologyreview.com/news/517686/spoofers-use-fake-gps-signals-to-knock-a-yacht-off-course.”

[2] M. Pajic, N. Bezzo, J. Weimer, R. Alur, R. Mangharam, N. Michael,G. J. Pappas, O. Sokolsky, P. Tabuada, S. Weirich et al., “Towardssynthesis of platform-aware attack-resilient control systems,” in Pro-ceedings of the 2nd ACM international conference on High confidencenetworked systems. ACM, 2013, pp. 75–76.

[3] “Black-I Robotics LandShark UGV.http://www.blackirobotics.com/LandShark UGV UC0M.html.”

[4] J. S. Warner and R. G. Johnston, “A simple demonstration that theglobal positioning system (gps) is vulnerable to spoofing,” Journal ofSecurity Administration, vol. 25, no. 2, pp. 19–27, 2002.

[5] “Car Hacking http://blog.ioactive.com/2013/08/car-hacking-content.html.”

[6] S. Checkoway, D. McCoy, B. Kantor, D. Anderson, H. Shacham,S. Savage, K. Koscher, A. Czeskis, F. Roesner, and T. Kohno,“Comprehensive experimental analyses of automotive attack surfaces,”in Proc. of USENIX Security, 2011.

[7] M. Zhu and S. Martinez, “On resilient consensus against replay attacksin operator-vehicle networks,” in American Control Conference (ACC),2012. IEEE, 2012, pp. 3553–3558.

[8] F. Pasqualetti, F. Dorfler, and F. Bullo, “Attack detection and identi-fication in cyber-physical systems,” IEEE Transactions on AutomaticControl, 2012, submitted.

[9] Y. Mo, E. Garone, A. Casavola, and B. Sinopoli, “False data injectionattacks against state estimation in wireless sensor networks,” in 49thIEEE Conference on Decision and Control (CDC), 2010. IEEE, 2010,pp. 5967–5972.

[10] H. Fawzi, P. Tabuada, and S. Diggavi, “Secure estimation and controlfor cyber-physical systems under adversarial attacks,” arXiv preprintarXiv:1205.5073, 2012.

[11] “Robotic Operating System. http://www.ros.org.”[12] J. McClean, C. Stull, C. Farrar, and D. Mascarenas, “A preliminary

cyber-physical security assessment of the robot operating system(ros),” in SPIE Defense, Security, and Sensing. International Societyfor Optics and Photonics, 2013, pp. 874 110–874 110.

[13] J. Weimer, N. Bezzo, M. Pajic, O. Sokolsky, and I. Lee, “Attack-resilient minimum-variance estimation,” in American Control Confer-ence (ACC), 2014 (to appear). IEEE, 2014.

[14] M. S. Grewal and A. P. Andrews, Kalman filtering: theory and practiceusing MATLAB. Wiley. com, 2011.

[15] D. Simon, Optimal state estimation: Kalman, H infinity, and nonlinearapproaches. Wiley. com, 2006.

[16] J. J. Nutaro, Building Software for Simulation: Theory and Algorithms,with Applications in C++. Wiley, 2010.

[17] H. Fawzi, P. Tabuada, and S. Diggavi, “Secure state-estimation fordynamical systems under active adversaries,” in Proceedings of the2011 49th Annual Allerton Conference on Communication, Control,and Computing (Allerton), 2011, pp. 337–344.

[18] ——, “Security for control systems under sensor and actuator attacks,”in Proceedings of the 51st IEEE Conference on Decision and Control,2012.

3698