university of rome la “sapienza” infocom department

1Perugia, February 13, 2007

University of Rome La “Sapienza”INFOCOM Department

BIO-INSPIRED SENSOR NETWORK DESIGN:

DISTRIBUTED DECISION THROUGH SELF-SYNCHRONIZATION

Sergio Barbarossa

Ack’s: WINSOC project (IST-FP6) and ARL/ERO - R&D 9989-CE-01 Collaborators: G. Scutari, L. Pescosolido

Perugia, February 13, 20072

Univ. of Rome “La Sapienza”

Overview

• Motivating remarks

• Was the problem already solved in some unpublished notes ?

• Fundamental limits in wireless communications: A tiny step towards semantic

• In-network processing: Distributed consensus algorithms

• Directed graphs: How to model interactions

• Decentralized decision through self-synchronization

• Entropy flow: How to monitor self-organization

• What is the price for self-organization ?

• Conclusion



• Fundamental motivating application: Sensor networks

• Requirements: High reliability, small energy consumption, economy of scale, adaptive MAC / routing capabilities, energy scavenging

• Criticalities: Energy consumption, survivability, vulnerability to node failures, sleep modes or intentional attacks, congestion around sink nodes, scalability

• Resources: very inexpensive, simple, unreliable nodes, with very limited energy supply and simple MAC / routing mechanisms

Motivating remarks

… it may look like a nightmare for an engineer !

… or maybe is an opportunity to apply for research funds ?



Was the problem already solved in some unpublished notes ?

Example: heartbeat

a single natural pacemaker cell has

• a life cycle much smaller than average human being lifetime

• limited individual reliability and precision

a population of mutually coupled pacemakers gives rise to a very stable and reliable system

nevertheless …

design sensor networks as a population of mutually coupled oscillators Idea:




Other examples

• fireflies in South East Asia

• brain neurons

• lasers

• menstrual cycle in women living in close contact

• muscular contraction in digestive system

• cellular mitotic division



References


Huyghens, 1658 nearby pendula tend to synchronize

Kaempfer, 1680 South-East Asia fireflies flash simultaneously

Kuramoto, 1984 chemical oscillations, waves and turbulence

Mirollo, Strogatz 1990a population of globally coupled

oscilllators may converge to a unique stable

equilibrium under very mild conditions



• Capacity of one-to-one link [Shannon ‘48]

• Transport capacity of many one-to-one wireless links [Gupta, Kumar, ‘00], [Xie, Kumar , ‘04]

• Transport capacity of many-to-one wireless links [Duarte-Melo, Liu, ‘03]

Fundamental limits in wireless communications: A tiny step towards semantic

logC

n n

1

Cn

1



Scaling laws for wireless sensor networks

Goal of a sensor network: Compute a function of the measurements

collected by N sensors

Data-centric view: is invariant to any permutation of the collected data

what is important is the value of the collected data, not which sensor has collected which measurement

transport capacity scales as

( , ,..., )Nf x x x1 2 , ,..., Nx x x1 2

( , ,..., )Nf x x x1 2

logC

n

1




Besides providing bounds on transport capacity, fundamental scalinglaws suggest also strategies to approach the bounds:

• Spatial reuse / multihop

• Distributed source / channel coding

• In-network processing / computing

• Hierarchical layering clustering


Basic message: Efficient network design should take into account the goal of the network data-centric and event-driven approaches



Hierarchical layering

environment

lower-level nodes

higher-level nodes

controlnodes

- Low level nodes pre-process the data and take local decisions

- High level nodes carry relevant information to control centers



In-network processing: Distributed consensus algorithms

References:

• Eisenberg, Gale, “Consensus of subjective probabilities: The pari-mutuel method”, 1959

• DeGroot, “Reaching a consensus”, 1974

• Borkar, Varaiya, “Asymptotic agreement in distributed estimation”, 1982

• J. N. Tsitsiklis, “Problems in decentralized decisions making and computation,” 1982

• Olfati-Saber and Murray, “Consensus Protocols for Networks of Dynamic Agents”, 2003

• Jadbabaie, Lin, and Morse, “Coordination of groups of mobile autonomous agents using nearest neighbour rules”, 2003

• Xiao, Boyd, and Lall, “A scheme for robust distributed sensor fusion based on average consensus,” 2005

• Barbarossa, Scutari “Decentralized ML estimation through nonlinearly coupled osc. ”, 2005




Motivating problems:

1. Distributed estimation

Given the vector measurements gathered by N nodes

can we achieve the globally optimal (ML or BLUE) estimate

using a totally decentralized approach (without a fusion center) ?

, 1, 2,...,i i i i N y A ξ v

-1N N1 1

i=1 i=1

ˆ T Ti i i i i i

ξ A R A A R y




Motivating problems:

2. Multiple hypothesis testing

Denoting by the conditional pdf of the observation vector ,

conditioned to the hypothesis Hk , with a priori known probability P(Hk) ,

assuming that different sensors collect conditionally independent measurements,

can we derive the minimum error rate (MAP) test

using a totally decentralized approach (without a fusion center) ?

( / )i i kp y H iy

1ˆ arg max ( / ) ( ) arg max ( / ) ( )Nm k k i i i k kk k

p P p P y yH H H H H



Proposed approach [Bar ‘05]: Each sensor takes an initial estimate (decision) as a function of its measurement and starts evolving as follows:

where

running estimate of each sensor

nonlinear, odd, increasing function

global coupling gain

local coupling attenuation (ci > 0)

( )f

ic

ix t

K


ij jia apj = transmit power of sensor j

dij = distance between nodes i and j

hij = channel fading coefficient



1. Can we design local functions , attenuation coefficients to guarantee that each sensor state (derivative) converges towards globally optimal sufficient statistic ?

2. What is the impact of nonlinear coupling ?

3. What is the impact of delays and (fading) channel coefficients ?

4. Which are the convergence conditions ?

Basic questions:




Network topologies

Directed graphs: How to model interactions

strongly connected(SC) digraph

quasi strongly connected(QSC) digraph

weakly connected(WC) digraph, witha two-tree forest

Every strongly connected component (SCC) can be substituted by a single node of the so called condensation digraph

If a node of the condensation digraph is a root node, thecorresponding strongly connected component is a root SCC (RSCC)



1. If the coupling function is linear, the network is SC, then all state derivatives

converge to the same function of the measurements (global consensus)

This state is globally asymptotically stable

Decentralized decision through self-synchronizationConvergence conditions in the no-delay case

The system is totally democratic: the final consensus depends on all



2. If the coupling function is linear, the network is not SC and it contains only one directed tree, then all state derivatives converge to the root decision


All the nodes obey to the decision taken by the leader (root node)




3. If the coupling function is linear, the network is not SC and it contains a forest of K RSCC components, then the state derivatives converge to a linear combination of the root decisions

The network forms K clusters of consensus

cluster Cq

is the i-th entry of the left eigenvector associated to the smallest eigenvalue of the graph Laplacian consensus depends on topology !



Decentralized decision through self-synchronizationConvergence conditions in the delayed case

4. If the coupling function is linear, the delays are not negligible, the network contains a forest of K RSCC components, then the state derivatives converge to a linear combination of the root decisions

Consensus (global or local) depends on channel coefficients and delays

Nevertheless, a two step iterative algorithm is sufficient to remove any bias, without knowing or estimating the channels



5. If coupling function is nonlinear (odd, monotonically increasing), the graph is

non directed and connected, and K > Kc, then all state derivatives converge

to a global consensus

The synchronized state is globally asymptotically stable

Decentralized decision through self-synchronizationConvergence conditions in the nonlinear, non-delayed case

All dynamical systems converge to the same value of the state derivative,

which is unique, irrespective of their initial conditions



The critical coupling coefficient is lower bounded by

depends only on network topology

depends only on measurement statistics

depends only on coupling function

• If the network is not connected,

the oscillators cannot reach a consensus

2 0A L

• If coupling is linear, and the network is connected,

the oscillators always reach a consensus

• The consensus speed is proportional to

maxf

2K L

Decentralized decision through self-synchronizationConvergence conditions in the nonlinear, non-delayed case



• If coupling function is linear, necessary and sufficient condition for reaching a

consensus is that the digraph associated to the network is QSC

• The final consensus depends on the topology (# of root nodes), but it does not

depend on the channel coefficients

• If coupling function is nonlinear (odd, monotonically increasing), global

consensus is achieved if network is QSC and coupling is sufficiently strong

• In the presence of delays, consensus depends on channel coefficients and

delays, but it is possible to remove the bias by running the consensus

algorithm twice, without the need to estimate neither channel coefficients nor

delays

Summary of convergence conditions



Any function of the collected measurements that can be expressed in the form

with ci >0, can be computed with a totally distributed strategy based on

self-synchronization

Examples:

parameter estimation, detection of known waveforms in noise, detection of Gaussian process in Gaussian noise, belief propagation, …

Decentralized decision through self-synchronizationDistributed computing through self-synchronization



Scalar observation model

where u is the unknown parameter and

Distributed estimation: run

In the non Gaussian, case, this estimator coincides with the Best Linear Unbiased Estimator (BLUE)

Decentralized decision through self-synchronizationHow to set network parameters to achieve optimal estimates



Decentralized decision through self-synchronizationEstimation of vector parameters

Vector model

Strategy: Initialize each node with local ML estimate and run

All nodes tend to the global ML estimate, without sending neither the observations,nor the mixing matrices, nor the covariance matrices to any fusion center



MAP test may be achieved by letting the network evolve with

and ci = 1, for as many times as the number of hypotheses

At convergence, each node applies the function

to the consensus value achieved under hypothesis k

The asymptotic consensus value, in the k-th iteration, is proportional to the argument of the MAP detector

Decentralized decision through self-synchronizationHow to choose network parameters to build MAP detector

( / ) log ( / ) ( )i i k i i k kg p Py yH H H

* *( ) exp( )k ku

*k

1 ( / ) ( )Ni i i k kp P y H H



Linear coupling with random locations and geometry dependent delays

Numerical examples



0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.580

85

90

95

100

105

110

time

mµ̂

§¾̂ µ

Estimated value +/- standard deviation vs. time

centralized MLE (BLUE)

decentralized - delayed

two step decentralized delayed

decentralized undelayed

Rayleigh fading channels with distance-dependent variance, random network topology, 40 nodes

Observation: with ,

Numerical examples



Example: random Gaussian (6x3) mixing matrices network topology: regular graph with fixed node degree = 4

101

102

10-4

10-3

10-2

Number of Sensors

Estim

ation

var

iance

Estimation variance decreases as 1 / N, even if degree is fixed

Performance improves adding nodes, without changing node Tx power (degree)

Average estimation variance vs. number of nodes

Examples: Estimation of vector parameters



Binary hypothesis testing

Signal model: Gaussian random patterns with known variances

Optimal centralized rule:

Decentralized solution: run

and compare with a threshold

Examples: Decentralized detection



Examples: Decentralized detection

0 1 2 3 4 5 6 7 8 9 100

1

2

3

4

5

6

7

8

9

10

x

y

0 1 2 3 4 5 6 7 8 9 100

1

2

3

4

5

6

7

8

9

10

x

y

0 1 2 3 4 5 6 7 8 9 100

1

2

3

4

5

6

7

8

9

10

x

y

0 1 2 3 4 5 6 7 8 9 100

1

2

3

4

5

6

7

8

9

10

x

y

0 1 2 3 4 5 6 7 8 9 100

1

2

3

4

5

6

7

8

9

10

x

y

Example of performance over a random grid: Colors encode detection decisions

Eventually, all oscillators end up with the same decision statistic



Detection probability vs. SNR and number of nodes, for a given Pfa

-10 -5 0 5 10 15 20 25 3010

-3

10-2

10-1

100

Peak SNR (dB)

Det

ectio

n P

roba

bilit

y

Detection Probability @ PFA = 10-3 , =2 , = 100.5

N = 4N = 16N = 36N = 64N = 100

Performance improves by increasing number of sensors, even if the degree is kept fixed

Decentralized detection

Degree is four,for any N

N



Entropy evolution [Bar-Scu ‘07]

Entropy flow: How to monitor self-organization

Are we contradicting thesecond law of thermodynamic ?

0 100 200 300 400 500 600 70050

60

70

80

90

100

110

120

time

Ent

ropy

cluster consensus system

global consensus systems

Entropy decreases !

Final value depends on how many nodes contribute to final decisions



Total energy spent for achieving consensus is inversely proportional to [Bar-Scu-Swami ‘07]:

The dark side of distributed consensus: iterations

There exists an optimal transmit power that minimizes total energy

Random spacing is equivalent to uniform spacing

10 11 12 13 14 15 16 17 18 19 2010

1

102

103

Transmit power at each node

Tota

l ene

rgy

random grid

uniform grid



Using K less than critical value, the network may be forced not tosynchronize

Example: Noisy temperature field observed by a regular grid of sensors

Spatial clustering

x

y

0 5 10 15 20 25 30 35 40

0

5

10

15

20

25

30

35

40

x

y

0 5 10 15 20 25 30 35 40

0

5

10

15

20

25

30

35

40

x

y

0 5 10 15 20 25 30 35 40

0

5

10

15

20

25

30

35

40

x

y

0 5 10 15 20 25 30 35 40

0

5

10

15

20

25

30

35

40

The network tends to form spontaneous clusters



If sensors are distributed over a line, they are sufficiently close to eachother and there is linear coupling only between adjacent sensors, i.e., except

Spatial smoothing or clustering

state evolution follows diffusion equation, triggered by initial observation

information propagates as a heat diffusion process

smoothing against observation noise is a result of diffusion



0 5 10 15 20 25 30

0

5

10

15

20

25

30

0 5 10 15 20 25 30

0

5

10

15

20

25

30

Spatial smoothing or clustering

Example: Noisy temperature field observed by a regular grid of sensors

Initial observation Smoothed phase



• Consensus through self-synchronization proves to be a very versatile strategy matched to the data-centric characteristic of WSN

• The approach allows for an easy implementation of radio transceivers

• Robustness and fault tolerance can be achieved through distributed coding / processing / computing / communicating …

• Great potentials for economy of scale and miniaturization

• Information can propagate as a diffusion wave percolating through the network in analog form

• Switching behavior from global consensus to local clustering or smoothing is possible using the same basic mechanism

Conclusion



self- synchronization may be a beautiful subject to study …

Conclusion



… sometimes it doesn’t work

Conclusion



… but when it works, it may be rewarding …

Conclusion



… sometimes, it may be just for fun… sometimes, it may be necessary

Conclusion

thank you !

university of rome la “sapienza” infocom department

Documents