Distributed solutions for visual sensor networks to detect targets

in crowds

Cheng Qian

Outline

• Visual sensor networks for target detection

• Computing paradigms in sensor networks

• Local processing

• A centralized solution

• Distributed solution I

• Distributed solution II

Visual sensor networksThe technology involves deploying (manually or from a plane) a large number of small, inexpensive motes over the area of interest.

Each mote carries

Visual sensors ( CCD or thermal ) with limited range and field of view (FOV).

Limited computing capacities and storage resources.

Wireless channels to communicate with other sensors

Why sensor networks for an application about target detection ?

Existence? A target may be occluded from the vision of “a” mote.

Localization? A visual mote is more like a orientation sensor.

3D Shape of the target? A mote only captures a 2D silhouette of the target.

www.ece.ncsu.edu

Collaboration among motes in the entire network.

Visual sensor networksDifference from a multi-perspective system where

each camera has no data processing capacity.

cameras are deployed with a planned strategy.

cameras are never enough to be called densely deployed

no energy and bandwidth concerns

Key sentences in visual sensor networks for target detection

The local processing capacity should be fully exploited.

Sensors should be dynamically aware of the location of neighboring sensors

Decision should be made by integrating information captured by the entire network.

Redundant information should be discarded and data should be only directed to related sensors.

A powerful central station Research Microsoft

Trade-off between information integration and information transmission.

Information integration: entire network local motes

Wireless transmission: long distance short distance

Networking traffic jam: almost certainly almost impossible

Visual sensor networks-Computing paradigms

Illustrations from Xiaolin Wang’s thesis

Centralized client/server model Distributive peer to peer model

Distributive cluster-based model

Related motes are clustered, but how to define “motes being related”??

In the context of sensor networks, computing paradigm refers to the information processing model deployed in the application layer of the protocol stack.

Local processing in a MoteEach mote encloses useful information into limited bits.

A target may be projected to a ridge in an image, and each ridge is represented by its central axis, height and colors.

Raw image

Silhouette

Silhouette boundary

Remove noises

Remove spikes

Ridges

A mote

A centralized solution

1. Sweep the ridge through the common ground. and drop votes to spots the ridge passes by

2. Select the spot with highest number of compatible votes ( same color, same height…). This spot is declared to be occupied by a target.

3. Find all the ridges contributing to this declaration. Cancel all the other votes created by these ridges. Remove those spots between the declared spot and the ridges.

4. Go back to step 2.

Integrating ridges from all the motes for detection and localization - A spot is seen having a ridge by multiple motes can be declared to be occupied by a target. The declaration gets more confirmed with the number of contributing motes increases.

To save memory and increase running time, the ground plane is implemented by a quad-tree structure instead of a grid.

A centralized solution Shape reconstruction - A visual hull.

Transmit all the critical points along the boundary of the ridge,

and stack up the “slices” of visual hull.

Cluster related sensors

A distributed solution I

1. Distance smaller than d

d

--- Ball Pivoting

2. FOV overlapped

--- A convex hull.

3. Information is exchanged between neighboring clusters through motes on the cluster boundaries.

What about meshing the motes, and each mesh with three vertex motes forms a cluster? Each cluster is only responsible for detecting local occurrences inside that mesh.

A distributed solution II Recall the centralized solution

1. Decomposed a central task requiring storage of a global map and computation about global optimization into local tasks belonging to each mesh.

2. Neighboring meshes can exchange information with each other to reduce redundancy.

Thanks