toward scalable activity recognition for sensor networks

MITSHUBISHI ELECTRIC RESEARCH LABORATORIESCambridge, Massachusetts

Toward Scalable Activity Recognition for Sensor Networks

Christopher R. WrenMERL Research

Emmanuel Munguia TapiaMIT House_n

MITSUBISHIELECTRIC

Changes for the better

MITSUBISHI ELECTRIC RESEARCH LABORATORIES

MITSUBISHIELECTRIC



t

Office

MITSUBISHIELECTRIC



t

t

Office

Hall

MITSUBISHIELECTRIC



A sensor network for building context.

MITSUBISHIELECTRIC



Second Implementation

Emmanuel’s MERL internship Summer 2005

Wireless Motion DetectorWires for Power

155 installed and operational

MITSUBISHIELECTRIC



Why Smarter Buildings?

• Safety & Security– Unusual Activity Detection– Economical Context Sensing– Tuned Escape Routes

• Eldercare – Activity Monitoring– Awareness

• Energy Efficient Buildings– Lighting– HVAC– Integrated building systems

• Socially–Aware Buildings– Presence– Availability– Awareness

• Entertainment• Elevator Demand Prediction• Retail

– Predict densities– monitor reaction to displays

• Industrial Process– Ergonomics– Safety– Efficiency

MITSUBISHIELECTRIC



How do you build a sensor network?

MITSUBISHIELECTRIC



MITSUBISHIELECTRIC



Ka

Photo courtesy of Kathy Ryall

MITSUBISHIELECTRIC



LoCA27 nodes1 receiver

MITSUBISHIELECTRIC



today155 nodes7 receivers

MITSUBISHIELECTRIC



How do you use a sensor network?

MITSUBISHIELECTRIC



SARSEN: Scalable Activity Recognition for Sensor Networks

•Locality of processing–Abstract information early–Reduce communication–Small data means small CPU

•Locality of Context–Build form small components–Small areas are more portable–Independent of context–A hallway is a hallway

•Build understanding–Tracking is not possible–Find consistent explanations–Abstractions reduce space

MITSUBISHIELECTRIC



Behavior Hierarchy

Action

Activity

Movement

Motion no context, no temporal information

limited local context, no temporal information

more context,significant temporal structure

activities in global context

MITSUBISHIELECTRIC



Behavior Hierarchy

Action

Activity

Movement

Motion

MITSUBISHIELECTRIC



Hardware

• 155 sensors, public spaces of a 2000m2 office building floor.• 7 weeks of data and counting• 3x106 observed motion events• ~90-150 people• Collaborative environment• Many kinds of spaces:

– laboratories – cubicles – offices – meeting rooms

• frequent visitors

MITSUBISHIELECTRIC



Behavior Hierarchy

Action

Activity

Movement

Motiontopology

MITSUBISHIELECTRIC



Automatically Recovered Topology

TR2005-003: “Similarity-based Analysis for Large Networks of Ultra-Low Resolution Sensors.” To Appear Pattern Recognition Special Issue on Similarity-Based Pattern Recognition

MITSUBISHIELECTRIC



Behavior Hierarchy

Action

Activity

Movement

Motion

MITSUBISHIELECTRIC



Movement

MITSUBISHIELECTRIC



MITSUBISHIELECTRIC



Turn Movement

MITSUBISHIELECTRIC



Walk Movement

MITSUBISHIELECTRIC



Still Movement

MITSUBISHIELECTRIC



Behavior Hierarchy

Action

Activity

Movement

Motion

MITSUBISHIELECTRIC



Activities

• Temporal Structure: Markov Chains• Trained from hand-written examples

– Visit: stop, stand around, start– Meet: exit, exit, exit.

MITSUBISHIELECTRIC



Meet Activity

MITSUBISHIELECTRIC



Visit Activity

MITSUBISHIELECTRIC



Behavior Hierarchy

Action

Activity

Movement

Motion

MITSUBISHIELECTRIC



Next Generation

>3 years on AA!802.15.4 radio

Easy to manufacture

Jonathan WesthuesDarren Leigh

MITSUBISHIELECTRIC



Summary

•Buildings as experts in themselves!

•Cheap to install and operate•Potentially cheap…

…also in terms of privacy

•Aware buildings can be safer, more efficient, richer.

•Hierarchical Behavior Elements…not Tracks

SARSEN

MITSUBISHIELECTRIC



MITSUBISHIELECTRIC



Thank You!

Christopher R. [email protected]

Emmanuel Munguia [email protected]

MITSUBISHIELECTRIC



MITSUBISHIELECTRIC



Why Motion Detectors?

They’re cheaper, require less bandwidth, less computation, less memory, but also….

Cameras provide a greater potential for abuse when you vary the intentionality of the person in control of the technology.

Dimensional Metaethics, Carson ReynoldsAnalytic framework for ethical comparison

TR2006-005: Worse is Better for Ambient Sensing, Pervasive 2006 Workshop on Privacy, Trust and

Identity Issues for Ambient Intelligence

toward scalable activity recognition for sensor networks

Documents