Scaling to several dancers…

High Speed Sensor FusionVocabulary of features

Capacitive proximity to 50 cm6-axis IMU - 1 Mbps TDMA radio100 Hz Full State Updates from 25

nodes

Capacitive Sensing System

• Capacitive system measures relative spacing between nodes

• Employs a “transmit mode” configuration

• One node transmits a sinusoidal pulse at 90kHz, others measure amplitude of received pulse

• Nodes trade roles as transmitters and receivers as arbitrated by the wireless basestation

• Sensitive up to 0.5 meter with bracelet-sized electrode and shared ground through the body

Typical Capacitive Sensor Response With Strap Electrode and Body Grounded

0

500

1000

1500

2000

2500

3000

3500

4000

4500

0 10 20 30 40 50 60

Spacing (cm)

Raw Sensor Values

New Developments

• Full deployment at a rehearsal with 20 sensor nodes running on five dancers

• Feature extraction and mapping algorithms which are fed logged data at the sample rate to simulate real time operation

QuickTime™ and aSorenson Video 3 decompressorare needed to see this picture.

Rough Demo Mapping

Direct Features

Intuitive Features

Musical Parameters

• Audio rendered in real-time as features are calculated

• Runs on a 1.6 GHz G5 w. 1GB RAM

The Disposable Wireless Sensors

• Very simple motion sensor– Cantilevered PVDF piezo strip with proof mass– Activates CMOS dual monostable when jerked– Sends brief (50 s) pulse of 300 MHz RF– 100 ms dead timer prevents multipulsing– Can zone to within ~10 meters via amplitude– Ultra low power – battery lasts up to shelf life– Extremely cheap – e.g., under $1.00 in large quantity

6

JAP7/05

Interactive Raves at MIT

Activity (# hits per 10 seconds) FFT (Tempo)

Device Details – Camera System

Deployed to cover entire building (>100 nodes)

3MP CameraMotorized Panning and

FocusDedicated Video DSP/ARM - (TI Davinci chip)Real-time Linux OSLCD displays what is being

captured at all timesContains Spinner

Gateway/Sensor board (detailed on next slide)

Device Details – Spinner GatewayWorks with or without

camera boardCommunicates with

wearable/mobile devices in mesh network

Serves as reference beacon for location system

EthernetAudio system with DSP

AVR32

Environmental SensingMotionHumidity and TemperatureLightInfrared Communication

and Detection/Proximity (talks to all our badge systems)

Device Details – Spinner Social SensorWearable on collar or as

pin/badgeAudio system with DSP for

analytics and CD quality recording

Compass for orientation3-axis AccelerometerIR communication and line of

sight detection/proximityLocation engineCaptures social signal and

group dynamics

Device Details – Spinner Wrist SensorWrist worn device3-axis AccelerometerGalvanic Skin Response (GSR) SensorLocation engineUI for interacting with networkStores and plays videos, providing

ownership of video to end userCaptures gesture and indications of

affective state