mobile sensing kolkata lab tac_tics2014
TRANSCRIPT
1Copyright © 2011 Tata Consultancy Services
Limited
Remote sensing and surveillance using mobile phone sensors and ground-robot mounted sensors
Dr. Arpan Pal
Principal Scientist and Research Head
Innovation Lab, Cyber physical Systems
Tata Consultancy Services
With Aniruddha Sinha and Ranjan Dasgupta, Innovation Lab, Kolkata
2
Sensing the Physical World - on the ground
Mobile phone based crowd sensing
Robot assisted sensing
www.popularmechanics.com
www.engadget.com
www.allthingssd.com
apollo2.cs.illinois.edu
Mobile Phone based Crowd-sensing
Robot Assisted Sensing
4
Mobile Phones as Sensor and Gateway
InternetWiFi, 2G/GPRS, 3G
On-board sensorsAccelerometer, GPS,
Compass
Camera, Microphone,
Magnetometer
Internet
Sensors (Location, Audio , Camera, …)
Gateway (Stand-alone
Device / Mobile Phones)
Backend
Server & database
Portal
USB, CAN, Zigbee, BT,
NFC, WiFi
GSM/GPRS, 3GOther InterfacesBluetooth and USB
Internet2G/GPRS
On-board sensorsMicrophone, Camera
Basic Phone
Smart Phone
Other InterfacesBluetooth and USB
5
Phone Accelerometer based Road Condition Monitoring
Acceleration a(t) = f (Road, Vehicle, Driver)
1. “A Smart Transport Application of Cyber-Physical Systems: Road Surface Monitoring with mobile devices”, 6th
International Conference on Sensing Technology (ICST 2012)
2. “Low Computational Approach for Road Condition Monitoring Using Smartphones”, Computer Society of India (CSI)
Annual Convention, Theme: Intelligent Infrastructure, Jan 2013
3. “Road condition monitoring and alert application: Using in-vehicle Smartphone as Internet-connected sensor”,
PerCom Workshops 2012
6
Phone Microphone based Sound Scaping
Solution Overview
Event driven with participatory sensing aided audio surveillance system
• Classification of Traffic Noise (Honk Detection) and Crowd Noise
1. "Two Stage Feature Extraction using Modified MFCC for Honk Detection", CODIS 2012
2. "TrigSense: Accelerometer Triggered Audio sensing for Traffic Condition Monitoring", ISSNIP 2013
3. "Fusion of spectral and time domain features for crowd noise classification system", ISDA 2013,
7
Phone Camera based 3D Reconstruction from 2D images
Input Images
Dense Reconstruction without using mobile
inertial sensors- 20 images, compute time (4 core, 1GPU) ~ 20 min
- 120 images, compute time (16 core, 1GPU) ~ 30 min
- Bandwidth saving ~ 8 times, if done on mobile
Sparse Reconstruction using Mobile Inertial
Sensors for Camera Position Estimation
• 20 images, compute time (4 core, 1GPU) ~ 3 min
(without using inertial sensors)
• 20 images – compute time (4 core, 1GPU) ~10
sec. (with inertial sensors)
• Bandwidth saving ~ 200 times, if done on mobile
• Sparse good enough for many applications
• Two papers in process of submission in ICIP
and ECCV
• Dense Reconstruction with mobile inertial
sensors under progress with more number of
images
target < 1min
Dense Reconstruction
-120 images Dense Reconstruction
- 20 images
• Low cost solution for 3D reconstruction from multiple 2D images captured from mobile device.
• Motion information from the inbuilt inertial sensors – for camera position estimation
• Applications in Agro-advisory service, Remote Diagnostics, Remote Healthcare
8
Mobile phone based Connected Healthcare
Use CaseAffordable Healthcare for Chronic Disease / Elderly People Monitoring @Home
Physiological Sensing Using Mobile Phones
• Accelerometer and other inertial sensors• Activity Monitoring and Classification
• Camera on fingertip• Heart-rate, Respiratory Rate, ECG, HRV, BP, SpO2
• Microphone • Spirometry
• Camera on eye• Retinopathy and Pupilometry
Papers in Mobihoc, Sensys, ACM-SAC, Mobiquitous, ICASSP
Continuous Data Stream
Windowed Data
Zero Normalization
Linear Interpolation
Low Pass Filtration
Frequency Spectrum
Identifying non-activity window
using frequency spectrum
Peak Detection and Step
Validation using IPA;
calculating step cycle lengths for
all valid steps in the window
Classification of window
activity using step frequencies
derived from step cycle lengths
Mobile Phone based Crowd-sensing
Robot Assisted Sensing
10
Multi-sensor Fusion for Robot-assisted Sensing
Application in remote sensing in hazard-prone areas
• Robot carries 2D camera and heat / chemical sensors on a rotating arm
• 3D reconstruction from the 2D vision
• Estimation of Heat / gas leak / sound Source (direction and range) through passive directional
signal processing
• Fusion of heat / gas / sound source on reconstructed 3D vision map
www.ese.wustl.edu
Ongoing Work
Possible reuse from 2D-3D reconstruction and sound classification
Cloud point
from 3D
vision
Possible
gas / heat
source
(ROI)
Source
direction
and intensity
11
RIPSAC – TCS IoT Platform
Internet
End Users &
Renderers
Administrators
Device Integration & Management Services
Analytics Services
CPS Modelling
Application Services
Scalable Storage – Sensor Data and Metadata
Messaging & Event Distribution Services
Ap
plic
atio
n S
erv
ice
s
Presentation Services
Application Support Services
OS & Device Drivers
Sensor Signal Processing
HHTP / Enhanced CoAP
Mid
dle
ware
(S
ecurity
/Privacy a
nd Inte
ropera
tion)
Edge Gateway
Sensors
Internet
Back-end
Already Available – making it actuation-enabled is the next step
