Sistemi rinconfigurabili a basso consumo per il monitoraggio distribuito

Tutor: prof. Luca Benini

Michele Magno

Outline

Wireless Video sensor NodeWireless Video sensor Network, WVNPower issueEnergy HarvestingPIR SensorMultomodal low cost systemApplication of video surveillanceResource manager for embedded computer

Wireless Video Sensor Network (WVSN)WSN with cameras for ambient monitoring

UnobtrusiveNo cables (remote locations)

Power availability issueBattery poweredSustainable operation

ProcessingWireless communication

Multimodal surveillance networksAugment informationReduce power consumption

Energy harvesting techniques to extend node lifetimeExtend nodes lifetime

Power issueEnergy consumption issue

Image acquisition and processingWireless transmission of recorded transmission

Several techniques can be used to extend network lifetime

Multimodal surveillanceEnergy harvestingLocal processing of imagesResource Manger

Energy efficient

Multimodal sensing

Energy harvesting

Local computation

Energy efficient Wireless Video Sensor NodeCombines three modalities

A Pyroelectric InfraRed (PIR) sensor provides a low power wake up signal. Sensor sensibility is modulated according to available energy and image contrast.

A PhotoVoltaic (PV) module is used to harvest energy from the environment.

Images are locally classified using Support Vector Machines (SVM) to detect relevant event and limit wireless transmission of undesired images

Wireless Video Sensor Node

Samrt Camera =Camera + intelligenceThe basis for new applications

Such as: detection, tracking, scene analysis

APPLICATIONS:• Automotive• Mobile Comm.• Surveillance• Consumer

Wireless Video Sensor Node –

(II)

Low costLow power consumptionLocal "intelligence" (on-board image

processing capability)A reconfigurable architecture to achieve

flexibilityWireless connectivity

Increase battery live

Wireless Video Sensor Node –

(III) Processing unit

Technology

Processor approach:DSP,

Media Processor,GPU,ARM

Programmable Logicapproach:

CPLD,FPGA,

Application

with

smart

camera People Detection

MICROCurrent Frame

Background Subtraction Winows Transfer Feature

ExtractionEmbedded Support

Vector Machine(ERSVM)

Background

FPGA

Video Processing Block

Application –

People detection IIARM9 – STR912

Three frame difference: background subtraction Region of Interest (ROI) detection (128x64 pixels)Feature extraction: average gray value over column and row (192 features)Classification: SVM-like hardware oriented algorithm

Sensor conditioning circuits: band pass filter (0.2-12Hz) + amplifier (1500x) + trigger generator

Low power: 1.5mWDetects changes in incident IR radiation

Moving bodiesPIR output dependencies

SpeedTemperatureDistance

Sensors PIR

+

+Out

Vdd

GND

Sensitive elements

Lens array Incident radiation

Absorbing structure

Architecture video sensor node + PIR

Typical use of PIR Sensor

VIDEO PROCESSING

Wireless Transceiver

2.4GHz

SLEEP MODE

Typical scenario

Application III –

abandoned /removed object detection with PIR Sensor

Active without video sensor

NO EVENTS!Sleep mode, only

PIR active

… …

EMPTY SCENE PIR->START

VIDEO PROCESSING

Application IV –

ZigBee WSN PIR- CAMERA

• Zibgee comunication• Scalable topology• Interaction Pir sensors and cameras motes

Application IV –

ZigBee WSN PIR- CAMERA

Power consumption measurementExtended battery life Node coopeation

Energy Harvesting & MPC ( ETH-

ZURICH)

• Model Predictive Control for rates and other parameters• Distributed MPC• Control the PIR sensitivity• Advance Multimodal video surveillance system

Resource Manager for embedded system (SCALOPES EU PRPJECT & STM)

Hw Design & Implementation

Thermal sensorPower IslandsClock gatingClock scaling

Sw DesignPorting of the Linux RM layer on SPEAr platformAdding RM feature to the IPs driver

SPEAR

ARCH-DRIVER

Normal Mode

Slow Mode

Doze Mode

Sleep Mode

Freq. of Cores

Freq of DDR

Freq of Perif

CPUIDLE CPUFREQACPI-DRIVER

RM-DRIVER

GOVERNORS

GENERIC INFRASTRUCTURE

SPECIFIC SPEAR GOVERNOS

USER LEVEL INTERFACES

APPLICATIONS/SURVEILLANCE APPLICATION

•/sys/devices/system/cpu/cpuidle

•/sys/devices/system/cpu/cpufreq

• Objective: A Linux-based RM infrastructure for the configurable SPEAR platform (multi-core processor + domain specific configurable accelerator)

CollaborationsPROJECTS

EU PROJECT: Scalopes ‘09-’10 SensactionAAL ‘08 – ‘09

National PROJECT:SUMMIT ‘07-’08PRIIN

Industrial STMicroelectronics ( Microcontroller, Vision, SCALOPES)Datalogic ( wireless applications, SUMMIT)

AccademicETH – Zurich ( 6 months intership) (MPC & Energy harvesting)EPFL (Andrea Acquaviva - reconfigurable architecture )University of Trento ( Support Vector Machine)PoliMI ( SCALOPES porjects)PoliTO ( SCALOPES porjects)

PublicationJournals:

Transactions on HiPEAC-1, Lecture Notes in Computer Science (LNCS), Springer-Verlag Berling Heidelberg New York Titolo: Exploration of Reconfiguration Strategies forEnvironmentally Powered DevicesAutori: Alex E. Susu, Michele Magno, Andrea Acquaviva, David Atienza, Giovanni De MicheliRivista: Journal of Real-Time Image Processing Titolo: A low-power wireless video sensornode for distributed object detectionAutori: Aliaksei Kerhet, Michele Magno, Francesco Leonardi, Andrea Boni and Luca Benini

Conferences:Congresso: Advanced Video and Signal based Surveillance Titolo: Distributed Video Surveillance Using Hardware-Friendly Sparse Large Margin Classifiers Autori: AliakseiKerhet, Francesco Leonardi, Andrea Boni, Paolo Lombardo, Michele Magno, Luca BeniniLuogo: Londra Data: Settembre 2007Congresso: 11th EUROMICRO Conference on Digital System Design Titolo: A Solar-powered Video Sensor Node for Energy Efficient Multimodal SurveillanceAutori: Magno, Michele; Brunelli, Davide; Zappi, Piero; Benini, LucaLuogo: ParmaData: Settembre 2008Congresso: Workshop on Multi-camera and Multi-modal Sensor Fusion Algorithms and Applications (M2SFA2) Titolo: Multi-modal Video Surveillance aided by Pyroelectric InfraredSensors Autori: Michele Magno; Federico Tombari; Davide Brunelli; Luigi Di Stefano; Luca Benini Luogo: Marsiglia Data: Ottobre 2008

Publication

(II)

Congresso: Sixth IEEE International Conference on Advanced Video and Signal BasedSurveillance Titolo: Multimodal Abandoned/Removed Object Detection for Low Power Video Surveillance SystemsAutori: Michele Magno; Federico Tombari; Davide Brunelli; Luigi Di Stefano; Luca BeniniLuogo: GenovaData: Settembre 2009Congresso: Third ACM/IEEE International Conference on Distributed Smart Cameras(ICDSC 2009) Titolo: Adaptive Power Control for Solar Harvesting Multimodal Wireless Smart Camera Autori: Michele Magno, Davide Brunelli, Lothar Thiele and Luca Benini Luogo: Como Data: Agosto 2009

POSTERS:Congresso: International Conference on Distributed Smart Cameras Titolo: A Low-PowerConfigurable Wireless Video Sensor Node for Distributed Vision Applications Autori: MAGNO M., L. BENINILuogo: Vienn aData: Settembre 2007Congresso: Fifth European conference on Wireless Sensor Networks, EWSN 2008 Titolo: A Self-powered Video Node Triggered by PIR Sensors Autori: M. Magno, D. Brunelli, P. Zappi and L. Benini Luogo: Bologna Data: Gennaio 2008Congresso: 6th European Conference on Wireless Sensor Networks Titolo: Detection ofabandoned/removed objects with a video sensor node aided by Infrared Sensor Autori: Michele Magno, Davide Brunelli, Luca Benini Luogo: Cork, Ireland Data: Febbraio 2009

Publication

(III) –

Work in progress

Congresso: The 9th ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN2010) Titolo Energy Efficient Cooperative Multimodal Ambient MonitoringAutori: Magno M., Zappi P., Brunelli D., L. BENINI STATUS: Submitted

TITOLO: Energy aware multimodal video surveillance embedded system. Autori: Magno M., Lanza A. Brunelli D., Di Stefano L., Benini L.

TITOLO: Resource manager for video surveillance embedded system. Autori: Magno M., Brunelli D., Benini L.

Looking for Transactions of Embedded computer and Embedded video and data signal processing

Conclusion and Future WorkEnergy efficient wireless video node

Multimodal surveillanceLocal preprocessing

Multi sensor node for distributed surveillanceVideo processing applicationData fusion for more energy efficiency

Energy Harvesting video sensor nodeResource managerFuture work

Augment information from low-power sensorsDistributed policies for collaborative surveillanceNew vision algorithms