![Page 1: Vision-based monitoring of people in private spaces](https://reader034.vdocument.in/reader034/viewer/2022052601/559725a31a28ab140e8b464b/html5/thumbnails/1.jpg)
Francisco Flórez-Revuelta Digital Imaging Research Centre
Vision-based monitoring of people in private spaces
![Page 2: Vision-based monitoring of people in private spaces](https://reader034.vdocument.in/reader034/viewer/2022052601/559725a31a28ab140e8b464b/html5/thumbnails/2.jpg)
Presentation Architecture Human action recognition Optimisation Privacy by the context Discussion
Overview
![Page 3: Vision-based monitoring of people in private spaces](https://reader034.vdocument.in/reader034/viewer/2022052601/559725a31a28ab140e8b464b/html5/thumbnails/3.jpg)
Associate Professor (now on leave) at the University of Alicante (Spain) Member of the "Domotics and Ambient Intelligence" research group Multi-disciplinary research: computer vision, computational intelligence (artificial neural networks, evolutionary computation), home automation… Research on ICT for smart environments (services at home, system integration, efficient use of energy, ambient assisted living, person-environment interaction…) Development of person-environment interfaces for the interaction of people with their environment, paying particular attention to the support of the independent living of elderly and disabled people
My background
![Page 4: Vision-based monitoring of people in private spaces](https://reader034.vdocument.in/reader034/viewer/2022052601/559725a31a28ab140e8b464b/html5/thumbnails/4.jpg)
![Page 5: Vision-based monitoring of people in private spaces](https://reader034.vdocument.in/reader034/viewer/2022052601/559725a31a28ab140e8b464b/html5/thumbnails/5.jpg)
Joined Kingston University in 2011 as Marie Curie fellow The Robot Vision Team at the Digital Imaging Research Centre had a huge experience on video-surveillance Combine their expertise on video-surveillance with mine on ambient assisted living
My background
![Page 6: Vision-based monitoring of people in private spaces](https://reader034.vdocument.in/reader034/viewer/2022052601/559725a31a28ab140e8b464b/html5/thumbnails/6.jpg)
A study on people activity in private spaces: towards a multisensor network that meets privacy requirements Project funded by the European Commission – 2011-2013
Objectives Study, design and development of a distributed vision system for human motion analysis in a private environment Development of appropriate techniques for maintaining the privacy of the individuals being monitored Integration of the vision system with other sensors forming a perceptual, unobtrusive and interoperable network Integration and validation of the system in a smart home environment
![Page 7: Vision-based monitoring of people in private spaces](https://reader034.vdocument.in/reader034/viewer/2022052601/559725a31a28ab140e8b464b/html5/thumbnails/7.jpg)
Senior Researcher since 2013 Strengthen research on ambient assisted living and intelligent environments at Kingston University
Current position
![Page 8: Vision-based monitoring of people in private spaces](https://reader034.vdocument.in/reader034/viewer/2022052601/559725a31a28ab140e8b464b/html5/thumbnails/8.jpg)
Platform for self-assessment and efficient management for informal caregivers Project funded by the European Commission Ambient Assisted Living Joint Programme (call 5) – 2013-2015
Activities and events to be detected: Recognition of activities of daily living (ADL): wake-up, going to the toilet, sleeping… Activity level: light, moderate, intensive Location and tracking of the person inside the home Risky situations: fall detection, dangerous objects; fire, flood, gas alarms… Social activity: leaving home, receiving visits, use of telephone… Detection of basic and advanced activities: cooking, watching TV…
![Page 9: Vision-based monitoring of people in private spaces](https://reader034.vdocument.in/reader034/viewer/2022052601/559725a31a28ab140e8b464b/html5/thumbnails/9.jpg)
Architecture
Camera 1
Motion Detection
Human Behaviour Analysis
Multi-view Human Behaviour Analysis
Privacy
Reasoning System
Alarm Actuators
Camera 2
Motion Detection
Human Behaviour Analysis
Camera N
Motion Detection
Human Behaviour Analysis
...
Setup and Profiles DB (Activities, Inhabitants, Objects, ...)
Log
Event
Long-term analysis
...
...
Environmental Sensor Information
Caregiver
![Page 10: Vision-based monitoring of people in private spaces](https://reader034.vdocument.in/reader034/viewer/2022052601/559725a31a28ab140e8b464b/html5/thumbnails/10.jpg)
Human behaviour analysis
![Page 11: Vision-based monitoring of people in private spaces](https://reader034.vdocument.in/reader034/viewer/2022052601/559725a31a28ab140e8b464b/html5/thumbnails/11.jpg)
Feature extraction
![Page 12: Vision-based monitoring of people in private spaces](https://reader034.vdocument.in/reader034/viewer/2022052601/559725a31a28ab140e8b464b/html5/thumbnails/12.jpg)
Bag of key poses
![Page 13: Vision-based monitoring of people in private spaces](https://reader034.vdocument.in/reader034/viewer/2022052601/559725a31a28ab140e8b464b/html5/thumbnails/13.jpg)
Multiple views
![Page 14: Vision-based monitoring of people in private spaces](https://reader034.vdocument.in/reader034/viewer/2022052601/559725a31a28ab140e8b464b/html5/thumbnails/14.jpg)
Results
Weizmann MuHAVi-‐8
MuHAVi-‐14
IXMAS
![Page 15: Vision-based monitoring of people in private spaces](https://reader034.vdocument.in/reader034/viewer/2022052601/559725a31a28ab140e8b464b/html5/thumbnails/15.jpg)
Use of RGB-D cameras
![Page 16: Vision-based monitoring of people in private spaces](https://reader034.vdocument.in/reader034/viewer/2022052601/559725a31a28ab140e8b464b/html5/thumbnails/16.jpg)
Optimisation
We need to recognise the action/activity in order to apply an appropiate privacy filter A high success rate is compulsory
![Page 17: Vision-based monitoring of people in private spaces](https://reader034.vdocument.in/reader034/viewer/2022052601/559725a31a28ab140e8b464b/html5/thumbnails/17.jpg)
Evolutionary feature subset selection
Some features are not needed or they introduce noise Use of an evolutionary algorithm as a wrapper Chromosomes are encoded as binary vectors, each gene representing whether a feature is or is not used for recognition
![Page 18: Vision-based monitoring of people in private spaces](https://reader034.vdocument.in/reader034/viewer/2022052601/559725a31a28ab140e8b464b/html5/thumbnails/18.jpg)
Results
Weizmann MuHAVi-‐8
MuHAVi-‐14
DHA
IXMAS
![Page 19: Vision-based monitoring of people in private spaces](https://reader034.vdocument.in/reader034/viewer/2022052601/559725a31a28ab140e8b464b/html5/thumbnails/19.jpg)
Evolutionary feature subset selection
Chromosome
Normalisa=on
Crossover
![Page 20: Vision-based monitoring of people in private spaces](https://reader034.vdocument.in/reader034/viewer/2022052601/559725a31a28ab140e8b464b/html5/thumbnails/20.jpg)
Results
![Page 21: Vision-based monitoring of people in private spaces](https://reader034.vdocument.in/reader034/viewer/2022052601/559725a31a28ab140e8b464b/html5/thumbnails/21.jpg)
Results
![Page 22: Vision-based monitoring of people in private spaces](https://reader034.vdocument.in/reader034/viewer/2022052601/559725a31a28ab140e8b464b/html5/thumbnails/22.jpg)
Cooperative coevolutionary optimisation
Simultaneous selection of the training set, feature set and parameters of the clustering algorithm to improve the success rate
![Page 23: Vision-based monitoring of people in private spaces](https://reader034.vdocument.in/reader034/viewer/2022052601/559725a31a28ab140e8b464b/html5/thumbnails/23.jpg)
Cooperative coevolutionary optimisation
![Page 24: Vision-based monitoring of people in private spaces](https://reader034.vdocument.in/reader034/viewer/2022052601/559725a31a28ab140e8b464b/html5/thumbnails/24.jpg)
Cooperative coevolutionary optimisation
![Page 25: Vision-based monitoring of people in private spaces](https://reader034.vdocument.in/reader034/viewer/2022052601/559725a31a28ab140e8b464b/html5/thumbnails/25.jpg)
Cooperative coevolutionary optimisation
![Page 26: Vision-based monitoring of people in private spaces](https://reader034.vdocument.in/reader034/viewer/2022052601/559725a31a28ab140e8b464b/html5/thumbnails/26.jpg)
Privacy preservation
Privacy by the context Context:
identity of the subject appearance location on-going activity identity of the observer
![Page 27: Vision-based monitoring of people in private spaces](https://reader034.vdocument.in/reader034/viewer/2022052601/559725a31a28ab140e8b464b/html5/thumbnails/27.jpg)
Privacy filters
![Page 28: Vision-based monitoring of people in private spaces](https://reader034.vdocument.in/reader034/viewer/2022052601/559725a31a28ab140e8b464b/html5/thumbnails/28.jpg)
Current work
Fusion of multiple vision devices: multiple RGB-D devices, RGB and RGB-D cameras… Feature selection from a large bank of features Incremental and adaptive learning Continuous activity recognition Validation in real scenarios Monitoring people with all kind of sensors, fusion with environmental sensors Anything related to intelligent environments
![Page 29: Vision-based monitoring of people in private spaces](https://reader034.vdocument.in/reader034/viewer/2022052601/559725a31a28ab140e8b464b/html5/thumbnails/29.jpg)
Papers
Human activity recognition Chaaraoui, A.A.; Padilla-López, J.R.; Ferrández-Pastor, F.J.; Nieto-Hidalgo, M.; Flórez-Revuelta, F.: A Vision-based System for Intelligent Monitoring: Human Behaviour Analysis and Privacy by Context, Sensors, 14(5):8895-8925, 2014. Chaaraoui, A.A.; Flórez-Revuelta, F.: Adaptive Human Action Recognition With an Evolving Bag of Key Poses, IEEE Transactions on Autonomous Mental Development (accepted for publication) Chaaraoui, A.A.; Padilla-López, J.R.; Flórez-Revuelta, F.: Fusion of Skeletal and Silhouette-based Features for Human Action Recognition with RGB-D Devices, 3rd Workshop on Consumer Depth Cameras for Computer Vision (CDC4CV), ICCV 2013 Workshop, Sydney (Australia), 2013. Chaaraoui, A.A.; Climent-Pérez, P.; Flórez-Revuelta, F.: Silhouette-based Human Action Recognition using Sequences of Key Poses, Pattern Recognition Letters, 34(15):1799–1807, 2013. Chaaraoui, A.A.; Climent-Pérez, P.; Flórez-Revuelta, F.: An Efficient Approach for Multi-view Human Action Recognition Based on Bag-of-Key-Poses, Lecture Notes in Computer Science, 7559:29-40, 2012. Chaaraoui, A.A.; Climent-Pérez, P.; Flórez-Revuelta, F., A review on vision techniques applied to Human Behaviour Analysis for Ambient-Assisted Living. Expert Systems with Applications, 39(12): 10873-10888, 2012.
![Page 30: Vision-based monitoring of people in private spaces](https://reader034.vdocument.in/reader034/viewer/2022052601/559725a31a28ab140e8b464b/html5/thumbnails/30.jpg)
Papers
Evolutionary optimisation Chaaraoui, A.A.; Padilla-López, J.R.; Climent-Pérez, P.; Flórez-Revuelta, F.: Evolutionary joint selection to improve human action recognition with RGB-D devices, Expert Systems with Applications, 41(3):786-794, 2014. Chaaraoui, A.A.; Flórez-Revuelta, F.: Optimizing human action recognition based on a cooperative coevolutionary algorithm, Engineering Applications of Artificial Intelligence, Volume 31:116–125, 2014. Chaaraoui, A.A.; Flórez-Revuelta, F.: Human action recognition optimization based on evolutionary feature subset selection, Proceeding of the 15th Annual Genetic and Evolutionary Computation Conference, Amsterdam (The Netherlands), 2013. Climent-Pérez, P.; Chaaraoui, A.A.; Padilla-López, J.R.; Flórez-Revuelta, F.: Optimal Feature Selection for Skeletal Data from RGB-D Devices using a Genetic Algorithm, 11th Mexican International Conference on Artificial Intelligence, San Luis Potosí (Mexico), 2012.
![Page 31: Vision-based monitoring of people in private spaces](https://reader034.vdocument.in/reader034/viewer/2022052601/559725a31a28ab140e8b464b/html5/thumbnails/31.jpg)
Papers
Privacy preservation Padilla-López, J.R.; Chaaraoui, A.A.; Flórez-Revuelta, F.: Visual privacy by context: A level-based visualisation scheme (submitted to conference) Padilla-López, J.R.; Chaaraoui, A.A.; Flórez-Revuelta, F.: Visual privacy protection methods: a survey (submitted to journal) Padilla-López, J.R.; Flórez-Revuelta, F.; Monekosso, D.N.; Remagnino, P.: The "Good" Brother: Monitoring People Activity in Private Spaces, International Symposium on Distributed Computing and Artificial Intelligence (DCAI 2012), Salamanca (Spain), pp. 49-56, 2012.
![Page 32: Vision-based monitoring of people in private spaces](https://reader034.vdocument.in/reader034/viewer/2022052601/559725a31a28ab140e8b464b/html5/thumbnails/32.jpg)
Papers
Home automation and intelligent environments Ferrández-Pastor, F.J.; García-Chamizo, J.M.; Romacho-Agud, V.; Flórez-Revuelta, F.: Using the wavelet transform to disaggregate electrical power consumption into the major end-uses (submitted to journal) Ferrández-Pastor, F.J.; Nieto-Hidalgo, M.; García-Chamizo, J.M.; Flórez-Revuelta, F.: DAI Virtual Lab: a Virtual Laboratory for Testing Ambient Intelligence Digital Services, V Congreso Internacional de Diseño, Redes de Investigación y Tecnología para Todos (DRT4ALL 2013), Madrid (Spain), 2013. Climent-Pérez, P.; Chaaraoui, A.A.; Flórez-Revuelta, F.: Useful Research Tools for Human Behaviour Understanding in the Context of Ambient Assisted Living, International Symposium on Ambient Intelligence (ISAmI 2012), Salamanca (Spain), pp. 201-205, 2012. Nieto-Hidalgo, M.; Limiñana-Benito, A.; Torregrosa-Jorqués, R.; Padilla-López, J.R.; Cabo-Díez, M.; Ferrández-Pastor, F.J.; Romacho-Agud, V.; Climent-Pérez, P.; Chaaraoui, A.; Flórez-Revuelta, F.: DAI Virtual Lab: a virtual lab for the co-design, development and validation of AmI services and technologies, International Joint Conference on Ambient Intelligence (AmI-11), Amsterdam (Nederlands), 2011.
![Page 33: Vision-based monitoring of people in private spaces](https://reader034.vdocument.in/reader034/viewer/2022052601/559725a31a28ab140e8b464b/html5/thumbnails/33.jpg)
Francisco (Paco) Flórez-Revuelta [email protected] @fflorezrevuelta www.dtic.ua.es/~florez franciscoflorezrevuelta