potenzialità dell’intelligenza artificiale come · the biorobotics institute 2019 8 professors 3...
TRANSCRIPT
Potenzialità dell’intelligenza artificiale come
strumento di pre-screening di massa: applicazioni wearable nello studio del movimento
umano
Andrea Mannini, PhD
Statistics for Health and Well-beingUniversità di Brescia, Dip. Economia e Management
25 – 27 Settembre 2019
www.bioroboticsinstitute.eu
Pisa: a “melting pot” of students, professors and scientists
Source: MIUR, data collected oct 2011
National Institute
of Nuclear Physics
300 Researchers
National Research Council
15 Research Institutes
1.500 Researchers
1810
87
450
1343
1450
53.000
1987
103
800
Founding date
Faculty staff
Students
The BioRobotics Institute 2019
8 Professors
3 Associate
Professors
15 Assistant Professors (4 in tenure track)
9 Administrative Assistants
7 Technical Assistants
8 Tecnologists
75 Post Doc Research Assistants
90 PhD Students
250+ people
40+ % women
0
5
10
15
20
25
30
2011 2012 2013 2014 2015 2016 2017 2018 2019
Assistant Associate Professor Total
Volterra annual meeting July 5-7, 2018
About me
Biomedical Engineer (2009)
PhD in BioRobotics (2013)
Assistant professor in
Bioengineering at Scuola
Sant’Anna (2017)
Work on machine learning
enabled solutions for wearable
sensors (mainly inertial sensors
and electromyography)
AI-based preliminary-diagnosis using wearables
AI momentum
Why is AI living a new explosion?
High computational power
available in our pockets
Amazing outlooks for costs
reduction using AI apps
150 Billion$ saving thanks to top AI
applications by 2026 (Accenture)
AI-based preliminary-diagnosis using wearables
The Iron Triangle in healthcare
In healthcare management, trying to
improve one factor usually harms
another…
More affordable, less effective
More effective, less accessible
…The Iron
Triangle
Access
Cost
Quality
AI appears as the short-circuit solution:
It can cut costs, improve treatment
quality and bolster accessibility
AI-based preliminary-diagnosis using wearables
How is this possible?
Transferring time consumining
human tasks to machines
Enabling patients to self service
their care needs when possible
Objection 1: we are talking about
healthcare, I do not trust robots!
Objection 2: robot will steal our
jobs!
AI-based preliminary-diagnosis using wearables
Breaking the Iron Triangle
What is really changing right now?
Most of the short-term investments are
about the operational/administrative
side rather than the clinical one;
The keyword of current solutions is:
Data-driven products and services
i.e. use data and previous experience
to improve quality, personalization and
reduce final costs
AI in Healthcare
Robotic surgery is shifting
towards automatic surgery
Nursing assistance
Administrative and
technical optimizations
Diagnostics
AI-based preliminary-diagnosis using wearables
AI in Healthcare for diagnostic applications
AI diagnostics
Image based
Genomeinterpretation
Biomarkerdiscovery
Clinicaloutcomeprediction
Wearables
OftalmologyDermatology
RadiologyPathology
Identify gene mutations related to pathologies
Identify molecular patterns associated with
pathologies by merging information using ML
Predict hospital stay length,
readmission rate, mortality
AI for prevention: wearables
Movement (accelerometer,
gyroscope, magnetometer, GNSS)
Heart rate,
Respiratory rate,
Oxygen levels,
Blood pressure,
Body temperature,
Galvanic skin response, …
The vocabulary of measurements that
are possible via smart devices or
connected intrumentation is huge
AI-based preliminary-diagnosis using wearables
Providing healthcare
before you knew you
even need it!
Do that ecologically, with
no time/space restrictions
AI for prevention: wearables
Providing healthcare before
you knew you even need it!
We can track health status and
flag abnormal events
We can identify changes
Early detection in ecological
settings
AI-based preliminary-diagnosis using wearables
Evaluate the health
status ecologically
Wearables has no space limitations
Current golden standard requires
dedicated facilities and
instrumentation or invasive analysis
They are super accurate but do
they alter the quantity they are
measuring?
AI-based preliminary-diagnosis using wearables
Evaluate the health
status ecologically
Wearables has no time limitations
They can be used to induce and
track long term behavior changes
They can be used to evaluate
alterations, building on previous
«history»
World Health Organization: 60% of related factors to individual health and quality of life are
correlated to lifestyle choices, including taking prescriptions such as blood-pressure medications
correctly, getting exercise, and reducing stress.
AI-based preliminary-diagnosis using wearables
The Christmas tree effect
AI for prevention using wearables: limitations
The accuracy of wearable solutions
is often lower than that of classical
solutions
The user comfort is not always ideal,
and then the user’s adherence to
long term monitoring studies can
be limited
One third of US costumers who have owned wearable devices stopped using them within six
months of receiving them, foreshadowing the utility of the devices in fostering long term
behavioral change
[Source: Wearables are totally failing the people who need them most. Wired, June 2014]
AI for prevention using wearables
Search string «wearable AND prevention»
Numbers of studies targeting prevention using wearables are significantly
growing
AI-based preliminary-diagnosis using wearables
Let us see an example in detail…
Wearable sensor systems
Motion sensors
Computational methods for human motion analysis
Movement assessment
AI-based preliminary-diagnosis using wearables
Body Sensor
Networks
Activity
recognition
Activity
assessment
Alteration
recognition
A. Mannini, S. S. Intille, M. Rosenberger, A. M. Sabatini, W. Haskell, "Activity recognition using a single accelerometer placed at the
wrist or ankle", Medicine & Science in Sports & Exercise, vol. 45, no. 11, pp. 2193-2203, 2013.
Wrist Ambulation Cycling Other Sedentary
Ac
tua
l Ambulation 2121(87.2 %) 112 (4.6 %) 74 (3.0 %) 126 (5.2 %)
Cycling 86 (8.3 %) 650 (62.9 %) 16 (1.5 %) 281 (27.2 %)
Other 59 (6.2 %) 16 (1.7 %) 778 (81.6 %) 100 (10.5 %)
Sedentary 36 (1.2 %) 156 (5.2 %) 70 (2.3 %) 2722(91.2 %)
Overall accuracy = 84.7%
Ankle Ambulation Cycling Other Sedentary
Ac
tua
l
Ambulation 2481(99.5 %) 10 (0.4 %) 2 (0.1 %) 0 (0 %)
Cycling 13 (1.3 %) 975 (93.9 %) 21 (2.0 %) 29 (2.8 %)
Other 6 (0.6 %) 12 (1.2 %) 802 (81.6 %) 163 (16.6 %)
Sedentary 0 (0.0 %) 16 (0.5 %) 105 (3.5 %) 2879(96.0 %)
Overall accuracy = 95.0%
Ambulation Cycling Other activities Sedentary
walking. carrying-load
stairs: inside and down
stairs: inside and up
treadmill: 3 mph 0% incline
treadmill: 3 mph 6% incline
treadmill: 3 mph 9% incline
treadmill: 2 mph 0% incline
treadmill: 4 mph 0% incline
walking, natural
70rpm. 50W. 0.7kg
cycling outdoor level
cycling outdoor uphill
cycling outdoor downhill
painting: roller
painting: brush
sweeping with broom
sitting, internet search
sitting, computer typing
sitting: writing
sitting: reading
sorting files / paperwork
lying: on back
lying on left side
lying on-right-side
sitting: legs straight
standing still
Activity recognition
Features
evaluation
(time and
frequency
domain)
Classifiers
cross-
validation
(SVM +
leave-1-
subject-out)
Translate windowed accelerometer data in
activity information
2 h of annotated data for each participant
Leave-one-subject-out cross-validation
A. Mannini, M. Rosenberger, W. Haskell, A.M. Sabatini e S. S. Intille, Activity recognition in youth using a single accelerometer placed at the wrist or ankle, Medicine and Science in Sports and Exercise, 49 (4), 801-812, 2017.A. Mannini et al., A smartphone-based implementation based on the embedded accelerometer of this method is also available, Bodynets 2015.
It can be implemented on mobile devices
using their embedded sensors!Wrist activity recognition Accuracy
LOSO CV on Adults (original feature set) 84.7 🗸
LOSO CV on Youths (original feature set) 85.9 🗸
LOSO CV on Adults (new feature set) 87.0 🗸
LOSO CV on Youth (new feature set) 91.0 🗸
Crossed tests:
Training on Adults testing on Youths (original
feature set)
58.8 ✘
Training on Adults testing on Youths (new feature
set)
69.8 ✘
Training on Youths testing on Adults (original
feature set)
71.7 ✘
Training on Youths testing on Adults (new feature
set)
71.8 ✘
LOSO CV on Adults +Youths 88.5 🗸
(Youth data contribution only) 90.7 🗸
(Adult data contribution only) 86.7 🗸
Activity recognition
It works across different age groups
A. Mannini, V. Genovese and A. M. Sabatini, "Online decoding if hidden Markov models for gait event detection using foot-
mounted gyroscopes", IEEE Journal on Biomedical and Health Informatics, vol. 18, no. 4, pp. 1122-1130, 2014 .
It works on pathological gait too! (Mannini et al. IEEE-EMBC 2015)
Activity assessment: gait
Hidden Markov
Model (HMM)
Early detect gait alterations in everyday life
Healthy (EL) Post-stroke (PS) Choreic (HD)
Differentpathological
conditions alter differently the way
we move
A. Mannini, , O. Martinez-Manzanera, T.F. Lawerman, D. Trojaniello, U. Della Croce, D. Sival, N. Maurits and A.M. Sabatini, Automatic classification of gait in
children with early-onset ataxia or developmental coordination disorder and controls using inertial sensors, Gait & Posture, vol. 52, pp.287--292, 2017.
Early detect gait alterations in everyday life
Healthy (EL)Post-stroke (PS)
Choreic (HD)
Model #1, EL group Model #2, PS group Model #3, HD group
A. Mannini, , O. Martinez-Manzanera, T.F. Lawerman, D. Trojaniello, U. Della Croce, D. Sival, N. Maurits and A.M. Sabatini, Automatic classification of gait in
children with early-onset ataxia or developmental coordination disorder and controls using inertial sensors, Gait & Posture, vol. 52, pp.287--292, 2017.
Early detect gait alterations in everyday life
A. Mannini, , O. Martinez-Manzanera, T.F. Lawerman, D. Trojaniello, U. Della Croce, D. Sival, N. Maurits and A.M. Sabatini, Automatic classification of gait in
children with early-onset ataxia or developmental coordination disorder and controls using inertial sensors, Gait & Posture, vol. 52, pp.287--292, 2017.
Early detect gait alterations in everyday life
Values in the upper part of the plots refers to relevant alterations
By observing classification outputs in relation to clinical scales we see that
misclassified patients are mild PS patients or severe HD patients. No
misclassification between healthy and pathological occurs
Parkinson’s Disease (PD), (1)
Palmerini et al, IEEE Trans Neur Sys Rehab Eng, 2013
Salarian et al, IEEE Trans Neur Sys Rehab Eng, 2010
TUG test
Since 2010-2011: correlations between measured gait and signal
parameters and clinical scales
Gait and TUG-test has been studied (Salarian et al, Weiss et al.)
Studies used single sensor solutions (lower back) or even more
complex setup with up to 7 inertial measurement units
Since 2013: Proper classification studies including PD-altered gait
recognition obtained accuracies >80% in discriminating between
PD and controls
Such studies used a single sensor (lower back, Palmerini et al
2013) or two shoe-fixed sensors (Klucken et al. 2013)
Large dataset evaluated (173 subjects in Klucken et al. 2013)
Klucken et al, PlosOne, 2013
AI-based preliminary-diagnosis using wearables
Parkinson’s Disease (2)
Since 2012-2013: PD symptoms affecting gait were also studied.
Several methods to automatically recognize freezing of gait
and levodopa induced diskynesia were proposed
Studies adopted complex sensing setups
(6 sensors at wrists, waist, trunk and ankles)
Control populations were involved
very high accuracies in the recognition of
such symptoms were obtained (>88%)
The features adopted spanned from entropy
of signals to signal energy
Several automatic classifiers were compared in cross-
validation.
Tripoliti et al, Computer Methods
in Biomedicine 2013
AI-based preliminary-diagnosis using wearables
Hemiplegic patients (HP)
Recognizing hemiplegic gait from controls is a much
simpler problem due to the asymmetry and highly
altered gait in most HP patients
Scheffer et al (2012) proposed a complex approach
involving a full Xsens MTV system (17 IMUs)
28 HP patients and 30 CTRL
24 features from gait temporal parameters and range
of motion, including left-right cross-correlations
Artificial Neural Network classification (no cross-
validation reported)
>99% recognition accuracy
AI-based preliminary-diagnosis using wearables
Alzheimer’s Disease (AD)
A study by Hsu et al, (2014) started
the evaluation of variables extracted from
wearable sensors to identify AD symptoms
in gait automatically
A 3 IMUs system was proposed
21 AD and 50 CTRL tested
Features included postural sway parameters, gait temporal parameters and stride length
Tests included single and dual task gait (walking while counting down)
Most gait parameters differ across the two groups, especially in dual task gait
Hsu et al, IEEE J Biomed Health Inf,2014
AI-based preliminary-diagnosis using wearables
Cognitive decline, frailty and fatigue
Frailty: Greene et al (2014) observed 184 non-frail, 185 pre-frail and 30 frail elderly doing a TUG test and
designed a frailty classifier based on logistic regression using 44 features (temporal gait params,
spatial gait params, angular velocity parameters, "turn" params) and including gender, age, heigth,
weigth. The study involved a 10-fold cross validation and achieved an accuracy of 78%
Cognitive decline: TUG test was also adopted by the same group (Greene and Kenny 2012) to
discriminate cognitive declined and intact subjects. 16 declined and 120 intact subjects were
tested achieving good classification performances. Discriminant analysis was done and a cross-
validation was performed.
Fatigue: Zhang et al (2014) used shank and sternum IMUs to discriminate normal and fatigue walk on
17 subjects. Stride-by-stride classification. 96% accuracy obtained, without running a proper cross-
validation.
AI-based preliminary-diagnosis using wearables
Gait is the most important movement, it is related to main aspects of quality
of life and its assessment has main relevance for clinical reasons.
But we also want to have fun…
But… what about other movements?
AI-based preliminary-diagnosis using wearables
Yoga & sports assessment
Recent collaboration with
TuringSense company and
Università di Roma Foro Italico
IMUs to recognize and analyze
movements and postures in sports
Recognize specific Yoga postures to
reduce errors in kinematic
estimation
Identify skill-related features in the
sport gesture
Fatigue assessment in soccer players&referees
Data acquisition involved referee from the
Florence section in Coverciano, April 2019
Data are being processed to extract fatigue
predictors
Target exercises were the Counter-movement
Jump and Gait
Other ongoing projects:
PRIN project that will start in late 2019:
TRAINED: "mulTifeature analysis of heaRt rate
variability and gaIt features in cliNical Evaluation of
Depression“, budget 450 k€ in 3 years.
PNRM project that will start in late 2019:
WAVE: “Wearable Assistant for VEterans in sport",
budget 1.2 M€ in 3 years.
AI-based preliminary-diagnosis using wearables
So, keeping focused
on the main topic …
AI-based preliminary-diagnosis using wearables
“A total of 1183 patients (51% response rate) were
enrolled between May and June 2018. Overall, 20%
considered that the benefits of technology (e.g.,
improving the reactivity in care and reducing the
burden of treatment) greatly outweighed the dangers.
Only 3% of participants felt that negative aspects
(inadequate replacement of human intelligence, risks
of hacking and misuse of private patient data) greatly
outweighed potential benefits.”
“We found that 35% of patients would refuse to
integrate at least one existing or soon-to-be
available intervention using biometric monitoring
devices and AI-based tools in their care.”
AI-based preliminary-diagnosis using wearables
Conclusion
AI will be one of the keywords of
reserach, in healthcare too
(Horizon EU)
Wearables and personal devices
will play a significant role in our
daily monitoring for pre-diagnostics
Ethical concerns and user
perception are central
AI-based preliminary-diagnosis using wearables
Thank you!2029 ?