classification and clustering of brain pathologies from motion data of patients in a virtual reality...
TRANSCRIPT
Classification and Clustering of Brain Pathologies from Motion
Data of Patientsin a Virtual Reality Environment
Via Machine Learning
Uri Feintuch, Hadassah- Hebrew University Medical Center
Larry Manevitz, University of Haifa,
Natan Silnitsky, University of Haifa
Data from: Assaf Dvorkin, Northwestern University
BISFAI, June 2011
Brain pathologies
• CVA - CerebroVascular Accident (Stroke)
– Hemispatial neglect
• TBI - Traumatic Brain Injury
Rehabilitation
• Diagnosis– Differential Diagnosis (e.g., Neglect vs.
Hemianopsia)
• Evaluation– Severity of deficit– Progress during intervention
(Deouell, Sacher & Soroker, 2005)
• HD applied for and received back his driver’s license, having shown intact visual fields at Perimetry and no signs of neglect.
• HD scored 143/146 on his BIT test.
• Since obtaining the license, however, he was involved in 9 car accidents, all concerning the left side of his car.
and their shortcomings…
VR in Rehab (1)
• Virtual Mall
A camera which films the patient and a monitor which displays her
The way the patient views himself within the virtual environment
VR in Rehab
• Replaces traditional methods
• Ecological validity
• Safety
• Absolute control of stimuli
VR in Rehab (2)
• Assaf Dvorkin, Rehabilitation Institute of Chicago, Northwestern Uni.
The VRROOM haptics/graphic system A target in the field of view
VR in Rehab - Challenges
• Human behavior is very complicated
• Vast amount of information
• geometry or physics formula (?)
• Simplistic analysis (e.g., RT, % Errors)
Research Goals
• Identify and differentiate between meaningful clinical conditions – Scarce data– Perhaps noisy
• Broad spectrum conditions like neglect– Mild, severe– Use unsupervised learning approach
Machine Learning Techniques Used
• Supervised Learning– Backpropogation NN– SVM
• Unsupervised Learning– Kohonen– K-means
2D Experiment
• Population: 54 HA, 11 CVA (without neglect), 9 TBI, 25 HC.
• Data Encoding: Vector of hand movement (dx,dy,dt)
NN Architecture for 2D
dx dy dt dx dy dt dx dy dt dx dy dt dx dy dt
…(Full connectivity)…
Data point (t) Data point (t+1) Data point (t+2) Data point (t+3) Data point (t+4)
Input Layer
Hidden Layer
Output Layer
NN Architecture for 2D (TBI vs. CVA)
dx
…(Full connectivity)…
Data point (t) Data point (t+1) Data point (t+2) Data point (t+3) Data point (t+4)
Input Layer
Output Layer
dy dt dx dy dt dx dy dt dx dy dt dx dy dt
…(Full connectivity)…
2D Results – 2 class
PopulationsBP NN
Average Success(by subjects)
Healthy/CVA90%
Healthy/TBI100%
TBI/CVA97%
3D Experiment
• Population: 9 H, 9 N, 10 S• Data Encoding: Vector of movement
(x,y,z,t)• Only trials where movement occurred at all• Phases
– Long vector: Entire trial from appearance of stimulus (includes pre-movement data)
– Movement: Vector only from commencement of movement
– Initial/Final segment – beginning/end of movement
NN Architecture for 3D
• 1400 elements for a long vector (1400-5-1)
• 1000 elements for a movement vector (1000-5-1)
• 130 elements for initial/final vectors (130-5-1)
3D Data Set
• Population of Healthy, Neglect, Stroke
• Movement Vectors (x,y,z,t) of different lengths
• Also tested on “snippets” for cross platform
• Resilient back-propagation
• 50 to 300 epochs
3D Results – 2 classVector sizePopulations BP NN Success Rate
By Patient and
By Data Point Including Reaction Time (470 time steps)
Healthy/CVA78%
62% (4010/6458)
Healthy/Neglect89%
79% (4557/5791)
Neglect/CVA72%
63% (3584/5703)
From Start of Motion
(330 time steps)
Healthy/CVA78%
68% (4378/6458)
Healthy/Neglect89%
78% (4515/5791)
Neglect/CVA72%
66% (3737/5703)
3D Results – 2 classVector sizePopulations BP NN Success Rate
By Patient and
By Data Point Small initial segment
(43 time steps)
Healthy/CVA39%
43% (2791/6458)
Healthy/Neglect83%
70% (4058/5791)
Neglect/CVA78%
61% (3470/5703)
Small final segment
(43 time steps)
Healthy/CVA67%
53% (3435/6458)
Healthy/Neglect89%
61% (3547/5791)
Neglect/CVA61%
52% (2962/5703)
3D expriment - 1 class
• Architecture
• Movement vectors – 1000-200-1000 • initial/final vectors - 130-26-130
3D expriment - 1 class
• Architecture
• Movement vectors – 1000-200-1000 • initial/final vectors - 130-26-130
1 class results for 3D
Vector sizeTraining Set Population
Compression NNAverage Success
From Start of Motion
(330 time steps) Healthy 93%
CVA 69%
Neglect 50% Non-Mild Neglect76%
Severe Neglect 100%
1 class results for 3D
• Neglect classifier for "Left targets trials only" - 62%
• Non-Mild Neglect classifier for "Left targets trials only" - 83%
Combined Platforms – 2 class
Vector sizeTraining Set Origin
BP NNAverage Success
30 time steps VRROOM and GestureXtreme
75%
90 time steps 90%
VRROOM only 50%
GestureXtreme only
50%
Summary of results
• 2D experiment – Differential Diagnosis:– Healthy vs. Patients– TBI vs. CVA
• 3D experiment – DD + Evaluation – Neglect vs CVA– Clusters by severity– 1 class classifiers (Severe Neglect)
Future Work
• Merging data across platforms
• Automatic Prognosis and Individualized Treatment Protocols
– Construct models of patients with their movement restrictions
– Run potential rehab protocols on the model– Prognosis: via best results on model– Apply best protocol to the patient