nick cercone vlado keselj calvin thomas computer science dalhousie university
DESCRIPTION
Analysis of Spontaneous Speech in Dementia of Alzheimer Type: Experiments with Morphological and Lexical Analysis. Nick Cercone Vlado Keselj Calvin Thomas Computer Science Dalhousie University. Kenneth Rockwood Medicine, Dalhousie University Elissa Asp English Deparment - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Nick Cercone Vlado Keselj Calvin Thomas Computer Science Dalhousie University](https://reader035.vdocument.in/reader035/viewer/2022062409/56815152550346895dbf74b9/html5/thumbnails/1.jpg)
1
Analysis of Spontaneous Speech in Dementia of Alzheimer Type: Experiments with Morphological and Lexical Analysis
Nick Cercone
Vlado Keselj
Calvin Thomas
Computer Science
Dalhousie University
PUL Workshop, Dalhousie University, Halifax, 23 Apr 2004
Kenneth Rockwood
Medicine, Dalhousie University
Elissa Asp
English Deparment
Saint Mary’s University
![Page 2: Nick Cercone Vlado Keselj Calvin Thomas Computer Science Dalhousie University](https://reader035.vdocument.in/reader035/viewer/2022062409/56815152550346895dbf74b9/html5/thumbnails/2.jpg)
2
Overview
Introduction Related work: Bucks et al, authorship
attribution CNG discrimination Pt/other rating dementia levels
use of attribute sets: MA-A, MA-B CNG and Ordinal CNG
Conclusion
![Page 3: Nick Cercone Vlado Keselj Calvin Thomas Computer Science Dalhousie University](https://reader035.vdocument.in/reader035/viewer/2022062409/56815152550346895dbf74b9/html5/thumbnails/3.jpg)
3
Introduction Effects of the Alzheimer’s disease (AD)
reduced communicative ability deterioration of linguistic performance
Can we detect it? Current methods rely on structured interviews
confrontation naming single word production word generation given context word generation given first letter picture description
![Page 4: Nick Cercone Vlado Keselj Calvin Thomas Computer Science Dalhousie University](https://reader035.vdocument.in/reader035/viewer/2022062409/56815152550346895dbf74b9/html5/thumbnails/4.jpg)
4
Analysis of spontaneous speech drawbacks of structured interviews:
sometimes insensitive to early signs of dementia observed by family
low scores are not reliable unless difficulty is observed in natural conversation
brake “natural speech” into components subjective, i.e., designed by a researcher
alternative solution: objective automatic analysis of spontaneous, i.e., natural, speech
![Page 5: Nick Cercone Vlado Keselj Calvin Thomas Computer Science Dalhousie University](https://reader035.vdocument.in/reader035/viewer/2022062409/56815152550346895dbf74b9/html5/thumbnails/5.jpg)
5
Speech characteristics in Dementia of Alzheimer Type (DAT) frequent use of functional words (closed
class) less rich vocabulary difficulty with constructing longer coherent
phrases more difficulties at lexical and morphological
level than phonetic and syntactic levels
![Page 6: Nick Cercone Vlado Keselj Calvin Thomas Computer Science Dalhousie University](https://reader035.vdocument.in/reader035/viewer/2022062409/56815152550346895dbf74b9/html5/thumbnails/6.jpg)
6
Related work: Bucks et al. (BSCW) Bucks, Singh, Cuerden, Wilcock 2000, 2001:
Analysis of spontaneous conversational speech in dementia of Alzheimer type (DAT)
use eight linguistic measures to analyze transcribed spontaneous speech:1) noun rate 5) clause-like semantic unit rate
2) pronoun rate (CSU)
3) verb rate 6) Brunet’s index (W)
4) adjective rate 7) token type ratio (TTR)
8) Honore’s statistic (R)
![Page 7: Nick Cercone Vlado Keselj Calvin Thomas Computer Science Dalhousie University](https://reader035.vdocument.in/reader035/viewer/2022062409/56815152550346895dbf74b9/html5/thumbnails/7.jpg)
7
Bucks et al.: Experiment design• experiment with 24 participants:
• 8 patients and 16 healthy individuals• discriminating between demented and
healthy individuals: • 100% on training data• 87.5% with cross-validation
![Page 8: Nick Cercone Vlado Keselj Calvin Thomas Computer Science Dalhousie University](https://reader035.vdocument.in/reader035/viewer/2022062409/56815152550346895dbf74b9/html5/thumbnails/8.jpg)
8
Related work: Automated authorship attribution Problem of identifying the author of an
anonymous text
One of Text Categorization Problems
. Spam detection
. Language and encoding identification
. Authorship attribution and plagiarism detection
. Text genre classification
. Topic detection
. Sentiment classification
![Page 9: Nick Cercone Vlado Keselj Calvin Thomas Computer Science Dalhousie University](https://reader035.vdocument.in/reader035/viewer/2022062409/56815152550346895dbf74b9/html5/thumbnails/9.jpg)
9
Related work (authorship attribution)
1. style analysis using style markers (features) relying on non-trivial NL analysis Stamatatos et al. 2000-02
2. language modeling Peng et al. 2003, EACL’03 Khmelev and Teahan 2003, SIGIR’03
3. N-gram-based text categorization Cavnar and Trenkle 1994
![Page 10: Nick Cercone Vlado Keselj Calvin Thomas Computer Science Dalhousie University](https://reader035.vdocument.in/reader035/viewer/2022062409/56815152550346895dbf74b9/html5/thumbnails/10.jpg)
10
Shortcomings of style analysis
• difficult to automatically extract some features
• feature selection is critical• language dependent• task dependent, i.e., does not
generalize well to other types of classification
![Page 11: Nick Cercone Vlado Keselj Calvin Thomas Computer Science Dalhousie University](https://reader035.vdocument.in/reader035/viewer/2022062409/56815152550346895dbf74b9/html5/thumbnails/11.jpg)
11
Character N-gram -based Methods Text can be considered as a concatenated
sequence of characters instead of words.
Advantages
1. small vocabulary
2. language independence
3. no word segmentation problems in many Asian languages such as Chinese and Thai
![Page 12: Nick Cercone Vlado Keselj Calvin Thomas Computer Science Dalhousie University](https://reader035.vdocument.in/reader035/viewer/2022062409/56815152550346895dbf74b9/html5/thumbnails/12.jpg)
12
How do character n-grams work?Marley was dead: to begin with. There
is no doubt whatever about that. …
n = 3Mararlrleleyey_y_w_wawas
_th 0.015 ___ 0.013 the 0.013 he_ 0.011 and 0.007 _an 0.007 nd_ 0.007 ed_ 0.006
sort by frequency
L=5
(from Christmas Carol by Charles Dickens)
…
![Page 13: Nick Cercone Vlado Keselj Calvin Thomas Computer Science Dalhousie University](https://reader035.vdocument.in/reader035/viewer/2022062409/56815152550346895dbf74b9/html5/thumbnails/13.jpg)
13
How do we compare two profiles?
_th 0.015 ___ 0.013 the 0.013 he_ 0.011 and 0.007
Dickens: Christmas Carol _th 0.016 the 0.014 he_ 0.012 and 0.007 nd_ 0.007
Dickens: A Tale of Two Cities
_th 0.017 ___ 0.017 the 0.014 he_ 0.014 ing 0.007
Carroll: Alice’s adventures in wonderland
?
?
![Page 14: Nick Cercone Vlado Keselj Calvin Thomas Computer Science Dalhousie University](https://reader035.vdocument.in/reader035/viewer/2022062409/56815152550346895dbf74b9/html5/thumbnails/14.jpg)
14
N-gram distribution
0.00E+00
5.00E-04
1.00E-03
1.50E-03
2.00E-03
2.50E-03
3.00E-03
3.50E-03
4.00E-03
4.50E-03
5.00E-03
1 4 7 10 13 16 19 22 25 28 31 34
6-grams
(From Dickens: Christmas Carol)
![Page 15: Nick Cercone Vlado Keselj Calvin Thomas Computer Science Dalhousie University](https://reader035.vdocument.in/reader035/viewer/2022062409/56815152550346895dbf74b9/html5/thumbnails/15.jpg)
15
CNG profile similarity measure
• a profile = the set of L the most frequent n-grams
• profile dissimilarity measure:
weight
2
profile 21
21
2
profile 21
21
)()(
))()((2
2)()()()(
nn nfnf
nfnfnfnfnfnf
![Page 16: Nick Cercone Vlado Keselj Calvin Thomas Computer Science Dalhousie University](https://reader035.vdocument.in/reader035/viewer/2022062409/56815152550346895dbf74b9/html5/thumbnails/16.jpg)
16
Authorship Attribution Evaluation
0
10
20
30
40
50
60
70
80
90
100
English Greek A Greek B Greek B+ Chinese
Style
Lang. M
CNG
![Page 17: Nick Cercone Vlado Keselj Calvin Thomas Computer Science Dalhousie University](https://reader035.vdocument.in/reader035/viewer/2022062409/56815152550346895dbf74b9/html5/thumbnails/17.jpg)
17
ACADIE Data Set• 189 GAS interviews (Goal Attainment Scaling)• 95 patients (2 interviews per patient, except 1
patient)• 6 sites; 17 MB of data (3.2 million words)• interview participants:
• FR – field researcher• Pt – patient• Cg – caregiver• other people
![Page 18: Nick Cercone Vlado Keselj Calvin Thomas Computer Science Dalhousie University](https://reader035.vdocument.in/reader035/viewer/2022062409/56815152550346895dbf74b9/html5/thumbnails/18.jpg)
18
Experiment set-up• preprocessing• patients divided into two groups
• 85 training group (169 interviews)• 10 testing group (20 interviews)
• patient speech in training group is used to build Alzheimer profile
• non-patient speech in training group is used to build non-Alzheimer profile
• two experiments:• classification• improvement detection
![Page 19: Nick Cercone Vlado Keselj Calvin Thomas Computer Science Dalhousie University](https://reader035.vdocument.in/reader035/viewer/2022062409/56815152550346895dbf74b9/html5/thumbnails/19.jpg)
19
Classification
• from each test interview patient and non-patient speech is extracted
• this produces 40 speech extracts• each speech extract is labelled by
the classifier as Alzheimer or non-Alzheimer
• accuracy is reported
![Page 20: Nick Cercone Vlado Keselj Calvin Thomas Computer Science Dalhousie University](https://reader035.vdocument.in/reader035/viewer/2022062409/56815152550346895dbf74b9/html5/thumbnails/20.jpg)
20
Experiment 1.1
training and testing part (90:10) use all speakers to generate profiles use both interviews
![Page 21: Nick Cercone Vlado Keselj Calvin Thomas Computer Science Dalhousie University](https://reader035.vdocument.in/reader035/viewer/2022062409/56815152550346895dbf74b9/html5/thumbnails/21.jpg)
21
ACADIE: Classification accuracy
n=1 2 3 4 5 6 7 8 9 10L = 20 88% 85% 83% 88% 98% 93% 95% 80% 85% 85%
50 73% 80% 78% 95% 95% 85% 93% 93% 95% 100%100 73% 78% 95% 95% 98% 98% 100% 98% 98% 100%200 73% 93% 98% 100% 98% 100% 100% 100% 100% 100%500 73% 80% 95% 100% 100% 98% 98% 100% 100% 100%
1000 73% 95% 100% 100% 100% 100% 100% 100% 100% 100%1500 73% 98% 98% 100% 100% 100% 100% 100% 100% 100%2000 73% 98% 93% 100% 100% 100% 100% 100% 100% 100%3000 73% 98% 93% 100% 100% 100% 100% 100% 100% 100%4000 73% 98% 98% 100% 100% 100% 100% 100% 100% 100%5000 73% 98% 98% 98% 100% 100% 100% 100% 100% 100%
![Page 22: Nick Cercone Vlado Keselj Calvin Thomas Computer Science Dalhousie University](https://reader035.vdocument.in/reader035/viewer/2022062409/56815152550346895dbf74b9/html5/thumbnails/22.jpg)
22
Improvement detection
) threshold(0.5 profileAlzheimer
withsimilarity normalized
profileAlzheimer -non with similarity
profileAlzheimer with similarity
ba
a
b
a
SS
SS
S
S
improvement is detected by observing an increase in S value between the first and second interview
![Page 23: Nick Cercone Vlado Keselj Calvin Thomas Computer Science Dalhousie University](https://reader035.vdocument.in/reader035/viewer/2022062409/56815152550346895dbf74b9/html5/thumbnails/23.jpg)
23
ACADIE: Detected improvement
n=1 2 3 4 5 6 7 8 9 10L = 20 50% 60% 70% 80% 70% 50% 50% 40% 60% 50%
50 50% 70% 60% 30% 60% 30% 30% 60% 50% 70%100 40% 60% 40% 40% 40% 40% 80% 60% 70% 60%200 40% 30% 30% 40% 50% 70% 40% 70% 50% 60%500 40% 80% 60% 80% 60% 50% 40% 60% 80% 70%
1000 40% 50% 90% 60% 70% 70% 70% 90% 60% 60%1500 40% 70% 80% 70% 80% 60% 80% 80% 60% 50%2000 40% 60% 90% 70% 70% 70% 70% 70% 60% 60%3000 40% 60% 70% 70% 70% 60% 60% 70% 60% 70%4000 40% 60% 70% 90% 80% 80% 70% 60% 70% 70%5000 40% 60% 70% 80% 80% 70% 60% 70% 70% 70%
![Page 24: Nick Cercone Vlado Keselj Calvin Thomas Computer Science Dalhousie University](https://reader035.vdocument.in/reader035/viewer/2022062409/56815152550346895dbf74b9/html5/thumbnails/24.jpg)
24
Experiment 1.2
use only first interviews to create Alzheimer and Non-Alzheimer profiles
![Page 25: Nick Cercone Vlado Keselj Calvin Thomas Computer Science Dalhousie University](https://reader035.vdocument.in/reader035/viewer/2022062409/56815152550346895dbf74b9/html5/thumbnails/25.jpg)
25
Exp. 1.2: Classification accuracy
n=1 2 3 4 5 6 7 8 9 10L = 20 85% 85% 83% 88% 93% 90% 95% 80% 80% 83%
50 70% 90% 83% 98% 95% 85% 93% 95% 95% 90%100 73% 98% 98% 98% 90% 98% 98% 98% 95% 98%200 73% 88% 98% 100% 100% 98% 100% 95% 100% 100%500 73% 83% 98% 100% 95% 98% 95% 100% 98% 100%
1000 73% 95% 100% 100% 100% 100% 100% 100% 100% 100%1500 73% 95% 95% 100% 100% 100% 100% 100% 100% 100%2000 73% 95% 93% 100% 100% 100% 100% 100% 100% 100%3000 73% 95% 95% 100% 100% 100% 100% 100% 100% 100%4000 73% 95% 98% 100% 100% 100% 100% 100% 100% 100%5000 73% 95% 100% 100% 100% 100% 100% 100% 100% 100%
Improvement detection: 0.6-0.9
![Page 26: Nick Cercone Vlado Keselj Calvin Thomas Computer Science Dalhousie University](https://reader035.vdocument.in/reader035/viewer/2022062409/56815152550346895dbf74b9/html5/thumbnails/26.jpg)
26
Experiment 1.3
use only first interviews only speech produced by patients,
caregivers, and other (not field researchers)
![Page 27: Nick Cercone Vlado Keselj Calvin Thomas Computer Science Dalhousie University](https://reader035.vdocument.in/reader035/viewer/2022062409/56815152550346895dbf74b9/html5/thumbnails/27.jpg)
27
Exp. 1.3: Classification accuracy
n=1 2 3 4 5 6 7 8 9 10L = 20 75% 90% 85% 80% 65% 75% 80% 70% 75% 70%
50 73% 88% 68% 80% 90% 75% 75% 75% 80% 83%100 73% 85% 88% 85% 88% 80% 83% 88% 88% 88%200 73% 83% 90% 90% 95% 88% 93% 88% 98% 93%500 73% 65% 95% 95% 95% 95% 98% 90% 93% 95%
1000 73% 83% 93% 93% 98% 93% 98% 98% 98% 95%1500 73% 78% 80% 95% 95% 100% 98% 98% 98% 95%2000 73% 80% 75% 95% 100% 98% 98% 98% 98% 95%3000 73% 83% 83% 88% 95% 98% 95% 98% 95% 93%4000 73% 83% 90% 95% 95% 95% 98% 98% 95% 95%5000 73% 83% 93% 98% 95% 98% 98% 98% 98% 93%
Improvement detection: 0.6-0.8
![Page 28: Nick Cercone Vlado Keselj Calvin Thomas Computer Science Dalhousie University](https://reader035.vdocument.in/reader035/viewer/2022062409/56815152550346895dbf74b9/html5/thumbnails/28.jpg)
28
Some experiment observations Alzheimer n-gram profile captures many
indefinite terms and negated (e.g., sometimes, don’t know, can not, …)
the profiles captures reduced lexical richness
Alzheimer
non-Alzheimer
n-gram rank
n-gram
frequency
![Page 29: Nick Cercone Vlado Keselj Calvin Thomas Computer Science Dalhousie University](https://reader035.vdocument.in/reader035/viewer/2022062409/56815152550346895dbf74b9/html5/thumbnails/29.jpg)
29
Second set of experiments
rating dementia levels
implement method BSCW (by Bucks et al.),
analysis and extension
comparison with CNG
application of a wider set of machine learning algorithms
![Page 30: Nick Cercone Vlado Keselj Calvin Thomas Computer Science Dalhousie University](https://reader035.vdocument.in/reader035/viewer/2022062409/56815152550346895dbf74b9/html5/thumbnails/30.jpg)
30
MMSE – Mini-Mental State Exam MMSE – a standard test for identifying
cognitive impairment in a clinical setting 17 questions, 5-10 minutes introduced in 1975 by Folstein et al. score range from 0 to 30 a variety of cut points suggested over years:
17.5, 21.5, 23.5, 25.5
![Page 31: Nick Cercone Vlado Keselj Calvin Thomas Computer Science Dalhousie University](https://reader035.vdocument.in/reader035/viewer/2022062409/56815152550346895dbf74b9/html5/thumbnails/31.jpg)
31
MMSE Score Gradation
we use the following gradation
four classes: severe moderate mild normal
two classes: low high
0 14.5 20.5 24.5 30
![Page 32: Nick Cercone Vlado Keselj Calvin Thomas Computer Science Dalhousie University](https://reader035.vdocument.in/reader035/viewer/2022062409/56815152550346895dbf74b9/html5/thumbnails/32.jpg)
32
MMSE Score distribution in data set
severe moderate mild normal
![Page 33: Nick Cercone Vlado Keselj Calvin Thomas Computer Science Dalhousie University](https://reader035.vdocument.in/reader035/viewer/2022062409/56815152550346895dbf74b9/html5/thumbnails/33.jpg)
33
![Page 34: Nick Cercone Vlado Keselj Calvin Thomas Computer Science Dalhousie University](https://reader035.vdocument.in/reader035/viewer/2022062409/56815152550346895dbf74b9/html5/thumbnails/34.jpg)
34
Part-of-speech tagging, MA-A following the BSCW method applied Hepple from NL GATE and Connexor Hepple is based on Brill’s tagger Connexor performed better set of attributes MA-A: attributes similar to
BSCW: excluded CSU-rate:
1. manually annotated
2. reported non-significant impact by BSCW
![Page 35: Nick Cercone Vlado Keselj Calvin Thomas Computer Science Dalhousie University](https://reader035.vdocument.in/reader035/viewer/2022062409/56815152550346895dbf74b9/html5/thumbnails/35.jpg)
35
Morphological Attribute Set: MA-B start with all POS attributes regression-based attribute selection 7 POS attributes selected (conjunctions
included) add TTR and Honore statistics
Brunet statistic shown to be non-significant use several machine learning algorithms with
cross-validation, using software tool WEKA
![Page 36: Nick Cercone Vlado Keselj Calvin Thomas Computer Science Dalhousie University](https://reader035.vdocument.in/reader035/viewer/2022062409/56815152550346895dbf74b9/html5/thumbnails/36.jpg)
36
![Page 37: Nick Cercone Vlado Keselj Calvin Thomas Computer Science Dalhousie University](https://reader035.vdocument.in/reader035/viewer/2022062409/56815152550346895dbf74b9/html5/thumbnails/37.jpg)
37
Ordinal CNG Method
• use two extreme groups to build profiles
severe dementia level normal level
profilesevere
profilenormal
test speechprofile
CNG similarity: SsevereSnormal
classify according tonormalsevere
severe
SS
S
![Page 38: Nick Cercone Vlado Keselj Calvin Thomas Computer Science Dalhousie University](https://reader035.vdocument.in/reader035/viewer/2022062409/56815152550346895dbf74b9/html5/thumbnails/38.jpg)
38
Ordinal CNG: Thresholds
range of values: [0,1] 0 corresponds to severe, 1 to normal
what are good threshold interesting observation:
the optimal threshold is very close to the “natural threshold” – 0.5 (varies from 0.5 to 0.512)
![Page 39: Nick Cercone Vlado Keselj Calvin Thomas Computer Science Dalhousie University](https://reader035.vdocument.in/reader035/viewer/2022062409/56815152550346895dbf74b9/html5/thumbnails/39.jpg)
39
![Page 40: Nick Cercone Vlado Keselj Calvin Thomas Computer Science Dalhousie University](https://reader035.vdocument.in/reader035/viewer/2022062409/56815152550346895dbf74b9/html5/thumbnails/40.jpg)
40
Conclusions extensive experiments on morphological and lexical
analysis of spontaneous speech for detecting dementia of Alzheimer type
methods: CNG and Ordinal CNG extension of method proposed by use of POS tags as
suggested by BSCW positive results in classification and detecting
dementia level: 100% discrimination accuracy (Pt and other) 93% - severe/normal 70% - two-class accuracy 46% - four-class accuracy
![Page 41: Nick Cercone Vlado Keselj Calvin Thomas Computer Science Dalhousie University](https://reader035.vdocument.in/reader035/viewer/2022062409/56815152550346895dbf74b9/html5/thumbnails/41.jpg)
41
Future work
improvement detection use of word CNG method stop-word frequency-based classifier syntactic analysis semantic analysis