the challenges of affect detection in the social programmer ecosystem
TRANSCRIPT
![Page 1: The Challenges of Affect Detection in the Social Programmer Ecosystem](https://reader035.vdocument.in/reader035/viewer/2022062823/587a26bd1a28abbd388b52b5/html5/thumbnails/1.jpg)
The Challenges of Affect Detection in the Social Programmer
Ecosystem
Nicole NOVIELLICOLLAB, Collaborative Development Group
Dipartimento di InformaticaUniversità degli Studi di Bari ‘Aldo Moro’
![Page 2: The Challenges of Affect Detection in the Social Programmer Ecosystem](https://reader035.vdocument.in/reader035/viewer/2022062823/587a26bd1a28abbd388b52b5/html5/thumbnails/2.jpg)
Outline
• Motivation• Studying Affect in SE• A preliminary investigation• Open Challenges
![Page 3: The Challenges of Affect Detection in the Social Programmer Ecosystem](https://reader035.vdocument.in/reader035/viewer/2022062823/587a26bd1a28abbd388b52b5/html5/thumbnails/3.jpg)
University of Bari• Since 1924• 2nd largest university of South Italy• 60,000 students, 1,500 professors, 1,400 employees• 4 main campuses: Science and Engineering, Economics, Law and
Humanities, Medicine
![Page 4: The Challenges of Affect Detection in the Social Programmer Ecosystem](https://reader035.vdocument.in/reader035/viewer/2022062823/587a26bd1a28abbd388b52b5/html5/thumbnails/4.jpg)
People at Collab
• Faculty– Filippo Lanubile– Fabio Calefato– Nicole Novielli
• Research Assistants– Alessandro Natilla
• Graduate Students• Final-year undergraduate students
Collaborative Development Group
Department of Computer Science - collab.di.uniba.it
![Page 5: The Challenges of Affect Detection in the Social Programmer Ecosystem](https://reader035.vdocument.in/reader035/viewer/2022062823/587a26bd1a28abbd388b52b5/html5/thumbnails/5.jpg)
Research at COLLAB
• Distributed Development– Addressing the problem of distance in distributed
workgroups• Collaborative / Global / Social Software
Engineering– Special focus on software development as an intense
collaborative process
![Page 6: The Challenges of Affect Detection in the Social Programmer Ecosystem](https://reader035.vdocument.in/reader035/viewer/2022062823/587a26bd1a28abbd388b52b5/html5/thumbnails/6.jpg)
Software Engineering involves social interaction• Programmers cooperate, directly or indirectly• Massive adoption of social media and rise of the ‘social programmer’ (Storey, ‘12)
and the surrounding ecosystem
![Page 7: The Challenges of Affect Detection in the Social Programmer Ecosystem](https://reader035.vdocument.in/reader035/viewer/2022062823/587a26bd1a28abbd388b52b5/html5/thumbnails/7.jpg)
From: A. Begel, T. Zimmermann. Analyze This! 145 Questions for Data Scientists in Software Engineering, ICSE 2014
Microsoft’s Top 10 Questions EssentialEssential + Worthwhile
1. How do users typically use my application? 80.0% 99.2%
2. What parts of a software product are most used and/or loved by customers?
72.0% 98.5%
3. How effective are the quality gates we run at checkin? 62.4% 96.6%
4. How can we improve collaboration and sharing between teams?
54.5% 96.4%
5. What are the best key performance indicators (KPIs) for monitoring services?
53.2% 93.6%
6. What is the impact of a code change or requirements change to the project and its tests?
52.1% 94.0%
7. What is the impact of tools on productivity? 50.5% 97.2%
8. How do I avoid reinventing the wheel by sharing and/or searching for code?
50.0% 90.9%
9. What are the common patterns of execution in my application? 48.7% 96.6%
10. How well does test coverage correspond to actual code usage by our customers?
48.7% 92.0%
![Page 8: The Challenges of Affect Detection in the Social Programmer Ecosystem](https://reader035.vdocument.in/reader035/viewer/2022062823/587a26bd1a28abbd388b52b5/html5/thumbnails/8.jpg)
The Role of Affect• Emotion Awareness in Social Software
Engineering– Do emotions affect the outcome of collaboration?– How to deal with troubles in emotion communication
in computer-mediated interaction?– How to appropriately convey sentiment through text?
![Page 9: The Challenges of Affect Detection in the Social Programmer Ecosystem](https://reader035.vdocument.in/reader035/viewer/2022062823/587a26bd1a28abbd388b52b5/html5/thumbnails/9.jpg)
Studying Emotions in Software Engineering
![Page 10: The Challenges of Affect Detection in the Social Programmer Ecosystem](https://reader035.vdocument.in/reader035/viewer/2022062823/587a26bd1a28abbd388b52b5/html5/thumbnails/10.jpg)
Studying Emotions in Software Engineering
MSR‘15
Mining Successful Answers in Stack
Overflow (Calefato et al.)
Peer J ‘13
Happy software developers solve problems better
Peer J(Graziotin et al.)
ESEC/FSE ‘13
Towards emotional
awareness in software
development teams.
(Guzman and Bruegge)
ICSE‘15
Stuck and Frustrated or In
Flow and Happy: Sensing
Developers’ Emotions and
Progress. (Muller and Fritz)
MSR ‘14
Sentiment analysis of commit comments in GitHub: an
empirical study (Guzman et al.)
RE‘14
How Do Users Like this
Feature? A Fine Grained
Sentiment Analysis of App
Reviews (Guzman and Maalej)
Security and emotion: sentiment analysis of security discussions
on GitHub (Pletea et al.)
CHASE ‘15
Exploring Causes of Frustration for Software
Developers. (Ford and Parnin)
Towards emotion-based collaborative software
engineering(Dewan)
Do developers feel emotions?(Murgia et al.)
Are Bullies More Productive? Empirical Study of Affectiveness vs. Issue Fixing Time
(Ortu et al.)
SCAM‘15
Recommending insightful comments
for source code using crowdsourced
knowledge. Source Code Analysis and
Manipulation (Rahman et al.)
2013 2014 2015
![Page 11: The Challenges of Affect Detection in the Social Programmer Ecosystem](https://reader035.vdocument.in/reader035/viewer/2022062823/587a26bd1a28abbd388b52b5/html5/thumbnails/11.jpg)
• Annotation study on the Apache Software Foundation issue tracking system– Wide range of both positive and negative emotions
Murgia et al., 2014 – Do developers feel emotions? An exploratiory analysis of emotions in software artifacts. MSR 2014.
Do Developers Feel Emotions?
Thanks for your input! You're, like, awesome.
I'm happy with the approach and the code looks good
I will come over to your work and slap you
Sorry for the delay Stephen
![Page 12: The Challenges of Affect Detection in the Social Programmer Ecosystem](https://reader035.vdocument.in/reader035/viewer/2022062823/587a26bd1a28abbd388b52b5/html5/thumbnails/12.jpg)
• Survey of 45 software developers – For 67% frustration is a severe problem
• Learning barriers are a major cause of frustration– New programming tools and languages– New projects– Unavailability of resources
• High cognitive complexity and time pressure– Misinterpretation of the task– Size of task, high cognitive complexity– Limited time, task less simple than expected
Ford and Parnin, 2015. Exploring Causes of Frustration for Software Developers. CHASE ’15.
Exploring Causes of Frustration for Developers
![Page 13: The Challenges of Affect Detection in the Social Programmer Ecosystem](https://reader035.vdocument.in/reader035/viewer/2022062823/587a26bd1a28abbd388b52b5/html5/thumbnails/13.jpg)
Do emotions affect performance? (1/3)
• Positive mood improves debugging performance
• Positive mood induced through physical exercises
Khan et al. 2011 - Do moods affect programmers’ debug performance? Cognition, Technology & Work
![Page 14: The Challenges of Affect Detection in the Social Programmer Ecosystem](https://reader035.vdocument.in/reader035/viewer/2022062823/587a26bd1a28abbd388b52b5/html5/thumbnails/14.jpg)
Do emotions affect performance? (2/3)
• Emotions is associated to issue tracking activity – More comments and watchers for issue where
Sadness is expressed– Higher fixing time when negative emotions are
expressed
Murgia et al., 2014 – Do developers feel emotions? An exploratiory analysis of emotions in software artifacts. MSR 2014.
![Page 15: The Challenges of Affect Detection in the Social Programmer Ecosystem](https://reader035.vdocument.in/reader035/viewer/2022062823/587a26bd1a28abbd388b52b5/html5/thumbnails/15.jpg)
• Emotion and cognition– Developers constantly face problem-solving tasks– Analysis and creativity skills are required – Emotions and moods deeply influence cognitive
processing abilities and performance
Do emotions affect performance? (3/3)
Graziotin et al., 2013 - Happy software developers solve problems better: psychological measurements in empirical software engineering. Peer J
![Page 16: The Challenges of Affect Detection in the Social Programmer Ecosystem](https://reader035.vdocument.in/reader035/viewer/2022062823/587a26bd1a28abbd388b52b5/html5/thumbnails/16.jpg)
- Google Careers - http://www.google.com/about/careers/lifeatgoogle/benefits/ Last accessed: Jan 2016.
![Page 17: The Challenges of Affect Detection in the Social Programmer Ecosystem](https://reader035.vdocument.in/reader035/viewer/2022062823/587a26bd1a28abbd388b52b5/html5/thumbnails/17.jpg)
Affective Computing as a New Method for Empirical Software Engineering
![Page 18: The Challenges of Affect Detection in the Social Programmer Ecosystem](https://reader035.vdocument.in/reader035/viewer/2022062823/587a26bd1a28abbd388b52b5/html5/thumbnails/18.jpg)
Multimodal recognition of affective states• Biometric measurements
- Empatica – www.empatica.com- Bos, EEG-based Emotion Recognition - Human Media Interaction
Need for individual training for each user
![Page 19: The Challenges of Affect Detection in the Social Programmer Ecosystem](https://reader035.vdocument.in/reader035/viewer/2022062823/587a26bd1a28abbd388b52b5/html5/thumbnails/19.jpg)
Multimodal recognition of affective states• Biometric measurements• Facial expression analysis
![Page 20: The Challenges of Affect Detection in the Social Programmer Ecosystem](https://reader035.vdocument.in/reader035/viewer/2022062823/587a26bd1a28abbd388b52b5/html5/thumbnails/20.jpg)
Multimodal recognition of affective states• Biometric measurements• Facial expression analysis• Affective natural language processing
![Page 21: The Challenges of Affect Detection in the Social Programmer Ecosystem](https://reader035.vdocument.in/reader035/viewer/2022062823/587a26bd1a28abbd388b52b5/html5/thumbnails/21.jpg)
Multimodal recognition of affective states• Biometric measurements• Facial expression analysis• Affective natural language processing• Social signal processing (multimodal)
A.Vinciarelli, M.Pantic, D.Heylen, C.Pelachaud, I.Poggi, F.D’Errico and M.Schroeder, 2012. “Bridging the Gap Between Social Animal and Unsocial Machine: A Survey of Social Signal Processing“, IEEE Transactions on Affective Computing.F.D’Errico, I.Poggi, A.Vinciarelli and L.Vincze (eds.), 2015. Conflict and Multimodal Communication
- Negotiation and conflicts- Communication effectiveness- Role recognition- Display of status - Power relationships and
dominance- Automatic personality perception
![Page 22: The Challenges of Affect Detection in the Social Programmer Ecosystem](https://reader035.vdocument.in/reader035/viewer/2022062823/587a26bd1a28abbd388b52b5/html5/thumbnails/22.jpg)
Affective computing as a new method for SE (1/3)• Sensing affect from biometric measures
• Recognizing “in flow” vs. stuck developers – In flow developers should not be disturbed– Stuck developers need recommendations
Muller and Fritz, 2015 - Stuck and Frustrated or In Flow and Happy: Sensing Developers’ Emotions and Progress. ICSE 2015
EEG sensor
Wrist band
Eye tracking
![Page 23: The Challenges of Affect Detection in the Social Programmer Ecosystem](https://reader035.vdocument.in/reader035/viewer/2022062823/587a26bd1a28abbd388b52b5/html5/thumbnails/23.jpg)
Affective computing as a new method for SE (2/3)• Affective natural language processing• Software requirements evolution
– Feature-based sentiment analysis of app reviews (Guzman and Maalej, 2015)
• Crowdsourced documentation– Exploiting sentiment polarity to assess usefulness
of comments in Stack Overflow (Rahman et al., 2015)
Guzman and Maalej, 2015 - How Do Users Like This Feature? A Fine Grained Sentiment Analysis of App Reviews - Requirements Engineering Conference (RE), 2014 Rahman et al., 2015 - Recommending insightful comments for source code using crowdsourced knowledge. Source Code Analysis and Manipulation (SCAM), 2015
![Page 24: The Challenges of Affect Detection in the Social Programmer Ecosystem](https://reader035.vdocument.in/reader035/viewer/2022062823/587a26bd1a28abbd388b52b5/html5/thumbnails/24.jpg)
Affective computing as a new method for SE (3/3)• Affective natural language processing• Improve team collaboration
– Sentiment analysis of communication artifacts for emotional awareness in development teams (Guzman and Bruegge, 2013)
• Crowdsourced knowledge– Investigating the role played by emotions in
success of information seeking in community-based Question & Answering (Calefato et al., 2015)
Guzman and Bruegge, 2013. Towards emotional awareness in software development teams. ESEC/FSE 2013.Calefato et al., 2015. Mining Successful Answers in Stack Overflow. MSR, 2015.
![Page 25: The Challenges of Affect Detection in the Social Programmer Ecosystem](https://reader035.vdocument.in/reader035/viewer/2022062823/587a26bd1a28abbd388b52b5/html5/thumbnails/25.jpg)
How can we model and detect emotions?
![Page 26: The Challenges of Affect Detection in the Social Programmer Ecosystem](https://reader035.vdocument.in/reader035/viewer/2022062823/587a26bd1a28abbd388b52b5/html5/thumbnails/26.jpg)
How can we model and detect emotions?
MSR‘15
Mining Successful Answers in Stack
Overflow (Calefato et al.)
Peer J ‘13
Happy software developers solve problems better
Peer J(Graziotin et al.)
ESEC/FSE ‘13
Towards emotional
awareness in software
development teams.
(Guzman and Bruegge)
ICSE‘15
Stuck and Frustrated or In
Flow and Happy: Sensing
Developers’ Emotions and
Progress. (Muller and Fritz)
MSR ‘14
Sentiment analysis of commit comments in GitHub: an
empirical study (Guzman et al.)
RE‘14
How Do Users Like this
Feature? A Fine Grained
Sentiment Analysis of App
Reviews (Guzman and Maalej)
Security and emotion: sentiment analysis of security discussions
on GitHub (Pletea et al.)
CHASE ‘15
Exploring Causes of Frustration for Software
Developers. (Ford and Parnin)
Towards emotion-based collaborative software
engineering(Dewan)
Do developers feel emotions?(Murgia et al.)
Are Bullies More Productive? Empirical Study of Affectiveness vs. Issue Fixing Time
(Ortu et al.)
SCAM‘15
Recommending insightful comments
for source code using crowdsourced
knowledge. Source Code Analysis and
Manipulation (Rahman et al.)
2013 2014 2015
![Page 27: The Challenges of Affect Detection in the Social Programmer Ecosystem](https://reader035.vdocument.in/reader035/viewer/2022062823/587a26bd1a28abbd388b52b5/html5/thumbnails/27.jpg)
MSR‘15
Mining Successful Answers in Stack
Overflow (Calefato et al.)
Peer J ‘13
Happy software developers solve problems better
Peer J(Graziotin et al.)
ESEC/FSE ‘13
Towards emotional
awareness in software
development teams.
(Guzman and Bruegge)
ICSE‘15
Stuck and Frustrated or In
Flow and Happy: Sensing
Developers’ Emotions and
Progress. (Muller and Fritz)
MSR ‘14
Sentiment analysis of commit comments in GitHub: an
empirical study (Guzman et al.)
RE‘14
How Do Users Like this
Feature? A Fine Grained
Sentiment Analysis of App
Reviews (Guzman and Maalej)
Security and emotion: sentiment analysis of security discussions
on GitHub (Pletea et al.)
CHASE ‘15
Exploring Causes of Frustration for Software
Developers. (Ford and Parnin)
Towards emotion-based collaborative software
engineering(Dewan)
Do developers feel emotions?(Murgia et al.)
Are Bullies More Productive? Empirical Study of Affectiveness vs. Issue Fixing Time
(Ortu et al.)
SCAM‘15
Recommending insightful comments
for source code using crowdsourced
knowledge. Source Code Analysis and
Manipulation (Rahman et al.)
2013 2014 2015
How can we model and detect emotions?
![Page 28: The Challenges of Affect Detection in the Social Programmer Ecosystem](https://reader035.vdocument.in/reader035/viewer/2022062823/587a26bd1a28abbd388b52b5/html5/thumbnails/28.jpg)
MSR‘15
Mining Successful Answers in Stack
Overflow (Calefato et al.)
Peer J ‘13
Happy software developers solve problems better
Peer J(Graziotin et al.)
ESEC/FSE ‘13
Towards emotional
awareness in software
development teams.
(Guzman and Bruegge)
ICSE‘15
Stuck and Frustrated or In
Flow and Happy: Sensing
Developers’ Emotions and
Progress. (Muller and Fritz)
MSR ‘14
Sentiment analysis of commit comments in GitHub: an
empirical study (Guzman et al.)
RE‘14
How Do Users Like this
Feature? A Fine Grained
Sentiment Analysis of App
Reviews (Guzman and Maalej)
Security and emotion: sentiment analysis of security discussions
on GitHub (Pletea et al.)
CHASE ‘15
Exploring Causes of Frustration for Software
Developers. (Ford and Parnin)
Towards emotion-based collaborative software
engineering(Dewan)
Do developers feel emotions?(Murgia et al.)
Are Bullies More Productive? Empirical Study of Affectiveness vs. Issue Fixing Time
(Ortu et al.)
SCAM‘15
Recommending insightful comments
for source code using crowdsourced
knowledge. Source Code Analysis and
Manipulation (Rahman et al.)
2013 2014 2015
How can we model and detect emotions?
![Page 29: The Challenges of Affect Detection in the Social Programmer Ecosystem](https://reader035.vdocument.in/reader035/viewer/2022062823/587a26bd1a28abbd388b52b5/html5/thumbnails/29.jpg)
Polarity classification• Classification of a text according to its positive,
negative or neutral semantic orientation• Several tools available
– NLTK• Outputs probability for each polarity class• Trained on tweets and movie reviews
– Stanford Sentiment Analyzer• Issues an overall polarity label + representation of the sentence
structure• Trained on movie reviews
– SentiStrength• Outputs a score for both positive and negative sentiment• Designed for and validated on general purpose social media
[1] – NLTK: http://text-processing.com/ [2] – Stanford Sentiment Analyser - http://nlp.stanford.edu/sentiment/[3] – SentiStrength - http://sentistrength.wlv.ac.uk/
![Page 30: The Challenges of Affect Detection in the Social Programmer Ecosystem](https://reader035.vdocument.in/reader035/viewer/2022062823/587a26bd1a28abbd388b52b5/html5/thumbnails/30.jpg)
Measuring the sentiment: SentiStrength
• Estimates the strength of both positive and negative sentiment in a text
• Robust also for informal language• Widely employed in social computing
Input textPositive Sentiment Score
Negative Sentiment Score
In [1,5]
In [-1,-5]
where ±1 indicates absence of pos/neg sentiment
Emotion Lookup table
![Page 31: The Challenges of Affect Detection in the Social Programmer Ecosystem](https://reader035.vdocument.in/reader035/viewer/2022062823/587a26bd1a28abbd388b52b5/html5/thumbnails/31.jpg)
SentiStrength
abandon* -2 LIWCabate -2 General Inquirer Feb 2010abdicate* -2 General Inquirer Feb 2010abhor* -4 General Inquirer Feb 2010abject -2 General Inquirer Feb 2010abnormal* -2 General Inquirer Feb 2010accusation* -2 General Inquirer Feb 2010
[…]
Emotion Lookup table
![Page 32: The Challenges of Affect Detection in the Social Programmer Ecosystem](https://reader035.vdocument.in/reader035/viewer/2022062823/587a26bd1a28abbd388b52b5/html5/thumbnails/32.jpg)
Sentiment annotation with SentiStrength
Input text Positive NegativeAny help would be really great! :-) 5 -1
@<username> - what’s the emoticon for being terribly embarassed!
1 -5
I have very simple and stupid trouble […] I am pretty confused, explain please, what is wrong? Any help would be greatly appreciated.
3 -3
![Page 33: The Challenges of Affect Detection in the Social Programmer Ecosystem](https://reader035.vdocument.in/reader035/viewer/2022062823/587a26bd1a28abbd388b52b5/html5/thumbnails/33.jpg)
What model and recognition approaches are suitable to detect the affect of
programmers in a reliable manner?
• Which is the best model?• How to operationalize it?• What tools/resources are available?
![Page 34: The Challenges of Affect Detection in the Social Programmer Ecosystem](https://reader035.vdocument.in/reader035/viewer/2022062823/587a26bd1a28abbd388b52b5/html5/thumbnails/34.jpg)
It depends…
• On research goals and application domain– Why do you want to model and detect affect?
• On the available source of data– Sensors: biometric measurements – Written text: natural language processing and
conversational analysis– Speech: acoustic and prosodic features– Cameras/Eye tracking devices: body posture, gaze,
facial analysis
![Page 35: The Challenges of Affect Detection in the Social Programmer Ecosystem](https://reader035.vdocument.in/reader035/viewer/2022062823/587a26bd1a28abbd388b52b5/html5/thumbnails/35.jpg)
Typology of Affective StatesD
ur
at
io
n
++
In
te
ns
it
y+
+
Scherer, 1984. Emotion as a Multicomponent Process: A model and some cross-cultural data. In P. Shaver, ed., Review of Personality and Social Psych 5: 37-63.
![Page 36: The Challenges of Affect Detection in the Social Programmer Ecosystem](https://reader035.vdocument.in/reader035/viewer/2022062823/587a26bd1a28abbd388b52b5/html5/thumbnails/36.jpg)
Continuous vs. Discrete Emotion Models (1/4)
Russel, 1980 - A circumplex model of affect. J.Personality and Social Psychology, 39, 1161-1178.
Polarity
ActivationCircumplex Model of Affect
![Page 37: The Challenges of Affect Detection in the Social Programmer Ecosystem](https://reader035.vdocument.in/reader035/viewer/2022062823/587a26bd1a28abbd388b52b5/html5/thumbnails/37.jpg)
Continuous vs. Discrete Emotion Models (2/4)
Ekman’s basic emotions
Anger Fear Disgust Surprise Happiness Sadness
![Page 38: The Challenges of Affect Detection in the Social Programmer Ecosystem](https://reader035.vdocument.in/reader035/viewer/2022062823/587a26bd1a28abbd388b52b5/html5/thumbnails/38.jpg)
Continuous vs. Discrete Emotion Models (3/4)
Lazarus’ framework
Negative Emotions
Positive Emotions
Mixed Emotions
angerfrightanxietyguiltshamesadnessenvyjealousyDisgust
happinesspriderelieflove
hopecompassiongratitude
Based on appraisal theory
![Page 39: The Challenges of Affect Detection in the Social Programmer Ecosystem](https://reader035.vdocument.in/reader035/viewer/2022062823/587a26bd1a28abbd388b52b5/html5/thumbnails/39.jpg)
Continuous vs. Discrete Emotion Models (4/4)
Plutchik’s Wheel
Many others…
![Page 40: The Challenges of Affect Detection in the Social Programmer Ecosystem](https://reader035.vdocument.in/reader035/viewer/2022062823/587a26bd1a28abbd388b52b5/html5/thumbnails/40.jpg)
Goal
Example
Resources
Sentiment Analysis
Negative vs. PositiveSentiment Strength
SentiStrength(Thelwall et al., 2012)
SentiWordNet(Esuli and Sebastiani, 2006)
MPQA Lexicon(Wilson et al., 2005)
- …
‘I can't solve this problem, it’s very frustrating’
Negative
Mining Affect from Text
Emotion Detection
Classification using Discrete Emotion Labels
WordNet Affect (Strapparava and Valitutti, 2004)
NRC Emotion Lexicon (Mohammad and Turney, 2010)
Depeche Mood (Staiano and Guerini, 2014)
- …
‘I can't solve this problem, it’s very frustrating’
Frustrated
![Page 41: The Challenges of Affect Detection in the Social Programmer Ecosystem](https://reader035.vdocument.in/reader035/viewer/2022062823/587a26bd1a28abbd388b52b5/html5/thumbnails/41.jpg)
SentiWordNet
http://sentiwordnet.isti.cnr.it/ Esuli and Sebastiani. Sentiwordnet: A publicly available lexical resource for opinion mining. LREC 2006 Baccianella et al. SentiWordNet 3.0: An Enhanced Lexical Resource for Sentiment Analysis and Opinion Mining, LREC 2010
Scores Sense ID and gloss as an adjective
Positive = 0.75Negative = 0Objective = 0.25
good#1 - Having desirable or positive qualities especially those suitable for a thing specified (as in ‘a good joke’)
as a noun
Positive = 0.5 Negative = 0Objective = 0.5
good#1 benefit (as in ‘for your own good’)
Positive = 0Negative = 0Objective = 1
good#4commodity, article of commerce
P N
O
![Page 42: The Challenges of Affect Detection in the Social Programmer Ecosystem](https://reader035.vdocument.in/reader035/viewer/2022062823/587a26bd1a28abbd388b52b5/html5/thumbnails/42.jpg)
Wordnet Affect
A-label Example of Synsets
EMOTION noun ‘anger’, verb ‘fear’
MOOD noun ‘animosity’, adjective ‘fear’
TRAIT noun ‘aggressiveness’, adj. ‘competitive’
COGNITIVE State noun ‘confusion’, adj. ‘dazed’
PHYSICAL State noun ‘illness’
HEDONIC Signal noun ‘hurt’, noun ‘suffering’
Emotion-eliciting SITUATION noun ‘awkwardness’
Emotional RESPONSE noun ‘cold sweat’, verb ‘tremble’
BEHAVIOR noun ‘offense’, adj. ‘inhibited’
ATTITUDE noun ‘intolerance’, noun ‘defensive’
SENSATION noun ‘coldness’, noun ‘feel’
Strapparava and Valitutti. WordNet-affect: an affective extension of WordNet. LREC 2004
![Page 43: The Challenges of Affect Detection in the Social Programmer Ecosystem](https://reader035.vdocument.in/reader035/viewer/2022062823/587a26bd1a28abbd388b52b5/html5/thumbnails/43.jpg)
Emotion Detection in the Social Programmer Ecosystem
• Data source: written, asynchronous communication on social media
• Goals– To prevent negative emotion from negatively
affecting outcome of collaboration– To support developers in appropriately convey
sentiment through text– To exploit emotion mining for improving the creation
of crowdsourced knowledge
![Page 44: The Challenges of Affect Detection in the Social Programmer Ecosystem](https://reader035.vdocument.in/reader035/viewer/2022062823/587a26bd1a28abbd388b52b5/html5/thumbnails/44.jpg)
Research Question
• What affect model should we adopt?• What are the main drawbacks of applying state-of-the-art
sentiment analysis techniques in the SE domain? • What tools and linguistic resources should be used and
how can they be improved and adapted to the SE domain?
How to reliably detect the affective states of programmers in textual communication artifacts?
![Page 45: The Challenges of Affect Detection in the Social Programmer Ecosystem](https://reader035.vdocument.in/reader035/viewer/2022062823/587a26bd1a28abbd388b52b5/html5/thumbnails/45.jpg)
A Preliminary Investigation on Stack Overflow
![Page 46: The Challenges of Affect Detection in the Social Programmer Ecosystem](https://reader035.vdocument.in/reader035/viewer/2022062823/587a26bd1a28abbd388b52b5/html5/thumbnails/46.jpg)
SSE@FSE 2014 46
![Page 47: The Challenges of Affect Detection in the Social Programmer Ecosystem](https://reader035.vdocument.in/reader035/viewer/2022062823/587a26bd1a28abbd388b52b5/html5/thumbnails/47.jpg)
Stack Overflow dataset
• Questions, Answers and relative comments from the Stack overflow official (May 2014)
• 800 items in our dataset– We opportunistically chose items with the highest
positive and negative polarity scores (SentiStrength)– 200 questions and 200 follow-up askers’ comments
(400 cases overall) – 200 top answers and 200 follow-up answerers’
comments (400 additional cases).
![Page 48: The Challenges of Affect Detection in the Social Programmer Ecosystem](https://reader035.vdocument.in/reader035/viewer/2022062823/587a26bd1a28abbd388b52b5/html5/thumbnails/48.jpg)
Negative and Positive Sentiment in Stack Overflow
Negative Sentiment detected Positive Sentiment detected
In Questions Negative attitude towards technical issues- Emotions such as Frustration - Negative Opinions about, tools,
languages, IDE, etc.
Positive attitude towards technical issues- Positive Opinions about, tools,
languages, IDE, etc.
In Comments Negative attitude towards technical issues- Negative Opinions about, tools,
languages, IDE, etc.Negative attitude towards interlocutor- Emotions such as Anger
Positive attitude towards technical issues- Positive Opinions about, tools,
languages, IDE, etc.Positive attitude towards interlocutor- Emotion such as Liking, Appreciation
or Gratitude
![Page 49: The Challenges of Affect Detection in the Social Programmer Ecosystem](https://reader035.vdocument.in/reader035/viewer/2022062823/587a26bd1a28abbd388b52b5/html5/thumbnails/49.jpg)
Negative attitude towards the interlocutor
• Disliking- ‘Didn't notice the horrid inline jQuery’ - ‘Added some instructions for the really hopeless cases
• Anger- ‘Do u know the answer? If not .... then u might know how hard is homework. Ignorance is bliss’- ‘Arrrghhh, how I hate those people who downvote answers without leaving a comment as for why the downvote…’
![Page 50: The Challenges of Affect Detection in the Social Programmer Ecosystem](https://reader035.vdocument.in/reader035/viewer/2022062823/587a26bd1a28abbd388b52b5/html5/thumbnails/50.jpg)
• Detecting hostile behaviour‘Didn't notice the horrid inline jQuery’
‘Added some instructions for the really hopeless cases’
‘Do u know [the] answer ? If not .... then u might know how hard is homework. Ignorance is bliss’
Relevant for community moderators
Negative attitude towards the interlocutor
![Page 51: The Challenges of Affect Detection in the Social Programmer Ecosystem](https://reader035.vdocument.in/reader035/viewer/2022062823/587a26bd1a28abbd388b52b5/html5/thumbnails/51.jpg)
51 SSE ‘15
• Hostile behavior is a problem for the Stack Exchange community‘Do u know [the] answer ? If not .... then u might know how hard is homework. Ignorance is bliss’
‘Didn't notice the horrid inline jQuery’
‘Added some instructions for the really hopeless cases’
Negative attitude towards the interlocutor
![Page 52: The Challenges of Affect Detection in the Social Programmer Ecosystem](https://reader035.vdocument.in/reader035/viewer/2022062823/587a26bd1a28abbd388b52b5/html5/thumbnails/52.jpg)
Positive attitude towards the interlocutor
• Positive polarity, positive attitude– Gratitude
• Thanks, that helped. Case closed!• Worked perfectly, thanks a lot
• Often positive feedback is provided for discussion threads without an accepted answer
• ‘Thanks for the feedback, it was a pleasure! Could please accept my answer?’
• Remind information seekers to accept useful answers!
Calefato et al., 2015 - Mining Successful Answers in Stack Overflow. MSR 2015.s
![Page 53: The Challenges of Affect Detection in the Social Programmer Ecosystem](https://reader035.vdocument.in/reader035/viewer/2022062823/587a26bd1a28abbd388b52b5/html5/thumbnails/53.jpg)
• Frustration– I am not sure what I did in a previous life to warrant this, it must have been bad! I am getting buried in a world of xml […]
– This is driving me nutz :-( – There's no way to do this I'm afraid :(
• Negative opinions– They use it to clean up connections, which is really scary
– It is very painful to add multiple tickets to Trac
– I find this incredibly annoying with Dreamweaver
Attitude towards technical issues
![Page 54: The Challenges of Affect Detection in the Social Programmer Ecosystem](https://reader035.vdocument.in/reader035/viewer/2022062823/587a26bd1a28abbd388b52b5/html5/thumbnails/54.jpg)
Exploiting negative affect towards technical issues
Detect negative sentiment
Detect controversial technical issues or questions that are not exhaustively answered
![Page 55: The Challenges of Affect Detection in the Social Programmer Ecosystem](https://reader035.vdocument.in/reader035/viewer/2022062823/587a26bd1a28abbd388b52b5/html5/thumbnails/55.jpg)
• Negative polarity, positive attitude– Sorry-for‘To explain my regrettably unfriendly comment (sorry about that)’‘I’m afraid I can’t help you any further with this issue!’
Does polarity capture the true sentiment? (1/2)
![Page 56: The Challenges of Affect Detection in the Social Programmer Ecosystem](https://reader035.vdocument.in/reader035/viewer/2022062823/587a26bd1a28abbd388b52b5/html5/thumbnails/56.jpg)
• Humor, an open challenge for affective NLP • Irony
– ‘Haha that comment made me laugh my heart out. Makes me kind of proud of how horrible my code can be’
(positive sentiment = 3; negative sentiment = -4)
• Sarcasm– ‘Ignorance is bliss’ (positive sentiment = 5; negative sentiment = -3)
Does polarity capture the true sentiment? (2/2)
![Page 57: The Challenges of Affect Detection in the Social Programmer Ecosystem](https://reader035.vdocument.in/reader035/viewer/2022062823/587a26bd1a28abbd388b52b5/html5/thumbnails/57.jpg)
• Positive polarity– Gratitude‘Thanks for the feedback, it was a pleasure!’
• Negative polarity, positive attitude– Sorry-for‘I’m afraid I can’t help you any further with this issue!’
Emotion or politeness?
True emotions might not be necessary involved! *Politeness, ‘Behabitive’ speech acts*
Austin, 1962. How to Do Things with Words. Oxford University Press.
![Page 58: The Challenges of Affect Detection in the Social Programmer Ecosystem](https://reader035.vdocument.in/reader035/viewer/2022062823/587a26bd1a28abbd388b52b5/html5/thumbnails/58.jpg)
Domain dependence of sentiment-analysis
• False positives in negative sentiment detection– Domain lexicon‘What is the best way to kill a critical process’
– Contextual semantics‘I am missing a parenthesis. But where?
– Context of interaction (Q&A)‘I have a problem, […] please explain what is wrong’
![Page 59: The Challenges of Affect Detection in the Social Programmer Ecosystem](https://reader035.vdocument.in/reader035/viewer/2022062823/587a26bd1a28abbd388b52b5/html5/thumbnails/59.jpg)
Main findings
• Emotions emerge especially in comments– Seen as a ‘free zone’, comments do not contribute to
reputation• Need to distinguish between the emotions
towards somebody (i.e. the reader) vs. something (i.e. technical issues)– Relevant to different contexts and research goals
• Distinguish politeness from true emotions• Domain-dependence of sentiment analysis
– Need for tuning tools and linguistic resources
![Page 60: The Challenges of Affect Detection in the Social Programmer Ecosystem](https://reader035.vdocument.in/reader035/viewer/2022062823/587a26bd1a28abbd388b52b5/html5/thumbnails/60.jpg)
Project EmoQuesthttp://collab.di.uniba.it/emoquest
Investigating the Role of Emotions in Social Q&A Sites
![Page 61: The Challenges of Affect Detection in the Social Programmer Ecosystem](https://reader035.vdocument.in/reader035/viewer/2022062823/587a26bd1a28abbd388b52b5/html5/thumbnails/61.jpg)
Investigating the Role of Emotions in Social Q&A Sites
Impact in Computer-Supported Cooperative Work
HUMAN-COMPUTER INTERACTION
SOFTWARE ENGINEERING
PSYCHOLOGY LINGUISTICS
Project EmoQuesthttp://collab.di.uniba.it/emoquest
![Page 62: The Challenges of Affect Detection in the Social Programmer Ecosystem](https://reader035.vdocument.in/reader035/viewer/2022062823/587a26bd1a28abbd388b52b5/html5/thumbnails/62.jpg)
Expected Results
AFFECTIVE COMPUTING
• Empirically-driven guidelines for emotional interface design• Fine-grained Sentiment Analysis tools exploiting lexical semantics and
unsupervised approaches• Real-time modeling of affect in collaboration dialogues in social media
PSYCHOLINGUISTICS
• New psycholinguistic theories and models of social emotions in computer-mediated interaction
APPLIED LINGUISTICS
• New models of affect expression through language in social computing
SOCIAL SOFTWARE ENGINEERING
• Netiquette: pragmatic and actionable synthesis for emotional awareness• Software tools for embedding emotional intelligence into Q&A platforms
![Page 63: The Challenges of Affect Detection in the Social Programmer Ecosystem](https://reader035.vdocument.in/reader035/viewer/2022062823/587a26bd1a28abbd388b52b5/html5/thumbnails/63.jpg)