New Directions in

Social Media

Tom ReamyChief Knowledge Architect

KAPS Group

http://www.kapsgroup.com

2

Agenda

Introduction – Social Media and Text Analytics Deeper than Positive-Negative

Building a Foundation for Social Media– Adding intelligence to BI, CI, and Sentiment

New Dimensions for Social Media– New taxonomies– Cognitive Science of Emotion

Applications– Expertise Analysis, Behavior Prediction, Crowd Sourcing– Integration with Predictive Analytics, Social Analytics

Conclusions

3

KAPS Group: General

Knowledge Architecture Professional Services – Network of Consultants Partners – SAS, SAP, IBM, FAST, Smart Logic, Concept Searching

– Attensity, Clarabridge, Lexalytics, Strategy – IM & KM - Text Analytics, Social Media, Integration Services:

– Taxonomy/Text Analytics development, consulting, customization– Text Analytics Fast Start – Audit, Evaluation, Pilot– Social Media: Text based applications – design & development

Clients: – Genentech, Novartis, Northwestern Mutual Life, Financial Times,

Hyatt, Home Depot, Harvard Business Library, British Parliament, Battelle, Amdocs, FDA, GAO, etc.

Applied Theory – Faceted taxonomies, complexity theory, natural categories, emotion taxonomies

Presentations, Articles, White Papers – http://www.kapsgroup.com

4

Introduction – Social Media & Text AnalyticsBeyond Simple Sentiment Beyond Good and Evil (positive and negative)

– Social Media is approaching next stage (growing up)– Where is the value? How get better results?

Importance of Context – around positive and negative words– Rhetorical reversals – “I was expecting to love it”– Issues of sarcasm, (“Really Great Product”), slanguage

Granularity of Application– Early Categorization – Politics or Sports

Limited value of Positive and Negative– Degrees of intensity, complexity of emotions and documents

Addition of focus on behaviors – why someone calls a support center – and likely outcomes

5

Introduction – Social Media & Text AnalyticsBeyond Simple Sentiment Two basic approaches:

– Statistical Signature of Bag of Words – Dictionary of positive & negative words

Beware automatic solutions – Accuracy, Depth Essential – need full categorization and concept extraction to get

full value from social media Categorization - Adds intelligence to all other components –

extraction, sentiment, and beyond Categorization/extraction rules – not just topical or sentiment

Combination with advanced social media analysis Opens up whole new worlds of applications

6

Text Analytics Foundation for Social MediaText Analytics Features Noun Phrase Extraction

– Catalogs with variants, rule based dynamic Sentiment Analysis

– Objects and phrases – statistics & rules – Positive and Negative Auto-categorization

– Training sets, Terms, Semantic Networks– Rules: Boolean - AND, OR, NOT– Advanced – DIST(#), ORDDIST#, PARAGRAPH, SENTENCE

Text Analytics as Foundation– Disambiguation - Identification of objects, events, context– Build rules based, not simply Bag of Individual Words

Case Study – Categorization & Sentiment

7

Case Study – Categorization & Sentiment

8

9

New Dimensions for Social Media

New Taxonomies – Appraisal – Appraisal Groups – Adjective and modifiers – “not very good”– Four types – Attitude, Orientation, Graduation, Polarity– Supports more subtle distinctions than positive or negative

Emotion taxonomies – Joy, Sadness, Fear, Anger, Surprise, Disgust– New Complex – pride, shame, embarrassment, love, awe– New situational/transient – confusion, concentration, skepticism

Beyond Keywords– Analysis of phrases, multiple contexts – conditionals, oblique – Analysis of conversations – dynamic of exchange, private language

10

New Applications in Social Media

Expertise Analysis – Experts think & write differently – process, chunks– Categorization rules for documents, authors, communities

Applications:– Business & Customer intelligence, Voice of the Customer– Deeper understanding of communities, customers – better models– Security, threat detection – behavior prediction, Are they experts?– Expertise location- Generate automatic expertise characterization

Behavior Prediction–TA and Predictive Analytics, Social Analytics Crowd Sourcing – technical support to Wiki’s Political – conservative and liberal minds/texts

– Disgust, shame, cooperation, openness

11

12

New Applications in Social Media

Analysis of Conversations– Techniques: self-revelation, humor, sharing of secrets, establishment

of informal agreements, private language– Detect relationships among speakers and changes over time– Strength of social ties, informal hierarchies

Combination with other techniques– Expertise Analysis – plus Influencers– Quality of communication (strength of social ties, extent of private

language, amount and nature of epistemic emotions – confusion+) Experiments - Pronoun Analysis – personality types Essay Evaluation Software - Apply to expertise characterization

– Model levels of chunking, procedure words over content

13

New Applications in Social MediaBehavior Prediction – Telecom Customer Service Problem – distinguish customers likely to cancel from mere threats Analyze customer support notes General issues – creative spelling, second hand reports Develop categorization rules

– First – distinguish cancellation calls – not simple– Second - distinguish cancel what – one line or all– Third – distinguish real threats

14

New Applications in Social MediaBehavior Prediction – Telecom Customer Service

Basic Rule– (START_20, (AND,  – (DIST_7,"[cancel]", "[cancel-what-cust]"),– (NOT,(DIST_10, "[cancel]", (OR, "[one-line]", "[restore]", “[if]”)))))

Examples:– customer called to say he will cancell his account if the does not stop receiving

a call from the ad agency. – cci and is upset that he has the asl charge and wants it off or her is going to

cancel his act– ask about the contract expiration date as she wanted to cxl teh acct

Combine sophisticated rules with sentiment statistical training and Predictive Analytics and behavior monitoring

15

New Applications: Wisdom of CrowdsCrowd Sourcing Technical Support Example – Android User Forum Develop a taxonomy of products, features, problem areas Develop Categorization Rules:

– “I use the SDK method and it isn't to bad a all. I'll get some pics up later, I am still trying to get the time to update from fresh 1.0 to 1.1.”

– Find product & feature – forum structure– Find problem areas in response, nearby text for solution

Automatic – simply expose lists of “solutions”– Search Based application

Human mediated – experts scan and clean up solutions

16

New Directions in Social MediaText Analytics, Text Mining, and Predictive Analytics Two Systems of the Brain

– Fast, System 1, Immediate patterns (TM)– Slow, System 2, Conceptual, reasoning (TA)

Text Analytics – pre-processing for TM– Discover additional structure in unstructured text– Behavior Prediction – adding depth in individual documents – New variables for Predictive Analytics, Social Media Analytics– New dimensions – 90% of information

Text Mining for TA– Semi-automated taxonomy development – Bottom Up- terms in documents – frequency, date, clustering– Improve speed and quality – semi-automatic

17

New Directions in Social MediaConclusions Social Media Analysis requires a hybrid approach

– Software, text analytics, human judgment – Contexts are essential

Text Analytics needs new techniques and structures – Smaller, more dynamic taxonomies– Focus – verbs, adjectives, broader contexts, activity

Value from Social Media Analysis requires new structures– Appraisal taxonomies, emotion taxonomies Plus– Better models of documents and authors – multi-dimensional

Result:– Enhanced sentiment analysis / social media applications– Develop whole range of new applications

Questions?

Tom Reamytomr@kapsgroup.com

KAPS Group

http://www.kapsgroup.com

Upcoming: Text Analytics Summit – Boston - June

Text Analytics World – Boston – October – Speakers?