open analytics: building effective frameworks for social media analysis
TRANSCRIPT
Building Effective Frameworks for Social Media Analysis
Open Analytics NYC – 11/08/2012
Agenda
• Social Media: An Intelligence perspective• Common Analytic Pitfalls• An Analytic Framework• Case Study: Brand Management– Problem Definition– Source Selection– Data Capture– Data Reporting– Data Analysis
• Ways Forward, Future Analysis• Questions?
Intelligence
• Intelligence is information that has been transformed to meet an operational need
Data Intelligence
Operational Lens
Intelligence Cycle
• No matter what methodology you use…
intelligence analysis is an iterative process.
Collect
Store
Analyze
Distribute
Social Media: Intelligence Perspective
HUMINT
SIGINTOSINT
• Social Media Intelligence is a combination of the best and worst features of:– HUMINT– OSINT– SIGINT
Social Media Analysis Goals
• Provide value to the organization – turn data into intelligence using an “operational lens”
• Ensure cyclical feedback occurs during collection, processing, analysis, and consumption
• Validate that a particular network is the right source of data for the questions you need answered
Common Misconceptions
• Social media is not a panacea– Not everyone uses social media– Users of social media use it unevenly– User behavior changes based on situations
• Just because people can talk about anything does not mean they talk about everything all the time.
Common Pitfalls
• Analyzing What Instead of Why:The important thing is often not what people are saying… but why they are saying it.
• Using the Wrong Analysis Tools:Reporting tools rarely help dig into the why. Many common tools, reports, and metrics are actually misleading:– Word clouds atomize message context– Sentiment metrics are often highly inaccurate– Information in aggregate hides more than it reveals
Pitfalls: An Example of the Challenge
Pitfalls: An Example of the Challenge
Dangers of Disintegration
Source: Matthew Auer, Policy Studies Journal, Volume 39, Issue 4, pages 709–736, Nov 2011
Analytic Framework
• Data Capture (DC)• Data Reporting (DR)• Data Analysis (DA)– What to measure– What the data is saying– What should be done based on the data
Source: Avinash Kaushik, Occam’s Razor Blog http://www.kaushik.net/avinash/web-analytics-consulting-framework-smarter-decisions/
Capture
ReportAnalyze
Choosing a Platform
• Social media, and the ways that it is used, is relatively new and evolving rapidly:– Static approaches to social media are flawed from
the outset– No one metric or set of metrics will always let you
know what is happening• Platforms need to be open and highly
adaptable to facilitate data capture, reporting, and analysis
Case Study: Brand Management
• Industry: Gaming– Experiencing 10% growth annually– Overall revenue expected to exceed $80 billion by
2014• In May, Zenimax Online Studios announced
Elder Scrolls Online– Elder Scrolls V: Skyrim 2nd largest game of 2011
Problem Definition
• Question: How can brand managers use social media to track and understand public attitudes toward a product?
• Challenge: Capture relevant information for social media sources.– Query too large = false positives– Query too small = miss potential information
• Twitter has excellent analytical potential:– Enormous volume, 400 million+ tweets per day– Large user base, 140 million+ accounts– Open API
• But its not without its limitations:– 140 characters– Limited historical (lookback) capacity without
using a 3rd party provider like DataSift or GNIP
Data Capture: Initial Query
• Twitter search for “Elder Scrolls Online”– Simplest possible way to access information– RSS feed for 10 days (Jun 27 – July 6 2012)
Data Capture: Entities & Associations
Hashtag TwitterHandle URL
Unstructured Keywords
Time / Date Stamp
WhoTwitterHandle
WhatHashtags, Keywords, URLs
WhenTime, Date
WhereGeo (if Available)
Data Reporting
Data Reporting
Data Analysis
• Analysis needs to be rooted in the operational need:
“How can I use social media to track and understand public attitudes toward my product”
• Emphasis on hypothesis generation, testing, and experimentation
Data Analysis: Hashtags
• Top hashtags were almost all generic or abstract– Undermines tracking and understanding– Top hashtags tied to franchise, not to the game
Hashtags#ElderScrolls #concept#games #nerd#online #geek#MMO #gamer#skyrim #ScreenShot
Data Analysis: Expanding the Query
• Hash tags from an initial subset of Tweets fed back into the initial query
Twitter Stream
Initial Query Results
Expanded Query Results
Data Analysis: Sentiment
• Sentiment analysis on small snippets of text like Tweets is generally poor
• Follow and convert linked URLs into derivative sources
• Larger text sources offer potential value with sentiment analysis that tweets alone cannot offer
Data Analysis: Sentiment
• Top negative and positive sentiment scores can provide a glimpse into aggregate attitudes
• Provide starting points for additional analysis
Next Steps: Shape the Conversation
• Create and promote hashtags that help shape the conversation and make it easier to collect and analyze the Twitter stream
Next Steps: Segment the Data
• Segment, or cluster, your data by:– User name or handle– Hashtags– Keywords– Geographic region
to explore patterns and trends at the micro level versus the entire dataset
Next Steps: Segment the Data
Next Steps: Graph Analysis
Lessons Learned
• Don’t:– Try drinking from a fire hose, sometimes less
really is more;– Use metrics you can’t tie to actions;– Use visualizations or reports that strip the data
from its context.
Lessons Learned
• Do:– Segment data rather than attempting to work in
the aggregate;– Look for the why behind the message;– Always return to the source material;– Explore alternative explanations;– Always consider the ultimate goal.
Thank You!
Craig Vitter
github.com/ikanow/Infinit.e