© 2013 ibm corporation version 1.0 the new eye insight through big data and analytics: a case study...
TRANSCRIPT
© 2013 IBM CorporationVersion 1.0
The New EyeInsight through Big Data and Analytics: A Case Study on Citizen Sentiment Analysis
Sandipan Sarkar, Executive ArchitectGlobal Government Center of Competence, IBM
Mobile: +91.98302.31038Email: [email protected]
© 2013 IBM Corporation2
The world is changing – there is an explosion of data
The volume, variety, and velocity of data is growing at an unprecedented rate.
1.3 Billion RFID tags in 200530 billion RFIDtags today
1 trillion devices are connected to the Internet
1 trillion devices are connected to the Internet
Twitter processes 12+ terabytes ofdata every day
80% of world’s information is unstructured content 25+ terabytes of
log data every day
4.6 billion camera phones world wide
76 million smart metersin 2009 … 200M by
2014
The Information base of the world doubles every 11 hours
The Information base of the world doubles every 11 hours
© 2013 IBM Corporation3
Why the data is “big” now?
Characteristics of Big Data
Source: IBM methodology
© 2013 IBM Corporation4
The challenge is also an opportunity: move analytics closer to big data
BI / Reportin
g
BI / Reporting
Exploration / Visualization
FunctionalApp
IndustryApp
Predictive Analytics
Content Analytics
Analytic Applications
IBM Big Data Platform
Systems Management
Application Development
Visualization & Discovery
Accelerators
Information Integration & Governance
HadoopSystem
Stream Computing
Data Warehouse
New analytic applications drive the requirements for a big data platform
• Integrate and manage the full variety, velocity and volume of data
• Apply advanced analytics to information in its native form
• Visualize all available data for ad-hoc analysis
• Development environment for building new analytic applications
• Workload optimization and scheduling
• Security and Governance
© 2013 IBM Corporation5
Governments are trying to move closer to citizens – sentiment analysis from social media can be a useful vehicle in this journey
How do citizens feel about the agency’s new programmes and policies?
What are the most talked about programmes? Is it good or bad?
What are the most positively talked about attributes in the agency’s programmes? Can the agency replicate it to other programmes?
Is there negative chatter that the agency should respond to?
Who are advocates and skeptics of the agency?
Where the agency should be actively listening?
Source: Gartner Open Government Maturity Model
Building such insight is a daunting task because of the volume, variety, velocity and veracity of information that social media can generate.
© 2013 IBM Corporation6
Citizen sentiment analysis in social media: a confluence of big data, natural language processing, information extraction and visual analytics
IBM® Cognos Consumer Insight
AdminUI Analysis UI
Hadoop
IBM® General Parallel File System
Data Fetcher
Topic Extractor
FlowManager
SystemT(Information Extractor)
Uploader
Lucene (Search Engine)
Topic Modeller
Administrator Analyzer
© 2013 IBM Corporation7
Citizen sentiment analysis in social media for a major social benefits organisation in US revealed valuable insights
Key Observations– Benefits and Services received more
than double the amount of coverage than Healthcare related buzz
– Disability Compensation and Employment Benefits are the most talked about topics among all the benefits and services offered by the agency. Mental Health is the most talked about topic among Healthcare initiatives
– Disability Compensation, Insurance, and Pension contribute heavily towards negative sentiments, whereas Employment Benefits, Dependent’s Assistance, and Home Loan Benefits are talked in positive light.
– July 2012 hit all time high negative sentiment, because of a single news
Root Cause Analysis– The agency was suffering from huge
back-logs in claims processing– Awareness of benefits and services
was little among its clients. Agency needed to transform its outreach activities.
– Agency had a poor social media strategy.
