insight analysis on big data1
TRANSCRIPT
Peng Lai
Ling Zheng
Analysis on News Stories about “Big Data” in Popular Press
（ 2010-2015）
1. For years, Big Data has been applied in fields including health care, technology, advertising, banks and even politics. Topics around big data has been changing over time since 2010
Topics overview from 2010 to 2015
Big data swept through academia, business and government.
China and India grow fast in big data market.
Talents for big data are rare, especially in Java field.
Legal issues including privacy and security are concerned by government.
TopicsCloud, Storage, Solution, Enterprise, ApplicationBank, Marketing, Buy, Campaign, SalePatient, Health, Care, Medical, DiseaseJob, Student, Skill, JavaChina, City, India, Growth, GovernmentPrivacy, Law, Government, Protection, Legal
2. ‘Cloud’ occurs in all the years from 2010 to 2015. This shows that cloud computing is a core technology for the development of big data.
2010
2011
2012 2013
2014 2015
Cloud, Storage, Solution, Enterprise, Hadoop
Bank, Credit, Loan, Financial, Buy
Patient, Health, Care, Medical, Clinical
China, City, Government, Chinese, Province
Greenberg, Trump, Republicans, Presidential
Student, Science, University, Skill, Job
Campaign, Voter, Election, Political, Media
Job, Status, Experience, Java, Applicant
Solution, Could, Analytics, Storage, IBM
Patient, Health, Medical, Care, Treatment
Privacy, Law, Government, Protection, White House
China, City, Billion, Investor, Growth
Cloud, Storage, Application, Hadoop, Server
Surveillance, Government, Security, Law, Privacy
Patient, Health, Disease, Care, Medical
India, Cent, Growth, Revenue, Investment
Student, Science, University, Skill, Job
Ad, Consumer, Advertising, Marketing, Google
Cloud, Server, Network, Switch, Ethernet
Investor, IPO, Valley, Proofpoint
Risk, Insurer, Organisation, Analytics, Hadoop
Scientist, Science, Data Scientist, Researcher, University
Voter, Campaign, Political, Obama, Privacy
India, Cloud, Mobility, Skill, Wipro
Management, Oracle, Application, Organisation, Database
Amazon, Cloud, Netflix, Computing, Backup
Age, Manyika, Privacy, Technological, Thought
Email, Card, Breach, Law, Credit
Enterprise Software, Venture, Investor, Capital, Produce
Teradata, Egypt, Active, Warehouse
Challenge, Analytics Amounts, Technology, Value Real
Cloud ‘Could Computing’, Open Cloudera, Computing Built
3. From 2011 to 2012, terms like venture and investor appeared frequently indicating that big data dragged investors’ attention in the early stage
2010
2011
2012 2013
2014 2015
Cloud, Storage, Solution, Enterprise, Hadoop
Bank, Credit, Loan, Financial, Buy
Patient, Health, Care, Medical, Clinical
China, City, Government, Chinese, Province
Greenberg, Trump, Republicans, Presidential
Student, Science, University, Skill, Job
Campaign, Voter, Election, Political, Media
Job, Status, Experience, Java, Applicant
Solution, Could, Analytics, Storage, IBM
Patient, Health, Medical, Care, Treatment
Privacy, Law, Government, Protection, White House
China, City, Billion, Investor, Growth
Cloud, Storage, Application, Hadoop, Server
Surveillance, Government, Security, Law, Privacy
Patient, Health, Disease, Care, Medical
India, Cent, Growth, Revenue, Investment
Student, Science, University, Skill, Job
Ad, Consumer, Advertising, Marketing, Google
Cloud, Server, Network, Switch, Ethernet
Investor, IPO, Valley, Proofpoint
Risk, Insurer, Organisation, Analytics, Hadoop
Scientist, Science, Data Scientist, Researcher, University
Voter, Campaign, Political, Obama, Privacy
India, Cloud, Mobility, Skill, Wipro
Management, Oracle, Application, Organisation, Database
Amazon, Cloud, Netflix, Computing, Backup
Age, Manyika, Privacy, Technological, Thought
Email, Card, Breach, Law, Credit
Enterprise Software, Venture, Investor, Capital, Produce
Teradata, Egypt, Active, Warehouse
Challenge, Analytics Amounts, Technology, Value Real
Cloud ‘Could Computing’, Open Cloudera, Computing Built
4. Privacy issues in big data field arose in early 2011 and related laws were discussed afterwards
2010
2011
2012 2013
2014 2015
Cloud, Storage, Solution, Enterprise, Hadoop
Bank, Credit, Loan, Financial, Buy
Patient, Health, Care, Medical, Clinical
China, City, Government, Chinese, Province
Greenberg, Trump, Republicans, Presidential
Student, Science, University, Skill, Job
Campaign, Voter, Election, Political, Media
Job, Status, Experience, Java, Applicant
Solution, Could, Analytics, Storage, IBM
Patient, Health, Medical, Care, Treatment
Privacy, Law, Government, Protection, White House
China, City, Billion, Investor, Growth
Cloud, Storage, Application, Hadoop, Server
Surveillance, Government, Security, Law, Privacy
Patient, Health, Disease, Care, Medical
India, Cent, Growth, Revenue, Investment
Student, Science, University, Skill, Job
Ad, Consumer, Advertising, Marketing, Google
Cloud, Server, Network, Switch, Ethernet
Investor, IPO, Valley, Proofpoint
Risk, Insurer, Organisation, Analytics, Hadoop
Scientist, Science, Data Scientist, Researcher, University
Voter, Campaign, Political, Obama, Privacy
India, Cloud, Mobility, Skill, Wipro
Management, Oracle, Application, Organisation, Database
Amazon, Cloud, Netflix, Computing, Backup
Age, Manyika, Privacy, Technological, Thought
Email, Card, Breach, Law, Credit
Enterprise Software, Venture, Investor, Capital, Produce
Teradata, Egypt, Active, Warehouse
Challenge, Analytics Amounts, Technology, Value Real
Cloud ‘Could Computing’, Open Cloudera, Computing Built
5. Talents in big data became a focus since 2012. Academic institutions, such as universities, pay efforts to offer students big data skills to satisfy the increasing big data job market.
2010
2011
2012 2013
2014 2015
Cloud, Storage, Solution, Enterprise, Hadoop
Bank, Credit, Loan, Financial, Buy
Patient, Health, Care, Medical, Clinical
China, City, Government, Chinese, Province
Greenberg, Trump, Republicans, Presidential
Student, Science, University, Skill, Job
Campaign, Voter, Election, Political, Media
Job, Status, Experience, Java, Applicant
Solution, Could, Analytics, Storage, IBM
Patient, Health, Medical, Care, Treatment
Privacy, Law, Government, Protection, White House
China, City, Billion, Investor, Growth
Cloud, Storage, Application, Hadoop, Server
Surveillance, Government, Security, Law, Privacy
Patient, Health, Disease, Care, Medical
India, Cent, Growth, Revenue, Investment
Student, Science, University, Skill, Job
Ad, Consumer, Advertising, Marketing, Google
Cloud, Server, Network, Switch, Ethernet
Investor, IPO, Valley, Proofpoint
Risk, Insurer, Organisation, Analytics, Hadoop
Scientist, Science, Data Scientist, Researcher, University
Voter, Campaign, Political, Obama, Privacy
India, Cloud, Mobility, Skill, Wipro
Management, Oracle, Application, Organisation, Database
Amazon, Cloud, Netflix, Computing, Backup
Age, Manyika, Privacy, Technological, Thought
Email, Card, Breach, Law, Credit
Enterprise Software, Venture, Investor, Capital, Produce
Teradata, Egypt, Active, Warehouse
Challenge, Analytics Amounts, Technology, Value Real
Cloud ‘Could Computing’, Open Cloudera, Computing Built
6. Health care industry is one of first industries to apply big data
2010
2011
2012 2013
2014 2015
Cloud, Storage, Solution, Enterprise, Hadoop
Bank, Credit, Loan, Financial, Buy
Patient, Health, Care, Medical, Clinical
China, City, Government, Chinese, Province
Greenberg, Trump, Republicans, Presidential
Student, Science, University, Skill, Job
Campaign, Voter, Election, Political, Media
Job, Status, Experience, Java, Applicant
Solution, Could, Analytics, Storage, IBM
Patient, Health, Medical, Care, Treatment
Privacy, Law, Government, Protection, White House
China, City, Billion, Investor, Growth
Cloud, Storage, Application, Hadoop, Server
Surveillance, Government, Security, Law, Privacy
Patient, Health, Disease, Care, Medical
India, Cent, Growth, Revenue, Investment
Student, Science, University, Skill, Job
Ad, Consumer, Advertising, Marketing, Google
Cloud, Server, Network, Switch, Ethernet
Investor, IPO, Valley, Proofpoint
Risk, Insurer, Organisation, Analytics, Hadoop
Scientist, Science, Data Scientist, Researcher, University
Voter, Campaign, Political, Obama, Privacy
India, Cloud, Mobility, Skill, Wipro
Management, Oracle, Application, Organisation, Database
Amazon, Cloud, Netflix, Computing, Backup
Age, Manyika, Privacy, Technological, Thought
Email, Card, Breach, Law, Credit
Enterprise Software, Venture, Investor, Capital, Produce
Teradata, Egypt, Active, Warehouse
Challenge, Analytics Amounts, Technology, Value Real
Cloud ‘Could Computing’, Open Cloudera, Computing Built
7. Both India and China have made rapid growth in IT field recent years. It can be seen in big data as well.
2010
2011
2012 2013
2014 2015
Cloud, Storage, Solution, Enterprise, Hadoop
Bank, Credit, Loan, Financial, Buy
Patient, Health, Care, Medical, Clinical
China, City, Government, Chinese, Province
Greenberg, Trump, Republicans, Presidential
Student, Science, University, Skill, Job
Campaign, Voter, Election, Political, Media
Job, Status, Experience, Java, Applicant
Solution, Could, Analytics, Storage, IBM
Patient, Health, Medical, Care, Treatment
Privacy, Law, Government, Protection, White House
China, City, Billion, Investor, Growth
Cloud, Storage, Application, Hadoop, Server
Surveillance, Government, Security, Law, Privacy
Patient, Health, Disease, Care, Medical
India, Cent, Growth, Revenue, Investment
Student, Science, University, Skill, Job
Ad, Consumer, Advertising, Marketing, Google
Cloud, Server, Network, Switch, Ethernet
Investor, IPO, Valley, Proofpoint
Risk, Insurer, Organisation, Analytics, Hadoop
Scientist, Science, Data Scientist, Researcher, University
Voter, Campaign, Political, Obama, Privacy
India, Cloud, Mobility, Skill, Wipro
Management, Oracle, Application, Organisation, Database
Amazon, Cloud, Netflix, Computing, Backup
Age, Manyika, Privacy, Technological, Thought
Email, Card, Breach, Law, Credit
Enterprise Software, Venture, Investor, Capital, Produce
Teradata, Egypt, Active, Warehouse
Challenge, Analytics Amounts, Technology, Value Real
Cloud ‘Could Computing’, Open Cloudera, Computing Built
8. Big data works in politics since 2012 which is intrigued by Obama’s successful political campaigns by implementing and using big data. Trump became a big word in big data In 2015.
2010
2011
2012 2013
2014 2015
Cloud, Storage, Solution, Enterprise, Hadoop
Bank, Credit, Loan, Financial, Buy
Patient, Health, Care, Medical, Clinical
China, City, Government, Chinese, Province
Greenberg, Trump, Republicans, Presidential
Student, Science, University, Skill, Job
Campaign, Voter, Election, Political, Media
Job, Status, Experience, Java, Applicant
Solution, Could, Analytics, Storage, IBM
Patient, Health, Medical, Care, Treatment
Privacy, Law, Government, Protection, White House
China, City, Billion, Investor, Growth
Cloud, Storage, Application, Hadoop, Server
Surveillance, Government, Security, Law, Privacy
Patient, Health, Disease, Care, Medical
India, Cent, Growth, Revenue, Investment
Student, Science, University, Skill, Job
Ad, Consumer, Advertising, Marketing, Google
Cloud, Server, Network, Switch, Ethernet
Investor, IPO, Valley, Proofpoint
Risk, Insurer, Organisation, Analytics, Hadoop
Scientist, Science, Data Scientist, Researcher, University
Voter, Campaign, Political, Obama, Privacy
India, Cloud, Mobility, Skill, Wipro
Management, Oracle, Application, Organisation, Database
Amazon, Cloud, Netflix, Computing, Backup
Age, Manyika, Privacy, Technological, Thought
Email, Card, Breach, Law, Credit
Enterprise Software, Venture, Investor, Capital, Produce
Teradata, Egypt, Active, Warehouse
Challenge, Analytics Amounts, Technology, Value Real
Cloud ‘Could Computing’, Open Cloudera, Computing Built
9. Hot topics of big data “technology” have been changed from software, cloud to internet of things
Technical Review
Prepare Data: split data sets to year by year
Parsing and Filtering 1:Set Synonyms, Ignore of Entities,
Stop List
Text Mining:Compare Text Cluster and
Text Topics Results
Technical Review
Text Filtering 2:Drop Words that are not
Informative based on previous results
Check Text Mining Results:Check words in Filter Viewer
Discover Concept Links
Technical Review
High frequency words comparison &
percentage of occurrence for certain term comparison
through time
Insights Building