skills required to integrate big data into public opinion ...skills required to integrate big data...
TRANSCRIPT
![Page 1: Skills Required to Integrate Big Data into Public Opinion ...Skills Required to Integrate Big Data into Public Opinion Research Abe Usher Chief Technology Officer, HumanGeo ... Data](https://reader033.vdocument.in/reader033/viewer/2022060300/5f0832647e708231d420d153/html5/thumbnails/1.jpg)
Skills Requiredto Integrate Big Data
into Public Opinion Research
Abe UsherChief Technology Officer, HumanGeo
![Page 2: Skills Required to Integrate Big Data into Public Opinion ...Skills Required to Integrate Big Data into Public Opinion Research Abe Usher Chief Technology Officer, HumanGeo ... Data](https://reader033.vdocument.in/reader033/viewer/2022060300/5f0832647e708231d420d153/html5/thumbnails/2.jpg)
• Big data demystified• Four layers of big data• Skills required• Easter eggs
Outline
![Page 3: Skills Required to Integrate Big Data into Public Opinion ...Skills Required to Integrate Big Data into Public Opinion Research Abe Usher Chief Technology Officer, HumanGeo ... Data](https://reader033.vdocument.in/reader033/viewer/2022060300/5f0832647e708231d420d153/html5/thumbnails/3.jpg)
Big Data Today
Courtesy of Google Trends: http://goo.gl/4H8Ttd
![Page 4: Skills Required to Integrate Big Data into Public Opinion ...Skills Required to Integrate Big Data into Public Opinion Research Abe Usher Chief Technology Officer, HumanGeo ... Data](https://reader033.vdocument.in/reader033/viewer/2022060300/5f0832647e708231d420d153/html5/thumbnails/4.jpg)
Big Data & Public Opinionvs Pop Culture
Courtesy of Google Trends: http://goo.gl/QHIQcN
![Page 5: Skills Required to Integrate Big Data into Public Opinion ...Skills Required to Integrate Big Data into Public Opinion Research Abe Usher Chief Technology Officer, HumanGeo ... Data](https://reader033.vdocument.in/reader033/viewer/2022060300/5f0832647e708231d420d153/html5/thumbnails/5.jpg)
5
Big data: De-mystified
What is big data?
What is Hadoop File System? (HDFS)
What is Hadoop MapReduce? (MR)
![Page 6: Skills Required to Integrate Big Data into Public Opinion ...Skills Required to Integrate Big Data into Public Opinion Research Abe Usher Chief Technology Officer, HumanGeo ... Data](https://reader033.vdocument.in/reader033/viewer/2022060300/5f0832647e708231d420d153/html5/thumbnails/6.jpg)
6
20th Century model of analysis
Tracy Morrow (aka “Ice T”)
How can you identify alegitimate hip-hop artist(versus someone who just getsup and rhymes)?
http://www.npr.org/2005/08/30/4824690/original-gangster-rapper-and-actor-ice-t
![Page 7: Skills Required to Integrate Big Data into Public Opinion ...Skills Required to Integrate Big Data into Public Opinion Research Abe Usher Chief Technology Officer, HumanGeo ... Data](https://reader033.vdocument.in/reader033/viewer/2022060300/5f0832647e708231d420d153/html5/thumbnails/7.jpg)
7
Tracy Morrow (aka “Ice T”)
How can you identify alegitimate hip-hop artist(versus someone who just getsup and rhymes)?
“Game knows game, baby.”
20th Century model of analysis
![Page 8: Skills Required to Integrate Big Data into Public Opinion ...Skills Required to Integrate Big Data into Public Opinion Research Abe Usher Chief Technology Officer, HumanGeo ... Data](https://reader033.vdocument.in/reader033/viewer/2022060300/5f0832647e708231d420d153/html5/thumbnails/8.jpg)
8
Tracy Morrow (aka “Ice T”)
How can you identify alegitimate hip-hop artist(versus someone who just getsup and rhymes)?
“If you have expert knowledge,then you are capable ofanswering complex questionsby interpreting domain specificinformation.” [paraphrased]
20th Century model of analysis
![Page 9: Skills Required to Integrate Big Data into Public Opinion ...Skills Required to Integrate Big Data into Public Opinion Research Abe Usher Chief Technology Officer, HumanGeo ... Data](https://reader033.vdocument.in/reader033/viewer/2022060300/5f0832647e708231d420d153/html5/thumbnails/9.jpg)
New model of analysis
Peter Gibbons hatches a plot towrite a computer virus that grabfractions of a penny from acorporate retirement account.http://goo.gl/rDg1U
9
Office Space
Takeaway point: Little bits of value (information)provide deep insights in the aggregate
Takeaway point: Little bits of value (information)provide deep insights in the aggregate
![Page 10: Skills Required to Integrate Big Data into Public Opinion ...Skills Required to Integrate Big Data into Public Opinion Research Abe Usher Chief Technology Officer, HumanGeo ... Data](https://reader033.vdocument.in/reader033/viewer/2022060300/5f0832647e708231d420d153/html5/thumbnails/10.jpg)
New model of analysis
10
Takeaway point: Hadoop simplifies the creation of massive counting machinesTakeaway point: Hadoop simplifies the creation of massive counting machines
![Page 11: Skills Required to Integrate Big Data into Public Opinion ...Skills Required to Integrate Big Data into Public Opinion Research Abe Usher Chief Technology Officer, HumanGeo ... Data](https://reader033.vdocument.in/reader033/viewer/2022060300/5f0832647e708231d420d153/html5/thumbnails/11.jpg)
11
Big Data: Layers
![Page 12: Skills Required to Integrate Big Data into Public Opinion ...Skills Required to Integrate Big Data into Public Opinion Research Abe Usher Chief Technology Officer, HumanGeo ... Data](https://reader033.vdocument.in/reader033/viewer/2022060300/5f0832647e708231d420d153/html5/thumbnails/12.jpg)
12
Big Data: Layers
Data Source(s)Data Source(s)
Data StorageData Storage
Data AnalysisData Analysis
Data OutputData Output
Examples: geolocated social media(Proxy variable for behavior of interest)
Example: Hadoop Distributed File System
Example: Hadoop MapReduce
Example: map visualization
![Page 13: Skills Required to Integrate Big Data into Public Opinion ...Skills Required to Integrate Big Data into Public Opinion Research Abe Usher Chief Technology Officer, HumanGeo ... Data](https://reader033.vdocument.in/reader033/viewer/2022060300/5f0832647e708231d420d153/html5/thumbnails/13.jpg)
Four roles related to big data:each provide different skills
![Page 14: Skills Required to Integrate Big Data into Public Opinion ...Skills Required to Integrate Big Data into Public Opinion Research Abe Usher Chief Technology Officer, HumanGeo ... Data](https://reader033.vdocument.in/reader033/viewer/2022060300/5f0832647e708231d420d153/html5/thumbnails/14.jpg)
14
Skills by role
System Administrator• Storage systems (MySQL, Hbase,
Spark)• Cloud computing:
• Amazon Web Services (AWS)• Google Compute Engine
• Hadoop ecosystem
Computer scientist• Data preparation• MapReduce algorithms• Python/R programming• Hadoop ecosystem
![Page 15: Skills Required to Integrate Big Data into Public Opinion ...Skills Required to Integrate Big Data into Public Opinion Research Abe Usher Chief Technology Officer, HumanGeo ... Data](https://reader033.vdocument.in/reader033/viewer/2022060300/5f0832647e708231d420d153/html5/thumbnails/15.jpg)
Big data enablesnew insights into human behavior
Geolocated social media activity in Washington DCduring a 15 minute time period generated by MR. TweetMap
![Page 17: Skills Required to Integrate Big Data into Public Opinion ...Skills Required to Integrate Big Data into Public Opinion Research Abe Usher Chief Technology Officer, HumanGeo ... Data](https://reader033.vdocument.in/reader033/viewer/2022060300/5f0832647e708231d420d153/html5/thumbnails/17.jpg)
17
Easter Eggs
Google “do a barrel roll”
“Google gravity”
Google search in Klingon www.google.com/?hl=kn