keepr presentation at mit cmsw

KeeprAlgorithm for Extracting Entities, Eyewitnesses and Amplifiers

Hong QuSeptember 19, 2013

http://twitpic.com/135xahttps://twitter.com/jkrums/status/1121915133

http://www.telegraph.co.uk/technology/twitter/4269765/New-York-plane-crash-Twitter-breaks-the-news-again.html

http://www2.sims.berkeley.edu/courses/is290-2/f04/sched.html

https://twitter.com/hqu/status/1745171763

Humans vs Machines

Pattern RecognitionValue Judgment

Computers count really, really fast!

What is a Tweet?

140 characters:● words● @user mentions● #hashtags● links

Hows does Keepr process tweets?

140 character * 100 tweets = 14,000 characters

❏ Parse it❏ Count it❏ Visualize it❏ Zoom in❏ Archive it

https://canvas.instructure.com/courses/812708/assignments/syllabus

Natural Language Processing

Humans are way better at at making value judgements and telling stories

Social media and the Boston bombings: When citizens and journalists cover the same story

Keepr’s Algorithm➔ Entity extraction

◆ Topics

➔ Media extraction◆ images, videos

➔ Link expansion◆ articles

➔ Conversation analysis◆ @ mentions

◆ source discovery

◆ amplification velocity

➔ Source verification◆ geo-location

◆ social media profiles

Topic Extraction by Term Frequency

http://trimc-nlp.blogspot.com/2013/04/tfidf-with-google-n-grams-and-pos-tags.html

https://twitter.com/hqu/stawww.cs.cornell.edu/home/kleinber/bhs.pdftus/1745171763

Journalists want

❏ Source discovery and curation

❏ Passive monitoring and alerts

❏ Saving and archiving

❏ Visualizations

❏ Parity with TweetDeck user interface

People want

“I want to catch up with a summary of key information about the breaking news story.”

I want to get a list of Twitter accounts who are official organizations related to that story.

My Musing

What’s next for keepr?

➔ refine algorithm➔ source classification➔ conversation analysis and visualization➔ archiving search results and tweets

Rolling out a Beta program for newsroomsSign up at www.keepr.com/beta

https://github.com/hqu/keepr

Verification Resources

● Verifying Social Media Content● verificationjunkie.tumblr.com● BBC processes for verifying social media content● Storyful’s validation process● InformaCam