personalized filtering of twitter stream
DESCRIPTION
With the rapid growth in users on social networks, there is a corresponding increase in user-generated content, in turn resulting in information overload. On Twitter, for example, users tend to receive un- interested information due to their non-overlapping interests from the people whom they follow. In this paper we present a Semantic Web ap- proach to filter public tweets matching interests from personalized user profiles. Our approach includes automatic generation of multi-domain and personalized user profiles, filtering Twitter stream based on the gen- erated profiles and delivering them in real-time. Given that users inter- ests and personalization needs change with time, we also discuss how our application can adapt with these changes.TRANSCRIPT
Personalized Filtering of the Twitter Stream
Pavan Kapanipathi 1,2, Fabrizio Orlandi1, Amit Sheth2 ,Alexandre Passant 1
11 Digital Enterprise Research Institute, Galway – Ireland
2 Kno.e.sis, Dayton, OH- USA
2
Motivation
Twitter – GrowthInformation Overload
http://www.cmswire.com/cms/customer-experience/35-key-twitter-statistics-infographic-012384.php
3
Motivation• How many people should I follow ?• Am I receiving latest/complete information ?
4
BackgroundTwarql – Streaming annotated tweets
Semantic Web TechnologiesAnnotate Tweets (DBpedia Entities)Filter Stream using SPARQL Queries formulated
Example:Stream all the tweets related to Semantic Web generated
in Germany?tweet moat:taggedWith ?topic .?topic dcterms:subject category:Semantic_Web .?tweet sioc:has_creator ?user .?user geonames:locatedIn dbpedia:Germany .
5
Approach -- Overview
Football
Apple
The new iPhone has a
3.5-inch screen,
released today User Profiles
Filter
Broadcast
6
Architecture
Semantic Filter
Semantic Hub
Profile Generator
RDF
ANNOTATOR
RDF
RSS
Store and Query Topics
Notify Update
Fetch Updates
Get Interested Subscribers
Push
Upd
ates
Subs
cribe
Create Profile
Store FOAF
The new iPhone has a
3.5-inch screen, released today
Annotate: iPhone?user foaf:interest
dbPedia:iPhoneUnion
?user foaf:interestCategory:Apple
Get Subscribers based on
preference
Push Updates to Interested
Users
Update RSS
7
ContributionProfile Generator
Automatic generation of User Profiles
Semantic FilterAnnotating Twitter Stream with concepts from
Linked Open Data
Semantic HubDelivering tweets to appropriate Interested Users
(near real-time)
8
Profile Generator
Semantic Filter
Semantic Hub
Profile Generator
RDF
ANNOTATOR
RDF
RSS
Store and Query Topics
Notify Update
Fetch Updates
Get Interested Subscribers
Push
Upd
ates
Subs
cribe
Create Profile
Store FOAFUpdate RSS
9Social Networking Sites as Walled Gardens by David Simonds (Used with permission)
Disconnected Social websites
Isolated data silos
Profile Generator
10
User Profile
Interlink social websites
Merge and model user data
Personalise users’ experience using their profile
Integration&
User Modelling
Recommendations
Search Personalisation
Adaptive Systems
11
Profile GeneratorData Extraction
Twitter, Facebook, LinkedIn Example: Tweets, FB Likes
Profile Generation Interests extracted from collected data
Entity spotting (user generated data)Explicit interests specified by user (Facebook likes etc)
Weighted Interests
Semantic Representation of Profiles FOAF profile
12
Semantic Filter
Semantic Filter
Semantic Hub
Profile Generator
RDF
ANNOTATOR
RDF
RSS
Store and Query Topics
Notify Update
Fetch Updates
Get Interested Subscribers
Push
Upd
ates
Subs
cribe
Create Profile
Store FOAFUpdate RSS
13
Semantic FilterTwitter Streaming API
Microblog Metadata Twitter provides metadata
Author, date, location etc.. Metadata Extracted
DBPedia Entities, URLs
Generate SPARQL Query representing interested Users Retrieved at Semantic Hub
14
Semantic Filter – RDF<http://twitter.com/rob/statuses/123456789>
rdf:type sioct:MicroblogPost ;sioc:content "P Groth and Y Gil, Linked Data for Network
Science http://bit.ly/owxcJg #iswc2011 #lisc2011 #linkeddata-“ �sioc:has_creator <http://twitter.com/rob> ;foaf:maker <http://example.org/rob> ;moat:taggedWith dbpedia:Linked_Data ;moat:taggedWith dbpedia:Network_Science ;
<http://twitter.com/rob/statuses/123456789#presence>rdf:type opo:OnlinePresence ;opo:startTime 2010-03-20T17:55:42+00:00 ;�opo:customMessage <http://twitter.com/rob/statuses/
123456789> .
<http://twitter.com/rob> geonames:locatedIn Dbpedia:Ohio .[...]
15
Semantic Filter– SPARQL Query
Generate SPARQL QueriesRepresenting FOAF of interested users
SELECT ?user WHERE { { ?user foaf:interest
dbpedia:Linked_Data .} UNION{ ?user foaf:interest
dbpedia:Network_Science .} }
16
Semantic Hub
Semantic Filter
Semantic Hub
Profile Generator
RDF
ANNOTATOR
RDF
RSS
Store and Query Topics
Notify Update
Fetch Updates
Get Interested Subscribers
Push
Upd
ates
Subs
cribe
Create Profile
Store FOAFUpdate RSS
17
PubSubHubbub Protocol
PubSubHubbub is an extension to RSS/Atom Open, web hook based, pubsub protocol for Real-time
notification of updates
Drawback Publisher has no control over the dissemination of his content
Extension – Semantic Hub Publisher controlled dissemination SPARQL Query representing the subset of target subscribers
18
PubSubHubbub Protocol Extension
Pub
Sub - A
Sub - B
Sub - C
Sub - D
Hey I have new content for feed X +
my preference Y
Social Graph
Get the subscribers of Pub
whose profile matches
preference Y
Here is the new
content of feed X
Give me the
new content
Here it is
Semantic Hub
19
Semantic HubRSS Extension
Preference – to include the sparql queries
Push content FOAF profiles of the subscribers are matched with the
preference Interested subscribers receive the content
Accepted as a full paper in the In-Use track at ISWC 2011
20
Conclusion Single consistent profile rather than profiles on multiple social
networks User Profile Generation
Architecture for Personalization of twitter stream Reduce load on users to follow others
Public tweets streamed Access to information from experts in domains
Are you following experts in your domain of interest? Experts public tweets will be streamed
Dynamic groups of users Interest Driven
Future work -- Why RDFTwarql features
Concept feeds as interests of the users
22
Future WorkPeriodic FOAF profile generation for users
Twitter Stream reflecting the changing interests
Extending to other social networks (G+, FB)
23
Contact us on Twitter @pavankaps@badmotorf@terraces@amit_p
Email: {pavan, amit}@knoesis.org {fabrizio.orlandi, alexandre.passant}@deri.org
This work is funded by (1) Science Foundation Ireland under grant number SFI/08/CE/I1380 (Lıon 2) and by an IRCSET scholarship supported by Cisco Systems (2) Social Media Enhanced Organizational Sensemaking in Emergency Response, National Science Foundation under award IIS-1111182, 09/01/2011 - 08/31/2014.
Thanks
24
25
Architecture
26
AgendaMotivationContributionArchitectureConclusion Future Work
27
Weighing function based on RTs and other active engagements of the user