maj david robinson online behavior analysis and modeling methodology (obamm)
TRANSCRIPT
Maj David Robinson
Online Behavior Analysis and Modeling Methodology
(OBAMM)
Overview
• Goals• Methodology
– Collect– Cluster/Categorize– Classify– Correlate
• Future Work
Goals
• Creation of online user fingerprints based on usage patterns and interest areas
• Insider threat detection, policy violation, social network analysis, business process analysis
Collect
• Active or passive
• Need to see “unique” users (in general)
• HTTP/GET Requests– Minimize chaff– Interested in src, dest, timestamp
08/24/07 17:30:06.203412,192.168.10.10, GET,www.ists.dartmouth.edu
Timestamp Source Destination
Cluster/Categorize
• Initial cluster on categories– Utilize reverse category lookup
• Then cluster on:– Content– Meta-data
Categorization Data• ODP – Open Directory Project
– 4,830,584 sites categorized
– 16 “core” categories
– Multi-language
• Blacklists– Approx 40 MB of URLs
– Approx 37 categories
– Capture “less common” URLs• Porn, hacking, drugs, weapons, warez• Shopping, webmail, finance, dating
ODP Data<Topic r:id="Top/Arts/Movies/Titles/1/10_Rillington_Place">
<catid>205108</catid>
<link r:resource="http://www.britishhorrorfilms.co.uk/rillington.shtml"/>
<link r:resource="http://www.shoestring.org/mmi_revs/10-rillington-place.html"/>
</Topic>
<ExternalPage about="http://www.britishhorrorfilms.co.uk/rillington.shtml">
<d:Title>British Horror Films: 10 Rillington Place</d:Title>
<d:Description>Review which looks at plot especially the shocking features of it.</d:Description> <topic>Top/Arts/Movies/Titles/1/10_Rillington_Place</topic>
</ExternalPage>
<ExternalPage about="http://www.shoestring.org/mmi_revs/10-rillington-place.html">
<d:Title>MMI Movie Review: 10 Rillington Place</d:Title>
<d:Description>Review includes plot, real life story behind the film and realism in the film.</d:Description>
<topic>Top/Arts/Movies/Titles/1/10_Rillington_Place</topic>
</ExternalPage> ge>
Blacklist Data
• Shopping Domainsbestbudsgarden.com
bestbudsgarden.pointshop.com
bestbuket.ru
bestbung.com
bestbuy.com
• Shopping URLs00sake.com/htdocs
horseland.com.au/shop
balance.com/products/shop
theawningman.com.au/catalog/
wigglywigglers.co.uk/shop
Single URL
• http://www.dartmouth.edu
Dartmouth College
New Hampshire
United States
North America
Colleges & Universities
Education
Reference
Multiple URLs
• news.google.com/?topic=s
• www.kicksology.com
• www.beckett.com/userpages/Meibin.html
• www.razbet.com
• www.fhlsim.com
Multiple URLs
Portals
News and Media
Resources
Sports
Footwear
Apparel
Basketball
Sports
Shopping
Basketball
Cards
Sports
Collecting
Recreation
Basketball
Tipping & Handicapping
Sports
Gambling
Games
Software
Simulation
Hockey
Fantasy
Sports
news.google.com/?topic=s www.kicksology.com becket.com/userpages www.razbet.com www.fhlsim.com
Multiple URLs
Portals
News and Media
Resources
Footwear
Apparel
Basketball
Shopping
Basketball
Cards
Sports
Collecting
Recreation
Basketball
Tipping & Handicapping
Gambling
Games
Software
Simulation
Hockey
Fantasy
Cluster/Categorize
• Issues– Not all URLs categorized– URLs in multiple categories
• Document Clustering Methods– Page Content– Page meta data
User Representation
User Details
Correlate
• Temporal
• Similarity
• Interactions
Temporal Relations
Time
Fre
quen
cy
ShoppingComputerAdultArt
Correlate
User 1
User 3
User 4
User 6
User 10
User 8
User 5
User 9
User 7
User 2
Correlate
User 1
User 3
User 4
User 6
User 10
User 8
User 5
User 9
User 7
User 2
Correlate
User 1
User 3
User 4
User 6
User 10
User 8
User 5
User 9
User 7
User 2
Engineers
Med Students
Business Students
Future Work
• Search terms
• Categorize functionalities
• Temporal correlation
• Social Network Analysis
• Pruning
• Data
