maj david robinson online behavior analysis and modeling methodology (obamm)

Maj David Robinson

Online Behavior Analysis and Modeling Methodology

(OBAMM)

Overview

• Goals• Methodology

– Collect– Cluster/Categorize– Classify– Correlate

• Future Work

Goals

• Creation of online user fingerprints based on usage patterns and interest areas

• Insider threat detection, policy violation, social network analysis, business process analysis

Collect

• Active or passive

• Need to see “unique” users (in general)

• HTTP/GET Requests– Minimize chaff– Interested in src, dest, timestamp

08/24/07 17:30:06.203412,192.168.10.10, GET,www.ists.dartmouth.edu

Timestamp Source Destination

Cluster/Categorize

• Initial cluster on categories– Utilize reverse category lookup

• Then cluster on:– Content– Meta-data

Categorization Data• ODP – Open Directory Project

– 4,830,584 sites categorized

– 16 “core” categories

– Multi-language

• Blacklists– Approx 40 MB of URLs

– Approx 37 categories

– Capture “less common” URLs• Porn, hacking, drugs, weapons, warez• Shopping, webmail, finance, dating

ODP Data<Topic r:id="Top/Arts/Movies/Titles/1/10_Rillington_Place">

<catid>205108</catid>

<link r:resource="http://www.britishhorrorfilms.co.uk/rillington.shtml"/>

<link r:resource="http://www.shoestring.org/mmi_revs/10-rillington-place.html"/>

</Topic>

<ExternalPage about="http://www.britishhorrorfilms.co.uk/rillington.shtml">

<d:Title>British Horror Films: 10 Rillington Place</d:Title>

<d:Description>Review which looks at plot especially the shocking features of it.</d:Description> <topic>Top/Arts/Movies/Titles/1/10_Rillington_Place</topic>

</ExternalPage>

<ExternalPage about="http://www.shoestring.org/mmi_revs/10-rillington-place.html">

<d:Title>MMI Movie Review: 10 Rillington Place</d:Title>

<d:Description>Review includes plot, real life story behind the film and realism in the film.</d:Description>

<topic>Top/Arts/Movies/Titles/1/10_Rillington_Place</topic>

</ExternalPage> ge>

Blacklist Data

• Shopping Domainsbestbudsgarden.com

bestbudsgarden.pointshop.com

bestbuket.ru

bestbung.com

bestbuy.com

• Shopping URLs00sake.com/htdocs

horseland.com.au/shop

balance.com/products/shop

theawningman.com.au/catalog/

wigglywigglers.co.uk/shop

Single URL

• http://www.dartmouth.edu

Dartmouth College

New Hampshire

United States

North America

Colleges & Universities

Education

Reference

Multiple URLs

• news.google.com/?topic=s

• www.kicksology.com

• www.beckett.com/userpages/Meibin.html

• www.razbet.com

• www.fhlsim.com

Multiple URLs

Portals

News and Media

Resources

Sports

Footwear

Apparel

Basketball

Sports

Shopping

Basketball

Cards

Sports

Collecting

Recreation

Basketball

Tipping & Handicapping

Sports

Gambling

Games

Software

Simulation

Hockey

Fantasy

Sports

news.google.com/?topic=s www.kicksology.com becket.com/userpages www.razbet.com www.fhlsim.com

Multiple URLs

Portals

News and Media

Resources

Footwear

Apparel

Basketball

Shopping

Basketball

Cards

Sports

Collecting

Recreation

Basketball

Tipping & Handicapping

Gambling

Games

Software

Simulation

Hockey

Fantasy

Cluster/Categorize

• Issues– Not all URLs categorized– URLs in multiple categories

• Document Clustering Methods– Page Content– Page meta data

User Representation

User Details

Correlate

• Temporal

• Similarity

• Interactions

Temporal Relations

Time

Fre

quen

cy

ShoppingComputerAdultArt

Correlate

User 1

User 3

User 4

User 6

User 10

User 8

User 5

User 9

User 7

User 2

Correlate

User 1

User 3

User 4

User 6

User 10

User 8

User 5

User 9

User 7

User 2

Correlate

User 1

User 3

User 4

User 6

User 10

User 8

User 5

User 9

User 7

User 2

Engineers

Med Students

Business Students

Future Work

• Search terms

• Categorize functionalities

• Temporal correlation

• Social Network Analysis

• Pruning

• Data

Questions

• Contacts– david.j.robinson@dartmouth.edu– gvc@dartmouth.edu– vincent.h.berk@dartmouth.edu