cset 2015 security reputation metrics for hosting providers · security reputation metrics for...
TRANSCRIPT
![Page 1: CSET 2015 Security Reputation Metrics for Hosting Providers · Security Reputation Metrics for Hosting Providers @CSET’15 10 Aug. 2015 Arman Noroozian, Maciej Korzcyński, Samaneh](https://reader036.vdocument.in/reader036/viewer/2022070608/5ac4fe427f8b9a5c558d1b5c/html5/thumbnails/1.jpg)
Security Reputation Metrics for Hosting Providers
@CSET’1510 Aug. 2015
Arman Noroozian, Maciej Korzcyński,
Samaneh Tajalizadehkhoob, Michel van Eeten
![Page 2: CSET 2015 Security Reputation Metrics for Hosting Providers · Security Reputation Metrics for Hosting Providers @CSET’15 10 Aug. 2015 Arman Noroozian, Maciej Korzcyński, Samaneh](https://reader036.vdocument.in/reader036/viewer/2022070608/5ac4fe427f8b9a5c558d1b5c/html5/thumbnails/2.jpg)
… to make and Interpret properly
Reputation Metrics are Hard !
2
![Page 3: CSET 2015 Security Reputation Metrics for Hosting Providers · Security Reputation Metrics for Hosting Providers @CSET’15 10 Aug. 2015 Arman Noroozian, Maciej Korzcyński, Samaneh](https://reader036.vdocument.in/reader036/viewer/2022070608/5ac4fe427f8b9a5c558d1b5c/html5/thumbnails/3.jpg)
Why Metrics ? The “Lemons Market” Problem
Information Asymmetry
Consumer / Policy maker/ Law enforcement officer Which provider is better/worse in security?
The provider (intermediary) itself doesn’t know either!
Erodes incentives to invest in security3
![Page 4: CSET 2015 Security Reputation Metrics for Hosting Providers · Security Reputation Metrics for Hosting Providers @CSET’15 10 Aug. 2015 Arman Noroozian, Maciej Korzcyński, Samaneh](https://reader036.vdocument.in/reader036/viewer/2022070608/5ac4fe427f8b9a5c558d1b5c/html5/thumbnails/4.jpg)
Hosting Providers Legitimate hosting provider types
Bulletproof Hosting!
(M3AAWG, Anti-Abuse Best Common Practices for Hosting and Cloud Service Providers. Technical Report March 2015.)
4
![Page 5: CSET 2015 Security Reputation Metrics for Hosting Providers · Security Reputation Metrics for Hosting Providers @CSET’15 10 Aug. 2015 Arman Noroozian, Maciej Korzcyński, Samaneh](https://reader036.vdocument.in/reader036/viewer/2022070608/5ac4fe427f8b9a5c558d1b5c/html5/thumbnails/5.jpg)
Concentration of Abuse
Attractive Pressure points Remediation
Policy making
Source: McAfee Threats Report Q2 2012 5
![Page 6: CSET 2015 Security Reputation Metrics for Hosting Providers · Security Reputation Metrics for Hosting Providers @CSET’15 10 Aug. 2015 Arman Noroozian, Maciej Korzcyński, Samaneh](https://reader036.vdocument.in/reader036/viewer/2022070608/5ac4fe427f8b9a5c558d1b5c/html5/thumbnails/6.jpg)
Source: http://krebsonsecurity.com/2010/03/naming-and-shaming-bad-isps/
Concentrations of Abuse (Cont.)
6
![Page 7: CSET 2015 Security Reputation Metrics for Hosting Providers · Security Reputation Metrics for Hosting Providers @CSET’15 10 Aug. 2015 Arman Noroozian, Maciej Korzcyński, Samaneh](https://reader036.vdocument.in/reader036/viewer/2022070608/5ac4fe427f8b9a5c558d1b5c/html5/thumbnails/7.jpg)
Source: http://hostexploit.com/downloads/world_hosts_report_201403.pdf
Hoster Size Matters
7
![Page 8: CSET 2015 Security Reputation Metrics for Hosting Providers · Security Reputation Metrics for Hosting Providers @CSET’15 10 Aug. 2015 Arman Noroozian, Maciej Korzcyński, Samaneh](https://reader036.vdocument.in/reader036/viewer/2022070608/5ac4fe427f8b9a5c558d1b5c/html5/thumbnails/8.jpg)
Measures of SizeAdvertised IP Space Hosted 2nd level domains
8
![Page 9: CSET 2015 Security Reputation Metrics for Hosting Providers · Security Reputation Metrics for Hosting Providers @CSET’15 10 Aug. 2015 Arman Noroozian, Maciej Korzcyński, Samaneh](https://reader036.vdocument.in/reader036/viewer/2022070608/5ac4fe427f8b9a5c558d1b5c/html5/thumbnails/9.jpg)
Indicators of Abuse
Indicators Why Challenge
Occurrence of Abuse(How often abused?)
Signals network hygiene and vulnerability
Hard to isolate provider efforts from other factors
Uptime of abuse(How long abused?)
Signals effectiveness of abuse handing
Hard to measure at scale
9
![Page 10: CSET 2015 Security Reputation Metrics for Hosting Providers · Security Reputation Metrics for Hosting Providers @CSET’15 10 Aug. 2015 Arman Noroozian, Maciej Korzcyński, Samaneh](https://reader036.vdocument.in/reader036/viewer/2022070608/5ac4fe427f8b9a5c558d1b5c/html5/thumbnails/10.jpg)
Sensitivity of Metrics Choice of abuse data
Biases and errors in abuse data
Errors in mapping abuse data
Biases and errors in size estimation data
10
![Page 11: CSET 2015 Security Reputation Metrics for Hosting Providers · Security Reputation Metrics for Hosting Providers @CSET’15 10 Aug. 2015 Arman Noroozian, Maciej Korzcyński, Samaneh](https://reader036.vdocument.in/reader036/viewer/2022070608/5ac4fe427f8b9a5c558d1b5c/html5/thumbnails/11.jpg)
Dutch Police:“Who are the worst hosting providers in our jurisdiction?”
A Dutch Case Study
11
![Page 12: CSET 2015 Security Reputation Metrics for Hosting Providers · Security Reputation Metrics for Hosting Providers @CSET’15 10 Aug. 2015 Arman Noroozian, Maciej Korzcyński, Samaneh](https://reader036.vdocument.in/reader036/viewer/2022070608/5ac4fe427f8b9a5c558d1b5c/html5/thumbnails/12.jpg)
Data Sources Abuse
StopBadware
Shadowserver Compromised servers
Outbound malware connections
Zeustracker C&Cs (Abuse.ch)
Mutual Legal Assistance Treaty (MLAT) requests
Dutch child pornography hotline
Phishtank
Anti-Phishing Working Group
IP Routing Data Python pyasn library
Passive DNS (pDNS) DNSDB from Farsight Security
750 million unique 2LDs
93 million unique IPv4 Addresses
12
![Page 13: CSET 2015 Security Reputation Metrics for Hosting Providers · Security Reputation Metrics for Hosting Providers @CSET’15 10 Aug. 2015 Arman Noroozian, Maciej Korzcyński, Samaneh](https://reader036.vdocument.in/reader036/viewer/2022070608/5ac4fe427f8b9a5c558d1b5c/html5/thumbnails/13.jpg)
Our Methodology
13
![Page 14: CSET 2015 Security Reputation Metrics for Hosting Providers · Security Reputation Metrics for Hosting Providers @CSET’15 10 Aug. 2015 Arman Noroozian, Maciej Korzcyński, Samaneh](https://reader036.vdocument.in/reader036/viewer/2022070608/5ac4fe427f8b9a5c558d1b5c/html5/thumbnails/14.jpg)
Abuse Feeds
p-DNS / IP Routing
• Shadow Server Compromise• Shadow Server Sandbox URL• Zeustracker C&Cs• MLAT requests• PhishTank• APWG• Child Pornography Hotline
• # Advertised IPs• # IPs in p‐DNS• # Domains Hosted
Abuse Mapping
Size Mapping
• Farsight Security p-DNS Data
• Internet IP Routing Data
• # Unique Abuse / AS
Abuse MapsAbuse Maps
PhishTankAS#1 100 AS#2 200
MLATAS#1 50AS#2 73
Size MapsSize Maps
Advertised IPsAS#1 256AS#2 1024
Domains HostedAS#1 23AS#2 1232
Step 1+2: Mapping
14
![Page 15: CSET 2015 Security Reputation Metrics for Hosting Providers · Security Reputation Metrics for Hosting Providers @CSET’15 10 Aug. 2015 Arman Noroozian, Maciej Korzcyński, Samaneh](https://reader036.vdocument.in/reader036/viewer/2022070608/5ac4fe427f8b9a5c558d1b5c/html5/thumbnails/15.jpg)
Abuse MapsAbuse Maps
PhishTankAS#1 100 AS#2 200
MLATAS#1 50AS#2 73
Size MapsSize Maps
Advertised IPsAS#1 256AS#2 1024
Domains HostedAS#1 23AS#2 1232
Normalized AbuseNormalized Abuse
PhishTank / Advrt. IPsAS#1 0.39AS#2 0.19
PhishTank / Domains HostedAS#1 4.34AS#2 0.16
MLAT / Advrt. IPsAS#1 0.19AS#2 0.07
MLAT / Domains HostedAS#1 2.17AS#2 0.05
Normalization
• # Abuse / Size
Step 3: Normalization
15
![Page 16: CSET 2015 Security Reputation Metrics for Hosting Providers · Security Reputation Metrics for Hosting Providers @CSET’15 10 Aug. 2015 Arman Noroozian, Maciej Korzcyński, Samaneh](https://reader036.vdocument.in/reader036/viewer/2022070608/5ac4fe427f8b9a5c558d1b5c/html5/thumbnails/16.jpg)
Normalized AbuseNormalized Abuse
PhishTank / Advrt. IPsAS#1 0.39AS#2 0.19
PhishTank / Domains HostedAS#1 4.34AS#2 0.16
MLAT / Advrt. IPsAS#1 0.19AS#2 0.07
MLAT / Domains HostedAS#1 2.17AS#2 0.05
Abuse RankingAbuse Ranking
PhishTank Ranking 1AS#1 834AS#2 833
PhishTank Ranking 2AS#1 834AS#2 833
MLAT Ranking 1AS#1 235AS#2 234
MLAT Ranking 2AS#1 235AS#2 234
Rank
Sort Rank High Low
Step 4: Ranking
16
![Page 17: CSET 2015 Security Reputation Metrics for Hosting Providers · Security Reputation Metrics for Hosting Providers @CSET’15 10 Aug. 2015 Arman Noroozian, Maciej Korzcyński, Samaneh](https://reader036.vdocument.in/reader036/viewer/2022070608/5ac4fe427f8b9a5c558d1b5c/html5/thumbnails/17.jpg)
Abuse RankingAbuse Ranking
PhishTank Ranking 1AS#1 834AS#2 833
PhishTank Ranking 2AS#1 834AS#2 833
MLAT Ranking 1AS#1 235AS#2 234
MLAT Ranking 2AS#1 235AS#2 234
Abuse RankingAbuse Ranking
Overall RankingAS#1 1AS#2 0.92AS#3 0.87AS#4 0.86
Combine Ranks
Borda Count
Step 5: Aggregation
17
![Page 18: CSET 2015 Security Reputation Metrics for Hosting Providers · Security Reputation Metrics for Hosting Providers @CSET’15 10 Aug. 2015 Arman Noroozian, Maciej Korzcyński, Samaneh](https://reader036.vdocument.in/reader036/viewer/2022070608/5ac4fe427f8b9a5c558d1b5c/html5/thumbnails/18.jpg)
Security Reputation Metrics20 worst Dutch hosting providers Abuse Rate vs Cleanup Rate
18
![Page 19: CSET 2015 Security Reputation Metrics for Hosting Providers · Security Reputation Metrics for Hosting Providers @CSET’15 10 Aug. 2015 Arman Noroozian, Maciej Korzcyński, Samaneh](https://reader036.vdocument.in/reader036/viewer/2022070608/5ac4fe427f8b9a5c558d1b5c/html5/thumbnails/19.jpg)
Abuse Metrics are Hard How to measure abuse and remediation
What abuse can be observed What does it tell us about remediation efforts
How to associate it with hosting providers What is a hosting provider How to identify them at scale
How to control for differences among providers and interpret metric(s) How to take size into account How to take different business models into account
How to aggregate indicators into a comprehensive metric (set of metrics) ?
19
![Page 20: CSET 2015 Security Reputation Metrics for Hosting Providers · Security Reputation Metrics for Hosting Providers @CSET’15 10 Aug. 2015 Arman Noroozian, Maciej Korzcyński, Samaneh](https://reader036.vdocument.in/reader036/viewer/2022070608/5ac4fe427f8b9a5c558d1b5c/html5/thumbnails/20.jpg)
Towards better metrics How to measure abuse and remediation
Increase coverage, add different global abuse feeds Add uptime data (e.g. phishing)
How to associate it with hosting providers Identify hosting providers from IP ownership data (WHOIS) instead of AS-level routing
data (BGP)
How to control for differences among providers and interpret metric(s) Extract ‘profiles’ from pDNS data (size, shared hosting, dedicated, non-webdomain)
How to aggregate indicators into a comprehensive metric (set of metrics) ? More sensitivity analysis of aggregation methods
20
![Page 21: CSET 2015 Security Reputation Metrics for Hosting Providers · Security Reputation Metrics for Hosting Providers @CSET’15 10 Aug. 2015 Arman Noroozian, Maciej Korzcyński, Samaneh](https://reader036.vdocument.in/reader036/viewer/2022070608/5ac4fe427f8b9a5c558d1b5c/html5/thumbnails/21.jpg)
Questions?
Thank you for attention21