“Botconomics” – Mastering the Underground Economy of Botnets. LACNIC May, 2008
Kleber Carriello de OliveiraConsulting Engineer
Arbor Networks
Page 2 - Company Confidential
Agenda
Malware, Botnets & DDoS
An Underground Economy: “Botconomics”
Questions & Answers
Page 3 - Company Confidential
What’s in a Denial of Service (DoS) Attack?
</attack> <attack id="122002" start="2006-10-14 02:21:47" stop="2006-10-14 03:36:11"> # About an hour and 15 minutes duration <severity importance="1" lrm="0.9077" red_rate="1e+06" unit="pps"/> <type class="3" subclass="5"/> # Misuse Null TCP <direction type="Incoming" name="anonymous" gid="756"/> <protocols>6</protocols> # IP Protocol 6, TCP <tcpflags></tcpflags> # No Flags - Null TCP <source> <ips>0.0.0.0/0</ips> # Very well distributed or Source-spoofed IPs <ports>0-65535</ports> # Very well distributed source ports </source> <dst> <ips>xx.xx.X.X/32</ips> # Surprise, undernet IRC Server… <ports>6667</ports> # 6667 IRC </dst> <infrastructure num_routers="19" num_interfaces="52" sum_bps="622878440000" sum_pps="15571961000" max_bps="1980325333" max_pps="6188517"/> </attack>
Source: ISC
Page 4 - Company Confidential
Threat Time Line: NBA is Another Layer of Defense
Time
DiscoverVulnerability
AV/IDS Available
New Version
Advisory
Patch
PATCH MANAGEMENT
NETWORK ADMISSION
Network Behavioral Analysis with PEAKFLOW X
zero-day
Exploit VariantReleased
Reverse Engineer/new exploit
Page 5 - Company Confidential
Anti-Virus and IDS Detection Rates
Projected that between 75k-250k new malware families or variants release in 2006 (one released every 1-3 minutes)
0.00%10.00%20.00%30.00%40.00%50.00%60.00%70.00%80.00%90.00%
100.00%
McAfee F-Prot ClamAV Trend Symantec
AV Vendor
Malware Detection Rates Across Datasets
Legacy (20 NOV 2006) Small (20 NOV 2006)
Small (21 MAR 2007) Large (31 MAR 2007)
•Source: Internet Malware Classification and Analysis; University of Michigan & Arbor Networks, Inc., 2007
• Some samples still not detected a year after collection of malware.
• Almost half the samples in the small dataset undetected, and one quarter in the large
•AV fails to detect malware between 20% and 62% of the time!
Page 6 - Company Confidential
Though Necessary, AV Performance Poor
• Research puts most AV performance very low– ~38 AV products (open source & commercial)– Average 28-32% hit on for newer threats– AV Vendors change heuristics to improve results - but raises false-
positives rate– Why?
• Signature 1: 1000100010011111• New variant: 1000100010010001 - No AV Match• Minor obfuscation techniques• Packers• Polymorphic; e.g., recompile
– Getting better; more behavior-based functions, less static file analysis
– Behavior-based solutions augment• Cisco CSA, Sana Security host behavior (file, process,
network state)• NBA, Network Behavioral Analysis coupled with threat feeds
(e.g., Arbor’s ATF & Peakflow X)
Page 7 - Company Confidential
Bots: Putting the ‘(D)’ in (D)DoS
“Got bot?” • A bot is a servant process on a compromised system
(unbeknownst by owner) usually installed by a Trojan or Worm.
• Communicates with a handler or controller via public IRC servers or other compromised systems.
• A botmaster or botherder commands bots to perform any of an number of different functions.
• System of bots and controller(s) is referred to as a botnet or zombie network.
Page 8 - Company Confidential
InternetBackbone
B
UK Broadband
US Corp US Broadband
B
JP Corp.Provider
B B
ThePeacefulVillage
B
BB
B
B
B
SystemsBecomeInfected
Bots connect to a C&C to create an overlay network (botnet)
ControllerConnectsBotnet masterIssues attack
Command
BM
C&C
Bots attack
Bye Bye!
Anatomy of a DDoS Attack
Page 9 - Company Confidential
Anatomy of Botnet Construction
Exploit vector (e.g., TCP/135) Second stage functions (e.g., TFTP, FTP, HTTP) to download
bot software, C&C instructions Bot is executed, connected to C&C infrastructure
– often IRC, identified by DNS– Bot connects to channel (e.g., USA|743634) of C&C– Passwords often required– C&C often employs encryption, anti-cloaking techniques
Page 10 - Company Confidential
Malware Delivery
• Traditionally, worms with self propagation vector, not remote control function
• Last real virus - Melissa; 1999• Today email and other application-level functions laden with Trojans• Now delivered via web sites - drive-by installs
– Projected 1 in 10 web sites hosts malicious content– Web-based deliver means outpacing email, viruses, etc..– Example: Dolphin stadium web site compromised to host malicious content
just before Super Bowl in early 2007– iframe functions popular today
• <iframe src="http://www.iframemoney.org/banner.php?id=yourid" width="460" height="60"...></iframe>
• Interesting read: The Ghost in the Browser– http://www.usenix.org/events/hotbots07/tech/full_papers/provos/provos.pdf
• Clever new attacks include multi-layer attacks:– Compromise– Grab proxy IP; arpspoof, proxy– iframe insertion, local malware delivery, etc..
Page 11 - Company Confidential
Upon compromise, perform browser-esque speed tests to the following sites using Mozilla/4.0 (compatible; MSIE 6.0; WIN NT 5.1; Hotbar 4.3.1.0 :www.nifty.comwww.d1asia.comwww.st.lib.keio.ac.jpwww.lib.nthu.edu.twwww.above.netwww.level3.comnitro.ucsc.eduwww.burst.netwww.cogentco.comwww.rit.eduwww.nocster.comwww.verio.comwww.stanford.eduwww.xo.netde.yahoo.comwww.belwue.dewww.switch.chwww.1und1.deverio.frwww.utwente.nlwww.schlund.net
Engineering Malware: disable updates, speed tests..
• Engineer around current AV DBs
• Disable auto-update functions
• Evaluate connectedness of asset
• Employ
Page 12 - Company Confidential
Sophisticated Botnet Management & Statistics
Graphical user interface
Performance Statistics
Page 13 - Company Confidential
Reflective Amplification Attacks
r v
Response
vrQuery
Attacker - a
Victim - v
Resolver - r
A botnet with as few as 20 DSL-connect homes (1 Mbps upstream each) can generate 1.5 Gbps of attack traffic with DNS reflective amplification attack vectors such as those employed for root server attacks in early 2006 (1:76 amplification factor). Most enterprises have little more than 155 Mbps Internet connectivity.
Source IP of Victim (v) spoofed when query sent to resolver, resolver receives, responds to v. 55-byte query elicits 4200-byte response
Page 14 - Company Confidential
Application of Anti-Spoofing Measures
• Still not ubiquitous deployment - far from (hence effectiveness of reflective attacks)
• Largest deployment burden– hardware support– configuration management– Authoritative IP ownership
repository• ‘Loose-mode RPF’ likely creates
false sense of protection
Anti-Spoofing Techniques Employed
0.00%
5.00%
10.00%
15.00%
20.00%
25.00%
30.00%
35.00%
40.00%
45.00%
50.00%
BCP 38 uRPF Loose uRPF Strict None Other
Perc
en
tag
e R
esp
on
den
ts
Broadband/Dial-Up Dedicated Customer Peering Edge
Should assume slightly more clueful respondent pool than in general, so actual numbers likely less
Page 15 - Company Confidential
Attack Scale Still Increasing Considerably
Proliferation of broadband connectivity Increased virulence of attack vectors Sophistication of bot management software ‘01 - ‘03 data projections based on public
and private information regarding prominent attacks
Largest attacks (22 & 24 Gbps) reported by large content provider and hosting providers
Both >20 Gbps attacks reported to have been DNS reflective amplification attacks
Most backbone link speeds have 10G maximum capacity today
Largest Attacks Observed - 12 Months
0%
5%
10%
15%
20%
25%
NoAnswer
< 100Mbps
100 -500
Mbps
500Mbps -1 Gbps
1 - 4Gbps
4 - 10Gbps
10 - 20Gbps
20+Gbps
Attack Size - Bits Per Second
Sustained Attack Size - Gbps
0.4 1.22.5
10
17
24
0
5
10
15
20
25
30
2001 2002 2003 2004 2005 2006
Page 16 - Company Confidential
1
2
3
4
5
6
ISP A
T1 AGG
RTR
T1
Transit ISP
GE
Target
3 Mbps DDoS - teeny tiny attack - well, to Transit ISP, not ISP A
512k Attack
DDoS Attacks: Taking Advantage of Our Broadband
Botnets take advantage of “our” unlimited broadband pipes and PCs for amplification attacks and brute-force flooding attacks
ISPs are taken offline in the process of trying to mitigate these attacks.
Target Gone
CollateralDamage
ISP n
Much BIGGER Attack
Page 17 - Company Confidential
DNS Attacks - When & What?
OCT 2002 JUN 2004 OCT 2004 JAN-FEB 2006NOV 2004NOV 2002 FEB 2007
Root Server AttackedDuration:1 hourMulti-modal: smurf, ICMP, port 53“7” Root Servers appear
unreachableImpact: No noticeable user effect
Root Server AttackedDuration:1 hourMulti-modal: smurf, ICMP, port 53“7” Root Servers appear
unreachableImpact: No noticeable user effect
UltraDNS TLD Servers AttackedDuration: 24 hours +ICMP 0,8 and then portEasily filtered -- uses pure volume
of packets to disableResults in 2-way traffic loadImpact: No noticeable user effect
UltraDNS TLD Servers AttackedDuration: 24 hours +ICMP 0,8 and then portEasily filtered -- uses pure volume
of packets to disableResults in 2-way traffic loadImpact: No noticeable user effect
Akamai attackedDuration: 4 hoursNo mitigation possiblePort 53, UDP, valid queriesMulti-millions queries per secondImpact: Global Impact
Akamai attackedDuration: 4 hoursNo mitigation possiblePort 53, UDP, valid queriesMulti-millions queries per secondImpact: Global Impact DDoS for hire (extortion)
The golden age for worms/trojansThe perfect DNS DDoS in the wildNo protocol based defense or mitigationAttack on Bandwidth, not applications or
servers - 11 Gbps+Impact: Significant collateral damage
DDoS for hire (extortion)The golden age for worms/trojansThe perfect DNS DDoS in the wildNo protocol based defense or mitigationAttack on Bandwidth, not applications or
servers - 11 Gbps+Impact: Significant collateral damage
January-February.com, .net (Verisign), .org (UltraDNS)Utilized open recursive serversAverage attack 7-10 GbpsTLD Operators have no successful
defenseImpact: Considerable user impact
January-February.com, .net (Verisign), .org (UltraDNS)Utilized open recursive serversAverage attack 7-10 GbpsTLD Operators have no successful
defenseImpact: Considerable user impact
G, L & M Root Servers, Other TLDs (UltraDNS)?
Utilized large bogus DNS UDP queries from many bots
Aggregate attacks 10 Gbps+Mitigate: Special HardwareImpact: 90% Traffic dropped
localized user impact
G, L & M Root Servers, Other TLDs (UltraDNS)?
Utilized large bogus DNS UDP queries from many bots
Aggregate attacks 10 Gbps+Mitigate: Special HardwareImpact: 90% Traffic dropped
localized user impact
NOV 2006
UUNet Attack - 2nd Level DNSUDP/53, auth servers for bank.fooSpoofed source IPs - 800 KppsImpact: End-user/customerMitigated with Cisco Guard-XTCollateral damage: 2x .gov & 2
7206s in network path
UUNet Attack - 2nd Level DNSUDP/53, auth servers for bank.fooSpoofed source IPs - 800 KppsImpact: End-user/customerMitigated with Cisco Guard-XTCollateral damage: 2x .gov & 2
7206s in network path
Root & TLD AttacksSpoofed source IPsLarge Bogus Queries10+ GbpsRegionalized User Impact
Root & TLD AttacksSpoofed source IPsLarge Bogus Queries10+ GbpsRegionalized User Impact
Page 18 - Company Confidential
Botconomics
• Amalgamation:: botnets && economics == botconomics
• Botconomics: it’s all about the $$$$
Page 19 - Company Confidential
Three Tiers of Cyber Criminals
Script KiddiesPolitical/Ego-driven; improve
halo reputation
Organized Crime Economically Motivated - all about the $$$
Cyber TerrorismCyber Espionage;
Asymmetric Warfare
Page 20 - Company Confidential
Religious, Political Estonia Denmark Cartoon Rage
Ego-driven (gaming, IRC)
Extortion (SuperBowl, World Cup - can your bookie afford to be offline?)
$2B US Each - $48B Market Player SLAs
Lift email, targeted spam, spear phishing (>90% spam through bots)
An Underground Economy: “Botconomics”
Page 22 - Company Confidential
Botconomics: Identity Theft & Fraud
Global organized crime
How many people here:Have every bought anything online? Bank online? Have a credit cardHave a mortgage or pay rent? Were in the militaryHave ever been to a medical office?If you said yes to any of the above, you’re at risk
‘full creds’
But who’d be dumb enough to fill this out?
Hey Kleber, quick question for
you. IF…..??
Page 23 - Company Confidential
Botconomics: It doesn’t matter if you don’t use your credit card on line!
The databases that contain all your in-person credit card transactions is where the money is.
Hits close to home.
But what do you do with 46 Million stolen credit card data sets?
•Sell them - individual, bundle, wholesale•Use them to buy stuff online (e.g., movietickets.com)•CC Forums - brokerage houses, printed cards..
•Buy stuff•Get cash advances•Need to monetize
•Item Advertised Price (US $)
•US-based credit card with card verification value $1 - $6•UK-based credit card with card verification value $2 - $12•List of 29,000 emails $5•Online banking account with a $9,900 balance$300•Yahoo Mail cookie exploit -- facilitates full access when successful $3•Valid Yahoo and Hotmail email cookies $3•Compromised computer $6 - $20•Phishing Web site hosting - per site $3 - 5•Verified PayPal account with balance (balance varies) $50 - $500•Unverified PayPal account with balance (balance varies) $10 - $50•Skype account $12•World of Warcraft account - one month duration $10
Source: Symantec Internet Security Threat Report - March 2007
•Item Advertised Price (US $)
•US-based credit card with card verification value $1 - $6•UK-based credit card with card verification value $2 - $12•List of 29,000 emails $5•Online banking account with a $9,900 balance$300•Yahoo Mail cookie exploit -- facilitates full access when successful $3•Valid Yahoo and Hotmail email cookies $3•Compromised computer $6 - $20•Phishing Web site hosting - per site $3 - 5•Verified PayPal account with balance (balance varies) $50 - $500•Unverified PayPal account with balance (balance varies) $10 - $50•Skype account $12•World of Warcraft account - one month duration $10
Source: Symantec Internet Security Threat Report - March 2007
Page 24 - Company Confidential
Botconomics: Increase in Sophistication and Marketing
Key loggers– Gotta get those “full creds”
Drop Sites Click Fraud Bot trading & Marketing
– .net - .$.05– .gov - $1.00– nasa.gov - $.05
“Better Marketing by the Botherders”– Excellent ping & uptime– Rotating IP addresses– Different ISPs– Intuitive User Interface– SLAs - 100 percent uptime guarantee!
Page 25 - Company Confidential
Botconomics: Closing the Loop
Phishing Systems– Command & Control – Hosting phishing sites– Lift email addresses– Spam phishing messages– Drop Sites– All bots!
Botnet Defense Systems– Attack anti-phishing, anti-spam
and anti-botnet companies
• BlueSecurity
• CastleCops
[19/Feb/2007:15:10:18 +0000] "GET / HTTP/1.1" 200 497 "http://you.shut.us.down.we.shut.you.down.is.it.a.trade.or.not.net" "Mozilla/4.0 (compatible)”[19/Feb/2007:15:10:18 +0000] "GET / HTTP/1.1" 200 497 "http://you.shut.us.down.we.shut.you.down.is.it.a.trade.or.not.net" "Mozilla/4.0 (compatible)”[19/Feb/2007:15:10:18 +0000] "GET /p898068-remove_quot_quot_from_the_domain_name.html#898068 HTTP/1.0" 200 497 "-" "Mozilla/4.0 (compatible)”[19/Feb/2007:15:10:18 +0000] "GET /p898068-remove_quot_quot_from_the_domain_name.html#898068 HTTP/1.1" 200 497 "-" "Mozilla/4.0 (compatible)”[19/Feb/2007:15:10:18 +0000] "GET / HTTP/1.1" 200 497 "http://even.prolexic.cant.protect.you.net.wanna.try.akamai.ill.drop.them.too" "Mozilla/4.0 (compatible)”
Page 26 - Company Confidential
From Arbor’s BLOG
Page 27 - Company Confidential
The Phish….
• Build the phishing site, host on bot; perhaps proxy actual site• Spam the phish message - perhaps targeted (spear)• ハ - Go to:
– <a href="http://cesantoni.com.mx/%20/update-wells-info/index.html">https://online.wellsfargo.com/signon/</a><br>
• Throw the spoils on a couple of drop sites - more bots• Use the spoils to transfer money directly, use to transfer money
internationally, etc..
Page 28 - Company Confidential
Where’s the Money Going?
• Funding an “online dating service for al-Qaeda?
• “investigators say they found some 37,000 stolen credit card numbers. Alongside each credit card record was other information on the ID theft victims, such as the account holder's address, date of birth, credit balances and limits.”
• “..jihadists might need for their battle against the American and allied forces in Iraq, including global positioning satellite (GPS) devices, night-vision goggles, sleeping bags, telephones, survival knives and tents.”
Page 29 - Company Confidential
Operation Spamalot
• ・On Friday, Dec. 15, 2006, shares in Apparel Manufacturing Associates, Inc. (APPM) closed at $.06, with a trading volume of 3,500 shares. After a weekend spam campaign distributed emails proclaiming, "Huge news expected out on APPM, get in before the wire, We're taking it all the way to $1.00," trading volume on Monday, Dec. 18, 2006, hit 484,568 shares with the price spiking to over 19 cents a share. Two days later the price climbed to $.45. By Dec. 27, 2006, the price was back down to $.10 on trading volume of 65,350 shares.
• On Dec. 19, 2006, trading in Goldmark Industries, Inc. (GDKI), closed at $.17 on trading volume of 126,286 shares. On Dec. 20, 2006, the spam campaign started, with e-mail proclaiming "GDKI IS MAKING EVERYONE BANK!," and setting a 5-day price target of $2. By Dec. 28, 2006, spam emails boasted of the price spike that had already been achieved -- "$.28 (Up 152% in 2 days!!!)" -- and promised a 5-day price target of $1. That same day, GDKI closed at $.35 on a volume of more than 5 million shares. By January 9, 2007, the closing share price was back
down to $.15.
Attack Vector?
Page 30 - Company Confidential
Good News?
• The financial losses are at a point where industry must invest - obvious from Financials to LEOs discernible uptick in activity
US
$ -
Bill
ion
s
Time - Losses Annually
Factored Losses, Tolerance Threshold
Cyber Crime Losses
Traditional Fraud
~$20B US
Page 31 - Company Confidential
Arbor’s Worldwide Infrastructure Security Report
Demographics:
− 70 self-classified tier-1, tier-2, and hybrid IP network operators in North America, Europe & Asia
Key Findings:− Most significant operational threats are:
• #1 Botnets, #2 DDoS
− Frequency, size and complexity of attacks are growing
•22 & 24 Gbps attacks reported
•More Application Layer attacks
− ISPs finish the job
−DDoS Managed Services activity grows 800%
− Less than 2% reported to Law Enforcement
Page 32 - Company Confidential
Detection without mitigation - hrmm…
DDoS Mitigation Techniques
• Good & bad news– Bad: SPs still effectively complete
attack (protect network availability)– Good: More mitigation solution
deployment (scrub- ARBOR TMS, flow spec, etc..) and service offerings - nearly 10x increase percentage wise, even with wider respondent pool
• Can’t win bandwidth game (e.g., consider Storm with reflective amplification)
• New mitigation infrastructure only applies to MS customers
• Mitigation highly fragmented - little incentive to follow-up with ingress (or even upstream/ adjacent) network for host cleanup - malicious activity recurrence factor considerable
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
50%
Primary Secondary
ACLs BGP Destination-based RTBH
BGP Source-based RTBH Scrubber
Other No AnswerDDoS Managed Services
0%
5%
10%
15%
20%
25%
30%
No Answer DetectionOnly
MitigationOnly
Detection&
Mitigation
InPlanning
Other
Page 33 - Company Confidential
• Netflow + DPI
The system talk with the scrub to clean the traffic
Mitigation process is started
Inteligent Mitigation
Flows sent to the collector system
System detects the attack
Inject BGP route (off-ramping)
Scrub inspects each packet against its rules and network behavior
Peakflow SP TMS
Peakflow SP
Page 34 - Company Confidential
Attack Scale & Frequency
• Attacks from perspective of single ISP and single attack vector, thus aggregate for many is likely to be much higher
• Cross-correlation of targets and times provides considerable insight
• Doesn’t necessarily matter - scale all about perspective
Attack Scale and Frequency (11 mos.)
0
200
400
600
800
1000
1200
Attack Days 1 2 2 3 15 38 67 114 188
Total Attacks 2 4 4 5 23 91 170 437 1059
9+ Mpps
8 Mpps
7 Mpps
6 Mpps
5 Mpps
4 Mpps
3 Mpps
2 Mpps
1 Mpps
Estonia Attacks 4 Mpps aggregate
at peak
Page 35 - Company Confidential
Even Cyber Criminals Take Some Time Off
Attack Size: BPS
1.00E+00
1.00E+01
1.00E+02
1.00E+03
1.00E+04
1.00E+05
1.00E+06
1.00E+07
1.00E+08
1.00E+09
1.00E+109/
12/0
6
9/19
/06
9/26
/06
10/3
/06
10/1
0/0
6
10/1
7/0
6
10/2
4/0
6
10/3
1/0
6
11/7
/06
11/1
4/06
11/2
1/06
11/2
8/06
12/5
/06
12/1
2/0
6
12/1
9/0
6
12/2
6/0
6
1/2/
07
1/9/
07
BP
S max_bpsavg_bps
• Data derived from Arbor products deployed in 70% of world’s ISPs
Page 36 - Company Confidential
Attack on Russia - Arbor’s Global Visibility
Detect multi-ISP distributed attackDetect multi-ISP distributed attack
Page 37 - Company Confidential
A Solution: Network Behavioral Analysis (NBA)
Network transactional information + control plane data enables baselines (statistical and relational) that allow abnormalities to be identified
Network-based mitigation can be performed based upon NBA
Even to detect zero-day threats (e.g., many families have same network behavioral fingerprint but different payload)
Based on compound temporal functions, as well as single packet transactions (e.g., know botnet C&C, UN Exported Restricted Nations, known malware distribution sites, etc..)
Page 38 - Company Confidential
Behavioral Fingerprinting
Unique variants require new virus detection definitions: – packers– polymorphism, recompile– minor obfuscation techniques for known packers– strings
E.g., 580+ Agobot variants Fingerprinting behaviors allows for more generalized detection
mechanisms– file status– process state– network transactions
Host and network-based detection models that employ relational modeling and network behavioral analysis provide substrate for zero-day threat identification
Page 40 - Company Confidential
InternetBackbone
B
UK Broadband
US Corp US Broadband
B
Anti-Bot/Spam.comProvider
B B
ThePeacefulVillage
B
BB
B
B
B
SystemsBecomeInfected
Bots connect to a C&C to create an overlay network (botnet)
ControllerConnectsBotnet masterIssues attack
Command
BM
Bots attack
Bye Bye!
Think of the Possibilities
PhishingSite
PhishingSite
DropSite
DropSite
C&C
SpamRelay
SpamRelay
OpenProxy
OpenProxy
Phishing DataPhishing Data
CD KeysKeylogger
Personal IDVideoEmail
CC & PWFinancial data
CD KeysKeylogger
Personal IDVideoEmail
CC & PWFinancial data
Page 42 - Company Confidential
Conclusions
• It’s all about layered [network] security - there IS NO silver bullet
• Behavioral models coupled with real-time threat intelligence (e.g., Arbor’s ATLAS) can minimize threats; provide gap insurance and help hardening and prevention
• Enable account transaction alerting and keep an eye on those credit reports…
Page 43 - Company Confidential