anomaly detection steven m. bellovin smb matsuzaki ‘maz’ yoshinobu 1
TRANSCRIPT
![Page 1: Anomaly Detection Steven M. Bellovin smb Matsuzaki ‘maz’ Yoshinobu 1](https://reader035.vdocument.in/reader035/viewer/2022080905/56649ce55503460f949b2bb9/html5/thumbnails/1.jpg)
1
Anomaly Detection
Steven M. Bellovinhttps://www.cs.columbia.edu/~smb
Matsuzaki ‘maz’ Yoshinobu<[email protected]>
![Page 2: Anomaly Detection Steven M. Bellovin smb Matsuzaki ‘maz’ Yoshinobu 1](https://reader035.vdocument.in/reader035/viewer/2022080905/56649ce55503460f949b2bb9/html5/thumbnails/2.jpg)
2
Why Anomaly Detection?
• Signatures defend against known attacks– You need a separate signature for each one– By definition, there are no signatures for things
that don’t exist• Anomaly detectors look for unusual activity:
things that normally don’t happen• Implication: must first know what is normal– “Normal” is different for every organization
![Page 3: Anomaly Detection Steven M. Bellovin smb Matsuzaki ‘maz’ Yoshinobu 1](https://reader035.vdocument.in/reader035/viewer/2022080905/56649ce55503460f949b2bb9/html5/thumbnails/3.jpg)
3
What’s An Anomaly?
Normal Infected
![Page 4: Anomaly Detection Steven M. Bellovin smb Matsuzaki ‘maz’ Yoshinobu 1](https://reader035.vdocument.in/reader035/viewer/2022080905/56649ce55503460f949b2bb9/html5/thumbnails/4.jpg)
4
Examples
• massive incoming traffic - periodic security update or DoS • unusual outbound traffic - video chat , flood attack or information theft • unusual protocol communication - new application or compromised host
![Page 5: Anomaly Detection Steven M. Bellovin smb Matsuzaki ‘maz’ Yoshinobu 1](https://reader035.vdocument.in/reader035/viewer/2022080905/56649ce55503460f949b2bb9/html5/thumbnails/5.jpg)
5
General Process
• Establish a baseline of normal activity– Sample activity from times when you’re not under
attack• Train your detectors on this baseline set• Continually match current behavior against
the baseline• Investigate “significant” deviations
![Page 6: Anomaly Detection Steven M. Bellovin smb Matsuzaki ‘maz’ Yoshinobu 1](https://reader035.vdocument.in/reader035/viewer/2022080905/56649ce55503460f949b2bb9/html5/thumbnails/6.jpg)
6
CV5: “Correlate Violations of Volume, Velocity, Values, Vertices”
• Correlate is obvious• Violations implies some "normal" model is violated• Volume and Velocity are standard metrics of
expected flow behavior (think highways)• Values pertain to any content analysis, packet heads,
datagrams, email bodies, URL, PHP variable argument values, etc.
• Vertices pertains to graphic theoretic constructs, connectivity between entities, IP addresses, MAC addresses, ports, etc.
![Page 7: Anomaly Detection Steven M. Bellovin smb Matsuzaki ‘maz’ Yoshinobu 1](https://reader035.vdocument.in/reader035/viewer/2022080905/56649ce55503460f949b2bb9/html5/thumbnails/7.jpg)
7
Establishing a Baseline
• Different strategies for different uses and kinds of attacks– What does your traffic flow normally look like?– What applications do users run?– What is the byte value distribution of certain file
types?• Word documents infected with shell code will have more
bytes that look like x86 machine code
• Different groups will have different normal behavior
![Page 8: Anomaly Detection Steven M. Bellovin smb Matsuzaki ‘maz’ Yoshinobu 1](https://reader035.vdocument.in/reader035/viewer/2022080905/56649ce55503460f949b2bb9/html5/thumbnails/8.jpg)
8
One Way to Define Normal
(Mathematically) find clusters. Points outside the clusters are abnormal.
![Page 9: Anomaly Detection Steven M. Bellovin smb Matsuzaki ‘maz’ Yoshinobu 1](https://reader035.vdocument.in/reader035/viewer/2022080905/56649ce55503460f949b2bb9/html5/thumbnails/9.jpg)
9
Limitations
• It’s hard to define “normal”– Was your training data really attack-free?– What if legitimate patterns change? New
employees? New versions of applications? • Relatively high false positive rate• Can miss subtle attacks• Must run anomaly detectors on many
different activities
![Page 10: Anomaly Detection Steven M. Bellovin smb Matsuzaki ‘maz’ Yoshinobu 1](https://reader035.vdocument.in/reader035/viewer/2022080905/56649ce55503460f949b2bb9/html5/thumbnails/10.jpg)
10
Advantages
• Can detect minor variants of existing attacks (a serious issue in the anti-virus world)
• Can detect 0-day attacks• No need to constantly update signature
database• Probably the wave of the future in intrusion
detection
![Page 11: Anomaly Detection Steven M. Bellovin smb Matsuzaki ‘maz’ Yoshinobu 1](https://reader035.vdocument.in/reader035/viewer/2022080905/56649ce55503460f949b2bb9/html5/thumbnails/11.jpg)
11
Example: Netflow
• router can export traffic flow information (incoming interface, packet headers) to a collector – useful to analyze traffic
netflow export
netflow collector
![Page 12: Anomaly Detection Steven M. Bellovin smb Matsuzaki ‘maz’ Yoshinobu 1](https://reader035.vdocument.in/reader035/viewer/2022080905/56649ce55503460f949b2bb9/html5/thumbnails/12.jpg)
12
Example: Mail Logs
• Look at the mail logs every day– Is someone sending significantly more mail than
they normally do?– Is someone sending to many more recipients than
normal?– Is the size of someone’s mail messages larger than
normal?– Anomalies can be benign: recently, someone
emailed me a 9 MB, 1600 page PDF, with many scanned images—and it was perfectly legitimate
![Page 13: Anomaly Detection Steven M. Bellovin smb Matsuzaki ‘maz’ Yoshinobu 1](https://reader035.vdocument.in/reader035/viewer/2022080905/56649ce55503460f949b2bb9/html5/thumbnails/13.jpg)
13
Example: Host Monitoring
• Monitor system calls– What system calls does an application normally
make?– What sequences of system calls does it normally
make?• Works before encryption or after decryption• But—attackers can look for and disable a host-
based IDS