the osquery file carver - osdfcon · #!/bin/bash nc -l -p 1337. what is osquery? ......
Post on 16-Jul-2020
9 Views
Preview:
TRANSCRIPT
The osquery File Carver
Nick AndersonSecurity Engineer
• Nick Anderson • Security Engineer at Facebook
• thor@fb.com • Super legit, not an alias
• Github - github.com/muffins
• Twitter - twitter.com/poppyseedplehzr
• Slack - thor
•
C:\> Get-Host
A quick story
Analyst
Enterprise Endpoint
Analyst
Enterprise Endpoint
/tmp/evil
Analyst
Enterprise Endpoint
/tmp/evil
Analyst
Enterprise Endpoint
/tmp/evil
#!/bin/bashnc -l -p 1337
Analyst
Enterprise Endpoint
/tmp/evil
#!/bin/bashnc -l -p 1337
What is osquery?
• FOSS host based IDS
What is osquery?
• FOSS host based IDS • Cross-platform
What is osquery?
• FOSS host based IDS • Cross-platform • Abstracts OS as SQLite tables
What is osquery?
• FOSS host based IDS • Cross-platform • Abstracts OS as SQLite tables • Performant
What is osquery?
• FOSS host based IDS • Cross-platform • Abstracts OS as SQLite tables • Performant • Extensible
What is osquery?
• FOSS host based IDS • Cross-platform • Abstracts OS as SQLite tables • Performant • Extensible • https://osquery.io
What is osquery?
osquery architecture
How does osquery work?dispatcher
SchedulerRunnerCarver
ExtensionRunnerCore
ConfigRefresh Runner
DistributedRunner
WatcherRunner
InternalRunnable
How does osquery work?dispatcher
Carver
ExtensionRunnerCore
ConfigRefresh Runner
DistributedRunner
WatcherRunner
InternalRunnable
SchedulerRunner
How does osquery work?dispatcher
Carver
ExtensionRunnerCore
ConfigRefresh Runner
DistributedRunner
SchedulerRunner
InternalRunnable
WatcherRunner
How does osquery work?dispatcher
Carver
ExtensionRunnerCore
ConfigRefresh Runner
SchedulerRunner
InternalRunnable
WatcherRunner
DistributedRunner
ad-hoc distributed live on-demand deployment
TLS
configuration
storage
Analyst
cloud
off prem
enterprise
TLS
configuration
storage
Analyst
cloud
off prem
enterprise
TLS
configuration
storage
Analyst
cloud
off prem
enterprise
TLS
configuration
storage
Analyst
cloud
off prem
enterprise
TLS
configuration
storage
Analyst
cloud
off prem
enterprise
TLS
configuration
storage
Analyst
cloud
off prem
enterprise
TLS
configuration
storage
Analyst
cloud
off prem
enterprise
TLS
configuration
storage
Analyst
cloud
off prem
enterprise
distributed queries
TLS
configuration
storageSELECT p.pid, p.name, lp.port, lp.address, lp.path FROM processes AS p JOIN listening_ports AS lp USING (pid);
Analyst
cloud
off prem
enterprise
Analyst
TLS
configuration
storage
SELECT p.pid, p.name, lp.port, lp.address, lp.path FROM processes AS p JOIN listening_ports AS lp USING (pid);
cloud
off prem
enterprise
TLS
configuration
storage
AnalystSELECT p.pid, p.name, lp.port, lp.address, lp.path FROM processes AS p JOIN listening_ports AS lp USING (pid);
cloud
off prem
enterprise
TLS
configuration
storage
SELECT p.pid, p.name, lp.port, lp.address, lp.path FROM processes AS p JOIN listening_ports AS lp USING (pid);
Analyst
cloud
off prem
enterprise
TLS
configuration
storage
SELECT p.pid, p.name, lp.port, lp.address, lp.path FROM processes AS p JOIN listening_ports AS lp USING (pid);
Analyst
cloud
off prem
enterprise
TLS
configuration
storage
SELECT p.pid, p.name, lp.port, lp.address, lp.path FROM processes AS p JOIN listening_ports AS lp USING (pid);
Analyst
cloud
off prem
enterprise
TLS
configuration
storage[ {"address":"0.0.0.0","name":"rapportd","path":"","pid":"507","port":"49177"}, {"address":"::","name":"rapportd","path":"","pid":"507","port":"49177"}, {"address":"0.0.0.0","name":"SystemUIServer","path":"","pid":"601","port":"64562"}, {"address":"127.0.0.1","name":"scm_daemon","path":"","pid":"749","port":"15432"}, {"address":"127.0.0.1","name":"Dropbox","path":"","pid":"771","port":"17603"}]
SELECT p.pid, p.name, lp.port, lp.address, lp.path FROM processes AS p JOIN listening_ports AS lp USING (pid);
Analyst
cloud
off prem
enterprise
Repurposing Runnables
How does osquery work?dispatcher
ExtensionRunnerCore
ConfigRefresh Runner
SchedulerRunner
InternalRunnable
WatcherRunner
DistributedRunnerCarver
How does osquery the carver work?dispatcher
ExtensionRunnerCore
ConfigRefresh Runner
SchedulerRunner
InternalRunnable
WatcherRunner
DistributedRunner
Carver
How does osquery the carver work?dispatcher
ExtensionRunnerCore
ConfigRefresh Runner
SchedulerRunner
InternalRunnable
WatcherRunner
DistributedRunner
Carver
SELECT * FROM carves WHERE carve=1 AND ...
How does osquery the carver work?dispatcher
ExtensionRunnerCore
ConfigRefresh Runner
SchedulerRunner
InternalRunnable
WatcherRunner
DistributedRunner
Carver
SELECT * FROM carves WHERE carve=1 AND ...
Carver::Carver
How does osquery the carver work?dispatcher
ExtensionRunnerCore
ConfigRefresh Runner
SchedulerRunner
InternalRunnable
WatcherRunner
DistributedRunner
Carver
SELECT * FROM carves WHERE carve=1 AND ...
Carver::Carver
1. Create a temp file store
How does osquery the carver work?dispatcher
ExtensionRunnerCore
ConfigRefresh Runner
SchedulerRunner
InternalRunnable
WatcherRunner
DistributedRunner
Carver
SELECT * FROM carves WHERE carve=1 AND ...
2. Grab the specified files
1. Create a temp file store
Carver::Carver
How does osquery the carver work?dispatcher
ExtensionRunnerCore
ConfigRefresh Runner
SchedulerRunner
InternalRunnable
WatcherRunner
DistributedRunner
Carver
SELECT * FROM carves WHERE carve=1 AND ...
3. Create a .tar
Optionally compress
2. Grab the specified files
1. Create a temp file store
Carver::Carver
How does osquery the carver work?dispatcher
ExtensionRunnerCore
ConfigRefresh Runner
SchedulerRunner
InternalRunnable
WatcherRunner
DistributedRunner
Carver
SELECT * FROM carves WHERE carve=1 AND ...
boost::filesystem::temp_directory_path()
Carver::carve(const boost::fs::path& p)
archive(std::set<boost::fs::path> paths)
compress(const boost::fs::path& p)
4. POST the data
3. Create a .tar
Optionally compress
2. Grab the specified files
1. Create a temp file store
Carver::Carver
How does osquery the carver work?dispatcher
ExtensionRunnerCore
ConfigRefresh Runner
SchedulerRunner
InternalRunnable
WatcherRunner
DistributedRunner
Carver
SELECT * FROM carves WHERE carve=1 AND ...
Update DB
{ 'time': 1537852244.4733381, 'sha256': '<sha256>', 'size': 123, 'path': '<Path of Carves', 'status': 'SUCCESS' }
4. POST the data
3. Create a .tar
Optionally compress
2. Grab the specified files
1. Create a temp file store
Carver::Carver
The file carver
TLS
configuration
storage
cloud
off prem
enterprise
configuration
storage
SELECT * FROM carves WHERE carve=1 AND path='/tmp/evil';
cloud
off prem
enterprise
TLS
off prem
configuration
storage
SELECT * FROM carves WHERE carve=1 AND path='/tmp/evil';
cloud
enterprise
TLS
configuration
storage
SELECT * FROM carves WHERE carve=1 AND path='/tmp/evil';
cloud
off prem
enterprise
TLS
configuration
storage
SELECT * FROM carves WHERE carve=1 AND path='/tmp/evil';
cloud
off prem
enterprise
TLS
Use Cases
• Auto Carving • New files • Case artifacts
Use Cases
• Auto Carving • New files • Case artifacts
• Asynchronous acquisitions • Endpoint is off corp network • Laptop is shut and then opened
Use Cases
Surprises
SELECT * FROM carves WHERE carve=1 AND path LIKE '/Users/%/Downloads/%';
Carver Wins
Limitations
• Watchdog and size limits
Carver Limitations
• Watchdog and size limits • "carves"
Carver Limitations
• Watchdog and size limits • "carves" • Block POST retry
Carver Limitations
• Watchdog and size limits • "carves" • Block POST retry • Limited to TLS endpoints
Carver Limitations
• Watchdog and size limits • "carves" • Block POST retry • Limited to TLS endpoints • How does one carve?
Carver Limitations
Carver Endpoints
• Zentral • github.com/zentralopensource/zentral
• SGT • github.com/OktaSecurityLabs/sgt
Carver Endpoints
• Doorman issue • github.com/mwielgoszewski/doorman/issues/120
• osquery integration test • osquery/tools/tests/test_http_server.py
Carver Endpoints Specs and Docs
• Nick Anderson • Security Engineer at Facebook
• thor@fb.com • Super legit, not an alias
• Github - github.com/muffins
• Twitter - twitter.com/poppyseedplehzr
• Slack - thor
•
Thanks! Questions?
top related