controlling system calls and protecting application data in … · 2011-02-28 · controlling...
TRANSCRIPT
![Page 1: Controlling System Calls and Protecting Application Data in … · 2011-02-28 · Controlling System Calls and Protecting Application Data in Virtual Machines Koichi Onoue* Yoshihiro](https://reader033.vdocument.in/reader033/viewer/2022041804/5e53732d29b5bc2b7c56a494/html5/thumbnails/1.jpg)
Controlling System Calls and Protecting Application Data
in Virtual Machines
Koichi Onoue* Yoshihiro Oyama** Akinori Yonezawa*
* The University of Tokyo** The University of Electro-Communications
![Page 2: Controlling System Calls and Protecting Application Data in … · 2011-02-28 · Controlling System Calls and Protecting Application Data in Virtual Machines Koichi Onoue* Yoshihiro](https://reader033.vdocument.in/reader033/viewer/2022041804/5e53732d29b5bc2b7c56a494/html5/thumbnails/2.jpg)
Protection for Applications
• Security systems has been widely applied to provide secure computing environments– Sandboxing systems
– Intrusion detecion/prevension systems (IDSes/IPSes)
– Anti-virus tools
OS kernel
Security system
Applicationcontrol
Confidential data
![Page 3: Controlling System Calls and Protecting Application Data in … · 2011-02-28 · Controlling System Calls and Protecting Application Data in Virtual Machines Koichi Onoue* Yoshihiro](https://reader033.vdocument.in/reader033/viewer/2022041804/5e53732d29b5bc2b7c56a494/html5/thumbnails/3.jpg)
Security Systems Can Also Be Compromised!
• Security systems and the other applications are running in the same execution space
Application 1
OS kernel
Security system
Application 2
Confidential data
Application 1 was compromised !
control✗
![Page 4: Controlling System Calls and Protecting Application Data in … · 2011-02-28 · Controlling System Calls and Protecting Application Data in Virtual Machines Koichi Onoue* Yoshihiro](https://reader033.vdocument.in/reader033/viewer/2022041804/5e53732d29b5bc2b7c56a494/html5/thumbnails/4.jpg)
Advantages of Virtual Machine Monitor(VMM) in Terms of Security
• VMM provides strong isolation between VMs– VMM prevents a
compromised VM from attacking the other VMs
• VMM can control access to physical resources such as physical memory and a disk– VMM is running at the
higher privileged level than VMs Hardware
Virtual machine monitor (VMM)
Virtual machine(VM)
OS (Guest OS)
Virtual machine(VM)
OS(Guest OS)
Application Application
![Page 5: Controlling System Calls and Protecting Application Data in … · 2011-02-28 · Controlling System Calls and Protecting Application Data in Virtual Machines Koichi Onoue* Yoshihiro](https://reader033.vdocument.in/reader033/viewer/2022041804/5e53732d29b5bc2b7c56a494/html5/thumbnails/5.jpg)
Our Goal
• Enhancing application security by a system running outside of VMs– In cooperation with VMM, the security system
controls behaviors of application and protects application data
VMM
Control VM Target VM
ApplicationSecuritysystem
![Page 6: Controlling System Calls and Protecting Application Data in … · 2011-02-28 · Controlling System Calls and Protecting Application Data in Virtual Machines Koichi Onoue* Yoshihiro](https://reader033.vdocument.in/reader033/viewer/2022041804/5e53732d29b5bc2b7c56a494/html5/thumbnails/6.jpg)
Our Approach
• Our system consists of program in VMM and program in control VM– They run outside of target VMs
• It controls system calls invoked by application process• It controls memory and file operations related to target
applications
Our system controls only the target applications specified by users
We extend a para-virtualization version of Xen
![Page 7: Controlling System Calls and Protecting Application Data in … · 2011-02-28 · Controlling System Calls and Protecting Application Data in Virtual Machines Koichi Onoue* Yoshihiro](https://reader033.vdocument.in/reader033/viewer/2022041804/5e53732d29b5bc2b7c56a494/html5/thumbnails/7.jpg)
Controlling System Calls fromOutside of target VMs
![Page 8: Controlling System Calls and Protecting Application Data in … · 2011-02-28 · Controlling System Calls and Protecting Application Data in Virtual Machines Koichi Onoue* Yoshihiro](https://reader033.vdocument.in/reader033/viewer/2022041804/5e53732d29b5bc2b7c56a494/html5/thumbnails/8.jpg)
Comparison between “w/o VMM” and “w/ VMM”
Securitysystems
(“w/o VMM”)
Security system in cooperation with VMM
(“w/ VMM”)
Attack against security systems
✗Not hard
○Hard
Execution states obtained by
security systems
○OS-level
✗Hardware-level
![Page 9: Controlling System Calls and Protecting Application Data in … · 2011-02-28 · Controlling System Calls and Protecting Application Data in Virtual Machines Koichi Onoue* Yoshihiro](https://reader033.vdocument.in/reader033/viewer/2022041804/5e53732d29b5bc2b7c56a494/html5/thumbnails/9.jpg)
Securitysystems
Security systems in cooperation with VMM
Attack against security systems
✗Not hard
○Hard
Execution states obtained by
security systems
○OS-level
✗Hardware-level
Goal for Controlling System Calls
Our goal
Semantic gap
![Page 10: Controlling System Calls and Protecting Application Data in … · 2011-02-28 · Controlling System Calls and Protecting Application Data in Virtual Machines Koichi Onoue* Yoshihiro](https://reader033.vdocument.in/reader033/viewer/2022041804/5e53732d29b5bc2b7c56a494/html5/thumbnails/10.jpg)
Approach to Controlling System Calls
• Controlling system calls from outside of VMs– Using information on target OSes
created in kernel build
– Conforming to security policies
VMM
Control VM Target VM
application
Securitysystem
Security policy System call is
invoked
control
![Page 11: Controlling System Calls and Protecting Application Data in … · 2011-02-28 · Controlling System Calls and Protecting Application Data in Virtual Machines Koichi Onoue* Yoshihiro](https://reader033.vdocument.in/reader033/viewer/2022041804/5e53732d29b5bc2b7c56a494/html5/thumbnails/11.jpg)
Bridging the Semantic Gap
• What a VMM can observe– Events :Privileged instructions, interrupts, …
– Execution States:Registers, memory pages, …
• What security systems require– Events :System calls, …
– Execution states: Process ID, system call number, …
Semantic gap
![Page 12: Controlling System Calls and Protecting Application Data in … · 2011-02-28 · Controlling System Calls and Protecting Application Data in Virtual Machines Koichi Onoue* Yoshihiro](https://reader033.vdocument.in/reader033/viewer/2022041804/5e53732d29b5bc2b7c56a494/html5/thumbnails/12.jpg)
Security Policy
• Specifies controlled system calls withpattern matching
...open default: allow
fileEq(“/etc/passwd”)or filePrefixEq(“/etc/cron.d”)
deny(EPERM)...
![Page 13: Controlling System Calls and Protecting Application Data in … · 2011-02-28 · Controlling System Calls and Protecting Application Data in Virtual Machines Koichi Onoue* Yoshihiro](https://reader033.vdocument.in/reader033/viewer/2022041804/5e53732d29b5bc2b7c56a494/html5/thumbnails/13.jpg)
Controlling Memory and File Operations Related to Application Data
![Page 14: Controlling System Calls and Protecting Application Data in … · 2011-02-28 · Controlling System Calls and Protecting Application Data in Virtual Machines Koichi Onoue* Yoshihiro](https://reader033.vdocument.in/reader033/viewer/2022041804/5e53732d29b5bc2b7c56a494/html5/thumbnails/14.jpg)
Goal for Protecting Application Data
• Prevent compromised programs from leaking target data and tampering with them– We assume attackers read/write application data
with ptrace system call and kernel modules, etc.
VM
Target
OS kernel Application
Confidential data
Compromised program
Leaking
Tampering
Memory
Target
Virtual disk
![Page 15: Controlling System Calls and Protecting Application Data in … · 2011-02-28 · Controlling System Calls and Protecting Application Data in Virtual Machines Koichi Onoue* Yoshihiro](https://reader033.vdocument.in/reader033/viewer/2022041804/5e53732d29b5bc2b7c56a494/html5/thumbnails/15.jpg)
Approach to Protecting Application Data
• Hiding “real” application data on memory and a virtual disk from compromised programs– Compromised programs include target OS kernel
• Application data on memory– Code region, data region, stack region, etc.➔VMM multiplexes physical pages
• Overshadow[Chen et al., 2008]• [Rosenblum et al.,2008]
• Application data on a virtual disk– Executables, configuration files, etc.➔ Control VM manages them
![Page 16: Controlling System Calls and Protecting Application Data in … · 2011-02-28 · Controlling System Calls and Protecting Application Data in Virtual Machines Koichi Onoue* Yoshihiro](https://reader033.vdocument.in/reader033/viewer/2022041804/5e53732d29b5bc2b7c56a494/html5/thumbnails/16.jpg)
OS Memory Management
Virtual address space
Physical address space
Application
Application
OS addresstranslation
![Page 17: Controlling System Calls and Protecting Application Data in … · 2011-02-28 · Controlling System Calls and Protecting Application Data in Virtual Machines Koichi Onoue* Yoshihiro](https://reader033.vdocument.in/reader033/viewer/2022041804/5e53732d29b5bc2b7c56a494/html5/thumbnails/17.jpg)
VMM Memory Management
Target VM virtual address space
(gVA)
Target VM physical address
space(gPA)
Application
application
gVA → hPA
VMM address translation
gPA → hPA
VMM physical address space
(hPA)
Application
17
![Page 18: Controlling System Calls and Protecting Application Data in … · 2011-02-28 · Controlling System Calls and Protecting Application Data in Virtual Machines Koichi Onoue* Yoshihiro](https://reader033.vdocument.in/reader033/viewer/2022041804/5e53732d29b5bc2b7c56a494/html5/thumbnails/18.jpg)
Protecting Memory (1/2)
• According to the operational mode, a VMM switches accessible physical pages
Target VM virtual address space
(gVA)
Target VMphysical address
space(gPA)
VMM physical address space
(hPA)
ApplicationApplication
“Real” data
Dummy data
Accessible page at user-mode
Accessible page at kernel-mode
![Page 19: Controlling System Calls and Protecting Application Data in … · 2011-02-28 · Controlling System Calls and Protecting Application Data in Virtual Machines Koichi Onoue* Yoshihiro](https://reader033.vdocument.in/reader033/viewer/2022041804/5e53732d29b5bc2b7c56a494/html5/thumbnails/19.jpg)
Protecting Memory (2/2)
• VMM switches page tableswhen the operational mode is changed– Exception/Interrupt handling
– System call handling
![Page 20: Controlling System Calls and Protecting Application Data in … · 2011-02-28 · Controlling System Calls and Protecting Application Data in Virtual Machines Koichi Onoue* Yoshihiro](https://reader033.vdocument.in/reader033/viewer/2022041804/5e53732d29b5bc2b7c56a494/html5/thumbnails/20.jpg)
Approach to Protecting Application File Data (1/5)
• Control VM manages “real” target files– Executables, configuration and data base files, etc.
– Security policy specifies target files
• Target VM manages “dummy” target files
VMM
Control VM Target VM
Security system
Security policy
“Real” configuration file
Application
“Real” executable
Dummy configuration
fileDummy
executable
![Page 21: Controlling System Calls and Protecting Application Data in … · 2011-02-28 · Controlling System Calls and Protecting Application Data in Virtual Machines Koichi Onoue* Yoshihiro](https://reader033.vdocument.in/reader033/viewer/2022041804/5e53732d29b5bc2b7c56a494/html5/thumbnails/21.jpg)
Approach to Protecting Application File Data (2/5)
VMM
Control VM Target VM
Security system
Configuration file
Application
Memory
Application readsa configuration file
![Page 22: Controlling System Calls and Protecting Application Data in … · 2011-02-28 · Controlling System Calls and Protecting Application Data in Virtual Machines Koichi Onoue* Yoshihiro](https://reader033.vdocument.in/reader033/viewer/2022041804/5e53732d29b5bc2b7c56a494/html5/thumbnails/22.jpg)
Approach to Protecting Application File Data (3/5)
VMM
Control VM Target VM
Security system
Configuration file
Application
Memory
VMM intercepts “read” system call
![Page 23: Controlling System Calls and Protecting Application Data in … · 2011-02-28 · Controlling System Calls and Protecting Application Data in Virtual Machines Koichi Onoue* Yoshihiro](https://reader033.vdocument.in/reader033/viewer/2022041804/5e53732d29b5bc2b7c56a494/html5/thumbnails/23.jpg)
Approach to Protecting Application File Data (4/5)
VMM
Control VM Target VM
Security system
Configuration file
Application
Security system emulates “read”
Memory
![Page 24: Controlling System Calls and Protecting Application Data in … · 2011-02-28 · Controlling System Calls and Protecting Application Data in Virtual Machines Koichi Onoue* Yoshihiro](https://reader033.vdocument.in/reader033/viewer/2022041804/5e53732d29b5bc2b7c56a494/html5/thumbnails/24.jpg)
Approach to Protecting Application File Data (5/5)
VMM
Control VM Target VM
Security system
Configuration file
Application
Memory
VMM notifies application of a result of “read”
![Page 25: Controlling System Calls and Protecting Application Data in … · 2011-02-28 · Controlling System Calls and Protecting Application Data in Virtual Machines Koichi Onoue* Yoshihiro](https://reader033.vdocument.in/reader033/viewer/2022041804/5e53732d29b5bc2b7c56a494/html5/thumbnails/25.jpg)
Conclusion
• We have proposed a system that enhances application security inside target VMs– Controlling of application behaviors
• Controlling of system calls from outside of target VMs
– Protecting application data on memory anda virtual disk
• Application memory data: VMM multiplexes target physical pages
• Application file data: Control VM manages them
![Page 26: Controlling System Calls and Protecting Application Data in … · 2011-02-28 · Controlling System Calls and Protecting Application Data in Virtual Machines Koichi Onoue* Yoshihiro](https://reader033.vdocument.in/reader033/viewer/2022041804/5e53732d29b5bc2b7c56a494/html5/thumbnails/26.jpg)
Thank you for your attention