Defense Against Defense Against The Dark Arts:The Dark Arts:Using Computer Security To Using Computer Security To Teach Core Computer Science Teach Core Computer Science ConceptsConcepts
Mark W. Bailey
Associate Professor, Hamilton CollegeVisiting Professor, University of Virginia
OutlineOutline
Goals, Context
Course Overview
Tools
Course Material
Assignment Examples
Core CS Concepts
Summary
ObservationsObservations
Declining CS enrollments nationwide
Need to demonstrate CS relevance
Difficult to include new courses in limited curricula
Security issues are of concern to everyone
Future decision makers must be literate to make informed policy decisions
Computer security provides an ideal framework for attracting and engaging students
GoalsGoals
Develop an innovative set of computer security courses
Include courses for students without computer programming expertise
Courses should be suitable across the spectrum of colleges and universities
Package and document courses to facilitate widespread adoption
Participating SchoolsParticipating Schools
Hamilton College (Liberal Arts)
Mark Bailey
University of Virginia(Research University)
Jack Davidson
Virginia State University(HBCU)
Jeff Zadeh
Proposed CoursesProposed Courses
C4: Advanced Topicsin Computer
Systems Security
C1: Introduction toComputer Systems
Security
C3: CounteringNetwork Threats:
Principles and Techniques
C2: Anti-virusPrinciples and
Techniques
C4: Advanced Topicsin Computer
Systems Security
C1: Introduction toComputer Systems
Security
C3: CounteringNetwork Threats:
Principles and Techniques
C2: Anti-virusPrinciples and
Techniques
Proposed CoursesProposed Courses
C1 and C4 require no programming experience
Suitable for liberal arts and engineering students wishing to obtain technical literacy
Use case study approach
CS majors could take all four to obtain a strong background in security
Courses C2 and C3 have programming assignments
C4: Advanced Topicsin Computer
Systems Security
C1: Introduction toComputer Systems
Security
C3: CounteringNetwork Threats:
Principles and Techniques
C2: Anti-virusPrinciples and
Techniques
C4: Advanced Topicsin Computer
Systems Security
C1: Introduction toComputer Systems
Security
C3: CounteringNetwork Threats:
Principles and Techniques
C2: Anti-virusPrinciples and
Techniques
OutlineOutline
Goals, Context
Course Overview
Tools
Course Material
Assignment Examples
Core CS Concepts
Summary
C2: Defense Against The Dark C2: Defense Against The Dark ArtsArtsFocuses on anti-virus principles and techniques
Prerequisites: Programming and assembly
Reinforces assembly language
Introduces programming vulnerabilities
Emphasizes virus prevention, detection, and disinfection
Designed for third and fourth year students
Anti-Virus Course Anti-Virus Course MotivationMotivation
Threats to computers systems, such as viruses and worms, are a serious problem
CS students need to understand malware schemes and defenses against them
Teaches how to detect and defeat malicious software
Analyzing programs demands application of core theoretical concepts of CS
Teaches these concepts in an application area of great topical interest
OutlineOutline
Goals, Context
Course Overview
Tools
Course Material
Assignment Examples
Core CS Concepts
Summary
ToolsTools
Anti-virus programs need toDisassemble binary codeAnalyze and reason about codeModify, or fix codeReassemble binary code
Many of these operations are performed by compilers
Phoenix Compiler SuitePhoenix Compiler Suite
A cutting-edge suite of compilers and tools from Microsoft Research
Scalable, configurable, extensible, compilation infrastructure
Configurable for new tools, and purposesEasy insertion of plug-ins at any point in analysis sequenceWell defined, API’s encouraging analysis and transformation reuse
Supports binary manipulation
Phoenix IR Phoenix IR Raising/LoweringRaising/Lowering
Notice that the flow arrows go in both directionsA binary (in EIR form) can be:
Raised all the way to HIR, transformed
Lowered to MIR, transformed
Lowered to LIR, transformed
Then written back out as a new binary
ASTs (Abstract Syntax Trees)
Source Code (C++)
C++ Front End
C# Front End VB Front End
Source Code (C#)
Source Code (Visual Basic)
HIR (High-level IR)
MIR (Medium-level IR)
LIR (Low-level IR)
EIR (Encoded IR)
OutlineOutline
Goals, Context
Course Overview
Tools
Course Material
Assignment Examples
Core CS Concepts
Summary
TopicsTopics
Introduction, ethics, threat modelsTerminology, x86 architecture
Tools: Disassembly tools, Phoenix intro
Phoenix binary analysis tools
Viruses: Boot, interrupt, memory resident, executable file
Detecting viruses, regular expressions, lex, Chomsky hierarchy
Ken Thompson’s Turing Award Lecture
Topics (Continued)Topics (Continued)
Obfuscation, SSA form and PhoenixAnti-anti-virus schemes, analyzing systems
Retroviruses, tunneling, armor, encryption, oligomorphic, polymorphic, metamorphic
Software Dynamic Translation (SDT)Strata SDT framework; SDT security applicationsCode vulnerabilities and exploits; secure coding, static security analyzersRoot kits
Antivirus AssignmentsAntivirus Assignments
“Tricky jump” illustrated (C/assembly)
Reinforces assembly, introduces DUMPBIN
Dumping Phoenix IRDemonstrates construction of Phoenix analysis and instrumentation tools
Virus code detection using lexFinding junk instructions using SSA formRemoving obfuscating jumps using Phoenix control flow informationUsing Phoenix to prevent stack smashingStudent research presentations
OutlineOutline
Goals, Context
Course Overview
Tools
Course Material
Assignment Examples
Core CS Concepts
Summary
Tricky Jump IllustratedTricky Jump Illustrated
Application code:xor eax, eaxret
Can be replaced withpush offset malicious_funcret
Causing a jump instead of a returnStudents build tricky jump program
assembly language and debugging skill building
Virus Code DetectionVirus Code Detection
A common virus excerpt changes the IVT:
mov eax, 4CH
mov dword ptr [eax], edx
Which register is used is irrelevant
Disassemble executable using dumpbin
Recognize pattern using lex
Introduces regular expressions and their limitations
Finding Junk InstructionsFinding Junk Instructions
Viruses obfuscate using junk instructions
code SSA form
x = 2; x1 = 2; (useless)
y = 3; y1 = 3;
x = 4; x2 = 4;
y = y*x; y2 = y1*x2;
Students use Phoenix’s SSA to find junk
Removing Obfuscating Removing Obfuscating JumpsJumps
Simple sequences like:
x += 4;y +- (z – x);z -= 3;printf(“%d\n”, x);
Can be obfuscated using jumps:
x += 4;goto lab2;
lab3:z -= 3;goto lab4;
lab2:y += (z – x);goto lab3;
lab4:printf(“%d\n”,
x);
OutlineOutline
Goals, Context
Course Overview
Tools
Course Material
Assignment Examples
Core CS Concepts
Summary
Core CS ConceptsCore CS Concepts
Viruses often detected by pattern matchingRegular expressions in context of suspicious code patternsCode obfuscations make pattern matching inadequate in practiceChomsky language hierarchy is used to understand this limitationEquivalence of obfuscated code applies concepts from computability and theory of computation Students learn anti-virus software must often approximately solve an infeasible problem
Core CS Concepts Core CS Concepts ContinuedContinued
Pattern matching limitations suggest semantic analyses found in compilers
Simple dataflow analysis and SA form for de-obfuscation
Simple register allocation/assignment used to defeat register renumbering—another obfuscation technique
The Ongoing BattleThe Ongoing Battle
Endless advances in both malicious software and the tools that combat itAs each generation is defeated by security software, new techniques are developed that defeat the security toolsExamples: Armoring of viruses, obfuscation techniques, evolutionary viruses such as polymorphic, and metamorphic virusesThis “warfare” between good and bad forces has been found to intensify student interest
OutcomesOutcomes
Taught twice at Virginia, once at Hamilton and VSU (Spring 2007)UVa course overenrolled (had to turn away students)Used Phoenix infrastructure as a vehicle for teaching anti-virus techniques (compilers in disguise)Student feedback very positive
Students wanted more Phoenix assignments. “Would have liked to do more with Phoenix and less with Lex, but I guess time was too much of a limitation.”
SummarySummary
Course focuses on topic of concern to everyoneUses core CS concepts in an interesting application areaStudents use state-of-the-art tools to analyze real code (but not real viruses…)Course theme, title, and subject helps attract and fill coursesCourse materials suitable at a wide range of institutionsCourse materials will be made available in the Microsoft Academic Alliance Curriculum Repository
For More InformationFor More Information
Mark Bailey ([email protected])
Jack Davidson ([email protected])
Jeff Zadeh ([email protected])
Microsoft Research Faculty Summit 2007