![Page 1: 2020 LAS Collaborators’ Week...Vulnerability Detection Symbolic Execution without Source Code Malware Evolution and Triage Polymorphic vs Metamorphic Obfuscation & Detection Techniques](https://reader036.vdocument.in/reader036/viewer/2022062508/5ff077719e4fb7579e31115c/html5/thumbnails/1.jpg)
1
2020 LAS Collaborators’ Week
Dr. Alyson WilsonLAS Principal Investigator
Dr. Matt SchmidtLAS Director of Programs
Jamie RoseboroughLAS Director of Outreach
and Engagement
Dr. Christine BrughLAS Technical Program Manager
Dr. Jascha SwisherLAS Technical Program Manager
Lori WachterLAS Technical Program Manager
June 15-18, 2020
![Page 2: 2020 LAS Collaborators’ Week...Vulnerability Detection Symbolic Execution without Source Code Malware Evolution and Triage Polymorphic vs Metamorphic Obfuscation & Detection Techniques](https://reader036.vdocument.in/reader036/viewer/2022062508/5ff077719e4fb7579e31115c/html5/thumbnails/2.jpg)
Contact Info● General Inquiries: [email protected]
● Specific Inquiries:○ Alyson Wilson, [email protected]○ Matt Schmidt, [email protected]○ Jamie Roseborough, [email protected]
● LAS Collaborators Week Website: https://ncsu-las.org/2020-las-collaborators-day/
2
![Page 3: 2020 LAS Collaborators’ Week...Vulnerability Detection Symbolic Execution without Source Code Malware Evolution and Triage Polymorphic vs Metamorphic Obfuscation & Detection Techniques](https://reader036.vdocument.in/reader036/viewer/2022062508/5ff077719e4fb7579e31115c/html5/thumbnails/3.jpg)
3
LAS Collaborators Week Schedule● Monday, June 15: Plenary Session
● Tuesday, June 16: “How to Work with LAS” Sessions
● Wednesday, June 17: Technical “Office Hour” Sessions○ Analytic Rigor and Performance○ Data Triage○ Influence Campaigns
● Thursday, June 18: Technical “Office Hour” Sessions○ Machine Learning Integrity○ Human Machine Collaboration○ Selected Cybersecurity Challenges○ Additional Use Cases
![Page 4: 2020 LAS Collaborators’ Week...Vulnerability Detection Symbolic Execution without Source Code Malware Evolution and Triage Polymorphic vs Metamorphic Obfuscation & Detection Techniques](https://reader036.vdocument.in/reader036/viewer/2022062508/5ff077719e4fb7579e31115c/html5/thumbnails/4.jpg)
Plenary SessionMonday, June 15
● Overview of LAS and how we work
● Overview of 2021 LAS interest areas
● Overview of the white paper submission process
4
![Page 5: 2020 LAS Collaborators’ Week...Vulnerability Detection Symbolic Execution without Source Code Malware Evolution and Triage Polymorphic vs Metamorphic Obfuscation & Detection Techniques](https://reader036.vdocument.in/reader036/viewer/2022062508/5ff077719e4fb7579e31115c/html5/thumbnails/5.jpg)
“How to Work with LAS” SessionsTuesday, June 16
● Purpose○ Answer questions about logistics of working with LAS○ Provide general suggestions about how your research interests might align
with the different LAS interest areas.
● Individual sessions conducted via Zoom○ Sign up for 10-minute time slot at: https://ncsu-las.org/2020-las-collaborators-day/○ Two available blocks of time slots
■ 09:00a – 11:00a (EDT)■ 01:00p – 03:00p (EDT)
5
![Page 6: 2020 LAS Collaborators’ Week...Vulnerability Detection Symbolic Execution without Source Code Malware Evolution and Triage Polymorphic vs Metamorphic Obfuscation & Detection Techniques](https://reader036.vdocument.in/reader036/viewer/2022062508/5ff077719e4fb7579e31115c/html5/thumbnails/6.jpg)
Technical “Office Hour” SessionsWednesday, June 17 & Thursday, June 18
● Purpose○ Provide an opportunity to speak with LAS staff who have related interests
about potential project ideas and collaborations.
● Individual sessions conducted via Zoom○ Sign up for 10-minute time slot at: https://ncsu-las.org/2020-las-collaborators-day/○ Wednesday, June 17
■ 09:00a – 11:30a (EDT) : Analytic Rigor and Performance (CFWP Section 3.1)■ 12:00p – 02:30p (EDT) : Influence Campaigns (CFWP Section 4.1)■ 02:30p – 05:00p (EDT) : Data Triage (CFWP Section 3.4)
○ Thursday, June 18■ 09:00a – 11:30a (EDT) : Machine Learning Integrity (CFWP Section 3.2)■ 12:00p – 02:30p (EDT) : Selected Cyber Security Challenges (CFWP Section 4.2)■ 12:00p – 02:30p (EDT) : Additional Use Cases (CFWP Section 4.3)■ 02:30p – 05:00p (EDT) : Human Machine Collaboration (CFWP Section 3.3) 6
![Page 7: 2020 LAS Collaborators’ Week...Vulnerability Detection Symbolic Execution without Source Code Malware Evolution and Triage Polymorphic vs Metamorphic Obfuscation & Detection Techniques](https://reader036.vdocument.in/reader036/viewer/2022062508/5ff077719e4fb7579e31115c/html5/thumbnails/7.jpg)
Questions● If you would like to ask a question please use the Q&A feature in
Zoom
● We have multiple places in the talk where we will pause to answer questions from the Q&A
● If you are unable to ask your question through the Q&A feature today, please email [email protected] with your question, and we will get back to you.
7
![Page 8: 2020 LAS Collaborators’ Week...Vulnerability Detection Symbolic Execution without Source Code Malware Evolution and Triage Polymorphic vs Metamorphic Obfuscation & Detection Techniques](https://reader036.vdocument.in/reader036/viewer/2022062508/5ff077719e4fb7579e31115c/html5/thumbnails/8.jpg)
What is the Laboratory for Analytic Sciences?LAS is a mission-oriented academic-industry-government research collaboration that works at the intersection of technology and tradecraft.
8https://ncsu-las.org/
![Page 9: 2020 LAS Collaborators’ Week...Vulnerability Detection Symbolic Execution without Source Code Malware Evolution and Triage Polymorphic vs Metamorphic Obfuscation & Detection Techniques](https://reader036.vdocument.in/reader036/viewer/2022062508/5ff077719e4fb7579e31115c/html5/thumbnails/9.jpg)
9
Advance the tradecraft of intelligence analysis while leveraging novel and recent advances in research and technology
● Investigate technical approaches with the potential to address analysis challenges
● Develop analytic tradecraft that leverages research and technology to address mission needs
● Transition technology and tradecraft to partners who can operationalize and scale solutions
![Page 10: 2020 LAS Collaborators’ Week...Vulnerability Detection Symbolic Execution without Source Code Malware Evolution and Triage Polymorphic vs Metamorphic Obfuscation & Detection Techniques](https://reader036.vdocument.in/reader036/viewer/2022062508/5ff077719e4fb7579e31115c/html5/thumbnails/10.jpg)
10
How do we work at LAS?● Mission-relevant projects
● ~90% of our work is unclassified
● Integrated, team-based approach
● Guidance is intentionally open-ended, as we are expecting you to help shape the direction of the projects
![Page 11: 2020 LAS Collaborators’ Week...Vulnerability Detection Symbolic Execution without Source Code Malware Evolution and Triage Polymorphic vs Metamorphic Obfuscation & Detection Techniques](https://reader036.vdocument.in/reader036/viewer/2022062508/5ff077719e4fb7579e31115c/html5/thumbnails/11.jpg)
● 24 faculty (and ≈ 35 students) at 9 unique universities
● 7 industry partners and 1 national lab
● ≈ 50 government staff/IC partners
● 14 NCSU staff
11
Who is participating with LAS in 2020?
![Page 12: 2020 LAS Collaborators’ Week...Vulnerability Detection Symbolic Execution without Source Code Malware Evolution and Triage Polymorphic vs Metamorphic Obfuscation & Detection Techniques](https://reader036.vdocument.in/reader036/viewer/2022062508/5ff077719e4fb7579e31115c/html5/thumbnails/12.jpg)
12
What are we looking for in 2021 Collaborators?
● Immersive○ Iterative approaches to solutions○ Opportunistic approaches to solutions
● Interdisciplinary○ Researchers, developers, and practitioners○ STEM, humanities, and social sciences
● Relevant Expertise○ Relevant to their own activities○ Potentially relevant to other activities
![Page 13: 2020 LAS Collaborators’ Week...Vulnerability Detection Symbolic Execution without Source Code Malware Evolution and Triage Polymorphic vs Metamorphic Obfuscation & Detection Techniques](https://reader036.vdocument.in/reader036/viewer/2022062508/5ff077719e4fb7579e31115c/html5/thumbnails/13.jpg)
13
What are we looking for in 2021 Projects?● Impact
Will a successful outcome have a positive impact for intelligence analysts?
● InnovationIs a new approach proposed, or does it utilize new capabilities?
● EngagementAre LAS stakeholders interested in collaborating on the project?
![Page 14: 2020 LAS Collaborators’ Week...Vulnerability Detection Symbolic Execution without Source Code Malware Evolution and Triage Polymorphic vs Metamorphic Obfuscation & Detection Techniques](https://reader036.vdocument.in/reader036/viewer/2022062508/5ff077719e4fb7579e31115c/html5/thumbnails/14.jpg)
● New Understanding○ Experimental data○ Research Papers
● New Tradecraft○ Storyboards ○ Documented Workflows
● New Capabilities○ Proofs-of-concept (e.g. Jupyter Notebooks)○ Prototypes
14
What are we looking for in 2021 Outcomes?
![Page 15: 2020 LAS Collaborators’ Week...Vulnerability Detection Symbolic Execution without Source Code Malware Evolution and Triage Polymorphic vs Metamorphic Obfuscation & Detection Techniques](https://reader036.vdocument.in/reader036/viewer/2022062508/5ff077719e4fb7579e31115c/html5/thumbnails/15.jpg)
Questions● If you would like to ask a question please use the Q&A feature in
Zoom
15
![Page 16: 2020 LAS Collaborators’ Week...Vulnerability Detection Symbolic Execution without Source Code Malware Evolution and Triage Polymorphic vs Metamorphic Obfuscation & Detection Techniques](https://reader036.vdocument.in/reader036/viewer/2022062508/5ff077719e4fb7579e31115c/html5/thumbnails/16.jpg)
16
Human-Machine Collaboration
Machine Learning Integrity
Influence, Cybersecurity, and Other Use Cases
Analytic Rigor and Performance
Triage
What are our areas of interest for 2021?
![Page 17: 2020 LAS Collaborators’ Week...Vulnerability Detection Symbolic Execution without Source Code Malware Evolution and Triage Polymorphic vs Metamorphic Obfuscation & Detection Techniques](https://reader036.vdocument.in/reader036/viewer/2022062508/5ff077719e4fb7579e31115c/html5/thumbnails/17.jpg)
17
Analytic Rigor and Performance
Analytic production and
journalism
Evaluating rigor in analytic workflows
Augmenting analytic
performance
Applying rigor to language analysis
Defining and Evaluating rigor and its components
Identifying the “Fundamental Five” of analyst performance
![Page 18: 2020 LAS Collaborators’ Week...Vulnerability Detection Symbolic Execution without Source Code Malware Evolution and Triage Polymorphic vs Metamorphic Obfuscation & Detection Techniques](https://reader036.vdocument.in/reader036/viewer/2022062508/5ff077719e4fb7579e31115c/html5/thumbnails/18.jpg)
from Amershi et al (2019)
Machine Learning Integrity
18
ML in ProductionDefine and support best practices for machine learning operations
People and MLImprove interactions between humans and algorithms
End UsersEncourage appropriate trust in automated predictions
ML human factors
Data ScientistsAccelerate development of reliable models
ML explainability
End Users as Data ScientistsEmpower individual end users to address their own challenges through ML
User-centric document classification
Label, build, deploy, monitor R&D not finished products
![Page 19: 2020 LAS Collaborators’ Week...Vulnerability Detection Symbolic Execution without Source Code Malware Evolution and Triage Polymorphic vs Metamorphic Obfuscation & Detection Techniques](https://reader036.vdocument.in/reader036/viewer/2022062508/5ff077719e4fb7579e31115c/html5/thumbnails/19.jpg)
Human Machine Collaboration
19
Recognizing IntentUnderstand what an analyst is trying to do
Modeling intent in open-world environments
Useful interventionsEffectively support analysts in achieving their goals
Comparative studies
Microsoft Office Assistant, used with permission from Microsoft. From Wikipedia User:Norm
from Crouser et al (2020)
from Farrell and Ware (2020)from Hong and Watson (to appear)from Guo et al (2020)
![Page 20: 2020 LAS Collaborators’ Week...Vulnerability Detection Symbolic Execution without Source Code Malware Evolution and Triage Polymorphic vs Metamorphic Obfuscation & Detection Techniques](https://reader036.vdocument.in/reader036/viewer/2022062508/5ff077719e4fb7579e31115c/html5/thumbnails/20.jpg)
20
Data Triage
Data TaggingData Retention
Information Retrieval
Data Prioritization
Data Exploration & Survey
Data Triage concerns the Classic Challenges of Big Data
![Page 21: 2020 LAS Collaborators’ Week...Vulnerability Detection Symbolic Execution without Source Code Malware Evolution and Triage Polymorphic vs Metamorphic Obfuscation & Detection Techniques](https://reader036.vdocument.in/reader036/viewer/2022062508/5ff077719e4fb7579e31115c/html5/thumbnails/21.jpg)
Influence CampaignsIndicators, origins, &
provenance
Message content
Impact & effectiveness
Countering malign
influence
21
“The collection of tactical information about an adversary as well as the dissemination of propaganda in pursuit of a competitive advantage over an opponent” (RAND)
Influence can be:
• Online or offline
• Authentic or inauthentic activity
• Targeted or broad
• “Innocuous” or malign
![Page 22: 2020 LAS Collaborators’ Week...Vulnerability Detection Symbolic Execution without Source Code Malware Evolution and Triage Polymorphic vs Metamorphic Obfuscation & Detection Techniques](https://reader036.vdocument.in/reader036/viewer/2022062508/5ff077719e4fb7579e31115c/html5/thumbnails/22.jpg)
Selected Cybersecurity Challenges
22
● Vulnerability Detection○ Symbolic Execution without Source Code
● Malware Evolution and Triage○ Polymorphic vs Metamorphic Obfuscation
& Detection Techniques
● Endpoint Detection and Response○ Machine Learning Research for EDR
● Cybersecurity Policy○ Effectiveness Assessment
![Page 23: 2020 LAS Collaborators’ Week...Vulnerability Detection Symbolic Execution without Source Code Malware Evolution and Triage Polymorphic vs Metamorphic Obfuscation & Detection Techniques](https://reader036.vdocument.in/reader036/viewer/2022062508/5ff077719e4fb7579e31115c/html5/thumbnails/23.jpg)
Prioritization of Voice Data
User-Centric Document
Categorization
Processing Uniquely
Structured Forms
Handwriting Recognition in Scanned Docs
Additional Use Cases
23
![Page 24: 2020 LAS Collaborators’ Week...Vulnerability Detection Symbolic Execution without Source Code Malware Evolution and Triage Polymorphic vs Metamorphic Obfuscation & Detection Techniques](https://reader036.vdocument.in/reader036/viewer/2022062508/5ff077719e4fb7579e31115c/html5/thumbnails/24.jpg)
Questions● If you would like to ask a question please use the Q&A feature in
Zoom
24
![Page 25: 2020 LAS Collaborators’ Week...Vulnerability Detection Symbolic Execution without Source Code Malware Evolution and Triage Polymorphic vs Metamorphic Obfuscation & Detection Techniques](https://reader036.vdocument.in/reader036/viewer/2022062508/5ff077719e4fb7579e31115c/html5/thumbnails/25.jpg)
25
LAS White Paper and Proposal Timeline
● May 28, 2020 Call for White Papers
● June 15-18, 2020 LAS Collaborators Week
● July 17, 2020 White Papers Due● Sept 15, 2020 Preliminary Notifications
● Nov 1, 2020 Final Notifications
● Jan 1, 2021 Begin Period of Performance
● Dec 31, 2021 End Period of Performance
![Page 26: 2020 LAS Collaborators’ Week...Vulnerability Detection Symbolic Execution without Source Code Malware Evolution and Triage Polymorphic vs Metamorphic Obfuscation & Detection Techniques](https://reader036.vdocument.in/reader036/viewer/2022062508/5ff077719e4fb7579e31115c/html5/thumbnails/26.jpg)
White Paper Submissions
● In order to propose work, you must submit a white paper ● More than one submission is fine. You should submit one white
paper for each project idea you have.
● You may submit team white papers with more than one performer.
White Papers Due July 17, 2020
26
![Page 27: 2020 LAS Collaborators’ Week...Vulnerability Detection Symbolic Execution without Source Code Malware Evolution and Triage Polymorphic vs Metamorphic Obfuscation & Detection Techniques](https://reader036.vdocument.in/reader036/viewer/2022062508/5ff077719e4fb7579e31115c/html5/thumbnails/27.jpg)
White Paper Submissions
● All white papers must be submitted through web-based tool○ Link: https://whitepapers.ncsu-las.net
● Each white paper submission must include:○ Title○ All Funded PIs and Main POC○ Abstract○ Budget Request○ Technical Description
We ask that your abstract and white paper NOT contain classified, proprietary, or sensitive information of any kind.
27
NOTE: These will be entered separately in the submission tool and do not have to be repeated in the Technical Description
![Page 28: 2020 LAS Collaborators’ Week...Vulnerability Detection Symbolic Execution without Source Code Malware Evolution and Triage Polymorphic vs Metamorphic Obfuscation & Detection Techniques](https://reader036.vdocument.in/reader036/viewer/2022062508/5ff077719e4fb7579e31115c/html5/thumbnails/28.jpg)
White Paper Content Guidelines● Detailed guidelines for what to include in a white paper are given in
Section 6.2 of the Call for White Papers○ Link: https://ncsu-las.org/2021-call-for-white-papers/
● Generally, the most important parts of your white paper will be the descriptions of:
○ The proposed effort (what question are you answering or problem are you solving)
○ The proposed approach (how will you address the question/problem)○ How the work aligns with LAS areas of interest○ The specific deliverables you expect from your work
28
![Page 29: 2020 LAS Collaborators’ Week...Vulnerability Detection Symbolic Execution without Source Code Malware Evolution and Triage Polymorphic vs Metamorphic Obfuscation & Detection Techniques](https://reader036.vdocument.in/reader036/viewer/2022062508/5ff077719e4fb7579e31115c/html5/thumbnails/29.jpg)
White Paper Structure Guidelines
● Each whitepaper should be no more than 2 pages
● Optional additional page to discuss possible extensions to 2022
● Optional additional page to describe team capabilities, although a link to a website is preferred
29
![Page 30: 2020 LAS Collaborators’ Week...Vulnerability Detection Symbolic Execution without Source Code Malware Evolution and Triage Polymorphic vs Metamorphic Obfuscation & Detection Techniques](https://reader036.vdocument.in/reader036/viewer/2022062508/5ff077719e4fb7579e31115c/html5/thumbnails/30.jpg)
White Paper Budget GuidelinesAcademic Partners● Standard award is equivalent to:
○ One month of summer salary support or academic release, plus○ One 12-month graduate student, plus ○ $3,600 in other direct costs
● Award can be used for post-docs, undergrads, etc., but must stay within total budget
● You may submit up to three additional scope options at the level of one additional graduate student each
30
![Page 31: 2020 LAS Collaborators’ Week...Vulnerability Detection Symbolic Execution without Source Code Malware Evolution and Triage Polymorphic vs Metamorphic Obfuscation & Detection Techniques](https://reader036.vdocument.in/reader036/viewer/2022062508/5ff077719e4fb7579e31115c/html5/thumbnails/31.jpg)
White Paper Budget GuidelinesIndustry Partners● Standard award is $250k or less, which includes all direct and
indirect costs
● You may submit up to three additional scope options at the level of an additional $100k each
● If these levels of effort do not seem appropriate to the work you would like to propose, please contact Dr. Matt Schmidt, LAS Director of Programs, [email protected], to discuss other options.
31
![Page 32: 2020 LAS Collaborators’ Week...Vulnerability Detection Symbolic Execution without Source Code Malware Evolution and Triage Polymorphic vs Metamorphic Obfuscation & Detection Techniques](https://reader036.vdocument.in/reader036/viewer/2022062508/5ff077719e4fb7579e31115c/html5/thumbnails/32.jpg)
Questions● If you would like to ask a question please use the Q&A feature in
Zoom
32
![Page 33: 2020 LAS Collaborators’ Week...Vulnerability Detection Symbolic Execution without Source Code Malware Evolution and Triage Polymorphic vs Metamorphic Obfuscation & Detection Techniques](https://reader036.vdocument.in/reader036/viewer/2022062508/5ff077719e4fb7579e31115c/html5/thumbnails/33.jpg)
33
LAS Collaborators Week Schedule● Monday, June 15: Plenary Session
● Tuesday, June 16: “How to Work with LAS” Sessions
● Wednesday, June 17: Technical “Office Hour” Sessions○ Analytic Rigor and Performance○ Data Triage○ Influence Campaigns
● Thursday, June 18: Technical “Office Hour” Sessions○ Machine Learning Integrity○ Human Machine Collaboration○ Selected Cybersecurity Challenges○ Additional Use Cases