software development life cycle – managing risk and measuring security
TRANSCRIPT
Thomas Malmberg
Risk & IT-Security Consultant, Mint Security Ltd
Software Development Life Cycle – Managing Risk and Measuring Security
Thomas Malmberg - @tsmalmbe tweet!
oWorks as an independent consultant @ Mint Security
oThis presentation contains anonymized and generalized cases based on real life experience during the past few years.
2010 2015
Mint Security Ltd
o Founded in 2015 – specialized in data & cyber security and IT-risk & security management
Software and Software Development Security
Purchasing, Project & Management Consulting
Architectures and Security Cyber & IT Risk Management
An Introduction
to this Presentation
(SECURE) Software Development Life Cycle
What is an SDLC?
oSoftware Development Life Cycle o Defines a process – does not imply any specific
model (agile, waterfall, whatever rocks your ship)
o Relevant for companies who are doing serious software development
o Is not the same as DevOps
o Works as a base framework for all development
o Can be audited, reviewed and assessed
o Can be aligned to security standards
o Can be communicated and understood
What’s your relation to the SDLC?
1. You develop software for yourself o You need quality and measurability
2. You develop products o You need reliability and continuity
3. You develop software for your customers
o You face contracts and warranty-demands
4. You buy custom developed software o You demand transparency and warranty
Why should my SDLC be secure?
“Organizations with a proper SDLC will experience an 80 percent decrease in critical vulnerabilities”
“Organizations that acquire products and services with just a 50 percent reduction in vulnerabilities will reduce configuration management and incident response costs by 75 percent each”
Is this presentation purely theoretical?
”SOMETHING
GOES WRONG”
Plant bad code
(add vulnerability)
CUSTOMER
Deliver (vulnerable) devices
CUSTOMER CUSTOMER
EXPLOIT !
Is this presentation purely theoretical?
Breached!?!
CUSTOMER
OTHER
CUSTOMER
CUSTOMER
OTHER
CUSTOMER
< CONFUSION > < CONFUSION^2 >
< CONFUSION^3 >
”SOMETHING GOES WRONG”
DEVELOPMENT ENVIRONMENT
BREACH
Juniper said that someone managed to get
into its systems and write “unauthorized
code” that “could allow a knowledgeable
attacker to gain administrative access.“
(CNN 18.12.2015)
How do I know my SDLC is secure?
o In a nutshell o You develop securely
o You develop secure software
o How do I achieve this? o Harden your processes
o Harden your development environment
o Harden your people
What we will cover today
1. Extending Your Security Policy to Cover the SDLC
2. Aligning with Development Standards and Best Practices
3. Assessing SDLC Risk – Different Viewpoints
4. Metrics and Correlations to the Rescue
5. Real Life Examples and Tools
Extending Your Security Policy
to Cover the SDLC
ISO27002 – PCI-DSS
Everyone has a security policy..
o ...but policies differ a lot – some are not formalized – or even written down!
oA proper SDLC may not demand a security policy – but a proper security policy demands an SDLC
o ISO27002 provides ”good common security ground” for this presentation
oThere are demands on the development process, the development environment as well as the deliverables
ISO27002 implicit requirements
oYou must protect your source code (9.4.5)
oDon’t mix development and production (12.1.4)
oYour development process must be secure (14.2.1)
oAssess your SDLC risk (14.2.6)
oPerform security testing (14.2.8)
NOTE! In addition to this, there are 1) OPERATIONAL requirements for running software and 2) other (but not as implicit) requirements that software development should adhere to.
Dissecting the ISO27002 for SDLC requirements
27002 Description SDLC viewpoints & examples
6.1.1 Information security roles and
responsibilities
Who is responsible for the environment
security
6.1.2 Segregation of duties Coders, administrators, network operators
7.2.2 Information security, awareness,
education and training
Let the people know there are policies and
requirements and how to work
8.1.3 Acceptable Use of Assets Guides for working in the development env
8.2.1 Classification of information Development code, production code, docs
9.1.1 Access Control Policy VCS system is not an island
9.2.1 Access to networks and network
services
Dev systems must be part of the
enterprise IAM
Dissecting the ISO27002 for SDLC requirements
27002 Description SDLC viewpoints & examples
9.2.3 Management of privileged access
rights
Limit roots and admins on the dev servers
9.4.1 Information access restriction Not all code is available to everyone
9.4.5 Access to program source code Prevent introduction of unauthorized
functionality
12.1.4 Separation of development,
testing and operational
environments
Self explanatory
12.4.1 Event logging Logging is not only for production systems
12.4.3 Administrator and operator logs What are your admin’s credentials doing
13.1.1 Network controls Segragate and protect
Dissecting the ISO27002 for SDLC requirements
27002 Description SDLC viewpoints & examples
13.1.3 Segregation in Networks Really, segregate
13.2.1 Information transfer policies and
procedures
Can your code travel
14.2.1 Secure development policy ”Code of conduct” for coding
14.2.2 System change control
procedures
Make the dev environment part of change
management
14.2.5 Secure system engineering
principles
Documented instructions for creating
systems
14.2.6 Secure development environment Assess your risks
14.2.8 System security testing Test the software you create
Suggestion for alignment
oAdd at least some concrete notes on the SDLC directly to your policy o If you have a well-documented SDLC that
includes security – put a reference to it into 14.2.1
o If you start fresh or have a big gap, write a policy for your SDLC based on hand-picked sections of the 27002 o This will help and guide your software team
What about PCI-DSS?
oThe only implicit demand is domain 6 – ”Develop and Maintain Secure Systems and Applications” – but o PCI-DSS and ISO27002 can be mapped to each
other (but it is far from 1:1 though so caveat emptor)
o Don’t force more than one set of requirements on your teams – so enforce and expose them to your security policy – and then
o Map your policy to other requirements and standards as you need to and see fit – behind the scenes
Dissecting the PCI-DSS for SDLC requirements
27002 PCI-DSS 27002 PCI-DSS 27002 PCI-DSS
6.1.1 7 9.2.3 7,2 13.1.3 1,6
6.1.2 7 9.4.1 3 13.2.1 4
7.2.2 12 9.4.5 6 14.2.1 6
8.1.3 12 12.1.4 3 14.2.2 6
8.2.1 - 12.4.1 10 14.2.5 2,6,11
9.1.1 7 12.4.3 10 14.2.6 6
9.2.1 8 13.1.1 1 14.2.8 6
Prepare to measure
oSpend some time thinking about KPI’s and KRI’s
oConsider adding some measuring guidelines to your policy as concrete demands
1. You get what you measure
2. You are what you measure
3. You can’t manage what you don’t measure
Aligning with Development Standards
and Best Practices
SAMM - DevOps
Ideas for a secure SDLC?
oDefining security domains and classifications – defining your ”security quality” o Managing development phase code access
o Software design
o Writing and auditing code
o Release management
o QA & pentesting
o Managing production environments
Example security domains
External
Services
Privacy
Intensive
Development process
Security requirements
Policies
PROCESSES
Internal
Services
SECURITY
DOMAINS
When writing detailed policies and setting demands,
choose and use an existing SDLC framework to ensure coverage.
Say hello to SAMM
OWASP SAMM - Software Assurance Maturity Model
Covers mostly development, not the dev environment
SAMM is a maturity model
1. Evaluate where you are – and more importantly – where you need to be
2. Evaluate your top-10 areas of risk – the areas where failure is not an option, or in any case very expensive – and create a roadmap
3. Evaluate areas where added security and quality can be capitalized upon
4. Start measuring as soon as possible to create a good baseline
Other frameworks than SAMM?
oNIST SP800-64
oBSIMM
oGAISP/GASSP
oCLASP
oMicrosoft SDL
oTouchpoints
oTSP-Secure
SAMM and ISO27002?
6.1.1
6.1.2
9.1.1
7.2.2
8.1.3
8.2.1
9.2.1
9.2.3
9.4.1
9.2.3
9.4.1
9.4.5
9.4.5
13.1.3
14.2.1
9.4.5
12.1.4
14.2.6
14.2.5 14.2.8
12.4.1
12.4.3
13.1.1
14.2.2
13.2.1
NOTE! Operations
security has
intentionally been
left out!
SAMM and DevOps?
SECOPS DEV
Underlying governance
and management
A quote on SAMM and DevOps
Adam Muntner, Security engineer at Mozilla
http://devops.com/2015/07/16/the-myth-of-devops-as-a-catalyst-to-improve-security/
“The thing to keep in mind is that the security
of an application environment is inherited –
it’s the aggregate result of all its component
pieces.”
”There is nothing inherent to DevOps that solves these [security]
problems and DevOps comes with its own potential pitfalls. The
problem is external to what DevOps can deliver. For organizations
that want to better understand the maturity of security in their
Software Development Lifecycle, OWASP OpenSAMM is a good place
to start.“
My take on SAMM and DevOps
oTo be truly secure, you will (anyway) need to spend endless amounts of time and money
oSAMM is about developing secure software, not developing in a secure environment
oSAMM does NOT fix everything
oDevOps is not a quick fix for your development environment security
Your secure (?) environment
VPN
QA
VCS & CI
DEV
Juniper said that someone
managed to get into its
systems and write
“unauthorized code”
I’m confused - how do I spend my time money then?
oMoney is spent based on risk assessments o Spend money where you can loose
money impact X probability
o Challenge – finding the risks and assessing them realistically is always hard and requires experience
o Saving on mapping out what your threats and weaknesses are is NOT a good way to spend money
Manage risk with tools already in your SDLC!
Assessing SDLC Risk –
Different Viewpoints
Risk Management for Software Development
The biggest risk is not assessing SDLC risk at all.
The second biggest risk is failing your risk assessment.
A key risk indicator (KRI) is a metric for
measuring the likelihood that the combined
probability of an event and its consequence will
exceed the organization's risk appetite and have
a profoundly negative impact on an
organization's ability to be successful.
Recap – Your role equals your risk-map
1. You develop software for yourself o You need quality and measurability
2. You develop products o You need reliability and continuity
3. You develop software for your customers
o You face contracts and warranty-demands
4. You buy custom developed software o You demand transparency and warranty
Risk – You develop software for yourself
1. Lack of (security) quality assurance
2. Unplanned and/or badly planned production installations
3. Neglicence towards architecture
4. Trivializing of threats
5. Nonexistent privilege management in the development environment
KRI – You develop software for yourself
1. The amount of security incidents related to your development
2. Missing/uninstalled security updates or patches
3. Security reviews (static code analysis) vs. production installations (releases)
4. Threat monitoring for installed software
5. Access management system vs. access logs
Risk – You develop products
1. Responsibility to your customer base – including legal liabilities
2. Difficulty to patch on time
3. Planting of rogue code in your environment
4. Outsourcing and IPR theft
KRI – You develop products
1. Quality Assurance results
2. Security testing results
3. Test pass rate
4. Test case comprehensiveness
5. External security reviews vs. internal security reviews
6. Access management system vs. access logs
Risk – You develop software for your customers
1. Responsibility to your customer base – including legal liabilities
2. Customer inability to define and verify security requirements
3. Nonexistent privilege management in the development environment – with the addition of customer access
KRI – You develop software for your customers
1. The contents of your agreements vs. change request logs
2. Access management system vs. access logs
3. External security reviews vs. internal security reviews
Risk – You buy custom developed software
1. Responsibilites are not fulfilled
2. Requirements and contracts do not reflect reality
3. Vulnerabilities due to bad quality code
4. Management of test-data (PII and EU requirements etc.)
KRI – You buy custom developed software
1. The contents of your agreements vs. change request logs
2. Supplier defect count vs. buyer acceptance tests vs. warranty fixes
3. Defect amounts in code audits and security tests
4. Process effectivity – deliverables vs. Vulnerabilities
Notes about measurements and KRI
Measuring is part of
your maturity Quantitative
measurements are
key – they provide
much more than
qualitative
guesswork KRI’s enable
proactivity instead of
reactivity
Metrics and Correlations
to the Rescue
Closer to practice with risk assessments and KRI’s
Where do we want to put our probes?
oThe deliverables – code o Code quality
o Vulnerabilities
o Changes
oThe development environment o Audit logs
o Access management
Do we have the necessary information?
oYes! Information is available but spread around different systems – examples: o Source code commits and fixes – in Git
o Builds, failures, QA tests – in the CI
o Static code analysis results – in the cloud
o Tickets and bugs from testing and production – in Jira
o Code audit and review results – somewhere
o QA and test environments – here and there
o Access – VPN-gateways, ssh-logs, firewalls, ...
o Bug Bounty programs
So how should this information be used?
o Extract information from the development process and measure and improve weak points and find and emphasize strong points
o Make agile more reportable without taking away the agility – enable both devs and managers
o C3 – Collect, Combine & Correlate o Results for release metadata
o Automated test results
o Static code analysis
o Tickets and bugs – from customers and manual test cycles
o Find problems in your releases, processes, code – find anomalies and “whoops” before the customers do
o Insight is everything
...you get some valuable KPI’s..
o Knowledge will let you optimize important business decisions o Amounts of features per release
o The need for external audits & code reviews
o Development team sizes & outsourcing
o Knowledge will let your team leaders & scrum masters better understand o How the teams work
o Where there is lack of knowledge
o Workload effects on quality
o Knowledge will enable your business and your development to work more closely together making decisions based on hard facts
..and insight into your level of security
o Audit trails – who did what and why it was approved – and by whom.
o Security defects and issues reported to issue-trackers are correlated to releases and projects
o Outsourced and “far-away-shored” teams can be monitored and assisted
o Source-code leaks and tampering can be detected
o Security scan results can be leveraged
o Automatic (static) code-analysis can be leveraged
o Correlate with threat intel
o Transparency and visibility in the process adds natural social pressure for correct behavior - You can even gamify quality and security!
Real Life
Examples and Tools
Quantitive Measurements
Important metrics
o Commits per release
o Commits per team
o Static code analysis results per release
o Issues & bugs per release
o New issues & bugs per week
o Average release time
o Open issues per release
o Code coverage
o Executed unit tests executed per relases
o Executed unit test per week
o Test coverage
o Team overall activity
o Failed test cycles per commit
o Failed unit tests per release
o Ad-hoc commits (”no ticket”)
o Access to version management system
o Access to test environments
o Dormant privileged users
o Build / version management system configuration changes
o Failed security tests
o Audit-logs for version management and CI
o New third-party libraries
o Deployers
o Deployment status
Correlations – KRI’s/KPI’s
o Commits vs. features
o Commits vs. reported issues
o Commits vs. test cycles
o Issues in static code analysis vs. manual code review issue amounts
o Open issues vs. closed issues
o Closed issues vs. added lines of code
o Test cycles vs. reported productions issues
o Unit test vs. static code analysis results
o Defect rate vs. test cycles
o Static code issues vs. commits
o Performance figures vs. commits
o Release frequency vs. code issues vs. defects
o Remote vs. local access to systems vs. threat intel
o Malicious patterns in remote access vs. normal access
Source: Secure Development Lifecycle – Eoin Keary & Jim Manico (OWASP)
The tools and the datasources
o System and access logs collected using Splunk forwarders (SSH and RDP) or syslog (VPN)
o Static code analysis using the Veracode platform API with Jenkins and integrated to Splunk using REST
o Security scan results can be integrated through logfiles or existing Splunk Apps (example: Nessus & Beyond Security)
o Performance data is integrated through logfiles
o Issue trackers (example: Jira) are integrated through database connectors or API’s
o CI, version management, release automation tools are integrated through logfiles
o Threat intel is added with Splunk Apps (example: Verizon)
Connect the bits and pieces
o A vast amount of sources can be hooked up and correlated
o Databases
o Text-based logs
o API’s
o Information can be visualized and displayed differently to various stakeholders
o NOTE! The schematic displays example sources and is used to serve an example and an overview – there is no limit to the imagination of the stakeholders – anything can be hooked up – but start simple
Generic case – Remote Development Teams
oReasoning and business case o Mitigate risk related to remote teams and/or
outsourcing.
o Issues o Sharing of VPN-tokens
o Tracking remote access IP’s against threat intel
o Alert on abnormal VPN-2-Code -behaviour
Generic case – Release Notes
oReasoning and business case o Add relevant and valuable security information
to your release notes.
o Issues o Commits not related to tickets
o Commits by people not part of the core team
o Changed vs. added vs. deleted code
o Residual static code scan findings
Generic case – Superuser access
oReasoning and business case o Development teams are managing their own
access, including superuser access.
o Privileges are handled ”under the radar”.
o Issues o Track all superuser accounts
o Monitor superuser access
o Alert on anomalous activities
Generic case – Policy violations
oReasoning and business case o We want to enforce and monitor our security
policies even in the farthest of IT-trenches
o Issues o Monitor password policies
o Address continuity requirements
This is the End
Almost
Studies and background material
o Coverity report o http://www.coverity.com/library/pdf/open_source_quality_report.pdf
o Department of Homeland Security, Software Assurance Working Group - Software Quality Metrics to Identify Risk o http://www.mccabe.com/ppt/SoftwareQualityMetricsToIdentifyRisk.ppt
o Software Quality Metrics for your Continuous Delivery Pipeline – Part I o http://apmblog.dynatrace.com/2014/03/13/software-quality-metrics-for-your-continuous-
delivery-pipeline-part-i/
o Secure Development LifeCycles (SDLC) – Bart De Win SecAppDev 2013 o https://handouts.secappdev.org/handouts/2013/Bart%20De%20Win/SecAppDev2013%20-
%20SDLC%20Session%20Bart%20De%20Win%20v1.0.pdf
o Secure Development Lifecycle – Eoin Keary & Jim Manico o https://www.owasp.org/images/7/76/Jim_Manico_%28Hamburg%29_-
_Securiing_the_SDLC.pdf
o Motivation for Security Testing – manu Khari & Chetna Bajaj o http://www.rroij.com/open-access/motivation-for-security-testing-26-32.php?aid=37877
Cologne, Germany Helsinki, Finland
Copenhagen, Denmark Helsinki, Finland Cologne, Germany
Prague, Czech Republic
@tsmalmbe
fi.linkedin.com/in/thomasmalmberg