© 2004 spire security, llc. all rights reserved. security i spre security measures & metrics...
TRANSCRIPT
© 2004 Spire Security, LLC. All rights reserved.
securityiSP RE
Security Measures & Metrics
Pete Lindstrom, CISSPResearch Director
Spire Security, [email protected]
© 2004 Spire Security. All rights reserved. 2
Security Metrics I
Security Metrics (Part 1): Building the Framework
There are obvious benefits to charting and quantifying the success of your security program. But where do you begin? This session -- part 1 of a 2-part mini-workshop -- outlines a practical approach to security metrics that links standard business practices with security functions. Find out from Information Security magazine contributing editor, Pete Lindstrom, Research Director for Spire Security, how to build a rock-solid foundation based on a model known as the "Four Disciplines of Security Management." Then learn about the elements of a cohesive security metrics program from a functional and resource-usage perspective. Plus, you leave with a solid understanding of the relative utility metrics for productivity, process efficiency, cost effectiveness and risk management.
© 2004 Spire Security. All rights reserved. 3
What is the Four Disciplines Model?
A way to think about securityo High-level without losing clarityo Detailed enough for technical folkso Identifies relationships
A taxonomy of objectives, functions, activities, and products.
A framework for security measurement.
© 2004 Spire Security. All rights reserved. 4
Introducing the Four Disciplines
Identity Mgt:Managing Users
and other sources
Threat Mgt:Monitoring
activities and events
Trust Mgt:Designing security policy and process
Vuln. Mgt:Hardening the systems
3
4
2
1
© 2004 Spire Security. All rights reserved. 6
Vulnerability Mgt Functions
Evaluate and harden configurationso By platform
Identify and remediate vulnerabilitieso Software bugs
Configure firewalls / other access control
Reduce/filter anomalous traffic
© 2004 Spire Security. All rights reserved. 8
Identity Management Functions
Validate user information
Create/modify user accounts and privileges
Disable/delete user accounts
Change/reset passwords
Validate sessions
Authorize access
© 2004 Spire Security. All rights reserved. 10
Trust Management Functions
Create/modify user policies
Create/modify system policies - technical baselines
Design security architecture
Design/implement controls to prevent sniffing or copying data.
Design/implement controls to prevent modifying data.
© 2004 Spire Security. All rights reserved. 12
Threat Management Functions
Identify anomalous activitieso Monitor network and componentso Aggregate alerts and logso Collect physical information
Manage/resolve incidents
Incident response - take corrective action
Conduct forensic analysis of systems/data
© 2004 Spire Security. All rights reserved. 14
Q1: Most Important?
Which Discipline is most important to a strong security program?
1. Vulnerability Management(firewalls, vuln assess, patch)
2. Identity Management(provision, acct mgt, authent.)
3. Trust Management(policies, tech guides, crypto)
4. Threat Management(monitor, incident, forensics)
© 2004 Spire Security. All rights reserved. 15
Q2: Most Time?
Which Discipline does your organization spend the most time on?
1. Vulnerability Management(firewalls, vuln assess, patch)
2. Identity Management(provision, acct mgt, authent.)
3. Trust Management(policies, tech guides, crypto)
4. Threat Management(monitor, incident, forensics)
© 2004 Spire Security. All rights reserved. 16
Fundamental Security Elements
People:DepartmentsAdmins
Costs:Salaries, ConsultingHW, SW, Maint.
Time:Hr/Day
Month/Yr
Resources:User accts,
systems, apps
Activities:Four
Disciplines
© 2004 Spire Security. All rights reserved. 17
Types of Metrics
Process Effectiveness – doing things right. (measure quality)
Staff Productivity – people doing more things. (measure volume)
Cycle Time – transaction time. (measure process efficiency)
Staff Efficiency – people doing things faster. (people / transaction / time)
Cost Effectiveness – transaction costs. (cost / activity)
© 2004 Spire Security. All rights reserved. 18
Process Effectiveness Metrics
“doing things right”Key Elements:• Activities• errors
Examples:• Acct request
errors• Remediation
errors• False alarm rate• Policy
exceptions
error rates
© 2004 Spire Security. All rights reserved. 19
Process Effectiveness
Measure quality by identifying error rates of activities
Identity Managemento User account request errors
Vulnerability Managemento Vulnerabilities not remediated
Threat Managemento Improper incident management
Trust Managemento Policy violations
© 2004 Spire Security. All rights reserved. 20
Staff Productivity Metrics
“people doing more things”
Elements:• People• Activities
Examples:• Accts per
person• Vulns per
person• Patches per
person
© 2004 Spire Security. All rights reserved. 21
Staff Productivity
Productivity and workload for all manual activities (activities/people)
Identity Managemento Requests per administratoro Account disablements per admino Password resets per admin
Vulnerability Managemento Vulnerabilities resolved per administrator
Threat Managemento Incidents per person
Trust Managemento Policy changes per person
© 2004 Spire Security. All rights reserved. 22
Cycle Time Metrics
avg “time to perform activity x”
Elements:• Time• Activities
Examples:• Accts per
month• Vulns fixed per
month• Patches per
month
© 2004 Spire Security. All rights reserved. 23
Cycle Time
Process efficiency
Identity Managemento User account request time to
complete
Vulnerability Managemento Remediation time to complete
Threat Managemento Incident response time to complete
Trust Managemento Policy creation time to complete
© 2004 Spire Security. All rights reserved. 24
Staff Efficiency Metrics
Admins by Department
2000 Hours per FTE
“people doing things” quickerElements:• People• Activities• Time
Examples:• Accts per
person/hr• Vulns per
person/hr• Patches per
person/hr
© 2004 Spire Security. All rights reserved. 25
Staff Efficiency
Combines staff productivity and cycle time metrics.
Identity Managemento User account requests completed per
person per day/week/month
Vulnerability Managemento Vulnerabilities remediated per person per
day/week/month
Threat Managemento Incidents closed per person per
day/week/month
Trust Managemento Policies reviewed per person per
day/week/month
© 2004 Spire Security. All rights reserved. 26
Cost Effectiveness Metrics
Cheaper transactions
Elements:• Activities• Costs
Examples:• Cost per
acct• Cost per
vuln fixed• Cost per
patch
© 2004 Spire Security. All rights reserved. 27
Cost Effectiveness
Dollars/activities; dollars/resources; dollars/demographics
Identity Managemento Cost per requesto Cost per password reset
Vulnerability Managemento Cost per vulnerabilityo Cost per system setting
Threat Managemento Cost per incident
Trust Managemento Cost per policyo Cost per project
© 2004 Spire Security. All rights reserved. 28
When to Use Metrics
Process Effectivenesso Six Sigma
Staff Productivityo ROI / promotions
Cycle Timeo Balanced Scorecard
Staff Efficiencyo ROI
Cost Effectivenesso Activity-based costingo ROI/TCO
© 2004 Spire Security. All rights reserved. 29
Q3: Most Useful?
Which metric type is most useful to your security program?
1. Process Effectiveness
2. Staff Productivity
3. Cycle Time
4. Staff Efficiency
5. Cost Effectiveness
© 2004 Spire Security. All rights reserved. 30
Conclusions
Security functions are spread throughout organizations.
You can’t improve security until you measure it.
Ultimately, security is a business operation that should be run like a business operation.
© 2004 Spire Security, LLC. All rights reserved.
securityiSP RE
Pete [email protected]
Agree? Disagree?