security metrics, part 1 -- building the framework

31
© 2004 Spire Security, LLC. All rights reserved. security i SP RE Security Measures & Metrics Pete Lindstrom, CISSP Research Director Spire Security, LLC www.spiresecurity.com [email protected]

Upload: sandra4211

Post on 26-May-2015

367 views

Category:

Documents


1 download

TRANSCRIPT

Page 1: Security Metrics, Part 1 -- Building the Framework

© 2004 Spire Security, LLC. All rights reserved.

securityiSP RE

Security Measures & Metrics

Pete Lindstrom, CISSPResearch Director

Spire Security, [email protected]

Page 2: Security Metrics, Part 1 -- Building the Framework

© 2004 Spire Security. All rights reserved. 2

Security Metrics I

Security Metrics (Part 1): Building the Framework

There are obvious benefits to charting and quantifying the success of your security program. But where do you begin? This session -- part 1 of a 2-part mini-workshop -- outlines a practical approach to security metrics that links standard business practices with security functions. Find out from Information Security magazine contributing editor, Pete Lindstrom, Research Director for Spire Security, how to build a rock-solid foundation based on a model known as the "Four Disciplines of Security Management." Then learn about the elements of a cohesive security metrics program from a functional and resource-usage perspective. Plus, you leave with a solid understanding of the relative utility metrics for productivity, process efficiency, cost effectiveness and risk management.

Page 3: Security Metrics, Part 1 -- Building the Framework

© 2004 Spire Security. All rights reserved. 3

What is the Four Disciplines Model?

A way to think about securityo High-level without losing clarityo Detailed enough for technical folkso Identifies relationships

A taxonomy of objectives, functions, activities, and products.

A framework for security measurement.

Page 4: Security Metrics, Part 1 -- Building the Framework

© 2004 Spire Security. All rights reserved. 4

Introducing the Four Disciplines

Identity Mgt:Managing Users

and other sources

Threat Mgt:Monitoring

activities and events

Trust Mgt:Designing security policy and process

Vuln. Mgt:Hardening the systems

3

4

2

1

Page 5: Security Metrics, Part 1 -- Building the Framework

© 2004 Spire Security. All rights reserved. 5

1. Harden Systems

Page 6: Security Metrics, Part 1 -- Building the Framework

© 2004 Spire Security. All rights reserved. 6

Vulnerability Mgt Functions

Evaluate and harden configurationso By platform

Identify and remediate vulnerabilitieso Software bugs

Configure firewalls / other access control

Reduce/filter anomalous traffic

Page 7: Security Metrics, Part 1 -- Building the Framework

© 2004 Spire Security. All rights reserved. 7

2. Identify/Manage Users

Page 8: Security Metrics, Part 1 -- Building the Framework

© 2004 Spire Security. All rights reserved. 8

 Identity Management Functions

Validate user information

Create/modify user accounts and privileges

Disable/delete user accounts

Change/reset passwords

Validate sessions

Authorize access

Page 9: Security Metrics, Part 1 -- Building the Framework

© 2004 Spire Security. All rights reserved. 9

3. Design/Strengthen Processes

Page 10: Security Metrics, Part 1 -- Building the Framework

© 2004 Spire Security. All rights reserved. 10

Trust Management Functions

Create/modify user policies

Create/modify system policies - technical baselines

Design security architecture

Design/implement controls to prevent sniffing or copying data.

Design/implement controls to prevent modifying data.

Page 11: Security Metrics, Part 1 -- Building the Framework

© 2004 Spire Security. All rights reserved. 11

4. Monitor Environment

Page 12: Security Metrics, Part 1 -- Building the Framework

© 2004 Spire Security. All rights reserved. 12

Threat Management Functions

Identify anomalous activitieso Monitor network and componentso Aggregate alerts and logso Collect physical information

Manage/resolve incidents

Incident response - take corrective action

Conduct forensic analysis of systems/data

Page 13: Security Metrics, Part 1 -- Building the Framework

© 2004 Spire Security. All rights reserved. 13

Putting It All Together

Page 14: Security Metrics, Part 1 -- Building the Framework

© 2004 Spire Security. All rights reserved. 14

Q1: Most Important?

Which Discipline is most important to a strong security program?

1. Vulnerability Management(firewalls, vuln assess, patch)

2. Identity Management(provision, acct mgt, authent.)

3. Trust Management(policies, tech guides, crypto)

4. Threat Management(monitor, incident, forensics)

Page 15: Security Metrics, Part 1 -- Building the Framework

© 2004 Spire Security. All rights reserved. 15

Q2: Most Time?

Which Discipline does your organization spend the most time on?

1. Vulnerability Management(firewalls, vuln assess, patch)

2. Identity Management(provision, acct mgt, authent.)

3. Trust Management(policies, tech guides, crypto)

4. Threat Management(monitor, incident, forensics)

Page 16: Security Metrics, Part 1 -- Building the Framework

© 2004 Spire Security. All rights reserved. 16

Fundamental Security Elements

People:DepartmentsAdmins

Costs:Salaries, ConsultingHW, SW, Maint.

Time:Hr/Day

Month/Yr

Resources:User accts,

systems, apps

Activities:Four

Disciplines

Page 17: Security Metrics, Part 1 -- Building the Framework

© 2004 Spire Security. All rights reserved. 17

Types of Metrics

Process Effectiveness – doing things right. (measure quality)

Staff Productivity – people doing more things. (measure volume)

Cycle Time – transaction time. (measure process efficiency)

Staff Efficiency – people doing things faster. (people / transaction / time)

Cost Effectiveness – transaction costs. (cost / activity)

Page 18: Security Metrics, Part 1 -- Building the Framework

© 2004 Spire Security. All rights reserved. 18

Process Effectiveness Metrics

“doing things right”Key Elements:• Activities• errors

Examples:• Acct request

errors• Remediation

errors• False alarm rate• Policy

exceptions

error rates

Page 19: Security Metrics, Part 1 -- Building the Framework

© 2004 Spire Security. All rights reserved. 19

Process Effectiveness

Measure quality by identifying error rates of activities

Identity Managemento User account request errors

Vulnerability Managemento Vulnerabilities not remediated

Threat Managemento Improper incident management

Trust Managemento Policy violations

Page 20: Security Metrics, Part 1 -- Building the Framework

© 2004 Spire Security. All rights reserved. 20

Staff Productivity Metrics

“people doing more things”

Elements:• People• Activities

Examples:• Accts per

person• Vulns per

person• Patches per

person

Page 21: Security Metrics, Part 1 -- Building the Framework

© 2004 Spire Security. All rights reserved. 21

Staff Productivity

Productivity and workload for all manual activities (activities/people)

Identity Managemento Requests per administratoro Account disablements per admino Password resets per admin

Vulnerability Managemento Vulnerabilities resolved per administrator

Threat Managemento Incidents per person

Trust Managemento Policy changes per person

Page 22: Security Metrics, Part 1 -- Building the Framework

© 2004 Spire Security. All rights reserved. 22

Cycle Time Metrics

avg “time to perform activity x”

Elements:• Time• Activities

Examples:• Accts per

month• Vulns fixed per

month• Patches per

month

Page 23: Security Metrics, Part 1 -- Building the Framework

© 2004 Spire Security. All rights reserved. 23

Cycle Time

Process efficiency

Identity Managemento User account request time to

complete

Vulnerability Managemento Remediation time to complete

Threat Managemento Incident response time to complete

Trust Managemento Policy creation time to complete

Page 24: Security Metrics, Part 1 -- Building the Framework

© 2004 Spire Security. All rights reserved. 24

Staff Efficiency Metrics

Admins by Department

2000 Hours per FTE

“people doing things” quickerElements:• People• Activities• Time

Examples:• Accts per

person/hr• Vulns per

person/hr• Patches per

person/hr

Page 25: Security Metrics, Part 1 -- Building the Framework

© 2004 Spire Security. All rights reserved. 25

Staff Efficiency

Combines staff productivity and cycle time metrics.

Identity Managemento User account requests completed per

person per day/week/month

Vulnerability Managemento Vulnerabilities remediated per person per

day/week/month

Threat Managemento Incidents closed per person per

day/week/month

Trust Managemento Policies reviewed per person per

day/week/month

Page 26: Security Metrics, Part 1 -- Building the Framework

© 2004 Spire Security. All rights reserved. 26

Cost Effectiveness Metrics

Cheaper transactions

Elements:• Activities• Costs

Examples:• Cost per

acct• Cost per

vuln fixed• Cost per

patch

Page 27: Security Metrics, Part 1 -- Building the Framework

© 2004 Spire Security. All rights reserved. 27

Cost Effectiveness

Dollars/activities; dollars/resources; dollars/demographics

Identity Managemento Cost per requesto Cost per password reset

Vulnerability Managemento Cost per vulnerabilityo Cost per system setting

Threat Managemento Cost per incident

Trust Managemento Cost per policyo Cost per project

Page 28: Security Metrics, Part 1 -- Building the Framework

© 2004 Spire Security. All rights reserved. 28

When to Use Metrics

Process Effectivenesso Six Sigma

Staff Productivityo ROI / promotions

Cycle Timeo Balanced Scorecard

Staff Efficiencyo ROI

Cost Effectivenesso Activity-based costingo ROI/TCO

Page 29: Security Metrics, Part 1 -- Building the Framework

© 2004 Spire Security. All rights reserved. 29

Q3: Most Useful?

Which metric type is most useful to your security program?

1. Process Effectiveness

2. Staff Productivity

3. Cycle Time

4. Staff Efficiency

5. Cost Effectiveness

Page 30: Security Metrics, Part 1 -- Building the Framework

© 2004 Spire Security. All rights reserved. 30

Conclusions

Security functions are spread throughout organizations.

You can’t improve security until you measure it.

Ultimately, security is a business operation that should be run like a business operation.

Page 31: Security Metrics, Part 1 -- Building the Framework

© 2004 Spire Security, LLC. All rights reserved.

securityiSP RE

Pete [email protected]

Agree? Disagree?