developing metrics for information security governance

34
Developing Metrics for Information Security Governance Presented By: John P. Pironti, CGEIT, CISA, CISM, CISSP, ISSAP, ISSMP Chief Information Risk Strategist Getronics Interop Las Vegas April 30, 2008

Upload: digitallibrary

Post on 22-Nov-2014

3.799 views

Category:

Business


0 download

DESCRIPTION

Information security has become a critical issue within organizations, and a key success factor for businesses. To effectively maintain the integrity and security of an organization's information infrastructure effective security metrics and measures must be developed, implemented and monitored. Learn about enterprise security metrics and the concepts that must be considered when developing, implementing, and monitoring them. Understand how to identify measurable points and activities, develop meaningful metrics and measures and monitor concepts. Case studies and scenarios demonstrate operational scenarios for the benefits and challenges of securing information.

TRANSCRIPT

Page 1: Developing Metrics for Information Security Governance

Developing Metrics for

Information Security Governance

Presented By:

John P. Pironti, CGEIT, CISA, CISM, CISSP, ISSAP, ISSMP

Chief Information Risk Strategist

Getronics

Interop Las Vegas

April 30, 2008

Page 2: Developing Metrics for Information Security Governance

Agenda• Why Measure

• Key Performance Indicators

• Defining the Measurement

• Metrics Frameworks

• Metrics Libraries– Organizational and Professional Metrics

– Operational Metrics

– Technological Metrics

– Business Process Metrics

– Business Metrics

– Compliance Metrics

• Reporting

• Final Thoughts

Page 3: Developing Metrics for Information Security Governance

Why Measure?• Key to Effective Governance

– Meaningful understanding of business

process effectiveness

– Ability to mature processes for constant

improvement

– Early warning radar for threats and

vulnerabilities

• Beneficial Reporting to Management

and Business

• Business aligned knowledge

• Business and Security Intelligence

• Trend analysis

• Anomaly detection

• Threat intelligence

3

Page 4: Developing Metrics for Information Security Governance

Subjective Vs. Objective

Measures• Subjective Measures

– Both powerful and harmful

– High Risk

• Dependant on knowledge and capability of

measuring party

– Hard to substantiate

– Human intuition can be the best and worst

indicator

• Objective Measures

– Low risk

– Supported by data

– Can be recreated

Page 5: Developing Metrics for Information Security Governance

Key Performance Indicators

• Business Aligned

Quantitative and Qualitative

Measures

– Success or failure of

processes, personal,

technology, organizational

effectiveness

• Enable Continuous

Improvement

• Facilitate Effective

Governance

Page 6: Developing Metrics for Information Security Governance

Defining the Measurement• What is being

measured?

• What is the business

value of the

measurement?

• Thresholds Should Be

Established

– Positive and negative

boundaries

– Realistic goals

– Range of values

Page 7: Developing Metrics for Information Security Governance

Gathering The Data

• Electronic Methods

– System Logs

– Automated System Monitoring

– Sensor Networks

• Non-Electronic Methods

– Statistical Tracking

– Human Feedback

– Business Process Monitoring

– Business Reporting

Page 8: Developing Metrics for Information Security Governance

Know The Audience• Who is being

communicated to?

– Business audience

– Technical audience

– External audience

• Cultural Considerations

– Localized content

– Generic language

• Benefits to the business

– Why does the audience

care about the information?

Page 9: Developing Metrics for Information Security Governance

Keep It Simple and Consistent

• Keep it simple

– Concise and direct

measures are most

effective

– Complicated metrics only

confuse audience

• Consistency essential for

credibility

– Collect and report in a

consistent fashion

– Provide integrity of

information for external

reviewers

Page 10: Developing Metrics for Information Security Governance

Business Goal Alignment

• Business leadership and

goals define measurement

requirements

– Measures only effective if

beneficial to the business

• Map business process to

define metrics

– Visualization can assist in

effective metric definition

• Understand motivation

– How will the metric help the

business to be successful?

Page 11: Developing Metrics for Information Security Governance

Baseline Framework of Metrics

• Baseline framework of

metrics

– Foundation of consistent

metrics

– Includes people, processes,

procedures, technology and

compliance

• Value Provided Vs. Cost

– Monetary impact

– Cost of labor

– Addition of complexity

– Impact on user experience

Page 12: Developing Metrics for Information Security Governance

Baseline Metrics Framework

Information Security Governance

Baseline Metrics Framework

People TechnologyProceduresProcess Compliance

Page 13: Developing Metrics for Information Security Governance
Page 14: Developing Metrics for Information Security Governance

Common Information

Security Metrics

• Number of information

security related events

• Number of information

security incidents

• Number of media

mentions and media

types

• Cost of incident

investigation and

remediation

• Cost of controls

• Elapsed time from

incident identification to

remediation

• Number of attacks

identified

• Number of policy

exceptions requested

• Number of policy

exceptions granted

• Effectiveness of

Controls

Page 15: Developing Metrics for Information Security Governance

Organizational and Performance

Metrics• Measure Organizational

Functions Ability to

Meet Business Goals

• Measure Effectiveness

– Provide information

infrastructure protection

– Enable business to run

efficiently and efficiently

• Assignment of goals

– Goals define success or

failure metrics

Page 16: Developing Metrics for Information Security Governance

Operational Metrics• Evaluate effectiveness and

capabilities of controls

– All key controls need to be

measureable

• Impact of controls on business

activities

• Measure ability of organization

to introduce countermeasures

and controls

– How long and how difficult

• Responsiveness to evolving

threats and identified

vulnerabilities

Page 17: Developing Metrics for Information Security Governance

Technological Metrics• Provide insight into

effectiveness of technological

controls

– Measure capabilities against

current attack trends

– Input from Threat and

Vulnerability Analysis used to

evaluate measure

• Example

– Number of spam e-mail messages

anti-spam capabilities able to

block

– Number of virus/malware

infections detected and prevented

Page 18: Developing Metrics for Information Security Governance

Business Process Metrics

• Impact of information security

activities on business activities

– Security as enabler or disabler to

business process

• Utilize business process flow to

identify measurable points

• Data points should be represented

in dollar values

– Savings vs. cost

– Return on investment

• Example:

– actual material and operational

costs for introducing and operating

strong authentication capabilities

Page 19: Developing Metrics for Information Security Governance

Business Metrics• Represent direct impact of

information security on business

– Shareholder Value

– Market Perception

– Profit and Loss

• Interaction with leadership team

essential for metric development

– What is important to them?

• Example:– Number of negative public media

items associated with security

incidents

Page 20: Developing Metrics for Information Security Governance

Compliance Metrics

• Measurement of controls

required by policy or regulation

• Identification of information

required to prove compliance

– External and internal review

• Trending to prove

improvements internal and

external auditors

• Example

– Time elapsed for removal of

credentials after termination

Page 21: Developing Metrics for Information Security Governance

Meaningful Reporting• Reporting of metrics are important as metrics

themselves

• Tiered reporting for different audience types

– Top Tier: Senior Leadership

– Middle Tier: Business Process Owners and

Managers

– Lower Tier: Operations and Stakeholders

• Trend Analysis

– Metrics over time periods

– Historical values and cycles

• Security Knowledge Management

– Central repository of data

– Customized reporting

Page 22: Developing Metrics for Information Security Governance

Reporting Formats and

Techniques• Visual Cues Most Effective

– Pictures and colors are widely

accepted and understood across

cultures

– Human eye most sensitive to

color green

– Red represents danger or

attention required

• Number oriented reports

useful for quantitative analysis

systems

– Event Correlation Tools

– Risk Management Tools

Page 23: Developing Metrics for Information Security Governance

Top Tier: Senior Leadership

• Risk oriented information and

business impact statements

• Costs and benefits

associated with information

security

• Market perceptions and

customer impacts

• Regulatory Impacts

– Can they go to jail?

Page 24: Developing Metrics for Information Security Governance

Middle Tier: Business Process

Owners• Effectiveness of controls

• Impact of controls on

business process capabilities

and efficiency

• Threats and Vulnerabilities

• Regulatory and Compliance

Requirement Achievement

• Policy exceptions and audit

findings

Page 25: Developing Metrics for Information Security Governance

Lower Tier: Operational

Metrics

• Operational status of controls

– Are they working as designed?

• Functional Boundaries

– Acceptable limits for controls

– Tolerance for failure

• Maturity of controls

– Should they be improved?

– How can they be better?

– Do they still make sense?

Page 26: Developing Metrics for Information Security Governance

Frequency of Reports

• Top Tier - Senior Leadership

– Quarterly

– Annually

• Aggregation of Quarterly Data

• Middle Tier – Business Process

Owners

– Monthly

• Lower Tier – Operational

– Daily or as activities occur

Page 27: Developing Metrics for Information Security Governance

Trend Analysis• Mapping metrics over periods of time

essential to success

– History is bound to repeat itself

– Early warning radar

• Establish boundaries and goals for

success and failure

– Positive and negative business impacts

– Ensure appropriate level of effort is being

applied

• Identification of trends

– Intelligence for threat and vulnerability

management capabilities

– Identification of new data points for

investigation and remediation

Page 28: Developing Metrics for Information Security Governance

Threat Trend Analysis – OSI+

ModelPolicy, Process,

and Procedure

Application

Presentation

Session

Transport

Network

Data

Physical

People

Page 29: Developing Metrics for Information Security Governance

Trend Analysis: Example

Business Impacting Information Security Events

0

5

10

15

20

25

Janu

ary

Febr

uary

Mar

ch

Apr

il

May

June

July

Aug

ust

Sep

tem

ber

Oct

ober

Nov

embe

rD

ecem

ber

Month

Nu

mb

er

of

Ev

en

ts

2005

2006

Page 30: Developing Metrics for Information Security Governance

Benchmark Reporting• Performance of controls and

capabilities compared to others

• Industry standards typically easiest

to benchmark against

– CoBit

– ISO 27001

– PCI

• Typically survey driven

– Actual testing provides more accuracy

• Spider maps best way to represent

data

– Immediately recognizable information

Page 31: Developing Metrics for Information Security Governance

Benchmark Report ExampleInformation Security Program

Functional Inventory Capability Maturity Model

0

1

2

3

4

5

CISO

Threat/Vulnerability Assessment

Vulnerability Management/Incident Response

Legal/Regulatory

Strategy

Policies, Procedures, Principals, and Standards

Business Continuance/Disaster RecoveryEducation/Communications

Program Governance

Architecture/Design

Technology Capability/Evaluation

Key Performance Analysis/Effectiveness

Information Security Oversight Board

Organization

Industry Benchmark

Page 32: Developing Metrics for Information Security Governance

Security Knowledge

Management

• Central repository for metric data

• Customized reporting capabilities

• Custom analysis for specific needs and

purposes

– Audit, Compliance, Business Marketing

• Automated report generation and alerting

• Examples include

– Archer Technologies

– Control Path

– RSAM

Page 33: Developing Metrics for Information Security Governance

Final Thoughts• Information Security is a Moving Target

– Threats, vulnerabilities, and requirements constantly

change

• Effective Metrics Bring Organizational Clarity

– What threats are realistic and business impacting

– What vulnerabilities exist in information

infrastructure

• Information security Governance Driven By

Information

– Metrics provide knowledge

• Reporting must make sense to the

audience– Information is useless if it cannot be understood,

analyzed, and utilized

Page 34: Developing Metrics for Information Security Governance

Thank You For Your Time!

John P. Pironti, CGEIT, CISA,

CISM, CISSP, ISSAP, ISSMP

Chief Information Risk Strategist

Getronics

01-978-625-6540

[email protected]