quick intro:

1 ISACA 2007, Jeffrey Blackmon

ISACA December 13th 2007

Auditing the Disaster Recovery Plan

What should be in a plan, and what should not

By:Jeffrey Blackmon CBCP, CISSP

2 ISACA 2007, Jeffrey Blackmon

Quick Intro: Jeff Blackmon, CBCP, CISSP

Started BC/DR planning in mid 80s Financial Petroleum Foreign Military Pharmaceutical

L3 Communications, Titan Group Support of Federal Government Contracts

(Kansas City and DC)

3 ISACA 2007, Jeffrey Blackmon

Format:

A little free format style

Open Discussion

Ask Questions

4 ISACA 2007, Jeffrey Blackmon

This may be somewhat a little different from the regular presentations

Usually have auditors speaking to auditors

Usually have computer people speaking to computer people

But not in this case

5 ISACA 2007, Jeffrey Blackmon

Computer person / business person speaking to the auditors

So expect a little different perspective

6 ISACA 2007, Jeffrey Blackmon

Computer Staff

7 ISACA 2007, Jeffrey Blackmon

The Auditors

8 ISACA 2007, Jeffrey Blackmon

Reason for some of the past relationships between Auditors and the Computer people

9 ISACA 2007, Jeffrey Blackmon

Why is BC and DR so difficult?

May not be well defined Big project Expensive Very difficult to take that 1st step

10 ISACA 2007, Jeffrey Blackmon

Topics

1. Goals and Reasons for doing Business Continuity and Disaster Recovery

2. What are BC and DR3. RTO/RPO 4. Good DR Plans5. Not so Good DR Plans6. Closing information

11 ISACA 2007, Jeffrey Blackmon

Goals and Reasons for BC and DR

12 ISACA 2007, Jeffrey Blackmon

Principle Goals

Provide for the safety of all employees

Minimize business downtime

13 ISACA 2007, Jeffrey Blackmon

Reasons for Doing BC and DR

Business Best Practices

FEMA Best Practices

Audit Requirements

14 ISACA 2007, Jeffrey Blackmon

Reasons for Doing BC and DR

Private Sector FSLIC √ HIPAA OCC √ GLBA Sarbanes Oxley √ NASD 3510

Government Sector FPC 65 √ NIST 800-34 A-123 Audit

15 ISACA 2007, Jeffrey Blackmon

Financial Reasons

Company Loss of $84,000 to $90,000 per hour of downtime

90% of companies that experience 1 week of data center down time go out of business within 12 months

(CIO INSIGHT, IDC)

16 ISACA 2007, Jeffrey Blackmon

More Financial Reasons‘The cost of being unprepared’

By Jim EllisEnergy $2,817,846Telecom $2,066,245Manufacturing $1,610,654Finance/Brokerage $1,495,134IT $1,344,461Insurance $1,202,444Retail $1,107,274Pharmaceuticals $1,082,252Banking $996,802Food processing $804,192Consumer $785,719Chemicals $704,101Average / hour $1,010,536

17 ISACA 2007, Jeffrey Blackmon

Costs(R. Witty, DRJ Fall 2006)

18 ISACA 2007, Jeffrey Blackmon

High Startup Costs

19 ISACA 2007, Jeffrey Blackmon

What are BC and DR?

20 ISACA 2007, Jeffrey Blackmon

21 ISACA 2007, Jeffrey Blackmon

DR Plan, what is it? IT Related

Major disruption has occurred that is not part of day to day SOP

Hardware / Software requirements Step by step directions for full

system recovery Very detailed documents required

22 ISACA 2007, Jeffrey Blackmon

DR Plan #1 Easy to use

Recovery of all major Computer systems based on Pre- determined priority (RTO)

Details, details, details

(Hardware, software, configurations, communications, disk storage, SAN connections……. )

23 ISACA 2007, Jeffrey Blackmon

BC Plan

#1 Easy to use

Recovery of all major business processes

People related Probably many manual processes

to be used for the short term

24 ISACA 2007, Jeffrey Blackmon

25 ISACA 2007, Jeffrey Blackmon

Plain and Simple

BC/DR are Risk Mitigation

No way to eliminate all risks

Proper planning will reduce the risks to an acceptable level

26 ISACA 2007, Jeffrey Blackmon

RTO and RPO

27 ISACA 2007, Jeffrey Blackmon

Recovery Time Objective (RTO)

The max allowable time that a business system, application or resource is allowed to be down or offline

RTO is determined by business owners, not IT department

28 ISACA 2007, Jeffrey Blackmon

Recovery Point Objective (RPO)

The amount of data that is acceptable to lose since the last successful backup was completed

RPO is determined by business owners, not IT department

29 ISACA 2007, Jeffrey Blackmon

Recovery Point Objective Recovery Time Objective

BackupTape Made

BackupTape Made

MidnightMondayNoon

MidnightTuesday

MidnightWednesday

NoonNoon

BackupTape Made

DISASTER

RPO (12 hours)

RTO (24 hours)Standard TapeBackup Recovery

30 ISACA 2007, Jeffrey Blackmon

Recovery Point Objective Recovery Time Objective

BackupTape Made

BackupTape Made

MidnightMondayNoon

MidnightTuesday

MidnightWednesday

NoonNoon

BackupTape Made

DISASTER

RPO (2 minutes)

RTO (12 hours, rebuild system)Replicated DataBackup Recovery

$$$ $

Real time replication

31 ISACA 2007, Jeffrey Blackmon

Find the Cost Effective Solution

Cost Effective Solution

Time

Costs

Business Interuption Cost Recovery Costs

32 ISACA 2007, Jeffrey Blackmon

RPO / RTO Example Major financial institutions on mission

critical systems RPO = 0 hours, on some applications RTO = 2 hours, on some applications

After 96 Hours, major financial institutions will probably not recover

By Jay Ranade, CISSP, CISA, CBCP, CISMPresident, Jay Ranade Consultants, Inc.

33 ISACA 2007, Jeffrey Blackmon

RPO / RTO Example Major breakfast cereal producer

RPO = 7 days RTO = 7 days

Put it all into perspective Very regular shipments to distributors by

boxcar Only breakfast cereal, if problems occur, then

re-ship

By DRII Classmate, 1999

34 ISACA 2007, Jeffrey Blackmon

RPO / RTO Expectations

‘Usually’ a large gap in management expectations as compared to actual recovery abilities

Talk with technical staff

35 ISACA 2007, Jeffrey Blackmon

What a plan should look like

36 ISACA 2007, Jeffrey Blackmon

Good DR plans

Be sure you keep in mind that DR plans are to recover computer and network systems

37 ISACA 2007, Jeffrey Blackmon

NIST 800-53, Recommended Security Controls for Federal Information System

FAMILY: CONTINGENCY PLANNING CP-1 CONTINGENCY PLANNING

POLICY AND PROCEDURES CP-2 CONTINGENCY PLAN CP-3 CONTINGENCY TRAINING CP-4 CONTINGENCY PLAN

TESTING CP-5 CONTINGENCY PLAN

UPDATE

38 ISACA 2007, Jeffrey Blackmon

NIST 800-53, Recommended Security Controls for Federal Information System

FAMILY: CONTINGENCY PLANNING CP-6 ALTERNATE STORAGE SITES CP-7 ALTERNATE PROCESSING SITES CP-8 TELECOMMUNICATIONS

SERVICES CP-9 INFORMATION SYSTEM

BACKUP CP-10 INFORMATION SYSTEM

RECOVERY AND RECONSTITUTION

39 ISACA 2007, Jeffrey Blackmon

Good DR plans

Disaster definition Who can activate the DR plan? Critical computer applications Escalation Plans / Decision Plans

40 ISACA 2007, Jeffrey Blackmon

Good DR plans

List of Recovery Team Members and contact info

Vendor Contact Information Communications Vendor Contact

Information Hotsite contact information Offsite storage contact information

41 ISACA 2007, Jeffrey Blackmon

Good DR plans Hardware / Software recovery for

each and every critical system based on RPO/RTO

Network recovery information

Detailed configuration information

42 ISACA 2007, Jeffrey Blackmon

Good DR plans

Up to date Information on last time this DR

plan was tested (Minimum is annually)

Change Log to the plan Returning to normal operations

43 ISACA 2007, Jeffrey Blackmon

Not so Good DR Plans

44 ISACA 2007, Jeffrey Blackmon

Not so Good DR plans No Executive Sponsor Unrealistic Budget

(< 2% of Data Center total budget) Unrealistic recovery strategy Not Exercised / Tested

Testing only partial of a system No training

No Priority on recovery of systems

45 ISACA 2007, Jeffrey Blackmon

Not so Good DR plans Copied from another site with no

updates General in nature 3 inch binder Overabundance of color charts and

slides High on fluff Short on useful information

46 ISACA 2007, Jeffrey Blackmon

Not so Good DR plans PURPOSE OBJECTIVES SCOPE AUTHORITIES REFERENCES MANAGEMENT RESPONSIBILITIES ORGANIZATION OF THE PLAN DEFINITIONS CANCELLATION DISTRIBUTION OVERVIEW POLICY ASSUMPTIONS CONCEPT OF ACTIVATION DEPLOYMENT CONDITIONS

47 ISACA 2007, Jeffrey Blackmon

With Logic like this

48 ISACA 2007, Jeffrey Blackmon

They may be trying to Bamboozal you!

49 ISACA 2007, Jeffrey Blackmon

Remember Review the plan at a high level Recovery of Systems and

Communications, that is key Who needs to be contacted? Where do we go? Acquire equipment Restore Operating Systems, applications

and data Restore Communication

50 ISACA 2007, Jeffrey Blackmon

Remember

Stick to the key points and don’t get distracted by all of the rest

Do not get bogged down in the fine detail

51 ISACA 2007, Jeffrey Blackmon

Closing

52 ISACA 2007, Jeffrey Blackmon

Front end security vs back end BC/DR

BC / DR activation are last resort efforts

Risk levels go high

Spend the time, effort & money to develop a very strong front end security program to avoid a disastrous event

53 ISACA 2007, Jeffrey Blackmon

Thank You for Attending!