co-location sites for business continuity and disaster recovery peter lesser [email protected] (212)...

Co-location SitesCo-location Sitesforfor

Business Continuity and Business Continuity and Disaster RecoveryDisaster Recovery

Peter LesserPeter [email protected]@kkl.com(212) 692-5617(212) 692-5617

Peter LesserPeter [email protected]@kkl.com(212) 692-5617(212) 692-5617

Kraft Kennedy & Lesser, Inc.Kraft Kennedy & Lesser, Inc.360 Lexington Avenue360 Lexington Avenue

New York, NY 10017 New York, NY 10017

Kraft Kennedy & Lesser, Inc.Kraft Kennedy & Lesser, Inc.360 Lexington Avenue360 Lexington Avenue

New York, NY 10017 New York, NY 10017

K raft Kennedy Lesser &

Inc.

AgendaAgendaAgendaAgenda Business ContinuityBusiness Continuity

ElementsElements TechnologiesTechnologies

Case Study - Skadden’s ApproachCase Study - Skadden’s Approach ConceptConcept AdoptionAdoption DesignDesign Implementation StatusImplementation Status August 14 – 17, 2003August 14 – 17, 2003

Top 10 Lessons LearnedTop 10 Lessons Learned Q & AQ & A

Business ContinuityBusiness Continuity ElementsElements TechnologiesTechnologies

Case Study - Skadden’s ApproachCase Study - Skadden’s Approach ConceptConcept AdoptionAdoption DesignDesign Implementation StatusImplementation Status August 14 – 17, 2003August 14 – 17, 2003

Top 10 Lessons LearnedTop 10 Lessons Learned Q & AQ & A


Inc.

Business Continuity Business Continuity ElementsElements


High AvailabilityHigh Availability Fault ToleranceFault Tolerance Disaster RecoveryDisaster Recovery

High AvailabilityHigh Availability Fault ToleranceFault Tolerance Disaster RecoveryDisaster Recovery


Inc.



Data AvailabilityData Availability Systems AvailabilitySystems Availability Ubiquitous AccessUbiquitous Access Business Continuity PlanBusiness Continuity Plan

Data AvailabilityData Availability Systems AvailabilitySystems Availability Ubiquitous AccessUbiquitous Access Business Continuity PlanBusiness Continuity Plan


Inc.

Business Continuity Business Continuity TechnologiesTechnologies

Business Continuity Business Continuity TechnologiesTechnologies

Basic Hardware - RedundancyBasic Hardware - Redundancy Environmental - UPS & A/CEnvironmental - UPS & A/C Network - HSRPNetwork - HSRP Storage - SANsStorage - SANs Servers - Server ClusteringServers - Server Clustering Data - ReplicationData - Replication Clients - Thin and Thick/CachedClients - Thin and Thick/Cached Printers - IP PrintingPrinters - IP Printing Data Centers – Co-location and DR/BC Hot-SiteData Centers – Co-location and DR/BC Hot-Site

Basic Hardware - RedundancyBasic Hardware - Redundancy Environmental - UPS & A/CEnvironmental - UPS & A/C Network - HSRPNetwork - HSRP Storage - SANsStorage - SANs Servers - Server ClusteringServers - Server Clustering Data - ReplicationData - Replication Clients - Thin and Thick/CachedClients - Thin and Thick/Cached Printers - IP PrintingPrinters - IP Printing Data Centers – Co-location and DR/BC Hot-SiteData Centers – Co-location and DR/BC Hot-Site

Skadden ArpsSkadden ArpsConceptConcept


Inc.

MissionMissionMissionMission

To create systems redundancies which aim To create systems redundancies which aim to eliminate interruptions in service to eliminate interruptions in service attributable to:attributable to:

Data center component failureData center component failure Data center facility failureData center facility failure Communications link failureCommunications link failure Building becomes temporarily or Building becomes temporarily or

permanently inaccessiblepermanently inaccessible

To create systems redundancies which aim To create systems redundancies which aim to eliminate interruptions in service to eliminate interruptions in service attributable to:attributable to:

Data center component failureData center component failure Data center facility failureData center facility failure Communications link failureCommunications link failure Building becomes temporarily or Building becomes temporarily or

permanently inaccessiblepermanently inaccessible


Inc.

AssumptionsAssumptionsAssumptionsAssumptionsThe plan directs investments toward protecting firm The plan directs investments toward protecting firm applications deemed most critical.applications deemed most critical.

Critical applications are defined as those which pose the most Critical applications are defined as those which pose the most risk to firm operations should they become unavailable for any risk to firm operations should they become unavailable for any period of time. They are: period of time. They are:

MessagingMessaging Document Management and Document ProcessingDocument Management and Document Processing Time EntryTime Entry Litigation SystemsLitigation Systems Contact ManagementContact Management ConflictsConflicts

The remaining less critical applications will be restored as The remaining less critical applications will be restored as needed if an outage is determined to be long-term.needed if an outage is determined to be long-term.

The plan directs investments toward protecting firm The plan directs investments toward protecting firm applications deemed most critical.applications deemed most critical.

Critical applications are defined as those which pose the most Critical applications are defined as those which pose the most risk to firm operations should they become unavailable for any risk to firm operations should they become unavailable for any period of time. They are: period of time. They are:

MessagingMessaging Document Management and Document ProcessingDocument Management and Document Processing Time EntryTime Entry Litigation SystemsLitigation Systems Contact ManagementContact Management ConflictsConflicts

The remaining less critical applications will be restored as The remaining less critical applications will be restored as needed if an outage is determined to be long-term.needed if an outage is determined to be long-term.


Inc.

Capabilities as of 2000/1Capabilities as of 2000/1Capabilities as of 2000/1Capabilities as of 2000/1Standard top tier law firm practices.Standard top tier law firm practices.

The firm’s data centers are protected by:The firm’s data centers are protected by: Fire control systemsFire control systems Temperature control systemsTemperature control systems Physical security systemsPhysical security systems Uninterruptible power suppliesUninterruptible power supplies

Hardware components are protected by:Hardware components are protected by: Duplicate power suppliesDuplicate power supplies Duplexed/RAID hard drivesDuplexed/RAID hard drives Redundant Internet access communications facilitiesRedundant Internet access communications facilities

Firm data is protected by tape backup procedures and data is Firm data is protected by tape backup procedures and data is regularly moved to off site storage.regularly moved to off site storage.

No real time ability to utilize that data in case of systems failure.No real time ability to utilize that data in case of systems failure.

Standard top tier law firm practices.Standard top tier law firm practices.

The firm’s data centers are protected by:The firm’s data centers are protected by: Fire control systemsFire control systems Temperature control systemsTemperature control systems Physical security systemsPhysical security systems Uninterruptible power suppliesUninterruptible power supplies

Hardware components are protected by:Hardware components are protected by: Duplicate power suppliesDuplicate power supplies Duplexed/RAID hard drivesDuplexed/RAID hard drives Redundant Internet access communications facilitiesRedundant Internet access communications facilities

Firm data is protected by tape backup procedures and data is Firm data is protected by tape backup procedures and data is regularly moved to off site storage.regularly moved to off site storage.

No real time ability to utilize that data in case of systems failure.No real time ability to utilize that data in case of systems failure.


Inc.

Options for increased Options for increased capabilitiescapabilities

Options for increased Options for increased capabilitiescapabilities

Establish disaster recovery hot-site.Establish disaster recovery hot-site. Establish primary data centers serving as Establish primary data centers serving as

“hot-sites” for each other.“hot-sites” for each other. Establish redundant off-site data centers.Establish redundant off-site data centers.

Establish disaster recovery hot-site.Establish disaster recovery hot-site. Establish primary data centers serving as Establish primary data centers serving as

“hot-sites” for each other.“hot-sites” for each other. Establish redundant off-site data centers.Establish redundant off-site data centers.


Inc.

Disaster Recovery Hot-Disaster Recovery Hot-SiteSite

Disaster Recovery Hot-Disaster Recovery Hot-SiteSite

Advantages:Advantages: Protects multiple offices from disaster in any single Protects multiple offices from disaster in any single

location.location. Hot-site is easily scaled to backup addition applications Hot-site is easily scaled to backup addition applications

or locations.or locations. No reliance on communications for normal operations.No reliance on communications for normal operations.

Disadvantages:Disadvantages: Does not provide full systems recovery.Does not provide full systems recovery. Does not protect from component failure.Does not protect from component failure. Does not leverage economies of consolidation.Does not leverage economies of consolidation. Requires large distributed IT staff.Requires large distributed IT staff. Solution is disruptive to implement.Solution is disruptive to implement.

Advantages:Advantages: Protects multiple offices from disaster in any single Protects multiple offices from disaster in any single

location.location. Hot-site is easily scaled to backup addition applications Hot-site is easily scaled to backup addition applications

or locations.or locations. No reliance on communications for normal operations.No reliance on communications for normal operations.

Disadvantages:Disadvantages: Does not provide full systems recovery.Does not provide full systems recovery. Does not protect from component failure.Does not protect from component failure. Does not leverage economies of consolidation.Does not leverage economies of consolidation. Requires large distributed IT staff.Requires large distributed IT staff. Solution is disruptive to implement.Solution is disruptive to implement.


Inc.

Internal Hot-SitesInternal Hot-SitesInternal Hot-SitesInternal Hot-SitesAdvantages:Advantages:

Allows for rapid recovery.Allows for rapid recovery. Leverages existing firm locations.Leverages existing firm locations. Reduces space required for some data centers.Reduces space required for some data centers. Reduces efforts for some systems upgrades.Reduces efforts for some systems upgrades.

Disadvantages:Disadvantages:

Need to acquire significant additional space in primary locations. Need to acquire significant additional space in primary locations. Existing locations do not have ideal facilities.Existing locations do not have ideal facilities. Does not fully leverage economies of consolidation.Does not fully leverage economies of consolidation. Requires distributed high level IT staff.Requires distributed high level IT staff. Solution is most disruptive to implement.Solution is most disruptive to implement. Solution is most time consuming to complete.Solution is most time consuming to complete. Must invest in redundant communications links from small offices Must invest in redundant communications links from small offices

to consolidated data centers.to consolidated data centers.

Advantages:Advantages: Allows for rapid recovery.Allows for rapid recovery. Leverages existing firm locations.Leverages existing firm locations. Reduces space required for some data centers.Reduces space required for some data centers. Reduces efforts for some systems upgrades.Reduces efforts for some systems upgrades.

Disadvantages:Disadvantages:

Need to acquire significant additional space in primary locations. Need to acquire significant additional space in primary locations. Existing locations do not have ideal facilities.Existing locations do not have ideal facilities. Does not fully leverage economies of consolidation.Does not fully leverage economies of consolidation. Requires distributed high level IT staff.Requires distributed high level IT staff. Solution is most disruptive to implement.Solution is most disruptive to implement. Solution is most time consuming to complete.Solution is most time consuming to complete. Must invest in redundant communications links from small offices Must invest in redundant communications links from small offices

to consolidated data centers.to consolidated data centers.


Inc.

Redundant Off-Site Data Redundant Off-Site Data CentersCenters

Redundant Off-Site Data Redundant Off-Site Data CentersCenters

Advantages:Advantages: Allows for rapid recovery.Allows for rapid recovery. Reduces space required for in office data centers.Reduces space required for in office data centers. Allows data centers to be located in less expensive locations.Allows data centers to be located in less expensive locations. Allows data centers to be located in buildings with enhanced Allows data centers to be located in buildings with enhanced

facilities.facilities. Increases in DR protection are realized during implementation Increases in DR protection are realized during implementation

process.process. Allows for IT staff consolidation.Allows for IT staff consolidation. Allows for simplification of systems design.Allows for simplification of systems design. Dramatically reduces efforts for systems upgrades.Dramatically reduces efforts for systems upgrades. Dramatically reduces time to support new locations.Dramatically reduces time to support new locations. Requires smallest amount of hardware and software.Requires smallest amount of hardware and software. Solution is least disruptive to implement.Solution is least disruptive to implement.

Disadvantages:Disadvantages: Must invest in redundant communications links from all offices to Must invest in redundant communications links from all offices to

data center locations. data center locations.







Inc.

RecommendationRecommendationRecommendationRecommendation

Implement Redundant Off-Site Data CentersImplement Redundant Off-Site Data Centers

Provides most comprehensive and highest Provides most comprehensive and highest levels of protection against data center levels of protection against data center and building disasters.and building disasters.

Option best meets defined mission.Option best meets defined mission. Utilizes financial industry best practices.Utilizes financial industry best practices.

Implement Redundant Off-Site Data CentersImplement Redundant Off-Site Data Centers

Provides most comprehensive and highest Provides most comprehensive and highest levels of protection against data center levels of protection against data center and building disasters.and building disasters.

Option best meets defined mission.Option best meets defined mission. Utilizes financial industry best practices.Utilizes financial industry best practices.

Skadden ArpsSkadden ArpsAdoptionAdoption


Inc.

Current Systems Architecture

4 Times Square

Skadden Office

Skadden Office Skadden Office

Skadden OfficeSkadden Office

Skadden Office Skadden Office

Skadden Office

1460 Broadway

Server

Server

Server

Server

ServerServer

Server

Server

Server

Server Server Server Server

Server

Server

Server Server Server

Server


Inc.

Future Systems Architecture

ServerServer ServerServer ServerServer ServerServer ServerServer ServerServer ServerServer ServerServer

Skadden Office

Skadden Office

Skadden OfficeSkadden Office

Skadden Office

SECONDARY DATA CENTERPRIMARY DATA CENTER

Skadden Office

Skadden OfficeSkadden Office Skadden Office


Inc.

Strengths & Weaknesses of Strengths & Weaknesses of Current System Current System

Architecture Architecture

Strengths & Weaknesses of Strengths & Weaknesses of Current System Current System

Architecture Architecture Advantages:Advantages: DMS and messaging are decentralized. Failure in one office DMS and messaging are decentralized. Failure in one office

does not affect the rest of the Firm.does not affect the rest of the Firm. DMS and messaging are not dramatically affected by local DMS and messaging are not dramatically affected by local

communications link failure.communications link failure.

Disadvantages:Disadvantages: Minimal system redundancy.Minimal system redundancy. Vulnerable to local system and power outages.Vulnerable to local system and power outages. Firm data is protected by tape backup only.Firm data is protected by tape backup only. No real time ability to utilize data in case of system failure.No real time ability to utilize data in case of system failure. Maximum of days to weeks of downtime and 24 hours of Maximum of days to weeks of downtime and 24 hours of

data loss.data loss.

Advantages:Advantages: DMS and messaging are decentralized. Failure in one office DMS and messaging are decentralized. Failure in one office

does not affect the rest of the Firm.does not affect the rest of the Firm. DMS and messaging are not dramatically affected by local DMS and messaging are not dramatically affected by local

communications link failure.communications link failure.

Disadvantages:Disadvantages: Minimal system redundancy.Minimal system redundancy. Vulnerable to local system and power outages.Vulnerable to local system and power outages. Firm data is protected by tape backup only.Firm data is protected by tape backup only. No real time ability to utilize data in case of system failure.No real time ability to utilize data in case of system failure. Maximum of days to weeks of downtime and 24 hours of Maximum of days to weeks of downtime and 24 hours of

data loss.data loss.


Inc.

Strengths & Weaknesses of Strengths & Weaknesses of Future System Architecture Future System Architecture Strengths & Weaknesses of Strengths & Weaknesses of Future System Architecture Future System Architecture Advantages:Advantages:

Allows for rapid recovery.Allows for rapid recovery. Reduces space required for in office data centers.Reduces space required for in office data centers. Allows data centers to be located in less expensive locations.Allows data centers to be located in less expensive locations. Allows data centers to be located in buildings with enhanced Allows data centers to be located in buildings with enhanced










Skadden ArpsSkadden ArpsDesignDesign


Inc.

Availability & Disaster Availability & Disaster Recovery GoalsRecovery Goals

Availability & Disaster Availability & Disaster Recovery GoalsRecovery Goals

High Availability High Availability Requirement Requirement

ApplicatiApplicationon

Disaster Recovery Disaster Recovery RequirementRequirement

Data LossData Loss………..………..……

Recovery Recovery time……..time……..

Disconnects……..Disconnects……..……

0 0

00

00

CriticalCritical Data LossData Loss…….…….….. Recovery ….. Recovery Time…..Time…..

1/2 hour1/2 hour

2 hours2 hours

Data LossData Loss………..………..……

Recovery Recovery Time…….Time…….

00

00Very Very

ImportanImportantt

Data LossData Loss…….…….….. Recovery ….. Recovery Time…..Time…..

2 hours2 hours

4 hours4 hours

Data LossData Loss………..... ………..... RecoveryTime……RecoveryTime…….. ..

00

24 hrs24 hrsImportanImportan

tt

Data LossData Loss…..…..……. Recovery ……. Recovery TimeTime .…..….

24 hours24 hours

72 hours72 hours


Inc.

Management GoalsManagement GoalsManagement GoalsManagement Goals

Reduce Systems Management EffortReduce Systems Management Effort Improve ConsistencyImprove Consistency Reduce Infrastructure Upgrade EffortReduce Infrastructure Upgrade Effort Reduce Time to Support New ApplicationsReduce Time to Support New Applications Reduce Time to Establish and Support New Reduce Time to Establish and Support New

LocationsLocations

Reduce Systems Management EffortReduce Systems Management Effort Improve ConsistencyImprove Consistency Reduce Infrastructure Upgrade EffortReduce Infrastructure Upgrade Effort Reduce Time to Support New ApplicationsReduce Time to Support New Applications Reduce Time to Establish and Support New Reduce Time to Establish and Support New

LocationsLocations


Inc.

Primary Data CenterPrimary Data CenterPrimary Data CenterPrimary Data Center

WTS/CitrixServer Farm200 Servers

Layer 3Switch

Layer 3Switch Fibre Channel

Storage

Fiber ChannelSwitch

Fiber ChannelSwitch

Netware 6.x cluster30 servers

Hosting Login/GroupWise/File & Print

HP 9000HP-UX cluster

15 ServersHosting Sybase Apps

Win2000 cluster55 servers

Server FarmHosting SQL apps

RDF


Inc.

Secondary Data CenterSecondary Data CenterSecondary Data CenterSecondary Data Center

WTS/CitrixServer Farm100 Servers

Layer 3Switch

Layer 3Switch

Fibre ChannelStorage

Fiber ChannelSwitch

Fiber ChannelSwitch

Netware 6.x cluster30 servers

Hosting Login/GroupWise/File & Print

HP 9000HP-UX cluster

15 ServersHosting Sybase Apps

Win2000 cluster55 servers

Server FarmHosting SQL apps

Fibre ChannelTape Library

RDF


Inc.

Communications DesignCommunications DesignCommunications DesignCommunications Design

Implementation StatusImplementation Status

Data Center I:Data Center I: December 1December 1stst, 2002, 2002Data Center II:Data Center II: February 1February 1stst, 2003, 2003Live Users:Live Users: 1500+1500+

August 14 – 17, 2003August 14 – 17, 2003 No Primary Data Center OutagesNo Primary Data Center Outages Migration of “Central Applications”Migration of “Central Applications”

Top 10Top 10Lessons LearnedLessons Learned


Inc.

#10Think Outside the Box

#10Think Outside the Box


Inc.

#9 Define your own 9’s

Separate High Availability and Disaster Recovery

#9 Define your own 9’s

Separate High Availability and Disaster Recovery


Inc.

#8Replication Comes in Many

Shapes and Sizes

#8Replication Comes in Many

Shapes and Sizes


Inc.

#7 Use Citrix MetaFrame and WTS as Your Swiss Army

Knife

#7 Use Citrix MetaFrame and WTS as Your Swiss Army

Knife


Inc.

#6Now is the Time for a

Better, Faster, Cheaper WAN

#6Now is the Time for a

Better, Faster, Cheaper WAN


Inc.

#5Leverage Your Vendors

#5Leverage Your Vendors


Inc.

#4Distributed Computing is an

IT Burden

#4Distributed Computing is an

IT Burden


Inc.

#3#3Some Data Centers Do Not Some Data Centers Do Not

Need a Park Avenue Need a Park Avenue Address, a View or Even an Address, a View or Even an

IT StaffIT Staff

#3#3Some Data Centers Do Not Some Data Centers Do Not

Need a Park Avenue Need a Park Avenue Address, a View or Even an Address, a View or Even an

IT StaffIT Staff


Inc.

#2Think Like a Lawyer

#2Think Like a Lawyer


Inc.

#1#1 You Can Never be Too You Can Never be Too Rich, Too Good Looking Rich, Too Good Looking

or Have Too Much or Have Too Much Project ManagementProject Management

#1#1 You Can Never be Too You Can Never be Too Rich, Too Good Looking Rich, Too Good Looking

or Have Too Much or Have Too Much Project ManagementProject Management

QuestionsQuestions&&

AnswersAnswers

co-location sites for business continuity and disaster recovery peter lesser [email protected] (212)...

Documents

kraft kennedy lesser

q ak raft kennedy lesser

primary data centers

firms data centers

systems recovery

firm applications

critical applications

disaster recoverypeter