north carolina digital government summit continuity of operations: technology trends and solutions...

North Carolina Digital Government Summit

Continuity of Operations: Technology Trends and Solutions

September 2, 200910:45am – 12:00pm

Ann Garrett, State CISOOffice of Information Technology Services

2

Welcome

House Bill 2436

Recommendations to General Assembly for Enterprise IT Business Continuity

Accomplishments / Successes• Standardized Tools & Reports• NCID Integration• Virtualization• Western NC Data Center

Vision

Questions & Answers


Outline

3

SECTION 6.10.(a) The State Chief Information Officer (CIO) shall utilize the business and disaster recovery plans submitted under G.S. 147-33.89 and any other information at the CIO's disposal to determine whether State agencies have made adequate preparations for backing up critical applications.

SECTION 6.10.(b) In cases where backup is not sufficient to minimize any disruptions in critical State services caused by natural or man-made disasters, the State CIO, in conjunction with the agencies and the Office of State Budget and Management, shall develop plans to utilize the Western Data Center for providing backup.

SECTION 6.10.(c) By December 1, 2008, the State CIO shall report to the Joint Legislative Oversight Committee on Information Technology on the number of critical State applications without adequate backup, the State agencies utilizing the applications, and the plans for providing adequate backup.

SECTION 6.10.(d) This section does not apply to the General Assembly, to the Judicial Department, or to The University of North Carolina and its constituent institutions.


House Bill 2436, Section 6.10 (2007 – 2008 Session) IMPROVE DISASTER RECOVERY AND

BUSINESS CONTINUITY

4

1. Improve, standardize and centrally manage business continuity planning tools.

2. Use the data from enterprise business continuity planning tools to help identify common gaps in BCPs for the recovery of critical applications.

3. Identify critical dependencies and plan for adequate support infrastructure.

4. Work with Dept. of Homeland Security and the Division of Emergency Management on security measures at the Eastern Data Center and Western Data Center.

5. Provide agencies with the opportunity to conduct both informal and formal business continuity tests and maintain records of agency critical application BCP tests.

http://www.scio.state.nc.us/News_Files/Disaster_Recovery_and_Business_Continuity.pdf


Recommendations To General Assembly – December 2008

5

1. Improve, standardize and centrally manage business continuity planning tools. One Business Continuity Planning Tool for BCP and COOP

- Living Disaster Recovery Planning System (LDRPS) With NCID front-end for user-id and password management Web based application in a virtual Windows environment Graphical dependency mappings

2. Use the data from enterprise business continuity planning tools to help identify common gaps in BCPs for the recovery of critical applications.

Standardized plans using Crystal reports and Oracle database All BCPs follow the same table of contents All have the same ‘look and feel’ Easier to locate critical plan elements Application Portfolio Mgt tool captures critical application data Produced the ‘IT Scorecard’ for the Governor


Accomplishments / Successes

6

3. Identify critical dependencies and plan for adequate support infrastructure.

Example:


Accomplishments and Successes

7

4. Work with Dept. of Homeland Security and the Division of Emergency Management on security measures at the Eastern Data Center and Western Data Center.

Completed

5. Provide agencies with the opportunity to conduct both informal and formal business continuity tests and maintain records of agency critical application BCP tests.

Semi-annual formal test cycles in place Informal testing can be considered Tracking of BCP testing is now more robust


Accomplishments and Successes

8


Western NC Data Center

The need for a second data center for NC Study in 2004 identified need for fully redundant,

round-the-clock disaster recovery capability in N.C. Growth causing strain on outside vendor’s capacity at

single recovery site Disaster recovery testing time consuming and

expensive Escalating costs Finite floor space and power capacity at main ITS site

in Raleigh

9



Additional factorsGrowing recognition in agencies and

General Assembly of disaster recovery needs for critical business programs

Future capacity needs because of growth and State CIO initiatives, such as consolidation of IT infrastructure

Administration desire to create jobs in state

10



Our approachBuilt case with administration, then N.C.

General AssemblyIdentified siteStudy recommended Western NC

Physical separation from Raleigh in case of hurricane, ice storm, earthquake, homeland security incident

Looked at 20 counties, 20+ sitesWorked with private sector and state and federal

government agencies

11

53,000 square feet15,000 square feet raised floor spaceFull redundancy in power, cooling and fiber24/7/365 operation



12



OutcomeCompleted on time and within budgetThree full-scale disaster recovery exercises

completed since June 200815 agencies participating in DR exercises; a

record numberITS ended contract with out of state vendor in

June 2008, and began backing up all core services at the Western Data Center

13


State Data Centers

Technologies Modernized gear to provide automatic switching from

“line”, “battery” and “generator” power sources at EDC

MPLS enabled, fully redundant, 10G ring architecture providing connectivity between the data centers

VMWare with virtual tape at WDC SAN at EDC VOIP

14

State of the Art Data Center going light green Redundancy Synchronization between EDC & WDC XRC – Extended Remote Copy for Mainframe Teleworking expanded

Enabling more staff to communicate and work remotely

No single points of failure Standardization and blending of the BCP and COOP


Vision

15


Questions?

north carolina digital government summit continuity of operations: technology trends and solutions...

Documents

technology trends

state agencies

critical state services

state cio

critical dependencies

business continuitysection

disaster recovery plans

adequate backup