Continuity of OperationsA Introduction to

Taxpayer Services’ Business Continuity Plan

OR…

What the heck is it? And why do we need a plan?

WelcomeWelcome

Rules of EngagementRules of Engagement

IntroductionsIntroductions

AgendaAgenda

Significance of COOP Planning

The World Trade Center The World Trade Center Building 7Building 7

At precisely the time when this office had a At precisely the time when this office had a critical service to deliver to the citizenry, they critical service to deliver to the citizenry, they were faced with a severe crisis - the loss of were faced with a severe crisis - the loss of their facility and its operational capabilitytheir facility and its operational capability

The office was able to re-establish an Emergency The office was able to re-establish an Emergency Operations Center at Pier 92, just three days after Operations Center at Pier 92, just three days after the collapse.the collapse.

Probability (likelihood that an incident will occur), frequency (how often an incident occurs), and severity (impact of an incident) are factors that weigh heavily into COOP planning.

COOP Planning

This is the reason why COOP This is the reason why COOP planning planning mustmust address the full address the full range of scenarios. (All Hazards)range of scenarios. (All Hazards)

COOP Planning

COOP Planning Includes…

• The safety of personnel and visitorsThe safety of personnel and visitors

• The ability to continue essential operations The ability to continue essential operations

• And provisions for the protection of:And provisions for the protection of: Critical equipmentCritical equipment Critical recordsCritical records Other assetsOther assets

COOP Planning Includes…

• Efforts to minimize damage and losses.Efforts to minimize damage and losses.

• Provisions for an orderly response and Provisions for an orderly response and recovery from any incident.recovery from any incident.

• A foundation for the continued survival of A foundation for the continued survival of leadership.leadership.

The consequences of The consequences of notnot developing a COOP developing a COOP capability are very high, as demonstrated by capability are very high, as demonstrated by the response efforts on September 11, 2001.the response efforts on September 11, 2001.

Legal Ramifications

COOP Plan

• The plan should have provisions or procedures for the following: Relocation sites for alternate operations For maintaining operations For Recovery and return to normal operations For Vital record maintenance and back-up

provisions / procedures For staff notification

COOP Elements

• Outline Phases of COOP to include: Activation procedures Alternate operations (sites) Reconstitution and termination Annexes with detailed information on each of

the essential elements Procedures for implementation, according to

the magnitude of the incident should also be included

Questions?

The Evolution of Hennepin County’s Business Continuity

Plan

Before Business Continuity PlanningBefore Business Continuity Planning

In the Beginning

• The Year Was 2003…• Richard Johnson (then Deputy County

Administrator) decided that… Hennepin County needed a plan to continue

business even after a disaster We would hire a consultant to write it We would train one

employee to supervise it

Business Impact

Analysis & Risk

Assessment

Plan Testing

Recovery Strategies

Plan Development

The Evolutionary Cycle of the BCP

What We Needed to Accomplish

1. Business Impact Analysis2. Strategies3. Emergency Response4. Develop a County Plan

• Departmental Plans

5. Develop Awareness & Training Programs6. Maintenance Program

Who Would Do All of This?

The Bottom Line

• Nobody knows what we do better than us so…

• We (Hennepin County) have to write the plan

• The consultants will…

• Organize the work

• Develop time line and...

• Supply the format

A Little Help From Our Friends

Business Impact Analysis • Conducted by Intertech, a division of the

Department of Administration, State of Minnesota

County Plan and Recovery Strategies

• Prepared by LBL Partners

Business Impact Analysis

• What were the business functions each

department performed?

• Which functions were critical?

• Which functions had to be recovered first?

• Recovery Time Objectives

First Things First

Information Collection

• Intertech conducted hundreds of interviews with HC personnel to determine each Department’s:• Prioritized Services, Processes and Functions.

• Estimate Recovery Time Objectives (RTO).

• Determine Resource Requirements.

• Business Impact Analysis Complete: June 2005

So…

• Business Impact Analysis

• Strategies

• Emergency Response

• Develop a County Plan• Departmental Plans

• Develop Awareness and Training Programs

• Maintenance Program

NEXT

• LBL • Furnished the template• Established deadlines for section completion• Collected final information• Assisted HC Departments with writing• Provided the software for housing the Plans

• Plans Basically Complete: Fall of 2007• Plans are kept in ContingencyPro software

program on a HC Server

Today

• Plan Map• Business Impact Analysis

• Strategies

• Emergency Response

• Develop a County Plan

• Develop Awareness and Training Programs

• Maintenance Program

Today • Our end product is sitting in front of you!• Points to remember as we move forward

• It took a long time and a lot of work to produce

• It’s our plan, to update and maintain

• We need to make it work in times of crisis

• Let’s hope we never need it!!

Incident Command for Managers

And…What We Can Learn From Fire Departments

Managers Often Need To…

• Manage IncidentsManage Incidents Security incidentsSecurity incidents Service outagesService outages Infrastructure failuresInfrastructure failures Power failuresPower failures Cooling failuresCooling failures Connectivity failuresConnectivity failures … … and so forthand so forth

Who manages emergencies daily?

• Public Safety Agencies Public Safety Agencies Fire departmentsFire departments Urban & suburbanUrban & suburban Forest & wildlandForest & wildland Police departmentsPolice departments Coast GuardCoast Guard … … etc.etc.

How do Public Safety agencies…• Organize themselves on the fly to deal with Organize themselves on the fly to deal with

a major incident?a major incident?

• Quickly and effectively coordinate the Quickly and effectively coordinate the efforts of multiple agencies?efforts of multiple agencies?

• Evolve the organization as the incident Evolve the organization as the incident changes in scope, scale, or focus?changes in scope, scale, or focus?

What could managers learn from this?

For Example…

• Car hits a fire hydrant – the car’s occupants are trapped and injured

• Water from hydrant floods an underground electrical transformer, causing a short circuit and an outage

• Who might be involved in response? Fire Dept: rescue trapped occupants Ambulance Service: treat & transport victims Police Dept: direct traffic & investigate Water Dept: shut off hydrant Electric Co: deal with flooded transformer

How do you manage all of that?

What needs to get done?

• The ambulance crew needs to treat & transport victims

• But first, the fire department needs to extricate them from wreckage

• Before the fire department can do that, the water company needs to shut off the water

• Which they can't do until the electric company safes the flooded transformer.

.

How do you organize this?

• Who is in charge?

• How do they figure out what needs to be done and who can do it?

• How do assignments get made, so that everything necessary gets done?

• No effort gets duplicated

• Everything is done safely

An Even Bigger Example…

• Southern California Wildfires Fast-changing situation Fire grows and moves as weather and winds shift Plan evolves as situation & resources change Many agencies involved Firefighters from dozens of

cities, plus CDF, USFS, BLM, and military Airborne water drop, transport, & scouting Law enforcement to deal with residents Support units (medical, kitchens, camps, fuel, etc.)

What do these types of incidents have in common?

• Time matters – need to respond quickly

• Situation not perfectly understood at start

• Learn as you go, and adjust on the fly

• Resources change over time

• People come and go; not all together at start

• Need ways to bring newcomers up to speed

• Need ways to transfer responsibilities

So, what’s the Incident Command System?

• The Incident Command System (ICS) is… Standardized organizational structure and set of

operating principles, tools for command, control,

and coordination of a response to an incident

Provides means to coordinate efforts of multiple

parties toward common goals

Uses principles that have been proven to

improve efficiency and effectiveness

History of ICS

• Developed in 1970 to coordinate agencies dealing with yearly Southern California wildfires

• Has since evolved into national standard

• Now used by nearly all US public safety agencies

• Mandated by Dept of Homeland Security in order to obtain State/Federal funding

1. Modular and scalable organizational structure

2. Manageable span of control

3. Unity of command

4. Explicit transfers of responsibility

5. Clear communications

6. Consolidated incident action plans

7. Management by objective

8. Comprehensive resource management

9. Designated incident facilities

Key ICS Principles

1 - Modular & Scalable

• Command Operations Logistics Planning/Status Admin/Finance

• Command functions are activated as needed for a particular incident

• All incidents will have a Command section• Almost all will have an Operations section• Rest of sections are used on larger/longer

incidents

1 – Modular and Scaleable

• Command Incident Commander (IC) responsible for

overall management of incident Incident Commander initially also performs all

four section chief roles until each is designated to somebody else

• Operations• Logistics• Planning/Status• Admin/Finance

1 – Modular and Scaleable

• Operations This is where the real work happens Operations develops and executes plans to

achieve the objectives set by Command Assists Command in development of

Consolidated Incident Action Plan Typically biggest section, by number of people Operations focuses on now; Planning worries

about later

1 – Modular and Scaleable

• Logistics Responsible for obtaining all resources,

services and support required to deal with the incident

Responsibilities include facilities, supplies, transportation, equipment maintenance and fueling, feeding and medical care of incident response personnel, etc.

Is more important on large, long-running incidents; may not be needed on small or short incidents

1 – Modular and Scaleable

• Planning/Status Collects & evaluates info needed to prepare

Incident Action Plan (IAP) Forecasts probable course of incident Plans for next day, next week, etc. (IAP) Keeps track of what has been done, and

what still needs to be done Keeps “current status & plans” info up to

date, so that new arrivals can brief themselves

1 – Modular and Scaleable

• Admin/Finance Responsible for tracking incident-related

costs, including time & materials, if

necessary for reimbursement

Also administers procurements arranged

by Logistics

Usually only activated on the very largest

and longest-running incidents

1 – Modular and Scaleable

• Growing the Incident Command System Initially, the most experienced first responder is

the Incident Commander (IC)

IC responsibility may transfer to somebody else later as incident grows, but isn't automatic

Generally better to keep the same IC if feasible

If IC transfer does happen, it needs to be explicit (face to face)

2 – Manageable Span of Control

• Each supervisor should have no more

than 3 – 7 subordinates

• 5 is ideal

• When necessary as organization

grows, create new levels Division might be functional or geographic

3 – Unity of Command

• On incident, each person has 1 boss

• Strict tree structure, all the way to the top

• Everybody knows who they work for

• Makes communication & coordination

easier, up /down tree as organization

grows & changes

• Reduces freelancing

4 – Explicit Transfer of Responsibility

• Changes to organization are made explicitly and announced

• More senior person doesn't automatically take over IC upon arrival

• Person already in place is often better suited to handle the current situation, and certainly is more up to speed

• Planning/Status keeps overall organizational chart updated

5 – Clear Communications

• Communicate clearly and completely, not

in code (Uses Plain English!)

• Reduces potential for confusion

• Reduces time spent clarifying

• Talk directly to resources, when possible

6 – Consolidated Action Plans• Command communicates top-level action plan

for current operational period (hour, shift, day, etc.) 6 – 12 – 24 hour operational periods

• Plan states, at a high level, what organization is trying to accomplish right now

• Written plan is best• Makes it easier to keep everybody on target• Makes it easier for new arrivals to brief

themselves

• Rule of thumb: if it crosses organizational or specialty boundaries, write it down

7 – Management By Objective• Tell people what you want them to

accomplish, not how

• Let them figure out how to get it done

• Gives them room to flexibly and creatively cope with changing circumstances

• Is generally faster to communicate, and the folks doing the work may know a better way than you

8 – Comprehensive Resource Management

• All assets & personnel need to be tracked so…

• New resources can be used most effectively

• Existing resources can be relieved

• Avoids duplication

• Folks should “sign in” through Admin function, then wait for assignment

• Helps ensure people are put to best use

9 – Designated Incident Facilities• Command Post is key facility to identify - that's

where everybody can expect to find IC

• If IC needs to leave Command Post, needs to transfer IC responsibility (temporarily or permanently) to someone who'll still be there

• EOC coordinates information & communication

• Also useful to designate “staging area” for new resources to report to upon arrival, for sign-in and assignment; may be at Command Post

ICS Tips

• Establish ICS early in an incident, If you get off to a disorganized start, you'll be playing catch-up forever

• Think of ICS as a toolbox full of tools Choose tools you need for the incident at hand Keep it simple

• Practice ICS at every opportunity If you use it for “routine” and pre-planned events like moves, social events or elections your dept. will be more comfortable using it for “surprise” events

Remember…

• The ICS organization changes as the

situation and resources change

• Following the ICS principles gives you a

way to keep it all under control

• You can keep the ICS going – indefinitely,

if needed

5 Minute Break?

Format

• In between Cover and Glossary there are 16 sections

• Open your plans to: Section 1, The Table of Contents

• If you are having a touch of insomnia please feel free to take this plan home!

What’s Important

• Directions on how we are to recover• How we are organized- Section 4

• Recovery Teams• What we have to recover- Section 7

• Critical Tasks• Recovery Time Objectives (RTO)

What’s Important

• Who is on what team – Forms Tab (Section 11)

• What to do when

• Checklists• Management Recovery Team – Section 5• Logistics Recovery Team - Section 6• Operations Recovery Team – Section 7• Technology Recovery Team – 8

Recovery Teams

• To facilitate recovery and bring the County back to normalcy the Business Continuity Plan calls for the formation of FOUR TEAMS:• Management• Logistics• Technology• Operations

Recovery Team Basics

• Emergencies require a coordinated response and each occurrence must be managed in a way that fosters consistency of effort and contains the damage

• The Recovery Teams are organized to address all issues related to an emergency and they are authorized to make specific decisions relative to recovery efforts.

Management Recovery Team

• Activating all or part of the Department Activating all or part of the Department Business Continuity PlanBusiness Continuity Plan

• Providing strategic direction to Department Providing strategic direction to Department Recovery Teams & personnelRecovery Teams & personnel

• Prioritizing objectives, actions & activities Prioritizing objectives, actions & activities

Logistics Recovery Team

• Determining immediate operating needs

• Arranging for alternate facilities, if needed

• Arranging for basic support services• Obtaining office equipment as necessary• Arranging transportation, travel and food• Arranging for temporary personnel

Technology Recovery Team

• Coordinating repairs• Restoring network equipment• Restoring the data communications network• Restoring systems to operational status• Restoring system databases and files from

backup copies of electronic media• Reconstituting to normal operations

Operations Recovery Team

• Initiating alternate procedures• Executing BCP procedures• Performing critical business functions• Reconstructing data• Reconstituting to normal operations

Section 7

The Heart & Soul of the Plan

Section 7

• In the event of disaster it is important that we continue to perform the critical business processes/functions identified in this Business Continuity Plan.

• Section 7 contains the interim and restoration procedures for the critical business functions and tasks that support these processes.

• The Operations Recovery Team will use this information to ensure that the staff is able to perform the critical business functions in a timely manner.

Priority Service Levels

• Each division has been assigned a Priority Service Level (One – Four)

• These priority services levels have been determined and categorized by the State of Minnesota

• Indicate the priority order in which services should be performed in the event a disaster impacts Hennepin County

Priority Service Levels Defined

• Priority Service Level One: Immediate threat to public health, safety or welfare

• Priority Service Level Two: Direct economic impact, constitutionally or statutorily mandated time frame or civil disorder may develop if not performed in a few days

• Priority Service Level Three: Regulatory services required by law, rule or order that can be suspended or delayed by law or rule during an emergency

• Priority Service Level Four: All other services

Discussion of Your Plan

• Open your plan to Section 7, Page 8

• What level of priority has been assigned to your services?

• What are your critical functions?

• Do you agree with them?

• Does anything need to change?

Musical Chairs

Your Recovery Teams

Continuity of Continuity of OperationsOperations

A Tabletop Exercise

• This exercise is a learning opportunity – it is not a test

• You are encouraged to think out loud and “outside of the box”

• Expect to identify some issues or problems that we cannot resolve today

• No one is expected to have all the answers – there are no “gotchas!”

• HAVE fun!!

Table Top Exercise

The Scenario

• Hennepin County Government Center• 7:30AM on a Tuesday• Fire on the “D” level involving electrical

transformers containing PCB’s• HCGC A Tower will not be available for

four to five days• There is no access to the tower to

recover records, computers or personal items

Questions to Discuss

• On what basis is the BCP activated?On what basis is the BCP activated? When?When? By Whom?By Whom?

• Who needs to be notified or involved?Who needs to be notified or involved?

• What documents and information do you need What documents and information do you need to resume operations?to resume operations?

• What alternate sites could be used to resume What alternate sites could be used to resume operations?operations?

The Scenario

• Building officials state that entry might not be possible for 3 – 4 months

• Officials consider everything in the building contaminated

• All other departments in the building are also not allowed inside the building

• Where are your alternate Where are your alternate sites?sites?

• Who are the essential Who are the essential employees that need to be employees that need to be called in?called in?

• Estimate how long it would Estimate how long it would take to resume essential take to resume essential services!services!

Questions to Discuss

Questions?

Hot Wash