high assurance smart grid - carnegie mellon universityelectricityconference/2010/overman_high... ·...

BOEING is a trademark of Boeing Management Company.Copyright © 2010 Boeing. All rights reserved. 1 of 28

High Assurance Smart Grid An Outcome of Power Engineering and Cyber Security Engineering

09 March 2010Tom OvermanChief Architect, Energy Solutions Cyber [email protected]

Smart Grid Cyber Securityis more than just applying IT security

to grid control linksIt is a total System Design approach

Copyright © 2010 Boeing. All rights reserved. 2 of 28

Agenda

Review of threat examples, with Lessons Learned

Grid integration

Threat Response:

– An Architectural Approach to achieve a High Assurance Smart Grid

High Assurance

– A broad term encompassing dimensions of both high security and high availability

Copyright © 2010 Boeing. All rights reserved.


Threats against the emerging Smart Grid range from hackers on the low end to terrorists and nation/state adversaries on the high end

The Threat


The ThreatLessons Learned

(IT Solutions Approach)Apply appropriate security to remote accessCritical patch installation needs to drive trusted agent statusData/command integrityDefense-in-depth strategies, Firewalls & IDSDelete user accounts after terminationsDon’t perform database updates on live systemsDon’t use administrative controls to solve system anomaliesIdentify controls to critical assetsIntegrated physical securityInvestigate anomalous system behaviorRole based accessSecure remote (trusted) access channelsTrusted agentsUse secure radio transmissions

All necessary, but not sufficientthese do not address grid control architecture



NIST Smart Grid Conceptual Reference Diagram

System Details



Example of Tomorrow’s Smart Highly Integrated Grid

CriticalLoads

Non-CriticalLoads Housing

Energy Consuming EquipmentElectric Vehicles

(Charging & Storage)Wind Solar

Distributed Energy Resources (DER)

On-Site Peaker

Other

Installation or Regional Networked Energy

Operations Center (NEOC)

Distributed Generators

Storage

Installation Utility Grid Interface

IntelligentSub Station

Energy DemandDriving InformationUtilities – Energy

Providers

Purchase/Demand Response/Stability Support

GeothermalPower

IntelligentTransformer Vault (HTV)



Example of Tomorrow’s Vulnerabilities

CriticalLoads

Non-CriticalLoads Housing

Energy Consuming EquipmentElectric Vehicles

(Charging & Storage)Wind Solar

Distributed Energy Resources (DER)

On-Site Peaker

Other

Installation or Regional Networked Energy

Operations Center (NEOC)

Distributed Generators

Storage

Installation Utility Grid InterfaceIntelligent

Sub Station

Energy DemandDriving InformationUtilities – Energy

Providers

Purchase/Demand Response/Stability Support

GeothermalPower

IntelligentTransformer Vault (HTV)

Points of System Entry

Every node on the System represents a Point of System Entry for an attack



Smart Grid Standards Overview and Relationships

Slide source:Scott NeumannUtility Integration Solutions, Inc.Used with permission

Behind the pretty pictures, the complex of interrelated standards will create vulnerabilities



Threat Vectors

Making grid control systems ‘smarter’(more interconnected) exponentially increases vulnerabilities

Transmission ISO/RTO

Distribution

Attack

AMI/HAN

Utility Applications

Bulk Power



Threat Response: High Assurance Smart Grid

Transmission ISO/RTO

Distribution

Protect

AMI/HAN

Utility Applications

1. In addition to good IT Security, implement distributed intelligence at end nodes and between entities. These agents analyze data from multiple sources before executing commands from potentially compromised links, systems and sensors

2. Physical and Cyber security can be applied within and between entities3. Defense in Depth within entities and reports to hierarchical SA systems

Includes traditional ‘IT Security’ controls such as encryption and authentication

Bulk Power



High Assurance Smart Grid AttributesIntegrated Energy Management, Cyber Security and Physical Security with Defense in Depth

– Including strong Role Based Access Control (RBAC) for people and devices

Secure distributed architecture enables autonomy and eliminates single point of failure

Assume compromise in the system(through malice or system failure), and engineer energy control systems accordingly

Creating a High Assurance Smart Gridrequires utilizing the best attributes from multiple disciplines

Firewall/External Gateway

Physical Security

PKI

Authentication, Authorization,

and Accounting

Encryption- Data in Transit

- Data at Rest

SIEM

HIDS

Host Firewall

Detection/Response

Layer

Data Protection

Layer

Service Protection

Layer

Boundary Protection

Layer

Honeypots

Protocol Access Lists

Routing Protocols

Authentication

SwitchportSecurity

NIDS Sensors Secure

NMS protocol


Physical Security

PKI


and Accounting


- Data at Rest

S IEM

HIDS

Host Firewall

Detection/Response

Layer

Data Protection

Layer

Service Protection

Layer

Boundary Protection

Layer

Honeypots

Protocol Access

Lists

Routing Protocols

Authentication

SwitchportSecurity

NIDS Sensors Secure

NMS protocol

Firewall/External Gateway Physical

Security (e.g. tamper protection)

PKI


and Accounting



- Data at Rest

Autonomous Sensors

Actuators

Secure NMS

protocolIDS

Sensors

Autonomous Protection

Layer

Data Protection

Layer

Service Protection

Layer

Boundary Protection

Layer


Physical Security

PKI


and Accounting


- Data at Rest

SIEM

HIDS

Host Firewall

Detection/Response

Layer

Data Protection

Layer

Service Protection

Layer

Boundary Protection

Layer

Honeypots

Protocol Access

Lists

Routing Protocols

Authentication

SwitchportSecurity

NIDS Sensors Secure

NMS protocol

BulkPower

Trans-mission

Distri-bution

AMIHANBMSControl

Room DG

DSFRR



Generic DOD “Security Onion”for Robust Cyber Security and Defense in Depth

Defense-In-Depth: Boundary to Core

Security Mgr (Log-On)

Defense-In-Depth: Architecture Layers

HMI

Security Mgr Service (Access Control)Cross-Domain Service

Key Mgt Security LabelRBAC

ServiceSecurityPolicy

Data MgtSecurity

VPNFilter

Router

PhysAccess

CSlv2SSL/TLS

PacketLabel

MediaEncryption

Audit

IDS

EnclaveBoundary

Protection Service Protection

Data ProtectionDetect/

Respond

Plus• Wireless

Networks• Classified

Interconnections• Key Mgt Infrastructure (KMI)/

Public Key Infrastructure (PKI)• Detect and Respond

Aircraft Demo SystemTailored IA Controls

Future System IA Controls

Green Text

Blue Text+ Upgraded Green

Interconnect Mgt TEMPESTVPN

Sanitization

COMSEC

AuthenticationRole-Based

Access Control Malicious CodeConfigMgmt

KeyMgmt

IdentityMgmt

SessionEncryption

MediaEncryption

Data Label

Backup/Restore

IntegrityControls

RAID

PhysicalAccess Controls

FilteringRouter Anti-

TamperIntrusionDetectionSystem

Incident Reporting

Firewall

OPSEC

ControlledInterface

Clearance/Need to Know

Service

Infra-structure

Operating System

Network

Physical

FIREWALL

GUARD

Aspects of DOD defense in depth model must be applied to secure the Smart Grid


High Assurance Smart Grid (HASG) Architecture


Physical Security

PKI


and Accounting


- Data at Rest

SIEM

HIDS

Host Firewall

Detection/Response

Layer

Data Protection

Layer

Service Protection

Layer

Boundary Protection

Layer

Honeypots


Routing Protocols

Authentication

SwitchportSecurity

NIDS Sensors Secure

NMS protocol


Physical Security

PKI


and Accounting


- Data at Rest

SIEM

HIDS

Host Firewall

Detection/Response

Layer

Data Protection

Layer

Service Protection

Layer

Boundary Protection

Layer

Honeypots


Routing Protocols

Authentication

SwitchportSecurity

NIDS Sensors Secure

NMS protocol



PKI


and Accounting



- Data at Rest

Autonomous Sensors

Actuators

Secure NMS

protocolIDS

Sensors


Layer

Data Protection

Layer

Service Protection

Layer

Boundary Protection

Layer


Physical Security

PKI


and Accounting


- Data at Rest

SIEM

HIDS

Host Firewall

Detection/Response

Layer

Data Protection

Layer

Service Protection

Layer

Boundary Protection

Layer

Honeypots


Routing Protocols

Authentication

SwitchportSecurity

NIDS Sensors Secure

NMS protocol

To raise the confidence level of the future grid, the complete architecture and points of entry must be properly engineered and architected

Use all appropriate technologies and controls to protect:

Centralized management systems

Data in transit between field devices and Control Room

Distributed sensors and actuators

HASG combines defense in depth with trust model

Engineer field sensors and actuators to be resistant to improper commands from potentially compromised Control Center management systems




Physical Security

PKI


and Accounting


- Data at Rest

SIEM

HIDS

DOD/DHSSystemHardeningCorrelation/

Response Layer

Data Protection

Layer

Service Protection

Layer

Boundary Protection

Layer

Honey-pots

Protocol Access

Lists

Routing Protocols

Authentication

SwitchportSecurity

IPS Sensors

Secure NMS

protocol

WirelessSecurity

DistributedCyber Agentsdeveloped for

DOD

Notes:Certification and Accreditation per DOE, DOD and/or DHS Methods performed on entire systemEntire security solution developed and evaluated using security systems engineering procedures from DOE, DOD, DHS

Control Center Cyber Security Defense in Depth

Each instantiation in the architecture requires a slightly modified approach to raise the Probability of Assurance



Sub Station and Field Device Cyber Security Defense in Depth

Each instantiation in the architecture requires a slightly modified approach to raise the Probability of Assurance


SwitchportSecurity

IPSSensors

DHS caliberPhysical

Security for unmanned

assets

Honey-pots

Protocol Access

Lists


- Data at Rest

Autonomous Sensors/Actuators

Secure NMS

protocol

Routing Protocols

Authentication


Layer

Data Protection

Layer

Service Protection

Layer

Boundary Protection

Layer

PKI


and Accounting

HIDS

WirelessSecurity

DistributedCyber Agentsdeveloped for

DOD

Notes:Certification and Accreditation per DOE, DOD and/or DHS Methods performed on entire systemEntire security solution developed and evaluated using security systems engineering procedures from DOE, DOD, DHS

DOD/DHSSystemHardening



Energy Management, Cyber Security and Physical Security

An integrated solution must bring together Energy Management, Cyber Security and Physical Security

S2GCOE capabilities ensuresecure integration of:

Energy ManagementCyber SecurityPhysical Security



Physical Security Architecture Proven by DHS

OpticalCamera

InfraredCamera

Secure Communications Link

UGS

UGS

UGS

900 MHz

OpticalCamera

InfraredCamera

900 MHz

900

MHz

MicrowaveFiber orCopper

Microwave,

Fiber or

Copper

Cyber / Physical Security Monitoring Center

HMIC2Server

RTURTU

CyberIntruder

PhysicalIntruder

SEIM

NIDSFeature Set

in Router

HASG incorporates DHS caliber physical security which was developed to protect unmanned critical assets along US borders



Integrated Energy Management, Cyber Security and Physical Security with Defense in Depth



Physical Security

PKI


and Accounting


- Data at Rest

SIEM

HIDS

Host Firewall

Detection/Response

Layer

Data Protection

Layer

Service Protection

Layer

Boundary Protection

Layer

Honeypots


Routing Protocols

Authentication

SwitchportSecurity

NIDS Sensors Secure

NMS protocol


Physical Security

PKI

Authentication, Authoriz ation,

and Accounting


- Data at Rest

S IEM

HIDS

Host Firewall

Detection/Response

Layer

Data Protection

Layer

Service Protection

Layer

Boundary Protection

Layer

Honeypots

Protocol Access

Lists

Routing Protocols

Authentication

SwitchportSecurity

NIDS Sensors Secure

NMS protocol



PKI


and Accounting



- Data at Rest

Autonomous Sensors

Actuators

Secure NMS

protocolIDS

Sensors


Layer

Data Protection

Layer

Service Protection

Layer

Boundary Protection

Layer


Physical Security

PKI


and Accounting


- Data at Rest

SIEM

HIDS

Host Firewall

Detection/Response

Layer

Data Protection

Layer

Service Protection

Layer

Boundary Protection

Layer

Honeypots

Protocol Access

Lists

Routing Protocols

Authentication

SwitchportSecurity

NIDS Sensors Secure

NMS protocol

High Assurance Smart Grid Attributes

Secure distributed architecture enables autonomy and eliminates single point of failure


Creating a High Assurance Smart Gridrequires utilizing the best attributes from multiple disciplines

BulkPower

Trans-mission

Distri-bution

AMIHANBMSControl

Room DG

DSFRR



Ener

gy F

low

Mesh with Distributed Intelligence

Mesh with Distributed GenerationQ: Why have a distributed control architecture?

A2: It reduces risk of the Control Room as a point of failure

EnergyMesh

Network

ControlMesh

NetworkFu

ture

Past

Pres

ent

HierarchicalPr

esen

t

Hierarchical

Grid

Con

trol

Flo

w

BulkPwr

T DBulkPwr

T D

Field Devices

CtrlRm

Substn

Substn

Substn

FFF

FFF

FFF

FField Devices

CtrlRm

Substn

Substn

Substn

FFF

FFF

FFF

F

A1: The Grid and Grid Control Architecture must match


High Assurance Smart Grid Substation Example

In both examples, Control Room sends command to closeGrid segments are out of phase, which will cause damage if actuator closes

High Assurance Smart Grid comes only from integratingCyber Security, Physical Security, and Distributed Energy Management

In Substation 1, Actuator 1 trusts the command, activates, resulting in damage

In Substation 2, Actuator 2 receives a command to close, directly validates of local sensor status and Substation 3 status,and refuses the command

BOEING PROPRIETARY

BOEING PROPRIETARY



Strong Distributed Cyber Security Enables Trusted Distributed Intelligence for Energy Control

Not just “Distributed Agents”but Distributed Intelligence

Many Agents are just “Rules-Based”

Autonomy requires Distributed Intelligence

Software Control Agents for:

Grid Management

Cyber Security

Physical Security

Smart Grid Control Node

EMI HardenedSingle Board Computer

Real Time Operating System

Control Agents

Pw

rFlo

wC

trl

Dem

and

/R

e spo

nse

I/F

FiberNIC

S2GCOEReal Time Version

Se n

sorI

nte g

r at io

n

Cy b

e rS

ecur

ity

Distributed Control Agents assure no action is taken based on a single input HASG leverages distributed cyber agents developed for DOD

BOEING PROPRIETARY

BOEING PROPRIETARY



High Assurance Smart Grid ArchitectureAssume Failure and/or Compromise of Control System Components

BMS = Building Management SystemAMI = Advanced Metering InfrastructureHAN = Home Area Network

Cata-strophic

Implement Multi-Tier Model For Impact of System Failure

High

Medium Low

High Assurance Trust Model: Assume failure or compromise of Control Systems

Limit impact of failure or compromise by requiring field devices to synthesize Control Room commands and direct read of distributed sensors

1. Objectively characterize system criticality based on impact of failure (e.g. # customers impacted )

2. Control Room should not be the most critical grid system3. Field devices must rely on at least two independent inputs before taking action4. In any given day, failure or compromise of any single element should be assumed

DG = Distributed GenerationDS = Distributed StorageFRR = Frequency Responsive Reserve (Loadbanks, flywheels, etc.)

BulkPower

Trans-mission

Distri-bution

AMIHANBMSControl

Room DG

DS

FRR

Each entity in Trust Model has its own defense in depth “onion”


High Assurance Smart Grid AttributesIntegrated Energy Management, Cyber Security and Physical Security with Defense in Depth


Secure distributed architecture (not hierarchical) enables autonomy and eliminates single points of failure


High Assurance Smart Grid Solutionsutilizing the best attributes from multiple disciplines


Physical Security

PKI


and Accounting


- Data at Rest

SIEM

HIDS

Host Firewall

Detection/Response

Layer

Data Protection

Layer

Service Protection

Layer

Boundary Protection

Layer

Honeypots


Routing Protocols

Authentication

SwitchportSecurity

NIDS Sensors Secure

NMS protocol


Physical Security

PKI

Authentication, Authoriz ation,

and Accounting


- Data at Rest

S IEM

HIDS

Host Firewall

Detection/Response

Layer

Data Protection

Layer

Service Protection

Layer

Boundary Protection

Layer

Honeypots

Protocol Access

Lists

Routing Protocols

Authentication

SwitchportSecurity

NIDS Sensors Secure

NMS protocol



PKI


and Accounting



- Data at Rest

Autonomous Sensors

Actuators

Secure NMS

protocolIDS

Sensors


Layer

Data Protection

Layer

Service Protection

Layer

Boundary Protection

Layer


Physical Security

PKI


and Accounting


- Data at Rest

SIEM

HIDS

Host Firewall

Detection/Response

Layer

Data Protection

Layer

Service Protection

Layer

Boundary Protection

Layer

Honeypots

Protocol Access

Lists

Routing Protocols

Authentication

SwitchportSecurity

NIDS Sensors Secure

NMS protocol

BulkPower

Trans-mission

Distri-bution

AMIHANBMSControl

Room DG

DSFRR


The Solution – A System Design Approach

Smart Grid Cyber Security is more than just applying IT securityto grid control links – It is a total System design approach

High Assurance Architectural Requirements


Physical Security

PKI


and Accounting


- Data at Rest

SIEM

HIDS

Host Firewall

Detection/Response

Layer

Data Protection

Layer

Service Protection

Layer

Boundary Protection

Layer

Honeypots


Routing Protocols

Authentication

SwitchportSecurity

NIDS Sensors Secure

NMS protocol


Physical Security

PKI


and Accounting


- Data at Rest

S IEM

HIDS

Host Firewall

Detection/Response

Layer

Data Protection

Layer

Service Protection

Layer

Boundary Protection

Layer

Honeypots

Protocol Access

Lists

Routing Protocols

Authentication

SwitchportSecurity

NIDS Sensors Secure

NMS protocol



PKI


and Accounting



- Data at Rest

Autonomous Sensors

Actuators

Secure NMS

protocolIDS

Sensors


Layer

Data Protection

Layer

Service Protection

Layer

Boundary Protection

Layer


Physical Security

PKI


and Accounting


-Data at Rest

SIEM

HIDS

Host Firewall

Detection/Response

Layer

Data Protection

Layer

Service Protection

Layer

Boundary Protection

Layer

Honeypots

Protocol Access

Lists

Routing Protocols

Authentication

SwitchportSecurity

NIDS Sensors Secure

NMS protocol

BulkPower

Trans-mission

Distri-bution

AMIHANBMSControl

Room DG

DSFRR

Apply appropriate security to remote accessCritical patch installation needs to drive trusted agent statusData/command integrityDefense-in-depth strategies, Firewalls & IDSDelete user accounts after terminationsDon’t perform database updates on live systemsDon’t use administrative controlsto solve system anomaliesIdentify controls to critical assetsIntegrated physical securityInvestigate anomalous system behaviorRole based accessSecure remote (trusted) access channelsTrusted agentsUse secure radio transmissions

IT Lessons Learned


The Integrated Solution for a High Assurance Smart Grid:Energy Management, Cyber Security and Physical Security

1. Engineer energy control systems using High Assurance principles

– From utility, aviation, space and government systems

2. Distributed Intelligence

– For Cyber Security, Physical Security and Grid Management

3. Implement Role Based Access Controls

– For people and for devices

4. Deploy Cyber and Physical Security sensor solutions

– Integrated with Energy Management network systems

5. Provide Common Operational Picture (COP) capability

– Consolidated view of Energy Management, Cyber and Physical Security

High Assurance Smart Grid Solutions –An integrated approach across multiple disciplines


Summary:

High Assurance Smart Grid Solutions require the best possible security and reliability,

achieved by integrating Utility Applications and Control Systems

withDOD Cyber and DHS Physical Security Solutions

and with Power & Aerospace Control Systems Engineering.

high assurance smart grid - carnegie mellon universityelectricityconference/2010/overman_high... ·...

Documents