Cybersecurity and

the Smart Utility

Branndon Kelley Chief Information Officer

Fast Facts: American

Municipal Power • Wholesale power supplier and services provide for

132 municipal electric systems in 9 states and service more than 637,000 customers.

• AMP members receive their power supply from a diversified resource mix that includes wholesale power purchases and energy produced utilizing fossil fuels and renewable resources.

• Focused on sustainability and increased use of renewable generation resources with plans to add more than 300 MW of new hydro capacity to the region.

History of AMP

• Founded in 1971 with the purpose to provide the generation, transmission, and distribution of electric power and energy to its members at lower costs. This purpose is served by: – Joint ownership of electric facilities

– Pooled buying power in energy markets

– Pursuing additional means of generating, transmitting and distributing electric power and energy

• Original members were all located in Ohio (AMP-Ohio). Name changed in 2009 to AMP.

1800s - Early days of

electricity

• Systems small and

localized

• Generation built close

to the end user

• Limited transmission

capabilities

The Pearl Street Station in New

York City

1900s – Establishment of The Modern Grid

AEP 765kV transmission

tower in Virginia

Prairie State Energy

Campus in Illinois

• Began in the late 1800s.

• Transmission lines make it possible to separate generation from the end user by many miles.

• More complex system but benefits outweigh challenges

1990s & 2000s

• 1992 - De-regulation

• Residential customer begins installing

their own generation

Rooftop Solar

• Even more complex

systems.

Future – The Smart Grid

• Many types and

sources of

generation

• Millions of

hackable utility

connected

devices

Evolution of the Utility

Smart Grid = Smart Utility

Smart controls on distribution poles

Microgrids and energy reduction

Solar & Advanced Metering (AMI)

Sensors on Assets in Power Plants

Smart Utility – Power Generation

• Distributed control systems & automation reduce the number

of people it takes to run a power plant.

• Sensors and system provide data for pro-active maintenance

to take place and reduce unnecessary maintenance.

• All resulting in safer facilities and less forced outages.

Smart Utility – T & D

• SCADA system allow for better monitoring of the grid and

identification of issues.

• Automated reclosers provides for better detection and

interruption of momentary faults

• All resulting in faster restoration during weather events and

more efficient system maintenance.

Smart Utility – Micro Grids

• Can operate with the main grid or independently as an

electrical island

• Locally controlled systems

• Often contain multiple generation types with battery storage

• Current State of

CyberSecurity

Latest in the News

Threat Vectors

• Physical Attacks

• Malware - Viruses/Exploits

• Phishing Attacks & Social Engineering

– Targeted Attacks to Extract Information

• Advanced Persistent Threats

– Well planned

– Often Nation State or Organization Sponsored

Top target roles – Spear Phishing

Symantec Internet Security Threat Report – April 2015, Volume 20

Vulnerabilities in ICS

The Structure of an Advanced Persistent Threat

Source: Dell Secureworks

Smart Enablement Cyber Risk

• Generation Example

• Attackers gain access to an unnamed plant’s office network

through a targeted malicious email

• Attacker’s are ultimately able to cross over into the production

network.

• The plant’s control systems are breached which results in an

incident where a turbine could not be shut down in the regular way

and the turbine was in an undefined condition which resulted in

massive damage to the whole system

Smart Enablement Cyber Risk

• Distribution Example

X X X

X

X

X X

X X

Smart Enablement Cyber Risk

• In the Home Example

Water Heater Thermostat

Connected utility and security can

co-exist.

• Must create a culture of cyber security

• Leveraging best practices for Physical and Cyber Security is key

• Standards do exist for implementing effective cyber security

– SANS 20 Critical Security Controls

– NIST Cybersecurity Framework

Physical Security Best Practices

• Review/Confirm security procedures and regular inspection of facilities

• Provide Security Training and awareness for staff

• Hold Security Briefings for key personnel

• Limit Access to Facilities and Systems to authorized personnel only

• Security Badges and Electronic Security Systems

• Procedures to prevent tailgating and unauthorized entry to facilities

Cyber Security Best Practices

• Adopt a Framework (SANS, NIST)

• Cyber Security Training

• Penetration Tests & Vulnerability Assessments

• Tabletop exercises

• Restrict Physical Access to IT Devices/Networks

• Practice Incident Response

Cyber Security Incident Response

• Take a not “if” but “when” approach

• Drill incident response and include

executive management.

• Reviewed layered defense strategy to

identify defense points.

Cyber Security Systems

• Firewalls, Intrusion Prevention Systems, and

Web Filters

• Sandboxing - Advanced Persistent Threats

• Endpoint based Protection and Whitelisting

– Traditional Antivirus is becoming less effective

• Network Access Control Systems

• Multi-Factor Authentication

• Separated Networks with Layered Defenses

Air Gapping is becoming more difficult

• USB drive plugged in

• Engineering laptop plugged in

• Researchers are discovering ways to bridge air gaps with cell phones

• IT and OT personnel have to work together to secure systems at all layers instead of creating a hardened outer perimeter with a weak inner network.

Defense in Depth / Layered Security

• Originally a military strategy that seeks to delay, rather

than prevent, the advance of an attacker by yielding

space in order to buy time.

• Test defenses with Red Team vs Blue Team Exercises

Source: NERC

30

Redefining AMP’s Strategy

What we know…

• The utility industry business is increasing its use of technology - in

the business, in field equipment, and by customers

• Our member municipalities have an emerging need

– Skill & talent not locally available

• Our operations are becoming more vulnerable to attack

– Cybersecurity engineering is of paramount importance

Members have recognized AMP’s ability to effectively

manage bulk power purchases, generation facilities

and power supply contracts

• AMP’s Board has identified the need to support members in their

adoption of technology in their operations

Redefining AMP’s Strategy

One of the eight teams is focused on technology

enablement - “Hosted Solutions"

• AMP members are evaluating many technologies in the

distribution and customer operations parts of the business

• Vendors, distributors, and independent providers have identified

the need within small municipal utility operators

• The term – “Hosted Solutions” – is reflective of what the

marketplace refers to these services

– Vendors providing these services to individual members

AMP’s Smart Grid Program Project launched on January 6, 2015

• Focus on simplifying AMI adoption for AMP members

• Recognize variability among member’s requirements

Pilot member utilities’ benefits

• Aggregating purchasing of equipment

• Mitigating the risks associated with local deployment of major

technology components like Meter Data Management Systems

• Support business case & financial modeling

• Assistance with presentations to leadership, where required

• Provide collateral material for customer communications

Program Leadership • Under supervision of AMP Chief Technology Officer,

Jared Price.

– Has been with AMP since 2011

– Has responsibility for Overall IT Enterprise Architecture, SCADA

and plant systems across AMP’s generation portfolio

– 10+ years of experience in infrastructure management, project

management, and enterprise architecture across multiple

industries including banking & finance, healthcare, education,

and utilities.

– Holds Global Industrial Cyber Security Professional Certification

(GICSP), #178

• Also retain a Smart Grid Consultant / Owner’s engineer

with 30+ years of large utility experience.

Program Overview • AMP will host the back-end AMI and Meter Data

Management System (MDMS) for individual

member utilities.

• AMP Will provide staffing and expertise to run

these systems.

• RFI and RFP process to major systems vendors

earlier this year.

• Pilot member committee helped in shaping the

program.

• Go live planned in early 2016

Member Business Drivers • Address aging meter assets and meter reading

equipment

• Improve customer service

• Support for emerging needs – rates, distributed

generation

• Leverage join action to gain lowest possible cost

• Defer to AMP (vs. Vendor) management of

technology

Current State - HHMR

Billing

System

• Manual meter reading process

• Aging meters, handheld equipment

• Support for new rates

• “Smart grid” platform & customer expectations

Advanced Metering Evolution - AMR

Meters

Meters replaced with “One-Way” RF System;

Reading with “drive by” equipment

• Improves efficiency (less estimates, lockouts)

• Continued shortcomings on advanced rates,

smart grid capabilities, & customer expectations

Billing

System

AMP Advanced Metering Solution - AMI

Back Office Infrastructure

MDM Customer

Portal

Utility

Portal Outage

viewer

AMP Managed Systems

Wireless

Network

AMI

Head-End Field

Infrastructure

Billing

System

Meters

Utility Systems

AMI Solution Security • AMP is able to leverage Cyber Security defenses and

best practices with the deployment and management of

this solutions

– Many of our members do not have the expertise to do

this on their own

• AMP is also able to leverage trusted partners that have a

forward thinking approach to cyber security like Kevin

Goodman and Bluebridge networks. AMP will host this

system like many other critical systems within the

Bluebridge datacenter.

References http://www.engin.umich.edu/college/about/news/stories/2011/may/living-off-the-grid-smart-grids-are-current-

technology-at-its-best

Living off the grid: smart grids are current technology at its best

By Marilyn Tsao

http://www.gereports.com/every-electron-gets-byte-digital-power-plant-makes-electricity-smart/

Every Electron Gets A Byte: Digital Power Plant Makes Electricity Smart

By Tomas Kellner

http://www.scmagazine.com/cyberattacks-costing-big-business-big-

bucks/article/443982/?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed:+SCMagazineHo

me+(SC+Magazine)

Cyberattacks costing big business big bucks

By Dough Olenick (SC Magazine)

http://www.infosecurity-magazine.com/news/dow-jones-hacked-affecting/

Dow Jones Hacked, Affecting Thousands

By Tara Seals (Infosecurity Magazine)

Workshop: Building a Utility Customer Digital Engagement Program

By Chet Geschickter (Gartner Symposium ITXPO 2015)

References (continued…) http://www.infosecurity-magazine.com/news/dow-jones-hacked-affecting/

Dow Jones Hacked, Affecting Thousands

By Tara Seals (Infosecurity Magazine)

http://www.infosecurity-magazine.com/news/uks-nuclear-industry-at-risk-of/

UK’s Nuclear Industry at Risk of Major Cyber-Attack

Phil Muncaster (Infosecurity Magazine)

https://www4.symantec.com/mktginfo/whitepaper/ISTR/21347932_GA-internet-security-threat-report-

volume-20-2015-social_v2.pdf

Symantec Internet Security Threat Report – April 2015, Volume 20

http://blogs.wsj.com/cio/2014/12/18/cyberattack-on-german-iron-plant-causes-widespread-damage-report/

Wall Street Journal (Dec 18, 2014) - Cyberattack on German Iron Plant Causes ‘Widespread Damage’:

Report

http://www.nist.gov/cyberframework/index.cfm

NIST Cyber Security Framework

http://www.sans.org/critical-security-controls/control/20

SANS Critical Security Control: 20

THANKS!