cip-003-6 for low impact bes cyber systems

CIP-003-6 for Low Impact BES Cyber Systems

Dave Cerasoli Senior CIP Auditor

[email protected]

Marie Kozub Senior Compliance Analyst

[email protected]

May 11, 2016

mailto:[email protected]


CIP-003-6 Applicability

BA, DP, GO, GOP, IA, RC, TO, TOP

See Implementation Plan for Enforceable

Dates of the Requirements and Attachment 1 Sections

5/4/2016 2

CIP-003-6 PURPOSE

To specify consistent and sustainable security management controls that establish responsibility and accountability to protect BES Cyber Systems (BCS) against compromise that could lead to misoperation or instability in the Bulk Electric System (BES).

5/4/2016 3

CIP-003-6 R1

R1. Each Responsible Entity, shall review and obtain CIP Senior Manager approval at least once every 15 calendar months for one or more documented cyber security policies that collectively address the following topics:

5/4/2016 4

CIP-003-6 R1.2 1.2 For its assets identified in CIP-002 containing low impact BES Cyber Systems, if any: 1.2.1. Cyber security awareness; 1.2.2. Physical security controls; 1.2.3. Electronic access controls for Low Impact External Routable Connectivity (LERC) and Dial- up Connectivity; and 1.2.4. Cyber Security Incident Response.

5/4/2016 5

CIP-003-6 R1.2 NOTE: use of common programs and procedures are permitted for High, Medium and Low Impact BES Cyber Systems and should be noted when explaining to auditors.

5/4/2016 6

CIP-003-6 R1.2 Considerations for Audit

• Policy documents • Revision History that reflects review and

approval of each cyber security policy at least once every 15 calendar months

• Records of Review, e.g., emails, meeting minutes

• Workflow evidence from a document management system

5/4/2016 7

CIP-003-6 R2 R2. Each Responsible Entity with at least one asset identified in CIP-002 containing low impact BES Cyber Systems shall implement one or more documented cyber security plan(s) for its low impact BES Cyber Systems that include the sections in Attachment 1.

5/4/2016 8

CIP-003-6 R2

Note: An inventory, list, or discrete identification of low impact BES Cyber Systems or their BES Cyber Assets is not required. Lists of authorized users are not required.

5/4/2016 9

Low Impact BES Cyber Systems HEADS UP!!!

Although an inventory, list, or discrete identification of low impact BCS or their BES Cyber Assets is not required… A list containing the name of “each asset that contains a

low impact BES Cyber System” is required, such as a list of: • Generating plants • Transmission stations • Certain distribution stations • Certain “small” control centers that contain low impact BCS • Blackstart resources and cranking paths

5/4/2016 10

CIP-003-6 R2 Attachment 1 4 Focus Areas for Lows

• Section 1. Cyber Security Awareness – Reinforce, at least every 15 calendar months, cyber security

practices • Section 2. Physical Security Controls

– Control of physical access based on need • Section 3. Electronic Access Controls

– Permit only necessary inbound and outbound bi-directional routable protocol access

– Authentication for all Dial-up Connectivity • Section 4. Cyber Security Incident Response

– Requires 6 elements (of the 9 required for Medium )from CIP-008-5

5/4/2016 11

Attachment 1 – Section 1

Section 1 – Cyber Security Awareness – Shall reinforce cyber security practices at least

every 15 months – May include physical security practices

5/4/2016 12

Attachment 1 – Section 2

Section 2 – Physical Security Controls

Shall control physical access, based on need, to: • The low impact BES Cyber Systems within the asset • The Low Impact BES Cyber Systems Electronic Access

Points (LEAPs), if any.

5/4/2016 13

Attachment 1 – Section 3 Section 3 – Electronic Access Controls 3.1 For Low Impact External Routable Connectivity

(LERC), if any, implement a LEAP (Low Impact Electronic Access Point) to permit only necessary inbound and outbound bi-directional routable protocol access 3.2 Implement authentication for all Dial-up

Connectivity, if any, that provides access to low impact BES Cyber Systems, per Asset capability

5/4/2016 14

Attachment 1 – Section 4 Section 4 – Cyber Security Incident Response Plan(s) 4.2 Determination of whether an identified Cyber

Security Incident is a Reportable Cyber Security Incident and subsequent notification to the Electricity Sector Information Sharing and Analysis Center (E-ISAC), unless prohibited by law; 4.3 Identification of the roles and responsibilities for

Cyber Security Incident response by groups or individuals; 4.1 Identification, Classification and Response to a

Cyber Security Incident 4.4 Incident handling for Cyber Security Incidents;

5/4/2016 15

Attachment 1 – Section 4 Section 4 – Cyber Security Incident Response plan(s) 4.5 Testing the Cyber Security Incident response

plan(s) at least once every 36 calendar months by: (1) responding to an actual Reportable Cyber Security Incident; (2) using a drill or tabletop exercise of a Reportable Cyber Security Incident; or (3) using an operational exercise of a Reportable Cyber Security Incident.

5/4/2016 16

Attachment 1 – Section 4 Section 4 – Cyber Security Incident Response plan(s) 4.6 Updating the Cyber Security Incident response

plan(s), if needed, within 180 calendar days after completion of a Cyber Security Incident response plan(s) test or actual Reportable Cyber Security Incident.

5/4/2016 17

CIP-003-6 R2

Considerations for Audit • Provide copies of a documented cyber security

plan(s) that collectively addresses each of the four Sections in Attachment 1.

• Have available for review - dated electronic or physical records accurately demonstrating that the cyber security plans were reviewed, implemented and followed.

5/4/2016 18

CIP-003-6 R2 Considerations for Audit

Lists of personnel with access to low impact BES Cyber Systems are not required, however, the responsible entity’s plan should identify and demonstrate: – Process for determining which personnel have a

“need” to access the low impact BES Cyber Systems. – How the electronic security protections and physical

protections are implemented to ensure that access is restricted only to those personnel that have a “need”.

– That personnel have completed required training and had access to the security awareness materials.

5/4/2016 19


Entities must demonstrate that low impact BCS locations have been afforded electronic and physical protections, and are included in recovery plans. The following may be beneficial towards demonstrating compliance: – Maintaining lists of BES Cyber Asset / BES Cyber Systems

(while not required) may assist in ensuring that all low impact BES Cyber Systems are afforded proper protections.

– Station, plant, or Control Center drawings showing all Cyber Assets at the location,

– Drawings showing computer network paths through identified LEAPS, and

– Drawings of physical locations showing required physical access controls.

5/4/2016 20

A Final Note for R2 CIP-002-5: Requires a list containing

the name of “each asset that contains a low impact BES Cyber System”, such as:

• Generating plants • Transmission stations • Certain distribution stations • Blackstart resources and cranking paths

5/4/2016 21

CIP-003-6 R3

R3. Each Responsible Entity shall identify a CIP Senior Manager by name and document any change within 30 calendar days of the change.

5/4/2016 22

CIP-003-6 R3

Considerations for Audit • Must provide documentation that specifically

designates someone by name. – email, memo, letter, company bulletin, etc.

• Documentation for changes should reflect date when the current person is stepping down and when the replacement becomes effective . 5/4/2016 23

CIP-003-6 R4 R4. The Responsible Entity shall implement a documented process to delegate authority, unless no delegations are used. Where allowed by the CIP Standards, the CIP Senior Manager may delegate authority for specific actions to a delegate or delegates. These delegations shall be documented, including the name or title of the delegate, the specific actions delegated, and the date of the delegation; approved by the CIP Senior Manager; and updated within 30 days of any change to the delegation. Delegation changes do not need to be reinstated with a change to the delegator.

5/4/2016 24


• Must provide a documented process for delegation of responsibility or actions.

• Must include: – delegate identified by name – identify the specific actions for which they are

responsible – be approved by the CIP Senior Manager and dated.

• Changes must be documented within 30 days.

5/4/2016 25

Suggestions • Insist on early participation from SMEs, plant,

and field personnel from both the IT and the Operations areas to promote a better understanding of the assets and their functions, resulting in a more collaborative and effective program of protection.

• Schedule weekly team meetings to maintain focus and keep the project moving forward.

5/4/2016 26

Suggestions, Cont. • Use existing policies as a foundation and expand

on them. • Ensure that Policies/Procedures accurately reflect

what you do. • Start early, anticipate events that may potentially

impact the project. • Build in extra time in the project timeline

providing for testing and feedback, budget cycles, and unplanned contingencies.

5/4/2016 27

Suggestions, Cont. • The CIP Standards are generic by design,

allowing more flexibility. Be cautious of the appearance of oversimplified requirements.

• Continually review the Standards and Requirements and ensure the teams have addressed all of the documentation/testing requirements.

5/4/2016 28

Summary • Compliance is a continuous process, not a one time

thing. Look for ways to exceed the Requirements. • Ensure you identify each asset that contains a low

impact BES Cyber System. • Ensure you implement all sections of CIP-003-6

Attachment 1 for each asset that contains a low impact BES Cyber System.

• Provide complete and appropriate evidence to support all sections of CIP-003-6 Attachment 1.

• Provide detailed narratives describing how you meet compliance for each requirement.

5/4/2016 29

Low Impact BES Cyber Systems

• Must demonstrate that low impact BCS locations have been afforded electronic and physical protections, and are included in recovery plans

• DON’T have to identify a discrete list of low impact BCS

• DO have to demonstrate compliance with CIP-003-6 R2 for each low impact BCS – A list of low impact BCS at each asset may be helpful

5/4/2016 30

??? QUESTIONS???

5/4/2016 31

/ ©2016 NAVIGANT CONSULTING, INC. ALL RIGHTS RESERVED1

BRIAN HARRELL, CPPDIRECTOR - RISK MANAGEMENT, COMPLIANCE, AND SECURITY

PETER SCALICI, CHPPNPCC-SENIOR CIP SPECIALIST

PHYSICAL SECURITY TRENDS: A CIP-014 PERSPECTIVE

NPCC SPRING WORKSHOP-COOPERSTOWN, NY MAY 11, 2016NPNpcc Spring

Workshop


IT’S AN ISSUE OF MAGNITUDE


NORTH AMERICAN INTERCONNECTED GRID

• The greatest engineering achievement of the 20th Century –US National Academy of Engineering

• Serves Canada, the US, and Baja California Norte, Mexico

• Demand of 830,000 megawatts• 211,000 miles of high-voltage

transmission lines• Total assets over one trillion dollars• The one critical infrastructure sector

that makes all the other critical infrastructure sectors possible

• The value of the grid to our quality of life is priceless

Integrated Transmission Grid, showing lines of 345kv or greater


Over 55,000 Substations over 100 Kv in size!


SECURING A REMOTE OR URBANASSET!

The Real Challenge…


CRITICAL TRANSFORMERS


NERC CIP-014 PHYSICAL SECURITY


CIP-014 PHYSICAL SECURITY STANDARD

• Purpose:- To identify and protect transmission stations and

transmission substations, their associated primary control centers, that if rendered inoperable or damaged as a result of physical attack could result in widespread instability, uncontrolled separation, or cascading within an interconnection.

• Applicability:- Transmission Owners (TO)- Transmission Operators (TOP)


KEY DATES

CIP-014-2 Implementation Timeline

Activity Implementation Not Later Than Days after 10/1/15

R1 Assessment Effective Date 10/1/2015 0 days

R2 Verification Effective + 90 12/30/2015 90 days

R2.3 Address Discrepancies R2.2 + 60 2/28/2016 150 days

R3 Notify Control Center R2 +7 1/6/2016 97 days

R4 Threat & Vulnerability Evaluation R2 + 120 6/27/2016 270 days

R5 Security Plan R2 + 120 6/27/2016 270 days


• Critical facility identification (R1) complete before effective date (six months following publication in the Federal Registry) Standard approved November 20, 2014 Mandatory and Enforceable October 1, 2015

• Third party verification (R2) complete within 90 days of completion of R1.• Notification of other parties (R3) complete within 7 days of completion of

R2.• Evaluate threats and vulnerabilities (R4) and develop security plans (R5)

within 120 days of completion of Requirement R2.• Third party review of threats and vulnerabilities and security plans (R6)

within 90 days of completion of R4/R5.

IMPLEMENTATION


REQUIREMENT 4• Advising utilities without a robust security departments to use the

NATF CIP-014 R4 Guideline• Provide all documentation of prior attacks, break-ins, sabotage

incidents• OE-417, RCIS, E-ISAC reports, LEO Reports

• Seek outside threat information:• E-ISAC• Fusion Center• Local, State, and Federal Law Enforcement• Has DHS identified you as “critical”?

• Design Basis Threat (DBT)• Accurate scenarios for potential attack

• Outside firearms attack• “Suspicious” device left behind or thrown over perimeter fence• Vehicle-borne Improvised Explosive Device• Breached control/station house


REQUIREMENT 5• Physical Security Plan (R5) should map directly to T&V

Assessment (R4)• Are you hardening the entire facility or specific critical assets and

infrastructure within the facility?• Discuss security measures designed to deter, detect, delay,

assess, communicate, and respond to potential physical threats• What is the response time of security staff? Local Law

Enforcement? • If you commit to paper, you are now obligated• Highlight mitigation measures that have been put in place as a

result of the attack scenarios• Remove line of sight to critical transformers• Suspicious package procedures and response plan• How are you slowing, checking, screening, and controlling

access to your facility?• How are you monitoring the station house? Patrols? Procedures

for reporting a breach? Cyber!!


PHYSICAL-TECHNOLOGY INTEGRATION

Site Specific Layered Approaches To:

Deter potential adversaries from considering the facilities in their pre-operational planning

Detect adversaries in their planning, surveillance, or approach stages

Delay adversaries from gaining access to critical facilities and equipment

Minimize the impact of any intrusions or attacks on BPS reliability

Rapidly respond to any attacks or intrusions Preserve and assist law enforcement in evidence

recovery for potential apprehension


DETERRENCE

Current systems and technologies used by industry security professionals:

• Motion activated video surveillance with intrusion deterrence technologies • Limited access smart locks and access card systems/readers• Employee screening (insider threat)• Security fencing to include solutions with blast and ballistic resistance• Environmental and physical vehicle barriers• Security lighting to include motion activated strobe illumination• Security signage• Prohibit non-critical storage and staging to reduce criminal draw• Annual security program and vulnerability assessment reviews • Security guards• Neighbor awareness security program


DETECTION

• External/internal video analytic systems• External/internal motion sensing systems• External seismic detection systems• External/internal gunshot detection systems• UAV detection systems


DELAY

• Environmental barriers• Access barriers• Perimeter fencing


REQUIREMENT 6• Consultant? DHS? Law Enforcement?

• Many government agencies do not want to sign off on “compliance”

• Avoid overnight firms that have recently boarded the “CIP-014 train”

• Use a firm with proven experience in the electricity sector ANDdoing physical security

• Client expectations:• Proposal response• Initial kick-off meeting to discuss timeline (R2, R4/R5 Dates?)

and milestones• Time to review R4 and R5 documentation ahead of site visits• On-site review of all CIP-014 sites to verify identified

vulnerabilities and mitigation measures• Informal exit presentation• Compliance documentation


REQUIREMENT 6 CONTINUED…

• Final document:• Identified facility• Unaffiliated 3rd Party statement• Statement suggesting that security measures “mitigate the

threats identified in the R4 threat assessment” • A determination that “the physical security plan is achievable” • Provide security suggestions, if applicable• Provide evidence of consultants expertise and certification(s)

▫ Do not tell me, show me• Official company letterhead with full contact details


BRIAN HARRELL, [email protected]

PETER SCALICI, CHPPNPCC-Senior CIP Specialist212 [email protected]

CONTACT

CIP Update

May 11, 2016

Agenda 1. CIPv6 Outreach Program 2. CIP-002-5.1 and CIP-014-2 Self Certifications 3. CIP-014 Outreach Program Update

CIPv6 Outreach Program

David Cerasoli, CISSP, C|EH Senior CIP Auditor

We will hold 10 outreach sessions this year. The focus of the sessions will be on CIP-003-6 Attachment 1.

Participation is limited to registered entities that only have Low Impact BES Cyber Systems. Participants will be asked to provide a list of questions for discussion during the session.

The duration of your session will depend on the number and complexity of your questions. Each session will be held via WebEx, but we are willing to visit your facility if necessary. Participants will be selected on a first-come, first-served basis. Please contact me ASAP if you are interested in participating in an outreach session.

[email protected]


CIP-002-5.1 and CIP-014-2 Self Certifications

CIP-014-2 Self Certification • Was due May 2, 2016 • Applicable to all TOs • Submitted via the CDAA with an attachment

CIP-002-5.1 Self Certification • Due July 15, 2016 • Applicable to BA, DP, GO, GOP, IC/IA, RC, TO

and TOP • Will be posted by June 15, 2016 • Submitted via the CDAA with an attachment

CIP-014 Outreach Program

Peter Scalici, CHPP Senior CIP Specialist

The Program-Overview In 2015 seven CIP-014 assessments were completed for entities that volunteered and have facilities that are applicable under the CIP-014 applicability section. For each assessment NPCC Staff went onsite and met with SMEs, reviewed policies, procedures and other documentation. After walkthroughs of the subject sites we discussed current and potential issues with SMEs.

Program Overview-cont’d During the tours, NPCC Staff made note of unique characteristics of the sites, possible mitigation of issues associated with those unique characteristics, what the entity had done to date, and what they planned to do in the future, if anything.

CIP-014 Sites Revisited In 2016 NPCC is revisiting the sites to observe the progress those entities have made in their implementation of upgrades, if necessary, to come into compliance with the requirements of CIP-014. To date, three sites have been revisited and the progress observed has been excellent. It appears as if the entities observed have good management support and have considered the latest technologies when planning and implementing updates to their physical security.

CIP-014 Sites Revisited It was observed that the entities had considered such unique characteristics as: • Urban vs rural locations • Surrounding infrastructure (airports, highways,

residential homes, etc.) • Community (economic, socio-political, people) • Cost estimates and budgets • Size (acreage) of the site

14

David Cerasoli, CISSP, C|EH

[email protected]

Daniel Grinkevich, CISSP, C|EH [email protected]

Peter Scalici, CHPP [email protected]




Questions?

Transient Cyber Asset and Removable MediaCIP-010-2 Attachment 1 & 2

Michael Bilheimer

Disclaimer

• UI has a compact territory where all substations can be reached in a short time period. UI has internal SME’s performing the majority of the work.

• Larger utilities, with widespread territories, or utilities who rely on vendors or contractors may need to deploy different controls.

UI Approach

1. Understand the TCA Requirements2. Develop a process to perform the

requirement3. Deploy solution and educate users

TCA/RM Summary Requirements

• Applicability High/Medium BES Cyber System and PCA.

• CIP-010-2 R4 Appendix 1&2 Transient Cyber Asset (TCA) and Removable Media (RM) – Authorization and Management– Software Vulnerability Mitigation/Malicious Code

Mitigation– Unauthorized Use – Vendor Control

• Implementation - 4/1/2017

Security Controls

• Write a Procedure• Each TCA is in UI NERC CIP Database• Two user groups authorized and recorded in Database:

– Relay Test Techs share a group of TCA’s.

– SCADA and IT/OT departments have TCA’s assigned to individual users.

• Special accounts are created to authenticate to UI SCADA environment.

– Direct relay connection do not authenticate but the laptop will authenticate user access and a separate password is required to access relays.

• Only authorized TCA’s are assigned IP addresses upon connection to the environment.

• Software Vulnerability (Patching)• Malicious Code Prevention• Physical Security when connected

Vendor TCA Control• UI only allow Vendor TCA (TCA-V) to connect to High and Medium

impact BES Cyber Systems if the Vendor TCA has proprietary software that cannot be loaded onto a UI TCA and is required to perform a necessary function.

• Prior to TCA-V connection to a UI BES Cyber System or PCA:– Authorization: TCA is authorized by the UI designated approver or assigned

delegate.– Anti-Virus: Vendor provide evidence that TCA-V is running anti-virus program

that has the most current signatures.– Patching: Vendor provides evidence that the TCA-V operating system, firmware

and applications have updated security patches installed.– Accounts: Vendor will provide evidence that the Vendor TCA enabled user

access controls.– Mitigation: Vendor will mitigate any deficiencies identified by UI prior to

connection. – Change Management Request: a UI Requestor will submit a Change

Management Request. – Prior to connection: Verify anti-virus signatures are up-to-date and Scan the

TCA-V with installed anti-virus software (Screenshot) .

UI Removable Media• UI only uses Removable Media distributed by the IT

Department. – Scan on a UI Corporate Laptop prior to connection.

• Vendor Removable Media is not allowed. – Transfer file to a UI Corporate machine. – Scan transferred file on UI Corporate machine. – Move files to SFTP server, UI TCA or UI RM.

More than just Cyber

Presenter: John Helme

TFIST Chairman [email protected]

5/4/2016 1

Recent Attacks

5/4/2016 2

E-ISAC Blog

5/4/2016 3 http://www.techinsider.io/redteam-hackers-power-grid-company-2016-4

Ukraine

5/4/2016 4

Ukraine • December 2015, in

Ukraine • 4 Distribution

organizations attacked • 3 had operational impact. • Estimated 225,000

customers lost power.

How did it Happen From the NERC Alert - Mitigating Adversarial Manipulation of Industrial Control Systems as

Evidenced By Recent International Events • Phase One – Reconnaissance • Phase Two – Delivery • Phase Three – Exploitation • Phase Four – Installation • Phase Five – Command and Control • Phase Six - Action

5/4/2016 5

More than just Cyber

• Phase One – Reconnaissance • Phase Two – Delivery • Phase Three – Exploitation • Phase Four – Installation • Phase Five – Command and Control

• Phase Six - Action

5/4/2016 6

Reconnaissance

5/4/2016 7

Delivery Spear Phishing

5/4/2016 8

5/4/2016 9

$4.1 Million inheritance/wining

“jailable offense under section 12 subsection 441 of the ? Tax Code. ” “Failure to comply may lead to your arrested, interrogation and/or you being prosecuted in the Court of Law “

Phishing Attack per Emails

5/4/2016 10

Symantec Internet Security Threat Report, March 2016

https://www.symantec.com/security_response/publications/monthlythreatreport.jsp

https://www.symantec.com/security_response/publications/monthlythreatreport.jsp

Removable Media

5/4/2016 11

Don’t Forget

5/4/2016 12

Action

5/4/2016 13

Protection By the CIP Standards

5/4/2016 14

Control System Business System

Cyber Security Awareness

5/4/2016 15

Cyber Security Incident Response

5/4/2016 16

TFIST Whitepaper • Based on the

recommendations of the SANS/E-ISAC, Analysis of the Cyber Attack on the Ukrainian Power Grid. (March 18, 2016)

• “Good Practice” for each recommendation

5/4/2016 17

Questions

“I was wrong to put my faith in the league. I've come to the conclusion this was never about doing what was fair and just.“

Robert Kraft

5/4/2016 18

Spring NPCC Compliance and Standards Workshop

May 10-12, 2016

NPCC Criteria Services UpdateGerry Dunbar

Manager, Reliability Criteria

5/5/2016 1

Outline• Background

• NPCC Regional Criteria

• Directory Project

• ERO Standards

• Current Reviews

5/5/2016 2

Background• History

– ‘A’ Criteria Documents– Members Develop and ‘own’ Criteria

• Criteria= Who, What and Why?– NPCC Full Members (By-Laws)– Other Agreements – More Stringent Criteria---A-10 Methodology

• NPCC BPS– Promote Reliability

• Augment ERO Standards

5/5/2016 3

NPCC Regional Criteria• NPCC Task Forces and Working Groups Develop• Post for Comments• Committee Review and Approval• NPCC Full Members Ballot Approval• NPCC Full Members Obligated to Comply

– CCEP • Criteria in Other NERC Regions

5/5/2016 4

NPCC Directories• NPCC Directory Project

– Consolidate ‘A’ Criteria with Functionally Related ‘B’ Guidelines and ‘C’ Procedures.

– 10 Directories– NPCC Directory and Revision Manual

• Criteria Revisions/Retirements• Criteria Clarifications

5/5/2016 5

ERO Standards• NERC Oversight of NPCC

– CMEP– Registration– Business Plan & Budgets

• Standard Development Process– Rules of Procedure (ROP)– Criteria ‘Not Inconsistent With’ Standards

• Criteria Revised or Retired As Standards Evolve

5/5/2016 6

Current Directory (Criteria) Reviews• Directory#8 System Restoration

– PRC-005 Protection Maintenance – Battery Testing

• Directories #9 and #10 Verification Criteria– MOD-25-2 – NPCC TOP Verification Program

• Directory #11 Disturbance Monitoring– Retirement of PRC-002-NPCC-1

• A-10 Classification of Bulk Power System Elements – Directory#1 Review

• Directory#7 Special Protection Systems– PRC -012-2 RAS

• Other B and C Documents

5/5/2016 7

5/5/2016 8

Questions or Comments ?

Misoperation Information Data Analysis System (MIDAS)

NPCC Compliance & Standard Workshop Cooperstown, NY, Spring 2016

Rafael Sahiholamal Senior RAPA Engineer

1

M: Misoperation I: Information D: Data A: Analysis S: System

3

4

• Background • Current Process • Future Process

Background

5

PRC-004 Analysis and Mitigation of Transmission and Generation

Protection System Misoperations

6

Current Process

7

https://cdaa.npcc.org/

8

PRC-004-2.1(i)a

9

Requirements R1. The Transmission Owner and any Distribution Provider that owns

a transmission Protection System shall each analyze its transmission Protection System Misoperations and shall develop and implement a Corrective Action Plan to avoid future Misoperations of a similar nature according to the Regional Entity’s procedures. R2. The Generator Owner shall analyze its generator and generator interconnection Facility Protection System Misoperations, and shall develop and implement a Corrective Action Plan to avoid future Misoperations of a similar nature according to the Regional Entity’s procedures. R3. The Transmission Owner, any Distribution Provider that owns a transmission Protection System, and the Generator Owner shall each provide to its Regional Entity, documentation of its Misoperations analyses and Corrective Action Plans according to the Regional Entity’s procedures. 10

NPCC Document C-45

Procedure for Analysis and Reporting of

Protection System Misoperations

https://www.npcc.org/Standards/Procedures/C-45.pdf

11






System Protection Working Group (SP-7)

• Review the analysis of misoperations of protection systems on the bulk electric system including SPS

• Maintain a record of all reviewed misoperations • Calculate statistic of protection system misoperations • Work with the NPCC Event Analysis Team • Share lessons learned with Members and industry from

review of misoperations • Recommend NPCC additions to the NERC reporting

template 12

NPCC Protection System Misoperations by Cause

13

NPCC Protection System Misoperation Rate (%)

14

Misoperation Rate by Region

15

Future Process

16

PRC-004-2.1(i)a WILL BE INACTIVE ON JUNE 30, 2016

17

Misoperations Standard Drafting Team (SDT)

The Protection System Misoperations Standard Drafting Team (SDT) in Project 2010-05.1 has removed the data reporting obligation included in Reliability Standard PRC-004-2.1a from the revised standard and recommended that NERC request the data required for performance analysis purposes pursuant to Section 1600 of the NERC Rules of Procedure.

18

Section 1600

19

In accordance with Section 1600 of the NERC Rules of Procedure, NERC may request data or information (“Data Request”) necessary in order to meet its obligations under Section 215 of the Federal Power Act, as authorized by Section 39.2(d)2 of the Federal Energy Regulatory Commission’s (“FERC” or “Commission”) regulations.

PRC-004-4(i)

20

The SAR for this project also included clarifying reporting requirements. Misoperation data, as currently collected and reported, is not optimal to establish consistent metrics for measuring Protection System performance. As such, the data reporting obligation for this standard is being removed and is being developed under the NERC Rules of Procedure, Section 1600 – Request for Data or Information (“data request”). The standard and data request have been developed in a manner such that evidence used for compliance with the standard and data request are intended to be independent of each other.

21

The purpose of this Data Request • Develop meaningful metrics to assess Protection

System performance • Identify trends in Protection System performance

that negatively impact reliability • Identify remediation techniques to reduce the rate

of occurrence and severity of Misoperations • Provide focused assistance to entities in need of

guidance; and • Publicize lessons learned to the industry.

22

MIDAS

23

The MIDAS project was created to enable consistent and reliable submission, processing, and reporting of misoperation data from Registered Entities to ERO Enterprise via a centralized system and standardized template for performance analysis. The FERC-approved Revised Reliability Standard PRC-004-4 requires retention of this data or evidence of compliance with the standard, but no longer requires periodic reporting of that information. Periodic, quarterly submittals of misoperation data will be associated with reporting under the Section 1600 Data Request approved by the NERC Board of Trustees (Board) in August 2014.

Operation Tab

25

Misoperation Tab

26

Optional Fields

27

MIDAS Video Tutorials

28

https://vimeopro.com/nerclearning/midas-video-library



• TO, GO, and DP* • PRC-004-2.1(i)a Inactive on 6/30/2016 • PRC-004-4(i) Subjects to Enforcement on 7/1/ 2016 • Misoperation/Operation data reporting NERC Rules of

Procedure, Section 1600 . (“data request”). • MIDAS starts 2nd Quarter 2016 • Submittal timeframe 60 Days after each quarter • First submittal due date August 31, 2016

29

Questions?

[email protected] 212-205-7064

30









Evolution of the Ontario Supply Mix

Leonard Kula P.Eng.

Director, Power System Assessments

Independent Electricity System Operator

NPCC Compliance and Standards Workshop

May 11 2016

Transforming Ontario’s Power System

2

0

2,000

4,000

6,000

8,000

10,000

12,000

0

2,000

4,000

6,000

8,000

10,000

12,000

2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015

Ne

w C

apac

ity

(MW

) A

dd

ed

Co

al C

apac

ity

(MW

) D

ecl

ine

Nuclear Oil/Gas/Steam Renewables Bio Energy Coal

1

2

3

4 5

6

7

1 - April 30, 2005: Lakeview Retirement

2 - October 1, 2010: Lambton G1, G2 and Nanticoke G3, G4 shutdowns

3 - December 31, 2013: Nanticoke G1, G2 shutdown

4 - September 11, 2012: Atikokan Shutdown

5 - September 26, 2013: Lambton shutdown

6 - December 31, 2013: Nanticoke shutdown

7 - April 1, 2014: Thunder bay shutdown

Emissions Management

3

0

50,000

100,000

150,000

200,000

250,000

2004 2005 2006 2007 2008 2009 2010 2011 2012 2013

To

tal

Em

issi

on

s (T

on

nes

)

Year Coal Gas Other

Source: Environment Canada

Emissions: SOx NOx CO TPM VOC

Ontario Electric Power Utilities Emission Totals (2004 – 2013)

• Maintaining reliability and adequacy

• Preserving operational flexibility

• Emissions control strategy

• Communicating progress and requirements

Key Elements of the Coal Replacement Plan

4

• Supply adequacy

– Replacement supply required to demonstrate reliable

performance before retiring coal units

• Location of replacement supply

– Need for reactive power support in critical locations to

maintain adequate voltages

– Maintain existing import and export capability

• Transmission enhancements

– Need for extensive changes and realignment of the

transmission infrastructure

Maintaining Reliability and Adequacy

5

6

10,882

7,564

4,976

7,676

66

2005 Q1 Installed Capacity

Nuclear

Coal

Oil/Gas

Hydro

Other

12,947

9,920

8,462

2,543 495

2015 Q1 Installed Capacity

Nuclear

Oil/Gas

Hydro

Wind

Other

Supply Mix – Then and Now

Wind/Solar Outlook

7

8

Grid/Transmission Enhancements

• Ensuring replacement supply has appropriate

governor response for frequency stabilization

• New system configuration after coal shutdowns

and new resources created the need for

significant transmission upgrades:

– Auto-transformers

– Capacitors

– Switching stations

– Static VAR compensators (SVCs)

– Bruce to Milton 500kV circuits

9

Preserving Operational Flexibility

• Ontario Energy Board`s Natural Gas

Electricity Interface Review

• Enhanced focus on gas-electric

coordination

• Studied operational characteristics

and established requirements for

replacement supply

– Connection requirements

– Performance validation

– Renewables Integration Initiative (RII)

• Public reporting on important aspects of the off-coal

transition

Communicating Progress and Requirements

10

Ontario Demand vs Potential Baseload Supply

11

0

5000

10000

15000

20000

25000

MW

Hours

Wind

Hydro

Nuclear

Ontario Demand

• The coal phase-out program created the need for

additional resources, transmission upgrades, new

operating methods, and collaboration with stakeholders

• Through planning and operability studies, coal was

successfully and reliably replaced by gas, nuclear

refurbishments, and renewable resources

• By April 2014, the coal shutdown was complete making

Ontario the first jurisdiction in North America with a

significant reliance on coal-fired electricity to eliminate

coal as a source of electricity production

Summary

12

Internal Controls Evaluation (ICE) Update

5/11/2016 1

NPCC Compliance WorkshopMay 11, 2016

• Where are the Hall of Fames for these 10 sports– Football (Pro)– Basketball– NASCAR– Hockey– Golf– Soccer– Tennis– Swimming– Track and Field– Sailing

5/11/2016 2

Equal Time for Other Sports

• Where are the Hall of Fames for– Football (Canton Ohio)– Basketball (Springfield MA)– NASCAR (Charlotte NC)– Hockey (Toronto Ontario; Eveleth MN – USA) – Golf (St. Augustine FL)– Soccer (Pachuca, Hidalgo, Mexico; Oneonta NY*) – Tennis (Newport RI)– Swimming (Ft. Lauderdale FL)– Track and Field (Washington Heights NY)– Sailing (Annapolis MD)* closed

5/11/2016 3

Equal Time for Other Sports

• What are the rankings for most popular watched sports? Pick the top 5 in order– Soccer– Basketball– Football– Baseball – NASCAR– Hockey– Golf– Tennis– Swimming– Track and Field– Sailing

5/11/2016 4

Is Baseball the most watched sport in the US?

• What are the rankings for most popular watched sports? Pick the top 5 in order– Soccer– Basketball (5 – 6%)– Football (1 – 35%) (3 – 11% College)– Baseball (2 – 14%)– NASCAR (4 – 7%)– Hockey– Golf– Tennis– Swimming– Track and Field– Sailing

5/11/2016 5

Is Baseball the most watched sport in the US?

• Status of Internal Controls Evaluations• Benefits of ICE• Lessons Learned• Appendix – references, examples

5/11/2016 2

Topics

Intro and Background

NERC Risk‐based Compliance Oversight Framework • Allows each Registered Entity optional participation in ICE assessment by Regional Entity (NPCC).

Internal Controls Evaluation (ICE)• refines scope, monitoring methods and frequency based on assessment of the organization’s Internal Controls and Internal Control Designs (ICD) implementation

• used to support and inform– ERO CMEP Implementation Plan– NPCC CMEP Implementation Plan (Appendix A3)

5/11/2016 3

5/11/2016 4

NPCC ERA “Filtration/Expansion” Diagram(Compliance Monitoring Lifecycle)

IRA results and Scoping are provided in the IRA Summary Report to the entity. Section 4 describes the pre‐ICE audit scope.

If entity participates in ICE, then post‐ICE proposed scope of audit and alternate monitoring methods are provided in the ICE Summary Report to the entity.

We are here

5/11/2016 5

Compliance Monitoring Lifecycle

ICE Metrics

5/11/2016 6

ICE Objectives• Primary objective:

– Recognize internal controls utilized by the Entity to manage and mitigate reliability risks.

• Other objectives of the ICE review :– Leverage ERO Enterprise Inherent Risk Assessment Framework;

– Align Regional Entity & registered entity compliance resources with BES reliability risk‐focus areas;

– Identify and assess entity’s risk mitigation and internal controls design;

– Obtain reasonable assurance that the entity internal controls and internal control designs are in place and operating effectively

– Develop Entity Compliance Oversight Plan including Compliance monitoring methods, scope & frequency.5/11/2016 7

Benefits of ICEExpected benefits derived from ICE review participation typically include the following:

– Attainment of BES reliability, Corporate Goals and Objectives;– Alignment of staff performance to Key Performance Indicators;– Improved operational performance (i.e., exceeding standards and

requirements); – Enhanced entity communication and interaction across organizational

business functions;– Targeted BES reliability risk‐focused scoping;– Reduction in audit duration;– Improved risk and control awareness;

• Internal Control Design evaluation including:– Functional and Business Process Assessment;– Risk Identification, Mitigation & Remediation;– Design and Gap Analysis– Non‐binding Recommendations for Internal Control Design Enhancement.

• ICE Training and Outreach.5/11/2016 8

Methodology• ICE review process to assess Entity ICD’s and related risk mitigation• Developed multiple worksheet types to facilitate Entity preferences and

review flexibility– Type 1 ICE Worksheet:

• Depicts internal controls on a requirement‐by‐requirement basis. – Type 3 ICE Worksheet:

• Depicts internal controls based upon Functional Businesses and/or Control Processes.

• Covers multitude of standards and requirements.• Aligns with organizational business functions and processes.• NPCC’s preferred approach• See Appendix for examples

• Pre‐ICE WebEx for every ICE participant• Frequent communication encouraged5/11/2016 9

Lessons LearnedAll key Internal Controls have not been identified and documented.

– Automation tools often overlooked, taken for granted (hence not taking credit for them)

• Work management systems • Document management tools• Compliance management tools• Inventory/Maintenance/Vegetation Management tools• Checklists, logs

– Tribal knowledge (“oh everybody knows that. I didn’t think I had to list it”)

– Existing processes (documented or undocumented) are not identified in ICE and therefore not credited.

5/11/2016 10

Lessons Learned

5/11/2016 11

Flowcharts really really help. Really!

– Flowcharts are recognized as a best practice– Facilitates comprehensive gap analysis of internal controls design

– Functional Responsibilities/Departments• inputs/outputs/handoffs/feedback loops.• two way “traffic” documentation (e.g. emails, meeting agendas/notes/lessons learned, regular meetings, logs

• Identify “control silo” components• “map” to the ICE worksheets

Type 1 ‐ ICE Example (Cont’d)• Flow Diagram

5/11/2016 12

NPCC Control Design Assessment and Comments

5/11/2016 13

Annual Blackstart Restoration Plan

Lessons Learned

5/11/2016 14

Submittals not always quality checked prior to submittal to NPCC ICE Team

– Mapping of flow diagrams to ICE worksheets– Additional undocumented controls (discovered during ICE evaluation walkthrough)

– Conduct a peer check of your SME’s flow diagram .– Conduct Independent check of flow diagram to ICE worksheet– Crosscheck the “who, what, when, where, why, how” an Internal Control is

implemented– Identify and verify all feedback loops, inter‐departmental inputs, outputs

and handoffs– Perform a mock walkthrough of ICE before NPCC ICE team review– If you’ve been “ICE’d” before, use that experience for future ICE.

Lessons Learned

5/11/2016 15

Audit evidence is not the same as Internal Controls evidence. SMEs need to understand the difference.

– Requirements ask for processes, procedures or programs, proof of actions, studies, calculations, communications, logs, test results to demonstrate compliance. (Maintenance, Testing, Restoration, Training, Operations, Misops, etc.)

– “Aren’t the above my internal control design? Isn’t the ICE evidence the same as the audit evidence?”

– Internal Control Silos– “Who, What, When, Where, Why, How” test– Key Internal control?

– Postulate failure mechanisms– Redundancy to avoid single points of failure

Audit

• Audit interested in pass/fail, No Finding or Possible Violation

• Once pass/fail, NF/PV is determined, there may be additional items provided to improve reliability (e.g. Areas of Concern, Recommendations, Suggestions)

• Backward looking (Audit Period)

5/11/2016 16

5/11/2016 17

OR“Please move along once you have received your inspection sticker”

Internal Controls Evaluation

• Preventative controls in place to– pass audits– exceed the requirements to improve reliability

• Detective controls to indicate degradation in reliability or identify drift from compliance

• Corrective controls confirm and mitigate non‐compliances

• Real time and forward looking

5/11/2016 18

ICE vs Audit (continued)Audits

• RSAWs are used for Audits– focused on evidence showing compliance– Some ask for brief narrative describing how the requirement was met.– Narrative does not delve into Internal Controls Design.– No credit given for exceeding the requirements to improve reliability

ICE• ICE Worksheets used for ICE

– Many columns requesting entity to describe many aspects of each component of the Internal Control Design

– Asks questions to determine who, what, when, where, why, how the internal control was implemented to ensure compliance.

– Acknowledges systematic approaches/designs to ensure/exceed reliability

5/11/2016 19

Internal Control Types

• Internal Controls ‐ Preventative [P], Detective [D] & Corrective [C]

• Institute of Internal Auditors (IIA) – recognizes Corrective controls as Good, Detective controls as Better, and Preventative controls as Best

• IIA ‐ suggests Preventative controls be bolstered with Detective and Corrective controls – to ensure Preventative control implementation and proper functioning

5/11/2016 20

Types of Controlspertaining to Kitchen Fires

Corrective Controls = Good

5/11/2016 21

Types of Controls ‐ Kitchen Fires

Detective Controls = Better

5/11/2016 22

Types of Controls ‐ Kitchen Fires

Preventative Controls = Best(Procedures, signage, training, monitoring/situational awareness, access)

5/11/2016 23

“Where there’s a will, there’s a way...”(to get around preventative controls)

5/11/2016 24

“Where there’s a will, there’s a way”

5/11/2016 25

"As my husband and I were entertaining guests in the other room, one of our dogs decided to break into the pizza box that was sitting on top of our gas stove. Her paw turned the gas ignition knobs as she tried to put her head further into the box. Luckily we had it on video and know who to blame! Lol let this serve as a gentle reminder not to place items on your stove top and also to carefully choose the placement of your fire detectors," the uploader wrote.”

5/11/2016 26

“How a NYC Traffic Cop helped me improve my Internal Control Design”

Lessons Learned

5/11/2016 27

Audit evidence is not the same as Internal Controls evidence

– Requirement says there must be a process, procedure or program in place for compliance. (Maintenance, Testing, Restoration, Training, Operations, etc.)

– “Isn’t the above my internal control design? The compliance evidence is the same as ICE evidence isn’t it?”

– Internal Control Silos– “Who, What, When, Where, Why, How” test– Key Internal control? (Failure mechanisms)– (Protection systems have redundancy, Backup Control Centers– Single point of failure for each silo (SHC uses training to the procedure to do

the right thing

Appendix

• References and additional examples(more can be found in prior presentations and on the NPCC ERA website)

5/11/2016 28

Analytical ToolsType 1 Worksheet • Navy Section from NERC Standard – filled by NPCC

5/11/2016 29

• Lavender Section – filled by Entity– Explanation of Headings provided on separate tab

Analytical Tools (Cont’d)

Type 3 Worksheet • Lavender Section – filled by Entity

– Explanation of Headings provided on separate tab

5/11/2016 30

Analytical Tools (Cont’d)Power Blue Section ‐ NPCC completes after assessment of Internal Controls Design and supporting documents

5/11/2016 31

Type 1 ‐ ICE Example (Cont’d)• NPCC Assessment:

5/11/2016 32

NPCC ERA website• More references and examples

5/11/2016 33

5/11/2016 34

QUESTIONS ?

5/11/2016 35

Thank You