nerc id name ncr id actions... · req. entity name ncr id noncompliance start date noncompliance...

ReliabilityFirst Corporation (ReliabilityFirst) FFT O&P

NERC Violation ID Reliability Standard Req. Entity Name NCR ID Noncompliance Start Date Noncompliance End Date Method of Discovery

Future Expected Mitigation Completion Date

RFC2019021406 COM‐002‐4 R3 Coldwater Board of Public Utilities NCR08012 7/1/2016 5/7/2019 Compliance Audit Completed Description of the Noncompliance (For purposes of this document, each noncompliance at issue is described as a “noncompliance,” regardless of its procedural posture and whether it was a possible, or confirmed noncompliance.)

On April 22, 2019, ReliabilityFirst determined that the entity, as a Distribution Provider, was in noncompliance with COM‐002‐4 R3. The noncompliance was identified during a Compliance Audit conducted on April 11, 2019. Specifically, ReliabilityFirst concluded that entity operating personnel had not completed the initial training on receiving oral two‐party, person‐to‐person Operating Instructions prior to receiving Operating Instructions.

The root cause of this noncompliance was a failure to realize that COM‐002‐4 R3 was applicable to the entity. This noncompliance involves the management practices of grid operations, workforce management, and reliability quality management. A basic tenet of grid operations management is providing sufficient operator training in order to support reliable operation. Workforce management involves ensuring that employees have baseline competencies to perform their compliance responsibilities, and this can be achieved, in part, through effective training. Reliability quality management should include: (a) efforts to identify and fully understand Reliability Standards and Requirements; and (b) evaluations for quality assurance of the organization’s Bulk Electric System reliability and resilience activities.

This noncompliance started on July 1, 2016, when the entity was required to comply with COM‐002‐4 R3 and ended on May 7, 2019, when entity operating personnel completed required training. Risk Assessment This noncompliance posed a minimal risk and did not pose a serious or substantial risk to the reliability of the Bulk Power System (BPS) based on the following factors. Failing to train operators on

predefined communication protocols increased the likelihood of a miscommunication that could have led to action or inaction harmful to the reliability of the BPS. The risk in this case was minimized based upon the following facts. The entity’s all‐time peak system load was 83.7 MW, and the entity’s interconnections are limited to a common 138kV bus owned and operated by ITC Interconnection, LLC (ITC). ITC is responsible for switching on the 138kV system. Based upon the foregoing, any miscommunication regarding an Operating Instruction issued to the entity would have had a minimal impact on the overall reliability and resilience of the BPS. Further, the entity receives only a couple of Operating Instructions per year on average, thereby reducing the number of opportunities for a miscommunication that would impact the BPS. Lastly, the primary operator at the entity has over thirty years of industry experience and has utilized three‐part communication over the course of his career, thereby reducing the likelihood of a miscommunication. No harm is known to have occurred.

ReliabilityFirst considered the entity’s compliance history and determined there were no relevant instances of noncompliance. Mitigation To mitigate this noncompliance, the entity:

1) trained staff on oral two‐party, person‐to‐person Operating Instructions; and2) implemented a more rigorous orientation process for new staff that would be able to receive an oral Operating Instruction.

ReliabilityFirst has verified the completion of all mitigation activity.

A-1 Public Non-CIP - Find, Fix, Track, and Report Consolidated Spreadsheet

Last Updated 03/31/2020 1




RFC2019021337 PRC‐005‐6 R3 Forked River Power LLC NCR00376 4/1/2017 4/1/2019 Self‐Report Completed Description of the Noncompliance (For purposes of this document, each noncompliance at issue is described as a “noncompliance,” regardless of its procedural posture and whether it was a possible, or confirmed noncompliance.)

On April 5, 2019, the entity submitted a Self‐Report stating that, as a Generator Owner, it was in noncompliance with PRC‐005‐6 R3. (On May 23, 2019, ReliabilityFirst discovered the same PRC‐005‐6 R3 during Compliance Audit conducted on May 20, 2019, because Self‐Report’s submitted during the 90‐day audit window are considered to be audit findings.)

The entity failed to complete 18‐month tests of battery terminal connection resistance and battery intercell or unit to unit connection resistance as required by PRC‐005‐6 Table 1‐4(a) from April 1, 2017, through April 19, 2019, for its two applicable Vented Lead Acid batteries. The entity did complete other 18 month tests including verifying battery continuity, verifying float voltage, and inspecting the condition of all cells where visible and additional battery tests including the 4‐month battery tests required under PRC‐005‐6. The reason the 18‐month tests of battery terminal connection resistance and battery intercell or unit to unit connection resistance tests were missed is that the entity failed to enter those tests into its work order system.

The root cause of this noncompliance was inadequate internal verification controls to assure that the 18‐month tests of battery terminal connection resistance and battery intercell or unit to unit connection resistance tests were entered into the entity’s work order system.

This noncompliance involves the management practices of verification and grid maintenance. Verification management is involved because the entity failed to assure that all PRC‐005‐6 compliance activities were included in related work orders. Grid maintenance management is involved because the noncompliance occurred in the entity’s Protection System Maintenance program.

This noncompliance started on April 1, 2017, when the entity was required to comply with PRC‐005‐6 R3 and ended on April 1, 2019, when the entity completed the 18‐month tests of battery terminal connection resistance and battery intercell or unit to unit connection resistance tests.

Risk Assessment This noncompliance posed a moderate risk and did not pose a serious or substantial risk to the reliability of the bulk power system (BPS) based on the following factors. The risk associated with failing to perform required maintenance activities within the required timeframe is that the device could fail to operate as expected, which could reduce the reliability of the BPS. The risk here is not minimal because the noncompliance had a duration of two (2) years. The risk here is not serious because the batteries were otherwise maintained and checked on a regular basis. The entity performed 4‐month testing, and some activities required for 18‐month testing including verifying battery continuity, verifying float voltage, and inspecting the condition of all cells where visible. Additionally, the batteries involved were relatively new, installed in 2015, and 2016, respectively which decreases the likelihood of substandard performance. ReliabilityFirst also notes that the entity contributes 86 MW of generation to the system. No harm is known to have occurred.


1) verified “Battery terminal connection resistance” and “Battery intercell or unit‐to‐unit connection resistance”; and2) aligned MP2 system with tracking and verifying that all 4‐month, 18‐month, and 6‐year PRC‐005‐6 tests under Table 1‐4(a) have been completed. For each test, a work order will be provided to

relevant employees indicating that the test must be completed. The entity also provided appropriate personnel with training on NERC responsibilities including PRC‐005‐6 maintenance and testing byconducting annual PRC‐005 training for all plant personnel who are involved in Protection System maintenance and testing. This will assure that plant personnel are apprised of the most currentstandards versions as well as any announced changes to the standard and implementation dates, as well as to allow the plant personnel to ask clarifying questions related to performance ofmaintenance obligations, documentation and compliance.






SERC2016016372 PRC-005-1 R2 Entergy NCR01234 11/19/2007 12/01/2017 Self-Report Completed

Description of the Noncompliance (For purposes of this document, each noncompliance at issue is described as a “noncompliance,” regardless of its procedural posture and whether it was a possible, or confirmed violation.)

On October 18, 2016, Entergy submitted a Self-Report stating that, as a Transmission Owner, it was in noncompliance with PRC-005-1 R2. Entergy failed to maintain a Protection System device within its required interval for the equipment at the 115 kV Dumas substation. On February 3, 2017, Entergy submitted an Expansion of Scope to the October 18, 2016 Self-Report stating that there was additional equipment at the 115 kV Dumas substation that had not received timely maintenance.

Regarding Instance 1, in 2008, Entergy suspended maintenance for a 115kV bus differential panel at the Dumas station in its Substation Work Management System (SWMS). The panel contained 10 Protection System devices: one microprocessor relay; four auxiliary trip relays; one lockout relay; four current transformers; and one potential transformer. Entergy issued no future work orders and was unable to provide the cause for the suspension of maintenance. As a result of the suspended maintenance, on December 31, 2013, Entergy missed the deadline to complete maintenance for the microprocessor relay; three auxiliary relays; and the lockout relay.

On December 2, 2015, Entergy completed a review of the Dumas station. The contractor performing the survey noted that the subject panel was located in the substation but was not included in the inventory. On October 24, 2016, Entergy completed required maintenance on the 115 kV Dumas station.

On November 28, 2017, Entergy submitted an additional Self-Report of PRC-005-1 R2 relating to the Adams Creek 230kV switchyard, which was designated NERC ID SERC2017018726. SERC determined that the noncompliance in SERC2017018726 was related to the instance in the original October 18, 2016 Self-Report. Therefore, SERC dismissed and consolidated SERC2017018726 with SERC2016016372.

In Instance 2, on February 10, 2010, Entergy suspended maintenance of the Adams Creek panel. The panel provides protection on the Adams Creek to Calpine 230 kV line. The panel included solid-state relays for protection of a short bus section and transmission line stub, which Entergy intentionally kept electrically isolated. Relay types included a distance relay; a current differential (86) relay; two associated auxiliary relays; and the lockout relays for the protection of the stub that remained energized. Entergy assigned a four-year maintenance interval for the panel devices. Entergy issued no future work orders and was unable to provide the cause for the suspension of maintenance. As a result of the suspended maintenance, on December 31, 2010, Entergy missed the first deadline to complete maintenance. On December 31, 2014, Entergy missed a second deadline to complete maintenance.

On July 7, 2017, an engineer was performing a data review of protective relay panels as part of an ongoing SWMS enhancement project and they discovered that regular maintenance was suspended in SWMS. Entergy performed the required maintenance on this date.

On December 28, 2017, Entergy submitted an Expansion of Scope to the November 28, 2017 Self-Report stating that Little Rock South substation had a missing interval of maintenance.

In Instance 3, on April 2, 2009, Entergy completed maintenance on the primary line protection relay at the Little Rock South station. The equipment has a four year requirement for maintenance. Entergy incorrectly scheduled the next maintenance for 2012 due to a mistake in the work order, but it was not performed since the required interval was not yet reached. On April 2, 2013, Entergy missed its maintenance deadline. On December 1, 2017, Entergy completed testing on the Little Rock South station.

On February 5, 2019, Entergy submitted another Self-Report of PRC-005-1 R2 relating to the Addis to Plaquemine 115kV line, which was designated NERC ID SERC2019021037. SERC determined that the noncompliance in SERC2019021037 involved similar maintenance failures to the instance in the original October 18, 2016 Self-Report. Therefore, SERC dismissed and consolidated SERC2019021037 with SERC2016016372.

Regarding Instance 4, in 2007, Entergy suspended maintenance on the tone set relay on the Addis to Plaquemine 115 kV line. The maintenance interval for the relay was set to 12 months. Entergy issued no future work orders and was unable to provide the cause for the suspension of maintenance. On November 19, 2007, Entergy missed the first deadline to complete maintenance. Entergy also missed the deadlines in 2008, 2009, 2010, and 2011. On June 23, 2012, Entergy instituted a revised Protection System Maintenance Program as part of the implementation plan for an upcoming NERC Standard. Entergy changed the maintenance interval from 12 months to 12 years. On January 10, 2019, Entergy discovered the missed maintenance as a part of an evaluation of maintenance records.

The duration includes multiple isolated instances and was not a continuous state of noncompliance. These instances of noncompliance started on November 19, 2007, when Entergy missed maintenance on the Addis to Plaquemine 115 kV tone set relay, and ended on December 1, 2017, when Entergy performed required maintenance on the Little Rock South station.

The cause for these four instances was management oversight. Management failed to verify implementation of adequate internal controls to ensure the closure of open work orders and the suspension of maintenance activities. For all instances, management failed to implement controls to identify the missed maintenance activities.



Risk Assessment These instances of noncompliance posed a moderate risk and did not pose a serious or substantial risk to the reliability of the bulk power system. Specifically, Entergy’s failure to perform maintenance on its Protection System devices could have caused improper operation of a relay. However, because of the setup of each system, it was unlikely that the operation would have caused a transmission line outage. For instance, the equipment at issue either had a backup system, which had been properly maintained or was unable to cause an outage to the transmission system. No harm is known to have occurred.

SERC considered Entergy’s PRC-005-1 R2 compliance history in determining the disposition track. Entergy has four relevant prior instances of noncompliance with PRC-005-1 R2. SERC determined that Entergy’s PRC-005-1 R2 compliance history should not serve as a basis for applying a penalty. The historical instances of noncompliance were primarily caused by Entergy misunderstanding its obligations, which led to missed deadlines through inaction. In contrast, the instant noncompliance was a direct result of Entergy actively switching maintenance tasks off in its tracking software by mistake. The historical Mitigation Plans focused on addressing issues due to inaction and therefore could not have prevented the instant noncompliance from occurring.

Mitigation To mitigate this noncompliance, Entergy:

1) unsuspended maintenance in SWMS and performed maintenance at the Dumas station;2) performed Panel DC Operational Test WR# 2105463 and Relay Calibration WR# 2105461 tasks at Adams Creek;3) tested Relays at Little Rock South;4) re-enabled the SWMS maintenance for the Addis tone set relay and the Plaquemine tone set relays;5) performed the channel integrity maintenance at Addis;6) issued Configuration Management (AM-CM-FAC-001) and Energization Notice (AM-CM-AD-001) procedures;7) updated AM-PC-AD-001, sections 1.2.1 and 4.2.1, to require approval when PRC-005 components are removed from maintenance;8) implemented user access level controls on the Override Reliability-Centered Maintenance (RCM) Rank option and limited access to Subject Matter Experts;9) updated AM-CM-AD-027 to provide guidance on selecting and configuring communication system schemes in SWMS;10) developed an exception report for SWMS work requests that flag suspect work requests;11) created a new monthly automated report to query SWMS for SERC applicable components without maintenance enabled;12) updated the WebTap training module - PRC-005 Substation Supervisor and Planner Scheduler Training - to align with the changes to AM-CM-AD-027 and reissued the WebTap training;13) implemented a change in SWMS to: expand the scope of RCM update script to look for changes in in-service date, application, availability, and a default of "RCM Low" for each new asset that is

created. This is currently set to "No Maintenance";14) developed requirements and guidance specific to the expectations of closing current open work to previously completed work requests;15) modified the Work Management Procedure to include guidance specific to the expectations of closing current open work to previous completed work requests based on information provided by AM

Transmission Protection; and16) created and issued training on work management procedure modifications.






RFC2019021379 VAR‐002‐4.1 R2 AEP Generation Resources Inc. NCR11401 2/15/2018 9/18/2018 Self‐Report Completed Description of the Noncompliance (For purposes of this document, each noncompliance at issue is described as a “noncompliance,” regardless of its procedural posture and whether it was a possible, or confirmed noncompliance.)

On April 16, 2019, the entity submitted a Self‐Report stating that, as a Generator Operator, it was in noncompliance with VAR‐002‐4.1 R2.

On September 18, 2018, the entity performed an annual internal document review and discovered that the AEP Generation Dispatch desk procedure did not match the AEP Transmission Operator (TOP) VAR‐002 Generation instruction letter for communicating deviations from the voltage or Reactive Power schedules provided by the TOP. The desk procedure required the dispatcher to call only AEP (TOP) for a voltage schedule deviation. The instruction letter required the dispatcher to call AEP (TOP) and PJM (TOP) when making notifications about a voltage schedule deviation.

As a result of this discovery, the entity performed an extent of condition and discovered that from February 15, 2018 to September 18, 2018, there were 40 instances of the AEP Generation Dispatcher not communicating voltage schedule deviations to PJM as is required.

Prior to revising the AEP Generation Dispatch desk procedure on September 18, 2018, the desk procedure stated that the dispatcher should call the AEP TOP when deviating from the voltage schedule. However, the AEP TOP instruction letter effective February 6, 2018 for the Cardinal facility and February 8, 2018, for the Conesville facility stated that dispatch must notify AEP TOP and PJM TOP. Because the dispatchers followed the desk procedure (which was inaccurate), the entity was notifying only one of the two TOPs it was required to notify (it was notifying AEP, but not PJM) in the case of a deviation from the voltage schedule.

The root cause of this noncompliance was a failure to integrate a new TOP notification requirement into the existing entity procedure for dispatchers resulting in a failure to notify PJM of voltage schedule deviation on 40 separate instances.

This noncompliance involves the management practices of grid operations and workforce management. Grid operations management is involved because by failing to communicate voltage deviations to PJM, the entity was adversely impacting PJM’s ability to operate with comprehensive situational awareness. Workforce management is involved because the entity did not train its dispatchers on the new process requirements created by AEP TOP’s instruction letter.

This noncompliance started on February 15, 2018, when the entity was required to notify both TOPs of a voltage deviation, and failed to notify PJM. The noncompliance ended on September 18, 2018, when the entity updated its dispatch notification procedure.

Risk Assessment This noncompliance posed a moderate risk and did not pose a serious or substantial risk to the reliability of the bulk power system (BPS) based on the following factors. The risk posed by this noncompliance is that the failure to notify the TOP of voltage deviations places the TOP in an informational disadvantage and adversely impacts situational awareness, potentially resulting in harm to the BPS. The risk is not minimal because of the 40 separate instances, some with durations lasting multiple hours. The risk is partially reduced because the AEP TOP was notified in all instances. (The AEP TOP is required to communicate with PJM for any reliability issues that may arise.) Further reducing the risk, the entity’s system voltage telemetry was readily available to PJM at all times. No harm is known to have occurred.


1) revised the Generation Dispatch desk procedure to align to the Transmission Operator (TOP) instruction for notification of voltage deviations;2) performed an extent of condition to investigate not notifying TOP as instructed. No gaps in communication were identified other than what was identified from this incident;3) changed the management process to address any changes to the awareness and readiness process of going live with any changes to VAR‐002 R2 operating procedures; and4) revised change management process for alignment to the current organization to update the awareness and readiness communication between the compliance organization and the business units

accountable for compliance roles to the NERC requirement. The flow of communication to provide feedback from business units to the NERC compliance organization providing readiness of updatedcompliance documents has been updated to have a primary central compliance group responsible for ensuring compliance readiness prior to going live with any changes to operation processes.

AEP as a GO/GOP/TOP has undergone a joint venture in continuous improvement of communication of VAR‐002 measures to enhance the real‐time system voltage data and statuses between Generation and Transmission for improved system performance and reliability of the Bulk Electric System from the latter part of 2017 through the end of 2018. Enhancements include more visibility and situational awareness into the real‐time system generator voltage and schedule to improve overall regional system voltages.







RFC2019021378 VAR‐002‐4.1 R2 American Electric Power Service Corporation as agent for etc. NCR00682 9/11/2018 9/19/2018 Self‐Report Completed

Description of the Noncompliance (For purposes of this document, each noncompliance at issue is described as a “noncompliance,” regardless of its procedural posture and whether it was a possible, or confirmed noncompliance.)

On April 15, 2019, the entity submitted a Self‐Report stating that, as a Generator Operator, it was in noncompliance with VAR‐002‐4.1 R2. On September 25, 2018, the entity discovered, as the result of an internal generation dispatch management review, that the generation dispatcher had not verbally communicated voltage deviations to the Transmission Operator (TOP) in two instances. In both instances, voltage deviated for more than 30 continuous minutes and the entity failed to adhere to the TOP voltage schedule letter guidance and provide verbal notification to both the TOP and PJM. The first instance occurred at Generator Amos Unit 2 on September 11, 2018. Voltage deviated from the scheduled tolerance at 05:53 AM, returned within the voltage schedule tolerance at 06:26 AM. The dispatcher missed a system voltage deviation alarm that alerts the dispatcher to make verbal notifications to the entity TOP and PJM. The plant operator also received a system voltage deviation alarm and likewise failed to make the required notification to generation dispatch. The second instance occurred at Mitchell Unit #1 on September 18‐19, 2018, when the entity failed to report to the entity TOP an exceedance deviation from the assigned voltage schedule. The dispatcher notified PJM as instructed but did not notify the entity TOP of the system voltage deviation. Voltage deviated from scheduled tolerance at 18:02 on September 18, 2018 and returned within voltage schedule tolerance at 15:12 on September 19, 2018. The root cause of this noncompliance was inadequate training as entity personnel were not effectively trained on the purpose of the system voltage deviation alarm and on the actions that need to be taken to respond to that alarm. That ineffective training resulted in the failure to notify the TOPs of voltage deviations from the assigned voltage schedule in excess of 30 continuous minutes. This noncompliance involves the management practices of grid operations and workforce management. Grid operations management is involved because by failing to verbally communicate voltage deviations to the TOPs, the entity was adversely impacting the TOP’s ability to operate with comprehensive situational awareness. Workforce management is involved because though the dispatchers were trained on verbal notifications, they were not trained adequately to assure that verbal notifications were executed during voltage deviations. This noncompliance started on September 11, 2018, when the first instance of the failure to notify the entity TOP of a voltage deviation occurred and ended on September 19, 2018, when the voltage was no longer deviating after the second failure to notify the TOP verbally.

Risk Assessment

This noncompliance posed a moderate risk and did not pose a serious or substantial risk to the reliability of the bulk power system (BPS) based on the following factors. The risk posed by this noncompliance is that the failure to notify the TOP of voltage deviations places the TOP in an informational disadvantage and adversely impacts situational awareness, potentially resulting in harm to the BPS. The risk is partially reduced because both TOPs were utilizing Supervisory Control and Data Acquisition (SCADA), Inter‐Control Center Communications Protocol, and other alarms for monitoring and tracking generator system voltage, in place of verbal notification. (The additional verbal notification requirements were put in place in 2017 as the result of a recommendations following a 2015 regional audit. These serve as an additional layer of monitoring, beyond SCADA and alarms, rather than a single method of monitoring and communication.) Further reducing the risk is the short three minute duration of the first instance and the fact that the entity self‐identified both instances. The entity also self‐reported additional related instances which are being resolved under a separate NERC Violation ID. All of the instances, taken together, pose a moderate risk to the BPS. No harm is known to have occurred. ReliabilityFirst considered the entity’s compliance history and determined there were no relevant instances of noncompliance.

Mitigation

To mitigate this noncompliance, the entity: 1) reviewed its processes for gaps and awareness of desired processes to Generation Dispatchers. As part of the review, the roles and responsibilities of notification to entity Transmission Operator and

PJM for Generator system voltage deviations was discussed to affirm operators understood their role in the communication process. Generation NERC compliance group reviewed and verified the desk procedure was meeting Transmission Operator expectations for voltage deviation notification;

2) helped achieve preventative recurrence by periodic or as needed reviews of source documents and operator procedures for alignment; and 3) revised rebooting to maintain awareness of generator voltage monitoring during entity Generation Dispatch shift change. The restarting and rebooting of the non‐CIP computer that normally occurs

at the beginning of each shift every day has been revised to reboot only on Monday and Thursday on day shift to ensure availability of support staff if issues are encountered. In 2017, the entity developed a revised communication process to enhance the awareness of monitoring generator system voltage, and communication of voltage deviations. This revised process was a result of a 2015 regional audit recommendation to improve voltage schedule communication. The improved communication workflows for voltage deviations provide an additional layer of monitoring and communication, beyond the existing SCADA/alarms, utilizing Generation Dispatch for increased awareness. The entity as a GO/GOP/TOP has undergone a joint venture in continuous improvement of communication of VAR‐002 measures to enhance the real‐time system voltage data and statuses between Generation and Transmission for improved system performance and reliability of the Bulk






RFC2019021378 VAR‐002‐4.1 R2 American Electric Power Service Corporation as agent for etc. NCR00682 9/11/2018 9/19/2018 Self‐Report Completed

Electric System from the latter part of 2017 through the end of 2018. Enhancements include more visibility and situational awareness into the real‐time system generator voltage and schedule to improve overall regional system voltages.






RFC2019021566 PRC‐005‐1 R2 The Dayton Power and Light Company NCR00748 6/18/2007 4/30/2019 Self‐Report Completed Description of the Noncompliance (For purposes of this document, each noncompliance at issue is described as a “noncompliance,” regardless of its procedural posture and whether it was a possible, or confirmed noncompliance.)

On May 14, 2019, the entity submitted a Self‐Report stating that, as a Transmission Owner, it was in noncompliance with PRC‐005‐1 R2. On April 29, 2019, during a review and update of the model used for short circuit analysis and protection engineering, the entity discovered that three 345 kV bus differential relays were missing from its asset management database, and consequently excluded from the entity’s routine maintenance schedule, which required these relays to be tested every 4 years. Upon investigation, the entity determined that these three relays were missed during its initial data migration from a paper‐based system to the asset management database in 2009/2010. That data migration included the transcription of information for approximately 8,000 total relays and over 11,000 associated paper test cards. The root cause of this noncompliance was an oversight during the initial data migration and subsequent failure to identify the oversight during the data validation process. This root cause involves the management practices of verification and validation. This noncompliance started on June 18, 2007, when the entity was required to comply with PRC‐005‐1 R2 and ended on April 30, 2019, when the entity completed the testing and maintenance activities on these three relays.

Risk Assessment

This noncompliance posed a moderate risk and did not pose a serious or substantial risk to the reliability of the bulk power system (BPS) based on the following factors. The risk posed by failing to perform required maintenance activities within the required timeframe is that the devices could fail to operate as expected, which could reduce the reliability of the BPS. The risk is not minimal in this case because of the length of time that the issue persisted. The risk is not serious or substantial in this case based on the following factors. First, these three relays represent only .13% of the entity’s total inventory of PRC‐005 components and only .3% of its total inventory of PRC‐005 relays, which indicates that this was an isolated issue and not a systemic fall down. Second, overlap protection from the bank high side and connected breakers are set to operate before other facilities would have operated to isolate the bus at issue. ReliabilityFirst also notes that throughout the time period that this issue persisted: (a) no events occurred that called upon the bus differential to operate; and, (b) the testing and maintenance of these three relays identified no operational issues. No harm is known to have occurred. ReliabilityFirst considered the entity’s compliance history and determined there were no relevant instances of noncompliance.

Mitigation

To mitigate this noncompliance, the entity: 1) performed the requisite maintenance and testing, and added the relays to its asset management database to ensure future maintenance and testing is done on time; 2) identified Bulk Electric System substations where potential devices are located and field verification is required; 3) developed work packets for first set of field verifications. The entity also performed field inspection of substation relays for the first set of substations; 4) developed work packets for second set of field verifications. The entity also performed field inspection of substation relays for the second set of substations; and 5) summarized results of field inspection program, submit any pertinent information from the field inspection verification program to ReliabilityFirst.






RFC2019020967 EOP‐005‐2 R14 Seneca Generation, LLC NCR11446 2/13/2014 11/13/2018 Compliance Audit Completed Description of the Noncompliance (For purposes of this document, each noncompliance at issue is described as a “noncompliance,” regardless of its procedural posture and whether it was a possible, or confirmed noncompliance.)

On January 10, 2019, ReliabilityFirst determined that the entity, as a Generator Operator, was in noncompliance with EOP‐005‐2 R14 identified during a Compliance Audit conducted from September 14, 2018 through December 4, 2018.

The ReliabilityFirst audit team determined that the entity did not have a specific process for starting its Blackstart Resources during an event (Blackstart Event Procedure) for the two entity Black Start units which are 229 MVA units. Rather, the entity had a Blackstart Test Procedure, and indicated that the steps to be taken in case of an event were included in the Blackstart Test Procedure, but not specifically noted in the Blackstart Test Procedure.

The root cause of this noncompliance was inadequate internal verification controls and lack of knowledge of EOP‐005‐2 R14 resulting in a failure to implement a distinct internal Blackstart Event Procedure.

This noncompliance involves the management practices of grid operations and verification. Grid operations management is involved because the entity failed to effectively define and implement operating procedures for the Blackstart units. Verification management is involved because the entity did not ensure that it had an actionable procedure for a Blackstart during a grid event.

The noncompliance began on February 13, 2014, the date the entity was required to comply with EOP‐005‐2 R14. The noncompliance ended on November 13, 2018, the date the entity implemented a new, distinct, Blackstart Event Procedure.

Risk Assessment This noncompliance posed a moderate risk and did not pose a serious or substantial risk to the reliability of the bulk power system based on the following factors. The risk of not having procedures in place for starting a Blackstart Resource is that if a blackout or partial blackout occurred, the entity may not have been able to start the Blackstart Resources in a timely manner if required to do so. The risk is not minimal because of the five year duration, and inability to identify the noncompliance via internal controls. The risk here is not serious because while the entity did not have a Blackstart Event Procedure, it did have a Blackstart Test Procedure which included the steps to be taken in case of an event (but were not outlined as event specific steps). The entity’s Blackstart Test Procedure had been in place since 2011, and employees had been trained on the procedure in 2015, 2017, and 2018 which meant employees were likely familiar with the steps involved in the case of an event. ReliabilityFirst also notes that during the period of the noncompliance, the entity was not directed to use its Blackstart Resources at any time. No harm is known to have occurred.

The entity has relevant compliance history. However, ReliabilityFirst determined that the entity's compliance history does not warrant an elevated risk and should not serve as a basis for an aggravating penalty because the prior noncompliance is distinguishable as it involved different conduct (i.e. failure to train) and a different root cause.

Mitigation To mitigate this noncompliance, the entity:

1) created a Blackstart Event Training work order to ensure plant operating personnel receive Blackstart Event Training which includes the documented procedures for starting each Blackstart Resourceand energizing a bus. The operating personnel will also receive training on the PJM restoration plan including coordination with the Transmission Operator;

2) created a new Blackstart Start up Procedure for both Units 1 and 2 which includes documented procedures for starting each Blackstart Resource and energizing a bus; and3) trained all Operating Personnel on the Blackstart Start up Procedure for both Units 1 and 2 and the Restoration Plan.






SERC2019021062 MOD-027-1 R5 Tennessee Valley Authority (TVA) NCR01151 02/03/2016 01/03/2019 Self-Report Completed


On February 15, 2019, TVA submitted a Self-Report stating that, as a Transmission Planner (TP), it was in noncompliance with MOD-027-1 R5. TVA did not provide a written response to the Generator Owner (GO) within 90 days of the turbine/governor and load control or active power/frequency control system model verification that the model was usable or not useable.

On November 19, 2018, TVA determined that it failed to provide a written response to the GO, within 90 days of the data submission by the GO, for units in November 2015, April 2016, June 2016, May 2017, and August 2017. Additionally, in some cases, TVA did not determine if the model was or was not useable within 90 days of model data submission. Ultimately, TVA determined that all of these models were useable. The model data for the five periods involved 36 units, which represented 4,498 MVA of TVA’s 33,526 MVA of generation (13.4%).

As part of its mitigating activities, TVA completed an extent-of-condition review and identified five additional instances that included nine additional units. In total, the number of days that elapsed from the date of submission to the date TVA responded ranged from 94 days to 1,155 days.

This noncompliance started on February 3, 2016, when TVA failed to provide the written response to the GO, and ended on January 3, 2019, when TVA submitted the last overdue response to the GO.

The cause of the noncompliance was management oversight (A4B1). Management failed to verify that all necessary internal controls, e.g., tracking software and an assignment of a secondary delegate responsible for tracking model submissions, were implemented to ensure timely responses to the GO. The TVA TP used an email inbox for the receipt of model data from GOs and assigned a single subject-matter-expert (SME) to monitor the mailbox. The SME failed to follow-up with written notification after discussions with the GO regarding models. The SME also went on leave after receiving the notification and failed to test the models and respond to the GO after returning from leave. Additionally, when the SME received a batch of submissions, the SME used the date of the last submission of the batch as the date to respond by, instead of addressing all units from the submission in the response.

Risk Assessment This noncompliance posed a moderate risk and did not pose a serious or substantial risk to the reliability of the bulk power system. Specifically, TVA’s failure to verify model usability, and notify the GO within 90 days that the models submitted are usable or not usable could have led to TVA using incorrect models that could have resulted in erroneous assessments or an incorrect corrective action plan. The noncompliance included 45 units and the response to the GO data submission ranged from 94 days to 1155 days. TVA was able to include the models that were slightly late in the applicable SERC data submittal, however, for those that had longer delays, TVA was unable to include those models in the applicable SERC data submittal. While TVA ultimately determined the model information was useable, if TVA determined the models were unusable, the GO would have been required to re-engage the contractor that had originally provided the verified models. This could have resulted in additional delays in obtaining a usable verified model for these units. In addition, TVA only identified the instant noncompliance when reviewing data for a previous GO MOD noncompliance. No harm is known to have occurred.

SERC considered TVA’s compliance history and determined that there were no relevant instances of noncompliance. Mitigation To mitigate this noncompliance, TVA:

1) sent replies to all outstanding model validations as required by the Standard;2) assigned a delegate, in addition to the SME, to assist in tracking the model submissions;3) implemented a new action tracking software. As models are received by the TP, they will be entered into the software manually, which will then track the appropriate due dates for the replies. E-mailsfrom the software are automatically sent to the SME each week, along with an additional e-mail as the due date for the reply approaches to both the SME and the assigned delegate;4) conducted an extent-of-condition (EOC) analysis to verify the results of the GO EOC analysis to be certain that all received models reports have been tested and the appropriate replies to the GO havebeen performed; and5) created a new Technical Procedure to document the process that the TP used to process and reply to incoming MOD-027 model submissions with enough details such that a new employee will be ableto track, process, and reply to a new MOD-027 model supplied by a GO. All impacted employees were notified and trained on the contents of the procedure.





TRE2017018733 PRC-019-2 R1 CPS Energy (CPS Energy1) NCR04037 07/01/2016 09/13/2017 Self-Report Completed


On November 29, 2017, CPS Energy1 submitted a Self-Report stating that, as a Generator Owner (GO), it was in noncompliance with PRC-019-2 R1. Specifically, CPS Energy1 failed to properly coordinate the voltage regulating system controls with the applicable equipment capabilities and settings of the applicable Protection System devices and functions for 40% of its applicable Facilities by July 1, 2016, and for 60% of its applicable Facilities by July 1, 2017, as required.

The Implementation Plan for PRC-019-2 required that 40% of an entity’s applicable Facilities be verified to meet the requirements of R1 by July 1, 2016, and that 60% of an entity’s applicable Facilities be verified by July 1, 2017. CPS Energy1 was late in meeting these requirements. The Implementation Plan also required that 80% of an entity’s applicable Facilities be verified by July 1, 2018.

On March 1, 2016, CPS Energy1’s Compliance department became aware of the PRC-019-2 R1 requirement. On July 24, 2017, which is after the July 1, 2016 implementation plan milestone, studies were completed and settings were coordinated on 40% of CPS Energy1’s applicable Facilities. On September 13, 2017, which is after the July 1, 2017 implementation plan milestone to coordinate settings on 60% of applicable Facilities and before the July 1, 2018 implementation plan milestone to coordinate settings on 80% of applicable Facilities, studies were completed and settings were coordinated on 60% of CPS Energy1’s applicable Facilities, ending the noncompliance.

The root cause of the noncompliance was twofold. First, CPS Energy1’s Compliance department did not fully monitor the development and implementation of PRC-019-2. As a result, CPS Energy1 did not become completely aware of the scope and breadth of PRC-019-2 R1 and the efforts needed to comply until the spring of 2016. Second, CPS Energy1 underestimated the amount of time and resources needed to perform the studies required by PRC-019-2 R1. CPS Energy1 attempted to perform the studies itself, but soon determined it lacked the necessary resources and would need to hire a contractor. CPS Energy1 was not initially aware of the cost and timeframe involved in hiring a contractor to gather and analyze data and create the coordination and evidence required for compliance with PRC-019-2 R1.

This noncompliance started on July 1, 2016, when PRC-019-2 became mandatory and enforceable, and ended on September 13, 2017, when CPS Energy1 completed the coordination of its voltage regulating system controls with the applicable equipment capabilities and settings of the applicable Protection System devices and functions for 60% of its applicable Facilities.

Risk Assessment This noncompliance posed a moderate risk and did not pose a serious or substantial risk to the reliability of the bulk power system.

Specifically, the failure to coordinate the voltage regulating system controls with the applicable equipment capabilities and settings of the applicable Protection System devices and functions could lead to a generator failing to trip before the equipment sustained damage, or tripping for a system event that should not have caused the generator to trip at all. CPS Energy1’s applicable Facilities fall into three groups with varying degrees of potential impact on the grid. The first group of applicable Facilities is connected to the 138 kV Transmission system and is located in San Antonio, a large load center. The second group of generation is connected to the 345 kV Transmission system, which provides this group the ability to serve load beyond its local area, and is located in the same large load center. Transmission planners and operators expect both groups of applicable Facilities to be available to serve load and provide voltage support in that load center. The final group of applicable facilities is connected to the 345 kV Transmission system, which provides this group the ability to serve load beyond its local area. This group is located between two large load centers, San Antonio and Austin, and Transmission planners and operators expect this group to be available to serve load in both the large load centers. Of the applicable Facilities that were not timely compliant with the Reliability Standard, CPS Energy1 was an average of 263 days late achieving compliance, bringing the applicable Facilities into compliance 23 days through 388 days after the applicable implementation plan deadline. Thirteen applicable Facilities of 24 total (as of (November 29, 2017)) were not timely compliant and needed settings changes to achieve compliance. CPS Energy1 made the settings changes an average of 95 days after the date of the study. The 13 applicable Facilities have a combined real power rating of 1,315 MW, with an average rating of 101 MW and the largest applicable Facility rated at 323 MW.

CPS Energy1 did not experience any Misoperations or trips related to the lack of coordination, and when the study results indicated that settings needed to be changed, CPS Energy1 determined that the old settings had not impacted equipment performance.

No harm is known to have occurred.

Texas RE considered CPS Energy1’s compliance history and determined there were no relevant instances of noncompliance.



Mitigation Activity To mitigate this noncompliance, CPS Energy1: 1) completed studies and coordinated settings on 60% of its applicable Facilities; 2) began holding recurring meetings to track Reliability Standard development and GO compliance status; 3) reorganized the Compliance department to more effectively utilize internal resources; 4) hired a Power Generation Regulatory Compliance Manager for increased oversight of compliance with PRC-019-2 Requirements; and 5) implemented a compliance tracking tool that assigns compliance responsibility for each applicable NERC Reliability Standard and Requirement and tasks related to compliance. Texas RE has verified the completion of all mitigation activity.





TRE2018019912 PRC-008-0 R1 Kerrville Public Utility Board (KPUB) NCR10179 12/06/2007 03/31/2015 Compliance Audit

Completed


During a Compliance Audit conducted from April 17, 2018, through April 19, 2018, Texas RE determined that KPUB, as a Distribution Provider (DP) was in noncompliance with PRC-008-0 R1. Specifically, KPUB failed to have an Underfrequency Load Shedding (UFLS) equipment maintenance and testing program in place that included UFLS equipment identification, the schedule for UFLS equipment testing, and the schedule for UFLS maintenance, for all of KPUB’s UFLS equipment. During the compliance audit, KPUB was asked to provide a copy of its UFLS equipment maintenance and testing program. In response, KPUB provided its Protection System Maintenance Program (PSMP) and an interconnection agreement between it and another Entity. KPUB’s PSMP and interconnection agreement did not address UFLS equipment maintenance and testing scheduling as required by PRC-008-0 with regard to the bus potential transformers at issue. The root cause of this noncompliance was KPUB’s failure to adhere to its PSMP. KPUB failed to adequately inventory its equipment and subsequently follow the applicable requirements of PRC-008-0. This noncompliance started on December 6, 2007, when KPUB was registered, and ended on March 31, 2015, the last day PRC-008-0 R1 was applicable to the equipment at issue.

Risk Assessment

This noncompliance posed a minimal risk and did not pose a serious or substantial risk to the reliability of the bulk power system. Failure to identify all UFLS equipment and provide a schedule for maintenance and testing of all component types could potentially result in inconsistent testing and maintenance activity and/or documentation thereof, which could limit KPUB’s awareness of the working order of its UFLS equipment. The risk of this noncompliance was reduced by the fact that the other Entity that was party to the interconnection agreement with KPUB was regularly testing and maintaining the bus potential transformers in question during maintenance and testing of its own UFLS equipment at these facilities. The risk of this noncompliance was further reduced by the fact that the 2017 ERCOT UFLS survey shows that the UFLS relays associated with KPUB’s potential transformers would trip 15.81 MW of Load at the time of the survey. This is approximately 2.6% of its Transmission Operator’s (TOP’s) UFLS enabled Load (609.03 MW) at the time of the survey. No harm is known to have occurred. Texas RE considered KPUB’s compliance history and determined there were no relevant instances of noncompliance.

Mitigation To mitigate this noncompliance, KPUB: 1) performed maintenance activities in 2011, 2013, and 2015 that met the requirements of PRC-005-2, Table 3, which became effective April 1, 2015; and 2) updated its PSMP to include tracking of maintenance on all PRC-005-6 applicable Protection System Components (including its UFLS Components). Texas RE has verified the completion of all mitigation activity.





TRE2018019913 PRC-005-1 R2.1 Kerrville Public Utility Board (KPUB) NCR10179 09/01/2009 02/20/2018 Compliance Audit

Completed


During a Compliance Audit conducted from April 17, 2018, through April 19, 2018, Texas RE determined that KPUB, as a Distribution Provider (DP) was in noncompliance with PRC-005-1 R2.1. Specifically, KPUB failed to have evidence its Protection System devices were maintained and tested within the defined intervals of its Protection System Program (PSMP). KPUB’s PSMP in effect during the period of the noncompliance required KPUB to verify the presence and rotation of all voltage and currents associated with transformers on a 48-month maintenance interval. During the compliance audit, Texas RE determined that KPUB failed to conduct the required maintenance in a timely fashion for five of ten of KPUB’s transformers designed to detect faults on Bulk Electric System (BES) Elements. The due date for the required maintenance and testing for the Protection System Devices that are the subject of this noncompliance expired prior to KPUB’s transition of these devises to its PRC-005-2(i) PSMP, therefore, the longer maintenance period for relays under PRC-005-2(i) is not applicable to these devices during this time period. The root cause of this noncompliance was KPUB’s failure to adhere to its PSMP. More specifically, KPUB failed to monitor and track maintenance activities performed by third-party contractors and ensure that the required performance was completed in a manner, and within a timeframe, required by its PSMP. This noncompliance started on September 1, 2009, the due date for the earliest missed maintenance interval for one of its transformers, and ended on February 20, 2018, when KPUB completed testing on the last transformer that was the subject of overdue testing.

Risk Assessment

This noncompliance posed a moderate risk and did not pose a serious or substantial risk to the reliability of the bulk power system. The failure to maintain transformers and associated protection systems in accordance with an Entity’s PSMP has the potential to affect the reliability of the bulk power system by increasing the risk that a protection system may fail and cause a misoperation. However, the risk regarding these instances of noncompliance was reduced because, while the periods for the actual maintenance exceeded the intervals prescribed within KPUB’s PRC-005-1 PSMP, they would not have exceeded the maximum 12-year maintenance intervals prescribed within PRC-005-6 for these transformers had KPUB transitioned this equipment to the new standard. KPUB is a municipal electric system that serves approximately 22,000 customers over an area of 146 square miles. No harm is known to have occurred. Texas RE considered KPUB’s compliance history and determined there were no relevant instances of noncompliance.

Mitigation To mitigate this noncompliance, KPUB: 1) completed the required testing on the applicable transformers; and 2) updated its PSMP to include tracking of maintenance on all PRC-005-6 applicable Protection System Components. Texas RE has verified the completion of all mitigation activity.






RFC2018020611 PRC‐005‐6 R3 CPV Maryland, LLC NCR11706 6/14/2017 10/10/2018 Self‐Report Completed Description of the Noncompliance (For purposes of this document, each noncompliance at issue is described as a “noncompliance,” regardless of its procedural posture and whether it was a possible, or confirmed noncompliance.)

On October 22, 2018, the entity submitted a Self‐Report to ReliabilityFirst stating that, as a Generator Owner, it was not in compliance with PRC‐005‐6 R3.

On February 14, 2017, the entity brought a natural gas‐fired 2x1 combined cycle 745 MW electric generation facility (the Plant) into commercial operation. Prior to the start of commercial operations the entity engaged a third party operations and maintenance contractor (the Contractor) to operate the Plant and to perform services necessary to ensure that the entity was in compliance with its obligations relating to the Plant. (The entity was responsible for overseeing the Contractor’s NERC compliance program. The entity performed its oversight duty by participating in multiple meetings with the Contractor before, during, and after commissioning of the Plant; communicating with the Contractor personnel who were on site at the Plant regarding compliance; and communicating with the Contractor’s corporate NERC personnel regarding compliance of the Plant.)

During his time with the entity, the Contractor turned over the management team at the Plant significantly, including three different plant managers. On July 23, 2018, based on ineffective communication and a lack of responsiveness from the Contractor, the entity replaced the Contractor with a new operations and management contractor, the Second Contractor. Upon hiring the Second Contractor, the entity directed the Second Contractor to perform a comprehensive review of the entity’s compliance program as it relates to the NERC standards which apply to a Generator Owner/Generator Operator. The Second Contractor identified the following noncompliance.

The Plant has four (4) battery locations housing the Protection System Station DC supply Vented Lead‐Acid batteries: 1) the battery racks at the steam turbine generator power distribution center; 2) the combustion turbine generator Unit 11; 3) the combustion turbine generator Unit 12; and 4) the main balance of the Plant. The Vented Lead‐Acid batteries provide backup DC power to plant equipment in the event that main AC power is lost.

While the entity had a procedure in place that outlined the compliance requirements of PRC‐005‐6 R3 and required actions of the Contractor; the Contractor failed to perform them, or failed to document performance resulting in this noncompliance. Specifically, the Contractor was to perform maintenance on the Protection System Station DC supply Vented Lead‐Acid batteries required by PRC‐005‐6 R3 Table 1‐4. During the performance of the internal audit, the Second Contractor found no records to establish that the Contractor performed either four‐month or eighteen month maintenance on the Protection System Station DC supply Vented Lead‐Acid batteries required by PRC‐005‐6 R3 Table 1‐4.

The root cause of this noncompliance was that the entity did not have an adequate verification control to assure that the Contractor responsible for NERC compliance took the steps necessary to be fully compliant with PRC‐005‐6 R3 Table 1‐4.

This noncompliance involves the management practices of external interdependencies and verification. External interdependencies is involved because the noncompliance arose from the failure of a contractor and the entity’s inadequate oversight of that contractor. Verification management is involved because the entity failed to confirm that the Contractor was properly performing its NERC compliance functions.

The noncompliance began on June 14, 2017, the date the entity was required to comply with PRC‐005‐6 R3. The noncompliance ended on October 10, 2018, the date the entity had completed both the four month and eighteen month battery maintenance activities.

Risk Assessment This noncompliance posed a moderate risk and did not pose a serious or substantial risk to the reliability of the bulk power system based on the following factors. The potential risk posed by this noncompliance is that unmaintained and untested batteries would fail and that failure could lead to local loss of load or transmission equipment at the substation. The risk is not minimal because the entity missed four different four month maintenance cycles. However, the risk is not serious because the entity operators performed basic daily checks of all batteries including voltage, amps, fan and visual inspection. Additionally, the entity performed battery bank testing during the commissioning phase in anticipation of coming online on February 14, 2017, which further reduces the risk of battery failure. ReliabilityFirst notes that there is no evidence of a loss of generation or downtime resulting from a battery related failure. No harm is known to have occurred.


1) conducted 4 calendar month battery maintenance with entity personnel;2) conducted the 18‐month battery maintenance activities;3) developed forms to be completed by personnel as they are completing the 4‐month and 18‐month battery maintenance activities;4) uploaded the required maintenance and testing activities into two different tracking software applications utilized at the Plant (Gensuite and Maximo). Gensuite will automatically notify Plant

Management of the 4‐month and 18‐month maintenance intervals, prior to the respective due dates. The Battery Maintenance Forms have been uploaded to Maximo to be generated as preventive






RFC2018020611 PRC‐005‐6 R3 CPV Maryland, LLC NCR11706 6/14/2017 10/10/2018 Self‐Report Completed maintenance attachments. The 4‐month forms will automatically generate every three calendar months and the 18‐month form will generate every 12 months, to be completed during the required maintenance interval by the entity.







RFC2018020612 PRC‐019‐2 R1 CPV Maryland, LLC NCR11706 2/14/2017 11/16/2018 Self‐Report Completed Description of the Noncompliance (For purposes of this document, each noncompliance at issue is described as a “noncompliance,” regardless of its procedural posture and whether it was a possible, or confirmed noncompliance.)

On October 22, 2018, the entity submitted a Self‐Report to ReliabilityFirst stating that, as a Generator Owner, it was not in compliance with PRC‐019‐2 R1. On February 14, 2017, the entity brought a natural gas‐fired 2x1 combined cycle 745 MW electric generation facility (the Plant) into commercial operation. Prior to the start of commercial operations the entity engaged a third party operations and maintenance contractor (the Contractor) to operate the Plant and to perform services necessary to ensure that the entity was in compliance with its obligations relating to the Plant. (The entity was responsible for overseeing the Contractor’s NERC compliance program. The entity performed its oversight duty by participating in multiple meetings with the Contractor before, during, and after commissioning of the Plant; communicating with the Contractor personnel who were on site at the Plant regarding compliance; and communicating with the Contractor’s corporate NERC personnel regarding compliance of the Plant.) During his time with the entity, the Contractor turned over the management team at the Plant significantly, including three different plant managers. On July 23, 2018, based on ineffective communication and a lack of responsiveness from the Contractor, the entity replaced the Contractor with a new operations and management contractor (the Second Contractor). Upon hiring the Second Contractor, the entity directed the Second Contractor to perform a comprehensive review of the entity’s compliance program as it relates to the NERC standards which apply to a Generator Owner/Generator Operator. The Second Contractor identified the following noncompliance. Since the Plant became operational as of February 14, 2017, after the implementation of PRC‐019‐2 R1, the entity was required to have: (a) 40 percent of its generation units, synchronous condenser voltage regulating controls, limit functions, equipment capabilities, and Protection System settings coordinated and verified in accordance with PRC‐019‐2 Section G Attachments 1, 2, & 3 no later than 2/14/2017, (b) 60 percent of such facilities coordinated and verified by July 1, 2017; and (c) 80 percent of such facilities coordinated and verified by July 1, 2018. When the Second Contractor performed an internal review of the entity’s NERC Compliance Program in the summer of 2018, they did not find evidence or documentation that the entity had adhered to PRC‐019‐2 R1. The root cause of this noncompliance was that the entity did not have an adequate verification control to assure that the Contractor responsible for NERC compliance took the steps necessary to be fully compliant with PRC‐019‐2 R1. This noncompliance involves the management practices of external interdependencies and verification. External interdependencies is involved because the noncompliance arose from the failure of a contractor and the entity’s inadequate oversight of that contractor. Verification management is involved because the entity failed to confirm that the Contractor was properly performing its NERC compliance functions. The noncompliance began on February 14, 2017, the date the entity was required to comply with PRC‐019‐2 R1. The noncompliance ended on November 16, 2018, the date the entity completed the coordination and verification of facilities required by PRC‐019‐2 R1.

Risk Assessment

This noncompliance posed a moderate risk and did not pose a serious or substantial risk to the reliability of the bulk power system based on the following factors. The risk posed by this instance of noncompliance is the discoordination of voltage controls which can result in a generator falsely tripping. The risk is not minimal because of the long duration of this noncompliance as the entity did not test any units until after the 80% deadline of July 1, 2018 (more than two years after the initial implementation date of July 1, 2016).However, the risk is not serious because the entity missed the 80% deadline by less than five months, and the entity identified the issue prior to the deadline to test 100% of its units (July 1, 2019) and was thereafter on schedule to meet the 100% deadline). The entity provides 750 MWs in generation with a gross capacity factor of 65.49%. ReliabilityFirst also notes that there is no record of a generator trip arising from incorrect relay or generator settings. Out of the 230 settings reviewed, 25 settings were changed (11%). Of the 25 settings that were changed, seven of those changes were greater than a 10% change. No harm is known to have occurred. ReliabilityFirst considered the entity’s compliance history and determined there were no relevant instances of noncompliance.

Mitigation

To mitigate this noncompliance, the entity: 1) engaged GE to provide a coordination report demonstrating that generating units, synchronous condenser voltage regulating controls, limit functions, equipment capabilities, and Protection System

settings are set per Section G Attachments 1, 2, and 3; 2) created recurring automated task reminders in Gensuite to remind the Plant Management to coordinate the voltage regulating system controls, limiters and protection functions with the applicable

equipment capabilities and settings of the applicable Protection System devices and functions at a maximum of every five calendar years. Gensuite will send an automated email to Plant Management as a reminder to complete this task several times prior to the due date;

3) implemented a documented PRC‐019 procedure that provides guidance for the entity to follow in order to ensure compliance with PRC‐019.






RFC2018020612 PRC‐019‐2 R1 CPV Maryland, LLC NCR11706 2/14/2017 11/16/2018 Self‐Report Completed ReliabilityFirst has verified the completion of all mitigation activity.






RFC2018020613 VAR‐002‐4 R2 CPV Maryland, LLC NCR11706 2/23/2017 10/9/2018 Self‐Report Completed Description of the Noncompliance (For purposes of this document, each noncompliance at issue is described as a “noncompliance,” regardless of its procedural posture and whether it was a possible, or confirmed noncompliance.)

On October 22, 2018, the entity submitted a Self‐Report to ReliabilityFirst stating that, as a Generator Operator, it was not in compliance with VAR‐002‐4 R2. On February 14, 2017, the entity brought a natural gas‐fired 2x1 combined cycle 745 MW electric generation facility (the Plant) into commercial operation. Prior to the start of commercial operations, the entity engaged a third party operations and maintenance contractor (the Contractor) to operate the Plant and to perform services necessary to ensure that the entity was in compliance with its obligations relating to the Plant. (The entity was responsible for overseeing the Contractor’s NERC compliance program. The entity performed its oversight duty by participating in multiple meetings with the Contractor before, during, and after commissioning of the Plant; communicating with the Contractor personnel who were on site at the Plant regarding compliance; and communicating with the Contractor’s corporate NERC personnel regarding compliance of the Plant.) During his time with the entity, the Contractor turned over the management team at the Plant significantly, including three different plant managers. On July 23, 2018, based on ineffective communication and a lack of responsiveness from the Contractor, the entity replaced the Contractor with a new operations and management contractor (the Second Contractor). Upon hiring the Second Contractor, the entity directed the Second Contractor to perform a comprehensive review of the entity’s compliance program as it relates to the NERC standards which apply to a Generator Owner/Generator Operator. The Second Contractor identified the following noncompliance. While the entity had a procedure in place that outlined the compliance requirements of VAR‐002‐4 and required actions of the Contractor; the Contractor failed to perform them, or failed to document performance resulting in this noncompliance. Specifically, the Contractor was to monitor voltage to ensure the Plant met the Transmission Operator’s (TOP) voltage schedule as required by VAR‐002‐4.1 R2. The entity operates within the Potomac Electric Power Company (PEPCO) zone of PJM and generates electricity at the 230 kV voltage level. The Second Contractor’s internal review included a review of previous correspondence between the TOP and the entity regarding voltage schedule specification. The TOP confirmed on September 28, 2018, that the entity must comply with the default PJM voltage of 235.0 KV +/‐4.0 kV. The Second Contractor then retrieved five minute operating data and examined every instance where the entity deviated outside the voltage scheduled for longer than 30 minutes without notifying the TOP. During the period starting February 23, 2017 and ending October 1, 2018, the entity deviated the required voltage schedule 125 times to varying degrees and all to the high side of 239 kV. Following is a table of the deviations:

Deviation Range Total Deviations (125) <0.5 kV 75 >0.51 kV <1.0 kV 23 >1.01 kV <1.5 kV 17 >1.51 kV <2.0 kV 6 >2.0 kV (2.06 kV, 2.55 kV, 2.46 kV, and 2.1 kV)

4

The root cause of this noncompliance was that the entity did not have an adequate verification control to assure that the Contractor responsible for NERC compliance took the steps necessary to be fully compliant with VAR‐002‐4. This noncompliance involves the management practices of external interdependencies and verification. External interdependencies is involved because the noncompliance arose from the failure of a contractor and the entity’s inadequate oversight of that contractor. Verification management is involved because the entity failed to confirm that the Contractor was properly performing its NERC compliance functions. The noncompliance began on February 23, 2017, the date the entity was required to comply with VAR‐002‐4 R2. The noncompliance ended on October 9, 2018, the date the entity completed its Mitigating Activities.

Risk Assessment

This noncompliance posed a moderate risk and did not pose a serious or substantial risk to the reliability of the bulk power system (BPS) based on the following factors. The risk posed by this instance of noncompliance is allowing voltage schedule deviations at levels detrimental to the BPS’s voltage level without the TOP having knowledge, which could result in harm to the BPS. The risk is not minimal because of the extended duration and multiple instances. The risk is not serious because a majority of the voltage deviations were less than 0.5 kV outside of the voltage schedule, and all but 4 deviations were less than 2 kV outside of the voltage schedule. No harm is known to have occurred.






RFC2018020613 VAR‐002‐4 R2 CPV Maryland, LLC NCR11706 2/23/2017 10/9/2018 Self‐Report Completed ReliabilityFirst considered the entity’s compliance history and determined there were no relevant instances of noncompliance.

Mitigation

To mitigate this noncompliance, the entity: 1) trained all operations personnel on VAR‐002‐4.1 R2 compliance through a Learning Management System. The entity also trained operations personnel on how to adjust generator voltage on the Plant

load control page of the Distributed Control System; 2) laminated and taped to the control room operating board, the required voltage schedule and instructions on who to contact in the event the Plant is unable to maintain the requisite voltages schedule.

The entity also dedicated a Distributed Control System monitor to show operating personnel the Plant’s line voltage at all times, which includes a visual alarm that switches the screen to red when the line voltage reaches 232.0kV and 238.0kV; and

3) implemented internal controls to assist management with on‐going monitoring activities, as follows: (a) The control room shift turnover, which occurs every 12 hours, now includes the following two questions: (i) was the voltage schedule of 235kV (+/‐ 3kV) maintained during the shift?; and (ii) if not, was Pepco and PJM notified? (b) Plant Management will now complete a weekly Compliance Attestation after he/she reviews five minute voltage data for the prior week to determine if the Plant maintained the requisite voltage schedule during that week. The Compliance Attestation for weekly VAR‐002‐4.1 R2 review will be filed electronically at the Plant. If this review indicates the operating voltage was outside the schedule, Plant Management will confirm the appropriate notification was given to Pepco and PJM and was documented on the aforementioned shift turnover. (c) Plant Management is now required to complete a Compliance Attestation the month following each calendar quarter to summarize how the entity complied with the four applicable requirements of the VAR‐002 Standard. Gensuite will automatically notify Plant Management to complete this Attestation at the end of each quarter.






SERC2018019676 PRC-001-1.1(ii) R3 Columbia Energy LLC (Columbia) NCR11480 01/18/2017 10/03/2017 Self-Report Completed


On May 11, 2018, Columbia submitted a Self-Report stating that, as a Generator Operator, it was in noncompliance with PRC-001-1.1(ii) R3. Columbia did not coordinate protective system changes with its Transmission Operator (TOP) and Host Balancing Authority (BA). On January 18, 2017, Columbia made changes to relay settings to meet compliance requirements of PRC-024-2. On October 3, 2017, during an internal compliance review, Columbia could not locate evidence to demonstrate it coordinated the relay changes with the TOP and BA. Columbia contacted the TOP and BA to request evidence that the facility coordinated the changes, but the TOP and BA did not provide any documentation to demonstrate that the coordination occurred. Columbia made 24 relay setting changes for which it is had no evidence of coordination with the TOP and BA. This noncompliance started on January 18, 2017, when Columbia changed relay settings without coordinating with its TOP and BA, and ended on October 3, 2017, when Columbia coordinated the relay setting changes with the TOP and BA. The cause of this noncompliance was a lack of a compliance facilitation procedure that requires coordination with its TOP and BA for protective system changes. Columbia made the relay setting changes for PRC-024 compliance but failed to realize that those changes should have been coordinated with the TOP and BA.

Risk Assessment

This noncompliance posed a moderate risk and did not pose a serious or substantial risk to the reliability of the bulk power system. Columbia’s failure to coordinate 24 relay changes for approximately nine months on a 749 MW facility could impact real-time operations. Specifically, it could create issues where facilities trip in unexpected ways or at unexpected times, result in inadequate protection for the interconnecting facility, and result in the BA and TOP not having an accurate depiction of its system. However, the relay setting changes that Columbia made only impacted its generators with respect to the trigger points as to when the units would trip in response to an event on the system. The relay setting changes results in the generators being less likely to trip for excursions. While using the previous settings, the TOP and BA would anticipate the generation trip sooner than what would occur with new settings implemented to meet compliance requirements of PRC-024-2. Additionally, the BA and TOP did not report any concerns with the relay setting changes made by Columbia after receiving notification. No harm is known to have occurred. SERC considered Columbia’s compliance history and determined that there were no relevant instances of noncompliance. Note: Columbia will be deregistered upon acceptance of this filing by FERC. SCE&G has purchased this facility.

Mitigation

To mitigate this noncompliance, Columbia: 1) sent an e-mail to its TOP and BA on October 3, 2017, describing the changes made to the relays to comply with PRC-024. The TOP and BA sent a response to Columbia on October 4, 2017 acknowledging receipt of the e-mail; and 2) implemented a compliance facilitation procedure. This procedure requires the performance of a monthly compliance assessment of the NERC Standards. The procedure explicitly states that new/changes to protection systems will be coordinated with TOP and BA, per PRC-001-1.1(ii). Training was completed on this procedure on August 10, 2017.





SERC2019021061 MOD-026-1 R6 Tennessee Valley Authority (TVA) NCR01151 02/03/2016 01/03/2019 Self- Report Completed


On February 15, 2019, TVA submitted a Self-Report stating that, as a Transmission Planner (TP), it was in noncompliance with MOD-026-1 R6. TVA did not provide a written response to the Generator Owner (GO) within 90 days of the verified excitation control system or plant volt/var control function model verification that the model was usable or not useable. On November 19, 2018, TVA determined that it failed to provide a written response to the GO, within 90 days of the data submission by the GO, for four units in November 2015, 12 units in April 2016, 20 units in June 2016, one unit in May 2017, and one unit in August 2017. Additionally, in some cases, TVA did not determine if the model was or was not useable within 90 days of model data submission. Ultimately, TVA determined that all of these models were useable. The model date for the five periods involved 38 units, which represented 4,590 MVA of TVA’s 33,526 MVA of generation (13.7%). As part of its mitigating activities, TVA completed an extent-of-condition review and identified eight additional instances that included 30 additional units. For these additional instances, the number of days that elapsed from the date of submission to the date TVA responded ranged from 91 days to 1,226 days. This noncompliance started on February 3, 2016, when TVA was required to provide the written response to the GO, and ended on January 3, 2019, when TVA submitted the last overdue response to the GO. The cause of the noncompliance was management oversight. Management failed to verify that all necessary internal controls, e.g., tracking software and an assignment of a secondary delegate responsible for tracking model submissions, were implemented to ensure timely responses to the GO. The TVA TP used an email inbox for the receipt of model data from GOs and assigned a single subject-matter-expert (SME) to monitor the mailbox. The SME failed to follow-up with written notification after discussions with the GO regarding models. The SME also went on leave after receiving the notification and failed to test the models and respond to the GO after returning from leave. Additionally, when the SME received a batch of submissions, the SME used the date of the last submission of the batch as the date to respond by, instead of addressing all units from the submission in the response.

Risk Assessment

This noncompliance posed a moderate risk and did not pose a serious or substantial risk to the reliability of the bulk power system (BPS). TVA’s failure to notify the GO, within 90 days, that the models submitted are usable or not usable, could have led to the GO using incorrect models that could have resulted in erroneous assessments or as part of an incorrect corrective action plan. The noncompliance included 68 units. The response to the GO data submission ranged from 94 days to 1,226 days. TVA was able to include the models that were slightly late in the applicable SERC data submittal, however, for those that had longer delays, TVA was unable to include those models in the applicable SERC data submittal. Although TVA ultimately determined that the model information was useable, had TVA determined that the models were unusable, the GO would have been required to re-engage the contractor that had originally provided the verified models, which would have resulted in additional delays in obtaining a usable verified model for these units. In addition, TVA only identified the instant noncompliance when reviewing data for a previous GO MOD noncompliance. No harm is known to have occurred. SERC considered TVA’s compliance history and determined that there were no relevant instances of noncompliance.

Mitigation

To mitigate this noncompliance, TVA: 1) sent replies to all outstanding model validations as required by the Standard; 2) assigned a delegate in addition to the SME to assist in tracking the model submissions; 3) implemented a new action tracking software. As models are received by the TP, they will be entered into the system, which will then track the appropriate due dates for the replies. E-mails from the software are automatically sent to the SME each week, along with an additional e-mail as the due date for the reply approaches to both the SME and the assigned delegate; 4) conducted an extent-of-condition (EOC) review to verify the results of the GO EOC analysis to be certain that all received models reports have been tested and the appropriate replies to the GO have been performed; and 5) created a new Technical Procedure to document the process that the TP uses to process and reply to incoming MOD-026 model submissions with enough details such that a new employee will be able to track, process, and reply to a new MOD-026 model supplied by a GO. All impacted employees were notified and trained on the contents of the procedure.





TRE2017017837 PRC-005-1.1b R2 Sharyland Utilities, LLC, (SU) NCR04119 04/01/2015 06/13/2018 Self-Report Completed


On June 30, 2017, SU submitted a Self-Log stating that, as a Transmission Owner (TO), it was in noncompliance with PRC-005-6 R3. Specifically, SU reported that it did not maintain its Protection Systems included within the time-based maintenance program in accordance with the minimum maintenance activities and maximum maintenance intervals prescribed by the Standard. On January 9, 2018, SU provided evidence of additional instances of noncompliance with PRC-005-6 R3, as well as instances of noncompliance with predecessor Standard PRC-005-1.1b R2. After subsequent review, Texas RE determined that the instances of noncompliance posed a moderate risk to the reliability of the bulk power system. As a result, Texas RE removed this issue from the Entity’s Self-Log.

Texas RE determined that, consistent with SU’s June 30, 2017 Self-Report, SU failed to complete battery inspections at five of its seven substations within the maximum maintenance interval prescribed in Table 1-4(b) of PRC-005-6. The Standard requires that for Valve Regulated Lead-Acid (VRLA) station batteries that do not have internal ohmic value monitoring and alarming, an inspection of the condition of the batteries by measuring battery cell/unit internal ohmic values, must be conducted no less than every six calendar months. SU conducted the overdue inspections on April 11, 2017, 11 days after the applicable due date.

Texas RE determined that, consistent with SU’s January 9, 2018 submission of additional evidence, SU also failed to conduct protection system maintenance in accordance with PRC-005-1.1b R2, at its Railroad 138kV substation since the in-service date of the Facility, April 1, 2007. According to SU’s Protection System Maintenance Program (PSMP) in effect at the time, such testing and maintenance was due every eight years, which indicated that the due date for the Railroad Facility was April 1, 2015. SU completed the required testing and maintenance at the Railroad Facility on June 13, 2018, three years, two months, and 13 days after the testing and maintenance was due.

Texas RE determined that, consistent with SU’s January 9, 2018 submission of additional evidence, SU was also noncompliant with the Implementation Plan for PRC-005-6 R3 in that it failed to test at least 30% of its vented lead-acid (Table 1-4a), and 30% of its nickel-cadmium station batteries (Table 1-4c) by April 1, 2017. SU completed testing on 13 of its 42 (31%) vented lead-acid batteries, and 3 of 3 (100%) of its nickel-cadmium batteries, on November 8, 2017, 221 days after the applicable due date.

The root cause of this noncompliance was based on several factors. First, SU’s PSMP lacked precision. SU failed to draft its PSMP with sufficient detail as to properly identify the responsible staff members, and the particular testing responsibilities required by the Standard. Second, SU failed to shift responsibility for compliance to alternative staff members during staffing changes. SU failed to ensure that the testing required by the standard was completed notwithstanding the fact that the staff position responsible for this activity was vacant. Third, SU’s failure to conduct a proper extent-of-condition analysis when the noncompliance was first discovered resulted in the discovery of additional instances of noncompliance after SU’s initial Self-Report.

This noncompliance started on April 1, 2015, when SU failed to complete the required testing and maintenance at the Railroad substation, and ended on June 13, 2018, when SU completed the required testing and maintenance at the Railroad substation.

Risk Assessment This issue posed a moderate risk and did not pose a serious or substantial risk to the reliability of the bulk power system. PRC-005-6 is intended to ensure that proper testing and maintenance of Protection Systems occurs so that these devices remain in good working order. The risk to the bulk power system caused by SU’s noncompliance was reduced by the fact that SU's transmission substation batteries and other Protection Systems, including the Amarillo-area VRLA batteries, are monitored continuously via SCADA. Any failure could have been addressed immediately. SU has not experienced a transmission failure due to Protection System hardware failure. Also, for some of the substations at issue, Sharyland has a looped transmission system that connects all substations to multiple feed points in the ERCOT grid, helping to stabilize voltage as well as feed from different points during emergency situations. For the Railroad substation, there are two feeds from two separate substations. The failure or loss of one of these lines does not result in an overload condition, thus allowing the high-voltage DC line to continue to be rated for full import/export capability even during the loss of one line. No harm is known to have occurred.

Texas RE considered the SU’s compliance history and determined there were no relevant instances of noncompliance. Mitigation To mitigate this noncompliance, SU:

1) completed the required testing and maintenance activities;2) created the staff position of Transmission Maintenance Coordinator with the responsibility of assisting the Maintenance Supervisor with ensuring that testing and maintenance inspections are

carried out; and3) revised its PSMP so that it is more precise and identifies the specific staff members and alternates with the responsibilities to complete the tasks required by the Standard.

Texas RE has verified the completion of all mitigation activity.






RFC2019021380 PRC‐005‐6 R3 Benton County Wind Farm, LLC NCR00170 1/1/2018 10/4/2018 Compliance Audit Completed Description of the Noncompliance (For purposes of this document, each noncompliance at issue is described as a “noncompliance,” regardless of its procedural posture and whether it was a possible, or confirmed noncompliance.)

On April 15, 2019, ReliabilityFirst determined that the entity, as a Generator Owner, was in noncompliance with PRC‐005‐6 R3 identified during a Compliance Audit conducted from September 21, 2018 through January 23, 2019.

Under PRC‐005‐6 R3 the entity was required to verify maintenance for 30% of components by December 31, 2017 for the following: (a)Table 1‐1: 6‐year relay verification; (b) Table 1‐2: 6‐year communications systems verification; (c) Table 1‐4: 6‐year battery load testing; (d) Table 1‐5: 6‐year circuit breaker trip coil verification; and (e) Table 1‐5: 6‐year electrical operation of electromechanical lockout device verification.

The entity did not perform any of the above verifications within six calendar years, and therefore, failed to comply with PRC‐005‐6. However, the entity had performed all four‐month and eighteen‐month battery testing required by PRC‐005‐6.

The root cause of this noncompliance was the entity’s failure to exercise sufficient oversight of a contractor who was tasked with managing NERC compliance. Specifically, the entity employed a contractor to manage all NERC compliance responsibilities, but failed to verify that the contractor was completing the necessary work to maintain compliance.

This noncompliance involves the management practices of grid maintenance and workforce management. Grid maintenance is involved because the entity failed to assure that the requisite maintenance verification activities were being performed to comply with PRC‐005‐6. Workforce management is involved because the entity failed to oversee the performance of an independent contractor who was responsible for managing NERC compliance.

This noncompliance started on January 1, 2018, when six calendar years had elapsed since the entity performed the requisite verifications under PRC‐005‐6 and ended on October 4, 2018, when the entity completed its final PRC‐005‐6 verification activities.

Risk Assessment This noncompliance posed a moderate risk and did not pose a serious or substantial risk to the reliability of the bulk power system (BPS) based on the following factors. The risk posed by this noncompliance is that not timely maintaining and testing all generation Protection System devices can have a negative effect on the reliable operation of the BPS. The risk here is partially reduced because the entity is a 150 MVA generating facility with a capacity factor below 30%. Further reducing the risk, the entity performed all four‐month and eighteen‐month battery testing required by PRC‐005‐6. Additionally, this is a phased‐in implementation, and the missed deadline was only an interim deadline; the entity reached 100% compliance before the 100% compliant date. ReliabilityFirst notes that upon testing, 11 of the 12 components did not show any deficiency. The risk is elevated because the entity failed to identify and remediate the noncompliance on its own and one cell of one of the batteries when tested was shown to be deficient. The bad cell had significantly below normal and accepted voltage ranges, at times as low as 83% below normal. The entity had to replace the battery, and the risk posed by the deficient battery was a failure for the Protection Systems to operate properly, potentially causing a total loss of generation. No harm is known to have occurred.


1) replaced battery #13;2) performed the PRC‐005‐6 R3 6‐year battery performance verification; and3) implemented a tracking system which will send multiple reminders for compliance activities necessary under PRC‐005‐6.






RFC2018019787 PRC‐005‐6 R1 City of Rochelle NCR00721 1/1/2016 9/25/2018 Compliance Audit Completed Description of the Noncompliance (For purposes of this document, each noncompliance at issue is described as a “noncompliance,” regardless of its procedural posture and whether it was a possible, or confirmed noncompliance.)

On May 18, 2018, ReliabilityFirst determined that the entity, as a Transmission Owner, was in noncompliance with PRC‐005‐6 R1 identified during a Compliance Audit conducted from September 28, 2017 through May 4, 2018. During the audit, ReliabilityFirst determined that the entity did not provide evidence that it had a documented, dated PRC‐005‐6 Protection System Maintenance Plan (PSMP) in place as of the audit notification date (i.e., June 12, 2017). Specifically, the entity’s PSMP did not identify which maintenance method it used to address each Protection System Component type as required by R1 Part 1.1. It identified a maintenance method for only relays. Additionally, the entity’s PSMP did not include the applicable monitored component attributes applied to the Protection System Component Type where monitoring is used to extend the maintenance intervals as required by R1.2. Notably, the entity did submit evidence indicating that it was actually performing time‐based maintenance for all Protection System Component Types. The root cause of this noncompliance was the entity’s general lack of awareness regarding applicable compliance obligations. This root cause involves the management practice of workforce management, which includes providing training, education, and awareness to employees. This noncompliance started on January 1, 2016, when the entity was required to comply with PRC‐005‐6 R1 and ended on September 25, 2018, when the entity implemented a full and complete PSMP.

Risk Assessment

ReliabilityFirst determined that the subject noncompliance posed a moderate risk to the reliability of the bulk power system (BPS) based on the following factors. The risk posed by failing to properly document these aspects of the entity’s PSMP is that the entity may not perform the requisite testing at the defined intervals, which increases the likelihood that the devices could fail to operate as expected, adversely affecting the reliability of the BPS. The risk is not minimal in this case due to the amount of time that the issue persisted and the fact that the issue was identified during an audit. The risk is not serious or substantial in this case based on the fact that the entity provided evidence that it was actually performing time‐based maintenance, which reduces the risk that these devices may fail to operate as expected. No harm is known to have occurred. ReliabilityFirst considered Rochelle’s compliance history and determined there were no relevant instances of noncompliance.

Mitigation

To mitigate this noncompliance, the entity: 1) drafted a Protection System Maintenance and Testing Program (PSMP) that reflects its approach to maintain Bulk Electric System equipment; 2) approved the PSMP; and 3) uploaded the PSMP and attachment A to CDMS mitigation plan. ReliabilityFirst has verified the completion of all mitigation activity.





SERC2017016784 FAC-014-2 R2 Duke Energy Progress, LLC (DEP) NCR01298 02/18/2014 10/15/2018 Compliance Audit Completed


During a Compliance Audit conducted per an existing multi-regional registered entity agreement from September 6, 2016 through January 11, 2017, SERC determined that Duke Energy Florida (DEF), as a Transmission Operator, was in noncompliance with FAC-014-2 R2. DEF failed to establish System Operating Limits (SOLs) in accordance with its Reliability Coordinator’s (RC’s) SOL Methodology. FRCC’s SOL Methodology for the Operating Horizon (FRCC-MS-OP-009r1) defines SOLs as the value (e.g., MW, MVar, Amperes, Frequency or Volts) that satisfies the most limiting of the prescribed operating criteria for a specified system configuration to ensure operation within acceptable reliability criteria. The methodology also states that SOLs shall not exceed associated facility ratings and that SOLs must provide Bulk Electric System performance, such that, following certain single contingencies identified in the requirements of the methodology, the system shall demonstrate transient, dynamic, and voltage stability; all Facilities shall be operating within their facility ratings and within their thermal, voltage, and stability limits; and cascading or uncontrolled separation shall not occur. SERC reviewed FRCC’s Winter 2016 Seasonal Assessment and Summer 2016 Seasonal Assessment and discovered that DEF identified several single contingency overloads that required corrective actions to mitigate overloads. However, DEF did not identify those contingency overloads as SOLs as required by its SOL Methodology. DEF explained that it believed the requirements of another Reliability Standard, FAC-011-2 R2, allowed it to excuse designating those overloads as SOLs because it had identified operator actions that would restore acceptable system performance. Because of that interpretation, DEF did not determine those contingencies to be SOLs, but it did establish and did communicate corrective actions in the operating horizon. This noncompliance started on February 18, 2014, the earliest known submission of a list of SOLs that DEF developed in accordance with its RC’s methodology, and ended on October 15, 2018, when DEP established SOLS for its transmission lines and revised its procedure to reflect the change in philosophy for establishing SOLs. The cause of this noncompliance was management oversight. Management failed to implement a practice to ensure its staff properly interpreted the standard and gained familiarity with the expectations to fully implement the standard. DEF erroneously believed that FAC-011-2 R2 excused the overloads from being designated as SOLs because DEF had identified operator actions that would restore acceptable system performance.

Risk Assessment

This noncompliance posed a moderate risk and did not pose a serious or substantial risk to the reliability of the bulk power system (BPS). By not establishing SOLs for Facilities per the RC’s methodology, DEF absolved itself from compliance obligations of operating to SOLs as required under TOP-002-2.1b, TOP-003-1, TOP-004-2, TOP-007-0, and TOP-008-1. Under those requirements, DEF would be required to communicate with the RC and to coordinate its corrective measures with the RC. Failing to do so could result in actions adverse to the operation of interconnected transmission systems. Although DEF did not identify the contingencies as SOLs, DEF did establish corrective action plans in the operating horizon for each of the contingencies. DEF did establish SOLs for Interconnection Reliability Operating Limit-related interconnections with neighboring transmission systems. No harm is known to have occurred. SERC considered DEF’s compliance history and determined that there were no relevant instances of noncompliance. SERC considered the compliance history of DEF’s affiliates Duke Energy Progress, Duke Energy Carolinas, and Duke Energy Corporation and determined that there were no relevant instances of noncompliance.

Mitigation

To mitigate this noncompliance, DEP: 1) developed SOLs for its 500 kV transmission lines and its 500/230 kV transformers and submitted those SOLs for approval to its RC; 2) developed SOLs for its 230 kV transmission lines and 230/115 kV transformers and submitted those SOLs for approval to its RC; 3) developed SOLs for its 115 kV transmission lines and submitted those SOLs for approval to its RC; 4) revised SORMF-TD-03 (DEF-TOP-OP22) to reflect the change in philosophy for establishing SOLs based in its new understanding of FAC-011-2 R2; and 5) notified affected operators of changes via email and added the revisions to future training.





SERC2017018144 VAR-002-4 R2 Duke Energy Carolinas, LLC (DEC) NCR01219 12/16/2016 10/10/2017 Self-Report

Completed


On August 9, 2017, DEC, on behalf of Duke Energy Corporation (DECorp), submitted a Self-Report stating that, as a Generator Operator (GOP), it was in noncompliance with VAR-002-4 R2. Also on that date, Duke Energy Progress, LLC (DEP), Duke Energy Florida, LLC (DEF), and DEC submitted Self-Reports, with tracking numbers SERC2017018142, SERC2017018143, and SERC2017018145, respectively, making the same assertion. DEC, DEP, DEF, and DECorp are subject to a Multi-Regional Registered Entity (MRRE) agreement and, as such, SERC rolled the Self-Reports for DEP, DEF and DEC into the instant Self-Report. Each Entity reported multiple instances where it failed to maintain the voltage schedule provided by their Transmission Operator (TOP) or otherwise failed to meet the conditions of notification for deviations. There are a total of twelve instances that are discussed below in chronological order. Hereafter, this documents refers to all four affiliates, collectively, as the Entities. From December 16, 2016 through March 15, 2017, DEC identified four instances of violations. In the first instance, DEC identified four periods where Allen Units 3 and 4 exceeded its 230 kV voltage schedule by up to 0.6 kV. In the second instance, DEC identified twenty six periods where Rogers Unit 5 and Unit 6 exceeded its 230 and 500 kV voltage schedules by up to 1 kV. In the third instance, DEC identified twenty-one periods where Keowee Hydro exceeded its reactive power schedule by up to 180 kVAR. In the fourth instance, DEC identified three periods where the Wateree Station exceeded its 100kV voltage schedule by up to 0.4 kV. DEC made no notifications to its TOP for any of these instances. The fifth instance occurred on December 16, 2016 when DECorp’s Vermillion Station operated at an average of 1% below its voltage schedule for 5 hours. DEC made no notification to its TOP. From December 18, 2016 through March 15, 2017, DEF identified the sixth instance, where Crystal River units had 98 periods where it operated at a maximum of 1 kV above and below its 230 kV voltage schedule and 0.4 kV below its 500 kV voltage schedule for more than 30 minutes. DEF made no notifications to its TOP. On January 8, 2017, DECorp identified the seventh instance where Edwardsport operated 0.6% below its voltage schedule for approximately 150 minutes. DECorp made no notification to its TOP. The eighth instance occurred from January 8, 2017 through February 20, 2017. DEF identified four periods where the units at Osprey Station operated at a maximum of 1 kV below its 230kV voltage schedule for more than 30 minutes. DEF made no notifications to its TOP. From January 21, 2017 through April 18, 2017, DEP identified a ninth instance where the Robinson Nuclear Plant operated up to 2.2kV above of its maximum voltage schedule 5.6% of the time. DEP made no notifications to its TOP. On March 8, 2017, DECorp identified a tenth instance where Cayuga Unit 1 operated an average of 1% below its minimum voltage schedule when it was in start-up with the automatic voltage regulator (AVR) that was not in service. This occurred for approximately 90 minutes until the AVR was back in service. DECorp made no notification to its TOP. The eleventh instance was discovered on March 13, 2017. DECorp’s Gallagher Unit 4 was an average of 1% below its voltage schedule for an undeterminable amount of time. The unit was out of compliance for at least 91 minutes, but may have been out of compliance up to 196 minutes. The twelfth instance occurred on October 10, 2017, at approximately 9:50 a.m., when DEC’s Wateree Unit 4 exceeded its 100 kV voltage schedule by a maximum of 0.7 kV for about 60 minutes. A programmable logic controller was set in ‘Off-Peak’ mode when it should have been in ‘Peak’ mode. DEC notified its TOP upon discovery, which ended the violation. DEC submitted this as an expansion of scope to SERC2017018144. The noncompliance started on December 16, 2016, when DEC’s Allen Units operated outside its voltage schedule without notification to the TOP, and ended on October 10, 2017, when DEC’s Wateree Unit 4 notified its TOP. The causes of these violations were a combination of inadequate operator training, inadequate alarms, insufficient alarm response procedures, and inadequate normal operating procedures. Instances one, two, three, four, five, seven, eight, eleven and twelve had elements of all four causes previously listed. The causes of instance six, nine, and ten were inadequate operator training and inadequate normal operating procedures.



Risk Assessment

This noncompliance posed a moderate risk and did not pose a serious or substantial risk to the reliability of the Bulk Power System (BPS). The Entities failed to maintain voltage or reactive power in accordance with the assigned schedule at multiple locations, with various causes, for long periods of time. The violations could have caused equipment damage, voltage drop, or, in worst case scenario, led to system instability. However, no single instance of the violations had a voltage deviation greater than 2% of the voltage schedule. Additionally, most violations had only moderate lengths of time outside their voltage schedule. While these mitigating circumstances by no means extinguish the risk, they do make the worst case scenario, Bulk Electric System instability, unlikely. No harm is known to have occurred. SERC considered the Entities’ VAR-002-4 R2 compliance history in determining the disposition track. The relevant prior noncompliance with VAR-002-4 R2 includes: NERC Violation ID SERC2013012516, SERC2015015365, and SERC2015015435. In SERC2013012516, four hydro units were not operating in accordance with the reactive power schedule provided by the TOP, and one generating station exceeded its allowable voltage schedule on one occasion. In SERC2015015365, a new voltage schedule to a nuclear power station was not implemented. In SERC2015015435, one unit operated outside of its voltage schedule for more than 30 minutes on seven consecutive days. SERC determined that the Entities’ compliance history should not serve as a basis for applying a penalty. In SERC2013012516, the hydro units typically operated remotely to provide peaking power and water management, not voltage support. Additionally, the mitigation for the prior noncompliance was completed over three years from start date of the instant noncompliance. In SERC2015015365, the noncompliance was an isolated instance and the underlying causes of the prior noncompliance was different than the instant noncompliance. The prior noncompliance was caused by inadequate training, as the person responsible for ensuring implementation of the new voltage schedule was on vacation, and the staff on duty did not understand now to implement the schedule. SERC2015015365 was an isolated instance where DECorp was unaware of the voltage exceedance. The Entities were individually responsible for drafting and implementing their own mitigation plan, and thus, the mitigation for the prior instance could not have prevented the instant noncompliance.

Mitigation

To mitigate this noncompliance, the Entities, in Fossil Hydro Operations (FHO):

1) revised OPS- PGDX- 00009 Operations NERC/SERC Compliance and trained applicable staff to ensure alignment with current VAR-002-4 standard requirements; 2) requested stations to identify operational procedures for potential gaps then updated said procedures; 3) required reading of the updated operational procedures, which was later formalized into its own training procedure; 4) implemented a VAR-002 fleet wide communication protocol; 5) documented and implemented a common tool and protocol for logging generation communication; 6) updated the current Corrective Action Program procedure and communicated the updates across the fossil and hydro organizations; 7) reviewed the alarm priority logic and corrected to enable implementation of high priority voltage alarms to effectively alerted them prior to reaching minimum or maximum thresholds; 8) ensured the GOP utilized common points, or received concurrence for the points being used from the TOP for monitoring voltage schedule compliance. 9) defined the responsibility for maintenance and calibration of the common point; 10) required all implementation of Preventative Maintenance Plan (PM) to verify Distributive Control System set points were consistent to operate within their regions' voltage schedules; and 11) utilized existing tools to ensure PGL362 Training, Generator Operation for Maintaining Network voltage schedules VAR-002 was taken by applicable FHO personnel on an annual basis.

To mitigate this noncompliance, the Entities, in Nuclear Operations: 1) implemented Interim guidance for VAR-002-4 at each Nuclear Station with procedural gaps for communications with the Transmission Control Center/Energy Control Center (TCC/ECC) during

voltage schedule excursions; 2) conducted a review of the Nuclear Fleet Operations Procedure that showed McGuire and Catawba needed procedure updates to make clear Control Room Communication requirements with

Transmission via the TCC/ECC as described above; and 3) communicated to the Duke Nuclear Fleet Operations CFAM (Corporate Functional Area Manager) expectations of Control Room communications with the TCC/ECC for voltage schedule, VAR

Schedule, AVR, and PSS issues to be immediately communicated and discussed with Transmission Operations (via TCC/ECC); and reviewed Robinson Plant's voltage schedule for potential update due to Robinson's position on the grid.






RFC2018020788 TOP‐001‐3 R9 Northern Indiana Public ServiceCompany LLC NCR02611 4/27/2018 12/11/2018 Self‐Report Completed


On November 29, 2018, the entity submitted a Self‐Report stating that, as a Transmission Operator, it was in noncompliance with TOP‐001‐3 R9. On August 19, 2019, the entity submitted a Self‐Report stating that, as a Transmission Operator, it was in noncompliance with TOP‐001‐3 R9. That second Self‐Report was consolidated into the first Self‐Report.

There are two separate instances in this noncompliance.

First, on April 27, 2018, at 16:40, entity staff discovered that the Real Time Contingency Analysis (RTCA) program status was in the “pending” state and the last completion had occurred at 13:23. At 16:41 the employee who discovered the issue notified MISO (the Reliability Coordinator (RC)) that the RTCA was not solving. At 16:42 the employee communicated with an internal CIP applications expert on how to restart the RTCA. At 16:49, the RTCA, following the restart, began running again and the employee notified MISO that the entity RTCA was once again running.

The root cause of this first noncompliance was that the entity did not have sufficient alarming in place to notify personnel when the RTCA stalled in this type of situation. Here, the RTCA is tethered to a secondary process, and that process stalled. The RTCA does not run unless the secondary process completes, thus the RTCA stalled as a result of the secondary process stalling.

This first noncompliance involves the management practices of work management and integration management. Work management is involved because the entity did not have sufficient alarming in place to notify relevant personnel of RTCA failure. That lack of alarming resulted in insufficient information being transmitted to MISO. Integration management is involved because failure in one interface resulted in the RTCA interface being offline which indicates problematic interface integration.

This first noncompliance started on April 27, 2018, when the RTCA went offline at 13:23 and ended on April 27, 2018, at 16:45 when the entity confirmed that the RTCA was once again solving following a restart.

Second, at 8:40 on December 11, 2018, the entity initiated an EOP‐008‐1 R7 test of the Operating Plan for loss of Control Center Functionality. (This was an unplanned drill. The entity normally runs through the procedure the day before the actual planned drill to ensure successful transfer of the energy management system (EMS) to the other servers at its disaster recovery site.) The MISO RC and neighboring entities were notified at this time. Also at this time, the entity confirmed that the RTCA was running normally.

RTCA last ran successfully at 10:31. Also, at 10:31, the EMS was switched over from the Primary Control Center (PCC) to the Backup Control Center (BCC). At 10:33, the Transmission System Supervisor (TSS) at the PCC acknowledged numerous switchover alarms, including one for RTNET failure (RTCA is part of RTNET).

At 11:58, System Operators at the PCC noticed RTCA was still stalled at which point they notified MISO. During the entire entity RTCA outage, MISO's RTCA was running and providing a Real‐time Assessment for the entity Transmission Operator Area.

At 12:03, RTCA restarted and began running again. The entity notified the RC that RTCA was once again working.

The root cause of this second noncompliance is that the entity did not have an effective control (alarm) in place to notify personnel when the RTCA stalled in this type of situation. Here, the RTCA stalled due to a duplicated path from host A at the PCC to host C at the BCC because of some unpatched bugs for duplicates paths from the vendor. That lack of notification resulted in insufficient information being transmitted to MISO. (The alarm that the entity installed as a result of mitigation for the first noncompliance did not work for the second noncompliance because this was a different condition than what the entity had observed in the first noncompliance. The stall alarm implemented in the first instance is based on the RTNET/RTCA task in procman stalling. In this case, the problem was in OLNETSEQ, and the tasks it scheduled remained pending, so the RTNET/RTCA tasks were not asked to run.)

This second noncompliance involves the management practices of information management, work management, and external interdependencies. The entity did not confirm with GE that everything had up‐to‐date patches which would have prevented the duplicate path issue that resulted in the RTCA stall. The entity also lacked an effective process to notify personnel when the RTCA failed due to this issue.

This second noncompliance started on December 11, 2018, when the RTCA went offline at 10:31 and ended on December 11, 2018, at 12:02 when the entity confirmed that the RTCA restarted and began running again.

Risk Assessment This noncompliance posed a moderate risk and did not pose a serious or substantial risk to the reliability of the bulk power system (BPS) based on the following factors. The risk posed by this noncompliance is that failure to perform post‐contingency real time analysis could result in the entity not being completely aware of the state of its system for contingent conditions. This could inhibit the






RFC2018020788 TOP‐001‐3 R9 Northern Indiana Public Service Company LLC NCR02611 4/27/2018 12/11/2018 Self‐Report Completed

entity’s ability to develop sufficient actions to prevent potential exceedance of a System Operating Limit, instability, uncontrolled separation, or cascading outages that could adversely impact the reliability of the BPS. The risk is not minimal because of the more than three hour duration of the first instance. The risk is lessened because for the entire noncompliance, MISO's RTCA was running, providing a real‐time assessment for the entity Transmission Operator Area. The risk is also reduced because the duration for the second instance was relatively short (approximately one hour and a half). No harm is known to have occurred. The entity has relevant compliance history. However, ReliabilityFirst determined that the entity’s compliance history should not serve as a basis for applying a penalty because both the root cause and the scope of the noncompliance differ. In the prior noncompliance, the root cause was improper training of notification processes for entity employees; in the current instances of noncompliance the root cause was the lack of an alarm and notification process to let relevant personnel know that the RTCA was not functioning.

Mitigation

To mitigate this noncompliance, the entity configured the Real‐time Contingency Analysis and Real‐time Network Analysis alarming correctly for the stall condition with audible and visible alarms. The entity also instituted the use of an independent timer to remind control room personnel to check the RTCA to be running inside of every 30 minutes, and added a screen that shows system time and last successful RTCA run. ReliabilityFirst has verified the completion of all mitigation activity.





SERC2017016879 FAC-009-1 R1

Entergy - Fossil & Hydroelectric Generation (EntergyFHG)

NCR11167

11/11/2011 04/24/2018 Self-Report

Completed


On January 26, 2017, Entergy - Fossil & Hydroelectric Generation (EntergyFHG) submitted a Self-Report stating that, as a Generator Owner, it was in noncompliance with FAC-008-3 R1. EntergyFHG reported five instances where it had not determined Facility Ratings in accordance with its Facility Rating methodology (FRM). SERC determined that the appropriate standard and requirement is FAC-009-1 R1. On December 11, 2017, EntergyFHG self-reported one additional Facility Rating error, which was assigned NERC2017018748. On May 6, 2019, this self-reported noncompliance was dismissed and consolidated with the initial self-reported instances in SERC2017016879. Thus, this noncompliance involves five instances of Facility Rating errors. In the first instance, prior to March 13, 2014, EntergyFHG used a manual process to verify compliance with FAC-009. In 2014, EntergyFHG internally developed a software tool to scan its Facility Ratings and identify possible errors. Specifically, at the Acadia Power facility, EntergyFHG had correctly determined the summer rating of the generator lead (Unit 026) to be 273 MVA but incorrectly determined the winter rating. EntergyFHG determined the winter rating of 286 MVA with a current transformer as the most limiting element, an increase of 4.6%. On March 13, 2014, EntergyFHG initiated an annual process to randomly select a minimum of 10 units to test with its new software tool. EntergyFHG did not identify any discrepancies in its reviews in 2014 and 2015. On February 2, 2015, EntergyFHG completed a project to standardize and update the facility rating workbooks. In the second instance, on July 7, 2016, EntergyFHG again used the software tool to review 10 randomly sampled Facility Ratings of generation facilities. EntergyFHG discovered that at the Hinds facility, it erroneously calculated the Unit GT2 transformer rating as 203 MVA, which was corrected to 193 MVA, a decrease of 5.2%. As a result, EntergyFHG expanded the review to include an additional six Facilities, and it discovered two additional errors (third instance). At the Sterlington facility, Units 7A and 7B shared a transformer. When EntergyFHG retired Unit 7B and reconfigured the high voltage bus, it did not adjust the Facility Rating from 120 MVA to 60 MVA as required by its FRM, a decrease of 50%. These instances started on February 2, 2015, when, in the first instance, EntergyFHG incorrectly determined the winter rating, and ended on April 20, 2018, when, in the first and second instances, EntergyFHG updated the Facility Ratings sheets with the accurate Facility Ratings. On April 20, 2018, while applying the revised procedure as part of the mitigating actions for the previously discovered instances, EntergyFHG discovered a fourth incorrect Facility Rating for Unit 2 at the Little Gypsy facility, which was submitted as a scope expansion on June 7, 2018. EntergyFHG applied an incorrect gas pressure rating from the Generator nameplate chart. The Generator MVA rating used was associated with gas pressure labeled as “design.” The rating that EntergyFHG should have used was associated with gas pressure labeled as “max cooling.” The result was that the Facility Rating documented the incorrect most limiting element. A “design” gas pressure of 30 PSIG corresponds to a 450 MVA generator rating, and a “max cooling” gas pressure of 45 PSIG corresponds to a 495 MVA generator rating. The Generator had been identified as the most limiting element at 450 MVA. When the correct gas pressure rating is used (45 PSIG), the Generator rating of 495 MVA becomes higher (less limiting) than the transformer rating of 465 MVA, an increase of 9.6%. This instance started on November 11, 2011, when EntergyFHG applied the incorrect generator nameplate rating, and ended on April 24, 2018, when EntergyFHG applied the correct generator nameplate and updated its applicable Facility Ratings sheet with the accurate Facility Rating. On December 11, 2017, EntergyFHG self-reported a fifth Facility Rating error, which was discovered on May 31, 2017, after EntergyFHG performed another survey using its software tool. For this instance, on April 28, 2013, EntergyFHG placed a new transformer in service at the Lewis Creek 2 facility. The previous transformer rating of 290 MVA was the most limiting Facility Rating. However, the new transformer has a higher cooling capability and is rated at 325 MVA making the generator at 312 MVA the most limiting element, an increase of 12%. This instance started on April 27, 2016, when EntergyFHG installed a new transformer and did not adjust the Facility Rating, and ended on August 7, 2017, EntergyFHG adjusted the Facility Rating. EntergyFHG performed an extent-of-condition survey by reviewing the Facility Rating determinations for all 75 of its generation facilities and determined that no other noncompliance existed. The root cause for these instances of noncompliance was a combination of lack of training and a deficient procedures for determining Facility Ratings. For instances one and two, personnel need additional training. Personnel was unfamiliar with the infrequently performed steps in the process, which resulted in calculation errors in the Facility Rating determinations. Additionally, for instances three and five, the procedure guidelines did not clearly define the roles and responsibility for updating Facility Ratings when making facility modifications. Moreover, for the fourth instance, the procedure did not clearly explain how to determine the correct MVA rating when multiple ratings are provided on generator nameplates.



Risk Assessment

This noncompliance posed a moderate risk and did not pose a serious or substantial risk to the reliability of the bulk power system (BPS). Incorrect Facility Ratings could result in errors in planning and operation of the Bulk Electric System and may affect System Operating Limits. Notwithstanding, the following factors reduced risk to the BPS. The Facility Rating error at Acadia (instance #1) was an omission of a calculation for the line rating during winter months, which was higher than the summer rating. At the Hinds facility (Instance #2), the incorrect Facility Rating was lower by only by 5% and it affected only one unit of a combined cycle power plant. Regarding the Sterlington facility (instance #3), the units have a capacity factor less than zero and the error related to a generator retirement, not the electrical capability of the series devices. For the Lewis Creek facility (instance #5), the Facility Rating was higher than previously calculated. These five instances are the only errors EntergyFHG identified in its fleet of 75 generators. According to EntergyFHG, the output for the plans never exceeded the correct Facility Rating and facility modifications did not affect transmission planning results or operation. No harm is known to have occurred.

Mitigation

To mitigate this noncompliance, EntergyFHG: 1) for all instances, updated Facility Ratings to identify the most limiting equipment that comprises the facility; 2) shared causal determination with Fossil NERC Champions during a monthly NERC Champion Call; 3) performed an analysis of Fossil FAC-008 violations and findings utilizing historical Entergy Compliance and Risk Took (ECART) audit results, completed causal analyses, and Paperless Condition Reporting System (PCRS) data to determine if there is a reoccurring Fossil wide problem associated with accurate completion of FAC-008 documentation and presented the results of this analysis to the Condition Review Group (CRG); 4) revised EF-PR-NERC-FAC-008 to include guidance on triggering a Facilities Rating document review when putting a new unit in service or a unit is retired (instance #3) and to clarify which MVA should be used when multiple ratings are provided on generator nameplates (instance #4); 5) transferred the unit rating data of all EntergyFHG plants to the new Unit Facility Ratings Sheet; 6) reviewed all ratings sheets to identify and resolved any issues and/or need for NERC Compliance retraining; 7) revised the current FAC-008 Computer Based Training (CBT) to make current with the standard and revised procedure; 8) verified that every NERC Champion has completed the updated FAC-008 CBT; 9) developed a FAC-008 NERC Champion classroom training; and 10) delivered a classroom training, once, to all NERC Champions.





TRE2017017827 FAC-008-3 R8; R8.1

Texas Municipal Power Agency (TMPA1)

NCR11456

03/25/2014

10/25/2018

Self-Report

Completed


On June 27, 2017, TMPA1 submitted a Self-Report stated that as a Transmission Owner (TO) it was in noncompliance with FAC-008-3 R8.1. In particular, it stated that it did not provide the correct Facility Ratings for the Gibbons Creek to Keith transmission line to the Electric Reliability Council of Texas, Inc. (ERCOT ISO). Subsequently, during a Compliance Audit conducted from November 27, 2017 through February 5, 2018, Texas RE determined that TMPA1, as a TO, had another potential noncompliance with FAC-008-3 R8.1. Specifically, TMPA1 did not provide Facility Ratings to ERCOT ISO for the West Denton to Roanoke transmission line. The root cause of the noncompliance was caused by two factors: 1) for the Self-Report instance, a lack of process for reviewing the Facility Rating for each line and 2) for the Compliance Audit instance, the fact that the regional process for Facility Ratings does not have controls in place for when facilities are jointly owned and the owners independently report conflicting facility ratings, causing the facility rating to default to the Facility Rating reported by an owner which is the most limiting rating. This noncompliance started on March 25, 2014, when TMPA1 registered as a TO, and ended on October 25, 2018, when TMPA1 submitted an updated rating for the West Denton to Roanoke Line to ERCOT ISO.

Risk Assessment

This noncompliance posed a moderate risk and did not pose a serious or substantial risk to the reliability of the bulk power system. In the Self-Report instance, the Facility Ratings that ERCOT ISO had were greater than the actual capability of that equipment. Specifically, the Facility Rating in that instance should have been 478 MVA, but ERCOT ISO’s records showed a rating of 753 MVA. In the Compliance Audit instance, a Facility Rating of 1,480 MVA was submitted to ERCOT ISO, when the Facility Rating identified in TMPA1’s documentation was 1,434 MVA. As a result of these two issues, ERCOT ISO could have overloaded the equipment in question, potentially resulting in damage to the affected Facilities and/or other pieces of in-series equipment. However, the risk posed by the issue was reduced by the following factors. The incorrect rating in the Self-Report had been used for more than 10 years with no adverse effects. Further, in the Compliance Audit instance, TMPA1 provided manufacturer’s documentation to demonstrate the Equipment Ratings identified in the Facility Ratings spreadsheets are consistent with Ratings provided by the equipment manufacturer. Additionally, the Transmission Operator (TOP) that operates the facilities as issue employs engineers and field personnel with the capability to respond to emergency situations at all times. No harm is known to have occurred. Texas RE considered the TMPA1’s compliance history and determined that an affiliate, the City of Garland, had a relevant instance of noncompliance documented in NERC Violation ID TRE2016015977, but that compliance history should not serve as a basis for aggravating the risk. Specifically, though the TMPA1 Self-Report instance and the City of Garland instance have the same root cause, the Self-Report instance was discovered during the extent of condition review conducted as a result of the City of Garland instance, and was reported promptly after it was discovered. Further, the TMPA1 Compliance Audit instance had a different root cause than the City of Garland instance.

Mitigation To mitigate this noncompliance, TMPA1: 1) for the Self-Report instance, submitted an updated rating for the line to ERCOT ISO on June 13, 2017; 2) for the Compliance Audit instance, submitted an updated rating for the line to ERCOT ISO on January 23, 2018; and 3) adopted a revised procedure whereby the rating of each line is reviewed annually to ensure that its Facility Ratings match the records maintained by ERCOT ISO and to investigate, document

and modify any incorrect numbers if they are different.






TRE2018019883 FAC-008-3 R8 Wind Energy Transmission Texas, LLC (WETT)

NCR11074 08/04/2015 04/26/2018 Compliance Audit Completed


During a Compliance Audit conducted from February 26, 2018, through June 12, 2018, Texas RE determined that WETT, as a Transmission Owner (TO), was in noncompliance with FAC-008-3 R8. Specifically, WETT did not provide facility ratings to ERCOT ISO for its jointly-owned facilities. Specifically, improper Facility Ratings were recorded with ERCOT ISO due to WETT’s failure to provide equipment ratings for WETT-owned equipment on jointly-owned transmission facilities. These Facilities were tie lines with adjacent Transmission Operators (TOP).

The root cause of this noncompliance was an insufficient process for developing and recording Facility Ratings for jointly-owned transmission Facilities.

This noncompliance started on August 4, 2015, the last date of WETT’s previous exit briefing, and ended on April 26, 2018, WETT provided Facility Ratings to ERCOT ISO for its jointly owned Facilities.

Risk Assessment This noncompliance posed a moderate risk and did not pose a serious or substantial risk to the reliability of the bulk power system. In some instances, the Facility Ratings that ERCOT had were greater than the actual capability of that equipment. The potential risk from not having accurate Facility Ratings is that equipment may be operated above its maximum ratings without the operators being aware and this may cause equipment degradation and potential failure. As a result, ERCOT could have overloaded the equipment in question, potentially resulting in damage to the affected Facilities and/or other pieces of in-series equipment. Providing incorrect Facility Ratings also reduces the Reliability Coordinator’s situational awareness, impacting its ability to develop accurate operational models of real-time demands, contingency analyses, and planning studies. In some instances, the Facility Ratings that ERCOT had were less than what WETT considered the actual capability of that equipment. Operating below the correct rating in emergencies can also cause undo load shedding and result in incorrect post contingency planning. The risk posed by this issue was reduced by the fact that WETT has a relatively small footprint (1,253 MW of interconnected generation) that does not have an appreciable effect on the BPS, and further, WETT employs engineers and field personnel with the capability to respond to emergency situations at all times. No harm is known to have occurred.

Texas RE considered WETT’s compliance history and determined there were no relevant instances of noncompliance.

Mitigation To mitigate this noncompliance, WETT:

1) WETT provided Facility Ratings to ERCOT ISO for its jointly owned Facilities;2) updated its procedure to ensure that the Facility Ratings in ERCOT’s models match those from WETT’s models, on an annual basis.







RFC2019020923 PRC‐005‐6 R3 Potomac Electric Power Company NCR00881 5/1/2018 10/31/2018 Self‐Report Completed Description of the Noncompliance (For purposes of this document, each noncompliance at issue is described as a “noncompliance,” regardless of its procedural posture and whether it was a possible, or confirmed noncompliance.)

On January 4, 2019, the entity submitted a Self‐Report stating that, as a Distribution Provider and Transmission Owner, it was in noncompliance with PRC‐005‐6 R3. On October 23, 2018, a crew working at a 138 kV substation notified the entity’s Substations Mobile Operations team (Mobile Ops) that there was equipment in need of routine maintenance. Upon review of past substation inspection documentation, Mobile Ops discovered that the substation was not included in the monthly substation inspection cycle and that the entity had missed three maintenance activities, which were required to be performed every 4 calendar months, for two 4 calendar month maintenance cycles. The missed maintenance activities included: (1) verifying station dc supply voltage; (2) inspecting electrolyte level; and, (3) inspecting for unintentional grounds.

The root cause of this noncompliance was the entity’s failure to include this substation in the monthly substation inspection cycle at the time of commissioning. The entity had designated one employee from the Mobile Ops team to perform this task. However, that employee retired in 2017 before this substation was commissioned and the entity failed to delegate this responsibility to another team member. This root cause involves the management practices of implementation, because the noncompliance involved an error in the commissioning process, and workforce management, which includes managing succession plans.

This noncompliance started on May 1, 2018, when the entity missed its first 4 calendar month interval and ended on October 31, 2018, when the entity completed the missed maintenance activities. Risk Assessment This noncompliance posed a minimal risk and did not pose a serious or substantial risk to the reliability of the bulk power system (BPS) based on the following factors. The risk associated with failing to

perform required maintenance activities within the required timeframe is that the device could fail to operate as expected, which could reduce the reliability of the BPS. This risk was mitigated in this case by the following factors. First, the batteries were newly installed and underwent testing during commissioning to ensure that they were functioning properly, which reduces the likelihood that the devices would fail. Second, the entity designed the substation with redundant batteries and chargers, which reduces the risk posed by failing to maintain these particular batteries for two intervals. Third, one of the batteries has its dc supply voltage monitored and alarmed to the control center as an additional method for ensuring the operability of the battery system. Fourth, the entity quickly identified and corrected the issue quickly. No harm is known to have occurred.

The entity has relevant compliance history. However, ReliabilityFirst determined that the entity’s compliance history should not serve as a basis for applying a penalty because while the result of some of the prior noncompliances were arguably similar, the prior noncompliances arose from different causes. Although the entity’s compliance history with respect to PRC‐005‐6 R3 is distinguishable, ReliabilityFirst considered the entity’s compliance history related to its maintenance and testing of Protection Systems and determined that FFT treatment was the appropriate disposition method in this case.


1) completed the required maintenance activities;2) provided training to Mobile Ops current management and supervisors on PRC‐005 station battery maintenance activities and evidence documentation, and require training to be completed during the

onboarding process for new managers and supervisors;3) required all Mobile Ops supervisors and business analyst(s) to complete training on preventative and corrective maintenance scheduling, and require training be completed during the onboarding

process for new supervisors and analysts;4) revised the Project Management substation commissioning documentation to require Project Management leads to verify that all applicable preventative maintenance is scheduled in the

maintenance scheduling database prior to commissioning; including monthly substation inspections; and5) revised the Substation Engineering commissioning documentation to require Substation Engineering personnel to verify that all applicable preventative maintenance is scheduled in the maintenance

scheduling database prior to commissioning; including monthly substation inspections.


Last Update 10/31/2019 1



SERC2016016490 TOP-002-2b R19 Associated Electric Cooperative, Inc. (AECI)

NCR01177 07/01/2012 10/13/2016 Compliance Audit Completed


During a Compliance Audit conducted from July 12, 2016 to October 31, 2016, SERC determined that AECI, as a Balancing Authority and Transmission Operator, was in noncompliance with TOP-002-2b R19. AECI did not maintain accurate computer models utilized for analyzing and planning system operations.

During the control center tour portion of the audit, SERC compared Facility Ratings in AECI’s Emergency Management System (EMS) to the Facility Ratings provided in the FAC-008-3 R6 Facilities Rating data submission for eight facilities. Since AECI uses dynamic ratings, AECI had to perform a reversion calculation to relate back to the FAC-008-3 R6 Facility Ratings. This calculation was based on the actual temperature AECI used for the Facility at the time of the control center tour. One of the Facilities selected for rating verification was the Huben to Coffman – 161kV transmission line. The rating shown in the Facility Ratings spreadsheet was 179 MVA. The rating shown on EMS, at the time of the control center tour, was 334.6 MVA. AECI performed the reversion calculation for this Facility and determined there was a discrepancy. While SERC was on-site, AECI determined that it had changed the actual rating of the Huben to Coffman – 161kV to 179 MVA based on relay loadability. However, AECI did not make the change in the EMS; therefore, the Facility Rating in the EMS was incorrect. In addition, SERC determined that the Facility Rating for the Boone to McBaine – 161kV transmission line, which was calculated per AECI’s Facility Rating Methodology (FRM), did not match the rating used in EMS. Once SERC identified this issue, AECI found six additional similar instances that had EMS values different than the values in the database. All six of the differing values were associated with not using the relay limit within the circuits. AECI corrected these issues before SERC concluded the audit.

Post audit, SERC issued a request for information, which required AECI to evaluate 23 Facilities that are included in six Medium Impact BES Cyber Systems. During its evaluation of the 23 Facilities, AECI identified 13 additional discrepancies that impacted the transmission’s Most Limiting Element (MLE). In all instances, the EMS values were the same as the values that AECI incorrectly calculated when it applied its FRM. SERC dispositioned these 13 instances as violations of FAC-009-1 R1 as SERC2016016489.

This noncompliance started on July 1, 2012, when AECI used inaccurate ratings in its Energy Management System (EMS), and ended on October 13, 2016, when AECI corrected the ratings in its EMS.

The root cause of this noncompliance was an internal control failure. The EMS software code did not take into account the relay limits within the circuit and, therefore, it failed to recognize the MLE of the circuit when the relay was the MLE.

Risk Assessment This noncompliance posed a moderate risk and did not pose a serious or substantial risk to the reliability of the bulk power system (BPS). Because AECI used inaccurate ratings in its EMS, there was a risk that AECI System Operators would respond inappropriately to planned system switching or to unplanned system events. Of the 20 identified discrepancies that impacted the transmission’s MLE, the largest MLE derate was an 86.59% winter rating derate on a 161 KV Facility, however, most derates were around 10%. AECI determined it did not exceed the correct MLE for any of the identified discrepancies. No harm is known to have occurred.

SERC considered AECI’s compliance history and determined that there were no relevant instances of noncompliance. Mitigation To mitigate this noncompliance, AECI:

1) performed a physical walk-down of FAC-008-3 applicable facilities to ensure all facility elements were correctly identified;2) entered asset information into its new asset management software, MinMax;3) propagated asset data into AECI transmission planning model database (Mantis);4) applied AECI FAC-008 Ratings Methodology to all entered information;5) propagated all limits to EMS and Real-time Contingency Analysis (RTCA);6) created the AECI Rating Application Procedure to document the process to keep the EMS, Mantis, and TVA CTR portal models accurate;7) reported to SERC all discrepancies found during the field verification which impact the MLE as well as those that do not impact the MLE; and8) implemented an internal audit process internal control to verify ratings.





SERC2017017604 PRC-005-2(i) R3 Duke Energy Progress, LLC (DEF) NCR01298 10/01/2015 09/20/2016 Self-Report Completed


On May 23, 2017, Duke Energy Progress, LLC (DEP), through an existing multi-regional registered entity agreement, submitted a Self-Report on the behalf of Duke Energy Florida, LLC (DEF) stating that, as a Transmission Owner (TO), DEF was in noncompliance with PRC-005-1.1b R2. DEF failed to perform required periodic maintenance and testing on a Protection System device. SERC later determined that PRC-005-2(i) R3 was the applicable Standard and Requirement.

On September 19, 2016, a DEF Construction and Maintenance (C&M) Relay Technician performed an investigation of a relay misoperation on the Largo to Anclote 230 kV Transmission Line and discovered that someone removed a carrier relay check-back module from the carrier set on the Largo end. DEF found that, since at least July 29, 2015, it had inhibited the alarm for the check-back failure at the DEF control center because it had not fully commissioned a new Supervisory Control and Data Acquisition (SCADA) alarm point. That action left the carrier relay system unmonitored since that date, and DEF had not recorded the inoperability of the alarm in the maintenance data system. As of November 1, 2015, the implementation plan for PRC-005-2 required that TOs verify any unmonitored communication as functional every four calendar months. Since DEF believed this circuit to be monitored, DEF did not perform the functionality maintenance that is required every four calendar months for unmonitored communication system.

On September 11, 2017, DEF submitted an expansion of scope and stated that, on September 19, 2016, while doing other work at Fort White substation, the Relay Technicians noticed that the check-back alarm for the Fort White to Newberry Carrier did not alarm to the Energy Control Center (ECC) on a failure of communication. DEF considered this communication system monitored and, therefore, it failed to perform the four calendar month interval maintenance tasks.

This noncompliance started on October 1, 2015, when DEF failed to perform the required four calendar month maintenance items on the carrier system, and ended on September 20, 2016, when DEF replaced and verified operability of the carrier relay.

The root cause of this noncompliance was inadequate controls for the documentation and maintenance of Protection System devices. Risk Assessment This noncompliance posed a moderate risk and did not pose a serious or substantial risk to the reliability of the bulk power system (BPS). Failure to properly maintain Protection System devices could

result in failure to protect the Bulk Electric system from faults or could cause unnecessary trips or other misoperations affecting reliability. In this case, failure to perform the required maintenance did result in a misoperation and unnecessary relay operation, but no load was lost. However, a moderate risk existed because the failure to properly document alarm inhibits, and failure to perform maintenance on Protection System devices on critical system elements, may have caused greater impact than actually occurred.

SERC determined that DEF's compliance history should not serve as a basis for applying a penalty. SERC reviewed the compliance history of PRC-005 for DEF and its affiliates, Duke Energy Carolinas, Duke Energy Corporation, and DEP, and did not identify circumstances similar to that of the instant issue. Each Duke Energy affiliate is responsible for its own maintenance and testing program and the completed mitigation plans would not have addressed the instant issue. Therefore, SERC staff did not consider the previous violations by affiliates as aggravating circumstances, nor did it identify a programmatic issue based on that history.

Mitigation To mitigate this noncompliance, DEF:

1) replaced the failed carrier unit;2) tested carrier unit alarms;3) outlined Job Plan Instructions to perform check-back module alarm checks;4) completed review, comment period and approval of Job Plan Instructions to perform check-back module alarm checks;5) developed a checklist to periodically review inhibited points and verify estimated in-service dates;6) executed the checklist, developed in Milestone 5, on an ongoing quarterly basis, and developed plans to address issues based on this review;7) reinforced the importance of creating corrective work order on broken equipment per DEF’s PSMP Sec. 12 with DEF Relay Crews through crew meetings; and8) verified that all monitored carriers have a check-back module and functioning alarms per job plan instructions.





RFC2018020367 PRC‐005‐2(i) R3 American Municipal Power Inc. NCR00683 1/4/2016 8/30/2018 Self‐Report Completed Description of the Noncompliance (For purposes of this document, each noncompliance at issue is described as a “noncompliance,” regardless of its procedural posture and whether it was a possible, or confirmed noncompliance.)

On August 31, 2018, the entity submitted a Self‐Report stating that, as a Generator Owner, it was in noncompliance with PRC‐005‐2(i) R3. Beginning in January 2016, the entity commissioned four hydro facilities. During the commissioning process, the entity relied on one individual to build the battery maintenance program for all four facilities. That individual left the entity before the commissioning process was complete and before a comprehensive list of battery maintenance tasks was entered into the Maximo system. As a result, each facility developed its own periodic battery maintenance practices.

In July 2018, the entity engaged a third‐party contractor to conduct a gap analysis for each of the hydro facilities and discovered that there were inconsistencies in battery testing activities and documentation. Generally, the entity either omitted some testing, conducted testing outside of the required interval, or failed to properly document testing. (The entity provided a complete, detailed list of all the inconsistencies in battery testing activities. All facilities had issues with the 6‐month testing of battery cell/unit internal ohmic value. At most, the entity missed three intervals of this testing. Two facilities had issues with 18‐month interval testing, including battery continuity, battery terminal connection resistance, and battery intercell or unit‐to‐unit connection resistance. At most, the entity missed one interval of this testing.) However, the entity was still generally maintaining and checking the batteries on a regular basis, which would have helped identify any issues with the batteries and would allow for corrective actions to be performed before degradation of the battery system occurred.

The root cause of the noncompliance was the entity’s reliance on one individual to set up the comprehensive battery testing program for all four hydro facilities and subsequent failure to verify on an ongoing basis that proper testing was being performed. This root cause involves the management practices of workforce management, which includes managing succession plans and managing employment status changes, and verification, in that the entity failed to verify that proper testing was occurring.

This noncompliance started on January 4, 2016, when the entity commissioned the first hydro facility and ended on August 30, 2018, when the entity completed all the required testing and updated the Maximo program to include all required maintenance activities for all facilities.

Risk Assessment This noncompliance posed a moderate risk and did not pose a serious or substantial risk to the reliability of the bulk power system based on the following factors. The risk posed by failing to conduct all of the required battery tests within the prescribed time intervals is that it could impair the entity’s ability to identify and correct issues that could lead to degradation of the battery system. The risk is not minimal in this case because the entity was unaware of the inconsistencies for approximately two years and only discovered them after engaging a third‐party contractor. The risk is not serious or substantial in this case because, although the entity had some inconsistencies in its battery testing program, the entity was still performing many of the battery maintenance activities on a regular basis, which would have helped identify any issues with the batteries and would allow for corrective actions to be performed before degradation of the battery system occurred. No harm is known to have occurred.


1) conducted a face‐to‐face meeting between plant personnel responsible for PRC‐005 maintenance and testing and a third‐party NERC contractor to discuss documentation practices, details on batterytesting, and overall responsibilities for PRC‐005;

2) developed a universal battery testing template to be used by all facilities to ensure consistent battery testing across all facilities; and3) updated Maximo program for all sites to include all PRC‐005 maintenance activities for the batteries installed at the site.






RFC2018020364 VAR‐002‐4.1 R3 American Municipal Power Inc. NCR00683 11/14/2017 11/14/2017 Self‐Report Completed Description of the Noncompliance (For purposes of this document, each noncompliance at issue is described as a “noncompliance,” regardless of its procedural posture and whether it was a possible, or confirmed noncompliance.)

On August 30, 2018, the entity submitted a Self‐Report stating that, as a Generator Operator, it was in noncompliance with VAR‐002‐4.1 R3. On November 14, 2017, at 8:10 am, the control room operator noticed a Distributed Control System alarm indicating an issue with the automatic voltage regulator (AVR) for the steam generation unit. The operator immediately began troubleshooting the issue and at 8:14 am notified the Energy Control Center (ECC). The entity returned the AVR to automatic voltage control mode at 8:43 am. Subsequently, the entity determined that the AVR switched into manual mode at 7:41 am, which resulted in a noncompliance because the entity failed to notify its Transmission Operator (TOP) within 30 minutes from the status change. The AVR was in manual mode for approximately 1 hour before the operator returned the AVR to automatic voltage control mode.

The root cause of the noncompliance was two‐fold. First, the ECC operator misunderstood the notification requirement. The operator thought that the 30 minute notification started to run from the time he was made aware of the AVR mode change, rather than the actual time of the AVR mode change. Because the AVR was returned to automatic mode within 30 minutes of when the ECC operator was made aware of the change, he did not believe he needed to notify the TOP. Second, the entity discovered that, although the two combustion turbines have audible alarms that activate in the event of a status change of the AVR, the steam turbine did not have a similar alarm for the status change of the AVR. The entity discovered the lack of an alarm when the under‐excitation limiter caused the trouble alarm to activate. The root cause involves the management practices of workforce management, which includes providing training, awareness, and education to employees, and grid operations.

This noncompliance started at 8:11 am on November 14, 2017, the time by which the entity was required to notify its TOP about the AVR status change, and ended later that same day when the entity switched the AVR back into automatic mode.

Risk Assessment This noncompliance posed a moderate risk and did not pose a serious or substantial risk to the reliability of the bulk power system based on the following factors. The risk posed by failing to notify the TOP of a status change on the AVR is that it would impair the TOP’s ability to maintain voltage. The risk is not minimal in this case because the entity only discovered the AVR status change after the under‐excitation limiter caused the trouble alarm to activate. Consequently, the turbine was at risk of entering a state of under‐excitation. Prolonged under‐excitation can lead to overheating of the stator core and windings resulting in physical damage. Additionally, in extreme cases, the generator can also lose synchronism with the system and trip. The risk is not serious or substantial in this case because the two combustion turbines were operating in automatic voltage control mode and were able to maintain the voltage at the point of interconnection for the facility. ReliabilityFirst also notes that, during the time that the steam combustion turbine’s AVR was in manual mode, no emergency events occurred and the TOP did not make any requests from the generator. No harm is known to have occurred.


1) created an audible alarm, in addition to the visual alarm, in the AMP Fremont Energy Center Operator Control Center for a change in the AVR status on the steam turbine;2) included the status of the AMP Fremont Energy Center AVR on the screens of the operators in the AMP Energy Control Center;3) developed comprehensive VAR 002 training material for AMP Energy Control Center Personnel to be included in the NERC Update for Energy Control Center training. The training will provide clarity

on the 30 minute reporting obligations for when an AVR changes status; and4) retrained AMP Energy Control Center operators on the time‐frame for reporting changes in a generation unit’s AVR status.






SERC2017018328 TOP-008-1 R4 Duke Energy Carolinas, LLC (DEC) NCR01219 1/12/2016 03/29/2017 Self-Report Completed


On September 8, 2017, Duke Energy Carolinas LLC (DEC) submitted a Self-Report stating that, as a Transmission Operator (TOP), it was in noncompliance with TOP-008-1 R4. DEC stated that it did not have sufficient analysis tools to determine the causes of System Operating Limit (SOL) violations.

DEC acts as the Reliability Coordinator (RC) agent for VACAR. The DEC Energy Management System (EMS) Real-time Contingency Analysis (RTCA) application is used by both the DEC TOP and VACAR South RC Agent System Operators to identify potential line overloads and other potential SOL issues.

On March 29, 2017, at 1:15, p.m., the South Carolina Electric & Gas (SCE&G) TOP contacted DEC, as the RC Agent for VACAR South, to report that the Newport-VC Summer 230 kV tie-line between a DEC substation and the SCE&G generating unit would exceed its Facility Rating for the contingency loss of the power plant. The DEC RTCA failed to indicate that this contingency loading would be present. On March 29, 2017, at 1:55 p.m., the RC Agent System Operator confirmed, using a Powerflow study, that the contingency overload was valid. The Powerflow study showed that the contingency would cause loading of 132% of the 1 hour rating on the SCE&G end of the line and 120% of the 1 hour rating on the DEC end of the line. If this contingency would have occurred, the resulting overload of the tie-line would have been an SOL exceedance.

The RC developed a mitigation plan that involved the DEC TOP and two adjacent TOPs. The TOPs performed the plan as a proactive measure to eliminate original contingency overload.

DEC notified its System Operations Engineering (SOE) and EMS Engineering that the RTCA failed to show the tie-line contingency. SOE worked with EMS Engineering and discovered that the tie-line was not properly flagged. In order to be included in the RTCA application, the tie-line should have been flagged as a monitored element in the EMS Network Monitored Element Definition display. On March 29, 2017, at 1:58 p.m., EMS Engineering and SOE added the tie-line to the RTCA monitoring by checking the eligible flag on the Network Monitored Element Definition display. As a result, RTCA indicated the contingency loading for the Newport–VC Summer 230 kV tie-line.

The EMS Engineering Staff performed an extent-of-condition assessment that consisted of a review of other flags in the EMS Network Monitored Elements display to check for missing flags on eligible facilities. That review identified an additional four transmission elements in the Duke Energy Progress, LLC (DEP) TOP area that were not flagged correctly in the DEC EMS RTCA application. The DEC EMS Engineering Staff was unable to determine the exact period of time that it had not properly flagged this instance and DEP’s four transmission elements, however, it appears to be linked to a data update that occurred on January 12, 2016.

This noncompliance started on January 12, 2016, when DEC performed an update to reset the flags in the RTCA database, and ended on March 29, 2017, when DEC reset the flags to include the transmission elements in its RTCA.

The root cause of this noncompliance was inadequate internal control to ensure EMS RTCA updates were accurate. There was no requirement to validate pre-upload flagged network monitored elements against the post-upload elements to identify any inconsistencies.

Risk Assessment This noncompliance posed a moderate risk and did not pose a serious or substantial risk to the reliability of the bulk power system (BPS). The potential risk to the BPS relates to the fact that DEC, as a TOP, and the RC Agent for VACAR South, was unable to use its RTCA tools to monitor for a potential SOL violation on five transmission elements. This could have resulted in overloads on those elements resulting in possible equipment damage, and reduced reliability. SERC determined that the issue was appropriate for FFT treatment because DEC and its neighbors did maintain Real Time monitoring of the elements and neighboring TOPs did include the elements in their RTCA. The neighbors had not identified other contingency overloads. No actual overloads occurred. The RC, DEC, and its neighbors communicated and collaboratively reached an action plan to address the contingency. No harm is known to have occurred.

SERC considered DEC’s compliance history and determined that there were no relevant instances of noncompliance. Mitigation To mitigate this noncompliance, DEC:

1) added the Newport Tie- VC Summer 230 kV Tie-line to the RTCA monitoring causing the RTCA to indicate the contingency loading for the Tie-line;2) included steps in the EMS Engineering Staff’s pre-upload procedure (Upload Process) to document and validate the pre-upload flagged Network Monitored Elements against the post-upload NetworkMonitored Elements to identify any discrepancies after each database upload for DEC, Duke Energy Progress, LLC (DEP), Duke Energy Corporation (DECorp), and Duke Energy Florida (DEF); 3) developed, trained, and implemented a new defined procedure clearly defining the roles and responsibilities and expectations for EMS Engineering when reviewing flags on eligible facilities;4) modified the process with the VACAR South RC Member TOPs to annually review the VACAR South RC “Key Facilities List”; as part of the annual review, the VACAR South RC Agent ensured that all of theVACAR South ‘Key Facilities’ were flagged correctly in the Network Monitored Element Definitions display in the DEC EMS for inclusion for monitoring by the RTCA application; 5) trained and implemented the System Operation Engineering Staff in DEC and DEP on the modified process (mitigating activity # 4); and



6) documented, by the VACAR South RC Agent, that all of the VACAR South RC ‘Key Facilities’ were flagged correctly in the Network Monitored Element Definitions display in the DEC EMS for inclusion formonitoring by the RTCA application.





SERC2017018331 IRO-005-3.1a R1, R1.3

VACAR South (VACS) NCR01365 01/01/2016 03/29/2017 Self-Report Completed


On September 12, 2017, VACS submitted a Self-Report stating that, as a Reliability Coordinator (RC), it was in noncompliance with IRO-005-3.1a R1, R1.3. VACS did not monitor current post-contingency element conditions for violations of a System Operating Limit (SOL).

Duke Energy Carolinas (DEC) acts as an RC agent for VACS. Both DEC and VACS use the DEC Energy Management System (EMS) real-time contingency analysis (RTCA) application to identify potential line overloads and other potential SOL issues. On March 29, 2017, at 1:15 p.m., an adjacent Transmission Operator (TOP) contacted DEC to report that a 230 kV tie-line between a DEC substation and the adjacent TOP generating unit (tie-line) would exceed its Facility Rating for the contingency loss of the power plant. The DEC RTCA application did not indicate that this contingency loading would be present.

On March 29, 2017, at 1:55 p.m., using a Powerflow study, DEC’s System Operator confirmed that the contingency overload was valid and showed that the contingency would cause loading of 132% of the one hour Rating on the adjacent TOP’s end of the line and 120% of the one hour rating on the DEC end of the line. If this contingency would have occurred, the resulting overload of the tie-line would have been an SOL exceedance.

DEC notified its System Operations Engineering (SOE) and Energy Management System (EMS) Engineering that the tie-line contingency was not showing up in RTCA results. SOE worked with EMS Engineering and discovered that it had not properly flagged the tie-line as a monitored element in the Network Monitored Element Definition display in the DEC EMS for inclusion in the RTCA application. The omission of the proper flag setting did not allow the VACS System Operators to identify and become aware of the potential overload of the tie-line.

DEC EMS Engineering performed a review of other flags in the Network Monitored Element Definition display in the EMS to check for missing flags on eligible facilities. The review identified four transmission elements in the Duke Energy Progress (DEP) TOP area as not flagged correctly for inclusion in the RTCA application. DEP TOP did monitor four elements in its RTCA and DEC and VACS did monitor the facilities in the EMS, but, DEC had not properly flagged them for analysis in the RTCA application. DEC EMS Engineering was not able to determine the exact date that it had not properly flagged the five transmission elements at issue as monitored elements in the Network Monitored Element Definition display in the DEC EMS. However, it appeared to be linked to a data update that occurred sometime in 2016. Therefore, January 1, 2016, was the earliest possible start date for this instance of noncompliance.

This noncompliance started on January 1, 2016, when DEC performed an update to the RTCA database that reset the flags, and ended on March 29, 2017, when DEC reset the flags to include the transmission elements in the RTCA application.

The root cause of this noncompliance was a lack of internal controls to properly flag the tie-line as a monitored element in the Network Monitored Element Definition display in the DEC EMS for inclusion in the RTCA application.

Risk Assessment This noncompliance posed a moderate risk and did not pose a serious or substantial risk to the reliability of the bulk power system (BPS). VACS’s inability to use its RTCA tools to monitor for a potential SOL violation on five transmission elements could cause overloads on those elements resulting in possible equipment damage and reduced reliability. However, the TOPs in the VACS footprint, other than DEC, maintained Real Time monitoring of the elements and the TOPs, other than DEC, included the elements in their RTCA application. Additionally, the adjacent TOPs did not identify other contingency overloads and no actual overloads occurred. Moreover, DEC and the adjacent TOPs communicated and collaboratively reached an action plan to address the contingency. No harm is known to have occurred.

SERC considered VACS’s compliance history and determined that there were no relevant instances of noncompliance.

Mitigation To mitigate this noncompliance, VACS:

1) added the Newport-VC Summer 230 kV tie-line to the RTCA application by checking the eligible flag in the Network Monitored Element Definition display in the EMS;2) revised the EMS Engineering staff pre-upload procedure (upload process) to document and validate the pre-upload flagged Network Monitored Elements against the post-upload Network Monitored

Elements to identify any discrepancies after each database upload for DEC;3) developed and implemented a process with the VACS RC Member TOPs to annually review the VACS RC ‘Key Facilities List.’ As part of the annual review, DEC ensured that all of the VACS ‘Key

Facilities’ were flagged correctly in the Network Monitored Element Definition display in the DEC EMS for inclusion for monitoring by the RTCA application;4) discussed the above-noted process change with its Members Committee, and, revised its Operating Limits Procedure ‘Key Facilities List’ section; and



5) documented that all VACS RC ‘Key Facilities’ were flagged correctly in the Network Monitored Element Definition display in the DEC EMS for inclusion for monitoring by the RTCA application.





SERC2017018332 IRO-003-2 R1 VACAR South (VACS) NCR01365 01/01/2016 03/29/2017 Self-Report Completed


On September 12, 2017, VACS submitted a Self-Report stating that, as a Reliability Coordinator (RC), it was in noncompliance with IRO-003-2 R1. VACS was unable to monitor all facilities in its RC area and adjacent RC areas to ensure that it was able to determine any potential System Operating Limit (SOL) and Interconnection Reliability Operating Limit violations within its RC Area.

Duke Energy Carolinas (DEC) acts as the RC agent for VACS. Both DEC and VACS use the DEC Energy Management System (EMS) real-time contingency analysis (RTCA) application to identify potential line overloads and other potential SOL issues. On March 29, 2017, at 1:15 p.m., an adjacent Transmission Operator (TOP) contacted DEC to report that a 230 kV tie-line between a DEC substation and that the adjacent TOP’s generating unit (tie-line) would exceed its Facility Rating for the contingency loss of the power plant. The DEC RTCA application did not indicate this contingency loading would be present.

On March 29, 2017, at 1:55 p.m., using a Powerflow study, DEC’s System Operator confirmed that the contingency overload was valid and showed that the contingency would cause loading of 132% of the one hour Rating on the adjacent TOP’s end of the line, and 120% of the one hour Rating on the DEC end of the line. If this contingency would have occurred, the resulting overload of the tie-line would have been an SOL exceedance.

DEC notified its System Operations Engineering (SOE) and Energy Management System (EMS) Engineering that the tie-line contingency was not showing up in RTCA results. SOE worked with EMS Engineering and discovered that it had not properly flagged the tie-line as a monitored element in the Network Monitored Element Definition display in the DEC EMS for inclusion in the RTCA application. The omission of the proper flag setting did not allow the VACS System Operators to identify and become aware of the potential overload of the tie-line.

DEC EMS Engineering performed a review of other flags in the Network Monitored Element Definition display in the DEC EMS to check for missing flags on eligible facilities. The review identified four transmission elements in the Duke Energy Progress (DEP) TOP area as not flagged for inclusion in the RTCA application. DEP TOP did monitor four elements in its RTCA and DEC and VACS did monitor the facilities in its EMS, but, DEC had not properly flagged them for analysis in the RTCA application. DEC EMS Engineering was not able to determine the exact date that it had not properly flagged the five transmission elements at issue as monitored elements in the Network Monitored Element Definition display in the DEC EMS. However, it appeared to be linked to a data update that occurred sometime in 2016. Therefore, January 1, 2016, was the earliest possible start date for this instance of noncompliance.

This noncompliance started on January 1, 2016, when DEC performed an update to the RTCA database that reset the flags, and ended on March 29, 2017, when DEC reset the flags to include the transmission elements in the RTCA application.

This root cause of this noncompliance was a lack of internal controls to properly flag the tie-line as a monitored element in the Network Monitored Element Definition display in the DEC EMS for inclusion in the RTCA application.

Risk Assessment This noncompliance posed a moderate risk and did not pose a serious or substantial risk to the reliability of the bulk power system (BPS). VACS’s inability to use its RTCA tools to monitor for a potential SOL violation on five transmission elements could cause overloads on those elements resulting in possible equipment damage and reduced reliability. However, the TOPs in the VACS footprint, other than DEC, maintained Real Time monitoring of the elements. Additionally, the TOPs, other than DEC, included the elements in their RTCA application. The adjacent TOPs did not identify other contingency overloads and no actual overloads occurred. Moreover, DEC and the adjacent TOPs communicated and collaboratively reached an action plan to address the contingency. No harm is known to have occurred.

SERC considered VACS’s compliance history and determined that there were no relevant instances of noncompliance.

Mitigation To mitigate this noncompliance, VACS:

1) added the Newport-VC Summer 230 kV tie-line to the RTCA application by checking the eligible flag in the Network Monitored Element Definition display in the EMS;2) revised the EMS Engineering staff pre-upload procedure (upload process) to document and validate the pre-upload flagged Network Monitored Elements against the post-upload Network MonitoredElements to identify any discrepancies after each database upload for DEC; 3) developed and implemented a process with the VACS RC Member TOPs to annually review the VACS RC ‘Key Facilities List.’ As part of the annual review, DEC ensured that all of the VACS ‘Key Facilities’were flagged correctly in the Network Monitored Element Definition display in the DEC EMS for inclusion for monitoring by the RTCA application; 4) discussed the above-noted process change with its Members Committee, and, revised its Operating Limits Procedure ‘Key Facilities List’ section; and5) documented that all VACS RC ‘Key Facilities’ were flagged correctly in the Network Monitored Element Definition display in the DEC EMS for inclusion for monitoring by the RTCA application.



NERC Violation ID Reliability Standard

Req. Entity Name NCR ID Noncompliance Start Date Noncompliance End Date Method of Discovery Future Expected Mitigation Completion Date

TRE2018019485 EOP-005-2 R4 Oncor Electric Delivery Company, LLC (Oncor)

NCR04109 01/01/2018 02/09/2018 Self-Report Completed


On April 2, 2018, Oncor Electric Delivery Company, LLC (Oncor) submitted a Self-Report stating that, as a Transmission Operator (TOP), it was in noncompliance with EOP-005-2 R4. Specifically, Oncor failed to update its restoration plan prior to implementing a planned Bulk Electric System (BES) modification that would change the implementation of its restoration plan.

In August of 2017, Oncor’s Reliability Coordinator (RC) awarded Blackstart Resource contracts for 2018-2019 that included Oncor’s Permian Basin Steam Electric Station (Permian Basin) as a Blackstart Resource. This award required Oncor to develop Primary and Secondary Synchronization Corridors to connect Permian Basin to its synchronization points. On October 31, 2017, Oncor submitted its Blackstart Plan, which was to be effective January 1, 2018, and included the Synchronization Corridors for Permian Basin. On December 6, 2017, an Oncor internal work request was submitted to plan certain System modifications that would alter the Primary Synchronization Corridor for Permian Basin. Oncor’s Transmission Outage Application (TOA) flagged the work requests as affecting Blackstart equipment identified in Oncor’s new Blackstart Plan. However, Oncor’s Transmission Grid Operations (TGO) Support Group was in the process of updating the TOA and failed to recognize the applicability of the notice. On December 15, 2017, Oncor implemented the planned system modifications, and Oncor’s Blackstart Plan became effective on January 1, 2018 with erroneous information regarding the Primary Synchronization Corridor for Permian Basin.

On February 5, 2018, Oncor discovered the noncompliance when another internal work order was evaluated that proposed System modifications to equipment affecting the Primary Synchronization Corridor for Permian Basin. During evaluation of the work order, Oncor recognized that the path for Permian Basin’s Primary Synchronization Corridor had been altered after submission of Oncor’s Blackstart Plan, and that the plan required revision. On February 9, 2018, Oncor revised its Blackstart Plan and submitted it to its RC for review, ending the noncompliance.

The root cause of this noncompliance was the Oncor TGO Support Group’s failure to adhere to standard procedures during its effort to update the TOA and ensure that work requests flagged by TOA were recognized by staff and appropriately addressed.

This noncompliance started on January 1, 2018, when Oncor’s Blackstart Plan became effective, and ended on February 9, 2018, when Oncor revised and resubmitted its Blackstart Plan.

Risk Assessment This noncompliance posed a minimal risk and did not pose a serious or substantial risk to the reliability of the bulk power system. First, Oncor's Transmission Management System (TMS) included the correct one-line diagrams at the time changes were made on December 15, 2017. Had it been necessary to implement Oncor’s Blackstart Plan and for Oncor to use its Primary Synchronization Corridor, Oncor's operators would have been able to use an alternative route illustrated in the one-line diagrams. Second, Oncor’s Secondary Synchronization Corridor was unaffected by the noncompliance and was available for use during the entirety of the noncompliance. Third, the duration of the noncompliance was relatively short, lasting 39 days. Finally, the noncompliance was discovered when Oncor’s TOA flagged a related request, and through due diligence by Oncor’s TGO Support group. No harm is known to have occurred.

Texas RE has determined that Oncor’s previous instance of noncompliance with EOP-005-2 R4 (TRE2016015567) should not serve as a basis for applying a penalty, but remains as an aggravating factor. The prior and current instances of noncompliance share a similar root cause in that in both instances Oncor failed to implement or adhere to standard procedures requiring it to identify equipment within a proposed work request and update its Blackstart restoration plan accordingly. Texas RE has determined that the recurrence of the activity at issue in the prior Compliance Exception indicates that this noncompliance is appropriate for Find, Fix, Track and Report (FFT) treatment.

Mitigation To mitigate this noncompliance, Oncor:

1) revised its Blackstart Plan to reflect the altered Primary Synchronization Corridor for Permian Basin and submitted a revised Blackstart Plan to its RC;2) created automatic email notifications from Oncor’s updated Transmission Outage Application (iTOA) to be sent to Transmission District Managers and TGO Support Group personnel to heightenawareness of outage requests submitted for equipment that is part of the Blackstart Plan; and 3) provided additional training to Oncor staff that emphasized:

(a) the need to notify the TGO Support Group during the annual review process of any work to be done on equipment included in Blackstart Synchronization Corridors, (b) Oncor's process for development of and revisions to Blackstart units, primary and Secondary Synchronization Corridors, and the Blackstart Plan, and (c) the NERC Standards applicable to the Blackstart Plan.






RFC2018020359 MOD-025-2 R1 American Municipal Power Inc. NCR00683 7/1/2016 8/22/2018 Self-Report Completed Description of the Noncompliance (For purposes of this document, each noncompliance at issue is described as a “noncompliance,” regardless of its procedural posture and whether it was a possible, or confirmed noncompliance.)

On August 30, 2018, the entity submitted a Self-Report stating that, as a Generator Owner, it was in noncompliance with MOD-025-2 R1.

Prior to July 1, 2016, entity personnel tested the reactive and real power output of its Bulk Electric System (BES) facilities. Entity personnel, however, misread the Standard and only tested the Facilities for maximum load. The entity also did not document that it provided its test data to its Transmission Planner for its generation facilities. Therefore, the entity did not have 40% of its generation units tested on or before July 1, 2016 or 60% of its generation units tested on or before July 1, 2017 as required by the implementation schedule for MOD-025-2. The entity discovered these issues while performing an internal review to assure that 80% of its generation facilities had been tested prior to July 1, 2018.

After discovering these issues, the entity engaged in an expedited effort to ensure that it had tested 80% of its generation facilities by July 1, 2018. The entity completed testing on 13 of its 17 BES generation units (77%) by July 1, 2018. The entity completed testing on 16 of its 17 BES generation units (94%) as of August 22, 2018.

This noncompliance involves the management practices of verification and validation. The entity did not verify that it timely and properly completed all of the necessary testing for MOD-025 and met the implementation plan. That failure to verify is a root cause of this noncompliance.

This noncompliance started on July 1, 2016, when the entity was required to comply with MOD-025-2 R1 and ended on August 22, 2018, when the entity completed testing on 16 of its 17 BES generation units, thereby meeting the 80% requirement.

Risk Assessment This noncompliance posed a moderate risk and did not pose a serious or substantial risk to the reliability of the bulk power system based on the following factors. The risk posed by this noncompliance is that by providing incorrect data regarding generating capacity, the veracity of generating models and power flow analyses for planning and operations would be affected. The risk is not minimal because of the long (approximately 28 month) duration. The risk is lessened because 14 of the entity’s 17 units are less than ten years old. The relative newness of these generating units means that there has been minimal degradation and wear resulting in performance levels at (or very close to) the capability curves determined at the time of commissioning. Another risk mitigating factor is that 15 of the 17 entity generating units are run-of-the-river hydro generators that operate within the limits of their variable resource. Therefore, those hydro units would be able to operate at the maximum real-power output of the unit only under the most ideal of circumstances. No harm is known to have occurred.


1) scheduled and completed, real and reactive testing at more than 80% of its generators and reported the results of the testing to the applicable Transmission Planners. This action brought the entityback into compliance with the implementation schedule for MOD-025-2; and

2) scheduled recurring maintenance activities for its generators in its MAXIMO system. To assure that future testing occurs, the entity added a reminder in its MAXIMO system for each BES generationunit (except Hamilton JV2 Gas Turbine) to complete MOD-025-2 testing. The reminders are set to alert the operators one year prior to the end of the five-year cycle mandated by MOD-025-2. ForHamilton JV2 Gas Turbine, a peaking unit that does not use the MAXIMO system, the entity added a calendar reminder to the plant operator’s calendar as well as the calendars of the entity’sGeneration Operations’ Electrical Engineer and the entity’s Director of Reliability Standards Compliance.





RFC2018020360 MOD-025-2 R2 American Municipal Power Inc. NCR00683 7/1/2016 8/22/2018 Self-Report Completed Description of the Noncompliance (For purposes of this document, each noncompliance at issue is described as a “noncompliance,” regardless of its procedural posture and whether it was a possible, or confirmed noncompliance.)

On August 30, 2018, the entity submitted a Self-Report stating that, as a Generator Owner, it was in noncompliance with MOD-025-2 R2. Prior to July 1, 2016, entity personnel tested the reactive and real power output of its Bulk Electric System (BES) facilities. Entity personnel, however, misread the Standard and only tested the Facilities for maximum load. The entity also did not document that it provided its test data to its Transmission Planner for its generation facilities. Therefore, the entity did not have 40% of its generation units tested on or before July 1, 2016 or 60% of its generation units tested on or before July 1, 2017 as required by the implementation schedule for MOD-025-2. The entity discovered these issues while performing an internal review to assure that 80% of its generation facilities had been tested prior to July 1, 2018. After discovering these issues, the entity engaged in an expedited effort to ensure that it had tested 80% of its generation facilities by July 1, 2018. The entity completed testing on 13 of its 17 BES generation units (77%) by July 1, 2018. The entity completed testing on 16 of its 17 BES generation units (94%) as of August 22, 2018. This noncompliance involves the management practices of verification and validation. The entity did not verify that it timely and properly completed all of the necessary testing for MOD-025 and met the implementation plan. That failure to verify is a root cause of this noncompliance. This noncompliance started on July 1, 2016, when the entity was required to comply with MOD-025-2 R1 and ended on August 22, 2018, when the entity completed testing on 16 of its 17 BES generation units thereby meeting the 80% requirement.

Risk Assessment

This noncompliance posed a moderate risk and did not pose a serious or substantial risk to the reliability of the bulk power system based on the following factors. The risk posed by this noncompliance is that by providing incorrect data regarding generating capacity, the veracity of generating models and power flow analyses for planning and operations would be affected. The risk is not minimal because of the long (approximately 28 month) duration. The risk is lessened because 14 of the entity’s 17 units are less than ten years old. The relative newness of these generating units means that there has been minimal degradation and wear resulting in performance levels at (or very close to) the capability curves determined at the time of commissioning. Another risk mitigating factor is that 15 of the 17 entity generating units are run-of-the-river hydro generators that operate within the limits of their variable resource. Therefore, those hydro units would be able to operate at the maximum real-power output of the unit only under the most ideal of circumstances. No harm is known to have occurred. ReliabilityFirst considered the entity’s compliance history and determined there were no relevant instances of noncompliance.

Mitigation

To mitigate this noncompliance, the entity: 1) scheduled and completed, real and reactive testing at more than 80% of its generators and reported the results of the testing to the applicable Transmission Planners. This action brought the entity

back into compliance with the implementation schedule for MOD-025-2; and 2) scheduled recurring maintenance activities for its generators in its MAXIMO system. To assure that future testing occurs, the entity added a reminder in its MAXIMO system for each BES generation

unit (except Hamilton JV2 Gas Turbine) to complete MOD-025-2 testing. The reminders are set to alert the operators one year prior to the end of the five-year cycle mandated by MOD-025-2. For Hamilton JV2 Gas Turbine, a peaking unit that does not use the MAXIMO system, the entity added a calendar reminder to the plant operator’s calendar as well as the calendars of the entity’s Generation Operations’ Electrical Engineer and the entity’s Director of Reliability Standards Compliance.






RFC2018020648 VAR-002-4 R2 Invenergy Nelson LLC NCR11513 8/18/2015 4/10/2017 Self-Report Completed Description of the Noncompliance (For purposes of this document, each noncompliance at issue is described as a “noncompliance,” regardless of its procedural posture and whether it was a possible, or confirmed noncompliance.)

On November 2, 2018, the entity submitted a Self-Report stating that, as a Generator Operator, it was in noncompliance with VAR-002-4 R2. Following an audit notification, the entity reviewed voltage activity for the requested sample dates and found no instances where it did not maintain the relevant schedule. However, the entity later conducted a broader review of its entire voltage history and identified 53 instances in which it did not maintain the relevant voltage schedule and failed to properly notify the Transmission Operator or receive an exemption. More specifically, on August 18, 2015, the entity experienced two voltage deviations that lasted approximately 36 and 38 minutes, respectively. The voltage deviations were less than 1% below the scheduled minimum. Then, throughout 2016, the entity experienced 27 voltage deviations, which lasted approximately between 31 and 232 minutes. The maximum deviation was less than 2% below the scheduled minimum. Also, between January and April 2017, the entity experienced 22 voltage deviations, which lasted approximately between 33 and 177 minutes. The maximum deviation was less than 2% below the scheduled minimum. The root cause of this noncompliance was the entity’s failure to have adequate internal controls in place to quickly detect and correct deviations. Furthermore, the entity’s insufficient training of its generation unit operators with respect to notification requirements contributed to the number of occurrences. This root cause involves the management practices of reliability quality management, which includes maintaining a system for deploying internal controls, and workforce management, which includes providing training, education, and awareness to employees. This noncompliance occurred multiple times over the course of approximately 20 months, from August 18, 2015, when the entity experienced its first voltage deviation, and April 10, 2017, when the entity experienced its last voltage deviation.

Risk Assessment

This noncompliance posed a moderate risk and did not pose a serious or substantial risk to the reliability of the bulk power system (BPS) based on the following factors. The risk posed by this noncompliance was that the entity not adhering to its voltage schedule increased the likelihood that the entity would be unable to respond to changes in voltage caused by reactive power demands and fail to provide voltage support to the BPS. The risk is not minimal in this case based on the number of occurrences (i.e., 53) and the long duration of the noncompliance (i.e., approximately 20 months). The risk is not serious or substantial in this case based on the following factors. First, the deviations were within a narrow band, between 98.43% of the scheduled voltage at the lower end and 100.65% of the scheduled voltage at the higher end. Second, the average duration of each deviation was approximately one hour, and none exceeded four hours. No harm is known to have occurred. ReliabilityFirst considered the entity’s compliance history and determined there were no relevant instances of noncompliance.

Mitigation

To mitigate this noncompliance, the entity: 1) held internal gap analysis discussions, which identified voltage management as an area of improvement; and 2) implemented improved real-time monitoring practices and system controls: An updated graphic display which shows a rolling 30-minute trend for voltage and reactive power, located on the central

control screen, audio and visual alarms, and end of day shift turnover reports. ReliabilityFirst has verified the completion of all mitigation activity.





SPP2018018989 IRO-001-4 R2 Red Dirt Wind Project, LLC (RDW) NCR11779 12/9/2017 12/9/2017 Self-Report Completed


On January 12, 2018, the RDW submitted a Self-Report stating that, as a Generator Operator, it was in noncompliance with IRO-001-4 R2. At the time of the noncompliance, RDW was part of an MRRE Group monitored under the Coordinated Oversight Program that included Osage Wind, LLC (NCR11554) that was registered in the SERC Reliability Corporation (SERC) Region, Smoky Hills Wind Farm, LLC (NCR11049), Caney River Wind Project, LLC (NCR11230), Rocky Ridge Wind Project, LLC (NCR11234), Chisholm View Wind Project, LLC (NCR11291), Buffalo Dunes Wind Project, LLC (NCR11407), Origin Wind Energy, LLC (NCR11496), Goodwell Wind Project, LLC (NCR11574), Prairie Rose Wind Project, LLC (NCR11293), Cimarron Bend Wind Project, LLC (NCR11693), Drift Sand Wind Project LLC (NCR11670), Lindahl Wind Project, LLC (NCR11699), Rock Creek Wind Project, LLC (NCR11762), Thunder Ranch Wind Project, LLC (NCR11778), and Smoky Hills Wind Project II, LLC (NCR10316) that were located in MRO’s Region. After the noncompliance, the aforementioned entities transferred their assets to Smoky Hills Wind Project II, LLC who changed its name to CHI Power, Inc.; CHI Power, Inc. is registered under the same NCR ID in the ReliabilityFirst (RF), SERC, and MRO Regions and is currently monitored under the Coordinated Oversight Program.

On December 7, 2017, RDW’s Reliability Coordinator (RC) determined that if the Woodring - Sooner 345 kV line were lost, there could be loading above the emergency rating of a 345/138 kV transformer at the Woodring substation. To mitigate against a post-contingent over loading of the transformer, the RC issued an Operating Instruction to RDW at 2:38 a.m. to limit its generation output to 75 MW. RDW implemented the order by pausing individual turbines. A few minutes later, there was an increase in local wind speeds and RDW’s generation rose, reaching 89.2 MW within five minutes of the implementation of the Operating Instruction. At 05:49 a.m., the RC called and again requested that generation be limited to 75 MW, RDW implemented the Operating Instruction by pausing additional turbines, however due to fluctuating wind speeds, it was not able to keep the output at or below 75 MW. At 6:38 a.m., RDW’s vendor called and requested authorization to run tests and RDW granted that request. The vendor raised the generation set point to 300 MW and generation rose to 238 MW by 7:02 a.m. At 7:02 a.m., the RC called RDW and again instructed RDW to comply with the existing Operating Instruction that generation be limited to 75 MW. RDW agreed and reduced the output to under 75 MW by 7:12 a.m.

The cause of the noncompliance is that RDW did not have sufficient controls in place to ensure that its operators could understand and implement Operating Instructions; in this case, there was confusion about implementing an Operating Instruction while the Facility was in commissioning.

This noncompliance started on December 9, 2017, when RDW produced generation in excess of the Operating Instruction, and ended later that day when RDW brought its generation below the maximum allowed by the Operating Instruction.

Risk Assessment This noncompliance posed a moderate risk and did not pose a serious or substantial risk to the reliability of the bulk power system (BPS). The noncompliance was not minimal because RDW’s Control Center controls over 1,000 MVA of nameplate generation. This means that the potential risk of failing to properly respond to an Operating Instructions is not limited to the RDW Facility as the Control Center has the potential to impact the wider BPS. However, the noncompliance was not serious or substantial because the noncompliance did not impact an Interconnection Reliability Operating Limit (IROL). Further, despite RDW’s noncompliance, based on the observed post-contingent loading while the Operating Instruction was in effect, if the 345 kV line had been lost, the transformer would have only exceeded its emergency rating between 6:00 a.m. and 6:10 a.m. Finally, if conditions had warranted, RDW’s RC or Transmission Operator could have removed RDW’s generation from service without significant adverse effects to the BPS. No harm is known to have occurred.

MRO reviewed the compliance history for RDW and all entities that were included in the MRRE Group; the entities have no relevant history of noncompliance.

Mitigation To mitigate this noncompliance, RDW:

1) complied with the Operating Instruction;2) reviewed the coordination process for commissioning tests;3) updated operator procedures including the distribution of a memo prohibiting new site testing while an Operating Instruction is in effect; and4) retrained control room operators.

A Mitigation Plan for this noncompliance was verified complete on May 25, 2018.





TRE2018020762 PRC-024-2 R1 Consolidated Edison Development, Inc. (CED)

NCR11605

07/01/2016

05/03/2019 Self-Report Completed


On November 29, 2018, Consolidated Edison Development, Inc. (CED) submitted a Self-Report stating that, as a Generator Owner (GO), it was in noncompliance with PRC-024-2 R1. In particular, CED failed to set its frequency protective relays to not trip within the “no trip zone” of PRC-024-2, Attachment 1, for its Alamo 5 and Alamo 7 solar generating Facilities by July 1, 2016. On June 7, 2016, through consultation with the Original Equipment Manufacturer (OEM), CED discovered that frequency relays at Alamo 5 and Alamo 7 were set to comply with ERCOT Nodal Operating Guide voltage and frequency ride-through requirements, and not with PRC-024-2. CED continued to work with the OEM to determine the precise relay settings at each Facility, but was unable to complete a full review until August 7, 2018. CED was unable to independently verify the relay settings at its Facilities because the OEM relies on a proprietary tool to review and update relay settings. On October 1, 2018, CED developed a baseline settings list and began working with the OEM to update the relay settings at Alamo 5 and Alamo 7. On November 29, 2018, CED Self-Reported the noncompliance. A Compliance Audit, in which PRC-024-2 was in-scope, was conducted by Texas RE March 18, 2019, through March 22, 2019. CED’s Facilities became compliant with PRC-024-2 on May 3, 2019. The root cause of this noncompliance was that CED failed to establish adequate controls around relay changes made by internal personnel and third-party vendors that impact plant control systems. This noncompliance started on July 1, 2016, when PRC-024-2 became mandatory and enforceable and ended on May 3, 2019, when CED updated relay settings at Alamo 5 and Alamo 7 so they did not trip within the “no trip zone.”

Risk Assessment

This noncompliance posed a minimal risk and did not pose a serious or substantial risk to the reliability of the bulk power system for the following reasons. During the period of the noncompliance with PRC-024-2, Alamo 5 and Alamo 7's relays were in compliance with ERCOT Nodal Operating Guide frequency ride-through requirements. While not identical to PRC-024-2, this partially mitigated the potential for frequency related trips. Additionally, Alamo 5 and Alamo 7 are intermittent solar photovoltaic facilities, which are online far fewer hours than conventional generation. Frequency trips at these Facilities at night would have zero impact to the bulk power system, and frequency trips during suboptimal production would have minimal impact. Although the maximum peak output of the generating facilities are over 75 MVA, over the past year, the average hourly generation output has been 23.8 MW at Alamo 5 and 29.2 MW at Alamo 7. No harm is known to have occurred. Texas RE considered CED’s and its affiliates’ PRC-024-2 compliance history and determined there were no relevant instances of noncompliance.

Mitigation To mitigate this noncompliance, CED: 1) implemented relay settings that are compliant with PRC-024; and 2) implemented a Change Management Process to establish controls around any changes made by internal personnel and third party vendors that impact plant control systems. All changes that

impact facility control systems must be reviewed and approved by CED's Engineering and Operations, and Maintenance staff in accordance with the Change Management Process prior to implementation of any proposed modifications.






TRE2019021481 PRC-024-2 R2 Consolidated Edison Development, Inc. (CED)

NCR11605

07/01/2016

05/03/2019 Compliance Audit Completed


During a Compliance Audit conducted from March 18, 2019 through March 22, 2019, Texas RE determined that Consolidated Edison Development, Inc. (CED), as a Generator Owner (GO), was in noncompliance with PRC-024-2 R2. In particular, CED failed to set its voltage protective relays to not trip within the “no trip zone” of PRC-024-2, Attachment 2, for its Alamo 5 and Alamo 7 solar generating Facilities by July 1, 2016. On June 7, 2016, through consultation with the Original Equipment Manufacturer (OEM), CED discovered that voltage relays at Alamo 5 and Alamo 7 were set to comply with ERCOT Nodal Operating Guide voltage and frequency ride-through requirements, and not with PRC-024-2. CED continued to work with the OEM to determine the precise relay settings at each Facility, but was unable to complete a full review until August 7, 2018. CED was unable to independently verify the relay settings at its Facilities because the OEM relies on a proprietary tool to review and update relay settings. On October 1, 2018, CED worked with the OEM and was able to develop a baseline settings list. During the Compliance Audit the audit team reviewed the baseline settings list and the noncompliance was confirmed. CED began working with the OEM to update the relay settings at Alamo 5 and Alamo 7 and its Facilities became compliant with PRC-024-2 on May 3, 2019. The root cause of this noncompliance was that CED failed to establish adequate controls around relay changes made by internal personnel and third-party vendors that impact plant control systems. This noncompliance started on July 1, 2016, when PRC-024-2 became mandatory and enforceable and ended on May 3, 2019, when CED updated relay settings at Alamo 5 and Alamo 7 so they did not trip within the “no trip zone.”

Risk Assessment

This noncompliance posed a minimal risk and did not pose a serious or substantial risk to the reliability of the bulk power system for the following reasons. During the period of the noncompliance with PRC-024-2, Alamo 5 and Alamo 7's relays were in compliance with ERCOT Nodal Operating Guide voltage ride-through requirements. While not identical to PRC-024-2, this partially mitigated the potential for voltage related trips. Additionally, Alamo 5 and Alamo 7 are intermittent solar photovoltaic facilities, which are online far fewer hours than conventional generation. Voltage trips at these Facilities at night would have zero impact to the bulk power system, and voltage trips during suboptimal production would have minimal impact. Although the maximum peak output of the generating facilities are over 75 MVA, over the past year, the average hourly generation output has been 23.8 MW at Alamo 5 and 29.2 MW at Alamo 7. No harm is known to have occurred. Texas RE considered CED’s and its affiliates’ PRC-024-2 compliance history and determined there were no relevant instances of noncompliance.

Mitigation To mitigate this noncompliance, CED: 1) implemented relay settings that are compliant with PRC-024-2; and 2) implemented a Change Management Process to establish controls around any changes made by internal personnel and third party vendors that impact plant control systems. All changes that

impact facility control systems must be reviewed and approved by CED's Engineering and Operations, and Maintenance staff in accordance with the Change Management Process prior to implementation of any proposed modifications.






TRE2017018281

PRC-005-1.1b R2 Hackberry Wind, LLC (HWF) (the “Entity”)

NCR00210

01/29/2016 10/24/2017 Compliance Audit Completed


During a Compliance Audit conducted from July 25, 2017, through August 30, 2017, Texas RE determined that the Entity, as a Generator Owner (GO), was in noncompliance with PRC-005-1.1b R2. Specifically, the Entity failed to provide evidence that its protective relays and DC control circuitry devices were maintained and tested within the defined intervals included in its Protection System Maintenance Program (PSMP). This noncompliance continued during the time when PRC-005-6 R3 was effective. After receiving notice of the Compliance Audit, the Entity identified this issue during an internal review following a change in compliance personnel. The Entity’s PSMP identified a time-based maintenance program for the Entity’s protective relay and DC control circuitry devices, with maintenance activities due every 90 months. Maintenance activities were performed for the devices at issue during July 28, 2008, through August 7, 2008. Therefore, the next interval of maintenance activities should have been performed by January 28, 2016, through February 7, 2016. The failure to timely perform maintenance and testing for these devices also caused the Entity to fail to meet the April 1, 2017, milestone for compliance with the implementation plan for PRC-005-6 R3. Thus, this issue resulted in noncompliance regarding both PRC-005-1.1b R2 and PRC-005-6 R3. The root cause of this issue is that the Entity did not have a sufficient process for compliance with PRC-005-1.1b and PRC-005-6. The Entity stated that it did not devote sufficient resources and personnel to compliance activities regarding PRC-005-1.1b and PRC-005-6. To address this root cause, the Entity revised its PSMP and devoted additional resources to its compliance program. This noncompliance started on January 29, 2016, when the Entity failed to timely perform the maintenance activities for its protective relays and DC control circuitry devices required by its PSMP, and ended on October 24, 2017, the Entity performed maintenance activities with six-year maximum intervals for the devices at issue pursuant to PRC-005-6 Table 1-1 and Table 1-5.

Risk Assessment This noncompliance posed a minimal risk and did not pose a serious or substantial risk to the reliability of the bulk power system (BPS). The risk posed by this issue is that the Entity would not be aware that a Protection System device was not functioning as intended. In addition, the duration of the issue was approximately 21 months, from January 29, 2016, to October 24, 2017. However, the risk to the reliability of the BPS was reduced by the following factors. First, the Entity’s generating Facility is relatively small, comprising a single wind generator site with a nameplate rating of 185 MVA. Second, the issue involved only 15 protective relays and 14 DC control circuitry devices, which represents approximately 42% of the 69 devices included in the Entity’s PSMP. Third, the Entity did not identify any devices that had failed when it performed the required maintenance activities. Finally, no trips or Misoperations were identified as resulting from the issues identified during the Compliance Audit. No harm is known to have occurred. Texas RE considered the Entity’s compliance history and determined there were no relevant instances of noncompliance.

Mitigation

To mitigate this noncompliance, the Entity: 1) performed the required maintenance activities for the devices at issue; 2) conducted training for the Entity’s employees, including an overview of applicable Reliability Standards; 3) added personnel and consulting services to improve its compliance program; and 4) created an automatic reminder for the next interval of maintenance activities with six-year maximum maintenance intervals for the devices at issue. Texas RE verified the completion of all mitigation activity.





WECC2016016576 EOP-008-1 R1 CXA Sundevil Holdco, Inc. (GRMA) NCR05169 7/1/2013 11/23/2013 Compliance Audit Completed

Description of the Noncompliance (For purposes of this document, each noncompliance at issue is described as a “noncompliance,” regardless of its procedural posture and whether it was a possible or confirmed violation.)

During a Compliance Audit conducted from September 26, 2016 through October 7, 2016, WECC determined that GRMA, as a Balancing Authority (BA), it was in potential noncompliance with EOP-008-1 R1.

Specifically, prior to the registration of GRID to perform the BA functions for GRMA, GRID was already contractually performing the BA functions and the Operating Plan was designed, documented and implemented by GRID on behalf of its clients. WECC found several issues with the Operating Plan GRMA utilized;

a. it defined the backup functionality as being provided by remotely accessing the BA functionality from specified hotel lobbies and using laptops instead of transferring operations to a specificbackup facility. GRMA incorporated an incorrect definition of facility, citing the use of laptops in a hotel lobby as implementing backup functionality in addition to an “alternate” Control Center,which did not meet the criteria of backup functionality provided by FERC’s directives in Order 6931 (R1.1);

b. it listed laptop batteries as the backup power supply to the hotel building power for use from the hotel lobbies (R1.2.4);c. it did not include physical or cyber security in the hotel lobbies (R1.2.5);d. it did not include a transition period between the loss of primary control center functionality and the time to transition to the alternate control center in Austin, Texas which was used for low

probability high impact events, such as hurricanes requiring evacuation of Houston, Texas. Specifically, the primary Control Center and the alternate Control Center were two and a half hours awayfrom each other by car resulting in a period over the two-hour limit (R1.5).

e. for these reasons, GRMA did not include actions to manage the risk to the BES during the transition from primary to backup functionality as well as during outages of the primary or backupfunctionality because GRMA assumed that its operators would be able to gain full operational functionality in under two hours from the hotel lobbies whenever required (R1.6.2).

After reviewing all relevant information, WECC determined that WECC determined that GRMA failed to have an Operating Plan describing the manner in which it continues to meet its functional obligations with regard to the reliable operations of the BES in the event that its primary control center functionality is lost that meets the requirements of EOP-008-1 R1, specifically R1.1, R1.2.4, R1.2.5, R1.5, and R1.6.2. There was a corresponding EOP-008-1 R1 violation for GRID, NERC Violation ID, WECC2016016377.

The root cause of the noncompliance was the incorrect assumptions regarding the criteria for its Operating Plan and not considering the specific sub-requirements of EOP-008-1 R1 nor FERC’s directives when it designed and created its Operating Plan.

WECC determined that this issue began on July 1, 2013, when the Standard became mandatory and enforceable and ended on November 23, 2013, when GRID registered to perform BA functions on behalf of GRMA, for a total of 146 days noncompliance.

Risk Assessment WECC determined that this issue posed a moderate risk and did not pose a serious or substantial risk to the reliability of the Bulk Power System (BPS). In this instance, GRMA failed to have an Operating Plan describing the manner in which it continues to meet its functional obligations with regard to the reliable operations of the BES in the event that its primary control center functionality is lost that meets the requirements of EOP-008-1 R1, specifically R1.1, R1.2.4, R1.2.5, R1.5, and R1.6.2. Such failure could result in GRMA not having the system functionality, power sources, nor physical and cyber security controls for backup functionality in place within the required transition period, which could result in a delay or failure in performing its BA obligations and a negative impact the BPS. In addition, personnel tasked with transferring functions to the backup or alternate control center may not understand the time requirement, prolonging the risk of a loss of generation or load. GRMA was responsible for 1,458 MW that was applicable to this issue. Therefore, WECC assessed the potential harm to the security and reliability of the BPS as intermediate.

GRMA did not have effective internal controls to detect or prevent this issue. However, as compensation, the Operating Plan was used successfully for backup Control Center functionality on December 14, 2012, due to a bomb threat. In addition, the Operating Plan was used successfully during hurricane evacuation conditions and for routine training and testing of remote functionality verifying all functions could be performed using remote access functionality from 2012 to 2013. Based on this, WECC determined that there was a moderate likelihood of causing intermediate harm to the BPS. No harm is known to have occurred.





WECC2016016576 EOP-008-1 R1

CXA Sundevil Holdco, Inc. (GRMA)

NCR05169

7/1/2013

11/23/2013 Compliance Audit

Completed

Mitigation

To mitigate this issue, GRMA: a. GRID registered to perform the BA functions on behalf of GRMA; b. engaged a real estate firm to assist with identification of a space that will be managed by the primary BA that is accessible in approximately 90 minutes or less; c. visited spaces that have been identified by the real estate firm as potential facilities; d. modified the Operating Plan to include a summary of the risk assessment for power supply needs during a loss of primary control center condition; e. negotiated the lease and build out requirements; f. established the new EOP-008 Operating Plan that is inclusive of the primary BA managed designated facility; g. established new Operating Plan inclusive of the primary BA managed facility; and h. built out the leased space to meet requirements for backup functionality established in the EOP-008 risk based assessment. On January 30, 2018, GRMA submitted a Mitigation Plan Completion Certification and on March 7, 2018, WECC verified GRMA’s completion of Mitigation Plan. Upon undertaking the actions outlined in the Mitigation Plan, GRMA took voluntary corrective action to remediate this issue. WECC notes that GRMA does not have any relevant previous violations of this or similar Standards and Requirements. WECC considered these factors in its designation of this remediated issue as an FFT.





TRE2018020184 IRO-008-2 R4 Electric Reliability Council of Texas, Inc. (ERCOT ISO)



On August 6, 2018, ERCOT ISO submitted a Self-Report to Texas RE stating that, as a Reliability Coordinator (RC), it was in noncompliance IRO-008-2 R4. Specifically, ERCOT ISO failed to ensure that a Real-time Assessment (RTA) was performed at least once every 30 minutes in one instance on June 29, 2018.

ERCOT ISO’s process to perform RTAs includes the use of a State Estimator, which creates a save case describing present conditions, and a Real-Time Contingency Analysis (RTCA), which evaluates the save case under various contingencies. On June 29, 2018, ERCOT ISO’s Real-Time Contingency Analysis (RTCA) failed to execute for a 43-minute period between 9:43 p.m. and 10:26 p.m., which prevented ERCOT ISO from performing an RTA during that period. As a result, ERCOT ISO exceeded the 30-minute deadline to perform an RTA pursuant to IRO-008-2 R4 by 13 minutes.

The root cause of the noncompliance was a flaw in the software used by ERCOT ISO to manually create contingencies to be used to create save cases that are evaluated by the RTCA software, as well as an insufficient process to verify a save case before it is saved for use in the RTA process. In this instance, based on the expected unavailability of certain Transmission Elements, ERCOT ISO personnel attempted to revise the save case created by the State Estimator by adding additional manual constraints evaluated by the RTCA software. However, the RTCA software failed to execute after the manual constraints were introduced, which was caused by a flaw in the software used to incorporate manual constraints in the save case. Several months prior to the noncompliance, ERCOT ISO was aware of the software defect and began work on a software update, but, during this time, ERCOT ISO did not implement sufficient controls to prevent personnel from inadvertently creating an invalid save case in the RTCA process. In particular, ERCOT ISO did not disable the ability to incorporate manual constraints until after the noncompliance occurred. In addition, ERCOT ISO has the ability to use its offline Study Contingency Analysis (STCA) process, which is intended to provide a backup to the online RTCA process, but, at that time, the most current save case to be used for the STCA process was already affected by the same software defect that prevented the RTCA software from executing. Subsequently, ERCOT ISO implemented a process to verify a save case before it can be saved for use in the RTCA or STCA processes.

During the noncompliance, ERCOT ISO’s State Estimator continued to execute, but ERCOT ISO’s Voltage Security Assessment Tool (VSAT), which calculates certain reliability limits, failed to execute between 9:42 p.m. and 10:28 p.m.

The violation started on June 29, 2018, at 10:14 p.m., which is 31 minutes after an RTA was performed, and ended on June 29, 2018, at 10:26 p.m., when an RTA was performed.

Risk Assessment This noncompliance posed a moderate risk and did not pose a serious or substantial risk to the reliability of the bulk power system. ERCOT ISO’s failure to perform an RTA for 13 minutes could have potentially reduced ERCOT ISO’s situational awareness. However, the risk posed by this issue was reduced by the following factors. First, during the noncompliance the State Estimator continued to execute, allowing ERCOT ISO to continue to perform the pre-Contingency portion of the RTA. ERCOT ISO was also able to observe any outages using the Forced Outage Detection Tool. Second, during the noncompliance, ERCOT ISO directed Transmission Operators (TOPs) to monitor their respective service areas, and no TOPs notified ERCOT ISO of any issues. Third, although VSAT failed to execute during the noncompliance, the existing voltage limits calculated by ERCOT ISO’s VSAT remained valid because no forced outages occurred that would have invalidated the previous calculated limits. ERCOT ISO continued to monitor actual flows relative to those limits. No harm is known to have occurred.

A Settlement Agreement covering IRO-002-2 R7 (TRE2016016699), IRO-003-2 R1 (TRE2016016700), IRO-003-2 R2 (TRE2016016701), IRO-005-3.1a R1 (TRE2016016702), IRO-008-2 R4 (TRE2017017719), and TOP-001-3 R13 (TRE2017017720) was filed with FERC under NP18-10-000 on April 30, 2018. On May 30, 2018, FERC issued an order stating it would not engage in further review of the Notice of Penalty.

Texas RE considered this compliance history in determining that this issue is appropriate for Find, Fix, and Track (FFT) treatment. While the instances were of the same or similar Reliability Standards and Requirements and involved a failure to timely perform an RTA, the underlying conduct of the instances was different. One prior instance was caused by a lack of detailed instructions during the system troubleshooting process to begin manually performing an RTA, and the other prior instance was the result of an insufficient process to prevent test data from being inadvertently loaded in to an active production environment.

Mitigation To mitigate this noncompliance, ERCOT ISO:

1) implemented software updates to allow the creation of an RTA save case only after RTCA has successfully executed and to disable the ability to create a manual constraint for a group ofgenerators;

2) implemented a software update to fix the software flaw that caused the RTCA execution failure; and3) created an additional offline backup to the existing RTA process, which automatically stores offline cases and performs RTAs using the last valid State Estimator and RTCA save cases.






TRE2018020185 TOP-001-3 R13 Electric Reliability Council of Texas, Inc. (ERCOT ISO)



On August 6, 2018, ERCOT ISO submitted a Self-Report to Texas RE stating that, as a Transmission Operator (TOP), it was in noncompliance TOP-001-3 R13. Specifically, ERCOT ISO failed to ensure that a Real-time Assessment (RTA) was performed at least once every 30 minutes in one instance on June 29, 2018.

ERCOT ISO’s process to perform RTAs includes the use of a State Estimator, which creates a save case describing present conditions, and a Real-Time Contingency Analysis (RTCA), which evaluates the save case under various contingencies. On June 29, 2018, ERCOT ISO’s Real-Time Contingency Analysis (RTCA) failed to execute for a 43-minute period between 9:43 p.m. and 10:26 p.m., which prevented ERCOT ISO from performing an RTA during that period. As a result, ERCOT ISO exceeded the 30-minute deadline to perform an RTA pursuant to TOP-001-3 R13 by 13 minutes.

The root cause of the noncompliance was a flaw in the software used by ERCOT ISO to manually create contingencies to be used to create save cases that are evaluated by the RTCA software, as well as an insufficient process to verify a save case before it is saved for use in the RTA process. In this instance, based on the expected unavailability of certain Transmission Elements, ERCOT ISO personnel attempted to revise the save case created by the State Estimator by adding additional manual constraints evaluated by the RTCA software. However, the RTCA software failed to execute after the manual constraints were introduced, which was caused by a flaw in the software used to incorporate manual constraints in the save case. Several months prior to the noncompliance, ERCOT ISO was aware of the software defect and began work on a software update, but, during this time, ERCOT ISO did not implement sufficient controls to prevent personnel from inadvertently creating an invalid save case in the RTCA process. In particular, ERCOT ISO did not disable the ability to incorporate manual constraints until after the noncompliance occurred. In addition, ERCOT ISO has the ability to use its offline Study Contingency Analysis (STCA) process, which is intended to provide a backup to the online RTCA process, but, at that time, the most current save case to be used for the STCA process was already affected by the same software defect that prevented the RTCA software from executing. Subsequently, ERCOT ISO implemented a process to verify a save case before it can be saved for use in the RTCA or STCA processes.

During the noncompliance, ERCOT ISO’s State Estimator continued to execute, but ERCOT ISO’s Voltage Security Assessment Tool (VSAT), which calculates certain reliability limits, failed to execute between 9:42 p.m. and 10:28 p.m.

The violation started on June 29, 2018, at 10:14 p.m., which is 31 minutes after an RTA was performed, and ended on June 29, 2018, at 10:26 p.m., when an RTA was performed.

Risk Assessment This noncompliance posed a moderate risk and did not pose a serious or substantial risk to the reliability of the bulk power system. ERCOT ISO’s failure to perform an RTA for 13 minutes could have potentially reduced ERCOT ISO’s situational awareness. However, the risk posed by this issue was reduced by the following factors. First, during the noncompliance the State Estimator continued to execute, allowing ERCOT ISO to continue to perform the pre-Contingency portion of the RTA. ERCOT ISO was also able to observe any outages using the Forced Outage Detection Tool. Second, during the noncompliance, ERCOT ISO directed TOPs to monitor their respective service areas, and no TOPs notified ERCOT ISO of any issues. Third, although VSAT failed to execute during the noncompliance, the existing voltage limits calculated by ERCOT ISO’s VSAT remained valid because no forced outages occurred that would have invalidated the previous calculated limits. ERCOT ISO continued to monitor actual flows relative to those limits. No harm is known to have occurred.


Texas RE considered this compliance history in determining that this issue is appropriate for Find, Fix, and Track (FFT) treatment. While the instances were of the same or similar Reliability Standards and Requirements and involved a failure to timely perform an RTA, the underlying conduct of the instances was different. One prior instance was caused by a lack of detailed instructions during the system troubleshooting process to begin manually performing an RTA, and the other prior instance was the result of an insufficient process to prevent test data from being inadvertently loaded in to an active production environment.


1) implemented software updates to allow the creation of an RTA save case only after RTCA has successfully executed and to disable the ability to create a manual constraint for a group ofgenerators;

2) implemented a software update to fix the software flaw that caused the RTCA execution failure; and3) created an additional offline backup to the existing RTA process, which automatically stores offline cases and performs RTAs using the last valid State Estimator and RTCA save cases.






TRE2019021584 IRO-002-5 R5 Electric Reliability Council of Texas, Inc. (ERCOT ISO) NCR04056 08/14/2018 08/21/2018 Compliance Audit Completed Description of the Noncompliance (For purposes of this document, each noncompliance at issue is described as a “noncompliance,” regardless of its procedural posture and whether it was a possible, or confirmed violation.)

During a Compliance Audit conducted from November 5, 2018 through November 16, 2018, Texas RE determined that ERCOT ISO, as a Reliability Coordinator (RC), was in noncompliance with IRO-002-5 R5. Between August 14, 2018, and August 21, 2018, ERCOT ISO failed to monitor certain Facilities to identify System Operating Limit (SOL) exceedances within its Reliability Coordinator Area under post-Contingency conditions for 947 contingencies.

ERCOT ISO’s process to perform a Real-time Assessment (RTA) includes the use of a State Estimator, which creates a save case describing present conditions, and a Real-Time Contingency Analysis (RTCA) tool, which evaluates the save case under various contingencies. On August 14, 2018, at 11:56 p.m., ERCOT ISO performed its weekly update to the contingencies to be examined by the RTCA tool. However, due to a flaw in the automated import tool, ERCOT ISO unintentionally disabled 947 out of a total of approximately 7,300 contingencies. ERCOT ISO discovered this issue on August 21, 2018, at 10:30 p.m. and replaced the disabled contingencies on August 21, 2018, at 11:55 p.m. As a result, during August 14, 2018, through August 15, 2018, ERCOT ISO was not monitoring certain Facilities to identify post-Contingency SOL exceedances in its Reliability Coordinator Area, in violation of IRO-002-5 R5. However, the RTCA software continued to execute, and ERCOT ISO continued to perform RTAs during the period, as required by IRO-008-2 R4.

The root cause of this issue was a flaw in the software used by ERCOT ISO to import lists of contingencies, combined with an insufficient process to verify the accuracy of the contingencies and models used in the RTCA process. The automated tool used by ERCOT ISO had a flaw that, under certain conditions, would result in the disabling of large groups of contingencies. ERCOT ISO was not previously affected by the software flaw because this instance was the first time that ERCOT ISO had used the software tool under the conditions that would result in the inadvertent disabling of contingencies. However, the Compliance Audit also identified that ERCOT ISO did not have sufficient internal controls to ensure timely correction of temporary data replacements in real-time systems. Specifically, while ERCOT ISO did automatically generate emails to notify certain personnel when the weekly changes were applied, ERCOT ISO did not have automatic alerts when large or aberrational changes were applied and did not have a sufficient process to perform real-time verifications during periods between weekly changes. As a result, ERCOT ISO did not discover this issue until a week after it began, when ERCOT ISO was performing the next weekly update of the list of contingencies.

The violation started on August 14, 2018, at 11:56 p.m., when ERCOT ISO unintentionally disabled 947contingencies monitored by the RTCA tool, and ended on August 21, 2018, at 11:55 p.m., when ERCOT ISO implemented a revised list of contingencies.

Risk Assessment This noncompliance posed a moderate risk and did not pose a serious or substantial risk to the reliability of the bulk power system. ERCOT ISO’s failure to monitor certain Facilities to identify SOLs under post-Contingency conditions for 947 contingencies for one week reduced ERCOT ISO’s situational awareness. Of the disabled contingencies, five contingencies were associated with post-Contingency SOL exceedances during the noncompliance. For these six disabled contingencies, Texas RE identified 41 instances of post-Contingency SOL exceedances that occurred during the noncompliance.

However, the risk posed by this issue was reduced by the following factors. First, during the noncompliance, the State Estimator continued to execute, allowing ERCOT ISO to monitor real-time conditions for the affected Facilities and would have been able to respond to SOL exceedances that occurred in the pre-Contingency time period. Second, the nature and number of the disabled contingencies also reduced the risk posed by this issue. The number of disabled contingencies was only 947 out of approximately 7,300 total contingencies evaluated by ERCOT ISO, which is approximately 13% of the total number of contingencies. None of the disabled contingencies impacted the assessment of the voltage stability limits for the Rio Grande Valley or Houston-area Import areas. ERCOT ISO also noted that all of the disabled contingencies were single-circuit contingencies involving 69 kV or 138 kV Facilities. Finally, many of the disabled contingencies are associated with Transmission Operators (TOPs) that have the ability to perform monitoring of post-Contingency conditions for their own systems. Of the five disabled contingencies that were associated with post-Contingency SOL exceedances, four are associated with TOPs that have post-Contingency analysis capabilities, and of the total of 947 disabled contingencies, 762 are associated with TOPS that have post-Contingency analysis capabilities. Further, all associated TOPs have pre-Contingency monitoring capabilities. No harm is known to have occurred.


Texas RE considered this compliance history in determining that this issue is appropriate for Find, Fix, and Track (FFT) treatment. While the instances were of the same or similar Reliability Standards and Requirements, the underlying conduct of the instances was different. The compliance history described above involved a failure to ensure that an RTA was timely performed, which is not an issue in this case. Further, this compliance history was caused by a lack of detailed instructions during the system troubleshooting process to begin manually performing an RTA and by an insufficient process to prevent test data from being inadvertently loaded in to an active production environment, which were also not issues in this case.


1) resumed monitoring the contingencies at issue;2) revised the weekly model loading and review process to add notifications to ERCOT ISO personnel showing the number of disabled contingencies and to add a review of disabled contingencies

to the agenda for weekly meetings;3) revised the documented process for model-loading procedures to include verifying the number of disabled contingencies when loading new model information;4) revised the documented process for the Advanced Network Applications group’s database-loading procedures to include verifying the number of active and inactive contingencies;



5) implemented a software update to prevent the unintentional disabling of contingencies when importing new contingency sets;6) created an automatic notification system to alert ERCOT ISO personnel of significant changes to the number of disabled contingencies; and7) modified the “Contingency Solution Results” display to include the total number of inactive contingencies.






RFC2018020645 PRC-005-6 R3 Public Service Electric & Gas Company NCR00896 10/23/2017 10/5/2018 Self-Report CompletedDescription of the Noncompliance (For purposes of this document, each noncompliance at issue is described as a “noncompliance,” regardless of its procedural posture and whether it was a possible, or confirmed noncompliance.)

On November 2, 2018, the entity submitted a Self-Report stating that, as a Transmission Owner, it was in noncompliance with PRC-005-6 R3 (The entity initially submitted the Self-Report under PRC-005-6 R1. After discussions with the entity, ReliabilityFirst determined that the instance of noncompliance was not a violation of PRC-005-6 R1, but, rather, was a violation of PRC-005-6 R3.)

The entity failed to verify a communication system as functional as per the PRC-005-6 Table 1-2 requirements. The entity discovered this noncompliance during a NERC Standards and Compliance Group (NS&C) internal controls review of all entity Power Line Carrier System maintenance activities. The entity power line carrier (PLC) systems which are part of the Bulk Electric System (BES) are maintained as per the NERC requirements specified in PRC-005-6 under Table 1-2 communication systems. The entity currently has a total of 36 power line carriers which have component attributes that meet the definition of either monitored or unmonitored and are maintained as per the specific requirements prescribed for each attribute.

The noncompliance occurred on the entity's 500 kV tie line 5016, which runs from the entity's Branchburg Switching Station to the PPL Alburtis Station. The power line carrier communication system on this line met the attributes associated with a monitored communication system described in Table 1-2. This communication system performed periodic automated testing for the presence of the channel function, and alarming for loss of function. A part of the communication system for tie line 5016 is the RFL 9785 carrier transceiver unit. A carrier check-back card had been installed into the existing RFL 9785 carrier transceiver unit to perform automatic check-back tests.

On October 23, 2017, an unsuccessful carrier check on tie line 5016 triggered a carrier check-back failure alarm. A Relay Technician was notified of the alarm and investigated the problem, but incorrectly assumed the alarm was caused by a PLC on the line that was previously retired in place and tagged as “out of service.” As a result, the Technician erroneously disconnected the alarm without further investigating the issue. The Relay Technician was unaware of the installation of the carrier check-back card on the RFL 9785, which caused the alarm.

A year later, on September 14, 2018, during the NS&C Group internal controls review of the entity's Branchburg Switching Station, an NS&C Relay Test Engineer observed that the tie line 5016 RFL 9785 carrier transceiver unit was in "Carrier Check Back Failure" alarm state, but that no alarm was posted on the panel. Following this, the entity conducted an internal investigation and determined that the relay technician admitted to making an incorrect assumption which led to the mistake (disconnecting the alarm) described above.

This noncompliance involves the management practices of workforce management, validation, and verification. The root cause of this noncompliance is the entity’s failure to verify a communication system as functional per the PRC-005-6 table 1-2 specified requirements because of ineffective training. The relay technician was not effectively trained and did not verify that his understanding of what he was required to do was correct.

This noncompliance started on October 23, 2017, when the relay technician incorrectly disconnected the alarm and ended on October 5, 2018, when the entity repaired the automatic check-back system, enabled the alarms, and returned it to service.

Risk Assessment This noncompliance posed a moderate risk and did not pose a serious or substantial risk to the reliability of the bulk power system based on the following factors. The risk posed by disconnecting the alarm is undetected failure of the communication system could lead to a misoperation and loss of the 500 kV tie line. The risk is not minimal because the line operates at 500 kV and is connected to two other 500 kV lines that are part of the Eastern Reactive Transfer Interface and because of the approximately one year duration. This risk is lessened by the following factors. First, the primary and backup protection systems were fully functional throughout the noncompliance. Only the communication systems monitoring experienced an issue. This means that the systems were in place and operating to protect the BPS. Second, the blocking carrier system was fully functional and would have operated to help prevent a line trip outside the relay's designated zone of protection. Third, the board failure would not have prevented the line protection from operating to clear a fault within its designated zones and the relay protection would have functioned as designed. The line protection consists of a primary and completely redundant back-up protection scheme with breaker failure and high speed remote tripping. (During the time this equipment remained out of service, there were no misoperations on this line.) No harm is known to have occurred.

ReliabilityFirst considered the entity’s compliance history and determined there were no relevant instances of noncompliance.

Mitigation To mitigate this noncompliance, the entity completed the following mitigation activities:

1) performed a manual check on the communication system;2) placed the Line 5016 Blocking Carrier System on the list of communication systems which are tested manually every three months at a maximum;3) repaired the Line 5016 Automatic Check-back System, enabled the alarms and returned it to service;4) placed all monitored power line carrier systems onto the list of communication systems which are tested manually every three months at a maximum, with the exception of Hope Creek Generating

Station (PSEG Nuclear);5) communicated these findings to all applicable Division personnel as lessons learned. Electric Division Managers were included in the communications;6) provided an update to ReliabilityFirst on the status;





RFC2018020645 PRC-005-6 R3 Public Service Electric & Gas Company NCR00896 10/23/2017 10/5/2018 Self-Report May 31, 2019

7) created a procedure which will allow for the Hope Creek monitored power line carrier systems to be tested every three months and will add the Hope Creek monitored power line carrier systems tothe manual testing schedule of every three months (maximum); and

8) performed an Internal Assessment and Focused Compliance Review of the entire PRC-005-6 standard.





RFC2018019282 PRC-004-5(i) R6 DTE Electric Company NCR00753 12/31/2017 2/16/2018 Self-Report Completed Description of the Noncompliance (For purposes of this document, each noncompliance at issue is described as a “noncompliance,” regardless of its procedural posture and whether it was a possible, or confirmed noncompliance.)

On February 22, 2018, the entity submitted a Self-Report stating that, as a Distribution Provider and Generator Owner, it was in noncompliance with PRC-004-5(i) R6. The entity failed to implement or update a Corrective Action Plan (CAP) designed to address the cause of a misoperation. The misoperation occurred on August 31, 2017, when Unit 7 at the St. Clair Power Plant cleared from the system after synchronizing and increasing output. The misoperation was caused by a Current Transformer (CT) shorting switch that was in the wrong position.

An after action review was conducted on September 29, 2017, and the entity developed a CAP, which included tasks and corresponding deadlines. However, the entity subsequently failed to implement or update the CAP before the target completion date of December 31, 2017. The entity discovered the issue on February 9, 2018, while collecting 2017 Q4 relay data.

The root cause of this noncompliance was an inadequate process to track and verify the performance and completion of tasks identified in the CAP. This noncompliance implicates the management practice of workforce management. An entity can minimize this type of violation by implementing adequate processes, procedures, and controls.

This noncompliance started on December 31, 2017, when the entity failed to implement or update the CAP prior to its scheduled completion date and ended on February 16, 2018, when the entity updated the CAP.

Risk Assessment This noncompliance posed a moderate risk and did not pose a serious or substantial risk to the reliability of the bulk power system based on the following factors. If an entity fails to implement or update a CAP regarding a protection system misoperation, then there is an increased likelihood of future misoperations of a similar nature. The risk was not minimal in this case because of the size of the generating unit (605 MVA) involved in the noncompliance. And, two separate relay protection schemes were impacted (i.e., a bus differential and a generator differential). The risk was not serious or substantial because the underlying issues were only present at a single location (i.e., this was not a fleet-wide issue). Although the entity did not implement the CAP before the initial target completion date, it took steps to investigate the misoperation, identify cause(s), and develop a plan to prevent recurrence in a timely manner, thus further reducing the risk in this case. No harm is known to have occurred.


1) performed St. Clair Power Plant operator refresher training on the use and purpose of shorting switches;2) reviewed the protective tagging restoration procedure for the St. Clair Power Plant Unit 7;3) reviewed protective tagging for all units at the St. Clair Power Plant;4) revised labeling on shorting switches for all units at the St. Clair Power Plant; and5) shared the results of the event review with the rest of the fleet. In addition, to address the root cause of this noncompliance, the entity improved its process for tracking and verifying the completion

of tasks identified in CAPs.






RFC2018020429 PRC-005-6 R3 Essential Power OPP, LLC (EPOPP) NCR00212 4/1/2017 5/28/2019 Compliance Audit July 31, 2019 Description of the Noncompliance (For purposes of this document, each noncompliance at issue is described as a “noncompliance,” regardless of its procedural posture and whether it was a possible, or confirmed noncompliance.)

On September 13, 2018, ReliabilityFirst determined that the entity, as a Generator Owner, was in noncompliance with PRC-005-6 R3. The noncompliance was identified during a Compliance Audit conducted from August 9, 2018 through September 13, 2018. Specifically, the entity failed to maintain/test the following components within required time periods: (a) unmonitored protective relays [Table 1-1]; and (b) control circuity [Table 1-5]. Pursuant to the implementation plan for PRC-005-6, thirty percent (30%) of the unmonitored protective relays should have been tested by April 1, 2017, but none were tested by that date. They were last tested in 2014 and were not scheduled to be tested again until 2020. Thirty percent (30%) of the control circuitry components also should have been tested by April 1, 2017, but none were tested by that date. They were last tested in 2013 and were not scheduled to be tested again until 2025. The root cause of this noncompliance was inadequate planning, which resulted in confusion regarding the implementation of changes relating to required maintenance and testing under PRC-005-6. This noncompliance implicates the management practice of planning, which includes the need to effectively identify and understand changing requirements and establish safeguards to avoid an unintentional adverse effect on Bulk Electric System reliability and resilience. This noncompliance started on April 1, 2017, which was the date on or before which certain testing should have been completed, and will end on May 28, 2019, after maintenance and testing is completed and in alignment with the PRC-005-6 implementation plan.

Risk Assessment

This noncompliance posed a moderate risk and did not pose a serious or substantial risk to the reliability of the bulk power system based on the following factors. Neglecting to maintain and test protection system, automatic reclosing, and sudden pressure relaying components could lead to device malfunction, premature or undetected device failure, or misoperation. Such issues could have significant consequences related to equipment damage and power system performance (e.g., generating or system instability, unacceptable loss of load or generation, cascading, or uncontrolled system separation). The risk was not minimal because the entity was unaware of the issue until it was identified during an audit, and based upon the existing maintenance schedule, the issue likely would have persisted for multiple years. However, the risk was not serious or substantial in this case because of the following mitigating factors. First, in the unlikely event that the components failed, only two units with a total generating output of 340 MW were at risk of being lost. Second, testing was performed on the subject relays and control circuitry in 2014 and 2013, respectively, and testing was scheduled to occur again in the future. Restated, the affected components were subject to an existing maintenance plan, which would have helped the entity identify any issues with the components and allowed for corrective actions to be performed. No harm is known to have occurred. In exercising its discretion to treat this issue as an FFT, ReliabilityFirst underscored to the entity the importance of staying abreast of maximum maintenance and testing intervals and new and updated standards and requirements. ReliabilityFirst considered the entity’s compliance history and determined there were no relevant instances of noncompliance.

Mitigation

To mitigate this noncompliance, the entity: 1) reviewed, and edited if required, the entity’s PRC-005 procedure; 2) conducted a review of its Protective System component attributes; 3) will complete all required testing to get the entity on track with the PRC-005-6 implementation plan, including testing sixty percent (60%) of the entity’s relays and control circuitry and thirty percent

(30%) of entity’s CT/PT’s, communication systems, and microprocessor relays on or before May 28, 2019; and 4) will develop a plan for completing the remainder of the testing prior to January 1, 2021. Then, the entity will continue maintenance and testing activities in accordance with the PRC-005-6

implementation plan and PRC-005-6. Mitigation is ongoing as the entity needs additional time to complete required maintenance and testing to get on track with the PRC-005-6 implementation plan.





FRCC2018020720 PRC-005-6 R3. Gainesville Regional Utilities (GRU) NCR00032 11/24/2017 09/26/2018 Self-Report Completed Description of the Noncompliance (For purposes of this document, each noncompliance at issue is described as a “noncompliance,” regardless of its procedural posture and whether it was a possible or confirmed noncompliance.)

On November 20, 2018, GRU submitted a Self-Report stating that, as a Distribution Provider, Generation Owner, and Transmission Owner, it was in violation of PRC-005-6 R3.

This violation started on November 24, 2017, when the Entity did not complete one (1) component of the 18-month battery maintenance activities required for one (1) Vented Lead-Acid (VLA) battery bank of twenty-four (24) banks, and ended on September 26, 2018, when all maintenance activities were complete.

During an internal review of battery maintenance records, it was discovered that the intercell/intracell testing for one (1) VLA battery bank had not been performed within the 18-month interval, as required by PRC-005-6 Table 1-4(a). The last recorded intercell/intracell test was conducted on May 23, 2016, making the next required test to be completed no later than November 23, 2017 (18 months). Prior to this discovery, the Entity had performed a complete test on September 26, 2018, or 28 months from the previous test (10 months beyond the maximum maintenance interval).

An extent of condition review was conducted verifying there were no additional occurrences.

The cause for this noncompliance was a lack of a uniform preventive maintenance (PM) schedule for the generating site batteries, which comprise eight (8) of the twenty-four (24) total PRC-005-6 batteries.

Risk Assessment This violation posed a minimal risk and did not pose a serious or substantial risk to the reliability of the bulk power system.

The risk is reduced because the Entity completed the monthly maintenance activities which included visual inspection of battery for cleanliness and corrosion, fluid level checks, measurement of cell specific gravities, electrolyte temperature, and terminal voltage. The Entity’s failure to take intercell/intracell readings on the battery could result in a lack of awareness of battery deterioration, which could lead to battery failure. A battery failure could result in a misoperation of a BES system device, a local service interruption, and/or loss of ability to provide peaking support of up to 72.5 MW.

Additionally, the Entity’s total generation of 630 MWs is 1.25% of the Region.

The Region determined that the Entity’s compliance history (FRCC200900166, FRCC200900174, FRCC201000402) should not serve as a basis for applying a penalty due to different facts and circumstances. No harm is known to have occurred. The instant noncompliance is a repeat of FRCC2011008750 resulting in FFT treatment; however, FRCC2011008750 should not serve as an aggravating factor in applying a penalty.

Mitigation To mitigate this violation, the Entity: 1) performed extent of condition review;2) conducted root cause analysis;3) established common battery PM schedules for all generation batteries;4) revised the Protection System Maintenance Program to reflect changes in generation battery PMs;5) created training materials; and6) trained all applicable personnel on changes





RFC2018018936 TOP-001-3 R9 Indianapolis Power & Light Company (IPL)



On December 20, 2017, the entity submitted a Self-Report stating that, as a Transmission Operator, it was in noncompliance with TOP-001-3 R9.

On September 21, 2017, the entity’s Real-Time Assessment (RTA) did not converge for a timeframe greater than 30 minutes. (Specifically, the RTA was not converging for 8 hours and 10 minutes on September 21, 2017) The entity examined the issue and learned that potential (post-contingency) operating conditions were the only item not being met by the entity regarding the RTAs being performed. Operators in the Transmission Operations Control Center (TOCC) were monitoring the entity transmission system during this time. However, the entity did not notify MISO or its impacted interconnected utilities. (MISO did not notify the entity of any issues resulting from the Real-Time Contingency Analysis during the period of time that the entity was having issues with its RTAs.) Furthermore, after identifying the noncompliance, ReliabilityFirst and the entity verified that MISO’s RTA and Contingency Analysis was fully functional on the date in question.

The root cause of this noncompliance was the fact that the alarms operators received for non-converging State Estimator solutions did not provide enough information to the operators to easily determine when the 30 minute threshold was reached. This major contributing factor involves the management practice of grid operations, which includes maintaining situational awareness of operations and validating operations tools.

This noncompliance started on September 21, 2017, when the entity’s operators failed to make the requisite notification to MISO and ended on January 4, 2018, when ReliabilityFirst confirmed that the entity had actually made the requisite notification to MISO.

Risk Assessment This noncompliance posed a moderate risk and did not pose a serious or substantial risk to the reliability of the bulk power system (BPS) based on the following factors. The risk involved in this noncompliance is a reduction in visibility of potential post-contingencies on the BPS. If these are not addressed and communicated properly, they could result in wider spread adverse impact on the BPS. This risk was mitigated in this case by the following factors. First, it was confirmed that MISO’s RTA and Contingency Analysis was fully functional and operational on the date in question. Therefore, if an issue had occurred, MISO would have been aware of it independently from the entity’s notification. Second, RTA is only one of many methods used to monitor the BPS and would not necessarily be the only indicator of a potential risk. For example, the entity’s TOCC Operators were constantly monitoring the transmission system during the time of non-convergence. Other examples include Supervisory Control and Data Acquisition displays and energy management system alarms. ReliabilityFirst also notes that both the entity and MISO observed no BPS issues during the time of non-convergence. No harm is known to have occurred.

ReliabilityFirst considered the entity's compliance history and determined there were no relevant instances of noncompliance.


1) created and tested the alarm functionality. The entity implemented a new alarm notification;2) created training based on the new alarm notification; and3) provided training to the Transmission Operations Control Center Operators.






RFC2017018693 VAR-002-4 R2 NRG Energy Services LLC - Morgantown (NRG Morgantown)



On November 12, 2017, the entity submitted a Self-Report stating that, as a Generator Operator, it was in noncompliance with VAR-002-4 R2. During an internal compliance review, the entity discovered several failures to notify PJM, the Transmission Operator (TOP), through First Energy (FE), when the 30 minute voltage exceeded the assigned high voltage schedule, during select dates in 2016 and 2017. In total, the entity identified a total of 87 instances of noncompliance where it failed to notify its TOP. However, the majority of the voltage exceedances were only slightly higher than the high voltage limit. In fact, the average exceedance was .21% above the threshold, with the single highest exceedance being 1.12% above the threshold. Furthermore, FE informed the entity that there is a difference of minus .3 kV voltage at the substation, which, when taken into account, would reduce the number of exceedances by almost half. Additionally, the entity discovered that voltage monitoring and control is maintained at the generator step-up (GSU) transformer output with no visibility at the Interconnection point in violation of VAR-002-4 R2.3. Consequently, the entity had no methodology to convert the voltage measurement at the GUS transformer output to the voltage measurement at the point of Interconnection.

The root cause of this noncompliance were (a) inadequate detective controls to notify operators to respond when nearing the voltage limit; and, (b) the fact that the entity did not have a methodology in place to convert the voltage at the GSU transformer output to the voltage level at the point of Interconnection.

This noncompliance started on January 31, 2016, when the first voltage exceedance occurred, and ended on September 7, 2017, when the last voltage exceedance ended.

Risk Assessment This noncompliance posed a moderate risk and did not pose a serious or substantial risk to the reliability of the bulk power system based on the following factors. The risk posed by a Generator Operator failing to inform the TOP of a voltage or reactive power exceedance is that it could cause the TOP to be uncertain of what generator was creating or contributing to an abnormal voltage condition on the BPS. This uncertainty could impede the TOP’s ability to take appropriate action. The risk posed by failing to accurately monitor and control the voltage at the point of Interconnection is that it could impede the generator’s ability to automatically regulate the voltage to maintain the proper voltage. These risks are not minimal in this case because of the number of exceedances (i.e., 87) and the length of time over which they occurred. However, these risks are not serious in this case based on the following factors. First, the generating unit does not have enough reactive capability, due to its size ﴾50 MWs﴿, to make a significant change in the voltage level at the point of Interconnection. Second, the majority of the voltage exceedances were found only slightly higher than the high voltage limit ﴾i.e., an average of 140.3 kV or .21% above the voltage threshold﴿ with the single largest exception at 141.580 which is 1.12 % above the scheduled maximum level of 140kV. No harm is known to have occurred.

ReliabilityFirst considered the entity’s compliance history and determined there were no relevant instances of noncompliance.


1) instituted voltage data report from the distributed control system (DCS) at this site for exception reporting to be used as Catsweb quarterly control evidence;2) changed entries in electronic log book to capture alarm limits and required response by operators;3) created an advanced alarm for high voltage schedule set to initiate at 139.5 with a high-high alarm at 140 KV to provide adequate notification for operators to respond to when nearing the voltage limit;4) created an advanced alarm for low voltage schedule set to initiate at 136.5 with a low-low alarm at 136 KV to provide adequate notification for operators to respond to when nearing the voltage limit;5) instituted an alarm response procedure to include the new alarm limits and response requirements for the operators when this occurs;6) monitored adherence to voltage schedule weekly until an automated or batch process of exception reported is developed;7) trained all control board operators and Operations supervision on updated procedure and NRG OCC-VAR-002 Compliance procedure;8) compared DCS voltage output with FE’s voltage readings at Interconnection to determine correlation in voltage readings over load range and determine feasibility of retrieving PJUM and FE real timevoltage profile for monitoring purposes; and 9) determined and implemented a means for definitive measurement or methodology of voltage monitoring at the Interconnection can be established.






SERC2018019780 VAR-002-2b R2 Carville Energy LLC (Carville) NCR11479 7/3/2014 05/07/2017 Self-Report Completed


On May 29, 2018, Carville submitted a Self-Report indicating that, as a Generator Operator, it was in noncompliance with VAR-002-3 R2. During an internal compliance review performed in August 2017, Carville determine that it had failed to meet the conditions of notification for deviations from the Voltage schedule provided by its TOP, Entergy Corporation.

Entergy’s Voltage Schedule Policy for Generating Facilities Interconnected to the Entergy Transmission System (Entergy Voltage Schedule) provides that during all times of the year, Carville is to maintain 232.3 kV with a tolerance band of +3 kV / -2 kV at its interconnection. Further Entergy’s Voltage Schedule provides:

Each plant should contact Entergy’s operations control center immediately (within 30 minutes) upon meeting both of the following conditions:

a. The discovery of a deviation from the prescribed schedule tolerance band.

b. The plant has exhausted all means of controlling voltage or reactive power.

Carville indicated that is has historically had difficulty maintaining the voltage schedule with all generators in automatic voltage control producing the respective reactive power during the summer months due to the increased system load demand during the summers. Further, Carville stated that until the internal compliance review, it believed it was in compliance with VAR-002-3 by following the terms of its January 2000 Interconnection and Operating Agreement (Operating Agreement) with Entergy. The Operating Agreement provides that “in the event that the voltage schedule is not being maintained, the Facility shall be operated (within the design limitations of the equipment in service at the time) to produce the maximum reactive power (MVAR) output available in an attempt to achieve the prescribed voltage schedule, provided that Entergy has requested other generators in the affective area (including but not limited to Entergy’s generators) to produce maximum reactive power (MVAR) in an attempt to achieve the prescribed voltage.”

Carville estimated that during the months of June, July and August for the years 2015, 2016 and 2017, it deviated from the voltage schedule for 201 hours or 9% of the time in 2015, 429 hours or 19% of the time in 2016, and 884 hours or 40% of the time in 2017. The maximum Voltage schedule deviation experienced during these times was + 1.8 kV (.76%) / - 7.7 kV (3.3%). Carville indicated that it was not always operating at its maximum reactive power capability during the aforementioned times and did not always notify its TOP of the deviation from the Voltage schedule.

The root cause of this noncompliance was Carville’s incorrect assumption that complying with the terms of its Operating Agreement with Entergy was sufficient to achieve compliance with VAR-002-2b R2.

This noncompliance started on July 3, 2014, the date of the first instance that Carville did not meet the conditions of notification for deviations from the Voltage schedule provided by its TOP, and ended on May 7, 2017, the last date Carville failed to meet the conditions of notification for deviations from the Voltage schedule provided by its TOP.

Risk Assessment This noncompliance posed a moderate risk and did not pose a serious or substantial risk to the reliability of the bulk power system (BPS). Specifically, Carville is a 555 MW, 2x1 combined cycle generating facility that is interconnected to the BPS at 230 kV. Additionally, the noncompliance occurred over an extended period of time (two years and seven months) and during the summer months, a time the BPS is heavily loaded. Nevertheless, Carville indicated that it is rarely called upon by the TOP for voltage support and has always responded by providing the maximum reactive power available. No harm is known to have occurred.

SERC determined that Carville's compliance history should not serve as a basis for applying a penalty. Carville has no relevant prior violations of VAR-002-3 R2 or any other standards that are similar in nature.

Mitigation To mitigate this noncompliance, Carville:

1) trained operations personnel on the requirements of VAR-002-4;2) requested and received a modification to the TOP’s voltage schedule, providing for a more sustainable voltage schedule;3) trained operations personnel and management on the new voltage schedule; and4) implemented an alarm system to notify plant operators when plant voltage deviates from the voltage schedule.



NERC Violation ID Reliability Standard Req. Entity Name NCR ID Violation Start Date Violation End Date Method of Discovery


SERC2016016658 PRC-023-1 R1 Georgia Power Company (GPC) NCR01247 7/1/2010 May 18, 2017 Self-Report Completed Description of the Violation (For purposes of this document, each violation at issue is described as a “violation,” regardless of its procedural posture and whether it was a possible, or confirmed violation.)

On December 15, 2016, GPC submitted a Self-Report stating that, as a Transmission Owner, it was in noncompliance with PRC-023-1 R1. On April 27, 2016, during quarterly monitoring of internal controls by GPC Transmission Compliance, a GPC Control Center Support Engineer discovered an error with a relay setting for the Fortson – Tenaska 500 kV transmission line. Following the replacement of a 500 kV switch in 2015, which resulted in an increase in the line rating for the Fortson – Tenaska transmission line, GPC failed to adjust the protective relay setting to greater than 150% of the transmission line Facility Rating. As a result of the line rating increase, the protective relay limited the loadability of the Fortson – Tenaska transmission line to 147% of its highest seasonal Facility Rating (R1.1).

Subsequent to the Self-Report submitted on December 15, 2016, GPC identified two additional instances of noncompliance with PRC-023-1 R1. In the first instance, as part of its mitigation plan for the December 15, 2016 self-reported instance, GPC reviewed all of its protective relays to ensure the protection relays were set above 150% of the highest seasonal Facility Rating of the associated transmission facility. The protective relays for the Big Shanty Bank A, Big Shanty Bank B, and Bowen Bank 10 autobank transformer facilities were set to limit the loadability of the autobank facilities to 149.85%, 149.85% and 149.91% of their respective highest seasonal Facility Rating (R1.11).

In the second instance, on March 23, 2017, GPC identified a disparity between the Facility Rating information in its PRC-023 spreadsheet and the GPC Operations’ Facility Ratings. Upon correcting the Facility Rating disparity, GPC discovered the protective relay for the McIntosh CC 1 - West McIntosh 230 kV transmission line limited the loadability of the transmission line to 140.4 % of its highest seasonal Facility Rating (R1.1).

The root cause of this noncompliance was lack of a formal process for changing Facility Ratings and lack of managerial oversight. By not having a formal process for changing Facility Ratings, GPC utilized inexact hand calculations for the Low Side Back-up Over-Current relays in the autobank spreadsheet to calculate relay settings, which resulted in Facility Rating errors. Proper managerial oversight should have identified the undocumented process for changing Facility Ratings.

This noncompliance started on July 1, 2010, the date PRC-023-1 became mandatory and enforceable and GPC failed to ensure the relay settings were set to limit the loadability, and ended on May 18, 2017, the date GPC corrected the relay settings for the relays that were not set to limit the loadability of the respective transmission facilities to greater than 150% of the highest seasonal Facility Rating.

Risk Assessment This noncomplaince posed a moderate risk and did not pose a serious or substantial risk to the reliability of the bulk power system (BPS). GPC’s failure to set protective relays so they did not operate at or below 150% of the highest seasonal Facility Rating of a transmission facility increased the risk that relays would unnecessary trip transmission facilities during system events that otherwise would not have caused a transmission facility outage. Notwithstanding, SERC determined that the risk to the BPS was mitigated because the incorrect 500 kV line relay settings would support at least 147% of the highest seasonal Facility Rating of the circuit. GPC operates its transmission system to withstand the loss of any single transmission facility without adversely affecting the continued operation of its transmission system The actual load on the 500 kV line during the period of non-compliance did not exceed 1084 amps, which is well below the original setting of 2,675 amps. Additionally, the improperly set protective relay errors were small – the largest error occurred for the McIntosh CC 1 – West McIntosh 230 kV transmission line where loadability of the transmission line was limited to 140.4 % of its highest seasonal rating. No harm is known to have occurred due to the incorrect relay settings.

A Spreadsheet Notice of Penalty covering violation of PRC-023-1 R1 (SERC2011007157) for GPC was filed with FERC under NP12-27-000 on May 30, 2012. On June 29, 2012, FERC issued an order stating it would not engage in further review of the Notice of Penalty. GPC identified seven protection relays that were not set to operate above 150% of the highest seasonal Facility Rating of the associated transmission facilities.

SERC determined that GPC’s compliance history should not serve as a basis for applying a penalty. The current issue does not involve recurring conduct that was the same or similar to the conduct in the prior noncompliance. Whereas the prior violation resulted from omissions and a failure to follow approved change procedures, the instant violation is the result of incorrect relay settings due to mathematical errors. SERC determined that the repeat violation of PRC-023-1 does not represent a systemic problem with GPC’s internal compliance program but is illustrative of lack of a formalized process and managerial oversight.

Mitigation To mitigate this noncompliance, GPC:

1) changed the incorrect protective relay settings to comply with PRC-023-1;2) implemented a formal process for changing Facility Ratings for GPC Operations;3) created an informational user interface module in the GPC lines rating database to identify transmission line Facilities with pending changes in Ratings;4) incorporated the calculations for Low Side Back-Up Over-Current (LSBUOC) load responsive relays into the autobank spreadsheet used to calculate relay settings;5) removed the McIntosh circuit from service;6) trained personnel in the use of the formal process for changing Facility Ratings for GPC Operations; and



NERC Violation ID Reliability Standard Req. Entity Name NCR ID Violation Start Date Violation End Date Method of Discovery


SERC2016016658 PRC-023-1 R1 Georgia Power Company (GPC) NCR01247 7/1/2010 May 18, 2017 Self-Report Completed 7) completed a 100% review of PRC-023 relays.





SERC2017018600 TOP-001-3 R13 Duke Energy Progress, LLC (DEP) NCR01298 07/21/2017 07/21/2017 Self-Report Completed


On November 7, 2017, Duke Energy Progress, LLC (DEP) submitted a Self-Report on behalf of Duke Energy Florida, LLC (DEF) stating that, as a Transmission Operator, DEF was noncompliant with TOP-001-3 R13. DEF did not perform a Real-time Assessment at least once every 30 minutes. DEP submitted the Self-Report under an existing Multi-regional Registered Entity Agreement.

On July 21, 2017, the DEF Energy Control Center (ECC) performed a Backup Control Center (BUCC) functionality test. At 6:10 a.m., prior to initiating transfer of operations to the BUCC, DEF completed a Real-time Contingency Analysis (RTCA) assessment, i.e., Real-time Assessment. No contingencies were identified on DEF’s transmission system by the RTCA assessment. Thereafter, DEF Energy Management System (EMS) Support Engineering initiated the EMS function at the primary ECC server to failover to the BUCC server.

At 6:19 a.m., an RTCA assessment utilizing the BUCC EMS identified an unsupported number of contingencies. Notwithstanding the perceived problem with the BUCC EMS RTCA results, System Operators determined the DEF BUCC EMS was operational and providing valid real-time flows, generator output, frequency, and other EMS data. DEF System Operators subsequently determined that although the RTCA was solving, the results of the RTCA were not valid.

At 6:47 a.m., System Operators attempted a manual RTCA assessment per DEF Standing Order TS115 using a parallel Study Contingency Analysis (STCA) program. System Operators determined the STCA assessment was also invalid. At 7:25 a.m., System Operators contacted EMS Support Engineering who began troubleshooting the RTCA application problem at 7:41 a.m. At 8:14 a.m., DEF completed a second STCA assessment utilizing modified 5:45 a.m. EMS data. Although, the STCA assessment did not utilize the most current EMS data, no contingencies were identified on DEF’s transmission system.

At 8:28 a.m., DEF informed its Reliability Coordinator (RC), Florida Reliability Coordinating Council that the DEF RTCA application was providing invalid solutions. The RC agreed to monitor its Real-time assessments to identify DEF contingencies until the DEF RTCA issue was resolved.

At 8:30 a.m., DEF performed a third STCA utilizing modified EMS data from 8:14 a.m. No contingencies were identified on DEF’s transmission system.

At 8:45 a.m., DEF EMS Support Engineering restored normal operation of the BUCC RTCA application. It was determined that the RTCA application failure occurred because the EMS database at the BUCC was updated with incorrect generating unit data during the failover of the primary ECC server to the BUCC server.

DEF’s failure to ensure the performance of a Real-time Assessment within 30 minutes was attributed to: failure to stop and restart all process on all EMS servers following an EMS database update to ensure a relink of generating unit indexes in the EMS Transfer Manager; inadequate controls to verify that a Real-time Assessment occurred every 30 minutes; the RTCA application failure occurring concurrent with the BUCC functionality test; System Operator failure to implement protocols to mitigate the loss of the RTCA function; System Operator involvement in the investigation of the cause of the RTCA application failure; and System Operator confusion regarding the continued operation of the BUCC EMS, i.e., notwithstanding the erroneous contingencies generated by the RTCA application, the BUCC EMS continued to provide valid real-time flows, generator output, frequency, and other EMS data.

This noncompliance started on July 21, 2017, at 6:40 a.m., the time by which DEF should have performed a Real-time Assessment following the, successful 6:10 a.m. Real-time Assessment, and ended on July 21, 2017, at 8:28 a.m., when the DEF RC began monitoring its Real-time Assessment for DEF contingences.



Risk Assessment This noncompliance posed a moderate risk to the reliability of the bulk power system (BPS). The failure to ensure the performance of a Real-time Assessment of the transmission system every 30 minutes increases the risk that System Operators could be unaware of system conditions that would impact the reliability of the BPS. This lack of system awareness increases the risk that System Operators would not proactively mitigate system conditions that could result in instability, uncontrolled separation, or cascading outages. Here, DEF failed to perform a Real-time Assessment of its transmission system for one hour and forty-eight minutes. Additionally, DEF failed to timely report its inability to perform a Real-time Assessment to its RC and seek the RC’s assistance in monitoring for contingencies on the DEF transmission system. SERC determined that this issue is appropriate for FFT disposition because: the failure to perform a Real-time Assessment occurred during morning hours when system conditions typically change, but change in a predictable manner; DEF continued to have Real-time EMS data available to monitor system status; the RC’s Real-time Assessment capabilities were unaffected by the DEF RTCA failure; DEF employed the STCA process to provide near Real-time assessments; and neither DEF nor its RC identified pre-contingency or post-contingency conditions that required mitigating actions during the noncompliance. No harm is known to have occurred.

SERC considered DEF’s compliance history and determined that there were no relevant instances of noncompliance. Mitigation To mitigate this noncompliance, DEF:

1) ECC system operators implemented DEF Standing Order TS115 to run manual real-time assessments using STCA;2) EMS support engineering troubleshot and then restarted the transfer manager application to relink the real-time EMS applications;3) updated DEF Standing Order TS115 to include:a. actions to be taken when RTCA solves but results are invalid;b. notifying the FRCC RC; andc. pulling the last valid real-time savecase into EMS applications;4) EMS support engineering updated failover procedures to include:a. pre-job brief with system operations prior to commencing;b. critical steps to notify system operations prior to performing;c. process to stop/start all servers after all database updates;d. EMS support engineering will be in a fixed location prior to any predetermined failovers; ande. opening a bridge line to be used for constant communication during failovers between system operations and EMS support engineering;5) EMS support engineering provided a copy of the revised failover procedures to system operations;6) updated EMS checklist for system operations to include RTCA contingency list after the failover coincides with the RTCA contingency list prior to the failover; and7) provided system operations training on the following:a. the new failover procedures and EMS checklist;b. hands on demonstration using the DTS creating a save case and performing a manual real-time assessment; andc. communications protocols.



nerc id name ncr id actions... · req. entity name ncr id noncompliance start date noncompliance...

Documents