us canada power system outgate task force

8/3/2019 US Canada Power System Outgate Task Force

1/30

U.S.-Canada Power System Outage Task Force

Final Report on the August 14, 2003 Blackout

in theUnited States and Canada:

Causes and

Recommendations

April 2004


2/30


3/30


Final Report on the August 14, 2003 Blackout

in theUnited States and Canada:

Causes andRecommendations

April 2004


4/30

iiG

U.S.-Canada Power System Outage Task ForceG

August 14th Blackout: Causes and RecommendationsG


March 31, 2004

Dear Mr. President and Prime Minister:

We are pleased to submit the Final Report of the U.S.-Canada Power System Outage

Task Force. As directed by you, the Task Force has completed a thorough investigation

of the causes of the August 14, 2003 blackout and has recommended actions to minimizethe likelihood and scope of similar events in the future.

The report makes clear that this blackout could have been prevented and that immediate

actions must be taken in both the United States and Canada to ensure that our electric

system is more reliable. First and foremost, compliance with reliability rules must be

made mandatory with substantial penalties for non-compliance.

We expect continued collaboration between our two countries to implement this reports

recommendations. Failure to implement the recommendations would threaten the

reliability of the electricity supply that is critical to the economic, energy and national

security of our countries.

The work of the Task Force has been an outstanding example of close and effective

cooperation between the U.S. and Canadian governments. Such work will continue as we

strive to implement the Final Reports recommendations. We resolve to work in

cooperation with Congress, Parliament, states, provinces and stakeholders to ensure that

North Americas electric grid is robust and reliable.

We would like to specifically thank the members of the Task Force and its Working

Groups for their efforts and support as we investigated the blackout and moved toward

completion of the Final Report. All involved have made valuable contributions. We

submit this report with optimism that its recommendations will result in better electric

service for the people of both our nations.

Sincerely,

U.S. Secretary of Energy Minister of Natural Resources Canada


5/30

ContentsPage

G



iii

1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

2. Overview of the North American Electric Power System and Its Reliability Organizations . . . . . . . 5

The North American Power Grid Is One Large, Interconnected Machine . . . . . . . . . . . . . . . . . . . . 5Planning and Reliable Operation of the Power Grid Are Technically Demanding . . . . . . . . . . . . . . 6Reliability Organizations Oversee Grid Reliability in North America . . . . . . . . . . . . . . . . . . . . . . . 10Key Parties in the Pre-Cascade Phase of the August 14 Blackout . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

3. Causes of the Blackout and Violations of NERC Standards. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17The Causes of the Blackout in Ohio . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17Linking Causes to Specific Weaknesses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17Institutional Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

4. Context and Preconditions for the Blackout: The Northeastern Power GridBefore the Blackout Began . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

Electric Demands on August 14 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25Generation Facilities Unavailable on August 14. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25Unanticipated Outages of Transmission and Generation on August 14. . . . . . . . . . . . . . . . . . . . . . 27Key Parameters for the Cleveland-Akron Area at 15:05 EDT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27Power Flow Patterns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28Voltages and Voltage Criteria . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31Past System Events and Adequacy of System Studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37Model-Based Analysis of the State of the Regional Power System at 15:05 EDT,Before the Loss of FEs Harding-Chamberlin 345-kV Line . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41System Frequency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

5. How and Why the Blackout Began in Ohio . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45Chapter Organization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45Phase 1: A Normal Afternoon Degrades: 12:15 EDT to 14:14 EDT . . . . . . . . . . . . . . . . . . . . . . . . . . 45Phase 2: FEs Computer Failures: 14:14 EDT to 15:59 EDT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51Phase 3: Three FE 345-kV Transmission Line Failures and Many Phone Calls:15:05 EDT to 15:57 EDT. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57Phase 4: 138-kV Transmission System Collapse in Northern Ohio: 15:39 to 16:08 EDT . . . . . . . . . 68

6. The Cascade Stage of the Blackout. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73Phase 5: 345-kV Transmission System Cascade in Northern Ohio and South-Central Michigan . . 77Phase 6: The Full Cascade . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82Phase 7: Several Electrical Islands Formed in Northeast U.S. and Canada: 16:10:46 EDTto 16:12 EDT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90

Why the Blackout Stopped Where It Did . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91Under-Frequency and Under-Voltage Load-Shedding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92Why the Generators Tripped Off . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93

7. The August 14 Blackout Compared With Previous Major North American Outages . . . . . . . . . . . . . 103Incidence and Characteristics of Power System Outages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103Outage Descriptions and Major Causal Factors. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104Common or Similar Factors Among Major Outages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107Comparisons With the August 14 Blackout . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110


6/30

Contents (Continued) Page

Appendixes

Tables

Figures

ivG



A. Members of the U.S.-Canada Power System Outage Task Force and Its Three Working Groups . . . 175B. Description of Outage Investigation and Process for Development of Recommendations . . . . . . . . 179C. List of Commenters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189D. NERC Actions to Prevent and Mitigate the Impacts of Future Cascading Blackouts . . . . . . . . . . . . . 193E. List of Electricity Acronyms. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211F. Electricity Glossary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213G. Transmittal Letters from the Three Working Groups. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221

8. Performance of Nuclear Power Plants Affected by the Blackout. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111Findings of the U.S. Nuclear Working Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112Findings of the Canadian Nuclear Working Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121Perspective of Nuclear Regulatory Agencies on Potential Changes to the Grid . . . . . . . . . . . . . . . . 129

9. Physical and Cyber Security Aspects of the Blackout . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131

Summary and Primary Findings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131SWG Mandate and Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132Cyber Security in the Electricity Sector . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132Information Collection and Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133Cyber Timeline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136

10. Recommendations to Prevent or Minimize the Scope of Future Blackouts . . . . . . . . . . . . . . . . . . . 139Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139Recommendations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140

4.1. Generators Not Available on August 14. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

4.2. Benchmarking Model Results to Actual . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 284.3. Comparison of Voltage Criteria (Percent) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

7.1. Changing Conditions That Affect System Reliability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104

8.1. U.S. Nuclear Plant Trip Times . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1168.2. Summary of Events for U. S. Nuclear Power Plants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1208.3. Summary of Shutdown Events for Canadian Nuclear Power Plants . . . . . . . . . . . . . . . . . . . . . . . . 128

2.1. Basic Structure of the Electric System. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52.2. North American Interconnections. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62.3. PJM Load Curve, August 18-24, 2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72.4. Normal and Abnormal Frequency Ranges. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72.5. NERC Regions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112.6. NERC Regions and Control Areas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122.7. NERC Reliability Coordinators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132.8. Reliability Coordinators and Control Areas in Ohio and Surrounding States. . . . . . . . . . . . . . . . . 13

4.1. August 2003 Temperatures in the U.S. Northeast and Eastern Canada. . . . . . . . . . . . . . . . . . . . . . 254.2. Load Forecasts Below Actuals, August 11 through 14. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 254.3. MW and MVAr Output from Eastlake Unit 5 on August 14. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 274.4. Generation, Demand, and Interregional Power Flows on August 14, 2003, at 15:05 EDT. . . . . . . . 29


7/30

Figures (Continued) Page

G



v

4.5. Scheduled Imports and Exports for the Northeast Central Region, June 1 throughAugust 13, 2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

4.6. Impacts of Transactions Flows on Critical Line Loadings, August 14, 2003 . . . . . . . . . . . . . . . . . . . 304.7. Actual Voltages Across the Ohio Area Before and On August 14, 2003. . . . . . . . . . . . . . . . . . . . . . . 344.8. Reactive Reserves Around Ohio on August 14, 2003, for Representative Generators in the Area. . . 374.9. Loss of the Perry Unit Hurts Critical Voltages and Reactive Reserves: V-Q Analyses . . . . . . . . . . . . 42

4.10. Impact of Perry Unit Outage on Cleveland-Akron Area Voltage Stability. . . . . . . . . . . . . . . . . . . . . 434.11. Frequency on August 14, 2003, up to 16:09 EDT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 434.12. Hourly Deviations in Eastern Interconnection Frequency for the Month of August 2003 . . . . . . . . 44

5.1. Timeline: Start of the Blackout in Ohio . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 465.2. Timeline Phase 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 485.3. Eastlake Unit 5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 495.4. Timeline Phase 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 525.5. FirstEnergy 345-kV Line Flows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 575.6. Voltages on FirstEnergys 345-kV Lines: Impacts of Line Trips . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 575.7. Timeline Phase 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 585.8. Harding-Chamberlin 345-kV Line . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 605.9. Hanna-Juniper 345-kV Line . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

5.10. Cause of the Hanna-Juniper Line Loss . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 625.11. Star-South Canton 345-kV Line . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 635.12. Cumulative Effects of Sequential Outages on Remaining 345-kV Lines . . . . . . . . . . . . . . . . . . . . . . 645.13. Timeline Phase 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 685.14. Voltages on FirstEnergys 138-kV Lines: Impact of Line Trips. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 685.15. Simulated Effect of Prior Outages on 138-kV Line Loadings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70

6.1. Rate of Line and Generator Trips During the Cascade . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 746.2. Area Affected by the Blackout . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 756.3. Sammis-Star 345-kV Line Trip, 16:05:57 EDT. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 776.4. Sammis-Star 345-kV Line Trip . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 786.5. Line Flows Into Michigan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78

6.6. Ohio 345-kV Lines Trip, 16:08:59 to 16:09:07 EDT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 796.7. New York-Ontario Line Flows at Niagara . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 796.8. First Power Swing Has Varying Impacts Across the Grid . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 806.9. Map of Zone 3 (and Zone 2s Operating Like Zone 3s) Relay Operations on August 14, 2003 . . . . . . 81

6.10. Michigan and Ohio Power Plants Trip. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 826.11. Transmission and Generation Trips in Michigan, 16:10:36 to 16:10:37 EDT . . . . . . . . . . . . . . . . . . 836.12. Flows on Keith-Waterman 230-kV Ontario-Michigan Tie Line. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 836.13. Simulated 345-kV Line Loadings from 16:05:57 through 16:10:38.4 EDT . . . . . . . . . . . . . . . . . . . . 846.14. Simulated Regional Interface Loadings from 16:05:57 through 16:10:38.4 EDT. . . . . . . . . . . . . . . . 846.15. Michigan Lines Trip and Ohio Separates from Pennsylvania, 16:10:36 to 16:10:38.6 EDT . . . . . . . 856.16. Active and Reactive Power and Voltage from Ontario into Detroit . . . . . . . . . . . . . . . . . . . . . . . . . . 866.17. Measured Power Flows and Frequency Across Regional Interfaces, 16:10:30 to 16:11:00 EDT,

with Key Events in the Cascade . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 866.18. Cleveland and Toledo Islanded, 16:10:39 to 16:10:46 EDT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 886.19. Generators Under Stress in Detroit, as Seen from Keith PSDR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 886.20. Western Pennsylvania Separates from New York, 16:10:39 EDT to 16:10:44 EDT . . . . . . . . . . . . . . 896.21. Northeast Separates from Eastern Interconnection, 16:10:45 EDT . . . . . . . . . . . . . . . . . . . . . . . . . . 896.22. PJM to New York Interties Disconnect. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 906.23. New York and New England Separate, Multiple Islands Form . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 916.24. Generator Trips by Time and Cause. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 956.25. Events at One Large Generator During the Cascade . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96


8/30

Figures (Continued) Page

viG



6.26. Measured Power Flows and Frequency Across Regional Interfaces, 16:10:45 to 16:11:30 EDT,with Key Events in the Cascade . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97

6.27. Frequency Separation Between Ontario and Western New York . . . . . . . . . . . . . . . . . . . . . . . . . . 996.28. Electric Islands Reflected in Frequency Plot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1006.29. Area Affected by the Blackout . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1006.30. Cascade Sequence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101

7.1. North American Power System Outages, 1984-1997. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103


9/30

1. Introduction

On August 14, 2003, large portions of the Midwestand Northeast United States and Ontario, Canada,experienced an electric power blackout. The out-age affected an area with an estimated 50 millionpeople and 61,800 megawatts (MW) of electricload in the states of Ohio, Michigan, Pennsylva-nia, New York, Vermont, Massachusetts, Connect-icut, New Jersey and the Canadian province ofOntario. The blackout began a few minutes after4:00 pm Eastern Daylight Time (16:00 EDT), andpower was not restored for 4 days in some parts ofthe United States. Parts of Ontario suffered rolling

blackouts for more than a week before full powerwas restored. Estimates of total costs in the UnitedStates range between $4 billion and $10 billion(U.S. dollars).1 In Canada, gross domestic productwas down 0.7% in August, there was a net loss of18.9 million work hours, and manufacturing ship-ments in Ontario were down $2.3 billion (Cana-dian dollars).2

On August 15, President George W. Bush andthen-Prime Minister Jean Chrtien directed that ajoint U.S.-Canada Power System Outage Task

Force be established to investigate the causes ofthe blackout and ways to reduce the possibility offuture outages. They named U.S. Secretary ofEnergy Spencer Abraham and Herb Dhaliwal,Minister of Natural Resources, Canada, to chairthe joint Task Force. (Mr. Dhaliwal was later suc-ceeded by Mr. John Efford as Minister of NaturalResources and as co-chair of the Task Force.)Three other U.S. representatives and three otherCanadian representatives were named to theTask Force. The U.S. members were Tom Ridge,Secretary of Homeland Security; Pat Wood III,Chairman of the Federal Energy Regulatory Com-mission; and Nils Diaz, Chairman of the NuclearRegulatory Commission. The Canadian memberswere Deputy Prime Minister John Manley, latersucceeded by Deputy Prime Minister AnneMcLellan; Kenneth Vollman, Chairman of theNational Energy Board; and Linda J. Keen, Presi-dent and CEO of the Canadian Nuclear SafetyCommission.

The Task Force divided its work into two phases:

u Phase I: Investigate the outage to determine itscauses and why it was not contained.

u Phase II: Develop recommendations to reducethe possibility of future outages and reduce thescope of any that occur.

The Task Force created three Working Groups toassist in both phases of its workan Electric Sys-tem Working Group (ESWG), a Nuclear WorkingGroup (NWG), and a Security Working Group

(SWG). The Working Groups were made up ofstate and provincial representatives, federalemployees, and contractors working for the U.S.and Canadian government agencies representedon the Task Force.

The Task Force published an Interim Report onNovember 19, 2003, summarizing the facts thatthe bi-national investigation found regarding thecauses of the blackout on August 14, 2003. AfterNovember 19, the Task Forces technical investi-gation teams pursued certain analyses that werenot complete in time for publication in the Interim

Report. The Working Groups focused on the draft-ing of recommendations for the consideration ofthe Task Force to prevent future blackouts andreduce the scope of any that nonetheless occur. Indrafting these recommendations, the WorkingGroups drew substantially on information andinsights from the investigation teams additionalanalyses, and on inputs received at three publicmeetings (in Cleveland, New York City, andToronto) and two technical conferences (in Phila-delphia and Toronto). They also drew on com-ments filed electronically by interested parties on

websites established for this purpose by theU.S. Department of Energy and Natural ResourcesCanada.

Although this Final Report presents some newinformation about the events and circumstancesbefore the start of the blackout and additionaldetail concerning the cascade stage of the black-out, none of the comments received or additional

G



1


10/30

analyses performed by the Task Forces investiga-tors have changed the validity of the conclusionspublished in the Interim Report. This report,however, presents findings concerning additionalviolations of reliability requirements and institu-tional and performance deficiencies beyond thoseidentified in the Interim Report.

The organization of this Final Report is similar tothat of the Interim Report, and it is intended toupdate and supersede the Interim Report. It isdivided into ten chapters, including this introduc-tory chapter:

u Chapter 2 provides an overview of the institu-tional framework for maintaining and ensuringthe reliability of the bulk power system in NorthAmerica, with particular attention to the rolesand responsibilities of several types of reliabil-ity-related organizations.

u

Chapter 3 identifies the causes of the blackoutand identifies failures to perform effectivelyrelative to the reliability policies, guidelines,and standards of the North American ElectricReliability Council (NERC) and, in some cases,deficiencies in the standards themselves.

u Chapter 4 discusses conditions on the regionalpower system on and before August 14 andidentifies conditions and failures that did anddid not contribute to the blackout.

u Chapter 5 describes the afternoon of August 14,starting from normal operating conditions, then

going into a period of abnormal but still poten-tially manageable conditions, and finally intoan uncontrollable blackout in northern Ohio.

u Chapter 6 provides details on the cascade phaseof the blackout as it spread in Ohio and thenacross the Northeast, and explains why the sys-tem performed as it did.

u Chapter 7 compares the August 14, 2003, black-out with previous major North American poweroutages.

u Chapter 8 examines the performance of thenuclear power plants affected by the August 14outage.

u Chapter 9 addresses issues related to physicaland cyber security associated with the outage.

u Chapter 10 presents the Task Forces recom-mendations for preventing future blackouts andreducing the scope of any that occur.

Chapter 10 includes a total of 46 recommenda-tions, but the single most important of them is thatthe U.S. Congress should enact the reliability pro-visions in H.R. 6 and S. 2095 to make compliance

with reliability standards mandatory and enforce-able. If that could be done, many of the other rec-ommended actions could be accomplished readilyin the course of implementing the legislation. Anoverview of the recommendations (by titles only)is provided on pages 3 and 4.

Chapter 2 is very little changed from the versionpublished in the Interim Report. Chapter 3 is newto this Final Report. Chapters 4, 5, and 6 have beenrevised and expanded from the correspondingchapters (3, 4, and 5) of the Interim Report. Chap-ters 7, 8, and 9 are only slightly changed from

Chapters 6, 7, and 8 of the Interim Report. TheInterim Report had no counterpart to Chapter 10.

This report also includes seven appendixes:

u Appendix A lists the members of the Task Forceand the three working groups.

u Appendix B describes the Task Forces investi-gative process for developing the Task Forcesrecommendations.

u Appendix C lists the parties who either com-mented on the Interim Report, provided sugges-

tions for recommendations, or both.

u Appendix D reproduces a document released onFebruary 10, 2004 by NERC, describing itsactions to prevent and mitigate the impacts offuture cascading blackouts.

u Appendix E is a list of electricity acronyms.

u Appendix F provides a glossary of electricityterms.

u Appendix G contains transmittal letters perti-nent to this report from the three Working

Groups.

2G




11/30

G



3

Overview of Task Force Recommendations: Titles Only

Group I. Institutional Issues Related to Reliability

1. Make reliability standards mandatory and enforceable, with penalties for noncompliance.

2. Develop a regulator-approved funding mechanism for NERC and the regional reliability councils,to ensure their independence from the parties they oversee.

3. Strengthen the institutional framework for reliability management in North America.

4. Clarify that prudent expenditures and investments for bulk system reliability (including invest-ments in new technologies) will be recoverable through transmission rates.5. Track implementation of recommended actions to improve reliability.6. FERC should not approve the operation of new RTOs or ISOs until they have met minimum

functional requirements.7. Require any entity operating as part of the bulk power system to be a member of a regional reli-

ability council if it operates within the councils footprint.8. Shield operators who initiate load shedding pursuant to approved guidelines from liability or

retaliation.9. Integrate a reliability impact consideration into the regulatory decision-making process.

10. Establish an independent source of reliability performance information.11. Establish requirements for collection and reporting of data needed for post-blackout analyses.

12. Commission an independent study of the relationships among industry restructuring, competi-tion, and reliability.

13. DOE should expand its research programs on reliability-related tools and technologies.14. Establish a standing framework for the conduct of future blackout and disturbance

investigations.

Group II. Support and Strengthen NERCs Actions of February 10, 2004

15. Correct the direct causes of the August 14, 2003 blackout.

16. Establish enforceable standards for maintenance of electrical clearances in right-of-way areas.17. Strengthen the NERC Compliance Enforcement Program.

18. Support and strengthen NERCs Reliability Readiness Audit Program.19. Improve near-term and long-term training and certification requirements for operators, reliability

coordinators, and operator support staff.20. Establish clear definitions for normal, alertand emergencyoperational system conditions. Clarify

roles, responsibilities, and authorities of reliability coordinators and control areas under eachcondition.

21. Make more effective and wider use of system protection measures.

22. Evaluate and adopt better real-time tools for operators and reliability coordinators.23. Strengthen reactive power and voltage control practices in all NERC regions.

24. Improve quality of system modeling data and data exchange practices.25. NERC should reevaluate its existing reliability standards development process and accelerate the

adoption of enforceable standards.26. Tighten communications protocols, especially for communications during alerts and emergen-

cies. Upgrade communication system hardware where appropriate.

27. Develop enforceable standards for transmission line ratings.28. Require use of time-synchronized data recorders.29. Evaluate and disseminate lessons learned during system restoration.

30. Clarify criteria for identification of operationally critical facilities, and improve dissemination ofupdated information on unplanned outages.

31. Clarify that the transmission loading relief (TLR) process should not be used in situations involv-ing an actual violation of an Operating Security Limit. Streamline the TLR process.

(continued on page 142)


12/30

Endnotes

4G



1 See The Economic Impacts of the August 2003 Blackout,Electric Consumer Research Council (ELCON), February 2,2004.

2 Statistics Canada, Gross Domestic Product by Industry,August 2003, Catalogue No. 15-001; September 2003 LabourForce Survey; Monthly Survey of Manufacturing, August 2003,Catalogue No. 31-001.

Overview of Task Force Recommendations: Titles Only (Continued)

Group III. Physical and Cyber Security of North American Bulk Power Systems

32. Implement NERC IT standards.

33. Develop and deploy IT management procedures.34. Develop corporate-level IT security governance and strategies.

35. Implement controls to manage system health, network monitoring, and incident management.

36. Initiate U.S.-Canada risk management study.37. Improve IT forensic and diagnostic capabilities.38. Assess IT risk and vulnerability at scheduled intervals.

39. Develop capability to detect wireless and remote wireline intrusion and surveillance.40. Control access to operationally sensitive equipment.

41. NERC should provide guidance on employee background checks.42. Confirm NERC ES-ISAC as the central point for sharing security information and analysis.

43. Establish clear authority for physical and cyber security.44. Develop procedures to prevent or mitigate inappropriate disclosure of information.

Group IV. Canadian Nuclear Power Sector

45. The Task Force recommends that the Canadian Nuclear Safety Commission request OntarioPower Generation and Bruce Power to review operating procedures and operator training associ-ated with the use of adjuster rods.

46. The Task Force recommends that the Canadian Nuclear Safety Commission purchase and installbackup generation equipment.


13/30

2. Overview of the North American Electric Power

System and Its Reliability Organizations

The North American Power GridIs One Large, InterconnectedMachine

The North American electricity system is one ofthe great engineering achievements of the past 100years. This electricity infrastructure representsmore than $1 trillion (U.S.) in asset value, more

than 200,000 milesor 320,000 kilometers (km)of transmission lines operating at 230,000 voltsand greater, 950,000 megawatts of generatingcapability, and nearly 3,500 utility organizationsserving well over 100 million customers and 283million people.

Modern society has come to depend on reliableelectricity as an essential resource for nationalsecurity; health and welfare; communications;finance; transportation; food and water supply;heating, cooling, and lighting; computers andelectronics; commercial enterprise; and even

entertainment and leisurein short, nearly allaspects of modern life. Customers have grown toexpect that electricity will almost always be avail-able when needed at the flick of a switch. Mostcustomers have also experienced local outagescaused by a car hitting a power pole, a construc-tion crew accidentally damaging a cable, or a

lightning storm. What is not expected is the occur-rence of a massive outage on a calm, warm day.Widespread electrical outages, such as the onethat occurred on August 14, 2003, are rare, butthey can happen if multiple reliability safeguardsbreak down.

Providing reliable electricity is an enormouslycomplex technical challenge, even on the mostroutine of days. It involves real-time assessment,

control and coordination of electricity productionat thousands of generators, moving electricityacross an interconnected network of transmissionlines, and ultimately delivering the electricity tomillions of customers by means of a distributionnetwork.

As shown in Figure 2.1, electricity is produced atlower voltages (10,000 to 25,000 volts) at genera-tors from various fuel sources, such as nuclear,coal, oil, natural gas, hydro power, geothermal,photovoltaic, etc. Some generators are owned by

the same electric utilities that serve the end-usecustomer; some are owned by independent powerproducers (IPPs); and others are owned by cus-tomers themselvesparticularly large industrialcustomers.

Electricity from generators is stepped up tohigher voltages for transportation in bulk over

G



5

Figure 2.1. Basic Structure of the Electric System


14/30

transmission lines. Operating the transmissionlines at high voltage (i.e., 230,000 to 765,000 volts)reduces the losses of electricity from conductorheating and allows power to be shipped economi-cally over long distances. Transmission lines areinterconnected at switching stations and substa-tions to form a network of lines and stations calleda power grid. Electricity flows through the inter-connected network of transmission lines from thegenerators to the loads in accordance with thelaws of physicsalong paths of least resistance,in much the same way that water flows through anetwork of canals. When the power arrives near aload center, it is stepped down to lower voltagesfor distribution to customers. The bulk power sys-tem is predominantly an alternating current (AC)system, as opposed to a direct current (DC) sys-tem, because of the ease and low cost with whichvoltages in AC systems can be converted from onelevel to another. Some larger industrial and com-

mercial customers take service at intermediatevoltage levels (12,000 to 115,000 volts), but mostresidential customers take their electrical serviceat 120 and 240 volts.

While the power system in North America is com-monly referred to as the grid, there are actuallythree distinct power grids or interconnections(Figure 2.2). The Eastern Interconnection includesthe eastern two-thirds of the continental UnitedStates and Canada from Saskatchewan east to theMaritime Provinces. The Western Interconnectionincludes the western third of the continentalUnited States (excluding Alaska), the Canadianprovinces of Alberta and British Columbia, and aportion of Baja California Norte, Mexico. The thirdinterconnection comprises most of the state ofTexas. The three interconnections are electrically

independent from each other except for a fewsmall direct current (DC) ties that link them.Within each interconnection, electricity is pro-duced the instant it is used, and flows over virtu-ally all transmission lines from generators toloads.

The northeastern portion of the Eastern Intercon-

nection (about 10 percent of the interconnectionstotal load) was affected by the August 14 blackout.The other two interconnections were notaffected.1

Planning and Reliable Operationof the Power Grid Are TechnicallyDemanding

Reliable operation of the power grid is complexand demanding for two fundamental reasons:

u First, electricity flows at close to the speed oflight (186,000 miles per second or 297,600km/sec) and is not economically storable inlarge quantities. Therefore electricity must beproduced the instant it is used.

u Second, without the use of control devices tooexpensive for general use, the flow of alternat-ing current (AC) electricity cannot be controlledlike a liquid or gas by opening or closing a valvein a pipe, or switched like calls over a long-distance telephone network.2 Electricity flowsfreely along all available paths from the genera-tors to the loads in accordance with the laws ofphysicsdividing among all connected flowpaths in the network, in inverse proportion tothe impedance (resistance plus reactance) oneach path.

Maintaining reliability is a complex enterprisethat requires trained and skilled operators, sophis-ticated computers and communications, and care-ful planning and design. The North AmericanElectric Reliability Council (NERC) and its tenRegional Reliability Councils have developed sys-

tem operating and planning standards for ensur-ing the reliability of a transmission grid that arebased on seven key concepts:

u Balance power generation and demandcontinuously.

u Balance reactive power supply and demand tomaintain scheduled voltages.

u Monitor flows over transmission lines and otherfacilities to ensure that thermal (heating) limitsare not exceeded.

6G



Figure 2.2. North American Interconnections


15/30

u Keep the system in a stable condition.

u Operate the system so that it remains in a reli-able condition even if a contingency occurs,such as the loss of a key generator or transmis-sion facility (the N-1 criterion).

u Plan, design, and maintain the system to oper-ate reliably.

u Prepare for emergencies.

These seven concepts are explained in more detailbelow.

1. Balance power generation and demand contin-uously. To enable customers to use as muchelectricity as they wish at any moment, produc-tion by the generators must be scheduled ordispatched to meet constantly changingdemands, typically on an hourly basis, and thenfine-tuned throughout the hour, sometimes

through the use of automatic generation con-trols to continuously match generation to actualdemand. Demand is somewhat predictable,appearing as a daily demand curvein thesummer, highest during the afternoon and eve-ning and lowest in the middle of the night, andhigher on weekdays when most businesses areopen (Figure 2.3).

Failure to match generation to demand causesthe frequency of an AC power system (nomi-nally 60 cycles per second or 60 Hertz) toincrease (when generation exceeds demand) or

decrease (when generation is less than demand)(Figure 2.4). Random, small variations in fre-quency are normal, as loads come on and offand generators modify their output to follow thedemand changes. However, large deviations infrequency can cause the rotational speed of gen-erators to fluctuate, leading to vibrations thatcan damage generator turbine blades and otherequipment. Extreme low frequencies can trigger

automatic under-frequency load shedding,which takes blocks of customers off-line inorder to prevent a total collapse of the electricsystem. As will be seen later in this report, suchan imbalance of generation and demand canalso occur when the system responds to majordisturbances by breaking into separateislands; any such island may have an excessor a shortage of generation, compared todemand within the island.

2. Balance reactive power supply and demand tomaintain scheduled voltages. Reactive powersources, such as capacitor banks and genera-tors, must be adjusted during the day to main-tain voltages within a secure range pertaining toall system electrical equipment (stations, trans-mission lines, and customer equipment). Mostgenerators have automatic voltage regulatorsthat cause the reactive power output of genera-

tors to increase or decrease to control voltages toscheduled levels. Low voltage can cause electricsystem instability or collapse and, at distribu-tion voltages, can cause damage to motors andthe failure of electronic equipment. High volt-ages can exceed the insulation capabilities ofequipment and cause dangerous electric arcs(flashovers).

3. Monitor flows over transmission lines andother facilities to ensure that thermal (heating)limits are not exceeded. The dynamic interac-tions between generators and loads, combined

with the fact that electricity flows freely acrossall interconnected circuits, mean that powerflow is ever-changing on transmission and dis-tribution lines. All lines, transformers, andother equipment carrying electricity are heatedby the flow of electricity through them. The

G



7

Figure 2.3. PJM Load Curve, August 18-24, 2003

Figure 2.4. Normal and Abnormal FrequencyRanges


16/30

flow must be limited to avoid overheating anddamaging the equipment. In the case of over-

head power lines, heating also causes the metalconductor to stretch or expand and sag closer toground level. Conductor heating is also affectedby ambient temperature, wind, and other fac-tors. Flow on overhead lines must be limited toensure that the line does not sag into obstruc-tions below such as trees or telephone lines, orviolate the minimum safety clearances betweenthe energized lines and other objects. (A shortcircuit or flashoverwhich can start fires ordamage equipmentcan occur if an energizedline gets too close to another object). Most trans-

mission lines, transformers and other current-carrying devices are monitored continuously toensure that they do not become overloaded orviolate other operating constraints. Multipleratings are typically used, one for normal condi-tions and a higher rating for emergencies. Theprimary means of limiting the flow of power ontransmission lines is to adjust selectively theoutput of generators.

4. Keep the system in a stable condition. Becausethe electric system is interconnected anddynamic, electrical stability limits must beobserved. Stability problems can develop veryquicklyin just a few cycles (a cycle is 1/60th ofa second)or more slowly, over seconds orminutes. The main concern is to ensure thatgeneration dispatch and the resulting powerflows and voltages are such that the system isstable at all times. (As will be described later inthis report, part of the Eastern Interconnectionbecame unstable on August 14, resulting in acascading outage over a wide area.) Stability

limits, like thermal limits, are expressed as amaximum amount of electricity that can be

safely transferred over transmission lines.

There are two types of stability limits: (1) Volt-age stability limits are set to ensure that theunplanned loss of a line or generator (whichmay have been providing locally critical reac-tive power support, as described previously)will not cause voltages to fall to dangerouslylow levels. If voltage falls too low, it begins tocollapse uncontrollably, at which point auto-matic relays either shed load or trip generatorsto avoid damage. (2) Power (angle) stability lim-its are set to ensure that a short circuit or anunplanned loss of a line, transformer, or genera-tor will not cause the remaining generators andloads being served to lose synchronism withone another. (Recall that all generators andloads within an interconnection must operate ator very near a common 60 Hz frequency.) Lossof synchronism with the common frequencymeans generators are operating out-of-step withone another. Even modest losses of synchro-nism can result in damage to generation equip-ment. Under extreme losses of synchronism,the grid may break apart into separate electrical

islands; each island would begin to maintain itsown frequency, determined by the load/genera-tion balance within the island.

5. Operate the system so that it remains in a reli-able condition even if a contingency occurs,such as the loss of a key generator or transmis-sion facility (the N minus 1 criterion). Thecentral organizing principle of electricity reli-ability management is to plan for the unex-pected. The unique characteristics of electricity

8G



Local Supplies of Reactive Power Are Essential to Maintaining Voltage Stability

A generator typically produces some mixture ofreal and reactive power, and the balancebetween them can be adjusted at short notice tomeet changing conditions. Real power, measuredin watts, is the form of electricity that powersequipment. Reactive power, a characteristic ofAC systems, is measured in volt-amperes reac-tive (VAr), and is the energy supplied to create orbe stored in electric or magnetic fields in andaround electrical equipment. Reactive power isparticularly important for equipment that relieson magnetic fields for the production of inducedelectric currents (e.g., motors, transformers,pumps, and air conditioning.) Transmission

lines both consume and produce reactive power.At light loads they are net producers, and atheavy loads, they are heavy consumers. Reactivepower consumption by these facilities or devicestends to depress transmission voltage, while itsproduction (by generators) or injection (fromstorage devices such as capacitors) tends to sup-port voltage. Reactive power can be transmittedonly over relatively short distances during heavyload conditions. If reactive power cannot be sup-plied promptly and in sufficient quantity, volt-ages decay, and in extreme cases a voltagecollapse may result.


17/30

mean that problems, when they arise, canspread and escalate very quickly if proper safe-guards are not in place. Accordingly, throughyears of experience, the industry has developeda network of defensive strategies for maintain-ing reliability based on the assumption thatequipment can and will fail unexpectedly uponoccasion.

This principle is expressed by the requirementthat the system must be operated at all times toensure that it will remain in a secure condition(generally within emergency ratings for currentand voltage and within established stabilitylimits) following the loss of the most importantgenerator or transmission facility (a worst sin-gle contingency). This is called the N-1 crite-rion. In other words, because a generator orline trip can occur at any time from random fail-ure, the power system must be operated in a

preventive mode so that the loss of the mostimportant generator or transmission facility

does not jeopardize the remaining facilities inthe system by causing them to exceed theiremergency ratings or stability limits, whichcould lead to a cascading outage.

Further, when a contingency does occur, theoperators are required to identify and assessimmediately the new worst contingencies,

given the changed conditions, and promptlymake any adjustments needed to ensure that ifone of them were to occur, the system wouldstill remain operational and safe. NERC operat-ing policy requires that the system be restoredas soon as practical but within no more than 30minutes to compliance with normal limits, andto a condition where it can once again with-stand the next-worst single contingency with-out violating thermal, voltage, or stabilitylimits. A few areas of the grid are operated towithstand the concurrent loss of two or more

facilities (i.e., N-2). This may be done, forexample, as an added safety measure to protect

G



9

Why Dont More Blackouts Happen?

Given the complexity of the bulk power systemand the day-to-day challenges of operating it,there are a lot of things that could go wrongwhich makes it reasonable to wonder why so fewlarge outages occur.

Large outages or blackouts are infrequentbecause responsible system owners and opera-tors practice defense in depth, meaning thatthey protect the bulk power system through lay-ers of safety-related practices and equipment.These include:

1. A range of rigorous planning and operatingstudies, including long-term assessments,year-ahead, season-ahead, week-ahead, day-ahead, hour-ahead, and real-time operationalcontingency analyses. Planners and operatorsuse these to evaluate the condition of the sys-

tem, anticipate problems ranging from likelyto low probability but high consequence, anddevelop a good understanding of the limits andrules for safe, secure operation under suchcontingencies. If multiple contingencies occurin a single area, they are likely to be interde-pendent rather than random, and should havebeen anticipated in planning studies.

2. Preparation for the worst case. The operatingrule is to always prepare the system to be safe

in the face of the worst single contingency thatcould occur relative to current conditions,which means that the system is also preparedfor less adverse contingencies.

3. Quick response capability. Most potentialproblems first emerge as a small, local situa-tion. When a small, local problem is handledquickly and responsibly using NERC operatingpracticesparticularly to return the system toN-1 readiness within 30 minutes or lessthe problem can usually be resolved andcontained before it grows beyond localproportions.

4. Maintain a surplus of generation and trans-mission. This provides a cushion in day-to-day operations, and helps ensure that smallproblems dont become big problems.

5. Have backup capabilities for all critical func-tions. Most owners and operators maintainbackup capabilitiessuch as redundantequipment already on-line (from generation inspinning reserve and transmission operatingmargin and limits to computers and otheroperational control systems)and keep aninventory of spare parts to be able to handle anequipment failure.


18/30

a densely populated metropolitan area or whenlines share a common structure and could beaffected by a common failure mode, e.g., a sin-gle lightning strike.

6. Plan, design, and maintain the system to oper-ate reliably. Reliable power system operationrequires far more than monitoring and control-

ling the system in real-time. Thorough plan-ning, design, maintenance, and analysis arerequired to ensure that the system can be oper-ated reliably and within safe limits. Short-termplanning addresses day-ahead and week-aheadoperations planning; long-term planningfocuses on providing adequate generationresources and transmission capacity to ensurethat in the future the system will be able towithstand severe contingencies without experi-encing widespread, uncontrolled cascadingoutages.

A utility that serves retail customers must esti-mate future loads and, in some cases, arrangefor adequate sources of supplies and plan ade-quate transmission and distribution infrastruc-ture. NERC planning standards identify a rangeof possible contingencies and set correspondingexpectations for system performance under sev-eral categories of possible events, ranging fromeveryday probable events to extreme eventsthat may involve substantial loss of customerload and generation in a widespread area. NERCplanning standards also address requirements

for voltage support and reactive power, distur-bance monitoring, facility ratings, system mod-eling and data requirements, system protectionand control, and system restoration.

7. Prepare for emergencies. System operators arerequired to take the steps described above toplan and operate a reliable power system, butemergencies can still occur because of externalfactors such as severe weather, operator error,or equipment failures that exceed planning,design, or operating criteria. For these rareevents, the operating entity is required to haveemergency procedures covering a crediblerange of emergency scenarios. Operators mustbe trained to recognize and take effective actionin response to these emergencies. To deal with asystem emergency that results in a blackout,such as the one that occurred on August 14,2003, there must be procedures and capabilitiesto use black start generators (capable ofrestarting with no external power source) and tocoordinate operations in order to restore the

system as quickly as possible to a normal andreliable condition.

Reliability Organizations OverseeGrid Reliability in North America

NERC is a non-governmental entity whose mis-

sion is to ensure that the bulk electric system inNorth America is reliable, adequate and secure.The organization was established in 1968, as aresult of the Northeast blackout in 1965. Since itsinception, NERC has operated as a voluntary orga-nization, relying on reciprocity, peer pressure andthe mutual self-interest of all those involved toensure compliance with reliability requirements.An independent board governs NERC.

To fulfill its mission, NERC:

u Sets standards for the reliable operation and

planning of the bulk electric system.u Monitors and assesses compliance with stan-

dards for bulk electric system reliability.

u Provides education and training resources topromote bulk electric system reliability.

u Assesses, analyzes and reports on bulk electricsystem adequacy and performance.

u Coordinates with regional reliability councilsand other organizations.

u Coordinates the provision of applications(tools), data and services necessary to supportthe reliable operation and planning of the bulkelectric system.

u Certifies reliability service organizations andpersonnel.

u Coordinates critical infrastructure protection ofthe bulk electric system.

u Enables the reliable operation of the intercon-nected bulk electric system by facilitating infor-mation exchange and coordination among

reliability service organizations.Recent changes in the electricity industry havealtered many of the traditional mechanisms,incentives and responsibilities of the entitiesinvolved in ensuring reliability, to the point thatthe voluntary system of compliance with reliabil-ity standards is generally recognized as not ade-quate to current needs.3 NERC and many otherelectricity organizations support the developmentof a new mandatory system of reliability standards

10G




19/30

and compliance, backstopped in the United Statesby the Federal Energy Regulatory Commission.This will require federal legislation in the UnitedStates to provide for the creation of a new electricreliability organization with the statutory author-ity to enforce compliance with reliability stan-dards among all market participants. Appropriategovernment entities in Canada and Mexico areprepared to take similar action, and some havealready done so. In the meantime, NERC encour-ages compliance with its reliability standardsthrough an agreement with its members.

NERCs members are ten regional reliabilitycouncils. (See Figure 2.5 for a map showing thelocations and boundaries of the regional councils.)In turn, the regional councils have broadenedtheir membership to include all segments of theelectric industry: investor-owned utilities; federalpower agencies; rural electric cooperatives; state,

municipal and provincial utilities; independentpower producers; power marketers; and end-usecustomers. Collectively, the members of the NERCregions account for virtually all the electricity sup-plied in the United States, Canada, and a portionof Baja California Norte, Mexico. The ten regionalcouncils jointly fund NERC and adapt NERCstandards to meet the needs of their regions. TheAugust 14 blackout affected three NERC regionalreliability councilsEast Central Area ReliabilityCoordination Agreement (ECAR), Mid-AtlanticArea Council (MAAC), and Northeast Power Coor-

dinating Council (NPCC).Control areas are the primary operational enti-ties that are subject to NERC and regional councilstandards for reliability. A control area is a geo-graphic area within which a single entity, Inde-pendent System Operator (ISO), or RegionalTransmission Organization (RTO) balances gener-ation and loads in real time to maintain reliableoperation. Control areas are linked with eachother through transmission interconnection tielines. Control area operators control generationdirectly to maintain their electricity interchange

schedules with other control areas. They alsooperate collectively to support the reliability oftheir interconnection. As shown in Figure 2.6,there are approximately 140 control areas in NorthAmerica. The control area dispatch centers havesophisticated monitoring and control systems andare staffed 24 hours per day, 365 days per year.

Traditionally, control areas were defined by utilityservice area boundaries and operations werelargely managed by vertically integrated utilities

that owned and operated generation, transmis-sion, and distribution. While that is still true insome areas, there has been significant restructur-ing of operating functions and some consolidationof control areas into regional operating entities.Utility industry restructuring has led to anunbundling of generation, transmission and dis-tribution activities such that the ownership andoperation of these assets have been separatedeither functionally or through the formation ofindependent entities called Independent SystemOperators (ISOs) and Regional TransmissionOrganizations (RTOs).

u ISOs and RTOs in the United States have beenauthorized by FERC to implement aspects of theEnergy Policy Act of 1992 and subsequent FERCpolicy directives.

u The primary functions of ISOs and RTOs are tomanage in real time and on a day-ahead basis

the reliability of the bulk power system and theoperation of wholesale electricity marketswithin their footprint.

u ISOs and RTOs do not own transmission assets;they operate or direct the operation of assetsowned by their members.

u ISOs and RTOs may be control areas them-selves, or they may encompass more than onecontrol area.

u ISOs and RTOs may also be NERC ReliabilityCoordinators, as described below.

Five RTOs/ISOs are within the area directlyaffected by the August 14 blackout. They are:

u Midwest Independent System Operator (MISO)

u PJM Interconnection (PJM)

G



11

Figure 2.5. NERC Regions


20/30

u New York Independent System Operator(NYISO)

u New England Independent System Operator(ISO-NE)

u Ontario Independent Market Operator (IMO)

Reliability coordinators provide reliability over-

sight over a wide region. They prepare reliabilityassessments, provide a wide-area view of reliabil-ity, and coordinate emergency operations in realtime for one or more control areas. They may oper-ate, but do not participate in, wholesale or retailmarket functions. There are currently 18 reliabil-ity coordinators in North America. Figure 2.7shows the locations and boundaries of theirrespective areas.

Key Parties in the Pre-CascadePhase of the August 14 Blackout

The initiating events of the blackout involved twocontrol areasFirstEnergy (FE) and American

Electric Power (AEP)and their respective reli-ability coordinators, MISO and PJM (see Figures2.7 and 2.8). These organizations and their reli-ability responsibilities are described briefly in thisfinal subsection.

1. FirstEnergy operates a control area in north-ern Ohio. FirstEnergy (FE) consists of seven

electric utility operating companies. Four ofthese companies, Ohio Edison, Toledo Edison,The Illuminating Company, and Penn Power,operate in the NERC ECAR region, with MISOserving as their reliability coordinator. Thesefour companies now operate as one integratedcontrol area managed by FE.4

2. American Electric Power (AEP) operates a con-trol area in Ohio just south of FE. AEP is both atransmission operator and a control areaoperator.

3. Midwest Independent System Operator(MISO) is the reliability coordinator forFirstEnergy. The Midwest Independent System

12G



Figure 2.6. NERC Regions and Control Areas


21/30

Operator (MISO) is the reliability coordinatorfor a region of more than 1 million square miles(2.6 million square kilometers), stretching fromManitoba, Canada in the north to Kentucky inthe south, from Montana in the west to westernPennsylvania in the east. Reliability coordina-tion is provided by two offices, one in Minne-sota, and the other at the MISO headquarters inIndiana. Overall, MISO provides reliabilitycoordination for 37 control areas, most of whichare members of MISO.

4. PJM is AEPs reliability coordinator. PJM is oneof the original ISOs formed after FERC orders888 and 889, but was established as a regionalpower pool in 1935. PJM recently expanded itsfootprint to include control areas and transmis-sion operators within MAIN and ECAR (PJM-West). It performs its duties as a reliability coor-dinator in different ways, depending on the

control areas involved. For PJM-East, it isboth the control area and reliability coordinatorfor ten utilities, whose transmission systemsspan the Mid-Atlantic region of New Jersey,most of Pennsylvania, Delaware, Maryland,West Virginia, Ohio, Virginia, and the District ofColumbia. The PJM-West facility has the reli-ability coordinator desk for five control areas(AEP, Commonwealth Edison, Duquesne Light,Dayton Power and Light, and Ohio Valley Elec-tric Cooperative) and three generation-onlycontrol areas (Duke Energys WashingtonCounty (Ohio) facility, Dukes LawrenceCounty/Hanging Rock (Ohio) facility, and Alle-gheny Energys Buchanan (West Virginia)facility.

Reliability Responsibilities of ControlArea Operators and ReliabilityCoordinators

1. Control area operators have primary responsi-bility for reliability. Their most importantresponsibilities, in the context of this report,are:

N-1 criterion. NERC Operating Policy 2.ATransmission Operations:

All CONTROL AREAS shall operate so thatinstability, uncontrolled separation, or cas-cading outages will not occur as a result ofthe most severe single contingency.

Emergency preparedness and emergencyresponse. NERC Operating Policy 5Emer-gency Operations, General Criteria:

Each system and CONTROL AREA shallpromptly take appropriate action to relieveany abnormal conditions, which jeopardizereliable Interconnection operation.

Each system, CONTROL AREA, and Regionshall establish a program of manual and auto-matic load shedding which is designed toarrest frequency or voltage decays that couldresult in an uncontrolled failure of compo-nents of the interconnection.

NERC Operating Policy 5.ACoordinationwith Other Systems:

A system, CONTROL AREA, or pool that isexperiencing or anticipating an operatingemergency shall communicate its currentand future status to neighboring systems,CONTROL AREAS, or pools and throughout theinterconnection . . . . A system shall inform

G



13

Figure 2.7. NERC Reliability Coordinators

Figure 2.8. Reliability Coordinators and ControlAreas in Ohio and Surrounding States


22/30

other systems . . . whenever . . . the systemscondition is burdening other systems orreducing the reliability of the Interconnec-tion . . . [or whenever] the systems line load-ings and voltage/reactive levels are such thata single contingency could threaten the reli-ability of the Interconnection.

NERC Operating Policy 5.CTransmissionSystem Relief:

Action to correct an OPERATING SECURITYLIMIT violation shall not impose unaccept-able stress on internal generation or transmis-sion equipment, reduce system reliabilitybeyond acceptable limits, or unduly imposevoltage or reactive burdens on neighboringsystems. If all other means fail, correctiveaction may require load reduction.

Operating personnel and training: NERC Oper-ating Policy 8.BTraining:

Each OPERATING AUTHORITY should period-ically practice simulated emergencies. Thescenarios included in practice situationsshould represent a variety of operating condi-tions and emergencies.

2. Reliability Coordinators such as MISO andPJM are expected to comply with all aspects ofNERC Operating Policies, especially Policy 9,Reliability Coordinator Procedures, and itsappendices. Key requirements include:

NERC Operating Policy 9, Criteria for Reliabil-ity Coordinators, 5.2:

Have detailed monitoring capability of theRELIABILITY AREA and sufficient monitoring

14G



Institutional Complexities and Reliability in the Midwest

The institutional arrangements for reliability inthe Midwest are much more complex than theyare in the Northeasti.e., the areas covered bythe Northeast Power Coordinating Council(NPCC) and the Mid-Atlantic Area Council(MAAC). There are two principal reasons for thiscomplexity. One is that in NPCC and MAAC, theindependent system operator (ISO) also serves asthe single control area operator for the individualmember systems. In comparison, MISO provides

reliability coordination for 35 control areas in theECAR, MAIN, and MAPP regions and 2 others inthe SPP region, and PJM provides reliability coor-dination for 8 control areas in the ECAR andMAIN regions (plus one in MAAC). (See tablebelow.) This results in 18 control-area-to-control-area interfaces across the PJM/MISO reli-ability coordinator boundary.

The other is that MISO has less reliability-relatedauthority over its control area members than PJM

has over its members. Arguably, this lack ofauthority makes day-to-day reliability operationsmore challenging. Note, however, that (1) FERCsauthority to require that MISO have greaterauthority over its members is limited; and (2)before approving MISO, FERC asked NERC for aformal assessment of whether reliability could bemaintained under the arrangements proposed byMISO and PJM. After reviewing proposed plansfor reliability coordination within and between

PJM and MISO, NERC replied affirmatively butprovisionally. FERC approved the new MISO-PJM configuration based on NERCs assessment.NERC conducted audits in November andDecember 2002 of the MISO and PJM reliabilityplans, and some of the recommendations of theaudit teams are still being addressed. The ade-quacy of the plans and whether the plans werebeing implemented as written are factors inNERCs ongoing investigation.

Reliability Coordinator (RC)

Control

Areas inRC Area

Regional Reliability

Councils Affected andNumber of Control Areas Control Areas of Interest in RC Area

MISO 37 ECAR (12), MAIN (9),MAPP (14), SPP (2)

FE, Cinergy,Michigan Electric Coordinated System

PJM 9 MAAC (1), ECAR (7),MAIN (1)

PJM, AEP,Dayton Power & Light

ISO New England 2 NPCC (2) ISONE, Maritime Provinces

New York ISO 1 NPCC (1) NYISO

Ontario Independent Market Operator 1 NPCC (1) IMO

Trans-Energie 1 NPCC (1) Hydro Qubec


23/30

capability of the surrounding RELIABILITYAREAS to ensure potential security violationsare identified.

NERC Operating Policy 9, Functions of Reliabil-ity Coordinators, 1.7:

Monitor the parameters that may have sig-nificant impacts within the RELIABILITY AREAand with neighboring RELIABILITY AREASwith respect to . . . sharing with otherRELIABILITY COORDINATORS any informationregarding potential, expected, or actual criti-cal operating conditions that could nega-tively impact other RELIABILITY AREAS. TheRELIABILITY COORDINATOR will coordinatewith other RELIABILITY COORDINATORS andCONTROL AREAS as needed to develop appro-priate plans to mitigate negative impacts ofpotential, expected, or actual critical operat-ing conditions . . . .

NERC Operating Policy 9, Functions of Reliabil-ity Coordinators, 6:

Conduct security assessment and monitor-ing programs to assess contingency situa-tions. Assessments shall be made in real timeand for the operations planning horizon atthe CONTROL AREA level with any identified

problems reported to the RELIABILITY CO-ORDINATOR. The RELIABILITY COORDINATORis to ensure that CONTROL AREA, RELIABILITYAREA, and regional boundaries are suffi-ciently modeled to capture any problemscrossing such boundaries.

Endnotes

G



15

What Constitutes an Operating Emergency?

An operating emergency is an unsustainablecondition that cannot be resolved using theresources normally available. The NERC Oper-ating Manual defines a capacity emergency aswhen a systems or pools operating generationcapacity, plus firm purchases from other sys-tems, to the extent available or limited by trans-fer capability, is inadequate to meet its demandplus its regulating requirements. It defines an

energy emergency as when a load-servingentity has exhausted all other options and canno longer provide its customers expectedenergy requirements. A transmission emer-gency exists when the systems line loadingsand voltage/ reactive levels are such that a singlecontingency could threaten the reliability of theInterconnection. Control room operators anddispatchers are given substantial latitude todetermine when to declare an emergency. (Seepages 66-67 in Chapter 5 for more detail.)

1 The province of Qubec, although considered a part of theEastern Interconnection, is connected to the rest of the East-ern Interconnection only by DC ties. In this instance, the DCties acted as buffers between portions of the Eastern Intercon-nection; transient disturbances propagate through them less

readily. Therefore, the electricity system in Qubec was notaffected by the outage, except for a small portion of the prov-inces load that is directly connected to Ontario by AC trans-mission lines. (Although DC ties can act as a buffer betweensystems, the tradeoff is that they do not allow instantaneousgeneration support following the unanticipated loss of a gen-erating unit.)2 In some locations, bulk power flows are controlled throughspecialized devices or systems, such as phase angle regula-tors, flexible AC transmission systems (FACTS), and high-voltage DC converters (and reconverters) spliced into the ACsystem. These devices are still too expensive for generalapplication.3 See, for example, Maintaining Reliability in a Competitive Electric Industry(1998), a report to the U.S. Secretary ofEnergy by the Task Force on Electric Systems Reliability;National Energy Policy(2001), a report to the President of theUnited States by the National Energy Policy DevelopmentGroup, p. 7-6; and National Transmission Grid Study (2002),U.S. Dept. of Energy, pp. 46-48.4 The remaining three FE companies, Penelec, Met-Ed, and

Jersey Central Power & Light, are in the NERC MAAC regionand have PJM as their reliability coordinator. The focus of thisreport is on the portion of FE in the ECAR reliability regionand within the MISO reliability coordinator footprint.


24/30


25/30

3. Causes of the Blackout

and Violations of NERC Standards

Summary

This chapter explains in summary form the causesof the initiation of the blackout in Ohio, based onthe analyses by the bi-national investigation team.It also lists NERCs findings to date concerningseven specific violations of its reliability policies,guidelines, and standards. Last, it explains howsome NERC standards and processes were inade-

quate because they did not give sufficiently cleardirection to industry members concerning somepreventive measures needed to maintain reliabil-ity, and that NERC does not have the authority toenforce compliance with the standards. Clearstandards with mandatory compliance, as con-templated under legislation pending in the U.S.Congress, might have averted the start of thisblackout.

Chapters 4 and 5 provide the details that supportthe conclusions summarized here, by describingconditions and events during the days before and

the day of the blackout, and explain how thoseevents and conditions did or did not cause or con-tribute to the initiation of the blackout. Chapter 6addresses the cascade as the blackout spreadbeyond Ohio and reviews the causes and events ofthe cascade as distinct from the earlier events inOhio.

The Causes of the Blackout in Ohio

A dictionary definition of cause is somethingthat produces an effect, result, or consequence.1

In searching for the causes of the blackout, theinvestigation team looked back through the pro-gression of sequential events, actions and inac-tions to identify the cause(s) of each event. Theidea of cause is here linked not just to what hap-pened or why it happened, but more specificallyto the entities whose duties and responsibilitieswere to anticipate and prepare to deal with thethings that could go wrong. Four major causes, orgroups of causes, are identified (see box on page18).

Although the causes discussed below producedthe failures and events of August 14, they did notleap into being that day. Instead, as the followingchapters explain, they reflect long-standing insti-tutional failures and weaknesses that need to beunderstood and corrected in order to maintainreliability.

Linking Causes

to Specific WeaknessesSeven violations of NERC standards, as identifiedby NERC,2 and other conclusions reached byNERC and the bi-national investigation team arealigned below with the specific causes of theblackout. There is an additional category of con-clusions beyond the four principal causesthefailure to act, when it was the result of precedingconditions. For instance, FE did not respond to theloss of its transmission lines because it did nothave sufficient information or insight to reveal the

need for action. Note: NERCs list of violations hasbeen revised and extended since publication ofthe Interim Report. Two violations (numbers 4and 6, as cited in the Interim Report) weredropped, and three new violations have beenidentified in this report (5, 6, and 7, as numberedhere). NERC continues to study the record andmay identify additional violations.3

Group 1: FirstEnergy and ECAR failed to assessand understand the inadequacies of FEs

system, particularly with respect to voltageinstability and the vulnerability of the

Cleveland-Akron area, and FE did not operateits system with appropriate voltage criteriaand remedial measures.

u FE did not monitor and manage reactivereserves for various contingency conditions asrequired by NERC Policy 2, Section B, Require-ment 2.

u NERC Policy 2, Section A, requires a 30-minuteperiod of time to re-adjust the system to prepareto withstand the next contingency.

G



17


26/30

18G



Causes of the Blackouts Initiation

The Ohio phase of the August 14, 2003, blackoutwas caused by deficiencies in specific practices,equipment, and human decisions by variousorganizations that affected conditions and out-comes that afternoonfor example, insufficientreactive power was an issue in the blackout, butit was not a cause in itself. Rather, deficiencies incorporate policies, lack of adherence to industrypolicies, and inadequate management of reactivepower and voltage caused the blackout, ratherthan the lack of reactive power. There are fourgroups of causes for the blackout:

Group 1: FirstEnergy and ECAR failed toassess and understand the inadequacies of

FEs system, particularly with respect to

voltage instability and the vulnerability ofthe Cleveland-Akron area, and FE did notoperate its system with appropriate voltagecriteria. (Note: This cause was not identified inthe Task Forces Interim Report. It is based onanalysis completed by the investigative teamafter the publication of the Interim Report.)

As detailed in Chapter 4:

A) FE failed to conduct rigorous long-term plan-ning studies of its system, and neglected toconduct appropriate multiple contingency orextreme condition assessments. (See pages37-39 and 41-43.)

B) FE did not conduct sufficient voltage analysesfor its Ohio control area and used operationalvoltage criteria that did not reflect actual volt-age stability conditions and needs. (See pages31-37.)

C) ECAR (FEs reliability council) did not con-duct an independent review or analysis ofFEs voltage criteria and operating needs,thereby allowing FE to use inadequate prac-tices without correction. (See page 39.)

D)Some of NERCs planning and operationalrequirements and standards were sufficientlyambiguous that FE could interpret them toinclude practices that were inadequate for reli-able system operation. (See pages 31-33.)

Group 2: Inadequate situational awarenessat FirstEnergy. FE did not recognize orunderstand the deteriorating condition of

its system.

As discussed in Chapter 5:A) FE failed to ensure the security of its transmis-

sion system after significant unforeseen con-tingencies because it did not use an effectivecontingency analysis capability on a routinebasis. (See pages 49-50 and 64.)

B) FE lacked procedures to ensure that its opera-tors were continually aware of the functionalstate of their critical monitoring tools. (Seepages 51-53, 56.)

C) FE control center computer support staff and

operations staff did not have effective internalcommunications procedures. (See pages 54,56, and 65-67.)

D) FE lacked procedures to test effectively thefunctional state of its monitoring tools afterrepairs were made. (See page 54.)

E) FE did not have additional or back-up moni-toring tools to understand or visualize the sta-tus of their transmission system to facilitateits operators understanding of transmissionsystem conditions after the failure of their pri-

mary monitoring/alarming systems. (Seepages 53, 56, and 65.)

Group 3: FE failed to manage adequately treegrowth in its transmission rights-of-way.

This failure was the common cause of the outageof three FE 345-kV transmission lines and one138-kV line. (See pages 57-64.)

Group 4: Failure of the interconnected gridsreliability organizations to provide effectivereal-time diagnostic support.

As discussed in Chapter 5:

A) MISO did not have real-time data fromDayton Power and Lights Stuart-Atlanta345-kV line incorporated into its state estima-tor (a system monitoring tool). This precluded

(continued on page 19)


27/30

u NERC is lacking a well-defined control area(CA) audit process that addresses all CA respon-sibilities. Control area audits have generally notbeen conducted with sufficient regularity andhave not included a comprehensive audit of thecontrol areas compliance with all NERC andRegional Council requirements. Compliancewith audit results is not mandatory.

u ECAR did not conduct adequate review or anal-

yses of FEs voltage criteria, reactive powermanagement practices, and operating needs.

u FE does not have an adequate automatic under-voltage load-shedding program in the Cleve-land-Akron area.

Group 2: Inadequate situational awarenessat FirstEnergy. FE did not recognize orunderstand the deteriorating condition of

its system.

Violations (Identified by NERC):

u

Violation 7: FEs operational monitoring equip-ment was not adequate to alert FEs operatorsregarding important deviations in operatingconditions and the need for corrective action asrequired by NERC Policy 4, Section A, Require-ment 5.

u Violation 3: FEs state estimation and contin-gency analysis tools were not used to assesssystem conditions, violating NERC OperatingPolicy 5, Section C, Requirement 3, and Policy4, Section A, Requirement 5.

Other Problems:

u FE personnel did not ensure that theirReal-Time Contingency Analysis (RTCA) was afunctional and effective EMS application asrequired by NERC Policy 2, Section A, Require-ment 1.

u FEs operational monitoring equipment was notadequate to provide a means for its operators toevaluate the effects of the loss of significanttransmission or generation facilities as requiredby NERC Policy 4, Section A, Requirement 4.

u FEs operations personnel were not providedsufficient operations information and analysistools as required by NERC Policy 5, Section C,Requirement 3.

u FEs operations personnel were not adequatelytrained to maintain reliable operation underemergency conditions as required by NERC Pol-icy 8, Section 1.

u NERC Policy 4 has no detailed requirements for:

(a) monitoring and functional testing of criticalEMS and supervisory control and data acquisi-tion (SCADA) systems, and (b) contingencyanalysis.

u NERC Policy 6 includes a requirement to planfor loss of the primary control center, but lacksspecific provisions concerning what must beaddressed in the plan.

u NERC system operator certification tests forbasic operational and policy knowledge.

G



19

Causes of the Blackouts Initiation (Continued)

MISO from becoming aware of FEs systemproblems earlier and providing diagnosticassistance or direction to FE. (See pages49-50.)

B) MISOs reliability coordinators were usingnon-real-time data to support real-timeflowgate monitoring. This prevented MISOfrom detecting an N-1 security violation inFEs system and from assisting FE in neces-sary relief actions. (See pages 48 and 63.)

C) MISO lacked an effective way to identify thelocation and significance of transmission linebreaker operations reported by their EnergyManagement System (EMS). Such informa-tion would have enabled MISO operators tobecome aware earlier of important line out-

ages. (See page 48.)

D) PJM and MISO lacked joint procedures orguidelines on when and how to coordinate asecurity limit violation observed by one ofthem in the others area due to a contingencynear their common boundary. (See pages

62-63 and 65-66.)

In the chapters that follow, sections that relate toparticular causes are denoted with the followingsymbols:

Cause 2InadequateSituationalAwareness

Cause 3InadequateTreeTrimming

Cause 4InadequateRC DiagnosticSupport

Cause 1InadequateSystemUnderstanding

8/3/2019

us canada power system outgate task force

Documents