real-time synchrophasor data quality analysis and ...€¦ · operations and analysis”, in...
TRANSCRIPT
Real-time Synchrophasor Data
Quality Analysis and Improvement
PSERC IAB Meeting
December 2-4, 2015
Research Team
Le Xie, Texas A&M University
P. R. Kumar, Texas A&M University
Mani Venkatasubramanian, Washington State University
Industry Advisors
• Aftab Alam, California ISO
• Frankie Zhang, ISO-New
England
• Chaitanya A. Baone, GE
Global Research
• Santosh Veda, GE Global
Research
• Paul T. Myrda, EPRI
• Liang Min, LLNL
• Mahendra Patel, EPRI
• Giuseppe Stanciulescu,
BC Hydro
• Vijay Sukhavasi, Alstom
• Gurudatha Pai, Alstom
• Jay Ramamurthy, Entergy
• Yingchen Zhang, NREL
• Prashant Kansal, AEP
• Harvey Scribner, SPP
• Jay Caspary, SPP
• Xiaoming Feng, ABB
• Floyd Galvan, Entergy
• Naim Logic, SRP
• Alan Engelmann, ComEd
2
Summary
3
This project aims at developing strategies for real-time data quality
management of streaming PMU data.
In this project, we will build a systematic online framework for
identifying and handling typical data quality issues such as clock
errors, transducer errors and network delays.
Based on the synchrophasor data’s spatio-temporal correlations,
the proposed approach is capable of identifying bad data during
both normal and fault-on conditions.
Real-world synchrophasor data as well as synthetic dynamic grid
models will be used to differentiate the root causes of data quality
issues and to validate the proposed strategies.
Project Description - Motivation
4
Current Practice Critical Needs
There is an urgent need to
develop scalable, real-time
methods to monitor and
improve synchrophasor data
quality.
Conventional bad data
detection algorithms are
rendered ineffective, novel
algorithms are needed.
Utilities and vendors are developing more and more synchrophasor-based decision making tools.
Synchrophasor data has much higher sampling rate and accuracy requirement compared with traditional SCADA data.
Typical bad data ratio of synchrophasors in California ISO ranges from 10% to 17% (in 2011) [6].
Project Description - Overview
5
Proposed Approach for Synchrophasor Data Quality Issues
Online Bad Data
Detection Algorithm
Root Cause of Data Quality Issues &
Bad Data Mitigation Strategies
Previous Experience
Recent success of dimensionality reduction of synchrophasor data.
Vast experience in handling real-time data quality.
Technical Approach - Overview
6
Thrust I: Real-time Data Quality
Monitoring via Spatio-temporal
Correlations (Xie, Kumar)
• Quantification of PMU Data Spatio-
Temporal Correlation.
• Development of Computationally
Efficient Metrics of PMU Data
Quality.
• Test of the Algorithm During
Normal and Fault-on Conditions.
Thrust II: First principle-based
Classification and Correction of
PMU Data Quality Issues (Mani, Xie)
• Root-cause
Detection/Classification of Bad
Data.
• Correction of Bad Data.
Data Accessibility
The team has access to more than 5TB of PMU data from the field.
A strong effort will be made to test the proposed algorithms based
on realistic data.
1 PMU Measurement Curve
7
Phase Angle Measured by A Western System PMU for A Recent Brake Test Event
Event Bad DataBad Data
Weak
Temporal
Correlation
Bad DataBad Data
Weak
Temporal
Correlation
Weak
Temporal
Correlation
Weak
Temporal
Correlation
Weak
Temporal
Correlation
2 PMU Measurement Curves
8
Event Bad DataBad Data Bad DataBad Data
Weak
Spatial
Correlation
Weak
Spatial
Correlation
Weak
Spatial
Correlation
Weak
Spatial
Correlation
Strong
Spatial
Correlation
Technical Approach – Thrust I
9
Criteria:
Normal Data VS Bad/Eventful Data
Criteria:
Bad Data VS Eventful Data
For a particular PMU measurement, its
low-quality data segment and fault-on
data segment have weak temporal
correlation with its normal data
segment.
For a particular PMU measurement, its
low-quality data segment has weak
spatial correlation with corresponding
data segments of its neighborhoods.
Its fault-on data segment has strong
spatial correlation with corresponding
data segments of its neighborhoods.
Proposed
Approach
Based on the above criteria, this task will develop an
appropriate algorithm in order to differentiate low-quality
PMU measurements from normal and fault-on PMU
measurements.
Technical Approach – Thrust I
10
Quantify the strength of
spatio-temporal correlations.
Develop an appropriate
distance metric.
Investigate a variance-based
distance metric.
Apply the distance metric to
calculate the local outlier
factor (LOF), which is a
density-based indicator of
data quality.
Use LOF to detect bad data.
Technical Approach – Thrust I
11
Test the algorithm using different types of bad data:
• Gaussian random noises.• Bad data caused by physical instrumentation errors.• Bad data caused by malicious attacks.
Test the algorithm under normal/fault-on system conditions.
Test of the Bad Data Algorithm using Synthetic PMU Data
Verify algorithm performance under practical conditions.
Test the algorithm under normal/fault-on system conditions.
Test of the Bad Data Algorithm using Real-world PMU Data
Technical Approach - Thrust II
12
Contributing Factors to
PMU Data Quality Degradation
• GPS time-lock issues
• Internal clock problems
• Communication network delays
• Incorrect settings at PMU
• Incorrect settings at PDC
• Wiring issues
• Bad transducers
• Others…
II.1: Detection of Data Quality
Issues.
II.2: Correction of Data Quality
Issues.
First principle-based Classification and
Correction of PMU Data Quality Issues
Voltage Phase Angle Example
13
Phase Angle Measured by A Western System PMU for A Recent Brake Test Event
No error flag from the PMU. Data supposedly good.
Spikes real or spurious? Spikes disrupt ringdown/ambient data analysis.
Magnitude comparable to values during event.
Event
? ? ??
Voltage Angle versus Voltage Magnitude
14
No spikes in voltage magnitude data.
Spikes in angle data likely spurious.
Event
? ? ? ?
Vo
lta
ge
Ma
gn
itu
de
Vo
lta
ge
An
gle
Voltage Phase Angle after Data correction
15
Spikes fixed using a smoothing filter
Corrected angle - excellent signal for analysis.
Event
? ? ? ?
Co
rre
cte
d A
ng
leR
aw
An
gle
Technical Approach – Thrust II
16
Task II.1: Data Quality Issues - Detection
Compare local measurements available from the same PMU
(example discussed) or from neighboring PMUs.
Study signatures and develop rules.
GPS clock issues and internal clock issues studied earlier at
ASU and WSU.
Study other data quality contributing mechanisms.
Data “holes” seen in PDCs from communication network
issues or PMU device issues.
Technical Approach – Thrust II
17
Task II. 2: Data Quality Issues - Correction
Once data is deemed to be spurious or unreliable, how to “fix” it?
Smoothing filters from signal processing
Dynamic state estimation using neighbor data
Kalman filter approach
Data interpolation techniques
Study different approaches
Recommend strategies for different applications
Field data from actual PMUs to be used for testing and tuning.
Technical Approach – Summary
18
Online Bad Data
Detection
Bad Data Classification
& Correction
Quantification of PMU
data spatio-temporal
correlation.
Development of
computationally efficient
metrics of PMU data
quality.
Test of the algorithm
during normal and fault-
on conditions.
Root-cause detection &
classification of bad data
Correction of bad data
- Signal processing tools.
- Dynamic state
estimation.
Filling missing data
- Interpolation strategies.
- Dynamic state
estimation.
Implementation Plan
• Formalize the
strategy into a
general formulation.
• Test the methodology
various root-cause
mechanisms.
Systematic comparison of the data under
consideration with other local measurements as
well as neighboring measurements.
Key
Idea
Potential Benefits
19
This project will provide vendors, utilities, and system operators
with the analytical framework and computational algorithms for
real-time cleaning up of synchrophasor data.
It will boost up the level of confidence for PMU-based applications.
A proper handling of these data quality issues is also essential for
designing synchrophasor based closed loop control solutions for
the power industry.
The root-cause analysis will benefit system operators to better
identify the sources and mechanisms in the data acquisition and
communication stages so that they can be better handled in the
future.
Expected Outcomes
20
A technical report that describes the theory, algorithm,
and case studies of the proposed data quality
monitoring and correction methods.
Software modules that are available to industry
members on spatio-temporal correlation-based bad
data detection algorithm and root cause based
detection and mitigation algorithms.
Potential Applications
21
Once successful, this research will contribute to data
quality modules in all PMU-based software.
The application will include computational methods,
root-cause analysis, and the potential visualization of
streaming synchrophasor data quality.
Work Plan
22
Number Task Year 1 Year 2
1 Real-time Data Quality Monitoring via Spatio-temporal Correlations
1.1 Quantification of PMU Data Spatio-Temporal Correlation
1.2 Development of Computationally Efficient Metrics of
PMU Data Quality
1.3 Test of the Algorithm During Normal and Fault-on
Conditions
2 First principle-based Classification and Correction of PMU Data Quality
Issues
2.1 Root-cause Detection/Classification of Bad Data
2.2 Correction of Bad Data
Related Work
23
PSERC project T-43 investigates the interoperability and application performance of PMUs and
PMU-enabled IEDs at the device and system level.
Electric Power Group proposes a three-level architecture to solve PMU data quality problems.
Pre-defined thresholds are suggested for bad data detection [1].
A PMU-based state estimator with the capability of identifying phasor angle bias, current
magnitude scaling, and line parameter errors is presented [2].
Researchers from Virginia Tech proposed a Kalman filter-based approach to cleaning PMU
data. System parameter and topology information is required [3].
Synergistic ideas of using singular value decomposition to detect system physical anomaly
are developed in the context of voltage stability monitoring [4].
Automatic PMU testing tool and PMU bad data detection tools based on state estimation and
data mining techniques are developed [5].
Our approach can effectively identify bad data issues during both normal and fault-on
conditions.
The proposed algorithm involves only simple matrix operations (no matrix inversion or
decomposition needed), making it very suitable for online applications.
The root-cause analysis and correction combines first principles into the data conditioning
process, which provides key insights to utilities and system operators.
Key References
24
[1] K. Martin, “Synchrophasor data diagnostics: detection & resolution of data problems for
operations and analysis”, in Electric Power Group Webinar Series, Jan 2014.
[2] S. Ghiocel, J. Chow, et al. "Phasor-measurement-based state estimation for
synchrophasor data quality improvement and power transfer interface monitoring," IEEE
Tran. Power Systems, 2014.
[3] K. D. Jones, A. Pal, and J. S. Thorp, “Methodology for performing synchrophasor data
conditioning and validation,” IEEE Tran. Power Systems, May 2015.
[4] J. Lim; C. L. DeMarco, "Model-free voltage stability assessments via singular value
analysis of PMU data," in Bulk Power System Dynamics and Control - IX Optimization,
Security and Control of the Emerging Power Grid (IREP), 2013.
[5] Anurag Srivastava, “Meeting PMU data quality requirements for mission critical
applications”, in PSERC Public Webinar Series, Nov 2015.
[6] California ISO, “Five year synchrophasor plan,” California ISO, Tech. Rep., Nov 2011.
[7] L. Xie, Y. Chen, and P. R. Kumar, “Dimensionality reduction of synchrophasor data for
early anomaly detection: linearized analysis,” IEEE Tran. Power Systems, 2014.
[8] M. Wu and L. Xie, “Online identification of bad synchrophasor measurements via
spatio-temporal correlations,” submitted to 19th Power Systems Computation Conference,
Genoa, Italy, 2016.
[9] Q. Zhang, and V. Venkatasubramanian. "Synchrophasor time skew: formulation,
detection and correction," North American Power Symposium (NAPS), 2014.