research highlights
Missing data recovery of large volumes of PMU data by exploiting
low-dimensional structures
by Dr. Joe Chow ([email protected]) and Dr. Meng Wang ([email protected])
Data losses happen due to either device malfunction or communication errors. The recovery of missing
PMU (Phasor Measurement Unit) data is an important preprocessing step for other monitoring and control
tasks such as state estimation and disturbance detection. We propose new online and offline data recovery
methods by collectively processing measurements from multiple PMUs simultaneously. We verify the
effectiveness of our proposed methods on PMU data from New York ISO (NYISO) and New York Power
Authority (NYPA). We can accurately recover the missing data of 53 PMUs in historical NYISO datasets, some
of which contain 10-15% data loss.
The central idea of our proposed methods for data recovery is to exploit the low-dimensional structures
of spatial-temporal PMU data blocks. Because PMU measurements are sampled at synchronized time
instants, and the measurements of nearby PMUs are correlated through the power system topology, the
high dimensional PMU data exhibits a coherence property. If measurements of multiple PMU channels are
represented by a matrix with each row representing the measurements of one channel across time, then the
matrix only contains a small number of significant singular values.
Leveraging the aforementioned approximate low-rank property of PMU data, we connect the problem of
missing PMU data recovery with recent advances in low-rank matrix completion method. The low-rank
property of data blocks has been studied in other applications, and various low-rank matrix completion
algorithms have been proposed. We proposed an online algorithm that can fill in the missing PMU
measurements for real-time applications and tested our methods on historical PMU data from NYISO. The
computational time for the 5-minute snapshot with 30 samples/second of PMU voltage magnitude data
(9000 by 53 matrix) is only about 6 seconds using our developed online algorithm for PMU data processing
OLAP algorithm (Intel i7-4770 with 12 GB RAM). Figure 1 compares the recovery performance of multiple
recovery methods, including our developed OLAP algorithm and existing methods such as singular value
thresholding (SVT) and information cascading matrix completion (ICMC). All three methods have similar
performance on this dataset and can correctly recover the missing points with negligible error.
Figure 1: Missing PMU Data Recovery by different recovery methods
Figure 2 shows the recovery performance on historical PMU data in New York State. All the missing points are
recovered with negligible error.
3
Figure 2: Missing data recovery of historical PMU data in New York State