AUTOMATED QUALITY CONTROL OF GEOPHYSICAL TIME SERIES
The algorithmic systems developed at Geophysical Center of Russian Academy of Sciences are intended for recognition of disturbances with defined morphology on time series. These algorithms were applied to 1-minute and 1-second INTERMAGNET data for recognition of artificial disturbances on the magnetograms. INTERMAGNET network is the basis for geomagnetic field monitoring so requirements for reliability of collected data are very high. Therefore, an important task is an objective and formalized recognition and further elimination of possible anthropogenic anomalies in data records.
Fuzzy comparisonson positive numbers
Nearness in finitemetrical space
Limit in finitemetrical space
Densityas measureof limitness
Smoothingtime series
Monotonoustime series
Fuzzy logicand geometry on
time series:geometricmeasuresPredication
of time series.Forecast
Anomalies ontime series.
DRAS. FLARS. FCARS
Extremumson time series
Convex time series
Separation of denseSubset.
Crystal. Monolith
Clusterization.Rodin
Search of linearStructure.
Tracing
Morphological time series
analysis
Morphological correlation
Stochasticcorridors
Multidimensial time series
analysis
1999-2008
2009
2010-
Discrete Mathematical Analysis (DMA) Scheme
A spike is a chain of interrelated singular record fragments representing disturbances that are substantial vertically and insignificant horizontally and that do not lead to a shift of the recording level [Bogoutdinov et al., 2010].
Artificial disturbances on geomagnetic records
Example of spike recognition on 1-minute data (FRD, X, 2005)
Jump recognition on 1-minute data
Jump recognition on satellite magnetic data (GOES, 2 Hz)
The results of training and testing show that SP, SPs and JM algorithms are efficient enough to recognize almost all artificial spikes and jumps detected by data experts manually. This also provides the possibility to carry out retrospective analysis and quality control of the magnetograms available at ICSU World Data Centers.
SP algorithm block scheme
( , ) max | ( ) ( ) |: , [ , ]yO t y t y t t t t t
Brute-force search of free parameter values: 4 600 sets of values
Spike recognition on 1-minute INTERMAGNET magnetograms
A.A. Soloviev1, A. Chulliat2, R.V. Sidorov1, Sh.R. Bogoutdinov1
1-Geophysical Center RAS, Moscow, Russia; 2 – Insitiut de Physique du Globe de Paris, France
SP algorithm recognition results
Jump recognition on INTERMAGNET magnetograms
SPs recognition statistics
Sh.R. Bogoutdinov, A.D. Gvishiani, S.M. Agayan, A.A. Solovyev, E. Kihn, Recognition of Disturbances with Specified Morphology in Time Series. Part 1: Spikes on Magnetograms of the Worldwide INTERMAGNET Network, Izvestiya, Physics of the Solid Earth, 2010, Vol. 46, No. 11, pp. 1004–1016 A. Soloviev, A. Chulliat, S. Bogoutdinov, A. Gvishiani, S. Agayan, A. Peltier, B. Heumez (2012), Automated recognition of spikes in 1 Hz data recorded at the Easter Island magnetic observatory, Earth Planets Space, Vol. 64 (No. 9), pp. 743-752, 2012, doi:10.5047/eps.2012.03.004
1( , )
a b
b an a b
, where 0
A fuzzy comparison ( , )n a b of non-negative numbers a and b measures in the scale segment [ 1,1] the rate of superiority of “b ” over “a ”: segment [ 1,1] the rate of superiority of “b ” over “a ”:
If 1 20 NA a a a is a finite set and 0b , then
( , ) ( ) [ 1,1]n a b es a b .
( , )( ) ( , ) [ 1,1]i in a b
es A b n A bN
Example of spike recognition on 1-second data
Comparison with classical methods
SPs algorithm block schemeSpike recognition on 1-second magnetograms
Comparison with F methodComparison with statistical
algorithms
Components XYZ and intensity F
PeriodNumber
of obser-vatories
Events recognize
d
Target miss
False alarm
Learning 2007 7 290 0% 5.2%Exam 1 2008 5 110 1.0% 8.2%Exam 2 2003* 7 1032 0% 15.4%Exam 3 2005* 7 535 0.2% 14.6%
* Increased magnetic activity
Jump is an anomaly on a record leading to its baseline shift.
(rinf [ , ] lsup [ , ], rsup [ , ] linf [ , ])[ , ]
(linf [ , ] rsup [ , ],lsup [ , ] rinf [ , ])
n y a b y a b y a b y a bjmesA a b
n y a b y a b y a b y a b
1 2[ , ] ( )( ) [ , ] arg min [ , , ]
def
a b T Tj A a b jmes y a b
[ , ] 0.5jmesA a b
JM Algorithm: Calculating measures of jumpiness using fuzzy bounds
where rinf, linf are the fuzzy lower bounds, rsup, lsup are the fuzzyupper bounds,
Potential jump (red), wings (green), fuzzy bounds (black)
References:
[Soloviev et al., 2012]
~ 140 days
Natural geomagnetic pulsations on 1-second data