rfi status: where and how to deal with it. lisbon, 11 … · rfi status: where and how to deal with...

1

RFI Status: Where and How to Deal with it.Lisbon, 11 - 13 March 2009

N. Skou, J. Balling, S. S. Søbjærg,

and S. S. Kristensen

National Space Institute

Technical University of Denmark

[email protected]

2

EMIRAD Radiometer System

• L-band, 1400 - 1427 MHz

• Fully polarimetric system

• Digital radiometer with subharmonic sampling

• Digital processing in FPGA - 2!nd and 4!th order moments of PDF are

calculated

– i.e. Kurtosis is calculated

• Data integrated to 8 msec (from 2008: 1 msec) and recorded

• Fast data integrated to 1.8 µsec (from 2008: 14.7 µsec) and recorded

• 2 antennas

– nadir

– 40 deg aft

3

EMIRAD on HUT Skyvan

4

Approaches to Detecting RFI

1. Time domain – look for pulses

2. Frequency domain – look for carrier frequencies

3. Amplitude domain – look for non-thermal distribution

Thermal waveformSinusoidal waveform

Gaussian pdfNon-Gaussian pdf

5

Brief Theory of Kurtosis Operation

• Desired radiometric (science) signals are noise-like

– Gaussian probability distribution function (PDF)

• RFI is man-made

– PDFs will be non-Gaussian in general

• Exploiting this distinction is the basis for RFI flagging

• Underlying Statistics:

– all higher-order moments of a Gaussian are uniquely determined byits lowest two moments

– the second central moment is nothing but the brightnesstemperature of the scene

– the ratio equals 3 for a Gaussian PDF

– or to say it differently: the moment ratio - Kurtosis - is 3 for naturaltargets and typically significantly different for RFI signals

– Threshold = Kurtosis mean + 4 times Kurtosis std. dev.

!

k =< x4 (t) >

< x2 (t) >2

6

Kurtosis Values and Blind Spot

Kurtosis is:

! = 3 for Gaussian

! > 3 for pulse

! = 1.5 for CW

! = 3 for 50% duty cycle

1

2

3

4

5

6

7

1 2 3 4 5 6 7 8 9 10 11

T

msec

K

K

TB

TBTB

TB + RFITB + RFI

7

Overview of CoSMOS Campaigns

Campaign: Where: When: What:

CoSMOS - Aus Australia Fall 05 Land

CoSMOS - OS Norway Spring 06 Ocean

POL-ICE Finland Winter 07 Ice

Demonstrator Finland Summer 07 Ocean

Rehearsal Germany - Spain Spring 08 Land

8

Flight Pattern off Norway

9

Example: RFI Seen Directly on TB

10

RFI Detected by Kurtosis

11

Detailed View: One 8 msec Frame - Heavily Contaminated

12

Coastal Crossing: Kurtosis Independent From TB

13

RFI Percentages, North Sea

Date Aft H Aft V Nadir H Nadir V

6/4 - 06 0.44% 0.53% 0.31% 0.49%

9/4 - 06 0.06% 0.37% 0.06% 0.44%

10/4 - 06 0.03% 0.36% 0.05% 0.43%

12/4 - 06 0.04% 0.31% 0.04% 0.36%

13/4 - 06 0.57% 1.02% 0.87% 1.16%

15/4 - 06 0.05% 0.61% 0.06% 0.72%

16/4 - 06 1.97% 2.44% 3.37% 1.52%

18/4 - 06 0.13% 0.46% 0.06% 0.56%

19/4 - 06 0.70% 1.45% 2.34% 0.90%

22/4 - 06 35.9% 41.6% 43.9% 18.0%

25/4 - 06 0.05% 0.27% 0.06% 0.29%

29/4 - 06 31.2% 35.4% 53.3% 18.7%

30/4 - 06 0.99% 0.28% 0.44% 0.60%

14

RFI Percentages, Australia and Finland


15/11 - 05 2.99% 0.83% 15.0% 2.58%

27/11 - 05 2.26% 0.55% 17.4% 2.69%

29/11 - 05 2.94% 0.78% 16.0% 1.22%

3/12 - 05 5.44% 5.70% 30.0% 5.34%

6/12 - 05 4.03% 2.10% 21.8% 4.97%


8/3 - 07 0.86% 1.6% 0.59% 0.81%

11/3 - 07 0.07% 0.16% 0.12% 0.10%

12/3 - 07 10.8% 21.5% 8.6% 20.8%

13/3 - 07 0.17% 0.22% 0.12% 0.09%

13/8 - 07 0.08% 0.50% 0.07% 0.56%

13/8 - 07 0.23% 3.8% 0.61% 0.46%

15/8 - 07 0.22% 0.42% 0.11% 0.38%

15

RFI Percentages, München and Valencia


8/4 - 08 2.74% 2.80% 1.79% 2.70%

14/4 - 08 1.66% 1.82% 1.70% 1.89%

18/4 - 08 4.16% 3.57% 3.46% 3.51%


22/4 - 08 3.28% 3.35% 1.82% 1.88%

24/4 - 08 8.93% 4.64% 3.34% 2.59%

28/4 - 08 4.50% 2.87% 2.39% 2.31%

2/5 - 08 34.6% 30.5% 38.5% 32.9%

16

RFI Percentages, Transit Back Home

Where? Aft H Aft V Nadir H Nadir V

Valencia –

Casablanca

42.4% 18.1% 10.7% 23.3%

Casablanca -

Marseille

33.2% 24.3% 20.63% 28.32%

Marseille - Dole 14.1% 17.7% 10.7% 12.9%

Dole - Luxembourg 7.09% 5.67% 4.20% 4.53%

Luxembourg -

Wurstenbach

0.11% 0.19% 0.04% 0.17%

Wurstenbach -

Paderhorn

3.83% 4.89% 3.23% 4.15%

Paderhorn - Kalmar 3.47% 2.63% 1.57% 1.75%

Kalmar - Helsinki 0.57% 0.73% 0.32% 0.42%

17

Impact of RFI Corruption in Terms of TB(K)

• München, 14 April

• Nadir horn, V-pol. as example

• Appr. 1.9 % data flagged

• An RFI pixel is about 800 x 800 m, and corresponds to 7000 1 msecsamples

18

RFI Flags Near München

19






• Using the 1 msec data, flagged samples are removed from the 7000samples that make up 1 RFI pixel in the map, and the “true” TB valuecalculated. Then the impact of the RFI is found.

20

RFI Distribution According to Impact on TB

21






• Using the 1 msec data, flagged samples are removed from the 7000samples that make up 1 RFI pixel in the map, and the “true” TB valuecalculated. Then the impact of the RFI is found.

• Most of the flagged RFI is below 1 K

• Much is actually below 0.1 K

• Significant amount in the 1 - 10 K region

• This is very difficult to detect using conventional threshold algorithms!

• Can however be done with very fast sampling rate (RFI of pulsed nature)

22

München - Overview

23

München - “DLR” Airport and Vicinity

24

München - Campaign Area

25

Valencia - Overview

26

Valencia - Airport and Industrial Area

27

Valencia - Campaign Area

28

Vercors

29

Marseille

30

Montelimar

31

Dijon

32

Troyes

33

Lübeck

34

Kurtosis Discussion

• In general only a few percent of data samples are flagged

• South France much worse, 15%! - or more (scale dependent), gradually

back to “normal” going north

• Significant fraction close to airports and cities - away from research areas.

– fine for airborne campaigns

– but for space systems!!?!

• Significant fraction - but surely not all - of flagged data only contribute

insignificantly to TB. This is especially true for land applications.

• Kurtosis is one method for RFI detection with its own advantages andproblems. Especially, there is a blind spot for 50% duty cycle signals!

• Kurtosis seems to work very well over North European seas

• Problems have been reported in München and Valencia: some clear andlarge signals are not flagged by kurtosis!??

• The kurtosis can be regarded as an offer: the user can always take the TBoutput and handle RFI by traditional means - Not recommended though!

35

Polarimetric Data

• Natural targets have very small 3!rd and 4!th Stokes

• Linearly polarized RFI normally not aligned with H and V of ourinstrument, hence we get 3!rd Stokes

• Many surveillance radars use circular polarization, hence we get 4!thStokes

• Experience with EMIRAD shows that often Kurtosis flagged data has

significant signals in 3!rd and 4!th Stokes, but not always. The oppositecan also be the case.

• Subject for further investigations

• Anyway, looking for signals in the 3!rd and 4!th Stokes channels ofSMOS can be an important method for RFI detection

36

Fast Data Example

• The 8 msec data sample has been flagged by Kurtosis

• All Stokes parameters of one 8 msec interval are shown

• Beware large !T due to minute integration time

• To the right of the H and V curves is shown the offset in K due to the RFIwhen integrated over 8 msec.

37

Fast Data Example (H and V polarization)

3.6K

7.5K

38

Fast Data Example – (3!rd and 4!th Stokes)

39

3!rd and 4!th Stokes used as RFI Indicators

• Compare with kurtosis

• Assume kurtosis is the “truth”

• Assume 3!rd and 4!th Stokes = 0 over natural targets

• If parameters are larger than prescribed value, then flag as RFI

• This value is horizontal axis

40


41


• Compare with kurtosis

• Assume kurtosis is the “truth”

• Assume 3!rd and 4!th Stokes = 0 over natural targets

• If parameters are larger than prescribed value, then flag as RFI

• This value is horizontal axis

• Set prescribed value = 10 K

• Catch 60% of kurtosis flagged data - the “low” value is not surprising:we know that about half of kurtosis flagged data have very low impacton TB and this may be what we see here

• Discards 5% of “clean” data

• Might these 5% actually be RFI not flagged by kurtosis?

• Proposal: a combination of kurtosis and Stokes > 10 K might be apowerful RFI detection tool (yet still simple and safe to calculate).

• This is under investigation.

42

Conclusions

• RFI is generally there

• RFI is very variable in nature, time, and space

• DTU flights have generally experienced 1% RFI flagging over ruralresearch areas - no problem for science data when you know

• But for airborne instruments RFI flagging seems a must!

• The traditional search for unusual, large TBs is difficult and unreliable

• Kurtosis flagging seems all in all to work very reliably

• Some additional cleaning might be needed

• Alternative methods:

– very fast sampling and recording (in this case 1.8 µsec) seems very

powerful for most (pulsed) RFI

– signatures in the 3!rd and 4!th Stokes channels seems to indicate RFI

– a combination of kurtosis and the above is being investigated.

– the kurtosis algorithm itself is also under investigation ( furtherintegration of kurtosis, different calculation time intervals).

rfi status: where and how to deal with it. lisbon, 11 … · rfi status: where and how to deal with...

Documents