fatima deliverable d221 v2 2 0 (004)fatima-h2020.eu/wp-content/uploads/2015/07/633945... · d2.2.1:...

D2.2.1: Methodology for dense high-‐resolution EO time series, gap filled WP2.2-‐ Time Series of satellite data from multiple satellites in near real

time

Guido D’Urso, Carlo De Michele (Ariespace)

with inputs from Francesco Vuolo, BOKU and Jesús Garrido, UCLM

This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 633945.

Ref. Ares(2015)5467399 - 30/11/2015

D2.2.1: Methodology for dense high-resolution EO time series, gap filled

27/11/2015 V2.2

2 | 28

Document Information

Grant Agreement Number 633945 Acronym FATIMA Full Title of Project Farming Tools for external nutrient inputs and water Management Horizon 2020 Call SFS-‐02a-‐2014: External nutrient inputs (Research and innovation Action) Start Date 1 March 2015 Duration 36 months Project website www.fatima-‐h2020.eu Document URL (insert URL if document is publicly available online) REA Project Officer Aneta RYNIAK Project Coordinator Anna Osann Deliverable D2.2.1 Methodology for dense high-‐resolution EO time series, gap filled

Work Package WP2.2 – EO for monitoring plant status and yield

Date of Delivery Contractual 30 November 2015 Actual 30 November 2015 Nature R -‐ Report Dissemination Level PU Lead Beneficiary 04_ARIESPACE Lead Author Guido D’Urso (ARIESPACE) Email durso@unina,it

Contributions from

internal Reviewer 1 Ali Gul (EA-‐TEK) Internal Reviewer 2 Nicos Spyropulos (SIGMA) Objective of document To describe the methodology for deriving dense time series from multi-‐

sensors (operated as in a virtual constellation of available satellites). E.O. data and related products (from Vegetation index to canopy parameters) to reduce the impact of noise, cloud cover, missing data (including Landsat7ETM+SLC-‐off) and to derive smooth curves at regular temporal intervals

Readership/Distribution All FATIMA Regional Teams; All WP leaders and other FATIMA team members; European Commission / REA

Keywords Forecast, EO time series, gap filling, remote sensing, monitoring data, crop growth model

Document History

Version Issue Date Stage Changes Contributor 1.0 3/8/2015 draft Main structure and contents G. D’Urso

1.1 16/9/2015 draft New contents F.Vuolo

1.2 18/9/2015 draft Figure and bibliography adjustments

G. D’Urso

2.0 19/11/2015 draft Text revise for crop growth models G. D’Urso

2.1 24/11/2015 draft Text revised for reviewers comments

G. D’Urso

2.2 27/11/2015 Final draft Contribution from Garrido included G.D’Urso


27/11/2015 V2.2

3 | 28

Disclaimer

Any dissemination of results reflects only the authors’ view and the European Commission is not responsible for any use that may be made of the information it contains.

Copyright

© FATIMA Consortium, 2015 This deliverable contains original unpublished work except where clearly indicated otherwise. Acknowledgement of previously published material and of the work of others has been made through appropriate citation, quotation or

both. Reproduction is authorised provided the source is acknowledged. Creative Commons licensing level

Executive summary

This Deliverable aims at describing the methodology for the derivation of dense time series from multi-‐

sensors Earth Observation (EO.) data and related products (from Vegetation Index to canopy parameters)

to reduce the impact of noise, cloud cover, missing data (including Landsat7ETM+SLC-‐off) and to derive

smooth curves at regular temporal intervals. The procedure to be implemented in FATIMA should take into

account current (e.g., Landsat, Spot, Deimos, RapidEye, Formosat, WorldView-‐2) and new platforms

(including Sentinel-‐2). In an operational system, some of the main constraints are the amount of data to

deal with and the management of cloud-‐cover. Previous experience has shown that even with a five-‐ days

period of return, some areas won’t will not receive enough data to follow crop development. Gap filling,

interpolation and above all, extrapolation, while the system is waiting for incoming data, are needed. We

will examine the various aspects of data lacks and gaps and implement procedures to fill them.


27/11/2015 V2.2

4 | 28

Table of Contents Executive summary ........................................................................................................................................... 3

1 Problem statement ................................................................................................................................... 6

2 Methodologies for gap filling and interpolation of existing E.O. data and related products .................... 7

3 Methodologies for the extrapolation of E.O. data and related products (forecast) ............................... 22

3.1 Forecast based on curve fitting ....................................................................................................... 22

3.2 Forecast based on crop growth model ............................................................................................ 23

References ...................................................................................................................................................... 26

List of Tables Table 1 : Summary of methods considered in the present study ..................................................................... 8 Table 2 -‐ Parameters of Whittaker Smoother function .................................................................................. 14

List of Figures Figure 1 -‐ By combining data from different sensors, differently affected by various deteriorating processes, we aim at creating a radiometrically uniform multi-‐spectral product with proper spectral signatures and realistic time profiles. ............................................................................................................... 7 Figure 2 -‐ Test for the interpolation procedure in Castilla-‐La Mancha [31] ............................................... 9 Figure 3 -‐ Comparison between actual and interpolated values of NDVI for an irrigated summer crop and for alfalfa [adapted from 31]. ................................................................................................................... 10 Figure 4 -‐ Dense time serie of NDVI derived with INTERPOLA for an irrigated summer crop [31]. ......... 10 Figure 5 -‐ Temporal profiles of forest raw MODIS NDVI data over 8-‐yeas ............................................... 11 Figure 6 -‐ TIMESAT software with sample-‐time series .................................................................................... 12 Figure 7 -‐ Asymmetric Gaussian function ........................................................................................................ 12 Figure 8 -‐ Savitzky – Golay Filter ..................................................................................................................... 13 Figure 9 -‐ Number of iteration for the upper envelope fitting on MODIS NDVI data (Whittaker filtering by means of R package). ...................................................................................................................................... 16 Figure 10 -‐ Scatter plots of Landsat CDR and surface reflectance data (corrected at BOKU) corresponding to Landsat bands 1–5 and 7 (Blue, Green, Red NIR, SWIR-‐1 and SWIR-‐2, respectively) for a set of satellite observations acquired over different seasons over the Austrian pilot area. The broken lines show the ordinary least squares linear regression fits [26]. ........................................................................................... 17 Figure 11 -‐ Two examples of pixel-‐based Landsat time series for the near-‐infrared spectral band. The red dots represent the high quality observations; the green crosses represent the lower quality observation; the red line is the resulting filtered and gap filled series. ............................................................................... 17 Figure 12 -‐ Times series of smoothed and gap filled data of Marchfeld (Austria) for the year 2009. The temporal resolution between gap-‐filled output data is 15-‐days. ................................................................... 18 Figure 13 -‐ Times series of smoothed and gap filled data of Barrax (Castilla la Mancha, Spain) for the year 2003. The temporal resolution between gap-‐filled output data is 15-‐days. ................................................... 19


27/11/2015 V2.2

5 | 28

Figure 14 -‐ Example of a time series (2009-‐2015, 15-‐days) for four vegetation indices (left) calculated from the smoothed/gap-‐filled reflectance (right, only near-‐infrared band is shown); Austrian pilot area of Marchfeld, one pixel randomly selected representing a forest land cover type. ........................................... 20 Figure 15 -‐ An example of a time series (2002-‐2015, 15-‐days) for four vegetation indices (left) calculated from the smoothed/gap-‐filled reflectance (right, only near-‐infrared band is shown); Spanish pilot area. one pixel randomly selected representing agricultural land cover type ................................................................ 21 Figure 16 -‐ Crop growth and NDVI for wheat (Calera, presentation at Mammamia45 conference, Enschede (NL), June 2015). ............................................................................................................................................. 22 Figure 17 – Linear extrapolation of the crop coefficient Kc(NDVI) and prediction of the reference ET0 from air temperature data [32]. ................................................................................................................................... 23 Figure 18 -‐ Crop growth model schematisation .............................................................................................. 24 Figure 19 -‐Ensemble LAI forecasted with a lead time of 5 days compared with observed values indirectly estimated from VIS-‐NIR satellite images [16] ................................................................................................. 25


27/11/2015 V2.2

6 | 28

1 Problem statement This activity in FATIMA includes the development of the methodology to derive dense time series from

multi-‐sensors (operated as in a virtual constellation of available satellites). E.O. data and related products

(from Vegetation Index to canopy parameters) can be filtered to reduce the impact of noise, cloud cover,

missing data (including Landsat7ETM+SLC-‐off) and to derive smooth curves at regular temporal intervals.

The procedure to be implemented in FATIMA should take into account current (e.g., Landsat, Spot, Deimos,

Geoeye, RapidEye, Formosat, World View 2) and new platforms (including Sentinel-‐2). In an operational

system, some of the main constraints are the amount of data to deal with and the management of cloud-‐

cover. Previous experience has shown that even with a five-‐day period of return, some areas will not

receive enough data to follow crop development. Gap filling, interpolation and above all, extrapolation,

while the system is waiting for incoming data, are needed. The present document will examine various

aspects of data lacks and gaps and implement procedures to fill them.Two main different problems can be

distinguished:

1 gap filling and interpolation of existing E.O. data and related products;

2 extrapolation of E.O. data and related products (forecast).

The problem of type 1), illustrated in Figure 1, has been extensively studied and there are several useful

methodologies that just need to be evaluated in each particular case, ranging from mosaicking to more

complex pixel-‐based compositing and data fusion [17, 18, 19, 20, 22, 23, 24, 25]. The techniques have been

mainly developed for smoothing time series of vegetation indexes such as NDVI, but they can be easily

applied also to surface reflectance data and crop related products, such as LAI, Kc, fractional vegetation

cover. In this case, the same techniques are applied for smoothing and for gap-‐filling problems. In addition,

available techniques do not differ when applied to reflectance data or derived products. However, most of

them have been tested for low-‐medium spatial resolution data i.e. AVHRR, MODIS, Spot Veg., but few

applications exist for finer resolution data (Landsat-‐like or better),

Diversely, the problem of type 2) needs to some extent a forecast of crop growth, and for this reason it can

be more easily afforded by looking at the possible evolution of crop parameters, based on crop growth

models. Forecasting surface reflectance or vegetation indexes is certainly more difficult and inaccurate.

One major output of these activities will be the assessment of uncertainties associated with the elaboration

of each product and the applicability conditions.


27/11/2015 V2.2

7 | 28

Figure 1 -‐ By combining data from different sensors, differently affected by various deteriorating processes, we aim at creating a radiometrically uniform multi-‐spectral product with proper spectral signatures and realistic time profiles.

2 Methodologies for gap filling and interpolation of existing E.O. data and related products

There are several methods of interpolation for time-‐series of vegetation indexes or surface reflectance

values. Four major classes of methods (Table 1) might be considered:

1. slope methods, including the best index slope extraction technique (BISE);

2. filter-‐based methods, including the Savitzky-‐Golay filter technique and its variants, and the mean

value iteration filter;

3. function fitting methods, such as the Asymmetric Gaussian fitting and the harmonic analysis of time

series (HANTS);

4. smoothing techniques, i.e. the Whittaker smoother.

Comparisons of these techniques have been carried out in several case-‐studies, by using different

indicators of performance. Each method has its own advantages and drawbacks, see for example [12]. New

techniques have been proposed in recent years, and in many cases there is not a rigorous comparative

analysis with other techniques. Almost all comparisons have been based on one sensor.

Besides the choice of the algorithm, it is important in the context of FATIMA to consider which tools are

available for performing these analyses. For this reason, within the categories given in Table 1, the

following techniques are considered for operational implementation in the context of FATIMA:


27/11/2015 V2.2

8 | 28

a) Filter techniques based on the Savitzky-‐Golay algorithm and its variants [3];

b) The Whittaker smoother [2].

These methods can be implemented in Matlab and in the open-‐source software package R; a software with

GUI is also available, named TIMESAT, developed by Lund University.

Table 1 : Summary of methods considered in the present study

Category Method Description Reference

1) Slope Interpol. Best-‐Index

slope extraction technique

Compares the current term value with the previous and the next term within a predefined sliding window, and replaces these values with the mean value of the previous and the next values if the percentage difference is greater than a predefined threshold (20%).

[13]

2) Filter based

Savitzky-‐Golay and its variants

Local polynomial fitting of the upper envelope of data series, based on two parameters: the length of the temporal window used and the order of the polynomial. As proposed by Chen et al. (2004), the values of these parameters have to be optimized for each case to get the best match between observations and reconstructed values. In newer variants, the temporal window may be asymmetric and variable in length.

[3]

Mean value iteration

Iteratively compares each date with the average of the dates before and after it, replacing the date with this average if the difference is above a certain threshold. The maximum difference date value will be removed in an iteration process. Iteration will stop when all differences are less than the threshold.

[14]

3) Function

fitting

Asymmetric Gaussian fitting

Fits local, nonlinear functions at intervals around the local maxima and minima, then merges these into a global function describing the full NDVI time series.

[10]

Fast Fourier and Harmonic analysis (HANTS)

Time series are decomposed into sum of sinusoidal functions; once derived phase and amplitudes, these parameters are used for reconstructing and analyzing the data set. [15]

4) Smoothing Whittaker smoother

Based on “penalized” least squares regression, it fits a discrete series to discrete data and penalizes the roughness of the smooth curve. In this way, it balances the reliability of the data and roughness of the fitted data.

[2]

The procedure “INTERPOLA” developed at the University of Castilla-‐La Mancha by Garrido et al. [31] is an

example of the first class of methods “Slope interpolation”. The algorithm is based on the following

equation:


27/11/2015 V2.2

9 | 28

( )( )

( )001

010 xx

xxyyyy −

−

−+=

where : § y: interpolated image pixel value § y0: pixel value on the Julian day x0 (first valid acquired image) § y1: pixel value on the Julian day x1 (second valid acquired image) § x: Julian day of the interpolated image § x0: Julian day of the first valid acquired image § x1: Julian day of the second valid acquired image.

This algorithm can be used to replace clouds or shadows, or an interely missing acquisition, and it can be

applied either to reflectance either to derived products (i.e. Kc). The procedure has been validated by using

Landsat images over an area in Castilla-‐La Mancha, falling within the overlap between the orbits paths no.

199 and 200 (row 33). Hence an interpolated image has been generated between two consecutive Landsat

acquisitions (path 199) and compared, for the overlapping portion, with the acquisition over path 200

(fig.2); hence the time difference (x1-‐x0) was 16 days, and the interpolated image was in the middle of this

interval.

Figure 2 -‐ Test for the interpolation procedure in Castilla-‐La Mancha [31]

The results of the comparison are shown in the plots of fig.3, and they are particularly satisfactory for

irrigated crops like maize (R2=0.90); diversely, cuttings falling between the two acquisitions on days x0 and

x1 introduce a large error in the interpolation. For crops which growth follows a monotonic curve, the

interpolation procedure is able of producing series of images at interval of 7 days or better, as shown in

fig.4 for a typical irrigated summer crop.

yoxo

x

x1

Tiem

po

y1

Análisis y evaluación

y

Imagen real -- (199-033)Imagen real -- (199-033)Imagen interpolada -- (199-033)Imagen real -- (200-033)Zona de solape y validación

yoyoyoxo

x

x1

Tiem

po

xo

x

x1

Tiem

po

y1y1

Análisis y evaluación

yyy

Imagen real -- (199-033)Imagen real -- (199-033)Imagen interpolada -- (199-033)Imagen real -- (200-033)Zona de solape y validación


27/11/2015 V2.2

10 | 28

Figure 3 -‐ Comparison between actual and interpolated values of NDVI for an irrigated summer crop and for alfalfa [adapted from 31].

Figure 4 -‐ Dense time serie of NDVI derived with INTERPOLA for an irrigated summer crop [31].

In the case of long time series, or for crop growth curves which may divert from a monotonic behaviour it is

needed to adopt more complex procedures, included in the categories 2) to 4) of Tab.1.

To this aim, considering the complexity of operations to be performed, two different software packages

can be considered as candidates for utilization in FATIMA.

The first one is TIMESAT, software originally intended for handling noisy time-‐series of AVHRR NDVI data

and to extract seasonality information from the data. The current version of the program has the capability

of handling different types of remotely sensed time-‐series, e.g. data from Terra/MODIS at different time

resolutions. The data analysis can be carried out by means of Savitzky-‐Golay filter (Chen et al., 2004). The

link for downloading the software (registration required) and further descriptions can be found at:

0,1 0,2 0,3 0,4 0,5 0,6 0,7 0,8 0,9

0,1

0,2

0,3

0,4

0,5

0,6

0,7

0,8

0,9

Regadío de Verano

Med

ia N

DV

I, Im

agen

Inte

rpol

ada

para

el 0

1-07

-04,

en

tre la

s im

ágen

es d

el 2

6-06

-04

y 10

-07-

04

Media NDVI, Imagen Real del 01-07-04Media NDVI, Imagen Real

Med

ia N

DV

I, Im

agen

Inte

rpol

ada

0,1 0,2 0,3 0,4 0,5 0,6 0,7 0,8 0,9

0,1

0,2

0,3

0,4

0,5

0,6

0,7

0,8

0,9

Regadío de Verano

Med

ia N

DV

I, Im

agen

Inte

rpol

ada

para

el 0

1-07

-04,

en

tre la

s im

ágen

es d

el 2

6-06

-04

y 10

-07-

04


Med

ia N

DV

I, Im

agen

Inte

rpol

ada

0,1 0,2 0,3 0,4 0,5 0,6 0,7 0,8 0,9

0,1

0,2

0,3

0,4

0,5

0,6

0,7

0,8

0,9

Alfalfa

Med

ia N

DV

I, Im

agen

Inte

rpol

ada

para

el 0

1-07

-04,

en

tre la

s im

ágen

es d

el 2

6-06

-04

y 10

-07-

04


Med

ia N

DV

I, Im

agen

Inte

rpol

ada

0,1 0,2 0,3 0,4 0,5 0,6 0,7 0,8 0,9

0,1

0,2

0,3

0,4

0,5

0,6

0,7

0,8

0,9

Alfalfa

Med

ia N

DV

I, Im

agen

Inte

rpol

ada

para

el 0

1-07

-04,

en

tre la

s im

ágen

es d

el 2

6-06

-04

y 10

-07-

04


Med

ia N

DV

I, Im

agen

Inte

rpol

ada

0

0,1

0,2

0,3

0,4

0,5

0,6

0,7

0,8

0,9

0 50 100 150 200 250 300 350DoY

Val

ores

de

ND

VI

NDVI Real - R.VeranoNDVI Interpolado - R.Verano


27/11/2015 V2.2

11 | 28

http://web.nateko.lu.se/timesat/timesat.asp

It is possible to load into TIMESAT time series of image data and plot temporal profiles pixel by pixels as

shown in Figure 5.

Figure 5 -‐ Temporal profiles of forest raw MODIS NDVI data over 8-‐yeas

The interface of TIMESAT appears as follows; the first step is the removal of cutoffs (spikes) has shown in

the Figure 6. Spikes and outliers removal is important to avoid seriously degrading in the final function fits.

In the interface box, the low amplitudes means that only time series with an amplitude higher than a

certain value are processed. This makes sure that uninteresting time series with low variation were not

processed.

The Seasonal fit means the choice of the function or filter technique. They are the Double Logistic or

asymmetric Gaussian (AG) functions (Figure 7) and the Savitzky-‐Golay (SG) filter (Figure 8).


27/11/2015 V2.2

12 | 28

Figure 6 -‐ TIMESAT software with sample-‐time series

Figure 7 -‐ Asymmetric Gaussian function


27/11/2015 V2.2

13 | 28

Figure 8 -‐ Savitzky – Golay Filter

Envelope adaption involves the fitting of the data. No. of fitting steps means the adaption to the upper

envelope, which is realistic because most noise in NDVI data is, even for clear data, negatively biased. It was

set to ‘3’. Adaption strength is set to ‘1’ and the SG window size: ‘4 5 6’ (the higher the numbers the greater

the gliding window, meaning an increased smoothing but probably less accuracy).

It should be noted that the filtering parameters, once defined, are fixed for the entire image.

The second procedure is based on the R software package, which is a programming environment for

statistical computing and graphics, providing also a wide variety of techniques, including filtering and

smoothing. R is available as Free Software under the terms of the Free Software Foundation’s GNU General

Public License in source code form. The installation package can be downloaded from the following link:

http://www.inside-‐r.org/download/cran

R includes among others the Whittaker smoother. Reference and sample script can be found at:

http://www.inside-‐r.org/packages/cran/pracma/docs/whittaker

It has been shown that Whittaker performs well against a number of other filters based either on curve

fitting or Fast Fourier transform (FFT) [5] and it can be considered as the method allows filtering data

without information on pixel quality [1,2]. In [1], it was shown that this filter permits a significant increase

in the signal-‐to-‐noise ratio (SNR) of coarse resolution VI time series. Besides VI usefulness information, the

smoother also takes into account land/water mask layers of the VI Quality Assessment Science Data Set [9].

This method is based on two assumptions [3]:


27/11/2015 V2.2

14 | 28

i. that the time-‐series of vegetation index follows an annual cycle of growth and decline as the index

is primarily related to vegetation density and plant vigor;

ii. that clouds and poor atmospheric conditions produce a negative bias in the vegetation index

values, requiring that sudden drops in vegetation index, which are not compatible with the gradual

process of vegetation change, are regarded as noise and will be removed.

The Whittaker smoother family was firstly presented by Whittaker in 1923 for life tables, based on

penalized least squares. These ideas were revived by Paul Eilers, Leiden University, in 2003. This approach is

also known as Whittaker-‐Henderson smoothing.

Whittaker smoother is based on penalized least squares, fits a discrete series to discrete data and penalizes

the roughness of the smooth curve. In this way, it balances the reliability of the data and roughness of the

fitted data. The smoother takes a time series of observations together with some parameters and outputs

the filtered time series. Whittaker filter for smoothing multi-‐temporal satellite sensor observations with the

ultimate purpose of deriving an appropriate annual vegetation growth cycle and estimating phenological

parameters reliably.

Diversely from other methods, the Whittaker adapt the filtering to each single pixel within the image,

thus providing the maximum adaptability to image itself.

Table 2 -‐ Parameters of Whittaker Smoother function

Lambda smoothing parameter. Allowed are (non-‐integer) values > 0 (scalar)

Weights weights (0 ... 1) used for weighting the time series to be filtered

n°iter scalar, indicating the number of iterations to be performed

Order scalar (integer) order of differences (default = 2).

min val scalar, indicating which values in the input time series are valid

max val scalar, indicating which values in the input time series are valid

min length

minimum number of valid observations in for that a filtering is performed

miss val scalar, used to indicate which values in the input time series are invalid


27/11/2015 V2.2

15 | 28

BOX 1: Main Parameters of the Whittaker Smoother in R

The smoothing parameter (λ)

The smoothing parameter (λ) of the Whittaker smoother determines the roughness of the smoothed curve. For the back-‐processing can be fixed, after some trial-‐and-‐error tests, to a constant value for the entire study region; acceptable considering balancing fidelity to the input EO data with the roughness of the resulting curve.

Quality Flags

As already applied with MODIS NDVI data, the operational filtering procedures of high spatial resolution data can take advantage of the quality flag during smoothing, since this information is available for Landsat Level-‐2A product (Landsat 4-‐5 Thematic Mapper (TM), Landsat 7 Enhanced Thematic Mapper Plus (ETM+) and Landsat 8 Operational Land Imager (OLI)) generated by the Landsat Ecosystem Disturbance Adaptive Processing System (LEDAPS)) and for Landsat Level-‐1T product (Landsat 8). Pixels having a VI usefulness value lower than three were considered to be acceptable and assigned a weight of one (very good to good quality), while a VI usefulness larger than seven was excluded from further processing with a weight of zero (not acceptable). VI usefulness values between three and seven were linearly scaled between one and zero.

Numbers of Iterations

Indicating the number of iterations to be performed. To further reduce the possible impact of undetected clouds and poor atmospheric conditions, three filtering iterations were performed to fit the upper envelope of the VI. Multiple filter runs were for example recommended by [8,10].

Similar to [11], two filtering iterations can be performed to fit the VI data to the upper envelope. Iterative filtering to the upper envelope is recommended, as undetected clouds and poor atmospheric conditions decrease the observed VI.

Order difference

This parameter of the Whittaker smoother can be set after some tests to two. Based on the order difference, the smoother calculates the roughness of the smoothed curve [1].


27/11/2015 V2.2

16 | 28

An example of application of the Whittaker filter is shown is fig.9 for MODIS NDVI time series. Preliminary

analyses were also carried out with surface reflectance data and they showed satisfactory results. In

particular, the filter was applied on a time series of the Landsat Surface Reflectance Climate Data Record

archive for some test sites in Austria, Spain and Italy. Landsat CDR is a Landsat Level-‐2A product generated

by the Landsat Ecosystem Disturbance Adaptive Processing System (LEDAPS) [21]. The data set includes

atmospherically corrected (BOA) Landsat 4-‐5 Thematic Mapper (TM), Landsat-‐7 Enhanced Thematic

Mapper Plus (ETM+) and Landsat-‐8 Operational Land Imager (OLI) data at global level. These sensors have

identical spatial resolutions and a comparable spectral resolution. The Landsat CDR data have

demonstrated to be as accurate as a currently available atmospherically corrected surface reflectance

products that can be obtained using an industrial-‐standard radiative transfer model of the atmosphere (e.g.

ATCOR-‐2) in combination with a detailed study, performed by a trained operator, of the atmospheric

conditions at the time of each satellite acquisition [26]. In Figure 10 a comparison is shown between

Landsat CDR and surface reflectance data independently corrected by using a manual fine-‐tuning of ATCOR-‐

2 parameters to reach the highest possible accuracy.

A similar procedure can be adopted for other sensors and for time series of different sensors, once a cross-‐

calibration has been performed and all image have the same geographical projections, geometrical

resolution and dimensions (number of rows and columns).

Training session might be organised within FATIMA in order to practice the proposed procedures and to

standardise them for the different datasets.

Figure 9 -‐ Number of iteration for the upper envelope fitting on MODIS NDVI data (Whittaker filtering by means of R package).


27/11/2015 V2.2

17 | 28

Figure 10 -‐ Scatter plots of Landsat CDR and surface reflectance data (corrected at BOKU) corresponding to Landsat bands 1–5 and 7 (Blue, Green, Red NIR, SWIR-‐1 and SWIR-‐2, respectively) for a set of satellite observations acquired over different seasons over the Austrian pilot area. The broken lines show the ordinary least squares linear regression fits [26].

Figure 11 shows two examples of a pixel-‐based filtering of surface reflectance of Landsat CDR data for the

near-‐infrared band for the period 2009-‐2015.

Figure 11 -‐ Two examples of pixel-‐based Landsat time series for the near-‐infrared spectral band. The red dots represent the high quality observations; the green crosses represent the lower quality observation; the red line is the resulting filtered and gap filled series.


27/11/2015 V2.2

18 | 28

The approach was applied to all spectral bands to derive RGB colour composites and various vegetation

indices. An example of RGB (Near-‐infrared, Red, Green) is shown in Figure 12 for the Austrian test site of

Marchfeld for the year 2009 (temporal resolution of gap-‐filled data is 15-‐days) and in Figure 13 for the

Spanish test site of Barrax for the year 2003. Tested vegetation indices included NDVI, NDWI, fAPAR and

Tasselled Cup Transformation (TCB). The temporal profile for two exemplary pixels is shown in Figure 14

(from the Austrian dataset) and Figure 15 (from the Spain dataset ), along with the correspondent

reflectance (raw and smoothed/gap-‐filled data) in the near-‐infrared band only.

Figure 12 -‐ Times series of smoothed and gap filled data of Marchfeld (Austria) for the year 2009. The temporal resolution between gap-‐filled output data is 15-‐days.


27/11/2015 V2.2

19 | 28

Figure 13 -‐ Times series of smoothed and gap filled data of Barrax (Castilla la Mancha, Spain) for the year 2003. The temporal resolution between gap-‐filled output data is 15-‐days.


27/11/2015 V2.2

20 | 28

Figure 14 -‐ Example of a time series (2009-‐2015, 15-‐days) for four vegetation indices (left) calculated from the smoothed/gap-‐filled reflectance (right, only near-‐infrared band is shown); Austrian pilot area of Marchfeld, one pixel randomly selected representing a forest land cover type.


27/11/2015 V2.2

21 | 28

Figure 15 -‐ An example of a time series (2002-‐2015, 15-‐days) for four vegetation indices (left) calculated from the smoothed/gap-‐filled reflectance (right, only near-‐infrared band is shown); Spanish pilot area. one pixel randomly selected representing agricultural land cover type


27/11/2015 V2.2

22 | 28

3 Methodologies for the extrapolation of E.O. data and related products (forecast)

3.1 Forecast based on curve fitting

As mentioned in the introduction, in the context of applications regarding crop management, the

extrapolation of data (forecast) is possible with regards to the crop parameters i.e. crop coefficients Kc,

fractional vegetation cover, height, Leaf Area Index, based on crop growth models. Crop growth is usually

described by smooth curves of known shape (Figure 16). In particular, it is well known that NDVI follows the

same shape of the crop coefficient Kc. It is possible to establish from actual EO data the current point along

the ideal Kc curve, and forecast the possible evolution for the considered crop. This approach can be used in

conjunction with forecast of reference ET0 to predict crop water requirements for the incoming 5-‐7 days.

In case of a plant growth diverting from the ideal curve, due to water or nutrient stresses, and such

occurrence is evidenced by the first available acquisition, it would be needed to adopt appropriate scaling

procedure of the ideal Kc curve in order to continue correctly the same fitting method. Alternatively,

complex growth model might be applied (see following section). In this case, assimilation or forcing

techniques to integrate E.O data products into the model should be adopted.

Figure 16 -‐ Crop growth and NDVI for wheat (Calera, presentation at Mammamia45 conference, Enschede (NL), June 2015).


27/11/2015 V2.2

23 | 28

The method of linear extrapolation of Kc(NDVI) has been applied in a case-‐study in the Castilla La-‐Mancha

region [32]. In this case, the reference ET0 was predicted by using forecast air temperature data for a

period of 7 days and the Hargraves-‐Samani formula. A reanalysis carried out with actual meteorological

data on a weekly basis, a very good agreement was found between the predicted ET0 and the

corresponding calculated by means of the standard FAO Penman-‐Monteith equation (Figure 17).

Figure 17 – Linear extrapolation of the crop coefficient Kc(NDVI) and prediction of the reference ET0 from air temperature data [32].

3.2 Forecast based on crop growth model

An alternative -‐ physically based -‐ procedure could be represented by the implementation of a crop model,

to predict its growth. A simplified crop growth model with the aim to assess the biomass and LAI growth at

daily time step scale can be used for this purpose. In this class of models the main biophysical processes are

conceptualized by a set of simplified analytical relations. A typical approach is the 3PG [30] where the net

primary production (NPP) is modeled according to a light-‐use efficiency approach with a constant carbon

use efficiency factor, similarly to other popular biomass growth models. In short, the LAI dynamics is based

on temperature and leaf dry matter supply, driven by the development stage of the crop. The conceptual

scheme is depicted in Figure 18. In most crop growth models, the main biophysical processes are

conceptualized by a set of simplified analytical relations, with the aim to assess the biomass and LAI growth

with a given time step i.e. daily. The net primary production (NPP) is modelled according to a light-‐use

efficiency approach; the LAI dynamics is based on temperature and leaf dry matter supply, driven by the

development stage of the crop.


27/11/2015 V2.2

24 | 28

Figure 18 -‐ Crop growth model schematisation

Within the crop growth model, there is an uncertainty related the spatial distribution of soil properties,

initial soil conditions, crop parameters, meteorological forcings. This uncertainty majorly influences the

simulation of two important physiological processes:

1 the simulation of crop canopy development, which determines light interception and

photosynthetic potential;

2 the simulation of soil moisture content, which determines the actual evapotranspiration and the

reduction of photosynthesis as a result of drought stress.

Because the crop growth process has inherent errors, including the errors on initial and boundary

conditions, the forward simulation of the crop model would result in an increasingly enlarged differences

between the simulated and observed results.

E.O. derived LAI can be coupled with crop growth models according to different strategies, i.e. by means of

an Ensemble Kalman Filter. In this case, a recursive Bayesian ensemble-‐based filter estimates the state

variable of a dynamic system from a series of noise corrupted measurements, to mitigate modeling

uncertainty, i.e. to update model state predictions. In this way the model state variables are continuously

updated when remote sensing information is available.

An example of application of a stochastic model is shown in Figure 19, where the assimilation has been

implemented for simulating the growth of the crop above ground biomass and LAI, starting from its seeding

[16]. In Figure 19 it is also possible to notice that the uncertainty on LAI decreases when more observations

are available, thus improving the estimates for the next 5 days.


27/11/2015 V2.2

25 | 28

Figure 19 -‐Ensemble LAI forecasted with a lead time of 5 days compared with observed values indirectly estimated from VIS-‐NIR satellite images [16]

Other more sophisticated model can be used while maintaining the same approach, for example:

• CGMS: The Crop Growth Monitoring System (operational crop yield forecasting)

• WOFOST: Within CGMS a version of the WOFOST crop model is implemented, which has been

adapted to the applications at European scale in the Agri4cast EU action.

• CropSyst: A multi-‐crop model for growth simulations, currently applied e.g. in climate change

impact studies

• STICS: internationally recognised as a dynamic, generic and robust model aiming to simulate the

soil-‐crop-‐atmosphere system, developed by INRA [27, 28 29]

• EPICS: crop model developed by Texas A & M Univ. initially to to estimate soil productivity as

affected by erosion and further on expanded to predict effects of management decisions on soil,

water, nutrient and pesticide movements.

The utilisation of these models is required to adequately consider the management of nutrients. For the

implementation and available software, the reader should follow the hyperlinks provided above.

100 150 200 250 3000

0.5

1

1.5

2

2.5

3

3.5

DOY

LAI

Observation5th day forecast


27/11/2015 V2.2

26 | 28

References

[1] Atzberger, C. and Eilers, P. H. C., "Evaluating the effectiveness of smoothing algorithms in the absence of ground reference measurements", International Journal of Remote Sensing, 32(13), 3689-‐3709, 2011.

[2] Eilers P. H. C., “A perfect smoother,” Analytical Chemistry, vol. 75, no. 14, pp. 3631–3636, 2003.

[3] Chen, J.; Jönsson, P.; Tamura, M.; Gu, Z.H.; Matsushita, B.; Eklundh, L. A simple method for reconstructing a high-‐quality NDVI time-‐series data set based on the Savitzky–Golay filter. Remote Sens. Environ. 2004, 91, 332–344.

[4] Mattiuzzi ,M., Verbesselt ,J., Hengl ,T., Klisch , A., Evans,B. & Lobo ,A., 2012b: MODIS download and processing package. Processing functionalities for (multi-‐temporal) MODIS grid data. R package version 0.5-‐3/r276. http://R-‐Forge.R-‐project.org/projects/modis (2.7.2014).

[5] P. M. Atkinson, C. Jeganathan, J. Dash, and C. Atzberger, “Inter-‐comparison of four models for smoothing satellite sensor time-‐series data to estimate vegetation phenology,” Remote Sensing of Environment, vol. 123, pp. 400–417, Aug. 2012.

[6] C. Atzberger and P. H. C. Eilers, “A time series for monitoring vegetation activity and phenology at 10-‐daily time steps covering large parts of South America,” International Journal of Digital Earth, vol. 4, no. 5, pp. 365–386, 2011.

[7] Vuolo, F.; Mattiuzzi, M.; Klisch, A.; Atzberger, C. Data service platform for MODIS vegetation indices time series processing at BOKU Vienna: Current status and future perspectives. Proc. SPIE 2012, 8538, doi:10.1117/12.974857.

[8] Beck P. S. a., C. Atzberger, K. A. Høgda, B. Johansen, and A. K. Skidmore, “Improved monitoring of vegetation dynamics at very high latitudes: A new method using MODIS NDVI,” Remote Sensing of Environment, vol. 100, no. 3, pp. 321–334, Feb. 2006.

[9] Solano, R.; Didan, K.; Jacobson, A.; Huete, A. MODIS Vegetation Index User’s Guide (MOD13 Series); Version 2.00, May 2010 (Collection 5); The University of Arizona: Tucson, AZ, USA, 2010.

[10] Jönsson, P.K.; Eklundh, L. Seasonality extraction by function fitting to time series of satellite sensor data. IEEE Trans. Geosci. Remote Sens. 2002, 40, 1824–1832.

[11] Rembold, F.; Atzberger, C.; Savin, I.; Rojas, O. Using low resolution satellite imagery for yield prediction and yield anomaly detection. Remote Sens. 2013, 5, 1704–1733

[12] Hird, J.N., McDermid, G.J. (2009): Noise reduction of NDVI time series: An empirical comparison of selected techniques. Remote Sensing of Environment 113, 248–258.

[13] Viovy, N.; Arino, O.; Belward, A. The Best Index Slope Extraction (BISE): A method for reducing noise in NDVI time-‐series. Int. J. Remote Sens. 1992, 13, 1585–1590

[14] Ma, M.M.; Veroustraete, F. Reconstructing pathfinder AVHRR land NDVI time-‐series data for the Northwest of China. Adv. Space Res. 2006, 37, 835–840

[15] Roerink, G.J.; Menenti, M.; Verhoef, W. Reconstructing cloudfree NDVI composites using Fourier analysis of time series. Int. J. Remote Sens. 2000, 21, 1911–1917

[16] Pelosi A., Falanga S., De Michele C., Medina Gonzalez H., Villani P., D’Urso G., Chirico G.B. 2015. A stochastic ensemble-‐based model to predict crop water requirements from numerical weather


27/11/2015 V2.2

27 | 28

forecasts and VIS-‐NIR high resolution satellite images in Southern Italy. EGU General Assembly 2015 , Vienna

[17] Bielski, C., Grazzini, J., & Soille, P. (2007). Automated morphological image composition for mosaicing large image data sets. In International Geoscience and Remote Sensing Symposium (IGARSS) (pp. 4068–4071). doi:10.1109/IGARSS.2007.4423743

[18] Eivazi, A., Kolesnikov, A., Junttila, V., & Kauranne, T. (2015). Variance-‐preserving mosaicing of multiple satellite images for forest parameter estimation: Radiometric normalization. ISPRS Journal of Photogrammetry and Remote Sensing, 105, 120–127. doi:10.1016/j.isprsjprs.2015.03.007

[19] Griffiths, P., van der Linden, S., Kuemmerle, T., & Hostert, P. (2013). A Pixel-‐Based Landsat Compositing Algorithm for Large Area Land Cover Mapping. Selected Topics in Applied Earth Observations and Remote Sensing, IEEE Journal of, PP(99), 1–14. doi:10.1109/JSTARS.2012.2228167

[20] Hermosilla, T., Wulder, M. a., White, J. C., Coops, N. C., & Hobart, G. W. (2015). An integrated Landsat time series protocol for change detection and generation of annual gap-‐free surface reflectance composites. Remote Sensing of Environment, 158, 220–234. doi:10.1016/j.rse.2014.11.005

[21] Masek, J. G., Vermote, E. F., Saleous, N. E., Wolfe, R., Hall, F. G., Huemmrich, K. F., Lim, T. K. (2006). A landsat surface reflectance dataset for North America, 1990-‐2000. IEEE Geoscience and Remote Sensing Letters, 3(1), 68–72. doi:10.1109/LGRS.2005.857030

[22] Roy, D. P., Ju, J., Kline, K., Scaramuzza, P. L., Kovalskyy, V., Hansen, M., Zhang, C. (2010). Web-‐enabled Landsat Data (WELD): Landsat ETM+ composited mosaics of the conterminous United States. Remote Sensing of Environment, 114(1), 35–49. doi:10.1016/j.rse.2009.08.011

[23] Vanonckelen, S., Lhermitte, S., & Van Rompaey, A. (2013). The effect of atmospheric and topographic correction methods on land cover classification accuracy. International Journal of Applied Earth Observation and Geoinformation, 24(1), 9–21. doi:10.1016/j.jag.2013.02.003

[24] Xu, L., Li, B., Yuan, Y., Gao, X., & Zhang, T. (2015). A Temporal-‐Spatial Iteration Method to Reconstruct NDVI Time Series Datasets. Remote Sensing, 7(7), 8906–8924

[25] Zhu, Z., Wang, S., & Woodcock, C. E. (2015). Improvement and expansion of the Fmask algorithm: cloud, cloud shadow, and snow detection for Landsats 4–7, 8, and Sentinel 2 images. Remote Sensing of Environment, 159, 269–277. doi:10.1016/j.rse.2014.12.014

[26] Vuolo, F., Mattiuzzi, M., & Atzberger, C. (2015). International Journal of Applied Earth Observation and Geoinformation Comparison of the Landsat Surface Reflectance Climate Data Record ( CDR ) and manually atmospherically corrected data in a semi-‐arid European study area. International Journal of Applied Earth Observations and Geoinformation, 42, 1–10. doi:10.1016/j.jag.2015.05.003

[27] Brisson N., Mary B., Ripoche D., Jeuffroy M.H., Ruget F., Gate P., Devienne-‐Barret F., Antonioletti R., Durr C., Nicoullaud B., Richard G., Beaudoin N., Recous S., Tayot X., Plenet D., Cellier P., Machet J.M., Meynard J.M., Delecolle R., 1998, STICS: a generic model for the simulation of crops and their water and nitrogen balance. I. Theory and parametrization applied to wheat and corn. Agronomie 18, 311-‐346


27/11/2015 V2.2

28 | 28

[28] Brisson N.,Ruget F.,Gate P., Lorgeou J., Nicoullaud B., Tayot X., Plenet D., Jeuffroy M.H., Bouthier A., Ripoche D., Mary B., Justes E., 2002, STICS: a generic model for the simulation of crops and their water and nitrogen balances. II. Model validation for wheat and corn. Agronomie, 22, 69-‐93

[29] Brisson N., Gary C., Justes E., Roche R., Mary B., Ripoche D., Zimmer D., Sierra J., Bertuzzi P., Burger P., Bussiere F., Cabidoche Y.M., Cellier P., Debaeke P., Gaudillere J.P., Maraux F., Seguin B., Sinoquet H., 2003, An overview of the crop model STICS. Eur. J. Agron. 18, 309-‐332

[30] Landsberg J.J., Waring R.H., 1997. A generalised model of forest productivity using simplified concepts of radiation-‐use efficiency, carbon balance and partitioning. Forest Ecology and Management, 95: 209–228

[31] Garrido-‐Rubio, J., Campos, I., Villa, G., Plaza, N., Calera, A., 2011. INTERPOLA. Herramienta para la generación de imágenes sintéticas y sustitución de valores reales con presencia de nubes-‐sombras por valores interpolados, XIV Congreso de la Asociación Española de Teledetección, Mieres, Asturias, Spain, p. 4

[32] Garrido-‐Rubio, J., González, L., Madurga del Cura, C., Arellano, I., Navarro Comalrena de Sobregau, M., López Tapia, J., Calera, A., 2015. Predicción de las necesidades hídricas con una semana de antelación mediante teledetección y meteorología en la finca experimental CENTER para cultivos de Maíz, XXXIII Congreso Nacional de Riegos, Valencia, p. 10

fatima deliverable d221 v2 2 0 (004)fatima-h2020.eu/wp-content/uploads/2015/07/633945... · d2.2.1:...

Documents