validation reportco-resyf.eu/wordpress/wp-content/uploads/2019/01/... · project start date:...

Validation Report | Research Applications

PUBLIC of 41

Coastal Waters Research Synergy Framework

Validation Report

Research Applications

Document Code: CORESYF-ACR-VVR-STR03-E-R

Date of delivery: SAR-2 meeting (T0+33) Deliverable identifier: D4.08 Version of document: 2.0 – last updated 5/11/2018 Dissemination level for document: PU

Table of Signatures

Name Function Signature

Prepared by

Romain Serra Anne Vallette

Nadim Dayoub Helen Snaith

Chiara Pratola Rory Scarrott

Walther Camaro Eimear Tuohy

Francisco Sancho Alberto Azevedo Florent Birrien

Ana Moura Nuno Grosso

ACRI-HE ACRI-HE

NOC NOC UCC UCC UCC UCC LNEC LNEC LNEC

IH Deimos

Reviewed by

Romain Serra

Anne Vallette

ACRI-HE, Application

Developer

Approved by Miguel Terra-Homem Executive Board Chair

Signatures and approvals appear on original


PUBLIC of 41

Project start date: 01/01/2016

Project duration: 36 months

Revision Records

Version Date Changes Authors

1.0 14/11/2017 First issue of document

Romain Serra Nadim Dayoub Paolo Cipollini Rory Scarrott Eirini Politi Eimear Tuohy Francisco Sancho Alberto Azevedo Ana Moura Nuno Grosso

1.1 29/11/2017 Minor additions to optical bathymetry acceptance tests results, §3.2 (comments added in steps 15 and 16).

Romain Serra

2.0 5/11/2018 Second issue of document Romain Serra Anne Vallette


PUBLIC of 41

Table of Contents

1 Introduction ............................................................................................................................. 5

1.1 Purpose and Scope .......................................................................................................... 5

1.2 Document Structure ........................................................................................................ 6

2 ACCEPTANCE TESTS RESULTS .................................................................................................. 6

3 Acceptance Test Procedures ................................................................................................... 7

3.1 Bathymetry determination from SAR imagery ................................................................ 8

3.2 Bathymetry, benthic classification and water quality determination from optical

imagery ...................................................................................................................................... 14

3.3 Vessel detection & Oil Spill Detection with SAR & Optical ............................................ 20

3.4 Time series processing for hyper temporal optical data analysis ................................. 27

3.5 Ocean and Coastal altimetry ......................................................................................... 32

4 References ............................................................................................................................. 40

List of Tables

Table 2-1: Test case to Test procedure traceability ........................................................................ 6

List of Figures

Aucune entrée de table d'illustration n'a été trouvée.


PUBLIC of 41

Acronyms and Abbreviations

AIS Automatic Identification System

AOI Area Of Interest

BBP Backscattering coefficient of particulates

BOA Bottom of Atmosphere

CDOM Coloured Dissolved Organic Matters

Co-ReSyF Coastal Waters Research Synergy Framework

DEM Digital Elevation Model

ECMWF European Centre for Medium-Range Weather Forecasts

EMSA European Maritime Safety Agency

EO Earth Observation

FFT Fast Fourier Transform

GUI Graphical User Interface

IH Instituto Hidrografico, Portugal

LNEC Laboratório Nacional de Engenharia Civil, Portugal

NE North East

NOC National Oceanography Centre, UK

NW North West

QGIS Geographic Information Systems Software

RGB Red Green Blue composite (i.e. in true colours)

RMSE Root Mean Square Error

SAR Synthetic aperture radar

SAR-1, SAR-2 System Acceptance Reviews

SE South East

SGDR Sensor Geophysical Data Records

SNAP SentiNel Application Platform

SSH Sea Surface Height

SSHA Sea Surface Height Anomaly

SST Sea Surface Temperature

SW South West

SWH Significant Wave Height

TS Time Series

UCC University College Cork, Ireland

UT Untested

VTP Validation Test Procedures

VTS Validation Test Specifications


PUBLIC of 41

1 Introduction The Co-ReSyF project will implement a dedicated data access and processing infrastructure,

with automated tools, methods and standards to support research applications using Earth

Observation (EO) data for monitoring of Coastal Waters, levering on the components deployed

SenSyF (www.sensyf.eu). The main objective is to facilitate the access to Earth Observation data

and pre-processing tools to the research community, towards the future provision of future

Coastal Waters services based on EO data.

Through Co-ReSyF‘s collaborative front end, even inexperienced researchers in EO will be able

to upload their applications to the system to compose and configure processing chains for easy

deployment on the cloud infrastructure. They will be able to accelerate the development of

high-performing applications taking full advantage of the scalability of resources available in the

cloud framework. The system’s facilities and tools, optimized for distributed processing, include

EO data access catalogues, discovery and retrieval tools, as well as a number of pre-processing

tools and toolboxes for manipulating EO data. Advanced users will also be able to go further and

take full control of the processing chains and algorithms by having access to the cloud back-end,

and to further optimize their applications for fast deployment for big data access and

processing.

The Co-ReSyF capabilities will be supported and initially demonstrated by a series of early

adopters who will develop new research applications on the coastal domain, guide the

definition of requirements and serve as system beta testers. A competitive call will be issued

within the project to further demonstrate and promote the usage of the Co-ReSyF release.

These pioneering researchers in will be given access not only to the platform itself, but also to

extensive training material on the system and on Coastal Waters research themes, as well as to

the project's events, including the Summer School and Final Workshop.

1.1 Purpose and Scope The main aim of this document is to check the correct implementation of the first version of the

Research Applications, demonstrating that the requirements have been satisfied.

Within the Co-ReSyF project, five research applications are being developed and deployed for

operation onto the Co-ReSyF platform:

i) Bathymetry using SAR data,

ii) Bathymetry using optical imagery,

iii) Oil Spill and Vessel detection using SAR and optical data,

iv) Time series processing for hyper temporal optical data analysis, and

v) Coastal Altimetry.

This document focuses solely on the V1 requirements of the Co-ReSyF Research Applications,

and presents the results of the acceptance tests, based on the plan defined in the frame of the

Co-ReSyF Validation Plan for Research Applications (D3.04).


PUBLIC of 41

1.2 Document Structure The structure of the document is as follows:

Chapter 2: describes the Validation and Acceptance approach.

Chapter 3: describes the Results of the Acceptance Tests executed.

Chapter 4: details the Reference Documents.

2 ACCEPTANCE TESTS RESULTS The plan for the acceptance of the software version is fully described in (Co-ReSyF, 2016c). The

following section presents the test procedures executed during the acceptance tests in order to

execute the test cases for V1 as defined in (Co-ReSyF, 2016c). The following table traces the

test procedures for each of the test cases.

Table 2-1: Test case to Test procedure traceability

VTS No. VTP Id. APPLI1-VTS-001 to

APPLI1-VTS-007 APPLI1-VTP-001

APPLI2-VTS-001 to APPLI2-VTS-006

APPLI2-VTP-001


APPLI3-VTP-001


APPLI4-VTP-001


APPLI5-VTP-001


PUBLIC of 41

3 Acceptance Test Procedures Each acceptance test procedure covers one or more test cases and their respective requirements, and has a unique ID in the form of APPLI#-VTP-xxx (where xxx is a sequential number). The possible results for a test step are mainly PASS or FAIL. However, due to the various degrees of maturity of the research applications, some could not be integrated in the Co-ReSyF platform, at the time of release of this deliverable. In this case, the test was performed offline, the test step result was noted “UT” for “untested online in the Co-ReSyF platform” and offline results are presented in the procedure comments as much as possible. These will be further detailed in the final version of this document (V2). In the case that the step is verifying a requirement, the Pass/Fail criteria will need to be specified (this is mandatory) and will be used to assess the result of the step. On cases where no P/F criteria is specified the result of the step is assessed by the ability to execute the step.


PUBLIC of 41

3.1 Bathymetry determination from SAR imagery

TEST PROCEDURE

VTP Id.: APPLI1-VTP-001 Id of Associated VTS:

APPLI1-VTS-01

APPLI1-VTS-02

APPLI1-VTS-03

APPLI1-VTS-04

APPLI1-VTS-05

APPLI1-VTS-06

APPLI1-VTS-07

Functionality to be Tested: This procedure tests the implementation and working functionality of the SAR-Bathymetry application

Required Test Environment: Desktop with a command line interface

Overview of the test procedure: The application was not still integrated in the platform, although specific module tests have been performed offline in a desktop environment using a command line interface. Thus, the following reports refer when applicable to the platform tests or to the offline environment tests.

Detailed description of the test procedure, including how to observe and verify the results:

Step Nb.

Description Result VTS Verified P/F Criteria Comments

1

Test the application as a whole. The aim is to check if everything is running without error and a bathymetry map is delivered.

FAIL APPLI1-VTS-01

The outputs of all processing steps shall be georeferenced maps, loadable in QGIS or SNAP software (GeoTIFF format)

The application has been integrated into the platform, but it fails running for a region with more than approximately 25 grid-points. Also, the GeoTIFF output format was not implemented..


PUBLIC of 41

Step Nb.


2

Check of the external wind

velocity at the centre point of

each SAR image. PASS APPLI1-VTS-02

Wind velocity should be comprised between 3 and 10 m/s for an image to be a good candidate for the application of the proposed methodology.

The module is running and gathers data from the ECMWF or the NCEP re-analysis databases. The module calculates the wind at the position closest to that at the centre of each image, for the KOMPSAT-5 images. The procedure remains to be tested for SENTINEL-1 images and other possible SAR image sources (e.g. TERRASAR-X). However, this module is not integrated in the running platform version.

3 Check external wave data for

each SAR image date PASS APPLI1-VTS-03

Existence of a well-defined ocean swell with one predominant direction and wavelengths adequate for the application of the method.

For the test region (the West coast of Portugal), a procedure was defined to extract data from an online available numerical model forecast (which can be used as hindcast) at http://ariel.lnec.pt/. Also, wave-related in situ past data have been collected from the Hidrográfico database, from the Leixões buoy, and made available for this project for the period 2002-2016. An algorithm was developed by Hidrografico in order to attribute to each wave condition a pass/fail criterion, based on the measured wave period and spectral width, which indicated the likelihood of the presence of a visible swell for a candidate image. However, this module is not integrated in the running platform version.

http://ariel.lnec.pt/


PUBLIC of 41

Step Nb.


4 Check of image selection

protocol FAIL APPLI1-VTS-04

1. The SAR image is visualized to verify the existence of wind shadow areas, in the bathymetry AOI. 2. The presence of swell is verified visually. 3. The presence of strong wave diffraction or wave refraction zones near the shore should be verified. 4. Other features are checked, such as the presence of different signatures like oil spills, vessels, internal waves, river plumes or other coastal processes that could affect the algorithm application.

This functionality was not implemented.


PUBLIC of 41

Step Nb.


5

Radiometric, speckle, sigma0 and crop and creation of the Bathymetry AOI with– visual check

PASS APPLI1-VTS-05

1. The polygon of the AOI

must be closed and

properly defined.

2. The grid size must be

within a default range,

depending on image

resolution.

3. The image within the AOI shall show clear wave crest patterns

For a given application, a North-South oriented rectangular polygon needs to be created as an area of interest (AOI). If this rectangular AOI includes land points, these are masked and excluded from the computation. It is expected that this algorithm evolves and allows for another closed-polygon shape that may adjust better to any given coastal configuration. The procedure also includes the SAR image calibration georeferencing and filtering. This step is incomplete, as remains the need to perform the flip or ellipsoid-correction. The grid size is presently defined by the user, within a default range given by the algorithm, which depends on the image resolution. This has been tested for SENTINEL-1 images. The version of the currently available processing modules does not mask the land pixels or perform the necessary ellipsoid correction and therefore the test of this module has failed. The other two functions (radiometric correction and speckle filtering) are already implemented and are working correctly.


PUBLIC of 41

Step Nb.


6

Determination of the FFT

boxes for each point of the

final grid PASS APPLI1-VTS-06

1. The image is cropped in

N FFT boxes which are

dependent of the

resolution of the grid.

2. Check if the wavelength

is comprised between a

pre-defined interval.

3. Check if the direction of the swell is comprised between an expected interval, from the visual inspection (APPLI1 -VTS-04).

The computation grid definition at the AOI is computed automatically, once the AOI and grid size are defined. For each grid point, a squared (box) sub-scene or image tilling with a user-defined size is cropped from the main image, centred at the grid point location. This is referred to as the “FFT boxes definition”. This box is further shifted (or linearly translated) a pre-defined spatial-lag length, along all the 4 main cardinal directions, and the four intermediate directions (NW, NE, SE, SW). This produces nine FFT boxes around each grid point. It follows the computation of the wave spectrum, through an image Fast Fourrier Transformation (FFT), yielding the peak wave length and wave direction at all the 9 FFT boxes, for each grid point. Several tests have been performed and several algorithms were developed to determine the peak wavelength and wave direction, using 1D and 2D filtering techniques and spectral estimates. The average of the valid results is then used as the grid-point wave length to compute the local bathymetry. The deep-water wave lengths are determined in order to retrieve the local wave peak period for each image AOI. Several methods to determine the wave length, based on 2D, and multiple 1D spectral estimates have been tested.


PUBLIC of 41

Step Nb.


7 Test of the depth inversion

algorithm PASS APPLI1-VTS-07

1. Creation of the

bathymetry map. The

estimated depths must be

coherent, particularly at

adjacent grid points.

2. Validation of the final

outputs with in situ data.

3. The outputs of all processing steps shall be georeferenced maps.

For a given image, areas (grid points) where the APPLI1 -VTS-06 protocol fails are flagged out from the bathymetry computation. These areas appear in maps as without a valid bathymetry, and are marked visually by a different colour. Validation of the final bathymetry outputs with field data has not been done yet, due to the lack of proper validation data.

8

Comparison of output with

EMODNET and local

soundings. PASS

Comparison of

output with

EMODNET and

local soundings.

The comparison of outputs with the EMODNET chart for the AOI is implemented. However, this functionality was not implemented in the running platform version.

Date of execution: 5/11/2018


PUBLIC of 41

3.2 Bathymetry, benthic classification and water quality determination from optical imagery

TEST PROCEDURE

VTP Id.: APPLI2-VTP-001 Id of Associated VTS: APPLI2-VTS-01

APPLI2-VTS-02

APPLI2-VTS-03

APPLI2-VTS-04

APPLI2-VTS-05

APPLI2-VTS-06

Functionality to be Tested: The good performance of the Optical Bathymetry application.

Required Test Environment: Co-ReSyF online platform available, with a valid user. No special input needed, Co-ReSyF catalogue is sufficient.

Overview of the test procedure: Test if the whole processing chain delivers a bathymetry map as expected. If this test fails, there must be an error so looking into other tests is important in order to find the failure.


Step Nb.



PUBLIC of 41

Step Nb.


1

Test the application as a whole. The aim is to check if everything is running without error and a bathymetry map is delivered

PASS APPLI2-VTS-01

The outputs of all processing steps shall be georeferenced maps (GeoTIFF format)

The application is running as expected, without any error. A GeoTIFF showing the resulting bathymetry is delivered. As GeoTIFF is not fitted for presentation display, a JPEG is also produced, showing the assessed bathymetry.

This test indicates a good general behaviour of the application as it provides the expected result as an output.


PUBLIC of 41

Step Nb.


2 Test if images are corrected from Rayleigh scattering and

gaseous absorption. PASS APPLI2-VTS-02

1. Source images contain land and water pixels

2. Atmospheric correction provides bottom of atmosphere data in the Area Of Interest

3. The lowest quality pixels are flagged (clouds, shadows)

The atmospheric correction algorithm developed by ACRI searches for water and land pixel to find a dark target and correct the atmospheric effects. Scenes not providing this target (because of absence of water, presence of cloud…) are not treated.

The others are treated and corrected. As an output of the application, intermediate images are delivered in order to evaluate the processing over the scene. For most modules, a before/after study can be done so the user can understand how the correction works and why it is necessary.

For example, images of a Before/After atmospheric correction are delivered such as the ones below (Sentinel2, over Venice area, Italy) and can be used to visually assess the quality of the correction or the presence of artefacts or of sun glint over water (like visible on the eastern part of these images).


PUBLIC of 41

Step Nb.


3

Test if external bathymetry data is available and

associated with each satellite image considered

NA APPLI2-VTS-03

The Co-ReSyF application allows access to bathymetry data available on the area of interest, and when possible at a date close to the one of the considered satellite image used.

Pairing In-situ bathymetry data with the area of Interest was useful for the V1 as the bathymetric model required in-situ data for calibration. For V2, the bathymetric model is autonomous and is only based on water colour. So even if this module was already integrated (test was PASSED for V1), it has finally been decided to remove this module as it is finally useless.

4 Water Masking PASS APPLI2-VTS-04 1. A Water mask covers all the water areas.

The water masking is running well and produce a GeoTIFF with masking non water pixels. Only remain the water pixels that can be assessed for bathymetry retrieval. The following example shows quite well how water pixels are successfully distinguished thanks to the mask.


PUBLIC of 41

Step Nb.


5 Bathymetry and water quality

inversion PASS APPLI2 -VTS-07

1. The combined inversion maps (water depths, chlorophyll-a, bbp and CDOM) have continuous values at the limits of two clusters. If the analysis shows spatial heterogeneity over the map, the inversion process shall be adapted (e.g. change number of cluster).

2. The areas with water depths deeper than 15 meters are not considered (flagged).

The water depth (top left), Albedo (top right), CDOM (middle left), Chl-A (middle right) and BBP (bottom) are adjusted in the model to fit satellite radiometry. Areas not suitable for bathymetry retrieval are masked on the Water Depth map.


PUBLIC of 41

Step Nb.


6 Water level correction PASS APPLI2 -VTS-06 1. Bathymetry maps are corrected according to the modelled tidal level

Depending on the time and location, the tide level is retieved thanks to FES2014 model. A chart displays the tide level during the sensing day and the accurate tide level at the exact time of the satellite image. This tide level is then added to the Water Depth data and so the final Bathymetry is obtained.

7 Averaged bathymetry and

estimate of variability FAIL APPLI2 -VTS-07

This step provides the median value of all the bathymetry results (and associated standard deviation) available over the same area of interest during a period shorter than six months.

It finally appeared that averaging dozens of bathymetry maps is not relevant or reliable enough as each of the maps can be affected by clouds, sun glint or other undesirable effect that lead to erroneous synthetic data. It has been decided not to integrate that module.



PUBLIC of 41

3.3 Vessel detection & Oil Spill Detection with SAR & Optical

TEST PROCEDURE


APPLI3-VTS-01

APPLI3-VTS-02

APPLI3-VTS-03

APPLI3-VTS-04

APPLI3-VTS-05

APPLI3-VTS-06

APPLI3-VTS-07

APPLI3-VTS-08

APPLI3-VTS-09

APPLI3-VTS-10

APPLI3-VTS-11

Functionality to be Tested: The good performance of the Vessel and Oil Spill Detection application through a python environment

Required Test Environment: Co-ReSyF online platform available, with a valid user. No special input needed, Co-ReSyF catalogue is sufficient. Overview of the test procedure: Test if each step of the vessel detection and oil spill processing chain is working as expected.. Successful requirements of the application are outlined for each test in the comment section. For the Oil spill detection algorithm some of the steps in the processing chain were already tested and accepted in the work developed in another project. Those steps are marked as PASS. Detailed description of the test procedure, including how to observe and verify the results:

Step Nb.



PUBLIC of 41

Step Nb.


1 Pre-processing analysis PASS APPLI3-VTS-01

All necessary pre-processing steps are executed within the application

Raw SAR image is successfully calibrated, despeckled, georeferenced and land is masked out. The user is able to define the conditions for the pre-processing analysis through the portal.

Sentinel 1 – RAW Image

Ellipsoid Correction - GG

Land Mask + Calibration + Speckle-Filter


PUBLIC of 41

Step Nb.


2 Select Area of Interest (AOI) PASS APPLI3-VTS-02 Pre-processed image is cropped to meet user defined AOI.

Input required – User defined AOI. Image is successfully cropped to geographical specifications

FORMAT: 'POLYGON ((minEst minNorth, minEst maxNorth, maxEst maxNorth, maxEst minNorth, minEst minNorth))‘


PUBLIC of 41

Step Nb.


3 Application of ship detection algorithm

PASS APPLI3-VTS-03

Vessels detections outputted as GeoTiff, which includes detection parameters & metadata

Vessel detections successfully outputted internally as a GeoTiff. The user is able to define the minimum and maximum target size through the portal.

4 Merging image tiles UT APPLI3-VTS-04

Generation of original AOI image from all image tiles used for parallel processing

Currently untested. Changes were done to the research application and the current version does not separate image into tiles.


PUBLIC of 41

Step Nb.


5 Shapefile generation PASS APPL13-VTS-05

Generation of shapefile (.shp) with the date and time of the SAR image acquisition.

The user is able to download an output shapefile through the portal.

6 All UT APPLI3-VTS-06 GeoTiff output and AIS match-up output

Currently untested because no AIS data in included in the platform. AIS match up to be confirmed. AIS data access and application to be confirmed.

7 Pre-processing analysis (optical)

UT APPLI3-VTS-07

Optical images are atmospherically corrected and the land /clouds are masked out

Currently untested in the Co-ReSyF platform. The platform will not provide an algorithm for oil spill detection in optical imagery.


PUBLIC of 41

Step Nb.


8 Application of oil spill detection algorithm (SAR)

PASS APPLI3-VTS-08

Detection of dark features on the sea surface and classify as oil spills or look-alikes

Tested in the Co-ReSyF platform but still requires to be tested in the Co-ReSyF platform. See examples of oil spill detected in image below.


PUBLIC of 41

Step Nb.


9 Application of oil spill detection algorithm (optical) UT APPLI3-VTS-09

Separation of oil spills from water or other ocean features is performed on pixel-based spectral signatures

Currently untested in the Co-ReSyF platform. The platform will not provide an algorithm for oil spill detection in optical imagery

10 Auxiliary data shapefile generation PASS APPLI3-VTS-10

Query based on satellite image (SAR and optical) date/time acquisition (allowing user-defined temporal window) generates list of auxiliary data that are suitable for match-up pairs

Tested in the scope of the SenSyF in a cloud environment similar to the Co-ReSyF platform but still requires to be tested in the Co-ReSyF platform (no AIS data available in the platform). See of detected oil spill shapefile/kml example below


PUBLIC of 41

Step Nb.


11 All UT APPLI3-VTS-11

Output data format ((i) AOI SAR and optical images in GeoTIFF format, (ii) binary (merged) result over the AOI in shapefile format (SAR), (iii) binary (merged) result over the AOI in shapefile format (optical), (iv) auxiliary match-up dates/times shapefile)

Currently only tested for SAR imagery in the Co-ReSyF platform. See example images below of shapefile and GeoTiff oil spill detection.


3.4 Time series processing for hyper temporal optical data analysis

TEST PROCEDURE

VTP Id.:

APPLI4-VTP Id of Associated VTS:

APPLI4-VTS-01

APPLI4-VTS-02

APPLI4-VTS-03

APPLI4-VTS-04

APPLI4-VTS-05

APPLI4-VTS-06

APPLI4-VTS-07

Functionality to be Tested: Operations of the application and comparative success to desktop-based tools.

Required Test Environment: Testing the various requirements involves the use of desktop software (ArcMap 10.0) to cross-compare desktop and platform outputs, ensuring values and spatial locations at times and in output datasets are the same.


PUBLIC of 41

Overview of the test procedure: The whole processing chain has been run on a desktop-compiled datacube of 5-day interval SST data for 2011. The output dataset is prepared and ready to be compared to an output dataset derived using the Co-ReSyF system, using the same input SST datafiles.

The confirmation of statistical patterns (using truly random numbers) for the process to build upon is reported on in the ATBD.

The defined processing chain from datacube to Ocean-surface Heterogeneity Map is detailed here, with the verification steps required that it is operating. The validation and product characterisation efforts have been detailed in the ATBD v2.3.

Note that as of November 4th, 2018, the full suite of hypertemporal component modules have not yet been integrated into the Co-ReSyF platform. Integration is ongoing at time of writing. Once the process (and component modules) has been integrated into the WINGS development platform, the verification tests below can be implemented. As such, all tests are currently marked as UT (Untested) in the results column. Furthermore, due to the focus on testing the process in the WINGS environment, and the revised process itself (see ATB V2.3), a number of requirements have been rendered redundant (indicated by N/A (No longer Applicable) in the results section. Progress towards achieving the test has been outlined in the comments section. An updated version of section will be submitted in due course before the final review.


Step Nb.

Description Result Requirement(s) Verified

P/F Criteria Comments

1 Ingestion into the system of a 3-d datacube consisting of cleaned level 4 data to be processed.

UT APPLI4-VTS-01

Updated

Multi-layer dataset successfully uploaded into the WINGS application development portal.

Currently untested due to research application not yet being integrated into Platform.

Progress made:

WINGS development portal for testing uploaded modules and dataset upload functionalities completed. The Classification modules is also compiled, and awaiting upload into the development area, upon which time the 3D dataset can be uploaded and used to test whether the upload has been completely successful


PUBLIC of 41

Step Nb.



2 Verification that the data cube generated, contains the correct data images, in the correct sequence.

N/A APPLI4-VTS-02 Independent desktop verification that each image has successfully been integrated into the data cube, in the correct order, with no loss of the required data values for further processing.

No longer applicable - Step has been bypassed by enabling upload of user-built datacubes into the WINGS environment.

3 Verification that the randomised image data, is generated within the statistical parameter range constraints sourced from the data cube

N/A APPLI4-VTS-03 Independent desktop verification that each image’s values lie within the range allowed by the statistical constraints.

No longer applicable – Research outlined in the ATBD has shown that the classification algorithm (in particular the separability analysis steps) are not operable on classifications of truly randomised data. If the process works, it is because the data contain real patterns.

4 Independent Verification that the 91 cluster outputs extracted from the data cube by the Co-ReSyF system are equivalent to those extracted by the same methodology operating on a desktop computer

UT APPLI4-VTS-04 Independent comparison of like versus like cluster images (e.g. image of platform-derived 10 clusters, versus image of desktop-derived 10 clusters), and verification that output images are equal.


Progress made:

ISODATA Clustering code written.

The module has been written for ISODATA and is being enhanced to produce a signature file containing the statistics of each cluster for each cluster output. Following this enhancement, the module can be uploaded into the WINGS environment and tested.


PUBLIC of 41

Step Nb.



5 Independent Verification that for each of the 91 cluster output image files extracted from the data cube by the Co-ReSyF system, the boundaries between clusters have been successfully captured by the boundary extraction algorithm, and integrated into boundary extraction output image file

UT APPLI4-VTS-05 Independent verification that boundary value pixels in the extracted boundary output image files, are spatially located in inter-cluster boundary areas in the classification output cluster image files.


Progress made:

The procedure is clearly defined for desktop applications.

Raster to polygon module written and in the process of being adapter for upload.

polygon to polyline module written and in the process of being adapter for upload.

Polyline to raster converter module written and in the process of being adapter for upload.

Awaiting a polyline field attribute manipulation module t be written and integrated into the platform alongside the above modules before the components can be tested.


PUBLIC of 41

Step Nb.



6 Verification that the raster calculator functionality is operating properly

UT APPLI4-VTS-06 (i) Determine whether the boundary file generated using the platform, and that generated on the desktop PC are equivalent.

(ii) Determine whether the land/ocean boundary areas have the highest value.

Currently untested due to required module not yet being integrated into Platform.

Progress made re. (i):

Final output boundary file has been generated from the desktop-generated datacube of level 4 SST data and is on file to be used to verify platform-generated outputs.

Progress made re. (ii):

Final output boundary file has been generated from the desktop-generated datacube of level 4 SST data. This exhibits areas of high boundary probability, including highest boundary probability areas along all coastlines.

This boundary dataset is on file to be used to verify platform-generated outputs.

Awaiting the composition and platform integration of a raster calculation module generate comparative datasets and complete tests.


PUBLIC of 41

Step Nb.



7 Verification that the statistical analyses are being conducted successfully by the system

N/A APPLI4-VTS-07 Verification that the same statistical conclusions are reached from the same data being processed using the two different system pathways (platform and desktop PC).

No Longer Applicable – the WINGS component of Co-ReSyF will produce OHMA maps for the user to download and analyse. Furthermore the required statistical analyses to validate the produced OHMA maps rely on underway data from multiple sources which are not yet integrated into the Co-ReSyF platform.

Progress made:

Statistical analysis procedure has beenexplored and tested, and is now outlined in the ATBD.

See step 3 for the Randomised comparison validation, now described in the ATBD.

Date of execution: Estimated completion date, December 2018.

3.5 Ocean and Coastal altimetry

TEST PROCEDURE


APPLI5- VTS-01

APPLI5- VTS-02

APPLI5- VTS-03

APPLI5- VTS-04

APPLI5- VTS-05

APPLI5- VTS-06

APPLI5- VTS-07

APPLI5- VTS-08

Functionality to be Tested: Test the good performance of the Ocean & Coastal Altimetry Application

Required Test Environment: Co-ReSyF online platform available, with a valid user account. No special input needed, Co-ReSyF catalogue is sufficient.


PUBLIC of 41

Overview of the test procedure: The test aims to assess the whole processing chain that delivers SWH, SSHA and Sigma0. This test assumes that input data

to the processor are present in the catalogue.

Detailed description of the test procedure, including how to observe and verify the results: Step Nb.


1

Test the application as a whole. The aim is to check if the application is running smoothly and delivering the required results

PASS APPLI5- VTS-01

The outputs of all processing steps shall be along-track CF-compliant NetCDF files, loadable by most common NetCDF readers (for instance Ferret)

The application3 for jason1, jason2 and jason3 altimetry mission was tested on the geoportal. It ran as expected and delivered the results in NetCDF format.

Step Nb.



PUBLIC of 41

2 User specifies mission, pass list, cycle list or time limits, latitude limits

PASS (partially)

APPLI5- VTS-02

Selection module locates and plots/lists the correct list of all SGDR data to process

At the geoportal, user can choose the mission, time limits and latitude limits. [Note that the choice of selecting data by cycle or pass is not yet available and will have to be tested at a later date]. The test was done successfully and the geoportal was able to define and plot all SGDRs available within a geographical box or polygon with a certain time limits as per picture below:

3 User specifies region from predefined list and time limits

PASS (partially)

APPLI5- VTS-03

Selection module locates and plots/lists correct list of all SGDR data to process

As for step no. 0, but polygon/region selection not yet enabled so it will have to be tested at a later stage

Step Nb.



PUBLIC of 41

4 User selects a few passes from list

PASS APPLI5- VTS-04

Module correctly passes to processor the sub-selection of SGDR data to process

The user then is able to create a data package and select the passes that are to be processed by the processor

Step Nb.



PUBLIC of 41

5

Retracking of example data from different missions with appropriate algorithm

PASS APPLI5- VTS-05

Retracked quantities (range, SWH, sigma0) for each mission and algorithm compare favourably (i.e. have equivalent or less noise) with original data

The retracked data compare well with the improved original SGDR data (see SSH, SWH plot).

Step Nb.



PUBLIC of 41

6 Different combinations of range corrections specified by the user

PASS APPLI5- VTS-06 Corrections are properly applied according to user selection

The user is able to choose from a set of improved corrections before running the processor or keep the default corrections. This functionality is working well within the geoportal.

Step Nb.



PUBLIC of 41

7 Output with different degrees of richness, as specified by the user

FAIL APPLI5- VTS-07

NetCDF output is generated at different levels of complexity (i.e. including different combinations of variables) according to user specification

There are still some issues in reading the along-track data provided in the NetCDF format by the Geoportal. Once these are sorted then the processor can be evolved to give the user flexibility in choosing the richness of the outcome. This will have to be tested again at a later stage

8 Validation of output reprocessed SSHA against selected tide gauges

FAIL APPLI5- VTS-07

1. Number valid measurements recovered in the coastal zone increases when using reprocessed data w.r.t. using standard data.

2. Correlation between time series (altimetry vs tide gauges) does not decrease when using reprocessed data w.r.t. using standard data.

There is still need to project the output onto reference tracks (i.e. co-locate the data) so time series analysis can be performed. And also more mission to be integrated within the Geoportal. At this moment only jason-1, jason-2 and Jason-3 are integrated .



PUBLIC of 41

3.6 Application synergy: SAR/Optical Bathymetry cross-validation and

fusion

During the project, it appeared that a synergy between SAR and Optical Bathymetry was highly

suitable and would have been an interesting demonstration for Co-Resyf. But the schedule was

too short and needed the SAR Bathymetry, the Optical Bathymetry and Workflow manager to

be ready and integrated at least 6 months before the end of the project, in order to have the

time to set up and deploy a prototype. For that reason, building this synergy was set aside but it

is to be kept in mind for later development. However, during this cooperation, a deep reflexion

about what can be done has been carried out and the algorithm has been defined and is for sure

a strong basis for later research.


PUBLIC of 41

4 References Co-ReSyF. (2016a). GRANT AGREEMENT-687289. European Commission, Research Executive

Agency.

Co-ReSyF (2016b). System Requirements Document – Tools, issue 1.0. European

Commission, Research Executive Agency.

Deimos (2016a). VERIFICATION AND VALIDATION PROCEDURE, EDG-CMS-PRO16-E, Issue

2.0. ELECNOR DEIMOS CORPORATE MANAGEMENT SYSTEM.

Co-ReSyF (2016c). Verification and Validation Plan - Tools, issue 1.1. European Commission,

Research Executive Agency.

Deimos (2016b). NON-CONFORMANCE MANAGEMENT PROCEDURE, EDG-CMS-PRO12-E,

Issue 3.0. ELECNOR DEIMOS CORPORATE MANAGEMENT SYSTEM.


PUBLIC of 41

END OF DOCUMENT