monthly operations report - european space agency€¦ · reporting period from 16-jan-2017 to...

MONTHLY OPERATIONS

REPORT

MOR#038

Reporting period from 16-Jan-2017 to 15-Feb-2017

Reference: PROBA-V_D5_MOR-038_2017-02_v1.0 Author(s): Dennis Clarijs, Sindy Sterckx, Alex Geboers, Erwin Wolters Version: 1.0 Date: 20/02/2017

Monthly Operations Report PROBA-V Operations Contract No. 4000111291/14/I-LG - 1310174

PROBA-V_D5_MOR-038_2017-02_v1.0 Monthly Operations Report of 20

DOCUMENT CONTROL

Signatures Author(s) Dennis Clarijs, Sindy Sterckx, Alex Geboers, Erwin Wolters Reviewer(s) Dennis Clarijs Approver(s) Dennis Clarijs Issuing authority

Change record

Release Date Pages Description Editor(s)/Reviewer(s)

1.0 20/02/2017 All Initial version



TABLE OF CONTENT

1. Summary .............................................................................................................................................. 4 2. System Infrastructure ........................................................................................................................... 5 3. Image Processing Services .................................................................................................................... 6 3.1. Ingested and archived products ........................................................................................................... 6 3.2. Generated and archived products ........................................................................................................ 6 3.3. Backup and archiving service ............................................................................................................... 7 3.4. Dissemination service .......................................................................................................................... 8 3.5. End-user activity .................................................................................................................................. 8 4. Image Calibration services ................................................................................................................... 11 4.1. Radiometric Calibration ...................................................................................................................... 11 4.2. Geometric Calibration ......................................................................................................................... 16 5. Anomalies ........................................................................................................................................... 17 5.1. System related issues .......................................................................................................................... 17 5.2. Image processing issues ...................................................................................................................... 18 6. Scheduled activities for the next period(s) .......................................................................................... 20 7. Operational remarks ........................................................................................................................... 20



1. Summary In this reporting period, most of the synthesis products were nearly complete. Six transfer frame files were not received by VITO due to very high winds at the data receptions station on 19 January and 13 February. Other smaller factors that influenced data gaps were geometric errors, automatic recoveries of the platform and some decompression errors. Especially the amount of geometric errors is closely monitored. There were no major issues with the image quality during this reporting period for the radiometric or geometric quality although a small peak was registered in terms of daily absolute location error between 4 February and 7 February. In this reporting period, the reprocessing campaign towards Collection 1 (C1) was completed on February 7th. All reprocessed PROBA-V data in all spatial resolutions is now available in HDF5 and geoTIFF format via the Product Distribution Facility, including the newly introduced L2A products. Collection 0 (C0) data will remain available to the public until 1 March 2017. A news letter to the entire user community was communicated on 15 February, while at the same time releasing the reprocessed collection 3 (C3) of SPOT-VGT. Furthermore, a quality page has been launched on the PROBA-V website containing information on the product evaluation, calibration and some info on the PROBA-V Quality Working Group. The products evaluation page contains links to documents related to the evaluation of PROBA-V C1, SPOT-VGT C3 and the comparison of PROBA-V C1, SPOT-VGT C3 collections. A new version of the PROBA-V Product User Manual was also published, mostly containing links to the above documents and more explanation on the PROBA-V Mission Exploitation Platform (MEP) applications. The MEP environment also has access to the new reprocessed collection of PROBA-V and SPOT-VGT. Due to a planned power interruption on February 9, 2017, most of the nominal PROBA-V processing servers were shut down for several hours to prevent any damage on the system. They were restarted the same day, and most of the products were nicely delivered on time. No new developments are expected in the coming period.



2. System Infrastructure

Category % Up Time % Down Time

Switches 100.0 0.0

Database Servers 99.99 0.01

Mid Term File Servers 99.98 0.02

Short Term File Servers 99.99 0.01

Master Servers 100.0 0.0

Worker Nodes 99.45 0.55

PDF 100.0 0.0

Table 1: System Infrastructure availability for this reporting period

Database servers: poracle4, poracle5 down due to power interruption on 09-02-2017

MTFS: pmtfs3 and pmtfs5 down due to power interruption on 09-02-2017

STFS: pstfs1, pstfs5, and pstfs6 down due to power interruption on 09-02-2017

Worker nodes down due to power interruption on 09-02-2017



3. Image Processing Services

3.1. Ingested and archived products

Product Type Total Received Missing data, ingested by VITO

Archived

METEO 236 236 0 236

TFF 310 304 6(*) 304

Table 2: Ingested and archived products for this reporting period

(*) 6 TFF missing: TFF 12248, TFF 12496, TFF 12497, TFF 12498, TFF 12499, TFF 12500: All these passes were lost due to strong winds at the DRS.

3.2. Generated and archived products Product Type Total Processed Error Archived

PROBAV_L1A - Calibration 476 476 0 476

PROBAV_L1A - Nominal 2557 2552 5(*) 2557

PROBAV_L1C 2552 2552 0 2552

PROBAV_L3_S1_TOA_100M 31 31 0 31

PROBAV_L3_S1_TOC_100M 31 31 0 31

PROBAV_L3_S1_TOC_NDVI_100M 31 31 0 31

PROBAV_L3_S5_TOA_100M 6 6 0 6

PROBAV_L3_S5_TOC_100M 6 6 0 6


PROBAV_L3_S1_TOA_300M 31 31 0 31

PROBAV_L3_S1_TOC_300M 31 31 0 31

PROBAV_L3_S10_TOC_300M 3 3 0 3


PROBAV_L3_S1_TOA_1KM 31 31 0 31

PROBAV_L3_S1_TOC_1KM 31 31 0 31

PROBAV_L3_S10_TOC_1KM 3 3 0 3

PROBAV_L3_S10_TOC_NDVI_1KM 3 3 0 3

Table 3: Generated and archived products for this reporting period

(*) 4x L1A Error: Error Geometric processing, 1x due to Error Generating L1A



3.3. Backup and archiving service

Product type Total Files Total File Size (GB)

TFF 316 790.95

L1A 2933 1349.0

Database transaction logs 1075 187.26

Database incremental back-up 0 0.0

Database full back-up 21 1437.61

Table 4: Back-up data volumes for this reporting period

Product type Total Files Total File Size (GB)

PROBAV_TRANSFERFRAMES 1880 5267.43

PROBAV_L1A 17815 9267.15

PROBAV_L1C 15739 18026.32

PROBAV_L3_S1_TOA_100M 418 10554.94

PROBAV_L3_S1_TOC_100M 424 10312.51

PROBAV_L3_S1_TOC_NDVI_100M 261 760.02

PROBAV_L3_S5_TOA_100M 91 8608.53

PROBAV_L3_S5_TOC_100M 86 8145.05


PROBAV_L3_S1_TOA_300M 381 4366.05

PROBAV_L3_S1_TOC_300M 389 4434.42

PROBAV_L3_S10_TOC_300M 37 810.81


PROBAV_L3_S1_TOA_1KM 380 623.01

PROBAV_L3_S1_TOC_1KM 389 631.45

PROBAV_L3_S10_TOC_1KM 37 115.29

PROBAV_L3_S10_TOC_NDVI_1KM 24 5.47

ICP_GEOMETRIC_CENTRE 0 0

ICP_ GEOMETRIC _LEFT 0 0

ICP_ GEOMETRIC _RIGHT 0 0

ICP_RADIOMETRIC_CENTRE 2 0.08

ICP_RADIOMETRIC_LEFT 2 0.08

ICP_RADIOMETRIC_RIGHT 2 0.08

METEO_ECMWF 232 0.29

METEO_METEOSERVICES 232 1.24

POLARMOTION 1 0.00

Table 5: Archived data volumes for this reporting period



3.4. Dissemination service Product type Added to catalogue Ordered Delivered

PROBAV_L1C 2554 113 114

PROBAV_L3_S1_TOA_100M 31 106 116

PROBAV_L3_S1_TOC_100M 31 1654 1775

PROBAV_L3_S1_TOC_NDVI_100M 31 86 123

PROBAV_L3_S5_TOA_100M 6 24 97

PROBAV_L3_S5_TOC_100M 6 76 86


PROBAV_L3_S1_TOA_300M 31 197 209

PROBAV_L3_S1_TOC_300M 32 1628 4448

PROBAV_L3_S10_TOC_300M 3 74 81


PROBAV_L3_S1_TOA_1KM 31 135 143

PROBAV_L3_S1_TOC_1KM 31 208 204

PROBAV_L3_S10_TOC_1KM 3 167 274

PROBAV_L3_S10_TOC_NDVI_1KM 3 518 653

Table 6: Ordered and delivered products for this reporting period

3.5. End-user activity 17 new user(s) were registered in this reporting period. The total number of users registered for PROBA-V data and that have ordered data is 1054 with 104 different nationalities representing 806 different companies/universities.

Product type Africa Asia Europe N-America Oceania S-America

PROBAV_L1C 0 0 238.77 0 0 0

PROBAV_L3_S1_TOA_100M 0.04 297.60 60.96 0 0 0.01

PROBAV_L3_S1_TOC_100M 0 1376.65 682.01 7796.77 0 0

PROBAV_L3_S1_TOC_NDVI_100M 0.64 1.10 0.57 0 0 0

PROBAV_L3_S5_TOA_100M 0 1033.69 41.35 0 0 0.01

PROBAV_L3_S5_TOC_100M 37.61 25.30 501.54 0.92 0 0.09

PROBAV_L3_S5_TOC_NDVI_100M 0.79 0.98 280.02 0.61 0.10 1.69

PROBAV_L3_S1_TOA_300M 0 1.98 1037.56 0 0 0

PROBAV_L3_S1_TOC_300M 0 0 2539.18 4634.06 0 0

PROBAV_L3_S10_TOC_300M 4.23 1.70 176.18 0 0 0

PROBAV_L3_S10_TOC_NDVI_300M 0.06 1.89 11.45 0 0.03 1.07

PROBAV_L3_S1_TOA_1KM 0.00 0 149.81 0 0 0.12



PROBAV_L3_S1_TOC_1KM 0.00 0 180.45 3.11 0 0

PROBAV_L3_S10_TOC_1KM 0 0 23.34 0.93 0 0

PROBAV_L3_S10_TOC_NDVI_1KM 6.64 0.42 2.24 24.99 0 0.02

Table 7: Data download (GB) in total per Origin of the User for the reporting period

Product Type Global

L1C 238.77

PROBAV_L3_S1_TOA_100M 358.60

PROBAV_L3_S1_TOC_100M 9855.42

PROBAV_L3_S1_TOC_NDVI_100M 2.31








PROBAV_L3_S1_TOA_1KM 149.93

PROBAV_L3_S1_TOC_1KM 183.56

PROBAV_L3_S10_TOC_1KM 24.27

PROBAV_L3_S10_TOC_NDVI_1KM 34.30

Table 8: Data download (GB) in total for the reporting period

Company # Downloads

UNIVERSITY OF LEICESTER 1821

GOOGLE 1430

VITO 1268

METEO FRANCE 1083

NSMC 534

UNESCO-IHE 425

XINJIANG INSTITUTE OF ECOLOGY AND GEOGRAPHY, CAS 314

CNES 243

ARMSTATEHYDROMET 226

GSI LTD 196

Table 9: Top 10 user companies for the reporting period



Country # Users

CHINA 104

BELGIUM 91

FRANCE 55

ITALY 55

BRAZIL 47

UNITED STATES 47

UNITED KINGDOM 43

INDIA 39

NETHERLANDS 35

GERMANY 34

Table 10: Top 10 countries with most registered users List of issues raised by users:

ProbaV: * information - Patagonia image * Problems uploading the ordered data * Software that can read Probav * Availability of PROBA V101 data * Lost username * Download in TIFF format and IDRISI cannot be imported because RGB Full Color Image is not supported * Subscription LST for Proba-v App team * Products of order "M0022500" (they do not include the .h5 format LAI data) * Quicklook data for PROBA-V Image Products * FTP Download Connection Problems * SPOT NDVI dataset (2015 and 2016) * Spot vegetation derived products PDF: * Documentation subscription process * Forgot my credentials



4. Image Calibration services

4.1. Radiometric Calibration

Calibration request type Total Processed Not received Error

CLOUDS 16 16 0 0

DARK CURRENT 22 22 0 0

MOON 1 1 0 0

RAYLEIGH 52 52 0 0

SNOW 95 94 1 0

SUN_GLINT 0 0 0 0

Table 11: Calibration Image requests for this reporting period

Calibration image type Total Valid Invalid(*)

PROBA_V_L1A_CALIBRATION 1 1 0

PROBA-V_L1B_CALIBRATION 475 423 52

PROBA-V_L1B_INTERSECTION 743 338 405

PROBA-V_L1B_OVERLAPREGION 0 0 0

Table 12: Processed calibration images for this reporting period

(*) Due to insufficient overlap with the calibration region of interest, not enough pixels (e.g. clouds contamination), site not sufficiently uniform (illumination), etc. Long-term monthly Libya-4 mean plots for different cameras are given in Figure 1, Figure 2 and Figure 3. Deep convective clouds interband calibration results are given in Figure 4. Please note that plots are given for collection 0 products. The DCC interband calibration results for LEFT and CENTER BLUE still show a decrease. For the NIR strips this is no longer the case. For the CENTER BLUE the decreasing trend is confirmed by the Libya-4 and Moon results. It will be evaluated if a degradation model will need to be applied to the center Blue absolute calibration coefficient. A clear vertical stripe is visible at pixel 131 (0-based numbering) of the CENTER SWIR3 strip. This pixel will be assigned the status BAD at the next ICP file update. Radiometric ICP file ICP dark values will be updated in the coming days and one bad pixel, i.e. Center SWIR3 pixel 131, will be added.



Another ICP update will follow end of the month where both the SWIR absolute calibration coefficients will be updated following the linear degradation model and dark currents will be updated. The current ICP files are

PROBAV_ICP_RADIOMETRIC#LEFT_20170201_V01

PROBAV_ICP_RADIOMETRIC#CENTER_20170201_V01

PROBAV_ICP_RADIOMETRIC#RIGHT_20170201_V01

Figure 1. Libya-4 desert calibration results: LEFT monthly averaged results (collection 0)



Figure 2. Libya-4 desert calibration results: CENTER monthly averaged results (collection 0)



Figure 3. Libya-4 desert calibration results: RIGHT monthly averaged results (collection 0)



Figure 4. DCC inter-band calibration results: LEFT, CENTER and RIGHT camera (collection 0)



4.2. Geometric Calibration

Calibration image type Total Processed Error

PROBA-V_L1C_INTERSECTION 13825 13825 0

Table13: Processed calibration images for this reporting period

During previous month, the average ALE was < 82 m (σ < 98 m). Daily values were rather stable (ALE 60 – 80 m) until 4/2, after which a sharp increase to a maximum of 131 m (7/2) occurred. This peak was followed by a sharp decline. During the remainder of the period, daily ALE values fluctuated between 75 and 95 m. The geometric accuracy was within the requirement of < 300 m, with an average compliance for all cameras of 99.1% (98.5 – 99.7% from BLUE to SWIR). Daily values showed the usual signature coinciding with the minimum and maximum ALE values, with an absolute minimum for the VNIR of 97.0% (BLUE) – 97.6% (NIR) at 7/2. Geometric ICP file

PROBAV_ICP_GEOMETRIC#LEFT_20160907_V01

PROBAV_ICP_GEOMETRIC#CENTER_20160907_V01

PROBAV_ICP_GEOMETRIC#RIGHT_20160907_V01

Figure 5 - Daily ALE evolution for all PROBA-V spectral bands for 16/1/2017 – 15/2/2017



5. Anomalies

5.1. System related issues A detailed description of each issue is available in the issue tracking system http://jira.vgt.vito.be

Key Summary Status Created Component/s

PROBAVUS-7 Very small images fail to process Resolved 10/01/2014 General

PROBAVUS-60 LTDA Restore fails when product destination already exists on disk

Resolved 22/01/2016 Software

PROBAVUS-63 Cloud shadow detection at high solar zenith angles not working properly

Open 11/05/2016 Software

PROBAVUS-65 Processing statuses L2 products Open 16/09/2016 Software

PROBAVUS-66 Cloud cover percentages on PDF products are not reliable

Open 19/10/2016 PDF

PROBAVUS-67 PDF Customisation.exe: L2 geotiff bands are not grouped

Resolved 19/10/2016 PDF

0 new issues were logged during this reporting period 0 issue(s) was resolved and closed during this reporting period 2 issues are resolved but remain to be closed formally 1 issues are resolved but remain in the list logging purposes 3 issue(s) is open and remain to be solved

http://jira.vgt.vito.be/browse/PROBAVUS-39









5.2. Image processing issues A detailed description of each issue is available in the Weekly Report and the image processing tracking system https://juniper.vgt.vito.be/ciptools The below table gives an overview of the S1’s of this reporting period:

# S1 Dates

Major Gaps (> 21600 km² (missing TFF)) 1 13/02

Large Gaps (< 21600 km²) 1 19/01

Medium Gaps (< 10000 km²) 0

Minor Gaps (< 3600 km²) 6 10/02, 15/02, 31/01, 16/01, 24/01, 09/02

Negligible Gaps (< 1000 km²) 23 05/02, 26/01, 21/01, 20/01, 25/01, 30/01, 04/02, 14/02, 23/01, 28/01, 18/01, 08/02, 03/02, 29/01, 01/02, 27/01, 17/01, 11/02, 06/02, 22/01,

12/02, 07/02, 02/02

Complete synthesis (no gaps) 0

Table14: Overview of S1 for this reporting period

https://juniper.vgt.vito.be/ciptools



Synthesis Missing Decom. Error

Geom. Error

Missing TFF

Autom. Recovery

VC4 Missing

Create Contours

Other

20170116 4.15% 6 44 2

20170117 0.85% 4 41

20170118 0.67% 1 39

20170119 11.36% 4 33 1 1

20170120 0.54% 37

20170121 2.11% 30

20170122 2.49% 1 25

20170123 2.22% 28

20170124 4.50% 4 28 1

20170125 1.51% 5 26

20170126 0.56% 34

20170127 1.37% 33

20170128 0.19% 2 23

20170129 0.14% 1 6

20170130 0.11% 7

20170131 0.08% 13 1

20170201 3.22% 3 11

20170202 2.11% 5 9

20170203 2.22% 19

20170204 1.01% 2 17

20170205 1.35% 1 11

20170206 0.05% 9

20170207 0.01% 1 4

20170208 0.07% 1 12

20170209 1.73% 3 13 1

20170210 2.50% 7 14 2 78

20170211 3.68% 8

20170212 3.64% 18

20170213 54.90% 2 10 5 1

20170214 2.48% 9

20170215 4.90% 4 22 3

Table 15: List of synthesis with an error overview of the missing percentages and errors for this reporting period



6. Scheduled activities for the next period(s)

Software upgrades: No software upgrades planned.

Hardware: As the PROBA-V reprocessing has ended, most of the machines will be moved to nominal production in order to phase out the oldest servers.

Development: No new developments are planned.

No other activities scheduled.

7. Operational remarks

Due to a planned power interruption on February 9, 2017, most of the nominal PROBA-V processing servers were shut down for several hours to prevent any damage on the system. They were restarted the same day, and most of the products were nicely delivered on time.

monthly operations report - european space agency€¦ · reporting period from 16-jan-2017 to...

Documents