d2.3 updated report on e-infrastructure requirements v1...mainly in first three scenarios (open land...

This document is issued within the frame and for the purpose of the EUXDAT project. This project has received funding from the European

Union’s Horizon2020 Framework Programme under Grant Agreement No. 777549. The opinions expressed and arguments employed herein

do not necessarily reflect the official views of the European Commission. This document and its content are the property of the EUXDAT Consortium. All rights relevant to this document are determined by the

applicable laws. Access to this document does not grant any right or license on the document or its contents. This document or its contents are

not to be used or treated in any manner inconsistent with the rights or interests of the EUXDAT Consortium or the Partners detriment and are

not to be disclosed externally without prior written consent from the EUXDAT Partners. Each EUXDAT Partner may use this document in conformity with the EUXDAT Consortium Grant Agreement provisions. (*) Dissemination level.-PU: Public, fully open, e.g. web; CO: Confidential, restricted under conditions set out in Model Grant Agreement;

CI: Classified, Int = Internal Working Document, information as referred to in Commission Decision 2001/844/EC.

D2.3 Updated Report on e-Infrastructure

Requirements v1

Document Identification

Status Final Due Date 5.11.2018

Version 1.0 Submission Date 5.11.2018

Related WP WP2 Document Reference D2.3

Related

Deliverable(s)

D2.1 Dissemination Level (*) PU

Lead Participant WRLS Lead Author Karel Jedlička

Contributors Michal Kepka

(WRLS), Miguel

Ángel Esbrí (ATOS)

Pavel Hájek (P4A),

Marcela Doubková

(PI), Dimitrios

Moshou (CERTH),

Karl Gutbrodt (MB),

Dimitrij Kozukh

(WRLS)

Reviewers Marcela Doubková (PI)

Fabien Castel (ATOS

FR)

Keywords:

Pilot definition, infrastructure, requirements, EUXDAT

Document name: D2.3 Description of Proposed Pilots and Requirements V1 Page: 2 of 98

Reference: D2.3 Dissemination: PU Version: 1.0 Status: Final

Document Information

List of Contributors

Name Partner

Karel Jedlička WRLS

Michal Kepka WRLS

Pavel Hájek P4A

Marcela Doubková Pessl Instr.

Miguel Ángel Esbrí ATOS ES

Dimitrios Moshou CERTH

Dmitrij Kozukh WRLS

Karl Gutbrodt Meteoblue

Document History

Version Date Change editors Changes

0.1 Susana Palomares

Fernández

D2.3 uploaded for editing

0.2 27. 9. 2018 Karel Jedlička Assigning tasks to partners

0.2.1 6. 10. 2018 Karel Jedlička Editing of Pilot 3 and adding related scenarios

0.3 9. 10. 2018 Karel Jedlička,

Michal Kepka, Pavel

Hájek

Executive Summary, introduction, pilot 3 related

scenarios and requirements. General guidelines for

authors added as comments.

0.4 19.10.2018 Miguel Ángel Esbrí Updated generic EUXDAT platform requirements

according to new information in deliverables D2.2

and D3.2

0.5 25.10.2018 Michal Kepka Consolidating of partners contributions, overall

formatting

0.6 31.10.2018 All contributors Review reactions

0.7 5.11.2018 Karel Jedlička Final version for quality review

0.8 5.11.2018 ATOS Quality review

1.0 5.11.2018 FINAL READY FOR SUBMISSION

Quality Control

Role Who (Partner short name) Approval Date

Deliverable leader Karel Jedlička (WRLS) 5.11.2018

Technical manager Fabien Castel (ATOS FR) 5.11.2018

Quality manager Susana Palomares (ATOSES) 5.11.2018

Project Manager Javier Nieto (ATOSES) 5.11.2018



Table of Contents

Document Information ............................................................................................................................ 2

Table of Contents .................................................................................................................................... 3

List of Tables ........................................................................................................................................... 5

List of Figures ......................................................................................................................................... 7

List of Acronyms ..................................................................................................................................... 8

Executive Summary ................................................................................................................................ 9

1 Introduction .................................................................................................................................. 10

1.1 Purpose of the document ....................................................................................................... 10

1.2 Structure of the document ..................................................................................................... 10

1.3 Relation to other project work ............................................................................................... 10

2 Precision farming ideas and scenarios .......................................................................................... 12

2.1 Precision farming ideas ......................................................................................................... 12

2.1.1 Open Land Use Map improvement ............................................................................... 12

2.1.2 Identification and forecast of crop types and production .............................................. 13

2.1.3 Monitoring of crop status .............................................................................................. 14

2.1.4 Delimiting of agro-climatic zones ................................................................................. 15

2.1.5 Looking for climatic patterns changes........................................................................... 16

2.1.6 Information support for field use recommendations ..................................................... 16

2.1.7 Effective utilization of natural resources ....................................................................... 17

2.2 Elaborated scenarios .............................................................................................................. 18

2.2.1 Open Land Use Map Improvement scenario ................................................................. 18

2.2.2 Monitoring of crop status scenario ................................................................................ 20

2.2.3 Delimiting Agro-climatic zones scenario ...................................................................... 22

2.2.4 Looking for climatic patterns changes scenario ............................................................ 24

2.2.5 Information support for field use recommendations scenario ....................................... 26

2.2.6 Effective utilization of natural resources scenario ........................................................ 30

2.3 Precision farming ideas and scenarios summary ................................................................... 31

3 Proposed pilots ............................................................................................................................. 32

3.1 Pilot 1: Land Monitoring and Sustainable Management ....................................................... 32

3.1.1 Vision of the pilot .......................................................................................................... 32

3.1.2 Used Datasets description .............................................................................................. 33

3.1.3 Pilot contribution to defined scenarios .......................................................................... 33



3.2 Pilot 2: Energy efficiency analysis ........................................................................................ 33


3.2.2 Used datasets description .............................................................................................. 34


3.3 Pilot 3: 3D farming ................................................................................................................ 34


3.3.2 Used Datasets description .............................................................................................. 35


3.4 Pilot scenario matching ......................................................................................................... 36



List of Tables

Table 1: Relevance of Open Land Use Improvement idea to criteria for further elaboration ................ 13 Table 2: Relevance of Forecast of crop types and production idea to criteria for further elaboration .. 13 Table 3: Relevance of monitoring of crop status idea to criteria for further elaboration........................ 14 Table 4: Relevance of delimiting of agro-climatic zones idea to criteria for further elaboration ............ 15 Table 5: Relevance of Looking for climatic patterns changes idea to criteria for further elaboration ... 16 Table 6: Relevance of information support for field use recommendations idea to criteria for further

elaboration ............................................................................................................................................. 17 Table 7: Relevance of Effective utilization of natural resources idea to criteria for further elaboration 17 Table 8 Pilot to Scenario matching ........................................................................................................ 36 Table 9: Informational requirement template .............................................................................................. 37 Table 10: Functional/non-functional requirement templates ........................................................................ 38 Table 11: Pilots Informational requirements .............................................................................................. 39 Table 12: Pilots - Functional/non-functional requirements .......................................................................... 40 Table 13: EUXDAT platform general functional and non-functional requirements ........................................ 41 Table 14: EUXDAT-REQ-PILOT-DATA-001 .............................................................................................. 48 Table 15: EUXDAT-REQ-PILOT-DATA-002 .............................................................................................. 49 Table 16: EUXDAT-REQ-PILOT-DATA-003 .............................................................................................. 49 Table 17: EUXDAT-REQ-PILOT-DATA-004 .............................................................................................. 50 Table 18: EUXDAT-REQ-PILOT-DATA-005 .............................................................................................. 51 Table 19: EUXDAT-REQ-PILOT-DATA-006 .............................................................................................. 52 Table 20: EUXDAT-REQ-PILOT-DATA-007 .............................................................................................. 52 Table 21: EUXDAT-REQ-PILOT-DATA-009 .............................................................................................. 55 Table 22: EUXDAT-REQ-PILOT-DATA-010 .............................................................................................. 55 Table 23: EUXDAT-REQ-PILOT-DATA-011 .............................................................................................. 56 Table 24: EUXDAT-REQ-PILOT-DATA-012 .............................................................................................. 57 Table 25: EUXDAT-REQ-PILOT-DATA-013 .............................................................................................. 57 Table 26: EUXDAT-REQ-PILOT-001 ........................................................................................................ 59 Table 27: EUXDAT-REQ-PILOT-002 ........................................................................................................ 59 Table 28: EUXDAT-REQ-PILOT-003 ........................................................................................................ 60 Table 29: EUXDAT-REQ-PILOT-004 ........................................................................................................ 60 Table 30: EUXDAT-REQ-PILOT-005 ........................................................................................................ 61 Table 31: EUXDAT-REQ-PILOT-006 ........................................................................................................ 61 Table 32: EUXDAT-REQ-PILOT-007 ........................................................................................................ 62 Table 33: EUXDAT-REQ-PILOT-008 ........................................................................................................ 62 Table 34: EUXDAT-REQ-PILOT-009 ........................................................................................................ 63 Table 35: EUXDAT-REQ-PILOT-010 ........................................................................................................ 63 Table 36: EUXDAT-REQ-PILOT-011 ........................................................................................................ 64 Table 37: EUXDAT-REQ-PILOT-012 ........................................................................................................ 64 Table 38: EUXDAT-REQ-PILOT-013 ........................................................................................................ 65 Table 39: EUXDAT-REQ-PLATF-001 ....................................................................................................... 67 Table 40: EUXDAT-REQ-PLATF-002 ....................................................................................................... 67 Table 41: EUXDAT-REQ-PLATF-003 ....................................................................................................... 68 Table 42: EUXDAT-REQ-PLATF-004 ....................................................................................................... 68 Table 43: EUXDAT-REQ-PLATF-005 ....................................................................................................... 69 Table 44: EUXDAT-REQ-PLATF-006 ....................................................................................................... 70 Table 45: EUXDAT-REQ-PLATF-007 ....................................................................................................... 70 Table 46: EUXDAT-REQ-PLATF-008 ....................................................................................................... 71



Table 47: EUXDAT-REQ-PLATF-009 ....................................................................................................... 72 Table 48: EUXDAT-REQ-PLATF-010 ....................................................................................................... 72 Table 49: EUXDAT-REQ-PLATF-011 ....................................................................................................... 73 Table 50: EUXDAT-REQ-PLATF-012 ....................................................................................................... 74 Table 51: EUXDAT-REQ-PLATF-013 ....................................................................................................... 74 Table 52: EUXDAT-REQ-PLATF-014 ....................................................................................................... 75 Table 53: EUXDAT-REQ-PLATF-015 ....................................................................................................... 76 Table 54: EUXDAT-REQ-PLATF-016 ....................................................................................................... 77 Table 55: EUXDAT-REQ-PLATF-017 ....................................................................................................... 77 Table 56: EUXDAT-REQ-PLATF-018 ....................................................................................................... 78 Table 57: EUXDAT-REQ-PLATF-019 ....................................................................................................... 79 Table 58: EUXDAT-REQ-PLATF-020 ....................................................................................................... 79 Table 59: EUXDAT-REQ-PLATF-021 ....................................................................................................... 81 Table 60: EUXDAT-REQ-PLATF-022 ....................................................................................................... 81 Table 61: EUXDAT-REQ-PLATF-023 ....................................................................................................... 82 Table 62: EUXDAT-REQ-PLATF-024 ....................................................................................................... 82 Table 63: EUXDAT-REQ-PLATF-025 ....................................................................................................... 83 Table 64: EUXDAT-REQ-PLATF-026 ....................................................................................................... 84 Table 65: EUXDAT-REQ-PLATF-027 ....................................................................................................... 84 Table 66: EUXDAT-REQ-PLATF-028 ....................................................................................................... 85 Table 67: EUXDAT-REQ-PLATF-029 ....................................................................................................... 87 Table 68: EUXDAT-REQ-PLATF-030 ....................................................................................................... 87 Table 69: EUXDAT-REQ-PLATF-031 ....................................................................................................... 88 Table 70: EUXDAT-REQ-PLATF-032 ....................................................................................................... 89 Table 71: EUXDAT-REQ-PLATF-033 ....................................................................................................... 89 Table 72: EUXDAT-REQ-PLATF-034 ....................................................................................................... 90 Table 73: EUXDAT-REQ-PLATF-035 ....................................................................................................... 91 Table 74: EUXDAT-REQ-PLATF-036 ....................................................................................................... 91 Table 75: EUXDAT-REQ-PLATF-037 ....................................................................................................... 92 Table 76: EUXDAT-REQ-PLATF-038 ....................................................................................................... 93 Table 77: EUXDAT-REQ-PLATF-039 ....................................................................................................... 93 Table 78: EUXDAT-REQ-PLATF-040 ....................................................................................................... 94 Table 79: EUXDAT-REQ-PLATF-041 ....................................................................................................... 95 Table 80: EUXDAT-REQ-PLATF-042 ....................................................................................................... 95 Table 81: EUXDAT-REQ-PLATF-043 ....................................................................................................... 97 Table 82: EUXDAT-REQ-PLATF-043 ....................................................................................................... 98



List of Figures

Figure 1 Temperature zones ................................................................................................................. 22 Figure 2: Planned releases for the proposed e-Infrastructure, according to EUXDAT Grant Agreement

[2] ........................................................................................................................................................... 32



List of Acronyms

Abbreviation /

acronym

Description

API Application programming interface

DEM Digital Elevation Model

EO Earth Observation

EU-DEM European Union Digital Elevation Model

FMIS Farm Management Information System

GEO/GEOSS Group on Earth Observations/ Global Earth Observation System of Systems

HPC High Performance Computing

HRM/LAM High resolution Models / Local Area Weather Models

INSPIRE INSPIRE Directive aims to create a European Union spatial data infrastructure for the

purposes of EU environmental policies and policies or activities which may have an impact

on the environment

LAI Leaf Area Index

LPIS Land Parcel Identification System

NDVI Normalized Difference Vegetation Index

OGC Open Geospatial Consortium

OLU Open Land Use map

SRTM Shuttle Radar Topography Mission

UAV Unmanned Aerial Vehicle

VW Verticillium Wilt

WP Work Package



Executive Summary

This deliverable aims to update the deliverable 2.1 Description of Proposed Pilots and Requirements

especially in the parts related to e-Infrastructure requirements. The D2.3 deliverable is second in a row

of deliverables tracking the e-infrastructure development (see the list of related deliverables in section

1.3).

Chapters 1-3 describe the high-level vision by describing the initial ideas (chapter 2.1), scenarios

(chapter 2.2) and pilots (chapter 3). Therefore, these chapters stay more or less the same as in D2.1.

The main development of the D2.3 text lies in chapter 4 EUXDAT Pilots and Platform requirements and

Annex 1 Detailed pilot’s requirements and Annex 2 Detailed EUXDAT Platform requirements.

D2.3 takes advantage of the parallel work on V1 version of e-infrastructure and consequent experiments,

mainly in first three scenarios (Open Land Use Map Improvement, Monitoring of Crop Status and

Delimiting of agro-climatic zones). Therefore, informational, functional and non-functional

requirements related to these scenarios are the most changed ones. Even the rest of the requirements is

updated, where possible, taking into account additional information and requirements gathered from

other technical deliverables in the project (e.g., D2.2 EUXDAT e-Infrastructure Definition and D3.2

End Users’ Platform).

Whereas in D2.1, the requirements were gathered pilot by pilot and then scenario by scenario,

requirement overlaps occurred. The analysis of these overlaps was realized and duplications were

eliminated in D2.3.



1 Introduction

1.1 Purpose of the document

The purpose of this document is to update the description of the pilots defined in EUXDAT proposal

and scenarios defined in D2.1 by describing the consortium ideas related to these pilots and precision

farming (chapters 2 and 3) which emerge during the M7-M12 project period in the consortium, while

working on WP2. Furthermore, the document refines both pilot specific and general functional and non-

functional requirements related to the pilots and scenarios. These requirements have been firstly defined

in D2.1 taking into account the information coming from the pilots, as well as the previous experience

of the partners (both from their technical background and from requirements observed also in previous

projects from similar domains). The D2.3 adds details to initial high-level design of the e-Infrastructure

architecture.

Therefore, this document is the basis to define the main features to be provided by EUXDAT and is

used for the e-infrastructure architecture design.

1.2 Structure of the document

This document is structured in 3 major chapters and 2 annexes:

Chapter 2 presents consortium updated ideas (2.1) related to precision farming themes in EUXDAT

pilots. Ideas worth to elaborate are then turned into scenarios (2.2)

Chapter 3 presents original EUXDAT pilots and their relation to updated ideas and scenarios

Chapter 4 presents both all three pilots’ specific requirements (4.1) and EUXDAT general platform

requirements (4.2)

Annex 1 describes pilot specific requirements in detail

Annex 2 describes EUXDAT general platform requirements

1.3 Relation to other project work

This document is the third deliverable of 6 deliverables taking place in Work Package 2:

• D2.1 Description of Proposed Pilots and Requirements - to deliver by month 4 - responsible

partner: WIRELESSINFO

• D2.2 EUXDAT e-Infrastructure Definition v1 - to deliver by month 6 - responsible partner:

ATOS SPAIN SA

• D2.3 Updated Report on e-Infrastructure Requirements v1 - to deliver by month 12 - responsible

partner WIRELESSINFO


ATOS SPAIN SA

• D2.5 Updated Report on e-Infrastructure Requirements v2 - to deliver by month 20 - responsible

partner: WIRELESSINFO


ATOS SPAIN SA

The Work Package 2 aims to:



• Describe the pilots proposed as the validators of the e-Infrastructure developments to be done

in the rest of WPs;

• Identify and describe those end users' communities that might benefit from the proposed e-

Infrastructure, which may have new data management-related needs;

• Gather requirements from the pilots and from other stakeholders, in order to understand the

current and future needs for computation and data management;

• Define the main features to be fulfilled by the e-Infrastructure, identifying the key components

to be modified and the potential bottlenecks for scaling up to extremely large data analysis.

The D2.3 is related to Tasks Task 2.1: Pilots and Future Problems Description (M1-M9) and 2.2:

Platform-Driven e-Infrastructure Requirements (M1-M20).



2 Precision farming ideas and scenarios

This chapter captures consortium ideas related to precision farming and discusses its business potential.

The chapter 2.1 describes the captured ideas and indicates their relevance to the EUXDAT project focus

by evaluating following 6 criteria:

1. Uses already available geographical databases from different sources.

2. Uses high spatial and temporal resolution datasets to permit value-adding services.

3. Creates services linking different existing datasets

4. Not creating new datasets, nor new complex algorithms, because both are either too time-

consuming, need too much ground-truthing (over vast areas), lack credibility in the respective

communities and are often already generated by specialised organisations, which have a closer

contact to their communities.

5. Focuses on large-area applications: either major crops, or applications suitable for multiple

crops.

6. Avoids regional specialisation and niche crops, because those do not serve the main

communities.

An idea is then selected for further elaboration, if it matches a majority of the criteria. A scenario is then

described for each selected idea in section 2.2.

2.1 Precision farming ideas

This section shortly describes identified ideas related to precision farming and discusses their business

potential and their relation to EUXDAT Pilots.

2.1.1 Open Land Use Map improvement

The Open Land Use Map (OLU) was created by consortium partners in previous projects (Foodie1,

SDI4Apps2) and is now maintained by the Plan4all association. The idea of the EUXDAT project is to

use Earth Observation (EO) data, EU-DEM (European Union Digital Elevation Model) and existing

country by country specific data to enrich the Open Land Use Map. The aim is to enrich OLU using

classified EO data, add morphologic characteristic to each zone and optionally integrate open data from

local resources into OLU. This approach can generate more detailed blocks in countries, where there no

open Land Parcel Identification System (LPIS) exists.

1 http://www.foodie-project.eu/

2 http://sdi4apps.eu/

http://www.foodie-project.eu/

http://sdi4apps.eu/



Table 1: Relevance of Open Land Use Improvement idea to criteria for further elaboration

Criterion

No

Relevance to the criterion

1. The idea satisfies this criterion as it uses already existing databases - Open Land Use Map, plus

possibly others such as SRTM data, open satellite and radar data.

2. The idea operates on quite detailed spatial level - level of land parcels and LPIS fields.

3. Within the idea, the attributes of Open Land Use Map will be extended by the attributes calculated

from other open datasets SRTM, satellite and radar imagery.

4. The idea will not create any new datasets - it will improve existing.

5. The idea fulfils this criterion. Application can be either for major crop or for any other.

6. The idea fulfils this criterion. Application can be either for major crop or for any other.

Decision: Elaborate the idea further, because Open Land Use Map is a key dataset and consortium partners

plan to use OLU as a base layer to support EUXDAT pilots.

2.1.2 Identification and forecast of crop types and production

Using classification of EO data for forecast of crop types according LPIS blocks - use time series of

images (Sentinel 2) to provide a forecast of production. The principal usage could would be food

industry, but it could be also used for crop management (by identifying critical growth stages for

intervation) and government agencies (to compare actual field use with reported uses).

This idea needs a dataset of crop rotation data from at least one large-scale farm, LPIS data and time

series. Then crop types can be identified on the base of multi-temporal statistics in areas outside the

training data form the farm.

A second approach developed in October 2018 would use land use maps, select cropping areas, then

build time series on those areas, classify the time series into winter and summer crops (by using fallow

periods for separation and validating this with meteorological data – e.g. precipitation sums to confirm

growth or absence thereof), then separating the crops using specific rules, such as typical periods

(sowing / emergence, booting, flowering, maturation, harvesting times), colours (e.g. green for winer

cereals vs. yellow for oilseed rape), temperature min/max and sums, soil moisture. The resulting

classification will be ground truthed with 3 methods: manually, overlay with statistics, and check by

users (customers).

Table 2: Relevance of Forecast of crop types and production idea to criteria for further elaboration

Criterion

No


1. The idea does not fully satisfy this criterion, since there is no existing database for crop type.

Satellite data would have to be converted into crop type using smart features, such as planting



date (agricultural land, slope, first appearance of crop, crop colour, density increase, harvest

date, eventually even previous crop).

2. The idea operates on quite detailed spatial level – resolution of satellite data is 300 meters to

below 10 meters.

3. Fulfilled: Would link satellite to land use, slope (and weather) data.

4. The idea will create new datasets – based on existing ones. This can be done on the fly and

become a very interesting service.

5. Very well fulfilled: Application will be for major crop or for any other.

6. Fulfilled. Application will be applicable to any crop and area – if the algorithms can be set by

users, then other uses could also be created.

Decision: Keep the idea as a reserve, it is highly dependent on training data from at least one farm, which

is hard to acquire.

2.1.3 Monitoring of crop status

The idea is to develop a system for monitoring of crops status. The system will include crop anomaly

detection component based on specific Sentinel 2 image analysis algorithms to monitor crop conditions

and presence of anomalies as deviation with one-class machine learning. It will then attribute anomaly

to many types of stress or disease by hyperspectral data and crowdsourcing data. According to stress

reason identification as related nutrient, drought, insect or disease, the appropriate tree of decisions will

be proposed (fertilizer, pesticide, other), combined with Meteo and Epidemic model would be desired

for improving risk forecasting activities.

A dataset already exists on multispectral and hyperspectral monitoring of olive orchards for condition

and crop anomaly detection where verticillium and drought stress were detected. More data will be

obtained with a hyperspectral camera on UAV. These data can be combined with meteo data and

epidemic model for future risk.

The system will build on Error! Reference source not found. (Identification and Forecast of crop types a

nd production) and be cross checked with meteo data and user feedback: users can annotate the accuracy

of the crop Identification and growth stage definition, and thereby differences between calculation and

actual developments can be determined. This process can be fully automated.

Table 3: Relevance of monitoring of crop status idea to criteria for further elaboration

Criterion

No


1. The idea satisfies this criterion. A dataset already exists on multispectral and hyperspectral

monitoring of olive orchard. Possibly others: Open satellite data Sentinel 2 multispectral, meteo

forecasting.



2. The idea operates with high spatial resolution due to UAV based hyperspectral and multispectral

additionally to S2 multispectral. Also temporal due to ad hoc use of UAV monitoring. These

permit added-value services

3. Within the idea the attributes of Level-1C multi-spectral imaging products from the Sentinel-2,

sensor data from UAV hyperspectral imagery, Meteorological forecasting

4. The idea will create partially new datasets in the form of hyperspectral datasets

5. Focuses on Large-area applications: olive orchards, suitable for multiple crops.

6. Application of monitoring can be for any tree crop that can be either for major crop or for any

other.

Decision: CONTINUE-ELABORATE FURTHER: a scenario for this idea concerns Field use

recommendations - providing information about field interventions in infected trees by curative

spray from monitoring maps.

2.1.4 Delimiting of agro-climatic zones

Current climate zones maps are very generic. These show large areas and display only some differences

in topography. Characteristics such as seaside buffer zones, weather divides or South-North differences

are usually not accounted. The idea is to provide local agro-climatic maps by processing Historic

weather databases and detailed Earth Observation data for topography and land cover.

Such improvements in the climate zones would support local/within-field management strategies. For

researchers it may be of interest to use this dataset for decisions related to field trial (climatic)

representativeness. Agronomists and insurances may find this dataset useful for risk assessment.

Last but not least, researchers and advisors may find important to check the impact of climate change

on given area and decide about future management strategies.

Table 4: Relevance of delimiting of agro-climatic zones idea to criteria for further elaboration

Criterion

No


1. The idea satisfies this criterion, since there is no existing database for climate zones.

2. The idea operates on quite detailed spatial level – resolution of weather data is 3 km, topography

data 100 meters, soil types; weather stations density is variable. This would produce very high-

resolution zoning.

3. Fulfilled: Would link weather to land slope (and vegetation) and weather station data.

4. The idea will create new datasets – based on existing ones. This can be used for various purposes

(frost protection, crop selection, risk management).

5. Very well Fulfilled: Application will be for any crop (and either non-ag uses, like roads).

6. Fulfilled. Application will be applicable to any crop and area.



Decision: Describe a scenario for this idea.

The ideal case would be that existing datasets are linked on the fly, so no need for a new dataset

is created.

2.1.5 Looking for climatic patterns changes

This idea is about processing a spatiotemporal analysis of temperature extremes, precipitation, soil

moisture, drought and other phenomena to detect changes in climatic patterns.

Table 5: Relevance of Looking for climatic patterns changes idea to criteria for further elaboration

Criterion

No


1. The idea satisfies this criterion, since there is no existing database for climate change patterns.

2. Mostly fulfilled: idea operates on average spatial level – resolution of long-term weather data is

30-15 km.

3. Partially fulfilled: Would link different weather data sources – but does not really need other

databases – unless linked to previous topic 2.1.5.

4. The idea will create new datasets – based on existing. This can be used for various purposes (frost

protection, crop selection, risk management).



Decision: Describe a scenario for this idea: Idea fulfils all criteria to a large extent. The ideal case would

be that existing datasets are linked on the fly, so no need for a new dataset is created.

2.1.6 Information support for field use recommendations

This idea is about an expert system that is linking spatial datasets (weather, soil, zones) to provide actual

use recommendations and past compliance checks for single fields and crops. This idea is focused

mainly on fields (in opposite to next idea 2.1.7) and is based on processing regional datasets in

combination with local in-situ measurements that are making results more precise. Datasets can be used

in models to output estimate of crop height or density, as well as on soil water content and actual weather

situation. This information is important for a) planning of management interventions in the field, b)

accessibility of the field with heavy machinery like harvester, manure spreader, base fertiliser spreader,

c) soil situation and water content. Actual weather at the field is needed to decide about pesticide sprays

or fertiliser applications to be possible at a particular time.

This idea is deemed to be the one with most potential, because it can be expected (in the best case) to

be used on each field of the EU for around 3-10 times per season, generating enormous aggregated

benefits by multiple improvements in management practices.



Table 6: Relevance of information support for field use recommendations idea to criteria for further

elaboration

Criterion

No


1. The idea satisfies this criterion, since there are no existing databases for use patterns.

2. Idea operates on quite detailed spatial level – resolution of weather data is 3 km, topography data

25 meters, soil types; permission areas, weather stations density is variable. This would produce

very high-resolution zoning.

3. Fulfilled: Would link weather to land slope, soil type, (and vegetation), soil restriction data.

4. The idea will not create new datasets – it will be based on existing. This can be used for various

purposes (spraying and fertilising planning, compliance checks, event mapping, crop analysis,

risk management).



Decision: Describe a scenario for this idea: Idea completely fulfils all criteria and appears seems to be

feasible, no existing solution yet. The idea would enable application in routine farm management

as well as monitoring and compliance check, based on independent, properly maintained datasets.

2.1.7 Effective utilization of natural resources

Main goal of this idea is to prepare a system providing analysis and appropriate visualization of spending

of natural resources to farmers and agronomists. In opposite to previous idea (2.1.6), this idea is more

machinery-centric and resource-centric. The idea is based on collecting telemetry data from machinery,

make detail evidence of field interventions and utilization of inputs on farms (energy, fuel, fertilizers,

pesticides etc.) and collecting agro-meteorological observations directly on fields. On the other hand,

this system can combine these collected data with other datasets like EO data, local geographical data

(DEM, soil, hydrological data etc.), and data from LPIS or records from other farm management

information systems (FMIS). Analyses based on integration of such datasets with appropriate

visualization of results can support agronomists and farmers in decision-making processes to utilize

natural resources more effectively.

Analyses of machinery trajectories during interventions on fields that are interwoven with interventions

planning based on the calculation of yield productivity zones from the satellite EO data can decrease

consumption of inputs and can improve effective utilization of resources on farms.

Analysis of soil status and weather forecast for next several days with knowledge of necessary time for

fertilizing can recommend appropriate day to start of fertilization before rain.

Table 7: Relevance of Effective utilization of natural resources idea to criteria for further elaboration

Criterion Relevance to the criterion



No

1. Idea is using available datasets - satellite data, DEM, LPIS, OLU

2. Idea is the added value will be in decision-making support

3. Analytical and visualization tool will link together different data sources

4. It can use existing algorithms for calculation of yield productivity zones

5. The idea is focusing on the way of analysing and visualizing integrated data

6. The idea can be applicable in different regions

Decision: Elaborate this idea further, because it is describing analytical system as a support for

agronomists in the decision-making process. It can be elaborated up to pilot definition.

2.2 Elaborated scenarios

This section further describes selected ideas, turning them into scenarios. Please note that the idea 2.1.2

is not further elaborated and therefore there is a numbering shift in the scenarios!

2.2.1 Open Land Use Map Improvement scenario

2.2.1.1 Description

This scenario intends to build Open Land Use Map for farmers. The basic unit of the dataset is a farmer’s

field. Usually farmer grows certain culture on a field and manages fields separately. For the big yield of

a certain culture – certain variables and their values are important. The aim of this pilot thus is to collect

the field borders and relevant information associated with the fields. Then based on the data – API for

selection of relevant data for the field will be created.

Variables that could be of an interest of farmers are:

1) Geomorphological information (aggregated information: minimum/maximum/median/mean

elevation, height, slope, orientation; raster maps of the field in 25m resolution displaying

elevation/contour lines/orientation/slope)

2) Climatic data ~ here it is good to integrate with meteoblue API to allow a farmer to get various

information of his interest – this could be for instance general information – average annual

temperature, average annual precipitation, but also more specific information: number of

growing degree days for barley or the average temperature for harvesting period (August-

September). All this queries are possible with meteoblue API.

3) Hydrologic data (queries that allow farmer to display water bodies in the certain buffer zone

from the field, also data about percentage of the field exposed to water erosion)

4) Information about vegetation index at field (aggregated information:

minimum/maximum/median/mean of index; raster map; comparison of the mean index value to

the fields in x km radius. Data source for this could be either Sentinel 2 imagery or directly it

could be taken from sen2agri (sen2agri calculates Leaf Area Index(LAI) by default.

Additional data could be added on the request.



It is worth mentioning that vector data about field borders and water bodies is not available everywhere

in Europe. Therefore, it will be necessary to train classification model that will be able to recognize

water bodies and field boundaries based on Sentinel 2 imagery.

At the first stage it is planned to put an effort to create API that would serve other pilots for querying

the relevant information about the fields, as well as it could be a product that could be sold to third party

developers/product managers (that develop products for farming).

In the second phase it is planned to create a web application that would allow farmer to log in, add his

fields and use power of API through GUI.

2.2.1.2 Specifications

At the beginning, there will be discussions with the other scenarios to decide on the exact additional

attributes OLU needs to be extended for. Also interviews with stakeholders will help to decide to which

exact attributes fields need to be extended. Independently of this, the model to identify field borders and

water bodies from Sentinel 2 imagery will be created. Also it will be discussed with meteoblue how to

integrate their API into our solution. In the last year of the project the web application that provides GUI

for farmers will be created.

2.2.1.3 Data sources

• Open Land Use map

• EU-DEM

• Sentinel 2

• meteoblue API

• Open Street Map

• LAI (leaf area index) from sen2agri (or if does not work out it could be replaced with NDVI

index that will be calculated directly from Sentinel 2 data)

• Soil maps (need to do data research in which countries this data is available)

• Tabular data (growing degree days for different cultures)

2.2.1.4 Users

• Initial product will be API for getting information about the certain field described above. From

this follows that in the beginning the main users of the pilot will be other scenarios and

developers/product managers that develop tools for precision farming.

• In the second phase when web application that allows to access API from GUI will be created.

The users of this application will be primarily farmers.

2.2.1.5 Challenges

There will be three main challenges: technology, data availability, finding customers. In terms of

technology it will be challenging to develop models for identification of water bodies and farms borders.

In terms of data availability – it is possible that crucial information that farmers need are not available

as open data (soil type for example). Also it is possible that we won‘t find users outside our project.

Those are the natural risks, that could be encountered not just in this pilot.



2.2.1.6 Benefits

The main benefit is in having harmonized big datasets with LPIS fields and attached to the field’s

attributes relevant for farming. This will make life for the people who would want to reuse the solutions

from the project pilots a lot easier.

2.2.2 Monitoring of crop status scenario

2.2.2.1 Description

The monitoring system will include a crop anomaly detection component based on specific Sentinel 2

image analysis algorithms to monitor crop condition and presence of anomalies as deviation in the

spatial distribution of indices that are related to health condition like CRI2 and NDVI. It will then

attribute anomalies to stress, disease, or insect by hyperspectral data and other data sources. The indices

of Sentinel-2 will demarcate the possible disease spots for wide areas. The hot spots will be scanned by

hyperspectral camera on a UAV to identify exactly the disease or stress and not just the anomaly of the

S-2 multispectral indices that work at coarser resolution than the hyperspectral indices, which is UAV

mounted. The hyperspectral indices have to capability of exact identification of the type of disease or

stress.

The methodology as presented in D5.1, page 27 will use anomaly detection in the spatial distribution of

the CRI2 index. Zartaloudis et al. (2015) [9] showed that the CRI2 is the most appropriate spectral index

for determining the early stress of olive trees by verticillium wilt. The CRI2 is an index that estimates

the carotenoid content using the mathematical difference of reciprocal reflectance in the red edge and

the reciprocal reflectance in a band centered at 510 nm. This difference is also multiplied by reflectance

at NIR. The scope of this index is to remove the effect of chlorophyll on the estimation of carotenoids,

as carotenoids and chlorophylls absorb in the blue (400 to 500 nm) and thus it is difficult to estimate

carotenoid concentration independently from chlorophyll concentration using nondestructive techniques

(Gitelson et al., 2012) [10]. According to Gitelson (2012) [10] the band in the red edge is sensitive for

determining the chlorophyll content of the plants, because the depth of the light penetration is found to

be higher in the red edge compared to blue and red bands (the light absorption by the chlorophyll is

reduced).

Input data:

Sentinel-2 L2A images are available from the DIAS facility (Copernicus Data and Information Access

Services), and more precisely the facility operated by Atos that is Mundi.).

Processing methods:

Carotenoid Reflectance Index 2 calculation

CRI2=(1/R520-1/R700)×R800

R520, R700 and R800 is the reflectance in the spectral regions of 520, 700 and 800 nm, corresponding

in the green, red-edge and NIR bands respectively. Specifically, in Sentinel-2 bands, the CRI2 equation

would appear like this:

CRI2=(1/Band3-1/Band5)×Band8

Verticillium wilt risk probability 0 or 1 levels calculation proportional to CRI2 is derived from

an equation similar to:

Infection = f (CRI2), as a binary function 1=infected, 0=healthy

This equation has been tested with UAV data in Greece (Zartaloudis et al., 2015) [9] and will be further

developed for Sentinel-2 within EUXDAT.



The optimal configuration foreseen is

Infection = f (CRI2, UAV-HS, semantic annotation), where semantic annotation is annotations of

canopy spots as real disease presence by expert pathologist or similar.

UAV-HS is hyperspectral described in D3.1, and other data correlated to the infection

Output data:

The result will be a map with the spatial distribution of possibly infected olive orchards across the

study area, repeated with every Sentinel-2 overpass every 6 days.

The crowdsourcing reference data will be mainly expert annotations regarding the actual presence of

disease.


The goal is to generate local risk maps by combining:

• Crop anomaly detection by indices of Sentinel-2

• Hyperspectral signatures of crop and soil

• Meteo data, soil moisture (transmission risk)

• Crowdsourcing data (expert annotation as reference or any surrogate method that can provide

this ground truth)


• Level-1C multi-spectral imaging products from the Sentinel-2

• Sensor data from the field sensors including UAV-enabled hyperspectral imagery

• Data from the hyperspectral and soil-condition sensors in combination with EO data

• Meteorological data, soil moisture

• Crowdsourcing data (smartphones on observations and photos)

2.2.2.4 Users

• Mainly, crop protection specialists, farmers, counsellors or technical farm organisations

• Frequency of use: many per year, to make protection decisions, data queries could be local to

regional (comparisons of several sites)

• Usage pattern: few queries with heavier data use (spatial queries)

2.2.2.5 Challenges

• Find the right algorithms for differentiate the reason of anomaly in crop signatures based on all

sources of data together

• Combine meteorological and soil moisture data for future risk

• Build trust in the farming community, that so these data are actually reliable enough to base

decisions upon

2.2.2.6 Benefits

• User-friendly interface that alerts farm advisory services timely in order to manage field actions

• Cost of phytochemicals is reduced as the applications are site-specific and smaller quantities are

sprayed



• The environmental footprint is reduced by avoiding unnecessary sprayings

2.2.3 Delimiting Agro-climatic zones scenario

2.2.3.1 Description

Today, the maps of climate zones are very generic: they show large areas and display some differences

in topography. Things like seaside buffer zones, weather divides and South-North differences are not

shown.

The Agro-climatic classification system will allow classification of land areas in agro-climatic classes

for crop production and management, based on long-term climate data, land cover and topography. It

will allow establishment of classifications such as:

• Frost (Date of last spring frost, date of first fall frost, average length of frost-free season).

• Growing degree days: average /minimum /maximum growing degree days (base 0°C, 3°C, 5°C,

8°C, 10°C, 15°C) for a given site /area.

• Precipitation volumes: average /minimum /maximum (mm) for a selected period on a given site

/area.

• Soil moisture volumes: average /minimum /maximum number of days with saturation, 50% field

capacity, below wilting point water reserve for a site / area.


The goal is to generate local climate maps which take into account:

• General weather conditions (large-scale weather models)

• Local topography, with North/South slopes

• Buffer effects, such as lakes, sea or swamps

• Soil types.

The existing databases (see DATA SOURCES) will be analysed, and downscaled for special local

effects, such as cold air flows, lakeside buffer effects, north and south slope effects.

A first test was made with temperature zones as a function of altitude and hydrology (water bodies). The

results showed a substantial improvement in detail for the area analysed.

Figure 1 Temperature zones



The concrete algorithms are documented in the presentation Jedlicka_Hajek-EUXDAT-V1 Experiments.pptx.

These algorithms need to be validated with real observations, which will be rather time consuming. The

project can be continued with competent partners.

Alternative applications will be:

1. Maps for temperature extremes: distribution of critical minima – maxima. The thresholds can

be set by users. A prototype historyPRO was demonstrated by meteoblue.com on 23.10.2018.

2. Maps for temperature sums for crop land on North/South slopes, based on topography,

insolation balances and local weather stations.

3. Queries for specific areas: selection of areas (districts, cropland, fields) with possible

downscaling algorithms based on 1 and 2.

4. Time series for extreme events, based on 3.


Detailed weather data, based on HRM/LAM (high resolution models/local area weather models), for

temperature, wind, cloudiness and precipitation.

• Weather datasets: ERA5 (ECMWF), NEMS30 (meteoblue).

• Topography maps: EU-DEM,

• Land cover / soil maps (JRC/Open Land Use Map)

o https://esdac.jrc.ec.europa.eu/resource-type/soil-functions-data

o https://esdac.jrc.ec.europa.eu/resource-type/soil-threats-data

o https://esdac.jrc.ec.europa.eu/resource-type/soil-point-data

o https://esdac.jrc.ec.europa.eu/resource-type/soil-projects-data

o https://sdi4apps.eu/open_land_use/

2.2.3.4 Users

• Mainly agriculture extension specialists, counsellors or technical farm organisations wishing to

make investment or varietal choice decisions (frost protection, irrigation, etc.)

• Potentially insurances and other financial institutions wishing to make decisions on quality and

risk of agricultural investments.

• Frequency of use: once per year, to make investment decisions, data queries could be local to

regional (comparisons of several sites)

• Usage pattern: few queries with heavier data use (spatial queries)

2.2.3.5 Challenges

• Find the right algorithms for downscaling; Several algorithms proposed and tested. Algorithm

definition is planned until December 2018 (meteoblue), Pessl Instruments)

• Distribution of information to end user groups: the history+ platform could be an existing option

(https://www.meteoblue.com/en/historyplus): this product ois already established in the market.

Further offers can be API services, integration into farm software, etc.

• Build trust in the farming community, that these data are actually reliable enough to base large

decisions upon. This can be done with

o Showing relation of simulations with measurements,

https://esdac.jrc.ec.europa.eu/resource-type/soil-functions-data

https://esdac.jrc.ec.europa.eu/resource-type/soil-threats-data

https://esdac.jrc.ec.europa.eu/resource-type/soil-point-data

https://esdac.jrc.ec.europa.eu/resource-type/soil-projects-data

https://sdi4apps.eu/open_land_use/

https://www.meteoblue.com/en/historyplus



o Integration of the services in services of established service providers (e.g. farm

software, insurances, weather services, others)

o Option for farmers to compare results with own weather measurements.

o Links to crop growth data from scenario 2.

• Build trust in the stakeholder community, that these data are actually reliable enough to base

large decisions upon - this seems the largest challenge, because such data cannot so easily be

verified, and decisions will have large (10’000 to several million €) impact.

o First tests show that results are of interest and visual verification generates substantial

confidence.

o Cooperation with providers of measurements and farm services will enhance

credibility;

o The growth of IoT will enhance ability of users to compare results.

• Maintain system operational with different data sources

o This is a generic problem of all EUXDAT services.

o Solution will be SLA, fallback options, and caching of data.

• Integrate new data sources once they appear.

2.2.3.6 Benefits

Such improvements in the climate zones would support local/within-field management strategies. For

researchers, it may be of interest to use this dataset for decisions related to field trial (climatic)

representativeness. Last but not least, researchers and advisors may find important to check the impact

of climate change on given area and decide about future management strategies.

Long-term decisions about crops will be better informed and resources used more efficiently if long-

term climate changes are known.

One example is frost protection: currently, significant parts of the Central EU Orchard and Vineyard

industry have been affected by late frosts, and have to make large decisions about anti-frost protection

measures, varietal changes and risk mitigation strategies.

Further examples are setting of minima and maximum thresholds, drought and flooding risk.

2.2.4 Looking for climatic patterns changes scenario

2.2.4.1 Description

Today, climate change is affecting most areas in Europe. The magnitude of the changes is not yet fully

known, however, the variability over the past 10 to 30 years is rather precisely understood.

This system will allow an assessment of trends, frequency distribution and extremes for the major

weather-related variables (temperature, precipitation, evapotranspiration, soil moisture) for agricultural

zones. It can link to the other system (2.1.4. Delimiting Agro-climatic zones scenario).

The differences to existing climate databases will be the following:

1. Analysis will be focussed on crop land (filter).

2. Analysis will be focussed on cropping seasons (filter applicable for different crops, e.g. summer

crops, winter crops, perennial crops).



3. Analysis will allow users to upload own records (e.g. temperature measurements) to benchmark

regional model data.

4. The service can (but does not have to) link to existing climate change models which quantify

future changes (https://www.eea.europa.eu/themes/climate/dc)

This service will create a unique ability to assess potential impact of climate change on local production.


These extremes change over time, and the mapping exercise would consist of defining long-term indices

for change (frequency of frost or heat, heavy rainfall or drought, storms) and plotting them on maps.

Local detail is not of such concern in these maps, since large-scale frequency changes occur over longer

periods of time and larger areas and are not subject to local micro-climate.

A prototype historyPRO was demonstrated by meteoblue.com on 23.10.2018.

Details must still be worked out, because this pattern change so far only uses weather input and does not

yet connect this input with other data.

Options to enhance the service:

1. Upload weather station data (for 1-2 years) to validate sources for long-term data.

2. Downscale simulations using topography/slope.

3. Integrate climate risk with soil data, e.g. impact of drought on soils with san, silt, clay, or peat

content.

4. Insert crop growth thresholds to show impact of climate variations on different crops.

5. Simulate effect of climate on different crops (link to scenario 2).


• Climate data everywhere, in highest possible resolution and minimum 30 years from single

source, to be comparable (e.g. from models GFS, NEMS30...)

• Land use map to limit query to agricultural land (open land use map, 2.1.1).

• Optional: Topography map to adjust risk calculations to altitude.

• Optional: Water body map - to assess buffer effects from lakes.

• Climate change database: https://www.eea.europa.eu/themes/climate/dc

• User datasets (temperature, precipitation measurements): requires upload function.

2.2.4.4 Users

• Mainly, agriculture extension specialists, counsellors or technical farm organisations wishing to

make investment decisions (frost protection, irrigation, etc.)

• Frequency of use: once every 1-5 years, to make investment decisions, data queries could be

local to regional (comparisons of several sites)

• Insurance companies assessing banks

• Usage pattern: very few queries with heavier data use (spatial queries)

2.2.4.5 Challenges

• Find the right query delimiters for long-term weather risk

https://www.eea.europa.eu/themes/climate/dc

https://www.eea.europa.eu/themes/climate/dc



a. Several queries tested and validated (frost risk, heat risk, drought risk, length of

drought, growing degree day variation between years), variable planting dates.

• Find enough use cases;

b. The queries are being increasingly requested by farm service providers, indicating

good demand.

c. Seasonal comparisons are the most frequently used.

d. Addition to farm software seem promising.

• Build trust in the stakeholder community, that these data are actually reliable enough to base

large decisions upon - this seems the largest challenge, because such data cannot so easily be

verified, and decisions will have large (million €) impact).

e. Showing relation of simulations with measurements,

f. Integration of the services in services of established service providers (e.g.

farm software, insurances, weather services, others)

g. Option for farmers to compare results with own weather measurements.

• Build trust in the farming community, that these data are actually reliable enough to base large

decisions upon.

h. Cooperation with providers of measurements and farm services will enhance

credibility;

• Integrate new data sources once they appear.

2.2.4.6 Benefits

• Stakeholders can query their location and find frequency of local weather extremes.

• Stakeholders can compare such extremes to places where comparisons with measurements exist.

• Stakeholders can analyse their own cropping success comparing with weather patterns and

extremes in particular seasons and find similar impact of suitable practices. This is known to

work for fertilisation, variety selection and other management decisions.

2.2.5 Information support for field use recommendations scenario

2.2.5.1 Description

The system is designed as an expert system supporting decision making for field interventions planning

and actual use recommendations. The system will link spatial datasets (actual weather and forecast, soil

data, land cover type, management zones etc.) to provide actual use recommendations and past

compliance checks for farmers for individual fields and crops. Such information is important for

planning of several actions: a) management interventions in the field, b) accessibility of the field with

heavy machinery like harvester, manure spreader or base fertiliser spreader, c) soil situation (e.g. NPK

and water content). Actual weather at the field is needed to decide about pesticide sprays or fertiliser

applications to be possible and effective at a particular time. This scenario is mainly focused on fields

and crops protection.




The following list shows concrete applications, made possible by queries to the designed expert system.

Note that all applications rely, among others, also on soil moisture in-situ data available from PESSL‘s

instrumentation at various depths (10 to 100 cm). As so, these services can be provided only for fields

where this or similar soil moisture instrumentation is installed.

• Can a tractor enter the field? – YES/NO. Answer based on:

▪ soil level = using artificial intelligence to classify the soil moisture level relative

to field capacity into i.e. very dry, dry, wet, very wet (Pessl Instruments)

▪ Crop coverage and phenological status using Leaf Area Index (LAI) from

Sentinel-2;

▪ machine type;

• Input: map of soil level, crop coverage/phenological status, machine type, water usage

zones (buffer strips), weather, crop protection products (optional), NPK in-situ

measurements (optional).

• Output: yes/no zone, “time-of-the-year” windows,

• Description: application of mineral nitrogen fertiliser is recommended with 50-90%

soil saturation. The query will for “today” (e.g. 10.05.2018) will show the field to be

too dry. A query for 5 days later (e.g. 15.05.2018) will show the field to be moist

enough, because rainfall is forecast on day 4. (Pessl Instruments)

• Can a harvester enter the field? – YES/NO. Answer based on:

▪ Soil level (as above)

▪ Crop coverage and phenological status

▪ Crop moisture (i.e. too moist vegetation damages the harvester)

▪ Map of hourly traffic ability, for next 6-7 (or past): yes/no zones, different crop

zones (?)

▪ machine type;

• Input: map of soil level, crop coverage/phenological status, in-situ vegetation wetness

(were available, optional), weather, machine type.

• Output: yes/no recommendation, “time-of-the-year” windows,

• Description: Harvesting is not recommended for very wet crop. The query will for

“today” (e.g. 10.05.2018) will show the field to be too wet (shortly after rain). A

query for 5 days later (e.g. 15.05.2018) will show the field to be dry enough, because

there was no rainfall for the 4 days. (Pessl Instruments)

• Crop protection recommendation/spraying? – YES/NO. Answer based on:

▪ Crop coverage and phenological status (Leaf Area Index (LAI)) from Sentinel-

2

▪ Soil level

▪ water usage zones (buffer strips, optional)

▪ weather

▪ crop protection product restrictions (by product class, optional)

▪ machine type;

• Input: map of soil level, Crop coverage and phenological status, water usage zones

(optional), crop protection status (optional), weather, machine type.



• Output: yes/no recommendation, “time-of-the-year” windows,

• Description: spraying is recommended at low crop stage and dry condition to avoid

soil compaction and run off of the herbicide. The query for “today” (e.g. 10.05.2018)

will show the field to be too wet with 50 – 90%. A query for 5 days later (e.g.

15.05.2018) will show the field to be dry enough and still at low development stage,

no rainfall falls for 4 days. (Pessl Instruments)

• Application map for fertilization

▪ availability of LAI or NDVI from Sentinel-2

▪ Soil samples from Mobilab mobile device

• Description: Generate management zone maps based on long-term heterogeneity of

the field using historical Sentinel-2 data acquired during vegetative period. Assign

fertilizer application values per zone given the allowed maximal amount or using the

indications on the level of NO3, K and PO4 using Mobilab device.

• Start irrigation to prevent damage of crops by radiation frost

▪ Meteorological measurements in situ

▪ Local weather forecast

• Description: To prevent damage by radiation frost is necessary to monitor phenomena

that are indicators of radiation frost and to be able to provide early warning in time

that any counteractions can be managed.

• Start ventilation in greenhouse to prevent overheating of crops

▪ Meteorological measurements in situ

▪ Local weather forecast

• Description: To prevent overheating of crops in greenhouses is necessary to observe

temperature and local forecast how quick the temperature will increase. It is expected

an early warning mechanism to warn before reaching temperature threshold and what

amount of time left to this threshold.


• Volumetric soil moisture data available from PESSL’s in-situ stations

• Weather datasets: these are available for forecast, as well as for historic data. ERA5 (ECMWF),

NEMS30 (Meteoblue).

• Level-1C multi-spectral imaging products from the Sentinel-2 available at field scale

• Dynamic cropland mask, crop type map and LAI from Sen2Agri system

• Corine Land Cover

• Soil maps (JRC) (optional): in particular, these

• https://esdac.jrc.ec.europa.eu/content/european-soil-database-v20-vector-and-

attribute-data

• https://esdac.jrc.ec.europa.eu/content/maps-storing-and-filtering-capacity-soils-europe

• https://esdac.jrc.ec.europa.eu/content/maps-indicators-soil-hydraulic-properties-

europe

• https://esdac.jrc.ec.europa.eu/content/lucas-2009-topsoil-data

• in-situ vegetation wetness from Pessl’s instrumentation (optional)

• NPK in-situ measurements (optional)

https://esdac.jrc.ec.europa.eu/content/european-soil-database-v20-vector-and-attribute-data


https://esdac.jrc.ec.europa.eu/content/maps-storing-and-filtering-capacity-soils-europe

https://esdac.jrc.ec.europa.eu/content/maps-indicators-soil-hydraulic-properties-europe


https://esdac.jrc.ec.europa.eu/content/lucas-2009-topsoil-data



• Optional: Water catchment areas: these datasets are available from some national data sources

and may not be consistent.

• Optional: Land use restrictions (water catchment area, limited fertilisation, others).

2.2.5.4 Users

• Farmers, checking sites for local conditions to work on field (tractor passage, soil preparation,

sowing), spraying (crop protection), fertilisation (mineral, organic), harvesting, to determine if

their applications are efficient, correct and lawful, warning system preventing damage of crops;

• Agronomist: checking sites for compliance of operations of spraying (crop protection),

fertilisation (mineral, organic), to make usage recommendations for areas;

• Counsellors or technical farm organisations wishing to make crop management

recommendations (frequency of application windows for an area).

• Regulators, to check if past applications were correct and lawful (the tool could even be used as

compliance check to grant subsidies).

Frequency of use: 10-20 times per farm and year, to make operational decisions, data queries would be

mostly local (single sites)

• Usage pattern: many queries with few data use (point queries)

• Many use cases possible

2.2.5.5 Challenges

• Prove the benefit of such applications to farmers, counsellors and regulators

• Link the datasets from various sources

• Building fast queries for quick data retrieval

• Refine use cases by user group feedback

2.2.5.6 Benefits

Such recommendations would support local/within-field management strategies. It will allow:

• Farmers and extensionists to find suitable timing for operations.

• Agronomists and extensionists can check compliance of past operations.

• Farmers and agronomists can document compliance of field operations.

• Stakeholders can compare actual practices to the recommendation or permissions available for

that particular field.

• Stakeholders may add own decision criteria to the data output.

In summary, compliant operations will increase energy efficiency of operations, by increasing yield per

unit of input, improve environmental compliance by reducing ineffective applications, and improve

profitability for farmers by reducing waste.



2.2.6 Effective utilization of natural resources scenario

2.2.6.1 Description

The scenario is focused on preparation of analytical and visualization tool for natural resources spending

and inputs (fertilizers, pesticides, fuel, electrical energy, feed material, etc.) on specific farm for

agronomists and farmers. This scenario will utilize results from other scenarios – OLU improvement

(chapter 2.2.1), Delimiting of Agro-climatic zones (chapter 2.2.3) and Information support for field use

recommendations scenario (chapter 2.2.5) and will integrate results under common application and

analytical possibilities.

The system will integrate data sets from different sources and provide analyses based on these data. One

of the data sources will be telemetry data from machinery, another one is evidence of field interventions

and utilization of inputs and last but not least collected observations from agro-meteorological sensors

as well. This scenario will focus on improving efficiency by reducing inputs while maintaining

production at the current level or reducing the production only slightly. The system will integrate data

from machine telemetry, local sensors, EO data, local geographical data, LPIS and records from local

FMIS.

One way to achieve cost savings without significant negative impact on production or even with a

positive impact on production is to optimize the trajectory of tractors and adjust the interventions taking

into account all possible relevant information about the field that can be obtained.


• Integration of heterogeneous data from different sources

a. LPIS, sensor and telemetry data can be combined to provide analyses on effectivity on

plot or management zone level

b. Satellite data and LPIS can be combined to provide analyses on yield productivity zones

based on long term vegetation indexes and to provide recommendations for fertilization

or pesticide application maps.

• Calculation of yield productivity zones

c. Zones can be calculated from satellite data in combination with local yield data from

harvesting and long-term satellite monitoring. By defining zones, it can be provided

zone-specific interventions planning mechanism.

• Analysis and visualization of results by utilization of modern technologies

d. Cluster analysis, data mining for analyses – searching for correlations and relations

e. Heat maps, cartograms for visualization – advanced visualization for delivery

information to target user group

• Support decision-making process and intervention planning

f. Defining rules and conditions by discussions with agronomists – support to derivate the

appropriate knowledge from information

g. Calculation of probability of upcoming weather events – providing early warning

system for intervention planning+




• Satellite data (Sentinel 2): Level-1C multi-spectral imaging products from the Sentinel-2

available at field scale

• DEM: EU-DEM and local DEM with higher resolution where available

• LPIS: geometry of plots with corresponding attribute from private LPIS system for target farm

• Machinery telemetry: telemetry data from active machinery with appropriate frequency of

collecting and details about utilization of machinery engine

• Agro-meteorological observations: local in-situ measurements in fields and local meteo stations

• OLU: improved OLU map, with water bodies and restriction areas

• FMIS records: data from target farm about interventions, spending fertilizers, pesticides

• Weather datasets: these are available for forecast, as well as for historic data. ERA5 (ECMWF),

NEMS30 (meteoblue).

2.2.6.4 Users

• Agronomists – can utilize analyses produced by scenario as data sources for their further

processing

• Farmers – can utilize analyses and visualization as support for decision-making process in

combination with own experience

2.2.6.5 Challenges

• Integrate appropriate data

• Provide analyses with appropriate granularity

• Provide visualization suitable for decision-making

• Collecting of all proper telemetry data

2.2.6.6 Benefits

• Integration of different data on one access point

• Analyses based on user demands

• New visualization trends

2.3 Precision farming ideas and scenarios summary

As can be seen there are 6 scenarios identified and described. The Open Land Use Map improvement is

related to data infrastructure. Agro-climatic zones and Climatic pattern changes are related to strategic

level of precision farming planning. Monitoring of crop status has actually two tiers: both strategic and

operational. Field use recommendations are naturally related mainly to operational level, but can take

advantage of outputs of strategic scenarios.



3 Proposed pilots

Pilots, as described in EUXDAT proposal, are technical pilots ~ the vision of each pilot firstly describe

its technological nature and then maps the pilot to above identified scenarios with business potential.

The pilots are planned to develop in three phases, according to three versions of e-infrastructure

described in EUXDAT proposal (see fig. 3.1):

1. Additionally, to operational software from the consortium, there should be prepared mock-ups

and initial designs for each component planned to use in the V1 version of EUXDAT e-

infrastructure.

2. V2 version should be hosted on a cloud and consist of working prototypes (composed of

orchestrated components) serving to the purposes of the pilots.

3. V3 e-infrastructure needs to be composed from fully functional tools, applications and services.

Figure 2: Planned releases for the proposed e-Infrastructure, according to EUXDAT Grant Agreement [2]

3.1 Pilot 1: Land Monitoring and Sustainable Management

3.1.1 Vision of the pilot

Olive tree cultivation accounts for nearly 2 billion euros in annual net income. The farmers are not able

to be proactive in their efforts against crop diseases that result in yield damage because dedicated,

comprehensive, and reliable services are currently missing. Extensive cultivations are hard to be

continuously and effectively monitored from the ground. The monitoring system will include crop

anomaly detection component based on specific Sentinel 2 image analysis algorithms to monitor crop

condition, presence of anomaly as a deviation. Then, it will attribute the anomaly to stress or disease or

insect by hyperspectral data and other data sources. By interpreting sensor inputs (e.g. plant health,

nutritional status, weed detection, soil health) and their correlation to ground truth, the service will also

provide maps of risk for possible future occurrence of infections at the orchard level. The service will

be available to farmers, agronomists, agro-consultants and regional agricultural ministries, which should

all work together in order to reduce infections. Verticillium Wilt (VW) is an elusive olive tree disease



and thus is difficult to be traced by simple human observation, especially at early stages. As a result, the

farmers either are not reacting at all, or they are following ad hoc solutions in order to protect their trees

from VW. The scenario is to develop a system for a holistic management. The system will include an

early detection component based on specific image analysis algorithms to monitor tree condition,

presence of VW infestation and its level. Also, the precise application of novel eco-friendly formulation

(e.g. inducing-resistance agents) will be targeted.

The service will provide maps of actual infections of the targeted olive plantations at the tree level.

Using these maps and the new treatments, farmers and agronomists will be able to be more efficient and

effective in their detection and treatment of VW (Sentinel 2 multispectral indices). Identification will be

achieved using UAV-enabled hyperspectral imagery and its correlation to ground truth.

3.1.2 Used Datasets description

• Level-1C multi-spectral imaging products from the Sentinel-2

• UAV-enabled hyperspectral imagery

• Meteo data, soil moisture (transmission risk)

• Crowdsourcing data (smartphone user annotations, photos)

3.1.3 Pilot contribution to defined scenarios

• This pilot relates to Monitoring of crop status scenario (section 2.2.2) by providing information

about stress status of crops, identifications of stress cause and calculation of risk of spread of

infection.

• This pilot relates to Field use recommendations scenario (section 2.2.5) by providing

information about field interventions from monitoring data and from farm management

information system.

3.2 Pilot 2: Energy efficiency analysis


As described in the EUXDAT Proposal, the energy sector is experiencing an increasing tension between

safety and cost-saving requirements. Of late, the need to ensure Europe’s long-term strategic interests

became another factor contributing to this tension. Agricultural production is influenced by and

influences lot of factors, e.g. ecosystems, economic factors, water quality and of course energy use and

supply. The effectiveness of agricultural production is determined by the ratio of the value of the

production outputs to the value of production inputs. The efficiency of the production is affected not

only by the internal factors of the production process, but also by external factors such as climate,

subsidies, the situation in the global market, among others. Acquiring knowledge about the energy and

carbon intensity of different crops on different lands, about the way farm processes work and how to

take care of the variability within fields in a single farm is a very demanding task. ICT technologies are

needed to collect sensitive data and evaluate it in the most accurate way, because any optimization

process cannot be performed without sufficient and objective knowledge.

The pilot will be focused on collecting and integrating data from different data sources (in-situ sensors,

machinery fleet monitoring, farm management systems, weather data etc.) on one side. And it will focus



on providing analyses and visualizations of analysis results as a support for the decision-making

processes of the farmers and agronomists on the other side.

Analyses of integrated data and data sets will be based on statistical, geoprocessing as well as Big Data

methods. The pilot will use exploratory visualisation methods, combining different types of data. The

exploratory visualisation involves an expert creating maps and other graphics while dealing with

relatively unknown geographic data. While working with the data, the expert should be able to rely on

cartographic expertise to be able to view data from different perspectives.

3.2.2 Used datasets description

The pilot will build mainly on collecting of agrometeorological data from local in-situ sensors and of

machinery fleet monitoring data. Other data sources for analyses will be Copernicus Sentinel 2 data,

potentially also Sentinel 1 and Copernicus Digital Elevation model, Landsat 8 data and available indexes

(NDVI, LAI) and yield potential derived from this data, Open Land Use Data, Land Parcel Information

System data, Soil Maps and National Digital Elevation Model. Pilot will also include historical

meteorological data from regional level. Standalone data source will be data from local farm

management information system of each farm.


This section shortly summarizes the pilot relation to defined scenarios.

• Effective utilization of natural resources (section 2.2.6) - provide analytical and visualization

functions based on integrated datasets from different sources

• Field use recommendations (section 2.2.5) - providing expert system about field interventions

planning from regional datasets and local agrometeorological data

• Agro-climatic zones scenario (section 2.2.3) - providing information of local observations from

in-situ and fleet monitoring sensors, providing information about field interventions on crops

and possibilities of weather extremes.

3.3 Pilot 3: 3D farming


As described in the EUXDAT Proposal, 3D precision farming is a new approach and will help to better

plan and manage a farm production. Current practically utilized precision farming systems are working

only with 2D data analysis and 2D visualization. There are new approaches focusing on 3D, using

different types of DEM including UAV. However, no one incorporates usage of the 3D visualization

and analysis of existing data, even if it can explain many problems inside the field. It all leads to soil

erosion prevention, nutrient leaching reduction and therefore to cost effective land management.

The pilot will do that on two levels: 3D analysis and 3D visualization.



3.3.1.1 3D analysis

The main goal of 3D analysis lies in categorizing fields into different productivity land/zones (with

appropriate humidity, slope and aspect, rich to fertilizers, well accessible, etc.). This goal will be

achieved by basic analytical steps combined together to better understand conditions on the fields as it

is known, that elevation has a large influence distribution on water, nutrients and soil particles in the

field. Using of spatially variable distribution of chemicals according to identified zones during

machinery movement and even in longer period for crop rotation will decrease costs of farm production

(e.g. by better soil protection and erosion control, etc.). As such, the 3D farming pilot is closely related

to Field use recommendations scenario. By adding average, mean, min and max, slope, aspect and

altitude characteristics to each particular zone, the pilot can contribute to Open Land Use Map scenario.

A calculation of water buffers, again the slopes and their orientation, watersheds delimitation and other

morphometric characteristics brings the local variability in Agro-climatic zones scenario.

3.3.1.2 Visualization of 3d data

The visualization of 3D data in a virtual environment (usually but not exclusively using perspective

projection on a common screen) can help to explore data in more natural way than a common two-

dimensional map. The 3D visualization will therefore be applicable to all scenarios as an option, but it

is expected that it will be used extensively in Open Land Use Map scenario as this is going to be a native

client of OLU visualization. Then the visualization of 3D data is going to be used for Field use

recommendations for interactive exploration of the geomorphometric situation of a farm fields (e.g. for

yield potential maps).

3.3.2 Used Datasets description

The pilot will build mainly on: Copernicus Sentinel 2 data, potentially also Sentinel 1 and Copernicus

Digital Elevation Model (DEM), Landsat 8 data and available indexes (NDVI, LAI) and yield potential

derived from this data Open Land Use Data, Land Parcel Information System data, Soil Maps,

hydrological layers and National Digital Elevation Models. We will also include historical

meteorological data. Usage of meteorological data in combination with yield maps will help us to better

understand dependencies among processes.


This section shortly summarizes the pilot relation to defined scenarios:

• Field use recommendations (section 2.2.5) - categorizing fields into different productivity

land/zones taking terrain morphometry into account.

• Open Land Use Map (section 2.2.1) - calculation of average, mean, min and max, slope, aspect

and altitude characteristics to each particular zone.

• Agro-climatic zones scenario (section 2.2.3) - calculation of water buffers, watersheds

delimitation, again slopes and their orientation and other morphometric characteristics.



3.4 Pilot scenario matching

This section recapitulates pilot relations to pilots via a matchmaking table below.

Table 8 Pilot to Scenario matching

Scenario 1 Scenario 2 Scenario 3 Scenario 4 Scenario 5 Scenario 6

Pilot 1 Pilot 2 Pilot 3



4 EUXDAT pilots and platform requirements

The design and specification of EUXDAT platform is based on two main sources of requirements:

i. Pilot specific requirements elicited and described using as basis the pilot descriptions provided

in section 3. These will be reported in section 4.1

ii. EUXDAT general platform requirements and other project-external requirements sources - such

as on-going and past experiences and initiatives agriculture-based projects dealing with large

amounts of data, Big Data architectures, etc. - providing a broader and more general view of the

requirements that EUXDAT platform should fulfil. These will be reported in section 4.2

In both cases, EUXDAT pilot specific and platform generic requirements will be subdivided in two main

categories:

1) Informational requirements, referring to all data sources of information necessary (by the

pilots and platform).

2) Functional and non-functional requirements, referring to features that the platform should

provide such as storage, query and visualization capabilities. Please, note that Non-

functional requirements can relate to aspects like Look & Feel, Usability & Humanity,

Performance, Operational, Maintainability & Support, Security, Cultural & Political and

Legal constraints.

In order to facilitate the requirements compilation, the following templates will be used (the templates

have been adapted from “Suzanne Robertson, James Robertson; ‘Mastering the Requirements Process:

Getting Requirements Right’; Addison-Wesley Professional, 2012.” [3]).

Table 9: Informational requirement template

# Id Unique identifier of a requirement according to the

following scheme:

EUXDAT-REQ-PILOT-DATA-<requirementID>

Priority Low/Medium/High

Name Name of the data source

Description Long description of the data source

Purpose Justify why the data source is required (e.g., to be used as input for a specific calculation)

Author Project partner (or external organization/project) that identified the use case/feature

Actors Actors involved in the requirement, taking into account stakeholders related to it.

Data Owner Institution who is the owner/provider of the data set

Data Access How to access the data set (url to ftp, http, OGC service, etc.)

Data

Licensing

Is the dataset free of charge or does it have any cost? What is its license model (if known)

Coverage Pilot area, country level, Europe, Global, etc. Priority area?



Data Format In which data format(s) is the dataset available (shapefile, json, geotiff, etc.)

Estimated Size (in MBs/GBs/TBs)

Metadata Provide link to metadata record (if available)

Relationships List here those requirements ids related to this one. For instance, requirements that are a

dependency for this one.

Table 10: Functional/non-functional requirement templates

# Id Unique identifier of a requirement according to the

following scheme for

i) specific pilot requirement: EUXDAT-REQ-PILOT-

<requirementID>

ii) platform and external requirement: EUXDAT-REQ-

PLATF-<requirementID>

Priority Low/Medium/High

Name Name for the requirement

Categories Will refer to categories in EUXDAT architecture (e.g.:

• "Cloud and High-Performance Computing (HPC)"

• "Data Management"

• "Data Processing"

• “Data Protection and Security”

• “Data Analytics”

• "Data Visualization and User Interaction"

• “Standards”

• etc.

Description Definition of the requirement. Describe what it is about.

Author Project partner (or external organization/project) that identified the use case/feature

Actors Actors involved in the requirement, taking into account stakeholders related to it.

For instance:

• Cloud providers

• HPC providers

• Data providers

• System integrator companies

• Agricultural services providers

• Farmers

• Developers

• Agronomists

• Data processing specialists

Validation

scenario

Determine some validation criteria which would check that the requirement is fulfilled

(i.e. small remark about the testing that should be done). It is necessary to have here some

indicative metric. Bear in mind it will be validated in the context of WP5.



Components

Include some mapping with the parts of the high-level architecture which are affected by

this requirement. In the first iteration, take a look at the components we mentioned in the

proposal.

Relationships List here those requirements related to this one. For instance, requirements that are a

dependency for this one.

4.1 Pilot specific requirements

The following subsections summarize the list of informational and functional/non-functional

requirements elicited from the three project pilots. More detailed information on each of the

requirements can be found in Annex 1 – Detailed pilot’s requirements,

4.1.1 Pilots Informational requirements

Table 11: Pilots Informational requirements

ID Name Format Size Priority Pilots

EUXDAT-REQ-

PILOT-DATA-

001

Level-1C multi-spectral

imaging products from the

Sentinel-2

Raster (Geotiff) Gb High

Pilot 1

Pilot 2

Pilot 3

EUXDAT-REQ-

PILOT-DATA-

002

UAV-enabled hyperspectral

imagery Raster (Geotiff)

Tb

Medium

Pilot 1

EUXDAT-REQ-

Pilots-DATA-

003

Climate data Raster Gb High

Pilot 1

EUXDAT-REQ-

Pilots-DATA-

004

Dynamic cropland mask,

crop type map and LAI from

Sen2-Agri system

Raster Gb High

Pilot 1

EUXDAT-REQ-

Pilots-DATA-

005

Copernicus European Digital

Elevation Model (EU-

DEM), version 1.1

Raster (Geotiff) Gb High

Pilot 1

Pilot 3

EUXDAT-REQ-

Pilots-DATA-

006

Land use map Raster Gb High Pilot 1

EUXDAT-REQ-

Pilots-DATA-

007

Soil map Raster Gb Low Pilot 1

EUXDAT-REQ-

PILOT-DATA-

008

Soil moisture data from

Pessl's instrumentation Xml file Mb High

Pilot 1



ID Name Format Size Priority Pilots

EUXDAT-REQ-

PILOT-DATA-

009

Open Land Use Map Shapefile, RDF GBs Medium

Pilot 2

Pilot 3

EUXDAT-REQ-

PILOT-DATA-

010

Land Parcel Identification

System (LPIS) Raster GBs Medium

Pilot 2

Pilot 3

EUXDAT-REQ-

PILOT-DATA-

011

Hydrology for EU Vector GBs Medium

Pilot 3

EUXDAT-REQ-

PILOT-DATA-

012

Actual weather Meteoblue

API Unknown Medium

Pilot2,

Pilot 3

EUXDAT-REQ-

PILOT-DATA-

013

Historic weather Unknown Unknown Medium

Pilot 3

4.1.2 Pilots Functional/non-functional requirements

Table 12: Pilots - Functional/non-functional requirements

ID Name Category Priority Pilots

EUXDAT-REQ-

PILOT-001

Atmospheric correction of

Multispectral Sentinel bands Data processing Medium

Pilot 1

EUXDAT-REQ-

PILOT-002

Enable calculation of

spectral indices from the 12

Sentinel multispectral bands

Data processing Medium

Pilot 1

EUXDAT-REQ-

PILOT-003

Calculation of Hyperspectral

indices relevant for stress

and disease


Pilot 1

EUXDAT-REQ-

PILOT-004

Availability of Sentinel-2

data at field scale/for a

given polygon for given

time period

Data Management

Data Processing High

Pilot 1

EUXDAT-REQ-

PILOT-005

2D visualization of time-

series over selected pixels,

provision of interfaces,

toolkits

Data Visualization and User

Interaction High

Pilot 1

EUXDAT-REQ-

PILOT-006

Installation of Sen2Agri

system and provision of

Dynamic cropland mask,

crop type map and LAI

Cloud and High-Performance

Computing (HPC) High

Pilot 1

EUXDAT-REQ-

PILOT-007

Enable statistics on multi-

temporal data for given

field, I.e. monthly averaging

of spatial datasets.


Pilot 1



EUXDAT-REQ-

PILOT-008

Collecting machinery

tracking data Data Management High

Pilot 2

EUXDAT-REQ-

PILOT-009

Collecting of agro-

meteorological data Data Management High

Pilot 2

EUXDAT-REQ-

PILOT-010

Calculation of yield

productivity zones Data Analytics High

Pilot 2

EUXDAT-REQ-

PILOT-011

Zone related morphometric

statistic


Computing (HPC)

Data Analytics

High Pilot 3

EUXDAT-REQ-

PILOT-012

Water influence to weather

conditions


Computing (HPC)

Data Analytics

Medium

Pilot 3

EUXDAT-REQ-

PILOT-013 3D visualization

Data Visualization and User

Interaction High

Pilot 3

4.2 EUXDAT Platform requirements

This section describes EUXDAT general platform requirements as well as other project-external

requirements sources - such as on-going and past experiences and initiatives agriculture-based projects

dealing with large amounts of data, Big Data architectures, etc. - providing a broader and more general

view of the requirements that EUXDAT platform should fulfil. Currently, inputs form the following

related projects were taken into account: FOODIE3, DataBio4 and MSO4SC5[6] .

The following table summarizes the general functional and non-functional requirements of EUXDAT

platform. More detailed information on each of these requirements can be found in Annex 2 – Detailed

EUXDAT Platform requirements.

Table 13: EUXDAT platform general functional and non-functional requirements

ID Name Category Priority

EUXDAT-REQ-

PLATF-001 Support for various HPC and Cloud providers

Cloud and High-

Performance Computing

(HPC)

High

EUXDAT-REQ-

PLATF-002 Monitor HPC and Cloud resources

Cloud and High-


(HPC)

High

EUXDAT-REQ-

PLATF-003 Applications monitoring and profiling

Cloud and High-


(HPC)

Medium

3 http://www.foodie-project.eu/ 4 https://www.databio.eu/en/ 5 mso4sc.eu

http://www.foodie-project.eu/




EUXDAT-REQ-

PLATF-004 Adequate operation of the platform

Cloud and High-


(HPC)

Low

EUXDAT-REQ-

PLATF-005 Optimize data movement

Cloud and High-


(HPC)

Data Management

High

EUXDAT-REQ-

PLATF-006

Support security and privacy in data

management

Cloud and High-


(HPC)

Data Management

Data Protection and

Security

Medium

EUXDAT-REQ-

PLATF-007

Automated deployment and execution of

applications

Cloud and High-


(HPC)

Data Management

Medium

EUXDAT-REQ-

PLATF-008 API access to pilots' data and services Data Management High

EUXDAT-REQ-

PLATF-009 User management

Data Protection and

Security High

EUXDAT-REQ-

PLATF-010 Access sensor observations

Data Management

Standards High

EUXDAT-REQ-

PLATF-011 Support information modelling

Data Management

Standards High

EUXDAT-REQ-

PLATF-012 Support integration of meta-information

Data Management

Standards High

EUXDAT-REQ-

PLATF-013 Compliance with INSPIRE specifications

Standards

Data Management High

EUXDAT-REQ-

PLATF-014 Compliance with GEO/GEOSS specifications

Standards

Data Management High

EUXDAT-REQ-

PLATF-015 Integrate Web map services

Standards

Data Visualization and

User Interaction

Data Management

High

EUXDAT-REQ-

PLATF-016 Multiple Data Centres in the Cloud

Cloud and High-


(HPC)

Data Management

Data Protection and

Security

Medium

EUXDAT-REQ-

PLATF-017 Cloud Data Storage

Cloud and High-


(HPC)

High




Data Management

Data Protection and

Security

EUXDAT-REQ-

PLATF-018 Dependability

Cloud and High-


(HPC)

High

EUXDAT-REQ-

PLATF-0219 Big Data Management

Cloud and High-


(HPC)

Data Management

High

EUXDAT-REQ-

PLATF-020 Identity Management & Access control

Data Management

Data Protection and

Security

High

EUXDAT-REQ-

PLATF-021 Scalability – Users growth

Cloud and High-


(HPC)

Data Management

High

EUXDAT-REQ-

PLATF-022

Scalability – Data growth and complex

analytics

Cloud and High-


(HPC)

Data Management

Data Processing

Data Analytics

High

EUXDAT-REQ-

PLATF-023 Data decentralization

Cloud and High-


(HPC)

Data Management

Medium

EUXDAT-REQ-

PLATF-024 Parallel data stream processing

Cloud and High-


(HPC)

Data Management

Data Processing

Data Analytics

Medium

EUXDAT-REQ-

PLATF-025

Reduction in energy consumption by

improved processing algorithms

Cloud and High-


(HPC)

Low

EUXDAT-REQ-

PLATF-026 Use of efficient hybrid architectures

Cloud and High-


(HPC)

High

EUXDAT-REQ-

PLATF-027 Visualization of large amounts of data

Data Management

Data Analytics


User Interaction

High

EUXDAT-REQ-

PLATF-028 Support of different formats for visualization


User Interaction

Standards

High

EUXDAT-REQ-

PLATF-029

Provide rich user interfaces for the interactive

visualization


User Interaction High




EUXDAT-REQ-

PLATF-030

Render high resolution data in N arbitrary

dimensions


User Interaction High

EUXDAT-REQ-

PLATF-031

Personalised end-user-centric reusable data

visualisation


User Interaction Medium

EUXDAT-REQ-

PLATF-032 Detection of abnormal sensor measurements

Data Management

Data Processing Medium

EUXDAT-REQ-

PLATF-033

Use of high-performance computing

techniques to the processing of extremely

huge amounts of data

Cloud and High-


(HPC)

Data Management

Data Processing

Data Analytics

High

EUXDAT-REQ-

PLATF-034

Heterogeneous data aggregation and

normalization

Data Management

Data Processing

Data Analytics


User Interaction

Standards

High

EUXDAT-REQ-

PLATF-035 Verification of data integrity and veracity

Data Management

Data Processing Low

EUXDAT-REQ-

PLATF-036

Support for structured, semi-structured and

un-structured data

Data Management

Data Processing

Data Analytics

Standards

Medium

EUXDAT-REQ-

PLATF-037 Provision of RESTful interfaces for accessing

processing capabilities of EUXDAT platform

Data Processing

Data Analytics

Standards

High

EUXDAT-REQ-

PLATF-038 Use of containerization solutions for

implementation and deployment of

processing algorithms

Cloud and High-


(HPC)

Data Processing

Data Analytics

High

EUXDAT-REQ-

PLATF-039

Provision of Data and Processes Catalogue

and Marketplace Data Analytics High

EUXDAT-REQ-

PLATF-040 Data ingestion and caching in the platform Data Management High

EUXDAT-REQ-

PLATF-041 EUXDAT shall provide an orchestration

mechanism that will allow sending tasks to

the underlying infrastructure in a transparent

way to EUXDAT users

Cloud and High-


(HPC)

Data Processing

Data Analytics

High

EUXDAT-REQ-

PLATF-042

EUXDAT shall provide a web development

frontend which will facilitate developers and

data processing experts users preparing,

testing and deploying their algorithms in the

platform, as well as publishing them as new

services.

Data Processing

Data Analytics


User Interaction

High




EUXDAT-REQ-

PLATF-042 EUXDAT General Frontend

Data Management


User Interaction

High

EUXDAT-REQ-

PLATF-044 EUXDAT Pilot Application Frontend


User Interaction

High



5 Conclusions

This deliverable expands the EUXDAT proposal and D2.1 whose acquired the initial e-Infrastructure

requirements. The deliverable starts with a description of ideas captured during an initial discussion in

consortium. Then the D2.3 further elaborates on selected ideas with a foreseen business potential, by

turning them into scenarios. One scenario is described per each selected idea. Then the deliverable

shortly outlines the vision of each EUXDAT pilot, as it was described in the EUXDAT proposal, then

summaries the initial datasets needed for the pilot and then describes the relation of the pilot to defined

scenarios and ideas. In the second part, the deliverable presents an initial collection of informational and

functional/non-functional requirements for each of the pilots (based on their descriptions) as well as a

set of more general and broader requirements that should be taken into account when designing the

platform. These, have been further refined and additional requirements have been collected based on

new information described in other technical deliverables in WP2 and WP3. Additional

refinements/additions in the requirements during next project period may be collected from the

discussions within the consortium progresses in the next period as well as derived from feedback from

consultations with external parties is gathered. These changes are going to be described in the final

planned deliverable D2.5 Updated Report on e-Infrastructure Requirements v2.



6 References

[1] European e-Infrastructure for Extreme Data Analytics in Sustainable Development (EUXDAT).

Project proposal. Nieto, Francisco Javier. 2017.

[2] European e-Infrastructure for Extreme Data Analytics in Sustainable Development (EUXDAT).

Grant Agreement. Nieto, Francisco Javier. 2017.

[3] Suzanne Robertson, James Robertson; ‘Mastering the Requirements Process: Getting Requirements

Right’; Addison-Wesley Professional, 2012

[4] FOODIE. D5.1.2 Pilots Description and Requirements Elicitation Report v2.4 (Annexes). Various.

2015

[5] DataBio. D4.1 Platforms and interfaces for trial 1. Various (to be published on June 2018)

[6] MSO4SC. D2.1 End Users’ Requirements Report. Various. 2017.

[7] EUXDAT. D2.2 EUXDAT e-Infrastructure Definition. Various. 2018

[8] EUXDAT project, deliverable D3.2 End Users’ Platform. Various. 2018

[9] Zartaloudis, Z. D., Iatrou, M. Savvidis, G., Savvidis, K., Glavenas, D., Kalogeropoulos, K. and

Kyparissi, S. (2015). Early and Timely detection of Verticillium dahliae in olive growing using

remote sensing. El Aceite de Oliva, Actas Simposio Expoliva 2015, Jaen, Espana, 6-8 Mayo.

[10] Gitelson, A. A. (2012). Nondestructive estimation of foliar pigment (chlorophylls, carotenoids,

and anthocyanins) contents: evaluating a semi-analalytical three-band model. Pages 141-165 in:

Hyperspectral remote sensing of vegetation. P.S. Thenkabail, G.J. Lyon, A. Huete, eds. Boca

Raton, FL, CRS Press.



7 Annexes

7.1 Annex 1 – Detailed pilots’ requirements

7.1.1 Pilots specific informational requirements

Table 14: EUXDAT-REQ-PILOT-DATA-001

# Id EUXDAT-REQ-PILOT-DATA-001 Priority High

Name Level-1C multi-spectral imaging product

Description All available bands

Purpose Change detection, yield loss, field user recommendation, spraying recommendation.

The system will use multispectral remote sensing instruments on satellite at field scale

Sentinel 2 data will also be used for yield productivity zones calculations

Author CERTH, Pessl Instruments, Meteoblue AG

Actors Farmers, Developers, Agronomists, Agricultural services providers, Data providers

Data Owner European Commission (Regulation (EU) No 377/2014 and Commission Delegated

Regulation (EU) No 1159/2013)

Data Access Sentinels Scientific Data Hub

Data

Licensing

Access to data is based on a principle of full, open and free access as established by the

Copernicus data and information policy Regulation (EU) No 1159/2013 of 12 July 2013.

This regulation establishes registration and licensing conditions for GMES/Copernicus

users and can be found here

Coverage Worldwide

Pilot area, Chalkidiki, Greece

Data Format Geotiff

Estimated

Size

TBs

Metadata Not decided yet

Relationships EUXDAT-REQ-PILOT-DATA-002, EUXDAT-REQ-PILOT-DATA-003, EUXDAT-

REQ-PILOT-DATA-004, EUXDAT-REQ-PILOT-DATA-005, EUXDAT-REQ-PILOT-

DATA-006, EUXDAT-REQ-PILOT-DATA-007, EUXDAT-REQ-PILOT-DATA-011

http://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A32013R1159




# Id EUXDAT-REQ-PILOT-DATA-002 Priority Medium

Name UAV-enabled hyperspectral imagery

Description After spotting the problem spots in EUXDAT-REQ-Pilot1-DATA-002, scan with UAV

hyperspectral in detected spots for identification. Develop disease related indices.

Calculate vegetation indices georeferenced to make maps of disease related indices to map

the diseased spots

Purpose To make disease maps of olive trees and treat only the infected according to severity

Author CERTH

Actors Farmers, Developers, Agronomists, Agricultural services providers, Data providers

Data Owner CERTH

Data Access EUXDAT will provide a repository with UAV scanned hyperspectral indices

Indices

Data

Licensing

Free of charge

Coverage Pilot area, priority area: Chalkidiki, Greece

Data Format Geotiff

Estimated

Size

TBs

Metadata DCAT type metadata

Relationships EUXDAT-REQ-PILOT-DATA-001



Name Climate data

Description Climate data everywhere, in highest possible resolution, coming from single source, to be

comparable (e.g. from models ICON, NEMS)

Purpose Climate data for Pilot 1 to develop forecasting for disease spread in Pilot 1

Author Meteoblue AG, CERTH



Actors Farmers, Agronomists, Developers, Agricultural services providers, Data providers

Data Owner Meteoblue AG, DWD, ECMWF

Data Access Via Meteoblue API

Data

Licensing

NEMS model: Owned by Meteoblue

ERA5: Owned by ECMWF, free to use, accessible via Meteoblue API

ICON: Owned by DWD, free to use, accessible via Meteoblue API

Coverage global

Data Format JSON or CSV

Estimated

Size

Gb

Metadata Various meteorological variables

Relationships EUXDAT-REQ-PILOT-DATA-007, EUXDAT-REQ-PILOT-DATA-008


# Id EUXDAT-REQ-PILOT-DATA-004 Priority high

Name Dynamic cropland mask, crop type map and LAI from Sen2-Agri system

Description Sentinel-2 for Agriculture system is designed to automatically generate key products for

agriculture monitoring, based on Sentinel-2 and Landsat-8 data (cloud-free surface

reflectance composite, dynamic cropland mask, cultivated crop type map and vegetation

indicators describing the vegetative development of crops). It was developed by the

Sentinel-2 for Agriculture project, which has been funded by the European Space Agency

(DUE programme)

Purpose All datasets are important for field management recommendations

Author Pessl Instruments

Actors Farmers, Agronomists, Developers, Agricultural services providers

Data Owner Université catholique de Louvain, Belgique

Université Toulouse III Paul Sabatier, France

CS systèmes d'information, France

CS ROMANIA SA, Romania

Data Access http://www.esa-sen2agri.org/operational-system/systemdownload/

Data

Licensing

The data are available via a free software that needs to be installed on EUXDAT

server.



Coverage worldwide

Data Format GeoTiff, other formats may be required later

Estimated

Size

Tbs

Metadata Not decided yet



DATA-006, EUXDAT-REQ-PILOT-DATA-007, EUXDAT-REQ-PILOT-DATA-011



Name Copernicus European Digital Elevation Model (EU-DEM), version 1.1

Description These products are already value-added products developed by the Sen2Agri consortium

and derived from Sentinel-2 and Landsat-8

Purpose Field management recommendations,

3D visualizations

Author Pessl Instruments, WRLS

Actors Agricultural services providers,

Data Owner European Environment Agency

Data Access https://www.eea.europa.eu/data-and-maps/data/eu-dem

Data

Licensing

Access to the data is governed by the draft delegated regulation on Copernicus data and

information policy, as approved by the EC on 12th of July 2013, and in the process of

decision making by the Council and European Parliament. This delegated act supplements

regulation (EU) No 911/2010 of the European Parliament and of the Council on the

European Earth monitoring programme (GMES). It establishes registration and licensing

conditions for GMES/Copernicus users and defines criteria for restricting access to

GMES/Copernicus dedicated data and GMES/Copernicus service information.

Coverage Europe

Data Format GeoTiff

Estimated

Size

GBs

Metadata https://www.eea.europa.eu/data-and-maps/data/eu-dem#tab-metadata

https://www.eea.europa.eu/data-and-maps/data/eu-dem

https://www.eea.europa.eu/data-and-maps/data/eu-dem#tab-metadata



Relationships EUXDAT-REQ-PILOT-012



Name Copernicus land cover map

Description https://land.copernicus.eu/pan-european/corine-land-cover/clc-2012?tab=metadata

Purpose Field management recommendation

Author Meteoblue AG, Pessl Instruments

Actors Farmers, agronomists, developers, Agricultural services providers

Data Owner EEA

Data Access https://land.copernicus.eu/pan-european/corine-land-cover/clc-2012/view

Data

Licensing

Access to data is based on a principle of full, open and free access as established by the

Copernicus data and information policy Regulation (EU) No 1159/2013 of 12 July 2013.

This regulation establishes registration and licensing conditions for GMES/Copernicus

users and can be found here.

Coverage worldwide

Data Format GeoTiff

Estimated

Size

Mbs for raster, Gbs for vector format

Metadata https://land.copernicus.eu/pan-european/corine-land-cover/clc-2012?tab=metadata



DATA-005, EUXDAT-REQ-PILOT-DATA-006, EUXDAT-REQ-PILOT-DATA-007,

EUXDAT-REQ-PILOT-DATA-011



Name Soil maps (JRC)

Description Information about land use, soil properties, soil function and threats

https://land.copernicus.eu/pan-european/corine-land-cover/clc-2012?tab=metadata

http://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A32013R1159




Author Meteoblue AG

Actors Farmers, agronomists, developers, Agricultural services providers

Data Owner Joint Research Center (JRC) at the European Commision (EC)

Data Access Web Download with Account Registration

Data Licensing Free to use

Coverage Europe

Data Format GeoTiff

Estimated Size GB

Metadata • Soil parameters, https://esdac.jrc.ec.europa.eu/content/european-soil-

database-v20-vector-and-attribute-data

• https://esdac.jrc.ec.europa.eu/content/maps-storing-and-filtering-capacity-

soils-europe

• https://esdac.jrc.ec.europa.eu/content/maps-indicators-soil-hydraulic-

properties-europe

• https://esdac.jrc.ec.europa.eu/content/lucas-2009-topsoil-data

Relationships EUXDAT-REQ-PILOT-DATA-003, EUXDAT-REQ-PILOT-DATA-007



Name Soil moisture data from Pessl's instrumentation

Description Volumetric soil moisture at multiple depths



Actors Farmers, agronomists, developers, Agricultural services providers, Data Providers

Data Owner Pessl Instruments

Data Access API via payed Account Registration

Data Licensing

Coverage Several thousand stations worldwide

Data Format Xml

Estimated Size Mb

Metadata Data from multiple stations will be provided within the project. Please note that the

ownership and the exact geographic location must stay unpublished. The geographic

location can be used within EUXDAT backend platform to extract other datasets.

Relationships







https://esdac.jrc.ec.europa.eu/content/lucas-2009-topsoil-data





Name Open Land Use

Description The base European dataset is derived from the set of available data sources that are

helping identify the land use in particular locality.

Purpose Identification of land use in particular areas, planning of operation on farms

Author WRLS

Actors Data providers, Developers, Agricultural services providers

Data Owner Open licence, Plan4All association

Data Access Shapefiles, RDF by Sparql Endpoint

Data

Licensing

Open database licence

Coverage European level

Data Format Shapefile, RDF

Estimated

Size

GBs

Metadata http://sdi4apps.eu/open_land_use/

Relationships



Name Land Parcel Identification System (LPIS)

Description Data about plots and identification from LPIS give information about selected agro

companies and their area where are farming.

Purpose Plot data will be used for identification, calculation of machinery trajectories, calculation

of effectiveness

Author WRLS



Actors Agricultural services providers, Farmers, Developers

Data Owner Ministry of agriculture of each Member state

Data Access XML files exported from Public Access, WMS services

Data

Licensing

Free license for public part of LPIS

Coverage European level, separated by countries

Data Format XML

Estimated

Size

GBs per country

Metadata Unknown

Relationships



Name Hydrology for EU

Description Water streams and water areas of EU

Purpose For calculation of water influence to temperature and other weather features. Data

have to be derived from EO or downloaded from existing source. OpenStreetMap is a

candidate for vector data

Author WRLS

Actors Agricultural services providers, Developers, Data Providers

Data Owner OpenStreetMap Community

Data Access Various, e.g. https://www.geofabrik.de/

Data Licensing ODbL, details here: https://www.openstreetmap.org/copyright/en

Coverage World

Data Format Depending on data access point, shapefiles for https://www.geofabrik.de/

https://www.geofabrik.de/

https://www.openstreetmap.org/copyright/en

https://www.geofabrik.de/



Estimated Size GBs

Metadata https://wiki.openstreetmap.org/wiki/Rivers

Relationships EUXDAT-REQ-Pilot3-001, EUXDAT-REQ-Pilot3-002


# Id EUXDAT-REQ-Pilot3-DATA-012 Priority

Name Actual weather

Description All majour weather variables (temperature, humidity, precipitation, radiation and

others) available in hourly intervals for any location on Earth via coordinate search,

with automated downscaling for topography and nearby measurements

Purpose Regular updates of actual conditions and 14-day forecast for applications like crop

growth forecasting.

Author meteoblue.com

Actors meteoblue, Pessl Instruments, Plan4All, others

Data Owner meteoblue

Data Access https://content.meteoblue.com/ru/what-we-offer/meteoblue-weather-api

Data Licensing Depends on business model:

1. Free use for base data, in combination with meteoblue branding or in exchange for

measurement data;

2. License fee from farm software providers

3. Pay-per use Tailored business model for EUXDAT purposes

Coverage Worldwide

Data Format JSON; csv: Also possible as images.

Estimated Size GB per day.

Metadata https://content.meteoblue.com/en/what-we-offer/meteoblue-weather-api

Relationships EUXDAT-REQ-Pilot3-002


https://wiki.openstreetmap.org/wiki/Rivers

https://content.meteoblue.com/ru/what-we-offer/meteoblue-weather-api

https://content.meteoblue.com/en/what-we-offer/meteoblue-weather-api



# Id EUXDAT-REQ-Pilot3-DATA-013 Priority

Name Historic weather

Description All major weather variables (temperature, humidity, precipitation, radiation and

others) available in hourly intervals for any location on Earth via coordinate search,

since 1984, with automated downscaling for topography and nearby measurements for

some variables.

Purpose Agro-climatic zoning (Scenario 3)

Climate risk monitoring (Scenario 4)

Crop growth (Scenario 5): Regular updates of past conditions

Author meteoblue.com

Actors meteoblue, Pessl Instruments, Plan4All, others

Data Owner meteoblue

Data Access https://content.meteoblue.com/en/time-dimensions/history

(meteoblue_API_history_data-packages_documentation_EN_v0.8.pdf)

Data Licensing Depends on business model:

1. Free use for base data, in combination with meteoblue branding or in exchange for

measurement data;

2. License fee from farm software providers

3. Pay-per use Tailored business model for EUXDAT purposes

Coverage Worldwide

https://content.meteoblue.com/en/time-dimensions/history



7.1.2 Pilots specific functional and non-functional requirements

Table 26: EUXDAT-REQ-PILOT-001

# Id EUXDAT-REQ-PILOT-001 Priority Medium

Name Atmospheric correction of Multispectral Sentinel bands

Categories Data processing

Description This requirement performs atmospheric correction on the 12 Multispectral Sentinel

bands by simulating atmospheric transmittance and removing the noise from the data

Author CERTH

Actors Agricultural services providers, Data Providers, Developers

Validation

scenario A service calculating atmospherically corrected values of multispectral bands for the

raster image of the field of the pilot

Components UMN MapServer or GeoServer for serving geographic data, optionally Virtuoso for

RDF data, HS Layers NG together with Cesium for the Web application framework




Name Calculation of spectral indices from the 12 Sentinel multispectral bands


Description We will calculate different indices from the 12 Multispectral Sentinel bands based on

the standard band combinations to use it for assessing crop health, it will be used as

input for the anomaly detection stage

Author CERTH

Actors Agricultural services providers, Developers

Validation

scenario

A service calculating maps of different indices for the field of the pilot and comparison

with expected values for the specific crop and conditions

Components

UMN MapServer or GeoServer for serving geographic data, optionally Virtuoso for

RDF data, HS Layers NG together with Cesium for the Web application framework






Name Calculation of Hyperspectral indices relevant for stress and disease


Description This requirement calculates hyperspectral indices utilizing the band values provided

by the UAV camera

Author CERTH

Actors Agricultural services providers

Validation

scenario

A service calculating hyperspectral index values for the field of the pilot and

comparison with expected values for the specific crop and conditions

Components UMN MapServer or Geoserver for serving geographic data, optionally Virtuoso for

RDF data, HS Layers NG together with Cesium for the Web application framework.

Relationships


# Id EUXDAT-REQ-PILOT-004 Priority High

Name Availability of sentinel-2 data at field scale

Categories Data Processing

Description The system will cut out information only based on a given polygon information

avoiding the necessity to download entire scene.


Actors Data providers, Cloud providers

Validation

scenario

Download data for multiple polygons (2-3 for each continent) in given projection

system.

Components not known yet

Relationships EUXDAT-REQ-PILOT-DATA-001, EUXDAT-REQ-PILOT-DATA-004,





# Id EUXDAT-REQ-Pilot1-005 Priority High

Name 2D visualization of time-series over selected pixels, provision of interfaces, toolkits

Categories Data Visualization and User Interaction

Description Perspective visualization of 2D data.


Actors Agricultural services providers, Developers, System integrator companies, Farmers,

Agronomists

Validation

scenario

There exists a web-based framework which allows to create tailored applications for

perspective visualization of multidimensional data in common formats


Relationships All data in Pilot 1



Name Installation of Sen2Agri system and provision of Dynamic cropland mask, crop type

map and LAI

Categories Cloud and High-Performance Computing (HPC)

Description Sentinel-2 for Agriculture system is designed to automatically generate key products

for agriculture monitoring, based on Sentinel-2 and Landsat-8 data (cloud-free

surface reflectance composite, dynamic cropland mask, cultivated crop type map and

vegetation indicators describing the vegetative development of crops). It was

developed by the Sentinel-2 for Agriculture project, which has been funded by the

European Space Agency

(DUE programme)


Actors Agricultural services providers

Validation not known yet



scenario



EUXDAT-REQ-PILOT-DATA-003, EUXDAT-REQ-PILOT-DATA-004,

EUXDAT-REQ-PILOT-DATA-005, EUXDAT-REQ-PILOT-DATA-006,




Name Enable statistics on multi-temporal data for given field, I.e. detecting anomalies,

monthly averaging of spatial datasets, management zone computation.


Description The system should enable to quickly (within max few sec) analyse multitemporal data

over given field. As an example may serve situation when 250 scenes are available

over 50 ha field and anomalies should be detected (computed as a stdev from a mean

always for a given month).

Author Pessl Instrumetns

Actors Agricultural services providers, Developers

Validation

scenario

not known yet


Relationships Not known yet



Name Collecting machinery tracking data

Categories Data Management



Description Machinery tracking data are necessary data source for analysis of effectiveness and

efficiency of utilization of machinery on farms. In combination with Farm

management data it provides evidence of field interventions.

Author WRLS

Actors Farmers, Developers, Agronomists, Agricultural services providers, Data providers,

Cloud providers

Validation

scenario

Data is necessary to collect with given granularity and continuity during whole

production year. There will be necessary to observe at least 2 farms to given valuable

comparison.

Components It can be used FarmTelemetry and SensLog components for data storage and analysis

with HSLayers library for visualization.




Name Collecting of agro meteorological data


Description Agro meteo data will produce additional information from fields and meteo conditions

of farms. They can provide feedback about state of soil and crops.

Author WRLS

Actors Farmers, Agronomists, Developers, Agricultural services providers, Data providers,

Cloud providers

Validation

scenario

Data is necessary to collect with given granularity and continuity during whole

production year. There will be necessary to observe at least 2 farms to given valuable

comparison.

Components SensLog for data storing and analysing, HSLayers library for visualization.






Name Calculation of yield productivity zones

Categories Data Analytics

Description Calculation of yield productivity zones for plots on focused farms will provide

information about effectiveness of field interventions on each plot. It provides input

to decision making process of agronomists on farms.

Author WRLS

Actors Agronomists, Data processing specialists, Developers, Agricultural services providers

Validation

scenario

Calculation of yield productivity zones can be validated by data of real yield from the

machinery sensors.

Components Sen2agri

Relationships EUXDAT-REQ-PILOT-008, EUXDAT-REQ-PILOT-009



Name Zone related morphometric statistic


Data Analytics

Description Calculation of average, mean, min and max, slope, aspect and altitude characteristics

(morphometric characteristics) to each particular zone.

Author WirelessInfo

Actors Agricultural services providers, Developers, Farmers,

Validation

scenario

A service calculating morphometric characteristics for zones exists. Inputs are EU-

DEM and zonal dataset (e.g. OLU or LPIS). Outputs are morphometric attributes for

each particular zone.

Components Geographic information systems algorithms implemented in Cloud and High-

Performance Computing environment.







Name Water influence to weather conditions


Data Analytics

Description Calculation of water buffers, optionally watersheds delimitation, again slopes and

their orientation and other morphometric characteristics

Author WirelessInfo

Actors HPC providers, Agricultural services providers, Developers, Data processing

specialists

Validation

scenario

A service calculating water buffers, optionally watersheds delimitation, again slopes

and their orientation and other morphometric characteristics exists. Inputs are Actual

weather data, Hydrology for EU and EU-DEM. Outputs is raster of water influence

to weather condition (e.g. temperature) and optionally a layer with watershed

delimitation.

Components Geographic information systems algorithms implemented in Cloud and High-

Performance Computing environment.





Name 3D visualization


Description Perspective visualization of multidimensional data (2D, 2,5D, 3D and optionally 4D

~ 3D + time) - called 3D visualization for short.

Author WirelessInfo

Actors Developers, Agricultural services providers, Farmers,

Validation

scenario

There exists a web-based framework which allows to create tailored applications for

perspective visualization of multidimensional data in common formats



Components UMN MapServer or Geoserver for serving geographic data, optionally Virtuoso for

RDF data, HS Layers NG together with Cesium for the Web application framework.

Relationships Potentially all data requirements; potential visualization tool for all other functional

requirements



7.2 Annex 2 – Detailed EUXDAT Platform requirements

Table 39: EUXDAT-REQ-PLATF-001

# Id EUXDAT-REQ-PLATF-001 Priority High

Name Support for various HPC and Cloud providers


Description The EUXDAT e-Infrastructure is expected to make use of Cloud and HPC resources,

depending on the action to be carried out. Therefore, the orchestration solution

should be able to deal with different Cloud and HPC resources providers. This means

to implement several connectors, as providers may use different workload managers

for HPC (Slurm or Torque) and for Cloud (OpenNebula, OpenStack, AWS, etc).

EUXDAT is expected to support, at least, 2 solutions of each kind.

Author Atos

Actors HPC providers

Cloud providers

Validation

scenario

Different providers will be used for validating the requirement. At least, the

EUXDAT solution will support Slurm and Torque for HPC. In the case of Cloud, at

least two Cloud solutions must be supported.

Components The component in charge of the orchestration and the monitoring will deal with this

requirement.




Name Monitor HPC and Cloud resources


Description Since EUXDAT is expected to support several HPC and Cloud providers, it is

necessary that it will provide a monitoring solution which retrieves information about

the available resources, since it will be also necessary from the orchestration

perspective, in order to select the most adequate providers.

Author Atos

Actors HPC Providers



Cloud providers

Validation

scenario

EUXDAT will support monitoring for Slurm and Torque, for HPC. In the case of

Cloud, at least two Cloud solutions must be supported by the monitoring component.

Components The component in charge of the monitoring will deal with this requirement.

Relationships EUXDAT-REQ-PLATF-001, EUXDAT-REQ-PLATF-003


# Id EUXDAT-REQ-PLATF-003 Priority Medium

Name Applications monitoring and profiling


Description The EUXDAT e-Infrastructure will be able to monitor the applications behaviour

(i.e. execution time, resources used, errors produced, other information gathered

from the logs, etc.). The monitoring system will be able to show this information and

all this data will be used for creating application profiles, which will indicate the kind

of applications executed in EUXDAT, as a way to improve resources assignment.

Author Atos


Cloud providers

Validation

scenario

All the mentioned pilots will be run and monitored with EUXDAT, showing

information about the resources used, time to execute and errors detected. A profile

will be created for each pilot, according to the resources used.

Components

The component in charge of the orchestration, monitoring and profiles management

will deal with this requirement.

Relationships EUXDAT-REQ-PLATF-002


# Id EUXDAT-REQ- PLATF-004 Priority Low

Name Adequate operation of the platform




Description The components which are in charge of the resource’s management will be managed

in such a way they can scale up as needed and they will be available as much time as

possible, as it happens with micro-services-based architectures. Components failing

will be automatically restarted and when more resources are needed, these will be

provided with the adequate configuration in an autonomous way.

Author Atos

Actors Cloud providers

HPC providers

Validation

scenario

This requirement will be validated with two scenarios. In the first one, an error will

be provoked in a component, so it will not be available, expecting the right recovery

of the system. In the second one, an important load will be generated, requiring to

scale up the resources allocated.

Components All components involved in the resource’s management operations will be affected.




Name Optimize data movement


Data Management

Description Data is not always in the location where the application is going to carry out the

computation. Therefore, taking into account the particularities of HPC and Cloud

systems, EUXDAT will deal with the challenge of moving data efficiently to the

computation resources as necessary. This may imply to use cache-like mechanisms,

to predict when data should be moved, to maintain data in certain locations and to

determine when to remove data which is not expected to be used anymore.

Author Atos


Cloud providers

Validation

scenario

This requirement will be validated by running applications using both HPC and

Cloud resources with different data loads: big files and data streams. In all cases,

EUXDAT must show that data movement policy provides, in general, better results

than moving all the data just before computing (in sequential workflows).







Name Support security and privacy in data management


Data Management

Data Protection and Security

Description In some cases, the data to be used is not open and it requires some measurements to

keep it secure. Therefore, the solutions in charge of resources management must

provide the means for secure data movement, storage and usage. It is necessary to

guarantee that unauthorized entities will be able to access such data in all the stages

(movement, processing and storage).

Author Atos


Cloud providers

Developers

System intergrator companies

Agricultural services providers

Farmers

Validation

scenario

This requirement will be validated by running applications using both HPC and

Cloud resources with different data loads: big files and data streams. In all cases,

EUXDAT must show that data has been secured for storage, processing and

movement.





Name Automated deployment and execution of applications


Data Management

Description In order to facilitate the usage of computation resources in EUXDAT, the e-

Infrastructure should provide a mechanism that allows to perform automatic



deployment of the applications to be executed and to manage such execution easily.

The complexity of all the process will be hidden to end users, who are not expected

to know about jobs and VMs creation, data movement policies, resources reservation,

etc… The interface provided to the upper layers will facilitate to carry out the

workflows defined for the applications.

Author Atos


Cloud providers

Developers

System integrator companies

Validation

scenario

All the EUXDAT pilots will be deployed and executed with a simple interface, just

providing the basic information (data source, cores/nodes to use, target provider).

EUXDAT will be able to deploy automatically the applications and to execute them

without requiring technical involvement.

Components

This requirement mainly concerns the component in charge of the Orchestration and

Data Management.




Name API access to pilots' data and services


Description The pilots should have an access layer beyond the portal GUI in the form of a HTTP

RESTful API service.

Ideally the e-Infrastructure already provides this API service and the end-user pilots

are making use of these endpoints as well. Special API endpoints per pilot could be

established if underlying functionality differs.

Exposing the API and documentation for external use would allow others to

integrate results from EUXDAT initiative into their portals or services as well.

Data of relevance for machine to machine communication could be single values,

lists, or images. The proposed data format might be JSON (or XML files).


Actors Developers, System integrator companies, Agricultural services providers, Data

processing specialists, Data providers

Validation A valid user having access to a pilot could download processed results using a

standard HTTP API client as well without necessity to use the pilot web portal.



scenario

Components Not known yet




Name User management

Categories Data Protection and Security

Description In order to manage access to the system components the system manager needs to be

able to specify users and groups of users to the system. Application users’ access

authorization will generally depend on who (and maybe where) they are, so it is

necessary to allow system managers to identify users of the system. This might also

include a group of anonymous users or users who registered themselves to the system.

Author ATOS

Requirement source: FOODIE project [4], deliverable D5.1.2 Pilots Description and

Requirements Elicitation Report v2.4 (Annexes)

Actors Developers, System integrator companies, Agricultural services providers, HPC

providers, Cloud providers

Validation

scenario

Registered users in EUXDAT platform can access to specific resources and features

that are not available for anonymous users

Components I&A Manager




Name Access sensor observations


Standards

Description The access to environmental observation values is already being standardized by the

OGC.



The platform shall provide instances of the OGC Sensor Observation Service with the

following properties:

• It shall offer a result model that allows the user to query for, retrieve and enter/update

a set of observation values according to a result model that is compliant with the OGC

Observations & Measurement model.

• It shall be able to enter observations acquired by human sensors.

Author ATOS

Requirement source: FOODIE project [4], deliverable D5.1.2 Pilots Description and


Actors Data providers, Developers,

Validation

scenario

Sensor observations can be stored, queried and accessed according to OGC SOS,

SensorThingsAPI standards or OMA NGSI9/10 interface.


Relationships EUXDAT-REQ-PLATF-008, EUXDAT-REQ-PLATF-013, EUXDAT-REQ-PLATF-

014



Name Support information modelling


Standards

Description Application developers will often need to access the models that carry the structure of

the application’s information and in some cases may need to develop or extend these

models.

Information models provide important structure to the various attributes of

information, and therefore require support both for the use of such models and the

development of them by application developers.

Author ATOS

Requirement source: FOODIE project, deliverable D5.1.2 Pilots Description and


Actors Agricultural services providers, Developers, Data processing especialists

Validation

scenario

Use of standardized models for storing agriculture and farm related information

facilitates the storage, retrieval and integration with other data sources.







Name Support integration of meta-information


Standards

Description Meta-information is a crucial adjunct to information resources discovered and intended

to be used in the application. The application will generally need access to meta-

information describing the data and other information sources brought into it.

Author ATOS



Actors Developers, Agricultural services providers, Data providers, Data processing

especialists

Validation

scenario

Use of metadata facilitates the query and access to the data sources stored in EUXDAT

platform





Name Compliance with INSPIRE specifications

Categories Standards

Data Management

Description The EUXDAT platform shall adopt standards and interfaces com-pliant with the

INSPIRE specifications. This does not necessarily mean that the EUXDAT

architecture will adopt internally the INSPIRE recommended standards and



specifications. Indeed, if necessary EUXDAT may implement specific mediation

components and tools to make interoperability with INSPIRE possible.

The interoperability of the EUXDAT platform with INSPIRE will allow:

a) to publish EUXDAT resources in INSPIRE where possible/needed, and

b) to access INSPIRE resources through the EUXDAT platform.

INSPIRE Implementing Rules, Technical Guidelines are available through the official

INSPIRE web site (http://inspire.jrc.ec.europa.eu)

Author ATOS



Actors Developers, Data providers, System integrator companies

Validation

scenario

By making EUXDAT platform INSPIRE compliant we are enabling external parties to

integrate our services and datasets with theirs. This also will enable to EUXDAT

platform to access to external repositories compliant with INSPIRE



012, EUXDAT-REQ-PLATF-015



Name Compliance with GEO/GEOSS specifications


Data Management

Description GEOSS will become a system of systems by adopting appropriate standards for the

interfaces through which the various GEOSS components exchange data and

information. This requires making these systems and components interoperable, so that

the data and information they produce can be pooled and combined.

EUXDAT shall adopt standards and interfaces compliant with the GEOSS

specifications. This does not necessarily mean that the EUXDAT architecture will

adopt internally the GEOSS recommended standards and specifications. Indeed if

necessary EUXDAT may implement specific mediation components and tools to make

interoperability with GEOSS possible.

The interoperability of the EUXDAT platform with GEOSS will allow:

a) to make the EUXDAT platform a GEOSS component, and



b) to access GEOSS resources through the EUXDAT platform.

The GEOSS Standards and Interoperability Registry provides information about

standards and other interoperability arrangements relevant to the implementation and

operation of GEOSS. In the registry it is possible to find those standards that have been

formally adopted for GEOSS, standards that are currently in use though are not yet

formally recognized, and standards that are potential candidates for use in GEOSS.

Author ATOS



Actors Developers, Data providers, System integrator companies

Validation

scenario

By making EUXDAT platform GEO/GEOSS compliant, we are enabling external

parties to integrate our services and datasets with theirs. This also will enable to

EUXDAT platform to access to external repositories compliant with GEO/GEOSS.



012, EUXDAT-REQ-PLATF-015



Name Integrate Web map services


Data Visualization and User Interaction

Data Management

Description Spatial data will play a crucial role in the application, and integration of these data

using standard tools is essential. Standardized services (such as OGC WMS, WCS and

WFS services) will greatly facilitate the integration and incorporation of spatial data

within applications.

Author ATOS



Actors Developers, Data providers, System integrator companies, Agricultural services

providers



Validation

scenario

Using OGC standards for data visualization (WMS) and data query and access (WCS

and WFS) enables end-users to access and integrate datasets in their own applications.

Components Data Browser

Data Catalogue




Name Multiple Data Centers in the Cloud


Data Management


Description EUXDAT partners may want to keep their components/data in their own infrastructure,

and interconnect it to the cloud infrastructure where EUXDAT platform will run.

Author ATOS



Actors Cloud providers

Validation

scenario

Not known yet





Name Cloud Data Storage


Data Management




Description EUXDAT platform users shall be able to store and serve content stored in the cloud

from their applications instead of providing their own storage resources. The cloud

data storage would be a multi-tenant service that can be offered to many users and

organizations while their data is safely partitioned. This service should interact

with the user management and access control components within the platform in

order to regulate access to the content.

Author ATOS



Actors Cloud providers, Data providers

Validation scenario Not known yet





Name Dependability


Description EUXDAT platform users shall be able to access the system and use the provided

services with a minimum downtime and failures, and can undergo modifications

and repairs easily. The level of operational performance should be prearranged

between data centres providing resources to the cloud (e.g., from 1000 requests for

a service, 990 must be satisfied).

Author ATOS



Actors Farmers, Developers, System integrator companies, Agricultural services

providers


Components Monitoring Component






Name Big Data Management


Data Management

Description Regarding the volume dimension, the system must be able to store datasets of sizes

in TB units. Regarding the variety, the system must be able to manage dozens of

different types of data, from structured to non-structured datasets. Regarding

velocity, the system must be able to manage growth of data with short update

window.

Author ATOS



Actors Cloud providers, HPC providers


Components Data Repository

Data Manager

Data Catalogue




Name Identity Management & Access control



Description Agents (e.g., users, services, devices) need to access multiple data sources,

services and devices in order to perform their tasks. The communication,



coordination and access across these different entities should be done easily and

securely.

The system must provide Identity Management capabilities that are necessary for

regulating users' access to data, services and applications. These capabilities may

include secure and private authentication from agents (e.g., users, smart devices)

to services and applications in the platform, Authorization management, User

Profile management, Single Sign-On (SSO) and Identity Federation if necessary.

Access control should be managed by the system based on the authorization and

user information. Additionally, the system may apply access control policies to

take final decisions.

Author ATOS



Actors Cloud providers, HPC providers, Developers, System integrator companies


Components I&A Manager






Name Scalability – Users growth


Data Management

Description The system shall be able to enlarge and accommodate to a growing amount of

users over the time.

Author ATOS



Actors Cloud providers, HPC providers, Data providers


Components User Management Component




Name Scalability – Data growth and complex analytics


Data Management

Data Processing

Data Analytics

Description EUXDAT platform shall be scalable in terms of storage and application of

complex analytics techniques in order to extract knowledge out of the data and

develop decision-support applications

Author ATOS

Requirement source:

DataBio project, deliverable D4.1 Platforms and interfaces for trial 1[5]

This requirement is reported in document "European Big Data Value Strategic

Research and Innovation Agenda, V4.0". Chapter 3: Technical Aspects, section

Priority 'Data Processing Architectures'.



Actors Cloud providers, HPC providers, Data providers, Developers, Data processing

specialists


Components Data Manager

Data Repository

Data Analytics Module




Name Data decentralization


Data Management

Description EUXDAT architecture shall manage data decentralization (Big Data producers and

consumers can be distributed and loosely coupled as in the Internet of Things) by

providing mechanisms to manage loose data agreements and missing contextual

data.

Author ATOS

Requirement source:

DataBio project, deliverable D4.1 Platforms and interfaces for trial 1




Actors Cloud providers, Data providers








Name Parallel data stream processing


Data Management

Data Processing

Data Analytics

Description The utilisation of a cluster of Big Data processing nodes in the EUXDAT platform

shall require using modern Big Data specific parallelization technique with

automated distribution of tasks overs the nodes in order to accomplish effective

stream processing

Author ATOS

Requirement source:





Actors Cloud providers, HPC providers, Data processing specialists, Developers


Components Orchestrator

Data Manager



# Id EUXDAT-REQ-PLATF-025 Priority Low

Name Reduction in energy consumption by improved processing algorithms


Description The performance of the algorithms in EUXDAT platform shall be able to scale up

by several orders of magnitude, while reducing energy consumption compatible

with the best efforts in the integration between hardware and software.

Author ATOS

Requirement source:







Actors Cloud providers, HPC providers


Components Monitoring Component




Name Use of efficient hybrid architectures


Description EUXDAT platform shall make use of efficient hybrid architectures (i.e., Hybrid

Big Data and High-Performance Computing architecture) that optimise the

mixture of Big Data (i.e. edge) and HPC (i.e. central) resources – combining local

and global processing – to serve the needs of the most extreme and/or challenging

data analytics at scale.

Author ATOS

Requirement source:





Actors Cloud providers, HPC providers, Developers








Name Visualization of large amounts of data


Data Analytics


Description It shall be possible to visualize large amounts of geospatial data in a map with

zoom, pan and filter functions on time/types

Author ATOS

Actors Developers, Agricultural services providers, Farmers, Data processing specialists

Validation scenario The end-user visualizes in an agile manner the datasets without perceiving delays

in the data processing and depiction on his/her screen.


Relationships EUXDAT-REQ-PLATF-015, EUXDAT-REQ-PLATF-019, EUXDAT-REQ-

PLATF-029, EUXDAT-REQ-PLATF-030



Name Support of different formats for visualization


Standards

Description It shall be possible to produce/ingest diversified file formats for visualization,

rendering and reporting

Author ATOS

Requirement source:


This requirement is reported in document "NIST Big Data Interoperability

Framework: Volume 6, Reference Architecture". Page 6, DCR-2: Diversified

output file formats for visualization, rendering and reporting

Actors Developers, Agricultural services providers, Farmers, Data processing specialists







Name Provide rich user interfaces for the interactive visualization


Description EUXDAT platform shall provide rich user interfaces for the interactive

visualization of various datasets types as well as results of data analytics processes,

presenting them in the form of maps, tables and graphic charts. Such visual rich

interfaces should be accessible (preferably) by means of common web browsers

Author ATOS

Requirement source:



Framework: Volume 6, Reference Architecture". Page 6, "DCR-3: Visual layout

for results presentation" and "DCR-4: Rich user interface for access using browser,

visualization tools"

Actors Developers, Agricultural services providers, Data processing specialists, Farmers


Components EUXDAT Frontend

Marketplace

Catalogue




Name Render high resolution data in N arbitrary dimensions


Description EUXDAT platform visualization tools shall be able to render high resolution data,

being also possible to visualize N arbitrary dimensions (1D, 2D, 3D, 4D, 5D, etc.)

if supported by the dataset.

Author ATOS

Requirement source:





Framework: Volume 6, Reference Architecture". Page 6, "DCR-5: High

resolution, multidimensional layer of data visualization"

Actors Developers, Agricultural services providers, Data processing specialists






Name Personalised end-user-centric reusable data visualisation


Description EUXDAT platform shall provide personalised end-user-centric reusable data

visualisation components. Such plug-and-play visualisation components shall

support the combination of any visualisation asset in real-time and can be adapted

and personalised to the needs of end-users.

Author ATOS

Requirement source:




Priority 'Data Visualization and User Interaction'.

Actors Developers, Agricultural services providers, System integrator companies








Name Detection of abnormal sensor measurements


Data Processing

Description EUXDAT platform shall support IoT data stream analysis for the detection of

abnormal sensor measurements

Author ATOS

Requirement source:


Actors Developers, Agricultural services providers, Farmers, Data processing especialists


Components Data Analytics Module




Name Use of high-performance computing techniques to the processing of extremely

huge amounts of data


Data Management

Data Processing

Data Analytics

Description EUXDAT platform shall build on high performance computing techniques to the

processing of extremely huge amounts of data (High Performance Data Analytics

(HPDA)) by taking advantage of a high-performance infrastructure that powers

different workloads, and starting to support workflows that actually accelerate

insights and lead to improved business results for enterprises.

Author ATOS

Requirement source:






Priority 'Data Analytics'.

Actors Developers, Data processing specialists, Cloud providers, HPC providers







Name Heterogeneous data aggregation and normalization


Data Processing

Data Analytics


Standards

Description EUXDAT platform shall support standardization, aggregation, and normalization

of data from disparate sources

Author ATOS

Requirement source:


Actors Developers, Agricultural services providers, Data processing specialists



Data Manager






# Id EUXDAT-REQ-PLATF-035 Priority Low

Name Verification of data integrity and veracity


Data Processing

Description EUXDAT platform shall provide new models and tools to check integrity and

veracity of data, through both machine-based and human-based (crowd-sourcing)

techniques.

Author ATOS

Requirement source:


Actors Data providers, Cloud providers, Developers, Agricultural services providers,

Data processing specialists



Data Repository




Name Support for structured, semi-structured and un-structured data


Data Processing

Data Analytics

Standards

Description EUXDAT platform shall support the ingestion, integration and analysis of

heterogeneous data (ranging from database records, excel and XML files, images,

and Word and PDF documents among others) according to their structured/semi-

structured/unstructured nature.

Author ATOS

Requirement source:



EUXDAT project, deliverable D2.2 EUXDAT e-Infrastructure Definition (pages

15-16)

Actors Developers, Agricultural services providers, Data processing especialists, Systems

integrator companies



Data Manager




Name Provision of RESTful interfaces for accessing processing capabilities of EUXDAT

platform


Data Analytics

Standards

Description EUXDAT platform shall provide standardized protocols (e.g., OGC WPS

standard) for invoking the various processing and data analytics capabilities

offered. These APIs will expose the input parameters required by each of the data

processing/analytics algorithms (e.g., calculation of NDVI indexes) as well as the

output parameters resulting from the execution of the process.

Author ATOS

Requirement source:

EUXDAT project, deliverable D2.2 EUXDAT e-Infrastructure Definition

Actors Developers, Agricultural services providers, Data processing specialists, System

integrator companies, Data providers


Components Catalogue






Name Use of containerization solutions for implementation and deployment of

processing algorithms


Data Processing

Data Analytics

Description There are many algorithms, working on different kind of data, and performing very

different activities. These algorithms will be implemented as several distinct

applications that will used a wide range of tools, libraries and pre-existing pieces

of code, most probably involving several languages (Python, R, Java, C…).

Therefore, it is necessary that EUXDAT supports several kinds of applications

written in different languages and with different ways to integrate analytics

algorithms. Therefore, EUXDAT platform should not constrain the application

developers to use any specific technical environment. Using containerization

technology (Docker + Kubernetes) is the best way to achieve this goal.

Encapsulating each application and all the technical artefacts it requires into a

container allows them to coexist on the platform with a similar interface and clear

manipulation procedures.

Author ATOS

Requirement source:

EUXDAT project, deliverable D2.2 EUXDAT e-Infrastructure Definition (page

19)

Actors Developers, System integrator companies, HPC providers, Cloud providers






Name Provision of Data and Processes Catalogue and Marketplace


Description EUXDAT Marketplace and the Data Catalogue shall enable the finding and access

to information about applications/algorithms and datasets, including metadata,



also enabling the possibility to ‘buy’ data and applications that can be used later

on. The Data Catalogue is focused on datasets metadata access and search, while

the Marketplace is focused on the interface for selling products.

Author ATOS

Requirement source:


19, 40)

Actors Developers, Agricultural services providers, Data providers, System integrator

companies


Components Platform portal

Marketplace

Data Browser

Catalogue




Name Data ingestion and caching in the platform


Description All remote data is not stored permanently on the platform. When a user sends to

execution a pilot application that requires remote data over a certain period of

space and time, it is the responsibility of the platform to retrieve the data and make

it available for the application. To do so, proper download requests have to be sent

prior to the application execution.

In order to avoid repetitive download of the same data, a data cache mechanism is

required. The downloaded files are stored on a local disk. Their description

(dataset ID, spatial and temporal coverage, location on the local disk…) have to

be stored in a database. This database is used to avoid redundant downloading.

Author ATOS

Requirement source:


20)



Actors Cloud provider, Data provider, System integrator companies, Agricultural services

providers


Components File Manager


PLATF-035, EUXDAT-REQ-PLATF-036, EUXDAT-REQ-PLATF-039



Name EUXDAT shall provide an orchestration mechanism that will allow sending tasks

to the underlying infrastructure in a transparent way to EUXDAT users


Data Processing

Data Analytics

Description Critical Big-Data environments need some especial aspects to be addressed and

guaranteed, such as performance (speed of data retrieval and processing), load-

balancing and operation in a distributed computing environments. In that regard,

EUXDAT shall be able to send jobs to HPC systems (like the HLRS one, managed

with Torque) and use VMs in different Cloud solutions, in a transparent manner

for the end-user.

Author ATOS

Requirement source :


23)

Actors Cloud providers, HPC providers, Developers








Name EUXDAT shall provide a web development frontend which will facilitate

developers and data processing experts preparing, testing and deploying their

algorithms in the platform, as well as publishing them as new services.


Data Analytics


Description Such web interface shall facilitate expert users (such as agricultural service

providers) the implementation and testing of their algorithms in a visual and easy

manner (by using Jupyter notebooks) with their preferred programming language.

This programming framework shall enable the programmer to visualize and make

use within its service the various datasets available in EUXDAT platform by

means of a series of APIs (abstracting the developer from the complexity of

accessing the data resources). The resulting service shall be easily deployed in

EUXDAT platform (in the form of a Docker container) and expose its functionality

via an API as well.

The development frontend shall also allow defining complex workflows, where

one or more existing services (within but also outside EUXDAT platform) can be

combined to create new added-value services which will be hosted by EUXDAT.

Newly created and deployed services should be registered (automatically) in the

platform Marketplace.

Author ATOS

Requirement source:


24, 39-40)

EUXDAT project, deliverable D3.2 End Users’ Platform (pages 15-17)

Actors Developers, Agricultural Service Providers, Data processing experts


Components EUXDAT Development Frontend


User Manager

Monitoring Component

Marketplace

Data Browser


PLATF-027, EUXDAT-REQ-PLATF-029, EUXDAT-REQ-PLATF-028,

EUXDAT-REQ-PLATF-038, EUXDAT-REQ-PLATF-037, EUXDAT-REQ-

PLATF-041





Name EUXDAT General Frontend



Description EUXDAT shall provide a general frontend that will serve as main entry point to

the platform features (it must be usable by any user, ranging from farmers, citizens,

to agricultural service providers). Such frontend shall comprehend:

- a marketplace, to present the various data access APIs available in the

platform as well as the pilot specific frontend applications)

- a data catalogue, to provide further specialized information (metadata)

concerning the base datasets hosted by the platform as well as those

generated from the three pilots).

- visualization tools, to get more insight (in the form of maps, graphics or

tables) on the datasets available in the catalogue

Author ATOS

Requirement source:


24, 39-40)

EUXDAT project, deliverable D3.2 End Users’ Platform (pages 15-17)

Actors Any


Components EUXDAT General Frontend

User Manager

Marketplace

Data Browser


PLATF-028, EUXDAT-REQ-PLATF-039





Name EUXDAT Pilot Application Frontend


Description EUXDAT shall provide for each of the project pilots a specific frontend, enabling

to its end-users (the farmers) to carry out – in an easy and visual manner -the

typical tasks described in each of the pilot.

Author ATOS

Requirement source:

EUXDAT project, deliverable D2.3 Description of Proposed Pilots and

Requirements.

Actors Farmers


Components EUXDAT Pilot Specific Frontend

User Manager

Data Browser

Map Viewer


PLATF-009, EUXDAT-REQ-PLATF-008, EUXDAT-REQ-PLATF-027,

EUXDAT-REQ-PLATF-028, EUXDAT-REQ-PLATF-029, EUXDAT-REQ-

PLATF-030

d2.3 updated report on e-infrastructure requirements v1...mainly in first three scenarios (open land...

Documents