2011 wise soe data requests for rivers, lakes, groundwater , tcm
DESCRIPTION
2011 WISE SoE data requests for rivers, lakes, groundwater , TCM. Miroslav Fanta ETC / ICM data manager [email protected]. Content of the presentation. Reporting process overview Step by step through annual activities Priority data flow - most important reporting categories - PowerPoint PPT PresentationTRANSCRIPT
Event / Date: Country visit on SoE water reporting, Podgorica 20.10.2011Author: Miroslav Fanta, ETC/ICM1
2011 WISE SoE data requests for
rivers, lakes, groundwater, TCM
Miroslav Fanta
ETC / ICM data manager
Event / Date: Country visit on SoE water reporting, Podgorica 20.10.2011Author: Miroslav Fanta, ETC/ICM2
Content of the presentation
•Reporting process overview•Step by step through annual activities •Priority data flow - most important reporting categories
•Rivers + Lakes water quality•Groundwater quality•Transitional, coastal and marine waters quality
•Description of data checking and processing
Event / Date: Country visit on SoE water reporting, Podgorica 20.10.2011Author: Miroslav Fanta, ETC/ICM3
Abbreviations and terms• WISE = Water Information System for Europe• SoE = State of Environment • ROD = Reporting Obligation Database• Data Dictionary = instructions for data preparation by
countries• CDR = Central Data Repository = web space where
data are uploaded by countries• Working database = database for primary data
procesing and analysis• Waterbase = published pan-European datasets on EEA
web page, annual output of reporting• Circa / EIONET Forum = web space for sharing data
and documents
Event / Date: Country visit on SoE water reporting, Podgorica 20.10.2011Author: Miroslav Fanta, ETC/ICM4
Reporting categories
•Rivers water quality (EWN-1)•Lakes water quality (EWN-2)•Groundwater quality (EWN-3)•Transitional, coastal and marine waters quality (ME-
1)•Water quantity (EWN-4)•Emmisions to water (WISE-1)•Biology in Rivers and Lakes (WISE-2)•Biology in Transitional and Coastal waters (WISE-2)
Event / Date: Country visit on SoE water reporting, Podgorica 20.10.2011Author: Miroslav Fanta, ETC/ICM5
WISE-SoE reporting processReporting obligation
Data Dictionary update
Validation questions update
Scoring criteria update
Reportnet automatic QA system update
Announcement letter to countries
Data preparation by countries
Data reporting by countries, checking of data by Reportnet automatic QA system
Data checking and processing in working database by DB manager,communication with countries, European dataset created
European dataset checked by EEA, published in waterbase, evaluation of countries
May-June
July
July-August
31st of July
August-October31st of October
November-January (February)
February-April
Event / Date: Country visit on SoE water reporting, Podgorica 20.10.2011Author: Miroslav Fanta, ETC/ICM6
Data Dictionary
•Detailed description of requested national dataset
•Data model
•List of tables, Instructions for filling tables
•Structure of tables incl. definition, methodology and
format of all fields
•Relations between tables
•Specification of mandatory fields in each table
•Unique combination of fields in each table
Event / Date: Country visit on SoE water reporting, Podgorica 20.10.2011Author: Miroslav Fanta, ETC/ICM7
Data Dictionary (cont.)
•Codelists relevant to specified fields
(list of allowed values)
•Summary of last update
•Templates for data in several formats (xls, mdb, xml)
•Contact to database manager
dd.eionet.europa.eu
Event / Date: Country visit on SoE water reporting, Podgorica 20.10.2011Author: Miroslav Fanta, ETC/ICM8
Event / Date: Country visit on SoE water reporting, Podgorica 20.10.2011Author: Miroslav Fanta, ETC/ICM9
Event / Date: Country visit on SoE water reporting, Podgorica 20.10.2011Author: Miroslav Fanta, ETC/ICM10
Scoring criteria•Individual criteria for each reporting category•Harmonised across categories if meaningful•Set of criteria for each level•Best level: 3 positive „smileys“• -> 2 -> 1 positive „smiley“•Worst level: 1 negative smiley (no data or unusable data reported)
•www.eionet.europa.eu/dataflows/pdf2011/criteria
•Published results of evaluation:- summary of all reporting obligations per country- detailed overview per country, year and reporting category
www.eionet.europa.eu/dataflows/pdf2011/benchmarking?year=2010
Event / Date: Country visit on SoE water reporting, Podgorica 20.10.2011Author: Miroslav Fanta, ETC/ICM11
Event / Date: Country visit on SoE water reporting, Podgorica 20.10.2011Author: Miroslav Fanta, ETC/ICM12
Letter to countries
•Invitation appeal for next reporting period•Released by EEA, published on ETC/ICM web page•Contains: overview of requested reportinglinks to all data dictionariessummary of all important reporting stepslink to central data repositorylinks to evaluation and scoring criterialink to helpdesk
•Data measured in the past year and older are requested
(2011 reporting: data sammpled in 2010 and older)
icm.eionet.europa.eu/announcements/wise_soe_data_2011
Event / Date: Country visit on SoE water reporting, Podgorica 20.10.2011Author: Miroslav Fanta, ETC/ICM13
Event / Date: Country visit on SoE water reporting, Podgorica 20.10.2011Author: Miroslav Fanta, ETC/ICM14
WISE-SoE reporting processReporting obligation
Data Dictionary update
Validation questions update
Scoring criteria update
Reportnet automatic QA system update
Announcement letter to countries
Data preparation by countries
Data reporting by countries, checking of data by Reportnet automatic QA system
Data checking and processing in working database by DB manager,communication with countries, European dataset created
European dataset checked by EEA, published in waterbase, evaluation of countries
May-June
July
July-August
31st of July
August-October31st of October
November-January (February)
February-April
Event / Date: Country visit on SoE water reporting, Podgorica 20.10.2011Author: Miroslav Fanta, ETC/ICM15
Reporting categories
•Rivers water quality (EWN-1)•Lakes water quality (EWN-2)•Groundwater quality (EWN-3)•TCM (transitional, coastal and marine) waters quality (ME-
1)•Water quantity (EWN-4)•Emmisions to water (WISE-1)•Biology in Rivers and Lakes (WISE-2)•Biology in Transitional and Coastal waters (WISE-2)
Event / Date: Country visit on SoE water reporting, Podgorica 20.10.2011Author: Miroslav Fanta, ETC/ICM16
Rivers water qualityConcentrations of chemical substances in the rivers according to the 2011 WISE SoE data request:
•Physical Characteristics of River Monitoring Stations
•Proxy Pressures on the Upstream Catchments of the Rivers
•Nutrients, Organic Matter and General Physico-Chemical Determinands in Rivers - Aggregated Data
•Hazardous Substances and Other Chemical Determinands in Rivers - Aggregated Data
•Hazardous Substances and Other Chemical Determinands in Rivers - Disaggregated Data (preferred)
•Hazardous Substances in Rivers - Supportive Determinands
Aggregated data = aggregated per monitoring station and period (year)
Event / Date: Country visit on SoE water reporting, Podgorica 20.10.2011Author: Miroslav Fanta, ETC/ICM17
Rivers water quality – data model
Event / Date: Country visit on SoE water reporting, Podgorica 20.10.2011Author: Miroslav Fanta, ETC/ICM18
River Monitoring Stations•Country code, National station ID, national name
•Water body ID, water body name
•WFD station (yes / no)
•Catchment name
•River basin district
•Geographical coordinates (longitude, latitude)in ETRS89 coordinate system
•Altitude
•Length from source
•….
Event / Date: Country visit on SoE water reporting, Podgorica 20.10.2011Author: Miroslav Fanta, ETC/ICM19
Proxy Pressures
• Proxy pressures are specified by different types of land use (urban, wetland, forests, total agricultural, pastures, arable,…), industry pressures, discharges of waste water treatment plants, fertilisation, irrigation, water abstraction, use of pesticides, using of cooling water, transport and other influencies
• Corine Land Cover dataset is one of the most important data sources for proxy pressures specification
• Data should be checked regularly by countries and updated in case of changes
Event / Date: Country visit on SoE water reporting, Podgorica 20.10.2011Author: Miroslav Fanta, ETC/ICM20
Nutrients in Rivers – agg. data• Most important for rivers water quality, used for
indicators
• Covers organic matter and general physico-chemical determinands as well (temperature, pH, …)
• 32 substances / determinands specified in codelist
• 5 preferred SoE nutrient groups:
1. BOD5 / BOD7 / Dissolved Organic Carbon2. Total Ammonium / Ammonium3. Total Phosphorus4. Orthophosphate5. Nitrate / Total Oxidised Nitrogen(at least 1 substance from each group should be provided for best scoring)
Event / Date: Country visit on SoE water reporting, Podgorica 20.10.2011Author: Miroslav Fanta, ETC/ICM21
Hazardous substances in Rivers
• Covers other chemical determinands as well (metals,...)
• List of more than 250 hazardous substancies and other determinands is available in the Data Dictionary, nearly all of them including the CAS-number (Chemical Abstract Service)
• About 60 substances are specified as preferred SoE hazadrous substances (pesticides, DDT-related substances, mercury,…) - please focus on these substances in your reporting above all
• Codelist of substances / determinands is available for download on Circa in Excel format as well
Event / Date: Country visit on SoE water reporting, Podgorica 20.10.2011Author: Miroslav Fanta, ETC/ICM22
Hazardous substances in Rivers
• Data can be delivered as disaggregated or aggregated per monitoring station and aggregation period (year in nearly all cases)
• Disaggregated data are allways preferred
• Requested parameters are described in data dictionary– Country code, Station ID– Day of the measurement (disaggregated data)– Specification of the substance– Concenration of determinand in sample (disagg. data) – Mean concentration of samples (aggregated data)
• Other substances than those available in the codelist can be reported too
Event / Date: Country visit on SoE water reporting, Podgorica 20.10.2011Author: Miroslav Fanta, ETC/ICM23
Hazardous substances in Rivers – disaggregated data
Rules for reporting of values below the Limit of detection or below the limit of quantification:
If the sample concentration value is below the limit of detection or limit of quantification, enter the limit of detection or limit of quantification value itself into the Concentration field and fill the field LOD_LOQ_Flag with appropriate character:[ sample concentration value is below the the limit of detection< sample concentration value is below the the limit of quantification
Event / Date: Country visit on SoE water reporting, Podgorica 20.10.2011Author: Miroslav Fanta, ETC/ICM24
Hazardous substances in Rivers supportive determinands
• Supportive determinands needed for correct interpretation of some hazardous substances
HardnessHCO3 Hydrogen carbonateK KaliumMg MagnesiumNa natriumSO4 Sulphate…..
• Table fields:country, station ID, date, supportive determinand, value,
unit
Event / Date: Country visit on SoE water reporting, Podgorica 20.10.2011Author: Miroslav Fanta, ETC/ICM25
Quality assurance / Quality controlList of rules available in Validation rules• Logical rules (agg. data): minimum <= mean <= maximum
minimum <= median <= maximum …
• Data consistency rules:– monitoring station ID used in the concentration table must be
available in the stations table (or already stored in the working database)
– coordinates of stations must be located in the country– …• Outliers: low / high values suspicious in the context of other
values provided for given substance in given station (1, 2, 1, 8, 1)
• Extremely high / low values (0.000001 microgrammes/l) high probability of wrong unit
Countries are asked for confirmation / correction
Event / Date: Country visit on SoE water reporting, Podgorica 20.10.2011Author: Miroslav Fanta, ETC/ICM26
Lakes water qualityAnalogical reporting as in the case of rivers - concentrations of chemical substancies in lakes according to the 2011 WISE SoE data request:
•Physical Characteristics of Lake Monitoring Stations
•Proxy Pressures on the Upstream Catchments of Lakes
•Nutrients, Organic Matter and General Physico-Chemical Determinands in lakes - Aggregated Data
•Hazardous Substances and Other Chemical Determinands in Lakes - Aggregated Data
•Hazardous Substances and Other Chemical Determinands in Lakes - Disaggregated Data (preferred)
•Hazardous Substances in Lakes - Supportive Determinands
Aggregated data = aggregated per monitoring station and period (year)
Event / Date: Country visit on SoE water reporting, Podgorica 20.10.2011Author: Miroslav Fanta, ETC/ICM27
Lakes water quality – data model
Event / Date: Country visit on SoE water reporting, Podgorica 20.10.2011Author: Miroslav Fanta, ETC/ICM28
Significant differences betweenLakes / Rivers reporting
• Secchi depth determinant
• Several samples taken from different depths in one station – „duplicate“ data - not to be erased in data processing!
Event / Date: Country visit on SoE water reporting, Podgorica 20.10.2011Author: Miroslav Fanta, ETC/ICM29
Groundwater qualityConcentrations of substances in groundwater according to the 2011 WISE SoE data request:
•Physical Characteristics of groundwater bodies (delineated according to the EIONET / WFD Art.5)
•Characteristics of groundwater monitoring stations
•Nutrients in groundwater - Aggregated Data (NO2, NO3, NH4, O2 only)
•Nutrients, Organic Matter and General Physico-Chemical Determinands in Groundwater - Disaggregated Data (preferred)
•Hazardous Substances and Other Chemical Determinands in Groundwater - Disaggregated Data
Aggregated data = aggregated per substance, groundwater body and period (year)
Event / Date: Country visit on SoE water reporting, Podgorica 20.10.2011Author: Miroslav Fanta, ETC/ICM30
Groundwater quality (cont.)
• Saltwater intrusionAny occurrences of saltwater intrusion (from seawater or deep
aquifers) caused by groundwater over-exploitation
• Groundwater body GIS boundaries (spatial data layer)Link to the groundwater bodies table via gw body ID
• Saltwater intrusion GIS boundaries (spatial data layer) Link to the saltwater intrusion table via Saltwater area ID
Event / Date: Country visit on SoE water reporting, Podgorica 20.10.2011Author: Miroslav Fanta, ETC/ICM31
Groundwater quality – data model
Event / Date: Country visit on SoE water reporting, Podgorica 20.10.2011Author: Miroslav Fanta, ETC/ICM32
Groundwater bodies
• Characteristics of groundwater bodies (delineated within the EIONET or according to the WFD Art.5)
• In the case gw bodies are not delineated up to now, regional administrative units (e.g. districts or municipalities) can be used instead as temporary solution
• The most important and required fields in this table are:GWB-Code-EIONET or GWB-Code-WFDNational CodeNameReference YearNumber of Horizon (different gw horizons can overlap)River Basin District
Event / Date: Country visit on SoE water reporting, Podgorica 20.10.2011Author: Miroslav Fanta, ETC/ICM33
Groundwater monitoring stations
•Station ID (1.+2. character = country code)
•GWB-Code-EIONET / GWB-Code-WFD(one of these two fields has to be filled)
•National station code, national name
•WFD station (yes / no), WFD station code
•Geographical coordinates (longitude, latitude)in ETRS89 coordinate systém
•Type of use(drinking water, industrial, surveilance, other)
•Well or spring
Event / Date: Country visit on SoE water reporting, Podgorica 20.10.2011Author: Miroslav Fanta, ETC/ICM34
Nutrients in gw – Aggregated data• 4 substances only can be reported as aggregated:NO2, NO3, NH4, O2 (preferred SoE Nutrients)Substance is specified by numerical DeterminandCode (see data
dictionary)
• Data should be aggregated by gw body, substance and period (year)
• Values: minimum, mean, maximum, median• Number of stations (of various types) and classification
classes are requested – details in Data dictionary• In the case given nutrients are provided as disaggregated for
certain gw body and period, do not fill Nutrients in gw – Aggregated data table by the data calculated from already provided disaggregated ones
Disaggregated data are allways preferred
Event / Date: Country visit on SoE water reporting, Podgorica 20.10.2011Author: Miroslav Fanta, ETC/ICM35
Nutrients in gw – Disaggregated data• 12 substances / determinands available in the codelist
for this table:NO2, NO3, NH4, O2, COD, TOC, pH, temperature, …Substance is specified by numerical DeterminandCode (see data
dictionary)
• Fields:Station ID, code of the substance, value, date
• Sample concentrations below the limit of detection should be indicated as [Value_of_the_limit_of_detection]
• Sample concentrations below the limit of quantification should be indicated as <Value_of_the_limit_of_quantification
Event / Date: Country visit on SoE water reporting, Prishtina 13.9.2011Author: Miroslav Fanta, ETC/ICM36
Hazardous substances in gw – Disaggregated data
• About 260 substances / determinands available in the codelist for this table:
• Focus on preferred SoE hazardous substances (cca 60)above all: pesticides (lindane, simazine, atrazine,..) DDT and related substances bisphenol-A, mercury other most important local contaminants
• Identical structure of the table and identical rules for reporting concentrations below the limit of detection or quantification as in the case of nutrients
Event / Date: Country visit on SoE water reporting, Prishtina 13.9.2011Author: Miroslav Fanta, ETC/ICM37
Quality assurance / Quality controlList of rules available in Validation rules• Logical rules (agg. data): minimum <= mean <= maximum
minimum <= median <= maximum …
• Data consistency rules:– groundwater body code used in the stations table must be
available in the gw bodies table– monitoring station ID used in the concentration table must be
available in the stations table– coordinates of stations must be located in the country– DeterminandCode (= specification of the substance) used in
the concentrations data tables must be available in the relevant codelist
– …• Outliers, extremely high / low values – analogical
rules as described in the case of rivers
Event / Date: Country visit on SoE water reporting, Podgorica 20.10.2011Author: Miroslav Fanta, ETC/ICM38
Transitional, coastal and marine waters quality
Concentrations of chemical substances in TCM waters according to the 2011 WISE SoE data request:
•Physical Characteristics of TCM waters Monitoring Stations
•Physical Characteristics of TCM waters Flux Stations
•Proxy Pressures of TCM waters Flux stations
•Nutrients, Organic Matter and General Physico-Chemical Determinands in Seawater - Disaggregated Data
•Hazardous Substances and Other Chemical Determinands in Seawater - Disaggregated Data
•Hazardous Substances and Other Chemical Determinands in Sediment - Disaggregated Data
Event / Date: Country visit on SoE water reporting, Podgorica 20.10.2011Author: Miroslav Fanta, ETC/ICM39
Transitional, coastal and marine waters quality (cont.)
• Hazardous Substances and Other Chemical Determinands in Biota - Disaggregated Data
• Riverine Input Loads
• Direct Discharges
Event / Date: Country visit on SoE water reporting, Podgorica 20.10.2011Author: Miroslav Fanta, ETC/ICM40
TCM waters quality – data model
Event / Date: Country visit on SoE water reporting, Podgorica 20.10.2011Author: Miroslav Fanta, ETC/ICM41
TCM waters Monitoring Stations, Flux Stations
•Country code, National station ID, national name
•Water category
•Water body type description (bay, estuary,…)
•WFD station (yes / no)
•River basin district
•Geographical coordinates (longitude, latitude)in ETRS89 coordinate system
•Sea area
•Marine convention area
•….
Event / Date: Country visit on SoE water reporting, Podgorica 20.10.2011Author: Miroslav Fanta, ETC/ICM42
TCM nutrients and hazardous substances tables
•Analogical definition of substances and concentrations as in the case of rivers
•Disaggregated data reported only
•Hazardous substances in seawater, sediment, biota
•Different units for sediment and biota concentrations
•….
Event / Date: Country visit on SoE water reporting, Prishtina 13.9.2011Author: Miroslav Fanta, ETC/ICM43
WISE-SoE reporting processReporting obligation
Data Dictionary update
Validation questions update
Scoring criteria update
Reportnet automatic QA system update
Announcement letter to countries
Data preparation by countries
Data reporting by countries, checking of data by Reportnet automatic QA system
Data checking and processing in working database by DB manager,communication with countries, European dataset created
European dataset checked by EEA, published in waterbase, evaluation of countries
May-June
July
July-August
31st of July
August-October31st of October
November-January (February)
February-April
Event / Date: Country visit on SoE water reporting, Podgorica 20.10.2011Author: Miroslav Fanta, ETC/ICM44
Checking of data by Reportnet automatic QA system
• Upload data into relevant country folder on cdr.eionet.europa.eu
• Run QA tests which are prepared
• Check the results, errors or inconsistencies (if any) are highlighted
• Correct the data, re-upload the corrected dataset
• Re-run the tests, check the results…
Event / Date: Country visit on SoE water reporting, Podgorica 20.10.2011Author: Miroslav Fanta, ETC/ICM45
Data processing by working database manager
• Rivers: [email protected]: [email protected], [email protected]: [email protected]: Joergen Norrevang Jensen [email protected] quantity: George Karavokiros [email protected]
• Database manager checks the data, comunicates with the country reporter to correct and clarify any detected data errors or problems
• Reported data are included into working database, records containing any errors are tagged
• Data are aggregated and sent to EEA for final checking and publication
Raw disaggregated data are not published
Event / Date: Country visit on SoE water reporting, Podgorica 20.10.2011Author: Miroslav Fanta, ETC/ICM46
Publishing data in the waterbase
Event / Date: Country visit on SoE water reporting, Podgorica 20.10.2011Author: Miroslav Fanta, ETC/ICM47
Publishing data in the waterbase
Event / Date: Country visit on SoE water reporting, Podgorica 20.10.2011Author: Miroslav Fanta, ETC/ICM48
Evaluation of countries
Event / Date: Country visit on SoE water reporting, Podgorica 20.10.2011Author: Miroslav Fanta, ETC/ICM49
Next year….Reporting obligation
Data Dictionary update
Validation questions update
Scoring criteria update
Reportnet automatic QA system update
Announcement letter to countries
Data preparation by countries
Data reporting by countries, checking of data by Reportnet automatic QA system
Data checking and processing in working database by DB manager,communication with countries, European dataset created
European dataset checked by EEA, published in waterbase, evaluation of countries
May-June
July
July-August
31st of July
August-October31st of October
November-January (February)
February-April
Event / Date: Country visit on SoE water reporting, Podgorica 20.10.2011Author: Miroslav Fanta, ETC/ICM50
Validation questions
• List of rules used for data checking and testing• List of remaining errors and unsolved „opened
questions“ not clarified up to now by communication with country
• QA/QC rules and list of remaining errors by country are available for download form Circa (EIONET Forum) annually by 31.7.
Circa: http://eea.eionet.europa.eu/Public/irc/eionet- circle/Home/mainForum: http://forum.eionet.europa.eu/
Event / Date: Country visit on SoE water reporting, Podgorica 20.10.2011Author: Miroslav Fanta, ETC/ICM51
Thank you for attention