quality control and measurement uncertainty
TRANSCRIPT
![Page 1: Quality Control and Measurement Uncertainty](https://reader034.vdocument.in/reader034/viewer/2022042618/58a56a751a28ab20528b6591/html5/thumbnails/1.jpg)
Quality Control and Measurement Uncertainty
Ympäristötiedosta palveluihin – seminaari, 23.9.2015
Mauno Rönkkö (UEF), Okko Kauhanen (UEF), Markus Stocker (UEF), Mikko Kolehmainen (UEF), Harri Hytönen (Vaisala), Olli Ojanperä (Vaisala), Esko Juuso (UO), Markku Ohenoja (UO), Ville Kotovirta (VTT), Maija Ojanen (VTT), Petri Koponen (VTT), Teemu Näykki (SYKE), Jari Koskiaho (SYKE), Niina Kotamäki (SYKE), Jari Silander (SYKE), Hanna Huitu (LUKE), Jussi Nikander (LUKE),…
![Page 2: Quality Control and Measurement Uncertainty](https://reader034.vdocument.in/reader034/viewer/2022042618/58a56a751a28ab20528b6591/html5/thumbnails/2.jpg)
24.9.2015 Mauno Rönkkö 2
Contents
1. Sources for Uncertainty in Environmental Monitoring
2. Quality Flagging by Nordic Meteorological Institutes
3. Extended Quality Flagging Scheme to Environmental Data
4. Automatic Monitoring – case Väänteenjoki
5. Water Quality Monitoring and MUkit
6. Conclusion
![Page 3: Quality Control and Measurement Uncertainty](https://reader034.vdocument.in/reader034/viewer/2022042618/58a56a751a28ab20528b6591/html5/thumbnails/3.jpg)
Sources for Uncertainty in
Environmental Monitoring
24.9.2015 Mauno Rönkkö 3
![Page 4: Quality Control and Measurement Uncertainty](https://reader034.vdocument.in/reader034/viewer/2022042618/58a56a751a28ab20528b6591/html5/thumbnails/4.jpg)
1. Case #1: Incomplete Understanding [1/8]
24.9.2015 Mauno Rönkkö 4
http://www.mmea.fi
![Page 5: Quality Control and Measurement Uncertainty](https://reader034.vdocument.in/reader034/viewer/2022042618/58a56a751a28ab20528b6591/html5/thumbnails/5.jpg)
1. Case #2: Indirect Measurements [2/8]
•The Finnish Environmental Institute (SYKE) monitors total phosphorus of lakes and rivers in Finland.
•Currently, there is no device that measures total phosphorus of a lake.
•The amount of total phosphorus is (1) estimated based on the amount of suspended solids (2) which is estimated based on measured turbidity at (3) a given location on a lake/river.
24.9.2015 Mauno Rönkkö 5
http://wwwi3.ymparisto.fi/i3/sakylapyhajarvi/sakylapyhajarvi.htm
![Page 6: Quality Control and Measurement Uncertainty](https://reader034.vdocument.in/reader034/viewer/2022042618/58a56a751a28ab20528b6591/html5/thumbnails/6.jpg)
1. Case #3: Heterogeneous Measurement Methods [3/8]
24.9.2015 Mauno Rönkkö 6
http://www.vaisala.com/en/products/ automaticweatherstations/Pages/default.aspx
http://www.biltema.fi/fi/Toimisto---Tekniikka/Kellot-ja-Lampomittarit/Lampomittari/Langaton-saaasema-84086/
![Page 7: Quality Control and Measurement Uncertainty](https://reader034.vdocument.in/reader034/viewer/2022042618/58a56a751a28ab20528b6591/html5/thumbnails/7.jpg)
1. Case #4: Sampling [4/8]
24.9.2015 Mauno Rönkkö 7
Is this a good enough sample?
![Page 8: Quality Control and Measurement Uncertainty](https://reader034.vdocument.in/reader034/viewer/2022042618/58a56a751a28ab20528b6591/html5/thumbnails/8.jpg)
1. Case #5: Inconsistent Treatment of Measurement Errors [5/8]
24.9.2015 Mauno Rönkkö 8
ERRORS (random errors + systematic errors): -Devices are individuals -Measurements drift over time -Devices are positioned badly -Devices are used in non-optimal conditions -Measurement noise is too large for measured quantity -Person measuring affects the measurements -Environmental conditions vary and affect measurements -Several different measurement devices used to get a dataset -Spatially and temporally different measurements are not preprocessed -Wrong devices are used in a specific measurement method -Different measurement methods with different devices are used -No calibration -Standards and not used or they are used improperly -Measurement data is treated with wrong statistical methods …
![Page 9: Quality Control and Measurement Uncertainty](https://reader034.vdocument.in/reader034/viewer/2022042618/58a56a751a28ab20528b6591/html5/thumbnails/9.jpg)
1. Case #5: Inconsistent Treatment of Measurement Errors [6/8]
24.9.2015 Mauno Rönkkö 9
ERRORS (the less recognized but most important): -Non-synchronized measurement clocks -Cognitive pitfalls
![Page 10: Quality Control and Measurement Uncertainty](https://reader034.vdocument.in/reader034/viewer/2022042618/58a56a751a28ab20528b6591/html5/thumbnails/10.jpg)
1. Case #6: Semantically Inconsistent Interoperability [7/8]
24.9.2015 Mauno Rönkkö 10
![Page 11: Quality Control and Measurement Uncertainty](https://reader034.vdocument.in/reader034/viewer/2022042618/58a56a751a28ab20528b6591/html5/thumbnails/11.jpg)
1. Case #7: Poorly Understood Uncertainties and Validities [8/8]
24.9.2015 Mauno Rönkkö 11
![Page 12: Quality Control and Measurement Uncertainty](https://reader034.vdocument.in/reader034/viewer/2022042618/58a56a751a28ab20528b6591/html5/thumbnails/12.jpg)
Quality Flagging by
Nordic Meteorological Institutes
24.9.2015 Mauno Rönkkö 12
F. Vejen (ed), C. Jacobsson, U. Fredriksson, M. Moe, L. Andresen, E. Hellsten, P. Rissanen, T. Palsdottir, and T. Arason. Quality Control of Meteorological Observa- tions. Automatic Methods Used in the Nordic Countries. Climate Report 8/2002, Norwegian Meteorological Institute, 2002.
![Page 13: Quality Control and Measurement Uncertainty](https://reader034.vdocument.in/reader034/viewer/2022042618/58a56a751a28ab20528b6591/html5/thumbnails/13.jpg)
2. Why bother?
24.9.2015 Mauno Rönkkö 13
THIS IS WATER CONSUMPTION!?
![Page 14: Quality Control and Measurement Uncertainty](https://reader034.vdocument.in/reader034/viewer/2022042618/58a56a751a28ab20528b6591/html5/thumbnails/14.jpg)
2. Why bother?
24.9.2015 Mauno Rönkkö 14
IS THIS WATER CONSUMPTION!?
![Page 15: Quality Control and Measurement Uncertainty](https://reader034.vdocument.in/reader034/viewer/2022042618/58a56a751a28ab20528b6591/html5/thumbnails/15.jpg)
2. Why bother?
24.9.2015 Mauno Rönkkö 15
THIS IS WATER CONSUMPTION!
![Page 16: Quality Control and Measurement Uncertainty](https://reader034.vdocument.in/reader034/viewer/2022042618/58a56a751a28ab20528b6591/html5/thumbnails/16.jpg)
2. Data to information
24.9.2015 Mauno Rönkkö 16
Measurement device
Server
Data storage
Data analysis and refinement
Human operator
![Page 17: Quality Control and Measurement Uncertainty](https://reader034.vdocument.in/reader034/viewer/2022042618/58a56a751a28ab20528b6591/html5/thumbnails/17.jpg)
2. Quality checks
24.9.2015 Mauno Rönkkö 17
QC0 QC1
QC2
HQC
Measurement device
Server
Data storage
Data analysis and refinement
Human operator
![Page 18: Quality Control and Measurement Uncertainty](https://reader034.vdocument.in/reader034/viewer/2022042618/58a56a751a28ab20528b6591/html5/thumbnails/18.jpg)
2. Quality checks
24.9.2015 Mauno Rönkkö 18
QC1
QC2
HQC
Measurement device
Server
Data storage
Data analysis and refinement
Human operator
QC0: real-time quality control on
individual data points about range, step and
consistency
![Page 19: Quality Control and Measurement Uncertainty](https://reader034.vdocument.in/reader034/viewer/2022042618/58a56a751a28ab20528b6591/html5/thumbnails/19.jpg)
2. Quality checks
24.9.2015 Mauno Rönkkö 19
QC2
HQC
Measurement device
Server
Data storage
Data analysis and refinement
Human operator
QC0: real-time quality control on
individual data points about range,
step and consistency
QC1: real-time quality control on individual data
points using statistical methods, including missing
and expected values
![Page 20: Quality Control and Measurement Uncertainty](https://reader034.vdocument.in/reader034/viewer/2022042618/58a56a751a28ab20528b6591/html5/thumbnails/20.jpg)
2. Quality checks
24.9.2015 Mauno Rönkkö 20
HQC
Measurement device
Server
Data storage
Data analysis and refinement
Human operator
QC0: real-time quality control on
individual data points about range,
step and consistency QC1: real-time quality
control on individual data points using statistical
methods, including missing and expected
values
QC2: non-real-time quality control on data sets
including spatial and temporal analysis with
corrective computations
![Page 21: Quality Control and Measurement Uncertainty](https://reader034.vdocument.in/reader034/viewer/2022042618/58a56a751a28ab20528b6591/html5/thumbnails/21.jpg)
2. Quality checks
24.9.2015 Mauno Rönkkö 21
Measurement device
Server
Data storage
Data analysis and refinement
Human operator
QC0: real-time quality control on
individual data points about range,
step and consistency QC1: real-time quality
control on individual data points using statistical
methods, including missing and expected
values
QC2: non-real-time quality control on data sets
including temporal and spatial analysis with
corrective computations
HQC: non-real-time quality inspection
including visualization; the final word
![Page 22: Quality Control and Measurement Uncertainty](https://reader034.vdocument.in/reader034/viewer/2022042618/58a56a751a28ab20528b6591/html5/thumbnails/22.jpg)
2. Measurement Data
24.9.2015 Mauno Rönkkö 22
556 2014-09-27T09:00:30 62.8925, 27.678333 22.6 42
time
humidity temperature
location
device-id
![Page 23: Quality Control and Measurement Uncertainty](https://reader034.vdocument.in/reader034/viewer/2022042618/58a56a751a28ab20528b6591/html5/thumbnails/23.jpg)
2. Measurement Data with a Quality Flag
24.9.2015 Mauno Rönkkö 23
556 2014-09-27T09:00:30 62.8925, 27.678333 22.6 42 9330
time quality flag
humidity temperature
location
device-id
![Page 24: Quality Control and Measurement Uncertainty](https://reader034.vdocument.in/reader034/viewer/2022042618/58a56a751a28ab20528b6591/html5/thumbnails/24.jpg)
2. The Flag Values
24.9.2015 Mauno Rönkkö 24
556 2014-09-27T09:00:30 62.8925, 27.678333 22.6 42 9330
time
quality flag
humidity temperature
location
device-id
C = 1000EHQC + 100EQC2
+ 10EQC1 + EQC0
![Page 25: Quality Control and Measurement Uncertainty](https://reader034.vdocument.in/reader034/viewer/2022042618/58a56a751a28ab20528b6591/html5/thumbnails/25.jpg)
2. Multiple Data Points
24.9.2015 Mauno Rönkkö 25
556 2014-09-27T09:00:30 62.8925, 27.678333 22.6 42 9330
C = 1000EHQC + 100EQC2
+ 10EQC1 + EQC0
556 2014-09-27T09:00:30 62.8925, 27.678333 15.3 55 4000
![Page 26: Quality Control and Measurement Uncertainty](https://reader034.vdocument.in/reader034/viewer/2022042618/58a56a751a28ab20528b6591/html5/thumbnails/26.jpg)
Extending the Quality Flagging Scheme
to Environmental Data
24.9.2015 Mauno Rönkkö 26
M. Rönkkö, O. Kauhanen, M. Stocker, H. Hytönen, V. Kotovirta, E. Juuso, M. Kolehmainen. Quality Control of Environmental Measurement Data with Quality Flagging. IFIP Advances in Information and Communication Technology, 2015, Volume 448, Environmental Software Systems. Infrastructures, Services and Applications, pages 343-350.
![Page 27: Quality Control and Measurement Uncertainty](https://reader034.vdocument.in/reader034/viewer/2022042618/58a56a751a28ab20528b6591/html5/thumbnails/27.jpg)
3. Generic Interpretation
24.9.2015 Mauno Rönkkö 27
Flag Original interpretation Generic interpretation
0 No check performed Value not checked
1 Observation is ok Approved value
2 Suspected small difference Suspicious value
3 Suspected big difference Anomalous value
4 Calculated value Corrected value
5 Interpolated value
Imputed value
6 (Not defined originally) Erroneous value
7 (Not defined originally) Frozen value
8 Missing value Missing value
9 Deleted value Deleted value
![Page 28: Quality Control and Measurement Uncertainty](https://reader034.vdocument.in/reader034/viewer/2022042618/58a56a751a28ab20528b6591/html5/thumbnails/28.jpg)
3. Example Service Architecture
24.9.2015 Mauno Rönkkö 28
![Page 29: Quality Control and Measurement Uncertainty](https://reader034.vdocument.in/reader034/viewer/2022042618/58a56a751a28ab20528b6591/html5/thumbnails/29.jpg)
3. Quality Control of Water Consumption Data
24.9.2015 Mauno Rönkkö 29
![Page 30: Quality Control and Measurement Uncertainty](https://reader034.vdocument.in/reader034/viewer/2022042618/58a56a751a28ab20528b6591/html5/thumbnails/30.jpg)
3. Quality Control of Water Consumption Data
24.9.2015 Mauno Rönkkö 30
QC1: measured and checked once a minute QC2: runs every 2 hours, used for spotting leaks and malfunctions HQC: done once a month, aims at resolving frozen data values
![Page 31: Quality Control and Measurement Uncertainty](https://reader034.vdocument.in/reader034/viewer/2022042618/58a56a751a28ab20528b6591/html5/thumbnails/31.jpg)
Automatic Monitoring - case Väänteenjoki
24.9.2015 Mauno Rönkkö 31
![Page 32: Quality Control and Measurement Uncertainty](https://reader034.vdocument.in/reader034/viewer/2022042618/58a56a751a28ab20528b6591/html5/thumbnails/32.jpg)
24.9.2015 Mauno Rönkkö 32
4. Automatic monitoring – case Väänteenjoki • The challenge in automating water quality monitoring is that the measurement data
have not only significant seasonal variation, but also erroneous values
• Thus, without proper quality control and reliable uncertainty estimation, the data has little value
• As a solution, we have implemented a computation service based on an Enterprise Service Bus Architecture. The service provides means for online quality control and integration of uncertainty estimation
• Case study: In the Karjaanjoki River Basin the Väänteenjoki site equipped with an OBS3+ turbidity sensor (Campbell Scientific inc.)
• OBS3+ sensor emits a near-infrared
light into the water, measures the light that scatters back from the suspended particles, and transforms this information into turbidity values in Nephelometric Turbidity Units (NTU)
![Page 33: Quality Control and Measurement Uncertainty](https://reader034.vdocument.in/reader034/viewer/2022042618/58a56a751a28ab20528b6591/html5/thumbnails/33.jpg)
24.9.2015 Mauno Rönkkö 33
4. Automatic monitoring – case Väänteenjoki • The “raw” turbidity recorded by the OBS3+ sensor had to be calibrated against the
turbidity determined from water samples taken near the sensor • Calibration equation was determined by linear regression between the values of the
water samples and the simultaneous values recorded by the sensor • Then, because turbidity does not denote the content of substance in water, the
calibrated turbidity data had to be converted to concentrations of susp.solids and total P
• We have implemented a computational service that automates and integrates uncertainty estimation to the sequence of operations
![Page 34: Quality Control and Measurement Uncertainty](https://reader034.vdocument.in/reader034/viewer/2022042618/58a56a751a28ab20528b6591/html5/thumbnails/34.jpg)
Water Quality Monitoring and MUkit
24.9.2015 Mauno Rönkkö 34
![Page 35: Quality Control and Measurement Uncertainty](https://reader034.vdocument.in/reader034/viewer/2022042618/58a56a751a28ab20528b6591/html5/thumbnails/35.jpg)
24.9.2015 Mauno Rönkkö 35
![Page 36: Quality Control and Measurement Uncertainty](https://reader034.vdocument.in/reader034/viewer/2022042618/58a56a751a28ab20528b6591/html5/thumbnails/36.jpg)
24.9.2015 Mauno Rönkkö 36
![Page 37: Quality Control and Measurement Uncertainty](https://reader034.vdocument.in/reader034/viewer/2022042618/58a56a751a28ab20528b6591/html5/thumbnails/37.jpg)
24.9.2015 Mauno Rönkkö 37
![Page 38: Quality Control and Measurement Uncertainty](https://reader034.vdocument.in/reader034/viewer/2022042618/58a56a751a28ab20528b6591/html5/thumbnails/38.jpg)
24.9.2015 Mauno Rönkkö 38
![Page 39: Quality Control and Measurement Uncertainty](https://reader034.vdocument.in/reader034/viewer/2022042618/58a56a751a28ab20528b6591/html5/thumbnails/39.jpg)
Conclusion
24.9.2015 Mauno Rönkkö 39
![Page 40: Quality Control and Measurement Uncertainty](https://reader034.vdocument.in/reader034/viewer/2022042618/58a56a751a28ab20528b6591/html5/thumbnails/40.jpg)
5. Conclusion [1/2]
•What you cannot measure, you cannot control.
•Sources for uncertainties
Incomplete understanding, Indirect measurements, Heterogeneous measurement methods, Sampling, Inconsistent treatment of measurement errors, Semantically inconsistent interoperability, Poorly understood uncertainties and validities
•Quality Flagging
– scheme by the Nordic Meteorological Institutes
– Quality checks at various stages; Real-time and non-real-time checks
•Quality Flagging of Environmental Data
– Generic interpretation
– ESB based architecture
24.9.2015 Mauno Rönkkö 40
![Page 41: Quality Control and Measurement Uncertainty](https://reader034.vdocument.in/reader034/viewer/2022042618/58a56a751a28ab20528b6591/html5/thumbnails/41.jpg)
5. Conclusion [2/2]
•Automatic monitoring – case Väänteenjoki
– proper quality control and reliable uncertainty estimation required
– implemented a computation service based on an ESB
•Water quality monitoring and MUkit
– Based on the Nordtest TR 537 guide and on the standard SFS-EN ISO 11352
– Automated turbidity measuring system for ”real-time” uncertainty estimation using AutoMUkit
– Several international publications available!
24.9.2015 Mauno Rönkkö 41