ntroduction to measurement uncertainty
TRANSCRIPT
-
7/27/2019 Ntroduction to Measurement Uncertainty
1/27
Doc.: IEEE 802.11-06/0333r0
Submission
March 2006
Dr. Michael D. Foegelle, ETS-LindgrenSlide 1
Introduction to Measurement Uncertainty
Notice:This document has been prepared to assist IEEE 802.11. It is offered as a basis for discussion and is not binding on the cont ributing individual(s) or organization(s). The material inthis document is subject to change in form and content after further study. The contributor(s) reserve(s) the right to add, amend or withdraw material contained herein.
Release:The contributor grants a free, irrevocable license to the IEEE to incorporate material contained in this contribution, and an y modifications thereof, in the creation of an IEEEStandards publication; to copyright in the IEEEs name any IEEE Standards publication even though it may include portions of this contribution; and at the IEEEs sole discretion to permit
others to reproduce in whole or in part the resulting IEEE Standards publication. The contributor also acknowledges and accepts that this contribution may be made public by IEEE 802.11.
Patent Policy and Procedures:The contributor is familiar with the IEEE 802 Patent Policy and Procedures , including the statement"IEEE standards may include the known use of patent(s), including patent applications, provided the IEEE receives assurance from the patent holder or applicant with respect to patents
essential for compliance with both mandatory and optional portions of the standard." Early disclosure to the Working Group of patent information that might be relevant to the standard is
essential to reduce the possibility for delays in the development process and increase the likelihood that the draft publicat ion will be approved for publication. Please notify the Chair
as early as possible, in written or electronic form, if patented technology (or technology under patent application) might be incorporated into a draft standard being
developed within the IEEE 802 .11 Working Group. If you have questions, contact the IEEE Patent Committee Administrator at < [email protected]>.
Date: 2006-3-02
Name Company Address Phone emailDr. Michael D. Foegelle ETS-Lindgren 1301 Arrow Point Drive
Cedar Park, TX 78613
(512) 531-6444 [email protected]
Authors:
http://%20ieee802.org/guides/bylaws/sb-bylaws.pdfmailto:[email protected]:[email protected]:[email protected]:[email protected]://%20ieee802.org/guides/bylaws/sb-bylaws.pdfhttp://%20ieee802.org/guides/bylaws/sb-bylaws.pdfhttp://%20ieee802.org/guides/bylaws/sb-bylaws.pdf -
7/27/2019 Ntroduction to Measurement Uncertainty
2/27
Doc.: IEEE 802.11-06/0333r0
Submission
March 2006
Dr. Michael D. Foegelle, ETS-LindgrenSlide 2
Abstract
This presentation introduces the common industry
concept of Measurement Uncertainty to represent the
quality of a measurement.
Other common terms such as accuracy, precision, error,
repeatability, and reliability are defined and their
relationship to measurement uncertainty is shown.
Basic directions on calculating uncertainty and an
example are included.
-
7/27/2019 Ntroduction to Measurement Uncertainty
3/27
Doc.: IEEE 802.11-06/0333r0
Submission
March 2006
Dr. Michael D. Foegelle, ETS-LindgrenSlide 3
Overview
Definitions
Measurement Uncertainty
Type A Evaluations
Type B Evaluations
Putting It All TogetherRSS
Reporting Uncertainty
Special Cases
Example Uncertainty Budget
Summary
References
-
7/27/2019 Ntroduction to Measurement Uncertainty
4/27
Doc.: IEEE 802.11-06/0333r0
Submission
March 2006
Dr. Michael D. Foegelle, ETS-LindgrenSlide 4
Definitions
ErrorThe deviation of a measured result from the
correct or accepted value of the quantity being
measured.
There are two basic types of errors, randomand
systematic.
Error
-
7/27/2019 Ntroduction to Measurement Uncertainty
5/27
Doc.: IEEE 802.11-06/0333r0
Submission
March 2006
Dr. Michael D. Foegelle, ETS-LindgrenSlide 5
Definitions
Random Errorscause the measured result to deviate
randomly from the correct value. The distribution of
multiple measurements with only random error
contributions will be centered around the correct value.
Some Examples
Noise (random noise)
Careless measurements
Low resolution instruments
Dropped digits Random Errors
-
7/27/2019 Ntroduction to Measurement Uncertainty
6/27
Doc.: IEEE 802.11-06/0333r0
Submission
March 2006
Dr. Michael D. Foegelle, ETS-LindgrenSlide 6
Definitions
Systematic Errorscause the measured result to deviate
by a fixed amount in one direction from the correct
value. The distribution of multiple measurements with
systematic error contributions will be centered some
fixed value away from the correct value.
Some Examples:
Mis-calibrated instrument
Unaccounted cable loss
Systematic Errors
-
7/27/2019 Ntroduction to Measurement Uncertainty
7/27
Doc.: IEEE 802.11-06/0333r0
Submission
March 2006
Dr. Michael D. Foegelle, ETS-LindgrenSlide 7
Definitions
Measurements typically contain some combination of
random and systematic errors.
Precisionis an indication of the level of random error.
Accuracyis an indication of the level of systematic error. Accuracy and precision are typically qualitativeterms.
Low Precision
Low Accuracy
Low Precision
High Accuracy
High Precision
Low Accuracy
High Precision
High Accuracy
-
7/27/2019 Ntroduction to Measurement Uncertainty
8/27
Doc.: IEEE 802.11-06/0333r0
Submission
March 2006
Dr. Michael D. Foegelle, ETS-LindgrenSlide 8
Definitions
Measurement Uncertaintycombines these concepts into asingle quantitative value representing the total expecteddeviation of a measurement from the actual value beingmeasured.
Includes a statistical confidence in the resulting uncertainty.
Contains contributions from all components of the measurementsystem, requiring an understanding of the expected statisticaldistribution of these contributions.
By definition, measurementuncertainty does not typically contain
contributions due to the variability of the DUT. The correct value of a measurement is the value generated by the DUT
at the time it is tested.
Variability of the DUT cannot be pre-determined.
Still, the uncertainty of a particular measurement result will include thisvariability.
-
7/27/2019 Ntroduction to Measurement Uncertainty
9/27
Doc.: IEEE 802.11-06/0333r0
Submission
March 2006
Dr. Michael D. Foegelle, ETS-LindgrenSlide 9
Definitions
Repeatabilityrefers to the ability to perform the same
measurement on the same DUT under the same test
conditions and get the same result over time.
By repeating the test setup between measurements of a
stable DUT, a statistical determination ofSystem
Repeatabilitycan be made. This is simply the level of
random error (precision) of the entire system, including
the contribution of the test operator, setup, etc.
-
7/27/2019 Ntroduction to Measurement Uncertainty
10/27
Doc.: IEEE 802.11-06/0333r0
Submission
March 2006
Dr. Michael D. Foegelle, ETS-LindgrenSlide 10
Definitions
Reproducibilitytypically refers to the stability of the
DUT and the ability to reproduce the same
measurement result over time using a system with a
high level of repeatability.
More generally, it refers to achieving the same
measurement result under varied conditions.
Different test equipment
Different DUT
Different Operator
Different location/test lab
-
7/27/2019 Ntroduction to Measurement Uncertainty
11/27
Doc.: IEEE 802.11-06/0333r0
Submission
March 2006
Dr. Michael D. Foegelle, ETS-LindgrenSlide 11
Definitions
Reliabilityrefers to producing the same result instatistical trials. This would typically refer to thestability of the DUT, and has connotations of
operational reliability of the DUT. Correction -value added algebraically to the
uncorrected result of a measurement to compensate forsystematic error.
Correction Factor- numerical factor by which theuncorrected result of a measurement is multiplied tocompensate for systematic error.
Resolutionindicates numerical uncertainty of testequipment readout. Actual uncertainty may be larger.
-
7/27/2019 Ntroduction to Measurement Uncertainty
12/27
Doc.: IEEE 802.11-06/0333r0
Submission
March 2006
Dr. Michael D. Foegelle, ETS-LindgrenSlide 12
Measurement Uncertainty
A measurement uncertainty represents a statisticallevel encompassing the remaining unknown error in ameasurement.
If the actual value of an error is known, then it is notpart of the measurement uncertainty. Rather, it shouldbe used to correct the measurement result.
The methods for determining a measurementuncertainty have been divided into two generic classes:
Type A evaluation produces a statistically determineduncertainty based on a normal distribution.
Type B evaluation represents uncertainties determinedby any other means.
-
7/27/2019 Ntroduction to Measurement Uncertainty
13/27
Doc.: IEEE 802.11-06/0333r0
Submission
March 2006
Dr. Michael D. Foegelle, ETS-LindgrenSlide 13
Type A Evaluations
Uncertainties are determined through Type A evaluationby performing repeated measurements and determiningthe statistical distribution of the results.
This approach works primarily for randomcontributions.
Repeated measurements with systematic deviations from a knowncorrect value gives an error value that should be corrected for.
However, when evaluating the resulting measurement,
the effect of many systematic uncertainties combine withrandom uncertainties in such a way that their effect canbe determined statistically.
Eg. A systematic offset in temperature can cause an increase in therandom thermal noise in the measurement result.
-
7/27/2019 Ntroduction to Measurement Uncertainty
14/27
Doc.: IEEE 802.11-06/0333r0
Submission
March 2006
Dr. Michael D. Foegelle, ETS-LindgrenSlide 14
Type A Evaluations
Type A evaluation is based on the standard deviation of
repeat measurements, which for nmeasurements with
results qk and average value q, is approximated by:
The standard uncertaintycontribution uiof a single
measurement qk
is given by:
Ifnmeasurements are averaged together, this becomes:
n
k
kk qqn
qs1
2)()1(
1)(
_
)( ki qsu
n
qsqsu ki
)()(
-
7/27/2019 Ntroduction to Measurement Uncertainty
15/27
Doc.: IEEE 802.11-06/0333r0
Submission
March 2006
Dr. Michael D. Foegelle, ETS-LindgrenSlide 15
Type B Evaluations
For cases where Type A evaluation is unavailable orimpractical, and to cover contributions not included inthe Type A analysis, a Type B analysis is used.
Determine potential contributions to the total meas. uncertainty. Determine the uncertainty value for each contribution.
Type A evaluation.
Manufacturers datasheet.
Estimate a limit value.
Note: Contribution must be in terms of the variation in the measuredquantity, not the influence quantity.
For each contribution, choose expected statistical distribution anddetermine its standard uncertainty.
Combine resulting uis and calculate the expanded uncertainty.
-
7/27/2019 Ntroduction to Measurement Uncertainty
16/27
Doc.: IEEE 802.11-06/0333r0
Submission
March 2006
Dr. Michael D. Foegelle, ETS-LindgrenSlide 16
Type B Evaluations
There are a number of common distributions foruncertainty contributions:
Normal distr ibution:
Examples:
Results of Type A evaluations
expanded uncertainties of components
-4s -3s -2s -1s 0 1s 2s 3s 4s
68%
99.7%
95%
kUu ii
where Uiis the expanded
uncertainty of the
contribution and kis the
coverage factor (k= 2for 95% confidence).
-
7/27/2019 Ntroduction to Measurement Uncertainty
17/27
Doc.: IEEE 802.11-06/0333r0
Submission
March 2006
Dr. Michael D. Foegelle, ETS-LindgrenSlide 17
-2ai
-ai
0 ai
2ai
100%
Type B Evaluations
Rectangular distr ibutionmeasurement result has an
equal probability of being anywhere within the range
ofaito ai.
3
ii
au
Examples:
Equipment manufacturer
accuracy values (not fromstandard uncertainty budget)
Equipment resolution limits.
Any term where only maximal
range or error is known.
-
7/27/2019 Ntroduction to Measurement Uncertainty
18/27
Doc.: IEEE 802.11-06/0333r0
Submission
March 2006
Dr. Michael D. Foegelle, ETS-LindgrenSlide 18
Type B Evaluations
U-shaped distr ibution
measurement result has a higher
likelihood of being some value
above or below the median thanbeing at the median.
2
ii
au
Examples:
Mismatch (VSWR)
Distribution of a sine wave
5% Resistors (Culling)
-2ai
-ai
0 ai
2ai
-2ai -ai 0 ai 2ai
-
7/27/2019 Ntroduction to Measurement Uncertainty
19/27
Doc.: IEEE 802.11-06/0333r0
Submission
March 2006
Dr. Michael D. Foegelle, ETS-LindgrenSlide 19
Type B Evaluations
Tr iangular distributionnon-normal distribution with
linear fall-off from maximum to zero.
6
i
i
a
u
Examples:
Alternate to rectangular or
normal distribution whendistribution is known to
peak at center and has a
known maximum
expected value.
-2ai
-ai
0 ai
2ai
100%
-
7/27/2019 Ntroduction to Measurement Uncertainty
20/27
Doc.: IEEE 802.11-06/0333r0
Submission
March 2006
Dr. Michael D. Foegelle, ETS-LindgrenSlide 20
Type B Evaluations
Another Look
-2ai
-ai
0 ai
2ai
Normal Distribution U-Shaped Distribution Triangular Distribution
-2ai
-ai
0 ai
2ai
-2ai
-ai
0 ai
2ai
-4 -3 -2 -1 0 1 2 3 4
.
Rectangular Distribution
-
7/27/2019 Ntroduction to Measurement Uncertainty
21/27
Doc.: IEEE 802.11-06/0333r0
Submission
March 2006
Dr. Michael D. Foegelle, ETS-LindgrenSlide 21
Putting It All Together - RSS
Once standard uncertainties have been determined forall components, including any Type A analysis, they arecombined into a total standard uncertainty (the
combined standard uncertainty, uc), for the resultantmeasurement quantity using the root sum of squaresmethod:
where Nis the number of standard uncertaintycomponents in the Type B analysis.
The combined standard uncertainty is assumed to havea normal distribution.
N
i
ic uu1
2
-
7/27/2019 Ntroduction to Measurement Uncertainty
22/27
Doc.: IEEE 802.11-06/0333r0
Submission
March 2006
Dr. Michael D. Foegelle, ETS-LindgrenSlide 22
Reporting Uncertainty
The standard uncertainty is the common term used forcalculations. It represents a 1s span (~68%) of anormal distribution.
Typically, measurement uncertainties are expressed asan Expanded Uncertainty, U = k uc, where k is thecoverage factor.
A coverage factor ofk=2 is typically used, representinga 95% confidence that the measured value is within thespecified measurement uncertainty.
Reporting of expanded uncertainties must include boththe uncertainty value and either the coverage factor orconfidence interval in order to assure proper use.
-
7/27/2019 Ntroduction to Measurement Uncertainty
23/27
Doc.: IEEE 802.11-06/0333r0
Submission
March 2006
Dr. Michael D. Foegelle, ETS-LindgrenSlide 23
Special Cases
For Type A analyses with only a small number ofsamples, the standard coverage factor is insufficient toensure that the expanded uncertainty covers the
expected confidence interval. Must use variable kp.
RSS math works for values in dB! However,distribution of a linear value may change whenconverted to dB.
Uncertainties typically always determined in measurement outputunits.
N-1 1 2 3 4 5 6 7 8 10 20 50
kp 14.0 4.53 3.31 2.87 2.65 2.52 2.43 2.37 2.28 2.13 2.05 2.00
-
7/27/2019 Ntroduction to Measurement Uncertainty
24/27
Doc.: IEEE 802.11-06/0333r0
Submission
March 2006
Dr. Michael D. Foegelle, ETS-LindgrenSlide 24
Special Cases
Not all distributions are symmetrical!
Can develop asymmetrical uncertainties (+X/-Y) treating
asymmetric inputs separately.
Can separate random portion of uncertainty from systematicportion and apply a systematic error correction to measurement.
(Convert asymmetric uncertainty to symmetric uncertainty.)
error correction = (X+Y)/2, U = (X-Y)/2
-
7/27/2019 Ntroduction to Measurement Uncertainty
25/27
Doc.: IEEE 802.11-06/0333r0
Submission
March 2006
Dr. Michael D. Foegelle, ETS-LindgrenSlide 25
Example Uncertainty Budget
Contribution Source Value Unit Distribution u_j (dB)
Mismatch: Transmit Side 0.00 dB U-Shaped 0.00Analyzer Output Port Source Reflectivity Manufacturer -35.00 dB
Analyzer Output Port VSWR 1.04
Antenna Input Port VSWR (1775-2000) Measured 1.45
Antenna Input Port Reflectivity -14.72 dB
Cable Loss (S21 & S12) Measured 8.00 dB
Mismatch: Receive Side 0.01 dB U-Shaped 0.00
Analyzer Input Port Load Reflectivity Manufacturer -42.00 dB
Analyzer Input Port VSWR 1.02
Antenna Output Port VSWR Measured 1.35Antenna Output Port Reflectivity -16.54 dB
Cable Loss (S21 & S12) Measured 3.00 dB
Network Analyzer Measurement Uncertainty Manufacturer 0.40 dB Rectangular 0.23
(Full Two-Port Calibration, 50 dB path loss, W ide Dynamic Range device)
Transmit Cable Loss Variation Measured 0.05 dB Rectangular 0.03
(Due to flexing, etc.)
Mounting Accuracy: Reference Antenna Calculated 0.00 Rectangular 0.00Antenna Mounting (PLS Laser Aligned & Custom Mounts) 0.13 inches
Range length 14.50 feet
Reference Antenna Gain Uncertainty Manufacturer 0.22 dB Normal 0.11
Miscellaneous Uncertainty CTIA 2.1 G.13 0.20 dB Normal 0.10
Total Uncertainty, u_c Type B RSS 0.28
Expanded Uncertainty, U k = 2 0.55
Validity Range: 1775-2000 MHz
-
7/27/2019 Ntroduction to Measurement Uncertainty
26/27
Doc.: IEEE 802.11-06/0333r0
Submission
March 2006
Dr. Michael D. Foegelle, ETS-LindgrenSlide 26
Summary
This presentation gives common definitions for various
terms that have been used and misused in the TGT
draft.
The concept of measurement uncertainty has been
introduced as the industry standard replacement for
terms such as accuracy, precision, repeatability, etc.
Basic information has been given for a general
knowledge of the concepts and components ofmeasurement uncertainty.
This document is notintended as a reference! Please
refer to the published documents referenced here.
-
7/27/2019 Ntroduction to Measurement Uncertainty
27/27
Doc.: IEEE 802.11-06/0333r0
Submission
March 2006
Dr. Michael D. Foegelle, ETS-LindgrenSlide 27
References
1. NIST Technical Note 1297-1994, Guidelines forEvaluating and Expressing the Uncertainty of NISTMeasurement Results, Barry N. Taylor and Chris E.Kuyatt. 2. NIS-81, The Treatment of Uncertainty in EMCMeasurements, NAMAS 3. ISO/IEC Guide 17025, General requirements forthe competence of testing and calibrationlaboratories.