physics 736 experimental methods in nuclear-, …neutrino.physics.wisc.edu/teaching/phys736/... ·...
TRANSCRIPT
Karsten Heeger, Univ. of Wisconsin Physics 736, Spring 2013
Physics 736
Experimental Methods in Nuclear-, Particle-, and Astrophysics
- Statistics and Error Analysis -
Karsten [email protected]
Karsten Heeger, Univ. of Wisconsin Physics 736, Spring 2013
Statistics & Error AnalysisTopics
• introduction to statistics and error analysis• probability distributions• treatment of experimental data• measurement process and errors• maximum likelihood• parameter estimation• method of least squares • Bayesian and frequentist approach• hypothesis testing and significance• confidence intervals and limits
Karsten Heeger, Univ. of Wisconsin Physics 736, Spring 2013
Statistics & Error AnalysisTopics
• introduction to statistics and error analysis• treatment of experimental data• probability distributions• maximum likelihood• parameter estimation• method of least squares • Bayesian approach• hypothesis and significance testing • intervals and limits
Karsten Heeger, Univ. of Wisconsin Physics 736, Spring 2013
Statistics & Error AnalysisTopics
• introduction to statistics and error analysis• probability distributions• treatment of experimental data• measurement process and errors• maximum likelihood• parameter estimation• method of least squares • Bayesian and frequentist approach• hypothesis testing and significance• confidence intervals and limits
Karsten Heeger, Univ. of Wisconsin NUSS, July 13, 2009
Probability Distributions - Revisited
Karsten Heeger, Univ. of Wisconsin Physics 736, Spring 2013
Statistical Distributions
ννννν
binomial
Poisson
Gaussian
chisquare distribution
Karsten Heeger, Univ. of Wisconsin Physics 736, Spring 2013
Statistical Distributions
Gaussian
binomial
Poisson
chisquare distribution
P (r) =N !
r!(N ! r)!pr(1 ! p)N!r
P (r) =µre!µ
r!
P (x) =1
!!
(2")e!
(x!µ)2
2!2A
B
C
P (u)du =(u/2)(u/2)!1e!u/2
2!(!/2)duD
what are distributions A-D?
Karsten Heeger, Univ. of Wisconsin Physics 736, Spring 2013
Probability and Statistics
binomial distribution
r r
Karsten Heeger, Univ. of Wisconsin Physics 736, Spring 2013
Probability and Statistics
binomial distribution
Karsten Heeger, Univ. of Wisconsin Physics 736, Spring 2013
Probability and Statistics
binomial distributionfor large N
Karsten Heeger, Univ. of Wisconsin Physics 736, Spring 2013
Probability and Statistics
binomial distribution
for large N
Karsten Heeger, Univ. of Wisconsin Physics 736, Spring 2013
Probability and Statistics
Poisson distribution
Karsten Heeger, Univ. of Wisconsin Physics 736, Spring 2013
Probability and Statistics
Gaussian distribution
Karsten Heeger, Univ. of Wisconsin Physics 736, Spring 2013
Probability and Statistics
Gaussian distribution
σ
Karsten Heeger, Univ. of Wisconsin Physics 736, Spring 2013
Probability and Statistics
Gaussian distribution
Karsten Heeger, Univ. of Wisconsin Physics 736, Spring 2013
Probability and Statistics
Chisquare distribution
ννννν
χ2
family of distributions
different χ2 distribution for each value of the degrees of freedom
Karsten Heeger, Univ. of Wisconsin Physics 736, Spring 2013
Statistical Distributions
p ! 0.05
p ! 0.05
Karsten Heeger, Univ. of Wisconsin Physics 736, Spring 2013
Mean, Median, Mode
mode=most frequent value in data set
Karsten Heeger, Univ. of Wisconsin NUSS, July 13, 2009
Measurement Process and Errors
Karsten Heeger, Univ. of Wisconsin NUSS, July 13, 2009
Probability and Statistics
Karsten Heeger, Univ. of Wisconsin NUSS, July 13, 2009
Probability and StatisticsMeasurement is a random process
Karsten Heeger, Univ. of Wisconsin NUSS, July 13, 2009
Probability and StatisticsMeasurement is a random process described by a
probability distribution
Karsten Heeger, Univ. of Wisconsin NUSS, July 13, 2009
Probability and StatisticsMeasurement is a random process described by a
probability distribution
σ=instrumental precision
Karsten Heeger, Univ. of Wisconsin NUSS, July 13, 2009
Data and Error BarsSystematic and Random Errors
Karsten Heeger, Univ. of Wisconsin NUSS, July 13, 2009
Probability and StatisticsWhat about errors?
Karsten Heeger, Univ. of Wisconsin NUSS, July 13, 2009
1
Figure 2: A historical perspective of values of a few particle properties tabulated in this Review as a function of date ofpublication of the Review. A full error bar indicates the quoted error; a thick-lined portion indicates the same but withoutthe “scale factor.”
Karsten Heeger, Univ. of Wisconsin Physics 736, Spring 2013
Data with Error Bars
Karsten Heeger, Univ. of Wisconsin Physics 736, Spring 2013
Data with Error Bars
• What fraction of data points (+error bars) do you not expect to fall on the line?
Karsten Heeger, Univ. of Wisconsin Physics 736, Spring 2013
Data with Error Bars
For ±1σ, 1/3 of data should be outside fit
Karsten Heeger, Univ. of Wisconsin NUSS, July 13, 2009
Sampling and Parameter Estimation
Karsten Heeger, Univ. of Wisconsin Physics 736, Spring 2013
Sampling and Parameter Estimation
some terminology
sample • data • set of N measurements
population• observable space • underlying parent distribution
estimate• best value
variance of estimate• error on best value
Karsten Heeger, Univ. of Wisconsin NUSS, July 13, 2009
Probability and StatisticsMeasurement is a random process described by a
probability distribution
how confident are we in our measurement?
best estimate
standard deviation
Karsten Heeger, Univ. of Wisconsin Physics 736, Spring 2013
Sampling and Parameter Estimation
best value minimizes variance between estimate and true value
• method of estimation– 1) determine best estimate– 2) determine uncertainty on best estimate
Karsten Heeger, Univ. of Wisconsin Physics 736, Spring 2013
Examples
mean and error from series of measurements
mental Dat.
(4.s2)
;ense that.51). TheEquationat unlikernent be-s it quire
(4.53)
(4.s4)
r fromen oc-lantityof the'moremeth-imum
valuelevia-tY oi.
f.55)
cor-
,56)
t in
rl5 Examples of Applications 97
1,5 Examples of Applications
15.1 Mean and Error from a Series of Messurements
f,remple 4.1 Consider the simple experiment proposed in Sect. 4.3.2 to measure theLngth of an object. The following results are from such a measurement:
17.62l7.61t7.61
17.6217.6217.615
| 7.6t 511.625t7.61
17.62l7.6217.@5
17.6117.6217.61
Uhat is the best estimate for the length of this object?
Since the errors in the measurement are instrumental, the measurements are Gaus-rian distributed. From (4.49), the best estimate for the mean value is then
I = 17.61533
rhile (4.52) gives the standard deviation
d: 5.855 x l0-3 .
This can now be used to calculate the standard error of the mean (4.50),
o(r) = A/V15:0.0015 .The best value for the length of the object is thus
x = 17.616x.0.002 .
Note that the uncertainty on the mean is given by the standard error of the mean andnot the standard deviation!
f5.2 Combining Data with Different Errors
Enmple 4.2 It is necessary to use the lifetime of the muon in a calculation. However,in searching through the literature, 7 values are found from different experiments:
2.19Et0.(X)1 rrs2.197t0.d)i ps2.t948t0.(X)10 us
2.203 t 0.004 ps2.198t0.(X)2 ps
2.202+0.003 ps2.1!X5t0.0020 us
What is the best value to use?One way to solve this problem is to take the measurement with the smallest error;
however, there is no reason for ignoring the results of the other measurements. Indeed,cven though the other experiments are less precise, they still contain valid informationon the lifetime of the muon. To take into account all available information we musttake the weighted mean. This then yields then mean value
t = 2.1%96
with an error
Karsten Heeger, Univ. of Wisconsin Physics 736, Spring 2013
Examples
combining data with different errors
mental Dat.
(4.s2)
;ense that.51). TheEquationat unlikernent be-s it quire
(4.53)
(4.s4)
r fromen oc-lantityof the'moremeth-imum
valuelevia-tY oi.
f.55)
cor-
,56)
t in
rl5 Examples of Applications 97
1,5 Examples of Applications
15.1 Mean and Error from a Series of Messurements
f,remple 4.1 Consider the simple experiment proposed in Sect. 4.3.2 to measure theLngth of an object. The following results are from such a measurement:
17.62l7.61t7.61
17.6217.6217.615
| 7.6t 511.625t7.61
17.62l7.6217.@5
17.6117.6217.61
Uhat is the best estimate for the length of this object?
Since the errors in the measurement are instrumental, the measurements are Gaus-rian distributed. From (4.49), the best estimate for the mean value is then
I = 17.61533
rhile (4.52) gives the standard deviation
d: 5.855 x l0-3 .
This can now be used to calculate the standard error of the mean (4.50),
o(r) = A/V15:0.0015 .The best value for the length of the object is thus
x = 17.616x.0.002 .
Note that the uncertainty on the mean is given by the standard error of the mean andnot the standard deviation!
f5.2 Combining Data with Different Errors
Enmple 4.2 It is necessary to use the lifetime of the muon in a calculation. However,in searching through the literature, 7 values are found from different experiments:
2.19Et0.(X)1 rrs2.197t0.d)i ps2.t948t0.(X)10 us
2.203 t 0.004 ps2.198t0.(X)2 ps
2.202+0.003 ps2.1!X5t0.0020 us
What is the best value to use?One way to solve this problem is to take the measurement with the smallest error;
however, there is no reason for ignoring the results of the other measurements. Indeed,cven though the other experiments are less precise, they still contain valid informationon the lifetime of the muon. To take into account all available information we musttake the weighted mean. This then yields then mean value
t = 2.1%96
with an error
muon lifetime measurements
Karsten Heeger, Univ. of Wisconsin Physics 736, Spring 2013
Examples
count rates and errors
98
o(r) = 0.00061.
Note that this value isThe best value for the
*.{tr
4. Stat ist ics and the Trca
smaller than the error on anv of the inclifetime is thus
t = 2.1970+ 0.0006 ps .
4.5.3 Determination of Count Rales and Their Errors
Example 4.3 Consider the following series of measuremenrs ofrom a detector viewing a 22Na source,
2201 2145 )r) ' ,
What is the decay rate and its uncertainty?
Since radioactive decay is described by a Poisson distributicfor this distribution to find
fr=i=2205.6 and
o(ir) =
The count rate is thus
Count Rate = (2206x.21) counts/min.
It is interesting to see what would happen if instead of cotriods we had counted the total 5 minutes without stopping.served a total of 11028 counts. This constitutes a sample of 4for 5 minutes is thus 11 208 and the error on this, o = l.counts per minute, we divide by 5 (see the next section) toidentical to what was found before. Note that the error takcncount rate in 5 minutes. A common error to be avoided is tominute and then take the square root of this number.
4.5.4 Null Experiments. Setting Confidence Limis \l 'her
Many experiments in physics test the validity of certain theosearching for the presence of specific reactions or decal's Isuch measurements, an observation is made for a cenain anif one or more events are observed. the theoretical lau isevents are observed. the converse cannot bc said to bc truc
23m21ffi
lE=---- ' - =215
Karsten Heeger, Univ. of Wisconsin Physics 736, Spring 2013
Examples
error propagation (e.g. polarization measurements)
! =R ! L
R + L
Karsten Heeger, Univ. of Wisconsin NUSS, July 13, 2009
Measurements and LimitsConfidence Levels
Karsten Heeger, Univ. of Wisconsin NUSS, July 13, 2009
Probability and StatisticsMeasurement is a random process described by a
probability distribution
how confident are we in our measurement?
best estimate
standard deviation
Karsten Heeger, Univ. of Wisconsin Physics 736, Spring 2013
Probability and Statistics
Gaussian distribution
Karsten Heeger, Univ. of Wisconsin Physics 736, Spring 2013
Data with Error Bars
For ±1σ, 1/3 of data should be outside fit
Karsten Heeger, Univ. of Wisconsin Physics 736, Spring 2013
One-parameter Confidence Level
Karsten Heeger, Univ. of Wisconsin Physics 736, Spring 2013
Multiparameter Confidence Levels
hold some parameters fixed, vary only subset
allow all parameters to vary
Karsten Heeger, Univ. of Wisconsin Physics 736, Spring 2013
rate orflux or# of events
x confidenceinterval (CL=68.3%)
confidenceinterval (CL=99%)
Confidence Intervals: Measurements and Limits
Karsten Heeger, Univ. of Wisconsin Physics 736, Spring 2013
Issue of Coverage
• Correct coverage
• Confidence intervals overcover (i.e. are too conservative)
• Reduced power to reject wrong hypotheses
• Confidence intervals undercover
• Measurement pretends to be more accurate than it actually is
Proper coverage can be tested by Monte Carlo simulations
Karsten Heeger, Univ. of Wisconsin NUSS, July 13, 2009