2 accuracy and precision accuracy how close a measurement is to the actual or “true value” high...

33
Estimation: Precision and accuracy, standard error, confidence intervals

Upload: loraine-norman

Post on 04-Jan-2016

214 views

Category:

Documents


0 download

TRANSCRIPT

Measurement and Significant Figures

Estimation: Precision and accuracy, standard error, confidence intervals2Accuracy and PrecisionAccuracyHow close a measurement is to the actual or true value high accuracy

true value low accuracy true value

3PrecisionHow well several measurements agree with each otherhigh precision low precision4Precision or the reproducibility of a set of measurements. A precise sample estimate will have a very small random error of estimation.Precision5Accuracy

Precision

doesn't necessarily need to be7Accuracy and Precision What can you say about the accuracy and precision in each of the following:

High precision, low accuracy

High precision, high accuracy

8Question:How will each of the following affect accuracy and precision?1. A meter stick that is missing the first centimeter.2. A scale that has a zero point that is really five pounds above zero. 9Solution How will each of the following affect accuracy and precision?1. The shortened meter stick will produce measurements that have poor accuracy, but good precision is possible.2. The poorly calibrated scale will give a weight that is not accurate, but good precision is possible.

10 A blood sample was taken from a patient and four different assays were used to measure blood glucose. The true value of the blood glucose was known to be 4.5 mmol/L. State whether the accuracy and the precision of each assay is high or low:Assay A: 4,6; 4,6; 4,8; 4,5; 4,5; 4,4;Assay B: 4,3; 3,5; 5,3; 4,6; 5,5; 3,7Assay C: 3,5; 3,6; 3,3; 3,5; 3,4; 3,5Assay D: 8,5; 6,4; 5,3; 7,6; 4,8; 9,3

11Literature incosistency?

High accuracy and low precision orboth low accuracy and low precision?How confident are we in the estimation of mean/proportion we have calculated?

14

15

Measures of precision:Standard error of mean, SEM Standard error of proportion, SE(p)

Confidence interval for mean Confidence interval for proportion

Standard error of mean, SEM

Number of patientsStandard deviation, SDSEM is smaller (estimate is more precise):the larger is N (number of patients)the smaller is SD (dispersion of data)

19

95% confidence interval for mean, 95% CITogether with SEM, 95% CI is also the measure of precision

Unlike SEM, 95% CI also estimates accuracy of the resultie. 95% is accurate that interval includes true (population) mean)

95% confidence interval for meanIf we draw a 100 samples from our population we would find the true population value within 95% confidence interval in 95 samples. 21

20 samplesCritical values for 90%, 95% and 99% level of confidence90% CI => mean 1.65 SEM95% CI => mean 1.96 SEM 99% CI => mean 2.58 SEM Level of Confidence - Critical Value 0.75, or 75% 1.15 0.80, or 80% 1.28 0.85, or 85% 1.44 0.90, or 90% 1.65 0.95, or 95% 1.96 0.98, or 98% 2.33 0.99, or 99% 2.58Example 1The average systolic BP before treatment in study A, of a group of 100 hypertensive patients, was 170 mmHg. After treatment with the new drug the mean BP dropped by 20 mmHg.If the 95% CI is 1525, this means:23we can be 95% confident that the trueeffect of treatment is to lower the BP by 1525 mmHg.Example 2In study B 50 patients were treated with the same drug, also reducing their mean BP by 20 mmHg, but with a wider 95% CI of -5 to +45. This CI includes zero (no change). This means:24there is more than a 5% chance that there was no true change in BP, and that the drug was actuallyineffective..Watch out for...

The size of a CI is related to the sample size of the study. Larger studies usually have a narrower CI.25Example 3 Meta analysis

Fig. Plot of 5 studies of a new antihypertensive drug.Which study showed the greatest change?Did all the studies show change in favour of the intervention?Were the changes statistically significant?

Results of studies A and B above are shown by the top two lines, i.e. 20 mmHg, 95% CI 1525 for study A and 20 mmHg, 95% CI -5 to +45 for study B.

26ProportionStandard error of proportion, SE(p) SE(p) = (p(1 p)/n)

Confidence interval for proportion

The standard deviation describes the variability of a sample; The standard error of the mean (SEM) does not describe the sample but describes the uncertainty of how the sample mean represents the population mean.Krebs NF, Westcott JE, Culbertson DL et. al. Comparison of complementary feeding strategies to meet zinc requirements of older breastfed infants. Am J Clin Nutr. 2012; 96:30-35 Mean (SEM) total absorbed zinc amounts were 0.80 0.08, 0.71 0.09, and 0.52 0.05 mg/d for the: meat, iron-and-zinc-fortified infant cereal, and whole-grain, iron-only-fortified infant cereal groups of infants.

SEMCIMeat Fe&Zn Fe

Meat Fe&Zn Fe

Common mistake in the literatureMisuse of standard error of the mean (SEM) when reporting variability of a sample. A critical evaluation of four anaesthesia journals. P. Nagele* British Journal of Anaesthesia 90 (4): 514-16 (2001)

SDCIStandard deviation tells us about the variability (spread) in a sample.The CI tells us the range in which the true value (the mean if the sample were infinitely large) is likely to be.What does a small standard error tell us about the sample estimate of the mean?That it is highly variableThat the population standard deviation may be smallThat the sample size is probably smallThat it is imprecise

What will tend to make the standard error larger?A small varianceA large standard deviationImprecise dataInaccurate data

33