statistics in validation sampling plans - cbinet.com session... · statistics in validation...

Statistics in Validation

Sampling Plans that Result in Statistical Confidence

Raul Soto, MSc, CQEIVT Conference - March 2017

Amsterdam, Netherlands

(c) 2017 Raul Soto 2

The contents of this presentation represent the opinion of the

speaker; and not necessarily that of his present or past employers.

IVT Amsterdam MAR2017

About the Author• 20+ years of experience in the medical devices, pharmaceutical, biotechnology, and consumer electronics industries

• MS Biotechnology, emphasis in Biomedical Engineering

• BS Mechanical Engineering

• ASQ Certified Quality Engineer (CQE)

• Led validation / qualification efforts in multiple scenarios:

• High-speed, high-volume automated manufacturing and packaging equipment; machine vision systems

• Manufacturing Execution Systems (MES)

• Enterprise resource planning applications (i.e. SAP)

• IT network infrastructure, reporting systems, Enterprise applications

• Laboratory information systems and instruments

• Mobile apps

• Product improvements, material changes, vendor changes

• Contact information:

• Raul Soto [email protected]

3

4

• Present the fundamental concepts of Acceptance Sampling

• OC Curves• Quality Indices: AQL and LQ (LTPD)

• Sampling by Attributes• Single and Double plans

• Sampling by Variables

• Introduction to ASQ z1.4 and z1.9

What this talk is about

(c) 2017 Raul SotoIVT Amsterdam MAR2017

Statistically Valid Sampling Plans

1. Select the appropriate AQL and LQ (LTPD)

• Based on these, select a sampling plan that provides the required protection to the patient

5(c) 2017 Raul SotoIVT Amsterdam MAR2017


2. KNOW what your sampling plan routinely accepts and rejects

• The protection provided by your sampling plan is defined by what it routinely accepts (defined by AQL) and what it routinely rejects (defined by LQ).

• KNOW the AQL and LQ for ALL your sampling plans

• YES, that includes the ones you get from ASQ z1.4 and z1.9



3.RANDOMIZE

• Units selected for inspection must be selected at random.

• Every single unit in your lot must have the SAME probability to be selected as a sample.

• If this condition is not met, your sample will be biased and your sampling decisions will be WRONG.


PART 1

FUNDAMENTALS

WHAT IS SAMPLING /WHEN TO USE IT / OC CURVES / AQLS AND LQS

9

Fundamentals

• What is Acceptance Sampling?

• When to use it

• Risks of sampling

• OC Curves

• Quality Indices : what they really tell us

•Attributes vs Variables Sampling Plans


10

What is Acceptance Sampling?

Process of evaluating a portion of the product in a lot/batch with the purpose of accepting or rejecting the entire lot/batch.

• Main advantage of acceptance sampling : economics.

• Despite the cost of designing and administering the plans, an overall cost reduction results from not having to inspect the whole lot.

• Sampling helps us to :

• Improve the quality level of the product going out to the customer• Protect us against the release of highly defective lots


11

Alternatives for Product Inspection

The four (4) main alternatives for product inspection are:

1. No inspection• PAT, Parametric Release

2. Small samples• Stable processes, homogeneous product

3. Large samples• Validation, new products, processes with large variation

4. 100% Inspection• When there is a cost-effective alternative (i.e. automation), or when sampling

determines a very high % defective


12

Limitations

• GOOD product may be REJECTED, and BAD product may be ACCEPTED.

• Acceptance sampling does NOT improve the quality of our manufacturing processes.

• Acceptance sampling does NOT provide an accurate estimate of lot quality. It determines an acceptance and rejection decision for each lot.

• The only way to provide 100% good product is to make 100% good product.

“You cannot inspect quality into a product, quality must be built into a product” - Deming


13

Sampling Risks

• Neither statistical sampling, nor 100% inspection, can guarantee that you will find every defect on a lot.

• Sampling risks can be defined as:

• The risk of rejecting a good lot = alpha (a) risk / error (Producer’s Risk)• Represented by AQL

• The risk of accepting a bad lot = beta (b) risk / error (Consumer’s Risk)• Represented by LQ / RQ / LTPD


AQL: Acceptable Quality Limit

• An attributes sampling plan (n = 82, a = 2) with AQL = 0.01 (1%) is used to inspect the following lots:

Accept or Reject?

• Lot A: 0.9 % defective _________________

• Lot B: 1.0 % defective _________________

• Lot C: 1.1 % defective _________________

• Lot D: 2.0 % defective _________________

• Lot E: 3.2 % defective _________________

• Lot F: 5.0 % defective _________________

• Which lots will be accepted, and which will be rejected?



LOT Probability of Acceptance (for n = 82, a = 2)

• Lot A: 0.9 % defective 96.2 %

• Lot B: 1.0 % defective 95.1 %

• Lot C: 1.1 % defective 93.8 %

• Lot D: 2.0 % defective 77.4 %

• Lot E: 3.2 % defective 51.0 %

• Lot F: 5.0 % defective 21.8%



An AQL of 1% means that a lot that is 1% defective will have a:

95% probability or better of being accepted… and

5% probability or lower of being rejected.


17

What the Quality Indices tell us

• Some people believe that lots with % defective < AQL will always be rejected. This is NOT correct.

• Lots with a % defective equal or lower than the AQL have a probability of acceptance of 95% or better.

• Lots with a % defective equal or higher than the LQ10 have a probability of acceptance of 10% or less.

• Lots with a % defective between the AQL and the LQ10 have a probability of acceptance between 10 – 95%.

• Recommendation: Generate sampling plans based on AQL and LQ


18

Quality Indices for Acceptance Sampling Plans

• AQL (Acceptable Quality Limit): • Usually defined as the p (percent defective)

with a 95% Pa (Probability of acceptance).• Focus of AQL is to protect the PRODUCER

from rejecting GOOD lots

• LQ (Limiting Quality)• Usually defined as the p with a 5% Pa (LQ5) or

a 10% Pa (LQ10)• Focus of LQ is to protect the CONSUMER from

accepting BAD lots• Also called Lot Tolerance Percent Defective

(LTPD), Rejectable Quality Level (RQL).

• IQL (Indifference Quality Limit): • The p with a 50% Pa. • Lots with p = IQL have the same probability of

being accepted and being rejected.• This is your sampling plan’s blind spot.

• AOQL (Average Outgoing Quality Limit): • The maximum % defective possible in the

outgoing material for this sampling plan.


19

The Operating Characteristic CurvePa

(p

rob

abili

ty o

f ac

cep

tan

ce)

p (% defective)0 1

1• The OC Curve for a sampling plan

quantifies the sampling risk graphically, by plotting the Probability of Acceptance (Pa, or P) of a lot as a function of the Lot % Defective (p).

• When the percent defective (p) is high, we want the probability of acceptance (Pa) to be low

• When the percent defective (p) is low, we want the probability of acceptance (Pa) to be high

0.95

0.10


21

Two (2) Types of OC Curves

• Type A:• Assumes that samples are taken from an isolated lot of finite size, where the effect of removing a

sample is significant. (Lot size / sample size < 10)

• This sampling process is modeled by the Hypergeometric distribution.

• Type B: • Assumes that the sample came from a large lot, where the effect of removing a sample has negligible

effect. (Lot size / sample size > 10)

• This sampling process is modeled by the Binomial distribution (for attributes).


Why is AQL not enough?

• In order to understand the level of protection provided by a sampling plan, you should know what the plan routinely accepts (AQL), and what it routinely rejects (LQ).

• AQL quantifies the producer’s risk for your sampling plan.

• A low AQL helps ensure you do not reject good product.

• LQ quantifies the consumer’s risk for your sampling plan.

• A low LQ helps ensure you do not accept bad product.

• To know this, you need to know the AQL, the LQ, and the OC Curve.


Why is AQL not enough?• These 4 sampling plans have

AQL ≈ 1% but different LQ10.

• They provide equivalent protection to the producer, but different levels of protection to the consumer.

23

Plan AQL IQ LQ10

n = 132, a = 3 1.0% 2.8% 5.01%

n = 65, a = 2 1.3% 4.2% 8.1%

n = 52, a = 2 1.6% 5.1% 9.9%

n = 25, a = 1 1.4% 6.6% 14.7%


AQL and LQ for Validations

• Depends on the claim we want to be able to make:

• Claim: “At 95% confidence, the % defective for individual lots is less or equal than 1%”

=> design a plan with LQ5 ≤ 1%

• Claim: “At 90% confidence, the % defective for individual lots is less or equal than 5%”

=> design a plan with LQ10 ≤ 5%

• LQs for validating new processes or process changes should be smaller than for regular manufacturing with mature processes.

• Depends on severity of a failure: critical / major / minor / cosmetic

• Your company’s Quality Engineering standards should have target quality levels based on severity.

• Main driver should always be the protection of patient safety.


25

Attributes vs Variables Data

• Attributes data

• Quality characteristics that are expressed on a go / no-go basis• Qualitative and discrete• Pass / Fail• Examples: seal appearance, colors, presence or absence

• Variables data

• Quality characteristics that are measured in a numerical basis• Quantitative and continuous• Examples: weight, diameter, length, thickness, hardness, seal strength, concentration of solute


26

Attributes vs Variables Sampling Plans

Attributes sampling plans

• Defined by:• a sample size n• an acceptance number a (or c)

• A random sample is taken from the lot• Each unit is inspected and classified as acceptable or defective• The number of defective units is then compared to the allowable number stated in the plan.• A decision is made of accepting or rejecting the lot• Modeled using the binomial or hypergeometric distribution


27

Attributes vs Variables Sampling Plans

Variables sampling plans

• Defined by:• a sample size n• an critical distance constant k

• A sample is taken and a measurement of a specified characteristic is made on each unit.• The sample average, sample standard deviation, and k are calculated.• The appropriate formula is used to analyze the lot.• A decision is made of accepting or rejecting the lot.• Modeled using the normal distribution.

• Normality of the data must be confirmed before using a variables sampling plan


PART 2

ATTRIBUTES SAMPLING PLANS

29

Attribute Sampling PlansSingle• lots inspected and decision to accept / reject is based on one sample• Simpler to design and implement• Larger sample sizes = less economically efficient

Double • lots inspected and decision to accept / reject is based on a maximum of two samples• More complex to design and implement• Smaller sample sizes in the long run = more economically efficient

Multiple• decision to accept / reject is based on maximum of seven samples (out of scope for this talk)• Most complex to design and implement• Fewer sampling, most economically efficient• Looks too much like “sampling until you pass”


30

Attribute Sampling Plans

• Defined by sample size (n) and acceptance number (a)

• Example:

N = lot size = 10,000 unitsn = sample size = 89 unitsa = acceptance number = 2 units

• This means that we will take a random sample of 89 units from the lot, and inspect them; if the number of observed defectives is less than or equal to 2, the lot will be accepted.

• In attributes sampling plans, each inspected unit is judged as either conforming or nonconforming.


31

Attribute Sampling Plans

• The OC Curve allows us to see the level of protection provided by a sampling plan

• In this case, since N/n >10, we can use the binomial distribution to construct the OC Curve

• For example, for p = 0.01 (1% defective), n = 89, a = 2

• In Excel: BINOM.DIST(2,89,0.01,TRUE) = 0.93969

• If we calculate Pa for multiple values of p, we obtain the OC Curve:

9397.0)99.0()01.0(!87!*2

!89)99.0()01.0(

!88!*1

!89

)99.0()01.0(!89!*0

!89}2{

872881

890

xPPa


0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 2 4 6 8 10 12 14 16 18

Pa

percent defective

OC Curve

32

% Defective po

Probability of Acceptance

Pa

0.005 0.9897

0.010 0.9397

0.020 0.7366

0.030 0.4985

0.040 0.3042

0.050 0.1721

0.060 0.0919

0.070 0.0468

0.080 0.0230

0.090 0.0109

AQL LQ10

0.95

0.10

0.5


33

AQL and LQ

• AQL is the p (% defective) that corresponds to a Pa of 95%

• LQ10 is the p that corresponds to a Pa of 10%.

• For this attributes plan: AQL = 0.0092 ; LQ10 = 0.0598 ; IQ = 0.0300

• The protection of this plan (n = 89, a = 2) can be interpreted from the OC curve as follows:

• A lot with 0.92% defective has a 95% probability of acceptance (AQL)

• A lot with 3% defective has a 50% probability of acceptance (IQL)

• A lot with 5.98% defective has a 10% probability of acceptance (LQ10)


34

Effects of N, n, a in the OC Curve

• Increasing the sample size n increases the precision of the plan, its ability to discriminate between good and bad lots. (Steeper OC curve)

• Decreasing the acceptance number a shifts the OC curve to the left, also increasing the protection of the plan.

• When a = 0, the OC curve is convex.

• The lot size does not have a significant effect on the OC curve unless the sample size is large (N/n < 10)


35

Effect of sample size n

A lot that is 2% defective will have a probability of acceptance of (approximately):

85% with n = 3065% with n = 6040% with n = 100


36

Effect of acceptance number


37

Effect of lot size N

Assuming lots are homogeneous


38

Effects of N, n, a in the OC Curve

• Sometimes the sample size is set as a fixed percentage of the lot size (for example, sampling 10% of every lot).

• Another common practice is to set the sample size as the square root of the lot size.

• The problem with these method is that they do not provide a consistent protection, since the sample size will vary as the lot size varies.

• More protection to large lots, less protection to small lots.

• You still need to know your plan’s AQLs and LTPDs. Remember, you should ALWAYS know what your sampling plans routinely accept and routinely reject.


What if we use n = 10% of N?

39

N = 1000

N = 5000

N = 2000

Inconsistent protectionLarger lots get better protection than

smaller lots


What if we use n = √N?

40

Inconsistent protectionLarger lots get better protection than

smaller lots


41

AOQ and AOQL

• Average Outgoing Quality (AOQ) is the average quality level of all accepted lots under a specific sampling plan

• AOQ is what the client will receive in the long term

• AOQ calculation assumptions:• Rejected lots are 100% inspected• 100% inspection finds all defective units• Defective units are replaced or repaired• Lot size is large compared to sample size

• AOQ is defined as AOQ(p) = p * Pa(p)

• The maximum value of the AOQ is the Average Outgoing Quality Limit, AOQL


42

AOQ and AOQL

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

0 1 2 3 4 5 6 7

Ave

rage

Ou

tgo

ing

Qu

alit

y A

OQ

percent defective p

AOQ Curve for n = 132, a = 3

AOQL = 1.47%• Assumptions

• 100% inspection of rejected lots

• All defective units found and discarded / repaired

• Large lot size (N > 10n)


43

“Zero Defects” Sampling Plans

• Using an acceptance number (a = 0) on a sampling plan does NOT guarantee zero defects.

• An a = 0 sampling plan is NOT a tool to drive a process to zero defects; all you achieve is to reject more product.

• Zero defects can only be achieved by eliminating the problems that cause defects, NOT by sampling.


44

“Zero Defects” Sampling Plans

• Alternatives:

• For critical defects with major impact to patient safety it is necessary to keep the lowest AQL possible. In this case, a large sample with an accept number of 0 is the most appropriate alternative.

• In these cases, we must be aware of the cost of implementing such a plan, vs the cost of releasing a critical defect to the market. For medical devices/ biopharma, the safety of the customer must be the most important consideration.

• For non-critical defects: Non-zero defects plan with equivalent LTPD, slightly higher AQL


Zero vs Non-Zero Defects Sampling Plansn = 50 a = 0

Pa% Index p95 AQL 0.102534990 0.210496750 IQ 1.376728010 LTPD10 4.50074205 LTPD5 5.8155100

NEARLY EQUIVALENT TO

n = 85 a = 1

Pa% Index p95 AQL 0.419673190 0.627401650 IQ 1.966671010 LTPD10 4.49905805 LTPD5 5.4596020


Double Sampling Plans

• Lots inspected and decision to accept / reject is based on a maximum of twosamples

• For mature processes with low % defective, double sampling plans provide equivalent protection with smaller sample sizes.

PROS:

• Very good or very bad lots will be rejected with less sampling overall.

• More economically favorable if cost of sampling is an issue

• Equivalent levels of producer and consumer risks with significantly less samples in the long term

CONS:

• More complex to design and administer


47


Take n1 = 60

# defects

Accept Reject

Take n2 = 82

Total # defects

Accept Reject

0 ≥ 4

1 - 3

≥ 4≤ 3

n1 = 60, a1 = 0, r = 4n2 = 82, a2 = 3

Total # defects: add up defects found in both rounds of inspection


48


• Example: Target AQL = 1%, Target LQ10 = 5%

• Single sampling plann = 132, a = 3

Pa (%) Index p (%)95 AQL 1.0490 1.3250 IQ 2.7710 LQ10 4.995 LQ5 5.77

AOQL = 1.47%


49


• Equivalent: Target AQL = 1%, Target LQ10 = 5%

• Double sampling plann1 = 60, a1 = 0, r = 4n2 = 82, a2 = 3 a2 → total cumulative # defects

Pa (%) Index p (%)95 AQL 1.0090 1.2850 IQ 2.7010 LQ10 4.995 LQ5 5.85

AOQL = 1.42%


Single vs Double Sampling Plan Comparison

50

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 1 2 3 4 5 6 7

Pro

bab

ility

of

Acc

ep

tan

ce (

Pa)

percent defective p

Single vs Double Sampling Plan

Pa (double)

Pa (single)

p Pa (double) Pa (single)

0 1 1

0.65 0.9871363 0.9888251

1.3 0.8952717 0.905678

1.95 0.7228923 0.7425666

2.6 0.5279285 0.5501373

3.25 0.3570842 0.3751051

3.9 0.2284081 0.2392301

4.55 0.1405285 0.1445002

5.2 0.08433088 0.08343898

5.85 0.04993715 0.04638324

6.5 0.02946017 0.02495489

Target AQL = 1%, LQ10 = 5%

• Single sampling plan• n = 132, a = 3

• Double sampling plan• n1 = 60, a1 = 0, r = 4• n2 = 82, a2 = 3

AQL

LQ10


Using Minitab

• Stats / Quality Tools/ Acceptance Sampling by Attributes


52

Acceptance Sampling by Attributes

Measurement type: Go/no goLot quality in percent defectiveUse binomial distribution to calculate probability of acceptance

Acceptable Quality Level (AQL) 1Producer's Risk (Alpha) 0.05

Rejectable Quality Level (RQL or LTPD) 10Consumer's Risk (Beta) 0.1

Generated Plan(s)

Sample Size 52Acceptance Number 2

Accept lot if defective items in 52 sampled <= 2; Otherwise reject.

Percent Probability ProbabilityDefective Accepting Rejecting

1 0.985 0.01510 0.097 0.90350 0.000 1.000

Using Minitab


53

MS EXCEL Functions for Attributes Sampling

BINOM.DIST (acceptance number, sample size, %defective, true) = returns the probability of acceptance PaTrue = cumulative functionFalse = probability mass function

BINOM.INV (sample size, AQL, alpha) = returns the minimum acceptance number for given AQL

BINOM.INV (sample size, LTPD, beta) = returns the minimum rejection number for given LTPD

POISSON.DIST (acceptance number, n * AQL, true) = Returns the probability of acceptance Pa

HYPGEOM.DIST (acceptance number, sample size, % defective, lot size) = Returns the probability of acceptance Pa


PART 3

VARIABLES SAMPLING PLANS

55

Variables Sampling Plans

• Variables sampling plans are defined by two quantities: • the sample size n• the critical distance constant k

• Generally based on sample mean and sample standard deviation of the quality characteristic.


56

Advantages and Disadvantages

• Advantages

• The same OC Curve can be obtained with a smaller sample size than with an attributes sampling plan. (The same protection with smaller sample sizes).

• Measurements data usually provide more information about the process than attributes data.


57

Advantages and Disadvantages

• Disadvantages

• The distribution of the quality characteristic should be normal.

• Separate sampling plans must be used for each quality characteristic of interest.

• It is possible to reject a lot even though the actual sample does not contain any defective items.


58

Types of Variable Sampling Plans

• Known vs Unknown Standard Deviation• Larger sample sizes are needed when the standard deviation is unknown.

• One or two specification limits• Some quality characteristics only have USL or LSL, not both• Example: sealing strength in some packaging

• All variables sampling plan are characterized by• Sampling size n• Critical distance constant k


59

Process

Standard

Deviation

Spec Limits Acceptance Criteria

Known LSL LSL + k * s ≤ x̄

Known USL x̄ ≤ USL – k * s

Known USL and LSL LSL + k * s ≤ x̄ ≤ USL – k * s

Unknown LSL LSL + k * s ≤ x̄

Unknown USL x̄ ≤ USL – k * s

Unknown USL and LSL LSL + k * s ≤ x̄ ≤ USL – k * s

Types of Variable Sampling Plans


60

Example:

• Sampling plan for variables: n = 20 and k = 2.0

• Known process standard deviation: s = 1.2 mg

• One-sided Specification: LSL = 12 mg

• Take a sample, calculate sample mean: x ̄ = 16.32 mg

• Do we accept or reject this lot?


61

Example:

For this example:

• Known process standard deviation, LSL only

• Accept if : LSL + k * s ≤ x ̄ acceptance criteria

• (12 + 2.0 * 1.2 ) ≤ x ̄

• 14.4 ≤ 16.32 YES

• Since the sample mean is larger than the acceptance criteria, the lot is accepted.


62

Another Example:

• Sampling plan for variables: n = 20 and k = 2.0

• Two-Sided Specs : LSL = 120 mg, USL = 140 mg

• Process standard deviation is unknown

• Take a sample, sample mean x ̄= 124.07 mgsample stdev s = 2.15 mg

• Do we accept or reject this lot?


63

Another Example:

For this example:

• LSL and USL ; Unknown process standard deviation

• Accept if: LSL + k * s ≤ x̄ ≤ USL – k*s Acceptance Criteria

• 120 + 2 * 2.15 ≤ 124.07 ≤ 140 – 2 * 2.15• 124.30 ≤ 124.07 ≤ 135.70 ??? NO

• Since the sample mean does not satisfy the acceptance criteria, the lot is rejected


64

Comparing Sampling Plan Types: Attributes vs Variables

• Compare plans: AQL = 1%, LTPD = 5% target

• Attributes:• n = 132, a = 3

• Variables, unknown process standard deviation:• n = 55, k = 1.95

• Variables, known process standard deviation:• n = 19, k = 1.95

• Using a variables sampling plan can provide equivalent levels of protection with a much smaller sample size.


65

Normality Assumption• The normality of the quality characteristic of interest must be tested before using a variables

sampling plan.

• Typical tests for normality:

• Visual methods: • Histograms, P-P plots (probability-probability), Q-Q plots (quartile-quartile)

• Normality tests:• Kolmogorov-Smirnov (KS), Lilliefors-corrected KS, Shapiro-Wilk test, Anderson-Darling,

Cramer-von Mises, D’Agostino skewness test, Anscombe-Glynn kurtosis test, D’Agostino-Pearson omnibus test, Jarque-Bera test.

• KS is frequently used, highly sensitive to extreme values• Some researches consider the Shapiro-Wilk (SW) test as the best choice• Minitab includes AD, KS, and Ryan-Joiner which is similar to SW.

Ghasemi A, Zahediasl S. Normality Tests for Statistical Analysis:

A Guide for Non-Statisticians.

Int J Endocrinol Metab. 2012;10(2):486-9. DOI: 10.5812/ijem.3505

http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3693611/


66

Normality Assumption

• For some processes the quality characteristic we want to measure does not follow a normal distribution

• Alternatives: transform the data (for example, take a natural logarithm) and check if the transformed data is normally distributed

• ALWAYS consult with your company’s statisticians before you do any data transformations


67

Effect of lot – to – lot variability

• If the mean of the quality characteristic varies from lot to lot, a variables sampling plan will overestimate the variability and will probably reject good lots.

• This type of variability will also affect your Cpk

• Variables sampling plans do not react well when a process has outliers or short periods of highly defective products.

• Use automated 100% inspection instead


Using Minitab

• Stats / Quality Tools/ Acceptance Sampling by Variables/ [Create / Compare]


Using Minitab

69

Acceptance Sampling by Variables - Create/Compare

Lot quality in percent defective

Lower Specification Limit (LSL) 7.5Upper Specification Limit (USL) 8.5

Acceptable Quality Level (AQL) 1Producer's Risk (Alpha) 0.05

Rejectable Quality Level (RQL or LTPD) 10Consumer's Risk (Beta) 0.1

Generated Plan(s)

Sample Size 20Critical Distance (k Value) 1.73910Maximum Standard Deviation (MSD) 0.244685

Z.LSL = (mean - lower spec)/standard deviationZ.USL = (upper spec - mean)/standard deviationAccept lot if standard deviation <= MSD, Z.LSL >= k and Z.USL >= k; otherwise reject.

Percent Probability ProbabilityDefective Accepting Rejecting

1 0.953 0.04710 0.111 0.88950 0.000 1.000


70

MS EXCEL Functions for Variables Sampling

Z.TEST (array, x, sigma)Returns the one-tailed P-value of a z-test

NORM.INV (probability, mean, standard_dev)Returns the inverse of the normal cumulative distribution for the specified mean and standard deviation

NORM.S.DIST (z)Returns the standard normal cumulative distribution (has a mean of zero and a standard deviation of one)

NORM.DIST (x, mean, standard_dev, cumulative)Returns the normal cumulative distribution for the specified mean and standard deviation

NORM.S.INV (probability)Returns the inverse of the standard normal cumulative distribution (has a mean of zero and a standard deviation of one)


PART 4

ASQ 1.4 AND 1.9

72

ASQ Z1.4 (2003)

• Most commonly used sampling system in the world… and a lot of people are using it WRONG.

• It is NOT intended to be a table of sampling plans from where you can pick the one you want… but that’s how a lot of people use it.

• ASQ z1.4 is available at www.asq.org


73

ASQ Z1.4 – Use of Switching Rules

• Z1.4 contains tables of matches single, double, and multiple sampling plans.

• It also contains OC Curves, percentiles, ASN curves, and AOQLs for these plans.

• Z1.4 includes reduced, normal, and tightened sampling plans, and a set of switchingrules

• ASQ z1.4 page 7, section 11.6:

• If we are NOT using the switching rules, we can’t state we are inspecting as per ASQ z1.4


74

ASQ

z1

.4 (

20

03

) p

age

7, s

ecti

on

11

.6


75

ASQ Z1.4 – for Individual Plans

• Section 11.6.2

“When employed in this way, this document simply represents a repository for a collection of individual plans indexed by AQL. The operational characteristics and other measures of a plan so chosen must be assessed individually for that plan from the tables provided.”


76

ASQ Z1.4 – How to Use It

• Specify a sampling plan by level of inspection, AQL, and lot size

• get the sample size letter code from Table I

• get the single sampling plan from Table II-A


77

ASQ Z1.4 – Inspection Level

“Three inspection levels: I, II, III, are given in Table I for general use. Normally, Inspection Level II is used. However, Inspection Level I may be used when less discrimination is needed, or Level III may be used for greater discrimination”

“Four additional Special levels, S-1, S-2, S-3, and S-4, are given in the same table, and may be used where relatively small sample sizes are necessary, and large sampling risks can or must be tolerated.”

• Level I has largest LTPD, Level III has smaller LTPDs

• Increasing the Inspection Level basically increases the sampling size letter code.


78

Why Increase the Inspection Level?

• Economics: to provide increased protection for larger lots

• Increase probability of acceptance at the AQL

• Decrease LTPD


79

z1.4 Single Sampling Plans for AQL = 1%

Letter

Code

Sampling

Plan

Pa @ AQL Actual

AQL (%)

LTPD (%)

E n = 13, a = 0 0.877 0.394 16.2

H n = 50, a = 1 0.911 0.715 7.56

J n = 80, a = 2 0.953 1.03 6.52

K n = 125, a = 3 0.963 1.10 5.27

L n = 200, a = 5 0.984 1.31 4.59

M n = 315, a = 7 0.985 1.27 3.71

N n = 500, a = 10 0.987 1.24 3.06

P n = 800, a = 14 0.983 1.16 2.51

Q n = 1250

a = 21

0.991 1.19 2.25


80

ASQ Z1.4

• Probability of acceptance (Pa) is NOT constant in z1.4

• Z1.4 changes the plan as the lot size increases, NOT to maintain a constant level of protection, but instead to give MORE protection to larger lots.

• Changes in lot size mean changing the sample size

• Alternatives:

• Use the average lot size to select a plan OR

• Use the largest lot size to select a plan, then inspect all lots using the same plan


81

ASQ Z1.4 Example

• Select a single sampling plan with AQL = 1%, Level 2 inspection. Lot sizes range from 4,000 to 8,000 units:

• From Table I (pg. 10), code letter is L

• From Table IIA (pg. 11), select plan n = 200, a = 5

• This plan provides the following protection (Table X-L) (pg. 51):

• AQL = 1.31% , slightly higher than the target AQL• LTPD10 = 4.64%• LTPD5 = 5.26%


85

Switching Rules

• z1.4 provides 3 inspection types: Normal, Tightened, and Reduced

• Normal: Default starting point

• Move to reduced when an improvement in lot quality is detected

• Move to tightened when a decrease in lot quality is detected


86

Switching Rules

• Reduced

•Smaller sample sizes than Normal, assumes quality is good

•If ten (10) consecutive lots pass the Normal inspection, move to Reduced

•If one (1) lot fails reduced inspection, move back to Normal


87

Switching Rules

• Tightened

•Lower acceptance numbers than Normal, therefore steeper OC curves

•If two (2) out of five (5) lots fail Normal, move to Tightened

•If five (5) consecutive lots pass Tightened, move back to Normal


88

Example:

• Find a plan with L-2 inspection, AQL = 1.0%• Lot sizes between 4,000 and 8,000 units

• From the Table:• Normal inspection : n = 200, a = 5• Tightened inspection: n = 200, a = 3• Reduced inspection: n = 80, a = 2, r = 5

• What does r mean in the Reduced inspection plan?

• For 0, 1, or 2 defects found in the sample, accept the lot and remain in Reduced inspection

• For 3, 4, or 5 defects fond in the sample, accept the lot but switch to Normal inspection.

• For 6 or more defects, reject the lot and switch to Normal inspection


89

Advantages and Disadvantages of Switching Rules

• Advantages• Improved protection during stationary conditions• Reduced sample size during periods of good quality• Work best in a well established process in good control.

• Disadvantages• Bad protection during periods of changing quality. • No protection against isolated bad quality batches or first batch in a run of bad

ones.• More complex to administer


90

ASQ Z1.9 (2003)

• Equivalent z1.4 but for variables

• Also has normal, reduced and tightened plans, with similar switching rules

• Plans are organized by letter code and AQL

• AQLs in z1.9 go from 0.04% to 15%

• z1.9 has 5 inspection levels : General I, II, III; Special S3, S4• (A7.1): “Normal” is General Level II• We can use Level I for less discrimination or Level III for more• S3 and S4 are used for small sample sizes and large sampling risks


91

ASQ z1.9 vs z1.4

Table on p.101 provides a comparison between:

z1.9 (2003)z1.4 (1972) z1.9 (2003)


92

ASQ Z1.9 : Methods

• Z1.9 includes the following methods

• Unknown variability, standard deviation method•Individual spec (LSL or USL) p.32•Double spec (LSL and USL) p. 38

• Unknown variability, range method•Individual spec (LSL or USL) p.55•Double spec (LSL and USL) p. 62

• Known variability•Individual spec (LSL or USL) p.80•Double spec (LSL and USL) p. 88


RS 93

Z1.9 Tables

IVT Amsterdam MAR2017

IVT Amsterdam MAR2017 94

95

Example:

• Our quality characteristic has LSL = 225. • AQL target = 1 %• Lot sizes = 100,000 units• Process standard deviation is unknown

• Lot size = 100,000 units, Normal sampling (Gen Lvl 2)• Sample size code letter = N

• From Table B-1 in p.36 for unknown variability• for N the sample size is n = 150• for an AQL = 1%, k = 2.03


Key Take-Aways:• The protection offered by a sampling plan is defined by what it routinely accepts and rejects.

• AQL represents your Producer’s Risk, quantifies what your plan routinely accepts.

• LQ represents your Consumer’s Risk, quantifies what your plan routinely rejects.

• Sample size has the most significant effect on the protection offered by a sampling plan.

• To select a valid sampling plan:• Establish the purpose of the inspection (validation, regular production, etc.)• Pick AQL and LTPD.• Select or calculate a plan that provides the desired level of protection.

• Knowing the AQL and LTPD of all our sampling plans allows us to know the level of protection they offer.


Selecting Sampling Plans with Statistical Confidence

Select AQLs and LTPDs that provide appropriate protection

KNOW the AQLs and LTPDs for ALL your sampling plans

RANDOMIZE your samples97(c) 2017 Raul SotoIVT Amsterdam MAR2017

98

Q & A


References1. Ferryanto, Liem. "Statistical Sampling Plan for Design Verification and Validation of Medical Devices." Journal of Validation Technology

(2015): n. pag. Web. http://www.ivtnetwork.com/article/statistical-sampling-plan-design-verification-and-validation-medical-devices

2. Flaig, John. “Does c = 0 Sampling Really Save Money?” Quality Digest (2013): n. pag. Web. https://www.qualitydigest.com/print/23830

3. Ghasemi A, Zahedias S. Normality Tests for Statistical Analysis: A Guide for Non-Statisticians. Int J Endocrinol Metab. 2012;10(2):486-9. DOI: 10.5812/ijem.3505 http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3693611/

4. Gojanovic, Tony. "Back to Basics: Zero Defect Sampling." ASQ Quality Progress (2007): n. pag. Web. http://asq.org/quality-progress/2007/11/basic-quality/zero-defect-sampling.html

5. Niles, Kim. "Sample Wise: Settling on a Suitable Sample Size for Your Project Is Half the Battle." ASQ Quality Progress (2009): n. pag. Web. http://asq.org/quality-progress/2009/05/back-to-basics/sample-wise.html

6. Schilling, Edward G., and Dean V. Neubauer. Acceptance Sampling in Quality Control. Boca Raton: CRC, 2009. Print. https://www.crcpress.com/Acceptance-Sampling-in-Quality-Control-Second-Edition/Schilling-Neubauer/p/book/9781584889526

7. Taylor, Wayne A. Guide to Acceptance Sampling. Lake Villa, IL: Taylor Enterprises, 1992. Print.

8. Taylor, Wayne. "Article - "Acceptance Sampling Update" N.p., n.d. Web. http://www.variation.com/techlib/as-9.html

9. Taylor, Wayne. "Article - "The Effect of Lot Size" N.p., n.d. Web. http://www.variation.com/techlib/as-3.html

10. Taylor, Wayne. "Article - "Statistically Valid Sampling Plans" N.p., n.d. Web. http://www.variation.com/techlib/as-2.html

11. "What Kinds of Lot Acceptance Sampling Plans (LASPs) Are There?" NIST Engineering Standards Handbook Section 6.2.2, n.d. Web. http://itl.nist.gov/div898/handbook/pmc/section2/pmc22.htm


http://www.ivtnetwork.com/article/statistical-sampling-plan-design-verification-and-validation-medical-devices

https://www.qualitydigest.com/print/23830

http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3693611/

http://asq.org/quality-progress/2007/11/basic-quality/zero-defect-sampling.html

http://asq.org/quality-progress/2009/05/back-to-basics/sample-wise.html

https://www.crcpress.com/Acceptance-Sampling-in-Quality-Control-Second-Edition/Schilling-Neubauer/p/book/9781584889526

http://www.variation.com/techlib/as-9.html



http://itl.nist.gov/div898/handbook/pmc/section2/pmc22.htm

statistics in validation sampling plans - cbinet.com session... · statistics in validation...

Documents