thoratec workshop in applied statistics for qa/qc, mfg, and r+d part 3 of 3: advanced applications...

Thoratec Workshop in Applied Statistics for QA/QC, Mfg, and R+D Part 3 of 3: Advanced Applications Instructor : John Zorich www.JOHNZORICH.COM [email protected] Part 3 was designed for students who have taken Part 1 and Part 2 of these workshops, or who have had a college-level statistics course.

John Zorich's Qualifications: 20 years as a "regular" employee in the medical device industry (R&D, Mfg, Quality) ASQ Certified Quality Engineer (since 1996) Statistical consultant+instructor (since 1999) for many companies, including Siemens Medical, Boston Scientific, Stryker, and Novellus Instructor in applied statistics for Ohlone College (CA), Pacific Polytechnic Institute (CA), and KEMA/DEKRA Past instructor in applied statistics for UC Santa Cruz Extension, ASQ Silicon Valley Biomedical Group, & TUV. Publisher of 9 commercial, formally validated, statistical application Excel spreadsheets that have been purchased by over 80 companies, world wide. Applications include: Reliability, Normality Tests & Normality Transformations, Sampling Plans, SPC, Gage R&R, and Power. Youre invited to connect with me on LinkedIn.

Self-teaching & Reference Texts RECOMMENDED by John Zorich Dovich: Quality Engineering Statistics Dovich: Reliability Statistics Juran: Juran's Quality [Control] Handbook Natrella: Experimental Statistics ( 0.425,... and even tho the correlation coefficient is very high... this plot is slightly curved; and therefore this data is not truly normal (it is almost Normal). Is "almost" good enough for critical products?

The "inverse" ( = 1 / X ) transformation gives a much straighter line on "Normal Probability Plotting" paper, and so the distribution is "Inverse Normal" rather than "Normal" continued from previous slide...

Using the 12-pt data set from the previous slide... Z(F) = 4.15 = 0.002% failure rate = 99.998% reliability at 95% confidence (solid triangle is the extrapolated value; the hollow triangle is the upper 1-tailed 95% confidence limit). By comparison, Normal K-tables yielded slightly less than 99.9 % reliability. Z (F) In John Zorich's view, Reliability Plotting is more accurate than Normal K-tables. In this case, we obtained a "better" result; but the reverse may occur on a different data set. Using Reliability Plotting to extrapolate transformed data to the transformed spec ( 1 / X = 1 / 5.5 = 0.1818 ), we have this result:

If you have replicate measurements in your data set (e.g., 4 data points each = 0.35), and if you plot each of the individual exact data points, the resulting line may be inappropriate. The reason it may be inappropriate is that "Linear Regression" (which is the mathematical tool used in Reliability Plotting) does not perform well when replicates are present (especially when the replicates are near one or the other of the ends of the straight line). Instead of using your individual "exact" data, it may be better to pool identical values (or very similar values) into groups ("intervals"), the way you do for a histogram. Then calculate the % cumulative for each of the cumulative groups. Reliability Plotting: EXACT vs. INTERVAL

EXACT %F 0.356.7% 0.3516.3% 0.3526.0% 0.3535.6% 0.6845.2% 1.2254.8%. 1.5664.4% 1.9174.0% 2.1783.7% 2.1893.3% Reliability Plotting: EXACT vs. INTERVAL

EXACT %F INTERVAL %Cumulative 0.356.7% 0.3540% 0.3516.3% 0.6850% 0.3526.0% 1.2260% 0.3535.6% 1.5670% 0.6845.2%1.9180% 1.2254.8% 2.18 99.9999..% 1.5664.4% 1.9174.0% 2.1783.7% 2.1893.3% Reliability Plotting: EXACT vs. INTERVAL (%F and %Cumulative are calculated differently) In cases such as this, plotting INTERVAL values will produce a more accurate line than plotting EXACT values, but you may need to "censor" the largest value. See next slide for this data plotted.

Reliability Plotting: EXACT vs. INTERVAL (continued from previous slide)

Burst strength ( actual data !!) 60 devices tested; the minimum spec was 0.40 psi. The (sorted) raw data was...

Using Reliability Plotting.xls using Z(F) vs. X(untransformed) (this is equal to Normal Probability Plotting paper) Burst strength Because this data does NOT form a straight line on NPP paper, it's not valid to use K-factor tables.

Also, notice that the data includes many replicate values... Burst strength

...and notice that on a basic cumulative plot ( = F(untransformed) vs. X(untransformed) ) the data seem to include 2 different populations: Burst strength A single population would look like a smooth " S " curve. This one has a break or corner in it, indicating a dual population. To use Reliability Plotting, must "censor" these data, (they appear as shoulder on a line chart -- see next slide)

(continued from previous slide) Mixtures of distributions appear as bi-modal frequency distributions, or as a single mode with a shoulder, like this: Frequency Distribution "Shoulder" (the data that must be "censored", to use Reliability Plotting)

Here is how to convert the ( n = 60 ) data, with its many replicate values, into interval data: Burst strength 48 -- 2 = 2/60 = 3.3 % 49 -- 1 = 3/60 = 5.0 % 51 -- 1 = 4/60 = 6.7 % 52 -- 1 = 5/60 = 8.3% 53 -- 4 = 9/60 = 15.0% 55 -- 1 56 -- 4 57 -- 1 58 -- 1 59 -- 3 60 -- 1 61 -- 7 etc. For "Interval" plots, must use % cumulative, not "F" Do NOT plot zero values e.g., do not plot 54 -- 0

Burst strength Important !! This pairing IS the best straight line but does NOT have the highest CC. If convert from "exact" to "interval" AND censor (i.e., do not plot) values above 0.8 psi, then... 95% confidence at 99.8707 % Reliability using Log(X) vs Z(%cum) (i.e., LogNormal) Log(X) vs Z(%cum) (i.e., LogNormal)

Statistical Analysis of Gages Calibration, Metrology, & Measurement Uncertainty

Regulatory Requirements... ISO 9001 & 13485 "The organization shall determine the monitoring and measurement to be undertaken and the monitoring and measuring devices needed to provide evidence of conformity of product to determined requirements." MDD: Annex II, V, + VI "Application of the quality system must ensure that the products conform to the provisions of this Directive which apply to them at every stage, from design to final inspection....It shall include in particular an adequate description of...the test equipment used; it must be possible to trace back the calibration of the test equipment adequately." Question: What does that word "adequately" mean? The combination of calibration records AND the process of choosing calibrated equipment must "provide evidence of conformity. If the wrong instrument is chosen, it provides no evidence of conformity, even if it is calibrated.

Vocabulary ACCURACY is defined using the mean of several measurements. Subtracting that mean from the "true value" gives the Inaccuracy or Bias. Divide Inaccuracy by the true value", then multiply by 100, to yield the "% Inaccuracy". Commonly, accuracy is assessed by taking only a single measurement; and if that measurement is within the tolerance allowed, then the instrument is said to be within tolerance. Calibration vendors use N = 1, i.e., a single measurement, unless explicitly told not to. That "N=1" may NOT be a good thing, because accuracy cannot be determined accurately without taking multiple readings !!

Vocabulary PRECISION (also called repeatability, especially in the specification section of measurement equipment owners manual) is assessed by taking several measurements of the same item, and calculating their standard deviation (= the Imprecision). Divide Imprecision by the true value", then multiply by 100, to yield the "% Imprecision". Typically, calibration vendors do not check for precision, unless you explicitly tell them to. This may NOT be a good thing, if product pass/fail decisions are based upon a single measurement (precision tells you how reliable is a single measurement) !!

Vocabulary RESOLUTION is equivalent to the number of digits you can read on an instrument, e.g. an instrument that can output this measurement 2.71634 inches has "higher resolution" than one that can output only this 2.716 inches Unfortunately, there is no reliable relationship between resolution and accuracy and/or precision !! However, when there is no other info available about measurement uncertainty, the standard deviation of repeated measurements is estimated as follows: The width of the smallest readable unit (e.g., 0.00001 and 0.001 in the examples above) divided by 3.464 NOTE: some micrometers read only 0 or 5 in the last digit, in which case, the smallest readable unit is not 1 as in the examples above, but rather 5 .

Uncertainty There is always some uncertainty as to the degree with which a sample represents the population from which it was drawn. That type of uncertainty cannot be reduced by anything other than larger sample sizes. In addition to that uncertainty, there is uncertainty caused by the measurement process itself. The science of "Metrology" aims to quantify, control, and reduce the uncertainty caused by the measurement process.

Some Tools of Metrology are... Calibration Assesses & corrects accuracy & precision Gage R&R Assesses precision only Gage Correlation Assesses accuracy only, vs. another device/person Gage Linearity & Bias Assesses accuracy only, vs. a gold std Gage Stability (i.e., instability) Studies Assesses systematic drift in accuracy over time Uncertainty Budgets Summarizes available measurement uncertainty info, and then suggests how QC specs should be modified. Not discussed in this seminar

Calibration concerns Product tolerance specification = target +/ 4 units This situation above ( a ratio of 4 : 1 ) is generally considered acceptable, since it mimics the practice that ISO 17025 mandates for Calibration companies. Before setting a design specification, a company should decide on a desired relationship between specification tolerance & measurement equipment accuracy, e.g.... Equipment calibrated accuracy = nominal +/ 1 unit

Calibration concerns Product tolerance specification = 100 +/ 4 units SOME MEDICAL DEVICE COMPANIES UNKNOWINGLY HAVE RATIOS OF 1 : 1, FOR EXAMPLE... EXAMPLE OF WORST-CASE SCENARIO WITH THAT RATIO Calibration data: NIST-traceable standard of 100 reads 96 (= meets requirement, but equipment is reading 4 units low). Equipment is then used to measure product; if result is 104, it passes spec, but "true" value is really 108. Thus, product should be rejected but instead is passed. Equipment calibrated accuracy = 100 +/ 4 units

Gage R&R A "Gage R&R" study quantitates measurement uncertainty that is due to the combination of the instrument & users. Typically, the output is the width of the interval that includes the middle 99% of the "Normal" distribution of individual measurements. Let's call that the "Uncertainty Interval". In an R&R study, we primarily identify the... Total Variation uncertainty interval (from all causes) Repeatability uncertainty interval (this is uncertainty caused by the inability of a measurement instrument [i.e., the gage] to produce the same measurement result when used repeatedly to measure the identical part) Reproducibility uncertainty interval (caused by the inability of different users of the same gage to produce the same result when measuring the identical part)

Equipment Control This is an example of a data input table for a simple Gage R&R study (a complicated one involves more than one gage). Typically, analysis of this data requires a computer program capable of the quite-involved Gage R&R calculations.

This shows that gages+people (i.e., repeatability+reproducibility) cause variation that consumes 12.98 / 40 = 32.5 % of the QC Spec Interval. In this case, re-training and/or standardizing personnel practices will help only a little to decrease that %. To decrease variation significantly, we need to buy better gages (gages caused most of this R&R variation, i.e., 32.2 % vs. 4.3 % ). Uncertainty

Gage R&R using Excels Data Analysis Add-in Option, on the DATA tab: ANOVA: Two Factor With Replication Reproducibility (99%) = 5.15 x StdDev( 4.63, 4.42, 4.65 ) Repeatability (99%) = 5.15 x sqrt [ ( 578.76 57.87 0.94 ) / ( 89 9 2 ) ] Gage R&R (99%) = sqrt ( Reproducibility 2 + Repeatability 2 )

Gage Correlation A "Gage Correlation" study typically is used to compare measurements of identical parts by 2 different companies --- for example, by the Supplier of the part, and their Customer. One practical use is to validate that the Supplier gets the "same" answer as the Customer, and thus the Customer justifies using Supplier-provided QC data rather than the Customer having itself to perform QC (the part could then justifiably go "dock to stock"). In a Gage Correlation study, we identify the... Linear Regression relationship between the measurements by the 2 companies Correlation Coefficient for that linear regression Offset Values that could be used to "correct" any identified differences in measurement between the 2 companies. Such a study could also evaluate R&D vs. Pilot Production, or Pilot Production vs. Manufacturing, in the same company.

Gage Correlation This is an example of a data input table for a simple Gage Correlation study (a complicated one would involve 3 or more gages). In this case, each "Set #" is a unique part to be measured by "Gage # 1" at one company (or department), and "Gage # 2" at the other company (or department).

Altho not identical, the measurements from the 2 companies look to be linearly related and to be highly correlated.

For Gage#1 to read like Gage #2, multiply each Gage# 1 result by 0.998 and then subtract 4.7732 from that result... 4.77 (based on equation shown on previous slide).

Gage Linearity A "Gage Linearity" study typically is used to evaluate performance over a wide range of values (e.g., over the entire range of values that the gage is capable of measuring). In a Gage Linearity study, we... Use "gold standards" or any parts for which we believe we know the "true" answers very accurately Make repeated measurements of the gold standards, using a single on-test gage Graph the "error" (a.k.a., "bias") for each measurement (i.e., how far off each measurement is from the "true" value) Determine if the error is statisticly significantly different from 0.000 (i.e., if there is no error, then there is no bias) If the gage is found to have (statisticly) "no" bias thruout its tested range, we say the gage has acceptable "linearity" in that range.

Gage Linearity This is an example of a data input table for a Gage Linearity study. The "gold standards" could be very accurate gage blocks.

0.000 Bias = error Consider the curved lines to be 95% confidence limits on the sample result avgs, in the measurement range of 2 thru 10 (formulas for such limits are found in advanced stat books). Because the horizontal "Y = 0.000 Bias" line is fully contained by the solid curved confidence interval lines, we conclude that this gage has acceptable linearity in the range of 2 thru 10.

Gage Bias A "Gage Bias" study is, in effect, a one-point Gage Linearity Study, in which is used either a gold standard ("Reference") calibrator or a gold standard ("Reference") gage. The difference between the on-test gage and either the gold-standard gage or the gold-standard calibrator is considered the "bias". The virtue of a Gage Bias study is that it is simple & quick --- it uses one person, one (on-test) gage, and one part measured several times at one sitting. Its analogous to a one-point calibration. See output of this study, on the next slide >>

Gage Bias Upper & Lower conf. limits of sample mean, calculated with t-tables per any intro stat book

Gage Linearity vs. Bias vs. Calibration In either case, nothing is known of uncertainty for any point in this range --- 0.000 is a given (a "tare point"), not a calibration pt. Value out-putted by On-Test gage True Value (= calibration standard or reference gage) Uncertainty is known in this range if all 3 levels are evaluated or calibrated. Uncertainty is known only here if only this 1 level is evaluated or calibrated 1-point calibration is OK, if only measure here for QC, Mfg., R&D etc.

Uncertainty Budget Estimate the standard deviation of uncertainty for each of the uncertainty sources for which you have information, e.g., for a 99% interval Gage R&R... StdDev = GageR&R interval divided by 5.15 for a calibration tolerance (e.g., "Mfg's specs")... StdDev = tolerance interval divided by 4.00 for uncertainty in the calibration calibrator (typical)... StdDev = calibration tolerance divided by 16.000 other (e.g., gage instability)...?? Square each StdDev, sum them, take square root of sum. Multiply that by factor for interval you wish to calculate (e.g., for 99%, factor = 5.15; for 95%, factor= 3.92 ); the result is called the "Expanded Uncertainty Interval" Divide the width of product's spec interval by that interval. There is general agreement in industry, based on ISO & NIST recommendations, that if that ratio is less than 4.00, for a 95% interval, then the measurement equipment or measurement process is NOT suitable for given product.

Graphical Summary of the Problem of & Solution to Measurement Uncertainty Design specification range 95% or 99% Expanded Uncertainty interval (based upon whatever is included in the "Uncertainty Budget") This guard-banded specification range has the advantage that any single measurement that falls within it is guaranteed to fall within the design specification range ( 95% or 99% probability). Without "guard banding", the actual range being used to pass/fail measurement results is this "expanded specification".

What is acceptable, if the goal is to... "provide evidence of conformity of product to determined requirements" per ISO 9001 & 13485 ? There is no regulation or official guidance document that discusses uncertainty budgets, expanded specifications, and guard-banding (e.g., ISO 14969 says only that the documented procedure should include details of equipment type, unique identification, location, frequency of checks, check method, and acceptance criteria). Therefore, ISO, CE, & FDA auditors have no firm basis on which to force companies to implement metrology policies. Therefore, it is up to the company whether or not its product is QCd vs. the expanded specification interval (which is always wider than the design-based specification interval).

Classic QC Sampling Plans (and their alternatives)

Standards & Regulations ISO 9001:2008 + ISO 13485:2003 8.1: "[Mfg] shall...implement...analysis...processes needed to demonstrate [product/process] conformity....This shall include determination of applicable...statistical techniques". FDA's "GMP" (21CFR820.250) (re: medical devices): "Sampling plans...shall be...based on a valid statistical rationale... Each manufacturer shall...ensure that sampling methods are adequate for their intended use." FDA's "Medical Device Quality Systems Manual" "...all sampling plans have a built-in risk of accepting a bad lot. This sampling risk is typically determined in quantitative terms by deriving the 'operating characteristic curve' [which]...can be used to determine the risk a sampling plan presents. A manufacturer should be aware of the risks the chosen plan presents....A manufacturer shall be prepared to demonstrate the statistical rationale for any sampling plan used. US Dept of Defense MIL-STD 1916 "...sampling inspection is...redundant...and...unnecessary." "...consider [using an] alternative acceptance method."

Basic Types of Sampling Plans In an attribute sampling plan, "quality" is measured by the observed % of the sample that meets specification. In a variables sampling plan, "quality" is measured by the estimated % of the population that meets specification (based upon Sample Mean & either Sample Range or Std Deviation, & assuming data Normality (see statement of normality requirement, in ANSI/ASQC Z1.9-1993, pp. 2-3). Only attribute sampling plans are discussed in this class, because they are currently still the dominant ones used in the medical device industry (see next slide).

Virtually 100% of U.S. medical device companies use AQL Attribute sampling plans for their IQC inspections ( IQC = Incoming or Receiving Quality Control ) less than 1% use a "variables" sampling plan less than 1% use an LQL sampling plan. That conclusion is based upon John Zorich's history of... full-time quality-system (& statistical) consulting, 19992013 working halftime as an auditor for European ISO / Notified- Body registrars, TUV and KEMA/DEKRA, 20002013 performing more than 500 quality-system audits at more than 200 medical-device companies in USA, 19992013 Information collected by John Zorich:

Attribute Sampling Plans An attribute sampling plan is a written procedure for... choosing a fraction of an incoming lot (the fraction = the sample) deciding on the acceptability of the entire lot based on the observed quality of the sample (the lot "passes" if the number of defects or defective parts is not more than the " C " = "acceptance number" that is allowed by the plan) Sampling-plan-use involves a RISK of approving a bad lot (a risk to end-user customer, possibly).

Attribute Sampling Plans AQL stands for Acceptable Quality Level or "Acceptance Quality Limit". The %AQL of an AQL sampling plan is the product quality ( = lots having that % defective) which the sampling plan will approve almost all the time (there is no generally accepted numerical definition of %AQL). %AQL = " I am happy with AQL% defective " LQL stands for Limiting Quality Level or "Lower Quality Limit". The %LQL of an LQL sampling plan is the product quality ( = lots having that % defective) which the sampling plan will reject almost all the time (there is no generally accepted numerical definition of %LQL). %LQL = " I'm not happy with LQL% defective"

99% of U.S. med device companies that do IQC inspection use one of these two plans: ANSI/ASQC-Z1.4 = ISO 2859-1 = MilStd105E AQL attribute sampling plan, widely used because of it's explicit endorsement by the FDA, in its Medical Device Quality Systems Manual: "[Sampling] Plans should be developed by qualified mathematicians or statisticians, or be taken from established standards such as ANSI Z1.4" The plan's stated purpose "is not intended as a procedure for estimating lot quality or for segregating lots"...but rather to "induce a supplier to maintain a process average...[and to control ] consumer's risk...." Squeglias "Zero Acceptance Number Sampling Plans" AQL attribute sampling plan, widely used in industry, because of its smaller sample sizes & implicit endorsement by ASQ (it's published by the official ASQ Quality Press). The plan's stated purpose is "provide essentially equal or greater LQ protection at the 0.10 consumer's risk level".

Classic (AQL Attribute) QC Sampling Plans (are they worth the effort?)

"FDA / ISO auditors won't ask any challenging questions." That is true, for field auditors (= untrained in statistics). PMA / CE auditors and their staff statisticians are much more statistically savvy, and have been known to ask you to justify your sampling plans, based on risk analysis, for critical parts (e.g., implant components). "Such plans provide statistical assurance that... 1.suppliers provide consistently high quality product, 2.we are not accepting low quality product, & 3.our Parts Storeroom has a known quality level." Let's now examine those 3 claims... What people say about why they use traditional AQL sampling plans is...

ANSI Z1.4

" Zero Acceptance Number Sampling Plans " ( by N. L. Squeglia, 4 th ed.)

Attribute Sampling Plans For lots of a given part #, when inspected using a given sampling plan, the % of lots (not the % of parts) that meet specification is called the "Pass Rate". The Pass Rate for a sampling plan is always... ( #1 )high for good lots ( = have low % defectives) ( #2 )low for bad lots ( = have high % defectives) ( #3 )intermediate for lots of intermediate quality.

Attribute Sampling Plans The manner in which lot quality and lot size affect the Pass Rate is described by 2 types of... Operating Characteristic curves = OC curves In this presentation, those 2 types of curves are called... % Defective OC Curves and Lot Size OC Curves examples of each are shown on upcoming slides...

Predicting Pass Rates % Defective OC curve for a 4% AQL sampling plan (ANSI Z1.4) N = 1000 n = 80 c = 7 This is the typical OC CURVE found in text books, i.e., Lot % Defective vs. Pass Rate OC curves "describe the long run behavior of a sampling plan. They do not tell the user what can be said about a particular lot that has just been accepted or rejected." D. J. Wheeler, 2006 in EMP III, pg. 152

Predicting Pass Rates % Defective OC curve for a 4% AQL, C=0 sampling plan (Squeglia's 4th edition) N = 1000 n = 15 c = 0

How can such variations in "4% AQL" pass rates ensure that "Suppliers provide consistently high quality product "? N = 1000 n = 80 or 15 c = 7 or 0 Which "4% AQL" plan should be used? In order to focus on consumer risk (per FDA & Squeglia), need to focus on this LQL point. LQL sampling plans focus on % defective that will be rejected almost all the time.

Attribute Sampling Plans Probability of Acceptance of a Single Lot from a Sequence of Lots from a Stable Process (MSExcel function) =binomdist(C,S,F,True) C = Number of defectives allowed in the sample S = Sample size F = Fraction of lot that is defective True = tells the program to add up the probabilities for 0, 1, 2, 3,.... thru to C. (continued on next slide)

(free) "Self-made Sampling Plans.xls"

It is NOT possible to use an AQL% to explain a "valid statistical rationale" for a sampling plan. The only way to achieve a "valid statistical rationale" for classic sampling plans is to... review the Risk Management documents (e.g., FMEA), to determine if "IQC" processes have been identified as being a "mitigation"; if they have, then... choose a sampling plan whose LQL supports Risk- Management statements such as..."In IQC, mitigation will involve using a sampling plan that ensures that component lots that are 1% or more defective are rejected approximately 90% or more of the time"). If Risk Management docs do not identify IQC inspection as mitigating a product or process risk, then it is reasonable to conclude that IQC does not pose any risk to the end-user (e.g., patient or doctor); in that case, only "business risks" are important, and therefore any sampling plan is "valid". Control of Sampling Plan's Consumer Risks

Suppose that your Risk Management docs state that "consumer risk" is not acceptable if component lots are more than 5% defective, & claim that the IQC AQL attribute sampling plan shown below provides "mitigation" so that such lots are rejected 95% of time. Is that claim true? % Defective OC curve for a 4% AQL sampling plan N=1000 n=80 c=7 NOT TRUE, because in order to ensure that "We are not accepting low quality product ", a 4 or 5% LQL sampling plan should be used, not this 4% AQL one. However, even with LQL plans, we don't know exactly what level of "statistical confidence" we can claim for a given lot being inspected.

Do AQL plans control consumer risk consistently? ASQC-Z1.4, general, level II, single, normal, 4% AQL Lot Sample If lot is 5% Defective SizeSize " C " Pass Rate is... 20 3 085 % 100 20 295 % 1,000 80 796 % This shows a consistent approval rate of about 90%; but "5% defective" is at the AQL top of the OC curve (where "supplier risk" is controlled), & so is irrelevant to control of "consumer risk".

ASQC-Z1.4, general, level II, single, normal, 4% AQL Lot Sample If lot is 15% Defective Size Size " C " Pass Rate is... 20 3 060 % 100 20 238 % 1,000 80 7 6 % Do AQL plans control consumer risk consistently? This shows an inconsistent approval rate at the LQL bottom of the OC curve (where "consumer risk" is controlled).

% Defective OC Curves for ASQC-Z1.4, general, level II, single, normal, 4% AQL All three of these "4% AQL" plans have same high Pass Rate when lot is low % defective, but differ greatly at high % defective. Do AQL plans control consumer risk consistently?

Lot Quality is Lot Size OC Curves for ASQC-Z1.4, general, level II, single, normal, 4% AQL ANOTHER KIND OF OC CURVE Do AQL plans control consumer risk consistently?

Lot Quality is Lot Size OC Curves for v4 ASQC-C=0, Single Sample, 4% AQL Do AQL plans control consumer risk consistently? Conclusion: Z1.4 and C=0 AQL sampling plans do not control consumer risk consistently, unless Lot Size is controlled.

Important lesson from 2 previous slides: When the Receiving-QC Inspection Pass Rate changes dramaticly (increasing or decreasing), you should not come to any conclusion about the cause (e.g. "Supplier is doing much better!" or "Supplier is doing much worse!"), until you examine the "Lot Size OC Curve" versus the size of the lots that have been received before and after that Pass Rate changed dramaticly. Such an examination may reveal that the dramatic "change" is a false impression, and that it is due to a change in size of lots received, not due to a change in the quality of the lots received!

Arbitrariness(?) of Sampling Plans ** = ASQC-Z1.4, general, level II, single, normal, 4% AQL Lot Sample If lot is 5% Defective SizeSize" C " Pass Rate is... 20 3** 0**85 %** 10020** 2**95 %** 1,00080 690 % 1,00080** 7**96 %** 1,00080 899 %

** = ASQC-Z1.4, general, level II, single, normal, 4% AQL Lot Sample If lot is 5% Defective SizeSize" C " Pass Rate is... 1,000125 1096 % 1,00080** 7**96 %** 1,00040 496 % During World War II (when these sampling plans first became common), one possible use for the larger-than- needed sample sizes was for discrimination between mediocre lots and excellent lots (both of which pass QC). Arbitrariness(?) of Sampling Plans

Sample Size for lot size of 1000 " C " for that sample size All 3 plans from the previous slide have the same high Pass Rate when lot is 5% defective, but differ greatly at high % defective. Middle line is ASQC Z1.4 4% AQL Arbitrariness(?) of Sampling Plans

" Zero Acceptance Number Sampling Plans " ( by N. L. Squeglia, 5 th ed.) This table from this 5th edition, has many sample-size changes (shown circled), compared to the 4th edition. In some cases, the sample size has increased dramatically (e.g., if Lot Size = 100 and AQL=1.5, then...Sample Size is now 19 instead of 12, which is a 58% increase, and... the pass rate for a 1.5% defective lot of that size drops from 83% in 4th edition to 75% in the 5th edition).

How much defective product is in your approved-parts Storeroom? The % of defective product that is in your Approved-parts Storeroom is a function of the... quality of lots received sampling plan used lot size received (as we saw on previous slides) A relevant term that defines that % is... AOQ (Average Outgoing Quality). AOQ is the resulting average % defective in the Approved Storeroom, assuming that LotSize, SampleSize, C value, and received Lot%Defective all remain constant. If the received Lot%Defective varies from lot to lot, then the potential AOQ varies lot to lot. The worst AOQ possible is then called the Average Outgoing Quality Limit or AOQL.

AOQ can be easily calculated using a classic formula found in any sampling-plan textbook, but AOQL is typically available in tables in the back of published Sampling Plans. Squeglia "C=0" (4th ed.) Therefore, unless only a specific small range of lot sizes (e.g., 91--150) is allowed to be purchased, AOQL (and possibly storeroom quality) varies lot-to-lot! This means that if a 1.0 % AQL Sampling plan is used, and if the parts Supplier consistently sends lots that are about 6 to 7% defective, and if Lot Size is consistently 91 to 150, then Approved Stores will consistently contain about 2.6% defective of that part.

The classic formula for AOQ assumes that good parts are used to replace all defective parts encountered either in any sample or in a 100% inspection of a rejected lot, before that lot is approved and moved into the Approved Storeroom. Based upon John Zorichs experience auditing more than 200 US medical device manufacturing companies from 1999 to 2014, no company follows those classic instructions. Instead, virtually all companies do NOT replace defective parts with good parts, but rather return defective parts to the Supplier for credit on future shipments of normal lot sizes. If N=100, n=16, C=1, & received Lot%Defective = 10%, then... AOQ = 4.20% using the classic formula AOQ = 4.92% when good parts do NOT replace defectives. % Defective in the Approved Storeroom is affected by IQC & Supplier-Control practices

When we use classic AQL attribute sampling plans, we settle for knowing almost nothing about the specific lot of product from which the sample came. If our boss were to ask, all we can say is "the lot passed". We don't know the % defective in the just-passed lot We may not know the actual % defective in Stores (we may know only theoretical worst case = the "AOQL") If all we do is focus on the AQL, we dont even have a clear definition of a bad lot (the chosen "AQL%" is considered good-enough %defective) Instead, for each Lot of product received, why don't we calculate what % of that lot is "in-spec" ? That is Why not calculate its reliability at 95% confidence ("reliability" here means "% in-specification"), and use % reliability specs (instead of % AQL specs)? Why do we do so much work for so little information?

Using "Reliability Calculations" instead of AQL Sampling Plans Duke Empirical (a well-known contract Design & OEM manufacturer in Santa Cruz, California, with a long list of medical device clients, including billion-dollar corporations) does not use any AQL sampling plans for IQC, unless mandated by the client. Instead of %AQL specifications, Duke uses %Reliability specs (all at 95% Confidence), as described in this seminar. The client is asked to choose an IQC %Reliability spec for each of its parts received by Duke. If the client is not ready to do that, Duke defaults to the %Reliability listed by Risk class in Duke SOPs (e.g., human-implant parts are high risk). The future is now:

thoratec workshop in applied statistics for qa/qc, mfg, and r+d part 3 of 3: advanced applications...

Documents