conformance testing measurement decision rules(nasa)(important)

8/12/2019 Conformance Testing Measurement Decision Rules(NASA)(IMPORTANT)

1/15

Abstract

Conformance Testing Measurement Decision Rules

Speaker /Author:Scott M. Mimbs

National Aeronautics and Space Administration (NASA)John F . Kennedy Space Center

Kennedy Space Center, FL 32899Phone: 321-861-5184Fax: 321-867-1740

sco tt .m .mimb s@ nasa. gov

The goal o fa Quality Management System (QMS) as specified in ISO 9001 and AS9100is to provide assurance to the customer that end products meet specifications. Measuringdevices , often called measuring and test equipment (MTE) , are used to provide theevidence of product conformity to specified requirements. Unfortunately, processes thatemploy MTE can become a weak link to the overall QMS if proper attention s not givento the measurement process design, capability, and implementation. Documented

decision rules establish the requirements to ensure measurement processes provide themeasurement data that supports the needs of the QMS.

Measurement data are used to make the decisions that impact all areas oftechno logy .Whether measurements support research, design , production , or maintenance, ensuringthe data supports the decision s crucial. Measurement data quality can be critical to theresulting consequences o f measurement-based decisions.

Historically , most industries required simplistic , one-size-fits -all decision rules formeasurements. One-si ze- fits-all rules in some cases are not rigorous enough to provide

adequate measurement results , while in other cases are overly conservative and too costlyto implement. Ideally , decision rules should be rigorous enough to match the criticalityof the parameter being measured, while being flexible enough to be cost effective. Thegoal o f a decision rule is to ensure that measurement processes provide data with asufficient level of quality to support the decisions being made - no more , no less.

This paper discusses the basic concepts of providing measurement-based evidence thatend products meet specifications . Although relevant to all measurement-basedconformance tests , the target audience s the MTE end-user , which is anyone using MTEother than calibration service providers . Topics include measurement fundamentals , theassociated decision risks, verifying conformance to specifications, and basicmeasurement decisions rules.

Introduction

Selection and calibration ofMT has traditionally been the so le emphasis ofmeasurement quality assurance, or n other words , ensuring measurement data adequatelysupports decisions. With ever increasing technology requirements , current measurementquality assurance needs to include all pertinent variables within the measurementprocesses that provide the measurement data. There are many general-purpose and

12010 NCSL Intem ational Workshop and Symposium


2/15

discipline-specific documents that provide guidelines for developing decision rules . Afew o these are listed at the end o this paper , under the heading Additional relatedinformation. The list is not exhaustive, so further research is recommended priordeveloping decision rules . Also , selection or development o appropriate decision rulesrequires a basic understanding o measurement quality assurance. Therefore, thi s paperwill briefly address the background and fundamentals o measurement quality assurance.

Measurements Decisions

In the simplest terms , measurements are made to gain knowledge or to make decision sFor example , measurements are made in basic research to extend our knowledge, or morecommonly, they are used in commerce and industry to verify product conformance priorto acceptance. Many factors can influence a measurement result , and the measurementresult can have consequences long after the mea sure ment. Negative consequences trommeasurement results can range from wasted resources to loss o mission or life.Therefore , before designing a measurement proces s or establishing a measurementdecision rule , the basics o why the measurement is being made, and any associated riskso incorrect decisions has to be understood and considered.

Place to Begin

n all measurements , including calibration, two que stions should be posed prior tomaking a measurement.

I How good does the measurement need to be?

2 How good can the mea surement be made?

Many factors can affect the answer to these que stions , including available funds andcurrent technolog y n mo st cases, the first question is answered in respect to a busine sscase, balancing between the cost and the required quality , while the second questioninvolves existing technology and processes.

For the first question , the required quality o a measurement depend s on the reason forthe mea surement. Not all measurements require the same level o rig or. By focusingfirst on the reason the measurement is required, the how good is good enough becomesclearer.

The answer to the second question requires an understanding o how to evaluate thegoodness o a measurement. There are many factors which can negatively influence a

measurement result. These factors have to be understood , evaluated, and controlled , tosome degree , to achieve measurement results that meet the requirements established bythe first question. A better instrument does not always guarantee a better measurement ,since even a perfect instrument cannot fix a poor measurement pro cess.

Risks ssociated with Measurements

There is almost always some type o risk associated with decisions , including deci sion sbased on measurement data. AS9100C [I] defines risk as, An undesirable situation orcircumstanc e that has both a likelihood o occurring and a potentially negativeco nseque nce . The focus o measurement quality assurance is to quantify, and/ormanage the likelihood o incorrect measurement-ba sed decisions. When doing so ,there mu st be a balance between the level o effort and the risks re sulting trom making an

-2-2010 NCSL Intemational Workshop and Symposium


3/15

incorrect decision . In balan c in g the effort vers us the risk s, the decision (direct ri sk) andthe co nseq uences (indirect ri sk) of the me as urement mu st be co nsidered .

1. Direct Risk : This ri sk is directly associa ted with the measurement dat a andimp acts th e deci sio ns inv o lving a measurement (e.g., acce pt , reject , rework ,scrap).

2. Indirect Risk : This risk affects the quality or performance of end products whichstem from measurements. In other words thi s is th e consequence of an incorrectdeci sion. This type of ri sk may not be evident until after the pr oduct is in service .

Figure 1 illustrates direct and indirect measurement risk through a li fecycle. Thenegative impact of mea surement deci sion s can carry through the entire li fecycle;therefore , managing measurem ent-ba se d ri sks durin g each phase of the li fecycle is anesse ntial part of a quality sys tem .

Researc h

Theory MeasurementK n o wledg e .

Ba

..... ... . ........ ....... ...... .............. . ................................... .Develollment . ~

:I " ,'.. Test Object ive Measurement o n c lusi on .i: .' ,:

. . . ................ ..... .. . . Production

, ~

' Design Measurement Perform ance'

I Direct Risk I I nd irect Risk IFigure I A partiallifec ycle pha se illustr ati ng Direct an d Indir ect Risk associated with

measurements. As illustrated , consequences can promulgate through the Iifccycle.

Metrology s Two Fundamental TenetsBalancing the cost of measurem ent processes and the consequences of in correct decisio nsca n be cha ll enging. Key to developing cost-effective measur ement pr ocesses thatadequately manage the deci sion ri sks is th e application of metrol ogy S tw o fundamentaltenet s.

Measurement Traceability

Measurement Uncert aint y

Proper applicatio n of these tw o tenets provide s a flexible , thus cost-effective,measurement quality assurance program.

-3-2010 NCSL Internatio na l Workshop a nd Symposium


4/15

Meas u rement Traceability

Traceability establishes the link for a given mea surement unit (e .g . kg , DC etc.) to thenationa l or international standard for that unit o mea sure. The Bureau International desPoids Mesures (BIPM) is the organization charged with maintaining the InternationalSystem o Units (SI) reference standards , which are the global references f or

measurement units . Each industriali zed countryal

so has its own National MeasurementInstitute with legal authority to maintain and disseminate measurement units , butultimately all are traceable to the BIPM.

Figure 2 illu strates thi s chain o traceability along with each link ' s level o ri sk to theend product , based on errors or incorrect decisions .

Highe s ll evel of Acc uracy(scientific metrology)

Errors ca u seLowest le vel of Risk tofinal p roduc t or service

Inlernatlonal System of Units (51)Bu reau Inte rnational des Poi ds Mesu res

BIPM)

JPrimary National StandardNalion al Me trology Ins titute

Second a ry StandardNa tional Metrology Ins titute o r Prim a ry

St an da rds La b

Work ing StandardsPrimary Standards Lab or other Labs

Calibrat ion sIn-house or Commercial Cali b ra tion l bs

l

Lowe st level of Ac cura cy(end-item usage)

Errors causeHighest leve of Risk tofina l p roducl or service

Measurement s by End-Item users

Figure : Traceability and level-of-risk flowing high to low for meas u rements.

Measurement Traceability is accomplished through an unbrok en sequence o competentand documented calibrations . nbrok en and comp etent mean that accepted me asurementbest practices were appl ied and all known error sources were taken into account in theca libration process . Traceability relates a measurement result to the corresponding SI

unit, thereby providing the ability to compare measurements (and the related decisions)within and between organi zations or even international borders .

Measurement Uncertainty

Uncertainty in measurements is the second fundamental tenet o metrology.Measurement uncertainty i s the doubt that exists about a measurement's result. Everymeasurement - even the most careful - al ways has a margin o doubt. Evaluating theuncertainty in the mea surement proce ss determines the goodne ss o a measurement.

-4-2010 NCSL Intemational Workshop and Symposium


5/15

Measurement uncertainty is an estimation of the p otential error in a me as urement resultthat i s caused b y v ariabilit y in the equipment, th e processe s, the environment, and otherso urces. Every element within a meas urement proce ss co ntribute s errors to themeasurement result , including characteristics of the item being te sted . Evaluation of themeas urement uncertaint y characterizes what is rea sona ble t o believe about ame as urement result ba se d on knowledge of the mea surement p rocess.

Evaluation of measurement uncertainty can be qualitative or quantitative. Not all tasksrequire the same level of quality , thus the rigor of the uncertainty evaluation should bedetermined by the importance of the measurement result to the function o f the end-item.Regardle ss of the level of rigor , uncertainty ana lys is is necessary to identify and possiblyreduce errors within a measur ement pr ocess . Typical error sources are:

Instrument Accuracy Repeatability Error Reso lution Error Digit a l Sampling Error Co mputation Error Operator Bias Environmental Factors Error

A lack of standardi za tion for qu antified estimation of measurement uncertainty oftencauses disagreements and confusion in trade , sc ientific findings , and legal i ss ues . TheInternational Organization for Standardization (ISO) Guide to the xpression oUncertai nty n Measurement (GUM) [2] i s the international standardized approach toestimating uncertainty. ANSlfNCSL Z540.2-l997 (R2007) , u s Guide to the xpressiono Uncertain ty n Meas ur eme nt (U.S. Guide) [3] , is the U.S. adoption of the ISO GUM .Additiona l guidance on estimating mea surement uncertainty i s available in variousengineering discipline- specific voluntary consensus standards and complimentary

documents . However , for consistentre

sult s, it is imp erative that the quantificati onof

me as urement uncertainty be based on the ISO GUM.

valuating Measurement Results

Under standing how measurement error impact s the functionalit y o f the end-item an swe rsthe que stion , How good doe s the mea surement need t o be ? Ha ving a rea so nableestimate of the size o f the error answers the que stion , How good can the measurementbe made ? Measurement uncertainty i s the best way to evaluate this error. t is anest imat e of the possible range of true values about a measurement result , caused byme as urement errors. This knowled ge of the mea surement uncertainty i s imp ortant fordetermining the usefulne ss of th e meas urement proces s as it applies to a given product or

serv ice. n other words , it provide s a way to determine if a me as urement re sult isadequate .

Figure 3 illustrates three differ ent mea surement process es and the correspondingmea surement uncert ainty for each , as indicated b y the error bars . Each o f the threeprocesses provides a different re sult. When three different me asure ment proc esses areused to mea su re the same item, it i s rea so nabl e to expect three different answers due t othe uncertainties of the proce sses. An example would be the di ffering mea surement

-5-2 1 NCSL Intemalional Workshop and Symposium


6/15

res ults obtained u sing a s tee l m ac hini st rul e, di al ca liper, and mi cro meter t o make thesam e meas urement.

g ure 3 : Thr ee different m ea surem ent indi cations with the nominal valuecontain ed b y each m easur ement process uncertainty rang e

No te that in Figu re 3 , th e nomin al va lu e is co ntained within eac h m easu rem ent s processunce rtaint y range. On e may as k which is co rrect . The a nswe r is all thr ee a re co rrectwithin th e ca pabi li ty of eac h m eas urement p rocess. However , the measurement w ith thelowes t unce rtaint y is a better es timate of the qu antit y of interes t. A better qu es ti on i swhich m eas urement p rocess w ill support the tes t objectives wi th acce ptabl e risk a nd cos t.

The key to eva luatin g a ny meas ureme nt process is an unde rstanding of the relat ions hipbetwee n the meas urement res ult and the fun ctionalit y o f th e e nd-item. In o ther wor dsknowing h ow goo d i s goo d enough. It is not n ecess ary to use a mi crometer i f a s tee lrul e prov ides a dequate res ult s for th e e nd- item .

Verifying Conformance to Tolerances

Demonstrati on of conf ormity i s not always s traight fo rw ard , yet i s cruci al t o manag in g therisks assoc iated with m eas urements. AS9 100 [ I] requires that organi za tions

de mons tr at e confo rm ity o he pr oduct , whic h is ano ther way of sayi ng prove aprodu ct i s in-t oleran ce . Re quir ements for qu ality prod ucts are no t n ew, but now therequ irement t o demonstrate co n fo rmit y mee ts head-o n with th e tec hn ica l challenge ofprov ing it. Thi s sec tion p rov id es the inform ation n eeded to unde rstand and deve lopdec ision rul es requir ed to m anage the ri sks associa ted w ith t o lerance tes tin g.

Ri sks in tolerance te sting

As di scusse d e arli er, dir ect risk i s assoc iated with d ec is ions at the tim e o f m eas urementand indirec t risk i s th e co nsequ ence of the meas urement deci sio n. For ve rifi ca tion oftolerances , direct risk ca n be broken d ow n into two ca tegories:

I . False Accept Ri sk i s the pr obabili ty of an o ut-or-to lerance item or p arameterbe ing unkn owin gly ac cepted b y tes t or ca libr ation.Depe ndin g o n th e c riti ca lit y of the measu rement , thi s type of error ca n lead to lossof mi ss ion or loss o f li fe, reduced end-it em fun ction or ca pac ity, damage dco rporate reput a tion, wa rranty ex penses, s hippin g a nd assoc iated cos ts forreturn ed items loss o f fu ture sa les, puniti ve dama ges, lega l fees etc [4).

2. False Reject Risk i s the probabilit y of an in-t olerance item or p arame ter b eingunknowing ly rejec ted b y tes ting o r ca libration.

6 20 10 NCS L Inte r a t ona l o r s hop a nd S y p osi um


7/15

False rejects in test and calibration processes lead to unnece ssary rew or k andhandling. For test processes , higher rejection rates impl y poorer productioncontrols, thus false rejects also create an excessively pessimistic view of thequality of the end-i tern production process. This view may le ad to more frequentdisassembly and repair o f end items than is necessary [4] .

It can be surmised from the definitions and descriptions that both types of risk can have acost impact. Although false rejects have an immediate cost imp ac t with unnecessaryrework or scrap, it is false accepts that have the potential for larger cost impacts. This isbecau se false accepts are hidden and their impact may not be felt until much later , afterthe end-item is in service.

Both false accept risk and false reject ri sk are functions of the uncertaint y in themeasurement proces s. Jointly , they comprise measurement deci sion risk , which is a ke yelement and metric o f mea surement quality assurance .

Tolerance testing

It cannot be overstated - uncertainty exists in all mea surement s The amount ofuncertainty in the measurement pr ocess and where the measurement result lies withrespect to the tolerance limit determines the probability of an incorrect deci sion . Figure 4illustrates this key problem in proving conformity to a specification. Using the samethree measurement processe s from Figures 3 , Figure 4 adds a tolerance limit o f L whichindicates the acceptable range of measurement results .

L

ANo m in al

- L

Figure : Three different measurement processes are used to verify anominal value with a tolerance of L. Although aU three have the same

indication measurement A has some probability o non-conformance.

In Figure 4, all three mea surement s indicate the same va lue which is of f -nominal. Aportio n of measurement A s uncert ai nty r ange extends be yond the L limit , which means

there is so me probability that the true va lu e estimated by mea surement A is, in reality ,outside of the specified limits .

-7-2010 NCSL Intemational Workshop a n Symposium


8/15

Figure 5 is a rotated view o Figure 4 , illu strating the assumed Gaussian nature o theerror distributions. Again , it is apparent that measurement A has s ome prob ability obeing out-of-tolerance .

No minal

c

- L + LFigure : Rotated view of Figure 4. Error distributions are assumed to e

Gaussian i.e., normal),

Figure 4 and Figure 5 illu strate different measurement proces ses with different sized errordistributions. The error bands o each measurement in Figure 4 represent the 95 .5%confidence level o the Gaussian di stribution shown in Figure 5 . This means there is a95.5% probability the true value o the measurement result lies within the error bands ,which represents two standard deviations 2cr about the mea surement result. For mo stme asurement systems , a 95% confidence level is adequate , although the criticality o theend product may dict ate a higher level of confidence . Lean Six Sigma uses 3cr , which isa 99.7 % confidence level. Figure 6 illustrates the Gaussian di stribution curve with thestandard deviation and corre sponding probabilities.

9 9 .7

No m ina l

95 .5%- - - - - j - - - - - _ ~

oStan dar d Dc ,; al o sigm a)

Figure 6: he Gaussian distribution Le., normal), illustrating the containment probabilities for onc ,two , and three standard deviations.

As a mea surement result approaches a specified tolerance limit , the measurement processuncertainty creates an area o transition sometimes called an ind e te rmin a te zo n e Thesize o the ind e te rminat e zo n e is dependent on the si ze o f the mea surement process

-8-2010 NCSL Intemational Worksh o p and Symp osium


9/15

uncerta in ty distribu tion - the larger t he measuremen t un certain ty, the larger th eind eter minat e zone ASME 8 89 .7.3.1 2001 , Guid elines or ecisio n R ules [5] , di scussesthese tran sitio n zo nes w ith r espect to es tabli shin g dec isio n rul es for co nfo rm ance testing.Accep tance and rejec tion zones can be es tabli shed insid e o r outs ide of the s pec ifiedto lerance dependin g o n th e c rit ica lity of th e meas urement. Us ing the sa me thr eemeas urem ent p rocess as before, Figu re 7 i llu strates the co ncept of th e ind eterm ina tezones with str inge nt and relax ed acce ptance zo nes for meas urement p rocesses 8 and C .The in deter min a te zo ne for m eas urement A s process cove rs the e nti re s pec ifi edto lerance.

No m nal

Indc tcnn ina Ie Zo nes St rin gent Acccp tanccZo ncs

BA A

- LRelaxe d Acce pta nce Zo nes

+ L

Figure : Indetermin a te zon es a re based on the size o th e mea sur em ent proce ss un ce rtainty According to ASM E 889 .7.3.1 2001 , a cceptance zone s can be es tablish ed inside or out sid e of th e

specification limits dep endin g on th e c riticalit y o th e measurement

Basic Measurement Decision RulesPerfo rmin g a rigo rous unce rtaint y ana lys is for eac h m eas urement r esult p rov ides th esures t way to o btain th e best m eas urement d ata poss ibl e. However , it is not alwaysnecessary. For most m eas urement-base d d ec is ions, less rigoro us meas urement processesare more than a dequ ate to o btai n th e requi site mea surement data qu ality . Dec isio n rul escan be deve loped to p rov ide qu ali ty meas urem ent d ata in a cos t effec tive manne r.

This sec tion cove rs a few basic deci sion rul es that can prov ide flex ibilit y to meet m os tmeas ureme nt qu alit y as surance requir ements .

Ratio-Based Deci sion Rules

One of the o ldest an d mos t co mm on d eci sion rul es in to lerance tes tin g invo lves aco mp ariso n of the test toleran ce to the acc uracy of the meas urin g instrum ent or to themeas urement pro cess unce rtaint y. O riginally ca lled acc uracy ratios, these methods we re

rul es o f thumb that we re not a lways di rec tly link ed to an eng ineerin g o r sc ientifi c bas is.In the ea rly part of the Twe ntie th Ce ntury , the most co mm on acc uracy ratio re quir ed themeas urin g i nstrum ent acc uracy to be ten tim es be tter than th e tole rance of the qu an titybei ng meas ured . Thi s was refe rred to as the I0 : I rule. In the late 1940 ' s an d ear ly1950 ' s, Alan Eag le, Frank Gru bbs , and Helen Coo n pioneered wo rk on co nsume r andpro du cer risk an alys is [6, 7] , wh ich was a ri gorous , sta tistica l metho d of contro ll ingmeas urement dec ision risk in man ufac tu ring. 8 uild ing u pon Eag le ' s p ubli shed wo rk ,Jerry Hayes es tabli shed a bas is for linkin g meas urement dec ision risk to acc uracy ratiosfo r appli ca tion in th e Navy ' s ca libr atio n pr ogr am [8]. T he o bjec tive was to pro vide a n

-9 - 10 NCS L Inte ma tiona l rks hop a nd Sy m pos ium


10/15

imp roved dec is ion rul e w ith o ut requi ri ng stat isti ca l calc ul at ions, which we re ve ry tedi ouswi th th e li m ited comp uti ng powe r o f the ea rl y 1950 ' s [9] . Haye s wo rk was t he genes iso f what is now known as th e 4: I tes t acc ur acy ratio (TAR) , a rul e o f thu mb usedexte nsivel y w ithi n the U .S . ca lib ra ti on sys tem.

A lth ough eas il y a ppli ed , there a re two m ajo r pit fa ll s w ith rudim ent ary rul es o f thumb

such asth

e 4 : I TAR .Fi

rst , a o ne s izefi

ts all

app roac h m ay not be a ppro

pri atefo

r all

circum stances. In th e case o fl ife or mi ss ion criti ca l m eas urement s , detail ed ana lysis o fthe m eas ur ement process s hould b e th e rul e . H oweve r, th ere m ay be va lid ra ti onale forusin g a ccur acy ra ti o m eth ods in th e majo rit y of less c ri tica l t es t p rocess s itu ati ons. Theseco nd pitfa ll is a lack o f standardiza tio n o f rules o f thum b th at can lead to varie d res ult sand a fa lse co nfi dence i n measu rement processes . Thi s is co m pou nded whe n p rod ucts areman ufac tur ed , asse mbled , and tes ted in d iffe ren t l oca tions.

Fo r NASA , it is imp era ti ve th at accep tance /rej ection rul es be s tan dardize d acrossprogra ms or proj ec ts to e nsure uni fo rm appli ca ti on, thu s res ult s. There a re two p reva lentacc uracy ra tio rul es th at have bee n u sed within NASA a nd Industry . Both can be a pp li edto two -sid ed , sym m etri ca l, or as ymm etri ca l to lerance limit s.

I . Test Uncertainty atio (TUR) . T he combin ati on o f th e un ce rt ai nti es o f a ll th eelem ent s o f th e meas urem ent p rocess mu st be no grea ter than a give n p erce nt (such as25 %) o f th e ove ra ll to lerance o f the item b eing measu red . Th e co mbin edun ce rt ain ties of th e m eas urem ent process wo ul d re quire a leve l o f confi dence , suc h as95%, w hi ch wo ul d t hen b e ca ll ed the 95 % expand ed un ce r tai n ty o f he mea s ur ementp r ocess.

Ex pan ded un certai nty is de fin ed in the ISO GUM (o r Z540-2) , where k is t hecove rage fac tor a nd u is the co mb ined un ce rt ai nt y o f th e m eas ure men t p rocess . T hefo ll owing is a mathem at ica l ex press ion of th e TU R base d on th e de finiti on fo und inAN S IINCS L Z5 40.3, Req uire ment s for the Calibration o f Meas uri ng a nd TestEq uip men t [ 10]. A lth ough taken from a ca libr ati on sta nd ard , th is de fini tion i srelevan t for all m eas ure ment app li catio ns w here a TU R i s used .

L -LTU R -_ upper lowe r kU =

2 U9

By acco untin g fo r a ll relevan t un ce rt ainti es , and w ith an ade qu ate rat io va lu e th eTU R can prov ide fo r effec ti ve meas urement qu alit y ass ura nce for m os t m eas urem entsce nari os . Meas ure m ent s B an d C i n Figure 4 illu st rate thi s poi nt.

2 . Te st Accurac y atio (T A R). The acc uracy o f the meas urin g inst rum ent w ill be nogrea ter than a g ive n p erce nt (s uch as 10%) o f th e meas ure ment to leran ce .

The TAR uses th e s pec ifie d acc ur acy o f th e meas urin g in st rum ent an d d oes notco nsider un ce rt ai nties d ue to o th er meas ur ement process e rrors . Thu s, ca utio n mus tbe exe rcised w hen u sing the TAR b eca use othe r so urces o f meas urem en t processerror can ofte n b e large r than the acc ur acy o f the m eas uri ng i nst rum ent. A lthough th eTAR h as th is limit a ti on , it ca n st ill b e used effec ti ve ly in low -c ri tica lity a ppli catio ns.

Fo r effec ti ve use in conformance tes tin g, th e T U R and TA R m ust have an appropria tera ti o va lue . Th is is es pec ia ll y t ru e fo r th e TAR beca use it does not acco unt fo r ot her

-10-2 1 NCSL Inte rna tional Wo r shop a nd Symposium


11/15

me a surement proce ss erro r sources, providing only an o ptimi s tic view of theme a surement quality. Th e larger the T URIT AR , th e higher the confidence of correctaccept/reject deci sio ns as the measurement re sult move s cl ose r to the specification limit s.Figure 8 illustr ate s thi s point for a TUR.

Figure 8 uses the sa m e three me a surement proce sses as in pre vio us figures. This time the

indicat ed me a surement s are posit ioned so that the extreme edge of the 95 uncertaintyrange is approximately even with the tolerance limit. To put the figure in pe r specti ve ,using th e Z540.3 TUR definition , the approximate ratios are: me as urement process A =I : I , measur e ment process 8 = 4: I and measurement proc ess C = 10 : I . Figur e 8illustrat es how cl ose to the tolerance L a measured value can be m ade and still h aveadequate confidence the mea surement is in-tolerance .

L

Nomina lA

- L

Figure 8: Three different measurement processes are used to verify a nominalvalue with a tolerance o L. Each measurement is shown at its capability limit

prior to exceeding the tolerance limit.

It i s imp ortant t o no te that mea surement proce ss A , with the I: I TUR, pro v id es th e sameconfidence at n o minal as me a surement proces ses 8 and C , although the y are much clo se rto the tolerance limit s. In a conformance te s t, it would be difficult to d e fend u singm eas urem ent proce ss A if better measurement proce sses were available , s uch as 8 o r C.

Decision rules for Single-Sided Tolerances

For single-sided t o ler ances , the uncertainty , or accuracy , is used to manage th e deci sio nri sk . As the mea surement re sult approaches the tolerance limit , the uncertainty of theproce ss creates an indeterminate zo ne , as di sc us sed above. The decisi on rule adds o rsubtracts th e uncertainty /accuracy to the limit in such a way as to create an ccept ncelimit , a lso known as a guardband. Figure 9 illustrates this concept , using the acce ptancezo ne term s /Tom ASME 889.7.3.1 .

S trin gen t Acccpta n ee Zo ne s Ind etenn inat c Zones

A

Re laxed Acceptance Zo ne s

Figure 9: Single-sided tolerance S L with three measurement processes . Their indet e rmin te zonesare used to create the acceptance zones.

-I2010 NC SL International Works hop and Symposium


12/15

Risk-based decision rules

As demonstrated in Figure 8, a smaller TUR (e .g., I : I ) offers significantly lowerconfidence for off-nominal measurements than a larger TUR (e .g ., 10 :1 . Thisconfidence can be quantified across a tolerance range for specific TURs . Figure 10 doesthis by linking the measurement result to the false accept risk over the tolerance range o

L. Using the T URo

the three measurements , Figure 10 illustrates that smaller ratiosare less capable o demonstrating conformity over the full range o a specification. Asshown in Figure 10, a TUR o I : I measured at the nomina l value has about a 5 % out-oftolerance probability , while a TUR o 10 : I wi ll have a negligible out-of-toleranceprobability for a large percentage o the to lerance range .

~

~

t(L- XJ-ct> X Jp p

P = c t > L u - XJ c t > ~ , XJ - Ip lip

Where ct>0is the standard normal distribution function found in most spreadsheetprograms . For Microsoft Excel , the function is NORMSDISTO.

122010 NCSL Inlemational Workshop and Symposium


13/15

Table I puts Figure 10's graph into tabular form for three in-tolerance confidence levels.A risk-based decision rule establishes a specific confidence level , and establishesacceptance limits for conformance testing. This approach allows for different confidencelevels depending on both the criticality of the item being measured and the capability ofthe measurement process.

Table 1: Percentage of usable tolerance for a desired confidence level.

In tolerancePercent of

TU confidence

leve lTolerance

10 :1 99.7 3a 86.3

4 :1 99 .7 3a 65 .7

1:1 99 .7 3a Not possible

10 :1 95.45 2a 91.5

4:1 95.45 2a 78.9 1:1 95.45 2a Nominal only

10 :1 68.3 la 97.6

4 :1 68 3 ( la 94 .0

1:1 68.3 la 76 .2

Considerations and CautionsAlthough no measurement system can provide 100 assurance of conformity , theprobability of an undetected non-conformance can be greatly reduced by usingappropriately designed measurement decision rules . The underlying foundation for thebest decision rules is the fundamental tenets of metrology. Without traceability andmeasurement uncertainty , assurance of conformity to speci fied requirements is greatlyreduced and acceptance or rejection becomes a gamble .

There are many point s to consider when developing or implementing measurementdecision rules. While the following list reiterates some points already discussed, it is farfrom exhaustive .

I Measurements support decisions - accept , reject, rework, scrap , or even launch aspace vehicle. In conformance testing , if a measurement does not support a decision ,it is unnecessary.

2 Know how good the measurement needs to be. Deci sions based on measurementdata can be routine or life critical. The criticality of the measurement is the same asthe criticality of the decision.

3. Know how good the measurement can be made. All measurements have error s.Measurement uncertainty analysis is the process that identifies and quantifiesmeasurement errors. The TUR differs from the TAR by the inclusion of all pertinent

3-2010 NCSL Int ernational Workshop and Symposium


14/15

measurement proce ss erro rs. Frequen tl y measurement process errors are larger thanthe erro r o the m eas uring instrum ent being used ; therefore , the TAR only provides anoptimistic view o the measurement quality .

4. All specified tol erances are not created equal The quality o specification limit scan complicate mea surement qu a lity assurance application s. Some design centersprovide excessive margin in tolerances , with out proper documentation. This canunnecessarily inflate costs to the program by margin-stacking i those re spon siblefor verifying the tolerance are not aware o the additional margin and apply strictmeasurement decision rules such as a 10 : I TUR. The converse can a lso be true whenthose responsible for conformance testing believe the mea surement reliabilitymargins are contained within the to lerance , when in reality they are not. n the lattercase , without the application o appropriate measurement decision rules , there is noton ly an increased false acce pt risk , but more importantly , there may also be increasedsafety risks i critica l limit s are unintentionally exceeded.

The mor e critica l th e d ec ision th e more critical the data. he mor ecritica l the data the more critica l the measurement.

142010 NCSL Internatio n l WorKshop a nd Symposiu m

N S Refe ren ce Publicati on 1342


15/15

References

I Quality Management Systems - Aerospace - Requirements, AS91 00 , Rev. C, SAEAerospace, 2009

2 Guide to the Expression o Unce rt ai nty in Measurement, Intern ational Organizationfor Standardization (ISO) , 1995

3 American National Standard or Ca libr a ti on - u s Guide to the Expression oUnce rtainty in Measurement, ANSlINCSL Z540-2-1997 (R2007), NCSLInternational , 2002

4 . Metrology - Ca libration and Measurement Processes Guidelines, NASA ReferencePublication 1342 , NASA, 1994

5 Gui delin es for Decision Rules: Co nsideri ng Measurement Unce rtain ty inDetermining Conformance to Specifications, ASME B89 .7.3. 1-2001 , AmericanSociety of Mechanical Engineers, March 2002

6. Eagle, Alan R., A Method for Handling Errors in Te sting and Measuring, Indu strial

Quality Co ntr ol, March 19547. Grubbs, Frank, and Coon, Helen , O n Setting Test Limits Relati ve to Specification

Limits, Industrial Qualit y Co ntrol , March 1954

8 Hayes , Jerry L , Factors Affecting Measurement Reliabi lity, U.S Naval OrdnanceLaboratory , TM No. 63-106, 24 March 1955

9 Mimb s, Scott , Measur ement Decision Risk - Th e Importan ce o Definitions, Proc.NCSL International Worksh op Symposium, St. Paul , August 2007

10 . Req uir ements or the Calibra tion o Measuring and Test Equipment, ANSlINCSLZ540.3-2006, NCSL International , 2006

Estimation and Evaluation o Measurement Decision Risk, NASA-HANDBOOK-8739.19-4 (Working Draft) , February 20 10

dditional related information

I Measurement Uncertainty Analysis: Principl es and Methods, KS C-UG-2809 Rev:Basic , NASA, November 2007

2. Measurement Uncertainty a nd Co n orman ce Tes ting : Risk Analysis , AS MEB89.7.4 .1-2005 (Technical Report) , American Society o f Mechanical Engineers,Februar y 2006

3 Geometrical Produ ct Specifications GPS) - In spec tion by meas ur ement oworkpieces an d m easuri ng eq uipm ent - Part 1: Decisio n rul es or provingconfo rman ce or non conformance wit h specifications, ISO 14253-1 : 199 8 (E) ,International Organization for Standardization (ISO) , 199 8

4 . The Expression o Uncertainty and Co nfidence in Measurement, M3003 , UnitedKingdom Accreditation Service (UKAS) , January 2007

-15-2010 NCSL International Workshop and Symposium

conformance testing measurement decision rules(nasa)(important)

Documents