Objective Read World Objective Read World Uncertainty AnalysisUncertainty Analysis

CMSC 2003 July 21-25 2003

Tim NielsenTim Nielsen Scott SandwithScott Sandwith

IntroductionIntroduction Confidently optimize production processes against their Confidently optimize production processes against their

requirementsrequirements Inputs vs. OutputsInputs vs. Outputs

Need to simulate process performance to optimize accuracy, Need to simulate process performance to optimize accuracy, speed, and costs speed, and costs

Need reliable (easy to understand) uncertainty estimates for Need reliable (easy to understand) uncertainty estimates for complex 3D measurements on the factory floorcomplex 3D measurements on the factory floor

Need to estimate the benefits of combining measurement Need to estimate the benefits of combining measurement systemssystems Common Type Network (n Trackers)Common Type Network (n Trackers) Hybrid Type Network (Scanners + Trackers etc.)Hybrid Type Network (Scanners + Trackers etc.)

Need real-time (easy to understand) feedback on Need real-time (easy to understand) feedback on measurement system performancemeasurement system performance

Need traceable measurement uncertainty for each assemblyNeed traceable measurement uncertainty for each assembly

Process DescriptionProcess Description InputsInputs

Object Characteristics Object Characteristics (e.g., Volume, Surface)(e.g., Volume, Surface)

Expected TolerancesExpected Tolerances Instrument Types and Instrument Types and

Number of StationsNumber of Stations Cycle TimeCycle Time Measurement Constraints Measurement Constraints

(e.g., line-of-sight, targeting (e.g., line-of-sight, targeting the actual critical features)the actual critical features)

OutputsOutputs GUM Compliant GUM Compliant

Uncertainty Estimates of Uncertainty Estimates of Feature MeasurementsFeature Measurements

Measurement Plan Measurement Plan Number of Instruments Number of Instruments

(Stations)(Stations) Types of InstrumentTypes of Instrument Instrument PlacementInstrument Placement Targeting RequirementsTargeting Requirements

Network/Orientation Network/Orientation RequirementsRequirements Transform vs. BundleTransform vs. Bundle Number of Common PtsNumber of Common Pts ClosureClosure

Analysis DependenciesAnalysis Dependencies

Background: UncertaintyBackground: Uncertainty Guide to Uncertainty in Measurement (GUM)Guide to Uncertainty in Measurement (GUM)

ISO way to express uncertainty in measurementISO way to express uncertainty in measurement Error and Uncertainty are not the sameError and Uncertainty are not the same

Quantify components of UncertaintyQuantify components of Uncertainty Type A vs. B depends on the estimation methodType A vs. B depends on the estimation method

A = Statistical Methods (e.g., Monte Carlo, 1A = Statistical Methods (e.g., Monte Carlo, 1stst-order Partials)-order Partials) B = Other means (e.g., measurements, experience, specs)B = Other means (e.g., measurements, experience, specs)

Random vs. Systematic Effects (e.g., Noise vs. Scale)Random vs. Systematic Effects (e.g., Noise vs. Scale) Both are components estimated with Type A or B methodsBoth are components estimated with Type A or B methods

Uncertainty Estimates can contain Type A & B methodsUncertainty Estimates can contain Type A & B methods GUM mandates uncertainty statements in order to provide GUM mandates uncertainty statements in order to provide

traceability for measurement resultstraceability for measurement results A measurement result is complete only when accompanied A measurement result is complete only when accompanied

by a quantitative statement of its uncertainty by a quantitative statement of its uncertainty 11

1 - Taylor and Kuyatt, 1994: NIST TN/1297

Background: UncertaintyBackground: Uncertainty SpecificationsSpecifications

Instrument specifications are not representative of the results from Instrument specifications are not representative of the results from actual use of the instrument in a networkactual use of the instrument in a network

3D Measurement Networks3D Measurement Networks 1 Instrument + References1 Instrument + References 1 Instrument in multiple locations + References1 Instrument in multiple locations + References n Instruments (types) + Referencesn Instruments (types) + References

Application of 3D Measurement SystemsApplication of 3D Measurement Systems Real use Real use multiple stations and different instruments in the same multiple stations and different instruments in the same

networknetwork Quantify coordinate data uncertainty fields in a networkQuantify coordinate data uncertainty fields in a network

Practical methods to estimate the uncertainty of specific systemsPractical methods to estimate the uncertainty of specific systems Combining measurement systems Combining measurement systems Combining measurement uncertaintiesCombining measurement uncertainties Results need to in an easy to understand and meaningful formatResults need to in an easy to understand and meaningful format

Monte CarloMonte Carlo What: Non-linear statistical techniqueWhat: Non-linear statistical technique Why: Difficult problems and expensive to state or Why: Difficult problems and expensive to state or

solvesolve When: Consequences are expensiveWhen: Consequences are expensive How:How:

List of possible conditions (where the activity being studied List of possible conditions (where the activity being studied is to large or complex to be easily stated)is to large or complex to be easily stated)

Random numbers (from estimates of each measured Random numbers (from estimates of each measured component) component)

Model of Network … interactionsModel of Network … interactions Large number of solution are runLarge number of solution are run Statistical inferences are drawnStatistical inferences are drawn

Monte Carlo technique was developed during World War II in Los Alamos for the atom bomb project

ModelsModels ModelingModeling InstrumentsInstruments

AxesAxes AnglesAngles RangingRanging OffsetsOffsets JoinsJoins

MeasurementsMeasurements Angles Angles ppm ppm Ranges Ranges ppm + offset ppm + offset

ConfidenceConfidence

Wing to Body JoinWing to Body JoinApplicationApplication

InputsInputs CAD Model includes Features, CAD Model includes Features,

Relationships, TolerancesRelationships, Tolerances Sweep, Dihedral, IncidenceSweep, Dihedral, Incidence

Scanners, Trackers, Local Scanners, Trackers, Local GPS, Robotics, Gap GPS, Robotics, Gap Measurement DevicesMeasurement Devices

Production Measurement + Production Measurement + Analysis < 3 minutesAnalysis < 3 minutes

Aluminum SurfaceAluminum Surface Targeted and Pre-measured Targeted and Pre-measured

Assembly Interface FeaturesAssembly Interface Features Transfer critical object control Transfer critical object control

to continuously visible featuresto continuously visible features

OutputsOutputs Surface: 0.080” @ 2Surface: 0.080” @ 2 Features: 0.004” @ 2Features: 0.004” @ 2 2 Scanners + Local GPS + 2 Scanners + Local GPS +

GAP Measurement ToolGAP Measurement Tool Optimized Instrument LocationOptimized Instrument Location Bundle Local GPS and Bundle Local GPS and

Transfer to (11) Common PtsTransfer to (11) Common Pts Local GPS updates at 2 HzLocal GPS updates at 2 Hz Aerodynamically matched Aerodynamically matched

orientation within process orientation within process uncertaintyuncertainty

ApplicationApplication

OutputsOutputs

OutputsOutputs

ResultsResults

ResultsResults

ConclusionsConclusions

AcknowledgementsAcknowledgements

John Palmateer (Boeing)John Palmateer (Boeing) Dr. Joe Calkins (New River Kinematics)Dr. Joe Calkins (New River Kinematics)

SummarySummary