auditing to sufficient depth

Registration Management Committee (RMC)Workshop – July 14-15, 2008“Auditing to Sufficient Depth”

Auditing to Sufficient Depth Auditing to Sufficient Depth

Prepared/Presented by: Ben Tuley SAI Global - Senior ConsultantPrepared/Presented by: Ben Tuley SAI Global - Senior Consultant (214) 274-8646 – [email protected](214) 274-8646 – [email protected] Lloyd (Sonny) Crile – Boeing Supplier Quality Lloyd (Sonny) Crile – Boeing Supplier Quality (316) 304-5294 – [email protected](316) 304-5294 – [email protected]


Auditing to Sufficient DepthAuditing to Sufficient Depth

• Why is this important?Why is this important?

– ANAB auditors issued over 75 NCRs in 20 ANAB auditors issued over 75 NCRs in 20 areas in 2005-2007 for not auditing to areas in 2005-2007 for not auditing to sufficient depthsufficient depth

– Many were repeat NCRsMany were repeat NCRs– Undermines the integrity of aerospace Undermines the integrity of aerospace

auditsaudits– Serious concern within aerospace Serious concern within aerospace

communitycommunity



• 75 NCRs in these 20 areas75 NCRs in these 20 areas



• Why is this important?Why is this important?

– OEMs beginning to conduct validation audits OEMs beginning to conduct validation audits (independent checks of suppliers with AS91XX (independent checks of suppliers with AS91XX certifications)certifications)

– OEMs can (& do) complain directly to ANAB OEMs can (& do) complain directly to ANAB regarding unresolved issues related to CB regarding unresolved issues related to CB conformance to established aerospace conformance to established aerospace requirements, including not auditing to sufficient requirements, including not auditing to sufficient depthdepth

– ANAB will respond ANAB will respond veryvery promptly (i.e. visit CB promptly (i.e. visit CB office within a few days)office within a few days)



• What should auditors do?What should auditors do?

– Understand the requirements to which the audit must be Understand the requirements to which the audit must be conductedconducted

– Research/get training in topics they do not understandResearch/get training in topics they do not understand– Understand the processes and interactions and Understand the processes and interactions and

involvement of all stakeholdersinvolvement of all stakeholders– Understand what objective evidence is required for an Understand what objective evidence is required for an

auditee to show conformance with a requirementauditee to show conformance with a requirement– Accomplish fair, unbiased, detailed audits to the Accomplish fair, unbiased, detailed audits to the

established criteriaestablished criteria– Avoid “soft” auditing or “glazing over” topicsAvoid “soft” auditing or “glazing over” topics



• What will we address in this session?What will we address in this session?

– Document control (including configuration Document control (including configuration management)management)

– Procurement (including purchasing and supplier Procurement (including purchasing and supplier control)control)

– Product acceptance softwareProduct acceptance software



Document Control and Configuration Document Control and Configuration Management RequirementsManagement Requirements

(13 ANAB NCRs in this area in the past three (13 ANAB NCRs in this area in the past three years, primarily in configuration years, primarily in configuration

management)management)


Auditing to Sufficient DepthAuditing to Sufficient DepthDocument ControlDocument Control

4.2.1 General: The quality management 4.2.1 General: The quality management system documentation shall include…system documentation shall include…

f. quality system requirements imposed by f. quality system requirements imposed by the applicable regulatory authoritiesthe applicable regulatory authorities


Auditing to Sufficient DepthAuditing to Sufficient DepthControl of RecordsControl of Records

4.2.4 Control of Records4.2.4 Control of Records

• The documented procedure shall define The documented procedure shall define the method for controlling records that the method for controlling records that are created and/or retained by suppliers.are created and/or retained by suppliers.

• Records shall be available for review by Records shall be available for review by customers and regulatory authorities in customers and regulatory authorities in accordance with contract or regulatory accordance with contract or regulatory requirements.requirements.


Auditing to Sufficient DepthAuditing to Sufficient DepthControl of Documents and RecordsControl of Documents and Records

Questions on Document and Questions on Document and

Record Control?Record Control?


Auditing to Sufficient DepthAuditing to Sufficient DepthConfiguration ManagementConfiguration Management

4.3 Configuration Management:4.3 Configuration Management:

The organization shall establish, document The organization shall establish, document and maintain a configuration and maintain a configuration management process appropriate to the management process appropriate to the product.product.

Note: Guidance on configuration Note: Guidance on configuration management is given in ISO 10007management is given in ISO 10007



What is configuration management?What is configuration management?

Per ISO 10007:Per ISO 10007:

““configuration: interrelated functional and configuration: interrelated functional and physical characteristics of a product defined physical characteristics of a product defined product configuration information”product configuration information”

““configuration management: coordinated configuration management: coordinated activities to direct and control configuration”activities to direct and control configuration”



What is configuration management?What is configuration management?Working level definition:Working level definition:

The cumulative actions an organization takes to The cumulative actions an organization takes to ensure that the exact status of a unit or ensure that the exact status of a unit or component is known at any given time and to component is known at any given time and to ensure that the status is as planned ensure that the status is as planned

Status refers to the makeup of the unit or Status refers to the makeup of the unit or component (i.e. which revision is being built?; component (i.e. which revision is being built?; what pieces or items are in the unit or what pieces or items are in the unit or component?)component?)



4.2.3 Document Control vs. 4.3 4.2.3 Document Control vs. 4.3 Configuration Management:Configuration Management:

Document control typically addresses Document control typically addresses controlling procedures, work instructions, controlling procedures, work instructions, contracts and other such documentscontracts and other such documents

Configuration management typically Configuration management typically addresses controlling a product print or addresses controlling a product print or drawing and items that are used to build drawing and items that are used to build the productthe product



• While we will address configuration While we will address configuration management in some detail, ISO 10007 management in some detail, ISO 10007 should be available at your registrarshould be available at your registrar

• Per AQMS Standards, configuration Per AQMS Standards, configuration management process must be management process must be documented (may change in next rev)documented (may change in next rev)

• Configuration management will move to Configuration management will move to clause 7.1 in next revision clause 7.1 in next revision



Configuration management actions:Configuration management actions:

Technical and organizational activities Technical and organizational activities comprising:comprising:

• configuration identificationconfiguration identification

• configuration control configuration control

• configuration status accountingconfiguration status accounting

• configuration auditingconfiguration auditing

(next rev adds configuration planning)(next rev adds configuration planning)



Configuration PlanningConfiguration Planning– Develop a planDevelop a plan– Describe CM actions, responsibilities and authorities Describe CM actions, responsibilities and authorities

throughout product life cyclethroughout product life cycle

Configuration IdentificationConfiguration Identification– Identify configuration itemsIdentify configuration items– Develop a baselineDevelop a baseline

Configuration ControlConfiguration ControlDefine a change methodologyDefine a change methodology

Define the information storage procedureDefine the information storage procedure



Configuration Status AccountingConfiguration Status Accounting– Gather & record data on a buildGather & record data on a build– Identify deviationsIdentify deviations

Configuration AuditsConfiguration Audits– Functional Configuration AuditFunctional Configuration Audit

• Verifies that a CI meets performance and Verifies that a CI meets performance and functional requirements cited in the configuration functional requirements cited in the configuration

documentationdocumentation – Physical Configuration AuditPhysical Configuration Audit

• Verifies that the as-built unit conforms to product Verifies that the as-built unit conforms to product configuration documentationconfiguration documentation



Some points an auditor should investigate:Some points an auditor should investigate:• Is the process defined/documented? Is the process defined/documented? • Has a CM plan been developed?Has a CM plan been developed?• Is there a methodology to accomplish CSA?Is there a methodology to accomplish CSA?• Are the steps being accomplished?Are the steps being accomplished?• How are the audits being documented?How are the audits being documented?• Are audit findings being acted upon?Are audit findings being acted upon?• Have roles and responsibilities been defined?Have roles and responsibilities been defined?• What happens when CM is not maintained?What happens when CM is not maintained?



Some points an auditor should investigate (con’t):Some points an auditor should investigate (con’t):• How does configuration management ripple throughout How does configuration management ripple throughout

the organization (i.e. purchasing, production, repair)? the organization (i.e. purchasing, production, repair)? • What impact does a change in supplier or source of a What impact does a change in supplier or source of a

material/component have on CM activities?material/component have on CM activities?• Is the CM system comprehensive enough for the Is the CM system comprehensive enough for the

product and/or contract flow down requirements?product and/or contract flow down requirements?• How are drawings controlled, both while in engineering How are drawings controlled, both while in engineering

and after release to the program?and after release to the program?• How do engineering changes interface with the How do engineering changes interface with the

configuration management process?configuration management process?• At what point does the CM function take control of the At what point does the CM function take control of the

configuration from the design function?configuration from the design function?• How does this transfer take place?How does this transfer take place?



Other questions on Other questions on

configuration management?configuration management?


Auditing to Sufficient DepthAuditing to Sufficient DepthProcurementProcurement

Addresses supplier control, purchasing Addresses supplier control, purchasing and verification of purchased productand verification of purchased product

(10 ANAB NCRs in this area (10 ANAB NCRs in this area

in the last three years)in the last three years)


Auditing to Sufficient DepthAuditing to Sufficient DepthProcurement - Supplier ControlProcurement - Supplier Control

7.4.1: The organization shall evaluate 7.4.1: The organization shall evaluate and select suppliers based on their and select suppliers based on their ability to supply product in accordance ability to supply product in accordance with the organization’s requirements. with the organization’s requirements. Criteria for selection, evaluation and re-Criteria for selection, evaluation and re-evaluation shall be established.evaluation shall be established.

Does a supplier having an ISO 9001 or AS9100 Does a supplier having an ISO 9001 or AS9100 certification automatically satisfy this requirement?certification automatically satisfy this requirement?



7.4.1: The organization shall be 7.4.1: The organization shall be responsible for the quality of all products responsible for the quality of all products purchased from suppliers, including purchased from suppliers, including customer-designated sources.customer-designated sources.

• What should an organization do if a customer directed What should an organization do if a customer directed supplier is not performing as needed?supplier is not performing as needed?

• How should an organization proceed if a customer How should an organization proceed if a customer directs use of a supplier that the organization would directs use of a supplier that the organization would not otherwise use?not otherwise use?



7.4.1: The organization shall:7.4.1: The organization shall:

a.a. Maintain a register of approved Maintain a register of approved suppliers that includes the scope suppliers that includes the scope of approvalof approval

How is scope of approval determined?How is scope of approval determined?– An exampleAn example



To identify scope, we divide our To identify scope, we divide our suppliers by commodities.suppliers by commodities.

The commodities we use are:The commodities we use are:



Hardware Hardware

and and

SoftwareSoftware



How can scope of approval be cited?How can scope of approval be cited?

• Specific part numbersSpecific part numbers

• Unique item numbersUnique item numbers

• CommoditiesCommodities

– Let’s look at commoditiesLet’s look at commodities



Commodity ListCommodity ListAC – Adhesives and CoatingsAC – Adhesives and CoatingsCB - Circuit BoardsCB - Circuit BoardsCH – ChemicalsCH – ChemicalsCS – ConsumablesCS – ConsumablesDI – DistributorDI – DistributorML - MetalsML - MetalsMP – Manufactured PartsMP – Manufactured PartsPL – PlasticsPL – PlasticsTP – Turned PartsTP – Turned Parts

Is this list sufficient?Is this list sufficient?



Commodity ListCommodity ListAC – Adhesives and CoatingsAC – Adhesives and Coatings

Adhesives – pressure sensitive adhesive, composite adhesiveAdhesives – pressure sensitive adhesive, composite adhesiveCoatings- metal coatings, optical coatingsCoatings- metal coatings, optical coatings

CH – ChemicalsCH – ChemicalsDye, photo resist, cleaning solventsDye, photo resist, cleaning solvents

CS – ConsumablesCS – ConsumablesWhat does this mean?What does this mean?

DI – DistributorDI – DistributorHow are distributors handled?How are distributors handled?

ML – MetalsML – MetalsRound stock, bar stock, sheet stock, aluminum, steel, gold, platinumRound stock, bar stock, sheet stock, aluminum, steel, gold, platinum

MP – Manufactured PartsMP – Manufactured PartsLathe, extrusion, moldLathe, extrusion, mold

Is this list sufficient?Is this list sufficient?



7.4.1 b7.4.1 bPeriodically review supplier performance; Periodically review supplier performance;

records of these reviews shall be used records of these reviews shall be used as a basis for establishing the level of as a basis for establishing the level of controls to be implementedcontrols to be implemented

How does this apply to customer directed How does this apply to customer directed suppliers?suppliers?

What are some examples of controls used for What are some examples of controls used for suppliers?suppliers?



7.4.1 c7.4.1 cDefine the necessary actions to take when Define the necessary actions to take when

dealing with suppliers that do not meet dealing with suppliers that do not meet requirementsrequirements

How does this apply to customer directed How does this apply to customer directed suppliers?suppliers?

What are some examples of actions that can be What are some examples of actions that can be taken?taken?



7.4.1 d7.4.1 dEnsure where required that both the Ensure where required that both the

organization and all suppliers use organization and all suppliers use customer-approved special process customer-approved special process sourcessources

How can we tell when this is required?How can we tell when this is required?

How can this requirement be met without flowing How can this requirement be met without flowing it down to suppliers?it down to suppliers?



7.4.1 e7.4.1 eEnsure that the function having Ensure that the function having

responsibility for approving supplier responsibility for approving supplier quality systems has the authority to quality systems has the authority to disapprove the use of sourcesdisapprove the use of sources

What must be in place before we can ensure this What must be in place before we can ensure this requirement is met?requirement is met?


Auditing to Sufficient DepthAuditing to Sufficient DepthProcurement - PurchasingProcurement - Purchasing

7.4.27.4.2d. Include positive identification of product, applicable specs, d. Include positive identification of product, applicable specs,

drawings, process requirements, inspection instructions & drawings, process requirements, inspection instructions & other relevant technical dataother relevant technical data

e. Include requirements for design, test, examination and related e. Include requirements for design, test, examination and related instructions for organizational acceptanceinstructions for organizational acceptance

f. Include requirements for test specimens (production method, f. Include requirements for test specimens (production method, number, storage conditions) for design approval, inspection, number, storage conditions) for design approval, inspection, investigation or auditinvestigation or audit

g. Include NCM (nonconforming material) notification g. Include NCM (nonconforming material) notification requirements (supplier notification to organization and requirements (supplier notification to organization and arrangements for organization approval of supplier NMC)arrangements for organization approval of supplier NMC)


Auditing to Sufficient DepthAuditing to Sufficient DepthProcurement - PurchasingProcurement - Purchasing

7.4.27.4.2

h. Include supplier notification to organization of h. Include supplier notification to organization of product or process definition change and obtain product or process definition change and obtain organizational approval if requiredorganizational approval if required

i. Include right of access by organization, customer, i. Include right of access by organization, customer, regulatory authorities to all involved facilities and regulatory authorities to all involved facilities and applicable recordsapplicable records

j. Include supplier requirements flow down to sub-j. Include supplier requirements flow down to sub-tier - applicable requirements, key characteristicstier - applicable requirements, key characteristics


Auditing to Sufficient DepthAuditing to Sufficient DepthProcurement – Verifying Purchased ProductProcurement – Verifying Purchased Product

7.4.3 7.4.3 Verification of purchased product Verification of purchased product • Positive verification must occur before supplied product Positive verification must occur before supplied product

is used - or release under positive recallis used - or release under positive recall• When verification is delegated to the supplier, When verification is delegated to the supplier,

requirements will be defined and a register of delegations requirements will be defined and a register of delegations maintainedmaintained

• Customer access to suppler & organization’s premises Customer access to suppler & organization’s premises when required to verify subcontracted product meets when required to verify subcontracted product meets requirementsrequirements

• Customer verification is not evidence of good supplier Customer verification is not evidence of good supplier control, does not relieve organization’s responsibility for control, does not relieve organization’s responsibility for supplying acceptable product & does not preclude supplying acceptable product & does not preclude customer rejectioncustomer rejection


Auditing to Sufficient DepthAuditing to Sufficient DepthProcurementProcurement

Questions or Comments Questions or Comments

on Procurement?on Procurement?


Auditing to Sufficient DepthAuditing to Sufficient DepthSummarySummary

• What should auditors do to be effective?What should auditors do to be effective?– Understand the requirements to which the audit must be Understand the requirements to which the audit must be

conductedconducted– Research/get training in topics they do not understandResearch/get training in topics they do not understand– Understand the processes and interactions and Understand the processes and interactions and

involvement of all stakeholdersinvolvement of all stakeholders– Understand what objective evidence is required for an Understand what objective evidence is required for an

auditee to show conformance with a requirementauditee to show conformance with a requirement– Accomplish fair, unbiased, detailed audits to the Accomplish fair, unbiased, detailed audits to the

established criteriaestablished criteria– Avoid “soft” auditing or “glazing over” topicsAvoid “soft” auditing or “glazing over” topics– Continue to gather information/data until a conclusion Continue to gather information/data until a conclusion

can be reachedcan be reached– Review their actions and results to ensure effectivenessReview their actions and results to ensure effectiveness


Auditing to Sufficient DepthAuditing to Sufficient DepthSummarySummary

• ExistenceExistence

• AdequacyAdequacy

• ConformanceConformance

• EffectivenessEffectiveness


Auditing to Sufficient DepthAuditing to Sufficient DepthProduct Acceptance SoftwareProduct Acceptance Software

Product Acceptance SoftwareProduct Acceptance Software


CMSC OverviewCMSC Overview

Product Acceptance Software (PAS) Product Acceptance Software (PAS) and Quality System Requirements - and Quality System Requirements - Taking Ownership of One's PAS Taking Ownership of One's PAS


AgendaAgenda

• Product Acceptance Software Defined Product Acceptance Software Defined • Processes AffectedProcesses Affected• PAS Requirements/GuidelinesPAS Requirements/Guidelines• Mathematical ComputationMathematical Computation• Validation MethodsValidation Methods• Validation SummaryValidation Summary• What can you do and whyWhat can you do and why• QuestionsQuestions


Product Acceptance Software DefinedProduct Acceptance Software Defined

• What is Product Acceptance Software What is Product Acceptance Software (PAS)(PAS)– PAS is considered software that performs PAS is considered software that performs

product or tooling acceptance without product or tooling acceptance without subsequent inspectionsubsequent inspection

– Software used to manage, store, or translate Software used to manage, store, or translate the sole-source authority copies of digital type the sole-source authority copies of digital type design, and is able to impact the integrity of design, and is able to impact the integrity of the type design the type design

– PAS may be commercial off the shelf softwarePAS may be commercial off the shelf software


Processes AffectedProcesses Affected

– Common PAS applications includeCommon PAS applications include• CMM softwareCMM software• PCMS applications (Laser Trackers, Laser PCMS applications (Laser Trackers, Laser

Radar, Laser Scanners, Photogrammetry, Radar, Laser Scanners, Photogrammetry, PCMM’s) PCMM’s)

• Optical Laser TemplatesOptical Laser Templates• CAD data translatorsCAD data translators• CAD Analysis SoftwareCAD Analysis Software• Post processorsPost processors


PAS RequirementsPAS Requirements

• FAA AC21-36 (1993) Quality assurance FAA AC21-36 (1993) Quality assurance controls for product acceptance software. controls for product acceptance software. (FAA Req’mnt)(FAA Req’mnt)

• AS9100 Quality management systems – AS9100 Quality management systems – Aerospace requirementsAerospace requirements– Section 7.6Section 7.6

• D6-51991 Quality Assurance Standard for D6-51991 Quality Assurance Standard for Digital Product Definition at Boeing SuppliersDigital Product Definition at Boeing Suppliers– Section3.0Section3.0


PAS GuidelinesPAS Guidelines

• ISO 10012 Requirements for measurement ISO 10012 Requirements for measurement processes and measuring equipment processes and measuring equipment (Guidelines)(Guidelines)– Section 6.2.2Section 6.2.2

• ASME B89.4.10 Methods for performance ASME B89.4.10 Methods for performance evaluation of coordinate measuring system evaluation of coordinate measuring system software (Guidelines)software (Guidelines)

• ARP9005 Aerospace Guidance for Non-ARP9005 Aerospace Guidance for Non-Deliverable SoftwareDeliverable Software



• FAA AC21-36 (1993) Quality assurance FAA AC21-36 (1993) Quality assurance controls for product acceptance software. controls for product acceptance software. (FAA Req’mnt)(FAA Req’mnt)– Provides information and guidance concerning Provides information and guidance concerning

control of softwarecontrol of software– ““This AC addresses only those sections of This AC addresses only those sections of

FAR Part 21, Subparts F,G,X and O where FAR Part 21, Subparts F,G,X and O where information on CAM/CAI/CAT software would information on CAM/CAI/CAT software would be helpful”.be helpful”.



• AS 9100_rev B_Sect 7.6f – AS 9100_rev B_Sect 7.6f –

When used in the monitoring and When used in the monitoring and measurement of specified requirements, measurement of specified requirements, the ability of computer software to satisfy the ability of computer software to satisfy the intended application shall be the intended application shall be confirmedconfirmed. . This shall be undertaken prior to initial use This shall be undertaken prior to initial use and reconfirmed as necessary.and reconfirmed as necessary.

NOTE: See ISO 10012-1 and ISO 10012-2 for NOTE: See ISO 10012-1 and ISO 10012-2 for guidance. guidance. (Replaced by ISO 10012 rev 2003)(Replaced by ISO 10012 rev 2003)



• D6-51991 section 3.1.4 (Boeing)D6-51991 section 3.1.4 (Boeing)– Supplier PAS must be verified prior to product Supplier PAS must be verified prior to product

acceptance use.acceptance use.– The supplier will establish and maintain a The supplier will establish and maintain a

procedure independent of the software procedure independent of the software developer to determine that the software, and developer to determine that the software, and subsequent revisions, subsequent revisions, accomplishes its accomplishes its intended function.intended function.


PAS GuidelinePAS Guideline

• ISO 10012_rev 2003_Sect 6.2.2ISO 10012_rev 2003_Sect 6.2.2

Software used in the measurement processes Software used in the measurement processes and calculations of results shall be and calculations of results shall be documented, identified and controlled to documented, identified and controlled to ensure suitability for continued use. Software, ensure suitability for continued use. Software, and any revisions to it, shall be tested and/or and any revisions to it, shall be tested and/or validated prior to initial use, approved for use, validated prior to initial use, approved for use, and archived. and archived. Testing shall be to the extent Testing shall be to the extent necessary to ensure valid measurement necessary to ensure valid measurement results.results.



• ASME B89.4.10-2000ASME B89.4.10-2000– Methods for Performance Evaluation of Methods for Performance Evaluation of

Coordinate Measuring System SoftwareCoordinate Measuring System Software• This document provides guidelines for evaluating This document provides guidelines for evaluating

the quality of solutions generated by CMS software the quality of solutions generated by CMS software and is concerned with testing the behavior of and is concerned with testing the behavior of algorithm implementation. algorithm implementation.

• This standard allows for either a “Digital Test” or a This standard allows for either a “Digital Test” or a “test work piece“ (artifact) can be used in lieu of the “test work piece“ (artifact) can be used in lieu of the listed tests. listed tests.



• ARP9005 Section 7 Verification & ValidationARP9005 Section 7 Verification & Validation– Verification is the process of evaluating developed Verification is the process of evaluating developed

software to assure meeting input requirements at the software to assure meeting input requirements at the end of a development stageend of a development stage. Verification can occur at . Verification can occur at various times during software development. Verification various times during software development. Verification includes review, analysis, inspection, and test. includes review, analysis, inspection, and test.

– Validation is the confirmation that the software Validation is the confirmation that the software requirements are fulfilled and the requirements are fulfilled and the software works as software works as intended in the target environmentintended in the target environment. Validation . Validation determines that the integrated software operates determines that the integrated software operates correctly, completely, and consistently with correctly, completely, and consistently with specifications and requirements.specifications and requirements.

NOTE: Verification is performed by the developer of the software and NOTE: Verification is performed by the developer of the software and Validation is performed by the purchaser of the softwareValidation is performed by the purchaser of the software


Mathematical ComputationMathematical Computation

What is required?What is required?• The ability of computer software to satisfy The ability of computer software to satisfy

the intended application shall be confirmed.the intended application shall be confirmed.• Testing shall be to the Testing shall be to the extent necessaryextent necessary to to

ensure valid measurement results.ensure valid measurement results.• The supplier will establish and maintain a The supplier will establish and maintain a

procedure independent of the software procedure independent of the software developer to determine that the software, developer to determine that the software, and subsequent revisions, accomplishes and subsequent revisions, accomplishes its its intended functionintended function..


Algorithms TestingAlgorithms Testing• The algorithms are the mathematical The algorithms are the mathematical

calculations that the software uses to produce calculations that the software uses to produce results or controls measurement hardwareresults or controls measurement hardware

• Common algorithms that require testing include:Common algorithms that require testing include:– GD&T functions GD&T functions

– Shapes fitsShapes fits

– Best Fits/Least Squares OrientationBest Fits/Least Squares Orientation

– Temperature compensationTemperature compensation

– CAD translationsCAD translations

– Software that controls measurement hardwareSoftware that controls measurement hardware


Algorithm TestingAlgorithm Testing

• ASME B89.4.10-2000 describes a ASME B89.4.10-2000 describes a methodology for testing PAS algorithms methodology for testing PAS algorithms using 2 sets of point data. One dataset using 2 sets of point data. One dataset represents a geometric shape. When this represents a geometric shape. When this data is analyzed by the PAS, the second data is analyzed by the PAS, the second set of data describes nominal values to set of data describes nominal values to validate the PAS algorithms analyzed the validate the PAS algorithms analyzed the shape correctly.shape correctly.


Example of validation of circle algorithm with point Example of validation of circle algorithm with point dataset pairs obtained from NISTdataset pairs obtained from NIST


• Values match !!Values match !!• This algorithm has been validated.This algorithm has been validated.

Measurement Software analysis results

Known NIST “Fit Dataset”


Validation SummaryValidation Summary

• Once the software has been validated; Once the software has been validated; – Document the revision level testedDocument the revision level tested– Maintain records of validation for each version Maintain records of validation for each version

of software testedof software tested– Validate and document any software revisionsValidate and document any software revisions


HistoryHistory


ExpectationsExpectations

• Comprehensive assessment of Comprehensive assessment of suppliers to AS9100 section 7.6suppliers to AS9100 section 7.6

• Consideration/Review of customers Consideration/Review of customers requirements.requirements.– D6-51991 (BOEING)D6-51991 (BOEING)– SQR004 (Vought)SQR004 (Vought)– MAA1-10009-1 (Spirit)MAA1-10009-1 (Spirit)


ReviewReview

• RequirementsRequirements– FAA AC, AS9100, D6-51991FAA AC, AS9100, D6-51991

• HistoryHistory– SER’s SER’s

• ExpectationsExpectations– AS9100 7.6 AS9100 7.6 – Customer RequirementsCustomer Requirements


FAA ConcernsFAA Concerns

• Configuration control and traceability for Boeing Configuration control and traceability for Boeing Datasets and all derivatives (NC programs, CMM Datasets and all derivatives (NC programs, CMM programs, Screen prints, Inspection plans, programs, Screen prints, Inspection plans, OLT’s etc).OLT’s etc).

• Verification of data translations Verification of data translations • Product Acceptance Software verification (PAS).Product Acceptance Software verification (PAS).• Calibration recordsCalibration records• Internal Audit records for DPD/MBDInternal Audit records for DPD/MBD• Supplier control records (Approval, Supplier control records (Approval,

Maintenance and Flow down)Maintenance and Flow down)• FAI’sFAI’s• Training Records that apply to DPD/MBDTraining Records that apply to DPD/MBD


QUESTIONSQUESTIONS


ContributorsContributors

• Sonny Crile Sonny Crile – DPD PQIT Team Member– DPD PQIT Team Member• Mike Sawyer Mike Sawyer – Tooling/DPD Rep– Tooling/DPD Rep• Rick HarcourtRick Harcourt – DPD PQIT Team Member– DPD PQIT Team Member• Randy Becker – DPD PQIT Team MemberRandy Becker – DPD PQIT Team Member