AS13100 Requirements for Process Failure Mode & Effects Analysis

(PFMEA) & Control Plans

WEBINAR QUESTIONS & ANSWERS

September 14th, 15th & 16th 2021

Dr Ian Riggs, Quality Executive, Rolls-Royce Civil Aerospace September 28th, 2021

2

Webinar Series – “AESQ Process Failure Mode & Effects Analysis (PFMEA) – AS13100 Process FMEA Requirements” presented by Dr. Ian Riggs, Rolls-Royce

Led by Dr Ian Riggs, this series of interactive webinars described the intent of the AESQ AS13100 requirements for Process FMEA and Control Plans and how they link to the effective deployment of Advanced Product Quality Planning (APQP) and a Zero Defect Strategy.

These webinars explained how AS13100 Process FMEA and Control Plans can be developed, maintained and improved using real examples of best practice from across the industry.

The webinars included an overview of Reference Process FMEAs to help improve the quality of the organization’s PFMEAs and to reduce the length of time taken to create them.

The Webinar series videos and presentation material is available at https://aesq.sae-itc.com/defect-prevention. Detailed guidance on all aspects covered in these Webinars is available in AS13100 Reference Manual RM13004 available at https://aesq.sae-itc.com/supplemental-material. Special thanks go to Rudolf Braunrieder (MTU) and Rebecca Lemon (SAE) who supported the delivery of these webinars. Dr Ian Riggs September 29th 2021

3

DAY 1 QUESTIONS: Question 1.1: Is the presentation available for download after the webinar? Yes the videos of each webinar and the presentation material in PowerPoint format is available to download from the AESQ website at https://aesq.sae-itc.com/defect-prevention. Question 1.2: A customer does not have to invoke AS13100 to mandate APQP, if they already invoke AS9145 do they? AS9145 APQP & PPAP is only one element of AS13100. There are many other requirements in AS13100 that are not included in AS9145. Also, AS13100 has some additional APQP / PPAP requirements that are beyond what is included in AS9145. AS13100 also provides more detail on how the defect prevention quality tools need to be conducted e.g. FMEA, Control Plans and MSA, etc. The engine manufacturer invokes AS13100 in order to reduce the number of unique requirements between companies i.e. RR, GE, Safran and P&W. Question 1.3: We have Customer Dictating what goes on our FMEA We recognize that PFMEAs are one of the most powerful of all the defect prevention tools and key to our aim of achieving Zero Defects. However, we see such great variability in how well they are applied across our businesses and supply chain. Therefore, we have defined the key requirements in AS13100 to help standardize how PFMEAs are deployed and to improve their quality of application. Question 1.4: What about a well-established company that has not adopted FMEA? Is it ever too late to start using FMEA? It is never too late to start using FMEA. FMEA is a key defect prevention tool that will provide value at any stage in the product lifecycle although it may be more limited in terms of the business case to make product and process changes as a result of its application. FMEA creates a structured approach to assessing risk and mitigation that any organization will get benefit from. The aerospace industry must become experts at applying this and all of the other defect prevention quality tools (see Chapter C of AS13100). Question 1.5: We are keen to understand how much time we should spend compiling a PFMEA. Can it be characterized by minutes per feature? It can be quite difficult to provide a definitive answer as it will vary company to company and will be influenced by;

• The part complexity (number of process steps, number of characteristics)

• The number of duplicate features

4

• Experience and skills of the cross functional team carrying it out. The more FMEAs that the team do then the more efficient they will become. It usually takes between 3 and 5 to become proficient.

• Use of Reference FMEAs

• Application of Software

• Availability of similar part FMEAs and Control Plans In my experience for PFMEAs with geometric characteristics early FMEAs can take up to 500-man hours but over time this can reduce to less than 100 (when using Reference PFMEAs too). There is mention of this on Webinar 3. Question 1.6: How can we know the application of the product if the design is not sharing information with us as a supplier? So, what shall we do if there is no KC's given by Customer? Our product often goes to stockholders. We don't know the end customer. Without design input the description of the potential effects and the scoring of severity will be limited to internal effects. However it is still valuable by using the manufacturing effects and severity. Also, history from similar parts may provide some insight into the impact of some nonconformance that has escaped to the customer to provide information that the designer would have provided. Question 1.7: Is AS13100 for manufacturers only? How does AS13100 or AS/RM13004 affect Distributors? Is it part of our flow downs to manufacturers? In AS13100 TABLE 1 describes the applicability of each requirement to the supplier classification. You can use this to determine the applicability of these requirements to your organization. Question 1.8: Is it possible for customers to provide over-arching PFMEAs and flow them to sub-tiers, similar to specification flow-down? for example, customer has a weld PFMEA that is flowed to sub-tier who then builds weld PFMEA based on more specific process used The closest thing to what you are describing is the sharing of Reference Process FMEAs by the customer. At Rolls-Royce we have made several our Reference PFMEAs available to our supply chain for them to use as a starting point. The supplier still needs to complete the scoring (S, O and D) and adjust the prevention and detection controls to suit their own processes, but it is a useful way to provide standardization for certain process types. The RM13004 subject matter interest group is looking to see how we could expand on these Rolls-Royce Reference PFMEAs to create AESQ versions that can be shared with our external supply chains, and beyond. Question 1.9: Should we consider pressure, temperatures and vacuum as failure mode or as the cause of failure? These appear to be typical causes of failure rather than failure modes. For it to be a failure mode then you must be able to see it or measure it on the product. As these are process parameters then they are the potential causes of a Failure Mode.

5

Question 1.10: In the example PFMEA only the most severe severity score was used - is this usual as there is the possibility that the more severe outcomes are much less likely to occur? Using the most severe score is standard industry practice across automotive and aerospace and assesses ‘the worst that could happen’. This ensures that we apply our resources to those failure modes that could have the biggest impact. See RM13004, page 77 for more details on calculating RPNs.

Question 1.11: Can Severity x Occurrence" be priority for improvement, why is there no S x O column in the PFMEA template? Severity x Occurrence is a good way to prioritize improvements. Many organizations use the priority grid as discussed in Webinar 2. In the spirit of standardization, we have adopted an industry recognized template as used across automotive (AIAG Blue Books) as well as aerospace. See RM13004, page 79 for more details on using the Risk Priority Grid. Question 1.12: How would you capture Human Factors in the PFMEA? We must be careful with the integration of Human Factors into a product PFMEA. We could easily end up with many of the ‘dirty dozen’ being listed as potential causes without adding much value. Human Factors tend not to be product specific but rather ‘environmental factors’ that need to be managed. In Rolls-Royce we have created a separate non product related Human Factors PFMEA which we have adapted for each process area to consider the main HF risks and mitigation plans. Question 1.13: How does PFMEA scoring or RPN align with Automotive scoring? The starting point for the scoring guidelines was the AIAG ‘Blue Book’ to ensure that the scoring principles aligned with automotive. The main change in RM13004 (and AS13004) was to provide aero engine examples to help score the severity rating.

6

DAY 2 QUESTIONS: Question 2.1: If we make a generic process change that impacts all our parts, is it expected that we update all our PFMEAS? Yes indeed. It is likely that although it is a generic change it may modify the risk profile (RPN) differently for specific part numbers. We expect Process FMEAs to be done for each part number (as introduced with APQP or as agreed with the customer) and therefore the need to update them when anything in the product specification or process design changes is a key requirement. Question 2.2: Do we need to include inspection / final inspection in the PFMEA if this operation does not transform the product? This example was covered in Webinar 2. Inspection process should be included to consider those ‘unintentional’ transformations such as damage, FOD or cleaning (if relevant). We should not include Failure Modes or Potential Causes that state ‘measurement error’ or ‘inspection error’ this will be considered as part of MSA evaluation. See RM13004, page 59 for more details on dealing with measurement & inspection processes. Question 2.3: What about operations /steps that do not create final feature (i.e. conforming prior to welding)? The intent of the PFMEA process would be including all process steps and product characteristics, even those which are not in their final condition i.e. manufacturing tolerances. The consequence of these being non-conforming will still result in disruption and additional costs and therefore there must be a business case to apply defect prevention. However not every customer may insist on this, even though it is good practice. See RM13004, page 62 for more details on dealing with manufacturing tolerances. Question 2.4: Controls of form, (roundness, flatness, etc.) can be invoked by other GD&T requirements noted on the drawing. Couldn't those form errors be considered as potential failure modes? All characteristics must be included in the PFMEA. Question 2.5: There are big concerns that the design-responsible customer will not provide the potential effects and the severity per characteristic / per failure mode regarding flight operations. Design Engineering resources may well be a constraint when it is in the customer organization, that is true. However, we can still create a useful PFMEA with our internal effects and severity ratings. We can also supplement this with our knowledge of customer escapes and what consequences they had.

7

If we complete the PFMEA as much as possible then it may be easier to convince the design team to support if they know that their input can be very focused on the effects and severity columns. Question 2.6: It seems to me that in these examples PFMEA failure modes are connected to DFMEA potential effects. As parts manufacturer with no Design responsibility, our potential effects are limited to "concession, scrap, rework, etc." It is unlikely that you will find these Failure Modes directly in the DFMEA. The DFMEA is evaluating the potential for the design to fail to meet the customer requirements. It will not get into the level of detail for every characteristic Where Design input may be limited see the answer in 2.5. Question 2.7: Dr Ian Riggs can you explain an example of composite part PFMEA, eager to know the failure modes and causes of failure case examples... I do not have any examples that I can share with you in this forum. Question 2.8: How should a PFMEA for an investment casting house look like? Do you have an example? I do not have any examples that I can share with you in this forum. Question 2.9: Occurrence Table: At what number are we talking about a Low Volume / High volume Criteria? Is there a limitation value? There is no defined number to use but we consider this to be used for things like development parts or legacy ‘alien and stranger’ parts where they are not routinely scheduled production. You will need to use your judgement. Question 2.10: Do we need to score the occurrence of every cause or only of the potential failure? RM13004 shows score against every cause? There must be an Occurrence score for every potential cause. Where there is no data to be able to determine the score for a potential cause e.g. spindle wear, then you can use the occurrence score for the Failure Mode until you have the data. See Occurrence Scoring Section in Webinar 2. See RM13004, page 77 for more details on calculating RPNs. Question 2.11: How could I improve my detection system if my inspection methodology is only visual inspection. Since we are a composite manufacturer, we foresee only the visual defects. In fact, the visual inspection rating falls between 7 or 8.? For visual inspection it can be difficult to lower the score without some level of automation e.g., camera systems. Depending on the application this may not be viable. The high score for visual inspection reflects the inherent problem with human visual inspection and the knowledge that even good inspectors will miss things.

8

Question 2.12: Does AQL level / Sample Check affect the detection rating? No as this is not criteria in the Detection Rating Table. However, it assumes that the occurrence score must be low in order to qualify for sample inspection instead of 100% Question 2.13: If we didn't have Key Characteristics from our client, do you have an example how to fix our KPCs? I do not understand the question – sorry. Some parts will have no design or customer KPCs but this will not change the expectations for completing the PFMEA. Question 2.14: If failure mode describes a physical property of the product, then how can we expect a Process Key characteristic as an output of PFMEA. A process key characteristic may be identified where the process parameter has a direct and significant effect on the product characteristic being met, particularly where the process capability is low or marginal e.g., temperature in a heat treatment furnace. Question 2.15: Are slides from yesterday’s session already available? Yes, they can be found at https://aesq.sae-itc.com/defect-prevention. Question 2.16: How to define the RPN value?

Please see Section 5 of the Webinar 2 presentation pack for an explanation of calculating the RPN. See RM13004, page 77 for more details on calculating RPNs. Question 2.17: To which RPN result should actions be taken to improve the result? Please refer to Section 6 of Webinar 2. Question 2.18: If no customer complaints occur in the last 2 years then what is the review frequency of PFMEA? The review of the PFMEA is not based solely on the number or frequency of customer escapes. The review should also include the review of internal non-conformance such as scrap, concessions and rework. It may also be updated with learning from similar products too. There is no set expectation for the frequency of reviews, you will need to use your judgement to determine what is right. Question 2.19: Is the general process performance used for the occurrence and not the occurrence for that specific product? (When you do an FMEA on an existing product) Where available the occurrence scoring should be based on the data from the specific part number i.e., Cpk, frequency of equipment failures, PPM, etc.

9

For new part numbers where there is no or insufficient data then you should use the data from similar parts and processes to help with the scoring until part number specific data is available.

10

DAY 3 QUESTIONS: Question 3.1: At what point should we formally start creating the PFMEA on an NPI program? On concept design, draft drawing or final issue? The PFMEA should be started very early in the NPI program, usually once the initial designs are being created. This timing is important as it is useful to be able to use the PFMEA output as an input to the design process so that potential process capability issues can be addressed by the product design. Question 3.2: We have concerns with the amount of administration we would need to keep the PFMEA updated with every drawing release. The PFMEA administration should be focused only on the impact of the drawing changes. It is important that the PFMEA is aligned to the latest drawing issue. Question 3.3: What do you do if the process flow of the Shell FMEA changes? Do you adapt ALL part number specific PFMEAs? The shell PFMEA will reflect the process flow of the specific part number, so if the process flow changes it should be a simple re-ordering of the elements within the PFMEA. The shell is created for each part number and so is by default ‘part specific’ Question 3.4: Is there any suggestions which software can be used for creating and managing Reference-, SHELL- and Part-number specific-PFMEAs and other APQP documents? Rolls-Royce uses xFMEA for Design FMEAs, Reference PFMEAs, Process FMEAs and Control Plans. There are other packages out there that can do this (possibly except for Reference PFMEAs) and some that mirror APQP and include Process Flows, etc. too. Question 3.5: There’s a basic request to provide an example of an assembly PFMEA. It’s more difficult therefore it would be good to see some additional to the machining process. It seems more difficult compared with a machining process. Assembly PFMEAs are more difficult than a machining process, in particular the definition of the requirements on which the rest of the PFMEA is built needs special care. It can be too easy to fall into the trap of blaming the operator for all assembly failure modes. This must be avoided. There is a simple example included in RM13004 and the RM13004 Subject Matter Interest Group are planning to develop more detailed Reference Process FMEAs for assembly soon. Question 3.6: Would you have a control plan per part number? Yes, the Control Plan includes part number specific specifications and controls that were identified in the Part Specific PFMEA.

11

Question 3.7: Is part specific the main criteria to determine whether a control should be in a Control Plan? Yes, as described in Webinar 3. Some organizations may put additional non part specific controls in there too, but this is not seen as good practice. Question 3.8: Do control plans need to include Inspection operations? The Control Plan will include all measurements and inspections required to validate the part is conforming. These maybe integrated into a make operation or a separate, dedicated inspection operation. Question 3.9: Is it expected to link the PFMEA and Control Plan in each router? The router (if you mean the sequence of operations), PFMEA and Control Plan are all linked. Question 3.10: When for example a fitting cell deals with 10 different engine types. Each of these then has various fitting requirements, riveting, assembly, torque tightening, pressure test, wiring etc. Would these various types of operation all require a control plan too? Yes, as each engine type are likely to have different specification requirements which are a key element of the Control Plan. Question 3.11: Is it intended to combine inspection plans with control plans? Are these different / separate documents? There should be no need to have two different types of documents. The control plan should include everything we need to define the inspection requirements. Therefore, only a Control Plan should be needed although different customers may have specific requirements for this.

12

Webinar Polls A number of polls were taken during the Webinars using Slido. We recognize that not everyone could access them but evenso more than 200 people responded to each.

13

14

15