DOE ApplicationsDOE Applications

Oct 16th 2002ASQ Section 702: San Gabriel Valley

byDr. Raj Palanna

Objective of Talk

Give a “flavor” for actual applications of DOE in industry

Provide an insight into how complex theoretical concepts are applied in real-world situations

Build on the ASQ 702 September presentation DOE - Basic Concepts by Dr. Kurt Palmer

Agenda

Deep Dive - 2 ExamplesQuick High Level Overview - Few

More ApplicationsQ&A

* These examples are taken from applications at different companies. Some of the data and facts have been “coded” to protect proprietary information.

1. ECS Pack Switch Failure InvestigationStarting February 2002 (until April

2002) a total of 21 switches have been rejected at XXX Aircraft Company’s aircraft test line, for failure to “open” at max. 200 deg. F.

Shipments of multi million dollar aircraft are at risk

Key Planning Aspects

Immediate containment actions takenTeams working at 3 tiers of supply chainEvaluate the “usual suspects”*

Testing setup and process Manufacturing process Application at system level, etc

Different schools of quality uses slightly different methods for this exercise

* This presentation will concentrate only on the DOE aspect of the problem

DOE Problem Statement

Parts that failed at customer passed at supplier test process. Test method is a suspect. A DOE was set up to understand the sensitivity of switches to the test parameters

6 Parts were chosen for DOE - 3 New, 3 Rejected

DOE Pre-planing

A complex DOE THOUGHT MAP was developed Factor Selection Response Selection Measurement Repeatability Studies Noise Impact Confounding Randomization Resource Logistics Management Approvals Observation/ Copious Notes etc

DOE Orthogonal Matrix Building Exercise

You Have Been Tasked to Optimize Gas Mileage on Your Car Team Needs to Test 4 Factors Has Resources for 8 Runs Res. IV Confounded Design in

Acceptable (Don’t worry if you don’t understand this bullet!!)

What is the DOE Matrix?

DOE Matrix

Factor A Factor B Factor C Factor D Response

Run # Test #

Depth of Switch (Plate) Ramp Rate

Oil Type/ Viscosity

(Flow)Mounting Technique

OutputSN 1

OutputSN 2

OutputSN 3

OutputSN 4

OutputSN 5

OutputSN 6

Average "Good"

Average "Rejected'

Average Total

1 1 Submerged 2DC 200 10CST Threaded 188.2 187.9 187.9 191.8 191.6 191.8 188.0 191.7 189.9

7 2Not

Submerged 2DC 200 10CST Set On 190.3 188.5 188.4 213.3 219.4 220.2 189.1 217.6 203.4

6 3 Submerged 10DC 200 10CST Set On 188.8 188.7 188.0 203.2 207.5 209.1 188.5 206.6 197.6

8 4Not

Submerged 10DC 200 10CST Threaded 188.7 188.0 187.3 195.3 200.6 200.5 188.0 198.8 193.4

3 5 Submerged 2DC 210 100CST Set On 192.8 190.6 190.8 215.6 222.0 222.0 191.4 219.9 205.6

5 6Not

Submerged 2DC 210 100CST Threaded 190.9 189.9 191.0 207.8 212.7 212.9 190.6 211.1 200.9

2 7 Submerged 10DC 210 100CST Threaded 189.0 187.9 189.3 204.4 213.7 213.8 188.7 210.6 199.7

4 8Not

Submerged 10DC 210 100CST Set On 199.6 191.6 195.4 253.1 246.4 250.8 195.5 250.1 222.8

24-1ResIV

0 10 20

AC

AB

B

AD

A

C

D

Pareto Chart of the Effects(response is REJECTED, Alpha = .30)

A: PlateB: Ramp RatC: Oil TypeD: Mounting

-20 -10 0 10 20

-1.5

-1.0

-0.5

0.0

0.5

1.0

1.5

Effect

Nor

mal

Sco

re

C

D

Normal Probability Plot of the Effects(response is REJECTED, Alpha = .30)

A: PlateB: Ramp RatC: Oil TypeD: Mounting

DOE Analysis:“Rejected” Switches Average Trigger Temperature

Plate Ramp Rate Oil Type Mounting

Not Submerged

Submerged

2 10 DC 200 10CST

DC 210 100CST

Set On

Threaded

204

209

214

219

224

RE

JEC

TED

Main Effects Plot - Boeing Rejected Switches (F)

Type of TESTING OIL (C) influences test results and Type of MOUNTING (D) influences test results

2 10 DC 200 10C

DC 210 100

Set OnThreaded

190

200

210

190

200

210

190

200

210Plate

Ramp Rate

Oil Type

Mounting

Not Submer

Submerged

2

10

DC 200 10C

DC 210 100

Interaction Plots - Boeing Rejected Switches (F)

DOE Analysis:“New” Switches Average Trigger Temperature

0 1 2 3

B

AD

AC

AB

A

D

C

Pareto Chart of the Effects(response is GOOD, Alpha = .30)

A: PlateB: Ramp RatC: Oil TypeD: Mounting

-3 -2 -1 0 1 2 3

-1.5

-1.0

-0.5

0.0

0.5

1.0

1.5

Effect

Nor

mal

Sco

re

C

Normal Probability Plot of the Effects(response is GOOD, Alpha = .30)

A: PlateB: Ramp RatC: Oil TypeD: Mounting

2 10 DC 200 10C

DC 210 100

Set OnThreaded

188.0

190.5

193.0

188.0

190.5

193.0

188.0

190.5

193.0Plate

Ramp Rate

Oil Type

Mounting

Not Submer

Submerged

2

10

DC 200 10C

DC 210 100

Interaction Plots - 'Good' Switches (F)

Plate Ramp Rate Oil Type Mounting

Not Submerged

Submerged

2 10 DC 200 10CST

DC 210 100CST

Set On

Threaded

188.5

189.3

190.1

190.9

191.7

GO

OD

Main Effects Plot - 'Good' Switches (F)

No Impact of Factors A, B, C and D on “New” Switches

180 185 190 195 200 205

LSL USL

Test Parameters Outside ATP - Trigger Temp Spread 14353

USL

Target

LSL

Mean

Sample N

StDev (ST)

StDev (LT)

Cp

CPU

CPL

Cpk

Cpm

Pp

PPU

PPL

Ppk

PPM < LSL

PPM > USL

PPM Total

PPM < LSL

PPM > USL

PPM Total

PPM < LSL

PPM > USL

PPM Total

200.000

*

180.000

191.037

8

2.81155

3.90992

1.19

1.06

1.31

1.06

*

0.85

0.76

0.94

0.76

0.00

0.00

0.00

43.23

716.94

760.17

2379.21

10945.68

13324.89

Process Data

Potential (ST) Capability

Overall (LT) Capability Observed Performance Expected ST Performance Expected LT Performance

STLT

185 187 189 191 193

LSL USL

Test Parameters Outside ATP - Trigger Temp Spread 14412

USL

Target

LSL

Mean

Sample N

StDev (ST)

StDev (LT)

Cp

CPU

CPL

Cpk

Cpm

Pp

PPU

PPL

Ppk

PPM < LSL

PPM > USL

PPM Total

PPM < LSL

PPM > USL

PPM Total

PPM < LSL

PPM > USL

PPM Total

200.000

*

180.000

189.137

8

1.32979

1.45172

2.51

2.72

2.29

2.29

*

2.30

2.49

2.10

2.10

0.00

0.00

0.00

0.00

0.00

0.00

0.00

0.00

0.00

Process Data

Potential (ST) Capability

Overall (LT) Capability Observed Performance Expected ST Performance Expected LT Performance

STLT

180 185 190 195 200

LSL USL

Test Parameters Outside ATP - Trigger Temp Spread 15235

USL

Target

LSL

Mean

Sample N

StDev (ST)

StDev (LT)

Cp

CPU

CPL

Cpk

Cpm

Pp

PPU

PPL

Ppk

PPM < LSL

PPM > USL

PPM Total

PPM < LSL

PPM > USL

PPM Total

PPM < LSL

PPM > USL

PPM Total

200.000

*

180.000

189.762

8

1.65907

2.74439

2.01

2.06

1.96

1.96

*

1.21

1.24

1.19

1.19

0.00

0.00

0.00

0.00

0.00

0.00

187.37

95.61

282.98

Process Data

Potential (ST) Capability

Overall (LT) Capability Observed Performance Expected ST Performance Expected LT Performance

STLT

150 170 190 210 230 250 270

LSL USL

Test Parameters Outside ATP - Trigger Temp Spread 14954

USL

Target

LSL

Mean

Sample N

StDev (ST)

StDev (LT)

Cp

CPU

CPL

Cpk

Cpm

Pp

PPU

PPL

Ppk

PPM < LSL

PPM > USL

PPM Total

PPM < LSL

PPM > USL

PPM Total

PPM < LSL

PPM > USL

PPM Total

200.000

*

180.000

210.562

8

15.1596

19.6889

0.22

-0.23

0.67

-0.23

*

0.17

-0.18

0.52

-0.18

0.00

750000.00

750000.00

21897.23

757021.74

778918.98

60298.58

704183.21

764481.79

Process Data

Potential (ST) Capability

Overall (LT) Capability Observed Performance Expected ST Performance Expected LT Performance

STLT

150 170 190 210 230 250 270

LSL USL

Test Parameters Outside ATP - Trigger Temp Spread 15007

USL

Target

LSL

Mean

Sample N

StDev (ST)

StDev (LT)

Cp

CPU

CPL

Cpk

Cpm

Pp

PPU

PPL

Ppk

PPM < LSL

PPM > USL

PPM Total

PPM < LSL

PPM > USL

PPM Total

PPM < LSL

PPM > USL

PPM Total

200.000

*

180.000

214.238

8

14.0578

16.9269

0.24

-0.34

0.81

-0.34

*

0.20

-0.28

0.67

-0.28

0.00

875000.00

875000.00

7435.83

844418.92

851854.75

21553.66

799858.81

821412.47

Process Data

Potential (ST) Capability

Overall (LT) Capability Observed Performance Expected ST Performance Expected LT Performance

STLT

150 170 190 210 230 250 270

LSL USL

Test Parameters Outside ATP - Trigger Temp Spread 14935

USL

Target

LSL

Mean

Sample N

StDev (ST)

StDev (LT)

Cp

CPU

CPL

Cpk

Cpm

Pp

PPU

PPL

Ppk

PPM < LSL

PPM > USL

PPM Total

PPM < LSL

PPM > USL

PPM Total

PPM < LSL

PPM > USL

PPM Total

200.000

*

180.000

215.137

8

14.7670

18.1465

0.23

-0.34

0.79

-0.34

*

0.18

-0.28

0.65

-0.28

0.00

875000.00

875000.00

8668.88

847340.05

856008.93

26414.19

797910.71

824324.90

Process Data

Potential (ST) Capability

Overall (LT) Capability Observed Performance Expected ST Performance Expected LT Performance

STLT

Switches from New Batch

Switches from Customer ‘Rejected” Batch

Spread of Results Under Different Testing Conditions

Higher Spread Has been Observed in “Rejected Switches”

ITT ATP Data 641121, Jan 1999 - Mar 2002 (Actuation)

170

175

180

185

190

195

200

205

210

1

134

267

400

533

666

799

932

1065

1198

1331

1464

1597

1730

1863

1996

2129

2262

2395

2528

2661

2794

2927

3060

3193

3326

3459

3592

3725

3858

3991

4124

4257

4390

4523

4656

4789

Test #

Deg

ree

F

High Variation?

Historical Review of Test Data to Evaluate Process

Conclusions

A particular batch of switches were found to be in under-fill condition and switches became “sensitive” to testing parameters/ operating conditions and lost its robustness to environment

Manufacturing process was fixedTest was augmented to prevent this

situation from happening again

2. Aircraft Motor Performance OptimizationProblem statement: shop failures at

test. Brake was releasing under the “brake release voltage specification”

Evaluate the “usual suspects” Testing setup and process Manufacturing process Application at system level, etc

Developed a DOE thought map

DOE Matrix .. MotorInput Factors Output

Average

Run # Kit #Static Brake

CW

Static Brake CCW

Static Brake(in-oz)

6 1 -1 0.00019 -1 0.0008 -1 0.003in -1 No -1 0.003in 18.23 15.7 16.965

4 2 1 0.00043 -1 0.0008 -1 0.003in -1 No 1 0.005in 9.12 9.26 9.19

1 3 -1 0.00014 1 0.0003 -1 0.003in -1 No 1 0.005in 8.35 9.26 8.805

7 4 1 0.00051 1 0.0003 -1 0.003in -1 No -1 0.003in 8.42 8.34 8.38

5 5 -1 0.0002 -1 0.0006 1 0.005in -1 No 1 0.005in 9.06 9.42 9.24

2 6 1 0.00037 -1 0.0006 1 0.005in -1 No -1 0.003in 9.09 8.6 8.845

8 7 -1 0.00016 1 0.0003 1 0.005in -1 No -1 0.003in 7.33 7.32 7.325

3 8 1 0.0004 1 0.0005 1 0.005in -1 No 1 0.005in 9.3 9.18 9.24

14 1-2 -1 0.00019 -1 0.0008 -1 0.003in 1 Yes -1 0.003in 22.03 19.38 20.705

12 2-2 1 0.00043 -1 0.0008 -1 0.003in 1 Yes 1 0.005in 18.65 18.13 18.39

9 3-2 -1 0.00014 1 0.0003 -1 0.003in 1 Yes 1 0.005in 14.17 13.3 13.735

15 4-2 1 0.00051 1 0.0003 -1 0.003in 1 Yes -1 0.003in 9.87 9.72 9.795

13 5-2 -1 0.0002 -1 0.0006 1 0.005in 1 Yes 1 0.005in 17.12 17.21 17.165

10 6-2 1 0.00037 -1 0.0006 1 0.005in 1 Yes -1 0.003in 15.38 14.85 15.115

16 7-2 -1 0.00016 1 0.0003 1 0.005in 1 Yes -1 0.003in 15.54 14.76 15.15

11 8-2 1 0.0004 1 0.0005 1 0.005in 1 Yes 1 0.005in 14.36 13.45 13.905

D=ABCEnd Play

ADisk Flatness

BArmature TIR

CBrake Gap

ERun In

25-1ResIV

DiskFlat ArmTIR Brake Gap Run-In End Play

-1 1 -1 1 -1 1 -1 1 -1 1

10.0

11.2

12.4

13.6

14.8

Sta

tic B

rake

Main Effects for DOE Part 2

Use : • Run-In• High Armature TIR• Flat Disk

-1 1 -1 1 -1 1 -1 1

8

13

18 8

13

18 8

13

18 8

13

18DiskFlat

ArmTIR

Brake Gap

Run-In

End Play

-1

1

-1

1

-1

1

-1

1

Interaction Plot for DOE 2

With Low Disk Flatness Use 3 mm (Low) End-Play.

0 1 2 3 4 5

E

AD

ADE

ACD

CD

DE

BD

AB

C

ABD

AC

A

AE

B

D

Pareto Chart of the Effects(response is Static B, Alpha = .30)

A: DiskFlatB: ArmTIRC: Brake GaD: Run-InE: End Play

-4 -3 -2 -1 0 1 2 3 4 5 6

-1

0

1

Effect

Nor

mal

Sco

re

D

AE

A

B

Normal Probability Plot of the Effects(response is Static B, Alpha = .30)

A: DiskFlatB: ArmTIRC: Brake GaD: Run-InE: End Play

Test for Statistical Significance

Results Are Statistically Significant

2 4 6 8 10 12 14 16

-1

0

1

Observation Order

Res

idua

l

Residuals Versus the Order of the Data(response is Static B)

10 15 20

-1

0

1

Fitted Value

Res

idua

l

Residuals Versus the Fitted Values(response is Static B)

-1 0 1

-1.5

-1.0

-0.5

0.0

0.5

1.0

1.5

Nor

mal

Sco

re

Residual

Normal Probability Plot of the Residuals(response is Static B)

-1.5 -1.0 -0.5 -0.0 0.5 1.0 1.5

0

1

2

3

4

Residual

Fre

que

ncy

Histogram of the Residuals(response is Static B)

Randomness Test

There is No Unusual Observations

Conclusions - Motor DOE

Experiment showed statistical significance.

Settings for Static BrakeBrake Cycling….Introduce Brake Cycling in MOT-

ChuckHigh Armature TIR.???? Do we ask for “bad parts”?Flat Disk….Within 1/10th..Need Process ControlWith Low Disk Flatness Use 3 mm(Low) EndPlay.

(With Flat Disk Set Low EndPlay). Change MOT - Chuck.

This will improve Static Brake Problems.

3. Nozzle Air Flow Optimization DOE

Problem statement: air cycle machine nozzle hole-to-hole variation need to be minimized to improve system performance

Process: multi spindle drillingDOE response: diameter sigma between

different holes in a single nozzleFactors: machining factors

Reamer speed, Reamer feed, Reamer diameter, Stock (after drilling)

4. Vapor Cycle Pack Bearing Failure Investigation

Problem statement: bearing failures of VCS pack on aircraft

Type of DOE: design robustness evaluation

Response: measure of fatigue on the bearing after specified run time

DOE factors: product design variables

5. DOE for Car Door Panel Delimitation Problem

Problem statement: delaminating of vinyl from the foam substrate of an car door interior

Process: door panel molding processResponse: lamination evaluation on a likert

scale + measure of area of delaminatingDOE factors: molding machine factor

Overall temp, Heat upper, Heat lower, Oven temp, Backing type, Density, Vacuum pressure, Vacuum delay

6. Actuator Pin Alignment DOE

Problem statement: aircraft door actuator connector pin alignment out of specification

Type of DOE: combination assembly process evaluation plus design features evaluation

DOE response: distance from nominal in circular domain

DOE factors: assembly technique variables plus part features Twist, Length, Sleeve

Practice Idea:Gas Mileage Optimization DOE Problem statement: pick a response variable

to optimize on the car (gas mileage…) Choose factors that might influence this output

response variable Setup the DOE, run the DOE, analyze DOE Other ideas:

Optimize volume of popcorn produced Etc……. Any suggestions from audience?

DOE Is Not the Easiest Application Tool Around Town….But If You Master DOE, You Will Be a Champion QE!!

Practice, Practice, Practice

Appendix

Reference Books Douglas C. Montgomery: Design and Analysis of

Experiments Ronald D. Moen Et Al.: Improving Quality Through Planned

Experimentation Madhav S. Phadke: Quality Engineering Using Robust

Design

Software Minitab JMP Etc