doe 5.1class notes

1

© TLC, DOE 10104

2

© TLC, DOE 10104

AudienceAudience

611 Whitby Lane Brentwood, CA [email protected]

This course is designed for those individuals directly working on product and process development to characterize, optimize and control product and process performance.

Presentation of course materials is designed for 16 hours of instruction.

Prerequisites:

Engineering Statistics and Data Analysis is recommended

Software:

JMP 5.1

3

© TLC, DOE 10104

Topic Page Number

Section I Introduction to DOE and robust design principles 6Section II Experimental preparation 49Section III Full factorial designs 71Section IV Screening designs 124Section V Taguchi designs (optional) 149 Section VI Custom designs

167Section VII Optimization designs 204Section VIII Mixture designs (optional) 219

DOE Table of ContentsDOE Table of Contents

4

© TLC, DOE 10104

Upon completion of the course the participants will be able to:

1. Apply the principles of robust design

2. Design experiments appropriate for the information of interest

3. Use and apply the structures of orthogonal arrays for industrial problem solving

4. Assure the experimental design is efficient

5. Use regression techniques in order to analyze the results and make process/product improvements

6. Use JMP software to design and analyze experiments

Course ContentCourse Content

5

© TLC, DOE 10104

Section ISection I

Introduction to DOE and robust design principles Experimental preparation Full factorial designs Screening designs Taguchi designs (optional) Custom designs Optimization designs Mixture designs (optional)

6

© TLC, DOE 10104

To improve process and design centering and design margins

Performance optimization

Improve product and process robustness

Establish valid design targets, transfer functions and sensitivity budgets

For new equipment characterization and qualification

Developing new process recipes

Problem solving

Variation reduction and performance enhancement

Yield improvement and defect reduction

When processes and systems are complex

DOE is typically the best approach to achieve breakthroughs in parameter design for new products and processes

General Use of DOEGeneral Use of DOE

7

© TLC, DOE 10104

Ad-Hoc Data Analysis Versus DOEAd-Hoc Data Analysis Versus DOE

Ad-Hoc

Results data from the product and process

Goal was to make all the products the same

Problem is some of the parts are not yielding

Poor range of the Xs

Some correlation of the Xs

Results are often muddy and the signal is weak

Structured Experiments

Results data from the product and process

Goal was to make the product differently to learn what are the effects

Experiment is typically off-line

The range of the Xs is purposefully manipulated

Zero or near zero correlation of the Xs

Results are often clear and the signal is strong

8

© TLC, DOE 10104

Experimental ProcessExperimental Process

1. Define the problem and goals

2. Brainstorm factors, experimental levels and responses

3. Design the experiment using JMP

a) Experimental matrix

b) Sampling plan

c) Error control plan

4. Run experiment, collect data

5. Analyze data, fit model and optimize the response

6. Validate the solution and implement the improvement

9

© TLC, DOE 10104

Factor (X, Input)

A feature in the design or process which influences a resulting quality characteristic. Cost and time of the experiment is largely a function of the number of factors in the study. An X can be nominal or continuous.

Response (Y, Output)

A measurable quality characteristic of the product. A Y can be either nominal or continuous.

Level (Settings for each X)

Parameter settings for the experiment. Typical designs have 2-3 levels during characterization, 4-5 during optimization. The number of categorical levels are as many as are considered useful.

Language of DOELanguage of DOE

Process or Process or ProductProduct

Factor 1

Factor 2

Factor 3

Factor 4

Factor 5

Response 1

Response 2

Response 3

Xs

Ys

Product Responses

Process or Process or ProductProduct

Factor 1

Factor 2

Factor 3

Factor 4

Factor 5

Response 1

Response 2

Response 3

Xs

Ys

Product Responses

10

© TLC, DOE 10104

ConstantFactors

Are held fixed during the experiment

ExperimentalFactors

Are changed during the experiment

CovariateFactors

Factors that cannot be controlled; however may influence the response

All constant factor effects during the experiment are considered to be zero. Single machine, single operator, one material lot etc. (error control).

The goal of the experiment is to characterize all experimental factors. Experimental factors are continuous, categorical or mixture.

Covariate factors must be measurable to be useful. The two types of covariates are insitu and fixed covariates. JMP only refers to fixed covariates during design.

More on FactorsMore on Factors

Effects of blocking factors are removed from the results as not to mix in with the estimation of other factors. (error control)

BlockingFactors

Factors included to control for potential sources of error

11

© TLC, DOE 10104

Custom Design JMP Factor DefinitionCustom Design JMP Factor Definition

Select DOE, Custom Design, Factors, Add Factor to see the factor types in designing an experiment.

Once factors are defined they can be saved by selecting Custom Design, Save Factors.

12

© TLC, DOE 10104

Bigger is Better Smaller is Better

One-sided upper specification limit only

USL

One-sided - lower specification limit only

LSL

Two-sided specifications

LSL USL

Target is Best

Three Types Of ResponsesThree Types Of Responses

Unacceptable Good

Good Unacceptable Unacceptable Good Unacceptable

What are some Company examples of each of these?

0

13

© TLC, DOE 10104

Defining Responses in JMPDefining Responses in JMP

Select Add Response when adding responses to an experiment.

Once responses are defined they can be saved by selecting Custom Design, Save Responses.

Goals are defined:

Maximize (Lower Limit only)

Match Target (Upper and Lower)

Minimize (Upper Limit only)

14

© TLC, DOE 10104

Linear Effects

Only 2 levels are required. Linear effects are about 70% + of all effects.

Interaction Effects

Two or more factors are required. Interactions are found in most systems. Interactions are about 10-20% of all effects.

Quadratic Effects

Three or more levels are required. Non linear effects are common. Curvature effects are about 5-15% of all effects.

Linear, Interaction & Quadratic EffectsLinear, Interaction & Quadratic Effects

15

© TLC, DOE 10104

Factor & Response ExamplesFactor & Response Examples

In Class Exercise: 5 Min

Pre-Experimental DesignExperiment Name:

Date:Experimental Problem, Objectives and Goals: Experimenter(s):

Do the following:

1. Open the file Factor Response Matrix.xls

2. Select a product, process or test condition for characterization or improvement

3. Define the problem for the experiment

4. Determine the objectives of the experiment

5. Determine the goals of the experiment (maximize, minimize or hit some target)

16

© TLC, DOE 10104

Responses:Responses:

Output: 1000 ft/hr or >

Dia.: 2.54” .03”

Cracks: < 10 per hour

Factors:Factors:

Speed: 200-100 rpmSpeed: 200-100 rpm

Temperature: 300-250Temperature: 300-250

Time: 10-5 minutesTime: 10-5 minutes

Pressure: 30-15 psiPressure: 30-15 psi

Manufacture of Extruded Plastic Rod

DOE simulationDOE simulation

17

© TLC, DOE 10104

20 Trials to Solve the Problem20 Trials to Solve the Problem

Run Speed Temperature Time Pressure Output Diameter Cracks123456789

1011121314151617181920

In Class Exercise: 45 Min

Factors Responses

18

© TLC, DOE 10104

Best Guess and Tweak

One Factor at a Time

All Combinations of all Factors

DOE

a structured development strategy for product/process engineering in order to characterize, optimize and control the product with minimal waste. This is accomplished by experimenting with many factors at the same time.

Commonly Used Development MethodsCommonly Used Development Methods

19

© TLC, DOE 10104

System Design: is the selection of the general technology and or approach for the design or process

Parameter Design: is the selection of the targets for the design or process. For the design they are product design targets, for the process they are process parameter targets.

Tolerance Design: is the allowable deviation or limits from the target parameter. Bigger is better, smaller is better or target is best.

System, Parameter & Tolerance DesignSystem, Parameter & Tolerance Design

Dr. Genichi Taguchi: All development/design can be broken down into the following:

20

© TLC, DOE 10104

Customer Requirements

Performance Requirements

System Design (Trade Studies)

Parameter Design (DOE)

Tolerance Design (DOE)

Design & Process Qualified?

(DPPM & Cpk)Yes

No

Control Product & Process (SPC)

Process Capability Data

Characterize, Optimize, & Control

Deliver to Customer

Systematic Product Development and ImprovementSystematic Product Development and Improvement

21

© TLC, DOE 10104

PLAN Activity Time Importance

Statement of Objectives 10 % 70%

Organization of Support Team

Background Research

Selection of the Factors and Levels

DO Design and Run the Experiments 65% 10%

CHECK Modeling, Analysis, and Validation 20% 15%

ACT Optimization 5% 5%

Begin With the End in MindBegin With the End in Mind

Identification of the right factors and right levels are critical for achieving optimal results out of the experimentation process

22

© TLC, DOE 10104

GOAL::

Ensure every design point and process operation is Robust with respect to its intended function.

ROBUST::

Insensitive to product or process variation. A robust product/process continues to provide high quality results even when variation is present.

Robust DesignRobust Design

23

© TLC, DOE 10104

Taguchi Quadratic Loss FunctionTaguchi Quadratic Loss Function

Loss = k(2 + (-T)2)

$ Loss

LSL USL

Taguchi quadratic loss function is a good conceptual model for improving robustness. We need the right targets and right tolerances in order to minimize all losses in cost and performance. Quality is not a step function.

Dollar loss (K) includes:

Yield loss Quality incidents Customer loss Consumer loss Reliability loss Product recalls Customer sat. Market share loss

Example is a target is best loss function. There are also smaller is better and bigger is better functions as well.

24

© TLC, DOE 10104

1. Design in Margin

2. Achieve Target

3. Minimize Variability

4. Characterize and Minimize Noise Effects

5. Design to the Flats

6. Use Parameter Combinations

7. Optimize Designs and Processes

8. Use Interactions to Tune Out Sensitivities

8 Robust Design Principles8 Robust Design Principles

25

© TLC, DOE 10104

Typical design margins are 15% greater than worst case

100100 100 100

100 lb. rated

No Margin

100 Base 100 WC+ 30 Margin 230 lb.

1. Design in Margin1. Design in Margin

System design is the primary focus for margin improvement; material selection, product design, capital equipment, etc.

26

© TLC, DOE 10104

GOAL: = Target

±1.5 Shifts inthe mean are commonduring productionRuns.

Keeping the product on target (not just within specification) is often the quickest and most simple way to make improvements in production yields and minimizes loss. This is particularly true for multi-step operations.

LSL USL

TARGET

LSL USL

TARGET

2. Achieve Target2. Achieve Target

27

© TLC, DOE 10104

Non-Robust RobustProcess Process

Even when changes in the average occur the product is still within customer specifications

3. Minimize Variation3. Minimize Variation

28

© TLC, DOE 10104

Variation Reduction MethodsVariation Reduction Methods

DFM Simplify product designs and follow design rules for a highly manufacturable product

Design of Experiments Improved targets, run conditions & controls

SPC Minimize over and under control errors

Correct Capital Equipment Determine if process is capable, reengineer, retool or capitalize if incapable

Buy better components Reduced assembly defects and improved products

POV/COV/REML studies Locate source of variation and make improvements

Supplier Qualification Allow only qualified parts into the assembly

MSA Eliminate or reduce gauge/tester error from the production system

Standardized work Clean, organized, ESD free, methods of work

29

© TLC, DOE 10104

Noise

Factors in the environment that effect the product characteristics of interest

4. Characterize and Minimize Noise Effects4. Characterize and Minimize Noise Effects

30

© TLC, DOE 10104

Process or Product

Raw

Mat

eria

l

Mac

hine

Ope

rato

r

Hum

idity

ES

D

Tem

perature

Factor 1

Factor 2

Factor 3

Factor 4

Factor 5

Response 1

Response 2

Response 3

Internal Noise (Factors or Covariates)

External Noise (Factors or Covariates)

Particles

4. Characterize and Minimize Noise Effects4. Characterize and Minimize Noise Effects

Xs

Xs

Xs

Ys

Consider using the internal and external noise Xs as factors during your experiments. Then determine how to minimize their influence based on sensitivities.

Design parameters, Materials and Machine Settings

Product Responses

31

© TLC, DOE 10104

Factor ATemperature

Volume

5. Design to the Flats5. Design to the Flats

Must include the quadratic term in the experiment and model in order to estimate curvature

32

© TLC, DOE 10104

Use parameter combinations to reduce variation and place response on-target

Low Setting of A High Setting of A

Off-TargetLow Variation

On-TargetHigh Variation

6. Use Parameter Combinations6. Use Parameter Combinations

33

© TLC, DOE 10104

High Setting of Factor A

Off-TargetLow Variation

On-TargetLow Variation

shift due to Factor B

Design or process characterization must be complete before an engineer can use this method

6. Use Parameter Combinations6. Use Parameter Combinations

34

© TLC, DOE 10104

05

101520253035404550

5 10 15 20 25 30 35 40

Tons Shoveled per Manper Day

21.5 lb. optimum shovel load for any material

Old Way New WayNo. of laborers 500 140Tons per man per day 16 59Earnings per man per day $1.15 $1.88Cost per Ton .07 .03

600 workers & 15 different shovel geometries were used

Shovel Load

7. Optimize7. Optimize

35

© TLC, DOE 10104

0123456789

10

1.0 1.2 1.4 1.6 1.8 2.0 2.2

BondStrength (lbs)

Cure Time (sec.)

Customer Requirement

7. Optimize7. Optimize

How does optimizing improve product robustness?

36

© TLC, DOE 10104

Interaction Profiles

49

34

49

34

Speed

115111.25107.5103.75100

3026.2522.518.7515

Temperature

Th

ickn

ess

Th

ickn

ess

8. Use Interactions to Tune Out Sensitivities8. Use Interactions to Tune Out Sensitivities

Interactions can be used to tune the design or process to the flattest most robust condition.

What speed setting will cause the change in temperature to have little to no affect on thickness?

37

© TLC, DOE 10104

Class III Class IVAffects only the Does not affectaverage the response

Class I Class IIAffects both the Affects the standard average and standard deviation only deviation

Types of Factor InfluenceTypes of Factor Influence

How would these look if displayed as scatter diagrams?

38

© TLC, DOE 10104

Replicates are multiple runs and or measurements at identical test conditions. Experimental replicates are an excellent way to gather information concerning process and product variation.

3 to 5 replicates of the test condition are typically sufficient to characterize the mean and standard deviation.

In order to determine factor class, replicates must be used in the experiment and the Y response is summarized into mean and standard deviation from the replicated experimental raw data for each treatment combination.

Between unit (independent) replicates are much more expensive and highly desirable. Between unit replicates require additional units run at the same settings. (between replicates will add 3-5X the cost)

Within unit (dependent) replicates are inexpensive and very valuable. Within unit replicates are multiple measurements taken on the same unit. Within unit replicates are sometimes referred to as pseudo replicates.

ReplicatesReplicates

39

© TLC, DOE 10104

1. Experimental Design

2. Statistical Tests and Analysis

No amount of statistical manipulation will correct for poorly planned and conducted experimentation

Good designs will result in good interpretation of the results

Experimental Design and Statistical TestsExperimental Design and Statistical Tests

40

© TLC, DOE 10104

Test Validity

Test appropriate for the measure

Tests the condition of interest

Valid test results are considered to be true

Test Repeatability

Same results time-after-time

Can a test be repeatable and not valid?

Test ValidityTest Validity

Patterned silicon substrates are expensive, un-patterned glass substrates are cheap. Which substrate should you select for the experiment?

41

© TLC, DOE 10104

Internal Validity

Results are valid within the test

The effects we see are real

Results are not confounded

DOE is excellent for internal validity

External Validity

Results can be generalized and repeatable

Given similar conditions (materials, machines, etc.) we will see similar results

Threats to ValidityThreats to Validity

42

© TLC, DOE 10104

1. Unknown Events

2. Systematic Change

3. Measurement Error

4. Lost Tests

5. Identification

6. Test Reaction

7. Selection

8. Regression

9. Outliers & Data Entry Errors

Threats to Internal ValidityThreats to Internal Validity

43

© TLC, DOE 10104

1. Unknown Events

An event occurs during the experiment that has an effect. The effect of the event is confused with another factor.

2. Systematic Change

Some systematic change occurs in the system during the experiment. The systematic change is confused with another factor.

3. Measurement Error

Problems with accuracy, linearity, repeatability, reproducibility, and stability of measurement. The measurement errors are mistaken for factor effects.

Threats to Internal ValidityThreats to Internal Validity

44

© TLC, DOE 10104

4. Lost Tests Missing data does not allow for proper analysis of the data and false conclusions are drawn.

5. Identification

Test results are misidentified or not identified at all. The link between results and the test conditions are corrupted. This is the #1 error in large experiments.

6. Test Reaction The parts and or materials change as a direct result of measurement.

45

© TLC, DOE 10104

46

© TLC, DOE 10104

47

© TLC, DOE 10104

48

© TLC, DOE 10104

49

© TLC, DOE 10104

50

© TLC, DOE 10104

51

© TLC, DOE 10104

52

© TLC, DOE 10104

53

© TLC, DOE 10104

54

© TLC, DOE 10104

55

© TLC, DOE 10104

56

© TLC, DOE 10104

57

© TLC, DOE 10104

58

© TLC, DOE 10104

59

© TLC, DOE 10104

60

© TLC, DOE 10104

61

© TLC, DOE 10104

62

© TLC, DOE 10104

63

© TLC, DOE 10104

64

© TLC, DOE 10104

65

© TLC, DOE 10104

66

© TLC, DOE 10104

67

© TLC, DOE 10104

68

© TLC, DOE 10104

69

© TLC, DOE 10104

70

© TLC, DOE 10104

71

© TLC, DOE 10104

72

© TLC, DOE 10104

73

© TLC, DOE 10104

74

© TLC, DOE 10104

75

© TLC, DOE 10104

76

© TLC, DOE 10104

77

© TLC, DOE 10104

78

© TLC, DOE 10104

79

© TLC, DOE 10104

80

© TLC, DOE 10104

81

© TLC, DOE 10104

82

© TLC, DOE 10104

83

© TLC, DOE 10104

84

© TLC, DOE 10104

85

© TLC, DOE 10104

86

© TLC, DOE 10104

87

© TLC, DOE 10104

88

© TLC, DOE 10104

89

© TLC, DOE 10104

90

© TLC, DOE 10104

91

© TLC, DOE 10104

92

© TLC, DOE 10104

93

© TLC, DOE 10104

94

© TLC, DOE 10104

95

© TLC, DOE 10104

96

© TLC, DOE 10104

97

© TLC, DOE 10104

98

© TLC, DOE 10104

99

© TLC, DOE 10104

100

© TLC, DOE 10104

101

© TLC, DOE 10104

102

© TLC, DOE 10104

103

© TLC, DOE 10104

104

© TLC, DOE 10104

105

© TLC, DOE 10104

106

© TLC, DOE 10104

107

© TLC, DOE 10104

108

© TLC, DOE 10104

109

© TLC, DOE 10104

110

© TLC, DOE 10104

111

© TLC, DOE 10104

112

© TLC, DOE 10104

113

© TLC, DOE 10104

114

© TLC, DOE 10104

115

© TLC, DOE 10104

116

© TLC, DOE 10104

117

© TLC, DOE 10104

118

© TLC, DOE 10104

119

© TLC, DOE 10104

120

© TLC, DOE 10104

121

© TLC, DOE 10104

122

© TLC, DOE 10104

123

© TLC, DOE 10104

124

© TLC, DOE 10104

125

© TLC, DOE 10104

126

© TLC, DOE 10104

127

© TLC, DOE 10104

128

© TLC, DOE 10104

129

© TLC, DOE 10104

130

© TLC, DOE 10104

131

© TLC, DOE 10104

132

© TLC, DOE 10104

133

© TLC, DOE 10104

134

© TLC, DOE 10104

135

© TLC, DOE 10104

136

© TLC, DOE 10104

137

© TLC, DOE 10104

138

© TLC, DOE 10104

139

© TLC, DOE 10104

140

© TLC, DOE 10104

141

© TLC, DOE 10104

142

© TLC, DOE 10104

143

© TLC, DOE 10104

144

© TLC, DOE 10104

145

© TLC, DOE 10104

146

© TLC, DOE 10104

147

© TLC, DOE 10104

148

© TLC, DOE 10104

149

© TLC, DOE 10104

150

© TLC, DOE 10104

151

© TLC, DOE 10104

152

© TLC, DOE 10104

153

© TLC, DOE 10104

154

© TLC, DOE 10104

155

© TLC, DOE 10104

156

© TLC, DOE 10104

157

© TLC, DOE 10104

158

© TLC, DOE 10104

159

© TLC, DOE 10104

160

© TLC, DOE 10104

161

© TLC, DOE 10104

162

© TLC, DOE 10104

163

© TLC, DOE 10104

164

© TLC, DOE 10104

165

© TLC, DOE 10104

166

© TLC, DOE 10104

167

© TLC, DOE 10104

168

© TLC, DOE 10104

169

© TLC, DOE 10104

170

© TLC, DOE 10104

171

© TLC, DOE 10104

172

© TLC, DOE 10104

173

© TLC, DOE 10104

174

© TLC, DOE 10104

doe 5.1class notes

Documents

bc abc ab

std dev yield

blade anglesolder

selecting

fractional

target low

factor response

cat alyst