SIXand

International Journal of

Competitive Advantage

SIGMA

Int. J. Six Sigma and Competitive Advantage, Vol. 7, Nos. 3, June 2012 (Accepted for Publication- Oct. 24, 2011) ____________________________________________________________________________

A Six Sigma Approach for R&D: Measuring Dissolved Oxygen_____________________________________________________________________________

Michael J. JohnsonDepartment of Mechanical EngineeringCase Western Reserve UniversityCleveland, OhioE-mail: mjjohnson2236@gmail.com

Monique T. ClaverieMicroptix Technologies, LLCWilton, MaineE-mail: mclaverie@microptix.com

Joseph E. Johnson*Microptix Technologies, LLCWilton, MaineE-mail: jjohnson@microptix.com; xjoej@juno.com*Corresponding author

Abstract: A Six Sigma approach was applied to an R&D application with the goal of understanding and reducing measurement variability. Typically, Six Sigma is applied to a manufacturing process. Many aspects of processes and R&D applications are similar, e.g., input variables, measurements, and output values. In the current study, variables were identified, a Gauge R&R test and Pareto analysis performed, and correction actions implemented to improve measurement variations. The Gauge R&R study included an analysis of variances (ANOVA) using the Fisher test. The system under study was a new method to quantitate dissolved oxygen (DO) by colorimetry. Specifically, measurements were performed using a newly developed i-LAB® spectrometer. Various statistical and mathematical tools were employed to improve measurement variation, resulting in the standard deviation decreasing from 0.66 to 0.23. Additionally, the Six Sigma approach may be applied to other relevant R&D systems.

Keywords: Six Sigma. R&D, Research and Development, analysis of variances (ANOVA), Gauge R&R, statistical analysis, quality improvement, dissolved oxygen (DO), colorimetry, spectrophotometer, quality, statistics

Reference to this paper should be made as follows: Johnson, Michael J., Claverie, Monique, and Johnson, Joseph E. (2011) ‘A Six Sigma Approach for R&D: Measuring Dissolved Oxygen’, Int. J. Six Sigma and Competitive Advantage, Vol. 7, No. 3, pp. X-XX, June 2012

Biographical notes: Michael J. Johnson is a Mechanical Engineering student at Case Western Reserve University in Cleveland, Ohio. He was an Engineering Intern at Microptix Technologies, LLC in Wilton, Maine for part of 2011. His areas of research at Microptix included engineering, application development, computer programming and statistical analysis.

Monique T. Claverie is an Application Engineer at Microptix Technologies. She received her BS degree in Chemical Engineering from the University of Maine at Orono. Her areas of research include application development, quality control, and new product development. She has several white papers and a patent application.

Joseph E. Johnson is the Vice-President of Technology and Operations at Microptix Technologies. He received his PhD from Clarkson University in Potsdam, New York and a BS degree in Chemistry from Syracuse University in Syracuse, New York. His areas of interest include R&D, operations, Six Sigma, and new product development. He has twenty patents and published eight papers in peer-reviewed journals.

______________________________________________________________________________

1 Introduction

Quality and process improvements have evolved greatly over the last few decades. Motorola and General Electric are two companies that initially implemented methodologies to reduce product defects, and coined the term, “Six Sigma” (Harry and Schroeder, 2000; Pande and Holpp, 2002). At that time, the Six Sigma concept was used to count or quantitate defects. However, prior to that, Edward W. Deming was teaching statistics and applying quality control in the automotive and other fields (Aguayo, 1990). Deming and his disciples were determining quality by correlating manufacturing tolerances (sigmas) to input variables. Quality, as defined by Montgomery, is “the fitness of use” with properties being “inversely proportional to variability” (Montgomery, 2009). Today, it is commonplace in numerous manufacturing processes to have a quality program based on Six Sigma concepts. The advent of enhanced measurement tools coupled with data capture, analysis, and modeling by use of computers has resulted in many automated quality control systems. The new Six Sigma programs are in stark contrast to those of a decade ago when complex, bureaucratic, and expensive programs were implemented by only large companies (Dusharme, 2001). Within the last ten years there has also been an emergence to apply Six Sigma and quality concepts to the service industries (Tennant, 2001; Uzzi, 2004; Guarraia, Carey, Corbett and Neuhaus, 2008). The measurable results are improved service quality, enhanced efficiency, and less wasted time. Thus, productivity and profitability have also improved for service industries, (e.g., insurance, banking, food service and hospitals). A common overlap between the manufacturing and service industries is that both have defined measurables that can be analyzed and related to input variables. In other words, the DMAIC (Design, Measure, Analyze, Improve, and Control) procedure may be applied to industries that center on operations and metrics. An operation may be defined as jobs or tasks consisting of

multiple elements or subtasks. Additionally, the Six Sigma goals are the same- improve quality, and understand input and output variables and the related tolerances.

An R&D (Research & Development) process for NPD (New Product Development) also has defined measurables which can be directly related to input variables. Additionally, the goals for a new product are to understand variability in measurement or function, as well as to optimize quality. Thus, it is conceivable to apply a Six Sigma methodology to an R&D process. However, differences between an R&D process and a manufacturing operation exist. The main difference is that the manufacturing operation has a known optimization point, e.g., in making a one inch nail, the desired length and centering point of the nail length is one inch. For an R&D process, the optimization point may be unknown. To circumvent this notion, multiple “points” can be measured and then the mean may be chosen as the optimization point. Additionally, in a manufacturing process variables may not include multiple machines. For NPD, multiple instruments need to be considered. The conventional DMAIC Procedure (Figure 1) has been modified to account for these and other differences, with the “R&D” DMAIC Procedure in Figure 2.

Figure 1 Conventional DMAIC Procedure

Figure 2 Modified “R&D” DMAIC Procedure

Note that Figures 1 and 2 are similar in many respects, but differ in their respective procedures. The DMAIC outline is followed for new product development, as it is for “conventional” operations and services. However, the measurement step varies from the operational procedure and that for the R&D step. In the operation method the capability of the system to be measured and current status is determined and a target value is known. In an R&D procedure the capability to measure is already known, as is the status, so they are unnecessary. A conventional operation is focused on obtaining a known target value. Using the example above, the center point is 1.00 inches for an operation to produce a one inch nail. For an R&D measurement, the focus is on

Determine what needs to be Measured and tolerancesDetermine what Variables effect measurementsEstablish a Metrics PLAN with Main Variables (ex. Operators, Unit-to-Unit, and Within Units)

Perform Measurements

Segment Measurements by VariableStatistically determine Measurement Variation contribution for each variablePareto Analysis of variable contributors

Determine ways to decrease variations for each VariableImplement means and determine improvement effectConfirm and verify improvements

Maintain changes and metrics programPUBLISH and share lessons for best practices

DEFINE

MEASURE

ANALYZE

IMPROVE

CONTROL

12 STEP Modified DMAIC PROCEDURE

obtaining the least amount of error for an unknown value. As an example, an average Absorption value for many spectrometer units may be calculated at 0.525. The 0.525 value, unknown before, now becomes the center point. Another difference between an operation and an R&D procedure is the variables. The variables used in an operation are CTQ, or those that are critical to quality. CTQ variables are used to adjust to a known value, say 1.00 inch for the nail in the previous example. The CTQ variables may be adjusted resulting in an increase or decrease in nail length. The variables in the R&D procedure are those that only contribute to the measurement error. R&D variables are adjusted to only decrease the measurement error around the average. In other words, adjustment results in a decrease of error, not a change in the center point. Even with these differences, the general scope of an R&D procedure is similar to that of an operational procedure. The goal for an operation is to stay within defined limits or control levels around a center point. An R&D procedure does not initially have defined limits, but creates a center point and then drives out error to reduce the limits by adjusting variables.

2 Case study

The goal of the case study was to optimize measurement values (reduce variability) of colorimetric measurements using a spectrometer (an i-LAB® instrument). Other goals are to understand, control, and optimize input variables. Calibrated dissolved oxygen (DO) colored standards were used in the study (Cat. No. K-7512, CHEMetrics, Calverton, VA). The colored standards have a blue dye that absorbs light at 580 nm. The standards varied in the amount of dye, so that an increase in dye concentration results in an increase in absorption. In other words, the more dye present, then the darker blue the standard. The dye solutions were in sealed ampoules that were well within their expiration date, and their color did not change over time. The standards are normally used as a visual comparison reference to determine dissolved oxygen content. This study used instrumentation to quantitate PPM value by measuring absorption. While the sealed standards contain blue liquid in varying depths of color, actual test vials contain a yellow-green reduced form of indigo carmine dye in an acidic solution that undergoes a colorimetric oxidation reaction to turn blue in the presence of oxygen (Gilbert, Behymer, and Castañeda, 1982). Typical applications for actual dissolved oxygen measurements include water (drinking, sewage, industrial waste and bio-habitats, e.g., lakes, oceans, and rivers), power plants (cooling water and boilers), and fermentation (Biofuels, beer, and liquors). The amount of oxygen has a large effect on a wide-range of measurements, e.g., determining the growth of flora and fauna in water, corrosion rate of metal pipes and cooling towers, and fermentation in ethanol and non-aerobic processes. Thus, it is imperative that the dissolved oxygen measurements be as accurate and reproducible as possible. Errors in accuracy and reproducibility can be due to measurement variance as well as sampling procedure. Thus, a major goal is to reduce measurement variance as much as possible.

The spectrometers used were Visible i-LAB® spectrophotometers, (Model S560, Microptix Technologies, Wilton, ME). Spectrometers are used to quantify the wavelength and intensity of light waves. Visible spectrometers are used to determine the amount of color at one (or more)

specific wavelength. The principle involved is to shine white light, with known intensity, through a material and to determine the Absorbance value. Light transmits through a colored liquid and the amount of absorbed light is determined. The amount of absorbed light (Absorbance) is proportional to the amount of color. In this way, the color of the DO standards may be measured.

Definition StageThe first step was to determine what needed to be measured and the acceptable tolerance level. The depth of color of a colored liquid in an ampoule was measured and reported as Absorbance. The acceptable tolerance or variation in measurement is strongly application dependent, but generally the smaller the better. The tolerance or variability of the dissolved oxygen standards was unknown, needed to be determined, and then improved upon.

The second step was to determine the variables that affect the measurement values. For this step a plan was conceived with an ANOVA (analysis of variation) gauge R&R (reliability and reproducibility) test in mind. In general, an ANOVA gauge study determines the contribution of variables to errors in measurements and compares them to errors of the total system. Several factors may affect a system, including:

• Measuring instruments- variability from one instrument to the other (Unit to Unit Variation) as well as variability within one instrument (Within A Unit Variability);variations may be caused by differences in optics, light sources and sensors for the current study

• Operators- variability in operator ability to clean and place samples, measure and use the instrument

• Environment- variability that may affect the process, instrument and operator measurements;this may include extraneous light and changing temperature

• Test Method- variability in protocol and operation (how the product or service is measured)

• Specifications- comparison of a measurement with a reference standard

• Sample- what is being measured and how does it change with time or placement;the current experiments use standards with a defined holder that did not change over time

In the current study, all factors were considered with the focus on reducing measurement error. The specifications and samples were merged together, with known calibration standards being measured. In that manner, the reference and the samples were the same, thus eliminating these variables. Attempts were made to minimize any environmental differences and any variations they would emerge in operator differences (if a cap wasn’t placed correctly so extraneously light was introduced) or within unit differences (if temperature differences caused light intensity or detection differences). Test methods remained consistent throughout the study, eliminating this

variable as well. So the variables that were examined were the within unit variability, unit to unit variability, and operator measurement.

With these variables in mind, the primary focus for the gauge R&R study was on determining: 1) consistency of readings for a specific sample with an individual spectrometer; 2) variability of readings between spectrometers, given the same sample and operator; and 3) the effect of operator on measurement readings and measurement variances. Normally, point 1) equates to repeatability, or variation in measurements taken by a single person or instrument on the same item and under the same conditions. Also point 3) equates to reproducibility, or variation in readings contributed by different operators measuring the same sample. It is very important to note that this study also includes point 2), which is unit to unit variability. This point expands a conventional gauge R&R study, but is absolutely necessary for R&D and new product development.

Using ANOVA, the contribution of the variables to measurement error was determined. After that, a Pareto analysis was completed to find the largest source of error. Pareto analysis is a statistical technique for decision making. It is used for selecting a limited number of task(s) to do, which would cause the most significant changes to an overall effect. In this case, focus would be on understanding the largest contributor(s) to measurement error. The next step was to determine how to adjust each specific variable to reduce error. Typically the Pareto principle is based on the concept that a relatively small (~ 20%) amount of critical variables contribute to a large (~80%) degree of errors. To extend the Pareto principle, one needs to limit or remove these critical causes to achieve significant quality improvement. In the current study, the Pareto analysis was used to identify the critical contributors that cause errors in measurement. Next, improvements will focus on decreasing the specific variables. And then a new Pareto assessment will be made verifying the contributors were the areas to focus on, and statistically significant improvements were obtained. The improvements should be monitored to ensure the variable changes work.

As mentioned previously, the main controllable variables were determined to be the individual unit, multiple units, and operator. A plan was made involving the main controllable variables and how they could be quantitated. For every standard, three measurements would be performed. There were nine blue standards of varying PPM levels of dissolved oxygen to be measured. Five units would be used in the study. Two highly-trained operators would perform the measurements. Each standard would be measured on each unit, by each operator, in triplicate. So, in total 270 measurements would be made based on three variables.

Measure StageThe next step was performing the actual measurements. The experimental set up was as follows:

The five instruments used were calibrated i-LAB® visible spectrometers each having their own ampoule adapter. The instrument specifications include the ability to measure 400 nm to 700 nm

wavelengths of light, with a resolution of ~0.7 nm. The adapters for the spectrometer were designed to accommodate glass ampoules. The adapter snapped onto the i-LAB unit such that light was focused through the ampoule, to the back side of the adapter, and then reflected through the sample again and to a detector. The light path was the same for every unit.

The ampoules standards were from CHEMetrics (Calverton, VA) catalog number C-7512 (Color Comparator), and were composed of uniform glass with dimensions of 4 inch long by ~1/4 inch outer diameter. The glass ampoule standards contained blue-colored aqueous liquids with differing dye concentrations. The degree of color correlated to the amount of dissolved oxygen (DO) for their ampoules kit K-7512 samples. The sample ampoules are meant to measure DO from 1-12 ppm (parts per million) using an indigo carmine reaction. Only the sealed colored standard ampoules were used in the study.

Two trained operators performed the measurements using the same standard testing procedure and software. The operators measured the standards on the same day, with the same instruments to minimize environmental variations. The measuring procedure consisted of initially calibrating the spectrometer with an ampoule filled with clear, distilled water and also with an ampoule filled with an opaque, black ink. The clear or black ampoule was cleaned, then placed into the adapter, aligned in the holder, and a 3 3/4 inch black plastic cap was placed on the adapter and over the ampoule to ensure the blockage of all ambient light. The calibration procedure was performed to ensure that the spectrometer LEDs and sensor contained the correct optical specifications. After calibration, the operator measured the ampoule filled with clear, distilled water to obtain the background. Next, the standard colored ampoules samples were cleaned, placed, and measured. The background or “blank” was subsequently accounted for during this sample measurement. Even though the spectrometer inherently measures light wave intensities from 400 to 700 nm at 1 nm increments, in this study only the Absorbance at 580 nm was used in accordance with the recommended procedure (Gilbert, Behymer, and Castañeda, 1982). Five visible i-LAB® visible spectrometers (unit numbers 719, 791, 805, 818, and 823) were used in the study. Three absorbance measurements at 580 nm were taken for each standard by each operator on each unit. Two trained operators performed the tests. In this manner, the 5x3x2 study yielded 30 reference points for each part of the study. So, together with the nine blue dissolved oxygen standards, the amount of measurements was two 270 points.

Analysis Stage

The data was analyzed in multiple ways. After the measurements were done in triplicate by each operator on five spectrometers, the average Absorbance (ABS) values were calculated and are shown in Table 1. It is apparent that ABS values increase as the standards increase in PPM of dissolved oxygen. Variation amongst measurements with differing units, variation within units and operators are also apparent, and will be explored.

Table 1: Absorbance Averages for each Unit and Operator as a Function of PPM Standards, with Operator Standard Deviation (Sigma)

Unit Number

PPM

V791

V818

V805

V719

V823

Ope

rato

r 1

(s

igm

a =

0.03

1) 1

0.128

0.163

0.164

0.138

0.107

2

0.259

0.29

0.305

0.281

0.225

3

0.374

0.431

0.408

0.389

0.347

4

0.498

0.551

0.539

0.503

0.47

5

0.603

0.656

0.652

0.633

0.588

Unit Number

PPM

V791

V818

V805

V719

V823

6

0.695

0.771

0.736

0.72

0.68

8

0.874

0.934

0.899

0.9

0.84

10

1.056

1.122

1.1

1.072

1.044

12

1.236

1.298

1.28

1.261

1.225

Ope

rato

r 2

(s

igm

a =

0.03

8) 1

0.103

0.179

0.151

0.116

0.079

2

0.227

0.294

0.303

0.259

0.206

3

0.343

0.418

0.38

0.37

0.326

4

0.46

0.533

0.521

0.489

0.449

5

0.571

0.631

0.635

0.605

0.562

Unit Number

PPM

V791

V818

V805

V719

V823

6

0.672

0.728

0.74

0.713

0.664

8

0.833

0.892

0.892

0.874

0.818

10

1.033

1.116

1.089

1.086

1.013

12

1.211

1.297

1.273

1.282

1.204

The average ABS values vs. the PPM standards for each operator were also plotted in Figs. 3 and 4, respectively. The graphs show linearity between Absorbance and the standard PPM values. The connected lines also show that each unit has variances, and that each unit performed differently for each operator. Error bars are at 2 standard deviations (2 sigma levels).

Figure 3: ABS vs. Colored PPM Standards Measured by Operator 1 (Error Bars at 2 Sigma)

0 2 4 6 8 10 120

0.2

0.4

0.6

0.8

1

1.2

1.4

V791

V818

V805

V719

V823

Series12

DO Standards (ppm)

Abso

rban

ce (5

80 n

m)

Figure 4: ABS vs. Colored PPM Standards Measured by Operator 2 (Error Bars at 2 Sigma)

0 2 4 6 8 10 120

0.2

0.4

0.6

0.8

1

1.2

1.4

V791

V818

V805

V719

V823

Series12

DO Standards (ppm)

Abso

rban

ce (5

80 n

m)

The first step to understanding variations in the Analyze Stage measurements is to identify variables. The second step is to quantify their contribution to variations. A broad way to see variances is to “normalize” the measurements with respect to the average for each PPM standard. Further classification by variables can also ease in seeing their contribution and magnitude.

So, in detail, for each colored PPM standard, the overall average absorbance values were determined. The difference (delta) between the measured absorbance value for each point and the average absorbance value was then calculated and plotted in Figure 5. Figure 5 highlights that

there is variation in all three areas; unit to unit, between operators, and within unit. Two standard deviations for between units was 2σ=0.039, within unit variations had 2σ=0.018, and between operators was 2σ=0.009. Thus, the variation contributions in percentage were: between unit 59%; within units 27%, and between operators 14%. The variables were further segmented and analyzed statistically for the degree of error and its significance.

Figure 5: Differences Between Absorbance and Mean Absorbance for Standards

Unit to Unit Variability

In order to determine if unit to unit variations were statistically significant, an analysis of variation (ANOVA) was done using the f-type distribution. The f-test is used to determine whether the expected values of a variable differ significantly from each other. In the case of comparing multiple units, the ANOVA analysis indicates if the instruments get similar results of if they differ substantially. Inherent in the test is a Fisher (or F) number that is calculated and is defined as the between group sum of squares divided by the within group sum of squares, Equation 1.

Equation 1: Fisher Number Calculation Equation

∑¿

❑

¿¿/(i-1)

F= _______________________________

∑nij

❑

(¿Y > ❑ij−¿

Y >❑ij )❑2

/a(i-1)

From the Fisher number, a probability (p) factor is calculated. The probability is that of observing a result as extreme or unlikely than a result using a null hypothesis that the sample measurements are not alike. The Fisher method is often used to determine the statistical

p. 15 of 23

likelihood, actually unlikelihood, for different measurements coming from the same variable (Fisher, 1925; Miller, Jr., 1998; Nolan and Speed, 2000).

In Equation 1, Y is the measurement average for a specific unit or measurement; i is the number of data values per group; j is the within group number; and a is the number of groups. After calculating the F number, a probability factor (p) was determined. The probability factor, which ranges from 0 to 1 (0-100%), indicates the likelihood of the measurements from differing groups being the same number. An α value or significance level is compared with the p value to determine if the groups (differing units) were significantly the same. For an p value of >0.05, there is a high statistical likelihood that the groups would yield the same measurement number.

Table 2 shows the results of the Fisher calculations for each of the PPM levels tested. For 6 of the 9 samples p < α = 0.05 and for 3 of the 9 samples p < α = 0.01. This shows that the variation between units is statistically significant (reproducibility is compromised). Thus the five units differ substantially in yielding expected results.

Table 2: Results from Unit To Unit AOVA Testing Using Fisher Analysis

1 PPM 2 PPM 3 PPM 4 PPM 5 PPM 6 PPM 8 PPM 10 PPM 12 PPM

Fisher # 3.43 6.10 5.17 2.36 2.37 4.83 2.21 4.15 3.43

p 0.023 0.001 0.004 0.081 0.080 0.005 0.097 0.010 0.023

Since this variable showed a statistically significant variation with Fisher Analysis as well as contributed most (59%) in the 2 sigma standard deviation analysis, it had potential for greatest improvement. Drastic reduction in variation was achieved after much brainstorming and testing. These alterations are outlined in the improvement section.

Within A Unit Variability

The Fisher analysis was also performed for the Within A Unit Variable, with results that were unlike that of the Unit to Unit results. Table 3 shows the probability, p value, was significantly greater than 0.05 for all standards. In fact the p value was >0.90 for each calculation. These values indicate there is a very high probability that each unit will produce values that are alike.

Table 3: Results from Within A Unit ANOVA Testing Using Fisher Analysis

1 PPM 2 PPM 3 PPM 4 PPM 5 PPM 6 PPM 8 PPM 10 PPM 12 PPM

Fisher # 0.10 0.05 0.13 0.14 0.29 0.09 0.16 0.03 0.02

p 0.991 0.998 0.984 0.981 0.914 0.993 0.975 0.999 1.000

Even though the Fisher Analysis did not show the Within A Unit Variation to be statistically significant, avenues for improvement were still explored and eventually achieved. These are again outlined in the improvement section.

p. 16 of 23

Operator Variability

Like the Within a Unit variability results, the analysis of variation (ANOVA) with the Fisher tests showed that significant variation did not appear between operators (Table 4). The ANOVA study showed that the p number for the Operator variables was >0.59, significantly larger than 0.05. This probability indicates that the measurements made by one operator are the same as those made by the other. So, the variation in measurements between operators is not statistically significant.

Table 4: Results from Operator Variability ANOVA Testing Using Fisher Analysis

1 PPM 2 PPM 3 PPM 4 PPM 5 PPM 6 PPM 8 PPM 10 PPM 12 PPM

Fisher # 0.11 0.08 0.24 0.18 0.30 0.13 0.30 0.05 0.02

p 0.743 0.779 0.628 0.675 0.588 0.721 0.588 0.825 0.889

The error in measurements attributed to operator differences can be shown schematically. In Figure 6, the center green bar shows the average difference between operators for all measurements with respect to the PPM numbers. The red and blue bars show the average delta and the standard deviations for operator 1 and operator 2, respectively. These bars show +/- one standard deviation, which illustrates that there is a good deal of measurement overlap. From this analysis and a Fischer analysis, it was determined that operator error did not play a significant role, relative to the other two variables, in contributing to the overall error. For this reason, improvement strategies were not explored for this variable.

p. 17 of 23

Figure 6: Measurement Errors for Operator 1 and Operator 2

-0.40

-0.30

-0.20

-0.10

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0.06

0.24

Ave - Std Dev-0.07

Operator 1 Ave0.24

Ave + Std Dev0.55

Ave- Std Dev-0.31

Operator 2 Ave0.06

Ave + Std Dev0.44

Measurement Error

AVER

AGE

DELT

A (P

PM)

Summary of Variables

A Pareto chart shows the relative contribution to error as evidenced by plotting two standard deviations for each variable (Figure 7). The graph shows the majority of error in measurements can be attributed to differences between unit to unit variations. Again, 59% of error results from unit to unit variation, and combined with within unit variation both account for 86% of the error. These results are not surprising and were shown before in Figure 5.

p. 18 of 23

Figure 7: Pareto Analysis of the Main Variables for the DO Standard Study

Unit to Unit Within Units Operator0

0.005

0.01

0.015

0.02

0.025

0.03

0.035

0.04

0.045

0%

20%

40%

60%

80%

100%2

Sigm

a in

Mea

sure

men

t Err

or

Perc

ent C

ontr

ibuti

on to

Err

or

Improvement Stage

Each variable was analyzed for ways to improve measurements. In other words, determine what can be done to decrease variation. Typically, improvements for one variable should not affect others. The ways may not be, and usually are not the same for each variable. However, any improvement in one area should also be assessed in other areas, especially if variables are dependent on each other. The three variables used in the study were independent of each other.

With that in mind, each improvement was analyzed for the specific variable.

Unit to Unit Variability

It became apparent during analysis that Unit to Unit Variation was the largest contributor to variation. It was determined that individual calibration of each unit could reduce the unit to unit variation. To accomplish this, the data from each unit was looked at individually. The absorption data of the standard ampoules was graphed to create Absorption vs. PPM graphs, complete with regression lines for each unit. The average slope and intercept of the regression lines was then used as the model. An automatic correction factor was programmed into the software of each unit to normalize the data in accordance with the individualized regression line. In a sense, each unit was “re-calibrated” in alignment with the standard ampoules. Each unit had slightly different

p. 19 of 23

slopes and intercept lines for the ABS vs. PPM standards due to differences in the light sources (i.e., the LEDs had some variation), the sensors, or the optical path. The newly improved software program corrected for any physical variances. New ANOVA tests showed no statistically significant variations; meaning the units gave statistically similar readings. Further testing of the standard ampoules resulted in much more uniform measurements between units, decreasing the overall standard deviation significantly. Figure 8 shows a graph like Figure 5, but with the new programming improvements for unit to unit variation.

Figure 8: Differences between Absorbance and Mean Absorbance for Standards Operator 1 Operator 2

+1 ϭ

-1 ϭ

+2 ϭ

-2 ϭ

Unit

V791

V818

V805

V719

V823

-0.08

-0.06

-0.04

-0.02

0

0.02

0.04

0.06

0.08

0 10 20 30 40 50 60 70 80 90 100

Within Unit Variability

Despite this variation contributing less than the Unit to Unit Variation, avenues to improve upon it were still explored. Focus on improvement centered on the software correlation used to calculate a PPM measurement from the Absorption data. Initial software used a linear regression line to correlate the absorption to PPM level, as seen in Equation 2.

Equation 2: Linear Correlation Equation

PPM = (9.8912*ABS) – 0.7003

Data generated with this equation resulted in an average delta standard deviation of 0.35PPM, with a 0.995 coefficient of determination (R2) number for a linear regression. The high

p. 20 of 23

correlation number shows that there is a strong relationship between the color of the PPM standards and the Absorbance values, especially the average value. However, improvements can be made by using a quadratic curve fit program. Equation 3 illustrates a quadratic regression line for one of the units. Figure 9 shows an even stronger correlation generated by using a quadratic regression line rather than a linear regression line.

Equation 3: Quadratic Correlation Equation

PPM = (1.894*ABS2) + (7.186*ABS)

Data generated with the quadratic regression line resulted in an average standard deviation of 0.22 PPM, thereby improving precision and going from an R2 of 0.995 to 0.9991.

p. 21 of 23

Figure 9: PPM vs. Absorption Data with both Linear vs. Quadratic Fitting

0 0.2 0.4 0.6 0.8 1 1.2 1.40

2

4

6

8

10

12

14

R² = 0.999122958336071R² = 0.995018782696659

Absorption

PPM

Operator Variability

No improvements were performed for this variable as it was determined in the analysis stage that average delta standard deviations for each individual operator had significant overlap (as seen in Figure 6) and operator differences were not a large contributor to variation. Further improvements may be made by having a more uniform procedure or having operators watch each other perform measurements to determine any differences in sample placement or reading.

Summary of Variables After Improvements

The next step in the improvement process is to implement means of improvement and determine their effect. With that in mind, figure 10 shows the standard deviation before and after improvements for between units and within unit variables. The standard deviation for between units went from 0.039 to 0.003, while that within unit decreased from 0.018 to 0.011. A Pareto chart (Figure 11) shows the new relative contribution to error for each variable. Note that the 2 sigma value decreased from 0.066 to 0.23 in total. The largest variable after the improvements is now within unit variation. Both charts show a large decrease in measurement error. Re-

p. 22 of 23

measuring the units showed the same significant improvements (as was previously seen the differences between Figures 5 and 8).

Figure 10: Impact of Improvements on 2 Standard Deviations

Before Improvement

(Red)

After Improvement

(Blue)0

0.01

0.02

0.03

0.04

Unit to Unit Within Units

2 Si

gma

Mea

sure

men

t Err

or

Figure 11: Pareto Analysis of the Main Variables after Improvements for the DO Standard Study

p. 23 of 23

Unit to Unit Within Units Operator0

0.002

0.004

0.006

0.008

0.01

0.012

0%

20%

40%

60%

80%

100%

2 Si

gma

in M

easu

rem

ent E

rror

Perc

ent C

ontr

ibuti

on to

Err

or

Control Stage

The next step is to maintain changes. Since the changes are primarily in software, they are not expected to change. The units will be periodically checked with the standards just to make sure that there are no added variances. It has become apparent that the methods used to achieve improvement in this study can be applied to also decrease variability in several other applications using the i-LAB® instrument. Learning from this study will be used to improve i-LAB® measurement variability for many applications. The last step is to publish and share the information.

3 Concluding Remarks

The study revealed that a Six Sigma methodology can be successfully applied to new product development (NPD). The modified DMAIC procedure incorporating a Gauge R&R study, a quantitative analysis of variations using Fisher testing, and a Pareto diagram resulted in an exceptional system understanding. Through the Gauge R&R study and ensuing analysis of variance (ANOVA) the magnitude of variance between the three main variables was determined. From the metrics, two variables were specifically improved. In regards to improvements, the measurement error at a 2ϭ level decreased by factor more than three, resulting in PPM variability going from 0.7 to 0.2. Although at 1 PPM that means a 20% error, another dissolved oxygen test kit, having greater sensitivity and ranging from 0 to 1 PPM, may be used. Thus the error ranges from a more viable 10% to 1.7% from 2 PPM to 12 PPM, respectively. A future focus will be to further

p. 24 of 23

reduce reading error from both the within unit variable and by different operators. In summary, the objective of implementing a Six Sigma approach to new product development is realistic. The approach is useful to understand, quantify and improve variables for NPD.

p. 25 of 23

References

Aguayo, R. (1990) Dr. Deming: The American Who Taught the Japanese about Quality, Fireside Press, New York.

Dusharme, D. ( 2001) ‘Six Sigma Survey: Breaking Through The Six Sigma Hype’, Quality Digest, Vol. 21, #11, pp. 27-34.

Fisher, R.A. (1925) Statistical Methods for ResearchWorkers, Oliver & Boyd, London available on-line at ‘Classics in the History of Psychology’ by Christopher D. Green, http://psy.ed.asu.edu/~classics/Fisher/Methods/

Gilbert, T. W., Behymer, T. D., Castañeda, H. B. (1982) ‘Determination of Dissolved Oxygen in Natural and Wastewaters’, American Laboratory, pp. 119-134.

Guarraia, P, Carey, G., Corbett, A. and Neuhaus, K. (2008) ‘Lean Six Sigma for the Services Industry’, Bain Brief, May 20, 2008 Bain &Co., Inc.

Harry, M. and Schroeder, R. (2000) Six Sigma: The Breakthrough Management Strategy Revolutionizing the World’s Top Corporations, Doubleday, New York.

Miller, Jr., R. G. (1998) Beyond ANOVA Basics of Applied Statistics, 1st Reprint, CRC Press, Boca Raton, FL.

Montgomery, D. C. (2009) Introduction to Statistical Quality Control, 6th ed., Wiley, New York.

Pande, P. and Holpp, L.(2002) What is Six Sigma?, McGraw Hill, New York.

Nolan, D. and Speed, T. (2000) ‘Can She Taste the Difference’, Ch. 5, Stat Labs: Mathematical Statistics through Applications, Springer-Verlag, New York.

Tennant, G. (2001) Six Sigma: SPC and TQM in Manufacturing and Services, Gower Publishing, Burlington,VT.

Uzzi, John A. (2004) ‘Six Sigma: A Measureable Approach To Providing Quality Services’, August 23, 2004 edition, Insurance J. West Magazine.

p. 26 of 23