Enhancements to IIIG LTMSBy: Todd Dvorak

01-29-09

Background– The LTMS Severity charts indicate that the test is severe and at the warning

limit.

– An analysis of the industry data indicates that there is statistical difference in current WPD parameter test results as compared to the period when the test was established.

– The Industry Severity Adjusted results for the WPD parameter are also below the established reference oil target values

– LTMS enhancements and WPD transforms should be explored to bring the test closer to on target performance.

– The enhancements and transformations explored in this presentation include:• Reference Oil selection (Section 1)• Changing LTMS factors (Section 2)• Applying WPD transforms (Section 3)

Section I – Reference Oil Selection

Reference Oil Selection• A plot of the WPD means by reference oil suggests that the performance is more

severe with reference oils 434 and 435. (Means based on PMNS & BC6-BC7 ring data)

• Eliminating 438 will result in severity adjustments that are better matched to the performance level of a passing candidate test oil.

• It would be advantageous to include a GF5 capable reference oil and/or change the reference oil frequency mix. (i.e. 2 x 434’s for each 435 reference, etc.)

4

2.50

3.00

3.50

4.00

4.50

5.00

WPD

Reference Oil

WPD Target & Average by Reference Oil(All Labs with PMNS Rods & BC6-BC7 Ring Data)

WPD Target WPD Avg

Oil 438 Oil 435 Oil 434

Reference Oil Selection• The following summarize the Severity Adjusted results with each of the

reference oil mix combinations. – Severity Adjusted results for reference oil 434, 435, and 438

– Severity Adjusted results for reference oil 434 and 435 (Pooled S for reference oil selection ~ 0.62)

– Severity Adjusted results for reference oil 434 (Pooled S for reference oil selection ~ 0.76)

5

Industry Data Summary (2007 & 2008 Data)SA Adj WPD WPD Stdev Prop > Target N

Oil 434 (Target = 4.80) 4.25 0.46 0 15Oil 435 (Target= 3.59) 3.62 0.32 0.65 23Oil 438 (Target = 3.20) 3.34 0.23 0.71 21

Industry Data Summary (2007 & 2008 Data)SA Adj WPD WPD Stdev Prop > Target N

Oil 434 (Target = 4.80) 4.44 0.47 0.13 15Oil 435 (Target= 3.59)Oil 438 (Target = 3.20)

Industry Data Summary (2007 & 2008 Data)SA Adj WPD WPD Stdev Prop > Target N

Oil 434 (Target = 4.80) 4.28 0.47 0 15Oil 435 (Target = 3.59) 3.60 0.34 0.61 23Oil 438 (Target = 3.20)

Reference Oil Selection• Reference Oil Selection summary:

– Severity adjusted results are affected by the reference oil selection

– Eliminating reference oil 438 will have a favorable affect on laboratory based SAs

– Changing the reference oil mix will have a favorable effect on Laboratory based SA’s:

• Increase 434 reference frequency mix (recommend a minimum of 2 x 434’s for each 435)

• Add a GF5 capable reference oil

6

Section II – LTMS Modifications

LTMS Modifications• The previous section concluded that the laboratory based severity

adjustments remain severe of target – even with modifications to the reference oil selection.

• Additional Changes to LTMS will be explored to determine if improvements can be made to bring the Severity Adjusted WPD results closer to the target values.

• The LTMS changes to be evaluated include analyzing the effects of (Action) K factor, Lambda, and SPC “8 in a row” rule changes.

• All changes will be evaluated with reference oils 434 & 435, exclusively.

8

LTMS Modifications• The “Action Limit” for a laboratory based severity adjustment is a function of

both and the K value.

• As shown in the below response plot, both and K values have an equivalent effect on the severity adjustment “Action Limit.”

9

λ2λK0limitAction

LTMS Modifications• A computer program evaluated a series of Lambda and K value combinations

- using reference oils 434 & 435. The results are summarized in the below contour plots.

• The plots indicate that a reduced (Action) K and Lambda solution set of {1.5, 0.15}, respectively will result in a more favorable (WPD) SA.

10

Lambda

Actio

n (K

)

0.50.40.30.20.1

2.5

2.0

1.5

1.0

0.5

> – – – – < 3.42

3.42 3.483.48 3.543.54 3.603.60 3.66

3.66

WPD(435)SA

Contour Plot of SA WPD(435) vs Action (K), Lambda

Lambda

Actio

n (K

)

0.50.40.30.20.1

2.5

2.0

1.5

1.0

0.5

> – – – – < 4.08

4.08 4.164.16 4.244.24 4.324.32 4.40

4.40

WPD(434)SA

Contour Plot of SA WPD(434) vs Action (K), Lambda

LTMS Modifications• One SPC rule indicates that a process mean change may have occurred if 8

observations occur above or below the centerline1. The effects of adding this rule to trigger a laboratory based severity adjustment are summarized below.

• The plots indicate that a reduced (Action) K and Lambda solution set of {1.0, 0.2}, respectively, will result in a more favorable (WPD) SA.

11Note 1: “Statistical Quality Design and Control”, DeVor, Change, Sutherland, 1992.

Lambda

Actio

n (K

)

0.50.40.30.20.1

2.5

2.0

1.5

1.0

0.5

> – – – – < 4.24

4.24 4.284.28 4.324.32 4.364.36 4.40

4.40

WPD(434)SA

(With "8 in a Row" SPC Rule)Contour Plot of SA WPD(434) vs Action (K), Lambda

Lambda

Actio

n (K

)

0.50.40.30.20.1

2.5

2.0

1.5

1.0

0.5

> – – – – < 3.600

3.600 3.6253.625 3.6503.650 3.6753.675 3.700

3.700

SA WPD(435)

(With "8 in a Row" SPC Rule)Contour Plot of SA WPD(435) vs Action (K), Lambda

LTMS Modifications• The below following summarizes the LTMS Severity Adjusted results for the

selected and K factors:

• Conclusions:– With selected Lambda’s and K values, the lower K value solution sets have a slight advantage.– The and K of 0.15 and 1.5, respectively, may be more preferable solution set – since it is

more similar to the current LTMS factor settings.– There appears to be no practical difference in the Severity Adjusted results with the “8 in a

Row” SPC rule.

12

Industry Data Summary (2007-2008)K Value Lambda Reference Oil SA Adj WPD WPD Stdev Prop > Target N 8 in A Row Rule

1.00 0.20 434 4.37 0.51 0.20 15 No1.00 0.20 435 3.68 0.29 0.74 23 No1.00 0.20 434 4.37 0.51 0.20 15 Yes1.00 0.20 435 3.70 0.29 0.74 23 Yes1.50 0.15 434 4.32 0.50 0.13 15 No1.50 0.15 435 3.62 0.33 0.70 23 No1.50 0.15 434 4.33 0.52 0.20 15 Yes1.50 0.15 435 3.68 0.34 0.78 23 Yes

LTMS Modifications• Supplemental thought for discussion:

– If the industry data indicates that the test is severe, then there is some justification for a reduced K value for Laboratory based Severity Adjustments.

13

Section III – WPD Transforms

WPD Transforms• Industry data suggests that the WPD variability is a function of the reference

Oil mean.

WPD Performance & Variability Relationship

438435434

6

5

4

3

2

IND

WPD

Individual Value Plot of WPD95% CI for the Mean

Diagnostics of Yi Metric for each Reference Oil Histogram Plot of Yi by Reference Oil Type

WPD Transforms for Control Charts

438

435

434

1.51.41.31.21.11.00.90.80.70.6

IND

95% Bonferroni Confidence Intervals for StDevs

Test Statistic 9.47P-Value 0.009

Test Statistic 0.90P-Value 0.407

Bartlett's Test

Levene's Test

Test for Equal Variances for WPDyi

• Two assumptions of Shewhart Control charts are constant mean and variance. • Multiple comparisons of WPD Yi data (with lab & oil factors) suggests

statistical differences between reference oil Yi means

• The descriptive statistics of Yi data by reference oil also suggest that it may be advantageous to explore WPD transforms.

4.53.01.50.0-1.5-3.0-4.5

20

15

10

5

0

4.53.01.50.0-1.5-3.0-4.5

20

15

10

5

0

434

WPDyi

Freq

uenc

y

0435

0438

Mean -0.6644StDev 0.7986N 70

434

Mean -0.2001StDev 0.9864N 80

435

Mean -0.1329StDev 1.153N 78

438

Normal

Panel variable: IND

Histogram of all Chartable WPDyi Data

(WPD, Lab, & Ref Oil) - GLM Residual Diagnostics

438435434

2.0

1.5

1.0

0.5

0.0

-0.5

-1.0

-1.5

IND

WPD

Mod

el R

esid

uals

95% CI for the MeanIndividual Value Plot of GLM WPD Model Residuals with Lab and Oil Factors

Model Fit Residuals Plot by Reference Oil Type

210-1-2

99.999

90

50

10

10.1

Residual

Perc

ent

4.54.03.53.0

2

1

0

-1

Fitted Value

Resi

dual

2.01.51.00.50.0-0.5-1.0-1.5

60

45

30

15

0

Residual

Freq

uenc

y

220200180160140120100806040201

2

1

0

-1

Observation Order

Resi

dual

Normal Probability Plot Versus Fits

Histogram Versus Order

Residual Plots for WPD with Lab and Oil FactorsIncludes all Chartable 434, 435, and 438 Reference Oil Data

• Severity Adjustments are based on General Linear Model (GLM) Pooled S• Two of the GLM assumptions require a constant variance and normal distribution of

the errors.

• The residual diagnostics of a fitted model also suggest that it may be advantageous to explore WPD transforms to better satisfy GLM assumptions.

WPD Transforms for General Linear Models

WPD Transforms• Several types of WPD transformations were explored to help satisfy GLM

and control charting assumptions.

• Two possible transforms are based on a natural log and inverse of the WPD parameter.

• A summary of both transforms are provided on the following slides.

Inverse WPD Transform• The first evaluated transform is based on the WPD inverse:

• The residual diagnostics indicate that this transform better satisfies the GLM assumptions.

• Multiple comparisons of 1/WPD Yi data (with lab & oil factors) indicates statistical differences between reference oil Yi means

• The model fit diagnostics of the LTMS Industry data are summarized below.

WPD1WPD'

(1/WPD, Lab, & Ref Oil) - GLM Residual Diagnostics Model Fit Residuals Plot by Reference Oil Type

0.10.0-0.1

99.999

90

50

10

10.1

Residual

Perc

ent

0.350.300.25

0.1

0.0

-0.1

Fitted Value

Resi

dual

0.160.120.080.040.00-0.04-0.08

60

45

30

15

0

Residual

Freq

uenc

y

220200180160140120100806040201

0.1

0.0

-0.1

Observation Order

Resi

dual

Normal Probability Plot Versus Fits

Histogram Versus Order

Residual Plots for 1/ WPD with Lab and Oil FactorsIncludes all Chartable 434, 435, and 438 Reference Oil Data

438435434

0.15

0.10

0.05

0.00

-0.05

-0.10

IND

1/W

PD M

odel

Res

idua

ls

95% CI for the MeanIndividual Value Plot of 1/ WPD Model Residuals with Lab and Oil Factors

LTMS Modifications with WPD Inverse Transform

• With the WPD Inverse Transform, a computer program evaluated a series of Lambda and K value combinations - using reference oils 434 & 435. The results are summarized in the below contour plots.

• Similar to the untransformed results, the plots indicate that a reduced (Action) K and Lambda value of {1.0, 0.2} or {1.5, 0.15}, respectively, will result in a more favorable (WPD) SA.

20

Lambda

Actio

n (K

)

0.50.40.30.20.1

2.5

2.0

1.5

1.0

0.5

> – – – – < 0.278

0.278 0.2830.283 0.2880.288 0.2930.293 0.298

0.298

SA 1/WPD(435)

Contour Plot of SA 1/ WPD(435) vs Action (K), Lambda

Lambda

Actio

n (K

)

0.50.40.30.20.1

2.5

2.0

1.5

1.0

0.5

> – – – – < 0.230

0.230 0.2350.235 0.2400.240 0.2450.245 0.250

0.250

SA 1/WPD(434)

Contour Plot of SA 1/ WPD(434) vs Action (K), Lambda

Natural Log based WPD Transform• The second evaluated transform is based on the natural log of WPD:

• The residual diagnostics indicate that this transform also helps to satisfy the GLM assumptions.

• The model fit diagnostics of the LTMS Industry data are summarized below.

(Ln(WPD*10+1), Lab, & Ref Oil) - GLM Residual Diagnostics Model Fit Residuals Plot by Reference Oil Type

1)10*Ln(WPDWPD'

0.500.250.00-0.25-0.50

99.999

90

50

10

10.1

Residual

Perc

ent

3.83.73.63.53.4

0.4

0.2

0.0

-0.2

-0.4

Fitted Value

Resi

dual

0.30.20.10.0-0.1-0.2-0.3-0.4

40

30

20

10

0

Residual

Freq

uenc

y

220200180160140120100806040201

0.4

0.2

0.0

-0.2

-0.4

Observation Order

Resi

dual

Normal Probability Plot Versus Fits

Histogram Versus Order

Residual Plots for Ln(WPD*10+1)

438435434

0.4

0.3

0.2

0.1

0.0

-0.1

-0.2

-0.3

-0.4

-0.5

IND

Ln(1

0*W

PD+

1) R

esid

uals

95% CI for the MeanIndividual Value Plot of Ln(10*WPD+1) Residuals with Lab and Oil Factors

LTMS Modifications with Ln(WPD*10+1) Transform• The below summarizes the Severity Adjusted results for reference oils 434 and

435 of all combinations of and K factors with the Ln(WPD*10+1) Transform.

• Similar to the untransformed results, the plots indicate that a reduced (Action) K and Lambda value of {1.0, 0.2} or {1.5, 0.15}, respectively, will result in a more favorable (WPD) SA.

22

Lambda

Actio

n (K

)

0.50.40.30.20.1

2.5

2.0

1.5

1.0

0.5

> – – – – < 3.555

3.555 3.5703.570 3.5853.585 3.6003.600 3.615

3.615

*10+1)Ln(WPD(435)

Contour Plot of Ln(WPD(435)*10+1) vs Action (K), Lambda

Lambda

Actio

n (K

)

0.50.40.30.20.1

2.5

2.0

1.5

1.0

0.5

> – – – – < 3.72

3.72 3.743.74 3.763.76 3.783.78 3.80

3.80

*10+1)Ln(WPD(434)

SA

Contour Plot of SA Ln(WPD(434)*10+1) vs Action (K), Lambda

LTMS Summary with WPD Transforms• The below summarizes the severity adjusted results of the analyzed WPD

transforms and the LTMS based , K, and “8 in a row” change proposals:

23

Industry Data Summary (2007-2008) for Selected Transform, Lambda, K Value, & 8 In a Row Rule

K Value LambdaReference

Oil Transform TypeSeverity Adj. Trans WPD

Untrans Severity Adj. WPD Result

WPD Severity Adj. Stdev Prop > Target N

8 in A Row Rule

1.00 0.20 434 1/WPD 0.233 4.29 0.032 0.20 15 No1.00 0.20 435 1/WPD 0.280 3.57 0.023 0.61 23 No1.00 0.20 434 1/WPD 0.232 4.31 0.033 0.27 15 Yes1.00 0.20 435 1/WPD 0.278 3.60 0.023 0.61 23 Yes1.50 0.15 434 1/WPD 0.236 4.24 0.031 0.13 15 No1.50 0.15 435 1/WPD 0.283 3.53 0.027 0.52 23 No1.50 0.15 434 1/WPD 0.232 4.31 0.034 0.27 15 Yes1.50 0.15 435 1/WPD 0.280 3.57 0.027 0.57 23 Yes

1.00 0.20 434 Ln(10*WPD+1) 3.79 4.33 0.13 0.27 15 No1.00 0.20 435 Ln(10*WPD+1) 3.62 3.62 0.08 0.70 23 No1.00 0.20 434 Ln(10*WPD+1) 3.79 4.33 0.13 0.27 15 Yes1.00 0.20 435 Ln(10*WPD+1) 3.62 3.65 0.08 0.70 23 Yes1.50 0.15 434 Ln(10*WPD+1) 4.32 4.31 0.50 0.13 15 No1.50 0.15 435 Ln(10*WPD+1) 3.62 3.63 0.33 0.70 23 No1.50 0.15 434 Ln(10*WPD+1) 3.79 4.32 0.13 0.20 15 Yes1.50 0.15 435 Ln(10*WPD+1) 3.62 3.61 0.09 0.70 23 Yes

1.00 0.20 434 WPD 4.37 0.51 0.20 15 No1.00 0.20 435 WPD 3.68 0.29 0.74 23 No1.00 0.20 434 WPD 4.37 0.51 0.20 15 Yes1.00 0.20 435 WPD 3.70 0.29 0.74 23 Yes1.50 0.15 434 WPD 4.32 0.50 0.13 15 No1.50 0.15 435 WPD 3.62 0.33 0.70 23 No1.50 0.15 434 WPD 4.33 0.52 0.20 15 Yes1.50 0.15 435 WPD 3.68 0.34 0.78 23 Yes

LTMS Change Proposal Conclusions• Transformation of WPD Summary:

– Transforms help to better satisfy GLM modeling assumptions

– With the selected values, K factors, the untransformed WPD results are closer to the reference oil targets than the transformed WPD results.

– Regardless of the WPD transform or changes to K or factors, a large majority of the Industry wide severity adjusted WPD results remain below the reference oil target values.

24

Recommendations

Summary• Recommend eliminating oil 438, establishing a new reference oil mix (2

or more 434 reference tests for each 435), and adding a GF5 capable oil.

• Recommend changing the IIIG WPD K and to be set to 1.5 and 0.15, respectively.– The K and changes to be applied to the WPD parameter, exclusively.

• Recommend a reduced K (i.e. K = 1 =0.2) when the industry is severe

• The addition of a new “8 in a Row” rule has a small effect on the LTMS severity adjusted results. (It is optional.)

• The application of a WPD transform is optional. – It has a minimal effect the Industry Wide Severity Adjusted (WPD) results.

• None of the proposed changes will result in a laboratory based SA 26

Summary

• None of the proposed changes will result in a laboratory based SA that has an “on target” performance.– It would be advantageous that a 434 reference test would result in an adjusted

WPD of 4.8

27