pq report kn 30jan2007

7/31/2019 PQ Report KN 30Jan2007

1/12

S O SSM

Services

Caterpillar SOS Services Evaluationof the

Analex PQ Ferrous Debris Monitor

Introduction:

The Analex PQ is designed to measure ferrous debris in lubricating oils. The PQinstrument exposes an oil sample to a magnetic field. If there are magneticparticles in the oil, the PQ records the change in the magnetic field. The result,of this measurement, is called the PQ index. The PQ index is proportional to themass of ferrous particles in the oil.

A small percentage ofCaterpillar dealers have purchased the Analex PQinstrument, and use them regularly as part of their oil analysis program. Also,

one of the Cat SOS Services regional laboratories provides PQ results. Some ofthese dealers use the PQ as a screening device to trigger more in-depth testing.This in-dept testing could be microscope analysis or filter patch analysis ofparticulates.

The PQ has not been used as a replacement for an optical particle counter.Instead, the PQ has been used as a supplement to optical particle counting.Optical particle counters are ineffective when used for samples with: high levelsof water contamination, dark oxidized oils, dark dyed oils, and engine oils withhigh soot content. Some oils contain so many particles that they exceed themeasuring capacity optical particle counters. And, most labs will not use an

optical particle counter on samples that contain visible particles. The PQ can beused in all of these cases.

In order to evaluate the value of PQ analysis, Caterpillar SOS Services launcheda 6 Sigma project. The 6 Sigma project evaluated PQ in three key areas:

1. Interpretation of PQ results2. Calibration, Accuracy, and Repeatability of the PQ instrument3. Worldwide instrument service capability

Each of these three areas will be summarized in this report, and a final

recommendation will be provided.

Interpretation of PQ results:

To understand how PQ results are used, the 6 Sigma team evaluatedinterpretations from four Cat dealers. Three of the dealers are located in NorthAmerica, and one of the dealers is in Europe. These dealers have been usingPQ results for two or more years.


2/12

S O SSM

Services Interpretations were selected from the following machine models: 777D, D8R,330C, D6R, and 980G. These machines were selected because of their highpopulation density and their wide range of applications. A total of 17,302 oilsamples were evaluated from these machines. The oil samples were evaluated

by individual compartment for each of the five models.

The objectives for this part of the project were as follows:1. Determine the best way to identify abnormal PQ results.2. Determine how PQ is used when analyzing sample results.3. Determine if PQ results add a needed dimension of information about the

sample.

Identifying Abnormal PQ ResultsBased on studies done by Caterpillar dealers, a set of guidelines have emergedfor the PQ. These guidelines were developed from samples taken exclusively on

Cat equipment. The guidelines were developed using the bottle method. Thebottle method measures the PQ of the sample in the original sample bottle. Apot method is also available. When the pot method is used, 2 ml of the sample isdecanted into a standard container (pot) supplied by Analex. The pot methodproduces lower PQ values.

PQ GuidelinesCompartment Bottle Method Average Pot Method - estimated

Engine 14 5Hydraulic System 14 5

Power Shift Transmission 22 7

Axles/Differentials 154 51Final Drives 108 36Tandems 297 99

The team discovered that these guidelines should only be used if there is nohistoric PQ data from the compartment. In some cases the guideline is too lowand there would be a very high percentage of samples flagged as abnormal. Inother cases the guideline is too high, and some abnormal samples would bemissed.

Instead of using these somewhat generic guidelines, the team found that

statistical guidelines could be developed, by model, for each compartment. Thestatistical guideline is the average value plus three times the standard deviation,(Average + 3Sigma). This statistical guideline flagged approximately 1.4% of thePQ samples as abnormal.

The team concluded that statistical guidelines would be needed, by model, foreach compartment. These statistical guidelines could be calculated with a tool


3/12

S O SSM

Services that is similar to the Wear Table generator. The statistical guidelines could thenbe written into SOS Services Manager and used to flag abnormal results.

If statistical guidelines are not available, the best way to identify abnormal resultsis through trend analysis.

How Interpreters use PQTo determine how the PQ results are used, the team closely investigated the1.4% of the samples flagged as abnormal for PQ. The team looked at theOverall Evaluation that was associated with these abnormal samples and theyalso looked at the way PQ was used in the interpretation.

24% of the abnormal samples produced a Red Alert on the SOS Services report.The percentage of Green Alerts and Yellow Alerts can be seen in the chartbelow.

Alert Level % Produced byAbnormal Samples

All Compartments

24%

35%

41%

This result was somewhat surprising since the PQ value for all these samplesexceeded the mean plus 3 sigma of the compartment population. Even thoughthe PQ values were quite high, for all of these samples, the interpreters did notassign a Red Alert to the report. After further investigation, the team discoveredthat many of the Yellow Alert and Green Alert samples did not have otherabnormal test results. Without collaborating evidence, the interpreters werereluctant to assign a Red Alert evaluation to the sample.

The team also broke down this data by compartment. Each major machinecompartment was evaluated for Red alerts, when the PQ was at abnormal levels.

The results, shown in the chart below, show that the interpreters found the mostvalue for PQ in evaluating geared compartments. Abnormal PQ values for finaldrive oils generated the most Red Alerts.

This chart produced another interesting result. Abnormal PQ values had theleast significance in the interpretation of engine oils. Since engine oils cannot beevaluated with optical particle counters, the team thought that abnormal PQvalues would be very important to engine oil interpretations. Apparently, many of


4/12

S O SSM

Services these abnormal engine PQ values were not supported by other abnormal testresults. Therefore, the interpreters did not apply a Red Alert to the overallevaluation.

% Red Alerts Produced by Abnormal PQ Values

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

Engine Hydraulic

System

Powershift

Transmission

Differential Final Drive

The Value of PQ in Red AlertsThe final part of this section was to evaluate the red alert samples and determinehow the interpreters used abnormal PQ results. For this evaluation, 58interpretations were studied. All of these interpretations had an abnormal PQresults, and they were assigned a Red Alert by a dealer interpreter.

The team found that 41% of these Red Alert samples absolutely needed the PQanalysis to make an accurate interpretation. In many of these cases, the

abnormal PQ value was accompanied by visible metal in the sample. The PQanalysis helped by placing a quantitative value to the visible metal.

% of Red Alert samples that needed PQ to make

an accurate interpretation

Maybe

20%

No

39%Yes

41%


5/12

S O SSM

Services 39% of the samples did not need an abnormal PQ value to make an accurateinterpretation. These samples had other results that alone would have led to aRed Alert evaluation. The remaining 20% of the samples may have needed theabnormal PQ for an accurate interpretation.

Interpretation SummaryIn this part of the project, the 6 Sigma team discovered that trend analysis orspecific PQ guidelines, by compartment, are needed to evaluate PQ results.General guidelines are not acceptable. The team also discovered that PQprovides the most value in the evaluation of Final Drive samples. Interestingly,PQ provided the least value in the interpretation of engine oil samples. Finallythe team found that about 41% of the Red Alert samples needed the PQ resultsfor an accurate interpretation. When compared to the 17,302 samples evaluated,PQ was needed for an accurate interpretation of 0.13% of these samples.

Calibration, Accuracy, and Repeatability of the PQ instrument:

CalibrationAnalex PQ instruments are factory calibrated to a PQ value of 3000. A ten-pointcalibration curve is used. The calibration process produces an R2 value of0.9969. For laboratory calibrations, Analex provides a single calibration standardwith a PQ value of 750. The calibration procedure uses zero as the starting pointand then measures the value of the standard. Calibration checks must be within1% of the PQ 750 standard. If the instrument is not within 1% of the PQ 750standard, the instrument will recalibrate to the PQ 750 standard.

The 6 Sigma team noted several shortcomings with this calibration procedure.

Laboratory best practices recommend that calibration procedures contain morethan two data points. However, Analex does not provide additional calibrationstandards with this instrument. Additional calibration standards can be custommade and purchased from Analex.

As an alternative to full calibration standards, Analex can provide checkstandards. Check standards are less costly than full calibration standards.These check standards are made to be within a PQ range. Check standards canbe used during calibration checks; however, they are not part of the instrumentcalibration process.

Another shortcoming is the material composition of the calibration standard. ThePQ 750 standard is made in an epoxy matrix, not oil. Again, laboratory bestpractices recommend that calibration be done in a material that is similar to thematerial under evaluation. Because of issues involving particle distribution andsettling, Analex does not provide calibration standards in oil.

The SOS Services lab in Malaga uses check standards and the PQ 750 standardto check calibration. Calibration checks are run four times a day. The results in


6/12

S O SSM

Services

the chart below show the repeatability of those checks. The last column in thechart shows how the results compare to the specified repeatability of theinstrument. The Analex PQ data sheet states the repeatability as +/- 4 PQ or1% of average reading, whichever is greater.

Standard Type

PQ Standard

value or (PQ

Range) value Average Sigma

Samples that

Meet Analex

Repeatability

check standard (0-30) 20.5 6 37%

check standard (90-150) 141.3 8.4 35%

check standard (625-685) 567.9 7.5 73%

calibration standard 750 751.4 5.3 88%

PQ Standards Results - Malaga Lab - June 2006

Analex Repeatability: +/-4 PQ or 1% of average reading, whichever is greater

This chart shows that this PQ instrument had difficulty meeting advertisedrepeatability in the Cat laboratory environment. Repeatability was improved atthe higher PQ values. The repeatability of the instrument will be furtherdiscussed later in this document.

AccuracyThe Analex brochure indicates that the PQ number is proportional to the mass offerrous (magnetic) wear debris. However, Analex does not provide thecorrelation between the PQ number and the mass of ferrous wear debris.

To investigate the relationship between PQ number and Fe, the team plotted PQ

verses Fe for typical compartments. The Fe values were generated by aspectrophotometer; therefore, the particle sizes are less than ten micron.Examples of these plots are shown below. This series of plots are from D8Rtractors. The same relationships existed for other Cat models in this study.

D8R Engines

R2

= 0.05

0

20

40

60

80

100

120

140

0 10 20 30 40 50 60

PQ

Fe-ppm


7/12

S O SSM

Services

D8R Hydraulic System

R2

= 0.08

10

20

30

40

50

Fe-ppm

0

0 10 20 30 40PQ

D8R Powertrain

R2

= 0.36

0

20

40

60

80

100

0 10 20 30 40 50 60 70

PQ

Fe-ppm

D8R Final Drives

R2

= 0.58

0

200

400

600

800

1000

1200

0 50 100 150 200 250 300 350 400PQ

Fe-p

pm


8/12

S O SSM

Services The plots show no correlation between the PQ number and Fe for the engine andhydraulic system compartments, as shown by the very low R2 values. Thepowertrain compartment begins to show some correlation with an R2 value of0.36. The final drives show the best correlation to Fe, with an R2 value of 0.58.The differences seen in these correlations could have been caused by particle

size differences. It is conceivable that an engine or hydraulic system wouldproduce smaller particles than a powershift transmission or a final drive.

This data indicates that the accuracy of the PQ number increases with higher PQvalues. It also may indicate that PQ has a better correlation to Fe when there arepotentially larger particles in the oil.

RepeatabilityThe repeatability of the PQ was evaluated by a series of controlled tests donewith the PQ instrument at the Malaga lab. 41 samples were selected from thesamples received for analysis. Compartment totals are shown in the chart below.

CompartmentNumber ofSamples

Engine 10

Final Drive 14

Powershift Trans 8

Hydraulic System 9

Total 41

The repeatability evaluation was done on the bottle method and on the potmethod. 2 ml from each sample was decanted into an Analex sample pot for thecomparisons. Each sample was then measured three times on the PQinstrument. The team set a repeatability target for this instrument at +/- 10% ofthe average.

PQ values within +/- 10% of Average

based on 3 tests per sample

50%

60%

70%

80%

90%

100%

Bottles

Pots

0%

10%

20%

30%

40%

Final Drives Transmissions Hydraluics Engines


9/12

S O SSM

Services le

mpared

the chart below. This slope of this line specifies that the bottle method will

2alue of 0.98. However, the data that strongly influenced this correlation is from

g of

no

The team concluded that this poor correlation was caused by the repeatability ofthe instrument at low PQ values.

The chart shows that the PQ results from the bottle method are more repeatabthan the PQ results from the pot method. The chart also shows the bestrepeatability from final drive and powershift transmission compartments.

The relationship between the average PQ values in bottles and pots is co

inproduce values that are 1.3 times higher than the pot method.

The correlation between the two methods appears to be very good, with the Rvresults where the PQ values exceeded 100. The red circle shows a groupinPQ results that are lest than 100. The chart below shows the data from the redcircle area. This grouping of data, with PQ values less than 100, showscorrelation.

Bottles vs Pots - All Compartments

y = 1.28x + 44.06

0

1000

2000

5000

0 500 1000 1500 2000 2500 3000 3500 4000

PQ in Pots

R2

= 0.98

3000

4000

PQi

nBottles

Bottles vs Pots with PQ < 100 - All Compartments

y = 1.11x + 44.77

R2

= 0.06

0

20

40

60

80

100

120

0 5 10 15 20 25 30 35 40PQ in Pots

PQi

nBottles


10/12

S O SSM

Services

n atn

nough data points. Additional calibration standards would be needed to perform

does notprovide the correlation between PQ number and the mass of ferrous wear debris.The instrument does show a reasonable correlation to spectrophotometer Feppm for final drive samples. The final drive samples have the potential to containlarger size particles and they generally generate higher PQ values.

The repeatability of the instrument is better for the bottle method than for the potmethod. The bottle method produces PQ values that are about 1.3 times higherthan the pot method. The repeatability is only acceptable with final drivesamples.

Worldwide Instrument Sales & Service Capability

Caterpillar SOS Services laboratories are located all over the world. Therefore,

Kittiwake maintains:1. There is no major maintenance associated with the PQ instrument itself.The sensor area should be kept clean.

2. All sales and service are done from a central location in the UnitedKingdom. If a PQ instrument requires service, it must be sent back to theUK.

3. If service is required on an instrument, Kittiwake does not provide atemporary instrument.

This level of service and support may not be acceptable to a large portion ofthe SOS Services labs.

Summary of Calibration, Accuracy, and RepeatabilityThe team concluded that Analex provides a thorough instrument calibratiothe factory. However, the laboratory calibration procedure does not contaiea better laboratory calibration.

The accuracy of the instrument cannot be validated since Analex

there must be adequate sales and service capability for instruments used inthese labs.

Analex PQ instruments are sold, distributed, and serviced by Kittiwake. The 6Sigma team discussed the sales and service capabilities with Kittiwakerepresentatives.


11/12

S O SSM

Services

terpretation of PQ Resultsd to

alibration, Accuracy, and Repeatability

lex does not provide data to check accuracy.

7. Accuracy is indirectly demonstrated by a reasonable correlation to

8. The repeatability of the instrument is better for the bottle method than for

rhe pot method.

10. The instrument meets SOS Services repeatability guidelines for final drive

11. Kittiwake does not maintain a staff of service technicians for this

erefore, the instrument must be returned to Kittiwake in the

12. instrument, if service is required

SOS Services Recommendations

s does not recommend the Analex PQ for the

the following problems:

A. Interpreters did not consistently utilize abnormal PQ results.B. The laboratory calibration procedure for the Analex PQ does not

contain a sufficient number of data points.C. The repeatability of the instrument for engines, transmissions,

and hydraulic system samples does not meet the +/- 10%guideline established by SOS Services.

Summary of Results

The following list summarizes the results obtained by the 6 Sigma teamregarding the evaluation of the Analex PQ instrument.

In1. Trend analysis or specific PQ guidelines, by compartment, are neede

evaluate PQ results. General guidelines are not acceptable.2. PQ provides the most value in the evaluation of final drive samples.3. PQ provided the least value in the interpretation of engine oil samples.

C4. Analex provides a thorough instrument calibration at the factory.5. The laboratory calibration procedure does not contain enough data points

for high level of confidence.6. Ana

spectrophotometer Fe ppm, for final drive samples.

the pot method.9. The bottle method produces PQ values that are about 1.3 times highe

than t

samples.

Service Capability

instrument. ThUK for repairs.Kittiwake does not provide a temporaryon your instrument.

1. Caterpillar SOS Servicegeneral analysis and interpretation of used oil samples.

The 6 Sigma team sighted


12/12

S O SSM

Services of Kittiwake to efficiently support and service an

ex PQ instruments was in

If an SOS Services laboratory has already purchased an Analex

ent should only be used a supplement to the

Daaterpillar SOS Services

D. The abilityestimated additional 100 Anelquestion.

PQ, the instrumcore program.

vid S. NyczC30 January 2007

pq report kn 30jan2007

Documents