soil compaction - 2017 (76) proficiency testing program report · report this report is available...

www.labsmartservices.com.au

Soil Compaction - 2017 (76)

PROFICIENCY TESTING

PROGRAM REPORT

Accredited for compliance with ISO/IEC 17043

Copyright: LabSmart Services Pty Ltd

Compaction Proficiency Testing Program - 2017(76)

Copyright: LabSmart Services Pty Ltd Issued - January 2018 of 34

Report This report is available on the LabSmart Services website. The issue of this proficiency report was authorised by Peter Young, Director, LabSmart Services Pty Ltd, February 2018. Contact Details

Email: [email protected] Mobile: 0432 767 706 Fax: (03) 8888 4987

Program Coordinator The program coordinator for this program was Peter Young, Director, LabSmart Services Pty Ltd. Contact Details

Email: [email protected] Mobile: 0432 767 706 Fax: (03) 8888 4987

Acknowledgements Please note that any technical questions regarding this program are to be directed to the program coordinator.

Accredited Proficiency Testing Provider LabSmart Services is accredited by NATA to ISO/IEC 17043, Conformity assessment – General requirements for proficiency testing. Accreditation number 19235. The accreditation provides additional assurance to participants of the quality and importance we place on our proficiency testing programs.

LabSmart Services As well as proficiency testing programs LabSmart Services also offers nuclear gauge calibration. Please see our website for further details.

www.labsmartservices.com.au

Copyright This work is copyright. No part of this publication may be reproduced in any form, transmitted or stored in any repository (e.g. mechanical, digital, electronic or photographic) without prior written permission of LabSmart Services Pty Ltd. Please contact LabSmart Services should you wish to reproduce any part of this report.

Amendment History Reports may be downloaded from the LabSmart Services website.

Version 1 – Issued 20 February 2018



CONTENTS PAGE 1. Program Aim 4

2. Performance

2.1 Identified outliers 2.2 Program summary

4

4 5

3. Technical Comment

3.1 Part A

3.1.1 Test methodology 3.1.2 Soil curing 3.1.3 MDD & OMC 3.1.4 Precision 3.1.5 Graphing

3.2 Part B

3.2.1 Data analysis and graphing 3.2.2 Compaction graph

3.3 Test Method

6

6

6 6 6 7 8

9 9 10

14

4. Statistics: Z – Score & Graph

4.1 Part A - Maximum Dry Density (MDD) 4.2 Part A - Optimum Moisture Content (OMC) 4.3 Part B - Maximum Dry Density (MDD) 4.4 Part B - Optimum Moisture Content (OMC)

16

16 18 20 22

5. Program Information

5.1 Z-score summary 5.2 Program design

5.2.1 Design 5.2.2 Selection of material for program 5.2.3 OMC & MDD 5.2.4 Role of proficiency testing 5.2.5 Participant assessment 5.2.6 Reporting of results – significant figures 5.2.7 Additional information requested 5.2.8 Data checks

5.3 Sample Preparation 5.4 Packaging and Instructions 5.5 Quarantine 5.6 Sample Dispatch 5.7 Homogeneity Testing 5.8 Participation 5.9 Statistics

5.9.1 Z-score Summary 5.9.2 Comparing statistics from one program to another 5.9.3 Measurement uncertainty 5.9.4 Metrological traceability

5.10 Non-statistical outlier

24

24 24

24 24 25 25 25 25 26 26

26 26 27 27 27 28 28

29 29 30 30

30

Appendix A Instructions for testers

Appendix B Results log

Appendix C Participant supplied test information

31

32

34



1. Program Aim The proficiency testing program was conducted in November 2017 with participants throughout Australia. The program involved the performance of the following tests.

▪ Standard Maximum Dry Density (MDD)

▪ Standard Optimum Moisture Content (OMC) AS 1289.5.1.1 (2017) was the preferred testing method but other equivalent methods were accepted. Each participant’s performance is statistically assessed and used as a measure of their competency relative to all those who participated. The program provides feedback and confidence to participants and the industry regarding the competency of laboratories to perform these tests. Details relating to the design and conduct of the program can be located in section 5.

2. Performance

2.1 Identified outliers Overall a satisfactory level of testing was achieved by 57% of the 30 participants returning results. There were 13 participants identified as having one or more outliers (43%). Participant’s test results are tabulated in section 4 along with the robust statistics and a z-score graph. The z-score indicates how far away a participant is from the program’s median value. A z-score of zero indicates a strong consensus with respect to all other participants and represents a very good outcome. The z-score graph gives a quick visual indication of how a result compares to others in the program. Outliers are classified as z-scores where the z-score value was greater than 3 or less than -3. It is recommended that participants with outliers investigate their performance of the test. Participants with outliers are detailed in table 2.1. Those participants with z-scores greater than 2 or less than -2 may wish to review their testing methodology. Only those approaching 3 (outside ± 2.70) have been specifically identified in table 2.1 as feedback. More detail on the robust statistics used can be found in section 5. Technical comment and feedback in section 3 is provided to assist participants investigate or review their results as well for those seeking to improve their testing performance.



Proficiency Component Test Investigate Review*

Part A Soil Sample

Maximum Dry Density (MDD) Z6 -

Optimum Moisture Content (OMC) A8, Z5, G7, P6, K3,

Q4, E3 -

Part B Reference

Data

Maximum Dry Density (MDD) A8, B3, D9, P2, U8 B5

Optimum Moisture Content (OMC) D9, V4, U8 A8, C7

* Those in brackets are not considered outliers but may wish to review their results (see section 3.2.1). Only z-scores greater than 2.70 and less than -2.70 have been identified in the above table.

Table 2.1 Identified outliers

2.2 Program Summary There were 31 participants that applied for the proficiency testing program. Of these 30 participants returned results in time to be included in the final report. The 57 % of participants performed well in this program. The spread of results was very small with standard deviations within the ranges expected. The overall spread of the program compares favourably to the precision suggested by the standard for both OMC and MDD. Laboratories need to ensure graphs are checked to ensure they are both accurate and appropriate prior to issuing results. Laboratories should consider plotting air voids on compaction graphs and check that air voids when plotted are meaningful. Attention to the spread of compaction points, where two on the dry leg and two on the wet leg is ideal, will result in greater accuracy of testing. The following (table 2.2) is a summary of the results obtained. Normalized IQR values approximate standard deviations.

Statistic

Part A - Soil Sample Part B - Reference Data

MDD OMC MDD OMC

t/m3 % t/m3 %

Number of participants 30 30 25 25

Median 2.093 8.6 1.482 14.4

Normalized IQR 0.022 0.15 0.0007 0.07

CV (%) 1.0 1.7 0.04 0.5

Min* 2.055 8.2 1.840 14.2

Max* 2.114 8.9 1.844 14.6

Range* 0.059 0.7 0.004 0.4

* Minimum, Maximum and Range are calculated with outliers excluded.

Table 2.2 Summary Statistics



The proficiency testing program indicates a satisfactory level of testing and within industry expectations for only half the participants. The proficiency program was a useful exercise, allowing participants to have greater confidence in their results while for others providing an opportunity to improve their competency with respect to the test method covered.

3. Technical Comment 3.1 Part A 3.1.1 Test Methodology

Participants were requested to provide additional details about the testing performed. These details were used to help analyse the proficiency program results. In addition, the information can help with the investigation of outliers arising from the program. See Appendix C for participant responses. All participants nominated the test methods used as AS 1289.5.1.1 and AS 1289.2.1.1. All participants reported that they used the ‘A’ sized mould and hand compacted. 3.1.2 Soil Curing

There was a large range of curing times used by participants despite the addition to the test method standard of further guidance on the approach to curing (see Appendix C). The curing times ranged from 2 hours to 96 hours. The material needs to be cured in such a manner as to ensure moisture is homogeneous throughout. 3.1.3 Maximum Dry Density (MDD) & Optimum Moisture Content (OMC)

The following participants had outliers for ‘Part A’ of the program that need to be investigated.

Test Investigate Review

Maximum Dry Density (MDD) Z6 -

Optimum Moisture Content (OMC) A8, Z5, G7, P6, K3, Q4, E3 -

For many with outliers the graphing was a contributing factor in the outlier. Re-graphing showed that many did not have an outlier. Some participants have inadvertently changed the scale of the density and moisture axis used to give a ‘better looking’ graph. Care should be exercised as this can be quite misleading. Those with outliers have raw data meeting the test method but as explained in section 3.3 the choice of test points can lead to greater variation in results.



The data from Z6 was re-graphed. Moisture was evenly spaced at around 2 % for each compaction. Re-graphing did not yield any improvement in the results obtained. Participant A8 did not supply a graph. When graphed there was only one point on the wet leg and an OMC of 8.5 was obtained. May need to look at moisture intervals and graphing technique. With Participant Z5 re-graphing did not yield any improvement in the results obtained. Participant G7 re-graphing yielded a OMC of 8.3 which was closer to the median value. Three points on the dry side and moisture spacing of 1 % may have resulted in the data not being definitive or accurate enough. Participant P6 did not supply a graph. When re-graphed similar results were obtained. There was only one point on the wet leg. May need to look at moisture intervals and graphing technique. Participant K3 re-graphing yielded a OMC of 8.3 and MDD of 2.130. The moisture was closer to the median value. Three points on the dry side have resulted in the data not being definitive or accurate enough. The data for Q4 was inconclusive when re-graphed. There densities were flat with only 0.051 in density between the lowest and highest on the dry leg. This may indicate an issue with compaction or calculations. Three points on the dry side may result in a less accurate curve. Participant E3 re-graphing yielded a OMC of 8.5 and MDD of 2.115. May need to look at graphing technique. 3.1.4 Precision

Proficiency programs that use a consensus median value to assess participants performance are limited as to what can be said about the accuracy of the test results. This is normal for many proficiency programs. Only testing against a reference sample or a method shown to be more accurate would yield information regarding the accuracy. It is hoped however that the larger the number of participants in a program the closer the move towards an accurate median value. The variation determined for the participants in this program is a measure of reproducibility (i.e. the variation from laboratory to laboratory). It is a measure of the precision of testing shown by the participating laboratories rather than the precision associated with repeat testing by a single operator. Reproducibility AS 1289.5.1.1 under section 7 indicates the reproducibility based on ranges and comparing two results from two different laboratories.



For both MDD and OMC the maths is the same. Find the difference between two results. Express the difference (range) as a percentage of the average of these two results. Next compare the value obtained to those shown in the test method i.e. 4% for MDD and 20% for OMC. The calculated value should be less than the test method quoted precision to be acceptable. The test method intends this approach to be used for comparing the single result from each of two laboratories. Without further information, as to the approach used to derive the data shown in the test method it is unclear as to how applicable this may be compared to a proficiency program. If the highest and lowest participants result from the proficiency program are chosen, the following is obtained.

Part A Max Min Range Mean Range ÷ Mean %

Limit Pass

MDD 2.114 2.055 0.059 2.093 2.8 <4 Yes

OMC 8.9 8.2 0.7 8.6 8.1 <20 Yes

Table 3.1 Precision check based on test method limits

The minimum and maximum values from the proficiency program represent the largest variation arising from the proficiency program (outliers excluded). As these are below the limits set in the test method it means all other participants also meet the limits indicated in the test method. Repeatability Determination of repeatability (i.e. the variation shown within a laboratory testing of the same sample multiple times) was not part of this proficiency program. The homogeneity results give an approximation of repeatability (precision within that laboratory) however it should be remembered that this result is for one particular laboratory only. 3.1.5 Graphing

This proficiency program specifically requested graphs to be submitted. The graphing technique used was predominately via software. The quality of graphing has improved considerably over the last few programs. Many had very good graphs such as participant J8. The test method indicates that a graph must be prepared to derive the OMC and MDD. Graphing the results is also the most practical approach for assessing the correct performance of this test and reliability of the result obtained. Unfortunately, the test method does not specifically define the derivation of the graph or other important aspects pertinent to the accuracy of this test. For example, it does not define when a result should be rejected.



Many participants had only one point on the wet leg side which is allowed by the test method. Reviewing participant results it did appear that if the single wet point was out it could throw out the curve and OMC/MDD obtained. If the same point was used to estimate the voids line, then this too could be out. Detailed feedback on graphing has been provided for past programs but with many now using ‘software based’ systems the graphs provided are too small to review. However, laboratories may have the ability to enlarge graphs during the checking process. Graphs that are produced need to be reviewed as there is still an element of judgement involved and the chance that the software used has not produced the best outcome. Graphs need to be ‘fit for purpose’. Many laboratories need to keep working towards this. Comments on the graphs submitted for Part A and B are covered in section 3.2.2 under ‘Compaction Graph’. 3.2 Part B

Most participants used the same graphing technique for both Part A & B.

There is still however an element of judgement involved and the chance that the software used to produce the graph has not produced the best outcome. For these reasons ‘Part B’ of the program is an important aspect of the program. Part B of the program is meant to find gross departures from practice. The data used while typical of test data is an ‘ideal set’. That is, it has two evenly spaced data results either side of OMC/MDD. See section 3.3 for further discussion. 3.2.1 Data analysis and graphing

Part B provided participants with a standard set of data and requested that the MDD & OMC be determined. The outcomes reflect only the variation associated with the graphing and interpretation component of the graph. Although the variation is small (see table 2.2) it does contribute significantly to the overall variation. The variation observed is smaller or similar to previous proficiency testing programs. An example plot is shown by graph 3.2-C. It is unlikely that everyone would get the same result as laboratories use slightly different approaches. It is expected that the results amongst participants should be close. In fact, the results are very close which is a great outcome. It should be noted however that the number of participants is not large and if five or more participants use the same ‘software’ to derive OMC and MDD then it can create a bias. The outcome should be put into perspective. That is the standard deviation is MDD ± 0.0007 and OMC ± 0.07. Three standard deviation is MDD ± 0.002 and OMC ± 0.21. The variation allowed is very small. Hand graphing is hard to compete for accuracy against computer graphed.



Five participants (A8, B3, D9, P2 & U8) had statistical outliers for MDD Part B that need to be investigated while participant B5 should review their results. Three participants (D9, V4, U8) had statistical outliers for OMC Part B that need to be investigated while participants A8, C7, P6 should review their results. A8 – Graph fit appears incorrect. B3 – Hand drawn – did not use first data point in preparing graph. D9 – Curve appears inaccurate – very small graph and hard to determine. B5 – Graph great but have misread the graph otherwise results would have been very good. P2 – Hand drawn U8 – Fit may be slightly out – tends to flatten at top – particle density 2.58. V4 – Hand drawn C7 – Hand drawn P6 – Not able to comment as no data supplied. 3.2.2 Compaction Graph

The compaction graph gives a visualization of the test results. It is useful as a quick means of determining how well the test has been performed. This is conveyed through the “fit” of the curve to the points and spacing of the compaction/moisture data points. An “air voids line” can be a very useful addition. The air voids line slope and y-intercept is determined by the soil particle density (See graph 3.2-A). It is important to note that it curves. The soil particle density may have been determined experimentally or as approximated via the ‘Note’ under clause 6(d) of AS 1289.5.1.1. Soil particle density = 1 / [ {(100 x (1 - (A/100))) - (B x C)} / (B*100) ] A = 0% Air Voids B = Dry density of the wettest compaction point C = Moisture at wettest compaction point plus 1% The test method note indicates that a 2% void line when plotted using this particle density should lie close to the compaction curve produced.



Graph 3.2-A

Void lines can be plotted at various amounts of entrapped air. See graph 3.2-B. Often 0 %, 2% or 5 % air void lines are useful. The ‘wet leg’ of a compaction curve should run approximately parallel to the 0 % air void line. The compaction curve plotted must also lie to the left of the 0% air void line. The dry leg should match the wet leg in slope (i.e. match the voids line). Compaction curves not corresponding to this should be investigated. Many laboratories plot one or more air void lines using an assumed particle density. This may not convey the information needed to fully interpret the plotted results.

Graph 3.2-B

1.40

1.50

1.60

1.70

1.80

1.90

2.00

2.10

2.20

2.30

2.40

2.50

2.60

2.70

2.80

2.90

3.00

0.0 2.0 4.0 6.0 8.0 10.0 12.0 14.0

Dry

Den

sit

y (

t/m

3)

Moisture Content (%)

Zero Air Voids

Soil particle density 2.8

Soil particle density 2.6

1.40

1.50

1.60

1.70

1.80

1.90

2.00

2.10

2.20

2.30

2.40

2.50

2.60

2.70

2.80

2.90

3.00

0.0 2.0 4.0 6.0 8.0 10.0 12.0 14.0

Dry

Den

sit

y (

t/m

3)


Air Voids - Soil Partcle Density of 2.60

0 % Air Voids

2 % Air Voids

5 % Air Voids



Soil particle densities lie generally between 2.6 and 2.8. The use of the equation as noted in the test method gives a more meaningful 2 % air void line. It is recommended that laboratories should consider showing air void lines. The air void line should be identified, and the particle density used indicated. As to the graphs ‘fit-for-purpose’ it is clear many of the submitted graphs could be improved but it is up to laboratories to determine what best suits both their needs and those of their clients. Many participants did not show any void line(s) on the graph supplied. Participants are encouraged to consider adding these in the future.



Graph 3.2C

1.700

1.720

1.740

1.760

1.780

1.800

1.820

1.840

1.860

1.880

1.900

10.0 10.5 11.0 11.5 12.0 12.5 13.0 13.5 14.0 14.5 15.0 15.5 16.0 16.5 17.0 17.5 18.0 18.5 19.0 19.5 20.0

Dry

Den

sit

y (

t/m

3)


Part B Data - OMC 14.5 & MDD 1.841

Spline Estimate

Raw Data

Air voids 0%

2%voids

Estimated particle density 2.65 t/m3



3.3 Test Method There are various road authority test methods for this test in use. The following comments however are related only to the AS 1141.5.1.1 method. Often as part of a proficiency testing programs there is a need to discuss aspects of the test that can be identified as contributing to the overall variation in the test results produced. It does not mean the test method needs to change only that it is important for laboratories to know which aspects of the test, if not performed well, could add to the variability of the outcome. The need to change the test method only arises if the accuracy and variability in the test results is not within the expected range. One aspect of the test that contributes significantly to the variation in test results is the graphing approach used and moisture points selected for compaction. With more laboratories using mathematical approaches to determine the OMC/MDD the variation has reduced compared to using hand drawn graphs.

Year (Program)

Statistic – CV (%)

MDD OMC

t/m3 %

2011(23) 1.4 3.4

2012(33) 0.8 8.3

2013(42) 0.9 3.9

2014(51) 0.7 4.4

2015(62) 0.8 3.3

2016(69) 0.9 2.6

2017(76) 1.0 1.7

Table 3.3A Summary Statistics showing improvement as hand drawn graphed results are

replaced with computer calculated results.

As detailed in table 3.3A the effect in the reduction in the variation shown due to increased use of electronic determination of OMC/MDD compared to hand drawn is most marked with the OMC determination. This is a general overview as other factors such as material tested, number of participants and competency may also have an effect. There are aspects of the test method that could be improved to further reduce the variation observed between laboratories. Even though test results are rounded (i.e. OMC to 0.5 and MDD to 0.01) it is still important to test as accurately as possible. In doing so, despite the rounding employed it will lead to better reproducibility between laboratories. Where the results are used elsewhere such as in the CBR test such improvements are important. Better definition of mathematical approach to be used and specifying where data must fall on the curve would decrease testing variation between laboratories.



Currently the test method allows the following: 1. Two points on the dry side of OMC/MDD and two on the wet side.

2. Three points on the dry side of OMC/MDD and one on the wet side.

3. Two points on the dry side of OMC/MDD, one at the apex and one on the

wet side.

Approach 1 above is optimum and is used in part B of the program. The other approaches (2 & 3) are more likely to show more variation because any error with the single point on the apex or wet leg has a dramatic effect on the curve and therefore the OMC determined. There is also a strong possibility that if different mathematics is used to fit a regression curve to the data then there could be a variation depending on the approach used. It appears in some case that computer programs may alter the curve, so it runs parallel with the voids line. Rounding

Rounding can affect the variation observed in some programs. For example, Part A results were rounded, and the z-scores recalculated with the following variation (table 3.3B). The OMC determined is more affected than the MDD determined.

Statistic

Part A Part A - Rounded

MDD OMC MDD OMC

t/m3 % t/m3 %

Number of participants 30 30 30 30

Median 2.093 8.6 2.090 8.5

Normalized IQR 0.022 0.15 0.020 0.38

CV (%) 1.0 1.7 1.0 4.5

Min* 2.055 8.2 2.06 8.0

Max* 2.114 8.9 2.11 9.0

Range* 0.059 0.7 0.05 1.0

* Minimum, Maximum and Range are calculated with outliers excluded.

Table 3.3B Summary statistics comparing rounded and unrounded test results



A2 2.091 -0.07 E3 2.109 0.75

P4 2.097 0.21 F7 2.065 -1.26

C3 L8 2.097 0.21

A8 2.098 0.25

C6 2.097 0.21

B3 2.066 -1.21

D9 2.066 -1.21

K2 2.079 -0.62

C7 2.110 0.80

Z5 2.098 0.25

X8 2.092 -0.02

B5 2.090 -0.11

J8 2.091 -0.07

G7 2.114 0.98

Z6 1.904 -8.62 #

V4 2.114 0.98

P2 2.073 -0.89

E9 2.112 0.89

M7 2.092 -0.02

P6 2.055 -1.71

W7 2.091 -0.07

K3 2.114 0.98

Q4 2.075 -0.80

R9 2.055 -1.71

U8 2.094 0.07

X3 2.108 0.71

L4 2.110 0.80

F9 2.093 0.02

Number of results 30

Median 2.093

Median MU 0.005

First Quartile 2.076

Third Quartile 2.106

IQR 0.030

Normalised IQR 0.022

CV (%) 1.0

Minimum 2.055 (1.904)

Maximum 2.114 (2.114)

Range 0.059 (0.210)

Note: A # indicates an outlier where the z-score obtained is either greater then

3 or less than -3. Codes for all participates are shown. The results column

shows a blank entry for those participants that did not submit a result for this

test. Minimum, Maximum and Range are calculated with outliers excluded,

those in brackets include outliers. Particpants results that have been corrected

are shown in green.

4.1 Part A - MDD: Z - Scores

Code

Test

Result

t/m3

Z Score Code

Test

Result

t/m3

Z Score

Statistic Value



ReviewWeak

Consensus

Weak

ConsensusReview

Z-score

Strong Consensus

4.1 Part A - MDD: Z - Score Graph

G7

V4

K3

E9

C7

L4

E3

X3

A8

Z5

P4

C6

L8

U8

F9

X8

M7

A2

J8

W7

B5

K2

Q4

P2

B3

D9

F7

P6

R9

-3 -2 -1 0 1 2 3

Z6



A2 8.6 0.34 E3 8.1 -3.04 #

P4 8.8 1.69 F7 8.8 1.69

C3 L8 8.2 -2.36

A8 8.0 -3.71 #

C6 8.5 -0.34

B3 8.6 0.34

D9 8.7 1.01

K2 8.6 0.34

C7 8.6 0.34

Z5 9.0 3.04 #

X8 8.6 0.34

B5 8.4 -1.01

J8 8.3 -1.69

G7 8.0 -3.71 #

Z6 8.5 -0.34

V4 8.5 -0.34

P2 8.4 -1.01

E9 8.5 -0.34

M7 8.5 -0.34

P6 9.4 5.73 #

W7 8.6 0.34

K3 7.6 -6.41 #

Q4 8.0 -3.71 #

R9 8.9 2.36

U8 8.6 0.34

X3 8.6 0.34

L4 8.5 -0.34

F9 8.6 0.34


Median 8.6

Median MU 0.03

First Quartile 8.4

Third Quartile 8.6

IQR 0.20

Normalised IQR 0.15

CV (%) 1.7

Minimum 8.2 (7.6)

Maximum 8.9 (9.4)

Range 0.7 (1.8)






are shown in green.

4.2 Part A - OMC: Z - Scores

CodeTest

Result %

Z Score CodeTest

Result %

Z Score

Statistic Value



ReviewWeak

Consensus

Weak

ConsensusReview

Z-score

Strong Consensus

4.2 Part A - OMC: Z - Score Graph

P6

Z5

R9

P4

F7

D9

A2

B3

K2

C7

X8

W7

U8

X3

F9

C6

Z6

V4

E9

M7

L4

B5

P2

J8

L8

E3

A8

G7

Q4

K3

-3 -2 -1 0 1 2 3



A2 1.844 2.70 E3 1.842 0.00

P4 1.841 -1.35 F7 1.842 0.00

C3 L8 1.842 0.00

A8 1.839 -4.05 #

C6 1.841 -1.35

B3 1.845 4.05 #

D9 1.839 -4.05 #

K2 1.842 0.00

C7 1.843 1.35

Z5 1.841 -1.35

X8 1.842 0.00

B5 1.840 -2.70

J8 1.841 -1.35

G7 1.842 0.00

Z6 1.842 0.00

V4 1.843 1.35

P2 1.846 5.40 #

E9 1.842 0.00

M7 1.842 0.00

P6 1.841 -1.35

W7 NR

K3 NR

Q4 NR

R9 NR

U8 1.839 -4.05 #

X3 1.841 -1.35

L4 1.841 -1.35

F9


Median 1.842

Median MU 0.000

First Quartile 1.841

Third Quartile 1.842

IQR 0.0010

Normalised IQR 0.0007

CV (%) 0.04

Minimum 1.840 (1.839)

Maximum 1.844 (1.846)

Range 0.004 (0.007)






are shown in green.

4.3 Part B - MDD: Z - Scores

Code

Test

Result

t/m3

Z Score Code

Test

Result

t/m3

Z Score

Statistic Value



ReviewWeak

Consensus

Weak

ConsensusReview

Z-score

Strong Consensus

4.3 Part B - MDD: Z - Score Graph

P2

B3

A2

C7

V4

K2

X8

G7

Z6

E9

M7

E3

F7

L8

P4

C6

Z5

J8

P6

X3

L4

B5

A8

D9

U8

-3 -2 -1 0 1 2 3



A2 14.4 0.00 E3 14.4 0.00

P4 14.5 1.35 F7 14.4 0.00

C3 NR L8 14.4 0.00

A8 14.6 2.70

C6 14.4 0.00

B3 14.3 -1.35

D9 14.7 4.05 #

K2 14.4 0.00

C7 14.2 -2.70

Z5 14.4 0.00

X8 14.5 1.35

B5 14.5 1.35

J8 14.4 0.00

G7 14.4 0.00

Z6 14.4 0.00

V4 14.1 -4.05 #

P2 14.3 -1.35

E9 14.3 -1.35

M7 14.5 1.35

P6 14.2 -2.70

W7 NR

K3 NR

Q4 NR

R9 NR

U8 14.8 5.40 #

X3 14.4 0.00

L4 14.4 0.00

F9 NR


Median 14.4

Median MU 0.02

First Quartile 14.4

Third Quartile 14.5

IQR 0.1

Normalised IQR 0.07

CV (%) 0.5

Minimum 14.2 (14.1)

Maximum 14.6 (14.8)

Range 0.4 (0.7)






are shown in green.

4.4 Part B - OMC: Z - Scores

Code

Test

Result

%

Z Score CodeTest

Result %Z Score

Statistic Value



ReviewWeak

Consensus

Weak

ConsensusReview

Z-score

Strong Consensus

4.4 Part B - OMC: Z - Score Graph

U8

D9

A8

P4

X8

B5

M7

A2

C6

K2

Z5

J8

G7

Z6

X3

L4

E3

F7

L8

B3

P2

E9

C7

P6

V4

-3 -2 -1 0 1 2 3



5. Program Information

5.1 Z-score summary The proficiency program was conducted in November 2017. A ‘Z-score Summary’ summary was issued on the 13 January 2018. A copy was e-mailed to all participants who submitted results. The summary is intended as an early indicator of participant performance. This program report supersedes the z - score summary. Further information can be found in section 5.9 ‘Statistics’. The z-scores generally do not vary significantly between the “summary” and the “Final Report”. An additional set of results for two participants have been included but this has not significantly affect the statistics or performance outcomes.

5.2 Program design 5.2.1 Design

Part of the design of each program involves asking for the right information. The correct analysis of the data collected then allows feedback to be offered to enable participants to improve in the performance of this test. In designing a proficiency program, it is sometimes necessary to minimise the effect of some inherent test method variability. E.g. By providing the OMC and MDD value to be used for a CBR program. At other times the use of different materials gives a better indication of performance than using a single “easy” material. Test methods often have more than one area where skill and technique have an effect. Where possible a proficiency testing program may be designed to test different aspects of the method. In this program the overall performance of the test is covered by ‘Part A’. The skill associated with graphing and data interpretation has been separated out by giving the same test results to all participants in ‘Part B’. This also gives a measure of the variably associated with just this part of the test method. The program was designed to provide technical feedback regarding performance as well as possible improvements in performance. Other considerations involving the design of the program are detailed below. 5.2.2 Selection of material used in the program

The test in this proficiency program is operator skill/experience dependant. In addition, certain types of soils require more knowledge to obtain consistent results than others. Different materials are selected for each program to mirror the range of materials encountered in practice. This program provides a sample that gives results in the range that would be commonly tested by laboratories.



It is expected that the level of experience/skill need to perform these tests will present a reasonable assessment of the overall competency of the tester and industry performance. 5.2.3 OMC & MDD

Sample B was reference data chosen as a set of typical test results. Its purpose was to establish how much the graphing and measurement form the graph of OMC & MDD contributed to the overall variation of the test. 5.2.4 Role of proficiency testing

The determination of outliers is an important task of this proficiency program. A secondary function is to provide feedback that can help those with outliers identify possible areas to investigate as well as assist all participants to improve. In addition to the statistics, proficiency programs often obtain other information that is not normally available to a laboratory. It allows for a better understanding of the testing and can provide information that can lead to improvements in the testing process or test method. Proficiency testing enables participants to measure competency against others. It is also a measure of staff performance and the equipment used. Apart from ‘measurement uncertainty’ it is the most useful tool a laboratory has in better understanding the performance of a test. 5.2.5 Participant assessment

Assessment of each participant is based on a z-score that is related to the program consensus value (median). This is used to determine any statistical outliers. Compliance to proficiency program requirements including the correct calculation of results and adherence to program and test method requirements may also be used as part of the assessment process. Participants may also be asked to investigate any discrepancies detected with the paperwork submitted. 5.2.6 Reporting of results - Significant figures

The number of decimal places (significant figures) reported for a test has a bearing on the statistical analysis and therefore the interpretation of the results. There is a need to strike a balance between what is desirable from a statistical viewpoint while recognising how the results are used in practice. Too few decimal places (e.g. due to rounding) can cause an increase in the observed spread of results. Increasing the number of decimal places (with respect to normal reporting) can distort the observed spread of results compared to that encountered in actual practice. Large numbers of similar, rounded results can also cause a distortion in the analysis. For example, rounding to 0.5 % means that any number between 10.75 and 11.25 will be 11.0%. If the largest value is 10.75 in a set of results it is pushed out to 11.0 through rounding. Rounded results are useful from “an end user” perspective but are not as useful when considering laboratory performance.



The test method acknowledges additional decimal places may be used for statistical purposes. For this program, it was decided that the benefits of using additional decimal places would complement the aim of the proficiency program. Participants results were analysed as received regardless of whether there were more or less significant figures used other than the number requested by the program. 5.2.7 Additional information requested

This program requested additional information as detailed in Appendix C not usually reported. The additional information is however consistent with the performance of the test and the records the test method requires laboratories to maintain. The additional information is used to interpret participant’s performance and assist with providing technical comment including feedback on outliers and possible participant improvement. 5.2.8 Data checks

As often observed ‘operator errors’ can occur in the result calculation process. Not every participant’s results were verified as reasonable only those with outliers. Spacing of moisture steps was checked as per the test method. Checks however are only as accurate as the raw data supplied by each participant. These checks also help ensure that the data is comparable. Any inconsistencies identified during this process are identified as possible feedback for participant improvement. In some cases, inconsistences identified may need to be investigated by participants. 5.3 Sample Preparation Samples for the program were drawn and packaged from a single, well mixed, lot. Samples were laid out in the order prepared. Ten samples were selected at approximately equal intervals from the set of samples. These were used for homogeneity testing. Each participant received randomly drawn samples from the remainder. A unique participation code was assigned to each sample. Each sample was placed in a plastic bag, sealed, labelled with the program name and packed into a sturdy box prior to dispatch. 5.4 Packaging and Instructions Each participant received one sealed plastic bag, marked ‘Soil Compaction Proficiency Program – 2017 (76) Sample’, containing approximately 12.5kg of soil. Participants were instructed to test per the nominated test method and report to the accuracy indicated on the ‘results log’ sheet. See ‘Appendix A’ for a copy of the instructions issued to participants and ‘Appendix B’ for the log sheet supplied.



5.5 Quarantine Samples sent to Western Australia are subject to quarantine regulations that require treatment of the soil prior to importation. Samples sent to WA are heat treated and compliance certificates enclosed with samples. Where applicable further instructions regarding preparation or handling of the sample may be included. 5.6 Sample dispatch Samples were dispatched to participants on the 2 November 2017 using Toll Priority. Dispatched samples are tracked from despatch to delivery to each participant by LabSmart Services.

Sample Code MDD t/m3

OMC %

H1 2.081 -

H2 2.094 8.4

H3 2.093 8.6

H4 2.087 8.8

H5 2.069 8.5

H6 2.074 -

H7 2.073 8.7

H8 2.076 8.4

H9 2.090 8.8

H10 2.061 -

Average 2.080 8.6

Standard Deviation 0.011 0.17

Minimum 2.061 8.4

Maximum 2.094 8.8

Range 0.033 0.4

Coefficient of Variation (%) 0.5 2.0

Table 5.7 Homogeneity Results

5.7 Homogeneity Testing Samples for homogeneity (Sample A) testing were packed in the same way as those for participants. Ten samples were selected at approximately equal intervals throughout the set of samples. To approximate the same conditions as participants the same instructions were given to the laboratory performing the homogeneity testing.



The homogeneity data was reviewed with the MDD indicating a small variation indicative of the sample being homogenous. The OMC however showed a large variation. This was investigated further and found to be because of the points selected to derive the curve and possibly the curve fitting technique used. Suspect results had only one point on the wet leg. The OMC value for these three sample was rejected from the pool of results. The effects are discussed in section 3.3, Test Method. The average of the homogeneity samples lies within 1 s.d of the participant’s median value. The homogeneity results also meets the precision requirements indicated in the test method. Overall this provides confidence that any outliers identified in the program represent statistically valid outliers. 5.8 Participation Thirty-one participants entered the program with a total of 30 results received. The nominated date for participants to return their results was 24 November 2017. There was one participant unable to return their results in time for inclusion in the final report. The z-score summary included 28 participants results while the final program report included 30 participants results.

5.9 Statistics Z-Scores were calculated for each test and used to assess the variability of each participant relative to the consensus median. A corresponding z-score graph was produced for each test. The use of median and quartiles reduces the effect that outliers have on the statistics and other influences. Therefore, z-scores provide a more realistic or robust method of assessment. Some results were reported by participants to more decimal places than requested as part of the proficiency program and by others to fewer decimal places. In all instances test results have been used as submitted by participants. Assessment of participant’s data is undertaken to ensure data is statistically comparable. Checks are undertaken to ensure the data calculated matches that reported by the participant and that the appropriate corrections etc. have been applied if required. The level of checking required varies from program to program. If inconsistencies are identified the data may be removed or amended with the discrepancy highlighted.

A z-score is one way of measuring the degree of consensus with respect to the grouped test results. The z-scores in this report approximate standard deviations. For each test a z-score graph is shown. Use the graph to visually check statistically how you compare to other participants. The following bar (Figure 5.2) is shown at the bottom of each graph. This helps to quickly visualize where each participant’s result falls.



Review Weak

Consensus Strong Consensus

Weak Consensus

Review

Figure 5.2 Z-score interpretation bar

For example:

• A strong consensus (i.e. agreement) means that your test result is close i.e. within 1 standard deviation of the median.

• A weak consensus means that your test result is satisfactory and is within 2 standard deviations of the median.

• If you have obtained a test result that is outside 2 standard deviations, then it

may be worth reviewing your testing processes to ensure that all aspects are satisfactory. Only those obtaining a z-score approaching 3 (I.e. outside 2.75 range) have been highlighted in the report for review.

If you have obtained a test result that is outside 3 standard deviations then you will need to investigate your testing processes to ensure that all aspects are satisfactory. Participant assessment is not based purely on statistical analysis. Compliance to proficiency program requirements including the correct calculation of results and adherence to program requirements may also be used as part of the assessment process. Participants may also be asked to investigate any discrepancies detected with the paperwork submitted. For further details on the statistics used in this proficiency program can be obtained from LabSmart Services or download the ‘Participant Guide’ from the LabSmart Services website. 5.9.1 Z-score summary

A “Z-Scores Summary” is issued soon after most results are received. It gives participants early feedback as to any program outliers. The summary is usually available on the LabSmart Services website up until the final report is issued. The final report supersedes the z-score summary. The final report contains detailed technical feedback regarding the performance of tests and revised z-scores. The inclusion of late results or corrections are at the discretion of the program coordinator. In some instances, this may change some of the z-scores slightly but generally the performance outcome remains the same. If there is any impact it will be discussed within section 5.1 of the report. 5.9.2 Comparing statistics from one program to another

The statistics generated from one proficiency program are not usually comparable to those from another proficiency testing program. Only very general comparisons may be possible. The reason statistics from one program may not be compared to another is due to the range of variables that differ from one proficiency program to another.



These variables include:

• Type of material selected,

• The number of participants,

• Experience of participants,

• Test methodology variations,

• Equipment used,

• Test methods used,

• Experience of supervisors,

• Range of organisations involved.

• Program design and the statistics employed. The program outcome represents a ‘snap shot’ of the competency within the industry and hence provides an overview of the industry. The more participants involved in the program then the more representative the overview. 5.9.3 Measurement uncertainty

The statistics detailed in this program do not replace the need for laboratories to separately calculated measurement uncertainties associated with each test when required by the client or NATA. The proficiency program does give information useful for calculating the MU and bench marking the MU calculated. 5.9.4 Metrological traceability

The assigned median value used in this proficiency testing program is derived from participant performance and is not metrologically traceable. 5.10 Non-statistical Outliers One of the issues faced by proficiency testing providers is what to do with an incorrect result even if its z-score is satisfactory. In many cases they cannot be detected but still can have a significant impact on the statistics calculated. This can cause biased (or unfair) outcomes for other participants. To limit the effect that erroneous results may have on a program additional information is requested to allow the main results to be recalculated. In some cases, results shown to be erroneous may be reject for inclusion in the program. If the result does not add any statistical bias it is left in the program. The result however is incorrect even though it may have a satisfactory z-score. To highlight that the participant needs to investigate erroneous results it is considered a ‘non-statistical outlier’. This may also be applied to non-compliance to program requirements e.g. incorrect reporting of results etc or incorrect partial calculations/data. Non-statistical outliers were not used as part of the assessment process for this program.



76 Appendix A PT Instructions V4 Compaction 2017.docx

LabSmart Services

Soil Compaction Proficiency Testing Program – 2017(76)

INSTRUCTIONS FOR TESTERS

1. Please check that the package you have received contains the following:

• Instructions (for tester)

• Results Log

• Approximately 12.5 kg soil sample sealed in a plastic bag.

Contact Peter on 0432 767 706 if the bag is damaged or any item is missing.

2. Please read and follow these instructions carefully. Examine the results log sheets. There is a Part A and a Part B to the proficiency program.

3. Please do not discuss aspects of this program with other organisations or other staff

within your own organisation who may also be testing a sample from this program. Confidentiality is important in order to ensure statistically valid measures of participant performance.

PART A

4. Due to segregation during transit please ensure the sample is thoroughly mixed prior

to starting. Ensure moisture is uniform throughout sample.

5. AS 1289.5.1.1, 4 points is the preferred test method.

6. You may perform the test even if you are not NATA accredited for the test. The use of mechanical compaction is allowed.

7. Sieve the sample over the 19.0 mm sieve. Discard any material retained on the 19.0

mm sieve. Thoroughly mix the material passing the 19.0 mm sieve and use for the compaction test. Check if material needs to dry back before commencing test.

8. Record all information and calculations as per the proficiency testing results log sheet

and to the accuracy shown on the results log sheet. A greater reporting accuracy is required compared to that nominated by the standard. Please forward graph used.

PART B 9. Use the moisture/dry density data shown under ‘Part B’ of the results log sheet. 10. Calculate the OMC and MDD for this data set using the same procedure as used in

‘Part A’ if possible. Enter your results onto ‘Part B’ of the results log. Please forward graph used.

11. If you are unable to use the same procedure as used in ‘Part A’ then use an alternative

method (e.g. manually graph etc).

Please fax or e-mail the “Results Log” to LabSmart Services by 24 November 2017

Fax: (03) 8888 4987 OR E-mail: [email protected]

Thank you for participating in this proficiency program. Page 1 of 1

Appendix A



mailto:[email protected]

76 Appendix B - Results Log V3 Compaction 2017.docx

LabSmart Services

Soil Compaction Proficiency Testing Program – 2017 (76)

RESULTS LOG for XXXXXXXX

Participation Code: XX

Please fax (03) 8888 4987 or e-mail ([email protected]) the completed results log by:

24 November 2017

PART A – Please attach graph

Date sample received:

Condition of sample received:

Tested by:

Name of Laboratory:

Test Report to: Result Method

Recommended Tick or enter method used

Standard Maximum Dry Density (MDD) Nearest 0.001 t/m3 AS 1289.5.1.1

Standard Optimum Moisture Content (OMC) Nearest 0.1 % AS 1289.5.1.1

Method used to determine moisture AS 1289.2.1.1

Data used to construct curve Dry Density Moisture

Report to Nearest 0.001 t/m3 Report to Nearest 0.1 %

Sample 1

Sample 2

Sample 3

Sample 4

Additional test details Report Result

Curing time used Nearest ½ hr

Has a mechanical compactor been used? (Yes or No)

Mould size used A or B

Number of blows used per compaction layer Blows

Page 1 of 2

Appendix B



mailto:[email protected]

76 Appendix B - Results Log V3 Compaction 2017.docx

PART B – Please attach graph

Data used to construct curve Dry Density Moisture

Sample 1 1.785 11.1

Sample 2 1.824 12.8

Sample 3 1.838 15.0

Sample 4 1.801 16.5

NOTE: If you are only able to use a three points method then use data for samples 1, 2 and 4 only.

Test Report Result

Standard Maximum Dry Density (MDD) Nearest 0.001 t/m3

Standard Optimum Moisture Content (OMC) Nearest 0.1 %

Method of determining OMC & MDD same as Part A?

(Yes or No)

COMMENTS:…………………………………………..............…………………… ………………………………...........................………..........…...……………………………………………………...........................………..........…...……………… …………………………………...........................………..........…...………………

------------------------------------ ---------------------------------- --------------- Supervisor Name (Please Print) Signature Date

In signing the above I acknowledge that the above results are approved and have been checked. I will also ensure that the results are kept confidential both internal and external to the laboratory until the final technical report covering this program has been issued.

Note:

Please retain the completed “Results Log” as this contains your participation code that will identify your results in the technical report covering the proficiency testing program. It is also recommended that a copy of completed worksheets be kept with the results log in your proficiency file.

Thank you for participating ________________________________________________________________________

Have a query? Contact Peter Young at LabSmart Services. Mobile: 0432 767 706

Page 2 of 2



A2 AS 1289.5.1.1 AS 1289.2.1.1 72 No A Yes

P4 AS 1289.5.1.1 AS 1289.2.1.1 113 No A Yes

C3 NR NR NR NR NR NR

A8 NR NR 2.5 No A Yes

C6 AS 1289.5.1.1 AS 1289.2.1.1 20 No A Yes

B3 AS 1289.5.1.1 AS 1289.2.1.1 17 No A Yes

D9 AS 1289.5.1.1 AS 1289.2.1.1 6 No A Yes

K2 AS 1289.5.1.1 AS 1289.2.1.1 22 No A Yes

C7 AS 1289.5.1.1 AS 1289.2.1.1 53 No A Yes

Z5 AS 1289.5.1.1 AS 1289.2.1.1 2 No A Yes

X8 AS 1289.5.1.1 AS 1289.2.1.1 38 No A Yes

B5 AS 1289.5.1.1 NR 4.0 No A Yes

J8 AS 1289.5.1.1 AS 1289.2.1.1 77 No A Yes

G7 AS 1289.5.1.1 AS 1289.2.1.1 5 No A Yes

Z6 AS 1289.5.1.1 AS 1289.2.1.1 48.5 No A NR

V4 AS 1289.5.1.1 AS 1289.2.1.1 19 No A Yes

P2 AS 1289.5.1.1 AS 1289.2.1.1 48 No A Yes

E9 AS 1289.5.1.1 AS 1289.2.1.1 97 No A Yes

M7 AS 1289.5.1.1 AS 1289.2.1.1 2.5 No A Yes

P6 AS 1289.5.1.1 AS 1289.2.1.1 NR No A Yes

W7 AS 1289.5.1.1 AS 1289.2.1.1 2 No A NR

K3 AS 1289.5.1.1 NR 2.5 No A NR

Q4 NR NR 2 No A NR

R9 AS 1289.5.1.1 NR 2.5 No A NR

U8 AS 1289.5.1.1 AS 1289.2.1.1 24 No A Yes

X3 AS 1289.5.1.1 AS 1289.2.1.1 2.5 No A Yes

L4 AS 1289.5.1.1 AS 1289.2.1.1 5 No A Yes

F9 NR NR NR NR NR NR

E3 AS 1289.5.1.1 AS 1289.2.1.1 2 No A Yes

F7 AS 1289.5.1.1 AS 1289.2.1.1 2 No A Yes

L8 AS 1289.5.1.1 AS 1289.2.1.1 2 No A Yes

Appendix C - Particpant Supplied Test Information

Code MDD Method MC Method

Cure

Time

(hrs)

Mechanical

Compaction

Mould

Size

Used

Graph method

same for Part

A & B



soil compaction - 2017 (76) proficiency testing program report · report this report is available...

Documents