labsmart report compaction 2020

Compaction Proficiency Testing Program – 2020 (94)

Copyright: LabSmart Services Issue - 27 October 2020 of 40

www.labsmartservices.com.au

Soil Compaction – 2020 (94)

Proficiency Testing Program Report

Accredited for compliance with ISO/IEC 17043

Copyright: LabSmart Services.



Report

This report is available on the LabSmart Services website. The issue of this proficiency report was authorised by Jeffrey Mulholland, General Manager, LabSmart Services, October 2020. Contact Details

Email: [email protected] Mobile: 0439 208 406

Program Coordinator

The program coordinator for this program was Jeffrey Mulholland, LabSmart Services. Contact Details

Email: [email protected] Mobile: 0439 208 406

Please note that any technical questions regarding this program are to be directed to the program coordinator. Z-scores Summary

A z-scores summary for this program was issued on the 4th of August 2020. This technical report supersedes the z-sores summary. Accredited Proficiency Testing Provider

LabSmart Services is accredited by NATA to ISO/IEC 17043, Conformity assessment – General requirements for proficiency testing. Accreditation number 20650. The accreditation provides additional assurance to participants of the quality and importance we place on our proficiency testing programs. LabSmart Services

Please see our website for further details.

www.labsmartservices.com.au Copyright

This work is copyright. No part of this publication may be reproduced in any form, transmitted or stored in any repository (e.g. mechanical, digital, electronic or photographic) without prior written permission of LabSmart Services. Please contact LabSmart Services should you wish to reproduce any part of this report. Amendment History

Reports may be downloaded from the LabSmart Services website. Version 1 – Issued 27 October 2020



Contents 1. Program Aim .......................................................................................................................... 4

2. Performance .......................................................................................................................... 5

2.1. Identified Outliers ............................................................................................................. 5

2.2. Program Summary ........................................................................................................... 6

3. Technical Comment ............................................................................................................... 8

3.1. Part A ............................................................................................................................... 8

3.2. Part B ............................................................................................................................. 11

3.3. Test Method ................................................................................................................... 17

4. Statistics: Z-Score & Graph ................................................................................................. 20

5. Program Information ............................................................................................................ 28

5.1. Z-score summary ........................................................................................................... 28

5.2. Program Design ............................................................................................................. 28

5.4. Packaging and Instructions ............................................................................................ 31

5.5. Quarantine ..................................................................................................................... 31

5.6. Sample dispatch ............................................................................................................ 32

5.7. Homogeneity Testing ..................................................................................................... 33

5.8. Participation ................................................................................................................... 33

5.9. Statistics ........................................................................................................................ 33

5.10. Non-statistical Matters ................................................................................................. 35

Appendix A: Instructions for testers ......................................................................................... 37

Appendix B: Results Log ......................................................................................................... 38

Appendix C: Participant Supplied Test Information ................................................................. 40



1. Program Aim

The proficiency testing program was conducted over June/July 2020 with participants throughout Australia. The program involved the performance of the following tests.

Standard Maximum Dry Density (MDD) Standard Optimum Moisture Content (OMC)

AS 1289.5.1.1 (2017) was the preferred testing method, but other equivalent methods were accepted.

Each participant’s performance is statistically assessed and used as a measure of their competency relative to all other Participants. The program provides feedback and confidence to participants and the industry regarding the competency of laboratories to perform these tests.

Details relating to the design and conduct of the program may be found under Section 5.



2. Performance

2.1. Identified Outliers

Overall a satisfactory level of testing was achieved by 88% of the participants returning results.

There were 6 participants identified as having one or more outliers (12%).

Participant’s test results are tabulated in section 4 along with the robust statistics and a z-score graph. The z-score indicates how far away a participant is from the program’s median value. A z-score of zero indicates a strong consensus with respect to all other participants and represents a very good outcome. The z-score graph gives a quick visual indication of how a result compares to others in the program.

Outliers are classified as z-scores where the z-score value was greater than 3 or less than -3. It is recommended that participants with outliers investigate their performance of the test. Participants with outliers are detailed in Table 1.

Those participants with z-scores greater than 2 or less than -2 may wish to review their testing methodology. Only those approaching 3 (outside ± 2.75) have been specifically identified in Table 1 as feedback.

More detail on the robust statistics used can be found in section 5.

Technical comment and feedback in section 3 is provided to assist participants to investigate and review their results, as well for those seeking to improve their testing performance.



Table 1: Identified outliers

Proficiency Component Test Investigate Review*

Part A Soil Sample

Maximum Dry Density (MDD) E9 V4, C2

Optimum Moisture Content (OMC) - -

Part B Reference

Data

Maximum Dry Density (MDD) K7, Q7, Z3, J5 -

Optimum Moisture Content (OMC) W5, K7, Q7, Z3, J5,

E9 -

* Those in brackets are not considered outliers but may wish to review their results (see section 3.2.1). Only z-scores greater than 2.75 and less than -2.75 have been identified in the above table.

2.2. Program Summary

There were 52 participants that applied for the proficiency testing program; of these 51 participants returned results in time to be included in the final report.

Of the 51 participants, 45 participants (88%) were found to have performed well in this program, With the spread of results in line with previous programs.

Laboratories need to ensure graphs are checked to ensure they are both accurate and appropriate before issuing results. Laboratories should consider plotting air voids on compaction graphs and check that air voids when plotted are meaningful. Attention to the spread of compaction points, where two on the dry leg and two on the wet leg is ideal, will result in greater accuracy of testing.

Table 2 summarises the results obtained. Normalized IQR values approximate standard deviations.



Table 2: Summary Statistics

Statistic Part A - Soil Sample Part B - Reference Data**

MDD OMC MDD OMC

t/m3 % t/m3 %

Number of participants

51 51 34 34

Median 2.020 11.7 1.771 17.1

Normalized IQR 0.016 0.70 0.0010 0.10

CV (%) 0.8 6.0 0.3 0.4

Min* 1.973 10.7 1.760 17.0

Max* 2.064 12.7 1.782 17.3

Range* 0.091 2.0 0.022 0.3

*Minimum, Maximum and Range are calculated with outliers excluded **Adjustments were made for Part B, See Section 3.2.1 for more details

The proficiency testing program indicates a satisfactory level of testing and is within industry expectations for most participants.

The proficiency program was a useful exercise, allowing participants to have greater confidence in their results while for others providing an opportunity to improve their competency with respect to the test method covered.



3. Technical Comment

3.1. Part A

3.1.1. Test Methodology

Participants were requested to provide additional details about the testing performed. These details were used to help analyse the proficiency program results. Also, the information can help with the investigation of outliers arising from the program. See Appendix C for participant responses.

Most participants nominated the test methods used for determining MDD and OMC as AS 1289.5.1.1 aside from Q7 and Z3 who nominated Q142A. All participants reported using AS 1289.2.1.1 for their determination of moisture. Additionally, all participants stated they used an ‘A’ sized mold, and most participants reported using hand compaction, with only four participants using a mechanical compactor.

3.1.2. Soil Curing

There was a broad range of curing times used by participants ranging from no curing (tested as received) to 360.5 hours (see Appendix C). The material needs to be cured in such a manner as to ensure moisture is homogeneous throughout. AS 1289.5.1.1 gives specific minimum curing times based on the sample’s liquid limit along with its deviation from OMC in its prepared condition.

3.1.3. Maximum Dry Density (MDD) & Optimum Moisture Content (OMC)

The following participants (Table 3) had outliers for ‘Part A’ of the program that needs to be investigated. Additionally, participants with Z-scores approaching a Z-score of 3 (greater than 2.75 or less than -2.75) should also review.

Table 3: Outliers for Part A

Test ‘Outlier’ Investigate Review

Maximum Dry Density (MDD) E9 C2, V4

Optimum Moisture Content (OMC) - -



Graph 1: Plot of participant MDD vs OMC results for ‘Part A’ with outliers shown

*The cross shows the variation at 1 s.d for MDD and OMC.

The above graph shows the supplied MDD vs OMC results for all the participants in Part A

For the outlier in ‘Part A’ the data supplied was re-graphed to check to see if plotting was a factor.

However, upon revising the additional information supplied by E9, there was an inconsistency with the data on the graph and the final result supplied (2.155t/m3). The participant only supplied a graph; however, a visual assessment of this graph shows a result of approximately 2.019t/m3, which would have had a strong consensus with the other participants. Participants should take to review all available data before finalising results.



3.1.4. Graphing

This proficiency program specifically requested graphs to be submitted. The graphing technique used was predominately via software. The quality of graphing has improved considerably over the last few programs. Many participants had very good graphs.

The test method indicates that a graph must be prepared to derive the OMC and MDD. Graphing the results is also the most practical approach for assessing the correct performance of this test and the reliability of the result obtained.

Unfortunately, the test method does not specifically define the derivation of the graph or other important aspects pertinent to the accuracy of this test. For example, it does not define when a result should be rejected.

Reviewing participant results, it did appear that if the single wet point was ‘out’ it could throw out the curve and OMC/MDD obtained. If the same point was used to estimate the voids line, then this too could be thrown ‘out’.

Detailed feedback on graphing has been provided for past programs, but with many now using ‘software-based’ systems, the graphs provided are too small to review. However, laboratories may have the ability to enlarge graphs during the checking process. Graphs that are produced need to be reviewed as there is still an element of judgement involved and the chance that the software used has not produced the best outcome.

As a final note, some participants in this program have inadvertently changed the scale of the density and moisture axis used, giving a ‘better looking’ graph visually. Care should be exercised as this can be misleading. Those with outliers have raw data meeting the test method, but as explained in section 3.3, the choice of test points can lead to greater variation in results.

Ultimately, Graphs need to be ‘fit for purpose’. Many laboratories need to keep working towards this.

Comments on the graphs submitted for Part A and B are covered in section 3.2.2 under ‘Compaction Graph’.



3.2. Part B

Most participants used the same graphing technique for both Part A & B.

Part B of the program is meant to find gross departures from practice. It should be noted that the data used, while typical of test data is an ‘ideal data set’. That is, it has two points on both side of OMC/MDD, with moisture points evenly spaced. See section 3.3 for further discussion.

An interesting situation has arisen for the participants of this program and is the result of the industry moving towards the same/similar calculation methods. As a whole, this is seen as a positive. however, it should be noted there is an element of judgement involved when choosing calculation/graphing techniques and there is a chance that the process used by anyone participant to produce their graph has not provided the best outcome.

The issue relates to over 50% of the participants in this program reaching the same result for OMC (17.1%), this along with it being spread across the First and Third Quartile associated with Interquartile Range. This resulted in a IQR of 0 and effected our ability to use our normal statistical processes.

In place of implementing an alternative statistical process, it was decided the review all companies that had supplied results for multiple samples (across different laboratories/personnel) and where they had supplied the same result for both MDD and OMC, these submissions would be treated as one result.



3.2.1. Data analysis and graphing

Part B provided participants with a standard set of data and requested that the MDD & OMC be determined. The outcomes reflect only the variation associated with the graphing and interpretation component of the graph.

For this program, the variation observed is similar to previous proficiency testing programs, and although the variation is small (see table 2), it does contribute significantly to the overall variation experienced across laboratories. An example plot is shown in graph 4. It is unlikely that everyone would get the same result as laboratories use slightly different approaches. Nevertheless, it is expected that the results amongst participants should be close.

Additionally, the outcome should be put into perspective. The NIQR (approximated standard deviation) for MDD is ± 0.005t/m3 and ± 0.07% for OMC. Three standard deviations (seen as an acceptable range as you would not be an outlier) is only ± 0.015 t/m3 for MDD and ± 0.21% for OMC. This is a very small range for any one participant to fall within and can easily be affected by such things as rounding or graphing techniques.

The following comments may assist those with outliers for Part B to fine-tune their graphing process.

Overall, for Part B there were four participants (K7, Q7, Z3 & J5) that had statistical outliers for MDD and six participants (W5, K7 Q7, Z3, J5 & E9) that had statistical outliers for OMC.

W5 – the graph supplied shows a moisture of approximately 17.1%. however, the handwritten note on the page shows 17.5%. This result of 17.5% appears to have be caried on to the submitted result on the ‘log sheet’. Participant W5 should review this first.

K7 – No Graph provided; it appears that the results for Part A have been duplicated into the results for Part B.

Q7 – No Graph provided; it appears that the results for Part A have been duplicated into the results for Part B.



Z3 – No Graph provided; it appears that the results for Part A have been duplicated into the results for Part B.

J5 – the graph supplied strongly agrees with the outcome of the Z-scores for both MDD (1.771t/m3) and OMC (17.1%). However, it appears that the results for Part A have been duplicated into the results for Part B.

E9 – Visual assessment of the graph supplied shows that the second point (meant to be 1.756t/m3 @ 16.0%) was incorrectly entered at approximately 17.6% moisture, participant E9 should review this first.

It was noted that a few graphs submitted by participants did not display the data points (only showing the calculated curve). Displaying these points can aid in assessing if the graph drawn by the software has produced the best outcome.



3.2.2 Compaction Graph

The compaction graph gives a visualisation of the test results. It is useful as a quick means of determining how well the test has been performed. This is conveyed through the “fit” of the curve to the points and spacing of the compaction/moisture data points. An “air voids line” can be a very useful addition.

The air voids line slope and the y-intercept is determined by the soil particle density (See Graph 2). It is important to note that it curves. The soil particle density may have been determined experimentally or as approximated via the ‘Note’ under clause 6(d) of AS 1289.5.1.1.

Soil particle density = 1 / [ {(100 x (1 - (A/100))) - (B x C)} / (B*100) ]

A = 0% Air Voids

B = Dry density of the wettest compaction point

C = Moisture at wettest compaction point plus 1%

The test method note indicates that a 2% void line when plotted using this particle density should lie close to the compaction curve produced.

Graph 2: Zero Air Voids

1.401.501.601.701.801.902.002.102.202.302.402.502.602.702.802.903.00

0.0 2.0 4.0 6.0 8.0 10.0 12.0 14.0

Dry

Den

sity

(t/

m3)

Moisture Content (%)

Soil particle density 2.8

Soil particle density 2.6



Void lines can be plotted at various amounts of entrapped air (Graph 3). Often 0 %, 2% or 5 % air void lines are useful.

The ‘wet leg’ of a compaction curve should run approximately parallel to the 0 % air void line. The compaction curve plotted must also lie to the left of the 0% air void line. The wet leg of the plotted curve should match the wet leg in slope (i.e. match the voids line). Compaction curves not corresponding to this should be reviewed. However, the curve should pass through the ‘wet’ data point if hand plotted at the expense of running parallel to the void line.

Many laboratories plot one or more air void lines using an assumed particle density. This may not convey the information needed to interpret the plotted results fully.

Graph 3: Air Voids - Soil Particle Density of 2.60

Soil particle densities generally lie between 2.6 and 2.8. The use of the equation, as noted in the test method, gives a more meaningful 2 % air void line.

It is recommended that laboratories should consider showing air void lines. The air void line should be identified, and the particle density used indicated.

As to the graphs ‘fit-for-purpose’, it is clear many of the submitted graphs could be improved but it is up to laboratories to determine what best suits both their needs and those of their clients.

1.401.501.601.701.801.902.002.102.202.302.402.502.602.702.802.903.00

0.0 2.0 4.0 6.0 8.0 10.0 12.0 14.0

Dry

Den

sity

(t/

m3)

Moisture Content (%)

0 % Air Voids

2 % Air Voids

5 % Air Voids



Participants without any void line(s) on their graphs are encouraged to consider adding these in the future.

Graph 4: Part B Data – OMC 17.1 & MDD 1.771



3.3. Test Method

There are various road authority test methods for this test in use. The following comments, however, are related only to the AS 1289.5.1.1 method.

Often as part of a proficiency testing programs, there is a need to discuss aspects of the test that can be identified as contributing to the overall variation in the test results produced. It does not mean the test method needs to change, only that it is important for laboratories to know which aspects of the test, if not performed well, could add to the variability of the outcome.

The need to change the test method only arises if the accuracy and variability in the test results are not within the expected range.

One aspect of the test that contributes significantly to the variation in test results is the graphing approach used and moisture points selected for compaction. With more laboratories using mathematical approaches to determine the OMC/MDD, the variation has reduced compared to using hand-drawn graphs.

Even though test results are rounded (i.e. OMC to 0.5 and MDD to 0.01) it is still important to test as accurately as possible. In doing so, despite the rounding employed it will lead to better reproducibility between laboratories where the results are used elsewhere such as in the CBR test, such improvements are important.

Some of the results submitted for this program had ‘rogue data’ points that affect the OMC/MDD result obtained. There is no industry recognised approach to dealing with these.

A better definition of the mathematical approach to be used and specifying where data must fall on the curve would decrease testing variation between laboratories.

Currently, the test method allows the following:

Two points on the dry side of OMC/MDD and two on the wet side. Three points on the dry side of OMC/MDD and one on the wet side. Two points on the dry side of OMC/MDD, one at the apex and one on the wet side.



Approach 1 above is optimum and is used in part B of the program. The other approaches (2 & 3) are more likely to show more variation because any error with the single point on the apex or wet leg has a dramatic effect on the curve and therefore the OMC determined.

There is also a strong possibility that if different mathematics is used to fit a regression curve to the data, then there could be a variation depending on the approach used.

Rounding

Rounding can affect the variation observed in some programs i.e. 0.01 for MDD and 0.5 for OMC. For example, Part A results were rounded, and the z-scores recalculated with the following variation (Table 4). In this program, the effect was negligible.

Table 4 Summary statistics comparing rounded and unrounded test results

Statistic Part A Part A - Rounded

MDD OMC MDD OMC

t/m3 % t/m3 %

Number of participants 51 51 51 51

Median 2.020 11.7 2.02 11.5

Normalized IQR 0.016 0.70 0.015 0.74

CV (%) 0.8 6.0 0.7 6.4

Min* 1.973 10.7 1.980 10.5

Max* 2.064 12.7 2.060 12.5

Range* 0.094 2.0 0.90 2.0

* Minimum, Maximum and Range are calculated with outliers excluded.



This page has been left blank intentionally for formatting purposes



4. Statistics: Z-Score & Graph

t/m³ t/m³A2 2.020 0.00 X6 2.016 -0.25N9 2.020 0.00 E9 2.155 8.47 #W5 2.027 0.44 W2 2.044 1.51M8 J7 2.030 0.63C2 2.064 2.76 W8 2.011 -0.56U5 2.019 -0.06 Q3 1.995 -1.57D6 2.014 -0.38 Z5 2.040 1.25T5 2.000 -1.25 V4 1.973 -2.95K7 1.989 -1.95 N6 2.001 -1.19B4 2.037 1.07 L8 2.03 0.63N5 2.033 0.82 M7 2.02 0.00Q7 2.009 -0.69 L4 2.020 0.00Z3 2.017 -0.19 A7 2.021 0.06G7 2.003 -1.07 G8 2.026 0.38M6 2.021 0.06 L7 2.005 -0.94B9 2.041 1.32 F7 2.035 0.94P5 1.993 -1.69 X4 2.010 -0.63J8 2.008 -0.75 M5 2.021 0.06E2 2.054 2.13 L5 2.007 -0.82F8 2.018 -0.13 X2 2.017 -0.19C9 2.030 0.63 F4 1.994 -1.63Y6 2.024 0.25 A4 1.983 -2.32V5 2.010 -0.63 G6 2.037 1.07K3 2.022 0.13 S6 2.030 0.63W6 2.006 -0.88C7 2.041 1.32J5 2.013 -0.44U8 2.011 -0.56

Number of results 51

Median 2.020

Median MU 0.0028

First Quartile 2.009

Third Quartile 2.030

IQR 0.022

Normalised IQR 0.016

CV (%) 0.8

Minimum 1.973 (1.973)

Maximum 2.064 (2.155)

Range 0.091 (0.180)

Note: A # indicates an outlier where the z-score obtained is either greater then3 or less than -3. Codes for all participates are shown. The results columnshows a blank entry or 'NR' for those participants that did not submit a resultfor this test. Results in green have been calculated by the programcoordinator. An R indicates an abnormal result rejected by the programcoordinator. Minimum, Maximum and Range are calculated with outliersexcluded, those in brackets include outliers.

Part A - Maximum Dry Density: Z - Scores

Code Z Score Code Z Score

Statistic Value

Test Result

Test Result



ReviewWeak

ConsensusWeak

ConsensusReview

Z-score

Strong Consensus

Part A - Maximum Dry Density: Z - Score Graph

E9C2E2W2B9C7Z5B4G6F7N5C9J7L8S6W5G8Y6K3M6A7M5A2N9M7L4

U5F8Z3X2X6D6J5

U8W8V5X4Q7J8L5

W6L7

G7N6T5Q3F4P5K7A4V4

-3 -2 -1 0 1 2 3



% %A2 11.9 0.28 X6 11.9 0.28N9 12.2 0.71 E9 11.6 -0.14W5 12.3 0.85 W2 11.6 -0.14M8 J7 11.2 -0.71C2 10.7 -1.42 W8 12.3 0.85U5 11.2 -0.71 Q3 12.1 0.57D6 11.4 -0.43 Z5 11.2 -0.71T5 12.3 0.85 V4 11.8 0.14K7 12.3 0.85 N6 12.3 0.85B4 10.8 -1.28 L8 12.2 0.71N5 11.3 -0.57 M7 12.2 0.71Q7 11.9 0.28 L4 11.1 -0.85Z3 12.0 0.43 A7 11.2 -0.71G7 11.7 0.00 G8 11.8 0.14M6 11.8 0.14 L7 12.4 0.99B9 12.0 0.43 F7 11.2 -0.71P5 12.7 1.42 X4 11.2 -0.71J8 11.7 0.00 M5 10.9 -1.14E2 10.9 -1.14 L5 11.3 -0.57F8 11.3 -0.57 X2 10.9 -1.14C9 11.2 -0.71 F4 12.3 0.85Y6 10.8 -1.28 A4 12.1 0.57V5 12.3 0.85 G6 11.0 -0.99K3 11.3 -0.57 S6 10.7 -1.42W6 11.9 0.28C7 10.8 -1.28J5 12.2 0.71U8 11.7 0.00


Median 11.7

Median MU 0.12

First Quartile 11.2

Third Quartile 12.2

IQR 0.95

Normalised IQR 0.70

CV (%) 6.0

Minimum 10.7

Maximum 12.7

Range 2.0

Note: A # indicates an outlier where the z-score obtained is either greater then3 or less than -3. Codes for all participates are shown. The results columnshows a blank entry or 'NR' for those participants that did not submit a resultfor this test. Results in green have been calculated by the programcoordinator. An R indicates an abnormal result rejected by the programcoordinator. Minimum, Maximum and Range are calculated with outliersexcluded, those in brackets include outliers.

Part A - Optimum Moisture Content: Z - Scores

Code Z Score Code Z Score

Statistic Value

Test Result

Test Result



ReviewWeak

ConsensusWeak

ConsensusReview

Z-score

Strong Consensus

Part A - Optimum Moisture Content: Z - Score Graph

P5L7W5T5K7V5W8N6F4N9J5L8M7Q3A4Z3B9A2Q7W6X6M6V4G8G7J8U8

E9W2D6N5F8K3L5

U5C9J7Z5A7F7X4L4

G6E2

M5X2B4Y6C7C2S6

-3 -2 -1 0 1 2 3



t/m³ t/m³A2 1.771 0.00 U8*N9 1.770 -0.21 X6*W5 1.766 -1.04 E9 1.760 -2.28M8 W2 1.771 0.00C2 1.767 -0.83 J7 1.780 1.87U5 1.768 -0.62 W8*D6 1.769 -0.42 Q3*T5 1.771 0.00 Z5 1.780 1.87K7 1.989 45.24 # V4*B4 1.769 -0.42 N6*N5 1.768 -0.62 L8*Q7 2.009 49.39 # M7*Z3 2.017 51.05 # L4*G7* A7 1.771 0.00M6* G8 1.768 -0.62B9* L7 1.767 -0.83P5* F7 1.768 -0.62J8* X4 NRE2* M5 NRF8* L5 NRC9* X2 1.778 1.45Y6 1.770 -0.21 F4*V5 1.782 2.28 A4*K3 1.771 0.00 G6*W6 1.777 1.25 S6*C7 1.771 0.00J5 2.013 50.22 #

Value


Median 1.771

Median MU 0.0010



IQR 0.006

Normalised IQR 0.005

CV (%) 0.3

Minimum 1.760 (1.760)

Maximum 1.782 (2.017)

Range 0.022 (0.260)

1.769 -0.42

1.771 0.00

1.77 -0.21

1.771 0.00

Part B - Maximum Dry Density: Z - Scores

Code Z Score Code Z ScoreTest

Result

1.771 0.00

1.771 0.00

Test Result

1.769 -0.42

*See section 3.2. Part B in relation to adjustments made

Note: A # indicates an outlier where the z-score obtained is either greater then3 or less than -3. Codes for all participates are shown. The results columnshows a blank entry or 'NR' for those participants that did not submit a result forthis test. Results in green have been calculated by the program coordinator.An R indicates an abnormal result rejected by the program coordinator.Minimum, Maximum and Range are calculated with outliers excluded, those inbrackets include outliers.

Statistic



ReviewWeak

ConsensusWeak

ConsensusReview


Z-score

Strong Consensus

Part B - Maximum Dry Density: Z - Score Graph

Z3

J5

Q7

K7

V5

J7

Z5

X2

W6

A2

T5

E2, F8, C9*

K3

C7

U8, X6*

W2

W8, Q3*

A7

F4, A4, G6, S6*

N9

Y6

L8, M7, L4*

D6

B4

G7, M6, B9, P5, J8*

V4, N6*

U5

N5

G8

F7

C2

L7

W5

E9

-3 -2 -1 0 1 2 3



% %A2 17.1 0.00 U8*N9 17.1 0.00 X6*W5 17.5 5.40 # E9 18.1 13.49 #M8 W2 17.1 0.00C2 17.1 0.00 J7 17.0 -1.35U5 17.1 0.00 W8*D6 17.1 0.00 Q3*T5 17.1 0.00 Z5 17.0 -1.35K7 12.3 -64.75 # V4*B4 17.1 0.00 N6*N5 17.3 2.70 L8*Q7 11.9 -70.15 # M7*Z3 12.0 -68.80 # L4*G7* A7 17.0 -1.35M6* G8 17.2 1.35B9* L7 17.2 1.35P5* F7 17.1 0.00J8* X4 NRE2* M5 NRF8* L5 NRC9* X2 17.2 1.35Y6 17.0 -1.35 F4*V5 17.0 -1.35 A4*K3 17.1 0.00 G6*W6 17.0 -1.35 S6*C7 17.1 0.00J5 12.2 -66.10 #

Value


Median 17.1

Median MU 0.02



IQR 0.10

Normalised IQR 0.07

CV (%) 0.4

Minimum 17.0 (11.9)

Maximum 17.3 (18.1)

Range 0.3 (6.2)

Test Result

Part B - Optimum Moisture Content: Z - Scores

Code Z Score Code Z ScoreTest

Result

17.2 1.35

17.1 0.00

17.1 0.00

Statistic


Note: A # indicates an outlier where the z-score obtained is either greater then3 or less than -3. Codes for all participates are shown. The results columnshows a blank entry or 'NR' for those participants that did not submit a result forthis test. Results in green have been calculated by the program coordinator.An R indicates an abnormal result rejected by the program coordinator.Minimum, Maximum and Range are calculated with outliers excluded, those inbrackets include outliers.

0.0017.1

17.1 0.00

17.1 0.00

17.1 0.00



ReviewWeak

ConsensusWeak

ConsensusReview


Z-score

Strong Consensus

Part B - Optimum Moisture Content: Z - Score Graph

E9

W5

N5

G7, M6, B9, P5, J8*

G8

L7

X2

A2

N9

C2

U5

D6

T5

B4

E2, F8, C9*

K3

C7

U8, X6*

W2

W8, Q3*

V4, N6*

L8, M7, L4*

F7

F4, A4, G6, S6*

Y6

V5

W6

J7

Z5

A7

K7

J5

Z3

Q7

-3 -2 -1 0 1 2 3



5. Program Information

5.1. Z-score Summary

The proficiency program was conducted in June/July 2020. A ‘Z-score Summary’ was issued on the 4th August 2020. A copy was e-mailed to all participants who submitted results as well as being available on the LabSmart Services website. The summary is intended as an early indicator of participant performance. This program report supersedes the ‘z-score summary’. Further information can be found in section 5.9 ‘Statistics’.

As noted in the ‘Z-Score Summary’, there was an issue with the final outcome for Part B. Ultimately, this can only be seen as a good outcome, as many participants are now moving to the same/similar graphing techniques. But with over 50% of all participants returning the same OMC value of 17.1%, it made the normal statistical practices (Interquartile Range), used in proficiency programs such as this, to no longer work. More information can be found in section 3.2 Part B.

5.2. Program Design

5.2.1. Design

Part of the design of each program involves asking for the right information. The correct analysis of the data collected then allows feedback to be offered and enable participants to improve in their performance of this test.

In this program, the overall performance of the test is covered by ‘Part A’. The skill associated with graphing and data interpretation has been separated out in to ‘Part B’ by giving all participants the same set of data. This also gives a measure of the variability associated with just this part of the test method.

The program was designed to provide technical feedback regarding performance as well as possible improvements in performance. Other considerations involving the design of the program are detailed below.



5.2.2. Selection of material used in the program

The test in this proficiency program is operator skill/experience-dependent. In addition, certain types of soils require more knowledge to obtain consistent results than others.

Different materials are selected for each program to mirror the range of materials encountered in practice. This program provides a sample that gives results in the range that would be commonly tested by laboratories.

It is expected that the level of experience/skill need to perform these tests will present a reasonable assessment of the overall competency of the tester and industry performance.

5.2.3. Role of proficiency testing

The determination of outliers is an important task of this proficiency program. A secondary function is to provide feedback that can help those with outliers identify possible areas to investigate as well as assist all participants to improve.

In addition to the statistics, proficiency programs often obtain other information that is not normally available to a laboratory. It allows for a better understanding of the testing and can provide information that can lead to improvements in the testing process or test method.

Proficiency testing enables participants to measure competency against others. It is also a measure of staff performance and the equipment used. Apart from ‘measurement uncertainty’ it is the most useful tool a laboratory has in better understanding the performance of a test.

5.2.4. Participant assessment

Assessment of each participant is based on a z-score that is related to the program consensus value (median). This is used to determine any statistical outliers. Compliance to proficiency program requirements including the correct calculation of results and adherence to program and test method requirements may also be used as part of the assessment process. Participants may also be asked to investigate any discrepancies detected with the paperwork submitted.



5.2.5. Reporting of results - Significant figures

The number of decimal places (significant figures) reported for a test has a bearing on the statistical analysis and therefore, the interpretation of the results. There is a need to strike a balance between what is desirable from a statistical viewpoint while recognising how the results are used in practice.

Too few decimal places (e.g. due to rounding) can cause an increase in the observed spread of results. Increasing the number of decimal places (with respect to normal reporting) can distort the observed spread of results compared to that encountered in actual practice. Large numbers of similar, rounded results can also cause a distortion in the analysis.

For example, rounding to 0.5 % means that any number between 10.75 and 11.25 will be 11.0%. If the largest value is 10.75 in a set of results, it is pushed out to 11.0 through rounding. Rounded results are useful from “an end-user” perspective but are not as useful when considering laboratory performance. The test method acknowledges additional decimal places may be used for statistical purposes.

For this program, it was decided that the benefits of using additional decimal places would complement the aim of the proficiency program.

Participants results were analysed as received regardless of whether there were more, or less significant figures used other than the number requested by the program.

5.2.6. Additional information requested

This program requested additional information as detailed in Appendix C not usually reported. The additional information is, however, consistent with the performance of the test and the records, the test method requires laboratories to maintain. The additional information is used to interpret participant’s performance and assist with providing technical comment, including feedback on outliers and possible participant improvements.



5.2.7. Data checks

As often observed ‘operator errors’ can occur in the result calculation process. Not every participant’s results were verified as reasonable only those with outliers. The spacing of moisture steps was checked as per the test method. Checks, however, are only as accurate as the raw data supplied by each participant. These checks also help ensure that the data is comparable. Any inconsistencies identified during this process are identified as possible feedback for participant improvement. In some cases, inconsistencies identified may need to be investigated by participants.

5.3. Sample Preparation

Samples for the program were drawn and packaged from a single, well-mixed, lot.

Samples were laid out in the order prepared, then ten samples were selected at equal intervals. These ten samples were used for homogeneity testing. Each participant received randomly drawn samples from the remainder. A unique participation code was assigned to each sample.

Each sample was placed in a plastic bag, sealed, labelled with the program name and packed into a sturdy box prior to dispatch.

5.4. Packaging and Instructions

Each participant received one sealed plastic bag containing up to 17kg of soil. Participants were instructed to test per the nominated test method and report to the accuracy indicated on the ‘results log’ sheet.

See ‘Appendix A’ for a copy of the instructions issued to participants and ‘Appendix B’ for the Results Log sheet supplied.

5.5. Quarantine

Samples sent to Western Australia are subject to quarantine regulations that require treatment of the soil before importation. Samples sent to WA are heat-treated, and compliance certificates are enclosed within sample packaging. Where applicable, further instructions regarding preparation or handling of the sample may be included.



5.6. Sample dispatch

Samples were dispatched to participants on the 10th June 2020 via Pack and Send. Dispatched samples are tracked from dispatch to delivery to each participant by LabSmart Services.

Table 5: Homogeneity Results

Code MDD

(t/m³)

OMC

(%)

H1 2.005 12.0

H2 2.012 11.8

H3 2.002 11.3

H4 2.013 11.4

H5 1.996 11.7

H6 2.015 11.6

H7 2.012 11.2

H8 2.014 11.9

H9 2.016 11.6

H10 2.011 11.8

Average 2.010 11.6

Standard Deviation

0.006 0.26

Minimum 1.996 11.2

Maximum 2.016 12.0

Range 0.020 0.8

Coefficient of Variation (%)

0.32 2.26



5.7. Homogeneity Testing

Samples for homogeneity (Part A) testing were packed in the same way as those for participants. Ten samples were selected at equal intervals. To approximate the same conditions as participants, the same instructions were given to the laboratory performing the homogeneity testing.

The homogeneity data was reviewed. the homogeneity data showed a small variation indicative of the sample being homogenous and therefore suitable for this program. A summary of these results can be seen in Table 5.

5.8. Participation

52 participants entered the program with a total of 51 participants returning results. The nominated date for participants to return their results was 6th of July 2020. There was one participant unable to return their results in time for inclusion in the final report.

5.9. Statistics

Z-Scores were calculated for each test and used to assess the variability of each participant relative to the consensus median. A corresponding z-score graph was produced for each test.

The use of median and quartiles reduces the effect that outliers have on the statistics and other influences. As a consequence, z-scores provide a more realistic or robust method of assessment.

Some results were reported by participants to more decimal places than requested as part of the proficiency program and by others to fewer decimal places. In all instances, test results have been used as submitted by participants.

A z-score is one way of measuring the degree of consensus with respect to the grouped test results. The z-scores in this report approximate standard deviations. For each test, a z-score graph is shown. Use the graph to visually check statistically how you compare to other participants.

The following bar is shown at the bottom of each graph. This helps to visualize where each participant’s result falls quickly.



Review Weak

Consensus Strong Consensus

Weak

Consensus Review

Figure 1: Z-score interpretation bar

For example:

A strong consensus (i.e. agreement) means that your test result is close i.e. within 1 standard deviation of the median.

A weak consensus means that your test result is satisfactory and is within 2 standard deviations of the median.

If you have obtained a test result that is outside 2 standard deviations, then it may be worth reviewing your testing processes to ensure that all aspects are satisfactory. Only those obtaining a z-score approaching 3 (I.e. outside 2.75 range) have been highlighted in the report for review.

If you have obtained a test result that is outside 3 standard deviations, then you will need to investigate your testing processes to ensure that all aspects are satisfactory.

For further details on the statistics used in this proficiency program can be obtained from LabSmart Services or download the ‘Participant Guide’ from the LabSmart Services website.

5.9.1. Z-score summary

A “Z-Scores Summary” is issued soon after most results are received. It gives participants early feedback as to any program outliers. The summary is usually available on the LabSmart Services website up until the final report is issued. The final report supersedes the z-score summary.

The final report contains detailed technical feedback regarding the performance of tests and revised z-scores. The inclusion of late results or corrections is at the discretion of the program coordinator. In some instances, this may change some of the z-scores slightly, but generally, the performance outcome remains the same. If there is any impact, it will be discussed in section 5.1 of the report.

5.9.2. Comparing statistics from one program to another

The statistics generated from one proficiency program are not usually comparable to those from another proficiency testing program. Only very general comparisons may be possible. The



reason statistics from one program may not be compared to another is due to the range of variables that differ from one proficiency program to another.

These variables include:

Type of material selected, The number of participants, Experience of participants, Test methodology variations, Equipment used, Test methods used, Experience of supervisors, Range of organisations involved. Program design and the statistics employed

The program outcome represents a ‘snapshot’ of the competency within the industry and hence provides an overview of the industry - The more participants involved in each program, the more representative the overview.

5.9.3. Measurement uncertainty

The statistics detailed in this program do not replace the need for laboratories to separately calculated measurement uncertainties associated with each test when required by the client or NATA. The proficiency program does give information useful for calculating the MU and bench-marking the MU calculated.

5.9.4. Metrological traceability

The assigned median value used in this proficiency testing program is derived from participant performance and is not metrologically traceable.

5.10. Non-statistical Matters

One of the issues faced by proficiency testing providers is what to do with an incorrect result even if its z-score is satisfactory. In many cases, they cannot be detected but still can have a significant impact on the statistics calculated. This can cause biased (or unfair) outcomes for other participants.



To limit the effect that erroneous results may have on a program, additional information is requested to allow the main results to be recalculated. In some cases, results shown to be erroneous may be rejected for inclusion in the program. If the result does not add any statistical bias, it is left in the program.

The result, however, is incorrect even though it may have a satisfactory z-score. To highlight that the participant needs to investigate erroneous results, it is considered a ‘non-statistical’ matter.

This may also be applied to non-compliance to program requirements, e.g. incorrect reporting of results or incorrect partial calculations/data.

Non-statistical matters were not used as part of the assessment process for this program.



Appendix A: Instructions for testers



Appendix B: Results Log



Appendix C: Participant Supplied Test Information

A2 1289.5.1.1 1289.2.1.1 6 Yes A Yes

N9 1289.5.1.1 1289.2.1.1 4 No A Yes

W5 1289.5.1.1 1289.2.1.1 164.0 No A Yes

M8

C2 1289.5.1.1 1289.2.1.1 2 No A Yes

U5 1289.5.1.1 1289.2.1.1 48 No A Yes

D6 1289.5.1.1 1289.2.1.1 2.5 No A NR

T5 1289.5.1.1 1289.2.1.1 72 No A Yes

K7 1289.5.1.1 1289.2.1.1 24 No A NR

B4 1289.5.1.1 1289.2.1.1 71 No A No

N5 1289.5.1.1 1289.2.1.1 74.5 No A Yes

Q7 Q142A 1289.2.1.1 6 No A NR

Z3 Q142A 1289.2.1.1 46 No A NR

G7 1289.5.1.1 1289.2.1.1 49 No A Yes

M6 1289.5.1.1 1289.2.1.1 55 No A NR

B9 1289.5.1.1 1289.2.1.1 48 No A Yes

P5 1289.5.1.1 1289.2.1.1 66 No A Yes

J8 1289.5.1.1 1289.2.1.1 51 No A Yes

E2 1289.5.1.1 1289.2.1.1 2 No A Yes

F8 1289.5.1.1 1289.2.1.1 2 No A Yes

C9 1289.5.1.1 1289.2.1.1 96.0 No A Yes

Y6 1289.5.1.1 1289.2.1.1 2.0 No A NR

V5 1289.5.1.1 1289.2.1.1 24 No A NR

K3 1289.5.1.1 1289.2.1.1 55 Yes A NR

W6 1289.5.1.1 1289.2.1.1 57 No A Yes

C7 1289.5.1.1 1289.2.1.1 24 No A NR

J5 1289.5.1.1 1289.2.1.1 264 No A Yes

U8 1289.5.1.1 1289.2.1.1 48 No A Yes

X6 1289.5.1.1 1289.2.1.1 48 No A Yes

E9 1289.5.1.1 1289.2.1.1 48 No A Yes

W2 1289.5.1.1 1289.2.1.1 120 No A NR

J7 1289.5.1.1 1289.2.1.1 74.5 No A NR

W8 1289.5.1.1 1289.2.1.1 2.0 No A NR

Q3 1289.5.1.1 1289.2.1.1 96.0 No A NR

Z5 1289.5.1.1 1289.2.1.1 2 No A Yes

V4 1289.5.1.1 1289.2.1.1 22.5 Yes A NR

N6 1289.5.1.1 1289.2.1.1 46.5 Yes A Yes

L8 1289.5.1.1 1289.2.1.1 360.5 No A Yes

M7 1289.5.1.1 1289.2.1.1 360.5 No A Yes

L4 1289.5.1.1 1289.2.1.1 NR No A Yes

A7 1289.5.1.1 1289.2.1.1 5 No A Yes

G8 1289.5.1.1 1289.2.1.1 2.0 No A Yes

L7 1289.5.1.1 1289.2.1.1 48 No A Yes

F7 1289.5.1.1 1289.2.1.1 2 No A Yes

X4 1289.5.1.1 1289.2.1.1 2.0 No A NR

M5 1289.5.1.1 1289.2.1.1 2.0 No A NR

L5 1289.5.1.1 1289.2.1.1 2.0 No A NR

X2 1289.5.1.1 1289.2.1.1 2 No A Yes

F4 1289.5.1.1 1289.2.1.1 2.0 No A Yes

A4 1289.5.1.1 1289.2.1.1 2.5 No A Yes

G6 1289.5.1.1 1289.2.1.1 29 No A NR

S6 1289.5.1.1 1289.2.1.1 3 No A NR

Code MDD Method MC MethodCure Time

(hrs)Mechanical Compaction

Mould Size Used

Graph method same for Part A & B

labsmart report compaction 2020

Documents