final report of the proficiency testing in vehicles ... · final report of the proficiency testing...

Relatório Preliminar do Ensaio de Proficiência de Emissões de

Automóveis - 7a Rodada

Final Report of the Proficiency Testing in Vehicles Emissions

9th Round

19/12/2018 Page 1 of 40

PROFICIENCY TESTING IN VEHICLES EMISSIONS – 9th ROUND

Period of completion: from 02/10/17 to 29/11/18

FINAL REPORT – Nº 005/2018 – Review 01

PROFICIENCY TESTING ORGANIZATION

Instituto Nacional de Metrologia, Qualidade e Tecnologia - Inmetro.

Diretoria de Metrologia Científica e Tecnologia - Dimci

Endereço: Av. Nossa Senhora das Graças, 50 - Xerém - Duque de Caxias

RJ - Brasil - CEP: 25250-020

E-mail para contato: [email protected]

ORGANIZING COMMITTEE

Adelcio Rena Lemos (Inmetro/Dimci)

José Ricardo Bardellini da Silva (Inmetro/Dimci) - Coordenador PEP-Inmetro

Paulo Roberto da Fonseca Santos (Inmetro/Dimci)

Carla Thereza Coelho (Inmetro/Dimci)

Marcelo Lima Alves (Inmetro/Dimci/Dimqt/Lafiq)

Paulo Roque Martins Silva (Inmetro/Dimci/Dimqt/Lafiq)

Valnei Smarçaro da Cunha (Inmetro/Dimci/Dimqt)

TECHNICAL COMMITTEE

Danilo Torres (AEA)

Gabriel Fonseca Sarmanho (Inmetro/Dimci/Dimqt)

Luiz Henrique da Conceição Leal (Inmetro/Dplan/Dgcor)

Marcelo Lima Alves (Inmetro/Dimci/Dimqt/Lafiq)

Paulo Roque Martins Silva (Inmetro/Dimci/Dimqt/Lafiq)

Valnei Smarçaro da Cunha (Inmetro/Dimci/Dimqt)

Final Report of the Proficiency Testing in Vehicles Emissions – 9th round- Rev. 01

Pag. 2 of 40

SUMMARY

1. Introduction .............................................................................................................................................. 3

2. Materials and Methods ............................................................................................................................. 4

2.1. Test Item................................................................................................................................................ 4

2.2. Methodology ......................................................................................................................................... 4

3. Test Item Integrity ................................................................................................................................... 5

4. Statistical Analysis of Participants’ Results ............................................................................................ 6

4.1. z Score .................................................................................................................................................. 6

5. Assigned Values ...................................................................................................................................... 6

6. Results Dispersion ................................................................................................................................... 8

6.1. Urban Cycle Emissions ........................................................................................................................ 9

6.2. Evaporative Emissions (g/test) ........................................................................................................... 13

6.3. Road Cicle Emissions ......................................................................................................................... 14

7. Participants’ Results .............................................................................................................................. 19

7.1. Urban Cicle Emissions ....................................................................................................................... 19

7.2. Evaporative Emissions ....................................................................................................................... 26


8. Analysis Testimony ............................................................................................................................... 36

8.1. Urban Cicle Emissions ....................................................................................................................... 36

8.2. Evaporative Emissions ....................................................................................................................... 36


9. Confidenciality....................................................................................................................................... 37

10. Conclusions ................................................................................................................................... 37

11. Participants .................................................................................................................................... 39

12. References ..................................................................................................................................... 40

13. Review History ............................................................................................................................. 40


Pag. 3 of 40

1. Introduction

The problem of air pollution is a serious threat to human health, decreasing their quality of life. Vehicles

are potential agents of pollution worldwide. Gas emissions from vehicles carry several toxic substances,

which, in some cases, in contact with the respiratory system, can produce several negative health effects

and cause traffic accidents due to decreased visibility.

The analysis of pollutants is one of the most delicate items of a vehicle or an engine emission test. The

Proficiency Testing Schemes (PT Scheme) of automotive emissions evaluate laboratories by the

determination of the compound amounts in vehicle emissions, then providing subsidies to laboratories

to identify and solve analytical problems, contributing to the harmonization of emission measurements

in the country.

Proficiency testing scheme is a quality tool for the identification of interlaboratory differences, but the

assessment is punctual. A PT Scheme aims to compare measurement results from different laboratories,

performed under similar conditions, and then to obtain an assessment of the technical competence of

participating laboratories in order to demonstrate the reliability of their measurement processes. The

participating laboratories, in their turn, have the opportunity to review their analysis procedures and

implement improvements in their processes, if necessary.

In this round, the following vehicle emission parameters were proposed to be evaluated: (CO, CO2, THC,

NOx, NMHC and total aldehydes in g/km and urban autonomy, road autonomy and combined autonomy

in km/L and evaporative emissions (g/test). Ten parameters were evaluated with participation of 20

(twenty) laboratories, three more than the last round.

This report presents the results of the performance evaluation of participants, the methodology used in

the tests and the procedure used for the statistical analysis.

The objectives of this PT scheme were:

• To determine the performance of laboratories for the proposed tests;

• To monitor the ongoing performance of the analytical vehicle emissions laboratories;

• To increase the confidence of the measuring emission process of the vehicle emission laboratories;

• To improve continuously the measurement techniques of vehicle emissions laboratories.


Pag. 4 of 40

2. Materials and Methods

2.1. Test Item

The test item is a vehicle supplied by Hyundai CAOA do Brasil having the following characteristics:

Model TUCSON, gray color, identification code REPCAR2, 2.0L motor, Flex Fuel, four-speed

automatic transmission, equivalent inertia of 1644 kg. The test vehicle was correlated with the purge

system of the blow-by gas and canister (the test item was supplied with the necessary changes), since,

in this edition, there was no evaporative emission measurement.

Each participating laboratory should use its own fuel (Gasool A22 as ABNT NBR 8689 standard in

force).

2.2. Methodology

The standard methods used for emission measurements were ABNT NBR 6601, 7024 and 12026 The

tests defined by these standard methods are complementary and were carried out simultaneously. The

values of deceleration times (coast down) were provided by Hyundai do Brasil emission laboratory,

vehicle owner, to participants in order to adjust their dynamometers to reproduce the deceleration time.

Three different tests were carried out, namely:

• Determination of THC, NMHC, total aldehydes, NOx, CO, CO2 and urban autonomy, according to

ABNT NBR 6601 and 12026 standards;

• Determination of hot stage evaporative emissions according to ABNT NBR 11481 standard;

• Determination of CO, CO2, THC, NOx and NMHC, road autonomy in road cycle and combined

autonomy according to ABNT NBR 7024 standard.

Participants should follow the test flow chart presented in figure 1 when performing the tests and

preferably start the tests at 25 °C temperature, aiming minimizing cold start effects in results. A CETESB

representative witnessed one of the three PT measurements of each participant.


Pag. 5 of 40

Figure 1 – Flow chart of PT measurement activities.

3. Test Item Integrity

Hyundai CAOA laboratory performed stability tests in the beginning and at the end of the cycle – first

analysis (Y_1), second analysis (Y_2).

GM do Brasil CPCA laboratory performed stability tests in the beginning, in the middle and at the end

do the cycle – first analysis (X_1), second analysis and third analysis (X_3).

It was verified if there were statistical differences between measurements of the 7 (seven) components

of urban cycle CO, CO2, THC, NOx, NMHC, total aldehydes in g/km and urban autonomy in km/L, of

the 8 (eight) components of road cycle, CO, CO2, THC, NOx e NMHC in g/km, urban autonomy, road

autonomy and combined autonomy in km/L and 1 (one) hot stage evaporative emissions component in

g/test.

All results were the same, with p-value greater than 0.05. Thereby, it can be assured that, to a level of

confidence of 95 %, there are no difference statistically significant between the mean and the sample

data can be considered as coming from the same population. Thus, the vehicle maintained integrity

during the performance of this Proficiency Test.

Thermal conditions stabilization:

12 h to 36 h

Emission test according to ABNT NBR 6601

standard (urban cycle)

Preconditioning according to ABNT NBR

6601 standard (5.4.3 item)

Preconditioning according to ABNT NBR

6601 standard (5.4.3 item)

Hot stage evaporative emission test or wait an

hour

Emission test according to ABNT NBR 7024

standard (road cycle)

Drainage and refueling with 40% of total tank

capacity


Pag. 6 of 40

Due to data confidentiality, once Hyundai CAOA do Brasil and GM do Brasil CPCA are also participants

of this PT, these results were not presented.

4. Statistical Analysis of Participants’ Results

4.1. z Score

For the participants’ results evaluation, it was followed one of ABNT NBR ISO/IEC 17043:2011

criteria, z score (distance measurement related of the laboratory measurement result in relation to the PT

designated value, that was calculated according to equation 1.

Xxz i

i

(1)

Where:

xi is the mean measurement result of the ith participant;

X is the PT designated value;

is the standard deviation for the proficiency testing, that in this round was established as described

in ISO 13528:2015 standard, that is, a robust standard deviation based on participants’ results.

The interpretation of z score is presented as follows:

|z| ≤ 2,0 - indicates “satisfactory” performance and generates no signal;

2,0 < |z| <3,0 – indicates “questionable” performance and generates a warning signal;

|z| ≥ 3,0 - indicates “unsatisfactory” performance and generates an action signal.

5. Assigned Values

According to available procedures for the establishment of designated values by ABNT NBR

ISO/IEC 17043:2011, the designated values of this PT were calculated by statistical methods described

in 7.7 item of ISO 13528:2015 standard, that is, consensus values from participant results.

ISO 13528:2015 standard describes the robust analysis involving employment of the A algorithm for the

calculation of designated value and standard deviation. The robust statistical techniques are used to

minimize the influence that extreme results can have on estimates of mean and standard deviation.

Initially, all values object of the analysis (values sent by participants) were put in ascending order. Next,

robust values and standard deviation of these data were denoted by (x*) and (s*). Initial values of (x*)

and (s*) were calculated according to equations below:


Pag. 7 of 40

x∗ = xi median (2)

s* = 1,483 x median |xi – x*| (3)

(x*) e (s*) values were updated as follows. It was calculated:

*s,51 (4)

For each xi (i = 1, 2,..., p), it was calculated:

otherwise,x

xxif,x

xxif,x

x

i

*i

*

*i

*

*i

--

(5)

new values of (x*) e (s*) should be calculated from the equations:

p/xx *i

* (6)

113412

p/xx,s **i

* (7)

Where the summation is over i.

The robust estimation (x*) and (s*) can be obtained by an iterative calculation, i.e. by updating the values

of (x*) and (s*) several times using the modified data, until the process converges. Convergence may be

assumed when there is no change from one iteration to the next in the third significant figure of the

robust standard deviation and of the equivalent figure in the robust average.

The results out of 2 standard deviation intervals after the robust average and robust standard deviation

calculation were considered as outliers and new assigned values as well new robust standard deviation

results were calculated for each parameter of the PT, removing those outliers.

Tables 1, 2 and 3 present the assigned values and the robust standard deviation for all parameters,

including all PT participants, as well as the new robust average and standard deviation values after

removal of the outlier results.

Each participant in this report is identified by the last three characters of its identification code in

tables, graphs and texts.


Pag. 8 of 40

Table 1 - Assigned values and standard deviation of the PT – urban cycle emissions.

Parameter Designated

value

Standard

deviation

Outliers

(participants)

Recalculated

designated value

Recalculated

standard

deviation

CO (g/km) 0.380 0.058 - - -

CO2 (g/km) 239.9 5.2 - - -

THC (g/km) 0.034 0.005 - - -

NOx (g/km) 0.065 0.013 064 - -

NMHC (g/km) 0.030 0.004 - - -

Total aldeydes

(g/km) 0.00103 0.00031 113 0.00105 0.00029

Urban autonomy

(km/L) 9.04 0.20 - - -

Table 2 - Assigned values and standard deviation of the PT – evaporative emissions cycle.


value

Standard

deviation

Outliers

(participants)

Recalculated

designated value

Recalculated

standard

deviation

Evaporative

emissions 0.342 0.102 052 0.356 0.090

Table 3 - Assigned values and standard deviation of the PT – road cycle.


value

Standard

deviation

Outliers

(participants)

Recalculated

designated value

Recalculated

standard

deviation

CO (g/km) 0.487 0.086 -- -- --

CO2 (g/km) 183.1 2.7 009, 015 e 045 182.9 2.0

THC (g/km) 0.013 0.001 015 0.013 0.001

NOx (g/km) 0.030 0.007 -- -- --

NMHC (g/km) 0.011 0.001 015 0.010 0.001

Urban autonomy 9.04 0.20 -- -- --

Road autonomy 11.82 0.18 015 e 045 11.83 0.14

Combined

autonomy 10.10 0.20 009 e 015 10.10 0.17

6. Results Dispersion

In the presented graphs for all tested parameters, a continuous line represents the assigned value and the

last three digits of its identification code identify each laboratory. Dotted lines are representations of Ref

± 1s and Ref ± 2s, where "Ref" is the assigned value (robust average) and "s" is the robust standard

deviation.


Pag. 9 of 40

6.1. Urban Cycle Emissions

Figures 2 to 8 graphically present the means and standard deviations of the reported emission results by

the laboratories for each analyzed parameter.

Figure 2 – Scatter plot of the results for CO determination – urban cycle.


Pag. 10 of 40

Figure 3 – Scatter plot of the results for CO2 determination – urban cycle.

Figure 4 – Scatter plot of the results for THC determination – urban cycle.


Pag. 11 of 40

Figure 5 – Scatter plot of the results for NOx determination – urban cycle.

Figure 6 – Scatter plot of the results for NMHC determination – urban cycle.


Pag. 12 of 40

Figure 7 – Scatter plot of the results for total aldehydes determination – urban cycle.

Figure 8 – Scatter plot of the results for urban autonomy determination – urban cycle.


Pag. 13 of 40

Through the graphs, it can be seen that:

CO (g/km): Among all 20 participants that reported results within the Ref ± 2s interval, participants

064, 113, 120,117 and 008 presented the greatest dispersions.

CO2 (g/km): Among all 20 participants that reported results within the Ref ± 2s interval, participants

086, 117, 030 and 061 presented the greatest dispersions.

THC (g/km): Among all 20 participants that reported results within the Ref ± 2s interval, participant

117 presented the greatest dispersion.

NOx (g/km): Among 19 participants that reported results within the Ref ± 2s interval, participant 064

presented result out of this interval and had the greatest data dispersion.

NMHC (g/km): Among all 20 participants that reported results within the Ref ± 2s interval, participants,


Total aldehydes (g/km): Among 19 participants that reported results within the Ref ± 2s interval,

participant 113 presented result out of this interval and participants 117 and 019 presented the greatest

dispersions.

Urban autonomy (km/L): Among all 20 participants that reported results within the Ref ± 2s interval,

participants, 061, 117, 030 and 086 presented the greatest dispersions.

6.2. Evaporative Emissions (g/test)

Figure 9 graphically presents the means and standard deviations of the reported results for evaporative

emissions data (g/test) by participants for each analyzed parameter.


Pag. 14 of 40

Figure 9 – Scatter plot of the results for evaporative emissions (g/test).


Evaporative Emissions (g/test): Among 12 participants that presented results within the Ref ± 2s

interval, participant 052 presented result out of this interval and participants 082 and 116 presented the

greatest data dispersions.

6.3. Road Cicle Emissions

Figures 10 to 17 graphically presents the means and standard deviations of the reported results for road

cycle emission data by participants for each analyzed parameter.


Pag. 15 of 40

Figure 10 – Scatter plot of the results for CO – road cycle.

Figure 11 – Scatter plot of the results for CO2 – road cycle.


Pag. 16 of 40

Figure 12 – Scatter plot of the results for THC – road cycle.

Figure 13 – Scatter plot of the results for NOx – road cycle.


Pag. 17 of 40

Figure 14 – Scatter plot of the results for NMHC – road cycle.

Figure 15 – Scatter plot of the results for urban autonomy (km/L) – road cycle.


Pag. 18 of 40

Figure 16 – Scatter plot of the results for road autonomy (km/L) – road cycle.

Figure 17 – Scatter plot of the results for combined autonomy (km/L) – road cycle.



Pag. 19 of 40

CO (g/km): Among all 20 participants that reported results within the Ref ± 2s interval, participants


CO2 (g/km): Among 16 participants that reported results within the Ref ± 2s interval, participants 009,

015 and 045 presented result out of this interval and participant 091 presented the greatest dispersion.

THC (g/km): Among 19 participants that reported results within the Ref ± 2s interval, participants 011,

079 and 091 were borderlines, participant 015 is out of this limit and participants 047 and 015 presented

the greatest dispersions.

NOx (g/km): Among all 20 participants that reported results within the Ref ± 2s interval, participants

015, 009, 031, 022, 047, 011, 091 and 045 presented the greatest dispersions.

NMHC (g/km): Among 19 participants that reported results within the Ref ± 2s interval, participant 015

is out of this interval and participants 047 and 015 presented the greatest dispersions.

Urban autonomy (km/L): 19 participants reported results within the Ref ± 2s interval. Participant 045

did not present results.

Road autonomy (km/L): Among 17 participants that reported results within the Ref ± 2s inteval,

participants 103, 009, 015 and 045 are out of this interval and participants 091 and 006 presented the

greatest dispersions.

Combined autonomy (km/L): Among 18 participants that reported results within the Ref ± 2s interval,

participants 009 and 015 are out of this interval and participants 091, 001 and 099 presented the greatest

dispersions.

7. Participants’ Results

Measurement results reported by participants in this PT are presented in sections 7.1 to 7.3.

In this report each participant is identified only by the final numbering of its identification code

in the tables and graphs.

Participants 050, 053, 075, 110 and 111 did not send their results report, due to problems in their

equipment and reported this situation to the PT coordination.

7.1. Urban Cicle Emissions

Tables 4 and 5 present the average and standard deviations of each participant, where the result is the

mean value of the replicates.


Pag. 20 of 40

Note: All decimal places were considered for calculations, but the values in the tables 4 and 5 were

rounded to the same number of decimal places as requested in the results form.

Table 4 – Average and standard deviation of participants for CO, CO2, THC, NOX and NMHC (g/km)

parameters – urban cycle.

Code

CO CO2 THC NOx NMHC

(g/km) (g/km) (g/km) (g/km) (g/km)

Average Standard

deviation Average

Standard

deviation Average

Standard

deviation Average

Standard

deviation Average

Standard

deviation

004 0.333 0.026 231.5 0.2 0.031 0.001 0.052 0.002 0.028 0.001

007 0.301 0.018 234.2 1.2 0.028 0.001 0.054 0.003 0.025 0.000

008 0.470 0.090 240.2 0.7 0.038 0.001 0.069 0.001 0.033 0.001

018 0.314 0.009 232.1 0.8 0.032 0.002 0.048 0.007 0.029 0.003

019 0.381 0.033 243.6 0.9 0.040 0.002 0.080 0.005 0.036 0.002

030 0.375 0.001 239.9 5.4 0.035 0.002 0.075 0.001 0.031 0.002

044 0.433 0.025 241.9 2.5 0.025 0.001 0.048 0.001 0.023 0.001

056 0.411 0.013 239.2 2.0 0.034 0.001 0.057 0.002 0.030 0.001

061 0.398 0.022 246.7 4.8 0.039 0.002 0.062 0.005 0.035 0.002

064 0.354 0.052 242.8 0.5 0.033 0.002 0.114 0.023 0.029 0.002

065 0.303 0.023 238.9 1.7 0.041 0.000 0.077 0.011 0.037 0.000

071 0.343 0.015 246.9 1.7 0.033 0.002 0.054 0.002 0.029 0.003

081 0.313 0.029 243.3 1.6 0.030 0.001 0.057 0.012 0.027 0.001

086 0.436 0.017 239.2 8.4 0.036 0.001 0.064 0.006 0.032 0.001

092 0.362 0.014 248.8 0.4 0.038 0.002 0.067 0.007 0.034 0.003

098 0.407 0.018 241.0 1.0 0.033 0.000 0.056 0.011 0.029 0.000

106 0.375 0.013 236.4 0.6 0.032 0.001 0.072 0.010 0.030 0.001

113 4.040 0.044 237.4 1.7 0,036 0.001 0.082 0.012 0,032 0.001

117 0.460 0.070 239.6 6.7 0.041 0.004 0.075 0.004 0.037 0.003

120 0.431 0.051 234.3 2.0 0.026 0.001 0.071 0.009 0.022 0.002

Table 5 – Average and standard deviation of participants for total aldehydes (g/km) and urban

autonomy (km/L) parameters – urban cycle.

Code

Total aldehydes

(g/km)

Urban autonomy

(km/L) Code

Total aldehydes

(g/km)

Urban autonomy

(km/L)

Average Standard

deviation Average

Standard

deviation Standard

deviation Average

Standard

deviation Average

004 0.00125 0.00004 9.37 0.01 065 0.00092 0.00003 9.11 0.06

007 0.00096 0.00008 9.25 0.05 071 0.00093 0.00015 8.76 0.05

008 0.00000 0.00003 9.02 0.03 081 0.00130 0.00004 8.91 0.06

018 0.00109 0.00008 9.34 0.03 086 0.00136 0.00021 9.10 0.32

019 0.00139 0.00033 8.89 0.04 092 0.00146 0.00015 8.71 0.01

030 0.00106 0.00015 9.04 0.20 098 0.00090 0.00001 9.00 0.04

044 - - 8.96 0.09 106 0.00065 0.00005 9.17 0.03


Pag. 21 of 40

Code

Total aldehydes

(g/km)

Urban autonomy

(km/L) Code

Total aldehydes

(g/km)

Urban autonomy

(km/L)

Average Standard

deviation Average

Standard

deviation Standard

deviation Average

Standard

deviation Average

056 0.00097 0.00007 9.06 0.07 113 0.00018 0.00002 9.13 0.07

061 0.00115 0.00010 8.78 0.17 117 0.00130 0.00040 9.01 0.26

064 0.00060 0.00010 8.93 0.02 120 0.00087 0.00000 9.24 0.07

For the performance evaluation of the participants, z-score values were calculated, after the exclusion of

the outlier results, using the robust average and robust standard deviation of the results for each

parameter as assigned value and its standard deviation. Tables 6 and 7 and figures 18 to 24 show these

results.

Table 6 – z-score values for the CO, CO2, THC, NOX and NMHC parameters – urban cycle.

Code CO (g/km) CO2 (g/km) THC (g/km) NOX (g/km) NMHC (g/km)

z score z score z score z score z score

004 -0.80 -1.60 -0.64 -1.00 -0.54

007 -1.35 -1.08 -1.16 -0.79 -1.13

008 1.53 0.05 0.76 0.34 0.63

018 -1.12 -1.49 -0.45 -1.27 -0.32

019 0.01 0.72 1.21 1.28 1.23

030 -0.09 -0.00 0.12 0.87 0.12

044 0.90 0.38 -1.73 -1.30 -1.80

056 0.52 -0.12 -0.03 -0.57 -0.08

061 0.30 1.30 0.95 -0.17 0.86

064 -0.44 0.56 -0.13 4.00 -0.25

065 -1.31 -0.18 1.34 1.06 1.45

071 -0.62 1.35 -0.26 -0.81 -0.32

081 -1.13 0.65 -0.71 -0.57 -0.76

086 0.94 -0.12 0.37 -0.03 0.26

092 -0.30 1.71 0.69 0.25 0.78

098 0.45 0.20 -0.32 -0.71 -0.32

106 -0.08 -0.67 -0.39 0.60 -0.25

113 0.40 -0.47 0.25 1.41 0.19

117 1.36 -0.04 1.40 0.87 1.45

120 0.86 -1.05 -1.61 0.52 -1.88

* Satisfactory result

* Questionable result

* Unsatisfactory result


Pag. 22 of 40

Table 7 – z-score values for the total aldehydes (g/km) and urban autonomy (km/L) parameters – urban

cycle.

Code

Total

aldehydes

(km/L)

Urban

autonomy

(km/L) Code

Total

aldehydes

(km/L)

Urban

autonomy

(km/L)

z score z score z score z score

004 0.68 1.63 065 -0.45 0.38

007 -0.31 1.07 071 -0.41 -1.38

008 -1.10 -0.10 081 0.85 -0.62

018 0.13 1.51 086 1.07 0.30

019 1.16 -0.72 092 1.42 -1.65

030 0.03 -0.00 098 -0.53 -0.20

044 - -0.40 106 -1.39 0.65

056 -0.26 0.09 113 -2.99 0.43

061 0.35 -1.27 117 0.85 -0.15

064 -1.56 -0.55 120 -0.61 1.00




Figure 18 – z-score graph for CO measurement – urban cycle


Pag. 23 of 40

Figure 19 – z-score graph for CO2 measurement – urban cycle

Figure 20 – z-score graph for THC measurement – urban cycle


Pag. 24 of 40

Figure 21 – z-score graph for NOx measurement – urban cycle

Figure 22 – z-score graph for NMHC measurement – urban cycle


Pag. 25 of 40

Figure 23 – z-score graph for total aldehydes measurement – urban cycle.

Figure 24 – z-score graph for urban autonomy measurement – urban cycle.


Pag. 26 of 40

Through z-score graph analysis, it can be seen that:

CO (g/km): 20 participants presented satisfactory results;

CO2 (g/km): 20 participants presented satisfactory results;

THC (g/km): 20 participants presented satisfactory results;

NOx (g/km): 19 participants presented satisfactory results. Participant 064 presented unsatisfactory

result;

NMHC (g/km): 20 participants presented satisfactory results;

Total aldehydes (g/km): 19 participants presented satisfactory results. Participant 113 presented

unsatisfactory result;

Urban autonomy (km/L): 20 participants presented satisfactory results.

7.2. Evaporative Emissions

Table 8 presents the average and standard deviation of each participant, where the result is the mean

value of the replicates.

Note:

Note: All decimal places were considered for calculations, but the values in the table 8 was rounded to

the same number of decimal places as requested in the results form.

Table 8 – Average and standard deviation of participants for evaporative emissions parameter (g/test).

Code

Evaporative emissions

Code


Average Standard

deviation Average

Standard

deviation

002 0.389 0.016 063 0.424 0.005

021 0.317 0.010 080 0.221 0.033

026 0.380 0.009 082 0.376 0.051

034 0.317 0.005 094 0.212 0.020

049 0.487 0.018 115 0.362 0.005

052 0.135 0.012 116 0.431 0.049

060 0.341 0.026 - - -



parameter as assigned value and its standard deviation. Table 9 and figure 25 show these results.


Pag. 27 of 40

Table 9 – z-score values for the evaporative emissions (g/teste) parameter.


Code z score Code z score

002 0.36 063 0.75

021 -0.42 080 -1.49

026 0.30 082 0.22

034 -0.43 094 -1.59

049 1.45 115 0.07

052 -2.44 116 0.83

060 -0.15 - -




Figure 25– z-score graph for evaporative emissions (g/test) measurement.


Evaporative emissions (g/test): the 12 participants presented satisfactory results. Participant 052

presented questionable result.


Table 10 presents the average and standard deviations of each participant, where the result is the mean

value of the replicates.


Pag. 28 of 40

Note: All decimal places were considered for calculations, but the values in the tables 10, 11 and 12

were rounded to the same number of decimal places as requested in the results form.

Table 10 – Average and standard deviation of participants for CO, CO2, THC, NOx and NMHC (g/km)

parameters– road cycle.

Code

CO CO2 THC NOx NMHC

(g/km) (g/km) (g/km) (g/km) (g/km)

Average Standard

deviation Average

Standard

deviation Average

Standard

deviation Average

Standard

deviation Average

Standard

deviation

001 0.559 0.096 184.3 1.3 0.013 0.000 0.038 0.002 0.011 0.000

006 0.388 0.073 184.5 3.2 0.014 0.001 0.039 0.003 0.012 0.001

009 0.530 0.043 171.4 0.4 0.010 0.000 0.030 0.005 0.010 0.000

011 0.401 0.017 181.5 0.2 0.017 0.000 0.038 0.005 0.014 0.000

013 0.533 0.118 181.0 0.6 0.012 0.000 0.027 0.000 0.010 0.000

015 0.550 0.009 192.4 2.7 0.020 0.004 0.025 0.005 0.018 0.004

020 0.389 0.036 185.4 2.2 0.011 0.001 0.018 0.002 0.009 0.001

022 0.613 0.071 182.1 0.8 0.011 0.000 0.034 0.004 0.009 0.001

031 0.361 0.040 182.0 0.6 0.012 0.000 0.027 0.005 0.010 0.000

045 0.499 0.138 196.2 0.8 0.014 0.001 0.043 0.010 0.011 0.001

047 0.437 0.017 182.8 0.3 0.015 0.003 0.037 0.007 0.012 0.002

070 0.425 0.072 184.6 0.9 0.013 0.001 0.024 0.003 0.010 0.000

073 0.564 0.084 182.7 0.6 0.011 0.001 0.028 0.002 0.010 0.001

079 0.476 0.052 186.2 0.6 0.017 0.001 0.030 0.002 0.013 0.000

084 0.537 0.024 183.2 0.5 0.013 0.000 0.033 0.004 0.011 0.001

091 0.605 0.177 185.2 5.3 0.017 0.002 0.042 0.006 0.014 0.001

095 0.467 0.128 181.5 0.9 0.013 0.000 0.027 0.002 0.011 0.000

099 0.520 0.090 180.5 0.8 0.011 0.001 0.030 0.001 0.009 0.000

103 0.486 0.081 177.8 1.2 0.013 0.001 0.024 0.003 0.011 0.001

112 0.395 0.049 183.0 0.5 0.012 0.001 0.024 0.001 0.010 0.000

Table 11 – Average and standard deviation of participants for urban autonomy (km/L), road autonomy

(km/L) and combined autonomy (km/L) parameters – road cycle.

Code

Urban autonomy

(km/L)

Road autonomy

(km/L)

Combined autonomy

(km/L)

Average Standard

deviation Average

Standard

deviation Average

Standard

deviation

001 9.10 0.33 11.78 0.08 10.14 0.21

006 8.90 0.04 11.74 0.21 9.98 0.08

009 9.37 0.01 12.62 0.04 10.59 0.02

011 9.12 0.07 11.98 0.02 10.22 0.05


Pag. 29 of 40

Code

Urban autonomy

(km/L)

Road autonomy

(km/L)

Combined autonomy

(km/L)

Average Standard

deviation Average

Standard

deviation Average

Standard

deviation

013 9.06 0.08 11.95 0.04 10.17 0.06

015 8.71 0.02 11.24 0.15 9.69 0.06

020 8.76 0.06 11.66 0.14 9.86 0.05

022 9.24 0.08 11.87 0.06 10.26 0.07

031 9.26 0.05 11.90 0.04 10.28 0.05

045 --- --- 11.03 0.03 9.77 0.03

047 9.02 0.03 11.85 0.02 10.10 0.02

070 9.00 0.04 11.74 0.05 10.05 0.04

073 8.96 0.09 11.84 0.03 10.06 0.06

079 8.79 0.17 11.63 0.03 9.87 0.12

084 9.13 0.07 11.81 0.03 10.17 0.04

091 9.01 0.26 11.64 0.33 10.03 0.27

095 9.35 0.03 11.93 0.06 10.36 0.04

099 9.04 0.20 11.99 0.06 10.16 0.16

103 9.17 0.02 12.18 0.08 10.32 0.03

112 8.92 0.06 11.84 0.03 10.04 0.05

Participant 045 did not present result for urban autonomy parameter.



parameter as assigned value and its standard deviation. Table 12 and figures 26 to 33 show these results.

Table 12 – z-score values for CO, CO2, THC, NOx and NMHC parameters – road cycle.

Code CO (g/km) CO2 (g/km) THC (g/km) NOx (g/km)

NMHC

(g/km)


001 0.83 0.65 0.19 0.92 0.46

006 1.14 0.76 0.73 1.09 0.66

009 0.53 -5.64 -0.35 -0.71 -0.77

011 0.99 -0.71 2.19 0.96 2.31

013 0.53 -0.93 -0.49 -0.43 -0.54

015 0.72 4.62 3.83 -0.76 4.58

020 1.13 1.21 -1.26 -1.58 -1.18


Pag. 30 of 40

Code CO (g/km) CO2 (g/km) THC (g/km) NOx (g/km)

NMHC

(g/km)


022 1.45 -0.41 -0.89 0.40 -1.18

031 1.45 -0.45 -0.71 -0.46 -0.56

045 0.14 6.49 0.37 1.65 0.05

047 -0.57 -0.07 0.92 0.79 0.87

070 0.71 0.76 -0.35 -0.89 -0.35

073 0.88 -0.11 -0.89 -0.33 -0.56

079 0.12 1.56 2.01 -0.11 1.69

084 0.57 0.11 0.01 0.31 0.05

091 1.36 1.07 2.01 1.52 2.11

095 0.22 -0.71 0.19 -0.41 0.25

099 0.37 -1.18 -0.89 -0.07 -0.97

103 0.01 -2.55 -0.35 -0.89 -0.15

112 1.06 0.00 -0.71 -0.89 -0.56




Table 13 – z-score values for urban autonomy (km/L), road autonomy (km/L) and combined autonomy

(km/L) parameters – road cycle.

Code

Urban

autonomy

(km/L)

Road

autonomy

(km/L)

Combined

autonomy

(km/L)

z score z score z score

001 0.26 -0.36 0.17

006 -0.73 -0.67 -0.71

009 1.56 5.54 2.78

011 0.34 1.06 0.64

013 0.05 0.81 0.34

015 -1.64 -4.22 -2.41

020 -1.38 -1.23 -1.40

022 0.94 0.22 0.90

031 1.01 0.43 1.01

045 --- -5.66 -1.93

047 -0.13 0.08 -0.02

070 -0.23 -0.67 -0.30

073 -0.43 0.01 -0.27

079 -1.27 -1.44 -1.34


Pag. 31 of 40

Code

Urban

autonomy

(km/L)

Road

autonomy

(km/L)

Combined

autonomy

(km/L)

z score z score z score

084 0.39 -0.15 0.36

091 -0.18 -1.37 -0.44

095 1.44 0.64 1.43

099 -0.04 1.07 0.32

103 0.60 2.41 1.20

112 -0.64 0.05 -0.40




Figure 26 – z-score graph for CO measurement – road cycle.


Pag. 32 of 40

Figure 27 – z-score graph for CO2 measurement – road cycle.

Figure 28 – z-score graph for THC measurement – road cycle.


Pag. 33 of 40

Figure 29 – z-score graph for NOx measurement – road cycle.

Figure 30 – z-score graph for NMHC measurement – road cycle.


Pag. 34 of 40

Figure 31 – z-score graph for urban autonomy measurement – road cycle.

Figura 32 – z-score graph for road autonomy measurement – road cycle.


Pag. 35 of 40

Figure 33 – z-score graph for combined autonomy measurement – road cycle.


CO (g/km): 20 participants presented satisfactory results;

CO2 (g/km): 16 participants presented satisfactory results. Participant 103 presented questionable result

and participants 009, 015 and 045 presented unsatisfactory results;

THC (g/km): 16 participants presented satisfactory results. Participants 011, 079 and 091 presented

questionable results and participant 015 presented unsatisfactory result;

NOx (g/km): 20 participants presented satisfactory results;

NMHC (g/km): 17 participants presented satisfactory results. Participants 011 e 091 presented

questionable results and participant 015 presented unsatisfactory result;

Urban autonomy (km/L): 19 participants presented satisfactory results.

Road autonomy (km/L): 16 participants presented satisfactory results. Participant 103 presented

questionable result and participants 009, 015 and 045 presented unsatisfactory results;

Combined autonomy (km/L): 18 participants presented satisfactory results. Participants 009 and 015

presented questionable results.


Pag. 36 of 40

8. Analysis Testimony

As established in the proficiency testing protocol, CETESB representative witnessed one of the PT three

measurements at each laboratory participant and sent the results to the PT coordination. After finishing

the tests, each participant sent their results to the coordination for statistical treatment.

Tables 14 to 16 show the comparison between the results sent by CETESB and those sent by the

participants to PT coordination for all measurement cycles performed at the testimony day. It is worth

saying that all values sent by CETESB were previously discussed with each one of the participants after

the testimony.

8.1. Urban Cicle Emissions

Discrepancies were found in 3 of 400 results sent (0.75 %) for the urban cycle. Three among 20

participants of this PT round showed some discrepancy between the values sent by CETESB and those

informed by the participants to the PT coordination. These discrepancies are mainly due to rounding or

digitation errors by the participant when sending the results to PT coordination.

Table 14– Comparison between testimony results sent by CETESB to PT coordination and those sent

by the PT participants for the urban cycle parameter.

Code

NMHC Total aldehydes

(g/km) (g/km)

CETESB Lab CETESB Lab

030 - - 0.00116 0.00120

106 - - 0.00061 0.00059

065 - - 0.00090 0.00091

8.2. Evaporative Emissions

Discrepancy was found in 1 of 39 results sent (2.5 %) for evaporative emissions. One of the 13

participants of this PT round showed some discrepancy between the values sent by CETESB and those

informed by the participants to the PT coordination.

Table 15 – Comparison between testimony results sent by CETESB to PT coordination and those sent

by the PT participants for the evaporative emissions parameter.

Code

Evaporative

emissions

CETESB Lab

082 0.315 0.320


Pag. 37 of 40


Discrepancy was found in 1 of 480 results sent (0.2 %) for road cycle. One of the 20 participants of this

PT round showed only one discrepancy between the values sent by CETESB and those informed by the

participant to the PT coordination, but this participant did not report any discrepant result and the

discrepancy was due to the non-forwarding of the result by CETESB.

Table 16 – Comparison between testimony results sent by CETESB to PT coordination and those sent

by the PT participants for the road cycle parameter.

Code

Combined

autonomy (km/L)

CETESB Lab

073 - 9,99

*Cetesb did not send this testimony.

9. Confidenciality

Each participant was identified by an individual code which is only known by the participant and the PT

coordination. As stated on the registration form, the identification of accredited laboratories and

laboratories in stage of accreditation will be forwarded for information of the General Accreditation

Coordination (Cgcre). The participant received, by email, his own identification code corresponding to

the participation in this PT. This code was used to identify the participant in the results registration form.

The results may be used in studies and publications by Inmetro respecting the confidentiality of each

participant.

As established in section 4.10.4 of ABNT ISO/IEC 17043:2011, in exceptional circumstances, a

regulatory authority may require the results and the identification of the participants to the PT provider.

If this occurs, the provider will notify the PT participants about this action.

10. Conclusions

Proficiency Testing Schemes in vehicle emissions is a type of study carried out only in Brazil and,

considering the particular features of such study, we can conclude that the results are quite satisfactory

and this initiative is very important to the industry and society along these nine rounds held in

collaboration between Inmetro and AEA.


Pag. 38 of 40

This PT round involved a large number of variables and the testimony of a regulation body (CETESB).

This large numbers of variables in the vehicle emissions PT certainly have influence in the reported

results. Therefore it is recommended that participants that showed questionable performance to critically

evaluate their measurement methods.

Some discrepancies between the reported results during CETESB testimony and those sent to PT

coordination were observed. They were digitation or round errors that did not influence the quality of

the results, but these discrepancies denote a checking problem before sending the results to PT

coordination. It is an opportunity to improve the data checking procedures after the analysis.

It is worth saying that the established acceptance limits were lowered in the 8th round, as those results

out of Ref ± 2s limits were considered outliers. In other rounds, these results could be considered as

satisfactory and they became questionable or unsatisfactory due to the new criteria. In general, the results

in this 9th round continue to improve compared to the results of the previous round.

In consequence, there was an improvement in the performance of participants, considering that urban

cycle had 97.8 % of satisfactory results, 0.71 % of non-correctly reported or non-measured results,

0.71 % questionable results and other 0.71 % unsatisfactory results.

For the evaporative emissions cycle 92.4 % of the reported results were considered satisfactory and

7.6 % were considered unsatisfactory.

For road cycle, 89.3 % were considered satisfactory, 5.6 % were considered questionable and 5.0 %

were considered unsatisfactory.

It should be emphasized the importance of different laboratory participation in a proficiency test scheme,

since it constitutes an useful tool to monitor the procedures in routine analysis and to evaluate the

laboratory measurement results, enabling the improvement of the results quality and ensuring greater

reliability of the measurements.

It is up to PT participant to carry out a critical analysis of the results, where the entire process and

laboratory experience must be considered. Therefore, the continuous participation in a proficiency test

can assure information to the laboratory about the measurement capability and it is of great importance

for monitoring the validity of the results.


Pag. 39 of 40

11. Participants

Twenty-two laboratories were registered in the 9th round of the Proficiency Testing in Vehicles

Emissions, but two participants did not send their results report because of equipment problems and

informed it to the PT coordination. Thus, 20 participants remained.

The list of laboratories that sent results to this PT coordination is presented in Table 17. It is important

to note that the numbering of laboratories in the table only indicates the number of PT participants, under

no circumstances it is associated to laboratory identification in presenting their results.

Table 17– Participants.

Instituição

1. AVL SOUTH AMERICA LTDA

2. CAOA Montadora de Veículos

Centro de Pesquisas e Eficiência Energética

3. Continental Brasil Indústria Automotiva Ltda.

Laboratório de Emissões Veiculares – Centro Tecnológico “Geraldo Negri Rangel”

4. FCA Fiat Chrysler Automóveis Brasil Ltda.

Laboratório de Emissões e Consumo

5. FEV America Latina Ltda.

6. Ford Motor Company Brasil Ltda.

Laboratório de Emissões do Campo de Provas de Tatuí

7. General Motors do Brasil Ltda.

Laboratório de Emissões do Campo de Provas de Cruz Alta

8. General Motors do Brasil Ltda.

Laboratório de Emissões Veiculares – Global Propulsion Systems

9. Honda Automóveis do Brasil Ltda.

Laboratório de Emissões Honda Automóveis

10. Hyundai Motor Brasil Montadora de Automóveis Ltda.

Centro de Pesquisa e Desenvolvimento HMB

11. Instituto de Tecnologia para o Desenvolvimento – Institutos LACTEC

LEME – Laboratório de Emissões Veiculares

12. Magneti Marelli Sistemas Automotivos Indústria e Comércio Ltda.

13. Petróleo Brasileiro S.A.

Laboratório de Ensaios Veiculares - CENPES

14. Peugeot Citroen do Brasil Automóveis Ltda.

15. Renault do Brasil S/A

LEV – Laboratório de Emissões Veiculares

16. Robert Bosch Ltda.

Laboratório de emissões veiculares – Robert Bosch

17 SENAI – Serviço Nacional de Aprendizagem Industrial

LEV – CIT SENAI FIEMG Campus CETEC

18 Toyota do Brasil Ltda.

Laboratório de Emissões Indaiatuba


Pag. 40 of 40

Instituição

19 Umicore Brasil Ltda.

Laboratório de Emissões Veiculares - Umicore

20 Volkswagen do Brasil Ltda.

Laboratório de Emissões Veiculares da Volkswagen do Brasil Ltda.

Total participants: 20 laboratories.

12. References

ABNT NBR ISO/IEC 17025:2005: Requisitos gerais para a competência de laboratórios de ensaio e

calibração.

ABNT NBR ISO/IEC 17043:2011: Avaliação de Conformidade – Requisitos Gerais para ensaios de

proficiência.

ISO 13528:2015 (E), “Statistical methods for use in proficiency testing by interlaboratory

comparisons”.

Vocabulário Internacional de Metrologia: conceitos fundamentais e gerais e termos associados (VIM

2012). 1ª Edição Luso–Brasileira.

13. Review History

Corrections due to results of urban autonomy parameter in road cycle not sent by participant 045.

_______________

final report of the proficiency testing in vehicles ... · final report of the proficiency testing...

Documents