emissions offset project auditing as quality assurance discipline: a forest inventory perspective

© SCS Global Services | 1

Emissions offset project auditing as quality assurance discipline: a

forest inventory perspective2013 Western Mensurationists Meeting, Leavenworth, WA

25 June 2013

© Scientific Certification Systems | 2© SCS Global Services | 2

Three questions:

What do forest inventory professionals know about quality assurance?

What do auditors know about quality assurance?

Can quality assurance ideas from the world of auditing be of any interest in a forest inventory setting?

© Scientific Certification Systems | 3© SCS Global Services | 3

“Timber cruiser”, Ken Brauner, 1992

http://www.kenbrauner.com/home/kb1/page_477_218/timber_cruiser_art_print.html

© Scientific Certification Systems | 4© SCS Global Services | 4

http://www.zazzle.com

© Scientific Certification Systems | 5© SCS Global Services | 5

Emissions offset projects

http://commons.wikimedia.org/wiki/File:Smokestacks_3958.jpg http://commons.wikimedia.org/wiki/File:Douglas_Firs_Mount_Hood_National_Forest.jpg

© Scientific Certification Systems | 6© SCS Global Services | 6

The number(usually expressed in metric tonnes CO2-equivalent)

© Scientific Certification Systems | 7© SCS Global Services | 7

The number(usually expressed in metric tonnes CO2-equivalent)

•Spreadsheets•Access/SQL Server databases•Scripted processes•Modeling software (e.g., growth-and-yield models)

© Scientific Certification Systems | 9© SCS Global Services | 9

(Adapted from financial auditing methods via ISO 14064-3:2006)

A framework for quality assurance

© Scientific Certification Systems | 10© SCS Global Services | 10

Complexity of the system

Multiple forest inventories?

Different growth models?

© Scientific Certification Systems | 11© SCS Global Services | 11

Key risks in the system

Manual transfer errors

Spreadsheet calculation errors

Errors in software operation

© Scientific Certification Systems | 12© SCS Global Services | 12

Quality control processes

Built-in data validation?

Manual quality control checks?

Sniff test?

© Scientific Certification Systems | 13© SCS Global Services | 13

Remaining areas of risk

Focus on biggest areas of “residual” risk

Likelihood of residual risk determines how much to check

© Scientific Certification Systems | 14© SCS Global Services | 14

Tools for data testing

© Scientific Certification Systems | 15© SCS Global Services | 15

Vouching

© Scientific Certification Systems | 16© SCS Global Services | 16

Retracing

© Scientific Certification Systems | 17© SCS Global Services | 17

Recomputation

© Scientific Certification Systems | 18© SCS Global Services | 18

The number(usually expressed in metric tonnes CO2-equivalent)

•Spreadsheets•Access/SQL Server databases•Scripted processes•Modeling software (e.g., growth-and-yield models)

•Measured forest inventory data•Assumptions/parameters•Other input values

© Scientific Certification Systems | 19© SCS Global Services | 19

Tools from emissions offset auditing can:

Help us to strategically assess our own quantitative procedures for weakness and possible improvements.

Facilitate critical evaluation of data originating from within other organizations (or even our own organization)

© Scientific Certification Systems | 20© SCS Global Services | 20

Questions?

Zane Haxtema zhaxtema@scsglobalservices.com

+1-510-292-5968 direct zane.haxtema Skype

© Scientific Certification Systems | 21© SCS Global Services | 21

Supplementary Slides

© Scientific Certification Systems | 22© SCS Global Services | 22

(Adapted from financial auditing principles*)

A sample size framework

MAR = AR / (IR * CR)

MAR = Max. audit risk (risk that errors not caught by audit)

AR = allowable risk (acceptable risk of a “false positive”)

IR = Inherent risk (inherent risk of errors occurring)

CR = Control risk (risk that errors not caught by controls)

*As specifically described in PCAOB AU Section 350, accessed online 13 June 2013 at http://pcaobus.org/standards/auditing/pages/au350.aspx

© Scientific Certification Systems | 23© SCS Global Services | 23

(Adapted from financial auditing principles*)

A sample size framework

MAR = AR / (IR * CR)

MAR = Max. audit risk (risk that errors not caught by audit)

AR = allowable risk (acceptable risk of a “false positive”)

IR = Inherent risk (inherent risk of errors occurring)

CR = Control risk (risk that errors not caught by controls)

*As specifically described in PCAOB AU Section 350, accessed online 13 June 2013 at http://pcaobus.org/standards/auditing/pages/au350.aspx

© Scientific Certification Systems | 24© SCS Global Services | 24

(Adapted from financial auditing principles*)

A sample size framework

MAR = AR / (IR * CR)

MAR = Max. audit risk (risk that errors not caught by audit)

AR = allowable risk (acceptable risk of a “false positive”)

IR = Inherent risk (inherent risk of errors occurring)

CR = Control risk (risk that errors not caught by controls)

*As specifically described in PCAOB AU Section 350, accessed online 13 June 2013 at http://pcaobus.org/standards/auditing/pages/au350.aspx

© Scientific Certification Systems | 25© SCS Global Services | 25

(Adapted from financial auditing principles*)

A sample size framework

MAR = AR / (IR * CR)

MAR = Max. audit risk (risk that errors not caught by audit)

AR = allowable risk (acceptable risk of a “false positive”)

IR = Inherent risk (inherent risk of errors occurring)

CR = Control risk (risk that errors not caught by controls)

*As specifically described in PCAOB AU Section 350, accessed online 13 June 2013 at http://pcaobus.org/standards/auditing/pages/au350.aspx

© Scientific Certification Systems | 26© SCS Global Services | 26

(Adapted from financial auditing principles*)

A sample size framework

MAR = AR / (IR * CR)

MAR = Max. audit risk (risk that errors not caught by audit)

AR = allowable risk (acceptable risk of a “false positive”)

IR = Inherent risk (inherent risk of errors occurring)

CR = Control risk (risk that errors not caught by controls)

*As specifically described in PCAOB AU Section 350, accessed online 13 June 2013 at http://pcaobus.org/standards/auditing/pages/au350.aspx

© Scientific Certification Systems | 27© SCS Global Services | 27

(Adapted from financial auditing principles*)

A sample size framework

MAR = AR / (IR * CR)

MAR = Max. audit risk (risk that errors not caught by audit)

AR = allowable risk (acceptable risk of a “false positive”)

IR = Inherent risk (inherent risk of errors occurring)

CR = Control risk (risk that errors not caught by controls)

*As specifically described in PCAOB AU Section 350, accessed online 13 June 2013 at http://pcaobus.org/standards/auditing/pages/au350.aspx

© Scientific Certification Systems | 28© SCS Global Services | 28

An example...

Allowable risk = 5%

Inherent risk = 50%

Control risk = 10%

Maximum audit risk = 5% / (50% * 10%) = 100%

© Scientific Certification Systems | 29© SCS Global Services | 29

(Adapted from financial auditing principles*)

A sample size framework

MAR = AR / (IR * CR)

MAR = Max. audit risk (risk that errors not caught by audit)

AR = allowable risk (acceptable risk of a “false positive”)

IR = Inherent risk (inherent risk of errors occurring)

CR = Control risk (risk that errors not caught by controls)

*As specifically described in PCAOB AU Section 350, accessed online 13 June 2013 at http://pcaobus.org/standards/auditing/pages/au350.aspx

© Scientific Certification Systems | 30© SCS Global Services | 30

FactorConditions leading to smaller sample

size

Conditions leading to larger sample

size

Inherent risk Lower risk Higher risk

Control risk Lower risk Higher risk

Number of items in the population

“Virtually no effect on sample size unless population is very small”

Adapted from Table 1, PCAOB AU Section 350, accessed online 13 June 2013 at http://pcaobus.org/standards/auditing/pages/au350.aspx