auditing and assessing the pms – government and contractor requirements presented by: dr. douglas...

Auditing and Assessing the PMS – Government and Contractor Requirements

Presented by:Dr. Douglas Goetz, CPPM, CF

NPMA Rocky Mountain Chapter19 May 2010

TASKING

• Over the past six months design develop and write a DOD Property Guide….

• What Professor Chuck Waszczak referred to as “The Property Book of Wisdom.”

• Its foundation – the DOD Property Manual Rewrite team:– Team Members were from the Air Force, Army,

DCMA, Navy, and NASA• The original rewrite died an ignominious death….

TASKING

• OSD tasked DAU to create six guides:– Property Administrators– Plant Clearance Officers– Program Managers– Procuring Contracting Officers– Administrative Contracting Officers– CORs/COTRs/TRCOs

Thinking!?!?!

• One Chapter of the PA Guide got me thinking!– In the words of Lyle Hestermann, it caused me to go,

“Hmmmmm?”– What were the FOUNDATIONAL ELEMENTS of an

ASSESSMENT? – What were the FOUNDATION ELEMENTS of an

AUDIT?– And were/are we RIGOROUSLY applying those

ELEMENTS?

FAR REQUIREMENTS

• GP CLAUSE– Requirement for a “SYSTEM”

• 52.245-1(b) Property management. – (1) The Contractor shall have a system to manage (control, use, preserve, protect,

repair and maintain) Government property in its possession.

– Requirement for “ASSESSMENT”• 52.245-1 (f) Contractor Plans and Systems

– (3) The Contractor shall establish and maintain procedures necessary to ASSESS its property management system effectiveness, and shall perform periodic internal REVIEWS and AUDITS.

– Allowance to “AUDIT”• 52.245-1(g) SYSTEM ANALYSIS

– (1) The Government shall have access to the contractor's premises and all Government property, at reasonable times, for the purposes of reviewing, inspecting and evaluating the Contractor's property management plan, systems, procedures, records

Contractor Requirement• ASSESSMENT of their PMS– What methods can the contractor use…

what exists out there today?

E 2452-05Standard Practice for

Equipment Management Process Maturity Model

Property Management

E-Tools

Plus Others!!!

Contractor Requirement

• There is no specific contractual requirement as to HOW the contractor is to “ASSESS,” “REVIEW” or “AUDIT” their PMS.– So long as they “assess” the outcomes/processes

set forth in the GP Clause or other requirements.• Contractors should determine the APPROPRIATE

methodology(ies) for their “environment.”• Ensure reliability, validity, and the ability to

replicate the findings.

• As our profession has matured the underlying concepts – and their accompanying semantics become more critical.

• The current perspective from DC (That 20 square miles

of land surrounded by reality) is that our work, i.e., our reviews need more rigor and consistency.

• Just listen to the Congressional Hearings on DCAA!

• What does this mean? Next Slide!!!

Government System “?????”What… Is it a survey or analysis or ???

Government System “?”ASP

R Sup

. 3

FOR DODYe’ Really Olden Days! Ye’ Recent Olden Days!

DCMAGUIDEBOOK

Current GuidanceFor DCMAers

Property ControlSYSTEM SURVEY

Property ControlSYSTEM ANALYSIS

Property ManagementSYSTEM ANALYSIS

DoDGUIDE

For the Performance Of Contract

Property Administration

Proposed GuidanceFor All of DOD

Property ManagementSYSTEM AUDIT

AUDIT REQUIREMENTS• If Property, within the Government, is going to

use the term AUDIT – they open themselves to the requirements of the AUDIT PROFESSION!

• AICPA

• IIA • GAGAS - (Generally Accepted Government Auditing Standards)

Are these VCSes/ILPs???

AUDIT PROTOCOLS

• PROCESS– Since the FAR has moved to a PROCESS orientation

it would be wise to view “Auditing” as a PROCESS.– And, since I am trying to move NPMA to a more

RESEARCH based body… Thought it would be good to support my contentions based upon RESEARCH and not opinion.

– So, think for a minute, looking at every reference I cited in earlier slides, are they not all based upon AUDIT PRINCIPLES and PRACTICES?

AUDIT PROTOCOLS

• The answer is yes!• So, what does that mean to every contractor

and every Government Employee involved with either ASSESSING their PMS or ANALYZING a PMS?

• The Answer…– We had best be well versed in AUDIT PROTOCOLS,

METHODOLOGIES, and… WAIT FOR IT….

STANDARDS!!!

AUDIT PROTOCOLS(Regardless of which methodology you use)

AUDIT PROCESS – what are the steps?

• Audit Objectives• Define the attributes or

transactions or characteristics

• Define the Population• Specify the acceptance

rejection rate• Sampling Methodology

• Select Sample• Perform the Audit• Analyze Sample• Determine Status• Issue report

Source:National Property Management Association,Intermediate Studies Manual, Clearwater, FL.2007. Pg. 7-2.

Please note that there are TONS of TextsOn the Auditing Process – encompassing VOLUMES!

So, how do I apply this?Let’s see – Some Examples!

• ASTM E 2452, 2005 Paragraph 8. Equipment Life-Cycle Phases

• 8.1.6 Receiving– 8.1.6.1 Identify and

resolve shipping discrepancies

– 8.1.6.2 Record and post receipts

• DoD Property Manual 4161.2-M (1991) App. A Function 3 – Receiving– Discrepancies

• 2. Causes of discrepancies are investigated and documented. II

• 3. Contractor initiates appropriate discrepancy reports….

– Receiving Process• 3. Receiving reports are

promptly prepared that document items and quantities received, etc., etc., etc.

Note: Neither of these documentsTell you the specific data elements Within each PROCESS. That is whereYour PROCESS REQUIREMENTS and PROCEDURES are critical!

Both of these items may apply steps within the AUDIT PROCESS.

• For example:– What is your population?– What is your sample size?– What specific DATA ELEMENTS will you evaluate/test under

this process:• Timeliness• Accuracy of data• Etc.

– How will you EVALUATE the “maturity” or “adequacy” of the sample/population?• The EMPM uses maturity levels (1-5) • The DoD Property Manual uses Confidence rates, i.e., acceptance

and rejection rates.Both take into consideration issues of Materiality & Significance.

THE ISSUE

• We in the property community, at least under DOD, have grown “comfortable” in doing our work ONE WAY, i.e., using the DoD Property Manual.

• In today’s environment we need:– The ability to adapt and change!– The ability to think and analyze – creatively!– Most importantly – the ability to LEARN!

We need to be well versed inA WIDE VARIETY of

AUDIT and ASSESSMENT PROTOCOLS!

An Interesting Quote• “There are known knowns. There are things we

know that we know. There are known unknowns. That is to say, there

are things that we know we don't know. But there are also unknown unknowns. There are

things we don't know we don't know.”• Donald Rumsfeld

• So, the concern is, what is the depth of your knowledge to properly apply an assessment or audit program/process?

Are you as comfortable with the Language of Auditing and Assessment as you are with Property?

Significance

MaterialityConfidence Rates

Sampling Audit Evidence

Attest

GeneralizabilityReliability

“Well, wait a minute Doug – this is all new, it’s different – and it ain’t easy!”

• What to do?

LEARN!NPMA

COURSESSTEVENS-

HENAGERCOURSES

READ!

DAU

COURSES

EVEN YOUR LOCAL COLLEGES…

CORPORATE PEERS

EVEN SELF-DIRECTED LEARNING

Now, if I were “King of the Forest” and was to design a curriculum for the Field…

• Foundations in Statistics• Statistical Process Control (SPC)– Including Metrics

• Auditing– With ALL of its embedded requirements

• Just take a look at the AICPA SAS or the DCAA Manuals

• Property Related Tools• Maturity Model Construct• AIA’s E-tools and Metrics

• Auditing and Assessment -- Relationship(s) to VCSesA few Tiny Questions - Why is it that DOD has just one two week on-line Property Audit Course yet DCAA and QA have extensive training & education requirements in auditing?And other Government agencies with Property responsibilities have NO training requirements….And contractors, you’re not innocent either. What does YOUR company require/support?

What’s the old adage?

• “Jack of all trades and master of none.”– Anonymous

• YES!!! We must have BREADTH to our knowledge and experience

• But… we must also have DEPTH to our knowledge!

Our Challenge here…

• Again, go out and learn – all that you can, such that you acquire and develop the knowledge, skills, and abilities to perform exceptional work in the area of PROPERTY Audits and Assessments.

THANK YOU!

Dr. Douglas N. Goetz, CPPM, CFOLD DAU PROFESSOR

Recently Retired – now Rehired.President of the newly incorporated GP Associates.

Hopefully a Good NPMA Instructor.Chief cook and bottle washer at Home, responsible for all “honeydue” requirements – though my wife has told me to stay out of HER house!

233 N. Maple AveFairborn, OH 45324

GPASSOCIATES@ATT.NET1- 888-576-0200