auditing and assessing the pms – government and contractor requirements presented by: dr. douglas...
TRANSCRIPT
Auditing and Assessing the PMS – Government and Contractor Requirements
Presented by:Dr. Douglas Goetz, CPPM, CF
NPMA Rocky Mountain Chapter19 May 2010
TASKING
• Over the past six months design develop and write a DOD Property Guide….
• What Professor Chuck Waszczak referred to as “The Property Book of Wisdom.”
• Its foundation – the DOD Property Manual Rewrite team:– Team Members were from the Air Force, Army,
DCMA, Navy, and NASA• The original rewrite died an ignominious death….
TASKING
• OSD tasked DAU to create six guides:– Property Administrators– Plant Clearance Officers– Program Managers– Procuring Contracting Officers– Administrative Contracting Officers– CORs/COTRs/TRCOs
Thinking!?!?!
• One Chapter of the PA Guide got me thinking!– In the words of Lyle Hestermann, it caused me to go,
“Hmmmmm?”– What were the FOUNDATIONAL ELEMENTS of an
ASSESSMENT? – What were the FOUNDATION ELEMENTS of an
AUDIT?– And were/are we RIGOROUSLY applying those
ELEMENTS?
FAR REQUIREMENTS
• GP CLAUSE– Requirement for a “SYSTEM”
• 52.245-1(b) Property management. – (1) The Contractor shall have a system to manage (control, use, preserve, protect,
repair and maintain) Government property in its possession.
– Requirement for “ASSESSMENT”• 52.245-1 (f) Contractor Plans and Systems
– (3) The Contractor shall establish and maintain procedures necessary to ASSESS its property management system effectiveness, and shall perform periodic internal REVIEWS and AUDITS.
– Allowance to “AUDIT”• 52.245-1(g) SYSTEM ANALYSIS
– (1) The Government shall have access to the contractor's premises and all Government property, at reasonable times, for the purposes of reviewing, inspecting and evaluating the Contractor's property management plan, systems, procedures, records
Contractor Requirement• ASSESSMENT of their PMS– What methods can the contractor use…
what exists out there today?
E 2452-05Standard Practice for
Equipment Management Process Maturity Model
Property Management
E-Tools
Plus Others!!!
Contractor Requirement
• There is no specific contractual requirement as to HOW the contractor is to “ASSESS,” “REVIEW” or “AUDIT” their PMS.– So long as they “assess” the outcomes/processes
set forth in the GP Clause or other requirements.• Contractors should determine the APPROPRIATE
methodology(ies) for their “environment.”• Ensure reliability, validity, and the ability to
replicate the findings.
• As our profession has matured the underlying concepts – and their accompanying semantics become more critical.
• The current perspective from DC (That 20 square miles
of land surrounded by reality) is that our work, i.e., our reviews need more rigor and consistency.
• Just listen to the Congressional Hearings on DCAA!
• What does this mean? Next Slide!!!
Government System “?????”What… Is it a survey or analysis or ???
Government System “?”ASP
R Sup
. 3
FOR DODYe’ Really Olden Days! Ye’ Recent Olden Days!
DCMAGUIDEBOOK
Current GuidanceFor DCMAers
Property ControlSYSTEM SURVEY
Property ControlSYSTEM ANALYSIS
Property ManagementSYSTEM ANALYSIS
DoDGUIDE
For the Performance Of Contract
Property Administration
Proposed GuidanceFor All of DOD
Property ManagementSYSTEM AUDIT
AUDIT REQUIREMENTS• If Property, within the Government, is going to
use the term AUDIT – they open themselves to the requirements of the AUDIT PROFESSION!
• AICPA
• IIA • GAGAS - (Generally Accepted Government Auditing Standards)
Are these VCSes/ILPs???
AUDIT PROTOCOLS
• PROCESS– Since the FAR has moved to a PROCESS orientation
it would be wise to view “Auditing” as a PROCESS.– And, since I am trying to move NPMA to a more
RESEARCH based body… Thought it would be good to support my contentions based upon RESEARCH and not opinion.
– So, think for a minute, looking at every reference I cited in earlier slides, are they not all based upon AUDIT PRINCIPLES and PRACTICES?
AUDIT PROTOCOLS
• The answer is yes!• So, what does that mean to every contractor
and every Government Employee involved with either ASSESSING their PMS or ANALYZING a PMS?
• The Answer…– We had best be well versed in AUDIT PROTOCOLS,
METHODOLOGIES, and… WAIT FOR IT….
STANDARDS!!!
AUDIT PROTOCOLS(Regardless of which methodology you use)
AUDIT PROCESS – what are the steps?
• Audit Objectives• Define the attributes or
transactions or characteristics
• Define the Population• Specify the acceptance
rejection rate• Sampling Methodology
• Select Sample• Perform the Audit• Analyze Sample• Determine Status• Issue report
Source:National Property Management Association,Intermediate Studies Manual, Clearwater, FL.2007. Pg. 7-2.
Please note that there are TONS of TextsOn the Auditing Process – encompassing VOLUMES!
So, how do I apply this?Let’s see – Some Examples!
• ASTM E 2452, 2005 Paragraph 8. Equipment Life-Cycle Phases
• 8.1.6 Receiving– 8.1.6.1 Identify and
resolve shipping discrepancies
– 8.1.6.2 Record and post receipts
• DoD Property Manual 4161.2-M (1991) App. A Function 3 – Receiving– Discrepancies
• 2. Causes of discrepancies are investigated and documented. II
• 3. Contractor initiates appropriate discrepancy reports….
– Receiving Process• 3. Receiving reports are
promptly prepared that document items and quantities received, etc., etc., etc.
Note: Neither of these documentsTell you the specific data elements Within each PROCESS. That is whereYour PROCESS REQUIREMENTS and PROCEDURES are critical!
Both of these items may apply steps within the AUDIT PROCESS.
• For example:– What is your population?– What is your sample size?– What specific DATA ELEMENTS will you evaluate/test under
this process:• Timeliness• Accuracy of data• Etc.
– How will you EVALUATE the “maturity” or “adequacy” of the sample/population?• The EMPM uses maturity levels (1-5) • The DoD Property Manual uses Confidence rates, i.e., acceptance
and rejection rates.Both take into consideration issues of Materiality & Significance.
THE ISSUE
• We in the property community, at least under DOD, have grown “comfortable” in doing our work ONE WAY, i.e., using the DoD Property Manual.
• In today’s environment we need:– The ability to adapt and change!– The ability to think and analyze – creatively!– Most importantly – the ability to LEARN!
We need to be well versed inA WIDE VARIETY of
AUDIT and ASSESSMENT PROTOCOLS!
An Interesting Quote• “There are known knowns. There are things we
know that we know. There are known unknowns. That is to say, there
are things that we know we don't know. But there are also unknown unknowns. There are
things we don't know we don't know.”• Donald Rumsfeld
• So, the concern is, what is the depth of your knowledge to properly apply an assessment or audit program/process?
Are you as comfortable with the Language of Auditing and Assessment as you are with Property?
Significance
MaterialityConfidence Rates
Sampling Audit Evidence
Attest
GeneralizabilityReliability
“Well, wait a minute Doug – this is all new, it’s different – and it ain’t easy!”
• What to do?
LEARN!NPMA
COURSESSTEVENS-
HENAGERCOURSES
READ!
DAU
COURSES
EVEN YOUR LOCAL COLLEGES…
CORPORATE PEERS
EVEN SELF-DIRECTED LEARNING
Now, if I were “King of the Forest” and was to design a curriculum for the Field…
• Foundations in Statistics• Statistical Process Control (SPC)– Including Metrics
• Auditing– With ALL of its embedded requirements
• Just take a look at the AICPA SAS or the DCAA Manuals
• Property Related Tools• Maturity Model Construct• AIA’s E-tools and Metrics
• Auditing and Assessment -- Relationship(s) to VCSesA few Tiny Questions - Why is it that DOD has just one two week on-line Property Audit Course yet DCAA and QA have extensive training & education requirements in auditing?And other Government agencies with Property responsibilities have NO training requirements….And contractors, you’re not innocent either. What does YOUR company require/support?
What’s the old adage?
• “Jack of all trades and master of none.”– Anonymous
• YES!!! We must have BREADTH to our knowledge and experience
• But… we must also have DEPTH to our knowledge!
Our Challenge here…
• Again, go out and learn – all that you can, such that you acquire and develop the knowledge, skills, and abilities to perform exceptional work in the area of PROPERTY Audits and Assessments.
THANK YOU!
Dr. Douglas N. Goetz, CPPM, CFOLD DAU PROFESSOR
Recently Retired – now Rehired.President of the newly incorporated GP Associates.
Hopefully a Good NPMA Instructor.Chief cook and bottle washer at Home, responsible for all “honeydue” requirements – though my wife has told me to stay out of HER house!
233 N. Maple AveFairborn, OH 45324
[email protected] 888-576-0200