real world consulting organizational performance measurement
DESCRIPTION
Mark Tregar, MS. Real World Consulting Organizational Performance Measurement. Agenda. Introduction and Bio Career Path What is Consulting Really? Example of Organizational Performance Management Project Questions?. Mark Tregar - Bio. Levels Across Firms: An example of my career path. 4. - PowerPoint PPT PresentationTRANSCRIPT
© 2013 IBM Corporation
Real World ConsultingOrganizational Performance Measurement
Mark Tregar, MS
© 2013 IBM Corporation
Agenda
1. Introduction and Bio2. Career Path3. What is Consulting Really?4. Example of Organizational Performance Management Project5. Questions?
© 2013 IBM Corporation
Mark Tregar - Bio
© 2013 IBM Corporation
Levels Across Firms: An example of my career path
CNA
Leaders
Analysts
Associate
Specialist
Assistant
IBM
Partners (B10+)
Associate Partners (B10)
Sr. Managing Consultant (B9)
Managing Consultant (B8)
Senior Consultant (B7)
Consultant (B6) – CBD Prog
Analyst/Assistant
Deloitte
Partner/Principal/Director
Sr. Manager
Manager
Senior Consultant
Consultant
Analyst
1
23
4
• My career trajectory has spanned 3 organizations in the past 8 years.• Possible UB graduates entry point: Between consultant and senior consultant!
© 2013 IBM Corporation
Why Companies Hire Consultants
1)Staff augmentation
2)External change force aka “political cover”
3)Best practices across industries and functions
4)Analytical horsepower
5)Fresh perspective
6)Training and skillset growth of staff
Management consulting is the practice of helping organizations to improve their performance, primarily through the analysis of existing organizational problems and development of plans for improvement. Organizations may draw upon the services of management consultants for a number of reasons, including gaining external (and presumably objective) advice and access to the consultants' specialized expertise. http://en.wikipedia.org/wiki/Management_consulting
A Review of Performance Measures Work at USAID
November 21, 2013
USAID HR LOB CHRRPS Technical Integration Support Project
© 2013 IBM Corporation
Table of Contents
1. Background 8
2. Purpose and Approach 9
1. Conceptual Design 9
2. Detailed Design 10
3. Data Gathering 13
4. Data Reporting and Analysis 14
3. Next Steps and Lessons Learned 16
© 2013 IBM Corporation
The purpose of the Performance Measure effort was to aid the measurement of the overall health of HR operations.
The team identified a series of specific Key Performance Indicators (KPIs) and then selected the appropriate KPIs to provide an accurate measure of current HR operations for USAID.– The identified KPIs were reported and measured in a one-time scorecard using
Balanced Scorecard-based categories
– Our approach to selecting KPIs was adapted from the 2009 OPM Shared Service Center (SSC) Human Resource Line of Business (HR LOB) Benchmarking Process, which was piloted with 8 agencies and expanded to 12 in 2010
Conceptual Design
Detailed Design
Data Gathering
Data Reporting
and Analysis
Del
iver
abl
esSt
eps
Nov Dec-May May-Jul Jul-Sep
ID of 10 OHR Process and HR LOB System KPIs
Delivered: 5/24/13
Scorecard Template
Delivered: 12/12/12
Initial Performance Report
Delivered: 8/9/13
Figure 1. Our 4-Step Approach Adapted from OPM’s SSC HR LOB Benchmarking Steps
© 2013 IBM Corporation
In Conceptual Design, IBM developed a conceptual model Scorecard Template and gathered feedback on inclusions.
Level 1 - Top OHR USAID Management Report/ Scorecard reports high-level results (i.e., Customer Service) with an overall status for a given time period.
Level 2 - HR Management and Staff Report/Scorecard reports at a lower level of results (i.e., results for a specific metric) for a given time period.
We envisioned a future state report/scorecard with two levels and four Balanced Scorecard Categories:
© 2013 IBM Corporation
In Detailed Design, the team refined our original list of 135 KPIs, to a smaller set that were most relevant to USAID.
We cut our list to 49 preliminary KPIs* based on initial performance measure research and a series of meetings with key stakeholders.
– Our original list of 135 was an agency list with additional potential KPIs from IBM’s Benchmarking wizard
– In addition, we added KPIs based on our first focus group which included HR Staff members and meetings across HR.
To gather feedback we conducted workshops with the following groups: HR leaders, HR staff (Customer Service Group), 3 HR teams, and one contractor team.
135KPIs
54KPIs
End Result:
Final KPIs selected and then
measured
* The umber of KPIs ranged between 26 and 55 throughout the stakeholder engagement process.
45 KPIs
49* KPIs
© 2013 IBM Corporation
In Detailed Design, we used a series of questions to help leadership stakeholders cut KPIs
Is the potential measure
Actionable?
Is the potential measure
Applicable?
Is the potential measure
Meaningful?
Is the potential measure Detailed?
Is the potential measure Easily Understood?
The data is under control of the agency (or other provider) currently available and practical to collect.
The measure is relevant to government processes and supports USAID HR goals.
The data will be meaningful to decision makers and the derived value is greater than cost of collecting.
The scope of the measure is defined and is quantifiable to the extent possible.
The data measured is clear and is easy to understand.
Discussion Items:
1.Which KPIs are most valuable for selection and use? If so, why?
2.Which KPIs should be removed from our potential list?
Example KPI: HR Servicing Ratio - This metric measures the number of employees receiving HR services from Agency HR employees. The metric provides insight into the size and support level of HR.
UB Discussion: What are some others that could be important to an agency?
Figure 1. Items to consider in reviewing KPIs:
© 2013 IBM Corporation
Following Detailed Design discussions, the IBM team rated on Relevance and Data Availability
The feedback we received from the HR Director and targeted HR groups helped us understand the key priorities for OHR.
• Relevance Rating: (Likert scale of 1-5)– 1: Not Relevant to OHR Priorities– 2: Little Relevance to OHR Priorities– 3: Some Relevance to OHR Priorities– 4: Moderate Relevance to OHR Priorities – 5: Very Relevant to OHR Priorities
• Data Availability Rating: (Likert scale of 1-5)– 1: Data is not available in any format– 2: Data is available sporadically and is maintained less than monthly– 3: Data is available but in a static spreadsheet format– 4: Data is fully available in a database and available to the team indirectly – 5: Data is fully available in a database and available to the team directly
• Based on this understanding of USAID, we rated each KPI on Relevance and Data Availability
• The sum of these two ratings forms the Overall KPI Usage Score
• Each KPI was also cataloged into one of 12 categories (e.g., Time to Hire, Processing) to help understand KPI type
• The final KPI determination for use was based on Overall KPI Usage Score and representation in 12 categories.
• This allowed for targeting KPIs with available data, while avoiding oversampling in categories
Relevance Rating + Data Availability Rating = Overall KPI Usage Score
© 2013 IBM Corporation
In Data Gathering, the team met with groups such as HR Training and Education (HR T&E) to locate and collect data
• In USAID, each KPI had a specific HR owner who had access to raw data and could provide data to the team.
• The team interviewed the data owners and tracked the status of data collection, the data source (identification of specific data base), and who could help us collect the data.
• The team also tracked the specific parameters of each data element (e.g., the date range, and specific queries used in WA, Peoplesoft, or other databases) in order to validate the metrics at a later date
List of Potential T&E KPIs Used in Discussion:
Automated ILTP Metrics, for example:-Percentage of staff currently entering ILTPs
Post-Course Training Evaluation Ratings
Percentage of students who feel the knowledge they gained from the training will be applied on their job
Percentage of students who feel training will improve their job performance
Percentage of staff taking USAID University Training
Sample KPI Discussion Questions:•In our current list of KPIs (right), which KPIs do you currently track?•If you track this information, where is it currently stored (e.g., Excel, a database)?•What metrics do you measure monthly? Annually?•Which KPIs are most important to your team? Least important? •In your ideal scorecard, which KPIs would be reported?
© 2013 IBM Corporation
In Data Reporting and Analysis, the team successfully collected and analyzed all data
• Each KPI was reported across 3 periods of time• We compared each result to IBM’s Benchmark in terms of median and a
specific public sector median.• The data was shared with the HR director in the form of the Initial
Performance Report. The HR Director reported our findings in multiple sessions with her Agency Leadership Council (ALC) and the USAID Administrator, Dr. Rajiv Shah
• Similar data analysis can be found in annual HR LOB Reports– http://www.opm.gov/services-for-agencies/hr-line-of-business/benchmarking/payroll-be
nchmarking/2011report.pdf– http://www.opm.gov/services-for-agencies/hr-line-of-business/benchmarking/hr-benchm
arking/2010report.pdf
© 2013 IBM Corporation
Sample of results in 2010 OPM HR LOB Report
– Agency HR Servicing Ratio: This metric measures the number of employees receiving HR services from Agency HR employees. The metric provides insight into the size and support level of HR.
– Source: MAESC 2010 HR LOB Benchmarking Report - http://www.opm.gov/services-for-agencies/hr-line-of-business/benchmarking/hr-benchmarking/2010report.pdf
© 2013 IBM Corporation
HR Leaders are moving forward with a new BI Tool Deployment
Next Steps•HR leaders are moving forward with BI Tool Deployment based on our ideas
•We hope to work with USAID later this year
Lessons Learned
Do:– Get leadership’s input and approval before discussing the effort with other leaders and
managers.– Use multiple ways of gathering information. Even if you are promised data, it may not
be available as needed.– Create ratings based on your understanding of KPIs, and work to validate this
understanding throughout the process.– Encourage transparency with the initiative’s goals, expected outcomes, and level of
accountability.
Don’t:– Hold up your progress for one person.– Look for one person to have all the answers.– Commit to more KPIs or data collection than necessary for your primary objective.
© 2013 IBM Corporation
PBC 2012 General Information Utilization PA Results
Name:
Title:
Band/Year in Band:
SAM Name:
Are you a SAM?
Client Delivery Business Development Lead or Mentor People
Picture Here
SAM Area: Administrative Compliance:
PA Score Average: Recommended PBC Rating:
Practice Giveback Thought Leadership
PA Name ResultsRelation-ships
Prof Eff Leading Bus &
FMChargeable Target:
Chargeable:
Total (110% Target):
Comments:
Comments: Client Proposal (Type Work)
Individual Contribution
TCV Result Current Client
Role (months)
# of resources led/mentored
2nd Client
Role (months)
# of resources led/mentored
Giveback Role DateThought Leadership/Strategic
ContributionRole Date
Comments:
Comments:
Yes No
Comments: Comments:
Comments:
90%
59%
• Led USAID’s Performance Measures development task,
while providing competency management expertise in the launch of the USAID LMS.• Received positive client feedback for USAID performance such as: ”I've been impressed with the quality of his work products and his ability to dig right in…and figure it out.“• Provide project management expertise at USAID including Financial and labor planning, and creating Project Launch documentation.• Led WellPoint communications team in a complex and competitive environment.• Led CPAC team delivering all deliverables on time and on budget, and developed executable plans for future work• Analyzed and presented performance management statistics to NASA HR leaders and received high client praise.
Mark Tregar
Managing Consultant
8 / 9 MonthsStephen Hutton
USAID
WellPoint
1-2
Team Lead / DPM
Team Lead
1
Contributed to IBM with work on over 10 proposals including VA HR LOB, OPM TMA, DHS S&T SETA IDIQ, Dept. of Energy, VA VISN 5, and State of New York.
Overall, I was very successful in my first 9 months including solid client contributions, as well as B&P and MOS project success. At USAID, I received “O” ratings and high client satisfaction. At WellPoint, I stabilized a challenging client environment which supported our win. At CPAC, I completed all deliverables on time and on budget.
3rd Client – VA CPAC, Role: Team Lead; Resources led/mentored: 1Provided mentorship and leadership to all lower level staff, as well as clear and consistent feedback on all products and deliverables.
x 112%
costs) at DoE HQ.
Developed an improved Public Sector Competency Development Methodology
Project 517/Career Pro – Competency Team Lead
Lead
Team Lead
Sep-Jan 2013
Apr-Aug 2012
Social Media for Human Capital White Paper
Department of Energy (DoE) Vforce
Lead & Author
Contributor
Nov-2012 (In Review)
April 2012
OPM TMA (Recompete)
DHS S&T SETA IDIQ
Dept. of Energy (New)
Section Lead (Cost Estimate, Admin)
Section Lead
Contributor
Section Lead
Win
TBD
TBD
TBD
USAID
WellPoint
O
VG
O
VG
O
O
O
O
O
VG
Provided expert input across Federal B&P efforts: Led the VA HR LOB cost proposal input development by interviewing proposal leaders, and facilitated large proposal editing sessions. I authored a section of the OPM TMA proposal and co-authored the VA Social Media RFI response. I provided research and analysis for the Dept. of Energy solution and DHS S&T SETA IDIQ response.
VA HR LOB (New)
© 2013 IBM Corporation
Overview of Competency Based Management
Knowledge - What you know about the work.
Skills - Individual know-how and techniques related to the work.
Ability - Your capability and capacity to accomplish the work.
Behaviors & Characteristics - Your attitude and state of mind.
Competencies embody an individual’s total experience, including both training and education.
A competency is an observable, measurable pattern of skills, knowledge, abilities, behaviors and other characteristics that an individual needs to perform work roles or occupational functions successfully.
19
© 2013 IBM Corporation20
Relationship between KSAs and Competencies
Competency: Internal Audit and ControlReview and validate
accounting and administrative controls to safeguard the integrity of programs and
achieve compliance in accordance with government
auditing standards
Knowledge
Understand accounting
and administrative
controls
Abilities
Able to audit standards
when dealing with financial information
SkillsOperate
internal audit software
programs to validate
compliance
A competency is more than just the sum of knowledge, skills, and abilities
A competency is a “bundle” of the KSAs associated with a given area of work. It is the description of the observable behaviors that makes it different from a single K, S, or A.
© 2013 IBM Corporation21
Competency Development Process
• Identify data collection methodology to meet project objectives
• Identify proper mix of interviews, focus groups, and surveys
• Identify & Select SMEs• Work with sponsor to
identify Resource Panel SMEs and Incumbent SMEs
• Identify required SME population details (i.e., numbers of staff, occupations, levels)
• Initiate effort by inviting attendees to Resource Panel
• Support effort with the use of an executive memo or message of support for effort
• Identify data collection methodology to meet project objectives
• Identify proper mix of interviews, focus groups, and surveys
• Identify & Select SMEs• Work with sponsor to
identify Resource Panel SMEs and Incumbent SMEs
• Identify required SME population details (i.e., numbers of staff, occupations, levels)
• Initiate effort by inviting attendees to Resource Panel
• Support effort with the use of an executive memo or message of support for effort
• Develop/refine competency model using Resource Panels
• Hold 2-3 facilitated sessions with SMEs
• Output: Straw man competency model
• Refine competency model using Incumbent SMEs via focus group or survey [initial validation]
• Refine model using SME input (i.e., ratings, behavioral examples) and factor analysis (100+ SMEs required)
• Output: Interim Competency Model
• Develop final competency model
• Analyze data and develop final model
• Present final model to sponsor
• Output: Final Competency Model
• Develop/refine competency model using Resource Panels
• Hold 2-3 facilitated sessions with SMEs
• Output: Straw man competency model
• Refine competency model using Incumbent SMEs via focus group or survey [initial validation]
• Refine model using SME input (i.e., ratings, behavioral examples) and factor analysis (100+ SMEs required)
• Output: Interim Competency Model
• Develop final competency model
• Analyze data and develop final model
• Present final model to sponsor
• Output: Final Competency Model
• Define Objectives• Build business case for
competency model• Link model to strategic
plan
• Collect Background Materials
• Request existing competency models, position descriptions, etc.
• Review previous related IBM competency models
• Research and Review• Review strategic plan, and
organizational background• Review related
occupational models
• Develop and Execute Communications Plan to Build Sponsor Support
• Deliver kick-off brief• Collect feedback and
create additional briefings
• Define Objectives• Build business case for
competency model• Link model to strategic
plan
• Collect Background Materials
• Request existing competency models, position descriptions, etc.
• Review previous related IBM competency models
• Research and Review• Review strategic plan, and
organizational background• Review related
occupational models
• Develop and Execute Communications Plan to Build Sponsor Support
• Deliver kick-off brief• Collect feedback and
create additional briefings
• Conduct final validation survey with Incumbent SMEs for preparation in deeper applications
• Develop and refine competency survey behavioral anchors
• Execute competency validation survey
• Note: This can double as a competency assessment (including proficiency items)
• Output: Validated Competency Model
• Apply model in workforce applications
• Includes workforce & succession planning, selection*, training needs assessment)
• Conduct final validation survey with Incumbent SMEs for preparation in deeper applications
• Develop and refine competency survey behavioral anchors
• Execute competency validation survey
• Note: This can double as a competency assessment (including proficiency items)
• Output: Validated Competency Model
• Apply model in workforce applications
• Includes workforce & succession planning, selection*, training needs assessment)
Plan InitiateDevelop and
RefineValidate and
Apply
Plan, envision, and communicate competency effort
Plan, envision, and communicate competency effort
Select and initiate specific competency modeling approach
Select and initiate specific competency modeling approach
Develop and refine competency modelsDevelop and refine competency models
Validate competency models and apply models in various applications
Validate competency models and apply models in various applications
11
22
33
© 2013 IBM Corporation22
• Behavioral Event Interviews (BEI) are a type of structured interview use for collecting competency data from SMEs
• BEIs can be used to collect additional qualitative information for use in competency model development.
• BEIs often use a Situation-Task-Action-Reasoning/Results format in response to a prompt (i.e., Tell me about a time when…)
• Behavioral Event Interviews (BEI) are a type of structured interview use for collecting competency data from SMEs
• BEIs can be used to collect additional qualitative information for use in competency model development.
• BEIs often use a Situation-Task-Action-Reasoning/Results format in response to a prompt (i.e., Tell me about a time when…)
Situation/Task
What was the situation or
context? What were you doing? What task were you working on?
Action Reasoning
What was the reasoning/
rationale that led to the action?
Results
What was the result/
outcome of the key
situation?
What did you do? What were the
steps you took to get to that effective
outcome?
An Example Of Data Collection for Competency Modeling
© 2013 IBM Corporation23
Class Discussion
1. Why do IBM consultants exist? What is their job – what is the end result for consultants?
•What types of competencies are useful?
•What are lacking in the workforce today?
Think of what this looks like on the job: Think of a recent situation where you helped a client, where you saw this happen (i.e., at work, in a movie). What types of behaviors helped? How would you classify those behaviors?
© 2013 IBM Corporation
As you move up the career leader there is an expectation of deeper involvement, independence, and relationship building.
Project Planning
Work Product Development
Customer Management
What is this chart for?Many beginning practitioners do not have a solid idea as to what is expected of them
based on their particular level. This chart make the expectations more clear.
Communication
© 2013 IBM Corporation25
Applying Competency Models in Recruiting/Hiring
Behavioral Interviewing: Once a validated model is created with a full set of
behavioral anchors (i.e., a model with 1, 3, 5
benchmarks) you can create behaviorally-based
interview questions.
Behavioral Interviewing: Once a validated model is created with a full set of
behavioral anchors (i.e., a model with 1, 3, 5
benchmarks) you can create behaviorally-based
interview questions.
1-Not Capable1-Not
Capable
• Understand asset management principles to budgeting and programming projects
• Know about available tools (unit price books, other contract estimate tools) in order to conduct complex end-to-end cost estimates
• Understand asset management principles to budgeting and programming projects
• Know about available tools (unit price books, other contract estimate tools) in order to conduct complex end-to-end cost estimates
2-Minimally Capable
2-Minimally Capable
3-Capable3-Capable
4-Fully Capable4-Fully Capable
5-Exceeds Capability
5-Exceeds Capability
Competency: Financial ManagementInterview Question: “In regards financial management, can you tell me a time when you dealt with a large budget in a complex project that was delivered on time and on budget?
Please be specific as to size of budget, team and any constraints.”
Competency: Financial ManagementInterview Question: “In regards financial management, can you tell me a time when you dealt with a large budget in a complex project that was delivered on time and on budget?
Please be specific as to size of budget, team and any constraints.”
• Understand asset management principles for budgeting and programming projects and able to determine rules for use
• Apply available tools in basic situations (unit price books, other contract estimate tools) in order to conduct complex end-to-end cost estimates
• Understand asset management principles for budgeting and programming projects and able to determine rules for use
• Apply available tools in basic situations (unit price books, other contract estimate tools) in order to conduct complex end-to-end cost estimates
• Apply asset management principles to budgeting and programming projects
• Interpret and apply available tools (unit price books, other contract estimate tools) in order to conduct complex end-to-end cost estimates
• Apply asset management principles to budgeting and programming projects
• Interpret and apply available tools (unit price books, other contract estimate tools) in order to conduct complex end-to-end cost estimates
• Apply asset management principles to budgeting and programming projects
• Selects the best available tools (unit price books, other contract estimate tools) in order to conduct complex end-to-end cost estimates
• Apply asset management principles to budgeting and programming projects
• Selects the best available tools (unit price books, other contract estimate tools) in order to conduct complex end-to-end cost estimates
• Evaluate asset management principles and execution in budgeting and programming projects
• Evaluate the use of available tools (unit price books, other contract estimate tools) in order to conduct complex end-to-end cost estimates
• Evaluate asset management principles and execution in budgeting and programming projects
• Evaluate the use of available tools (unit price books, other contract estimate tools) in order to conduct complex end-to-end cost estimates
© 2013 IBM Corporation
All Top Consulting Firms Use Competencies in Hiring
Situation/Task
What was the situation or
context? What were you doing? What task were you working on?
Action Reasoning
What was the reasoning/
rationale that led to the action?
Results
What was the result/
outcome of the key
situation?
What did you do? What were the
steps you took to get to that effective
outcome?
A pro-tip: Use the STAR response method. It will keep you from going beyond what people want.
• Companies are looking for examples of your past work. Remember, history repeats itself!
• What you did in the past, chances are you will continue – good and bad• Think about what they are hiring for (look at the job posting) then think
about:1. What competencies are do they need in their hires?2. What BEHAVIORS can I talk about?