Life Cycle Logistics Validation and Workforce Assessment

LOG FIPT Presentation

March 28, 2008LOG FIPT

AT&L HCSP Competency Initiative … Enabling a High-Performing, Agile Workforce

2

Participation

• Exceeded our aim of 1092 employees

• Adequate Participation Achieved: Of the 5635 LCL professionals randomly identified, 1295 completed the assessment for a total response rate of 22.9%.– This number is more than sufficient for the current analysis

• Focus is on Validation of the model – Ensuring competencies are related to the “job” of a Life Cycle Logistics

professional– Not enough supervisor participation: Did not allow for workforce

proficiency gap assessment• Appropriate to focus on validation for this level of development of our

competency model• There are many options for building off the the current work to look into

workforce assessment and workforce planning aspects

3

Participation In Detail

Career Level Frequency Percent

Entry Level 18 1.4

Journey Level 1084 83.7

Senior Level 193 14.9

Total 1295 100.0

Military/Civilian Status Frequency Percent

Civilian 1241 95.8

Military 54 4.2

Total 1295 100.0

Primary Role Frequency Percent

System Sustainment 688 53.1

Acquisition Logistics 607 46.9

Total 1295 100.0

Years LCL Frequency Percent

1 to 5 Years LCL Experience 289 22.3

6 to 10 Years LCL Experience 304 23.5

11 to 15 Years LCL Experience 141 10.9

16 to 20 Years LCL Experience 90 6.9

21+ Years LCL Experience 438 33.8

Missing 33 2.5

Total 1295 100.0

Average 14.1 years

Component Frequency Percent

Army 466 36.0

Navy (including USMC) 485 37.5

Air Force 302 23.3

4th Estate (DCMA, DLA, Other) 42 3.2

Total 1295 100.0

4

Technical Competency Assessment

What Did We Ask The Participants?Frequency, Criticality, and Proficiency

5

Competency Results – ValidationFrequency and Criticality

Earned Value Management

Provide supportability data to ensure inclusion in EVM tracking where appropriate. (E6)

Utilizing Simulation and Modeling Techniques

Use modeling and simulation to estimate cost and/or performance parameters. (E2)

Integrated Product and Processes Development (IPPD)

Contribute to an IPT to ensure logistics considerations are represented in the program. (E12)

What competencies were rated highest across all ratings?

• Integrated Product and Process Development (IPPD)• Maintaining and Managing Customer Relationships

What competencies were rated lowest across all ratings?• Utilizing Simulation and Modeling• Earned Value Management

Maintaining and Managing Customer Relationships

Communicate with the customer to verify that logistics requirements are stated accurately in the system design. (E14)

Use customer feedback tools or methods to refine, maintain, and/or share logistics information. (E15)

Competency Element In Assessment

Competency Element In Assessment

6

Exploratory Factor Analysis

Test and Evaluation

Technical Data Management

Configuration Management

Supportability Analyses and Design Tradeoffs

Maintenance Planning

IT process Management

Human System Integration

Incorporating Environmental Safety and Occupational Health

Obsolescence and DMSMS Planning

PBL Planning

PBL Implementation and Execution

Supply Chain Management

Minimizing Life Cycle Costs

Managing and Developing Performance Based Agreements PBAs Risk Management

Contracting for Supplies and Services

Integrated Product and Process Development IPPD

Design for Support Supportability

Joint Operational Interface

Maintaining and Managing Customer Relationships

Alternative Sourcing

Business Case Analysis

Earned Value Management

Utilizing Simulation and Modeling

Unit 2: Performance-Based

Logistics Planning

and Management

Unit 3: Collaborate with

Customers to Enhance

Supportablity

Unit 1: Utilizing Test Data for

Best Design, Configuration,

and Processes

Unit 4:

Cost Evaluation

for Sustainment of Systems

• Unit 1: Utilizing Test Data for Best Design, Configuration, and Processes• Unit 2: Performance-Based Logistics Planning and Management• Unit 3: Collaborate with Customers and IPT members to Enhance Supportability

• Unit 4: Cost Evaluation for Sustainment of Systems

7

Competency Results – By Unit Level

What Unit Was Rated Highest Across All Ratings?

•Unit 3 Collaborate with Customers and IPT members to Enhance Supportability is the highest rated across frequency, criticality, and proficiency

•The next highest was Unit 2: Performance-Based Logistics Planning and Management across all ratings

Which Units Were Rated Lowest?

•The Lowest Rated Units then vary between Unit 1 and Unit 4 in Frequency, Criticality, and Proficiency

• Unit 4: Cost Evaluation for Sustainment of Systems• Unit 1: Utilizing Test Data for Best Design, Configuration, and Processes

8

Schedule and Deliverables

• Sponsor review version delivered to FIPT Mid-April

• Collect comments and feedback during the review from

• Complete April 30

• We Plan to Formally deliver to Mr. Hall and the FIPT in May

9

Logistics Competency Management Workforce Validation

Questions?

10

Competency Results – Differences by Component

• Army is different in Frequency & Criticality• Army (mean 2.4) does significantly less Unit 1,2, & 3 activities than

Air Force or Navy• Army participants see Unit 1,2, & 3 as significantly less critical than

Air Force or Navy

• The job of an Army Life Cycle Logistician may be significantly different than that of a practitioner in Air Force or Navy.

• We know there are more Army LCLoggies than others• They may have a more wide variety of duties, beyond our competency

model

• There are no differences in Proficiency between the major Service Components

11

Framework Development

Framework Development

Phase I - Convene an expert panel (EP)

Actions:• Develop a competency

framework & input model• EP identifies Subject Matter

Experts (SMEs)• EP communicates

competency effort to the SMEs

• Develop communications package

Goal: • Establish baseline of

existing competency model.• Communicate effortProducts: • FA provides list of targeted

high-performing SMEs• Obtains expert panel

concurrence on baseline competency framework

• Obtain approval from Dir, HCI and FA on competency model input

Phase I - Convene an expert panel (EP)

Actions:• Develop a competency

framework & input model• EP identifies Subject Matter

Experts (SMEs)• EP communicates

competency effort to the SMEs

• Develop communications package

Goal: • Establish baseline of

existing competency model.• Communicate effortProducts: • FA provides list of targeted

high-performing SMEs• Obtains expert panel

concurrence on baseline competency framework

• Obtain approval from Dir, HCI and FA on competency model input

Phase IV – Validate and Assess

Actions: • Launch competency

assessment tool• Analyze results to evaluate

model validity and generalizability to the workforce

Goal: • Identify competencies

required for superior performance

• Evaluate proficiency gaps for validated competencies

• Plan for continual updates and use of competency model

Products: • Deliver proven (validated)

competency model in HR XML format

• Provide competency validation and assessment and obtain Dir, HCI and FA approval

Phase IV – Validate and Assess

Actions: • Launch competency

assessment tool• Analyze results to evaluate

model validity and generalizability to the workforce

Goal: • Identify competencies

required for superior performance

• Evaluate proficiency gaps for validated competencies

• Plan for continual updates and use of competency model

Products: • Deliver proven (validated)

competency model in HR XML format

• Provide competency validation and assessment and obtain Dir, HCI and FA approval

AT&L Competency Management Process

Model Testing & Refinement

Model Testing & Refinement

Model DevelopmentModel Development

V 1.0 V 1.0 CompetencyCompetency

Model Model

Proposed Proposed Competency Model Report Competency Model Report

Approved InitialApproved InitialCompetency Model V 0.5Competency Model V 0.5

Phase III – Perform a beta test & refine model

Actions: • Collect and synthesize

feedback from proposed model report

• Pre-assessment communications to workforce

• Identify stratified workforce sample

Goal: • Further refine model to

include input from functional leads

• Obtain FA and Dir, HCI approval for validation assessment

Products: • Obtain concurrence from

FIPT on competency model

• Obtain approval from Dir, HCI and FA on competency model

Phase III – Perform a beta test & refine model

Actions: • Collect and synthesize

feedback from proposed model report

• Pre-assessment communications to workforce

• Identify stratified workforce sample

Goal: • Further refine model to

include input from functional leads

• Obtain FA and Dir, HCI approval for validation assessment

Products: • Obtain concurrence from

FIPT on competency model

• Obtain approval from Dir, HCI and FA on competency model

Phase II – Develop the model

Actions:• SMEs review the

competency framework and provide essential job data through structured interviews and online data collection tools.

• SMEs engaged to identify key “work” situations and competencies contributing to successful performance

• Analyze results and develop competency model content

Goal: • Model development and

identification of key behaviors

Products: • Deliver Proposed Model

Report to Dir, HCI and FA for review

Phase II – Develop the model

Actions:• SMEs review the

competency framework and provide essential job data through structured interviews and online data collection tools.

• SMEs engaged to identify key “work” situations and competencies contributing to successful performance

• Analyze results and develop competency model content

Goal: • Model development and

identification of key behaviors

Products: • Deliver Proposed Model

Report to Dir, HCI and FA for review

Collect Existing

Competency Data

Competency Competency Validation Validation

& Assessment& AssessmentReport Report

Approved InputApproved InputCompetency Model Competency Model

Competency Validation, Assessment, and Sustainment

Competency Validation, Assessment, and Sustainment

May 06 – July 06 Sep - 06 - Jun 07 Nov 07 - Present Jan 22 08 - Apr 08

12

Sample Size Criteria

Assumption 50% Population #Sample Size based on

Component Breakdown

PM 12775 2600

LCL 12331 2184

SPRDE SE 35142 2516

BCEFM 7608 2220

Property 530 530

Using components to stratify the sample we need:

Assumptions:1. Assume a 50 percent response rate2. Stratified sample based on

component population

Note: Samples are different with similar population numbers due to differing stratification across components. If you have a large concentration of staff in one or two major components you will need less than expected from those areas

Assmumption of Participation Army Navy AF 4th Est Total

LCL Population 2006 6,319 4,155 1,781 76 12,331Sample Needed 100% 361 351 317 63 1092

Sample Needed 50% 722 702 634 126 2184

13

Logistics Competency Management Workforce Validation

What has been done?• LCL experts met with CNA to refine competency elements

• Website has been tested and feedback has been collected

• A random sample from each component has been selected and will be e-mailed by CNA with login information

• Launch details have been finalized• Changes are being implemented and a final round of testing will

close Jan-14• E-mail to be sent co-signed from Mr. Hall and Mr. Anderson to

be sent Jan-14

14

Analysis

Business Rules for Inclusion/Exclusion in Sample– Extreme proficiency rating differences between supervisor and

employee will be discarded – If employee-supervisor pair incomplete, only importance and

frequency ratings will be factored into final results

Assumptions/Approach for Reporting Results– Gap scores are combined to make a composite score – Composite score of employee and supervisor data will be

compared to a standard– Gaps will be reported based on the magnitude of difference to

the proficiency standard to identify most to least critical

Component Army Navy Air Force4th

Estate TotalNumber Selected

Randomly

722 702 634 126 2184