Methods for Efficient Job Analysis – Part II

If your computer does not have speakers, please dial in at:1-415-655-0069

Access Code: 595-368-411

Objectives

Recap of Job Analysis Steps 1 - 3

CPS HR Job Analysis Methodology Steps 4 - 6

Methods for optimizing efficiency in the job analysis process

Question & Answers

6-Step Job Analysis Methodology

1. Conducting a literature review2. Preparing tasks and KSA statements

(questionnaires)3. Obtaining SME ratings of task and KSA statements4. Analyzing the Subject Matter Expert (SME) ratings5. Linking KSAs to tasks6. Documenting the procedures used for the study

Planning for Data Collection & AnalysisThere are several questions that should be answered in order to plan appropriate data collection and analysis approaches. Some examples include:1. How many incumbents are there?2. What is the desired/required response rate?

1. Watch for transfers, updates to incumbent lists, etc.3. What type of job is being studied (e.g., professional, physical)?4. Do incumbents work on more than one shift or at more than one work location?5. How long should the questionnaire remain active?

1. Initial introduction email from management, supervisors, leadership?2. Will reminders be sent? If so, how often?

Step 4: Analyzing the Subject Matter Expert (SME) Ratings

Rating Approaches in Job Analysis

Consensus Ratings: SMEs work collectively in rating each task and KSA statement and reach agreement as to the ratings assigned. This approach is typically used with a small job class.

Independent Ratings: SMEs independently rate each task and KSA statement. Mean ratings are calculated for each task and KSA based upon the SME independent ratings. This approach is typically used with a large job class.

Rating Scales used in Job Analysis

Task Scales

• Task Frequency• Task Importance

KSA Scales

• KSA Importance• KSA When Needed• Relationship to Job Performance

Typical Task Frequency ScaleHow frequently is this task performed on the job?

0 – This task is not performed on the job1 – Performed less than yearly2 – Performed every few months to yearly3 – Performed every few weeks to monthly4 – Performed every few days to weekly5 – Performed approximately daily

Typical Task Importance ScaleHow important is the performance of this task for successful job performance?0 – This task is not performed or is trivial to successful job performance1 – Performance of this task is somewhat important for successful job 

performance2 – Performance of this task is important for successful job performance3 – Performance of this task is critical for successful job performance

Rating Criteria for TasksFor consensus ratings, an important or essential task of the job would meet both of the following thresholds:

Task Frequency Scale: Consensus rating of 2 or greater performed at least every few months to yearly

Task Importance Scale: Consensus rating of 2 or greater at least important for job performance

For independent ratings, an important or essential task of the job would meet both of the following thresholds:

Task Frequency Scale: Mean rating of 1.5 or greater at least halfway between performed less than yearly and performed every few months to yearly

Task Importance Scale: Mean rating of 1.5 or greater at least halfway between somewhat important and important

Typical KSA Importance ScaleHow important is possession of this KSA for successful job performance?

0 – Possession of this KSA is not needed or is trivial to successful job performance

1 – Possession of this KSA is somewhat important for successful job performance

2 – Possession of this KSA is important for successful job performance

3 – Possession of this KSA is critical for successful job performance

Typical KSA When Needed Scale

When is this KSA needed for successful job performance?0 – This KSA is not needed for successful job performance1 – This KSA is needed after hire for successful job 

performance2 – This KSA is required upon entry to the job for successful 

job performance

Typical Relationship to Job Performance Scale

Does possession of more of this KSA (beyond some minimum threshold) result in better job performance?

0 – NO – possessing more of this KSA does not result in better job performance

1 – YES – possessing more of this KSA does result in better job performance

Rating Criteria for KSAs for recruitment/selection activitiesFor consensus ratings, an important KSA would meet all of the following thresholds: KSA Importance Scale: Consensus rating of 2 or greater

at least important for successful job performance

KSA When Needed Scale: Consensus rating of 2

required upon entry

Relationship to Job Performance Scale: a consensus rating of 1 is needed to rank-order candidates on an eligible list

yes, this KSA leads to better job performance

For independent ratings, an important KSA would meet all of the following thresholds: KSA Importance Scale: Mean rating of 1.5 or greater

at least halfway between somewhat important and important

KSA When Needed Scale: Mean rating of 1.5 or greater

at least half way between needed after hire and needed upon entry

Relationship to Job Performance Scale: a mean rating of greater than 0.5 is needed to rank-order candidates on an eligible list

at least half of the respondents say more of this KSA does result in better job performance

Rating Criteria for KSA Statements for Other HR ActivitiesFor consensus ratings, an important KSA would meet each of the following thresholds:

KSA Importance Scale: Consensus rating of 2 or greater at least important

KSA When Needed Scale: Consensus rating of 1 needed after hire for successful job performance

For independent ratings, an important KSA would meet each of the following thresholds:

KSA Importance Scale: Mean rating of 1.5 or greater at least halfway between somewhat important and important for successful job performance

KSA When Needed Scale: Mean rating greater than 0.5 but less than 1.5 at least half of respondents rate it as needed after hire, but not all the way to a 2, which is needed upon entry

The Relationship to Job Performance Scale does not apply to other HR activities since this scale is used for rank ordering in selection/recruitment.

Step 5: Linkage of KSAs to Tasks

Linkage of KSAs to Tasks

The purpose of the linkage process is to document and justify how KSAs are used on the job, and their relationship to tasks performed on the job.

A linkage of KSAs to tasks is required by the Uniform Guidelines on Employee Selection Procedures.

Typically only tasks and KSAs that have been identified as important are included in the linkage process.

Can use consensus method or independent method with SMEs, or HR Analyst can link and SMEs confirm.

The results of the linkage are used to identify KSAs for selection, to develop a test plan, and to determine weighting of selection components.

KSA/Task Linkage KSAs

Task# Tasks

Skill to track expenditures

Skilltou

se phone

Timekeeping

Outlook

functions

Advanced know

ledge of payroll

Linkage Scale

0=Not Required (NO)1=Required (YES)

1 Track misc. office purchases

Complete payroll report

2 4 5 8 11

Reserve conference room for monthly staff meetingsScreen incoming phone calls and route to appropriate staff

4

7

9

KSA/Task Linkage KSAs

Task# Tasks

Skill to track expenditures

Skilltou

se phone

Timekeeping

Outlook

functions

Advanced know

ledge of payroll

Linkage Scale

0=Not Required (NO)1=Required (YES)

1 Track misc. office purchases

Complete payroll report

2 4 5 8 11

Reserve conference room for monthly staff meetingsScreen incoming phone calls and route to appropriate staff

4

7

9

1 0 0 0 0

0 1 1 0 1

Consensus Linkage Ratings

When using the consensus rating approach:

A KSA must link to at least one task with a consensus rating of 1 in order for the KSA to remain in the KSA inventory.

A KSA that fails to link to a single task in the task inventory with a consensus rating of 1 is eliminated from the KSA inventory.

Independent Linkage Ratings

When using the independent rating approach:

A KSA must link to at least one task with a mean linkage rating of greater than .5 in order for the KSA to remain in the KSA inventory. (This is at least halfway between not required and required.)

A KSA that fails to link to a single task in the task inventory with a mean linkage rating greater than .5 is eliminated from the KSA inventory.

Step 6: Documentation of the Job Analysis Study

DocumentationProper documentation of the job analysis study is important for several reasons: Captures the important information needed to support numerous HR programs

Provides the necessary legal defensibility

Establishes the content validity for selection procedures as required by the Uniform Guidelines on Employee selection procedures

Allows for the easy and efficient update or replication of the study in the future

Documentation

What You’d Want to Document…• Documentation of the SMEs involved in all steps of the process Data collection methods that were used Scales that were used (tasks, KSAs, linkage) Rating approach used (i.e., consensus or independent ratings) Scale cutoffs used to identify important tasks & KSAs, and linkage

(justification & consistency) Final list of important tasks & KSAs of the job (for selection)

Other Methods for Efficient Job Analysis

• Remote meetings• Online surveys• Rating process• Templates

Remote Meetings

Utilize programs and software, such as • Lync – part of Microsoft Office suite • Go to Meeting ‐ http://www.gotomeeting.com• Doodle ‐ http://doodle.com/• Meeting Wizard ‐ http://meetingwizard.com/

Question #1

Do you schedule or conduct job analysis online meetings?

If so, what tools or programs do you use for those meetings?

Remote Meetings

• Our recommendations:• For specific job classes, no more than 10 people on call • Ask attendees to try out the technology beforehand• Block out enough time for meeting • Provide an agenda ahead of time / Goal of the meeting• Provide training & materials before meeting• Keep people engaged throughout meeting (Call out specific people to share) • Delegate certain tasks to SMEs prior to meeting (utilize track changes) • Management buy‐in (initial communication & follow‐up)• Sign‐in sheet (documentation)

Online Surveys

When using online surveys, here are a couple to try: • Qualtrics• Survey Monkey

Question #2

What programs do you utilize for distributing online surveys to respondents?

Online SurveysOur Recommendations:• Review survey to make sure there are no mistakes• Pilot survey on a few people before distributing to larger group• Work w/ stakeholders to determine best way to distribute (i.e., email, intranet, all vs some)

• Determine appropriate timeline for data collection• Identify who will do data analysis• Determine best way to report the results • For test development (test validation), identify the critical KSAs

Rating Process

• For small classifications, utilize consensus panels (if not enough SMEs) • Gather ratings while refining the task/KSA statements 

TemplatesOur recommendations: Rather than recreating the wheel, put in place templates • Excel/SPSS templates (built in formulas/syntax)• Technical report• Messages for SMEs (meeting invites, reminders, message from management, etc.)

• Demographics questions – defensibility of process• O*NET/ previous job analysis materials• Google the classification under study (refer to other agency’s job classifications; rewrite duty statements into task statements) 

Question #3

What other templates do you all use or would consider using for a job analysisstudy?

Potential Challenges to a Job Analysis Study Insufficient number of SMEs included in the study (no incumbents involved; only supervisors)

Insufficient data collection (e.g., shifts, work locations)

Insufficient number of tasks and KSAs

Lack of clarity or specificity in task and KSA statements 

Incorrect application and/or interpretation of the scale cutoffs

Failure to understand all of the requirements of the process # of SMEs involved, linking tasks to KSA instead of dimensions, response rate

Questions?

Thank You!