2013 APTMetrics, Inc.

Synthesizing the Measurement of Leadership Potential

John Scott & Kevin Tomczak

November 4, 2015

2013 APTMetrics, Inc.

39TH International Congress on Assessment Center Methods November 3-4, 2015

1

Leadership Assessment Benchmarks

15%

47%

75%

90%

0% 25% 50% 75% 100%

Campus hires

Supervisors

High potentials

Senior executives

Target of Assessments

Source: Church & Rotolo (2015)

2

0%

10%

20%

30%

40%

50%

60%

70%

80%Hi Potentials

Senior Executives

Source: Church, Rotolo, Ginther &

Levine (2015)

What is Being Measured?

3

Setting the Context:

Talent Segmentation

4

How Best to Differentiate Talent?

• Majority of organizations use some form of “High Potential” concept as their key talent differentiator

• Many models exist of high potential

• Some are grounded in research… some are not

• Research on high potentials is limited

• Potential for what?

5

Example Models of High Potential

Model Example Observations

Role Move into more senior roles in the organization

Sometimes generic but often defined as specific end-points; can devolve to simple replacement planning

Level Move into positions two levels above current role

Most commonly used; often generic in nature; works best at lower levels in the organization

Breadth Capability to take on broader responsibilities

Wide open definition that can vary across different parts of the organization

Performance Consistent track record of performance over time

Very specific indicator; less commonly used today than in the past (e.g., Nine-Box framework)

Source: Church, A.H., Waclawski, J. & Scott, J.C. (2012, April). Talent management in action: Game of thrones. Workshop conducted at the 27th annual

conference of the Society for Industrial and Organizational Psychology, San Diego.

6

Foundational Dimensions

Growth Dimensions

Career Dimensions

Leadership Potential Blueprint

Performance (In-role track record)

Personality Cognitive Ability

Learning Motivation

Leadership Capability Functional Capability

Fit

Adapted from Silzer, R., & Church, A. H. (2009). The Pearls and Perils of Identifying Potential.

Industrial and Organizational Psychology: Perspectives on Science and Practice, 2, 377-412.

7

Identify, Develop, and Deploy Potential

C-suite Leaders

Senior Leaders

Mid-level Managers

Supervisors/Pivotal Roles

Team Leads

Purpose Assessment

Identify and address development gaps; Inform movement decisions

1. Virtual BCS 2. Hogan Suite 3. Ravens APM 4. Qualitative 360 5. Interview

Verify global enterprise talent; succession planning

1. Assessment Center 2. Hogan Suite 3. Ravens APM 4. 360

Confirm potential of individuals early in career

1. Virtual Assessment 2. Hogan Suite 3. Ravens APM 4. 360

Identify & differentiate; Conduct initial segmentation of talent

1. SJT 2. Biodata/Personality 3. Ravens

Focus #4: Senior Leaders

Focus #3: Advanced Leaders

Focus #2: Emerging Leaders

Focus #1: Potential Leaders

Source: Adapted from Church, Waclawski, & Scott (2012, April). Talent management in action: Game of thrones. Workshop conducted at the 27th annual conference of the Society for Industrial and

Organizational Psychology, San Diego.

8

A Case for Multiple Assessment Methods

9

Benchmarks: Assessment Methods Used

0%

10%

20%

30%

40%

50%

60%

70%Hi Potentials

Senior Executives

Source: Church & Rotolo, (2015)

10

Assessment Matrix

Leadership Potential Dimensions

Assessment Tools

Ravens

Hogan

360-degree

feedback

Structured

Interview

AC Business

Simulation

Foundational

(Cognitive & Personality) X X X

Growth

(Learning & Motivation) X X X X X

Career

(Leadership & Functional)

• Transformational Leadership X X X X

• Builds Strategic Alliances X X X X

• Champions the Customer X X X

• Drives the Business X X X X X

• Manages Talent X X

• Fosters Open Communication X X X X

• Builds an Ethical Culture X X

• Cultivates Change

and Diversity X X X

• Domain Expertise X

X X

11

Assessment Framework

Multi-rater Feedback

Cognitive & Personality Measures

Accomplishment Record Behavioral Interview

Future What you are capable of

Today How you are currently seen

History What you’ve done

Foundation Who you are

Source: Adapted from Church, Rotolo, Wade, Tuller, & Ginther, (2014, May). Leveraging assessment technology for senior leaders. In C. Wade (Chair), Breaking the assessment glass

ceiling. Symposium conducted at the 29th annual conference of the Society for Industrial and Organizational Psychology, Honolulu

12

What to Assess

13

Searching for the Perfect Predictor

Source: Schmidt & Hunter (1998)

+1.00 Perfect Prediction

0.00 Random Prediction

GMA Tests

Structured Interviews

Peer Ratings

Assessment Centers Biodata Personality Tests

Educational Qualifications

Graphology

+.51

+.51

+.49

+.37 +.35

+.31

+.10

+.02

GMA Test & Other Predictors +.53 -.65

14

Limitations of Current Assessment Model

Levels of Analysis

Behavioral Consistency Assumption

Thin Slices of Behavior

Non-representative Behavior

Adverse Impact

1

2

3

4

5

Source: Cascio & Aguinis (2008)

Context

Time

Broaden Criterion Space

Extend Measurement Perspective

Ability to predict performance has not improved over past 80 years

15

Job Analysis is the Foundation

Job

Job

Job

Success Profiles

Talent Pool Criteria

Promotional Readiness Criteria

Assessment Specifications

Minimum & Preferred Qualifications

Job

An

alys

is

Key Responsibilities

Scope and Impact of Work

Functional Expertise

Competencies Required

Education/Experience

Requirements

Contextual/Strategic/

Environmental Factors

16

Potential Simulation Challenges

• Potential major acquisition

• Competitive threats (small, agile)

• Crisis management issues

• Potential regulatory challenges

• Innovations that change the market realities

Competitive Challenges

Market Challenges

Stakeholder Management

Business Challenges

• Activist shareholder threat

• Leadership team challenges, resistance

• Conflict within the Board of Directors

• Underperforming products, regions

• Manufacturing challenges

• Flat to declining market share

17

Leveraging the 3rd Dimension

18

Competency / Challenge Matrix

Leadership Potential Dimensions

Business Challenges

Competitive

(Acquisition)

Market

(Crisis Mgmt)

Market

(Shareholder)

Business

(Performance)

Stakeholder

(BoD)

Foundational

(Cognitive & Personality) X X X X X

Growth

(Learning & Motivation) X X

Career

(Leadership & Functional)

• Transformational Leadership X X X X X

• Builds Strategic Alliances X X X X

• Champions the Customer X X X

• Drives the Business X X X

• Manages Talent X X

• Fosters Open Communication X X X X

• Builds an Ethical Culture X X X X

• Cultivates Change

and Diversity X X X X

• Domain Expertise X X X X X

19

Business Case Simulation - NGO

Story Overview

• Participant assumes role of an emerging leader of Children Frist, a large Non-

Governmental Organization (NGO) with offices around the world

• Simulation takes place in several “chapters” occurring over the first few weeks on

the job for the new leader

Technical Specifications

• Multi-media simulation design for administration either in one 2-3 hour sitting, or in

multiple 1-2 hour sittings over a specified time

• Assessment simulation can be taken from any desktop or laptop computer

• Multi-media delivered through cloud and optimized for limited bandwidth

• Assessment items include video-taped constructed responses scored by trained

assessors

• Integrated assessment reports are delivered within 3-5 days

20

DEMO

21

Multi-Trait, Multi-Method

Integration

22

Assessment Outcomes

FUTURE CAPABILITIES

Capacity to perform when faced with the challenges and strategic issues of a leadership role two levels above current role

CURRENT CAPABILITIES

Extent to which individual is engaging in the leadership behaviors relevant for role

FOUNDATIONAL CAPABILITIES

Core thinking styles and personality characteristics that are the building blocks of leadership potential

Source: Church & Silzer (2014) . Going behind the Corporate Curtain with a Blue Print for Leadership Potential: An

Integrated Framework for Identifying High-Potential Talent. The Professional Journal of HRPS, Volume 36 Issue 4 2014 (2014)

23

Competency by Assessment Matrix

= Strength = Proficiency = Opportunity for Growth

Foundational History Current Future

Competencya Hogan/ Raven’s

Behavioral Interview

360 AC

Simulation OAR

Leading the Business

Transformational Leadership

Drives the Business

Domain Expertise

Leading People

Develops Talent

Champions the Customer

Builds Strategic Alliances & Relationships

Champion for Inclusion

Cultivates Change and Diversity of Thought

Builds and Ethical Culture

Fosters Open Communication

24

Impact of Integrated Assessment Programs

• 65% of benchmark companies view assessment as impacting organizational results

• 80% use assessments currently and another 15% plan to do so

• Multi-method, multi-trait approach key to success

Source: Church, Rotolo, Ginther & Levine (2015)

25

Back-up Slides – Examples of Assessment Tools

26

Raven’s Advanced Progressive Matrics

27

Hogan

360 Screens: Rater Experience

28

360 Screens: Rater Experience

29

Feedback Report: Executive Summary

30

Feedback Report: Highest & Lowest Scores

31

Feedback Report: Gap Analysis

32