professional coach training evaluation research beyond participants’ self-reports

Professional Coach Training

Evaluation Research Beyond Participants’ Self-Reports

Arthur DrexlerUniversity of InnsbruckDepartment of Communication in the Professional Field and PsychotherapySchöpfstrasse 3 A-6020 Innsbruck

Heidi MöllerUniversity of Kassel

Department 04Arnold-Bode-Straße 10

D-34109 Kassel

http://cms.uni-kassel.de/

University of KasselDepartment 04

University of InnsbruckDepartment of Communication in the Professional Field and Psychotherapy

ICTP/ St. Petersburg 2008

Overview

• Background• The Research Focus• Characteristics of a Coach• Evaluation Design and Instruments• Results• Outlook

„How can hard facts for soft skills be collected?“




Background: Lack of Evaluation Tools

• No tradition of evaluating coach trainings

• No valid and practicable specific tools for evaluation and quality assurance

• No benchmarks available




Background: The Coach Programme

• 9 weekend seminars (=190 training units)• Duration of 16 months• Each seminar focused on specific topic

• conflicts• communication skills• group dynamics• etc.




The Research Focus

The research addresses three main questions:

What theoretical knowledge, skills and personality traits does a coach need to successfully meet the clients’ needs?

Do the participants have the knowledge and skills at their disposal after coach training?

What instruments can make these factors visible?




Characteristics of a Coach

Motivation Personality factors Concept competence Key qualifications




Sample Description

• Sample size: 17• Gender: 6 men, 11 women• Mean age: 44 years (s = 6.6)• Profession: 23.5% work in a social

profession, 76.5% in other professions (e.g. business consulting, HR, medicine)




Evaluation Design and Instruments

Pre-measurement at the beginning of the training

QuestionnairesCase StudyAffect Recognition TestsBusiness Knowledge TestSculpting

Continuous measurements after each seminar

Self and trainers’ assessments of learning progress through questionnaires

Post-measurement

QuestionnairesCase StudyAffect Recognition TestsBusiness Knowledge Test




Initial Motivation Results

• Goal of their own personal development through coach training

• Desire for a change to improve their professional lives – not necessarily as coaches

• Motivation to help clients (and themselves) reach professional goals




Results Personality

FPI-Scales T df Sig.

Satisfaction/Contentedness ,891 13 ,389

Social Orientation -6,204 13 ,000*

Performance Orientation -5,264 13 ,000*

Inhibition 2,104 13 ,055

Excitability -,900 13 ,385

Aggression -,306 13 ,765

Pressure 1,236 13 ,238

Physical Complaints 10,262 13 ,000*

Concerns about Health -1,000 13 ,336

Openness -2,918 13 ,012*

Extraversion -4,954 13 ,000*

Emotionality 1,127 13 ,280




Results Affect Recognition

No significant changes for anxiety, joy, surprise, loathing, mourning and anger




Business Knowledge

0

10

20

30

40

50

60

70

80

1 2 3 4 5 6 7 8 9 10 11 12

Overall-changes are not significant (p = 0.119)

Individual changes (blue bars: pre / red bars: post-measurement)




Results Case Studies

• Case analyses improved– Considered different levels (relationship, competence,

emotional)– Considered psychodynamics– Considered group dynamics in the organisation

• Hypotheses improved– More hypotheses– More complex hypotheses

• Problem-solving skills improved– Beyond mere description – Concrete interventions




Self-Assessments

Self-assessments of improvement, after the conflict management seminar

23

18

16

37

35

29

46

12

27

18

38

22

27

0 5 10 15 20 25 30 35 40 45 50

Knowledge of leadership

Knowledge of management

Knowledge of organisational structures

Analytical skills

Coaching planning skills

Communication skills

Conflict management skills

Stress management skills

Change management skills

Media skills

Analysing psychodynamic processes

Personal self-reflection

Self-reflection of coaching performance




Self-Assessments

Self-assessments of improvement after completion of the programme

1,67

1,27

1,52

2,61

2,27

2,22

1,65

0,93

1,32

1,71

1,78

2,15

2,24

0,00 0,50 1,00 1,50 2,00 2,50 3,00

Knowledge of leadership

Knowledge of management

Knowledge of organisational structures

Analytical skills

Coaching planning skills

Communication skills

Conflict management skills

Stress management skills

Change management skills

Media skills

Analysing psychodynamic processes

Personal self-reflection

Self-reflection of coaching performance




Summary

Motivation to attend dominated by expectations of personal and professional development

Social orientation, performance orientation, extraversion and openness increase

Tendencies of improvement in recognising emotions Tendencies of improved business knowledge Better development of hypotheses based on interpretive

frameworks and improved problem-solving skills Seminar-related improvements correlate mostly with the

seminar topic Strong improvements of analytical skills and self-reflection




Summary

Personal and professional development

More Social Orientation

More Performance Orientation

Extraversion

Openness

+

Motivation

Coaching skills




Outlook

The multi-methodological approach to evaluate coach training can be prototypic for adult education and training evaluation in general.

Results allow the comparison of the development of competencies in different education and training contexts.

Participants and course designers can benefit from the feedback provided by the evaluation process.

“Hard facts” can be collected for quality assurance and quality development in training.

professional coach training evaluation research beyond participants’ self-reports

Documents