evidence-based hr what is it and is it really happening … · u stage 3 – decline: ten reasons...

1

EVIDENCE-BASED HR

WHAT IS IT AND IS IT REALLY HAPPENING YET?

Rob B Briner

CEBMa center forEvidence-Based Management

2

Most stuff available www.cebma.org

2

3

Objectivesu What is evidence-based practice in HR?u Why all practitioners need evidence-based

practiceu How evidence-based is HR?u Mini case of how evidence-based approaches

can lead to more effective decisions

3

4

What is evidence-based practice for?u To help with this big problem: Practitioners

in all fields use evidence (or information) in their decision-making

u BUT the evidence used is often limited– Not enough of it– Not from/of multiple and diverse sources/types– Not critically appraised for its quality or relevance– Not used in a systematic way (e.g., not focusing

on the best available evidence, not weighting, not summarized)

4

5

The logic of evidence-based practiceu Using limited quantities, sources, and types

of evidence in an uncritical and unsystematic way will increase the chances of poor decision processes and unintended outcomes

u Using greater quantities, sources and types of evidence in a critical and systematic way will increase the chances of better decision processes and intended outcomes

5

6

What is it EBMgt?Evidence-based practice is about making decisions through the conscientious, explicit and judicious use of the best available

evidence from multiple sources by

1. Asking: translating a practical issue or problem into an answerable question2. Acquiring: systematically searching for and retrieving the evidence3. Appraising: critically judging the trustworthiness and relevance of the evidence4. Aggregating: weighing and pulling together the evidence5. Applying: incorporating the evidence in the decision-making process6. Assessing: evaluating the outcome of the decision taken

to increase the likelihood of a favourable outcome.6

7

What is EBMgt?u Making decisions through theu conscientious (you try hard, you are

systematic)u explicit (you write down, show and share

your evidence) andu judicious (you judge the quality of the

evidence) use of theu best available evidence fromu multiple sources (not just one or two) 7

88

9

Example: Evidence-Based Performance Managementu Suppose a senior manager believes that

there is a problem with low employee performance in the organization

u They believe the employees could perform better

9

1010

Example: Evidence-Based Performance Management

Element 1: Professional expertiseu Why do I and my experienced colleagues think there is

low employee performance?u Have I seen this before? What happened?u What are my personal beliefs about the causes of low

performance?u What’s worked to improve performance in the past and

why?u What are my hunches?u What do I think are the causes and possible solutions?u Is this situation occurring elsewhere?u How relevant and applicable is my experience?

1111

Example: Evidence-Based Performance Management

Element 2: Organizational datau What actually is the performance level?u What type of poor performance and where?u What are local explanations for poor performance?u What performance management is currently in place and

is it working?u What do managers think is happening?u What are the possible costs and benefits of

interventions? Is it worth intervening here?u What is happening or what is going to happen that

might be affecting performance?

1212

Example: Evidence-Based Performance Management

Element 3: Scientific research findingsu What are the average levels of employee performance in

my sector and location – is the performance level really ‘low’?

u What does systematically reviewed research evidence suggest to be the major causes of poor performance?

u How relevant and applicable is that evidence here?u What does research evidence from systematic reviews

suggest as effective interventions to increase employee performance?

u How well might the interventions the research describes work here?

1313

Example: Evidence-Based Performance Management

Element 4: Stakeholders’ values and concerns

u How do employees feel about their work performance?u What do employees feel about the proposed

interventions?u Do they see downsides or unintended negative

consequences?u How do managers feel about the performance

interventions?u How practical do those responsible for implementing the

interventions feel they are?u What alternative explanations and proposed solutions for

poor performance do others have?

14

Why is it a particular problem in HR?u It may not be but management and HR has

come very late to evidence-based practiceu The concept is not well-known or understood by

organizations, managers or management schools

u It seems that much HR practice…– is dominated by fads and fashions– not into basing decisions on good quality evidence

from multiple sources– applies ‘solutions’ without identifying problems

u HR is not alone in this…

14

1515

1616

1717

1818

1919

2020

2121

2222

23

24

Why do we need it? Because our decisions strongly shaped by…u Strong beliefs which don’t seem to be trueu General cognitive biases that limit decision-

making and judgementu Management fads, fashions and quick fixesu Power and politics that determine seniorityu Consultants/advisors/fad vendors

24

25

The dangers of holding strong beliefs if they aren’t quite right

u It ain't what you don't know that gets you into trouble. It's what you know for sure that just ain't so. (Mark Twain)

u The greatest enemy of knowledge is not ignorance, it is the illusion of knowledge(Stephen Hawking)

u Ignorance is fine – false beliefs not so

25

26

Some examples…u What’s happened to average job tenure in

the past 10 years? Up? Down? Stayed same?

u Job satisfaction in past 15 years? Up? Down? Stayed same?

u World moving faster – more VUCA (volatility, uncertainty, complexity and ambiguity)? Faster? Slower? Same?

u Generational differences in work attitudes? Large? Small? None? 26

27

Job tenure UK

27

2828

2929

30

More VUCA?

30

31

Economist: Creed of Speed December 2015u The idea that time is speeding up is clearly

popular. It is also plausible. There is just one problem. It is very hard to prove that it is actually happening.

u Hard evidence of a great acceleration is hard to come by. The Economist has considered a variety of measures by which the speed of business in America can be quantified. A few do show some acceleration. But a lot do not.

31

3232

33

Generational differences in work attitudes?

33

34

Employee engagement: What do we know for sure that just ain't so?1. Engagement is clearly defined2. There are valid and reliable measures of

engagement3. Engagement is different from previous concepts

(e.g., satisfaction, commitment)4. There plenty of good quality evidence that

demonstrates value of engagement• Increases in engagement > increases in performance• Engagement interventions > increases in engagement >

increases in performance

5. Claims made about the importance and role of engagement are reasonable and accurate

34

3535

3636

37

Cognitive biases

A bat and ball cost one pound and ten pence. The bat costs a pound more than the ball. How much does the ball cost?

37

38

Error and biases in problem-solving and decision-making – some examplesu Confirmation bias: Tendency to interpret and search for

information consistent with one’s prior beliefsu Hindsight bias: Tendency to see past events as being more

predictable than they were before the event occurredu Loss aversion: Tendency to prefer avoiding losses than to

acquiring gainsu Anchoring effect: Tendency to rely too heavily or over-

emphasize one piece of information (e.g., restaurant wine lists, large reductions in price in sales)

u Framing effect: Drawing different conclusions from exactly the same information presented in different ways (e.g., would you prefer a ready meal that’s “85% fat free” or “15% fat”?)

u Meta-cognitive bias: The belief we are immune from such biases38

39

Evidence-based practice can help because it’s about the conscientious, explicit, and judicious use of different sources of information

39

40

Example fads and fashionsu Business process re-engineeringu Total quality managementu Quality circlesu Talent managementu Leanu Outsourcing

40

4141

4242

4343

4444

4545

46

Fad and fashion article titles: Miller et al (2004)u Stage 1 - Ascendancy: Total Quality: Wave of the

Future, Reengineering: It’s Totally Radical, Welcome to the Revolution, The Promise of Reengineering, How to Work Wonders, Completely.

u Stage 2 – Maturity: Reengineering: The Hot New Managing Tool, The Reengineering Rage, Warning: This Good Idea May Become a Fad, Reengineering: Beyond the Buzzword.

u Stage 3 – Decline: Ten Reasons Why TQM Doesn't Work, TQM: The Mystique, the Mistakes, The Hocus-Pocus of Reengineering, Why TQM Fails and What to Do About It.

46

4747

48

Related concept of the quick fixu Focus on style and presentation not content or

processu Not be evaluatedu Not be as quick as had been hopedu Not be effective so followed by another quick fixu Become subject to organizational amnesia*u Can be career-enhancing for managers (e.g.,

issue selling, kick-ass CEOs)

*Kitchen equipment analogy

48

4949

50

So why are fads and fashions and quick fixes followed?u Promise to deliver a lot and fastu Appear simpleu New and shinyu Will make everything alrightu Help contain anxieties around intractable problemsu Help user feel effective and cutting edgeu Seems very ‘human’ to want to find quick, easy answers

that other people are adopting too

Evidence-based management not really much of a fad, fashion or quick fix using these criteria

50

51

Evidence-based practice can help because it’s about the conscientious, explicit, and judicious use of different sources of information

51

52

Power and politics as barriers to evidence-based managementu How do people really get ahead and into

positions of power?u Is it by being experts in and advocates of

evidence-based management?

52

53

Espoused and implicit goals of managers that deter them from EBPESPOUSED GOALSu To do what works (but few

evaluations)u To help organization fulfil its

missionu To identify and solve important

problemsu To do what mattersu Treating everyone equallyu Speak truth to power(?)

IMPLICIT GOALSu To get things doneu To further careeru To avoid troubleu To fix political or presenting

problemsu To meet targetsu To do what gets measuredu Favour those who help

advance personal goalsu Say what higher-ups want to

hear

53

54

Evidence-based practice can (maybe) help because it’s about the conscientious, explicit, and judicious use of different sources of information

54

55

What are the incentives for fad-selling consultancies to be evidence-based?u Get the work, get more work, and keep getting

work (so depends almost entirely on what clients want)– Persuade (may not take much) clients they need some

new thing, idea, technique, approach– Sell them the relevant product or service or intervention

based on that idea, technique or approach– Saturate the market until everyone’s bought it– Invent or borrow new ideas, techniques and

approaches clients do not yet use– Sell them the relevant product or service or intervention– Repeat

55

56

So why do managers and HR need evidence-based management?u Because their decisions are shaped by

– Strong beliefs which may not be true– General cognitive biases that limit decision-

making and judgement– Management fads, fashions and quick fixes– Power and politics that determine seniority– Consultants/advisors/fad vendors

56

57

Is there such a thing as evidence-based is HR?

57

5858

5959

6060

6161

62

How evidence-based is HR?u Evidence is always used but…u Too much reliance on

– Professional expertise– Benchmarking/copying– Fads and fashions– ‘Analytics’ alone

u HR professional training does not support evidence-based practice

u Do HR teams have the capability?62

63

Mini-case: Risk culture in global banku Managers believed serious errors caused by

poor ‘risk culture’u Regulator required banks to do something

around risk cultureu What evidence was collected?u What other evidence could have been used?

63

6464

problem solution

Practitionersprofessional expertise

Organizationinternaldata

Stakeholdersvaluesandconcerns

Scientificliteratureempiricalstudies

AskAcquireAppraiseAggregateApplyAssess

65

What evidence was collected?u Some stakeholder evidence (managers and

regulators)u No scientific evidenceu Limited professional expertiseu Little analysis of organizational data (though

much available)

65

66

What was done?u Interviews/working groups around the world

(about 80)u Did not ask whether there was a problem or

what that problem might beu Asked participants what they thought a ‘sound’

risk culture looked like (assumption)u From this developed a survey and framework –

not psychometrically sound not properly validated

u So what would a more evidence-based approach be?

66

67

What organizational data could have been collected?u A thorough analysis of the types and

frequency of errors being madeu A judgement of the quality of the available

organizational datau An analysis of other data that might help

explain the causes of errors (e.g., location, managers, shift, employee training, age, etc)

67

68

What scientific evidence could have been collected?u What is known, generally, about why errors

are made? What is number one cause?u Concept of risk culture not really known –

general view that ‘culture’ is not something that can be measured (no validated surveys) and little agreement on meaning

u However, concept of safety climate well known: Free to use well-validated questionnaires – links to errors across industries (e.g., aviation, medicine, oil, etc)

68

69

Questions already raised in industry about risk culture conceptu McKinsey (2008) analyzed the 'risk culture' at

Goldman Sachs: “A model for all firms … were the firm to stumble it would not be for the lack of a powerful risk-informed culture that combines intelligence, discipline and common sense".

u 2010, Goldman Sachs > the largest penalty ever for a Wall Street firm at the time (500 mil), in a settlement with the SEC concerning misleading "investors in a subprime mortgage product just as the U.S. housing market was starting to collapse" (Securities Exchange Commission 2010).

69

70

What stakeholder evidence could have been collected?u A closer examination of what the regulators

actually wantedu A clearer analysis of what senior managers

really felt the problem to beu A critical appraisal of the quality of this

evidence and what could be taken from it

70

71

How could an evidence-based approach helpu More evidence used to provide a stronger

analysis of the problemu More evidence from a range of sources about

a range of possible solutionsu A critical analysis of that informationu Greater learning for the organizationu A solution that’s more likely to work

71

72

Disadvantages of EBMgtu Challenges beliefs, prejudices and pet projectsu Requires more effortful critical thinkingu Requires collecting and using more data (of

different types from different sources)u Makes it hard to follow fadsu Challenge to authority and power based on

hierarchy or “because I say so” or JFDI or HiPPOu Reduces action-orientation of management

where doing stuff more important than doing stuff that works

72

73

Advantages of EBMgtu All the disadvantages also advantages

– Challenges beliefs, prejudices and pet projects– Requires more effortful critical thinking– Requires collecting and using more data (of different types from

different sources)– Makes it hard to follow fads– Challenge to authority and power based on hierarchy or

“because I say so” or JFDI or HiPPO– Reduces action-orientation of management where doing stuff

more important than doing stuff that worksu Develops deeper knowledge and understanding (of

organizational and other data)u Develops essential skillsu More likely to do stuff that will lead the outcomes you

want73

74

It’s a sort of choiceu How do you personally want to practice as a

professional?u How would you like your skills, knowledge

and effectiveness to develop over the course of career?

u How would you like your team to develop its capabilities and function to help the organization meet its objectives?

74

75

How to become a little more evidence-basedu Check our CEBMa websiteu Accept our ignorance and limits of all forms

of knowledge and evidence and experienceu Allow yourself to think and say the three

hardest words in management…

75

76

“I don’t know”

But I know how to find out…

Evidence-Based Management

76

77

Thank youThoughts, comments, criticisms, etc?

r.b.briner@bath.ac.uk

77