Michigania 02005

A Logic of Diversity II

Scott E Page

Complex Systems, Political Science, Economics and

Institute for Social Research

University of MichiganSanta Fe Institute

Michigania 02005

Enlarging The Mantra

Identity,

Training,

Experiential

Diversity

Diverse

Perspectives

Better

Outcomes

Michigania 02005

Monday’s Talk: Unpacking The First Box

Diverse

Perspectives

Michigania 02005

Monday’s Talk: Unpacking The First Box

Perspectives

Heuristics

Interpretations

Michigania 02005

Today’s Talk: Demonstrating Causality

Diverse

Perspectives

Better

Outcomes

Michigania 02005

Specific Tasks

Problem SolvingPrediction

Preference Aggregation

Michigania 02005

Why Construct Models?Models allow us to provide conditions for

when a statement is true.

The Pythagorean Theorem: ``A-squared equals B-squared plus C squared’’ only holds for right triangles.

Michigania 02005

Finding the Conditions``Two heads are better than one!’’

``Too many cooks spoil the broth’’

Which one wins? Which do we apply in a given setting.

Michigania 02005

Finding the Conditions``Two heads are better than one!’’

``Too many cooks spoil the broth’’

Condition: For an irreversible process, too many cooks spoil the broth.

Michigania 02005

Swarm of BeesAlmost all of social science looks at

averages and changes in those averages.

Analogy: if you look at a swarm of bees, the path of any one bee is hard to predict and understand, but in the swarm all of those idiosyncratic behaviors cancel out and we can identify general trends.

Michigania 02005

The BuzzBee hives must stay around 96 degrees in

order for bees to reach maturation. Bees achieve this by genetic mechanisms that drive two behaviors:

When hot: fan out or leave the hiveWhen cool: huddle together

Michigania 02005

Diversity and HomeostasisGenetically homogeneous bees: All

get cool (or hot) at the same time. Temperature in hive fluctuates wildly. (1930’s heating system)

Genetically diverse bees: Get cool (or hot) at different temperatures. Temperature stabilizes.

Michigania 02005

Rethinking the SwarmThe logic of cancellation does not hold

because there are feedbacks between the bees. Those feedbacks imply we cannot look at averages.

Groups of people solving problems, making predictions, and making choices create feedbacks in abundance.

Michigania 02005

Problem Solving

Michigania 02005

Problem Solving

Perspectives

Heuristics

Michigania 02005

The Idea

135 91 A,b,x x,M

Thermometer: Toolbox: SAT,IQ skills, heuristics

Michigania 02005

Perspectivenumber of chunks

size

Ben & Jerry’s Ice Cream Array

Michigania 02005

Heuristicnumber of chunks

size

Ben & Jerry’s Ice Cream Array

Michigania 02005

Consultant

perspective: caloric rank

Michigania 02005

Consultant

perspective: caloric rankheuristic: look left and

right

Michigania 02005

Performance

• Average Performance Given– solution in perspective– application of heuristics

• Ben and Jerry– average quality of solution = 82

• Consultant- average quality of solution = 74

Michigania 02005

Perspective Diversity

8375

7381

80

Ben and Jerrystuck at 83

Michigania 02005

Perspective Diversity

8375

7381

80

86 8380 74

Ben and Jerrystuck at 83

consultantGets to 86

Michigania 02005

Diversity or Ability: A Test

Create a bunch of artificial problem solving agents and rank these agents by their average performances on a difficult problem.

All of the agents must be “smart”

Michigania 02005

Two Groups• Group 1: Best 20 agents• Group 2: Random 20 agents

Have each group work collectively - when one agent gets stuck at a point, another agent tries to find a further improvement. Group stops when no one can find a better solution.

Michigania 02005

The IQ View

138

75

121 84

Alpha Group Diverse Group

132 135

139

135

137

135

111

31

Michigania 02005

And the winner is..

“Most of the time” the diverse group outperforms the group of the best by a substantial margin.

See Lu Hong and Scott Page Proceedings of the National Academy of Sciences (2002)

Michigania 02005

The Toolbox View

ABD

EZ

AHK FD

Alpha Group Diverse Group

ADE BCD

ABC

BCD

ACD

BCD

AEG

IL

Michigania 02005

Formal Version

Theorem: Given a set of diverse problem solvers, a random collection outperforms a collection of the “best” individual problem solvers provided

-the set is large-the problem is hard-the problem solvers are smart

Michigania 02005

Prediction.

Michigania 02005

Prediction.

Interpretations Mental Models

Michigania 02005

The Madness of Crowds

We tend to think of crowds of people as irrational mobs. And that can be true. When people hear the ideas and opinions of others, they often succumb to peer pressure rather than speaking their own minds.

Michigania 02005

Which Line is Longer?

A: _____________

B: ___________

Michigania 02005

The dim boy claps because the others clap.

- Richard Hugo

Michigania 02005

The Wisdom of Crowds

If people do not hear the opinions of others, or if they render their true predictions anyway crowds can be incredibly wise.

Michigania 02005

Suroweicki’s Examples

Morton Thiokol’s stock plungePrediction Markets

Hollywood Stock ExchangeIowa Electronic MarketSports Betting Markets

Who Wants to be a Millionaire1906 West of England Fat Stock and

Poultry Exhibition

Michigania 02005

Two Separate Phenomena1. Information known by part of the crowd

2. Aggregative diverse predictive models

Michigania 02005

Revealing Known Information

Which of the following books would you NOT find in the Point o’ Pines Library

A. The Periwinkle Steamboat - LancasterB. Curtains - Agatha ChristieC. Unabridged Crossword Puzzle DictionaryD. I am Charlotte Simmons - Tom Wolfe

Michigania 02005

Information RisingSuppose that no one know the answer but

that 18 people know one of the books on the list is in the library and that 18 people know two of the books on the list are in the library. This means that 64 people guess randomly.

Michigania 02005

Information Rising

Of 64 Clueless: Correct answer gets 16Of 18 know one: Correct answer gets 6Of 18 know two: Correct answer gets 9

Total 31

Other answers get 23 (on average)

Michigania 02005

The Answer Is…Which of the following books would you NOT find in

the Point o’ Pines Library

B. Curtains - Agatha Christie

Michigania 02005

Aggregating Diverse Predictions

In most of the situations described, people do not know the answer yet. We can assume that people have diverse predictive models. We’d like to understand how that aggregation occurs and what roles diversity and ability play.

Michigania 02005

Reality

CharismaH MH ML L

H Experience

MH ML

L

G G G

G G G G

G

G

B

B

B

B

B

B

B

B

Michigania 02005

Experience Interpretation

HExperience MH

ML

L

G G G

G G G G

G

G

B

B

B

B

B

B

B

B

B

B

B

B

B G

G

75 % Correct

Michigania 02005

Charisma Interpretation

H MH ML L

75% Correct

G B

G BBBG

G G

BG G

BG G

BG G

G

G

B

B

B

B B

Michigania 02005

Balanced Interpretation

H MH ML L

75% Correct H

Good to beextreme on one MHmeasure, bad on other ML

L

G G G

G B G G

G

G

B

B

B

B

B

B

B

B

B

G

G

Michigania 02005

Voting OutcomeCharisma

H MH ML L

H

MH ML

L

GGB GGG GBG

GGG GGB G GBG

BGG

BGG

BGB

GBB

BBG

BBB

BBB

BBG

BBG

BGB

Michigania 02005

The Mathematics of Prediction

Prediction: # runs scored by winning softball team

Mon Tue WedBrad 8 10 9Matt 10 12 8

Michigania 02005

“Crowd’’ Prediction

Mon Tue WedBrad 8 10 10Matt 10 12 8Crowd 9 11 9

Michigania 02005

Actual Numbers

Mon Tue WedBrad 8 10 10Matt 10 12 8Crowd 9 11 9Actual 8 12 9

Michigania 02005

Squared Errors

Brad: (8-8)2 +(10-12)2 +(10-9)2 = 5

Matt : (10-8)2 +(12-12)2 +(8-9)2 = 5

Crowd: (9-8)2 +(11-12)2 +(9-9)2 = 2

Michigania 02005

Diversity of Predictions

(Brad-Crowd)2 = 1 + 1 + 1 = 3 (Matt-Crowd)2 = 1 + 1 + 1 = 3

Michigania 02005

Notice: 2 = 5 - 3

Crowd Error = Average Error - Diversity

Michigania 02005

Diversity Prediction Theorem

Crowd Error = Average Error - Diversity

(note: proven by statisticians, computer scientists, and economists)

Michigania 02005

Crowd = Average - Diversity

• Diversity as important as ability

• Limit to how much diversity(otherwise crowd error would be negative)

Michigania 02005

Experts on NFL DraftPlayer #1 #2 #3 #4 #5 #6 #7 #8 Alex Smith 1 1 1 1 1 1 1 2 Ronnie Brown 2 2 4 2 2 5 2 6 Braylon Edwards 3 3 2 7 3 2 3 3 Cedric Benson 4 4 13 4 8 4 8 4 Carnell Williams 8 5 5 5 4 13 4 8 Adam Jones 16 9 6 8 6 6 9 17

Error^2 158 89 210 235 112 82 39 300

Average Error: 153.13Diversity: 101.52

Michigania 02005

Crowd of Experts on NFL Draft

Player CrowdAlex Smith 1.13 Ronnie Brown 3.13Braylon Edwards 3.25Cedric Benson 6.13Carnell Williams 6.50Adam Jones 9.63

Error^2 51.61

Michigania 02005

Crowds Beat Averages Law

Crowd Error < Average Error

Michigania 02005

Does Crowd Beat Best?

In the NFL draft example, the best predictor Pete Brisco had an error of only 39. He outperformed the crowd, which had an error of 51.6.

Michigania 02005

Novices and Experts

Novices: Base their models on only a few variables or a few boxes.

Experts: Base their models on many variables or many boxes.

Michigania 02005

In Praise of Experts

Theorem: If an expert contains every variable considered by any one of the novices, the expert predicts better than the crowd of novices.

Michigania 02005

Crowds vs Experts

Test Set: Linear functions defined over 20 variables.

Crowd: Each of 100 novices looks at N randomly chosen variables

Expert: Looks at E>N variablesTraining: 300 independent

variablesContest: 300 independent variables

Michigania 02005

Crowds vs Experts% of Time

N E Expert Wins3 20 94.66%3 15 34.66%5 10 29.33%5 7 9%

Michigania 02005

What’s Happening

Expert: Getting best fit over all his variables.

Crowd: Getting an average of many fits over many distinct subsets of variables.

Michigania 02005

Put Another Way

Expert: Great partial view

Crowd: So-so complete view

Michigania 02005

Diversity and Prediction

Diverse predictors generate better predictions unless someone’s head is large enough and data is sufficient enough for a complete model.

Michigania 02005

Preference Aggregation

Michigania 02005

Instrumental vs Fundamental

Fundamental Preferences: Preferences over outcomes

Instrumental Preferences: Preferences over policies to attain outcomes

Michigania 02005

Instrumental Politics“I am the _____ candidate”

A.Pro crimeB.Anti childC.Anti environmentalD.Pro drug addictionE. Higher health care costs

Michigania 02005

Preference Diversity Problems

• Preference Cycles

• Manipulation

• Underprovision

Michigania 02005

Preference Cycle

A = Arts & Crafts, B = Boating, T = Tennis

Lindsey: A > B > T Samuel: B > T > ABecca : T > A > B

Michigania 02005

Preference Cycle

Lindsey: A > B > T Samuel: B > T > ABecca : T > A > B

• Majority Vote Outcome: A > B > T > A

Michigania 02005

ManipulationGiven any voting rule, people with

diverse preferences will always have an incentive to misrepresent themselves.

Implication: People in diverse groups will not trust one another as much.

Michigania 02005

Under Provision

If we want different outcomes and have a fixed budget, we are likely to spread our money too thin.

Idea: Rather than have a good car or a nice boat, we have a lousy car and a lousy boat.

Michigania 02005

Theoretical Summary

Tasks involve- Solution Generation (problem solving)- Evaluation (prediction)- Choice (preference aggregation)

Michigania 02005

“Diversity is Ability”

To be different is to be able to make a contribution.

Diversity Trumps Ability: Diverse group does better than “able” group at problem solving

Michigania 02005

“Diversity is Ability”

Diversity Prediction TheoremCrowd Error = individual error -

diversity

(ability and diversity enter equally)

Michigania 02005

ComplicationPreference diversity creates cycles.

It creates incentives to act strategically and to manipulate agendas.

At the same time, preference diversity may be a primary cause of the other types of diversity.

Michigania 02005

SummaryThe empirical evidence suggests

that diverse perspectives, mental models, and tools lead to “better outcomes” but that value diversity creates problems.

Michigania 02005

Pudding

Michigania 02005

Quick Look at the ``Facts’’• Growth of modern civilization• National level GDP• City level productivity• Diverse team performance

Michigania 02005

Rise of Modern Civilization• Jared Diamond: diversity/easy

problems• Joel Mokyr: exploiting diversity• Michael Kremer: 1 million years of

data shows growth and population size correlated

Michigania 02005

National Level GDP• Paul Romer: Diversity crucial to

economic growth• Ethnic Linguistic Fractionalization

(ELF): strongly negatively correlated with economic growth

Michigania 02005

Performance of Cities (42)• Doubling of city size increases

productivity by 6% to 20%• Arrow, Lucas: spillovers within an

industry (silicon valley)• Jacobs, Auerbach: spillovers

between industries (just in time)*

Michigania 02005

Identity Diverse Teams• Generate more solutions (many

worse)• Thomas and Ely: do better if they

have diverse heuristics and perspectives

• People in diverse groups are less happy - world views are challenged - feel like outcomes were

manipulated

Michigania 02005

The End of Great Scientists First 10

Physics Nobels: 14 Chemistry Nobels: 10

Michigania 02005

The End of Great Scientists First 10 Last 10

Physics Nobels: 14 28Chemistry Nobels: 10 27

(There’s a maximum of three)

Michigania 02005

Final ThoughtIndividual ability not likely to grow

much.

Collective diversity can grow.

Diversity is our best hope to solve problems and to create innovations.

Michigania 02005

www.cscs.umich.edu/~spage