www.culturalcognition.net the climate-science literacy measurement problem—and how to fix it dan...

www.culturalcognition.net

The climate-science literacy measurement problem—and how to fix it

Research Supported by: Annenberg Center for Public Policy at the University of Pennsylvanalia

Dan M. KahanYale University

1. The identity-knowledge confound

2. How to detect & correct it

3. Report from an ongoing research program

What am I talking about?

The knowledge-identity confound . . . .


1. The Slovic-Peters-Leiserowitz model (SPL)



* The affect heuristic



Affect +/-

Risk perception




Affect +/-

Risk perception


perceivedbenefit

perceivedcost

anything else



Affect +/-

Risk perception


perceivedbenefit

perceivedcost

“trust”

Poortinga, W. & Pidgeon, N.F. Trust in Risk Regulation: Cause or Consequence of the Acceptability of GM Food? Risk Analysis 25, 199-209 (2005)



* The cultural theory of risk* The affect heuristic

Affect +/-

Risk perception

perceivedbenefit

perceivedcost

“trust”

cultural worldview

* Interpretive community



* The cultural theory of risk* The affect heuristic

Risk perception

perceivedbenefit

perceivedcost

“trust”

cultural worldview

* Interpretive community



2. “Climate change literacy” & SPL . . .


climate change science lit.

climate change risk perception





happening

caused by humans




Affect +/-





climate change risks perception

climate change “everything else”

Affect +/-





Affect +/-

happening

caused by humans





Cultural worldviews happening

caused by humans



GWRISK: “How much risk do you believe global warming pose to human health, safety, or prosperity?” [0 “None at all” to 7 “Extremely high”]

“How strongly do you disagree or agree with the following statements?” [1 “Strongly disagree” to 6 “strong agree”]

Source: Kahan, D.M., Hank, J.-S., Tarantola, T., Silva, C. & Braman, D. Geoengineering and Climate Change Polarization: Testing a Two-Channel Model of Science Communication. Annals of the American Academy of Political and Social Science 658, 192-222 (2015).

[0 “None at all” to 7 “Extremely high”]

0

1

2

3

4

5

6

7

-1.6 -1 0 1 1.6


0

1

2

3

4

5

6

7

-1.6 -1 0 1 1.6

0

1

2

3

4

5

6

7

-1.6 -1 0 1 1.6


0

1

2

3

4

5

6

7

-1.6 -1 0 1 1.6

0

1

2

3

4

5

6

7

-1.6 -1 0 1 1.6

r = - 0.65, p < 0.01

0%

25%

50%

75%

100%

Human caused Naturally caused No warming

Beliefs on global temperature “increase in recent decades”

N = 1,885. Annenberg Public Policy Center & Cultural Cognition Project. Nationally representative sample, June 2014 (YouGov). CIs are 0.95 confidence intervals for estimated general population means.


happening

caused by humans





1994 2010





happening

caused by humans

Affect +/-

causes skin cancer

caused by aerosols





happening

caused by humans

Cultural worldview

causes skin cancer

caused by aerosols




* “climate literacy” correlates with worldview

* two scales, not 1: “cause” & “no cause”

* worldview-“climate literacy” interaction



Essential tool: Item response theory

How to detect it . . . .

0

.1

.2

.3

.4

.5

.6

.7

.8

.9

1

-2.5 -1 0 1 2.5

0

.1

.2

.3

.4

.5

.6

.7

.8

.9

1

-2.5 -1 0 1 2.5

0

.1

.2

.3

.4

.5

.6

.7

.8

.9

1

-2.5 -1 0 1 2.5

“Group 2”

“Group 1”

“Group 2”

“Group 1”

“Group 2”

“Group 1”



0

.1

.2

.3

.4

.5

.6

.7

.8

.9

1

-2.5 -1 0 1 2.5

0

.1

.2

.3

.4

.5

.6

.7

.8

.9

1

-2.5 -1 0 1 2.5

0

.1

.2

.3

.4

.5

.6

.7

.8

.9

1

-2.5 -1 0 1 2.5

“Group 2”

“Group 1”

“Group 2”

“Group 1”

“Group 2”

“Group 1”



0

.1

.2

.3

.4

.5

.6

.7

.8

.9

1

-2.5 -1 0 1 2.5

“Group 2”

“Group 1”

0

.1

.2

.3

.4

.5

.6

.7

.8

.9

1

-2.5 -1 0 1 2.5

0

.1

.2

.3

.4

.5

.6

.7

.8

.9

1

-2.5 -1 0 1 2.5

0

.1

.2

.3

.4

.5

.6

.7

.8

.9

1

-2.5 -1 0 1 2.5

> avg. religiosity

< avg religiosity

> avg. religiosity

< avg religiosity

> avg. religiosity

< avg religiosity

> avg. religiosity


0

.1

.2

.3

.4

.5

.6

.7

.8

.9

1

-2.5 -1 0 1 2.5

> avg. religiosity

< avg religiosity


0

.1

.2

.3

.4

.5

.6

.7

.8

.9

1

-2.5 -1 0 1 2.5

0

.1

.2

.3

.4

.5

.6

.7

.8

.9

1

-2.5 -1 0 1 2.5

0

.1

.2

.3

.4

.5

.6

.7

.8

.9

1

-2.5 -1 0 1 2.5

> avg. religiosity

< avg religiosity

> avg. religiosity

< avg religiosity

> avg. religiosity

< avg religiosity

< avg religiosity

0

.1

.2

.3

.4

.5

.6

.7

.8

.9

1

-2.5 -1 0 1 2.5

There is “solid evidence” of recent global warming due “mostly” to “human activity such as burning fossil fuels.” [agree, disagree]



0

.1

.2

.3

.4

.5

.6

.7

.8

.9

1

-2.5 -1 0 1 2.5

0

.1

.2

.3

.4

.5

.6

.7

.8

.9

1

-2.5 -1 0 1 2.5

0

.1

.2

.3

.4

.5

.6

.7

.8

.9

1

-2.5 -1 0 1 2.5

> avg. religiosity

< avg religiosity

> avg. religiosity

< avg religiosity

> avg. religiosity

< avg religiosity

< avg religiosity

0

.1

.2

.3

.4

.5

.6

.7

.8

.9

1

-2.5 -1 0 1 2.5



0

.1

.2

.3

.4

.5

.6

.7

.8

.9

1

-2.5 -1 0 1 2.5

Liberal Democrat

Conservative Republican


0

.1

.2

.3

.4

.5

.6

.7

.8

.9

1

-2.5 -1 0 1 2.5

0

.1

.2

.3

.4

.5

.6

.7

.8

.9

1

-2.5 -1 0 1 2.5

0

.1

.2

.3

.4

.5

.6

.7

.8

.9

1

-2.5 -1 0 1 2.5

> avg. religiosity

< avg religiosity

< avg religiosity

< avg religiosity

< avg religiosity

0

.1

.2

.3

.4

.5

.6

.7

.8

.9

1

-2.5 -2 -1.5 -1 -.5 0 .5 1 1.5 2 2.5

0

.1

.2

.3

.4

.5

.6

.7

.8

.9

1

-2.5 -2 -1.5 -1 -.5 0 .5 1 1.5 2 2.5

0

.1

.2

.3

.4

.5

.6

.7

.8

.9

1

-2.5 -2 -1.5 -1 -.5 0 .5 1 1.5 2 2.5

0

.1

.2

.3

.4

.5

.6

.7

.8

.9

1

-2.5 -1 0 1 2.5

Liberal Democrat


Liberal Democrat


Liberal Democrat




0

.1

.2

.3

.4

.5

.6

.7

.8

.9

1

-2.5 -1 0 1 2.5

Liberal Democrat



How to fix it . . . .

How to fix it . . . . by unconfounding identity & knowledge!

0

.1

.2

.3

.4

.5

.6

.7

.8

.9

1

-2.5 -1 0 1 2.5

99th percentile1st percentile 84th percentile16th percentile 50th percentile

Prob

abili

ty o

f cor

rect

resp

onse

There is “solid evidence” of recent global warming due “mostly” to “human activity such as burning fossil fuels.”

Ordinary Science Intelligence

LiberalDemocrat

ConservativeRepublican

How to fix it . . . . by unconfounding identity & knowledge!

Report from ongoing research program . . .

“Ordinary Climate Science Intelligence” battery


“What gas do most scientists believe causes temperatures in the atmosphere to rise? Is it [hydrogen, helium, carbon dioxide, radon]?”





“Climate scientists believe that human-caused global warming will result in flooding of many coastal regions.” [True or False]

“Climate scientists believe that human-caused global warming will increase the risk of skin cancer in human beings.” [True or false]


OCSI: item response functions

0.1

.2.3

.4.5

.6.7

.8.9

1

-2 -1 0 1 2

0.1

.2.3

.4.5

.6.7

.8.9

1

-2 -1 0 1 2

bars denote 0.95 CIs


0.1

.2.3

.4.5

.6.7

.8.9

1

-2 -1 0 1 2




0

.1

.2

.3

.4

.5

.6

.7

.8

.9

1

-2.5 -1 0 1 2.5


Prob

abili

ty o

f cor

rect

resp

onse

There is “solid evidence” of recent global warming due “mostly” to “human activity such as burning fossil fuels.”

Ordinary Science Intelligence

LiberalDemocrat

ConservativeRepublican

OSI: item response functions


-2-1

01

2C

lima

te s

cie

nce

lite

racy

-2 -1 0 1 2science comprehension

Ord

inar

y cl

imat

e sc

ienc

e in

telli

genc

e

r = 0.32, p < 0.01

Ordinary science intelligence

1st percentile

86th percentile

14th percentile

99th percentile

50th percentile


shaded area denotes 0.95 CIs


OSI & OCSI: best of friends!

-2-1

01

2C

lima

te s

cie

nce

lite

racy


Ord

inar

y cl

imat

e sc

ienc

e in

telli

genc

e

r = 0.32, p < 0.01

Ordinary science intelligence


1st percentile

86th percentile

14th percentile

99th percentile

50th percentile

> avg Left_Right< avg Left_Right

-2-1

01

2C

lima

te s

cie

nce

lite

racy


shaded area denotes 0.95 CIs


OSI & OCSI: best of friends!

OCSI & positions on global warming

0

1

2

3

4

5

6

7

8

9

Human caused Naturally caused No warming

Positions on global warming in “past few decades”

0

1

2

3

4

5

6

7

8

9

0

1

2

3

4

5

6

7

8

9

No

. co

rrec

t





0.1

.2.3

.4.5

.6.7

.8.9

1

-2 -1 0 1 2


Report from ongoing research program . . .Unconfounding

What we know from Who we are

0.1

.2.3

.4.5

.6.7

.8.9

1

-2 -1.5 -1 -.5 0 .5 1 1.5 2

0.1

.2.3

.4.5

.6.7

.8.9

1

-2 -1.5 -1 -.5 0 .5 1 1.5 2

0.1

.2.3

.4.5

.6.7

.8.9

1

-2 -1.5 -1 -.5 0 .5 1 1.5 2

0.1

.2.3

.4.5

.6.7

.8.9

1

-2 -1.5 -1 -.5 0 .5 1 1.5 2

“Climate scientists believe that the increase of atmospheric carbon dioxide associated with the burning of fossil fuels will reduce photosynthesis by plants.” [True or False]


“Climate scientists believe that human-caused global warming will result in flooding of many coastal regions .” [True or False]

0.1

.2.3

.4.5

.6.7

.8.9

1

-2 -1.5 -1 -.5 0 .5 1 1.5 2

“Climate scientists believe that if the North Pole icecap melted as a result of human-caused global warming, global sea levels would rise.” [True or False]

“Climate scientists believe that human-caused global warming will increase the risk of skin cancer in human beings.” [True or False]

“Climate scientists believe that human-caused global warming has increased the number and severity of hurricanes around the world in recent decades.” [True or false]

“Climate scientists believe that nuclear power generation contributes to global warming” [True or false]

“Climate scientists believe that here will be positive as well as negative effects from human-caused global warming.” [True or false]

“Climate scientists believe that globally averaged surface air temperatures were higher for the first decade of the twenty-first century (2000-2009) than for the last decade of the twentieth century (1990-1999) [True or false]

0.1

.2.3

.4.5

.6.7

.8.9

1

-2 -1.5 -1 -.5 0 .5 1 1.5 2

0.1

.2.3

.4.5

.6.7

.8.9

1

-2 -1.5 -1 -.5 0 .5 1 1.5 2

0.1

.2.3

.4.5

.6.7

.8.9

1

-2 -1.5 -1 -.5 0 .5 1 1.5 2

0.1

.2.3

.4.5

.6.7

.8.9

1

-2 -1.5 -1 -.5 0 .5 1 1.5 2

Ordinary climate science intellience

prob

abili

ty o

f cor

rect

ans

wer


prob

abili

ty o

f cor

rect

ans

wer


prob

abili

ty o

f cor

rect

ans

wer

prob

abili

ty o

f cor

rect

ans

wer

prob

abili

ty o

f cor

rect

ans

wer

prob

abili

ty o

f cor

rect

ans

wer

Ordinary climate science intellience Ordinary climate science intellience



prob

abili

ty o

f cor

rect

ans

wer


prob

abili

ty o

f cor

rect

ans

wer

prob

abili

ty o

f cor

rect

ans

wer


“Ordinary climate science intelligence” item response curves

Figures plot the predicted probability of correctly responding to the item conditional on score on OCSI scale. Black bars denote 0.95 CIs. The figures can thus be used to assess the relative difficulty of the items and the precision with which they measure differences in comprehension.



“Ordinary Climate Science Intelligence” 2.0!


* test Shi/Tobler/Visschers/Siegrist on US sample

* retest best OCSI plus some rookies

* use IRT to construct best unconfounded synthesis

* test conjectures on OCSI & “scientific consensus”


The end!




New data: shame & critical reasoning!www.culturalcognition.net

www.culturalcognition.net the climate-science literacy measurement problem—and how to fix it dan...

Documents

humansthe slovicpeters

risk regulation

risk analysis

cultural theory of risk

correct it3

ongoing research programwhat

human health

global warming