false choices, policy framing, and the promise of “big data” · and the promise of “big...

33
False Choices, Policy Framing, and the Promise of “Big Data” Presidential Address November 8, 2013 Paul T. Decker, 2013 APPAM President; President, Mathematica Policy Research

Upload: others

Post on 11-Jul-2020

2 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: False Choices, Policy Framing, and the Promise of “Big Data” · and the Promise of “Big Data” ... • Realizing the full potential of “Big Data” is in our hands • Ask

False Choices, Policy Framing, and the Promise of “Big Data”

Presidential Address

November 8, 2013

Paul T. Decker, 2013 APPAM President; President, Mathematica Policy Research

Page 2: False Choices, Policy Framing, and the Promise of “Big Data” · and the Promise of “Big Data” ... • Realizing the full potential of “Big Data” is in our hands • Ask

It All Starts with Data

“In God we trust; all others bring data.” - W.E. Deming, Statistician

“Not everything of value can be measured.”

- Wall Street Journal, May 17, 2013

2

Page 3: False Choices, Policy Framing, and the Promise of “Big Data” · and the Promise of “Big Data” ... • Realizing the full potential of “Big Data” is in our hands • Ask

The False Choice Equation

3

Page 4: False Choices, Policy Framing, and the Promise of “Big Data” · and the Promise of “Big Data” ... • Realizing the full potential of “Big Data” is in our hands • Ask

The Nuanced Choice

4

Page 5: False Choices, Policy Framing, and the Promise of “Big Data” · and the Promise of “Big Data” ... • Realizing the full potential of “Big Data” is in our hands • Ask

Case in Point: Upward Bound

5

Page 6: False Choices, Policy Framing, and the Promise of “Big Data” · and the Promise of “Big Data” ... • Realizing the full potential of “Big Data” is in our hands • Ask

Advocates’ Response

6

1.Cast doubt on findings

2.Work to prohibit further research

Page 7: False Choices, Policy Framing, and the Promise of “Big Data” · and the Promise of “Big Data” ... • Realizing the full potential of “Big Data” is in our hands • Ask

How to Argue with Research You Don’t Like

Dylan Mathews, Wonkblog, Washington Post

7

Page 8: False Choices, Policy Framing, and the Promise of “Big Data” · and the Promise of “Big Data” ... • Realizing the full potential of “Big Data” is in our hands • Ask

Advocates’ Response

8

1.Cast doubt on findings

2.Work to prohibit further research

Page 9: False Choices, Policy Framing, and the Promise of “Big Data” · and the Promise of “Big Data” ... • Realizing the full potential of “Big Data” is in our hands • Ask

Advocates’ Response

“You can’t tell a kid, ‘You’re going to be in this life-changing program,’ and then say, ‘no, you’re only going to be in a control group.’”

9

Education Week

Page 10: False Choices, Policy Framing, and the Promise of “Big Data” · and the Promise of “Big Data” ... • Realizing the full potential of “Big Data” is in our hands • Ask

Advocates’ Response

“Random assignment is the wrong way to study their efforts because program operators typically require students to undergo an elaborate application process as a way to show commitment and winnow out candidates unlikely to succeed.”

10

Education Week

Page 11: False Choices, Policy Framing, and the Promise of “Big Data” · and the Promise of “Big Data” ... • Realizing the full potential of “Big Data” is in our hands • Ask

Spend More Time on Approaches

“If I had an hour to solve a problem I'd spend 55 minutes thinking about the problem and 5 minutes thinking about solutions.”

- Albert Einstein

11

Page 12: False Choices, Policy Framing, and the Promise of “Big Data” · and the Promise of “Big Data” ... • Realizing the full potential of “Big Data” is in our hands • Ask

Reframing the Evidence Conversation

12

• Stable commitment of resources

• Evidence is generated on a continual basis

• Research becomes an ally not a threat

• Programs evolve in their populations’ best interest

Page 13: False Choices, Policy Framing, and the Promise of “Big Data” · and the Promise of “Big Data” ... • Realizing the full potential of “Big Data” is in our hands • Ask

Premature Obituary: Job Corps

13

Page 14: False Choices, Policy Framing, and the Promise of “Big Data” · and the Promise of “Big Data” ... • Realizing the full potential of “Big Data” is in our hands • Ask

Job Corps Earning Impacts

14

Average Earnings Per Week by Quarter (1995 Dollars)

Page 15: False Choices, Policy Framing, and the Promise of “Big Data” · and the Promise of “Big Data” ... • Realizing the full potential of “Big Data” is in our hands • Ask

Paul’s Policy and Practice Takeaways

15

1. Continue to support and generate evidence

2. Re-frame the evidence conversation beyond the binary choice

3. Don’t be shy about connecting the dots

Page 16: False Choices, Policy Framing, and the Promise of “Big Data” · and the Promise of “Big Data” ... • Realizing the full potential of “Big Data” is in our hands • Ask

APPAM and “Big Data”

16

Page 17: False Choices, Policy Framing, and the Promise of “Big Data” · and the Promise of “Big Data” ... • Realizing the full potential of “Big Data” is in our hands • Ask

A Symbiotic Relationship With Technology

Page 18: False Choices, Policy Framing, and the Promise of “Big Data” · and the Promise of “Big Data” ... • Realizing the full potential of “Big Data” is in our hands • Ask

Disruptive Innovation Defined

“Technologies, processes or business models that bring to market products or services that are more affordable and simpler to use than what is currently available. It enables more consumers in that market to afford and/or have the skill to use the product or service. The change caused by such an innovation is so big that it eventually replaces, or disrupts, the established approach to providing that product or service.”

– Clayton Christensen

18

Page 19: False Choices, Policy Framing, and the Promise of “Big Data” · and the Promise of “Big Data” ... • Realizing the full potential of “Big Data” is in our hands • Ask

“Big Data”

Page 20: False Choices, Policy Framing, and the Promise of “Big Data” · and the Promise of “Big Data” ... • Realizing the full potential of “Big Data” is in our hands • Ask

“Big Data”

20

Page 21: False Choices, Policy Framing, and the Promise of “Big Data” · and the Promise of “Big Data” ... • Realizing the full potential of “Big Data” is in our hands • Ask

“Big Data”

21

“Less than $1 out of every $100 of government spending is backed by even the most basic evidence that the money is being spent wisely.”

Page 22: False Choices, Policy Framing, and the Promise of “Big Data” · and the Promise of “Big Data” ... • Realizing the full potential of “Big Data” is in our hands • Ask

Data Tsunami

22

Page 23: False Choices, Policy Framing, and the Promise of “Big Data” · and the Promise of “Big Data” ... • Realizing the full potential of “Big Data” is in our hands • Ask

“Juking the Numbers”

23

Page 24: False Choices, Policy Framing, and the Promise of “Big Data” · and the Promise of “Big Data” ... • Realizing the full potential of “Big Data” is in our hands • Ask

APPAM’s Sweet Spot

24

Page 25: False Choices, Policy Framing, and the Promise of “Big Data” · and the Promise of “Big Data” ... • Realizing the full potential of “Big Data” is in our hands • Ask

No Research Expertise Required

25

Page 26: False Choices, Policy Framing, and the Promise of “Big Data” · and the Promise of “Big Data” ... • Realizing the full potential of “Big Data” is in our hands • Ask

The APPAM Advantage

26

• Public management experience

• Methods expertise

• Policy analysis and evaluation

• Translation of data and research into practice

Page 27: False Choices, Policy Framing, and the Promise of “Big Data” · and the Promise of “Big Data” ... • Realizing the full potential of “Big Data” is in our hands • Ask

Democratization of Data

27

Page 28: False Choices, Policy Framing, and the Promise of “Big Data” · and the Promise of “Big Data” ... • Realizing the full potential of “Big Data” is in our hands • Ask

Half Full?

28

Page 29: False Choices, Policy Framing, and the Promise of “Big Data” · and the Promise of “Big Data” ... • Realizing the full potential of “Big Data” is in our hands • Ask

Conclusion

29

• Commit to research and evidence, especially when it’s inconsistent with our initial beliefs.

• We must remain nonpartisan and objective

• Realizing the full potential of “Big Data” is in our hands

• Ask the questions and set the framework, what gets asked gets answered.

Page 30: False Choices, Policy Framing, and the Promise of “Big Data” · and the Promise of “Big Data” ... • Realizing the full potential of “Big Data” is in our hands • Ask

Pythagoras

30

Page 31: False Choices, Policy Framing, and the Promise of “Big Data” · and the Promise of “Big Data” ... • Realizing the full potential of “Big Data” is in our hands • Ask

John Snow and the Broad Street Water Pump

31

Page 32: False Choices, Policy Framing, and the Promise of “Big Data” · and the Promise of “Big Data” ... • Realizing the full potential of “Big Data” is in our hands • Ask

Joel Fleischman

32

•Seed a problematic field with research and trials

•Define clear and achievable goals

•Devise evidence-based strategies to get there

•Help guide society toward that sweet spot on the horizon