10-05-19 | 1

Rink Hoekstra

DFH Meetup

May 9th, 2019

Open Science: inevitable but how?

Why interested in Open science?

› A brief personal history

• 2003: Start PhD project on “use and usability of statistical techniques”

• 2005: Interview study on researchers’ handling of data sets

• 2011: Discovery fraud Diederik S.• > 2011: increasing attention for Open Science,

transparency, questionable research practices

10-05-19 | 2

10-05-19 | 3

Why Open Science?

10-05-19 | 4

› Enabling checking and verifying outcomes, which is (or should be) central to science

› Making outcomes accessible for stakeholders inside and outside academia (thus increasing its credibility)

› Improving quality of scientific research (possibly at the cost of quantity)

› Without it, some basic scientific norms are hard to follow

Norms in science (Merton, 1942)

› communism: all scientists should have common ownership of scientific goods (intellectual property), to promote collective collaboration; secrecy is the opposite of this norm.

› universalism: scientific validity is independent of the sociopolitical status/personal attributes of its participants

› disinterestedness: scientific institutions act for the benefit of a common scientific enterprise, rather than for the personal gain of individuals within them

› organized scepticism: scientific claims should be exposed to critical scrutiny before being accepted: both in methodology and institutional codes of conduct

10-05-19 | 5

› Norms are widely supported among scientists at very high rates (Anderson et al. 2007)

› However, current research practice has made it difficult to follow such norms

10-05-19 | 6

Open science is not dichotomous

› Often perceived as “for” or “against”, “in” or “out”

› An active open science community is advantageous to the cause, but it can also be perceived as divisive/exclusive

› Nevertheless, there’s hardly a serious debate on whether Open science, but merely on how

10-05-19 | 7

Progress, but practices not yet widely adopted

› Quite a few obstacles/challenges

› What are they? And how can be they overcome?

10-05-19 | 8

Challenges: 1) “Not everything can be open”

• Of course!• “As open as possible, as closed as necessary”

› Ideally, the default should be “Open, unless…”, instead of being dependent on individual people making choices • Which is not necessarily in contrast with academic

freedom!

10-05-19 | 9

A grass-roots example

› Peer reviewers’ openness initiative

› Group of researchers who accept reviewing conditional on whether…• …either data and materials are shared, OR• …a statement is made in the published paper why they

aren’t

10-05-19 | 10

The social dilemmaAs authors, incentives are...› Publish faster› Avoid work that doesn't benefit us

directly› Avoid possible embarrassment of

public mistakes

10-05-19 | 11

As reviewers, incentives are...› Encourage "better" science in

general› Get others to share their data› Avoid extra review workload› See possible mistakes

Challenge 2: No consensus among researchers

› Definitely true.

› However, there are multiple explanations1) Not everybody agrees2) Not everybody has the same information3) Not everybody has the same interests (personal and

scientific interests are not always in line; reluctance to change well-honed practices)

10-05-19 | 12

Challenge 3: Open science doesn’t make life easier

› Partly true.

› More information to be processed

› The simple (too simplistic?) metrics for evaluation need to be replaced• Shift from quantity to quality

› However, more ownership of scientific goods, scrutinizing much easier

10-05-19 | 13

Challenge 4: Getting everyone moving together is not easy

› Definitely not!

› Changing behaviour is difficult• Only a rational approach is insufficient• Incentive structures should change as well

10-05-19 | 14

› We should acknowledge personal interests, and try to minimize damage• NB: there are also many people who lose in the

current system

› Incentive structure should support rather than thwart those who adopt Open sciences practices

› Metrics should be a reflection of this

10-05-19 | 15

How?› Combination of top-down and bottom-up approach inevitable

› Top-down:• Create an environment in which open science practices are not

at odds with personal interests• Disseminate a coherent vision regarding open science

• Including ideas for a transition phase• Try to adopt (and propose) between-university initiatives

› Bottom-up• Inform policy makers/superiors of solutions and problems

10-05-19 | 16

› One bottom up initiative: • founding of Open science community Groningen

10-05-19 | 17