missingevidence digitalpdf singlepages (2)

8/16/2019 MissingEvidence DigitalPDF SinglePages (2)

http://slidepdf.com/reader/full/missingevidence-digitalpdf-singlepages-2 1/60



CONTENTS

1 Introduction

2 Summary

3 Section 1: Is there a problem?

5 Section 2: The system of conductin , commissionin and publishin overnment research

11 Section 3: Reasons for the delayed publication or withholdin of overnment research

30 Section 4: Remedies and recommendations

37 Afterword Tracey Brown, director of Sense about Science

39 Appendix I: Inquiry methods and call for evidence

41 Appendix II: List of contributors

45 Appendix III: Freedom of Information requests

47 Appendix IV: Biblio raphy



5+# D"2*#, E2".,(/ .(-#)"/#"* 7;#",7 &0(1* KL4M 02882(" ! a year on research intended to guide,develop, modify and monitor policy on a wide variety of issues. Some of this policy researchis done in-house. Some of it is conducted by outside individuals and bodies who are paid to do it.

=*7 &2/7 %#)# *(?

N Understand what has happened in known casesof delay in research publication.

N C## +(% ;)(08#/&*2> )#7#&)>+ >&" 0# +&",8#, %#884

N =,#"*23' ;(() ;)&>*2># 2" *+# >(//2772("2".and dissemination of government research.

N G)&% &**#"*2(" *( .((, ;)&>*2># *+&* >(18,and should be replicated.

5+27 27 &" (;;()*1"# /(/#"* *( >("72,#) *+# -2720282*'(3 *+# #-2,#"># 0&7# 3() ;1082> ;(82>'4 O(-#)"/#"*&", >2-28 7(>2#*' &)# ,#-#8(;2". *+# DE67 *+2), F&*2("&8Action Plan on Open Government, which will proposeways to make government more open and accountable.The Independent Commission on Freedomof Information has reinforced the principle of openness.

Introducing its report to parliament in March 2016,Lord Bridges said, “We are committed to being the mosttransparent government in the world.” In 2015 the WorldWide Web Foundation ranked the UK as the mosttransparent government in the world. Yet, as this reportdemonstrates, the withholding of external research from;1082> 7>)1*2"' )#/&2"7 & ;)(08#/4

At a time of public spending constraints, it is alsoto be expected that government will seek opportunitiesto increase the value derived from research, reduce wasteand identify ineffective or mistaken policy decisions.

This inquiry was initiated in September 2015. Its scopewas determined by initial consultation with researchers,civil servants and politicians. The individualsin the scienti c, policy and political sectors whoprovided written submissions, and those with whomwe conducted interviews, are listed in Appendix II.Some are quoted below, " but this report has bene tedgreatly from contact with all of them. Its content,though, remains the author’s responsibility.

5+# 2":12)' &", *+# ;)(,1>*2(" (3 *+27 )#;()* %#)#71;;()*#, 0' & .)&"* 3)(/ *+# PQCC5 R+&)2*&08#Trust (Registered Charity No: 247498) and by Nick

Q(774 G#*&287 (3 *+# 2":12)'67 /#*+(,7 &", *+# >&88for evidence are in Appendix I.

INTRODUCTION

5+27 )#;()* 27 >(">#)"#, %2*+ *+# ;1082>&*2("of research which government has !"##$%%$"&'( 3)(/#B*#)"&8 )#7#&)>+#)74 5+# )#&7("7 3() *+27 &)# 0(*+practical and legal. The volume of research done withindepartments, and the variety of forms it takes, are hugeand (apart from the estimated global cost) unquanti ed.The routine announcement and publication of allsuch research would not only create a considerable&,/2"27*)&*2-# 01),#"S 2* %(18, )#:12)# *+# 7#.)#.&*2("of research ndings from policy advice to ministerswhich at present, whether rightly or wrongly, enjoys2//1"2*' 3)(/ ,27>8(71)# #-#" 2" 82*2.&*2("4 5+# >&7#3() *)&"7;&)#">' &0(1* 2"*)&9,#;&)*/#"*&8 2";1*72"*( ;(82>' 3()/&*2(" 27 *+#)#3()# ,233#)#"* 3)(/the case, which this report examines, for invariablepublication of expert research and ndings externally>(//2772("#, 0' .(-#)"/#"*4

Government, which readily recognises that the public27 #"*2*8#, *( 7## *+# ;)(,1>* (3 #B*#)"&8 )#7#&)>+ ;&2,for with its money, has set guidelines and protocols thatrequire prompt publication. All departments re ect this)#:12)#/#"* 2" *+#2) )#7#&)>+ 7*)&*#.2#74

F("#*+#8#77 .(-#)"/#"* )#7#&)>+ 27 "(* &8%&'7published in suf cient time for informed public,27>1772("4 5+#)# +&-# 0##" 2"7*&">#7 (3 ;1082>&*2("being held back when the ndings were politicallyawkward. Such cases have included research on foodbanks, immigration, choice of GP and drugs policy.

5+#7# 2"7*&">#7 +&-# 8#, *( >(">#)" &0(1*?N Failure to make publicly-funded research available

to the public.

N A lack of transparency about the basisof government decisions and the role thatevidence played in reaching them.

N T )#,1>#, %2882"."#77 (3 )#7#&)>+#)7 *( &7727*in policymaking.

In response to these public concerns, Sense aboutC>2#"># 2"2*2&*#, *+27 2":12)' 2"*( *+# ,#8&'#, ;1082>&*2("of government research, and into possible remedies.

G#*&287 (3 *+# &1*+()7+2; (3 *+# )#;()* &)# 7#* (1*in Appendix I.

1 INTRODUCTION



No comprehensive account exists of how much government research is commissioned,nor of how much of it is published and when. This in itself is a signi cant problem,and it is why this report recommends that there should be a single publicly searchabledatabase of all external government research.

has already been submitted for use by government.Similarly, a journal’s print schedule should not affectthe timeliness of research for public discussionof policy; reviewed preprints should be usedwhere necessary.

Analysis of the reasons for delayed publication,and of practices that allow dif cult researchto be handled well, points to the following)#>(//#",&*2("7?

) & 7*&",&),27#, >#"*)&8 )#.27*#) (3 &88 #B*#)"&88'>(//2772("#, .(-#)"/#"* )#7#&)>+4

)) >8&)2*' (" %+&* >("7*2*1*#7 #B*#)"&88'>(//2772("#, .(-#)"/#"* )#7#&)>+4

))) & >8#&) >(//2*/#"* *( ;)(/;* ;1082>&*2("in research contracts.

)* )(1*2"# ;1082>&*2(" (3 )#7#&)>+ .(-#)"/#"*has considered in policy formulation with,if appropriate, reasons for rejecting it.

* & >8#&) 7*&*#/#"* (3 *+# >1))#"* )#:12)#/#"*7for prompt publication and adherence to them.

*) *)&2"2". 2" )#7#&)>+ 3() ;(82>' >(//1"2>&*()74

5+#7# )#>(//#",&*2("7 &)# 7#* (1* 2" /()# ,#*&28at the end of the report.

SUMMARY

The most prominent issue is delay in the publicationof research ndings to accommodate political concernsabout policy announcements. If government policyis formulated and announced before the mediaand the public know what expert evidence ministershave received, public discussion (including parliamentaryscrutiny) may be handicapped or sti ed. Delay designedsimply to avoid political embarrassment is not ethicallyacceptable, and this report puts forward proposalsto ensure timely public access to research even —indeed especially — where it con icts with policy.It will be suggested that research should ordinarilybe published as soon as it is ready, and ahead of policy&""(1">#/#"*7 1"8#77 *+27 27 .#"12"#8' 2/;)&>*2>&08#4G#3#))&8 0#'(", *+# *2/# (3 ;(82>' ;)(/18.&*2(" 7+(18,require very strong justi cation.

A' ;1082>&*2(" %# /#&" ("82"# &>>#77 *( *+# )#;()*of the research. It appears currently that uncertainties(-#) 3()/&* &", &"' )#-2#% ;)(>#77 >&" ,#8&'publication, and some witnesses have describedconcerns that this uncertainty can be used intentionally.5+# ;)&>*2># 7+(18, 0# *( &.)## *+# *';# &", 3()/&*of the research report as part of the commission.T"' )#-2#% ;)(>#77 &", *2/2". (3 *+&* 7+(18, &87(be agreed in advance. Plans for a journal publicationare not a reason for delaying publication where a repor t

! The most recent year for which Of ce for National Statistics (ONS) gures areavailable is 2013. The 2013 ONS bulletin on UK Government Expenditure on Science,Engineering and Technology also shows that government spent £5 billion on in-houseR&D, purchased R&D and on funding external organisations to run R&D, both in theUK and overseas. It is dif cult to tell how much of this was spent on commissionedresearch. Government separately spent £5.9 billion on funding research councils.The research done by these institutional bodies is not within the scope of this inquiry.

" Quotations, unless otherwise stated in the textor footnotes, are from submissions to the inquiry.

LSUMMARY



!"! =* /2.+* +&-# 0##" *+(1.+* *+&* &"' ;)(08#/ (3 ,#8&'#, ;1082>&*2(" (3 .(-#)"/#"* )#7#&)>+ >(18,be readily identi ed and quanti ed. Each department of state and executive agency must knowwhat research contracts it has issued in, say, the last three years, and be able to say whetherthe researchers have reported and, if they have, when their report was made public.

!"# It has been disturbing to nd that this is not how things are. The extent of reporting and recordkeeping varies enormously between government departments. Using Freedom of Information(FoI) requests, it has emerged that some departments — eg the Department for Environment,Food and Rural Affairs, the Department for Transport and the Department of Health — give onlineaccess to databases that set out what research has been commissioned and what stageit has reached. Others — eg the Department for Communities and Local Government (DCLG) —only list published studies without indicating whether the lists contain all research that has beencommissioned. See Appendix III for full details of the FoI requests.

!"$ Some departments responded to FoI requests by supplementing their public lists with more completeinformation. DCLG provided a comprehensive list of studies commissioned, completed and publishedbetween 2013 and 2015. By contrast, the Department of Energy and Climate Change and the ForeignOf ce did not know what had been commissioned because that information was dispersed withinthe department and not held centrally. Adding to the dif culty of nding out about governmentresearch, it has been suggested to us that incoming administrations may sometimes consign

directly to the National Archives research bespoken by their predecessors.!"% In the absence of the comprehensive information that ought to be available, it has been necessary

*( #B&/2"# %+&* +&7 +&;;#"#, 2" >&7#7 %+#)# *+# 3&281)# *( ;10827+ .(-#)"/#"* )#7#&)>+ +&, 0##")#;()*#, 2" *+# /#,2& () #87#%+#)#4

!"& We know from a Supreme Court judgment that in 2007 the home secretary withheld publicationof research that failed to support a proposed policy of denying visas to spouses where oneof the couple was under 21. But concerns over the independence of government research reachfurther back: Professor Roger Tarling, former head of the Home Of ce Research and Planning Unit,wrote in 2011 about strained relations between government and social researchers during the 1980sand 90s, and in 1988 the Association of Social Research Organisations reviewed conditions aboutpublication in research contracts because their members viewed them as becoming “increasingly

("#)(17 &", )#7*)2>*2-#4V!"' There is no recent evidence of the inde nite suppression of research. However, there is good

evidence that research publication does get delayed. The reasons, in addition to political concernsabout the timing of publication, may include uncertainty about peer review, about what countsas government research, and about what should be published in relation to policy announcements.Yet delayed publication can be as damaging as inde nite suppression 0#>&17# 2* ,#;)2-#7parliamentarians, the media, NGOs and others of the timely access they need in order to be ableto engage with policy formation in the light of contemporaneous evidence.

!"( There will always be cases in which government is doubtful about or dissatis ed with the qualityof the research. This does not ordinarily justify delay in publication: the more appropriate courseis for openly stated grounds of doubt or disagreement to accompany publication.

T7 Professor Sir Nigel Shadbolt *(8, 17?

“A signi cant reason to publish openly is that ‘many eyes’ on the materialis one way to quickly highlight shortcomings in the work itself —open publication tends to improve the quality of data and analysis.”

!") 5+# 2/;(7720282*' (3 &77#/082". & 3188 ;2>*1)# (3 .(-#)"/#"* )#7#&)>+ +&7 2"#81>*&08' 0#>(/#an aspect of this inquiry. This report includes some remedies. First, however, it is necessaryto describe what counts as government research and to sketch the rules presently governingit (Section 2). Section 3 describes features of the problem of delayed publication from evidencein the public domain or obtained in the course of the inquiry. Section 4 describes some encouraging#B&/;8#7 (3 .(-#)"/#"* 0(,2#7 ;10827+2". )#7#&)>+ ,#7;2*# >+&88#".#7 () >("*)&,2>*2("7 %2*+ >1))#"*policy. The report concludes with recommendations for making decisions about publication moreconsistent, fair and open, bearing in mind that protocol and practice should not be such as to inhibitministers and civil servants from commissioning research on politically dif cult questions.

WSECTION 1: IS THERE A PROBLEM?



#"' At the start of this inquiry it was assumed that departmental chief scienti c advisers (CSAs) would>(">#)" *+#/7#8-#7 %2*+ &88 )#7#&)>+ 2" *+#2) ,#;&)*/#"* &", +&-# &" (-#)-2#% (3 %+&* +&;;#"7 *( 2*4It appears this is not always the case. The Government Of ce for Science’s document Chief Scienti c Advisers and their of cials: an introduction (2015) sets out how CSAs provide scienti c adviceto ministers and help prepare their department’s research strategies that set broad objectives.They also oversee research in the natural sciences. But we found that the way CSAs carry out their role,

their seniority and their access to ministers and permanent secretaries varies between departments.CSAs have a minimal role in overseeing social scienti c research, which forms a signi cant proportionof research in many departments and is governed by the separate government social research(GSR) profession.

Government research

#"( The system of government research is vast, complicated and fragmented. 5+# LW /2"27*#)2&8departments in Whitehall, and the hundreds of public bodies and agencies that report to them,take very different approaches to conducting, commissioning and publishing research. So dothe devolved administrations of Scotland, Northern Ireland and Wales. This section summarisesthe system, the recognition in existing guidance and protocols that ndings should be publishedpromptly, and whether this requirement is well-understood and followed.

#") O(-#)"/#"* 31",7 )#7#&)>+ 2" *%( %&'7? *+)(1.+ *+# 7>2#"># 01,.#* X&,/2"27*#)#, 0' *+# G#;&)*/#"*for Business, Innovation and Skills and mainly distributed through research councils and universities),and through research conducted or commissioned by departments of state, executive agenciesand arm’s length bodies in their own policy areas.

#"* A range of researchers carries out government research: civil servants working within Whitehall,#;&)*/#"*7S *+(7# 2" &)/67 8#".*+ 0(,2#7 &", #B#>1*2-# &.#">2#7 2">81,2". ;1082> 7#>*() )#7#&)>+bodies; independent researchers in universities and research institutes; and consultancies.

#"!+ O(-#)"/#"* >(",1>*7 )#7#&)>+ 3() (0@#>*2-#7 )1""2". 3)(/ &77#772". *+# #B*#"* (3 "&*1)&8 &", 7(>2&8;)(08#/7 *( 2"3()/2". *+# ,#-#8(;/#"* (3 ;(82>' &", #-&81&*2". 2*7 2/;&>*4

#"!! The Government Of ce for Science explained to us: “As with research in any other context, thereis a spectrum of increasing formality, ranging from basic ‘desk research,’ through more formallymanaged projects, to research which is subject to formal peer review.”

#"!# Research conducted for government is governed by a variety of rules, principles, guidance and protocols,which use their own de nitions of what counts as research and when and how it should be published.These are discussed in paragraphs 2.20-2.23. Section 3 addresses whether these de nitions createuncertainty over what should be published and when, and makes recommendations for greater clarity.

Reasons for publishin overnment research

#"!$ Publishing government research facilitates scrutiny of what is being done or proposed on behalfof the public. This matters more in an era when government frequently points to evidenceto justify policy.

#"!% The rules and statements that cover government research acknowledge that research fundedby taxpayers should be made publicly available.

#"!& Publication of government-commissioned studies is among other things a mechanism for ensuringthat departments know what research they have done themselves in the past, and enables them to avoidduplicating their own research. Although one cannot estimate the extent of such duplication becauseof the lack of data about research, civil servants say that departments spend signi cant time tryingto nd past studies that they conducted or paid for. Y10827+2". .(-#)"/#"* )#7#&)>+ &87( #"&08#7departments to learn from each other’s ndings.

Z SECTION 2: THE SYSTEM OF CONDUCTING, COMMISSIONING AND PUBLISHING GOVERNMENT RESEARCH



#"!' An important value of external research is that it reduces the risk that government research willbe slanted, or perceived to be so.

#"!( Full publication can also help to ensure that the evidence from government research informs widerscienti c knowledge by giving researchers access to a larger body of ndings.

#"!) Researchers who have experienced delays in their research being published told this inquiry thatit has affected their willingness to put themselves forward to run government-commissionedresearch in the future. 5+# 0&, ;1082>2*' &77(>2&*#, %2*+ *+# >&7#7 ,#7>)20#, 2" C#>*2(" U /&' +&-#in uenced others in this regard. It is clearly not in government’s interest, or in the public interest,if there is a narrowing pool of people who are willing to work on government contracts.

#"!* H2**8# () "( ,277#"* 3)(/ *+#7# 0)(&, ;)(;(72*2("7 %&7 #">(1"*#)#, 2" *+# >(1)7# (3 *+27 2":12)'4There is a general expectation that the results of government research should be made publiclyand promptly available.

For example, Andrew Miller, former MP and chair of the House of Commons science and technologyselect committee , said:

“The goal should be a culture where government aims to ‘suck in’ as much goodquality advice and data [as] it can, putting as much as possible in the public domain— with the default question being ‘why shouldn’t we publish all of this?”

Sir Venki Ramakrishnan, president of the Royal Society , 7&2,?

“It is important that government is transparent about how policy decisions havebeen made. To enable policymaking to be properly scrutinised, the evidenceon which it is made should be available. This includes data held by the governmentand the ndings of commissioned research.”

Dr Sarah Wollaston MP, chair of the House of Commons health select committee , said:

“If research is withheld, how can the public judge whether it re ects scienti cconsensus, whether it’s correct, and how it links if at all to policy decisions?Of course those decisions take account of other considerations, but the evidencebase for them should be open to scrutiny.”

Rules require the prompt and full publication of overnment research

#"#+ Government recognises the public expectation of publication, codifying it in rules requiring promptand complete publication of research conducted or commissioned by Whitehall depar tments. Howeverthese rules are not clearly de ned; in some areas they are strongly stated and in others weakly implied.

#"#! O(-#)"/#"* )#7#&)>+ 2" *+# "&*1)&8 7>2#">#7 27 .(-#)"#, 0' *+# +2.+98#-#8 ;)2">2;8#7 7#* (1*in Principles of scienti c advice to government (Government Of ce for Science, 2010a). Furtherguidelines set out how these principles should be applied by scienti c advisory committeesand departments. These include: The Government Chief Scienti c Adviser’s Guidelines on the Useof Scienti c and Engineering Advice in Policy Making (Government Of ce for Science, 2010b),which set out “how scienti c and engineering advice should be sought and applied,” and Chief Scienti c Advisers and their of cials: an introduction , which “sets out broad guidelines for the work of chiefscienti c advisers in departments.” In addition, each department sets a research strategy reaf rmingthe commitment to publishing its research in full.

#"## 5+# ;1082>&*2(" (3 .(-#)"/#"* 7(>2&8 7>2#"># )#7#&)>+ 27 >(-#)#, 0' *+# OCQ ;)(3#772("67 ;1082>&*2("protocol (Government Economic & Social Research Team, 2015).

[SECTION 2: THE SYSTEM OF CONDUCTING, COMMISSIONING AND PUBLISHING GOVERNMENT RESEARCH



$"& The creation of host research In those departments where there is no mechanism to nd or track research, there is a risk that of cials

will not be able to nd research at all. It may be that only the of cials who commissioned the researchknow about it. Once of cials leave their post the knowledge is lost and, if the research has not beenpublished in an accessible body of literature, it leaves no record and its results will never be used.It becomes ghost research.

$"' The lack of a full account of what happens to government research makes it dif cult to assess,from the known cases of delay, how much the system and procedure contribute.

T7 David Walker, head of policy at the Academy of Social Sciences , said:

“Evidence has to be more than anecdotal for the problem [of research beingwithheld] to be considered systemic.”

$"( 5+# "##, 3() )18#7 &", .12,#82"#7 ;)2()2*272". #&)8' ;1082>&*2(" /&' 2" 3&>* 2",2>&*# )#>(."2*2("of pressures within the system to do the opposite. We note that procedures for the publicationof government social research were recently upgraded from guidance to protocol.

Professor Dame Sally Davies, chief medical of cer for England , told the inquiry that the systemsin place now support publication:

“Although a decade or more ago there may have been more of a problem withresearch being delayed, clearer guidance and publication frameworks in place todaymean there isn’t a major problem anymore.”

$") This report looks at some prominent examples of delayed publication that have come to light chie ythrough media reports. But evidence submitted to the inquiry, including that on the publication of policytrials and statistical analysis, suggests that delays may be more frequent.

Ed Humpherson, head of assessment, UK Statistics Authority , said:

“UKSA nd it unacceptable for [public] claims to be made without everyone havingaccess to the analysis behind them, so we press for publication. That we haveto do this repeatedly means that the principles of prompt publication found in codes

governing research might not be that strongly embedded.”

$"* 5+# 8#-#8 (3 ,27().&"27&*2(" ,#7>)20#, &0(-# /&' %#88 0# >("*)201*2". *( 7;#>18&*2(" *+&* ;1082>&*2("of research is routinely delayed for political reasons. If so, this is both unnecessary and harmfulto government, since it is likely that the majority of external research studies are published without1",1# ,#8&'4

$"!+ Although systematic appraisal of publication patterns is currently impossible, it is possible to assessthe circumstances in which known delays have occurred. Cases of delayed publication and evidencesubmitted to this inquiry suggest that government research may be held back for reasons including:

N the desire to align publication with the announcement of policy, resulting in delayed publicationof ndings that are politically inconvenient;

N 1">#)*&2"*' (-#) +(% ;##) )#-2#% &", ,27>1772("7 &0(1* )#7#&)>+ :1&82*' &", -&82,2*' 7+(18,be handled; and

N uncertainty over what counts as government research, and differences in the interpretationof the rules.

$"!! 5+# *+()"2#7* ;)(08#/ 27 *+# &,@17*/#"* (3 *+# *2/2". (3 ;1082>&*2(" 3() ;(82*2>&8 )#&7("74

SECTION 3: REASONS FOR THE DELAYED PUBLICATION OR WITHHOLDING OF GOVERNMENT RESEARCH13



Uncertainty over how peer review and discussions about research qualityand validity should be handled

$"#$ 5+27 )#;()* "(*#7 *+)## >&7#7 %+#)# *+# ;##) )#-2#% ;)(>#77 %&7 >2*#, &7 ;&)* (3 *+# )#&7("for delayed publication. When publication of research on the rising use of food banks (Case Study 6),

and on evaluating changes to GP practice boundaries (Case Study 7), was delayed, governmentspokespersons said that the reports were awaiting peer review.

$"#% Case Study 8 shows the dif culties that can arise if government withholds research,even if this is because of doubts over the quality of research, without explaining its reasons.

Case Studies

! 5+# #33#>*7 (3 /2"2/1/ &8>(+(8 ;)2>2". Y&.# LU

# Q#,1>2". 71.&) >("71/;*2(" Y&.# LW

$ 5+# +()7#/#&* 7>&",&8 Y&.# LM

% The effect of immigration on unemployment Page 26

& ="*#)"&*2("&8 >(/;&)27(" (3 ,)1. 8&%7 Y&.# LZ

' The increasing use of food banks Page 28

( R+((72". & OY &%&' 3)(/ %+#)# '(1 82-# Y&.# L]

) 5+# /2"2/1/ &.# 3() & /&))2&.# #"*)' -27& Page 30

* The effect of fracking on house prices Page 31

17 SECTION 3: REASONS FOR THE DELAYED PUBLICATION OR WITHHOLDING OF GOVERNMENT RESEARCH



Awkward research does et published

%"! Contrary to the impression created by the behaviour in cases where publication has been delayed,many researchers and civil servants are familiar with strategies that can help governmentto commission and publish controversial research without fear of how the public or the media

might respond. Some examples are described below. These include publishing more informationabout departmental research, appointing ad hoc committees to oversee research, commissionersand researchers working closely together to set questions and agree on publication timing and process,and allowing the reanalysis of sensitive data. These examples inform the recommendationsat the end of this section to improve the visibility and utility of government research.

Publishin more information about overnment research

%"# The Department for Environment, Food and Rural Affairs (Defra), Department for InternationalDevelopment, Department for Transport (DfT) and Department of Health have public databaseswhere research is registered and progress to publication is tracked. There should be a single, publicly

searchable database used by all departments *+&* 0128,7 (" *+# /(7* 17#318 3#&*1)#7 (3 *+#7#,#;&)*/#"*&8 7'7*#/7 XQ#>(//#",&*2(" =\4 5+27 %(18, #"&08# ;#(;8# *( (0*&2" 2"3()/&*2(" &0(1*planned research as well as to locate and use the reports of completed work.

%"$ Departments might also consider arrangements, such as those used by the Welsh governmentin its Emerging Findings Reports, to make early research ndings or interim ndings available,with the necessary caveats about peer review, in certain cases where there is a pressing needfor research to contribute to current policy discussions.

%"% Publicly searchable research databases G#3)&67 C>2#"># C#&)>+ %#072*# +&7 & ;1082>8' 7#&)>+&08# ,&*&0&7# (3 &88 )#7#&)>+ >(//2772("#,

from third parties. This is based on what Defra staff, who are civil servants, enter into Omnicom,their internal database for managing all information related to research. Internal quality assuranceand audit is undertaken by the Omnicom support team to ensure that Defra research managersand procurement staff are using the database and completing information appropriately. Omnicomis also used as a document repository to help the department track pre-project ideas, procurementdocuments such as contracts, meeting notes during the live stage of the project, and reports.

%"& Omnicom allows the department to review its whole research landscape. Intended publication date is,however, not a eld in the system, although start and nish dates are — all deliverables are to be nalisedby nish date, after which peer review and other quality assurance activities start.

%"' Mappin research on to overnment policy =* %&7 3()/#)8' *+# ;)&>*2># *+&* .(-#)"/#"* ,#;&)*/#"*7 %(18, ;10827+ & ;)(@#>*2(" *+&* 7+(%#, %+&*

research was planned over the coming year, along the lines of the Home Of ce’s research programme.

5+27 ;)&>*2># &;;#&)7 *( +&-# 8&).#8' #",#, 3(88(%2". 1">#)*&2"*' (-#) ,#;&)*/#"*&8 )#7#&)>+budgets in 2010. Publishing a forward look would enable other sectors and departments, the researchcommunity, parliament and the public to see planned research alongside the government’s policypriorities. The Welsh government’s Forthcoming Publications web page continues to ful ll this function.

%"( Although policy often moves faster than research, this does not mean that the public cannotscrutinise the extant evidence base. The Welsh government has developed Emerging Findings Reports,which they ask researchers and departments to prepare when an urgent policy announcement needsto be made before full research ndings are available.

31 SECTION 4: REMEDIES AND RECOMMENDATIONS



Ensurin the independent commissionin and publication of overnment research

%") A number of individuals giving evidence to this inquiry warned that a blanket call for all government-commissioned research to be openly published might chill policymakers’ willingness to commissionresearch. This does not appear to have been the case where a research register is in operation,

eg at Defra, and all research is tracked in public view.%"* =* +&7 0##" 71..#7*#, *+&* ("# %&' *( &88#-2&*# *+# ;)#771)#7 (" *+# >(//2772("2". &", ;1082>&*2("

of government research would be to separate the analytical and policymaking functions of every,#;&)*/#"*4

Jonathan Portes, National Institute of Economic and Social Research , said:

“If you tell ministers that research will automatically be published on a xedtimetable with no scope for delays or interference, then (with honourableexceptions) they will commission much less of it or none. And in current budgetarycircumstances there will be no particular penalty for this. This tradeo simplycannot be resolved, except by making the commissioning of research independent…

it’s generally recognised internationally, and is the reason that the IMF and WorldBank, say, have strong, constitutionally independent evaluation o ces. The UKneeds the same, for each major department (there could be some grouping) withindependent heads, not from government, some protection of budgets (at leastthree or ve year budgets), the freedom to evaluate what they see t and publishthe results, and a direct reporting line to Commons departmental committees(à la NAO [National Audit O ce]). There is nothing particularly radical, complicated,or di cult about any of this, except that ministers wouldn’t like it.”

%"!+ There is some support for the principle of separating research from policymaking amongst membersof the research community, civil servants and politicians, but these proposals raise the question of who%(18, *+#" >(//2772(" *+# )#7#&)>+4 C1>+ & 7#;&)&*2(" /2.+* &87( 0)2". 2"*( :1#7*2(" *+# )#8&*2("7+2;

between policymakers who commission research and academics carrying it out and indeed it might2*7#83 +&-# *+# #33#>* (3 )#,1>2". ;(82*2>2&"76 2"*#)#7* 2" >(//2772("2". &", ;&'2". 3() )#7#&)>+4

%"!! T"(*+#) 71..#7*2(" /&,# *( *+27 2":12)' %&7 *+&* )#7#&)>+#)7 >(18, )#;()* ,2)#>*8' *( ;&)82&/#"*4It is understood that the former head of what was then the Home Of ce Research Unit concludedin the 1950s that this should be the norm, so as to preserve the independence of research. While bothof the suggestions above are worthy of consideration, they imply an unlikely major overhaul of a system*+&* >(18, /##* ;1082>&*2(" )#:12)#/#"*7 /1>+ /()# 72/;8'4

Appointin ad hoc committees

%"!#

Appointing ad hoc committees to oversee potentially dif cult research can be especially useful, because2* &88(%7 .(-#)"/#"* *( ;(2"* *( 2",#;#",#"* #-2,#"># &", *( 7#* (1* 2*7 ;(82>' )#7;("7# 7#;&)&*#8'4Here are some examples:

%"!$ Biofuels and food crops In April 2008, the government’s Renewable Transport Fuel Obligation committed the UK to ensuring

that biofuels make up 2.5% by volume of road transport fuel sales, increasing by 1.25% a yearto 5% by 2010/11. This was a result of an EU Directive in 2003 promoting the use of biofuels.

ULSECTION 4: REMEDIES AND RECOMMENDATIONS



%"!% Evidence had emerged that raised “concern about the role of biofuels in rising food prices, acceleratingdeforestation and doubts about the climate bene ts.” In February 2008, a paper by Searchinger et al inScience suggested that in the US, biofuels caused land use changes that led to increased net emissions(3 .)##"+(17# .&7#74 %# 5+27 ;)(/;*#, *+# >+2#3 7>2#"*27*7 (3 G#3)& &", G35 *( >(//2772(" & )#-2#%of the evidence behind the policy of increasing biofuel use and its indirect effects, and whether there7+(18, 0# & /()&*()21/4

%"!& The review was chaired by Professor Ed Gallagher, chair of the Renewable Fuels Agency(the independent agency created to implement the policy). It concluded that while there was no needfor a moratorium and that “there is a future for a sustainable biofuels industry,” the rate of increasein the use of biofuels should be reduced to avoid agricultural land being displaced.

%"!' Because the UK was committed to a rapid increase in the use of biofuels, and because the impacton food crops was controversial, the research and recommendations could have been awkwardfor the government. It appears that, because of a collaborative effort between researchers and civilservants to ensure that counter-evidence and uncertainty were properly addressed during the research,it was possible to publish the nal report within six months of the research being commissioned.

%"!( The effect of mobile phones on health Media stories and campaign groups claimed that mobile phones and transmitter masts were causing

cancer, despite a balance of scienti c evidence to the contrary. An independent expert group on mobilephones had concluded in 2000 that although mobile phones and transmitters are unlikely to causedisease, further research was required to address possible biological effects of low-level radiofrequencyradiation. The Mobile Telecommunications and Health Research (MTHR) programme was set upin 2001 to commission and oversee this research, jointly funded by government and the mobilephone industry.

%"!) The MTHR’s research spanned 11 years, and concluded that there was no association betweenmobile phone use and cancer. The MTHR’s committee was effective in commissioning controversialand complex research and communicating it to the public. The interposing of an independentcommittee helped government to publish reliable research in a framework that was recognised

by commentators as independent.

%"!* The Depleted Uranium Oversi ht Board The Ministry of Defence set up this independent board to oversee the development of a laboratory test

to check for exposure to depleted uranium (DU) amongst veterans of the Gulf War and other con ictsin the early 1990s. The board included established scientists, veterans’ groups and campaigners,to ensure that a broad range of views was considered. It concluded that there was no detectableexposure to DU. The research was published without delay, alongside a minority review that questioned*+# -&82,2*' (3 *+# *#7*4

Research commissioners and researchers workin to ether to set questions

and a ree on publication

%"#+ Or anisational preparedness for unconventional terrorist attacks Research into how prepared public and private organisations are to respond to chemical, biological,

radiological or nuclear terrorist incidents has been published promptly and in full, notwithstandingconcerns that the ndings or even the questions could be seen as identifying weaknesses in nationalsecurity. Researchers, civil servants and other stakeholders met to discuss the questions to be asked,research methods and ways to communicate ndings to the public. Such events appear to help buildtrust between academics and those commissioning research.

%# Searchinger, T et al (2008)

UU SECTION 4: REMEDIES AND RECOMMENDATIONS



Inquiry methods

The Rt Hon Sir Stephen Sedley, who is leading the inquiry, is a privy counsellor and former Lord Justice of Appeal.He has served as an ad hoc judge of the European Court of Human Rights and has been a visiting professorat the University of Oxford. He is a trustee of Sense about Science.

The report has been researched and co-authored by Dr Prateek Buch, policy associate and Dr Síle Lane,director of campaigns and policy at Sense about Science.

Initial consultations with members of the research community, civil servants, and politicians helped us to framethe inquiry’s call for evidence (see below), which we issued in November 2015. We invited written submissionsand interviewed interested parties to gather evidence.

We have bene ted greatly from the written submissions, oral contributions and advice from the people listedin Appendix II, but the content of this report is the responsibility of its authors.

5+# 2":12)' &", *+# ;)(,1>*2(" (3 *+27 )#;()* %#)# 71;;()*#, 0' & .)&"* 3)(/ *+# PQCC5 R+&)2*&08# 5)17*(Registered Charity No: 247498) and by Nick Ross.

Call for evidence Tuesday 24th November 2015

Sense about Science is seeking information and views about how government commissions and publishes researchfrom the academic community, professional bodies, the civil service, charities, the media and other interested parties.

Research conducted or commissioned by central government is an important part of guiding, developing,/(,23'2". &", /("2*()2". ;(82>'4 T8*+(1.+ ,#;&)*/#"*&8 .12,#82"#7 )#:12)# ;)(/;* &", >(/;8#*# ;1082>&*2("of such research, there have been repeated allegations in recent years of publication being held back.

5+27 +&7 )&27#, ;1082> >(">#)"7 3() & "1/0#) (3 )#&7("7?

N Failure to make publicly-funded research available to the public.

N A lack of transparency on the basis of government decisions and the role of evidence in reaching them.

N Potential effects on the willingness of researchers to assist in policymaking.

It is not known how signi cant or widespread the delayed publication or withholding of government researchis, and there is little comparative evidence of how different departments or agencies behave. There will alsobe examples of good practice, for instance where potentially awkward research has been published promptly.

Sense about Science is conducting an inquiry, to be led by one of its trustees, the Rt Hon Sir Stephen Sedley,into the scale and sources of the problem, and into possible remedies. An initial scoping exercise was conductedbetween 18th September and 30th October 2015 to establish this call for evidence.

The inquiry will consider the publication of research, including analyses of of cial statistics, initiated by Whitehall,#;&)*/#"*7 &", *+# &)/67 8#".*+ 0(,2#7 *+&* )#;()* *( *+#/4 <# &87( %#8>(/# 710/2772("7 )#8&*2". *( ,#-(8-#,

administrations, but not to local authorities. The inquiry will consider what has happened in established instancesof research being held back, and will suggest how controversial or inconclusive research can be handled.

We invite submissions about your experience of commissioning, conducting, publishing or accessing governmentresearch, particularly in the following contexts:

! Contracts and rules: do research contracts make publication principles clear? Are these principles ethicallyand legally appropriate? Who is responsible within departments for ensuring that research is publishedpromptly and fully?

# Expectations of publication: these vary, especially as regards the timing of the announcement of policyand in relation to controversial topics. What justi cations may there be for delaying or withholding publicationof government-commissioned research?

$ Potential improvements: what changes would improve the way government commissions, conductsand publishes research? How can good practice be spread?

40APPENDIX I: INQUIRY METHODS AND CALL FOR EVIDENCE



We have had the bene t of written submissions, oral discussions and advice from the followingpeople. With the authors’ permission, written submissions are published in full at ---./0102/34!567!/8.9/:;17<=!11!951

Richard Bartholomew

Professor Sir John Beddin ton

Professor Vir inia Berrid e

Nicola Blackwood MP

Sir Iain Chalmers

Professor David Co on

Tim Colbourne

Professor Brian S Collins

Professor Dame Sally Davies

Fiona Fox

Dr Emma Gordon

Dr Michael Hallsworth

Deborah Har reaves

Rod Harrison

Professor Marianne Hester

Former joint head of government social research

Professor of applied population biology, Imperial College London;senior adviser, Oxford Martin School, University of Oxford;former government chief scienti c adviser

Profes sor of history and director, Centre for History in Public Health,London School of Hygiene and Tropical Medicine (written submission)

Chair, House of Commons science and technology select committee

Coordinator, James Lind Initiative ; founding member of CochraneR(88&0()&*2(" &", R&/;0#88 R(88&0()&*2("

Professor of occupational and environmental medicine,D"2-#)72*' (3 C(1*+&/;*("

Director of policy, Open Reason; former special adviserand deputy chief of staff to deputy prime minister Nick Clegg MP

Professor of engineering policy and director, International Centrefor Infrastructure Futures, UCL; former chief scienti c adviser,Department of Business Innovation and Skills and Department for Transport

Chief medical of cer for England, Department of Health; emeritus professor,=/;#)2&8 R(88#.# H(",("

Chief executive, Science Media Centre (written submission)

Head of government economic and social research, Treasury

Director, health & tax, Behavioural Insights Team

Director, High Pay Centre; chaired High Pay Commission

Senior principal research of cer, Scottish government

Profes sor of gender, violence & international policy, University of Bristol

Rt Hon Patricia Hewitt Former MP and secretary of state for health

Ed Humpherson Head of assessment, UK Statistics Authority

Dr Julian Huppert Former MP

Dr Eric Jensen Associate professor, University of Warwick

Professor Lord Krebs Emeritus professor of zoology and former principal,Jesus College, University of Oxford

Dr Daniel Lambauer Director for value for money development, National Audit Of ce

Rt Hon David Laws Former MP, minister of state for schools, minister of state for Cabinet Of ce

and chief secretary to the Treasury; executive chairman, CentreForum

WLAPPENDIX II: LIST OF CONTRIBUTORS



Chief scienti c adviser, Department for Education

Senior journalist, Pulse

Head of international, Academy of Medical Sciences

Department for Environment, Food and Rural Affairs

Chief researcher and head of Education Analytical Services Division,C>(**27+ .(-#)"/#"*

Chief social research of cer, Welsh government

Lecturer, University of Dundee (written submission)

Former MP and chair of the House of Commons science&", *#>+"(8(.' 7#8#>* >(//2**##

Social scientist and research programme manager,Department for Environment, Food and Rural Affairs

G#;&)*/#"*&8 8#>*1)#) 2" #-2,#">#90&7#, 7(>2&8 2"*#)-#"*2(" &", ;(82>'evaluation, University of Oxford

Head of policy, Wellcome Trust

Head of research, Department for Education

Principal research fellow (former director), National Institute of Economicand Social Research; former chief economist, Cabinet Of ce & Number 10G(%"2". C*)##* X%)2**#" 710/2772("\

Editor,Pulse

Head of public affairs, Royal Society

Y)#72,#"* (3 *+# Q('&8 C(>2#*' X%)2**#" 710/2772("\

Reader in risk and terror, Kings College London

Senior policy analyst, Transform Drug Policy Foundation

Programme director, Institute for Government

Chief scientist and head of research and evaluation, BehaviouralInsights Team; post-doctoral research fellow, Harvard Kennedy Schoolof Government; research associate, Centre for Market and PublicOrganisation, University of Bristol

Principal of Jesus College, Oxford; professor of computer science, University(3 !B3(),S >+&2)/&" (3 *+# !;#" G&*& ="7*2*1*#

Co-founder, Scienceogram; post-doctoral researcher, Francis Crick Institute

Director, RAND Europe

Dr Tim Leuni

Soa Lind

Catherine Luckin

Simon Luker

Dr A udrey MacDou all

Dr Steven Marshall

Dr Jonathan Mendel

Andrew Miller

Rachel Muckle

Dr Kathryn Oliver

Nicola Perrin

Vicky Petrie

Jonathan Portes

Ni el Praities

Becky Purvis

Sir Venki Ramakrishnan

Dr Brooke Ro ers

Steve Rolles

Jill Rutter

Dr Michael Sanders

Professor Sir Ni el Shadbolt

Dr Andrew Steele

Dr Christian van Stolk

Professor Ro er Tarlin University of Surrey; former head of the Home Of ce Researchand Planning Unit

WU APPENDIX II: LIST OF CONTRIBUTORS



Dr Suzanne Taylor Research fellow, London School of Hygiene and Tropical MedicineX%)2**#" 710/2772("\

Jane Tinkler Head of social science, Parliamentary Of ce of Science and Technology

Dr Arnaud Va anay Research associate, London School of Economics and Political Science

David Walker Former head of policy, Academy of Social Sciences (written submission)

Professor Sir Mark Walport Government chief scienti c adviser, Government Of ce for Science(written submission); honorary distinguished professor of medicine,=/;#)2&8 R(88#.# H(",("

Professor Jeremy Watson Professor of engineering systems and vice-dean, Faculty of EngineeringSciences, UCL; chief scientist and engineer, Building ResearchEstablishment; former chief scienti c adviser, Department for Communitiesand Local Government

Naomi Weir Assistant director, Campaign for Science and Engineering

Professor James Wilsdon Professor of research policy, Department of Politics and director of impactand engagement, Faculty of Social Sciences, University of Shef eld

Dr Sarah Wollaston MP R+&2) (3 *+# J(17# (3 R(//("7 +#&8*+ 7#8#>* >(//2**##

WWAPPENDIX II: LIST OF CONTRIBUTORS



Alliance for Useful Evidence (2016). Using evidence:What works? http://www.alliance4usefulevidence.org/assets/Alliance-Policy-Using-evidence-v4.pdf(Accessed 17th May 2016).

BBC News (2013). Horsemeat scandal: Withdrawnproducts and test results. BBC News, [online]22nd March. http: //www.bbc.co.uk/news/world-21412590 (Accessed 17th May 2016).

Brennan, A et al (2014). Potential bene ts of minimumunit pricing for alcohol versus a ban on below costselling in England 2014: modelling study. BMJ,349:g5452. http://www.bmj.com/content/349/bmj.g5452.full.pdf+html (Accessed 18th May 2016).

Butler, P (2013). Food banks: where is the ‘suppressed’government report on UK food poverty? The Guardian ,Patrick Butler’s cuts blog, [blog] 12th November.http://www.theguardian.com/society/patrick-butler-cuts-

blog/2013/nov/12/food-banks-government-suppressed-defra-report-on-food-aid (Accessed 17th May 2016).

Civil Service Learning (no date). Government SocialResearch Competency Framework. https://civilservicelearning.civilservice.gov.uk/sites/default/ les/gsr_professional_competency_framework_-_ nal.docx

Collins, B (2013). Engineering policy: evidence, adviceand execution. In: Doubleday and Wilsdon (eds.). Futuredirections for scienti c advice in Whitehall.http://www.csap.cam.ac.uk/media/uploads/ les/1/fdsaw.pdf (Accessed 17th May 2016).

Cook, C (2014). Immigration impact report withheldby Downing Street. BBC News, [online] 5th March.http://www.bbc.co.uk/news/uk-politics-26435000(Accessed 17th May 2016).

Department for Business, Innovation & Skills (2013).Science, engineering and technology statistics 2013. https://www.gov.uk/government/statistics/science-engineering-and-technology-statistics-2013(Accessed 18th May 2016).

Department for Education (2014). Teachers’ workloaddiary survey 2013: Research report. https://www.gov.uk/government/uploads/system/uploads/attachment_data/

le/285941/DFE-RR316.pdf (Accessed 17th May 2016).

Department for Environment, Food and Rural Affairs(2014a). Shale Gas Rural Economy Impacts.[Draft report] March. https://www.gov.uk/government/uploads/system/uploads/attachment_data/ le/440791/,)&3*97+&8#9.&79)1)&89#>("(/'92/;&>*9)#;()*4;,3(Accessed 20th May 2016).

Department for Environment, Food and Rural Affairs(2014b). Request for information: correspondencecovering the economics of shale gas. [Letter] 28th

July. https://www.gov.uk/government/uploads/system/

uploads/attachment_data/ le/337645/RFI6751_Economics_of_Shale_Gas_Redacted.pdf(Accessed 17th May 2016).

Department for Environment, Food and Rural Affairs(2015). Draft Shale Gas Rural Economy Impacts paper.https://www.gov.uk/government/uploads/system/uploads/attachment_data/ le/440791/draft-shale-gas-)1)&89#>("(/'92/;&>*9)#;()*4;,3(Accessed 26th May 2016).

Department for Environment, Food and Rural Affairsand Food Standards Agency (2014). Elliott Review intothe integrity and assurance of food supply networks:

nal report. https://www.gov.uk/government/uploads/system/uploads/attachment_data/ le/350726/elliot-review- nal-report-july2014.pdf(Accessed 17th May 2016).

Department of Health (2012). Pilot scheme to allow

patients to choose convenient GP practice. [News story,online] 4th April. https://www.gov.uk/government/news/;28(*97>+#/#9*(9&88(%9;&*2#"*79*(9>+((7#9>("-#"2#"*9gp-practice (Accessed 17th May 2016).

Depleted Uranium Oversight Board (2007). Final reportof the Depleted Uranium Oversight Board. http://www.bandepleteduranium.org/en/docs/27.pdf(Accessed 17th May 2016).

Devlin, C et al (2014). Impacts of migration on UK nativeemployment: an analytical review of the evidence.Home Of ce and Department of Business, Innovationand Skills. https://www.gov.uk/government/uploads/system/uploads/attachment_data/ le/287287/occ109.pdf (Accessed 18th May 2016).

Dugan, E (2014). Government accused of suppressingthe damning report that suggests its agship welfarereforms are forcing ever more people to resort to foodbanks. The Independent , [online] 20th February.http://www.independent.co.uk/news/uk/politics/.(-#)"/#"*9&>>17#,9(3971;;)#772".9*+#9,&/"2".9report-that-suggests-its- agship-welfare-reforms-are-9142310.html (Accessed 17th May 2016).

Engineering and Physical Sciences Research Council

(no date). Sandpits. [Online]. https://www.epsrc.ac.uk/funding/howtoapply/routes/network/ideas/whatisasandpit/ (Accessed 17th May 2016).

Fox, F (2014). A battle too far? Science Media Centre,Fiona Fox’s blog, [blog] 22nd September.http://www.sciencemediacentre.org/a-battle-too-far/(Accessed 17th May 2016).

Fox, F (2015). Fiona Fox: ‘Time to end the cultureof silence in government science’. ResearchProfessional , [online] 29th April. http://www.researchresearch.com/news/article/?articleId=1351698(Accessed 17th May 2016).

W[APPENDIX IV: BIBLIOGRAPHY



Gallagher, E (2008). The Gallagher Review of theindirect effects of biofuels production. 5+# Q#"#%&08#Fuels Agency. http://webarchive.nationalarchives.gov.uk/20110407094507/http://renewablefuelsagency.gov.uk/sites/renewablefuelsagency.gov.uk/ les/_documents/Report_of_the_Gallagher_review.pdf

(Accessed 17th May 2016).Government Economic & Social Research Team (2015).Publishing research and analysis in government: GSRPublication Protocol. https://www.gov.uk/government/uploads/system/uploads/attachment_data/ le/431367/GSR_publication_protocol_2015_FINAL.pdf(Accessed 17th May 2016).

Government Of ce for Science (2010a).Principles of scienti c advice to government.https://www.gov.uk/government/publications/scienti c-advice-to-government-principles/principles-of-scienti c-advice-to-government (Accessed 20th May 2016).

Government Of ce for Science (2010b). TheGovernment Chief Scienti c Adviser’s Guidelines onthe Use of Scienti c and Engineering Advice in PolicyMaking. https://www.gov.uk/government/uploads/system/uploads/attachment_data/ le/293037/10-669-gcsa-guidelines-scienti c-engineering-advice-policy-making.pdf (Accessed 20th May 2016).

Government Of ce for Science (2015). Chief Scienti cAdvisers and their of cials: an introduction.https://www.gov.uk/government/uploads/system/uploads/attachment_data/ le/426307/15-2-chief-

scienti c-advisers-and-of cials-introduction.pdf(Accessed 20th May 2016).

Haynes, L et al (2012). Test, learn, adapt: Developingpublic policy with randomised controlled trials.Cabinet Of ce. https://www.gov.uk/government/uploads/system/uploads/attachment_data/ le/62529/TLA-1906126.pdf (Accessed 17th May 2016).

Hester, M et al (2007). Forced marriage: the riskfactors and the effect of raising the minimum age fora sponsor, and of leave to enter the UK as a spouse or

ancé(e). http://www.bristol.ac.uk/media-library/sites/sps/migrated/documents/rk6612 nalreport.pdf(Accessed 18th May 2016).

HM Government (2010). Drug strategy 2010 ReducingDemand, Restricting Supply, Building Recovery:Supporting People to Live a Drug Free Life. https://www.gov.uk/government/uploads/system/uploads/attachment_data/ le/98026/drug-strategy-2010.pdf (Accessed 17th May 2016).

HM Government (2012). The Government’s AlcoholStrategy. https://www.gov.uk/government/uploads/system/uploads/attachment_data/ le/224075/alcohol-strategy.pdf (Accessed 17th May 2016).

Holmes, J (2014). Ban on below cost alcohol saleswould be 40 times less effective than minimum pricing.The Conversation , [online] 8th January.https://theconversation.com/ban-on-below-cost-alcohol-sales-would-be-40-times-less-effective-than-minimum-pricing-21865 (Accessed 18th May 2016).

Home Of ce (2012). Alcohol strategy consultation.https://www.gov.uk/government/consultations/alcohol-strategy-consultation (Accessed 17th May 2016).

Home Of ce (2013). Next steps following theconsultation on delivering the Government’s alcoholstrategy. https://www.gov.uk/government/uploads/system/uploads/attachment_data/ le/223773/Alcohol_consultation_response_report_v3.pdf(Accessed 17th May 2016).

Home Of ce (2014). Drugs: International comparators.https://www.gov.uk/government/uploads/

system/uploads/attachment_data/ le/368489/G)1.7="*#)"&*2("&8R(/;&)&*()74;,3(Accessed 17th May 2016).

House of Commons health select committee (2015a).Correspondence on childhood obesity strategy. http://www.parliament.uk/business/committees/committees-a-z/commons-select/health-committee/publications/?type=37&session=27&sort=false&inquiry=2370(Accessed 18th May 2016).

House of Commons health select committee (2015b).Childhood obesity-brave and bold action.http://www.publications.parliament.uk/pa/cm201516/cmselect/cmhealth/465/465.pdf(Accessed 18th May 2016).

House of Commons home affairs select committee(2008). Domestic Violence, Forced Marriage and‘Honour’-based violence. http://www.publications.parliament.uk/pa/cm200708/cmselect/cmhaff/263/263i.pdf and http://www.publications.parliament.uk/pa/cm200708/cmselect/cmhaff/263/263ii.pdf(Accessed 18th May 2016).

Information Commissioner’s Of ce (2015). EnvironmentalInformation Regulations 2004 Decision notice (reference

FER0562043). https://ico.org.uk/media/action-weve-taken/decision-notices/2015/1431897/fer_0562043.pdf(Accessed 26th May 2016)

International Committee of Medical Journal Editors(2004). Clinical Trial Registration: A Statement fromthe International Committee of Medical Journal Editors. [Editorial, online]. http://www.icmje.org/news-and-editorials/clin_trial_sep2004.pdf(Accessed 18th May 2016).

Kenny, C (2015). The impact of academia on Parliament:45 percent of Parliament-focused impact case

W] APPENDIX IV: BIBLIOGRAPHY



studies were from social sciences. 5+# H(",("School of Economics and Political Science ImpactBlog, [blog] 19th October. http://blogs.lse.ac.uk/impactofsocialsciences/2015/10/19/the-impact-of-uk-academia-on-parliament/ (Accessed 17th May 2016).

King’s College London and Digital Science (2015).The nature, scale and bene ciaries of research impact:An initial analysis of Research Excellence Framework (REF) 2014 impact case studies. Higher EducationFunding Council for England. http://www.kcl.ac.uk/sspp/policy-institute/publications/Analysis-of-REF-impact.pdf(Accessed 17th May 2016).

Lambie-Mumford, H et al (2014). Household FoodSecurity in the UK: A Review of Food Aid. https://www.gov.uk/government/uploads/system/uploads/attachment_data/ le/283071/household-food-security-uk-140219.pdf (Accessed 17th May 2016).

Lawrence, F (2014). Row as horsemeat le shelved.The Guardian , [online] 15th August. http://www.theguardian.com/uk-news/2014/aug/15/of cial-report-+()7#/#&*97>&",&89,#8&'#,9"#%93((,97&3#*'93#&)7(Accessed 17th May 2016).

Lean, G (2014). Want to know how fracking will affectyou? Sorry, that’s a state secret! The Daily Telegraph ,[online] 11th August. No longer available.

Lind, S (2013). Decision on extending practice boundarypilots to be made without ‘robust’ economic data.Pulse , [online] 24th September. http://www.pulsetoday.co.uk/your-practice/practice-topics/practice-income/,#>272("9("9#B*#",2".9;)&>*2>#90(1",&)'9;28(*79*(90#9made-without-robust-economic-data/20004351.article(Accessed 18th May 2016).

Loopstra, R et al (2015). Austerity, sanctions, and therise of food banks in the UK. BMJ, 350:h1775.http://www.bmj.com/content/350/bmj.h1775(Accessed 17th May 2016).

Lord Rose (2015). Better leadership for tomorrow:NHS leadership review. https://www.gov.uk/government/uploads/system/uploads/attachment_data/

le/445738/Lord_Rose_NHS_Report_acc.pdf

(Accessed 17th May 2016).Mason, R (2014). Fracking campaigners criticise‘censored’ report on house prices. The Guardian ,[online] 10th August. http://www.theguardian.com/environment/2014/aug/10/fracking-censored-house-price-repor t (Accessed 17th May 2016).

May, T (2012). An immigration system that worksin the national interest. [Speech] 12th December.https://www.gov.uk/government/speeches/home-7#>)#*&)'97;##>+9("9&"92//2.)&*2("97'7*#/9*+&*9works-in-the-national-interest (Accessed 17th May 2016).

Mendel, J (submitted). Unpublished policy trials, therisk of false positives and the persistence of authority-based policy. Evidence & Policy.

Meng, Y et al (2013). Modelled income group-speci cimpacts of alcohol minimum unit pricing in England2014/15: Policy appraisals using new developmentsto the Shef eld Alcohol Policy Model X-L4M\4The University of Shef eld. http://www.shef.ac.uk/polopoly_fs/1.291621!/ le/julyreport.pdf(Accessed 17th May 2016).

Migration Advisory Committee (2012). Analysis of theImpacts of Migration. https://www.gov.uk/government/uploads/system/uploads/attachment_data/ le/257235/analysis-of-the-impacts.pdf (Accessed 17th May 2016).

Mobile Telecommunications and Health ResearchProgramme management committee (2007).Mobile Telecommunications and Health Research

Programme: Report 2007. J#&8*+ Y)(*#>*2(" T.#">'4http://www.mthr.org.uk/documents/MTHR_report_2007.pdf (Accessed 17th May 2016).

Morris, N (2013). Demand for food banks has nothingto do with bene ts squeeze, says Work minister LordFreud. The Independent , [online] 2nd July. http://www.independent.co.uk/news/uk/politics/demand-for-food-banks-has-nothing-to-do-with-bene ts-squeeze-says-work-minister-lord-freud-8684005.html(Accessed 17th May 2016).

Morris, N (2014). Conservatives suppressed vitalreports on drug abuse, says Home Of ce minister.The Independent , [online] 3rd October.http://www.independent.co.uk/news/uk/politics/tories-71;;)#77#,9-2*&89)#;()*79("9,)1.9&017#97&'79+(/#9of ce-minister-9773938.html (Accessed 17th May 2016).

Oakeshott, I (2013). Ministers hide surge in use of foodbanks. The Sunday Times , [online] 3rd November.http://www.thesundaytimes.co.uk/sto/news/Politics/article1335533.ece (Accessed 17th May 2016).

Oliver, K et al (2014). A systematic reviewof barriers to and facilitators of the use of evidenceby policymakers. BMC Health Services Research ,

14(2). http://bmchealthservres.biomedcentral.com/articles/10.1186/1472-6963-14-2(Accessed 17th May 2016).

Page, E (2013). There is suf cient evidenceto suggest Whitehall is leaning on researchersto produce politically useful research. 5+# H(",("School of Economics and Political Science ImpactBlog, [blog] 31st October. http://blogs.lse.ac.uk/impactofsocialsciences/2013/10/31/governments-lean-on-researchers/ (Accessed 17th May 2016).

50APPENDIX IV: BIBLIOGRAPHY



Portes, J (2012). British jobs and foreign workers:today’s reports on immigration and unemployment.National Institute of Economic and Social Research blog,[blog] 10th January. http://www.niesr.ac.uk/blog/british- jobs-and-foreign-workers-todays-reports-immigration-and-unemployment#.VzsVqt8rLeT (Accessed 17th May

2016).Portes, J (2015). Let statisticians cry foul whenpoliticians bend the truth. Financial Times , [online] 25thJanuary. http://www.ft.com/cms/s/0/dcf46a5a-a335-11e4-9c06-00144feab7de.html#axzz41rJQGucX(Accessed 17th May 2016).

Price, C (2014). One-third of practice boundary pilotsurgeries failed to register a single patient. Pulse ,[online] 20th March. http://www.pulsetoday.co.uk/political/practice-boundaries/one-third-of-practice-0(1",&)'9;28(*971).#)2#793&28#,9*(9)#.27*#)9&972".8#9patient/20006157.article#.UzKs9 _t8E(Accessed 17th May 2016).

Public Health England (2014). Sugar reduction:responding to the challenge. https://www.gov.uk/government/uploads/system/uploads/attachment_data/

le/324043/Sugar_Reduction_Responding_to_the_Challenge_26_June.pdf (Accessed 26th May 2016).

Public Health England (2015). Sugar reduction:the evidence for action. https://www.gov.uk/government/uploads/system/uploads/attachment_data/

le/470179/Sugar_reduction_The_evidence_for_action.pdf (Accessed 18th May 2016).

Research Excellence Framework (REF) 2014 impactcase studies. [Database] Digital Science. http://impact.ref.ac.uk/casestudies (Accessed 17th May 2016).

The Royal Society and the Royal Academy ofEngineering (2012). Shale gas extraction in the UK:a review of hydraulic fracturing. https://royalsociety.org/~/media/policy/projects/shale-gas-extraction/2012-06-28-shale-gas.pdf (Accessed 17th May 2016).

The Royal Statistical Society (2016). Putting evidenceat the heart of policy debate. http://www.rss.org.uk/Images/PDF/in uencing-change/2016/data-manifesto1-

;1**2".9#-2,#">#9&*9*+#9+#&)*9(39*+#9;(82>'9,#0&*#4;,3(Accessed 17th May 2016).

Rutter, J (2012). Evidence and evaluation in policymaking: a problem of supply or demand? ="7*2*1*#for Government. ht tp://www.instituteforgovernment.org.uk/sites/default/ les/publications/evidence%20and%20evaluation%20in%20template_ nal_0.pdf(Accessed 17th May 2016).

Rutter, J and Gold, J (2015). Show your workings:assessing how government uses evidenceto make policy. ="7*2*1*# 3() O(-#)"/#"*4

http://www.instituteforgovernment.org.uk/publications/show-your-workings (Accessed 17th May 2016).

Searchinger, T et al (2008). Use of U.S. Croplandsfor Biofuels Increases Greenhouse Gases ThroughEmissions from Land-Use Change. Science , 319(5867),pp1238-1240. http://science.sciencemag.org/content/319/5867/1238 (Accessed 20th May 2016).

Sense about Science (2006). I don’t know whatto believe: making sense of science stories. http://www.senseaboutscience.org/data/ les/resources/16/IDontKnowWhatToBelieve_web2011.pdf(Accessed 17th May 2016).

Sense about Science (2012). Peer review: the nutsand bolts. http://www.senseaboutscience.org/data/

les/resources/99/Peer-review_The-nuts-and-bolts.pdf(Accessed 17th May 2016).

Sense about Science (2013). Making senseof uncertainty. http://www.senseaboutscience.org/data/

les/resources/127/SAS012_MSU_reprint_compressed.pdf (Accessed 17th May 2016).

Sense about Science (2014). “Fracking risk comparedto thalidomide and asbestos in Walport report”? Sense about Science, For The Record, [online]29th November (updated 16th December).http://www.senseaboutscience.org/resources.php/170/quotfracking-risk-compared-to-thalidomide-and-asbestos-in-walport-reportquot (Accessed 17th May 2016).

Spencer, B (2014). The fracking cover-up: Defra censors

key report 63 times in 13 pages in move describedas ‘comical’ by campaigners. Mail Online, [online] 11thAugust (updated 12th August). http://www.dailymail.co.uk/news/article-2722407/The-fracking-cover-Defra-censors-key-report-63-times-13-pages-described-comical-campaigners.html (Accessed 17th May 2016).

The supreme court (2011). ! #$% &'( )**+,-)&,$% $. /0,+))%1 )%$&'(23 #453 #!(6*$%1(%&63 7 8(-2(&)29 $. 8&)&(for the Home Department (Appellant) [2011] UKSC 45.https://www.supremecourt.uk/cases/uksc-2011-0022.html (Accessed 18th May 2016).

5arling, R (2011). Relations between GovernmentResearchers and Academics. The Howard Journalof Criminal Justice , 50(3), pp307-313. http://onlinelibrary.wiley.com/doi/10.1111/j.1468-2311.2011.00666.x/abstract(Accessed 17th May 2016).

Transform Drug Policy Foundation (2010). Epic threeyear FOI battle to release suppressed Home Of ce drugstrategy evaluation reaches its denouement. 5)&"73()/Drug Policy Foundation, Transform Media Blog, [blog] 20thJanuary (last updated 26th June). http://transform-drugs.blogspot.co.uk/2010/01/epic-three-year-foi-quest-to-release.html (Accessed 17th May 2016).

51 APPENDIX IV: BIBLIOGRAPHY



C#"7# &0(1* C>2#"># 27 &" 2",#;#",#"* >&/;&2."2". >+&)2*' *+&*challenges the misrepresentation of science and scienti c evidencein public life. We advocate openness and honesty about research

ndings, and work to ensure the public interest in sound scienceand evidence is represented and recognised in public discussionand policymaking.

Sense about Science is a small team working with thousandsof supporters, from world-leading researchers to community groups.

This report was published in 2016 by Sense about Science,which has nal responsibility for the content.

Designed by Francesca Tortora

For more copies or for further information, please contact:

>0510 2<97? >3!053014a Clerkenwell GreenLondon EC1R 0DP+44 (0)20 7490 9590

%%%47#"7#&0(1*7>2#">#4().#":12)2#7_7#"7#&0(1*7>2#">#4().

Registered charity No. 1146170Company No. 6771027

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License



>0510 2<97? >3!053014a Clerkenwell GreenLondon EC1R 0DP+44 (0)20 7490 9590

The inquiry and the production of this report were supported by a grant fromth JRSST Ch it bl T t (R gi t d Ch it N 247498) d b Ni k R

missingevidence digitalpdf singlepages (2)

Documents