This may be the author’s version of a work that was submitted/acceptedfor publication in the following source:

Bedford, Laura & Neyroud, Peter(2018)Organisational learning from field research in policing: How police canimprove policy and practice by implementing randomised controlled trials.European Journal of Policing Studies (EJPS), 5(4), pp. 28-49.

This file was downloaded from: https://eprints.qut.edu.au/120149/

c© Maklu BV 2018

This work is covered by copyright. Unless the document is being made available under aCreative Commons Licence, you must assume that re-use is limited to personal use andthat permission from the copyright owner must be obtained for all other uses. If the docu-ment is available under a Creative Commons License (or other specified license) then referto the Licence for details of permitted re-use. It is a condition of access that users recog-nise and abide by the legal requirements associated with these rights. If you believe thatthis work infringes copyright please provide details by email to qut.copyright@qut.edu.au

Notice: Please note that this document may not be the Version of Record(i.e. published version) of the work. Author manuscript versions (as Sub-mitted for peer review or as Accepted for publication after peer review) canbe identified by an absence of publisher branding and/or typeset appear-ance. If there is any doubt, please refer to the published source.

http:// www.maklu-online.eu/ en/ tijdschrift/ ejps/ volume-5/special-issue-changes-policing-improve-service-del/

1 

Bedford, L.1, and Neyroud, P.2 Organisational learning from field research in policing: how police can improve policy and practice by implementing randomised controlled trials Abstract Organisational Learning (OL) perspectives suggest that all organisations use evidence to adapt and change their policies and practices. The special case of how police organisations adapt and change in response to experiences with the implementation of Randomised Controlled Trials (RCTs) is, however, not well understood in the extant literature. The value of policing RCTs may, on the one hand, lie in their ability to provide hard evidence or results of ‘what works’ in policing. On the other hand, RCTs may be powerful change processes which serve to generate OL and potential service improvements even in the absence of results. This paper presents an analysis of the results of in-depth interviews conducted in a policing organisation that has implemented a RCT. Using an integrated perspective of OL, we show that processes related to the experience with implementing a RCT may have a specific potential to leverage organisational change. Keywords: Policing; Randomised Controlled Field Trial; Organisational Learning; Police Reform; Evidence Based Policing 1LauraBedford,SchoolofJustice,FacultyofLaw,QueenslandUniversityofTechnology.2PeterNeyroud,Institute of Criminology, University of Cambridge.

2 

INTRODUCTION Many routes to police reform and service improvement have been proposed and most have proved difficult to deliver (Skogan, 2008). We will argue in this paper that one route – the process of rigorously testing police practices and innovations – may act to catalyse organisational learning and improvements in policing. The debates on testing police practice, particularly by conducting randomised controlled trials (RCTs), has become a ‘methods war’ which centres on arguments as to whether RCTs are an effective way of evaluating police practice (Neyroud, 2017). These arguments stem, in part, from the evident difficulty of translating RCT research results in policing. We suggest that advocates of an evidence based approach have insufficiently countered these critiques by highlighting the way the process of testing can contribute to wider improvements. We argue that better understanding and maximising the process of OL associated with the implementation of RCTs increases their value and potential to improve policing (Bedford, 2017; Neyroud, 2017). This is a timely moment to consider the role of RCTs in policing, because, as Neyroud has shown (Neyroud, 2016), the number of RCTs being conducted in policing has risen sharply in recent years (Figure 1). Moreover, there has been a significant shift from externally managed academic research to practitioner-led experimental research (Neyroud, 2017). Weisburd and Neyroud (2011) propose that such police ownership of the scientific process could be an important agent for the transformation of policing. By contrast, Sparrow (2011) has cast doubt on experimental testing as a research strategy. He concludes that Weisburd and Neyroud’s proposal ‘offers virtually no benefits for the police’. Further, he argues that the ‘best that they can hope for is that the scientists they have invited in, after months or years of work, will finally confirm what police thought they knew already’ (Sparrow, 2011, p. 26). [Figure 1 near here]. Sparrow is, however, basing his arguments largely on an ‘outsider’ model of research where academics undertake research and deliver research results to police as knowledge products. In contrast, the recent expansion of experimental research in policing has led to the creation or trialling of several key research stations (Sherman, 2010) – the Queensland Police Service and Western Australia Police in Australia and West Midlands Police in the UK – where batches of experiments are being run by frontline staff supported by embedded criminologists, not by ‘drop in’ outsiders. The lessons of these experiments – the challenges, processes and results – are being shared not just in academic journals (another of Sparrow’s criticisms) but through growing networks of police like the Society for Evidence Based Policing, which is now established in the UK, Australasia, USA and Canada. These developments suggest that Shepherd’s (2003) vision of police practice and police research combining to improve policing through continuous field testing is both capable of being realised and capable of offering real benefits to frontline service. The increased attention to understanding how RCTs can be most effectively implemented (Sherman, 2010; Neyroud, 2017), alongside efforts to understand how best to translate the lessons from RCTs into practice in the field (Lum, Telep, Koper & Grieco, 2012) to improve service delivery, is vitally important to the evidence based policing agenda. However, very little attention has, so far, been focused on the ways in which RCTs can impact wider organisational learning in policing (Bedford and Mazerolle 2014). The value of policing RCTs in organisational learning (OL) may, on the one hand, lie in their ability to provide hard evidence or results of ‘what works’ in policing. On the other hand, RCTs may also be

3 

powerful change processes, which serve to generate OL and improvements aside from results (Bedford, 2017). Thismayimproveservicedeliverybyincreasingthecapacityofpoliceorganisationstoembracechangeandbecomemoreopentolearningandreflecting. In particular, the process of testing a practice systematically in a controlled experiment can provide opportunities for the organisation to learn how to deliver an existing or adapted practice more effectively. In the case study explored in this article, amongst other innovations, the force developed a new ‘prescribing tool’ to enable more consistent delivery of tailored treatment conditions to offenders in pre-court disposals managed by the police (Slothower, 2014a), a new approach to engaging the victims in the setting of conditions (Slothower, 2014b) and an improved approach to programme implementation (Slothower, Sherman & Neyroud, 2015). Our research data comprises a series of transcripts of face-to face interviews with staff involved in one RCT in West Midlands Police (WMP) area in the United Kingdom. We investigate the perceptions of police regarding the role of the RCT in generating OL through the entire process of knowledge acquisition, dissemination, interpretation and organisational change (Huber 1991). To begin, in the Literature Review section below, we provide an overview of key theoretical constructs central to the OL literature explored by Bedford (2017) in her PhD Thesis. We then explore the evidence from the interviews in light of these key constructs. We conclude with a discussion of the potential role of the RCTs in organisational change and service improvement. LITERATURE REVIEW The origin of the notion that organisations ‘learn’ is usually attributed to Cyert and March (1963). They highlighted the role of OL as an adaptive biological type of response in defense against external shocks, challenges and changes. The emphasis is on organisational problem solving and search where problems are identified as the failure to achieve organisational goals. In ‘problemistic search’ (Huber 1991), so long as their goals are not met through their current activities, organisations will scan and search for knowledge and solutions in their external environments. This search process is purposeful and triggered by negative performance feedback (Cyert & March 1963). Cyert and March (1963) differentiate between adaptive response to ‘short-run feedback from the environment according to some fairly well-defined rules’ and, ‘organisational adaptation which results in changes to rules themselves’ (Cyert & March, 1963, pp. 101-102). This is considered one of the origins of the theory of single-loop and double-loop learning, later developed by Argyris and Schőn (1978) who argued that negative feedback loops are built into self-regulating systems (organisations) to provide stability and allow them to generate and recognize negative feedback (errors). Single-loop learning is like the thermostat that detects if the room it is too hot or cold and adjusts the temperature to a programmed setting. This adjustment will be repeated so long as the feedback loop suggests that it is performing correctly (Argyris 1994). Here, OL is based on a ‘what works’ trial and error learning where results which are considered to be effective are repeated and those considered ineffective are discontinued (Argyris, 2004; Cyert & March, 1963). Single-loop organisational learning occurs whenever ‘error is detected and corrected without questioning or altering the underlying values of the system … [and] … the

4 

error detected and corrected permits the organisation to carry on its current policies or achieve its current objectives’ (Argyris 1994, p. 8). When the organisation’s measures of success, and means to achieve success, are questioned, adaptive single-loop learning becomes insufficient to correct error. When this occurs, double-loop generative learning is required for the organisation to continue to meet its goals and involves ‘forging new paths’ (Nutley, et al. 2007, p. 163) and modification to the organisation’s ‘underlying norms, policies and objectives’ (Argyris, 1994, p. 8). In double-loop learning, the thermostat ‘questions’ the setting itself e.g. why it was programmed to a specific temperature and why it measures temperature at all (Argyris, 2004). For some theorists (e.g. Huber, 1991), OL requires changes to potential organisational behavior where information, knowledge or lessons learnt may impact future behavior and constitutes the organisational memory. Others argue that OL requires actual organisational behavior change along with cognitive change (Argyris and Schőn 1978, Fiol and Lyles 1985). Argyris (2004, p. 17), argues that OL requires internal, external and ‘implementable validity’, not just good ideas. OL is thus defined as the process which leads to a change in the range of potential organisational behavior or a change in actual organisational behavior (Argote & Miron-Spektor, 2011; Huber, 1991). Our study maps to a series of key OL constructs identified by organisational learning scholars including: knowledge acquisition, knowledge dissemination, knowledge interpretation and organisational change (Huber, 1991); experiential learning (Nonaka & Takeuchi, 1995); absorptive capacity (Cohen & Levinthal, 1990); productive and defensive reasoning; single-loop and double-loop OL (Argyris & Schőn, 1978); organisational memory (Argyris, 2004; Tsang, 1997). These constructs are captured and integrated in Bedford’s (2017) heuristic model of OL depicted in Figure 2 which constitutes our analytical framework [Figure 2 near here]. METHODOLOGY Our study focuses on the Operation Turning Point RCT undertaken between 2011 and 2014 by the West Midlands Police (WMP) across Birmingham, a city of 1.1 million people in the WMP area in the United Kingdom. Operation Turning Point tested a deferred prosecution with conditions (treatment) against prosecuting low harm offenders (control). In the three WMP local policing areas engaged in the trial, custody staff at five custody centres, police offender managers and middle and senior managers (in total more than 150 staff) were directly involved and trained to implement to the RCT. Eighteen in-depth, semi-structured interviews were conducted with a purposive sample of staff who were involved in implementing the RCT. The interviews were conducted in 2013 during the fourth phase of the trial. In this paper, the RCT, the organisation and the organisational units involved are discussed in generic terms which sufficiently captures the essence of the accounts, issues and debates raised by the participants in the interviews. The breakdown of interviewees was as follows:

OU1: Custody Sergeant N = 3 OU2: Offender Manager N = 6 OU3: Referral and Support Service N = 3

5 

SL: Strategic Leadership N = 2 OM: Operational Management N = 4

Total N = 18 The interviews were based on an interview protocol developed from analysis of the literature on implementing police RCTs (Neyroud, 2017). Questions focussed on categories identified by open coding articles on 63 police based RCT’s identified by Braga et al. (2014). The questions explored elements of the RCT implementation process including the role of the external context, police leadership, the role of University partners, and organisational learning processes. The interviews were coded and analysed by Bedford (2017) using an interpretive approach to qualitative analysis to organise the data so that it made sense in light of themes (Mason, 1996, p. 54). Instead of letting the ‘data speak for itself’ and for themes to emerge as the analysis progressed, the analysis derived from the theoretical constructs in the OL Framework (Bedford, 2017: Figure 2). The perspectives of different organisational units and ranks were compared and found to be highly consistent across both individual respondents and groups. RESULTS In presenting our analysis we engage the main constructs of the OL Framework to describe and discuss the findings. The use of direct quotations is illustrative of the analytical points made in the article and reflects the balance of opinion of the respondents unless otherwise stated. The supporting quotations relate to the key themes developed and reported by Bedford (2017) which were tied to the interview protocol (Neyroud, 2017) and reflected across all the transcripts. Knowledge acquisition and dissemination As suggested by the OL Framework (Figure 2), the RCT attributes and the external context influence the way the organisation acquires knowledge through the process of experiential learning at the operational unit level, and knowledge dissemination at the unit and organisational level. The ability of organisations to seek for solutions and learn through experience is also linked to their levels of absorptive capacity. These two constructs - experiential learning and absorptive capacity - are explored below in the light of the in-depth interviews. Experiential learning: Organisations learn through experience and experiential learning enhances organisational performance (e.g. Argote & Miron-Spektor, 2011; Huber, 1991; Lewin, 1951; Nonaka, 1994). Experiential learning is especially important in establishing whether or not an innovation, intervention or program is practically implementable (Argyris, 2004; Moingeon & Edmondson. 1996). Throughout the conduct of a RCT the exchange of tacit and explicit knowledge may enact a ‘spiral of knowledge creation’ (Nonaka & Takeuchi, 1995, p. 71) where experiential learning is conceived of as a purposive process of reflection-in-action through the constant conversion of tacit (individual) to explicit (organisational) knowledge (Nonaka 1994). For example, as Bedford (2017) notes, training may be conceived of as explicit to tacit knowledge. ‘Making mistakes and articulating and sharing lessons from them within the project team and wider organisation can be conceptualised as tacit to explicit knowledge’

6 

(Bedford, 2017, p.163). We found that both of these types of knowledge exchange were embedded within the Operation Turning Point RCT as illustrated by the following quotations:

Training, making mistakes – best way to learn sometimes really, isn’t it? … So [learning through experience] is the way we have learned pretty much (RS03:OM) You learn by experience and by doing (RS17:OU3). We learned a lot. And to be able to test and adapt and learn and improve was really important (RS10:OM).

The extent to which the RCT leveraged new possibilities for learning through a collective process of ‘doing’ was also highlighted and is captured in the following statements:

[S]ometimes [police] can be working silos with blinkers on [… it] can become very isolated within an organisation ... I think for those guys it has probably opened up a new alternative for them (RS03:OM). [I]t has been quite easy to have advice and give positive or constructive feedback, or criticism […] And teething problems seem to have been ironed out easily from good communication (RS16:OU2).

The flexibility and incremental changes to the way the experiment was implemented was also seen to be important. The ‘incorporation of feedback from the officers implementing the experiment into the RCT process was seen as critical to their buy-in. So too was their ability to exercise their own agency within the project’ (Bedford, 2017, p. 64):

People have been allowed to be creative. So that is probably the key … that we are police officers and have been given the freedom to be creative with our ideas and try them out (RS17:OU3).

The data suggest that the initial intention of the police in conducting the RCT was to find a better way (what works) to meet the stated goals (reducing crime efficiently) and bring the organisation back on course. However, in so doing, the underlying goals, rules and values of the organisation began to be tested and reconceptualised by actors within the RCT as it progressed. This was not based on the results of the RCT, but the experiential learning associated with the process of implementation. Absorptive capacity: Absorptive capacity is linked to the capacity of organisations to innovate (Greenhalgh et al., 2004; Huber, 1991) and recognize ‘the value of new external knowledge and information through exploratory learning; assimilate new knowledge through transformative learning; and, apply it to make high-quality decisions through exploitative learning’ (Cohen & Levinthal, 1990, p. 128). Absorptive capacity is also a function of the knowledge base of the organisation because ‘organisations may not necessarily make changes in behavioural terms as they acquire new knowledge; they may instead acquire the flexibility necessary to make adjustments as the (perceived) need arises’ (Shipton, 2006, p. 244). Our analysis found that the trial had enhanced the capacity of the police to successfully implement the specific RCT intervention if it were to be to be rolled out more widely:

7 

I think the way [the experiment] has been set up; we have done all the difficult parts now you know. If [the intervention] was going to go out to the rest of the force for example, the hard work has been done (RS03:OM).

Beyond the specific adaptation being tested in the RCT ‘the ‘knowledge base’ necessary to question existing values and assumptions had also been expanded’ as the trial was implemented. It had enhanced ‘the organisation’s existing level of flexibility and absorptive capacity’ (Bedford, 2017, p. 65):

[T]here is learning in relation to how to run experiments, there is learning in relation to initial findings on [other elements of policing activity]. And it is moving in now to how you diagnose most appropriately what the problems are in the first place. And that learning has gone way outside [the experiment] … (RS01:SL) [Working on the experiment] really makes you reassess. Rather than making sweeping generalisations that you have no evidence for, you start thinking a bit differently. So [you] think: ‘Where is the evidence to support what I want to do here?’ (RS18:OU3).

Knowledge interpretation RCTs are purposive feedback processes, that potentially influence the knowledge interpretation process. On the one hand, the RCT feedback process may influence the organisation to engage in a mindful process of OL based on productive reasoning. On the other hand, it may generate a reflex response based on defensive reasoning (Argyris and Schőn 1989). Productive and defensive reasoning: In the case of organisations, error is defined as a mismatch between intended and obtained organisational outcomes (Argyris 2004). The identification and articulation of this error may open the organisation to embarrassment or threat which leads to the enactment of defensive organisational routines and defensive reasoning (Argyris, 2004). In the case of the WMP, the strategic goal was ‘to reduce crime’ (RS01:SL), and to do so efficiently. Conventional methods of meeting this goal were seen to be failing – characterised by a revolving-door of low harm offending, apprehension by police and repeated court appearances:

[The police] are obsessed with preventing crime. [Business-as-usual] doesn’t seem to be working so we take it into our own hands, and try and affect crime in a different way (RS01:SL).

Argyris (2004) argues that, in contrast to defensive reasoning, organisational responses to perceived error that emphasise rigorous enquiry and testing potentially lead to productive reasoning to overcome the error. Productive reasoning may be sustained by agents alongside ‘organisational norms that encourage good dialectic and effective communication’ (Argyris, 2004, p. 12). The role of the RCT in overcoming resistance is captured by one respondent as follows:

A lot of staff in the organisation have had an initial resistance to working in this way. And therefore, a lot of [the experiment] has been about overcoming that

8 

resistance to a different way of working … not so much about running an experiment … [it’s not just about shifting] practice – [its] about shifting people, values and beliefs … it is not something you can do in an email or in a presentation … it is face-to-face: You-me having a conversation and talking it through (RS10:OM).

The ‘organisation had embarked on a trial of one of its core strategies, potentially highlighting its inability to meet organisational goals using the tried and mainstreamed methods. [Despite this], there was a significant openness on the part of the police leadership to confront potential ‘embarrassment’ and engage in productive reasoning’ (Bedford, 2017, p.67):

The [top police leadership] bought into it, [and] was prepared to take the risk – because there is a certain amount of risk in this experiment, both from resources but also from the proverbial [newspaper] headlines [denigrating the police strategy] (RS01:SL).

For some police officers, however, the commitment to the trial (and evidence-based policing) was based less on understanding than expediency. In other words, organisational commitment could in fact reflect defensive reasoning (Argyris 2004).

[T]here was quite a level of resistance from the police. [Support from] the high-level command was obviously very important … and they have inevitably told people involved ‘you will do this’ […] there wasn’t always buy-in further down the chain … (RS18:OU3).

The risks of the Turning Point approach were alluded to by numerous respondents. The main concern was with the way the trial opened the organisation to criticism (embarrassment; threat) from the general public and aligned to this, political stakeholders.

I think one of the difficult things is going to be getting the political message across to Jo Public, that this may be a better way … the general public (GP) see [business-as-usual approach] … as a success and winning. And I think to get the public onside with it may be the next challenge (RS15:OU1).

There was, moreover, a tendency with some in the organisation to only see the outcomes of a RCT to be worthwhile if they were ‘positive’, that is, if the RCT results demonstrated the alternative approach to be better than business-as usual approach:

There had been another [experiment] run in [the organisation] that has shown no benefit … and I have heard senior managers talk about that as a failure …. [That] ‘he has had a failure’. It is not that we have learned: ‘good here is something that we could have invested in, and we are not going to do it now’. It is a case of ‘oh, it hasn’t shown it, has it? It shows the whole evidence-based policing thing is nonsense.’ (RS10:OM)

As Bedford notes, ‘countering resistance was couched in terms of the need to shift ‘culture’ – as a proxy for underlying organisational assumptions, norms and values. This was particularly the case with police outside of the main implementing units who lacked an understanding of the trial and evidence-based policing in general’ (Bedford, 2017, p. 69).

9 

Organisational change OL is a process that happens over time and should not be conceptualised as a discrete event (Tsang, 1997). For theorists such as Argote (1999), Argyris (1994), Huber (1991), Nonaka and Takeuchi (1995), and Simon (1991), the internal production and conversion of organisational knowledge to organisational action and change is understood to be moderated by an OL process. Single loop learning: Argyris (2004, p. 199) suggests that without underlying changes to organisational governing values, organisational behaviour changes may be single-loop learning or they may be nothing more than short-run ‘managerial gimmicks and fads’. When organisational behavior changes in the absence of cognitive changes of organisational members, it is not OL but rather ‘forced learning’ or ‘imitation’ (Tsang, 1997, p. 77). Certainly, this was considered a potential risk with the trial:

Because not only are we doing it because we are being told to do it – let’s be honest we are doing it because we have been told to do things in the past that haven’t worked haven’t we? […] And I think in the early days this might have been another scheme or gimmick, that line management who have got limited resources … would have (RS16:OU2).

Even where “buy-in had been achieved, it needed to be constantly re-invigorated as the RCT unfolded” to counter resistance (Bedford, 2017, p. 75). Respondents highlighted the need to engage in persuasion on an ongoing basis throughout the implementation of the RCT:

It was going to happen anyway because we were going to make it happen, but we know that there is no point in having pressed people, and so there was an element of persuasion – ‘this is why you are doing it’, and ‘you are going to do it anyway so’ … It is [important through the implementation process to] remind them why it is good, why it is sensible, why we need to do this (RS01:SL).

Another view in OL theory is that behaviour change alone (single-loop learning) may lead to OL in the absence of a priori cognitive change. ‘In this view, behaviour change works backwards into the system to shift cognition and potential organisational behaviour over time’ (Burke 2002 cited in Bedford, 2017, p. 75). This view was held by some of the respondents:

I think certainly there needed to be more engagement [with staff] all the way through and a clearer mandate [from the organisation management] will do this. That is not to replace the attempt to win hearts and minds but to cover both sides of the equation. Some people respond to different things, some people respond because they agree with it and they are being persuaded, other people won’t be persuaded and whether [they] agree or not they still have to do it (RS18:OU3)

Double loop learning: In double-loop learning, modifications to actions need to be accompanied by modifications to underlying values, routines, beliefs and assumptions. We found that individual mindsets or assumptions and values of those participating in the RCT were changed through the course of the experiment.

10 

Custody Sergeants, particularly older Custody Sergeants … are generally regarded as probably being the most cynical bunch of police officers that you will ever come across, and the most difficult to convince that radical change is ever going to be a good thing. But very, very few Custody Sergeants failed to see the potential benefits of [the experiment] (RS15:OU1).

There was also a ‘clear sense that the RCT was more than a passing managerial ‘gimmick’ or ‘fad’’(Bedford, 2017, p.76):

There is lots of different pilots. They happen and then they stop … they happen and then they stop … [for] somebody that is within the service it gets frustrating […] Whereas with this it is impressive, really to know … how everybody is looking to drive it forwards (RS05:OU3).

The implementing teams not only felt that the RCT had played a role in changing values and culture at a team level. ‘Beyond this, they also saw it as their role not only to expand buy-in for the intervention, but also to shift values and culture to areas of the organisation not involved in the RCT’ (Bedford, 2017, p.76):

It is not as simple as simply sending a bunch of Custody Sergeants on a course about [the experiment] and expecting that to work fluidly at first. There is some cultural changes that have to be overcome as well, within a large organisation (RS03:OM).

At an organisational level, Bedford (2017) found that the shift from single-loop to double-loop learning aligned to a shift from an emphasis on RCT results (instrumental learning) to a later pre-occupation with understanding the implications of the RCT process as it unfolded (experiential learning). This extended way beyond the boundaries of what the RCT was testing:

We have learnt a whole lot about […] … we are therefore is a much better place now if we were to start again, to do something much more effective, much slicker, much better. But of course that is going to be the way, you know. You don’t know until you start something … and maybe the [aligned strategy] will be a thousand times better because we have had a chance to experiment and watch and practice with [this experiment] (RS10:OM).

Our research suggests that the RCT provided knowledge as an impetus for organisational learning. Nonetheless, ‘the knowledge provided from the RCT is a necessary, but not sufficient condition for OL: The learning needs to be operationalised at an organisational level. This was captured by the respondents who noted that organisational commitment had expanded over time’ (Bedford, 2017, p. 77).

[P]eople have gone away realising actually what they have always assumed was the best thing to do wasn’t actually very effective, and it wasn’t a very good service to … the community, and there were other alternatives. Initially they would have thought: ‘Oh this is nonsense’; whereas now a lot of them are thinking ‘I see the rationale in that’. So I think [the experiment] has had a much wider impact. It has had an impact as being a change program for values in [the organisation] (RS10:OM).

11 

Some of the more fundamental organisational change may involve a process of ‘unlearning’, of ‘forgetting’ (Hedberg 1981, Tsang 1997). This is because much of what organisations do is based on embedded routines and customs which may block the capacity for renewal (Nutley et al. 2007). Unlearning or ‘unfreezing’ (Lewin 1951) may be required before new learning can take place (Huber 1991). This was also highlighted in the interviews.

You have almost got to deconstruct one view before you can reconstruct a new view. (RS10:OM)

Learning how to improve organisational learning was also highlighted in the interviews and was articulated as learning about evidence-based policing.

It rebalances the craft science debate a bit as well where ‘we always know best’ to, ‘in actual fact sometimes we don’t always know best’ (RS01:SL). I have learnt what an academic trial is … I have learnt some important management lessons on how to get this over to reluctant members of staff… And I have got a real commitment now to evidence-based policing … (RS09:OM).

Further, the RCT was seen to have facilitated organisational capacity and openness to learning to learn, beyond the results of the specific RCT findings:

[The organisation] can learn from [the experiment] as a randomised experiment, and that is the way forwards for the police is to - before doing things - [ask] ‘are you doing it for the right reasons and good reasons?’ … (RS15:OU1).

This relates to the Huber’s distinction between OL aligned with a specific adaptation and OL associated with enhancing organisational adaptability or ‘flexibility’. The need for organisations to adapt in a rapidly changing or competitive environment is associated with the concept of ‘experimenting organisations’ which strive for adaptability and not just short-term adaptation to a specific context (Huber 1991; Starbuck 1984, cited in Bedford, 2017). This was articulated as follows:

[T]here is something coming in with this that says the police should be more scientific and should be more of a professionalised institution … So [the experiment] is not just about evaluating a new measure – it is about a way of operating (RS18:OU3)

As (Huber 1991, p. 93) suggests, organisational experiments are ‘generally directed toward enhancing adaptation, while maintaining organisational experiments is generally directed toward enhancing adaptability’. This was captured by some respondents as the shift towards a more evidence-based approach.

And that is what I want, I want every experiment just to be a launch into making an evidence-based approach part of the DNA of [the organisation] (RS01:SL).

Advancing organisational adaptability and evidence-based policing by learning from RCTs was, however, seen as the fundamental challenge by some senior officers within WMP (Bedford, 2017).

12 

…we have also learnt about doing experiments … how do we harness that enthusiasm to take the force on and continue working in this way, rather than finish[ing] [the experiment] and everyone going back to normal business? ‘We have been in a little bubble for a bit, but now we are back to normal police work’. It is actually about trying to change the course of the organisation a bit as well. And, oh heck, I am not quite sure how we are going to do that (RS10:OM).

Organisational memory Huber (1991) argues that OL requires change in cognition and potential organisational behaviour. OL has occurred if ‘through its processing of information, the range of its potential behaviours changed’ (Huber 1991, p. 89). The organisational memory is the cumulative information and knowledge that the organisation has acquired and retained (Huber, 1991). With staff involved in Turning Point there was a great deal to suggest that the new knowledge associated with the RCT had become normal procedure over the course of the implementation of the trial. ‘This included data and information systems that had been developed for the trial, but which were proving to be useful beyond the boundaries of the RCT’ (Bedford, 2017, p. 63). For example, one respondent stated:

We started [the experiment] with paper records. We now use a very basic online portal … Sitting behind that you have a database that academics can get access to, and we can pick bones out of … [This] is [potentially] something that won’t just be used locally but might just have an impact and influence nationally (RS17:OU3).

Harnessing the learning about evidence-based policing from the RCT – committing it to the organisational memory – was also a theme raised by a number of the respondents. For example, one interviewee said:

I think the challenge for the force is: Is it a one-off that the impact fades quite rapidly over time. Or is it something we can harness and we can continue? And we can foster? Which is quite hard. And that is an organisational issue [beyond individuals or the experiment itself], really (RS10:OM).

Argyris and Schőn (1996, p. 16) suggest that in order to become organisational, learning must be ‘embedded’ in the people or in the artefacts of the organisation. The Operation Turning Point case suggests that the design and extended roll out of the RCT led to the intervention (experimental condition) becoming embedded within the implementing organisational units for the duration of the RCTs. The invented solution was produced and tested over the course of the trial. ‘The RCT’s implementability was, in fact, a cornerstone of the success of the experimental process. The data suggest that the Turning Point approach started to feel like normal routine to a number of the respondents. It had become embedded in the people and processes in those units that were implementing it’ (Bedford, 2017, p. 73).

[I]t kind of feels [like] it is accepted and normal practice which is good (RS08:OM).

Tsang (1997, p. 77) suggests that it ‘is not advisable for descriptive studies [of OL] to incorporate actual behaviour change in their definitions’. This is because definitions that focus on actual observable behavioural change inadequately take into account the ‘long time

13 

lag between the time when knowledge is acquired and when it is put into practice’ (Tsang, 1997, p. 77). It is not possible to conclusively suggest that Turning Point generated observable change. Rather, the RCT has significantly influenced the organisational memory and changed the range of potential behaviours of the organisation. Our study shows that a significant amount of knowledge has been acquired through the RCT and this largely relates to the experience of the implementation of the RCT.

That stuff that the force learns out of it will be worth much more than what we were interested in. That is like added value that we never expected that we have just got. And that is like 80% of the value of that the force gets out of it will be added learning. Four times more! (RS10:OM).

The new knowledge is largely disseminated by and captured within the individuals involved in the experiment. ‘Data systems, formal meetings and interim results provided hard repositories of learning captured within the organisational memory. These have increased the absorptive capacity of the target organisation and enhanced its potential for double-loop organisational learning’ (Bedford, 2017, p. 81). Discussion of the Findings: We have explored how an RCT may impact police organisational change through OL by focusing on the views of police involved in the implementation of Operation Turning Point. Our aim was to understand their views of the specific processes of knowledge acquisition, dissemination and interpretation associated with that RCT. Our research did not set out to compare the extent to which RCTs may generate OL more effectively than other research methods. It also did not set out to provide evidence that OL had taken place. The focus was on the respondent’s views of the RCT processes, underpinned by OL theory. A strength of our study is its in-depth focus on the Turning Point RCT – a large trial, covering the whole of a major city – conducted in as real a world an environment as possible. However, this approach also has some limitations: Turning Point is not typical of many other smaller scale experiments, and WMP is not typical of other smaller police organisations. Nonetheless the findings from our study indicate that RCTs are potentially powerful change processes that serve to generate OL in the absence of results. Further research into the attributes of research and OL processes that foster OL in police organisations should move beyond an individual case approach and, indeed, beyond considerations of translation of research results. Our research suggests that more needs to be done to understand OL processes associated with RCTs in policing. There are four main findings from our study: experiential learning from a RCT potentially expands productive reasoning; RCTs may support specific adaptations, but they may also expand absorptive capacity, adaptability and double-loop OL; RCTs may enhance organisations’ ability to learn about learning (deutero learning); and, organisational learning may be embedded at team level in the organisational memory. Experiential learning from a RCT potentially expands productive reasoning The RCT generated OL beyond its formal findings or results, notably in the learning respondents associated with the process of implementation. Experiential learning was

14 

abundant, ongoing and welcome within the RCT. For example, formal and informal training and articulating and sharing lessons all allowed new knowledge associated with the RCT to become embedded within the implementing units as normal procedure over the course of the trial. Moreover, ‘the collective process of ‘doing’ and communication demonstrated a willingness to admit difficulties, own up to mistakes, and retain a commitment to scientific rigour, reflecting productive reasoning’ (Bedford, 2017, p.83). RCTs may support specific adaptations, but they may also expand absorptive capacity and double-loop OL The interviewees placed as much emphasis, if not more, on the concept of learning how to be adaptable as an organisation as they did on the adaptation of specific practice based on the results of the RCT. This was linked to the need to be more flexible and responsive in light of the increasingly complex role played by police. The trial served not only to expand the knowledge base for the implementation of the specific RCT intervention; it also impacted the organisation’s existing level of flexibility and absorptive capacity. Respondents suggested, for example, that over and above the trial of the experimental intervention itself, unanticipated and oblique learning had taken place related to other elements of the implementation process, such as improved treatment support IT and training and feedback processes (Slothower, 2014a; Slothower, Sherman & Neyroud, 2015). Throughout the interviews, participants highlighted a perceptible shift within the implementing team towards questioning of existing routines and incorporating scientific enquiry as an organisational goal. RCTs may enhance an organisation’s ability to learn about learning (deutero learning) Our respondents highlighted the role of the RCT in driving forward a broader organisational commitment to evidence-based policing in general: not only ‘what works’, but also ‘why and how it works’. The respondents emphasised the role of the RCT in expanding the professional capacity of individuals and the ability of individuals and teams to recognise, assimilate and apply new knowledge. So, the initial motivation for the research may have been adaptive, based on the identification of a mismatch between stated goals and outcomes, but the nature of the research process drove a commitment to expanding adaptability and organisational capacity for experimentation, productive reasoning and double-loop learning. A concrete example of this was the way in which the staff involved in the frontline and middle management engaged in Turning Point developed their understanding that the success of the experiment extended beyond a binary judgment about the outcome to a much broader range of outcomes from better tools for treatment, improved methods of implementation and a future capability to test. It is notable that WMP have continued to experiment and, at time of writing, had committed to a further four experimental tests of pre-court diversion and neighbourhood justice. Organisational learning may be embedded at team level in the organisational memory In order to become organisational, learning must be embedded in the people or in the artefacts of the organisation (Argyris 1994). The design and extended roll out of the RCT facilitated learning from it becoming embedded within the implementing organisational units for the duration of its existence. The invented solution was produced and tested over the course of the trial and started to feel like normal routine to a number of the respondents. It had become embedded in the people and processes in those areas which were implementing it and changed the range of potential behaviours of the organisation. Results from the RCT

15 

were needed, however, to realise that potential and ‘sell’ the project within the organisation beyond the learning at a team level. While significant and meaningful change in values had taken place within the project teams, our research suggests that only hard results would allow these changes to be operationalised – to become organisational. The implementing teams have effectively become the repository of organisational memory – and they are the key to unlocking the broader learning when the organisation is ready to accept the RCT results. Key Considerations for the Implementation of RCTs:

Bedford’s study of OL from RCTs (2017), draws a number of key lessons from the case of Operation Turning Point RCT and ties these back to her OL Framework, Figure 2. She found that elements of the external context acted to both catalyse and determine the role the RCT played in OL. Specifically, a key factor in determining the level of top leadership support for the intervention is the extent to which it is seen to align with the overarching strategic direction of the organisation and the organisational field.

In the case of Operation Turning Point, respondents suggested that ‘selling’ the message beyond the implementing units would be dependent on a formal set of conclusive results once the RCT had run its course. However, formal results would only be acted on by the organisation if the RCT demonstrated the effectiveness of the intervention, that is, the results were ‘positive’. So ‘while economic and political interests were catalysts in sending the police on a quest for a better and cheaper way of doing business’ (Bedford, 2017, p. 141), police leaders within the organisation and the implementation team must be prepared, in advance of implementation of the RCT, to accept weak, ‘negative’, and null findings. In other words, they must be actively prepared to embrace the scientific process. Results will almost certainly be buried, ignored or disputed if null or backfire results are presented without the implementing team preparing leadership to expectat them. This does not mean that the RCT has ‘failed’. Choosing not to invest in a changed course of action, innovation, or technology based on hard evidence is of value to police.

The value of RCTs, however, extends beyond the evidence and research results. RCTs may not only support specific adaptations, they may also enhance organisational adaptability and absorptive capacity even in the face of null or ‘negative’ results. As Huber notes organisational experiments are ‘generally directed toward enhancing adaptation, while maintaining organisational experiments is generally directed toward enhancing adaptability’ (Huber, 1991, p. 93). OL is not, therefore, a discrete event, but a process that involves increasing organisational flexibility and adaptability such that organisations are better prepared to meet challenges as they arise (Shipton, 2006). The OL process, thus, involves the development of an organisational knowledge base over time (Huber, 1991).

In the case of Operation Turning Point, officers involved in the RCT were interested in the specific policing adaptation associated with the Operation Turning Point intervention, but they also emphasised the value of the RCT in terms of its broader role in learning how to be adaptable as an organisation. They linked this need for adaptability to the need for more flexible and responsive policing service delivery, and the increasingly complex role played by police. So, ‘while Operation Turning Point served to expand the knowledge base for the implementation of the specific RCT intervention, it also impacted the organisation’s existing level of flexibility and absorptive capacity’ (Bedford, 2017, p. 146). Further, the RCT generated a form of reflexive process learning and facilitated critical reflection on business-as-usual practices in general – not just the Operation Turning Point intervention. Participants noted the increasing tendency of officers involved in the RCT to question existing routines more generally, and value scientific enquiry as an organisational goal.

Whereas all organisations incorporate O-I learning systems, the challenge in OL is to incorporate Model II theories-in-use within these. Model II theories-in-use emphasise informed choice, the production of valid information, enquiry and rigorous testing, to break down defensive routines and allow productive reasoning (Argyris and Schőn, 1996). In this regard, RCTs are especially valuable, because ‘the very

16 

purpose of an RCT is to produce valid and reliable information through enquiry and rigorous, public testing of the effectiveness of systems and activities to promote informed choice’ (Bedford, 2017, p. 142). Bedford’s (2017) research supports the view that RCTs may constitute ‘repositories’ of Model II learning systems and when incorporated into Model I type organisation that is receptive, they may promote and sustain productive reasoning by the continued use of Model II theories-in-use by organisational agents alongside ‘organisational norms that encourage good dialectic and effective communication’ (Argyris, 2004, p. 12). In this way, we argue, the Operation Turning Point RCT had significant value well beyond its formal findings or results. At a sub-organisational level, the shift from single-loop to double-loop learning was associated with a shift from a focus on RCT results to an emphasis on the implications of the RCT process (Bedford, 2017). While the initial motivation for the RCT may have been ‘based on a reflexive adaptive response to identifying a mismatch between stated goals and outcomes, the nature of the research process drove a commitment to expanding the organisational capacity for experimentation, use of research, productive reasoning and double-loop learning’ (Bedford, 2017, p. 148).

To summarise, Bedford (2017) found that when implemented in such a way as to maximise experiential learning and productive reasoning – by incorporating Model II organisational learning systems – the process of implementing an RCT may break down defensive reasoning and expand organisational absorptive capacity and double-loop OL. In the case of Operation Turning Point, experiential learning from the RCT shifted underlying values and assumptions of the officers involved in its implementation and entered the organisational memory. RCTs are most likely to generate OL if they aim to test alternatives to business-as-usual approaches and issues that relate to error identified by police themselves. Further, when implemented by police as repositories of Model II systems, they may do away with the need for research translation because the police, even if only at sub-unit level, ‘already speak the language’ (Bedford, 2017, p.156). The implementation of RCTs without capitalising on the potential for OL and research translation is potentially a wasted opportunity to deliver innovation, service delivery improvement and change. As Bedford (2017) notes,

the process of OL is reflexive and iterative. Much like a game of pinball where the pinballs represent the RCT and its attributes (who pulled the lever, when, where, why, and how) and the pinball table represents the organisation and its context. From the moment the balls are unleashed into the table matrix, they are in play, potentially striking targets and generating ‘points’ (experiential learning, absorptive capacity, productive reasoning, double-loop OL) as long as they are active. On a high score, the points may carry over (organisational memory and change). The pinballs may, of course, lack traction, fail to strike any target at all, and fall through the paddles at the base of the table and into the gutter (Bedford, 2017, p 155).

CONCLUSIONS More than forty years ago Clarke and Cornish (1972) demonstrated that RCTs were difficult and challenging in the context of a criminal justice agency. In the subsequent methods debate the potential for learning from testing was largely lost in the noise. As more and more RCTs are led by practitioners or managed in partnership between police practitioners and academics (Neyroud & Weisburd, 2014) focusing on their wider implications for increasing organisational flexibility and enhancing organisational learning should be accorded more attention. The value of policing RCTs in organisational learning (OL) appears to lie, on the one hand, in their ability to provide hard evidence or results of ‘what works’ in policing. On the other hand, RCTs may be powerful change processes which act to catalyse reform and enhance policing adaptability through double-loop organisational learning (Bedford, 2017). For police forces, the challenge is to turn that potential into a sustainable reality. Whilst this article has centred on one randomised trial, a small group of police forces – three examples being West Midlands Police, the Queensland Police Service and Western Australia

17 

Police – have since decided to embed the process of learning and testing into organisational strategy and routines. These ‘field stations’ (Sherman, 2010) have set up evidence-based policing units to manage and facilitate both testing and the implementation of developed models which may enhance the capacity of organisations to embed Model II governing values and enhance absorptive capacity. They are committed to delivering a series of tests designed to support the agencies’ strategy for service improvement and OL. In this model, experiments in policing cease to be exclusively owned by academics and researchers and become part of the profession of policing’s process of development and self-improvement. REFERENCES Argote, L. (1999). Organisational learning: Creating, retaining and transferring knowledge. Norwell, MA: Kluwer Academic Publishers. Argote, L., and Miron-Spektor, E. (2011). Organisational learning: From experience to knowledge. Organisation Science, 22(5), pp. 1123-1137. Argyris, C. (1994). On organisational learning. Cambridge, Massachusetts: Blackwell Publishers. Argyris, C. (2004). Reasons and rationalizations: The limits to organisational knowledge. New York: Oxford University Press. Argyris, C. & Schőn, D. A. (1989). Participatory action research and action science compared: A commentary. American Behavioural Scientist, 32. Argyris, C. & Schőn, D. A. (1978). Organisational learning: A theory of action perspective. Reading, MA: Addison-Wesley. Argyris, C., & Schőn, D. A. (1996). Organisational learning II. Reading, MA: Addison-Wesley. Bedford, L. (2017). Randomised controlled field trials in policing: An organisational learning perspective. PhD Thesis, Institute for Social Science Research, University of Queensland.  Bedford, L., & Mazerolle, L. (2014). Beyond the evidence: Organisational learning from RCTs in policing. Policing, 8(4), pp. 402-416. Braga, A. A. & Weisburd, D. L. (eds.). (2014). Advances in randomised experiments and quasi experiments in criminology and criminal justice. Evaluation Review, 38(1). Burke, W. W. (2002). Organisational change: Theory and practice. Thousand Oaks, CA: Sage Publications, Inc. Cohen, W., & Levinthal, D. (1990). Absorptive capacity: A new perspective of learning and innovation. Administrative Science Quarterly, 35(1), pp. 128-153. Clarke, R. V. G., & Cornish, D. B. (1972). The controlled trial in institutional research: Paradigm or pitfall for penal evaluators?. London, UK: HM Stationery Office.

18 

Cyert, R. M., & March, J. G. (1963). A behavioural theory of the firm. Englewood Cliffs, NJ, Prentice-Hall. Fiol, C. M., & Lyles, M. A. (1985). Organisational learning. Academy of Management Review, 10(4), pp. 803-813. Greenhalgh, T., Robert, G., Macfarlane, F., Bate, P., & Kyriakidou, O. (2004). Diffusion of innovations in service organisations: Systematic review and recommendations. The Milbank Quarterly, 82(4). Hedberg, B. (1981). How organisations learn and unlearn. In P. Nystrom and W. Starbuck (Eds.), Handbook of Organisational design (Vol. 1, pp. 3-27). Oxford: Oxford University Press. Huber, G. P. (1991). Organisational learning: The contributing processes and the literatures. Organisation Science, 2(1), pp. 88-115. Lewin, K. (1951). Field theory in social science: selected theoretical papers. In D. Cartwright (Ed.). New York: Harper and Row. Lum, C., Telep, C. W., Koper, C. S., & Grieco, J. (2012). Receptivity to research in policing Justice Research and Policy, 14(1). Mason, J. (1996). Qualitative researching. London: Sage. Moingeon, B., & Edmondson, A. (1996). When to learn how and when to learn why: Appropriate organisational learning processes as a source of competitive advantage. In B. Moingeon and A. Edmondson (Eds.), Organisational learning and competitive advantage (pp. 17-37). Thousand Oaks, CA: Sage. Neyroud, P.W. (2016), Researching the police: Inside-outside perspectives in a new world of police professionalism and practitioner research, In M. Cowburn, L. Gelsthorpe and A. Wahidin. (Eds.) Research Ethics in Criminology: Dilemmas, Issues and Solutions, London: Routledge (pp. 77-95). Neyroud, P.W. (2017) Learning to field test in policing: Using an analysis of completed randomised controlled trials involving the police to develop a grounded theory on the factors contributing to high levels of treatment integrity in police field experiments. PhD Thesis, Institute of Criminology, University of Cambridge. Neyroud, P., & Weisburd, D. (2014). Transforming the police through science: The challenge of ownership. Policing, 8(4), pp. 287-293. Nonaka, I. (1994). A dynamic theory of organisational knowledge creation. Organisation Science, 5(1), pp. 14-47. Nonaka, I., & Takeuchi, H. (1995). The knowledge-creating company: How Japanese companies create the dynamics of innovation. New York: Oxford University Press.

19 

Nutley, S. M., Walter, I., & Davies, H. T. O. (2007). Using evidence: How research can inform public services. Bristol, UK: The Policy Press. Shepherd, J.P. (2003) ‘Explaining feast or famine in randomized field trials: Medical science and criminology compared’, Evaluation Review, 27(3), pp. 290-315. Sherman, L. W. (1998). Evidence-based policing. Washington, DC: Police Foundation. Shipton, H. (2006). Cohesion or confusion? Towards a typology for organisational learning research1. International Journal of Management Reviews, 8(4), pp. 233-252. Sherman, L.W. (2010). An introduction to experimental criminology, In Piquero, A.R. and Weisburd, D. (Eds.) Handbook of Quantitative Criminology, New York: Springer, pp. 399-437 Sherman, L. W. (2013). The rise of evidence-based policing: Targeting, testing, and tracking. Crime and Justice, 42(1), pp. 377-451. Shipton, H. (2006). Cohesion or confusion: Towards a typology for organisational learning research. International Journal of Management Reviews, 8(4), pp. 233-252. Simon, H. (1991). Bounded rationality and organisational learning. Organisation Science, 2(1), pp. 125-134. Skogan, W. (2008). Why reforms fail. Policing and Society, 18(1). pp. 23-34. Slothower, M.P. (2014a). Strengthening police professionalism with decision support: bounded discretion in out-of-court disposals. Policing, 8(4), pp. 353-367. Slothower, M. (2014b). Victim satisfaction and perceptions of police and criminal justice legitimacy: Mediating impacts of perceptions of problem-solving motive achievement and matching, [Unpublished M.Phil. Thesis, University of Maryland]. Slothower, M.P., Sherman, L.W. & Neyroud, P.W. (2015). Tracking quality of police actions in a victim contact program: A case study in training, tracking and feedback (TTF) in evidence-based policing. International Criminal Justice Review, 25(1), pp. 98-116. Sparrow, M. K. (2011). Governing science. Washington DC: National Institute of Justice. Starbuck, W. (1984). Organisations as action generators. Americal Sociological Review, 48, pp. 91-102. Tsang, E. W. K. (1997). Organisational learning and the learning organisation: A dichotomy between descriptive and prescriptive research. Human Relations, 50(1), pp. 73-89. Weisburd, D., & Neyroud, P.W. (2011), Police science: Toward a new paradigm. New Perspectives in Policing: Harvard Kennedy School.

20 

Figure 1: Randomised Trials in Policing 1970-2015

Source: (Neyroud, 2017)

1

0 0

1

0

1

0

6

0

1

0 0

5

0

2 2

7

1

0

2

3 3

4

1 1

3

1 1

4

6

2

3 3

0

1

2

6

3 3

2

1

8

4

6

1111

19701972197419761978198019821984198619881990199219941996199820002002200420062008201020122014

Randomised Trials in Policing 1970‐2015 (n.=122)Neyroud (forthcoming) 

21 

Figure 2: The Organisational Learning Framework (Source: Bedford, 2017)

Source: (Bedford, 2017)