Next Generation Technologies eBook Series

Trial Blazers

The Truth About Forecasting Clinical Trial Supplies

Part II

2

The

Trut

h A

bout

For

ecas

ting

Clin

ical

Tria

l Sup

plie

s P

art

II

Please note: this eBook will sometimes use the three terms—IWR, IVRS, IRT—interchangeably to describe a type of system in a general sense, unless a specific interface is being discussed.

In Part I of the Next-Generation Technologies eBook series, we discussed the new

class of IWR/IVRS/IRT support systems as they compare to legacy systems: uncovering

the benefits as well as the limitations and challenges. We established that the next-

generation is solving the painful problems of the current state of IVRS — cutting trial

setup times and adapting better to changes that can occur during the process of a trial.

Click here for Part I: Next-Generation IWR/IVRS.

Clinical trial supply forecasting and simulation is the focus for Part II of the Next-

Generation Technologies eBook series. Clinical Trial [Supply] Simulation is the

practice of modeling all important aspects of protocol design and supply chain, adding

assumptions where needed and then producing an approximation of kit demand over

time (in the simple case) or a daily global snapshot for each day (in a robust case).

This eBook series uses concepts which will be key to understand. Some of them are

technical in nature, but all result in dramatic real-world impacts and therefore should

be laid out clearly. Please click on the highlighted words throughout the paper to link

to the descriptions in the Important Concept section located at the back of this eBook.

3

The

Trut

h A

bout

For

ecas

ting

Clin

ical

Tria

l Sup

plie

s P

art

II

Not all Clinical Trial Simulators are created equal. Some are built specifically for trial startup logistics, some focus on

enrollment and others focus on the possible results of adaptive or dynamic randomizations. However, none of these

focus on clinical supply. The simulator discussed in this eBook takes clinical supply factors as input and generates a

forecast. Hence, it’s a “Clinical Trial Supply Simulator”.

The Business Value of SimulationIn a clinical trial, significant business value can be derived from the proper management of supply. In order to realize

this, the complexities and challenges in clinical trials and how to overcome them must first be understood.

Complexities and Challenges There is a multitude of complexities in modern clinical trials. An IVRS is built to support these complexities. Clinical

Supply Simulators are built to model the IVRS. However, the scope of what may be modeled in a clinical trial is

daunting. For example:

Clinical Trial Supply Simulation

INPUT(Trial Structure and

Supply Chain Details)

OUTPUT(When & Where the Supply is Being Used)

IVRSIMULATOR

4

The

Trut

h A

bout

For

ecas

ting

Clin

ical

Tria

l Sup

plie

s P

art

II

1. “When a site requires a shipment and not all of the drug is available, do we ship anyway (partial

shipments)?”

2. “A trial is dispensing different drugs in three different regions due to availability of different dose

strengths, and in one country the comparator will be sourced locally.”

Complexities such as these are not trivial to model. Simulation cannot generate a valid forecast if these

complexities are modeled incorrectly. The validity of the forecast is directly proportional to business value.

The higher the validity, the higher the business value.

The first level of complexity and challenge that must be overcome is trial design and logistical plan.

This must be clear and well understood. Without this, little else can be realized.

The second level is to understand the IVRS resupply algorithm and its control parameters. This can be particularly

challenging, especially when working with multiple vendors who use different algorithms. Nevertheless, a basic

understanding of the IVRS resupply algorithms and control parameters is critical for a simulator to generate a valid

forecast.

The third and final level of complexity and challenge is dealing with simulation tools themselves. When

representing all the various levels of complexity that can occur, it is easy to produce an overwhelming amount of

available functionality. This can be overcome by collaborating with simulation vendors to improve and simplify the

use of their tools.

Read more about our benchmarking survey by clicking here.

Power and ValueDespite challenges, the power of clinical supply simulation is such that it can provide value at a scale and speed

unavailable in any other facet of the clinical trial business. Global pharmaceutical companies (large and small) are

under tremendous pressure to operate more efficiently, even as clinical trial designs and the demands of regulatory

agencies continue to increase. Operational efficiency can be realized via Clinical Supply Simulation.

5

The

Trut

h A

bout

For

ecas

ting

Clin

ical

Tria

l Sup

plie

s P

art

II

Two sources of inefficiencies in trials are stockouts and unnecessary overage. Stockouts result from trials that are run with

inventory levels that are too low and can yield delays and additional costs. To prevent stockouts, excess drug is produced.

However, this results in unnecessary overage: excess drug that is produced

and never shipped. Clinical Supply Simulation can determine the necessary

overage for a trial and thereby help a small biopharma company run a set

of trials on a budget or save a large biopharma company tens of millions of

dollars in drug that is produced and never even shipped.

The concept of “necessary overage” is very trial specific. In our experience,

we have seen trials that require 0% overage, trials that require 1500%

overage and everything in between. Each trial has the potential for both

savings and risk reduction.

The bottom line is that clinical supply simulation can save hundreds of

thousands or even millions of dollars in a single trial and reduce the risk

of stockouts. In some cases, it enables otherwise failing trials to run. This

significant cost savings are possible for the low cost of a simulation tool and

a couple weeks of modeling.

The Technical Details of SimulationWe now understand that while intuition is powerful and sometimes accurate

for planning supply in simple trials, it is not sufficient to address complex

trials. Simple math operations (+, -, *, /), on paper, in Excel, or in a purpose-built web application only produce a baseline

supply consumption. Simulation produces a forecast.

BEFORE SIMULATION: Unnecessary OverageIntuition (from experience) is used to plan how much drug is needed at each site - to cover predictable & unpredictable demand

Monte Carlo simulation output reveals the amount of drug neededat each site - to cover predictable & unpredictable demand

AFTER SIMULATION: Necessary Overage

6

The

Trut

h A

bout

For

ecas

ting

Clin

ical

Tria

l Sup

plie

s P

art

II

Types of SimulationSimple simulation (a.k.a. “statistically-driven simulation”) employs statistical formulae to approximate trial activity and does

not model all the important elements of a protocol. This type of simulation (if designed correctly) can produce a more accurate

forecast.

Robust simulation is capable of modeling each element of the study by representing all the driving forces of a trial as objects,

on each day in the future. This method is more readily understood and issues can thereby be identified and resolved quickly.

SUPPLY CHAIN — including but not limited to:

• Depots• Sites• Shipping lines and lead times• Patient enrollment (estimate)• Drug lot, label and expiry information

(if available)• Country and site start dates• Intended IVRS resupply algorithm

and settings

• Shows when and where your supply is being used. • Reduces risk of stockout with smarter use of supply. • See supply problems before they happen.

OUTPUT

INPUT

STUDY DESIGN — including but not limited to:

• Kits (dispensing units)

• Treatment Groups • Visit Schedule, including - Durations and windows - Drop rates - Phased visits or skips - Screening, randomization (and re-randomization)

Plus any further complexities that drive the progress of patients through the study.

All culminating in the dispensing matrix for the trial:

• What kit(s) should be dispensed to which subjects by visit, by Treatment Group? • What titration options may be in place, and what probabilities? • What rules based on prior dosing or other classification?

7

The

Trut

h A

bout

For

ecas

ting

Clin

ical

Tria

l Sup

plie

s P

art

II

In both simple and robust simulation, the simulator may be executed multiple times while varying the randomness in each

“run” or vision of the future. This is called Monte Carlo simulation. Monte Carlo is especially powerful since it can vary random

events like randomization, patient arrival and titration, which forces the supply plan to account for many different possibilities

— just like reality. Since robust simulation models all the important elements of a trial, it is a more powerful method for

forecasting and monitoring clinical trials.

Click here to check out our blog “Taking the ‘art’ out of ‘The Art of Clinical Drug Supply Forecasting”.

IVRS in SimulationSince IVRS tends to be at the center of most modern clinical trials whenever clinical supply is a concern, it naturally plays a

large role in simulation. The chosen IVRS typically moves clinical supply from depot to depot, from depot to site, and dispenses

drug to patients at the site. In some cases it facilitates complex titration rules — all the while taking into account expiry,

country labeling and other characteristics that are stipulated by trial design, drug packaging, local regulatory authorities, or

some combination thereof. It would be fair to say that IVRS is the driving or originating system for all actions that influence

clinical supply during the normal course of a trial.

A good simulator then, must act like there is an IVRS in the middle of it. In fact, most major IVRS functionality that drives

drug supply must be at least approximated by the simulation, or else the simulation will not faithfully represent what actually

happens during the trial — which would defeat the very purpose of simulation in the first place!

If IVRS functionality is the core or “inside” of a simulation model, a host of other capabilities form the outside layer. This Starts

with the ability to introduce patients and drug into the trial via projected enrollment rates and packaged lots respectively. Once

simulated sites, patients, drug, etc., are introduced, time can be moved forward, future days begin counting and these outside

elements — simulated patients, kits, shipments, etc. — start interacting with the inside element: the IVRS.

In the end of a single simulation run, one of the possible outcomes of the trial should be evident. With robust simulators, it is

even possible to examine the global state of the trial and all elements in it at any future date in the study.

8

The

Trut

h A

bout

For

ecas

ting

Clin

ical

Tria

l Sup

plie

s P

art

II

Parameter Driven SystemsIf IVRS is the core of a modern simulator, then the resupply algorithm is: the epicenter. The resupply algorithm ultimately

determines how much value IVRS can bring to a sponsor in clinical supply savings. But the resupply algorithm itself is only half

of the solution. The second half of the solution is the parameters and settings that control the algorithm. The best algorithm will

always supply a large safety stock threshold to every site, if that is what the parameters tell it to do.

From a simulation point of view, there are two aspects to the resupply algorithm that need to be addressed:

1. Is the simulator capable of matching the resupply algorithm that the sponsors chosen IVRS is using?

2. Can the simulator shed light on what settings and parameters ought to be used to drive the IVRS resupply

algorithm?

In order to address the first consideration, the simulation tool vendor must be aware of the resupply algorithms that exist at

IVRS companies. Obtaining technical details on the various IVRS resupply algorithms can be a challenge because this topic is

complex and too few people possess the knowledge. Furthermore, since IVRS companies can invent new algorithms, it can be

nearly impossible to state at any given time that a simulator “contains approximations of all IVRS resupply algorithms”.

A solution is for the simulator to contain bits and pieces of different solutions, which may be included or disabled as needed in

order to cobble together an approximate match for the IVRS.

In order to address the second consideration, the simulator can be employed to derive values for the control parameters that

drive the resupply algorithm. The important parameters to understand are the buffers. The floor and ceiling buffers exist

in order to supply unpredictable demand and thereby reduce or eliminate risk. Clinical supply simulators can automatically

determine the necessary buffer stock values by country, site, or over time. This capability is the only scientific way to decide

what site and depot buffers should be, and when. The alternative is to employ full time staff to continuously analyze and tune

the site buffers globally— which is costly, error prone and produces waste. Establishing appropriate buffers for the resupply

algorithm is a requirement for running a clinical trial in order to maximize business value.

In summary, the IVRS resupply algorithm and its properly configured control parameters drive supply efficiency during a clinical

9

The

Trut

h A

bout

For

ecas

ting

Clin

ical

Tria

l Sup

plie

s P

art

II

trial. If the parameters are poorly configured, then risk and waste is introduced. A simulator is a convenient and inexpensive way to

pilot resupply algorithms and their control parameters. Why not figure out if a resupply algorithm and a set of parameters is any good

before shipping drugs globally?

Parameters and Pluggable Code in SimulationIn retrospect, it may seem obvious that throughout this eBook, we have been focused on parameter driven systems. The idea of

putting the system directly in the hands of the sponsors or business experts, the difficulties in approximating IVRS and IVRS resupply

algorithms, even the section on challenges which describes the daunting task of learning simulation tools, all imply that the tools in

question must be parameter driven. A custom-programmed system for each trial would have none of these challenges — although

the programmers responsible for the system would certainly have to do some work! But neither would it be available as a “shrink-

wrapped” commercial off-the-shelf product, which can be used by ordinary people to set up and simulate models of clinical trials.

A subtle point emerges here: in order to put the power to model complex clinical trials into the hands of the sponsor or business

experts, then the system must be parameter driven. Trials must be able to be modeled without specifications and programmers on

hand. And so, most modern clinical trial supply simulators are in fact parameter driven.

Parameter driven systems are by nature quite complex. They are also prone to complexities because they must stand ready to model

any trial regardless of its complexity. A robust clinical trial modeling system, such as a simulator, is usually ready to model a wide

variety of clinical trial and supply chain features which, on any given trial, the sponsor will not need. So a natural challenge is

inherent there and techniques are currently being developed to address this.

With that challenge aside, parameter driven clinical supply simulators also face the same challenge that is described previously about

parameter driven systems: no matter how complex they are, they cannot be built to handle all possible cases.

As a careful reader of the concepts section might assume, the solution for a fully robust parameter driven system to be able to handle

unexpected features or events, is the use of custom pluggable code. Custom plugins can take a variety of forms, but in their simplest

incarnation, are simply scripts, usually written in the same programming language as the product is written in; this can be inserted

into the simulation then picked up and run at run-time (i.e. during the simulation itself).

10

The

Trut

h A

bout

For

ecas

ting

Clin

ical

Tria

l Sup

plie

s P

art

II

With the more advanced clinical trial simulators, these plugins are able to make use of and extend or even modify some of the

existing functionality of the tool itself. Ideally, these can be presented directly in an editor so that none of the usual build and

deployment tasks are necessary to literally customize a running simulation product, on the fly.

Make no mistake, custom plugins certainly are programming code — but they are typically very small snippets of programming

code which can make use of powerful system code already present in the simulation tool, and they can be carefully developed

in an isolated area, either by sponsor personnel or more likely by vendor staff.

With custom plugins to simulation tools, one can achieve nearly anything imaginable — from simple things like sending

arbitrary shipments between depots, or modifying IVRS settings on the fly based on arbitrary milestones, to modifying or even

literally replacing a randomization, or dispensing or resupply algorithm with something more representative of what will actually

happen in the trial.

Custom plugins are not required in the majority of cases for clinical supply simulation, but they can be immensely handy when

something odd pops up.

In this eBook, we have discussed the types of clinical trial simulations, methods of forecasting and simulation modeling.

Further, we learned that despite the many complexities and challenges associated throughout the process, the power of clinical

trial supply simulation is such that it can provide value at a scale and speed unavailable in any other facet of the clinical trial

business.

Part III of this eBook series will reveal the relationship between simulation and IWR/IVRS — recognizing similarities, modeling

structures and data processing as described; the 360 concept.

Conclusion

11

The

Trut

h A

bout

For

ecas

ting

Clin

ical

Tria

l Sup

plie

s P

art

II

Clinical trials involve a variety of unknowns. Where in the world will subjects enroll and at what rate? What treatment

group will they randomize to? What titration path will they take within a treatment group? What about dropout rates? What

unplanned visits will occur? What about expiring drug? Despite these unknowns, a clinical supply plan must be created to

properly supply and run a clinical trial.

In the simplest case, it is usually possible to calculate baseline drug consumption for a clinical trial by multiplying the number

of intended patients in each group by the number and types of intended doses for that group. However, this ignores the

complexities of how much “overage” to place at sites and depots, in order for the trial to run. Since drugs cannot be instantly

delivered to sites where patients enroll, it usually has to be pre-positioned. This may drive the necessary or required overage for

a trial — that is, the amount drug that you will definitely not use in the trial but you need anyway.

There are approximately four different methods that people use to calculate

overage for a given clinical trial. In order of effectiveness, they are:

• Simulation

• Excel®

• Intuition

• Basic math (on paper or in a simple web application;

actually does not produce overage)

Although all four methods are still in use. The method employed depends on the sponsor, complexity of the trial in question

and tools at hand for the person creating the supply plan. Since the middle of the last decade, simulation has emerged as the

leading and most accurate method of forecasting clinical supply requirements.

Concept 1: Clinical Supply Forecasting / Simulation

Important Concepts

SIMULATION

EXCEL®

OR

DE

R O

F E

FFE

CTI

VEN

ES

SINTUITION

BASIC MATH ON PAPER OR IN A SIMPLE WEB APPLICATION

12

The

Trut

h A

bout

For

ecas

ting

Clin

ical

Tria

l Sup

plie

s P

art

II

Concept 2: IWR / IVRS / IRTIn clinical trials, “IVRS” is an imperfect name for a very specific type of technology. This phone-based technology (literally:

“Interactive Voice Response System”) was originally built to support patient randomization over the phone, but has evolved

to include management of clinical supply (from depots to sites to dispensing to subjects). Other common functions of IVRS

include screening / registration (e.g. generation of the patient ID), changing a subject status (e.g. screen failed), changing a

kit status (e.g. lost or damaged), break blind (e.g. subject emergency), and advanced dispensing (e.g. titration or investigator

choice). All along, clinical supply algorithms have been improving and becoming more complex, with most serious IVRS now

including complex predictive algorithms for just-in-time delivery of future patient dosing. In summary IVRS has evolved from a

phone based randomization systems into an advanced clinical supply management dispensing system.

Accordingly, the primary driver for companies to use IVRS is the significant savings in drug supply, relative to non-IVRS trials,

and therefore most IVRS companies spend a good deal of effort attempting to do this well.

As IVRS became more advanced, migration to the web began. Either as an option included with IVRS or as a separately built

system, this was dubbed “IWR” for “Interactive Web Response.” Yet another common acronym is “IRT” for “Interactive

Response Technology” or “Interactive Randomization Technology.” This eBook commonly uses the terms IVRS and IWR/IVRS,

either of which should be taken to mean all systems that fall into that category regardless of their presentation on web, phone or

both.

Concept 3: Resupply AlgorithmsSignificant supply savings is a common driver for companies that adopt IVRS for multi-center trials. The non-IVRS method

of providing clinical supply to patients is to label entire courses of medication with the patient randomization number in

advance and then ship all of that drug to sites in blocks (usually four-patient blocks at a minimum) — in advance of seeing

any patients at all. This method is incredibly wasteful compared to the IVRS paradigm of using patient interchangeable

supply, which is shipped to the sites just in time for patient consumption.

13

The

Trut

h A

bout

For

ecas

ting

Clin

ical

Tria

l Sup

plie

s P

art

II

The IVRS vs. non-IVRS debate being long concluded, the next challenge was for IVRS to be able to supply trials efficiently.

For example, if the IVRS knows about all subjects in the world and knows what treatment groups that they are in and what

their upcoming visit schedule will require for dispensing, then it should be possible to pre-position patient kits (dispensing

units) for each patient at each site. Add to this a small buffer for breakage or loss; in theory it is possible to have an

efficient site supply/resupply algorithm that will perform considerably better than a simple buffer or floor/ceiling algorithm.

However, most trials have at least some element of unpredictability. Many are centrally or dynamically randomized, which

means that the treatment required by a patient is not known until moments before it is required (at the randomization

event). Others have elements of titration, variable dosing and variable visit schedules.

Some IVRS’ address the unpredictability via predict-all-options (which is a worst case scenario) or via probability based

resupply. The most advanced IVRS resupply algorithms mitigate this uncertainty by adding buffers or floor/ceiling values

to the predictive calculations for known future demand.

All of these methods of figuring out how much to send to sites in advance are controlled by parameters: from the very

simple (buffer, floor/ceiling) to the very complex (look-ahead time windows, trigger windows, and patient counting rules).

Before we move onto the next concept, keep in mind:

• IVRS resupply algorithms are complex and controlled by a sets of parameters

• IVRS resupply algorithms are poorly understood

• IVRS resupply algorithms is where significant supply savings is realized

14

The

Trut

h A

bout

For

ecas

ting

Clin

ical

Tria

l Sup

plie

s P

art

II

Concept 4: Parameter Driven SystemsA “Parameter Driven” system is a system that can be configured by an administrator to perform all of the required functions

via user interface (UI) controls (e.g. switches, knobs, lists, and dropdown menus).

By contrast, a traditional IVRS system needs to have a specification written according to the protocol design. The specification

is then given to developers to program the system. The custom built system is then given to testers to validate the system. Since

each clinical trial is different and because fully featured parameter driven systems are very difficult to create, most IVRS systems

are traditional programmed systems.

Some more modern IVRS systems rely upon validated libraries of functions and therefore are only partially programmed —

“partially parameter driven.”

The next-generation IWR/IVRS are fully parameter driven — for example, all aspects of the system may be configured by a non-

programmer resource, using features of the system itself in order to complete the setup of a clinical trial.

The primary limitation of parameter driven systems is that it is not practical to build parameters for every possible trial design.

The parameters will satisfy 80% of most protocols. The last 20% is best addressed by extending IVRS via pluggable custom

code.

Pluggable custom code is one modern solution to the limitations of parameter driven systems. Code plugins allow small pieces of

programmed functionality to be inserted into a system to perform very specific functions of a study, which could not be realized

via parameters.

Study-specific functions, such as sending a specific shipment based on an arbitrary trigger or performing a logical operation on

data which derives some other operation (such as a dosing or stratification decision), are examples of custom code plugins. In

Concept 5: Pluggable Code

15

The

Trut

h A

bout

For

ecas

ting

Clin

ical

Tria

l Sup

plie

s P

art

II

addition to enhancing existing functions, a custom code plugin can also add novel functions like trial adaptation or cold chain

temp tales.

The important point with plugins is that the amount of specification, programming and testing is significantly minimized because

they are small isolated bits of code.