Simulation and Analysis of the Blood CoagulationCascade Accelerated on GPU

Matteo Bellini†, Daniela Besozzi†¶∗∗, Paolo Cazzaniga‡¶∗∗, Giancarlo Mauri§¶ and Marco S. Nobile§¶†Universita degli Studi di Milano – Dipartimento di Informatica, 20135 Milano, Italy

‡Universita degli Studi di Bergamo – Dipartimento di Scienze Umane e Sociali, 24129 Bergamo, Italy§Universita degli Studi di Milano-Bicocca – Dipartimento di Informatica, Sistemistica e Comunicazione, 20126 Milano, Italy

¶SYSBIO Centre for Systems Biology, 20126 Milano, Italy∗∗Istituto di Analisi dei Sistemi ed Informatica “Antonio Ruberti”, Consiglio Nazionale delle Ricerche, 00185 Roma, Italy

Abstract—The use of Graphics Processing Units (GPUs) hasrecently witnessed ever growing applications for different com-putational analyses in the field of Life Sciences. In this work wepresent a CUDA-powered computational tool, named coagSODA,that was purposely developed and applied for the analysis of alarge model of the blood coagulation cascade defined as a systemof ordinary differential equations, based on both mass-actionkinetics and Hill functions. We discuss the biological results ofthe parameter sweep analyses of this model, and show that GPUscan boost the computational performances up to 177× speedup.

I. INTRODUCTION

Since the introduction of general-purpose Graphics Pro-cessing Units (GPUs) and of CUDA, the Nvidia’s GPUprogramming language, the adoption of graphics engines ex-perienced a great boost in scientific applications related toBioinformatics, Systems Biology and Computational Biology(see [1], [2] and references therein). Indeed, the use of GPUsfor in silico investigations of complex biological systems –where Central Processing Units (CPUs) traditionally repre-sented the standard workhorses – is motivated by the need ofperforming large numbers of simulations to analyze the systembehavior in different conditions, which necessitate a computingpower that usually overtakes the capability of standard desktopcomputers, therefore requiring high-performance computingsolutions. Despite the considerable advantage in terms ofthe computational speedup, scientific applications on GPUsrisk to remain a niche for few specialists, since GPU-basedprogramming substantially differs from CPU-based program-ming and, therefore, GPU computing requires the developmentand the implementation of ad hoc algorithms. To avoid suchlimitations, several packages and software tools were recentlyreleased (see, e.g., [2], [3], [4]), so that also users with noknowledge of GPUs hardware and programming can access thecomputing power of graphics engines. In this work we presentcoagSODA, a simulator based on the GPU-powered tool thatwe previously developed, called cupSODA [5], which allowsto run deterministic simulations of a given mass-action basedsystem of biochemical reactions, using the LSODA algorithm[6]. coagSODA was specifically designed for the analysis ofthe blood coagulation cascade (BCC).

Blood is an essential component in human life and is thesubject of an intense scientific research, thanks to its keyrole in making diagnosis of numerous diseases [7]. All bloodcomponents are kept within appropriate concentration ranges

by means of fine-tuned regulatory mechanisms, ruled by sev-eral feedback controls; the preservation of blood compositionis maintained thanks to the circulation through an intricatenetwork of vessels. In particular, humans have evolved a com-plex hemostatic system that, under physiological conditions, isable to maintain blood in a fluid state; however, in responseto any vascular injury, this system is able to rapidly reactand seal the defects in the vessels wall in order to stop theblood leakage [8]. These regulation processes are governed bymeans of the BCC, a complex network of cellular reactionswhich, under physiological conditions in vivo, are inhibitedby the presence of intact endothelium. In order to allow bloodcoagulation, in humans there exist 13 blood clotting proteins,called coagulation factors, which are usually designated byRoman numerals I through XIII. Following a vascular injury,platelets become active and the tissue factor (TF, or factor III)is exposed in the subendothelial tissue, starting the BCC. Theultimate goal of BCC is to convert prothrombin (factor II)into thrombin (factor IIa - that is, the activated factor II), theenzyme that catalyzes the formation of a clot. Traditionally,the BCC is divided into the extrinsic and intrinsic pathways,both leading to the activation of factor X. The last part ofthe cascade, downstream of this factor, is called the commonpathway and leads to the formation of fibrin monomers, whosepolymers finally constitute the backbone of the clot. Excludingthrombin, all the enzymes involved in blood clotting arecharacterized by a low activity, which increases upon bindingto a specific protein cofactor (e.g., factors V and VIII) or toappropriate phospholipid surfaces (e.g., the plasma membranesof active platelets). In BCC pathways there exist also inhibitoryfactors, which limit the activity of the various active proteasesand allow to regulate the whole reactions cascade. Whenthe hemostatic system is unregulated, thrombosis (i.e., theformation of a blood clot obstructing the blood flow in vessels)may occur due to an impairment in the inhibitory pathway, orbecause the functioning of the natural anticoagulant processesis overwhelmed by the strength of the hemostatic stimulus [8].

In order to investigate the individuals’ variations in bloodcoagulation components and the corresponding response toperturbed conditions, we consider here a Systems Biologyperspective to analyze the alterations (prolongation or reduc-tion) of the time necessary to form the clot (i.e., the clottingtime). Numerous mathematical models of blood coagulationhave been developed in the last years, as they represent auseful tool for systematic studies of the intricate structureof the coagulation cascade and allow to obtain a suitable

2014 22nd Euromicro International Conference on Parallel, Distributed, and Network-Based Processing

1066-6192/14 $31.00 © 2014 IEEE

DOI 10.1109/PDP.2014.52

590

reconstruction of empirical observations (see, e.g., [9], [10],[11]). Here, we exploit a mathematical model of the BCC [12],built upon a previous model [13], that describes the intrinsic,extrinsic and common pathways and, more importantly, itaccounts for the platelet activation, as well as the presence ofseveral inhibitors. In particular, the model we consider in thiswork is a slightly reduced version of the deterministic modelgiven in [12], where we include only the reactions modelingthe coagulation in vivo until fibrin formation, excluding invitro species and a small set of reactions downstream of theclot formation – that is, the interactions between the fibrinpolymers, thrombin and antithrombin III – since they do nothave any effect on the clotting time. This model consists ina system of ordinary differential equations (ODEs) based onmass-action kinetics, except for 14 reactions that are influencedby specifically defined Hill functions, which describe thephysiological levels of thrombin concentration as a functionof platelet activation, as thoroughly explained in [12].

coagSODA, the GPU-accelerated simulator that we presentand exploit in this work for the analysis of the BCC model,is a user-friendly and efficient tool that circumvents the needof manually defining the system of ODEs that describe theblood coagulation network. More precisely, coagSODA is ableto automatically derive the system of (mass-action and Hillfunction-based) ODEs – and then to perform their numericalintegration – starting from the given set of 101 biochemicalreactions, which fully describe the molecular interactions be-tween all the species involved in the BCC.

II. SIMULATION OF THE BCC MODEL ON GPUS

Compute Unified Device Architecture (CUDA) is a com-puting platform and programming model that provides pro-grammers with a framework to exploit the parallel computingcapabilities of modern GPUs in general-purpose computationaltasks. CUDA gives access to tera-scale computing on commonworkstations, but it requires specialized skills to fully exploitthe peculiar architecture of GPUs. In particular, CUDA relieson a hierarchy of memories, all having different dimensionsand access latencies. To achieve the best performances, thecode must be optimized to fully exploit the most efficientresources.

The systematic analysis of models of biological systemsoften consists in the execution of large batches of simulations;for instance, one of the standard investigation regards an in-tensive exploration of the multidimensional parameters space.In order to reduce the corresponding computational burden,we previously implemented on the CUDA architecture one ofthe most efficient numerical integration algorithms for ODEs,LSODA [6]. This GPU-powered tool is called cupSODA [5],it exploits CUDA’s massive parallelism to execute differentand independent simulations in each thread, thus reducing thecomputational time required by a standard CPU counterpart ofLSODA. cupSODA was conceived as a black-box simulator,to be easily used without any programming skills: it consists ina tool that automatically converts a mechanistic reaction-basedmodel of a biological system into the corresponding system ofODEs, and that directly encodes the derived system – alongwith the corresponding Jacobian matrix – as C arrays.

This fully automatic methodology can be exploited for thesimulation of models whose chemical kinetics is based on the

mass-action law. The BCC model that we investigate in thispaper, though, includes a set of reactions whose kinetics isdescribed by Hill functions. In particular, the platelet activityis realized by modulating the rate of the dissociation of thecomplexes formed on a platelet’s surface by means of aspecific factor ε, which is calculated with a special equationduring the integration steps (see [12] for details). For thisreason, coagSODA was purposely designed as an extensionof cupSODA in order to consider the run-time computation ofthe specific kinetics of this set of reactions.

The accesses to the global memory and the computationalcosts due to the additional calculations required by the BCCmodel slow down the integration process, with respect to theintegration of ODEs performed according to a strictly mass-action based kinetics (as in the cupSODA simulator). Never-theless, in Section III we show that our parallel implementationlargely outperforms a sequential counterpart.

III. RESULTS

In this section we discuss the results of the parametersweep analysis (PSA) carried out on the BCC model, to inves-tigate the prolongation and the reduction of the clotting timein response to perturbed values of some reaction constants,chosen according to their meaning within the whole pathway.PSA was performed by generating a set of different initialconditions – corresponding to different parameterizations ofthe model – and then automatically executing the deterministicsimulations with coagSODA. The sweep analysis for singleparameters (PSA-1D) was performed considering a logarithmicsampling of numerical values of each reaction constant withina specified range (with respect to its physiological referencevalue). The sweep analysis over pairs of parameters (PSA-2D) was performed by simultaneously varying the values oftwo reaction constants within a specified range, considering alogarithmic sampling on the resulting lattice.

The output of the PSA is the prothrombin time (PT) –generally used in laboratory tests for monitoring the therapywith anticoagulant drugs – which is defined as the timenecessary to convert the 70% of the fibrinogen into fibrin,conventionally assumed to correspond to the time necessary toform the clot [14]. According to the model defined in [12], thePT in vivo is around 300 seconds in physiological conditions.We investigate here the response of the BCC by evaluating theclotting time in different conditions – namely, we carry outPSA-1D over the constant of two reactions (hereafter calledreactions α and β) taken separately, and PSA-2D over the twoconstants together – in order to monitor both their individualeffects and any possible coupled effect on the robustness ofthe BCC model. In each PSA, the value of the constants(hereafter denoted as kα and kβ) was varied over six orders ofmagnitude with respect to their physiological values, that is,three below and three above the reference value. Reaction α,which describes the catalytical activation of factor V by factorIIa (IIa + V → IIa + Va), was chosen because it represents themain positive feedback within the common pathway; reactionβ, which is involved in the activation of factor XI by binding tofactor IIa (IIa + XI → IIa-XI), was chosen because it representsthe main positive feedback in the intrinsic pathway of the BCCmodel. Moreover, preliminary PSA over all reaction constants

591

280

300

320

340

360

380

400

1e+04 1e+05 1e+06 1e+07 1e+08 1e+09 1e+10

Pro

thro

mb

in T

ime [

s]

kα

[M-1

s-1

]

310

320

330

340

350

360

370

1e+05 1e+06 1e+07 1e+08 1e+09 1e+10 1e+11

Pro

thro

mb

in T

ime [

s]

kβ [M

-1s-1

]

Fig. 1. Variation of prothrombin time (PT) according to PSA-1D over constantkα of reaction α : IIa + V → IIa + Va (top) and over constant kβ of reactionβ : IIa + XI → IIa-XI (bottom). The sweep ranges of kα and kβ are [2 ·104, 2 · 1010] and [1 · 105, 1 · 1011], respectively.

of the BCC model evidenced that these two reactions areamong the most relevant steps of the coagulation network.

The PSA-1D over kα shows that the PT is very sensitive tothe perturbation of this reaction, when the physiological valueof its constant (i.e., kα = 2·107 M−1s−1) is either increased ordecreased; in particular, when kα is very low, a plateau in PT isreached since the strength of the positive feedback exerted byfactor IIa is largely reduced, a condition where the contributionof the amplification of the hemostatic stimulus (due by thecommon pathway) to the formation of the clot is basicallynot effective (Figure 1, top). On the other side, the PSA-1Dover kβ shows that while a decrease of the constant from itsphysiological value (i.e., kβ = 1 · 108 M−1s−1) does not havea substantial effect on the clotting time, an increase of its valueleads to a progressive increase in the PT (Figure 1, bottom).This increase is due to the fact that factor IIa is sequestered inthe formation of a complex with factor XI, and hence it is nolonger available as a free component in blood to participatein other reactions, especially those reactions of the extrinsicpathway which principally lead to the clot formation in vivo.This behavior demonstrates that the intrinsic pathway has asecondary role in blood coagulation in vivo, compared with theextrinsic pathway, as also evidenced by various experimentalobservations [15].

Figure 2 shows the effect of the simultaneous variation

1e+041e+05

1e+061e+07

1e+081e+09

1e+101e+11 1e+05

1e+06

1e+07

1e+08

1e+09

1e+10

1e+11

280

300

320

340

360

380

400

420

440

Pro

thro

mb

in T

ime [

s]

kα

kβ

Pro

thro

mb

in T

ime [

s]

280

300

320

340

360

380

400

420

440

Fig. 2. Variation of prothrombin time (PT) according to PSA-2D over reactionconstants kα and kβ .

of kα and kβ (over the same sweep ranges considered inthe two PSA-1D). This result remarks a synergistic interplaybetween the two analyzed reactions, especially when the valueof constant kα is low and constant kβ is high, since inthis condition both the extrinsic and common pathways areinhibited and the PT is higher with respect to the values itreaches when only a single constant is changed (see Figure1). The anticoagulant effect achieved in this condition is dueto the lack of the positive feedback of factor IIa, especiallyon factor V, which results in a reduced conversion rate of thefibrinogen into fibrin.

To the aim of showing the relevant speedup achieved bycoagSODA, we consider the computational effort required byGPU and CPU for the simulation of the BCC model. Theperformances of our GPU simulator are compared with thoseobtained using the LSODA algorithm implemented in thesoftware COPASI [16], by executing on the CPU the same setof simulations that were run on the GPU. In all simulations,executed both on GPU and CPU, we stored 100 samples –uniformly distributed in the time interval considered for eachsimulation, i.e., [0, 700] seconds – of the dynamics of allchemical species involved in the BCC model. The settingsfor the LSODA algorithm were the following: relative error1 ·10−7, absolute error 1 ·10−13, maximum number of internalsteps set to 20000. The GPU used for the simulation is a NvidiaTesla K20c, with 2496 cores, equipped with 5 GB of DDR5RAM. The performance of the GPU were compared against apersonal computer equipped with a dual-core CPU Intel Corei5, frequency 2.3 GHz, 4 GB of DDR3 RAM, running theoperating system Mac OS X Lion 10.7.5.

In Table I we show the comparison of the running timesrequired to run several batches of simulations, executed tocarry out the PSA-1D over the reaction constant kα. Resultsclearly show that coagSODA always performs better than theCPU counterpart and, in particular, while the CPU performancelinearly increases with the number of simulations, the runningtimes are strongly reduced on the GPU. Indeed, the differentrunning times are due to the effort to manage the executionof blocks of threads on the GPU, as well as to the differentparameterizations used in the PSA, both factors that can greatly

592

TABLE I. CPU VS GPU PERFORMANCE COMPARISON

Number of simulations CPU time (sec) GPU time (sec) Speedup

10 1060.4 120.81 8.7 ×100 11652.8 902.12 12.9 ×1000 107358.2 2070.89 51.8 ×

10000 1069981.8* 6027.69 177.5 ×* Estimated running time

affect the computational performances. Table I also highlightsthat the advantage of exploiting the GPUs for the simulationof the BCC model becomes more evident as the number ofsimulations increases, with a 177× speedup when a PSA with10000 different parameterizations is executed.

IV. CONCLUSION

Thanks to the high-performance computing capabilities andto the very low costs, GPUs nowadays represent a suitabletechnology for the development and the application of parallelcomputational methods to analyze the emergent behaviors ofcomplex biological systems in different conditions. However,the implementation of efficient computational tools able tofully exploit the large potentiality of GPUs is still challenging,since good programming skills are required to implementGPU-based algorithmic methods, and to handle specific fea-tures of GPU computing, as the efficient usage of memoryor the communication bandwidth between GPU and CPU.Moreover, algorithms cannot be directly ported on the GPUbecause of the limited programming capabilities allowed byGPU kernels; as a matter of fact, they need to be redesignedto meet the requirements of the underlying SIMD architecture.

In this work we presented coagSODA, a GPU-poweredsimulator specifically developed for the efficient simulationof the BCC model firstly introduced in [12]. coagSODA wasdesigned to offer a black-box solution usable by any finaluser in an easy way. It relies on cupSODA, a numericalintegrator for ODEs that we previously implemented for theGPU architecture, based on the LSODA algorithm and capableof automatically translating a reaction-based model into a setof coupled ODEs. coagSODA implements specific functions tocompute the kinetics of mass-action and Hill function basedreactions, such as those involved in the platelets activity ofthe BCC model. coagSODA exploits the massive parallelcapabilities of modern GPUs, and our results demonstratedthat it can achieve a relevant reduction of the computationaltime required to execute many concurrent and independentsimulations of the BCC dynamics. The mutual independenceof the simulations allows to fully exploit the underlying SIMDarchitecture; moreover, coagSODA benefits from an additionalspeedup, thanks to our choice of storing each system state intothe low-latency shared memory (a solution that was alreadyimplemented in cupSODA). Since the BCC model is large(101 reactions, 81 chemical species), a large amount of sharedmemory was assigned to each thread, strongly reducing thetheoretical occupancy of the GPU, i.e., the ratio of active warpswith respect to the maximum number of warps supportedby each streaming multi-processor of the GPU. However,the results of the analysis presented in this work, performedon the BCC model, show that coagSODA allows a relevant

boost with respect to a reference CPU implementation. Ourresults indicate that, for biological models consisting in manyreactions and many species, our GPU implementation becomesmore profitable than the CPU counterpart as the number ofconcurrent simulations increases, making it suitable especiallywhen performing demanding computational analysis such as,e.g., parameter sweep, parameter estimation or sensitivity anal-ysis. Indeed, the GPU accelerated analysis of the BCC modelwith coagSODA represents a novel and relevant computationalmean to investigate the behavior of this complex biologicalsystem under non-physiological conditions, and could be ex-ploited to efficiently determine the response of the BCC todifferent therapeutical drugs.

The coagSODA software is available from the authors uponrequest.

REFERENCES

[1] L. Dematte and D. Prandi, “GPU computing for systems biology,”Briefings in Bioinformatics, vol. 11, no. 3, pp. 323–33, 2010.

[2] M. J. Harvey and G. De Fabritiis, “A survey of computational molec-ular science using graphics processing units,” Wiley InterdisciplinaryReviews: Computational Molecular Science, vol. 2, no. 5, pp. 734–742,2012.

[3] Y. Zhou, J. Liepe, X. Sheng, M. P. H. Stumpf, and C. Barnes, “GPUaccelerated biochemical network simulation,” Bioinformatics, vol. 27,no. 6, pp. 874–876, 2011.

[4] M. Vigelius, A. Lane, and B. Meyer, “Accelerating reaction-diffusionsimulations with general-purpose graphics processing units,” Bioinfor-matics, vol. 27, no. 2, pp. 288–290, 2011.

[5] M. S. Nobile, D. Besozzi, P. Cazzaniga, G. Mauri, and D. Pescini,“cupSODA: a CUDA-powered simulator of mass-action kinetics,” inProceedings of 12th International Conference on Parallel ComputingTechnologies (PaCT 2013), V. Malyshkin, Ed., vol. LNCS 7979, 2013,pp. 344–357.

[6] L. Petzold, “Automatic selection of methods for solving stiff andnonstiff systems of ordinary differential equations,” SIAM Journal ofScientific and Statistical Computing, vol. 4, no. 1, pp. 136–148, 1983.

[7] B. E. Publishing and K. Rogers, Blood: Physiology and Circulation,ser. The Human Body. Britannica Educational Pub., 2010.

[8] R. Colman, J. Hirsh, V. Marder, A. Clowes, and J. George, Eds.,Hemostasis and thrombosis: basic principles and clinical practice.Lippincott Williams & Wilkins, 2006.

[9] T. Wajima, G. Isbister, and S. Duffull, “A comprehensive model forthe humoral coagulation network in humans,” Clinical Pharmacology& Therapeutics, vol. 86, no. 3, pp. 290–298, 2009.

[10] K. Lo, W. S. Denney, and S. L. Diamond, “Stochastic modelingof blood coagulation initiation,” Pathophysiology of Haemostasis andThrombosis, vol. 34, no. 2-3, pp. 80–90, 2006.

[11] C. M. Danforth, T. Orfeo, K. G. Mann, K. E. Brummel-Ziedins, andS. J. Everse, “The impact of uncertainty in a blood coagulation model,”Mathematical Medicine and Biology, vol. 26, no. 4, pp. 323–336, 2009.

[12] M. S. Chatterjee, W. S. Denney, H. Jing, and S. L. Diamond, “Systemsbiology of coagulation initiation: Kinetics of thrombin generation inresting and activated human blood,” PLoS Computational Biology,vol. 6, no. 9, p. e1000950, 2010.

[13] M. F. Hockin, K. C. Jones, S. J. Everse, and K. G. Mann, “A modelfor the stoichiometric regulation of blood coagulation,” Journal ofBiological Chemistry, vol. 277, no. 21, pp. 18 322–18 333, 2002.

[14] B. H. Esteridge, A. P. Reynolds, and N. J. Walters, Basic medicallaboratory techniques, 4th ed. Cengage Learning, 2000.

[15] T. Renne, A. H. Schmaier, K. F. Nickel, M. Blomback, and C. Maas,“In vivo roles of factor XII,” Blood, vol. 120, no. 22, pp. 4296–4303,2012.

[16] S. Hoops, S. Sahle, R. Gauges, C. Lee, J. Pahle, N. Simus, M. Singhal,L. Xu, P. Mendes, and U. Kummer, “COPASI - a COmplex PAthwaySImulator,” Bioinformatics, vol. 22, no. 24, pp. 3067–3074, 2006.

593