A joint model of unpaired data from scRNA-seq and spatial transcriptomics forimputing missing gene expression measurements

Romain Lopez 1 * Achille Nazaret 1 2 * Maxime Langevin 1 2 * Jules Samaran 1 3 * Jeffrey Regier 1 *

Michael I. Jordan 1 Nir Yosef 1

Spatial studies of transcriptomes provide biologists withgene expression maps of heterogeneous and complex tis-sues such as the mouse nervous system (Zeisel et al., 2018).A wide array of experimental protocols exist based oneither single-molecule fluorescence in-situ hybridisation(smFISH) (Shah et al., 2016; Codeluppi et al., 2018), acombination of imaging and sequencing (starMAP) (Wanget al., 2018), or high-resolution RNA sequencing in-situ(e.g., slide-seq) (Rodriques et al., 2019). The first two tech-niques have a high degree of sensitivity (Codeluppi et al.,2018) but suffer from the need of a priori selecting a smallfraction of genes to be quantified out of the entire transcrip-tome (varying from fifty for smFISH to a few hundreds forstarMAP). The third technique is not limited to a predeter-mined subset of genes and in principle can capture any gene(out of thousands in the transcriptome). However, the num-ber of genes efficiently captured in slide-seq measurementsis substantially lower than what is obtained with standard(i.e., non-spatial) single-cell RNA sequencing (scRNA-seq),which is also more prevalent and easier to implement (Kleinet al., 2015; Saunders et al., 2018).

As a consequence, an important research problem is to inte-grate spatial transcriptomics data with scRNA-seq measure-ments, in order to either transfer cell-type annotation fromscRNA-seq to spatial datapoints (Welch et al., 2018; Zhuet al., 2018), impute gene missing in the spatial assay (Stuartet al., 2018) or to approximate the physical location in atissue of cells measured with standard scRNA-seq (Satijaet al., 2015). State-of-the-art methods focus on embed-ding both spatial and standard datasets into a latent space—using matrix factorization techniques (Liger and Seurat An-chors) (Welch et al., 2018; Stuart et al., 2018)— which isthen corrected via mutual nearest neighbours (Haghverdiet al., 2018) or quantile normalization. However, because(i) the embedding step relies on a linear model of the data

1Department of Electrical Engineering and Computer Sci-ences, UC Berkeley, USA 2Centre de Mathématiques Ap-pliquées, École polytechnique, Palaiseau, France 3École des Minesde Paris, France . Correspondence to: Romain Lopez <ro-main_lopez@berkeley.edu>, Nir Yosef <niryosef@berkeley.edu>.

though there is no basis for supposing linearity and (ii) thealignment is executed ad hoc, such methods will have ahigher chance of overlaying samples who show little bi-ological resemblance. In this manuscript, we focus onthe problem of imputing missing genes in spatial transcrip-tomics data based on (unpaired) standard scRNA-seq datafrom the same biological tissue. Notably, our problem isrelated to domain adaptation methods (CORAL) (Sun et al.,2016) as well as unpaired image-to-image translation (Zhuet al., 2017), which has been applied to genomic data (MA-GAN) (Amodio & Krishnaswamy, 2018).

Since a certain fraction of the features (i.e., genes) is presentin both datasets, we propose building up on recent advancesin the field of domain adaptation (Ganin et al., 2017) and in-troduce gene imputation with Variational Inference (gimVI),a deep generative model for integrating spatial transcrip-tomics data and scRNA-seq data which can be used to im-pute missing genes. gimVI is based on scVI (Lopez et al.,2018), with a different architecture as well as an alternativechoice of conditional distributions to better take into accountthe technology-specific covariate shift.

After describing our generative model (Section 1) and aninference procedure for it (Section 2), we compare gimVIto alternative methods on real datasets (Section 3). Oursource code, based on PyTorch, is publicly available athttps://github.com/YosefLab/scVI.

1. The gimVI probabilistic modelFigure 1 represents the probabilistic model graphically.

Shared biology For each cell n, binary variable sn speci-fies whether it was captured by the scRNA-seq or the spatialexperimental protocol (such a framework can be extended tohandle multiple datasets). Conditioned on nuisance variablesn, we treat each cell as an independent replicate from thefollowing generative process. Latent variable

zn ∼ N (0, Id) (1)

is a low-dimensional random vector describing cell n. Sucha latent variable is commonly interpreted as cell type orcell identity and can be used for downstream analysis suchas clustering or visualization. Let G be the set of genes

arX

iv:1

905.

0226

9v1

[cs

.LG

] 6

May

201

9

A joint model of unpaired data from scRNA-seq and spatial transcriptomics

captured by the scRNA-seq experiment. We assume G isa superset of G′, the set of genes captured by the spatialexperiment. Let fη be a neural network with parameters ηand a softmax non-linearity on the last layer. Variable

ρn = fη(zn, sn) (2)

is a value on the probability simplex. It can be interpretedas the expected normalized frequencies of each gene g ofa individual cell. This intermediate value has a biologicalinterpretation and can be useful for downstream analysissuch as imputation and detecting differentially expressedgenes (Lopez et al., 2018). In this manuscript we focus onimputation, leaving hypothesis testing across modalities forfuture research.

scRNA-seq measurements Let µn, σn ∈ R2+

be the em-pirical mean and variance of the log-library size, fixed beforeinference. Latent variable

`n ∼ LogNormal(µn, σ2n) (3)

represents a scalar nuisance factor for the scRNA-seq cellswhich corresponds to discrepancies of sequencing depth andcapture efficiency. Let fν be a neural network with parame-ters ν. Let θ ∈ R∣G∣

+denote a vector of gene-specific inverse

dispersion parameters, estimated via variational Bayesianinference. For each gene g ∈ G, we treat each observedgene expression level xng as independent conditionally on{`n, zn, sn} and sampled from an overdispersed count con-ditional distribution:

xng ∼⎧⎪⎪⎪⎨⎪⎪⎪⎩

ZINB(`nρng, θg, fνg (zn, sn)),or

NB(`nρng, θg),(4)

which can be dataset specific (Svensson, 2019). In thefollowing experiments, we will use a zero-inflated negativebinomial (ZINB) conditional distribution.

Spatial measurements For this part of the model, onlythe genes in G′ are captured. We therefore re-normalize ourexpected frequencies as

∀g′ ∈ G′, ρ′ng′ =ρng′

∑g∈G ρng. (5)

As we expect to see little to no technical variation becauseof size effects in spatial data (especially imaging based tech-nologies), we use observed random variable `′n to representthe number of transcripts in a given cell, which originatedfrom the genes included in the assay. Let θ′ ∈ R∣G∣

′

+de-

note a vector of gene-specific inverse dispersion parameters,estimated as before. For each gene g′ ∈ G′, we also treateach observed gene expression level x′ng′ as independentconditionally on {`n, zn, sn}. However, we model x′ng′ as

x′ng′ ∼ {Poisson(`′nρ′ng′), for smFISHNB(`′nρ′ng′ , θ′g′), for starMAP. (6)

Such a choice of distribution is method specific and de-pends on the properties of the protocol. Notably, osm-FISH (Codeluppi et al., 2018) is reported to have a nearlyperfect sensitivity (i.e., capturing molecules originatingfrom the genes included in the assay), which allows us tomodel it with a flat mean-variance relationship distributionsuch as Poisson.

xng x′ng

zn

sn

`n `′n

G′FISHGRNA

NCells

Figure 1. The proposed graphical model. Shaded vertices representobserved random variables. Empty vertices represent latent ran-dom variables. Edges signify conditional dependency. Rectangles(“plates”) represent independent replication.

2. Posterior inferenceBayesian inference aims at optimizing the data likelihoodunder the given model. Let Θ = {η, ν, θ, θ′} denote theparameters of our generative model. The likelihood of theobserved data decomposes as

N

∑n=1sn=0

log pΘ (xn ∣ sn) +N

∑n=1sn=1

log pΘ (x′n ∣ sn, `′n) . (7)

As exact posterior inference is intractable, we use insteadvariational inference (Kingma & Welling, 2014) to approx-imate the posterior distributions. Our variational distri-butions qφ(z, ` ∣ x, s) = qφ(z ∣ x, s)qφ(` ∣ x, s) andqψ(z ∣ x′, s) are Gaussian with diagonal covariance ma-trices, parameterized by neural networks. The last layer isshared between the encoder networks for qφ(z ∣ x, s) andqψ(z ∣ x′, s), which helps learning a joint latent space forthe multiple modalities. The modality-specific variationallower bounds are

log pΘ(x ∣ s) ≥ Eqφ(z,`∣x,s) log pΘ(x ∣ z, s, `)−DKL(qφ(` ∣ x, s) ∣∣ p(`))−DKL(qφ(z ∣ x, s) ∣∣ p(z)),

(8)

log pΘ(x′ ∣ `′, s) ≥ Eqψ(z∣x′,s) log pΘ(x′ ∣ z, s, `′)−DKL(qψ(z ∣ x′, s) ∣∣ p(z)).

(9)

We maximize these lower bounds using the stochastic back-propagation algorithm.

A joint model of unpaired data from scRNA-seq and spatial transcriptomics

Missing gene imputation Let us derive a condition forwhich we can accurately impute a given gene g ∈ G − G′ fora given spatial measurement x′. To this end, we would liketo sample z from the posterior distribution pΘ(z ∣ x′, s = 1)and use neural network fηg to impute the counterfactualgene expression x∗g(z). Let pspa(z) (resp. pseq(z)) denotethe aggregated posterior for the spatial data (resp. scRNA-seq data). From domain adaptation theory (Blitzer et al.,2008), for a given bounded loss function L, there exists aconstant κ such that the following bound holds

Epspa(z)L (fηg (z), x∗g(z)) ≤Epseq(z)L (fηg (z), x∗g(z))+ κdH(pspa(z), pseq(z)),

(10)

where dH designates the H-divergence between the twolatent spaces, and can be approximated by the loss of an ad-versarial classifier (Ganin et al., 2017). This result explainsthat imputing a missing gene is possible whenever (i) sucha gene is well fitted in the scRNA-seq data model and (ii)when the samples mix well in latent space.

3. Performance benchmarksTo assess the performance of gimVI, we will present thestatistical trade-offs for integrating cells into a joint latentspace (Section 3.1) and then benchmark our method forimputing missing genes (Section 3.2). Throughout, wecompare gimVI (for different values of κ) with vanilla scVI,Liger and Seurat. Only gimVI is able to make use of theheld-out genes in G when learning the latent space — akey advantage relative to other methods which focus onthe gene set G′ and only uses the complementary genes fordownstream analyses like imputation. For benchmarkingpurposes, we run all the methods with d = 10 (dimension ofthe latent space).

We apply gimVI on two pairs of real datasets. First, we usea scRNA-seq dataset of 3,005 mouse somatosensory cortexcells (Zeisel et al., 2015) and a osmFISH dataset of 4,462cells and 33 genes from the same tissue (Codeluppi et al.,2018) (referred to as mSMS). Second, we use a scRNA-seqdataset of 71,639 mouse pre-frontal cortex cells (Saunderset al., 2018) and a starMAP dataset of 3,704 cells and 166genes from the same tissue (Wang et al., 2018) (referred toas mPFC). For each pair of datasets, we hold out 20% of thegenes in G′ and define G as G′ with the addition of the held-out genes. In future work, we will investigate the model’srobustness for wider ranges of ratio G′/G. Since Liger throwsa runtime error (probably a memory issue) on a Intel i7-4500U addressing 8GB of RAM while running on the mPFCpair, we randomly subsampled the corresponding scRNA-seq dataset to 15,000 cells for benchmarking purposes. Weran scVI and gimVI on a NVIDIA Tesla K80 GPU. For allthe algorithms, fitting the data took less than a few minutes.

3.1. Integrating cells into a joint latent space

We assess our model’s ability to integrate information fromtwo drastically different datasets. First we compute the en-tropy of mixing (Haghverdi et al., 2018) to quantify howwell the algorithms integrate the datasets in the latent space.As the pairs of datasets are unbalanced, we use the negativeKL divergence between the k-nearest neighbors (k-NN) lo-cal sn distribution and the global sn distribution, which gen-eralizes the entropy of mixing for that setting. Because thismetric can be easily maximized by an algorithm that wouldignore the input, we propose to use the k-NN purity (Xuet al., 2019) to evaluate whether an algorithm would returna similar latent space either by integrating the data or onindividual datasets. For Seurat (resp. Liger, gimVI), we usePCA (resp. NMF, scVI) as the corresponding method forindividual datasets and compute a Jaccard index to measurethe overlap of the k-NN graphs. As such metrics coulddepend on the size of the neigborhood, we report these overa wide range of values for k in Figure 2.

0 2 4 6 8 10 k (% total cells)

0.1

0.2

0.3

0.4

0.5

0.6

Jacc

ard

inde

x

k-nearest neighbors purity

2 4 6 8 10 k (% total cells)

0.85

0.90

0.95

1.00

Nor

mal

ized

neg

ativ

e KL

Entropy of mixing

2 4 6 8 10 k (% total cells)

0.2

0.3

0.4

0.5

0.6

Jacc

ard

inde

x

k-nearest neighbors purity

2 4 6 8 10 12 k (% total cells)

0.70

0.75

0.80

0.85

0.90

0.95N

orm

aliz

ed n

egat

ive

KL

Entropy of mixing

SeuratLigerscVIgimVI κ = 1gimVI κ = 0

a) b)

c) d)

Figure 2. Integration metrics on the mPFC cells (a), (b) and themSMS cortex cells (c), (d).

Across the two pairs of datasets, Liger has the lowest k-NNpurity, suggesting that it may more significantly overlaysample of little biological resemblance compared to theother methods. Furthermore, gimVI has lower k-NN pu-rity than scVI, which is expected since gimVI also uses itslatent variables to impute genes which are unobserved inscVI. This behavior is exacerbated for higher values of κ(see the statistical trade-off described in Eq. 10). In termsof entropy of mixing, gimVI with κ = 1 outperforms allmethods. Notably, scVI has the second best median entropyof mixing for the mPFC cells, but the worse for the mSMScells, which may be due to the inadequacy of the ZINBdistribution to model osmFISH data. Overall, these resultssuggest that gimVI reaches the best trade-off (adjustablewith κ) between merging the datasets and keeping intact thebiological information.

A joint model of unpaired data from scRNA-seq and spatial transcriptomics

3.2. Imputing missing genes

Since certain genes are held-out for benchmarking purposes,we propose to impute them from the unpaired scRNA-seqdata and use the held-out information as groundtruth. AsLiger, and scVI do not explicitly model the unseen genes, weimpute their measurements using a k-NN regressor trainedon scRNA-seq data in the joint latent space— we selectedk = 5% number of cells. For Seurat, we used Seurat An-chors’ ad hoc method for imputation. Furthermore, becausespatial measurements and scRNA-seq have sensibly differ-ent distributions and gene-specific biases, we expect ourimputed values to carry these biases from scRNA-seq. Inparticular, we choose to evaluate our model using the Spear-man correlation ρ over cells for each gene instead of themean-square error, because the correct range of imputationmay be unidentifiable without more information about theprotocol-specific capture efficiencies. For this task, we alsoran CORAL and MAGAN. However, MAGAN returneduniformly random values, despite our efforts to train themodel (unshown). We report our results in Table 1.

mSMS mPFCρ̃ δ̃ρ ρ̃ δ̃ρ

Seurat 0.15 −57% 0.08 −55%Liger 0.22 −28% 0.09 −55%scVI 0.20 −36% 0.06 −65%CORAL 0.18 −38% 0.17 −15%gimVI κ = 1 0.30 −12% 0.22 −3%gimVI κ = 0 0.33 _ 0.22 _gimVI κ = κ∗ 0.37 +23% 0.22 +3%

Table 1. Imputation results. For each algorithm and datasets, wereport the median ρ̃ over Spearman correlations ρ and the medianδ̃ρ over per gene relative change δρ with respect to gimVI.

First, we observe that the median of ρ values are relativelylow for all the algorithms. This can be explained by the factthat G′ is a random subset of all the spatially transcribedgenes in our experiment: thus, some of the held-out genes inG may have small correlation with the genes in G′, makingthe imputation task significantly harder. To analyze the cor-relation distribution closely, we therefore report the medianover genes of the relative change δρ on ρ with respect togimVI. For both datasets, all versions of gimVI provides asignificant improvement over state-of-the-art methods. Forexample, in the mSMS dataset, gimVI κ = 0 has more than38 % relative improvement on ρ for more than half of theimputed genes compared to CORAL. Second, on the mSMSdataset κ = 1 in gimVI decreases the imputation perfor-mance compared to κ = 0. However, it is possible to findκ∗ ∈ (0,1) such that imputation is significantly improvedover both κ = 0 and κ = 1 (which is reflective of Eq 10).This result is less significant in the mPFC dataset, whichmay be indicative of the similarity between starMAP andscRNA-seq datasets. In future work, we will propose aprincipled algorithmic procedure to choose this parameter.

Another fundamental difference between gimVI and otherintegration methods such as Liger and Seurat is that gimVIis a generative probabilistic method, able to carry uncer-tainty along with the imputed values. We can thereforederive the confidence of our model from the variance of itspredictions. Comparatively, we computed for each gene theleast-square residuals of a linear regression predicting hid-den genes from observed genes on the scRNA-seq data only,to account for the “predictability” of the gene counts. Pre-cisely, the variance on our model’s prediction was computedby sampling fifty times from the variational posterior, andthe linear regression was performed after normalization andlog-transformation of the scRNA-seq data. We report ourresults for the mPFC cells in Figure 3 and observe that themodel is more uncertain for genes that are harder to predict.Similarly, gimVI has almost perfect predictions on valueshe is sure about (no variance). A next direction for researchwould be to detect which genes are worth imputing.

−2 −1 0 1Imputation log-relative-error

−8.0

−7.5

−7.0

−6.5

−6.0

−5.5

−5.0

Impu

tatio

n lo

g-va

rianc

e

0.0 0.5 1.0 1.5 2.0 2.5Least square residuals

−8.0

−7.5

−7.0

−6.5

−6.0

−5.5

−5.0

Impu

tatio

n lo

g-va

rianc

e

Figure 3. Imputation uncertainty of gimVI

A supplementary sanity check for the imputed values is tolook at them in the context of discovering spatial patterns.In particular, we investigate held-out gene Lamp5, a knownmarker gene for excitatory neurons and visualize them ac-cording to the cells’ locations on Figure 4 for the mSMSspatial dataset. Our result suggests that gimVI provides animputation spatially more coherent than its competitors whoguess high expression of this gene on wrong regions of thebrain (e.g., layer 6). In future work, we will systematizethis approach by extracting a curated list of positive andnegative control and relying on hypothesis testing procedureto detect such patterns (Svensson et al., 2018).

Groundtruth Liger

gimpVI Seurat

050100150200250300

020406080100120140

0

5

10

15

20

0

5

10

15

20

25

Figure 4. Discovering spatial motifs on imputed genes for themSMS spatial dataset

A joint model of unpaired data from scRNA-seq and spatial transcriptomics

ReferencesAmodio, M. and Krishnaswamy, S. MAGAN: Aligning

Biological Manifolds. In International Conference onMachine Learning, 2018.

Blitzer, J., Crammer, K., Kulesza, A., Pereira, F., and Wort-man, J. Learning Bounds for Domain Adaptation. InAdvances in Neural Information Processing Systems 20,2008.

Codeluppi, S., Borm, L. E., Zeisel, A., La Manno, G., vanLunteren, J. A., Svensson, C. I., and Linnarsson, S. Spa-tial organization of the somatosensory cortex revealed byosmFISH. Nature Methods, 2018.

Ganin, Y., Ustinova, E., Ajakan, H., Germain, P., Larochelle,H., Laviolette, F., Marchand, M., and Lempitsky, V.Domain-Adversarial Training of Neural Networks. InJournal of Machine Learning Research, 2017.

Haghverdi, L., Lun, A. T. L., Morgan, M. D., and Marioni,J. C. Batch effects in single-cell rna-sequencing data arecorrected by matching mutual nearest neighbors. NatureBiotechnology, 2018.

Kingma, D. P. and Welling, M. Auto-Encoding VariationalBayes. In International Conference on Learning Repre-sentations, 2014.

Klein, A. M., Mazutis, L., Akartuna, I., Tallapragada, N.,Veres, A., Li, V., Peshkin, L., Weitz, D. A., and Kirschner,M. W. Droplet barcoding for single-cell transcriptomicsapplied to embryonic stem cells. Cell, 2015.

Lopez, R., Regier, J., Cole, M. B., Jordan, M. I., and Yosef,N. Deep generative modeling for single-cell transcrip-tomics. Nature Methods, 2018.

Rodriques, S. G., Stickels, R. R., Goeva, A., Martin, C. A.,Murray, E., Vanderburg, C. R., Welch, J., Chen, L. M.,Chen, F., and Macosko, E. Z. Slide-seq: A scalabletechnology for measuring genome-wide expression athigh spatial resolution. Science, 2019.

Satija, R., Farrell, J. A., Gennert, D., Schier, A. F., andRegev, A. Spatial reconstruction of single-cell gene ex-pression data. Nature Biotechnology, 2015.

Saunders, A., Macosko, E., Wysoker, A., Goldman, M.,Krienen, F., de Rivera, H., Bien, E., Baum, M., Wang,S., Goeva, A., Nemesh, J., Kamitaki, N., Brumbaugh, S.,Kulp, D., and McCarroll, S. A. A single-cell atlas of celltypes, states, and other transcriptional patterns from nineregions of the adult mouse brain. bioRxiv, 2018.

Shah, S., Lubeck, E., Zhou, W., and Cai, L. In Situ Tran-scription Profiling of Single Cells Reveals Spatial Orga-nization of Cells in the Mouse Hippocampus. Neuron,2016.

Stuart, T., Butler, A., Hoffman, P., Hafemeister, C., Papalexi,E., Mauck, W. M., Stoeckius, M., Smibert, P., and Satija,R. Comprehensive integration of single cell data. bioRxiv,2018.

Sun, B., Feng, J., and Saenko, K. Return of frustratinglyeasy domain adaptation. In Thirtieth AAAI Conferenceon Artificial Intelligence, 2016.

Svensson, V. Droplet scrna-seq is not zero-inflated. bioRxiv,2019.

Svensson, V., Teichmann, S. A., and Stegle, O. SpatialDE:identification of spatially variable genes. Nature Methods,2018.

Wang, X., Allen, W. E., Wright, M. A., Sylwestrak, E. L.,Samusik, N., Vesuna, S., Evans, K., Liu, C., Ramakrish-nan, C., Liu, J., Nolan, G. P., Bava, F.-A., and Deisseroth,K. Three-dimensional intact-tissue sequencing of single-cell transcriptional states. Science, 2018.

Welch, J., Kozareva, V., Ferreira, A., Vanderburg, C., Martin,C., and Macosko, E. Integrative inference of brain cellsimilarities and differences from single-cell genomics.bioRxiv, 2018.

Xu, C., Lopez, R., Mehlman, E., Regier, J., Jordan, M. I.,and Yosef, N. Harmonization and Annotation of Single-cell Transcriptomics data with Deep Generative Models.bioRxiv, 2019.

Zeisel, A., Muñoz-Manchado, A. B., Codeluppi, S., Lönner-berg, P., La Manno, G., Juréus, A., Marques, S., Munguba,H., He, L., Betsholtz, C., Rolny, C., Castelo-Branco, G.,Hjerling-Leffler, J., and Linnarsson, S. Cell types in themouse cortex and hippocampus revealed by single-cellrna-seq. Science, 2015.

Zeisel, A., Hochgerner, H., Lönnerberg, P., Johnsson, A.,Memic, F., van der Zwan, J., Häring, M., Braun, E., Borm,L. E., La Manno, G., Codeluppi, S., Furlan, A., Lee, K.,Skene, N., Harris, K. D., Hjerling-Leffler, J., Arenas, E.,Ernfors, P., Marklund, U., and Linnarsson, S. Moleculararchitecture of the mouse nervous system. Cell, 2018.

Zhu, J.-Y., Park, T., Isola, P., and Efros, A. A. Unpairedimage-to-image translation using cycle-consistent adver-sarial networks. In Proceedings of the IEEE internationalconference on computer vision, 2017.

Zhu, Q., Shah, S., Dries, R., Cai, L., and Yuan, G.-C. Iden-tification of spatially associated subpopulations by com-bining scRNAseq and sequential fluorescence in situ hy-bridization data. Nature Biotechnology, 2018.