kipoi: : model zoo for genomics · transfer learning: adapting existing models to new tasks conv....
TRANSCRIPT
![Page 1: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/1.jpg)
Kipoi: : model zoo for genomics
Žiga AvsecPhD candidate, Technical University of Munich
www.gagneurlab.in.tum.de
@gagneurlab, @KipoiZoo, @Avsecz
![Page 2: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/2.jpg)
GenomicsACGTGTCAGTAGTTAAGCTAGTAGCTGATCGGTAACGTAGTGCACGTGTCAGTAGTTAAGCTAGTAGCTGATC
3 billion letters (x2) = 1 genome
37 trillion cells
![Page 3: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/3.jpg)
3
Proteins = main building blocks
Genome
Protein1
Protein1
Protein2
Protein3
Protein3
Protein3
~100k - 1M
![Page 4: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/4.jpg)
4
Proteins = main building blocks
Genome
Protein1
Protein1
Protein2
Protein3
Protein2
~100k - 1M
![Page 5: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/5.jpg)
5
Proteins = main building blocks
Genome
Protein1
Protein1
Protein2
Protein3
Protein2
Protein2
Protein1
Pro
tein
2Protein3
~100k - 1M
Protein complex
Function1
Function2
![Page 6: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/6.jpg)
How to make proteins from the genome?
![Page 7: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/7.jpg)
7
atcgtatatatcatgatatggatacgcatagatcatgactcaggatacgDNA (2)
aucaugauauggauacgcauagaucaugacuca
Transcription
precursor RNA
Protein (~100,000)
Gene expression: How information in DNA is read out
aucaugauacauagaucaugacuca
Splicing
mature RNA
Translation
Gene (~20,000, 1% of DNA)
Intron Exon
![Page 8: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/8.jpg)
8
atcgtatatatcatgatatggatacgcatagatcatgactcaggatacg
Transcriptionfactor
TranscriptionFactor binding site
Gene expression: How information in DNA is read out
![Page 9: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/9.jpg)
9
atcgtatatatcatgatatggatacgcatagatcatgactcaggatacg
RNA polymerase
Gene expression: How information in DNA is read out
![Page 10: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/10.jpg)
10
atcgtatatatcatgatatggatacgcatagatcatgactcaggatacg
Gene expression: How information in DNA is read out
![Page 11: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/11.jpg)
11
atcgtatatatcatgatatggatacgcatagatcatgactcaggatacg
Gene expression: How information in DNA is read out
![Page 12: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/12.jpg)
12
The regulatory elements control:● The position of transcription initiation (what)● The frequency of transcription (how much)
atcgtatatatcatgatatggatacgcatagatcatgactcaggatacg
Gene expression: How information in DNA is read out
![Page 13: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/13.jpg)
13
atcttatatatcatgatatggatacgcatagatcatgactcaggatacg
Genetic variants can disrupt regulatory elements
ReferencePatient
atcgtatatatcatgatatggatacgcatagatcatgactcaggatacg
![Page 14: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/14.jpg)
14
aucaugauauggauacgcauagaucaugacuca
aucaugauacauagaucaugacuca
Transcription
Thousands of regulatory elementsacross all steps of gene expression
Splicing
Translation RNA degradation
Ø
Protein degradation
Ø
cttatcacagtgtatatcatgatatggatacgcatagatcatgactcaggatacg
![Page 15: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/15.jpg)
15
Experimental data
Measuring the regulatory steps via sequencing
![Page 16: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/16.jpg)
16
Experimental data Predictive models
Learning the regulatory steps
![Page 17: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/17.jpg)
17
Experimental data Predictive models
Learning the regulatory steps
![Page 18: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/18.jpg)
18
GATA TAL
cttatcacagtgtatatcatgatatggatacgcatagatcatgactcaggatacg
Detecting regulatory elementswith convolutional neural networks
![Page 19: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/19.jpg)
19Eraslan*, Avsec* et al Nature Review Genetics 2019 (In press)
![Page 20: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/20.jpg)
20Eraslan*, Avsec* et al Nature Review Genetics 2019 (In press)
![Page 21: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/21.jpg)
21Eraslan*, Avsec* et al Nature Review Genetics 2019 (In press)
![Page 22: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/22.jpg)
22Eraslan*, Avsec* et al Nature Review Genetics 2019 (In press)
![Page 23: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/23.jpg)
23Eraslan*, Avsec* et al Nature Review Genetics 2019 (In press)
![Page 24: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/24.jpg)
24Eraslan*, Avsec* et al Nature Review Genetics 2019 (In press)
![Page 25: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/25.jpg)
25
That’s why we need GPUs in regulatory genomics.
Eraslan*, Avsec* et al Nature Review Genetics 2019 (In press)
![Page 26: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/26.jpg)
26
Experimental data Predictive models
Learning the regulatory steps
![Page 27: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/27.jpg)
List of published predictive models
27
- Transcriptional regulation- TF Binding
- PWM Scanning (Jaspar, Cis-BP, MEME)- DeepBind- Improved DeepBind- FactorNet- GERV- DanQ- CKN-seq
- Chromatin- DeepSEA- DeepChrome- Basenji
- DNA methylation- CpGenie- DeepCpG
- DNA Accessibility- Basset
- TSS: - FIDDLE
- Gene-Expression- Basenji- Expecto
- Post-transcriptional regulation- RBP binding
- iDeep- rbp_eclip (Avsec et al)
- miRNA binding- TargetScan- deepMiRGene
- Splicing- MaxEntScan 5’, 3’- Labranchor- HAL- MMSplice- SpliceAI
- mRNA half-life- Cheng et al 2017
- Polyadenylation- APARENT
- Translation- Optimus_5Prime- Cuperus et al 2017
See also: https://github.com/greenelab/deep-review
![Page 28: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/28.jpg)
List of published predictive models
28
- Transcriptional regulation- TF Binding
- PWM Scanning (Jaspar, Cis-BP, MEME)- DeepBind- Improved DeepBind- FactorNet- GERV- DanQ- CKN-seq
- Chromatin- DeepSEA- DeepChrome- Basenji
- DNA methylation- CpGenie- DeepCpG
- DNA Accessibility- Basset
- TSS: - FIDDLE
- Gene-Expression- Basenji- Expecto
- Post-transcriptional regulation- RBP binding
- iDeep- rbp_eclip (Avsec et al)
- miRNA binding- TargetScan- deepMiRGene
- Splicing- MaxEntScan 5’, 3’- Labranchor- HAL- MMSplice- SpliceAI
- mRNA half-life- Cheng et al 2017
- Polyadenylation- APARENT
- Translation- Optimus_5Prime- Cuperus et al 2017
Can we easily apply these models to new data?
Can we easily re-use these models?
See also: https://github.com/greenelab/deep-review
![Page 29: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/29.jpg)
29
Lack of standardization leads to poor sharing and re-use of trained predictive models in genomics
Trained predictive models
Code repository Paper supplements Author-maintained web page
![Page 30: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/30.jpg)
30
Lack of standardization leads to poor sharing and re-use of trained predictive models in genomics
Trained predictive models
Code repository Paper supplements Author-maintained web page
Data
....
Bioinformatics software
![Page 31: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/31.jpg)
31
The “Findable Accessible Interoperable Reusable” principlenot only for data but also for trained predictive models
![Page 32: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/32.jpg)
32
Challenges
- Making predictions end-to-end- predict <model> -i input.data -o output.data
![Page 33: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/33.jpg)
33
Challenges
- Making predictions end-to-end- predict <model> -i input.data -o output.data
- Data heterogeneity (think one paper = one dataset)
![Page 34: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/34.jpg)
34
Challenges
- Making predictions end-to-end- predict <model> -i input.data -o output.data
- Data heterogeneity (think one paper = one dataset)
- Model heterogeneity (from deep learning frameworks to custom code)- Dependency issues
![Page 35: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/35.jpg)
35with A. Kundaje, Stanford, and O. Stegle, DKFZ
Kipoi.org [Kípi]
Avsec et al, Nature Biotechnology (In press)
![Page 36: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/36.jpg)
36
Trained model (model.yaml)
![Page 37: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/37.jpg)
37
Model
TGATCGAGGGTAGCTAGC
CGTGAGTTT
Output
Model
Input
Parameters
Can be implemented using:
data-loader model
“Parameterized function”
![Page 38: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/38.jpg)
38
Model data-loader model
![Page 39: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/39.jpg)
Data-loader data-loader model
chr1 1000 2000chr2 5000 7000
>chr1NNNNNNNNNNNN...
intervals.bed
genome.faresize
extract transform
array([[[1, 0, 0, 0], [0, 1, 0, 0], [0, 0, 1, 0], [1, 0, 0, 0], [ … ]],
[[0, 1, 0, 0], [0, 0, 1, 0], [1, 0, 0, 0], [0, 0, 0, 1], [ … ]]])
github.com/kipoi/kipoiseq
![Page 40: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/40.jpg)
40
Dependencies data-loader model
![Page 41: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/41.jpg)
41
Test predictions
...
TGATCGAGGGTAGCTAGC
CGTGAGTTT
TGATCGAGGGTAGCTAGC
CGTGAGTTTTGATCGAGGGTAGCTAGC
CGTGAGTTT
x
![Page 42: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/42.jpg)
42
Schema
TGATCGAGGA
...
Supports multiple inputs/outputs
![Page 43: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/43.jpg)
43
General information
![Page 44: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/44.jpg)
44
Model repository
![Page 45: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/45.jpg)
45
![Page 46: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/46.jpg)
46
![Page 47: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/47.jpg)
47
# model groups: 20
# of models: 2073
![Page 48: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/48.jpg)
48
- Install new conda environment- Test the pipeline: data -> dataloader -> model- Test the predictions match
Testing models
![Page 49: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/49.jpg)
49
- Install new conda environment- Test the pipeline: data -> dataloader -> model- Test the predictions match
Testing models
When?- Pull-request
- Nightly (all model groups)
![Page 50: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/50.jpg)
50
Using models
![Page 51: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/51.jpg)
For the impatient:30 seconds introduction to Kipoi
![Page 52: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/52.jpg)
52
![Page 53: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/53.jpg)
53
Case study 1: Benchmarking models
![Page 56: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/56.jpg)
56
Benchmarking alternative models
# Create and activate a new conda environment with# all model dependencies installedkipoi env create <Model>source activate kipoi-<Model># Run model predictionkipoi predict <Model> \ --dataloader_args='{
“intervals_file”: “intervals.bed”, “fasta_file”: “hg38.fa”}' \
-o ‘<Model>.preds.h5'
![Page 57: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/57.jpg)
57
Benchmarking alternative models
# Create and activate a new conda environment with# all model dependencies installedkipoi env create <Model>source activate kipoi-<Model># Run model predictionkipoi predict <Model> \ --dataloader_args='{
“intervals_file”: “intervals.bed”, “fasta_file”: “hg38.fa”}' \
-o ‘<Model>.preds.h5'
<- Works very nicely with workflow-management tools like Snakemake
![Page 58: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/58.jpg)
58
# Create and activate a new conda environment with# all model dependencies installedkipoi env create <Model>source activate kipoi-<Model># Run model predictionkipoi predict <Model> \ --dataloader_args='{
“intervals_file”: “intervals.bed”, “fasta_file”: “hg38.fa”}' \
-o ‘<Model>.preds.h5'
Why not a single command?
predict <model> -i input.data -o output.data
![Page 59: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/59.jpg)
59
# Run model predictionkipoi predict <Model> \ --dataloader_args='{
“intervals_file”: “intervals.bed”, “fasta_file”: “hg38.fa”}' \
-o ‘<Model>.preds.h5' --singularity
Why not a single command?
predict <model> -i input.data -o output.data
![Page 60: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/60.jpg)
60
# Run model predictionkipoi predict <Model> \ --dataloader_args='{
“intervals_file”: “intervals.bed”, “fasta_file”: “hg38.fa”}' \
-o ‘<Model>.preds.h5' --singularity
Why not a single command?
predict <model> -i input.data -o output.data
input.data
Container
Model output.data
![Page 61: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/61.jpg)
61
# Run model predictionkipoi predict <Model> \ --dataloader_args='{
“intervals_file”: “intervals.bed”, “fasta_file”: “hg38.fa”}' \
-o ‘<Model>.preds.h5' --singularity
Why not a single command?
predict <model> -i input.data -o output.data
input.data
Container
Model output.data
In-progress:- --docker- --docker-gpu <- Container will be
available on NGC
![Page 62: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/62.jpg)
62
Transfer learning
![Page 63: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/63.jpg)
Transfer learning: Adapting existing models to new tasks
Conv. layers
Dense
Dense
Dense
Model with transferred parameters
Pre-trained model
DNA accessibility in 421 cell-types
DNA accessibility in new cell type
Conv. layers
Dense
Dense
Dense Area under thePrecision-recallcurve
Training epoch
See also Kelley et al. Gen. res. 2016
Randomly initialized(>1day)
Transferred (<4h)
Takes a few days to train(Divergent421 model in Kipoi)
![Page 64: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/64.jpg)
Transfer learning: Adapting existing models to new tasks
Training epoch
Area under thePrecision-recallcurve
See also Kelley et al. Gen. res. 2016
![Page 65: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/65.jpg)
65
Interpreting models
![Page 66: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/66.jpg)
66Eraslan*, Avsec* et al NRG 2019 (In press)
![Page 67: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/67.jpg)
67Eraslan*, Avsec* et al NRG 2019 (In press)
![Page 68: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/68.jpg)
68Eraslan*, Avsec* et al NRG 2019 (In press)
![Page 69: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/69.jpg)
69
# Pythonimport kipoifrom kipoi_interpret.importance_scores.gradient import GradientXInput
model = kipoi.get_model("model”)
imp_score = GradientXInput(model)
scores = imp_score.score(seqs)
# CLIkipoi interpret create_mutation_map \ <Model> \ --dataloader_args='{
“intervals_file”: “intervals.bed”, “fasta_file”: “hg38.fa”}' \
-o ‘mmap.h5'
kipoi-interpret
![Page 70: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/70.jpg)
70
Feature importance score plugin● Variant rs35703285, near the beta globin gene HBB, is pathogenic (ClinVar) and
linked to Beta thalassemia
![Page 71: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/71.jpg)
71
Feature importance score plugin● Variant rs35703285, near the beta globin gene HBB, is pathogenic (ClinVar) and
linked to Beta thalassemia
Methods- ISM- grad- input*grad- DeepLift
![Page 72: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/72.jpg)
72
Scoring genetic variants
![Page 73: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/73.jpg)
73
atcttatatatcatgatatggatacgcatagatcatgactcaggatacg
Scoring genetic variants
ReferencePatient
atcgtatatatcatgatatggatacgcatagatcatgactcaggatacg
#CHROM POS ID REF ALT …chr22 41320486 . G T …
- Each one of us carries ca 1,000,000 variants
![Page 75: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/75.jpg)
75
Variant effect prediction plugin
# Annotate VCF file with variant scoreskipoi veff score_variants <Model> \ --dataloader_args='{
“fasta_file”: “hg38.fa”}' \ --vcf_path 'input.vcf' \ -o ‘annotated.vcf'
● Supported by 12/20 model groups, runnable on VCF files
● In-silico mutagenesis
![Page 76: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/76.jpg)
76
Kipoi variant scoring as a DNAnexus applet
![Page 77: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/77.jpg)
77
atcgtatatatcatgatatggatacgcatagatcatgactcaggatacg
aucaugauauggauacgcauagaucaugacuca
Splicing: an essential step of protein production
aucaugauacauagaucaugacuca
Splicing
![Page 78: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/78.jpg)
78
atcgtatatatcatgatatggatactcatagatcatgactcaggatacg
aucaugauauggauactcauagaucaugacuca
Splicing: an essential step of protein production
aucaugauacauataucaugacuca
Splicing
![Page 79: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/79.jpg)
79
Scotti & Swanson, 2016 NRG
Splicing is a complex, multi-step process
![Page 80: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/80.jpg)
80
Different models model different regions
Donor AcceptorBranchpoint
MaxEntScan/3prime MaxEntScan/5primeHAL
labranchor
![Page 81: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/81.jpg)
81
Different models model different regions
Donor AcceptorBranchpoint
MaxEntScan/3prime MaxEntScan/5primeHAL
labranchor
Binary classification:- Pathogenic (ClinVar)- Benign (ClinVar)
![Page 82: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/82.jpg)
Ensemble model predicting pathogenic variants near splice sites
See also MMSplice from Cheng et al., Genome Biology, CAGI Splicing challenge 2018 winner
Kipoi models:KipoiSplice/4KipoiSplice/4consMMSplice
![Page 83: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/83.jpg)
Summary
83
![Page 84: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/84.jpg)
84
Experimental data Predictive models
Learning the regulatory steps
![Page 85: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/85.jpg)
85with A. Kundaje, Stanford, and O. Stegle, DKFZ
Kipoi.org [Kípi]
Avsec et al, Nature Biotechnology (In press)
![Page 86: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/86.jpg)
86
Kundaje lab Stanford● Anshul Kundaje● Avanti Shrikumar● Nancy Xu● Abhimanyu Banerjee● Chuan Sheng Foo
Gagneur lab TU Munich● Julien Gagneur● Jun Cheng
Stegle lab Cambridge EMBL-EBI● Oliver Stegle● Roman Kreuzhuber● Thorsten Beider● Lara Urban
Acknowledgements.org@KipoiZoo
Roman Kreuzhuber
Nvidia
● Jonny Israeli● Fernanda Foertter● Gary Dunn● Adam Simpson
Thorsten Beider
DNA Nexus
● Jason Chin● Maria Simbirsky● Andrew Carroll
![Page 87: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/87.jpg)
Kipoi: : model zoo for genomics
Žiga AvsecPhD candidate, Technical University of Munich
www.gagneurlab.in.tum.de
@gagneurlab, @KipoiZoo, @Avsecz
![Page 88: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA](https://reader035.vdocument.in/reader035/viewer/2022071418/6116b867fca27b34bc414a9f/html5/thumbnails/88.jpg)
88