dockerized clarin dspace with kubernetes · introduction...
TRANSCRIPT
![Page 2: Dockerized CLARIN DSpace with Kubernetes · Introduction Thispresentationwillgivesomeinsightintoourexperiencewith CLARINDSpace Docker Kubernetes andhowtomakesomethingnewwithallofthem](https://reader035.vdocument.in/reader035/viewer/2022071007/5fc50387d8208315bc08a15e/html5/thumbnails/2.jpg)
Outline
1 Introduction
2 The Idea: Containerising CLARIN DSpace
3 The Implementation: How did we approach the problem?
4 The Problems: Challenges and Common Pitfalls
5 The Future: Open Points and Possible Next Steps
König— Kubernetified DSpace 21 May 2019 | 1 / 19
![Page 3: Dockerized CLARIN DSpace with Kubernetes · Introduction Thispresentationwillgivesomeinsightintoourexperiencewith CLARINDSpace Docker Kubernetes andhowtomakesomethingnewwithallofthem](https://reader035.vdocument.in/reader035/viewer/2022071007/5fc50387d8208315bc08a15e/html5/thumbnails/3.jpg)
Introduction
This presentation will give some insight into our experience with
CLARIN DSpace
Docker
Kubernetes
and how to make something new with all of them
I’ll briefly describe how we did it, where we encountered problems and
how we solved them
König— Kubernetified DSpace 21 May 2019 | 2 / 19
![Page 4: Dockerized CLARIN DSpace with Kubernetes · Introduction Thispresentationwillgivesomeinsightintoourexperiencewith CLARINDSpace Docker Kubernetes andhowtomakesomethingnewwithallofthem](https://reader035.vdocument.in/reader035/viewer/2022071007/5fc50387d8208315bc08a15e/html5/thumbnails/4.jpg)
Introduction
This presentation will give some insight into our experience with
CLARIN DSpace
Docker
Kubernetes
and how to make something new with all of them
I’ll briefly describe how we did it, where we encountered problems and
how we solved them
König— Kubernetified DSpace 21 May 2019 | 2 / 19
![Page 5: Dockerized CLARIN DSpace with Kubernetes · Introduction Thispresentationwillgivesomeinsightintoourexperiencewith CLARINDSpace Docker Kubernetes andhowtomakesomethingnewwithallofthem](https://reader035.vdocument.in/reader035/viewer/2022071007/5fc50387d8208315bc08a15e/html5/thumbnails/5.jpg)
Introduction
This presentation will give some insight into our experience with
CLARIN DSpace
Docker
Kubernetes
and how to make something new with all of them
I’ll briefly describe how we did it, where we encountered problems and
how we solved them
König— Kubernetified DSpace 21 May 2019 | 2 / 19
![Page 6: Dockerized CLARIN DSpace with Kubernetes · Introduction Thispresentationwillgivesomeinsightintoourexperiencewith CLARINDSpace Docker Kubernetes andhowtomakesomethingnewwithallofthem](https://reader035.vdocument.in/reader035/viewer/2022071007/5fc50387d8208315bc08a15e/html5/thumbnails/6.jpg)
Introduction
This presentation will give some insight into our experience with
CLARIN DSpace
Docker
Kubernetes
and how to make something new with all of them
I’ll briefly describe how we did it, where we encountered problems and
how we solved them
König— Kubernetified DSpace 21 May 2019 | 2 / 19
![Page 7: Dockerized CLARIN DSpace with Kubernetes · Introduction Thispresentationwillgivesomeinsightintoourexperiencewith CLARINDSpace Docker Kubernetes andhowtomakesomethingnewwithallofthem](https://reader035.vdocument.in/reader035/viewer/2022071007/5fc50387d8208315bc08a15e/html5/thumbnails/7.jpg)
Introduction
This presentation will give some insight into our experience with
CLARIN DSpace
Docker
Kubernetes
and how to make something new with all of them
I’ll briefly describe how we did it, where we encountered problems and
how we solved them
König— Kubernetified DSpace 21 May 2019 | 2 / 19
![Page 8: Dockerized CLARIN DSpace with Kubernetes · Introduction Thispresentationwillgivesomeinsightintoourexperiencewith CLARINDSpace Docker Kubernetes andhowtomakesomethingnewwithallofthem](https://reader035.vdocument.in/reader035/viewer/2022071007/5fc50387d8208315bc08a15e/html5/thumbnails/8.jpg)
Introduction
This presentation will give some insight into our experience with
CLARIN DSpace
Docker
Kubernetes
and how to make something new with all of them
I’ll briefly describe how we did it, where we encountered problems and
how we solved them
König— Kubernetified DSpace 21 May 2019 | 2 / 19
![Page 9: Dockerized CLARIN DSpace with Kubernetes · Introduction Thispresentationwillgivesomeinsightintoourexperiencewith CLARINDSpace Docker Kubernetes andhowtomakesomethingnewwithallofthem](https://reader035.vdocument.in/reader035/viewer/2022071007/5fc50387d8208315bc08a15e/html5/thumbnails/9.jpg)
Introduction: CLARIN DSpace
CLARIN DSpace in brief
A fork of DSpace with some modifications and add-ons for CLARIN
Developed at UFAL in Prague, but now a community project
Easy and quick to set up (if you’re happy with the default behavior)
König— Kubernetified DSpace 21 May 2019 | 3 / 19
![Page 10: Dockerized CLARIN DSpace with Kubernetes · Introduction Thispresentationwillgivesomeinsightintoourexperiencewith CLARINDSpace Docker Kubernetes andhowtomakesomethingnewwithallofthem](https://reader035.vdocument.in/reader035/viewer/2022071007/5fc50387d8208315bc08a15e/html5/thumbnails/10.jpg)
Introduction: CLARIN DSpace
CLARIN DSpace in brief
A fork of DSpace with some modifications and add-ons for CLARIN
Developed at UFAL in Prague, but now a community project
Easy and quick to set up (if you’re happy with the default behavior)
König— Kubernetified DSpace 21 May 2019 | 3 / 19
![Page 11: Dockerized CLARIN DSpace with Kubernetes · Introduction Thispresentationwillgivesomeinsightintoourexperiencewith CLARINDSpace Docker Kubernetes andhowtomakesomethingnewwithallofthem](https://reader035.vdocument.in/reader035/viewer/2022071007/5fc50387d8208315bc08a15e/html5/thumbnails/11.jpg)
Introduction: CLARIN DSpace
CLARIN DSpace in brief
A fork of DSpace with some modifications and add-ons for CLARIN
Developed at UFAL in Prague, but now a community project
Easy and quick to set up (if you’re happy with the default behavior)
König— Kubernetified DSpace 21 May 2019 | 3 / 19
![Page 12: Dockerized CLARIN DSpace with Kubernetes · Introduction Thispresentationwillgivesomeinsightintoourexperiencewith CLARINDSpace Docker Kubernetes andhowtomakesomethingnewwithallofthem](https://reader035.vdocument.in/reader035/viewer/2022071007/5fc50387d8208315bc08a15e/html5/thumbnails/12.jpg)
Introduction: Docker
Docker in brief
Under heavy development (sometimes still breaks backwards
compatibility)
Available for all major OSes (with a focus on Unix-based ones)
The most popular containerization software
Bundle all necessary software and libraries
Without all the bloat that is not needed (but installed by default in a VM)
Closer coupling to the host resources (less overhead)
König— Kubernetified DSpace 21 May 2019 | 4 / 19
![Page 13: Dockerized CLARIN DSpace with Kubernetes · Introduction Thispresentationwillgivesomeinsightintoourexperiencewith CLARINDSpace Docker Kubernetes andhowtomakesomethingnewwithallofthem](https://reader035.vdocument.in/reader035/viewer/2022071007/5fc50387d8208315bc08a15e/html5/thumbnails/13.jpg)
Introduction: Docker
Docker in brief
Under heavy development (sometimes still breaks backwards
compatibility)
Available for all major OSes (with a focus on Unix-based ones)
The most popular containerization software
Bundle all necessary software and libraries
Without all the bloat that is not needed (but installed by default in a VM)
Closer coupling to the host resources (less overhead)
König— Kubernetified DSpace 21 May 2019 | 4 / 19
![Page 14: Dockerized CLARIN DSpace with Kubernetes · Introduction Thispresentationwillgivesomeinsightintoourexperiencewith CLARINDSpace Docker Kubernetes andhowtomakesomethingnewwithallofthem](https://reader035.vdocument.in/reader035/viewer/2022071007/5fc50387d8208315bc08a15e/html5/thumbnails/14.jpg)
Introduction: Docker
Docker in brief
Under heavy development (sometimes still breaks backwards
compatibility)
Available for all major OSes (with a focus on Unix-based ones)
The most popular containerization software
Bundle all necessary software and libraries
Without all the bloat that is not needed (but installed by default in a VM)
Closer coupling to the host resources (less overhead)
König— Kubernetified DSpace 21 May 2019 | 4 / 19
![Page 15: Dockerized CLARIN DSpace with Kubernetes · Introduction Thispresentationwillgivesomeinsightintoourexperiencewith CLARINDSpace Docker Kubernetes andhowtomakesomethingnewwithallofthem](https://reader035.vdocument.in/reader035/viewer/2022071007/5fc50387d8208315bc08a15e/html5/thumbnails/15.jpg)
Introduction: Docker
Docker in brief
Under heavy development (sometimes still breaks backwards
compatibility)
Available for all major OSes (with a focus on Unix-based ones)
The most popular containerization software
Bundle all necessary software and libraries
Without all the bloat that is not needed (but installed by default in a VM)
Closer coupling to the host resources (less overhead)
König— Kubernetified DSpace 21 May 2019 | 4 / 19
![Page 16: Dockerized CLARIN DSpace with Kubernetes · Introduction Thispresentationwillgivesomeinsightintoourexperiencewith CLARINDSpace Docker Kubernetes andhowtomakesomethingnewwithallofthem](https://reader035.vdocument.in/reader035/viewer/2022071007/5fc50387d8208315bc08a15e/html5/thumbnails/16.jpg)
Introduction: Docker
Docker in brief
Under heavy development (sometimes still breaks backwards
compatibility)
Available for all major OSes (with a focus on Unix-based ones)
The most popular containerization software
Bundle all necessary software and libraries
Without all the bloat that is not needed (but installed by default in a VM)
Closer coupling to the host resources (less overhead)
König— Kubernetified DSpace 21 May 2019 | 4 / 19
![Page 17: Dockerized CLARIN DSpace with Kubernetes · Introduction Thispresentationwillgivesomeinsightintoourexperiencewith CLARINDSpace Docker Kubernetes andhowtomakesomethingnewwithallofthem](https://reader035.vdocument.in/reader035/viewer/2022071007/5fc50387d8208315bc08a15e/html5/thumbnails/17.jpg)
Introduction: Docker
Docker in brief
Under heavy development (sometimes still breaks backwards
compatibility)
Available for all major OSes (with a focus on Unix-based ones)
The most popular containerization software
Bundle all necessary software and libraries
Without all the bloat that is not needed (but installed by default in a VM)
Closer coupling to the host resources (less overhead)
König— Kubernetified DSpace 21 May 2019 | 4 / 19
![Page 18: Dockerized CLARIN DSpace with Kubernetes · Introduction Thispresentationwillgivesomeinsightintoourexperiencewith CLARINDSpace Docker Kubernetes andhowtomakesomethingnewwithallofthem](https://reader035.vdocument.in/reader035/viewer/2022071007/5fc50387d8208315bc08a15e/html5/thumbnails/18.jpg)
Introduction: Kubernetes
Kubernetes in brief
Orchestration tool for (Docker) containers
Helps with
Managing volumes (e.g. through connecting a ceph filesystem)
Scaling by spinning up more containers and
replacing dead containers by restarting them
Managing (HTTP and HTTPS) access from outside through a central nginx
proxy (Ingress)
König— Kubernetified DSpace 21 May 2019 | 5 / 19
![Page 19: Dockerized CLARIN DSpace with Kubernetes · Introduction Thispresentationwillgivesomeinsightintoourexperiencewith CLARINDSpace Docker Kubernetes andhowtomakesomethingnewwithallofthem](https://reader035.vdocument.in/reader035/viewer/2022071007/5fc50387d8208315bc08a15e/html5/thumbnails/19.jpg)
Introduction: Kubernetes
Kubernetes in brief
Orchestration tool for (Docker) containers
Helps with
Managing volumes (e.g. through connecting a ceph filesystem)
Scaling by spinning up more containers and
replacing dead containers by restarting them
Managing (HTTP and HTTPS) access from outside through a central nginx
proxy (Ingress)
König— Kubernetified DSpace 21 May 2019 | 5 / 19
![Page 20: Dockerized CLARIN DSpace with Kubernetes · Introduction Thispresentationwillgivesomeinsightintoourexperiencewith CLARINDSpace Docker Kubernetes andhowtomakesomethingnewwithallofthem](https://reader035.vdocument.in/reader035/viewer/2022071007/5fc50387d8208315bc08a15e/html5/thumbnails/20.jpg)
Introduction: Kubernetes
Kubernetes in brief
Orchestration tool for (Docker) containers
Helps with
Managing volumes (e.g. through connecting a ceph filesystem)
Scaling by spinning up more containers and
replacing dead containers by restarting them
Managing (HTTP and HTTPS) access from outside through a central nginx
proxy (Ingress)
König— Kubernetified DSpace 21 May 2019 | 5 / 19
![Page 21: Dockerized CLARIN DSpace with Kubernetes · Introduction Thispresentationwillgivesomeinsightintoourexperiencewith CLARINDSpace Docker Kubernetes andhowtomakesomethingnewwithallofthem](https://reader035.vdocument.in/reader035/viewer/2022071007/5fc50387d8208315bc08a15e/html5/thumbnails/21.jpg)
Introduction: Kubernetes
Kubernetes in brief
Orchestration tool for (Docker) containers
Helps with
Managing volumes (e.g. through connecting a ceph filesystem)
Scaling by spinning up more containers and
replacing dead containers by restarting them
Managing (HTTP and HTTPS) access from outside through a central nginx
proxy (Ingress)
König— Kubernetified DSpace 21 May 2019 | 5 / 19
![Page 22: Dockerized CLARIN DSpace with Kubernetes · Introduction Thispresentationwillgivesomeinsightintoourexperiencewith CLARINDSpace Docker Kubernetes andhowtomakesomethingnewwithallofthem](https://reader035.vdocument.in/reader035/viewer/2022071007/5fc50387d8208315bc08a15e/html5/thumbnails/22.jpg)
Introduction: Kubernetes
Kubernetes in brief
Orchestration tool for (Docker) containers
Helps with
Managing volumes (e.g. through connecting a ceph filesystem)
Scaling by spinning up more containers and
replacing dead containers by restarting them
Managing (HTTP and HTTPS) access from outside through a central nginx
proxy (Ingress)
König— Kubernetified DSpace 21 May 2019 | 5 / 19
![Page 23: Dockerized CLARIN DSpace with Kubernetes · Introduction Thispresentationwillgivesomeinsightintoourexperiencewith CLARINDSpace Docker Kubernetes andhowtomakesomethingnewwithallofthem](https://reader035.vdocument.in/reader035/viewer/2022071007/5fc50387d8208315bc08a15e/html5/thumbnails/23.jpg)
Introduction: Kubernetes
Kubernetes in brief
Orchestration tool for (Docker) containers
Helps with
Managing volumes (e.g. through connecting a ceph filesystem)
Scaling by spinning up more containers and
replacing dead containers by restarting them
Managing (HTTP and HTTPS) access from outside through a central nginx
proxy (Ingress)
König— Kubernetified DSpace 21 May 2019 | 5 / 19
![Page 24: Dockerized CLARIN DSpace with Kubernetes · Introduction Thispresentationwillgivesomeinsightintoourexperiencewith CLARINDSpace Docker Kubernetes andhowtomakesomethingnewwithallofthem](https://reader035.vdocument.in/reader035/viewer/2022071007/5fc50387d8208315bc08a15e/html5/thumbnails/24.jpg)
Outline
1 Introduction
2 The Idea: Containerising CLARIN DSpace
3 The Implementation: How did we approach the problem?
4 The Problems: Challenges and Common Pitfalls
5 The Future: Open Points and Possible Next Steps
König— Kubernetified DSpace 21 May 2019 | 6 / 19
![Page 25: Dockerized CLARIN DSpace with Kubernetes · Introduction Thispresentationwillgivesomeinsightintoourexperiencewith CLARINDSpace Docker Kubernetes andhowtomakesomethingnewwithallofthem](https://reader035.vdocument.in/reader035/viewer/2022071007/5fc50387d8208315bc08a15e/html5/thumbnails/25.jpg)
The Idea
The situation
IAL at Eurac Research wanted to set up a repository to become a CLARIN
Centre
CLARIN DSpace quickly turned out to be the obvious choice
We decided to dockerize it for multiple reasons
it seems like a cleaner, more easily reproducible set-up than doing it
inside a VM
Our IT had recently set up a Kubernetes cluster
CLARIN-IT is still in the beginning and doesn’t have much money, so we
thought creating a (more or less) one-click installation for a
CLARIN-compatible repo might be interesting to other institutes in Italy
König— Kubernetified DSpace 21 May 2019 | 7 / 19
![Page 26: Dockerized CLARIN DSpace with Kubernetes · Introduction Thispresentationwillgivesomeinsightintoourexperiencewith CLARINDSpace Docker Kubernetes andhowtomakesomethingnewwithallofthem](https://reader035.vdocument.in/reader035/viewer/2022071007/5fc50387d8208315bc08a15e/html5/thumbnails/26.jpg)
The Idea
The situation
IAL at Eurac Research wanted to set up a repository to become a CLARIN
Centre
CLARIN DSpace quickly turned out to be the obvious choice
We decided to dockerize it for multiple reasons
it seems like a cleaner, more easily reproducible set-up than doing it
inside a VM
Our IT had recently set up a Kubernetes cluster
CLARIN-IT is still in the beginning and doesn’t have much money, so we
thought creating a (more or less) one-click installation for a
CLARIN-compatible repo might be interesting to other institutes in Italy
König— Kubernetified DSpace 21 May 2019 | 7 / 19
![Page 27: Dockerized CLARIN DSpace with Kubernetes · Introduction Thispresentationwillgivesomeinsightintoourexperiencewith CLARINDSpace Docker Kubernetes andhowtomakesomethingnewwithallofthem](https://reader035.vdocument.in/reader035/viewer/2022071007/5fc50387d8208315bc08a15e/html5/thumbnails/27.jpg)
The Idea
The situation
IAL at Eurac Research wanted to set up a repository to become a CLARIN
Centre
CLARIN DSpace quickly turned out to be the obvious choice
We decided to dockerize it for multiple reasons
it seems like a cleaner, more easily reproducible set-up than doing it
inside a VM
Our IT had recently set up a Kubernetes cluster
CLARIN-IT is still in the beginning and doesn’t have much money, so we
thought creating a (more or less) one-click installation for a
CLARIN-compatible repo might be interesting to other institutes in Italy
König— Kubernetified DSpace 21 May 2019 | 7 / 19
![Page 28: Dockerized CLARIN DSpace with Kubernetes · Introduction Thispresentationwillgivesomeinsightintoourexperiencewith CLARINDSpace Docker Kubernetes andhowtomakesomethingnewwithallofthem](https://reader035.vdocument.in/reader035/viewer/2022071007/5fc50387d8208315bc08a15e/html5/thumbnails/28.jpg)
The Idea
The situation
IAL at Eurac Research wanted to set up a repository to become a CLARIN
Centre
CLARIN DSpace quickly turned out to be the obvious choice
We decided to dockerize it for multiple reasons
it seems like a cleaner, more easily reproducible set-up than doing it
inside a VM
Our IT had recently set up a Kubernetes cluster
CLARIN-IT is still in the beginning and doesn’t have much money, so we
thought creating a (more or less) one-click installation for a
CLARIN-compatible repo might be interesting to other institutes in Italy
König— Kubernetified DSpace 21 May 2019 | 7 / 19
![Page 29: Dockerized CLARIN DSpace with Kubernetes · Introduction Thispresentationwillgivesomeinsightintoourexperiencewith CLARINDSpace Docker Kubernetes andhowtomakesomethingnewwithallofthem](https://reader035.vdocument.in/reader035/viewer/2022071007/5fc50387d8208315bc08a15e/html5/thumbnails/29.jpg)
The Idea
The situation
IAL at Eurac Research wanted to set up a repository to become a CLARIN
Centre
CLARIN DSpace quickly turned out to be the obvious choice
We decided to dockerize it for multiple reasons
it seems like a cleaner, more easily reproducible set-up than doing it
inside a VM
Our IT had recently set up a Kubernetes cluster
CLARIN-IT is still in the beginning and doesn’t have much money, so we
thought creating a (more or less) one-click installation for a
CLARIN-compatible repo might be interesting to other institutes in Italy
König— Kubernetified DSpace 21 May 2019 | 7 / 19
![Page 30: Dockerized CLARIN DSpace with Kubernetes · Introduction Thispresentationwillgivesomeinsightintoourexperiencewith CLARINDSpace Docker Kubernetes andhowtomakesomethingnewwithallofthem](https://reader035.vdocument.in/reader035/viewer/2022071007/5fc50387d8208315bc08a15e/html5/thumbnails/30.jpg)
The Idea
The situation
IAL at Eurac Research wanted to set up a repository to become a CLARIN
Centre
CLARIN DSpace quickly turned out to be the obvious choice
We decided to dockerize it for multiple reasons
it seems like a cleaner, more easily reproducible set-up than doing it
inside a VM
Our IT had recently set up a Kubernetes cluster
CLARIN-IT is still in the beginning and doesn’t have much money, so we
thought creating a (more or less) one-click installation for a
CLARIN-compatible repo might be interesting to other institutes in Italy
König— Kubernetified DSpace 21 May 2019 | 7 / 19
![Page 31: Dockerized CLARIN DSpace with Kubernetes · Introduction Thispresentationwillgivesomeinsightintoourexperiencewith CLARINDSpace Docker Kubernetes andhowtomakesomethingnewwithallofthem](https://reader035.vdocument.in/reader035/viewer/2022071007/5fc50387d8208315bc08a15e/html5/thumbnails/31.jpg)
Outline
1 Introduction
2 The Idea: Containerising CLARIN DSpace
3 The Implementation: How did we approach the problem?
4 The Problems: Challenges and Common Pitfalls
5 The Future: Open Points and Possible Next Steps
König— Kubernetified DSpace 21 May 2019 | 8 / 19
![Page 32: Dockerized CLARIN DSpace with Kubernetes · Introduction Thispresentationwillgivesomeinsightintoourexperiencewith CLARINDSpace Docker Kubernetes andhowtomakesomethingnewwithallofthem](https://reader035.vdocument.in/reader035/viewer/2022071007/5fc50387d8208315bc08a15e/html5/thumbnails/32.jpg)
The Implementation: All-in-One
One Container To Rule Them All
As a first step (as Docker newbies) we followed the DSpace installation
instructions and set it all up within one container
Typical docker pitfalls
some system services like systemd are not available
some helper programs (less,vim,ping) are not available (for debugging)
communications to the outside can be tricky (especially non HTPP(S))
The installation process needed some manual steps (e.g. generating the
admin user) which we needed to automatize
End result: ”Dockerized CLARIN DSpace”
König— Kubernetified DSpace 21 May 2019 | 9 / 19
![Page 33: Dockerized CLARIN DSpace with Kubernetes · Introduction Thispresentationwillgivesomeinsightintoourexperiencewith CLARINDSpace Docker Kubernetes andhowtomakesomethingnewwithallofthem](https://reader035.vdocument.in/reader035/viewer/2022071007/5fc50387d8208315bc08a15e/html5/thumbnails/33.jpg)
The Implementation: All-in-One
One Container To Rule Them All
As a first step (as Docker newbies) we followed the DSpace installation
instructions and set it all up within one container
Typical docker pitfalls
some system services like systemd are not available
some helper programs (less,vim,ping) are not available (for debugging)
communications to the outside can be tricky (especially non HTPP(S))
The installation process needed some manual steps (e.g. generating the
admin user) which we needed to automatize
End result: ”Dockerized CLARIN DSpace”
König— Kubernetified DSpace 21 May 2019 | 9 / 19
![Page 34: Dockerized CLARIN DSpace with Kubernetes · Introduction Thispresentationwillgivesomeinsightintoourexperiencewith CLARINDSpace Docker Kubernetes andhowtomakesomethingnewwithallofthem](https://reader035.vdocument.in/reader035/viewer/2022071007/5fc50387d8208315bc08a15e/html5/thumbnails/34.jpg)
The Implementation: All-in-One
One Container To Rule Them All
As a first step (as Docker newbies) we followed the DSpace installation
instructions and set it all up within one container
Typical docker pitfalls
some system services like systemd are not available
some helper programs (less,vim,ping) are not available (for debugging)
communications to the outside can be tricky (especially non HTPP(S))
The installation process needed some manual steps (e.g. generating the
admin user) which we needed to automatize
End result: ”Dockerized CLARIN DSpace”
König— Kubernetified DSpace 21 May 2019 | 9 / 19
![Page 35: Dockerized CLARIN DSpace with Kubernetes · Introduction Thispresentationwillgivesomeinsightintoourexperiencewith CLARINDSpace Docker Kubernetes andhowtomakesomethingnewwithallofthem](https://reader035.vdocument.in/reader035/viewer/2022071007/5fc50387d8208315bc08a15e/html5/thumbnails/35.jpg)
The Implementation: All-in-One
One Container To Rule Them All
As a first step (as Docker newbies) we followed the DSpace installation
instructions and set it all up within one container
Typical docker pitfalls
some system services like systemd are not available
some helper programs (less,vim,ping) are not available (for debugging)
communications to the outside can be tricky (especially non HTPP(S))
The installation process needed some manual steps (e.g. generating the
admin user) which we needed to automatize
End result: ”Dockerized CLARIN DSpace”
König— Kubernetified DSpace 21 May 2019 | 9 / 19
![Page 36: Dockerized CLARIN DSpace with Kubernetes · Introduction Thispresentationwillgivesomeinsightintoourexperiencewith CLARINDSpace Docker Kubernetes andhowtomakesomethingnewwithallofthem](https://reader035.vdocument.in/reader035/viewer/2022071007/5fc50387d8208315bc08a15e/html5/thumbnails/36.jpg)
The Implementation: Docker Compose
Separating Services using Docker Compose
We now have three containers nginx including shibboleth, psql and a big
one with all the rest
Separation problems
Most services talk to each other over the network, but some need local
files or sockets
Figuring out which libraries are necessary for which program is not always
easy
It became obvious that some internal program manager (e.g.
supervisor) is often necessary
König— Kubernetified DSpace 21 May 2019 | 10 / 19
![Page 37: Dockerized CLARIN DSpace with Kubernetes · Introduction Thispresentationwillgivesomeinsightintoourexperiencewith CLARINDSpace Docker Kubernetes andhowtomakesomethingnewwithallofthem](https://reader035.vdocument.in/reader035/viewer/2022071007/5fc50387d8208315bc08a15e/html5/thumbnails/37.jpg)
The Implementation: Docker Compose
Separating Services using Docker Compose
We now have three containers nginx including shibboleth, psql and a big
one with all the rest
Separation problems
Most services talk to each other over the network, but some need local
files or sockets
Figuring out which libraries are necessary for which program is not always
easy
It became obvious that some internal program manager (e.g.
supervisor) is often necessary
König— Kubernetified DSpace 21 May 2019 | 10 / 19
![Page 38: Dockerized CLARIN DSpace with Kubernetes · Introduction Thispresentationwillgivesomeinsightintoourexperiencewith CLARINDSpace Docker Kubernetes andhowtomakesomethingnewwithallofthem](https://reader035.vdocument.in/reader035/viewer/2022071007/5fc50387d8208315bc08a15e/html5/thumbnails/38.jpg)
The Implementation: Docker Compose
Separating Services using Docker Compose
We now have three containers nginx including shibboleth, psql and a big
one with all the rest
Separation problems
Most services talk to each other over the network, but some need local
files or sockets
Figuring out which libraries are necessary for which program is not always
easy
It became obvious that some internal program manager (e.g.
supervisor) is often necessary
König— Kubernetified DSpace 21 May 2019 | 10 / 19
![Page 39: Dockerized CLARIN DSpace with Kubernetes · Introduction Thispresentationwillgivesomeinsightintoourexperiencewith CLARINDSpace Docker Kubernetes andhowtomakesomethingnewwithallofthem](https://reader035.vdocument.in/reader035/viewer/2022071007/5fc50387d8208315bc08a15e/html5/thumbnails/39.jpg)
The Implementation: Docker Compose
Separating Services using Docker Compose
We now have three containers nginx including shibboleth, psql and a big
one with all the rest
Separation problems
Most services talk to each other over the network, but some need local
files or sockets
Figuring out which libraries are necessary for which program is not always
easy
It became obvious that some internal program manager (e.g.
supervisor) is often necessary
König— Kubernetified DSpace 21 May 2019 | 10 / 19
![Page 40: Dockerized CLARIN DSpace with Kubernetes · Introduction Thispresentationwillgivesomeinsightintoourexperiencewith CLARINDSpace Docker Kubernetes andhowtomakesomethingnewwithallofthem](https://reader035.vdocument.in/reader035/viewer/2022071007/5fc50387d8208315bc08a15e/html5/thumbnails/40.jpg)
The Implementation: Docker Compose
Separating Services using Docker Compose
We now have three containers nginx including shibboleth, psql and a big
one with all the rest
Separation problems
Most services talk to each other over the network, but some need local
files or sockets
Figuring out which libraries are necessary for which program is not always
easy
It became obvious that some internal program manager (e.g.
supervisor) is often necessary
König— Kubernetified DSpace 21 May 2019 | 10 / 19
![Page 41: Dockerized CLARIN DSpace with Kubernetes · Introduction Thispresentationwillgivesomeinsightintoourexperiencewith CLARINDSpace Docker Kubernetes andhowtomakesomethingnewwithallofthem](https://reader035.vdocument.in/reader035/viewer/2022071007/5fc50387d8208315bc08a15e/html5/thumbnails/41.jpg)
The Implementation: Kubernetes
Konverting Docker Compose to Kubernetes
Handy tool Kompose (https://kompose.io/) creates basic Kubernetesyaml files from docker-compose.yml
Volumes need to be added using Kubernetes volume management (we
use a ceph filesystem that is connected to the Kube cluster)
All secret information (e.g. usernames and passwords) is moved to
Kubernetes secrets
Every outside access to containers is managed by a central Ingress proxy
where also the SSL certificates are stored
makes it hard to distinguish different services via IP (setting up two
handle servers)
non-standard ports (e.g. handle net) are difficult to set up
Kubernetes fetches Docker images from registry (but as there are
multiple nodes, sometimes the same image needs to be reloaded upon
restart, which is mostly a problem with large images)
König— Kubernetified DSpace 21 May 2019 | 11 / 19
![Page 42: Dockerized CLARIN DSpace with Kubernetes · Introduction Thispresentationwillgivesomeinsightintoourexperiencewith CLARINDSpace Docker Kubernetes andhowtomakesomethingnewwithallofthem](https://reader035.vdocument.in/reader035/viewer/2022071007/5fc50387d8208315bc08a15e/html5/thumbnails/42.jpg)
The Implementation: Kubernetes
Konverting Docker Compose to Kubernetes
Handy tool Kompose (https://kompose.io/) creates basic Kubernetesyaml files from docker-compose.yml
Volumes need to be added using Kubernetes volume management (we
use a ceph filesystem that is connected to the Kube cluster)
All secret information (e.g. usernames and passwords) is moved to
Kubernetes secrets
Every outside access to containers is managed by a central Ingress proxy
where also the SSL certificates are stored
makes it hard to distinguish different services via IP (setting up two
handle servers)
non-standard ports (e.g. handle net) are difficult to set up
Kubernetes fetches Docker images from registry (but as there are
multiple nodes, sometimes the same image needs to be reloaded upon
restart, which is mostly a problem with large images)
König— Kubernetified DSpace 21 May 2019 | 11 / 19
![Page 43: Dockerized CLARIN DSpace with Kubernetes · Introduction Thispresentationwillgivesomeinsightintoourexperiencewith CLARINDSpace Docker Kubernetes andhowtomakesomethingnewwithallofthem](https://reader035.vdocument.in/reader035/viewer/2022071007/5fc50387d8208315bc08a15e/html5/thumbnails/43.jpg)
The Implementation: Kubernetes
Konverting Docker Compose to Kubernetes
Handy tool Kompose (https://kompose.io/) creates basic Kubernetesyaml files from docker-compose.yml
Volumes need to be added using Kubernetes volume management (we
use a ceph filesystem that is connected to the Kube cluster)
All secret information (e.g. usernames and passwords) is moved to
Kubernetes secrets
Every outside access to containers is managed by a central Ingress proxy
where also the SSL certificates are stored
makes it hard to distinguish different services via IP (setting up two
handle servers)
non-standard ports (e.g. handle net) are difficult to set up
Kubernetes fetches Docker images from registry (but as there are
multiple nodes, sometimes the same image needs to be reloaded upon
restart, which is mostly a problem with large images)
König— Kubernetified DSpace 21 May 2019 | 11 / 19
![Page 44: Dockerized CLARIN DSpace with Kubernetes · Introduction Thispresentationwillgivesomeinsightintoourexperiencewith CLARINDSpace Docker Kubernetes andhowtomakesomethingnewwithallofthem](https://reader035.vdocument.in/reader035/viewer/2022071007/5fc50387d8208315bc08a15e/html5/thumbnails/44.jpg)
The Implementation: Kubernetes
Konverting Docker Compose to Kubernetes
Handy tool Kompose (https://kompose.io/) creates basic Kubernetesyaml files from docker-compose.yml
Volumes need to be added using Kubernetes volume management (we
use a ceph filesystem that is connected to the Kube cluster)
All secret information (e.g. usernames and passwords) is moved to
Kubernetes secrets
Every outside access to containers is managed by a central Ingress proxy
where also the SSL certificates are stored
makes it hard to distinguish different services via IP (setting up two
handle servers)
non-standard ports (e.g. handle net) are difficult to set up
Kubernetes fetches Docker images from registry (but as there are
multiple nodes, sometimes the same image needs to be reloaded upon
restart, which is mostly a problem with large images)
König— Kubernetified DSpace 21 May 2019 | 11 / 19
![Page 45: Dockerized CLARIN DSpace with Kubernetes · Introduction Thispresentationwillgivesomeinsightintoourexperiencewith CLARINDSpace Docker Kubernetes andhowtomakesomethingnewwithallofthem](https://reader035.vdocument.in/reader035/viewer/2022071007/5fc50387d8208315bc08a15e/html5/thumbnails/45.jpg)
The Implementation: Kubernetes
Konverting Docker Compose to Kubernetes
Handy tool Kompose (https://kompose.io/) creates basic Kubernetesyaml files from docker-compose.yml
Volumes need to be added using Kubernetes volume management (we
use a ceph filesystem that is connected to the Kube cluster)
All secret information (e.g. usernames and passwords) is moved to
Kubernetes secrets
Every outside access to containers is managed by a central Ingress proxy
where also the SSL certificates are stored
makes it hard to distinguish different services via IP (setting up two
handle servers)
non-standard ports (e.g. handle net) are difficult to set up
Kubernetes fetches Docker images from registry (but as there are
multiple nodes, sometimes the same image needs to be reloaded upon
restart, which is mostly a problem with large images)
König— Kubernetified DSpace 21 May 2019 | 11 / 19
![Page 46: Dockerized CLARIN DSpace with Kubernetes · Introduction Thispresentationwillgivesomeinsightintoourexperiencewith CLARINDSpace Docker Kubernetes andhowtomakesomethingnewwithallofthem](https://reader035.vdocument.in/reader035/viewer/2022071007/5fc50387d8208315bc08a15e/html5/thumbnails/46.jpg)
The Implementation: Kubernetes
Konverting Docker Compose to Kubernetes
Handy tool Kompose (https://kompose.io/) creates basic Kubernetesyaml files from docker-compose.yml
Volumes need to be added using Kubernetes volume management (we
use a ceph filesystem that is connected to the Kube cluster)
All secret information (e.g. usernames and passwords) is moved to
Kubernetes secrets
Every outside access to containers is managed by a central Ingress proxy
where also the SSL certificates are stored
makes it hard to distinguish different services via IP (setting up two
handle servers)
non-standard ports (e.g. handle net) are difficult to set up
Kubernetes fetches Docker images from registry (but as there are
multiple nodes, sometimes the same image needs to be reloaded upon
restart, which is mostly a problem with large images)
König— Kubernetified DSpace 21 May 2019 | 11 / 19
![Page 47: Dockerized CLARIN DSpace with Kubernetes · Introduction Thispresentationwillgivesomeinsightintoourexperiencewith CLARINDSpace Docker Kubernetes andhowtomakesomethingnewwithallofthem](https://reader035.vdocument.in/reader035/viewer/2022071007/5fc50387d8208315bc08a15e/html5/thumbnails/47.jpg)
The Implementation: Kubernetes
Konverting Docker Compose to Kubernetes
Handy tool Kompose (https://kompose.io/) creates basic Kubernetesyaml files from docker-compose.yml
Volumes need to be added using Kubernetes volume management (we
use a ceph filesystem that is connected to the Kube cluster)
All secret information (e.g. usernames and passwords) is moved to
Kubernetes secrets
Every outside access to containers is managed by a central Ingress proxy
where also the SSL certificates are stored
makes it hard to distinguish different services via IP (setting up two
handle servers)
non-standard ports (e.g. handle net) are difficult to set up
Kubernetes fetches Docker images from registry (but as there are
multiple nodes, sometimes the same image needs to be reloaded upon
restart, which is mostly a problem with large images)
König— Kubernetified DSpace 21 May 2019 | 11 / 19
![Page 48: Dockerized CLARIN DSpace with Kubernetes · Introduction Thispresentationwillgivesomeinsightintoourexperiencewith CLARINDSpace Docker Kubernetes andhowtomakesomethingnewwithallofthem](https://reader035.vdocument.in/reader035/viewer/2022071007/5fc50387d8208315bc08a15e/html5/thumbnails/48.jpg)
The Implementation: Customization
Implementing Eurac Branding and Adapting some defaults
Used CLARIN WAYF instead of built-in UFAL one
Adapted documentation and privacy policy (GDPR)
Adapted color style to Eurac (proved problematic because some
functionality seems to be hardcoded in the UFAL theme)
Added a static start page that introduces the ERCC
König— Kubernetified DSpace 21 May 2019 | 12 / 19
![Page 49: Dockerized CLARIN DSpace with Kubernetes · Introduction Thispresentationwillgivesomeinsightintoourexperiencewith CLARINDSpace Docker Kubernetes andhowtomakesomethingnewwithallofthem](https://reader035.vdocument.in/reader035/viewer/2022071007/5fc50387d8208315bc08a15e/html5/thumbnails/49.jpg)
The Implementation: Customization
Implementing Eurac Branding and Adapting some defaults
Used CLARIN WAYF instead of built-in UFAL one
Adapted documentation and privacy policy (GDPR)
Adapted color style to Eurac (proved problematic because some
functionality seems to be hardcoded in the UFAL theme)
Added a static start page that introduces the ERCC
König— Kubernetified DSpace 21 May 2019 | 12 / 19
![Page 50: Dockerized CLARIN DSpace with Kubernetes · Introduction Thispresentationwillgivesomeinsightintoourexperiencewith CLARINDSpace Docker Kubernetes andhowtomakesomethingnewwithallofthem](https://reader035.vdocument.in/reader035/viewer/2022071007/5fc50387d8208315bc08a15e/html5/thumbnails/50.jpg)
The Implementation: Customization
Implementing Eurac Branding and Adapting some defaults
Used CLARIN WAYF instead of built-in UFAL one
Adapted documentation and privacy policy (GDPR)
Adapted color style to Eurac (proved problematic because some
functionality seems to be hardcoded in the UFAL theme)
Added a static start page that introduces the ERCC
König— Kubernetified DSpace 21 May 2019 | 12 / 19
![Page 51: Dockerized CLARIN DSpace with Kubernetes · Introduction Thispresentationwillgivesomeinsightintoourexperiencewith CLARINDSpace Docker Kubernetes andhowtomakesomethingnewwithallofthem](https://reader035.vdocument.in/reader035/viewer/2022071007/5fc50387d8208315bc08a15e/html5/thumbnails/51.jpg)
The Implementation: Customization
Implementing Eurac Branding and Adapting some defaults
Used CLARIN WAYF instead of built-in UFAL one
Adapted documentation and privacy policy (GDPR)
Adapted color style to Eurac (proved problematic because some
functionality seems to be hardcoded in the UFAL theme)
Added a static start page that introduces the ERCC
König— Kubernetified DSpace 21 May 2019 | 12 / 19
![Page 52: Dockerized CLARIN DSpace with Kubernetes · Introduction Thispresentationwillgivesomeinsightintoourexperiencewith CLARINDSpace Docker Kubernetes andhowtomakesomethingnewwithallofthem](https://reader035.vdocument.in/reader035/viewer/2022071007/5fc50387d8208315bc08a15e/html5/thumbnails/52.jpg)
Outline
1 Introduction
2 The Idea: Containerising CLARIN DSpace
3 The Implementation: How did we approach the problem?
4 The Problems: Challenges and Common Pitfalls
5 The Future: Open Points and Possible Next Steps
König— Kubernetified DSpace 21 May 2019 | 13 / 19
![Page 53: Dockerized CLARIN DSpace with Kubernetes · Introduction Thispresentationwillgivesomeinsightintoourexperiencewith CLARINDSpace Docker Kubernetes andhowtomakesomethingnewwithallofthem](https://reader035.vdocument.in/reader035/viewer/2022071007/5fc50387d8208315bc08a15e/html5/thumbnails/53.jpg)
The Problems: Versioning of software libraries
Software used in the Dockerfile has to be strictly versioned
Our DSpace Dockerfile uses a lot of sometimes very specific software
Some like Shibboleth are built from various sources that are fetched
with a script
For some libraries the URL where the source can be downloaded cannot
be guessed from the version number alone (manual investigation
necessary)
Sometimes working URLs stop resolving because old library versions are
archived in some way (=> build script suddenly stops working)
König— Kubernetified DSpace 21 May 2019 | 14 / 19
![Page 54: Dockerized CLARIN DSpace with Kubernetes · Introduction Thispresentationwillgivesomeinsightintoourexperiencewith CLARINDSpace Docker Kubernetes andhowtomakesomethingnewwithallofthem](https://reader035.vdocument.in/reader035/viewer/2022071007/5fc50387d8208315bc08a15e/html5/thumbnails/54.jpg)
The Problems: Versioning of software libraries
Software used in the Dockerfile has to be strictly versioned
Our DSpace Dockerfile uses a lot of sometimes very specific software
Some like Shibboleth are built from various sources that are fetched
with a script
For some libraries the URL where the source can be downloaded cannot
be guessed from the version number alone (manual investigation
necessary)
Sometimes working URLs stop resolving because old library versions are
archived in some way (=> build script suddenly stops working)
König— Kubernetified DSpace 21 May 2019 | 14 / 19
![Page 55: Dockerized CLARIN DSpace with Kubernetes · Introduction Thispresentationwillgivesomeinsightintoourexperiencewith CLARINDSpace Docker Kubernetes andhowtomakesomethingnewwithallofthem](https://reader035.vdocument.in/reader035/viewer/2022071007/5fc50387d8208315bc08a15e/html5/thumbnails/55.jpg)
The Problems: Versioning of software libraries
Software used in the Dockerfile has to be strictly versioned
Our DSpace Dockerfile uses a lot of sometimes very specific software
Some like Shibboleth are built from various sources that are fetched
with a script
For some libraries the URL where the source can be downloaded cannot
be guessed from the version number alone (manual investigation
necessary)
Sometimes working URLs stop resolving because old library versions are
archived in some way (=> build script suddenly stops working)
König— Kubernetified DSpace 21 May 2019 | 14 / 19
![Page 56: Dockerized CLARIN DSpace with Kubernetes · Introduction Thispresentationwillgivesomeinsightintoourexperiencewith CLARINDSpace Docker Kubernetes andhowtomakesomethingnewwithallofthem](https://reader035.vdocument.in/reader035/viewer/2022071007/5fc50387d8208315bc08a15e/html5/thumbnails/56.jpg)
The Problems: Versioning of software libraries
Software used in the Dockerfile has to be strictly versioned
Our DSpace Dockerfile uses a lot of sometimes very specific software
Some like Shibboleth are built from various sources that are fetched
with a script
For some libraries the URL where the source can be downloaded cannot
be guessed from the version number alone (manual investigation
necessary)
Sometimes working URLs stop resolving because old library versions are
archived in some way (=> build script suddenly stops working)
König— Kubernetified DSpace 21 May 2019 | 14 / 19
![Page 57: Dockerized CLARIN DSpace with Kubernetes · Introduction Thispresentationwillgivesomeinsightintoourexperiencewith CLARINDSpace Docker Kubernetes andhowtomakesomethingnewwithallofthem](https://reader035.vdocument.in/reader035/viewer/2022071007/5fc50387d8208315bc08a15e/html5/thumbnails/57.jpg)
The Problems: General Kubernetes Problems
Kubernetes Problems and Peculiarities
Proxying every access through Ingress needed some config tweaking to
not lose origin IP (for statistics)
Kubernetes often assumes that every user is admin which means that
some info is not available to a non-privileged user (e.g. Ingress logs =>
makes debugging harder)
Kubernetes is heavily developed, which means that sometimes urgent
updates are needed, which may result in some services becoming
unavailable
König— Kubernetified DSpace 21 May 2019 | 15 / 19
![Page 58: Dockerized CLARIN DSpace with Kubernetes · Introduction Thispresentationwillgivesomeinsightintoourexperiencewith CLARINDSpace Docker Kubernetes andhowtomakesomethingnewwithallofthem](https://reader035.vdocument.in/reader035/viewer/2022071007/5fc50387d8208315bc08a15e/html5/thumbnails/58.jpg)
The Problems: General Kubernetes Problems
Kubernetes Problems and Peculiarities
Proxying every access through Ingress needed some config tweaking to
not lose origin IP (for statistics)
Kubernetes often assumes that every user is admin which means that
some info is not available to a non-privileged user (e.g. Ingress logs =>
makes debugging harder)
Kubernetes is heavily developed, which means that sometimes urgent
updates are needed, which may result in some services becoming
unavailable
König— Kubernetified DSpace 21 May 2019 | 15 / 19
![Page 59: Dockerized CLARIN DSpace with Kubernetes · Introduction Thispresentationwillgivesomeinsightintoourexperiencewith CLARINDSpace Docker Kubernetes andhowtomakesomethingnewwithallofthem](https://reader035.vdocument.in/reader035/viewer/2022071007/5fc50387d8208315bc08a15e/html5/thumbnails/59.jpg)
The Problems: General Kubernetes Problems
Kubernetes Problems and Peculiarities
Proxying every access through Ingress needed some config tweaking to
not lose origin IP (for statistics)
Kubernetes often assumes that every user is admin which means that
some info is not available to a non-privileged user (e.g. Ingress logs =>
makes debugging harder)
Kubernetes is heavily developed, which means that sometimes urgent
updates are needed, which may result in some services becoming
unavailable
König— Kubernetified DSpace 21 May 2019 | 15 / 19
![Page 60: Dockerized CLARIN DSpace with Kubernetes · Introduction Thispresentationwillgivesomeinsightintoourexperiencewith CLARINDSpace Docker Kubernetes andhowtomakesomethingnewwithallofthem](https://reader035.vdocument.in/reader035/viewer/2022071007/5fc50387d8208315bc08a15e/html5/thumbnails/60.jpg)
The Problems: Various Issues
Open Issues and Common Problems
Implementing CLARIN WAYF resulted in local login to be unavailable
Rebasing lindat-common (and to a lesser degree) clarin-dspace repo
often results in conflicts because we had to change some general things
Eurac theming is spread across three projects (forks of clarin-dspace &
lindat-common and the general docker project)
König— Kubernetified DSpace 21 May 2019 | 16 / 19
![Page 61: Dockerized CLARIN DSpace with Kubernetes · Introduction Thispresentationwillgivesomeinsightintoourexperiencewith CLARINDSpace Docker Kubernetes andhowtomakesomethingnewwithallofthem](https://reader035.vdocument.in/reader035/viewer/2022071007/5fc50387d8208315bc08a15e/html5/thumbnails/61.jpg)
The Problems: Various Issues
Open Issues and Common Problems
Implementing CLARIN WAYF resulted in local login to be unavailable
Rebasing lindat-common (and to a lesser degree) clarin-dspace repo
often results in conflicts because we had to change some general things
Eurac theming is spread across three projects (forks of clarin-dspace &
lindat-common and the general docker project)
König— Kubernetified DSpace 21 May 2019 | 16 / 19
![Page 62: Dockerized CLARIN DSpace with Kubernetes · Introduction Thispresentationwillgivesomeinsightintoourexperiencewith CLARINDSpace Docker Kubernetes andhowtomakesomethingnewwithallofthem](https://reader035.vdocument.in/reader035/viewer/2022071007/5fc50387d8208315bc08a15e/html5/thumbnails/62.jpg)
The Problems: Various Issues
Open Issues and Common Problems
Implementing CLARIN WAYF resulted in local login to be unavailable
Rebasing lindat-common (and to a lesser degree) clarin-dspace repo
often results in conflicts because we had to change some general things
Eurac theming is spread across three projects (forks of clarin-dspace &
lindat-common and the general docker project)
König— Kubernetified DSpace 21 May 2019 | 16 / 19
![Page 63: Dockerized CLARIN DSpace with Kubernetes · Introduction Thispresentationwillgivesomeinsightintoourexperiencewith CLARINDSpace Docker Kubernetes andhowtomakesomethingnewwithallofthem](https://reader035.vdocument.in/reader035/viewer/2022071007/5fc50387d8208315bc08a15e/html5/thumbnails/63.jpg)
Outline
1 Introduction
2 The Idea: Containerising CLARIN DSpace
3 The Implementation: How did we approach the problem?
4 The Problems: Challenges and Common Pitfalls
5 The Future: Open Points and Possible Next Steps
König— Kubernetified DSpace 21 May 2019 | 17 / 19
![Page 64: Dockerized CLARIN DSpace with Kubernetes · Introduction Thispresentationwillgivesomeinsightintoourexperiencewith CLARINDSpace Docker Kubernetes andhowtomakesomethingnewwithallofthem](https://reader035.vdocument.in/reader035/viewer/2022071007/5fc50387d8208315bc08a15e/html5/thumbnails/64.jpg)
What comes next?
Open Issues
Figure out a way to have both Shibboleth and local login available
Investigate whether/how the handle server can be moved to its own
container
Improve documentation
(If possible) enable docker-compose again (in a separate branch?) for
users without Kubernetes
Shibbolize our search interfaces (e.g. Annis) and integrate them with
DSpace
Investigate the use of a templating engine (e.g. Helm) to make updates
more robust (e.g. version number only needs to be changed once)
Various optimizations (e.g. reduce the container size, main container is
currently > 3GB)
König— Kubernetified DSpace 21 May 2019 | 18 / 19
![Page 65: Dockerized CLARIN DSpace with Kubernetes · Introduction Thispresentationwillgivesomeinsightintoourexperiencewith CLARINDSpace Docker Kubernetes andhowtomakesomethingnewwithallofthem](https://reader035.vdocument.in/reader035/viewer/2022071007/5fc50387d8208315bc08a15e/html5/thumbnails/65.jpg)
What comes next?
Open Issues
Figure out a way to have both Shibboleth and local login available
Investigate whether/how the handle server can be moved to its own
container
Improve documentation
(If possible) enable docker-compose again (in a separate branch?) for
users without Kubernetes
Shibbolize our search interfaces (e.g. Annis) and integrate them with
DSpace
Investigate the use of a templating engine (e.g. Helm) to make updates
more robust (e.g. version number only needs to be changed once)
Various optimizations (e.g. reduce the container size, main container is
currently > 3GB)
König— Kubernetified DSpace 21 May 2019 | 18 / 19
![Page 66: Dockerized CLARIN DSpace with Kubernetes · Introduction Thispresentationwillgivesomeinsightintoourexperiencewith CLARINDSpace Docker Kubernetes andhowtomakesomethingnewwithallofthem](https://reader035.vdocument.in/reader035/viewer/2022071007/5fc50387d8208315bc08a15e/html5/thumbnails/66.jpg)
What comes next?
Open Issues
Figure out a way to have both Shibboleth and local login available
Investigate whether/how the handle server can be moved to its own
container
Improve documentation
(If possible) enable docker-compose again (in a separate branch?) for
users without Kubernetes
Shibbolize our search interfaces (e.g. Annis) and integrate them with
DSpace
Investigate the use of a templating engine (e.g. Helm) to make updates
more robust (e.g. version number only needs to be changed once)
Various optimizations (e.g. reduce the container size, main container is
currently > 3GB)
König— Kubernetified DSpace 21 May 2019 | 18 / 19
![Page 67: Dockerized CLARIN DSpace with Kubernetes · Introduction Thispresentationwillgivesomeinsightintoourexperiencewith CLARINDSpace Docker Kubernetes andhowtomakesomethingnewwithallofthem](https://reader035.vdocument.in/reader035/viewer/2022071007/5fc50387d8208315bc08a15e/html5/thumbnails/67.jpg)
What comes next?
Open Issues
Figure out a way to have both Shibboleth and local login available
Investigate whether/how the handle server can be moved to its own
container
Improve documentation
(If possible) enable docker-compose again (in a separate branch?) for
users without Kubernetes
Shibbolize our search interfaces (e.g. Annis) and integrate them with
DSpace
Investigate the use of a templating engine (e.g. Helm) to make updates
more robust (e.g. version number only needs to be changed once)
Various optimizations (e.g. reduce the container size, main container is
currently > 3GB)
König— Kubernetified DSpace 21 May 2019 | 18 / 19
![Page 68: Dockerized CLARIN DSpace with Kubernetes · Introduction Thispresentationwillgivesomeinsightintoourexperiencewith CLARINDSpace Docker Kubernetes andhowtomakesomethingnewwithallofthem](https://reader035.vdocument.in/reader035/viewer/2022071007/5fc50387d8208315bc08a15e/html5/thumbnails/68.jpg)
What comes next?
Open Issues
Figure out a way to have both Shibboleth and local login available
Investigate whether/how the handle server can be moved to its own
container
Improve documentation
(If possible) enable docker-compose again (in a separate branch?) for
users without Kubernetes
Shibbolize our search interfaces (e.g. Annis) and integrate them with
DSpace
Investigate the use of a templating engine (e.g. Helm) to make updates
more robust (e.g. version number only needs to be changed once)
Various optimizations (e.g. reduce the container size, main container is
currently > 3GB)
König— Kubernetified DSpace 21 May 2019 | 18 / 19
![Page 69: Dockerized CLARIN DSpace with Kubernetes · Introduction Thispresentationwillgivesomeinsightintoourexperiencewith CLARINDSpace Docker Kubernetes andhowtomakesomethingnewwithallofthem](https://reader035.vdocument.in/reader035/viewer/2022071007/5fc50387d8208315bc08a15e/html5/thumbnails/69.jpg)
What comes next?
Open Issues
Figure out a way to have both Shibboleth and local login available
Investigate whether/how the handle server can be moved to its own
container
Improve documentation
(If possible) enable docker-compose again (in a separate branch?) for
users without Kubernetes
Shibbolize our search interfaces (e.g. Annis) and integrate them with
DSpace
Investigate the use of a templating engine (e.g. Helm) to make updates
more robust (e.g. version number only needs to be changed once)
Various optimizations (e.g. reduce the container size, main container is
currently > 3GB)
König— Kubernetified DSpace 21 May 2019 | 18 / 19
![Page 70: Dockerized CLARIN DSpace with Kubernetes · Introduction Thispresentationwillgivesomeinsightintoourexperiencewith CLARINDSpace Docker Kubernetes andhowtomakesomethingnewwithallofthem](https://reader035.vdocument.in/reader035/viewer/2022071007/5fc50387d8208315bc08a15e/html5/thumbnails/70.jpg)
What comes next?
Open Issues
Figure out a way to have both Shibboleth and local login available
Investigate whether/how the handle server can be moved to its own
container
Improve documentation
(If possible) enable docker-compose again (in a separate branch?) for
users without Kubernetes
Shibbolize our search interfaces (e.g. Annis) and integrate them with
DSpace
Investigate the use of a templating engine (e.g. Helm) to make updates
more robust (e.g. version number only needs to be changed once)
Various optimizations (e.g. reduce the container size, main container is
currently > 3GB)
König— Kubernetified DSpace 21 May 2019 | 18 / 19
![Page 71: Dockerized CLARIN DSpace with Kubernetes · Introduction Thispresentationwillgivesomeinsightintoourexperiencewith CLARINDSpace Docker Kubernetes andhowtomakesomethingnewwithallofthem](https://reader035.vdocument.in/reader035/viewer/2022071007/5fc50387d8208315bc08a15e/html5/thumbnails/71.jpg)
Fin
Thank you for your attention!
https://gitlab.inf.unibz.it/commul/docker/clarin-dspace/
König— Kubernetified DSpace 21 May 2019 | 19 / 19