docker @ foss4g 2016, bonn
TRANSCRIPT
An overview of Docker images for geospatial
applicationsFOSS4G conference 2016, Bonn, Germany
Daniel NüstInstitute for Geoinformatics
University of Münster
Talk abstract
http://2016.foss4g.org/talks.html#146
Docker is a growing open-source platform for building and shipping applications as cloud services in so called containers. But containers can be more than that! Following the idea of DevOps, Dockerfiles are a complete scripted definition of an application with all it's dependencies, which can be build and published as ready to use images. As each container is only running "one thing" (e.g. one application, one database, a worker instance), multiple containers can be configured with the help of docker-compose.
More and more geospatial open source projects or third parties provide Dockerfiles. In this talk, we try to give an overview of the existing Docker images and docker-compose configurations for FOSS4G projects. We report on test runs that we conducted with them, informing about the evaluation results, target purposes, licenses, commonly used base images, and more. We will also give a short introduction into Docker and present the purposes that Docker images can be used for, such as easy evaluation for new users, education, testing, or common development environments.
This talk integrates and summarizes information from previous talks at FOSS4G and FOSSGIS conferences, so I'd like to thank Sophia Parafina, Jonathan Meyer, and Björn Schilberg for their contributions.
Agenda
What is Docker? Why?
What can it be used for?
Overview of geo-related containers
Select containers Live Demo
Why containerization?Why Docker?
Motivation
http://www.slideshare.net/gmccance/cern-data-centre-evolution
Pets vs. Cattle
Motivationhttps://www.docker.com/use-cases
Science
Reproducibility is at the of
Slide by Docker inventor & Docker, Inc. CTO Solomon Hykes, DockerCon 2014
https://www.docker.com/what-dockerhttps://en.wikipedia.org/wiki/Operating-system-level_virtualizationhttps://youtu.be/ki8CZkutoxQ
Containerization using
kernel featuresnamespaceslibcontainer, LXCcgroupsresources
Houses vs. Appartments
Docker basics
Dockerfile
…
…
Docker Image
Docker Container
build run
pausestop/killstartlogscpexecrmstats
docker-compose configuration
…
…
use n
Docker Container
Docker Container
Docker Container
updown
Docker CLI
Docker Engine
Docker Registry
run
“Official Docker registry”: Docker Hub
https://hub.docker.com/u/osgeo/
$ docker run -it --rm osgeo/proj.4root@f1141552cee9:/# projRel. 4.9.3, dd Month yyyyusage: proj [ -beEfiIlormsStTvVwW [args] ] [ +opts[=arg] ] [ files ]root@f1141552cee9:/#
Process and evaluation
GoogleSearch Docker HubSearch FOSS4G talksOSGeo project list
Criteria and data points
Goals / target users / applications (e.g. testing, dev env,
new users, demos, …)Maintainer (3rd party, original devs?)LicenseBase image
“geo” on Docker Hub
“gis” on Docker Hub
Degree
tfr42/deegreetfr42/teamenginemdillon/postgis
FROM tomcat:8-jre7MAINTAINER deegree TMC <[email protected]>
# set deegree versionENV DEEGREE_VERSION 3.3.18
# download deegreeRUN wget http://repo.deegree.org/content/repositories/public/org/deegree/deegree-webservices/${DEEGREE_VERSION}/deegree-webservices-${DEEGREE_VERSION}.war -O /usr/local/tomcat/webapps/deegree-webservices.war
# run tomcatCMD ["catalina.sh", "run"]
gdal - Live Demo
docker run --rm geodata/gdal
Play around with gdal, easily use different versions (of library combinations)
https://github.com/geo-data/gdal-docker
Use it as a “binary executable” in scripts, e.g. https://gist.github.com/spara/73972116fe751bf374e29618a10d5bed
https://hub.docker.com/r/spara/gdal_ef/
QGIS - Live Demo
xhost +# --rm will remove the container as soon as it endsdocker run --rm --name="qgis-desktop-2.14ltr" \ -i -t \ -v ${HOME}:/home/${USER} \ -v /tmp/.X11-unix:/tmp/.X11-unix \ -e DISPLAY=unix$DISPLAY \ kartoza/qgis-desktop:2.14xhost -
docker run --name "qgis-server-webclient" -p 8081:80 -d -t opengisch/qgis-server-webclient
PostGIS
PostGIS - Live Demo
docker run --name some-postgis -e POSTGRES_PASSWORD=mysecretpassword -d mdillon/postgis
docker run -it --link some-postgis:postgres --rm postgres \ sh -c 'exec psql -h "$POSTGRES_PORT_5432_TCP_ADDR" -p "$POSTGRES_PORT_5432_TCP_PORT" -U postgres'
# \list# \q
tileserver-mapnik - Live Demo
wget -c https://osm2vectortiles-downloads.os.zhdk.cloud.switch.ch/v1.0/extracts/zurich.mbtiles
git clone https://github.com/mapbox/mapbox-studio-osm-bright.tm2.git
docker run -v $(pwd):/data -p 8085:80 klokantech/tileserver-mapnik
52°North WPS - Live Demo
docker run -p 80:8080 52n-wps-base
http://localhost/wps
pycsw Live Demo
docker run --rm -it -p 8000:8000 geocontainers/pycsw
http://localhost:8000/
GeoNode Live Demo
docker run -p 8000:8000 -p 8080:8080 openmarble/arch-geonode
http://localhost:8080/
GeoMOOSE Live Demo
docker run -p 80:80 --rm=true --name geomoose_demo geomoosehttp://localhost/geomoose/geomoose_dev.html
Great example: Instructions for demo, testing, and development usage of the image
Osmosis Live Demo
docker run --rm -t -i -v $(pwd)/tmp:/data farberg/openstreetmap-osmosis-docker --read-pbf /data/monaco.osm.pbf --write-xml /data/monaco.xml
docker run --rm -t -i -v $(pwd)/tmp:/data ferimer/osmosis:20160525 --read-pbf /data/monaco.osm.pbf --write-xml /data/monaco2.xml
diff tmp/monaco2.xml tmp/monaco.xml
PDAL
http://www.pdal.io/workshop/docker.html
PDAL stands on the shoulders of giants. It uses GDAL, GEOS, and many other dependencies. Because of this, it is very challenging to build it yourself. We could easily burn an entire workshop learning the esoteric build miseries of PDAL and all of its dependencies. Fortunately, Docker will allow us to use a fully-featured known configuration to run our examples and exercises without having to suffer so much.
docker run -t pdal/pdal pdal --help
CartoDB
docker run -d -p 3000:3000 -p 8080:8080 -p 8181:8181 --name carto sverhoeven/cartodb
sudo sh -c 'echo 127.0.1.1 cartodb.localhost >> /etc/hosts'
docker run -p 80:80 --link carto:cartodb.localhost -d spawnthink/cartodb-nginx
http://cartodb.localhost
Join on the OSGeo wiki
https://wiki.osgeo.org/wiki/DockerImages
https://wiki.osgeo.org/wiki/DockerImagesMeta
Core arguments for FOSS4G community
(all the Docker advantages… write once, biz ops, cloud, etc.)
Lower entrance barrier for users AND devs(e.g. sandboxed stack per developer, J. Meyer)
Easier transition from testing to production
Tasks & Challenges for the community
Put your project on Docker Hub as an automated build and add a license
Use base images, extend them to provide useful variants
Collaborate with existing images!
Make services and apps configurable using ENVironment parameters and files
Projects and their developers must build up knowledge
Communicate intention of Dockerfile because there is not one Dockerfile fitting all problems
UbuntuGIS easy, but huge images (try alpine!)
Packaging UI / Desktop applications still in flow
https://hub.docker.com/search/?isAutomated=0&isOfficial=0&page=1&pullCount=0&q=foss4g&starCount=0 as of Aug 23 2016
Thanks for your attention!
What are your questions?
http://o2r.info @[email protected]
Thanks to…Björn Schilberg (Intevation) - https://www.fossgis.de/konferenz/2015/programm/events/847.de.html and https://www.youtube.com/watch?v=CeI8S_tnCZ0Jonathan Meyer - https://2016.foss4g-na.org/session/image-mosaics-automation Sophia Parafina - https://2016.foss4g-na.org/session/spatial-data-processing-docker