geode on docker

36
Docker and Geode William Markito [email protected]

Upload: apache-geode

Post on 17-Aug-2015

117 views

Category:

Technology


3 download

TRANSCRIPT

Docker and GeodeWilliam Markito

[email protected]

Docker and GeodeUsing Apache Geode and Docker

2

DisclaimerApache Geode (incubating) has not yet a release under Apache Software Foundation Incubator. The current bits available are for

development purposes arising from a nightly build. Use at your own risk.

3

‣ Docker basics

‣ Demo

‣ Building Apache Geode image

‣ Docker and Geode basics

‣ Demo

‣ Troubleshooting

‣ Demo

Agenda

4

‣ Enterprise Architect for Pivotal GemFire

‣ Helping customers & field design GemFire

solutions

‣ Helping (?) engineering with features and bugs

‣ Focusing on Apache Geode

Who Am I ?

5

Enterprise in the next year ?

• Containers • FreeBSD Jails (2000) • Solaris Zones (2004) • Docker (2013)

• Operating system level virtualization • Isolated user space instances

Some history…

7

* https://linuxcontainers.org/

• Fast churn (create/delete) • For applications it’s seamless

• Each instance has its own FileSystem, RAM, CPU

• Keep environment consistent • Useful for troubleshooting, CI, tests

• Used in production at scale (Google/Twitter/Netflix)

Why containers ?

8

9

Container vs VM

“..while the hypervisor abstracts the entire device, containers just abstract the operating system kernel"

10

Docker vs Vagrant

Docker Vagrant

Virtual Environment Virtual Machine Manager

Linux* Any platform

Few seconds Few minutes

Partial isolation Full isolation

Dockerfile Vagrantfile

* https://www.scriptrock.com/articles/docker-vs-vagrant

11

Docker vs Vagrant

Dockerfile VagrantfileFROM  centos:latest  

RUN  yum  install  -­‐y  dos2unix  

ENV  MY_VAR  my_value  

EXPOSE    8080  

RUN  mkdir  /data  

CMD  ["bash"]  

Vagrant.configure("2")  do  |config|      config.vm.box  =  "chef/centos-­‐7.0"      config.vm.hostname  =  "mybox"      config.vm.network  :private_network,  ip:  "192.168.0.42"  

   config.vm.provider  :virtualbox  do  |vb|          vb.customize  [              "modifyvm",  :id,              "-­‐-­‐cpuexecutioncap",  "50",              "-­‐-­‐memory",  "256",          ]      end  

   config.vm.provision  :shell,  path:  "bootstrap.sh"      end

• Commands

12

docker  create  docker  run  docker  exec  docker  stop  docker  start  docker  restart  docker  rm  docker  kill  docker  attach  

docker  ps  docker  logs  docker  inspect  docker  events  docker  port  docker  top  docker  stats  docker  diff

Actions Monitoring

Docker 101

• Commands

13

docker  images  docker  import  docker  build  docker  commit  docker  load  

docker  history  docker  tag  docker  login  docker  search  docker  pull  docker  push  

Images Other useful commands

Docker 101

https://github.com/wsargent/docker-cheat-sheet

• Dockerfile • Image descriptor • Step-by-step instructions

14

FROM  centos:latest  

RUN  yum  install  -­‐y  dos2unix  

ENV  MY_VAR  my_value  

EXPOSE    8080  

RUN  mkdir  /data  

CMD  ["bash"]  

Docker 101

15

Demo

Docker 101

16

• Dockerfile • Layers

• Union mount

"A union mount is a way of mounting that allows several filesystems or directories to be simultaneously mounted and visible through a single mount point, appearing to be one filesystem.”*

* Pendry, Jan-Simon; Marshall Kirk McKusick (December 1995). "Union Mounts in 4.4BSD-Lite”

Docker 101

17

FROM  centos:latest  

RUN  yum  install  -­‐y  dos2unix  RUN  yum  install  -­‐y  zlib-­‐devel  

ENV  MY_VAR  my_value  

EXPOSE    8080  

RUN  mkdir  /data  

CMD  ["bash"]  

Building Apache Geode image

• Docker Layers

Previous image: 236.1MB (docker history <id>) - Install a header file of 49k

18

Demo

Building Apache Geode image

• Docker Layers

Previous image: 236.1MB - Install a header file of 49k Current image: 299.7MB

19

FROM  centos:latest  

RUN  yum  install  -­‐y  dos2unix  zlib-­‐devel  

ENV  MY_VAR  my_value  

EXPOSE    8080  

RUN  mkdir  /data  

CMD  ["bash"]  

Building Apache Geode image

Building Apache Geode image

20

FROM  centos:latest  MAINTAINER  William  Markito  <[email protected]>  

LABEL  Vendor="Apache  Geode  (incubating)"  LABEL  version=unstable  

#  download  JDK  8  ENV   JAVA_HOME  /jdk1.8.0_51  

RUN   yum  install  -­‐y  wget  which  tar  git  \     &&  wget  -­‐-­‐no-­‐cookies  -­‐-­‐no-­‐check-­‐certificate  -­‐-­‐header  "Cookie:  gpw_e24=http%3A%2F%2Fwww.oracle.com%2F;  oraclelicense=accept-­‐securebackup-­‐cookie"  "http://download.oracle.com/otn-­‐pub/java/jdk/8u51-­‐b16/jdk-­‐8u51-­‐linux-­‐x64.tar.gz"  \     &&  tar  xf  jdk-­‐8u51-­‐linux-­‐x64.tar.gz  \     &&  git  clone  -­‐b  develop  https://github.com/apache/incubator-­‐geode.git  \     &&  cd  incubator-­‐geode  \     &&  ./gradlew  build  -­‐Dskip.tests=true  \     &&  ls  /incubator-­‐geode  |  grep  -­‐v  gemfire-­‐assembly  |  xargs  rm  -­‐rf  \     &&  rm  -­‐rf  /root/.gradle/  \     &&  rm  -­‐rf  /incubator-­‐geode/gemfire-­‐assembly/build/distributions/  \     &&  rm  -­‐rf  /jdk-­‐8u51-­‐linux-­‐x64.tar.gz  \     &&  rm  -­‐rf  $JAVA_HOME/*src.zip  \               $JAVA_HOME/lib/missioncontrol  \               $JAVA_HOME/lib/visualvm  \               $JAVA_HOME/lib/*javafx*  \               $JAVA_HOME/jre/lib/plugin.jar  \               $JAVA_HOME/jre/lib/amd64/libfxplugins.so  \               $JAVA_HOME/jre/lib/amd64/libglass.so  \               $JAVA_HOME/jre/lib/amd64/libgstreamer-­‐lite.so  \               $JAVA_HOME/jre/lib/amd64/libjavafx*.so  \               $JAVA_HOME/jre/lib/amd64/libjfx*.so  \     &&  rm  -­‐rf  /usr/share/locale/*  \     &&  yum  remove  -­‐y  perl  \     &&  yum  clean  all  

ENV  GEODE_HOME  /incubator-­‐geode/gemfire-­‐assembly/build/install/apache-­‐geode  ENV  PATH  $PATH:$GEODE_HOME/bin:$JAVA_HOME/bin  

#  Default  ports:  #  RMI/JMX  1099  #  REST  8080  #  PULE  7070  #  LOCATOR  10334  #  CACHESERVER  40404  EXPOSE    8080  10334  40404  1099  7070  VOLUME  ["/data/"]  CMD  [“gfsh"]  

21

Demo

Building Apache Geode image

22

• Docker-compose (Fig) • Simple YAML file • Orchestration

• Ordering • Scale • Network wiring

locator:      #build:  .      image:  apachegeode/geode:unstable      hostname:  locator      expose:        -­‐  "10334"        -­‐  "1099"        -­‐  "7575"      ports:        -­‐  "1099:1099"        -­‐  "10334:10334"        -­‐  "7575:7575"      volumes:        -­‐  scripts/:/data/      command:  gfsh  start  locator  -­‐-­‐name=locator  -­‐-­‐mcast-­‐port=0  

server:      image:  apachegeode/geode:unstable      links:        -­‐  locator:locator      expose:        -­‐  "8080"        -­‐  "40404"        -­‐  "1099"      ports:        -­‐  "40404"      volumes:        -­‐  scripts/:/data/      command:  gfsh  start  server  —name=server  —locators=locator[10334]

Apache Geode - Container cluster

23

Demo

Apache Geode - Container cluster

24

Geode on Containers: Use cases

‣ Scale systems

‣ Microservices architecture

‣ REST services on Geode

‣ Testing or reproducing scenarios

‣ Network partitions

‣ Production environment (multiple nodes)

‣ Deploy on cloud infrastructure

25

Docker ecosystem

Docker ecosystem

26

https://hub.docker.com/u/apachegeode/

Agenda

27

‣ Docker basics

‣ Demo

‣ Building Apache Geode image

‣ Docker and Geode basics

‣ Demo

‣ Troubleshooting

‣ Demo

Troubleshooting

28

29

Troubleshooting

• Postmortem • docker ps -a• docker inspect <id>• Volumes!

• Online • docker stats• docker logs• bash/shell

• Your current knowledge still applies!

30

Demo

Troubleshooting

Concerns

31

• Performance (prefer —net=host) • Storage (persistence & backup) • Statistics

• /proc is not containerized • Metrics are still a bit confusing

• Fair access to system resources • Security

• Everything as root… • VM’s do a better job here ?

• Speed + easy of use always wins!

• Portability + consistency for dev, test and prod

• Containers are good for applications and services

• Easy to scale for applications

• Easy to scale for services such as Apache Geode

• Apache Geode as Cloud Native “data storage"

Conclusions

32

• Code • New features • Bug fixes • Writing tests

• Documentation • Wiki • Web site • User guide

33

How to Contribute

• Community • Join the mailing list

• Ask or answer • Join our HipChat • Become a speaker • Finding bugs • Testing an RC/Beta

Q&A

34

35

https://github.com/wsargent/docker-cheat-sheet

https://www.docker.com/tryit/

https://docs.docker.com/

References:

https://hub.docker.com/u/apachegeode/

https://geode.incubator.apache.org

Thanks

36

William [email protected]

https://hub.docker.com/u/apachegeode/

https://github.com/apache/incubator-geode http://geode.incubator.apache.org