docker service registration and discovery
TRANSCRIPT
![Page 1: Docker Service Registration and Discovery](https://reader031.vdocument.in/reader031/viewer/2022020208/55a687031a28ab391e8b4588/html5/thumbnails/1.jpg)
Service Registration
and Discovery with
Michael Richardson @m_richo
![Page 2: Docker Service Registration and Discovery](https://reader031.vdocument.in/reader031/viewer/2022020208/55a687031a28ab391e8b4588/html5/thumbnails/2.jpg)
$whoami
• Michael Richardson
• Engineer @ odecee
• @m_richo
• http://www.mricho.com
• I like DevOps*
• Docker has grown on me in the last 6 months
* My definition of DevOps is likely different from yours
![Page 3: Docker Service Registration and Discovery](https://reader031.vdocument.in/reader031/viewer/2022020208/55a687031a28ab391e8b4588/html5/thumbnails/3.jpg)
Agenda
• What is Service Registration & Discovery?
• Consul
– tool to store information about services
• Registrator
– tool to register Docker services
• Consul-template
– tool to query registered services and apply configuration.
• Demo
![Page 4: Docker Service Registration and Discovery](https://reader031.vdocument.in/reader031/viewer/2022020208/55a687031a28ab391e8b4588/html5/thumbnails/4.jpg)
Service Registration and Discovery
What is it?
![Page 5: Docker Service Registration and Discovery](https://reader031.vdocument.in/reader031/viewer/2022020208/55a687031a28ab391e8b4588/html5/thumbnails/5.jpg)
Service Registration and Discovery
What is it?
“Service Discovery Protocols (SDP) are
network protocols which allow automatic
detection of devices and services offered by
these devices on a computer network.”
Source: http://en.wikipedia.org/wiki/Service_discovery
![Page 6: Docker Service Registration and Discovery](https://reader031.vdocument.in/reader031/viewer/2022020208/55a687031a28ab391e8b4588/html5/thumbnails/6.jpg)
Service Registration and Discovery
What is it?
Basically… how do clients determine the
network details of desired services.
![Page 7: Docker Service Registration and Discovery](https://reader031.vdocument.in/reader031/viewer/2022020208/55a687031a28ab391e8b4588/html5/thumbnails/7.jpg)
Service Registration and Discovery
Existing techniques?
• DNS (perhaps Dynamic DNS)
• Puppet exported resources
• Chef Search
• many others
![Page 8: Docker Service Registration and Discovery](https://reader031.vdocument.in/reader031/viewer/2022020208/55a687031a28ab391e8b4588/html5/thumbnails/8.jpg)
Service Registration and Discovery
These approaches can struggle in a docker world
Image source: http://www.ideachampions.com/weblogs/archives/2011/09/the_top_100_lam.shtml
![Page 9: Docker Service Registration and Discovery](https://reader031.vdocument.in/reader031/viewer/2022020208/55a687031a28ab391e8b4588/html5/thumbnails/9.jpg)
Service Registration and Discovery
These approaches can struggle in a docker world
• DNS
![Page 10: Docker Service Registration and Discovery](https://reader031.vdocument.in/reader031/viewer/2022020208/55a687031a28ab391e8b4588/html5/thumbnails/10.jpg)
Service Registration and Discovery
These approaches can struggle in a docker world
• DNS
– TTL and caching can often be an issue
![Page 11: Docker Service Registration and Discovery](https://reader031.vdocument.in/reader031/viewer/2022020208/55a687031a28ab391e8b4588/html5/thumbnails/11.jpg)
Service Registration and Discovery
These approaches can struggle in a docker world
• DNS
– TTL and caching can often be an issue
– Just try doing interesting DNS things in an enterprise.
![Page 12: Docker Service Registration and Discovery](https://reader031.vdocument.in/reader031/viewer/2022020208/55a687031a28ab391e8b4588/html5/thumbnails/12.jpg)
Service Registration and Discovery
These approaches can struggle in a docker world
• DNS
– TTL and caching can often be an issue
– Just try doing interesting DNS things in an enterprise.
– Containers don’t have a unique IP address. Simple IP
based DNS isn’t sufficient. *
* SRV records can be an option
![Page 13: Docker Service Registration and Discovery](https://reader031.vdocument.in/reader031/viewer/2022020208/55a687031a28ab391e8b4588/html5/thumbnails/13.jpg)
Service Registration and Discovery
These approaches can struggle in a docker world
• DNS– TTL and caching can often be an issue
– Just try doing interesting DNS things in an enterprise.
– Containers don’t have a unique IP address. Simple IP based DNS isn’t sufficient. *
• Config Management agents
* SRV records can be an option
![Page 14: Docker Service Registration and Discovery](https://reader031.vdocument.in/reader031/viewer/2022020208/55a687031a28ab391e8b4588/html5/thumbnails/14.jpg)
Service Registration and Discovery
These approaches can struggle in a docker world
• DNS– TTL and caching can often be an issue
– Just try doing interesting DNS things in an enterprise.
– Containers don’t have a unique IP address. Simple IP based DNS isn’t sufficient. *
• Config Management agents– I’m resistant to running agents within a container.
– CM agent run intervals are probably not frequent enough.
* SRV records can be an option
![Page 15: Docker Service Registration and Discovery](https://reader031.vdocument.in/reader031/viewer/2022020208/55a687031a28ab391e8b4588/html5/thumbnails/15.jpg)
Service Registration and Discovery
Are there any new tools for Services Discovery?
![Page 16: Docker Service Registration and Discovery](https://reader031.vdocument.in/reader031/viewer/2022020208/55a687031a28ab391e8b4588/html5/thumbnails/16.jpg)
Service Registration and Discovery
• Zookeeper
• Doozer
• Etcd
• SmartStack
• Eureka
• SkyDNS
• consul
Yep. Quite a few…
![Page 17: Docker Service Registration and Discovery](https://reader031.vdocument.in/reader031/viewer/2022020208/55a687031a28ab391e8b4588/html5/thumbnails/17.jpg)
Service Registration and Discovery
• Zookeeper
• Doozer
• Etcd
• SmartStack
• Eureka
• SkyDNS
• consul
Yep. Quite a few…
![Page 18: Docker Service Registration and Discovery](https://reader031.vdocument.in/reader031/viewer/2022020208/55a687031a28ab391e8b4588/html5/thumbnails/18.jpg)
Consul
“Service Discovery and
configuration made easy.
Distributed, highly available
and datacenter aware.”
![Page 19: Docker Service Registration and Discovery](https://reader031.vdocument.in/reader031/viewer/2022020208/55a687031a28ab391e8b4588/html5/thumbnails/19.jpg)
Consul
Key features…
• Service Discovery
• Health Checking
• Key/Value Storage
• Multi Datacenter
![Page 20: Docker Service Registration and Discovery](https://reader031.vdocument.in/reader031/viewer/2022020208/55a687031a28ab391e8b4588/html5/thumbnails/20.jpg)
Consul
Key features…
• Service Discovery
• Health Checking
• Key/Value Storage
• Multi Datacenter
![Page 21: Docker Service Registration and Discovery](https://reader031.vdocument.in/reader031/viewer/2022020208/55a687031a28ab391e8b4588/html5/thumbnails/21.jpg)
Consul - example
![Page 22: Docker Service Registration and Discovery](https://reader031.vdocument.in/reader031/viewer/2022020208/55a687031a28ab391e8b4588/html5/thumbnails/22.jpg)
Consul - example
Consul service definition for nginx web server
![Page 23: Docker Service Registration and Discovery](https://reader031.vdocument.in/reader031/viewer/2022020208/55a687031a28ab391e8b4588/html5/thumbnails/23.jpg)
Consul + Docker
How should I use consul with docker?
![Page 24: Docker Service Registration and Discovery](https://reader031.vdocument.in/reader031/viewer/2022020208/55a687031a28ab391e8b4588/html5/thumbnails/24.jpg)
Consul + Docker
How should I use consul with docker?
Option 1: run 1 Consul client agent in every
container to register its service
Or
Option 2: run 1 consul client agent per Docker
host that is responsible for all the containers on it
![Page 25: Docker Service Registration and Discovery](https://reader031.vdocument.in/reader031/viewer/2022020208/55a687031a28ab391e8b4588/html5/thumbnails/25.jpg)
Consul + Docker
How should I use consul with docker?
Option 1: run 1 Consul client agent in every
container to register its service
Or
Option 2: run 1 consul client agent per Docker
host that is responsible for all the containers on it
Option 2 makes more sense to me
![Page 26: Docker Service Registration and Discovery](https://reader031.vdocument.in/reader031/viewer/2022020208/55a687031a28ab391e8b4588/html5/thumbnails/26.jpg)
Consul + Docker
Docker host running some containers
![Page 27: Docker Service Registration and Discovery](https://reader031.vdocument.in/reader031/viewer/2022020208/55a687031a28ab391e8b4588/html5/thumbnails/27.jpg)
Consul + Docker
Add a Consul Agent as a container
![Page 28: Docker Service Registration and Discovery](https://reader031.vdocument.in/reader031/viewer/2022020208/55a687031a28ab391e8b4588/html5/thumbnails/28.jpg)
Consul + Docker
Which talks to the consul cluster
![Page 29: Docker Service Registration and Discovery](https://reader031.vdocument.in/reader031/viewer/2022020208/55a687031a28ab391e8b4588/html5/thumbnails/29.jpg)
Consul + Docker
Next Question?
How do we register the
container services with the
consul agent?
![Page 30: Docker Service Registration and Discovery](https://reader031.vdocument.in/reader031/viewer/2022020208/55a687031a28ab391e8b4588/html5/thumbnails/30.jpg)
Consul + Docker
Question?
How do we register the
container services with the
consul agent?
Use
Registrator!
![Page 31: Docker Service Registration and Discovery](https://reader031.vdocument.in/reader031/viewer/2022020208/55a687031a28ab391e8b4588/html5/thumbnails/31.jpg)
Registrator
• https://github.com/gliderlabs/registrator
• Automatically registers/deregisters services for docker
containers
• Supported Registry services include consul, etcd and SkyDNS
2
• Reads events from Docker Socket (/var/run/docker.sock)
• Run on each Docker host*
* Ideally run it as a container
![Page 32: Docker Service Registration and Discovery](https://reader031.vdocument.in/reader031/viewer/2022020208/55a687031a28ab391e8b4588/html5/thumbnails/32.jpg)
Registrator
Installation instructions
![Page 33: Docker Service Registration and Discovery](https://reader031.vdocument.in/reader031/viewer/2022020208/55a687031a28ab391e8b4588/html5/thumbnails/33.jpg)
Registrator
Installation instructions
![Page 34: Docker Service Registration and Discovery](https://reader031.vdocument.in/reader031/viewer/2022020208/55a687031a28ab391e8b4588/html5/thumbnails/34.jpg)
Registrator
Starting Registrator
![Page 35: Docker Service Registration and Discovery](https://reader031.vdocument.in/reader031/viewer/2022020208/55a687031a28ab391e8b4588/html5/thumbnails/35.jpg)
Registrator
Starting Registrator
![Page 36: Docker Service Registration and Discovery](https://reader031.vdocument.in/reader031/viewer/2022020208/55a687031a28ab391e8b4588/html5/thumbnails/36.jpg)
Registrator
Starting Registrator
pass docker socket to container
Specify service registry endpoint (eg consul)
![Page 37: Docker Service Registration and Discovery](https://reader031.vdocument.in/reader031/viewer/2022020208/55a687031a28ab391e8b4588/html5/thumbnails/37.jpg)
How does it work
![Page 38: Docker Service Registration and Discovery](https://reader031.vdocument.in/reader031/viewer/2022020208/55a687031a28ab391e8b4588/html5/thumbnails/38.jpg)
How does it work
• Start containers for consul
and registrator
![Page 39: Docker Service Registration and Discovery](https://reader031.vdocument.in/reader031/viewer/2022020208/55a687031a28ab391e8b4588/html5/thumbnails/39.jpg)
How does it work
• Start containers for consul
and registrator
• Registrator watches for
container start and die
events
![Page 40: Docker Service Registration and Discovery](https://reader031.vdocument.in/reader031/viewer/2022020208/55a687031a28ab391e8b4588/html5/thumbnails/40.jpg)
How does it work
• Start containers for consul
and registrator
• Registrator watches for
container start and die
events
• Start new container
![Page 41: Docker Service Registration and Discovery](https://reader031.vdocument.in/reader031/viewer/2022020208/55a687031a28ab391e8b4588/html5/thumbnails/41.jpg)
How does it work
• Start containers for consul
and registrator
• Registrator watches for
container start and die
events
• Start new container
• Registrator informs consul
of service A’s metadata
(IP/port/name/tag)
![Page 42: Docker Service Registration and Discovery](https://reader031.vdocument.in/reader031/viewer/2022020208/55a687031a28ab391e8b4588/html5/thumbnails/42.jpg)
Service information is now in
Consul. How can we access it?
![Page 43: Docker Service Registration and Discovery](https://reader031.vdocument.in/reader031/viewer/2022020208/55a687031a28ab391e8b4588/html5/thumbnails/43.jpg)
• Consul’s HTTP API
• Consul’s DNS API
• Consul-template (via HTTP
API)
Consul Access
![Page 44: Docker Service Registration and Discovery](https://reader031.vdocument.in/reader031/viewer/2022020208/55a687031a28ab391e8b4588/html5/thumbnails/44.jpg)
• Consul’s HTTP API
• Consul’s DNS API
• Consul-template (via HTTP
API)
Consul Access
![Page 45: Docker Service Registration and Discovery](https://reader031.vdocument.in/reader031/viewer/2022020208/55a687031a28ab391e8b4588/html5/thumbnails/45.jpg)
Consul-template
What does it do?
![Page 46: Docker Service Registration and Discovery](https://reader031.vdocument.in/reader031/viewer/2022020208/55a687031a28ab391e8b4588/html5/thumbnails/46.jpg)
Consul-template
What does it do?
• Populate template files with data from
Consul.
• Optionally run commands when consul
data changes.
![Page 47: Docker Service Registration and Discovery](https://reader031.vdocument.in/reader031/viewer/2022020208/55a687031a28ab391e8b4588/html5/thumbnails/47.jpg)
Consul-template – example 1
![Page 48: Docker Service Registration and Discovery](https://reader031.vdocument.in/reader031/viewer/2022020208/55a687031a28ab391e8b4588/html5/thumbnails/48.jpg)
Consul-template – example 1
![Page 49: Docker Service Registration and Discovery](https://reader031.vdocument.in/reader031/viewer/2022020208/55a687031a28ab391e8b4588/html5/thumbnails/49.jpg)
Consul-template – example 2
![Page 50: Docker Service Registration and Discovery](https://reader031.vdocument.in/reader031/viewer/2022020208/55a687031a28ab391e8b4588/html5/thumbnails/50.jpg)
Consul-template – example 2
![Page 51: Docker Service Registration and Discovery](https://reader031.vdocument.in/reader031/viewer/2022020208/55a687031a28ab391e8b4588/html5/thumbnails/51.jpg)
Time for a Demo
![Page 52: Docker Service Registration and Discovery](https://reader031.vdocument.in/reader031/viewer/2022020208/55a687031a28ab391e8b4588/html5/thumbnails/52.jpg)
links
• https://consul.io/
• https://github.com/hashicorp/consul-template
• https://github.com/gliderlabs/registrator
• https://github.com/CiscoCloud/microservices-
infrastructure
• http://jasonwilder.com/blog/2014/02/04/service-
discovery-in-the-cloud/
![Page 53: Docker Service Registration and Discovery](https://reader031.vdocument.in/reader031/viewer/2022020208/55a687031a28ab391e8b4588/html5/thumbnails/53.jpg)
Questions?
Thanks for listening
@m_richo