node collaboration - exported resources and puppetdb
DESCRIPTION
Node Collaboration - How can your servers share information with each other. Exploring Exported Resources, PuppetDB and other methods. This talk was given at Sydney Puppet Users Meetup on 14/08/2014.TRANSCRIPT
![Page 1: Node collaboration - Exported Resources and PuppetDB](https://reader035.vdocument.in/reader035/viewer/2022081516/54b767484a79595d768b45f5/html5/thumbnails/1.jpg)
Node collaboration
How can your servers share information with each other?
Michael Richardson@m_richo
![Page 2: Node collaboration - Exported Resources and PuppetDB](https://reader035.vdocument.in/reader035/viewer/2022081516/54b767484a79595d768b45f5/html5/thumbnails/2.jpg)
Warning!!
This talk contains both Facts and Opinions*.
*don’t believe everything I say.
![Page 3: Node collaboration - Exported Resources and PuppetDB](https://reader035.vdocument.in/reader035/viewer/2022081516/54b767484a79595d768b45f5/html5/thumbnails/3.jpg)
Puppet’s answer
![Page 4: Node collaboration - Exported Resources and PuppetDB](https://reader035.vdocument.in/reader035/viewer/2022081516/54b767484a79595d768b45f5/html5/thumbnails/4.jpg)
Puppet’s answer
Exported Resources!
![Page 5: Node collaboration - Exported Resources and PuppetDB](https://reader035.vdocument.in/reader035/viewer/2022081516/54b767484a79595d768b45f5/html5/thumbnails/5.jpg)
Exported Resources
* https://docs.puppetlabs.com/puppet/latest/reference/lang_exported.html
Definition:An exported resource declaration specifies a desired state
for a resource, does not manage the resource on the target system, and publishes the resource for use by other nodes. Any node (including the node that exported it) can
then collect the exported resource and manage its own copy of it. *
![Page 6: Node collaboration - Exported Resources and PuppetDB](https://reader035.vdocument.in/reader035/viewer/2022081516/54b767484a79595d768b45f5/html5/thumbnails/6.jpg)
Exported Resources
![Page 7: Node collaboration - Exported Resources and PuppetDB](https://reader035.vdocument.in/reader035/viewer/2022081516/54b767484a79595d768b45f5/html5/thumbnails/7.jpg)
Exported Resources
For those that can’t afford DNS servers…
![Page 8: Node collaboration - Exported Resources and PuppetDB](https://reader035.vdocument.in/reader035/viewer/2022081516/54b767484a79595d768b45f5/html5/thumbnails/8.jpg)
Exported Resources
Common Examples• Backend servers configured in a load balancer• Monitoring Servers updated with monitoring
clients (nagios_host / nagios_service)• Distribute public keys (ssh and openssl)
![Page 9: Node collaboration - Exported Resources and PuppetDB](https://reader035.vdocument.in/reader035/viewer/2022081516/54b767484a79595d768b45f5/html5/thumbnails/9.jpg)
Exported Resources
Question for the audience
![Page 10: Node collaboration - Exported Resources and PuppetDB](https://reader035.vdocument.in/reader035/viewer/2022081516/54b767484a79595d768b45f5/html5/thumbnails/10.jpg)
Exported Resources
Question for the audience
What do you think of Exported Resources?
![Page 11: Node collaboration - Exported Resources and PuppetDB](https://reader035.vdocument.in/reader035/viewer/2022081516/54b767484a79595d768b45f5/html5/thumbnails/11.jpg)
Exported Resources
Question for the audience
What do you think of Exported Resources?
Great. Alright. So so.Let’s explore alternatives
![Page 12: Node collaboration - Exported Resources and PuppetDB](https://reader035.vdocument.in/reader035/viewer/2022081516/54b767484a79595d768b45f5/html5/thumbnails/12.jpg)
What about our friends @
![Page 13: Node collaboration - Exported Resources and PuppetDB](https://reader035.vdocument.in/reader035/viewer/2022081516/54b767484a79595d768b45f5/html5/thumbnails/13.jpg)
Ohai Node data (think facter) is automatically added, indexed and
searchable in the Chef Server.
No need to export facts or resources.
![Page 14: Node collaboration - Exported Resources and PuppetDB](https://reader035.vdocument.in/reader035/viewer/2022081516/54b767484a79595d768b45f5/html5/thumbnails/14.jpg)
How to use Chef Search
Query the the Chef server data via
• Method within recipe (think puppet manifest)• Search subcommand of ‘knife’• /search or /search/INDEX endpoints of Chef
Server API
![Page 15: Node collaboration - Exported Resources and PuppetDB](https://reader035.vdocument.in/reader035/viewer/2022081516/54b767484a79595d768b45f5/html5/thumbnails/15.jpg)
How to use Chef Search
1. Method within recipe (think puppet manifest)
![Page 16: Node collaboration - Exported Resources and PuppetDB](https://reader035.vdocument.in/reader035/viewer/2022081516/54b767484a79595d768b45f5/html5/thumbnails/16.jpg)
Chef Search
What do people think?
![Page 17: Node collaboration - Exported Resources and PuppetDB](https://reader035.vdocument.in/reader035/viewer/2022081516/54b767484a79595d768b45f5/html5/thumbnails/17.jpg)
Chef Search
What do people think?
Personally I feel this approach can sometimes be better than exported resources approach. *
* Remember this is an opinion
![Page 18: Node collaboration - Exported Resources and PuppetDB](https://reader035.vdocument.in/reader035/viewer/2022081516/54b767484a79595d768b45f5/html5/thumbnails/18.jpg)
But what about Puppet?
![Page 19: Node collaboration - Exported Resources and PuppetDB](https://reader035.vdocument.in/reader035/viewer/2022081516/54b767484a79595d768b45f5/html5/thumbnails/19.jpg)
Introducing PuppetDB
![Page 20: Node collaboration - Exported Resources and PuppetDB](https://reader035.vdocument.in/reader035/viewer/2022081516/54b767484a79595d768b45f5/html5/thumbnails/20.jpg)
PuppetDB
Definition:collects data generated by Puppet. It enables
advanced Puppet features like the inventory service and exported resources, and can be the foundation
for other applications that use Puppet’s data. *
https://docs.puppetlabs.com/puppetdb/2.1/index.html
![Page 21: Node collaboration - Exported Resources and PuppetDB](https://reader035.vdocument.in/reader035/viewer/2022081516/54b767484a79595d768b45f5/html5/thumbnails/21.jpg)
PuppetDB
• Written in Clojure.• Runs on the JVM• Backed by PostgreSQL• Scales horizontally
![Page 22: Node collaboration - Exported Resources and PuppetDB](https://reader035.vdocument.in/reader035/viewer/2022081516/54b767484a79595d768b45f5/html5/thumbnails/22.jpg)
PuppetDB
PuppetDB stores:• The most recent facts from every node• The most recent catalog for every node• Optionally, 14 days of event reports for
every node• Exported Resources
![Page 23: Node collaboration - Exported Resources and PuppetDB](https://reader035.vdocument.in/reader035/viewer/2022081516/54b767484a79595d768b45f5/html5/thumbnails/23.jpg)
PuppetDB
PuppetDB REST APIV3 API includes the following endpoints:
• Facts• Resources• Nodes• Fact-names,• Metrics
• Reports• Events• And more
![Page 24: Node collaboration - Exported Resources and PuppetDB](https://reader035.vdocument.in/reader035/viewer/2022081516/54b767484a79595d768b45f5/html5/thumbnails/24.jpg)
PuppetDBGET /v3/nodes/
![Page 25: Node collaboration - Exported Resources and PuppetDB](https://reader035.vdocument.in/reader035/viewer/2022081516/54b767484a79595d768b45f5/html5/thumbnails/25.jpg)
PuppetDBGET /v3/nodes/
QUERY = (JSON array of queries)
![Page 26: Node collaboration - Exported Resources and PuppetDB](https://reader035.vdocument.in/reader035/viewer/2022081516/54b767484a79595d768b45f5/html5/thumbnails/26.jpg)
PuppetDBGET /v3/nodes/<NODE>/facts
Return all facts for given node
![Page 27: Node collaboration - Exported Resources and PuppetDB](https://reader035.vdocument.in/reader035/viewer/2022081516/54b767484a79595d768b45f5/html5/thumbnails/27.jpg)
PuppetDBGET /v3/nodes/<NODE>/facts/<NAME>Return facts by their name
![Page 28: Node collaboration - Exported Resources and PuppetDB](https://reader035.vdocument.in/reader035/viewer/2022081516/54b767484a79595d768b45f5/html5/thumbnails/28.jpg)
PuppetDB
I know what you’re thinkingSo much awesome information
How can I make use of it?
![Page 29: Node collaboration - Exported Resources and PuppetDB](https://reader035.vdocument.in/reader035/viewer/2022081516/54b767484a79595d768b45f5/html5/thumbnails/29.jpg)
PuppetDB
checkoutPuppetdbquery module
by Erik Dalén
https://forge.puppetlabs.com/dalen/puppetdbqueryhttps://github.com/dalen/puppet-puppetdbquery
![Page 30: Node collaboration - Exported Resources and PuppetDB](https://reader035.vdocument.in/reader035/viewer/2022081516/54b767484a79595d768b45f5/html5/thumbnails/30.jpg)
PuppetDB
Puppetdbquery module• Command line tools• Puppet functions to query PuppetDB• Hiera backend to return query results from
PuppetDB
![Page 31: Node collaboration - Exported Resources and PuppetDB](https://reader035.vdocument.in/reader035/viewer/2022081516/54b767484a79595d768b45f5/html5/thumbnails/31.jpg)
PuppetDBPuppetdbquery CLI - nodes
facts
resources
![Page 32: Node collaboration - Exported Resources and PuppetDB](https://reader035.vdocument.in/reader035/viewer/2022081516/54b767484a79595d768b45f5/html5/thumbnails/32.jpg)
PuppetDBPuppetdbquery CLI – facts
![Page 33: Node collaboration - Exported Resources and PuppetDB](https://reader035.vdocument.in/reader035/viewer/2022081516/54b767484a79595d768b45f5/html5/thumbnails/33.jpg)
PuppetDBPuppetdbquery CLI – facts
factsquery
![Page 34: Node collaboration - Exported Resources and PuppetDB](https://reader035.vdocument.in/reader035/viewer/2022081516/54b767484a79595d768b45f5/html5/thumbnails/34.jpg)
PuppetDBPuppetdbquery functions in manifests
• query_nodes
• query_facts
![Page 35: Node collaboration - Exported Resources and PuppetDB](https://reader035.vdocument.in/reader035/viewer/2022081516/54b767484a79595d768b45f5/html5/thumbnails/35.jpg)
PuppetDBPuppetdbquery functionsquery fact
![Page 36: Node collaboration - Exported Resources and PuppetDB](https://reader035.vdocument.in/reader035/viewer/2022081516/54b767484a79595d768b45f5/html5/thumbnails/36.jpg)
PuppetDBPuppetdbquery functions
note: Output is abbreviated
array
hash
![Page 37: Node collaboration - Exported Resources and PuppetDB](https://reader035.vdocument.in/reader035/viewer/2022081516/54b767484a79595d768b45f5/html5/thumbnails/37.jpg)
Where to from here?
Exported Resourcesvs
PuppetDB + puppetdbquery
![Page 38: Node collaboration - Exported Resources and PuppetDB](https://reader035.vdocument.in/reader035/viewer/2022081516/54b767484a79595d768b45f5/html5/thumbnails/38.jpg)
Where to from here?What about something
completely different
![Page 39: Node collaboration - Exported Resources and PuppetDB](https://reader035.vdocument.in/reader035/viewer/2022081516/54b767484a79595d768b45f5/html5/thumbnails/39.jpg)
Where to from here?What about something
completely different
• Confd• Etcd• Consul
• Zookeeper• Doozer• Skydns
![Page 40: Node collaboration - Exported Resources and PuppetDB](https://reader035.vdocument.in/reader035/viewer/2022081516/54b767484a79595d768b45f5/html5/thumbnails/40.jpg)
Where to from here?What about something
completely different
• Confd• Etcd• Consul
• Zookeeper• Doozer• Skydns
Really interesting space… but that’s for another talk.
![Page 41: Node collaboration - Exported Resources and PuppetDB](https://reader035.vdocument.in/reader035/viewer/2022081516/54b767484a79595d768b45f5/html5/thumbnails/41.jpg)
SummaryHow can you share information between your nodes/servers?
![Page 42: Node collaboration - Exported Resources and PuppetDB](https://reader035.vdocument.in/reader035/viewer/2022081516/54b767484a79595d768b45f5/html5/thumbnails/42.jpg)
SummaryHow can you share information between your nodes/servers?
Plenty of ways• Exported resources• PuppetDB• Many, many others.
![Page 43: Node collaboration - Exported Resources and PuppetDB](https://reader035.vdocument.in/reader035/viewer/2022081516/54b767484a79595d768b45f5/html5/thumbnails/43.jpg)
SummaryHow can you share information between your nodes/servers?
Plenty of ways• Exported resources• PuppetDB• Many, many others.
Pick the method that works best for you
![Page 44: Node collaboration - Exported Resources and PuppetDB](https://reader035.vdocument.in/reader035/viewer/2022081516/54b767484a79595d768b45f5/html5/thumbnails/44.jpg)
Thank you
![Page 45: Node collaboration - Exported Resources and PuppetDB](https://reader035.vdocument.in/reader035/viewer/2022081516/54b767484a79595d768b45f5/html5/thumbnails/45.jpg)
Thank you
#protipgo buy this book nowBy Alessandro FranceschiIt is excellent!
![Page 46: Node collaboration - Exported Resources and PuppetDB](https://reader035.vdocument.in/reader035/viewer/2022081516/54b767484a79595d768b45f5/html5/thumbnails/46.jpg)
IS HIRING
![Page 47: Node collaboration - Exported Resources and PuppetDB](https://reader035.vdocument.in/reader035/viewer/2022081516/54b767484a79595d768b45f5/html5/thumbnails/47.jpg)
QUESTIONS?