cloud postgresql automation management with...
TRANSCRIPT
![Page 1: Cloud PostgreSQL Automation Management with Ansibleinfo.citusdata.com/rs/235-CNE-301/images/Cloud... · Simple systems automation/orchestration framework Easy to learn and use Little](https://reader033.vdocument.in/reader033/viewer/2022060512/5f2ae336638fc942116469b6/html5/thumbnails/1.jpg)
Cloud PostgreSQL Automation Management with Ansible
Kenny GormanChief Technologist; Data
Rackspace
@rackspace @kennygorman
![Page 2: Cloud PostgreSQL Automation Management with Ansibleinfo.citusdata.com/rs/235-CNE-301/images/Cloud... · Simple systems automation/orchestration framework Easy to learn and use Little](https://reader033.vdocument.in/reader033/viewer/2022060512/5f2ae336638fc942116469b6/html5/thumbnails/2.jpg)
My Background
● Database Engineer, Developer, DBA, Architect, Founder, Chief
Technologist
● Oracle, MySQL, PostgreSQL, MongoDB, Apache Spark
● Ran a very large PG implementation at Hi5
● Contributed to early pg_reorg tooling
● Co-Founded ObjectRocket; MongoDB as a Service
![Page 3: Cloud PostgreSQL Automation Management with Ansibleinfo.citusdata.com/rs/235-CNE-301/images/Cloud... · Simple systems automation/orchestration framework Easy to learn and use Little](https://reader033.vdocument.in/reader033/viewer/2022060512/5f2ae336638fc942116469b6/html5/thumbnails/3.jpg)
You have more data than yesterday
![Page 4: Cloud PostgreSQL Automation Management with Ansibleinfo.citusdata.com/rs/235-CNE-301/images/Cloud... · Simple systems automation/orchestration framework Easy to learn and use Little](https://reader033.vdocument.in/reader033/viewer/2022060512/5f2ae336638fc942116469b6/html5/thumbnails/4.jpg)
![Page 5: Cloud PostgreSQL Automation Management with Ansibleinfo.citusdata.com/rs/235-CNE-301/images/Cloud... · Simple systems automation/orchestration framework Easy to learn and use Little](https://reader033.vdocument.in/reader033/viewer/2022060512/5f2ae336638fc942116469b6/html5/thumbnails/5.jpg)
Simple systems automation/orchestration framework
Easy to learn and use
Little overhead and footprint
Parallel execution
Ansible is
![Page 6: Cloud PostgreSQL Automation Management with Ansibleinfo.citusdata.com/rs/235-CNE-301/images/Cloud... · Simple systems automation/orchestration framework Easy to learn and use Little](https://reader033.vdocument.in/reader033/viewer/2022060512/5f2ae336638fc942116469b6/html5/thumbnails/6.jpg)
● Written in python
● Agentless/Daemon-less
● OpenSSH
● Idempotent
Ansible attributes
![Page 7: Cloud PostgreSQL Automation Management with Ansibleinfo.citusdata.com/rs/235-CNE-301/images/Cloud... · Simple systems automation/orchestration framework Easy to learn and use Little](https://reader033.vdocument.in/reader033/viewer/2022060512/5f2ae336638fc942116469b6/html5/thumbnails/7.jpg)
● Human readable● Machine readable
Ansible is
![Page 8: Cloud PostgreSQL Automation Management with Ansibleinfo.citusdata.com/rs/235-CNE-301/images/Cloud... · Simple systems automation/orchestration framework Easy to learn and use Little](https://reader033.vdocument.in/reader033/viewer/2022060512/5f2ae336638fc942116469b6/html5/thumbnails/8.jpg)
● pip/yum/apt-get or repo
● SSH keys
● yaml files
Setup
![Page 9: Cloud PostgreSQL Automation Management with Ansibleinfo.citusdata.com/rs/235-CNE-301/images/Cloud... · Simple systems automation/orchestration framework Easy to learn and use Little](https://reader033.vdocument.in/reader033/viewer/2022060512/5f2ae336638fc942116469b6/html5/thumbnails/9.jpg)
● Config: /etc/ansible/ansible.cfg
● Inventory: /etc/ansible/hosts
● Command line interface
● Modules
● Playbooks
Components
![Page 10: Cloud PostgreSQL Automation Management with Ansibleinfo.citusdata.com/rs/235-CNE-301/images/Cloud... · Simple systems automation/orchestration framework Easy to learn and use Little](https://reader033.vdocument.in/reader033/viewer/2022060512/5f2ae336638fc942116469b6/html5/thumbnails/10.jpg)
● /etc/ansible/hosts
Inventory
![Page 11: Cloud PostgreSQL Automation Management with Ansibleinfo.citusdata.com/rs/235-CNE-301/images/Cloud... · Simple systems automation/orchestration framework Easy to learn and use Little](https://reader033.vdocument.in/reader033/viewer/2022060512/5f2ae336638fc942116469b6/html5/thumbnails/11.jpg)
● Easy command line interface
Command Line
![Page 12: Cloud PostgreSQL Automation Management with Ansibleinfo.citusdata.com/rs/235-CNE-301/images/Cloud... · Simple systems automation/orchestration framework Easy to learn and use Little](https://reader033.vdocument.in/reader033/viewer/2022060512/5f2ae336638fc942116469b6/html5/thumbnails/12.jpg)
● Core vs User● Ansible Galaxy
Modules
![Page 13: Cloud PostgreSQL Automation Management with Ansibleinfo.citusdata.com/rs/235-CNE-301/images/Cloud... · Simple systems automation/orchestration framework Easy to learn and use Little](https://reader033.vdocument.in/reader033/viewer/2022060512/5f2ae336638fc942116469b6/html5/thumbnails/13.jpg)
● TemplatePlaybooks
![Page 14: Cloud PostgreSQL Automation Management with Ansibleinfo.citusdata.com/rs/235-CNE-301/images/Cloud... · Simple systems automation/orchestration framework Easy to learn and use Little](https://reader033.vdocument.in/reader033/viewer/2022060512/5f2ae336638fc942116469b6/html5/thumbnails/14.jpg)
Ansible + PostgreSQL
![Page 15: Cloud PostgreSQL Automation Management with Ansibleinfo.citusdata.com/rs/235-CNE-301/images/Cloud... · Simple systems automation/orchestration framework Easy to learn and use Little](https://reader033.vdocument.in/reader033/viewer/2022060512/5f2ae336638fc942116469b6/html5/thumbnails/15.jpg)
● Do I need Ansible?
● Database Provisioning
● Database Management
● DBA Sanity
![Page 16: Cloud PostgreSQL Automation Management with Ansibleinfo.citusdata.com/rs/235-CNE-301/images/Cloud... · Simple systems automation/orchestration framework Easy to learn and use Little](https://reader033.vdocument.in/reader033/viewer/2022060512/5f2ae336638fc942116469b6/html5/thumbnails/16.jpg)
Idempotent
![Page 17: Cloud PostgreSQL Automation Management with Ansibleinfo.citusdata.com/rs/235-CNE-301/images/Cloud... · Simple systems automation/orchestration framework Easy to learn and use Little](https://reader033.vdocument.in/reader033/viewer/2022060512/5f2ae336638fc942116469b6/html5/thumbnails/17.jpg)
Provisioning PG
![Page 18: Cloud PostgreSQL Automation Management with Ansibleinfo.citusdata.com/rs/235-CNE-301/images/Cloud... · Simple systems automation/orchestration framework Easy to learn and use Little](https://reader033.vdocument.in/reader033/viewer/2022060512/5f2ae336638fc942116469b6/html5/thumbnails/18.jpg)
…
![Page 19: Cloud PostgreSQL Automation Management with Ansibleinfo.citusdata.com/rs/235-CNE-301/images/Cloud... · Simple systems automation/orchestration framework Easy to learn and use Little](https://reader033.vdocument.in/reader033/viewer/2022060512/5f2ae336638fc942116469b6/html5/thumbnails/19.jpg)
Core: ● postgresql_db● postgresql_user● postgresql_privs● ...
User:● Ansible Galaxy● ansible_roles_pipelinedb
PostgreSQL Modules
![Page 20: Cloud PostgreSQL Automation Management with Ansibleinfo.citusdata.com/rs/235-CNE-301/images/Cloud... · Simple systems automation/orchestration framework Easy to learn and use Little](https://reader033.vdocument.in/reader033/viewer/2022060512/5f2ae336638fc942116469b6/html5/thumbnails/20.jpg)
Ops: Too many servers? Too little time? Too many mistakes?
![Page 21: Cloud PostgreSQL Automation Management with Ansibleinfo.citusdata.com/rs/235-CNE-301/images/Cloud... · Simple systems automation/orchestration framework Easy to learn and use Little](https://reader033.vdocument.in/reader033/viewer/2022060512/5f2ae336638fc942116469b6/html5/thumbnails/21.jpg)
● Ansible Management Host (or two)
● virtualenv/python/ansible
● git/github + your management code
● Inventory
● ssh key on every DB host
![Page 22: Cloud PostgreSQL Automation Management with Ansibleinfo.citusdata.com/rs/235-CNE-301/images/Cloud... · Simple systems automation/orchestration framework Easy to learn and use Little](https://reader033.vdocument.in/reader033/viewer/2022060512/5f2ae336638fc942116469b6/html5/thumbnails/22.jpg)
Ansible Management Host
US-EastPrivate DC
LON
![Page 23: Cloud PostgreSQL Automation Management with Ansibleinfo.citusdata.com/rs/235-CNE-301/images/Cloud... · Simple systems automation/orchestration framework Easy to learn and use Little](https://reader033.vdocument.in/reader033/viewer/2022060512/5f2ae336638fc942116469b6/html5/thumbnails/23.jpg)
Database Management
![Page 24: Cloud PostgreSQL Automation Management with Ansibleinfo.citusdata.com/rs/235-CNE-301/images/Cloud... · Simple systems automation/orchestration framework Easy to learn and use Little](https://reader033.vdocument.in/reader033/viewer/2022060512/5f2ae336638fc942116469b6/html5/thumbnails/24.jpg)
● start/stop
● pg_hba.conf hax
● upgrades
● user management
● schema uniformity
![Page 25: Cloud PostgreSQL Automation Management with Ansibleinfo.citusdata.com/rs/235-CNE-301/images/Cloud... · Simple systems automation/orchestration framework Easy to learn and use Little](https://reader033.vdocument.in/reader033/viewer/2022060512/5f2ae336638fc942116469b6/html5/thumbnails/25.jpg)
DB Management one liners
![Page 26: Cloud PostgreSQL Automation Management with Ansibleinfo.citusdata.com/rs/235-CNE-301/images/Cloud... · Simple systems automation/orchestration framework Easy to learn and use Little](https://reader033.vdocument.in/reader033/viewer/2022060512/5f2ae336638fc942116469b6/html5/thumbnails/26.jpg)
Are my DB machines up?
![Page 27: Cloud PostgreSQL Automation Management with Ansibleinfo.citusdata.com/rs/235-CNE-301/images/Cloud... · Simple systems automation/orchestration framework Easy to learn and use Little](https://reader033.vdocument.in/reader033/viewer/2022060512/5f2ae336638fc942116469b6/html5/thumbnails/27.jpg)
Is PG responding?
![Page 28: Cloud PostgreSQL Automation Management with Ansibleinfo.citusdata.com/rs/235-CNE-301/images/Cloud... · Simple systems automation/orchestration framework Easy to learn and use Little](https://reader033.vdocument.in/reader033/viewer/2022060512/5f2ae336638fc942116469b6/html5/thumbnails/28.jpg)
Make sure production indexes exist
![Page 29: Cloud PostgreSQL Automation Management with Ansibleinfo.citusdata.com/rs/235-CNE-301/images/Cloud... · Simple systems automation/orchestration framework Easy to learn and use Little](https://reader033.vdocument.in/reader033/viewer/2022060512/5f2ae336638fc942116469b6/html5/thumbnails/29.jpg)
Rolling new code/push to site
![Page 30: Cloud PostgreSQL Automation Management with Ansibleinfo.citusdata.com/rs/235-CNE-301/images/Cloud... · Simple systems automation/orchestration framework Easy to learn and use Little](https://reader033.vdocument.in/reader033/viewer/2022060512/5f2ae336638fc942116469b6/html5/thumbnails/30.jpg)
Are my DB getting smoked?
![Page 31: Cloud PostgreSQL Automation Management with Ansibleinfo.citusdata.com/rs/235-CNE-301/images/Cloud... · Simple systems automation/orchestration framework Easy to learn and use Little](https://reader033.vdocument.in/reader033/viewer/2022060512/5f2ae336638fc942116469b6/html5/thumbnails/31.jpg)
Triage
![Page 32: Cloud PostgreSQL Automation Management with Ansibleinfo.citusdata.com/rs/235-CNE-301/images/Cloud... · Simple systems automation/orchestration framework Easy to learn and use Little](https://reader033.vdocument.in/reader033/viewer/2022060512/5f2ae336638fc942116469b6/html5/thumbnails/32.jpg)
Whats is the slow query?
![Page 33: Cloud PostgreSQL Automation Management with Ansibleinfo.citusdata.com/rs/235-CNE-301/images/Cloud... · Simple systems automation/orchestration framework Easy to learn and use Little](https://reader033.vdocument.in/reader033/viewer/2022060512/5f2ae336638fc942116469b6/html5/thumbnails/33.jpg)
kill it
![Page 34: Cloud PostgreSQL Automation Management with Ansibleinfo.citusdata.com/rs/235-CNE-301/images/Cloud... · Simple systems automation/orchestration framework Easy to learn and use Little](https://reader033.vdocument.in/reader033/viewer/2022060512/5f2ae336638fc942116469b6/html5/thumbnails/34.jpg)
Gettin’ jiggy with Ansible
![Page 35: Cloud PostgreSQL Automation Management with Ansibleinfo.citusdata.com/rs/235-CNE-301/images/Cloud... · Simple systems automation/orchestration framework Easy to learn and use Little](https://reader033.vdocument.in/reader033/viewer/2022060512/5f2ae336638fc942116469b6/html5/thumbnails/35.jpg)
● pg_shard
● backups, cloning
● replica creation
● performance tuning
![Page 36: Cloud PostgreSQL Automation Management with Ansibleinfo.citusdata.com/rs/235-CNE-301/images/Cloud... · Simple systems automation/orchestration framework Easy to learn and use Little](https://reader033.vdocument.in/reader033/viewer/2022060512/5f2ae336638fc942116469b6/html5/thumbnails/36.jpg)
DBA Sanity
![Page 37: Cloud PostgreSQL Automation Management with Ansibleinfo.citusdata.com/rs/235-CNE-301/images/Cloud... · Simple systems automation/orchestration framework Easy to learn and use Little](https://reader033.vdocument.in/reader033/viewer/2022060512/5f2ae336638fc942116469b6/html5/thumbnails/37.jpg)
● Use modules wherever you can, if there are none, make one. Standardize.
● Git and Github are powerful when combined with Ansible.
● Use the cloud. Setup Dynamic Inventory.
● Use ad-hoc and playbooks in unison selecting the right time to use each.