automated upgrades of postgresql clusters in cloudpostgres_old_dsn:...
TRANSCRIPT
![Page 1: Automated Upgrades of PostgreSQL Clusters in Cloudpostgres_old_dsn: "dbname={{pglupgrade_database}} host={{groups['old-primary'][0]}} user={{pglupgrade_user}}" ... Database clusters](https://reader033.vdocument.in/reader033/viewer/2022060308/5f0a04f87e708231d4299f68/html5/thumbnails/1.jpg)
Automated Upgrades ofPostgreSQL Clusters in Cloud
Gülçin Yıldırım
1
![Page 2: Automated Upgrades of PostgreSQL Clusters in Cloudpostgres_old_dsn: "dbname={{pglupgrade_database}} host={{groups['old-primary'][0]}} user={{pglupgrade_user}}" ... Database clusters](https://reader033.vdocument.in/reader033/viewer/2022060308/5f0a04f87e708231d4299f68/html5/thumbnails/2.jpg)
select * from me;Site Reliability Engineer @
Board Member @
MSc Comp. & Systems Eng. @
Writes on 2ndQuadrant
Does some childish
Loves independent films
From Turkey
2ndQuadrant
PostgreSQL Europe
Tallinn University of Technology
blog
paintings
@
Skype: gulcin2ndq
Github:
apatheticmagpie
gulcin 2
![Page 3: Automated Upgrades of PostgreSQL Clusters in Cloudpostgres_old_dsn: "dbname={{pglupgrade_database}} host={{groups['old-primary'][0]}} user={{pglupgrade_user}}" ... Database clusters](https://reader033.vdocument.in/reader033/viewer/2022060308/5f0a04f87e708231d4299f68/html5/thumbnails/3.jpg)
AgendaDatabase UpgradesWhy Logical Replication?Platform ImplementationCase Studies & ResultsApplicability & LimitationsConclusion
3
![Page 4: Automated Upgrades of PostgreSQL Clusters in Cloudpostgres_old_dsn: "dbname={{pglupgrade_database}} host={{groups['old-primary'][0]}} user={{pglupgrade_user}}" ... Database clusters](https://reader033.vdocument.in/reader033/viewer/2022060308/5f0a04f87e708231d4299f68/html5/thumbnails/4.jpg)
Problems?DOWNTIME
Revenue Loss Reputation Loss
HighAvailability?
SLAs? Low Capacity?
4
![Page 5: Automated Upgrades of PostgreSQL Clusters in Cloudpostgres_old_dsn: "dbname={{pglupgrade_database}} host={{groups['old-primary'][0]}} user={{pglupgrade_user}}" ... Database clusters](https://reader033.vdocument.in/reader033/viewer/2022060308/5f0a04f87e708231d4299f68/html5/thumbnails/5.jpg)
Database Anyone?Banks
Social NetworksDesktop Apps
Startups
Medium-sizeEnterprises
5
![Page 6: Automated Upgrades of PostgreSQL Clusters in Cloudpostgres_old_dsn: "dbname={{pglupgrade_database}} host={{groups['old-primary'][0]}} user={{pglupgrade_user}}" ... Database clusters](https://reader033.vdocument.in/reader033/viewer/2022060308/5f0a04f87e708231d4299f68/html5/thumbnails/6.jpg)
Upgrade,or not to Upgrade
New featuresSecurity patchesPerfomance UpdatesBug fixes
Outdated, no supportVulnerable to attacksPoor PerfomanceBuggy, hard to maintain6
![Page 7: Automated Upgrades of PostgreSQL Clusters in Cloudpostgres_old_dsn: "dbname={{pglupgrade_database}} host={{groups['old-primary'][0]}} user={{pglupgrade_user}}" ... Database clusters](https://reader033.vdocument.in/reader033/viewer/2022060308/5f0a04f87e708231d4299f68/html5/thumbnails/7.jpg)
Why Automate?
Risk & ErrorsCostTime-to-market
ReproducibilityRepeatabilityEfficiency
Updating nasa.gov: 1 hr to 5 minsPatching updates: Multi-day to 45 minsApplication stack setup: 1-2 hrs to 10 mins
7
![Page 8: Automated Upgrades of PostgreSQL Clusters in Cloudpostgres_old_dsn: "dbname={{pglupgrade_database}} host={{groups['old-primary'][0]}} user={{pglupgrade_user}}" ... Database clusters](https://reader033.vdocument.in/reader033/viewer/2022060308/5f0a04f87e708231d4299f68/html5/thumbnails/8.jpg)
Ansible Loves PostgreSQL( in the s )
Postgres Modules: 6AWS Modules: 100
PostgreSQL252
Database208
Cloud116
Upgrade50
8
![Page 9: Automated Upgrades of PostgreSQL Clusters in Cloudpostgres_old_dsn: "dbname={{pglupgrade_database}} host={{groups['old-primary'][0]}} user={{pglupgrade_user}}" ... Database clusters](https://reader033.vdocument.in/reader033/viewer/2022060308/5f0a04f87e708231d4299f68/html5/thumbnails/9.jpg)
Database Upgrades1 2
3 4
same or compatiblestorage formathard to guaranteeperformanceoptimization - datastructures
logical copy (dump)load into new servertraditional approachoffline (downtime)
convert data fromold format to newcan be online (perf?)offline (downtime)often shorter (2nd)
logical dump (restore)capture changes whileupgradereplicate after restoremin downtime
9
![Page 10: Automated Upgrades of PostgreSQL Clusters in Cloudpostgres_old_dsn: "dbname={{pglupgrade_database}} host={{groups['old-primary'][0]}} user={{pglupgrade_user}}" ... Database clusters](https://reader033.vdocument.in/reader033/viewer/2022060308/5f0a04f87e708231d4299f68/html5/thumbnails/10.jpg)
Logical Replication Rocks!Offline Conversion
pg_dump/pg_restorepg_upgrade
Online Conversion
pglogicalpglupgrade
1
210
![Page 11: Automated Upgrades of PostgreSQL Clusters in Cloudpostgres_old_dsn: "dbname={{pglupgrade_database}} host={{groups['old-primary'][0]}} user={{pglupgrade_user}}" ... Database clusters](https://reader033.vdocument.in/reader033/viewer/2022060308/5f0a04f87e708231d4299f68/html5/thumbnails/11.jpg)
Elements of the solutionpglogical pgbouncer
Ansible AWSpglupgrade 11
![Page 12: Automated Upgrades of PostgreSQL Clusters in Cloudpostgres_old_dsn: "dbname={{pglupgrade_database}} host={{groups['old-primary'][0]}} user={{pglupgrade_user}}" ... Database clusters](https://reader033.vdocument.in/reader033/viewer/2022060308/5f0a04f87e708231d4299f68/html5/thumbnails/12.jpg)
Pglupgrade playbook[old-primary]54.171.211.188
[new-primary]54.246.183.100
[old-standbys]54.77.249.8154.154.49.180
[new-standbys:children]old-standbys
[pgbouncer]54.154.49.180
$ ansible-playbook -i hosts.ini pglupgrade.yml
Inventory file host.ini
Running pglupgrade playbook
8 plays
config.yml
host.ini
orchestrates
the upgrade
operation
12
![Page 13: Automated Upgrades of PostgreSQL Clusters in Cloudpostgres_old_dsn: "dbname={{pglupgrade_database}} host={{groups['old-primary'][0]}} user={{pglupgrade_user}}" ... Database clusters](https://reader033.vdocument.in/reader033/viewer/2022060308/5f0a04f87e708231d4299f68/html5/thumbnails/13.jpg)
Pglupgrade playbookansible_user: admin
pglupgrade_user: pglupgradepglupgrade_pass: pglupgrade123pglupgrade_database: postgres
replica_user: postgresreplica_pass: ""
pgbouncer_user: pgbouncer
postgres_old_version: 9.5postgres_new_version: 9.6
subscription_name: upgradereplication_set: upgrade
initial_standbys: 1
postgres_old_dsn: "dbname={{pglupgrade_database}} host={{groups['old-primary'][0]}} user={{pglupgrade_user}}"postgres_new_dsn: "dbname={{pglupgrade_database}} host={{groups['new-primary'][0]}} user={{pglupgrade_user}}"
postgres_old_datadir: "/var/lib/postgresql/{{postgres_old_version}}/main"postgres_new_datadir: "/var/lib/postgresql/{{postgres_new_version}}/main"
postgres_new_confdir: "/etc/postgresql/{{postgres_new_version}}/main"
config.yml13
![Page 14: Automated Upgrades of PostgreSQL Clusters in Cloudpostgres_old_dsn: "dbname={{pglupgrade_database}} host={{groups['old-primary'][0]}} user={{pglupgrade_user}}" ... Database clusters](https://reader033.vdocument.in/reader033/viewer/2022060308/5f0a04f87e708231d4299f68/html5/thumbnails/14.jpg)
HowDoes ItWork?
1 2 3
4
5
6
7
8
9
10
11
12 13
14
15
16
14
![Page 15: Automated Upgrades of PostgreSQL Clusters in Cloudpostgres_old_dsn: "dbname={{pglupgrade_database}} host={{groups['old-primary'][0]}} user={{pglupgrade_user}}" ... Database clusters](https://reader033.vdocument.in/reader033/viewer/2022060308/5f0a04f87e708231d4299f68/html5/thumbnails/15.jpg)
1st Case: High Availability
15
![Page 16: Automated Upgrades of PostgreSQL Clusters in Cloudpostgres_old_dsn: "dbname={{pglupgrade_database}} host={{groups['old-primary'][0]}} user={{pglupgrade_user}}" ... Database clusters](https://reader033.vdocument.in/reader033/viewer/2022060308/5f0a04f87e708231d4299f68/html5/thumbnails/16.jpg)
1st Case: High Availability
16
![Page 17: Automated Upgrades of PostgreSQL Clusters in Cloudpostgres_old_dsn: "dbname={{pglupgrade_database}} host={{groups['old-primary'][0]}} user={{pglupgrade_user}}" ... Database clusters](https://reader033.vdocument.in/reader033/viewer/2022060308/5f0a04f87e708231d4299f68/html5/thumbnails/17.jpg)
2nd Case: Read Scalability
17
![Page 18: Automated Upgrades of PostgreSQL Clusters in Cloudpostgres_old_dsn: "dbname={{pglupgrade_database}} host={{groups['old-primary'][0]}} user={{pglupgrade_user}}" ... Database clusters](https://reader033.vdocument.in/reader033/viewer/2022060308/5f0a04f87e708231d4299f68/html5/thumbnails/18.jpg)
2nd Case: Read Scalability
18
![Page 19: Automated Upgrades of PostgreSQL Clusters in Cloudpostgres_old_dsn: "dbname={{pglupgrade_database}} host={{groups['old-primary'][0]}} user={{pglupgrade_user}}" ... Database clusters](https://reader033.vdocument.in/reader033/viewer/2022060308/5f0a04f87e708231d4299f68/html5/thumbnails/19.jpg)
Test EnvironmentAmazon EC2 t2.medium instances2 Virtual CPUs4 GB of RAM for memory110 GB EBS for storage pgbench scale factor 2000
PostgreSQL 9.5.6PostgreSQL 9.6.1 Ubuntu 16.04PgBouncer 1.7.2Pglogical 2.0.0
19
![Page 20: Automated Upgrades of PostgreSQL Clusters in Cloudpostgres_old_dsn: "dbname={{pglupgrade_database}} host={{groups['old-primary'][0]}} user={{pglupgrade_user}}" ... Database clusters](https://reader033.vdocument.in/reader033/viewer/2022060308/5f0a04f87e708231d4299f68/html5/thumbnails/20.jpg)
ResultsMetric (1st case) pg_dump/pg_restore pg_upgrade pglupgrade
Primary Downtime 00:24:27 00:16:25 00:00:03
Partial cluster HA 00:24:27 00:28:56 00:00:03
Full cluster capacity 01:02:27 00:28:56 00:38:00
Length of upgrade 01:02:27 00:28:56 01:38:10
Extra disk space 800 MB 27 GB 10 GB
Metric (2nd case) pg_dump/pg_restore pg_upgrade pglupgrade
Primary Downtime 00:23:52 00:17:03 00:00:05
Partial cluster HA 00:23:52 00:54:29 00:00:05
Full cluster capacity 00:23:52 03:19:16 00:00:05
Length of upgrade 00:23:52 03:19:16 01:02:10
Extra disk space 800 MB 27 GB 10 GB 20
![Page 21: Automated Upgrades of PostgreSQL Clusters in Cloudpostgres_old_dsn: "dbname={{pglupgrade_database}} host={{groups['old-primary'][0]}} user={{pglupgrade_user}}" ... Database clusters](https://reader033.vdocument.in/reader033/viewer/2022060308/5f0a04f87e708231d4299f68/html5/thumbnails/21.jpg)
Interpreting the Results
Database size growth during logical replication initialization21
![Page 22: Automated Upgrades of PostgreSQL Clusters in Cloudpostgres_old_dsn: "dbname={{pglupgrade_database}} host={{groups['old-primary'][0]}} user={{pglupgrade_user}}" ... Database clusters](https://reader033.vdocument.in/reader033/viewer/2022060308/5f0a04f87e708231d4299f68/html5/thumbnails/22.jpg)
Interpreting the Results
Transaction rate and latency during standby cloning process22
![Page 23: Automated Upgrades of PostgreSQL Clusters in Cloudpostgres_old_dsn: "dbname={{pglupgrade_database}} host={{groups['old-primary'][0]}} user={{pglupgrade_user}}" ... Database clusters](https://reader033.vdocument.in/reader033/viewer/2022060308/5f0a04f87e708231d4299f68/html5/thumbnails/23.jpg)
Interpreting the Results
Transaction rate and latency during the upgrade process23
![Page 24: Automated Upgrades of PostgreSQL Clusters in Cloudpostgres_old_dsn: "dbname={{pglupgrade_database}} host={{groups['old-primary'][0]}} user={{pglupgrade_user}}" ... Database clusters](https://reader033.vdocument.in/reader033/viewer/2022060308/5f0a04f87e708231d4299f68/html5/thumbnails/24.jpg)
Back to the FutureNeed for a near-zero downtime automated upgrade solutionfor PostgreSQL [ ]The tool will have a use in and our customersPostgreSQL 10 will have built-in logical replication
PgCon 2017 Developer Meeting2ndQuadrant
24
![Page 25: Automated Upgrades of PostgreSQL Clusters in Cloudpostgres_old_dsn: "dbname={{pglupgrade_database}} host={{groups['old-primary'][0]}} user={{pglupgrade_user}}" ... Database clusters](https://reader033.vdocument.in/reader033/viewer/2022060308/5f0a04f87e708231d4299f68/html5/thumbnails/25.jpg)
ApplicabilityTraditional data centers (bare-metal or virtual)Other Operating Systems (i.e Windows)Other DBMSs through logical replication (i.e MySQL, Oracle)Can work without PgBouncer
25
![Page 26: Automated Upgrades of PostgreSQL Clusters in Cloudpostgres_old_dsn: "dbname={{pglupgrade_database}} host={{groups['old-primary'][0]}} user={{pglupgrade_user}}" ... Database clusters](https://reader033.vdocument.in/reader033/viewer/2022060308/5f0a04f87e708231d4299f68/html5/thumbnails/26.jpg)
LimitationsSpare resources on primary server (initial data copy)Cluster with too many writes (logical rep. catchup)Tables with PKs (or insert-only tables) - Postgres 10No transparent DDL replication
26
![Page 27: Automated Upgrades of PostgreSQL Clusters in Cloudpostgres_old_dsn: "dbname={{pglupgrade_database}} host={{groups['old-primary'][0]}} user={{pglupgrade_user}}" ... Database clusters](https://reader033.vdocument.in/reader033/viewer/2022060308/5f0a04f87e708231d4299f68/html5/thumbnails/27.jpg)
ConclusionDatabase clusters can be upgraded with minimaldowntime without users being affected while theupgrade is happening.
An application can still respond to the request onlywith a small drop in performance.
Case studies prove that pglupgrade minimizes thedowntime to the level of 3-5 seconds.
27
![Page 28: Automated Upgrades of PostgreSQL Clusters in Cloudpostgres_old_dsn: "dbname={{pglupgrade_database}} host={{groups['old-primary'][0]}} user={{pglupgrade_user}}" ... Database clusters](https://reader033.vdocument.in/reader033/viewer/2022060308/5f0a04f87e708231d4299f68/html5/thumbnails/28.jpg)
Thanks!
28
![Page 29: Automated Upgrades of PostgreSQL Clusters in Cloudpostgres_old_dsn: "dbname={{pglupgrade_database}} host={{groups['old-primary'][0]}} user={{pglupgrade_user}}" ... Database clusters](https://reader033.vdocument.in/reader033/viewer/2022060308/5f0a04f87e708231d4299f68/html5/thumbnails/29.jpg)
Questions
29