PostgreSQL ReplicationPostgreSQL Replicationin 2018in 2018PGConf.ASIA 2018
Tokyo, Japan
Magnus Hagander [email protected]
Magnus HaganderMagnus HaganderRedpill Linpro
Infrastructure servicesPrincipal database consultant
PostgreSQLCore Team memberCommitterPostgreSQL Europe
ReplicationReplication
"PostgreSQL doesn't have"PostgreSQL doesn't havereplication"replication"
"PostgreSQL doesn't have"PostgreSQL doesn't havereplication"replication"
"So we have to use MySQL"
ReplicationReplicationWasn't true back thenEven less true now!Now there are too many choices?
But you have to pick oneAnd can be hard to use
ReplicationReplicationCan be done at different layersFrom hardware
(ish)To application
Replication layersReplication layers↓ Application↓ App-in-database↓ Database logical↓ Database physical↓ Operating system↓ Hardware
Start from the bottomStart from the bottom
SAN replicationSAN replicationHardware takes care of replicationBlock levelTransparent to OS
And to PostgreSQLCommon enterprise solution
Especially with VMs
SAN replicationSAN replicationFrom single rackTo multi-siteSynchronousGuaranteed to never fail
Riiiiight...
Replication layersReplication layers↓ Application↓ App-in-database↓ Database logical↓ Database physical↓ Operating system↓ Hardware
DRBDDRBDSimilar in style to SANImplementation in OS driverPerformance?
Replication layersReplication layers↓ Application↓ App-in-database↓ Database logical↓ Database physical↓ Operating system↓ Hardware
Database physicalDatabase physicalWAL based replicationFile based from 8.3Streaming since 9.0Synchronous since 9.1
Transaction level mixingQuorum commit since 10(etc)
wal_level = 'replica'wal_level = 'replica'
Synchronous modeSynchronous modeofflocalremote_writeonremote_apply
Synchronous modeSynchronous modeSingle (9.1)
First (9.6)
Quorum (10)
synchronous_standby_names = s1,s2,s3
synchronous_standby_names = 2(s1,s2,s3)
synchronous_standby_names = ANY 2(s1,s2,s3)
Streaming replicationStreaming replicationPrimary choice todayEasy to set upHard to get wrongEfficientBuilt-in
Streaming replicationStreaming replication$ pg_basebackup -D /var/lib/pgsql \ -h primary -U replica \ -R -S replica1 -P $ sudo service postgresql-9.6 start
for pre-10, add -X stream
Streaming replicationStreaming replicationArchitecture/compile flag dependentWhole cluster onlyStandby completely read-onlyPrimary → standby onlyExcellent for availability
Streaming replicationStreaming replicationNo built-in cluster management
Manual or automaticProvides infrastructure
No fail-back(no easy one)
Easy to get started, harder to maintain
Cluster managementCluster managementPatroniPatroni
Designed for automatic managementIncluding automatic failoverUses etcd, zookeeper, or consulIntegrates with haproxy
Cluster managementCluster managementrepmgrrepmgr
Fewer pre-requisitesEasier for manual management
Comes with repmgrd that does automaticDoes not handle connection management
Use e.g. rebouncerOr haproxy
Cluster managementCluster managementPAFPAF
Integrates with pacemaker/corosyncManagement of other servicesManages virtual IP
Replication layersReplication layers↓ Application↓ App-in-database↓ Database logical↓ Database physical↓ Operating system↓ Hardware
Database logicalDatabase logicalLogical decoding since 9.4Logical replication since 10
Built-in, that isPiggy-backs on WALVery low overhead
wal_level = 'logical'wal_level = 'logical'
Logical replicationLogical replicationReconstructs changes by rowReplicates row content
not SQL statementsFully transactional
Logical replicationLogical replicationTable-level partial replicationTable-level bi-directional replication
Logical replicationLogical replicationCREATE TABLE testtable (a int PRIMARY KEY, b text); CREATE PUBLICATION testpub FOR TABLE testtable;
Logical replicationLogical replicationCREATE TABLE testtable (a int PRIMARY KEY, b text); CREATE SUBSCRIPTION testsub CONNECTION 'host=/tmp port=5500 dbname=postgres user=mha' PUBLICATION testpub;
Logical replicationLogical replicationData replication only
No schemaNo sequences
Suitable for data distributionBut not for HALacks failover slots!
pglogicalpglogicalExternal version of logical replicationMerged piece by pieceMore capabilities!Not as deeply integrated
pglogicalpglogicalSequence replicationRow based filteringColumn based filteringMerging and conflict resolution...
pglogicalpglogicalSupports PostgreSQL 9.4Zero (or close to zero) downtime upgrades!
Replication layersReplication layers↓ Application↓ App-in-database↓ Database logical↓ Database physical↓ Operating system↓ Hardware
App-in-databaseApp-in-databaseTrigger based systems
SlonyBucardoLondiste...
Trigger basedTrigger basedFor a long time the only choiceNow mostly supersededMuch higher overhead than logicalComplex scenarios
Multimaster?Multimaster?I I needneed it! it!
Do you really need it?Do you really know what it means?No transparent options
Because...
Multimaster?Multimaster?OptionsOptions
BDRFork!
BucardoOr just don't?
Replication layersReplication layers↓ Application↓ App-in-database↓ Database logical↓ Database physical↓ Operating system↓ Hardware
ApplicationApplicationReplication done entirely in applicationVery difficult for transactionalUseful in limited cases
SummarySummary
High AvailabilityHigh AvailabilityUse streaming replicationMix of sync and asyncConsider patroni or repmgr
Read query offloadingRead query offloadingUse streaming replication(see previous slide)
Data distributionData distributionLogical replication in 10pglogical in 9.4+
Or in 10 if built-in is not enoughUpgrade away from your Slony...
Need both?Need both?Use both!
Thank you!Thank you!Magnus Hagander
[email protected] @magnushagander
https://www.hagander.net/talks/
This material is licensed