amazon rds for postgresql: what's new and lessons learned - ny 2017

Grant McAlister – Senior Principal Engineer - RDS

March 2017

Amazon RDS for PostgreSQLWhat’s New and Lessons Learned

Amazon Aurora with PostgreSQL Compatibility

• PostgreSQL 9.6+• Cloud Optimized• Log based• 6 Copies across 3 Availability Zones• Up to 15 Read Replicas • Faster Failover• Enhanced Scaling• Autoscaling of storage to 64TB

Logging + Storage

Transactions

Caching

Amazon S3

PREVIEW

RDS Version Updates

New Major Version – 9.6

New Minor Releases (soon)• 9.6.2• 9.5.6 • 9.4.11 • 9.3.16

Extension Support Additions

9.6.1 bloom & pg_visibility9.6.2 log_fdw, pg_hint_plan & pg_freespacemap

rds-postgres-extensions-request@amazon.com

9.3 Original - 32

9.3 Current - 35

9.4 Current - 399.5 Current - 44

Future - ???9.6 Current - 49

log_fdw set log_destination to csvlog

postgres=> create extension log_fdw;

postgres=> CREATE SERVER log_fdw_server FOREIGN DATA WRAPPER log_fdw;

postgres=> select * from list_postgres_log_files(); file_name | file_size_bytes----------------------------------+----------------- postgresql.log.2017-03-28-17.csv | 2068 postgres.log | 617

postgres=> select create_foreign_table_for_log_file('pg_csv_log','log_fdw_server','postgresql.log.2017-03-28-17.csv');

postgres=> select log_time, message from pg_csv_log where message like 'connection%'; log_time | message----------------------------+-------------------------------------------------------------------------------- 2017-03-28 17:50:01.862+00 | connection received: host=ec2-54-174-205.compute-1.amazonaws.com port=45626 2017-03-28 17:50:01.868+00 | connection authorized: user=mike database=postgres

log_fdw - continued

can be done without csvpostgres=> select create_foreign_table_for_log_file('pg_log','log_fdw_server','postgresql.log.2017-03-28-17');

postgres=> select log_entry from pg_log where log_entry like '%connection%';

log_entry----------------------------------------------------------------------------------------------------------------------------------------------------2017-03-28 17:50:01 UTC:ec2-54-174.compute-1.amazonaws.com(45626):[unknown]@[unknown]:[20434]:LOG: received: host=ec2-54-174-205..amazonaws.com 2017-03-28 17:50:01 UTC:ec2-54-174.compute-1.amazonaws.com(45626):mike@postgres:[20434]:LOG: connection authorized: user=mike database=postgres2017-03-28 17:57:44 UTC:ec2-54-174.compute-1.amazonaws.com(45626):mike@postgres:[20434]:ERROR: column "connection" does not exist at character 143

pg_hint_plan

Add to shared_preload_libraries

• pg_hint_plan.debug_print• pg_hint_plan.enable_hint• pg_hint_plan.enable_hint_table• pg_hint_plan.message_level• pg_hint_plan.parse_messages

pg_hint_plan - examplepostgres=> EXPLAIN SELECT * FROM pgbench_branches bpostgres-> JOIN pgbench_accounts a ON b.bid = a.bid ORDER BY a.aid; QUERY PLAN------------------------------------------------------------------------------------------- Sort (cost=15943073.17..15993073.17 rows=20000000 width=465) Sort Key: a.aid -> Hash Join (cost=5.50..802874.50 rows=20000000 width=465) Hash Cond: (a.bid = b.bid) -> Seq Scan on pgbench_accounts a (cost=0.00..527869.00 rows=20000000 width=97) -> Hash (cost=3.00..3.00 rows=200 width=364) -> Seq Scan on pgbench_branches b (cost=0.00..3.00 rows=200 width=364)

postgres=> /*+ NestLoop(a b) */postgres-> EXPLAIN SELECT * FROM pgbench_branches bpostgres-> JOIN pgbench_accounts a ON b.bid = a.bid ORDER BY a.aid; QUERY PLAN------------------------------------------------------------------------------------------------------------------- Nested Loop (cost=0.58..44297240.44 rows=20000000 width=465) -> Index Scan using pgbench_accounts_pkey on pgbench_accounts a (cost=0.44..847232.44 rows=20000000 width=97) -> Index Scan using pgbench_branches_pkey on pgbench_branches b (cost=0.14..2.16 rows=1 width=364) Index Cond: (bid = a.bid)

Major version upgrade

Prod9.5

Prod9.6

pg_upgrade

Backup Backup

No PITR

Test9.5

Test9.6

pg_upgrade

Restore to a test instance

Application Testing

Security

Forcing SSL on all connections

DB Instance

SnapshotApplication

HostSSL

Log Backups

Security Group

DB Instance

SnapshotApplication

HostSSL

Log Backups

Security Group

DB Instance

SnapshotApplication

HostSSL

Log Backups

Security Group

Encryption at Rest

DB Instance

SnapshotApplication

HostSSL

Log Backups

Security Group

Encryption at Rest

ssl_mode=disable

DB Instance

SnapshotApplication

HostSSL

Log Backups

Security Group

Encryption at Rest

ssl_mode=disable

rds.force_ssl=1 (default 0)

Unencrypted Snapshot Sharing

DB Instance

Snapshot

Prod Account

Test Account

Snapshot

Share with account

DB Instance

Snapshot

Prod Account

Test Account

Snapshot

Share with account

DB Instance

Snapshot

Prod Account

Test Account

SnapshotDB Instance

Snapshot

Share with account

Share to Public

Encrypted Snapshot Sharing

DB Instance

Snapshot

Prod Account

Test Account

Snapshot

Share with account

Encryption at Rest

Default

DB Instance

Snapshot

Prod Account

Test Account

Snapshot

Share with account

Encryption at Rest

CustomKey

Add external account

DB Instance

Snapshot

Prod Account

Test Account

SnapshotDB Instance

Snapshot

Share with account

Encryption at Rest

CustomKey

Add external account

Cross Region Replicas – Encrypted

SecondarySync

Application

Primary

AZ1 AZ2

Application

US-EAST-1

Cross Region Replicas – Encrypted

SecondarySync

Application

Primary

AZ1 AZ2

Application

Read Replica

Application

AZ1Async Replication

US-EAST-1 EU-WEST-1

HIPAA-eligible service & FedRAMP

• RDS PostgreSQL is now a HIPAA-eligible service• https://aws.amazon.com/compliance/hipaa-compliance/

• FedRAMP in AWS GovCloud (US) region • https://aws.amazon.com/compliance/fedramp/

Data movement

Move data to the same or different database engine Keep your apps running during the migrationStart your first migration in 10 minutes or lessReplicate within, to, or from AWS EC2 or RDS

AWSDatabase Migration

Service(DMS)

CustomerPremises

Application Users

Internet

Start a replication instance

Keep your apps running during the migration

AWS Database Migration Service

CustomerPremises

Application Users

Internet

Connect to source and target databases

Select tables, schemas, or databases

CustomerPremises

Application Users

Internet

Let the AWS Database Migration Service create tables and load data

CustomerPremises

Application Users

Internet

Let the AWS Database Migration Service create tables and load data Uses change data capture to keep them in sync

CustomerPremises

Application Users

Internet

Let the AWS Database Migration Service create tables and load data Uses change data capture to keep them in syncSwitch applications over to the target at your convenience

AWS Database Migration Service - PostgreSQL

• Source - on premise or EC2 PostgreSQL (9.4+) RDS (9.4.9+ or 9.5.4+ or 9.6.1+)

• Destination can be EC2 or RDS• Initial bulk copy via consistent select• Uses PostgreSQL logical replication support to provide

change data capture

https://aws.amazon.com/dms/

Schema Conversion Tool - SCT

Downloadable tool (Windows, Mac, Linux Desktop)

Source Database Target Database on Amazon RDSMicrosoft SQL Server Amazon Aurora, MySQL, PostgreSQLMySQL PostgreSQLOracle Amazon Aurora, MySQL, PostgreSQLPostgreSQL Amazon Aurora, MySQL

SCT - Analysis

SCT - Detailed

Logical Replication Support• Supported with 9.6.1+, 9.5.4+ and 9.4.9+• Set rds.logical_replication parameter to 1• As user who has rds_replication & rds_superuser role

SELECT * FROM pg_create_logical_replication_slot('test_slot', 'test_decoding');

pg_recvlogical -d postgres --slot test_slot -U master --host $rds_hostname -f - --start

• Added support for Event Triggers

Logical Decoding Space Usage

CloudWatch – Replication Lag

CloudWatch – Slot usage for WAL

Logical Replication Support - Example

RDS Postgres

Logical Replica

Redshift

RDS Postgres

Logical Replica

Redshift

On PremisePostgres

RDS Postgres

Logical Replica

RedshiftEC2 Postgres

On PremisePostgres

RDS Postgres

Logical Replica

On PremisePostgres

S3(new)

RDS Postgres

Logical Replica

On PremisePostgres

EC2 Oracle

S3(new)

RDS Postgres

Logical Replica

On PremisePostgres

EC2 Oracle

CustomLogicalHandler

S3(new)NoSQL DB

Lessons

Vacuum parameters

Will auto vacuum when• autovacuum_vacuum_threshold +

autovacuum_vacuum_scale_factor * pgclass.reltuples

How hard auto vacuum works • autovacuum_max_workers• autovacuum_nap_time• autovacuum_cost_limit• autovacuum_cost_delay

Transaction IDWrap Around

RDS autovacuum logging (9.4.5+)

log_autovacuum_min_duration = 5000 (i.e. 5 secs)rds.force_autovacuum_logging_level = LOG

…[14638]:ERROR: canceling autovacuum task…[14638]:CONTEXT: automatic vacuum of table "postgres.public.pgbench_tellers"…[14638]:LOG: skipping vacuum of "pgbench_branches" --- lock not available

RDS autovacuum visibility(9.3.12, 9.4.7, 9.5.2)pg_stat_activity

BEFORE usename | query----------+------------------------------------------------------------- rdsadmin | <insufficient privilege> rdsadmin | <insufficient privilege> gtest | SELECT c FROM sbtest27 WHERE id BETWEEN 392582 AND 392582+4 gtest | select usename, query from pg_stat_activity

CloudWatch Metric

Scale and availability

M4 Instance Class – pgbench read only

1 2 4 8 160

db.m3.large db.m4.large

Threads

46% Better Price/Performance

37% TPS Increase

$0.195 $0.182

Enhanced Operating System (OS) metrics

1-60 second granularity

cpuUtilization• guest• irq• system• wait• idl: • user • total • steal • nice

diskIO • writeKbPS• readIOsPS• await • readKbPS• rrqmPS • util • avgQueueLen • tps • readKb • writeKb • avgReqSz • wrqmPS • writeIOsPS

memory • writeback• cached • free • inactive• dirty • mapped • active • total • slab • buffers• pageTable• Hugepages

swap • cached • total • free

tasks• sleeping • zombie • running • stopped • total • blocked

fileSys • used• usedFiles• usedFilePercent• maxFiles • total • usedPercent

loadAverageMinute • fifteen • five • one

uptime

processList• name• cpuTime• parentID• memoryUsedPct• cpuUsedPct• id • rss• vss

Process List

OS metrics

Performance Insights – In Preview

Aurora PostgreSQL

Read Replicas = Availability

Secondary

Application

Read Replica

SyncReplication

Multi-AZ

Primary

Eventually Consistent Reads

Writes & Consistent

Async Replication

Application

Read Replica

Read ReplicaSecondary Primary

Writes & Consistent

Async Replication

Application

Read Replica

Writes & Consistent

Async Replication

Application

Read Replica

Writes & Consistent

Async Replication

Upgrade

Application

Read Replica

Writes & Consistent

Async Replication

Modify

Application

AZ-1 AZ-2

Aurora Storage

Aurora

Aurora Storage

4/6 sync writes

Aurora Timing Example

Location 1

Location 2

Location 3

Start Finish

Only need 4/6 sync writes

50 90 99.9 99.990

6.110.4

123High Concurrency Sync Write Test

2 Node (4 copy) 3 Node (6 Copy)

Percentile

Concurrency

Queued Work

Log Buffer

PostgreSQL Aurora

Storage

Queued Work

Storage

Concurrency

Queued Work

Log Buffer

PostgreSQL Aurora

Storage

Queued Work

Storage

Concurrency

Queued Work

Log Buffer

PostgreSQL Aurora

Storage

Queued Work

Storage

Concurrency

Queued Work

Log Buffer

PostgreSQL Aurora

Storage

Queued Work

Storage

Concurrency

Queued Work

Log Buffer

PostgreSQL Aurora

Storage

Queued Work

Storage

Concurrency

Queued Work

Log Buffer

PostgreSQL Aurora

Storage

Queued Work

Storage

Concurrency

Queued Work

Log Buffer

PostgreSQL Aurora

Storage

A Queued Work

Storage

Concurrency

Queued Work

Log Buffer

PostgreSQL Aurora

StorageA

Queued Work

StorageB C D E

Concurrency

Queued Work

Log Buffer

PostgreSQL Aurora

StorageA

Queued Work

StorageB C D E

2 2 1 0 1A B C D E

DurabilityTracking

Concurrency

Queued Work

Log Buffer

PostgreSQL Aurora

StorageA

Queued Work

StorageB C D E

4 3 4 2 4A B C D E

DurabilityTracking

Concurrency

Queued Work

Log Buffer

PostgreSQL Aurora

StorageA

Queued Work

StorageB C D E

4 3 4 2 4A B C D E

DurabilityTracking

Concurrency

Queued Work

Log Buffer

PostgreSQL Aurora

StorageA

Queued Work

StorageB C D E

6 5 6 3 5A B C D E

DurabilityTracking

AZ-1 AZ-2 AZ-3

Aurora Storage and Replicas

Application

AZ-1 AZ-2 AZ-3

Application

AZ-1 AZ-2 AZ-3

Application

AsyncInvalidation& Update

Aurora – Writing Less

Block in Memory

PostgreSQL Aurora

update t set y = 6;

Block in Memory

AuroraStorage

Block in Memory

PostgreSQL Aurora

update t set y = 6;

Full Block

Block in Memory

AuroraStorage

Block in Memory

PostgreSQL Aurora

update t set y = 6;

Full Block

Block in Memory

AuroraStorage

Block in Memory

PostgreSQL Aurora

update t set y = 6;

Full Block

Block in Memory

AuroraStorage

Block in Memory

PostgreSQL Aurora

update t set y = 6;

Checkpoint

Datafile

Full Block

Archive

Block in Memory

AuroraStorage

Block in Memory

PostgreSQL Aurora

update t set y = 6; update t set y = 6;

Checkpoint

Datafile

Full Block

Archive

Block in Memory

AuroraStorage

Block in Memory

PostgreSQL Aurora

Checkpoint

Datafile

Full Block

Archive

Block in Memory

AuroraStorage

Block in Memory

PostgreSQL Aurora

Checkpoint

Datafile

Full Block

Archive

Block in Memory

AuroraStorage

Block in Memory

PostgreSQL Aurora

Checkpoint

Datafile

Full Block

Archive

Block in Memory

AuroraStorage

Amazon Aurora Loads Data 3x Faster Database initialization is three times faster than PostgreSQL using the standard PgBench benchmark

Command: pgbench -i -s 2000 –F 90

Amazon Aurora Delivers up to 85x Faster Recovery

SysBench oltp(write-only) 10GiB workload with 250 tables & 150,000 rows

PostgreSQL12.5GB

Checkpoint

PostgreSQL8.3GB Checkpoint

PostgreSQL2.1GB Checkpoint

Amazon AuroraNo Checkpoints

0 20,000 40,000 60,000 80,000

0 20 40 60 80 100 120 140

Writes per Second 69,620

Recovery Time (seconds) 102.0

Crash Recovery Time - SysBench 10GB Write Workload

Writes Per Second

Recovery Time in Seconds

Transaction-aware storage system recovers almost instantly

Amazon Aurora is >=2x Faster on PgBench

pgbench “tpcb-like” workload, scale 2000 (30GiB). All configurations run for 60 minutes

Amazon Aurora is 2x-3x Faster on SysBenchAmazon Aurora delivers 2x the absolute peak of PostgreSQL and 3x PostgreSQL performance at high client counts

SysBench oltp(write-only) workload with 30 GB database with 250 tables and 400,000 initial rows per table

Amazon Aurora Gives >2x Faster Response TimesResponse time under heavy write load >2x faster than PostgreSQL(and >10x more consistent)

SysBench oltp(write-only) 23GiB workload with 250 tables and 300,000 initial rows per table. 10-minute warmup.

Amazon Aurora Has More Consistent ThroughputWhile running at load, performance is more than three timesmore consistent than PostgreSQL

PgBench “tpcb-like” workload at scale 2000. Amazon Aurora was run with 1280 clients. PostgreSQL was run with 512 clients (the concurrency at which it delivered the best overall throughput)

Amazon Aurora is 3x Faster at Large Scale Scales from 1.5x to 3x faster as database grows from 10 GiB to 100 GiB

SysBench oltp(write-only) – 10GiB with 250 tables & 150,000 rows and 100GiB with 250 tables & 1,500,000 rows

10GB 100GB0

20,000

40,000

60,000

80,000

100,000

120,000

75,666

27,491

112,390

82,714

SysBench write-only

PostgreSQL Amazon Aurora

SysBench Test Size

Thank you!

Questions?

amazon rds for postgresql: what's new and lessons learned - ny 2017

Data & Analytics

postgresql on amazon rds

a journey down the amazon gabrielle roth scale 14x · rds...

aws webcast - introducing amazon rds for postgresql

amazon rds for postgresql grant mcalister – senior...

rds-14p rds-14p · rds-14p rds-14p.78 * • aa-rds •...

postgresql notes for professionals - goalkicker.com ·...

(sdd409) amazon rds for postgresql deep dive | aws re:invent...

migration to postgresql - preparation and … · overview...

modular architecture for amazon aurora postgresql · you...

machines to cloud, from virtual serverless options mysql...

postgresql, postgresql monitoring and monitoring...

what's new in postgresql 9.6

postgresql: introduction and concepts -...

performance optimization and monitoring with pmm open...

postgresql extensions -a deeper dive · postgresql...

amazon rds for postgresql: what's new and lessons learned -...

postgresql 9.1 - what's new

postgresql what's next

peter burkholder - home - tech at gsa · relational...

the open analytics platform · •data storage –ms azure...