make your postgresql 10x faster on cloud in minutes€¦ · postgresql file system linux page cache...
TRANSCRIPT
![Page 1: Make Your PostgreSQL 10x Faster on Cloud in Minutes€¦ · PostgreSQL File system Linux page cache fsync() Linux issues I/Os in 4 KiB unit ... Configure. Test setup on AWS Performance](https://reader036.vdocument.in/reader036/viewer/2022063013/5fcc46812899160a333daf9c/html5/thumbnails/1.jpg)
Make Your PostgreSQL 10x Faster
on Cloud in Minutes
Sangwook Kim
![Page 2: Make Your PostgreSQL 10x Faster on Cloud in Minutes€¦ · PostgreSQL File system Linux page cache fsync() Linux issues I/Os in 4 KiB unit ... Configure. Test setup on AWS Performance](https://reader036.vdocument.in/reader036/viewer/2022063013/5fcc46812899160a333daf9c/html5/thumbnails/2.jpg)
All storages are not created equal
Physical vs. Cloud Storage
2
≠Purchase H/W
HW-controlled
Pay-as-you-go
SW-controlled
![Page 3: Make Your PostgreSQL 10x Faster on Cloud in Minutes€¦ · PostgreSQL File system Linux page cache fsync() Linux issues I/Os in 4 KiB unit ... Configure. Test setup on AWS Performance](https://reader036.vdocument.in/reader036/viewer/2022063013/5fcc46812899160a333daf9c/html5/thumbnails/3.jpg)
I/O below 256 KiB consumes one IOPS
Example: AWS SSD Storage
3
PostgreSQL/Oracle
MySQL/MariaDB
Linux
![Page 4: Make Your PostgreSQL 10x Faster on Cloud in Minutes€¦ · PostgreSQL File system Linux page cache fsync() Linux issues I/Os in 4 KiB unit ... Configure. Test setup on AWS Performance](https://reader036.vdocument.in/reader036/viewer/2022063013/5fcc46812899160a333daf9c/html5/thumbnails/4.jpg)
Sequential and random I/O show identical performance
Example: AWS HDD Storage (1)
4
![Page 5: Make Your PostgreSQL 10x Faster on Cloud in Minutes€¦ · PostgreSQL File system Linux page cache fsync() Linux issues I/Os in 4 KiB unit ... Configure. Test setup on AWS Performance](https://reader036.vdocument.in/reader036/viewer/2022063013/5fcc46812899160a333daf9c/html5/thumbnails/5.jpg)
Sequential and random I/O show identical performance
Example: AWS HDD Storage (2)
5
![Page 6: Make Your PostgreSQL 10x Faster on Cloud in Minutes€¦ · PostgreSQL File system Linux page cache fsync() Linux issues I/Os in 4 KiB unit ... Configure. Test setup on AWS Performance](https://reader036.vdocument.in/reader036/viewer/2022063013/5fcc46812899160a333daf9c/html5/thumbnails/6.jpg)
Test setup on AWS cloud
Illustrative Experiment (1)
6
r5.xlarge4 vCPUs16GiB memorypgbench
AWS Seoul Region
Availability Zone A
r5d.4xlarge16 vCPUs128GiB memory1,000 IOPS EBS orNVMe SSDPostgreSQL v11.5
![Page 7: Make Your PostgreSQL 10x Faster on Cloud in Minutes€¦ · PostgreSQL File system Linux page cache fsync() Linux issues I/Os in 4 KiB unit ... Configure. Test setup on AWS Performance](https://reader036.vdocument.in/reader036/viewer/2022063013/5fcc46812899160a333daf9c/html5/thumbnails/7.jpg)
Test setup on AWS cloud
Illustrative Experiment (1)
7
![Page 8: Make Your PostgreSQL 10x Faster on Cloud in Minutes€¦ · PostgreSQL File system Linux page cache fsync() Linux issues I/Os in 4 KiB unit ... Configure. Test setup on AWS Performance](https://reader036.vdocument.in/reader036/viewer/2022063013/5fcc46812899160a333daf9c/html5/thumbnails/8.jpg)
PostgreSQL can fully utilize SSD
Illustrative Experiment (2)
8
loading run
13,000 TPS
![Page 9: Make Your PostgreSQL 10x Faster on Cloud in Minutes€¦ · PostgreSQL File system Linux page cache fsync() Linux issues I/Os in 4 KiB unit ... Configure. Test setup on AWS Performance](https://reader036.vdocument.in/reader036/viewer/2022063013/5fcc46812899160a333daf9c/html5/thumbnails/9.jpg)
PostgreSQL cannot fully utilize cloud storage
Illustrative Experiment (3)
9
loading run
1,000 TPS
13,000 TPS
![Page 10: Make Your PostgreSQL 10x Faster on Cloud in Minutes€¦ · PostgreSQL File system Linux page cache fsync() Linux issues I/Os in 4 KiB unit ... Configure. Test setup on AWS Performance](https://reader036.vdocument.in/reader036/viewer/2022063013/5fcc46812899160a333daf9c/html5/thumbnails/10.jpg)
PostgreSQL cannot fully utilize cloud storage
Illustrative Experiment (4)
10
loading run
Thput =
IO size * 1,000
![Page 11: Make Your PostgreSQL 10x Faster on Cloud in Minutes€¦ · PostgreSQL File system Linux page cache fsync() Linux issues I/Os in 4 KiB unit ... Configure. Test setup on AWS Performance](https://reader036.vdocument.in/reader036/viewer/2022063013/5fcc46812899160a333daf9c/html5/thumbnails/11.jpg)
Linux issues I/Os in 4 KiB unit
What’s the Problem? (1)
11
File
Block
PostgreSQL
File system
Linux page cache
8KiB unit
4KiB unit
4KiB unit
![Page 12: Make Your PostgreSQL 10x Faster on Cloud in Minutes€¦ · PostgreSQL File system Linux page cache fsync() Linux issues I/Os in 4 KiB unit ... Configure. Test setup on AWS Performance](https://reader036.vdocument.in/reader036/viewer/2022063013/5fcc46812899160a333daf9c/html5/thumbnails/12.jpg)
Linux issues I/Os in 4 KiB unit
What’s the Problem? (2)
12
File
Block
PostgreSQL
File system
Linux page cache
write()
DB file
Storage mapping
4KiB
dirty block…
![Page 13: Make Your PostgreSQL 10x Faster on Cloud in Minutes€¦ · PostgreSQL File system Linux page cache fsync() Linux issues I/Os in 4 KiB unit ... Configure. Test setup on AWS Performance](https://reader036.vdocument.in/reader036/viewer/2022063013/5fcc46812899160a333daf9c/html5/thumbnails/13.jpg)
File
Block
PostgreSQL
File system
Linux page cache
fsync()
Linux issues I/Os in 4 KiB unit
DB file
Storage mapping
4KiB
dirty block…
Issue 8 IOs
…
Linux issues I/Os in 4 KiB unit
What’s the Problem? (3)
13
![Page 14: Make Your PostgreSQL 10x Faster on Cloud in Minutes€¦ · PostgreSQL File system Linux page cache fsync() Linux issues I/Os in 4 KiB unit ... Configure. Test setup on AWS Performance](https://reader036.vdocument.in/reader036/viewer/2022063013/5fcc46812899160a333daf9c/html5/thumbnails/14.jpg)
Tuning helps to get 30% better TPS
Tuning PostgreSQL (1)
14
tuned:
max_wal_size = 128GB
checkpoint_timeout = 30min
vm.background_dirty_ratio = 1
![Page 15: Make Your PostgreSQL 10x Faster on Cloud in Minutes€¦ · PostgreSQL File system Linux page cache fsync() Linux issues I/Os in 4 KiB unit ... Configure. Test setup on AWS Performance](https://reader036.vdocument.in/reader036/viewer/2022063013/5fcc46812899160a333daf9c/html5/thumbnails/15.jpg)
Be aware of cache space contention inside OS
Tuning PostgreSQL (2)
15
![Page 16: Make Your PostgreSQL 10x Faster on Cloud in Minutes€¦ · PostgreSQL File system Linux page cache fsync() Linux issues I/Os in 4 KiB unit ... Configure. Test setup on AWS Performance](https://reader036.vdocument.in/reader036/viewer/2022063013/5fcc46812899160a333daf9c/html5/thumbnails/16.jpg)
Using huge pages additionally improves 20%
Tuning PostgreSQL (3)
16
![Page 17: Make Your PostgreSQL 10x Faster on Cloud in Minutes€¦ · PostgreSQL File system Linux page cache fsync() Linux issues I/Os in 4 KiB unit ... Configure. Test setup on AWS Performance](https://reader036.vdocument.in/reader036/viewer/2022063013/5fcc46812899160a333daf9c/html5/thumbnails/17.jpg)
What if we can issue 1 IO instead of 8?
Possible Solution for AWS EBS SSD
17
File
Block
PostgreSQL
File system
Linux page cache
fsync()
DB file
Storage mapping
4KiB
dirty block…
256 KiB IO
…
![Page 18: Make Your PostgreSQL 10x Faster on Cloud in Minutes€¦ · PostgreSQL File system Linux page cache fsync() Linux issues I/Os in 4 KiB unit ... Configure. Test setup on AWS Performance](https://reader036.vdocument.in/reader036/viewer/2022063013/5fcc46812899160a333daf9c/html5/thumbnails/18.jpg)
Each cloud has different policy for its storage
Cloud Storage Diversity
18
…
IOPS Thput Note
ebs-gp2 3/GiB 250 Burstable
ebs-io1 $67/1K/M 500
ebs-st1 500 40/TiB Burstable
ebs-sc1 250 12/TiB Burstable
IOPS Thput Note
pd-ssd 30/GiB 0.48/GiB
pd-ssd-
16kb
30/GiB 0.48/GiB 16 KiB
pd-std 1.5/GiB 0.12/GiB
pd-std-
16kb
1.5/GiB 0.12/GiB 16 KiB
IOPS Thput Note
ultra $49/1K/M $1/MB/M
pre-ssd 20,000 900 Burstable
std-ssd 6,000 750 Pay # IOs
std-hdd 2,000 500 Pay # IOs
![Page 19: Make Your PostgreSQL 10x Faster on Cloud in Minutes€¦ · PostgreSQL File system Linux page cache fsync() Linux issues I/Os in 4 KiB unit ... Configure. Test setup on AWS Performance](https://reader036.vdocument.in/reader036/viewer/2022063013/5fcc46812899160a333daf9c/html5/thumbnails/19.jpg)
AppOS is a file engine built for diverse types of storage
AppOS Extension
19
AppOS extension
Bare metal Virtual machine Container Container
![Page 20: Make Your PostgreSQL 10x Faster on Cloud in Minutes€¦ · PostgreSQL File system Linux page cache fsync() Linux issues I/Os in 4 KiB unit ... Configure. Test setup on AWS Performance](https://reader036.vdocument.in/reader036/viewer/2022063013/5fcc46812899160a333daf9c/html5/thumbnails/20.jpg)
AppOS Design Goals
20
No PostgreSQL code changes
PostgreSQL-aware I/O processing
Storage-aware I/O processing
![Page 21: Make Your PostgreSQL 10x Faster on Cloud in Minutes€¦ · PostgreSQL File system Linux page cache fsync() Linux issues I/Os in 4 KiB unit ... Configure. Test setup on AWS Performance](https://reader036.vdocument.in/reader036/viewer/2022063013/5fcc46812899160a333daf9c/html5/thumbnails/21.jpg)
AppOS extension is a collection of libraries
AppOS Architecture
21
pg_appos Syscall handler
PostgreSQL adapter
Storage
optimizer
Virtual file system
Page cache
File mapping
AppOS
I/O engine
Spec
Configure
![Page 22: Make Your PostgreSQL 10x Faster on Cloud in Minutes€¦ · PostgreSQL File system Linux page cache fsync() Linux issues I/Os in 4 KiB unit ... Configure. Test setup on AWS Performance](https://reader036.vdocument.in/reader036/viewer/2022063013/5fcc46812899160a333daf9c/html5/thumbnails/22.jpg)
Test setup on AWS
Performance on AWS Storage (1)
22
r5.xlarge4 vCPUs16 GiB memorypgbench
AWS Seoul Region
Availability Zone A
r5.4xlarge16 vCPUs128 GiB memory1,000 IOPS EBSPostgreSQL v11.5
![Page 23: Make Your PostgreSQL 10x Faster on Cloud in Minutes€¦ · PostgreSQL File system Linux page cache fsync() Linux issues I/Os in 4 KiB unit ... Configure. Test setup on AWS Performance](https://reader036.vdocument.in/reader036/viewer/2022063013/5fcc46812899160a333daf9c/html5/thumbnails/23.jpg)
PostgreSQL processes 23x more transactions with AppOS
Performance on AWS Storage (2)
23
avg: 4.6 ms
stddev: 1.7 ms
avg: 82 ms
stddev: 87 ms
![Page 24: Make Your PostgreSQL 10x Faster on Cloud in Minutes€¦ · PostgreSQL File system Linux page cache fsync() Linux issues I/Os in 4 KiB unit ... Configure. Test setup on AWS Performance](https://reader036.vdocument.in/reader036/viewer/2022063013/5fcc46812899160a333daf9c/html5/thumbnails/24.jpg)
AppOS makes PostgreSQL to fully utilize storage bandwidth
Performance on AWS Storage (3)
24
![Page 25: Make Your PostgreSQL 10x Faster on Cloud in Minutes€¦ · PostgreSQL File system Linux page cache fsync() Linux issues I/Os in 4 KiB unit ... Configure. Test setup on AWS Performance](https://reader036.vdocument.in/reader036/viewer/2022063013/5fcc46812899160a333daf9c/html5/thumbnails/25.jpg)
AppOS Internals (1)
25
AppOS does not require code changes
Linux kernel
syscall
prehook
syscall
posthook
AppOS VFSinode, dentry, dc
ache, icache, fdm
ap, context, …
POSIX
Interface
Ring 3
Ring 0
Ring 0
PostgreSQL
Physical Virtual
Ring 3
![Page 26: Make Your PostgreSQL 10x Faster on Cloud in Minutes€¦ · PostgreSQL File system Linux page cache fsync() Linux issues I/Os in 4 KiB unit ... Configure. Test setup on AWS Performance](https://reader036.vdocument.in/reader036/viewer/2022063013/5fcc46812899160a333daf9c/html5/thumbnails/26.jpg)
AppOS provides PostgreSQL-aware I/O processing
AppOS Internals (2)
26
AppOS
Logical sector size =
atomic unit
PG adapter
Tag I/O priority
Classify file types
Set page size
Prealloc WAL file
File syscalls
I/O engine (priority)
Page cache (8 KiB page)
File mapping
(8 KiB logical sector)
![Page 27: Make Your PostgreSQL 10x Faster on Cloud in Minutes€¦ · PostgreSQL File system Linux page cache fsync() Linux issues I/Os in 4 KiB unit ... Configure. Test setup on AWS Performance](https://reader036.vdocument.in/reader036/viewer/2022063013/5fcc46812899160a333daf9c/html5/thumbnails/27.jpg)
AppOS provides storage-aware I/O processing
AppOS Internals (3)
27
Storage
optimizer
Page cache
File mapping
AppOS
I/O engine
Spec
Per core
IO workers
Logical block
Local SSD
as memory extension(work in progress)
IO based on
logical block size
File Sharding
![Page 28: Make Your PostgreSQL 10x Faster on Cloud in Minutes€¦ · PostgreSQL File system Linux page cache fsync() Linux issues I/Os in 4 KiB unit ... Configure. Test setup on AWS Performance](https://reader036.vdocument.in/reader036/viewer/2022063013/5fcc46812899160a333daf9c/html5/thumbnails/28.jpg)
Test setup on GCP
Performance on GCP Storage (1)
28
n1-highcpu-88 vCPUs7.2 GiB memorypgbench
Availability Zone B
n1-standard-3232 vCPUs120 GiB memory500 GiB PD-SSDPostgreSQL v11.5
GCP Tokyo Region
![Page 29: Make Your PostgreSQL 10x Faster on Cloud in Minutes€¦ · PostgreSQL File system Linux page cache fsync() Linux issues I/Os in 4 KiB unit ... Configure. Test setup on AWS Performance](https://reader036.vdocument.in/reader036/viewer/2022063013/5fcc46812899160a333daf9c/html5/thumbnails/29.jpg)
PostgreSQL processes 3.7 more transactions with AppOS
Performance on GCP Storage (2)
29
![Page 30: Make Your PostgreSQL 10x Faster on Cloud in Minutes€¦ · PostgreSQL File system Linux page cache fsync() Linux issues I/Os in 4 KiB unit ... Configure. Test setup on AWS Performance](https://reader036.vdocument.in/reader036/viewer/2022063013/5fcc46812899160a333daf9c/html5/thumbnails/30.jpg)
Test setup on Azure
Performance on Azure Storage (1)
30
Ds4 v34 vCPUs16 GiB memorypgbench
Availability Zone B
Ds32 v332 vCPUs128 GiB memory256 GiB PremiumSSDPostgreSQL v11.5
Azure Seoul Region
![Page 31: Make Your PostgreSQL 10x Faster on Cloud in Minutes€¦ · PostgreSQL File system Linux page cache fsync() Linux issues I/Os in 4 KiB unit ... Configure. Test setup on AWS Performance](https://reader036.vdocument.in/reader036/viewer/2022063013/5fcc46812899160a333daf9c/html5/thumbnails/31.jpg)
PostgreSQL processes 11.5x more transactions with AppOS
Performance on Azure Storage (2)
31
![Page 32: Make Your PostgreSQL 10x Faster on Cloud in Minutes€¦ · PostgreSQL File system Linux page cache fsync() Linux issues I/Os in 4 KiB unit ... Configure. Test setup on AWS Performance](https://reader036.vdocument.in/reader036/viewer/2022063013/5fcc46812899160a333daf9c/html5/thumbnails/32.jpg)
Test setup on AWS
Performance on SSD (1)
32
r5.xlarge4 vCPUs16 GiB memorypgbench
AWS Seoul Region
Availability Zone A
r5d.4xlarge16 vCPUs128 GiB memory300 GiB NVMe SSDPostgreSQL v11.5
![Page 33: Make Your PostgreSQL 10x Faster on Cloud in Minutes€¦ · PostgreSQL File system Linux page cache fsync() Linux issues I/Os in 4 KiB unit ... Configure. Test setup on AWS Performance](https://reader036.vdocument.in/reader036/viewer/2022063013/5fcc46812899160a333daf9c/html5/thumbnails/33.jpg)
PostgreSQL processes 2.7x more transactions with AppOS
Performance on SSD (2)
33
![Page 34: Make Your PostgreSQL 10x Faster on Cloud in Minutes€¦ · PostgreSQL File system Linux page cache fsync() Linux issues I/Os in 4 KiB unit ... Configure. Test setup on AWS Performance](https://reader036.vdocument.in/reader036/viewer/2022063013/5fcc46812899160a333daf9c/html5/thumbnails/34.jpg)
AppOS can seamlessly work with PostgreSQL-derived DBs
Extensibility
34
![Page 35: Make Your PostgreSQL 10x Faster on Cloud in Minutes€¦ · PostgreSQL File system Linux page cache fsync() Linux issues I/Os in 4 KiB unit ... Configure. Test setup on AWS Performance](https://reader036.vdocument.in/reader036/viewer/2022063013/5fcc46812899160a333daf9c/html5/thumbnails/35.jpg)
TimescaleDB Performance (1)
35
Test setup on AWS
m5.2xlarge8 vCPUs32 GiB memoryTime-SeriesBenchmark Suitehttps://github.com/timescale/tsbs
AWS Seoul Region
Availability Zone A
m5.2xlarge8 vCPUs32 GiB memory16 TiB EBS HDD or1,000 IOPS EBS SSDPostgreSQL v11.5TimescaleDB v1.5
![Page 36: Make Your PostgreSQL 10x Faster on Cloud in Minutes€¦ · PostgreSQL File system Linux page cache fsync() Linux issues I/Os in 4 KiB unit ... Configure. Test setup on AWS Performance](https://reader036.vdocument.in/reader036/viewer/2022063013/5fcc46812899160a333daf9c/html5/thumbnails/36.jpg)
36
TimescaleDB Performance (2)
AppOS improves TimescaleDB inserts by 50%~3x
![Page 37: Make Your PostgreSQL 10x Faster on Cloud in Minutes€¦ · PostgreSQL File system Linux page cache fsync() Linux issues I/Os in 4 KiB unit ... Configure. Test setup on AWS Performance](https://reader036.vdocument.in/reader036/viewer/2022063013/5fcc46812899160a333daf9c/html5/thumbnails/37.jpg)
37
We’re partnering!
Try out AppOS via https://apposha.io
Join our Slack channel
https://apposha.io/support
https://appos-postgres.slack.com/