scaling postgresql
If you can't read please download the document
TRANSCRIPT
Click to edit the outline text formatSecond Outline LevelThird Outline LevelFourth Outline LevelFifth Outline LevelSixth Outline LevelSeventh Outline Level
Click to edit the outline text formatSecond Outline LevelThird Outline LevelFourth Outline LevelFifth Outline LevelSixth Outline LevelSeventh Outline Level
Click to edit the outline text formatSecond Outline LevelThird Outline LevelFourth Outline LevelFifth Outline LevelSixth Outline LevelSeventh Outline LevelClick to edit Master text stylesSecond levelThird levelFourth level
Fifth level
PostgreSQL
IBM POWER 8
PostgreSQL
PostgreSQL - (ORDBMS)
ANSI SQL (1992...2011),
NoSQL (key-value, JSON, JSONB)
Web: http://www.postgresql.org
: BSD, MIT-like
PostgreSQL - , . , , , . PostgreSQL -, .
PostgreSQL . PostgreSQL : Skype ( Microsoft), Yahoo, Instagram, BASF, Afilias .PostgreSQL . , PostgreSQL: Rambler, Yandex, Avito, Headhunter.
PostgreSQL PostgreSQL , PostgreSQL .PostgreSQL Unix, Linux, FreeBSD, Solaris, HPUX, Mac OS X, MS Windows. PostgreSQL (MVCC) . PostgreSQL . Postgres XL .
PostgreSQL , , . PostgreSQL BSD, . PostgreSQL ! PostgreSQL , , . PostgreSQL . PostgreSQL . .
, ,
PostgreSQL:
PostgreSQL . open source , . PostgreSQL , , ( ). : (MVCC), (Vacuum), (WAL), , .
, PostgreSQL, : , , (jsonb, hstore), , 1
Postgres Professional
PostgreSQL , , ,
2011 .
(, , )
PgConf.Russia - PostgreSQL
, , PostgreSQL * , , ,
20
PostgreSQL
4 : 3 PostgreSQL
Postgres Professional. , PostgreSQL.
Postgres Professional
*
*
PostgreSQL.
DISK
RAMRAMDISKCPUCPUCPUCPU
StorageShared Buffers
BackendBackendBackend
BgWorkers
PostgreSQL.
PostgreSQL , , , backend'a.
CPU, - CPU.
, PostgreSQL.
9.2 (select-only)
CPU . Robert Haas, EnterpriseDB, PostgreSQL.
2011 : PG 9.1 16 .
2012 , PG 9.2, 64
PostgreSQL .
9.4-9.5 (select-only)
IBM IBM 9119-MHE, 8 8 .
, , 9.2 PostgreSQL .
PostgreSQL 9.5 9.4 1.5 .
9.4-9.5 (select-only)
, X , PostgreSQL, Y .
, 60 9.4 150 9.5
CPU
perf top33.48% postgres [.] s_lock
2.51% postgres [.] GetSnapshotData
1.82% postgres [.] PinBuffer
, CPU.
perf.
Perf linux, . - . Perf : (~100-1000 ) , . , , , , . perf, postgres s_lock.
?
gdb#0 0x00003fffac40a858 in ___newselect_nocancel () from /lib64/power8/libc.so.6
#1 0x00000000106105f0 in pg_usleep (microsec=) at pgsleep.c:53
#2 0x00000000103e5f18 in s_lock (lock=0x3fe607980be0, file=0x10718398 "bufmgr.c", line=) at s_lock.c:110
#3 0x00000000103aea10 in UnpinBuffer (buf=0x3fe607980bc0, fixOwner=1 '\001') at bufmgr.c:1540
#4 0x00000000103b4910 in ReleaseAndReadBuffer (buffer=, relation=0x3fe6067073e0, blockNum=) at bufmgr.c:1401
, .
, . 10 .
, random probe gdb. , s_lock
(1.82% perf)
, Pin/UnPinBuffer
PostgreSQL Instance
Shared BuffersBackendBgWorkerspage X
BackendBackend
ClientClientClient
page 1page 2page 3page ... - PostgreSQL.
.
, , .
GDB , s_lock - .
.
s_lock?
* Semaphores+ (Posix)
-
* (Fetch And Add, ...)+ (gcc/clang, xlc, msvc)
- ,
* Spin lock+ - .
s_lock , BottleNeck. , ( ), . PostgreSQL PostgreSQL .1. : , , . Posix. .2. , .3. SPIN LOCK , - sleep.
s_lock spin lock
CPU
:perf top
13.75% postgres [.] GetSnapshotData
4.88% postgres [.] AllocSetAlloc
2.47% postgres [.] LWLockAcquire
:perf top
33.48% postgres [.] s_lock
2.51% postgres [.] GetSnapshotData
1.82% postgres [.] PinBuffer
, , BufferManager s_lock .
PG 9.5 vanila vs PG 9.5 atomic patch
900 .
350 .
!
PostgreSQL use-case 250 CPU
IBM Power 8 64 Cores
PostgreSQL LPAR *
Power Hypervisor PostgreSQL**
* Stable PostgreSQL 9.4 cpu > 120
** ! !
?
, PostgreSQL LPAR.
LPAR PostgreSQL .
C PostrgreSQL , , .
, .
PostgreSQL .
Buffer Manager*
* 2 (!!!)
Buffer Manager'.
Buffer Manager PostgreSQL 9.6
*
* +10%
, 10%
Buffer Manager NUMA- * **
* +10-15%
**
NUMA-.
, 10-15% , numa-.
1,8 IBM Power 8 / 64 Core * **
* ,
** PostgreSQL, LPAR.
:)
!
Postgres Professional
http://www.postgrespro.ru/[email protected]
!
!
M
X
Linux PPC :)