what’s new in postgresql 9.6, by postgresql · pdf fileoperation postgresql 9.4...
TRANSCRIPT
![Page 1: What’s New in PostgreSQL 9.6, by PostgreSQL · PDF fileOperation PostgreSQL 9.4 PostgreSQL 9.5 PostgreSQL 9.6 SELECT - Support foreign table inheritance Support foreign table inheritance](https://reader031.vdocument.in/reader031/viewer/2022012320/5a78f0557f8b9a5a148dfaa4/html5/thumbnails/1.jpg)
Copyright©2016 NTT corp. All Rights Reserved.
What’s New in PostgreSQL 9.6,by PostgreSQL contributor
NTT OpenSource Software CenterMasahiko Sawada
@PGConf.ASIA 2016 (2 Dec)
![Page 2: What’s New in PostgreSQL 9.6, by PostgreSQL · PDF fileOperation PostgreSQL 9.4 PostgreSQL 9.5 PostgreSQL 9.6 SELECT - Support foreign table inheritance Support foreign table inheritance](https://reader031.vdocument.in/reader031/viewer/2022012320/5a78f0557f8b9a5a148dfaa4/html5/thumbnails/2.jpg)
2Copyright©2016 NTT corp. All Rights Reserved.
Who am I?
Masahiko Sawada
Twitter : @sawada_masahiko
PostgreSQL Contributor
Freeze Map
Multiple Sync Replication
PostgreSQL Support
Attended a developer meeting at PGCon 2016
Photo by Oleg Bartunov
![Page 3: What’s New in PostgreSQL 9.6, by PostgreSQL · PDF fileOperation PostgreSQL 9.4 PostgreSQL 9.5 PostgreSQL 9.6 SELECT - Support foreign table inheritance Support foreign table inheritance](https://reader031.vdocument.in/reader031/viewer/2022012320/5a78f0557f8b9a5a148dfaa4/html5/thumbnails/3.jpg)
3Copyright©2016 NTT corp. All Rights Reserved.
1. What’s new in PostgreSQL 9.6
2. Towards to PostgreSQL 10
3. Conclusion
Agenda
![Page 4: What’s New in PostgreSQL 9.6, by PostgreSQL · PDF fileOperation PostgreSQL 9.4 PostgreSQL 9.5 PostgreSQL 9.6 SELECT - Support foreign table inheritance Support foreign table inheritance](https://reader031.vdocument.in/reader031/viewer/2022012320/5a78f0557f8b9a5a148dfaa4/html5/thumbnails/4.jpg)
4Copyright©2016 NTT corp. All Rights Reserved.
• Open source Relational Database Management System.
• Great features.• Window function.
• Transactional DDL.
• etc.
• 20th anniversary!
• Latest version is 9.6.1 (27th Oct).• Version 9.1 is EOL.
PostgreSQL
![Page 5: What’s New in PostgreSQL 9.6, by PostgreSQL · PDF fileOperation PostgreSQL 9.4 PostgreSQL 9.5 PostgreSQL 9.6 SELECT - Support foreign table inheritance Support foreign table inheritance](https://reader031.vdocument.in/reader031/viewer/2022012320/5a78f0557f8b9a5a148dfaa4/html5/thumbnails/5.jpg)
5Copyright©2016 NTT corp. All Rights Reserved.
New Features
1. Parallel queries.
2. Avoid VACUUM on all-frozen page (Freeze Map).
3. Monitoring progress of VACUUM.
4. Phrase full text search.
5. Multiple synchronous replication.
6. synchronous_commit = ‘remote_apply ’
7. postgrse_fdw support remote jois, sorts, UPDATEs and DELETEs.
8. Trigger Kernel write-back.
9. Better wait information in pg_stat_activity.
10. pg_blocking_pids().
![Page 6: What’s New in PostgreSQL 9.6, by PostgreSQL · PDF fileOperation PostgreSQL 9.4 PostgreSQL 9.5 PostgreSQL 9.6 SELECT - Support foreign table inheritance Support foreign table inheritance](https://reader031.vdocument.in/reader031/viewer/2022012320/5a78f0557f8b9a5a148dfaa4/html5/thumbnails/6.jpg)
6Copyright©2016 NTT corp. All Rights Reserved.
What’s new in
PostgreSQL 9.6
bridging knowledge to health by paul bica
![Page 7: What’s New in PostgreSQL 9.6, by PostgreSQL · PDF fileOperation PostgreSQL 9.4 PostgreSQL 9.5 PostgreSQL 9.6 SELECT - Support foreign table inheritance Support foreign table inheritance](https://reader031.vdocument.in/reader031/viewer/2022012320/5a78f0557f8b9a5a148dfaa4/html5/thumbnails/7.jpg)
7Copyright©2016 NTT corp. All Rights Reserved.
Parallel queries(Robet Haas, Amit Kapila, David Rowley, many others)
![Page 8: What’s New in PostgreSQL 9.6, by PostgreSQL · PDF fileOperation PostgreSQL 9.4 PostgreSQL 9.5 PostgreSQL 9.6 SELECT - Support foreign table inheritance Support foreign table inheritance](https://reader031.vdocument.in/reader031/viewer/2022012320/5a78f0557f8b9a5a148dfaa4/html5/thumbnails/8.jpg)
8Copyright©2016 NTT corp. All Rights Reserved.
Perpendicularly aggregation
Aggregate
Result
Seq Scan
Table
=# SELECT count(*) FROM test_table;
![Page 9: What’s New in PostgreSQL 9.6, by PostgreSQL · PDF fileOperation PostgreSQL 9.4 PostgreSQL 9.5 PostgreSQL 9.6 SELECT - Support foreign table inheritance Support foreign table inheritance](https://reader031.vdocument.in/reader031/viewer/2022012320/5a78f0557f8b9a5a148dfaa4/html5/thumbnails/9.jpg)
9Copyright©2016 NTT corp. All Rights Reserved.
Parallel aggregation
Partial Agg Partial Agg Partial Agg
Gather
ResultResult Result
Parallel Seq Scan Parallel Seq Scan Parallel Seq Scan
Result Result
Table
Result
rows/3
rows/3
rows/3
Final Aggregate
=# SELECT count(*) FROM test_table;
![Page 10: What’s New in PostgreSQL 9.6, by PostgreSQL · PDF fileOperation PostgreSQL 9.4 PostgreSQL 9.5 PostgreSQL 9.6 SELECT - Support foreign table inheritance Support foreign table inheritance](https://reader031.vdocument.in/reader031/viewer/2022012320/5a78f0557f8b9a5a148dfaa4/html5/thumbnails/10.jpg)
10Copyright©2016 NTT corp. All Rights Reserved.
Plans
=# EXPLAIN (ANALYZE on, VERBOSE on, COSTS off, TIMING off) SELECT count(*) FROM lineitem;
QUERY PLAN
-------------------------------------------------------------------------------------------Aggregate (actual rows=1 loops=1)
Output: count(*)
-> Seq Scan on public.lineitem (actual rows=29999795 loops=1)
Output: l_orderkey, l_partkey, l_suppkey, l_linenumber, l_quantity, ...
=# EXPLAIN (ANALYZE on, VERBOSE on, COSTS off, TIMING off) SELECT count(*) FROM lineitem;
QUERY PLAN
-----------------------------------------------------------------------------------------
Finalize Aggregate (actual rows=1 loops=1)
Output: count(*)
-> Gather (actual rows=3 loops=1)
Output: (PARTIAL count(*))
Workers Planned: 2
Workers Launched: 2
-> Partial Aggregate (actual rows=1 loops=3)
Output: PARTIAL count(*)
Worker 0: actual rows=1 loops=1
Worker 1: actual rows=1 loops=1
-> Parallel Seq Scan on public.lineitem (actual rows = ...
Worker 0: actual rows=9838356 loops=1
Worker 1: actual rows=10019336 loops=1
■ Parallel
■ Non Parallel
![Page 11: What’s New in PostgreSQL 9.6, by PostgreSQL · PDF fileOperation PostgreSQL 9.4 PostgreSQL 9.5 PostgreSQL 9.6 SELECT - Support foreign table inheritance Support foreign table inheritance](https://reader031.vdocument.in/reader031/viewer/2022012320/5a78f0557f8b9a5a148dfaa4/html5/thumbnails/11.jpg)
11Copyright©2016 NTT corp. All Rights Reserved.
• Support• Sequential Scan
• Aggregate (e.g. count, sum, avg)
• Nested Loop Join
• Hash Join
• Not Support• UPDATE, DELETE
• Index Scan
• Sorting
• Merge Join
• DDL
• etc
As of 9.6 parallel queries
![Page 12: What’s New in PostgreSQL 9.6, by PostgreSQL · PDF fileOperation PostgreSQL 9.4 PostgreSQL 9.5 PostgreSQL 9.6 SELECT - Support foreign table inheritance Support foreign table inheritance](https://reader031.vdocument.in/reader031/viewer/2022012320/5a78f0557f8b9a5a148dfaa4/html5/thumbnails/12.jpg)
12Copyright©2016 NTT corp. All Rights Reserved.
• HPE ProLiant DL580 GEN9
• Intel Xeon E7-8890 v4 2.20GHz (4P/192 core (96-HT))
• 2TB RAM
• Workload Accelerator(PCIe SSD)
• Read : 715,000 IOPS, 3.0GB/s
• Write : 95,000 IOPS, 2.5GB/s
Evaluation on a great machine
Supported by
Thank you!
![Page 13: What’s New in PostgreSQL 9.6, by PostgreSQL · PDF fileOperation PostgreSQL 9.4 PostgreSQL 9.5 PostgreSQL 9.6 SELECT - Support foreign table inheritance Support foreign table inheritance](https://reader031.vdocument.in/reader031/viewer/2022012320/5a78f0557f8b9a5a148dfaa4/html5/thumbnails/13.jpg)
13Copyright©2016 NTT corp. All Rights Reserved.
0
50
100
150
200
250
300
350
400
0 20 40 60 80 100 120 140 160 180 200
Exe
cuti
on
Tim
e(s
)
Parallel Degree
Parallel Query on 192 cores machine
• All table data(400GB) is on the shared buffer; no disk access.
• Simple aggregation, count(*).
• Parallel query makes aggregation 19x faster!!
385 sec
21 sec
![Page 14: What’s New in PostgreSQL 9.6, by PostgreSQL · PDF fileOperation PostgreSQL 9.4 PostgreSQL 9.5 PostgreSQL 9.6 SELECT - Support foreign table inheritance Support foreign table inheritance](https://reader031.vdocument.in/reader031/viewer/2022012320/5a78f0557f8b9a5a148dfaa4/html5/thumbnails/14.jpg)
14Copyright©2016 NTT corp. All Rights Reserved.
• Compare DBT-3 benchmark result with Hive• Single PostgreSQL node, 192 parallel degree.
• 7 of 22 queries timed out(30min) at SF500.• shared_buffers = 800GB, work_mem = 3GB.
• 12 node Hadoop cluster (master 2, slave 10).
For your reference
3,006 3,966
8,272
102,417
40
6,000
28 177 1,029 2,111
18,346
0
20,000
40,000
60,000
80,000
100,000
120,000
SF1 SF10 SF50 SF100 SF500 SF1000
Tota
l Exe
cuti
on
Tim
e(se
c)
Hive PostgreSQL 9.1.3 PostgreSQL 9.6.0
- CDH4.1.2
- 24 cores (12-HT)
- 32G RAM
- 2TB SATA
- PostgresSQL 9.1.3
- 8 cores (4-HT)
- 16G RAM
- SATA
![Page 15: What’s New in PostgreSQL 9.6, by PostgreSQL · PDF fileOperation PostgreSQL 9.4 PostgreSQL 9.5 PostgreSQL 9.6 SELECT - Support foreign table inheritance Support foreign table inheritance](https://reader031.vdocument.in/reader031/viewer/2022012320/5a78f0557f8b9a5a148dfaa4/html5/thumbnails/15.jpg)
15Copyright©2016 NTT corp. All Rights Reserved.
3,006
3,966
8,272
40
6,000
28 177
1,029
2,111
0
1,000
2,000
3,000
4,000
5,000
6,000
7,000
8,000
9,000
SF1 SF10 SF50 SF100
Tota
l Exe
cuti
on
Tim
e(se
c)
Hive PostgreSQL 9.1.3 PostgreSQL 9.6.0
• Compare DBT-3 benchmark result with Hive (SF1 - SF100)
• Hive flushes data to disk whenever finished each job, which could be overhead.• Now in 2016 we should use Hive on Tez.
• On the other hand, PostgreSQL has the all table data in shared buffer.
• Parallel execution in memory bring us much performance improvement.
SF1 to SF100
- CDH4.1.2
- 24 cores (12-HT)
- 32G RAM
- 2TB SATA
- PostgresSQL 9.1.3
- 8 cores (4-HT)
- 16G RAM
- SATA
![Page 16: What’s New in PostgreSQL 9.6, by PostgreSQL · PDF fileOperation PostgreSQL 9.4 PostgreSQL 9.5 PostgreSQL 9.6 SELECT - Support foreign table inheritance Support foreign table inheritance](https://reader031.vdocument.in/reader031/viewer/2022012320/5a78f0557f8b9a5a148dfaa4/html5/thumbnails/16.jpg)
16Copyright©2016 NTT corp. All Rights Reserved.
• The queries executed using parallel queries take less than 300 sec.• In Q13, PostgreSQL planner selects the merge join.
• For DWH query, using parallel query is a key for SQL tuning.
Per DBT-3 query (SF100)
0.0
200.0
400.0
600.0
800.0
1000.0
1200.0
Q01 Q02 Q03 Q04 Q05 Q06 Q07 Q08 Q09 Q10 Q11 Q12 Q13 Q14 Q15 Q16 Q17 Q18 Q19 Q20 Q21 Q22
Exe
cuti
on
Tim
e(se
c)
Hive PostgreSQL
![Page 17: What’s New in PostgreSQL 9.6, by PostgreSQL · PDF fileOperation PostgreSQL 9.4 PostgreSQL 9.5 PostgreSQL 9.6 SELECT - Support foreign table inheritance Support foreign table inheritance](https://reader031.vdocument.in/reader031/viewer/2022012320/5a78f0557f8b9a5a148dfaa4/html5/thumbnails/17.jpg)
17Copyright©2016 NTT corp. All Rights Reserved.
Avoid VACUUM on
all-frozen page(Masahiko Sawada, Robert Haas, Andres Freund)
![Page 18: What’s New in PostgreSQL 9.6, by PostgreSQL · PDF fileOperation PostgreSQL 9.4 PostgreSQL 9.5 PostgreSQL 9.6 SELECT - Support foreign table inheritance Support foreign table inheritance](https://reader031.vdocument.in/reader031/viewer/2022012320/5a78f0557f8b9a5a148dfaa4/html5/thumbnails/18.jpg)
18Copyright©2016 NTT corp. All Rights Reserved.
• Necessary to prevent transaction ID(XID) wraparound failures.
• anti-wraparound VACUUM is invoked every 200 million transaction by default
• Previously it always scanned all table pages.• Could be performance degradation.
Freezing of database
![Page 19: What’s New in PostgreSQL 9.6, by PostgreSQL · PDF fileOperation PostgreSQL 9.4 PostgreSQL 9.5 PostgreSQL 9.6 SELECT - Support foreign table inheritance Support foreign table inheritance](https://reader031.vdocument.in/reader031/viewer/2022012320/5a78f0557f8b9a5a148dfaa4/html5/thumbnails/19.jpg)
19Copyright©2016 NTT corp. All Rights Reserved.
• Keep track of which pages are completely frozen.
• Avoid VACUUM on all-frozen pages.• Very effective for mostly-read tables.
Avoid VACUUM on all-frozen page (Freeze Map)
-- 9.6
=# VACUUM FREEZE large_table;
VACUUM
Time: 703509.523 ms
=# VACUUM FREEZE large_table;
VACUUM
Time: 222.719 ms
-- 9.5
=# VACUUM FREEZE large_table;
VACUUM
Time: 685363.793 ms
=# VACUUM FREEZE large_table;
VACUUM
Time: 711380.587 ms
![Page 20: What’s New in PostgreSQL 9.6, by PostgreSQL · PDF fileOperation PostgreSQL 9.4 PostgreSQL 9.5 PostgreSQL 9.6 SELECT - Support foreign table inheritance Support foreign table inheritance](https://reader031.vdocument.in/reader031/viewer/2022012320/5a78f0557f8b9a5a148dfaa4/html5/thumbnails/20.jpg)
20Copyright©2016 NTT corp. All Rights Reserved.
Monitoring progress of VACUUM
(Amit Langote, Robert Haas, Vinayak Pokale, Rahila Syed)
![Page 21: What’s New in PostgreSQL 9.6, by PostgreSQL · PDF fileOperation PostgreSQL 9.4 PostgreSQL 9.5 PostgreSQL 9.6 SELECT - Support foreign table inheritance Support foreign table inheritance](https://reader031.vdocument.in/reader031/viewer/2022012320/5a78f0557f8b9a5a148dfaa4/html5/thumbnails/21.jpg)
21Copyright©2016 NTT corp. All Rights Reserved.
• Introduce new system view pg_stat_progress_vacuum.
• Report progress of running VACUUM.
• Not supported for VACUUM FULL and CLUSTER.
Progress information of vacuum
=# ¥d pg_stat_progress_vacuum
Column | Type | Modifiers
--------------------+---------+----------
pid | integer |
datid | oid |
datname | name |
relid | oid |
phase | text |
heap_blks_total | bigint |
heap_blks_scanned | bigint |
heap_blks_vacuumed | bigint |
index_vacuum_count | bigint |
max_dead_tuples | bigint |
num_dead_tuples | bigint |
![Page 22: What’s New in PostgreSQL 9.6, by PostgreSQL · PDF fileOperation PostgreSQL 9.4 PostgreSQL 9.5 PostgreSQL 9.6 SELECT - Support foreign table inheritance Support foreign table inheritance](https://reader031.vdocument.in/reader031/viewer/2022012320/5a78f0557f8b9a5a148dfaa4/html5/thumbnails/22.jpg)
22Copyright©2016 NTT corp. All Rights Reserved.
=# SELECT pid, datname, relname,
now() - query_start as duration,
((heap_blks_scanned / heap_blks_total::numeric(10,2)) * 100) as percentage,
p.phase,
index_vacuum_count
FROM pg_stat_progress_vacuum as p, pg_class as c
WHERE p.relid = c.oid;
-[ RECORD 1 ]-------+-----------------
pid | 100026
datname | postgres
relname | pgbench_accounts
duration | 01:23:45.000000
percentage | 19.72
phase | scanning heap
index_vacuum_count | 10
-[ RECORD 2 ]-------+-----------------
pid | 100027
datname | postgres
relname | very_large_table
duration | 02:35:12.123456
percentage | 95.12
phase | scanning heap
index_vacuum_count | 300
Monitoring progress of VACUUM
- Table name- Duration- Progress (percentage)- Phase- Index vacuum count
![Page 23: What’s New in PostgreSQL 9.6, by PostgreSQL · PDF fileOperation PostgreSQL 9.4 PostgreSQL 9.5 PostgreSQL 9.6 SELECT - Support foreign table inheritance Support foreign table inheritance](https://reader031.vdocument.in/reader031/viewer/2022012320/5a78f0557f8b9a5a148dfaa4/html5/thumbnails/23.jpg)
23Copyright©2016 NTT corp. All Rights Reserved.
Phrase full text search (Teodor Sigaev, Oleg Bartunov, Dmitry Ivanov)
![Page 24: What’s New in PostgreSQL 9.6, by PostgreSQL · PDF fileOperation PostgreSQL 9.4 PostgreSQL 9.5 PostgreSQL 9.6 SELECT - Support foreign table inheritance Support foreign table inheritance](https://reader031.vdocument.in/reader031/viewer/2022012320/5a78f0557f8b9a5a148dfaa4/html5/thumbnails/24.jpg)
24Copyright©2016 NTT corp. All Rights Reserved.
• Search for words positioned relative to other words.
• Added new tsquery operators ‘<->’ and ‘<N>’
• ‘index <-> scan’ means that ‘scan’ follows by ‘index’.• Match to ‘index scan’.
• ‘index <2> scan’ means that ‘index’ and ‘index’ separated by at most 1 other word.• Match to ‘index only scan’.
Search for Phrases
![Page 25: What’s New in PostgreSQL 9.6, by PostgreSQL · PDF fileOperation PostgreSQL 9.4 PostgreSQL 9.5 PostgreSQL 9.6 SELECT - Support foreign table inheritance Support foreign table inheritance](https://reader031.vdocument.in/reader031/viewer/2022012320/5a78f0557f8b9a5a148dfaa4/html5/thumbnails/25.jpg)
25Copyright©2016 NTT corp. All Rights Reserved.
=# SELECT title, body FROM pgdoc
WHERE content @@ to_tsquery('parallel <-> execute') LIMIT 1;
-[ RECORD 1 ]--------------------------
title | Foreign Data Wrapper Callback Routines
body | A ForeignScan node can, optionally, support
| parallel execution. …
=# SELECT title, body FROM pgdoc
WHERE content @@ to_tsquery('parallel <2> execute') LIMIT 1;
-[ RECORD 1 ]--------------------------
title | When Can Parallel Query Be Used?
body | Even when parallel query is generated for …
| impossible to execute that plan in parallel at execution …
=# SELECT title, body FROM pgdoc
WHERE content @@ to_tsquery('parallel <3> execute') LIMIT 1;
-[ RECORD 1 ]--------------------------
title | How Parallel Query Works
body | Every background worker process which is …
| parallel query will execute the portion of …
Search for ‘parallel <->/<N> execute’
![Page 26: What’s New in PostgreSQL 9.6, by PostgreSQL · PDF fileOperation PostgreSQL 9.4 PostgreSQL 9.5 PostgreSQL 9.6 SELECT - Support foreign table inheritance Support foreign table inheritance](https://reader031.vdocument.in/reader031/viewer/2022012320/5a78f0557f8b9a5a148dfaa4/html5/thumbnails/26.jpg)
26Copyright©2016 NTT corp. All Rights Reserved.
Multiple synchronous replication
(Masahiko Sawada, Beena Emerson, Michael Paquier, Fujii Masao, Kyotaro Horiguchi)
![Page 27: What’s New in PostgreSQL 9.6, by PostgreSQL · PDF fileOperation PostgreSQL 9.4 PostgreSQL 9.5 PostgreSQL 9.6 SELECT - Support foreign table inheritance Support foreign table inheritance](https://reader031.vdocument.in/reader031/viewer/2022012320/5a78f0557f8b9a5a148dfaa4/html5/thumbnails/27.jpg)
27Copyright©2016 NTT corp. All Rights Reserved.
History of Replication feature
9.0(2010)
9.4(2014)
9.5(2015)
9.6(2016)
9.2(2012)
9.3(2013)
9.1(2011)
Synchronous Replication
Cascade Replication
Fast Failover
- Replication Slot- Logical Decoding
- WAL Compression- Fast Failback
(pg_rewind)
M S S
M S
S
AsynchronousReplication
![Page 28: What’s New in PostgreSQL 9.6, by PostgreSQL · PDF fileOperation PostgreSQL 9.4 PostgreSQL 9.5 PostgreSQL 9.6 SELECT - Support foreign table inheritance Support foreign table inheritance](https://reader031.vdocument.in/reader031/viewer/2022012320/5a78f0557f8b9a5a148dfaa4/html5/thumbnails/28.jpg)
28Copyright©2016 NTT corp. All Rights Reserved.
Synchronous Replication (~9.5)
sync
async
async
=# SHOW synchronous_standby_names;
synchronous_standby_names
------------------------------
standby1, standby2, standby3
=# SELECT application_name, sync_state, sync_priority
FROM pg_stat_replication;
application_name | sync_state | sync_priority
------------------+------------+---------------
standby1 | sync | 1
standby2 | potential | 2
standby3 | potential | 3
standby1
standby2
standby3
![Page 29: What’s New in PostgreSQL 9.6, by PostgreSQL · PDF fileOperation PostgreSQL 9.4 PostgreSQL 9.5 PostgreSQL 9.6 SELECT - Support foreign table inheritance Support foreign table inheritance](https://reader031.vdocument.in/reader031/viewer/2022012320/5a78f0557f8b9a5a148dfaa4/html5/thumbnails/29.jpg)
29Copyright©2016 NTT corp. All Rights Reserved.
Multiple Synchronous Replication (9.6~)
=# SHOW synchronous_standby_names;
synchronous_standby_names
---------------------------------
2(standby1, standby2, standby3)
=# SELECT application_name, sync_state, sync_priority
FROM pg_stat_replication;
application_name | sync_state | sync_priority
------------------+------------+---------------
standby1 | sync | 1
standby2 | sync | 2
standby3 | potential | 3
sync
sync
async
standby1
standby2
standby3
![Page 30: What’s New in PostgreSQL 9.6, by PostgreSQL · PDF fileOperation PostgreSQL 9.4 PostgreSQL 9.5 PostgreSQL 9.6 SELECT - Support foreign table inheritance Support foreign table inheritance](https://reader031.vdocument.in/reader031/viewer/2022012320/5a78f0557f8b9a5a148dfaa4/html5/thumbnails/30.jpg)
30Copyright©2016 NTT corp. All Rights Reserved.
New syntax of synchronous_standby_names
‘2 ( standby1, standby2, standby3)’
The number of
sync standbys Standby names
• When all three standbys are available,
• standby1 and standby2 are synchronous standbys.
• standby3 is potential standby (async).
• After standby1 crashed,
• standby3 becomes to synchronous standby.
• standby2 and standby3 are synchronous standbys.
Standby namesStandby names
![Page 31: What’s New in PostgreSQL 9.6, by PostgreSQL · PDF fileOperation PostgreSQL 9.4 PostgreSQL 9.5 PostgreSQL 9.6 SELECT - Support foreign table inheritance Support foreign table inheritance](https://reader031.vdocument.in/reader031/viewer/2022012320/5a78f0557f8b9a5a148dfaa4/html5/thumbnails/31.jpg)
31Copyright©2016 NTT corp. All Rights Reserved.
synchronous_commit= ‘remote_apply’
(Thomas Munro)
![Page 32: What’s New in PostgreSQL 9.6, by PostgreSQL · PDF fileOperation PostgreSQL 9.4 PostgreSQL 9.5 PostgreSQL 9.6 SELECT - Support foreign table inheritance Support foreign table inheritance](https://reader031.vdocument.in/reader031/viewer/2022012320/5a78f0557f8b9a5a148dfaa4/html5/thumbnails/32.jpg)
32Copyright©2016 NTT corp. All Rights Reserved.
On master server
On standby server
Flush WAL Write WAL Flush WAL Apply WAL
off Not Wait Not Wait Not Wait Not Wait
local Wait Not Wait Not Wait Not Wait
remote_write Wait Wait Not Wait Not Wait
on Wait Wait Wait Not Wait
remote_apply Wait Wait Wait Wait
synchronous_commit = remote_apply
synchronosu_commit = [off | local | remote_write | on | remote_apply]New!
![Page 33: What’s New in PostgreSQL 9.6, by PostgreSQL · PDF fileOperation PostgreSQL 9.4 PostgreSQL 9.5 PostgreSQL 9.6 SELECT - Support foreign table inheritance Support foreign table inheritance](https://reader031.vdocument.in/reader031/viewer/2022012320/5a78f0557f8b9a5a148dfaa4/html5/thumbnails/33.jpg)
33Copyright©2016 NTT corp. All Rights Reserved.
• With remote_apply and synchronous replication, committed data is visible on both the master server and the slave server.
• The client can always see updated data even on the slave server.
Read balancing with remote_apply
![Page 34: What’s New in PostgreSQL 9.6, by PostgreSQL · PDF fileOperation PostgreSQL 9.4 PostgreSQL 9.5 PostgreSQL 9.6 SELECT - Support foreign table inheritance Support foreign table inheritance](https://reader031.vdocument.in/reader031/viewer/2022012320/5a78f0557f8b9a5a148dfaa4/html5/thumbnails/34.jpg)
34Copyright©2016 NTT corp. All Rights Reserved.
Postgres_fdw support remote joins, sorts,
UPDATEs and DELETEs(Etsuro Fujita, Shigeru Hanada, Ashutosh Bapat)
![Page 35: What’s New in PostgreSQL 9.6, by PostgreSQL · PDF fileOperation PostgreSQL 9.4 PostgreSQL 9.5 PostgreSQL 9.6 SELECT - Support foreign table inheritance Support foreign table inheritance](https://reader031.vdocument.in/reader031/viewer/2022012320/5a78f0557f8b9a5a148dfaa4/html5/thumbnails/35.jpg)
35Copyright©2016 NTT corp. All Rights Reserved.
What’s the FDW?
ID, COL100,Tokyo200,Akihabara
id col
100 Tokyo
200 Akihabara
id col
100 Tokyo
200 Akihabara
CSV file
SELECT ..
id col
100 Tokyo
200 Akihabara
A lot of FDW are available!!
You can have FDW from
• postgres_fdw
• oracle_fdw
• mysql_fdw
• redis_fdw
:
file_fdw postgres_fdw mysql_fdw
![Page 36: What’s New in PostgreSQL 9.6, by PostgreSQL · PDF fileOperation PostgreSQL 9.4 PostgreSQL 9.5 PostgreSQL 9.6 SELECT - Support foreign table inheritance Support foreign table inheritance](https://reader031.vdocument.in/reader031/viewer/2022012320/5a78f0557f8b9a5a148dfaa4/html5/thumbnails/36.jpg)
36Copyright©2016 NTT corp. All Rights Reserved.
Operation PostgreSQL 9.4 PostgreSQL 9.5 PostgreSQL 9.6
SELECT -Support foreign
table inheritanceSupport foreign
table inheritance
WHERE clause Push down Push down Push down
Aggregate Local Local Local
Sort Local Local Push down
Join Local Local Push down
UPDATE,DELETE
Tuple basedusing CURSOR
Tuple basedusing CURSOR
Directly execution
INSERTINSERT to remote server using Prepare/Execute
INSERT to remote serverusing Prepare/Execute
INSERT to remote serverusing Prepare/Execute
Improvement of FDW API and postgres_fdw
postgres_fdw
SELECT colFROM foreign_tableORDER BY col DESC;
SELECT colFROM foreign_tableORDER BY col DESC;
![Page 37: What’s New in PostgreSQL 9.6, by PostgreSQL · PDF fileOperation PostgreSQL 9.4 PostgreSQL 9.5 PostgreSQL 9.6 SELECT - Support foreign table inheritance Support foreign table inheritance](https://reader031.vdocument.in/reader031/viewer/2022012320/5a78f0557f8b9a5a148dfaa4/html5/thumbnails/37.jpg)
37Copyright©2016 NTT corp. All Rights Reserved.
Sort push down
-- 9.6
=# EXPLAIN (VEROBSE on, COSTS off) SELECT * FROM f_table ORDER BY col DESC;
QUERY PLAN
----------------------------------------------------------------------------
Foreign Scan on public.f_table
Output: col
Remote SQL: SELECT col FROM public.f_table ORDER BY col DESC NULLS FIRST
-- 9.5
=# EXPLAIN (VEROBSE on, COSTS off) SELECT * FROM f_table ORDER BY col DESC;
QUERY PLAN
----------------------------------------------------
Sort
Output: col
Sort Key: f_table.col DESC
-> Foreign Scan on public.f_table
Output: col
Remote SQL: SELECT col FROM public.f_table
![Page 38: What’s New in PostgreSQL 9.6, by PostgreSQL · PDF fileOperation PostgreSQL 9.4 PostgreSQL 9.5 PostgreSQL 9.6 SELECT - Support foreign table inheritance Support foreign table inheritance](https://reader031.vdocument.in/reader031/viewer/2022012320/5a78f0557f8b9a5a148dfaa4/html5/thumbnails/38.jpg)
38Copyright©2016 NTT corp. All Rights Reserved.
Join push down
-- 9.5
=# EXPLAIN (VERBOSE on, COSTS off) SELECT * FROM f_table a JOIN f_table2 b ON a.col = b.col LIMIT 10;
QUERY PLAN
------------------------------------------------------------------------
Limit
Output: a.col, b.col
-> Nested Loop
Output: a.col, b.col
Join Filter: (a.col = b.col)
-> Foreign Scan on public.f_table a
Output: a.col
Remote SQL: SELECT col FROM public.f_table
-> Materialize
Output: b.col
-> Foreign Scan on public.f_table2 b
Output: b.col
Remote SQL: SELECT col FROM public.f_table2
-- 9.6
=# EXPLAIN (VERBOSE on, COSTS off) SELECT * FROM f_table a JOIN f_table2 b ON a.col = b.col LIMIT 10;
QUERY PLAN
--------------------------------------------------------------------------------------------------
Limit
Output: a.col, b.col
-> Foreign Scan
Output: a.col, b.col
Relations: (public.f_table a) INNER JOIN (public.f_table2 b)
Remote SQL: SELECT r1.col, r2.col FROM (public.f_table r1 INNER JOIN public.f_table2 r2 ON (((r1.col = r2.col))))
![Page 39: What’s New in PostgreSQL 9.6, by PostgreSQL · PDF fileOperation PostgreSQL 9.4 PostgreSQL 9.5 PostgreSQL 9.6 SELECT - Support foreign table inheritance Support foreign table inheritance](https://reader031.vdocument.in/reader031/viewer/2022012320/5a78f0557f8b9a5a148dfaa4/html5/thumbnails/39.jpg)
39Copyright©2016 NTT corp. All Rights Reserved.
Trigger kernel writeback(Fabien Coelho, Andres Freund)
![Page 40: What’s New in PostgreSQL 9.6, by PostgreSQL · PDF fileOperation PostgreSQL 9.4 PostgreSQL 9.5 PostgreSQL 9.6 SELECT - Support foreign table inheritance Support foreign table inheritance](https://reader031.vdocument.in/reader031/viewer/2022012320/5a78f0557f8b9a5a148dfaa4/html5/thumbnails/40.jpg)
40Copyright©2016 NTT corp. All Rights Reserved.
• PostgreSQL writes data to the kernel’s disk cache.
• Write-back could be cause I/O storms
• Can be configured on global level• vm.dirty_background_ratio etc
Kernel write-back configurations
![Page 41: What’s New in PostgreSQL 9.6, by PostgreSQL · PDF fileOperation PostgreSQL 9.4 PostgreSQL 9.5 PostgreSQL 9.6 SELECT - Support foreign table inheritance Support foreign table inheritance](https://reader031.vdocument.in/reader031/viewer/2022012320/5a78f0557f8b9a5a148dfaa4/html5/thumbnails/41.jpg)
41Copyright©2016 NTT corp. All Rights Reserved.
• New configure parameters• checkpoint_flush_after
• 256kB by default
• bgwriter_flush_after
• 512kB by default
• backend_flush_after
• 0 by default
• wal_writer_flush_after
• 1MB by default
• Enable by default on Linux only
Kernel write-back configurations
![Page 42: What’s New in PostgreSQL 9.6, by PostgreSQL · PDF fileOperation PostgreSQL 9.4 PostgreSQL 9.5 PostgreSQL 9.6 SELECT - Support foreign table inheritance Support foreign table inheritance](https://reader031.vdocument.in/reader031/viewer/2022012320/5a78f0557f8b9a5a148dfaa4/html5/thumbnails/42.jpg)
42Copyright©2016 NTT corp. All Rights Reserved.
Better wait information in pg_stat_activity
(Amit Kapila, Ildus Kurbangaliev)
![Page 43: What’s New in PostgreSQL 9.6, by PostgreSQL · PDF fileOperation PostgreSQL 9.4 PostgreSQL 9.5 PostgreSQL 9.6 SELECT - Support foreign table inheritance Support foreign table inheritance](https://reader031.vdocument.in/reader031/viewer/2022012320/5a78f0557f8b9a5a148dfaa4/html5/thumbnails/43.jpg)
43Copyright©2016 NTT corp. All Rights Reserved.
• Tracking of wait event.
• Removed ‘waiting’ coliumn
• Added ‘wait_event_type’ column• The type of event for which the backend is waiting.
• Added ‘wait_event’ column• Wait event name.
Details of wait information in pg_stat_activity
=# SELECT pid, query, wait_event_type, wait_event
FROM pg_stat_activity;
pid | query | wait_event_type | wait_event
-------+---------------------+-----------------+--------------------
12345 | SELECT * FROM hoge; | |
90000 | VACUUM FULL hoge; | Lock | relation
10000 | SELECT * FROM bar; | LWLockNamed | XidGenLock
20000 | UPDATE bar SET…; | LWLockTranche | replication_slot_io
30000 | INSERT INTO … | BufferPin | BufferPin
(5 rows)
![Page 44: What’s New in PostgreSQL 9.6, by PostgreSQL · PDF fileOperation PostgreSQL 9.4 PostgreSQL 9.5 PostgreSQL 9.6 SELECT - Support foreign table inheritance Support foreign table inheritance](https://reader031.vdocument.in/reader031/viewer/2022012320/5a78f0557f8b9a5a148dfaa4/html5/thumbnails/44.jpg)
44Copyright©2016 NTT corp. All Rights Reserved.
pg_blocking_pids() function
(Tom Lane)
![Page 45: What’s New in PostgreSQL 9.6, by PostgreSQL · PDF fileOperation PostgreSQL 9.4 PostgreSQL 9.5 PostgreSQL 9.6 SELECT - Support foreign table inheritance Support foreign table inheritance](https://reader031.vdocument.in/reader031/viewer/2022012320/5a78f0557f8b9a5a148dfaa4/html5/thumbnails/45.jpg)
45Copyright©2016 NTT corp. All Rights Reserved.
pg_blocking_pids() function
• Returns an array of the PIDs that are blocking the session given PID.
=# SELECT pid, query, wait_event_type, wait_event
FROM pg_stat_activity;
pid | query | wait_event_type | wait_event
-------+--------------------+-----------------+------------
12345 | SELECT * FROM hoge; | |
90000 | VACUUM FULL hoge; | Lock | relation
55555 | SELECT * FROM bar; | Lock | transactionid
(3 rows)
=# SELECT pg_blocking_pids(90000);
pg_blocking_pids
------------------
{12345}
(1 row)
![Page 46: What’s New in PostgreSQL 9.6, by PostgreSQL · PDF fileOperation PostgreSQL 9.4 PostgreSQL 9.5 PostgreSQL 9.6 SELECT - Support foreign table inheritance Support foreign table inheritance](https://reader031.vdocument.in/reader031/viewer/2022012320/5a78f0557f8b9a5a148dfaa4/html5/thumbnails/46.jpg)
46Copyright©2016 NTT corp. All Rights Reserved.
Towards to
PostgreSQL 10
Future Energy by Floris Oosterveld
![Page 47: What’s New in PostgreSQL 9.6, by PostgreSQL · PDF fileOperation PostgreSQL 9.4 PostgreSQL 9.5 PostgreSQL 9.6 SELECT - Support foreign table inheritance Support foreign table inheritance](https://reader031.vdocument.in/reader031/viewer/2022012320/5a78f0557f8b9a5a148dfaa4/html5/thumbnails/47.jpg)
47Copyright©2016 NTT corp. All Rights Reserved.
New versioning scheme
~ 9.6 10 ~
9.6.0
9.5.0Major ver. Minor ver.
9.5.1 11.0
10.0
10.1
Major ver. Minor ver.
![Page 48: What’s New in PostgreSQL 9.6, by PostgreSQL · PDF fileOperation PostgreSQL 9.4 PostgreSQL 9.5 PostgreSQL 9.6 SELECT - Support foreign table inheritance Support foreign table inheritance](https://reader031.vdocument.in/reader031/viewer/2022012320/5a78f0557f8b9a5a148dfaa4/html5/thumbnails/48.jpg)
48Copyright©2016 NTT corp. All Rights Reserved.
• A number of companies publish its roadmap for PostgreSQL 10.• https://wiki.postgresql.org/wiki/PostgreSQL10_Roadmap
• Feedback is very welcome!!
PostgreSQL 10 Roadmap is available
![Page 49: What’s New in PostgreSQL 9.6, by PostgreSQL · PDF fileOperation PostgreSQL 9.4 PostgreSQL 9.5 PostgreSQL 9.6 SELECT - Support foreign table inheritance Support foreign table inheritance](https://reader031.vdocument.in/reader031/viewer/2022012320/5a78f0557f8b9a5a148dfaa4/html5/thumbnails/49.jpg)
49Copyright©2016 NTT corp. All Rights Reserved.
Conclusion
![Page 50: What’s New in PostgreSQL 9.6, by PostgreSQL · PDF fileOperation PostgreSQL 9.4 PostgreSQL 9.5 PostgreSQL 9.6 SELECT - Support foreign table inheritance Support foreign table inheritance](https://reader031.vdocument.in/reader031/viewer/2022012320/5a78f0557f8b9a5a148dfaa4/html5/thumbnails/50.jpg)
50Copyright©2016 NTT corp. All Rights Reserved.
• Over 200 new feature and improvement.
• Scale up with Parallel Query.
• Scale out with Synchronous Replication and postgres_fdw.
• Easier to use of very large database.
Conclusion
![Page 51: What’s New in PostgreSQL 9.6, by PostgreSQL · PDF fileOperation PostgreSQL 9.4 PostgreSQL 9.5 PostgreSQL 9.6 SELECT - Support foreign table inheritance Support foreign table inheritance](https://reader031.vdocument.in/reader031/viewer/2022012320/5a78f0557f8b9a5a148dfaa4/html5/thumbnails/51.jpg)
51Copyright©2016 NTT corp. All Rights Reserved.
Question?