managing thousands of databases

32
MANAGING THOUSANDS OF DATABASE SERVERS Emre Hasegeli

Upload: emre-hasegeli

Post on 15-Jan-2017

232 views

Category:

Software


1 download

TRANSCRIPT

Page 1: Managing thousands of databases

MANAGING THOUSANDS OF DATABASE SERVERSEmre Hasegeli

Page 2: Managing thousands of databases

2

TOPICS

2

1

3

4

Configuration

Monitoring

Backups & Replication

Problems

Page 3: Managing thousands of databases

3

Page 4: Managing thousands of databases

4

Debian 8 Jessie

PostgreSQL 9.5 - 9.6

Community packages (dep, rpm)

`puppetlabs-postgresql` module

INSTALLATION

Page 5: Managing thousands of databases

5

INSTALLATION & CONFIGURATION

Page 6: Managing thousands of databases

6

INSTALLATION & CONFIGURATION

Page 7: Managing thousands of databases

7

INSTALLATION & CONFIGURATION

Page 8: Managing thousands of databases

8

INSTALLATION & CONFIGURATION

Page 9: Managing thousands of databases

9

INSTALLATION & CONFIGURATION

Page 10: Managing thousands of databases

10

INSTALLATION & CONFIGURATION

Page 11: Managing thousands of databases

11

INSTALLATION & CONFIGURATION

Page 12: Managing thousands of databases

12

INSTALLATION & CONFIGURATION

Page 13: Managing thousands of databases

13

INSTALLATION & CONFIGURATION

Page 14: Managing thousands of databases

14

INSTALLATION & CONFIGURATION

Page 15: Managing thousands of databases

15

MONITORING

Page 16: Managing thousands of databases

16

MONITORING

https://bucardo.org/wiki/Check_postgres

Page 17: Managing thousands of databases

17

MONITORING

Page 18: Managing thousands of databases

18

MONITORING

Page 19: Managing thousands of databases

19

MONITORING

Page 20: Managing thousands of databases

20

MONITORING

Page 21: Managing thousands of databases

21

MONITORING

Page 22: Managing thousands of databases

22

MONITORING

Page 23: Managing thousands of databases

23

MONITORING

https://labs.consol.de/nagios/check_logfiles/index.html

Page 24: Managing thousands of databases

24

BACKUP & REPLICATION

Page 25: Managing thousands of databases

25

BACKUP & REPLICATION

Page 26: Managing thousands of databases

26

BACKUP & REPLICATION

http://www.pgbarman.org

Page 27: Managing thousands of databases

27

BACKUP & REPLICATION

http://www.repmgr.org

Page 28: Managing thousands of databases

28

CONNECTION POOLING

https://pgbouncer.github.io

Page 29: Managing thousands of databases

29

CONNECTION POOLING

Page 30: Managing thousands of databases

30

CONNECTION POOLING

Page 31: Managing thousands of databases

31

CONNECTION POOLING

Page 32: Managing thousands of databases

32

PROBLEMS

Bloat

Unexpected WAL increase

Long running database migrations

Complex failover

pg_upgrade