database sharding with oracle rdbms - first impressions · –troubleshooting/performance tuning...
Post on 07-Jun-2018
218 Views
Preview:
TRANSCRIPT
BASEL BERN BRUGG DÜSSELDORF FRANKFURT A.M. FREIBURG I.BR. GENEVA
HAMBURG COPENHAGEN LAUSANNE MUNICH STUTTGART VIENNA ZURICH
Database Sharding with Oracle RDBMS First Impressions
Robert Bialek Principal Consultant
Who Am I
DOAG Conference 2016: Database Sharding with Oracle RDBMS - First Impressions
Principal Consultant and Trainer at Trivadis GmbH in Munich
– MSc in Computer Engineering
Focus:
– Oracle Database High Availability
– Database Architecture/Internals
– Backup/Recovery
– Troubleshooting/Performance Tuning
– Linux Operating System
Trainer for the following Trivadis courses:
– Oracle Grid Infrastructure, RAC, Data Guard
17.11.2016 2
Our company.
3
Trivadis is a market leader in IT consulting, system integration, solution engineering
and the provision of IT services focusing on and
technologies
in Switzerland, Germany, Austria and Denmark. We offer our services in the following
strategic business fields:
Trivadis Services takes over the interacting operation of your IT systems.
O P E R A T I O N
DOAG Conference 2016: Database Sharding with Oracle RDBMS - First Impressions 16.11.2016
COPENHAGEN
MUNICH
LAUSANNE
BERN
ZURICH
BRUGG
GENEVA
HAMBURG
DÜSSELDORF
FRANKFURT
STUTTGART
FREIBURG
BASEL
VIENNA
With over 600 specialists and IT experts in your region.
4
14 Trivadis branches and more than
600 employees
200 Service Level Agreements
Over 4,000 training participants
Research and development budget:
CHF 5.0 million
Financially self-supporting and
sustainably profitable
Experience from more than 1,900
projects per year at over 800
customers
DOAG Conference 2016: Database Sharding with Oracle RDBMS - First Impressions 16.11.2016
5
Technology on its own won't help you. You need to know how to use it properly.
DOAG Conference 2016: Database Sharding with Oracle RDBMS - First Impressions 16.11.2016
Database Scalability – Limits?
DOAG Conference 2016: Database Sharding with Oracle RDBMS - First Impressions
For some business requirements:
– extreme scalability/elasticity, global data distribution,…
know database techniques might not be sufficient.
The solution: Database Sharding (horizontal scaling)
– Not really a new concept: Cassandra, MongoDB, MySQL, …
One Database
Split one
into many
16.11.2016 6
Only 1/N
components
…
Agenda
DOAG Conference 2016: Database Sharding with Oracle RDBMS - First Impressions
1. Sharded Database Architecture
2. Data Sharding Methods
3. Data High Availability/Disaster Recovery
4. Session/Statement Routing
5. Conclusions
16.11.2016 7
DOAG Conference 2016: Database Sharding with Oracle RDBMS - First Impressions
Sharded Database Architecture
16.11.2016 8
SDB Architecture – Distributed Partitioning
DOAG Conference 2016: Database Sharding with Oracle RDBMS - First Impressions
Horizontally distributed data partitioning using shared-nothing database architecture
– Sharded tables: data exist in one shard
– Duplicated tables: data exist in all shards
Un
sh
ard
ed D
B (
Ta
ble
s)
Server B
Shard 1 Shard 2 Shard 3
Server C Server D
Sharded Database (SDB)
Server A
Distributed
Partitioning
16.11.2016 9
Don’t share physical resources
No cluster software
Up to 1000
non-CDB shards
SDB Architecture – Big Picture
DOAG Conference 2016: Database Sharding with Oracle RDBMS - First Impressions
Shard Director 1 Shard Director 2 Shard Director 3 Shard Director 4
Global Services: OLTP_RW, OLTP_RO, …
Shard Catalog Shard Catalog
Shards Shards
Data Center (Region): Germany Data Center (Region): Switzerland
16.11.2016 10
DOAG Conference 2016: Database Sharding with Oracle RDBMS - First Impressions
All tables in one table family must be equi-partitioned (sharding key)
– Two methods to create a table family: reference partitioning or the PARENT clause
16.11.2016 11
SDB Architecture – Sharded Tables
CUSTID FIRSTNAME …
100
101
102
ORDERID CUSTID ORDERDATE …
300 100
301 101
302 102
LINEITEMID CUSTID ORDERID …
1000 100 300
1001 100 300
1002 101 301
1003 101 301
1004 102 302
CUSTOMERS
ORDERS
ORDERITEMS
CREATE SHARDED TABLE customers (...
CREATE SHARDED TABLE orders (...
Ta
ble
Fa
mily
SDB Architecture – Duplicated Tables
DOAG Conference 2016: Database Sharding with Oracle RDBMS - First Impressions
Used to avoid cross-shard operations
Use materialized view replication
– The master table is created in the Shard Catalog database
– R/O materialized views in all shards
– All supporting objects created automatically in SDB
– SHRD_DUPL_TABLE_REFRESH_RATE initialization
parameter controls refresh frequency
Cannot coexist in a tablespace with sharded tables
16.11.2016 12
Shard 1 Shard 2 Shard 3
CREATE DUPLICATED TABLE products(
productid INTEGER PRIMARY KEY,
...
) TABLESPACE PRODUCTS_TBS;
Shard Catalog
SDB Architecture – DDL Execution
DOAG Conference 2016: Database Sharding with Oracle RDBMS - First Impressions
The application schema name as well as object names are identical in all shards
DDLs are executed in the Shard Catalog database or using GDSCTL
– Automatically propagated to all shards
16.11.2016 13
SQL> CONNECT SYS@SH_CATALOG
SQL> ALTER SESSION ENABLE SHARD DDL;
SQL> CREATE USER <app_name>...
SQL> GRANT CREATE TABLE TO <app_name>...
SQL> CREATE DUPLICATED TABLE <name>...
SQL> CREATE SHARDED TABLE <name>...
Shard Catalog
GDSCTL> sql "CREATE USER ..."
GDSCTL> sql "CREATE TABLESPACE
SET ..."
...
SDB Architecture – Sharded Tables (Chunks)
DOAG Conference 2016: Database Sharding with Oracle RDBMS - First Impressions
Physical data distribution based on chunks
– Each table partition is stored in different tablespace
16.11.2016 14
Chunk #1 CUSTOMERS_P1
ORDERS_P1 TBS_P1
Shard 1
Chunk #2 CUSTOMERS_P2
ORDERS_P2 TBS_P2
…
Chunk #7 CUSTOMERS_P7
ORDERS_P7 TBS_P7 Shard 2
Chunk #8 CUSTOMERS_P8
ORDERS_P8 TBS_P8
Chunks: 1-6
Chunks: 7-12
357913941
0
715827882
2147483646
2505397587
2863311528
SDB Architecture – Resharding (Hotspot Elimination)
Database Sharding with Oracle RDBMS 12c Release 2 - First Impressions
Adding/Removing shards or hotspot elimination requires chunk movement
– Automatically or manually
A chunk can also be spit manually within a shard
Hotspot
09.09.2016 15
Chunk R/O on
source
Chunk level 1
RMAN backup
Chunk level 0
RMAN backup
Copy files and
RMAN restore
FAN R/O
notification to
clients
File move/
restore
FAN chunk
up/down events
Drop the chunk
on the source
GDSCTL> MOVE CHUNK -CHUNK 12 -SOURCE sh01 -TARGET sh12
GDSCTL> SPLIT CHUNK -CHUNK 12
DOAG Conference 2016: Database Sharding with Oracle RDBMS - First Impressions
Data Sharding Methods
16.11.2016 16
Data Sharding Methods – System-Managed Sharding
DOAG Conference 2016: Database Sharding with Oracle RDBMS - First Impressions
Data is automatically distributed across shards using partitioning by consistent hash
– Expression: MOD(ORA_HASH('<SHARDING_KEY'),4294967295)
– Useful for data distribution
Automatic resharding in case of cardinality change
The number of chunks is defined during CREATE SHARDCATALOG and cannot be
changed later
A tablespace set needs to be created in the Shard Catalog database
SQL> CREATE TABLESPACE SET TBS_SET USING TEMPLATE
(DATAFILE SIZE 1G AUTOEXTEND ON NEXT 512M MAXSIZE UNLIMITED
EXTEND MANAGEMENT LOCAL SEGMENT SPACE MANAGEMENT AUTO);
16.11.2016 17
Data Sharding Methods – System-Managed Sharding
DOAG Conference 2016: Database Sharding with Oracle RDBMS - First Impressions
CREATE SHARDED TABLE customers
(
custid varchar2(20) CONSTRAINT
pk_customers PRIMARY KEY,
firstname varchar2(50),
lastname varchar2(50),
...
) TABLESPACE SET tbs_set
PARTITION BY CONSISTENT HASH (custid)
PARTITIONS AUTO;
16.11.2016 18
C001TS_SET
C002TS_SET
C003TS_SET
C004TS_SET
customers_P1
customers_P2
customers_P3
customers_P4
#Chunks=12
#Shards=3
C005TS_SET
C006TS_SET
C007TS_SET
C008TS_SET
customers_P5
customers_P6
customers_P7
customers_P8
C009TS_SET
C00ATS_SET
C00BTS_SET
C00CTS_SET
customers_P9
customers_P10
customers_P11
customers_P12
Tab
lesp
ace
Set
Shard Catalog
Shards
Data Sharding Methods – Composite Sharding
DOAG Conference 2016: Database Sharding with Oracle RDBMS - First Impressions
Data is first partitioned by range or list method across multiple shardspaces (set of
shards), then by consistent hash in all shards within a shardspace
16.11.2016 19
GDSCTL> ADD SHARDSPACE germany;
GDSCTL> ADD SHARDSPACE switzerland;
GDSCTL> ADD SHARD –CONNECT shdb01 –SHARDSPACE germany;
GDSCTL> ADD SHARD –CONNECT shdb02 –SHARDSPACE germany;
GDSCTL> ADD SHARD –CONNECT shdb02 –SHARDSPACE switzerland;
...
SQL> CREATE TABLESPACE SET de_tbs IN SHARDSPACE germany;
SQL> CREATE TABLESPACE SET ch_tbs IN SHARDSPACE switzerland;
Data Sharding Methods – Composite Sharding
DOAG Conference 2016: Database Sharding with Oracle RDBMS - First Impressions
CREATE SHARDED TABLE customers
(
custid varchar2(20),
country varchar2(2),
...
)
PARTITIONSET BY LIST (country)
PARTITION BY CONSISTENT HASH (custid)
PARTITIONS AUTO (
PARTITIONSET germany VALUES ('DE')
TABLESPACE SET de_tbs
PARTITIONSET switzerland VALUES ('CH')
TABLESPACE SET ch_tbs
...
16.11.2016 20
C001DE_TBS
C002DE_TBS
customers_P1
customers_P2
customers_P3
customers_P4
de_t
bs
Shardspace
GERMANY
Shards
C003DE_TBS
C004DE_TBS
C005CH_TBS
C006CH_TBS
customers_P5
customers_P6
customers_P7
customers_P8
ch_t
bs
Shardspace
SWITZERLAND C007CH_TBS
C008CH_TBS
DOAG Conference 2016: Database Sharding with Oracle RDBMS - First Impressions
Data High Availability
Disaster Recovery
16.11.2016 21
Data High Availability / Disaster Recovery – Overview
DOAG Conference 2016: Database Sharding with Oracle RDBMS - First Impressions
Data replication with Data Guard is a crucial component in SDB environment
– High availability, disaster recovery, read offloading
– Replication deployment performed fully automatically
– The logical unit of data replication is a shardgroup
Configuration
– Data Guard Broker with FSFO
– Local replica (HA), remote replica (DR)
– With or without ADG
16.11.2016 22
Shard Director 1 Shard Director 2
Primary Standby
Observers
Data Replication – For System-Managed Sharding
DOAG Conference 2016: Database Sharding with Oracle RDBMS - First Impressions
Server B
Shard 1 Shard 2 Shard 3 shgrp1
Server B
Shard 7 Shard 8 Shard 9 shgrp3
DC1
DC2
shgrp2 Shard 4 Shard 5 Shard 6
16.11.2016 23
HA
DR
Data Replication – For System-Managed Sharding
DOAG Conference 2016: Database Sharding with Oracle RDBMS - First Impressions
GDSCTL> create shardcatalog -database shd01:1521:repo
-chunks 12 -user mygdsadmin/<pwd> -sdb shdb
-region germany,switzerland –repl DG –sharding system
-protectmode maxavailability
...
GDSCTL> add shardgroup -shardgroup shgrp1 -deploy_as primary
-region germany
GDSCTL> add shardgroup -shardgroup shgrp2 -deploy_as active_standby
-region germany
GDSCTL> add shardgroup -shardgroup shgrp3 -deploy_as active_standby
-region switzerland
16.11.2016 24
HA
DR
Data Replication – For System-Managed Sharding
DOAG Conference 2016: Database Sharding with Oracle RDBMS - First Impressions
GDSCTL> create shard -shardgroup shgrp1 -destination sh01
-credential oracle_cred
GDSCTL> create shard -shardgroup shgrp2 -destination sh02
-credential oracle_cred
...
GDSCTL> create shard -shardgroup shgrp3 -destination sh03
-credential oracle_cred
...
GDSCTL> deploy
16.11.2016 25
Data Replication – For Composite Sharding
DOAG Conference 2016: Database Sharding with Oracle RDBMS - First Impressions
Server B
Shard 1 Shard 2 DC1
Server B
Shard 3 Shard 4
Server B
Shard 5
Shardspace GERMANY Shardspace SWITZERLAND
Shard 6
shgrp1
shgrp2
shgrp3
Shard 9 Shard 10
Shard 7 Shard 8 shgrp4
shgrp5
DC2
DC3
16.11.2016 26
Data Replication – For Composite Sharding
DOAG Conference 2016: Database Sharding with Oracle RDBMS - First Impressions
GDSCTL> create shardcatalog -database shd01:1521:repo
-user mygdsadmin/<pwd> -sdb shdb –chunks 12
-region germany,switzerland
–repl DG –sharding COMPOSITE
...
GDSCTL> add shardspace –shardspace germany -PROTECTMODE
maxavailability
GDSCTL> add shardspace –shardspace switzerland -PROTECTMODE
maxavailability
16.11.2016 27
Data Replication – For Composite Sharding
DOAG Conference 2016: Database Sharding with Oracle RDBMS - First Impressions
GDSCTL> add shardgroup -shardgroup shgrp1 -shardspace germany
-deploy_as primary -region germany
GDSCTL> add shardgroup -shardgroup shgrp2 -shardspace switzerland
-deploy_as active_standby -region switzerland
...
GDSCTL> add shardgroup -shardgroup shgrp4 -shardspace switzerland
-deploy_as primary -region switzerland
GDSCTL> create shard -shardgroup shgrp1 -destination sh01
-credential oracle_cred
...
GDSCTL> deploy
16.11.2016 28
DOAG Conference 2016: Database Sharding with Oracle RDBMS - First Impressions
Session/Statement Routing
16.11.2016 29
Session Routing
DOAG Conference 2016: Database Sharding with Oracle RDBMS - First Impressions
Session routing is done based on a sharding key (super sharding key)
– SQL operations are related to the specified sharding key (shard)
– To work on another sharding key value, a new session needs to be created
OLTP_RW.TRIVADIS.COM =
(DESCRIPTION=
(CONNECT_TIMEOUT=5)(RETRY_COUNT=5)(TRANSPORT_CONNECT_TIMEOUT=3)
(ADDRESS_LIST=
(LOAD_BALANCE=ON)
(ADDRESS=(PROTOCOL = TCP)(HOST=gsm01.trivadis.com)(PORT=1530))
(ADDRESS=(PROTOCOL = TCP)(HOST=gsm02.trivadis.com)(PORT=1530))
...
(CONNECT_DATA=
(SERVICE_NAME=oltp_rw.shdb.trivadis)
(SHARDING_KEY=robert.bialek@trivadis.com)(SUPER_SHARDING_KEY=germany)
(REGION=germany))) Required
Shard Directors
16.11.2016 30
Only Composite
Sharding
Session Routing (GDS)
DOAG Conference 2016: Database Sharding with Oracle RDBMS - First Impressions
Shard
Director 1
Region: GERMANY Region: SWITZERLAND
Data Replication
Shard Catalog
1
16.11.2016 31
Shard
Director 2
Shard
Director 3
Shard
Director 4
SHARDING_KEY=robert.bialek@trivadis.com
2
3
4
Shard Catalog
Connection
Pool
1 2
3
Statement Routing/Cross-Shard Query
DOAG Conference 2016: Database Sharding with Oracle RDBMS - First Impressions
Client connection to the Coordinator (Catalog) Database is required
– No sharding key necessary in the connect descriptor
SQL excuted via DB-Link on Shards
– Partition and Shard pruning
Shard Catalog
16.11.2016 32
Data Replication
@<host>:<port>/GDS$CATALOG.trivadis
Shard
Director 1
Region: GERMANY Region: SWITZERLAND
Shard
Director 2
Shard
Director 3
Shard
Director 4
Global Services (GDS)
Affinity to a Local Region
– Connection requests are routed to the least-loaded database only in a local region
Affinity to a Local Region with Interregion Failover
– Connection requests are routed only to the least-loaded database in a local region
– If the global service cardinality drops to 0, client is routed to other region
DOAG Conference 2016: Database Sharding with Oracle RDBMS - First Impressions 16.11.2016 33
GDSCTL> add service -service oltp_ro –role physical_standby
-locality LOCAL_ONLY -region_failover –lag 20
Data Replication
Region: GERMANY Region: SWITZERLAND
Global Services: OLTP_RW, OLTP_RO, …
DOAG Conference 2016: Database Sharding with Oracle RDBMS - First Impressions
Conclusions
16.11.2016 34
Conclusions
DOAG Conference 2016: Database Sharding with Oracle RDBMS - First Impressions
Use case
– Custom OLTP applications which require extreme scalability (global data
distribution)
Existing database applications need to be adopted for database sharding
– Primarily single shard access
– With cross-shard operations performance may suffer
Existing database frameworks/features have been extended to allow for massive
data scalability and fault isolation
– Special business requirements can now also be addressed with Oracle RDBMS
– But sharding is complex and requires a lot of knowledge!
16.11.2016 35
Trivadis @ DOAG 2016
Booth: 3rd Floor – next to the escalator
Know how, T-Shirts, Contest and more
We look forward to your visit
Because with Trivadis you always win !
36 DOAG Conference 2016: Database Sharding with Oracle RDBMS - First Impressions 17.11.2016
top related