Architecture and Interface of
Scalable Distributed Database System
SD-SQL Server
Soror SAHRI Witold LITWIN Thomas SCHWARTZ
[email protected] [email protected] [email protected]
Ceria Laboratory Comp. Eng. Dep. Paris-Dauphine University Santa
Clara U.
The IASTED International Conference onDBA 2006
IASTED-DBA 2006, Innsbruck, Feb 15th, 2006
2
Outline
IntroductionOverall ArchitectureApplication InterfaceImplementationConclusion
IASTED-DBA 2006, Innsbruck, Feb 15th, 2006
3
Most of DBSs have distributed/parallel versions SQL Server, Oracle, DB2
DBSs do not provide dynamically scalable tables. All require manual repartitioning when tables scale-up.
A Scalable Distributed Database System:
SD-DBS
Solution?
Introduction
IASTED-DBA 2006, Innsbruck, Feb 15th, 2006
4
Applies SDDS technology to DBSs
Why SDDSs?
Provide many scalable distributed partitioning schemes.
LH*, RP*, k-RP*, LH*RS…
These schemes can serve as the basis for SD-DBS architecture
Introduction
IASTED-DBA 2006, Innsbruck, Feb 15th, 2006
5
SD-SQL Server Architecture
Split
User/ApplicationUser/Application
LinkedSQL
Servers
D1NDBs D2 Di Di+1
_D1_T
SD-SQLserver
SD-SQLserver
SD-SQLclient
D1_T
SD-SQL Server
Managers
_D1_T_D1_T
T
sd_select
SD-SQLpeer
sd_insert
IASTED-DBA 2006, Innsbruck, Feb 15th, 2006
6
DB1 SDB
SD-SQL Server ArchitectureNodes, SDBs, NDBs
Node1 Node2 Node3 Node i
DB1
……
DB1 DB1
DB2 SDB
DB2 DB2
MDB
IASTED-DBA 2006, Innsbruck, Feb 15th, 2006
7
A scalable (distributed) table is a collection of segments Segments are SQL tables
A scalable table has, initially, only one primary segment At some server or peer NDB
The number of segments in a scalable table is variable. If a segment overflows, its split is triggered
SD-SQL Server ArchitectureScalable Tables: Segments
IASTED-DBA 2006, Innsbruck, Feb 15th, 2006
8
SD-SQL Server ArchitectureScalable Tables: Split
A split occurs when an insert overflows the segment capacity
Splits produce other segments for a scalable table. Each is located at a different NDB
Within the SDB
If there is not enough NDBs, splits dynamically append new ones
IASTED-DBA 2006, Innsbruck, Feb 15th, 2006
9
SD-SQL Server ArchitectureImages
Images hide the scalable table segments An image is a distributed updateable partitioned
view of a scalable tableUnion-all view with check constraints
An image presents the scalable table partitioning It do not address any new segments resulted from a split
Each scalable table has only one primary image and one or several secondary images
IASTED-DBA 2006, Innsbruck, Feb 15th, 2006
10
DB1 SDB
N1.DB1 N2.DB1 Ni.DB1
T Scalable
Table
Primary Image
CREATE VIEW T AS SELECT * FROM N1.DB1._N3_T
UNION ALL SELECT * FROM N2.DB1._N3_T
UNION ALL SELECT * FROM Ni.DB1._N3_T
SD-SQL Server ArchitectureImages
…
IASTED-DBA 2006, Innsbruck, Feb 15th, 2006
11
SD-SQL Server Application Interface
The application interface manipulates scalable tables through SD-SQL Server commands.
The SD-SQL Server commands start with ‘sd_’ to distinguish from SQL Server commands for static tables.
INSERT sd_insert
CREATE TABLE sd_create_table
IASTED-DBA 2006, Innsbruck, Feb 15th, 2006
12
SD-SQL Server Application Interface
Use of the SkyServer DB as benchmark
http://research.microsoft.com/~gray/SDSS PhotoObj table as a scalable table. PhotoObj has 158,426 tuples (about 260 MB)
Use of the laboratory machines Ceria, Dell1, Dell2….
IASTED-DBA 2006, Innsbruck, Feb 15th, 2006
13
Node Creation sd_create_node ‘Dell1’ /* Server by default */ sd_create_node ‘Ceria’, ‘client’
Node Alteration sd_alter_node ‘Ceria’, ‘ADD server’ /* Becomes peer*/
Node Removal sd_drop_node ‘Ceria’
SD-SQL Server Application Interface
Nodes Management
IASTED-DBA 2006, Innsbruck, Feb 15th, 2006
14
SD-SQL Server Application Interface
SDB/NDB Management SDB Creation
sd_create_scalable_database
‘SkyServer’, ‘Dell1’, ‘Server’, 2 /* Creates the primary SkyServer NDB as well at Dell1*/
SDB Alteration sd_create_node_database
‘SkyServer’, ‘Ceria’, ‘Client’ SDB Removal
sd_drop_scalable_database ‘SkyServer’
IASTED-DBA 2006, Innsbruck, Feb 15th, 2006
15
Scalable Table Creation sd_create_table ‘PhotoObj (objid BIGINT
PRIMARY KEY…)’, 10000
Scalable Table Alteration sd_alter_table ‘PhotoObj ADD t INT’, 1000 sd_create_index ‘run_index ON Photoobj (run)’ sd_drop_index ‘PhotoObj.run_index’
Scalable Table Removal sd_drop_table ‘PhotoObj’
SD-SQL Server Application Interface
Scalable Tables
IASTED-DBA 2006, Innsbruck, Feb 15th, 2006
16
SD-SQL Server Application Interface Images
Secondary Image Creation sd_create_image ‘Ceria’, ‘PhotoObj’ sd_create_image ‘Ceria2’, ‘PhotoObj’
Secondary Image Removal sd_drop_image 'PhotoObj’
IASTED-DBA 2006, Innsbruck, Feb 15th, 2006
17
SD-SQL Server Application Interface
Scalable Queries
USE Skyserver /* SQL Server command */ Scalable Update Queries
sd_insert ‘INTO PhotoObj SELECT * FROM Ceria5.Skyserver-S.PhotoObj’
Scalable Search Queries sd_select ‘* FROM PhotoObj’ sd_select ‘TOP 5000 * INTO PhotoObj1 FROM
PhotoObj’, 500
IASTED-DBA 2006, Innsbruck, Feb 15th, 2006
18
SD-SQL Application Interface
IASTED-DBA 2006, Innsbruck, Feb 15th, 2006
19
SD-SQL Application InterfaceCommand Processing
SD-SQL Server uses distributed stored procedures with dynamic SQL…
SD-SQL Server processes every command as a distributed transaction at Repeatable Read isolation level
See details in the papers “SD-SQL Server: a Scalable Distributed Database System”,
CERIA Research Report 2005-12-13, December 2005 “Overview of Scalable Distributed Database System SD-SQL
Server”, Intl. Workshop on Distributed Data and Structures, WDAS 2006, Santa Clara, CA, Carleton Scientific.
IASTED-DBA 2006, Innsbruck, Feb 15th, 2006
20
Conclusion
Scalable distributed databases with scalable tables are now a reality with SD-SQL Server No more manual repartitioning
Unlike in any other DBS we know about See the “Related Work” in the paper
The performance analysis proves Efficiency of our design Immediate utility of SD-SQL Server
Thank you.
For more details:
http://ceria.dauphine.fr