Chaps26.1
CSE 4701
Chapter 26, 6e - 24, 5e
Distributed Databases
Prof. Steven A. Demurjian, Sr. Computer Science & Engineering Department
The University of Connecticut191 Auditorium Road, Box U-155
Storrs, CT 06269-3155 [email protected]
http://www.engr.uconn.edu/~steve(860) 486 - 4818
A portion of these slides are being used with the permission of Dr. Ling Lui, Associate Professor, College of Computing, Georgia Tech.
Remaining slides represent new material.
Chaps26.2
CSE 4701
Classical and Distributed Architectures Classic/Centralized DBMS Dominated the
Commercial Market from 1970s Forward Problems of this Approach
Difficult to Scale w.r.t. Performance Gains If DB Overloaded, replace with a Faster Computer this can Only Go So Far - Disk Bottlenecks
Distributed DBMS have Evolved to Address a Number of Issues Improved Performance Putting Data “Near” Location where it is Needed Replication of Data for Fault Tolerance Vertical and Horizontal Partitioning of DB Tuples
Chaps26.3
CSE 4701
Common Features of Centralized DBMS Data Independence
High-Level Representation via Conceptual and External Schemas
Physical Representation (Internal Schema) Hidden Program Independence
Multiple Applications can Share Data Views/External Schema Support this Capability
Reduction of Program/Data Redundancy Single, Unique, Conceptual Schema Shared Database
Almost No Data Redundancy Controlled Data Access Reduces Inconsistencies Programs Execute with Consistent Results
Chaps26.4
CSE 4701
Common Features of Centralized DBMS Promote Sharing: Automatically Provided via CC
No Longer Programmatic Issue Most DBMS Offer Locking for Key Shared Data
Oracle Allows Locks on Data Item (Attributes) For Example, Controlling Access to Shared Identifier
Coherent and Central DB Administration Semantic DB Integrity via the Automatic Enforcement
of Data Consistency via Integrity Constraints/Rules Data Resiliency
Physical Integrity of Data in the Presence of Faults and Errors
Supported by DB Recovery Data Security: Control Access for Authorized Users
Against Sensitive Data
Chaps26.5
CSE 4701
Shared Nothing Architecture In this Architecture, Each DBMS
Operates Autonomously There is No Sharing
Three Separate DBMSs on Three Different Computers
Applications/Clients Must Know About the External Schemas of all Three DBMSs for Database Retrieval Client Processing
Complicates Client Different DBMS
Platforms(Oracle, Sybase, Informix, ..)
Different Access Modes(Query, Embedded, ODBC)
Difficult for SWE to Code
Chaps26.6
CSE 4701
Difficulty in Access – Manage Multiple APIs Each Platform has a Different API
API1 , API3 , …. , APIn An App Programmer Must Utilize All three APIs
which could differ by PL – C++, C, Java, REST, etc.
Any interactions Across 3 DBs – must be programmatically handled without DB Capabilities
API1 API2APIn
Chaps26.7
CSE 4701
NW Architecture with Centralized DB High-Speed NWs/WANs Spawned Centralized DB
Accessible Worldwide Clients at Any Site can Access Repository Data May be “Far” Away - Increased Access Time In Practice, Each Remote Site Needs only Portion
of the Data in DB1 and/or DB2 Inefficient, no Replication w.r.t. Failure
Chaps26.8
CSE 4701
Fully Distributed Architecture The Five Sites (Chicago, SF, LA, NY, Atlanta) each
have a “Portion” of the Database - its Distributed Replication is Possible for Fault Tolerance Queries at one Site May Need to Access Data at
Another Site (e.g., for a Join) Increased Transaction Processing Complexity
Chaps26.9
CSE 4701
Distributed Database Concepts A transaction can be executed by multiple networked
computers in a unified manner. A distributed database (DDB) processes a Unit of
execution (a transaction) in a distributed manner. A distributed database (DDB) can be defined as
Collection of multiple logically related database distributed over a computer network
Distributed database management system as a software system that manages a distributed database while making the distribution transparent to the user.
Chaps26.10
CSE 4701
Goals of DDBMS Support User Distribution Across Multiple Sites
Remote Access by Users Regardless of Location Distribution and Replication of Database Content
Provide Location Transparency Users Manipulate their Own Data Non-Local Sites “Appear” Local to Any User
Provide Transaction Control Akin to Centralized Case Transaction Control Hides Distribution CC and Serializability - Must be Extended
Minimize Communications Cost Optimize Use of Network - a Critical Issue Distribute DB Design Supported by Partitioning
(Fragmentation) and Replication
Chaps26.11
CSE 4701
Goals of DDBMS Improve Response Time for DB Access
Use a More Sophisticated Load Control for Transaction Processing
However, Synchronization Across Sites May Introduce Additional Overhead
System Availability Site Independence in the Presence of Site Failure Subset of Database is Always Available Replication can Keep All Data Available, Even
When Multiple Sites Fail Modularity
Incremental Growth with the Addition of Sites Dedicate Sites to Specific Tasks
Chaps26.12
CSE 4701
Advantages of DDBMS There are Four Major Advantages1. Transparency
Distribution/NW Transparency User Doesn’t Know about NW Configuration (Location
Transparency) User can Find Object at any Site (Naming
Transparency) Replication Transparency (see next PPT)
User Doesn’t Know Location of Data Replicas are Transparently Accessible
Fragmentation Transparency Horizontal Fragmentation (Distribute by Row) Vertical Fragmentation (Distribute by Column)
Chaps26.13
CSE 4701
Data Distribution and Replication
Chaps26.14
CSE 4701
Other Advantages of DDBMS
2. Increased Reliability and Availability Reliability - System Always Running Availability - Data Always Present Achieved via Replication and Distribution Ability to Make Single Query for Entire DDBMS
3. Improved Performance Sites Able to Utilize Data that is Local for Majority
of Queries4. Easier Expansion
Improve Performance of Site by Upgrading Processor of Computer Adding Additional Disks Splitting a Site into Two or More Sites
Expansion over Time as Business Grows
Chaps26.15
CSE 4701
Challenges of DDBMS Tracking Data - Meta Data More Complex
Must Track Distribution (where is the Data) V & H Fragmentation (How is Data Split) Replication (Multiple Copies for Consistency)
Distributed Query Processing Optimization, Accessibility, etc., More Complex Block Analysis of Data Size Must also Now
Consider the NW Transmitting Time Distributed Transaction Processing
TP Potentially Spans Multiple Sites Submit Query to Multiple Sites Collect and Collate Results
Distributed Concurrency Control Across Nodes
Chaps26.16
CSE 4701
Challenges of DDBMS Replicated Data Management
TP Must Choose the Replica to Access Updates Must Modify All Replica Copies
Distributed Database Recovery Recovery of Individual Sites Recovery Across DDBMS
Security Local and Remote Authorization During TP, be Able to Verify Remote Privileges
Distributed Directory Management Meta-Data on Database - Local and Remote Must maintain Replicas of this - Every Site Tracks
the Meta-Data for All Sites
Chaps26.17
CSE 4701
A Complete Schema with Keys ...
Keys Allow us to Establish Links Between Relations
Chaps26.18
CSE 4701
…and Corresponding DB Tables
which Represent Tuples/Instances of Each Relation
1455
ASCnullWBnullnull
Chaps26.19
CSE 4701
…with Remaining DB Tables
Chaps26.20
CSE 4701
What is Fragmentation? Fragmentation Divides a DB Across Multiple Sites Two Types of Fragmentation
Horizontal Fragmentation Given a Relation R with n Total Tuples, Spread Entire
Tuples Across Multiple Sites Each Site has a Subset of the n Tuples Essentially Fragmentation is a Selection
Vertical Fragmentation Given a Relation R with m Attributes and n Total
Tuples, Spread the Columns Across Multiple Sites Essentially Fragmentation is a Projection Not Generally Utilized in Practice
In Both Cases, Sites can Overlap for Replication
Chaps26.21
CSE 4701
Horizontal Fragmentation A horizontal subset of a relation which contain those
of tuples which satisfy selection conditions. Consider Employee relation with condition DNO = 5 All tuples satisfying this create a subset which will be
a horizontal fragment of Employee relation. A selection condition may be composed of several
conditions connected by AND or OR. Derived horizontal fragmentation:
Partitioning of a primary relation to other secondary relations which are related with Foreign keys.
Chaps26.22
CSE 4701
Horizontal Fragmentation Site 2 Tracks All Information Related to Dept. 5
Chaps26.23
CSE 4701
Horizontal Fragmentation Site 3 Tracks All Information Related to Dept. 4 Note that an Employee Could be Listed in Both Cases,
if s/he Works on a Project for Both Departments
Chaps26.24
CSE 4701
Refined Horizontal Fragmentation Further Fragment from Site
2 based on Dept. that Employee Works in
Notice that G1 + G2 + G3 is the Same as WORKS_ON5
there is no Overlap
Chaps26.25
CSE 4701
Refined Horizontal Fragmentation Further Fragment from Site
3 based on Dept. that Employee Works in
Notice that G4 + G5 + G6 is the Same as WORKS_ON4
Note Some Fragments can be Empty
Chaps26.26
CSE 4701
Vertical Fragmentation Subset of a relation created via a subset of columns.
A vertical fragment of a relation will contain values of selected columns.
There is no selection condition used in vertical fragmentation.
A strict vertical slice/partition Consider the Employee relation.
A vertical fragment of can be created by keeping the values of Name, Bdate, Sex, and Address.
Since no condition for creating a vertical fragment Each fragment must include the primary key
attribute of the parent relation Employee. All vertical fragments of a relation are connected.
Chaps26.27
CSE 4701
Vertical Fragmentation Example Partition the Employee Table as Below Notice Each Vertical Fragment Needs Key Column
EmpDemo EmpSupvrDept
Chaps26.28
CSE 4701
Homogeneous DDBMS Homogeneous
Identical Software (w.r.t. Database) One DB Product (e.g., Oracle, DB2, Sybase) is
Distributed and Available at All Sites Uniformity w.r.t. Administration, Maintenance,
Client Access, Users, Security, etc. Interaction by Programmatic Clients is Consistent
(e.g., JDBC or ODBC or REST API …)
Chaps26.29
CSE 4701
Non-Federated Heterogeneous DDBMS Non-Federated Heterogeneous
Different Software (w.r.t. Database) Multiple DB Products (e.g., Oracle at One Site,
MySQL at another, Sybase, Informix, etc.) Replicated Administration (e.g., Users Needs
Accounts on Multiple Systems) Varied Programmatic Access - SWEs Must Know
All Platforms/Client Software Complicated Very Close to Shared Nothing Architecture
Chaps26.30
CSE 4701
Federated DDBMS Federated
Multiple DBMS Platforms Overlaid with a Global Schema View
Single External Schema Combines Schemas from all Sites
Multiple Data Models Relational in one
Component DBS Object-Oriented in
another DBS Hierarchical in a
3rd DBS
Chaps26.31
CSE 4701
Federated DBMS Issues Differences in Data Models
Reconcile Relational vs. Object-Oriented Models Each Different Model has Different Capabilities These Differences Must be Addressed in Order to
Present a Federated Schema Differences in Constraints
Referential Integrity Constraints in Different DBSs Different Constraints on “Similar” Data Federated Schema Must Deal with these Conflicts
Differences in Query Languages SQL-89, SQL-92, SQL2, SQL3 Specific Types in Different DBMS (Oracle Blobs )
Differences in Key Processing & Timestamping
Chaps26.32
CSE 4701
Heterogeneous Distributed Database Systems Federated: Each site may run different database system but the
data access is managed through a single conceptual schema. The degree of local autonomy is minimum. Each site must adhere to a centralized access policy There may be a global schema.
Multi-database: There is no one conceptual global schema For data access a schema is constructed dynamically as
needed by the application software.
Communications network
Site 5Site 1
Site 2Site 3
NetworkDBMS
Relational
Site 4
ObjectOriented
LinuxLinux
Unix
Hierarchical
ObjectOriented
RelationalUnix
Window
Chaps26.33
CSE 4701
Query Processing in Distributed Databases
Issues Cost of transferring data (files and results) over the network.
This cost is usually high so some optimization is necessary. Example relations: Employee at site 1 and Department at Site 2
– Employee at site 1. 10,000 rows. Row size = 100 bytes. Table size = 106 bytes.
– Department at Site 2. 100 rows. Row size = 35 bytes. Table size = 3,500 bytes.
Q: For each employee, retrieve employee name and department name Where the employee works.
Q: Fname,Lname,Dname (Employee Dno = Dnumber Department)
Fname Minit Lname SSN Bdate Address Sex Salary Superssn Dno
Dname Dnumber Mgrssn Mgrstartdate
Chaps26.34
CSE 4701
Query Processing in Distributed Databases Result
The result of this query will have 10,000 tuples, assuming that every employee is related to a department.
Suppose each result tuple is 40 bytes long. The query is submitted at site 3 and the result is
sent to this site. Problem: Employee and Department relations are
not present at site 3.
Chaps26.35
CSE 4701
Query Processing in Distributed Databases Strategies:
1. Transfer Employee and Department to site 3. Total transfer bytes = 1,000,000 + 3500 = 1,003,500
bytes.2. Transfer Employee to site 2, execute join at site 2 and send
the result to site 3. Query result size = 40 * 10,000 = 400,000 bytes. Total
transfer size = 400,000 + 1,000,000 = 1,400,000 bytes.3. Transfer Department relation to site 1, execute the join at site
1, and send the result to site 3. Total bytes transferred = 400,000 + 3500 = 403,500 bytes.
Optimization criteria: minimizing data transfer. Preferred approach: strategy 3.
Chaps26.36
CSE 4701
Query Processing in Distributed Databases Consider the query
Q’: For each department, retrieve the department name and the name of the department manager
Relational Algebra expression: Fname,Lname,Dname (Employee Mgrssn = SSN Department)
Chaps26.37
CSE 4701
Query Processing in Distributed Databases Result of query has 100 tuples, assuming that every
department has a manager, the execution strategies are:1. Transfer Employee and Department to the result site and
perform the join at site 3. Total bytes transferred = 1,000,000 + 3500 = 1,003,500
bytes.2. Transfer Employee to site 2, execute join at site 2 and send
the result to site 3. Query result size = 40 * 100 = 4000 bytes. Total transfer size = 4000 + 1,000,000 = 1,004,000 bytes.
3. Transfer Department relation to site 1, execute join at site 1 and send the result to site 3. Total transfer size = 4000 + 3500 = 7500 bytes.
Preferred strategy: Choose strategy 3.
Chaps26.38
CSE 4701
Query Processing in Distributed Databases Now suppose the result site is 2. Possible strategies :
1. Transfer Employee relation to site 2, execute the query and present the result to the user at site 2. Total transfer size = 1,000,000 bytes for both queries Q
and Q’.
2. Transfer Department relation to site 1, execute join at site 1 and send the result back to site 2. Total transfer size for Q = 400,000 + 3500 = 403,500
bytes and for Q’ = 4000 + 3500 = 7500 bytes.
Chaps26.39
CSE 4701
DDBS Concurrency Control and Recovery Distributed Databases encounter a number of
concurrency control and recovery problems which are not present in centralized databases, including: Dealing with multiple copies of data items
How are they All Updated if Needed? Failure of individual sites
How are Queries Restarted or Rerouted? Communication link failure
Network Failure Distributed commit
How to Know All Updates Done at all Sites? Distributed deadlock
How to Detect and Recover?
Chaps26.40
CSE 4701
The Next Big Challenge Interoperability
Heterogeneous Distributed Databases Heterogeneous Distributed Systems Autonomous Applications
Scalability Rapid and Continuous Growth
Amount of Data Variety of Data Types Dealing with personally identifiable information (PII)
and personal health information (PHI) Emergence of Fitness and Health Monitoring Apps Google Fit and Apple HealthKit New Apple ResearchKit for Medical Research
Chaps26.41
CSE 4701
Interoperability: A Classic View
FDB Global Schema
FederatedIntegration
Local Schema
Local Schema
Local Schema
FDB Global Schema 4
FederatedIntegration
FDB 1Local
SchemaFDB3
Federation Federation
Simple Federation Multiple Nested Federation
Chaps26.42
CSE 4701
LegacyApplication
Network
Java Client
Java Application Code
WRAPPER
Mapping Classes
JAVA LAYER
NATIVE LAYER
Native Functions (C++)RPC Client Stubs (C)
Interactions Between Java Clientand Legacy Appl. via C and RPC
C is the Medium of Info. Exchange
Java Client with C++/C Wrapper
Java Client with Wrapper to Legacy Application
Chaps26.43
CSE 4701
Network
Java Application Code
JAVA NETWORK WRAPPER
Mapping Classes
NATIVE LAYER
JAVA LAYER
Native Functions that Map to COTS Appl
Java Client Java Client
Java Application Code
JAVA NETWORK WRAPPER
Mapping Classes
NATIVE LAYER
JAVA LAYER
Native Functions that Map to Legacy Appl
COTS Application Legacy Application
Java is Medium of Info. Exchange - C/C++ Appls with Java Wrappers
COTS and Legacy Appls. to Java Clients
Chaps26.44
CSE 4701
Java Client
LegacyApplication
Relational Database
System(RDS)
Transformed Legacy Data
Updated Data
Extract and Generate Data
Transform andStore Data
Java Client to Legacy App via RDBS
Chaps26.45
CSE 4701
Information Broker
•Mediator-Based Systems•Agent-Based Systems
Database Interoperability in the Internet Technology
Web/HTTP, JDBC/ODBC, CORBA (ORBs + IIOP), XML, SOAP, REST API, WSDL
Architecture
Chaps26.46
CSE 4701
Driver Driver
Java Application
JDBC API
Driver Manager
Oracle SybaseAccess
Driver
JDBC
Driver Driver
JDBC API Provides DB Access Protocols for Open, Query, Close, etc.
Different Drivers for Different DB Platforms
Chaps26.47
CSE 4701
Connecting a DB to the Web
Web Server are Stateless
DB Interactions Tend to be Stateful
Invoking a CGI Script on Each DB Interaction is Very Expensive, Mainly Due to the Cost of DB Open
DBMS
Web Server
Browser
Internet
CGI Script Invocationor JDBC Invocation
Chaps26.48
CSE 4701
Connecting More Efficiently
To Avoid Cost of Opening Database, One can Use Helper Processes that Always Keep Database Open and Outlive Web Connection
Newly Invoked CGI Scripts Connect to a Preexisting Helper Process
System is Still Stateless
DBMS
Web Server
Browser
Internet
CGI Scriptor JDBC Invocation
Helper Processes
Chaps26.49
CSE 4701
DB-Internet Architecture
WWW Client(Netscape)
WWW Client(HotJava)
WWW client(Info. Explore)
Internet
HTTP Server
DBWeb Gateway
DBWeb Gateway
DBWeb Gateway
DBWeb Gateway
DBWeb Dispatcher
Chaps26.50
CSE 4701
EJB Architecture
Chaps26.51
CSE 4701
The focus is to make information available to users, in the right form, at the right time, in the appropriate place.
Technology Push Computer/Communication Technology (Almost Free)
Plenty of Affordable CPU, Memory, Disk, Network Bandwidth
Next Generation Internet: Gigabit Now Wireless: Ubiquitous, High Bandwidth
Information Growth Massively Parallel Generation of Information on
the Internet and from New Generation of Sensors Disk Capacity on the Order of Peta-bytes
Small, Handy Devices to Access Information
Chaps26.52
CSE 4701 Ubiquitous/Pervasive
Many computers and information appliances everywhere,
networked together
Research Challenges Inherent Complexity:
Coping with Latency (Sometimes Unpredictable)
Failure Detection and Recovery (Partial Failure)
Concurrency, Load Balancing, Availability, Scale
Service Partitioning Ordering of Distributed Events
“Accidental” Complexity: Heterogeneity: Beyond the Local
Case: Platform, Protocol, Plus All Local Heterogeneity in Spades.
Autonomy: Change and Evolve Autonomously
Tool Deficiencies: Language Support (Sockets,rpc), Debugging, Etc.
Chaps26.53
CSE 4701
Problem: too many sources,too much information
Internet:Information Jungle
Clean, Reliable,Timely Information,Anywhere
DigitalEarth
Sensors
PersonalizedFiltering &Info. Delivery
Infopipes
Resou
rce A
dapta
tion Property Mgmt
Information QualityContinual Queries
Mic
rofe
edba
ck
specializationInfosphere
Chaps26.54
CSE 4701
ThinClient
WebServer
MainframeDatabaseServer
Current State-of-Art – Has Mobile Changed This?
Chaps26.55
CSE 4701 Infotaps &
Fat Clients
Varietyof Servers
Sensors
DatabaseServer
Many sources
Infosphere Scenario – Where Does Mobile Fit?
Chaps26.56
CSE 4701
Heterogeneity and Autonomy Heterogeneity:
How Much can we Really Integrate? Syntactic Integration
Different Formats and Models XML/JSON/RDF/OWL/SQL Query Languages
Semantic Interoperability Basic Research on Ontology, Etc. DoD Maps (Grid, True, and Magnetic North)
Autonomy No Central DBA on the Net Independent Evolution of Schema and Content Interoperation is Voluntary Interface Technology DCOM: Microsoft Standard
CORBA, Etc...
Chaps26.57
CSE 4701
Security and Data Quality Security
System Security in the Broad Sense Attacks: Penetrations, Denial of Service System (and Information) Survivability
Security Fault Tolerance Replication for Performance, Availability, and
Survivability Data Quality
Web Data Quality Problems Local Updates with Global Effects Unchecked Redundancy (Mutual Copying) Registration of Unchecked Information Spam on the Rise
Chaps26.58
CSE 4701
Legacy Data Challenge Legacy Applications and Data
Definition: Important and Difficult to Replace Typically, Mainframe Mission Critical Code Most are OLTP and Database Applications
Evolution of Legacy Databases Client-server Architectures Wrappers Expensive and Gradual in Any Case
Chaps26.59
CSE 4701
Potential Value Added/Jumping on Bandwagon
Sophisticated Query Capability Combining SQL with Keyword Queries
Consistent Updates Atomic Transactions and Beyond
But Everything has to be in a Database! Only If we Stick with Classic DB Assumptions
Relaxing DB Assumptions Interoperable Query Processing Extended Transaction Updates
Commodities DB Software A Little Help is Still Good If it is Cheap Internet Facilitates Software Distribution Databases as Middleware
Chaps26.60
CSE 4701
Concluding Remarks Goals of Distributed DBS
Support User Distribution Across Multiple Sites Provide Location Transparency Provide Transaction Control Akin to Centralized
Case Minimize Communications Cost
Advantages of Distributed DBS Transparency Increased Reliability and Availability Improved Performance Easier Expansion