SQL Server Replication

Presented by Tarek GhazaliIT Technical SpecialistIT Technical SpecialistMicrosoft SQL Server MVPWeb Development MCPWeb Development MCPLebDev Vice President

© 2006 Tarek Ghazali. All rights reserved.

Replication OverviewReplication Overview

What is Replication?What is Replication?– ““a set of technologies for copying and distributing a set of technologies for copying and distributing

data and database objects from one database to data and database objects from one database to another and then synchronizing between another and then synchronizing between databases to maintain consistency”databases to maintain consistency”

Transactional ReplicationTransactional Replication– Server to Server solutionServer to Server solution

Merge ReplicationMerge Replication– Offline Database Client solutionOffline Database Client solution

Snapshot ReplicationSnapshot Replication– Point in time copy of objects from one database to Point in time copy of objects from one database to

anotheranother

Should I use Should I use replication?replication? Replication is data distributionReplication is data distribution Replication may not always be the best solutionReplication may not always be the best solution Other options areOther options are

– Distributed TransactionsDistributed Transactions Software-MSDTCSoftware-MSDTC

– Backup/RestoreBackup/Restore– Log Shipping (continuous Backup/Restore)Log Shipping (continuous Backup/Restore)– Linked ServersLinked Servers– Distributed TransactionsDistributed Transactions– Triggered SolutionTriggered Solution– DTS/SSISDTS/SSIS– ClusteringClustering– Hardware/Software Data Mirroring Hardware/Software Data Mirroring

Ordered by cost of latency/downtimeOrdered by cost of latency/downtime

Business Problems Business Problems That Require That Require ReplicationReplication Multiple users or locations need copies Multiple users or locations need copies

of the same dataof the same data– Remote users at distant locationsRemote users at distant locations– Mobile disconnected usersMobile disconnected users

Need to improve local performanceNeed to improve local performance– Physically separate data based on usagePhysically separate data based on usage– Distribute database processing across Distribute database processing across

multiple serversmultiple servers– Scale out read-only dataScale out read-only data

Customer ScenariosCustomer Scenarios

Offloading reporting and Data Offloading reporting and Data WarehousingWarehousing

Database scale out for websitesDatabase scale out for websites Empowering mobile usersEmpowering mobile users Enabling Point of Sale applicationsEnabling Point of Sale applications Consolidating and distributing Consolidating and distributing

regional dataregional data Integrating heterogeneous dataIntegrating heterogeneous data

Reporting and Data Reporting and Data WarehousingWarehousing

Reporting Reporting databases are databases are continuously continuously synchronized in near synchronized in near real-time with one or real-time with one or more source OLTP more source OLTP databasesdatabases

Database Scale OutDatabase Scale Out

London

Chicago

Tokyo

Queries scaled out Queries scaled out (often (often geographically) geographically) similar to reporting similar to reporting casescases

Redundancy provides Redundancy provides fault tolerance and fault tolerance and lowers maintenance lowers maintenance downtimedowntime

Online upgrades Online upgrades possiblepossible

Maximize website Maximize website uptime uptime

– (Database upgrades or (Database upgrades or failures shouldn’t bring failures shouldn’t bring down the application down the application system)system)

Mobile UserMobile User(Sales Orders (Sales Orders Scenario)Scenario)

Point of Sale ScenarioPoint of Sale Scenario

•High latency low bandwidth network

•Need low cost software and administration

•Sync at scheduled times

•Upload orders then remove

Regional Replication Regional Replication ScenarioScenario

•Central Central headquarters data headquarters data replicated to 1200 replicated to 1200 storesstores

•Each store also Each store also replicates between replicates between two databases to two databases to provide a backup provide a backup in case one failsin case one fails

•Database Database administrators are administrators are centralizedcentralized

The Publisher-The Publisher-Subscriber MetaphorSubscriber Metaphor

PublisherPublisherPublisherPublisher

Maintains source databases

Makes data available for replication

DistributorDistributorDistributorDistributor

Receives data changes

Stores metadata, history, and receives and stores changes

May forward changes to Subscribers

Holds copy of data

SubscriberSubscriberSubscriberSubscriber

Publications and Publications and ArticlesArticles PublicationsPublications

– One or more articles One or more articles – Basis of subscriptionBasis of subscription– One or more per databaseOne or more per database

ArticlesArticles– All tables or parts of tablesAll tables or parts of tables– ViewsViews– Stored proceduresStored procedures– User-defined functionsUser-defined functions

SubscriptionsSubscriptions

Push subscription

Push subscription

Push subscription

Push subscription

Pull subscriptionPull subscriptionPull subscriptionPull subscription

SubscriberSubscriber

SubscriberSubscriber

Publication

Remote DistributorRemote Distributor

PublisherSubscribers

Distributor

Local DistributorLocal Distributor

SubscribersPublisher/Distributor

SQL ServerSQL ServerReplication Types Replication Types

Microsoft SQL Server supports the Microsoft SQL Server supports the following types of replication:following types of replication:

Snapshot Replication Snapshot Replication Transactional Replication Transactional Replication

/Updatable subscriptions for /Updatable subscriptions for transactional replicationtransactional replication

Merge Replication Merge Replication

What Is Snapshot What Is Snapshot Replication?Replication? Distributes data as it appears at a Distributes data as it appears at a

moment in timemoment in time This type is mostly used when the This type is mostly used when the

amount of data to be replicated is amount of data to be replicated is small and data/DB objects are small and data/DB objects are static or does not change static or does not change frequently. frequently.

What Is Transactional What Is Transactional Replication?Replication? Permits immediate replication of data Permits immediate replication of data

modificationsmodifications– Minimal latency is required Minimal latency is required

Publisher and the subscriber are Publisher and the subscriber are always in synchronization and should always in synchronization and should always be connected. always be connected.

This type is mostly used when This type is mostly used when subscribers always need the latest subscribers always need the latest data for processing. data for processing.

What Is Merge What Is Merge Replication?Replication? Merges the updates between sites Merges the updates between sites

when they are connectedwhen they are connected Multiple Subscribers need to update at Multiple Subscribers need to update at

various timesvarious times Subscribers need to receive data, Subscribers need to receive data,

make changes offline, and later make changes offline, and later synchronize changes with the synchronize changes with the Publisher and other Subscribers. Publisher and other Subscribers.

Each Subscriber requires a different Each Subscriber requires a different partition of datapartition of data

Replication AgentsReplication Agents

SnapshotSnapshotSnapshotSnapshot

Snapshot AgentSnapshot Agent

Distribution AgentDistribution Agent

TransactionalTransactionalTransactionalTransactional

Snapshot AgentSnapshot Agent

Distribution AgentDistribution Agent

Log Reader AgentLog Reader Agent

Queue Reader AgentQueue Reader Agent

MergeMergeMergeMerge

Snapshot AgentSnapshot Agent

Merge AgentMerge Agent

Agents are the processes that are responsible for copying and distributing data between publisher and subscriber. There are different types of agents supporting different types of replication.

Multimedia: Microsoft Multimedia: Microsoft SQL Server ReplicationSQL Server Replication This animation reviews the This animation reviews the

concepts behind SQL Server concepts behind SQL Server replication.replication.– Snapshot replicationSnapshot replication– Transactional replicationTransactional replication– Merge replicationMerge replication

BreakBreak

Replication in Replication in SQL Server 2005SQL Server 2005 Focused on your requestsFocused on your requests

– Monitoring & diagnosticsMonitoring & diagnostics– Oracle PublishingOracle Publishing– TransactionalTransactional

Better high availability storyBetter high availability story Initialization from backupInitialization from backup

– Merge Merge PerformancePerformance ““Logical record” consistencyLogical record” consistency Synchronization over the webSynchronization over the web

– Schema changes made using the following data Schema changes made using the following data definition language (DDL) statements are automatically definition language (DDL) statements are automatically replicated replicated

– Wizard simplicity Wizard simplicity – Great documentation!Great documentation!

Replication Monitoring Replication Monitoring (Goals)(Goals) Independent from SQL Server Independent from SQL Server

Management StudioManagement Studio Provide answers to common Provide answers to common

questionsquestions Why is the system slow?Why is the system slow? How long until it catches up or How long until it catches up or

finishes?finishes? Estimate how long it will take for subscribers to catch upEstimate how long it will take for subscribers to catch up

Where are the potential problems?Where are the potential problems?

Oracle PublishingOracle PublishingTransactional ReplicationTransactional Replication

YukonYukonDistributorDistributor

SubscribersSubscribers

Designed Designed specificallyspecifically for Oracle Publishers for Oracle Publishers– v8+ on any operating system v8+ on any operating system

Administered like SQL Server, from SQL ServerAdministered like SQL Server, from SQL Server– No Oracle side software install necessaryNo Oracle side software install necessary

– Requires minimal knowledge of OracleRequires minimal knowledge of Oracle

Peer to Peer Peer to Peer ReplicationReplication BenefitsBenefits

– High availability & scale out of DB applicationsHigh availability & scale out of DB applications– No single point of failureNo single point of failure– Allows maintenance without taking applications Allows maintenance without taking applications

offlineoffline– Can scale queries across databasesCan scale queries across databases

SetupSetup– Create “Create “Tranasactional Publication with Updateable Tranasactional Publication with Updateable

SubscriptionsSubscriptions” publications and subscriptions from ” publications and subscriptions from each database to the other databaseseach database to the other databases

– Publications must all have the same namePublications must all have the same name

Distribution Distribution AgentAgent

DistDistDBDB

Logreader Logreader AgentAgent

Distribution Distribution AgentAgent

DistDistDBDB

Logreader Logreader AgentAgent

Distribution Distribution AgentAgent

DistDistDBDB

Logreader Logreader AgentAgent

““West”West” ““East”East”

““South”South”

Peer To Peer Topology Peer To Peer Topology Transactional ReplicationTransactional ReplicationPeer To Peer Topology Peer To Peer Topology Transactional ReplicationTransactional Replication

Transactional - Transactional - Initialization using a DB Initialization using a DB BackupBackup BenefitsBenefits

– Restore a publisher DB backup to deliver Restore a publisher DB backup to deliver the initial data to a subscriber (instead of the initial data to a subscriber (instead of snapshot)snapshot)

– Subscriber automatically picks up Subscriber automatically picks up subsequent changes on the publisher that subsequent changes on the publisher that occurred after the backupoccurred after the backup

SetupSetup– Set the “allow initialization with backup” Set the “allow initialization with backup”

publication optionpublication option– Restore a subsequent publisher backup to Restore a subsequent publisher backup to

the subscriber the subscriber

Merge Replication Merge Replication Performance Performance ImprovementsImprovements Differing levels of conflict detection Differing levels of conflict detection

improve performanceimprove performance– Download onlyDownload only

Data is read only and can only be modified on publisher (for Data is read only and can only be modified on publisher (for example, lookup data, salary, etc.)example, lookup data, salary, etc.)

No metadata sent to subscriberNo metadata sent to subscriber– Subscription based filteringSubscription based filtering

Changes occur at only one subscriber (Point of Sale, Package Changes occur at only one subscriber (Point of Sale, Package Delivery)Delivery)

Metadata is minimal and cleaned up at completion of syncMetadata is minimal and cleaned up at completion of sync– Partition based filteringPartition based filtering

Changes occur within a workgroup (CRM, sales management Changes occur within a workgroup (CRM, sales management [SFA])[SFA])

Conflicts can be resolved amongst common peersConflicts can be resolved amongst common peers Metadata is present; however, performance is improved Metadata is present; however, performance is improved

when applying changes to publisherwhen applying changes to publisher

Parameterized Row Parameterized Row Filter Filter Partition ValuesPartition Values

Description Value in Add Filter and

Edit Filter Value in Article Properties

Data in the partitions is overlapping, and the Subscriber can update columns referenced in a parameterized filter.

A row from this table will go to multiple subscriptions

Overlapping

Data in the partitions is overlapping, and the Subscriber cannot update columns referenced in a parameterized filter.

N/A1 Overlapping, disallow out-of-partition data changes

Data in the partitions is not overlapping, and the data is shared between subscriptions. The Subscriber cannot update columns referenced in a parameterized filter.

N/A1 Nonoverlapping, shared between subscriptions

Data in the partitions is not overlapping, and there is a single subscription per partition. The Subscriber cannot update columns referenced in a parameterized filter.2

A row from this table will go to only one subscription

Nonoverlapping, single subscription

1 If the underlying filtering option is set to 0, or 1, or 2, the Add Filter and Edit Filter dialog boxes will display A row from this table will go to multiple subscriptions.2 If you specify this option, there can only be a single subscription for each partition of data in that article. If a second subscription is created in which the filtering criterion of the new subscription resolves to the same partition as the existing subscription, the existing subscription is dropped.

Logical RecordsLogical RecordsMerge ReplicationMerge Replication

Web SynchronizationWeb Synchronization Merge ReplicationMerge Replication

Publisher/DistributorPublisher/Distributor

FirewallFirewall

IISIISSubscriberSubscriber

Replisapi.dllReplisapi.dll

•No VPN or port required•Eases deployment of mobile applications•Off by default•Secure by default• Only authenticated users can send request • Requires HTTPS •Wizard turns off anonymous access

XML over HTTPS Binary over TCP/IP

Mapping SQL Server Mapping SQL Server 2005 Data Types2005 Data Types SQL Server 2005 has SQL Server 2005 has

introduced a number introduced a number of new data types. of new data types. These new data These new data types are mapped to types are mapped to compatible data compatible data types at the types at the Subscriber if push Subscriber if push subscriptions from a subscriptions from a SQL Server 2005 SQL Server 2005 Distributor are used.Distributor are used.

SQL Server 2005 Data Type

SQL Server 2000 Data Type

XMLXML NTEXTNTEXT

CLR User Defined CLR User Defined Types (UDT)Types (UDT) IMAGEIMAGE

VARCHAR(max)VARCHAR(max) TEXTTEXT

NVARCHAR(max)NVARCHAR(max) NTEXTNTEXT

VARBINARY(max)VARBINARY(max) IMAGEIMAGE

Replicating Data to Replicating Data to SQL Server ExpressSQL Server Express Microsoft SQL Server 2005 Express Edition can serve Microsoft SQL Server 2005 Express Edition can serve

as a Subscriber for all types of replication, providing as a Subscriber for all types of replication, providing a convenient way to distribute data to client a convenient way to distribute data to client applications that use this edition of SQL Server. applications that use this edition of SQL Server.

When using SQL Server 2005 Express Edition in a When using SQL Server 2005 Express Edition in a replication topology, keep the following replication topology, keep the following considerations in mind: considerations in mind: – SQL Server 2005 Express Edition cannot serve as a Publisher or SQL Server 2005 Express Edition cannot serve as a Publisher or

Distributor. However, merge replication allows changes to be Distributor. However, merge replication allows changes to be replicated in both directions between a Publisher and replicated in both directions between a Publisher and Subscriber.Subscriber.

– SQL Server 2005 Express Edition does not include SQL Server SQL Server 2005 Express Edition does not include SQL Server Agent, which is typically used to run replication agents. If you Agent, which is typically used to run replication agents. If you use a pull subscription (in which agents run at the Subscriber), use a pull subscription (in which agents run at the Subscriber), you must synchronize the subscription using you must synchronize the subscription using WindowsWindows SynchronizationSynchronization ManagerManager or or RMORMO. .

Contrasting Replication with Contrasting Replication with Other Data Distribution Other Data Distribution MethodsMethods

MethodMethodMethodMethod

ReplicationReplication

Distributed transactionsDistributed transactions

Back up and restoreBack up and restore

AutonomyAutonomyAutonomyAutonomy

VariableVariable

LowLow

HighHigh

Data Transformation (SSIS)Data Transformation (SSIS) Special use onlySpecial use only

LatencyLatencyLatencyLatency

Variable, depending on typeVariable, depending on type

LowLow

HighHigh

Special use onlySpecial use only

BreakBreak

DemoDemo

Resources & QuestionsResources & Questions

Microsoft Resources:Microsoft Resources:– msdn.microsoft.com/sqlservermsdn.microsoft.com/sqlserver//– www.microsoft.com/www.microsoft.com/sqlsql/community/community

Contact me: Contact me: – tghazali@sqlmvp.comtghazali@sqlmvp.com– www.sqlmvp.comwww.sqlmvp.com

Download Presentation :Download Presentation :– www.lebdev.netwww.lebdev.net– www.devconnect.netwww.devconnect.net