Hands-On LabSQL Azure Data Sync

Lab version: 1.0.0

Last updated: 5/24/2023

Page | 1

CONTENTS

OVERVIEW................................................................................................................................................. 3

GETTING STARTED: PROVISIONING DATA SYNC................................................................................5Task 1 – Provisioning Data Sync...........................................................................................................5

EXERCISE 1: CREATING A SYNC GROUP USING SQL AZURE MANAGEMENT PORTAL.................9Task 1 – Creating Sync Group..............................................................................................................9

Task 2 – Adding a SQL Azure Database Sync Hub..............................................................................12

Task 3 – Adding a SQL Azure Database to Sync..................................................................................14

Task 4 – Adding an On-Premises SQL Server Database to Sync.........................................................17

Task 5 –Configuring Sync Group........................................................................................................28

VERIFICATION......................................................................................................................................... 35Task 1 – Synchronizing Data..............................................................................................................35

Task 2 – Conflict Resolution According Precedence Settings.............................................................44

Task 3 – Exploring Data Sync Objects.................................................................................................48

SUMMARY................................................................................................................................................ 50

Page | 2

Overview

The SQL Azure Data Sync is a cloud-based data synchronization service, which provides uni-directional and bi-directional data sync, allowing data to be easily shared between SQL Azure and on-premises SQL Server databases as well as between multiple SQL Azure databases, within the same or different data centers. The SQL Azure Data Sync Previews can be accessed directly from the Windows Azure Management Portal.

Figure 1A functional view of the SQL Azure Data Sync

In this Lab, you will synchronize two databases, a SQL Azure database against an on-premises SQL Server database, changing the sync settings available in a Sync Group.

Objectives

In this hands-on lab, you will learn how to:

Provision a SQL Azure Data Sync Server

Create a Sync Group Topology with a SQL Azure Database, a SQL Azure Sync Hub and an on-premises SQL Server database.

Prerequisites

The following is required to complete this hands-on lab:

Page | 3

Microsoft .NET Framework 4.0

SQL Server Data Tools

Setup

In order to execute the exercises in this hands-on lab you need to set up your environment.

1. Open a Windows Explorer window and browse to the lab’s Source folder.

2. Open the configuration.xml file and enter your SQL Azure database, user and password. Enter the server name for the On-Premises SQL Server. You can leave the database name.

3. Double-click the Setup.cmd file in this folder to launch the setup process that will configure your environment and install the Visual Studio code snippets for this lab.

4. If the User Account Control dialog is shown, confirm the action to proceed.

Note: Make sure you have checked all the dependencies for this lab before running the setup.

Exercises

This hands-on lab includes the following exercise:

1. Provisioning Data Sync

2. Creating a Sync Group Using SQL Azure Management Portal

Estimated time to complete this lab: 45 minutes.

Page | 4

Getting Started: Provisioning Data Sync

In this exercise, you will provision a SQL Azure Data Sync server, as this is required to host your sync groups. You may provision only one Data Sync server per Windows Azure subscription. However, a single Data Sync server can host multiple sync groups.

Task 1 – Provisioning Data Sync

In this task, you will use the SQL Azure Management Portal to provision your SQL Azure Data Sync Server.

1. Open Internet Explorer and navigate to the Windows Azure Management Portal at https://windows.azure.com.

2. Log in to your Windows Live account.

Figure 2Logging into the Azure Management Portal

3. At the Windows Azure Management Portal home page, click Data Sync.

Page | 5

Figure 3Provisioning Data Sync

4. Click Provision on the ribbon.

Figure 4Provisioning Data Sync Server

5. It will prompt a wizard to provision the Data Sync server. Check the Terms of Use Agreement and click Next.

Page | 6

Figure 5Provision Wizard –Terms of Use

6. Select your subscription under which your Data Sync server will be created from the Subscription list and click Next.

Figure 6Provision Wizard –Select Subscription

Page | 7

7. Select a region to setup your Data Sync Server from the Region list and click Finish.

Figure 7Provision Wizard – Select Region

8. On the left pane, you will see a folder structure like the following.

Figure 8Data Sync structure

Page | 8

Exercise 1: Creating a Sync Group Using SQL Azure Management Portal

In this exercise, you will create a Sync Group and define a Topology with the databases you want to synchronize. You will also define some sync settings, such as Conflicts Resolutions, Sync Dataset structure and synchronization schedule.

Note: Make sure you added the necessary Firewall Rules to allow you access to the SQL Azure Server using SSDT. Also, add the Microsoft Services rule (IP Address range “0.0.0.0” to “0.0.0.0”).

Task 1 – Creating Sync Group

In this task, you will add a Sync Group to your Data Sync Server.

1. If not already opened, open Internet Explorer and navigate to the Windows Azure Management Portal at https://windows.azure.com.

2. Log in to your Windows Live account.

Figure 9Logging into the Azure Management Portal

Page | 9

3. At the Windows Azure Management Portal home page, click Data Sync.

Figure 10Managing Data Sync

4. Click Create on the ribbon, within Sync Group section.

Figure 11Creating a Sync Group

5. It will prompt a Create Sync Group dialog. Select your subscription from the Subscription list and the Data Sync Server you created in the Getting Started section. Click Finish to complete the Sync Group creation.

Page | 10

Figure 12Create Sync Group

6. Now, you will need to define the Sync Group Name and its Topology. In the Management Portal, locate the Sync Group Name field and set its value to DataSyncGroup. You will define the Topology in the next task.

Figure 13Setting Sync Group Name

Page | 11

Task 2 – Adding a SQL Azure Database Sync Hub

In this task, we will add a SQL Azure database as the Sync Hub in the Data Sync Group we created in the previous task.

1. In the same page you defined the Sync Group Name you will define the Sync Group Topology. SQL Azure Data Sync Service supports a hub-spoke topology. The hub database must be a SQL Azure database and the members can be any combination of SQL Azure and SQL Server databases. To add a SQL Azure database as the Sync Hub, click the Add a SQL Azure database as the Sync Hub icon within the Cloud side in the Topology section.

Figure 14Adding SQL Azure database as the Sync Hub

Note: SQL Azure Data Sync uses a hub/spoke topology. The hub is the central database in a Sync Group and must be a SQL Azure database. It is best if the hub is either (a) in the same data center (when databases are not geographically disbursed or, (b) in the location most central to the other databases. The two most significant factors in selecting a hub database is latency and cost. Choosing a hub database that is close to the majority of your data traffic minimizes both latency and cost.

Once the hub is set, it cannot be changed to a different database or removed. Data changes in member databases are written to the hub database or discarded in accordance with the Conflict Resolution policy. After all the member databases have sent their changes to the hub, the hub then write changes to all the member databases.

Data Sync Service topology supports the following scenarios: sync between the Enterprise and the Cloud, and sync between the Cloud and the Cloud. Once the hub database is added to a

Page | 12

sync group, other databases can be added to or removed from the sync group at any time, as business needs dictate

2. Complete the Add Database to Sync Group dialog with your SQL Azure Server Name (i.e. yourservername.database.windows.net). Set the Database Name field to DataSyncHub (we created this database in the Setup) and complete the Credentials fields with your User ID and Password. Click Test to make sure you can access to the server. If the Connection Successful message appears, click Add.

Figure 15Adding SQL Azure Database

3. You will see the SQL Azure Server added as a Hub in your Sync Group Topology.

Page | 13

Figure 16Data Sync Topology

Task 3 – Adding a SQL Azure Database to Sync

In this task, we will add a second SQL Azure Server to synchronize with your Data Sync Hub.

1. Now that you have added the SQL Azure server as the sync hub to your Data Sync Group, you will add a second SQL Azure Server to synchronize with the Data Sync Hub. To do that, click the Add a SQL Azure database icon within the Cloud side in the Topology section.

Page | 14

Figure 17Adding SQL Azure Database

2. Complete the Add Database to Sync Group dialog with your SQL Azure Server Name (i.e. yourservername.database.windows.net). Set the Database Name field to DataSyncHOL (we created this database in the Setup) and complete the Credentials fields with your User ID and Password. Leave the Sync Direction with the default Bi-Directional value. Click Test to make sure you can access to the server. If the Connection Successful message appears, click Add.

Page | 15

Figure 18Adding SQL Azure Database

Note: When you add a database to a Sync Group you can specify the Sync Direction. There are three sync directions supported:

Bi-Directional: allows the changes to be synced between the selected database and the hub database bi-directionally.

Sync from the Hub: allows the changes to be synced from the hub database to the selected database in one direction only.

Sync to the Hub: allows the changes to be synced from the selected database to the hub database in one direction only.

3. You will see the SQL Azure Server added in your Sync Group Topology.

Page | 16

Figure 19Adding SQL Azure Database

Note: As you can see in the picture, you can add many SQL Azure Servers to synchronize with the SQL Azure Sync Hub. In the following task, you will see that you can also add many On-Premises SQL Servers.

Task 4 – Adding an On-Premises SQL Server Database to Sync

In this task, we will add an On-Premises SQL Server to the Data Sync Group.

1. Now that you have added the second SQL Azure Database to your data sync group, you will add an On-Premises SQL Server to synchronize with the SQL Azure Sync Hub. To do that, click the Add a SQL Server database icon within the On-Premises side in the Topology section.

Page | 17

Figure 20Adding On-Premises SQL Server

2. It will prompt a wizard to add your SQL Server database to the sync group. To do this, choose the Add a new SQL Server Database to the sync group as Database and leave the Sync Direction with the default Bi-Directional value. Click Next.

Page | 18

Figure 21Adding SQL Server Database

3. Each SQL Server database must be registered through a Client Sync Agent before it can be added to a sync group, to do this we first need to install the agent locally. Select Install a new Agent and click Next.

Page | 19

Figure 22Adding SQL Server Database

4. Now you will install the Client Sync Agent. The wizard dialog will show three steps to configure it. First, you need to download and install the Sync Agent locally. Click Download button, an Internet Explorer window will be prompted with the download page. Download and Install the Data Sync Agent.

Page | 20

Figure 23Installing Sync Agent

5. Once the Data Sync Agent is installed, follow the Step 2 and complete the Agent Name. Set the value to DataSyncOnPremises. Finally, in the Step 3, click Generate Agent Key and click Copy to save the generated value in your clipboard. Click Next.

Page | 21

Figure 24Installing Sync Agent

6. Open the SQL Azure Data Sync Agent from Start | All Programs | Microsoft SQL Azure Data Sync and click Submit Agent Key Configuration.

Page | 22

Figure 25Configuring SQL Azure Data Sync Preview

7. Complete the Agent Key field with the value you copied in the Step 5 and click OK.

Figure 26Configuring SQL Azure Data Sync Preview

Page | 23

8. In the SQL azure Data Sync agent’s window, click Register in the ribbon.

Figure 27Registering SQL Server

9. In the prompted dialog, set the SQL Server configuration with your credentials. Complete with your Server Name instance and the Database value with DataSyncHol. Use Windows Authentication. Click Save.

Page | 24

Figure 28Configuring SQL Server Connection

10. In the SQL Azure Data Sync agent’s window, you will see the recently added SQL Server in the Databases list.

Page | 25

Figure 29 Adding SQL Server Database to SQL Azure Data Sync

11. Now that you added the SQL Server to SQL Azure Data Sync agent, switch back to the portal in order to complete the wizard. Click Get Database List to obtain the currently registered database and choose DataSyncHOL from the Select Database list. Click Finish.

Page | 26

Figure 30Adding SQL Server to Sync Group

12. You will see the SQL Server added in your Sync Group Topology. In the next task, you will configure this Sync Group schedule and conflict resolution policy.

Page | 27

Figure 31Data Sync Group Topology

Task 5 –Configuring Sync Group

In this task, you will configure the Sync Group setting the sync frequency and the resolution policy to apply when conflicts occur (i.e. changes are made to the same field in multiple databases). You will also define the Sync Dataset.

1. Configure the Sync Schedule. In the portal, locate the Configuration section and check Enable. Set the Every value to 5 minutes.

Page | 28

Figure 32Data Sync Group – Sync Schedule Configuration

Note: The schedule defines the interval at which scheduled synchronizations are run. When synchronization is run it is only among the databases in the associated Sync Group.

2. Select the Conflict Resolution policy to determine what data is kept and what data is lost whenever the same record is changed in different databases. We will use Hub Wins.

Page | 29

Figure 33Data Sync Group – Conflict Resolution Settings

Note: There are two Conflict Resolution policies:

Hub wins: If there is a change in the hub’s record it will always be kept and a data change in the member is lost. The first change written to the hub is maintained throughout the synchronization.

Client wins: If there is a change in the member’s record it is written to the hub. If the record is changed in multiple members the last member’s change is written to the hub and is propagated out to all members.

Once the Conflict Resolution policy is set for Sync Group it cannot be changed later except by recreating the Sync Group.

3. Finally, you need to define the Sync Data Set by selecting the specific tables, columns, and rows (by filtering on a particular value) to be included for synchronization with this sync group (when a synchronization is performed only the tables, columns, and rows selected are synchronized). Click Edit Dataset.

Page | 30

Figure 34Data Sync Group – Defining Sync Dataset

4. In the prompted dialog, select the DataSyncHub database to import its schema and click Get Latest Schema to make sure it is updated.

Page | 31

Figure 35Defining Dataset Synchronization

5. Select the tables and columns for synchronization. In this case, we will synchronize Products table and all its columns. You can also add row filtering to specify which rows you want to synchronize. Make sure Products table and Id, Description, Price and IsLocal columns are selected and click OK.

Page | 32

Figure 36Defining Dataset Synchronization

6. You will see the Sync Dataset definition specifying the Synced Table and the Synced Columns.

Page | 33

Figure 37Sync Dataset

7. Finally, you need to deploy this sync group to SQL Azure. In the ribbon, click Deploy and wait until the deployment finishes. The sync group status will be updated to Good in order to show that the deployment has correctly finished.

Figure 38Deploying Data Sync Group

Page | 34

Figure 39Deployed Data Sync Group

You have successfully created a Data Sync Group, added two SQL Azure servers and an On-Premises SQL Server. To verify that the synchronization is working, follow the Verification section.

Verification

Now that we created the Sync Group and set the sync configurations, we will do some tests in different scenarios in order to verify that the synchronization is correctly working according the conflict resolution and dataset proposed.

Note: In order to run this verification you will need to execute each task in a time-window of five minutes as we defined in the Sync Schedule interval. If you want, you can update this value to a higher one as we did in Task 5 of Exercise 1.Take into account that the waiting time will increase.

Task 1 – Synchronizing Data

Page | 35

In this task, we will add some rows into the SQL Azure and the On-Premises SQL databases in order to verify if the Data Sync is working and SQL Azure Hub gets those rows and synchronize them between the databases.

1. Open SSDT. To do this, in Start | All Programs, click the Microsoft SQL Server Data Tools shortcut.

2. In the View menu, select SQL Server Object Explorer.

3. In the SQL Server Object Explorer pane, click Add SQL Server. It will prompt a Connect to Server dialog. Fill it with your SQL Azure Server Name and Credentials and click Connect.

Figure 40Connecting to SQL Azure Server

4. Now, we will add the On-Premises SQL Server to the Server Explorer. Click Add SQL Server and set your SQL Server Name. Change the Authentication mode to Windows Authentication and click Connect.

Page | 36

Figure 41Connecting to SQL Server

5. In the SQL Server Object Explorer pane expand the SQL Server node. Locate your SQL Azure Server and expand Databases folder. Locate the DataSyncHOL and DataSyncHub databases you created in the setup.

Page | 37

Figure 42SQL Azure Server

6. Right click in DataSyncHub database and select New Query. It will open a new tab. To make it easier for you identify each of the SQL Servers in the tabs we will add a commented line with the description of the database that is pointing that query. In this case, we will also add a line to list the Products table content and show that it is empty:

T-SQL

--SQL Azure Database – DataSyncHub

SELECT * FROM [Products]

7. Press CTRL + Q to run the query and verify that the table is empty. In the next steps, you will add some rows to the databases, after the synchronization, you will execute again this query and verify that the table is now populated with those rows.

Figure 43Empty Products Table in SQL Azure Hub

8. In the Databases folder, within the SQL Azure Server node, right click DataSyncHOL database and select New Query. Add the following lines in the new tab.

T-SQL

--SQL Azure Database – DataSyncHOL

INSERT INTO [Products] (Id, Description, Price, IsLocal) VALUES (1,'SQL Azure DataSyncHOL Product 1',10.99,1)

Page | 38

SELECT * FROM [Products]

9. Press CTRL + Q to execute the query and insert the product. After the insertion, the query will show Products table’s rows. You will see only the recently added product.

Figure 44Adding Products to SQL Azure Database

10. Finally, you will create a new query for the On-Premises SQL Server. To do this, locate your SQL Server in the Server Explorer. Expand Databases folder and right click DataSyncHOL database. Select New Query and add the following lines.

Page | 39

Figure 45On-Premises SQL Server

T-SQL

--SQL Server Database – DataSyncHOL

INSERT INTO [Products] (Id, Description, Price, IsLocal) VALUES (2,'SQL Server DataSyncHOL Product 2',2.99,1)

SELECT * FROM [Products]

11. Press CTRL + Q to execute the query and insert the product. After the insertion, the query will show the Products table’s rows. If you did complete this verification in the time-window of 5 minutes, you will see only the recently added product. If you did not, you will probably see the previously added product in the SQL Azure database because of the defined sync schedule.

Page | 40

Figure 46Adding Products to On-Premises SQL Server

12. Now that you added the rows, you need to wait a time-window of five to ten minutes in order to let the Sync Group execute the synchronization schedule. To verify that everything was synced correctly, replace the tabs content with the following script to obtain the list of products each one has.

T-SQL

SELECT * FROM [Products]

13. Execute the scripts by pressing CTRL + Q. You should see the same result in the three queries. First, all the databases synchronize their content with the SQL Azure Hub and then, this hub, sends the updates to each database in order to distribute the new rows.

Page | 41

Figure 47 SQL Azure Database - DataSyncHub Rows

Figure 48 SQL Azure Database - DataSyncHOL Rows

Page | 42

Figure 49 SQL Azure Database - DataSyncHOL Rows

After the sync schedule time elapsed, both SQL Azure and On-Premises databases rows were synchronized by the SQL Azure Sync Hub.

Task 2 – Conflict Resolution According Precedence Settings

In this task we will show how the conflicts are solved according the Conflicts Resolution we set during the Sync Group Configuration. To show this, we will update the same row in the SQL Azure Hub and the On-Premises SQL Server and verify which takes more relevance in the synchronization.

1. In the Server Explorer pane expand the SQL Server node. Locate your SQL Azure Server and expand Databases folder. Right click the DataSyncHub database, select New Query.

2. In the opened query tab, add the following lines.

T-SQL

--SQL Azure Database – DataSyncHub

UPDATE [Products] SET Description='SQL Azure Hub update' WHERE Id=2

SELECT * FROM [Products]

3. Press CTRL + Q to execute the query. You will see the description of the Product 2 updated to “SQL Azure Hub update”.

Page | 43

Figure 50Updating product in SQL Azure Hub

4. Now, you will create a new query for the On-Premises SQL Server. To do this, locate your SQL Server in the Server Explorer. Expand Databases folder and right click DataSyncHOL database. Select New Query and add the following lines.

T-SQL

--SQL Server Database – DataSyncHOL

UPDATE [Products] SET Description='On-Premises SQL Server update' WHERE Id=2

INSERT INTO [Products] (Id, Description, Price, IsLocal)VALUES (3,'SQL Server DataSyncHOL Product 3',2.99,1)

SELECT * FROM [Products]

5. Press CTRL + Q to execute the query. You will see the description of the Product 2 updated to “On-Premises SQL Server update” and a third row with “SQL Server DataSyncHOL Product 3” description.

Page | 44

Figure 51Updating and adding products in On-Premises SQL Server

6. Now that you updated the same product in the SQL Azure Hub and the On-Premises SQL Server, you need to wait a time-window of five to ten minutes in order to let the Sync Group execute the synchronization schedule. To verify that the conflict resolution applied in the synchronization was the one we set in the Sync Group Configuration, replace the tabs content with the following query to obtain the list of products each one has.

T-SQL

SELECT * FROM [Products]

7. Execute the scripts by pressing CTRL + Q. You should see the same result in both queries. When the On-Premises SQL Server synchronized the products with the SQL Azure Hub there was a conflict between the row with Id=2 as we updated the same row in both databases. When we defined the Conflict Resolution policy, we set the Hub wins policy (if there is a change in the hub’s record it will always be kept and a data change in the member is lost), so we should find the Product 2’s description with the value set in the SQL Azure Hub query.

Page | 45

Figure 52 SQL Azure Database - DataSyncHub Rows

Figure 53 SQL Server Database - DataSyncHOL Rows

Page | 46

8. Now we will check that the SQL Azure database was also updated with these changes. To do that, in the Databases folder within the SQL Azure Server node, right click DataSyncHOL database and select New Query. Add the following line in the new query tab.

T-SQL

SELECT * FROM [Products]

9. Press CTRL + Q to execute the query. It will show the same results you obtained in the previous step.

Figure 54SQL Azure Database – DataSyncHOL Rows

After the sync schedule time elapsed, the SQL Azure and On-Premises databases rows were synchronized by the SQL Azure Sync Hub. As we updated the same row in different databases, we generated a conflict that was solved by keeping the data from the SQL Azure Hub (Hub wins policy).

Task 3 – Exploring Data Sync Objects

In this task, we will show you some objects Data Sync creates in order to execute the synchronization between the Sync Group members.

Page | 47

1. Locate your SQL Server in the Server Explorer. Expand DataSyncHOL within Databases folder.

2. Expand the Tables folder. You will see the Products table we were using in this lab and many others with DataSync prefix that were created for Data Sync process.

Figure 55Tables

3. Now, expand Stored Procedures within Programmability folder. You will find many store procedures with DataSync prefix that were created for Data Sync process.

Figure 56Stored Procedures

4. Finally, expand Schemas within Security folder. You will find many schemas defined, but one named as DataSync created for Data Sync process.

Page | 48

Figure 57Schemas

Summary

In this Lab, you learned to provision a SQL Azure Data Sync Server and create a new sync group topology, synchronizing data between a SQL Azure and on-premises databases.

Page | 49