Data in the cloud - Building highly scalable database ( Azure Storage Vs SQL Data Services ) Saranya Sriram

Developer EvangelistMicrosoft Corporation India

Azure™ Services Platform

Microsoft’s Cloud – ReCap

AgendaData is KingMotivation ? Why data outside your premise?Microsoft’s Data Storage offerings

Windows Azure SQL Services

Programming & Coding for data on cloud OS

What we will not learn?Deep Dive into SQL ServicesAny SQL Services 2008 specificsArchitectural considerations to model data in real life

Azure™ Services Platform

Windows Azure Storage and SQL Data Services are different storage offerings for the Azure Services Platform

Today’s discussion – next 60 minutes

Windows Azure Storage Service

…

Fabric

Compute Storage

Application

Blobs Queues

HTTP

Tables

Windows Azure StorageStorage that is

DurableScalable (capacity and throughput)Highly AvailableSecurityPerformance EfficientPay for what you use

Rich Data AbstractionsService communication: queues, locks, …Large user data items: blobs, blocks, …Service state: tables, caches, …

Simple and Familiar Programming InterfacesREST Accessible and ADO.NET

Windows Azure Storage Account

Account

Blob Table Queue

User creates a globally unique storage accountReceived a 256 bit secret key when creating accountGeo-location & Co-location coming soon

Provides security for accessing the store Use secret key to create HMACSHA256 signature for each requestUse signature to authenticate request at server

Blobs Storage• Storage Account• An account can have many Containers

• Container• A container is a set of blobs• Sharing policies are set at the container level

• Public READ or Private• Associate Metadata with Container

• Metadata is <name, value> pairs

• Up to 8KB per container• List the blobs in a container

BlobContainerAccount

sally

pictures

IMG001.JPG

IMG002.JPG

movies MOV1.AVI

Slide Share for Tech Ed India 2009 -Basic

demo

Table StorageTable

A storage account can create many tablesTable name is scoped by account

Data is stored in tablesA table is a set of entities (rows)An entity is a set of properties (columns)

EntityTwo “key” properties that together

are the unique ID of the entity in the table

PartitionKey – enables scalabilityRowKey – uniquely identifies the

entity within the partition

EntityTableAccount

sally

users

photo index

Table . . .Table Table

Windows Azure StorageA closer look at tables

Entity . . .Entity Entity

Property PropertyProperty

Name Type Value

Storage Accounts

Tables - CapabilitiesWhat tables don’t do

Not relationalNo Referential Integrity

No JoinsLimited Queries

No Group byNo AggregationsNo TransactionsNo Transactions

What tables can do

CheapVery Scalable

FlexibleDurable

If these are important to you, use:

What tables contain

Partition key & Row key

Timestamp255 properties for your data & data

of size 1MB

Query a Table

LINQDataServiceContext context = new DataServiceContext(“http://myaccount.table.core.windows.net”);

var customers = from o in context.CreateQuery<Customer>(“Customer”)

where o.PartitionKey == “Lee”select o;

foreach (Customer customer in customers) { }

GET http://myaccount.table.core.windows.net/Customer? $filter= PartitionKey eq ‘Lee’

REST

Slide Share for Tech Ed India 2009 -Revisited

demo

Queue Storage

An account can create many queuesQueue Name is scoped by the account

A Queue contains messagesNo limit on number of messages stored in a queueA message is stored for at most a week in a queue

http://<Account>.queue.core.windows.net/<QueueName>Messages

Message size <= 8 KBTo store larger data, store data in blob/entity storage, and the blob/entity name in the message

2 1

C1

C2

Dequeue and Delete Messages

1234

Producers Consumers

P2

P1

3

2. Dequeue(Q, 30 sec) msg 2

1. Dequeue(Q, 30 sec) msg 1

12

C1

C2

Dequeue and Delete Messages

34

Producers Consumers

P2

P1

1

2

2. Dequeue(Q, 30 sec) msg 23. C2 consumed msg 24. Delete(Q, msg 2)7. Dequeue(Q, 30 sec) msg 1

1. Dequeue(Q, 30 sec) msg 15. C1 crashed

12 1 6. msg1 visible 30 seconds after Dequeue3

BenefitInsures that every message can be processed at least once

Slide Share for Tech Ed India 2009 -Revisited

demo

Account

Container Blobs

Table Entities

Queue Messages

Windows Azure Data Storage Concepts

http://<account>.blob.core.windows.net/<container>

http://<account>.table.core.windows.net/<table>

http://<account>.queue.core.windows.net/<queue>

Slide Share for Tech Ed India 2009 -Upload onto the Azure Storage & take REST

demo

Enables developers to access storageMassively scalable, durable, and availableAnywhere at anytime accessAutomatically scale to meet peek traffic demandsOnly pay for what the service uses

Easy to use REST and .NET InterfacesBlobs, tables, and queues

TakeawaysWindows Azure Storage

SQL ServicesSQL Services

.NET Services

Windows AzureLive Services

Applications

Applications

OthersWindowsMobile

WindowsVista/XP

WindowsServer

SQL Data Services

Others (Future)

Place

Type

Service

Data Platform: Device to Cloud

Mobile/Desktop

AnalysisCaching

Integration

Sync

Search

Data ServicesServerDesktop

RelationalBLOB

QueryBackup

Load

MultiDim

Mega-server

InMemory FileXML

ReportingMiningProtection

RDBMS

Model & policy based development and management

Extending SQL Data Platform to Cloud

Initial services – core RDBMS capabilities as a service (SDS), Data Sync and Data HubFuture Offerings

Additional data platform capabilities as a service: BI/DSS, DWNew services: Reference Data, Secure Data Hub

SQL Data Services (SDS)Database-as-a-Service

Data SyncReference DataReporting Business

Intelligence

Symmetric Programming Model Data Hub Aggregation

Enable new uses of data to deliver differentiated business value

The New SQL Data Services

Familiar SQL Server relational modelUses existing APIs & toolsBuilt for the Cloud with availability and scaleAccessible to all from PHP, Ruby, and Java

Focus on combining the best features of SQL Server running at scale with low friction

Accessing SDS

User Database connStringBuilder.InitialCatalog = "mydatabase";

// Specify your user database to connect

Master/Server DatabaseconnStringBuilder.DataSource = "myserver.data.dev.mscds.com“;

// Specify the DNS name of my SDS server master db)connStringBuilder.UserID = "jeff@myserver";

// Specify my user id

SQL Client (ADO.net) code snippetJust change the connection string values:static void Main(string[] args) {

SqlConnectionStringBuilder connStringBuilder = new SqlConnectionStringBuilder(); connStringBuilder.DataSource = "myserver.data.dev.mscds.comconnStringBuilder.InitialCatalog = "mydatabase"; connStringBuilder.Encrypt = true; connStringBuilder.TrustServerCertificate = true; connStringBuilder.UserID = "jeff@myserver"; connStringBuilder.Password = "****";

string createTableSql = @"CREATE TABLE [dbo].[tbl_Person] ( [FirstName] NVARCHAR(64) NOT NULL, [LastName] NVARCHAR(64) NOT NULL CONSTRAINT [personName_PK] PRIMARY KEY CLUSTERED ( [FirstName] ASC, [LastName] ASC ) )";using (SqlConnection conn = new SqlConnection(connStringBuilder.ToString())) {

conn.Open(); using (SqlCommand cmd = conn.CreateCommand()) {

cmd.CommandText = createTableSql; cmd.CommandType = CommandType.Text; cmd.ExecuteNonQuery(); conn.Close();

} }}

Example of SQL Compatibility

In Scope for v1 (likely)Tables, indexes and viewsStored ProceduresTriggersConstraintsTable variables, session temp tables (#t)…

Out of Scope for v1 (likely)Distributed TransactionsDistributed QueryCLRService BrokerSpatialPhysical server or catalog DDL and views

Foundation of the SDS relational platformBuilt using core SQL Server product codebase

Delivery of rich relational database service – PDC ’09Key capability of the Azure platformSupports broad spectrum of scenarios

TakeawaysSQL Data Services

Azure Services Platform Roadmap

First CTPDeveloper CTP for

Services SDKs, and Tools

Fall 2008 Spring 2009

Updated CTPsEnable Full trust & Fast CGI

Geo-location support.NET Services Open CTP

Live Framework Open CTPSDS Invitation-only CTP

Fall 2009

Commercial Availability

Windows Azure.NET ServicesSQL Services

Summer 2009

Pricing and SLA confirmationSDS Public CTP

Getting Started & Resources

1. Download the SDKs and tools from http://www.microsoft.com/azure

2. Get invitation for Windows Azure and start coding3. Review the hands-on labs and videos available on the

site3. SSDS blog: http://blogs.msdn.com/ssds 4. Azure Services Training Kit

Downloadable hands-on labs, demos, and presentations http://www.azure.com

5. Check Mix09 and PDC 08 sessions

question & answer

Related Content

Breakout Sessions (session codes and titles)

1. Cloud Track – Introduction to cloud computing and Windows Azure

2. Cloud Track – .NET services

3. Cloud Track - Code Camp

© 2009 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS,

IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.