storage in the windows azure platform - ericnel

Windows Azure PlatformWindows Azure Platform

Storing data in the cloud

Eric NelsonMicrosoft UK

Email: eric.nelson@microsoft.com Blog: http://geekswithblogs.net/iupdateableTwitter: http://twitter.com/ericnel and http://twitter.com/ukmsdnPodcast: http://bit.ly/msdnpodcast Newsletter: http://msdn.microsoft.com/en-gb/flash

Windows Azure Platform

2

Windows Azure Platform

• Storage in the Cloud• Queues

• Blobs

• Tables• Relational

Agenda

Windows Azure Platform

Windows Azure Platform

STORAGE IN THE CLOUD…

Windows Azure Storage and SQL Azure

Windows Azure PlatformSolve using the Cloud aka Windows Azure Platform

Web Role

AzureStorag

e

Browser

Browser

Browser

Browser

Browser

Web Role

Web Role

N L B

Worker Role

Worker RoleN

L B

p1 p2 p3

You don’t see this bit

You don’t see this bit

You don’t see this bit

Worker Role

or…Maybe you do

Windows Azure PlatformSolve using the Cloud aka Windows Azure Platform

Web Role

AzureStorag

e

Browser

Browser

Browser

Browser

Browser

Web Role

Web Role

N L B

Worker Role

Worker RoleN

L B

p1 p2 p3

You don’t see this bit

You don’t see this bit

You don’t see this bit

Worker Role

Ok, you

definitely do

SQLAzure

Windows Azure Platform

3 Important Services• Windows Azure– Compute and

Storage• SQL Azure– Storage

• .NET Services– Connecting

3 Critical Concepts• Computation– Web and Worker

• Storage– Table, Blob,

Relational• Messaging– Queues, Service Bus

Windows Azure Platform

BLOBS, TABLES, RELATIONAL

Windows Azure Platform

QUEUES

Windows Azure Platform

BLOBS, TABLES, RELATIONAL

Windows Azure Platform

BLOBS, TABLES, RELATIONAL

Windows Azure Platform

12

Windows Azure Platform

Provides structured storage• Massively scalable tables (TBs of data)• Self scaling• Highly available• Durable

Familiar and easy-to-use API, layered• .NET classes and LINQ• ADO.NET Data Services – .NET 3.5 SP1• REST – with any platform or language

Introduction to Tables

Windows Azure Platform

13

Windows Azure Platform

No join No group by No order by “No Schema”

Not a Relational Database

Windows Azure Platform

14

Windows Azure Platform

Table• A Table is a set of Entities (rows)• An Entity is a set of Properties (columns)

Entity• Two “key” properties form unique ID• PartitionKey – enables scale• RowKey – uniquely ID within a partition

Data Model

Windows Azure Platform

15

Windows Azure Platform

Getting all of dunnry’s blog posts is fast• Single partition

Getting all posts after 2008-03-27 is slow• Traverse all partitions

Key Example – Blog PostsPartition Key

AuthorRow Key

PermalinkProperty 3ChangedOn

Property 4Title

smarx i-love-tables 2009-07-04 I Love Tables!

smarx tables-are-awesome 2009-07-12 Tables are

Awesome!

dunnry cheetos 2008-03-27 I Love Cheetos

dunnry blogging-again 2009-07-15 Finally Blogging

Again

dunnry phluffyfotos 2008-04-09 PhluffyFotos!

Partition 1

Partition 2

Windows Azure PlatformQuery a Table

Worker Role

AzureTable

Storage

http://account.table.core.windows.net

REST: GET http://account.table.core.windows.net/Customer?$filter=%20PartitionKey%20eq%20value LINQ:var customers = from o in context.CreateQuery<customer>(“Customer”) where o.PartitionKey == value select o;

Windows Azure Platform

DEMO: WINDOWS AZURE STORAGE

Windows Azure Platform

18

Windows Azure Platform

Different Entity types in the same table E.g. PK = CustomerId

• Customer, Order and OrderDetails in the same table

Lessons LearnedAzure Storage – Entity Group Transactions

PK RK

Cust1 Cust1 Name=“MSFT” Desc=“...”

Cust1 Order1 Date=“25/12/09”

Cust1 OrdeDetail1 Product=12 Quantity=6

Cust1 OrderDetail2 Product=13 Quantity=1

Cust2 Cust2 Name=“IBM” Desc=“...”

Windows Azure Platform

BLOBS, TABLES, RELATIONAL

Windows Azure Platform

SQL Azure (July 2009)aka SQL Data Services

aka SQL Server Data Services

Windows Azure PlatformOn Premise Programming Model

SQL Server

Client RDBMS

TDSData

This is what we do on-premise...

Windows Azure PlatformSame for the cloud?

SQL Server

Client RDBMS

TDSData

So, is this is what we would like to do in the cloud...

Windows Azure PlatformSQL Azure can do this

SQL Azure

Client RDBMS

TDSData

Windows Azure PlatformSQL Azure can also do this

SQL Azure

Browser RDBMS

TDSWeb Role

HTTP

Windows Azure PlatformAnd this!

SQL Azure

Browser RDBMS

TDS

Web RoleHTTP

Worker Role

Queue

Windows Azure PlatformWhich means you can easily migrate from this

Browser RDBMS

Web TierHTTP

Bus. Logic

“The Data Center”

SQL Server

TDS

Windows Azure PlatformTo this… Windows Azure and SQL Azure

SQL Azure

Browser RDBMS

TDS

Web RoleHTTP

Worker Role

Queue

“The Cloud”

Windows Azure Platform

DEMO: SQL AZURE

Windows Azure Platform

29

Windows Azure Platform

From the database “down” it’s just SQL Server• Well, almost …• Many tools don’t work today

• System catalog is different

Above the database is taken care of for you• You can’t really change anything

Lessons LearnedSQL Azure

Windows Azure Platform

30

Windows Azure Platform

Tooling• SSMS partially works – “good enough”• Can not create connection using Visual Studio designer• Other tools may work better• No BCP (currently)

DDL• Must be a clustered index on every table• No physical file placement• No indexed views• No “not for replication” constraint allowed• No Extended properties• Some index options missing (e.g. allow_row_locks, sort_in_tempdb ..)• No set ansi_nulls on

Lessons LearnedSQL Azure

Windows Azure Platform

31

Windows Azure Platform

Types• No spatial or hierarchy id• No Text/images support.

• Use nvarchar(max)• XML datatype and schema allowed but no XML index or schema

collection.

Security• No integrated security

Lessons LearnedSQL Azure

Windows Azure Platform

32

Windows Azure Platform

Development• No CLR• Local temp tables are allowed • Global temp tables are not allowed• Cannot alter database inside a connection• No UDDT’s• No ROWGUIDCOL column property

Lessons LearnedSQL Azure

Windows Azure Platform

33

Windows Azure Platform

SQL Server is very familiar• SQL Azure *is* SQL Server in the cloud

Windows Azure Storage is…very different Make the right choice

• Understand Azure storage• Understand SQL Azure• Understand they are totally different

You can use both

Lessons LearnedSQL Azure vs Windows Azure Tables

Windows Azure Platform

34

Windows Azure Platform

SQL Azure is not always the best storage option SQL Azure costs more

• Delivers a *lot* more functionality

SQL Azure is more limited on scale

Lessons Learned SQL Azure vs Windows Azure Tables

Windows Azure Platform

35

Windows Azure Platform

Can be done• Many 10GB databases

Not fun

Lessons Learned SQL Azure and Sharding

Windows Azure Platform

IN SUMMARY

Windows Azure Platform

Resources

• Slides, links and more– http://geekswithblogs.net/iupdateable– Azure Training Kit (August update)

• www.azure.com– Sign up, links to resources etc

• http://www.cerebrata.com