storage options on windows azure

Database options on Azure

Himanshu Desai

Principal Consultant, Technical Specialist (WCF)1

Page

Agenda› Understand Azure Database Offering

› Overview of Azure SQL Database

› Overview of Azure Table Storage

› Overview of DocumentDB

› Suitability of Each options

/ Copyright ©2014 by Readify Pty Ltd2

Page / Copyright ©2015 by Readify Pty Ltd4

DB

SQL

DB

SQL

DB

SQL

App

SQL Azure

DB

Virtual Machine

SQL Server

Document DB

Partition KeyPartition

1

2

A

Table Storage

1

A

Row Key

Entity

Page

Who is responsible?› Developers

› Testers

› Operations


SQL Azure

Page

Features› Database as a service

› Elastic Scale with Sharding*

› Predicatableperformance

› Availability backed by replica and SLA

› Data protection via auditing, restore and geo-replication

› Self managed

› Three replica are running at one time (one primary and two or more seconday)

› Availability in 3 Tiers: Basic (2GB), Standard (250GB) & Premium (500GB)


Page

Sharding – Why?› The total amount of data is too large to fit within the constraints of a single database

› The transaction throughput of the overall workload exceeds the capabilities of a single

database

› Tenants may require physical isolation from each other, so separate databases are needed

for each tenant

› Different sections of a database may need to reside in different geographies for

compliance, performance or geopolitical reasons.


Page

Horizontal Vs Vertical Scaling


Page

Single Vs Multi-Tenant


Page

Backup & Restore› Point in time restore

› Geo-Restore

› Standard Geo -replication

› Active Geo-replication.


Page

Management› SQL Management Studio

› REST API’s

› Powershell Scripts

› Azure Management portal


Page

What is not supported› https://msdn.microsoft.com/en-us/library/azure/ee336281.aspx


Page

Demo


Page

Azure Table Storage


Page

What is Table Storage› Auto-sharded, key value store

› Single table can contain upto 500TB at a cost of

7c/Gb/month

› Scalability Target of 20K ops/second


Page

Key Features› Schema less with only 3 predefined fields

› Structured Storage

› Massively scalable tables

› Highly available and durable

› Data is replicated several times

› Usage via REST API or Windows Azure Storage client

libraries

› Storing data for efficient retrieval and presistence/ Copyright ©2014 by Readify Pty Ltd17

Page

Required Entity Properties› PartitionKey & RowKey

› Uniquely Identifies an entity

› Defines the sort order

› Use them to scale your app

› Timestamp

› Readonly

› Optimistic concurrency


Page

Table Storage Concepts


BooksOnline

Users

Books

Email

Name

Email

Name

Title

ISBN

Title

ISBN

Accounts Tables Entities

Page

Table Data Model› Table

› A Storage account can create many tables

› Table name is scoped by account

› Set of entities (i.e. rows)

› Entity

› Set of properties (columns)

› Required properties

› PartitionKey, RowKey & Timestamp/ Copyright ©2014 by Readify Pty Ltd20

PagePage / Copyright ©2014 by Readify Pty Ltd21

Partition Key

1

A

Row Key

Entity

Partition: ABC / Machine 1

1

2

A

Partition: DEF / Machine 2

1

2

A

Partition : HIJ / Machine 3

1

2

A

PartitionId RowKey Name Address Timestamp

ABC 1 …

ABC 2 ….

ABC 3 ….

PartitionId RowKey Name Address Timestam

p

DEF 4

DEF 5

DEF 6

PartitionId RowKe

y

Name Address Timestam

p

HIJ 7

HIJ 8

HIJ 9

Page

PartitionKey and Partitions› PartitionKey

› Used to group entities in the table into parititions

› A table partition

› All entities with same partition key value

› Unit of scale

› Control entity locality

› Row key provides uniqueness within a partition


Page

Partitioning Tables- Why› Scale

› Each individual partitions are distributed across multiple storage nodes

› System monitors the partitions usage and automatically balances

partitions across multiple storage nodes

› A partitions i.e. all entities with same partition key, will be served by a

single node

› Entity group transactions

› Allow the application to auto perform multiple Create/Update/Delete

operations across multiple entities using batch request


Page

Query Speed› Fast

› Single paritionkey and rowkey with equality

› Medium

› Single partion but a small range of rowkey

› Entire partition or table that is small

› Slow

› Large single scan

› Large table scan

› “OR” predicates on keys => no query optimisation =>results in scan

› Expect Continuation tokens/ Copyright ©2014 by Readify Pty Ltd24

Page

Make Faster Queries› Large Scans

› Split the range and parallelize queries

› Create and maintain own views that help queries

› “Or” Predicates

› Execute individual query in parallel

› User Interactive

› Cache the result to reduce scan frequency


Page

Continuation token› Max of 1k rows in a response

› At the end of partition range boundary

› Max of 5 secs to execute the query

› Expect continuation token always

› If the query times out, Server returns continuation token so that client

can make another query

› When the scan crosses the partition boundaries, continuatin token is

returned.


Page

Pagination› Use IQueryable<>.Take(N) to fetch the top results

› Use continuation

› http://<serviceUri>/Something?<originalQuery>&NextPartitionKey=<

someValue>&NextRowKey=<someOtherValue>


Page

Retrieve latest value› Have the row key as

› DateTime.MaxValue.Ticks-DateTime.UtcNow.Ticks


Page

Prefix based retrieval› Use CompareTo and ‘>’ and ‘<‘ function effectively

› myTable.Partitionkey.CompareTo(“somekey”)>=0


Page

Demo


Page

DocumentDB


Page

What is DocumentDB?› Document database that supports fully manage

storage of JSON documents in a highly performant

way.

› Fully managed distributed database.

› (Still in preview and available in West US,North & West

Europe, cost around 73 cents a day which includes 50%

discount.)


Page

Key Features› Schema less, NoSQL document database

› Fully managed, with provisioned capacity

› Allows for SQL like queries on Semi structured data (i.e. JSON files)

› Supports strong consistency for transactional applications

› Designed to scale into petabytes

› Includes libraries for Node,.NET, REST,Javascript (Client & Server),Python

› Includes Hekaton technologies (In Memory SQL)


Page

Management› Supported in new portal

› Manage database account, collections, users etc

› View consumption statistics

› http://portal.azure.com

› Use tools like

› Query explorer

› Document explorer

› DocumentDB studio


Page

Overview of DocumentDB


Page

Database Accounts


Page

Resource Model› Database Account

› Database

› Collection

› Document

› Attachment

› Stored Procedure

› Trigger

› User-defined functions

› User

› Permission

› Media/ Copyright ©2014 by Readify Pty Ltd37

Page

Resource Addresssing› Restful

› Unique ID

› API Url:

› https://{databaseaccount}.documents.azure.com

› Document Path

› /dbs/{database id}/colls/{collectionid}/docs/{documentid}

› Client libraries

› .NET

› Javascript

› Python/ Copyright ©2014 by Readify Pty Ltd38

Page

Capacity Unit› Database accounts comes with administrative quota of

› 100 databases

› 500,000 users

› 2,000,000 permissions

It includes quota for collections*

If you run out of CU, add more. They are stackable backed by SSD.

Each CU comes with 3 elastic collections,2000 RU & 10 GB of SSD

backed document storage

› Elastic by default. Document portioned by collection can have unlimited

storage. Not scoped to machine. Acts as a transaction domain


Page

REST API› Standard Operations

› Create,Delete,Put,Get,Post

› Permanent Uri on Create

› HMAC authentication using management or resource

key

› DocumentDB request headers


Page

.NET Development› You need following from the portal after provision a

database

› Endpoint Url

› Authorization key

› Nu-get package

› Install-Package Microsoft.Azure.Documents.Client –

Pre

› Use them to create DocumentClient/ Copyright ©2014 by Readify Pty Ltd41

Page

Demo


PagePage / Copyright ©2014 by Readify Pty Ltd43

Table Storage SQL Azure DocumentDB

Semi-structured Fully Structured Semi-Structured

Loosely typed Strongly typed Document Based Access

Non-Relational (Not RDMS) Relational (RDMS) Non-Relational

Massively scalable Highly Scalable Massively Scalable

REST API & Client SDK for

querying

Full SQL Support Good SQL Support for

querying hierarchical data

within a collection

N/A Stored Procs, UDFs etc Stored Procs, UDFs etc

Each CU comes with 3

collections. In Preview,

collection can grow upto

10GB and 2K RU of

throughput.

Page

Thank you

Himanshu Desai

[email protected]


storage options on windows azure

Technology

durable storage

storage offeringpagelets

offcourse storage

main topics azure sql

azure files butfor

isolated storage available

main categorieslocal

microsofts sql database