![Page 1: Hands On Windows Azure Building A Twitter Clone](https://reader033.vdocument.in/reader033/viewer/2022061303/54927423b4795907178b46f3/html5/thumbnails/1.jpg)
Session Code: NE.09
Windows Azure: Building a Twitter Clone
Nik Kalyani
DotNetNuke Corporation
![Page 2: Hands On Windows Azure Building A Twitter Clone](https://reader033.vdocument.in/reader033/viewer/2022061303/54927423b4795907178b46f3/html5/thumbnails/2.jpg)
About
Nik Kalyani• Co-founder, DotNetNuke Corporation• Technology Entrepreneur, 15 years• Microsoft MVP (2005-2009)
Online • Email: [email protected]• Twitter: @techbubble• Blog: http://www.techbubble.net
![Page 3: Hands On Windows Azure Building A Twitter Clone](https://reader033.vdocument.in/reader033/viewer/2022061303/54927423b4795907178b46f3/html5/thumbnails/3.jpg)
Session Objective
Provide insights on how to architect a large-scale, asynchronous application with the same capabilities as Twitter on Azure
while leveraging its non-RDMS capabilities
![Page 4: Hands On Windows Azure Building A Twitter Clone](https://reader033.vdocument.in/reader033/viewer/2022061303/54927423b4795907178b46f3/html5/thumbnails/4.jpg)
Agenda
• Azure Architecture
• Storage Concepts
• Micro-blogging Application – Requirements– Architecture– Processing
![Page 5: Hands On Windows Azure Building A Twitter Clone](https://reader033.vdocument.in/reader033/viewer/2022061303/54927423b4795907178b46f3/html5/thumbnails/5.jpg)
ServiceBus
AccessControl
Workflow
…
Database
Reporting
Analytics
…
Compute Storage Manage
Identity
Devices
Contacts
…
…
…
Your Applications
Azure Architecture
![Page 6: Hands On Windows Azure Building A Twitter Clone](https://reader033.vdocument.in/reader033/viewer/2022061303/54927423b4795907178b46f3/html5/thumbnails/6.jpg)
Azure Application Architecture
Cloud Storage (blob, table, queue)
Web RoleLBn
Worker Role
m
![Page 7: Hands On Windows Azure Building A Twitter Clone](https://reader033.vdocument.in/reader033/viewer/2022061303/54927423b4795907178b46f3/html5/thumbnails/7.jpg)
Azure Durable Storage
• Durable, scalable, available store• Simple interface
– REST– ADO.NET Data Services
• Simple abstractions
Blobs Tables
…
Queues
![Page 8: Hands On Windows Azure Building A Twitter Clone](https://reader033.vdocument.in/reader033/viewer/2022061303/54927423b4795907178b46f3/html5/thumbnails/8.jpg)
Azure Data Storage Concepts
Account
Container Blobs
Table Entities
Queue Messages
http://<account>.blob.core.windows.net/<container>
http://<account>.table.core.windows.net/<table>
http://<account>.queue.core.windows.net/<queue>
![Page 9: Hands On Windows Azure Building A Twitter Clone](https://reader033.vdocument.in/reader033/viewer/2022061303/54927423b4795907178b46f3/html5/thumbnails/9.jpg)
Blob Storage Concepts
BlockBlobContainerAccount
erica
pictures
IMG001.JPG
IMG002.JPG
movies MOV1.AVI
Block 1
Block 2
Block 3
![Page 10: Hands On Windows Azure Building A Twitter Clone](https://reader033.vdocument.in/reader033/viewer/2022061303/54927423b4795907178b46f3/html5/thumbnails/10.jpg)
Blobs: Features and Functions
• Store large objects
• Associate metadata with blob– Metadata is <name, value> pairs– Set/Get with or separate from blob data bits
• Standard REST Interface
• Limitations– Block Id - 64 Bytes Per Block– PutBlob - 64 MB Total– Blob MetaData - 8 KB Per Blob– PutBlock / PutBlockList - 50 GB Each– Blocks - 4 MB Each
![Page 11: Hands On Windows Azure Building A Twitter Clone](https://reader033.vdocument.in/reader033/viewer/2022061303/54927423b4795907178b46f3/html5/thumbnails/11.jpg)
Table Storage Concepts
EntityTableAccount
erica
users
Name =…
Email = …
Name =…
Email = …
photo index
Photo ID =…
Date =…
Photo ID =…
Date =…
![Page 12: Hands On Windows Azure Building A Twitter Clone](https://reader033.vdocument.in/reader033/viewer/2022061303/54927423b4795907178b46f3/html5/thumbnails/12.jpg)
Tables: Features and Functions• Massively scalable, highly available and
durable structured storage– Not RDMS -- no joins, no maintenance of foreign keys
• LINQ and REST interfaces via ADO.Net Data Services– Partition key is exposed to the application
• Limitations– Up to 255 Entity Properties (including PartitionKey,
RowKey, and Timestamp)– 1 MB Max for Combined Size of All Data in an Entity's
Properties
![Page 13: Hands On Windows Azure Building A Twitter Clone](https://reader033.vdocument.in/reader033/viewer/2022061303/54927423b4795907178b46f3/html5/thumbnails/13.jpg)
Queue Storage Concepts
MessageQueueAccount
erica
thumbnail jobs
128x128, http://…
256x256, http://…
photo processing
jobs
http://…
http://…
![Page 14: Hands On Windows Azure Building A Twitter Clone](https://reader033.vdocument.in/reader033/viewer/2022061303/54927423b4795907178b46f3/html5/thumbnails/14.jpg)
Queues: Features and Functions• An account can create many queues..
– Queue Name is scoped by the account
• A Queue contains messages..– No limit on number of messages stored in a queue– A message is stored for at most a week in a queue
• http://<Account>.queue.core.windows.net/<QueueName>
• Messages..– Message size <= 8 KB– To store larger data, store data in blob/entity storage,
and the blob/entity name in the message
![Page 15: Hands On Windows Azure Building A Twitter Clone](https://reader033.vdocument.in/reader033/viewer/2022061303/54927423b4795907178b46f3/html5/thumbnails/15.jpg)
• Relationships
– Any user can “follow” any other user
– A user can “unfollow” another user he/she is “following”
Micro-blogging ApplicationRequirements
![Page 16: Hands On Windows Azure Building A Twitter Clone](https://reader033.vdocument.in/reader033/viewer/2022061303/54927423b4795907178b46f3/html5/thumbnails/16.jpg)
• Micro-blogging
– Any user can post a message of up to 140 characters in length
– All message posts are public
Micro-blogging ApplicationRequirements
![Page 17: Hands On Windows Azure Building A Twitter Clone](https://reader033.vdocument.in/reader033/viewer/2022061303/54927423b4795907178b46f3/html5/thumbnails/17.jpg)
• Timeline
– A public timeline of all posted messages is available to anybody for up to 24 hours from the time the message was first posted
– A user timeline of all messages posted by a user is available to anybody forever
– A “following” timeline of all messages posted by all users that a user is following in the prior 24 hours is available to each user
Micro-blogging ApplicationRequirements
![Page 18: Hands On Windows Azure Building A Twitter Clone](https://reader033.vdocument.in/reader033/viewer/2022061303/54927423b4795907178b46f3/html5/thumbnails/18.jpg)
• Micro-blogging
– The number of users will determine volume of messages
– However, common situation is that a small number of users will post majority of the messages
Micro-blogging ApplicationArchitectural Considerations
![Page 19: Hands On Windows Azure Building A Twitter Clone](https://reader033.vdocument.in/reader033/viewer/2022061303/54927423b4795907178b46f3/html5/thumbnails/19.jpg)
• Timelines
– The public timeline is transient and will have a high frequency of reads and writes
– A user timeline is permanent and will have a small number of writes and a large number of reads
– A “following” timeline is transient and will have a large number of irregularly timed writes and a small number of reads
Micro-blogging ApplicationArchitectural Considerations
![Page 20: Hands On Windows Azure Building A Twitter Clone](https://reader033.vdocument.in/reader033/viewer/2022061303/54927423b4795907178b46f3/html5/thumbnails/20.jpg)
Micro-blogging ApplicationProfile, Messages and Relationships Storage
Table = UserPartitionKey = UsernameRowKey=string.Format("{0:D19}", DateTime.MaxValue.Ticks – DateTime.UtcNow.Ticks);
• Use Table storage sharded by user
• Entities with the same PartitionKey are stored on the same storage node
• Obtaining recent messages, followers or following list require only a single query for each
• Adding or Deleting Follower or Following requires two writes (i.e. one for each user)
Properties
EntityType = “Message”Source = UsernameMessage Text
EntityType=“Follower”Username
EntityType=“Following”Username
EntityType=“Profile”<Any profile property>
![Page 21: Hands On Windows Azure Building A Twitter Clone](https://reader033.vdocument.in/reader033/viewer/2022061303/54927423b4795907178b46f3/html5/thumbnails/21.jpg)
Micro-blogging ApplicationFollow
User A Partition
Follower[User C]
User B Partition
Following[User D]
User C Partition
Follower[User Y]
Follower[User Z]
Following[User A]
Following[User X]
Message Queue
Maintenance Queue
User A follows User B
![Page 22: Hands On Windows Azure Building A Twitter Clone](https://reader033.vdocument.in/reader033/viewer/2022061303/54927423b4795907178b46f3/html5/thumbnails/22.jpg)
Micro-blogging ApplicationFollow
User A Partition
Follower[User C]
Following[User B]
User B Partition
Follower[User A]
Following[User D]
User C Partition
Follower[User Y]
Follower[User Z]
Following[User A]
Following[User X]
Message Queue
Maintenance Queue
![Page 23: Hands On Windows Azure Building A Twitter Clone](https://reader033.vdocument.in/reader033/viewer/2022061303/54927423b4795907178b46f3/html5/thumbnails/23.jpg)
Micro-blogging ApplicationFollow
User A Partition
Follower[User C]
User B Partition
Following[User D]
User C Partition
Follower[User Y]
Follower[User Z]
Following[User A]
Following[User X]
Message Queue
Maintenance Queue
User B follows User A
Follower[User A]
Following[User B]
![Page 24: Hands On Windows Azure Building A Twitter Clone](https://reader033.vdocument.in/reader033/viewer/2022061303/54927423b4795907178b46f3/html5/thumbnails/24.jpg)
Micro-blogging ApplicationFollow
User A Partition
Follower[User B]
Follower[User C]
Following[User B]
User B Partition
Follower[User A]
Following[User A]
Following[User D]
User C Partition
Follower[User Y]
Follower[User Z]
Following[User A]
Following[User X]
Message Queue
Maintenance Queue
![Page 25: Hands On Windows Azure Building A Twitter Clone](https://reader033.vdocument.in/reader033/viewer/2022061303/54927423b4795907178b46f3/html5/thumbnails/25.jpg)
Micro-blogging ApplicationMessage Posting
User A Partition
Follower[User B]
Follower[User C]
Following[User B]
User B Partition
Follower[User A]
Following[User A]
Following[User D]
User C Partition
Follower[User Y]
Follower[User Z]
Following[User A]
Following[User X]
Message Queue
Maintenance Queue
User A posts a message
![Page 26: Hands On Windows Azure Building A Twitter Clone](https://reader033.vdocument.in/reader033/viewer/2022061303/54927423b4795907178b46f3/html5/thumbnails/26.jpg)
Micro-blogging ApplicationMessage Posting
User A Partition
Follower[User B]
Follower[User C]
Following[User B]
User B Partition
Follower[User A]
Following[User A]
Following[User D]
User C Partition
Follower[User Y]
Follower[User Z]
Following[User A]
Following[User X]
Message Queue
Maintenance Queue
Message added to Message Queue
User AMessage
![Page 27: Hands On Windows Azure Building A Twitter Clone](https://reader033.vdocument.in/reader033/viewer/2022061303/54927423b4795907178b46f3/html5/thumbnails/27.jpg)
Micro-blogging ApplicationMessage Posting
User A Partition
Follower[User B]
Follower[User C]
Following[User B]
User B Partition
Follower[User A]
Following[User A]
Following[User D]
User C Partition
Follower[User Y]
Follower[User Z]
Following[User A]
Following[User X]
Message Queue
Maintenance Queue
After Message Worker Role processes…
Msg (A)
Msg (A)
Msg (A)
User B User C
![Page 28: Hands On Windows Azure Building A Twitter Clone](https://reader033.vdocument.in/reader033/viewer/2022061303/54927423b4795907178b46f3/html5/thumbnails/28.jpg)
Micro-blogging ApplicationMessage Posting
User A Partition
Follower[User B]
Follower[User C]
Following[User B]
User B Partition
Follower[User A]
Following[User A]
Following[User D]
User C Partition
Follower[User Y]
Follower[User Z]
Following[User A]
Following[User X]
Message Queue
Maintenance Queue
Msg (A)
Msg (A)
Msg (A)
After Maintenance Worker Role processes…
![Page 29: Hands On Windows Azure Building A Twitter Clone](https://reader033.vdocument.in/reader033/viewer/2022061303/54927423b4795907178b46f3/html5/thumbnails/29.jpg)
Micro-blogging ApplicationMessage Maintenance
User A Partition
Follower[User B]
Follower[User C]
Following[User B]
User B Partition
Follower[User A]
Following[User A]
Following[User D]
User C Partition
Follower[User Y]
Follower[User Z]
Following[User A]
Following[User X]
Message Queue
Maintenance Queue
Msg (A)
Msg (A)
Msg (A)
Msg (D)Msg (D)25 hours
Msg (X)
User A posts a message
![Page 30: Hands On Windows Azure Building A Twitter Clone](https://reader033.vdocument.in/reader033/viewer/2022061303/54927423b4795907178b46f3/html5/thumbnails/30.jpg)
Micro-blogging ApplicationMessage Maintenance
User A Partition
Follower[User B]
Follower[User C]
Following[User B]
User B Partition
Follower[User A]
Following[User A]
Following[User D]
User C Partition
Follower[User Y]
Follower[User Z]
Following[User A]
Following[User X]
Message Queue
Maintenance Queue
Msg (A)
Msg (A)
Msg (A)
Msg (D)Msg (D)25 hours
Msg (X)
Message added to Message Queue
User AMessage
![Page 31: Hands On Windows Azure Building A Twitter Clone](https://reader033.vdocument.in/reader033/viewer/2022061303/54927423b4795907178b46f3/html5/thumbnails/31.jpg)
Micro-blogging ApplicationMessage Maintenance
User A Partition
Follower[User B]
Follower[User C]
Following[User B]
User B Partition
Follower[User A]
Following[User A]
Following[User D]
User C Partition
Follower[User Y]
Follower[User Z]
Following[User A]
Following[User X]
Message Queue
Maintenance Queue
Msg (A)
Msg (A)
Msg (A)
Msg (D)Msg (D)25 hours
Msg (X)
After Message Worker Role processes…
User B User C
Msg (A)
Msg (A)
Msg (A)
![Page 32: Hands On Windows Azure Building A Twitter Clone](https://reader033.vdocument.in/reader033/viewer/2022061303/54927423b4795907178b46f3/html5/thumbnails/32.jpg)
Micro-blogging ApplicationMessage Maintenance
User A Partition
Follower[User B]
Follower[User C]
Following[User B]
User B Partition
Follower[User A]
Following[User A]
Following[User D]
User C Partition
Follower[User Y]
Follower[User Z]
Following[User A]
Following[User X]
Message Queue
Maintenance Queue
Msg (A)
Msg (A)
Msg (A)
Msg (D)
Msg (X)
Msg (A)
Msg (A)
Msg (A)
After Maintenance Worker Role processes…
![Page 33: Hands On Windows Azure Building A Twitter Clone](https://reader033.vdocument.in/reader033/viewer/2022061303/54927423b4795907178b46f3/html5/thumbnails/33.jpg)
• Cloud app storage significantly different from RDMS…requires shift in thinking
• Queues make apps efficient … processing is done asynchronously
• Duplication of data is often required and this is OK
• Sharding is an important consideration for the scalability of your cloud application
• Absence of join, sort make application design challenging but with planning this is manageable
Micro-blogging ApplicationTakeaways
![Page 34: Hands On Windows Azure Building A Twitter Clone](https://reader033.vdocument.in/reader033/viewer/2022061303/54927423b4795907178b46f3/html5/thumbnails/34.jpg)
Evaluation form
Vul je evaluatieformulier in en maak kans op een van de prachtige prijzen!!
Fill out your evaluation form and win one of the great prizes!!
Session Code: NE.09