brokered messaging in windows azure
DESCRIPTION
An overview of the Brokered Messaging feature on the Windows Azure Platform. Brokered Messaging supports Queues, Topics and Subscriptions providing message-based sollutions for load balancing, load leveling and pub/sub scenarios.TRANSCRIPT
NEIL MACKENZIESATORY GLOBAL
Brokered Messagingin Windows Azure
Who Am I?
Neil MackenzieWindows Azure MVPBook:
Microsoft Windows Azure Development Cookbook
Blog: http://convective.wordpress.com/Twitter: @mknz
Content
Messaging overviewConceptsService Bus (digression)Brokered Messaging APIDemo
Messaging
Facilitates service-oriented architecture
Windows Azure Platform supports: Direct
Push - WCF Relayed
Push – Service Bus Relayed Messaging Brokered
Pull – Service Bus Brokered Messaging
Brokered Messaging
Windows Azure Service Bus Brokered Messaging Provides persistent store for messages Exposes
Send endpoint Listen endpoint
Supports: Structured message content Serializable message body Sophisticated message receipt semantics Message sessions Correlated messages
Scenarios
Temporal decoupling Senders and receivers don’t need to be active
simultaneouslyLoad leveling
Receiver processes messages at its own paceLoad balancing
Scale up number of receivers to handle loadPub/Sub - Multicasting
Multiple subscriptions
Message Store
Managed by Brokered MessagingExposed through a service path on the
Service BusAccess controlled through Service Bus ACSMaximum size of queue/topic 1, 2, 3, 4, 5 GBMaximum message size: 256KBMessage comprises:
Header (name/value pairs) < 64KB Body (serializable)
At-Most Once or At-Least Once
Brokered Messaging supports: At-most once semantics
ReceiveMode.ReceiveAndDelete At-least once semantics
ReceiveMode.PeekLock (default)
Message disposition with PeekLock: Abandon Complete DeadLetter Defer
Queues
S
R
R
S
CompetingReceivers
MultipleSenders
Queue
S
Topics & Subscriptions
S
R
RS
IndependentlyCompetingReceivers
MultipleSenders
R
R
R
R
Subscriptions
Rules(filters & actions)
Topic
S
Rules, Filters and Actions
Rule comprises a Filter and an Action Filter uses Properties bag of a subscription to filter
messages in the subscription Action uses SQL92 syntax to modify the Properties
bag of a subscriptionFilters:
CorrelationFilter - uses CorrelationId FalseFilter - no messages SqlFilter - SQL 92 syntax TrueFilter - all messages (default)
Aside: Service Bus Namespaces
Format: [namespace].servicebus.windows.net/[ServicePath]
Examples: gress.servicebus.windows.net/services/echo
gress.servicebus.windows.net/interestingtopic
gress.servicebus.windows.net/news/subscriptions/subscriber Namespace: gress Topic name: news Subscription name: subscriber
Aside: Service Identity
“Owner” – admin account for entire namespaceFollowing claims used for each service path:
manage send listen
Create specific service identity for service pathUse minimal set of claims
e.g. only “send” claim needed for a send-only applicationSBAzTool
SDK sample – application to manage service identities
Token Providers
TokenProvider exposes factory methods creating various authentication token providers: Saml token Shared secret token Simple web token
Uri serviceUri = ServiceBusEnvironment.CreateServiceUri( "sb", serviceNamespace, String.Empty);TokenProvider tokenProvider = TokenProvider.CreateSharedSecretTokenProvider( issuer, issuerKey);
Managing Queues, Topics and Subscriptions
NamespaceManager exposes factory methods to create and delete: queues topics subscriptions
NamespaceManager namespaceManager = new NamespaceManager( serviceUri, tokenProvider);namespaceManager.CreateQueue("queueName");
Queue, Topic and Subscription Metadata
QueueDescription, TopicDescription and SubscriptionDescription used to configure queues, topics and subscriptions.
Properties (e.g.): DefaultMessageTimeToLive (qts) LockDuration (qs) (Max 300s, def
60s) MessageCount (qs) Name (s) Path (qt) RequiresSession (qs)
Client Classes
MessagingFactory exposes factory methods to create the MessagingClientEntity derived objects used to send and receive brokered messages: MessageReceiver MessageSession MessageSender QueueClient SubscriptionClient TopicClientMessagingFactory messagingFactory = MessagingFactory.Create( serviceUri, tokenProvider);
Class: Brokered Message
BrokeredMessage Represents the brokered message to be sent to a
queue or topic, or retrieved from a queue or subscription.
Comprises a header with various properties and a body created from a serializable class.
Properties Properties bag can be used, instead of the message
body, to store the content of a brokered message Very powerful technique enabling rules, filters and
actions
Class: QueueClient
QueueClient Handles sending and receiving brokered messages for
a queue. Implements standard message disposition methods
(Abandon, Complete, DeadLetter and Defer) when using ReceiveMode.PeekLock
Supports synchronous and asynchronous versions of all methods
No support for rules, filters and actions
Class: TopicClient
TopicClient sends brokered messages to a topic. Supports synchronous and asynchronous versions of
all methods
TopicClient sender = messagingFactory.CreateTopicClient("topicName");BrokeredMessage brokeredMessage = new BrokeredMessage(serializableObject);sender.Send(brokeredMessage);
Class: SubscriptionClient
SubscriptionClient receives brokered messages from a subscription implements standard message disposition methods
(Abandon, Complete, DeadLetter and Defer) when using ReceiveMode.PeekLock
supports synchronous and asynchronous versions of all methods
Classes: MessageReceiver & MessageSender
MessageSender Abstracts functionality of QueueClient and TopicClient
so that it can send messages to either queues or topics.
MessageReceiver Abstracts functionality of QueueClient and
SubscriptionClient so that it can receive messages from either queues or subscriptions.
Easiest way to access the deadletter queue.
Class: MessageSession
MessageSession provides specialized MessageReceiver used to handle
sessions receives messages only for a particular session
Sessions are supported only if queue or subscription created with RequiresSession property
Session is a sequence of brokered messages with the same SessionId
Deadletter Subqueue
Subqueue containing “failed” messagesMessages are added to deadletter subqueue
When explicitly deadlettered On message expiration (if configured) On filter evaluation failure (if configured)
Deadletter queue processed like any other queue
Named by extending service path, e.g.: /queue/$deadLetterQueue /topic/Subscriptions/subscription/$deadLetterQueue
WCF Interface
WCF implementation of brokered messaging uses: NetMessagingBinding
Binding: Folds brokered message into a WCF message Handles message polling for receiver
See: Rick Hollander post on MSDN http://bit.ly/s2Zagg
REST Interface
REST interface supports: Send Receive Filters More senders and receivers than .NET API
REST interface does not support: Sessions Client batching
Comparison With Azure Storage Queues
Feature Azure Queues Brokered Messaging
API REST, .NET .NET, REST, WCF
Authentication Storage Service HMAC
Service Bus ACS
Maximum queue size 100TB 5GB
Maximum message size
64KB 256KB
Maximum message TTL
7 days 10,675,199 days
At most once delivery
No Yes
At least once delivery
Yes Yes
Maximum message lock
7 days 5 minutes
Hosted service affinity
Yes No
Receive behavior Non-blocking Long polling (<24 days)
Throughput 5,000 msgs/second 800-3,000 msgs/sec
See: http://bit.ly/tY96CZ
Performance
Throughput: 800-3000 1KB messages / second through connection Scales down by number of subscriptions
Control performance through: Client-side batching (default: on - factory)
send/complete Database access (default: on - client entity)
Server caching Prefetching (default: off – client entity)
Use multiple queues and topics if necessary
Best Practices
Use: Non-blocking asynchronous senders & receivers
Asynchronous programming model (BeginX/EndX) Transient Fault Handling Framework Single message factory (connection) Single client entity (thread safe) Client-side batching
Do not use: Default version of Task Parallel Library
See http://bit.ly/t5jUdm
References
Alan Smith: Developers Guide to AppFabric http://bit.ly/uB0aHS
Rick Garibay: Code magazine article http://bit.ly/sRm9iK
Clemens Vasters: Build 2011 http://bit.ly/t5ip6k
Neil Mackenzie: blog post http://bit.ly/oZdewm
MSDN documentation http://bit.ly/v9dcPD