which messaging layer to use in a loosely coupled distributed app (with notes)
DESCRIPTION
slides from my talk at PyCon 2014 in Montreal, CanadaTRANSCRIPT
Which messaging layer to use in a loosely coupled
distributed app?!
Narahari (Hari) Allamraju
!
java developer; python enthusiast working with messaging systems for 9.5 years
!
http://github.com/supercoderz [email protected]
• brief intro about work done • what to expect from this talk?
• a outline of what messaging is about and where it is used • how you should apply this to your situation and what to watch out for • advice based on experience from working on medium to large scale applications
Messeveryone talks to everyone
2
• as the organisations grow, they build up a large number of IT systems performing different functions • sometimes it is not possible to throw away all the systems and build a more integrated solution for legacy reasons • sometimes the problem is just a question of improving efficiency • sometimes its about the best utilisation of legacy systems, resources and costs • these factors drive us to build connections between the applications. • there are many ways to do this - good and bad ways.
Messagingeveryone talks to everyone through a single medium
3
• a properly designed messaging system lets the applications and components work with each other without getting in their way • its part of the architecture and is transparent enough that no one feels they need to do a lot of boiler plate extra work to get data across • its efficient and shows an improvement from the previous state of the system - you can tell that you are no longer doing time consuming manual work to ensure all systems see the same data. • and more importantly you should be able to see a consistent view of the entire organisation from any application - real time or a consistent view as of a given time.
Work Queueeveryone queues their tasks at the same place and waits
for the results4
• the other kind of use for messaging queues - more commonly seen today across all the web applications and user centric web applications is the work queue • these messaging systems follow the same logic as the ones that help connect all the systems in an organisations - help increase the efficiency without getting in the way or increasing work load.
So you want to build a loosely
coupled app!
why? do you know what side effects you will have?
!
ok you know … !
what do you need to do?
5
• why? • what do we get when we build a loosely coupled app - apart from the increase in efficiency?
• scalability • distributed load • reliability - since we know the flow and it is proven to work without manual intervention • ease of making changes
• it also adds a lot more moving parts :)
The basics• Will the application benefit by
splitting up into components?
• What are the components? How are they dependent on each other?
• Do we need more than one instance of each to cope up with the load?
• Do we need newer components to handle certain cases that arise from the new messaging layer?
6
• just like everything else, you need to think out carefully about what you want - because once its built you don’t want to be in doubt about what you were expecting, and you should see the results • in some cases, splitting up work or adding a messaging layer can cause other unexpected issues
• case 1 : a file based data transfer was changed to a pseudo real time process through a message queue. however, this meant that in case the data had to be replayed or re-loaded in case of certain operational issues, it could not be done. this could have been solved by adding a record/replay component.
• case 2 : a consumer application was working slow, this caused the messages to pile up and trigger an alert for the support team due to disk/memory usage. while not a problem, this is an operational issue. •
Identify the patterns• What sort of interface do we
need - request-reply, publish subscribe or queuing?
• Do we need to separate out messages into various types?
• Do we need schemas for the messages or free form?
• Are there any components that have a higher or lower throughput than others?
7
• its always good to know how the components you will integrate work - helps you to build in protections and handle unexpected corner cases • it also helps you to define the best architecture for your needs - this is important because if the architecture that you choose does not reflect the most common use cases, then you will not get the most out of your solution • it also helps you to determine the kind of setup you will need and the resources like storage etc
Latency, routing, priority• Which components need the
message yesterday; and which can wait till end of day?
• Do we need to expire messages when they are too late?
• How easily can we route messages?
• Can we make some messages take highest priority?
8
• messages are always important and you can never lose one • but the application that consumes them can decide whether it wants all of the, or the last one or if it can afford to lose a message once in a while • sometimes it is even required to separate out the applications that consume messages slower into their own partitions in the architecture •
Build, Manage and Support
• Do we have a suitable API?
• Easy to provision and manage
• Good monitoring
• Guaranteed delivery of messages
• Should be persist messages to disk for crash recovery?
• Cleanup in case of issues
9
• developers come first because like any application, they need to code an interface to the messaging layer from their existing application • also if they want to setup a developer environment, or a test instance - it should be very easy to do • support teams would love features that let them find issues faster • built in fail safe - pull the plug
A few options
• RabbitMQ
• ZeroMQ
• Redis
• RDBMS or NOSQL database
• RPC
10
Considering future growth
• What if the loads increased by 15%?
• What if we had a consistent heavy run for 24 hours or more?
• Storage costs etc
• If we have to switch to a different underlying - how hard is it?
Tokyo Metro
11
• this is something that actually happens on the train at peak hours • the best part of this - even with that rush, everyone understands, accommodates and everyone ensures that people get to where they need to get to - the whole system manages to handle that load, just at the limit and come out of it. almost 365 days a year.
Questions!• This presentation can be
found on slideshare at - http://goo.gl/KVJxoE !
• The version with speaker notes will be available soon !
• you can also email me at [email protected]
12