Scaling the server side of
occasionally-connected,
mobile systems with MongoDB
Thomas Huber - Fertl EDV-Systeme
MongoBerlin
October 4th
source: istockphoto.com/epicurean
Introduction Occasionally Connected Systems
• For e. g. Exchange Server & Outlook
• Every user has an Inbox, Calendar …..
• If you get a new Laptop – install Outlook and synch your Exchange account (synch takes time)
• After synching it is possible to go offline and read all mails offline
• booorrriing -> what about DB replication?
Definitions around Replication
• Lazy / Optimistic Replication <-> Eager ReplicationEager Replication – all nodes online are in synch, the transaction-steps are atomic Optimistic Replication – changes are logged, if the node is online, all changes are synched with other nodes, if not online, the node will synch later on. Conflicts are possible.
• Master-Slave <-> Peer-to-Peer One master, one or more slaves, not symmetric, all synchronization through the master (no Slave-to-Slave-Synch) Peer-to-Peer – all instances are talking with each other
Here is our solution
• Lazy / Optimistic Replication Smartphones can be out of coverage – offline functionality for mobile appsrobust in terms of going out of coverage and coming backlost data package detection
• Master-SlaveOne webserver / webportal, the user is able to change data on the smartphone and in the webportal, data will be synched
• One Security-Credential for Web-Portal & SmartphoneAuthentication (Login/Pwd) and Authorization (Permissions) are shared
• But keep in mind -> Exchange Server & OutlookData in the „inbox“ is synched whith the smartphonenot all data of all users
• It is possible to connect 990 different mobile devices to one user account (mixed scenarios - Symbian, iPhone, Android)
Mobile-Assistance-Framework
WTF – WHERE IS MONGODB????!!!!!!
• To scale this replication-scenario you need cheap power
• 1996 - Microsoft published a scientific paper about big replicationscenarios – one among many „The dangers of replication and a solution”... a ten-fold increase in nodes and traffic gives a thousand foldincrease in deadlocks or reconciliations ...
• It is not that ugly ;)
• But mobile apps will change the characteristics in typical load scenarios – think about more writes – think about longer transactions – because of GPRS
http://blog.appstorehq.com/post/1212703421/mashable-crushed-us-heres-how-we-bounced-back
Mongo helps in two Dimensions
• Think about 6-channel-ECG – 200Hz 16bit resolution35MByte per person per day -> 12GB per year
• 10.000 participants one year -> 120TB
• First dimension – the size -> with every new shard MongoDB is able save more data
• Second dimension – ops per seconds scales with every new shard
• if the problem is equally distributed over the machines the number of ops per second scales (it is very important to chose the right parameter)
Custom Development – maximizing the things covered in the framework
Hybrid – MongoDB and SQL
Behind the scenes
We are experimentingwith the ratio
WebServer______________ MongoMachines
Put things on the right track
• The portal is interlinked with the SQL Server (stored procedures)
• The important functions have good caching mechanisms
• Even 5 Billion User Credentials is not a real problem for a good SQL Server
• If Data grows per User per day,MongoDB helps
Easiest NoSQL-Engine (as far as I know)to port from SQL to MongoDB
• But there are no joins – so we had to implement it in our App-Server
• Compared to CassandraDB - MongoDB is much easier to query but not that easy to up-scale – but this is not a problem for us
• Easy to query http://www.mongodb.org/display/DOCS/Advanced+Queries
Thank you for your attention!More informations here:
www.mhealth-it.com
www.crossgeneration.info
I‘m presenting at the CloudConf
November 18th