Zalo Real-time System: Architecture
& ExperienceBy Nguyen Quang Nam
Zing Senior Software Manager
Web-Technical
Content
• About Zalo• Technology in Zalo• Real-time System Architecture• Experience• Q&A
About Zalo
Features
• Communication: message (voice, text), call• Social: “Nhật ký”, “Phòng trò chuyện”
Communication
Social
Updated stat
• Users: 3.0M total• Messages: 30M per day
Technology in Zalo
Technology
• Programming languages: C/C++ and Java for services at server-side• Client platforms: IOS, Android, Symbian (S40, S60),
Windows Phone (soon)
Technology (cont)
Connection servers are key points of the Zalo real-time system
Free / Opensource Inhouse
Load balancing HA proxy ZaCacheProxy, ZaRouter
Web servers Ngnix, Jetty ZaHttp
Searching Elastic, Solr Prefix Search Service
Database PostgreSQL, MySQL ZaDB, List32, List64, …
Connection servers ZaCommServer, ZEventbus
Real-time System Architecture
N-Tier
ZaCommServer
ZaCommServer (cont)
• Programming language: C/C• IO event mechanism: Epoll (libevent)• Binary protocol with authenticated connections &
encrypted messages
ZaCommServer (cont)
• Capacity per one box:• 1M concurrent connections tested• 200K throughput messages per second
Experience
Break the thing into multi-layers andlet they be independent each other
Move biz out of high performance services
Lessons Zalo learned
• Http long polling is not a good choice for a good app• Do not spend much time in optimizing small data• Memory fragment is a really big trouble in high
performance services, develop our own memory allocator lib instead of using default system glibc