telestax: processing billions of sms records daily for a top tier telecom
TRANSCRIPT
Telecom Big Data Processing Billions of SMS Records Daily with
Cassandra NoSQL
Ivelin IvanovCEO, co-founder
Athar Rafiq Sr Manager - IT, UFone, Etisalat Group
About Ufone
● Ufone is a wholly owned subsidiary of Pakistan Telecommunication Company Limited (PTCL).
● Started operations in January 2001● Built a subscriber base of over 24 million in less
than a decade● As a result of PTCL's privatization, Ufone became
a part of the Etisalat in 2006.● Ufone offers Pakistan's fastest growing 3G
network services
The Business Challenge
● New business use case for SMS marketing● Requires processing of hundreds of millions of
SMS per day and billions of CDR records● Too expensive to purchase additional legacy
infrastructure for SMS
The Technical Challenge
● Legacy telecom infrastructure● Black boxes that cannot be extended● Legacy inflexible and very very expensive vendor
Objective
To deploy a cost effective, highly robust and carrier proven SMSC that can be integrated with Ufone’s Campaign Management Solution.
Telestax SMSC in Ufone
● Very Robust handling 40-50 M SMS/day● Cost Effective/virtual Infrastructure● Highly Reliable● Store & Forward● Integrated with IBM’s Omni Channel Marketing
(formerly known as UNICA)● Handling Multiple LA (short codes)
Future Plans
● Expansions in existing capacity● Integrating Telestax SMSC with Digital Services
Platforms● Applications access to Big Data analytics using
APIs
About Telestax
The #1 Open Source Communications
Middleware
Restcomm.com
Worldwide customers
9
Telestax Customers
Direct paying customers in over 50 countries. Not counting sales via channel partners.
Powered by the TeleStax Communications Platform based on Mobicents
RestComm allows Service Providers and Enterprises to
improve end user experience and optimize business workflows
via Real Time Communications
The Road From Blag Box to Big Data
Early failures
● Started with RDBMS, soon faced below challenges○ How to be
■ Scalable with massive writes and reads■ Distributed■ Control over performance characteristics■ High Availability■ Low Latency■ Low hardware + software cost
○ No strict ACID requirements. Doesn’t matter much if SMS is not delivered instantaneously.
Early failures
● Started with naive NoSQL use - Cassandra● Beginners mistake of using normalized data
structures as SQL habits advise● Inserts were super fast● Reads failed horribly● Deletes were amazingly slow
Final solution
● Forget normalization!● Single Table, dynamically add columns
○ One Table each for live (retry) SMS and archived (delivered or permanently failed) SMS
● Tables created for each day○ Easy management ○ Archiving○ Fast Read
DestinationNumber TimeSlot SMS1 SMS2 SMS3 ... ... ...
15129999999 t1 xxx yyy zzz ... ... ...
Dynamically add columns
Composite Primary Key
Final solution (cont.)
● Never Delete rows!○ SSTables are immutable. We can’t remove rows○ Cassandra creates Tombstones for deleted data
● Any SMS failed, is inserted in new row with new timeslot
● Drop Tables
DestinationNumber TimeSlot SMS3 ... ... ... ... ...
15129999999 (t1+10) zzz ... ... ... ... ...
Dynamically add columns
Composite Primary Key
Final solution - Architecture
Final solution - Architecture
Lessons Learned
● Can’t ignore Cassandra’s internal workings● Read performance is heavily dependent on the
temporal patterns of your writes● Various write patterns make read performance
slowly decrease● Avoid deleting data, instead think of ways where
the whole table can be dropped
● Sales. Improving web site user conversion rates via intelligent integration of web calls data into marketing funnel.
● Support. Improving customer support by replacing archaic 1-800 menus and blind transfers into contextual intelligent agent response.
● Marketing. Measuring campaigns by intelligently blending call data with web and mobile app user data.
● We are barely scratching the surface of RTC analytics
Future Opportunities in Telco Analytics
From Dreaded Telecom to Programmable Contextual RTC
@telestax
http://www.telestax.com
Q & A