carrier grade voip systems with kamailio - kamailio sip server

Welcome!

Henning WesterholtKamailio project1&1 Internet AG

Linuxtag 2009, 24.06.2009

Carrier grade VoIP systems with Kamailio

Kamailio project, 1&1 Henning WesterholtLinuxtag, 24.06.2009

1. 1&1 VoIP backendpurpose and usagearchitecture2. Kamailio SIP server3. High-availability and failure-tolerancepractical problemscustomer expectations and legal requirementsmonitoring, automation and proceduresmaintainability and system complexity4. Performance and scalabilityscaling issuescaching, partioning and data localityover-optimization

Outline


1&1 voice over IP backend

purposeprovide telephony services for our DSL customersbasic call routing and also supplemental servicessome numbersover 1000 million minutes per month to the PSTNmore than 2 Million customers on the platformredundant infrastructure on several levelsclustering for applications and databasesinterfacing to other carrier networks and internal systemscustom testing and monitoring systems

About Kamailio

building block of VoIP infrastructuresprovides core servicesproxyregistrarbalancer or routerapplication serverno PBX, more like a routercares only about signaling, no RTP datafoundation of custom high-performance SIP services


About Kamailio

an open source projectlicenced under GPL (version 2 or later)over 200,000 lines of C codefrequent time-based releasesmanaged from a board of core developerscommunity aspectsover 20 developers provide support and contribute new featuresfriendly and healthy user communityregular meetings at international free and open source conferencesa mature productused from carriers like 1&1, QSC, Telefonica..several companies use it to provide turn-key solutions, also sold as appliancemerge with the SER project, Kamailio forked several years ago


Why Kamailio

Scalabilityusable from small embedded systems to carrier grade systemsfrom a few hundreds user up to several millionsPerformanceon a standard server several thousands calls per seconds throughput is no problema server with enough memory can manage 600.000 usersFlexibilitysmall core written in Cfunctionality can be added with modules, over 90 already availableconfiguration script allows access to any part of the SIP messagerouting decisions can be derived from many different sourcesfurther informationsextensive documentation available on kamailio.org, see also sip-router.org

High-availability and failure-tolerance

what happens if..the primary and secondary DNS of your carrier diesyour carrier SBCs don't like your SIP anymoresomehow IPs of some carrier GWs are firewalledand don't forget your own infrastructurethe call routing proxies suddendly starts to crashyour database replication don't like to work anymorehuman error destroyes some critical databasesBut:customers expect (of course) the same reliability as in the "normal" PSTNlegal requirements (e.g. for emergency call routing) applies also to VoIP systems


Monitoring and redundancy

quality related parameter and utilization informationsJitter, Paket loss, ASR, NER, SIP status codes..minutes per minutes and ASR countincoming and outgoing traffic and its distributionboth human and automatic checking of important valuesservice level monitoringcheck if the service is still running, listen on the ports, writes logs..use tools like "sipp" to check if basic functions are availableredundancy solutionsSRV load-balancing for front-end balancersautomatic or manual failover for proxies and registrarsautomatic failure routing if one GW or carrier failsmanual routing changes also necessary by quality problems


Automation and procedures

manual processes are risky because of human errorstry to automate as much as you canexample: least-cost-routing data updategenerate DB content from a description language that is managed in a repositorypush this automatically to a master DB in order to replicate ittrigger cache re-load on all machines after finishing the uploadEstablish procedures for common tasksexample: update to a new software releaseestablish rules what and how much you change for every releasehave a pre-production test suite on a dedicated test system, and use itcreate release announcements with an update plancommunicate and follow the plan


Quality assurance and maintainability

find bugs before they reach your production systemby catching them in your internal test-suiteby getting your code in the public repositoryby catching them in a upstream test-suiteprefer general solutions over custom implementationsgetting input from the community helps here a lotalso good motivation for updatesget your code in the public repository, or even better improve existing partsprovide feedback that your requirements are heardsupport the projectfor example with infrastructure, donations, organisational or development work..fix bugs directly in the upstreamparticipate in discussions and events

carrier infrastructure at 1&1DSL reseller, no own network to the customeruses network from Telekom, Telefonica, QSC..interface to PSTN via Telefonica, Broadnet, BT..different product lines and product flavours (i.e. bitstream access vs. full PSTN line)Routing problemsprefix based routingcustomer specific routingrouting is different in case of failureshigh number of routesNot possible with plain OpenSER

Performance and scalability



storage and retrieving of location datadifficult to scale because of frequent access and changeswhat works for two proxies don't work for moreavailability is critical for call setupno stable and suitable clustering solution from MySQL in the pastproprietary partioning solution implemented as Kamailio moduleprovides also error-handling and automatic failovertry to minimize database accesspartioning also helps here to ensure data localityuse modules that cache their content in RAMsetup local read-only database slavesuse local DNS cachesuse plenty of RAM



server tuningincrease PKG_MEM and SHM_MEM pool to a few times the default valueuse more worker children for network connectionsset TCP send and connect timeout to small values, to prevent blockingmake sure you use non-blocking syslog file writingdisable dynamic blacklisting and DNS search list usage, use the port for host namesconfiguration tuningtry to optimize the common path, special cases comes laterlow-level tuning normally not necessaryuse the benchmark module to find bottlenecksdata optimizationminimize least-cost-routing rules, e.g. by combining prefixestry to get rid of obselete entries in DB based logicthink about how often data must be changed (registration interval, expire logic..)


Performance and maintainability

CPU power is not a problemgiven todays multi-core CPUsso try to find the bottleneck and optimize theremost probably it will be something related to IOprefer general solutions over custom implementationse.g. use script logic with PVs instead of coding a custom moduleimplement DB queries with sqlops/ avpops and not in Cuse the perl module for quick hacksdesign in a modular wayhierachical sub-routes for certain areas (i.e. PSTN, presence..)use small routes as "functions" to provide common functionality (normalization, relaying)place the necessary SIP functional blocks on different hostsuse only needed functionality, e.g. balancers can be stateless to save resources

Thanks for your attention!

More informations and contact:[email protected] user and sip-router developer mailing listextensive documentation available at http://kamailio.org and http://sip-router.org

Pictures:slide 1: spring flowers, central Turkeyslide 5: © Thorsten Wagner, http://www.flickr.com/people/thunderstar/slide 8: © Randi Hausken, http://www.flickr.com/people/randihausken/slide 12: © gato-gato-gato, http://www.flickr.com/people/gato-gato-gato/slide 16: sunrise at Byala beach, Bulgaria

Licence of the slides: http://creativecommons.org/licenses/by-nc-sa/2.0/

carrier grade voip systems with kamailio - kamailio sip server

Documents