Getting More 9s from Your Deployment

Amila Maha ArachchiSenior Tech Lead - Cloud TeamWSO2 Inc.

98.99999999

99.9

Agenda

● Challenges which affects the 9s● Architecting the deployment● Importance of DevOps● Monitoring● Communication● Best practices

Challenges on 9s

● Failures are inevitable● Software have bugs● Maintenance work● Upgrades● Unexpected loads● Human factor

https://upload.wikimedia.org/wikipedia/commons/0/0a/Aircraft_Rescue_Firefighting_training.jpg

Architecting the deployment

Architecting the deployment

● You don’t architect the software● Yet you need to know how it works● Weak points of it● How to address the weak points● High availability for components● Load balancing● Capacity planning● Resource allocation● Potential failures

DevOps

● Automate the deployment - every part of it● Configuration management● Schedule maintenance● Backups and restoration

“a cross-disciplinary community of practice dedicated to the

study of building, evolving and operating rapidly-changing

resilient systems at scale.”

After the deployment is done...

● Monitor, Monitor and Monitor...● Tune the parameters

○ Again, monitoring is the key to this○ Memory allocation, thread pools, cache sizes etc.

Monitoring

● Alert in advance● Be smart with alerts● Three ways of monitoring

○ Physical server or VM health■ Disk space, Load average, etc.

○ JVM health

○ Functionality health

Monitoring

Communication

● Product/Vendor -> You○ Initial deployment

○ Upgrades

○ Patching etc.

● You -> DevOps○ Submit maintenance jobs

○ Define maintenance days

○ Use checklists

Communication

● You -> End user○ Notify in advance

○ Try to make at least a part of deployment available during maintenance

Best Practices

● Use checklists● Define processes - no ad hoc work● Automated tests● Maintain a staging setup

○ With real data● Verify every change before pushing to production● Script the operations and review them in advance● Drills● Do postmortems● HA your DevOps

Takeaways

● Define processes● Create checklists● Review the scripts● Verify before pushing to live

Fight for a resilient architecture

WSO2 Managed Cloud Service

http://wso2.com/cloud/managed/

Thank You!