how we took our server side application to the cloud and liked what we got
DESCRIPTION
Taking traditional Java server-side applications to the multi-tenant Cloud introduces lots of challenges. In this session, we will share our experience of creating a SaaS offering, which is currently being used successfully by the Java community. We will start by reviewing the challenges we faced during the SaaS conversion. Next, we will share our experience with the EC2 platform. We will discuss the importance of automation and how we use tools like Chef and Puppet for SaaS provisioning. Finally, we will describe how creating a SaaS version of our product shifted our way of thinking about software release. We will recommend what’s required to successfully release both SaaS and downloadable versions of your product.TRANSCRIPT
HOW WE TOOK OUR SERVER-SIDE APPLICATION TO THE CLOUD……and liked what we got
Where we started What we did Paradigm shift
2
Agenda
BACKGROUND
4
Fred Simon @freddy33 Chief Architect of JFrog Also Head of DevOps Co-author of Artifactory
Who’s talking?
5
Since 2006 Binary Repository Manager Open Source Standard Java Web Application
> Spring> Spring Security> Wicket> JCR> Jersey
Artifactory what?
6
Yet Another Java Server App
7
Community success JFrog established in 2008 JavaOne 2009
>Pro version>Software as a Service (SaaS version)
› Heavly used by OSS community
Moving Forward
That’s what this session is about!
8
Benefits for the user:>Zero maintenance
› Backups› Updates› Infrastructure
>Support Drawbacks for the user:
>Can’t install user plugins
Artifactory SaaS
9
etc.
Artifactory SaaS
10
3 product flavors in production>Challenging support tasks>Completing, not competing
30 million requests/week>Mostly in the Pacific Time Monday mornings
4 TB of artifacts
Moving Forward
11
A load of Artifacts!
JOURNEY TO THE CLOUD
THE *AAS BUZZ
* As A Service
Self Service Multi-tenant
15
Product Self Service
Multi-tenant
* aaS
Gmail Not *aaS, web-app
Google Apps SaaS
Google App Engine PaaS
Google Compute Engine IaaS
GaaE: Google as an Example
16
Product Self Service
Multi-tenant
* aaS
Amazon store Not *aaS, web-app
aStore SaaS
Amazon Elastic Beanstalk PaaS
Amazon Elastic Cloud IaaS
AaaE: Amazon as an Example
17
Multi-tenancy Platform selection
>PaaS or IaaS DB schema updates
The SaaS Pains - Overview
18
Java 9 already! Until then select one of the
following:>Use single application, separate data>Use separated applications>Use separated processes
Multi-tenancy
19
GaaE for Multi Tenancy Types
Product Muli-tenancy Type
Google Apps Data Separation
Google App Engine Application Separation
Google Compute Engine Process Separation
20
Strategy Pros ConsSeparating data Normal Java Application Manual state
separation Complicated and
critical schemaSeparating application
No shared state Or is it?
Simple transition from existing
Stay tuned…
Separating processes No shared state Simple transition from
existing
JVM per tenant!
Comparing the Strategies
Strategy Pros ConsSeparating data Normal Java Application Manual state
separation Complicated and
critical schemaSeparating application
No shared state Or is it?
Simple transition from existing
Stay tuned…
Separating processes No shared state Simple transition from
existing
JVM per tenant!
Comparing the Strategies
21
Strategy Pros ConsSeparating data Normal Java Application Manual state
separation Complicated and
critical schemaSeparating application
No shared state Or is it?
Simple transition from existing
Stay tuned…
Separating processes No shared state Simple transition from
existing
JVM per tenant!
Comparing the Strategies
22
Strategy Pros ConsSeparating data Normal Java Application Manual state
separation Complicated and
critical schemaSeparating application
No shared state Or is it?
Simple transition from existing
Stay tuned…
Separating processes No shared state Simple transition from
existing
JVM per tenant!
Comparing the Strategies
P
23
24
Separate Application: Tomcat Root
┌── lib├── webapps│ ├── customer-name│ ├── other-customer-name│ └── many other customers└── other dirs (bin, conf, log, etc)
Tenant WARs
25
Using standard Java WAR classloader isolation Creating standard WAR with dependencies in
‘lib’, classes in ‘classes’ Creating per-user database Done!
Separate Application
26
Using standard Java WAR classloader isolation
Creating standard WAR with dependencies in ‘lib’, classes in ‘classes’
Creating per-user database Done! Perm Gen explodes after deploying 4 WARs
>Even when set to 1Gb!
Separate Application
27
The solution – move libraries to shared classloader>Both 3rd party and application>25 Mbs of JAR files
Not as easy as it sounds Review the source code of 3rd party libraries
one by one>Open Source FTW
The Quest for Shared Libraries
THE QUEST FOR SHARED LIBRARIES
29
Perfect Make sure the context is bounded to
thread/war
Spring Framework
30
public class AppCtxHolder implements ApplicationContextAware {private static ApplicationContext ctx;
public AppCtxHolder() { }
public void setApplicationContext(ApplicationContext applicationContext) {ctx = applicationContext;
}
public static ApplicationContext getApplicationContext() {return ctx;
} }
Spring Framework
31
public class AppCtxHolder implements ApplicationContextAware {private static ApplicationContext ctx;
public AppCtxHolder() { }
public void setApplicationContext(ApplicationContext applicationContext) {ctx = applicationContext;
}
public static ApplicationContext getApplicationContext() {return ctx;
} }
Spring Framework
32
public class AppCtxHolder implements ApplicationContextAware {private static ApplicationContext ctx;
public AppCtxHolder() { }
public void setApplicationContext(ApplicationContext applicationContext) {ctx = applicationContext;
}
public static ApplicationContext getApplicationContext() {return ctx;
} }
Spring Framework
33
Was good New versions added static map of
applications>Lesson learned: Recheck on each
version upgrade
Apache Wicket
34
They are not shared – good! Need global management to adjust
the size to match all the tenants>Without global thread pool
Thread Pools
35
Logger fields are (almost) always static>Can’t fight it>We MITM-ed the
LoggerFactory >Keeps Loggers per
application instead of static
Logger
36
Examples>Transaction ID>Tomcat ClassLoader
Solving race condition generates synchronous initialization
Race conditions on central locks
37
CATS Software
38
Heap-wide temp file cleaner thread >A static thread destroying the temp files
of the other tenants>Not as funny as it sounds
Jackrabbit
39
Lucene and other cache solutions are adjusted for the whole heap size
Use SoftReference based caches
Heap-wide Caches
THE QUEST FOR SHARED LIBRARIES: RESULTS
41
┌── lib├── webapps│ ├── customer-name│ │ ├── favicon.ico│ │ ├── META-INF│ │ └── WEB-INF│ │ ├── web.xml│ │ └── classes│ │ └── DUMMY.TXT│ ├── other-customer-name│ │ ├── favicon.ico│ │ │ └── META-INF│ │ └── WEB-INF│ └── many other customers└── other dirs (bin, conf, log, etc)
Tomcat Root – Where’s the JARs?
Tenant WAR
Look, ma!No jars, nor classes!
42
┌── lib│ ├── artifactory│ │ ├── artifactory-*.jar│ │ ├── jackrabbit-core-jfrog-2.2.8c.jar│ │ ├── spring-core-3.1.1.RELEASE.jar│ │ ├── wicket-core-1.5.3.jar│ │ └── other jars│ ├── catalina.jar│ ├── servlet-api.jar│ └── other jars├── webapps└── other dirs (bin, conf, log, etc)
Tomcat Root – In Global Lib!
Artifactory dependencies
Shared libs folder
Artifactory jars
Tenant WARs (without jars)
43
Custom Tomcat (our own jars in Tomcat’s lib)
Custom Machine Image with our customized software and configuration
Infrastructure as a Service
PaaS or IaaS?
44
Distributed software is concerned with DB upgrade procedures>User experience involved
Since we came from there, we have “Converters”>Our SaaS version got it for free
What was it about DB upgrade?
THE DEVOPS
46
Self Service Platform>Registration>Instance generation
› DB› Backup› Virtual Host› WAR
>Customer account management
Providing Self Service
47
Customer account is only in SaaS Centralized Self-Service portal Intercommunication with the provisioned
application of specific tenant>UI Branding>Virtual host control>Backups
Separating Concerns
48
4 TB of storage 33 TB of traffic per month Increasing overall performance
in an order of magnitude in the last 18 months
Outcome: DevOps
THE PROCESS
50
3 Versions 3 Deployment formats
>WAR>ZIP>RPM
Lots of dependencies
Diversity
51
We eat our own dog food
Artifactory is built with Artifactory
Recursive Slide (w. baby picture)
52
Snapshots are built, tested & deployed on your build server of choice>With Artifactory plugin
Artifactory “snapshot promotion” to sandbox
Right Tools for the Job
53
Deploying to production using Chef Other options:
>Puppet>Build Server plugins>ZT LiveRebel>Scripts
Right Tools for the Job
THE PARADIGM SHIFT
55
Before:1. Planning downloadable version release2. Converting it to SaaS version
Today:1. Continuously release to the cloud2. Once in a while “freezing” it to
downloadable version
Reversing the Cycle
56
>Rapid feedback>Host server access>Developers must think big>Standalone updates well tested by SaaS>Standalone application is kind of LTS
version
Consequences