how we took our server side application to the cloud and liked what we got

HOW WE TOOK OUR SERVER-SIDE APPLICATION TO THE CLOUD……and liked what we got

Where we started What we did Paradigm shift

2

Agenda

BACKGROUND

4

Fred Simon @freddy33 Chief Architect of JFrog Also Head of DevOps Co-author of Artifactory

Who’s talking?

5

Since 2006 Binary Repository Manager Open Source Standard Java Web Application

> Spring> Spring Security> Wicket> JCR> Jersey

Artifactory what?

6

Yet Another Java Server App

7

Community success JFrog established in 2008 JavaOne 2009

>Pro version>Software as a Service (SaaS version)

› Heavly used by OSS community

Moving Forward

That’s what this session is about!

8

Benefits for the user:>Zero maintenance

› Backups› Updates› Infrastructure

>Support Drawbacks for the user:

>Can’t install user plugins

Artifactory SaaS

9

etc.

Artifactory SaaS

10

3 product flavors in production>Challenging support tasks>Completing, not competing

30 million requests/week>Mostly in the Pacific Time Monday mornings

4 TB of artifacts

Moving Forward

11

A load of Artifacts!

JOURNEY TO THE CLOUD

THE *AAS BUZZ

* As A Service

Self Service Multi-tenant

15

Product Self Service

Multi-tenant

* aaS

Gmail Not *aaS, web-app

Google Apps SaaS

Google App Engine PaaS

Google Compute Engine IaaS

GaaE: Google as an Example

16

Product Self Service

Multi-tenant

* aaS

Amazon store Not *aaS, web-app

aStore SaaS

Amazon Elastic Beanstalk PaaS

Amazon Elastic Cloud IaaS

AaaE: Amazon as an Example

17

Multi-tenancy Platform selection

>PaaS or IaaS DB schema updates

The SaaS Pains - Overview

18

Java 9 already! Until then select one of the

following:>Use single application, separate data>Use separated applications>Use separated processes

Multi-tenancy

19

GaaE for Multi Tenancy Types

Product Muli-tenancy Type

Google Apps Data Separation

Google App Engine Application Separation

Google Compute Engine Process Separation

20

Strategy Pros ConsSeparating data Normal Java Application Manual state

separation Complicated and

critical schemaSeparating application

No shared state Or is it?

Simple transition from existing

Stay tuned…

Separating processes No shared state Simple transition from

existing

JVM per tenant!

Comparing the Strategies

Strategy Pros ConsSeparating data Normal Java Application Manual state

separation Complicated and

critical schemaSeparating application

No shared state Or is it?

Simple transition from existing

Stay tuned…

Separating processes No shared state Simple transition from

existing

JVM per tenant!

Comparing the Strategies

21

Strategy Pros ConsSeparating data Normal Java Application Manual state

separation Complicated and

critical schemaSeparating application

No shared state Or is it?

Simple transition from existing

Stay tuned…

Separating processes No shared state Simple transition from

existing

JVM per tenant!

Comparing the Strategies

22

Strategy Pros ConsSeparating data Normal Java Application Manual state

separation Complicated and

critical schemaSeparating application

No shared state Or is it?

Simple transition from existing

Stay tuned…

Separating processes No shared state Simple transition from

existing

JVM per tenant!

Comparing the Strategies

P

23

24

Separate Application: Tomcat Root

┌── lib├── webapps│ ├── customer-name│ ├── other-customer-name│ └── many other customers└── other dirs (bin, conf, log, etc)

Tenant WARs

25

Using standard Java WAR classloader isolation Creating standard WAR with dependencies in

‘lib’, classes in ‘classes’ Creating per-user database Done!

Separate Application

26

Using standard Java WAR classloader isolation

Creating standard WAR with dependencies in ‘lib’, classes in ‘classes’

Creating per-user database Done! Perm Gen explodes after deploying 4 WARs

>Even when set to 1Gb!

Separate Application

27

The solution – move libraries to shared classloader>Both 3rd party and application>25 Mbs of JAR files

Not as easy as it sounds Review the source code of 3rd party libraries

one by one>Open Source FTW

The Quest for Shared Libraries

THE QUEST FOR SHARED LIBRARIES

29

Perfect Make sure the context is bounded to

thread/war

Spring Framework

30

public class AppCtxHolder implements ApplicationContextAware {private static ApplicationContext ctx;

public AppCtxHolder() { }

public void setApplicationContext(ApplicationContext applicationContext) {ctx = applicationContext;

}

public static ApplicationContext getApplicationContext() {return ctx;

} }

Spring Framework

31

public class AppCtxHolder implements ApplicationContextAware {private static ApplicationContext ctx;

public AppCtxHolder() { }

public void setApplicationContext(ApplicationContext applicationContext) {ctx = applicationContext;

}

public static ApplicationContext getApplicationContext() {return ctx;

} }

Spring Framework

32

public class AppCtxHolder implements ApplicationContextAware {private static ApplicationContext ctx;

public AppCtxHolder() { }

public void setApplicationContext(ApplicationContext applicationContext) {ctx = applicationContext;

}

public static ApplicationContext getApplicationContext() {return ctx;

} }

Spring Framework

33

Was good New versions added static map of

applications>Lesson learned: Recheck on each

version upgrade

Apache Wicket

34

They are not shared – good! Need global management to adjust

the size to match all the tenants>Without global thread pool

Thread Pools

35

Logger fields are (almost) always static>Can’t fight it>We MITM-ed the

LoggerFactory >Keeps Loggers per

application instead of static

Logger

36

Examples>Transaction ID>Tomcat ClassLoader

Solving race condition generates synchronous initialization

Race conditions on central locks

37

CATS Software

38

Heap-wide temp file cleaner thread >A static thread destroying the temp files

of the other tenants>Not as funny as it sounds

Jackrabbit

39

Lucene and other cache solutions are adjusted for the whole heap size

Use SoftReference based caches

Heap-wide Caches

THE QUEST FOR SHARED LIBRARIES: RESULTS

41

┌── lib├── webapps│ ├── customer-name│ │ ├── favicon.ico│ │ ├── META-INF│ │ └── WEB-INF│ │ ├── web.xml│ │ └── classes│ │ └── DUMMY.TXT│ ├── other-customer-name│ │ ├── favicon.ico│ │ │ └── META-INF│ │ └── WEB-INF│ └── many other customers└── other dirs (bin, conf, log, etc)

Tomcat Root – Where’s the JARs?

Tenant WAR

Look, ma!No jars, nor classes!

42

┌── lib│ ├── artifactory│ │ ├── artifactory-*.jar│ │ ├── jackrabbit-core-jfrog-2.2.8c.jar│ │ ├── spring-core-3.1.1.RELEASE.jar│ │ ├── wicket-core-1.5.3.jar│ │ └── other jars│ ├── catalina.jar│ ├── servlet-api.jar│ └── other jars├── webapps└── other dirs (bin, conf, log, etc)

Tomcat Root – In Global Lib!

Artifactory dependencies

Shared libs folder

Artifactory jars

Tenant WARs (without jars)

43

Custom Tomcat (our own jars in Tomcat’s lib)

Custom Machine Image with our customized software and configuration

Infrastructure as a Service

PaaS or IaaS?

44

Distributed software is concerned with DB upgrade procedures>User experience involved

Since we came from there, we have “Converters”>Our SaaS version got it for free

What was it about DB upgrade?

THE DEVOPS

46

Self Service Platform>Registration>Instance generation

› DB› Backup› Virtual Host› WAR

>Customer account management

Providing Self Service

47

Customer account is only in SaaS Centralized Self-Service portal Intercommunication with the provisioned

application of specific tenant>UI Branding>Virtual host control>Backups

Separating Concerns

48

4 TB of storage 33 TB of traffic per month Increasing overall performance

in an order of magnitude in the last 18 months

Outcome: DevOps

THE PROCESS

50

3 Versions 3 Deployment formats

>WAR>ZIP>RPM

Lots of dependencies

Diversity

51

We eat our own dog food

Artifactory is built with Artifactory

Recursive Slide (w. baby picture)

52

Snapshots are built, tested & deployed on your build server of choice>With Artifactory plugin

Artifactory “snapshot promotion” to sandbox

Right Tools for the Job

53

Deploying to production using Chef Other options:

>Puppet>Build Server plugins>ZT LiveRebel>Scripts

Right Tools for the Job

THE PARADIGM SHIFT

55

Before:1. Planning downloadable version release2. Converting it to SaaS version

Today:1. Continuously release to the cloud2. Once in a while “freezing” it to

downloadable version

Reversing the Cycle

56

>Rapid feedback>Host server access>Developers must think big>Standalone updates well tested by SaaS>Standalone application is kind of LTS

version

Consequences