softshake 2013 proud to be a polyglot

Monday, October 28, 13

Polyglot Applica-on Development

Tugdual “Tug” Grall

Technical Evangelist

@[email protected]


mailto:[email protected]


Modern Applica-ons

• Modern Applica-ons Must Manage any type of data Scale Be fault tolerant Adapt to change


Standard Architecture

• Mono-‐Technology Same “language/architecture” Single Database : RDBMS

• One size fits all approachJSP/JSFServicesMessaging

JTADAO/JPA


Is it good for me?

• Many files XML, Sources, ..

• Does Hibernate/JPA is good for all? Impedance mismatch

• Need to scale from 0 to 50millions users in 6 weeks (real life) can I do that easily with such architecture?

• How to add new features easily? code, schema change, ... and all this for yesterday!

• Lot of code


One size does NOT fit all!


Look at our Data....


0

0.50

1.00

1.50

2.00

2011 2006 2000

Source: IDC 2011 Digital Universe Study (h<p://www.emc.com/collateral/demos/microsites/emc-‐digital-‐universe-‐2011/index.htm)

Trillions of G

igabytes (Ze<

abytes)

Big DataHigh Data Variety and Velocity

Unstructured and Semi-‐Structured Data

Structured Data

Text, Log Files, Click Streams, Blogs, Tweets, Audio, Video, etc.

More Flexible Data Model Required


50 Million Users in 50 Days

191715131197533/12826242220181614121082/6

Draw Something by OMGPOPDaily Ac)ve Users (millions)

21

2

4

6

8

10

12

14

16


RDBMS is good for many thing, but hard to scale

RDBMS Scales UpGet a bigger, more complex server

Users

ApplicaIon Scales OutJust add more commodity web servers

Users

System CostApplicaTon Performance

Rela:onal Database

Web/App Server Tier


Won’t scale beyond this point

How do you take this growth?


Scaling out fla?ens the cost and performance curves

NoSQL Database Scales OutCost and performance mirrors app Ier

Users

NoSQL Distributed Data Store

Web/App Server Tier

ApplicaIon Scales OutJust add more commodity web servers

Users


ApplicaTon Performance System Cost

NoSQL Technology Scales Out


ClouderaHortonworks

Mapr

OperaTonal vs. AnalyTc Databases

CouchbaseMongoDB

CassandraHbase

AnalyFcDatabases

Get insights from data

Real-‐Fme, InteracFve Databases

Fast access to data

NoSQL


Polyglot Persistence

Document & K/V

• Products• User Profiles• Game AcTons• Sessions• Shopping Cart

RDBMS

• Financial Data• ReporTng

Big Data/Analysis

• Log Capture• RecommendaTons• AdCampaign

Indexing

•Full Text Search•Ad hoc queries


Developer New Skills

• Understand the pros/cons of each solu-ons Programming API How to access the data? (API, Query Languages)

• Learn how to design data-‐model De-‐normalizing and duplicate data is not a problem

• Integrate each solu-on to your applica-on

• Note: This is something you already do ! OperaTonal RDBMS and Data Warehouse


Data Service Layer

Data Service

• Wrap datastore in Services loosely coupled


Use Case : Content Driven Site


Use Case: Mobile Services

PIM Database

-‐ Legacy Applica:on-‐ Product Informa:on

NoSQL

-‐ Product Data-‐ Addi:onal Metada

Web/App Server Tier Web/App Server Tier


Now what?


Why mul-ple languages?

• Some-mes we have no choice HTML/CSS/JavaScript + Server Side Mobile NaTve ApplicaTons

• Java/.Net have not evolved that must these past years Java EE 5.0 is sTll mainstream (2006)

• Many languages and frameworks Scala, Clojure, Groovy, Dart, Go, Ruby, Python, F#, Erlang Node.js, Rails, Play!, Grails, ...


Examples

• Real-me interac-on with WebSockets : Wait for Java EE 7 to be out? Hack your app server and applicaTon with ConTnuaTon ? Use Node.js and Socket.io ?

• Data Collec-on and Treatment? Find a library? Create your library? Wait for Java 8 and Lambdas? Use Scala? ... or other


Examples

• Simple CRUD Applica-on Use the JavaEE stack with JPA and JSF? Use Spring ? Use tools like Play!, Grails, Rails ?


How to chose?

• Maturity/Supportability

• Features Set

• Learning Curve

• Produc-vity


Polyglot Programming in Ac-on

• Build a distributed database Manage distributed processes Save and retrieve data on disk Cache the data on RAM Build a query engine


Polyglot Programming in Ac-on

• Manage Distributed Process Erland and OTP (Open Telecom Plaform) has been built for that Taking this will bootstrap a project quickly instead of implemenTng in

another language

• Data access and Caching C/C++ is the best/fastest way to interact with I/O Leverage exisTng Caching layer, for example Memcached wrigen in C

• Query & Indexing Easy to code for developer : Let’s use JavaScript


Couchbase Server Architecture

ReplicaTon, Rebalance, Shard State Manager

REST management API/Web UI

8091Admin Console

Erlang /O

TP

11210 / 11211Data access ports

Object-‐managedCache

Storage Engine

8092Query API

Que

ry Engine h\p

Data Manager Cluster Manager

Cluster ManagerWriOen in Erlang

Data ManagerWriOen in C/C++


Use Case: Second Screen Gaming

Java/Play Applica:onPython scrips

Administra-on

Node.js + Socket IO


Conclusion

• Use the good tool for the good task Do not try to “hack” something when you have a shorter soluTon

Think about your users first... so deliver and get feedback

• Learning is part of our job ! And this will help you and your project

• What about maintenance? Do you think it is easier to maintain complex code?


Q&A

@[email protected]




softshake 2013 proud to be a polyglot

Technology

development tugdualtuggrall

manyles xml