proud to be polyglot!
DESCRIPTION
TRANSCRIPT
Proud to be Polygot!Tugdual “Tug” Grall
@tgrall
Technical Evangelist at .
Wednesday, November 6, 13
{about : “me”}
•Tugdual “Tug” Grall
Couchbase
Technical Evangelist
eXo
CTO
Oracle
Developer/Product Manager
Mainly Java/SOA
Developer in consulting firms
• Web
@tgrall
http://blog.grallandco.com
tgrall
NantesJUG co-founder
Pet Project :
http://www.resultri.com
Wednesday, November 6, 13
Modern Applications•Modern Applications Must
Manage any type of data
Scale
Be fault tolerant
Adapt to change
Wednesday, November 6, 13
Standard Architecture•Mono-Technology
Same “language/architecture”
Single Database : RDBMS
•One size fits all approach
JSP/JSFServices
MessagingJTA
DAO/JPA
Wednesday, November 6, 13
Is it good for me?•Many files
-XML, Sources, ..
• Hibernate/JPA is good for all?
- Impedance mismatch
•Need to scale
- from 0 to 50millions users in 6 weeks (real life)
- can I do that easily with such architecture?
• How to add new features easily?
- code, schema change, ...
- and all this for yesterday!
• Lot of code
Wednesday, November 6, 13
One size does NOT fit all!
Wednesday, November 6, 13
Look at our Data....
Wednesday, November 6, 13
0
0.50
1.00
1.50
2.00
2006 2009 2012Source: IDC
Trillions of G
igabytes (Ze:
abytes)
Big Data
Unstructured and Semi-‐Structured Data
Structured Data
Text, Log Files, Click Streams, Blogs,
Tweets, Audio, Video, etc.
High Data Variety and Velocity
More Flexible Data Model Required
Wednesday, November 6, 13
50 Million Users in 50 Days
191715131197533/12826242220181614121082/6
Draw Something by OMGPOPDaily Ac)ve Users (millions)
21
2
4
6
8
10
12
14
16
Wednesday, November 6, 13
How do you take this growth?
RDBMS Scales UpGet a bigger, more complex server
Users
ApplicaXon Scales OutJust add more commodity web servers
Users
System CostApplicaMon Performance
RelaXonal Database
Web/App Server Tier
System CostApplicaMon Performance
Won’t scale beyond this point
Wednesday, November 6, 13
NoSQL Database Scales OutCost and performance mirrors app Xer
Users
NoSQL Distributed Data Store
Web/App Server Tier
ApplicaXon Scales OutJust add more commodity web servers
Users
System CostApplicaMon Performance
ApplicaMon Performance System Cost
NoSQL Technology Scales Out
Wednesday, November 6, 13
ClouderaHortonworks
Mapr
CouchbaseMongoDB
AnalyXcDatabases
Get insights from data
Real-‐Xme, InteracXve Databases
Fast access to data
NoSQL
OperaMonal vs. AnalyMc Databases
Wednesday, November 6, 13
Polyglot Persistence
Document & K/V
• Products• User Profiles• Game AcMons• Sessions• Shopping Cart
RDBMS
• Financial Data• ReporMng
Big Data/Analysis
• Log Capture• RecommendaMons• AdCampaign
Indexing
•Full Text Search•Ad hoc queries
Wednesday, November 6, 13
Developer New Skills
• Understand the pros/cons of each solutions
Programming API
How to access the data? (API, Query Languages)
• Learn how to design data-model
De-normalizing and duplicate data is not a problem
• Integrate each solution to your application
Wednesday, November 6, 13
Data Service Layer
Data Service
•Wrap datastore in Services
- Loosely coupled
Wednesday, November 6, 13
Use Case : Content Driven Site
Wednesday, November 6, 13
Use Case: Mobile Services
PIM Database
-‐ Legacy ApplicaXon-‐ Product InformaXon
NoSQL
-‐ Product Data-‐ AddiXonal Metada
Web/App Server Tier Web/App Server Tier
Rabbit MQ
Wednesday, November 6, 13
Now what?
Wednesday, November 6, 13
Why multiple languages?• Sometimes we have no choice
HTML/CSS/JavaScript + Server Side
Mobile Native Applications
• Java/.Net have not evolved that must these past years
Java EE 5.0 is still mainstream (2006)
• Many languages and frameworks
Scala, Clojure, Groovy, Dart, Go, Ruby, Python, F#, Erlang
Node.js, Rails, Play!, Grails, ...
Wednesday, November 6, 13
Examples• Realtime interaction with WebSockets
- Wait for Java EE 7 to be out?
- Hack your app server and application with Continuation ?
- Use Node.js and Socket.io ?
• Data Collection and Treatment?
- Find a library?
- Create your library?
- Wait for Java 8 and Lambdas?
- Use Scala? ... or other
Wednesday, November 6, 13
Examples
• Simple CRUD Application
Use the JavaEE stack with JPA and JSF?
Use Spring ?
Use tools like Play!, Grails, Rails ?
Wednesday, November 6, 13
How to chose?
•Maturity/Supportability
• Features Set
• Learning Curve
• Productivity
Wednesday, November 6, 13
Polyglot Programming in Action
• Build a distributed database
Manage distributed processes
Save and retrieve data on disk
Cache the data on RAM
Build a query engine
Wednesday, November 6, 13
Polyglot Programming in Action• Manage Distributed Process
Erlang and OTP (Open Telecom Platform) has been built for that
Taking this will bootstrap a project quickly instead of implementing in another language
• Data access and Caching
C/C++ is the best/fastest way to interact with I/O
Leverage existing Caching layer, for example Memcached written in C
• Query & Indexing
Easy to code for developer : Let’s use JavaScript
Wednesday, November 6, 13
ReplicaMon, Rebalance, Shard State Manager
REST management API/Web UI
8091Admin Console
Erlang /O
TP
11210 / 11211Data access ports
Object-‐managedCache
Storage Engine
8092Query API
Que
ry Engine hbp
Data Manager Cluster Manager
Cluster ManagerWriben in Erlang
Data ManagerWriben in C/C++
Couchbase Server
Wednesday, November 6, 13
Use Case: Second Screen Gaming
Java/Play ApplicaXonPython scrips
AdministraXon
Node.js + Socket IO
Users
Wednesday, November 6, 13
Conclusion• Use the good tool for the good task
Do not try to “hack” something when you have a shorter solution
Think about your users first... so deliver and get feedback
• Learning is part of our job !
And this will help you and your project
•Operations / DevOps
Think about it from day one... as a developer
•What about maintenance?
Do you think it is easier to maintain complex code?
Wednesday, November 6, 13