internet apps powered by nosql and javascript

(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 1www.orientechnologies.com

Luca Garulli – Founder and CEO @NuvolaBase LtdAuthor of OrientDB Doc/Graph DB

Nov 17, 2012 – Paris, France

A new collaborative way to developInternet Apps powered by NoSQL

and Javascript

(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 2

Agenda

(1) Developing Web App today

(2) Goal: reduce complexityand improve performance

(3) The Next Generation Web Apps

(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 3

Can you remember when developingApplications was simpler?

WebApp

ClientClient Client

DB

Few clients, usually running inside a Intranet

(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 4

Can you remember when developingApplications was simpler?

WebApp

ClientClient Client

DB

Pages were always rendered

at the server side

(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 5

Cache

Why today are more similar to this?

ClientClient ClientClientClient ClientClientClient ClientClientClient Client

ClientClientClientClient

DB DB

WebApp 2 WebApp 3 Batch tasksBatch tasksWebApp 1

Balancer

Batch tasksBatch tasksBatch tasksBatch tasksBatch tasksBatch tasks

(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 6

Mainly because the Internet

100x clients!From few tens of

concurrent clients tohundreds or more

(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 7

Mainly because the Internet

Response Times:Network connection is not anymore

the bottleneck,users now expect response

times less than 1 second per operation

(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 8

Mainly because the Internet

Big Data:databases from

thousands of recordsto millions or even billions

(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 9

Cache

How many languages are you using?

ClientClient ClientClientClient ClientClientClient ClientClientClient Client

ClientClientClientClient

DB DB

WebApp 2 WebApp 3WebApp 1

Balancer

Batch tasksBatch tasksBatch tasksBatch tasksBatch tasksBatch tasksBatch tasksBatch tasks

(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 10

Cache

How many languages are you using?

ClientClient ClientClientClient ClientClientClient ClientClientClient Client

ClientClientClientClient

DB DB

Javascript! (Coffescript?)

WebApp 2 WebApp 3WebApp 1

Balancer

Java? PHP? .Net?

Ruby? Python?

Oracle PL/SQL?

Batch tasksBatch tasksBatch tasksBatch tasksBatch tasksBatch tasksBatch tasksBatch tasks

Bash? Perl? ANT?

(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 11

Managing complex architecture isvery costly because:

-(1) requires very skilled people,-specially if multiple technologies

-are involved

-(2) huge time for development-and testing

(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 12

and...........................

(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 13

Developing WebAppsSometimes the tuning takesmore time than developing*!

*Blood Bath

(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 14

Do we really need this levelof complexity to scale up?

(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 15

Main features

(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 16

OrientDB, the multi-modelNoSQL = {

flexibility of Document databases+ complexity of the Graph model

+ Object Oriented concepts}

(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 17

+14 yearsof research

(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 18

+4 yearsof design and development

(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 19

Ø configdownload, unzip, run!

cut & paste the db directory

(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 20

150,000records per second

(flat records, no index, on commodity hw)

(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 21

Schema-lessschema is not mandatory, relaxed model,collect heterogeneous documents all together

(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 22

Schema-fullschema with constraints on fields and validation rules

Customer.age > 17Customer.address not null

Customer.surname is mandatoryCustomer.email matches '\b[A-Z0-9._%+-]+@[A-Z0-9.-]+\.[A-Z]{2,4}\b'

(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 23

Schema-mixedschema with mandatory and optional fields + constraints

the best of schema-less and schema-full modes

(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 24

ACID Transactionsdb.begin();try{ // your code ... db.commit();

} catch( Exception e ) { db.rollback();}

(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 25

Complex typesnative support for collections, maps (key/value)

and embedded documentsno more additional tables to handle them

(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 26

SQLselect * from employee where name like '%Jay%' and status=0

(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 27

SQL & relationshipsselect from Account where address.city.country.name = 'Italy'

select from Account where addresses contains (city.country.name = 'Italy')

(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 28

SQL & trees/graphsselect out[label='friend'].in from V where name = 'Luca' and surname = 'Garulli'

select out[@class='knows'].in from V where name = 'Jay' and surname = 'Miner'

traverse friends from Profile while $depth < 7

(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 29

SQL sub queriesselect from ( traverse friends from Profile where $depth <= 7 ) where home.city.name = ‘Paris’

(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 30

SQL & stringsselect from Profile where name.toUpperCase() = 'LUCA'

select from City where country.name.substring(1,3).toUpperCase() = 'TAL'

select from Agenda where phones contains ( number.indexOf( '+39' ) > -1 )

select from Agenda where email matches '\bA-Z0-9._%+-?+@A-Z0-9.-?+\.A-Z?{2,4}\b'

(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 31

SQL & schema-lessselect from Profile where any() like '%Jay%'

select from Stock where all() is not null

(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 32

SQL & collectionsselect from Tree where children contains ( married = true )

select from Tree where children containsAll ( married = true )

select from User where roles containsKey 'shutdown'

select from Graph where edges.size() > 0

(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 33

Java®runs everywhere is available JRE1.6+

robust engine

(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 34

Language bindingsJava as native

JRuby, PHP, C, C++, Scala, .NET,Ruby, Clojure, Node.js,

Python, Javascript and more!

(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 35

Born for the Internet

Supports natively HTTP/RESTful protocolDocuments are transferred in JSON

(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 36

MVRB-Tree index

the best of B+Tree and RB-Treefast on browsing, low insertion cost

it's a new algorithm!

(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 37

Securityusers and roles, encrypted passwords

fine grain privileges(similar to what RDBMSs offer)

(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 38

CacheYou can avoid using 3°party caches

like Memcached

2 Levels of cache:Level1: Database level, 1 per thread

Level2: Storage level, 1 per JVM

(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 39

OGraphVertex (V)

PersonAddress : Address

Inheritance

CustomertotSold : float

ProvidertotBuyed : float

Vehiclebrand : BRANDS

(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 40

OGraphVertex (V)

PersonAddress : Address

Polymorphic SQL Query

CustomertotSold : float

ProvidertotBuyed : float

Vehiclebrand : BRANDS select * from Person

where city.name = 'Rome‘

Queries are polymorphicsand subclasses of Person can be

part of result set

(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 41

“A graph database is anystorage system

that providesindex-free adjacency”

- Marko Rodriguez(author of TinkerPop Blueprints and Gremlin language)

(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 42

GraphDB handles relationships as aphysical LINK to the record

assigned when the relationship is created

on the other side

RDBMS computes therelationship every time you query a

database!

(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 43

This means jumping from aO(log N) algorithm to a near O(1)

traversing cost is not more affectedby database size!

This is huge in the BigData age

(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 44

OrientDB in the Blueprints micro-benchmark,on common hw, with a hot cache,

traverses 29,6 Millionsof records in less than 5 seconds

about 6 Millions of nodes traversed per sec!

*unless you live in the Google’s server farm

Do not try this at home with any

RDBMS*!

(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 45

Now let’s start modellingA Web App using

OrientDB

(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 46

Customername: String

surname: String

Ordertotal : decimal

Addressstreet : String

Cityname : String

country : String

Our simple domain

(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 47

Customername: String

surname: String

Ordertotal : decimal

Addressstreet : String

Cityname : String

country : String

Our simple domain

Relationships in OrientDB are „links“

The link between two or more documents is

created at the document save time

(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 48

Client

HTTP/RESTful (+ JSON)

JSON

http://localhost:2480/jsdemo/command/sql/select * from Customer

{ ‘result’ : [ { '@rid' = ‘#8:0’, '@class' = ‘Customer', 'name' : 'Luca', 'surname' : 'Garulli', ‘orders’ : [ ‘#10:33’, ‘#10:232’ ], ‘address’ : { ‘@class’ : ‘Address’, ‘street’ : ‘Via del Corso 15’, ‘city’ : ‘#9:0’ } } ]}

(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 49

Document‘s uniqueRID (Record ID)

Assigned at creation time and never

changes!

{ '@rid' = ‘#8:0’, '@class' = ‘Customer', 'name' : 'Luca', 'surname' : 'Garulli', ‘orders’ : [ ‘#10:33’, ‘#10:232’ ], ‘address’ : { ‘@class’ : ‘Address’, ‘street’ : ‘Via del Corso 15’, ‘city’ : ‘#9:0’ }}

(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 50

One-to-Many relationship as a array

of linksThe link is the RID of the

target document(s)

{ '@rid' = ‘#8:0’, '@class' = ‘Customer', 'name' : 'Luca', 'surname' : 'Garulli', ‘orders’ : [ ‘#10:33’, ‘#10:232’ ], ‘address’ : { ‘@class’ : ‘Address’, ‘street’ : ‘Via del Corso 15’, ‘city’ : ‘#9:0’ }}

(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 51

Embedded documentis saved inside the owner document

Has no own identity

{ '@rid' = ‘#8:0’, '@class' = ‘Customer', 'name' : 'Luca', 'surname' : 'Garulli', ‘orders’ : [ ‘#10:33’, ‘#10:232’ ], ‘address’ : { ‘@class’ : ‘Address’, ‘street’ : ‘Via del Corso 15’, ‘city’ : ‘#9:0’ }}

(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 52

Create a new document of class «Customer»

>curl --basic -u admin:admin localhost:2480/document/jsdemo -d '{"@class":"Customer","name":“Luca","surname":“Garulli"}'#8:0

Retrieve the document just created

>curl --basic -u admin:admin localhost:2480/document/jsdemo/8:0{ "@type": "d", "@rid": "#8:0", "@version": 0, "@class": "Customer", "name": "Luca", "surname": "Garulli"}

(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 53

Since OrientDB talks nativelyHTTP and JSON

Is it too insane using it directly from the Web Pages bypassing any servers-side

layer?

(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 54

Pros

+ concurrency and response time:It’s super fast, all happens at the

client side

+ development time having justone layer to develop and maintain

(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 55

Javascript driver example I

<script src="js/orientdb-api.js" type="text/javascript"></script><script>var db = new ODatabase(‘http://localhost:2480/jsdemo’);var info = db.open(‘admin’, ‘admin’);

var customer = { “@class” : “Customer”, “name” : “Luca”, “surname“ : “Garulli” };

var rid = db.save( customer );

// reload the customer backvar customer = db.load(‘8:0’);</script>

(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 56

Javascript driver example IIDisplay in a input field the total of orders

var date = $(‘#date’).val(); // jQuery

var resultset = db.query( ‘select sum(total) as total from Order where date > ?’, [ date ] );

$(‘#total’).val( resultset.result[0].total ); // jQuery

(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 57

Cons

- Client-side JS can be easily hacked

- Sometimes it’s not a good ideawriting queries in the web pagesrevealing your database schema

to the users

(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 58

Security

To improve security we supportedRecord-Level access control

Even if the JSON is hacked at the clientside, the user can

read/write own records only

Can be enabled only for some classes of documents

(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 59

Record Level Security

Business DocumentsClass «Doc»

A B

> insert into Doc (name) values (‘A’)> select * from Docresult: A

> insert into Doc (name) values (‘B’)> select * from Docresult: B

(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 60

Server-Side functions

Let to OrientDB executing Javascriptat the server side. The code of thefunction is stored in the database

Similar to the «old» Stored-Procedures,but written using the Internet Language:

Javascript

(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 61

Server-Side Functions can access to specialobjects in context:

db, as the current database instance. Thisobject is like the JS driver API to make easy

moving code between client and serverrequest, as the HTTP request

response, as the HTTP responseutil, as utility helper class

(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 62

function user_create( name, roleName ) { var role = db.query("select from ORole where name = ?", roleName ); if( role == null ){ response.send(404, "Role name not found", "text/plain", "Error: role name not found" ); } else { db.begin(); try{ var result = db.save({ "@class" : "OUser", name : "Luca", password : "Luc4", roles : role}); db.commit(); return result; } catch ( err ) { db.rollback(); response.send(500, "Error on creating new user", "text/plain", err.toString() ); }}

(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 63

Server-Side Functions are automaticallypublished and reachable via HTTP REST.

Example:http://localhost:2480/function/jsdemo/sum/3/20

Server-Side Functions must declare if areIdempotent. Non-idempotent functions

can be only be called using thePOST HTTP method

(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 64

Demo, please

(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 65

OrientDB uses Mozilla Rhino as JS engine

Why Rhino?

+ Rhino allows execution of Java code,very useful to access to tons of libraries

+ Financial and BI apps allows usingthe java.math.BigDecimal class

+ Mature technology+ Fast, (even if probably not the fastest?)

(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 66

Fetch plansChoose what to fetch on query and document loading

Optimizes network latencyLoad many records in one shot!

(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 67

Load root + all the ordersfetchPlan = “*:1 orders:1”

Fetch plansCustomer Luca | | address city +-------> Address -----> City | 10th street Italy | orders +--------->* [Order Order ] [Order1 Order2]

(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 68

{ '@rid' = ‘#8:0’, '@class' = ‘Customer', 'name' : 'Luca', 'surname' : 'Garulli', ‘orders’ : [ { ‘@class’ : ‘Order’, ‘@rid’: ‘#10:33’, ‘total’ : 12000.50 } , { ‘@class’ : ‘Order’, ‘@rid’: ‘#10:232’, ‘total’ : 788.33 } ], ‘address’ : { ‘@class’ : ‘Address’, ‘street’ : ‘Via del Corso 15’, ‘city’ : ‘#9:0’ }}

Fetched linked documents

(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 69

Future plans

Support debugging via Studio

«Live Objects» by providing a true ObjectOriented interface to persistent

documents using JS to implement methods

Try Google V8 in place of Rhino

(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 70

Always Free

Open Source Apache 2 licensefree for any purposes,

even commercials

(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 71

www.NuvolaBase.com

Professional supportfor production

systems

TrainingCourses andcertification

DB as serviceon the cloud

always available

The OrientDB company

(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 72

Network of partners

To provide Professional Services through NuvolaBase Ltd:support, training, consulting, mentoring

Want to be part of the network as partner?

contact info@nuvolabase.com

(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 73

The OrientDB Academy

Course + Certification in

all the worldas soon as we reach

20 attendees!

Want it in your city?Send an email to:

info@nuvolabase.com

(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 74

OrientDBfor Java developers

8 hours

OrientDBMaster Development

14 hours

OrientDBfor SOA

6 hours

OrientDBand the power of graphs

6 hours

OrientDBfor DBA

6 hours

OrientBOXfor Web Developers

6 hours

Courses

(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 75

“OrientDB in Action”book

by Manning Publicationsis coming...

(begin of 2013)

(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 76

The VisionThe next generation Web Applications are

going to be written in JS that talkin HTTP/RESTful protocol against

Browser and multi-devices Smart Phones andTablets where the Business Logicand Data are totally integrated

to achieve the maximum of the speed,scalability and easy of deployment

(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 77

The Vision

WebApp

MobileWWW Tablet

Business Logic (JS)

Database (documents)

HTTP RESTful

(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 78

Summary

(1) Reduce complexity(2) Improve performance removing

layers(3) Use 1 (one) language for all: JS

(c) Luca Garulli Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License Page 79

Q&A

Luca Garulli

www.twitter.com/lgarulli

CEO atAuthor of

Document-Graph NoSQLOpen Source project Ltd, London UK