developing for node.js with mysql and nosql
DESCRIPTION
Introducing mysql-js, an easy-to-use JavaScript API for MySQL and MySQL Cluster.TRANSCRIPT
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Insert Picture Here
1
Developing for Node.JS with MySQL and NoSQL
J.D. Duncan, [email protected] Craig Russell, [email protected]
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.2
MySQL Cluster Architecture
MySQL Cluster Data Nodes
Data Layer
Clients
Application Layer
Management
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.3
MySQL Cluster Architecture
MySQL Cluster Data Nodes
Data Layer
Clients
Application Layer
ManagementManagement
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.4
MySQL Cluster Architecture
MySQL Cluster Data Nodes
Data Layer
Application Layer
ManagementManagement
Clients
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.5
mysql-js: a MySQL connector for Node.JS
Included in MySQL Cluster 7.3 in storage/ndb/nodejs
Also at github: https://github.com/mysql/mysql-js
One JavaScript API with two back-end adapters The MySQL back end uses node-mysql (Felix Geisendorfer's all-
JavaScript MySQL connector) The NDB (MySQL Cluster) back end uses native NDBAPI
Twitter-like demo application in samples/tweet/
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.6
What is Node.JS
A server-side web application platform Single-threaded* and event-driven Application code is written in JavaScript JavaScript code runs in a very fast VM Node.JS includes a useful set of standard libraries
* One JavaScript thread; a pool of background worker threads perform blocking I/O.
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.7
Why Node.JS
Servers like lighttpd, nginx, and tornado can solve the C10K problem for static or cached content
Node.JS can solve the C10K problem for a dynamic web application– ... but what about 10,000 back-end database connections?
mysql-js with NDB aims to solve the C10K problem for a dynamic and database-driven web application
The "C10K" problem: how to handle 10,000+ client connections
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.8
Sample CodeNode HTTP Server
function handleRequest(request, response) { var page = getContent(request.url); response.statusCode = page ? 200 : 404; response.write(page); response.end();}
function startWebServer() { http.createServer(handleRequest).listen(8080); console.log("Server started");}
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.9
Sample Code with database accessNode HTTP Server and mysql-jsfunction dbConnected(connectionError, sessionFactory) { function handleRequest(request, response) { function onDbResponse(err, data) { var page = renderPage(data); response.write(page); response.end(); } function onDbSession(err, session) { runDbOperation(session, request, onDbResponse); } sessionFactory.openSession(null, onDbSession); } http.createServer(handleRequest).listen(8080);}nosql.connect(... , dbConnected);
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
User's View
10
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Twitter-like Demo Application
11
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Twitter-like Demo Application
12
Create user Find user by name Delete user by name
(cascading) Create a tweet Find tweet by id Delete tweet by id (cascading) Make User A a follower of
User B List followers of user
List who user is following Get the most recent tweets Get the 20 latest tweets by a
user Get the 20 latest tweets
@mentioning a user Get the 20 latest tweets
containing a #hashtag
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.13
JavaScript Constructor
function Tweet(author, message) { if(author !== undefined) { this.date_created = new Date(); this.author = author; this.message = message; }}
Creates an instance of a Domain Object
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.14
SQL Table
CREATE TABLE tweet ( id bigint unsigned auto_increment not null, author varchar(20), message varchar(140), date_created timestamp(2), ...)
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.15
Mapping
var nosql = require("../mysql-js");
// Map SQL Tables to JS Constructorsvar mappings = [];mappings.push( new nosql.TableMapping('tweet'). applyToClass(Tweet));
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.16
Explicit Mappings are not required
All operations take either a constructor or table name If table name is used, default mapping is created Default mapping maps all columns to JS properties, where the JS type
of each property is the default for its corresponding SQL Data Type
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.17
connect function
connect is asynchronous First parameter is object with connection properties Second parameter is array of table names or constructors
– Metadata for these will be resolved before returning session factory Third parameter is callback (err, sessionFactory) err will be falsy if no error
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.18
connect Example/* Connection Properties */dbProperties = { adapter : "ndb", ndb_connectstring : "localhost:1186" };
// Map SQL Tables to JS Constructorsmappings = [];mappings.push(new nosql.TableMapping('tweet'). applyToClass(Tweet));// Connectnosql.connect(dbProperties, mappings, runCmdlineOperation, operation);
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.19
The connect callbackfunction dbConnected(connectionError, sessionFactory) { function handleRequest(request, response) { function onDbResponse(err, data) { var page = renderPage(data); response.write(page); response.end(); } function onDbSession(err, session) { runDbOperation(session, request, onDbResponse); } sessionFactory.openSession(null, onDbSession); } http.createServer(handleRequest).listen(8080);}nosql.connect(dbProperties, mappings, dbConnected);
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.20
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.21
find Example
function findAuthor(name, callback) { session.find(Author, name, function(err, result) { if (err) { callback(err); } else { // result is the row in the author table callback(null, result); });}
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.22
find function on Session
find is asynchronous First parameter is table name or constructor (mapped class) Second parameter is key to use locating the record
– Primary key– Unique key
Third parameter is callback (err, result)– err will be falsy if no error– result will be new object with mapped properties
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.23
persist Example
function persistAuthor(name, full, callback) {
var author = new Author(name, full);
session.persist(author, function(err) { if (err) {
callback(err);
} else {
callback(null, author);
}
});
}
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.24
persist function on Session
persist is asynchronous First parameter is instance of mapped class Second parameter callback (err)
– err will be falsy if no error Variants allow persist into default mapped table (no constructor)
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.25
update Example
function updateAuthor(author, new_full_name, callback) { author.full_name = new_full_name; session.update(author, callback); }
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.26
update function on Session
update is asynchronous First parameter is instance of mapped class
– Modified properties are written to database Second parameter callback (err)
– err will be falsy if no error Variants allow update of default mapped table (no constructor)
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.27
remove Example
function removeByAuthorName(name, callback) { session.remove(Author, name, function(err, result) {
if (err) { callback(err); } else { // row has been removed callback(null); });}
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.28
remove function on Session
remove is asynchronous– First parameter is table name or constructor (mapped class)– Second parameter is key to use locating the record
Primary key Unique key
– Third parameter is callback (err) err will be falsy if no error
Alternately– First parameter is instance of constructor– Second parameter is callback (err)
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.29
Transactions
/* Insert a tweet. - Start a transaction. - Persist the tweet. - Create & persist #hashtag & @user records - Increment the author's tweet count. - Then commit the transaction. */function InsertTweetOperation(params, data) { session.currentTransaction().begin(); ... session.currentTransaction().commit();}
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.30
Query Example// Last 20 tweets @user
TweetsAtUserOperation.run = function(tag) { if(tag.charAt(0) == "@") tag = tag.substring(1); this.session.createQuery(Mention, function(error, query) { var queryParams = { "order" : "desc", "limit" : 20 , "tag" : tag}; query.where(query.at_user.eq(query.param("tag"))); query.execute(queryParams, fetchTweetsInBatch); });};
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.31
query function
Session is Query factory using asynchronous api query has a filter based on constructor (mapped class) properties Filter has comparators and boolean operations
– eq, ne, gt, ge, le, lt, between,isNull,isNotNull comparators– and, or, not,andNot,orNot operations
Query execution is asynchronous– Filter determines query strategy
Primary/unique key lookup; index scan; table scan– Properties govern query execution– Results are given in callback
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.32
Query Comparators
Comparators compare properties to parameters Query Domain Type property names correspond to Constructor field
names (properties) Parameters are created by name
– qdt.param('date_low')
Properties are referenced by field name in Constructor– qdt.date_created
Comparators are properties of qdt properties– qdt.date_created.gt(qdt.param('date_low'));
Comparators return predicates
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.33
Query Operators
Predicates are used as query filters via where function Predicates are results of comparators or operators Operators combine predicates using and, or, andNot, orNot, not
– var predicate1 = qdt.date_created.gt(qdt.param('date_low'));
– var predicate2 = qdt.date_created.lt(qdt.param('date_high'));
– var predicate = predicate1.and(predicate2);
– var predicate = predicate3.andNot(predicate4);
– qdt.where(predicate)
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.34
Batching
function fetchTweetsInBatch(error, mentions, finalCallback) { var resultData = []; function addTweetToResults(err, tweet) { if(tweet && ! err) resultData.push(tweet); } var batch = session.createBatch(); while(var mention = mentions.pop()); batch.find(Tweet, mention.id, addTweetToResults);
batch.execute(finalCallback, resultData); }
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.35
Batching
Batching allows parallel database operations– Any combination of find, persist, update, remove
Batch is created using session.createBatch() Operations (with their own callbacks) are added to the batch Batch is executed with its own callback
– Operation callbacks are run first– Then the batch callback
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.36
Get mysql-js & run the Tweet demo### First install MySQL Cluster 7.3
### Then get the latest code from github:% git clone https://github.com/mysql/mysql-js.git
### build it% node configure.js% node-gyp configure build
### Go to the demo directory% cd samples/tweet
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.37
Get mysql-js & run the Tweet demo### create the schema% mysql -u root < tweet.sql
### Run with the mysql adapter% node tweet.js -a mysql put user mr_jdd \ 'John David Duncan'
### Run with the ndb adapter% node tweet.js -a ndb get user mr_jdd
### Look at some more examples% cat test_tweet.sh