lnug: having your node.js cake and eating it too

Having Your Node.js Cake and Eating It Too

Rob TweedM/Gateway Developments Ltd

Twitter: @rtweed

http://qewdjs.com

A bit of background

• Royal Marsden Hospital: 1980s• Touche Ross Management Consultants

(now Deloitte): early 1990s– NHS-wide Networking Project

• Independent consultant & software developer since 1993– Specialising in Web and associated

technologies, particularly in healthcare

A bit of background

• Node.js:– Since August 2010 (v0.2 had just come out!)– Gave one of the talks at LNUG inaugural

meeting

And So, 6 Years Later

Node.js Architecture

• Everything executes in a single process– You might have 1000's of users doing stuff

concurrently• They're all sharing the same process!

Node.js Architecture

• Non-blocking I/O– No user activity must block the process, or everyone

grinds to a halt

• Event-driven, asynchronous logic– Fire off the request, but don't wait for the results

• Carry on with the next task

– When results come back, handle them at the next opportunity

Node.js = server-side JavaScript

• Ryan Dahl wasn't actually a big JavaScript fan• However, he realised that it was a convenient

language for such an environment, as it's designed for exactly the same kind of way of working in the browser

• Google's V8 JavaScript engine was Open Sourced, so he used it to provide the language for Node.js– (which is actually mostly written in C++)

Asynchronous Syntax in Other Languages?

• Node.js isn't unique in supporting asynchronous logic and non-blocking I/O

• Available also in most other modern languages– Java– Python, etc

• Allows efficient use of resources when making parallel requests for resources– Files, external web services etc

Asynchronous Syntax in Node.js

• Node.js is unique in that you have no option but to use Asynchronous logic

Drawbacks of Node.js

• In every other language, zealots will encourage you to use it for everything!

• But even Node's greatest exponents will tell you not to use it for certain things

• Don't use Node.js if you have:– CPU-intensive logic– Complex database manipulation

• Particularly relational database handling

• Don't use Node.js if you have:– CPU-intensive logic

• Tie up the CPU and everyone else using the process will be blocked

– Complex database manipulation• Particularly relational database handling

• Don't use Node.js if you have:– CPU-intensive logic– Complex database manipulation

• Particularly relational database handling– Due to the limitations of asynchronous logic, eg you can't

chain functions» Limits creation of very high-level database

abstractions

Main criticism of newbies to Node.js

• Asynchronous logic

Async is the New Sync?

• Node.js version 8 now supports Async/Await which greatly improves the syntax needed to handle asynchronous logic– Avoids "callback hell"– Very like synchronous logic

Async/Await Doesn't solve:

• Node.js concurrency– Still need to avoid CPU-intensive code

• High-level database abstractions– Proper chaining of functions requires

synchronous logic

Usual Solutions

• Use a third-party queue to offload CPU-intensive work to some other environment– RabbitMQ– ZeroMQ

• Use another language such as Rails for database-intensive work

The down-sides

• Heterogeneous environment• Multiple skill-sets• Multiple moving parts

Node.jsRabbitMQ

Database

What does Ryan Dahl Think?

"..within a single process we could handle many requests by being completely asynchronous. I believed strongly inthis idea at the time, but over the past couple of years, I think that’s probably not the be-all and end-all idea for programming."

"..when I first started hearing about Go, which was around 2012, they had really easy to use abstractions, that make "blocking I/O", because it’s all in green threads at the interface between Go and the operating system.

I think it is actually all non-blocking I/O."

"I think Node is not the best system to build a massive server web. I would definitely use Go for that.

And honestly, that’s basically the reason why I left Node. It was the realization that: oh, actually, this is not the bestserver side system ever."

What do I think?

I want to have my Node.js Cake and Eat it!

I want my Node.js Cake & Eat it

• I like JavaScript• I want just one language for everything

– 1 skill set– 1 set of moving parts

• No extra technologies, thank you– Keep it simple

• And I want to be able to use Node.js for all situations

The Problem is Concurrency

• All concurrent users in Node.js share the same process

The Problem is Concurrency

• All concurrent users in Node.js share the same process

• As it happens, Amazon Web Services accidentally created a solution– And nobody (including AWS) seems to have

realised the consequences of what they've done

AWS Lambda

• "Function As A Service"• AKA "Serverless"

AWS Lambda

• "Function As A Service"• AKA "Serverless"

• You upload functions• AWS will execute them

– You don't worry about how or on what physical machine(s)

• You pay per invocation of your function(s)

AWS Lambda

• The first technology they supported was Node.js

AWS Concurrency?

• Your function will be invoked in a private computation container of some sort

• Your function has that container all to itself for the duration of its execution

• No competition with other concurrent users

Lambda Node.js Examples

• AWS examples show use of asynchronous APIs

• Node.js users of Lambda use asynchronous APIs and Async/Await

Lambda Node.js Examples

• AWS examples show use of asynchronous APIs

• Node.js users of Lambda use asynchronous APIs and Async/Await

• But unless your Lambda function really needs to be asynchronous, why use asynchronous logic?

Asynchronous Syntax in Other Languages?

• Available also in most other modern languages– Java

– Python, etc

• Allows efficient use of resources when making parallel requests for resources– Files, external web services etc

• But no programmer in those languages would use async logic if they didn't have to

• Why would Node.js developers use asynchronous logic in Lambda functions if they don't have to?– Partly because that's what you do, right?

• Why would Node.js developers use asynchronous logic in Lambda functions if they don't have to?– Partly because that's what you do, right?– Partly because almost no synchronous APIs

exist, particularly for:• Database integration

• Web/REST service access

Such APIs are possible

Any other way…

• To have your Node.js cake and eat it?– Not everyone will want to use Lambda

• Would it be possible to create a locally-available environment where my Node.js code runs in an isolated container where concurrency isn't an issue?

What Is QEWD?

• Essentially it's a multi-purpose Node.js-based run-time Platform

What Is QEWD?

• Essentially it's a multi-purpose Node.js-based run-time Platform

• Creates an isolated run-time container for your message/request handler functions, allowing:– CPU-intensive work– Database abstractions using synchronous

QEWD's Architecture

• Master Process• Pool of Worker Processes

QEWD's Architecture

• Master Process– Handles and queues all incoming requests

from client• HTTP/REST requests via Express or Koa.js

• WebSocket requests via socket.io

– Returns responses to client

QEWD's Architecture

• Pool of Persistent Worker Processes– Where all the processing occurs– A single queued request is dispatched to an

available worker process– Each Worker process handles a single

request at a time

QEWD's Architecture

• The queue / dispatcher / worker-process pool management part of QEWD is handled by a module named ewd-qoper8

Master Node.js Process

Queueprocessor/dispatcher

IncomingRequests

QEWD Architecture

Every incoming requestis passed from Expressand placed in a queue

No further processingof requests occurs inthe master process

Expressor

Koa.js

socket.io

HTTPREST

WebSocket

QEWD Architecture

Queue dispatcher isinvoked whenever arequest is added tothe queue

QEWD Architecture

Node.js Worker Process

Worker processstarted if none

available

QEWD Architecture

Redis/Cache/GT.M

QEWD &application-specific

Modules loaded

CustomWorkerModule

QEWD Architecture

CustomWorkerModule

Node.js Worker ProcessCustomWorkerModule

Request passedto worker

Redis/Cache/GT.M

QEWD Architecture

CustomWorkerModule

Worker flagged as Unavailable

Begin processing message

Redis/Cache/GT.M

QEWD Architecture

Unavailable / processing

Another incomingrequest Node.js Worker Process

CustomWorkerModule

Redis/Cache/GT.M

QEWD Architecture

CustomWorkerModule

If worker pool size not exceeded,another worker is startedand request passed to it

Redis/Cache/GT.M

QEWD Architecture

Redis/Cache/GT.M

If entire Worker Pool is busy:

Redis/Cache/GT.M

QEWD Architecture

If entire Worker Pool is busy:

New requests remain

in queue

Redis/Cache/GT.M

QEWD Architecture

Redis/Cache/GT.M

As soon as a worker is available again,a queued message can be passed to it

Redis/Cache/GT.M

Available

Redis/Cache/GT.M

QEWD Architecture

Finished

Redis/Cache/GT.M

A user's handler function signalscompletion using the function:

finished(responseObject);

This returns the responseobject to the master

process

QEWD Architecture

Finished

And the response isreturned to the client

that sent the original request

(via Express/Koa/socket.io)

Redis/Cache/GT.M

QEWD Architecture

Available

The finished() functionalso automatically returnsthe worker process back

to the available pool

So it can now handlethe next queued request

Redis/Cache/GT.M

QEWD Architecture

Available

Worker processes, once started, are persistent

No start-up / tear-down cost

Redis/Cache/GT.M

QEWD Architecture

Node.js concurrency is handledby the master process.

100% asynchronous logic

The master process doesalmost nothing

The perfect Node.js application:no CPU-intensive or long-

running tasks, so veryhigh-performance

MultipleConcurrentIncomingrequests

QEWD Architecture

Node.js concurrency is handledby the master process.

100% asynchronous logic

The master process doesalmost nothing

The perfect Node.js application:no CPU-intensive or long-

running tasks, so veryhigh-performance

All the actual work happens in theisolated worker processes

MultipleConcurrentIncomingrequests

QEWD Architecture

Available

Worker processes only handlea single request at a time

Completely isolated run-timeenvironment for handler functions

No need for concerns aboutNode.js concurrency, so

synchronous APIs can be used

Redis/Cache/GT.M

QEWD Architecture

Available

Redis/Cache/GT.M

Long-running or CPU-intensivelogic has no direct impact on

other worker processes

Why not just use child_process?

• To avoid the very high performance cost of starting and tearing down each child process– At least 30ms to start a child_process

• Each QEWD worker is a child_process– But keeps running, and is re-used

Why not just use Cluster?

• Cluster simply spreads the concurrent requests across a pool of persistent child_processes– If one of them is CPU intensive, all other

requests in that cluster are held up

• QEWD forces each child_process to handle a single request only– Creates an isolated run-time environment for

each request

Why not just use RabbitMQ or ZeroMQ?

• I just want to just use a single technology: Node.js / JavaScript

• Initial tests showed no performance benefit in using ZeroMQ

Performance?

• ewd-qoper8 module in isolation:– Handles the queue/master/worker architecture

• Raspberry Pi 3 Model B– 4 core CPU– Readily-available commodity item– Low-cost

• Easily-replicable benchmark

Performance

• ewd-qoper8 benchmark script provided:– How many workers? 3– How many messages? 500,000– Create a steady state of messages added to

the queue as they're being processed:• Add a batch of 622 messages at a time

• Wait 100ms between each batch

– Messages are sent to workers which echo them straight back

Performance

• 5,800 messages/second sustained throughput

• Limiting factor is master process hitting 100% CPU

• Workers only 30% CPU, so plenty of capacity to do real work at this rate

What can QEWD be Used For?

• Full-stack platform for browser & Native mobile applications– WebSocket or Ajax messaging

• Seamlessly swap between transports• All automated and abstracted out of the way

– With security also automated for you too

• API server for REST applications• Optional session management

– Automated token-based authentication– Server-side (persistent JSON) session storage

• Or automated JSON Web Token based:– Authentication– Client-side session storage

• MicroService Platform– With each MicroService supported by its own

shared or dedicated QEWD server– Using JWTs

• Shared secret on each QEWD server

– Using socket.io to provide high-performance, persistent connections between QEWD servers

• Secured over HTTPS if required

QEWD MicroService Fabric

ewd-qoper8queue

Express

Node.js

socket.io

CacheGT.M,YottaDBRedisNode.js

WorkerProcess

ewd-qoper8queue

Express

Node.js

socket.io

WorkerProcess

ewd-qoper8queue

Express

Node.js

socket.io

WorkerProcess

User authentication

Demographics

Pharmacy

ewd-qoper8queue

Express

Node.js

socket.io

CacheGT.M,YottaDBRedis

Node.jsWorkerProcess

Client Orchestration

HTTPSWebSocketConnections

• Federation platform– Providing a REST or WebSocket-based

middle tier to multiple REST-based back-end servers

• of any type, not just QEWD servers

– One request in• Routed to all back-end systems and responses

combined

• Performing a "dance" between federated servers

Federated Access to OpenEHR

Browser QEWD

GT.M orRedis

ewd-qoper8queue

qewd-rippleModule

Express

OpenEHRServer

AQL overHTTP(S)

Worker

OpenEHRServer

• Interface to particular databases known as Global Storage databases– GT.M / YottaDB

– Cache / IRIS

– Redis (ewd-globals-redis module)

• All of which are abstracted as:– Persistent JavaScript Objects

– Fine-grained Document Database• Accessible at any level down to individual name/value pair

anywhere in a JSON object

QEWD.js

• Resilient / Audit mode– When enabled, permanent record kept in the

database of:• Queued requests

• Processing status

• Response(s)

– If QEWD restarted, database is examined for queued, unprocessed requests

• Automatically re-queued

– Can specify retention period of database records

QEWD.js

• Supports all browser-side JavaScript frameworks

• Some cool 3rd-party tooling and support available for React and Vue.js

Have Your Node.js CakeAnd Eat It Too

http://qewdjs.com

lnug: having your node.js cake and eating it too

Software

node.js workshop- node.js unit testing

introducing in - finn technology · pdf fileagenda what is...

node.js workshop- node.js basics

glenn livingston, ph.d. and yoav ezer lost 86 pounds by...

node.js workshop

node.js + nosql

introduction node.js

having your cake and eating it too: jointly optimal erasure...

node.js @ rubyslava

node.js introduction

mastering node.js

having your cake and eating it too · having your cake and...

exploring node.js

private investments in new environmental technologies:...

node.js lab

offshore node.js-development- hire node.js developers-...

corporate inversions: a case of having the cake...

having your cake and eating it too

having your cake, and eating it too!

node.js primer -...