turbocharging client-side processing: leveraging asm.js

TURBOCHARGING CLIENT-SIDE PROCESSING

LEVERAGING ASM.JSVladimir Vukićević / @vvuk

OVERVIEWWhat is asm.js?

Writing asm.js by hand

Using Emscripten

Integrating libraries

What can you do with this

WHAT IS ASM.JS?

This is . It's JavaScript, with some rigid rules.

var g = Float32Array(32768); function main() { var a = 0, b = 0; do { a = 8 + (b << 2) | 0; g[a >> 2] = +g[a >> 2] + 1.0; b = b + 1 | 0; } while ((b | 0) < 5000); }

asm.js

asm.js operates on an ArrayBuffer heap

asm.js operates only on numeric values

asm.js can call regular JavaScript

asm.js can be called by regular JavaScript

asm.js provides predictable performance

TWO WAYS TO USE ASM.JSTo bring existing C/C++ apps to the web

Many of the current flashy demos do thisGreat for gamesNot the focus here

To accelerate processing of JS appsLeverage existing librariesOffload and speed up data processing and analysis

WRITING ASM.JS BY HANDvar heap = new ArrayBuffer(4096);var asm = function(stdlib, foreign, heap) { "use asm"; var HEAPF32 = new stdlib.Float32Array(heap); function averageValues(first, last) { first = first|0; last = last|0; var count = 0; var avg = 0.0; count = (last - first) >> 2; while ((first|0) < (last|0)) { avg = avg + +HEAPF32[first >> 2]; first = first + 4|0; } return +(avg / +(count|0)); } return { averageValues: averageValues, };}(window, {}, heap);/* put some values into the heap */var HEAPF32 = new Float32Array(heap);for (var i = 0; i < 20; ++i) { HEAPF32[i] = i;}/* make the call */var avg = asm.averageValues(0<<2, 20<<2);console.log(avg);

WRITING ASM.JS BY HAND... DOABLE, BUT PAINFUL.

Doable by hand for simple things

e.g., a matrix math library

It's intended to be a compile target

Not just from C/C++!

LLJS can target asm.js

TypeScript is interested

Emscripten for C/C++ source

USING EXISTING LIBRARIESTHROUGH EMSCRIPTEN

LOTS OF EXISTING CODE OUT THEREFor example, open source libraries for...

Computer Vision — OpenCV

Physics — Box2D, Bullet

Compression — zlib, lz4, etc.

Text-to-Speech — eSpeak

Convex Hull Computation — qhull

...

LOTS OF EXISTING CODE OUT THEREMany already have Emscripten ports

Some have nice JS layers on top

INTEGRATING WITH ASM.JS CODE1. Get data into asm.js heap

var canvasData = ctx.getImageData(0, 0, w, h); /* allocate space in asm.js heap */ var heapPtr = Module._malloc(w * h * 4); var heapData = new Uint8Array(HEAP, heapPtr, w * h * 4); /* copy canvasData.data into heap */ heapData.set(canvasData.data);

2. Call function(s) asmModule.doProcessing(heapPtr, w, h);

3. Get data out of heap /* copy data back to ImageData */ canvasData.data.set(heapData); /* free space in asm.js heap */ Module._free(heapPtr); /* draw image data back */ ctx.putImageData(canvasData, 0, 0);

INTEGRATING WITH ASM.JS CODEYou have to deal with manual

memory management and allocation

Likely already thinking about itif you're heavily optimizing your JS

Make some nice wrappers and avoidhaving to think about this again

NICE WRAPPERS?function processImageData(imageData) { var w = imageData.width, h = imageData.height; var heapPtr = Module._malloc(w * h * 4); var heapData = new Uint8Array(HEAP, heapPtr, w * h * 4); heapData.set(canvasData.data);

asmModule.doProcessing(heapPtr, w, h);

imageData.data.set(heapData); Module._free(heapPtr);}

JUST ANOTHER MODULE

asm.jscode

JavaScript App

App Code

EmberJS

jQuery

...

SOME ODDS AND ENDSUse Worker Threads

Use async Script Loading

USING WORKER THREADSPlace your asm.js module on a worker

Asynchronous execution, no jank!

Main App code postMessage ⇒

request

⇐ postMessagereply

Worker

asm.jsmodule

(ASIDE) WORKERS ARE AWESOMEAnd will get even more awesome

We've got lots of APIs coming to workers, including...

WebRTC

WebGL & Canvas 2D

Web Sockets

Web Audio

Synchronous message passing

USING ASYNCHRONOUS SCRIPT LOADINGasm.js code can be large, and parsing and compiling takes time.

<script async src="lib/myAsmLib.js"></script>

Add async. Magic.

In the (very near) future, such compilation will also be cached.

WHAT DO I DO WITH THIS?Processing/resizing of photos client-side

(color correction, face identification)

Data analysis(download raw data to client)

WHAT DO I DO WITH THIS?Face login/confirmation

(WebRTC + UserMedia + canvas)

Add text-to-speech(read notices in the background)

WHAT DO I DO WITH THIS?Implement new image (and video?) codecs

(specialized or experimental)

Recognize barcodes(handy on mobile!)

PERFORMANCE?"Within 2x of native."

... pretty consistently.

Even on mobile! Take advantage to make mobile apps feel morelike native.

IT'S DANGEROUS TO GO ALONEYou'll need...

Some familiarity with C/C++(or someone nearby familiar with C/C++)

Emscripten and Emscripten SDK

NOT JUST FOR C/C++Focus has been on C/C++ due to large body of existing code

Lua, Python, etc. all exist

For example, give users ability to write scripts in Lua

WRAP-UPasmjs.org

emscripten.org

QUESTIONS?Vladimir Vukićević / @vvuk