@affectiva

JavaScript SDK

Jason Radice

Overview

@affectiva

● What is the JavaScript SDK?● How does it work under the

covers?● Quick notes on how to use it● Demo Apps

What is the JSSDK?

@affectiva

● Emscripten compiled version of our C++ SDK○ Output is our SDK in asm.js

● Runs completely in the browser - no server communications● Does not require any plugins or third party JS libraries● Analyzes an image, a sequence of frames, or a video in real-time● In a nutshell - you pass it some image data and it returns some

metrics about itEmotions: {"joy":100, "sadness":0, "disgust":0, "contempt":0, "anger":0, "fear":0, "surprise":2, "valence":98, "engagement":100},

Expressions: {"smile":100, "innerBrowRaise":0, "browRaise":0, "browFurrow":0, "noseWrinkle":0, "upperLipRaise":5, "lipCornerDepressor":0, "chinRaise":0, "lipPucker":0, "lipPress":0, "lipSuck":0, "mouthOpen":100, "smirk":0, "eyeClosure":0, "attention":98, "lidTighten":0, "jawDrop":0, "dimpler":0, "eyeWiden":0, "cheekRaise":41, "lipStretch":0}

How does it work?

@affectiva

● Simply include the SDK JavaScript file in any web page

● Create a new detector instance and then start!

● Once the detector is started, it spawns a Web Worker thread● Compiled SDK is downloaded and initialized on the worker● Worker sends a message back to the detector once initialization is

complete● Detector will trigger the "onInitializeSuccess" event

<script src="https://download.affectiva.com/js/3.1/affdex.js"/>

var detector = new affdex.PhotoDetector();detector.start();

detector.addEventListener("onInitializeSuccess", function() { // now you're ready to start processing frames!});

How to process an image

@affectiva

● Draw an image to a hidden Canvas, and pass the ImageData to the detector

● To process an image or frame, the detector posts a message to the SDK worker with the ImageData○ Worker thread does not have access to the DOM, so the raw

image data is sent to the SDK via postMessage()● After processing the image data, worker will post a message to

the detector with the results● Detector will trigger an event in your code with the results

var img = new Image();img.onload = function() { var context = document.createElement("canvas").getContext("2d"); context.drawImage(img, 0, 0, img.width, img.height);

var img_data = context.getImageData(0, 0, img.width, img.height); detector.process(img_data, 0);};img.src = "http://example.com/some_image.jpg";

Metrics

@affectiva

● Attach an event listener to the detector and listen for onImageResultsSuccess()

● The faces object contains an array of faces found. ○ Currently, the JSSDK will only return metrics for 1 face○ Future versions will return results for multiple faces

● Each face object will have 3 metric categories: Emotions, Expressions, Emoji○ Feature point data is also available in the FeaturePoints

category● Get the "joy" emotion metric for the image that was just

processed

● Metric value will be between 0 and 100

detector.addEventListener("onImagesResultsSuccess", function(faces, img, ts) { // metrics are in the 'faces' object});

var joy_value = faces[0]["emotions"]["joy"];

Demos and sample code

@affectiva

● Some simple demonstration applications on the Affectiva JSSDK Github page○ https://github.com/Affectiva/js-sdk-sample-apps

● Detect emotions from a photo on your hard drive○ https://jsfiddle.net/affectiva/h6p64vwg/show/

● Detect emotions from your webcam○ https://jsfiddle.net/affectiva/opyh5e8d/show/

Photo Detector Demohttps://jsfiddle.net/affectiva/h6p64vwg/show/

@affectiva

Camera Detector Demohttps://jsfiddle.net/affectiva/opyh5e8d/show/

@affectiva

Instagram Demo

@affectiva

Game feedback demohttps://labs.affectiva.com/jssdk-game-demo/

@affectiva

@affectiva

Emotion-enable your own apps and digital experiences with our SDK

Get started today:affectiva.com/SDK