webrtc + socket.io: building a skype-like video chat with native javascript
TRANSCRIPT
WEBRTC + SOCKET.IOBUILDING A SKYPE-LIKE VIDEO CHAT WITH
NATIVE JAVASCRIPT
/ MICHELE DI SALVATORE @MIKDISALJavascript Architect @ Objectway
Audio and video communication and peer-to-peer data
sharing through a web application
Native javascript (no plugins)
Open source
appear.in
HOW IS BORNGlobal IP Solutions acquired by Google in 2010
Google open sourced the technologies
SUPPORTED BROWSERS AND PLATFORMS
IE AND SAFARI? freeTemasys Plugin
JAVASCRIPT WEBRTC APIMediaStream
RTCPeerConnection
RTCDataChannel
MEDIASTREAM (AKA GETUSERMEDIA)Acquiring audio and video
Can contain multiple 'tracks'var constraints = { video: { mandatory: { minWidth: 640, minHeight: 360 } };
function successCallback(stream) { var video = document.querySelector("video"); video.src = window.URL.createObjectURL(stream); }
function errorCallback(error) { console.log("navigator.getUserMedia error: ", error); }
Constraints controls the contents of the MediaStream
RTCPEERCONNECTION
Communicating audio and video
Signal processingCodec handlingPeer to peer communicationSecurityBandwidth management...
RTCDATACHANNEL
Communicating arbitrary data
Same API as WebSocketsUltra-low latencyUnreliable or reliableSecure
SIGNALINGExchanging 'session description' objects
What formats I support, what I want to sendNetwork information for peer-to-peer setup
Any message mechanism and protocol
JSEPJavaScript Session Establishment Protocol
RTCSESSIONDESCRIPTION EXAMPLE
v=0 o=- 7614219274584779017 2 IN IP4 127.0.0.1 s=- t=0 0 a=group:BUNDLE audio video a=msid-semantic: WMS m=audio 1 RTP/SAVPF 111 103 104 0 8 107 106 105 13 126 c=IN IP4 0.0.0.0 a=rtcp:1 IN IP4 0.0.0.0 a=ice-ufrag:W2TGCZw2NZHuwlnf a=ice-pwd:xdQEccP40E+P0L5qTyzDgfmW a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level a=mid:audio a=rtcp-mux a=crypto:1 AES_CM_128_HMAC_SHA1_80 inline:9c1AHz27dZ9xPI91YNfSlI67/EMkjHHIHORiClQea=rtpmap:111 opus/48000/2 ...
P2P IN REAL WORLD
NAT & Firewalls?
STUN SERVERGives my public IP addressSimple server, cheap to runData flows peer-to-peer
STUN SERVER
TURN SERVERFallback for p2p failingData via serverCall works everywhere
STUN + TURN SERVERS
ICEInteractive Connectivity Establishment
A framework to connect peers via UDP, enablingRTCPeerConnection via STUN and TURN server
WEBRTC SECURITYOn Chrome only via https
Encryption for media and data
Sandboxed
WEBRTC LIBRARIES & CO.
by by by
by ...
adapter.js webrtcrtc-everywhere contraSimpleWebRTC &YetRTCMultiConnection Muaz Khan
RTCMULTICONNECTION
A javascript wrapper library supportingapproximately all possible peer-to-peer
features.
WEBRTC EXPERIMENTS
Tons of open source experiments based onRTCMultiConnection:
webrtc-experiment.com
WEBSOCKETS
Standardized in 2011
Interactive communication session between browser andserver
Event driven
Sending strings and binary data
WEBSOCKETS BROWSER SUPPORT
All and IE10+
ajax polling as fallback
SOCKET.IO
A popular open source library with client and serverimplementations
Server side written in Node and easy to startvar server = require('http').createServer(); var io = require('socket.io')(server);
io.on('connection', function(socket){ socket.on('hi!', function(data){ socket.emit('a-message', {hello: 'world'}); });
socket.on('disconnect', function(){}); });
server.listen(3000);
SOCKET.IO CLIENT
var socket = io.connect('http://localhost:3000/');
socket.on('connect', function () { socket.emit('hi!'); });
socket.on('a-message', function (data) { console.log(data.hello); });
GOALA SKYPE LIKE CHAT
MAIN ISSUESWebRTC architecture like IRC
rooms with shared chat: text, video and datadirect chat: only one to oneroom managed by initiatorconversation type can change without agreement: text tovideo
IMPLEMENTATION
STEP 1
UserA is logged in and is connected to the socket server
STEP 2
The app sends a "presence" event to other users throughthe socket
The server forwards the message only to user's contacts
Each present user replies with the same "presence" event
SOCKET.IO FORWARD ROLE
STEP 3
Each present user replies with the same "presence" event
STEP 4
When a user receives a "presence" event, it automaticallyknows that the sender is online
Also the server sends the “presence” event to all, when auser socket is disconnected
STEP 5
UserA starts a conversation and the app creates the webrtcconnection
STEP 6
UserA sends a message via webrtc
STEP 7
UserB receives a notification
STEP 8
UserB starts writing a message
STEP 9
UserA receives the "is typing" info sent via webrtc
STEP 10
UserA starts a video call sending a request to UserB viasocket
STEP 11
UserB receives the request and accepting starts a video callvia webrtc
STEP 12
The 2 users can video chat via webrtc
Each user can close the video at any time or start a text chat
STEP 13
A user can send text messages also
STEP 14
A user can start an audio chat in the same way as a videochat
chat
BEHIND THE SCENES
WHERE IS WEBRTC?
Behind this actions
chat messagesvideo / audio call"is typing" info
AND THE OTHERS?via WebSockets
presencevideo / audio initializationall status infos
THAT'S ALL!THANK YOU ALL
/ Michele Di Salvatore @MikDiSal