webrtc media in the cloud - dialogic | cloud-optimized real …/media/products/docs/putting... ·...
TRANSCRIPT
![Page 1: WebRTC Media in the Cloud - Dialogic | Cloud-Optimized Real …/media/products/docs/Putting... · 2014-10-07 · Title: WebRTC Media in the Cloud Author: Chad Hart Created Date: 10/7/2014](https://reader033.vdocument.in/reader033/viewer/2022042119/5e98bbb2955e8b37057b7d2f/html5/thumbnails/1.jpg)
COMPANY CONFIDENTIAL © COPYRIGHT 2014 DIALOGIC INC. ALL RIGHTS RESERVED.
WebRTC Media in the Cloud
Chad Hart
IIT RTC Conference 2014
![Page 2: WebRTC Media in the Cloud - Dialogic | Cloud-Optimized Real …/media/products/docs/Putting... · 2014-10-07 · Title: WebRTC Media in the Cloud Author: Chad Hart Created Date: 10/7/2014](https://reader033.vdocument.in/reader033/viewer/2022042119/5e98bbb2955e8b37057b7d2f/html5/thumbnails/2.jpg)
COMPANY CONFIDENTIAL © COPYRIGHT 2014 DIALOGIC INC. ALL RIGHTS RESERVED. 2
Why do you need a media server?
Multi-party conferencing InterworkingTranscoding
Stream processing Person-to-machineRecording
![Page 3: WebRTC Media in the Cloud - Dialogic | Cloud-Optimized Real …/media/products/docs/Putting... · 2014-10-07 · Title: WebRTC Media in the Cloud Author: Chad Hart Created Date: 10/7/2014](https://reader033.vdocument.in/reader033/viewer/2022042119/5e98bbb2955e8b37057b7d2f/html5/thumbnails/3.jpg)
COMPANY CONFIDENTIAL © COPYRIGHT 2014 DIALOGIC INC. ALL RIGHTS RESERVED. 3
What do you care more about?
![Page 4: WebRTC Media in the Cloud - Dialogic | Cloud-Optimized Real …/media/products/docs/Putting... · 2014-10-07 · Title: WebRTC Media in the Cloud Author: Chad Hart Created Date: 10/7/2014](https://reader033.vdocument.in/reader033/viewer/2022042119/5e98bbb2955e8b37057b7d2f/html5/thumbnails/4.jpg)
COMPANY CONFIDENTIAL © COPYRIGHT 2014 DIALOGIC INC. ALL RIGHTS RESERVED. 4
Bandwidth is not always ubiquitous or free
Client-side Server-side
![Page 5: WebRTC Media in the Cloud - Dialogic | Cloud-Optimized Real …/media/products/docs/Putting... · 2014-10-07 · Title: WebRTC Media in the Cloud Author: Chad Hart Created Date: 10/7/2014](https://reader033.vdocument.in/reader033/viewer/2022042119/5e98bbb2955e8b37057b7d2f/html5/thumbnails/5.jpg)
COMPANY CONFIDENTIAL © COPYRIGHT 2014 DIALOGIC INC. ALL RIGHTS RESERVED. 5
CPU & processing power is expensive
Client view Provider View
![Page 6: WebRTC Media in the Cloud - Dialogic | Cloud-Optimized Real …/media/products/docs/Putting... · 2014-10-07 · Title: WebRTC Media in the Cloud Author: Chad Hart Created Date: 10/7/2014](https://reader033.vdocument.in/reader033/viewer/2022042119/5e98bbb2955e8b37057b7d2f/html5/thumbnails/6.jpg)
COMPANY CONFIDENTIAL © COPYRIGHT 2014 DIALOGIC INC. ALL RIGHTS RESERVED. 6Image source: http://thewrestlinglifeupdate.tumblr.com/
![Page 7: WebRTC Media in the Cloud - Dialogic | Cloud-Optimized Real …/media/products/docs/Putting... · 2014-10-07 · Title: WebRTC Media in the Cloud Author: Chad Hart Created Date: 10/7/2014](https://reader033.vdocument.in/reader033/viewer/2022042119/5e98bbb2955e8b37057b7d2f/html5/thumbnails/7.jpg)
COMPANY CONFIDENTIAL © COPYRIGHT 2014 DIALOGIC INC. ALL RIGHTS RESERVED. 7Image source: http://thewrestlinglifeupdate.tumblr.com/
Multi-PARTY! Video Conferencing
![Page 8: WebRTC Media in the Cloud - Dialogic | Cloud-Optimized Real …/media/products/docs/Putting... · 2014-10-07 · Title: WebRTC Media in the Cloud Author: Chad Hart Created Date: 10/7/2014](https://reader033.vdocument.in/reader033/viewer/2022042119/5e98bbb2955e8b37057b7d2f/html5/thumbnails/8.jpg)
COMPANY CONFIDENTIAL © COPYRIGHT 2014 DIALOGIC INC. ALL RIGHTS RESERVED. 8
Easy & cheap approach to multi-party: Mesh
8
Full Mesh
Works for a few parties
No server cost
Lowest latency
![Page 9: WebRTC Media in the Cloud - Dialogic | Cloud-Optimized Real …/media/products/docs/Putting... · 2014-10-07 · Title: WebRTC Media in the Cloud Author: Chad Hart Created Date: 10/7/2014](https://reader033.vdocument.in/reader033/viewer/2022042119/5e98bbb2955e8b37057b7d2f/html5/thumbnails/9.jpg)
COMPANY CONFIDENTIAL © COPYRIGHT 2014 DIALOGIC INC. ALL RIGHTS RESERVED. 9
Mesh does not scale for many video conferees
9
Full Mesh
Clients get overloadedEncode costs more than decode
Limited uplink bandwidth
Inconsistent performance across participants
![Page 10: WebRTC Media in the Cloud - Dialogic | Cloud-Optimized Real …/media/products/docs/Putting... · 2014-10-07 · Title: WebRTC Media in the Cloud Author: Chad Hart Created Date: 10/7/2014](https://reader033.vdocument.in/reader033/viewer/2022042119/5e98bbb2955e8b37057b7d2f/html5/thumbnails/10.jpg)
COMPANY CONFIDENTIAL © COPYRIGHT 2014 DIALOGIC INC. ALL RIGHTS RESERVED. 10
Traditional Telephony Approach: MCU
10
Multipoint Control Unit (MCU)Good at manipulating all media
Transcoding
Transizing
Transrating
Interworking
Server-side CPU intensive
Client can request different conference mixes
Usually client friendly
Downsizing & mixing can reduce bandwidth
Fewer client streams to process
MCU
![Page 11: WebRTC Media in the Cloud - Dialogic | Cloud-Optimized Real …/media/products/docs/Putting... · 2014-10-07 · Title: WebRTC Media in the Cloud Author: Chad Hart Created Date: 10/7/2014](https://reader033.vdocument.in/reader033/viewer/2022042119/5e98bbb2955e8b37057b7d2f/html5/thumbnails/11.jpg)
COMPANY CONFIDENTIAL © COPYRIGHT 2014 DIALOGIC INC. ALL RIGHTS RESERVED. 11
Video UX: Traditional method – mixed screen
MCU1 2
3 4
Mixed stream 1 2
3 4
3G Device
![Page 12: WebRTC Media in the Cloud - Dialogic | Cloud-Optimized Real …/media/products/docs/Putting... · 2014-10-07 · Title: WebRTC Media in the Cloud Author: Chad Hart Created Date: 10/7/2014](https://reader033.vdocument.in/reader033/viewer/2022042119/5e98bbb2955e8b37057b7d2f/html5/thumbnails/12.jpg)
COMPANY CONFIDENTIAL © COPYRIGHT 2014 DIALOGIC INC. ALL RIGHTS RESERVED. 12
Video UX: Traditional method – active talker
MCU1 2
3 4
Mixed stream 1 23
3G DeviceActive Talker
Media Server Chooses Layout
![Page 13: WebRTC Media in the Cloud - Dialogic | Cloud-Optimized Real …/media/products/docs/Putting... · 2014-10-07 · Title: WebRTC Media in the Cloud Author: Chad Hart Created Date: 10/7/2014](https://reader033.vdocument.in/reader033/viewer/2022042119/5e98bbb2955e8b37057b7d2f/html5/thumbnails/13.jpg)
COMPANY CONFIDENTIAL © COPYRIGHT 2014 DIALOGIC INC. ALL RIGHTS RESERVED. 13
Client appApp Server
Modern UX with MCU mixing, HTML5 & JavaScript
Browser
MCU1 2
3 4
HTML5 CanvasJavascript
Mixed streamLayout
"Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum."
![Page 14: WebRTC Media in the Cloud - Dialogic | Cloud-Optimized Real …/media/products/docs/Putting... · 2014-10-07 · Title: WebRTC Media in the Cloud Author: Chad Hart Created Date: 10/7/2014](https://reader033.vdocument.in/reader033/viewer/2022042119/5e98bbb2955e8b37057b7d2f/html5/thumbnails/14.jpg)
COMPANY CONFIDENTIAL © COPYRIGHT 2014 DIALOGIC INC. ALL RIGHTS RESERVED. 14
Client appApp Server2
Modern UX with MCU mixing, HTML5 & JavaScript
Browser
MCU1 2
3 4
HTML5 CanvasJavascript
Mixed streamLayout
"Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum."
![Page 15: WebRTC Media in the Cloud - Dialogic | Cloud-Optimized Real …/media/products/docs/Putting... · 2014-10-07 · Title: WebRTC Media in the Cloud Author: Chad Hart Created Date: 10/7/2014](https://reader033.vdocument.in/reader033/viewer/2022042119/5e98bbb2955e8b37057b7d2f/html5/thumbnails/15.jpg)
COMPANY CONFIDENTIAL © COPYRIGHT 2014 DIALOGIC INC. ALL RIGHTS RESERVED. 15
Client appApp Server2
Modern UX with MCU mixing, HTML5 & JavaScript
Browser
MCU1 2
3 4
HTML5 CanvasJavascript
Mixed stream
Application-defined UX
Layout
"Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum."
Benefits:
Application defined UX
User specific & customizable layout
Lower client BW & CPU vs. mesh
![Page 16: WebRTC Media in the Cloud - Dialogic | Cloud-Optimized Real …/media/products/docs/Putting... · 2014-10-07 · Title: WebRTC Media in the Cloud Author: Chad Hart Created Date: 10/7/2014](https://reader033.vdocument.in/reader033/viewer/2022042119/5e98bbb2955e8b37057b7d2f/html5/thumbnails/16.jpg)
COMPANY CONFIDENTIAL © COPYRIGHT 2014 DIALOGIC INC. ALL RIGHTS RESERVED. 16
Newer approach: SFU
16
SFU
Selective Forwarding Unit (SFU) routing
Clients send one & receive many
Client can instruct SFU which streams to send
High throughput
Can be lots of downlink bandwidth
Low latency
![Page 17: WebRTC Media in the Cloud - Dialogic | Cloud-Optimized Real …/media/products/docs/Putting... · 2014-10-07 · Title: WebRTC Media in the Cloud Author: Chad Hart Created Date: 10/7/2014](https://reader033.vdocument.in/reader033/viewer/2022042119/5e98bbb2955e8b37057b7d2f/html5/thumbnails/17.jpg)
COMPANY CONFIDENTIAL © COPYRIGHT 2014 DIALOGIC INC. ALL RIGHTS RESERVED. 17
Newer approach: SFU
17
SFU
Selective Forwarding Unit (SFU) routing
Clients send one & receive many
Client can instruct SFU which streams to send
High throughput
Can be lots of downlink bandwidth
Low latency
![Page 18: WebRTC Media in the Cloud - Dialogic | Cloud-Optimized Real …/media/products/docs/Putting... · 2014-10-07 · Title: WebRTC Media in the Cloud Author: Chad Hart Created Date: 10/7/2014](https://reader033.vdocument.in/reader033/viewer/2022042119/5e98bbb2955e8b37057b7d2f/html5/thumbnails/18.jpg)
COMPANY CONFIDENTIAL © COPYRIGHT 2014 DIALOGIC INC. ALL RIGHTS RESERVED. 18
New WebRTC approach: Simulcast
18
SFU
High bitrate
Low bitrate
Selective Forwarding Unit (SFU) with Simulcast
Clients send multiple streams to SFU
one high-bit rate
one or more lower-bit
Client directs SFU which streams to receive
Reduces bandwidth vs. SFU
Simulcast in WebRTC coming
![Page 19: WebRTC Media in the Cloud - Dialogic | Cloud-Optimized Real …/media/products/docs/Putting... · 2014-10-07 · Title: WebRTC Media in the Cloud Author: Chad Hart Created Date: 10/7/2014](https://reader033.vdocument.in/reader033/viewer/2022042119/5e98bbb2955e8b37057b7d2f/html5/thumbnails/19.jpg)
COMPANY CONFIDENTIAL © COPYRIGHT 2014 DIALOGIC INC. ALL RIGHTS RESERVED. 19
Future Approach with VP9 – SVC?
19
SFU
MediumHigh LowLayered bitrates:
Selective Forwarding Unit (SFU) with SVC
Clients send layered stream to SFU
Varying bitrates – time, size, quality
SFU directs who gets what
Coming to WebRTC eventually
![Page 20: WebRTC Media in the Cloud - Dialogic | Cloud-Optimized Real …/media/products/docs/Putting... · 2014-10-07 · Title: WebRTC Media in the Cloud Author: Chad Hart Created Date: 10/7/2014](https://reader033.vdocument.in/reader033/viewer/2022042119/5e98bbb2955e8b37057b7d2f/html5/thumbnails/20.jpg)
COMPANY CONFIDENTIAL © COPYRIGHT 2014 DIALOGIC INC. ALL RIGHTS RESERVED. 20
Typical media server network model
Load BalancerLoad Balancer
Client Client Client Client Client Client
App ServerApp Server
App Server
Media ServerMedia Server
Media Server
![Page 21: WebRTC Media in the Cloud - Dialogic | Cloud-Optimized Real …/media/products/docs/Putting... · 2014-10-07 · Title: WebRTC Media in the Cloud Author: Chad Hart Created Date: 10/7/2014](https://reader033.vdocument.in/reader033/viewer/2022042119/5e98bbb2955e8b37057b7d2f/html5/thumbnails/21.jpg)
COMPANY CONFIDENTIAL © COPYRIGHT 2014 DIALOGIC INC. ALL RIGHTS RESERVED. 21
Pseudo Call-flow with a media server
AS LB MS1 MS2 Client
Call signaling
Media control
MS1 status
MS2 status
Media control
Media
![Page 22: WebRTC Media in the Cloud - Dialogic | Cloud-Optimized Real …/media/products/docs/Putting... · 2014-10-07 · Title: WebRTC Media in the Cloud Author: Chad Hart Created Date: 10/7/2014](https://reader033.vdocument.in/reader033/viewer/2022042119/5e98bbb2955e8b37057b7d2f/html5/thumbnails/22.jpg)
COMPANY CONFIDENTIAL © COPYRIGHT 2014 DIALOGIC INC. ALL RIGHTS RESERVED. 22
Pseudo Call-flow – traditional fail-over with RE-INVITE
AS LB MS1 MS2 Client
Call signaling – REINVITE with new addresses
Media control
MS1 status
MS2 status
Media control + call states
Media
![Page 23: WebRTC Media in the Cloud - Dialogic | Cloud-Optimized Real …/media/products/docs/Putting... · 2014-10-07 · Title: WebRTC Media in the Cloud Author: Chad Hart Created Date: 10/7/2014](https://reader033.vdocument.in/reader033/viewer/2022042119/5e98bbb2955e8b37057b7d2f/html5/thumbnails/23.jpg)
COMPANY CONFIDENTIAL © COPYRIGHT 2014 DIALOGIC INC. ALL RIGHTS RESERVED. 23
Pseudo Call-flow – fail-over the cloud way
AS LB MS1 MS2 Client
Media control
MS1 status
MS2 status
New container
Media
Memcache & Virtual IP Address
![Page 24: WebRTC Media in the Cloud - Dialogic | Cloud-Optimized Real …/media/products/docs/Putting... · 2014-10-07 · Title: WebRTC Media in the Cloud Author: Chad Hart Created Date: 10/7/2014](https://reader033.vdocument.in/reader033/viewer/2022042119/5e98bbb2955e8b37057b7d2f/html5/thumbnails/24.jpg)
COMPANY CONFIDENTIAL © COPYRIGHT 2014 DIALOGIC INC. ALL RIGHTS RESERVED. 24
When you are serious about scale
Declarative
Concurrency
Soft real-time
Robustness
Distribution
Hot code loading
External interfaces
Portability
![Page 25: WebRTC Media in the Cloud - Dialogic | Cloud-Optimized Real …/media/products/docs/Putting... · 2014-10-07 · Title: WebRTC Media in the Cloud Author: Chad Hart Created Date: 10/7/2014](https://reader033.vdocument.in/reader033/viewer/2022042119/5e98bbb2955e8b37057b7d2f/html5/thumbnails/25.jpg)
COMPANY CONFIDENTIAL © COPYRIGHT 2014 DIALOGIC INC. ALL RIGHTS RESERVED. 25
ICE, STUN, and TURN save the day
Use of Public IP’s mitigates need for TURN servers
Open source TURN servers do reasonably well
Running the TURN server co-resident with the media server works well when you need it
Image source: webrtcHacks- http://webrtchacks.com/an-intro-to-webrtcs-natfirewall-problem/