bringing internet video to prime timenetseminar.stanford.edu/past_seminars/seminars/02_16_12.pdf ·...

Bringing Internet Video to Prime Time Ion Stoica UC Berkeley / Conviva February 16, 2012

Background: Conviva !   Co-founded with Hui Zhang (CMU) in fall, 2006

!   Conviva optimizes video quality for premium content properties !   E.g., HBO, ESPN, ABC, Disney, and Turner

! Conviva's powered some of the world's largest on-line events !   E.g., Olympics, FIFA World Cup, NCAA College Basketball March

Madness, Major League Baseball, and Academy Awards.

Internet Video Ecosystem : Video Data-plane

Video Source

Encoders & Video Servers

CMS and Hos4ng Content Delivery

Networks (CDN)

ISP & Home Net

Screen

Video Player

Key components: Video Player & CDN

Trends

Trend: CDN Pricing

!   CDN pricing has decreased x1.5-2 every year for the last 6 years

2006 2007 2008 2009 2010 2011

cents/GB

Trend: Bitrate for Premium Content

!   Average bitrate has increased 20-40% every year

0 200 400 600 800 1000 1200 1400 1600 1800

2006 2007 2008 2009 2010 2011

Trend: Per-hour Streaming Cost

!   Per-hour streaming cost has decreased 15-30% every year

0 1 2 3 4 5 6 7 8 9 10

2006 2007 2008 2009 2010 2011

cents/ho

HTTP Chunking

!   Trends fueled by switching from proprietary streaming technologies (e.g., Adobe’s FMS) to HTTP chunking !   Move Networks (2005)

!   Apple (2008)

!   Microsoft (2009)

!   Adobe (2011)

How does HTTP Chunking Work?

Client A

Client B

ISP A ISP B

Client C

… origin

hUp cache

Why HTTP Chunking?

!   Scale: more HTTP caches/servers than streaming servers

!   Cost: HTTP servers cheaper than proprietary streaming protocol

!   Reachability: HTTP traffic penetrates enterprises firewalls easier than proprietary streaming protocols

!   High bitrate: can use multiple TCP connections to download subsequent chunks in parallel

What Does this Mean?

!   Ad supported premium content !   CPM (cost per thousand of ad impressions): $20-$40

!   One ad covers one hour of streaming!

!   Paid content !   $0.99 episode, distribution cost < 2%

!   Subscription based premium content !   It costs less than $1/month to stream content to an user watching 2

hour per day

!   Production & rights dominate!

Quality Matters

We’ve seen patterns across many sites … and billions of streams

Video Player Monitoring : Player Model

Stopped/

Player

States Joining Playing Buffering Playing

Network/

stream

connection

established

buffer

filled upVideo

buffer

Buffer

replenished

sufficiently

action

Events

Player

Monitoring

Video download rate,

Available bandwidth,

Dropped frames,

Frame rendering rate, etc.

JoinTime (JT) BufferingRa2o(BR) RateOfBuffering(RB)

AvgBitrate(AB)

RenderingQuality(RQ)

Video Player Monitoring : Data Collection

Automa/c Monitoring

•  Automa4c and consistent monitoring of default streaming modules –  Flash: NetStream, VideoElement –  Silverlight: MediaElement, SmoothStreamingMediaElement –  iOS: MPMoviePlayerController

Streaming Module

UI Controller

Content Manager

Messaging & Serializa/on

Player Insight

To backend

Player App

Empirical Study [Sigcomm’11]

!   Goal: impact of quality on engagement (play time)

!   A week of data from multiple premium video sites

!   There genres: Live, Long VoD (LVoD), Short VoD

!   Five quality metrics: !   Buffering Ratio (BR)

!   Rate of Buffering (RB)

!   Join/Start Time (JT)

!   Rendering Quality (RQ)

!   Average Bit rate (AR)

!   Two granularities: view/viewers

Highlight of Results [Sigcomm’11]

!   Quality has substantial impact on engagement

!   Buffering ratio is most critical across genres !   Highest impact for live: 1% of buffering reduced play time by 3min

!   Bitrate more important for live than VoD

!   Join time impact engagement at viewer level but less at view level

LVoD at View Level [Sigcomm’11]

Buffering ratio correlates with engagement the most

LVoD vs. Live [Sigcomm’11]

Buffering Ratio remains most significant, but Bitrate and Rate of Buffering bigger impact

Live LVoD

Engagement vs. Buffering Ratio [Sigcomm’11]

!!!!!!!!

!!!!!!!!!!!!!!!!!!!!!!!

!!!!!!!!!!!!!!!!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

0 20 40 60 80 100

Buffering ratio (%)

time (

Correlation coefficient (kendall): −0.96, slope: −3.25

1% increase in buffering reduces engagement by 3 min

Engagement vs. Join Time: Viewer Level [Sigcomm’11]

0 10 20 30 40 50 60 70Join time (s)

Correlation coefficient (kendall): -0.74

Join time critical for user retention

What about Rendering Quality? [Sigcomm’11]

0 10 20 30 40 50 60 70 80 90 100

Rendering Quality (%)

0 10 20 30 40 50 60 70 80 90 100P

)Rendering Quality (%)

Live LVoD

Why doesn’t play time correlate with rendering quality for Live?

Example Video Site Quality Summary

!   31.68% of views had quality issues

!   32.2% of viewers had recurring quality issues

!   Viewers with good quality watched 1.5X more video than viewers with poor quality

!   Good video quality for all viewers can add 10.9% more minutes of viewed video

!   Good video quality can add $120K more revenue per month*

5.7 0.68

Total Views

!!"#$%&!'()%*+!

!"video did not start

video buffered

video had low resolu4on

video had good quality

0 20 40 60 80 100 120

Minutes per Viewer

Poor Quality Viewers Good Quality Viewers

* Assumes $30 CPM ad every 8 minutes

Total Views = 66,447,919 Total Viewers = 3,291,204 Total Minutes Viewed = 290,260,395

Example Video Site Quality Summary

!   31.68% of views had quality issues

!   32.2% of viewers had recurring quality issues

!   Viewers with good quality watched 1.5X more video than viewers with poor quality

!   Good video quality for all viewers can add 10.9% more minutes of viewed video

!   Good video quality can add $120K more revenue per month*

5.7 0.68

Total Views

!!"#$%&!'()%*+!

!"video did not start

video buffered

video had low resolu4on

video had good quality

0 20 40 60 80 100 120

Minutes per Viewer

Poor Quality Viewers Good Quality Viewers

* Assumes $30 CPM ad every 8 minutes

Total Views = 66,44,79,19 Total Viewers = 3,291,204 Total Minutes Viewed = 290,260,395

1 2 3 4 5 6 7 8 9 Poor experience on 9 different sites

Viewers Return More When Video Is Not Interrupted by Buffering

1% increase in buffering leads to more than 60% loss in audience

1% difference in buffering between two ISPs

68% monthly loss in uniques for ISP with poor performance

CDN Performance Varies Widely

CDNs Vary in Performance over Geographies and Time

•  Used one month aggregated data-‐set

•  Considered 31,744 DMA-‐ASN-‐hours with > 100 views in each CDN

There is no single best CDN across geographies, network, and time

CDNs Vary in Performance over Geographies and Time

100.0%

Washington DC

(Hagerstow

n):CMCS(33657)

aukee:RO

-‐CEN

TRAL(20231)

Green Ba

y -‐ A

ppleton:SCRR

-‐7015(7017)

Denver:ASN

-‐QWEST(20

CharloUe

R-‐11426(11426)

Washington DC

(Hagerstow

n):ASN

-‐CXA

-‐ALL-‐CCI-‐22773-‐

Philade

lphia:VZ

GNI-‐T

RANSIT(19262)

San Diego:SBIS-‐AS(7132)

egas:ASN

-‐CXA

-‐LV-‐13432-‐CB

S(13432)

Madiso

ARTER-‐NET-‐HKY-‐NC(20115)

Indianapolis:SBIS-‐AS(7132)

Providen

ce -‐ New

ford:ASN

-‐CXA

-‐ALL-‐CCI-‐22773-‐RDC

Washington DC

(Hagerstow

n):VZG

NI-‐T

RANSIT(19262)

Harnord & New

:SBIS-‐AS

(7132)

Houston:SBIS-‐AS(7132)

Grand Ra

pids -‐ Kalamazoo

-‐ Ba

Ule Creek:SBIS-‐AS(7132)

Atlanta:BE

-‐NET-‐BLK(6389)

lulu:HAW

AIIAN-‐TELCO

M(36149)

Atlanta:CO

ST-‐7725(7725)

Washington DC

(Hagerstow

n):SPC

S(10507)

ndo -‐ D

aytona Beach -‐ Melbo

RQ-‐W

Denver:CMCS(33652)

Norfolk -‐ Po

outh -‐ New

port New

N-‐CXA

-‐ALL-‐

Saint Lou

ARTER-‐NET-‐HKY-‐NC(20115)

Cincinna4:FU

SE-‐NET(6181)

nix:AS

N-‐QWEST(209)

Dallas -‐ Fort W

orth:VZG

NI-‐T

RANSIT(19262)

burgh:CC

CH-‐3(7016)

Saint Lou

is:SBIS-‐AS(7132)

San Francisco -‐ O

akland

-‐ San Jose:SBIS-‐AS

(7132)

San Diego:RO

-‐WEST(20001)

Greenville -‐ Spartansburg -‐ A

sheville -‐

Kansas City

:SBIS-‐AS

(7132)

Columbu

s -‐ OH:SCRR

-‐10796(10796)

Louisville:INSIGH

T-‐CO

MMUNICAT

IONS-‐CO

RP-‐AS1(36727)

Chicago:AT

T-‐INTERN

ET3(6478)

Kansas City

-‐CXA

(22773)

Miami -‐ Fort Laude

rdale:BE

-‐NET-‐BLK(6389)

Cleveland:NEO

-‐RR-‐CO

M(11060)

Chicago:VZ

GNI-‐T

RANSIT(19262)

Columbia -‐ SC:SCRR

-‐11426(11426)

Dallas -‐ Fort W

orth:SBIS-‐AS

(7132)

Detroit:S

BIS-‐AS

(7132)

Kansas City

R-‐11955(11955)

ngeles:CHA

-‐NET-‐HKY-‐NC(20115)

Miami -‐ Fort Laude

rdale:CO

ST-‐20214(20214)

Cleveland:SCRR

-‐10796(10796)

West P

alm Beach -‐ Fort Pierce:CO

ST-‐20214(20214)

San Diego:AS

N-‐CXA

(22773)

le -‐ Tacoma:AS

N-‐QWEST(209)

York:CA

BLE-‐NET-‐1(6128)

Portland

-‐ OR:AS

N-‐QWEST(209)

Oklahom

a City:SBIS-‐AS

(7132)

nix:AS

N-‐CXA

(22773)

Chicago:SBIS-‐AS(7132)

Greensbo

ro -‐ High Point -‐ Winston

-‐Salem

R-‐11426

Albany -‐ Sche

nectady -‐ T

roy:RR

-‐NYSRE

GION-‐ASN

-‐01

Tyler -‐ Lon

gview (Luq

in & Nacogdo

ches):S

ENLINK-‐

Grand Ra

pids -‐ Kalamazoo

-‐ Ba

Ule Creek:CM

CS(33668)

Houston:CM

CS(33662)

Harnord & New

:COMCA

ST-‐7015(7015)

Eugene

-‐QWEST(209)

San Francisco -‐ O

akland

-‐ San Jose:CMCS(33651)

CDN Streaming Failures Are Common Events

Opportunities for Improving Quality

Refresh: What is high quality?

!   Prevent startup failures

!   Start the video quickly

!   Play the video smoothly and without interruptions

!   Play the video at the highest bit rate possible

Possible Actions to Improve Quality

!   Switch bitrate ↓  Buffering, high frame drops, high start time, … ↑  High available bandwidth, …

!   Switch CDN ↔ Connection error, missing content, buffering on low bitrate, ...

!   Typically two categories of actions !   Start time selection !   Midstream switching

Potential Improvement Example: CDN Switching Only

!   Partition clients by (ASN, DMA, CDN) !   DMA: Designated Market Area

!   For each partition compute: !   Buffering ratio !   Start time !   Failure ratio !   ….

Akamai (buffering ra,o)

Level3 (buffering ra,o)

!   Oracle: !   For each partition select best CDN and

assume all clients in same partition selected that CDN

!   Essentially, pick partition with best quality across CDNs Akamai (buffering ra,o)

Best CDN (buffering ra,o)

!   Essentially, pick partition with best quality across CDNs

!   Historical: !   For each partition select best CDN in

previous epoch, and assign clients to that CDN in next epoch

!   Details !   If a partition has not enough clients

use a larger partition

!   Details !   If a partition has not enough clients

use a larger partition !   Use quality metric distribution to

predict quality of a client on new CDN Akamai (buffering ra,o)

Potential Improvements: Oracle

!   Customer1: large UGV site !   Customer2: large content provider !   Note: * denotes improvements when using mid-stream

switching

Metric Customer1 Customer2

Current Projected Current Projected

Buffering ratio (%) 6.8 2.5 / 1* 1 0.3 / 0.1*

Start time (s) 6.41 2.91 1.36 0.9

Failure ratio (%) 16.57 2.4 1.1 0.7

Between x2.7 and x10 improvement in buffering ratio

Customer1: Start-time vs. Midstream CDN Switching

Customer1: Oracle vs. Historical

0 5 10 15 20

rage b

uffering r

Epoch (hour)

CurrentOracle

Use previous epoch

A Possible Architecture for Delivering High Quality Video Over the Internet

Three Concepts for High Quality Video Delivery

!   Continuous measurement and optimization !   Multi-bit rate streams delivered using multiple CDNs !   Optimization algorithms based on

!   individual client, and

!   aggregate statistics at multiple time scales

Possible Optimization Architecture

Real-‐4me Alerts

Con4nuous real-‐4me measurements from every client

Real-‐4me and historical Insights

Real-‐4me global op4miza4ons

Inference Engine

Bit Rates

Decision Engine

Op4mize viewer performance by selec4ng the best op4on within the set of bit rates and CDNs

Akamai

ASN DMA

Limelight

Level3 Time of day

Localize issues by region, network, CDN, and 4me

Real-‐4me Global Data Aggrega4on and

Correla4on (Streaming Map-‐reduce)

Historical Data Aggrega4on and Analysis

(Hadoop+Hive+Spark) Global Inference, Decision & Policy

Engine

Example: Local vs. Global Optimization

0 2000 4000 6000 8000 10000 12000 14000 16000 18000 20000

width Fluctua4o

Peak Concurrent Viewers

Bandwidth fluctua4on = (Max Bandwidth – Min Bandwidth)/(Average Bitrate)

Example: Local vs. Global Optimization

Akamai DMA

Limelight

Level3

0 2000 4000 6000 8000 10000 12000 14000 16000 18000 20000

width Fluctua4o

0 5000 10000 15000 20000 25000 30000 35000

width Fluctua4o

ASN/DMA saturated on all CDNs Don’t switch CDN; reduce bitrates, instead

Conviva Optimization in the Wild

… increased average bit-rate from 1.7 Mbps to 2.1 Mbps…

Reduced views impacted by buffering from 16.13% to 5.56% …

… and raised engagement by 36%

10.00%

15.00%

20.00%

25.00%

Views Impacted by Buffering

Average Bit Rate

Concluding Remarks

!   We are in the middle of a key transition of main-stream video to the Internet

!   Video quality presents opportunity and challenge !   Follow the traffic: 60% Internet traffic today, will be more than 95% in the

next 2-3 years !   Premium video on big screens zero tolerance for poor quality

!   Video player continuous monitoring and global optimization has

the best chance of delivering high quality video

bringing internet video to prime timenetseminar.stanford.edu/past_seminars/seminars/02_16_12.pdf ·...

Documents

akamai site analyzer_service_description

akamai presentation sydney

akamai dsa sb

2016년 4분기 인터넷 현황 보안 보고서 인포...

akamai q111

2017년 3분기 executive summary - akamai · 2019. 4....

the akamai network

akamai soti q213_exec_summary

akamai data sigcomm04

akamai isms

akamai white paper

akamai cloud security solutions brochure | akamai · akamai...

dma 2006 dma 2006

improving web application...

akamai initiative presentation

akamai marketing group

dma and dma controller 8237

big data intelligence - or katz, akamai and tsvika klein,...

volume 5 issue 3: akamai community news · volume 5 issue...

akamai for m_e