bringing internet video to prime timenetseminar.stanford.edu/past_seminars/seminars/02_16_12.pdf ·...

Bringing Internet Video to Prime Time Ion Stoica UC Berkeley / Conviva February 16, 2012

Background: Conviva !   Co-founded with Hui Zhang (CMU) in fall, 2006

!   Conviva optimizes video quality for premium content properties !   E.g., HBO, ESPN, ABC, Disney, and Turner

! Conviva's powered some of the world's largest on-line events !   E.g., Olympics, FIFA World Cup, NCAA College Basketball March

Madness, Major League Baseball, and Academy Awards.

Internet Video Ecosystem : Video Data-plane

Video Source

Encoders & Video Servers

CMS and Hos4ng Content Delivery

Networks (CDN)

ISP & Home Net

Screen

Video Player

Key components: Video Player & CDN

Trends

Trend: CDN Pricing

!   CDN pricing has decreased x1.5-2 every year for the last 6 years

0

5

10

15

20

25

30

35

40

45

2006 2007 2008 2009 2010 2011

cents/GB

Trend: Bitrate for Premium Content

!   Average bitrate has increased 20-40% every year

0 200 400 600 800 1000 1200 1400 1600 1800

2006 2007 2008 2009 2010 2011

Kbps

Trend: Per-hour Streaming Cost

!   Per-hour streaming cost has decreased 15-30% every year

0 1 2 3 4 5 6 7 8 9 10

2006 2007 2008 2009 2010 2011

cents/ho

ur

HTTP Chunking

!   Trends fueled by switching from proprietary streaming technologies (e.g., Adobe’s FMS) to HTTP chunking !   Move Networks (2005)

!   Apple (2008)

!   Microsoft (2009)

!   Adobe (2011)

How does HTTP Chunking Work?

Client A

CDN1

Client B

CDN2

ISP A ISP B

Client C

… origin

hUp cache

Why HTTP Chunking?

!   Scale: more HTTP caches/servers than streaming servers

!   Cost: HTTP servers cheaper than proprietary streaming protocol

!   Reachability: HTTP traffic penetrates enterprises firewalls easier than proprietary streaming protocols

!   High bitrate: can use multiple TCP connections to download subsequent chunks in parallel

What Does this Mean?

!   Ad supported premium content !   CPM (cost per thousand of ad impressions): $20-$40

!   One ad covers one hour of streaming!

!   Paid content !   $0.99 episode, distribution cost < 2%

!   Subscription based premium content !   It costs less than $1/month to stream content to an user watching 2

hour per day

!   Production & rights dominate!

Quality Matters

We’ve seen patterns across many sites … and billions of streams

Video Player Monitoring : Player Model

Stopped/

Exit

Player

States Joining Playing Buffering Playing

Network/

stream

connection

established

Video

buffer

filled upVideo

buffer

empty

Buffer

replenished

sufficiently

time

User

action

Events

Player

Monitoring

Video download rate,

Available bandwidth,

Dropped frames,

Frame rendering rate, etc.

JoinTime (JT) BufferingRa2o(BR) RateOfBuffering(RB)

AvgBitrate(AB)

RenderingQuality(RQ)

Video Player Monitoring : Data Collection

Automa/c Monitoring

•  Automa4c and consistent monitoring of default streaming modules –  Flash: NetStream, VideoElement –  Silverlight: MediaElement, SmoothStreamingMediaElement –  iOS: MPMoviePlayerController

Streaming Module

UI Controller

Content Manager

Messaging & Serializa/on

Player Insight

To backend

HTTPS

Player App

lica4

on

Empirical Study [Sigcomm’11]

!   Goal: impact of quality on engagement (play time)

!   A week of data from multiple premium video sites

!   There genres: Live, Long VoD (LVoD), Short VoD

!   Five quality metrics: !   Buffering Ratio (BR)

!   Rate of Buffering (RB)

!   Join/Start Time (JT)

!   Rendering Quality (RQ)

!   Average Bit rate (AR)

!   Two granularities: view/viewers

Highlight of Results [Sigcomm’11]

!   Quality has substantial impact on engagement

!   Buffering ratio is most critical across genres !   Highest impact for live: 1% of buffering reduced play time by 3min

!   Bitrate more important for live than VoD

!   Join time impact engagement at viewer level but less at view level

LVoD at View Level [Sigcomm’11]

Buffering ratio correlates with engagement the most

LVoD vs. Live [Sigcomm’11]

Buffering Ratio remains most significant, but Bitrate and Rate of Buffering bigger impact

Live LVoD

Engagement vs. Buffering Ratio [Sigcomm’11]

!

!

!

!

!

!

!

!

!

!

!!

!!

!!!

!!!!!!!!

!!

!!!!!!!!!!!!!!!!!!!!!!!

!!!!!!!!!!!!!!!!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

0 20 40 60 80 100

010

2030

40

Buffering ratio (%)

Play

time (

min)

Correlation coefficient (kendall): −0.96, slope: −3.25

1% increase in buffering reduces engagement by 3 min

Engagement vs. Join Time: Viewer Level [Sigcomm’11]

0 10 20 30 40 50 60 70Join time (s)

0

10

20

30

40

50

60

Tota

lpla

ytim

e(m

in)

Correlation coefficient (kendall): -0.74

Join time critical for user retention

What about Rendering Quality? [Sigcomm’11]

0

5

10

15

20

25

30

35

40

45

50

0 10 20 30 40 50 60 70 80 90 100

Pla

y T

ime (

min

)

Rendering Quality (%)

0

5

10

15

20

25

30

35

40

45

50

0 10 20 30 40 50 60 70 80 90 100P

lay

Tim

e (

min

)Rendering Quality (%)

Live LVoD

Why doesn’t play time correlate with rendering quality for Live?

Example Video Site Quality Summary

!   31.68% of views had quality issues

!   32.2% of viewers had recurring quality issues

!   Viewers with good quality watched 1.5X more video than viewers with poor quality

!   Good video quality for all viewers can add 10.9% more minutes of viewed video

!   Good video quality can add $120K more revenue per month*

68.3

25.3

5.7 0.68

Total Views

!!"#$%&!'()%*+!

!"video did not start

video buffered

video had low resolu4on

video had good quality

100.9

62.2

0 20 40 60 80 100 120

Minutes per Viewer

Poor Quality Viewers Good Quality Viewers

* Assumes $30 CPM ad every 8 minutes

Total Views = 66,447,919 Total Viewers = 3,291,204 Total Minutes Viewed = 290,260,395

67.8

32.2

Example Video Site Quality Summary

!   31.68% of views had quality issues

!   32.2% of viewers had recurring quality issues

!   Viewers with good quality watched 1.5X more video than viewers with poor quality

!   Good video quality for all viewers can add 10.9% more minutes of viewed video

!   Good video quality can add $120K more revenue per month*

68.3

25.3

5.7 0.68

Total Views

!!"#$%&!'()%*+!

!"video did not start

video buffered

video had low resolu4on

video had good quality

100.9

62.2

0 20 40 60 80 100 120

Minutes per Viewer

Poor Quality Viewers Good Quality Viewers

* Assumes $30 CPM ad every 8 minutes

Total Views = 66,44,79,19 Total Viewers = 3,291,204 Total Minutes Viewed = 290,260,395

0

10

20

30

40

50

1 2 3 4 5 6 7 8 9 Poor experience on 9 different sites

Viewers Return More When Video Is Not Interrupted by Buffering

1% increase in buffering leads to more than 60% loss in audience

1% difference in buffering between two ISPs

68% monthly loss in uniques for ISP with poor performance

CDN Performance Varies Widely

CDNs Vary in Performance over Geographies and Time

•  Used one month aggregated data-‐set

•  Considered 31,744 DMA-‐ASN-‐hours with > 100 views in each CDN

25%

50%

25%

There is no single best CDN across geographies, network, and time

CDNs Vary in Performance over Geographies and Time

0.0%

10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

70.0%

80.0%

90.0%

100.0%

Washington DC

(Hagerstow

n):CMCS(33657)

Milw

aukee:RO

ADRU

NNER

-‐CEN

TRAL(20231)

Green Ba

y -‐ A

ppleton:SCRR

-‐7015(7017)

Denver:ASN

-‐QWEST(20

9)

CharloUe

:SCR

R-‐11426(11426)

Washington DC

(Hagerstow

n):ASN

-‐CXA

-‐ALL-‐CCI-‐22773-‐

Philade

lphia:VZ

GNI-‐T

RANSIT(19262)

San Diego:SBIS-‐AS(7132)

Las V

egas:ASN

-‐CXA

-‐LV-‐13432-‐CB

S(13432)

Madiso

n:CH

ARTER-‐NET-‐HKY-‐NC(20115)

Indianapolis:SBIS-‐AS(7132)

Providen

ce -‐ New

Bed

ford:ASN

-‐CXA

-‐ALL-‐CCI-‐22773-‐RDC

Washington DC

(Hagerstow

n):VZG

NI-‐T

RANSIT(19262)

Harnord & New

Haven

:SBIS-‐AS

(7132)

Houston:SBIS-‐AS(7132)

Grand Ra

pids -‐ Kalamazoo

-‐ Ba

Ule Creek:SBIS-‐AS(7132)

Atlanta:BE

LLSO

UTH

-‐NET-‐BLK(6389)

Hono

lulu:HAW

AIIAN-‐TELCO

M(36149)

Atlanta:CO

MCA

ST-‐7725(7725)

Washington DC

(Hagerstow

n):SPC

S(10507)

Orla

ndo -‐ D

aytona Beach -‐ Melbo

urne

:EMBA

RQ-‐W

NPK

Denver:CMCS(33652)

Norfolk -‐ Po

rtsm

outh -‐ New

port New

s:AS

N-‐CXA

-‐ALL-‐

Saint Lou

is:CH

ARTER-‐NET-‐HKY-‐NC(20115)

Cincinna4:FU

SE-‐NET(6181)

Phoe

nix:AS

N-‐QWEST(209)

Dallas -‐ Fort W

orth:VZG

NI-‐T

RANSIT(19262)

PiUs

burgh:CC

CH-‐3(7016)

Saint Lou

is:SBIS-‐AS(7132)

San Francisco -‐ O

akland

-‐ San Jose:SBIS-‐AS

(7132)

San Diego:RO

ADRU

NNER

-‐WEST(20001)

Greenville -‐ Spartansburg -‐ A

sheville -‐

Kansas City

:SBIS-‐AS

(7132)

Columbu

s -‐ OH:SCRR

-‐10796(10796)

Louisville:INSIGH

T-‐CO

MMUNICAT

IONS-‐CO

RP-‐AS1(36727)

Chicago:AT

T-‐INTERN

ET3(6478)

Kansas City

:ASN

-‐CXA


(22773)

Miami -‐ Fort Laude

rdale:BE

LLSO

UTH

-‐NET-‐BLK(6389)

Cleveland:NEO

-‐RR-‐CO

M(11060)

Chicago:VZ

GNI-‐T

RANSIT(19262)

Columbia -‐ SC:SCRR

-‐11426(11426)

Dallas -‐ Fort W

orth:SBIS-‐AS

(7132)

Detroit:S

BIS-‐AS

(7132)

Kansas City

:SCR

R-‐11955(11955)

Los A

ngeles:CHA

RTER

-‐NET-‐HKY-‐NC(20115)

Miami -‐ Fort Laude

rdale:CO

MCA

ST-‐20214(20214)

Cleveland:SCRR

-‐10796(10796)

West P

alm Beach -‐ Fort Pierce:CO

MCA

ST-‐20214(20214)

San Diego:AS

N-‐CXA


(22773)

SeaU

le -‐ Tacoma:AS

N-‐QWEST(209)

New

York:CA

BLE-‐NET-‐1(6128)

Portland

-‐ OR:AS

N-‐QWEST(209)

Oklahom

a City:SBIS-‐AS

(7132)

Phoe

nix:AS

N-‐CXA


(22773)

Chicago:SBIS-‐AS(7132)

Greensbo

ro -‐ High Point -‐ Winston

-‐Salem

:SCR

R-‐11426

Albany -‐ Sche

nectady -‐ T

roy:RR

-‐NYSRE

GION-‐ASN

-‐01

Tyler -‐ Lon

gview (Luq

in & Nacogdo

ches):S

UDD

ENLINK-‐

Grand Ra

pids -‐ Kalamazoo

-‐ Ba

Ule Creek:CM

CS(33668)

Houston:CM

CS(33662)

Harnord & New

Haven

:COMCA

ST-‐7015(7015)

Eugene

:ASN

-‐QWEST(209)

San Francisco -‐ O

akland

-‐ San Jose:CMCS(33651)

CDN Streaming Failures Are Common Events

Opportunities for Improving Quality

Refresh: What is high quality?

!   Prevent startup failures

!   Start the video quickly

!   Play the video smoothly and without interruptions

!   Play the video at the highest bit rate possible

Possible Actions to Improve Quality

!   Switch bitrate ↓  Buffering, high frame drops, high start time, … ↑  High available bandwidth, …

!   Switch CDN ↔ Connection error, missing content, buffering on low bitrate, ...

!   Typically two categories of actions !   Start time selection !   Midstream switching

Potential Improvement Example: CDN Switching Only

!   Partition clients by (ASN, DMA, CDN) !   DMA: Designated Market Area


!   Partition clients by (ASN, DMA, CDN) !   DMA: Designated Market Area

!   For each partition compute: !   Buffering ratio !   Start time !   Failure ratio !   ….

Akamai (buffering ra,o)

DMA

ASN

Level3 (buffering ra,o)

DMA

ASN


!   Oracle: !   For each partition select best CDN and

assume all clients in same partition selected that CDN


DMA

ASN


DMA

ASN




!   Essentially, pick partition with best quality across CDNs Akamai (buffering ra,o)

DMA

ASN


DMA

ASN

Best CDN (buffering ra,o)

DMA

ASN




!   Essentially, pick partition with best quality across CDNs

!   Historical: !   For each partition select best CDN in

previous epoch, and assign clients to that CDN in next epoch


DMA

ASN


DMA

ASN


!   Details !   If a partition has not enough clients

use a larger partition

?


DMA

ASN


!   Details !   If a partition has not enough clients

use a larger partition !   Use quality metric distribution to

predict quality of a client on new CDN Akamai (buffering ra,o)

DMA

ASN


DMA

ASN

Potential Improvements: Oracle

!   Customer1: large UGV site !   Customer2: large content provider !   Note: * denotes improvements when using mid-stream

switching

Metric Customer1 Customer2

Current Projected Current Projected

Buffering ratio (%) 6.8 2.5 / 1* 1 0.3 / 0.1*

Start time (s) 6.41 2.91 1.36 0.9

Failure ratio (%) 16.57 2.4 1.1 0.7

Between x2.7 and x10 improvement in buffering ratio

Customer1: Start-time vs. Midstream CDN Switching

78%

84%

90%

Customer1: Oracle vs. Historical

0.01

0.02

0.03

0.04

0.05

0.06

0.07

0 5 10 15 20

Ave

rage b

uffering r

atio

Epoch (hour)

CurrentOracle

Use previous epoch

A Possible Architecture for Delivering High Quality Video Over the Internet

Three Concepts for High Quality Video Delivery

!   Continuous measurement and optimization !   Multi-bit rate streams delivered using multiple CDNs !   Optimization algorithms based on

!   individual client, and

!   aggregate statistics at multiple time scales

Possible Optimization Architecture

Real-‐4me Alerts

Con4nuous real-‐4me measurements from every client

Real-‐4me and historical Insights

Real-‐4me global op4miza4ons

Inference Engine

Bit Rates

CDNs

Decision Engine

Op4mize viewer performance by selec4ng the best op4on within the set of bit rates and CDNs

Akamai

DMA

ASN

DMA

ASN DMA

ASN

Limelight

Level3 Time of day

Localize issues by region, network, CDN, and 4me

Real-‐4me Global Data Aggrega4on and

Correla4on (Streaming Map-‐reduce)

Historical Data Aggrega4on and Analysis

(Hadoop+Hive+Spark) Global Inference, Decision & Policy

Engine

Example: Local vs. Global Optimization

0

20

40

60

80

100

0 2000 4000 6000 8000 10000 12000 14000 16000 18000 20000

Band

width Fluctua4o

n

Peak Concurrent Viewers

Bandwidth fluctua4on = (Max Bandwidth – Min Bandwidth)/(Average Bitrate)

Example: Local vs. Global Optimization

Akamai DMA

ASN

DMA

ASN

DMA

ASN

Limelight

Level3

0

20

40

60

80

100

0 2000 4000 6000 8000 10000 12000 14000 16000 18000 20000

Band

width Fluctua4o

n


0

10

20

30

40

0 5000 10000 15000 20000 25000 30000 35000

Band

width Fluctua4o

n


ASN/DMA saturated on all CDNs Don’t switch CDN; reduce bitrates, instead

Conviva Optimization in the Wild

… increased average bit-rate from 1.7 Mbps to 2.1 Mbps…

Reduced views impacted by buffering from 16.13% to 5.56% …

… and raised engagement by 36%

0.00%

5.00%

10.00%

15.00%

20.00%

25.00%

1600

1700

1800

1900

2000

2100

2200

Views Impacted by Buffering

Average Bit Rate

Concluding Remarks

!   We are in the middle of a key transition of main-stream video to the Internet

!   Video quality presents opportunity and challenge !   Follow the traffic: 60% Internet traffic today, will be more than 95% in the

next 2-3 years !   Premium video on big screens zero tolerance for poor quality

!   Video player continuous monitoring and global optimization has

the best chance of delivering high quality video

bringing internet video to prime timenetseminar.stanford.edu/past_seminars/seminars/02_16_12.pdf ·...

Documents