Contributions

IPv6 PerformanceFailures

Latency

YouTube

Happy Eyeballs

Last-mile Latency

Thanks!

Q/A

Understanding the Impact of Network Infrastructure Changesusing Large Scale Measurement Platforms

Vaibhav BajpaiTU Munich

IM 2017 ConferenceLisbon, Portugal

Dissertation Committee

Jürgen SchönwälderJacobs University Bremen, Germany

Kinga LipskochJacobs University Bremen, Germany

Filip De TurckUniversity of Ghent, Belgium

May 2017

1 / 19

Contributions

IPv6 PerformanceFailures

Latency

YouTube

Happy Eyeballs

Last-mile Latency

Thanks!

Q/A

Survey on Internet Performance Measurement Platforms [1] [COMST ′15]

Measuring IPv6 Performance

▶ Measuring Web Similarity [2] [CNSM ′16]

▶ Measuring TCP Connect Times [3] [NETWORKING ′15]

▶ Measuring YouTube Performance [4] [PAM ′15]

▶ Measuring Effects of Happy Eyeballs [5] [ANRW ′16]

Measuring Access Network Performance

▶ RIPE Atlas Vantage Point Selection [6] [IM ′17]

▶ Dissecting Last-mile Latency Characteristics [∗]

▶ Lessons Learned from using RIPE Atlas [7] [SIGCOMM CCR ′15]

* entries are papers currently under review.

2 / 19

Contributions

IPv6 PerformanceFailures

Latency

YouTube

Happy Eyeballs

Last-mile Latency

Thanks!

Q/A

IPv6 Performance

▶ Literature focus largely on IPv6 adoption.

▶ Very little work on measuring IPv6 performance.

▶ This study closes the gap. 2009 2010 2011 2012 2013 2014 2015 2016 20170%5%10%15%

Google IPv6 Adoption

shaded region represents the duration of the longitudinal study.

We measure from ∼100 dual-stacked SamKnows probes.

NETWORK TYPE #

RESIDENTIAL 78

NREN / RESEARCH 10

BUSINESS / DATACENTER 08

OPERATOR LAB 04

IXP 01

RIR #

RIPE 60

ARIN 29

APNIC 10

AFRINIC 01

LACNIC 01

3 / 19

Contributions

IPv6 PerformanceFailures

Latency

YouTube

Happy Eyeballs

Last-mile Latency

Thanks!

Q/A

Complete Failures

2010 2011 2012 2013 2014 2015 2016 20170.0%10.0%20.0%30.0%40.0%50.0%

W6D

W6LD

ALEXA 1M with AAAA entries

HTTP Failure

▶ Failures reduced from 40% (2009) to 3% today.

0.1K 1K 10K 100K 1000KALEXA Rank

0.00.20.40.60.81.0

CDF

Failing AAAA Websites

4.3K[M

ar '

17]

▶ 88% failing websites rank > 100K.

▶ 1% rank < 10K, six websites rank < 300.

100101102103

www.bing.com

102

103www.detik.com

100101102103

www.engadget.com

102

103www.nifty.com

100101102103104

www.qq.com

Jan2013

Jan2014

Jan2015

Jan2016

Jul Jul Jul102

103www.sakura.ne.jp

IPv6 IPv4

TCP Connect Times (ms)

Metrics should account for changes in IPv6-readiness.

4 / 19

Contributions

IPv6 PerformanceFailures

Latency

YouTube

Happy Eyeballs

Last-mile Latency

Thanks!

Q/A

Partial FailuresALEXA top 100 websites with AAAA entries.

▶ 27% show some rate of failure over IPv6.

▶ 9% exhibit more than 50% failures over IPv6.

0 20 40 60 80 100Success Rate (%)

0.00.20.40.60.81.0

CDF

IPv6 (100)IPv4 (100)

▶ Limiting to root webpage can lead tooverestimation of IPv6 adoption numbers

▶ Unclear whether websites with partialfailures can be deemed IPv6-ready

▶ ISOC now supporting [8] development oftools that identify such partial failures

# Webpage Success Rate (%) W6LDIPv6(↓) IPv4

01 www.bing.com 0 100 302 www.detik.com 0 100 303 www.engadget.com 0 100 304 www.nifty.com 0 100

05 www.qq.com 0 100

06 www.sakura.ne.jp 0 100

07 www.flipkart.com 09 99 308 www.folha.uol.com.br 13 100

09 www.aol.com 48 100 3

10 www.comcast.net 52 100 311 www.yahoo.com 72 100 312 www.mozilla.org 84 100 313 www.orange.fr 86 100 314 www.seznam.cz 89 100 315 www.mobile.de 90 100 316 www.wikimedia.org 90 100

17 www.t-online.de 93 100 318 www.free.fr 95 100

19 www.usps.com 95 100

20 www.vk.com 95 100 321 www.wikipedia.org 95 100 322 www.wiktionary.org 95 100

23 www.elmundo.es 96 100 324 www.uol.com.br 96 100 325 www.marca.com 97 100 326 www.terra.com.br 98 100 327 www.youm7.com 99 100

5 / 19

Contributions

IPv6 PerformanceFailures

Latency

YouTube

Happy Eyeballs

Last-mile Latency

Thanks!

Q/A

Partial Failures | Root Cause Analysis

0 30 60 90www.youm7.com (1%)

www.terra.com.br (2%)www.marca.com (3%)www.uol.com.br (4%)www.elmundo.es (4%)

www.wiktionary.org (5%)www.wikipedia.org (5%)

www.vk.com (5%)www.usps.com (5%)www.free.fr (5%)

www.t-online.de (7%)www.wikimedia.org (10%)

www.mobile.de (10%)www.seznam.cz (11%)www.orange.fr (14%)

www.mozilla.org (16%)www.yahoo.com (28%)

www.comcast.net (48%)www.aol.com (52%)

www.folha.uol.com.br (87%)www.flipkart.com (91%)www.sakura.ne.jp (100%)

www.qq.com (100%)www.nifty.com (100%)

www.engadget.com (100%)www.detik.com (100%)www.bing.com (100%)

Network LevelCURLE_OKCURLE_COULDNT_RESOLVE_HOSTCURLE_COULDNT_CONNECTCURLE_OPERATION_TIMEDOUTCURLE_GOT_NOTHINGCURLE_RECV_ERROR

0 30 60 90Contribution (%)

Content Level

*/css*/html*/javascript, */json*/octet-stream*/plain*/rdf*/xmlimage/*

0 30 60 90

Service Level

SAME ORIGINCROSS ORIGIN

Website failing over IPv6

▶ Failures silently exist; clients do notnotice them due to IPv4 fallback.

▶ Identification of operational issuesrelevant for upcoming IPv6-onlynetworks

▶ Failures due to DNS resolution error on image/*, */javascript, */json and */css content.

▶ 12% websites have more than 50% content that belongs to same-origin source and fails over IPv6,

▶ Content failing from cross-origin sources consists of analytics and third-party advertisements.

6 / 19

Contributions

IPv6 PerformanceFailures

Latency

YouTube

Happy Eyeballs

Last-mile Latency

Thanks!

Q/A

Latency | Websites

∆sa(u) = t4(u) − t6(u)

where t(u) is the time taken to establish TCP connection to website u.

▶ ISPs in early stages of IPv6 deployment shouldensure their CDN caches are dual-stacked.

−150−100−50050

TCP Connect Times [∆sa (ms)]

www.bing.comwww.facebook.com

www.wikipedia.orgwww.youtube.com

2013 2014 2015 2016 2017−60−40−20020

www.blogspot.*www.google.*

www.netflix.comwww.yahoo.com

▶ TCP connect times to popular websites over IPv6 have considerably improved over time.

▶ Inflated latency over IPv6 was due to missing content caches over IPv6

7 / 19

Contributions

IPv6 PerformanceFailures

Latency

YouTube

Happy Eyeballs

Last-mile Latency

Thanks!

Q/A

Latency | Websites - Who connects faster?

ALEXA top 10K websites (as of Jan 2017):

▶ 40% are faster over IPv6.

▶ 94% of the rest are at most 1 ms slower.

▶ 3% are at least 10 ms slower.

▶ 1% are at least 100 ms slower. −1.0 −0.5 0.0 0.5 1.0∆sa (ms)

0.0

0.2

0.4

0.6

0.8

1.0

CDF

netflixyahoo

google

linkedinmicrosoft facebook

wikipedia

cloudflare heise

openstreetmap

ALEXA (10K)

[01/

2017

]

∆sa(u) = t4(u) − t6(u)

▶ Relevant for content providers to get insights on how their service delivery compares over IPv6.

8 / 19

Contributions

IPv6 PerformanceFailures

Latency

YouTube

Happy Eyeballs

Last-mile Latency

Thanks!

Q/A

YouTube

Latency is consistently higher over IPv6.

▶ TCP connect times

▶ < 1 ms slower over IPv6▶ Higher towards webpages

▶ Prebuffering durations

▶ > 25 ms slower over IPv6

▶ Startup delay

▶ > 100 ms slower over IPv6

▶ ISPs should make their GGC nodes dual-stacked.

−5−4−3−2−10

∆t (ms)

TCP Connect Times

Web

−0.4−0.3−0.2−0.10.0

∆t (ms)

TCP Connect Times

AudioVideo

−120−80−400

∆p (ms)

Prebuffering Duration

Oct Jan2015

Apr Jul Oct Jan2016

Apr−400−300−200−100

0

∆s (ms)

Startup Delay

∆t(y) = tc4(y) − tc6(y)∆p(y) = pd4(y) − pd6(y)∆s(y) = sd4(y) − sd6(y)

9 / 19

Contributions

IPv6 PerformanceFailures

Latency

YouTube

Happy Eyeballs

Last-mile Latency

Thanks!

Q/A

Happy Eyeballs

▶ Only ∼1% of samples aboveHE timer value > 300 ms

Samples where HE prefers IPv6 −

▶ HE prefers slower IPv6connections 90% of the time.

▶ HE timer of 150 ms maintainssame IPv6 preference levels.

10-2 10-1 100 101 102 103 104TCP Connect Times (ms)

0.00.20.40.60.81.0

CDF

300

ms

IPv6 (462K)IPv4 (462K)

['13

- '

17]

−40 −30 −20 −10 0 10∆sa (ms)

0.00.20.40.60.81.0

CDF

1% 2% 7%30%

93%99%

462K

['13

- '

17]

▶ RFC 6555 should have used 150 ms timer. Measurements should inform protocol engineering.▶ Drive an RFC 6555 update with operational experience within the IETF.

10 / 19

Contributions

IPv6 PerformanceFailures

Latency

YouTube

Happy Eyeballs

Last-mile Latency

Thanks!

Q/A

Survey on Internet Performance Measurement Platforms [COMST ′15]

Measuring IPv6 Performance

▶ Measuring Web Similarity [CNSM ′16]

▶ Measuring TCP Connect Times [NETWORKING ′15]

▶ Measuring YouTube Performance [PAM ′15]

▶ Measuring Effects of Happy Eyeballs [ANRW ′16]

Measuring Access Network Performance

▶ RIPE Atlas Vantage Point Selection [IM ′17]

▶ Dissecting Last-mile Latency Characteristics [∗]

▶ Lessons Learned from using RIPE Atlas [SIGCOMM CCR ′15]

* entries are papers currently under review.

11 / 19

Contributions

IPv6 PerformanceFailures

Latency

YouTube

Happy Eyeballs

Last-mile Latency

Thanks!

Q/A

Last-mile Latency

▶ Latency becomes a critical factor [9] when downstream throughput > 16 Mb/s.▶ Last-mile latency is a major contributor[9] to end-to-end latency.▶ However, little is known [10, 11] about characteristics of last-mile latency.

▶ 696 RIPE Atlas v3 residential probes (blue)

▶ 1245 SamKnows residential probes (red)

Methodology described to isolate residential probes useful forfuture broadband measurement studies using these platforms.

12 / 19

Contributions

IPv6 PerformanceFailures

Latency

YouTube

Happy Eyeballs

Last-mile Latency

Thanks!

Q/A

Last-mile Latency | Home Network Latency

The home network should not be accounted when measuring last-mile latency.

10-1 100 101 102 103 104hop1/hop2 (%)

0.00.20.40.60.81.0

CDF

Residential Probes

SamKnows (1.1K)RIPE Atlas (0.6K)

▶ hop1 > 10% of hop2 latency (∼19% probes).

Last-mile latency should be the difference between the hop2 and hop1 latency.13 / 19

Contributions

IPv6 PerformanceFailures

Latency

YouTube

Happy Eyeballs

Last-mile Latency

Thanks!

Q/A

Last-mile Latency | Interleaving Depths

▶ DSL networks not only enable interleaving [11] but …▶ …also employ multiple interleaving depth levels that change with time.

1 4 16 640.20.61.0

CDF

FREE

1 4 16 64

FREE

1 4 16 64

PLUSNET

1 4 16 64

TISCALI

29/ 12/ 26/141664

RTT (ms)

29/ 12/ 26/ 05/ 19/ 29/ 12/ 26/

hop1 hop2

▶ Interleaving depths show a step-wise functional change.▶ hop2 latency transitions correlate with corresponding timeseries.

14 / 19

Contributions

IPv6 PerformanceFailures

Latency

YouTube

Happy Eyeballs

Last-mile Latency

Thanks!

Q/A

Last-mile Latency | Time of Day Effects

▶ Last-mile latencies are stable over time.

▶ Last-mile latencies do not exhibit diurnal load patterns.

▶ Simulation studies can now accurately model access links.

▶ CDN providers benefit from characteristics of the last-mile.

▶ Promotes ISPs to cache popular content close to the CPE.

01h05h09h13h17h21h

[35

days

]

01h05h09h13h17h21h

[35

days

]

1 2 4 8 163264Last-mile latency (ms)

01h05h09h13h17h21h

[35

days

]

424424425426425425

[# P

robe

s]

DSL (RIPE Atlas)

223223223223223223

[# P

robe

s]

CABLE (RIPE Atlas)

363636363636

[# P

robe

s]

FIBRE (RIPE Atlas)

15 / 19

Contributions

IPv6 PerformanceFailures

Latency

YouTube

Happy Eyeballs

Last-mile Latency

Thanks!

Q/A

Last-mile Latency | Subscriber Location

▶ Not all cable deployments [10, 11] show last-mile latencies < DSL.

1 2 4 8 16 32 64Last-mile latency (ms)

0.00.20.40.60.81.0

CDF (84 probes) COMCAST (RIPE Atlas)

EST (44)PST (40)

▶ Last-mile latencies:

▶ can depend on geographic location of the subscriber.▶ are considerably different along US east (∼32 ms) and west (∼8 ms) coast.

16 / 19

Contributions

IPv6 PerformanceFailures

Latency

YouTube

Happy Eyeballs

Last-mile Latency

Thanks!

Q/A

Last-mile Latency | Broadband Speeds

Last-mile latencies vary by broadband speeds.

20 21 22 23 24 25 26Last-mile latency (ms)

0.00.20.40.60.81.0

CDF

(306

)

BT (SamKnows)80 Mbps (100)40 Mbps (37)20 Mbps (88)8 Mbps (81) ▶ Input for future standards (QUIC, TLS 1.3) work

that targets operation in 0-RTT mode.

▶ ADSL2+ and VDSL with higher transmission rates help reduce interleaving delays.

▶ Last-mile latencies for VDSL < ADSL/ADSL2+

17 / 19

Contributions

IPv6 PerformanceFailures

Latency

YouTube

Happy Eyeballs

Last-mile Latency

Thanks!

Q/A

This thesis would not have been possible without these amazing people!

12

Sam Crawford

• SamKnows Ltd WP1 leader, Leone Project

• Working on / interested in: – Large scale active measurements, mainly fixed-line,

but also mobile now – Better cross-traffic identification – Integration into existing CPE – LMAP architecture and practical implementations – New metrics (mainly for testing compatibility/viability)

• What’s  missing:  Many  things,  but  in-home measurements in particular

18 / 19

Contributions

IPv6 PerformanceFailures

Latency

YouTube

Happy Eyeballs

Last-mile Latency

Thanks!

Q/A

1. Survey on Internet Performance Measurement Platforms [COMST ′15]

2. Measuring IPv6 Performance

▶ Measuring Web Similarity [CNSM ′16]

▶ Measuring TCP Connect Times [NETWORKING ′15]

▶ Measuring YouTube Performance [PAM ′15]

▶ Measuring Effects of Happy Eyeballs [ANRW ′16]

3. Measuring Access Network Performance

▶ RIPE Atlas Vantage Point Selection [IM ′17]

▶ Dissecting Last-mile Latency Characteristics [∗]

▶ Lessons Learned from using RIPE Atlas [SIGCOMM CCR ′15]

www.vaibhavbajpai.com

bajpaiv@in.tum.de | @bajpaivaibhav

19 / 19

Contributions

IPv6 PerformanceFailures

Latency

YouTube

Happy Eyeballs

Last-mile Latency

Thanks!

Q/A

References[1] V. Bajpai and J. Schönwälder, “A Survey on Internet Performance

Measurement Platforms and Related Standardization Efforts,” ser.IEEE Communications Surveys and Tutorials, 2015. [Online].Available: http://dx.doi.org/10.1109/COMST.2015.2418435

[2] S. J. Eravuchira, V. Bajpai, J. Schönwälder, and S. Crawford,“Measuring Web Similarity from Dual-stacked Hosts,” ser.Conference on Network and Service Management, 2016, pp. 181–187.[Online]. Available: http://dx.doi.org/10.1109/CNSM.2016.7818415

[3] V. Bajpai and J. Schönwälder, “IPv4 versus IPv6 - who connectsfaster?” ser. IFIP Networking Conference, 2015, pp. 1–9. [Online].Available: http://dx.doi.org/10.1109/IFIPNetworking.2015.7145323

[4] S. Ahsan, V. Bajpai, J. Ott, and J. Schönwälder, “Measuring YouTubefrom Dual-Stacked Hosts,” ser. Passive and Active MeasurementConference, 2015, pp. 249–261. [Online]. Available:http://dx.doi.org/10.1007/978-3-319-15509-8_19

[5] V. Bajpai and J. Schönwälder, “Measuring the Effects of HappyEyeballs,” ser. Applied Networking Research Workshop, 2016.[Online]. Available: http://dl.acm.org/citation.cfm?id=2959429

[6] V. Bajpai, S. J. Eravuchira, J. Schönwälder, R. Kisteleki, and E. Aben,“Vantage Point Selection for IPv6 Measurements: Benefits and

Limitations of RIPE Atlas Tags,” ser. International Symposium onIntegrated Network Management (IM), 2017 (to appear).

[7] V. Bajpai, S. J. Eravuchira, and J. Schönwälder, “Lessons LearnedFrom Using the RIPE Atlas Platform for Measurement Research,” ser.Computer Communication Review, vol. 45, no. 3, 2015, pp. 35–42.[Online]. Available: http://doi.acm.org/10.1145/2805789.2805796

[8] “NAT64 Check,” nat64check.ipv6-lab.net, [Accessed 15-Apr-2017].

[9] S. Sundaresan, N. Feamster, R. Teixeira, and N. Magharei, “Measuringand Mitigating Web Performance Bottlenecks in Broadband AccessNetworks,” ser. IMC, 2013. [Online]. Available:http://doi.acm.org/10.1145/2504730.2504741

[10] M. Dischinger, A. Haeberlen, P. K. Gummadi, and S. Saroiu,“Characterizing Residential Broadband Networks,” ser. InternetMeasurement Conference, 2007, pp. 43–56. [Online]. Available:http://doi.acm.org/10.1145/1298306.1298313

[11] S. Sundaresan, W. de Donato, N. Feamster, R. Teixeira, S. Crawford,and A. Pescapè, “Broadband Internet Performance: A View From theGateway,” ser. SIGCOMM, 2011, pp. 134–145. [Online]. Available:http://doi.acm.org/10.1145/2018436.2018452

19 / 19