reorganizing website architecture for http/2 and beyond
TRANSCRIPT
Copyright(C)2016DeNACo.,Ltd.AllRightsReserved.
Reorganizing Website Architecture for HTTP/2 and Beyond
DeNA Co., Ltd.Kazuho Oku
Copyright(C)2016DeNACo.,Ltd.AllRightsReserved.
Who am I
n lead developer of H2O HTTP/2 server⁃ one of the most sophisticated HTTP/2 impl.⁃ initial public release: 2014/10 (license: MITL)⁃ used by Fastly, etc.
n author of HTTP performance-related I-Ds⁃ Cache Digests for HTTP/2⁃ 103 Early Hints
n lives in Japan, works for DeNA Co., Ltd.
Reorganizing Website Architecture for HTTP/2 and Beyond
Copyright(C)2016DeNACo.,Ltd.AllRightsReserved.
Current State of HTTP
Reorganizing Website Architecture for HTTP/2 and Beyond
Copyright(C)2016DeNACo.,Ltd.AllRightsReserved.
Why use HTTP/2?
n latency has become the bottleneck of the Webn HTTP/2 to conceal latency by raising concurrency
⁃ 6 concurrent requests in HTTP/1⁃ ~100 in HTTP/2
Reorganizing Website Architecture for HTTP/2 and Beyond
Copyright(C)2016DeNACo.,Ltd.AllRightsReserved.
Current state of HTTP
n HTTP/2 (RFC 7540) released on May 2015
1: https://github.com/HTTPWorkshop/workshop2016/blob/master/talks/http2-review-data.pdf
Reorganizing Website Architecture for HTTP/2 and Beyond
45
28
37
41
18
31
0% 20% 40% 60% 80% 100%
2015/7
2016/7
#oftransac+onsbyFirefox1
HTTP HTTPS(H1) HTTPS(H2)
Copyright(C)2016DeNACo.,Ltd.AllRightsReserved.
Achievements
n header compression (HPACK)⁃ compression ratio: 90% (at median) according to
Mozillan multiplexing
⁃ the tiled benchmark
Reorganizing Website Architecture for HTTP/2 and Beyond
Copyright(C)2016DeNACo.,Ltd.AllRightsReserved.
So is everything optimal now? Not quite.
Reorganizing Website Architecture for HTTP/2 and Beyond
Copyright(C)2016DeNACo.,Ltd.AllRightsReserved.
Issues
n errors in prioritizationn TCP head-of-line blockingn inefficiency of HTTP/2 pushn compression ratio of small responses
Reorganizing Website Architecture for HTTP/2 and Beyond
Copyright(C)2016DeNACo.,Ltd.AllRightsReserved.
Prioritization
Reorganizing Website Architecture for HTTP/2 and Beyond
Copyright(C)2016DeNACo.,Ltd.AllRightsReserved.
イメージを表示できません。メモリ不足のためにイメージを開くことができないか、イメージが破損している可能性があります。コンピューターを再起動して再度ファイルを開いてください。それでも赤い x が表示される場合は、イメージを削除して挿入してください。イメージを表示できません。メモリ不足のためにイメージを開くことができないか、イメージが破損している可能性があります。コンピューターを再起動して再度ファイルを開いてください。それでも赤い x が表示される場合は、イメージを削除して挿入してください。
イメージを表示できません。メモリ不足のためにイメージを開くことができないか、イメージが破損している可能性があります。コンピューターを再起動して再度ファイルを開いてください。それでも赤い x が表示される場合は、イメージを
イメージを表示できません。メモリ不足のためにイメージを開くことができないか、イメージが破損している可能性があります。コンピューターを再起動して再度ファイルを開いてください。それでも赤い x が表示される場合は、イメージを削除して挿入してください。イメージを表示できません。メモリ不足のためにイメージを開くことができないか、イメージが破損している可能性があります。コンピューターを再起動して再度ファイルを開いてください。それでも赤い x が表示される場合は、イメージを削除して挿入してください。
The prioritization tree
Reorganizing Website Architecture for HTTP/2 and Beyond
Root
LeaderG
FollowerGweight:1
HTMLweight:32
Imageweight:22
Imageweight:22
Imageweight:22
CSSweight:32
CSSweight:32
n hybrid approach using weights and chaining⁃ servers are expected to obey to the priority
specified by the clientsn Firefoxʼs prioritization tree is shown below
JSweight:32
JSweight:32
イメージを表示できません。メモリ不足のためにイメージを開くことができないか、イメージが破損している可能性があります。コンピューターを再起動して再度ファイルを開いてください。それでも赤い x が表示される場合は、イメージを削除して挿入してください。
イメージを表示できません。メモリ不足のためにイメージを開くことができないか、イメージが破損している可能性があります。コンピューターを再起動して再度ファイルを開いてください。それでも赤い x が表示される場合は、イメージを削除して挿入してください。
Copyright(C)2016DeNACo.,Ltd.AllRightsReserved.
The prioritization tree
n some web browsers fail to specify priority⁃ Safari, Blink⁃ older versions of Chrome also had issues⁃ server-side countermeasures required
Reorganizing Website Architecture for HTTP/2 and Beyond
イメージを表示できません。メモリ不足のためにイメージを開くことができないか、イメージが破損している可能性があります。コンピューターを再起動して再度ファイルを開いてください。それでも赤い x が表示される場合は、イメージを削除して挿入してください。
イメージを表示できません。メモリ不足のためにイメージを開くことができないか、イメージが破損している可能性があります。コンピューターを再起動して再度ファイルを開いてください。それでも赤い x が表示される場合は、イメージを削除して挿入してください。
イメージを表示できません。メモリ不足のためにイメージを開くことができないか、イメージが破損している可能性があります。コンピューターを再起動して再度ファイルを開いてください。それでも赤い x が表示される場合は、イメージを削除して挿入してください。
イメージを表示できません。メモリ不足のためにイメージを開くことが
イメージを表示できません。メモリ不足のためにイメージを開くことができないか、イメージが破
イメージを表示できません。メモリ不足のためにイメージを開くことができないか、イメージが破損している可能性があります。コンピューターを再起動して再度ファイルを開いてください。それでも赤い x が表示される場合は、イメージを削除して挿入してください。
イメージを表示できません。メモリ不足のためにイメージを開くことができないか、イメージが破損している可能性があります。コンピューターを再起動して再度ファイルを開いてください。それでも赤い x が表示される場合は、イメージを削除して挿入してください。
Root
HTMLweight:16
CSSweight:16
JSweight:16
Imageweight:16
Imageweight:16
Imageweight:16
Copyright(C)2016DeNACo.,Ltd.AllRightsReserved.
HTTP/2 prioritization: the solution
n bandwidth distribution on server-side:⁃ use Weighted Fair Queuing (WFQ) or Deficit
Round Robin (DRR)⁃ some servers do it right:• nghttp2 (and Apache) implements WFQ in O(log N)• H2O approximates WFQ in O(1)
n detect dumb clients and fallback to server-driven prioritization⁃ H2O reprioritizes CSS, JS for clients that do not
use priority chains
Reorganizing Website Architecture for HTTP/2 and Beyond
Copyright(C)2016DeNACo.,Ltd.AllRightsReserved.
HTTP/2 prioritization: benchmark
n differences between the times spent until first-paint (red bar)
Reorganizing Website Architecture for HTTP/2 and Beyond
Copyright(C)2016DeNACo.,Ltd.AllRightsReserved.
TCP head-of-line blocking
Reorganizing Website Architecture for HTTP/2 and Beyond
Copyright(C)2016DeNACo.,Ltd.AllRightsReserved.
Typical sequence of HTTP/2
Reorganizing Website Architecture for HTTP/2 and Beyond
HTTP/2 200 OK
<!DOCTYPE HTML>...<LINK HREF="style.css"...
client server
GET /
GET /style.css
needtoswitchfromsendingHTMLtosendingCSSatthisverymoment,orfromsendingimagestosendingCSSincasetheCSSindiscoveredlate(i.e.hiddenassets)
1RTT
*
Copyright(C)2016DeNACo.,Ltd.AllRightsReserved.
TCP head-of-line blocking
n head-of-line blocking:⁃ high-priority data blocked by preceding data in
flightn TCP head-of-line blocking:
⁃ data in TCP send buffer blocks following data of higher priority
Reorganizing Website Architecture for HTTP/2 and Beyond
Copyright(C)2016DeNACo.,Ltd.AllRightsReserved.
TCP head-of-line blocking
Reorganizing Website Architecture for HTTP/2 and Beyond
n typical H2 server writes much more than that can be sent immediately⁃ unsent data in TCP send buffer (and TLS buffer)
head-of-line blocks following data
TCPsendbuffer
CWNDunacked pollthreshold
TLSbuf.
TLSRecords
sentimmediately notimmediatelysent
HTTP/2frames
Copyright(C)2016DeNACo.,Ltd.AllRightsReserved.
TCP head-of-line blocking: the solution
n write only what can be sent immediately⁃ obtain CWND and unacked size using TCP_INFO
n adjust poll threshold to delay write notification until TCP becomes ready to send some data immediately
Reorganizing Website Architecture for HTTP/2 and Beyond
CWNDunacked pollthreshold
TLSRecords
sentimmediately notimmediatelysent
HTTP/2frames
TCPsendbuffer
Copyright(C)2016DeNACo.,Ltd.AllRightsReserved.
TCP head-of-line blocking: benchmark 1
Reorganizing Website Architecture for HTTP/2 and Beyond
n conditions:⁃ server in Ireland, client in Tokyo (RTT 250ms)⁃ load tiny js at the top of a large HTML
n result: delay decreased from 511ms to 250ms⁃ i.e. JS fetch latency was 2 RTT, became 1 RTT• similar results in other environments
Copyright(C)2016DeNACo.,Ltd.AllRightsReserved.
TCP head-of-line blocking: benchmark 2
n using same data as previousn server and client both in Japan (25ms RTT)
Reorganizing Website Architecture for HTTP/2 and Beyond
0
50
100
150
200
250
300
HTML JS
millisecon
ds�
downloadingHTML(andJSwithin)RTT~25ms�
master latopt
Copyright(C)2016DeNACo.,Ltd.AllRightsReserved.
TCP buffers exist everywhere
n HTTP/2 serverʼs TCP send buffer⁃ the negative impact can be avoided by a clever
HTTP/2 server (as shown by H2O)n TLS terminator
⁃ better to avoid using, if performance mattersn mobile networks
⁃ cannot be controlled by a web-site admin⁃ QUIC would be the final solution
Reorganizing Website Architecture for HTTP/2 and Beyond
Copyright(C)2016DeNACo.,Ltd.AllRightsReserved.
Push
Reorganizing Website Architecture for HTTP/2 and Beyond
Copyright(C)2016DeNACo.,Ltd.AllRightsReserved.
Push
n positive reports:⁃ “20-30% speedup on page render time”2
n negative comments:⁃ many unnecessary pushes (47% are reset2)⁃ increased render time in anti-patterns3
⁃ “consider preload instead of push”3
n what is the cause of these negative views?
2: https://github.com/HTTPWorkshop/workshop2016/blob/master/talks/server-push.pdf3: https://docs.google.com/document/d/1K0NykTXBbbbTlv60t5MyJvXjqKGsCVNYHyLEXIxYMv0/edit
Reorganizing Website Architecture for HTTP/2 and Beyond
Copyright(C)2016DeNACo.,Ltd.AllRightsReserved.
Itʼs the cached resources
n server shouldnʼt push resources already cached⁃ but HTTP/2 doesnʼt provide a way to determine⁃ pushing same resources multiple times is waste
of bandwidthn solution: cache-digests
⁃ now an IETF HTTP WG draft⁃ https://tools.ietf.org/html/draft-ietf-httpbis-
cache-digest-01
Reorganizing Website Architecture for HTTP/2 and Beyond
Copyright(C)2016DeNACo.,Ltd.AllRightsReserved.
Cache-digests
n client sends a digest of cached responses along with an HTTP request
n server examines the digest, pushes only the resources that arenʼt cached
n digest: list of SHA(URL)s stored in Golomb-coded sets (a space-efficient variant of bloom filter)
n size of digest: ~ N*log2(1/P) bits⁃ N: # of resources⁃ P: false-positive-rate
Reorganizing Website Architecture for HTTP/2 and Beyond
Copyright(C)2016DeNACo.,Ltd.AllRightsReserved.
Example
n set false positive rate to: P=1/27
n calculate SHA256 of two URLs (N=2)⁃ https://example.com/style.css (SHA256: baf9…)⁃ https://example.com/script.js (SHA256: 1174…)
n truncate to log2(N*1/P) bits and sort⁃ 0x11, 0xba
n emit log2(N), log2(1/P) in 5 bits, then the deltas of sorted list using Golomb coding (subtracted by 1)⁃ 00001 00111 1 0010001 01 010 1000 N 1/P 0x11 0xa8 (0xba-0x11-1)
Reorganizing Website Architecture for HTTP/2 and Beyond
Copyright(C)2016DeNACo.,Ltd.AllRightsReserved.
Overhead of sending a Cache Digest
n # of fresh responses in browser cache per domain is expected to be <~1K⁃ looking at about:cache• *.facebook.com: 790• *.google.com: 373
n HTTP request + cache-digest can be packed into a single TCP packet⁃ when setting P to 1/128 or 1/256• or, P can be set to a bigger value to conserve space,
or cache-digest could be split into multiple packets following the HTTP request
Reorganizing Website Architecture for HTTP/2 and Beyond
Copyright(C)2016DeNACo.,Ltd.AllRightsReserved.
Implementation Status
n three ways to send a cache-digest⁃ I-D proposes the use of HTTP/2 frame⁃ header-based scheme is TBD• so that websites can implement cache-digest using
ServiceWorker (e.g. cache-digest.js)
1: the listed softwares may have interoperability issues due to implementing different versions of the draft
2: approximating the cache state using cookies has accuracy issues
Reorganizing Website Architecture for HTTP/2 and Beyond
Conveyer ServerImplementa+on ClientImplementa+on
HTTP/2frame nghYp2 N/A
HTTPheader1 Apache,H2O cache-digest.js
HTTPcookie2 H2O notneeded
Copyright(C)2016DeNACo.,Ltd.AllRightsReserved.
but even after solving the never-push-cached-resources issue, push doesnʼt necessarily improve
performance
Reorganizing Website Architecture for HTTP/2 and Beyond
Copyright(C)2016DeNACo.,Ltd.AllRightsReserved.
Benchmark scores
n first-paint does not change⁃ since the browser needs both HTML and CSS⁃ push only changes the order between the two
n less time spent showing a blank page⁃ since unload happens when HTML arrives
Reorganizing Website Architecture for HTTP/2 and Beyond
Copyright(C)2016DeNACo.,Ltd.AllRightsReserved.
so is push an useless technology? no
Reorganizing Website Architecture for HTTP/2 and Beyond
Copyright(C)2016DeNACo.,Ltd.AllRightsReserved.
Three use-cases of H2 push
n prioritizationn push while processing requestn push from edge
Reorganizing Website Architecture for HTTP/2 and Beyond
Copyright(C)2016DeNACo.,Ltd.AllRightsReserved.
Pushing for prioritization
Reorganizing Website Architecture for HTTP/2 and Beyond
client server
GET /
GET/style.css
HTTP/2 200 OK<html><link rel=style.css...
HTTP/2 200 OKbody: ...#title: ...
1. send CSS, JS first2. then send HTML
(can be rendered progressively)
withoutpush
client server
GET /GET /style.css HTTP/2HTTP/2 200 OKbody: ...#title: ...
withpush
HTTP/2 200 OK<html> <link rel=style.css ...
Copyright(C)2016DeNACo.,Ltd.AllRightsReserved.
Push while processing request
n web applications involving DB access, etc.
Reorganizing Website Architecture for HTTP/2 and Beyond
req.�
processrequest�push-asset�
HTML�
push-asset�
push-asset�
push-asset�
req.�
processrequest�
asset�
HTML�
asset�
asset�
asset�
req.�
450ms(5RTT+processing=m
e)�
250ms(1RTT+processing=m
e)�
withoutpush� withpush�
Copyright(C)2016DeNACo.,Ltd.AllRightsReserved.
Push from edge
n CDNsʼ use-case⁃ utilize the conn. while waiting for app. response
Reorganizing Website Architecture for HTTP/2 and Beyond
req.�
push-asset�
HTML�
push-asset�
push-asset�
push-asset�
client� edgeserver(CDN)� app.server�
req.�
HTML�
Copyright(C)2016DeNACo.,Ltd.AllRightsReserved.
Impact of push
n prioritization doesnʼt have much benefit⁃ as discussed
n push-while-processing, and push-from-edge are the real use-cases⁃ but how to use?
Reorganizing Website Architecture for HTTP/2 and Beyond
Copyright(C)2016DeNACo.,Ltd.AllRightsReserved.
Push while processing
Reorganizing Website Architecture for HTTP/2 and Beyond
req.�
processrequest�
push-asset�
HTML�
push-asset�
push-asset�
push-asset�
req.�
processrequest�
asset�
HTML�
asset�
asset�
asset�
req.�
450ms(5RTT+processing=m
e)�
250ms(1RTT+processing=m
e)�
withoutpush� withpush�n One way is to configure the H2 server
mruby.handler: | Proc.new do |env| push_paths = [] if /(\/|\.html)$/.match(env["PATH_INFO"]) push_paths << "/style.css” ... end [399, push_paths.empty? ? {} : { "link" => push_paths.map{|p| "<#{p}>; rel=preload” }.join("\n") }, []] endfastcgi.connect: ...
Copyright(C)2016DeNACo.,Ltd.AllRightsReserved.
Push while processing
Reorganizing Website Architecture for HTTP/2 and Beyond
n or use 103 Early Hints⁃ application sends 103 then processes the request
GET / HTTP/1.1Host: example.com
HTTP/1.1 103 Early HintsLink: </style.css>; rel=preload
HTTP/1.1 200 OKContent-Type: text/htmlLink: </style.css>; rel=preload
<!DOCTYPE HTML>...
HTTP/2server app.server
GET/
103EarlyHintsLink:…
200OK
processrequest
WebBrowser
GET/
200OK
push
Copyright(C)2016DeNACo.,Ltd.AllRightsReserved.
103 Early Hints
n a new I-D (published last week)⁃ https://tools.ietf.org/html/draft-kazuho-early-
hints-status-code-00n the status code indicates that the server is likely to
send a final request with the headers included in the informational response⁃ can be used to hint preloading 3rd party assets
(e.g. ads) as welln already supported by: nghttp2, H2O
⁃ other server / web-application framework developers have shown interest
Reorganizing Website Architecture for HTTP/2 and Beyond
Copyright(C)2016DeNACo.,Ltd.AllRightsReserved.
Push from edge
n at least some CDNs are likely to provide configuration directives at the edge-server level
Reorganizing Website Architecture for HTTP/2 and Beyond
req.�
push-asset�
HTML�
push-asset�
push-asset�
push-asset�
client� edgeserver(CDN)� app.server�
req.�
HTML�
Copyright(C)2016DeNACo.,Ltd.AllRightsReserved.
compression ratio of small responses
Reorganizing Website Architecture for HTTP/2 and Beyond
Copyright(C)2016DeNACo.,Ltd.AllRightsReserved.
File size vs. compression ratio
n claim: no need to concatenate, since H2 improves request concurrency
n reality: small files suffer from low compression ratio⁃ problem for API as well
note: first N bytes of jquery-3.1.1.js compressed with best quality settings
Reorganizing Website Architecture for HTTP/2 and Beyond
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
256 1,024 4,096 16,384 65,536 86,709filesize�
Compressionra/oofaJavaScriptfile�
gzip
brotli
Copyright(C)2016DeNACo.,Ltd.AllRightsReserved.
Two proposals
n Shared Dictionary Compression over HTTP (SDCH)⁃ uses a pre-shared compression dictionary⁃ https://tools.ietf.org/html/draft-lee-sdch-spec-00
n Compression Dictionaries for HTTP/2⁃ re-uses the compression context of a previously
transmitted response⁃ https://tools.ietf.org/html/draft-vkrasnov-h2-
compression-dictionaries-01
Reorganizing Website Architecture for HTTP/2 and Beyond
Copyright(C)2016DeNACo.,Ltd.AllRightsReserved.
Differences between the approaches
n pre-shared dictionary:⁃ achieves high compression ratio when the
dictionary is cached by the client (i.e. when revisiting)
⁃ cannot be used at first visitn re-using compression context of another response:
⁃ works effectively at first visit⁃ easier to use, since no configuration needed⁃ may have security issues• i.e. CRIME attack
Reorganizing Website Architecture for HTTP/2 and Beyond
Copyright(C)2016DeNACo.,Ltd.AllRightsReserved.
Compression ratio of small responses
n beneficial to concatenate small files (to ~32KB or larger) for the time being
Reorganizing Website Architecture for HTTP/2 and Beyond
Copyright(C)2016DeNACo.,Ltd.AllRightsReserved.
Summary
Reorganizing Website Architecture for HTTP/2 and Beyond
Copyright(C)2016DeNACo.,Ltd.AllRightsReserved.
HTTP/2 is not perfect (yet)
n choice of servers / CDNs causes difference⁃ be careful when looking at benchmarks
n avoid using:⁃ TLS terminators⁃ hidden assets (or un-hide them w. preload links)
n use push for either of:⁃ push-while-processing (103 Early Hints)⁃ push-from-edge (configure your CDN)
n continue to concatenate files, at least the tiny ones
Reorganizing Website Architecture for HTTP/2 and Beyond