url to html

URL to HTMLa minute in the life of a webpage

François Marier @fmarier mozilla

https://twitter.com/fmarier

mozilla newmarket

8 engineers

8 engineers

1 designer

8 engineers

1 designer

1 manager

video & mediamarketplacecloud servicescrash investigation

> 1,000 employees world-wide

1. Learn HTML

2. ?

3. Profit !

abstractions

asbtractionconstruct used to understand acomplicated topic at a high level

asbtractionextra layer added to avoid writingthe same code over and over

drawSquare()

drawLine()

drawSquare()

if you don't understand thelayers below, you won't know whatto do when the abstraction breaks

masteryrequires a high-level understandingof the rest of the stack

web performance

web performancehow bytes make it to the user

web performancehow bytes make it to the userhow the browser renders the page

URL

DNS

IP

TCP

HTTP / TLS

HTML

URLuniform ressource locator

http://www.example.com

http://www.example.com/articles/

http://www.example.com /articles/tutorial.cgi

http://www.example.com/articles/tutorial.cgi?showsolutions=0&topic=web

http://www.example.com/articles/tutorial.cgi?showsolutions=0&topic=web#part5

http://www.example.com:80 /articles/tutorial.cgi?showsolutions=0&topic=web#part5

http://username:[email protected]:80 /articles/tutorial.cgi?showsolutions=0&topic=web#part5

DNSdomain name system

$ cat /etc/resolv.conf

nameserver 208.67.222.222nameserver 208.67.220.220

www.cs.auckland.ac.nz

www.cs.auckland.ac.nz

130.216.158.22

$ dig nz NS @199.7.83.42

$ dig nz NS @199.7.83.42

; <<>> DiG 9.8.1-P1 <<>> nz NS @199.7.83.42;; global options: +cmd;; Got answer:;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 41219;; flags: qr rd; QUERY: 1, ANSWER: 0, AUTHORITY: 7, ADDITIONAL: 12;; WARNING: recursion requested but not available

;; QUESTION SECTION:;nz. IN NS

;; AUTHORITY SECTION:nz. 172800 IN NS ns1.dns.net.nz.nz. 172800 IN NS ns2.dns.net.nz.nz. 172800 IN NS ns3.dns.net.nz.nz. 172800 IN NS ns4.dns.net.nz.nz. 172800 IN NS ns5.dns.net.nz.nz. 172800 IN NS ns6.dns.net.nz.nz. 172800 IN NS ns7.dns.net.nz.

;; ADDITIONAL SECTION:ns1.dns.net.nz. 172800 IN A 202.46.190.130ns2.dns.net.nz. 172800 IN A 202.46.187.130ns3.dns.net.nz. 172800 IN A 202.46.188.130ns4.dns.net.nz. 172800 IN A 202.46.189.130ns5.dns.net.nz. 172800 IN A 156.154.100.14ns6.dns.net.nz. 172800 IN A 156.154.101.14ns7.dns.net.nz. 172800 IN A 194.146.106.54ns1.dns.net.nz. 172800 IN AAAA 2001:dce:2000:2::130ns2.dns.net.nz. 172800 IN AAAA 2001:dce:7000:2::130ns5.dns.net.nz. 172800 IN AAAA 2001:502:ad09::14ns6.dns.net.nz. 172800 IN AAAA 2001:502:2eda::14ns7.dns.net.nz. 172800 IN AAAA 2001:67c:1010:13::53

;; Query time: 20 msec;; SERVER: 199.7.83.42#53(199.7.83.42);; WHEN: Thu May 22 15:27:37 2014;; MSG SIZE rcvd: 406

$ dig ac.nz NS @ns1.dns.net.nz

$ dig ac.nz NS @ns1.dns.net.nz

; <<>> DiG 9.8.1-P1 <<>> ac.nz NS @ns1.dns.net.nz;; global options: +cmd;; Got answer:;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 39124;; flags: qr aa rd; QUERY: 1, ANSWER: 7, AUTHORITY: 0, ADDITIONAL: 12;; WARNING: recursion requested but not available

;; QUESTION SECTION:;ac.nz. IN NS

;; ANSWER SECTION:ac.nz. 86400IN NS ns7.dns.net.nz.ac.nz. 86400IN NS ns4.dns.net.nz.ac.nz. 86400IN NS ns2.dns.net.nz.ac.nz. 86400IN NS ns1.dns.net.nz.ac.nz. 86400IN NS ns6.dns.net.nz.ac.nz. 86400IN NS ns3.dns.net.nz.ac.nz. 86400IN NS ns5.dns.net.nz.

;; ADDITIONAL SECTION:ns1.dns.net.nz. 86400IN A 202.46.190.130ns1.dns.net.nz. 86400IN AAAA 2001:dce:2000:2::130ns2.dns.net.nz. 86400IN A 202.46.187.130ns2.dns.net.nz. 86400IN AAAA 2001:dce:7000:2::130ns3.dns.net.nz. 86400IN A 202.46.188.130ns4.dns.net.nz. 86400IN A 202.46.189.130ns5.dns.net.nz. 86400IN A 156.154.100.14ns5.dns.net.nz. 86400IN AAAA 2001:502:ad09::14ns6.dns.net.nz. 86400IN A 156.154.101.14ns6.dns.net.nz. 86400IN AAAA 2001:502:2eda::14ns7.dns.net.nz. 86400IN A 194.146.106.54ns7.dns.net.nz. 86400IN AAAA 2001:67c:1010:13::53


$ dig auckland.ac.nz NS @ns1.dns.net.nz

$ dig auckland.ac.nz NS @ns1.dns.net.nz

; <<>> DiG 9.8.1-P1 <<>> auckland.ac.nz NS @ns1.dns.net.nz;; global options: +cmd;; Got answer:;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 59814;; flags: qr rd; QUERY: 1, ANSWER: 0, AUTHORITY: 3, ADDITIONAL: 2;; WARNING: recursion requested but not available

;; QUESTION SECTION:;auckland.ac.nz. IN NS

;; AUTHORITY SECTION:auckland.ac.nz. 86400IN NS pubsec.domainz.net.nz.auckland.ac.nz. 86400IN NS dns1.auckland.ac.nz.auckland.ac.nz. 86400IN NS dns2.auckland.ac.nz.

;; ADDITIONAL SECTION:dns1.auckland.ac.nz.86400IN A 130.216.1.2dns2.auckland.ac.nz.86400IN A 130.216.1.1


$ dig cs.auckland.ac.nz NS @dns1.auckland.ac.nz

$ dig cs.auckland.ac.nz NS @dns1.auckland.ac.nz

; <<>> DiG 9.8.1-P1 <<>> cs.auckland.ac.nz NS @dns1.auckland.ac.nz;; global options: +cmd;; Got answer:;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 4856;; flags: qr aa rd; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 6;; WARNING: recursion requested but not available

;; QUESTION SECTION:;cs.auckland.ac.nz. IN NS

;; ANSWER SECTION:cs.auckland.ac.nz. 10800 IN NS dns2.auckland.ac.nz.cs.auckland.ac.nz. 10800 IN NS kronos2.cs.auckland.ac.nz.cs.auckland.ac.nz. 10800 IN NS dns1.auckland.ac.nz.cs.auckland.ac.nz. 10800 IN NS kronos1.cs.auckland.ac.nz.

;; ADDITIONAL SECTION:dns1.auckland.ac.nz.1800 IN A 130.216.1.2dns2.auckland.ac.nz.1800 IN A 130.216.1.1kronos1.cs.auckland.ac.nz. 10800 IN A 130.216.35.35kronos1.cs.auckland.ac.nz. 10800 IN AAAA 2001:df0:0:2003::35kronos2.cs.auckland.ac.nz. 10800 IN A 130.216.35.135kronos2.cs.auckland.ac.nz. 10800 IN AAAA 2001:df0:0:2003::135


$ dig www.cs.auckland.ac.nz @kronos1.cs.auckland.ac.nz

$ dig www.cs.auckland.ac.nz @kronos1.cs.auckland.ac.nz

; <<>> DiG 9.8.1-P1 <<>> www.cs.auckland.ac.nz A @kronos1.cs.auckland.ac.nz;; global options: +cmd;; Got answer:;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 17586;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 4, ADDITIONAL: 6;; WARNING: recursion requested but not available

;; QUESTION SECTION:;www.cs.auckland.ac.nz. IN A

;; ANSWER SECTION:www.cs.auckland.ac.nz. 10800 IN A 130.216.158.22

;; AUTHORITY SECTION:cs.auckland.ac.nz. 10800IN NS kronos2.cs.auckland.ac.nz.cs.auckland.ac.nz. 10800IN NS dns2.auckland.ac.nz.cs.auckland.ac.nz. 10800IN NS dns1.auckland.ac.nz.cs.auckland.ac.nz. 10800IN NS kronos1.cs.auckland.ac.nz.

;; ADDITIONAL SECTION:dns1.auckland.ac.nz.1800 IN A 130.216.1.2dns2.auckland.ac.nz.1800 IN A 130.216.1.1kronos1.cs.auckland.ac.nz. 10800 IN A 130.216.35.35kronos1.cs.auckland.ac.nz. 10800 IN AAAA 2001:df0:0:2003::35kronos2.cs.auckland.ac.nz. 10800 IN A 130.216.35.135kronos2.cs.auckland.ac.nz. 10800 IN AAAA 2001:df0:0:2003::135


l.root-servers.net

ns1.dns.net.nz

ns1.dns.net.nz

dns1.auckland.ac.nz

kronos1.cs.auckland.ac.nz

IPinternet protocol

www.bbc.co.uk

212.58.246.94

$ mtr 212.58.246.94

1. 192.168.1.1 2. llu.bng1.tvc.orcon.net.nz 3. xe-3-3-0.cre1.sky.orcon.net.nz 4. 121.98.9.137 5. ae1-0.cre2.nct.odyssey.net.nz 6. ORCON-INTER.bar1.SanFrancisco1.Level3.net 7. xe-5-0-0.bar1.SanFrancisco1.Level3.net 8. ae-0-11.bar2.SanFrancisco1.Level3.net 9. ae-6-6.ebr2.SanJose1.Level3.net10. ae-62-62.csw1.SanJose1.Level3.net11. ae-61-61.ebr1.SanJose1.Level3.net12. ae-2-2.ebr2.NewYork1.Level3.net13. ae-62-62.csw1.NewYork1.Level3.net14. ae-61-61.ebr1.NewYork1.Level3.net15. ae-43-43.ebr2.London1.Level3.net16. ae-57-222.csw2.London1.Level3.net17. ae-229-3605.edge4.London1.Level3.net18. BBC-TECHNOL.edge4.London1.Level3.net19. ???20. ???21. ae0.er01.cwwtf.bbc.co.uk22. 132.185.255.16523. bbc-vip015.cwwtf.bbc.co.uk

130.216.158.22

212.58.246.94

130.216.158.22

212.58.246.94

router drops packetspackets arrive in wrong order

130.216.158.22

212.58.246.94

router drops packets

cable is cutpackets arrive in wrong order

130.216.158.22

212.58.246.94

router drops packets

cable is cut

packets arrive in wrong order

ideal network actual network

TCPtransmission control protocol

guarantees in-orderdelivery of packets

abstraction of a reliable point-to-pointconnection with built-in re-try logic

applications have a lotless errors to deal with

UDPuser datagram protocol

TCP UDP

reminder: abstractions are leaky

3-way handshakeestablishing a new connection

how are you?

good, you?

client server

clientSYNx=42

server

clientSYNx=42

SYN+ACK

y=10,x=43

server

clientSYNx=42

SYN+ACK

y=10,x=43

ACKy=11

server

HTTPhypertext transfer protocol

http://www.example.com

clear text protocol

client

request

server

client

request

response

server

Host: www.example.com User-Agent: Mozilla/5.0 (rv:29.0) Firefox/29.0DNT: 1

Host: www.example.com User-Agent: Mozilla/5.0 (rv:29.0) Firefox/29.0DNT: 1

<request body goes in here>

Content-Type: text/htmlDate: Thu, 22 May 2014 05:34:47 GMTLast-Modified: Fri, 09 Aug 2013 23:54:35 GMTContent-Length: 1270

<!doctype html><html><head> <title>Example Domain</title></head><body><div> <h1>Example Domain</h1> <p>This domain is established to be used for illustrative examples in documents. You may use this domain in examples without prior coordination or asking for permission.</p></div></body></html>

200 OK

404 Not Found

$ curl http://www.example.com

<!doctype html><html><head> <title>Example Domain</title></head>

<body><div> <h1>Example Domain</h1> <p>This domain is established to be used for illustrative examples in documents. You may use this domain in examples without prior coordination or asking for permission.</p></div></body></html>

$ curl --head http://www.example.com

HTTP/1.1 200 OKAccept-Ranges: bytesCache-Control: max-age=604800Content-Type: text/htmlDate: Thu, 22 May 2014 05:42:26 GMTEtag: "359670651"Expires: Thu, 29 May 2014 05:42:26 GMTLast-Modified: Fri, 09 Aug 2013 23:54:35 GMTServer: ECS (cpm/F858)X-Cache: HITx-ec-custom-error: 1Content-Length: 1270

verbs(fancy word for commands)

GET /article/43228

GET /article/43228GET /article/43228

GET /article/43228

POST /article/delete/last

GET /article/43228

POST /article/delete/lastPOST /article/delete/last

GET /article/43228

POST /item/20/buyPOST /item/20/buy

$$$$$$

TLStransport layer security

SSLsecure sockets layer

HTTPShypertext transfer protocol secure

secure(sometimes)

client server(pk, sk)

clienthello!

server(pk, sk)

clienthello!

hello!pubkey

server(pk, sk)

clienthello!

hello!pubkey

server(pk, sk)

session key

clienthello!

hello!pubkey

i'm done!encryptpk(session key)

server(pk, sk)

session key

clienthello!

hello!pubkey

i'm done!encryptpk(session key)

server(pk, sk)

session key

session key

man-in-the-middle


NSA(pk, sk)

clienthello!

server(pk, sk)

NSA(pk, sk)

clienthello!

server(pk, sk)

NSA(pk, sk)

hello!

clienthello!

hello!pubkey

server(pk, sk)

NSA(pk, sk)

hello!

clienthello!

hello!pubkey

server(pk, sk)

NSA(pk, sk)

hello!

hello!pubkey

clienthello!

hello!pubkey

server(pk, sk)

key

NSA(pk, sk)

hello!

hello!pubkey

clienthello!

hello!pubkey

i'm done!encrypt(key)

server(pk, sk)

key

NSA(pk, sk)

hello!

hello!pubkey

clienthello!

hello!pubkey


server(pk, sk)

key

NSA(pk, sk)

hello!

hello!pubkey

key

clienthello!

hello!pubkey


server(pk, sk)

key

NSA(pk, sk)

hello!

hello!pubkey

i'm done!encrypt(key)key

clienthello!

hello!pubkey


server(pk, sk)

key

NSA(pk, sk)

hello!

hello!pubkey

i'm done!encrypt(key)keykey

authentication(of the server)

clienthello!

hello!pubkey

server(pk, sk)

session key

clienthello!

hello!

signed pubkey

server(pk, sk)

session key

verifysignature

clienthello!

hello!

signed pubkey

server(pk, sk)

session key

verifysignature

i'm done!encryptpk(session key) session key

clienthello!

hello!signed pubkey

server(pk, sk)

NSA(pk, sk)

hello!

hello!signed pubkeykey

clienthello!

hello!signed pubkey

server(pk, sk)

NSA(pk, sk)

hello!

hello!signed pubkeykey

abort!

how can you tell you'retalking to the right person?

(and not to the NSA)

trusted third-partycertificate authority

EFF has foundmore than

650certificate

authoritiesin the wild

*.google.com

*.google.com 7 different domains

$1,000

$100

$1,000$1,000

$100

HTMLhypertext markup language

parsing

.png

.jpg .js .css

resolve all hostnames

establish TCP connections

negotiate TLS session

URL

DNS

IP

TCP

HTTP / TLS

HTML

@fmarier [email protected]

questions?

https://twitter.com/fmarier

Copyright © 2014 Francois Marier <[email protected]>This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

leaky pipe: https://www.flickr.com/photos/ifl/3920636654leaky pipe with elephant: https://www.flickr.com/photos/rcrhee/10785374875sky tower: https://www.flickr.com/photos/elisfanclub/6120863439golden gate: https://www.flickr.com/photos/jeffgunn/6663212147san jose: https://www.flickr.com/photos/the_tahoe_guy/3183673224statue of liberty: https://www.flickr.com/photos/suewaters/7574642942big ben: https://www.flickr.com/photos/timmorris/3103896345bbc house: https://www.flickr.com/photos/redvers/532073098fingers crossed: https://www.flickr.com/photos/bearpark/6861722073prince charles : http://en.wikipedia.org/wiki/File:Prince_Charles_2012.jpgsouthern cross cable: https://en.wikipedia.org/wiki/File:Southern_Cross_Cable_route.svg

image credits

http://creativecommons.org/licenses/by-sa/4.0/

https://www.flickr.com/photos/ifl/3920636654

https://www.flickr.com/photos/rcrhee/10785374875

https://www.flickr.com/photos/elisfanclub/6120863439

https://www.flickr.com/photos/jeffgunn/6663212147

https://www.flickr.com/photos/the_tahoe_guy/3183673224

https://www.flickr.com/photos/suewaters/7574642942

https://www.flickr.com/photos/timmorris/3103896345

https://www.flickr.com/photos/redvers/532073098

https://www.flickr.com/photos/bearpark/6861722073

http://en.wikipedia.org/wiki/File:Prince_Charles_2012.jpg

https://en.wikipedia.org/wiki/File:Southern_Cross_Cable_route.svg

url to html

Internet