content delivery - in a devops world

1

Leif Hedstrom

Content Delivery- in a DevOps World

Chief Architect, Cisco Cloud Services

July 8, 2012

2

Who am I?• One of the drivers to Open Source Traffic Server• Committer for Apache Traffic Server• VP of Apache Traffic Server PMC• ASF member• Chief Architect, Cisco Cloud Services

zwoop@apache.org

lhedstro@cisco.com

@zwoop

+zwoop

3

Why DevOps? A short story

4

It’s a disaster!

http://www.flickr.com/photos/historyinanhour/4775644390/

5

Smart people trying to figure it out…

6

Still a disaster, oh the humanity!

http://www.flickr.com/photos/historyinanhour/4775644390/

7

In the mean time …

http://people.apache.org/~weijin/weijin.jpg

8

Small fix, big results

index 30ed932..b74defa 100644--- a/iocore/net/UnixNetVConnection.cc+++ b/iocore/net/UnixNetVConnection.cc@@ -574,6 +574,9 @@ UnixNetVConnection::do_io_close(int alerrno /* = -1 */ ) write.vio.nbytes = 0; write.vio.op = VIO::NONE; + EThread *t = this_ethread();+ bool close_inline = !recursion && nh->mutex->thread_holding == t;+ INK_WRITE_MEMORY_BARRIER; if (alerrno && alerrno != -1) this->lerrno = alerrno;@@ -582,11 +585,8 @@ UnixNetVConnection::do_io_close(int alerrno /* = -1 */ ) else closed = -1; - if (!recursion) {- EThread *t = this_ethread();- if (nh->mutex->thread_holding == t)- close_UnixNetVConnection(this, t);- }+ if (close_inline)+ close_UnixNetVConnection(this, t); }

9

I’m happy!

http://www.flickr.com/photos/mhedstrom/6763754839/

10

DevOps reward (well, maybe … )

11

So what is DevOps?• To me, it’s a collaborative, community driven

way of developing software, services and IT solutions.

• Tear down the borders (and rifts!) between Development and Operations.

• Thrives on healthy Open Source communities.

12

13

14

QPS is no longer a meaningful metric

15

…Other things matters…

16

… and don’t forget features• Handling thousands of concurrent users• Extensible (e.g. plugins, or scripts)• Correctness• HTTP/1.1

• Keep-Alive• Chunking• etc.

17

When all you have is a hammer …

http://www.flickr.com/photos/aai/6936657289/

18

Evaluate the features that matters to you!ATS HAproxy ngin

xSquid Varnish mod_proxy

Worker Threads Y N N N Y Y

Multi-Process N Y Y N Y YEvent-driven Y Y Y Y sometimes YPlugin APIs Y N Y part Y YForward Proxy Y N N Y N YReverse Proxy Y Y Y Y Y YTransp. Proxy Y Y N Y N NLoad Balancer weak Y Y Y Y YPersistent Cache Y N Y Y sorta YESI Y N N Y kinda NICP Y N N Y N NKeep-Alive Y N Y Y Y YSSL Y N Y Y N YPipeline Y N Y Y N Y

19

Forward proxy

20

Transparent (or intercepting) proxy

21

Reverse proxy

22

23

Enemies of all content delivery

Server latency3-Way handshake

Congestion controlDNS lookups

(Page flow)

24

Enemies of all content delivery

Server latency3-Way handshake

Congestion controlDNS lookups

(Page flow)

25

TCP 3-way handshake

26

Enemies of all content delivery

Server latency3-Way handshake

Congestion controlDNS lookups

(Page flow)

27

Congestion control

28

Enemies of all content delivery

Server latency3-Way handshake

Congestion controlDNS lookups

(Page flow)

29

http://news.example.com/http://finance.example.com/http://groups.example.com/

30

http://news.example.com/http://finance.example.com/http://groups.example.com/

vshttp://www.example.com/newshttp://www.example.com/financehttp://www.example.com/groups

31

gzip matters

http://www.flickr.com/photos/marcovdz/4520986339/

32

Reduce your round-trips!

plain gzipTheoretical limit on 20Mbps 78ms 15msLow latency (17ms RTT) 182ms 105msHigh latency (159ms RTT) 1,340ms 840ms

33

Congestion control

34

Real example (200KB uncompressed)

200KB (8 RTs) 40KB (5 RTs)17ms RTT 136ms 85ms159ms RTT 1,272ms 795ms

35

Keep-alive• A primary tool to fight 3-way handshake

latency• Helps against congestion control too

• But not always… Slow starts begins again after some idle time

36

Old Congestion Window Size Algorithm:

IW = min (4*SMSS, max (2*SMSS, 4380 bytes))

(RFC2581)

Typically, 2-4 packets

37

New Linux Initial Congestion Window Size Algorithm:

10

38

SPDY• Addresses many of the problems related to

latency• Not a standard (yet)• SSL requirements, and other quirks, can be a

hurdle

39

40

Some history

41

The Concurrency problem

42

Solution 1: Multi-threading

43

Problems with Multi-threading

44

Solution 2: Event Processing

45

Problems with event processing

46http://www.flickr.com/photos/stuartpilbrow/3345896050/

47

Traffic Server model

48

Four Horseman of Bad Performance• Data copies• Context Switches• Memory allocation• Lock contention

49http://www.flickr.com/photos/simpologist/16734948/

50

http://www.flickr.com/photos/airosan/2232394342/

51

http://trafficserver.apache.org/

Thank you.