Download - Communication in Python and the C10k problem
Jose Ignacio Galarza @igalarzab
Communication in Python and the C10k problem
MANOLOOOOO!!!!
select the best method to
communicate
select the best method to scale
Index✤ From polling… to pushing ✤ Concurrency ✤ C10k ✤ Asynchronous I/O
PYTHON 3
PYTHON 3 EVERYWHERE
From polling to pushing
polling
Client Server
polling➡ Nothing new to implement ➡ Good support in all platforms ➡ Proxy friendly ➡ Uni-directional ➡ Very low efficiency ➡ No cross-domain
while True: r = requests.get('http://api.yo.com/messages') messages.extend(r.json()) time.sleep(5)
[email protected]("/messages") def messages(): message = check_new_message() return message
long polling
ServerClient
long polling➡ Good support in all platforms ➡ “Proxy friendly” ➡ Uni-directional ➡ Low efficiency (better than polling) ➡ No cross-domain
long polling
while True: try: r = requests.get('http://yo.com/messages', timeout=60) except TimeOut: continue
messages.extend(r.json())
@app.route("/messages") def messages(): message = wait_until_new_message() return message
HTTP Streaming
ServerClient
HTTP Streaming➡ Better efficiency than (long) polling ➡ “Proxy friendly” ➡ Uni-directional ➡ You need to parse the data manually
HTTP Streaming➡ Type 1: Connection Close
HTTP/1.1 200 OK Content-Type: text/plain Connection: close
Hello world This a connection close response
HTTP Streaming➡ Type 2: Chunked response
HTTP/1.1 200 OK Content-Type: text/plain Transfer-Encoding: chunked
E Hello World!
19 I am a chunked response
0
HTTP Streaming
r = requests.get('http://yo.com/messages', stream=True) for message in r.iter_lines(): messages.extend(message.decode(‘utf8’))
@app.route("/messages") def messages(): def content(): for sentence in wait_until_next_sentence(): yield sentence
return Response(content())
SSE
ServerClient
SSE➡ Well known protocol (HTTP) ➡ Good efficiency ➡ JS API ➡ Few client implementations ➡ Uni-directional
WebSockets
ServerClient
WebSockets➡ Bidirectional ➡ Great efficiency ➡ A lot of implementations (JS API) ➡ Cross-domain ➡ Handshake to update from HTTP ➡ Complete different protocol
GET /chat HTTP/1.1 Host: server.example.com Upgrade: websocket Connection: Upgrade Sec-WebSocket-Key: x3JJHMbDL1EzLkh9GBhXDw== Sec-WebSocket-Protocol: chat, superchat Sec-WebSocket-Version: 13 Origin: http://example.com
WS Handshake
HTTP/1.1 101 Switching Protocols Upgrade: websocket Connection: Upgrade Sec-WebSocket-Accept: HSmrc0sMlYUkAGmm5OPpG2HaGWk= Sec-WebSocket-Protocol: chat
Wikipedia
WebSockets
def msg(ws, msg): messages.extend(msg)
ws = websocket.WebSocketApp(‘ws://api.yo.com/', on_message=msg) ws.on_open = on_open ws.run_forever()
@sockets.route('/messages') def messages(ws): while True: message = wait_until_new_message() ws.send(message)
Bytes IN Bytes Out Total Bytes Time (seconds)
Polling 14640 13564 28204 159
Polling 2 7503 7636 15139 200
Long Polling 7503 7636 15139 120
Streaming 183 1549 1732 120
WebSockets 209 1605 1814 120
0
3000
6000
9000
12000
15000
18000
21000
24000
27000
30000
Polling
Polling 2
Long Polling
Streaming
WebSocketsTotal Bytes Bytes Out Bytes IN
Concurrency and the
C10k
C1how to handle more than
10k connections simultaneously?
0k?
concurrency is…
the ability of running in overlapping time periods,
not necessarily at the same time
concurrency !=
parallelism
processes➡ Preemptive scheduling by the OS ➡ Separate memory space ➡ No GIL related issues ➡ How to communicate them? ➡ They are really heavy
threads➡ Preemptive scheduling by the OS ➡ Same memory space ➡ Faster and lighter than processes ➡ You may suffer the GIL ➡ What about synchronisation? ➡ Race conditions
user threads➡ Same space address ➡ Lightest alternative ➡ No race-conditions ➡ You may suffer the GIL ➡ Very bad with CPU bound tasks
GIL
I/O methods
again…how to handle more than
10k connections simultaneously?
option 1
use 10k machines, one per client, VIP service
option 2use blocking and
synchronous calls, one per client
option 3non-blocking calls to start
I/O and then readiness notifications to know
when the socket is ready
option 4asynchronous calls to
start I/O and completion notifications to know when they’ve finished
but… in Python?
twistedtornadogevent
asyncio (aka tulip)
asyncio➡ Single-threaded ➡ Async I/O ➡ Coroutines ➡ Multiplexes I/O events ➡ …
COROUTINES
websockets
asyncio
demo
XGame Of Life
10K connections
questions?@igalarzab
thanks!