mongodb world 2016: implementing async networking in mongodb 3.2

Implementing Async Networking in MongoDB

Samantha Ritter MongoDB Engineer

Why?

mongosapp

shards

connect auth send recv done

mongosapp

shards

connect auth send recv done

Execution engine

Standalone ASIOhttp://think-async.com/

connect auth send recv done

{work queue

B: send

B: recv

D: auth

A: send

F: done

C++11 lambdas

Constructs a closure: an unnamed function object capable of capturing variables in scope.

auto lambda = [capture list](params) { // body };

…

lambda(); // runs body

send

// the “send” task void send_task(NetworkOp* op) { // pass a lambda to async_send async_send(op->socket, op->command, [op](error_code err) { if (err) { return done(op); } receive_task(op); }); }

connect

auth

recv

send

done

mongos

Network Errors

connect

auth

recv

send

done

mongos

XNetworkError!

Network errors are fine: they are on the primary path of execution

The primary path controlsoperation lifetime

// the “send” task void send_task(NetworkOp* op) { // pass a lambda to async_send async_send(op->socket, op->command, [op](error_code err) { if (err) { return done(op); } receive_task(op); }); }

{work queue

B: send

B: recv

D: auth

A: send

F: done

B: recv

XNetworkError!

clean up B

Cancellations

connect

auth

recv

send

done

mongos

recv !Warning!

cancel job

// the “send” task void send_task(NetworkOp* op) { // pass a lambda to async_send async_send(op->socket, op->command, [op](error_code err) { if (err) { return done(op); } receive_task(op); }); }

Cancellations are NOT fine: they are on the secondary path of execution

On the secondary path we can’t make assumptions about lifetime

Only the primary path can end an operation

Rule of ownership:

// Basic “network operation” class class NetworkOp { bool cancelled; };

// Primary path if (op->cancelled) { done(op); }

// Secondary path cancel(NetworkOp *op) { op->cancelled = true; }

// the “send” task void send_task(NetworkOp* op) { // pass a lambda to async_send async_send(op->socket, op->command, [op](error_code err) { if (err || op->cancelled) return done(op); receive_task(op); }); }

connect

auth

recv

send

done

mongos

Please cancel yourself

Ok!

connect

auth

send

mongos

recvdonePoof!

Please cancel your…

?!@!&?

// Secondary path cancel(NetworkOp *op) { // op could be a null pointer! op->cancelled = true; }

Operation access is protected

Rule of cooperation:

// “network operation” class class NetworkOp { bool cancelled; };

// "access control" object class SafeOp { mutex lock; NetworkOp* op; };

shared_ptr

// Primary path done(shared_ptr<SafeOp> safe) { // lock before cleanup safe->lock.lock(); // cleanup safe->op = nullptr;

safe->lock.unlock(); }

// Secondary path cancel(shared_ptr<SafeOp> safe) { // once we lock, can’t change under us safe->lock.lock(); if (safe->op) { safe->op->cancelled = true; } safe->lock.unlock(); }

connect

auth

send

mongos

recvdonePoof!

Please cancel your…

JK!!

Why?

Threading is better

Engineering Process

1. Iterate!

2. Use language features where possible

3. Use external libraries where appropriate

@SamWhoCodes

mongodb.com/careers

Thanks!