T o r u M a e s a k a

memcached Binary Protocolin a Nutshell

http://torum.net

memcached Night #1 in TokyoSeptember 17th, 2008

Binary Protocol

A new protocol that will beintroduced from version 1.3

For those that aren’tfamiliar…

bradfitz++

The original memcachedprotocol is a very simpleand small, yet an effectivetext based protocol.

Then why a new protocol?

• Performance• Extensibility

• Performance

Protocol parsing can beperformed millions of times in

a given instance’s lifetime.

• Performance

Protocol parsing canhappen thousands

of times per second.

• Performance

Parse Text String

Parse Binary Stringvs

• Performance

Ingredient for afaster Protocol

Parser :)

Benchmark

• Performance

memslap v1.0Concurrent GET OP on 100,000

pre-loaded items

• Performance

• Performance

Binary ProtocolPerformed better than

ASCII with moreconcurrent access(more parse/sec)

Extensibility

• Extensibility

Standard support ofadditional data in the

protocol.

• Extensibility

Able to give memcachedmore information about

the request

• Extensibility

Opens up newpossibilities

So… a…

New Protocol

Worried that yourapplication would break

after upgrading?

Don’t ;)

memcached-1.3 isBilingual!

Understands both ASCIIand the Binary Protocol

Specification

CommunityDriven

Specification

Packet Structure

EXTRA FIELD(Command Specific)

HEADER

KEY

VALUE

24 bytes Required

As neededSizes aredefined inthe header

* Values must be in network byte order.

Packet Structure

HEADER

Two types of headers:• Request Header• Response Header

Inside the Request Header

MAGIC(1 byte)

Key Length(2 bytes)

Opcode(1 byte)

Extra Length(1 byte)

Data Type(1 byte)

Reserved(2 bytes)

Total Body Length(4 bytes)

Opaque(4 bytes)

CAS (Compare and Swap)(8 bytes)

Inside the Response Header

MAGIC(1 byte)

Key Length(2 bytes)

Opcode(1 byte)

Extra Length(1 byte)

Data Type(1 byte)

Status(2 bytes)

Total Body Length(4 bytes)

Opaque(4 bytes)

CAS (Compare and Swap)(8 bytes)

MAGIC Field• Distinguish between Response and Request messages

• Check the validity of the packet

0x80 = Request Packet0x81 = Response Packet

OPCODERepresents the Command Type

0x00 => Get0x01 => Set0x02 => Add0x03 => Replace0x04 => Delete0x05 => Increment0x06 => Decrement0x07 => Quit

0x08 => Flush0x09 => GetQ0x0A => No-op0x0B => Version0x0C => GetK0x0D => GetKQ0x0E => Append0x0F => Prepend0x11 => Stats

Key Length• Length of the key in bytes

Note that the key length fieldis 2 bytes, so theoretically a

key can be up to 65,536characters long!

Extra Length

Length of the ‘extra’field in the packet

Data Type

Reserved forfuture use

Status

Status of theresponse

(e.g. if an error had occurred or not)

Total Body Length

Length of extra field,key, and value

combined in bytes.

Opaque

(like echoing)

The same opaque valuein the request is copiedback in the response.

CAS

A 64bit uint for theCompare and Swap

check.

Much moreinformation in the

binary protocoldocumentation

http://github.com/trondn/memcached/tree/binprot/doc/protocol-binary.txt

Latest SpecificationFile available from

Trond Norbye’s tree:

Thank You!and…

for providing thefacilities for

memcached Night #1