messagepack - an efficient binary serialization format
TRANSCRIPT
![Page 1: MessagePack - An efficient binary serialization format](https://reader033.vdocument.in/reader033/viewer/2022051504/5a64a0447f8b9a52568b4ec7/html5/thumbnails/1.jpg)
MESSAGEPACK - AN EFFICIENT
BINARY SERIALIZATION FORMAT
Larry Nung
![Page 2: MessagePack - An efficient binary serialization format](https://reader033.vdocument.in/reader033/viewer/2022051504/5a64a0447f8b9a52568b4ec7/html5/thumbnails/2.jpg)
AGENDA
Introduction
Format
MsgPack.Cli
Messagepack VS Protobuf
Reference
Q & A
2
![Page 3: MessagePack - An efficient binary serialization format](https://reader033.vdocument.in/reader033/viewer/2022051504/5a64a0447f8b9a52568b4ec7/html5/thumbnails/3.jpg)
INTRODUCTION
3
![Page 4: MessagePack - An efficient binary serialization format](https://reader033.vdocument.in/reader033/viewer/2022051504/5a64a0447f8b9a52568b4ec7/html5/thumbnails/4.jpg)
INTRODUCTION
An efficient binary serialization format
Like JSON. but fast and small.
4
![Page 5: MessagePack - An efficient binary serialization format](https://reader033.vdocument.in/reader033/viewer/2022051504/5a64a0447f8b9a52568b4ec7/html5/thumbnails/5.jpg)
FORMAT
5
![Page 6: MessagePack - An efficient binary serialization format](https://reader033.vdocument.in/reader033/viewer/2022051504/5a64a0447f8b9a52568b4ec7/html5/thumbnails/6.jpg)
FORMAT
6
![Page 7: MessagePack - An efficient binary serialization format](https://reader033.vdocument.in/reader033/viewer/2022051504/5a64a0447f8b9a52568b4ec7/html5/thumbnails/7.jpg)
FORMAT
7
![Page 8: MessagePack - An efficient binary serialization format](https://reader033.vdocument.in/reader033/viewer/2022051504/5a64a0447f8b9a52568b4ec7/html5/thumbnails/8.jpg)
FORMAT
format name first byte (in binary) first byte (in hex)
positive fixint 0xxxxxxx 0x00 - 0x7f
fixmap 1000xxxx 0x80 - 0x8f
fixarray 1001xxxx 0x90 - 0x9f
fixstr 101xxxxx 0xa0 - 0xbf
nil 11000000 0xc0
(never used) 11000001 0xc1
false 11000010 0xc2
true 11000011 0xc3
bin 8 11000100 0xc4
bin 16 11000101 0xc5
bin 32 11000110 0xc6 8
![Page 9: MessagePack - An efficient binary serialization format](https://reader033.vdocument.in/reader033/viewer/2022051504/5a64a0447f8b9a52568b4ec7/html5/thumbnails/9.jpg)
FORMAT
format name first byte (in binary) first byte (in hex)
ext 8 11000111 0xc7
ext 16 11001000 0xc8
ext 32 11001001 0xc9
float 32 11001010 0xca
float 64 11001011 0xcb
uint 8 11001100 0xcc
uint 16 11001101 0xcd
uint 32 11001110 0xce
uint 64 11001111 0xcf
int 8 11010000 0xd0
int 16 11010001 0xd1 9
![Page 10: MessagePack - An efficient binary serialization format](https://reader033.vdocument.in/reader033/viewer/2022051504/5a64a0447f8b9a52568b4ec7/html5/thumbnails/10.jpg)
FORMAT
format name first byte (in binary) first byte (in hex)
int 32 11010010 0xd2
int 64 11010011 0xd3
fixext 1 11010100 0xd4
fixext 2 11010101 0xd5
fixext 4 11010110 0xd6
fixext 8 11010111 0xd7
fixext 16 11011000 0xd8
str 8 11011001 0xd9
str 16 11011010 0xda
str 32 11011011 0xdb
array 16 11011100 0xdc 10
![Page 11: MessagePack - An efficient binary serialization format](https://reader033.vdocument.in/reader033/viewer/2022051504/5a64a0447f8b9a52568b4ec7/html5/thumbnails/11.jpg)
FORMAT
format name first byte (in binary) first byte (in hex)
array 32 11011101 0xdd
map 16 11011110 0xde
map 32 11011111 0xdf
negative fixint 111xxxxx 0xe0 - 0xff
11
![Page 12: MessagePack - An efficient binary serialization format](https://reader033.vdocument.in/reader033/viewer/2022051504/5a64a0447f8b9a52568b4ec7/html5/thumbnails/12.jpg)
FORMAT
12
Fixed length types
Integer
Floating point
Boolean
Nil
![Page 13: MessagePack - An efficient binary serialization format](https://reader033.vdocument.in/reader033/viewer/2022051504/5a64a0447f8b9a52568b4ec7/html5/thumbnails/13.jpg)
FORMAT
13
Nil => 11000000 => 0xc0
Boolean
False => 11000010 => 0xc2
True => 11000011 => 0xc3
Positive fixint
0 => 00000000 => 0x00
1 => 00000001 => 0x01
![Page 14: MessagePack - An efficient binary serialization format](https://reader033.vdocument.in/reader033/viewer/2022051504/5a64a0447f8b9a52568b4ec7/html5/thumbnails/14.jpg)
FORMAT
14
Variable length types
Raw bytes
Array
Map
![Page 15: MessagePack - An efficient binary serialization format](https://reader033.vdocument.in/reader033/viewer/2022051504/5a64a0447f8b9a52568b4ec7/html5/thumbnails/15.jpg)
FORMAT
Fixstr
Compact => 10100007 01100011 01101111 01101101
01110000 01100001 01100011 01110100 => a7 63 6f 6d
70 61 63 74
15
![Page 16: MessagePack - An efficient binary serialization format](https://reader033.vdocument.in/reader033/viewer/2022051504/5a64a0447f8b9a52568b4ec7/html5/thumbnails/16.jpg)
MSGPACK.CLI
16
![Page 17: MessagePack - An efficient binary serialization format](https://reader033.vdocument.in/reader033/viewer/2022051504/5a64a0447f8b9a52568b4ec7/html5/thumbnails/17.jpg)
MSGPACK.CLI
Install-Package MsgPack.Cli
17
![Page 18: MessagePack - An efficient binary serialization format](https://reader033.vdocument.in/reader033/viewer/2022051504/5a64a0447f8b9a52568b4ec7/html5/thumbnails/18.jpg)
MSGPACK.CLI
using MsgPack.Serialization;
...
public static byte[] Serialize<T>(T thisObj) {
var serializer = SerializationContext.Default.GetSerializer<T>();
using (var ms = new MemoryStream()) {
serializer.Pack(ms, thisObj);
return ms.ToArray();
}
}
public static T Deserialize<T>(byte[] bytes) {
var serializer = SerializationContext.Default.GetSerializer<T>();
using (var byteStream = new MemoryStream(bytes)) {
return serializer.Unpack(byteStream);
}
} 18
![Page 19: MessagePack - An efficient binary serialization format](https://reader033.vdocument.in/reader033/viewer/2022051504/5a64a0447f8b9a52568b4ec7/html5/thumbnails/19.jpg)
MSGPACK.CLI
public class OldPerson {
[MessagePackMember(0)]
public String Name { get; set; }
[MessagePackMember(1)]
public String NickName { get; set; }
[MessagePackIgnore]
public Object Tag { get; set; }
}
public class NewPerson {
[MessagePackMember(2)]
public String ID { get; set; }
[MessagePackMember(0)]
public String Name { get; set; }
[MessagePackMember(1)]
public String NickName { get; set; }
}
19
![Page 20: MessagePack - An efficient binary serialization format](https://reader033.vdocument.in/reader033/viewer/2022051504/5a64a0447f8b9a52568b4ec7/html5/thumbnails/20.jpg)
MSGPACK.CLI
…
var larry = new OldPerson {
Name = "Larry Nung",
NickName = "Larry"
};
var bytes = Serialize(larry);
var person = Deserialize<NewPerson>(bytes);
Console.WriteLine("{0} ({1})", person.NickName,
person.Name);
…20
![Page 21: MessagePack - An efficient binary serialization format](https://reader033.vdocument.in/reader033/viewer/2022051504/5a64a0447f8b9a52568b4ec7/html5/thumbnails/21.jpg)
MESSAGEPACK VS PROTOBUF
21
![Page 22: MessagePack - An efficient binary serialization format](https://reader033.vdocument.in/reader033/viewer/2022051504/5a64a0447f8b9a52568b4ec7/html5/thumbnails/22.jpg)
MESSAGEPACK VS PROTOBUF
private static void SerializeToStream<T>(Stream stream, T obj) {
stream.Seek(0, SeekOrigin.Begin);
Serializer.Serialize(stream, obj);
}
private static T DeSerializeFromStream<T>(Stream stream) {
stream.Seek(0, SeekOrigin.Begin);
return Serializer.Deserialize<T>(stream);
}
public static void Serialize<T>(Stream stream, T thisObj) {
stream.Seek(0, SeekOrigin.Begin);
var serializer = SerializationContext.Default.GetSerializer<T>();
serializer.Pack(stream, thisObj);
}
public static T Deserialize<T>(Stream stream) {
stream.Seek(0, SeekOrigin.Begin);
var serializer = SerializationContext.Default.GetSerializer<T>();
return serializer.Unpack(stream);
}22
![Page 23: MessagePack - An efficient binary serialization format](https://reader033.vdocument.in/reader033/viewer/2022051504/5a64a0447f8b9a52568b4ec7/html5/thumbnails/23.jpg)
MESSAGEPACK VS PROTOBUF
var counts = new int[] { 100, 1000, 10000, 100000, 1000000, 10000000, 100000000 };
...
Serialize(ms, larry);
Console.WriteLine("MsgPack Size: {0}", ms.Length);
SerializeToStream(ms, larry);
Console.WriteLine("ProtoBuf Size: {0}", ms.Length);
...
foreach (var count in counts)
{
…
Console.WriteLine("MsgPack Serialize: {0} ms", DoTest(count, () => { Serialize(ms, larry); }));
Console.WriteLine("ProtoBuf Serialize: {0} ms", DoTest(count, () =>{ SerializeToStream(ms, larry); }));
Console.WriteLine("MsgPack DeSerialize: {0} ms", DoTest(count, () =>{ Deserialize<Person>(ms); }));
Console.WriteLine("ProtoBuf DeSerialize: {0} ms", DoTest(count, () =>{ DeSerializeFromStream<Person>(ms); }));
}
23
![Page 24: MessagePack - An efficient binary serialization format](https://reader033.vdocument.in/reader033/viewer/2022051504/5a64a0447f8b9a52568b4ec7/html5/thumbnails/24.jpg)
MESSAGEPACK VS PROTOBUF
24
![Page 25: MessagePack - An efficient binary serialization format](https://reader033.vdocument.in/reader033/viewer/2022051504/5a64a0447f8b9a52568b4ec7/html5/thumbnails/25.jpg)
MESSAGEPACK VS PROTOBUF
Serialize
250
5000
10000
15000
20000
25000
30000
35000
40000
45000
100 1000 10000 100000 1000000 10000000 100000000
MsgPack
Protobuf
![Page 26: MessagePack - An efficient binary serialization format](https://reader033.vdocument.in/reader033/viewer/2022051504/5a64a0447f8b9a52568b4ec7/html5/thumbnails/26.jpg)
MESSAGEPACK VS PROTOBUF
Deserialize
26
0
10000
20000
30000
40000
50000
60000
70000
80000
100 1000 10000 100000 1000000 10000000 100000000
MsgPack
Protobuf
![Page 27: MessagePack - An efficient binary serialization format](https://reader033.vdocument.in/reader033/viewer/2022051504/5a64a0447f8b9a52568b4ec7/html5/thumbnails/27.jpg)
REFERENCE
27
![Page 28: MessagePack - An efficient binary serialization format](https://reader033.vdocument.in/reader033/viewer/2022051504/5a64a0447f8b9a52568b4ec7/html5/thumbnails/28.jpg)
REFERENCE
MessagePack: It's like JSON. but fast and small.
http://msgpack.org/
28
![Page 29: MessagePack - An efficient binary serialization format](https://reader033.vdocument.in/reader033/viewer/2022051504/5a64a0447f8b9a52568b4ec7/html5/thumbnails/29.jpg)
Q&A29
![Page 30: MessagePack - An efficient binary serialization format](https://reader033.vdocument.in/reader033/viewer/2022051504/5a64a0447f8b9a52568b4ec7/html5/thumbnails/30.jpg)
QUESTION & ANSWER
30