skyway: connecting managed heaps in distributed big data ... · •java serializer benchmark set...

Skyway:Connecting Managed Heaps

in Distributed Big Data Systems

Khanh Nguyen, Lu Fang, Christian Navasca,Harry Xu, Brian Demsky

University of ChicagoUniversity of California, IrvineShan Lu

2

2

2

3

3

JVMMR,

Spark Apps

3

JVMMR,

Spark Apps

3

JVMMR,

Spark Apps

3

3

3

3

4

4

4

4

outDataset

4

OutputStream out = Shuffler.GetOutputStream(receiver_id);

for (Object o: outDataset) {out.writeObject(o);

}

serialization

outDataset

4

OutputStream out = Shuffler.GetOutputStream(receiver_id);

for (Object o: outDataset) {out.writeObject(o);

}

serialization

outDataset

4

InputStream in = Shuffler.GetInputStream(sender_id);

while (in.hasData()) {Object o = in.readObject();inDataset.store(o)

} deserialization

4

InputStream in = Shuffler.GetInputStream(sender_id);

while (in.hasData()) {Object o = in.readObject();inDataset.store(o)

} deserialization

inDataset

Tim

e (s

ec)

Serializers0

350

700

1050

1400

1750

Data transfer costs

5TriangleCounting over LiveJournal on Spark 2.1.0 with 3 slaves

Tim

e (s

ec)

Serializers0

350

700

1050

1400

1750

Data transfer costs

5TriangleCounting over LiveJournal on Spark 2.1.0 with 3 slaves

14%18%

17%

16%

Data transfer

6

Receiver

Object

Sender

Data transfer

6

Receiver

Serialization

Object

Sender

Data transfer

6

Receiver

Serialization

Object

Reflection.getField

Sender

Data transfer

6

Receiver

Serialization

Object

Reflection.getField

Sender

Data transfer

6

Receiver

Serialization

Object

Reflection.getField

Sender

Data transfer

6

Receiver

Serialization

Object

Binary

Reflection.getField

Sender

Network

Data transfer

6

Receiver

Serialization

Object

Binary

Reflection.getField

Sender

Network

Data transfer

6

Receiver

Object

Binary

Sender

Data transfer

6

Receiver

Deserialization

Object

Binary

Sender

Data transfer

6

Receiver

Deserialization

Object

Binary

Reflection.setField

Reflection.allocate

Sender

Data transfer

6

Receiver

Deserialization

Object

Binary

Reflection.setField

Reflection.allocate

Sender

Data transfer

6

Receiver

Deserialization

Object

Binary

Reflection.setField

Reflection.allocate

Sender

Data transfer

7

Receiver

Reflection.getField

Reflection.setField

Reflection.allocate

Sender

Data transfer

7

ReceiverSender

8

8

8

8

8

8

Our solution

9

Our solution

9

ReceiverSender

Our solution

9

Object

ReceiverSender

Our solution

9

Object

ReceiverSender

Skyway

Our solution

9

Object

Reflection.getField

Reflection.setField

Reflection.allocate

ReceiverSender

Skyway

Our solution

9

Object

Reflection.getField

Reflection.setField

Reflection.allocate

ReceiverSender

Skyway

Our solution

9

Object

Reflection.getField

Reflection.setField

Reflection.allocate

ReceiverSender

Skyway

Skyway Overview

10

Skyway Overview

– Implemented in OpenJDK 8• Modified the class loader, the object/heap layout,

the Parallel Scavenge GC– Efficiently handle data transfer:

• Outperforms 90 serializers• Improves Spark by 36% (Java) - 16% (Kryo)• Improves Flink by 19%

10

Challenges

11

Challenges

1. Type representation

11

Challenges

1. Type representation Automated global type numbering

11

Challenges

1. Type representation Automated global type numbering

2. Pointer representation

11

Challenges

1. Type representation Automated global type numbering

2. Pointer representation Use relative offsets

11

Challenges

1. Type representation Automated global type numbering

2. Pointer representation Use relative offsets

3. Local JVM adaptation

11

Challenges

1. Type representation Automated global type numbering

2. Pointer representation Use relative offsets

3. Local JVM adaptation Visible for garbage collection

11

Challenges

1. Type representation Automated global type numbering

2. Pointer representation Use relative offsets

3. Local JVM adaptation Visible for garbage collection

4. Work pipelining

11

Challenges

1. Type representation Automated global type numbering

2. Pointer representation Use relative offsets

3. Local JVM adaptation Visible for garbage collection

4. Work pipelining Buffering

11

Type registries

12

Metadata Object

Worker A

TypeString ID12

“java. lang.Object”“org.apache.spark.

rdd.RDD”“java. lang.String” 5

Type Registry A

klass for “java.lang.Object”

klass for “java.lang.String”

Worker B

TypeString ID15

“java. lang.Object”“java. lang.String”

120

Type Registry B

klass for “org.apache.spark.scheduler.Task”

klass for “java.lang.Object”

“org.apache.spark.scheduler.Task”

TypeString ID12

4

“java. lang.Object”“org.apache.spark.

rdd.RDD”“java.util.HashMap”

...

3“java.util.HashSet”“java. lang.String” 5

...“org.apache.spark.

scheduler.Task” 120

Type Registry

Cluster

klass for “org.apache.spark.rdd.RDD”

Metadata Object

klass for “java.lang.String”

Master

Output & Input buffer

13

Output & Input buffer

13

Output buffer

Output & Input buffer

13

Output buffer

Output & Input buffer

13

Output buffer

•Segregated by receivers•One for each receiver

Output & Input buffer

13

Output buffer

•Segregated by receivers•One for each receiver•In native, off-the-heap memory

Output & Input buffer

13

Output bufferInput buffer

•Segregated by receivers•One for each receiver•In native, off-the-heap memory

Output & Input buffer

13

Output bufferInput buffer

•Segregated by receivers•One for each receiver•In native, off-the-heap memory

Output & Input buffer

13

Output bufferInput buffer

•Segregated by receivers•One for each receiver•In native, off-the-heap memory

•Segregated by senders•Multiple for each sender

Output & Input buffer

13

Output bufferInput buffer

•Segregated by receivers•One for each receiver•In native, off-the-heap memory

•Segregated by senders•Multiple for each sender•In managed heap

Example: Serialization

14

Integer[] 0xbb 0xcc0xaa3

Integer 20 Integer 30Integer 10

Example: Serialization

14

Integer[] 0xbb 0xcc0xaa3

Integer 20 Integer 30Integer 10

writeObject()

Example: Serialization

14

Integer[] 0xbb 0xcc0xaa3

Integer 20 Integer 30Integer 10

writeObject()

Example: Serialization

14

Integer[] 0xbb 0xcc0xaa3

Integer 20 Integer 30Integer 10

Output bufferin native memory

writeObject()

Example: Serialization

14

Integer[] 0xbb 0xcc0xaa3

Integer 20 Integer 30Integer 10

0Offset

Output bufferin native memory

writeObject()

Example: Serialization

14

Integer[] 0xbb 0xcc0xaa3

Integer 20 Integer 30Integer 10

0Offset

Output bufferin native memory

writeObject()

Example: Serialization

14

Integer[] 0xbb 0xcc0xaa3

Integer 20 Integer 30Integer 10

0Offset

Output bufferin native memory

“java.lang.Integer” 6“[java.lang.Integer” 7

TypeString ID

writeObject()

Example: Serialization

14

Integer[] 0xbb 0xcc0xaa3

Integer 20 Integer 30Integer 10

0xbb 0xcc0xaa37

0Offset

Output bufferin native memory

“java.lang.Integer” 6“[java.lang.Integer” 7

TypeString ID

writeObject()

Example: Serialization

14

Integer[] 0xbb 0xcc0xaa3

Integer 20 Integer 30Integer 10

0xbb 0xcc0xaa37

0Offset

Output bufferin native memory

“java.lang.Integer” 6“[java.lang.Integer” 7

TypeString ID

writeObject()

Example: Serialization

14

Integer[] 0xbb 0xcc0xaa3

Integer 20 Integer 30Integer 10

0xbb 0xcc0xaa37

0Offset

Output bufferin native memory

“java.lang.Integer” 6“[java.lang.Integer” 7

TypeString ID

7

writeObject()

Example: Serialization

14

Integer[] 0xbb 0xcc0xaa3

Integer 20 Integer 30Integer 10

0xbb 0xcc0xaa37 106

0Offset

Output bufferin native memory

“java.lang.Integer” 6“[java.lang.Integer” 7

TypeString ID

7

writeObject()

Example: Serialization

14

Integer[] 0xbb 0xcc0xaa3

Integer 20 Integer 30Integer 10

0xbb 0xcc0xaa37 106

0Offset

Output bufferin native memory

7

“java.lang.Integer” 6“[java.lang.Integer” 7

TypeString ID

7

writeObject()

Example: Serialization

14

Integer[] 0xbb 0xcc0xaa3

Integer 20 Integer 30Integer 10

0xbb 0xcc0xaa37 106 206

0 11Offset

Output bufferin native memory

7 11

“java.lang.Integer” 6“[java.lang.Integer” 7

TypeString ID

7

writeObject()

Example: Serialization

14

Integer[] 0xbb 0xcc0xaa3

Integer 20 Integer 30Integer 10

0xbb 0xcc0xaa37 106 206 306

0 11 15Offset

Output bufferin native memory

7 11 15

“java.lang.Integer” 6“[java.lang.Integer” 7

TypeString ID

7

writeObject()

Example: Deserialization

15

0 7 11 15

0xfb 0xff 20 300xf7 103

Offset

7 6 6 611 157

11 15737 10 20 306 6 6

Example: Deserialization

15

0 7 11 15

0xfb 0xff 20 300xf7 103

Offset

7 6 6 611 157

11 15737 10 20 306 6 6

readObject()

Example: Deserialization

15

0 7 11 15

0xfb 0xff 20 300xf7 103

Offset

7 6 6 611 157

Input buffer in heap

readObject()

Example: Deserialization

15

0 7 11 15

0xfb 0xff 20 300xf7 103

Offset

7 6 6 611 157

Input buffer in heap

java.lang.Integer6java.lang.Integer[]7

MetadataObjectID

readObject()

Example: Deserialization

15

0 7 11 15

0xfb 0xff 20 300xf7 10Integer[] 3

Offset

6 6 611 157

Input buffer in heap

java.lang.Integer6java.lang.Integer[]7

MetadataObjectID

readObject()

Example: Deserialization

15

0 7 11 15

0xfb 0xff 20 300xf7 10Integer[] 3

Offset

6 6 611 15

Input buffer in heap

java.lang.Integer6java.lang.Integer[]7

MetadataObjectID

readObject()

Example: Deserialization

15

0 7 11 15

0xfb 0xff 20 300xf7 10Integer[] 3

Offset

6 6 615

Input buffer in heap

java.lang.Integer6java.lang.Integer[]7

MetadataObjectID

readObject()

Example: Deserialization

15

0 7 11 15

0xfb 0xff 20 300xf7 10Integer[] 3

Offset

6 6 6

Input buffer in heap

java.lang.Integer6java.lang.Integer[]7

MetadataObjectID

readObject()

Example: Deserialization

15

0 7 11 15

0xfb 0xff 20 300xf7 10Integer[] 3 Integer

Offset

6 6

Input buffer in heap

java.lang.Integer6java.lang.Integer[]7

MetadataObjectID

readObject()

Example: Deserialization

15

0 7 11 15

0xfb 0xff 20 300xf7 10Integer[] 3 IntegerInteger

Offset

6

Input buffer in heap

java.lang.Integer6java.lang.Integer[]7

MetadataObjectID

readObject()

Example: Deserialization

15

0 7 11 15

0xfb 0xff 20 300xf7 10Integer[] 3 IntegerIntegerInteger

Offset

Input buffer in heap

java.lang.Integer6java.lang.Integer[]7

MetadataObjectID

readObject()

In the paper

• Cyclic references• Shared objects• Support for threads• Interaction with GC• Integrating Skyway in real systems

16

Evaluations - Microbenchmark

17

• Java Serializer Benchmark Set– Extensive performance evaluation with

existing 90 serializers

18

SKYWAY

18

1.8x

SKYWAY

GOOGLE’s Protobuf

18

1.8x

2.2x

SKYWAY

GOOGLE’s Protobuf

Kryo(rec. by Spark)

Evaluations – Real Systems

19

• Flink 1.3.2– 5 query answering applications– TPC-H datasets

Evaluations – Real Systems

19

• Flink 1.3.2– 5 query answering applications– TPC-H datasets

Evaluations – Real Systems

19

• Flink 1.3.2– 5 query answering applications– TPC-H datasets

On average, reduces end-to-end time by 19%

0

0.2

0.4

0.6

0.8

1

1.2

Improvement Summary: Flink

20Execution TimeSer. Time Deser. Time

Norm

alize

d Pe

rform

ance

to b

uilt-

in se

rializ

er

0

0.2

0.4

0.6

0.8

1

1.2

Improvement Summary: Flink

20Execution TimeSer. Time Deser. Time

Norm

alize

d Pe

rform

ance

to b

uilt-

in se

rializ

er

0

0.2

0.4

0.6

0.8

1

1.2

Improvement Summary: Flink

20Execution TimeSer. Time Deser. Time

Norm

alize

d Pe

rform

ance

to b

uilt-

in se

rializ

er

0

0.2

0.4

0.6

0.8

1

1.2

Improvement Summary: Flink

20Execution TimeSer. Time Deser. Time

Norm

alize

d Pe

rform

ance

to b

uilt-

in se

rializ

er

Evaluations – Real Systems

21

• Spark 2.1.0– 4 applications: WordCount, PageRank,

ConnectedComponents, and TriangleCounting

– 4 datasets:LiveJournal, Orkut, UK-2005, and Twitter

Evaluations – Real Systems

21

• Spark 2.1.0– 4 applications: WordCount, PageRank,

ConnectedComponents, and TriangleCounting

– 4 datasets:LiveJournal, Orkut, UK-2005, and Twitter

On average, reduces end-to-end time by 16% (w.r.t. Kryo)

by 36% (w.r.t. Java serializer)

Kryo Skyway0

0.2

0.4

0.6

0.8

1

1.2

1.4

Kryo Skyway Kryo Skyway

Improvement Summary: Spark

22

Norm

alize

d Pe

rform

ance

to

Java

Ser

ialize

r

Execution TimeSer. Time Deser. Time

Kryo Skyway0

0.2

0.4

0.6

0.8

1

1.2

1.4

Kryo Skyway Kryo Skyway

Improvement Summary: Spark

22

Norm

alize

d Pe

rform

ance

to

Java

Ser

ialize

r

Execution TimeSer. Time Deser. Time

Kryo Skyway0

0.2

0.4

0.6

0.8

1

1.2

1.4

Kryo Skyway Kryo Skyway

Improvement Summary: Spark

22

Norm

alize

d Pe

rform

ance

to

Java

Ser

ialize

r

Execution TimeSer. Time Deser. Time

Kryo Skyway0

0.2

0.4

0.6

0.8

1

1.2

1.4

Kryo Skyway Kryo Skyway

Improvement Summary: Spark

22

Norm

alize

d Pe

rform

ance

to

Java

Ser

ialize

r

Execution TimeSer. Time Deser. Time

Kryo Skyway0

0.2

0.4

0.6

0.8

1

1.2

1.4

Kryo Skyway Kryo Skyway

Improvement Summary: Spark

22

Norm

alize

d Pe

rform

ance

to

Java

Ser

ialize

r

Execution TimeSer. Time Deser. Time

Kryo Skyway0

0.2

0.4

0.6

0.8

1

1.2

1.4

Kryo Skyway Kryo Skyway

Improvement Summary: Spark

22

Norm

alize

d Pe

rform

ance

to

Java

Ser

ialize

r

Execution TimeSer. Time Deser. Time

Conclusion• Goal: Reduce data transfer costs in

Big Data systems

• Solution: Skyway, the first JVM-based serializer– Efficiently transfer data– Easy to integrate

23

Thank You!

24