performance comparison of dtn bundle protocol implementations

Performance Comparison of DTN Bundle ProtocolImplementations

Wolf-Bastian Pottner, Johannes Morgenroth, Sebastian Schildt, Lars Wolf

September 23, 2011: CHANTS Workshop, Las Vegas, NV

Motivation

Several Bundle Protocol (BP) implementations for Linux exist

DTN2: DTNRG’s reference implementation

IBR-DTN: Implementation for embedded OpenWRT devices

ION: JPL’s implementation for spacecrafts running on RTOS

Postellation: Viagenie’s closed-source (?) BP implementation

Spindle: Closed-source BP implementation by Raytheon BBNTechnologies

Performance?

No quantitative evaluation and comparison has been performed(yet)

Pottner, Morgenroth, Schildt, Wolf | Bundle Protocol Performance | 2

Motivation (cont’d)

Why is performance evaluation of BP implementations important?

Contacts are rare and short, communication has to be efficient

Broken interoperability can point out ambiguities in the specs

Goals of this paper

Concentrate on open-source TCPCL implementations for Linux

Quantitative “test bench” performance evaluation

Quantitative opportunistic performance evaluation

Systematic evaluation of interoperability

Pottner, Morgenroth, Schildt, Wolf | Bundle Protocol Performance | 3

Metrics

Application Layer Throughput

For different storage backends

Between different implementations

In an opportunistic scenario

Interoperability

Can implementations discover each other?

Throughput between different implementations

Communication Efficiency

In an opportunistic scenario

Pottner, Morgenroth, Schildt, Wolf | Bundle Protocol Performance | 4

Experimental Setup

Hardware

Athlon II X4 2.8 GHz CPUs with 4 GiB RAM

1 GBit Ethernet (good for 940 MBit/s TCP goodput with iperf)

Different Ethernet ports for measurement and control traffic

Samsung Spinpoint F3 500GB (HD502HJ) hard drive

Software

Ubuntu Linux 11.04

DTN2 v2.7, IBR-DTN v0.6.3, ION v2.4.0

Connection blocked via iptables (IBR-DTN) or daemon’s controlinterface (DTN2, ION)

Pottner, Morgenroth, Schildt, Wolf | Bundle Protocol Performance | 5

Storage Backends: BP Throughput

0

200

400

600

800

1000

1 10 100 1000 10000 100000 1e+06 1e+07

Bu

nd

le T

hro

ug

hp

ut

[MB

it/s

]

Bundle Payload Size [bytes]

DTN2 DiskDTN2 Mem

0

200

400

600

800

1000

1 10 100 1000 10000 100000 1e+06 1e+07

Bu

nd

le T

hro

ug

hp

ut

[MB

it/s

]

Bundle Payload Size [bytes]

DTN2 DiskDTN2 Mem

IBR-DTN DiskIBR-DTN Mem

0

200

400

600

800

1000

1 10 100 1000 10000 100000 1e+06 1e+07

Bu

nd

le T

hro

ug

hp

ut

[MB

it/s

]

Bundle Payload Size [bytes]

DTN2 DiskDTN2 Mem

IBR-DTN DiskIBR-DTN Mem

ION Disk (T)ION Mem (T)

Pottner, Morgenroth, Schildt, Wolf | Bundle Protocol Performance | 6

Interoperability: Discovery, Bundle Exchange

Discovery Capabilities

DTN2 uses a “proprietary” discovery beacon format

IBR-DTN supports IPND v0 & v1 and DTN2 discovery format

ION is focussed on scheduled contacts only

Interoperability Evaluation Results

DTN2 and IBR-DTN discover each other just fine

Bundles can be exchanged in both directions

ION exchanges bundles with DTN2 and IBR-DTN

Pottner, Morgenroth, Schildt, Wolf | Bundle Protocol Performance | 7

Interoperability: BP Throughput

Throughput (memory storage, 1 MByte payload size)HHHH

HHTX

RXDTN2 IBR-DTN ION

DTN2 687 MBit/s 635 MBit/s 93 MBit/s

IBR-DTN 881 MBit/s 679 MBit/s 90 MBit/s

ION 872 MBit/s 926 MBit/s 449 MBit/s

Observations

DTN2 may have a bottleneck in the TX component

IBR-DTN and ION show unclear bottlenecksshows unclearbottlenecks

Fastest result is between ION and IBR-DTN

Pottner, Morgenroth, Schildt, Wolf | Bundle Protocol Performance | 8

Opportunistic Scenario: Setup

Scenario

A M B

Memory-based storage backends

A creates bundles for B at constant rate, M commutes from A to B

TCP throughput limited to 12.7 MBit/s (based on measurements)

Cycle: 72 s contact time, 30 s travel, 72 s contact time, 30 s travel

(almost) 9 complete cycles in 30 minutes measurement time

Pottner, Morgenroth, Schildt, Wolf | Bundle Protocol Performance | 9

Opportunistic Scenario: Transferred Data

0

200

400

600

800

1000

250000 500000 1e+06 2e+06

Tra

nsf

erre

d D

ata

[MB

]

Bundle Payload Size [bytes]

DTN2IBR-DTN

theoretical Maximum

Pottner, Morgenroth, Schildt, Wolf | Bundle Protocol Performance | 10

Opportunistic Scenario: Communication Efficiency

0

20

40

60

80

100

250000 500000 1e+06 2e+06

Co

mm

un

icat

ion

Eff

icie

ncy

[%

]

Bundle Payload Size [bytes]

DTN2IBR-DTN

theoretical Maximum

Pottner, Morgenroth, Schildt, Wolf | Bundle Protocol Performance | 11

Conclusions

Motivation

Performance of BP implementations is important!

Comparison of three BP implementations: DTN2, IBR-DTN, ION

Results

All three implementations are interoperable

Performance varies widely and unpredictably

Fastest result between ION and IBR-DTN

Working with the implementations is not easy

More results

More results in our technical report (WiP, look for updates)

Mail me: poettner@ibr.cs.tu-bs.de

Pottner, Morgenroth, Schildt, Wolf | Bundle Protocol Performance | 12

BACKUP SLIDES

BACKUP SLIDES

Pottner, Morgenroth, Schildt, Wolf | Bundle Protocol Performance | 13

Throughput (linear y)

0

200

400

600

800

1000

1 10 100 1000 10000 100000 1e+06 1e+07

Bundle

Thro

ughput

[MB

it/s

]

Bundle Payload Size [bytes]

DTN2 DiskDTN2 Mem

IBR-DTN DiskIBR-DTN Mem

ION Disk (T)ION Mem (T)

Pottner, Morgenroth, Schildt, Wolf | Bundle Protocol Performance | 14

Throughput (logarithmic y)

0.001

0.01

0.1

1

10

100

1000

1 10 100 1000 10000 100000 1e+06 1e+07

Bundle

Thro

ughput

[MB

it/s

]

Bundle Payload Size [bytes]

DTN2 DiskDTN2 Mem

IBR-DTN DiskIBR-DTN Mem

ION Disk (T)ION Mem (T)

Pottner, Morgenroth, Schildt, Wolf | Bundle Protocol Performance | 15

DTN2 Slowdown

0

50

100

150

200

250

1 2 3 4 5 6 7 8 9 10

Thro

ughput

[MB

it/s

]

Measurement Run

50e+03 Bytes Payload 100e+03 Bytes Payload

500e+03 Bytes Payload1000e+03 Bytes Payload

Pottner, Morgenroth, Schildt, Wolf | Bundle Protocol Performance | 16

Interoperability: BP Throughput

Throughput (memory storage, 100 KByte payload size)HH

HHHHTX

RXDTN2 IBR-DTN ION

DTN2 197 MBit/s 193 MBit/s 72 MBit/s

IBR-DTN 677 MBit/s 542 MBit/s 76 MBit/s

ION 455 MBit/s 420 MBit/s 268 MBit/s

Throughput (memory storage, 1 MByte payload size)HHH

HHHTX

RXDTN2 IBR-DTN ION

DTN2 687 MBit/s 635 MBit/s 93 MBit/s

IBR-DTN 881 MBit/s 679 MBit/s 90 MBit/s

ION 872 MBit/s 926 MBit/s 449 MBit/s

Pottner, Morgenroth, Schildt, Wolf | Bundle Protocol Performance | 17