using openssl to boost tomcat · concurency 320 tomcat 8.5 coyote_apr_https coyote_nio_jssehttps...

Using OpenSSL to boost Using OpenSSL to boost TomcatTomcatJean-Frederic ClereJean-Frederic Clere

What I will coverWhat I will cover

● Who I am.

● Connectors

– NIO, NIO2, APR

– OpenSSLImplementation

– HTTP/2 and ALPN in Tomcat.

● Performance tests

– With ab and h2load as client load generator.

● Questions?

11/10/16 2

Who I amWho I am

Jean-Frederic Clere

Red Hat

Years writing JAVA code and server software

Tomcat committer since 2001

Doing OpenSource since 1999

Cyclist/Runner etc

Lived 15 years in Spain (Barcelona)

Now in Neuchâtel (CH)

11/10/16 3

TomcatTomcat

11/10/16 4

What is a Connector?What is a Connector?

● Tomcat's interface to the world

● Binds to a port

● Understands a protocol and possible upgrades.

● Dispatches requests (example)

– protocol="org.apache.coyote.http11.Http11AprProtocol"

– protocol="org.apache.coyote.http11.Http11NioProtocol"

– protocol="org.apache.coyote.http11.Http11Nio2Protocol"

11/10/16 5

Tomcat ConnectorsTomcat Connectors

● Java Non-blocking I/O (NIO)

● Native / Apache Portable Runtime (APR)

● Java NIO.2

Technically, there are combinations of all of the above with HTTP and AJP protocols.

The presentation focuses on HTTP and on NIO/NIO2.

11/10/16 6

What is new in Tomcat 9 / 8.5What is new in Tomcat 9 / 8.5

● Property sslImplementationName

– Allows replacement of the SSL code

● OpenSSLImplementation (use OpenSSL)

● JSSEImplementation (use JSSE)

● UpgradeProtocol

– Allows protocol upgrade from HTTP/1.1

● HTTP/2 (yes)

● Websocket (cool) / Speedy (no plan to support it).

11/10/16 7

Why a new SSLImplementationWhy a new SSLImplementation● JSSE:

– Very slow

– Missing features: like ALPN (JEP 244: TLS Application-Layer Protocol Negotiation)

– Hardware acceleration very partial (like AES in java8)

● Native connector:

– Fast but a lot of native code

– Use OpenSSL for SSL/TLS.

● New OpenSSL implemetation:

– Fast.

– Uses only a OpenSSL for native code (no native socket, poller etc).

– Works with NIO and NIO2.

– Uses OpenSSL for SSL/TLS. (warp, unwarp, handshake etc).

11/10/16 8

OpenSSLImplementationOpenSSLImplementation● Code originates from netty-tcnative a forked Tomcat Native

● Prototype (last year):

– Done with the BeFriNe University

– Tested and ported to tc_trunk last summer

● SSL Configuration compatible with the JSSE connection (*)

● Uses keystores (*)

● Uses SSL BIO to wrap/unwarp, handshake

● Uses java NIO or NIO2 Sockets for the reads and writes

● Automatically enabled when TC native is installed/enabled (*)

11/10/16 9

How TLS is done in TomcatHow TLS is done in Tomcat

11/10/16 10

Tomcat

JSSE Con.

Java std lib

JSSE SSL Engine

NIO/NIO2

Tomcat Native

APR JNIs

Webserver

APR Internals

APR Connector

OpenSSL OS Sockets

JavaC

/ Native

Webserver

OpenSSL Impl.

11

How does that worksHow does that works

SSLContext

JSSESSLContext OpenSSLContext

SSLEngine

SSLContext

OpenSSLEngine

createSSLEngine() createSSLEngine()

wrap()unwrap()

getSession()etc...

Overrides

11/10/16 11

12

How does wrap worksHow does wrap works

wrap(plaintext, encrypted)

internalBIO networkBIO

BIO_new_bio_pair

SSL_set_bio

writePlainTextDatawrite_ToSSLSSL_write

readEncryptedDatareadFromBIOBIO_read

11/10/16 12

13

How does unwrap worksHow does unwrap works

unwrap(encrypted, plaintext)

internalBIO networkBIO

BIO_new_bio_pair

SSL_set_bio

writeEncryptedDatawriteToBIOBIO_write

readPlaintextDatareadFromSSLSSL_read

11/10/16

Connector PerformanceConnector Performance

● Compare connectors throughput against each other

● Only static content was compared, varying file sizes

● Run on “fast” machines, 10 Gbps local network

● Tests:

– Compare the connectors (trunk) NIO, NIO2 and APR

– Using JSSE and OpenSSL

– First without “sendfile”

11/10/16 14

Connector Throughput (c8)Connector Throughput (c8)

4KiB.bin8KiB.bin

16KiB.bin32KiB.bin

64KiB.bin128KiB.bin

256KiB.bin512KiB.bin

1MiB.bin2MiB.bin

4MiB.bin8MiB.bin

16MiB.bin32MiB.bin

0

100000

200000

300000

400000

500000

600000

700000

Concurency 8

coyote_apr_https

coyote_nio2_openssl_https

coyote_nio_jsse_https

coyote_nio_openssl_https

File Size

Th

rou

gh

pu

t Kb

yte

s/se

c

11/10/16 15


4KiB.bin8KiB.bin

16KiB.bin32KiB.bin

64KiB.bin128KiB.bin


1MiB.bin2MiB.bin

4MiB.bin8MiB.bin

16MiB.bin32MiB.bin

0

100000

200000

300000

400000

500000

600000

700000

Concurency 40

coyote_apr_https




File Size

Th

rou

gh

pu

t Kb

yte

s/se

c

11/10/16 16


4KiB.bin8KiB.bin

16KiB.bin32KiB.bin

64KiB.bin128KiB.bin


1MiB.bin2MiB.bin

4MiB.bin8MiB.bin

16MiB.bin32MiB.bin

0

100000

200000

300000

400000

500000

600000

700000

concurency 80

coyote_apr_https




File Size

Th

rou

gh

pu

t Kb

yte

s/se

c

11/10/16 17

Connector CPU UseConnector CPU Use

4KiB 16KiB 64KiB 128KiB 512KiB 2MiB 8MiB 32MiB40

50

60

70

80

90

100

Concurency 8


50

60

70

80

90

100

concurency 40


50

60

70

80

90

100

Concurency 80

coyote_apr_https




File Size

CP

U u

sag

e

11/10/16 18

Connector TC8.5Connector TC8.5

11/10/16 19

4KiB.bin8KiB.bin

16KiB.bin32KiB.bin

64KiB.bin128KiB.bin


1MiB.bin

0

100000

200000

300000

400000

500000

600000

700000

800000

Concurency 320 tomcat 8.5

coyote_apr_https

coyote_nio_jssehttps

coyote_nio_opensslhttps

File Size

Kb

yte

s / s

eco

nd

4KiB8KiB

16KiB32KiB

64KiB128KiB

256KiB512KiB

1MiB

0

20

40

60

80

100

120

Concurency 320 tomcat8.5

coyote_apr_https

coyote_nio_jssehttps

coyote_nio_opensslhttps

File Size

CP

U u

sag

e


● With sendfile

– In fact with TLS/SSL sendfile is emulated

11/10/16 20


4KiB.bin8KiB.bin

16KiB.bin32KiB.bin

64KiB.bin128KiB.bin


1MiB.bin2MiB.bin

4MiB.bin8MiB.bin

16MiB.bin32MiB.bin

0

100000

200000

300000

400000

500000

600000

700000

800000

Concurency 8

coyote_apr_https




File Size

Th

rou

gh

pu

t in

Kb

yte

s/se

c

11/10/16 21


4KiB.bin8KiB.bin

16KiB.bin32KiB.bin

64KiB.bin128KiB.bin


1MiB.bin2MiB.bin

4MiB.bin8MiB.bin

16MiB.bin32MiB.bin

0

100000

200000

300000

400000

500000

600000

700000

800000

Concurency 40

coyote_apr_https




File Size

Th

rou

gh

pu

t in

Kb

yte

s/se

c

11/10/16 22


4KiB.bin8KiB.bin

16KiB.bin32KiB.bin

64KiB.bin128KiB.bin


1MiB.bin2MiB.bin

4MiB.bin8MiB.bin

16MiB.bin32MiB.bin

0

100000

200000

300000

400000

500000

600000

700000

800000

Concurency 80

coyote_apr_https




File Size

Th

rou

gh

t in

Kb

yte

s/se

c

11/10/16 23

Connector CPU UseConnector CPU Use


50

60

70

80

90

100

Concunreny 8


50

60

70

80

90

100

Concurency 40


50

60

70

80

90

100

Concurency 80

coyote_apr_https




11/10/16 24


● Conclusion:

– OpenSSL performs better that JSSE

– NIO and NIO(2) give similar results

– Emulated sendfile doesn't help a lot (bigger files better).

– APR isn't needed

– Until Java9 is released OpenSSL is needed for HTTP/2

11/10/16 25

Questions?Questions?Thank you!Thank you!

● [email protected]

● [email protected]

● Repo with the scripts for the tests:

– https://github.com/jfclere/AC2014scripts

11/10/16 26

mailto:[email protected]

mailto:[email protected]

Jean-Frederic Clere@[email protected]

using openssl to boost tomcat · concurency 320 tomcat 8.5 coyote_apr_https coyote_nio_jssehttps...

Documents