building high performance scalable tcp/ip servers with...
TRANSCRIPT
![Page 1: Building High Performance Scalable TCP/IP Servers with ...mina.apache.org/mina-project/resources/ACEU2006.pdf · •Since we will be using MINA's built-in support for building protocols,](https://reader030.vdocument.in/reader030/viewer/2022040914/5e8b9c1fa3ae4850261854fa/html5/thumbnails/1.jpg)
Building High Performance Scalable TCP/IP Servers with Apache MINAOriginally presented at ApacheCon Europe 2006 in Dublin
Latest slides and code samples at http://people.apache.org/~proyal
Presented by Peter Royal, <[email protected]>
1
![Page 2: Building High Performance Scalable TCP/IP Servers with ...mina.apache.org/mina-project/resources/ACEU2006.pdf · •Since we will be using MINA's built-in support for building protocols,](https://reader030.vdocument.in/reader030/viewer/2022040914/5e8b9c1fa3ae4850261854fa/html5/thumbnails/2.jpg)
•Introduction to MINA
•Demonstration of what it can do
•Converting blocking-IO code to MINA
•Hopefully inspire you to use it :)
Goals of this presentation
2
![Page 3: Building High Performance Scalable TCP/IP Servers with ...mina.apache.org/mina-project/resources/ACEU2006.pdf · •Since we will be using MINA's built-in support for building protocols,](https://reader030.vdocument.in/reader030/viewer/2022040914/5e8b9c1fa3ae4850261854fa/html5/thumbnails/3.jpg)
•Multipurpose Infrastructure for Networked Applications
•A framework (the F word!) for building networked clients and servers based on non-blocking IO
•http://directory.apache.org/subprojects/mina/
What is MINA
3
![Page 4: Building High Performance Scalable TCP/IP Servers with ...mina.apache.org/mina-project/resources/ACEU2006.pdf · •Since we will be using MINA's built-in support for building protocols,](https://reader030.vdocument.in/reader030/viewer/2022040914/5e8b9c1fa3ae4850261854fa/html5/thumbnails/4.jpg)
•Started out as Netty2 from Trustin Lee
•Joined the Directory Project as the SEDA-based directory needed an asynchronous I/O layer.
Brief history of MINA
4
![Page 5: Building High Performance Scalable TCP/IP Servers with ...mina.apache.org/mina-project/resources/ACEU2006.pdf · •Since we will be using MINA's built-in support for building protocols,](https://reader030.vdocument.in/reader030/viewer/2022040914/5e8b9c1fa3ae4850261854fa/html5/thumbnails/5.jpg)
Architectural overview
IoFilter
IoFilter
IoFilter
IoHandler
IoProcessor
IoAcceptor
Inbound
Outbound
Network
5
![Page 6: Building High Performance Scalable TCP/IP Servers with ...mina.apache.org/mina-project/resources/ACEU2006.pdf · •Since we will be using MINA's built-in support for building protocols,](https://reader030.vdocument.in/reader030/viewer/2022040914/5e8b9c1fa3ae4850261854fa/html5/thumbnails/6.jpg)
•Holder of state for a connection (either client-side or server-side)
•Passed along with every event
•Important Methods
•write
•close
•get/setAttribute
IoSession
6
![Page 7: Building High Performance Scalable TCP/IP Servers with ...mina.apache.org/mina-project/resources/ACEU2006.pdf · •Since we will be using MINA's built-in support for building protocols,](https://reader030.vdocument.in/reader030/viewer/2022040914/5e8b9c1fa3ae4850261854fa/html5/thumbnails/7.jpg)
•Akin to a Servlet
•Endpoint of a filter chain
•Important Methods
•sessionOpened
•messageReceived
•sessionClosed
IoHandler
IoProcessor
IoFilter
IoFilter
IoFilter
IoAcceptor
IoHandler
7
![Page 8: Building High Performance Scalable TCP/IP Servers with ...mina.apache.org/mina-project/resources/ACEU2006.pdf · •Since we will be using MINA's built-in support for building protocols,](https://reader030.vdocument.in/reader030/viewer/2022040914/5e8b9c1fa3ae4850261854fa/html5/thumbnails/8.jpg)
•Chain of IoFilter's for each IoSession
•Can setup template chains per IoConnector/IoAcceptor
•Dynamic addition/removal
IoFilterChain
8
![Page 9: Building High Performance Scalable TCP/IP Servers with ...mina.apache.org/mina-project/resources/ACEU2006.pdf · •Since we will be using MINA's built-in support for building protocols,](https://reader030.vdocument.in/reader030/viewer/2022040914/5e8b9c1fa3ae4850261854fa/html5/thumbnails/9.jpg)
•Akin to a ServletFilter
•View/Hack/Slash the event stream
•Important Methods
•sessionOpened
•messageReceived
•filterWrite
•sessionClosed
IoFilters
IoProcessor
IoAcceptor
IoHandler
IoFilter
IoFilter
IoFilter
9
![Page 10: Building High Performance Scalable TCP/IP Servers with ...mina.apache.org/mina-project/resources/ACEU2006.pdf · •Since we will be using MINA's built-in support for building protocols,](https://reader030.vdocument.in/reader030/viewer/2022040914/5e8b9c1fa3ae4850261854fa/html5/thumbnails/10.jpg)
•Server-side entry point.
•Accepts incoming connections and fires events to an IoHandler
•Important Methods
•bind
IoAcceptor
IoProcessor
IoFilter
IoFilter
IoFilter
IoHandler
IoAcceptor
10
![Page 11: Building High Performance Scalable TCP/IP Servers with ...mina.apache.org/mina-project/resources/ACEU2006.pdf · •Since we will be using MINA's built-in support for building protocols,](https://reader030.vdocument.in/reader030/viewer/2022040914/5e8b9c1fa3ae4850261854fa/html5/thumbnails/11.jpg)
•Client-side entry point
•Initiate connections to a remote service, and fires events to an IoHandler
•Important Methods
•connect
IoConnector
11
![Page 12: Building High Performance Scalable TCP/IP Servers with ...mina.apache.org/mina-project/resources/ACEU2006.pdf · •Since we will be using MINA's built-in support for building protocols,](https://reader030.vdocument.in/reader030/viewer/2022040914/5e8b9c1fa3ae4850261854fa/html5/thumbnails/12.jpg)
•Internal component
•Handles reading and writing data to an underlying connection
•Each connection is associated with a single IoProcessor (shared amongst multiple connections)
IoProcessor
IoAcceptor
IoHandler
IoFilter
IoFilter
IoFilter
IoProcessor
12
![Page 13: Building High Performance Scalable TCP/IP Servers with ...mina.apache.org/mina-project/resources/ACEU2006.pdf · •Since we will be using MINA's built-in support for building protocols,](https://reader030.vdocument.in/reader030/viewer/2022040914/5e8b9c1fa3ae4850261854fa/html5/thumbnails/13.jpg)
•Persistent connections from clients
•Serialize java objects across the wire
•Clients connect and are given a unit of work, which in this case, is just an instruction for how long to wait until getting their next instruction.
Our Sample Application
13
![Page 14: Building High Performance Scalable TCP/IP Servers with ...mina.apache.org/mina-project/resources/ACEU2006.pdf · •Since we will be using MINA's built-in support for building protocols,](https://reader030.vdocument.in/reader030/viewer/2022040914/5e8b9c1fa3ae4850261854fa/html5/thumbnails/14.jpg)
•Thread activity via jconsole
•CPU Activity via Activity Monitor
•(or your favorite tool)
Monitoring Performance
14
![Page 15: Building High Performance Scalable TCP/IP Servers with ...mina.apache.org/mina-project/resources/ACEU2006.pdf · •Since we will be using MINA's built-in support for building protocols,](https://reader030.vdocument.in/reader030/viewer/2022040914/5e8b9c1fa3ae4850261854fa/html5/thumbnails/15.jpg)
Limitations
15
![Page 16: Building High Performance Scalable TCP/IP Servers with ...mina.apache.org/mina-project/resources/ACEU2006.pdf · •Since we will be using MINA's built-in support for building protocols,](https://reader030.vdocument.in/reader030/viewer/2022040914/5e8b9c1fa3ae4850261854fa/html5/thumbnails/16.jpg)
•JVM limit on number of threads
•The lovely java.lang.OutOfMemoryError: unable to create new native thread
Scalability
16
![Page 17: Building High Performance Scalable TCP/IP Servers with ...mina.apache.org/mina-project/resources/ACEU2006.pdf · •Since we will be using MINA's built-in support for building protocols,](https://reader030.vdocument.in/reader030/viewer/2022040914/5e8b9c1fa3ae4850261854fa/html5/thumbnails/17.jpg)
•Server side first
•(Client to come soon)
Lets convert to MINA!
17
![Page 18: Building High Performance Scalable TCP/IP Servers with ...mina.apache.org/mina-project/resources/ACEU2006.pdf · •Since we will be using MINA's built-in support for building protocols,](https://reader030.vdocument.in/reader030/viewer/2022040914/5e8b9c1fa3ae4850261854fa/html5/thumbnails/18.jpg)
Re-testing
18
![Page 19: Building High Performance Scalable TCP/IP Servers with ...mina.apache.org/mina-project/resources/ACEU2006.pdf · •Since we will be using MINA's built-in support for building protocols,](https://reader030.vdocument.in/reader030/viewer/2022040914/5e8b9c1fa3ae4850261854fa/html5/thumbnails/19.jpg)
•Java Serialization takes up CPU time
•(a profiler would reveal this)
•OS limit of per-process open files
•(consult the documentation for your OS)
•sysctl / ulimit to view/change on unix-like systems
New Limitations?
19
![Page 20: Building High Performance Scalable TCP/IP Servers with ...mina.apache.org/mina-project/resources/ACEU2006.pdf · •Since we will be using MINA's built-in support for building protocols,](https://reader030.vdocument.in/reader030/viewer/2022040914/5e8b9c1fa3ae4850261854fa/html5/thumbnails/20.jpg)
•Since we will be using MINA's built-in support for building protocols, the ProtocolCodecFilter
•Any socket client can talk to MINA
•We’re using MINA on both sides for simplicity in our examples.
MINA on the client
20
![Page 21: Building High Performance Scalable TCP/IP Servers with ...mina.apache.org/mina-project/resources/ACEU2006.pdf · •Since we will be using MINA's built-in support for building protocols,](https://reader030.vdocument.in/reader030/viewer/2022040914/5e8b9c1fa3ae4850261854fa/html5/thumbnails/21.jpg)
•IoHandler and IoFilter's
•Can re-use filters on both client and server sides.
Client is just like the server
21
![Page 22: Building High Performance Scalable TCP/IP Servers with ...mina.apache.org/mina-project/resources/ACEU2006.pdf · •Since we will be using MINA's built-in support for building protocols,](https://reader030.vdocument.in/reader030/viewer/2022040914/5e8b9c1fa3ae4850261854fa/html5/thumbnails/22.jpg)
Implementing the protocol.
22
![Page 24: Building High Performance Scalable TCP/IP Servers with ...mina.apache.org/mina-project/resources/ACEU2006.pdf · •Since we will be using MINA's built-in support for building protocols,](https://reader030.vdocument.in/reader030/viewer/2022040914/5e8b9c1fa3ae4850261854fa/html5/thumbnails/24.jpg)
•Logging
•Compression
•Blacklist
•SSL
•Requires Java 5
Filters that ship today
24
![Page 25: Building High Performance Scalable TCP/IP Servers with ...mina.apache.org/mina-project/resources/ACEU2006.pdf · •Since we will be using MINA's built-in support for building protocols,](https://reader030.vdocument.in/reader030/viewer/2022040914/5e8b9c1fa3ae4850261854fa/html5/thumbnails/25.jpg)
•Traffic Shaping
•Rate Limiting
•Performance Monitoring
Filters we are working on
25
![Page 26: Building High Performance Scalable TCP/IP Servers with ...mina.apache.org/mina-project/resources/ACEU2006.pdf · •Since we will be using MINA's built-in support for building protocols,](https://reader030.vdocument.in/reader030/viewer/2022040914/5e8b9c1fa3ae4850261854fa/html5/thumbnails/26.jpg)
Some things built on MINA
•LDAP - Apache DS <http://directory.apache.org/subprojects/apacheds/index.html>
•Flash - red5 <http://www.osflash.org/red5>
•HTTP - AsyncWeb <http://asyncweb.safehaus.org/>
26
![Page 27: Building High Performance Scalable TCP/IP Servers with ...mina.apache.org/mina-project/resources/ACEU2006.pdf · •Since we will be using MINA's built-in support for building protocols,](https://reader030.vdocument.in/reader030/viewer/2022040914/5e8b9c1fa3ae4850261854fa/html5/thumbnails/27.jpg)
Performance Tips
•Set the number of IoProcessor’s to be equal to the number of CPU cores.
•Benchmark! Users have found both heap and direct buffers, pooled and not pooled, to be beneficial depending on their workloads.
•For ultra-low latency with small message sizes on a local lan, disable Nagle’s algorithm; the TCP_NODELAY flag.
27
![Page 28: Building High Performance Scalable TCP/IP Servers with ...mina.apache.org/mina-project/resources/ACEU2006.pdf · •Since we will be using MINA's built-in support for building protocols,](https://reader030.vdocument.in/reader030/viewer/2022040914/5e8b9c1fa3ae4850261854fa/html5/thumbnails/28.jpg)
Questions?
28