high performance rpc with finagle

@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #finagle #scala

High Performance RPC with

Sam BESSALAH @samklr


RPC Redux

• Control of clients and servers• Need for predictable performance • Communication between different languages• Isolated computations• Fan out, Fan In distributed systems• ...


Open sourced by Twitter in 2011 to solve this :


AsynchronousNon-Blocking

Protocol Agnostic Full Stack RPCbuilt on top of

@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #finagle #scalahttp://monkey.org/~marius/funsrv.pdf


Load Balancing : Heap BalancerClient Backoff Strategy

BackPressureFailure Detection

Failover/RetryConnection PoolingTracing (via ZipKin)

...

Production ready distributed toolbox


• Box• Foursquare• Nest• Uber• Pinterest• NY Times• Rdio• PagerDuty• Soundcloud• Strava• StumbleUpon• ...

Known Users ...


@nivdul


Networking Layer

@nivdul


Building Blocks

Futures

Services

Filters


Futures

• Concurrency primitive in Finagle• Different from scala.concurrent.Future and

java.util.concurrent.Future• Less context-switching, easier cancelability• From com.twitter.util.Future• Composables• Error and Exception Handling


Futures Combinators

• Can be composed with map, flatMap, handle and rescue• Exception handling via:


Service


Servicesclass DummyService extends Service[httpx.Request, httpx.Response] {

def apply(req: httpx.Request): Future[httpx.Response] = { val response = req.response response.setStatusCode(200) response.setContentTypeJson() response.write(requestHandler(req.getContentString())) Future.value(response)

}


Filters

class Filter[-ReqIn, +RepOut, +ReqOut, -RepIn] extends ((ReqIn, Service[ReqOut, RepIn]) => Future[RepOut])

or type Filter[Req, Rep] = (Req, Service[Req, Rep]) => Future[Rep]


Filters

• Wrap services to modify inputs or outputs (or both) …• Protocol and Network agnostic• Composable


Filters

val myTimeoutFilter = new TimeoutFilter[String,Int](1.second,DefaultTimer.twitter)


Filters

val myTimeoutFilter = new TimeoutFilter[String, Int](1.second,DefaultTimer.twitter)

val myService = new DummyService


Filters

val myTimeoutFilter = new TimeoutFilter[String, Int](1.second,DefaultTimer.twitter)


val server = myTimeOutFilter andThen myService


Filtersval myTimeoutFilter =

new TimeoutFilter[String,Int]( 1.second, DefaultTimer.twitter)


val service = myService andThen myTimeOutFilter

val server = Httpx.serve( “:8080”, service)

Await.ready(server)


Filters

Can be used for:

• Retries• Logging• Serialization• Authentication• Retries• Timeouts• ...


Clients


Clients

Finagle client are just full stack services

val myClient: Service[Request, Response] = Httpx.newService("rtb-bidder.com:8081")

val bidder = Httpx.serve(":8080", myClient)


Finagle in a nutshell


Protocols

• Thrift• Memcache• Redis• HTTP• SMTP• MySQL• Zookeeper• Mux

Protocol.newService( … )


Finagle Mux

• RPC Protocol Multiplexing • One network connection per client-server session

• Maximization of available bandwidth without incurring the cost of opening additional sockets

• Elimination of head-of-line blocking

• Explicit queue management

• Pure session layer (OSI )


Finagle Serial


Finagle Serial

Use binary format. Scodec, Protobuf, Thrift, Avro, etc.


Service discovery• Finagle ServerSets via ”Names”• Uses Zookeeper as Name server

val client = Http.newService(

“zk!myzkhost.home.com:2181!/my/zk/path” )


Twitter-Server

• Template for servers at Twitter built on top of Finagle• Flags for configuration• Logging and Metrics• Admin HTTP UI• Lifecycle management endpoints, handy for Docker and/or Mesos

• Tracing via ZipKin


Finagle JavaService<Request, Response> service = new Service<Request, Response>() { public Future<Response> apply(Request request) { Response response = Response.apply(); response.setContent(ChannelBuffers.wrappedBuffer("yo".getBytes())); Future<Response> future = Future.value(response); return future; }};

ServerBuilder.safeBuild( service, ServerBuilder.get() .codec(Http.get()) .bindTo(new InetSocketAddress(InetAddress.getLoopbackAddress(), 0)) .name("HttpServer")); }


Finatra• Library to easily build API Services on top of the Twitter Stack (Finagle,

Twitter-Server, Twitter-util, etc.)

• Supports Http and Thrift services

• Brings Sinatra like Style to Finagle for Rest Services


Finatra

class HelloWorldController extends Controller { get("/hi") { request: Request => "Hello " + request.params.getOrElse("name", "unnamed") }

post("/hi") { hiRequest: HiRequest => "Hello " + hiRequest.name + " with id " + hiRequest.id }

}


Finatraobject HelloWorldServerMain extends HelloWorldServer

class HelloWorldServer extends HttpServer { override def modules = Seq(Slf4jBridgeModule)

override def configureHttp(router: HttpRouter) { router .filter[LoggingMDCFilter[Request, Response]] .filter[TraceIdMDCFilter[Request, Response]] .filter[CommonFilters] .add[HelloWorldController] }}


● Thin layer of purely functional basic blocks, to build composable REST APIs.

● Built on top of Finagle● Functional wrapper of Finagle-http● Performance close to Bare Metal Finagle

http://github.com/finagle/finch


Zipkin

• Zipkin is a distributed tracing system that helps us gather timing data for all the disparate services at Twitter. It manages both the collection and lookup of this data through a Collector and a Query service.

• Implementation of Google’s Dapper paper

• Moved to OpenZipkin


ZipKin - Open ZipKin


Finagle : Cons

• Stuck to old versions of Netty. Currently Netty 3 and now moving to 4• API Changes since 6.x. No more Netty API Leaking through Finagle’s• Mismatch between scala.concurrent.Future and com.twitter.util.Future• Slow to update• Documentation not exhaustive. Thanks for gitter.im and the Mailing List• Can be painful to plug to other Metrics and Monitoring systems. Hopefully

it’s changing


Quick Demo


Conclusion

high performance rpc with finagle

Technology