Codemotion 2015,Berlin, Germany
Building a
Reactive RESTful APIwith Akka Http & Slick
Dan Persa - @danpersa
EUROPE’S LEADING ONLINE FASHION PLATFORM
15 countries3 fulfillment centers16+ million active customers2.2+ billion € revenue 2014130+ million visits per month9.000+ employees
Visit us: tech.zalando.com
ZALANDOTECHNOLOGY
500+Apps
800+ Tech employees
August
Conway’s Law
“organizations which design systems ...are constrained to produce designs which are copies of the communication structures of these organizations”
ARCHITECTURE
AN ARCHITECTURE FOR INNOVATION
API FIRSTRESTSAAS
MICRO SERVICESCLOUD
OPEN SOURCE
THE SHOP MONOLITH
http://blog.codinghorror.com/new-programming-jargon/
We call it “Jimmy”
Thousands of Java classes, undocumented featuresBusiness logic on all layers (including the database)
MICROSERVICES
Internet
LB
SKIPPERSKIPPER
SKIPPERSKIPPER
JIMMYMOSAIC
INNKEEPERINNKEEPER
INNKEEPERLB
REST APIs
HIGHLY AVAILABLE0 DOWNTIMERELIABLEFAST
INNKEEPER github.com/zalando/innkeeper
API FIRSTREST
FORMAL SPECIFICATION FOR RESTful APIs
The World's Most Popular Framework for APIs.
REST RESOURCES
GET /routesGET /routes/{id}POST /routesDELETE /routes/{id}
GET /updated-routes/{id}
GOING REACTIVE
MODEL DBCONTROLLER
BUFFERBUFFERDB RecordsDTOsJSON
CLIENT
TRADITIONAL APPLICATION
HTTP Request Method Call SQL
Blocking IO
MODEL DBCONTROLLER
DB StreamDTOsJSON
CLIENT
REACTIVE APPLICATION
HTTP Request Method Call SQL
Non-Blocking IO
StreamStreamStream
ScalaTypesafeComposable
Meet
Separation of I/0ResilienceReactive Streams
Meet
def routesModifiedSince(localDateTime: LocalDateTime): DatabasePublisher[RouteRow] = {
}
val routesTable = TableQuery[RoutesTable]
val q = for { routeRow <- routesTable if (routeRow.createdAt > localDateTime | routeRow.deletedAt > localDateTime)} yield routeRow
db.stream { q.result }
ComposableMaterializableReactive Streams
Meet
Streams
import akka.stream.scaladsl.Source
def findRoutesModifiedSince(localDateTime: LocalDateTime): Source[Route, Unit] = {
}
Source(
).mapConcat(_.toList)
routesRepo.selectModifiedSince(localDateTime).mapResult { row =>
}
row.id.map { id => Route( id = id, route = row.routeJson.parseJson.convertTo[NewRoute], row.createdAt, row.deletedAt )}
val route = path("hello") { get { complete { <h1>Hello World</h1> } } }
Meet
Http
object Main extends App {
}
implicit val system = ActorSystem("my-system")implicit val materializer = ActorMaterializer()val route = ...val bindingFuture = Http().bindAndHandle(route, "localhost", 8080)...
Innkeeper’s AKKA HTTP Routesval route = path("updated-routes" / Rest) { lastModifiedString => get { complete(…) } } ~ path("routes") { get { complete(…) } ~ post { complete(…) } } ~ path("routes" / LongNumber) { id => get { … } ~ delete { … } }
path("routes") { get {
complete { HttpResponse(
) } }}
entity = HttpEntity.Chunked( MediaTypes.`application/json`, chunkedStreamSource )
val chunkedStreamSource = jsonService.sourceToJsonSource(routesService.allRoutes)
def sourceToJsonSource[T](source: Source[T, Unit]) (implicit writer: JsonWriter[T]):
Source[ChunkStreamPart, ((Unit, Unit), Unit)] = {
val commaSeparatedRoutes: Source[ChunkStreamPart, Unit] = source .map(t => Some(t.toJson.compactPrint)) .scan[Option[ChunkStreamPart]](None)({ case (None, Some(sourceElement)) => Some(ChunkStreamPart(sourceElement)) case (_, Some(sourceElement)) => Some(ChunkStreamPart(s", $sourceElement")) }) .mapConcat(_.toList)
Source.single(ChunkStreamPart("[")) ++ commaSeparatedRoutes ++ Source.single(ChunkStreamPart("]")) }
SAAS
OAUTH
val route: RequestContext => Future[RouteResult] = authenticationToken { token => authenticate(token, authService) { authenticatedUser => path("updated-routes" / Rest) { lastModifiedString => get { hasOneOfTheScopes(authenticatedUser)(scopes.READ) { ...
import akka.http.scaladsl._import akka.http.scaladsl.server.directives.HeaderDirectives._
trait OAuthDirectives { def authenticationToken: Directive1[String] = headerValue(optionalValue("authorization")) | reject {
AuthenticationFailedRejection( CredentialsMissing, HttpChallenge("", "")
) }...
CLOUD
STUPS.IO
FROM zalando/openjdk:8u45-b14-5MAINTAINER Team Spearheads <[email protected]>
EXPOSE 8080
RUN mkdir -p /opt/innkeeper
ADD target/scala-2.11/innkeeper-assembly-0.0.1.jar /opt/innkeeper/
WORKDIR /opt/innkeeper
ENTRYPOINT java $(java-dynamic-memory-opts) -Dinnkeeper.env=prod -server -jar innkeeper-assembly-0.0.1.jar
docker:sbt assemblydocker build -t innkeper:latest-SNAPSHOT .
docker-push:docker push innkeper:latest-SNAPSHOT
docker-run:docker run -p 8080:8080 -t innkeper:latest-SNAPSHOT
test-db:docker run -e POSTGRES_PASSWORD=innkeeper-test -e
POSTGRES_USER=innkeeper-test -p 5433:5432 postgres:9.4
tick:senza create senza.yaml tick latest-SNAPSHOT
tock:senza create senza.yaml tock latest-SNAPSHOT
OPENSOURCE
github.com/zalando/innkeeper
DOOSF
RADICAL AGILITY
TRUSTINSTEAD OF CONTROL
# use Docker-based container (instead of OpenVZ)sudo: false
cache: directories: - $HOME/.sbt - $HOME/.ivy2
language: scala
script: - sbt ++$TRAVIS_SCALA_VERSION test - sbt ++$TRAVIS_SCALA_VERSION it:test
# Trick to avoid unnecessary cache updates - find $HOME/.sbt -name "*.lock" | xargs rm
scala: - 2.11.7jdk: - oraclejdk8
addons: postgresql: "9.4"
before_script: - psql -c 'CREATE ROLE innkeepertest superuser login createdb;' -U postgres - psql -c 'CREATE DATABASE innkeepertest;' -U postgres
NEXTSTEPS
APPLICATION LOGS: SCALYR
TODO: Screenshot
ZMON
Where to Find Us:Tech Blog: tech.zalando.com
GitHub: github.com/zalando
Innkeeper: github.com/zalando/innkeeper
Twitter: @ZalandoTech
Instagram: zalandotech
Jobs: http://tech.zalando.com/jobs
THANK YOU!QUESTIONS?