scaling business app development with play and scala
TRANSCRIPT
@PeterHilton
http://hilton.org.uk/
Scaling business app development with Play & Scala
M A N N I N G
Peter HiltonErik BakkerFrancisco CanedoFOREWORD BY James Ward
Covers Play 2
Play for Scala(Manning)
Peter HiltonErik BakkerFrancisco Canedo
http://bit.ly/playscala2p
Business web app development
Business applications
Business applications support things like data management, process visibility and process automation.
A special-purpose intranet application may only have 10-100 users.
4@PeterHilton •
Business app development projects
5@PeterHilton •
Development cost is the toughest issue.
The following is a true story of how Scala made us awesome.
Scaling is usually for runtime performance This is not that talk.
With simplicity in the right places, building a web application with the Typesafe platform is* easier and faster than with PHP
* probably6@PeterHilton •
Case study:Happy Melly
robin-berjon / CC BY-SA 2.0
Happy Melly
‘Happy Melly is a network of businesses that self-organize around a purpose: creating happy workers.’ http://www.happymelly.com/about/
Several member organisations No head office or other central location
9@PeterHilton •
10@PeterHilton •
11@PeterHilton •
Working with an experienced remote product owner
13@PeterHilton •
Release early: no ‘sprint 0’ - first release on day one
Public Internet test server: Play/Scala web app hosted on Cloudbees
Continuous delivery - release per feature Push to master → test server deployment
Technical approach
Play Framework 2.1 (later upgraded to 2.2) Scala 2.10 on JDK 1.7 Slick 1.0 MySQL 5.6 Twitter Bootstrap 2 (later upgraded to 3)
and some helpful libraries…14@PeterHilton •
15@PeterHilton •
15@PeterHilton •
15@PeterHilton •
/** * HTML form mapping for creating and editing. */ def organisationForm(implicit request: SecuredRequest[_]) = Form(mapping( "id" -‐> ignored(Option.empty[Long]), "name" -‐> nonEmptyText, "street1" -‐> optional(text), "street2" -‐> optional(text), "city" -‐> optional(text), "province" -‐> optional(text), "postCode" -‐> optional(text), "country" -‐> nonEmptyText, "vatNumber" -‐> optional(text), "registrationNumber" -‐> optional(text), "category" -‐> optional(categoryMapping), "webSite" -‐> optional(webUrl), "blog" -‐> optional(webUrl), "active" -‐> ignored(true),
"name" -‐> nonEmptyText, "street1" -‐> optional(text), "street2" -‐> optional(text), "city" -‐> optional(text), "province" -‐> optional(text), "postCode" -‐> optional(text), "country" -‐> nonEmptyText, "vatNumber" -‐> optional(text), "registrationNumber" -‐> optional(text), "category" -‐> optional(categoryMapping), "webSite" -‐> optional(webUrl), "blog" -‐> optional(webUrl), "active" -‐> ignored(true), "created" -‐> ignored(DateTime.now()), "createdBy" -‐> ignored(request.user.fullName), "updated" -‐> ignored(DateTime.now()), "updatedBy" -‐> ignored(request.user.fullName) )(Organisation.apply)(Organisation.unapply))
"name" -‐> nonEmptyText, "street1" -‐> optional(text), "street2" -‐> optional(text), "city" -‐> optional(text), "province" -‐> optional(text), "postCode" -‐> optional(text), "country" -‐> nonEmptyText, "vatNumber" -‐> optional(text), "registrationNumber" -‐> optional(text), "category" -‐> optional(categoryMapping), "webSite" -‐> optional(webUrl), "blog" -‐> optional(webUrl), "active" -‐> ignored(true), "created" -‐> ignored(DateTime.now()), "createdBy" -‐> ignored(request.user.fullName), "updated" -‐> ignored(DateTime.now()), "updatedBy" -‐> ignored(request.user.fullName) )(Organisation.apply)(Organisation.unapply))
private def validateWebUrl(url: String): Boolean = { try { val uri = new java.net.URI(url) val validScheme = ValidURLSchemes.contains( Option(uri.getScheme).getOrElse("").toLowerCase) val host = Option(uri.getHost).getOrElse("") val validDomain = DomainNameRegex.findFirstIn( host.toLowerCase).isDefined validScheme && validDomain } catch { case _: Throwable ⇒ false
} }
// Web site URL form mapping. val webUrl = text(maxLength = 1024) verifying ("error.url.web", validateWebUrl(_))
18@PeterHilton •
private val FacebookDomain = "facebook.com" private val LinkedInDomain = "linkedin.com" private val GoogleDomain = "google.com"
private def validateDomain(url: String, domain: String): Boolean = { try { val host = Option(new java.net.URI(url). getHost).getOrElse("").toLowerCase host == domain || host.endsWith("." + domain) } catch { case _: Throwable ⇒ false } }
val facebookProfileUrl = webUrl verifying (error = "error.url.profile", validateDomain(_, FacebookDomain))
Simplifying front-end development
20@PeterHilton •
Twitter Bootstrap No custom CSS or JavaScript* Master-detail pages (HTML tables, straightforward layout) Edit pages (mostly standard Bootstrap form layout)
* hardly any
http://cloc.sourceforge.net v 1.58 -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ Language files blank comment code -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ Scala 81 1063 2747 4494 HTML 40 321 0 2855 SQL 49 248 14 655 Javascript 5 30 150 284 CoffeeScript 1 15 3 51 XML 1 4 24 14 -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ SUM: 177 1681 2938 8353 -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐
How to cheat at front-end dev
We only supported the latest version of Google Chrome … and the usual pain just went away
Reminder: intranet application, few users, and an experienced product owner.
22@PeterHilton •
// Code example: Scala Slick database access
// Nice: select * from LICENSE where id=? val query = Query(Licenses).filter(_.id === id)
// Code example: Scala Slick database access
// Nice: select * from LICENSE where id=? val query = Query(Licenses).filter(_.id === id)
// Code example: Scala Slick database access
// Nasty: select b.NUMBER, b.DATE, p.NAME, o.NAME from BOOKING b // inner join ACCOUNT a on (a.ID=b.FROM_ID) // left outer join PERSON p on (p.ID=a.PERSON_ID) // left outer join ORGANISATION o on (o.ID=a.ORGANISATION_ID) val query = for { entry ← BookingEntries ((fromAccount, fromPerson), fromOrganisation) ← Accounts leftJoin People on (_.personId === _.id) leftJoin Organisations on (_._1.organisationId === _.id) if fromAccount.id === entry.fromId } yield ( entry.bookingNumber, entry.bookingDate, fromPerson.name.?, fromOrganisation.name.?)
Scalariform
25@PeterHilton •
Source code formatter, integrated with sbt
We liked it so much we set it up to reformat code on every compilation and replace ASCII art arrows with ⇒ and ←
https://github.com/mdr/scalariform
// project/Build.sbt …
val main = play.Project(appName, appVersion, appDependencies resolvers += Resolver.url("sbt-‐plugin-‐releases", url(" resolvers += Resolver.url("Objectify Play Snapshot Repository resolvers += Resolver.url("Objectify Play Repository", routesImport += "binders._" ).settings( // Reformat code before every compilation :) ScalariformKeys.preferences := FormattingPreferences(). setPreference(SpacesWithinPatternBinders, false). setPreference(PreserveSpaceBeforeArguments, true). setPreference(RewriteArrowSymbols, true) )
SecureSocial
28@PeterHilton •
Social network authentication: Twitter, Facebook, Google, LinkedIn Less effort and better UX than the usual sign-up, log-in, reset password features
http://securesocial.ws
DataTables
30@PeterHilton •
HTML tables with client-side filter and sort, in this case from server-side HTML tables. http://datatables.net
DataTables-Bootstrap integrates styling. http://datatables.net/manual/styling/bootstrap
pegdown & JSoup
32@PeterHilton •
Markdown processing - an easy way to use standard HTML forms to edit HTML https://github.com/sirthias/pegdown
JSoup sanitises the resulting HTML using an HTML whitelist http://jsoup.org
Joda Money
34@PeterHilton •
Currency arithmetic and conversion API.
Money type for an amount with a currency.
Arithmetic and currency conversion, with an explicit rounding policy.
http://www.joda.org/joda-money/
Lessons learned
35@PeterHilton •
You can save a lot of time on front-end development if you cheat. Development is very fast with two experienced developers. Slick had a steep learning curve* and some scary queries, but we still liked it.
* writing/publishing Slick tutorials helped
One more thing…
Halfway through the project, the customer decided to open source the application https://github.com/happymelly/teller
37@PeterHilton •
Case study:NIIOS
Netherlands Institute for Innovative Ocular Surgery (NIIOS)Independent eye surgery clinic in Rotterdam, the Netherlands. ISO accreditation requires quality management and detailed reporting.
Status quo: lots of spreadsheets.
39@PeterHilton •
Technical approach
Play Framework 2.2 Scala 2.10 on JDK 1.7 Slick 2.0 PostgreSQL 9.3 Twitter Bootstrap 2
… and jXLS, webjars, play-plugins-mailer44@PeterHilton •
http://cloc.sourceforge.net v 1.58 -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ Language files blank comment code -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ Scala 55 548 572 2497 HTML 20 179 0 1456 SQL 13 157 30 668 CoffeeScript 6 43 39 133 XML 2 8 6 39 -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ SUM: 96 935 647 4793 -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐
jXLS
46@PeterHilton •
Parse and generate Excel spreadsheets
More useful than CSV because it supports workbooks with multiple sheets
Simplifying data maintenance with spreadsheet integration
// build.sbt
libraryDependencies ++= Seq( jdbc, "com.github.tototoshi" %% "slick-‐joda-‐mapper" % "1.1.0", "com.typesafe.slick" %% "slick" % "2.0.2", "com.typesafe.play" %% "play-‐slick" % "0.6.0.1" , "net.sf.jxls" % "jxls-‐core" % "1.0.5", "net.sf.jxls" % "jxls-‐reader" % "1.0.5", "org.postgresql" % "postgresql" % "9.3-‐1101-‐jdbc41", "org.webjars" %% "webjars-‐play" % "2.2.1-‐2", "org.webjars" % "bootstrap" % "3.1.0", "org.webjars" % "datatables" % "1.10.0", "org.webjars" % "datatables-‐bootstrap" % "2-‐20120202-‐2", "com.typesafe" %% "play-‐plugins-‐mailer" % "2.2.0" )
WebJars
48@PeterHilton •
JavaScript/front-end library management
Specify dependencies in sbt Use Play reverse routing to resolve URLs: @routes.WebJarAssets.at(WebJarAssets.locate("jquery.min.js"))
class EmailActor extends Actor {
override def receive = { case m@EmailMessage(to, subject, body) => {
import com.typesafe.plugin val mailer: MailerAPI = plugin.use[MailerPlugin].email
mailer.setRecipient(to: _*) mailer.setSubject(subject) mailer.setFrom(from) mailer.send(body.trim) } } }
play-plugins-mailer
50@PeterHilton •
Sending e-mail. Send asynchronously from an Akka actor.
It Just Works.
Lessons learned
Development is fast and predictable if you’ve used the same architecture before.
‘We only support Chrome’ is possible twice.
One thing didn’t work: authentication via NTLM challenge-response on Microsoft IIS
51@PeterHilton •
Scaling app dev: lessons learned
Scale down
53@PeterHilton •
High-performance technology can scale down, as well as up. Who knew?
It turns out that Play and Scala make simple applications easier to build. Bonus: maintainability and performance
Get Play framework benefits
Template system allows simple HTML and using existing front-end frameworks.
HTML form validation API results in clear, understandable code.
No XML.54@PeterHilton •
Get Scala benefits
Strong types capture the domain model more explicitly and clearly (DDD FTW!)
Less verbose code, with immutable types, is easier to debug and maintain.
Third-party Java libraries remain essential.55@PeterHilton •
Scale down the architecture
56@PeterHilton •
No web front-end development (no custom JavaScript or CSS) Standard action-based MVC (server-side form validation only) Database most familiar to the team (avoid surprises and getting stuck) No reactive programming (would be premature optimisation here)
Scaling projects: lessons learned
Scaling down the scope
The first version of a business application has a lot in common with a start-up’s MVP (although a start-ups usually include front-end dev and branding in ‘minimum viable’)
59@PeterHilton •
Scaling up productivity
Throughput. Cycle time.
One developer on the team needs to know enough about agile software development to be able to get people using the software before the project gets cancelled.
60@PeterHilton •
Scaling down the team
Scale down the architecture first.
The team size trade-off is: communication overhead (big team) vs skills gaps (small team)
61@PeterHilton •
Vertical and horizontal scaling
In this context, vertical scaling is about making each developer more productive.
Horizontal scaling means more developers … at the cost of exponentially increasing overhead.
62@PeterHilton •
