naïveté vs. experience

Navet vs. ExperienceHow We Thought We Could Use Scala and Clojure, and How We Actually Did

Who

AI, Distributed Simulation, Code Generation, Machine Vision

C, C++, Java, CLIPS, Clojure, Scala

Co-author of The Joy of Clojure http://joyofclojure.com

Local Boy

Michael Fogus: Programmer

Why

Java

Compression Potential

Compression Potential

Essential Boilerplate


equals

hashCode

get

set

public Foo

toString


equals

hashCode

get

set

public Foo

toString

try

catch

finally

;

{

}

@override

static

int

System.out

boolean

.

void

import

String

Exception

throws

toString

toString

throws

throws

finally

finally

finally

get

get

equals

hashCode

try

}

System.out

boolean

toString

throws

throws

finally

get

public Foo

catch

finally

;

int

import

String

Exception

throws

toString

get

equals

get

toString

try

finally

static

import

String

toString

throws

finally

equals

try

toString

throws

get

finally

String

Exception

hashCode

finally

{

@override

int

void

Exception

toString

throws

finally

hashCode

toString

throws

finally

public Foo

finally

int

Exception

@override

static

finally

hashCode

try

throws

throws

static

@override

hashCode

throws

hashCode

@override

void

toString

toString

int

hashCode

void

finally

finally

public Foo

int

hashCode

@override

void

toString

toString

int

hashCode

void

finally

finally

public Foo

int

hashCode

@override

void

toString

toString

int

hashCode

void

finally

finally

public Foo

int

toString

static

finally

equals

try

finally

String

toString

throws

finally

toString

finally

What

Java

Java

Compiler

Problem

We can speak to the compiler or the problem-space. In Java the compiler is often king.Java

List foo = new ArrayList();foo.add(new HashMap() {{ put(bar, baz); }});Compiler

Problem

Java

Compiler

Problem

Placating the compiler can often occlude the problem at hand.

Java != JVM

Reprieve

JRuby

Jython

Jess

Scala

Clojure

Groovy

Rhino

JRuby was out because we use Python/Jython

Jess was replaced by Drools and relegated to a supporting role

Groovy provided nothing over Jython

Rhino was too scaryJRuby

Jython

Jess

Scala

Clojure

Groovy

Rhino

Scala

Clojure

Scala

Scala

Compiler

Problem

Using Scala allows us to speak more to the problem at hand.Scala

Compiler

Problem

val foo = List[Map[String,String]]()foo ++ Map("bar" -> "baz")

Scala

Compiler

Problem

The problem is only partially occluded. This can increase the closer to remain to Java's semantics.Clojure

Clojure

Compiler

Problem

You rarely talk to the compiler, and when you do it's often at the end when trying to gain speed.(def foo {})(assoc foo bar baz)Clojure

Compiler

Problem

Clojure

Compiler

Problem

zzz

Clojure

Compiler

Problem

Runtime

We've changed the model entirely. Runtime is king and this is scary for some coming from a static language like Java.

DSLs help to address the problem space directly.

Scala has good support for DSLs too, but not to the insane levels of Clojure/Lisp.Essential Boilerplate

public class Person { private String lastName; private String firstName; private Person spouse;

public Person(String firstName, String lastName, Person spouse) { this.lastName = lastName; this.firstName = firstName; this.spouse = spouse; }

public String getFirstName() { return firstName; } public String getLastName() { return lastName; } public Person getSpouse() { return spouse; }

public String toString() { return firstName + " " + lastName +

(spouse != null ? " married to " + spouse.getFirstName() + "." : "."); }

}

This was my initial pitch...Less Boilerplate

case class Person(val firstName:String, val lastName:String, val spouse:Option[Person]) {

public Person(String firstName, String lastName, Person spouse) { this.lastName = lastName; this.firstName = firstName; this.spouse = spouse; }




}

Kittens





}

These intermediate transformations are irrelevant given that they were not legal Scala nor Java... so why not kittens?Kittens and Aquaman


override def toString():String = firstName + " " + lastName +

return firstName + " " + lastName +


}

Scala


override def toString():String = firstName + " " + lastName +

(spouse match {

case None => "."

case Some(s) => " married to " + s.firstName + "."

});

}

By default, Scala classes are immutable.

This transformation demonstration was a huge hit! Things were beautiful, and the Java devs could understand the code fairly well.

But...Disingenuous

This exercise is a dishonest pitch.

1) It doesn't speak directly to Scala's strengths

2) Often interoperability hinders such drastic code reductions

3) ...Of Course...

(defrecord Person [fname lname spouse] Object

(toString [_]

(str fname " " lname

(when-let [n (:fname spouse)]

(str " married to " n))

\.)))


Demonstrating a similar transformation for Clojure was less than successful...

Scala Won
(mostly)

Mostly...

Scala is used in the mainline source.

Clojure is relegated to support tools and one-offs.

(I also use it for cron jobs)How

Embedded XML

Everything is an Object

Functional Programming

List Comprehensions

Case Classes

Pattern Matching

Interoperability

Currying

Implicits

Architecture

ExecDispatchServiceMiddleware

The Internetservices

This is a simplified for clarity and to make the points at hand.

Four layers per service.

Recursive definitions.



Inbound requests touch 3 layers

Outbound responses touch 1

This is also a simplification, but good as an overview.



Service layer is just a public facade.



Middleware is responsible for the actions to be performed (gathering, filtering, etc...). This is also where caching occurs (where appropriate).



Dispatch determines where the sub-services are located. Sometimes there are local and sometimes remote.



Services do the actual work...



Exec is just an executor. Sometimes results come back directly and sometimes they are delayed. Exec knows what to do in either case. Exec also handles execptions.




The Internet


The Internet

Let's zoom into one of the little-s services to see what it looks like...


The Internet


A service is also a ServiceUsing this abstraction we are able to gain a high-level of reusability.


The Internet


Navet

Embedded XML



List Comprehensions

Case Classes

Pattern Matching

Interoperability

Currying

Implicits

Interoperability


Embedded XML


List Comprehensions

Case Classes

Pattern Matching

Currying

Implicits

Interoperability


List Comprehensions

Pattern Matching

Currying

Experience

Interoperability

Type Aliases

Traits


List Comprehensions

Immutability

Pattern Matching

Currying

Closures

Disingenuous

Disingenuous

Interoperability
(near seamless essential)

Type Aliases

D.I.

Type Aliases

class Service extends IService {

def get(ref:ReferenceParam):ComponentReturn = { ... }

...

}


Although not functional, type aliases turned out to be extremely useful.Type Aliases



...

}

Type Aliases



...

}

type ReferenceParam = java.util.Map[String,String]

type ComponentReturn = java.util.Map[String,Object]

We could describe interfaces using logical names of types and somewhere else describe the concrete types.Traits

D.I.

D.I.

Traits



...

}

trait RPCTypes {



}

The concrete types were aggregated in traits. (this is a simplified view... in reality we split the return and parameter types and used implicits as glue)Traits

class Service extends IService with RPCTypes {


...

}

trait RPCTypes {



}


The Internet


RPCTypesLocalTypesCachedTypesDelayedTypes...

RPCTypesLocalTypes...

Now we just mix in the types (and the conversions as needed)Functional Programming

List Comprehensions

Immutability

When a tree falls in a lonely forest ... does it make a sound?

Charles Riborg Mann and George Ransom Twiss

If a pure function mutates some local data in order to produce an immutable return value, is that ok?

Rich Hickey

Scala and Java do not enforce immutability and in fact in interop the very opposite is the case. We were able to be quite successful using tree falls in the woods immutability however.Pattern Matching
(a language for error shapes)

Currying
(gratuitous, but cool, control structures)

Closures
(gratuitous, but cool, simplified delays)

Woe

Breaking Changes

Optional Immutability

Weak Laziness

Less Functional

Grand Hierarchies

Literals-lite

Breaking Changes

Optional Immutability

Weak Laziness

Less Functional

Grand Hierarchies

Literals-lite

Clojure!

Thanks To

You

My employer

Dean Wampler for feedback

Rich Hickey and Martin Odersky for Clojure and Scala

Chris Houser my co-author

Ryan Tomayko for the screaming face

The fam

Questions?
(example questions below)

Joel or Mike?

Is Scala too complicated?

Best zombie movies?

Kobaia is de hundin?

Favorite hangout in Baltimore?

What's with the tie?

naïveté vs. experience

Technology