devnation'15 - using lambda expressions to query a datastore

Using Lambda Expressions to Query a

Datastore

Xavier Coulon - Red Hat @xcoulon

The idea

The idea

Lambda Expressions:

• provide type safety

• describe the "what", not the "how"

The idea

The idea

u -> ( u.firstName.equals("john") && u.lastName.equals("doe")) || u.userName.equals("jdoe"))

final List<User> users = userCollection.filter( u -> ( u.firstName.equals("john") && u.lastName.equals("doe")) || u.userName.equals("jdoe")).toList();

Introducing project

Lambdamatic

Part I. The building blocks

Building blocks

• Metamodel generation • Lambda Expression analysis

Metamodel generation (à la JPA)

package com.example.domain;

public class User { private String id; private String firstName; private String lastName; private String userName;

}

Metamodel generation


@Document(collection="users") public class User {

@DocumentId private String id;

@DocumentField private String firstName;

private String lastName;

@DocumentField(name="uName") private String userName; }


@Generated(value="LambdamaticAnnotationsProcessor") public class QUser implements QueryMetadata<User> {

@DocumentField(name="_id") public StringField firstName;

@DocumentField(name="firstName") public StringField firstName;

@DocumentField(name="lastName") public StringField lastName;

@DocumentField(name="uName") public StringField userName; }

Query Metadata class generation



@Generated(value="o.l.m.a.DocumentAnnotationProcessor") public class UserCollection extends LambdamaticMongoCollectionImpl<User, QUser> {

public UserCollection(final MongoClient mongoClient, final String databaseName) { super(mongoClient, databaseName, "users", User.class); }

}

Collection classpackage com.example.domain;

@Document(collection="users") public class User {


@DocumentField private String firstName;

private String lastName;

@DocumentField(name="uName") private String userName; }


CDI Integration

@Generated(value="org.lambdamatic.mongodb.apt.DocumentAnnotationProcessor") @ApplicationScoped public class UserCollectionProducer { @Produces public UserCollection getUserCollection(final MongoClient mongoClient, final MongoClientConfiguration mongoClientConfiguration) { return new UserCollection(mongoClient, mongoClientConfiguration.getDatabaseName(), "users"); }

}

final List<User> users = userCollection.filter( u -> ( u.firstName.equals("john") && u.lastName.equals("doe")) || u.userName.equals("jdoe")).toList();

Metamodel in action

public class UserCollection extends LambdamaticMongoCollectionImpl<User, QUser> { ... }


@Generated(value="LambdamaticAnnotationsProcessor") public class QUser implements QueryMetadata<User> {

@DocumentField(name="_id") public StringField firstName;

@DocumentField(name="firstName") public StringField firstName;

... }

Metamodel benefits

Geospatial queries

Metamodel benefits

public class Location {

/** The latitude value.*/ private double latitude;

/** The longitude value.*/ private double longitude;

}

public List<BikeStation> findWithin(final Location[] corners) { return bikeStationCollection.find(s -> s.location.geoWithin(corners)) .toList(); }

public class QBikeStation implements QueryMetadata<BikeStation> {

@DocumentField(name="location") public LocationField location;

...

}

@Document(collection="bikestations") public class BikeStation { @DocumentField private Location location;

...

}

public interface LocationField {

public boolean geoWithin(final Location[] corners);

... }

Lambda Expression Analysis

(this is the fun part, really)


@FunctionalInterfacepublic interface FilterExpression<T> extends Predicate<T>, Serializable { }

Step#1 : Use a Serializable Functional Interface


public static <T> SerializedLambda getSerializedLambda( final FilterExpression<T> expression) { ... final Class<?> cl = expression.getClass(); final Method m = cl.getDeclaredMethod("writeReplace"); m.setAccessible(true); return (SerializedLambda) m.invoke(expression); ... }

Step#2 : Locate the generated method


Step#3 : read the generated bytecode Label L1125736023 VarInsn ALOAD #0 FieldInsn GETFIELD com/sample/QUser.firstName (desc=Lorg/lambdamatic/mongodb/metadata/StringField;) LdcInsn john MethodInsn INVOKEVIRTUAL java/lang/Object.equals (desc=(Ljava/lang/Object;)Z) JumpInsn IFEQ L1636182655 VarInsn ALOAD #0 FieldInsn GETFIELD com/sample/QUser.lastName (desc=Lorg/lambdamatic/mongodb/metadata/StringField;) Label L1122606666 LdcInsn doe MethodInsn INVOKEVIRTUAL java/lang/Object.equals (desc=(Ljava/lang/Object;)Z) JumpInsn IFNE L350068407 Label L1636182655 VarInsn ALOAD #0 FieldInsn GETFIELD com/sample/QUser.userName (desc=Lorg/lambdamatic/mongodb/metadata/StringField;) LdcInsn jdoe MethodInsn INVOKEVIRTUAL java/lang/Object.equals (desc=(Ljava/lang/Object;)Z) JumpInsn IFNE L350068407 Insn ICONST_0 JumpInsn GOTO L1820383114 Label L350068407 Insn ICONST_1 Label L1820383114 Insn IRETURN Label L369049246 LocalVariable u (desc=Lcom/sample/QUser;) index=0


Step#4 : build an AST

if(u.firstName.equals("john")) { if(u.lastName.equals("doe")) { return true } else { if(u.userName.equals("jdoe")) { return true } else { return false } } } else { if(u.userName.equals("jdoe")) { return true } else { return false } }

if(u.firstName.equals("john")) { if(u.lastName.equals("doe")) { return true } else { if(u.userName.equals("jdoe")) { return true } else { return false } } } else { if(u.userName.equals("jdoe")) { return true } else { return false } }

if(u.firstName.equals("john")) { if(u.lastName.equals("doe")) { return true } else { if(u.userName.equals("jdoe")) { return true } else { return false } } } else { if(u.userName.equals("jdoe")) { return true } else { return false }

(u.firstName.equals("john") && u.lastName.equals("doe"))

|| (u.firstName.equals("john") && !u.lastName.equals("doe") && u.userName.equals("jdoe"))

|| (!u.firstName.equals("john") && u.userName.equals("jdoe"))


Step#5 : Thin out the AST





Step#6 : Further simplify the AST





Boolean algebra to the rescue !a.(b.c) = a.b.c a + (b + c) = a + b + c a.(!a + b).(!a + c) = a.b.c a + (!a.b) + (!a.c) = a + b + c a.(a + b).(a + c) = a a + (a.b) + (a.c) = a (a.b) + (a.c) + (a.d) = a.(b + c + d) (a + b).(a + c).(a + d) = a + (b.c.d) a + a = a a.a = a a.!a = O a + !a = I O + a = a O.a = O 1 + a = 1 1.a = a a.(b + c + d) = (a.b) + (a.c) + (a.d) a + (b.c.d) = (a + b).(a + c).(a + d)


Step#6 : further simplify the statement tree




(u.firstName.equals("john") && u.lastName.equals("doe")) || u.userName.equals("jdoe")

Performances ?

• Byte code is analyzed on first access to Lambda Expression ( <100ms)

• "Raw" AST is cached for subsequent calls

• Placeholders in AST for captured arguments

Part II. Managing data on MongoDB

Managing Data on MongoDB

• Providing Codecs for Lambda Expressions and Documents

• Skipping the "DBObject" DTOs layer

MongoDB Java Driver Integration

Querying MongoDB

Service Layer

Lambda Expression

MongoDB Java Driver

BSON Document

MongoDB

FilterExpression Codec

BSON Document

Domain object(s)

Document Codec

Demo #1 Geolocation queries

Three more things (yes, three)

Query on Arrays

Query on Arrays

final List<BlogEntry> blogEntries = blogEntryCollection.filter( e -> e.comments.elementMatch(

c -> c.author.equals("anonymous") && c.date.greaterOrEquals(twoDaysAgo))) .toList();

final List<BlogEntry> blogEntries = blogEntryCollection.filter( e -> e.comments.author.equals("anonymous") && e.comments.date.greaterOrEquals(twoDaysAgo)) .toList();

Query Projections

Query Projection

Projection Metadata class generation

@Generated(value="o.l.m.a.DocumentAnnotationProcessor") public abstract class PBlogEntry implements ProjectionMetadata<BlogEntry> {

@DocumentField(name="_id") public ProjectionField id; @DocumentField(name="authorName") public ProjectionField authorName; @DocumentField(name="publishDate") public ProjectionField publishDate;

... }

@Document(collection="blog") public class BlogEntry {


@DocumentField private String authorName;

private Date publishDate;

... }

Query Projection

import static org.lambdamatic.mongodb.Projection.include;

final BlogEntry blogEntry = blogEntries.filter(e -> e.id.equals("1")).projection(e -> include(e.authorName, e.title, e.publishDate))

.first();

@Generated(value="o.l.m.a.DocumentAnnotationProcessor") public abstract class PBlogEntry implements ProjectionMetadata<BlogEntry> {

@DocumentField(name="_id") public ProjectionField id; @DocumentField(name="authorName") public ProjectionField authorName; @DocumentField(name="publishDate") public ProjectionField publishDate; ... }

public interface Projection { @IncludeFields public static void include(final ProjectionField... fields) {}

@ExcludeFields public static void exclude(final ProjectionField... fields) {} }

Query Projection

Projecting array elements

final BlogEntry blogEntry = blogEntryCollection.filter(e -> e.id.equals("1")) .projection(e -> include(e.authorName, e.title, e.publishDate, e.comments.elementMatch(c -> c.author.equals("anonymous") && c.date.greaterOrEquals(twoDaysAgo)))) .first();

Update Operations

Update Operations


@DocumentId private String id; /** Name of author of the blog entry. */ private String authorName; /** Title of the Blog Entry */ private String title; /** list of comments. */ private List<BlogEntryComment> comments; ... }

Update Metadata class generation@Generated(value="o.l.m.a.DocumentAnnotationProcessor") public abstract class UBlogEntry implements UpdateMetadata<BlogEntry> {

@DocumentField(name="_id") public String id; @DocumentField(name="authorName") public String authorName; @DocumentField(name="title") public String title; @DocumentField(name="comments") public UBlogEntryCommentArray comments; ... }

Update Operations

BlogEntryComment comment = ... ; blogEntries.filter(e -> e.id.equals("1")). forEach(e -> { e.lastUpdate = new Date(); e.commentsNumber++; e.comments.push(comment);});


@DocumentId private String id; /** date of last update. */ private Date lastUpdate; /** Number of comments */ private int commentsNumber; /** list of comments. */ private List<BlogEntryComment> comments;

@EmbeddedDocument public class BlogEntryComment {

/** comment author. */ private String author; /** comment date. */ private Date date; /** comment content. */ private String content;

Demo #2 Filter with projections

and Update

Project Status

Experimental

Project Info

http://github.com/lambdamatic

• Give it a try !

• Clone it / Fork it / Star it

• Open issues to discuss about API and features

http://github.com/lambdamatic

Thanks ! Q/A

devnation'15 - using lambda expressions to query a datastore

Technology