why we built kundera - the polyglot object mapper for nosqls?

© 2014 Impetus Technologies - Confidential1

Impetus Technologies Inc.

Why we built Kundera - The Polyglot Object Mapper for

NoSQLs?Vivek Shrivastava

software architect


Evolution of Data


10 LET A = 1

20 LET B = 2

30 LET C = A + B

40 PRINT C

50 END

Data

First Introduction To Data


Relational Databases

• Tables

• Relations

• SQL

• Transactions

• Normalization


Current Landscape

http://blogs.the451group.com/information_management/2014/03/18/updated-data-platforms-landscape-map-february-2014/


Infrastructure

Datastores Designed To Support Partitions


Relationships / Transaction

Datastores Designed To Support Aggregates

Item 1

Order

Item 2


Consistency

DC1

N1

DC2

N2

DC3

N3

Transactional Consistency

Replication Consistency

T=0, user1 fetches O

T=1, user2 fetches O

T=2, user1 edits and saves O

T=3, user2 edits and saves O


Consistency

Availability

Partition Tolerance

Not Possibl

e

CAP Theorem

Datastores Designed To Support Eventual Consistency


Shape

Datastores Designed To Support Richer Data Models

1 David 34

2 John 46

3 Lisa 32

4 Steve 46

5 JP 49

Tables

Graph

Columnar

Key-Value

Document


Cluster/Cloud FriendlySchema less Aggregates With

Redefined Transactions

Eventual ConsistencyRicher Data Model With Minimum Impedance

Mismatch

NoSQL

Finally


Emergence of a Polyglot World


A Single Application Using Multiple Data Storage Technologies

Polyglot Persistence


Data streams

Cassandra, HBase

User information

RDBMS

In memory cache/session

Redis

Orders / Articles / Tickets

MongoDB, Couchbase

User Social graph

Neo4J, Titan

Batch processing & analytics

Hadoop, Spark

Polyglot World


Decision Implementation

Is It Easy?

No. It’s A Bumpy Ride.


Decision

Degree of Polyglot? • X% RDBMS• Y% NoSQLComposition of Polyglot? • Datastores?

Type of Polyglot?

• Don’t choose for pride, passion and trends

• Don’t start with more than 1 NoSQL store

• Don’t start with 100% uses cases


Degree of Polyglot

Scale Relational Behavior RDBMS Non-RDBMS

Low Any 100% 0%

High High 90% 10% (New SQL)

High Moderate 75% 25% (NoSQL)

High Low 50% 50-100% (NoSQL)


RDBMS NoSQL NewSQL

Datastore + Cache Local + Global Client + Server

Composition of Polyglot - Forms


Composition of Polyglot - Datastores

Neo4j, Titan, Objectivity

Cassandra, HBase, BigTable

Oraclekv, Redis, Riak

MongoDB, Couch

base

Graph Columnar

Key-value

Document

Data Model / QueriesConsistency / Performance

Architecture / Non-Functional


Operating In The Polyglot World


Weak Polyglot

Data LayerReporting Lane

(MySQL)

Orders Lane(MongoDB)

Log Lane(Cassandra)Se

rvic

e La

yer

Web

Lay

er


Strong Polyglot

Data Layer

Polyglot MapperReports(MySQL)

+Orders

(MongoDB)+

Logs(Cassandra)Se

rvic

e La

yer

Web

Lay

er


Harmonization

Query / Indexing

Operational

Mapping / Interaction

CRUD

Atomicity

Switching / Migrating


And We Built Kundera


What It Is?

?Object Mapper• RDBMS

• NoSQLs (Cassandra, MongoDB, Hbase, Redis, Oracle NoSQL, CouchDB, Neo4j)

JPA Compliant

Polyglot


JPA

Customer(MySQL)

Order 1 (MongoDB)

{Item11, Item12}

Order 2 (MongoDB){Item21, Item22}

One to

many

Aggregate

<persistence-unit name=“cassandrapu"> <provider>com.impetus.kundera.KunderaPersistence</provider> <class>com.impetus.kvapps.entities.Order</class> <exclude-unlisted-classes>true</exclude-unlisted-classes> <properties> <property name="kundera.nodes" value="localhost" /> <property name="kundera.port" value="9160" /> <property name="kundera.keyspace" value=“orderstore" /> <property name="kundera.dialect" value="cassandra" /> <property name="kundera.client.lookup.class" value="com.impetus.client.cassandra.thrift.ThriftClientFactory" /> <property name="kundera.ddl.auto.prepare" value="create" /> <property name="index.home.dir" value="lucene"/> </properties></persistence-unit>

<persistence-unit name=“mongopu"> <provider>com.impetus.kundera.KunderaPersistence</provider> <class>com.impetus.kvapps.entities.Invoice</class><exclude-unlisted-classes>true</exclude-unlisted-classes> <properties> <property name="kundera.nodes" value="localhost" /> <property name="kundera.port" value="27017" /> <property name="kundera.keyspace" value=“invoicestore" /> <property name="kundera.dialect" value="mongodb" /> <property name="kundera.client.lookup.class" value="com.impetus.client.mongodb.MongoDBClientFactory" /> <property name="kundera.ddl.auto.prepare" value="create" /></properties> </persistence-unit>

//create entity manager factory. EntityManagerFactory emf = Persistence.createEntityManagerFactory(“cassandrapu,mongopu”, properties); EntityManager em = emf.createEntityManager(); ….. em.persist(order);

Configuration : Persistence.xml

Persist Data

At Its Simplest


Supports all major datastores and

Extensible

Polyglot CRUD / Query / Object

Mapping

Schema Generation /

Atomicity / Lazy and Eager fetching

REST Based AccessInterceptors /

Events / Caching / Connection Pool

JPQL / Native Query / Fallback

Indexing Flexibility

Additionally


Commitment to the Polyglot

Kundera.js (Polyglot JavaScript Persister)

Polyglot Data Viewer

Support for more datastores


Thank you.Questions??

https://github.com/impetus-opensource/Kundera

why we built kundera - the polyglot object mapper for nosqls?

Technology

2014 impetus

impetus

polyglot

kundera

nosql

cassandra

mongodb

redis