webinar - ad-hoc querying with nosql - n1ql

N1QL (“Nickel”)

Don Pinto

Product Manager

The new query language from Couchbase

Agenda

• Data Models

- Rich vs. Basic Data Model

• N1QL Concepts

- What is N1QL

- What you get with N1QL

- Foundation

• Use-case example

- E-commerce

• Resources and Next Steps

The Real World

Needs a rich data model to represent it

Simple Data Model

And why it is not sufficient

A simple data model for a book productModel representing

a book product

Book specific attributes

Product

IdSKUProductDescription

…AuthorISBN NumberPublisherPublish DateLanguage…

Adding music productsModel representing

a music product

Music record specific attributes

Product

IdSKUProductDescription

…Label NameArtistTitleRelease DateFormat…

Adding jeans productsModel representing

a jeans product

Jeans attributes are different and not consistent across

brands and makes

Product

IdSKUProductDescription

…BrandMakeWidthLengthColorStyle…

The basic data model

1. Works well for a while

2. When few new types are introduced becomes harder

but manageable

3. Not possible when more product types are introduced

Comparing the data models

Basic Data Model Rich Data Model

Technology Relational Document

Modeling the Real World

o Tables and dependencieso Model-driven joins

Rich objects Domain-driven joins

ApplicationFit

o Model translationo Data translation

Model match Data match

Flexibilityo Data uniformityo Change is costly

Data variety Change is embraced

Summarizing the Models

Basic Data Model Rich Data Model

…is about Rectangles

• Decompose objects into tables

• Assemble tables to recover objects

…is about Triangles

• Represent object structure and depth

• Represent object connections

N1QL

What is N1QL

The new query language from Couchbase

Embraces the JSON document model

SQL-like syntax eases transition for app developers

Familiarity of SQL but also goes beyond SQL

Enables you to easily build rich apps using Couchbase

What you get with N1QL

Next generation query language for structured, semi-structured and unstructured data

SQL look and feel but with the power of a flexible data model

JSON fully supported, additional formats could be supported in the future

Will be tightly integrated with Couchbase Server, supported by backend infrastructure

What you get with N1QL (contd.)

Aggregations, filtering, transformations

Paths, Arrays, Dates

Execute queries from your application

Navigate documents and document structures

Retain the flexibility offered by Couchbase Server

Robust Foundation

N1NF = Non-1st normal form = Document Multivalued attributes Nested objects Superset of relational model

Infrastructure High performance declarative indexes Per document atomicity Tools – interactive shell, online tutorial, etc.

BASICS

A simple query in N1QL has three parts to it:

• SELECT – Parts of document to return

• FROM – Data bucket, or data store to work with

• WHERE – Conditions the document must satisfy

• Output (resultset) is in the form of a JSON document

SELECT *

FROM customer

WHERE lastName = "Tremblay"

STRINGS

N1QL supports string operations as well as matching, such as:

• Concatenation

• Pattern Matching using wildcards

SELECT firstName || " " || lastName as fullNameFROM customerWHERE emailAddress LIKE "%.biz"

"resultset": [ {

"emailAddress": "don@armstronghaley.biz",

"fullName": "Don Pinto" }, .....

Advanced SELECT

N1QL provides support for sorting and grouping data

• GROUP BY, ORDER BY

• Pagination – LIMIT, OFFSET clauses

SELECT count(*), stateFROM customerWHERE customer.ccInfo.cardType="discover" GROUP BY customer.stateORDER BY customer.stateLIMIT 5 OFFSET 5

SIMPLE ARITHMETIC

N1QL supports simple arithmetic expressions such as:

• AVG, ROUND, TRUNC

• SUM, MIN, MAX

SELECTAVG(reviews.rating) / 5 as normalizedRating, ROUND((avg(reviews.rating) / 5), 2) as roundedRating,TRUNC((avg(reviews.rating) / 5), 3) as truncRating FROM reviews AS reviewsWHERE reviews.customerId = "customer62"

Duplicates, NULLs, Missing Values

• DISTINCT keyword can be used to eliminate duplicates

• Distinguish between explicit NULL versus Missing Values

SELECT COUNT( DISTINCT customerId ) FROM purchases

SELECT fname, children FROM tutorial

WHERE children IS NULL

SELECT fname, children FROM tutorial

WHERE children IS MISSING

ARRAYS and OBJECTS

N1QL provides first class support for nested objects and Arrays

• Dot “.” operator used to access fields nested inside objects

• Bracket “[ ]”

SELECTAVG(reviews.rating) / 5 as normalizedRating, ROUND((avg(reviews.rating) / 5), 2) as roundedRating,TRUNC((avg(reviews.rating) / 5), 3) as truncRating FROM reviews AS reviewsWHERE reviews.customerId = "customer62"

Collection Expressions

Collection predicates allow you to test a boolean condition over the elements of a collection

To search for purchase orders with a particular item purchased 5 times or more –

SELECT *FROM purchasesWHERE ANY item IN purchases.lineItems

SATISFIES item.count >= 5 END

JOINs

A JOIN in N1QL is similar to SQL; a single result is produced for each matching left and right-hand input.

Identify non-performing products –

SELECT product, avg(reviews.rating) avgRating, count(reviews) numReviews

FROM product join reviews keys product.reviewList

GROUP BY product having avg(reviews.rating) < 1

JOINs - UNNEST

The UNNEST clause allows you to take contents of a nested array and join them with the parent object.

To list products belonging to a particular category -

SELECT pFROM product pUNNEST p.categories as category WHERE category = "Appliances"

E-Commerce Example

E-commerce Shoppers

• Buy things online from the e-commerce site

• They do things like

- Browsing and searching items

- Checking out deals

- Viewing item ratings

- Viewing the Top 10 items

- And more..

Shopping for hotels

{“ID”: 1,“NAME”: “Fairmont San Francisco”,“DESCRIPTION”: “Historic grandeur…”,“AVG_REVIEWER_SCORE”: “4.3”,“AMENITY”: {“TYPE”: “gym”,

DESCRIPTION: “fitness center”},{“TYPE”: “wifi”,“DESCRIPTION”: “free wifi”},

“RATE_TYPE”: “nightly”,“PRICE”: “$199”,“REVIEWS”: [“review_1”, “review_2”],“ATTRACTIONS”: “Chinatown”,

}

JSON

{“ID”: 2,“NAME”: “W San Francisco”,“DESCRIPTION”: “Chic, hip accommodations..”,“AVG_REVIEWER_SCORE”: “4.0”,“AMENITY”: {“TYPE”: “spa”,

DESCRIPTION: “Bliss Spa”},{“TYPE”: “wifi”,“DESCRIPTION”: “free wifi”},{“TYPE”: “dining”,“DESCRIPTION”: “bar/lounge”},

“RATE_TYPE”: “nightly”,“PRICE”: “$194”,“REVIEWS”: [“review_1”, “review_2”],

} JSON

Hotels

Reviews and Ratings

{“ID”: 1,“NAME”:

“Fairmont San Francisco”,…}

JSON

Hotel Descriptions

Reviews{“REVIEW_ID”:

1,“REVIEW”:

“Loved Hotel…”,…}

JSON

{“REVIEW_ID”:

2,“REVIEW”:

“Nice, but …”,…}

JSON

User Profiles{“USER_ID”: 1,

“DISPLAY”: “Ted’s Trip…”,…}

JSON

{“USER_ID”: 2,“DISPLAY”:

“WhatWhat …”,…}

JSON

Document IDs associates related objects

Hotels points to reviews

Reviews points to users

E-commerce Merchants

• Run the e-commerce business online

• They do things like

- View business reports

• Month-over-month sales report

• Top 10 purchased / reviewed products

- Review and ship products to the customer by purchase order

- Monitor activity of a shopper on the site

- And more..

Customer

{"resultset": [

{"ccInfo": {

"cardExpiry": "2013-09-12","cardNumber": "1228-1221-1221-1431","cardType": "discover"

},"customerId": "customer0","dateAdded": "2013-06-23T04:32:31Z","dateLastActive": "2013-07-23T04:32:31Z","emailAddress": "zion@armstronghaley.biz","firstName": "Rosella","lastName": "Tremblay","phoneNumber": "1-543-962-9861 x534","postalCode": "75832","state": "PR","type": "customer"

}]

}

Product{"resultset": [{"categories": ["Luggage”,“Accessory”

],"color": "indigo","dateAdded": "2013-09-20T01:42:49Z","dateModified": "2014-03-19T01:42:49Z","description": "This product is available on","imageURL": "http://ecx.images-amazon.com/images/I/41i6-Jg3D8L._SL300_.jpg","name": "Briggs & Riley 17 Inch Executive Expandable Rolling Briefcase","productId": "product0","reviewList": ["review534”,"review8672"

],"type": "product","unitPrice": 367.2

}]

}

Purchases{

"resultset": [{"customerId": "customer845",

"lineItems": [{

"count": 5,"product": "product659"

},{

"count": 5,"product": "product379"

},{

"count": 5,"product": "product294"

},{

"count": 3,"product": "product620"

},{

"count": 2,"product": "product626"

}],"purchaseId": "purchase0","purchasedAt": "2013-06-22T01:42:55Z","type": "purchase"

}]

}

Reviews

{"resultset": [{"customerId": "customer62","productId": "product864","rating": 5,"reviewId": "review0","reviewedAt": "2013-11-19T01:42:50Z","type": "review"

}]

}

N1QL HomeN1QL

N1QL Developer Preview 3

available today

at

http://query.couchbase.com

Find documentation, tutorial and useful resources here

Demo – E-Commerce N1QL Queries

N1QL HomeN1QL

N1QL Developer Preview 3

available today

at

http://query.couchbase.com

Find documentation, tutorial and useful resources here