ibm db2 json an overview of db2 10.5 capabilities as a...
TRANSCRIPT
#IDUG
IBM DB2 JSON An overview of DB2 10.5 capabilities as a JSON document store Olivier Bernin
IBM
Session Code: 4
April 16th, 14:45 | Platform: DB2 LUW
#IDUG
Agenda
Introduction – what's JSON, requirements, setting up, ...
Using IBM DB2 JSON Command line tool
Java API
NoSQL Wire Listener
Under the bonnet
#IDUG
What is JSON ?
JavaScript Object Notation “open standard format that uses human-readable text to transmit data objects consisting of attribute–value pairs”
Alternative to XML
Simple
Human-readable
Flexible
Compact
JSON is JavaScript
Example
#IDUG
How is JSON used ?
Rich Internet Applications
Desktop Applications
Mobile Applications
Application Server
Database
JSON (no schema)
#IDUG
What is DB2 JSON ?
A DB2 built-in solution that brings together the flexibility of the JSON data representation with the enterprise features and quality of service associated with RDBMs.
In practice: Natively embedded in DB2 (out of the box)
Uses the MongoDB programming model and query language
Provides support for additional data types (date, etc ...)
3 ways to access data Interactive command line shell
Java driver – access JSON data through familiar JDBC API
MongoDB protocol driver – allow access through a number of programming languages
JavaScript, PHP, Python, Ruby, Perl, C, C++, etc ...
#IDUG
DB2 JSON – Architecture
DB2
DB2 Engine JSON_VAL JSON UDFs
DB2 JSON Java API
NoSQL Wire Listener
Applications
NoSQL CLP (db2nosql)
MongoDB Wire
Protocol
SQL
JDBC
#IDUG
Setting up DB2 for JSON – Software Requirements
IBM® DB2® Version 10.5 Fix Pack 1 for Linux, UNIX, and Windows
Requires JDBC db2jcc.jar for the IBM Data Server Driver Package
Requires the DB2 NoSQL JSON package:
DB2 NoSQL JSON library nosqljson.jar
Rhino JavaScript engine js.jar, which is required if you are using the command-line processor
Java™ Runtime Environment 1.5 or greater
#IDUG
Setting up DB2 for JSON – Configuration Steps
Add java to the PATH
Add db2jcc.jar to the CLASSPATH or the -cp in the script Other JAR files will be needed depending on technique used
Ensure that the user has the following administrative privileges:
CREATETAB and CREATE FUNCTION in the SYSTOOLS schema.
DBADM authority for the target database
Create or use an existing UTF-8 database with a 32K page size
Ensure that the DFT_TABLE_ORG database configuration parameter is set to ROW
#IDUG
Using the command line tool - db2nosql
Add the nosqljson.jar & js.jar to the CLASSPATH
Start the client db2nosql -db <dbname> -user <user> -password <password>
-hostname <host> -port <port>
nosql> prompt
Enable the database
nosql>enable(true)
Create required system objects. Run only once pe DB
That's it !
#IDUG
A word about the programming model
DB2 JSON deals with “Collections” of “Documents” A document is a JSON object
Document always have an ID (“_id”) - maybe implicit
A collection is a “heap” of document
Collections do not enforce any schemas on documents !!!
Documents can be added, removed, searched, etc ... in collections
Collections have names, optionally a namespace
Collections are stored in the DB2 database
Support for additional data types on top of native JSON ones $string, $number
$int, $date, $timestamp, $binary, $oid
Predicates for search, filtering, sorting, etc ... are JSON objects Use special operators
Example: { “price”: { $lt : 9.99 } }
#IDUG
Demo – Creating a collection & inserting a document
db.createCollection(“movies”)
db.movies.insert({
title: "2001: A Space Odyssey",
director: "Stanley Kubrick",
year: 1968,
duration: 160,
category: "Science Fiction",
cast: [
{ name: "Keir Dullea", character: "Dr. Dave Bowman" },
{ name: "Gary Lockwood", character: "Dr. Frank Poole" },
{ name: "William Sylvester", character: "Dr. Heywood R. Floyd"
}],
rating: 8.3})
#IDUG
Demo – Creating a collection & inserting a document
#IDUG
Demo – Importing documents
db.movies.importFile(“movies.js”)
#IDUG
Demo – Search & query documents
Find all movies directed by Stanley Kubrick db.movies.find({ director: "Stanley Kubrick" })
#IDUG
Demo – Search & query documents
Find all movies released in 1968, show the titles & directors db.movies.find({ year: 1968 }, { title: 1,
director: 1 })
#IDUG
Demo – Search & query documents
Find all movies released before 2008, show the titles & directors db.movies.find({ year: { $lt: 2008 }}, { title: 1,
director: 1 }))
#IDUG
Demo - Aggregation
Find the average rating for all movies from a given director db.movies.group({ _id: "$director", avgRating: {
$avg: "$rating"}} )
#IDUG
Using the command line shell – a few more options
Showing the current database connection: db -> Database: jdbc:db2:JSON Schema: TEST
Setting the JSON collection namespace: use e.g. use mynamespace
Default one is “TEST”
Displaying the current namespace: show dbs
#IDUG
Connecting using the DB2 JSON Java API (JDBC)
Enable the database using the db2nosql script
Use the -setup enable option
Use the com.ibm.nosql.json.api.NoSQLClient class to obtain a DB object that represents a connection to the JSON store
Establishes a JDBC connection to the DB2 database
Pass an existing java.sql.Connection or connection parameters (url, user, pwd, etc ...)
DB db = NoSQLClient.getDB(jdbcUrl, user, pwd)
#IDUG
Objects provided by the DB2 JSON Java API
Same programming model as with the command line shell
JSON Collection <=> com.ibm.nosql.json.api.DBCollection
JSON Document <=> com.ibm.nosql.json.api.BasicDBObject
JSON Arrays <=> com.ibm.nosql.json.api.BasicDBList
Queries return cursor as com.ibm.nosql.json.api.BasicDBObject Can be iterated over using Java “foreach” construct
#IDUG
An example
Attached file - Sample.java
#IDUG
More advanced features
Index
Transactions
Batch processing Send multiple documents to be processeds at once
Performance Lazy fetch
Fetch documents in successive groups instead of all at once
Fire-and-forget mode
Multi-threaded high-performance mode of update
#IDUG
Using the IBM NoSQL Wire Listener for DB2
The MongoDB Wire Protocol is a simple socket-based, request-response style protocol allowing the transmission over TCP/IP of JSON documents between clients and compatible JSON store
Provides programming language independant access
Requires a MongoDB driver for the onsidered language
JavaScript, Python, C, C++, Perl, PHP, Ruby, etc ...
DB2 provides an IBM NoSQL Wire Listener allowing compatible drivers to connect to DB2 JSON
MongoDB API JSON Java API
IBM NoSQL Wire Listener
Applications
DB2
#IDUG
Getting started with the NoSQL Wire Listener
Enable the database using the db2nosql script
Use the -setup enable option
Start the listener
wplistener -start -mongoPort <port> -dbName
<db> -userId <user> -password <password>
To stop it
wplistener -shutdown -noSQLHost -mongoPort
<port> -dbName <db> -userId <user>
#IDUG
Under the bonnet
The command line tool debug option: debug(true) Will display the SQL executing under the cover
Creating a collection: db.createCollection(“movies”)
A table named after the collection is created
2 columns: ID & DATA, type is BLOB
CREATE TABLE TEST."movies" ( ID CHAR(12) FOR BIT DATA NOT NULL, DATA BLOB(16M) INLINE LENGTH 25000, PRIMARY KEY(ID)) ORGANIZE BY ROW
#IDUG
Under the bonnet What does the data look like ?
SELECT * FROM TEST.”movies”
Making sense of it ... SELECT SYSTOOLS.BSON2JSON(DATA) AS JSON_DATA FROM TEST."movies"
SYSTOOLS.BSON2JSON() UDF created when database is enabled
See what happens by running enable(false)
ID DATA --------------------------- -------------------------- x'534645A535375FFAF80E7B65' x'0359010000027469746C6... x'534645CC35375FFAF80E7B66' x'0336010000027469746C6... x'534645CC35375FFAF80E7B67' x'0349010000027469746C6... x'534645CC35375FFAF80E7B68' x'0341010000027469746C6...
JSON_DATA ------------------------------------------------------ {"title":"2001: A Space Odyssey","director":"Stanle... {"title":"E.T. The Extra-Terrestrial","director":"S... {"title":"Raiders of the Lost Ark","director":"Stev... {"title":"Lincoln","director":"Steven Spielberg","y...
#IDUG
BSON ?
Binary JSON BSON [bee · sahn], short for Binary JSON, is a binary encoded serialization of JSON-like documents.
- bsonspec.org
Following properties Lightweight
Traversable
Efficient
More at bsonspec.org
#IDUG
Under the bonnet - Searching
db.movies.find({ director: "Steven Spielberg" })
New SYSIBM.JSON_VAL function SYSIBM.JSON_VAL(<BSON_BLOB>, <JSON_PATH>, <TYPE_SPEC>)
SELECT ID, DATA FROM TEST."movies" WHERE JSON_VAL(DATA, 'director', 's:2048:na')=?
SELECT JSON_VAL(DATA, 'title', 's:32') AS TITLE, JSON_VAL(DATA, 'year', 'i') AS YEAR FROM TEST.\"movies\"
TITLE YEAR -------------------------------- ----------- 2001: A Space Odyssey 1968 E.T. The Extra-Terrestrial 1982 Raiders of the Lost Ark 1981 Lincoln 2012
#IDUG
Under the bonnet What we have seen so far ...
JSON Documents are stored inside tables as BSON BLOBs Table represents the “collection”
Schema name is collection “namespace”
Set of new SQL functions provided to manipulate the documents JSON_VAL, BSON2JSON, JSON2BSON, etc ...
JSON_VAL used to parse BSON BLOB and extract a typed value
What about performances ??
#IDUG
Under the bonnet - Indexing
Creating an index on a JSON document field db.movies.ensureIndex({ director: 1 })
Expression based-index Indexing the results of a functional expression on one or more columns
New feature introduced with DB2 10.5
Expression-based indexes in IBM DB2 10.5 InfoCenter
CREATE INDEX TEST."movies_xdirector" ON TEST."movies" (JSON_VAL(DATA,'director','s:1024:na'))
#IDUG
IBM DB2 JSON on DB2 for z/OS
Supported from DB2 10.5 FP3
More required components
Database enablement is installation task performed by Database or System administrator.
No supported from script – see APAR PM98357
#IDUG
Thank you !
Questions ?
#IDUG
Olivier Bernin IBM [email protected]
Session 4 IBM DB2 JSON An overview of DB2 10.5 capabilities as a JSON document store
Please fill out your session
evaluation before leaving!