introduction to oracle nosql database - aioug nosql db over… · title: how to use the powerpoint...

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

Introduction to Oracle NoSQL Database

Anand Chandak Ashutosh Naik

[email protected] & [email protected]

1

mailto:[email protected]


Agenda

NoSQL Background

Oracle NoSQL Database Overview

Technical Features & Performance

Use Cases


2



Why NoSQL?

1. The four V’s of Big Data

– More Data, More Queries, Low Latency Operations (SLAs) Driving Business Innovation

2. Cost

– Cost effective horizontally scalable simple operations on simple data, Requires less HW, Lower license costs than traditional RDBMS databases Lower COO

3. Time to Market

– Developer-centric Data Management, Data Variety, Rapid Evolution Shorter Time to Market

[email protected] [email protected] 3



Where is NoSQL used?

4

Simple Data Management

Globally Distributed,

“Always On” data

Competitive Advantages of

“Fast Data”

Lower TCO, commodity

HW scale-out

ERP

EAM

Inventory Control

Accting & Payroll

Process Mgmt

Business Analytics

CRM

…

Driver

Application Real Time Event

Processing

Customer Portals

Online Banking

…

Mobile Data Management

Time Series & Sensor Data Mgmt


The NoSQL Landscape

NoSQL

Key/Value

• Keyspaces, Tables & Records

• Easy to Model

• Key-based access

• Limited Transactions

• Broad set of use cases

Columnar

• Data stored in cells grouped in columns

• Stored in continuous disk

• Data aggregation use cases

Document

• Collections

• Document-based access

• JSON & XML

• “Objects as documents” use cases

Graph

• Interconnected graphs

• Relatedness-based access

• Properties and Graphs, RDF

• Specific use cases

• Developer- centric APIs • Flexible schemas • Partitioned/sharded data

• Horizontally scalable • High Availability via Replication • Integrated with Hadoop

5 [email protected] & [email protected]

What we hear from our customers: • There is no “one size fits all NoSQL solution”

• NoSQL and SQL (RDBMS) must work together • Application version 2.0 is the one that really works



Agenda

NoSQL Background



Use Cases

6




What is Oracle NoSQL Database?

7

101100101001001001101010101011100101010100100101

Reliable Flexible Fast Simple

Cost-effective and horizontally scalable solution that addresses the needs of developers, IT and the business.

less is more


• Features

– Elastic

– Configurable transaction semantics

– Tables / JSON / Binary

– Online management

• Differentiators

– Oracle technology integrated

– Engineered Systems and Commodity HW

– Online rolling upgrades

– Streaming large object support

Architecture and features

9

Application

Storage Nodes Datacenter B

Storage Nodes Datacenter A

Application

NoSQL DB Driver

Application

NoSQL DB Driver

Application – Data Center Support

– Secondary Indexes

– Secure Access

– Flexible schema




• Query NoSQL data from Oracle Database

• Sync data using Database Mobile Server

• Access NoSQL data from Hadoop and Spark for DW and analytics

• Share data with Oracle Coherence for extensible in-memory cache grid

• Persist history & event streams for processing with Oracle Event Processing

• Store & query RDF data using Oracle RDF for NoSQL

• Replicate changes in Oracle Database to NoSQL DB using Oracle Golden Gate

• Monitor your NoSQL cluster using Oracle Enterprise Manager

10

Enterprise ready -- Integrated out of the box




Forrester Wave TM: NoSQL Key-Value Databases, Q3 2014

Oracle Confidential – Internal/Restricted/Highly Restricted 11

• “Oracle’s NoSQL database, based on BerkelyDB, has strong adoption and maturity.”

• “…Oracle continues to do well in the NoSQL category with a comprehensive solution to support most enterprise needs. “

• “Oracle NoSQL key-value database is a mature, high-performing, scale-out storage engine and provides transactional semantics, fine-grained concurrency, primary and secondary indexes, and high availability features. “

Comments from Forrester:

Source: Forrester


Agenda

NoSQL Background



Use Cases

13




Architecture (logical) Linear scaling and replication

14

• Elastic Auto Sharding

(split, add, contract)

Store

Shard

M

Shard

R

R

R R

Application

NoSQL Driver

M

• Writes to elected

node with flexible

durability

• Reads from any

node in system

Shard

R

R

M

Expand and Rebalance

Shard

M

R

R • Auto re-balance of

data on expansion




Architecture (physical) Linear scaling and replicated

15

M1

M2

M3

R2

R2 R1

R1 R3

R3

Shard1 Shard2 Shard3

Agents

A

A

A

Machine1

Machine2

Machine3

Application

NoSQL Driver

D

D D

D

D D

– Elastic Auto Sharding

– Elected Master

– Read from any node

– Data rebalance on expand


Data Center Support

• Availability Zones

• Flexible configuration

• Primary Zones

– Durability guarantees

– Low latency writes, HA

• 2nd’ary Read-Only Zones

– Asynchronous replication

– Analytic workloads

– Report generation

• Topology Aware Client Driver

• Provides business continuity and distributed workload management

17

DC1 DC2 DC3

PrimaryZones

Reports

Batch Analytics




Configurable Durability and Consistency

• Greater Flexibility

– Configurable • Durability per operation

• Consistency per operation

– Transaction scope is single API call • Records must share same shard key

• Multiple operations supported




Large Object Support

• Parallel Streaming Interfaces

– Efficient storage and retrieval of large objects

– Client side streaming interface for low memory consumption

– Server side splitting and distribution of object chunks across nodes for better read/write latency

– Automatic partial LOB detection

19

Large Object

No

SQL

DB

Dri

ver

Ap

plic

atio

n

Shard 2

Shard N

Shard 1



Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 20

Flexible Data Model

Oracle NoSQL Database

advanced key-value database

1. KV API, Application specific opaque values

Primary key & Shard key Automatic Data sharding & local indexing ACID Transactions

1. KV API, Application specific opaque values 2. JSON API, JSON Structures 1. Application specific opaque values 2. JSON API, JSON Structures 3. Table API, Tables




Security

• Data Access Protection

– Configurable enforcement

– Authentication • User/Password

• Configurable client time-outs

• Oracle Wallet integration

• Internal components self-authenticate

– Encryption over the wire • All channels SSL encrypted

– Authorization • Read/Write Access

• Administration

– Kerberos Integration

22

Store

Shard

M

Shard

R

R

Shard

R

R R

NoSQL Driver

M R

M

Username Password

SSL

SSL SSL

Application

Read/Write or Admin




Online Rolling Upgrades

• Linear time impact for upgrade – Admin commands available to describe

safe upgrade order – Scripted available hands-free upgrade

experience – Read/Write availability throughout the

upgrade process

200 Node version upgrade in 15 minutes

23

0

2.5

5

7.5

10

12.5

15

17.5

72 (24x3) 144 (48x3) 216 (72x3)

Tim

e t

o U

pgr

ade

(m

in)

Total Nodes (Shards x Rep. Factor)

Online Rolling Upgrade




Developer Centric APIs • Java API for Key-Value, JSON and Table data models

• R, JRuby, Jython community drivers

• C API for Key-Value & JSON

• C API for Table data model

– Implementing using a new Thrift, JNI-less protocol

– Additional Thrift-based APIs to follow (JavaScript, Python)

• REST API available through Oracle Rest Data Services

– Deployments using Oracle Web Logic Server, Glassfish, Tomcat

– Directs Web Service calls to NoSQL DB

• Client-side DDL API

– Create and Alter tables and schemas inside an application

• Node.js and Python clients




Integration with Apache Hive and Oracle Big Data SQL

• Oracle Big Data SQL

– Common SQL access layer for data stored in Hadoop HDFS, Hive and Oracle NoSQL Database

– Defines new external table access driver types. ORACLE_HIVE for accessing data stored in or referenced by Hive Tables

– Query Oracle NoSQL Data from Hive or Oracle Database

– Runs Map Reduce Jobs against data stored in Oracle NoSQL Database

• Support for primitive and complex data types

• Common data definition between Oracle NoSQL DB, Hive and Oracle Database

25

BIG DATA SQL


NoSQL As Mobile/Embedded Data Repository

• Embedded data store (BDB)

• Automatic rule-based sync

• Device Management

Devices

• Secure data transport and conversion

• Mobile Device Management

Database Mobile Server • Scalable data repository

• Fast data analytics

NoSQL

26

Complete Mobile/Embedded to NoSQL Integration


Developer and Admin Tools

• Standards based tooling

– SNMP / JMX metrics

– Oracle Enterprise Manager

– Cloud ready HTML5 browser admin

– Command line interface • Cluster Configuration & Status

• DDL & DML

• Scripting

• Data load

• Aggregates




Predictability, Reliability & Support

• Global, mission-critical application deployment experience

• Decades of enterprise-grade non-relational database technology

• Oracle Support available for both Enterprise and Community Edition

• Designed for Predictability and Manageability

Bulk Insert Test Cluster Expansion Test Rolling Upgrade Test

29


YCSB on SSD-backed commodity servers

• What’s the big deal

– Twitter sees ~500M tweets/day • This is 350K a minute

• Capture all tweets with 3 commodity servers

– 1.25M ops/sec

– 2 billion records

– 2 TB of data

– 95% read, 5% update

– Low latency, High Scalability

30

0

1

2

3

4

0

200,000

400,000

600,000

800,000

1,000,000

1,200,000

1,400,000

6 (2x3) 12 (4x3) 24 (8x3) 30 (10x3)

Ave

rage

Lat

en

cy (

ms)

Thro

ugh

pu

t (o

ps/

sec)

Cluster Size ( 3->15 servers )

Mixed Throughput

Throughput (ops/sec) Write Latency (ms)

Read Latency (ms)




What’s New in Oracle NoSQL DB 2014 & 2015

Data Management • Table Data Model*

• Secondary Indices*

• Client-side, SQL-like DDL*

Application

Enablement • Maven Repository

• C API Binaries

APIs • C API for Tables & Indices

• REST API*

• Thrift Proxy

• Javascript (Node.js)*

• Python*

Server & Performance • Admin Repair

• Diagnostic enhancements

Security • Authentication*

• Authorization*

• User-Defined Roles*

• Table-Level Permissions*

• Wire-Level Encryption*

Administration • Secondary Data Centers

Integration • OEM*

• Oracle Big Data SQL*

• Oracle Database Mobile

Server

• Oracle Golden Gate

• Apache Spark

• Apache Hive

(*) Customer-driven requests

32


Hardware, License, Delivery

• Customer Installs

– Commodity servers (x86 or SPARC – including M7)

– Big Data Appliance (BDA)

• Two Types of License

– Community Edition (AGPLv3 license) – Support available

– Enterprise Edition (Commercial license) – includes integration and security over CE.

• Cloud: Monthly subscription pricing (coming soon)

33


Agenda

NoSQL Background



Use Cases

35


Finance

• Large credit card network provider

• Fraud rule and history management

• 6,500 ops/sec, 85% update on read

Manufacturing (Airbus)

• Large airline manufacturer

• Developing next generation aircraft

• 750K key inserts/sec, 6TB/hour

36

Telecoms

• Large mobile telecom operator

• Mobile personalized advertizing

• 24,000 ops/sec, 100% update on read

Customer Loyalty

• Operationalize analytics to NoSQL Customer Portal

• Addressed processing conflicts

• 20M+ customers in multiple countries

Oracle NoSQL Database customer successes…

……many more


Web-Scale Personalization

(Customer Portals & Digital Commerce)

• Direct-to-Consumer web services

• Real-time transaction capture

• High volume, Low latency

• Retail, Telco, Financial Svcs

• Oracle CGBU BRM uses NoSQL Database

Time Series & Sensor Data Mgmt (aka Internet of Things)

• Efficient, scalable data persistence

• Requires analytics and visualization

• Scalability, Cost management

• Utilities, Oil & Gas, Financial Svcs

• Oracle’s IoT cloud service uses NoSQL Database

37

Use case growth in the last 12 months

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 46

High Level NoSQL DB Roadmap

• Oracle NoSQL Database Cloud Services Cloud

• Oracle Stack

• Open Source Stack Integration

• SQL for NoSQL

• Full Text Search

• Additional Security Product


Join NoSQL Database Community

Twitter https://twitter.com/#!/OracleNoSQL

LinkedIn http://www.linkedin.com/groups?gid=4147754

Oracle’s NoSQL DB blog https://blogs.oracle.com/nosql

Oracle Technology Network

http://bit.ly/1f0d8wU

Developer Webcast Series

http://bit.ly/1doV2jl

Oracle.com/BigData

47


Q&A

48

introduction to oracle nosql database - aioug nosql db over… · title: how to use the powerpoint...

Documents