postgres nosql - delivering apps faster

Postgres NoSQL - Delivering Applications Faster Marc Linster – SVP, Products and Services

•  EDB Introduction

•  BigData and NoSQL: Volume, Variety and Velocity

•  Postgres – NoSQL for the Enterprise •  Performance Evaluation – Phase 2

•  FDW – Coexistence of NoSQL only and SQL

•  Leveraging a proven skill set to create Web 2.0 applications

•  Postgres as a NoSQL/SQL Integration Platform

Agenda

POSTGRES innovation

ENTERPRISE reliability

24/7 support

Services & training

Enterprise-class features & tools

Indemnification

Product road-map

Control

Thousands of developers

Fast development

cycles

Low cost

No vendor lock-in

Advanced features

Enabling commercial adoption of Postgres

Postgres Plus Advanced Server Postgres Plus

Cloud Database

High Availability Performance Management

REMOTE DBA 24x7

SUPPORT PROFESSIONAL

SERVICES

TRAINING

EDB Serves All Your Postgres Needs

PostgreSQL

Security

•  Common Myth (or misconception) − Volume, Velocity, Variety and ACID are not compatible! − You cannot have it all!

•  Their (incorrect) reasoning: − Very High Data Volumes (log analysis, append only applications,

social media) applications cannot be handled by relational systems − The data is sent to the application at high speeds and has to be

ingested quickly – not in batch mode − Web 2.0 applications require more than SQL – they also need

documents, IMs, emails, graphs, links − ACID (Atomicity, Consistency, Isolation and Durability) compliant

systems are too inflexible, too slow and cannot handle the new data types

BigData, NoSQL and SQL: Myths about Volume, Variety, Velocity and ACID

$$$$$$

ACID, Volume, Velocity and Variety in Context ACID Transactional

Ingestion Velocity

Variety

Data Volume

Postgres

Data Mining/BigData

Document/NoSQL-Only

•  Performance Benchmarks −  Postgres is very fast and can handle huge

amounts of data −  Postgres can selectively relax key ACID features

to increase performance

•  Data Types −  Postgres has JSON, JSONB, Key-Value Pair,

plus arrays, ranges, timezones, dates, integer, floating point, etc.

•  Proven track record −  ACID compliant −  Open source −  ANSI SQL −  Huge developer and vendor community

Postgres – NoSQL for the Enterprise

•  HSTORE −  Key-value pair −  Simple, fast and easy −  Postgres v 8.2 – pre-dates many

NoSQL-only solutions

•  JSON −  Hierarchical document model −  Introduced in Postgres 9.2, perfected in 9.3

•  JSONB −  Binary version of JSON −  Faster, more operators and even more robust −  Postgres 9.4

NoSQL Data in Postgres

•  Creating a table with a JSONB field CREATE TABLE json_data (data JSONB);!

•  Simple JSON data element: {"name": "Apple Phone", "type": "phone", "brand": "ACME", "price": 200, "available": true, "warranty_years": 1}!

•  Inserting this data element into the table json_data INSERT INTO json_data (data) VALUES !

!(’ { !"name": "Apple Phone", !! !"type": "phone", !! !"brand": "ACME", !! !"price": 200, !! !"available": true, !! !"warranty_years": 1 ! !!!} ')!

JSON Examples

SELECT DISTINCT !!data->>'name' as products !

FROM json_data; ! products !------------------------------! Cable TV Basic Service Package! AC3 Case Black! Phone Service Basic Plan! AC3 Phone! AC3 Case Green! Phone Service Family Plan! AC3 Case Red! AC7 Phone!

A simple query for JSON data

This query does not return JSON data – it returns text values associated with the key ‘name’

SELECT data FROM json_data;!

data !

------------------------------------------!

{"name": "Apple Phone", "type": "phone", "brand": "ACME", "price": 200, "available": true, "warranty_years": 1}!

A query that returns JSON data

This query returns the JSON data in its original format

•  JSON is naturally integrated with ANSI SQL in Postgres

•  JSON and HSTORE are elegant and easy to use extensions of the underlying object-relational model

•  JSON and SQL queries use the same language, the same planner, and the same ACID compliant transaction framework

JSON and ANSI SQL – A Great Fit

SELECT DISTINCT product_type, data->>'brand' as Brand,

data->>'available' as Availability FROM json_data JOIN products ON (products.product_type=json_data.data->>'name') WHERE json_data.data->>'available'=true; product_type | brand | availability ---------------------------+-----------+-------------- AC3 Phone | ACME | true

JSON and ANSI SQL Example

ANSI SQL

No need for programmatic logic to combine SQL and NoSQL in the application – Postgres does it all

Bridging between SQL and NoSQL Simple ANSI SQL Table Definition

CREATE TABLE products (id integer, product_name text );! Select query returning standard data set

SELECT * FROM products; ! id | product_name !----+--------------! 1 | iPhone! 2 | Samsung! 3 | Nokia!

Select query returning the same result as a JSON data set

SELECT ROW_TO_JSON(products) FROM products; {"id":1,"product_name":"iPhone"} {"id":2,"product_name":"Samsung"} {"id":3,"product_name":"Nokia”}

•  Start unstructured, and become structured as you learn more − Use the quick-to-get-started capabilities of NoSQL − Complete the initial sprints without a DBA − Move data between unstructured and structured −  Embrace corporate data standards as you move

from the stand-alone application towards integrated applications with a bigger value proposition

Postgres Provides Great Flexibility

By 2017, 50% of data stored in NoSQL DBMSs will be damaging to the business due to lack of applied information governance policies and programs. Gartner, December 2013

•  Goal −  Help our customers understand when to chose Postgres and

when to chose a specialty solution −  Help us understand where the NoSQL limits of Postgres are

•  Setup −  Compare Postgres 9.4 to Mongo 2.6 −  Single instance setup on AWS M3.2XLARGE (32GB)

•  Test Focus −  Data ingestion (bulk and individual) −  Data retrieval

Postgres NoSQL Performance Evaluation

Load Generator

Postgres 9.4

MongoDB 2.6

Postgres NoSQL Performance Evaluation Generate JSON Documents

10 M documents & 50 M documents

Load into MongoDB 2.6 (IMPORT)

Load into Postgres 9.4

(COPY)

10 Million individual INSERT commands

Multiple SELECT statements

Test 1: 10 Million Documents

Test Criterion MongoDB 2.6 Postgres 9.4

Data load (s) 2,624 1,193

Inserts (s) 16,491 5,637

Selects (s) 187 67

DB Size (GB) 20.05 14.89

Data load (s) Inserts (s) Selects (s) DB Size (GB)

Mongo DB 2.4/Postgres 9.4 Rela9ve Performance Comparison (10M Documents)

MongoDB 2.6

Postgres 9.4

Test 2: 50 Million Documents

Test Criterion MongoDB 2.6 Postgres 9.4

Data load (s) 15,391 7,319

Inserts (s) 85,639 29,125

Selects (s) 1,929 753

DB Size (GB) 92.63 69.36

Data load (s) Inserts (s) Selects (s) DB Size (GB)

Mongo DB 2.4/Postgres 9.4 Rela9ve Performance Comparison (50M Documents)

MongoDB 2.6

Postgres 9.4

•  Tests confirm that Postgres can handle many NoSQL workloads

•  EDB is making the test scripts publically available

•  EDB encourages community participation to better define where Postgres should be used and where specialty solutions are appropriate

•  Download the source at https://github.com/EnterpriseDB/pg_nosql_benchmark

•  Join us to discuss the findings at http://bit.ly/EDB-NoSQL-Postgres-Benchmark

Postgres NoSQL Performance Evaluation

•  FDW implements SQL/MED ("SQL Management of External Data")

•  PostgreSQL 9.1 - read-only support

•  PostgreSQL 9.3 – read/write support •  FDW

−  Makes data on other servers (or services) look like tables in Postgres

−  available for databases (MongoDB, MySQL, Oracle, …), files, services (Twitter, …)

•  MongoDB FDW: https://github.com/EnterpriseDB

Foreign Data Wrappers – Co-Existence Platform

CREATE EXTENSION mongo_fdw;!

CREATE SERVER mongo_server !FOREIGN DATA WRAPPER mongo_fdw !OPTIONS (address '172.24.39.129', port '27017');!

CREATE USER MAPPING FOR enterprisedb !SERVER mongo_server !OPTIONS (username 'mongo', password 'mongo');!

CREATE FOREIGN TABLE mongo_data( !name text, !brand text, !type text) !SERVER mongo_server !OPTIONS ( ! !database 'benchmark', ! !collection 'json_tables');!

MongoDB FDW Example

SELECT * FROM mongo_data WHERE brand='ACME' limit 10;!

(10 rows)!

MongoDB FDW Example

INSERT INTO mongo_data(name, brand, type) !VALUES('iphone6 phone','Apple Inc','phone');!

SELECT* FROM mongo_data WHERE brand='Apple Inc';!

UPDATE mongo_data SET brand='Apple Product' !WHERE brand='Apple Inc’;!

SELECT * FROM mongo_data WHERE brand='Apple Product’;!

MongoDB FDW Example

•  Postgres allows you to −  Use the same Infrastructure DBAs (no change!!!)

−  Use the same management, maintenance, backup, restore, high availability, disaster recovery and compliance processes for SQL and NoSQL

−  Start unstructured and align with corporate governance standards as your project matures

−  Start unstructured (data logic in the application) and migrate shared data logic into the database è enhances reuse

−  Build on the existing PL/pgSQL programming skillset

−  EDB Postgres Plus adds Oracle PL/SQL compatibility (fully integrated with JSON and HSTORE)

Leveraging a Proven Skill Set

•  Postgres has many NoSQL features without the drawbacks: −  Schema-less data types, with sophisticated indexing

support −  Schema changes with rapid addition and removal of

keys and tags − Durable by default, but configurable per-table or

per-transaction −  Foreign Data Wrappers (FDW) support co-existence

and tight integration −  Standards based with very low technology risk − Highly available skill set

Postgres: The Best of Both Worlds

•  Structures and standards emerge! •  Data has references (products link to catalogues;

products have bills of material; components appear in multiple products; storage locations link to ISO country tables)

•  When the database has duplicate data entries, then the application has to manage updates in multiple places – what happens when there is no ACID transactional model?

“No SQL Only” or “Not Only SQL”?

•  Postgres is Not Only SQL (NoSQL is No SQL only) •  Fully ACID compliant •  Proven track record

•  Fully capable of handling the variety, velocity and volume requirements of most applications

•  Tackle NoSQL projects without leaving the capabilities of the relational model behind you

Postgres is the best of both worlds, and it’s open source!

Say ‘Yes’ to ‘Not Only SQL’

•  Contact us for an Enterprise NoSQL Assessment (sales@enterprisedb.com)

•  Review EDB’s NoSQL Resources −  Whitepaper -

http://www.enterprisedb.com/wp-using-nosql-features-postgres −  NoSQL Solution Information -

http://www.enterprisedb.com/nosql-for-enterprise −  Benchmarks- https://github.com/EnterpriseDB

postgres nosql - delivering apps faster

data data jsonb

data data values

postgres json

nosql data

data insert

simple json data element

table json

enterprisedb corporation

Software

postgres enterprise manager · postgres enterprise manager,...

nosql on acid - meet unstructured postgres

using the nosql capabilities in postgres · using the nosql...

nosql postgres - pg day · nosql postgres briefly • 2003...

postgres enterprise manager installation guide · postgres...

postgres open, september 2014: postgres interface...

nosql postgres - pg daynosql postgres briefly • 2003 —...

visualizing postgres - 日本postgresqlユーザ会 ·...

nosql...

postgres presentation

adventures in migrations nosql to postgresql · migrating...

postgres user

session 1 nosql vs sql: history and evolution...relational...

using vmware vfabric postgres - vfabric postgres 9.3 · pdf...

postgres plus advanced server installation...

manual postgres

fujitsu enterprise postgres - enhanced open source...

who am i?assets.astrails.com/.../wtf-is-mysql.pdf ·...

using vmware vfabric postgres - vfabric postgres 9.1

configuring edb postgres advanced server streaming...