second step to the nosql side: mysql json functions

Post on 05-Dec-2014

262 Views

Category:

Software

3 Downloads

Preview:

Click to see full reader

DESCRIPTION

Slides from my session "Second Step to the NoSQL Side: MySQL JSON Functions" at Oracle OpenWorld 2014

TRANSCRIPT

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

Second Step to the NoSQL Side: MySQL JSON Functions

Sveta SmirnovaSenior Principal Technical Support EngineerMySQL SupportSeptember 29, 2014

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

Safe Harbor Statement

The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

Program Agenda

Overview

Function descriptions

Where to get

How to install

References

1

2

3

4

5

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

Overview

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

JSON functions in MySQL

• 2012: version 0.1

– Presented at MySQL Connect

– Internally

• 2013: version 0.2–First public release 0.2.0

–0.2.1

• 2014: version 0.3–First Community contribution

–0.3.0, 0.2.2, 0.3.1

–Current version 0.3.22012 2013 2014

0

0.05

0.1

0.15

0.2

0.25

0.3

0.1Never published

0.2

0.3

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

JSON functions overview

• Functions–Manipulate JSON text

• Validate

• Search

•Modify

• UDF functions

–Easy to install

– Independent from MySQL Server version

• Work on all MySQL supported platforms

• Binaries for Linux, Mac OS X 10.9 and Windows

What are they doing?

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

JSON functions improvements after MySQL Connect 2013

• 26 bugs fixed

• 13 features implemented

• 1 Community contribution

• Simplified build

• Automatic package builder

• Error messages in the error log file

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

Function descriptions

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

json_valid(doc)

• Checks if doc is valid JSON document.

• Returns 1 if document is valid, 0 if document is invalid.

• Strict format as described at ● http://json.org

● http://www.ietf.org/rfc/rfc4627.txt?number=4627

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

json_valid(doc)mysql> select json_valid('{"MySQL Central": ["conference", 2014]}'),

json_valid('["conference", 2014]'),

json_valid('"conference"'),

json_valid('{"MySQL Central"}')\G

*************************** 1. row ***************************

json_valid('{"MySQL Central": ["conference", 2014]}'): 1

json_valid('["conference", 2014]'): 1

json_valid('"conference"'): 1

json_valid('{"MySQL Central"}'): 0

1 row in set (0.00 sec)

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

Functions, accessing elements by a key

• json_contains_key

• json_extract

• json_append

• json_replace

• json_set

• json_remove

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

Functions, accessing elements by a key

• json_contains_key

• json_extract

• json_append

• json_replace

• json_set

• json_remove

Checks if the document contains key specified

Extracts the element by keyAppends the elementReplaces the element

Perform a kind of INSERT ON DUPLICATE KEY UPDATE operation

Removes the element

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

json_contains_key(doc, keypart1, keypart2, ...)SET optimizer_trace=1;

mysql> select user from mysql.user;

...

mysql> select json_contains_key(trace, 'steps', '1', 'join_optimization', 'steps', '0', 'condition_processing') as contains from information_schema.optimizer_trace;

+----------+

| contains |

+----------+

| 0 |

+----------+

1 row in set (0.01 sec)

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

json_contains_key(doc, keypart1, keypart2, ...)

mysql> select user from mysql.user where user='Sveta';

mysql> select json_contains_key(trace, 'steps', '1', 'join_optimization', 'steps', '0', 'condition_processing') as contains from information_schema.optimizer_trace;

+----------+

| contains |

+----------+

| 1 |

+----------+

1 row in set (0.01 sec)

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

json_extract(doc, keypart1, keypart2, ...)SET optimizer_trace=1;

mysql> select user from mysql.user;

...

mysql> select json_extract(trace, 'steps', '1', 'join_optimization', 'steps', '0', 'condition_processing') as contains from information_schema.optimizer_trace;

+----------+

| contains |

+----------+

| NULL |

+----------+

1 row in set (0.03 sec)

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

Search path{“steps”:

[

...

{“join_optimization”:

{“steps”:

[

{“condition_processing”: .....

json_extract(trace, 'steps', '1', 'join_optimization', 'steps', '0', 'condition_processing')

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

json_extract(doc, keypart1, keypart2, ...)mysql> select user from mysql.user where user='Sveta';

mysql> select json_extract(trace, 'steps', '1', 'join_optimization', 'steps', '0', 'condition_processing') as contains from information_schema.optimizer_trace\G

*************************** 1. row ***************************

contains: {

"condition": "WHERE",

"original_condition": "(`mysql`.`user`.`User` = 'Sveta')",

"steps": [

...

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

json_append(doc, keypart1, keypart2, ...,new_element)mysql> select json_append('{"MySQL Central": ["conference", 2014]}', 'MySQL Central', 2, '"San Francisco"') as el2,

-> json_append('{"MySQL Central": ["conference", 2014]}', 'MySQL Central', -1, '"San Francisco"') as `el-1`,

-> json_append('{"MySQL Central": ["conference", 2014]}', 'MySQL Central', 1, '"San Francisco"') as el1\G

*************************** 1. row ***************************

el2: {"MySQL Central": ["conference", 2014, "San Francisco"]}

el-1: {"MySQL Central": ["conference", 2014, "San Francisco"]}

el1: {"MySQL Central": ["conference", 2014]}

1 row in set (0.01 sec)

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

json_replace(doc, keypart1, keypart2, ...,new_value)mysql> select json_replace('{"MySQL Central": ["conference", 2014]}', 'MySQL Central', 0, '"User conference"') as el0,

-> json_replace('{"MySQL Central": ["conference", 2014]}', 'MySQL Central', 2, '"User conference"') as el2,

-> json_replace('{"MySQL Central": ["conference", 2014]}', 'MySQL Central', -1, '"User conference"') as `el-1`\G *************************** 1. row ***************************

el0: {"MySQL Central": ["User conference", 2014]}

el2: {"MySQL Central": ["conference", 2014]}

el-1: {"MySQL Central": ["conference", 2014]}

1 row in set (0.00 sec)

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

json_set(doc, keypart1, keypart2, ..., new_value)mysql> select json_set('{"MySQL Central": ["conference", 2014]}', 'MySQL Central', 0, '"User conference"') as el0,

-> json_set('{"MySQL Central": ["conference", 2014]}', 'MySQL Central', 2, '"San Francisco"') as el2,

-> json_set('{"MySQL Central": ["conference", 2014]}', 'MySQL Central', -1, '"San Francisco"') as `el-1`\G

*************************** 1. row ***************************

el0: {"MySQL Central": ["User conference", 2014]}

el2: {"MySQL Central": ["conference", 2014, "San Francisco"]}

el-1: {"MySQL Central": ["conference", 2014, "San Francisco"]}

1 row in set (0.00 sec)

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

json_remove(doc, keypart1, keypart2, ...)mysql> select json_remove('{"MySQL Central": ["conference", 2014]}', 'MySQL Central', 1) as el1,

-> json_remove('{"MySQL Central": ["conference", 2014]}', 'MySQL Central', 2) as el2,

-> json_remove('{"MySQL Central": ["conference", 2014]}', 'MySQL Central', -1) as `el-1`\G

*************************** 1. row ***************************

el1: {"MySQL Central": ["conference"]}

el2: {"MySQL Central": ["conference", 2014]}

el-1: {"MySQL Central": ["conference", 2014]}

1 row in set (0.00 sec)

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

json_search(doc, value)

• Searches for specified value in the document

• Wildcards supported since version 0.3.2

• Returns key path of the element which contains the value in reverse order or NULL if not found or parsing failed

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

json_search(doc, value)mysql> select json_search(trace, '"trivial_condition_removal"') as `full`,

-> json_search(trace, '"trivial_condition"') as `partial`,

-> json_search(trace, '"trivial_condition%"') as `wildcard` from information_schema.optimizer_trace\G *************************** 1. row ***************************

full: transformation:0:steps:condit...essing:0:steps:join_optimization:0:steps::

partial: NULL

wildcard: transformation:0:steps:condit...essing:0:steps:join_optimization:0:steps::

1 row in set (0.01 sec)

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

Functions, merging documents

• json_merge

• json_safe_merge

• json_deep_merge

Merge two or more documents into one

Return first document with followings appended

Does not handle duplicate keys, does not check for validity, open and

closing brackets must match

+ Checks for validity

+ Duplicate keys are updated

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

json_[safe_|deep_]merge(doc1, doc2, ...)mysql> select json_merge('{"OOW": ["conference", 2013]}', \ '{"OOW": ["conference" 2014]}') as 'json_merge',

-> json_safe_merge('{"OOW": ["conference", 2013]}', \ '{"OOW": ["conference" 2014]}') as 'json_safe_merge',

-> json_safe_merge('{"OOW": ["conference", 2013]}', \ '{"OOW": ["conference", 2014]}') as 'json_safe_merge',

-> json_deep_merge('{"OOW": ["conference", 2013]}', \ '{"OOW": ["conference", 2014]}') as 'json_deep_merge'\G*************************** 1. row ***************************

json_merge: {"OOW": ["conference", 2013], "OOW": ["conference" 2014]}

json_safe_merge: {"OOW": ["conference", 2013]}

json_safe_merge: {"OOW": ["conference", 2013], "OOW": ["conference", 2014]}

json_deep_merge: {"OOW": ["conference", 2013, "conference", 2014]}

1 row in set (0.01 sec)

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

json_depth(doc)

• Returns depth of the document

• mysql> select json_depth('{"MySQL Central": ["conference", 2014]}') as 'json_depth';

+------------+

| json_depth |

+------------+

| 3 |

+------------+

1 row in set (0.00 sec)

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

json_count(doc[, keypart1[, keypart2[, ...]]])

• Returns number of childs of the key specified

• mysql> select json_count('{"MySQL Central": ["conference", 2014]}') as 'root count',

-> json_count('{"MySQL Central": ["conference", 2014]}', 'MySQL Central') as 'first element count'\G ************************ 1. row ************************

root count: 1

first element count: 2

1 row in set (0.02 sec)

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

json_version()

• Returns version number of the functions

• mysql> select json_version();+----------------------------+

| json_version() |

+----------------------------+

| MySQL JSON UDFs 0.3.2-labs |

+----------------------------+

1 row in set (0.00 sec)

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

json_test_parser(doc)

• Returns text representation of parse tree of the JSON document, partial parse tree or empty string if document is invalid.

• This function is supposed to use for tests only and should not be used in production.

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

json_test_parser(doc)mysql> select json_test_parser('{"MySQL Central": ["conference", 2014]}') as 'parse tree'\G

*************************** 1. row ***************************

parse tree: => "conference";

=> 2014;

"MySQL Central" => ["conference", 2014];

=> {"MySQL Central": ["conference", 2014]};

1 row in set (0.00 sec)

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

json_test_parser(doc)mysql> select json_test_parser('{"MySQL Central": ["conference", 2014]') as 'parse tree'\G

*************************** 1. row ***************************

parse tree: => "conference";

=> 2014;

"MySQL Central" => ["conference", 2014];

1 row in set (0.00 sec)

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

Where to get?

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

Source Code and Binaries

• MySQL Labs

– Source code

– Binaries• x86 and x86_64

• Generic Linux

• Mac OSX 10.9

• Windows 7

• http://labs.mysql.com/

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

More Information

• Manuals and articles

– README file

– Author's blog: https://blogs.oracle.com/svetasmirnova/tags/json

• Announces–Author's twitter: https://twitter.com/svetsmirnova

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

How to install?

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

How to install?UNIX

create function json_valid returns integer

soname 'libmy_json_udf.so';

create function json_search returns string

soname 'libmy_json_udf.so';

create function json_extract returns string

soname 'libmy_json_udf.so';

create function json_replace returns string

soname 'libmy_json_udf.so';

...

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

How to install?Windows

create function json_remove returns string

soname 'my_json_udf.dll';

create function json_set returns string

soname 'my_json_udf.dll';

create function json_merge returns string

soname 'my_json_udf.dll';

create function json_contains_key returns integer

soname 'my_json_udf.dll';

...

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

How to install or remove?Ready-to use scripts

mysql db_name < install_jsonudf.sql

mysql db_name < uninstall_jsonudf.sql

• Thank you, Daniel van Eeden!

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

Dependencies

• PCRE library

• Bundled with the functions

• Statically compiled with binaries

• UNIX

– You can use system's• Windows

● Always compiles statically

• You should not install any additional libraries to use functions!

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

How to Compile?UNIX

• You need:

– Cmake

– PCRE library (bundled with sources)

– Compiler

• How to compile:

cmake . -DMYSQL_DIR=/home/sveta/build/mysql-5.6

make

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

How to Compile?Windows

• You need:

– Cmake

– PCRE library (bundled with sources)

– Visual Studio

• How to compile:

"C:\Program Files (x86)\CMake 2.8\bin\cmake.exe" -G "Visual Studio 11 Win64" . -DMYSQL_DIR="C:/MySQL/mysql-5.6.21"

devenv my_json_udf.sln /build Release

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

References

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

Feature Requests and Bug Reports

• MySQL Community Bugs Database

– https://bugs.mysql.com

– Category• “MySQL Server: JSON User-defined function (UDF)”

• Oracle's Bugs Database for engineers and paying customers– Ask MySQL Support engineers to open a bug report for you

– Category

• “UDFJSON”

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

More information

• https://blogs.oracle.com/svetasmirnova/

• https://twitter.com/#!/svetsmirnova

• http://json.org/

• http://www.pcre.org/

• http://dev.mysql.com/doc/refman/5.6/en/adding-functions.html

• http://bugs.mysql.com/

• https://support.oracle.com

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

?

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

Thank you!

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

top related