contents · pdf filecontents 1. dql, release 0.1.0-0 ... dql supports the use of python...

35
dql Release 0.1.0-0-gadaccee-dirty February 05, 2014

Upload: dinhtuyen

Post on 06-Mar-2018

222 views

Category:

Documents


6 download

TRANSCRIPT

Page 1: Contents · PDF fileContents 1. dql, Release 0.1.0-0 ... DQL supports the use of python expressions anywhere that you would ... For this reason, all of your table queries must

dqlRelease 0.1.0-0-gadaccee-dirty

February 05, 2014

Page 2: Contents · PDF fileContents 1. dql, Release 0.1.0-0 ... DQL supports the use of python expressions anywhere that you would ... For this reason, all of your table queries must
Page 3: Contents · PDF fileContents 1. dql, Release 0.1.0-0 ... DQL supports the use of python expressions anywhere that you would ... For this reason, all of your table queries must

Contents

i

Page 4: Contents · PDF fileContents 1. dql, Release 0.1.0-0 ... DQL supports the use of python expressions anywhere that you would ... For this reason, all of your table queries must

ii

Page 5: Contents · PDF fileContents 1. dql, Release 0.1.0-0 ... DQL supports the use of python expressions anywhere that you would ... For this reason, all of your table queries must

dql, Release 0.1.0-0-gadaccee-dirty

A simple, SQL-ish language for DynamoDB

Contents 1

Page 6: Contents · PDF fileContents 1. dql, Release 0.1.0-0 ... DQL supports the use of python expressions anywhere that you would ... For this reason, all of your table queries must

dql, Release 0.1.0-0-gadaccee-dirty

2 Contents

Page 7: Contents · PDF fileContents 1. dql, Release 0.1.0-0 ... DQL supports the use of python expressions anywhere that you would ... For this reason, all of your table queries must

CHAPTER 1

User Guide

1.1 Getting Started

Install DQL with pip:

pip install dql

Since DQL uses boto under the hood, the authentication mechanism is the same. You may either set theAWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables, or pass them in on the commandline:

$ dql -a <access key> -s <secret key>

DQL uses us-west-1 as the default region. You can change this by setting the AWS_REGION variable or passing itin on the command line:

$ dql -r us-east-1

You can begin using DQL immediately. Try creating a table and inserting some data

us-west-1> CREATE TABLE posts (username STRING HASH KEY,> postid NUMBER RANGE KEY,> ts NUMBER INDEX(’ts-index’))> THROUGHPUT (5, 5);

us-west-1> INSERT INTO posts (username, postid, ts, text)> VALUES (’steve’, 1, 1386413481, ’Hey guys!’),> (’steve’, 2, 1386413516, ’Guys?’),> (’drdice’, 3, 1386413575, ’Fun fact: dolphins are dicks’);

us-west-1> lsName Status Read Writeposts ACTIVE 5 5

You can query this data in a couple of ways. The first should look familiar

us-west-1> SELECT * FROM posts WHERE username = ’steve’;

This performs a table query using the hash key, so it should run relatively quickly. The second way of accessing datais by using a scan

us-west-1> SCAN posts FILTER username = ’steve’;

This returns the same data as the SELECT, but it retrieves it with a table scan instead, which is much slower.

You can also perform updates to the data in a familiar way

3

Page 8: Contents · PDF fileContents 1. dql, Release 0.1.0-0 ... DQL supports the use of python expressions anywhere that you would ... For this reason, all of your table queries must

dql, Release 0.1.0-0-gadaccee-dirty

us-west-1> UPDATE posts SET text = ’Hay gusys!!11’ WHERE> username = ’steve’ AND postid = 1;

The Queries section has more details. Check it out for more information on the performance implications and queryoptions.

1.2 Advanced Usage

This section is devoted to some of the advanced uses of DQL

1.2.1 API

DQL execution is available as a python library. For example:

import dql

engine = dql.Engine()engine.connect_to_region(’us-west-1’)results = engine.execute("SCAN mytable LIMIT 10")for item in results:

print dict(item)

The return value will vary based on the type of query.

1.2.2 Embedded Python

DQL supports the use of python expressions anywhere that you would otherwise have to specify a data type. Justsurround the python with backticks. Create your variable scope as a dict and pass it to the engine with the commands:

>>> engine.scope = {’foo1’: 1, ’foo2’: 2}>>> engine.execute("INSERT INTO foobars (foo) VALUES (‘foo1‘), (‘foo2‘)")

The interactive client has a special way to modify the scope. You can switch into ‘code’ mode to execute python, andthen use that scope as the engine scope:

us-west-1> code>>> foo1 = 1>>> import time>>> endcodeus-west-1> INSERT INTO foobars (foo) VALUES (‘foo1‘), (‘time.time()‘)success

You can also spread the expressions across multiple lines by using the format m‘<expr>‘. If you do this, you willneed to return a value. The difference here is between using eval and exec in python.

us-west-1> UPDATE foobars SET foo = m‘> if bar % 2 == 0:> return bar**2> else:> return bar - 1> ‘;

4 Chapter 1. User Guide

Page 9: Contents · PDF fileContents 1. dql, Release 0.1.0-0 ... DQL supports the use of python expressions anywhere that you would ... For this reason, all of your table queries must

dql, Release 0.1.0-0-gadaccee-dirty

1.3 Queries

1.3.1 ALTER

Synopsis

ALTER TABLE tablenameSET[INDEX index]THROUGHPUT throughput

Examples

ALTER TABLE foobars SET THROUGHPUT (4, 8);ALTER TABLE foobars SET THROUGHPUT (7, *);ALTER TABLE foobars SET INDEX ts-index THROUGHPUT (5, *);

Description

Alter changes the read/write throughput on a table. Dynamo stipulates that you may only increase the throughput bya factor of 2 at a time. If you attempt to increase the throughput by more than that, DQL will have to make multiplecalls, waiting for the updates to go through in the interim. You may only decrease the throughput twice per day. Ifyou wish to change one of the throughput values and not the other, pass in 0 or * for the value you wish to remainunchanged.

Parameters

tablename The name of the table

throughput The read/write throughput in the form (read_throughput, write_throughput)

index The name of the global index

1.3.2 COUNT

Synopsis

COUNT[ CONSISTENT ]tablenameWHERE expression[ USING index ]

Examples

COUNT foobars WHERE foo = ’bar’;COUNT CONSISTENT foobars WHERE foo != ’bar’ AND baz >= 3;COUNT foobars WHERE (foo = ’bar’ AND baz >= 3) USING ’baz-index’;

1.3. Queries 5

Page 10: Contents · PDF fileContents 1. dql, Release 0.1.0-0 ... DQL supports the use of python expressions anywhere that you would ... For this reason, all of your table queries must

dql, Release 0.1.0-0-gadaccee-dirty

Description

This counts the number of matching items in your table. It is making a query, so you must search using the hash key,and you may optionally also provide the range key or an index.

The WHERE clause is mandatory. If you want a count of all elements in a table, look at the table description.

Parameters

CONSISTENT If this is present, use a read-consistent query

tablename The name of the table

index When the WHERE expression uses an indexed attribute, this allows you to manually specify which index nameto use for the query. It should generally not be needed, as the DQL engine will automatically detect the correctindex to use for a query.

Where Clause

Examples

WHERE hkey = ’a’ AND bar > 5 AND baz <= 16WHERE hkey = 1 AND bar BEGINS WITH "prefix"WHERE hkey = 1 AND bar BETWEEN (1, 100)

1.3.3 CREATE

Synopsis

CREATE TABLE[IF NOT EXISTS]tablenameattributes[GLOBAL [ALL|KEYS|INCLUDE] INDEX global_index]

Examples

CREATE TABLE foobars (id STRING HASH KEY);CREATE TABLE IF NOT EXISTS foobars (id STRING HASH KEY);CREATE TABLE foobars (id STRING HASH KEY, foo BINARY RANGE KEY,

THROUGHPUT (1, 1));CREATE TABLE foobars (id STRING HASH KEY,

foo BINARY RANGE KEY,ts NUMBER INDEX(’ts-index’),views NUMBER INDEX(’views-index’));

CREATE TABLE foobars (id STRING HASH KEY, bar STRING) GLOBAL INDEX(’bar-index’, bar, THROUGHPUT (1, 1));

CREATE TABLE foobars (id STRING HASH KEY, bar STRING, baz NUMBER,THROUGHPUT (2, 2))GLOBAL INDEX (’bar-index’, bar, baz)GLOBAL INDEX (’baz-index’, baz, THROUGHPUT (4, 2));

6 Chapter 1. User Guide

Page 11: Contents · PDF fileContents 1. dql, Release 0.1.0-0 ... DQL supports the use of python expressions anywhere that you would ... For this reason, all of your table queries must

dql, Release 0.1.0-0-gadaccee-dirty

Description

Create a new table. You must have exactly one hash key, zero or one range keys, and up to five indexes. You musthave a range key in order to have any indexes.

Parameters

IF NOT EXISTS If present, do not through an exception if the table already exists.

tablename The name of the table that you want to alter

attributes A list of attribute declarations of the format (name data type [key type]) The available data types areSTRING, NUMBER, and BINARY. You will not need to specify SET types, because these fields are only usedfor index creation and it is (presently) impossible to index a SET. The available key types are HASH KEY,RANGE KEY, and [ALL|KEYS|INCLUDE] INDEX(name). At the end of the attribute list you may specifythe THROUGHPUT, which is in the form of (read_throughput, write_throughput). If throughputis not specified it will default to (5, 5).

global_index A global index for the table. You may provide up to 5. The format is (name, hash key, [range key],[non-key attributes], [throughput]). The hash key and range key must be in the attributes declaration. non-keyattributes should only be provided if it is an INCLUDE index. If throughput is not specified it will default to(5, 5).

Schema Design at a Glance

When DynamoDB scales, it partitions based on the hash key. For this reason, all of your table queries must includethe hash key in the where clause (and optionally the range key or a local index). So keep that in mind as you designyour schema.

The keypair formed by the hash key and range key is referred to as the ‘primary key’. If there is no range key, it’s justthe hash key. The primary key is unique among items in the table. No two items may have the same hash key andrange key.

From a query standpoint, local indexes behave nearly the same as a range key. The main difference is that they do notneed to be unique.

Global indexes can be thought of as adding additional hash and range keys to the table. They allow you to querya table on a different hash key than the one defined on the table. Global indexes have throughput that is managedindependently of the table they are on. Global index keys do not have a uniqueness constraint (there may be multipleitems in the table that have the same hash and range key in a global index).

Read Amazon’s documentation for Create Table for more information.

1.3.4 DELETE

Synopsis

DELETE FROMtablenameWHERE expression[ USING index ]

1.3. Queries 7

Page 12: Contents · PDF fileContents 1. dql, Release 0.1.0-0 ... DQL supports the use of python expressions anywhere that you would ... For this reason, all of your table queries must

dql, Release 0.1.0-0-gadaccee-dirty

Examples

DELETE FROM foobars WHERE foo = ’bar’;DELETE FROM foobars WHERE foo != ’bar’ AND baz >= 3;DELETE FROM foobars WHERE KEYS IN (’hkey1’), (’hkey2’);DELETE FROM foobars WHERE KEYS IN (’hkey1’, ’rkey1’), (’hkey2’, ’rkey2’);DELETE FROM foobars WHERE (foo = ’bar’ AND baz >= 3) USING ’baz-index’;

Description

Parameters

tablename The name of the table

index When the WHERE expression uses an indexed attribute, this allows you to manually specify which index nameto use for the query. It should generally not be needed, as the DQL engine will automatically detect the correctindex to use for a query.

Where Clause

Examples

WHERE hkey = ’a’ AND bar > 5 AND baz <= 16WHERE hkey = 1 AND bar BEGINS WITH "prefix"WHERE hkey = 1 AND bar BETWEEN (1, 100)WHERE KEYS IN (’hkey1’, ’rkey1’), (’hkey2’, ’rkey2’)WHERE KEYS IN (’hkey’), (’hkey2’)

Notes

Using the KEYS IN form is much more efficient because DQL will not have to perform a query first to get the primarykeys.

1.3.5 DROP

Synopsis

DROP TABLE[ IF EXISTS ]tablename

Examples

DROP TABLE foobars;DROP TABLE IF EXISTS foobars;

8 Chapter 1. User Guide

Page 13: Contents · PDF fileContents 1. dql, Release 0.1.0-0 ... DQL supports the use of python expressions anywhere that you would ... For this reason, all of your table queries must

dql, Release 0.1.0-0-gadaccee-dirty

Description

Deletes a table and all its items.

Warning: This is not reversible! Use with extreme caution!

Parameters

IF EXISTS If present, do not through an exception if the table does not exist.

tablename The name of the table

1.3.6 DUMP

Synopsis

DUMP SCHEMA [ tablename [, ...] ]

Examples

DUMP SCHEMA;DUMP SCHEMA foobars, widgets;

Description

Print out the matching CREATE statements for your tables.

Parameters

tablename The name of the table(s) whose schema you want to dump. If no tablenames are present, it will dump alltable schemas.

1.3.7 INSERT

Synopsis

INSERT INTO tablenameattributesVALUES values

Examples

INSERT INTO foobars (id) VALUES (1);INSERT INTO foobars (id, bar) VALUES (1, ’hi’), (2, ’yo’);

1.3. Queries 9

Page 14: Contents · PDF fileContents 1. dql, Release 0.1.0-0 ... DQL supports the use of python expressions anywhere that you would ... For this reason, all of your table queries must

dql, Release 0.1.0-0-gadaccee-dirty

Description

Insert data into a table

Parameters

tablename The name of the table

attributes Comma-separated list of attribute names

values Comma-separated list of data to insert. The data is of the form (var [, var]...) and must contain the samenumber of items as the attributes parameter.

Data Types

Below are examples of how to represent the different dynamo data types in DQL

NUMBER: 123 STRING: ’abc’ or "abc" BINARY: b’abc’ or b"abc" NUMBER SET: (1, 2, 3)STRING SET: (’a’, ’b’, ’c’) BINARY SET: (b’a’, b’b’, b’c’)

1.3.8 SCAN

Synopsis

SCAN tablename[ FILTER expression ][ LIMIT limit ]

Examples

SCAN foobars;SCAN foobars FILTER id = ’a’ AND foo = 4;SCAN foobars FILTER id = ’a’ AND foo CONTAINS 4 LIMIT 100;

Description

Sequentially iterate over items in a table. This does not use the indexes, so it can be significantly slower than a query.

Parameters

tablename The name of the table

limit Maximum number of results to return

Filter Clause

Examples

10 Chapter 1. User Guide

Page 15: Contents · PDF fileContents 1. dql, Release 0.1.0-0 ... DQL supports the use of python expressions anywhere that you would ... For this reason, all of your table queries must

dql, Release 0.1.0-0-gadaccee-dirty

FILTER hkey = ’a’ AND bar > 5 AND baz != 16FILTER hkey = 1 AND bar BEGINS WITH "prefix"FILTER hkey = 1 AND bar BETWEEN (1, 100)FILTER hkey = 1 AND bar IS NULL AND baz IS NOT NULLFILTER hkey = 1 AND bar CONTAINS 5 AND baz NOT CONTAINS 5FILTER hkey = 1 AND bar IN (1, 3, 5, 7, 9)

Notes

See the AWS docs for more information on scan parameters.

1.3.9 SELECT

Synopsis

SELECT[ CONSISTENT ]attributesFROM tablenameWHERE expression[ USING index ][ LIMIT limit ][ ASC | DESC ]

Examples

SELECT * FROM foobars WHERE foo = ’bar’;SELECT CONSISTENT * foobars WHERE foo = ’bar’ AND baz >= 3;SELECT foo, bar FROM foobars WHERE id = ’a’ AND ts < 100 USING ’ts-index’;SELECT * FROM foobars WHERE foo = ’bar’ AND baz >= 3 LIMIT 50 DESC;

Description

Query a table for items.

Parameters

CONSISTENT If this is present, use a read-consistent query

attributes Comma-separated list of item attributes to fetch. * is a special case meaning ‘all attributes’.

tablename The name of the table

index When the WHERE expression uses an indexed attribute, this allows you to manually specify which index nameto use for the query. It should generally not be needed, as the DQL engine will automatically detect the correctindex to use for a query.

limit Maximum number of results to return

ASC | DESC Sort the results in ASCending (the default) or DESCending order.

1.3. Queries 11

Page 16: Contents · PDF fileContents 1. dql, Release 0.1.0-0 ... DQL supports the use of python expressions anywhere that you would ... For this reason, all of your table queries must

dql, Release 0.1.0-0-gadaccee-dirty

Where Clause

Examples

WHERE hkey = ’a’ AND bar > 5 AND baz <= 16WHERE hkey = 1 AND bar BEGINS WITH "prefix"WHERE hkey = 1 AND bar BETWEEN (1, 100)WHERE KEYS IN (’hkey1’, ’rkey1’), (’hkey2’, ’rkey2’)WHERE KEYS IN (’hkey’), (’hkey2’)

Notes

When using the KEYS IN form, DQL will perform a batch get instead of a table query. See the AWS docs for moreinformation on query parameters.

1.3.10 UPDATE

Synopsis

UPDATE tablenameSET values[ WHERE expression ][ RETURNS (NONE | ( ALL | UPDATED) (NEW | OLD)) ]

Examples

UPDATE foobars SET foo = ’a’;UPDATE foobars SET foo = ’a’, bar += 4 WHERE id = 1 AND foo = ’b’;UPDATE foobars SET foo = ’a’, bar += 4 RETURNS ALL NEW;UPDATE foobars SET myset << (5, 6, 7), mytags << ’new tag’ WHERE KEYS IN (’a’, ’b’);UPDATE foobars SET foo = ‘bar + 1‘;

Description

Update the attributes on items in your table.

Parameters

tablename The name of the table

values Comma-separated list of attribute assignments. The supported operators are =, +=, -=, <<, and >>. The‘shovel’ operators (<< & >>) are used to atomically add/remove items to/from a set. Likewise, the += and -=perform atomic inc/decrement and may only be used on NUMBER types.

expression Only return elements that match this expression. The supported operators are =, !=, >, >=, <, and <=.The only conjunction allowed is AND.

There is another form available that looks like KEYS IN (pkey) [, (pkey)]... The pkey is a singlevalue if the table only has a hash key, or two comma-separated values if there is also a range key.

12 Chapter 1. User Guide

Page 17: Contents · PDF fileContents 1. dql, Release 0.1.0-0 ... DQL supports the use of python expressions anywhere that you would ... For this reason, all of your table queries must

dql, Release 0.1.0-0-gadaccee-dirty

When using the first form, DQL does a table query to find matching items and then performs the updates. Thesecond form performs the updates with no reads necessary. If no expression is specified, DQL will perform afull-table scan.

RETURNS Return the items that were operated on. Default is RETURNS NONE. See the Amazon docs for Up-dateItem for more detail.

Notes

When using python expressions to set values, you may reference attributes on the table row:

UPDATE foobars SET foo = ‘bar + 1‘

If you aren’t sure if the attribute will exist or not, you can reference the row dict directly:

us-west-1> UPDATE foobars SET foo = m‘if row.get(’bar’):> return bar + 1> else:> return 1‘;

This syntax will NOT WORK if you are using the KEYS IN form of the query, as that performs the update withoutdoing any table reads.

1.4 Developing

To get started developing dql, run the following command:

wget https://raw.github.com/mathcamp/devbox/master/devbox/unbox.py && \python unbox.py [email protected]:mathcamp/dql

This will clone the repository and install the package into a virtualenv

1.4.1 Running Tests

The command to run tests is python setup.py nosetests. Some of these tests require DynamoDB Local.There is a nose plugin that will download and run the DynamoDB Local service during the tests. It requires the java6/7 runtime, so make sure you have that installed.

1.4. Developing 13

Page 18: Contents · PDF fileContents 1. dql, Release 0.1.0-0 ... DQL supports the use of python expressions anywhere that you would ... For this reason, all of your table queries must

dql, Release 0.1.0-0-gadaccee-dirty

14 Chapter 1. User Guide

Page 19: Contents · PDF fileContents 1. dql, Release 0.1.0-0 ... DQL supports the use of python expressions anywhere that you would ... For this reason, all of your table queries must

CHAPTER 2

API Reference

2.1 dql package

2.1.1 Subpackages

dql.grammar package

Submodules

dql.grammar.common module Common use grammars

dql.grammar.common.upkey(name)Shortcut for creating an uppercase keyword

dql.grammar.query module Grammars for parsing query strings

dql.grammar.query.create_filter()Create a grammar for filtering on table scans

dql.grammar.query.create_filter_constraint()Create a constraint for a scan FILTER clause

dql.grammar.query.create_limit()Create the gramar for a ‘limit’ clause

dql.grammar.query.create_query_constraint()Create a constraint for a query WHERE clause

dql.grammar.query.create_select_where()Create a grammar for the ‘where’ clause used by ‘select’

dql.grammar.query.create_where()Create the grammar for a ‘where’ clause

Module contents

DQL language parser

dql.grammar.create_alter()Create the grammar for the ‘alter’ statement

15

Page 20: Contents · PDF fileContents 1. dql, Release 0.1.0-0 ... DQL supports the use of python expressions anywhere that you would ... For this reason, all of your table queries must

dql, Release 0.1.0-0-gadaccee-dirty

dql.grammar.create_count()Create the grammar for the ‘count’ statement

dql.grammar.create_create()Create the grammar for the ‘create’ statement

dql.grammar.create_delete()Create the grammar for the ‘delete’ statement

dql.grammar.create_drop()Create the grammar for the ‘drop’ statement

dql.grammar.create_dump()Create the grammar for the ‘dump’ statement

dql.grammar.create_insert()Create the grammar for the ‘insert’ statement

dql.grammar.create_parser()Create the language parser

dql.grammar.create_scan()Create the grammar for the ‘scan’ statement

dql.grammar.create_select()Create the grammar for the ‘select’ statement

dql.grammar.create_throughput(variable=primitive)Create a throughput specification

dql.grammar.create_update()Create the grammar for the ‘update’ statement

2.1.2 Submodules

dql.cli module

Interative DQL client

class dql.cli.DQLClient(completekey=’tab’, stdin=None, stdout=None)Bases: cmd.Cmd

Interactive commandline interface

Attributes

running bool(x) -> boolddbengine

complete_display(text, *_)Autocomplete for display

complete_file(text, line, *_)Autocomplete DQL file lookup

complete_ls(text, *_)Autocomplete for ls

16 Chapter 2. API Reference

Page 21: Contents · PDF fileContents 1. dql, Release 0.1.0-0 ... DQL supports the use of python expressions anywhere that you would ... For this reason, all of your table queries must

dql, Release 0.1.0-0-gadaccee-dirty

ddb = None

default(command)

display = None

do_EOF(arglist)Exit

do_code(arglist)Switch to executing python code

do_display(arglist)Get or set the type of display to use when printing results

do_endcode(arglist)Stop executing python code

do_exit(arglist)Exit

do_file(arglist)Read and execute a .dql file

do_ls(arglist)List all tables or print details of one table

do_pagesize(arglist)Get or set the page size of the query output

do_shell(arglist)Run a shell command

do_use(arglist)Switch the AWS region

You may also specify ‘use local host=localhost port=8000’ to use the DynamoDB Local service

do_width(arglist)Get or set the width of the formatted output

do_x(arglist)Toggle expanded display format

You can set smart formatting with ‘x smart’

emptyline()

engine = None

formatter = None

help_alter()Print the help text for ALTER

help_count()Print the help text for COUNT

help_create()Print the help text for CREATE

help_delete()Print the help text for DELETE

help_drop()Print the help text for DROP

2.1. dql package 17

Page 22: Contents · PDF fileContents 1. dql, Release 0.1.0-0 ... DQL supports the use of python expressions anywhere that you would ... For this reason, all of your table queries must

dql, Release 0.1.0-0-gadaccee-dirty

help_dump()Print the help text for DUMP

help_help()Print the help text for help

help_insert()Print the help text for INSERT

help_scan()Print the help text for SCAN

help_select()Print the help text for SELECT

help_update()Print the help text for UPDATE

initialize(region=’us-west-1’, host=’localhost’, port=8000, access_key=None, secret_key=None)Set up the repl for execution

load_config()Load your configuration settings from a file

postcmd(stop, line)

region = None

running = False

save_config_value(key, value)Save your configuration settings to a file

start()Start running the interactive session (blocking)

update_prompt()Update the prompt

dql.cli.connect(region, host=’localhost’, port=8000, access_key=None, secret_key=None)Create a DynamoDB connection

dql.cli.repl_command(fxn)Decorator for cmd methods

Parses arguments from the arg string and passes them to the method as *args and **kwargs.

dql.engine module

Execution engine

class dql.engine.Engine(connection=None)Bases: object

DQL execution engine

Parameters connection : DynamoDBConnection, optional

If not present, you will need to call Engine.connect_to_region()

18 Chapter 2. API Reference

Page 23: Contents · PDF fileContents 1. dql, Release 0.1.0-0 ... DQL supports the use of python expressions anywhere that you would ... For this reason, all of your table queries must

dql, Release 0.1.0-0-gadaccee-dirty

Notes

One of the advanced features of the engine is the ability to set variables which can later be referenced in queries.Whenever a STRING or NUMBER is expected in a query, you may alternatively supply a variable name. Thatname will be looked up in the engine’s scope. This allows you to do things like:

engine.scope[’myfoo’] = ’abc’engine.execute("SELECT * FROM foobars WHERE foo = myfoo")

Attributes

scope dict Lookup scope for variables

cloudwatch_connectionLazy create a connection to cloudwatch

connect_to_region(region, **kwargs)Connect the engine to an AWS region

connectionGet the dynamo connection

describe(tablename, refresh=False, metrics=False)Get the TableMeta for a table

describe_all()Describe all tables in the connected region

execute(commands)Parse and run a DQL string

Parameters commands : str

The DQL command string

get_capacity(tablename)Get the consumed read/write capacity

resolve(val, scope=None)Resolve a value into a string or number

class dql.engine.FragmentEngine(connection)Bases: dql.engine.Engine

A DQL execution engine that can handle query fragments

execute(fragment)Run or aggregate a query fragment

Concat the fragment to any stored fragments. If they form a complete query, run it and return the result. Ifnot, store them and return None.

partialTrue if there is a partial query stored

pformat_exc(exc)Format an exception message for the last query’s parse error

reset()Clear any query fragments from the engine

2.1. dql package 19

Page 24: Contents · PDF fileContents 1. dql, Release 0.1.0-0 ... DQL supports the use of python expressions anywhere that you would ... For this reason, all of your table queries must

dql, Release 0.1.0-0-gadaccee-dirty

dql.engine.float_to_decimal(f)Monkey-patched replacement for boto’s broken version

dql.help module

Help text for the CLI

dql.models module

Data containers

class dql.models.GlobalIndex(name, index_type, status, hash_key, range_key, read_throughput,write_throughput, size, item_count, includes=None)

Bases: object

Container for global index data

classmethod from_description(description, attrs)Create an object from a boto response

pformat()Pretty format for insertion into table pformat

schemaThe DQL fragment for constructing this index

class dql.models.IndexField(name, data_type, index_type, index_name, includes=None)Bases: dql.models.TableField

A TableField that is also part of a Local Secondary Index

schemaThe DQL syntax for creating this item

class dql.models.TableField(name, data_type, key_type=None)Bases: object

A DynamoDB table attribute

Parameters name : str

data_type : str

The type of object (e.g. ‘STRING’, ‘NUMBER’, etc)

key_type : str, optional

The type of key (e.g. ‘RANGE’, ‘HASH’, ‘INDEX’)

index_name : str, optional

If the key_type is ‘INDEX’, this will be the name of the index that uses the field as arange key.

schemaThe DQL syntax for creating this item

to_index(index_type, index_name, includes=None)Create an index field from this field

20 Chapter 2. API Reference

Page 25: Contents · PDF fileContents 1. dql, Release 0.1.0-0 ... DQL supports the use of python expressions anywhere that you would ... For this reason, all of your table queries must

dql, Release 0.1.0-0-gadaccee-dirty

class dql.models.TableMeta(name, status, attrs, global_indexes, read_throughput, write_throughput,decreases_today, size, item_count)

Bases: object

Container for table metadata

Parameters name : str

status : str

attrs : dict

Mapping of attribute name to :class:‘.TableField‘s

global_indexes : dict

Mapping of hash key to GlobalIndex

read_throughput : int

write_throughput : int

decreases_today : int

consumed_read_capacity : int

May be None if unknown

consumed_write_capacity : int

May be None if unknown

size : int

Size of the table in bytes

item_count : int

Number of items in the table

decreases_remainingNumber of remaining times you may decrease throughput today

classmethod from_description(description)Factory method that uses the boto ‘describe’ return value

pformat()Pretty string format

primary_key(hkey, rkey=None)Construct a primary key dictionary

You can either pass in a (hash_key[, range_key]) as the arguments, or you may pass in an Item itself

schemaThe DQL query that will construct this table’s schema

total_read_throughputCombined read throughput of table and global indexes

total_write_throughputCombined write throughput of table and global indexes

2.1. dql package 21

Page 26: Contents · PDF fileContents 1. dql, Release 0.1.0-0 ... DQL supports the use of python expressions anywhere that you would ... For this reason, all of your table queries must

dql, Release 0.1.0-0-gadaccee-dirty

dql.output module

Formatting and displaying output

class dql.output.BaseFormat(width=100, pagesize=1000)Bases: object

Base class for formatters

format(result, ostream)Format a single result and stick it in an output stream

format_field(field)Format a single Dynamo value

write(results, ostream)Write results to an output stream

class dql.output.ColumnFormat(width=100, pagesize=1000)Bases: dql.output.BaseFormat

A layout that puts item attributes in columns

format(results, columns, ostream)

write(results, ostream)

class dql.output.ExpandedFormat(width=100, pagesize=1000)Bases: dql.output.BaseFormat

A layout that puts item attributes on separate lines

format(result, ostream)

class dql.output.SmartFormat(*args, **kwargs)Bases: dql.output.ColumnFormat

A layout that chooses column/expanded format intelligently

format(results, columns, ostream)

dql.output.get_default_display()Get the default display function for this system

dql.output.less_display(*args, **kwds)Use smoke and mirrors to acquire ‘less’ for pretty paging

dql.output.stdout_display(*args, **kwds)Print results straight to stdout

dql.output.truncate(string, length, ellipsis=u’...’)Truncate a string to a length, ending with ‘...’ if it overflows

dql.output.wrap(string, length, indent)Wrap a string at a line length

2.1.3 Module contents

Simple SQL-like query language for dynamo

dql.main()Start the DQL client

22 Chapter 2. API Reference

Page 28: Contents · PDF fileContents 1. dql, Release 0.1.0-0 ... DQL supports the use of python expressions anywhere that you would ... For this reason, all of your table queries must

dql, Release 0.1.0-0-gadaccee-dirty

24 Chapter 3. Versions

Page 29: Contents · PDF fileContents 1. dql, Release 0.1.0-0 ... DQL supports the use of python expressions anywhere that you would ... For this reason, all of your table queries must

CHAPTER 4

Changelog

4.1 0.1.0

• First public release

25

Page 30: Contents · PDF fileContents 1. dql, Release 0.1.0-0 ... DQL supports the use of python expressions anywhere that you would ... For this reason, all of your table queries must

dql, Release 0.1.0-0-gadaccee-dirty

26 Chapter 4. Changelog

Page 31: Contents · PDF fileContents 1. dql, Release 0.1.0-0 ... DQL supports the use of python expressions anywhere that you would ... For this reason, all of your table queries must

CHAPTER 5

Indices and tables

• genindex

• modindex

• search

27

Page 32: Contents · PDF fileContents 1. dql, Release 0.1.0-0 ... DQL supports the use of python expressions anywhere that you would ... For this reason, all of your table queries must

dql, Release 0.1.0-0-gadaccee-dirty

28 Chapter 5. Indices and tables

Page 33: Contents · PDF fileContents 1. dql, Release 0.1.0-0 ... DQL supports the use of python expressions anywhere that you would ... For this reason, all of your table queries must

Python Module Index

ddql, ??dql.cli, ??dql.engine, ??dql.grammar, ??dql.grammar.common, ??dql.grammar.query, ??dql.help, ??dql.models, ??dql.output, ??

29

Page 34: Contents · PDF fileContents 1. dql, Release 0.1.0-0 ... DQL supports the use of python expressions anywhere that you would ... For this reason, all of your table queries must

dql, Release 0.1.0-0-gadaccee-dirty

30 Python Module Index

Page 35: Contents · PDF fileContents 1. dql, Release 0.1.0-0 ... DQL supports the use of python expressions anywhere that you would ... For this reason, all of your table queries must

Python Module Index

ddql, ??dql.cli, ??dql.engine, ??dql.grammar, ??dql.grammar.common, ??dql.grammar.query, ??dql.help, ??dql.models, ??dql.output, ??

31