how to understand and use the query optimizer – couchbase connect 2016

©2016 Couchbase Inc.

Understand Query Optimizer to Tune QueriesSitaram Vemulapalli | Senior Software Engineer, Couchbase R&D | [email protected]

Keshav Murthy | Director, Couchbase R&D | [email protected]

1

©2016 Couchbase Inc.©2016 Couchbase Inc.

Agenda

• Brief Introduction to N1QL

• Understand Query Plan

• How Query Plan Generated

• Query + Indexing features

• Q&A

2

©2016 Couchbase Inc. 3

Introduction to N1QL


SQL

4

ResultSet

Input: Relations Output: Relation


NoSQL

5

{"Name" : "Jane Smith","DOB" : "1990-01-30","Billing" : [

{"type" : "visa","cardnum" : "5827-2842-2847-3909","expiry" : "2019-03"

},{

"type" : "master","cardnum" : "6274-2842-2847-3909","expiry" : "2019-03"

}],"Connections" : [

{"CustId" : "XYZ987","Name" : "Joe Smith"

},{

"CustId" : "PQR823","Name" : "Dylan Smith"

}{


}],"Purchases" : [

{ "id":12, item: "mac", "amt": 2823.52 }{ "id":19, item: "ipad2", "amt": 623.52 }

]}

LoyaltyInfo ResultDocuments

Orders

CUSTOMER

Built Manually; Expensive

Input: JSON Documents Output: JSON Documents


N1QL Changes the Possibilities for NoSQL


NoSQL

{"Name" : "Jane Smith","DOB" : "1990-01-30","Billing" : [

{"type" : "visa","cardnum" : "5827-2842-2847-3909","expiry" : "2019-03"

},{

"type" : "master","cardnum" : "6274-2842-2847-3909","expiry" : "2019-03"

}],"Connections" : [

{"CustId" : "XYZ987","Name" : "Joe Smith"

},{


}{


}],"Purchases" : [

{ "id":12, item: "mac", "amt": 2823.52 }{ "id":19, item: "ipad2", "amt": 623.52 }

]}

LoyaltyInfo ResultDocuments

Orders

CUSTOMER

Input: JSON Documents Output: JSON Documents


Goal of N1QL: SQL for JSON

Give developers and enterprises an

expressive, powerful, and complete language

for querying, transforming, and manipulating

JSON data.

©2016 Couchbase Inc.©2016 Couchbase Inc. 9

SELECT Customers.ID,

Customers.Name,

SUM(OrderLine.Amount)

FROM Orders UNNEST

Orders.LineItems AS OrderLine

JOIN Customers ON KEYS Orders.CustID

GROUP BY Customers.ID, Customers.Name

HAVING SUM(OrderLine.Amount) > 10000

ORDER BY SUM(OrderLine.Amount) DESC

LIMIT 100 ;


Understanding Query Plan


N1QL: Query Execution Flow

Clients

1. Submit the query over REST API 8. Query result

2. Parse, Analyze, create Plan 7. Evaluate: Filter, Join, Aggregate, Sort, Paginate

3. Scan Request; index filters

6. Fetch the documents

Index Service

Query Service

Data Service

4. Get qualified doc keys

5. Fetch Request, doc keys

SELECT c_id,

c_first,

c_last,

c_max

FROM CUSTOMER

WHERE c_id = 49165;

{

"c_first": "Joe","c_id": 49165,"c_last": "Montana","c_max" : 50000

}


Inside a Query Service

Client

FetchParse Plan Join FilterPre-Aggregate

Offset Limit ProjectSortAggregateScan

Query Service

Index Service

Data Service


Inside a Query Service

FetchParse Plan Join FilterPre-Aggregate

Offset Limit ProjectSortAggregateScan

Query Service

Index Service

Data Service

Prepare Phase

Projection Phase

Run Phase

ScanPhase

FetchPhase


Query Execution: Plan

Fetch

Parse

Plan

Join

Filter

Offset

Limit

Project

Sort

Aggregate

Scan

• Each query can be executed in several ways

• Create the query execution plan• Access path for each keyspace referenced

• Decide on the filters to push down

• Determine Join order and join method

• Create the execution tree

• For each keyspace reference:• Look at the available indices

• Match the filters in the query with index condition and keys

• Choose one or more indices for each keyspace

• Decide if a covering index can be used

• Covering index is preferred

• Create index filters and post scan, post join filters



Fetch

Parse

Plan

Join

Filter

Offset

Limit

Project

Sort

Aggregate

Scan

• Covering Index Scan

• Predicate pushdown to Indexer

• Pushdown limit to IndexScan (4.5)

• Use Index Order (4.5)

CREATE INDEX ts_c_id ON `travel-sample` (country, id) WHERE type = "airline";

EXPLAIN SELECT country, idFROM `travel-sample` tWHERE type = "airline" AND country = "United States" AND id BETWEEN 10 AND 1000ORDER BY country, idOFFSET 5LIMIT 10;

{"#operator": "IndexScan","covers": [

"cover ((`t`.`country`))","cover ((`t`.`id`))","cover ((meta(`t`).`id`))"

],"filter_covers": {

"cover ((`t`.`type`))": "airline"},"index": "ts_c_id","index_id": "ccd0f0c297114733","keyspace": "travel-sample","limit": "(5 + 10)","namespace": "default","spans": [

{"Range": {

"High": ["\"United States\"","1000"

],"Inclusion": 3,"Low": [

"\"United States\"","10"

]}

}],"using": "gsi"

}



Fetch

Parse

Plan

Join

Filter

Offset

Limit

Project

Sort

Aggregate

Scan

• Non Covering Index Scan

• Predicate pushdown to Indexer

• Pushdown limit to IndexScan (4.5)

• Use Index order (4.5)

CREATE INDEX ts_c_id ON `travel-sample`(country, id) WHERE type = "airline";

EXPLAIN SELECT country, id, nameFROM `travel-sample` tWHERE type = "airline" AND country = "United States" AND id BETWEEN 10 AND 1000ORDER BY country, idOFFSET 5LIMIT 10;

{"#operator": "IndexScan","index": "ts_c_id","index_id": "ccd0f0c297114733","keyspace": "travel-sample","limit": "(5 + 10)","namespace": "default","spans": [

{"Range": {




]}

}],"using": "gsi"

}


N1QL: Covering Index and Covered Query

Clients

1. Submit the query over REST API 6. Query result

2. Parse, Analyze, create Plan 5. Evaluate: Filter, Join, Aggregate, Sort, Paginate

3. Scan Request; index filters

Index Service

Query Service

4. Get qualified doc keys

create index i1 on CUSTOMER(c_id, c_max, c_first, c_last)

SELECT c_id,

c_first,

c_last,

c_max

FROM CUSTOMER

WHERE c_id = 49165;

{

"c_first": "Joe","c_id": 49165,"c_last": "Montana","c_max" : 50000

}



Fetch

Parse

Plan

Join

Filter

Offset

Limit

Project

Sort

Aggregate

Scan

• Non Covering Query

• Fetch (Only for Non Covering Scan)

• Filter


EXPLAIN SELECT country, id, nameFROM `travel-sample` tWHERE type = "airline" AND country = "United States" AND id BETWEEN 10 AND 1000ORDER BY country, idOFFSET 5LIMIT 10;

{"#operator": "Fetch","keyspace": "travel-sample","namespace": "default"

}

{"#operator": "Filter","condition": "((((`t`.`type`) =

\"airline\") and ((`t`.`country`) = \"United States\")) and ((`t`.`id`) between 10 and 1000))"}



Fetch

Parse

Plan

Join

Filter

Offset

Limit

Project

Sort

Aggregate

Scan

• IntersectScan

• For non covered queries only

CREATE INDEX ts_c ON `travel-sample`(country) WHERE type = "airline" ;

CREATE INDEX ts_n ON `travel-sample`(name) WHERE type = "airline" ;

EXPLAIN SELECT country, nameFROM `travel-sample` tWHERE type = "airline" AND country = "United States" AND name = "US Airways";

{"#operator": "IntersectScan","scans": [

{ "#operator": "IndexScan","index": "ts_c","keyspace": "travel-sample","spans": [

{"Range": {

"High": [ "\"United States\”"],"Inclusion": 3,"Low": [ "\"United States\””]

}}

]},{ "#operator": "IndexScan",

"index": "ts_n","keyspace": "travel-sample","spans": [

{"Range": {

"High": ["\"US Airways\”” ],"Inclusion": 3,"Low": ["\"US Airways\”” ]

}}

]}

]}



Fetch

Parse

Plan

Join

Filter

Offset

Limit

Project

Sort

Aggregate

Scan

• Array Index Key Scan (4.5)

CREATE INDEX ts_aix1 ON `travel-sample` (DISTINCT ARRAY v.day FOR v IN schedule END) WHERE type = "route";

SELECT META(t).idFROM `travel-sample` tWHERE type = "route" AND ANY v IN schedule SATISFIES v.day IN [1,2] END;

{ "#operator": "DistinctScan","scan": {

"#operator": "IndexScan","index": "ts_aix1","keyspace": "travel-sample","spans": [

{"Range": {

"High": ["1"


"1"]

}},{

"Range": {"High": [

"2"],"Inclusion": 3,"Low": [

"2"]

}}

]}

}



Fetch

Parse

Plan

Join

Filter

Offset

Limit

Project

Sort

Aggregate

Scan

• IndexCountScan (4.5)


EXPLAIN SELECT COUNT(1) AS countFROM `travel-sample` tWHERE type = "airline" AND country = "United States" AND id BETWEEN 10 AND 1000;

{ "#operator": "IndexCountScan","covers": [

"cover ((`t`.`country`))","cover ((`t`.`id`))","cover ((meta(`t`).`id`))"

],"index": "ts_c_id","keyspace": "travel-sample","spans": [

{"Range": {




]}

}]

}{ "#operator": "IndexCountProject",

"result_terms": [{

"as": "count","expr": "count(1)"

}]

}



Fetch

Parse

Plan

Join

Filter

Offset

Limit

Project

Sort

Aggregate

Scan

• JOIN, LEFT JOIN.

• NEST, LEFT NEST will have operator NEST.

• LEFT side keyspace of JOIN can be covered.

• Non-covered query.

CREATE INDEX ts_rix1 ON `travel-sample` (DISTINCT ARRAY v.day FOR v IN schedule END, airlineid, schedule) WHERE type = "route";

EXPLAIN SELECT r, a FROM `travel-sample` r LEFT JOIN `travel-sample` a ON KEYS r.airlineidWHERE r.type = "route" AND ANY v IN r.schedule SATISFIES v.day = 1 ENDLIMIT 1;

"scan": {"#operator": "IndexScan","index": "ts_rix1","keyspace": "travel-sample","spans": [

{"Range": {

"High": ["1"


"1"]

}}

]}

{"#operator": "Join","as": "a","keyspace": "travel-sample","namespace": "default","on_keys": "(`r`.`airlineid`)","outer": true

}



Fetch

Parse

Plan

Join

Filter

Offset

Limit

Project

Sort

Aggregate

Scan

• JOIN LEFT side covered query.

CREATE INDEX ts_rix1 ON `travel-sample` (DISTINCT ARRAY v.day FOR v IN schedule END, airlineid, schedule) WHERE type = "route";

EXPLAIN SELECT r.schedule, META(r).idFROM `travel-sample` r LEFT JOIN `travel-sample` a ON KEYS r.airlineidWHERE r.type = "route" AND ANY v IN r.schedule SATISFIES v.day = 1 ENDLIMIT 1;

"scan": {"#operator": "IndexScan","covers": [

"cover ((distinct (array (`v`.`day`) for `v` in (`r`.`schedule`) end)))",

"cover ((`r`.àirlineid`))","cover ((`r`.`schedule`))","cover ((meta(`r`).ìd`))"


"cover ((`r`.`type`))": "route"}"index": "ts_rix1","keyspace": "travel-sample","spans": [

{"Range": {

"High": ["1"


"1"]

}}

]}

{"#operator": "Join","as": "a","keyspace": "travel-sample","namespace": "default","on_keys": ”cover ((`r`.àirlineid`))","outer": true

}


How does this work?

CREATE INDEX brewery_state ON `beer-sample`(state);CREATE INDEX beer_brewery_id ON `beer-sample`(brewery_id);

SELECT brewery.name as breweryname, ARRAY_AGG(beer.name) beerlistFROM `beer-sample` brewery

LEFT OUTER JOIN `beer-sample` beer ON KEY beer.brewery_id FOR brewery

WHERE brewery.state = "California"GROUP BY brewery.name;

brewerybrewery_state

[“California”, “21st_sfo_ale”]

beer_brewery_id

(brewery_id = “21st_sfo_ale”)

beer

[“21st_sfo_ale”, ”pale_ale”]

(state = ‘California’)



Fetch

Parse

Plan

Join

Filter

Offset

Limit

Project

Sort

Aggregate

Scan

• Index JOIN, LEFT Index JOIN.

• Index NEST, LEFT Index NEST will have operator NEST.

• LEFT side keyspace of JOIN can be covered.

• In 4.6 RIGHT side keyspace of Index JOIN can be covered. NOT for Index NEST.

CREATE INDEX ts_c ON `travel-sample`(country) WHERE type = "airline" ;CREATE INDEX ts_aid ON `travel-sample`(airlineid, airline) WHERE type = "route" ;

EXPLAIN SELECT r, aFROM `travel-sample` a JOIN `travel-sample` r ON KEY r.airlineid FOR aWHERE r.type = "route" AND a.type = "airline" AND a.country = "United States"LIMIT 1;

{"#operator": "IndexScan","index": "ts_c","keyspace": "travel-sample","spans": [

{"Range": {

"High": ["\"United States\""


"\"United States\""]

}}

]}

{"#operator": "IndexJoin","as": "r","for": "a","keyspace": "travel-sample","on_key": "(`r`.`airlineid`)","scan": {

"index": "ts_aid"}

}



Fetch

Parse

Plan

Join

Filter

Offset

Limit

Project

Sort

Aggregate

Scan

• Covered Index Join (4.5)

CREATE INDEX ts_c ON `travel-sample`(country) WHERE type = "airline" ;CREATE INDEX ts_aid ON `travel-sample`(airlineid, airline) WHERE type = "route" ;

EXPLAIN SELECT a.country, r.airlineFROM `travel-sample` a JOIN `travel-sample` r ON KEY r.airlineid FOR aWHERE r.type = "route" AND a.type = "airline" AND a.country = "United States";


"cover ((à`.`country`))","cover ((meta(à`).ìd`))"


"cover ((à`.`type`))": "airline"},"index": "ts_c","keyspace": "travel-sample","spans": [

{"Range": {

"High": ["\"United States\""



}}

]}

{"#operator": "IndexJoin","as": "r","for": "a","keyspace": "travel-sample","on_key": "(`r`.àirlineid`)","scan": {

"index": "ts_aid"}

}



Fetch

Parse

Plan

Join

Filter

Offset

Limit

Project

Sort

Aggregate

Scan

• LEFT & RIGHT Covered Index Join (4.6)

CREATE INDEX ts_c ON `travel-sample`(country) WHERE type = "airline" ;CREATE INDEX ts_aid ON `travel-sample`(airlineid, airline) WHERE type = "route" ;EXPLAIN SELECT a.country, r.airlineFROM `travel-sample` a JOIN `travel-sample` r ON KEY r.airlineid FOR aWHERE r.type = "route" AND a.type = "airline" AND a.country = "United States";


"cover ((à`.`country`))","cover ((meta(à`).ìd`))"


"cover ((à`.`type`))": "airline"},"index": "ts_c”

………}

{"#operator": "IndexJoin","as": "r","for": "a","keyspace": "travel-sample","on_key": "cover ((`r`.àirlineid`))","scan": {

"covers": ["cover ((`r`.àirlineid`))","cover ((`r`.àirline`))","cover ((meta(`r`).ìd`))"


"cover ((`r`.`type`))": "route"},"index": "ts_aid"

}}



Fetch

Parse

Plan

Join

Filter

Offset

Limit

Project

Sort

Aggregate

Scan

• Aggregate


EXPLAIN SELECT country, count(1) countFROM `travel-sample` tWHERE type = "airline"GROUP BY country;

{"#operator": "InitialGroup","aggregates": [

"count(1)"],"group_keys": [

"(`t`.`country`)"]

}

{"#operator": "FinalGroup","aggregates": [


"(`t`.`country`)"]

}

{"#operator": "IntermediateGroup","aggregates": [


"(`t`.`country`)"]

}



Fetch

Parse

Plan

Join

Filter

Offset

Limit

Project

Sort

Aggregate

Scan

• Sort

• Offset

• Limit


EXPLAIN SELECT country, id, nameFROM `travel-sample` tWHERE type = "airline" AND country = "United States" AND id BETWEEN 10 AND 1000ORDER BY country, id DESCOFFSET 5LIMIT 10;

{"#operator": "Order","limit": "10","offset": "5","sort_terms": [

{"expr": "(`t`.`country`)"

},{

"desc": true,"expr": "(`t`.`id`)"

}]

}

{"#operator": "Offset","expr": "5"

}

{"#operator": "Limit","expr": "10"

}



Fetch

Parse

Plan

Join

Filter

Offset

Limit

Project

Sort

Aggregate

Scan

• Project


EXPLAIN SELECT country, id, nameFROM `travel-sample` tWHERE type = "airline" AND country = "United States" AND id BETWEEN 10 AND 1000ORDER BY country, id DESCOFFSET 5LIMIT 10;

{"#operator": "InitialProject","result_terms": [

{"expr": "(`t`.`country`)"

},{

"expr": "(`t`.`id`)"},{

"expr": "(`t`.`name`)"}

]}

{"#operator": "FinalProject"

}


How Query Plan Generated


Query Plan Generation

• Rule based optimization

• The index chosen by N1QL will satisfy the query

• Each query block has is optimized separately & stitched.

32



• Index Selection Criteria• If the query contains a USE KEYS clause, no index scan or full / primary scan is performed.

The input document keys are taken directly from the USE KEYS clause.

• If there is an explicit or implicit query predicate:• Covering secondary scan;

• Regular secondary scan -- longest satisfying keys, intersect scan;

• UNNEST scan;

• Regular primary scan

• If the query does not contain a predicate • covering primary scan;

• regular primary scan.

• If a primary scan is selected, and there is no primary index available, the query errors out.

33



• Covering Primary Scan• A covering primary scan is a primary scan that does not perform a subsequent document fetch. It is

used for queries that need a full / primary scan and only reference META().id.

• Regular Primary Scan• A regular primary scan also performs a subsequent document fetch. It is used for queries that need a

full / primary scan and reference some document data other than META().id.

34

SELECT META(t).id FROM `travel-sample` t;

SELECT META(t).cas FROM `travel-sample` t;

SELECT * FROM `travel-sample` t;

SELECT t.type FROM `travel-sample` t;



Preliminaries :• Online indexes

• Only online indexes are considered

• Preferred indexes• USE INDEX hint is provided the indexes in that list are only considered

• Satisfying Index condition• Partial / filtered indexes that index condition is super set of query predicate are

considered

• Satisfying Index keys• Indexes whose leading keys satisfy query predicate are considered

• Longest satisfying index keys• Redundancy is eliminated båy keeping longest satisfying index keys in same order.

• Index with satisfying keys (a,b,c) is retained over index with satisfying (a,b)

35



Covering Secondary Scan• Each satisfied index with most number of index keys is examined for query coverage

• Shortest covering index will be used.

Regular Secondary Scan• Indexes in with most number of matching index keys are used

• When more than one index are qualified, IntersectScan is used.

• To avoid IntersectScan provide hint with USE INDEX.

36

CREATE INDEX ts_name ON `travel-sample`(country, name) WHERE type = "airline";

SELECT country, name, type, META().id FROM `travel-sample` WHERE type = "airline" AND country = "United States";

SELECT country, name, type, id FROM `travel-sample` WHERE type = "airline" AND country = "United States";



UNNEST Scan• Only array indexes are considered. And only queries with UNNEST clauses are considered

IndexCountScan• Queries with single projection of COUNT aggregate, NO JOIN’s, GROUP BY is considered

• Chosen Index needs to be covered with single range, exact range will be able to push to indexer and argument to COUNT needs to be constant or leading key

37

CREATE INDEX ts_c ON `travel-sample`(country) WHERE type = "airline";

SELECT COUNT(1) AS countFROM `travel-sample` WHERE type = "airline" AND country = "United States";



• In order to satisfy an AND predicate, an index only needs to satisfy any term in the AND predicate.

• In order to satisfy an OR predicate, an index must satisfy every term in the OR predicate.

38



Index Satisfies these predicates Index doesn’t satisfy these predicates

CREATE INDEX idx_dept ON employee( dept_id, hire_date, base_comp );

hire_date >= "2010" AND dept_id = "xyz"hire_date >= "2010" AND dept_id = "xyz" AND last_name = "Smith"dept_id = "abc" AND base_comp > 50000dept_id = "abc" AND bonus > 2000

dept_id = "abc" OR dept_id = "xyz"(hire_date >= "2010" AND dept_id >= "xyz") OR (hire_date >= "2010" AND dept_id = "abc")dept_id = "abc" OR (hire_date >= "2010" AND dept_id >= "xyz")(hire_date >= "2010" AND dept_id >= "xyz") OR (dept_id = "abc" AND bonus > 2000)

hire_date >= "2010" AND base_comp > 50000hire_date >= "2010" AND bonus > 2000bonus > 2000 AND base_comp > 50000bonus > 2000 AND last_name = "Smith"

dept_id = "abc" OR hire_date >= "2010"dept_id = "abc" OR dept_id = "xyz" OR base_comp > 50000dept_id = "abc" OR dept_id = "xyz" OR bonus > 2000hire_date > "2010" OR base_comp > 50000


Query Optimization To Exploit Index Features



• N1QL Query engine can exploit the following index features.

• By creating right index and query that takes advantage performs better.

• Pushing predicates hint to Index

• Index Filters

• Using Index Order

• Pushing LIMIT to Index

• Pushing Offset as LIMIT (limit = limit + offset) to Index

• Top n Heap for Order

• Index Count (GSI Index only)

• Pushing MIN to Index

• Duplicate Indexes and Load balancing

• USE INDEX

41



USE INDEX

• Queries can specify which index to use with USE INDEX clause

• Multiple indexes qualify for query and one performs better than others

• To avoid Intersect Scan

• If there are many indexes on data source and queries are ad-hoc USE INDEX can reduce prepare time

Pushing predicates

• Predicates are pushed to indexer when possible

• These can be examined as spans in Index section of the EXPLAIN plan

• The spans can be exact or superset and never be subset of the predicates

• If the spans are exact the applying query predicates again will not eliminate the document

Index Filters

• If Index filter has equal on expression, that expression not required to include in the index keys. This allows keep the index LEAN . Queries can exploit index filter condition for covered queries.

42

CREATE INDEX ts_c ON `travel-sample`(country) WHERE type = "airline" ;SELECT type, country FROM `travel-sample`WHERE type = "airline" AND country = "United States";



Using Index Order

• Query ORDER BY list matches with index keys list order left to right with single range span

• Query doesn’t have any JOINs, GROUP BY or other clauses that can change the order produced by indexer

• The query can exploit index order and avoid expensive sort and fetching lot of data unnecessarily in some cases

43

CREATE INDEX ts_c_id ON `travel-sample`(country, id) WHERE type = "airline" ;

SELECT country, id FROM `travel-sample`WHERE type = "airline" AND country = "United States”ORDER BY id; --4.6 Uses Index order without any change

SELECT country, id FROM `travel-sample`WHERE type = "airline" AND country = "United States”ORDER BY country, id; --4.5 Uses Index order

CREATE INDEX ts_c_nid ON `travel-sample`(country, -id) WHERE type = "airline" ;

SELECT country, -(-id) FROM `travel-sample`WHERE type = "airline" AND country = "United States”ORDER BY country, -id; -- id prodcued in DESC order



Push LIMIT to Index

• Pushing the LIMIT is hint to indexer when to stop

• Pushing LIMIT can be done only when the predicates are pushed to indexer, spans are exact and matches index order

• In case multiple spans LIMIT can be pushed to each span

44

CREATE INDEX ts_c_id ON `travel-sample`(country, id) WHERE type = "airline" ;

SELECT country, id FROM `travel-sample`WHERE type = "airline" AND country = "United States" AND id BETWEEN 0 AND 1000LIMIT 10; --LIMIT can be pushed

SELECT country, id FROM `travel-sample`WHERE type = "airline" AND country IN ["United States", "United Kingdom"] AND id BETWEEN 0 AND 1000LIMIT 10; --LIMIT can be pushed

CREATE INDEX ts_c_id ON `travel-sample`(id, country) WHERE type = "airline" ;

SELECT country, id FROM `travel-sample`WHERE type = "airline" AND country = "United States" AND id BETWEEN 0 AND 1000LIMIT 10; --LIMIT can’t be pushed. Indexer produces {"id":11, "country":"France"}



Push Offset

• If LIMIT is present offset is added to limit and pushed limit to Indexer

Top-n Heap for Order

• Query requires order and has LIMIT less than 64K it uses Top-n Heap to eliminate unwanted documents before order.

Index Count

• Count is done by indexer

• Only GSI index supports Index count

• Queries with single projection of COUNT aggregate, NO JOIN’s, GROUP BY is considered

• Chosen Index needs to be covered with single range, exact range will be able to push to indexer and argument to COUNT needs to be constant or leading key

45


SELECT COUNT(1) AS countFROM `travel-sample` WHERE type = "airline" AND country = "United States";



MIN

• Queries with single projection of MIN aggregate, NO JOIN’s, GROUP BY is considered.

• Chosen Index needs to be covered with single range, exact range will be able to push to indexer and argument to MIN needs to be leading key.

• If the required conditions are satisfied limit 1 pushed to indexer.

46


SELECT MIN(country)FROM `travel-sample` t WHERE type = "airline" AND country > "United States";


"cover ((`t`.`country`))","cover ((meta(`t`).`id`))"


"cover ((`t`.`type`))": "airline"},"index": "ts_c","keyspace": "travel-sample","limit": "1","spans": [

{"Range": {

"Inclusion": 0,"Low": [


}}

]}



Duplicate Indexes and Load balancing

• Allows duplicate indexes (i.e. different name but identical schema) on same or different Indexer services

• Query Service will choose one of the index during prepare time

• During execution Indexer client does load balancing and uses one of the indexer to service query

47


Q&A


Keshav MurthyDirector

Contact information

Sitaram VemulapalliSr. Software Engineer