relational - university of washington · 2018. 1. 22. · relational algebra operators union ∪,...

CSE 344JANUARY 22ND –RELATIONAL ALGEBRA

ASSORTED MINUTIAE• HW2 and Online Quiz 2 due on Wednesday• Azure accounts will be created tonight

• You will be added to your account with your @cs.washington.edu email address

• If you don’t have one, email me and I will attach a different address

• When you get access, make sure that you only run queries that you need

• Due next Friday (some overlap)

TODAY’S LECTURE• Finalizing Subqueries• Queries as Relational algebra

MONOTONE QUERIESDefinition A query Q is monotone if:

• Whenever we add tuples to one or more input tables, the answer to the query will not lose any of the tuples

MONOTONE QUERIESTheorem: If Q is a SELECT-FROM-WHERE query that does not have subqueries, and no aggregates, then it is monotone.

CSE 344 - 2017au 5

MONOTONE QUERIESTheorem: If Q is a SELECT-FROM-WHERE query that does not have subqueries, and no aggregates, then it is monotone.

Proof. We use the nested loop semantics: if we insert a tuple in a relation Ri, this will not remove any tuples from the answer

CSE 344 - 2017au 6

SELECT a1, a2, …, akFROM R1 AS x1, R2 AS x2, …, Rn AS xnWHERE Conditions

for x1 in R1 dofor x2 in R2 do

…for xn in Rn doif Conditionsoutput (a1,…,ak)

MONOTONE QUERIESThe query:

is not monotone

7

Find all companies s.t. all their products have price < 200

Product (pname, price, cid)Company (cid, cname, city)


is not monotone

8


pname price cid

Gizmo 19.99 c001

cid cname city

c001 Sunworks Bonn

cname

Sunworks



is not monotone

Consequence: If a query is not monotonic, then we cannot write it as a SELECT-FROM-WHERE query without nested subqueries

9


pname price cid

Gizmo 19.99 c001

cid cname city

c001 Sunworks Bonn

cname

Sunworks

pname price cid

Gizmo 19.99 c001

Gadget 999.99 c001

cid cname city

c001 Sunworks Bonn

cname


QUERIES THAT MUST BE NESTEDQueries with universal quantifiers or with negation

CSE 344 - 2017au 10

QUERIES THAT MUST BE NESTEDQueries with universal quantifiers or with negation

Queries that use aggregates in certain ways

• sum(..) and count(*) are NOT monotone, because they do not satisfy set containment

• select count(*) from R is not monotone!

CSE 344 - 2017au 11

GROUP BY V.S. NESTED QUERIESSELECT product, Sum(quantity) AS TotalSalesFROM PurchaseWHERE price > 1GROUP BY product

SELECT DISTINCT x.product, (SELECT Sum(y.quantity)FROM Purchase yWHERE x.product = y.productAND y.price > 1)

AS TotalSalesFROM Purchase xWHERE x.price > 1

Why twice ?

Purchase(pid, product, quantity, price)

MORE UNNESTING

CSE 344 - 2017au 13

Author(login,name)Wrote(login,url)

Find authors who wrote ≥ 10 documents:

MORE UNNESTING

SELECT DISTINCT Author.nameFROM AuthorWHERE (SELECT count(Wrote.url)

FROM WroteWHERE Author.login=Wrote.login)

>= 10

This isSQL bya novice

Attempt 1: with nested queries



MORE UNNESTING

Attempt 1: with nested queries



SELECT Author.nameFROM Author, WroteWHERE Author.login=Wrote.loginGROUP BY Author.nameHAVING count(wrote.url) >= 10

This isSQL by

an expert

Attempt 2: using GROUP BY and HAVING

FINDING WITNESSES


For each city, find the most expensive product made in that city

FINDING WITNESSES

SELECT x.city, max(y.price)FROM Company x, Product yWHERE x.cid = y.cidGROUP BY x.city;

Finding the maximum price is easy…

But we need the witnesses, i.e., the products with max price

For each city, find the most expensive product made in that city


FINDING WITNESSESTo find the witnesses, compute the maximum pricein a subquery (in FROM or in WITH)


WITH CityMax AS (SELECT x.city, max(y.price) as maxpriceFROM Company x, Product yWHERE x.cid = y.cidGROUP BY x.city)

SELECT DISTINCT u.city, v.pname, v.priceFROM Company u, Product v, CityMax wWHERE u.cid = v.cid

and u.city = w.cityand v.price = w.maxprice;

FINDING WITNESSESTo find the witnesses, compute the maximum pricein a subquery (in FROM or in WITH)

SELECT DISTINCT u.city, v.pname, v.priceFROM Company u, Product v,

(SELECT x.city, max(y.price) as maxpriceFROM Company x, Product yWHERE x.cid = y.cidGROUP BY x.city) w

WHERE u.cid = v.cidand u.city = w.cityand v.price = w.maxprice;


FINDING WITNESSES

Or we can use a subquery in where clause

SELECT u.city, v.pname, v.priceFROM Company u, Product vWHERE u.cid = v.cidand v.price >= ALL (SELECT y.price

FROM Company x, Product y WHERE u.city=x.cityand x.cid=y.cid);


FINDING WITNESSES

There is a more concise solution here:

SELECT u.city, v.pname, v.priceFROM Company u, Product v, Company x, Product yWHERE u.cid = v.cid and u.city = x.cityand x.cid = y.cidGROUP BY u.city, v.pname, v.priceHAVING v.price = max(y.price)


RELATIONAL ALGEBRASet-at-a-time algebra, which manipulates relationsIn SQL we say what we wantIn RA we can express how to get itEvery DBMS implementations converts a SQL query to RA in order to execute itAn RA expression is called a query plan

BASICS

• Relations and attributes• Functions that are applied to relations

– Return relations– Can be composed together– Often displayed using a tree rather than linearly– Use Greek symbols: σ, p, δ, etc

SETS V.S. BAGSSets: {a,b,c}, {a,d,e,f}, { }, . . .Bags: {a, a, b, c}, {b, b, b, b, b}, . . .

Relational Algebra has two flavors:Set semantics = standard Relational AlgebraBag semantics = extended Relational Algebra

DB systems implement bag semantics (Why?)

RELATIONAL ALGEBRA OPERATORSUnion ∪, intersection ∩, difference -Selection σProjection πCartesian product X, join ⨝(Rename ρ)Duplicate elimination δGrouping and aggregation ɣSorting 𝛕

RA

Extended RA

All operators take in 1 or more relations as inputs and return another relation

UNION AND DIFFERENCE

What do they mean over bags ?

R1 ∪ R2R1 – R2

Only make sense if R1, R2 have the same schema

WHAT ABOUT INTERSECTION ?

Derived operator using minus

Derived using join

R1 ∩ R2 = R1 – (R1 – R2)

R1 ∩ R2 = R1 ⨝ R2

SELECTIONReturns all tuples which satisfy a condition

Examples• σSalary > 40000 (Employee)• σname = “Smith” (Employee)

The condition c can be =, =, combined with AND, OR, NOT

σc(R)

σSalary > 40000 (Employee)

SSN Name Salary1234545 John 200005423341 Smith 600004352342 Fred 50000

SSN Name Salary5423341 Smith 600004352342 Fred 50000

Employee

PROJECTIONEliminates columns

Example: project social-security number and names:• πSSN, Name (Employee) à Answer(SSN, Name)

π A1,…,An (R)

Different semantics over sets or bags! Why?

π Name,Salary (Employee)

SSN Name Salary1234545 John 200005423341 John 600004352342 John 20000

Name SalaryJohn 20000John 60000John 20000

Employee

Name SalaryJohn 20000John 60000

Bag semantics Set semantics

Which is more efficient?

COMPOSING RA OPERATORS

no name zip disease1 p1 98125 flu2 p2 98125 heart3 p3 98120 lung4 p4 98120 heart

Patient

σdisease=‘heart’(Patient)

no name zip disease2 p2 98125 heart4 p4 98120 heart

zip disease98125 flu98125 heart98120 lung98120 heart

πzip,disease(Patient)

πzip,disease(σdisease=‘heart’(Patient))

zip disease98125 heart98120 heart

CARTESIAN PRODUCTEach tuple in R1 with each tuple in R2

Rare in practice; mainly used to express joins

R1 × R2

Name SSNJohn 999999999Tony 777777777

Employee

EmpSSN DepName999999999 Emily777777777 Joe

Dependent

Employee X Dependent

Name SSN EmpSSN DepNameJohn 999999999 999999999 EmilyJohn 999999999 777777777 JoeTony 777777777 999999999 EmilyTony 777777777 777777777 Joe

CROSS-PRODUCT EXAMPLE

NATURAL JOIN

Meaning: R1⨝R2 = PA(sq (R1 × R2))

Where:• Selection sq checks equality of all common attributes (i.e.,

attributes with same names)• Projection PA eliminates duplicate common attributes

R1 ⨝R2

NATURAL JOIN EXAMPLEA BX YX ZY ZZ V

B CZ UV WZ V

A B CX Z UX Z VY Z UY Z VZ V W

R S

R ⨝ S =PABC(sR.B=S.B(R × S))

NATURAL JOIN EXAMPLE 2

age zip disease54 98125 heart20 98120 flu

AnonPatient P Voters V

P V

name age zipAlice 54 98125Bob 20 98120

age zip disease name

54 98125 heart Alice

20 98120 flu Bob

THETA JOINA join that involves a predicate

Here q can be any conditionNo projection in this case!For our voters/patients example:

R1 ⨝q R2 = sq (R1 X R2)

P ⨝ P.zip = V.zip and P.age >= V.age -1 and P.age

EQUIJOINA theta join where q is an equality predicate

By far the most used variant of join in practiceWhat is the relationship with natural join?

R1 ⨝q R2 = sq (R1 × R2)

EQUIJOIN EXAMPLE

age zip disease54 98125 heart20 98120 flu

AnonPatient P Voters V

P P.age=V.age V

name age zipp1 54 98125p2 20 98120

P.age P.zip P.disease V.name V.age V.zip

54 98125 heart p1 54 98125

20 98120 flu p2 20 98120

JOIN SUMMARYTheta-join: R ⨝q S = σq (R × S)

• Join of R and S with a join condition θ• Cross-product followed by selection θ• No projection

Equijoin: R ⨝θ S = σθ (R × S)• Join condition θ consists only of equalities• No projection

Natural join: R ⨝S = πA (σθ (R × S))• Equality on all fields with same name in R and in S• Projection πA drops all redundant attributes

SO WHICH JOIN IS IT ?When we write R ⨝S we usually mean an equijoin, but we often omit the equality predicate when it is clear from the context

MORE JOINSOuter join

• Include tuples with no matches in the output• Use NULL values for missing attributes• Does not eliminate duplicate columns

Variants• Left outer join• Right outer join• Full outer join

OUTER JOIN EXAMPLE

age zip disease54 98125 heart20 98120 flu33 98120 lung

AnonPatient P

P ⋊ J

P.age P.zip P.disease J.job J.age J.zip

54 98125 heart lawyer 54 98125

20 98120 flu cashier 20 98120

33 98120 lung null null null

AnnonJob J

job age ziplawyer 54 98125cashier 20 98120

SOME EXAMPLESSupplier(sno,sname,scity,sstate)Part(pno,pname,psize,pcolor)Supply(sno,pno,qty,price)

Name of supplier of parts with size greater than 10πsname(Supplier ⨝Supply ⨝(σpsize>10 (Part))

Name of supplier of red parts or parts with size greater than 10πsname(Supplier ⨝Supply ⨝(σ psize>10 (Part) ∪ σpcolor=‘red’ (Part) ) )πsname(Supplier ⨝Supply ⨝(σ psize>10 ∨ pcolor=‘red’ (Part) ) )

Can be represented as trees as well

REPRESENTING RA QUERIES AS TREESSupplier(sno,sname,scity,sstate)

Part(pno,pname,psize,pcolor)Supply(sno,pno,qty,price)

πsname(Supplier ⨝Supply ⨝(σpsize>10 (Part))

Part

Supplyσpsize>10

πsname

Answer

Supplier

RELATIONAL ALGEBRA OPERATORSUnion ∪, intersection ∩, difference -Selection σProjection πCartesian product X, join ⨝(Rename ρ)Duplicate elimination δGrouping and aggregation ɣSorting 𝛕

RA

Extended RA

All operators take in 1 or more relations as inputs and return another relation

EXTENDED RA: OPERATORS ON BAGSDuplicate elimination dGrouping g

• Takes in relation and a list of grouping operations (e.g., aggregates). Returns a new relation.

Sorting t

• Takes in a relation, a list of attributes to sort on, and an order. Returns a new relation.

USING EXTENDED RA OPERATORS

SELECT city, sum(quantity)FROM salesGROUP BY cityHAVING count(*) > 100

T1, T2 = temporary tables sales(product, city, quantity)

g city, sum(quantity)→q, count(*) → c

s c > 100

P city, q

Answer

T1(city,q,c)

T2(city,q,c)

TYPICAL PLAN FOR A QUERY (1/2)

R S

join condition

σselection condition

πfields

join condition

…

SELECT-PROJECT-JOINQuery

AnswerSELECT fieldsFROM R, S, …WHERE condition

TYPICAL PLAN FOR A QUERY (1/2)

πfields

ɣfields, sum/count/min/max(fields)

σhaving condition

σwhere condition

join condition

… …

SELECT fieldsFROM R, S, …WHERE conditionGROUP BY fieldsHAVING condition

HOW ABOUT SUBQUERIES?

Supplier(sno,sname,scity,sstate)Part(pno,pname,psize,pcolor)Supply(sno,pno,price)

SELECT Q.snoFROM Supplier QWHERE Q.sstate = ‘WA’ and not exists(SELECT *FROM Supply PWHERE P.sno = Q.sno

and P.price > 100)


and P.price > 100)


Correlation !



and P.price > 100)


De-Correlation

SELECT Q.snoFROM Supplier QWHERE Q.sstate = ‘WA’and Q.sno not in(SELECT P.snoFROM Supply PWHERE P.price > 100)


SELECT Q.snoFROM Supplier QWHERE Q.sstate = ‘WA’and Q.sno not in(SELECT P.snoFROM Supply PWHERE P.price > 100)


(SELECT Q.snoFROM Supplier QWHERE Q.sstate = ‘WA’)

EXCEPT(SELECT P.snoFROM Supply PWHERE P.price > 100)

EXCEPT = set difference


Un-nesting

(SELECT Q.snoFROM Supplier QWHERE Q.sstate = ‘WA’)

EXCEPT(SELECT P.snoFROM Supply PWHERE P.price > 100)


Supply

σsstate=‘WA’

Supplier

σPrice > 100

−Finally…

πsnoπsno


SUMMARY OF RA AND SQLSQL = a declarative language where we say what data we want to retrieveRA = an algebra where we say how we want to retrieve the dataTheorem: SQL and RA can express exactly the same class of queries

RDBMS translate SQL à RA, then optimize RA

SUMMARY OF RA AND SQLSQL (and RA) cannot express ALL queries that we could write in, say, JavaExample:

• Parent(p,c): find all descendants of ‘Alice’• No RA query can compute this!• This is called a recursive query

Next lecture: Datalog is an extension that can compute recursive queries

relational - university of washington · 2018. 1. 22. · relational algebra operators union ∪,...

Documents