why is materialize? - github pages · 2019. 9. 10. · planning materialize. timely dataflow a...
TRANSCRIPT
view
view
Why is Materialize?
OLTPmoderate OLTP load
view
view
view
increased
DenormalizationCaches
Caches
CachesOssification
What is Materialize?“incremental view maintenance offload”
Incremental View Maintenance Read-only
ChangeLog
SQL ClientSQL ClientSQL Client
Control plane
Data plane
ChangeLogs
ChangeLogs
ChangeLogs
CREATE VIEW v AS ...
SELECT COUNT(*) FROM ...
SELECT COUNT(*) FROM v;
The Materialize Stack
Timely Dataflow
Differential Dataflow
Indices AST Render
MaterializePlanning
Timely Dataflow
a dataflow
op op
op
op
data exchange
another dataflow
op
op
yet another dataflow
} workers
yet another dataflowyet another dataflowyet another dataflow
Differential Dataflow
a dataflow
GroupJoinMap
MapInput
Streams of “updates”: (data, time, diff)
e.g. String
e.g. u64
e.g. i64Operators produce “correct” output stream.
Differential Dataflow
a dataflow
GroupJoinMap
MapInput
Arrange : Stream -> Index + Stream(by key) (of indices)
Advanced Topic 1: Arrangements
Arrange
Differential Dataflow
a dataflow
GroupJoinArrange
ArrangeInput
Advanced Topic 1: Arrangements
: Stream -> Index + Stream(by key) (of indices)
Arrange
Differential Dataflow
a dataflow
GroupJoinArrangeInput
Advanced Topic 1: Arrangements
: Stream -> Index + Stream(by key) (of indices)
Group
Differential Dataflow
a dataflow
GroupJoinArrange
Advanced Topic 1: Arrangements
Group
Input
Differential Dataflow
a dataflow
GroupJoinArrange
Advanced Topic 1: Arrangements
Group
another dataflowImport Group
yet another dataflowImport Distinct
yet another dataflowImport Count
Input
Differential Dataflow
a dataflow
Group
JoinInput
Input
Advanced Topic 2: Delta Queries
JoinInput
materialization
Differential Dataflow
a dataflow
Group
dJoinInput
Input
Advanced Topic 2: Delta Queries
dJoinInput
Differential Dataflow
a dataflow
Group
Advanced Topic 2: Delta Queries
dJoinInput
Input Input
dJoin
dJoinInput
Input Input
dJoin
dJoinInput
Input Input
dJoin
Differential Dataflow
a dataflow
Group
dJoinInput
Input
Advanced Topic 2: Delta Queries
Input
dJoin
dJoinInput
Input Input
dJoin
dJoinInput
Input Input
dJoin
Differential Dataflow
a dataflow
Advanced Topic 3: Iteration
Works great!
The Materialize Stack
Timely
Differential
Indices AST Render
MaterializePlanning
The Materialize Stack
Indices AST Render
MaterializePlanning
Differential
Timely
.io@frankmcsherry