defer and stream directives in graphql · liliana matos, director, front-end engineering @ 1stdibs...
TRANSCRIPT
![Page 1: Defer and Stream Directives in GraphQL · Liliana Matos, Director, Front-End Engineering @ 1stdibs Rob Richard, Senior Director, Front-End Engineering @ 1stdibs Defer and Stream Directives](https://reader035.vdocument.in/reader035/viewer/2022070917/5fb7f13d8cff9a74a10c98e5/html5/thumbnails/1.jpg)
Liliana Matos, Director, Front-End Engineering @ 1stdibs Rob Richard, Senior Director, Front-End Engineering @ 1stdibs
Defer and Stream Directives in GraphQLImprove Latency with Incremental Delivery
![Page 2: Defer and Stream Directives in GraphQL · Liliana Matos, Director, Front-End Engineering @ 1stdibs Rob Richard, Senior Director, Front-End Engineering @ 1stdibs Defer and Stream Directives](https://reader035.vdocument.in/reader035/viewer/2022070917/5fb7f13d8cff9a74a10c98e5/html5/thumbnails/2.jpg)
About
• Online marketplace for luxury items across multiple verticals
• Front-End stack: Node, GraphQL, React, and Relay
• Offices in New York, Vilnius, Lithuania, and Bangalore, India
![Page 3: Defer and Stream Directives in GraphQL · Liliana Matos, Director, Front-End Engineering @ 1stdibs Rob Richard, Senior Director, Front-End Engineering @ 1stdibs Defer and Stream Directives](https://reader035.vdocument.in/reader035/viewer/2022070917/5fb7f13d8cff9a74a10c98e5/html5/thumbnails/3.jpg)
What are @defer and @stream?
• @defer and @stream are proposed directives to the GraphQL Specification to support incremental delivery for state-less queries
• Championing since January 2020
• In collaboration with GraphQL Working Group
• In this talk, we will discuss:
• Motivation
• Specification proposal overview
• Code Examples
• Reference implementation in GraphQL.js
• Open-source contribution
• Best practices
query TalksQuery { talks(first: 6) @stream ( label: "talkStream", initialCount: 3 ) { name ...TalkComments @defer(label: "talkCommentsDefer") } }
fragment TalkComments on Talk { comments { body } }
![Page 4: Defer and Stream Directives in GraphQL · Liliana Matos, Director, Front-End Engineering @ 1stdibs Rob Richard, Senior Director, Front-End Engineering @ 1stdibs Defer and Stream Directives](https://reader035.vdocument.in/reader035/viewer/2022070917/5fb7f13d8cff9a74a10c98e5/html5/thumbnails/4.jpg)
Why @defer and @stream?
• Large datasets may suffer from latency
• All requested data may not be of equal importance
• Current options for applications to prioritize data, such as query splitting and pre-fetching, come with undesirable trade-offs
• @defer and @stream would allow GraphQL clients to communicate priority of requested data to the server without undesirable trade-offs
![Page 5: Defer and Stream Directives in GraphQL · Liliana Matos, Director, Front-End Engineering @ 1stdibs Rob Richard, Senior Director, Front-End Engineering @ 1stdibs Defer and Stream Directives](https://reader035.vdocument.in/reader035/viewer/2022070917/5fb7f13d8cff9a74a10c98e5/html5/thumbnails/5.jpg)
Query Splitting
• Fetch expensive/non-essential fields in a separate query after initial query
• Trade-offs:
• Increased latency for lower priority fields
• Client resource contention
• Increased server cost
/** Original Query */query SpeakerQuery($speakerId: String!) { speaker(speakerId: $speakerId) { name ...SpeakerPicture }}
fragment SpeakerPicture on Speaker { picture { height width url } }
/** Split Queries */query SpeakerInitialQuery($speakerId: String!) { speaker(speakerId: $speakerId) { name }}
query SpeakerFollowUpQuery($speakerId: String!) { speaker(speakerId: $speakerId) { ...SpeakerPicture }}
![Page 6: Defer and Stream Directives in GraphQL · Liliana Matos, Director, Front-End Engineering @ 1stdibs Rob Richard, Senior Director, Front-End Engineering @ 1stdibs Defer and Stream Directives](https://reader035.vdocument.in/reader035/viewer/2022070917/5fb7f13d8cff9a74a10c98e5/html5/thumbnails/6.jpg)
Pre-fetching
• Optimistically fetching data based on a prediction that a user will execute an action
• Trade-offs:
• Increased server cost due to incorrect predictions
![Page 7: Defer and Stream Directives in GraphQL · Liliana Matos, Director, Front-End Engineering @ 1stdibs Rob Richard, Senior Director, Front-End Engineering @ 1stdibs Defer and Stream Directives](https://reader035.vdocument.in/reader035/viewer/2022070917/5fb7f13d8cff9a74a10c98e5/html5/thumbnails/7.jpg)
https://www.apollographql.com/blog/introducing-defer-in-apollo-server-f6797c4e9d6e/
![Page 8: Defer and Stream Directives in GraphQL · Liliana Matos, Director, Front-End Engineering @ 1stdibs Rob Richard, Senior Director, Front-End Engineering @ 1stdibs Defer and Stream Directives](https://reader035.vdocument.in/reader035/viewer/2022070917/5fb7f13d8cff9a74a10c98e5/html5/thumbnails/8.jpg)
What about Subscriptions?
• Intention is for real-time and long connections
• @defer and @stream, intention is to lower latency for short-lived connections
![Page 9: Defer and Stream Directives in GraphQL · Liliana Matos, Director, Front-End Engineering @ 1stdibs Rob Richard, Senior Director, Front-End Engineering @ 1stdibs Defer and Stream Directives](https://reader035.vdocument.in/reader035/viewer/2022070917/5fb7f13d8cff9a74a10c98e5/html5/thumbnails/9.jpg)
Specification Proposal for @defer and @stream
![Page 10: Defer and Stream Directives in GraphQL · Liliana Matos, Director, Front-End Engineering @ 1stdibs Rob Richard, Senior Director, Front-End Engineering @ 1stdibs Defer and Stream Directives](https://reader035.vdocument.in/reader035/viewer/2022070917/5fb7f13d8cff9a74a10c98e5/html5/thumbnails/10.jpg)
@defer
• The @defer directive may be specified on a fragment spread.
• if: Boolean
• When true fragment may be deferred, if omitted defaults to true.
• label: String
• A unique label across all @defer and @stream directives in an operation.
directive @defer(if: Boolean, label: String) on FRAGMENT_SPREAD
![Page 11: Defer and Stream Directives in GraphQL · Liliana Matos, Director, Front-End Engineering @ 1stdibs Rob Richard, Senior Director, Front-End Engineering @ 1stdibs Defer and Stream Directives](https://reader035.vdocument.in/reader035/viewer/2022070917/5fb7f13d8cff9a74a10c98e5/html5/thumbnails/11.jpg)
@deferExample
query SpeakerQuery($speakerId: String!) { speaker(speakerId: $speakerId) { name ...SpeakerPicture @defer(label: “speakerPictureDefer”) }}
fragment SpeakerPicture on Speaker { picture { height width url } }
// Response Payloads
// Payload 1
{ "data": { "speaker": { "name": "Jesse Rosenberger" } }, "hasNext": true}
// Payload 2{
"label": "speakerPictureDefer", "path": ["speaker"], "data": { "picture": { "height": 200, "width": 200, "url": "jesse-headshot.jpg" } }, "hasNext": false}
![Page 12: Defer and Stream Directives in GraphQL · Liliana Matos, Director, Front-End Engineering @ 1stdibs Rob Richard, Senior Director, Front-End Engineering @ 1stdibs Defer and Stream Directives](https://reader035.vdocument.in/reader035/viewer/2022070917/5fb7f13d8cff9a74a10c98e5/html5/thumbnails/12.jpg)
@stream
• The @stream directive may be provided for a field of List
• if: Boolean
• When true fragment may be deferred, if omitted defaults to true.
• label: String
• A unique label across all @defer and @stream directives in an operation.
• initialCount: Int
• The number of list items the server should return as part of the initial response.
directive @stream(if: Boolean, label: String, initialCount: Int) on FIELD
![Page 13: Defer and Stream Directives in GraphQL · Liliana Matos, Director, Front-End Engineering @ 1stdibs Rob Richard, Senior Director, Front-End Engineering @ 1stdibs Defer and Stream Directives](https://reader035.vdocument.in/reader035/viewer/2022070917/5fb7f13d8cff9a74a10c98e5/html5/thumbnails/13.jpg)
@streamExample
query SpeakersQuery { speakers(first: 3) @stream(label: “speakerStream", initialCount: 1) { name }}
// Response Payloads
// Payload 1{ "data": { "speakers": [ { "name": "Jesse Rosenberger" } ] }, "hasNext": true}
// Payload 2{ "label": "speakerStream", “path": ["speakers", 1], "data": { "name": "Liliana Matos" }, "hasNext": true}
// Payload 3{ "label": "speakerStream", "path": ["speakers", 2], "data": { "name": "Rob Richard" }, "hasNext": false}
![Page 14: Defer and Stream Directives in GraphQL · Liliana Matos, Director, Front-End Engineering @ 1stdibs Rob Richard, Senior Director, Front-End Engineering @ 1stdibs Defer and Stream Directives](https://reader035.vdocument.in/reader035/viewer/2022070917/5fb7f13d8cff9a74a10c98e5/html5/thumbnails/14.jpg)
Response FormatOverview
• When an operation contains @defer or @stream directives, the GraphQL execution will return multiple payloads
• The first payload is the same shape as a standard GraphQL response
• Any fields that were only requested on a fragment that is deferred will not be present in this payload
• Any list fields that are streamed will only contain the initial list items
![Page 15: Defer and Stream Directives in GraphQL · Liliana Matos, Director, Front-End Engineering @ 1stdibs Rob Richard, Senior Director, Front-End Engineering @ 1stdibs Defer and Stream Directives](https://reader035.vdocument.in/reader035/viewer/2022070917/5fb7f13d8cff9a74a10c98e5/html5/thumbnails/15.jpg)
Response FormatDetails
• label — The string that was passed to the label argument of the @defer or @stream directive that corresponds to this results
• path — A list of keys from the root of the response to the insertion point
• hasNext — A boolean that is present and true when there are more payloads that will be sent for this operation.
• data — The data that is being delivered incrementally.
• errors — An array of errors that occurred while executing deferred or streamed selection set
• extensions — For implementors to extend the protocol
![Page 16: Defer and Stream Directives in GraphQL · Liliana Matos, Director, Front-End Engineering @ 1stdibs Rob Richard, Senior Director, Front-End Engineering @ 1stdibs Defer and Stream Directives](https://reader035.vdocument.in/reader035/viewer/2022070917/5fb7f13d8cff9a74a10c98e5/html5/thumbnails/16.jpg)
Response FormatExample
query SpeakersQuery { speakers(first: 2) @stream(label: "speakerStream", initialCount: 1) { name ...SpeakerPicture @defer(label: "speakerPictureDefer") }}
fragment SpeakerPicture on Speaker { picture { url } }
// Response Payloads
// Payload 1{ "data": { "speakers": [ { "name": "Jesse Rosenberger" } ] }, "hasNext": true}
// Payload 2{ "label": "SpeakerPictureDefer", "path": ["speakers", 0, "picture"], "data": { "url": "jesse-headshot.jpg" }, "hasNext": true}
// Payload 3{ "label": "SpeakerStream", "path": ["speakers", 1], "data": { "name": "Liliana Matos" }, "hasNext": true}
// Payload 4{ "label": "SpeakerPictureDefer", "path": ["speakers", 1, "picture"], "data": { "url": "liliana-headshot.jpg" }, "hasNext": false}
![Page 17: Defer and Stream Directives in GraphQL · Liliana Matos, Director, Front-End Engineering @ 1stdibs Rob Richard, Senior Director, Front-End Engineering @ 1stdibs Defer and Stream Directives](https://reader035.vdocument.in/reader035/viewer/2022070917/5fb7f13d8cff9a74a10c98e5/html5/thumbnails/17.jpg)
How to use @defer and @stream in your GraphQL Server
• @defer - existing resolvers will work effectively
• @stream - need to consider how resolvers return data
![Page 18: Defer and Stream Directives in GraphQL · Liliana Matos, Director, Front-End Engineering @ 1stdibs Rob Richard, Senior Director, Front-End Engineering @ 1stdibs Defer and Stream Directives](https://reader035.vdocument.in/reader035/viewer/2022070917/5fb7f13d8cff9a74a10c98e5/html5/thumbnails/18.jpg)
Execution
fragment SpeakerPicture on Speaker { picture { height width url } }
{ "data": { "speaker": { "name": "Jesse Rosenberger" } }, "hasNext": true}
query SpeakerQuery($speakerId: String!) { speaker(speakerId: $speakerId) { name ...SpeakerPicture @defer(label: “speakerPictureDefer”) }}
{
"label": "speakerPictureDefer", "path": ["speaker"], "data": { "picture": { "height": 200, "width": 200, "url": "jesse-headshot.jpg" } }, "hasNext": false}
Fork execution to dispatcher
Initial Payload
Subsequent Payload
![Page 19: Defer and Stream Directives in GraphQL · Liliana Matos, Director, Front-End Engineering @ 1stdibs Rob Richard, Senior Director, Front-End Engineering @ 1stdibs Defer and Stream Directives](https://reader035.vdocument.in/reader035/viewer/2022070917/5fb7f13d8cff9a74a10c98e5/html5/thumbnails/19.jpg)
How to use @stream in your GraphQL Server
• Any List field can use the @stream directive
• What you return from your resolver matters
![Page 20: Defer and Stream Directives in GraphQL · Liliana Matos, Director, Front-End Engineering @ 1stdibs Rob Richard, Senior Director, Front-End Engineering @ 1stdibs Defer and Stream Directives](https://reader035.vdocument.in/reader035/viewer/2022070917/5fb7f13d8cff9a74a10c98e5/html5/thumbnails/20.jpg)
List return types in GraphQL-JS• GraphQL-JS supports returning several different data types in List resolvers.
• Array<T>, any Iterable, Promise<Iterable<T>>
• GraphQL engine will get all results at once
• Initial payload will be held up by this resolver
• Subsequent payloads will be sent immediately after
const resolvers = { Query: { items: async function (_, { filters }): Promise<Array<Item>> { const items = await api.getFilteredItems({ filters }); return items; }, },};
![Page 21: Defer and Stream Directives in GraphQL · Liliana Matos, Director, Front-End Engineering @ 1stdibs Rob Richard, Senior Director, Front-End Engineering @ 1stdibs Defer and Stream Directives](https://reader035.vdocument.in/reader035/viewer/2022070917/5fb7f13d8cff9a74a10c98e5/html5/thumbnails/21.jpg)
List return types in GraphQL-JS
• Array<Promise<T>>
• GraphQL engine will start waiting for all results
• Initial payload will be sent as soon as the "initialCount" values are ready
• Subsequent payloads will be sent as each promise resolves
• Requires knowing how many results there will be before the resolver returns
![Page 22: Defer and Stream Directives in GraphQL · Liliana Matos, Director, Front-End Engineering @ 1stdibs Rob Richard, Senior Director, Front-End Engineering @ 1stdibs Defer and Stream Directives](https://reader035.vdocument.in/reader035/viewer/2022070917/5fb7f13d8cff9a74a10c98e5/html5/thumbnails/22.jpg)
Returning Array<Promise<T>>
const resolvers = { Query: { items: async function (_, { filters }): Array<Promise<<Item>> { const itemIds = await api.filterItems({ filters }); return itemIds.map(async itemId => await api.getItemById(itemId)); }, },};
![Page 23: Defer and Stream Directives in GraphQL · Liliana Matos, Director, Front-End Engineering @ 1stdibs Rob Richard, Senior Director, Front-End Engineering @ 1stdibs Defer and Stream Directives](https://reader035.vdocument.in/reader035/viewer/2022070917/5fb7f13d8cff9a74a10c98e5/html5/thumbnails/23.jpg)
List return types in GraphQL-JS
• AsyncIterable, Async Generator function
• GraphQL engine will yield each result from the iterable
• Initial payload will be sent as soon as the "initialCount" values are ready
• Subsequent payloads will be sent as each new value is yielded
• Can determine asynchronously if the list is completed
![Page 24: Defer and Stream Directives in GraphQL · Liliana Matos, Director, Front-End Engineering @ 1stdibs Rob Richard, Senior Director, Front-End Engineering @ 1stdibs Defer and Stream Directives](https://reader035.vdocument.in/reader035/viewer/2022070917/5fb7f13d8cff9a74a10c98e5/html5/thumbnails/24.jpg)
Async Generator Function Resolverconst resolvers = { Query: { users: async function* (): AsyncIterable<User> { const db = new Database(); while (true) { // select one document const result = await db.getNext();
// end iteration if there are no documents returned if (!result) { break; }
yield result; }
return; }, },};
![Page 25: Defer and Stream Directives in GraphQL · Liliana Matos, Director, Front-End Engineering @ 1stdibs Rob Richard, Senior Director, Front-End Engineering @ 1stdibs Defer and Stream Directives](https://reader035.vdocument.in/reader035/viewer/2022070917/5fb7f13d8cff9a74a10c98e5/html5/thumbnails/25.jpg)
Server - Client Communication
• No websockets or any other stateful connection mechanism
• Works with common infrastructure and old browsers
• Spec is transport-agnostic, so you could use websockets
![Page 26: Defer and Stream Directives in GraphQL · Liliana Matos, Director, Front-End Engineering @ 1stdibs Rob Richard, Senior Director, Front-End Engineering @ 1stdibs Defer and Stream Directives](https://reader035.vdocument.in/reader035/viewer/2022070917/5fb7f13d8cff9a74a10c98e5/html5/thumbnails/26.jpg)
"Chunked transfer encoding (CTE) is a mechanism in which the encoder sends data to the player in a series of chunks. The player doesn’t have to wait until the complete segment is available"
transfer-encoding: chunked
![Page 27: Defer and Stream Directives in GraphQL · Liliana Matos, Director, Front-End Engineering @ 1stdibs Rob Richard, Senior Director, Front-End Engineering @ 1stdibs Defer and Stream Directives](https://reader035.vdocument.in/reader035/viewer/2022070917/5fb7f13d8cff9a74a10c98e5/html5/thumbnails/27.jpg)
Multipart HTTP
• Standard for encoding multiple payloads in a single HTTP request response
• Used for File Uploads to attach binary file data to form requests
• Used for Emails to add attachments to email body
• GraphQL Response can be encoded as multipart data of multiple JSON payloads
![Page 28: Defer and Stream Directives in GraphQL · Liliana Matos, Director, Front-End Engineering @ 1stdibs Rob Richard, Senior Director, Front-End Engineering @ 1stdibs Defer and Stream Directives](https://reader035.vdocument.in/reader035/viewer/2022070917/5fb7f13d8cff9a74a10c98e5/html5/thumbnails/28.jpg)
Multipart HTTP---Content-Type: application/jsonContent-Length: 590{ "data": { "talks": [{ "title": "Opening Keynote", "time": "10:30-11:00", "speaker": { "name": "Jesse Rosenberger" }, }, ...] }}---Content-Type: application/jsonContent-Length: 140{ "path": ["talks", 0, "comments"], "data": [{ "body": "Loved this!" }]}
query ConferenceQuery { talks { title time speaker { name } ...commentsFragment @defer }}
fragment commentsFragment on Talk { body}
![Page 29: Defer and Stream Directives in GraphQL · Liliana Matos, Director, Front-End Engineering @ 1stdibs Rob Richard, Senior Director, Front-End Engineering @ 1stdibs Defer and Stream Directives](https://reader035.vdocument.in/reader035/viewer/2022070917/5fb7f13d8cff9a74a10c98e5/html5/thumbnails/29.jpg)
![Page 30: Defer and Stream Directives in GraphQL · Liliana Matos, Director, Front-End Engineering @ 1stdibs Rob Richard, Senior Director, Front-End Engineering @ 1stdibs Defer and Stream Directives](https://reader035.vdocument.in/reader035/viewer/2022070917/5fb7f13d8cff9a74a10c98e5/html5/thumbnails/30.jpg)
GraphQL over HTTP
• GraphQL over HTTP is a proposed specification to define how GraphQL should be served over HTTP
• We have an RFC to add incremental delivery to this spec using chunked encoding and multipart responses
• https://github.com/graphql/graphql-over-http/pull/124
![Page 31: Defer and Stream Directives in GraphQL · Liliana Matos, Director, Front-End Engineering @ 1stdibs Rob Richard, Senior Director, Front-End Engineering @ 1stdibs Defer and Stream Directives](https://reader035.vdocument.in/reader035/viewer/2022070917/5fb7f13d8cff9a74a10c98e5/html5/thumbnails/31.jpg)
Server Codefunction sendPartialResponse( response: $Response, result: AsyncExecutionResult,): void { const json = JSON.stringify(result, null, 2); const chunk = Buffer.from(json, 'utf8'); const data = [ '', '---', 'Content-Type: application/json; charset=utf-8', 'Content-Length: ' + String(chunk.length), '', chunk, '', ].join('\r\n'); response.write(data);}// Close connectionresponse.end();
![Page 32: Defer and Stream Directives in GraphQL · Liliana Matos, Director, Front-End Engineering @ 1stdibs Rob Richard, Senior Director, Front-End Engineering @ 1stdibs Defer and Stream Directives](https://reader035.vdocument.in/reader035/viewer/2022070917/5fb7f13d8cff9a74a10c98e5/html5/thumbnails/32.jpg)
Client Code - Fetch
const response = await fetch(url, { method, headers, body })
// Don't call response.json()! // That waits for the whole response to finish loading.// const json = await response.json()
const reader = response.body.getReader();while (true) { const { value, done } = await reader.read(); if (done) return; handleChunk(value);}
![Page 33: Defer and Stream Directives in GraphQL · Liliana Matos, Director, Front-End Engineering @ 1stdibs Rob Richard, Senior Director, Front-End Engineering @ 1stdibs Defer and Stream Directives](https://reader035.vdocument.in/reader035/viewer/2022070917/5fb7f13d8cff9a74a10c98e5/html5/thumbnails/33.jpg)
Client Code - XMLHttpRequest
// This works in Internet Explorer 7!const xhr = new XMLHttpRequest();let index = 0;xhr.open(method, url);xhr.addEventListener("readystatechange", function onReadyStateChange() { const chunk = xhr.response.substr(index); handleChunk(chunk); index = xhr.responseText.length;});xhr.send(body);
![Page 34: Defer and Stream Directives in GraphQL · Liliana Matos, Director, Front-End Engineering @ 1stdibs Rob Richard, Senior Director, Front-End Engineering @ 1stdibs Defer and Stream Directives](https://reader035.vdocument.in/reader035/viewer/2022070917/5fb7f13d8cff9a74a10c98e5/html5/thumbnails/34.jpg)
Open Source Implementationfetch-multipart-graphql
• https://github.com/relay-tools/fetch-multipart-graphql
![Page 35: Defer and Stream Directives in GraphQL · Liliana Matos, Director, Front-End Engineering @ 1stdibs Rob Richard, Senior Director, Front-End Engineering @ 1stdibs Defer and Stream Directives](https://reader035.vdocument.in/reader035/viewer/2022070917/5fb7f13d8cff9a74a10c98e5/html5/thumbnails/35.jpg)
References
• Spec RFC: https://github.com/graphql/graphql-spec/blob/master/rfcs/DeferStream.md
• Spec Proposal: https://github.com/graphql/graphql-spec/pull/742
• GraphQL-JS: https://github.com/graphql/graphql-js/pull/2319
• express-graphql: https://github.com/graphql/express-graphql/pull/583
• fetch-multipart-graphql: https://github.com/relay-tools/fetch-multipart-graphql
• GraphQL over HTTP RFC: https://github.com/graphql/graphql-over-http/pull/124
![Page 36: Defer and Stream Directives in GraphQL · Liliana Matos, Director, Front-End Engineering @ 1stdibs Rob Richard, Senior Director, Front-End Engineering @ 1stdibs Defer and Stream Directives](https://reader035.vdocument.in/reader035/viewer/2022070917/5fb7f13d8cff9a74a10c98e5/html5/thumbnails/36.jpg)