event-driven api strategies · 2019-12-27 · • how can you design, deploy and manage your apis?...
TRANSCRIPT
Hospitality
Hospitality
Luis WeirCode Monsters11th December, 2019
Event-driven API strategiesfrom WebHooks to GraphQL Subscriptions
© 2019 Oracle. All rights reserved.1
Hospitality@luisw19
About Me
2
Luis [email protected] of Technology & Developer
@luisw19
www.soa4u.co.uk
• The API lifecycle• What Is API-Led — An Architectural Approach• A brief look at the evolution of interface protocols
leading to modern APIs• The 7 Deadly Sins of API Design• Setting the vision, strategy and direction — the
CTO’s role• How can you design, deploy and manage your
APIs?• The Spotify's Engineering Culture. My interpretation
and summary
• A comparison of API Gateways communication styles
• Is BPM Dead, Long Live Microservices?• Five Minutes with Luis Weir• 2nd vs 3rd Generation API Platforms - A
Comprehensive Comparison• Podcast: Are Microservices and APIs Becoming
SOA 2.0?• 3rd-Generation API Management: From Proxies to
Micro-Gateways
apiplatform.cloud/Released Q2 2018
tinyurl.com/apim15Released in Set. 2015
tinyurl.com/soagov13Released in Set. 2013
BO
OK
SA
RT
ICLE
Sluisw19
tinyurl.com/eapim18 July 2019
Foreword byZednek “Z” Nemec
Hospitality@luisw19
This is an opinionated presentation expressing my own views.
Hospitality
Sync vs Async APIs1
2
3
Asynchronous on the web?Trendy Async API StylesDemo
4 © 2019 Oracle. All rights reserved.
Agenda
4 Conclusions & Q & A
Hospitality@luisw19
Synchronous communication
© 2019 Oracle. All rights reserved.5
Definition of Synchronous
“existing or occurring at the same time”
Hospitality@luisw19
Synchronous APIs
© 2019 Oracle. All rights reserved.6
• Single http thread for processing a requestand a response
• Synchronous can be very good for many use cases (instant response needed) but not so good when response occurs in a different time (event-driven)
Req Res Req Res Req Res Req Res
Consuming Applications
API Exposure
API
Service• Regular pulling as common practice to obtain updates
Hospitality@luisw19
Asynchronous communication
© 2019 Oracle. All rights reserved.7
Definition of Asynchronous
“not existing or occurring at the same time”
Hospitality@luisw19
Asynchronous communication
© 2019 Oracle. All rights reserved.8
Have you ever thought about the name in the cup from a programmers point of view?
Hospitality@luisw19
Asynchronous communication
© 2019 Oracle. All rights reserved.9
Have you ever thought about the name in the cup from a programmers point of view?
The name actually a unique identifier
it’s there to allow non-blocking processing
and asynchronous delivery
Hospitality@luisw19
Think about it….
© 2019 Oracle. All rights reserved.10
1) an order is placed under one’s name. Then you get out of the queue.[a command with an UUID]
Hospitality@luisw19
Think about it….
© 2019 Oracle. All rights reserved.11
2) The barrister makes the coffee whilst other orders are placed[non-blocking command executed]
Hospitality@luisw19
Think about it….
© 2019 Oracle. All rights reserved.12
3) The order is delivered by calling one’s name again[a push event is made to fulfilled request]
Hospitality
Sync vs Async APIs1
2
3
Asynchronous on the web?Trendy Async API StylesDemo
13 © 2019 Oracle. All rights reserved.
Agenda
4 Conclusions & Q & A
Hospitality@luisw19
When talking about Web APIs there are some key considerations to be wary of
© 2019 Oracle. All rights reserved.14
HTTP(S), DNS, SSH, FTP, SMTP,…
TLS, SSL, MIME, XDR, LFP, AFP, …
PAP, ASP, ADS, NetBios, SOCKS,…
TCP, UDP, UDP-L, µTP, ATP, IL, …
IP, IPsec, IPX, ICMP, IGMP, DDP, …
ARP, Ethernet, 802.x, BT, MPLS, …
Wire, Base-T, GSM, G.x, BT, IrDA, …
6. Presentation
5. Session
4. Transport
3. Network
2. Data Link
1. Physical
• HTTP(s) as the main protocol supported in API Gateways and other layer 7 appliances e.g. (e.g. load balancers, web app firewalls, CDNs)
• Majority of corporate firewalls (cloud and on-prem) configured to block non-http(s) traffic
7. Application
• JSON currently most popular data format in modern APIs -event though for async comms Avro, Protobuf to name a few picking up fast.
• HTTP/2 adoption rapidly increasing but still early days in the context of API related tech.
Network Access
Internet
Transport
Application
OSI TCP
Hospitality@luisw19
WS-
Ad.
P S
TCP
HTTP/1.1**
P S
T T
Async technology landscape
P S P S
IP
Ethernet
xDSLWi-Fi 1xBase-T2-5G Fibre / OTN
LLC / MACVarious
A Arch. Pattern / Technique
Rabbit MQ
Active MQ
GraphQLT
P
S
Protocol / Specification
Industry Standard
Tool
D Draft or Recommendation
T T
UDP
rSocket
P
Aeron
P
P S P P
6. Presentation
5. Session
7. Application
4. Transport
3. Network2. Data Link1. Physical
HTTP/2WebSockets
P S P S
WebSocketsP S P
LongPolling
BOSHP PP S PT
P D
P
© 2019 Oracle. All rights reserved.15 ** no natively asynchronous
P
P N
N New / recent
NA
P DCom
et
A
SSEP S T
WebSub
Web-hooks
AA D
Hospitality@luisw19
TCP
Async technology landscape
P S P S
IP
Ethernet
xDSLWi-Fi 1xBase-T2-5G Fibre / OTN
LLC / MACVarious
A Arch. Pattern / Technique
Rabbit MQ
Active MQ
GraphQLT
P
S
Protocol / Specification
Industry Standard
Tool
D Draft or Recommendation
T T
UDP
rSocket
P
Aeron
P
P S P P
© 2019 Oracle. All rights reserved.16 ** no natively asynchronous
N New / recent
6. Presentation
5. Session
7. Application
4. Transport
3. Network2. Data Link1. Physical
WS-
Ad.
P S
HTTP/1.1**
P S
T T
HTTP/2WebSockets
P S P S
WebSocketsP S P
LongPolling
BOSHP PP S PT
P D
P
P
P N
NA
P DCom
et
A
SSEP S T
WebSub
Web-hooks
AA D
Hospitality@luisw19
rSocket
STOMP
Vulcain / Mercure
SSEWebSub
MQTT
XMPP
SOAP
REST
Comet
BOSH
GraphQLWebSockets
gRPC
Webhooks
Async Landscape Trends on Stack Overflow
© 2019 Oracle. All rights reserved.17
https://insights.stackoverflow.com/trends?tags=xmpp%2Cwebhooks%2Cwebsocket%2Cmqtt%2Cgraphql%2Cgrpc%2Csoap%2Crest
% o
f Sta
ck O
verf
low
que
stio
ns th
at m
onth
Hospitality@luisw19
Trendy Asynchronous API Styles
© 2019 Oracle. All rights reserved.18
Webhooks
WebSockets
GraphQL
gRPC
Subscriptions
Long Polling
Hospitality@luisw19
Terminology
Consuming Application A consuming application of any type unless otherwise explicitly indicated (e.g. browser based app, server side app, etc).
API ExposureAny layer from which a service interface can be accessed (e.g. API gateway, ingress, HTTP proxy, load balancer, web firewall, etc)
Service A (business or technical) capability encapsulated in a service and that is accessible via a programming interface (API).
Hospitality@luisw19
Long Polling APIs
© 2019 Oracle. All rights reserved.20
• Server can response with header “Transfer-Encoding: chunked” to avoid creating a new connexions
• As new events occur (e.g. changes in resource) responses are sent back.
Message HubStorage
API
Service
API ExposureLong-lived HTTP Connexion
Consuming Application(e.g. Browser App)
API
HTTP 1.1 [Verb] http(s)://server/resource(connexion opens)
Long-livedSessionHandler
Websocket Client
• Consuming application makes standard HTTP 1.1 request call however the server holds the response (connexion remains open)
• Long-polling/streaming endpoints can be designed/defined with OAS 2/3 by defining the relevant response headers (e.g. Transfer-Encoding)
IDLR
esponse
Response
Response
OAS 2/3
(connexion closes)
Hospitality@luisw19
Long Pooling Samples
© 2019 Oracle. All rights reserved.21
• Twitter Streaming API: https://developer.twitter.com/en/docs/tweets/filter-realtime/overview“Get only the Tweets you need by using advanced filtering tools with the realtime streaming API. BC”
• SalesForce Streaming API: https://developer.salesforce.com/docs/atlas.en-us.api_streaming.meta/api_streaming/intro_stream.htm“Streaming API enables streaming of events using push technology and provides a subscription mechanism for receiving events in near real time. The Streaming API subscription mechanism supports multiple types of events, including PushTopic events, generic events, platform events, and Change Data Capture events”
Hospitality@luisw19
Webhook APIs
© 2019 Oracle. All rights reserved.22
Consuming Application(e.g. Server App)
API
API Exposure
Message Hub
HTTP v1.1 [POST|PUT|DEL] /{resource}/subscribe{call-back endpoint}
API
ResourceEvent
Listener
Change of state
event
Call-back endpoint
HTTP [verb] callback url
Message Hub
Call-back Commands
• Callback URL registered via subscription endpoint
• Process execution fully detached from http thread
• Message request and response in separate HTTP calls
• Relevant events are pushed to the client as reverse http calls
HTTP 1.1 [POST] callback url
SubscribersDatabase
Get Subscribers
New Subscriber
2xx2xx
IDLOAS 3.0
API
Subscriber Capability
Call-back Capability
• Interface can be designed/defined with OAS 3.0 (callbacks)*
(akas REST-Hooks)
* AsyncAPISpec working on samples
* supermodel.io can be used as as domain (ubiquitous) modelling language
Hospitality@luisw19
Webhook API Samples
© 2019 Oracle. All rights reserved.23
• GitHub: https://developer.github.com/webhooks/“Webhooks allow you to build or set up integrations, such as GitHub Apps or OAuth Apps, which subscribe to certain events on GitHub.com. When one of those events is triggered, we'll send a HTTP POST payload to the webhook's configured URL. Webhooks can be used to update an external issue tracker, trigger CI builds, update a backup mirror, or even deploy to your production server. You're only limited by your imagination.”
• W3C: https://w3c.github.io/w3c-api/webhooks“Webhooks allow you to subscribe to certain events happening at W3C. When one of these events is triggered, we'll send a HTTP POST payload to the webhook's configured URL.”
• Paypal: https://developer.paypal.com/docs/api/webhooks/v1/“The PayPal REST APIs use webhooks for event notification. Webhooks are HTTP callbacks that receive notification messages for events. After you configure a webhook listener for your app, you can create a webhook, which subscribes the webhook listener for your app to events. The notifications namespace contains resource collections for webhooks.”
Hospitality@luisw19
WebSocket APIs
© 2019 Oracle. All rights reserved.24
• WebSocket client required (majority of browsers already support it)
• Once connexion is upgraded a full-duplexcommunication is established via single TCP connection
Message HubStorage
API
Service
API ExposureWebsocket Server
Consuming Application(e.g. Browser App)
API
HTTP 1.1 GET ws(s)://<ws endpoint>Connection: UpgradeUpgrade: websocket
WebSocketServer
HTTP/1.1 101 HandshakeConnection: UpgradeUpgrade: WebSocket
Websocket Client
WS Frames
WS Frames
• Client initiates WebSocket handshake to establish connection (HTTP call with upgrade headers)
• Interface can be designed/defined with AsyncAPISpec*
IDL
* supermodel.io can be used as as domain (ubiquitous) modelling language
Hospitality@luisw19
WebSocket API Samples
© 2019 Oracle. All rights reserved.25
• Slack: https://api.slack.com/rtm“The Real Time Messaging API is a WebSocket-based API that allows you to receive events from Slack in real time and send messages as users. It's sometimes referred to as simply the "RTM API"”
• Blockchain.com: https://www.blockchain.com/api/api_websocket“Our WebSocket API allows developers to receive Real-Time notifications about new transactions and blocks”
• FitBit: https://dev.fitbit.com/build/guides/communications/messaging/“The Messaging API allows developers to easily send simple messages between the app and it's companion using a socket based communications channel. This API has an identical implementation in the Device API and the Companion API, so code examples work the same in both locations.”
Hospitality@luisw19
GraphQL Subscriptions
© 2019 Oracle. All rights reserved.26
• Subscriptions are also client-driven meaning the consuming app defines what data to include in the event
• A subscription is a read only push stream (not bidirectional)
• Transport protocol agnostic however popularimplementations based on Server-Sent events (e.g. Sangria) and WebSockets (e.g. Apollo)
• Requires a Pub/Sub system as events generated by mutations (resolvers) are captured and pushed
Subscription Transport
(e.g. WebSocket)
GraphQL Service
Consuming App 1(e.g. Browser App)
subscription
HTTP 1.1 POST /graphqlsubscription {
EventName {data fields
} }
Subscription Client
mutation
App 2(e.g. Browser
App)
API Exposure
Subscription Resolver
Mutation Resolver
Pub/Sub System
Schema
Server
Common Libs
• GraphQL subscriptions can be designed/defined with theGraphQL Schema Definition Language (SDL)*
Database Legacy
SOAPREST
Service
gRPC
Service
• Allows subscription to events using the Subscription operation type
* AsyncAPISpec working on samples
* supermodel.io can be used as as domain (ubiquitous) modelling language
Hospitality@luisw19
GraphQL Live Queries
© 2019 Oracle. All rights reserved.27
• Any query could potentially be a live query by using the @live directive
• Transport protocol agnostic
• Requires the implementation of a reactive data layer (e.g. to tail a query) in the graphql server
Live Query T
ransport(e.g. W
ebSocket, Long Polling)
GraphQLService
Consuming App 1(e.g. Browser App)
query
Subscription Client
API Exposure
Query Resolver
Schema
Server
Common Libs
• Although queries can be designed/defined with the GraphQL Schema Definition Language (SDL), live queries are not formally defined in the spec (a directive used instead)*
ReactiveData Store
HTTP 1.1 POST /graphql@livequery {
objectName {data fields
} }
• A live query is a read only push stream (not bidirectional)
Reactive Stack(e.g. rxjs)
* supermodel.io can be used as as domain (ubiquitous) modelling language
Hospitality@luisw19
GraphQL Subscription & Live Query Samples
© 2019 Oracle. All rights reserved.28
• Hasura.io Graphql Subscriptions & Live Queries: https://hasura.io/all-features“Build powerful applications with GraphQL & Serverless”
• AWS AppSync: https://aws.amazon.com/appsync/“AppSync simplifies application development by letting you create a flexible API to securely access, manipulate, and combine data from one or more data sources. AppSync is a managed service that uses GraphQL to make it easy for applications to get exactly the data they need”
• Samsara Live Queries: https://www.youtube.com/watch?v=g-asVW9JFPw“At Samsara, we've used live queries in production for the last two years to render live data in our applications by default withminimal boilerplate and plumbing. We'll discuss our experiences: reactive backends, developer happiness, and how you too can add live queries to an existing system”
• Reactive (Live) Queries at Facebook: https://www.youtube.com/watch?v=BSw05rJaCpA“At Facebook, we've been developing a new interaction model to enable live GraphQL queries. Leveraging reactive backends and implicit dependency capture, live queries enable a developer-friendly and efficient means for keeping data on clients up to date”
Hospitality@luisw19
© 2019 Oracle. All rights reserved.29
BrowserAppgRPCWebClient
NativeorServerAppsgRPCStub
Service(e.g.Java,Node)
gRPCServer
• Supports unary, client/server streaming and full bidirectional communication
• Protocol buffers over HTTP/2 to serialise / deserialise data and define the service interface (.proto file).
• Makes use of HTTP/2 as transport protocol
• Communication over HTTP 1.1 possible by implementing a HTTP 1.1 to HTTP 2 proxy (e.g. envoy) and the gRPC WebClient library
APIExposure
Proto over HTTP2Request
APIExposure
• gRPC servers and stubs can be generated (in multiple languages) from the .proto file using the protoc CLI
.protofile
Proto over HTTP2Request
Proto over HTTP2Request
Hospitality
Sync vs Async APIs1
2
3
Asynchronous on the web?Trendy Async API StylesDemo
30 © 2019 Oracle. All rights reserved.
Agenda
4 Conclusions & Q & A
Hospitality@luisw19
{ data: {newTweet :[
{ … }]
} }
Browser
OJET Page
OJET FormWeb Component
OJET ListWeb Component
Oracle JET Single Page Application
Service
Apollo GraphQL Client
Demo
© 2019 Oracle. All rights reserved.31
GraphQL Service
Schema
Server
Modules
Subscription Resolver Query Resolver
apollo pub/sub controller twitter client
WS
Fram
es HTTP 1.1 POST /graphqlquery{
searchTweets {…
} }
{ data : {searchTweets :[
{ … }]
} }
HTTP 1.1 POST /graphqlsubscription {
newTweet {…
} }
HTTP/1.1 101 HandshakeConnection: UpgradeUpgrade: WebSocket…
https://github.com/luisw19/graphql-subscriptions/tree/master/sample-2
log polling
req/rep
Hospitality
Sync vs Async APIs1
2
3
Asynchronous on the web?Trendy Async API StylesDemo
32 © 2019 Oracle. All rights reserved.
Agenda
4 Conclusions & Q & A
Hospitality@luisw19
Comparison (subjective)
© 2019 Oracle. All rights reserved.33
(++) Brilliant(+) Good(~) Neutral / depends on other factors(-) Not very good / partly supported(--) Bad / not supported
Webhooks
WebSockets
GraphQL Live Queries
GraphQL Subscriptions
Long Polling
Service to Browser
--
++
++
++
-
+
Service to Service
(Internet)
++
-
-
-
-
+
ClientDriven
Contracts
-
-
++
++
--
-
FullDuplex
Transport
--
++
--
--
++
--
Learning Curve
++
+
--
-
-
+
Broad Adoption
++
+
-
-
+-
++
Service to Service
(Internal)
+
-
-
-
++
~
Hospitality@luisw19
Questions?
34
Hospitality
Thank you
Luis Weir
Oracle
35 © 2019 Oracle. All rights reserved.