Building an API Strategy:
Introduction and the Business of APIs
Ronnie Mitra
Principal API Architect - Europe
Layer 7 API Academy
Business Drivers
API Styles
-- Break --
The Developer Experience
API Architecture
Securing APIs
-- Break --
Principles of URI Design
Agenda
API
All programmers are API designers
Connections between modules
Language Dependant
APIs are constrained by the syntax of the language
Web APIs
Language Independent
APIs are constrained by the syntax of the web
Most API Design principles can be applied
Some design principles are unique to Web APIs
Innovation
Consumer Reach
Revenue Source
Marketing
Integration
Light Bulb designed by Jean-Philippe Cabaroc from The Noun Project
Innovation
Consumer Reach
Revenue Source
Marketing
Integration
Light Bulb designed by Jean-Philippe Cabaroc from The Noun Project
Revenue Source
Strategy Implications:
• Maximize uptime and reliability
• Target high revenue consumers
• Competitive differentiators are a must
Innovation
Consumer Reach
Revenue Source
Marketing
Integration
Light Bulb designed by Jean-Philippe Cabaroc from The Noun Project
Consumer Reach
Strategy Implications:
• UX driven interfaces
• Specialization may be required
• Difficult to update applications
Innovation
Consumer Reach
Revenue Source
Marketing
Integration
Light Bulb designed by Jean-Philippe Cabaroc from The Noun Project
Marketing
Strategy Implications:
• Terms and Services are very important
• Identify and reward top performers
• Limit calls and restrict access when needed
Innovation
Consumer Reach
Revenue Source
Marketing
Integration
Light Bulb designed by Jean-Philippe Cabaroc from The Noun Project
Innovation
Light Bulb designed by Jean-Philippe Cabaroc from The Noun Project
Innovation from within
Innovation
Light Bulb designed by Jean-Philippe Cabaroc from The Noun Project
Innovation outside your borders
Innovation
Light Bulb designed by Jean-Philippe Cabaroc from The Noun Project
When does innovation happen?
Innovation
Light Bulb designed by Jean-Philippe Cabaroc from The Noun Project
Strategy Implications:
• Design interface for general use
• Identify success stories
• Eliminate misuse
Innovation
Consumer Reach
Revenue Source
Marketing
Integration
Light Bulb designed by Jean-Philippe Cabaroc from The Noun Project
Estimated revenue: $1.5B in 2012
http://wikibon.org/wiki/v/Cloud_Computing_2013%3A_The_Amazon_Gorilla_Invades_the_Enterprise
Web API Modern Timeline
2000
Salesforce API ebay API
2002
Amazon API
2004
Flickr API 2006
Twitter API Facebook API Google (Maps) API
2012
Programmableweb.com has 7144 registered APIs
Sources: apievangelist.com
programmableweb.com
internetarchive.com
Steve Yegge Rant
oreilly.com
2005
ebay makes APIs free
2004
First Web 2.0 Conference
2010
Salesforce adds HTTP API
2008
Programmableweb.com has 1000 registered APIs
2005
Programmableweb.com launched 54 APIs registered.
Original APIs are still successful
New business models have emerged
Know your drivers – design accordingly
Summary
Building an API Strategy:
Introduction and the Business of APIs
Ronnie Mitra
Principal API Architect - Europe
Layer 7 API Academy
Building an API Strategy:
URI Style Design Tips
Ronnie Mitra
Principal API Architect - Europe
Layer 7 API Academy
What is ‘good’ API Design? • Easy to learn
• Easy to use, even w/o documentation
• Hard to misuse
• Easy to read and maintain code that uses it
• Sufficiently powerful to satisfy requirements
• Easy to extend
• Appropriate to audience
Joshua Bloch, Principal Software Engineer, Google.
Principles of URI Style API Design
• URIs should be intuitive and ‘hackable’
• The interface should adhere to standards (RFC 2616 and RFC 3986)
• The design should be extendable
Naming URIs
• Names matter!
• Establish reserved words and key words
• Names should be meaningful (to the application developer)
Defining Resources
• Translate interactions into nouns
• Build a resource model
• Avoid RPC/Tunnel style names
• Not everything fits well into the CRUD + Object space
Not Always Easy
Interaction: “Perform a spell-check on this message” Object: Message? What method is ‘spell-check’?
Relationships
Don’t expose relationships unless they are useful to the developer
Each path segment should be actionable
GET
• Retrieve a representation
• ‘safe’ method according to RFC
• no user-requested side effects
• won’t impact data
• ‘Conditional GET’ is supported with caching
• Don’t abuse for non-read operations
PUT
• Write a representation
• Store the entity (full replacement)
• Idempotent
• Example:
PUT /myapi/messages/14
{Message}
Idempotence
No side-effects on identical calls PUT /myapi/messages/14
Result: Message Replaced
PUT /myapi/messages/14
Result: Message Replaced
Full Replacement
PUT /myapi/messages/14
{ “title”: “Welcome”}
{ “id”:”14” “title”:”Wlecome” “author”:”Ronnie” “body”: “Hi Glen, welcome to the team!” }
On the Server: { “title”:”Wlecome” }
Why not use PUT for partial update?
• Breaks HTTP specification
• No defined semantic – can produce unexpected results from a dev perspective
PATCH (Partial Update)
• RFC 5789 (HTTP Patch)
• Partially update an identified resource with the supplied entity
• Example:
PATCH /myapi/message/14
{Partial Message}
Patch Media Type
• RFC 6902 – JSON Patch
• Content-Type: application/json-patch+json
PATCH /myapi/message/14 HTTP/1.1
[
{ "op": “replace", "path": "/subject", "value": “new" },
{ "op": “add", "path": "/tags", "value": “urgent" }
]
Challenges with PATCH
• Not part of HTTP 1.1 spec
• Not widely adopted in implementations
• May not be familiar to developer audience
PATCH Workarounds
Use a unique URL:
HTTP POST /myapi/patches/messages/14
HTTP POST /myapi/messages/14/patches
HTTP POST /myapi/messages/14;patch
POST
• Write/Process an entity
• Accept entity as sub-ordinate resource
• Not Idempotent
• No identifier specified (factory pattern):
POST /myapi/messages
Non-Idempotent
POST /myapi/messages
Result: Message #14 Created
POST /myapi/messages
Result: Message #15 Created
Method Tunneling
• Older platforms may not support all verbs
• Need to resort to embedding the verb in a header or parameter
• Example:
GET myapi/shops?method=POST
• Avoid doing this
Representations
Expose object properties that are relevant to the developer
Embed child objects and properties, but need to decide on granularity
Design structures that are extensible – be careful when implementing schema
Representations - Granularity
{ “id” : “14” “title” : “Welcome” “body” : “Hello!” “author” : “38820” }
Representations - Granularity
{ “id” : “14” “title” : “Welcome” “body” : “Hello!” “author” : [ “id” : “38820”, “firstName” : “Ronnie” ] }
Representations – Granularity Considerations
Chattiness vs. Latency
Frequency of change
Interaction (what data is needed?)
Retrieve a Filtered Collection of Data
• Filter by requesting children
• GET /shops/london
• GET /shops/amsterdam
• Limited to objects and sub-objects
• Difficult to retrieve unions/joins of data
Retrieve a Filtered Collection of Data
• Example: “Retrieve all store locations in London”
• Use query parameter from URI spec
• GET /shops?location=London
• GET /shops?location=London,Amsterdam
• GET /shops?location=London&sort=distance
Complex Queries
“retrieve all shops within a radius of 10 km from a specific location that are open within specified hours and sell specific phones, devices and account plans”
GET /shops?radius=10&location=8882,28832&open_time=38882034&close_time=23882343&phones=iphone,blackberry,samsung&plans=monthly_3GB,monthly_4GB,pay_go_2GB
Complex Queries
URI space may be limited
Long queries can become difficult to manage
Use POST on an operator resource:
POST /shopsquery
{
“radius” : “10”, “location” : “388203,838200”,
“phones” : [“blackberry”, “iphone”]
}
Returning Collections
• array of results
• all properties and child elements?
• collection responses can be BIG!
Pagination
• data page mechanism
/api/resource?page=3
• fixed page size
Page 1 Page 2 Page 3 Page 4 Page 5 Page 6
• easy to navigate
Pagination
• offset + count mechanism
/api/resource?offset=10&count=20
• client app dictates page size
10 30
• client calculates offset and count for pages
Pagination
• use links to navigate
{“href”:“/api/resource?offset=11&count=20”, “rel”:“next”}
• client doesn’t have to calculate location
• easier to navigate through pages
Pagination
• use defaults to reduce ‘friction’
• reduces learning curve for new developers
/api/resource?offset=10&count=20
/api/resource
/api/resource?offset=10&count=10
Field Projection
•‘property selection’, ‘zooming’
• Collections can be too big for some client
• Allow client to select properties in representations
Field Projection - Example
GET /myapi/messages?fields=title,body
{
[{“id”: “1”, “title” : “hi!”, “body” : “hello”}…]
}
Linking
• Use links as identifiers
{
[
{“id”: “/myapi/messages/13”},
{“id”: “/myapi/messages/14”}
]
}
Linking
Advantages:
• Developer doesn’t need to construct URI
• URIs can change!
Trade-offs:
• Query parameters increase complexity
Implementing Linking
Lots of Standards:
• HAL
• SIREN
• Collection + JSON
• ATOM
• XML LINKING
• HTML
Implementing Versioning
• myapi/v1/path • Try to extend instead of version • Don’t break what is already there • Clients should ignore what they don’t understand
• Introduce breaking changes if you want to drive developers away.
XML
• Not the same as SOAP / Tunnel Style
• Widely used (AJAX, mobile, server)
• W3C Standard, RFC 3023
XML vs. JSON?
• Not very different
• ‘<‘ vs. ‘{‘
• Similar performance overhead
• Most clients support both (XML more widely supported)
• What do your developer’s prefer?
HTML
• Hypermedia content type
• Web Form: application/x-www-form-urlencoded
• Useful for simple name/value pair input
• Easy for developers to implement
Selecting a Representation
• Content Negotiation • HTTP Accept Header
• URI based • /myapi/messages.xml
Status Codes
• 1xx: Informational • 2xx: Success • 3xx: Redirection • 4xx: Client Error • 5xx: Server Error
Status Codes
• You MUST use the correct category
• The second part of the code (xx) is largely informational, but still important
• Reason phrases can be customized
Error Handling
• You might include an application level error code • Definitely Include a human readable error message
Asynchronous Request
Client Resource
202 Accepted
<link href=“…” rel=“status”/>
Status
Resource
200 OK
<status>complete</status> <link href=“…” rel=“result”/>
Optimizations
• Mobile and device platforms have unique bandwidth constraints
• Transport Level (HTTP)
• API Design
HTTP Optimizations
• Compression • Negotiation (Accept-Encoding) • Example:
• Is JSON more efficient than XML?
Accept-Encoding: compress, gzip
HTTP Optimizations
• Pipelining
• Multiple requests without waiting for response
• Supported in HTTP 1.1 (persistent connections)
• Idempotent actions only
API Design Optimizations
Reduce data size (pagination, field projection)
Reduce number of calls: • Composition • RPC batch • Caching
Composition
URI style APIs can be chatty
Combine interactions and expose on the server
Example:
/myapi/composer
Batching
URI style APIs can be chatty
Combine and invoke calls on the client
Can be complicated for the developer
Adaptive Responses
• Provide responses that are the best fit for the calling application •Examples: • Reduced granularity • Different defaults • UI Driven
URI Design Summary
• Some standards, but lots of choices • Design with developer in mind • Consistency and structure are important
Building an API Strategy:
URI Style Design Tips
Ronnie Mitra
Principal API Architect - Europe
Layer 7 API Academy
Building an API Strategy:
The Developer Experience
Ronnie Mitra
Principal API Architect - Europe
Layer 7 API Academy
Learning
WADLs and WSDLs are nice
But provide real documentation for humans!
Hypermedia =/= zero documentation
Security
Security can hurt Usability
But… security can also improve the overall experience!
We need to think about the system as a whole
Complexity
• Sometimes complexity is necessary – that is ok
• Enough features to meet requirements
• Don’t hurt the DX – use structure and modularity
Structure and Consistency
Define consistent standards for: • Naming • Collection structure • Content negotiation • Links
Structure and Consistency
Enforcing standards requires organizational discipline
Especially difficult in large organizations!
1. Define the problem space
2. Design interactions
3. Map the interaction to an API style
4. Prototype and get feedback
5. Iterate
Innovation
Consumer Reach
Revenue Source
Marketing
Integration
Light Bulb designed by Jean-Philippe Cabaroc from The Noun Project
Interaction Model
Define requirements:
What interactions will benefit the developer?
What information is required to support the interaction?
Prototype
Don’t bind to real data or backend
Use something lightweight and easy to change
Do this early
Practical Prototyping
Write simple code or script in a language that is easy for you to implement.
var express = require('express'),
app = express();
var port = 8080;
app.listen(port);
app.get("/tasks", function(req, res) {
res.status(200).send(‘<response>
<tasks>
<task>
<name>Pick up Kai</name>
<priority>1</priority>
</tasks>
</response>’);
}
Practical Prototyping
Apply minimal security
Try writing throwaway client code
Ask target developers to write code and use the API
Make quick changes and try it again
Usability Summary
• Focus on the developer
• Start by thinking in terms of interactions
• Effective for public and private APIs
Building an API Strategy:
The Developer Experience
Ronnie Mitra
Principal API Architect - Europe
Layer 7 API Academy
Building an API Strategy:
Architecture Foundations
Ronnie Mitra
Principal API Architect - Europe
Layer 7 API Academy
Representation Layer
Representation happens in the Connector
HTTP supports content negotiation
- Accept
- Content-Type
Differing clients (user-agents) === differing representations
- Desktop
- Browser
- Tablet
- Smartphone
Be prepared to support multiple representations
Security implementations are difficult:
• Mistakes are costly • Hard to understand specifications • Performance can suffer
Caching Layer
Caching happens EVERYWHERE
HTTP supports Expiration Model and Validation Model Caching
Expiration Model
- Expires
- Cache-Control: max-age
Validation Model
- Last-Modified
- Etag, If-Match
Be prepared to support caching for both client and server
Squid, Varnish, Nginx, MemCacheD, NSURLConnection etc.
• Chaining multiple calls • Aggregating and enriching data • ‘mashup’ external data with internal data
Orchestration:
Gateway Pattern
Abstraction of multiple interfaces
In Software Engineering: Façade Pattern
Benefits:
- Deliver a consistent experience
- Centralize API functionality
http://martinfowler.com/eaaCatalog/gateway.html
Nuts and Bolts of API Management
Developer Registration
Access Control
API Explorer
API Documentation
Social Engagement
Tracking and Reporting
Architecture Summary
• Use a layered architecture
• Deploy a gateway for runtime
• Deploy a portal for developers
Building an API Strategy:
Architecture Foundations
Ronnie Mitra
Principal API Architect - Europe
Layer 7 API Academy
Building an API Strategy:
The Security Challenge
Ronnie Mitra
Principal API Architect - Europe
Layer 7 API Academy
Injection Attacks
Utilizing input parameters to inject data that compromises
the security of the targeted system.
Examples:
- SQL Injection
- Command Injection
- Code Injection
- Argument Injection
API Attack Example:
SQL Injection Attacks: APIs
GET http://host.com/aresource?token=
%E2%80%98or%20%E2%80%981%3D1
GET http://host.com/aresource?token=‘ or ‘1=1
select * from tokens where token = ‘’ or ‘1=1’;
APIs May Be A Direct Conduit
292
HTTP
Server
App
Server
Database
App
Objects
Often: • Self-documenting
• Closely mapped to object space
Denial Of Service Attacks
An attack which has the objective of making a service
unavailable to all users
Examples:
- XML/JSON parser attacks
- Jumbo messages
- Server overload
Overflow Attack
Intentionally sending too much data in order to exploit a
target systems by exceeding expected boundaries.
Examples:
Buffer Overflow
Cash Overflow
XSS API Example
296
Attacker
Web App Server
(browser+APIs)
Victim: Web
Browser
Client
<SCRIPT …>
1. API injects
script in
3. Browser loads
content with
embedded script
2. Server fails to
perform FIEO: Filter
Input, Escape Output
API
OWASP Top Ten (2010 Edition)
Source: http://www.owasp.org/index.php/Top_10
Impersonating a registered application in order to access
an API resource.
Examples:
- Guessing application ID by brute force
- Retrieving application ID by sniffing traffic
- Cracking application to retrieve application ID
App Spoofing
What the Fudge*! I didn’t make 10000 calls yesterday!!!!!!
I’m not paying that.
*This is what WTF actually stands for.
New platforms, new languages:
• Ruby on Rails
• Node.js
• Scala
• Nginx
• Squid/Varnish/Traffic Manager
http://www.flickr.com/photos/grumbler/571106054/
http://www.flickr.com/photos/roboppy/238406811/
Your Money
This Shop Needs Your Money
You need to grant access to your money
Your google+ data
This app needs to access your Google+ data
You need to grant access to your resources
OAuth 2 Grant Types
- Authorization Code
- Implicit
- Resource Owner Password Credentials
- Client Credentials
Authorization Code Grant
326
Client Application
Resource Owner
Using
Application
Resource Server
I Wish I could access my resources through
this application…
Authorization Code Grant
327
Client Application
Resource Owner
Using
Application
Resource Server
…but I don’t trust this app enough to give it
my credentials.
Authorization Code Grant
Initiation
328
Client Application
Resource Owner Authorization Server
Resource Server
User Agent
Issue GET
request via
User-Agent
Authorization Code Grant
Initiation
329
Client Application
Resource Owner Authorization Server
Resource Server
User Agent
Issue GET
request via
User-Agent
response_type
client_id
redirect_uri
scope
state
OAuth 2 Authorization Request
response_type – indicates grant type
client_id –application identifier
redirect_uri (optional) – address which the UA can use to respond to client
scope (optional) – space delimitted string: what the client wants to do
state (optional)– opaque string used to defeat CSRF attacks
Sample Authorization GET URL:
https://azserver/oauth2/authorize?response_type=code&client_id=my_id&state=state&r
edirect_uri=http%3A%2F%2Flocalhost%3A8080%2Fcallback
Authorization Code Grant
Resource Owner Authentication
331
Client Application
Resource Owner Authorization Server
Resource Server
User Agent
Send
User
Authentication
Form
?
Authenticate
Authorization Code Grant
Authorization
332
Client Application
Resource Owner Authorization Server
Resource Server
User Agent
Deliver
Grant
Screen
? ? ?
Approve
Grant
Request
Authorization Code Grant
Receipt of Authorization Code
333
Client Application
Resource Owner Authorization Server
Resource Server
User Agent
Redirect
User-Agent
Client
Application
! Redirected
To
Client
Application
code
state
302
Authorization Code Grant
Access Token Request
334
Client Application
Resource Owner Authorization Server
Resource Server
Request
Access
Token Return
Access
Token
and Optional
Refresh Token
grant_type
code
redirect_uri
client_id
200
AZ Code
AZ Code
Authorization Code Grant
Access Protected Resource
335
Client Application
Resource Owner Authorization Server
Resource Server
Request
Resource
Using
Application
Return
Resource
200
Authorization Code Grant - Summary
Most Complex of OAuth 2 Grant Types
Provides full OAuth 2 capability
OAuth 2 Challenges
New attack surfaces
Flexible, but complex for API publishers to implement
Utilizes redirection URIs (should be validated with strong rules)
Poor implementations will be exposed (see Facebook)
Not a solution to user authentication
OpenID Connect
Identity Access and Authentication (when combined with Open ID)
Built on top of OAuth 2
Not tied to any single vendor or identity provider
Open ID, Open ID Connect and OAuth 2
OAuth 2 allows an end-user to grant an application access to protected resources
However:
- The authorization server must still authenticate the end-user
- The client application is unable to determine information about the end-user
Client Application
Resource Owner Authorization Server
User Agent
Send
User
Authentication
Form
?
Authenticate
OpenID Authentication can help the server authenticate the end-user
OpenID Connect provides a mechanism for the application to learn about the end-
user
Open ID, Open ID Connect and OAuth 2
Client Application
Resource Owner Authorization Server
User Agent
Send
OpenID
Authentication
Form Authenticate
Retrieve User
Information
OpenID
Resource
Server
Summary:
Challenge: Balancing Usability and Security
Old Threats Still Exist
New Styles and Access Models create new surfaces