api design what makes a good api? - jax londonco-author 1st oracle ipaas book, 2017 togaf 9...

Phil [email protected]

uk.linkedin.com/in/philWilkins

@PhilConsultant /

@MP3Monster

Blog.MP3Monster.org /

Oracle-Integration.Cloud /

APIPlatform.Cloud

API Design – What Makes A Good API?

‹#›© 2018 Capgemini. All rights reserved.

Today ....

1

3

Introduction

Good API a PoV

4 Good API Illustration

2 What is an API anyway?

• Technical Enterprise Architect specializing in Integration, APIs and PaaS.

• Part of a multi-award winning team• Started out as a developer working on Radar s/w• Moved into integration space – using Open Source Tech e.g.

JBoss App Server & Fuse, Apache Camel etc.• Worked with Oracle middleware & PaaS >9yrs

• Worked in end user companies, ISVs, niche & SI consultancies.

• Oracle Ace

About: Phil Wilkins

Peer technical review on a variety of books

published byThomas Erl (Prentice

Hall), Packt etc

Articles published in a range of

Journals

Co-Author of Implementing Oracle API Platform, 2018

Co-Author 1st Oracle iPaaS Book, 2017

TOGAF 9 Certified2013

• Co-authored books on API Platform CS & Oracle Integration Cloud• contributing to development of more than a dozen other titles ranging from Apache Camel to Cloud Computing

Design • Articles published in a range of journals on Cloud Strategy, PaaS, Integration & APIs.

Phil Wilkins

[email protected]

uk.linkedin.com/in/philWilkins

@PhilConsultant /

@MP3Monster

Blog.MP3Monster.org /

Oracle-Integration.Cloud /

APIPlatform.Cloud

{ developer }

LONDON

#OracleDeveloperMeetup

https://www.meetup.com/Oracle-Developer-Meetup-London/

http://bit.ly/OracleDevMeetup

Capgemini is One of the World's Largest

Consulting, Technology, and Outsourcing Firms

& a global “full service” business transformation

provider

Asia Pacific

Latin America

Canada

United States

Mexico

Brazil

Argentina

Europe

Morocco

Australia

People’s Republic of China

India

Chile

Guatemala

Russia

Singapore

Hong Kong

North America

UK & Ireland

Nordics

Benelux

“It is the quality of our people, and their capacity to deliver fitting solutions, with you and for you, that drive real business results.”

5Businesses

Revenue

12,8Billion EUR (2017)

Central Europe

Morocco

Net Profit

€1,18B

▪ Targeting Value▪ Mitigating Risk▪ Optimising

Capabilities▪ Aligning the

Organisation

Elements to successful collaboration

Application ServicesInfrastructure

ServicesBusiness Process

OutsourcingConsulting

(Capgemini Consulting)Local Professional

4 Across 40+ countries, 100 nationalities

What is an API anyway?


In computer programming, an application programming interface (API) is a set of subroutine definitions, communication protocols, and tools for building software. In general terms, it is a set of clearly defined methods of communication among various components. A good API makes it easier to develop a computer program by providing all the building blocks, which are then put together by the programmer.

Wikipedia

What is an API?



Wikipedia

What is an API?

An API does not need to be bound to REST or SOAP, it can be equally …• Header file (.h)• A messaging interaction (JMS, Kafka, gRPC)• Database



Wikipedia

What is an API?

Mock implementations, SDKs are different ways of providing tooling



Wikipedia

What is an API?

An agreed exchange or contract between two or more components



Wikipedia

What is an API?

Needs to level of human understanding for it to be used including …• Purpose• Payload• Happy & unhappy paths



Wikipedia

What is an API?

Needs to level of human understanding for it to be used including …• Purpose• Payload• Happy & unhappy paths

An agreed exchange or contract between two or more components

An API does not need to be bound to REST or SOAP, it can be equally …• Header file (.h)• A messaging interaction (JMS, Kafka,

gRPC)• Database

Mock implementations, SDKs are different ways of providing tooling


What is a REST API?

01

03

05

06

0207

Representational State Transfer (REST) - First

defined by Roy Fielding in 2000 as part of his PhD http://bit.ly/RESTPaper

08

Code on Demand (Optional)• Ability to pass code for execution

e.g. JavaScript to be executed

Layered• Client does not know (or

need to) whether it is connecting to the source or anything between

Cacheable• Client can cache

the responses• Tell the client

whether it is safe to cache

Self Describing

Resource manipulation through representations

Client/Server

Stateless• Everything needed is

provided as part of the request

04

Uniform Interface• Resource Based• Self describing messages• HATEOAS

http://bit.ly/RESTPaper


API Waterfall

Long Feedback-loop(weeks or months)

Design TryBuild Package & Deploy

TimeIf you’re working CODE

1st!


API Agile

Feedback

Design Build Package & Deploy

Try Continuous Test

Feedback

RunAnalyse

Feedback


API 1st

Feedback

Design

Build Package & Deploy

Try Continuous Test

Feedback

Run

Analyse

Feedback

Build Package & Deploy

Try Continuous Test

API Provider

API Consumer


But I Don’t Know Who My Consumers are!

On larger projects, or when building solutions for clients, you may not get close enough to the real API consumer. But there are things we can do …• Test Driven Development – creating the test case 1st will make you more mindful of

how you might consume your own service – therefore how others are likely to experience it

• Evaluate your design against best practise guidance, if you start rationalizing the design based on how the system is implemented, STOP!

• Colleague as substitute• Business Analyst –they will be looking from a business/consumer view of the world• A techie outside your project - use a developer who isn’t involved with the solution

(so they don’t know how things have been built) and ask them to try use the API• If it’s difficult to explain to someone outside the team, then the API is difficult to use!

• If its going to be a Public API – Use social/forums to share thoughts & encourage feedback


API Designers - the new Front Line of Business

Not all of us, have the benefit of working on APIs for those poster children of the API Economy, But …APIs done right can…• Help organizations create (better) decoupling of systems internally, greater

agility – respond to opportunities OR disruptors – SOA 2.0• See opportunities to expand on current services – offer the same service in

different ways – Walgreens photo printing, PSD2 Banking

From this you could say over 60% of organizations see good API Design as key to their future

Clo

ud E

lem

ents

Sta

te o

f API In

tegra

tion

Report 2

018

Good API Docs is key to this

Good API Design is key to this

https://smartbear.com/SmartBearBrand/media/pdf/SmartBear_State_of_API_2019.pdf

https://smartbear.com/SmartBearBrand/media/pdf/SmartBear_State_of_API_2019.pdf

Good API


How have we arrived at what makes an API good?

Looking at the guidelines for API definition that the successful API providers point to (Google etc)

Looking at well adopted APIs to see what they offer / include

Considering maturity models

Looking at how API definitions impact our ability to implement good solutions

How common Non Functional Requirements relate to API definition

Experience – from working with APIs from HL7 to SEPA, OAGIS to using public APIs and designing integrations & payloads

Taking time to understand standards like HTTP and how they can/should be used


The BasicsUse unary (single) naming for entities except arrays of values

Entity/attribute names should be meaningful and Semantically correct, but avoid words that have programmatic meaning or meaning is overloaded

Have an agreed convention for naming syntax e.g. language, UK or US English, or local language. Casing – consistent e.g CamelCase, snake_case

Agree when special characters can and should be used e.g. _ $ etc –generally best to avoid

Use JSON types properly, so numbers are numbers type not strings. For string consider including info such as min and max length

When dealing with dates, currencies etc use standards such as ISO3166 for country codes, ISO 8601 for date & time

Only use nested structures to provide semantic/logical meaning, not for arbitrary / development based convenience

Are null values needed? Perhaps they should be optional


The more advanced considerations…

Error responses should not give indication to implementation technologies

Ensure HTTP headers a fully utilized, e.g. content/type, Cache-Control values

SSL/TLS is not a get out of jail for bad practise. Infra will have your cert to root the call, headers/URIs get logged by network infra for audit

If APIs use common behaviours e.g. pagination, HATEOAS references ensure consistent naming & approach are adopted across all of your API endpoints e.g nextPage, previousPage

Keep query parameters simple, complex values/expressions make the query parameter easier to exploit for malicious intent e.g

http://example.com/myEntity?x=y&xEval=eq&a=b&aEval=gt

http://example.com/myEntity?filter=“x equals y AND a > b”


Richardson Maturity Model

• One way to look at what makes a good API is through the lens of a maturity model

• Best one is the Richardson Maturity model – FOR Rest APIs

• First defined 2008• Most cases are likely to

have passed Level 1 these days

• Note - Richardson was only considering REST

• Controversial BUT is time for the model to be extended as APIs can be more than pure REST now??


OWASP Top 10 - 2017

A1:2017-Injection

A2:2017- Broken Authentication

A3:2017-Sensitive Data Exposure

A4:2017- XML External Entities

A5:2017-Broken Access Control

A6:2017- Security Misconfiguration

A7:2017- Cross Site Scripting

A8:2017- Insecure Deserialization

A10:2017-Insufficient Logging

& Monitoring

A9:2017- Using Components with

known vulnerabilities

https://www.owasp.org/index.php/Category:OWASP_Top_Ten_2017_Project



OWASP Top 10 (API)

A1:2019- Broken Object Level Authorization


A3:2019-Excessive Data Exposure

A4:2019 - Lack of Resources & Rate Limiting

A5:2019-Broken Function Level Authorization

A6:2019- Mass Assignment

A7:2019 -Security

Misconfiguration

A8:2019 -Injection


& Monitoring

A9:2019- Improper Assets Management

https://www.owasp.org/index.php/Category:OWASP_Top_Ten_2017_Projecthttps://www.owasp.org/index.php/OWASP_API_Security_Project

Consideration as to the data being exposed.• Is there a

requirement for sharing each data value?

• How do I accept the data?

• Do I check & manage the accepted values?


https://www.owasp.org/index.php/OWASP_API_Security_Project


OWASP Top 10 (API)







A7:2019 -Security

Misconfiguration

A8:2019 -Injection


& Monitoring



• secure against established organizational security frameworks

• Use prebuilt/proven frameworks to help

• Review rights / privileges at the most granular level

• Ensure you’re maximising your API gateway




OWASP Top 10 (API)







A7:2019 -Security

Misconfiguration

A8:2019 -Injection


& Monitoring



• Make sure your API is making full use of HTTP(S)

• Make use of your firewall & API Gateway products – they should be able to protect these factors




Correct use of HTTP VerbsVerb Description Idem-

potentCorrect HTTP Response Codes

Get Used for retrieving information, but we can make it conditional with headers for example If-Modified-Since.

Yes • 200 • 404 (Not Found), if ID not found or invalid.

Post Typically used for delivering updates, as the URL is known. But can be used for a Create operation if the address for the creation is known.PUT and POST are both unsafe methods (i.e. entities can change). However, PUT is idempotent, while POST is not.

No • 200 success with a response payload• 201 if the result is a new entity• 204 success but no body• 404 (Not Found)• 409 (Conflict) if resource already exists

Put Typically used for creating new entities. As you address the create operation to the parent entity.

Yes • 200 reflects success of an update with a response

• 201 if the result is a new entity• 204 if the response contains no content• 501 reflects the inability to action

Delete Deleting the identified resource(s) Yes • 200 if you’re including response information• 202 if the request has been accepted, but

not executed• 204 if no content is being returned, but

actioned• 404 (Not Found) or id not valid


Correct use of HTTP Verbs

Verb Description Idem-potent

Correct HTTP Response Codes

Patch Update the identified resource(s) No • 200 reflects success of an update with a response

• 204 if the response contains no content

• 501 reflects the inability to action

Head Used for communicating meta information in the header. Must never have a body in the request, but response headers should reflect those provided by a Get

Yes • 200 (but a 0 length)• 404 (Not Found), if ID not found or

invalid.

Options Primarily used to support the pre-flight requests for Cross Origin Script checks. The pre-flight check is performed to verify whether the server will accept a cross origin request from the named domain. More generally allows a client to ask a server what ‘options’ are available for communicating with it, e.g. content type.

Yes • 200 success with a response payload (but a payload defined of 0 length)

• 204 success but no body• 404 (Not Found) or id not valid


Use All The Power of HTTP, headers…

Header Description Examples

Accept A family of header values describing the payload types that can be handled, to character sets, encoding, language

• Accept-Encoding: gzip, deflate • Accept-Language: en-US

Accept-Control-Request-Method

For helping to manage Cross Origin use cases • Access-Control-Request-Method: GET

Content-Length, Content-MD5

Describing the content, can be used to help detect tampering for example

• Content-MD5: dfghkojdfgbfgb==

If-Unmodified-Since

Only send me the data if it has changed since • If-Unmodified-Since: Sat, 29 Oct 1994 19:43:31 GMT

If-Match Only perform the action if the client supplied entity matches the same entity on the server. This can be used as an aid to locking

• If-Match: “v123345"

Warning Allows warning information about the payload. • Warning: 199 Miscellaneous warning

Expect The client can ask the server to behave in particular ways

• Expect: 100-continue

The HTTP standard provides a broad range of headers that can be leveraged and should be considered. Here are some examples, that can be particularly helpful …


API Versioning

By versioning we can manage change.But how to version?

If you were to ask Roy Fielding, ‘father’ of REST, he may tell you not to version your API at all

Semantic Versioning (semver.org) ?

What ever answer you choose, be consistent, be clear & be understood!!

Common options …• URL & Domain Versioning• Query Parameter use• Header Attribute• Accepts Header Attribute• Versioning the Payload• Non-breaking evolution

https://www.infoq.com/articles/roy-fielding-on-versioning


Approach: URL & Domain Versioning

Pros

Cons

Examples

• Most commonly adopted model for versioning

• Is clear and visible• Introducing new versions won’t break

lazy consumers

• Encourages thinking along major versioning and breaking of compatibility – tendency to stockpile changes & have big break, lots of change work

• Discourages the development of ‘tolerant reader’ thinking

• Entities evolve they don’t break

http://v1.myDomain.com/entity

http://myDomain.com/entity/v1

http://myDomain.com/v1/entity

Use subdomains for versioning

Do I have an entity called v1?

Most common location for version, but means major changes could be perceived to impact all entities


Approach: Header Attribute

Pros

Cons

Examples

• Not concreted into the URL – so less invasive for the consumer – easier for conditional consumption

• Easy to process – getting header elements is quick.

• Using Accept reflects the intent of the HTTP standard

• Can encourage major version changing & breaking of compatibility (stockpile changes & have big break, lots of change work)


• Need to implement explicit header version checks, rather than failing on the URI

The HTTP header contains a value like:x-api-version:2.0

Arbitrary naming convention

Use an obvious versioning scheme such as Sermver

What would header absence infer? Use latest?


Approach: Header Attribute - Accepts

Pros

Cons

Examples• Not concreted into the URL – so don’t need to change requesting code

• The client is telling the provider what response versions can be handled –negotiated exchange

• Does require prior knowledge of version• Using Accept reflects the intent of the

HTTP standard & original REST principles

• More work for the provider – to extract and route to correct implementation of backend


• Accept headers are harder to test & increase the workload on the caller

The HTTP header contains a value like:Accept: application/vnd.myDomain.v2+json

Consumers need to tell us what version(s) they will accept. Bulky if the are good citizens and work with latest


Approach: Nonbreaking Evolution

Pros

Cons

Examples• As long as rules about changes are

observed, then no breaking changes occur

• Less disruptive to consumers• Avoids the problem of where to put

version information• An approach that will support

effectively adding versioning retrospectively

• Kind of approach GraphQL promotes

• Overtime collect a lot of duplicate or overlapping fields

• Communication of redundant values (either new data values that won’t be consumed to old clients OR old values not used by new clients)

• Clarity overtime as to which values to use• Demands governance to ensure breaking

changes aren’t introduced

• Duplication and Deprecation. In order to change an existing field (rename or change structure), you can add the new one next to the old element and deprecate the old one in the documentation. After a while, you can remove the old element.

• Utilize Hypermedia and HATEOAS. As long as the API client uses the links in the response to navigate through the API, you can safely change the URLs without breaking the clients.

• Create new resources with new names. If new business requirements lead to a completely new domain model and workflows, you can create new resources. That’s often quite intuitive as the domain model has a new name anyway (derived from the business name). E.g: A rental service now also rents bikes and vans. So the old concept car with the resource /cars doesn’t work anymore. A new domain model with a new resource /vehicle is introduced. It’s provided along with the old /car resource.


Hypermedia as the Engine of Application State (HATEOAS)

{"productId": “A113","productName": “Go Go Gadget","description": “Inspector Gadgets secret tool.""links": [{

"rel": "self","href": "https://myDomain/shop/api/products/A113 "

}, {"rel": "details","href":

"https://myDomain/shop/api/products/A113/details"}, {

"rel": “shippingCost","href": "https://courier/api/calculate?weight=10"

}, {"rel": "addToCart","href": "https://myDomain/shop/api/addToCart/A113"

}]}

HATEOAS…• Its value can be contentious despite it being the

top layer of the Richardson Maturity Model• Some argue, it doesn’t help as…

• Limited support to make it easy• Not many clients make use of it (the idea of

dynamic consumption is rarely implemented)

The important thing here, is that we provide links to REST services so that …• We can navigate to the related entity• We can invoke relevant operation(s) for the entity

This makes it very easy to do things like request a list of entities and then iterate over them – no need to construct the link.If the definition of the product is with a 3rd party

But the real magic is by identifying the operations appropriate to the object’s state. So if we had no stock of “Go Go Gadget” we would exclude the addToCart reference.


Good APIs are more than Payload Descriptions …

In addition to the actual URL & Payload descriptions, you may wish to consider …• Providing background & contextual information – use cases designed to be supported• The language being used, will a glossary help – linking references / definitions• Information about how versioning is being applied across the APIs

What policies about access are being applied …• is the API restricted to only being available from certain locations or networks?• Rate limits?• Guidance on caching of API call content?• Access controls from OAuth to simple tokens?

If the API is public, you may also wish to consider …• Terms & Conditions of use of the API e.g. misuse, utilization in a criminal act, damage your

organization’s reputation etc• Legal obligations – law imposed upon you that may impact the consumer e.g. data privacy or

countries that can not use your APIs & implementation e.g. UN Embargoed countries• Rate limiting – how much traffic to allow taking into account capacity of your backend• Rate limiting – to ensure all clients get fair access• Rate limiting – based on how much access is being purchased• Quality of Service you commit to – if the backend of the API becomes unavailable, will you

compensate customers?


Good APIs are more than Payload Descriptions - Driving Adoption…

A well documented API is a great step forward, BUT example code says more …• Remember not everyone is as experienced or as talented as you – so a helping hand to use an API

is always good• People learn & understand in different ways – some prefer reading docs, other prefer seeing

working code• Sometimes people are under-pressure and just need some code to get them started

Consumers of your services including APIs are likely to want to know how things will be secured…• Share in general terms how security works and/or how you validate security e.g If your API is

passing through an F5 firewall with DoS Heuristics enabled – you might want to just say there is a firewall layer

• DON’T give specifics as this advertises what attack vectors are most likely to work• Security should reflect the classification of data e.g. requiring PCI compliance, Personal data

legislation, HIPAA etc• If your API requires checksums or signing of the payload then show how the client would do this,

but don’t reveal how / if you do anything on the backend


Good APIs are more than Payload Descriptions – Providing an SDK

Sometimes an SDK may ease adoption for the common ways of using an API…• Your API may use an approach less commonly used e.g. BSON, gRPC etc – why increase the

learning curve, provide an SDK that makes it easy• Opportunity to incorporate additional metadata about the use of the API, by allowing the SDK to

capture additional information• If your API needs metadata to describe the content being communicated, the SDK can determine

this for the consumer• If you’re APIs have been defined using one of the lesser known notations e.g. YAML, an SDK can

reduce this as a possible barrier• Making it easier to use your API, particularly for devices & mobile platforms ..

• Coding against a SDK means development or compile time we’re more likely to spot usage errors (class mismatches etc etc)

• Using dependent libraries is something every developer learns very early on

There are tools that can make this process a lot simpler e.g. • APIMatic• APITools• RESTUnited• Swagger CodeGen• AutoRest


What’s wrong with creating APIs from Code?

A typical solution, regardless of microservice or monolith ideally a solution is structured is comprised as ….

Presentation Layer

API Layer

Business Logic

Persistence Layer

When we generate API directly from the persistence or code layers we see abstractions collapse ….

Business Logic +

Persistence Layer

API Layer

Presentation Layer

Presentation layer has decoupling from core logic, so the way information is presented & interacted with is driven by good UX practises, not logic or data models. Likely to leverage other business logic modules

Persistence layer structured to optimise storage processes – maybe document model, or fully normalized relational. By abstracting we can adjust as necessary to be optimal

API layer separated to provide the ability to interact in a meaningful way – we may associate processes with specific conceptual entities that do not have a correlation to persistence

Persistence layer structured to optimise storage processes – maybe document model, or fully normalized relational. By abstracting we can adjust as necessary to be optimal

API layer becomes impacted by any persistence change

A worst performance is dictated by the storage performance

A change on persistence has a knock on chain effect

UI is dictated by implementation concepts not UX optimization


Mocking & Testing API Endpoints

When developing an API backend or API consumer, having the means to verify consistency is a good thing.• We saw briefly that Swagger Hub & Apiary provide mock end points – but this

supports the API Design phase more than on going development• Consider providing a live set of end points reflecting the current production state of

APIs• If you provide an SDK, then why not include a test framework with it?• There are tools that can take the API definition and then evaluate a test call or test

response against the API specification for compliance .e.g are data types correct, entities exist …

DreddSwagger Test

Templates

API Fortress

There are a wide range of testing for APIs from Postman and SOAPUI to Karate and others. But many of them do NOT derive their tests from the API definition.

Good API Illustration


Look at Google Maps … as an example of Good API

Provide both APIs and SDKs to make adoption easy


Understanding the consuming audience

Example content


Explanation on how the API use is paid for and requirements to use the API

Note how the Query params are clear and straight forward


Values clearly explained with examples

Legal values communicated


Not Convinced / Conclusion …

The following are drawn from a new SmartBear State of API report …

With more than 190,000 people, Capgemini is present in over 40 countries and

celebrates its 50th Anniversary year in 2018. A global leader in consulting, technology

and outsourcing services, the Group reported 2016 global revenues of EUR 12.5 billion.

Together with its clients, Capgemini creates and delivers business, technology and

digital solutions that fit their needs, enabling them to achieve innovation and

competitiveness. A deeply multicultural organization, Capgemini has developed its own

way of working, the Collaborative Business Experience™, and draws on Rightshore®, its

worldwide delivery model.

About Capgemini

Learn more about us at

www.capgemini.com

This message contains information that may be privileged or confidential and is the property of the Capgemini Group.Copyright © 2018 Capgemini. All rights reserved.

Rightshore® is a trademark belonging to Capgemini. This message is intended only for the person to whom it is addressed. If you are not the intended recipient, you are not authorized to read, print, retain, copy, disseminate, distribute, or use this message or any part thereof. If you receive this message in error, please notify the sender immediately and delete all copies of this message.

http://www.linkedin.com/company/capgemini

http://www.slideshare.net/capgemini

http://www.twitter.com/capgemini

http://www.youtube.com/capgeminimedia

http://www.facebook.com/capgemini

http://www.capgemini.com/about/how-we-work/the-collaborative-business-experiencetm

http://www.capgemini.com/about/how-we-work/rightshorer

http://www.capgemini.com/

api design what makes a good api? - jax londonco-author 1st oracle ipaas book, 2017 togaf 9...

Documents