APIMETRICS

PSD2, APIS, AND PERFORMANCE AND

QUALITY MONITORING

A WHITE PAPER FROM APIMETRICSPAUL M. CRAY

AUGUST 2017

2

APIMETRICS

AbstractThis introduction of an API-driven mechanism for

banking interactions puts all institutions involved

in a financial transaction at the mercy of the worst

performing party. A universally agreed method

to measure performance and quality of PSD2

APIs against agreed service levels is essential.

APIs often do not behave as expected according

to their specifications and their behavior can vary

over time and among call locations. They often

appear to be operating normally according to

gateway transaction logs, while end users find

working with them difficult or impossible. It is there-

fore necessary for organizations to use synthetic

transactions to monitor the performance and

quality of APIs from the end user perspective.

Regulators and end users will expect PSD2

APIs to meet agreed service levels. Using

an independent tool for performance and

quality monitoring with synthetic transactions

provides all parties with assurance that the

APIs are meeting the agreed service levels.

“Payments Service Directive 2 (PSD2)

is set to revolutionize payment and

banking services in the European Single

Market. For PSD2, both regulators and

users will insist that the APIs exposed

by banks meet strict performance and

quality criteria.”

In the rapidly growing and evolving digital

economy, APIs are increasingly used for the

time-sensitive exchange of mission-critical data

and information between organizations. As of

January 2018, all European banks will be required

to expose APIs that allow authorized third parties

to access banking systems and customer account

information to facilitate a wide range of services.

Who should read this report?

This report is aimed at anybody with

responsibility for the oversight and management

of APIs relating to PSD2 and how they force

interaction between different institutions.

3

APIMETRICS

The European Single Market was established in 1993 to provide for the efficient movement of goods and

services throughout the current 28 EU members, plus the three additional European Economic Area

countries (Iceland, Lichtenstein and Norway) and Switzerland.

For many sectors, the Single Market has worked well. But one sector where it has not is payment services

and related financial products.

There are several reasons for this. Even after the introduction of the euro, many Single Market

members retained their own currencies. Because of the nature of what banks do (store people’s money),

there is often a high level of loyalty to familiar national banking brands. Also, expectations and practices

vary widely between countries, as does the regulatory environment – even where it is supposedly

harmonized across the Single Market.

To help facilitate Single Market payment services, during the mid- to late 2000s, the EU established the

Single Euro Payments Area (SEPA), the legal foundation of which is the Payment Services Directive (PSD).

SEPA did help in reducing the variations associated with certain kinds of services, but did not lead to the

creation of a genuine single market in payment services.

Historical Background

With the increasing growth of the digitized economy in the 2010s, the emergence of

novel products based on cutting-edge fintech, and the demand from non-traditional

organizations to be able to provide payment services, it was decided that further

measures were necessary to accelerate the evolution of the payment services sector

in the Single Market. Thus was born the Revised Directive on Payment Services (PSD2).

4

APIMETRICS

What is PSD2?PSD2 is a radical departure from PSD. PSD2 splits the payment ecosystem into several actors called

Third-Party Payment Service Providers (TPPs): Payment Initiation Services Providers (PISPs), Account

Servicing Payment Service Providers (ASPSPs), and Account Information Service Providers (AISPs).

Third-Party Payment

Service Provider (TPP)Description Example

Account Information

Service Provider (AISP)

Accesses the account infor-

mation of bank customers

An account consolidator

such as mint.com in the US

Account Servicing Payment

Service Provider (ASPSP)

Holds the customer’s

payment account

The customer’s bank

such as BBVA

Payment Initiation Services

Provider (PISP)

Initiates a payment on

behalf of the user

Ecommerce company

such as Amazon

€

€

€

€

€

Customer

Financial institution 1, e.g., BBVA

Financial institution 2, e.g., Banco Santander MasterCard

Financial institution 3, e.g., PayPal

Financial institution 4, e.g., AmEx

AUTHENTICATION

ACCOUNT INFORMATION

€

€

€

€

€

Customer

Financial institution 1, e.g., BBVA

Financial institution 2, e.g., Banco Santander MasterCard

Financial institution 3, e.g., PayPal

Financial institution 4, e.g., AmEx

AUTHENTICATION

ACCOUNT INFORMATION

AISPe.g. GoCompare

Fin. Inst. 1 Fin. Inst. 2 Fin. Inst. 3

€ € €

Fin. Inst. 4

€

5

APIMETRICS

AISP: Before

The implementation of PSD2 is supported by

Regulatory Technical Standards (RTS) defined by

the European Banking Authority. The RTS specify

the APIs that are used by TPPs for payment

services. Banks will now be obligated to expose

the APIs specified in the RTS to allow TPPs to

access payment services and other banking

functions. The banks will become platforms. They

will still be able to provide branded value-added

services to their customers (and to the customers

of other banks), but they will be competing in

a potentially very different marketplace.

AISP: After

There are likely to be many new entrants into the

payment services markets. Some of these will be

well-established pan-European brands such as

Amazon, Facebook, Google and PayPal. This may

well drive the establishment of a true European

market in payment services. It is also likely that

innovative fintech companies will attempt to

enter the market. In many cases, smaller entrants

will focus on niche segments in one national

market or across a number of countries.

Crucially, in the new ecosystem, all SEPA actors will

expose or consume PSD2 APIs and rely on them

to exchange mission-critical, time-sensitive data

and information to make and receive payments

and related banking and financial transactions.

The service an end user triggers in the portal of

Bank X or an ecommerce company might well be

provided by Bank Y. The question then becomes,

who will the user blame for underperformance?

€

€

Customer

Retailer (online)e.g. Ocado

CARD DETAILS

MONEY

Merchant acquirere.g. Vantiv

Card Schemee.g. Visa

Customer’s banke.g. Barclays

€€

Customer

Retailer (online)e.g. Ocado

AUTHENTICATION

MONEY

Customer’s banke.g. Barclays

(PISP)

6

APIMETRICS

PISP: Before

What impact will this have on revenues or sales

opportunities for banks when they have to rely on

services from competitors provided via PSD2 APIs?

What measures can banks, other TTPs and

regulators put in place to reduce the risk

from underperforming actors in the SEPA

ecosystem, and ensure that everyone is meeting

the required performance and quality?

PISP: After

Brexit and PSD2The UK is anticipated to leave the EU, the Single

Market and the SEPA in spring 2019. However,

it will still be necessary for UK banks to be fully

compliant with PSD2 by January 2018. The UK

Open Banking Working Group has established

the framework for an Open Banking Standard

that reminds of PSD2 in some ways. Because the

UK has made the greatest progress in defining

a standard on how APIs might be used in open

banking, it is even possible that the RTS will

be based on the UK standard (the European

Banking Authority is currently in London). Even

with Brexit looming, it is clear that UK banks will

have embraced the challenge of creating banking

platforms that use APIs to enable open banking.

€€

Customers

Proprietary AppsCustomer’s banke.g. Barclays

€ €

€ €

CustomersCustomers Customers

Third party apps

Financial Institution 1e.g. Barclays

Financial Institution 2e.g. BBVA

Public APIsPublic APIs

7

APIMETRICS

APIs for PSD2Banks and other organizations will have a degree of freedom in defining the interfaces that satisfy

RTS. A number of open-source efforts provide organizations with guidance on the API endpoints to be

exposed. These include the Open Bank Project, which provides a PSD2 sandbox and Open Banking.

Banking APIs – Current Situation

Banking APIs – Post-PD2 Situation

8

APIMETRICS

API ManagementIn the 2010s, RESTful web APIs became the plumbing

of the information economy. They allow organi-

zations to exchange data and information easily

within organizations, between business units,

and with customers, partners and suppliers.

Banks already make extensive use of APIs.

With the advent of PSD2, banks in the Single

Market will be required to expose APIs to TPPs

for payment and other banking services.

Components of an API management system include:

• Gateway

• Receives API requests, passes them to the back-end, and then passes responses back to requester

• Security functionality

• Handles authentication and authorization of requests through standardized mechanisms such

as OAuth2

• Developer portal

• Includes access to documentation and sandbox to let users develop apps that work with the

exposed APIs

• Monetization functionality

• Handles charging for the use of the API

9

APIMETRICS

• Monitoring

• Monitors load on exposed APIs, and performance and quality, including monitoring of

third-party APIs

• Reporting and analytics

• Analyze data on APIs and creates summary reports on API behavior

API management products such as those offered by

Axway, Mulesoft and others simplify the process of

integrating all the PSD2 APIs needed to provide open

services to other banks and TTPs. However, they do

not guarantee a level playing field on API quality. Only

through the use of independent tools for performance

and quality monitoring of PSD2 APIs with synthetic

transactions can SEPA stakeholders be confident that

all APIs are meeting their mandated service levels.

API MonitoringTo manage something, it is necessary to measure

it. Therefore, it is important to monitor both

exposed and consumed APIs. For instance, TPPs

will rely on third-party PSD2 APIs for the payment

services to meet the needs of their users.

Load monitoring (number and nature of requests

to an API in a given period) and the creation of

gateway transaction logs from which information

about latency and HTTP errors can be gleaned are

important. However, this form of passive monitoring

gives an incomplete picture of API behavior.

It is never simply a question of making sure your

own APIs are working properly; you must also

make sure the third-party APIs that are part of your

mission-critical business flows are working properly

as well. A key aspect of API management is managing

the third-party APIs you consume, a situation in

which you do not have access to load monitoring

data and gateway transaction logs. But you can

monitor third-party APIs with synthetic transactions.

10

APIMETRICS

Performance and Quality MonitoringAPI endpoints can return a HTTP 200 response even

when a back-end error is present. Furthermore, if

fields in the returned payload are missing or empty,

from the end user’s perspective, the transaction will

be perceived as a failure. Equally, if call latency is too

long, the requesting system might time out, again

producing the perception of failure. API performance

may also be inconsistent across geography and time.

In the case of PSD2 APIs, it is particularly important

that latency is acceptable from locations in the Single

Market, some of which are separated by thousands

of kilometers. Time zone differences across

Europe should also not affect API performance.

Inconsistency in performance is also an issue. An

API might have high overall availability in a given

period, but an API that has downtime or periods

of increased latency each day may be perceived

as having a worse performance than one with a

longer period of degraded behavior, but only on

a particular day. In other words, which is worse

– one 1-minute outage every day for a month,

or one 30-minute outage one day a month?

API quality isn’t just about latency or availability.

It’s about how different users experience the API.

An overall blended quality metric that combines

a number of metrics such as average latency,

availability and number of outliers (particularly

slow calls) helps you understand the quality of

your API at a glance, see whether the perfor-

mance is improving or worsening over time, and

if the API is meeting agreed service level criteria.

11

APIMETRICS

It also lets you compare APIs from different

providers, which is particularly important in

the context of PSD2; organizations might want

to give preference to the best performing APIs,

and organizations and regulators will want

to benchmark APIs against one another.

API quality isn’t just about latency or

availability. It’s about how different

users experience the API.

gateway transaction logs only; synthetic testing

actually lets know how your APIs are behaving.

And vice-versa, banks impacted by poorly

performing competitors need a mechanism to

identify offenders and a system for self-regula-

tion of offenders, perhaps through the publication

of PSD2 API blended quality league tables.

Although it is possible to use cURL scripts or

other scripting/programming tools to create

synthetic transactions for performance

and quality monitoring, it is better to use a

dedicated synthetic monitoring product.

An API performance and quality monitoring product should includes a comprehensive

suite of tools that makes it easy to:

• Set up test calls

• Manage authentications

• Look at results

• Generate reports

• Analyze and visualize performance data

• Configure dashboards

• Set conditions for user-defined alerts (for instance, if call latency is greater than a certain value

or the size of a payload is less than an expected amount)

• Create back-to-back calls to simulate workflows by using part of the response of an earlier

call in the input to a later one

• Handle WebHooks and other mechanisms, such as exposing an API monitoring system API,

for raising alarms in and integrating with higher level management systems

PSD2 APIs you expose should be proven to

meet certain performance and quality criteria.

You can’t do this with load monitoring for

Using ad-hoc or bundled tools that come with an API management system might seem be attractive.

But even more so in the context of PSD2 APIs than general APIs, it is false economy not to use a

12

APIMETRICS

ConclusionPSD2 is coming quickly. The banking sector needs

to develop new systems to manage relationships

that they would previously have considered to be

outside of their scope. Accountability and mutual

confidence in the services will be paramount in

ensuring regulators and consumers alike are happy.

You will need to monitor the performance and

quality of your PSD2 APIs for regulatory internal

management. If you are a TPP, you will want to

monitor the APIs of the platforms you use with

synthetic transaction, as you won’t have access

to gateway transaction logs. This requires a

comprehensive API management solution that

includes independent tools for synthetic trans-

action to allow API performance and quality

monitoring of internal and external tools.

A system for self-regulation of PSD2 APIs based

on blended API quality league tables generated

by an API performance and quality monitoring

system would be ideal for ensuring that the

SEPA ecosystem is meeting their obligations to

provide the APIs that will drive innovation

in payment services and banking in the Single Market

into the next decade, creating values for banks, TPPs

and consumers. PSD2 is in many ways a harbinger

of the API-driven world of the 2020s. And that world

is going to be one in which the only businesses that

survive and thrive truly understand how the APIs

they are exposing and consuming actually behave.

dedicated product. As your business relies on

your own APIs and those of others and others rely

on your APIs, by using a best-of-breed, dedicated

API performance and quality monitoring solution

you are going to be able to understand how the

APIs your organization depends on are really

working from the end-user perspective. Using

such an API performance and quality monitoring

system will give you the evidence to prove to

internal stakeholders, regulators and third party

users that your APIs really are working according

to spec (and those of other parties are not).

Some fintech companies have expressed concern

that banks might limit or even block altogether

access to their PSD2 APIs. Performance and

quality monitoring ensures honesty on all sides.

For instance, a fintech company can monitor a

bank’s PSD2 APIs. If the bank wants to make sure

that what the fintech company is saying about its

PSD2 APIs is true, the bank can carry out the exact

same synthetic transactions by using the same

or an equivalent tool. By doing this, everyone has

access to the same API quality data and it is much

harder to make false claims about API quality.

13

APIMETRICS

About APImetricsAPImetrics is a specialized provider of API performance and quality metrics. Their Cloud API

Service Consistency scoring system provides a simple and effective way to measure the

performance of cloud APIs. See http://apimetrics.io/casc for more details.

About the author Dr. Paul M. Cray is a computational physicist currently working with APImetrics on the application of

Machine Learning and Artificial Intelligence to API performance analysis.