Lessons learned from building enterprise APIsGustaf Nyman
Apitek Aktiebolag
as presented at the Nordic APIs 2016 Platform Summit
About Enterprise APIs
Enterprises have:
• In-house developed systems
• Acquired system
• External SaaS
• Clients as mobiles, tablets etc.
Most systems communicate and expose APIs
Enterprises has many internal APIs
Enterprises has legacy* APIs using old technology
You need API management
Single point API gateway:
• Route message to backend service
• Sometimes convert or translate messages
• Necessary if using microservice architecture
Administration:
• Keep track of routing configuration and metadata
Monitoring:
• Does it work? If not why?
• Is it fast enough?
Miscellaneous:
• Authentication, throttling, signing, validation, archiving etc.
API gateway
Internal API client Internal API server
External API serverExternal API client
API gateway
Internet
IntranetCloud vnet
API gateway
Routing of messages to back-end services
Protocol translation - to support legacy APIs
Keep track of all routing info and metadata
• WSDL/XML Schema, Swagger/JSON Schema etc.
Logging, monitoring and analytics:
• Simplifies monitoring and debugging of connected systems
Performance and reliability
Admin tool
The natural choice for developers and others
Real world: Using legacy API
Task 2002: Access legacy API (monolith) using modern tech?
Legacy API:
• Queue based transport
• Text based message format (structure and value)
• 70 message types
Solution legacy API
Use XML, XML Schema, SOAP and WSDL.
Convert between XML and legacy format using XML Schema
Generate WSDL/XML Schema from repository
API client Legacy API server
XML Schema driven converter
XML Schema
{getspeaker\Nordic APIs\Lesson learned\}
<getspeaker><conference>Nordic APIs<conference><presentation>Lesson learned</presentation>
</getspeaker>
XML Schema driven converter
Queue adapter
SOAP/HTTP Text/Queue
API gateway
Internal API client
Legacy API server
External API serverExternal API client
API Gateway
Internet
Intranet
Converter
Real World: Using legacy API
Task 2015: Access legacy API using modern tech?
Legacy API:
• SOAP/HTTP based transport
• XML message format described in WSDL/XML Schema
Move to modern technology, Microservices?
Solution: legacy API
Use JSON and Swagger/JSON Schema.
Convert between XML and JSON using existing XML Schema
Generate Swagger from WSDL using mapping rules
Lesson: Use metadata
Description of data, message exchange and contracts:
• WSDL/XML schema and Swagger/JSON schema
Use of metadata:
• Validate data and message exchange
• Generate stubs for client and server code
• Generate documentation, test data, user interfaces
• Data format conversion – with certain constraints
But remember:
• API/contract needs to be the first thing you design
• Think before generating from code or tools
• Make sure schemas are comprehensible!
Real World: B2B communication
Swedish insurance business 2002:
• Documents on paper communicated through surface mail.
Task: Define B2B communication guidelines, a ”standard”
Business requirements:
• Non repudiation
• Confidentiality
• Simple to implement on any platform/language
Solution: SOAP/WS-Security
Use standards, but define how and simplify:
• SOAP/HTTPS and WSDL/XML Schema
• X509 certificates and WS-Security for signing
• X509 client certificates
Add what’s missing:
• Organization identification header and correlation id
• A few standardized message types
10 years later still the natural B2B choice in the Swedish insurance business.
Lesson: Simplify and select
If you want things to last..
• Do not use ”everything” technologies provide – select the good stuff!
• Read the specifications – do not rely only on ready-made toolkits or frameworks.
• Remember: Simple things are easier to migrate and adapt to new technology
Problem: API failures
All APIs will eventually fail…
Ongoing task: improve analysis and production debugging support
Monitor and log
Log types:
• Processing logs – what happened?
• Data logs – what was the message content at specific point?
• Analytics logs – what is the response times and failure rate?
Logging subsystem:
• Should be always on
• Must be high performance and
• Must handle large amount of logs
• Must correlate and cluster related log data
What to log?
“Logga lagom”, i.e. log just enough information
• Too much and it becomes incomprehensible and unmanageable
• Too little and it is useless
Use experience and an evolutionary approach:
• You debug a difficult problem – improve log to enable easy identification next time
• Always strive to enable less experienced personnel to effectively debug using logs
Real world: Logging subsystem
Always on log system logs to disk – remove after 24 hours
On API failure related log files are stored in database
Web site to:
• Access and view error log data
• Automatic classification of errors using patterns
• Searching and reading logs on disk
• View other logs and system parameters
Take care of all your users
Your platform should be the natural choice for developers
• Cater to their needs.
More than developers:
• Many more are interested in what you are doing
• Many users are not computer savvy – they need simple tools
Typical non-technical users:
• People responsible for the business
• Testers with extensive business skills
• Customer service
Problem: Old technology
If we only redesign our system with…
• RPC
• CORBA/MTS
• SOAP/SOA
• REST/HTTP
• Event driven
• Microservices
• [your choice of the next big thing]
We will solve all problems and everything will be much better…
Everything will be legacy
…It is just a matter of time.
If redesign is not an option, put legacy APIs behind converters, translators and adapters:• Expose the modern API you prefer
Be prepared…• Use standards – avoid vendor specific extensions• Be selective – do not use everything• Simplify
…mitigate the consequences of systems turning legacy
Summary
Use an API gateway
Use metadata
Simplify and select
Monitor and log
Prepare for systems turning legacy
https://www.apitek.se/
Do not miss my talk tomorrow 14.15:
Build, Deploy and Test APIs and Microservices with Apitek Prisma