choosing an integration platform -...
TRANSCRIPT
Choosing an
Integration Platform
Technical Whitepaper
Authors: Kent Weare & Michael Stephenson
Brought to you by:
Choosing an Integration Platform
2
Table of Contents
1 About the Authors............................................................................................................. 4
2 Technical Reviewer .......................................................................................................... 5
3 About the Whitepaper ..................................................................................................... 5
3.1 PAPER INTRODUCTION .......................................................................................................................... 5
3.2 WHO THIS PAPER IS FOR? ...................................................................................................................... 6
3.3 PAPER FORMAT ................................................................................................................................... 6
4 General Considerations ................................................................................................... 7
4.1 WHAT ABOUT SOA/ESB/PUB-SUB/BROKER? ........................................................................................ 7
4.2 VENDOR PLATFORM COMPOSITION – SINGLE VENDOR VS BEST OF BREED? ............................................ 7
4.3 SINGLE VENDOR .................................................................................................................................. 7
4.4 OPEN SOURCE .................................................................................................................................... 8
4.5 VENDOR PLATFORM COMPOSITION – SUMMARY ................................................................................... 8
4.6 VENDOR ROADMAP ............................................................................................................................ 9
5 Feature Considerations .................................................................................................. 10
5.1 TOOLING ........................................................................................................................................... 10
Integrated Development Environment (IDE) ............................................................. 10
Administration Console ................................................................................................ 11
Deployments.................................................................................................................. 11
Exception Management and Troubleshooting ......................................................... 12
Monitoring ...................................................................................................................... 12
Unit Testing ..................................................................................................................... 12
Tooling Summary ........................................................................................................... 13
5.2 PLATFORM ......................................................................................................................................... 13
Runtime .......................................................................................................................... 13
Scalability ....................................................................................................................... 14
Application Isolation ..................................................................................................... 14
Message Formats .......................................................................................................... 14
Platform Summary ......................................................................................................... 14
5.3 TRANSFORMATIONS ............................................................................................................................ 15
Transformations Summary ............................................................................................ 15
5.4 ADAPTERS ......................................................................................................................................... 16
LOB Out of Box .............................................................................................................. 16
SaaS ................................................................................................................................ 17
Custom Adapters .......................................................................................................... 18
Protocol or Application Adapters ............................................................................... 18
Level of Implementation .............................................................................................. 19
Adapters – Summary .................................................................................................... 20
5.5 BUSINESS RULE ENGINE ....................................................................................................................... 20
Business Rule Engine – Summary ................................................................................. 20
5.6 BUSINESS ACTIVITY MONITORING ........................................................................................................ 21
Business Activity Monitoring – Summary ..................................................................... 21
5.7 API MANAGEMENT ........................................................................................................................... 22
Service Virtualization ..................................................................................................... 22
Identity Management .................................................................................................. 22
Security ........................................................................................................................... 22
Choosing an Integration Platform
3
Governance and Policy Management ..................................................................... 22
Analytics, Metering and Service Level Agreements ................................................. 22
Scalability ....................................................................................................................... 23
API Design and Developer Collaboration ................................................................. 23
API Management – Summary ..................................................................................... 23
5.8 DEPLOYMENT MODELS ....................................................................................................................... 24
Installation and Configuration ..................................................................................... 24
Deployment Options .................................................................................................... 24
On-Premises/Cloud ....................................................................................................... 24
Infrastructure as a Service (IaaS)................................................................................. 24
Integration Platform as a Service (IPass) .................................................................... 25
Hybrid.............................................................................................................................. 25
Deployment Models - Summary .................................................................................. 26
5.9 ENVIRONMENT MANAGEMENT ............................................................................................................ 26
Environment Isolation .................................................................................................... 26
Configuration Setting Management .......................................................................... 27
Cost & Licensing ............................................................................................................ 27
Environment Management – Summary ..................................................................... 27
5.10 INDUSTRY SUPPORT ............................................................................................................................. 28
Industry Support – Summary......................................................................................... 28
5.11 COMMUNITY AND ECO-SYSTEM .......................................................................................................... 29
Independent Software Vendors (ISVs) ....................................................................... 29
Blogs ................................................................................................................................ 29
Wikis................................................................................................................................. 29
Webcasts, Recordings, Webinars ................................................................................ 29
Forums ............................................................................................................................ 29
Events and User groups ................................................................................................ 29
Community Tools ........................................................................................................... 30
Community and Eco-System – Summary ................................................................... 30
5.12 PLATFORM IMPLEMENTATION .............................................................................................................. 30
Getting Help to Implement your Integration Platform ............................................. 30
Platform Implementation – Summary ......................................................................... 31
5.13 COMMERCIAL ................................................................................................................................... 32
Licensing Questions....................................................................................................... 32
Customers ...................................................................................................................... 32
Support ........................................................................................................................... 32
Success and Challenges .............................................................................................. 33
Platform Investments .................................................................................................... 33
Aggressive Sales and Marketing teams ..................................................................... 34
Commercial – Summary ............................................................................................... 34
6 Looking Forward .............................................................................................................. 34
6.1 MARKETPLACES ................................................................................................................................. 34
6.2 MICROSERVICES ................................................................................................................................ 35
7 How does Microsoft stack up? ...................................................................................... 35
8 Conclusion ....................................................................................................................... 36
9 Call to Action................................................................................................................... 37
Choosing an Integration Platform
4
1 About the Authors
Kent Weare
Kent Weare has been involved in integrating systems for over the past 10 years in a
variety of capacities including being a developer, an architect, leading teams,
stakeholder and a sponsor. These projects have had budgets that ranged from $50000
to $15 million dollars and generally focused on Government, Healthcare and Energy.
In addition to extensive project experience, Kent has been responsible for supporting
and leading a Middleware support team. He has been on the other end of a 2 am
alert notification and been involved in introducing pro-active monitoring so that he or
his team did not receive regular alerts notifications in the middle of the night.
Kent’s integration experience comes from a variety of platforms, including Microsoft
BizTalk Server/Services, MuleSoft, IBM and Intersystems Ensemble.
In addition to Kent's project and operations experience, he has been very active in
the integration communities having published many blog posts, webcasts,
conference presentations and has also co-authored 3 BizTalk books. For these efforts,
Kent has been recognized as Microsoft MVP for the past 7 years.
Michael Stephenson
Michael is a highly experienced freelance integration architect with many years of
experience of designing and delivering integration projects which leverage the
Microsoft technology stack. He has deep, practical knowledge of delivering complex
solutions with BizTalk, Microsoft .NET, Microsoft Azure and associated technologies.
Michael has also been a technical lead on 30+ projects which have leveraged
Microsoft's cloud platform using many of the different features it offers.
Michael is heavily involved in community activities around Microsoft technologies
through the Microsoft MVP and Advisor/Insider programmes and also speaks at user
groups on a regular basis. Michael is also an author for Pluralsight having produced a
very popular courses on .net and RabbitMQ. He has also written a popular set of
articles about building a real world agile integration platform using Microsoft Azure.
http://tinyurl.com/azure-integration-platform
Michael is the creator of the BizTalk Maturity Assessment an initiative to help
companies to measure the maturity of how they work with BizTalk and compare how
they do BizTalk against recognized good practices. For more info refer to
http://biztalkmaturity.com
More info about Michael is available on his website http://MicrosoftIntegration.guru.
Choosing an Integration Platform
5
2 Technical Reviewer
Steef-Jan Wiggers
Steef-Jan Wiggers has been in the integration space for more than 10 years in various
projects in fulfilling the role of developer, designer, architect, team-lead, subject
matter expert and senior support engineer. He has predominantly worked with the
BizTalk Server product, Microsoft Azure and web service technology. During this time
Steef-Jan has developed extensive domain experience in the areas of Banking,
Insurance, Logistics, Energy, Aviation, Real Estate, and (local) Government.
Since 2006 Steef-Jan has been very involved in the integration community having
contributed to blogs, TechNet Wiki, forums, and newsgroups. He is the author of the
BizTalk Server 2010 Cookbook, reviewer of various other BizTalk related books and
white papers, and an (inter)national speaker at various conferences and user groups.
For these efforts he has been recognized by Microsoft the last four years as a “Most
Valuable Professional” in the areas of BizTalk and Integration.
3 About the Whitepaper
3.1 Paper Introduction
The introduction of Cloud, SaaS, Internet of Things, Outsourcing, Mergers and
Acquisitions has fuelled the need for Integration. No longer is Integration an
afterthought as organizations are starting to include budgets for Integration activities.
Integration is no longer a necessary evil and more recently, considered by some
organizations as being strategic. It is usually seen this way by organizations who have
made the proper investment in an Integration platform and have reaped the benefits
of this investment.
For other organizations, their business is integration. Whether it is exposing internal data
as APIs or performing B2B transactions, integration is at the forefront. As Integration
technologies evolve, they may introduce a competitive advantage for firms, so it is
important to invest in the right platform(s).
We are firm believers in investing in platforms. We do subscribe, for most enterprise
organizations, standardization is a good thing as it allows for solutions to be built in a
consistent manner where economies of scale and skill are leveraged. This allows
organizations to turn around projects in an efficient and cost effective manner. When
a team has been armed with a comprehensive integration platform, that has a
vibrant community, it results in Integration being a true enabler that leads to business
results.
Choosing an Integration platform can be a very daunting task. There are many
established vendors in the marketplace today and there is also a handful of ambitious
start-up companies that have begun to make waves in the industry.
The purpose of this paper is to provide readers and decision makers with a simple tool
that will help them evaluate vendors in a requirements-driven manner. There are
many vendors that want to tell you how great they are, but what I hope readers can
benefit from in this paper is leveraging a checklist approach that helps you evaluate
a vendor based upon requirements that are important to you.
Choosing an Integration Platform
6
3.2 Who this paper is for?
This paper is primarily intended for CTOs and Architects who are involved in the
decision making process. The paper may also be valuable for managers who have a
technical background. All too often, integration platforms are being sold based upon
a lot of 'fluff and marketecture'. As a result of this unfortunate trend, this paper has
been created and will focus on the necessary details that are often overlooked. This
paper is going to include many technical details without any apologies. All too often
vendors want to gloss over the details and will try to jump right into the "signing check"
phase.
The contributors of this paper have seen, first hand, fallout that has resulted from
customers selecting a vendor without performing proper due diligence. Sometimes
this is the result of being blatantly misled by sales organizations, the result of a poor
vendor selection process, an overzealous decision-maker trying to pad their resume
or a decision is made based upon bad assumptions.
Some 'horror stories' that the contributors have experienced include:
Software that is riddled with bugs
Software that is very complex to install
Software that was misrepresented by the vendor during sales cycles
Software that does not meet business requirements
Vague or incomplete documentation
Limited internal or external technical resources available to support
implementation
Poor performance
Limited visibility into the health of deployed interfaces
Poor support offered by the Vendor
Escalating costs
3.3 Paper Format
The paper has been broken into a few main sections. The first being a General
Considerations section. Within this section we offer up some guidance based upon
our experience in the field. This section does not include specific requirements but
things to consider when evaluating Integration Platforms.
The next section is the Feature Considerations section. Within this section, a detailed
description of the requirement and why it is important will be included. These
requirements are features that the paper contributors have run into on their own
projects but also while visiting other prospects and customers who have integration
needs.
The integration landscape has changed considerably in the past five years and we
expect it to continue to evolve. As a result, we have included a section called Looking
Forward that will outline some trends that we anticipate in the near future.
Choosing an Integration Platform
7
The following section, How does Microsoft stack up, will focus on how the Microsoft
platform stacks up against these requirements. In this section, we will focus on the
Microsoft stack which includes both server and service based products such as BizTalk
Server and Azure including API Management, BizTalk Services and Service Bus.
An accompanying spreadsheet (can be downloaded here) will also be included that
allows you to assign different weights to each of these requirements so that you really
evaluate what is important to you.
Finally, we will wrap up this paper with a Conclusion and a Call to Action.
4 General Considerations
4.1 What about SOA/ESB/Pub-Sub/Broker?
This paper is trying to avoid, being type casted into a specific architectural style which
is really what each of these types of implementations are. Regardless of which
architecture style you are looking to implement, the requirements that have been
included within this paper are pretty agnostic. You will also have the ability to select
the requirements that are most applicable to your scenario within the checklist
spreadsheet by assigning the appropriate weight.
4.2 Vendor Platform Composition – Single Vendor vs Best of Breed?
This has been a highly debated question. All too often, the term ‘Best of Breed’ has
been a loaded and biased term. Some vendors position the aggregation of several
disparate components as being best of breed. But be careful, to ensure these
components are not just ‘best of what is available’. Using ‘best of what is available’
components lead to a fragmented platform that is full of bugs and interoperability
issues. A question to ask yourself is, do you want your developers either fixing the bugs
that exist in these open source components, or do you want your developers solving
business problems?
4.3 Single Vendor
Leveraging a Single Vendor has its advantages and disadvantages as well. On the
positive side, you will have a polished product that will work across the technology
stack seamlessly. The technology stack may include Authentication and Identity
management, IDE, Web Servers, Operating Systems, Messaging platforms
(Queues/Topics), Database platforms and collaboration platforms.
Using a single platform also allows for a consistent skill set across your team. Leveraging
existing skills reduces time to deliver and lowers Total Cost of Ownership (TCO).
On the flip side, leveraging a single platform may introduce vendor lock-in. Yes,
leveraging open protocols and standards such as REST, SOAP and AMQP reduce this
risk but it still exists. If you are going to go with a single platform, then you want to
ensure that you are considering this single vendor as a valued technology partner,
are comfortable and aware of their roadmap.
Choosing an Integration Platform
8
4.4 Open Source
Open Source platforms are becoming more and more prominent within the industry.
Choosing whether or not to adopt Open Source really becomes a decision that an
organization needs to make. For some organizations, they want to standardize on their
infrastructure as much as possible and this oftentimes results in running Windows on
the VM Ware or Hyper V stack. For organizations that are in this mode, bringing in open
source ESBs or messaging brokers is going to cause some negative disruption. Often
times these products just do not perform well on Windows platforms, or support and
knowledge is limited since these products are ‘tried and tested’ on Linux platforms.
For some organizations, running messaging and ESB platforms on Linux distributions is
not a problem. For these organizations, leveraging open source platforms is not very
disruptive and may be something that can be embraced.
When considering Open Source, really take a look at their Open Source model. Is it
truly Open Source or are their roots in Open Source and they are very much in ‘for
profit’ mode while embellishing their Open Source stance? For some companies they
may offer an Open Source version of their product, but then charge for Enterprise
Features. Features that you would expect to exist within a platform that you are paying
really good money for. Also, even if a vendor has made their core engine available
via Open Source, are you really going to fix a bug for that vendor? Is that the value
that you are expecting from a vendor that you are paying money to? Even if you are
not going to fix that bug, do you really want your developers chasing those bugs? Isn’t
that the vendor’s responsibility?
Some vendors really rely upon Open Source projects and frameworks in order to stitch
a platform together. This approach is prone to interoperability issues and lack of
accountability across the platform as who really owns the bug. If a platform consists
of these projects such as CXF, Jersey, CloverETL, Drools, Eclipse, Tomcat and Maven
as examples, then why are you paying a license and where are your license fees
going?
Open Source may not be for everyone and an organization needs to gauge its
comfort level with these technologies prior to their adoption. Organizations also
cannot blindly buy into the Open Source hype cycle without understanding what a
vendor’s business model really is and how the use of Open Source technology will
benefit the customer.
4.5 Vendor Platform Composition – Summary
In summary, the key elements in this section as follows:
Minimum
o Understand a vendor’s stance towards Single Vendor and Open Source. Be aware
of any trade-offs that may exist with either approach.
o If the Open Source is chosen, understand who is responsible for fixing bugs in
Open Source components.
Choosing an Integration Platform
9
Preferred
o Make decisions that align with your Enterprise Architecture principles. Adopting one
an approach that is en-vogue will have serious downstream effects.
Be cautious if:
o The vendor selectively chooses which components they will provide through
Open Source licensing and which components they will charge money for.
4.6 Vendor Roadmap
When you choose to invest in a technology platform, you are making a major
commitment and multi-year investment which you intend to be a foundation piece
of your organization’s IT capabilities. With this in mind, you are making decisions which
need to be aligned with a short/medium and long term view. When choosing an
Integration Platform you want to choose technologies which are going to be
complimentary to your own business and technology roadmap. With that in mind it is
important that you have a degree of confidence the vendor, or vendors, you choose
will have a roadmap for their technology offerings which is aligned to your needs.
If you see that a vendor has a good product offering to suit you now but their road
map indicates they will be making most of their investments in technologies which you
will not use, you may end up with a product that the vendor does not invest in. This
leads to a lack of innovation within the platform.
In summary, the key elements in this section as follows:
Minimum
o The vendor has a published road map.
o The road map is relatively up to date.
Preferred
o The vendor is regularly communicating about their plans for the short and long
term.
o There are regular new releases of their products.
o The vendor has well known employees who regularly talk about the future of their
platform.
o Industry recognized people are talking about how the vendor’s future road map
may look.
o The vendor has insider’s programs which customers can join under an NDA to help
shape the vendor’s future roadmap.
Be cautious if:
o The vendor is uncomfortable explaining their road map.
o The vendor seems to disregard industry trends.
Choosing an Integration Platform
10
5 Feature Considerations
Within this section, detailed requirements will be included in the following areas:
Tooling
Platform
Transformations
Adapters
Business Rules Engine
Business Activity Monitoring
API Management
Deployment Models
Industry Support
Community and Ecosystem
Platform Implementation
Commercial
5.1 Tooling
Tooling includes the tools that Developers and Administrators use to become more
productive. The number one competitor for any Integration Platform is developers
coding point to point interfaces. Tooling allows developers to configure rather than
build and really provides value to the organization that is considering an Integration
Platform. Tooling also reduces the redundant plumbing that developers will continue
to write as they build interfaces.
Integrated Development Environment (IDE)
IDEs are tools that allow developers will use to build out interfaces. Developers need
to be productive with these tools. A developer who deals with a clunky interface or
buggy IDE becomes less productive than they would otherwise.
The tool must be responsive and include tools that accelerate development.
The IDE must have little or no bugs, load quickly and does not crash.
The tool must be polished, and very precise. Dragging and dropping ‘widget’s
needs to be smooth.
All options must be available through the User Interface. No partial
implementations where some options need to be enabled from other places.
The IDE should be supported by the middleware vendor and should not be
dependent upon other organizations for innovating the development
experience.
Choosing an Integration Platform
11
Administration Console
The Administration Console is used by developers as they are building out their
applications as it gives them the opportunity to configure and test their application.
Once the application has been promoted through the various non-production
environments, an application package is typically handed off to an Administrator
who is responsible for deploying the application. Furthermore, the Administrator will
use the Administration Console to track exceptions, message throughput,
configuration settings and has visibility into the health of the integration environment.
An Administration console needs to provide Administrators with the visibility to quickly
diagnose and resolve interface issues.
The Console needs to provide granular functions that allow administrators to
manage all aspects of an interface including:
o Modifying endpoint configuration without requiring a redeployment.
o Enabling and disabling endpoint configuration.
o Exporting endpoint configuration.
o Viewing message tracking information, including message bodies.
o Visual tracing tools that allow administrators to view the path that a message took.
o Modifying throttling, threads and thresholds from single location for entire farm.
o Modifying endpoint security related credentials.
o Providing inflight message status, including how many messages are currently
inflight, where they are connected to and whether they are awaiting for another
message event (dehydration/correlation).
The ability to resume a message that has been suspended due to an
exception or and end point being unavailable.
o Role based authorization for administration and operator purposes.
o The ability to administrate the environment without logging into a server and
preferably over a web based console.
Deployments
Once a developer has built their application, performed some unit and integration
testing, it then needs to be promoted to a subsequent environment. In Non-
Production environments, developers may be able to perform these installations on
their own, but generally do not have the ability to perform the production
deployment. With this in-mind, have the ability to package your artifacts in a
meaningful and efficient manner is very important.
The following requirements should be considered when deploying integration
applications:
Deployment packages include a separation of application artifacts and
configuration.
Endpoint configuration can be tweaked after deployments by authorized
administrators.
Deployments can be completely automated via scripts or APIs.
Choosing an Integration Platform
12
Configuration for different environments is easily distinguishable.
Exception Management and Troubleshooting
Middleware platforms are inherently reliable. They often times are configured to
support durable messaging and have redundant components. With the Integration
Platform residing in the middle of other systems, errors that occur up, or down, stream
will ultimately propagate to the middleware. It is therefore critical to provide robust
Exception Management and Troubleshooting tools. These tools should offer some level
of self-service but also simplify supporting the Integration Platform.
The following requirements should be considered when evaluating an Integration
Platform:
Exceptions are easily identifiable and are stored centrally.
Exceptions can be viewed via a Management Console.
Any exceptions that have been raised can be resumed or terminated from
their current state.
Supporting data that indicate why the exception occurred and the path that
a message took is available.
The message body, at the time of exception, is available and its contents can
be exported or saved.
An end user portal is available for either a technical or business audiences.
This portal will provide visibility into exceptions without requiring a user to log
into the Management Console.
The end user portal allows for some simple reports and analytics that will
graphically display what application raised the event, the frequency of the
event and last time the event occurred.
End users can subscribe to events that are being captured in this portal and
receive notifications.
Message Repair and Resubmit capabilities exist via a User Interface.
Monitoring
When exceptions, or failures do occur, how are they communicated to the various
support teams?
The following requirements should be considered:
Monitoring across products. If there are platforms dependencies, they can be
monitored by a single tool.
Does enabling monitoring require subject matter expertise across different
domains and components?
Unit Testing
The platform must provide facilities to unit test discrete components in an
automated fashion including messages and transformations.
Unit Testing is achieved within the same IDE, is tracked and can be
automated.
Choosing an Integration Platform
13
Tooling Summary
In summary, the key elements in this section include:
Minimum
o The IDE needs to be responsive and stable.
o Exceptions are easily identified and located in a central location.
o Some level of monitoring is provided by the platform.
Preferred
o The IDE exposes all operations required to build applications.
o A portal allows both technical and end users to discover exceptions and subscribe
to relevant alerts.
o Real time alerting or viable third party monitoring options are available.
Be cautious if:
o You need to download external libraries in order to use the vendor’s IDE.
o Exceptions are logged to disparate files located on a server’s file system and you
are encouraged to purchase a 3rd party log aggregation tool.
o If a vendor tells you don’t need a monitoring tool because nothing wrong will
happen.
5.2 Platform
The next major subject that we want to cover in this paper is the Platform. In this
context we consider the Platform to include the Runtime, how the Integration Platform
scales, provides isolation across other applications running in the environment,
Message formats, Industry Specifications, Transformations, Rules Engine and Business
Activity Monitoring.
Runtime
The platform must provide an ability to support a publish/subscribe architecture
without requiring installing 3rd party products or projects.
The platform must support complex messaging patterns including:
o Aggregators
o Splitters
o Convoys
o Scatter-gather
o First-in-first-out (FIFO) ordered delivery
o Content based routing
o Context based routing
o The platform abstracts the underlying communication protocol(s) from any
Workflow or Orchestration in order to promote loose coupling. You should be able
to swap in and out different adapters without having to update your Workflow.
o The middleware platform provides out-of-the-box durable messaging to prevent
message loss without putting the burden on the developer to implement custom
processes that need to leverage other messaging infrastructure.
Choosing an Integration Platform
14
Scalability
The platform provides the ability to scale up or out depending upon system
resource pressure.
The platform exposes performance indicators that provide visibility into system’s
performance. These settings can be subsequently modified in order tune the
environment and get more value out of existing infrastructure and licenses.
Tools are available that allow you to benchmark an environment.
Additional nodes can be added to the Integration farm with minimal, or no,
downtime.
Application Isolation
The platform provides mechanisms that prevent an application from
negatively impacting another application’s performance through a throttling
mechanism that is built into the platform.
The throttling mechanism is not dependent upon a specific protocol (i.e. REST
or SOAP). Traffic originating from a non-HTTP source can also be throttled (FILE,
Queues/Topics).
Applications can be deployed without impacting existing applications.
Message Formats
• The integration platform must support a variety of message formats including:
o XML
o JSON
o Binary
o CSV
o Fixed Width Flat Files
o SAP IDOCs
o Industry Specifications (see following section)
Platform Summary
In summary, the key elements in this section include:
Minimum
o The platform provides Publish/Subscribe capabilities out-of-the-box.
o Many popular messaging patterns are documented and supported by vendor
product.
o The platform provides durable messaging capabilities out-of-the-box.
o The platform allows you to scale up or out as required.
o A variety of message formats are supported
Preferred
o There is a decoupling of Workflow/Orchestration from transport.
o Additional nodes can be introduced with limited, or no, downtime.
Choosing an Integration Platform
15
o The platform provides application isolation so that one application cannot interfere
with another
Be cautious if:
o A vendor suggests running an application on its own server
o A vendor does not have a strategy for protecting one application from another
5.3 Transformations
Transformations, or mapping, is a fundamental requirement for any Integration
platform. Mapping tools are seen as productivity tools as they reduce the amount of
custom code that is required when turning a source message structure into a
destination message structure.
Some Mapping tools include, pre-built functions that allow you to leverage commonly
used operations to further accelerate development. Inevitably, you will run into a
unique scenario where you may need to write some custom code to address your
requirement. Having a mapping platform that offers an extensibility point to call out
to manage code is one way of addressing these types of requirements.
• The transformation engine must provide support for large messages which are
greater than 20 MB.
• Multiple Source documents should be able to be transformed into a single
destination message.
• The mapping solution must include pre-built reusable functions that accelerate
development, including logical evaluations, string manipulation, date and
math functions.
• Mapping complex structures, including parent- child relationships and
message flattening are easily demonstrated.
• Leveraging managed code or XSLT, from a map, for more complex logic is
easily achievable.
• Reference data lookups such as Master Data from internal or external data
sources is supported.
Transformations Summary
In summary, the key elements in this section include:
Minimum
o The platform provides support for large messages.
o There are extensibility features to solve more complex mapping i.e. Managed
code, scripting, XSLT.
Preferred
o Reusable mapping functions are bundled by the vendor to accelerate map
development.
Be cautious if:
o A mapping solution contains memory leaks.
o Your existing messages cannot be mapped by the vendor in a Proof of Concept
phase.
Choosing an Integration Platform
16
5.4 Adapters
Note: For the purpose of this discussion, we will use the term Adapter. Quite often,
vendors will use different nomenclature when describing Adapters or Connectors.
Within this paper, when we refer to code that connects to external systems, we will
use the term Adapter instead of a Connector.
Adapters are components that wrap a system specific connectivity implementation
into a developer friendly package that can be configured in order to communicate
with different systems. Some adapters may be classified as being Protocol Adapters
where they implement a specific protocol. An example of this would be an SAP
Adapter that is going to implement SAP’s RFC communication protocol. Another type
of Adapter may be an Application Adapter, such as SalesForce. SalesForce is going
to expose an API(s) that can be consumed directly using REST or SOAP, but there is
usually some additional plumbing required in order to establish a connection and
exchange data. This additional plumbing often times includes dealing with security.
Certainly a developer can build this security related plumbing themselves, but
wrapping this up in an out-of-the-box Adapter provides productivity advantages for
customers.
LOB Out of Box
Out of the Box Adapters are included in your base license. Some vendors will charge
you a premium to use some Enterprise Adapters and some include them in the box.
These Enterprise Adapters that may incur additional costs include ERP Adapters such
as SAP, Seibel, Oracle, and Microsoft Dynamics. Every organization is going to have
different requirements when it comes to out-of-box Adapters so mileage will vary, but
keep an eye out for the following Adapters:
SQL Server
Oracle Database
SAP
Seibel
PeopleSoft
Dynamics AX
Dynamics CRM
SharePoint
The other considerations that should be monitored include:
Has the vendor kept up with support for new versions of LOB Platforms?
Are perquisite libraries required in order for the Adapter to function? If so, are
there any additional licensing costs involved?
Choosing an Integration Platform
17
SaaS
Within recent years, the ability to communicate with SaaS platforms has become
more and more important as organizations move workloads to the cloud. SaaS
platforms usually expose their own API or Services that allow external systems to call in
order to exchange data. So by nature, any integration platform that provides a SOAP
or REST adapter should be able to integrate with SaaS platforms. While consuming a
SOAP or REST endpoint gets organizations part way, it isn’t enough. Organizations
purchase Integration Platforms to accelerate development. Having to consume
services in a redundant manner does not improve productivity. Also, SaaS platforms
usually have their own nuances when it comes to security. This requires organizations
to figure out this security on their own, which takes time away from solving the business
requirement.
When you consider security and adding redundant references to SaaS services and
APIs, there is still room for SaaS adapters. The Integration platform needs to manage
versioning of APIs that are being exposed from these SaaS platforms in order to
achieve backwards compatibility. If a SaaS provider updates their API, is the
Integration Platform going to provide an updated SaaS Adapter in order to take
advantage of the new capabilities exposed by the SaaS platform? Vendors also need
to be able to provide a simple security model and manage sessions across each API
call. For a developer it should be as simple as providing a URI, Username, Password,
Shared Secret or equivalent. It should be a configuration based experience that
involves little or no coding.
Each organization’s requirements will vary depending upon which SaaS systems are
in use, but consider the following SaaS adapters when evaluating SaaS connectivity:
SalesForce
Workday
ServiceNow
NetSuite
SAP SuccessFactors
Dynamics CRM Online
Office 365
Marketo
SugarCRM
Concur
Twilio
Dropbox
• Box
Choosing an Integration Platform
18
There are some additional SaaS adapters that you may want to consider depending
upon your line of business. These connectors target social media platforms, including:
Google Docs
Custom Adapters
Inevitably, customers will run into a requirement where an Adapter is not included out-
of-the box and they must look to a custom solution. Custom Adapters usually address
vendor gaps and often times are the result of connecting to legacy systems or brand
new systems.
When considering a solution for Custom Adapters, the following aspects should be
considered:
An SDK is provided by the vendor which greatly reduces the amount of custom
code required.
Samples are provided by the vendor.
Best practices and guidance is provided by the vendor.
• A testing framework is provided by the vendor that aids in the validation of the
Adapter.
Protocol or Application Adapters
A Protocol Adapter is something that has been built upon a proprietary protocol vs
an Application Adapter that has been built upon an open protocol, but has been
tailored to be specific for a distinct application.
An example of a Protocol Adapter is an ERP Adapter. For example, SAP has
implemented a specific RFC protocol that is proprietary to their platform. It is for this
reason that libraries are required by middleware platforms in order to connect.
Similarly a java application would use a JCO connector and a .Net application could
use a .Net connector in order to establish communication with SAP. Whereas an
example of an Application Adapter is SalesForce. SalesForce exposes APIs via SOAP
and REST. Both SOAP and REST leverage HTTP for communication. There is nothing
stopping a developer from consuming these REST or SOAP endpoints without a
specific adapter. The value-add of leveraging an Application Adapter is that it should
accelerate the development experience. Usually this acceleration is the result of
simplifying security, testing connectivity or exposing metadata that will allow for
streamlined data mapping.
In general the industry is moving away from Protocol Adapters in favor of Application
Adapters. This, in part, is related to the explosion of SaaS applications, SOA and the
use of open standards.
Choosing an Integration Platform
19
Both Protocol and Application adapters have their place, but having too much of
one type of adapter is not a good indicator. With this in mind, here is some guidance
in this area:
A vendor who only provides Protocol or Application Adapters is probably
missing some much needed Adapters. Failing to expose Protocol adapters
likely means they are missing some key legacy systems like SAP or message
queuing bridges. Failing to expose Application Adapters likely means the
platform is not very modern and not capable of efficiently connecting to SaaS
systems.
When evaluating Protocol and Application Adapters, really focus on the
security aspects of the Adapter. If you are living in a Microsoft environment,
how does the Adapter leverage Windows Authentication/NTLM/Kerberos? Is
there any consistency in their security paradigm across the Adapter set?
How does Federation Services such as Active Directory Federation Services or
Windows Azure Active Directory play a role in adapter authentication?
Is OAuth 2.0 supported? Are you able to leverage delegated authentication
providers to avoid spreading passwords around the web?
Level of Implementation
The purpose of this section is to provide some cautionary guidance around adapters.
For some Vendors, they will include a laundry list of adapters within their platform. One
thing to consider when performing your evaluation is to determine whether or not all
the adapters have all operations built, or at least the operations that you are
interested in.
The following is an example of what can go wrong when a customer runs into a
situation where the adapter they thought they were getting didn’t live up to
expectations.
A customer thought they were getting a rich adapter but when they tried to
implement their solution they found the adapter didn’t support the operations they
needed. This gave them the choice of extending the adapter themselves or paying
for the vendor’s professional services team to extend the adapter.
The customer chose to extend the adapter themselves and this was initially successful,
but when they subsequently upgraded to the next version of the adapter they had
additional problems because their custom extensions were no longer compatible.
Initially, the adapter they used, looked like a good choice in the shop window, but
the net result of the customers experience was a sizable unexpected cost and some
project delays combined with an increase in the overall cost of ownership of the
solution.
Choosing an Integration Platform
20
Adapters – Summary
In summary, the key elements in this section include:
Minimum
o Both LOB and SaaS based adapters are available.
o An ISV community provides supported adapters that are certified by the
Integration Platform vendor.
Preferred
o Out-of-the-box, the core adapters that you need for your organization are
included.
o The vendor has developed a program around its adapters and ensures that
adapters are compliant with the latest versions of the software that it is connecting
to.
o A simple, easy to use, SDK is available that allows organizations to build their own
adapters.
Be cautious if:
o A vendor has adapters that contain partial implementations.
o If a professional services organization has written a lot of adapters, but they have
not been transitioned into the core platform.
o A vendor struggles with specific authentication schemes i.e. NLTM/Windows
Authentication.
5.5 Business Rule Engine Rule Engines provide a logical and physical separation of business rules, from
integration logic, that will be executed within an Integration Platform. Consider a
situation where you are processing an insurance quote. The logic rules that are used
to generate the quote should be separated from the actual Integration Platform.
Business Rule Engine guidance:
The Integration Platform provides an ability to leverage a Rules Engine
framework out-of-the-box.
The Rules Engine allows for a clean separation of Rules from the middleware
runtime. This allows rules to be updated without updating middleware
application.
Rules and Vocabularies are easy to use through an intuitive interface.
Business Rule Engine – Summary
In summary, the key elements in this section include:
Minimum
o The platform provides support for Rules Engines.
Preferred
o Rules and Integration logic are managed and executed independently of each
other.
Choosing an Integration Platform
21
Be cautious if:
o The Vendor does not have proven approach to supporting Rules Engine.
o The Rules Engine is supported by a different vendor.
5.6 Business Activity Monitoring Middleware inherently does not have a user interface, yet some business users want
visibility into the state of their business process. No longer is letting people know when
something goes wrong enough.
As an industry we continue to move towards a customer self-service model, so why
not provide visibility to the business for specific invoices or claims? Even better, why
not let end users subscribe to different business events as they occur? This is where
Business Activity Monitoring comes into the picture.
A Business Activity Monitoring solution should include the following features:
The ability to configure data elements that need to be captured as part of a
business process and display data elements to end users through a user
interface such as a portal.
Milestones can be configured that will update the state of a business process
as different events take place.
An exposed API that allows a developer to communicate with the BAM
platform from the edge of the integration platform or from other platforms such
as a Web Application.
A federated approach that allows BAM events to be tracked from a cloud
environment to an on-premises environment and vice versa.
Business Activity Monitoring is performed asynchronously and does not have an
impact on the performance of messages being processed.
Other Business Intelligence tools can easily consume the Integration Platform’s
BAM data.
Business Activity Monitoring – Summary
In summary, the key elements in this section include:
Minimum
o The integration platform needs to provide analytics to a business audience that
provides insight and visibility into the business process (es) that the middleware is
supporting.
Preferred
o The BAM environment can span different applications and deployment models i.e.
Cloud and IPaaS.
Be cautious if:
o Business Activity Monitoring is synchronous and therefore degrades performance.
o The vendor suggests logging to files instead.
o The vendor suggests using external products or projects to satisfy requirements.
Choosing an Integration Platform
22
5.7 API Management API Management is becoming a very loaded term within the industry. For the purpose
of this document we will focus on the following characteristics of an API Management
Platform: Service Virtualization, Identity Management, Security, Governance and
Policy Management, Analytics, Metering, Service Level Agreements (SLAs), Scalability,
API Design and Developer collaboration.
Service Virtualization
The API Management platform provides a layer abstraction for existing SOAP
and REST services.
A repository exists for these APIs that allows a developer to easily identify and
consume these APIs.
Service(API) abstraction is available for both On-Premise and Cloud based
services
Identity Management
• Support is available for a variety of Identity Brokers including support for Oath
2.0, SAML, Open ID, Active Directory, Active Directory Federation Services and
Azure Active Directory.
Security
• APIs can secured at a very granular, operation level, leveraging a variety of
Authentication and Authorization schemes.
Governance and Policy Management
• Policies are introduced and managed through configuration to control the
following aspects of the API including:
o Rate limit based throttling.
o Control messaging formats XML – JSON and JSON – XML.
o Authentication and Authorization
o API Key Management including Developer Self Service.
o API Versions
o API Header manipulation and management.
Analytics, Metering and Service Level Agreements
Near real time availability of service utilization is available.
Visual drill downs are available that allow users to slice and dice data based
upon geographies, time frames, products, and authentication.
Track analytics per API, API Key and API Version.
Monitory Service Level Agreement compliance
Choosing an Integration Platform
23
Scalability
5.7.6.1 Performance
As load increases, the API Management platform can scale through
configuration and without calling up a sales representative or operations team.
5.7.6.2 Caching
Configurable Caching can be enabled to prevent unnecessary round trips
when data remains relatively static.
• Cache eviction is automatically performed and updated data is automatically
loaded into cache.
API Design and Developer Collaboration
The API Management platform supports importing or designing APIs based
upon well-known specifications including WADL, Swagger and RAML.
• Orchestration or code level scaffolding is possible through importing an API. For
example, can you import a Swagger specification and automatically generate
code, orchestration and workflow stubs in your backend.
API Management – Summary
In summary, the key elements in this section include:
Minimum
o API Management tool is able to convert XML to JSON and vice versa through policy
configuration.
o Multiple security schemes are supported by the platform.
o Caching and rate limit policies are supported out-of-the-box.
o The API Management solution can proxy both On-Premises and Cloud based
endpoints.
o Rich analytics are exposed by the platform.
Preferred
o Both SOAP and REST services can be proxied.
o The API Management solution supports importing design-first specifications such as
Swagger, WADL or RAML.
Be cautious if:
o Scaling cannot be provisioned through self-service means.
o Significant upfront investments are required when establishing an API practice.
Choosing an Integration Platform
24
5.8 Deployment Models
Installation and Configuration
Installation can be a tricky subject, especially as you scale out your environment and
account for requirements such as high availability and redundancy. Some platforms
claim a very simple installation process, but it is important to take into account just
how many features are in the box. Obviously, the smaller the feature set, the smaller
the installation. Installing Notepad is going to be simpler and faster than installing
Microsoft Word. The purpose of this section is to discuss, how comprehensive the
installation packages are, whether the instructions are well documented and
understood and whether components can be installed in an automated fashion.
The installation process must include self-contained installation packages that
include necessary components or automatically reaches out to sources to
download these components.
The deployment, or portions of it, can be automated and scripted to ensure of
repeatability across environments.
Documentation is available and comprehensive for all components included
in the installation package.
Account requirements are described in a clear and concise manner which
clearly identify the separation of duties across components.
Wizard based interfaces guide users through the process and clearly indicate
when an environment has been successfully installed and configured. Having
a Wizard also ensures of a consistent and repeatable installation experience
which lowers complexity and reduces troubleshooting.
Deployment Options
New Integration Platform deployment paradigms continue to emerge. Customers
have an increasing number of options as to how and where they deploy their
interfaces. Within this section we will discuss the following deployment models:
On-Premises/Private Cloud
Infrastructure as a Service (IaaS)
Integration Platform as a Service (IPaaS)
Hybrid
On-Premises/Cloud
The traditional deployment model is installing the Integration Platform within your own
Data Center. The customer organization is typically responsible for maintaining,
monitoring and supporting the platform.
Infrastructure as a Service (IaaS)
The next evolution in hosting Integration Platforms is within someone else’s data center
where they are providing the underlying infrastructure, but you are responsible for
maintaining the application.
Choosing an Integration Platform
25
Integration Platform as a Service (IPass)
IPaaS is the variant of Platform of a Service where a vendor is responsible for providing
the Integration Platform and Infrastructure. The Infrastructure is provided, but
abstracted away from customers. Instead logical containers, or units, are used to
describe system resources that support the underlying integration applications and
interfaces.
Hybrid
Hybrid refers to the ability to bridge a cloud hosted integration platform with on-
premises assets or even an on-premises Integration Platform. There are often variations
in the techniques to solve hybrid requirements. It may be solved through ground to
cloud (or vice versa) messaging through a messaging broker, it could be an agent is
installed on-premises which communicates with the cloud platform. Networking
technologies may also be used to create secure tunnels between the on-premises
data center and the cloud entity. Another technique includes Relay communication
that can be deployed without the alteration of firewalls.
Every customer scenario will have many variables that will lend themselves to a
particular deployment model. Some variables may be driven by regulatory
requirements, cost structures and ultimately comfort levels. With so many variances of
customer requirements, at a high level, these principles should be considered when
evaluating different deployment models.
The platform should provide the ability to run integration applications in a
variety of different deployment scenarios including On-Premise/IaaS/IPaaS
without re-coding the application. This may also include lift and shift models
where you transition an existing deployment into a new model.
The On-Premise platform must be able to exchange messages seamlessly with
the Cloud platform and vice versa.
On-Premise services can be exposed utilizing a variety of different approaches.
These approaches may include network level integration, application
integration without opening firewall ports or leveraging a software agent
deployed on-premises.
If a Vendor provides a cloud based offering, how many Data Center locations
do they have? Is there any affinity to a particular region? i.e. A customer in
Europe runs their workflows in Europe, but must use a queuing system that is
hosted in US East.
Cloud based provisioning process does not require the intervention of a Sales
or Sales Operations teams. Cloud based services can be self-provisioned
through a Web based portal and a management API.
Cloud based services are metered at discrete unit such as per hour or per
minute to enable customers to leverage a “Pay as you Grow” model. This
model also enables “Cloud burst” scenarios without locking customers into
capacity for terms that are not required.
Choosing an Integration Platform
26
Deployment Models - Summary
In summary, the key elements in this section include:
Minimum
o The vendor provides multiple deployment models that allow customers to choose
the right configuration based upon their needs.
o Scaling in a cloud environment can be done through self-service tools.
Preferred
o Integration solutions are completely symmetric and can be deployed to any
platform model without re-compilation or modifications.
Be cautious if:
o Certain artifacts, or features, are only supported in one type of platform.
o Specific IPaaS components have an affinity to a specific data center or region.
5.9 Environment Management Environment Management is one of the most important, yet most neglected, areas of
many projects that don’t run smoothly. Getting the right kind of setup in place for
dev/test/production environments can be a key factor which can affect the
perception of how easy the Integration Platform you choose can be perceived to
work with.
As an example, if during the test iterations of your projects the Integration Platform is
perceived to be fragile and un-reliable then it can have a big negative effect on the
whole perception of integration in your organization.
When choosing a vendor for your Integration Platform you should carefully consider
what environments may look like, how they will be built and managed. There have
been large shifts in the industry around environment management in recent years as
organizations have tried to adopt practices such as DevOps but regardless of the
approach you take some fundamentals remain true. The fundamentals include:
Environment Isolation
Environment Isolation is a common challenge in managing environments. Some of the
worst problems in the implementation of Integration Platforms during the dev and test
stages come from scenarios where test environments are crossed. Imagine a bit like
in Ghost Busters where you don’t want to “cross the streams”. In environment
management it can get really painful when your system test instance of the CRM
application is pointing to the UAT instance of the Integration Platform which is then
pointing at the UAT instance of the ERP application. This makes understanding the
environment setup difficult and an effect of that is that deployment and
management becomes error prone and this leads to fragile and unreliable
environments.
The best way to tackle this problem is if it is easy to provision new environments and
decommission old environments. This means you can keep them clean and matching
(e.g.: UAT CRM UAT Integration UAT ERP). Having a vendor which makes it easy
for you to do the provisioning through self-service is important and also not having to
Choosing an Integration Platform
27
share multiple instances of the same components deployed to the same container
but supporting different environments.
Configuration Setting Management
Configuration management settings are all about the configuration values that
change when your integration solution is deployed into a given environment. As an
example, in the system test environment I might point a REST adapter to the system
test instance of my CRM application, whereas in the production environment I would
point the REST adapter to the production instance of my CRM application.
In the deployment section we talked about the importance of this separation
between the code and configuration which makes up your solution. This is where the
realization of that benefit becomes important. The Integration Platform is likely to have
a lot of dependencies because it is the piece that connects systems. To do
environment management well, you need to be able to easily visualize these
dependencies in terms of configuration settings and then have an approach of how
to apply these to different environments.
In this space we are looking for vendors who understand the importance of this
challenge and who have features in their product to help make this easier and take
away some of your pain.
Cost & Licensing
In the world of the cloud, cost has become a much bigger factor than it used to be.
Perhaps in the world of the Microsoft Integration professional we took for granted in
the past that MSDN could support our dev/test scenarios significantly reducing those
costs and we simply needed to get some virtual machines and the rest was pretty
much free. In the cloud world and the world of IPaaS and pay as you go these non-
production environments still consume resources and the model of free licenses for
servers does not really fit the cloud unless you stick with IaaS only approaches. With
this in mind you need to think about what your non-production environment costs will
be.
As an example one customer was considering switching from their server based on
premise integration environment to an IPaaS offering. When comparing the costs the
increase of cost for IPaaS non production environments was double what they were
previously paying for the on premise server equivalent.
Environment Management – Summary
In summary, the key elements in this section include:
Minimum
o The products can easily be configured to point at different systems during
deployment.
o The products can be managed in isolation and self-contained e.g. 1 instance per
environment.
o There are optimized costs for non-production environments.
o The deployment models for the products support good automation.
Choosing an Integration Platform
28
Preferred
o Non-production environments can be turned off and on as a way to save money.
o Provisioning and de-provisioning environments through self-service.
Be cautious if:
o You are not clear around the costs for non-production environments.
5.10 Industry Support Industry Specifications are built to reduce the uniqueness, and costs, across businesses
that participate within the same domain. If businesses are involved in trading partner
scenarios, they do not want to construct unique messaging contracts for each of
these different business partners. It is therefore in their best interests to conform to an
industry standard which will bring some commonality to these different businesses.
From an Integration Platform perspective, supporting a wide variety of industry
specifications will further accelerate your team’s development effort. If your existing
platform does not support EDI and you have to build your own or try to bring in a 3rd
party component, your development effort will take longer and the less value you are
extracting from your Integration Platform.
The platform must include support for popular industry standards such as:
o HL7/HIPPA (Healthcare)
o EDI
o SWIFT (Finance)
o FIX (Finance)
o Pidex (Oil and Gas)
o MultiSpeak (Utilities)
• These artifacts must be published by the vendor on a regular cadence and are
available shortly after the specification changes. Where this is not possible, a
partner eco-system is able to fill in any gaps.
Industry Support – Summary
In summary, the key elements in this section include:
Minimum
o The vendor understands and values supporting Industry Specifications.
o The vendor ensures of timely updates to Industry Specifications.
Preferred
o The vendor participates in Industry specific working committees.
Be cautious if:
o The vendor does not have out-of-the-box support for the industry and is
encouraging you to build custom solutions.
Choosing an Integration Platform
29
5.11 Community and Eco-System No vendor can provide it all. A vibrant community empowers customers to build
solutions required to deliver business value. A vendor with a small community should
be carefully scrutinized, as should a company that relies too heavily on their
community without making significant investments themselves.
Communities come in many forms such as blogs, forums, wikis, free or open sourced
tools, commercial tools, free or paid events, webcasts, and have a designated
experts group such as a MVP, ACE or Champions.
Independent Software Vendors (ISVs)
Does the vendor have ISVs building solutions on top of their platform? These
solutions may address industry vertical gaps or enhance an existing feature set.
• How engaged is the vendor with its ISV community? Does it embrace, support
or endorse these partners?
Blogs
An energized eco-system should be happy to and excited to blog about new
announcements, features and solutions that others can benefit from. Does an
active blogging community exist where announcements, feedback and
solutions are available?
Do aggregators or RSS feeds exists for these sources?
Wikis
A vendor-provided Wiki allows the community to enhance existing
documentation or outline alternative approaches to solving a particular
problem.
• In the event there is an error, it provides an opportunity for that feedback to be
recorded without it falling into black hole such as an unmonitored email alias.
Webcasts, Recordings, Webinars
New features, announcements, tutorials captured on video and made publicly
available.
• These channels are open and do not sit behind a paywall.
Forums
Peer based support is available and monitored by vendor employees and
influential community members.
Influential participants are recognized as experts so that people who are asking
questions are confident they are getting a reliable response.
Events and User groups
Community hosted Events and User groups are extremely powerful measure of
how vibrant a community is. Events may be in the form of paid or free events
that are supported by sponsors. Are regular community events hosted? Do
consolidated lists of these events exist?
Choosing an Integration Platform
30
• User groups are also very beneficial as they allow for local networking and
collaboration amongst local companies. These meet-ups also provide the
opportunity for people to develop other skillsets such as public speaking in less
stressful environments.
Community Tools
• Community Tools that accelerate or simplify a task are also signs of a strong
community. These tools may include automated deployment frameworks,
automated testing suites, performance benchmark tools, and adapters.
Community and Eco-System – Summary
In summary, the key elements in this section include:
Minimum
o A vibrant community and ISV community is easily identifiable.
o The vendor promotes and recognizes influential people within its community.
Preferred
o The vendor collaborates with its community members and publishes collateral on
public sites using a variety of media.
Be cautious if:
o The vendor is the only entity talking about its product.
o Wild claims about the vendor’s population are unsubstantiated.
o Vague metrics are used to describe its popularity or market penetration.
5.12 Platform Implementation
Getting Help to Implement your Integration Platform
Integration Platform implementations can vary in size from a small implementation
which can support only 1 or 2 interfaces to very large implementations covering an
entire enterprise. One of the secrets to success is in making an investment which is of
the right size, but can grow as your use of integration grows. With this in mind it is
important to consider how the skills and experience within your team will be able to
leverage your chosen platform and where you might go to seek help.
Ideally you will develop the skills of your internal team as you invest in your new
platform so your core team can reuse the platform for future projects. At the same
time, unless you already have skills or experience in the chosen platform, it could be
worth considering your options for external help and guidance. A common mistake is
that organizations just get some ‘java/.net people’ as it’s the same underlying
technologies anyways and they will figure it out. In the same way that a DBA thinks
differently to a UI developer, an experienced integration person understands that
integration is not the same as application development. The key difference with
integration is that you need to be aware of all of your external dependencies and
have a degree of understanding of how they work and how you can build a solution
to make these external systems work together effectively. You need to understand
how to leverage the API or the CRM system and how it maps to the ERP system and
Choosing an Integration Platform
31
where the gaps are in data and functionality and many other similar considerations.
This is all on top of the skills required for your chosen platform.
With this in mind we would recommend you consider what options you might consider
for getting help if you chose a particular vendor. This may be in the form of professional
services from the vendor itself. It could be via system integration partners the vendor
has, or independent experts or people in the community around the vendor’s
products.
One word of caution is that we have seen some cases where organizations have
worked with an implementation partner to create or implement the platform but no
effort was put into developing the internal team to manage, support or further
develop the platform. That can lead to a lock-in with the implementation partner. This
can be fine if you want to outsource your integration and have a long term
partnership but can also have some problems if you are unhappy with the vendor
sometime in the future. Generally we see the most successful implementations are
usually done with blended teams seeking experienced or temporary man-power.
These resources may be from external sources but complimenting that with the
business context knowledge provided by an internal team is usually a winning
combination.
Platform Implementation – Summary
In summary, the key elements in this section include:
Minimum
o The vendor has implementation partners you can work with.
Preferred
o The vendor has a professional services team you can work with.
o There are people you can approach in the community around these products to
seek advice on implementation or to engage with them.
o The vendor has training or training partners to develop your own team.
Be cautious if:
o The vendor wants to sell you the platform, but then is not interested in how you
implement it.
o The vendor or a partner wants to do the whole implementation, but not really
develop your internal team’s skills.
Choosing an Integration Platform
32
5.13 Commercial
Licensing Questions
Licensing is often times a difficult phase is the evaluation cycle. Every vendor will have
different motivations, formulas and models. Before signing any contracts, consider the
following:
Licensing costs are transparent and available online.
Cloud based offerings allow you to scale up and down without talking to an
Account Executive or Sales Operations team first.
Cloud based environments truly ‘Pay as you Go’.
Non-Production environments require licenses that are free or at a significant
discount compared to the Production environment.
• Disaster Recovery environments require licenses that are free or at a significant
discount compared to the Production environment.
Customers
Understanding a vendor’s client base is some important due diligence that must be
performed. While it is not possible to develop requirements in this area, it is important
to understand the following:
How many paying customers do you have?
Do you have any reference customers in my domain and location?
How frequently do you provide updates and fixes?
What reference-able whitepapers have you published that are relevant to my
industry or geography?
• What customer(s) with similar use cases can I speak with to better understand
their experiences with your tools?
Support
Inevitably there will come a time when a customer organization needs to leverage
the vendor’s support system. Problems may arise in the development phase of a
project or in the testing phase when some lag may be tolerated. But, there may also
be situations where a mission critical application is down and urgent support is
required. Therefore, it is import to understand the following:
The vendor’s support model is flexible and offers different tiers of service based
upon well-defined Service Level Agreements.
A support engineer is available for each call.
A Screen sharing activity is available with every support call if the customer
wants it.
A customer portal is available where organizations can log tickets and review
status and resolutions.
Choosing an Integration Platform
33
Success and Challenges
Every vendor has had some excellent implementations and some poor
implementations. It is very useful to understand what made an implementation
successful and implementation where they have learned valuable lessons. Consider
asking the following of the vendor:
Ask the vendor where they thrive and where they are weak. If they have no
weaknesses, then proceed carefully.
Query the vendor on some recently learnings. What has the vendor learned
from some of their recent projects? What has surprised them the most about
how their product is used in the marketplace?
• Do they have any recent product benchmarks that they are willing to share
that are applicable to your scenario?
Platform Investments
Since the Integration Platform landscape is changing so quickly, it is very important to
gauge the level of investment that vendors are putting into their platform. The
following questions may aid in better understanding where a customer is placing
investment.
Where is the vendor investing in their platform?
o SaaS Connectivity
o Hybrid Connectivity
o Industry Specifications
What mega-trends are they seeing?
o Containers
o Microservices
o Internet of Things (IoT)
o Complex Event Processing
o Application Integration Marketplaces
How will they modernize existing on-premises assets?
o Legacy Data Stores
o Enterprise Resource Planning Systems (ERP)
What tools are they providing to move existing customers onto modern
platforms?
o Data mapping migration
o Workflow migration
o Endpoint configuration migrations
o Trading Partner migrations
Choosing an Integration Platform
34
Aggressive Sales and Marketing teams
Sales teams have a job to do and that is to push product. Regardless of industry, there
is a common theme when it comes to selling a product. Similarly, marketing teams
exist in order to construct the right messages that really enable sales teams.
The biggest challenge, as a customer, is to cut through the hype and really ask the
right questions when it comes to be boisterous claims that a vendor may be making.
It may be as simple as asking for more details, indicating that you would like to better
understand the inner mechanics of a solution. If you can’t get the right information
that you are looking for, then ask to speak with a more technical resource. If the
vendor cannot come up with a person that can explain a reference solution at the
level of detail that you are interested in then it is a sign that the reference solution, or
whitepaper, is far-fetched.
Commercial – Summary
In summary, the key elements in this section include:
Minimum
o Licensing terms are transparent and easily verified.
o Customer whitepapers and references are available and can be verified.
Preferred
o Different support models are available and can be tailored to a customer’s specific
need.
Be cautious if:
o A vendor is not transparent about licensing terms.
o Sales teams are very aggressive and are not interested in discussing requirements
in great detail.
o Trial software is not available to be installed on the customer site.
6 Looking Forward
This topic is always difficult to predict, but there are definitely some leading indicators
to consider including how active is this organization in the industry? Are they working
on common standards in order to improve interoperability? Is their platform evolving?
Are they focusing on disruptive technologies like Big Data, Complex Event Processing,
MQTT, AMQP, Containers and Microservices?
Since the future is always difficult to predict, it is impossible to measure this. However,
one consideration is whether the vendor has roadmap that they are willing to share
with customers, analysts and the broader integration community.
6.1 Marketplaces Another phenomenon to watch is Integration focused Marketplaces. Marketplaces
emerged for mobile platforms within the past decade. It is anticipated that this
concept will be adopted by Integration vendors as a way to centralize efforts made
by the vendor, ISVs and the community. Having a centralized location where
Adapters can be hosted, distributed, and reviewed can be a great enabler.
Choosing an Integration Platform
35
While this area is very new in the integration space it is difficult to project requirements,
but there are some questions to consider when evaluating a vendor:
Who is moderating the Marketplace?
Where are support requests routed for Adapters?
o To the Integration Platform vendor?
o To the author, or publisher of the Adapter?
What is the SLA for support requests?
What is the pricing model for the Adapter?
o What percentage is retained by original author/publisher?
What quality checks are imposed on the Adapter publishers?
6.2 Microservices At the Integrate 2014 Summit, Microsoft unveiled its plan involving Microservices. We
are witnessing the beginning of the next evolution of Integration Platforms. Moving
forward, there will be a preference to adopt a series of lighter-weight services that
can be deployed independently of each other. This will eliminate, or reduce, the
dependency on large monolithic platforms.
The winner in this space will be the vendor that is able to stich these different
Microservices together without compromising the requirements that we have
discussed in this document. Having the ability to provide durable messaging over a
variety of endpoints is something that isn’t going to go away regardless of the
architectural style that was used to implement it.
As we move away from shared dependencies in favor of more isolation, the ability to
manage all of the different version of components will also be extremely important. It
may be quicker and easier to publish a new version of a Microservice, but at some
point version sprawl will catch up with organizations if they are not managing it
somehow.
7 How does Microsoft stack up?
An Excel spreadsheet has been included as part of this whitepaper that goes through
each requirement within this paper and provides a score out of 5. There are over 110
requirements that have been created so the list is very comprehensive. In addition to
the numerical score, some insight has been provided that identifies how this score was
established and any gaps that contributed to some points being lost.
The following chart provides a high level view of the Microsoft results. Please refer to
this spreadsheet for an in-depth analysis of how Microsoft stacks up.
Choosing an Integration Platform Matrix
Choosing an Integration Platform
36
The approach that has been used for scoring is based upon the Microsoft stack. This
stack includes the following technologies:
BizTalk Server
Azure BizTalk Services
Azure Service Bus
Azure Infrastructure as a Service
• Azure API Manager
The use of third party products, or open source projects, do not contribute to the
scores that have been assigned with the exception of the Community and ISV
requirements. In some areas where there are gaps, an ISV or community project may
have been referenced for completeness but it has not altered the score.
Knowing that every organization is going to prioritize certain requirements over others,
a weight column has been created that allows an evaluator to assign a value
between 0 and 1. For the purpose of this paper, all requirements hold the same weight
of 1.
8 Conclusion
The results for the Microsoft platform have come back strong. Some of the contributing
reasons for this is the maturity of the BizTalk Server product and the innovation that is
occurring within the Microsoft Azure platform.
0102030405060708090
100Adapters
API Management
Community and Eco System
Deployment ModelsPlatform
Tooling
Commercial
Microsoft
Choosing an Integration Platform
37
The challenge for Microsoft in the coming years is how they can simplify the migration
from historical On-Premises deployments into the newer Integration Platform as a
Service (IPaaS) offering.
Microservices also provides some tremendous opportunities to really put Microsoft in
the driver’s seat moving forward. BizTalk has the luxury of leveraging the massive
investment that Microsoft has made, and will continue to make, in Microsoft Azure.
Azure provides several, foundational building blocks that smaller niche players just
cannot make based upon the capital investment and time to market that is involved
in establishing these services. The introduction of Microservices is no different in that
Microservices will be a broad fabric that many teams at Microsoft will be able to plug
into. The BizTalk team will have this same opportunity to introduce integration
components into this core platform.
9 Call to Action
When presented with an opportunity to select an Integration platform, the following
actions are encouraged:
A very exhaustive list of requirements has been identified within this document.
Obviously, some requirements are going to be more important for you than
others. Identify the requirements that make the most sense to you and then
assign a weight to them.
Pass your requirements spreadsheet off to the vendors that you are interested
in. Ask them to score themselves.
Invite vendors to participate in a bake off based upon the requirements that
have laid out. Evaluate vendors based upon what you have seen in their
demonstrations.
Compare your results with your vendor’s results. How large of a gap is there?
What type of transparency are you seeing? Can this vendor be trusted as a
valued business partner?
Collect Quotes for Software and Services.
Request 3 references from customers within your industry and ideally within your
region
Shortlist your vendors and then negotiate. Negotiate training, Dev/Test/DR
environments and Professional Services hours.
Make a decision
If a vendor is not willing to participate in this sort of exercise, move on. These
requirements are not unreasonable and any vendor who feels they have a
competent platform should be happy and eager to engage. On the flip side, you
may find a vendor who is looking to establish a very transparent partnership and is
interested in better understanding your requirements. These learnings may contribute
to new features or provide some education on another way of solving that
requirement.