integration daysbiztalk used for real time payments

Introducing Luup

© Luup International Ltd. #1

BizTalk Used for Real Time Payments

Luup provides payment services for mobile phones in emerging markets in Asia and the Middle East. This presentation will discuss challenges in building a mobile payment platform, which supports high transaction volumes and a minimum of latency, using BizTalk for internal workflow services in a SOA architecture.

Introducing Luup


Topics

• Introduction to Luup– Real-time Payments– The Luup System Platform– The Main Architectural Driving Factors– How Luup Uses BizTalk– The Development Teams

• Our Experiences and Challenges With BizTalk– Performance– Consistency– Maintainability– Automated Testing– Automated Deployment

Introducing Luup


Luup

• Started in Norway > 10 years ago• Development in Oslo, HQ in London, Sales Office in Dubai• Experience with several mobile payment products• Main product is a global remittance eco-system• Offers bank-grade mobile payment solutions• A managed service• An integrated banking platform• Our projects are centered around:

- Integration towards new partners and customers- Make business processes work on our standard platform - UI: SMS, Mobile App and Web

Introducing Luup


Real-time Payments Examples

• Remittances - customers send money to their families conveniently, quickly and securely using their mobile devices

• Domestic Payments - customers pay people, bills, charities and merchants

• Disbursements - customers receive their wages or social benefits into an account linked to their mobile device

Introducing Luup


Factors Driving Architectural Decisions

• Decided to use standard software as much as possible (came from a history where everything was in-house and custom made)

• Robust orchestration of services is important• Building a scalable platform is vital, since our

business model relies on a high volume of transactions to be profitable

Introducing Luup


Luup Payment Processing Platform

The Luup Platform

BizTalk

Core/BPM PartnerGateway

Core banking system

Luup Services

http http

Employee UI web site

End user UI web site / web services

Q QQ

Q

SMS server

Bank web services

Telco

Orc

hest

rati

onof

ser

vice

s

MsgBox

MsgBox PartnerGateway

PartnerGateway

Integration –

Typical usage of

BizTalk

QQ

Introducing Luup


The Luup Team

• Our internal developers:– Had no experience with BizTalk 2 years ago– Were mostly experienced .Net developers –

learned quickly– Now this background helps in deciding whether

code should be made in BizTalk shapes, C# components or services

• Offshore team with 3-5 developers:– Developed the Core/BPM application

Introducing Luup


End of Introduction

• Now over to challenges we have faced and experiences made during the last 2 years

Introducing Luup


Luup’s Experiences and Challengesin Using BizTalk

• Introduction• Performance• Consistency• Maintainability• Automated Testing• Automated Deployment

Introducing Luup



The Luup Platform - Recap

BizTalk


MsgBox

MsgBox

Core banking system

Luup Services

http http



Q Q Q Q

SMS server

Q Q

Bank web services

Telco

PartnerGateway

PartnerGateway

Introducing Luup


PERFORMANCE

Introducing Luup


Challenges

• Complex business process orchestrations– Many (10-20) web service calls in each workflow• Important for our design choices

• Low latency required for some scenarios– Serve requests from web and mobile app

frontends

• Performance tuning of BT is complicated!– Each system/setup is unique

Introducing Luup


Current Configuration

• Shared server for Partner Gateway and BPM• Active-Passive cluster– Due to licence costs, until the number of

transactions increases

• Separate hosts for receiving, sending and processing

• We use the defaults for most settings

Introducing Luup


Design Considerations

• The database will be our bottleneck– The database is the hardest component to scale– We need to limit database roundtrips

Introducing Luup


Latency Example

• Message box round trips (polling every 0,5s):– If 10 service calls:• Adapter polls for messages to send, orchestration polls

for response messages• 10 requests and 10 responses = 20 msg box roundtrips

– 20 msg box roundtrips * 0,25 sec = 5 sec (for polling msg box only)

– If 1 sec used on actual processing: 5 + 1 = 6 sec

• How can we avoid this latency?

Introducing Luup


Low Latency Considerations

• Alternatives for calling web services from workflows– WCF Adapters• Polling interval can be tuned down to 50 ms

– Use .Net code – «Service Caller»• Used in our solution

Introducing Luup


Using Adapters

• Pros – The default and recommended way when using

BizTalk

• Cons– Orchestration is persisted for each web service

call– Message Box roundtrips for request and response

message

Introducing Luup


The «Service Caller»

• Uses C# code to do SOAP calls• Two versions:– Old version using WebRequest– New version using WCF

• We only use it in the CORE/BPM application– Only calls to locally deployed services

• We use BAM for logging requests and responses

Introducing Luup


Service Caller - Advantages

• Low latency and DB load– No message box roundtrips– Orchestration is not persisted after the request is

sent

Introducing Luup


Service Caller - Disadvantages

• Need to be careful about when the orchestration is persisted– When replaying request we need to use the

original request ids– Generate request ids up-front and start using the

service caller after the ids have been persisted

Introducing Luup


Service Caller - Disadvantages

• We take control of resource usage throttling away from BizTalk– Threads are used while waiting for web service

replies– Use short timeouts (< 60s) and short retry

intervals

Introducing Luup


Summarizing Performance

• Performance tuning BizTalk is hard and time consuming

• Using .Net code to call web services has improved our latency

• We need to focus on performance in the near future as the number transactions increases

Introducing Luup


CONSISTENCY

Introducing Luup


Consistency


BizTalk


MsgBox

MsgBox

Core banking system

Luup Services

http http



Q Q Q Q

SMS server

Q Q

Bank web services

Telco

PartnerGateway

PartnerGateway

Inside ci rcle: Ide mpoten t servic es

called with serv ice calle r.

Introducing Luup


Consistency


BizTalk


MsgBox

MsgBox

Core banking system

Luup Services

http http



Q Q Q Q

SMS server

Q Q

Bank web services

Telco

PartnerGateway

PartnerGateway

Through circle: Async using ports

Introducing Luup


MAINTAINABILITY

Introducing Luup


Size of Orchestrations

• Complex business logic gives complex orchestrations– Results in orchestrations which are hard to:• Understand• Change• Test

Introducing Luup


Shapes vs C# Code

• Use shapes as much as possible?– Results in huge orchestrations which are • Hard to test• Hard to read • Has duplicated logic

– Pros• All the logic is visible in the orchestration designer and

debugger

– Call Orchestration can be used to split out orchestration parts

Introducing Luup


Shapes Example

Introducing Luup


Shapes vs C# Code

• Or use C# code as much as possible?– Keeps the orchestrations dense and focused– Use the orchestration only for controlling the

transaction (at a high level)• Which is the main reason why we needed BizTalk

– C# code is easy to unit test

Introducing Luup


Using Internal Contracts

• Vital for having a clean modular architecture• Hide gory details from external systems– Internal contracts which are easier to understand• Mapping complex external contracts• Greatly improves maintainability

– Anti-corruption layers• Keeps the core stable even when external services

change

• Small and reusable orchestrations

Introducing Luup




BizTalk


MsgBox

MsgBox

Core banking system

Luup Services

http http



Q Q Q Q

SMS server

Q Q

Bank web services

Telco

PartnerGateway

PartnerGateway

Introducing Luup



Cont racts

Cont ractsContracts

Reusable orchestrations

Introducing Luup


Summarizing Maintainability

• Keep orchestrations small• Don’t be afraid of using C# code – But be careful about orchestration persistence

when changing state in C# code

• Use internal contracts

Introducing Luup


AUTOMATED TESTS

I don’t usually test my code.

But when I do, I do it in Production.

Introducing Luup


Unit and Integration Tests

• Using unit tests for– Maps– C# code

• Integration tests– Using BizUnit for testing orchestrations

Introducing Luup


Testing Maps – The Built-in Test Func

Introducing Luup


Testing Maps – The Built-in Test Func

• Disadvantages of built-in test functionality:– Compiles maps inheriting from test base class• References test assembly

Microsoft.BizTalk.TestTools.dll which must be deployed to production

– Exceptions are swallowed• Hard to figure out why a test failed

Introducing Luup


Testing Maps – Our Solution

• Custom-made library• Calls XLANG component to do transformation• Fluent language for writing assertions

Introducing Luup


Testing Maps – Our Solution

Schema validation

Field validations

Introducing Luup


Testing Orchestrations

• Complex workflows– Requires automated tests

• Custom made fluent interface on top of BizUnit– Reuse of code– Terse tests which are refactor-friendly

• Currently we have around 350 BizUnit orchestration tests

Introducing Luup


Testing Orchestrations - Example

[Test]public void RemittanceSend_WhenInvalidBankTransactionDate_ShouldReturnGeneralError(){ const string invalidTransactionDate = "12/24/2012"; var testCase = new BizUnitTestCaseBuilder() .WithTestCaseSteps(SharedTestSteps.CreateAndCleanupDirectories()) .WithCreateXmlFileStep( new CreateXmlFileStepBuilder(@"TestData\MtoRemittanceSendRequest_ToAccount.xml", SharedConstants.Pgw.ReceiveDirectory + TestContext.CurrentContext.Test.Name + ".xml") .WithUpdateXml("SenderBankTransactionDate", invalidTransactionDate, "//*[local-name()='SenderBankTransactionDate']") ) .WithWaitForFileStep(SharedConstants.Pgw.SendDirectory) .WithXmlFileValidation(new XmlFileValidationBuilder(SharedConstants.Pgw.SendDirectory) .WithSchemaValidation(@"..\..\..\SharedBizTalk\Schemas\PartnerGateway\MtoRemittanceSendResponse.xsd", "http://www.luup.com/schemas/") .WithXPathValidation("ErrorId", MtoRemittanceSendResultCode.GeneralFailure.ToString(),

"//*[local-name()='ErrorId']") ) .Build();

var bizUnit = new BizUnit.BizUnit(testCase); bizUnit.RunTest();}

Wait for orchestration execution to finish

Verify the result message

Initialize the test input file

Introducing Luup


Build Server Integration

• After each code checkin:– Build packages– Deploy all BizTalk applications to a dedicated

BizUnit server– Run BizUnit integration tests

Introducing Luup


Summarizing Automated Testing

• The built-in unit testing functionality for maps is no good– Using BizUnit is a better choice

• Both maps and orchestrations can and should have automated tests

• Build server runs automated tests

Introducing Luup


AUTOMATED DEPLOYMENT

TFS Source Control

Build Servers

Code Changes Deployment

Test Environments

Production

Introducing Luup


Our Deployment Challenges

• Our solution consists of about 20 BizTalk applications

• 9 test environments• Time consuming deployments to Production

Introducing Luup


How We Do Deployments

• Each application use BizTalk Deployment Framework (BTDF)

• Using the Sprinkler framework to control deployment of multiple applications

• The build server creates installation packages each time we check in code– Also deploys and runs BizUnit integration tests

• Automated deployments to test environments

Introducing Luup


The Sprinkler Framework

• Open source on CodePlex• A set of PowerShell script built on top of BTDF• A bit immature– We had to make a couple of changes to the scripts

in order to make it work with Active-Passive clusters

• Great when having multiple applications and servers

• Great for automated deployment

Introducing Luup


Deploying to Production

• Active-passive cluster• Using Sprinkler for parallel deployment to the

nodes• IT Ops handle deployment– Developers should not touch the Production

environment

Introducing Luup


Summarizing Automated Deployment

• BTDF and Sprinkler are good frameworks• Automate your deployments• Use the same binaries when deploying to test

and production• Deploy early and often

Introducing Luup


Main Summary

• Flexible and scalable solution• Usage of BizTalk both as a traditional integration tool

and orchestration of services• Reduce latency by reducing message box roundtrips• Improve maintainability by creating a modularized

architecture with clean interfaces• Testing of maps and orchestrations can and should be

automated• Automated deployment is useful when having many

BizTalk applications

Introducing Luup


Questions?

integration daysbiztalk used for real time payments

Technology