end2end EA - Application Architecture

end2end Enterprise ArchitectureApplication Architecture

Author: Lars SorensenEnterprise ArchitectAugust 2016

e2eEA

Welcome to the end2end Enterprise Architecture training programme. This slide deck will complement the 1st, 2nd and 4th sessions and feed into the final 5th session when it all is pulled together in Enterprise Architecture. But it will also stand alone and will provide an insight into WHAT TO DO to deliver on business expectations and what NOT to do. Unlike other courses the E2E EA series gives you the hands on architecture decisions and deliverables, not just possible options or concepts but the tangible actions for your day to day work.1

ContentThe Enterprise Architecture StackSeparation of ConcernsReusabilityApplication Architecture EvolutionNon-Functional Application ArchitectureAvailabilityPerformanceMaintenanceScalability

Slide 2 of 31

e2eEA

I will take you through the Architecture Stack how does the layers hang together and what is the handshake from one to the other. We will talk a lot about Separation of Concerns because at the end of the day that is what architecture is all about. I will take you through reusability opportunities TOGAF talks about the Architecture Continuum, I will take it to the next level at tell you how to achieve that reusability. It is important to understand where weve come from and where we are going, understanding the application architecture evolution just might give you that AHA moment. Finally Non-functional requirements specifically in relation to Application Architecture its a packed program, so hang on and enjoy the ride.2

Positioning theEnterprise Architecture

e2eEA

Enterprise Architecture

Slide 4 of 31

e2eEA

This slide shows you that when you have 4 architects in a room you are likely to have 5 opinions. Some of these statements are poles apart from the slogan from Enterprise Architects to the comprehensive definition from Gartner, which is what you should keep in mind when working at the domain architecture level.4

Enterprise Architecture Thinking MACRO ENVIRONMENTINDUSTRY SCANSWOTPERFORMANCEFinancialCustomerInternal (current)Internal (long-term)Ref: Enterprise Architects Source: InternetVisionINFLUENCERSASSESSMENTMEANSENDSMOTIVATION MODEL

SERVICE MODEL

CAPABILITY MODELMARKET MODEL

StrategyPeople/Process

Information

Technology

Application

Blue-printing

ROADMAPRoad-mapping

GOVERNANCE

Governance

BUSINESS MODEL

e2eEA

This slide I have to contribute to Enterprise Architects. It is giving you the bigger picture that Application Architecture is a component of delivering the business vision and company strategy. When you do design your application it is important you keep this perspective in mind dont get into the rabbit hole and delivering for a single purpose.5

The architecture stack contains five layers with a stream of governance filtering through the whole stackThe governance includes:Architecture governanceData governanceSecurityStrategic governance

Architectural Domains

e2eEA

Getting into the Architecture Stack is difficult without writing a war and peace. The slide speaks for itself about how the layers are connected and in the end you cant achieve any one of the layers without all the others. Today we will only cover the Business Applications layer, but look out for the other releases in this training series, including the final one Enterprise Architecture which will bring them all together. Outside each layer and across the entire stack you must always observe Architecture Governance, it is applicable to each layer and we will cover the relevance to Application Architecture today.6

Slide 7 of 31Architecture PrinciplesPrinciples will protect the business from the opportunistic acts of a transient organization.Failing to set out architecture principles gives individuals the right to adopt any standard or method they choose.Foundation PrinciplesStrategic PrinciplesBusiness PrinciplesApplication PrinciplesInformation & Data PrinciplesTechnology Principles

e2eEA

Architecture principles provides a boundary for you application architecture and offer a prioritisation of options e.g. rent before buy buy before build. On request you can receive a sample principle definition upon which you can establish your own for your organisation, because one size does not fit all you have to adopt to business language and priorities.

7

Separation of Concerns

Slide 7 of 31

e2eEA

The most important principle within the application architecture domain, there are several others which we will touch on but your focus should be here.8

Business ProcessAbusiness processis a collection of linked activities with a defined starting and end point, which always will involve the consumption and/or production of data.A business application is the full or partial automation of a business process.

GoLeanSixSigma

e2eEA

The is no application without a business process Whoa I hear you say, we just need business requirements!!! Yes, but business requirements are the definition of a business process. As an Application Architect you must review the business requirements and make sure they are completed to a level where you can design an application to meet business needs. I propose you adopt the Lean Six Sigma business process model in your business requirements definition because it provides you with everything you need. The SIPOC gives you the first level of Separation of Concerns in Application Architecture it defines your input/output and your source/target as well as the business rules for the process itself usually business requirements only defines the business rules or process steps leaving you guessing as to what data to use in what currency, where to get it and where to deliver it to.9

Application StructureOrganised for ease of maintenance Optimised for Parallel DevelopmentCommon / reusable application building blocksTechnology independence without over-diversity

Randomly Ordered Application FunctionalityLayered Application Functionality

Clearly Structured Application Functionality

Slide 9 of 31

e2eEA

Your application structure is the second level of Separation of Concerns. In the early days of IT we would reuse sections of application logic from within other sections of the same (or other) applications some of you smile, you remember.. JUST DONT DO IT. Clearly structure your application and break it out into separate components if some of them could be reused by other applications Yes, we are talking SOA application architecture here, and it makes sense from an application architecture perspective, we will deal with the technology and data implications in separate sessions.10

Separation of Concerns

LeatherMeatMilk

Applications must be multi-layered with a clear separation of the Client, Presentation, Integration, Business Logic and Data Access Tiers.This means:No logic or data on the Client. Period.Only format validation and state logic on the Web ServerOnly Workflow, Orchestration and State in Integration ServerALL business logic resides on the Application ServerOnly referential integrity logic on the database server

e2eEA

In every walk of life you have Separation of concerns. Albeit from the same source you would not buy shoes from a butcher or milk from a cobbler! Likewise with your application. Each part of the application must be dedicated for that particular purpose. End-to-End your application architecture must cover all six tiers and as the architect you must make sure that each tier remains true to its purpose. No ifs and not buts you must stay true for your architecture to succeed in terms of performance, scalability, portability, availability, flexibility, maintainability. Thats your target and thats the reason you must stay true to the Separation of Concerns.11

When layers are physically separated, multi-layering allows a better balance load between the separate instances.PerformanceThe Business Logic Layer serves as the Proxy to the DatabaseScalabilityIntegration layer detaches workflow, orchestration and application dependencyScalabilityFlexibilityUltra-thin Client removes dependency of an ever changing Client device marketFlexibilitySeparation of Concerns

Slide 11 of 31

e2eEA

By observing Separation of Concerns in your application architecture you will improve several key objectives:Performance: When you clearly separate the layers of your application you can load-balance individual tiers e.g. you may deploy the business logic layer across multiple application servers and load-balance across them, but may only have two servers in each of the other tiers. Scalability: It is a lot easier and cheaper to scale application servers than database servers, so keep your business logic on the application tier. Also leave workflow controls in the integration layer, if you includes this in the application tier you will create extensive connectivity activity to and from the application server consuming processing resources that could be used for business logic processing.Flexibility: By managing orchestration, workflow and alerting in the integration layer you dont need to change the application when any of these highly volatile components changes. Likewise keeping business and process logic away from the Client tier gives you flexibility to keep up with the latest technology in the very volatile Client device market. 12

Layer technology independence FlexibilityTiers can satisfy different Security levelsFlexibilityAbility to replace or create alternate layers without affecting the applicationFlexibilityTiers can satisfy different Performance requirementsCost ControlSeparate development team per tierCost ControlResponsiveness

Separation of Concerns

e2eEA

Flexibility: by clearly separating your application tiers enables change of technology in one tier without affecting the others, likewise it is possible to implement stronger security controls on one tier than in another. Changes to either the front-end or back-end can be introduced without making changes to the application business logic and hence functional testing and deployment is saved.Cost control: Cheaper technology can be deployed in some tiers and development can be conducted in parallel by separate teams providing better speed-to-market.13

Reusability

Slide 13 of 31

e2eEA

Reusability is the key to a successful application architecture.Gone are the days when we developed an application for a specific purpose. Doing things once and right is the name of the game.14

Creating a business application ecosystem that can serve the enterprise across business units and business functions Reusability

e2eEA

Let me introduce the term application ecosystem to you. Every component has its place and purpose and that purpose is not filled in part of in full by any other component. Each component of the application can be activated across organisational and system boundaries.15

Architecturally there is no difference between an Application a Service or a Remote ProcedureThey all follow the SIPOC structure

Vertical Silos of DevelopmentHorizontal Perspective The Software EcoSystem

Physical and Virtual PlatformsHosting EnvironmentsANDPhysical and Virtual PlatformsACROSS at different levelsReusability

Slide 15 of 31

e2eEA

We touched on the SIPOC structure earlier. Now we make sure it the business process break down the barriers of vertical and horizontal siloes. If the SIPOC (your business definition) is constrained then your application will likely be constrained because the same business process component, which could be a candidate application component, is likely done slightly differently in different siloes.16

The Software Ecosystem concept defines business requirements as the Orchestration of Services across platforms and across hosting environmentsAll interactions between Provider and Consumer must be self-contained, loosely coupled, and independent of any other interactions.technological isolation: Consumer need not know how the Provider has been implemented or where it is installedasynchronous communication: activity not confined in timeReusability

e2eEA

Your application architecture MUST cover business process definition (SIPOC) as well as the orchestration of services across platforms and hosting environments. As an Application Architect you have to this enterprise wide, dont leave it to the enterprise architect its your job.Your application components (some will be services others will not) must be self contained small applications in their own right. This will give you further technology isolation and freedom, but please dont create a mishmash of technology across your application landscape. Finally a pet of mine Asynchronous communication wherever possible, this gives you better reusability, scalability, flexibility, performance and availability.17

Coarse-Grained services can be identified as having a role in a ProcessCoarse-Grained Services can be used as building blocks for new processes or applications

An architecture which emphasizes the importance of reusable services with a framework for Consumers to easily discover and use service Providers is referred to as a Service Oriented Architecture.Reusability

Slide 17 of 31

e2eEA

What we have described in the previous slides is the core of Service Oriented Architecture. Make sure you make your services (or application components) general enough so they can be used by multiple applications no application will ever use ALL the data or functions made available in a service.18

Application Architecture Evolution

e2eEA

Im not going to bring you back to when we wrote programs back in the 70s in Assembler and facilitated reusability by go to, do until and perform statements that was the bad old days. On the road of Application Architecture the journey is guided by roadmaps, frameworks, principles, standards and a lot of other useless stuff. I will describe this briefly the get back on track on WHAT you need to do to deliver business benefits using application architecture. 19

Object-Oriented and Component-based architectures were too tightly-coupled and synchronous by nature. Also too fine grained to be easily reusable.Message-based architecture introduced the asynchronous communication model (message queues)Best of all worlds = Software Ecosystem:Component-based architecture, applied to coarse-grained functions, using Message-based architectures loose coupling and asynchronous communication

Application Architecture Evolution

Slide 19 of 31

e2eEA

Post the days of my programming days when we though it was cool to call a subroutine in a colleague's program without him/she knowing calling it reusability, some standards emerged. Firstly with Object and Component based architectures but it was basically externalisation of specific program routines, might as well stayed internal. Then the big break-through with message based integration, who has not been working with IBMs MQ Series, asynchronous fire and forget, but it moved significant part of the business logic onto the message broker and created another single point of failure. Now combine these two key developments in application architecture evolution and add the latest definition of service orientation at all levels, and we have a brave new world.20

Applications have become distributed and technologies evolve in ways to provide application scalability. Web APIs are now the forefront for application communication whether this be U2A or A2A.

Monolithic Application LibrariesAny change required would necessitate a recompilation of the entire application.CORBA/IIOPSOAPRESTImplementation was complex and sometimes non-implementable.Costly to implementInconsistent APIs across providers.Consistent approach for web services to make function calls using GET, PUT, POST, DELETE.Technological Advancements in APIsApplication Architecture Evolution

e2eEA

One should not differentiate one whether application components facilitates user, application or external functional connectivity think ECO-System. Everything about your application architecture should evolve around scalability, availability and performance and sometimes one has to sacrifice one to achieve the other. WRONG, WRONG, WRONG. If that is your frame of mind you need to snap out of your tunnel vision and see your application architecture holistically end to end. We will touch on some of those details in the coming slides. 21

Non-functionalApplication Architecture

Slide 21 of 31

e2eEA

Non-functional requirements is NOT anything not functional non-functional requirements defines the delivery of the functional requirements. The things that will keep your application surviving mergers and acquisitions, infrastructure upgrades and changes in business strategy. 22

AvailabilityLoose coupling between applications and services are imperative to availability. If you tightly couple your website to a backend business application and that becomes unavailable, user experience is unavailable. Caching data in non-persistent storage is another lever to provide a higher level of availabilityAsynchronous processing wherever possible. Avoid technology sprawl as much as possible.

Slide 23 of 31

e2eEA

Loose coupling = asynchronous processing, only use synchronous processing when it is absolutely necessary. Always use a integration layer e.g. Enterprise Service Bus or the like, then your user experience can remain intact by accepting the update and guarantee processing later. Stored Procedures is almost a religious subject but the fact is that a stored procedure must load into a contiguous piece of memory and especially large SPs may not find that result the application hang, SPs also goes against Loose Coupling as it is not possibe to migrate MS SQL SPs to Oracles PL/SQL or any other combination. YES but performance I hear you say, Ill address that in a later slide!!!23

PerformancePerformance is often an issue when new business applications are delivered the causes can be many and varied. Some key ones to look out for are: Synchronous integration. Should only be used if you absolutely must wait for a response.Coarse-grained services minimize the number of client-service interactions and provides cohesive units of work.Data caching can save database access, especially for frequently used reference data.

e2eEA

We have touched on the Sync/Async integration already. Chatty applications are a major cause to poor performance, try to minimise the number of round trips between tiers and components. When using data caching make sure it does not become another instance of a repository but it is truly a caching facility refreshed frequently from the source of truth.24

PerformanceResource pooling/sharing. Avoid making a database or network connection every time and do not hold the connection longer than necessary.Database tuning and indexation. The application design must define data structures, dependencies and accessibility. This design is implemented on the database to provide application performance.Progressive Processing. Validate data entry as it happens and perform incremental updates wherever practicable.

Slide 25 of 31

e2eEA

As an application architect one must have at least appreciation of database and network architectures and technologies, otherwise these resources will be used poorly and affect the performance of the application. While the application architect may produce a logical database design based on the enterprise architects conceptualisation this logical design must not be implemented as the physical deployment of the database, because it will most likely not work. Get a database administrator involved to define access paths for optimal performance.When designing the user interface be mindful of the user experience, validate what is possible on the fly and in the background. Give the user a different task to complete while the application validates the previous input.25

ScalabilitySynchronous processing reserves resources on every tier through the stack and should be used with great caution.Non-persisted asynchronous state reserve resources on a specific node and force the response to return through the same node. State should be persisted e.g. in shared memory cacheParallelisation. Design processing to facilitate parallel processing of the one end-to-end transaction.

Slide 27 of 31

e2eEA

A few of these have been mentioned in previous slides. Focus on your applications resource utilisation a synchronous request is hugging memory and cpu and if you have a bunch of those it can bring your application to a halt. 26

ScalabilityUsing Stored Procedures will ultimately impact scalability

BLBLBLBLDADADB SvcsDB SvcsStored ProcedureLoad BalanceA sequence of services with no escape route yields significantly less system throughput than one where there is such a choice

BLBLDADADB SvcsDB Svcs

Load BalanceLoad BalanceDatabase servers are harder to scale and more expensive than application serversDatabase ServersDatabase ServersBusinessLogicServersI/O ChannelI/O ChannelBL= Business LogicDA= Data AccessDBSvcs = Database Service

e2eEA

27

MaintenanceApplication maintenance can become costly and difficult to the point where replacement is the only way forward.Business logic proliferation is the biggest and most common issue.Retain business logic in the business logic tierNo business logic in: stored proceduresWeb Servers or front-endFTP servers or integration layerPoorly structured applicationsPoorly designed or documented applications

e2eEA

Best of bread was a term popular in the 70s and 80s and have cost many corporations dearly in maintenance. Make sure your application architecture is coherent and use industry/open standard and possibly open source depending on the company risk and security policies. Make sure maintenance of each tier require as few as possible skill sets, e.g. if you distribute your business logic across all tiers you will required SQL skills, Java Skills, C# ckills and HTTP skills plus skills to integrate them all. For most effective maintenance make sure you structure your application as defined in the previous slides and DOCUMENTATION is non-negotiable, and documentation is not comments in the code it is a written document structured to a standard.28

Summary

Slide 29 of 31

e2eEA

Right. I have taken you through from the enterprise architecture stack to the depth of application designs. What I have given you here is not a framework or guideline to be copied into a company policy this is very personal to you as the application architect, for you to have in your back pocket when you open your mouth in meetings or tell some vendor how you want things done. I wish you luck and happy to provide further advice on your professional journey, just email your question through on question@e2eea.com or use the form on our website.29

Application ArchitectureDont loose sight that application architecture is the manifestation of business strategy.The most important message in this presentation is that of Separation of Concerns Without observing this key principle, it is not possible to service the business strategy nor is it possible to fulfil non-functional requirementsKeep an eye on where the application evolution is going and be cautious when adopting new trends, watch out for hidden tight coupling of components.

e2eEA

Your key objective as an application architect is to know and understand the business strategy, objectives and short/medium goals. Without this appreciation you cannot claim to be an application architect, because the business applications must serve these always.I cannot say it enough SEPARATION OF CONCERNS this must become a mantra in your mind.Finally a word of warning dont believe all the new fads and trends coming out, never go bleeding edge. It took me 5 years from when I was introduced to data vault before I included it in my architecture.30