© cgi group inc. confidential€¦ · (e.g. esb). in order to support complex data exchange...
TRANSCRIPT
![Page 1: © CGI Group Inc. CONFIDENTIAL€¦ · (e.g. ESB). In order to support complex data exchange scenarios and business process, SOA is typically based on messages. Examples of message](https://reader035.vdocument.in/reader035/viewer/2022071003/5fc05370473dd732042fe1b7/html5/thumbnails/1.jpg)
© CGI Group Inc. CONFIDENTIAL
Testing of SOA based systems
Staged acceptance
Chris C. Schotanus
![Page 2: © CGI Group Inc. CONFIDENTIAL€¦ · (e.g. ESB). In order to support complex data exchange scenarios and business process, SOA is typically based on messages. Examples of message](https://reader035.vdocument.in/reader035/viewer/2022071003/5fc05370473dd732042fe1b7/html5/thumbnails/2.jpg)
2
Introduction
CHRIS SCHOTANUS
Principal IT Consultant
REQUIREMENTS & QUALITY MANAGEMENT
41 YEARS OF IT EXPERIENCE
OF WHICH 34 WITH CGI (AND ITS ANCESTORS)
27 YEARS IN TEST AUTOMATION, TEST PROCESS
IMPROVEMENT & PROCESS ARCHITECTURE
![Page 3: © CGI Group Inc. CONFIDENTIAL€¦ · (e.g. ESB). In order to support complex data exchange scenarios and business process, SOA is typically based on messages. Examples of message](https://reader035.vdocument.in/reader035/viewer/2022071003/5fc05370473dd732042fe1b7/html5/thumbnails/3.jpg)
3
Today’s Agenda
01 Introducing SOA
02 Test specific items
03
04
SOA governance aspects
The real life experience
![Page 4: © CGI Group Inc. CONFIDENTIAL€¦ · (e.g. ESB). In order to support complex data exchange scenarios and business process, SOA is typically based on messages. Examples of message](https://reader035.vdocument.in/reader035/viewer/2022071003/5fc05370473dd732042fe1b7/html5/thumbnails/4.jpg)
SOA based systems defined
(Ideal) SOA based systems are composite systems made of parts that are
• Virtual: the consumer is not aware of the implementation of the service
• Standardised: there is only one implementation of a function or responsibility
• Modular (replaceable) and compositional
• Abstract: generic, not dedicated to one specific consumer
• Detached: consumer and supplier are fully independent of each others implementation
• Agnostic: there is no direct link or relation between services
4
![Page 5: © CGI Group Inc. CONFIDENTIAL€¦ · (e.g. ESB). In order to support complex data exchange scenarios and business process, SOA is typically based on messages. Examples of message](https://reader035.vdocument.in/reader035/viewer/2022071003/5fc05370473dd732042fe1b7/html5/thumbnails/5.jpg)
These parts are called "services"
A service is a bundle of resources that belong to the same context and that are able to solve
operational problems and to support processes. Services are characterised by autonomy, loose
coupling, reusability and a service contract, which abstracts the service implementation
(technology and logic) via a well-defined service interface.
Types of service contracts:
• Basic contract: service operations incl. input/output parameters
• Behavioural contract: parameter combinations, pre/post conditions, service call sequences
• Synchronisation contract: simultaneously service calls
• QoS contract: non-functional requirements
5
![Page 6: © CGI Group Inc. CONFIDENTIAL€¦ · (e.g. ESB). In order to support complex data exchange scenarios and business process, SOA is typically based on messages. Examples of message](https://reader035.vdocument.in/reader035/viewer/2022071003/5fc05370473dd732042fe1b7/html5/thumbnails/6.jpg)
SOA principals: service triangle
A service call involves the service provider, the service consumer and the service registry. All
information needed to perform a service call can be gathered during run-time („service binding“)
which enables SOA to allocate resources flexibly and dynamically.
6
Service registry
Service provider Service consumer
Service usage
![Page 7: © CGI Group Inc. CONFIDENTIAL€¦ · (e.g. ESB). In order to support complex data exchange scenarios and business process, SOA is typically based on messages. Examples of message](https://reader035.vdocument.in/reader035/viewer/2022071003/5fc05370473dd732042fe1b7/html5/thumbnails/7.jpg)
SOA principals: service composition
Services can be categorised according to their functional stage of expansion:
• Basic services
• Composed services, using compositions of other services to perform a complex action
• Process services, representing long running business processes
Service composition can be achieved in two ways:
• Orchestration (service calls are coordinated by a central instance)
• Choreography (self-governed peer-to-peer communication between services)
Ideally, SOA implementations provide sophisticated workflow engines to allow
configuration of user-defined process definitions.
7
![Page 8: © CGI Group Inc. CONFIDENTIAL€¦ · (e.g. ESB). In order to support complex data exchange scenarios and business process, SOA is typically based on messages. Examples of message](https://reader035.vdocument.in/reader035/viewer/2022071003/5fc05370473dd732042fe1b7/html5/thumbnails/8.jpg)
SOA principals: service mediation
When a service call is performed, the code of the service implementation is not integrated into the
service consumer‘s environment but remains and is run in the service provider‘s environment. The
interoperability issues which result from such an approach are addressed by a mediation system
(e.g. ESB). In order to support complex data exchange scenarios and business process, SOA is
typically based on messages.
Examples of message exchange patterns:
• One way (Fire and Forget)
• Request/Response
• Request/Call-back
• Publish/Subscribe
8
![Page 9: © CGI Group Inc. CONFIDENTIAL€¦ · (e.g. ESB). In order to support complex data exchange scenarios and business process, SOA is typically based on messages. Examples of message](https://reader035.vdocument.in/reader035/viewer/2022071003/5fc05370473dd732042fe1b7/html5/thumbnails/9.jpg)
9
What this presentation is about
Questions often asked regarding SOA are:
• Who will accept a services?
• How about ownership of services?
• What should we test if one or more services change?
• Interface testing versus end-to-end integration testing?
• Who owns the maintenance budget?
These are the questions I will try to answer
![Page 10: © CGI Group Inc. CONFIDENTIAL€¦ · (e.g. ESB). In order to support complex data exchange scenarios and business process, SOA is typically based on messages. Examples of message](https://reader035.vdocument.in/reader035/viewer/2022071003/5fc05370473dd732042fe1b7/html5/thumbnails/10.jpg)
10
Today’s Agenda
01 Introducing SOA
02 Test specific items
03
04
SOA governance aspects
The real life experience
![Page 11: © CGI Group Inc. CONFIDENTIAL€¦ · (e.g. ESB). In order to support complex data exchange scenarios and business process, SOA is typically based on messages. Examples of message](https://reader035.vdocument.in/reader035/viewer/2022071003/5fc05370473dd732042fe1b7/html5/thumbnails/11.jpg)
11
Testing services
Accept the product
Define user needs,
Business processes,
Requirements
The supplier/consumer model as reference
Design Build Test Deliver
Consumer
responsibility
Supplier
responsibility
Either Agile or Sequential Process
![Page 12: © CGI Group Inc. CONFIDENTIAL€¦ · (e.g. ESB). In order to support complex data exchange scenarios and business process, SOA is typically based on messages. Examples of message](https://reader035.vdocument.in/reader035/viewer/2022071003/5fc05370473dd732042fe1b7/html5/thumbnails/12.jpg)
The SOA acceptance process
• User applications are assembled from or use (web)services
• Service requirements are derived from business requirements
• Services are selected from service registry
• Systems are assembled using services
12
It is a Staged Acceptance Process
![Page 13: © CGI Group Inc. CONFIDENTIAL€¦ · (e.g. ESB). In order to support complex data exchange scenarios and business process, SOA is typically based on messages. Examples of message](https://reader035.vdocument.in/reader035/viewer/2022071003/5fc05370473dd732042fe1b7/html5/thumbnails/13.jpg)
Enterprise Messaging Service
The SOA specific test process
13
End user app
SOA Service SOA Service SOA Service
Governance
Operations
Business Owner
Assembly
![Page 14: © CGI Group Inc. CONFIDENTIAL€¦ · (e.g. ESB). In order to support complex data exchange scenarios and business process, SOA is typically based on messages. Examples of message](https://reader035.vdocument.in/reader035/viewer/2022071003/5fc05370473dd732042fe1b7/html5/thumbnails/14.jpg)
Enterprise Messaging Service
This might as well be a hybrid or a full agile process like Nexus
14
End user app
SOA Service SOA Service SOA Service
Governance
Operations
Business Owner
![Page 15: © CGI Group Inc. CONFIDENTIAL€¦ · (e.g. ESB). In order to support complex data exchange scenarios and business process, SOA is typically based on messages. Examples of message](https://reader035.vdocument.in/reader035/viewer/2022071003/5fc05370473dd732042fe1b7/html5/thumbnails/15.jpg)
Service Integration (Regression) Testing
What should we test if a service is changed?
End-to-End testing versus
Interface testing
15
![Page 16: © CGI Group Inc. CONFIDENTIAL€¦ · (e.g. ESB). In order to support complex data exchange scenarios and business process, SOA is typically based on messages. Examples of message](https://reader035.vdocument.in/reader035/viewer/2022071003/5fc05370473dd732042fe1b7/html5/thumbnails/16.jpg)
Basic servicesS8 S9 S10
Cons 1 Cons 2 Cons 3
Orchestrated business
process
Orchestrated service
Data handling services
Business applications
(consumers)
S1 S2 S3
S4 S5 S6
S11 S12
S7
S13
DB2 IMSExternal
Service
What must be tested?
16
![Page 17: © CGI Group Inc. CONFIDENTIAL€¦ · (e.g. ESB). In order to support complex data exchange scenarios and business process, SOA is typically based on messages. Examples of message](https://reader035.vdocument.in/reader035/viewer/2022071003/5fc05370473dd732042fe1b7/html5/thumbnails/17.jpg)
Basic servicesS8 S9 S10
Cons 1 Cons 2 Cons 3
Orchestrated business
process
Orchestrated service
Data handling services
Business applications
(consumers)
S1 S2 S3
S4 S5 S6
S11 S12
S7
S13
DB2 IMSExternal
Service
Just the ones directly involved
17
E2E borders
Stubs, Virtual services or
Services running in test
environment
![Page 18: © CGI Group Inc. CONFIDENTIAL€¦ · (e.g. ESB). In order to support complex data exchange scenarios and business process, SOA is typically based on messages. Examples of message](https://reader035.vdocument.in/reader035/viewer/2022071003/5fc05370473dd732042fe1b7/html5/thumbnails/18.jpg)
Pro’s and cons
Pro: Less effort, relatively easy to manage
Con: Perhaps less insight in risks
18
Pro: Security, certainty
Con: Difficult to manage, expensive
![Page 19: © CGI Group Inc. CONFIDENTIAL€¦ · (e.g. ESB). In order to support complex data exchange scenarios and business process, SOA is typically based on messages. Examples of message](https://reader035.vdocument.in/reader035/viewer/2022071003/5fc05370473dd732042fe1b7/html5/thumbnails/19.jpg)
19
Ideally, in “real SOA”
there should be
no need for
end-to-end testing!
![Page 20: © CGI Group Inc. CONFIDENTIAL€¦ · (e.g. ESB). In order to support complex data exchange scenarios and business process, SOA is typically based on messages. Examples of message](https://reader035.vdocument.in/reader035/viewer/2022071003/5fc05370473dd732042fe1b7/html5/thumbnails/20.jpg)
End-2-end testing: where do we stop?
• Chain testing is expensive and complex
• Stakeholders define the product risks
• Only high risk systems justify the effort for chain testing
• In other cases stick to interface based testing!
20
Product Risk Assessment as starting point
![Page 21: © CGI Group Inc. CONFIDENTIAL€¦ · (e.g. ESB). In order to support complex data exchange scenarios and business process, SOA is typically based on messages. Examples of message](https://reader035.vdocument.in/reader035/viewer/2022071003/5fc05370473dd732042fe1b7/html5/thumbnails/21.jpg)
Test environments and test data management
21
How to deal with connections to non-existent services
or services that are under change
![Page 22: © CGI Group Inc. CONFIDENTIAL€¦ · (e.g. ESB). In order to support complex data exchange scenarios and business process, SOA is typically based on messages. Examples of message](https://reader035.vdocument.in/reader035/viewer/2022071003/5fc05370473dd732042fe1b7/html5/thumbnails/22.jpg)
Mock services (Stubs and Drivers )
• Specific pieces of software
• Simulate behaviour of services not (yet) available
• Allow for isolated testing of services
• Divide the chain into manageable parts
• Are adaptable to the wish of the tester
• Provide good control of data
• Good and fault situations are predictable
22
Driver
Testobject
MockMocks are interchangeable with virtualization tools
![Page 23: © CGI Group Inc. CONFIDENTIAL€¦ · (e.g. ESB). In order to support complex data exchange scenarios and business process, SOA is typically based on messages. Examples of message](https://reader035.vdocument.in/reader035/viewer/2022071003/5fc05370473dd732042fe1b7/html5/thumbnails/23.jpg)
• Every service can virtually be
available
• Flexible and adjustable test data
• Maintained by testers
• High degree of reuse
But:
• Relatively expensive (additional
tooling required)
• Additional (functional) management
• Knowledge and training needed
• Initially cheap
• “Tailor made”
• Rapidly deployable
• Risk of expensive maintenance
But:
• Made by developers
• Additional, non-standard
maintenance
• Additional communication
tester <> developer
• Inflexible
• Poorly reusable
Stubs or Service virtualisation?
23
Mocks Virtual Services
![Page 24: © CGI Group Inc. CONFIDENTIAL€¦ · (e.g. ESB). In order to support complex data exchange scenarios and business process, SOA is typically based on messages. Examples of message](https://reader035.vdocument.in/reader035/viewer/2022071003/5fc05370473dd732042fe1b7/html5/thumbnails/24.jpg)
Alternative: Several test environments
24
Production
Research
Copy of
production
Component test
of isolated
services by
system
developers
System test
of isolated
services against
functional and
non-functional
design
Service
Integration test
Acceptance of
individual
services
Test AcceptanceIntegration
TestDevelopment
Development
User Acceptance
Test
Of integrated
system
Service development(internal & external)
Consumer IT Business Operation
Actual use in live
environment
![Page 25: © CGI Group Inc. CONFIDENTIAL€¦ · (e.g. ESB). In order to support complex data exchange scenarios and business process, SOA is typically based on messages. Examples of message](https://reader035.vdocument.in/reader035/viewer/2022071003/5fc05370473dd732042fe1b7/html5/thumbnails/25.jpg)
Role of the Test manager
• Is responsible for all test activities
• Compiles a test strategy based on product risks analysis
• Defines requirements to test environments
• Formulates demands on service delivery
25
Test manager is often also "Integration Manager"
![Page 26: © CGI Group Inc. CONFIDENTIAL€¦ · (e.g. ESB). In order to support complex data exchange scenarios and business process, SOA is typically based on messages. Examples of message](https://reader035.vdocument.in/reader035/viewer/2022071003/5fc05370473dd732042fe1b7/html5/thumbnails/26.jpg)
First level Acceptance
• Does the service meet the requirements as stated in the repository?
• Is backwards compatibility guaranteed?
• Can the service safely replace the one that's already in the registry?
26
Fist level acceptance by the service owner
![Page 27: © CGI Group Inc. CONFIDENTIAL€¦ · (e.g. ESB). In order to support complex data exchange scenarios and business process, SOA is typically based on messages. Examples of message](https://reader035.vdocument.in/reader035/viewer/2022071003/5fc05370473dd732042fe1b7/html5/thumbnails/27.jpg)
27
Today’s Agenda
01 Introducing SOA
02 Test specific items
03
04
SOA governance aspects
The real life experience
![Page 28: © CGI Group Inc. CONFIDENTIAL€¦ · (e.g. ESB). In order to support complex data exchange scenarios and business process, SOA is typically based on messages. Examples of message](https://reader035.vdocument.in/reader035/viewer/2022071003/5fc05370473dd732042fe1b7/html5/thumbnails/28.jpg)
SOA Governance
SOA governance is a concept used for activities related to exercising control over services in
a service-oriented architecture (SOA).
IBM and others, state that SOA governance is an extension (subset) of IT governance which
itself is an extension of corporate governance.
Source: https://en.wikipedia.org/wiki/SOA_governance
SOA governance refers to the processes used to oversee and control the adoption and
implementation of service-oriented architecture (SOA) in accordance with recognized practices,
principles and government regulations. SOA governance provides optimum service quality,
consistency, predictability and performance, ensures that personnel follow prescribed policies and
corrects system problems or policy infractions as they occur.Source: https://searchmicroservices.techtarget.com/definition/SOA-governance
28
![Page 29: © CGI Group Inc. CONFIDENTIAL€¦ · (e.g. ESB). In order to support complex data exchange scenarios and business process, SOA is typically based on messages. Examples of message](https://reader035.vdocument.in/reader035/viewer/2022071003/5fc05370473dd732042fe1b7/html5/thumbnails/29.jpg)
Service repositories and service registries
29
Source: Boris Lublinsky - Explore the role of service repositories and registries in Service-Oriented Architecture (SOA) IBM-2007
A service repository is the foundation of enterprise SOA
governance, supporting centralized management of all of
services-related information, including design,
implementation, and usage artefacts.
A service registry is the foundation of service location
virtualization, allowing for centralized control over location
and invocation policies for all of the enterprise services.
Two important aspects of SOA Governance
![Page 30: © CGI Group Inc. CONFIDENTIAL€¦ · (e.g. ESB). In order to support complex data exchange scenarios and business process, SOA is typically based on messages. Examples of message](https://reader035.vdocument.in/reader035/viewer/2022071003/5fc05370473dd732042fe1b7/html5/thumbnails/30.jpg)
Use of registry and repository
30
Defining, redefining
Services and meta data
modelling
Services
Developping and testing
Services Rolling out
Services Making up
Services
Design time Run time
Modifying, updating
Services and meta data
Supervising
Services
Executing
Services
RegistryRepository
![Page 31: © CGI Group Inc. CONFIDENTIAL€¦ · (e.g. ESB). In order to support complex data exchange scenarios and business process, SOA is typically based on messages. Examples of message](https://reader035.vdocument.in/reader035/viewer/2022071003/5fc05370473dd732042fe1b7/html5/thumbnails/31.jpg)
31
Stakeholders and the service repository
The details about your services and the services you use:
• Functionality
• Availability
• Capacity
• Use
• Service owners
• Service Sepcifications
SLA between suppliers and consumers
![Page 32: © CGI Group Inc. CONFIDENTIAL€¦ · (e.g. ESB). In order to support complex data exchange scenarios and business process, SOA is typically based on messages. Examples of message](https://reader035.vdocument.in/reader035/viewer/2022071003/5fc05370473dd732042fe1b7/html5/thumbnails/32.jpg)
Business model
32
who will pay for the changes and test them if one consumer needs them
and others don’t.
Passenger Cargo
Business
IT
Agreement on
use and
maintenance
Req
uest
![Page 33: © CGI Group Inc. CONFIDENTIAL€¦ · (e.g. ESB). In order to support complex data exchange scenarios and business process, SOA is typically based on messages. Examples of message](https://reader035.vdocument.in/reader035/viewer/2022071003/5fc05370473dd732042fe1b7/html5/thumbnails/33.jpg)
33
Today’s Agenda
01 Introducing SOA
02 Test specific items
03 SOA governance aspects
04 The real life experience
![Page 34: © CGI Group Inc. CONFIDENTIAL€¦ · (e.g. ESB). In order to support complex data exchange scenarios and business process, SOA is typically based on messages. Examples of message](https://reader035.vdocument.in/reader035/viewer/2022071003/5fc05370473dd732042fe1b7/html5/thumbnails/34.jpg)
Testing of services in a live environment
The generic presentation of components in an Enterprise Service Bus (ESB)
System A PES EMS CES System B
A: Provider,
source of information
PES:
Providing Adapter
“Translates” A into CDM
According to specs in a mapping sheet
EMS:
Enterprise Messaging
Service
CES:
Consuming Adapter
“Translates” CDM into B
According to specs in a mapping sheet
B: Consumer,
Uses the information
Interface format
as used in System A
Common Data Model
format and behaviour
Standards on EMS Interface format
as used in System B
Service
Mapping A to CDM Mapping CDM to B
![Page 35: © CGI Group Inc. CONFIDENTIAL€¦ · (e.g. ESB). In order to support complex data exchange scenarios and business process, SOA is typically based on messages. Examples of message](https://reader035.vdocument.in/reader035/viewer/2022071003/5fc05370473dd732042fe1b7/html5/thumbnails/35.jpg)
Testing of services in a live environment
Multiple consumers per service
System A PES EMS
CES System B
B: Consumer,
Uses the information
Service
Mapping A to CDM
Mapping CDM to B
CES System C
C: Consumer,
Uses the information
Mapping CDM to C
![Page 36: © CGI Group Inc. CONFIDENTIAL€¦ · (e.g. ESB). In order to support complex data exchange scenarios and business process, SOA is typically based on messages. Examples of message](https://reader035.vdocument.in/reader035/viewer/2022071003/5fc05370473dd732042fe1b7/html5/thumbnails/36.jpg)
System development related to SOA
Client
needs
ES
B
Inte
gra
tion
Ce
ntr
e
Pro
ject
Exte
rna
l
Pro
gra
mm
ing
pa
rty
Integration
Req’s
Integration
Architecture Design
Intake &
Acceptance
Service Integration
Test
Business
Acceptance Test
Functional and Non-
functional Test
Detailed design &
Build
Delivery
Test
Architecture
& Design
Non-Integration
Req’s
Non-Integration
Development
Functional and Non-
functional Test
![Page 37: © CGI Group Inc. CONFIDENTIAL€¦ · (e.g. ESB). In order to support complex data exchange scenarios and business process, SOA is typically based on messages. Examples of message](https://reader035.vdocument.in/reader035/viewer/2022071003/5fc05370473dd732042fe1b7/html5/thumbnails/37.jpg)
Testing of services in a live environment
37
System A PES EMS CES System B
Adapter
system test
Adapter
system test
The developing party in their own SOFa in development environment
ESB Integration Centre in their own Test environment
EIC Delivery test
The client IT department in the Integration Environment (T)
Service Integration Test
End users in the Acceptance Envrionment (A)
Business Acceptance Test.
The Test Process
![Page 38: © CGI Group Inc. CONFIDENTIAL€¦ · (e.g. ESB). In order to support complex data exchange scenarios and business process, SOA is typically based on messages. Examples of message](https://reader035.vdocument.in/reader035/viewer/2022071003/5fc05370473dd732042fe1b7/html5/thumbnails/38.jpg)
Generic Test Agreements
Overall test strategy and agreements
• Clear agreements on who test what and when
• Overall acceptance criteria per test phase
• Handover criteria between test phases
Like any test strategy: It prevents "I did not mean or expect that" discussions
38
![Page 39: © CGI Group Inc. CONFIDENTIAL€¦ · (e.g. ESB). In order to support complex data exchange scenarios and business process, SOA is typically based on messages. Examples of message](https://reader035.vdocument.in/reader035/viewer/2022071003/5fc05370473dd732042fe1b7/html5/thumbnails/39.jpg)
Summary: testing in SOA environments
39
Accepting parties
▪ The users accept the application
▪ The service-owner accepts
individual SOA services on
functionality and capacity
▪ Operations accepts SOA services
op load, stress and behaviour
Integration testing
▪ Preferably in isolation
▪ End-to-end if stakeholders
consider it necessary and product
risks indicate to
Regression testing
▪ All owners of services that
communicate directly with the
changed one
▪ Owners of the changed services
▪ Operations
▪ Supervising the test process
▪ Advising the stakeholders
▪ Integration of services
The test managers role