sept. 8, 2008 seminar: paderborn university towards a service-oriented approach to testing web...
TRANSCRIPT
Sep
t. 8,
200
8
Seminar: Paderborn University
Towards A Service-Oriented Approach to Testing Web Services
Hong ZhuDepartment of ComputingOxford Brookes University
Oxford OX33 1HX, UK
Sep
t. 8,
200
8
Seminar: Paderborn University
Acknowledgement
Mr. Qingning Huo, PhD student at Oxford Brookes University, UK
Mr. Yufeng Zhang, MSc student at the National University of Defence Technology, China
Dr. Sue Greenwood, Oxford Brookes University, UK
Sep
t. 8,
200
8
Seminar: Paderborn University
Overview
Analyse the impact of the novel features of service-orientation on software testing
Identify the requirements on infrastructural support to testing WS applications.
Propose an framework to support testing WSReport some preliminary work
Sep
t. 8,
200
8
Seminar: Paderborn University
Characteristics of Web Services Web services is a distributed computing technique that
offers more flexibility and looser coupling so that it is more suitable for internet computing.
The dominant of program-to-program interactions The components of WS applications (such as service
providers): Autonomous: control their own resources and their own behaviours Active: execution not triggered by message, and Persistent: computational entities that last long time
Interactions between components: Social ability: discover and establish interaction at runtime Collaboration: as opposite to control, may refuse service, follow a
complicated protocol, etc.
Sep
t. 8,
200
8
Seminar: Paderborn University
WS technique stack Basic standards:
WSDL: service description and publication UDDI: for service registration and retrieval SOAP for service invocation and delivery
More advanced standards for collaborations between service providers and requesters. BPEL4WS: business process and workflow models. OWL-S: ontology for the description of semantics of services
Registry
Provider Requester
Search for services
registered services
register service
request service
deliver service
Sep
t. 8,
200
8
Seminar: Paderborn University
A typical scenario Car Insurance Broker
Suppose that a fictitious car insurance broker CIB is developing a web-based system that provides a complete service of car insurance.
End users: Submit car insurance requirements to CIBGet quotes from various insurers Select one insurer to insure the carSubmit payment informationGet insurance document//confirmation
Broker:Take information about the user and the carCheck the validity of user’s informationGet quotes from insurers and pass them to the userGet user’s selection of the insurerGet insurance from the insurer and pass the payment to the selected insurer Take commissions from the insurer or the user
Sep
t. 8,
200
8
Seminar: Paderborn University
Structure of the CIB application
CIB’s Services
Bank B’s Services
Insurance A1’s Services
Insurance A2’s Services
Insurance An’s Services
GUI Interface
CIB’s service requester
WS Registry
End users Other service users
Could be statically integrated
Should be dynamically integrated for business flexibility and competence, and lower operation and maintenance cost
Sep
t. 8,
200
8
Seminar: Paderborn University
Testing own side services (1)
Similar to test software components. Many existing work on software component testing can
be applied or adapted with special considerations: The stateless feature of HTTP protocol; XML encoding of the data passing between services as in SO
AP standard; Confirmation to the published descriptions:
WSDL for the syntax of the services workflow specification in BPEL4WS semantic specification in e.g. OWL-S.
These are the easy aspects to deal with, but…
Sep
t. 8,
200
8
Seminar: Paderborn University
Testing own side services (2) Dealing with requesters’ abnormal behaviours
The requesters are autonomous, thus they may stop cooperation in the middle of a transaction for many reasons, such as intentional quit, network failure, or failures of requester’s software system due to fault.
Burdens are on the testers to ensure that the system handles such abnormal behaviours properly.
Dealing with unexpected usages/loads As all web-based applications, load balance is essential. But, the knowledge of the usage of a WS may not be available during the
design and implementation of the system. Dealing with incomplete systems
A service may have to rely on other services to perform its functionality properly, thus hard to separate the testing of the own services from the integration testing, especially when it involves complicated workflows.
In the worst case, when WS is dynamically bound to the other services, the knowledge of their format and semantics can only be based on assumptions and standards.
Sep
t. 8,
200
8
Seminar: Paderborn University
Testing of other side services in composition
Some similarity to component integration, however, the differences are dominant
Problems in the application of existing integration testing techniques: Lack of software artifacts Lack of control over test executions Lack of means of observation on system behaviour
Sep
t. 8,
200
8
Seminar: Paderborn University
Lack of software artifacts The problem:
No design documents, No source code, No executable code The impacts:
For statically bound services, Techniques that automatically derive stubs from source code are not a
pplicable Automatic instrumentation of original source code or executable code
is not applicable For dynamic bound services,
Human involvement in the integration becomes completely impossible.
Possible solutions: (a) Derive test harness from WS descriptions; (b) The service provider to make the test stubs and drivers available for integ
ration.
Sep
t. 8,
200
8
Seminar: Paderborn University
Lack of control over test executions Problem:
Services are typically located on a computer on the Internet that testers have no control over its execution.
Impact: Control over the execution has been essential to apply existing testing
techniques and will continue to be essential for testing services: An invocation of the service as a test must be distinguished from a rea
l request of the service. System may be need to be restarted or put into a certain state to test it. The situation could become much more complicated when a WS is si
multaneously tested by many service requesters. Possible solution:
The service provider must provide a mechanism and a service that enable service requesters control the testing executions of the service.
Currently, there is no support to such mechanisms in W3C standards of WS.
Sep
t. 8,
200
8
Seminar: Paderborn University
Lack of means of observation
The problem: A tester cannot observe the internal behaviours of the services
The Impacts: No way to measure test coverage
Possible solutions: The service provider provides a mechanism and the services t
o the outside tester to observe its software’s internal behaviour in order to achieve the test adequacy that a service requester requires.
The service provider opens its document, source code as well as other software artifacts that are necessary for testing to some trusted test service providers.
Sep
t. 8,
200
8
Seminar: Paderborn University
The proposed approach A WS should be accompanied by a testing service.
functional services: the services of the original functionality testing services: the services to enable test the functional services.
Testing services can be either provided by the same vendor of the functional services, or by a third party.
Independent testing services: Provider:
testing tool vendors companies of specialized in software testing
The services: to generate test cases, to measure test adequacy, to extract various types of diagrams from source code or design and s
pecification documents, etc.
Sep
t. 8,
200
8
Seminar: Paderborn University
Architecture of service oriented testing
Broker
T-services of A1
F-services of A 1
Matchmaker
UDDI Registry
GUI
F-services of Tester T1
T-services of Tester T1
F-services of Tester T2
T-services of Tester T2
T-services of A2
F-services of A 2
Ontology m
anagement
Sep
t. 8,
200
8
Seminar: Paderborn University
Illustration of service oriented testing
CIB’s F-Services
Bank B’s F-Services
Insurance A1’s F-Services
Insurance A2’s F-Services
Insurance An’s F-Services
GUI Interface
WS Registry
CIB’s service requester
Bank B’s T-Services
Insurance A1’s T-Services
Insurance A2’s T-Services
Insurance An’s T-Services
CIB’s T-Services
Tester T1 F-Services
Tester T1 T-Services
Test Broker F-Services
Test Broker T-Services
Tester T2 F-Services
Tester T2 T-Services
Sep
t. 8,
200
8
Seminar: Paderborn University
How does the system work
The Scenario Suppose the car insurance broker want to search for we
b services of insurers and test the web service before making quote for its customers.
Car Insurance Broker CIB
Insurer Web Service IS
customer
Information about the car and
the user
Insurance quotes
Testing the integration of two services
Sep
t. 8,
200
8
Seminar: Paderborn University
Car Insurance Broker CIB
Insurer Web Service IS (F-Service)
Test Broker: TB
3.Request test service
Insurer Web Service: IS (T-Service)
Matchmaker
Testing Service: TG (Test case
Generator)
Testing Service: TE (Test Executor)
8.Testing related meta-data
16.Test report
7.Request service meta-data
11.Request service meta-data
12.Testing related meta-data
Intended composition of services
9.Test case
6.Request test service
2.Registerservice
10.Request test service
15.Test results
1.Register service
5. List of testers4.Search for testers
13.Test invocation of services
14.Results of test invocation of services
Sep
t. 8,
200
8
Seminar: Paderborn University
Automating Test Services The key technique issues to enable automated online te
st of WS: How a testing service should be described, published and regi
stered at WS registry; How a testing service can be retrieved automatically even for
testing dynamically bound services; How a testing service can be invoked by both a human tester
and a program to dynamically discover a service and then test it before bind to it.
How testing results can be summarized and reported in the forms that are suitable for both human beings to read and machine to understand.
These issues can be resolved by the utilization of a software testing ontology (Zhu & Huo 2003, 2005).
Sep
t. 8,
200
8
Seminar: Paderborn University
STOWS: Software Testing Ontology for WS Ontology defines the basic terms and relations
comprising the vocabulary of a topic area as well as the rules for combining them to define extensions to the vocabulary
STOWS is base on an ontology of software testing originally developed for agent oriented software testing (Zhu & Huo 2003, 2005). The concepts of software testing are divided into two
groups. Knowledge about software testing are also
represented as relations between concepts
Sep
t. 8,
200
8
Seminar: Paderborn University
STOWS (1): Basic concepts Tester: a particular party who carries out a testing activity. Activity: consists of actions performed in testing process, inclu
ding test planning, test case generation, test execution, result validation, adequacy measurement and test report generation, etc.
Artefact: the files, data, program code and documents etc. inovlved in testing activities. An Artefact possesses an attribute Location expressed by a URL or a URI.
Method: the method used to perform a test activity. Test methods can be classified in a number of different ways.
Context: the context in which testing activities may occur in software development stages to achieve various testing purposes. Testing contexts typically include unit testing, integration testing, system testing, regression testing, etc.
Environment. The testing environment is the hardware and software configurations in which a testing is to be performed.
Sep
t. 8,
200
8
Seminar: Paderborn University
Structure of basic concepts: Examples
Test Activity
Test planning
Test Case Generation
Test Execution
Result validation
Adequacy measurement
Report generation
Tester
Atomic Service
Composite Service
Sep
t. 8,
200
8
Seminar: Paderborn University
STOWS (2): Compound concepts
Capability: describes what a tester can do Capability
MethodActivity
Environment Context
Capability Data
Artefact
<<enumeration>>Capability Data Type
InputOutput
1 1
0-1 0-1
0-*
1-*
• the activities that a tester can perform • the context to perform the activity• the testing method used• the environment to perform the testing • the required resources (i.e. the input) • the output that the tester can generate
Sep
t. 8,
200
8
Seminar: Paderborn University
Task: describes what testing service is requested
A testing activity to be performedHow the activity is to be
performed:the context the testing method to be usedthe environment in which the
activity must be carried outthe available resources the expected outcomes
Task
MethodActivity
Environment Context
Task Data
Artefact
<<enumeration>>Task Data Type
InputOutput
0-1 0-1
1 1 1-*
1-*
Sep
t. 8,
200
8
Seminar: Paderborn University
STOWS (3): Relations between concepts Relationships between concepts are a very important
part of the knowledge of software testing: Subsumption relation between testing methods Compatibility between artefacts’ formats Enhancement relation between environments Inclusion relation between test activities Temporal ordering between test activities
How such knowledge is used: Instances of basic relations are stored in a knowledge-base as
basic facts Used by the testing broker to search for test services through
compound relations
Sep
t. 8,
200
8
Seminar: Paderborn University
Compound relations MorePowerful relation: between two capabilities.
MorePowerful(c1, c2) means that a tester has capability c1 implies that the tester can do all the tasks that can be done by a tester who has capability c2.
Contains relation: between two tasks. Contains(t1, t2) means that accomplishing task t1 implies accomplishing t2.
Matches relation: between a capability and a task. Match(c, t) means that a tester with capability c can fulfil the task t.
Capability
Tester
MorePowerful
*
*IsMorePowerful
C2 C1
Task
ContainsT1
T2C TMatches
Match
Contain
* * **
Sep
t. 8,
200
8
Seminar: Paderborn University
Definition of the MorePowerful relation
A capability C1 is more powerful than C2, written MorePowerful(C1, C2), if and only if C2’s capability is included in C1’s activities C1 and C2 have the same context. Environment of C1 is the enhancement of the environment
of C2. The method of C2 is subsumed by C1. For each input artefact of C1 , there is a corresponding co
mpatible input in the input artefact of C2 For each output artefact of C2 there is a corresponding co
mpatible output artefact of C1.
Sep
t. 8,
200
8
Seminar: Paderborn University
Definition of the Contains relation
A task T1 contains task T2, written Contains(T1, T2), if and only if T1 and T2 have the same context, T1’s activities include and T2’s activities, The method of T1 subsumes the method of T2, The environment of T2 is an enhancement of the
environment of T1, For each input artefact of T1, there is a corresponding
compatible the input artefact of T2, For each output artefact of T2 , there is a corresponding
compatible the output artefact of T1.
Sep
t. 8,
200
8
Seminar: Paderborn University
Definition of the Matches relation
A capability C matches a task T, written Matches(C, T), if and only if
C and T have the same context, C’s activities include T’s activity, The method of C subsumes the method of T, The environment of T is an enhancement of
environment of C, For each input artefact of T , there is a corresponding
compatible input artefact of C, For each output artefact of C, there is a
corresponding compatible the output artefact of T.
Sep
t. 8,
200
8
Seminar: Paderborn University
Properties of the compound relations
(1) The relations MorePowerful and Contains are reflexive and transitive.
(2) c1, c2Capability, tTask,
MorePowerful(c1, c2) Matches(c2, t)
Matches(c1, t).
(3) cCapability, t1, t2Task,
Contains(t1, t2) Matches(c, t1)
Matches(c, t2).
Sep
t. 8,
200
8
Seminar: Paderborn University
Prototype implementation
STOWS is represented in OWL-S Basic concepts as XML data definition Compound concepts defined as service profile
UDDI /OWL-S registry server (as the test broker): Using OWL-S/UDDI Matchmaker The environment:
Windows XP, Intel Core Duo CPU 2.16GHz, Jdk 1.5, Tomcat 5.5 and Mysql 5.0.
Sep
t. 8,
200
8
Seminar: Paderborn University
Representation of STOWS in OWL-S
ActivityContextEnvironmentMethodCapability dataInput ArtefactsOutput Artefacts
ServiceCategoryINPUT PARAMETERS ContextMark EnvironmentMark MethodMark Artefacts…OUTPUT PARAMETERS Artefacts…
Capability Service profile
Sep
t. 8,
200
8
Seminar: Paderborn University
Case Study: Overview
An automated software testing tool CASCAT is wrapped into a test service Registered:
Capability is described in the ontology represented in OWL-S
Searchable: It can be searched when the testing task matches its
capability
Invoked through the internet As a web services to generation test cases based on
algebraic specification
Sep
t. 8,
200
8
Seminar: Paderborn University
Case study (1): The subjectThe subject CASCAT:
Automated component testing tool for EJB Generate test cases from algebraic specification writt
en in CASOCC Execution of EJB on test cases and reports errors if
any axiom in the specification is violated
Bo Yu, Liang Kong, Yufeng Zhang, and Hong Zhu, Testing Java Components Based on Algebraic Specifications, Proc. of ICST 2008, April 9-11, 2008, Lillehammer, Norway.
Liang Kong, Hong Zhu and Bin Zhou, Automated Testing EJB Components Based on Algebraic Specifications, Proc. of TEST’07/ COMPSAC’07, Vol. 2, pp717-722. IEEE CS Press 2007.
Sep
t. 8,
200
8
Seminar: Paderborn University
Case study (2): Registry
Service Profile of CASCAT ServiceCategory: “TestCaseGenerationServices”. Input artefact: specified by the class CasoccSpecifica
tion, which is a subclass of Specification and stands for algebraic specification in CASOCC.
Context: “ComponentTest”. Environment: ‘not limited’. Method is CASOCC-method, which is a subclass of
SpecificationBased method. Output artefact: test case.
Sep
t. 8,
200
8
Seminar: Paderborn University
Case study (3): Submitting search requests
Test requester: a service was built that plays the role of test requeste
r. It constructs test tasks and submits them to the test b
roker to search for a test service Test task that it produced is to generate test case fro
m CASOCC specification in the context of the test as component test.
Sep
t. 8,
200
8
Seminar: Paderborn University
Example of test task<Task rdf:ID="testcaseTask"><inputArtefact rdf:resource="#CasoccSpec"/><outputArtefact>
<TestCase rdf:ID="testcase"><Location rdf:datatype="http://...">
http://resourse.nudt.edu.cn/testcase/ fictitioustestcase.txt
</Location> </TestCase></outputArtefact>
<hasActivity> <TestCaseGeneration rdf:ID="GenerateTestcase"/></hasActivity>
<hasMethod><CASOCC-method rdf:ID="CASOCCBasedMethod"/>
</hasMethod><hasContext rdf:resource="#ComponentTest"/>
</Task>
Sep
t. 8,
200
8
Seminar: Paderborn University
Case study (4): Search and discovery
Test Broker: Once receives a test task, it generates a capability de
scription from the test task Constructs a Service Profile. Then calls the API of the Matchmaker Client to searc
h for test service providers.
Sep
t. 8,
200
8
Seminar: Paderborn University
Case study (5): Invocation
A Java Enterprise Bean was deployed on Jboss platform
A formal specification of the bean was written in CASOCC.
The web service of CASCAT is invoked as a web service to generate test case of the component.
The result: Test cases as an instance of the OWL class TestCase. Stored as a file on a web server The Location attribute of the file is returned by the service.
Sep
t. 8,
200
8
Seminar: Paderborn University
Conclusion Testing imposes challenges to testing web services
applications are analysed: Testing a web service as own software
Solvable by adaption of existing software technologies
Integration testing at development and at run-time No support in current WS standard stack Grant challenge to existing software testing technology
A service oriented approach to solve the problem is proposed; Architecture fits well into service oriented architecture Supported by software testing ontology
Feasibility of the approach tested via a case study.
Sep
t. 8,
200
8
Seminar: Paderborn University
Advantages Automated process to meet the requirements of on-the-
fly service integration testing Automation without human involvement Testing without interference to providing normal functional
services Testing without affect the real world state
Security and IPR can be managed through a certification and authentication mechanism for third party specialised testing services
Business opportunities for testing tool vendors and software testing companies to provide testing services online as web services
Sep
t. 8,
200
8
Seminar: Paderborn University
Remaining challenges and future work Technical challenges
To develop a complete ontology of software testing (e.g. the formats of many different representations of testing related artefacts)
To implement the test brokers efficiently To device the mechanism of certification and authentication
for testing services Social challenges
For the above approach to be practically useful, it must be adopted by web service developers, testing tool vendors and software testing companies
Need standards, such as a standard of software testing ontology
Sep
t. 8,
200
8
Seminar: Paderborn University
References Zhang, Y. and Zhu, H., Ontology for Service Oriented Testing of
Web Services, Proc. of The Fourth IEEE International Symposium on Service-Oriented System Engineering (SOSE 2008) , Dec. 18-19, 2008, Taiwan. In press.
Zhu, H., A Framework for Service-Oriented Testing of Web Services, Proc. of COMPSAC’06, Sept. 2006, pp679-691.
Zhu, H. and Huo, Q., Developing A Software Testing Ontology in UML for A Software Growth Environment of Web-Based Applications, Chapter IX: Software Evolution with UML and XML, Hongji Yang (ed.). IDEA Group Inc. 2005, pp263-295.
Zhu, H. Cooperative Agent Approach to Quality Assurance and Testing Web Software, Proc. of QATWBA’04/COMPSAC’04, Sept. 2004, IEEE CS, Hong Kong,pp110-113.
Zhu, H., Huo, Q. and Greenwood, S., A Multi-Agent Software Environment for Testing Web-based Applications, Proc. of COMPSAC’03, 2003, pp210-215.