arc tools for revision and nightly functional tests · – submission of a simple jsdl job to an...
TRANSCRIPT
EMI IN
FSO-RI-261611
EMI IN
FSO-RI-261611
ARC tools for revision and nightly functional tests
Jozef Cernak, Marek Kocan, Eva Cernakova
(P. J. Safarik University in Kosice, Kosice, Slovak Republic)
EMI IN
FSO-RI-261611
EMI IN
FSO-RI-261611
• Testing strategy in ARC
• ARC-EMI testing tools
– Build testing
• Nightly (every night)
• Revision (after each commit of developers)
– Automatic functional testing of nightly builds
• Future plans
– Increasing number of automatic functional tests
– Automatic performance testing
• Conclusion
Use Insert Header & Footer to set this field
Use Insert Header & Footer to set this field 2
Outline
EMI IN
FSO-RI-261611
EMI IN
FSO-RI-261611
• Several roles in project: developers, testers and ARC release manager
• Adopted tests:
– Unit tests(developers)
– Build tests (testers)• Revision (Kosice)
• Nightly (Copenhagen)
• EMI-RC (ETICS)
– Functional tests (testers, developers)• Manual
• Automatic (Kosice)
– Regression testing (testers, developers)
– Performance testing (testers)• Manual
• Automatic in development phase
– Large scale testing (in preparation phase)
Use Insert Header & Footer to set this field
Use Insert Header & Footer to set this field 3
Testing practice in ARC
EMI IN
FSO-RI-261611
EMI IN
FSO-RI-261611
ARC-EMI testing tools Relation between ARC and EMI testing tools
Use Insert Header & Footer to set this field
Use Insert Header & Footer to set this field 4
EMI IN
FSO-RI-261611
EMI IN
FSO-RI-261611
Block diagram of automatic revision and Functional toolsSimple database server with PHP and Python scripts
Use Insert Header & Footer to set this field
Use Insert Header & Footer to set this field 5
EMI IN
FSO-RI-261611
EMI IN
FSO-RI-261611
• Infrastructure (local cluster)– 2 SUN 2200M2 servers (2x4GB RAM, 250GB
disk, AMD 64 bits)– 2 desktops (2x 1GB RAM, 500GB disk 32 bits)– 1 MAC mini (2GB RAM, 160GB, 32 bits)
• Software– MySQL, PHP, Python– Code is free
• Distributed data processing – SVN
Technical details
Use Insert Header & Footer to set this field
Use Insert Header & Footer to set this field 6
EMI IN
FSO-RI-261611
EMI IN
FSO-RI-261611
• Features
– Supported are several platforms of OS:
• CentOS (32,64),
• Debian(32,64),
• and MAX OS (64)
– Detailed identification of errors during building process
• Autogen
• Configuration
• make
• make dist
• Build
Revision automatic testing
Use Insert Header & Footer to set this field
Use Insert Header & Footer to set this field 7
EMI IN
FSO-RI-261611
EMI IN
FSO-RI-261611
– User friendly interface to define search criteria• Daily
• Weakly
• Monthly
• Custom– Revision number, etc (see next examples)
• Direct connection with SVN
• Warning messages for developers if code is broken
– CCCC metric
– Quick mechanisms to identify breaking event
…continued.
Use Insert Header & Footer to set this field
Use Insert Header & Footer to set this field 8
EMI IN
FSO-RI-261611
EMI IN
FSO-RI-261611
Example of www interface for automatic testing toolsHome page: http://arc-emi.grid.upjs.sk/ (Revision database is slow)
Use Insert Header & Footer to set this field
Use Insert Header & Footer to set this field 9
EMI IN
FSO-RI-261611
EMI IN
FSO-RI-261611
Several criteria to search results of revision testing
Use Insert Header & Footer to set this field
Use Insert Header & Footer to set this field 10
EMI IN
FSO-RI-261611
EMI IN
FSO-RI-261611
Example of the results of daily revisions
Use Insert Header & Footer to set this field
Use Insert Header & Footer to set this field 11
EMI IN
FSO-RI-261611
EMI IN
FSO-RI-261611
• Features
– Automatic download code from SVN based on several criteria:
• trunk,
• revision code,
• nightly builds
– Automatic build
– Automatic deployment
– User friendly interface for:
• submitting proposals of test cases
• to search the results of functional tests
– Tests are grouped into two main groups:
• functional tests of server,
• functional tests of client.
Automatic functional tests
Use Insert Header & Footer to set this field
Use Insert Header & Footer to set this field 12
EMI IN
FSO-RI-261611
EMI IN
FSO-RI-261611
Interface to enter search criteria
Use Insert Header & Footer to set this field
Use Insert Header & Footer to set this field 13
EMI IN
FSO-RI-261611
EMI IN
FSO-RI-261611
The results of automatic functional tests
Use Insert Header & Footer to set this field
Use Insert Header & Footer to set this field 14
EMI IN
FSO-RI-261611
EMI IN
FSO-RI-261611
• client_arcsub_to_arex_gridmap_simplejob_jsdl– Tries to submit a simple JSDL job to a grid-map secured A-REX service.
• client_arcsub_to_grid-manager_gridmap_simplejob_jsdl– Tries to submit a simple JSDL job to a grid-map secured ARC0 CE.
• client_arcsub_to_GIIS_simplejob_jsdl– Submission of a simple JSDL job to a GIIS service.
• client_arcsub_to_ISIS_simplejob_jsdl– Submission of a simple JSDL job to an ISIS cloud.
• client_arcsub_random_broker_isis_simplejob_jsdl– Submission of a simple JSDL job to an ISIS cloud using random broker.
• client_arcsub_random_broker_giis_simplejob_jsdl– Submission of a simple JSDL job to an GIIS cloud using random broker.
• client_arcsub_fastestqueue_broker_isis_simplejob_jsdl– Submission of a simple JSDL job to an ISIS cloud using fastest queue broker.
• client_arcsub_fastestqueue_broker_giis_simplejob_jsdl– Submission of a simple JSDL job to a GIIS service using fastest queue broker.
• Failed– There was some error before we managed to start testing. The error could have been in code
retrieval, installation or somewhere else.
Functional test scenaria
Use Insert Header & Footer to set this field
Use Insert Header & Footer to set this field 15
EMI IN
FSO-RI-261611
EMI IN
FSO-RI-261611
• Requirements:– several servers and clients– Monitoring of several distributed processes:
• resources: – Memory– CPU– Network usage
• Performance– Scalability– Number of concurrent requests
• Reliability of services:– Ratio between successful tasks and total tasks. Examples of
tasks: job submission, file transfer and etc.
Automatic performance tests
Use Insert Header & Footer to set this field
Use Insert Header & Footer to set this field 16
EMI IN
FSO-RI-261611
EMI IN
FSO-RI-261611
• Proposed infrastructure:– Grid servers and clients, we plan to use
existing infrastructure– Communication layer • XML-RPC client-server architecture
– Central database server– Common Python library on each server and
client (for example start stop service)– Central control program
… continued.
Use Insert Header & Footer to set this field
Use Insert Header & Footer to set this field 17
EMI IN
FSO-RI-261611
EMI IN
FSO-RI-261611
• Automatic testing covers:– Build process (ARC, ETICS),– Functional,– Performance tests (in progress).
• Utilization of automatic test tools increases probability to find weak part of the software. On the other hand, manual tests are important.
Use Insert Header & Footer to set this field
Use Insert Header & Footer to set this field 18
Conclusions
EMI IN
FSO-RI-261611
EMI IN
FSO-RI-261611
• Testing in ARC– http://wiki.nordugrid.org/index.php/Testing
• Revision tests– http://download.nordugrid.org/builds/– http://arc-emi.grid.upjs.sk/revisionTests.php
• Functional tests– http://arc-emi.grid.upjs.sk/functionalTests.php
• Code:– http://svn.nordugrid.org/trac/workarea/browser/ARCT
estScriptshttp://svn.nordugrid.org/trac/workarea/browser/ARCTestScripts
References
Use Insert Header & Footer to set this field
Use Insert Header & Footer to set this field 19
EMI IN
FSO-RI-261611
EMI IN
FSO-RI-261611
Thank you
Use Insert Header & Footer to set this field
20Use Insert Header & Footer to set this field
EMI is partially funded by the European Commission under Grant Agreement INFSO-RI-261611