sc7 wg6 rome engineering ingegneria informatica s.p.a. infso-ri-223782 isabel matranga etics...
Post on 20-Dec-2015
213 views
TRANSCRIPT
SC7 WG6Rome Engineering Ingegneria Informatica S.p.A.
INFSO-RI-223782
Isabel Matranga
ETICSAutomated Building,Testing and Quality Assurance
ETICS
INFSO-RI-223782
2
Rome, 19 October 2009
ETICS projects
E-Infrastructure for
Testing
Integration and
Configuration of
Software
ETICS
INFSO-RI-223782
3
Rome, 19 October 2009
ETICS automates and improves …
ETICS automates and improves theexecution of:
• Builds
• Tests
• Quality Verification
… offered as a service to its users
ETICS
INFSO-RI-223782
4
Rome, 19 October 2009
The context
4
ETICS
INFSO-RI-223782
5
ETICS Keywords
All-in-one solution building/testing/quality– Support for execution of continuous software build– Automatic creation of distributed packages– Dependency management
Quality made easy– Verification of the quality of the software produced– Offers the Automated-Quality Certification Model
compliant with ISO standards– Collection of test information
Language / platform / tool independent– ETICS is non-intrusive and easily extensible– Plugin based system
Distributed environment– Support for syncronization of developpers– Distributed builds and test– Multiplatform build and tests
ETICS
INFSO-RI-223782
6
Rome, 19 October 2009
ETICS supports software professionals
Requirementsand
Specification
Requirementsand
Specification
MaintenanceReview
and Improvement
MaintenanceReview
and Improvement
Architecture and Design
Architecture and Design
Testing, Validation and
Verification
Testing, Validation and
Verification
DeploymentDeployment
Implementationand Build
Implementationand Build
ETICS
INFSO-RI-223782
7
Rome, 19 October 2009
ETICS users
Main users (apart from ETICS 2 project itself)
“We use ETICS to build these software components across a variety of platforms, By using ETICS we are able to benefit from an automated build environment which can be reproducible at a later date”
Steven Newhouse, EGEE III Technical Director
“ETICS makes build and integration nightmare activity a simple configuration activity”
Pasquale Pagano, D4Science Technical Director
EDGeS “A-QCM certification, will contribute to the successful promotion of EDGeS achievements and solutions towards industrial partners”
Prof. Peter Kacsuk, coordinator of EDGeS consortium
ETICS
INFSO-RI-223782
8
Rome, 19 October 2009
What is A-QCM?
How does ETICS work?
ETICS
INFSO-RI-223782
9
ETICS System demoAutomated Testing and Quality Assurance
ETICS
INFSO-RI-223782
10
Table of Content
• Building with ETICS
• Testing with ETICS
• QA with ETICS
ETICS
INFSO-RI-223782
11
Table of Content
• Building with ETICS
• Testing with ETICS
• QA with ETICS
ETICS
INFSO-RI-223782
12
1
Build/TestArtefacts
ConfigurationWeb Service
Report/MetricsDB
ConfigurationDB
Execution Engines(Metronome, gLite, UNICORE, etc)
Command Line User Interface
RepositoryWeb Service
ETICS Infrastructure
Physical WorkerNodes
Web Portal
Virtual OS Images
ArchitectureBehind the scene
ETICS
INFSO-RI-223782
13
ETICS PortalWelcome
ETICS
INFSO-RI-223782
14
WA AdministrationSecurity
Authentication
Access to ETICS services is based on X509 certificates and fine-grained user roles:
Authorization based on RBAC model– Developers: can do local builds of project, subsystems and
components– Integrators: can create and run integration configurations– Testers: can register and submit tests attached to
configurations– Release Managers: can create and publish configurations,
create release notes and other documentation– Project Managers: can create and manage projects, assign
permissions, set policies, etc
ETICS
INFSO-RI-223782
15
ETICS PortalThe configurations
ETICS
INFSO-RI-223782
16
Web ApplicationDependencies
ETICS
INFSO-RI-223782
17
Web ApplicationRemote build
ETICS
INFSO-RI-223782
18
Repository
ETICS
INFSO-RI-223782
19
ReportsPresenting results
Testing with ETICS
ETICS
INFSO-RI-223782
20
Table of Content
• Building with ETICS
• Testing with ETICS
• QA with ETICS
ETICS
INFSO-RI-223782
21
Build & Testpackages
Reports &Metrics
Execution Engines
AWS, Metronome, gLite, Unicore
Repository WS
- source code- test code
continuous build and testing
- job submission- deployment
The ETICS visionin testing
ETICS
INFSO-RI-223782
22
ETICS Testing services
The multi-node testing functionality
• Automating execution of complex tests configuration: involving the deployment of several services, applications, clients on different hosts and their interaction
ETICS supports natively several types of testing:
• Unit test - ETICS build and test system is designed to run the already existing unit tests, to collect the results in form of metrics but not to create new unit test cases
• Static test - this technique provides a foundation for producing solid code by exposing structural errors and preventing entire classes of errors. ETICS integrating static analysis tools and techniques into the build/test process can yield significant reductions in system testing and field failures
ETICS
INFSO-RI-223782
23
Testing with ETICSThe plugins
• The ETICS plugins are small units of functionality or wrappers around external tools
• Plugins are highly configurable in ETICS
• ETICS user can activate them according to policies called profiles. Profiles can be set as a static attribute of a configuration or dynamically using build properties
• The plugins collect information and publish data as Metrics
• Metrics are published with each build/test reports and are also stored individually in the ETICS Repository
Optional plugins can be implemented and installed by users
ETICS
INFSO-RI-223782
24
Plugins and Metrics
Available plugins in ETICS:
• Sloccount (Single line of code count) – static analysis
• PyUnit, Junit, CppUnit – unit testing
• Pycoverage, Jcoverage
• Findbugs – static analysis
• CCCC
• IPv6, WSI
• CKJM
• Checkstyle
• PMD
and more coming soon ...
ETICS
INFSO-RI-223782
25
Testing use caseAn ETICS success
A typical scenario
These are the typical issues of a complex system:
• Having a running DB
• Having a software dependency installed
• Running the test suites
• Accessing the reports
A
ETICS
INFSO-RI-223782
26
ETICS Answers
• Recently presented “Experiences with the ETICS support for testing in voms” at EGEE 09 faced the described scenario:
• ETICS can configure and running additional services during a build by adding a dependency on the third-party modules (like a DB) or towards a version of a software module
• ETICS provides two choices:• ‘Test’ target in the “Build Commands
• Test Commands
• ETICS allows user-defined reports to be added to its own report page.
ETICS
INFSO-RI-223782
27
Table of Content
• Building with ETICS
• Testing with ETICS
• QA with ETICS
ETICS
INFSO-RI-223782
28
The ETICS visionin automating Quality Certification
Defines metrics
Runs measures
Define thresholdsTrials analysis
ETICS grid infrastructure at CERN,INFN, UoW (NMI)
ETICS
INFSO-RI-223782
29
A-QCM: Implementation
QA
EM
METRIC
Plugins
ETICS
INFSO-RI-223782
30
A-QCM: Web interface report example
Project level results
Subsystem level results
Component level results
The result can be expanded to reach the
metrics output related to the specific classes
ETICS
INFSO-RI-223782
31
Future plans
Really soon …• Results from A-QCM trial certification on ETICS users’
software publicly available • Enhancement of the multi-node testing feature• More plug-ins for testing integrated in ETICS• AWS cloud and Microsoft Azure Cloud under testing …
In 2010 …• ETICS offered as a service • ETICS software available under Apache 2.0 license
ETICS
INFSO-RI-223782
32
Thanks!
http://www.eticsproject.eu