Software Integration and Test Software Integration and Test Techniques in a Large Techniques in a Large Distributed Project: Distributed Project: Evolution, Process Evolution, Process

Improvement, ResultsImprovement, Results

Paola Sivera - ESO

ICALEPCS 2007, 15-19 Oct 2007, Knoxville TN 2

Center of Array

Pampa La BolaV. Licancabur Cº Chajnantor

Chajnantor Plateau – looking north

ICALEPCS 2007, 15-19 Oct 2007, Knoxville TN 3

Antenna Prototypes at ALMA Test Facility: 2005

ICALEPCS 2007, 15-19 Oct 2007, Knoxville TN 4

ALMA Geographical distribution

ALMA

DRAOPenticton

ALMAATF

ArcetriObservatory

Jodrell Bank

BreraObservatory

IRAMGrenoble

ATCEdinburgh

NAOJ

ESO

NRAO

Obs deParis

Univ.Calgary

c

c

Santiago

MPI Bonn

DAMIR/IEMMadrid

ICALEPCS 2007, 15-19 Oct 2007, Knoxville TN 5

ALMAManagement

B. GlendenningG. Raffi

K.Tatematsu

Science Software Requirements

R. Lucas

High Level Analysis

J. Schwarz

Software EngM.Zamparelli

Common SWG. Chiozzi

ExecutiveP. Grosbol

ControlA. Farris

ArchivingA. Wicenec

Observation Preparation

A. Bridger

Operations Support

M. Chavan

OfflineJ. McMullen

PipelineL. Davis

Telescope Calibration

R. Lucas

CorrelatorJ. Pisano

IntegrationP. Sivera

SchedulerA.Farris

ACAM.Watanabe

Trilateral Computing IPT Organisation

Total Bilateral staff now: 40 FTEs

Total trilateral staff now: 65 FTEs

Total Bilateral staff now: 40 FTEs

Total trilateral staff now: 65 FTEs

ICALEPCS 2007, 15-19 Oct 2007, Knoxville TN 6

ICALEPCS 2007, 15-19 Oct 2007, Knoxville TN 7

ICALEPCS 2007, 15-19 Oct 2007, Knoxville TN 8

ALMA System Dataflow

ICALEPCS 2007, 15-19 Oct 2007, Knoxville TN 9

ObsPrep Visual Editor (to select spectral setup)

ICALEPCS 2007, 15-19 Oct 2007, Knoxville TN 10

EXEC: Operator Master Client

ICALEPCS 2007, 15-19 Oct 2007, Knoxville TN 11

CASA Viewer• Visibility displays (raw, corrected data)• Image, data cube displays

– Statistics; profiles; raster, contour, vector support

ICALEPCS 2007, 15-19 Oct 2007, Knoxville TN 12

Standards and Tools

• Alma Common Software

• Environment

• CVS organization

• Makefile

• Standard Test Environment (STE)

• Tool for automated testing

• Bug tracking tool (JIRA)

ICALEPCS 2007, 15-19 Oct 2007, Knoxville TN 13

ALMA Release Cycle

Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

Minor Release: Mar 31 Major Release: Sep 30

• • • • • • • • • • • •

Monthly integration = last day of the month

ICD Freeze = 15th of the month

ICALEPCS 2007, 15-19 Oct 2007, Knoxville TN 14

Lessons learned

• Releases twice per year

but…

• Subsystem developers working in isolation

• Results thrown over the wall to ITS at release time– Each integration took ~2 months

– At completion of integration code was already obsolete

ICALEPCS 2007, 15-19 Oct 2007, Knoxville TN 15

Function-based Teams

ICALEPCS 2007, 15-19 Oct 2007, Knoxville TN 16

FBTs organization

Z SharedSimulator-2006-02-B (branch) SharedSimulator-2006-02-RootOfBranch (tag of the HEAD)

A SharedSimulator-2006-02-BeforeMerge

B TRUNK-SharedSimulator-2006-02-BeforeMerge

C TRUNK-SharedSimulator-2006-02-AfterMerge (tag of the HEAD)

SharedSimulator-2006-02-AfterMerge-B (branch)

F SharedSimulator-2006-02-FINAL

ICALEPCS 2007, 15-19 Oct 2007, Knoxville TN 17

Summing it up

• FBTs advantages:– Improve communication

– More effective involvement of testers

• Testing end-to-end since the beginning– Good for object-oriented software

– A lot of maintenance on the test cases

– It is not enough -> user tests

ICALEPCS 2007, 15-19 Oct 2007, Knoxville TN 18

Where we are

Testing maturity (according to Boris Beizer)

1. there is no difference between testing and debugging: often during FBTs’ work

2. the purpose of testing is to show that the software works: at FBTs validation

3. the purpose of testing is to show that the software does not work: release validation

4. the purpose of testing is not to prove anything, but to reduce the perceived risk of not working to an acceptable value: release validation

ICALEPCS 2007, 15-19 Oct 2007, Knoxville TN 19

Future developments

• Make software more testable from its inception • adoption of the test maturity model (TMM) • … the list can be very long …

For the near future we want to focus on: • Stability tests• More automation

ICALEPCS 2007, 15-19 Oct 2007, Knoxville TN 20

AOS Technical Building

AOS Technical Building

ICALEPCS 2007, 15-19 Oct 2007, Knoxville TN 21

ALMA - 2012

ICALEPCS 2007, 15-19 Oct 2007, Knoxville TN 22

ICALEPCS 2007, 15-19 Oct 2007, Knoxville TN 23

MELCO : August 2007