Reproducible GeophysicsArchiving Experiments

in the MADAGASCAR Project

Sergey Fomel

Jackson School of GeosciencesThe University of Texas at Austin

May 25, 2010

Outline

Computational Geophysics

Reproducible Research

MADAGASCAR Project

Largest Moving Object on Earth

1912 2009

Black Magic inGeophysicalProspectingL. W. Blau, 1936

Black Magic in Computational Science

Within the world of science, computation is nowrightly seen as a third vertex of a trianglecomplementing experiment and theory. However, as itis now often practiced, one can make a good case thatcomputing is the last refuge of the scientificscoundrel [...] Where else in science can one get awaywith publishing observations that are claimed to provea theory or illustrate the success of a techniquewithout having to give a careful description of themethods used, in sufficient detail that others canattempt to repeat the experiment?Randall LeVeque, ICM, 2006

(Hale, 1984)

(Hale, 1984)

Outline

Computational Geophysics

Reproducible Research

MADAGASCAR Project

What is Science?

What is Science?

Science is the systematic enterprise of gatheringknowledge about the universe and organizing andcondensing that knowledge into testable laws andtheories. The success and credibility of science areanchored in the willingness of scientists toindependent testing and replication by otherscientists. This requires the complete and openexchange of data, procedures and materials.American Physical Society, What is Science?

What is Reproducible Research?

I Attaching software code and data to publications

An article about computational science in a scientificpublication is not the scholarship itself, it is merelyadvertising of the scholarship. The actual scholarshipis the complete software development environmentand the complete set of instructions which generatedthe figures. Jon Buckheit and David Donoho, WaveLab

Reproducible Research Discussions

I http://www.reproducibleresearch.net

ICASSP 2007Berlin-6 2008CiSE 2009

I Donoho et al.I LeVequeI Ping & EckelI Stodden

IEEE Signal Processing Magazine 2009I Vandewalle et al.

Yale Roundtable 2009

Personal Experience

1991–2001 Jon F. ClaerboutI Stanford Exploration ProjectI Generations of Ph.D. studentsI The principal beneficiary is the author

2003–Present MADAGASCAR packageI Software code requires continuous maintenanceI Maintenance requires an open communityI http://www.reproducibility.org/

Outline

Computational Geophysics

Reproducible Research

MADAGASCAR Project

Facts

http://www.ahay.org/

http://www.reproducibility.org/

I Publicly released in 2006 (GPL)I Approaching 1.0 release in 2010I 25+ developersI 250,000+ lines of codeI 10,000 downloads from SourceForgeI 80 reproducible papers; 3,000 reproducible results

I http://www.ahay.org/wiki/Reproducible Documents

MADAGASCAR architectureRecipes

I Main programs operating on data filesI C, C++, Fortran, Java, Python, MatlabI Regularly Sampled FormatI Unix pipesI SCONS data processing flows

InputsI Data repository

ResultsI Figures included in publicationsI Archived to serve as regression testsI Links from programs to source code and recipesI Links from recipes to programs and dataI Hierarchy book/chapter/projectI LATEX2HTML

ThanksI Vladimir Bashkardin, Jules Browaeys, William Burnett,

Cody Brown, Maria Cameron, Lorenzo Casasanta,Joseph Dellinger, Jeff Godwin, Gilles Hennenfent,Trevor Irons, Jim Jennings, Long Jin, Roman Kazinnik,Siwei Li, Guochang Liu, Yang Liu, Doug McCowan,Henryk Modzelewski, Colin Russell, Paul Sava,Jeffrey Shragge, Xiaolei Song, Eduardo Filpo Silva,Ioan Vlad, Jia Yan

ConclusionsI Reproducible research

I Attaching software and data to publicationsI Computational experimentsI Reproducibility is not the goal!

I Lessons from experience in computational geophysicsI The principal beneficiary is the authorI Software code requires continuous maintenanceI Maintenance requires an open community!