national alliance for medical image computing testing with ctest and cdash julien jomier kitware...
TRANSCRIPT
National Alliance for Medical Image Computing http://na-mic.org
Testing with CTest and CDash
Julien JomierKitware Inc.
National Alliance for Medical Image Computing http://na-mic.org
Testing Overview
CVS/SVN
CDash
Web Browser
CTest
Developers
National Alliance for Medical Image Computing http://na-mic.org
Testing Terminology
Sites
dash5.kitware
iris.elmtech
Linux-g++
Builds
Linux-gcc3
Linux-gcc4
Dashboard
National Alliance for Medical Image Computing http://na-mic.org
Why Test Daily?
• Large code base too large for any single developer to understand
• Developers distributed around the world
• Identify problems as they occur
• Insure that object API remains unchanged
• Provide feedback to developers as they experiment with new implementations
National Alliance for Medical Image Computing http://na-mic.org
“If it’s not tested it doesn’t work”
• Dasboards and Software Process– Roots in GE Six Sigma initiative
• Cross platform testing
• Nightly regression testing Memory (Purify/ValGrind) testing
• Coverage testing
• Style checking
• Documentation evaluation
National Alliance for Medical Image Computing http://na-mic.org
How to write a good test?
• Use empirical values (5/2=2.5)
• Test most of the use cases (even the boring/most difficult ones)
• Make your tests as efficient as possible
• Have someone else write the test for your algorithm
National Alliance for Medical Image Computing http://na-mic.org
Regression Testing
• Compare generated image against standard “correct” image– pixel-by-pixel comparison– can use a threshold metric– adjusted for effects like dithering
National Alliance for Medical Image Computing http://na-mic.org
CTest
• Testing client
• Distributed with CMake
• Submit test results to the dashboard
• XML submission
• Independent of CMake
National Alliance for Medical Image Computing http://na-mic.org
CTest with CMakeCMakeLists.txtINCLUDE(CTest)ENABLE_TESTING()# Add a simple testADD_TEST(CDashTest1 CDashTest CDashTest) ADD_EXECUTABLE(CDashTest main.cxx)
CTestConfig.txtset(CTEST_PROJECT_NAME "MyProject")set(CTEST_NIGHTLY_START_TIME "21:00:00 EDT")
set(CTEST_DROP_METHOD "http")set(CTEST_DROP_SITE "localhost")set(CTEST_DROP_LOCATION "/CDash/submit.php?
project=MyProject")set(CTEST_DROP_SITE_CDASH TRUE)
National Alliance for Medical Image Computing http://na-mic.org
Testing Submission
XML Submission
XML Parsing
HTTP PUT
CTest
CDash
Backup XML File
Fill Database
Send Email
Update RSS Feed
Update.xml
Configure.xml
Build.xml
Test.xml
Coverage.xml
DynamicAnalysis.xml
Compiler/Tester
National Alliance for Medical Image Computing http://na-mic.org
CDash
• New generation of Dart• Written in Php/Javascript• MySQL• Open Source• Run on top of a web server• XSL implementation• Web 2.0
National Alliance for Medical Image Computing http://na-mic.org
CDash Features
• Ajax requests• Graphs: build, test times• Builds group• User management• Email notifications• Timing defects
National Alliance for Medical Image Computing http://na-mic.org
Coverage Support
• Email Low Coverage• Bullseye support
National Alliance for Medical Image Computing http://na-mic.org
Build Notes
• Share with other developers• Keep it simple
National Alliance for Medical Image Computing http://na-mic.org
My CDash
• Personal Page• Subscribe to project• View sites• Email preferences
National Alliance for Medical Image Computing http://na-mic.org
Project Roles
• Normal user• Site maintainer• Project administrator
National Alliance for Medical Image Computing http://na-mic.org
Claiming Sites
• Site information• Automatic (CTest 2.6)• Information history• Manual editing• Personal view/email
National Alliance for Medical Image Computing http://na-mic.org
CDash Misc.
• Automatic Geolocation from IP addresses
• Support for iPhone
National Alliance for Medical Image Computing http://na-mic.org
Installing CDash
• (L/W)AMP : Apache, MySQL, PHP- XAMPP, EasyPHP
• Download CDash (www.cdash.org) in your Apache web directory
• Edit config.php• Point Web Browser to CDash
- Setup the database- Create first project
National Alliance for Medical Image Computing http://na-mic.org
Style Checking
• KWStyle• C++ Style checker• Configuration via XML• HTML Generator (TBI in CDash) • Open Source• Written in C++
National Alliance for Medical Image Computing http://na-mic.org
Contribute• Website
- http://www.cdash.org- http://public.kitware.com/KWStyle
• Bug Tracker- http://www.cmake.org/Bug
• Mailing List- http://public.kitware.com/cgi-bin/mailman/listinfo/cdash
National Alliance for Medical Image Computing http://na-mic.org
Happy Testing!