Article: Source Code Review SystemsAuthor: Jason Remillard

Presenter: Joe Borosky

Class: Principles and Applications of Software Design

Date: 11/2/2005

Inspections: A Well Established Cost Effective Way to Find Defects

• Yet they are not universally used, WHY?– Lack of Training on how to do Inspections well

– The need for Project Managers to move resources Away From Testing into Inspections

– Large amount of paperwork required by Formal Inspections

Soluris and Inspections

• Because of the Previously stated problems the software inspections program at Soluris had faded away

• When Soluris wanted to reestablish the inspection process they did 2 things– First they purchased “Peer Reviews in Software”

by Karl Wiegers– Second they selected a software tool to automate

the inspection process and thus eliminate the paper forms that would be needed

Tools Compared

• Open Source– Bugzilla– Codestriker

• Commercial– CodeReview add-on for Visual Studio .NET– CodeReviewer– ReviewPro

Bugzilla (1) (www.bugzilla.org)

• Open Source Bug Tracking System• Originally built to support Netscape Navigator• It spun off in 1998 as part of the Mozilla Web

Browser• CGI-based Web Application• Written in Perl• Runs under Unix and Windows

Bugzilla (2) (www.bugzilla.org)

• The Database Backend uses the open source MySQL

• It requires Reviews to occur within an open Bug Report– Developers enter all Enhancements as Bugs so each

task performed has an associated Bug

• When an Enhancement is made or a Bug is fixed a Unified Difference Text File (or Patch File) is created

Bugzilla (3) (www.bugzilla.org)

• The Patch File only contains the changes made and is uploaded as a Bug Attachment.

• Using the existing Bug Commenting Systems you can state questions, concerns, or suggestions.

• The Bugzilla Patch Viewer is integrated with CVS (Concurrent Versions System) so you can view unchanged parts of files.

Bugzilla (4) (www.bugzilla.org)

• It does NOT support other revision control systems.

• It Cannot collect Metrics on the Review or Track the state of each comment, which are disadvantages when considering its use for Formal Inspections.

• Its support for Formal Inspections is minimal and it focuses on Spot Check-ins

Codestriker (1)(http://codestriker.sourceforge.net)• Written by David Sitsky in 2001• It started out as a simple Web-based Review

System for patches.• It has evolved into a tool with good support

for Formal Inspections with Metrics and for Inspection Meetings.

• CGI-Application written in Perl• The Web server runs on Windows and Unix

Codestriker (2)(http://codestriker.sourceforge.net)• Advantages over Bugzilla

– It can store data in Oracle, MySQL, PostgresSQL or Microsoft SQL Server

– It can integrate with many source code control systems, including CVS, Subversion, Clearcase, Visual Source Safe, Perforce, and Bugzilla

Codestriker (3)(http://codestriker.sourceforge.net)• Installation

– 1. Unpack the Codestriker tar or zip file contents into a directory on your web server

– 2. Create a new database in your RDB of choice

– 3. Configure the Web server to call the Codestriker CGI Perl Scripts

– 4. Configure the Codestriker site-specific option in the codestriker.conf file with a text editor

– If needed the manual gives detailed instructions

Codestriker (4)(http://codestriker.sourceforge.net)• To use Codestriker for a Review you must

set up a Topic which includes a description, a reviewer list, and the document to review.

• 2 ways to Create a Topic– 1. Generate it from the Revision Control

System– 2. Upload a File

Codestriker (5)(http://codestriker.sourceforge.net)

• Uploading a file (using patch Files)– This is usually a single command in most

Revision Control Systems. For example, cvs diff -u > my_diff.txt (similar to Bugzilla)

– Patch files do not need to be formatted as a Unified diff file like in Bugzilla

– You can upload a Text File but you cannot upload complex files like Word documents or PDF files

Codestriker (6)(http://codestriker.sourceforge.net)• Generating Topics from the Revision Control

System– First check in the files being reviewed

– Next Enter the baseline revision’s name in the start tag

– Then enter the end tag field of the new version of the files to compare

– Finally the topic author lists the reviewer’s email addresses and enters a comment and title.

Codestriker (7)(http://codestriker.sourceforge.net)• Codestriker sends email to the reviewers

with a link pointing to a dynamically created web page that shows the topic under review.

• Reviewers can make comments and Codestriker sends email to the topic author for each commit submitted. Comments are tracked in a separate comment page.

Codestriker (8)(http://codestriker.sourceforge.net)• When the author makes the appropriate changes

he/she closes the topic.• Soluris uses Codestriker for both Spot Checking

and for Formal Inspections.• Spot Checks are not as rigorous as Formal

Inspections but they are useful for finding obvious problems and style guide violations

• For the same amount of work Soluris now gets a review of all check ins on the revision control system using Codestriker

Codestriker (9)(http://codestriker.sourceforge.net)

• Metrics– It automatically collects metrics on each review– It knows how large each topic is, who participated, how long they

spent, and how many defects they found, all without any extra data entry

– It can manage External Metrics (overview meeting time and preparation time) & Inspection Metrics (monitor effective ness of inspections)

– It showed finding defects during inspections is more cost effective than finding them during integration testing or after software release.

Codestriker (10)(http://codestriker.sourceforge.net)• Problems

– It is limited to reviewing text files (can’t be used for documents with formatting, tables, or images). Thus high level documents require manual review.

– Soluris uses it only for Code Reviews, Detailed Design Reviews, and Check-in Spot Checks

– It sends a lot of emails (email is sent every time a topic is created or a comment is made)

– It does not support checklists as the commercial products do

Code Review Add-on for Visual Studio .NET (www.macadamian.com/products/codereview)

• By Macadamian Technologies

• Focuses on Pre-check in Spot Inspections

• Commercial product

• It has similar capabilities as Bugzilla (open source)

Code Reviewer(www.codehistorian.com/codereviewer-

overview.php)• By SmartBear Software

• Focuses on Pre-check in Spot Inspections

• Commercial product

• It has similar capabilities as Bugzilla (open source)

ReviewPro(www.sdtcorp.com/reviewpro.html• By Software Development Technologies• It offers Excellent support for Formal

Inspections (including: inspection metrics, fine grained user security, and customizable process flow).

• It assumes that the item being inspected is printed or viewable in another application.

• It cannot be used for check-in spot checks• Commercial Product

Conclusion

• Codestriker is the best product in terms of support for BOTH Check-In Spot Checks and for Formal Inspections.

• For your own needs evaluate available tools and see what works best for the types of inspections you need to do.

• No Current product is a complete solution for all kinds of inspections

Table 1 the 5 Review Products

Questions/Comments