lecture 9: debugging & testing. 9-2 microsoftintroducing cs using.netj# in visual studio.net...

Lecture 9:

Debugging & Testing

9-2MicrosoftIntroducing CS using .NETJ# in Visual Studio .NET

Objectives

“Unfortunately, errors are a part of life. Some are detected by the compiler, some are detected through testing, and the remainder go uncaught. Modern languages and tools like J#, Visual Studio .NET, and NUnit provide many techniques for quickly identifying, dealing with, and eliminating errors. But you have to apply them…”

• Debugging options• Testing• Testing with NUnit


Part 1

• Debugging options…


Print debugging

• Everyone understands print debugging

• In .NET, you have various options:

– console app: System.out.println(...);

– GUI app:System.Windows.Forms.MessageBox.Show(...);

– inside VS: System.Diagnostics.Debug.WriteLine(...);


Source-level debugging

• Most development environments supply a debugger– allow you to debug your program line-by-line, look at vars, etc.– Visual Studio .NET contains a very sophisticated debugger

• Basic idea:– you set a breakpoint to pause your program at a certain point– run program in order to reach breakpoint– once paused, you can look at variables, single-step, etc.

click in margin to set breakpoint


Debug menu

• Once you reachbreakpoint, you can:

– single-step– view local vars– change values

• Example:– Watch window let’s

you expand &inspect objects!

• See Debug menu forthings you can do…


Examples

• When using constructors, set a breakpoint where you do “new Class1(…)”, run to hit breakpoint, and then single-step, showing how constructor is automatically called…

• When using dynamic binding, single-step through calls to see how different method implementations are called dynamically…


Part 2

• Testing…


Testing

• Good testing is a skill that students must develop:– algorithm boundary cases– single calls & sequences of calls– parameter validation– exception handling– etc.

• Good testing is automated– so there’s no reason not to do it

• Good testing is repeatable (regression testing)– so you can retest after any change to the code


Basic testing methodology

• For every class C, create a test harness TC to test it

– TC = one or more methods that exercise C

• Test harness should:

– exercise all fields & methods in C

– strive for 100% code coverage (exercise all paths of execution)

– generate no output other than a final testing summary• i.e. test harness should *not* rely on user to determine correctness

• user will forget what the “correct” output should be…

class Ctest

harnessTC


Example

• StudentsIO class reads student info from a file– returns an ArrayList of Student objects…

public class StudentsIO{ /** * Reads student info from given file, returning arraylist */ public static java.util.ArrayList read(String filename) ... { java.util.ArrayList students; students = new java.util.ArrayList(); . . .

return students; }

public class StudentsIO{ /** * Reads student info from given file, returning arraylist */ public static java.util.ArrayList read(String filename) ... { java.util.ArrayList students; students = new java.util.ArrayList(); . . .

return students; }


Example test harness

• Tests StudentsIO.read( ) against various input files…

public class StudentsIOTest{ // tests against an empty input file… public static void readEmpty() throws java.lang.Exception { java.util.ArrayList students; students = StudentsIO.read("empty.txt"); if (students == null || students.size() != 0) throw new java.lang.Exception("readEmpty failed!"); }

// tests against an input file with one student, "Jim Bag" public static void readOne() throws java.lang.Exception { java.util.ArrayList students; students = StudentsIO.read("one.txt"); . . .

public class StudentsIOTest{ // tests against an empty input file… public static void readEmpty() throws java.lang.Exception { java.util.ArrayList students; students = StudentsIO.read("empty.txt"); if (students == null || students.size() != 0) throw new java.lang.Exception("readEmpty failed!"); }

// tests against an input file with one student, "Jim Bag" public static void readOne() throws java.lang.Exception { java.util.ArrayList students; students = StudentsIO.read("one.txt"); . . .


Running test harness

• Common approach is to define a main method in test harness– Reconfigure VS to startup app using this main. Right-click on

project in Solution Explorer (or use Project menu), select Properties, then Common Properties, then General, and set "Startup Object" in this case to "StudentApp.StudentsIOTest".

public class StudentsIOTest{ …

public static void main(String[] args) { int failed = 0; System.out.println("** Testing StudentsIO **");

try { readEmpty(); } catch(Exception ex) { System.out.println(ex.toString()); failed++; } … }}//class

public class StudentsIOTest{ …

public static void main(String[] args) { int failed = 0; System.out.println("** Testing StudentsIO **");

try { readEmpty(); } catch(Exception ex) { System.out.println(ex.toString()); failed++; } … }}//class


Part 3

• Testing with NUnit…


NUnit

• NUnit is based on the highly-successful Java tool JUnit

– NUnit is for .NET code

– NUnit is for testing "units", i.e. classes

– available for free from http://nunit.org/


NUnit methodology

• NUnit automates the running of the test harness

– it doesn't write the test harness for you (that's our job)

– it runs the test harness & collects statistics…

.EXE / .D.EXE / .DLLLL


How does NUnit help?

• NUnit helps in many ways:

– you don't have to write the code to collect & output statistics

– tells you exactly which tests failed / skipped

– catches exceptions that testing methods may throw

– provides a framework for regression testing, i.e. re-testing after source code has been modified


Using NUnit

• Five-step process:

1. Download & install NUnit

2. Create test harness, with instance test methods & no main( )

3. Set a reference to NUnit:• Project menu, Add Reference…• Browse…• Navigate to C:\Program Files\NUnit V2.1\bin• Open "nunit.framework.dll"• Ok

4. Add NUnit attributes to test harness, rebuild project

5. Startup Nunit-Gui via Start menu, open compiled code, & run!• "Green" light means test passed• "Yellow" light means test was skipped• "Red" light means test failed


Example

• Let's revisit test harness for StudentsIO.read( )…

• Same as before, except:– no main method; test methods should be instance methods– addition of NUnit attributes to identify test harness & methods

/** @attribute NUnit.Framework.TestFixture() */public class StudentsIOTest{ /** @attribute NUnit.Framework.Test() */ public void readEmpty() throws java.lang.Exception { java.util.ArrayList students; students = StudentsIO.read("empty.txt"); if (students == null || students.size() != 0) throw new java.lang.Exception("readEmpty failed!"); } . . .

/** @attribute NUnit.Framework.TestFixture() */public class StudentsIOTest{ /** @attribute NUnit.Framework.Test() */ public void readEmpty() throws java.lang.Exception { java.util.ArrayList students; students = StudentsIO.read("empty.txt"); if (students == null || students.size() != 0) throw new java.lang.Exception("readEmpty failed!"); } . . .


Working with NUnit

• Idea:

– compile test harness, etc. in Visual Studio .NET

– startup Nunit-Gui, file >> open compiled code, run test harness• Green is good, Yellow is caution, Red is bad

– leave both tools open, compiling in VS & testing in NUnit

.EXE / .D.EXE / .DLLLL


Debugging with NUnit

• Visual Studio's debugger is still available• Process for debugging:

– open Visual Studio & Nunit-Gui as usual

– set breakpoints in VS as usual

– in VS, attach debugger to Nunit-Gui as follows:• Debug menu• Processes…• select "nunit-gui.exe" from list• click "Attach"• check the box for "Common Language Runtime" & click OK• close Processes window, notice that VS is now in "Run" mode

– in Nunit-Gui, click Run button!• when you are done testing/debugging, you'll need to stop VS…


Writing a test method

• If test method returns, counted as success• If test method throws an exception, counted as failure• Implication:

– test method must throw an exception to denote failed test…

/** @attribute NUnit.Framework.Test() */public void readEmpty() throws java.lang.Exception{ java.util.ArrayList students;

students = StudentsIO.read("empty.txt"); if (students == null || students.size() != 0) throw new java.lang.Exception("readEmpty failed!");}


students = StudentsIO.read("empty.txt"); if (students == null || students.size() != 0) throw new java.lang.Exception("readEmpty failed!");}


Assertions

• Assertions are a common way of denoting failure• Idea:

– you assert that a particular boolean condition is true– if condition turns out to be true, nothing happens– if condition turns out to be false, an exception is thrown


students = StudentsIO.read("empty.txt"); System.Diagnostics.Debug.Assert(students != null); System.Diagnostics.Debug.Assert(students.size() == 0);}


students = StudentsIO.read("empty.txt"); System.Diagnostics.Debug.Assert(students != null); System.Diagnostics.Debug.Assert(students.size() == 0);}


Part 4

• Other tools & ideas…


Other tools & ideas

• NUnit short demo videos by Martin Schray: http://www.msdnacademicalliance.net/curriculum/pfv.aspx?5935

• WinForms GUI testing?– NUnitForms, http://sourceforge.net/projects/nunitforms/

• Web app testing?– NUnitASP, http://sourceforge.net/projects/nunitasp/

• Other strategies?– programming with invariants, listing pre & post-conditions

– these are based on the idea of using assertions in your code:

…

// at this point in the program, count should be 1, if not through an exception & stop nowSystem.Diagnostics.Debug.Assert(count == 1, "Count==1 failed");

…

// at this point in the program, count should be 1, if not through an exception & stop nowSystem.Diagnostics.Debug.Assert(count == 1, "Count==1 failed");


Summary

• Unfortunately, errors are a part of being human• Errors can be eliminated, but it takes a great deal of effort:

– good logic– good design– good programming– good testing

• You have the tools:– J#, Visual Studio .NET, NUnit, etc.

lecture 9: debugging & testing. 9-2 microsoftintroducing cs using.netj# in visual studio.net...

Documents

testing slide

microsoftintroducing

testing good testing

options testing testing

breakpoint slide

debugging options slide

nunit slide

code slide