introduction to testing with mstest, visual studio, and team foundation server 2010

Getting to know MSTestWith Visual Studio 2010 and Team Foundation Server 2010

Thomas Wellerwww.thomas-weller.deinfo@thomas-weller.de July 2010

Outline

I. Authoring tests (with MSTest)II. Visual Studio 2010III. Team Foundation Server 2010IV. Advanced testing conceptsV. UI Automation

What‘s in a test ?

• Anatomy of a test class • What makes a good (unit) test...• Test context• Initalize/Cleanup• Attributes:

• [ExpectedException]• [Description] • [TestProperty]• [TestCategory]

Sample „test target“

Anatomy of a test class (I)

Ordinary C# class with attributes:

Containing class: [TestClass] Test: [TestMethod], public void, paramless

Anatomy II: Assert class

To verify a fact:

Throws a special exception to fail a test StringAssert, CollectionAssert …

Anatomy III: AAA structure

General structure of a test method – AAA:o Arrangeo Acto Assert

What makes a good (unit) test…

verify only one fact

try to isolate code under test

Test documentation, usage example

short – simple – fast (CI) – readable

no dependencies on other tests (random order!)

TestContext class

run-time environment data (e.g. details, directories…) WriteLine() for additional info in test results:

TestContext class (II)

simply declare an automatic property:

Inizalization and Cleanup

methods marked with attributes (xxxInitialize/xxxCleanup)

three levels:[TestInitialize][TestCleanup] Before/after each test method

[ClassInitialize][ClassCleanup]

- static - Before/after the first/last test method of a class

[AssemblyInitialize][AssemblyCleanup]

- static - Before/after any test class of the assembly

all methods are optional

[ExpectedException]

testing, that a certain exception is thrown by the method under test:

counterpart to Assert class, very frequently used also Exception messages, if needed

[Description]

short summary text (similar to xml comments):

shows up in the Properties window:

[TestProperty]

arbitrary name/value pairs:

show up in the Properties window:

[TestCategory]

assign (arbitrary) categories to a test:

filter by category on automated builds:

[TestCategory]: Usage example

define some constants:

apply to test methods as appropriate:

Visual Studio 2010

• Test run configuration (*.testsettings) • Test lists/metadata (*.vsmdi) • Tool Windows:

• Test View window• Test List Editor window• Test Results window• Test Runs window• Test Results Details window

Test run config (*.testsettings)

set of test run and environment settings etc. (e.g. naming schemes, directories, depl. scripts etc.)

*.testsettings *.xml

Test lists/metadata (*.vsmdi)

*.vsmdi *.xml

organizing / grouping tests with lists e.g. for different test runs

similar use to that of TestCategory attribute edit with Test List Editor window (see below)

Test View window

run, view, group, filter, select tests on different criteria

Test List Editor window manage, create, and organize test lists ( *.vsmdi)

Test Results window

opens automatically when running a test inside VS

Test Results Details window

test details + error information (e.g. Stack trace, logs, console output ...)

Team Foundation Server 2010

• Walkthrough• Creating a Bug work item• Creating the test• Creating a Test Case work item for the Bug• Associating test and work item• Check in

• Tests and Automated Builds

Creating a (Bug) work item (I)

Project Portal (Web):

Visual Studio (Team Explorer):

Creating a (Bug) work item (II)

Writing the test

Create a Test Case for the Bug In Visual Studio (on Bug work item):

Associating test and work item From Test View window:

From work item:

Check in on check-in:

version history:

Tests and Automated Builds

Manual: „Dev build“ UI automation tests ClickOnce

CI run: only fast running tests Project „heartbeat“ Nightly run: longer running tests xml documentation

Some more advanced concepts

• Testing private code: PrivateObject/PrivateType• Testing internal code: InternalsVisibleTo• Mocking• „Moling“• Data-driven tests: xml• Data-driven tests: database

PrivateObject/PrivateType

wrappers around reflectionMETHOD DESCRIPTION

GetArrayElement Returns the selected item from a private array.

GetField Returns the value of the target field.

GetFieldOrProperty Returns the value of the target field or property.

GetProperty Returns the value of the target.

Invoke Invokes a method, optionally passing arguments.

SetArrayElement Assigns the given value to the indicated array element.

SetField Assigns the supplied object to the target field.

SetFieldOrProperty Assigns the supplied object to the target field or property.

SetProperty Assigns the supplied object to the target property.

PrivateType class static elements

PrivateObject : Example

method to test:

test method:

InternalsVisibleTo

better alternative (no reflection)

in AssemblyInfo.cs:

test method:

Mocking

http://code.google.com/p/moq/wiki/QuickStart

Creating dummy objects on the fly

Isolating the tested code from dependencies

Moq

Specify arguments and return values

Verify method calls

http://code.google.com/p/moq/

Other (OS) alternatives: Rhino.Mocks, NMock...

Mocking: Example (I)

Mocking: Example (II)

„Moling“

Mocking: Virtuals and Interface implementations MS Moles: Replace anything with a delegate

Visual Studio Gallery download: http://visualstudiogallery.msdn.microsoft.com/en-us/b3b41648-1c21-471f-a2b0-f76d8fb932ee

Generates moles-assembly, uses runtime instrumentation (performance!)

„Moling“: Example (I)

Add New Item ...:

Assembly references:

„Moling“: Example (II)

„Moling“: Example (III) method to test:

test method:

Data-driven tests: xml (I)

create an xml file:

Data-driven tests: xml (II)

test method:

data accessible via TestContext

Data-driven tests: xml (III)

Test Results Details window:

Data-driven tests: database (I) create a database/table:

Data-driven tests: database (II)

Properties window (wizard) …

… or manually

Automated UI testing

• The White Framework• The recorder• UI Spy• UI object model• Example (Concurrency)• how to proceed…

The White Framework

http://white.codeplex.com/

.NET based, OS, by ThoughtWorks (CC.Net) OO wrapper around MS UIAutomation library

The recorder (alpha)

generates C# sourcecode

UI Spy

Similar to Spy++ Windows SDK 6.0

http://www.microsoft.com/downloads/details.aspx?displaylang=en&FamilyID=4377f86d-c913-4b5c-b87e-ef72e5b4e065

UI object model (partial)

Example - Concurrency (I)

Example - Concurrency (II)

…

?