Test apps 1

Creating manual tests using the web portal 4

Customizing Code Coverage Analysis 12

Developer testing tools scenarios and capabilities 21

Exploratory test web apps directly in your browser 28

Exploratory Testing - Capture, Create and Collaborate 32

How to Configure Unit Tests to Target An Earlier Version of the .NET Frame 35

IntelliTest Reference Manual 40

Introduction to Coded UI Tests with Visual Studio Enterprise 2015 43

Introduction to IntelliTest with Visual Studio Enterprise 2015 67

Manual and exploratory testing scenarios and capabilities 85

Parallel and Context Sensitive Test Execution 92

Unit testing native code with Test Explorer 98

Unit Testing, Code Coverage and Code Clone Analysis with Visual Studio 2015 110

Manual and exploratory testing

Overview (manual-exploratory-

testing/manual-testing)

Get started (../work/kanban/add-run-

update-tests)

Test with Visual Studio Team Services

(manual-exploratory-testing/getting-

started/create-a-test-plan)

Test with Team Foundation Server

Performance testing

Overview (performance-

testing/performance-testing)

Get started (performance-

testing/getting-started/getting-

started-with-performance-testing)

Performance test your app

(performance-testing/run-

performance-tests-app-before-release)

Test apps early and often

Team Services | Visual Studio 2015 | Previous version (https://msdn.microsoft.com/library

/ms182409%28v=vs.120%29.aspx)

Test continuously while you code, build, and deploy your app. Find problems before launching your

app or updates into production. You can then better assess whether your app meets your customers'

needs and is ready for release.

Team Services (/docs/overview) > (/docs/test/overview)

Test apps early and often | Visual Studio Team Services and Team Founda... https://www.visualstudio.com/docs/test/overview

1 of 3 02.09.2016 13:24

(https://msdn.microsoft.com/library

/dd380763%28v=vs.140%29.aspx)

Test with Microsoft Test Manager

(https://msdn.microsoft.com/library

/jj635157%28v=vs.140%29.aspx)

URL-based load tests (performance-

testing/getting-started/get-started-

simple-cloud-load-test)

Test Azure web apps (performance-

testing/app-service-web-app-

performance-test)

Continuous testing

Overview (continuous-

testing/continuous-testing)

Get started (continuous-

testing/getting-started/getting-

started-with-continuous-testing)

Run tests with your builds (continuous-

testing/test-build)

Run Selenium tests (continuous-

testing/getting-started/continuous-

test-selenium)

Test Java apps (continuous-

testing/getting-started/continuous-

test-java)

Developer tools and unit testing

Overview (developer-

testing/developer-testing)

Get started (developer-testing/getting-

started/getting-started-

with-developer-testing)

Unit test your code

(https://msdn.microsoft.com/library

/dd264975)

Perform system tests with Visual Studio

(https://msdn.microsoft.com/library

/jj620889%28v=vs.140%29.aspx)

Automate system tests

(https://msdn.microsoft.com/library

/ff472576%28v=vs.140%29.aspx)

Set up test environments

Set up test machines

(https://msdn.microsoft.com/library

/dd293551%28v=vs.140%29.aspx)

Administer your test lab

(https://msdn.microsoft.com/library

/dd936084%28v=vs.140%29.aspx)

Troubleshoot lab management

(https://msdn.microsoft.com/library

/ee853230%28v=vs.140%29.aspx)

Extend your test tools

API reference for test tools

(https://msdn.microsoft.com/library

/dd465178.aspx)

Help and support

Submit bugs through Connect (https://connect.microsoft.com/visualstudio), make suggestions on

Uservoice (http://visualstudio.uservoice.com/forums/121579-visual-studio), and send quick thoughts

Test apps early and often | Visual Studio Team Services and Team Founda... https://www.visualstudio.com/docs/test/overview

2 of 3 02.09.2016 13:24

using the Send-a-Smile link in the Visual Studio, Team Services, or TFS title bar. We look forward to

your feedback.

© 2016 Microsoft

Test apps early and often | Visual Studio Team Services and Team Founda... https://www.visualstudio.com/docs/test/overview

3 of 3 02.09.2016 13:24

Creating manual tests using the web portal

 

Create manual test cases to check that each of the deliverables meet your users' needs. You organize your test cases by

adding test cases to test suites.

Create manual test cases

If you haven't already, create a test plan and requirement-based test suites.1.

From the Test hub in the web portal, choose a requirement-based test suite. Then create a test case for that suite.

A requirement-based test suite is created from a backlog item. If you add a test case to this type of suite, it is

automatically linked to that backlog item.

2.

Add test steps with actions and expected results so that any team member can run the test. You can add

attachments to a step if you want.

3.

Creating manual tests using the web portal https://msdn.microsoft.com/en-us/library/dd286729(d=printer).aspx

1 of 8 02.09.2016 14:22

Now you have created a test case that you can run.

Q&AQ: Can I add an extra line into a test step?

A: Yes. Press Shift+Enter in the action or expected results field to add an extra line.

Q: How do I insert a test step into a test case?

A: Select a test step. Press Alt+P to insert a test step above the selected step.

Q: Is there a way to quickly add multiple test cases at the same time?

A: Yes. Use the grid view when you add test cases to the test suite.

Creating manual tests using the web portal https://msdn.microsoft.com/en-us/library/dd286729(d=printer).aspx

2 of 8 02.09.2016 14:22

In the grid view, use the context menu to insert, clear or delete a row.

Q: Can I bulk edit multiple test cases?

A: Yes. Switch the view from list to grid. All the test cases for the current test suite will be displayed in the grid with their

test steps. This is a helpful view if you want to review your test cases with other team members jointly. When you review,

you can update and add new test cases.

Creating manual tests using the web portal https://msdn.microsoft.com/en-us/library/dd286729(d=printer).aspx

3 of 8 02.09.2016 14:22

Or you can filter and sort the test cases. Then select only the ones that you want to bulk edit.

To return to the test suite view, switch the view from grid back to list.

Q: Can I copy test cases and test steps from an existing Excel worksheet?

A: Yes. Copy the columns from Excel that you want to use for the title, action and expected results fields. Any formatting

in the columns, other than multiline, will not be copied into the worksheet. Paste these columns into the grid view, edit if

necessary, and save them. (This is only supported for Internet Explorer and Chrome browsers.)

Q: Can I copy test cases from the grid view to an Excel worksheet?

A: Yes. Copy the data and paste it into your Excel worksheet. Any formatting in the test steps, other than multiline, will not

be copied into the worksheet. (This is only supported for Internet Explorer and Chrome browsers.)

Q: Can I tag test cases so that I can view only tests with specific tags?

A: Yes. Tag test cases in a suite with any tag that is important to you. For example, tag all the tests related to login so that

Creating manual tests using the web portal https://msdn.microsoft.com/en-us/library/dd286729(d=printer).aspx

4 of 8 02.09.2016 14:22

you can rerun these tests if a bug is fixed for the login page. Then you can filter on that tag from the test hub. You can add

and edit tags when you edit a test case, or bulk edit tags in the grid view.

Q: Can I share test steps between test cases?

A: Yes. Choose the steps that you want to share. More details are here.

Q: Can I add parameters to a test case to run multiple times with different data?

A: Yes. Choose a test step and then add the parameter. Scroll down to add values for a parameter. More details are here.

Creating manual tests using the web portal https://msdn.microsoft.com/en-us/library/dd286729(d=printer).aspx

5 of 8 02.09.2016 14:22

Q: Can I share the data for parameters between test cases?

A: Yes. That way test cases that use the same parameters can be run with the same data for consistent results. Convert

existing parameters to shared parameters.

After you have created a shared parameter set, open another test case and add the shared parameter set to that test case.

More details are here.

Add, edit and rename your shared parameter sets from the parameters tab. View the test cases that reference them in the

test cases pane.

Creating manual tests using the web portal https://msdn.microsoft.com/en-us/library/dd286729(d=printer).aspx

6 of 8 02.09.2016 14:22

Each shared parameter set is a work item. From the Properties tab, you can view or make changes to this work item. For

example, you can assign owners and track changes.

From Microsoft Test Manager using Test Professional or Visual Studio Enterprise, you can view shared parameters for a

test case but you cannot update them. To update shared parameters, you must use the web portal. There is a hyperlink in

Microsoft Test Manager that takes you to the correct location to edit the shared parameters.

Q: How do I reassign a test to another tester?

A: Select the tests that you want to reassign. Then choose the tester who you want to run the tests.

Q: How can I find out if a test case has been added to other test suites?

A: Select a test case, then choose to view the test suites details. The associated test suites pane shows you any test suite

Creating manual tests using the web portal https://msdn.microsoft.com/en-us/library/dd286729(d=printer).aspx

7 of 8 02.09.2016 14:22

for any test plan that contains this test case. This includes all team projects. Click the test suite to view it. Hover over the

test suite in the pane to view the team project and the test plan for that test suite.

Q: What happens when I delete a test case from a requirement-based test suite?

A: The test case still exists in your team project, but the test case is removed from the test suite. In addition, it is no longer

linked to the backlog item for that test suite.

Try this nextRunning manual tests using the web portal

Or, dig deeper:Share steps between test cases

Test configurations: specifying test platforms

Repeat a test with different data

Explore your app without planning in advance.

© 2016 Microsoft

Creating manual tests using the web portal https://msdn.microsoft.com/en-us/library/dd286729(d=printer).aspx

8 of 8 02.09.2016 14:22

Customizing Code Coverage Analysis

 

By default, the Visual Studio Code Coverage tool analyzes all solution assemblies (.exe/.dll) that are loaded during unit tests.

We recommend that you retain this default, because it works well most of the time. For more information, see Using Code

Coverage to Determine How Much Code is being Tested.

Before customizing the code coverage behavior, consider some alternatives:

I want to exclude the test code from the code coverage results and include only the application code.

Add the ExcludeFromCodeCoverage Attribute to your test class.

I want to include assemblies that are not part of my solution.

Obtain the .pdb files for these assemblies and copy them into the same folder as the assembly .dll files.

To customize the code coverage behavior, copy the sample at the end of this topic and add it to your solution using the file

extension .runsettings. Edit it to your own needs, and then on the Test menu, choose Test Settings, Select Test Settings

file. The remainder of this topic describes this procedure in more detail.

The .runsettings fileAdvanced code coverage settings are specified in a .runsettings file. This is the configuration file used by unit testing

tools. We recommend you copy the sample at the end of this topic and edit it to suit your own needs.

What happened to the .testsettings file I used in Visual Studio 2010?

In Visual Studio 2010, the .testsettings file applies only to unit tests based on the MSTest framework. In Visual

Studio 2012, the testing tools apply not only to MSTest, but also other frameworks such as NUnit and xUnit.net.

The .testsettings file will not work with these. The .runsettings file is designed to customize the test tools in a way

that works with all testing frameworks.

To customize code coverage, you will need to add a .runsettings file to your solution:

Add an .xml file as a solution item with the extension .runsettings:

In Solution Explorer, on the shortcut menu of your solution, choose Add, New Item, and select XML File. Save the

file with a name ending such as CodeCoverage.runsettings

1.

Add the content given in the sample at the end of this topic, and then customize it to your needs as described in

the following sections.

2.

On the Test menu, choose Test Settings, Select Test Settings File and select the file.3.

Visual Studio 2015

Customizing Code Coverage Analysis https://msdn.microsoft.com/en-us/library/jj159530(d=printer).aspx

1 of 9 02.09.2016 13:45

Now when you run Analyze Code Coverage, this .runsettings file will control its behavior. Don’t forget that you

must run code coverage again: your previous coverage results and code coloring aren’t automatically hidden when

you run tests or update your code.

4.

To turn the custom settings off and on, deselect or select the file in the Test, Test Settings menu.5.

Other aspects of unit tests can be configured in the same .runsettings file. For more information, see Unit Test Your Code.

Specifying symbol search paths

Code coverage requires symbols (.pdb files) for assemblies to be present. For assemblies built by your solution, symbol

files are generally present alongside the binary files, and code coverage works automatically. But in some cases, you

might want to include referenced assemblies in your code coverage analysis. In such cases, the .pdb files might not be

adjacent to the binaries, but you can specify the symbol search path in the .runsettings file.

Warning

Symbol resolution can take time, especially when using a remote file location with a lot of assemblies. Therefore,

consider copying remote .pdb files to the same local location as the binary (.dll and .exe) files.

Excluding and including

You can exclude specified assemblies from code coverage analysis. For example:

<SymbolSearchPaths>

<Path>\\mybuildshare\builds\ProjectX</Path>

<!‐‐More paths if required‐‐>

</SymbolSearchPaths>

<ModulePaths>

<Exclude>

<ModulePath>Fabrikam.Math.UnitTest.dll</ModulePath>

<!‐‐ Add more ModulePath nodes here. ‐‐>

XML

minterastlib

Customizing Code Coverage Analysis https://msdn.microsoft.com/en-us/library/jj159530(d=printer).aspx

2 of 9 02.09.2016 13:45

As an alternative, you can specify which assemblies should be included. This approach has the drawback that when you

add more assemblies to the solution, you have to remember to add them to the list:

If <Include> is empty, then code coverage processing includes all assemblies (.dll and .exe files) that are loaded and

for which .pdb files can be found, except for items that match a clause in an <Exclude> list.

Include is processed before Exclude.

Regular expressions

Include and exclude nodes use regular expressions. For more information, see Using Regular Expressions in Visual

Studio. Regular expressions are not the same as wildcards. In particular:

.* matches a string of any characters1.

\. matches a dot ".")2.

\( \) matches parentheses "( )"3.

\\ matches a file path delimiter "\"4.

^ matches the start of the string5.

$ matches the end of the string6.

All matches are case-insensitive.

For example:

</Exclude>

</ModulePaths>

<ModulePaths>

<Include>

<ModulePath>Fabrikam.Math.dll</ModulePath>

<!‐‐ Add more ModulePath nodes here. ‐‐>

</Include>

</ModulePaths>

<ModulePaths>

<Include>

<!‐‐ Include all loaded .dll assemblies (but not .exe assemblies): ‐‐>

<ModulePath>.*\.dll$</ModulePath>

</Include>

<Exclude>

<!‐‐ But exclude some assemblies: ‐‐>

minterastlib

XML

Customizing Code Coverage Analysis https://msdn.microsoft.com/en-us/library/jj159530(d=printer).aspx

3 of 9 02.09.2016 13:45

Warning

If there is an error in a regular expression, such as an unescaped and unmatched parenthesis, then code coverage

analysis will not run.

Other ways to include or exclude elements

See the sample at the end of this topic for examples.

ModulePath – Assemblies specified by assembly file path.

CompanyName – matches assemblies by the Company attribute.

PublicKeyToken – matches signed assemblies by the public key token. For example to match all Visual Studio

components and extensions, use <PublicKeyToken>^B03F5F7F11D50A3A$</PublicKeyToken>.

Source – matches elements by the path name of the source file in which they are defined.

Attribute – matches elements to which a particular attribute is attached. Specify the full name of the attribute,

including "Attribute" at the end of the name.

Function – matches procedures, functions, or methods by fully qualified name.

Matching a function name

Your regular expression must match the fully qualified name of the function, including namespace, class name, method

name and parameter list. For example,

C# or Visual Basic: Fabrikam.Math.LocalMath.SquareRoot(double)

C++: Fabrikam::Math::LocalMath::SquareRoot(double)

<ModulePath>.*\\Fabrikam\.MyTests1\.dll$</ModulePath>

<!‐‐ Exclude all file paths that contain "Temp": ‐‐>

<ModulePath>.*Temp.*</ModulePath>

</Exclude>

</ModulePaths>

<Functions>

<Include>

<!‐‐ Include methods in the Fabrikam namespace: ‐‐>

<Function>^Fabrikam\..*</Function>

<!‐‐ Include all methods named EqualTo: ‐‐>

<Function>.*\.EqualTo\(.*</Function>

XML

Customizing Code Coverage Analysis https://msdn.microsoft.com/en-us/library/jj159530(d=printer).aspx

4 of 9 02.09.2016 13:45

How to specify .runsettings files while running tests

To customize runsettings in Visual Studio tests

Choose Test, Test Settings, Select Test Settings File and select the .runsettings file. The file appears on the Test

Settings menu, and you can select or cancel it. While selected, your .runsettings file applies whenever you use Analyze

Code Coverage.

To customize run settings in a command line test

To run tests from the command line, use vstest.console.exe. The settings file is a parameter of this utility. For more

information, see Using VSTest.console from the command line.

Launch the Visual Studio Developer Command Prompt:

On Windows Start, choose All Programs, Microsoft Visual Studio, Visual Studio Tools, Developer

Command Prompt.

1.

Run:

vstest.console.exe MyTestAssembly.dll /EnableCodeCoverage

/Settings:CodeCoverage.runsettings

2.

To customize run settings in a build definition

You can get code coverage data from a team build.

</Include>

<Exclude>

<!‐‐ Exclude methods in a class or namespace named UnitTest: ‐‐>

<Function>.*\.UnitTest\..*</Function>

</Exclude>

</Functions>

Customizing Code Coverage Analysis https://msdn.microsoft.com/en-us/library/jj159530(d=printer).aspx

5 of 9 02.09.2016 13:45

Make sure your .runsettings file is checked in.1.

In Team Explorer, open Builds, and then add or edit a build definition.2.

On the Process page, expand Automated Tests, Test Source, Run Settings. Select your .runsettings file.

But Test Assembly appears instead of Test Source. When I try to set the Run Settings field, I can only

select .testsettings files.

Under Automated Tests, select Test Assembly, and choose [...] at the end of the line. In the Add/Edit

Test Run dialog box, set Test Runner to Visual Studio Test Runner.

3.

The results are visible in the summary section of the build report.

Sample .runsettings fileCopy this code and edit it to suit your own needs. This is the default .runsettings file.

(For other uses of the .runsettings file, see Configure unit tests by using a .runsettings file.)

<?xml version="1.0" encoding="utf‐8"?>

<!‐‐ File name extension must be .runsettings ‐‐>

<RunSettings>

<DataCollectionRunSettings>

XML

Customizing Code Coverage Analysis https://msdn.microsoft.com/en-us/library/jj159530(d=printer).aspx

6 of 9 02.09.2016 13:45

<DataCollectors>

<DataCollector friendlyName="Code Coverage" uri="datacollector://Microsoft

/CodeCoverage/2.0"

assemblyQualifiedName="Microsoft.VisualStudio.Coverage.DynamicCoverageDataCollector,

Microsoft.VisualStudio.TraceCollector, Version=11.0.0.0, Culture=neutral,

PublicKeyToken=b03f5f7f11d50a3a">

<Configuration>

<CodeCoverage>

<!‐‐

Additional paths to search for .pdb (symbol) files. Symbols must be found for modules

to be instrumented.

If .pdb files are in the same folder as the .dll or .exe files, they are automatically

found. Otherwise, specify them here.

Note that searching for symbols increases code coverage runtime. So keep this small and

local.

‐‐>

<!‐‐

<SymbolSearchPaths>

<Path>C:\Users\User\Documents\Visual Studio 2012\Projects\ProjectX

\bin\Debug</Path>

<Path>\\mybuildshare\builds\ProjectX</Path>

</SymbolSearchPaths>

‐‐>

<!‐‐

About include/exclude lists:

Empty "Include" clauses imply all; empty "Exclude" clauses imply none.

Each element in the list is a regular expression (ECMAScript syntax). See

http://msdn.microsoft.com/library/2k3te2cs.aspx.

An item must first match at least one entry in the include list to be included.

Included items must then not match any entries in the exclude list to remain included.

‐‐>

<!‐‐ Match assembly file paths: ‐‐>

<ModulePaths>

<Include>

<ModulePath>.*\.dll$</ModulePath>

<ModulePath>.*\.exe$</ModulePath>

</Include>

<Exclude>

<ModulePath>.*CPPUnitTestFramework.*</ModulePath>

</Exclude>

</ModulePaths>

<!‐‐ Match fully qualified names of functions: ‐‐>

<!‐‐ (Use "\." to delimit namespaces in C# or Visual Basic, "::" in C++.)

‐‐>

<Functions>

<Exclude>

<Function>^Fabrikam\.UnitTest\..*</Function>

<Function>^std::.*</Function>

<Function>^ATL::.*</Function>

<Function>.*::__GetTestMethodInfo.*</Function>

<Function>^Microsoft::VisualStudio::CppCodeCoverageFramework::.*

Customizing Code Coverage Analysis https://msdn.microsoft.com/en-us/library/jj159530(d=printer).aspx

7 of 9 02.09.2016 13:45

</Function>

<Function>^Microsoft::VisualStudio::CppUnitTestFramework::.*</Function>

</Exclude>

</Functions>

<!‐‐ Match attributes on any code element: ‐‐>

<Attributes>

<Exclude>

<!—Don't forget "Attribute" at the end of the name ‐‐>

<Attribute>^System\.Diagnostics\.DebuggerHiddenAttribute$</Attribute>

<Attribute>^System\.Diagnostics

\.DebuggerNonUserCodeAttribute$</Attribute>

<Attribute>^System\.Runtime

\.CompilerServices.CompilerGeneratedAttribute$</Attribute>

<Attribute>^System\.CodeDom

\.Compiler.GeneratedCodeAttribute$</Attribute>

<Attribute>^System\.Diagnostics

\.CodeAnalysis.ExcludeFromCodeCoverageAttribute$</Attribute>

</Exclude>

</Attributes>

<!‐‐ Match the path of the source files in which each method is defined: ‐‐>

<Sources>

<Exclude>

<Source>.*\\atlmfc\\.*</Source>

<Source>.*\\vctools\\.*</Source>

<Source>.*\\public\\sdk\\.*</Source>

<Source>.*\\microsoft sdks\\.*</Source>

<Source>.*\\vc\\include\\.*</Source>

</Exclude>

</Sources>

<!‐‐ Match the company name property in the assembly: ‐‐>

<CompanyNames>

<Exclude>

<CompanyName>.*microsoft.*</CompanyName>

</Exclude>

</CompanyNames>

<!‐‐ Match the public key token of a signed assembly: ‐‐>

<PublicKeyTokens>

<!‐‐ Exclude Visual Studio extensions: ‐‐>

<Exclude>

<PublicKeyToken>^B77A5C561934E089$</PublicKeyToken>

<PublicKeyToken>^B03F5F7F11D50A3A$</PublicKeyToken>

<PublicKeyToken>^31BF3856AD364E35$</PublicKeyToken>

<PublicKeyToken>^89845DCD8080CC91$</PublicKeyToken>

<PublicKeyToken>^71E9BCE111E9429C$</PublicKeyToken>

<PublicKeyToken>^8F50407C4E9E73B6$</PublicKeyToken>

<PublicKeyToken>^E361AF139669C375$</PublicKeyToken>

</Exclude>

</PublicKeyTokens>

Customizing Code Coverage Analysis https://msdn.microsoft.com/en-us/library/jj159530(d=printer).aspx

8 of 9 02.09.2016 13:45

See AlsoUsing Code Coverage to Determine How Much Code is being Tested

Unit Test Your Code

© 2016 Microsoft

<!‐‐ We recommend you do not change the following values: ‐‐>

<UseVerifiableInstrumentation>True</UseVerifiableInstrumentation>

<AllowLowIntegrityProcesses>True</AllowLowIntegrityProcesses>

<CollectFromChildProcesses>True</CollectFromChildProcesses>

<CollectAspDotNet>False</CollectAspDotNet>

</CodeCoverage>

</Configuration>

</DataCollector>

</DataCollectors>

</DataCollectionRunSettings>

</RunSettings>

Customizing Code Coverage Analysis https://msdn.microsoft.com/en-us/library/jj159530(d=printer).aspx

9 of 9 02.09.2016 13:45

Developer testing tools scenarios andcapabilities

Visual Studio 2015 | Previous version (https://msdn.microsoft.com/library

/dd264943%28v=vs.120%29.aspx)

Maintain code health with unit testing. Visual Studio and Visual Studio Team Services provide a wide

range of powerful tools and techniques for developers to use when testing applications:

Scenarios and capabilities:

Avoid regressions and achieve code coverage with IntelliTest

User interface testing with Coded UI and Selenium

Effective unit testing with Visual Studio Code Coverage

Unit testing with any framework using the high performance Test Explorer

Get started with developer testing tools (getting-started/getting-started-with-developer-

testing)

Avoid regressions and achieve code coverage with IntelliTest

In traditional unit test suites, each test case represents an exemplary usage scenario, and the

assertions embody the relationship between the input and output. Verifying a few such scenarios

might well be enough, but experienced developers know that bugs lurk even in well-tested code,

when correct but untested inputs provoke wrong responses.

Improve coverage, and avoid regressions with IntelliTest. IntelliTest dramatically reduces the effort to

create and maintain unit tests for new or existing code.

Team Services (/docs/overview) > (/docs/test/overview)

Maintain code health with unit testing in Visual Studio Team Services https://www.visualstudio.com/docs/test/developer-testing/developer-testing

1 of 7 02.09.2016 13:26

Introduction to IntelliTest with Visual Studio Enterprise 2015 (http://download.microsoft.com

/download/6/2/B/62B60ECE-B9DC-4E8A-A97C-EA261BFB935E

/Docs/Introduction%20to%20IntelliTest%20with%20Visual%20Studio%20Enterprise%202015.docx

IntelliTest – One Test to rule them all (http://blogs.msdn.com/b/visualstudioalm/archive/2015/07

/05/intellitest-one-test-to-rule-them-all.aspx)

IntelliTest Videos (https://channel9.msdn.com/series/test-tools-in-visual-studio)

Get Started with IntelliTest (https://msdn.microsoft.com/library/dn823749.aspx)

IntelliTest reference manual (intellitest-manual/index)

User interface testing with Coded UI and Selenium

Test your user interface (UI) with best of breed or community approved UI Testing. Coded UI tests

provide a way to create fully automated tests to validate the functionality and behavior of your

application’s user interface. They can automate UI testing across a variety of technologies, including

XAML-based Windows Store apps, browser apps, and SharePoint apps.

Whether you choose best of breed Coded UI Tests or generic browser based UI testing with Selenium,

Visual Studio provides all the tools you need.

Maintain code health with unit testing in Visual Studio Team Services https://www.visualstudio.com/docs/test/developer-testing/developer-testing

2 of 7 02.09.2016 13:26

Use UI Automation To Test Your Code (https://msdn.microsoft.com/library/dd286726.aspx)

Get started creating, editing and maintaining a coded UI test (https://msdn.microsoft.com

/library/ff977233.aspx)

Test Windows Store 8.1 Apps with Coded UI Tests (https://msdn.microsoft.com/library

/dn305948.aspx)

Test Windows Phone 8.1 Apps with Coded UI Tests (https://msdn.microsoft.com/library

/dn747198.aspx)

Test SharePoint 2010 Applications with Coded UI Tests (https://msdn.microsoft.com/library

/jj715713.aspx)

Introduction to Coded UI Tests with Visual Studio Enterprise 2015 (Lab)

(http://download.microsoft.com/download/6/2/B/62B60ECE-B9DC-4E8A-A97C-EA261BFB935E

/Docs/Introduction%20to%20Coded%20UI%20Tests%20with%20Visual%20Studio%20Enterprise%202015.docx

Effective unit testing with Visual Studio Code Coverage

To determine what proportion of your project’s code is actually being tested by coded tests such as

unit tests, you can use the code coverage feature of Visual Studio. To guard effectively against bugs,

your tests should exercise or ‘cover’ a large proportion of your code.

Maintain code health with unit testing in Visual Studio Team Services https://www.visualstudio.com/docs/test/developer-testing/developer-testing

3 of 7 02.09.2016 13:26

Code coverage analysis can be applied to both managed (CLI) and unmanaged (native) code.

Code coverage is an option when you run test methods using Test Explorer. The results table shows

the percentage of the code that was run in each assembly, class, and method. In addition, the source

editor shows you which code has been tested.

Using Code Coverage to Determine How Much Code is being Tested

(https://msdn.microsoft.com/library/dd537628.aspx)

Unit Testing, Code Coverage and Code Clone Analysis with Visual Studio 2015 (Lab)

(http://download.microsoft.com/download/6/2/B/62B60ECE-B9DC-4E8A-A97C-EA261BFB935E

/Docs/Unit%20Testing,%20Code%20Coverage%20and%20Code%20Clone%20Analysis%20with%20Visual%20Studio%202

Customizing Code Coverage Analysis (https://msdn.microsoft.com/library/jj159530.aspx)

Unit testing with any framework using the high performanceTest Explorer

Test Explorer help developers create, manage, and get maximum benefit from unit testing.

Maintain code health with unit testing in Visual Studio Team Services https://www.visualstudio.com/docs/test/developer-testing/developer-testing

4 of 7 02.09.2016 13:26

Get started with Unit Testing (https://msdn.microsoft.com/library/dd264975.aspx)

Run unit tests with Test Explorer (https://msdn.microsoft.com/library/hh270865.aspx)

Writing Unit tests for C/C++ (https://msdn.microsoft.com/library/hh598953.aspx)

Install third-party unit test frameworks (https://msdn.microsoft.com/library/hh598952.aspx)

Visual Studio is also extensible and opens the door for third-party unit testing adapters such as NUnit

and xUnit.net. In addition, the code clone capability goes hand-in-hand with delivering high quality

software by helping you identify blocks of semantically similar code which may be candidates for

common bug fixes or refactoring.

Maintain code health with unit testing in Visual Studio Team Services https://www.visualstudio.com/docs/test/developer-testing/developer-testing

5 of 7 02.09.2016 13:26

Also see

Get started with developer testing tools (getting-started/getting-started-with-developer-

testing)

Speeding up Unit Test Execution in Team Foundation Server (http://blogs.msdn.com

/b/visualstudioalm/archive/2015/07/30/speeding-up-test-execution-in-tfs.aspx)

Parallel and Context Sensitive Unit Test Execution (https://blogs.msdn.microsoft.com

/visualstudioalm/2016/02/08/parallel-and-context-sensitive-test-execution-with-visual-studio-

2015-update-1/)

Unit Testing, Code Coverage and Code Clone Analysis with Visual Studio 2015 (Lab)

(http://download.microsoft.com/download/6/2/B/62B60ECE-B9DC-4E8A-A97C-EA261BFB935E

/Docs/Unit%20Testing,%20Code%20Coverage%20and%20Code%20Clone%20Analysis%20with%20Visual%20Studio%202

Maintain code health with unit testing in Visual Studio Team Services https://www.visualstudio.com/docs/test/developer-testing/developer-testing

6 of 7 02.09.2016 13:26

Help and support

Submit bugs through Connect (https://connect.microsoft.com/visualstudio), make suggestions on

Uservoice (http://visualstudio.uservoice.com/forums/121579-visual-studio), and send quick thoughts

using the Send-a-Smile link in the Visual Studio, Team Services, or TFS title bar. We look forward to

your feedback.

© 2016 Microsoft

Maintain code health with unit testing in Visual Studio Team Services https://www.visualstudio.com/docs/test/developer-testing/developer-testing

7 of 7 02.09.2016 13:26

Exploratory test web apps directly in your browserLast Updated: 8/12/2016

IN THIS ARTICLE

Team Services | TFS 2015

The Exploratory Testing extension lets everyone in the team, such as developers, testers, product owners, leads, and managers, contribute to thequality of the product.

What is the Exploratory Testing extension?

The Exploratory Testing extension is a simple browser‐based extension you can use to test web apps anytime and anywhere, and is simple enoughfor everyone in the team to use. It helps to improve productivity by allowing you to spend more time finding issues, and less time filing them.

Using the extension is a simple, three step process.

 ShareTeam Services  > Test  > Exploratory test your web app in Visual…

Table of contents

Testing scenarios

Manual and exploratory testing

Overview

Get started with manual testing

Advanced manual testingtechniques

Manual testing with MicrosoftTest Manager

Get started with exploratorytesting

Install the Exploratory Testingextension

Test in Connected mode

Test in Standalone mode

Advanced exploratory testingtechniques

Exploratory testing with MicrosoftTest Manager

Performance testing

Continuous testing

Developer testing tools

Test lab management

API reference for test tools

Free Visual Studio 

Page 1 of 4   Assembled by RunPDF.com

Capture your findings quickly and easily using the tools in the extension. The Exploratory Testing extension helps capture a large volume and widevariety of information that clearly highlights issues, and includes rich diagnostic data that can serve as a starting point for debugging. In Connectedmode, it can capture screenshots, notes, and screen recordings; which can be annotated. The extension automatically records user actions as animage action log; system information about the browser, operating system, memory, and more; and page load data including resource usage andnavigation timelines. In Standalone mode, the extension can capture screenshots and inline annotations.

Create work items such as bugs, tasks, and test cases directly from the Exploratory Testing extension. The captured findings automatically become apart of the work item. Users can file a bug to report an issue with the product, or create a task that indicates a new work requirement. The extensioncan also be used to create test cases for scenarios discovered during exploration.

Collaborate with your team by sharing your findings. In Connected mode, the powerful integration with team systems maximizes traceability, andsimplifies bug or task tracking and management. In Standalone mode you can export your session report and share it with the rest of the team.

Install the Exploratory Testing extension

1. Download and install Chrome, then open it, if you haven't already.

2. Go to Visual Studio Marketplace > Exploratory Testing and install the extension.

3. Add the Exploratory Testing extension to Chrome.

Select an exploratory testing mode

1. Go to Chrome and open the extension.

Page 2 of 4   Assembled by RunPDF.com

2. Decide if you want to use the extension in Connected or Standalone mode.

Connected mode is a full functional mode that supports Visual Studio Team Services and Team Foundation Server 2015 and above. In thismode, users can quickly capture all their issues using annotated screenshots, notes, and screen recordings; create bugs, tasks, and test casesdirectly; and collaborate with the rest of the team using the integrated experience to provide traceability, avoid duplication, update existingwork items, and get insights.

Standalone mode is a self‐contained mode with no dependency on Visual Studio Team Services or Team Foundation Server. Any team, largeor small, can use this mode to capture issues using screenshots with inline annotations and notes, then share the results using a session reportand manually create work items based on the results.

Try this next

Use the Exploratory Testing extension in Connected modeUse the Exploratory Testing extension in Standalone mode

Q&A

Q: Does the extension support other browsers?

A: The Exploratory Testing extension is currently available only for Chrome. Edge and Firefox support is planned.

Q: Does the extension support Team Foundation Server?

A: See extension details here.

Help and support

Submit bugs through Connect, make suggestions on Uservoice, and send quick thoughts using the Send‐a‐Smile link in the Visual Studio, TeamServices, or TFS title bar. We look forward to your feedback.

Visual StudioMy Visual Studio

Manage Visual Studio

Marketplace

Integrate

Related SitesVisual Studio documentation

MSDN Home

Channel 9

Azure

Visual Studio Blog

ProductsVisual Studio

SupportGet Support

Page 3 of 4   Assembled by RunPDF.com

Visual Studio Team Services

Visual Studio Code

Download

Compare

How to Buy

Submit a Bug

Submit an Idea

Forums

United States ﴾English﴿ Contact us Jobs Privacy Terms of use Trademarks

© 2016 Microsoft

Is this page helpful?

YES NO

Page 4 of 4   Assembled by RunPDF.com

| Marketplace

Visual Studio Team Services > Test > Exploratory Testing ‐ Capture, Create & Collaborate

Exploratory Testing ‐ Capture, Create & CollaborateMicrosoft | 14250 installs | ﴾27﴿

Exploratory Testing for everyone

Install

Categories

Test

Tags

adhoc testing exploratory testing manual testing

test execution

Works withVisual Studio Team Services

ResourcesSupportGet StartedLicense

More InfoVersion 1.0.105.1

Last updated Friday, September 02, 2016 06:53:31

Publisher Microsoft

Report Report Abuse

Sign in

Exploratory Testing now in 3 easy steps – capture, create & collaborate.

Everyone in the team, be it developers, testers, product owners, UX, leads/managers etc., can perform exploratorytesting of web‐apps, right from the Chrome browser on any platform ﴾Windows, Mac or Linux﴿. Support for Edgebrowser coming soon.

Capture

Use various capture formats‐ notes, screenshots with annotations, image action logs ﴾user actions﴿, and screenrecordings. Test your applications on real devices using cloud providers like Perfecto, or test them on browser basedemulators.

Create

Quick creation of bugs, tasks and test cases with all the captured information automatically attached for you.

Page 1 of 3   Assembled by RunPDF.com

Collaborate

Export your session report in the standalone mode and share your findings with the rest of the team. Additionallyconnect to your account on Visual Studio Team Services / Team Foundation Server ﴾2015 RTM or above﴿ to leveragean integrated experience spanning E2E traceability, avoiding duplicate bugs, simplified tracking & triaging of bugs/tasksand gathering rich insights across exploratory testing sessions.

Features and Pricing

Free during Preview*

Only Visual Studio Team Services users ﴾not stakeholders﴿ can be assigned the Exploratory Testing extension to beused in Connected ﴾Team﴿ mode. Learn more about Visual Studio Team Services user pricing.

After the Preview period is over, the Exploratory Testing extension will be included with the Test Manager extensionand with these Visual Studio subscriptions:

Visual Studio Enterprise ‐ monthly

Page 2 of 3   Assembled by RunPDF.com

© 2016 Microsoft

Write a review

See more

User Reviews

﴾27﴿

Fredrik Almén 8/24/2016

Love the concept! Really nice work!Have some questions,1. I cant find the export button under session timeline, and i cant find any sessions i have done either, I get forwardedto tfs but dont see ... Read More

Microsoft8/24/2016

Thanks Fredrik for appreciation!

About your questions ‐ ﴾1﴿ ‐ Export button is visible in Standalone mode ﴾From Connection page﴿, which is not connected toTFS/VSTS. It is for easy access of all s... Read More

Nimesh Sankhalpara 8/12/2016

Great little product.. Any chances of adding this extension to IE??

Microsoft8/17/2016

Thanks Nimesh for the appreciation! We do have plans to get this extension on Edge and Firefox, howeverwe have no plans around getting it on IE.

Valdir Viana 8/9/2016

Awesome!!

Carly Rimmer 8/8/2016

Can you advise where this tool saves the session reports? there is no indication as to where this is?

Microsoft8/9/2016

You can view your completed sessions using the Exploratory Testing ‐ Insights page. There are 2 ways to get to the insights page:a. From within the extension, click on “View completed exploratory s... Read More

sphereqube13 7/28/2016

We are trying to use Exploratory Testing chrome Plugin. We trying to connect our project TFS URL with Exploratory, weare not getting our team project into Exploratory. We can see its just connecting... Read More

Microsoft8/3/2016

Hi

Thanks for reporting this issue. What is the TFS version you are on? We support TFS 2015 onwards. In caseit is TFS 2015+, please write to vsoxt [at] microsoft [dot] com, and we will take this d... Read More

Contact us Jobs Privacy Terms of use Trademarks

Visual Studio Enterprise ‐ annualVisual Studio Enterprise with MSDNMSDN PlatformsVisual Studio Test Professional with MSDN

Page 3 of 3   Assembled by RunPDF.com

How to: Configure Unit Tests to Target AnEarlier Version of the .NET Framework

 

When you create a test project in Microsoft Visual Studio, the most recent version of the .NET Framework is set as the target,

by default. Additionally, if you upgrade test projects from previous versions of Visual Studio, they are upgraded to target

the most recent version of the .NET Framework. By editing the project properties, you can explicitly re-target the project to

earlier versions of the .NET Framework.

You can create unit test projects that target specific versions of the .NET Framework. The targeted version must be 3.5 or

later, and cannot be a client version. Visual Studio enables the following basic support for unit tests that target specific

versions:

You can create unit test projects and target them to a specific version of the .NET Framework.

You can run unit tests that target a specific version of the .NET Framework from Visual Studio on your local machine.

You can run unit tests that target a specific version of the .NET Framework by using MSTest.exe from the command

prompt.

You can run unit tests on a build agent as part of a build.

Testing SharePoint Applications

The capabilities listed above also enable you to write unit tests and integration tests for SharePoint applications using Visual

Studio. For more information about how to develop SharePoint applications using Visual Studio, see Create SharePoint

Solutions, Building and Debugging SharePoint Solutions and Verifying and Debugging SharePoint Code.

Limitations

The following limitations apply when you re-target your test projects to use earlier versions of the .NET Framework:

In the .NET Framework 3.5, multitargeting is supported for test projects that contain only unit tests. The .NET

Framework 3.5 does not support any other test type, such as coded UI or load test. The re-targeting is blocked for

test types other than unit tests.

Execution of tests that are targeted at an earlier version of the .NET Framework is supported only in the default host

adapter. It is not supported in the ASP.NET host adapter. ASP.NET applications that have to run in the ASP.NET

Development Server context must be compatible with the current version of the .NET Framework.

Data collection support is disabled when you run tests that support .NET Framework 3.5 multitargeting. You can run

code coverage by using the Visual Studio command-line tools.

Unit tests that use .NET Framework 3.5 cannot run on a remote machine.

You cannot target unit tests to earlier client versions of the framework.

Visual Studio 2015

How to: Configure Unit Tests to Target An Earlier Version of the .NET F... https://msdn.microsoft.com/en-us/library/gg601487(d=printer).aspx

1 of 5 02.09.2016 13:53

Re-targeting to a specific version of the .NET Framework for Visual

Basic Unit Test Projects

Create a new Visual Basic unit test project. On the File menu, choose New and then choose Project.

The New Project dialog box is displayed.

1.

Under Installed Templates, expand Visual Basic. Select Test and then select the Test Project template.2.

In the Name text box, type a name for your Visual Basic test project and then choose OK.3.

In Solution Explorer, choose Properties from the shortcut menu of the new Visual Basic test project.

The properties for your Visual Basic test project are displayed.

4.

On the Compile tab choose Advanced Compile Options as shown in the following illustration.5.

Use the Target framework (all configurations) drop-down list to change the target framework to .NET

Framework 3.5 or a later version as shown in callout B in the following illustration. You should not specify a client

version.

6.

How to: Configure Unit Tests to Target An Earlier Version of the .NET F... https://msdn.microsoft.com/en-us/library/gg601487(d=printer).aspx

2 of 5 02.09.2016 13:53

Re-targeting to a specific version of the .NET Framework for Visual

C# Unit Test Projects

Create a new Visual C# unit-test project. On the File menu, choose New and then choose Project.

The New Project dialog box is displayed.

1.

Under Installed Templates, expand Visual C#. Select Test and then select the Test Project template.2.

In the Name text box, type a name for your Visual C# test project and then choose OK.3.

In Solution Explorer, choose Properties from the shortcut menu of your new Visual C# test project.

The properties for your Visual C# test project are displayed.

4.

On the Application tab choose Target framework and then choose .NET Framework 3.5 or a later version from

the drop-down list to change the target framework.as shown in the following illustration. You should not specify a

client version.

5.

How to: Configure Unit Tests to Target An Earlier Version of the .NET F... https://msdn.microsoft.com/en-us/library/gg601487(d=printer).aspx

3 of 5 02.09.2016 13:53

Re-targeting to a specific version of the .NET Framework for

C++/CLI Unit Test Projects

Create a new C++ unit-test project. On the File menu, select New and then click Project.

The New Project dialog box is displayed.

Warning

To build C++/CLI unit tests for a previous version of the .NET framework for Visual C++, you must use the

corresponding version of Visual Studio. For example, to target the .NET Framework 3.5, you must install Visual

Studio 2008 and Visual Studio 2008 Service Pack 1.

1.

Under Installed Templates, expand Visual C ++. Select Test and then select the Test Project template.2.

In the Name text box, type a name for your Visual C++ test project and then click OK.3.

In Solution Explorer, choose Unload Project from your new Visual C++ test project.4.

In Solution Explorer, choose the unloaded Visual C++ test project and then choose Edit <project name>.vcxproj.

The .vcxproj file opens in the editor.

5.

Set the TargetFrameworkVersion to version 3.5 or a later version in the PropertyGroup labeled "Globals". You

should not specify a client version:

6.

<PropertyGroup Label="Globals">

<TargetName>DefaultTest</TargetName>

<ProjectTypes>{3AC096D0‐A1C2‐E12C‐1390‐A8335801FDAB};{8BC9CEB8‐8B4A‐

11D0‐8D11‐00A0C91BC942}</ProjectTypes>

<ProjectGUID>{CE16D77A‐E364‐4ACD‐948B‐1EB6218B0EA3}</ProjectGUID>

<TargetFrameworkVersion>3.5</TargetFrameworkVersion>

<Keyword>ManagedCProj</Keyword>

How to: Configure Unit Tests to Target An Earlier Version of the .NET F... https://msdn.microsoft.com/en-us/library/gg601487(d=printer).aspx

4 of 5 02.09.2016 13:53

Save and close the .vcxproj file.7.

In Solution Explorer, choose select Reload Project from the shortcut menu of your new Visual C++ test project.8.

See AlsoCreating and Running Unit Tests for Existing Code

Create SharePoint Solutions

Building and Debugging SharePoint Solutions

Advanced Compiler Settings Dialog Box (Visual Basic)

© 2016 Microsoft

<RootNamespace>CPP_Test</RootNamespace>

</PropertyGroup>

How to: Configure Unit Tests to Target An Earlier Version of the .NET F... https://msdn.microsoft.com/en-us/library/gg601487(d=printer).aspx

5 of 5 02.09.2016 13:53

IntelliTest Reference ManualLast Updated: 8/4/2016

IN THIS ARTICLE

| Visual Studio 2015 |

Contents

Overview of IntelliTest

The Hello World of IntelliTestLimitations

NondeterminismConcurrencyNative codePlatformLanguageSymbolic reasoningIncorrect stack traces

Further reading

Get started with IntelliTest

Important attributesImportant static helper classes

Test Generation

Test generatorsParameterized unit testingGeneric parameterized unit testingAllowing exceptionsTesting internal typesAssumptions and assertionsPreconditionPostconditionTest failuresSetup and tear downFurther reading

Input Generation

Constraint solverDynamic code coverageIntegers and floatsObjectsInstantiating existing classesVisibilityParameterized mocksStructsArrays and stringsObtaining additional inputs

 ShareTeam Services  > Test  > IntelliTest Reference Manual | Microsoft…

Table of contents

Testing scenarios

Manual and exploratory testing

Performance testing

Continuous testing

Developer testing tools

Overview

Get started

Create Unit Tests command

Generate tests with IntelliTest

Run tests with Test Explorer

Determine code coverage

System test with Visual Studio

Create system tests with VS

Run system tests with VS

Submit bugs in VS

Submit bugs in TFS

IntelliTest reference manual

Overview

Get started

Index to sections

Test generation

Dynamic symbolic execution

Exploration bounds

Attribute glossary

Settings waterfall

Static helper classes

Warnings and errors

Test lab management

API reference for test tools

Free Visual Studio 

Page 1 of 3   Assembled by RunPDF.com

Further reading

Exploration Bounds

MaxConstraintSolverTimeMaxConstraintSolverMemoryMaxBranchesMaxCallsMaxStackMaxConditionsMaxRunsMaxRunsWithoutNewTestsMaxRunsWithUniquePathsMaxExceptionsTestExcludePathBoundsExceededTestEmissionFilterTestEmissionBranchHits

Attribute Glossary

PexAssumeNotNullPexClassPexGenericArgumentsPexMethodPexExplorationAttributeBasePexAssemblySettingsPexAssemblyUnderTestPexInstrumentAssemblyAttributePexUseTypePexAllowedExceptionPexAllowedExceptionFromAssemblyPexAllowedExceptionFromTypePexAllowedExceptionFromTypeUnderTest

Settings Waterfall

Static Helper Classes

PexAssumePexAssertPexChoosePexObservePexSymbolicValue

Warnings and Errors

MaxBranches exceededMaxConstraintSolverTime exceededMaxConditions exceededMaxCalls exceededMaxStack exceededMaxRuns exceededMaxRunsWithoutNewTests exceededCannot concretize solutionNeed help to construct objectNeed help to find typesUsable type guessedUnexpected failure during explorationTargetInvocationExceptionUninstrumented method calledExternal method calledUninstrumentable method called

Page 2 of 3   Assembled by RunPDF.com

Testability issueLimitationObserved call mismatchValue stored in static field

Got feedback?

Post your ideas and feature requests on UserVoice.

Tell us what you think with Send‐a‐Smile on the title bar.

Visual StudioMy Visual Studio

Manage Visual Studio

Marketplace

Integrate

Related SitesVisual Studio documentation

MSDN Home

Channel 9

Azure

Visual Studio Blog

ProductsVisual Studio

Visual Studio Team Services

Visual Studio Code

Download

Compare

How to Buy

SupportGet Support

Submit a Bug

Submit an Idea

Forums

United States ﴾English﴿ Contact us Jobs Privacy Terms of use Trademarks

© 2016 Microsoft

Is this page helpful?

YES NO

Page 3 of 3   Assembled by RunPDF.com

Hands-On LabIntroduction to Coded UI Tests with Visual Studio Enterprise 2015

Lab version: 14.0.23107.0

Last updated: 9/11/2015

TABLE OF CONTENT

INTRODUCTION TO CODED UI TESTS WITH VISUAL STUDIO ENTERPRISE 2015 ...................................................... 1

........................................................................................................................................................................... 1

OVERVIEW .......................................................................................................................................................... 4

CODED UI TESTS PROVIDE A WAY TO CREATE FULLY AUTOMATED TESTS TO VALIDATE THE FUNCTIONALITY AND BEHAVIOR OF YOUR APPLICATION’S USER INTERFACE. IN THIS LAB, YOU WILL GAIN A BASIC UNDERSTANDING OF CODED UI TESTS BY CREATING A NEW TEST AND ADDING VALIDATION LOGIC TO IT. ............................................. 4

PREREQUISITES ................................................................................................................................................... 4

IN ORDER TO COMPLETE THIS LAB YOU WILL NEED THE VISUAL STUDIO 2015 VIRTUAL MACHINE PROVIDED BY MICROSOFT. FOR MORE INFORMATION ON ACQUIRING AND USING THIS VIRTUAL MACHINE, PLEASE SEE THIS BLOG POST. ......................................................................................................................................................... 4

EXERCISES ........................................................................................................................................................... 4

THIS HANDS-ON LAB INCLUDES THE FOLLOWING EXERCISES: ............................................................................... 4

INTRODUCTION TO CODE GENERATION FROM ACTION RECORDINGS ................................................................... 4

INTRODUCTION TO CODE GENERATION USING CODED UI TEST BUILDER .............................................................. 4

DATA DRIVEN DEMONSTRATION FOR CODED UI TEST ........................................................................................... 4

ESTIMATED TIME TO COMPLETE THIS LAB: 60 MINUTES. ...................................................................................... 4

EXERCISE 1: CODE GENERATION FROM ACTION RECORDINGS .............................................................................. 5

LOG IN AS JULIA (VSALM\JULIA). ALL USER PASSWORDS ARE P2SSW0RD. ............................................................ 5

LAUNCH VISUAL STUDIO 2015 FROM THE TASKBAR AND OPEN TEAM EXPLORER. ................................................ 5

SELECT THE CONNECT TO TEAM PROJECTS BUTTON. ............................................................................................ 5

........................................................................................................................................................................... 5

FIGURE ............................................................................................................................................................... 5

CONNECTING TO A DIFFERENT TEAM PROJECT ..................................................................................................... 5

IN TEAM EXPLORER – CONNECT, DOUBLE-CLICK ON THE TAILSPIN TOYS PROJECT. ................................................ 5

........................................................................................................................................................................... 5

FIGURE ............................................................................................................................................................... 5

LOADING THE TAILSPIN TOYS PROJECT ................................................................................................................. 5

START A NEW TESTING PROJECT (FILE | NEW | PROJECT…). .................................................................................. 5

IN THE NEW PROJECT WINDOW, SELECT THE CODED UI TEST PROJECT TEMPLATE FROM VISUAL C# | TEST, THEN SELECT THE OK BUTTON TO CREATE THE TEST PROJECT. ....................................................................................... 5

........................................................................................................................................................................... 5

FIGURE ............................................................................................................................................................... 5

CREATING A CODED UI TEST PROJECT .................................................................................................................. 5

THERE ARE TWO WAYS TO GENERATE CODE FOR THIS NEW CODED UI TEST. THE FIRST AND DEFAULT OPTION IS TO USE THE CODED UI TEST BUILDER, WHICH ALLOWS YOU TO GENERATE TEST CODE BY MANUALLY WALKING THROUGH A TEST SCENARIO. THE SECOND OPTION IS TO USE AN EXISTING ACTION RECORDING. SELECT THE SECOND OPTION TO USE AN EXISTING ACTION RECORDING AND SELECT THE OK BUTTON TO CONTINUE. ............ 5

........................................................................................................................................................................... 5

FIGURE ............................................................................................................................................................... 5

USING AN EXISTING ACTION RECORDING FOR TEST GENERATION ........................................................................ 5

SELECT THE IDS RADIO BUTTON AND ENTER 41. FOR THE PURPOSES OF THIS LAB, ASSUME THAT WE ALREADY KNOW THAT THIS IS THE ID OF A TEST CASE WITH AN ACTION RECORDING. ......................................................... 6

SELECT THE FIND BUTTON TO EXECUTE THE WORK ITEM QUERY. ......................................................................... 6

........................................................................................................................................................................... 6

FIGURE ............................................................................................................................................................... 6

FINDING A TEST CASE WITH ACTION RECORDING ................................................................................................. 6

SELECT THE OK BUTTON TO GENERATE A CODED UI TEST FROM THE ACTION RECORDING. ................................... 6

........................................................................................................................................................................... 6

FIGURE ............................................................................................................................................................... 6

SELECTING A TEST CASE WITH ACTION RECORDING ............................................................................................. 6

NAVIGATE TO THE CODEDUITESTMETHOD1 METHOD IN THE GENERATED CODEDUITEST1.CS FILE. EACH LINE REPRESENTS A STEP FROM THE ACTION RECORDING USED DURING TEST GENERATION. ....................................... 6

PLACE THE CURSOR SOMEWHERE ON THE LINE OF THE FIRST METHOD CALL, OPENHTTPWWWTAILSPINTOYSCOM(), AND THEN PRESS ALT + F12 TO PEEK AT ITS DEFINITION. THIS WILL LOAD THE UIMAP CLASS FROM THE UIMAP.DESIGNER.CS FILE, WHICH CONTAINS THE GENERATED TEST LOGIC. THIS GENERATED METHOD LAUNCHES INTERNET EXPLORER AND NAVIGATES TO A SPECIFIED URL. .............................. 6

........................................................................................................................................................................... 6

FIGURE ............................................................................................................................................................... 6

USING PEEK DEFINITION ...................................................................................................................................... 6

IN THE PEEK DEFINITION WINDOW, SCROLL DOWN TO THE CLICKFOURTHCOFFEEFLYER METHOD. THIS GENERATED METHOD TESTS CLICKING ON A “FOURTH COFFEE FLYER” HYPERLINK THAT IS IN THE TAILSPIN TOYS WEB APPLICATION. ............................................................................................................................................. 6

........................................................................................................................................................................... 6

FIGURE .............................................................................................................................................................. 6

GENERATED TEST METHOD EXAMPLE .................................................................................................................. 6

THE CLICKFOURTHCOFFEEFLYER TEST METHOD DOES NOT SPECIFY THE HYPERLINK PARAMETERS DIRECTLY, BUT INSTEAD REFERS TO THE

“UIBLANKPAGEWINDOWSINTEWINDOW.UIHOMETAILSPINTOYSDOCUMENT1.UIFOURTHCOFFEEFLYERHYPERLINK” PROPERTY. SELECT THE “UIFOURTHCOFFEEFLYERHYPERLINK” PROPERTY (MAY NEED TO SCROLL TO THE RIGHT) AND THEN PRESS ALT + F12 TO VIEW ITS DEFINITION. .......................................................................................... 6

........................................................................................................................................................................... 6

FIGURE ............................................................................................................................................................... 6

DEFINITION OF HYPERLINK PROPERTY ................................................................................................................. 6

NOTE: THE HTMLHYPERLINK INSTANCE THAT IS CREATED FOR THE UIFOURTHCOFFEEFLYERHYPERLINK PROPERTY HAS A NUMBER OF SEARCH AND FILTER PROPERTIES APPLIED THAT AID THE TEST FRAMEWORK IN LOCATING THECORRECT HTML HYPERLINK. IN THE EVENT THAT THE WEB APPLICATION CHANGES SOME OF THE LINK PROPERTIES, SUCH AS THE INNER TEXT, THE TEST HARNESS MAY STILL BE ABLE TO FIND THE HYPERLINK USING THE REMAINING SEARCH PROPERTIES. ................................................................................................................ 7

PRESS ESCAPE TO CLOSE THE CODE PEEK WINDOW AND RETURN TO THE CODEDUITEST1.CS FILE. ....................... 7

RIGHT-CLICK SOMEWHERE WITHIN THE CODEDUITEST1.CS SOURCE FILE AND SELECT RUN TESTS. DO NOT TOUCH THE MOUSE OR KEYBOARD DURING THE TESTS. .................................................................................................. 7

........................................................................................................................................................................... 7

FIGURE ............................................................................................................................................................... 7

LOCATION OF RUN TESTS COMMAND .................................................................................................................. 7

AS THE TESTS RUN, AN INSTANCE OF INTERNET EXPLORER WILL BE OPENED AND ACTIONS AUTOMATICALLY TAKEN AS THEY ARE DEFINED IN THE CODED UI TEST. THE TEST RUNS MORE THAN ONCE BECAUSE THE ORIGINAL MANUAL TEST THAT THIS CODED UI TEST WAS GENERATED FROM HAD MULTIPLE ROWS OF TEST PARAMETERS. . 7

........................................................................................................................................................................... 7

FIGURE .............................................................................................................................................................. 7

EXAMPLE OF CODED UI TEST RUNNING ............................................................................................................... 7

VERIFY THAT THE TEST PASSED BY VIEWING THE TEST EXPLORER WINDOW. IN THIS CASE, HOWEVER, WE ARE NOT PERFORMING ANY VALIDATION AFTER ANY OF THE STEPS. ........................................................................... 7

........................................................................................................................................................................... 7

FIGURE ............................................................................................................................................................... 7

TEST EXPLORER WINDOW SHOWING PASSED TEST .............................................................................................. 7

EXERCISE 2: CODE GENERATION USING CODED UI TEST BUILDER .......................................................................... 7

IN THIS EXERCISE, YOU WILL LEARN HOW TO USE THE CODED UI TEST BUILDER TO GENERATE TEST CODE FOR THETAILSPIN TOYS WEB APPLICATION AND MODIFY THE GENERATED CODE IN ORDER TO ENABLE DATA DRIVEN TESTING. ............................................................................................................................................................. 7

OPEN INTERNET EXPLORER AND SELECT THE TAILSPIN TOYS BUTTON FROM THE FAVORITES BAR. ........................ 7

SELECT THE MODEL AIRPLANES LINK. .................................................................................................................. 7

SELECT THE FOURTH COFFEE FLYER LINK. ............................................................................................................. 7

SELECT THE ADD TO CART LINK TO LOAD THE SHOPPING CART. ............................................................................ 7

RETURN TO VISUAL STUDIO, LOCATE THE CODEDUITESTMETHOD1 METHOD IN THE CODEDUITEST1.CS FILE, AND ADD A BLANK LINE AFTER THE CALL TO THE “THIS.UIMAP.CLICKONWHITESPACEINWEBSITE” METHOD. ................ 8

........................................................................................................................................................................... 8

FIGURE ............................................................................................................................................................... 8

ADDING BLANK LINE TO TEST SOURCE ................................................................................................................. 8

RIGHT-CLICK AT THE LOCATION OF THE BLANK LINE AND SELECT GENERATE CODE FOR CODED UI TEST | USE CODED UI TEST BUILDER… FROM THE CONTEXT MENU. THIS WILL LOAD THE CODED UI TEST BUILDER WINDOW (WHICH IS ALWAYS DISPLAYED OVER OTHER WINDOWS) AND THE INTERNET EXPLORER INSTANCE THAT WE PREVIOUSLY LEFT OPEN. ...................................................................................................................................... 8

........................................................................................................................................................................... 8

FIGURE ............................................................................................................................................................... 8

STARTING THE CODED UI TEST BUILDER ............................................................................................................... 8

NOTE: THE CODED UI TEST BUILDER IS USED TO RECORD ACTIONS AND ASSERTIONS WITHIN THE USER INTERFACE WHICH ARE THEN CONVERTED TO CODE. ........................................................................................... 8

NOW WE WILL ADD AN ASSERTION TO VERIFY THAT THE QUANTITY TEXTBOX IS EQUAL TO A VALUE OF 1. DRAG AND DROP THE TARGET ICON FROM THE CODED UI TEST BUILDER TOOL WINDOW ONTO THE QUANTITY TEXTBOX IN INTERNET EXPLORER. THIS ACTION WILL LOAD THE CODED UI TEST BUILDER WINDOW. ................... 8

........................................................................................................................................................................... 8

FIGURE ............................................................................................................................................................... 8

SELECTING AN ELEMENT TO USE FOR ASSERTION ................................................................................................ 8

IN THE CODED UI TEST BUILDER WINDOW, SELECT THE TEXT PROPERTY AND CLICK ON ADD ASSERTION. THIS WILL LOAD A DIALOG TO FINALIZE THE ASSERTION OPTIONS TO USE. .................................................................. 8

........................................................................................................................................................................... 8

FIGURE ............................................................................................................................................................... 8

CODED UI TEST BUILDER WINDOW ...................................................................................................................... 8

USE THE AREEQUAL COMPARATOR AND A COMPARISON VALUE OF ‘1’. SELECT THE OK BUTTON TO CONTINUE. ... 8

........................................................................................................................................................................... 8

FIGURE ............................................................................................................................................................... 8

SELECTING THE COMPARATOR TYPE AND VALUE .................................................................................................. 8

VERIFY THAT A CHECKBOX HAS BEEN ADDED TO THE TEXT PROPERTY ROW. ........................................................ 9

........................................................................................................................................................................... 9

FIGURE ............................................................................................................................................................... 9

TEXT PROPERTY SHOWING ASSERTION CHECKBOX .............................................................................................. 9

SELECT THE GENERATE CODE BUTTON FROM THE CODED UI TEST BUILDER TOOL WINDOW. ................................ 9

........................................................................................................................................................................... 9

FIGURE ............................................................................................................................................................... 9

GENERATE CODE BUTTON LOCATION ................................................................................................................... 9

IN THE GENERATE CODE WINDOW, USE QUANTITYEQUALSONE FOR THE METHOD NAME AND SELECT THE ADD AND GENERATE BUTTON TO GENERATE THE VALIDATION CODE. .......................................................................... 9

........................................................................................................................................................................... 9

FIGURE ............................................................................................................................................................... 9

GENERATING ASSERTION CODE ........................................................................................................................... 9

REMOVE THE “FOURTH COFFEE FLYER” ITEM FROM THE SHOPPING CART. ........................................................... 9

CLOSE THE CODED UI TEST BUILDER. .................................................................................................................... 9

CLOSE THE INTERNET EXPLORER WINDOW AND RETURN TO VISUAL STUDIO. ...................................................... 9

NOTE THAT THE ASSERTION CODE GENERATION HAS ADDED THE NEW VALIDATION STEP. .................................... 9

........................................................................................................................................................................... 9

FIGURE ............................................................................................................................................................... 9

NEWLY CREATED ASSERTION STEP FOR CODED UI TEST ........................................................................................ 9

RIGHT-CLICK AND SELECT RUN TESTS TO RUN THE TESTS WITH THE NEW VALIDATION STEPS. THE TEST SHOULD COMPLETE SUCCESSFULLY. ................................................................................................................................... 9

EXERCISE 3: DATA DRIVEN DEMONSTRATION FOR CODED UI TEST ...................................................................... 10

IN THIS EXERCISE, YOU WILL ADD ANOTHER SET OF TEST PARAMETER VALUES TO THE TEST CASE IN ORDER TO DEMONSTRATE THAT THESE TEST PARAMETERS ARE HOOKED UP TO THE CODED UI TEST AND THAT THE VALIDATION THAT WE RECENTLY ADDED IN IS PERFORMING AS EXPECTED. ........................................................ 10

LAUNCH MICROSOFT TEST MANAGER FROM THE TASKBAR. ............................................................................... 10

IF NOT ALREADY CONNECTED, CONNECT TO THE ITERATION 2 TEST PLAN FROM THE TAILSPIN TOYS TEAM PROJECT. YOU CAN CHANGE THIS SELECTION BY CLICKING ON THE HOME BUTTON. ........................................... 10

......................................................................................................................................................................... 10

FIGURE ............................................................................................................................................................. 10

CONNECTING TO TAILSPIN TOYS TEAM PROJECT ................................................................................................ 10

......................................................................................................................................................................... 10

FIGURE ............................................................................................................................................................. 10

CONNECTING TO ITERATION 2 TEST PLAN .......................................................................................................... 10

......................................................................................................................................................................... 10

FIGURE ............................................................................................................................................................. 10

LOCATION OF CURRENTLY SELECTED TEAM PROJECT AND TEST PLAN ................................................................. 10

SELECT THE TEST TAB AND SELECT TEST SUITE 7 NAMED “AS A CUSTOMER I SHOULD BE ABLE TO REMOVE ITEMS FROM MY SHOPPING CART”. ............................................................................................................................. 10

......................................................................................................................................................................... 10

FIGURE ............................................................................................................................................................. 10

TEST SUITE 7 HAS BEEN SELECTED ...................................................................................................................... 10

SELECT THE FIRST TEST CASE WITH ID = 41 AND SELECT THE OPEN TEST CASE BUTTON. ...................................... 10

......................................................................................................................................................................... 10

FIGURE ............................................................................................................................................................. 10

OPENING TEST CASE WITH ID=41 ....................................................................................................................... 11

IN THE PARAMETER VALUES SECTION AT THE BOTTOM, ADD A NEW ROW WITH QUANTITY 10. ......................... 11

NOTE: A VALUE OF 10 IS A LEGITIMATE VALUE FOR THE SHOPPING CART, SO THE CART WILL REFRESH TO SHOW AQUANTITY OF 10 WHEN THIS VALUE IS ENTERED. HOWEVER, SINCE THE PURPOSE OF THIS EXERCISE IS TO SHOW WHAT HAPPENS WHEN A TEST ITERATION FAILS, WE WILL PRETEND THAT THIS VALUE CAUSES THE TEST TO FAIL IN ORDER TO DEMONSTRATE A TEST CASE FAILURE. THE ASSERTION THAT WE ADDED EXPECTS THAT THE QUANTITY WILL REMAIN AT 1. ........................................................................................................................... 11

......................................................................................................................................................................... 11

FIGURE ............................................................................................................................................................. 11

PARAMETER VALUE CHANGE ............................................................................................................................. 11

SAVE THE CHANGES TO THE TEST CASE AND RETURN TO VISUAL STUDIO. .......................................................... 11

RUN THE TESTS AGAIN AND NOTE THAT THE TEST FAILS ON THE FOURTH ITERATION. ........................................ 11

SELECT THE FAILED TEST WITHIN THE TEST EXPLORER WINDOW TO SEE THE DETAILS. AT THE BOTTOM OF THE TEST EXPLORER WINDOW WE ARE NOTIFIED THAT 3 OUT OF 4 TESTS PASSED, WITH THE FOURTH DATA ROW FAILING. ............................................................................................................................................................ 11

......................................................................................................................................................................... 11

FIGURE ............................................................................................................................................................. 11

TEST RESULTS SHOWING FAILED ASSERTION ...................................................................................................... 11

Overview

Coded UI tests provide a way to create fully automated tests to validate the functionality and behavior ofyour application’s user interface. In this lab, you will gain a basic understanding of coded UI tests by creating a new test and adding validation logic to it.

Prerequisites

In order to complete this lab you will need the Visual Studio 2015 virtual machine provided by Microsoft.For more information on acquiring and using this virtual machine, please see this blog post.

Exercises

This hands-on lab includes the following exercises:

Introduction to Code Generation from Action Recordings

Introduction to Code Generation using Coded UI Test Builder

Data Driven Demonstration for Coded UI Test

Estimated time to complete this lab: 60 minutes.

Exercise 1: Code Generation from ActionRecordings

In this exercise, you will be introduced to the code generation features in Visual Studio that allow testers to quickly and easily create coded UI tests directly from existing action recordings. Action recordings contain the steps taken during manual testing of an application. To learn more about manual testing and action recordings, please see the “Authoring and Running Manual Tests using Microsoft Test Manager 2015” lab.

Log in as Julia (VSALM\Julia). All user passwords are P2ssw0rd.

Launch Visual Studio 2015 from the taskbar and open Team Explorer.

Select the Connect to Team Projects button.

Figure 1Connecting to a different team project

In Team Explorer – Connect, double-click on the Tailspin Toys project.

Figure 2Loading the Tailspin Toys project

Start a new testing project (File | New | Project…).

In the New Project window, select the Coded UI Test Project template from Visual C# | Test, then select the OK button to create the test project.

Figure 3Creating a Coded UI Test Project

There are two ways to generate code for this new coded UI test. The first and default option is to use the Coded UI Test Builder, which allows you to generate test code by manually walking through a test scenario. The second option is to use an existing action recording. Select the second option to use an existing action recording and select the OK button to continue.

Figure 4Using an existing action recording for test generation

Select the IDs radio button and enter 41. For the purposes of this lab, assume that we already know that this is the ID of a test case with an action recording.

Select the Find button to execute the work item query.

Figure 5Finding a test case with action recording

Select the OK button to generate a coded UI test from the action recording.

Figure 6Selecting a test case with action recording

Navigate to the CodedUITestMethod1 method in the generated CodedUITest1.cs file. Each line represents a step from the action recording used during test generation.

Place the cursor somewhere on the line of the first method call, Openhttpwwwtailspintoyscom(), and then press Alt + F12 to peek at its definition. This will load the UIMap class from the UIMap.Designer.cs file, which contains the generated test logic. This generated method launches Internet Explorer and navigates to a specified URL.

Figure 7Using Peek Definition

In the peek definition window, scroll down to the ClickFourthCoffeeFlyer method. This generated method tests clicking on a “Fourth Coffee Flyer” hyperlink that is in the Tailspin Toys web application.

Figure 8 Generated test method example

The ClickFourthCoffeeFlyer test method does not specify the hyperlink parameters directly, but instead refers to the “UIBlankPageWindowsInteWindow.UIHomeTailspinToysDocument1.UIFourthCoffeeFlyerHyperlink” property. Select the “UIFourthCoffeeFlyerHyperlink” property (may need to scroll to the right) and then press Alt + F12 to view its definition.

Figure 9Definition of hyperlink property

Note: The HtmlHyperlink instance that is created for the UIFourthCoffeeFlyerHyperlink property has a number of search and filter properties applied that aid the test framework in locating the correct HTML hyperlink. In the event that the web application changes some of the link properties, such as the inner text, the test harness may still be able to find the hyperlink using the remaining search properties.

Press Escape to close the code peek window and return to the CodedUITest1.cs file.

Right-click somewhere within the CodedUITest1.cs source file and select Run Tests. Do not touch the mouse or keyboard during the tests.

Figure 10Location of Run Tests command

As the tests run, an instance of Internet Explorer will be opened and actions automatically taken as they are defined in the coded UI test. The test runs more than once because the original manualtest that this coded UI test was generated from had multiple rows of test parameters.

Figure 11 Example of coded UI test running

Verify that the test passed by viewing the Test Explorer window. In this case, however, we are not performing any validation after any of the steps.

Figure 12Test Explorer window showing passed test

Exercise 2: Code Generation using Coded UI Test Builder

In this exercise, you will learn how to use the Coded UI Test Builder to generate test code for the Tailspin Toys Web application and modify the generated code in order to enable data driven testing.

Open Internet Explorer and select the Tailspin Toys button from the favorites bar.

Select the Model Airplanes link.

Select the Fourth Coffee Flyer link.

Select the Add To Cart link to load the shopping cart.

Return to Visual Studio, locate the CodedUITestMethod1 method in the CodedUITest1.cs file, and add a blank line after the call to the “this.UIMap.Clickonwhitespaceinwebsite” method.

Figure 13Adding blank line to test source

Right-click at the location of the blank line and select Generate Code for Coded UI Test | Use CodedUI Test Builder… from the context menu. This will load the Coded UI Test Builder window (which is always displayed over other windows) and the Internet Explorer instance that we previously left open.

Figure 14Starting the Coded UI Test Builder

Note: The Coded UI Test Builder is used to record actions and assertions within the user interface which are then converted to code.

Now we will add an assertion to verify that the Quantity textbox is equal to a value of 1. Drag and drop the target icon from the Coded UI Test Builder tool window onto the Quantity textbox in Internet Explorer. This action will load the Coded UI Test Builder window.

Figure 15Selecting an element to use for assertion

In the Coded UI Test Builder window, select the Text property and click on Add Assertion. This will load a dialog to finalize the assertion options to use.

Figure 16Coded UI Test Builder window

Use the AreEqual comparator and a comparison value of ‘1’. Select the OK button to continue.

Figure 17Selecting the comparator type and value

Verify that a checkbox has been added to the Text property row.

Figure 18Text property showing assertion checkbox

Select the Generate Code button from the Coded UI Test Builder tool window.

Figure 19Generate Code button location

In the Generate Code window, use QuantityEqualsOne for the Method Name and select the Add and Generate button to generate the validation code.

Figure 20Generating assertion code

Remove the “Fourth Coffee Flyer” item from the shopping cart.

Close the Coded UI Test Builder.

Close the Internet Explorer window and return to Visual Studio.

Note that the assertion code generation has added the new validation step.

Figure 21Newly created assertion step for coded UI test

Right-click and select Run Tests to run the tests with the new validation steps. The test should complete successfully.

Exercise 3: Data Driven Demonstration for Coded UI Test

In this exercise, you will add another set of test parameter values to the test case in order to demonstrate that these test parameters are hooked up to the coded UI test and that the validation that we recently added in is performing as expected.

Launch Microsoft Test Manager from the taskbar.

If not already connected, connect to the Iteration 2 test plan from the Tailspin Toys team project. You can change this selection by clicking on the Home button.

Figure 22

Connecting to Tailspin Toys team project

Figure 23Connecting to Iteration 2 test plan

Figure 24Location of currently selected team project and test plan

Select the Test tab and select test suite 7 named “As a customer I should be able to remove items from my shopping cart”.

Figure 25Test suite 7 has been selected

Select the first test case with ID = 41 and select the Open Test Case button.

Figure 26Opening test case with ID=41

In the Parameter Values section at the bottom, add a new row with quantity 10.

Note: A value of 10 is a legitimate value for the shopping cart, so the cart will refresh to show a quantity of 10 when this value is entered. However, since the purpose of this exerciseis to show what happens when a test iteration fails, we will pretend that this value causes the test to fail in order to demonstrate a test case failure. The assertion that we added expects that the quantity will remain at 1.

Figure 27Parameter Value change

Save the changes to the test case and return to Visual Studio.

Run the tests again and note that the test fails on the fourth iteration.

Select the Failed test within the Test Explorer window to see the details. At the bottom of the Test Explorer window we are notified that 3 out of 4 tests passed, with the fourth data row failing.

Figure 28Test results showing failed assertion

Hands-On LabIntroduction to IntelliTest with Visual Studio Enterprise 2015

Lab version: 14.0.23107.0

Last updated: 9/23/2015

TABLE OF CONTENT

INTRODUCTION TO INTELLITEST WITH VISUAL STUDIO ENTERPRISE 2015 ............................................................. 1

OVERVIEW .......................................................................................................................................................... 3

INTELLITEST EXPLORES YOUR .NET CODE TO GENERATE TEST DATA AND A SUITE OF UNIT TESTS. FOR EVERY STATEMENT IN THE CODE, A TEST INPUT IS GENERATED THAT WILL EXECUTE THAT STATEMENT. A CASE ANALYSIS IS PERFORMED FOR EVERY CONDITIONAL BRANCH IN THE CODE. FOR EXAMPLE, IF STATEMENTS, ASSERTIONS, AND ALL OPERATIONS THAT CAN THROW EXCEPTIONS ARE ANALYZED. THIS ANALYSIS IS USED TO GENERATE TEST DATA FOR A PARAMETERIZED UNIT TEST FOR EACH OF YOUR METHODS, CREATING UNIT TESTS WITH HIGH CODE COVERAGE. ................................................................................................................................................ 3

WHEN YOU RUN INTELLITEST, YOU CAN EASILY SEE WHICH TESTS ARE FAILING AND ADD ANY NECESSARY CODE TO FIX THEM. YOU CAN SELECT WHICH OF THE GENERATED TESTS TO SAVE INTO A TEST PROJECT TO PROVIDE A REGRESSION SUITE. AS YOU CHANGE YOUR CODE, RERUN INTELLITEST TO KEEP THE GENERATED TESTS IN SYNC WITH YOUR CODE CHANGES. ............................................................................................................................... 3

PREREQUISITES ................................................................................................................................................... 4

IN ORDER TO COMPLETE THIS LAB YOU WILL NEED THE VISUAL STUDIO 2015 VIRTUAL MACHINE PROVIDED BY MICROSOFT. FOR MORE INFORMATION ON ACQUIRING AND USING THIS VIRTUAL MACHINE, PLEASE SEE THIS BLOG POST. ......................................................................................................................................................... 4

EXERCISES ........................................................................................................................................................... 4

THIS HANDS-ON LAB INCLUDES THE FOLLOWING EXERCISES: ............................................................................... 4

ESTIMATED TIME TO COMPLETE THIS LAB: 30 MINUTES. ...................................................................................... 4

EXERCISE 1: INTRODUCTION TO INTELLITEST ........................................................................................................ 5

IN PRACTICAL TERMS, WHITE BOX UNIT TEST DEVELOPMENT INCLUDES AN ITERATIVE WORKFLOW INFORMED BYCODE COVERAGE - WRITE A UNIT TEST, SEE WHAT PARTS OF THE CODE ARE NOT COVERED BY THE TEST, WRITE MORE TESTS TO COVER THOSE PARTS, REPEAT UNTIL ALL OF THE CODE IS COVERED. THIS WORKFLOW IS SIMILARTO WHAT WE WOULD USE WHILE WORKING WITH INTELLITEST, AS YOU WILL SEE IN THIS EXERCISE. ................... 5

TASK 1: RUNNING INTELLITEST ............................................................................................................................. 5

........................................................................................................................................................................... 5

........................................................................................................................................................................... 5

........................................................................................................................................................................... 5

TASK 2: UNDERSTANDING INTELLITEST WARNINGS .............................................................................................. 5

........................................................................................................................................................................... 5

........................................................................................................................................................................... 5

........................................................................................................................................................................... 5

........................................................................................................................................................................... 6

........................................................................................................................................................................... 6

........................................................................................................................................................................... 6

........................................................................................................................................................................... 6

........................................................................................................................................................................... 6

........................................................................................................................................................................... 6

TASK 3: PROVIDING MOCK IMPLEMENTATIONS .................................................................................................... 6

........................................................................................................................................................................... 6

........................................................................................................................................................................... 7

........................................................................................................................................................................... 7

........................................................................................................................................................................... 7

........................................................................................................................................................................... 7

........................................................................................................................................................................... 7

........................................................................................................................................................................... 7

TASK 4: FOCUSING ON ‘JUST MY CODE’ ................................................................................................................ 8

........................................................................................................................................................................... 8

........................................................................................................................................................................... 8

........................................................................................................................................................................... 8

TASK 5: MODIFYING THE PARAMETERIZED UNIT TEST TO INCREASE CODE COVERAGE .......................................... 8

IN ORDER TO EXERCISE THE CODE-UNDER-TEST FURTHER, WE NEED TO MODIFY THE PARAMETERIZED UNIT TEST METHOD IN ORDER TO RETURN DATA FROM CALLS TO THE GETDATA METHOD. IN THE PUT, THE ‘TARGET’ IS THE OBJECT THAT CONTAINS DATA TO BE RETURNED BY CALLS TO GETDATA<T>. MORE SPECIFICALLY, T IS EITHER A LOOKUPGAMEPLAYERSDTO ARRAY OR A GETPLAYERSTATISTICSDTO ARRAY. OUR TASK NOW IS TO FILL UP FAKEQUERYDATASTORE WITH CONCRETE INSTANCES OF THESE TYPES. ................................................................ 8

........................................................................................................................................................................... 9

......................................................................................................................................................................... 10

......................................................................................................................................................................... 10

......................................................................................................................................................................... 10

......................................................................................................................................................................... 10

......................................................................................................................................................................... 10

Overview

IntelliTest explores your .NET code to generate test data and a suite of unit tests. For every statement in the code, a test input is generated that will execute that statement. A case analysis is performed for every conditional branch in the code. For example, if statements, assertions, and all operations that can throw exceptions are analyzed. This analysis is used to generate test data for a parameterized unit test for each of your methods, creating unit tests with high code coverage.

When you run IntelliTest, you can easily see which tests are failing and add any necessary code to fix them. You can select which of the generated tests to save into a test project to provide a regression suite. As you change your code, rerun IntelliTest to keep the generated tests in sync with your code changes.

Prerequisites

In order to complete this lab you will need the Visual Studio 2015 virtual machine provided by Microsoft.For more information on acquiring and using this virtual machine, please see this blog post.

Exercises

This hands-on lab includes the following exercises:

Introduction to IntelliTest

Estimated time to complete this lab: 30 minutes.

Exercise 1: Introduction to IntelliTest

In practical terms, white box unit test development includes an iterative workflow informed by code coverage - write a unit test, see what parts of the code are not covered by the test, write more tests to cover those parts, repeat until all of the code is covered. This workflow is similar to what we would use while working with IntelliTest, as you will see in this exercise.

Task 1: Running IntelliTest

Log in as Adam Barr (VSALM\Adam). All user passwords are P2ssw0rd.

Open an Explorer window and then open PokerLeagueManager.sln in Visual Studio from c:\samples\pokerleaguemanager\src. This application tracks stats for a weekly poker league. It has table that has the stats on each player (Games Played, Total Winnings, Total Profit, and so on).

In Solution Explorer, search for GetPlayerStatisticsHandler.cs in the search box and then open the associated file in the code editor.

Scroll down and locate the Handle method that takes a GameDeletedEvent parameter. When a Game is deleted, this method is responsible for updating the stats of the affected players. As with most real-world code, this code interacts with other objects and layers. Our goal with this demonstration is to enable IntelliTest reach 100% code coverage on the Handle method.

Right-click somewhere within this Handle method and then select Run IntelliTest.

Task 2: Understanding IntelliTest Warnings

After IntelliTest runs, only two tests are generated and there is low coverage of the code (6/42 blocks). In addition, there are 5 warnings reported.

Click on the Warnings button.

The first warning is a Runtime Warning, and it indicates that IntelliTest has discovered, and will use, “PokerLeagueManager.Queries.Core.QueryDataStore” as IQueryDataStore. Browsing through the code, we can discover that IQueryDataStore is the type returned by the getter from the QueryDataStore property on the base class BaseHandler. In order to unit test this method, a concrete instantiation of this type is required. However, this may not be the type that you want to use for testing.

Select the Object Creation warning category. IntelliTest has also discovered publicly accessible APIs though which to instantiate QueryDataStore (in this case that happens to be the public constructor). The APIs need to be publicly accessible because IntelliTest needs to actually call them to instantiate the type.

Select the first Object Creation warning. This warning alerts us about the APIs that it discovered. If we prefer, those calls could be persisted as a Factory method by clicking the Fix button, althoughwe will not do so now.

Select the second Object Creation warning. This warning alerts us that IntelliTest was not automatically able to instantiate the object and indicates that we have some more work to do inorder to get this working.

Next, select the Uninstrumented Method warning category followed by the only warning from the list.

It turns out that the QueryDataStore constructor ends up calling into some, as of yet, uninstrumented code, which you can see if you take a quick look through the provided stack trace.

This information is important to note, because IntelliTest works by instrumenting code and monitoring execution. However, it does not instrument the entire universe of code for two reasons, 1) it cannot know a priori what comprises that universe of code and 2) that would makethe system very slow. Therefore, that is why we see this “uninstrumented method” warning.

Select the Boundary warning category followed by the only warning from the list.

When the number of branches in the code path that IntelliTest is exploring is large, it can trip an internal boundary that has been configured for fast interactive performance. Hence, it raises a warning and stops the exploration.

Task 3: Providing Mock Implementations

To proceed further, we need to answer that first question: is that the type you want to use? To unit test the method, we need to provide a mock implementation of IQueryDataStore. Browsing through the solution, we can discover a FakeQueryDataStore. Let’s tell IntelliTest to use that (instead of the QueryDataStore that it discovered).

To start assisting IntelliTest like this, we first need to setup the Parameterized Unit Test (PUT). Click on the Warnings button once again to toggle it off.

Select the two tests and then click the Save button.

IntelliTest will generate a new project named PokerLeagueManager.Queries.Core.Tests, with the generated Parameterized Unit Test found in GetPlayerStatisticsHandlerTest.cs.

In Solution Explorer, expand GetPlayerStatisticsHandlerTest.cs and delete the generated unit test file ending with .g.cs.

In the test project, right-click on the References node and select Add Reference.

In the Reference Manager window, select the Projects node and then add a reference to PokerLeagueManager.Common.Tests. Click OK.

Open GetPlayerStatisticsHandlerTest.cs in the code editor.

Add the following Using statements to the top of the file:

using Microsoft.Pex.Framework.Using;

using PokerLeagueManager.Common.Tests;

To specify that IntelliTest should use FakeQueryDataStore, add the following attribute to the Handle method:

[PexUseType(typeof(FakeQueryDataStore))]

In the IntelliTest Exploration Results window, click the Run button.

Once the IntelliTest run completes, note that the bounds exceeded warning is gone.

Task 4: Focusing on ‘Just my Code’

Click the Warnings button.

The Object Creation warning now shows that IntelliTest has discovered how to instantiate FakeQueryDataStore. The Details alert us about the APIs it can use to instantiate it, and if we prefer, we can persist this as a factory method.

Select the Uninstrumented Method category and note the warnings shown. These indicate that IntelliTest has ended up calling into uninstrumented code once again. If you inspect the stack trace associated with these warnings, it is possible to see that the calls where execution transitions into uninstrumented code is at the constructor and GetData<T> methods.

Since we are not testing the mock, let us suppress these uninstrumented method warnings. Select one of the warnings and then click the Supress button.

Open PexAssemblyInfo.cs from the test project and note the added assembly attribute, PexSupressUninstrumentedMethodFromType.

Run IntelliTest once again and verify that only the object creation warning remains.

Task 5: Modifying the Parameterized Unit Test to Increase Code Coverage

In order to exercise the code-under-test further, we need to modify the parameterized unit test method in order to return data from calls to the GetData method. In the PUT, the ‘target’ is the object that contains data to be returned by calls to GetData<T>. More specifically, T is either a LookupGamePlayersDto array or a GetPlayerStatisticsDto array. Our task now is to fill up FakeQueryDataStore with concrete instances of these types.

Since IntelliTest can synthesize data values, we will add this to the PUT’s signature. We want 2 instances of LookupGamePlayersDto, and 1 instance of GetPlayerStatisticsDto. Further, we want to associate the statistics for the first player.

Add a reference to PokerLeageManager.Common.DTO in the test project (right-click on References, Add Reference…)

Add the following Using statements to the top of GetPlayerStatisticsHandlerTest.cs:

using PokerLeagueManager.Common.DTO;

using PokerLeagueManager.Common.DTO.DataTransferObjects.Lookups;

Modify the signature of the PUT method by adding the following parameters:

LookupGamePlayersDto[] lookupGamePlayers,

GetPlayerStatisticsDto[] getPlayerStatistics

The signature of the PUT method should now look like the following screenshot.

We should also add in some additional hints to IntelliTest about the assumptions we would like to make about the input parameters. Insert the following code snippet to the beginning of the PUT:

// assume

PexAssume.IsNotNull(lookupGamePlayers);

PexAssume.IsTrue(lookupGamePlayers.Length == 2);

PexAssume.IsNotNull(lookupGamePlayers[0]);

PexAssume.IsNotNull(lookupGamePlayers[1]);

PexAssume.IsNotNull(getPlayerStatistics);

PexAssume.IsTrue(getPlayerStatistics.Length == 1);

PexAssume.IsNotNull(getPlayerStatistics[0]);

PexAssume.IsTrue(lookupGamePlayers[0].PlayerName == getPlayerStatistics[0].PlayerName);

Next, we will prime the target with these steps (add this code just after the previous code that you inserted):

// arrange

foreach (var lookupGamePlayer in lookupGamePlayers)

{

target.QueryDataStore.Insert<LookupGamePlayersDto>(lookupGamePlayer);

}

target.QueryDataStore.Insert<GetPlayerStatisticsDto>(getPlayerStatistics[0]);

The next step is to exercise the code under test, but the code to do that is already in place:

target.Handle(e);

At the very end of the PUT, we simply query for the statistics and then assert the observed value of its fields. Add a Using statement to the top of the file to System.Linq and then add the following snippet to the very end of the PUT:

// assert

var playerStats = target.QueryDataStore.GetData<GetPlayerStatisticsDto>().Single();

PexObserve.ValueAtEndOfTest("playerStats", playerStats);

Delete the .g.cs file once again, since we changed the signature of the PUT.

Run IntelliTest and note that we now have full code coverage (52/52 blocks), with 3 passing tests, 4 failing tests, and a number of warnings.

Take a quick look at the warnings, and note that none is related to the code-under-test. Therefore, go ahead and Suppress all of the warnings.

Run IntelliTest and verify that the warnings are gone.

Two of the tests fail because they uncover a NullReferenceException when the ‘e’ parameter is null.

One of the tests uncovers a potential DivideByZeroException. This will happen if stats.GamesPlayed has a value of ‘1’. In this case, the statement stats.GamesPlayed-- will make it ‘0’, and subsequently stats.Profit / stats.GamesPlayed will raise the exception.

To see where in code the DivideByZeroException was thrown, select the test in the IntelliTest Exploration Results window, expand the Stack Trace on the right-hand side, and then double-click on the first line shown.

Another test uncovered an OverflowException.

Select the failed test in the IntelliTest Exploration Results window and take a moment to scroll through the Details section. This shows the specific test and parameters that were used against the code-under-test in order to generate the exception.

This shows that IntelliTest has generated tests that uncovered previously unknown errors in the code. If we were to add additional assertions about the expected behavior of the code-under-test, then it would generate tests for validating that as well.

Manual and exploratory testing scenarios andcapabilities

Visual Studio 2015 | Team Services | TFS 2015

Drive quality and collaboration throughout the development process.

Scenarios and capabilities:

Manual testing

Exploratory testing

User acceptance testing

Reports and dashboards

Browser-based testing

Rich testing environment

Manual testing

Visual Studio Test Professional provides access to the Test hub in Team Foundation Server and Team

Services. Use this to coordinate all test management activities including test planning, authoring,

execution, and tracking from a central location. The Test hub gives product owners and business

analysts critical insight into progress against defined acceptance criteria and quality metrics.

Team Services (/docs/overview) > (/docs/test/overview)

Manual and exploratory testing of your applications - Visual Studio Team ... https://www.visualstudio.com/en-us/docs/test/manual-exploratory-testing...

1 of 7 02.09.2016 13:29

Get started with test case management (getting-started/create-a-test-plan)

Web-based test case management with TFS (https://msdn.microsoft.com/magazine

/dn890369.aspx)

Videos: Test case management (https://channel9.msdn.com/series/test-tools-in-visual-studio)

Exploratory testing

Visual Studio Test Professional enables you to perform ad-hoc and exploratory tests without requiring

predefined test cases or test steps. The exploratory testing tool and browser extensions include action

recording, commenting with annotations, rich data capture, and bug-reporting capabilities. Use it to

quickly and easily create test cases from exploratory test recordings and add them to test plans and

suites for future manual or automated test passes.

Manual and exploratory testing of your applications - Visual Studio Team ... https://www.visualstudio.com/en-us/docs/test/manual-exploratory-testing...

2 of 7 02.09.2016 13:29

Get started with exploratory testing (getting-started/perform-exploratory-tests)

Learn more and download about exploratory testing extension now

(https://marketplace.visualstudio.com/items?itemname=ms.vss-exploratorytesting-web)

Videos: Exploratory testing (http://aka.ms/vstest)

User acceptance testing

With the today's faster development pace, tools that enable test teams to more easily verify that value

customers have requested has been delivered, and is of high quality, have become increasingly

important. This type of testing is often referred to as user acceptance testing (UAT). Visual Studio Team

Services and Team Foundation Server have been enhanced to make this easy.

Manual and exploratory testing of your applications - Visual Studio Team ... https://www.visualstudio.com/en-us/docs/test/manual-exploratory-testing...

3 of 7 02.09.2016 13:29

Get started with user acceptance testing (getting-started/user-acceptance-testing)

Acceptance testing (https://msdn.microsoft.com/library/ff649646.aspx)

Video: User acceptance testing (http://aka.ms/vstest)

Create actionable test results with reports and dashboards

Get end-to-end traceability across all artifacts by running tests and logging defects from your browser,

or use the Microsoft Test Manager (MTM) client. Track and assess quality throughout your testing

lifecycle, and verify requirements with user acceptance testing. All your quality engineers and user

acceptance testing groups can use the same test tools, and you can assign pre-defined test cases to

your user acceptance testers. You pay for the tools only when you need them.

Manual and exploratory testing of your applications - Visual Studio Team ... https://www.visualstudio.com/en-us/docs/test/manual-exploratory-testing...

4 of 7 02.09.2016 13:29

Get started with reports and dashboards (getting-started/track-test-status)

Web-based test case management with TFS (https://msdn.microsoft.com/magazine

/dn890369.aspx)

Video: Create actionable test results (http://aka.ms/vstest)

Browser-based testing

Execute manual and acceptance tests with only a browser. Run manual tests and record test results for

each test step using a toolset optimized around the needs of testers. The web-based test runner

enables pass-fail results, tracking of test steps, rich commenting, and bug reporting capabilities. You

can also use the test runner for your user acceptance testing needs to run and report test case

execution by multiple acceptance testers.

Manual and exploratory testing of your applications - Visual Studio Team ... https://www.visualstudio.com/en-us/docs/test/manual-exploratory-testing...

5 of 7 02.09.2016 13:29

Get started with browser-based testing (getting-started/run-manual-tests)

Create manual tests using the web portal (https://msdn.microsoft.com/library/dd286729.aspx)

Video: Browser based testing video (http://aka.ms/vstest)

Rich experience for running, recording, and repeating manualtests

Microsoft Test Manager (MTM) offers a fully configurable test runner that captures detailed records of

steps performed, behaviors observed, and the status of each test step. Pause testing to report a bug,

even if it’s not directly related to the current test. High-fidelity defect reports go directly to the

development team, including all the technical data needed to reproduce and determine the root

cause. You can record tests for later playback to speed test sessions and enable automation.

Manual and exploratory testing of your applications - Visual Studio Team ... https://www.visualstudio.com/en-us/docs/test/manual-exploratory-testing...

6 of 7 02.09.2016 13:29

Download Microsoft Test Manager (https://marketplace.visualstudio.com

/items?itemname=ms.vss-testmanager-web)

Plan manual tests with Microsoft Test Manager (https://msdn.microsoft.com/library

/dd286659.aspx)

Video: Manual testing (http://aka.ms/vstest)

Help and support

Submit bugs through Connect (https://connect.microsoft.com/visualstudio), make suggestions on

Uservoice (http://visualstudio.uservoice.com/forums/121579-visual-studio), and send quick thoughts

using the Send-a-Smile link in the Visual Studio, Team Services, or TFS title bar. We look forward to

your feedback.

© 2016 Microsoft

Manual and exploratory testing of your applications - Visual Studio Team ... https://www.visualstudio.com/en-us/docs/test/manual-exploratory-testing...

7 of 7 02.09.2016 13:29

Parallel and Context Sensitive Test Execution withVisual studio 2015 Update 1

0 2 37

February 8, 2016 by Charles Sterling // 1 Comments

﴾Editors Note: One of the most popular series of blog posts on the ALM Blog was Terje’s posts on Unit Testing. Sowhen he asked to republish his Norwegian post on the ALM blog the answer was of course an enthusiastic YES!. Alittle off topic this post also highlights we need a post on executing parallel test runs with the new “Run Functional TestTask” as that paradigm is different using separate machines and “test containers” ﴾think assemblies﴿﴿

Visual studio 2015 Update 1 : Parallel andContext Sensitive Test ExecutionVisual Studio 2015 Update 1 contains a bunch of improvements and bug fixes. In this post we will have a look at twoof the performance improvements, that may give a significant impact on your overall development performance.

Parallel test executionIn Visual Studio 2015 Update 1 the test engine can execute test assemblies in parallel. This can improve your testperformance significantly. This is off by default, so you need to enable it.

The system runs the test assemblies in parallel on as many cores as you specify, and of course, up to the maximumnumber of cores on your machine. If you only have a single test project you will of course have no benefit of this. Thisis a solution for the slightly larger projects.

It is independent of the type of test framework you use, and therefore works with both MSTest, NUnit 2, NUnit 3 andXUnit.

This mechanism is completely independent on whether the underlying framework supports parallelism or not.

To turn on the feature, you must use a runsettings file, which you must enable in Visual Studio. The easy way to get acorrect runsettings file in, is to use one of these templates, note that you need at least version 3.1 of these. When youhave this item extension installed, you have 3 templates, and if you’re just after the parallelism, use the one namedParallel.

Do as follow:

Select Solution, Right click and choose Add/New item

You will now see a list like the one below:

Subscribe

Download Visual Studio

Download TFS 2015

Visual Studio Team Services

Search this blogSearch all blogs

Share This Post

Tags

Administration Agile ALMAnnouncementAutomated Testing AzureBuild Process cloud load testing

Coded UI Test Coded UITesting Code Visualization

Customizing Debugging

DevOps

DiagnosticsExtensibility Feedback GitGuidance insightsIntelliTrace Load TestingManaging Technical DebtManual Testing Process Templates

Rangers ReleaseManagement SonarQube

Team Foundation Build

★★★★★★★★★★★★★★★

Search MSDN with Bing

Server & Tools Blogs > Developer Tools Blogs > Microsoft Application Lifecycle Management

Microsoft Application LifecycleManagementLearn to use Visual Studio, Visual Studio Team Services, Application Insights and Team Foundation Server to decreaserework, increase transparency into your application and increase the rate at which you can ship high quality softwarethroughout the application lifecycle

ExecutiveBloggers

Visual Studio ApplicationLifecycleManagement

Languages .NET PlatformDevelopment

Sign in

Page 1 of 6   Assembled by RunPDF.com

Select Parallel and you will see an instance of it under Solution Items in your solution.

That’s all it takes to enable parallel execution !

Just remember to activate it under the Visual Studio Test menu, do like this:

﴾1﴿ Go to Select Test Settings file from the top menu Test/Test settings.

﴾2﴿ Then select the wanted Runsettings file, in this case I named it Parallel2.runsettings.

﴾3﴿ Verify on the same menu that it is selected and checked.

The parameters:Looking at the content of the runsettings file, we see this:

There is only one item here we can change, the MaxCpuCount value. It controls how many parallel processes we canrun. The value 0 means “run as much as you can, limited by availability of cores on your machine”. The value 1 meansit will run sequentially, that is the same as the default with only one process activated. Any other number will limit atthat number, or the number of available cores on your machine.

Why don’t we include Code Coverage?You might have noticed that in the description for Parallel it says code coverage is not included. This means that whenyou activate this runsetting, you will not active code coverage during the same run. The other two templates do exactlythat, it doesn’t really give you any benefits.

Team Foundation SDK TeamFoundationServer Testing Test

Management TFS TFServiceUnit Testing Version Control

Visual Studio Visual Studio

2010 Visual Studio 2012 Visual

studio 2013 VisualStudio ALM VisualStudio Online Visual

Studio Team Services WorkItems

Archives

September 2016 ﴾1﴿August 2016 ﴾21﴿July 2016 ﴾15﴿June 2016 ﴾20﴿May 2016 ﴾17﴿April 2016 ﴾18﴿March 2016 ﴾24﴿February 2016 ﴾13﴿January 2016 ﴾22﴿All of 2016 ﴾151﴿All of 2015 ﴾186﴿All of 2014 ﴾173﴿All of 2013 ﴾154﴿All of 2012 ﴾207﴿All of 2011 ﴾81﴿All of 2010 ﴾28﴿All of 2009 ﴾1﴿All of 2008 ﴾5﴿All of 2007 ﴾30﴿All of 2006 ﴾76﴿All of 2005 ﴾22﴿

Subscribe Blog via Email

Subscribe to this blog and receivenotifications of new posts by email.

Email Address

Subscribe! Unsubscribe

Page 2 of 6   Assembled by RunPDF.com

If you activate the Code Coverage the performance will be reduced, because code coverage also needs to run, and itruns sequentially after the test runs, and will therefore increase the overall time significantly. And, it will not give youanything you really need.

The fact that you don’t active Code Coverage does not imply that you can’t run code coverage from Team Explorer –you can! Further, the results from this code coverage run is not used in the Test Explorer – strangely enough. Theactivation done in the two other templates just activate a background code coverage run, which results are just storedin a code coverage result file under the Test Results folder.

This file doesn’t really give you any benefits in Visual Studio, but is useful if you want these coverage data into anotherprogram , one good example is NDepend.

One benefit is that this coverage section not only enables the background run of code coverage, but also sets up thefiltering of the code coverage data, and this might be interesting for you. These filters will also take effect when you runthe Test Explorer “Analyze code coverage from selected tests”

In the linked article above there is a bunch of settings for code coverage, instead of using these, just use the templateCompleteRunSettings, which includes this, plus the other settings. The XML in the linked article is not up to date.

The templaten CompleteRunsettings has the MaxCpuCount set to 0, it will then run as parallel as possible.

The templaten CoverageNoParallel is nearly identical, with exactly the same fields and settings, except theMaxCpuCount which is set to 1, thus sequential test running.

There is also an MSDN artikkel about configuration of unit tests which are more correct,m but it doesn’t contain bthesame detailed information as the link above. In this however, all fields are explained.

Test project for parallel runsI have made a simple test project for parallel runs. It has 4 test projects, with one test each, and each test runs for 5seconds. You can download the source code from here, and then run the tests yourself.

Page 3 of 6   Assembled by RunPDF.com

﴾1﴿ Running without settings at all. It is then runs sequentially, and we see the total time is 26 seconds. Since each testuses 5 seconds we have a 6 seconds overhead.

﴾2﴿ Running with CodeCoverageNoParallel, which is identical to the old runsettings file, we will, get code coverage run inaddition, but no parallel runs. The total time increases to 31 seconds.

﴾3﴿ We then uses the complete settings, using both CodeCoverage and Parallel. The total time then decreases to 18seconds.

﴾4﴿ And finally using only Parallel without code coverage, we’re down in 12 seconds.

So from ﴾4﴿ to ﴾1﴿, we gain more than a 2:1 in performance. That is pretty good!

The machine this was tested on has more cores – 8 in all, but it also runs a bunch of other programs, so not all corescan be made available for the testing processes. You will gain some, but not equal to the number of cores – in practice. Anyway, a gain of 2:1 is pretty good and pretty significant in your daily work.

Parallel on build serverDoes this work on a build server? The answer is Yes, in a build definition you can define which runsettingfile to run, sothis will also have an effect on the build server run. But, do you really want that? That all depends on how you manybuild servers you have, how they are configured, and how many check‐ins/commits you have. That si, how loadedyour build servers are. Normally you set up a build agent to run one per core on the machine. If you have many builddefinitions and many commits/Checkins happening continuously during the day, the build agents will be pretty busy.,then there will be little benefit in setting up the tests to run in parallel in addition, as they will “steal” cores from theother builds. There can also be an overhead that eat up all your gain.

If you, on the other hand, have a builds machine that is not that much loaded, and not so many commits continuously,then by all means – why not? Grab the extra performance by running your tests in parallel.

There are also other ways to run in parallel, see this article for a different approach.

Context sensitive test executionContext sensitive test execution is new in Update 1 – so what is that? Under given circumstances, the new TestExplorer will only test the assemblies that it detects tests code that you have changed.

This means that if you work with a certain piece of code in a given module of your system, and your code is distributedover multiple modules, then for each build, only the code that you changes is being build. That is the normalincremental build which is by default. New is that also only the tests touching this code is being run, and then not thetests for code that has not been touched. It is pretty obvious that for larger systems this can mean a pretty dramatic

Page 4 of 6   Assembled by RunPDF.com

speedup in the test run performance.

There is one condition for this to work: You must activate “Run tests after build”.

You do this either in the Test Explorer ﴾1﴿ or from the top menu Test/Test Settings/Run Tests after Build ﴾2﴿.

The effect of this can be seen in the screenshots below:

﴾1﴿ Activate “Run tests after build”

﴾2﴿ Do some code changes

﴾3﴿ Just select Build, or use the keyboard shortcut

﴾4﴿ Just the touched test are being run, the other ones are shown dimmed green. They ran green the last time theywere run, but they were not run this time, because no code what changed that affected them. .

If you have larger projects this can give a significant gain in performance, and make your turn around from buildthrough test back to edit mode much faster – and that is what is important during development, right?

TestingTags

7 months agoScott Harmen

How does this work with running CodedUI tests in parallel?

From Charles SterlingHello Scott,As CodedUI Tests expect to the only keyboard input they need to be run in series.Running a CodedUI Test in parallel on a single machine would not work as the tests wouldinterfere with each other.﴾This regularly happens when developers accidently add CodedUI Tests to a Load Test with

Back totop

Page 5 of 6   Assembled by RunPDF.com

multiple users﴿.That said if you do want to scale out your Coded UI Tests there is a new build task called “RunFunctional Test” and “Deploy Visual Studio Test Agent”. If you partition your tests into separateassemblies this would enable you to run your CodedUI Test in parallel…using computers.ThanksChuck

Comments are closed.

© 2016 Microsoft Corporation. Terms of Use Trademarks Privacy & Cookies

Page 6 of 6   Assembled by RunPDF.com

Unit testing native code with Test Explorer

In Visual Studio, you can create unit tests for unmanaged code written in C++. Unmanaged code is sometimes referred to as

native code.

The following procedure contains the essential information that will get you started. The later sections provide a

walkthrough that describes the steps in more detail.

To write unit tests for an unmanaged code DLL

Use the Native Test Project template to create a separate Visual Studio project for your tests.

The project contains some sample test code.

1.

Make the DLL accessible to the test project:

#include a .h file that contains declarations of the DLL’s externally-accessible functions.

The .h file should contain function declarations marked with _declspec(dllimport). Alternatively, you can

export the methods using a DEF file. For more information, see Importing and Exporting.

Your unit tests can access only functions that are exported from the DLL under test.

Add the DLL project to the References of the test project:

In the Properties of the test project, expand Common Properties, Framework and References, and choose

Add Reference.

2.

In the test project, create test classes and test methods by using the TEST macros and Assert class in the following

way:

Assert contains several static functions that you can use to verify the result of a test.

3.

#include "stdafx.h"

#include <CppUnitTest.h>

#include "..\MyProjectUnderTest\MyCodeUnderTest.h"

using namespace Microsoft::VisualStudio::CppUnitTestFramework;

TEST_CLASS(TestClassName)

{

public:

TEST_METHOD(TestMethodName)

{

// Run a function under test here.

Assert::AreEqual(expectedValue, actualValue, L"message", LINE_INFO());

}

}

C++

Unit testing native code with Test Explorer https://msdn.microsoft.com/en-us/library/hh270864(d=printer).aspx

1 of 12 02.09.2016 13:47

The LINE_INFO() parameter is optional. In cases where there is no PDB file, it allows the test runner to identify

the location of a failure.

You can also write test setup and cleanup methods. For more information, open the definition of the

TEST_METHOD macro, and read the comments in CppUnitTest.h

You cannot nest test classes.

Use Test Explorer to run the tests:

On the View menu, choose Other Windows, Test Explorer.a.

Build the Visual Studio solution.b.

In Test Explorer, choose Run All.c.

To investigate any test in more detail in Test Explorer:

Select the test name to see more details such as a failure message and stack trace.i.

Open the test name (for example by double-clicking) to go to the failure location or to the test code.ii.

On the shortcut menu for a test, choose Debug Selected Test to run the test in the debugger.iii.

d.

4.

Walkthrough: Developing an unmanaged DLL with Test ExplorerYou can adapt this walkthrough to develop your own DLL. The principal steps are as follows:

Create a Native Test Project. The tests are created in a separate project from the DLL that you are developing.1.

Create a DLL Project. This walkthrough creates a new DLL, but the procedure for testing an existing DLL is similar.2.

Make the DLL functions visible to the tests.3.

Iteratively augment the tests. We recommend a "red-green-refactor" cycle, in which development of the code is led

by the tests.

4.

Debug failing tests. You can run tests in debug mode.5.

Refactor while keeping the tests unchanged. Refactoring means improving the structure of the code without

changing its external behavior. You can do it to improve the performance, extensibility, or readability of the code.

Because the intention is not to change the behavior, you do not change the tests while making a refactoring

change to the code. The tests help make sure that you do not introduce bugs while you are refactoring. You can

therefore make such changes with much more confidence than if you did not have the tests.

6.

Check coverage. Unit tests are more useful when they exercise more of your code. You can discover which parts of

your code have been used by the tests.

7.

Isolate units from external resources. Typically, a DLL is dependent on other components of the system that you are

developing, such as other DLLs, databases, or remote subsystems. It is useful to test each unit in isolation from its

dependencies. External components can make tests run slowly. During development, the other components might

not be complete.

8.

Unit testing native code with Test Explorer https://msdn.microsoft.com/en-us/library/hh270864(d=printer).aspx

2 of 12 02.09.2016 13:47

Create a native unit test project

On the File menu, choose New, Project.

In the dialog box, expand Installed, Templates, Visual C++, Test.

Choose the Native Test Project template.

In this walkthrough, the test project is named NativeRooterTest.

1.

In the new project, inspect unittest1.cpp

Notice that:

Each test is defined by using TEST_METHOD(YourTestName){...}.

You do not have to write a conventional function signature. The signature is created by the macro

TEST_METHOD. The macro generates an instance function that returns void. It also generates a static

function that returns information about the test method. This information allows the test explorer to find

the method.

2.

Unit testing native code with Test Explorer https://msdn.microsoft.com/en-us/library/hh270864(d=printer).aspx

3 of 12 02.09.2016 13:47

Test methods are grouped into classes by using TEST_CLASS(YourClassName){...}.

When the tests are run, an instance of each test class is created. The test methods are called in an unspecified

order. You can define special methods that are invoked before and after each module, class, or method. For

more information, see Organizing C++ Tests.

Verify that the tests run in Test Explorer:

Insert some test code:

Notice that the Assert class provides several static methods that you can use to verify results in test

methods.

a.

On the Test menu, choose Run , All Tests.

The test builds and runs.

Test Explorer appears.

The test appears under Passed Tests.

b.

3.

Create an Unmanaged DLL project

Create a Visual C++ project by using the Win32 Project template.

In this walkthrough, the project is named RootFinder.

1.

TEST_METHOD(TestMethod1)

{

Assert::AreEqual(1,1);

}

C++

Unit testing native code with Test Explorer https://msdn.microsoft.com/en-us/library/hh270864(d=printer).aspx

4 of 12 02.09.2016 13:47

Select DLL and Export Symbols in the Win32 Application Wizard.

The Export Symbols option generates a convenient macro that you can use to declare exported methods.

2.

Declare an exported function in the principal .h file:

The declarator __declspec(dllexport) causes the public and protected members of the class to be visible

3.

Unit testing native code with Test Explorer https://msdn.microsoft.com/en-us/library/hh270864(d=printer).aspx

5 of 12 02.09.2016 13:47

outside the DLL. For more information, see Using dllimport and dllexport in C++ Classes.

In the principal .cpp file, add a minimal body for the function:4.

Couple the test project to the DLL project

Add the DLL project to the project references of the test project:

Open the properties of the test project and choose Common Properties, Framework and References.a.

Choose Add New Reference.

In the Add Reference dialog box, select the DLL project and choose Add.

b.

1.

In the principal unit test .cpp file, include the .h file of the DLL code:2.

Add a basic test that uses the exported function:3.

// Find the square root of a number.

double CRootFinder::SquareRoot(double v)

{

return 0.0;

}

#include "..\RootFinder\RootFinder.h"

TEST_METHOD(BasicTest)

C++

C++

C++

Unit testing native code with Test Explorer https://msdn.microsoft.com/en-us/library/hh270864(d=printer).aspx

6 of 12 02.09.2016 13:47

Build the solution.

The new test appears in Test Explorer.

4.

In Test Explorer, choose Run All.5.

You have set up the test and the code projects, and verified that you can run tests that run functions in the code project.

Now you can begin to write real tests and code.

Iteratively augment the tests and make them pass

Add a new test:

Tip

1.

{

CRootFinder rooter;

Assert::AreEqual(

// Expected value:

0.0,

// Actual value:

rooter.SquareRoot(0.0),

// Tolerance:

0.01,

// Message:

L"Basic test failed",

// Line number ‐ used if there is no PDB file:

LINE_INFO());

}

TEST_METHOD(RangeTest)

{

CRootFinder rooter;

for (double v = 1e‐6; v < 1e6; v = v * 3.2)

{

double actual = rooter.SquareRoot(v*v);

Assert::AreEqual(v, actual, v/1000);

}

}

C++

Unit testing native code with Test Explorer https://msdn.microsoft.com/en-us/library/hh270864(d=printer).aspx

7 of 12 02.09.2016 13:47

We recommend that you do not change tests that have passed. Instead, add a new test, update the code so that

the test passes, and then add another test, and so on.

When your users change their requirements, disable the tests that are no longer correct. Write new tests and

make them work one at a time, in the same incremental manner.

Build the solution, and then in Test Explorer, choose Run All.

The new test fails.

Tip

Verify that each test fails immediately after you have written it. This helps you avoid the easy mistake of writing

a test that never fails.

2.

Enhance the code under test so that the new test passes:3.

Build the solution and then in Test Explorer, choose Run All.

Both tests pass.

4.

#include <math.h>

...

double CRootFinder::SquareRoot(double v)

{

double result = v;

double diff = v;

while (diff > result/1000)

{

double oldResult = result;

result = result ‐ (result*result ‐ v)/(2*result);

diff = abs (oldResult ‐ result);

}

return result;

}

C++

Unit testing native code with Test Explorer https://msdn.microsoft.com/en-us/library/hh270864(d=printer).aspx

8 of 12 02.09.2016 13:47

Tip

Develop code by adding tests one at a time. Make sure that all the tests pass after each iteration.

Debug a failing test

Add another test:1.

#include <stdexcept>

...

// Verify that negative inputs throw an exception.

TEST_METHOD(NegativeRangeTest)

{

wchar_t message[200];

CRootFinder rooter;

for (double v = ‐0.1; v > ‐3.0; v = v ‐ 0.5)

{

try

{

// Should raise an exception:

double result = rooter.SquareRoot(v);

_swprintf(message, L"No exception for input %g", v);

Assert::Fail(message, LINE_INFO());

}

catch (std::out_of_range ex)

{

continue; // Correct exception.

}

catch (...)

{

_swprintf(message, L"Incorrect exception for %g", v);

Assert::Fail(message, LINE_INFO());

}

}

}

C++

Unit testing native code with Test Explorer https://msdn.microsoft.com/en-us/library/hh270864(d=printer).aspx

9 of 12 02.09.2016 13:47

Build the solution and choose Run All.2.

Open (or double-click) the failed test.

The failed assertion is highlighted. The failure message is visible in the detail pane of Test Explorer.

3.

To see why the test fails, step through the function:

Set a breakpoint at the start of the SquareRoot function.a.

On the shortcut menu of the failed test, choose Debug Selected Tests.

When the run stops at the breakpoint, step through the code.

b.

4.

Insert code in the function that you are developing:5.

All tests now pass.6.

#include <stdexcept>

...

double CRootFinder::SquareRoot(double v)

{

// Validate parameter:

if (v < 0.0)

{

throw std::out_of_range("Can't do square roots of negatives");

}

C++

Unit testing native code with Test Explorer https://msdn.microsoft.com/en-us/library/hh270864(d=printer).aspx

10 of 12 02.09.2016 13:47

Refactor the code without changing tests

Simplify the central calculation in the SquareRoot function:1.

Build the solution and choose Run All, to make sure that you have not introduced an error.

Tip

A good set of unit tests gives confidence that you have not introduced bugs when you change the code.

Keep refactoring separate from other changes.

2.

Next steps

Isolation. Most DLLs are dependent on other subsystems such as databases and other DLLs. These other

components are often developed in parallel. To allow unit testing to be performed while the other components are

not yet available, you have to substitute mock or

Build Verification Tests. You can have tests performed on your team’s build server at set intervals. This ensures

that bugs are not introduced when the work of several team members is integrated.

Checkin tests. You can mandate that some tests are performed before each team member checks code into

source control. Typically this is a subset of the complete set of build verification tests.

You can also mandate a minimum level of code coverage.

// old code:

// result = result ‐ (result*result ‐ v)/(2*result);

// new code:

result = (result + v/result)/2.0;

Unit testing native code with Test Explorer https://msdn.microsoft.com/en-us/library/hh270864(d=printer).aspx

11 of 12 02.09.2016 13:47

See AlsoTasks

Walkthrough: Creating and Using a Dynamic Link Library (C++)

Concepts

Importing and Exporting

Other Resources

An Overview of Managed/Unmanaged Code Interoperability

Debugging Native Code

© 2016 Microsoft

Unit testing native code with Test Explorer https://msdn.microsoft.com/en-us/library/hh270864(d=printer).aspx

12 of 12 02.09.2016 13:47

Hands-On LabUnit Testing, Code Coverage and Code Clone Analysis with Visual Studio 2015

Lab version: 14.0.23107.0

Last updated: 8/12/2015

TABLE OF CONTENT

UNIT TESTING, CODE COVERAGE AND CODE CLONE ANALYSIS WITH VISUAL STUDIO 2015 .................................... 1

OVERVIEW .......................................................................................................................................................... 4

1.IN THIS LAB, YOU WILL LEARN ABOUT HOW THE UNIT TESTING ENGINE IN VISUAL STUDIO IS EXTENSIBLE AND OPENS THE DOOR FOR 3RD PARTY UNIT TESTING ADAPTERS SUCH AS NUNIT AND XUNIT.NET. IN ADDITION, YOU WILL SEE SOME OF THE IMPROVEMENTS MADE TO CODE COVERAGE SUPPORT. YOU WILL ALSO LEARN ABOUT HOW THE CODE CLONE CAPABILITY GOES HAND-IN-HAND WITH DELIVERING HIGH QUALITY SOFTWARE BY HELPING YOU IDENTIFY BLOCKS OF SEMANTICALLY SIMILAR CODE WHICH MAY BE CANDIDATES FOR COMMON BUG FIXES OR REFACTORING. .............................................................................................................................. 4

PREREQUISITES ................................................................................................................................................... 4

1.IN ORDER TO COMPLETE THIS LAB YOU WILL NEED THE VISUAL STUDIO 2015 VIRTUAL MACHINE PROVIDED BY MICROSOFT. FOR MORE INFORMATION ON ACQUIRING AND USING THIS VIRTUAL MACHINE, PLEASE SEE THIS BLOG POST. ......................................................................................................................................................... 4

ABOUT THE FABRIKAM FIBER SCENARIO .............................................................................................................. 4

1.THIS SET OF HANDS-ON-LABS USES A FICTIONAL COMPANY, FABRIKAM FIBER, AS A BACKDROP TO THE SCENARIOS YOU ARE LEARNING ABOUT. FABRIKAM FIBER PROVIDES CABLE TELEVISION AND RELATED SERVICES TO THE UNITED STATES. THEY ARE GROWING RAPIDLY AND HAVE EMBRACED WINDOWS AZURE TO SCALE THEIR CUSTOMER-FACING WEB SITE DIRECTLY TO END-USERS TO ALLOW THEM TO SELF-SERVICE TICKETS AND TRACK TECHNICIANS. THEY ALSO USE AN ON-PREMISES ASP.NET MVC APPLICATION FOR THEIR CUSTOMER SERVICE REPRESENTATIVES TO ADMINISTER CUSTOMER ORDERS. ..................................................................................... 4

EXERCISES ........................................................................................................................................................... 4

1.THIS HANDS-ON LAB INCLUDES THE FOLLOWING EXERCISES: ............................................................................ 4

2.UNIT TESTING ............................................................................................................................................................ 4 3.CODE COVERAGE ........................................................................................................................................................ 4 4.CODE CLONE ANALYSIS ................................................................................................................................................. 4

1.ESTIMATED TIME TO COMPLETE THIS LAB: 30 MINUTES. ................................................................................... 4

EXERCISE 1: UNIT TESTING ................................................................................................................................... 5

1.IN THIS EXERCISE, YOU WILL LEARN ABOUT THE UNIT TESTING STORY IN VISUAL STUDIO 2015. ........................ 5

2.LOG IN AS JULIA (VSALM\JULIA). ALL USER PASSWORDS ARE P2SSW0RD. ............................................................................... 5 3.LAUNCH VISUAL STUDIO 2015 FROM THE TASKBAR AND OPEN TEAM EXPLORER. YOU SHOULD NOW BE CONNECTED TO THE FABRIKAMFIBER TEAM PROJECT. IF YOU ARE NOT AUTOMATICALLY CONNECTED TO THE FABRIKAMFIBER PROJECT, SELECT THE CONNECT TO TEAM PROJECTS BUTTON () TO DO SO. ...................................................................................................................................................... 5 4.IN TEAM EXPLORER – HOME, DOUBLE-CLICK ON THE FIRST FABRIKAMFIBER.CALLCENTER.SLN SOLUTION. ........................................... 5

4.A. ..................................................................................................................................................................... 5

4.B.FIGURE ......................................................................................................................................................... 5

4.C.LOADING FABRIKAM FIBER SOLUTION ........................................................................................................... 5

5.PRESS CTRL + SHIFT + B TO BUILD THE SOLUTION. .............................................................................................................. 5 6.OPEN THE TEST EXPLORER WINDOW FROM TEST | WINDOWS | TEST EXPLORER. NOTE THAT DISCOVERED TESTS ARE INITIALLY SET TO THE NOT RUN STATE. ............................................................................................................................................................ 5

6.A. ..................................................................................................................................................................... 5

6.B.FIGURE ......................................................................................................................................................... 5

6.C.TEST EXPLORER WINDOW .............................................................................................................................. 5

7.SELECT THE RUN ALL LINK TO EXECUTE ALL DISCOVERED UNIT TESTS. ........................................................................................ 5

7.A. ..................................................................................................................................................................... 5

7.B.FIGURE ......................................................................................................................................................... 5

7.C.RUNNING ALL UNIT TESTS .............................................................................................................................. 5

7.E.NOTE: VISUAL STUDIO ALSO PROVIDES CONTINUOUS TEST RUNNER FUNCTIONALITY THAT CAN BE ENABLED WITH THE TEST | TEST SETTINGS | RUN TESTS AFTER BUILD OPTION. WITH THIS OPTION SELECTED, TESTS WILL ALWAYS RUN AFTER BUILDS. ............................................................................................................................... 5

8.EXPAND THE PASSED TESTS GROUP (IF NECESSARY) AND DOUBLE-CLICK ON THE TEST WITH THE NAME CREATEINSERTSCUSTOMERANDSAVES TO OPEN THE SOURCE CODE. ............................................................................................................................................. 5

8.A. ..................................................................................................................................................................... 5

8.B.FIGURE ......................................................................................................................................................... 5

8.C.NAVIGATING TO A TEST METHOD THAT USES MSTEST .................................................................................... 5

9.IN THE CUSTOMERSCONTROLLERTEST.CS FILE THAT OPENS, NOTE THAT THE TEST METHOD USES THE EXPECTED TESTMETHOD ATTRIBUTE USED BY MSTEST TO MARK UNIT TESTS. ...................................................................................................................................... 6

9.A. ..................................................................................................................................................................... 6

9.B.FIGURE ......................................................................................................................................................... 6

9.C.MSTEST UNIT TEST CODE ............................................................................................................................... 6

10.IN THE SEARCH BOX AT THE TOP OF THE TEST EXPLORER WINDOW, ENTER “INDEX” AND NOTE THE AVAILABLE SEARCH FILTERS. ............... 6

10.A. ................................................................................................................................................................... 6

10.B.FIGURE ....................................................................................................................................................... 6

10.C.SEARCHING WITHIN TEST EXPLORER ............................................................................................................ 6

11.IN THE SEARCH RESULTS, DOUBLE-CLICK ON THE ONLY TEST LISTED WITH THE NAME INDEXRETURNSNONNULLVIEW TO OPEN THE SOURCE CODE. ......................................................................................................................................................................... 6

11.A. ................................................................................................................................................................... 6

11.B.FIGURE ....................................................................................................................................................... 6

11.C.NAVIGATING TO A TEST METHOD THAT USES XUNIT ..................................................................................... 6

12.IN THE HOMECONTROLLERTEST.CS FILE THAT OPENS, YOU CAN SEE THAT THE INDEXRETURNSNONNULLVIEW TEST ACTUALLY USES THE XUNIT TESTING FRAMEWORK. BOTH THE XUNIT FRAMEWORK AND THE TEST RUNNER THAT INTEGRATES WITH TEST EXPLORER CAN BE ADDED TO A PROJECT VIA NUGET. ................................................................................................................................................ 6

12.A. ................................................................................................................................................................... 6

12.B.FIGURE ....................................................................................................................................................... 6

13.SELECT X BUTTON TO CLEAR THE CURRENT SEARCH BOX IN TEST EXPLORER. .............................................................................. 6

13.A. ................................................................................................................................................................... 6

13.B.FIGURE ....................................................................................................................................................... 6

13.C.LOCATION OF X BUTTON TO CLEAR SEARCH BOX .......................................................................................... 7

14.NOTE THAT RESULTS ARE GROUPED BY TEST OUTCOME BY DEFAULT, WITH EXECUTION TIMES LISTED. ................................................ 7

14.A. ................................................................................................................................................................... 7

14.B.FIGURE ....................................................................................................................................................... 7

14.C.TEST RUN RESULTS ....................................................................................................................................... 7

15.TEST EXECUTION PERFORMANCE HAS BEEN GIVEN SOME ADDITIONAL ATTENTION IN RECENT VERSIONS. TAKE NOTE OF THE EXECUTION TIMES AND THEN SELECT THE RUN… BUTTON FOLLOWED BY THE REPEAT LAST RUN OPTION. NOTE THE DIFFERENCE IN EXECUTION TIME. ............... 7

15.A. ................................................................................................................................................................... 7

15.B.FIGURE ....................................................................................................................................................... 7

15.C.REPEATING THE LAST TEST RUN ................................................................................................................... 7

15.D.NOTE: THE EXECUTION TIMES THAT YOU SEE WILL BE DIFFERENT FROM THOSE SHOWN IN THE SCREENSHOTS. THE FIRST TIME YOU RUN UNIT TESTS THEY WILL EXECUTE MORE SLOWLY THAN ON SUBSEQUENTRUNS AS THE TESTS AND TESTING ENGINE ARE BEING LOADED FOR THE FIRST TIME. ........................................... 7

15.F. .................................................................................................................................................................... 7

15.G.FIGURE ....................................................................................................................................................... 7

15.H.SECOND TEST RUN WITHOUT CHANGES TO TESTS IS FASTER ........................................................................ 7

16.SINGLE-CLICK THE FAILED TEST NAMED “SCHEDULEACTIONCORRECTLYUPDATESREPOSITORIES” TO VIEW A SUMMARY OF TEST RESULTS. ..... 7

16.A. ................................................................................................................................................................... 7

16.B.FIGURE ....................................................................................................................................................... 7

16.C.SELECTING THE FAILED TEST ......................................................................................................................... 7

16.E.NOTE: YOU CAN ALSO RIGHT-CLICK ON TEST RESULTS AND USE THE COPY COMMAND TO COPY DETAILS TO THE CLIPBOARD. THIS WOULD BE USEFUL WHEN SENDING AN EMAIL, FOR EXAMPLE. ......................................... 7

17.THE SUMMARY VIEW OF THE FAILED TEST RUN SHOWS THAT AN ARGUMENTNULLEXCEPTION EXCEPTION OCCURRED DURING THE TEST RUN AND EVEN PROVIDES THE STACK TRACK AT THE TIME OF THE EXCEPTION. NOTE THAT WE CAN FOLLOW LINKS TO GO DIRECTLY TO THE SOURCE CODE FOR THE TEST OR TO POINTS WITHIN THE STACK TRACE. SELECT THE SOURCE LINK TO GO TO THE SOURCE CODE FOR THE TEST METHOD. . . 7

17.A. ................................................................................................................................................................... 8

17.B.FIGURE ....................................................................................................................................................... 8

17.C.NAVIGATE TO THE SOURCE OF THE TEST METHOD ........................................................................................ 8

17.E. ................................................................................................................................................................... 8

17.F.FIGURE ........................................................................................................................................................ 8

17.G.INITIAL VIEW AFTER NAVIGATING TO SOURCE OF THE TEST METHOD ........................................................... 8

18.FIND THE LINE OF CODE THAT IS COMMENTED OUT AND UN-COMMENT IT. ASSUME THAT THIS IS THE ROOT CAUSE FOR THE FAILED TEST. . . . 8

18.A. ................................................................................................................................................................... 8

18.B.FIGURE ....................................................................................................................................................... 8

18.C.FIXING THE ROOT PROBLEM OF THE FAILED TEST ......................................................................................... 8

19.PRESS CTRL+S TO SAVE YOUR CHANGES. ......................................................................................................................... 8 20.RIGHT-CLICK ON THE FAILED TEST IN TEST EXPLORER AND SELECT RUN SELECTED TESTS TO MAKE SURE THAT THE PROBLEM IS FIXED. ........ 8

20.A. ................................................................................................................................................................... 8

20.B.FIGURE ....................................................................................................................................................... 8

20.C.RUNNING THE FAILED TEST AGAIN TO VALIDATE FIX ..................................................................................... 8

21.SO FAR, WE HAVE SEEN HOW TO RUN ALL DISCOVERED TESTS, SEARCH FOR SPECIFIC TESTS, AND GROUP BY TEST OUTCOME. NOW LET’S TAKE AT A FEW OTHER WAYS THAT YOU CAN ORGANIZE AND NAVIGATE UNIT TESTS TO MAKE UNIT TESTING EASIER. FOR SOME MOTIVATIONAL CONTEXT, REMEMBER HOW LONG YOU NEEDED TO WAIT FOR THE CODED UI TEST TO COMPLETE, AND THEN IMAGINE THAT THERE ARE MANY MORE OF THOSE. IT WOULD BE NICE IF THE TESTS WERE ORGANIZED SUCH THAT YOU COULD EASILY AVOID KICKING OFF THOSE TESTS WITHOUT HAVING TO BE CAREFUL OF THE TESTS THAT YOU SELECT. ........................................................................................................... 8 22.TO START WITH, THERE ARE A NUMBER OF USEFUL GROUPING OPTIONS. RIGHT-CLICK SOMEWHERE WITHIN THE TEST EXPLORER WINDOW AND SELECT GROUP BY | CLASS. ....................................................................................................................................... 8

22.A. ................................................................................................................................................................... 8

22.B.FIGURE ....................................................................................................................................................... 8

22.C.GROUPING TESTS BY CLASS .......................................................................................................................... 8

23.ASSUMING THAT TESTS ARE WELL ORGANIZED WITHIN DESCRIPTIVE CLASS NAMES, THIS CAN MAKE IT MUCH EASIER TO SELECT AND RUN TESTS. FOR EXAMPLE, YOU COULD SELECT JUST THE TESTS FROM THE CUSTOMERSCONTROLLERTEST CLASS AND RUN THEM. ........................ 8

23.A. ................................................................................................................................................................... 9

23.B.FIGURE ....................................................................................................................................................... 9

23.C.GROUPING TESTS BY CLASS .......................................................................................................................... 9

24.RIGHT-CLICK SOMEWHERE WITHIN THE TEST EXPLORER WINDOW AND SELECT GROUP BY | PROJECT. .............................................. 9

24.A. ................................................................................................................................................................... 9

24.B.FIGURE ....................................................................................................................................................... 9

24.C.GROUPING TESTS BY PROJECT ...................................................................................................................... 9

25.GROUPING TESTS BY PROJECT WOULD OBVIOUSLY BE USEFUL FOR NAVIGATING AND RUNNING TESTS AT THE PROJECT LEVEL. .................... 9

25.A. ................................................................................................................................................................... 9

25.B.FIGURE ....................................................................................................................................................... 9

25.C.GROUPING TESTS BY PROJECT ...................................................................................................................... 9

26.YOU CAN ALSO USE TRAITS IN YOUR TEST CODE TO PROVIDE CUSTOM GROUPING. LET’S SAY THAT WE WANT TO GROUP ALL CODED UI TESTS TOGETHER. OPEN CODEDUITEST1.CS FROM THE FABRIKAMFIBER.WEB.UITESTS PROJECT. ................................................................ 9

26.A. ................................................................................................................................................................... 9

26.B.FIGURE ....................................................................................................................................................... 9

26.C.LOADING TEST CODE .................................................................................................................................... 9

27.UN-COMMENT THE CODEDUITEST ATTRIBUTE FROM THE TOP OF THE CODEDUITEST1 CLASS DEFINITION. ........................................ 9

27.A. ................................................................................................................................................................... 9

27.B.FIGURE ....................................................................................................................................................... 9

27.C.CODEDUITEST ATTRIBUTE ............................................................................................................................ 9

28.ADD THE TESTCATEGORY ATTRIBUTE TO THE CODEDUITESTMETHOD1 METHOD WITH A CATEGORY OF “UI”. .................................... 9

28.A. ................................................................................................................................................................... 9

28.B.FIGURE ..................................................................................................................................................... 10

28.C.ADDING CUSTOM TRAIT TO TEST METHOD ................................................................................................. 10

29.BUILD THE SOLUTION BY PRESSING CTRL+SHIFT+B. .......................................................................................................... 10 30.RIGHT-CLICK WITHIN TEST EXPLORER AND SELECT GROUP BY | TRAITS. ................................................................................. 10 31.WITH THE CODED UI TESTS CATEGORIZED APPROPRIATELY, IT IS NOW EASY TO SELECT AND RUN JUST THE UI TESTS IF DESIRED. .............. 10

31.A. ................................................................................................................................................................. 10

31.B.FIGURE ..................................................................................................................................................... 10

31.C.TEST EXPLORER WINDOW SHOWING TESTS GROUPED BY TRAITS ............................................................... 10

32.YOU CAN ALSO CREATE SETS OF TESTS CALLED PLAYLISTS. THIS PROVIDES CUSTOM GROUPING WITHOUT THE NEED TO MODIFY YOUR UNIT TESTING CODE. RIGHT-CLICK ON THE CODEDUITESTMETHOD1 TEST AND SELECT ADD TO PLAYLIST | NEW PLAYLIST. ............................. 10

32.A. ................................................................................................................................................................. 10

32.B.FIGURE ..................................................................................................................................................... 10

32.C.CREATING A NEW TEST PLAYLIST ................................................................................................................ 10

33.ENTER “UI TESTS” FOR THE PLAYLIST FILE NAME AND THEN SELECT SAVE. .............................................................................. 10

33.A. ................................................................................................................................................................. 10

33.B.FIGURE ..................................................................................................................................................... 10

33.C.SAVING A NEW TEST PLAYLIST .................................................................................................................... 10

34.SELECT THE PLAYLIST DROP-DOWN AND THEN SELECT THE “UI TESTS” PLAYLIST. ....................................................................... 10

34.A. ................................................................................................................................................................. 10

34.B.FIGURE ..................................................................................................................................................... 10

34.C.SELECTING A TEST PLAYLIST ........................................................................................................................ 10

35.WITH JUST THE “UI TESTS” PLAYLIST SELECTED, YOU WILL ONLY SEE THOSE SPECIFIC TESTS SHOWN IN TEST EXPLORER, MAKING IT MUCH EASIER TO FOCUS ON A PARTICULAR SELECTION OF TESTS. ........................................................................................................ 10

35.A. ................................................................................................................................................................. 11

35.B.FIGURE ..................................................................................................................................................... 11

35.C.TEST PLAYLIST SELECTED ............................................................................................................................. 11

36.TEST PLAYLISTS ARE SIMPLE XML FILES THAT DEFINE THE INDIVIDUAL TESTS TO INCLUDE. FOR EXAMPLE, HERE IS WHAT THE “UI TESTS” XML LOOKS LIKE (LOADED IN VISUAL STUDIO EDITOR). .......................................................................................................... 11

36.A. ................................................................................................................................................................. 11

36.B.FIGURE ..................................................................................................................................................... 11

36.C.CONTENTS OF A TEST PLAYLIST ................................................................................................................... 11

37.PLAYLISTS CAN BE SHARED WITH TEAM MEMBERS VIA A TEAM WEBSITE, EMAILED, AND EVEN ADDED TO SOURCE CONTROL IF DESIRED. TO LOAD A PLAYLIST FILE, YOU WOULD SELECT THE PLAYLIST DROP-DOWN AND THEN SELECT THE OPEN PLAYLIST FILE OPTION. YOU DO NOT NEED TO DO THIS FOR THIS LAB. ............................................................................................................................................... 11

37.A. ................................................................................................................................................................. 11

37.B.FIGURE ..................................................................................................................................................... 11

37.C.OPENING A PLAYLIST .................................................................................................................................. 11

EXERCISE 2: CODE COVERAGE ............................................................................................................................ 11

1.IN THIS EXERCISE, YOU WILL LEARN ABOUT CODE COVERAGE FEATURES THAT MAKE IT EASIER TO USE AND INTEGRATE INTO THE DEVELOPMENT CYCLE. ...................................................................................................... 11

2.YOU SHOULD ALREADY HAVE THE TEST EXPLORER WINDOW OPEN FROM THE PREVIOUS EXERCISE, BUT GO AHEAD AND OPEN IT IF NECESSARY (FROM TEST | WINDOWS | TEST EXPLORER). ..................................................................................................................... 11 3.RETURN TO THE DEFAULT PLAYLIST THAT INCLUDES ALL TESTS BY SELECTING THE PLAYLIST DROP-DOWN AND THEN SELECTING THE ALL TESTS OPTION. .................................................................................................................................................................... 11

3.A. ................................................................................................................................................................... 11

3.B.FIGURE ....................................................................................................................................................... 11

3.C.SELECTING THE ALL TESTS PLAYLIST .............................................................................................................. 11

4.TO ANALYZE CODE COVERAGE FOR ALL TESTS, SELECT THE RUN DROP-DOWN AND THEN SELECT THE ANALYZE CODE COVERAGE FOR ALL TESTS OPTION. THIS WILL INITIATE THE PROCESS OF BUILDING, TESTING, AND GATHERING CODE COVERAGE RESULTS. ........................................ 11

4.A. ................................................................................................................................................................... 11

4.B.FIGURE ....................................................................................................................................................... 11

4.C.ANALYZING CODE COVERAGE ....................................................................................................................... 11

5.YOU CAN VIEW THE RESULTS IN THE CODE COVERAGE RESULTS WINDOWS TO GET AN IDEA OF THE COVERED/NOT COVERED STATISTICS FOR ALL OF THE TESTS. IN THE SCREENSHOT BELOW, NOTE THAT COVERAGE IS MEASURED IN BLOCKS OF CODE BY DEFAULT, WHERE A BLOCK IS CODE WITH EXACTLY ONE ENTRY AND EXIT POINT. .......................................................................................................................... 12

5.A. ................................................................................................................................................................... 12

5.B.FIGURE ....................................................................................................................................................... 12

5.C.CODE COVERAGE RESULTS SUMMARY .......................................................................................................... 12

5.E.NOTE: IF YOU WOULD LIKE TO SEE THE RESULTS IN TERMS OF LINES, YOU CAN DO THAT BY RIGHT-CLICKING IN THE CODE COVERAGE RESULTS WINDOW AND SELECTING THE ADD/REMOVE COLUMNS OPTION. ................. 12

6.EXPAND THE ROOT NODE OF THE CODE COVERAGE RESULT TO VIEW THE COVERAGE BROKEN DOWN BY ASSEMBLY. BY DEFAULT, WE SEE ALL ASSEMBLIES THAT ARE LOADED DURING THE TEST RUN (AND FOR WHICH A .PDB FILE IS AVAILABLE). ..................................................... 12

6.A. ................................................................................................................................................................... 12

6.B.FIGURE ....................................................................................................................................................... 12

6.C.CODE COVERAGE RESULTS BY ASSEMBLY ...................................................................................................... 12

6.D.NOTE: YOU CAN CUSTOMIZE CONTROL OVER WHICH ASSEMBLIES ARE SELECTED FOR CODE COVERAGE ANALYSIS BY WRITING A .RUNSETTINGS FILE. FOR MORE INFORMATION, SEE THE MSDN ARTICLE CUSTOMIZING CODE COVERAGE ANALYSIS. .............................................................................................................................. 12

7.EXPAND THE FABRIKAMFIBER.WEB.DLL NODE TO VIEW THE COVERAGE BROKEN DOWN BY NAMESPACE. THIS ALLOWS US TO SEE THAT, ALTHOUGH WE DO HAVE SOME TEST COVERAGE OF THE CONTROLLER CLASSES, THE TEST TEAM HAS A LOT OF WORK TO DO IN ORDER TO PROVIDE COVERAGE TO THE OTHER NAMESPACES. ............................................................................................................................. 12

7.A. ................................................................................................................................................................... 12

7.B.FIGURE ....................................................................................................................................................... 12

7.C.CODE COVERAGE RESULTS FOR NAMESPACES WITHIN ASSEMBLY ................................................................. 12

8.EXPAND THE FABRIKAMFIBER.WEB.CONTROLLERS NAMESPACE NODE TO VIEW THE COVERAGE BROKEN DOWN BY CLASS. THIS SHOWS THAT THE HOMECONTROLLER CLASS IS COVERED WELL AND THAT THE EMPLOYEESCONTROLLER CURRENTLY HAS NO COVERAGE. ........................ 12

8.A. ................................................................................................................................................................... 12

8.B.FIGURE ....................................................................................................................................................... 12

8.C.CODE COVERAGE RESULTS FOR CLASSES WITHIN NAMESPACE ...................................................................... 12

9.FINALLY, LET’S DRILL DOWN INTO THE CLASS NODES TO SEE COVERAGE DOWN TO THE METHOD LEVEL BY EXPANDING THE CUSTOMERSCONTROLLER CLASS NODE. .............................................................................................................................. 12

9.A. ................................................................................................................................................................... 13

9.B.FIGURE ....................................................................................................................................................... 13

9.C.CODE COVERAGE RESULTS FOR METHODS .................................................................................................... 13

10.DOUBLE-CLICK ON THE CREATE(FABRIKAMFIBER.DAL.MODELS.CUSTOMER) CONSTRUCTOR TO NAVIGATE TO THE SOURCE CODE TO VISUALIZE THE BLOCK COVERAGE. ...................................................................................................................................... 13

10.A. ................................................................................................................................................................. 13

10.B.FIGURE ..................................................................................................................................................... 13

10.C.NAVIGATING TO SOURCE ........................................................................................................................... 13

11.IN THE EDITOR WINDOW FOR SOURCECONTROLLER.CS, YOU CAN SEE THAT THE CODE HIGHLIGHTED BLUE REPRESENTS THE BLOCK THAT WAS COVERED BY TESTS WHEREAS THE RED REPRESENTS THE BLOCK THAT WAS NOT COVERED. .................................................................. 13

11.A. ................................................................................................................................................................. 13

11.B.FIGURE ..................................................................................................................................................... 13

11.C.SOURCE VIEW SHOWING CODE COVERAGE COLORING ............................................................................... 13

12.IT IS ALSO POSSIBLE TO GET CODE COVERAGE FOR A SPECIFIC SELECTION OF TESTS. IN THE TEST EXPLORER WINDOW, SELECT THE CREATEINSERTSCUSTOMERANDSAVES TEST METHOD, RIGHT-CLICK, AND THEN SELECT ANALYZE CODE COVERAGE FOR SELECTED TESTS. ....... 13

12.A. ................................................................................................................................................................. 13

12.B.FIGURE ..................................................................................................................................................... 13

12.C.ANALYZE CODE COVERAGE FOR SPECIFIED TESTS ........................................................................................ 13

13.ONCE TEST EXECUTION COMPLETES FOR THE SELECTED TEST, EXPAND THE CODE COVERAGE NODE AND NOTE THAT ONLY THE ASSEMBLIES LOADED DURING TEST EXECUTION ARE SHOWN WITH STATISTICS. ................................................................................................ 13

13.A. ................................................................................................................................................................. 13

13.B.FIGURE ..................................................................................................................................................... 13

13.C.CODE COVERAGE RESULTS .......................................................................................................................... 13

14.IT IS ALSO EASY TO NAVIGATE BETWEEN DIFFERENT CODE COVERAGE RESULTS BY USING THE DROP-DOWN IN THE CODE COVERAGE RESULTS WINDOW. GO AHEAD AND SELECT THE FIRST CODE COVERAGE RESULTS FILE. ................................................................................. 13

14.A. ................................................................................................................................................................. 13

14.B.FIGURE ..................................................................................................................................................... 14

14.C.SELECTING SPECIFIC CODE COVERAGE RESULTS .......................................................................................... 14

15.LET’S SAY THAT WE WANT TO USE THESE CODE COVERAGE RESULTS IN A REPORT OR SIMPLY SHARE THEM EXTERNALLY. TO DO THAT, CLICK ON THE EXPORT RESULTS BUTTON. ........................................................................................................................................ 14

15.A. ................................................................................................................................................................. 14

15.B.FIGURE ..................................................................................................................................................... 14

15.C.EXPORTING CODE COVERAGE RESULTS ....................................................................................................... 14

16.IN THE SAVE COVERAGE DATA AS XML WINDOW, YOU COULD SAVE THE CODE COVERAGE DATA TO AN XML FILE, BUT FOR THE PURPOSES OF THIS LAB, GO AHEAD AND SIMPLY CANCEL OUT OF THE DIALOG AND CONTINUE ON TO THE NEXT EXERCISE. ....................................... 14

16.A. ................................................................................................................................................................. 14

16.B.FIGURE ..................................................................................................................................................... 14

16.C.SAVING CODE COVERAGE RESULTS TO FILE ................................................................................................. 14

EXERCISE 3: CODE CLONE ANALYSIS ................................................................................................................... 14

1.IN THIS EXERCISE, YOU WILL LEARN ABOUT THE CODE CLONE ANALYSIS FEATURE THAT LOOKS FOR SEMANTICALLY SIMILAR CODE USING A HEURISTIC SEARCH TECHNIQUE, RATHER THAN SIMPLY SEARCHING FOR EXACT MATCHES. .............................................................................................................................................. 14

2.SELECT ANALYZE | ANALYZE SOLUTION FOR CODE CLONES FROM THE MAIN MENU IN VISUAL STUDIO. ............................................ 14

2.A. ................................................................................................................................................................... 14

2.B.FIGURE ....................................................................................................................................................... 14

2.C.ANALYZE SOLUTION FOR CODE CLONES ........................................................................................................ 14

3.ONCE THE ANALYSIS IS COMPLETE, THE CODE CLONE ANALYSIS RESULTS WINDOW WILL SHOW CLONE CANDIDATES GROUPED BY MATCH STRENGTH. EXPAND THE STRONG MATCH GROUP TO EXPOSE THE TWO FILES THAT HAD A STRONG MATCH. ........................................... 14

3.A. ................................................................................................................................................................... 14

3.B.FIGURE ....................................................................................................................................................... 14

3.C.VIEWING THE STRONG FILE MATCHES .......................................................................................................... 14

4.EACH LINE SHOWS THE CLASS AND METHOD, SPECIFIC FILE, AND LINES THAT WERE DETERMINED TO BE STRONG MATCHES. IF YOU MOUSE OVER EACH MATCH, A SNIPPET OF CODE GIVES YOU QUICK GLANCE AT SOME OF THE MATCHING CODE. ................................................. 15

4.A. ................................................................................................................................................................... 15

4.B.FIGURE ....................................................................................................................................................... 15

4.C.MOUSE OVER RESULTS TO SEE DETAILS ........................................................................................................ 15

5.DOUBLE-CLICK ON EACH MATCH TO OPEN THEM IN CODE EDITOR WINDOWS, AND THEN RIGHT-CLICK ON THE TITLE TAB FOR ONE OF THEM AND SELECT NEW HORIZONTAL TAB GROUP FROM THE CONTEXT MENU. ..................................................................................... 15

5.A. ................................................................................................................................................................... 15

5.B.FIGURE ....................................................................................................................................................... 15

5.C.OPENING BOTH FILES FOR COMPARISON ..................................................................................................... 15

6.SCROLL THROUGH THE CODE TO LOCATE THE ASSIGNSCHEDULE METHOD FOR EACH FILE AND NOTE THAT IT IS IDENTICAL EXCEPT FOR THE LAST LINE THAT CALLS THE REDIRECTTOACTION METHOD. THIS INDICATES THAT THIS METHOD IS A GOOD CANDIDATE FOR REFACTORING. THIS KIND OF BROAD SEARCH IS PARTICULARLY USEFUL WHEN LOOKING FOR CODE THAT CAN BE REFACTORED FOR EASIER MAINTENANCE IN THE FUTURE. . 15

6.A. ................................................................................................................................................................... 15

6.B.FIGURE ....................................................................................................................................................... 15

6.C.COMPARING THE STRONG MATCH FROM THE CODE CLONE ANALYSIS .......................................................... 15

7.SELECT WINDOW | CLOSE ALL DOCUMENTS FROM THE MAIN MENU TO CLEAR UP SOME SCREEN REAL ESTATE SPACE. .......................... 15 8.YOU CAN ALSO NARROW THE FOCUS OF THE SEARCH FOR CODE CLONES IF DESIRED. IN SOLUTION EXPLORER, NAVIGATE TO CONTROLLERS FOLDER OF THE FABRIKAMFIBER.WEB PROJECT AND OPEN CUSTOMERSCONTROLLER.CS IN THE CODE EDITOR. ........................................ 15

8.A. ................................................................................................................................................................... 15

8.B.FIGURE ....................................................................................................................................................... 15

8.C.OPENING THE CUSTOMERSCONTROLLER.CS SOURCE FILE ............................................................................. 15

9.SCROLL DOWN TO THE CREATE METHOD THAT TAKES A CUSTOMER PARAMETER AND SELECT THE THREE LINES OF CODE WITHIN THE FIRST “IF” STATEMENT. ................................................................................................................................................................ 15

9.A. ................................................................................................................................................................... 15

9.B.FIGURE ....................................................................................................................................................... 15

9.C.SELECTING CODE FOR A CODE CLONE ANALYSIS ........................................................................................... 15

10.RIGHT-CLICK ON THE SELECTED LINES OF CODE AND THEN SELECT THE FIND MATCHING CLONES IN SOLUTION OPTION FROM THE CONTEXT MENU. ...................................................................................................................................................................... 16

10.A. ................................................................................................................................................................. 16

10.B.FIGURE ..................................................................................................................................................... 16

10.C.FINDING MATCHING CLONES FOR THE SELECTED CODE ............................................................................... 16

11.AFTER THE SEARCH IS COMPLETE, THE CODE CLONE SEARCH RESULTS WINDOW SHOWS SNIPPET MATCHES OF VARYING STRENGTH. ........ 16

11.A. ................................................................................................................................................................. 16

11.B.FIGURE ..................................................................................................................................................... 16

11.C.RESULTS OF CODE CLONE SEARCH .............................................................................................................. 16

12.EXPAND ALL OF THE CLONE GROUPS TO EXPOSE THE DISCOVERED MATCHES, INCLUDING THE ORIGINAL CLONE GROUP. YOU MAY WANT TO INCREASE THE SIZE OF THE CODE CLONE SEARCH RESULTS WINDOW SO THAT YOU CAN SEE ALL MATCHES AT A GLANCE. ........................... 16

12.A. ................................................................................................................................................................. 16

12.B.FIGURE ..................................................................................................................................................... 16

12.C.CODE CLONE RESULTS WITH GROUPS EXPANDED ....................................................................................... 16

13.HOLD THE MOUSE CURSOR OVER THE ORIGINAL CODE SNIPPET TO REMIND OURSELVES WHAT THE CODE CLONES ARE BEING COMPARED TO. ............................................................................................................................................................................... 16

13.A. ................................................................................................................................................................. 16

13.B.FIGURE ..................................................................................................................................................... 16

13.C.ORIGINAL CODE SNIPPET ........................................................................................................................... 16

14.HOLD THE MOUSE CURSOR OVER THE EXACT MATCH RESULT AND NOTE THAT THE EDIT METHOD USES EXACTLY THE SAME CODE AS THE CODE FROM THE CREATE METHOD. .................................................................................................................................... 16 15.HOLD THE MOUSE CURSOR OVER THE STRONG MATCH RESULT AND NOTE THAT THE ONLY DIFFERENCE IS THAT THE FIRST LINE IS A CALL TO A DELETE METHOD. ......................................................................................................................................................... 16

15.A. ................................................................................................................................................................. 16

15.B.FIGURE ..................................................................................................................................................... 16

15.C.CODE SNIPPET RESULT WITH STRONG MATCH TO ORIGINAL ....................................................................... 16

16.HOLD THE MOUSE CURSOR OVER THE FIRST MEDIUM MATCH RESULT AND NOTE THAT THE SNIPPET IS SIMILAR TO THE ORIGINAL, BUT NOW WE ARE WORKING WITH AN ENTIRELY DIFFERENT OBJECT (NOW EMPLOYEEREPOSITORY). .................................................................. 16

16.A. ................................................................................................................................................................. 16

16.B.FIGURE ..................................................................................................................................................... 17

16.C.CODE SNIPPET RESULT WITH STRONG MATCH TO ORIGINAL ....................................................................... 17

17.IN SUMMARY, THERE ARE THREE MAIN SCENARIOS WHERE THE IDENTIFICATION OF CODE CLONES MAY BE USEFUL TO DEVELOPERS: .......... 17 17.a.Identification of candidates for code refactoring ........................................................................................ 1717.b.Bug fixes or enhancements are made in code and the developer wants to see if there are other similar locations that should be updated ........................................................................................................................ 1717.c.As a learning tool for a new developer starting work on a new team project, e.g. the developer adds code to update a customer record and wants to see if there are practices used by the rest of the codebase such as using a Try… Catch block ...................................................................................................................................... 17

1.

Overview

1. In this lab, you will learn about how the unit testing engine in Visual Studio is extensible and opens the door for 3rd party unit testing adapters such as NUnit and xUnit.net. In addition, you will see some of the improvements made to code coverage support. You will also learn about how the code clone capability goes hand-in-hand with delivering high quality software by helping you identify blocks of semantically similar code which may be candidates for common bug fixes or refactoring.

Prerequisites

1. In order to complete this lab you will need the Visual Studio 2015 virtual machine provided by Microsoft. For more information on acquiring and using this virtual machine, please see this blog post.

About the Fabrikam Fiber Scenario

1. This set of hands-on-labs uses a fictional company, Fabrikam Fiber, as a backdrop to the scenarios you are learning about. Fabrikam Fiber provides cable television and related services to the United States. They are growing rapidly and have embraced Windows Azure to scale their customer-facing web site directly to end-users to allow them to self-service tickets and track technicians. They also use an on-premises ASP.NET MVC application for their customer service representatives to administer customer orders.

In this set of hands-on labs, you will take part in a number of scenarios that involve the development andtesting team at Fabrikam Fiber. The team, which consists of 8-10 people, has decided to use Visual Studioapplication lifecycle management tools to manage their source code, run their builds, test their web sites, and plan and track the project.

Exercises

1. This hands-on lab includes the following exercises:

2. Unit Testing

3. Code Coverage

4. Code Clone Analysis

1. Estimated time to complete this lab: 30 minutes.

Exercise 1: Unit Testing

1. In this exercise, you will learn about the unit testing story in Visual Studio 2015.

2. Log in as Julia (VSALM\Julia). All user passwords are P2ssw0rd.

3. Launch Visual Studio 2015 from the taskbar and open Team Explorer. You should now be connected to the FabrikamFiber team project. If you are not automatically connected to the

FabrikamFiber project, select the Connect to Team Projects button ( ) to do so.

4. In Team Explorer – Home, double-click on the first FabrikamFiber.CallCenter.sln solution.

4.a.4.b. Figure 14.c. Loading Fabrikam Fiber solution

4.d.

5. Press Ctrl + Shift + B to build the solution.

6. Open the Test Explorer window from Test | Windows | Test Explorer. Note that discovered tests are initially set to the Not Run state.

6.a.

6.b. Figure 26.c. Test Explorer window

6.d.

7. Select the Run All link to execute all discovered unit tests.

7.a.7.b. Figure 37.c. Running all unit tests

7.d.

7.e. Note: Visual Studio also provides Continuous Test Runner functionality that can be enabled with the Test | Test Settings | Run Tests After Build option. With this option selected, tests will always run after builds.

8. Expand the Passed Tests group (if necessary) and double-click on the test with the name CreateInsertsCustomerAndSaves to open the source code.

8.a.8.b. Figure 48.c. Navigating to a test method that uses MSTest

8.d.

9. In the CustomersControllerTest.cs file that opens, note that the test method uses the expected TestMethod attribute used by MSTest to mark unit tests.

9.a.9.b. Figure 59.c. MSTest unit test code

9.d.

Note: The test status indicator just above the method definition is a CodeLens indicator. It tells us that the last run was a success. You can learn more about the CodeLens feature in the “Collaboration Experiences for Development Teams using Team Foundation Server 2015” lab.

10. In the Search box at the top of the Test Explorer window, enter “index” and note the available search filters.

10.a.10.b. Figure 610.c. Searching within Test Explorer

10.d.

11. In the search results, double-click on the only test listed with the name IndexReturnsNonNullView to open the source code.

11.a.11.b. Figure 711.c. Navigating to a test method that uses XUnit

11.d.

12. In the HomeControllerTest.cs file that opens, you can see that the IndexReturnsNonNullView test actually uses the XUnit testing framework. Both the xUnit framework and the test runner that integrates with Test Explorer can be added to a project via NuGet.

12.a.12.b. Figure 8Example of a XUnit test in Visual Studio

12.c.

13. Select X button to clear the current search box in Test Explorer.

13.a.13.b. Figure 913.c. Location of X button to clear search box

13.d.

14. Note that results are grouped by test outcome by default, with execution times listed.

14.a.14.b. Figure 1014.c. Test run results

14.d.

15. Test execution performance has been given some additional attention in recent versions. Take note of the execution times and then select the Run… button followed by the Repeat Last Run option. Note the difference in execution time.

15.a.

15.b. Figure 1115.c. Repeating the last test run

15.d. Note: The execution times that you see will be different from those shown in the screenshots. The first time you run unit tests they will execute more slowly than on subsequent runs as the tests and testing engine are being loaded for the first time.

15.e.

15.f.15.g. Figure 1215.h. Second test run without changes to tests is faster

15.i.

16. Single-click the failed test named “ScheduleActionCorrectlyUpdatesRepositories” to view asummary of test results.

16.a.16.b. Figure 1316.c. Selecting the failed test

16.d.

16.e. Note: You can also right-click on test results and use the Copy command to copy details to the clipboard. This would be useful when sending an email, for example.

16.f.

17. The summary view of the failed test run shows that an ArgumentNullException exception occurred during the test run and even provides the stack track at the time of the exception. Note that we can follow links to go directly to the source code for the test or to points within thestack trace. Select the source link to go to the source code for the test method.

17.a.17.b. Figure 1417.c. Navigate to the source of the test method

17.d.

17.e.

17.f. Figure 1517.g. Initial view after navigating to source of the test method

17.h.

18. Find the line of code that is commented out and un-comment it. Assume that this is the root cause for the failed test.

18.a.

18.b. Figure 1618.c. Fixing the root problem of the failed test

18.d.

19. Press Ctrl+S to save your changes.

20. Right-click on the failed test in Test Explorer and select Run Selected Tests to make sure that the problem is fixed.

20.a.20.b. Figure 1720.c. Running the failed test again to validate fix

20.d.

21. So far, we have seen how to run all discovered tests, search for specific tests, and group by test outcome. Now let’s take at a few other ways that you can organize and navigate unit tests tomake unit testing easier. For some motivational context, remember how long you needed to wait for the coded UI test to complete, and then imagine that there are many more of those. It would be nice if the tests were organized such that you could easily avoid kicking off those tests without having to be careful of the tests that you select.

22. To start with, there are a number of useful grouping options. Right-click somewhere within the Test Explorer window and select Group By | Class.

22.a.

22.b. Figure 1822.c. Grouping tests by class

22.d.

23. Assuming that tests are well organized within descriptive class names, this can make it much easier to select and run tests. For example, you could select just the tests from the CustomersControllerTest class and run them.

23.a.23.b. Figure 1923.c. Grouping tests by class

23.d.

24. Right-click somewhere within the Test Explorer window and select Group By | Project.

24.a.24.b. Figure 2024.c. Grouping tests by project

24.d.

25. Grouping tests by project would obviously be useful for navigating and running tests at the project level.

25.a.25.b. Figure 2125.c. Grouping tests by project

25.d.

26. You can also use traits in your test code to provide custom grouping. Let’s say that we want to group all coded UI tests together. Open CodedUITest1.cs from the FabrikamFiber.Web.UITests project.

26.a.26.b. Figure 2226.c. Loading test code

26.d.

27. Un-comment the CodedUITest attribute from the top of the CodedUITest1 class definition.

27.a.27.b. Figure 2327.c. CodedUITest attribute

27.d.

28. Add the TestCategory attribute to the CodedUITestMethod1 method with a category of “UI”.

28.a.

28.b. Figure 2428.c. Adding custom trait to test method

28.d.

29. Build the solution by pressing Ctrl+Shift+B.

30. Right-click within Test Explorer and select Group By | Traits.

31. With the coded UI tests categorized appropriately, it is now easy to select and run just the UI tests if desired.

31.a.31.b. Figure 2531.c. Test Explorer window showing tests grouped by traits

31.d.

32. You can also create sets of tests called playlists. This provides custom grouping without the need to modify your unit testing code. Right-click on the CodedUITestMethod1 test and select Add to Playlist | New Playlist.

32.a.

32.b. Figure 2632.c. Creating a new test playlist

32.d.

33. Enter “UI Tests” for the playlist file name and then select Save.

33.a.

33.b. Figure 2733.c. Saving a new test playlist

33.d.

34. Select the Playlist drop-down and then select the “UI Tests” playlist.

34.a.34.b. Figure 2834.c. Selecting a test playlist

34.d.

35. With just the “UI Tests” playlist selected, you will only see those specific tests shown in Test Explorer, making it much easier to focus on a particular selection of tests.

35.a.35.b. Figure 2935.c. Test playlist selected

35.d.

36. Test playlists are simple XML files that define the individual tests to include. For example, here is what the “UI Tests” XML looks like (loaded in Visual Studio editor).

36.a.

36.b. Figure 3036.c. Contents of a test playlist

36.d.

37. Playlists can be shared with team members via a team website, emailed, and even added tosource control if desired. To load a playlist file, you would select the Playlist drop-down and thenselect the Open Playlist File option. You do not need to do this for this lab.

37.a.37.b. Figure 31

37.c. Opening a playlist

Exercise 2: Code Coverage

1. In this exercise, you will learn about code coverage features that make it easier to use and integrate into the development cycle.

2. You should already have the Test Explorer window open from the previous exercise, but go ahead and open it if necessary (from Test | Windows | Test Explorer).

3. Return to the default playlist that includes all tests by selecting the Playlist drop-down and then selecting the All Tests option.

3.a.3.b. Figure 323.c. Selecting the All Tests playlist

3.d.

4. To analyze code coverage for all tests, select the Run drop-down and then select the Analyze Code Coverage for All Tests option. This will initiate the process of building, testing, andgathering code coverage results.

4.a.

4.b. Figure 334.c. Analyzing code coverage

4.d.

5. You can view the results in the Code Coverage Results windows to get an idea of the Covered/Not Covered statistics for all of the tests. In the screenshot below, note that coverage ismeasured in blocks of code by default, where a block is code with exactly one entry and exit point.

5.a.

5.b. Figure 345.c. Code coverage results summary

5.d.

5.e. Note: If you would like to see the results in terms of lines, you can do that by right-clicking in the Code Coverage Results window and selecting the Add/Remove Columns option.

6. Expand the root node of the code coverage result to view the coverage broken down by assembly. By default, we see all assemblies that are loaded during the test run (and for which a .pdb file is available).

6.a.

6.b. Figure 356.c. Code coverage results by assembly

6.d. Note: You can customize control over which assemblies are selected for code coverage analysis by writing a .runsettings file. For more information, see the MSDN article Customizing Code Coverage Analysis.

7. Expand the fabrikamfiber.web.dll node to view the coverage broken down by namespace. This allows us to see that, although we do have some test coverage of the controller classes, the test team has a lot of work to do in order to provide coverage to the other namespaces.

7.a.

7.b. Figure 367.c. Code coverage results for namespaces within assembly

7.d.

8. Expand the FabrikamFiber.Web.Controllers namespace node to view the coverage broken down by class. This shows that the HomeController class is covered well and that the EmployeesController currently has no coverage.

8.a.

8.b. Figure 378.c. Code coverage results for classes within namespace

8.d.

9. Finally, let’s drill down into the class nodes to see coverage down to the method level by expanding the CustomersController class node.

9.a.

9.b. Figure 389.c. Code coverage results for methods

9.d.

10. Double-click on the Create(FabrikamFiber.DAL.Models.Customer) constructor to navigate to the source code to visualize the block coverage.

10.a.10.b. Figure 3910.c. Navigating to source

10.d.

11. In the editor window for SourceController.cs, you can see that the code highlighted blue represents the block that was covered by tests whereas the red represents the block that was not covered.

11.a.11.b. Figure 4011.c. Source view showing code coverage coloring

11.d.

12. It is also possible to get code coverage for a specific selection of tests. In the Test Explorer window, select the CreateInsertsCustomerAndSaves test method, right-click, and then select Analyze Code Coverage for Selected Tests.

12.a.12.b. Figure 4112.c. Analyze code coverage for specified tests

12.d.

13. Once test execution completes for the selected test, expand the code coverage node and note that only the assemblies loaded during test execution are shown with statistics.

13.a.

13.b. Figure 4213.c. Code coverage results

13.d.

14. It is also easy to navigate between different code coverage results by using the drop-down in the Code Coverage Results window. Go ahead and select the first code coverage results file.

14.a.14.b. Figure 4314.c. Selecting specific code coverage results

14.d.

15. Let’s say that we want to use these code coverage results in a report or simply share them externally. To do that, click on the Export Results button.

15.a.15.b. Figure 4415.c. Exporting code coverage results

15.d.

16. In the Save Coverage Data as XML window, you could save the code coverage data to an XML file, but for the purposes of this lab, go ahead and simply Cancel out of the dialog and continue on to the next exercise.

16.a.

16.b. Figure 4516.c. Saving code coverage results to file

Exercise 3: Code Clone Analysis

1. In this exercise, you will learn about the Code Clone analysis feature that looks for semantically similar code using a heuristic search technique, rather than simply searching for exact matches.

2. Select Analyze | Analyze Solution for Code Clones from the main menu in Visual Studio.

2.a.2.b. Figure 462.c. Analyze solution for code clones

2.d.

3. Once the analysis is complete, the Code Clone Analysis Results window will show clone candidates grouped by match strength. Expand the Strong Match group to expose the two files that had a strong match.

3.a.3.b. Figure 473.c. Viewing the strong file matches

3.d.

4. Each line shows the class and method, specific file, and lines that were determined to be strong matches. If you mouse over each match, a snippet of code gives you quick glance at some of the matching code.

4.a.

4.b. Figure 484.c. Mouse over results to see details

4.d.

5. Double-click on each match to open them in code editor windows, and then right-click on the title tab for one of them and select New Horizontal Tab Group from the context menu.

5.a.5.b. Figure 495.c. Opening both files for comparison

5.d.

6. Scroll through the code to locate the AssignSchedule method for each file and note that it is identical except for the last line that calls the RedirectToAction method. This indicates that this method is a good candidate for refactoring. This kind of broad search is particularly useful when looking for code that can be refactored for easier maintenance in the future.

6.a.

6.b. Figure 506.c. Comparing the strong match from the code clone analysis

6.d.

7. Select Window | Close All Documents from the main menu to clear up some screen real estate space.

8. You can also narrow the focus of the search for code clones if desired. In Solution Explorer, navigate to Controllers folder of the FabrikamFiber.Web project and open CustomersController.cs in the code editor.

8.a.8.b. Figure 518.c. Opening the CustomersController.cs source file

8.d.

9. Scroll down to the Create method that takes a Customer parameter and select the three lines of code within the first “if” statement.

9.a.9.b. Figure 529.c. Selecting code for a code clone analysis

9.d.

10. Right-click on the selected lines of code and then select the Find Matching Clones in Solution option from the context menu.

10.a.10.b. Figure 5310.c. Finding matching clones for the selected code

10.d.

11. After the search is complete, the Code Clone Search Results window shows snippet matches of varying strength.

11.a.11.b. Figure 5411.c. Results of code clone search

11.d.

12. Expand all of the clone groups to expose the discovered matches, including the original clone group. You may want to increase the size of the Code Clone Search Results window so thatyou can see all matches at a glance.

12.a.12.b. Figure 5512.c. Code clone results with groups expanded

12.d.

13. Hold the mouse cursor over the original code snippet to remind ourselves what the code clones are being compared to.

13.a.13.b. Figure 5613.c. Original code snippet

14. Hold the mouse cursor over the Exact Match result and note that the Edit method uses exactly the same code as the code from the Create method.

15. Hold the mouse cursor over the Strong Match result and note that the only difference is that the first line is a call to a Delete method.

15.a.15.b. Figure 5715.c. Code snippet result with strong match to original

16. Hold the mouse cursor over the first Medium Match result and note that the snippet is similar to the original, but now we are working with an entirely different object (now employeeRepository).

16.a.16.b. Figure 5816.c. Code snippet result with strong match to original

17. In summary, there are three main scenarios where the identification of code clones may be useful to developers:

17.a. Identification of candidates for code refactoring

17.b. Bug fixes or enhancements are made in code and the developer wants to see if there are other similar locations that should be updated

17.c. As a learning tool for a new developer starting work on a new team project, e.g. the developer adds code to update a customer record and wants to see if there are practices used by the rest of the codebase such as using a Try… Catch block