Identify & Fix Performance Problems with Visual Studio 2012 UltimateBenjamin DayBenjamin Day Consulting, Inc.benday.com/blog | @benday

About Me

Cambridge, MAConsultant, Coach, & TrainerMicrosoft MVP for Visual Studio ALMTeam Foundation Server, Software Testing, Scrum, Software ArchitectureScrum.org Classes

Professional Scrum Developer (PSD)Professional Scrum Foundations (PSF)

www.benday.com, benday@benday.com, @benday

Have you heard of Scrum.org?

http://www.pluralsight.com/training/Courses/TableOfContents/alm-fundamentals

http://pluralsight.com/training/Courses/TableOfContents/alm-for-developers

Thanks.

To my content reviewersJerri Chiu, MicrosoftDennis Bass, Microsoft

The Agenda

Basics of Web Performance TestsLoad TestsLoad Test RigsPerformance Explorer

Existing application Identify performance problemsGo from bad performance to good

The Purpose of Load Testing

Identify the capabilities of your applicationPossible goals

ExploreVerify Find the limitsCrush

The Obligatory “Eat Your Vegetables” Slide.

Load test throughout the development cycle.

Catch problems earlyEstablish performance baselineWatch for performance trends

Cheaper to fix early

Plan for performance

Best Practice

Cheaper to Fix Early

Early Less Early Nearly Done Production0

10

20

30

40

50

60

Time

Cost

to fi

x

Dose Of Reality

You know you should work that way.You probably don’t.

This talk is for you.

You have problems.How do you figure out what’s going wrong?

Inspiration for this talk

Real customer engagementSpent years developing a web applicationReleased it Seriously grumpy customersHuge hardware~20 simultaneous usersFixed it

4000+ simultaneous usersReduced hardware

The Tools

Visual Studio Ultimate

• Web Performance Tests• Load Tests• Load Test Rigs

Visual Studio Premium

• Profiling Tools• Performance Wizard

Tour of the application.

Disaster Strikes.

The application is deployed to production.Users are losing their minds.

Intermittent errors on the site.Site is slow.

Your boss is seriously angry.“Fix it!”

The Problem.

“No Repro”“Works on my box.”You don’t know what the errors are.The site seems to work fine.

Wild guess: it dies under load.

You need to create some load.

The Plan.

1. Script user activity with Web Performance Tests2. Simulate a mix of users using Load Tests3. Simulate lots of users with a Load Test Rig4. Reproduce the errors...hopefully5. Find code problems with the Performance Wizard6. Fix the code

Web Performance Tests.

The Plan.

1. Script user activity with Web Performance Tests

2. Simulate a mix of users using Load Tests3. Simulate lots of users with a Load Test Rig4. Reproduce the errors...hopefully5. Find code problems with the Performance Wizard6. Fix the code

What is a Web Performance Test?

Test type in Visual Studio 2012 UltimateSimulate a user using an ASP.NET application

HTTP trafficBuilding block of Load Tests

Tip: Web Tests & User Stories

Web Tests should simulate a User StoryExample:

As an administrator, I need to search for a person by username so that I can deactivate that person’s account.As a customer, I want to order a pizza and pay with a credit card.

Helps to organize your suite

Create a basic Web Test

Data-driven Web Tests

Run web test once per row in data sourceSimulate multiple users doing *similar* actionsData source types

OLE DB, CSV, XMLParameterization

Better tests, better coverage

Web Tests in the underperforming app

Load Tests.

The Plan.

1. Script user activity with Web Performance Tests2. Simulate a mix of users using Load Tests3. Simulate lots of users with a Load Test Rig4. Reproduce the errors...hopefully5. Find code problems with the Performance Wizard6. Fix the code

What are Load Tests?

Mix of Web Performance and Unit Tests

Simulates a mix of users doing a mix of things

Simulate Network speedsDifferent browsersVarying user activity loads

Why would you put a Unit Test in a Load Test?

Helpful for testing WCFCumbersome to simulate WCF via Web Test

(You’d have to re-invent the wheel.)Unit tests can re-use the WCF libraries

(Done and done.)

Important for Silverlight and SOAhttp://wcfloadtest.codeplex.com/http://sqlloadtest.codeplex.com/

(Hooray!)

“Hey Kids,Let’s create some Load Tests!”

Running load tests.

The Plan.

1. Script user activity with Web Performance Tests2. Simulate a mix of users using Load Tests3. Simulate lots of users with a Load Test Rig4. Reproduce the errors...hopefully5. Find code problems with the Performance Wizard6. Fix the code

Two ways to run load tests.

Generate load from Visual Studio 2012Uses only 1 core/processorLimited to 250 simulated users

Generate load from a Test RigFar better testSimulate lots more users

Load Test Rig & The Application

ControllerOrchestratesLicensingPerfMon Counters

Agent (1..n)Simulates users

Controller

Agent 1

Web Server

Database

Agent 2

Agent n

Coordinates & Gathers Information

Generates Load

Application Under Test

Load Test Rig: Manage Test Controller

Load Test Rig: Test Settings

Load Test Rig: Remote Execution

The Plan.

1. Script user activity with Web Performance Tests2. Simulate a mix of users using Load Tests3. Simulate lots of users with a Load Test Rig4. Reproduce the errors...hopefully5. Find code problems with the Performance Wizard6. Fix the code

Run Load Tests on the Rig

Before: Load Test Summary

Before: Load Test Graphs

The Core Problem.

‘sp_who2’ says lots of open connections.

After: Load Test Summary

After: Load Test Graphs

Performance Profiling

The Plan.

1. Script user activity with Web Performance Tests2. Simulate a mix of users using Load Tests3. Simulate lots of users with a Load Test Rig4. Reproduce the errors...hopefully5. Find code problems with the Performance

Wizard6. Fix the code

Profiling Tools

Load Tests help you find issues in a system“Surface” issues

Profiling helps you find issues in the code

Unit Tests, Web Tests, Load TestsRepeatableGood for targeted tuning

System under “natural” loadNot repeatableLooking for rough patterns

Profiling Methods•Gathers information at intervals (clock cycles)•No code modifications•Use this for first pass explorationsSampling•Modifies your code•Gathers detailed timing and count information

Instrumentation

•Multi-threaded code profilingConcurrency

•Object creation•Garbage collection.NET Memory

•Interaction between your application and SQL Server via ADO.NET

Tier Interaction

Load Test Performance Sessions Via Wizard

Must be only Web Performance TestsCan’t profile via Load Test Rig

Let’s go profile some stuff.

The Plan.

1. Script user activity with Web Performance Tests2. Simulate a mix of users using Load Tests3. Simulate lots of users with a Load Test Rig4. Reproduce the errors...hopefully5. Find code problems with the Performance Wizard6. Fix the code

Any last questions?

For More Information: Articles

Uses Visual Studio 2012

http://tinyurl.com/8sdjjak

For More Information: Articles

Uses Visual Studio 2010

http://tinyurl.com/3xjqgvz

http://tinyurl.com/2ulvvvr

For More Information: Microsoft Links

Visual Studio Performance Testing Quick Reference Guidehttp://vsptqrg.codeplex.com/ Content Index for Web Tests & Load Testshttp://tinyurl.com/ye97e52

For More Information: Microsoft Bloggers

VS Team Test Blog Site http://blogs.msdn.com/b/vstsqualitytools/Ed Glas - http://blogs.msdn.com/edglas/Bill Barnett - http://blogs.msdn.com/billbar/Sean Lumley - http://blogs.msdn.com/slumley/Dennis Stone - http://blogs.msdn.com/densto/Mike Taute - http://blogs.msdn.com/mtaute/Neelesh Kamkolkar - http://blogs.msdn.com/b/nkamkolkar/

Thank you.

http://www.benday.com | benday@benday.com

© 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to

be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS

PRESENTATION.

Miscellaneous tips.

Include Load Tests In Your Nightly Build

Create a Test List (*.vsmdi)Include the Test List in the buildCaptures trend data at 24 hours intervalsCompare the load test data with Excel

Static Code Analysis / FxCop

Help you to find issues in your codeShow you IDispose errors

Rules are configurableUse your judgment about which are relevant for your team

Avoid Static Methods & CA1822

Static Code Analysis Rule CA1822 is evil.

Sacrifices maintainability for perfomance.

Blog post: “Static Methods Are A Code Smell”http://bit.ly/cnEsuV

Only optimize known performance problems.

You’d be surprised by what *ISN’T* a performance problemDon’t spend a ton of time coding fancy solutions to performance problems you *THINK* you might haveFind and fix *REAL* problems

Tip: WCF Message Size

Data structure formatting can make a huge differenceExample: Banks and the States they operate inOption 1: Dictionary<string, string>

Key = Bank IdValue = State Abbreviation

Option 2: Dictionary<string, string[]>Key = State AbbreviationValue = Array of Bank Id’s

Option 2 was approximately 90% smaller

Tip: Add Performance Counters

Create Performance Counters (perfmon) in your appDo this early in the development cycle

More directed profiling of your app during Load Tests

Good for managing your app when it’s in production(Operations people love this.)

Using Performance Sessions on x64

You can’t profile x64 IIS processes from the IDE

Option #1: Profile from the command lineStart ASP.NET profiling using VSPerfAspNetCmd.exe Run the web test

Option #2: Run the ASP.NET process in x86 mode

How To Run ASP.NET in x86 Mode

Set “Enable 32-bit Applications” to true on your application’s AppPool

Resources

Connect. Share. Discuss.

http://northamerica.msteched.com

Learning

Microsoft Certification & Training Resources

www.microsoft.com/learning

TechNet

Resources for IT Professionals

http://microsoft.com/technet

Resources for Developers

http://microsoft.com/msdn

Complete an evaluation on CommNet and enter to win!

MS Tag

Scan the Tagto evaluate thissession now onmyTechEd Mobile

Required Slide *delete this box when your slide is finalized

Your MS Tag will be inserted here during the final scrub.

© 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to

be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS

PRESENTATION.