testing in dot net
TRANSCRIPT
-
8/14/2019 Testing in Dot Net
1/31
1
Automated Testing in
.NET:
What you need to know about
Integrating Testing &Development
Mary Romero SweeneySammamish Software Services
Copyright Sammamish Software Services 2008; All rights Reserved
-
8/14/2019 Testing in Dot Net
2/31
2
Former developer both embedded andbusiness systems for the Boeing
Company
Published Visual Basic for Testers
Apress Aug, 2001
Published A Testers Guide to .NETProgramming Apress, Feb 2006
Quick Bio
-
8/14/2019 Testing in Dot Net
3/31
3
Todays talk
This talk is notintended only for Microsoft
customers; we want to talk about a newapproach to test development
In this talk I will provide only an overview
of the features available in Visual StudiosTeam Test software in order to give you afeel for its power and capabilities forcomparison to your own effortsWe will notbe covering the entire feature set of
this new addition; this is not a Microsoft-sponsored presentation!
-
8/14/2019 Testing in Dot Net
4/31
4
Test Techniques Timeline
Note: All along many have had support from white box testers with codingskills enabling companies to create their own unique automated tests
Manual Testing
Automated Testing using3rd party tools
Automated TestingUsing Open Source Tools
Test and Development IntegrationInto Development platform software
-
8/14/2019 Testing in Dot Net
5/31
5
Traditional Automation Software vs
Integrated Dev and Automated Test
ApplicationSoftware
Automation
Software
Integrated
Development andAutomated Test
-
8/14/2019 Testing in Dot Net
6/316
Problems with the status quo
Tool bundlingThis occurs because vendor tools often dont have all the
capability needed
Decreases communication due to the lack of integration
Adding diverse tools from different makers makescommunication transfer difficult. Many vendors add to this
issue by acquiring new features that are just acquisitionof other companies tools
Expensive vendor tools vs. risky open source tools is a
tough choiceDo it yourself test harnessesLack of commonality and lack of enough available talent
-
8/14/2019 Testing in Dot Net
7/317
Microsofts Team Edition for Testers
With the release of Visual Studio 2005,Microsoft has integrated a significant amountof test capability into Visual Studio with itsVisual Studio Team Test software
The idea behind the Team Test software is
the integration of testing tools into thesoftware development environment.Developers create code and then create Unit Tests
and Test Engineers use these unit test shells tocreate more sophisticated system tests
OR, Testers can create the Unit Tests fromdevelopment code
-
8/14/2019 Testing in Dot Net
8/318
Cross Organization communication
Intent is to create a setting whereinformation can be shared across
allthe groups involved in softwaredevelopment.
Tests and their results can then beshared and managed across theorganizationfrom management to
the development and test teams,
and back again.
Test Results can be published by thesoftware into reports and graphs
Testdevelopment/
publication
Dev
Mgt
Test
-
8/14/2019 Testing in Dot Net
9/319
Team Edition for Testers
There are a variety of test typesto
address different organizationalprocesses and softwaredevelopment methodologies,
including Agile, Extreme, and TestFirst testing methods, as well as thetraditional software lifecycle model.
Types of Tests:
Web,
Load,
Ordered,
Manual,
Unit,
Generic
-
8/14/2019 Testing in Dot Net
10/3110
Test Types
Web:
Verify the functionality of web applications. Navigate through the site,clicking on different buttons, links, etc. and all web traffic gets saved andcan be played back
Load:
Take other tests (except manual tests) and run them simultaneouslyunder simulated load using virtual users. Allows you to set thresholdsand to see where performance points have failed. Resulting performancedata can be collected and reported.
Ordered: Place a set of existing tests in an ordered container.
Continued
-
8/14/2019 Testing in Dot Net
11/3111
Test Types, continued
Manual:Describe the steps for a tester to follow manually. Provides a
way to keep track of your manual tests along with the othertests in your project.
Unit:
Create code-generated tests for your components. You canalso create data-driven unit tests that use a database to readand store data.
Generic:
The generic test type allows you to take an existing programyou have and wrap it to function as a test that can beexecuted inside Visual Studio. Harness existing nUnit teststhis way.
-
8/14/2019 Testing in Dot Net
12/3112
Test Team support for Component Testing
But, Unit (component) testing is traditionallyconsidered the domain of the development
team!?
Not so in Agile and Test First methodologiesNo one is suggesting that developers stop doing
unit testing; however, a test professional can add
great value to the project by incorporating
professional test expertise into the unit testingprocess.
-
8/14/2019 Testing in Dot Net
13/3113
The Process Developer creates component Depending on methodology employed the
skeleton test harness for the component iscreated by either the developer or a tester
Test professional enhances the harness with
proper test expertise Including equivalence classing, boundary analysis, etc.
Test Team executes and provides results to
management and developer. Developer uses integrated environment to
directly and quickly address test issues
Warning! Code on next page..
-
8/14/2019 Testing in Dot Net
14/3114
Creating a Unit Test in an Integrated
EnvironmentHuman error
sometimes accountsfor missed methodsand classes
Code generation candevelop the skeletonof your unit tests
Namespaces
Classes
Methods
-
8/14/2019 Testing in Dot Net
15/3115
'''A test for NewInvoiceNumber()
''' _Public Sub NewInvoiceNumberTestMethod()Dim CustID As Integer = 0
Dim target As Invoice = New Invoice(CustID)Dim expected As Integer = 75Dim actual As Integeractual = target.NewInvoiceNumber
Assert.AreEqual(expected, actual, _"InvoiceLib.Invoice.NewInvoiceNumber did not return the
expected value.")End Sub
Generated Test Code
-
8/14/2019 Testing in Dot Net
16/3116
Whats the effect on best practices
for development and test? The developer often does not have the time and
certainly does not always have the correctmindset to perform thorough Quality Assurancetesting
In fact, in many organizations whats being done is morealong the lines of Unit Exercisingor simply ensuring basicfunctionality not full professional testing
Integration of test with development allows formore detail support from the test team and cross-department communication of metrics
Eases the introduction of test support earlier in thesoftware develo ment rocess
-
8/14/2019 Testing in Dot Net
17/31
17
Problems and advantages
Downside:
Only the VS Team Edition for SoftwareDevelopers (VSTESD) gives you the fullcapability including reporting
Expensive! Due to the cost, many .NETcustomer companies will stick with nUnit andother automated testing tool options
-
8/14/2019 Testing in Dot Net
18/31
18
Upside:
Gets test and development not just thinkingabout but doingtesting MUCH earlier in thesoftware process
Increases the role and capability of the TestEngineer
Provides far more visibility to management on
test development and progressProvides for more quick and direct reporting and
resolution for certain kinds of bugs
Problems and advantages
-
8/14/2019 Testing in Dot Net
19/31
19
How new integrated testing compares
to traditional and current practices
Traditional: black box & manual testing
If your organization separates test anddevelopment and provides little or no white box
test support this will involve a major culture change
Test First and Agile methods
If you are a Windows/Microsoft project then this
provides more specific and convenient supportIf you are notWindows/Microsoft, look for more
support along these lines from your own platform
vendors
-
8/14/2019 Testing in Dot Net
20/31
20
-
8/14/2019 Testing in Dot Net
21/31
21
Test Manager window
The Test Manager window allows you to organizeand manage your tests and test runs (suites)
-
8/14/2019 Testing in Dot Net
22/31
22
Test View window
Create, run, andorganize tests from
Test View
-
8/14/2019 Testing in Dot Net
23/31
23
Test Results Window
Test results window provides integratedresolution of test and the opportunity to view
more info, debug and rerun
-
8/14/2019 Testing in Dot Net
24/31
24
Code Coverage Support
Do you pay for a code coverage tool? Its built-inhere:
-
8/14/2019 Testing in Dot Net
25/31
25
Test Run Configuration window
Tests and test runs are configurable
-
8/14/2019 Testing in Dot Net
26/31
26
Load Test Settings
-
8/14/2019 Testing in Dot Net
27/31
27
Load Testing Results
-
8/14/2019 Testing in Dot Net
28/31
28
Summary/Conclusions
Now using .NET we can test using the
development software; this produces manyadvantages in cross-departmentcommunication development
Larger software development companies can
make effective use of these integrated toolsto save a lot of coding time for automatedtests in Visual Studio.
The expense of this Visual Studio versionmay not be necessary for smallerorganizations and is prohibitive for individualtesters who want to write quick and simpletests.
-
8/14/2019 Testing in Dot Net
29/31
29
Finally a word to the Microsoft users:
If you havent upgraded to VS TeamSystem, should you?If you do white box testing and coding
currently then this could save you hundredsof hours in test development
Product is fairly stable
If you dont do white box testing and coding
you need to analyze your current talent anddetermine if this is a good fit; it may be costprohibitive for anything but the larger shops -- at this time
Can you do without it? Sure.
-
8/14/2019 Testing in Dot Net
30/31
30
The future
Prediction: Integrated test and developmentplatforms will be a standard in the future
Remember when integrating a debugger into thedevelopment platform was a new thing? (OK Im old)
Well be saying the same thing about having code
coverage and test benches in the platform as well. Isnt the cost prohibitive? Not really compared to
purchasing multiple tools; the advantage over open
source is the same vendor/seamless integration.Like most things cost will go down (?)
How does this affect major test tool vendors? Not goodnews for them.
Are youready for this next phase?????
-
8/14/2019 Testing in Dot Net
31/31
Resources
Testers Guide to .NET Programming by Mary Romero Sweeney and Randal Root (Apress, Feb 2006)
Pro Visual Studio Team System by Jeff Levinson (Apress, May 2006)
Microsoft (of course) http://msdn.microsoft.com/vstudio/teamsystem
Lots of Users groups and white papers
Product comparison website: http://msdn.microsoft.com/vstudio/products/compare/default.aspx
http://msdn.microsoft.com/vstudio/teamsystemhttp://msdn.microsoft.com/vstudio/products/compare/default.aspxhttp://msdn.microsoft.com/vstudio/products/compare/default.aspxhttp://msdn.microsoft.com/vstudio/teamsystem