© 2012 autodesk automated testing with the autocad ®.net api scott mcfarlane senior software...
TRANSCRIPT
© 2012 Autodesk
Automated Testing with the AutoCAD® .NET API
Scott McFarlaneSenior Software Engineer, Woolpert, Inc.
© 2012 Autodesk
Learning Objectives
At the end of this class, you will be able to: Recognize the obstacles to doing software testing, particularly with
programs written for AutoCAD Understand the goals, philosophy and principles of test automation Appreciate the benefits of test automation Use the correct Terminology: What is a mock, a stub, a fake, a dummy? Use abstraction and dependency inversion to write testable code
© 2012 Autodesk
Automated Testing
© 2012 Autodesk
Why Am I Not Doing Automated Testing?
My programs always work correctly I never need to modify or refactor my code I enjoy spending hours tracking down bugs I haven’t yet realized the benefits I haven’t learned to do it well
© 2012 Autodesk
Code-Driven Testing Frameworks
The growing family of code-driven testing frameworks have come to be known collectively as xUnit
Based on a design by Kent Beck, originally implemented for Smalltalk as SUnit, then ported to Java as JUnit.
Wikipedia lists over 360 testing frameworks for over 70 different languages and environments.
© 2012 Autodesk
Basic xUnit Architecture
Test Case Also called Testcase Class Implemented as an interface, abstract base class, or class attribute Contains one or more Test Methods Typically shares the same Test Fixture
Test Method Contains the logic for each test
© 2012 Autodesk
Basic xUnit Architecture
Test Suite A set of tests that share the same Test Fixture
SUT (System Under Test) The actual class/method we are testing
Test Fixture All the things we need to have in place to run a test
4-Step Test Execution 1. Setup 2. Exercise 3. Verify 4. Teardown
© 2012 Autodesk
Topics
The Goals of Test Automation Philosophy and Principles of Test Automation Terminology: What is a mock, a stub, a fake, a dummy?
© 2012 Autodesk
© 2012 Autodesk
Goals of Test Automation
Tests should help us improve quality Tests should help us understand the system under test (SUT) Tests should reduce (and -not introduce) risk Tests should be easy to run Tests should be easy to write and maintain Tests should require minimal maintenance as the system evolves
around them
© 2012 Autodesk
Tests Should Help Us Improve Quality
What is Quality? 1. Is the software built correctly? 2. Have we built the correct software?
Tests act as an executable specification Tests help prevent bugs Tests help localize bugs But… Quality is not just testing
“Trying to improve the quality of software by doing more testingis like trying to lose weight by weighing yourself more often.”
- Steve McConnell
© 2012 Autodesk
Tests Should Help Us Understand the SUT
Rather than pouring over the code to answer: What is this code supposed to do? What would the result be if . . . ?
Tests should show the test reader how the code is supposed to work. Tests are less likely to become out-of-sync with code than
documentation
© 2012 Autodesk
Tests Should Reduce (Not Introduce) Risk
Tests provide a safety net for incrementaldevelopment and refactoring Without tests, we are reluctant to make
changes or try different approaches – risk is too high. Writing good tests is not easy – It takes lots of practice
Keep tests simple, clean, easy to read Don’t be afraid to use Test Utility Methods.
Dangers (Risks) of Automated Testing Putting test logic in production code Testing too many things at once Testing too little
© 2012 Autodesk
Tests Should Be Easy to Run
Tests must be… Fully automated Self-checking Independent Repeatable Fast
© 2012 Autodesk
Tests Should Require Minimal Maintenance as the System Evolves Around Them Test only public interfaces and outcomes when possible Avoid testing private methods directly (implementation details) Use Mocks only when necessary Minimize duplicate code in tests
Tests should facilitate (not hinder) our ability to make changes to the production code.
© 2012 Autodesk
Summary of Goals
Trustworthiness Are you able to run your test and trust the results?
Maintainability Treat your test code with the same care as your production code.
Readability Can you look at a test and quickly understand what is being tested and why it
failed?
© 2012 Autodesk
Testing Philosophy
Test first versus test after Test-by-test versus test all-at-once Outside-in versus inside-out Behavior verification versus state verification
© 2012 Autodesk
Definitions (Meszaros)
Test Double – Any object that is used in place of a real object for the purposes of testing. Mocks, stubs, fakes and dummies are all test doubles. Mock – an object that can monitor the indirect outputs (behavior) of the SUT. Your
tests assert against the mock object (rather than the SUT itself) to see if it has passed.
Stub – an object that can control indirect inputs into the SUT. A test can use multiple subs, but can only have one mock object.
Fake – an object used by the SUT that behaves exactly like the real object, but may be used to isolate the SUT from external resources. An in-memory database is an example of a Test Double.
Dummy – an object that is required by the SUT to run, but plays no active role in the test.
© 2012 Autodesk
Behavior Verification vs. State Verification
Behavior Verification State Verification
Class under test Mock
Test
Communicate
Assert
Class under test Stub
Test
Communicate
Asser
t
DependentComponent
DependentComponent
© 2012 Autodesk
Unit Tests vs. Integration Tests
Unit Tests Should run w/o config Isolated No teardown required Run in memory Fast Repeatable Run often
Integration Tests May require config Use external resources May require teardown May use disk or database Slow May not be repeatable Run less often
© 2012 Autodesk
Unit Tests vs. Integration Tests
Do not mix - Keep in separate projects/assemblies
© 2012 Autodesk
Unit Testing “Thorns”
MessageBox.Show() Timer WebClient System.IO (File, etc.) DateTime.Now Singletons/Statics
© 2012 Autodesk
Testing Tools
Testing Framework (xUnit) MSTest NUnit MbUnit/Gallio
Isolation Framework NSubstitute Moq FakeItEasy
Other Fluent Assertions NCrunch
© 2012 Autodesk
Obstacles to Testing AutoCAD Code
© 2012 Autodesk
Obstacles to Testing AutoCAD Code
Our programs are hosted by AutoCAD Heavy user interaction Tightly coupled with AutoCAD API
© 2012 Autodesk
Example
© 2012 Autodesk
Abstraction and Dependency Injection
© 2012 Autodesk
Why is Abstraction Important?
The Problem Classes often contain dependencies on other classes to do their work. If a class makes assumptions about how its dependent classes are
implemented, the class becomes difficult to reuse in combination with other components.
Such classes are also very difficult to unit test because they cannot be isolated from their dependencies.
© 2012 Autodesk
Why is Abstraction Important?
The Solution Establish a common set of protocols by which classes interact, separately
from the classes themselves. Promotes the use of software design patterns (particularly dependency
injection) that result in code that is more testable, extensible, maintainable, scalable, and reusable.
© 2012 Autodesk
Typical Application
Business Logic
Logger
FILE
User Interface
Exception Handler
Data Access
Web Service Client
Component
Component
Configuration
© 2012 Autodesk
Business Logic
User Interface
Logger
FILE
Exception Handler
Data Access
Web Service Client
Component
Component
Configuration
IDataRepository
ILogger
IMessageBox
IProgressBar
IExceptionHandler
ISomeWebService
Interface
Interface
IConfiguration
IHostApplication
Host Application
Typical Application
© 2012 Autodesk
Business Logic
Test Double
Test Double
Test Double
Test Double
Test Double
Test Double
Test Double
Test Double
IDataRepository
ILogger
IMessageBox
IProgressBar
IExceptionHandler
ISomeWebService
Interface
Interface
IConfiguration
Test DoubleIHostApplication
Typical Application
© 2012 Autodesk
Example 1public class Example1{
public void DoTheWork(){
DataRepository dataRepository = new DataRepository();Logger logger = new Logger();
logger.Log("Getting the data");
DataSet theData = dataRepository.GetSomeData();
// Do some work with the data...
logger.Log("Done.");}
}
© 2012 Autodesk
Example 2public class Example2{
private readonly IDataRepository _dataRepository;private readonly ILogger _logger;
public Example2(IDataRepository dataRepository, ILogger logger){
_dataRepository = dataRepository;_logger = logger;
}
public void DoTheWork(){
_logger.Log("Getting the data");
DataSet theData = _dataRepository.GetSomeData();
// Do some work with the data...
_logger.Log("Done.");}
}
© 2012 Autodesk
Software Engineering Principles
Separation of Concerns – This class is now only responsible for the specific job it was designed to do.
Abstraction – By using interfaces, we have established a set of protocols by which the components interact, separately from the classes themselves.
Inversion of Control – The class has relinquished control of the creation and initialization of its dependencies.
Dependency Injection – This pattern is based on Inversion of Control, and describes the way in which an object obtains references to its dependencies.
© 2012 Autodesk
Application Architecture
AutoCADMain Plug-
in Code (Bootstrapper)
AutoCAD Specific
LogicBusiness Logic
AcDbMgd.dllAcCoreMgd.dll
AcMgd.dll
Shared Abstraction
s
User Interface
Other Dependencies
LoggingConfigurationDatabasesWeb ServicesEtc…
Depends on AutoCAD API
No dependency on AutoCAD API
InterfacesDomain Objects
Implements
AcCoreConsole.exe
Implem
ents
Unit Tests
© 2012 Autodesk
Example
© 2012 Autodesk
Autodesk, AutoCAD* [*if/when mentioned in the pertinent material, followed by an alphabetical list of all other trademarks mentioned in the material] are registered trademarks or trademarks of Autodesk, Inc., and/or its subsidiaries and/or affiliates in the USA and/or other countries. All other brand names, product names, or trademarks belong to their respective holders. Autodesk reserves the right to alter product and services offerings, and specifications and pricing at any time without notice, and is not responsible for typographical or graphical errors that may appear in this document. © 2012 Autodesk, Inc. All rights reserved.