agile developers go testing part ii
DESCRIPTION
Agile Developers Go Testing Part II. Darcy Casselman @ flying_squirrel Alexei Zheglov @az1. November 2010. What are we talking about today?. A lot of things… They may seem disconnected, but they all: improve software quality involve testing done by developers - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Agile Developers Go Testing Part II](https://reader035.vdocument.in/reader035/viewer/2022062411/56816935550346895de091bf/html5/thumbnails/1.jpg)
Agile Developers Go TestingPart II
Darcy Casselman @flying_squirrelAlexei Zheglov @az1
November 2010
![Page 2: Agile Developers Go Testing Part II](https://reader035.vdocument.in/reader035/viewer/2022062411/56816935550346895de091bf/html5/thumbnails/2.jpg)
What are we talking about today?
• A lot of things…• They may seem disconnected, but they all:
– improve software quality– involve testing– done by developers
• None of them can do the job alone• We cannot:
– Talk about all of them– Give them equal time
![Page 3: Agile Developers Go Testing Part II](https://reader035.vdocument.in/reader035/viewer/2022062411/56816935550346895de091bf/html5/thumbnails/3.jpg)
Unit Testing
• We showed you a very simple unit test example earlier
• We’ll show you a more difficult example later• Let’s step back first
![Page 4: Agile Developers Go Testing Part II](https://reader035.vdocument.in/reader035/viewer/2022062411/56816935550346895de091bf/html5/thumbnails/4.jpg)
Focus on Quality
quotes
"...in 2000, during the dot-com bubble, ... software quality for North American teams... ranged from 6 defects per function point to less than 3 per 100 function point, a range of 200 to 1. The midpoint is approximately 1 defect per 0.6 to 1.0 function points. This implies that it is common for teams to spend more than 90 percent of their effort fixing defects.”
http://www.amazon.com/Kanban-David-J-Anderson/dp/0984521402/http://www.amazon.com/Software-Assessments-Benchmarks-Best-Practices/dp/0201485427/
![Page 5: Agile Developers Go Testing Part II](https://reader035.vdocument.in/reader035/viewer/2022062411/56816935550346895de091bf/html5/thumbnails/5.jpg)
Logical Connection?
“The midpoint is approximately 1 defect per 0.6 to 1.0 function points. This implies that it is common for teams to spend more than 90 percent of their effort fixing defects.”
How does he know what they are spending their time on?
![Page 6: Agile Developers Go Testing Part II](https://reader035.vdocument.in/reader035/viewer/2022062411/56816935550346895de091bf/html5/thumbnails/6.jpg)
Capacity Allocation To Fixing Defects
![Page 7: Agile Developers Go Testing Part II](https://reader035.vdocument.in/reader035/viewer/2022062411/56816935550346895de091bf/html5/thumbnails/7.jpg)
=
Capacity Allocation To Fixing Defects
Defect Injection Rate
![Page 8: Agile Developers Go Testing Part II](https://reader035.vdocument.in/reader035/viewer/2022062411/56816935550346895de091bf/html5/thumbnails/8.jpg)
Bugs, Features
http://www.flickr.com/photos/dratz/1045336659/
![Page 9: Agile Developers Go Testing Part II](https://reader035.vdocument.in/reader035/viewer/2022062411/56816935550346895de091bf/html5/thumbnails/9.jpg)
Another View
• Root-cause analysis of established practices
• E.g. unit testing• Five-whys applied to
the solution, not the problem
• The link between the third and fourth whys?
Source: Karl Scotland http://availagility.co.uk/2010/10/04/a-root-cause-analysis-of-agile-practices/
![Page 10: Agile Developers Go Testing Part II](https://reader035.vdocument.in/reader035/viewer/2022062411/56816935550346895de091bf/html5/thumbnails/10.jpg)
Compare Two Teams
Team A• Just finished 40 “features”
(units of work)• Introduced 10 “bugs” (units
of rework)
Team B• 10 features• 8 bugs
![Page 11: Agile Developers Go Testing Part II](https://reader035.vdocument.in/reader035/viewer/2022062411/56816935550346895de091bf/html5/thumbnails/11.jpg)
Team A Progression
• Terms beyond the first three are negligible• Most defects eliminated in the next 2 releases• The 2nd maintenance release scope is very small• Illusion that any team can do it• Time spent fixing defects: 25%
3153...5.21040
![Page 12: Agile Developers Go Testing Part II](https://reader035.vdocument.in/reader035/viewer/2022062411/56816935550346895de091bf/html5/thumbnails/12.jpg)
Team B Progression
• The sum of the first 3 terms: only 24.4 (less than half of the total!)
• The sum of the first 10: approx. 45• Time spent fixing defects: 80%
50...12.54.6810
![Page 13: Agile Developers Go Testing Part II](https://reader035.vdocument.in/reader035/viewer/2022062411/56816935550346895de091bf/html5/thumbnails/13.jpg)
We want to be more like Team A
and certainly not like Team B
(or better)
![Page 14: Agile Developers Go Testing Part II](https://reader035.vdocument.in/reader035/viewer/2022062411/56816935550346895de091bf/html5/thumbnails/14.jpg)
Testing
• Is what we are talking about today• Leaving out other technical practices• Remembering that testing itself does not
provide quality
![Page 15: Agile Developers Go Testing Part II](https://reader035.vdocument.in/reader035/viewer/2022062411/56816935550346895de091bf/html5/thumbnails/15.jpg)
![Page 16: Agile Developers Go Testing Part II](https://reader035.vdocument.in/reader035/viewer/2022062411/56816935550346895de091bf/html5/thumbnails/16.jpg)
Test MatrixWhy?
Unit Did we build the code right? Does it work as specified?
![Page 17: Agile Developers Go Testing Part II](https://reader035.vdocument.in/reader035/viewer/2022062411/56816935550346895de091bf/html5/thumbnails/17.jpg)
Test MatrixWhy?
Unit Did we build the code right? Does it work as specified?
Acceptance Did we build the right code? Does it do what the user wants?
![Page 18: Agile Developers Go Testing Part II](https://reader035.vdocument.in/reader035/viewer/2022062411/56816935550346895de091bf/html5/thumbnails/18.jpg)
Test MatrixWhy?
Unit Did we build the code right? Does it work as specified?
Acceptance Did we build the right code? Does it do what the user wants?
Integration Do system components work together? What about performance?
![Page 19: Agile Developers Go Testing Part II](https://reader035.vdocument.in/reader035/viewer/2022062411/56816935550346895de091bf/html5/thumbnails/19.jpg)
Test MatrixWhy?
Unit Did we build the code right? Does it work as specified?
Acceptance Did we build the right code? Does it do what the user wants?
Integration Do system components work together? What about performance?
System Is the system installed and configured correctly?
![Page 20: Agile Developers Go Testing Part II](https://reader035.vdocument.in/reader035/viewer/2022062411/56816935550346895de091bf/html5/thumbnails/20.jpg)
Test MatrixWhy?
Unit Did we build the code right? Does it work as specified?
Acceptance Did we build the right code? Does it do what the user wants?
Integration Do system components work together? What about performance?
System Is the system installed and configured correctly?
Exploratory What can we find out about the system that is not specified by these tests?
![Page 21: Agile Developers Go Testing Part II](https://reader035.vdocument.in/reader035/viewer/2022062411/56816935550346895de091bf/html5/thumbnails/21.jpg)
Test MatrixCoverage
%Auto
?Tools Developers’ Role
Unit
Acceptance
Integration
System
Exploratory
![Page 22: Agile Developers Go Testing Part II](https://reader035.vdocument.in/reader035/viewer/2022062411/56816935550346895de091bf/html5/thumbnails/22.jpg)
Test Matrix
Sources: Robert C. Martin, Rob Diana http://regulargeek.com/2009/06/12/how-much-testing-is-enough/
Coverage %
Auto?
Tools Developers’ Role
Unit Yes NUnit/JUnit and similar toolsisolation (mocking) frameworks
Acceptance Yes acceptance testing frameworks (Cucumber, FitNesse, Robot etc.)domain-specific languages (DSLs)
Integration Yes UI testing frameworks (e.g. Selenium)
System Yes same
Exploratory No -
![Page 23: Agile Developers Go Testing Part II](https://reader035.vdocument.in/reader035/viewer/2022062411/56816935550346895de091bf/html5/thumbnails/23.jpg)
Test Matrix
Sources: Robert C. Martin, Rob Diana http://regulargeek.com/2009/06/12/how-much-testing-is-enough/
Coverage %
Auto?
Tools Developers’ Role
Unit 90% Yes NUnit/JUnit and similar toolsisolation (mocking) frameworks
Acceptance 50% Yes acceptance testing frameworks (Cucumber, FitNesse, Robot etc.)domain-specific languages (DSLs)
Integration 20% Yes UI testing frameworks (e.g. Selenium)
System 5% Yes same
Exploratory - No -
![Page 24: Agile Developers Go Testing Part II](https://reader035.vdocument.in/reader035/viewer/2022062411/56816935550346895de091bf/html5/thumbnails/24.jpg)
Test Matrix
Sources: Robert C. Martin, Rob Diana http://regulargeek.com/2009/06/12/how-much-testing-is-enough/
Coverage %
Auto?
Tools Developers’ Role
Unit 90% Yes NUnit/JUnit and similar toolsisolation (mocking) frameworks
take personal responsibility
Acceptance 50% Yes acceptance testing frameworks (Cucumber, FitNesse, Robot etc.)domain-specific languages (DSLs)
collaborateIntegration 20% Yes UI testing frameworks (e.g. Selenium)
System 5% Yes same
Exploratory - No - -
![Page 25: Agile Developers Go Testing Part II](https://reader035.vdocument.in/reader035/viewer/2022062411/56816935550346895de091bf/html5/thumbnails/25.jpg)
A Very Short Introduction to Acceptance Testing
![Page 26: Agile Developers Go Testing Part II](https://reader035.vdocument.in/reader035/viewer/2022062411/56816935550346895de091bf/html5/thumbnails/26.jpg)
Consider This Test Script
• A new customer registers• The customer places three books in the
shopping cart• The customer goes to check-out• The customer fills in their delivery address
(province: Ontario)• The system offers free delivery to the customer
Source: Gojko Adzic. Bridging the Communication Gap, pp. 78-79http://www.amazon.com/gp/product/0955683610
![Page 27: Agile Developers Go Testing Part II](https://reader035.vdocument.in/reader035/viewer/2022062411/56816935550346895de091bf/html5/thumbnails/27.jpg)
Compare the Test Script to This
The system offers free delivery to Ontario customers when they place their first order and their order contains at least three books
![Page 28: Agile Developers Go Testing Part II](https://reader035.vdocument.in/reader035/viewer/2022062411/56816935550346895de091bf/html5/thumbnails/28.jpg)
Convert to the Tabular Form
number of books order number ship to free delivery?
3 1 Ontario Yes
![Page 29: Agile Developers Go Testing Part II](https://reader035.vdocument.in/reader035/viewer/2022062411/56816935550346895de091bf/html5/thumbnails/29.jpg)
number of books order number ship to free delivery?
3 1 Ontario Yes
2 1 Ontario No
![Page 30: Agile Developers Go Testing Part II](https://reader035.vdocument.in/reader035/viewer/2022062411/56816935550346895de091bf/html5/thumbnails/30.jpg)
number of books order number ship to free delivery?
3 1 Ontario Yes
2 1 Ontario No
3 2 Ontario No
![Page 31: Agile Developers Go Testing Part II](https://reader035.vdocument.in/reader035/viewer/2022062411/56816935550346895de091bf/html5/thumbnails/31.jpg)
number of books order number ship to free delivery?
3 1 Ontario Yes
2 1 Ontario No
3 2 Ontario No
3 1 Alberta No
![Page 32: Agile Developers Go Testing Part II](https://reader035.vdocument.in/reader035/viewer/2022062411/56816935550346895de091bf/html5/thumbnails/32.jpg)
How To Do It
• Collaborate:– developers– business analysts– testers– product owner
• Which tool to use is not as important• Don’t look up terminology in Wikipedia
![Page 33: Agile Developers Go Testing Part II](https://reader035.vdocument.in/reader035/viewer/2022062411/56816935550346895de091bf/html5/thumbnails/33.jpg)
Books
http://www.amazon.com/gp/product/0955683610http://manning.com/adzic/
![Page 34: Agile Developers Go Testing Part II](https://reader035.vdocument.in/reader035/viewer/2022062411/56816935550346895de091bf/html5/thumbnails/34.jpg)
Unit-TestingWhen System Under Test (SUT)
Has Dependencies
Break the dependencies
![Page 35: Agile Developers Go Testing Part II](https://reader035.vdocument.in/reader035/viewer/2022062411/56816935550346895de091bf/html5/thumbnails/35.jpg)
What If We Don’t?
• Configuration• Database data• Remote services• Permissions• Time to run• Infrequent errors• Dependencies in development
![Page 36: Agile Developers Go Testing Part II](https://reader035.vdocument.in/reader035/viewer/2022062411/56816935550346895de091bf/html5/thumbnails/36.jpg)
Can You Run Unit Tests Like This?
http://www.flickr.com/photos/theredproject/2878006012/
![Page 37: Agile Developers Go Testing Part II](https://reader035.vdocument.in/reader035/viewer/2022062411/56816935550346895de091bf/html5/thumbnails/37.jpg)
Code Example
NotificationService
StudentRepositoryCaller
GradingEngine
![Page 38: Agile Developers Go Testing Part II](https://reader035.vdocument.in/reader035/viewer/2022062411/56816935550346895de091bf/html5/thumbnails/38.jpg)
Code Example
Mock
StubUnit Test
System Under Test
![Page 39: Agile Developers Go Testing Part II](https://reader035.vdocument.in/reader035/viewer/2022062411/56816935550346895de091bf/html5/thumbnails/39.jpg)
(Open IDE)
![Page 40: Agile Developers Go Testing Part II](https://reader035.vdocument.in/reader035/viewer/2022062411/56816935550346895de091bf/html5/thumbnails/40.jpg)
37.62%
34.42%
9.01%
4.05%
3.12%
1.99%
1.77%
1.21%
0.92%
0.57%
3.97% 1.35%
Popular Frameworks (.NET)
MoqRhinoMocksHand-rolledTypemock IsolatorMolesJustMockNSubstituteNMock2NMockFakeItEasyNoneOther
Source http://www.osherove.com/blog/2010/9/10/2010-poll-which-isolation-framework-do-you-use-in-net.html September 2010
![Page 41: Agile Developers Go Testing Part II](https://reader035.vdocument.in/reader035/viewer/2022062411/56816935550346895de091bf/html5/thumbnails/41.jpg)
Two Camps
Design for Testability:Moq, RhinoMocks, etc.
• Dynamically fake objects implementing interfaces/base classes in their own way
• Require software under test to be designed to be testable
• Programmers to follow SOLID principles (esp. Dependency Inversion Principle)
Mock Anything, e.g.Typemock Isolator, JustMock
• Rely on Profiler API• Can intercept anything and
run something else instead• Can fake statics, sealed
classes, etc.• Don’t force design for
testability
![Page 42: Agile Developers Go Testing Part II](https://reader035.vdocument.in/reader035/viewer/2022062411/56816935550346895de091bf/html5/thumbnails/42.jpg)
Popular Frameworks (Java)
• “Conventional”:– Mockito http://code.google.com/p/mockito/– EasyMock http://easymock.org/– JMock http://www.jmock.org/
• Alternative:– JMockit http://code.google.com/p/jmockit/
![Page 43: Agile Developers Go Testing Part II](https://reader035.vdocument.in/reader035/viewer/2022062411/56816935550346895de091bf/html5/thumbnails/43.jpg)
Roy Osherove
• The Chief Architect at TypeMock• Wrote: “A customer switches to Moq – and
I’m happy”• And: “This is pure awesome. This is why I
come to work every day.”• How is it possible?• Read it:
http://www.osherove.com/blog/2010/9/29/a-customer-switches-to-moq-and-im-happy.html
![Page 44: Agile Developers Go Testing Part II](https://reader035.vdocument.in/reader035/viewer/2022062411/56816935550346895de091bf/html5/thumbnails/44.jpg)
Back to Our Code Example
• Grading engine operates on Student and Grade
• But Student has a Department• And Grade has a Course and Course has a
Semester• We start mocking• Soon, we have a lot of mocks
![Page 45: Agile Developers Go Testing Part II](https://reader035.vdocument.in/reader035/viewer/2022062411/56816935550346895de091bf/html5/thumbnails/45.jpg)
“Every time a mock returns another mock a fairy dies”
• Who said it?• Not the guy whose tweet comes up first in
Google search results• I am not 100% sure• What the heck does it mean anyway?
![Page 46: Agile Developers Go Testing Part II](https://reader035.vdocument.in/reader035/viewer/2022062411/56816935550346895de091bf/html5/thumbnails/46.jpg)
“Every time a mock returns another mock a fairy dies”
• Who said it?• Not the guy whose tweet comes up first in
Google search results• I am not 100% sure• What the heck does it mean anyway?• We don’t have time for it in this presentation
![Page 47: Agile Developers Go Testing Part II](https://reader035.vdocument.in/reader035/viewer/2022062411/56816935550346895de091bf/html5/thumbnails/47.jpg)
http://www.amazon.com/Growing-Object-Oriented-Software-Guided-Tests/dp/0321503627/
![Page 48: Agile Developers Go Testing Part II](https://reader035.vdocument.in/reader035/viewer/2022062411/56816935550346895de091bf/html5/thumbnails/48.jpg)
Creating Objects
• Constructor injection (as in our code example)• Factory• What if dependencies have dependencies?• What if dependencies of dependencies have
dependencies?
![Page 49: Agile Developers Go Testing Part II](https://reader035.vdocument.in/reader035/viewer/2022062411/56816935550346895de091bf/html5/thumbnails/49.jpg)
Pretty Soon, Things Begin To Look Like This
Source: http://ayende.com/Blog/archive/2010/01/22/rejecting-dependency-injection-inversion.aspx
(different box color = different life span)
(and this is a simple example)
![Page 50: Agile Developers Go Testing Part II](https://reader035.vdocument.in/reader035/viewer/2022062411/56816935550346895de091bf/html5/thumbnails/50.jpg)
These Tools Can Help
• StructureMap• Castle Windsor• Unity• NInject• autofac
![Page 51: Agile Developers Go Testing Part II](https://reader035.vdocument.in/reader035/viewer/2022062411/56816935550346895de091bf/html5/thumbnails/51.jpg)
Conclusions
![Page 52: Agile Developers Go Testing Part II](https://reader035.vdocument.in/reader035/viewer/2022062411/56816935550346895de091bf/html5/thumbnails/52.jpg)
Quality: More Important Than This
http://www.flickr.com/photos/markhillary/2681349855/
![Page 53: Agile Developers Go Testing Part II](https://reader035.vdocument.in/reader035/viewer/2022062411/56816935550346895de091bf/html5/thumbnails/53.jpg)
More Conclusions
• You have to test• No single testing strategy is nearly enough• Employ multiple strategies• Each of them is difficult to master• We barely scratched the surface today• Agile brings testing into focus• Combine hard and soft skills• Collaborate
![Page 54: Agile Developers Go Testing Part II](https://reader035.vdocument.in/reader035/viewer/2022062411/56816935550346895de091bf/html5/thumbnails/54.jpg)
Q&A