julian harty - alternatives to testing - eurostar 2010
DESCRIPTION
EuroSTAR Software Testing Conference 2010 presentation on "Presentation Title" by "Speaker Name". See more at: http://conference.eurostarsoftwaretesting.com/past-presentations/TRANSCRIPT
1
Traditional Testing R.I.P.
Alternative Testing
Do We Have to Test Like We Always Have?
by Julian Harty
14 Nov 2010
2
Today's climate
We live in an era where quality has been replaced by speed - people are trained to expect little to none, and are perfectly happy getting it.
In such world, testing for quality must redefine itself and find a new purpose.
Goranka Bjedov (Google)
3
Cost of “doing the right thing”
“ the cost of "doing it right" is far higher than the cost of only responding when someone does find a problem”
John Viega: CTO McAfee: The myths of security
4
Business Reality...
Police, private security experts and internet service providers said on Wednesday they had shut down the Mariposa botnet , one of the largest networks of hacked personal computers discovered.
It had siphoned off financial and other information from inside half of the largest 1,000 US companies.
March 3rd, 2010http://www.ft.com/cms/s/0/f6960e5a-2711-11df-b84e-0 0144feabdc0.html
5
Are Testers impotent?
Why didn't any of the testers find the security flaw(s)?
What can we do to improve our testing competences?
6
Why test at all?
Testing slows projects down
It can mislead, or divert attention from other, more important work
Most testing is futile!
What's the risk of failure?
7
Some impacts of “testing”
Incorrect diagnosis
Incorrect changes
Irrelevant bugs
Still unfixed bugs
(what we know could hurt us...)
8
Testers as second-class citizens
“Anyone can test”
Therefore, one tester is as good as another, and no/little better than 'anyone'
Therefore, testers are nothing special...
9
Speed is the competitive advantage
Faster, iterative, releases
Testing for
vs Productivity Quality
10
What do you actually do?
(when you think you're testing...)
11
Checking vs. Testing
Checking Is Confirmation
Testing Is Exploration and Learning
Checks Are Machine-Decidable; Tests Require Sapience
http://www.developsense.com/blog/category/testing-v s-checking/
12
Some outputs of testing
Data
Error logs
Can we get these from other sources?
15
Use Canaries
What are they good for?
16
Canaries
What are they good for?
We expect some to die
Cheep(2010)
R.I.P.
17
Automate the Checking
When practical, create automated 'tests' that check:
Results
Data
Logs
The state of the software
18
Good uses of automated tests?
T.D.D.
Continuous Builds
Regression Tests
Security Tests
RED
GREENREFACTOR
19
Contributory Success Factors(for 'not' testing)
Quality and Speed of feedback
Perception of failure
Time to address and fix issues
Risk and cost of failure
The Art of Customer Relationships
20
What about the Testers?
“Traditional Testing” is waning
Change brings Opportunity
We can choose to apply our skills to more valuable & rewarding work
21
Now what?
What can we do with all the redundant testers?
22
Strengths of Testers
Recognizing there may be alternatives
Produces answers sooner
Reduce doubt / Increase confidence Advocate for: bugs, users, developers, software
Be Trusted Advisors
Debunk mysteries & superstitions
Test for productivity (better, faster, cheaper)
Test for quality (of the end product)
Be disruptive; yet likeable :)
23
Test to provide answers sooner
A good reason to test is to provide answers sooner
We need to decide:
What the aims are
What to measure, and how
How to run the test (environment)
Then we need to: run the tests, analyze the results, & present our assessment clearlySee: Gordon Bragg & Design of Experiments
24
Testing as a Leadership skill
Probe
Experiment
Execute
Decide
Move on...
25
What now?
Rethink what you do
Consider:
The value you offer
The value you deliver
Your competencies
How you work
Who you work for
Be willing to give up any “ kindergarten testing ”
26
Facets of Software Testing
0
2
4
6
8
10
Software design
Testing techniques
Test automation
Industry knowledge
Product knowledge
People skills
Ability to apply stuff
27
Improving in your current role
Sapient Testing:Engage your brain when testing software
http://www.satisfice.com/blog/archives/99
http://www.satisfice.com/blog/archives/358
Trinity Testing http://julianharty-softwaretesting.blogspot.com
Use automation appropriately
28
Honing our Testing Skills
Beta Testing for other software
“Be a better tester, be a Beta tester”
(Alan Richardson)
Work for a crowd-sourcing company
Mob4hire
uTest
Learn from others
Pairing
Exercises
29
Possible new roles?
Technical Due Diligence
Disruptive Agent of Change
Chief Engineer
Spy?
30
Read on... Gordon M. Bragg: Principles of Experimentation and Measurement
ISBN 0-13-701169-5
Lloyd Roden: Trusted Advisors
http://www.stickyminds.com/s.asp?F=S7849_CP_2
Alan Richardson: Be a better tester, be a Beta test er
http://www.compendiumdev.co.uk/eurostar2003/defa ult.php
Julian Harty: Trinity Testing
http://julianharty-softwaretesting.blogspot.com
Dr Stuart Reid + Julian Harty
Non Functional Testing (NFT) http://www.commercetes t.com/
P.T.M.M. http://julianharty-softwaretesting.blogsp ot.com
Tom Van Vleck: Three Questions About Each Bug You F ind
http://www.multicians.org/thvv/threeq.html