alpha and beta testing
Post on 05-Jul-2018
227 Views
Preview:
TRANSCRIPT
-
8/15/2019 Alpha and Beta Testing
1/72
Alpha and Beta TestingByrne Reese
October 2, 2006
-
8/15/2019 Alpha and Beta Testing
2/72
Product Manager, Six Apart
Founder, Test Run LLC
Open Source Developer
byrne@majordojo.com
http://www.majordojo.com/
AIM: byrnereese
Byrne Reese
-
8/15/2019 Alpha and Beta Testing
3/72
A test-centric development methodology.
Real world use cases.
What I am here to talk
about
-
8/15/2019 Alpha and Beta Testing
4/72
“Since you know version 1 is
going to suck , hurry up and
ship it so you can start
working on version 2.”
Google Proverb
-
8/15/2019 Alpha and Beta Testing
5/72
Alpha and Beta testing is notabout flushing out bugs…
-
8/15/2019 Alpha and Beta Testing
6/72
It’s about enabling afeedback loop.
-
8/15/2019 Alpha and Beta Testing
7/72
Stop thinking about
alpha and beta testingas phases in a
development cycle.
-
8/15/2019 Alpha and Beta Testing
8/72
Alpha and Beta Testing is an integral part
of emerging development methodologies.
And for that, you have to look at the
whole process.
-
8/15/2019 Alpha and Beta Testing
9/72
Phase OneRequirements
-
8/15/2019 Alpha and Beta Testing
10/72
Wait, wait, I forgot…
Before you begin you need to:a) define your target audience.
b) start collecting sign-ups.
-
8/15/2019 Alpha and Beta Testing
11/72
Phase OneRequirements
-
8/15/2019 Alpha and Beta Testing
12/72
Test on “paper.”Test on “paper.”
-
8/15/2019 Alpha and Beta Testing
13/72
Iterate on paper, whiteboards,
post it notes, index cards...Test team members’ ideas with
one another.
-
8/15/2019 Alpha and Beta Testing
14/72
And remember:
It doesn’t haveto look perfect
(It’s a frick’n wireframe)
-
8/15/2019 Alpha and Beta Testing
15/72
Then, as soon as you can…
-
8/15/2019 Alpha and Beta Testing
16/72
STOPtesting on paper!
-
8/15/2019 Alpha and Beta Testing
17/72
Don’t get trapped inwireframe-land.
-
8/15/2019 Alpha and Beta Testing
18/72
Phase Two
Development
-
8/15/2019 Alpha and Beta Testing
19/72
Create a tangible mock up.
Use what you create.
Watch others use what you create.
Learn from that experience.
-
8/15/2019 Alpha and Beta Testing
20/72
-
8/15/2019 Alpha and Beta Testing
21/72
Watching others…
-
8/15/2019 Alpha and Beta Testing
22/72
Select your subjects carefully. Do they align with
your target market?
Get an impartial party trained to presentoptions and collect feedback from subjects.
Record the session.
Then have the whole team watch it.
-
8/15/2019 Alpha and Beta Testing
23/72
Tools I Use:
Macromedia Breeze
Ethnio
-
8/15/2019 Alpha and Beta Testing
24/72
Establish a
development
rhythm early.
-
8/15/2019 Alpha and Beta Testing
25/72
Create an engineering pipeline.
Requirements
Release
Design
Requirements Design
Development Testing
Requirements
Development
-
8/15/2019 Alpha and Beta Testing
26/72
Phase Three
User Testing
-
8/15/2019 Alpha and Beta Testing
27/72
-
8/15/2019 Alpha and Beta Testing
28/72
Create as many open
communicationchannels as you can.
-
8/15/2019 Alpha and Beta Testing
29/72
-
8/15/2019 Alpha and Beta Testing
30/72
Create a discussion forum.Answer questions.
-
8/15/2019 Alpha and Beta Testing
31/72
Buy a help ticketing system.Build your knowledge base.
-
8/15/2019 Alpha and Beta Testing
32/72
-
8/15/2019 Alpha and Beta Testing
33/72
Talk to your customers.
-
8/15/2019 Alpha and Beta Testing
34/72
Listen to them too.
-
8/15/2019 Alpha and Beta Testing
35/72
-
8/15/2019 Alpha and Beta Testing
36/72
even if it is easy to do.
-
8/15/2019 Alpha and Beta Testing
37/72
But definitely do some of the
things they tell you.
Make your customers
feel heard.
-
8/15/2019 Alpha and Beta Testing
38/72
Make it crazy-easy
for customersto report bugs.
-
8/15/2019 Alpha and Beta Testing
39/72
Use automated error reports.
-
8/15/2019 Alpha and Beta Testing
40/72
-
8/15/2019 Alpha and Beta Testing
41/72
Log the error anyway.Send a daily report of all errors to the
entire team.
Meet daily to review open bugs and
to schedule their fix.
Schedule “bug-a-thons.”
-
8/15/2019 Alpha and Beta Testing
42/72
Don’t just record errors.
Record everything.
Understand your users’ behavior.
-
8/15/2019 Alpha and Beta Testing
43/72
-
8/15/2019 Alpha and Beta Testing
44/72
-
8/15/2019 Alpha and Beta Testing
45/72
Don’t forget: early beta testers are
perhaps your most important asset.
They will form the core
of your user base.
-
8/15/2019 Alpha and Beta Testing
46/72
Reward your betatesters.
-
8/15/2019 Alpha and Beta Testing
47/72
Praise them for helping.
Give them credit for their ideas.
Send them product discounts.
Buy them something off their wish list.
Hire some of them.
Release frequently.
-
8/15/2019 Alpha and Beta Testing
48/72
Lessons from
-
8/15/2019 Alpha and Beta Testing
49/72
Sandoz Laboratory produced a methodfor isolating the active ingredients of
medicinal plants in order to create
proper dosages for patients.
And now?
-
8/15/2019 Alpha and Beta Testing
50/72
-
8/15/2019 Alpha and Beta Testing
51/72
-
8/15/2019 Alpha and Beta Testing
52/72
The Internet was created to build acommunications network impervious
to nuclear assault.
And now?
“Tubes”
-
8/15/2019 Alpha and Beta Testing
53/72
-
8/15/2019 Alpha and Beta Testing
54/72
Ludicorp saw what users werereally using and adapted.
And now?
-
8/15/2019 Alpha and Beta Testing
55/72
Early on Friendster actively stoppedpeople from friending movies,
products and other “stuff.”
And now?
-
8/15/2019 Alpha and Beta Testing
56/72
96,697 friends and counting…
-
8/15/2019 Alpha and Beta Testing
57/72
Blogspot was originally a simplenews board in a larger project
management software suite.
And now?
-
8/15/2019 Alpha and Beta Testing
58/72
-
8/15/2019 Alpha and Beta Testing
59/72
After 6 months of beta testing, Swivelshut the product down and
went back to the drawing board.
And now?
-
8/15/2019 Alpha and Beta Testing
60/72
Summary
-
8/15/2019 Alpha and Beta Testing
61/72
The customer really is
always right.
-
8/15/2019 Alpha and Beta Testing
62/72
The customer really is
always right.(most of the time)
-
8/15/2019 Alpha and Beta Testing
63/72
Don’t be afraid tomake Hard Decisions.
-
8/15/2019 Alpha and Beta Testing
64/72
Alpha and Beta Testing
is not just aboutflushing out bugs…
-
8/15/2019 Alpha and Beta Testing
65/72
It’s abouttesting your ideas.
-
8/15/2019 Alpha and Beta Testing
66/72
It’s abouttesting your process.
-
8/15/2019 Alpha and Beta Testing
67/72
-
8/15/2019 Alpha and Beta Testing
68/72
And it’s aboutflushing out bugs.
-
8/15/2019 Alpha and Beta Testing
69/72
fin.
-
8/15/2019 Alpha and Beta Testing
70/72
The Slide Graveyard:
where slides go to die.
-
8/15/2019 Alpha and Beta Testing
71/72
Alpha and Beta testingare not the last phases of
development.
Requirements Development QARelease
alpha beta
-
8/15/2019 Alpha and Beta Testing
72/72
They are the last phase of every
phase of development.
Requirements Development User TestingRelease
alpha betaalpha betaalpha beta
top related