validation missteps making us full time firefighters
TRANSCRIPT
![Page 1: Validation Missteps Making Us Full Time Firefighters](https://reader038.vdocument.in/reader038/viewer/2022110310/55953b751a28aba1098b4688/html5/thumbnails/1.jpg)
Presented to Silicon Valley DVClub January 26th, 2010 Allison Goodman, [email protected]
![Page 2: Validation Missteps Making Us Full Time Firefighters](https://reader038.vdocument.in/reader038/viewer/2022110310/55953b751a28aba1098b4688/html5/thumbnails/2.jpg)
My Background
Products that survived the fires to market (some minus a few features)
Products were consumed by fires late in development
4
SSD
3
![Page 3: Validation Missteps Making Us Full Time Firefighters](https://reader038.vdocument.in/reader038/viewer/2022110310/55953b751a28aba1098b4688/html5/thumbnails/3.jpg)
What missteps along the way turn us into firefighters?
Insufficient Planning Not Designing for Test Not creating and integrating
Feedback Loops Lack of Transparency
![Page 4: Validation Missteps Making Us Full Time Firefighters](https://reader038.vdocument.in/reader038/viewer/2022110310/55953b751a28aba1098b4688/html5/thumbnails/4.jpg)
![Page 5: Validation Missteps Making Us Full Time Firefighters](https://reader038.vdocument.in/reader038/viewer/2022110310/55953b751a28aba1098b4688/html5/thumbnails/5.jpg)
Insufficient Planning- Don’t have what you need to test Original Test Coverage Not Enough
Undocumented Assumptions
Missed Dependencies
SCOPE
Scope
Lack of Coverage
![Page 6: Validation Missteps Making Us Full Time Firefighters](https://reader038.vdocument.in/reader038/viewer/2022110310/55953b751a28aba1098b4688/html5/thumbnails/6.jpg)
If you don’t plan for it…it will surprise you.
Every surprise will end up as a fire …
As you get closer to trying to ship a product you become a full time firefighter.
![Page 7: Validation Missteps Making Us Full Time Firefighters](https://reader038.vdocument.in/reader038/viewer/2022110310/55953b751a28aba1098b4688/html5/thumbnails/7.jpg)
Plan! (in writing!)
Take the time to write it all down -who does what (owners!) -how does a feature really work -what “done” is -checkpoints along the way -success criteria
Keep track of the assumptions
And what everyone really needs
![Page 8: Validation Missteps Making Us Full Time Firefighters](https://reader038.vdocument.in/reader038/viewer/2022110310/55953b751a28aba1098b4688/html5/thumbnails/8.jpg)
test
![Page 9: Validation Missteps Making Us Full Time Firefighters](https://reader038.vdocument.in/reader038/viewer/2022110310/55953b751a28aba1098b4688/html5/thumbnails/9.jpg)
Not Designing for Test Very difficult to repeat tests for debug Waste time
My Design is Perfect No plan tests- late implementation
Pass? Fail? Who knows?
Getting lucky vs knowing it works
Getting test to fail again
![Page 10: Validation Missteps Making Us Full Time Firefighters](https://reader038.vdocument.in/reader038/viewer/2022110310/55953b751a28aba1098b4688/html5/thumbnails/10.jpg)
If you don’t design for test, you won’t find or replicate issues … they will surprise you.
Every surprise will end up as a fire …
As you get closer to trying to ship a product you become a full time firefighter.
![Page 11: Validation Missteps Making Us Full Time Firefighters](https://reader038.vdocument.in/reader038/viewer/2022110310/55953b751a28aba1098b4688/html5/thumbnails/11.jpg)
Design for Testing!
Hooks & Bread Crumbs • Where you are (state) • How you got there • How to get back
Ability to Artificially inject fails
Figure out coverage
Define Pass & Fail with Team Partner with Architects & Design, you are helpful!
![Page 12: Validation Missteps Making Us Full Time Firefighters](https://reader038.vdocument.in/reader038/viewer/2022110310/55953b751a28aba1098b4688/html5/thumbnails/12.jpg)
tests?
![Page 13: Validation Missteps Making Us Full Time Firefighters](https://reader038.vdocument.in/reader038/viewer/2022110310/55953b751a28aba1098b4688/html5/thumbnails/13.jpg)
Not creating & integrating Feedback Loops-
Planned Testing
New Market / Customer Requirements
Planned Testing Stale
Testing
Hardware Software Firmware Marketing
Working in Silos Implementation != Documentation
![Page 14: Validation Missteps Making Us Full Time Firefighters](https://reader038.vdocument.in/reader038/viewer/2022110310/55953b751a28aba1098b4688/html5/thumbnails/14.jpg)
If you don’t create and implement feedback loops, you’ll miss issues…they will surprise you.
Every surprise will end up as a fire …
As you get closer to trying to ship a product you become a full time firefighter.
![Page 15: Validation Missteps Making Us Full Time Firefighters](https://reader038.vdocument.in/reader038/viewer/2022110310/55953b751a28aba1098b4688/html5/thumbnails/15.jpg)
Create & integrate Feedback Loops
Validation Team
Tight & Ongoing Feedback -Problems found on current product -new info about system interactions -actual implementation changes
Hardware Software Firmware Marketing
Planned Testing
New Market Requirements
Additional Testing
Adaptable to new requirements/usages
Is it Effective? Efficient?
Change it!
Run Test
![Page 16: Validation Missteps Making Us Full Time Firefighters](https://reader038.vdocument.in/reader038/viewer/2022110310/55953b751a28aba1098b4688/html5/thumbnails/16.jpg)
![Page 17: Validation Missteps Making Us Full Time Firefighters](https://reader038.vdocument.in/reader038/viewer/2022110310/55953b751a28aba1098b4688/html5/thumbnails/17.jpg)
Lack of Transparency
Team telling mgmt…or worse each other, it’s all fine
What you feel is really happening
DONE!
Schedules run late, validation is squeezed
Fewer iterations, narrower
coverage, less thorough
investigations
![Page 18: Validation Missteps Making Us Full Time Firefighters](https://reader038.vdocument.in/reader038/viewer/2022110310/55953b751a28aba1098b4688/html5/thumbnails/18.jpg)
If your team is not transparent, you’ll ignore risks and coverage gaps…escapes will surprise you.
Every surprise will end up as a fire …
As you get closer to trying to ship a product you become a full time firefighter.
![Page 19: Validation Missteps Making Us Full Time Firefighters](https://reader038.vdocument.in/reader038/viewer/2022110310/55953b751a28aba1098b4688/html5/thumbnails/19.jpg)
Foster a Transparent Team
Everyone plays a role…
Don’t declare DONE until it is really there…. Don’t punish for calling it out
Tell the real story, Encourage others to do the same
![Page 20: Validation Missteps Making Us Full Time Firefighters](https://reader038.vdocument.in/reader038/viewer/2022110310/55953b751a28aba1098b4688/html5/thumbnails/20.jpg)
Stop the Firefighting!
Plan Design for Test Create & Integrate Feedback Loops Foster a Transparent Team