hugs instead of bugs: dreaming of quality tools for devs and testers
TRANSCRIPT
1 @Dynatrace
- Of Tools That Make Dev, Test and Ops Hug each Other
- More on http://blog.dynatrace.com
- Dynatrace Free Trial: http://bit.ly/dttrial
Hosted by: Andreas Grabner - @grabnerandi
I have a Dream …
Tester Developer
NOBODY wants this …
Unless you work for Google or Microsoft
Nor this …
As it leads to this …
The “War Room”
Facebook – December 2012
And potentially to this …
And this …
And that’s why Business doesn’t like it either …
~80% of problems
caused by ~20% patterns
YES we know this
80% Dev Time in Bug Fixing
$60B Defect Costs
BUT
3 Situations on
WHY this happened,
HOW to avoid it,
METRICS to look at
#Push without a Plan
Mobile Landing Page of Super Bowl Ad
434 Resources in total on that page:230 JPEGs, 75 PNGs, 50 GIFs, …
Total size of ~ 20MB
http://apmblog.dynatrace.com/2014/01/31/technical-and-business-web-performance-tips-for-super-bowl-ad-landing-pages/
m.store.com redirects to www.store.com
ALL CSS and JS files are
redirected to the www domain
This is a lot of time “wasted” especially on high latency mobile
connections
http://apmblog.dynatrace.com/2013/12/02/the-terrible-website-performance-mistakes-of-mobile-shopping-sites-in-2013/
Fifa.com during Worldcup
http://apmblog.dynatrace.com/2014/05/21/is-the-fifa-world-cup-website-ready-for-the-tournament/
# Images
# Redirects
Size of Resources
#“Blindly” (Re)use Existing
Components
Requirement: We need a report
Using Hibernate results in 4k+ SQL Statements to display 3 items!
Hibernate Executes 4k+ Statements
Individual Execution VERY
FAST
But Total SUM takes 6s
http://apmblog.dynatrace.com/2014/04/23/database-access-quality-metrics-for-your-continuous-delivery-pipeline/
# SQL Executions
# of SAME SQLs
Architectural Decisions gone
Bad
We wanted Web 2.0 and Mobile Ready!
http://apmblog.dynatrace.com/2014/03/13/software-quality-metrics-for-your-continuous-delivery-pipeline-part-i/
# Items per Page
# AJAX per Page
•# Images
•# Redirects
•Size of Resources
•# SQL Executions
•# of SAME SQLs
•# Items per Page
•# AJAX per Page
Remember: Use Tools that provide these Metrics
•Time Spent in API
•# Calls into API
•# Functional Errors
•3rd Party calls
•# of Domains
•Total Size
If we do all that …
… so there is more
time for …
Before you get too drunk …
blog.dynatrace.combit.ly/dttrial
apmchallenge.eu
A FINAL THOUGHT!
Putting it into Test Automation
12 0 120ms
3 1 68ms
Build 20 testPurchase OK
testSearch OK
Build 17 testPurchase OK
testSearch OK
Build 18 testPurchase FAILED
testSearch OK
Build 19 testPurchase OK
testSearch OK
Build # Test Case Status # SQL # Excep CPU
12 0 120ms
3 1 68ms
12 5 60ms
3 1 68ms
75 0 230ms
3 1 68ms
Test Framework Results Architectural Data
We identified a regresesion
Problem solved
Exceptions probably reason for
failed testsProblem fixed but now we have an
architectural regression
Problem fixed but now we have an
architectural regressionNow we have the functional and
architectural confidence
Let’s look behind the
scenes
And in your Pipeline
Commit Stage• Compile• Execute Unit Test• Code Analysis• Build installers
Automated Acceptance
Testing
Automated Capacity Testing
Manual testing• Key showcases• Exploratory testing Release
Unit & Integration Tests
Functional Tests
Performance TestsProductionMonitoring
Functional Tests
57 @Dynatrace
Andreas GrabnerFree Tools @ http://bit.ly/dttrial
Follow me @grabnerandi
Email me [email protected]
http://blog.dynatrace.com
Win a GPS Watch: http://apmchallenge.eu