implementing a performance centre of excellence
DESCRIPTION
An old presentation, but just as relevant today as it was when I presented this at the British Computer Society in 2006. This presentation showed how building a performance test team using a shared knowledge base with shared code libraries and best practice techniques made the performance test team a valuable part of the project team at a large UK bank. Making test results and reports accessible to the entire project team and acting as a intermediary between the development teams and business users made the test team vital to the success of many projects at HBoS.TRANSCRIPT
Implementing a Implementing a Performance Test Performance Test “Centre of excellence”“Centre of excellence”
Richard BishopRichard Bishop
Senior Performance Test AnalystSenior Performance Test Analyst
IntroductionIntroduction
•HBOS Formed by Merger – Sep 2001•Halifax Building Society•Bank of Scotland
•70,000 employees•UK, Ireland, Spain, Australia
•UK’s largest mortgage provider•UK’s largest savings provider•£440bn assets
Why am I here?Why am I here?
•Have worked alongside Mercury PS•Experienced performance tester – 6 years•Performance Center™ 8.0 (beta)•LR 8.1 WebGUI (beta)•J2EE diagnostics (beta)• .NET diagnostics (beta)•Scripting standards•Team structure / mentoring•Results publication and analysis
Team evolution and growthTeam evolution and growth
Team websiteTeam website
•Team knowledge base•Central repository for results•Visible throughout HBOS
Performance Test ServicesPerformance Test Services
•EPT – Early testing• Informal• Iterative testing• Developer involvement• Aimed at improving performance
•PAT – Acceptance testing• Formal validation of application• Final test before deployment
ChallengesChallenges
•Delivering testing to meet growing business demands•Keeping pace with developments
(eg .NET 2 / J2EE / Web Services / Citrix)•Resource constraints•Demonstrating ROI•Fixed deadlines
Test ExperienceTest Experience
•Web 80%•COM / DCOM 10%•Web Services 5%•RTE 2%•Citrix 2%•Other 1%
•Citrix use likely to increase post LR 8.1
Test StagesTest Stages
•Planning•Alongside Developers
•Early access to code•Discussion of key features•Standard page IDs•Recommendations
•Key Business Processes•Knowledge pooling
Test Stages (Continued)Test Stages (Continued)
•Preparation• Technical documentation• Volumes calculations
•Test plan•Normal load•Peak load•Duration test
• Application familiarisation• Scenario design
Test Stages (Continued)Test Stages (Continued)
•Scripting
• Script recording• Script standards• Test data• Error checking
Test Stages (Continued)Test Stages (Continued)
•Test Execution• Prove scripts in test environment• Prove data• Re-state objectives
• Don’t test for the sake of it………….
Don’t test for the sake of itDon’t test for the sake of it
“A test a day keeps the boss away”
Test Stages (Continued)Test Stages (Continued)
•Results Analysis•Web-based reporting•LoadRunner Analysis templates•PERFMON charts
• Involve “panel of experts”• Publish results daily• Appropriate for audience
(See example site)
FeedbackFeedback
•No longer a “hurdle”•Seen as desirable•Actively requested by the business
“Overall the experience was one of helpful experts who gave us useful guidance in our testing. We fully intend to make further use of SI as we enter more early performance testing phases as part of the PTP project” – Darren Blackett, Senior Systems Developer - RBIT.
HBOS futuresHBOS futures
•.NET2 diagnostics•J2EE diagnostics•BAC / Tivoli integration•MAM•RUM•Hyperformix / MCP•End-to-end performance projects
Q&AQ&AE-Mail E-Mail [email protected]@hbosplc.com
DDI: +44 (0) 1422 338084DDI: +44 (0) 1422 338084
Mobile: +44 (0) 7909 610098Mobile: +44 (0) 7909 610098