testing leadership: from 2nd class citizens to best of breed
DESCRIPTION
TRANSCRIPT
IBM Quality thru Software Engineering
29 August 2006 © 2006 IBM Corporation
Bill WoodworthDirector, IBM Quality thru Software Engineering
Testing Leadership:From 2nd Class Citizens
to Best of Breed
Keynote at TAIC PARTConference
© 2006 IBM Corporation2
IBM Quality thru Software Engineering
Testing Leadership in IBM – TAIC PART Conference – 29 August 2006
Agenda1. Testing challenges to overcome in 1998
and pitfalls to watch out for…
2. IBM’s Response to these challenges
3. IBM wide Teaming of S/W Test Professionals
4. Results of Testers’ Leadership and Community
5. Keys to Making It All Happen…
Testing Progress in IBM
© 2006 IBM Corporation3
IBM Quality thru Software Engineering
Testing Leadership in IBM – TAIC PART Conference – 29 August 2006
Testing Challenges – in 1998
Low Recognition of the Value2nd Class Citizens
Isolated Islands of TestersNot Knowing Best Practices & ToolsEach Doing Their Own Thing; Tool Proliferation
Unclear customer usage/expectations of new functionsNot clear how to really verify the new functionReal customer scenarios not tested
Lack of disciplined process between Development & TestEntry / exit criteria weak or not applied with disciplineLack of early involvement by testers; Late deliveries from dev
Test technology had lagged behind Development
$?
Industry & academiaLack of automation
© 2006 IBM Corporation4
IBM Quality thru Software Engineering
Testing Leadership in IBM – TAIC PART Conference – 29 August 2006
Testing Pitfalls
1.Poor test planningBlown schedule Easiest items handled first
& High risk items later
3.Weak enforcement of entry & exit criteriaFalse starts, wasted timePoor quality code and
documentation on test entry
2.Poor test designTests redundant & inefficientProduct exercised - not verifiedWeak exception/boundary testsDifferent sequences not tested;
Error paths not well tested
4.System / Integration TestsOverlap between some testsHoles / gaps between others
© 2006 IBM Corporation5
IBM Quality thru Software Engineering
Testing Leadership in IBM – TAIC PART Conference – 29 August 2006
IBM’s RESPONSE =
CREATED A COMPANY WIDE COMMUNITY OF TESTERS
VisionTeaming to Become BEST OF BREED
Mission Delivering VALUE to Each Business Unit
© 2006 IBM Corporation6
IBM Quality thru Software Engineering
Testing Leadership in IBM – TAIC PART Conference – 29 August 2006
Key FOCUS AREAS
1. COMMUNITY Development
2. PEOPLE: Critical Skill / Competency
3. Selecting / Using PRACTICES & TOOLS
4. PROCESS and METRICS
5. Driving for IMPROVEMENTS
© 2006 IBM Corporation7
IBM Quality thru Software Engineering
Testing Leadership in IBM – TAIC PART Conference – 29 August 2006
1. COMMUNITY DEVELOPMENT
Executives Raising Expectations – Key
Team of Impassioned LeadersTechnical and Management
First Principle = “IBM T.E.S.T.” Integrating Better Methods Together
Enhancing Software Testing
Do It Once
Deploy to Many
Many
Many
Do it Once….. Deploy it to ALL !
© 2006 IBM Corporation8
IBM Quality thru Software Engineering
Testing Leadership in IBM – TAIC PART Conference – 29 August 2006
Highly Successful Software Communities•Drive and Provide the Means for:
Community Community
Developing Resourcesneeds basedcommittees +
workgroupsbest technology /
source
Driving Improvementsself assessmentusing community
resourcesremembering lessons
learned
Developing Peoplecareer & skill focus
training & recognition
ChangingDeploying
Gro
win
g
DeliveringValue
Sharingnurturing
communicationswebsite, T.R.s, DBs
newsletters; meetings
Networkingin-person meetings
building relationships
site visits; contacts
Leading & Decidingimpassioned unit
leaders‘change leaders’
TeamingInfo
rming
Gu
idin
g
OperationalInfrastructure
© 2006 IBM Corporation9
IBM Quality thru Software Engineering
Testing Leadership in IBM – TAIC PART Conference – 29 August 2006
2. PEOPLE
Establish Testing as a Recognized Critical Skill and Competency
Ensure Testers Have or Obtain the Required Skills
Actions New recognition opportunities Tester career roadmaps & guidance Test education & skills development
via classes, ITU, conferences, etc.
Management / executive emphasis and feedback
© 2006 IBM Corporation10
IBM Quality thru Software Engineering
Testing Leadership in IBM – TAIC PART Conference – 29 August 2006
TESTING: Professional Excellence with UNLIMITED Future
Skills Developed *Programming*Problem solving *Knowledge - product - system *Leadership *Creativity *Multi-languages
TESTERS:TESTERS:Customer Expectations
Marketplace needs..
System view: The Big PictureHow the products interact &
fit together; Breadth & Depth
Business functionality How used &
What it does & does not do
LEARNLEARN
Many Roles *Software Engineer *Solution architect *Sol’n consultant *Customer advocate *Systems engineer *Automation expert*Jack of all trades
A Multi-Technology
A Multi-TechnologyCareerCareer
A Multi-Industry
A Multi-Industry
ProfessionProfession
© 2006 IBM Corporation11
IBM Quality thru Software Engineering
Testing Leadership in IBM – TAIC PART Conference – 29 August 2006
3. SELECTING Practices & Tools
Architecture Based on Eclipse + Rational
Best Technologies Labs: internal tools Research: teaming on next generation Using / enhancing IBM Rational products
Overall StrategyBecome Intensely Automated
Deployment = Challenging & Critical Practices based with supporting tools Creating resources + info once ‘available to all’ Saturating community with ‘right’ information Facilitating ‘how to’, successes, free consultants Marketing it….. Energizing local teams
© 2006 IBM Corporation12
IBM Quality thru Software Engineering
Testing Leadership in IBM – TAIC PART Conference – 29 August 2006
4. Test Process Background
Test process = key element of product development process
As a company, we develop a lot of code: Some we delivery to our customers = ‘software’ Some we use to verify our software = ‘testware’
We are re-engineering the processes and practices applied by our testers and developers via IRUP IBM’s (customization of ) Rational Unified Process Selecting best practices from RUP, Test Community,
Development Community and IBM SMEs
© 2006 IBM Corporation13
IBM Quality thru Software Engineering
Testing Leadership in IBM – TAIC PART Conference – 29 August 2006
Test Project Manager
Test Architect
Test Customer
Advocate
Test Designer
Test Implementer
Roles = Skills & Responsibilities
Test Discipline Workflow
4. Test Process – enhancing via IRUP
© 2006 IBM Corporation14
IBM Quality thru Software Engineering
Testing Leadership in IBM – TAIC PART Conference – 29 August 2006
4. Closed Loop Process
Key = Learning & Remembering From Mistakes Disciplined & focused causal analysis . Orthogonal Defect Classification (ODC) .
Improving
EmphasisPlanningDesign / DevelopmentExecutionAutomation
Reducing escapes
TEST
Learning/Changing
Experiences fewer problems / defectsImproved solution availability and consumability
Learning/Changing
CUSTOMERS
Learning / Changing a Must
Customer
FocusReducing escapes
ProcessEase of UseAvailabilityReliability
Development
© 2006 IBM Corporation15
IBM Quality thru Software Engineering
Testing Leadership in IBM – TAIC PART Conference – 29 August 2006
4. PROCESS - Metrics
● Critical RoleCritical Role in Improving S/W ProcessesFocus on Quality Process & Cost of Quality
Type of Key MetricsDelivered Quality MetricsIn Process Metrics
How Used…..Business Decision Making ProcessDriving Improvements
Used to demonstrate progress and value Causal Analysis (lessons learned & remembered) Process Self Assessments (by testers and now developers) Scorecards / Dashboards (lab or business area basis)
© 2006 IBM Corporation16
IBM Quality thru Software Engineering
Testing Leadership in IBM – TAIC PART Conference – 29 August 2006
5. DRIVING for Improvements
Assessing where you are & where to improve… via annual Test Self Assessment
by each business unit
Ensuring process discipline applied
Using metrics to demonstrate Value
Gaining Executive Buy-in and Support
Motivating Test Leaders to be ‘Change Leaders’‘Change Leaders’ ‘System Test’ as ‘First Customer’ Drive across Test AND Development
© 2006 IBM Corporation17
IBM Quality thru Software Engineering
Testing Leadership in IBM – TAIC PART Conference – 29 August 2006
5. Two Keys for Unit Self Improvement
1. ITCL Unit Leaders Aggressive ‘Change Leaders’
2. Meaningful and disciplined use of the IBM Test Self Assessment to affect the most beneficial improvements – business wise
SoftwareDevelopment
Process
TestSelf-Assessment
Improvement Actions
IRUP
IBM Best Practices
Identify Strengths and WeaknessesDevelop and Implement
Improvement Plan
Review with VP;Track and
Assess Value
Begin Improvement
Process
© 2006 IBM Corporation18
IBM Quality thru Software Engineering
Testing Leadership in IBM – TAIC PART Conference – 29 August 2006
Agenda1. Testing challenges to overcome in 1998
and pitfalls to watch out for…
2. IBM’s Response to these challenges
3. IBM wide Teaming of S/W Test Professionals
4. Results of Testers’ Leadership and Community
5. Keys to Making It All Happen…
Testing Progress in IBM
© 2006 IBM Corporation19
IBM Quality thru Software Engineering
Testing Leadership in IBM – TAIC PART Conference – 29 August 2006
THE RESULTS
PeoplePeopleUNLIMITED CAREERSUNLIMITED CAREERS
BOTH Higher Quality& Available Sooner
Product SolutionsProduct Solutions
Major Test Effectiveness& Efficiency Gains
BusinessBusiness
LeadershipLeadershipLeading Unit’s Improvement Drive…Leading Unit’s Improvement Drive…
© 2006 IBM Corporation20
IBM Quality thru Software Engineering
Testing Leadership in IBM – TAIC PART Conference – 29 August 2006
THE RESULTS
LeadershipLeadership
•Within Each Product Team, Testers: teamed with developers involved from the start part of architecture / design efforts
•As ‘Change Leaders’, Testers: led the charge for improvements from requirements thru delivery
•Test = Foundation for total SW Eng’g focus
© 2006 IBM Corporation21
IBM Quality thru Software Engineering
Testing Leadership in IBM – TAIC PART Conference – 29 August 2006
Test Foundation for SW Engineering Focus
2003Software Developers Community Leadership
1998 IBM Test Community Leadership
1998SoftwareQualityChampions
1998: Created S/W Testing Process & Quality FocusesCharged by Corporate Technical
Committee (CEO, Senior Execs) Formed IBM S/W Testers
Community and Software Quality Champions
2003: Added Development Process Focus:Because of and based on
success of Test CommunityCommissioned by IBM S/W
and Corp Tech Senior ExecsFormed IBM S/W Developers
Community
2005: Unified 3 into 1 QSE
© 2006 IBM Corporation22
IBM Quality thru Software Engineering
Testing Leadership in IBM – TAIC PART Conference – 29 August 2006
Quality thru Software Engineering Community
QSE Community Vision
All Units are Better Achieving their Business Goals by Applying Better Quality Software Engineering !
(customer sat, speed to market, time to value, profit contribution, etc.)
QSE Community MissionTeaming to Improve Units’ Software Operations
and our Clients’ Satisfaction !
© 2006 IBM Corporation23
IBM Quality thru Software Engineering
Testing Leadership in IBM – TAIC PART Conference – 29 August 2006
QSE Transforming Our S/W Engineering
Team Building & Networking
Developing QSE Resources
Deploying QSE Resources
Driving QSE Improvements…….
Re-Engineering via IRUP
_1998_ _1999_ _2000_ _2001_ 2002-04 _2005_Type text
Type text
Type text
Typ
e t
ext
_2006_
Yielded 100s of Improvement Actions and $10,000,000s of Benefits Each Year
© 2006 IBM Corporation24
IBM Quality thru Software Engineering
Testing Leadership in IBM – TAIC PART Conference – 29 August 2006
KEYS TO SUCCESS THRU TEAMING
●NetworkingNetworking●TrustingTrusting●TeamingTeaming
●DeliveringDelivering Resources Resources
●DevelopingDeveloping Resources Resources
●DrivingDriving Improvements Improvements
●DeployingDeploying Practices Practices
VALUE
TO MAKE REALTO MAKE REALPROGRESS TAKES:PROGRESS TAKES:
Executive Support
TimeImpassioned
drivePersistence