improving software testing by observing process -ossi taipale -kari smolander lappeenranta...
Improving Software Testing by
Observing Process-Ossi Taipale
-Kari SmolanderLappeenranta University of
Presented by Albert Saryan and Karo Mazidzhyan
Breakdown Introduction Related Research Research Process Analysis Results
Process Improvement Propositions Conclusion
Introduction The objective of this study was to
understand how software testing is conducted by observation.
From observations propose improvement to the testing process.
Improvements by reducing development and testing costs, and improving quality.
Software Costs and Quality Software Engineering strives to reduce
development costs and improving quality. Software Process Improvements (SPI) are
the means to reaching these goals. Commitment to SPI’s by all from all
organizational levels is key to success Quality can be tested into products or
developed and built into products
Software Costs and Quality Cont. External events such as deadlines affect
software quality. The cost of software testing is high,
therefore SPI’s are necessary to reduce cost.
Related Research Involvement of testing during
development occurs when testers develop test for developers to analyze
The complexity of testing increases as a function of the complexity of the systems under testing.
Testing strategy defines the contents of testing.
Related Research Cont. Communication and interaction between
development and testing processes requires cooperation and coordination.
The use of software components are increasing rapidly
Design outsourcing and distributed development increase the use of components.
Cost of Quality is “Free”, but being late with products may be more costly than fixing faults.
Research Process This study consisted of Organizational
Units (OU) which develop and test technical software for automation or telecommunication in Finland.
Initial Sample included 26 OU’s, from which 5 were used as case studies.
Cases were chosen to show polar types
Research Process Cont. Data for the research was collected by a
series interviews. Each interview had a different theme in
mind and possibly a different interviewee in mind.
The interviews took place during five rounds, based on the theme.
Research Process Cont. Interview Rounds1. Development and Testing Managers were
asked to define their testing process .2. Managers of Testing were asked to
define their testing process in depth.3. Testers were interviewed.4. Systems Analysts interviewed.
Research Process Cont.Case Breakdowns
Research Process Cont. Data Analysis
Information gathered from these interviews were then categorized.
Categories were then analyzed to see how they were connected.
The categories were then used to identify factors which affected testing.
Analysis Results Description of Cases:
Case A - Developed and tested Manufacturing Execution Systems :
Turnover 50% product, 50% service Services included systems integration and customization Testing against requirements was a challenge because
customers had special in-house requirements standards Developers and testers worked physically close to each
other Time allocated to testing was consistent, although over
time it has been reduced Use of components low, hinders testing
Analysis Results Cont. Case B - Tested in house products and
provided testing services for external customers:
Turnover 75% service, 25% product Majority of requirements specifications were based
on standards. Delays in development allowed for fewer time
allocated for testing Communication flexible, developers talked face-to-
face with testers Use of components high, testability of components
must be considered
Analysis Results Cont. Case C – Customized Software Development:
Turnover 2/3 service, 1/3 product Testers were involved early, involved in development
process Testers often had issues due to lack of advisement
from developers Delays in development does not often result in
reduction of testing time Developers and testers communicate face-to-face Use of components low Testing of software components seen as difficult b/c
of different implementation.
Analysis Results Cont. Case D - Electronics:
Turnover 100% product High product orientation required high quality
because recalls are very expensive Avoided testing of unfinished product Testing tasks clear and well documented Testing involved in development late but planning of
testing automation provided information on upcoming tests
Communication between developers and tester planned, formal and transparent
Use of components high Components tested initially by suppliers then again at
Analysis Results Cont. Case E – Software Testing Services:
Turnover 100% service Working as an external testing organization required the
adaptation of the process of the customer Early involvement of testing was necessary for the testing
company to increase the testability of the software Sometimes testing was involved late Budgets for testing affected testing time Communication was handle through a contact person, but
was active and clear Use of components depends on customer As an external testing organization it was hard to receive
information about customers purchased components.
Analysis Results Cont.
Analysis Results Cont. Cause and effect
Analysis Result Cont. Cause and Effect
Business Orientation Directly associated with use of components and testing
schedules Business Model – value adding process
Purely Service Oriented System integration Customizing Consulting Customers directly affect development and testing process
Purely Product Oriented Product Development Marketing Customers do not directly affect development and testing
Analysis Results Cont. Process Improvement Propositions
Testing ought to be adjusted to business orientation
Product oriented should adopt formal planned testing process
Service oriented should adopt a flexible testing process
Enhanced Testability of software Consider testability when selecting components Review testing process of suppliers
Analysis Results Cont. Effective Communication and interaction
between development and testing Early involvement of testing and planning of
testing Use of risk based testing
Conclusion Proposals by observing best practices using
grounded theory Better documentation improved testability of
software and components Efficient communication between development
and testing improved quality When time is a major issue, risk based testing is
the best solution Business orientation affects:
Use of components Amount and quality of communication Allocated testing time and the planning of testing