evaluation of quality assurance factors in agile methodologies
DESCRIPTION
Mehreen Sirshar & Dr. Fahim ArifInternational Journal of Advanced Computer Science, Vol. 2, No. 2, Pp. 73-78, Feb. 2012.TRANSCRIPT
International Journal of Advanced Computer Science, Vol. 2, No. 2, Pp. 73-78, Feb. 2012.
Manuscript Received:
Revised:
Accepted:
Published:
Keywords
Agility,
Software
Quality
assurance,
Quality
factors,
Testing
Techniques
Abstract__ The agile software development
claims to improve the quality of the software
products. This claim has increases its usage
in the new industry of the software and
information technology. But, due to its
immature nature, very few frameworks for
the evaluation of quality exist. The proposed
research has introduced a tool for the
evaluation of quality in agile software
development. Various factors addressing the
quality have been evaluated among different
phases in various practiced methodologies of
agile software development. The proposed
tool has provided a new platform for
application of various types of testing in agile
methodologies. It has opened a new horizon
of research in software industry
1. Introduction
The traditional methodologies for the software
development focused on waterfall approach. After
achieving the freezing point of requirements the system was
developed in increments. However, volatile nature of
market, rapid change in customer requirements ,dynamic
infrastructure of business organization , exhaustive need for
short time deliverables and highly variations in pace of
development technologies and tools added the competitive
pressure to sustain in the market. As a result many
organizations adapted agile method for software
development. Practical implementation of software
development is highly dependent on the practices followed
by different organizations. The introduction of Agile as
software methodology in literature was introduced in late
90's.After that several methodologies have been proposed.
With the above mention factors the conformance to
quality standards was also challenging task. The agile
development promises to provide quality assurance. In fact
agile development is popular for quality delivery of the
project in small deliverables. The element of the proposed
research also targets the issues of quality in agile
development.
This work was supported by the National University of Science and
Technology , Rawalpindi ,Pakistan
M.Sirshar, with National University of Science and Technology,
Software Engineering MS Research scholar ([email protected])
F.Arif, with National University of Science and Technology,Rawalpindi
Pakistan([email protected])
The paper is organized into five sections. Section contains
literature review. Section 3 includes quality assurance
(QA) factors being evaluated in the research. Section 4
introduces tools and applications on agile methodologies.
Section 5 discusses the analysis .Section 6 draws
conclusion
2. Literature Review
Many definitions have been proposed for defining
quality and agile development. The literature survey reveals
very few definitions regarding agile and quality together.
A. Defining Quality
Many researchers agree on the consensus that quality is
conformance to customer needs. Many researches enhance
this view to define quality as characteristics bear by the
software product which must reflect the needs specified by
the customer.
Meyer [1] defines software quality according to an
adapted number of quality factors as defined by McCall [2],
which are; correctness, robustness, extendibility, reusability,
compatibility, efficiency, portability, integrity, verifiability,
and ease of use etc.
Pressman agrees with Crosby and defines quality as
“conformance to explicitly stated functional requirements,
explicitly documented development standards, and implicit
characteristics that are expected of all professionally
developed software” [3]. Sommerville [4] defines software
quality as a management process concerned with ensuring
that software has a low number of defects and that it reaches
the required standards of maintainability, reliability,
portability and so on. Pfleeger [5] agrees with Garvin who
views quality from five different perspectives namely:
• The transcendental meaning that quality can be recognized
but not defined,
• User view meaning that quality is fitness for purpose,
• Manufacturing meaning that quality is conformance to
specification,
• Product view meaning that quality is tied to inherent
product characteristics
• The value-based view meaning that quality depends on the
amount the customer is willing to pay for the product.
A. Defining agility
In literature, agility is commonly refers to the readiness
to motion. The Agile manifesto was published by group of
software engineers and consultants in 2001. It is stated as
Evaluation of Quality Assurance Factors in Agile
Methodologies
Mehreen Sirshar & Dr. Fahim Arif
International Journal of Advanced Computer Science, Vol. 2, No. 2, Pp. 73-78, Feb. 2012.
International Journal Publishers Group (IJPG)©
74
follows [6]-[7]:
Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan
Ambler [8] considers agile quality to be a result of
practices such as effective collaborative work, incremental
development, and iterative development as implemented
through techniques such as refactoring, test-driven
development, modeling, and effective communication
techniques. The definition of quality from view of agile
development is still dependent on organizational practices
and the agile methodology being followed.
From the agile perspective quality has been defined as
McBreen [9] defines agile quality assurance as the
development of software that can respond to change as the
customer requires it to change. This implies that the
frequent delivery of tested, working, and
customer-approved software at the end of each iteration is
an important aspect of agile quality assurance.
B. Quality assurance factors in agility
Table 1, defines different quality attributes followed in
agile development. Each quality factor is evaluated in
different agile methodologies. Some organizations follow
mixed methodologies to achieve different quality
requirements. Analysis of methodologies reveals some
practices which are suggested to be followed to achieve
high quality.
All agile methodologies do not address all of the quality
assurance factors .Several methodologies address multiple
factors. However combinations of methodologies are
followed to ensure the required quality factors. Quality
factors are non functional attributes of the system which are
implicitly stated .Therefore, they are most difficult to define
and evaluate.
Compatibility
Platform independence can be achieved by multiple
agile methodologies especially open source development
methodologies. It can also be achieved by following OO
(object Oriented) design in local practices of an
organization.
Cost Effectiveness
Iterative and incremental nature helps to achieve cost
effectiveness by delivering the prioritized requirements in
small increments.
Correctness
Many agile methodologies involve active involvement
of the customer therefore correctness in requirements is
handled and the correctness in the behavior of the desired
system is also achieved.
Ease of use
Active involvement of the customers increases the
understanding of the system thus increasing in ease of use
of the system when designing its interface. Interface is
highly important when web layouts are designed. However,
this may create problems when open source development is
followed but it is easily achieved through scrum, XP
(extreme programming), RUP (rational unified process) and
JAD (joint application development) techniques.
Extendibility
OOP design, COTS and modular development help to
cater changing requirements with greater flexibility the
system can be easily extended by achieving low coupling.
Almost all of the agile methodologies follow OOP designs
to achieve it.
Maintainability
Modular design and compatibility between interfaces
help to achieve maintainability. Feature driven development
and crystal methodologies play important role to achieve
maintainability.
Performance
Sprint, OO designs, code ownerships and common class
activities help to increase performance in almost all of the
existing agile methodologies.
Portability
Distributed computing and web service design has
TABLE 1:
QUALITY ASSURANCE FACTORS AND THEIR DEFINITIONS [20]
Quality
Parameters
Definitions
Correctness The ability of a system to perform according to defined specification.
Robustness Appropriate performance of a system under cases not covered by the specification. This is complementary to correctness.
Extendibility A system that is easy to adapt to new specification.
Reusability Software that is composed of elements that can be used to construct different applications
Compatibility Software that is composed of elements that can easily combine with other elements.
Efficiency The ability of a system to place as few demands as possible to hardware resources, such as memory, bandwidth used in
communication and processor time.
Portability The ease of installing the software product on different hardware and software platforms
Timeliness Releasing the software before or exactly when it is needed by the users.
Integrity How well the software protects its programs and data against unauthorized access
Verification and
validation
How easy it is to test the system.
Ease of use The ease with which people of various backgrounds can learn and use the software.
Maintainability The ease of changing the software to correct defects or meet new requirements
Performance Low utilization of resources, lower response time and mean time of failure and recovery define the performance of the system.
Cost effectiveness The ability of a system to be completed within a given budget.
Mehreen Sirshar et al.: Evaluation of Quality Assurance Factors in Agile Methodologies.
International Journal Publishers Group (IJPG)©
75
increased portability in agile methodologies.
Integrity
Low coupling among design increments help to achieve
integrity of the data as well as the integrity of system
Reusability
OO (object Oriented) designs are based on the principle
of reusability. Furthermore, organizations following agile
development keep track of all design increments to make
them available in short time boxes.
Robustness
Robustness is ensured by following development
standards however robustness is implicit. It cannot be
explicitly stated.
Timeliness
Iterative development and short cycles are followed to
achieve timeliness.
Validation and verification
Unit testing, integration testing and regression testing
and interface testing are commonly followed methodologies
to ensure validation and verification. Further more user
involvement ensures verification with each deliverable.
3. Tool for evaluation of quality
for agile methodologies
This section provides a brief overview of existing agile
methodologies, quality factors in agile development and
different quality assurance factors. The proposed tool
provides assurance factors. The proposed tool provides
different testing techniques that can be applied to different
quality attributes supported by specific methodology
through quality assurance factors. One such tool was
proposed by Hou et al [10] based on comparative study
between waterfall methodologies and existing agile
methodologies. The tool had certain limitations found in
analysis as mentioned below:
• Singles out two main aspects of quality management
namely; quality assurance and verification and validation.
• Overlooks other vital techniques used in agile processes to
achieve higher quality management.
• And is mainly based on Extreme Programming which on it
own is not a complete representation of agile
methodologies.
To address these limitations another tool was proposed
by Mankandla et al [11] .But the tool failed in its
applicability to all agile methodologies. It focuses only on
two of the widely used agile methodologies. In the
subsequent development effort similar tool has been
proposed in this research paper to address the limitations
provided by proposed tool. The proposed tool is applicable
to all agile methodologies.
4. Agile Methodologies.
Agile methodologies take the advantage of object
Oriented programming and component based software
development paradigms. Commonly followed agile
methodologies have been discussed below:
A. Extreme Programming
Beck et al [12] defines five phases of extreme
Programming: Exploration, Planning, Iteration to release,
Productionizing, Maintenance and death. In exploration
phase user requirements are collected on story cards,
planning focuses on making plan for deliverable defined
through story cards, extra testing and checking is done in
productionizing phase, maintenance phase deals with
keeping interest of market, customer and product in
connection to changing requirements where as death phase
is end or complete delivery of the product. It is notable that
the focus of the maintenance phase is on execution of
quality assurance activities.
B. Scrum
Scrum focuses on self organizing teams especially in the
scenarios where the environment and resources are highly
meant to variant. According to Schwaber[13] Scrum follows
three phases. First phase is pre-game phase which includes
planning and architecture development. Second is
development phase and third is post-game phase which
deals with closure of release. Major focus is on carrying the
activity known as Sprint in the development phase. Sprint is
iterative in nature and focus is on dealing with changing
demands and requirements.
C. Crystal Methodologies
Crystal Methodology is among recently developed
methodologies. Two well known methodologies are Crystal
Clear Methodology and Crystal Orange Methodology.
Crystal methodologies follows incremental delivery, direct
user involvement, timeliness defined with milestones,
regression testing , two user reviews per release, workshop
and interviews for product and methodology tuning, quality
and design standards, version control strategies and
compilation tools. According to CockBurn [14] each
increment of crystal include the development of
requirement specification document ( not detailed )
followed by the phase of staging, includes planning and
scheduling, followed by several iterations development,
monitoring, stability with milestones and review before
the final release.
D. Feature Driven development.
Feature driven development focuses on design and
implementation features in five steps using object oriented
methodology. Palmer et al [15] defines, develop and build
overall model, build feature list, plan by feature, and design
by feature and build by feature as five distinct stages.
Domain or context modeling is an important aspect of this
methodology which helps to increase performance by clear
understanding of the system. Inspection is commonly
followed defect prevention strategy. Feature defines the
functionalities required from the product by the customer.
E. Rational Unified process (RUP)
RUP has been developed to complement UML modeling
while following the agile methodologies. It is an iterative
paradigm based on OO (Object Oriented) Modeling.
Kructhen [16] states inception, elaboration, construction and
International Journal of Advanced Computer Science, Vol. 2, No. 2, Pp. 73-78, Feb. 2012.
International Journal Publishers Group (IJPG)©
76
transition as major processes followed in RUP methodology.
Business Modeling and environment workflows are
important part of RUP methodology. The transition phase in
RUP is totally dedicated to testing to ensure quality.
Inception is focused on gathering requirements, elaboration
on building models and construction on development of the
systems based on modeling content.
F. Dynamic System Development Methodology
(DSDM)
DSDM has been evolved to support RAD (Rapid
application Development framework).DSDM is based on
learn, speculate and collaborate phases. [17] DSDM is
component oriented approach. It is preferably followed for
large and complex systems. In speculation phase the project
is instantiated and planning is done. In collaboration phase
various component based methodologies are coordinated. In
last phase of learning quality reviews are carried out, quality
assurance is achieved and product is release.
G. Open Source Software Development(OSSD)
OSS is volunteer based distributed development
paradigm .It is based on versioning control systems located
on geographically distributed zones. Apache Server, Perl
and Linux are common examples of it. No project planning
and documentation is done. However the code is tested
before launching an open source platforms. Two common
open source platforms are 'Sourcefourge' and 'Freshmeat'
which provide free web based hosting. Sherma et al [18]
proposed problem discovery, finding volunteers, solution
identification, code testing and reviewing, code change view,
code commit documentation and release management as
commonly followed phases.
H. Agile Modeling or Lean Development
It is among most recently introduced methodologies. It
is based on repairing acute problems in design and
documentation, and support vast cultural boundaries and
geographically distributed organizations [19].
I. Pragmatic Programming
Pragmatic programming has no defined processes, tools
and standards .It is at the liberty of the organization
following some internal methods as well upon the
collaboration and communication among geographically
dislocated teams. Automation of tasks is preferred in
TABLE 2: EVALUATION OF METHODOLOGIES AGAINST QUALITY FACTORS.
Quality Parameters Agile Methodologies and quality activities
XP (extreme
programming)
Srcum Crystal methodologies Feature driven
development
RUP
Correctness User stories Review meetings Interviews and formal
meetings, printing
whiteboards
Feature lists prepared
with customers
Inception phase.
Interviews with
stakeholders
Robustness Generic OO design OO Design Tuning methods OO design OO design
Extendibility Simple design Prioritization Versioning systems Simple design +
versioning system
UML design
Reusability OO design Existing design Configuration management
tools
OO design UML design, COTS
development
Compatibility Inherit in OO deign OO design Version control Version control Inherit in OO design
Efficiency Pair programming Team meeting White boards Class code ownership UML modeling
Portability OO design practices OO designs Compilation tools Configuration
management support
Component Based architecture
Timeliness Incremental and iterative
development
Iterative Incremental 4-6 weeks Iterative development Iterative development
Integrity OO design OO design Common object model Class code ownership UML modeling
Verification and
validation
Unit testing Testing and reviews Regression testing , Interface
testing
Inspection Acceptance testing
Ease of use Simple design Simple design GUI Designs, User manuals Feature list UML modeling
Maintainability Simple design Compatible with
existing system
Versioning system OO design and versioning
control
UML compliances
Performance Productioniz-ing phase Sprint phase Common object Model Class code ownership OO design
Cost effectiveness Iterative development Iterative development Incremental delivery on
regular basis
Reusable design features
in iterative manner
Iterative development with OO
design
Correctness User stories Review meetings Interviews and formal
meetings, printing
whiteboards
Feature lists prepared
with customers
Inception phase.
Interviews with
stakeholders
Robustness Generic OO design OO Design Tuning methods OO design OO design
Extendibility Simple design Prioritization Versioning systems Simple design +
versioning system
UML design
Reusability OO design Existing design Configuration management
tools
OO design UML design, COTS
development
Compatibility Inherit in OO deign OO design Version control Version control Inherit in OO design
Efficiency Pair programming Team meeting White boards Class code ownership UML modeling
Portability OO design practices OO designs Compilation tools Configuration
management support
Component Based architecture
Timeliness Incremental and iterative
development
Iterative Incremental 4-6 weeks Iterative development Iterative development
Integrity OO design OO design Common object model Class code ownership UML modeling
Verification and
validation
Unit testing Testing and reviews Regression testing , Interface
testing
Inspection Acceptance testing
Ease of use Simple design Simple design GUI Designs, User manuals Feature list UML modeling
Maintainability Simple design Compatible with
existing system
Versioning system OO design and versioning
control
UML compliances
Performance Productioniz-ing phase Sprint phase Common object Model Class code ownership OO design
Cost effectiveness Iterative development Iterative development Incremental delivery on
regular basis
Reusable design features
in iterative manner
Iterative development with OO
design
Mehreen Sirshar et al.: Evaluation of Quality Assurance Factors in Agile Methodologies.
International Journal Publishers Group (IJPG)©
77
pragmatic programming.
Table 2 provides a comparison among different
discussed methodologies and quality assurance factors
obtained from them in different processes. Pragmatic
Programming has been intentionally left blank due to its
immature nature and unavailability in literature.
5. Analysis and Suggestions
Testing techniques which are suggested to be followed
in particular methodology are tabulated in Table 3.
Complex documentation is avoided. Automated testing
techniques are preferred due to short time boxes. Beta
testing, regression testing and integration testing are usually
followed.
Analysis of the table reveals the fact that apart from
following complex and formal testing techniques simple
and commonly followed techniques are used in nature. It is
due to the fact that almost all of the agile methodologies
are based upon design simplicity , to promote reuse and
efficiency as well as they are based on Object oriented
Methodologies.
6. Conclusion
This research has introduced a innovative tool for
evaluation of agile methodologies against quality assurance
factors. The quality management in agile methodologies is
also new era of research. The limitation of tool is that is
applicability in industry has not been verified. It may be due
to difference in practices followed from organization to
organization. The proposed tool would spark new era of
research in field of agile software development especially
the scenarios of agile modeling architecture for enterprise or
offshore development.
References
[1] Meyer, B., Object-Oriented Software Construction, Prentice
Hall PTR, pp.4-20, 2000
[2] McCall, J.A., Richards, P. K., & Walters, G. F., Factors in
Software Quality, Vols. 1, 2 and 3, National Technical Information
Service, 1977
[3] Pressman, R.S., Software Engineering a Practitioner’s
Approach, Mcgraw-Hill, 2001
[4] Sommerville, I., Software Engineering. Addison-Wesley,
2004
[5] Pfleeger, S. L., Software Engineering: Theory and Practice,
Prentice Hall, 2005
[6] CockBurn, A., Agile Software Development, Boston,
Addison –Wisely, 2002 (a)
[7] Beck, k., Beedle, M., Van Bennekum, A., CockBurn, A.,
Cunningham, W., Folwer, M., Gernning, J., Highsmith, j., Hunt, A.,
Jeffries, R., Kern, J., Maerik, B., Martin, R., Mellor, S., Schwaber,
K., Sutherland, J., & Thomas, D., Manifesto for software Agile
Development (22.3.2002) ,2001, http//: agileManifesto.org
[8] Ambler, S., Quality in an Agile World, Software Quality
Professional, Vol. 7, No. 4, pp. 34-40, 2005
[9] McBreen, P., Quality Assurance and Testing in Agile Projects,
Available from: http://www.mcbreen.ab.ca/talks/CAMUG.pdf
[Accessed 26 July 2006].
[10] M. Huo, J. Verner, Y. Zhu, & M.A. Babar, "Software Quality
and Agile Methods," (2004) Proceedings of the 28th Annual
International Computer Software and Applications Conference
(COMPSAC04), IEEE Computer
[11] E. Mnkandla, & B. Dwolatzky, "Defining Agile Software
Quality Assurance," (2006) Proceedings of the International
Conference on Software Engineering Advances (ICSEA'06), IEEE
[12] K. Beck, "Extreme Programming Explained: Embrace
Change, Reading Mass, Addison Wesely," (1999) IEEE Computer
vol. 32, no. 10, pp. 70-77
[13] Schwaber, K., & Beedle, M., Agile Software Development
with Scrum, Upper Saddle River, NJ, Prentice-Hall, 2002
[14] CockBurn, A., Writing Effective Use cases, The crystal
collection for software professionals, Addison-Wisely Professional,
2000 (b)
[15] Palmer, S.R., & Fesling, J.M., A practical Guide to Feature
driven Development, Upper saddle River, Prentice Hall, 2002
[16] Kructhen, P., the Rational Unified Process, an Introduction,
Addison-Wesely, 2000
[17] Highsmith, J. A., Adaptive software Development, a
Collaborative Approach to managing complex systems, New York,
NY, Dorset House Publishing, 2000
[18] S. Sharma, V. Sugumaran, & B. RajaGopalan, "A
Framework for creating hybrid open source software
communities," (2002) Information system Journal, vol. 12, no. 1,
pp. 7-25
[19] S. Ambler, "Lessons in agility from Internet based
development," (2002) International proceedings on information
systems, IEEE
[20] IEEE Standard Glossary of Software Engineering
Terminology, IEEE Std 610, 1990
TABLE 3:
TESTING TECHNIQUES AND AGILE METHODOLOGIES
Agile methodology Testing techniques
Extreme Programming Unit Testing, acceptance testing, integration testing, functional testing by tester or user, customer
Feedbacks
Scrum Review meetings, unit testing , functional testing, and sprint backlogs
Crystal Methodologies
Crystal clear and crystal Orange)
Regression testing , interface testing and repeated GUI testing
Feature driven Development Inspection, formal testing and unit testing
Rational Unified Process Alpha testing, Beat testing, component integration testing
DSDM (dynamic system Development
Methodology)
Regression testing, Converge testing
Open Source software development(OSSD) Beta testing
Lean development or agile Modeling Beta testing, Inspection
International Journal of Advanced Computer Science, Vol. 2, No. 2, Pp. 73-78, Feb. 2012.
International Journal Publishers Group (IJPG)©
78
Mehreen Sirshar , was born in Rawlpindi , Pakistan ,in 1985. She
Received her B.S.E degree from Fatima Jinnah
University ,Rawalpindi Pakistan.She is Currently Reseach
Scoholar for M.S (S,E) at National University of Science and
Technology Rawalpindi ,Pakistan.She is Serving as Lecturer in
Software Engineering in Fatima Jinnah Women University .She is
author of 3 international publications, reviewer of various
international conferences and supervisor of couple of graduate
degrees. She is currently conducting her Ms-Phd courses in
Millitary College of Signals.(a campus of National Univeristy of
Science and Technology) Rawalpindi Pakistan.She is currently
conducting her research in Digital video tracking projects.
Dr Fahim Arif was commissioned in
Signal Corps of Pakistan Army in 1988.
He has performed duties in Army units,
Research and Development setup of Corps
of Signals and National University of
Sciences and Technology. He has served
as Telecommunication Engineer in Special
Communication Organization and Frontier
Works Organization in southern Sindh
area. He has done Bachelors in Telecommunication from Military
College of Signals (UET Lahore) in 1995 and Master in Sciences
in Computer Software Engineering from National University
Science and Technology in 2003. He has won NUST Endowment
fund scheme scholarship for NUST in 2003 and International
Research Support Initiative Program Fund from HEC in 2007. He
has completed his PhD degree from National University Science
and Technology in 2009. His contribution to international research
in recent few years is excellent. He has presented/published eight
research papers in different international conferences including
USA and Canada. In addition to his research publications, he is
doing as reviewer for various international conferences. He work
as international researcher in System and Computer Engineering
Department, Carleton University, Ottawa, Canada in 2007 and
participated numerous research and academic activities. Recently,
his biography has been published by South Asian Publication
Who’s Who in the World 2008 Edition and awarded with Star
Laureate 2008 in recognition to his contributions to knowledge and
research. Higher Education Commission has nominated him as
their official PhD supervisor in 2010. Currently, he is teaching
various academic courses at MCS and supervising one PhD
student and different MS/ BE students in their final projects/thesis
in NUST.