evaluating and selecting software packages a review
DESCRIPTION
TRANSCRIPT
![Page 1: Evaluating and selecting software packages a review](https://reader034.vdocument.in/reader034/viewer/2022042606/5480a6755806b5d8108b4584/html5/thumbnails/1.jpg)
Evaluating and selecting
software packages: A review
Anil S. Jadhav*, Rajendra M. Sonar
Information and Software Technology
vol.51 pp.555–563, 2009.
Presenter :
Big Wu
Jin Liu
Dean Yeh
![Page 2: Evaluating and selecting software packages a review](https://reader034.vdocument.in/reader034/viewer/2022042606/5480a6755806b5d8108b4584/html5/thumbnails/2.jpg)
Keyword
• Software Evaluation
• Software Selection
• Evaluation Criteria
• Software Selection Tools
2
![Page 3: Evaluating and selecting software packages a review](https://reader034.vdocument.in/reader034/viewer/2022042606/5480a6755806b5d8108b4584/html5/thumbnails/3.jpg)
Overview
• Evaluating and selecting software packages
• That meet an organization’s requirements is a difficult
software engineering process.
• The aim of this paper is to provide a basis to improve the
process
• Selection of a wrong software package
• Can turn out to be costly and adversely affect business
processes. 3
![Page 4: Evaluating and selecting software packages a review](https://reader034.vdocument.in/reader034/viewer/2022042606/5480a6755806b5d8108b4584/html5/thumbnails/4.jpg)
Overview
• This paper reports a systematic review of papers
published in journals and conference proceedings.
• The review investigates methodologies that support
decision makers include:
4
Selecting Software Packages
Software Evaluation Techniques
Software Evaluation
Criteria System
![Page 5: Evaluating and selecting software packages a review](https://reader034.vdocument.in/reader034/viewer/2022042606/5480a6755806b5d8108b4584/html5/thumbnails/5.jpg)
Overview
• The key findings of the review are:
1. Analytic Hierarchy Process(AHP) has been widely
used for evaluation of the software packages.
2. Lack of a common list of generic software
evaluation criteria and its meaning.
3. Need to develop a framework comprising to assist
decision makers in software selection.5
![Page 6: Evaluating and selecting software packages a review](https://reader034.vdocument.in/reader034/viewer/2022042606/5480a6755806b5d8108b4584/html5/thumbnails/6.jpg)
Introduction
• B. Hecht (1997)
• Selecting the right solution is an exhausting process for
companies.
• J. Verville, A. Hallingten (2002)
• As ERP packages cost hundreds of thousands and even
millions of dollars, purchasing an ERP solution is a high
expenditure activity that consumes a significant portion of
companies’ capital budgets.6
![Page 7: Evaluating and selecting software packages a review](https://reader034.vdocument.in/reader034/viewer/2022042606/5480a6755806b5d8108b4584/html5/thumbnails/7.jpg)
Research Questions
• What is the contribution of the literature in the field of evaluation and selection of the software packages?RQ1:
• What are the methodologies for selecting software packages? RQ2:
• What are the systems/tools to assist decision makers in evaluating and selecting software packages?RQ3:
• What are the software evaluation techniques?RQ4:
• What are the software evaluation criteria?RQ5:7
![Page 8: Evaluating and selecting software packages a review](https://reader034.vdocument.in/reader034/viewer/2022042606/5480a6755806b5d8108b4584/html5/thumbnails/8.jpg)
Multiple Criteria Decision Making
(MCDM)
• K. Yoon, C. Hwang (1995) & E. Triantaphyllou (2000)
• MCDM refers to making preference decisions over the available
alternatives that are characterized by multiple, usually conflicting, attributes.
• M. Mollaghasemi, J. Pet-Edwards (1997)
• The goal of the MCDM is to help
1. Decision makers choose the best alternative of those studied.
2. Dort out alternatives that seem good among the set of alternatives studied.
3. Rank the alternatives in decreasing order of performance. 8
![Page 9: Evaluating and selecting software packages a review](https://reader034.vdocument.in/reader034/viewer/2022042606/5480a6755806b5d8108b4584/html5/thumbnails/9.jpg)
Literature review
• A formal process for evaluating COTS software products, IEEE Proceedings Software
P.K. Lawlis, K.E. Mark, D.A. Thomas, T. Courtheyn, (1997)
• Selection of components for OTS component based systems, IEEE
B. Kizzort, (2001)
• A Process for COTS Software Product Evaluation, ICCBSS S. Comella-Dorda, J.C. Dean, E.
Morris, P. Oberndorf, (2002)
• On the efficiency of domain-based COTS product selection method, Information and Software Technology
K.R.P.H. Leung, Hareton K.N. Leung, (2002)
• A process for COTS software product evaluation, ICCBSSC.-D. Santiago, C. John, M. Edwin, O. Patricia, (2002)
9
![Page 10: Evaluating and selecting software packages a review](https://reader034.vdocument.in/reader034/viewer/2022042606/5480a6755806b5d8108b4584/html5/thumbnails/10.jpg)
Literature review
• STACE: Social Technical Approach to COTS Software Evaluation Component Based Software Quality, LNCS,D. Kunda, (2003)
• A selection process of COTS components based on the quality of the software in a special attention to internet, HIS
K. Oh, N. Lee, S. Rhew, (2003)
• DesCOTS: a software system for selecting COTS components, in: Proceedings of the 30th EUROMICRO Conference, IEEE
G. Grau, J. Pablo Carvallo, X. Franch, C. Quer, (2004)
• A Method for Compatible COTS Component Selection, ICCBSS J. Bhuta, B. Boehm, (2005)
10
![Page 11: Evaluating and selecting software packages a review](https://reader034.vdocument.in/reader034/viewer/2022042606/5480a6755806b5d8108b4584/html5/thumbnails/11.jpg)
Literature review
• In recent years, researchers have focused on models and methods for
reusable off-the-shelf (OTS) software selection.
1. Concentrate on evaluation and selection of specific software products.
2. Describe automated systems/tools that assist decision makers in various
activities involved in software evaluation and selection.
3. Describe only criteria for software selection and methodology for software
selection.
4. Relate to the evaluation of a single software attribute, quality or some quality sub-
attribute, for a software product.11
![Page 12: Evaluating and selecting software packages a review](https://reader034.vdocument.in/reader034/viewer/2022042606/5480a6755806b5d8108b4584/html5/thumbnails/12.jpg)
Software evaluation problem
situations
• Stamelos & Tsoukias (2003)
• analyzed the contents of different ‘‘problem situations” and
suggested a basic classification of software evaluation
problem situations:
12
Keep or Change Make or BuyCommercial
product evaluation
Tender evaluation
Software certification
Software process
evaluation.
Software system design
selection.
![Page 13: Evaluating and selecting software packages a review](https://reader034.vdocument.in/reader034/viewer/2022042606/5480a6755806b5d8108b4584/html5/thumbnails/13.jpg)
Research method
1. Inclusion criteria
2. Search strategy and search
3. Paper selection
4. Data extraction
13
![Page 14: Evaluating and selecting software packages a review](https://reader034.vdocument.in/reader034/viewer/2022042606/5480a6755806b5d8108b4584/html5/thumbnails/14.jpg)
Inclusion criteria
1. Methodology for selecting software
packages
2. Software evaluation criteria
3. Software evaluation technique
4. System/Tool to assist decision makers in
evaluating software packages14
![Page 15: Evaluating and selecting software packages a review](https://reader034.vdocument.in/reader034/viewer/2022042606/5480a6755806b5d8108b4584/html5/thumbnails/15.jpg)
Search strategy and search
ACM Portal
Elsevier’s
Science
Direct
IEEE XploreSpringer-
Verlag
15
• Search Database
![Page 16: Evaluating and selecting software packages a review](https://reader034.vdocument.in/reader034/viewer/2022042606/5480a6755806b5d8108b4584/html5/thumbnails/16.jpg)
Paper selection
1. Initial selection from the search results, based
on reading the abstract of the papers.
2. Final selection from the initially selected list of
papers, based on reading of entire paper.
Reading all 130 papers in detail and considered
60 papers to be included in the final list for review.16
![Page 17: Evaluating and selecting software packages a review](https://reader034.vdocument.in/reader034/viewer/2022042606/5480a6755806b5d8108b4584/html5/thumbnails/17.jpg)
Data extraction
17
![Page 18: Evaluating and selecting software packages a review](https://reader034.vdocument.in/reader034/viewer/2022042606/5480a6755806b5d8108b4584/html5/thumbnails/18.jpg)
Research Results
• The contribution of the reviewed literature in the
field of evaluation and selection.
• Contribution of the literature in the field of evaluation and
selection of the software packages (RQ1)
• The Software packages is presented stage-based
methodologies for selecting software packages.
• Software selection methodologies (RQ2) 18
![Page 19: Evaluating and selecting software packages a review](https://reader034.vdocument.in/reader034/viewer/2022042606/5480a6755806b5d8108b4584/html5/thumbnails/19.jpg)
Contribution of the literature in the field
of evaluation and selection
of the software packages (RQ1)
19
![Page 20: Evaluating and selecting software packages a review](https://reader034.vdocument.in/reader034/viewer/2022042606/5480a6755806b5d8108b4584/html5/thumbnails/20.jpg)
Software selection
methodologies (RQ2)
Step 1:• Determining the need for purchasing by vendor.
Step 2:• Short listing of candidate packages.
Step 3:• Eliminating most candidate package.
Step 4:• Using an evaluation technique of them. 20
![Page 21: Evaluating and selecting software packages a review](https://reader034.vdocument.in/reader034/viewer/2022042606/5480a6755806b5d8108b4584/html5/thumbnails/21.jpg)
Software selection
methodologies (RQ2)
Step 5:• Doing further scrutiny by obtaining trial copy.
Step 6:
• Negotiating a contract specifying to terminate any agreement.
Step 7:
• Purchasing and implementing most appropriate software package. 21
![Page 22: Evaluating and selecting software packages a review](https://reader034.vdocument.in/reader034/viewer/2022042606/5480a6755806b5d8108b4584/html5/thumbnails/22.jpg)
Research Results
• Describes systems/tools to assist decision
makers in evaluating software packages.
• Systems/tools for evaluation and selection of software
packages(RQ3)
22
![Page 23: Evaluating and selecting software packages a review](https://reader034.vdocument.in/reader034/viewer/2022042606/5480a6755806b5d8108b4584/html5/thumbnails/23.jpg)
Systems/Tools for evaluation and
selection of software packages (RQ3)
23
![Page 24: Evaluating and selecting software packages a review](https://reader034.vdocument.in/reader034/viewer/2022042606/5480a6755806b5d8108b4584/html5/thumbnails/24.jpg)
Research Results
• Techniques for evaluating software packages
and software evaluation criteria are
described in :
• Software evaluation techniques (RQ4)
• Evaluation criteria (RQ5)
24
![Page 25: Evaluating and selecting software packages a review](https://reader034.vdocument.in/reader034/viewer/2022042606/5480a6755806b5d8108b4584/html5/thumbnails/25.jpg)
Software evaluation techniques
(RQ4)
Analytic Hierarchy Process(AHP)
Feature Analysis
Weighted Average Sum (WAS)
Fuzzy Based Approach25
![Page 26: Evaluating and selecting software packages a review](https://reader034.vdocument.in/reader034/viewer/2022042606/5480a6755806b5d8108b4584/html5/thumbnails/26.jpg)
Analytic Hierarchy Process(AHP)
Strengths:• AHP enables decision makers to
structure a decision making problem
into a hierarchy, helping them to
understand and simplify the problem.
• It is flexible and powerful tool for
handling both qualitative and
quantitative multi-criteria problems.
• AHP procedures are applicable to
individual and group decision making.
Weaknesses:• AHP is time consuming because of the
mathematical calculations and number of pair-wise
comparisons that increases as the number of
alternatives and criteria increases.
• The decision makers need to re-evaluate
alternatives when the number of criteria or
alternatives are changed.
• Ranking of alternatives depends on the
alternatives consider edfor evaluation hence
adding or deleting alternatives can lead to changes
in the final rank. 26
![Page 27: Evaluating and selecting software packages a review](https://reader034.vdocument.in/reader034/viewer/2022042606/5480a6755806b5d8108b4584/html5/thumbnails/27.jpg)
Feature Analysis
Strengths:
• Evaluation can be done to any
required level of detail by organizing
evaluation in different ways such as
screening mode, case study, formal
experiment and survey.
• It is used not only for technical
evaluation but also for evaluation of
viability of supplier.
Weaknesses:
• Producing the single number from the
individual scores may be misleading
because many different combinations
of numbers can produce the same
aggregate score.
27
![Page 28: Evaluating and selecting software packages a review](https://reader034.vdocument.in/reader034/viewer/2022042606/5480a6755806b5d8108b4584/html5/thumbnails/28.jpg)
Weighted Average Sum (WAS)
Strengths:
• Main advantage of WAS is its
ease of use.
Weaknesses:
• Weights to the attribute are assigned
arbitrary and it is very difficult to assign
weight when number of criteria is high.
• To obtain a score using this method a
common numerical scaling is required.
• Difficulties emerge when WAS is applied to
multi-dimensional MCDM problems.
28
![Page 29: Evaluating and selecting software packages a review](https://reader034.vdocument.in/reader034/viewer/2022042606/5480a6755806b5d8108b4584/html5/thumbnails/29.jpg)
Fuzzy Based Approach
Strengths:
• The decision makers can use
linguistic terms to evaluate
alternatives easily and intuitively.
• It improves decision making
procedure by accommodating the
vagueness and ambiguity occurred
during human decision making.
Weaknesses:
• Difficult to compute fuzzy
appropriateness index values
and ranking values for all
alternatives.
29
![Page 30: Evaluating and selecting software packages a review](https://reader034.vdocument.in/reader034/viewer/2022042606/5480a6755806b5d8108b4584/html5/thumbnails/30.jpg)
Evaluation criteria (RQ5)
Software quality characteristics
30
![Page 31: Evaluating and selecting software packages a review](https://reader034.vdocument.in/reader034/viewer/2022042606/5480a6755806b5d8108b4584/html5/thumbnails/31.jpg)
Evaluation criteria (RQ5)
Vendor & Functional characteristics
31
![Page 32: Evaluating and selecting software packages a review](https://reader034.vdocument.in/reader034/viewer/2022042606/5480a6755806b5d8108b4584/html5/thumbnails/32.jpg)
Evaluation criteria (RQ5)
Cost and Benefits & Hardware and Software
32
![Page 33: Evaluating and selecting software packages a review](https://reader034.vdocument.in/reader034/viewer/2022042606/5480a6755806b5d8108b4584/html5/thumbnails/33.jpg)
Evaluation criteria (RQ5)
Opinions from technical and non-technical sources & Output
33
![Page 34: Evaluating and selecting software packages a review](https://reader034.vdocument.in/reader034/viewer/2022042606/5480a6755806b5d8108b4584/html5/thumbnails/34.jpg)
Limitations
• This study has the usual limitations associated with any systematic
literature review.
• With respect to the search process, we have limited ourselves to
English language studies and to four major electronic databases
and search terms related to the terms ‘‘evaluation” and ‘‘selection”.
• This strategy will not find non-English language papers, paper in
many national journals and conferences, or papers that use
unusual terminology.
34
![Page 35: Evaluating and selecting software packages a review](https://reader034.vdocument.in/reader034/viewer/2022042606/5480a6755806b5d8108b4584/html5/thumbnails/35.jpg)
Conclusions
• This study provides an overview of the
literature associated with evaluation and
selection of software packages.
• On the basis of literature review we propose
generic software selection methodology and
evaluation criteria.35
![Page 36: Evaluating and selecting software packages a review](https://reader034.vdocument.in/reader034/viewer/2022042606/5480a6755806b5d8108b4584/html5/thumbnails/36.jpg)
Thanks for Your Listening
Q & A
36