Chapter 9: Evaluating the Outcomes of Information Systems Plans
Managing information technology evaluation – techniques and processes*
By: Satchanawit Phongkriangkrai
* An earlier version of this chapter appeared in the European Management Journal, Vol. 10, No.2. June, pp. 220–229.
Overview: Information systems planning
Agenda
Evaluation: emerging problems Strategy and information systems Evaluating feasibility: findings Linking strategy and feasibility techniques CODA: From development to routine operations
Introduction
The size and continuing growth in IT investments from the early 1990 served to place IT issue above the ranks in most organization .
Many organizations have encountered IT investment problems, e.g., high risks, and hidden cost processes.
At least 20% of IT investment is wasted and between 30% and 40% of IT projects realize no net benefits.
Major problems occur in how the IT investment is evaluated and controlled.
Evaluation: emerging problems
Evaluation brings into question the notion of cost, benefit, risk, value, and process.
There are many problems that impact on IT evaluation: inappropriate measures understating human and organizational costs overstating cost neglecting intangible benefits, etc.
The fundamental and common failure relates to strategy alignment.
Strategy and information systems
Organizational investment climate is affected by: The financial health and market position of the organization Industry sector pressures The organizational business strategy and direction The management and decision-making culture
Decision-making culture Conservative Innovative
Decision-making style Directive Consensus-driven
Alignment
Alignment creates strategic climate in which IT investment can be related to business/organizational direction.
Lack of alignment is a common problem in public sector information.
When the organization lacks alignment of IT evaluation practice, they tend to become separated from business needs and plans, and from organizational realities.
Another critical alignment is that between: what is done with IT how it fits with the information needs of the organization
IT Strategic grid
Developed by McFarlan and McKenney in 1983 Classifies systems where IT investment has been made and
where it should be applied. Demonstrates whether the IT vestments are being made:
into core system, business growth, competitiveness.
Identifies four classes of firms.
Figure Strategic grid analysis
Value chain
Established by Porter and Millar in 1991 Consists of the primary and support activities.
Primary activities of a typical manufacturing company: inbound and outbound logistics operations marketing and sales support service
Support activities: firm infrastructure human resource management technology development procurement
IT investment mapping
Developed by Peters in 1993 The basic dimensions of map were arrived at after reviewing
the main investment concerns. Two dimension over 50 IT projects
investment orientation Infrastructure – software/hardware environment Business Process – finance and accounts Market Influence – increasing repeat sales
and benefit: Business Expansion Risk Minimization Enhance Productivity
Figure IT investment mapping
Figure Investment map comparing business and IT plans
Multiple methodology
Developed by Earl in 1989 A multiple methodology inquires IT investment more closely
with the strategic aims and direction of the organization, and its key needs.
Three strategy formulas are: A top-down approach – use critical factor success (CFC) to
establish objectives; A bottom-up evaluation – evaluate current systems; Inside-out innovation – create new strategic options.
The purpose is to identify through internal and external analysis of business needs and opportunities to relate the development of IS applications to business strategy.
Evaluating feasibility: findings
Evaluating IT projects at its feasibility stage are based on the right strategic climate.
Feasibility evaluation provides results of interest and to sponsor of research.
Value of IT/IS are justified by understating cost and using notational figure for benefit realization.
Willcocks and Lester looked at 50 organizations from both private and public section of manufacturing and service.
There is little evidence of a concern for assessing risk in any formal manner.
Figure IT evaluation: feasibility findings
Linking strategy and feasibility techniques
A method uses evaluation techniques to measure the type of IT project, develop techniques relating the IT investment to
business/organization value. The method of evaluation needs to be
reliable, consistent in its measurement over time.
The three evaluation techniques are: Return on management (ROM), Matching objectives, projects and techniques, From cost-benefit to value.
Linking strategy and feasibility techniques (Return on management (ROM)) Developed by Strassman in 1990 ROM is modern IT investment evaluation. ROM measures the performance based on the added value –
the difference between net revenues and payment external supplier – to an organization provided by management.
However, ROM is not widely used in UK. The problem is whether it really represents what IT has
contributed to business performance.
Linking strategy and feasibility techniques (Matching objectives, projects and techniques)
Butler Cox (1990) suggests five main purposes: surviving and functioning as a business improving business performance by cost reduction/increasing sales achieving a competitive leap enabling the benefits of other IT investments to be realized being prepared to compete effectively in the future
The matching of IT investment can now be categorized as follows:
Mandatory investments Investments to improve performance Competitive edge investments Infrastructure investments Research investments
Figure Classifying IT investments
Figure Matching projects to techniques
Linking strategy and feasibility techniques(From cost-benefit to value)
Known as the information economics approach Developed by Parker et al. 1988 This technique copes with many problems of IT evaluation,
both at the level of methodology and of process. Information economics looks beyond benefit to value. Value is the sum of:
Enhanced return on investment, Business domain assessment, Technology domain assessment.
Figure The information economic approach
Linking strategy and feasibility techniques(From cost-benefit to value (Cont’d)) Enhanced ROI
Value Linking – assesses indirect benefits added to other departments. Value Acceleration – reduced time-scales for operations. Value Restructuring – measures the benefits of restructuring a department, or
job. Innovation valuation – considers the value of competitive advantage and
calculate risks, pioneer cost, or failing cost. Business Domain Assessment
Strategic Match: matches the business goals to the proposed project/IT investment.
Competitive Advantage – assesses the degree of an advantage in the market. Management Information – assesses the contribution toward the management
need for information. Competitive Response – assesses the degree of corporate risk associated with
not undertaking the project. Organization Risk – implements the project in terms of personnel, skills, and
experience.
Linking strategy and feasibility techniques(From cost-benefit to value (Cont’d))
Technology Domain Assessment Strategic IS Architecture – measures the degree to which the project
fits into the overall information system plan. Definitional Uncertainty – assesses the complexity of the area and
probability of non-routine change. IS Infrastructure Risk – evaluates a project’s dependence on new or
untried technologies.
CODA: From development to routine operations
CODA: From development to routine operations (Cont’d) The evaluation cycle is useful for
controlling a specific project, building organizational know-how on IT and management.
However, some of limitations in evaluation techniques and processes are: Weak linkage between evaluations carried out different
stages, Projects shelved, Stakeholders not included in feasibility stage, Continual problems from feasibility stage.
CODA: From development to routine operations (Cont’d) Suggestion to improve evaluation:
Linking evaluation across stages and time Determining stakeholders who are participants in
evaluation at all stages Avoiding the fall-of interest in evaluation at later stage Providing adequate evaluation techniques for the long
term process.
Conclusions
The profile of IT evaluation has increased high expenditure, disappointed expectations, underdeveloped and undermanaged area.
Need more effective evaluation practice: traditional techniques, tailor modern techniques.
In the past of the evaluation practice, value is price. The future challenge is to continue the measurement
of value and build techniques and processes.
Thank youQ&A
