software evolution_se lect3 btech

Types of software products

1

Syllabus: Unit 1

• Role of Software Engineering, Software Evolution, Legacy system structures, Legacy system design, Legacy System Assessment, Software Development Life Cycle.

2

Difference between generic and customized software

• The generic software product specifications are produced internally by the marketing department of the product company. They reflect what they think will sell. They are usually flexible and non-prescriptive.

• For customized systems are often the basis for the contract between customer and developer. They are usually defined in detail and changes have to be negotiated and carefully costed.

3

What are the main ingredients of Software Engineering.

4

What are the main ingredients of Software Engineering

• Principle

• Methods and Techniques

• Methodology

• Tools

• How above are correlated…

5

• Relationship Between Principal, Methods and Techniques, Methodology and Tools

6

What are the key challenges facing software engineering?

7

What are the key challenges facing software engineering?

• Heterogeneity, delivery and trust.• Heterogeneity

– Developing techniques for building software that can cope with heterogeneous platforms and execution environments;

• Delivery– Developing techniques that lead to faster delivery of software;

• Trust– Developing techniques that demonstrate that software can be trusted by its

users.

8

SOFTWARE EVOLUTION

9

Software change

• Software change is a common requirement – New requirements emerge when the software is used;– The business environment changes;– Errors must be repaired;– New computers and equipment is added to the system;– The performance or reliability of the system may have to be improved.

• A key problem for organisations is implementing and managing change to their existing software systems.

10

Importance of evolution

• Organisations have huge investments in their software systems - they are critical business assets.

• To maintain the value of these assets to the business, they must be changed and updated.

• The majority of the software budget in large companies is devoted to evolving existing software rather than developing new software.

11

Spiral model of evolution

Specification Implemention

ValidationOperation

Start

Release 1

Release 2

Release 3

12

• Program evolution dynamics is the study of the processes of system change.

• After major empirical studies, Lehman and Belady proposed that there were a number of ‘laws’ which applied to all systems as they evolved.

• There are sensible observations rather than laws. They are applicable to large systems developed by large organisations. Perhaps less applicable in other cases.

Program evolution dynamics

13

Evolution processes

• Evolution processes depend on– The type of software being maintained;– The development processes used;– The skills and experience of the people involved.

• Proposals for change are the driver for system evolution. Change identification and evolution continue throughout the system lifetime.

14

Change identification and evolution

Change proposalsNew system

Change identificationprocess

Software evolutionprocess

15

The system evolution process

Releaseplanning

Changeimplementation

Systemrelease

Impactanalysis

Changerequests

Platformadaptation

Systemenhancement

Fault repair

16

Change implementation

Requirementsupdating

Softwaredevelopment

Requirementsanalysis

Proposedchanges

17

WHAT ARE LEGACY SYSTEMS?

25

What are legacy systems?

• Systems developed for a specific client that have been in service for a long-time

• Many of these systems were developed years ago using obsolete technologies

• They are likely to be business critical systems required for normal operation of a business

• These are the systems that everyone worried about when the Y2K concerns became public

26

Background to legacy systems

• Software systems are a major investment

• Systems may be long lived to obtain return on investment (ROI) and thus become “legacy in nature”

• They may be critical for operation of an organisation

• Legacy systems may have evolved over many years (customisations)

27

Definition

• A ‘Legacy System’ refers to any computer system (typically, although not always a mini or mainframe computer system), which has been in existence for a long time.

• ‘Legacy Data’ relates to information collected by an organisation, which is of value to that organisation, but which has been created or stored by the use of software and/or hardware that has been rendered outmoded or obsolete.

28

Socio Technical Systems

• Legacy Systems have been called “Socio Technical Systems”– Systems that include technical systems but also

operational processes and people who use and interact with the technical system. Socio-technical systems are governed by organizational policies and rules.

29

Causal Dimensions of Legacy Status

• System suitability– Suitability to organisation’s

mission– Suitability to organisation

structure– Suitability to process

• Underlying platform– Hardware, Operating system,

Networking, Development environment and Data management

• Software quality– Component quality– Design quality– Change management quality

System Suitability

Software Quality

Underlying platform

30

Legacy Effects–Asset value goes down

•Mission criticality

•reliability

–Ease of operation goes down•User satisfaction

•Ease of testing and auditing

–Ease of migration / evolution

declines•Ease of use of new technology

•Scalability

–Ease of maintenance

declines• Cost of maintenance and

resistance to it

• Availability of resources

• Program size and

complexity

• Dependence on individuals

31

Finding a solution

• Slee and Slovin (1997) proposed a 4R portfolio matrix: -

Low business value Low technology condition

Retire

Low business value High technology condition

Reassess

High business value Low technology condition

Redevelop

High business value High technology condition

Renew

32

Legacy System Replacement

• The business risks associated with the strategy of scrapping a legacy system and replacing it with a new one are not insignificant– legacy systems rarely have complete specifications– changes are not likely to be documented well at all– business processes are reliant on the system– the legacy system may contain embedded business

rules that are not formally documented elsewhere– software development is risky and not is always

successful33

Changing Legacy Systems

• All systems must change to remain useful• Changes to legacy systems can be very expensive

– components may be implemented with different programming styles as changes are implemented

– system may be written in an obsolete language– system documents often out-of-date– system structure may be corrupted by years of maintenance

activities– techniques used to save space or increase speed may have

obscured understandability

– file structures used may be incompatible with each other

34

Legacy System Risks

• Replacing a legacy system is both expensive and risky• Maintaining a legacy system is also expensive and risky• Sometimes a the decision is made (based on the costs

and risks involved) to extend system life-time using techniques like re-engineering

35

Socio-Technical Systems

• Lagacy systems are more than just software systems• Sommerville describes legacy systems as socio-

technical systems• Socio-Technical System Layers

– business processes– application software– support software– hardware

36

Legacy System Structures

• System Hardware– could be a mainframe

• System Software• Application Software• Application Data

– business critical data often used by several programs

• Business Processes– processes that support a business objective and rely on the legacy

systems hardware and software

• Business Policies and Rules– business operation constraints

37

Legacy System Components

Systemhardware

Businessprocesses

Applicationsoftware

Business policiesand rules

Supportsoftware

Application data

ConstrainsUsesUsesRuns-onRuns-on

Embedsknowledge of

Uses

38

System Change

• In theory– it should be possible to replace one layer in a socio-technical system

without making any changes to the other layers

• In practice– changing one layer introduces new facilities that must be used in higher

level layers– changing the software may require hardware changes to maintain system

performance– it may be impossible to maintain hardware interfaces because of the huge

differences between mainframe and client-server architectures

39

Legacy Application System

File 1 File 2 File 3 File 4 File 5 File 6

Program 2Program 1 Program 3

Program 4 Program 5 Program 6 Program 7

40

Database Management System

Program1

Program2

Program3

Program4

Databasemanagement

system

Logical andphysical

data models

describes

41

Transaction Processing

Serialisedtransactions

Teleprocessingmonitor

Accountsdatabase

ATMs and terminals

Account queriesand updates

42

Architecture

• Component based– Not necessarily object-oriented– Good software component and design quality

• Object oriented– Good software component and design quality– Infrastructures may be too ‘leading edge’

• Layered architecture– Promotes flexibility in adapting applications– Requires sophisticated understanding of platforms

43

Dilemma

• Businesses with multiple legacy systems face a dilemma– Continue using Legacy systems - increased maintenance

costs.– Replace Legacy Systems - costly, risks associated with

changing business systems

• Alternative is to use modern software engineering techniques to extend lifetime of legacy systems

44

Legacy System Categories

• Low quality, Low business value– scrap the system

• Low quality, High business value– should be re-engineered or replaced if a suitable

system is available

• High-quality, Low business value– Replace or scrap system, or maintain

• High-quality, High business value– continue operation using normal maintenance practices

45

Legacy System Assessment

• Strategies for obtaining best ROI– Scrap Legacy System : system is not making a

contribution to business– Continue maintaining system: system is stable and

minimal changes being requested– Transform system to improve maintainability: changes

are degrading quality and changes are still required– Replace System: modern systems / technology provide a

viable cost effective alternative

46

Legacy System Assessment

• Low quality, low business value: Expensive to maintain with low business value so candidates for scrapping.

• Low quality, high business value: Expensive to maintain but high business value thus cannot be scrapped. Candidates for system transformation or replacement.

• High quality, low business value: Inexpensive to maintain with low business value. Avoid risk of replacement by maintaining or scrapping.

• High quality, high business value: Must be kept in operation; high quality so transformation or system replacement not necessary. Maintain system.

47

Business Value Assessment

• Establish business value of legacy system by consulting:– End-users: establish level of system usage and

perceived effectiveness.– Customers: establish level of transparency; flexibility;

responsiveness; errors – Line managers: Establish benefits to business unit; are

costs justified; criticality of system.– IT managers: Establish skills availability; level of

resource usage– Senior managers: Establish the level of the systems

contribution to the business goals48

Business Value Assessment

• System usage: if legacy system usage is low then it has a low business value.

• Business processes: legacy system may not be flexible in accommodating changing business processes, thus has a low business value

• System dependability: if legacy system is unreliable then it has a low business value

• System outputs: business depends on legacy system outputs (high business value), if output can be produced in alternate way (low business value)

49

Environmental Assessment

• Supplier stability: Still in existence? Financially stable? Alternate supplier providing system maintenance?

• Failure rate: Level of failure (reboot v’s random app failure). Hardware / software failures.

• Age: With age comes obsolescence, increased maintenance costs

• Performance: Is performance adequate?• Support requirements: Is a high level of support required? Are

costs high? • Maintenance costs: Costs of hardware/software maintenance

increase with system age.• Interoperability: Are there issues interfacing with other

systems50

Application Technical Assessment

• Clarity of source code (understandable?)• Documentation (consistency, quality)• Data (data model?, level of duplication)• Performance (adequate, poor)• Programming Language (modern/obsolete)• Level of Configuration Management (complete, partial,

none)• Test Data ( data & regression test availability)• Personnel Skills (availability?)

51

Legacy System Layers

• A legacy system may be viewed as a set of layers• Each layer depends on and interfaces with the layer

below• If interfaces are “clean” then “in theory” you should be

able to make changes within a layer without affecting other layers. Rare in practice!

Business processes

Application software

Support software

Hardware52

Summary

• Legacy system: an “old system” still providing essential business services. – Encompass business processes, application software, support software

and system hardware. – May be a collection of applications and shared data using files or

obsolete database management system

• Assess business value and quality of a legacy system hardware/software before decision to replace, transform or maintain the system. – Business value of a system is an assessment of the effectiveness of the

system in supporting business goals. – System Quality is determined by business processes, application

software and hardware and support software.

53

A Software Process is

A structured set of activities required to develop a software system

54

Ad hoc Software Development

• Developing software without planning for each phase, and without specifying tasks, deliverables, or time constraints.

• Relies entirely on the skills and experience of the individual staff for performing the work.

• The software process is constantly changed or modified as the work progresses.

55

Software Process Model

A Software process Model which is

“an abstract representation of a process. It presents a description of a process from some particular perspective.”

It provides guidelines to organize how software process activities should be performed and in what order.

56

The Capability Maturity Model

• What is the Capability Maturity Model (CMM)?– The application of process management and quality

improvement concepts to software development and maintenance.

– A guide for evolving toward a culture of engineering excellence.

– A model for organizational improvement.

57

• Focuses on practices that are under control of the software group

• Presents a minimum set of recommended practices that have been shown to enhance a software development and maintenance capability– It defines the expectation (the “what”)– Without overly constraining the implementation (the “how”)

Capability Maturity Model

58

Why We Chose CMM

• CMM today serves as a “seal of approval” in software development

• CMM helped guide us towards standard, repeatable processes – reduced learning time on how to get things done

• Standard practices mean time savings to our team - everyone knows what to expect and what to deliver

• Our quality activities became more aligned within the project rather than thought of as a separate event

• We rely on our processes and our people together, not just one or the other

• Ideas in CMM creates an environment of improvement – if you don’t like things one way, make it better!

59

Level Focus Process Areas

5 OptimizingContinuousProcess Improvement

Organizational Innovation and DeploymentCausal Analysis and Resolution

4 Quantitatively Managed

Quantitative Management

Organizational Process PerformanceQuantitative Project Management

3 Defined ProcessStandardization

Requirements DevelopmentTechnical SolutionProduct IntegrationVerificationValidationOrganizational Process FocusOrganizational Process DefinitionOrganizational TrainingIntegrated Project Mgmt (with IPPD extras)Risk ManagementDecision Analysis and ResolutionIntegrated Teaming (IPPD only)Org. Environment for Integration (IPPD only)Integrated Supplier Management (SS only)

Requirements ManagementProject PlanningProject Monitoring and ControlSupplier Agreement ManagementMeasurement and AnalysisProcess and Product Quality AssuranceConfiguration Management

2 ManagedBasicProject Management

1 Initial

QualityProductivity

RiskRework

Stages of Process Maturity

60

Level 1: the “Initial” Level Success depends on heroes

Good performance is possible - but• Requirements often misunderstood, uncontrolled• Schedules and budgets frequently missed• Progress not measured• Product content not tracked or controlled• Engineering activities nonstandard, inconsistent• Teams not coordinated, not trained• Defects proliferate

61

CMMI Level 2: “Managed”

Baseline the product requirements Baseline the product requirements

Estimate project parameters,Estimate project parameters,Develop plans and processesDevelop plans and processes

Measure actual progress to enable Measure actual progress to enable timely corrective actiontimely corrective action

Measure for mgmt. info needsMeasure for mgmt. info needsVerify adherence of processes Verify adherence of processes and products to requirementsand products to requirements

Identify and control products,Identify and control products, changes, problem reports changes, problem reports

Select qualified suppliers / vendors; Select qualified suppliers / vendors; manage their activitiesmanage their activities

CLARIFY REQUIREMENTSCLARIFY REQUIREMENTS

DOCUMENT PLANSDOCUMENT PLANS

TRACK PROGRESSTRACK PROGRESS

CONTROL PRODUCTSCONTROL PRODUCTS

7 Process Areas7 Process Areas

– Requirements Management Requirements Management (REQM)(REQM)

Project Planning Project Planning (PP)(PP)

– Project Monitoring Project Monitoring and Control and Control (PMC)(PMC)

– Measurement & AnalysisMeasurement & Analysis (M&A)(M&A)– Process & ProductProcess & Product

Quality Assurance Quality Assurance (PPQA)(PPQA)

– Configuration Configuration Management Management (CM)(CM)

– Supplier Agreement Supplier Agreement Management Management (SAM(SAM))

62

What Happens During Level 2

• Processes become easier to digest and understand• Managers and team members spend less time

explaining how things are done and more time doing• Projects are better estimated, better planned, and

more flexible• Quality is integrated into the project• Costs may go up initially, but do go down over time• And yes, there may be more documentation and

paper

63

• Clarify customer requirements• Solve design requirements; develop implementation processes• Assemble product components, deliver• Ensure products meet requirements• Ensure products fulfill intended use• Analyze decisions systematically

• Follow integrated, defined processes• Identify and control potential problems

• Establish org. responsibility for PI• Define the org’s best practices• Develop skills and knowledge

ENGINEER THE PRODUCTENGINEER THE PRODUCT

MANAGE THE PROCESSESMANAGE THE PROCESSES

PROVIDE ORG. INFRASTRUCTUREPROVIDE ORG. INFRASTRUCTURE

CMMI Level 3: “Defined” 11 Process Areas*11 Process Areas*

– Requirements Definition Requirements Definition (RD)(RD)– Technical Solution Technical Solution (TS)(TS)

– Product IntegrationProduct Integration (PI)(PI)– VerificationVerification (Ver)(Ver)– ValidationValidation (Val)(Val)– Decision AnalysisDecision Analysis

& Resolution & Resolution (DAR)(DAR)

– Integrated Project MgmtIntegrated Project Mgmt (IPM) (IPM) – Risk ManagementRisk Management (RSKM)(RSKM)

– Org. Process FocusOrg. Process Focus (OPF) (OPF) – Org. Process Definition Org. Process Definition (OPD)(OPD)– Org. TrainingOrg. Training (OT)(OT)

64

What Happens During Level 3

• Process Improvement becomes the standard – Cross-Functional teams look for ways to “short-cut” the system

• Solutions go from being “coded” to being “engineered”• Quality gates appear throughout the project effort with

the entire team involved in the process, reducing rework• Risks are managed and don’t take the team by surprise

65

MANAGE PROJECTS QUANTITATIVELYMANAGE PROJECTS QUANTITATIVELY

MANAGE THE ORGANIZATION MANAGE THE ORGANIZATION QUANTITATIVELYQUANTITATIVELY

CMMI Level 4: “Quantitatively Managed”

• Statistically manage the project’s processes and sub-processes

• Understand process performance; quantitatively manage the organization’s projects


– Quantitative ProjectQuantitative ProjectManagementManagement(QPM)(QPM)

– OrganizationalOrganizationalProcess PerformanceProcess Performance (OPP)(OPP)

66

CMMI Level 5: “Optimizing”

• Identify and eliminate

the cause of defects early

• Identify and deploy new tools and process improvements to meet needs and business objectives

OPTIMIZE PERFORMANCEOPTIMIZE PERFORMANCE

ADOPT IMPROVEMENTSADOPT IMPROVEMENTS


– Causal AnalysisCausal Analysisand Resolutionand Resolution (CAR)(CAR)

– Organizational Innovation Organizational Innovation and Deploymentand Deployment (OID)(OID)

67

The CMM Maturity Levels

Maturity Level 1

~Maturity Level 2

~ ~~Maturity Level 3

~~~~ ~ ~

Maturity Level 4

~~~~~ ~ ~

Maturity Level 5

68

Proving Maturity Levels

• Five characteristics must be demonstrated in each practice to be assessed in that maturity level practice areas:– Commitment to Perform – Policies, procedures, and resources to perform

the work– Ability to Perform – Personnel, tools, and templates in place– Activities Performed – Documentation and interviews demonstrating that

policies are implemented– Measurement and Analysis – Metrics and other tools used to evaluate

effectiveness of processes– Verifying Implementation – Independent review and evaluation of the

processes• Maturity levels are proven through documentation (policies,

procedures, templates) and interviews of staff (to prove institutionalization).

69

Implementation Best Practices

• Be Realistic – Some processes will be more ready than others.

• Be Flexible – Allowing tailoring is key to adoption.

• Be Open – The key is to learn how to do things better, not how to “comply”.

• Be Patient – It does not happen overnight.

70

Classifications of SDLC ModelSDLC Model

Sequential Iterative

WaterfallV Model

Progressive

71

SW Process Models

• Waterfall model• Evolutionary or Propgressive models• Component-based development model• Iterative Model

72

The Waterfall Model

• Oldest model, it’s been around since 1970.

• Called “Linear Sequential Model”.

• Most widely used model for SW engineering

• Documentation is produced at each stage.

73

Phases

1. Requirements analysis and definition

2. System and software design

3. Implementation and unit testing

4. Integration and system testing

5. Operation and maintenance

74

Waterfall model diagram

Requirements

Operation & Maintenance

Test & Integration

Code & Unit Test

Design

75

Disadvantages

• Inflexible partitioning of the project into distinct stages makes it difficult to respond to changing customer requirements.

• Only appropriate when the requirements are well-understood and changes will be fairly limited during the design process.

• The waterfall model is mostly used for large systems engineering projects.

76

Evolutionary Models

77

The Exploratory Model

Objective is to work with customers and evolve a final system from an initial outline specification.

Should start with well-understood requirements and add new features as proposed by the customer.

78


Concurrentactivities

ValidationFinal

version

DevelopmentIntermediate

versions

SpecificationInitial

version

Outlinedescription

79


• Problems– Lack of process visibility;– Systems are often poorly structured;

• Applicability– For small or medium-size interactive systems;– For parts of large systems (e.g. the user interface);– For short-lifetime systems.

80

The Prototyping Model

When a customer defines a set of general objectives for a software but does not identify detailed input, processing, or output requirement.

It consists of the iterating phases:1. Requirements gathering2. Design and build SW prototype3. Evaluate prototype with customer4. Refine requirements

81


82


• Advantages– Users get a feel for the actual system– Developers get to build something immediately– Specifications can be developed incrementally

• Disadvantages– The developer may make implementation compromises in

order to get a prototype working quickly.– The process in not visible (few documents that reflect every

version of the system)– Systems poorly structured

83

Component Based Software Engineering (CBSE)

• Based on systematic reuse where systems are integrated from existing components.

• Process stages– Component analysis;– Requirements modification;– System design with reuse;– Development and integration.

• This approach is becoming increasingly used as component standards have emerged.

84


Requirementsspecification

Componentanalysis

Developmentand integration

System designwith reuse

Requirementsmodification

Systemvalidation

85


• Advantages:– Reduce amount of software to be developed– Reduce costs and risks– Faster delivery

• Disadvantages:– Requirements compromises, system does not meet real

needs of users– Limited features

86

Iterative Models

87

The Incremental Model

Rather than deliver the system as a single delivery, the development and delivery is broken down into increments with each increment delivering part of the required functionality.

User requirements are prioritised and the highest priority requirements are included in early increments.

Once the development of an increment is started, the requirements are frozen though requirements for later increments can continue to evolve.

88


Validateincrement

Develop systemincrement

Design systemarchitecture

Integrateincrement

Validatesystem

Define outline requirements

Assign requirements to increments

System incomplete

Finalsystem

89


Advantages:• Customer value can be delivered with each increment so

system functionality is available earlier.• Early increments act as a prototype to help elicit

requirements for later increments.• Lower risk of overall project failure.• The highest priority system services tend to receive the

most testing.

90


Disadvantages:

• Increments should be relatively small (20,000 lines of code)

• Can be difficult to map the customer’s requirements onto increments of the right size

• Hard to identify common functions

91

The Spiral Model

• Defined by Barry Boehm in his 1988 article A Spiral Model of Software Development and Enhancement.

• Process is represented as a spiral rather than as a sequence of activities with backtracking.

• Each loop in the spiral represents a phase in the process.

• Suitable for large, expensive and complicated projects

92

http://en.wikipedia.org/wiki/Barry_Boehm

The Spiral Model

Riskanalysis

Riskanalysis

Riskanalysis

Riskanalysis Proto-

type 1

Prototype 2

Prototype 3Opera-tionalprotoype

Concept ofOperation

Simulations, models, benchmarks

S/Wrequirements

Requirementvalidation

DesignV&V

Productdesign Detailed

design

Code

Unit test

IntegrationtestAcceptance

testService Develop, verifynext-level product

Evaluate alternatives,identify, resolve risks

Determine objectives,alternatives and

constraints

Plan next phase

Integrationand test plan

Developmentplan

Requirements planLife-cycle plan

REVIEW

93

The Spiral Model

• Objective setting– Specific objectives for the phase are identified.

• Risk assessment and reduction– Risks are assessed and activities put in place to reduce the key risks.

• Development and validation– A development model for the system is chosen which can be any of

the generic models.

• Planning– The project is reviewed and the next phase of the spiral is planned.

94

The Spiral Model

• Risk driven process model– Different risk patterns can lead to choosing different process

models

• What is a risk?– Situations or possible events that may cause a project to fail to

meet its goal. – Example risks:

• Experienced staff leave the project

• Hardware which is essential for the system will not be delivered on schedule

– (more about risks in Chapter 3)95

The Spiral Model

Advantages:

• Risks are explicitly assessed and resolved throughout the process.

• Software engineers can start working on the project earlier rather than wading through a lengthy early design process.

96

The Spiral Model

Disadvantages:

• Requires highly skilled people in risk analysis and planning

• Requires more time, and is more expensive

• Estimates of budget and time are harder to judge at the beginning of the project since the requirements evolve through the process

97

software evolution_se lect3 btech

Education

existing software systems

new software

customized software

type of software

building software

software budget

organisation legacy

system change