editorial universidad manuela beltrán - pregrados - …...sistemas, certificado internacionalmente...

Editorial Universidad Manuela Beltrán

Software Certification

2018


Autores

Antonio José Segovia Henares

Carlos Augusto Sánchez Martelo

Henry Leonardo Avendaño Delgado

Manuel Antonio Sierra Rodríguez

Carlos Andrés Collazos Morales

Domingo Alirio Montaño Arias

Breed Yeet Alfonso Corredor

José Daniel Rodríguez Munca

Edición


Autores

Antonio José Segovia Henares

Magíster en Seguridad de la

Información, Ingeniería Técnica

Informática Sistemas por la

Universitat Oberta de

Catalunya, Perito Informático,

Especializado En Hacking

Ético, AENOR S16, IS 05, IS

02, IS 01

Carlos Augusto Sanchez

Martelo

Dr. (c) en Pensamiento

Complejo, Maestría en Diseño,

Gestión y Dirección de

Proyectos, Ingeniero de

sistemas, Certificado

Internacionalmente en ITIL Foundation v3,

Procesos en Desarrollo de Software y TIC

Henry Leonardo Avendaño

Delgado

Dr. (c) en Educación línea de

investigación Tecnologías de

la Información y

Comunicación para la

inclusión, Magister en

Educación, Especialista en Gerencia de

Telecomunicaciones, Ingeniero Electrónico.

Manuel Antonio Sierra

Rodríguez

Dr. (c) en Proyectos en la línea

de investigación en

Tecnologías de la Información

y Comunicación, Magíster en

Software Libre, Especialista en

Seguridad en Redes, Ingeniero de Sistemas,

Consultor en Seguridad de la Información y

Comunicaciones.

Domingo Alirio Montaño Arias

Dr. En Genética, Magister en

Biología, Biólogo, Investigador

Asociado, Universidad Manuela

Beltrán, BSc, MSc, PhD

Intereses de investigación en

Neurociencias, Genética y TIC

Aplicadas a la Educación. Miembro comité

editorial revista Journal of Science Educations.

Miembro fundador de la Sociedad Iberoamericana

de Biología Evolutiva.

Carlos Andrés Collazos

Morales

Postdoctorado en Ciencia y

Tecnología Avanzada, Dr. en

Ciencias, Magister en

Ingeniería Electrónica y

Computadores, Físico.

Breed Yeet Alfonso Corredor

Dr. (c) en Proyectos, Magister

en Educación, Especialista en

Desarrollo e Implementación

de Herramientas Telemáticas,

Ingeniera Electrónica,

Directora Académica y

Calidad, Consultora Nacional e Internacional

Académica de Educación Superior.

José Daniel Rodríguez Munca

Magister en Ciencias de la

Educación, Master en

Estrategias y Tecnologías

para el Desarrollo,

Especialista en docencia

mediada por las TIC e

Ingeniero Electrónico.

Daniela Suarez Porras

Corrección de estilo (Editor secundario)

Diagramación: Beatriz del Pilar Mejía

Diseño de portada: Beatriz del Pilar Mejía

Publicado en Diciembre de 2018

Formato digital PDF (Portable Document Format)


Avenida Circunvalar Nº 60-00

Bogotá – Colombia

Tel. (57-1) 5460600

Antonio José Segovia Henares, Carlos Augusto Sánchez Martelo,

Henry Leonardo Avendaño Delgado, Manuel Antonio Sierra

Rodríguez, Carlos Andrés Collazos Morales, Domingo Alirio

Montaño Arias, Breed Yeet Alfonso Corredor, José Daniel

Rodríguez Munca

Software Certification, Bogotá, UMB

© Antonio José Segovia Henares, Carlos Augusto Sánchez Martelo,

Henry Leonardo Avendaño Delgado, Manuel Antonio Sierra

Rodríguez, Carlos Andrés Collazos Morales, Domingo Alirio

Montaño Arias, Breed Yeet Alfonso Corredor, José Daniel

Rodríguez Munca

© Universidad Manuela Beltrán

Bogotá, Colombia

http:// www.umb.edu.co

Queda prohibida la reproducción total o parcial de este libro por

cualquier proceso gráfico o fónico, particularmente por fotocopia,

Ley 23 de 1982

Software Certification. / Antonio José Segovia Henares… (y otros 7) - Bogotá:

Universidad Manuela Beltrán, 2018.

108 p.: ilustraciones, gráficas, tablas; [versión electrónica]

Incluye bibliografía

ISBN: 978-958-5467-21-7

1. Desarrollo de software 2. Patrones de diseño de software 3. Ingeniería de

Software. . i. Sánchez Martelo, Carlos Augusto. ii. Avendaño Delgado, Henry

Leonardo. iii. Sierra Rodríguez, Manuel Antonio. iv. Collazos Morales, Carlos

Andrés. v. Montaño Arias, Domingo Alirio. vi. Alfonso Corredor, Breed Yeet. vii.

Rodríguez Munca, José Daniel.

005.068 cd 23 ed.

CO- BoFUM

Catalogación en la Publicación – Universidad Manuela Beltrán

Autoridades Administrativas

Gerente

Juan Carlos Beltrán Gómez

Secretario General

Juan Carlos Tafur Herrera

Autoridades Académicas

Rectora

Alejandra Acosta Henríquez

Vicerrectoría de Investigaciones

Fredy Alberto Sanz Ramírez

Vicerrectoría Académica

Claudia Milena Combita López

Vicerrectoría de Calidad

Hugo Malaver Guzman

ISBN: 978-958-5467-21-7

13

TABLE OF CONTENTS


Capítulo 1: Quality of the Process ........................................................................................... 19

1. Quality of the Process ........................................................................................................ 19

1.1. Introducción Quality of the process ............................................................................ 19

1.2. Conceptual Framework ................................................................................................... 19

1.2.1. Introduction to the Maturity Model ....................................................................................... 19

1.2.2. Approach About SPICE ........................................................................................................... 19

1.2.3. ISO/IEC 15504 structure .......................................................................................................... 19

1.2.4. Software life cycle processes – ISO/IEC 12207 ................................................................ 20

1.2.5. SPICE certification ................................................................................................................... 20

1.3. Examples (not to exceed 2 pages) ............................................................................... 20

1.4. Exercises (not to exceed 2 pages) ............................................................................... 20

1.5. Conclusions (not to exceed 1 page) ............................................................................ 20

1.6. Maturity Model ................................................................................................................... 21

1.6.1. Introduction to the Maturity Model....................................................................................... 21

1.6.2. Approach About SPICE ........................................................................................................... 26

1.6.3. Structure of ISO/IEC 15504 .................................................................................................... 31

1.6.4. Software Life Cycle Processes – ISO 12207 ...................................................................... 40

1.6.5. SPICE Certification ................................................................................................................... 50

Capítulo 2: Quality of the Product and Service .................................................................... 59

2.1. Introducción ....................................................................................................................... 59

2.2. Conceptual Framework ................................................................................................... 59

2.2.1. Introduction ................................................................................................................................ 59

2.2.2. Quality of the Software Product ........................................................................................... 59

2.2.3. Main Standards About the Quality of the Software Product ......................................... 60

2.2.4. Standard ISO/IEC 25000 (SQuaRE) ...................................................................................... 60

2.2.5. Tools for the Assessment of the Quality of the Software Product .............................. 60

2.2.6. Example of an Environment for the Evaluation of the Software Product .................. 60

2.2.7. Introduction to ISO/IEC 20000 ............................................................................................... 60

2.2.8. Structure of the ISO/IEC 20000 ............................................................................................. 61

2.2.9. Global management of ISO/IEC 20000 ................................................................................ 61

2.2.10. Model and key areas .............................................................................................................. 61

14

2.2.11. Exercises .................................................................................................................................. 61

2.2.12. Conclusions ............................................................................................................................. 62

2.3. Quality of the Product ..................................................................................................... 62

2.3.1- Introduction ............................................................................................................................... 62

2.3.2. Quality of the Software Product ........................................................................................... 63

2.3.3. Main Standards About the Quality of the Software Product ......................................... 65

2.3.4. Standard ISO/IEC 25000 (SQuaRE) ...................................................................................... 77

2.3.5. Tools for the Assessment of the Quality of the Software Product .............................. 83

2.3.6. Example of an Environment for the Evaluation of the Software Product .................. 84

2.3.7. Quality of the Service (ISO 20000) ....................................................................................... 87

2.3.8. Introduction to ISO/IEC 20000 ............................................................................................... 87

2.3.9. Structure of the ISO/IEC 20000 ............................................................................................. 88

2.3.10. Global Management of ISO/IEC 20000 .............................................................................. 91

2.3.11. The PDCA model and key areas ......................................................................................... 97

15

Foreword

Anyone with basic knowledge about programming can develop an application,

and this implies that you can find all kinds of solutions in the market.

The problem is that not everyone applies quality controls when developing

software. Imagine that we want to manufacture a new type of car, by copying

designs from other cars, and without establishing some minimum controls of

security. We can probably sell many units; but when any disaster occurs, problems

may arise.

Therefore, it is important to define a quality model, in terms of software

engineering, with the aim of establishing some controls that ensure the quality of

the developments. As far as it is the main objective of this text, we will focus on

quality models for software processes, products, and services.

We can consider a process for the measurement of the quality, but it is not only

the unique parameter, we can also consider directly the product, and also the

service that the company offers (generally related to the development and

maintenance of software).

So basically during this module, we will see new standards that can help us for

the establishment of quality in our product, and in our service.

17

Capítulo I

Qu

alit

y o

f th

e P

roce

ss

Introduction to the Maturity Model

Approach About SPICE

ISO/IEC 15504 Structure

Software Life Cycle Processes – ISO 12207

SPICE Certification

Quality of the Process

19

CAPÍTULO 1: QUALITY OF THE PROCESS

In this module, we will see the software quality for processes, so we will cover the

SPICE model, which is composed mainly by the ISO/IEC 15504 (evaluation) and

ISO/IEC 12207 (processes) standards.

1. Quality of the Process

1.1. Introducción Quality of the process

In terms of quality, you cannot improve a product or a service if you do not

improve the processes involved.

There are many specific processes in the software area, that can be used to

improve the product or the service, so we will see different models and their

processes.

1.2. Conceptual Framework

1.2.1. Introduction to the Maturity Model

The maturity model allows us to know the maturity of an organization, based on

its processes, in relation to the software quality. However, there is also a model of

capacity, which measures the capacity of each of the software processes within the

organization.

1.2.2. Approach About SPICE

The SPICE model allows to evaluate the maturity and capacity of an organization.

It is similar to CMMI, but it has a big difference. SPICE is based in an ISO

standard, and it has a certification scheme. SPICE defines mainly 6 levels for the

evaluations.

1.2.3. ISO/IEC 15504 structure

ISO/IEC 15504 is composed by 10 different parts, which can be used to

understand the standard better, and to know how to implement it.

20

1.2.4. Software life cycle processes – ISO/IEC 12207

ISO/IEC 12207 defines a group of software processes, which can be used to

improve the quality. SPICE can be based on these processes to perform the

evaluation.

1.2.5. SPICE certification

If you have already implemented SPICE, the next step is to certificate it.

Depending on the processes implemented, you will get a level according to the

SPICE model.

1.3. Examples (not to exceed 2 pages)

AENOR model:

http://www.aenor.es/AENOR/certificacion/calidad/calidad_software_15504.asp#.

WCWsltxhrAo

1.4. Exercises (not to exceed 2 pages)

1.- There are many companies developing software, but do all companies have

the same model for quality system?

2.- If two companies use the same model, do these companies have the same

level for quality?

3.- How can we prove that we have a standard –related to the software quality-

implemented in the organization?

1.5. Conclusions (not to exceed 1 page)

By improving software processes, software quality can be also improved, so our

customers will be more satisfied with our service.

http://www.aenor.es/AENOR/certificacion/calidad/calidad_software_15504.asp#.WCWsltxhrAo

http://www.aenor.es/AENOR/certificacion/calidad/calidad_software_15504.asp#.WCWsltxhrAo

21

1.6. Maturity Model

1.6.1. Introduction to the Maturity Model.

In the software engineering industry, errors are typical. Probably, your computer

or any application you use to surf the internet has shown you an error message

more than once, not allowing you to work with it.

These errors were much more common in the past, because few people worked

on the quality of the product.

In fact, in most of the systems developed in the past, quality focused on detecting

errors in a test phase, once the software was developed.

Nowadays, systems have improved and error detection of errors is not the only

important aspect, but also the definition of process as well as to use them for the

development of the software, in order to improve the quality of the developed

product.

Therefore, it is important to test the product once it has been developed; but it is

also very important to establish a series of processes, to be used for the

development of the product, to ensure the quality of the software.

Figure 1 – Software quality

Software quality

Software

Software process

Error detection

22

The concept of software process includes all the activities that are carried out, the

methods that are defined, and the practices that can be used for the software

development and maintenance.

A process can be basically compared to a black box, where a series of activities

can be carried out, taking a series of entries as the basis, and providing a series of

results as the output.

Figure 2 - Process

Therefore, the activities will provide results on the basis of some parameters.

These activities may consist of the following actions:

Establishment of a methodology and a systematic common work.

Setting of the different roles (developers, analysts, head of project, etc.)

Assignment of responsibilities (for example: programming developers,

analysts for the review of the source code, project manager for the

coordination of the project implementation, etc.)

Definition of the technologies to be used (for example: C++ and Java for

the logic of business, PHP for the web layer, etc.)

Inputs Activities Outputs

23

The main objective of these processes is the development of a final quality

product, which is obviously much better in order to satisfy the needs of current

customers, or to be able to get the attention of future clients.

You can also see this process approach as an opportunity for the positioning on

the market: not all organizations that develop software are really worried about the

quality of the product to develop. Bearing in mind the processes’ approach, we can

give greater quality guarantees.

On the other hand, to ensure that the processes improve the quality of the

product, they have to meet certain requirements:

They have to agree with the results that were expected, or established.

They must be based on a proper definition of the process.

They have to be constantly improved, depending on the organization

objectives.

In addition to gaining a competitive advantage, the processes approach can

provide cost savings, since it may allow an organization to establish an order,

which will involve a significant optimization of the tasks that are carried out within

the organization. This will lead to a time reduction and has a clear effect on the

costs (it is not the same to run a project in 12 months, than in 6 months).

An organization that does not define its processes or does not define them

properly, does not measure, control or improve them, is considered as an

immature organization. Obviously, if the company meets all these aspects, it is

considered a mature organization.

Therefore, the best way of ensuring the quality of a software product is to define

processes by establishing a method to measure control and improve them.

24

Figure 3 – Steps of the process software

In this way, a mature organization, with software-defined, measured, controlled,

and improved processes, will develop software according to the required

specifications, agree with the time limits that were set forth in the project plan,

achieve customer satisfaction, get the staff of the project team united, coordinated

and functioning all in one line.

Another advantage of mature organizations, is that the knowledge is usually kept

in the organization (through defined processes), while immature organizations are

made up by people who can possibly change their company, and therefore bring

their knowledge elsewhere. So they may leave a disclosure of knowledge within

the organization (a company may not depend on one or more people, it must have

its own work strategies, etc.).

Other of the advantages provided by this model, is that different levels of maturity

Defining the process

Measuring the process

Controlling the process

Improving the process

25

can be established, in order to know the degree of maturity of an organization in

relation to its processes.

On the other hand, you can also measure the capacity of each process, i.e., the

degree of capacity that each individual process has.

So, two main models of evaluation can be found:

Assessment of capacity model: Focused on each individual process.

Evaluation of maturity model: Focused on all the processes within the

organization

The evaluation of the maturity model is generally more common, since it gives a

global picture of the organization. However, these models are not incompatible,

i.e., they are often used in a parallel way; thus, the level of maturity of the

organization, and the level of capacity for each of their processes can be known.

Figure 4 – Capacity and maturity models

Therefore, it can be said that the quality of a product will be determined by the

quality of the processes used for its development and maintenance.

• Individual processesCapacity

evaluation

• All processesMaturity

evaluation

26

1.6.2. Approach About SPICE

From now on, it is clear that within the software engineering field, the process

model is a necessary step if we want to offer quality products, as well as to have a

powerful tool to be able to be competitive in the market.

But, how can an organization prove that it has its own processes, and uses them

to develop quality software? The answer is, by getting a certification.

The CMMI-DEV certification offers further insights about the maturity of the

organization, i.e., it uses a model of evaluation of the maturity to determine the

maturity of processes, and it is well known in the software engineering industry.

However, this certification is also known to be especially expensive, and in many

cases, it can also be considered complex, especially for small organizations (that

abound in this sector).

It can be interesting to consider other options, for example SPICE (Software

Process Improvement and Capability Determination), which is very similar to

CMMI-DEV.

SPICE is based on the standard ISO/IEC 15504, and it is used to certify the

maturity of processes in software development, which is basically the same thing

that makes CMMI-DEV.

The difference is that the SPICE certification tends to be cheaper, and also easier

to evaluate. In addition, being based on an ISO standard, makes it easily

integrated with other ISO standards, as for example ISO 27001 (information

security), ISO 22301 (business continuity), ISO 20000 (IT services), ISO 9001

(quality), etc.

27

Another important difference is the certification scheme. In the case of CMMI-

DEV, assessments (known as "appraisal"), are made via the SCAMPI method

(Standard CMMI Appraisal Method for Process Improvement), which identifies

strengths and weaknesses in the processes, and can reveal risks of development.

SPICE follows the certification scheme of management systems: ISO standards,

so that annual audits are performed (there is an initial audit, consisting of a stage 1

and a stage 2, and in the following years, surveillance audits are performed each

year, and a recertification audit is performed every 3 years).

Therefore, since SPICE it is easy to understand and implement, and it is cheaper,

we will give more information about this standard throughout this module.

The different levels that the SPICE model established (you already know that it is

based on ISO/IEC 15504) for capacity are the following:

Figure 5 – Capacity levels

• The process is improved continuouslyLevel 5: Optimized

• The process is managed using quantitative techniquesLevel 4: Predictable

• An adapted process is used, based on a standard processLevel 3: Established

• The process is managedLevel 2: Realizado

• The process is performed and there are clear evidencesLevel 1: Performed

• The process is not completeLevel 0: Incomplete

28

On the other hand, the levels of maturity that ISO / IEC 15504 defines are the

following:

Figure 6 – Maturity levels

The majority of organizations certified in the SPICE model, are found level 2 and

3, being the level 2 the first certifiable level. So, levels 0 and 1 are not certifiable

(especially level 0, which indicates that the processes are not implemented).

On the other hand, the ISO/IEC 15504 also relies on another standard, the

ISO/IEC 12207, which defines a common framework for software life cycle

processes. We will talk more in detail about this standard later.

•The processes are improved continuouslyLevel 5: Optimized

•The processes are managed using quantitative techniques

Level 4: Predictable

•The organization uses adapted processes, based on standards

Level 3: Established

•The processes are managedLevel 2: Managed

•The organization implements the processes and achieves the objectivesLevel 1: Basic

•The organization has not implemented the processesLevel 0: Immature

29

Therefore, an organization of software development based on the SPICE model,

can implement the ISO/IEC 12207 processes.

There is a specific SPICE model focused in the automotive industry, which is

known as Automotive SPICE, and it is composed by specific processes of this

industry.

CMMI-DEV defines the following levels of maturity:

Figure 7 – Capacity levels of CMMI-DEV

As you can observe, it is very similar to the SPICE model. Therefore, it is very

simple to map their levels.

• The processes are improved continuouslyLevel 5:

Optimized

• The processes are measured and controlledLevel 4:

Managed

• The processes are characterized by the organization, and it acts proactively

Level 3: Defined

• The projects are characterized by the processes, and the organization often acts reactively

Level 2: Repeatable

• The processes are unpredictable, poorly controlled, and the organization acts reactivelyLevel 1: Initial

30

Finally, as it was mentioned in a previous section, the SPICE certification scheme

is similar to the ISO management systems certification scheme (ISO 27001, ISO

20000, ISO 9001, ISO 14001, ISO 22301, etc). Therefore, a certification audit of

SPICE is composed by the following types of audit:

Initial audit: Composed by stage 1 (the audit is planned and the

documentation is reviewed), and stage 2 (the audit is performed to verify

that the organization has implemented the system, in this case the SPICE

model).

Surveillance audit: Similar to stage 2 of the initial audit, i.e. the audit is

conducted to check that the organization maintains the system.

Recertification audit: The certificate issued by a certification body usually

has a duration of 3 years, after this period, it is necessary to renew it, and

to repeat the certification process again (without stage 1 and stage 2, that

is, there will be surveillance audits each year, until a new recertification

audit is done.

Figure 8 – Main types of certification audits

Initial auditSurveillance

audit

Recertification audit

31

After each audit, the lead auditor is responsible for issuing a final report with the

results of this audit. This report may contain findings that the company needs to

treat, developing a corrective actions plan.

Once the organization presents the corrective actions plan, if the lead auditor

validates it, he gives a decision for the certification to the certification body, and if

everything is all right, the certification body gives the certificate to the company (it

is valid for 3 years).

The organization must ensure, during the validity time of the certificate, to keep

the system implemented and maintained, and may lose certification if during any

surveillance or recertification audit, it has not worked on the weaknesses identified

during an audit.

There is another type of audit, an unusual one, which is called extraordinary

audit. It is used in those cases where very important findings are detected and the

certification body must verify in situ, after a normal audit, if the organization has

treated and solved these very important findings.

These audits are performed by qualified auditors within certification bodies. They

also have to conduct audits according to another standard: the ISO/IEC 17021. In

other words, this standard defines requirements for the certification bodies and the

activities carried out during the certification audit.

1.6.3. Structure of ISO/IEC 15504

The ISO/IEC 15504, consists of 10 parts.

a.- ISO/IEC 15504-1:2004 Information technology. Process assessment. Part

1: Concepts and vocabulary

32

This first part of the standard provides information about basic concepts related to

the evaluation of processes. On the other hand, it also provides information about

the terminology used in the standard for the evaluation of processes.

Therefore, this standard provides information about basic concepts and

vocabulary that is used in all the standards for the evaluation of the processes.

b.- ISO/IEC 15504-2:2003 Information technology. Process assessment. Part

2: Performing an assessment

It sets the minimum requirements to carry out the evaluation of processes, so it

tends to be used by certification bodies to perform certification audits.

For the evaluation of the process, this part of the standard is based on a model

composed by 2 dimensions:

Dimension of the process: It relies on an external reference model

(usually ISO/IEC 12207, which we will see later), for the definition of a set

of processes.

Dimension of capacity: It defines 6 levels of capacity (which we saw in a

previous chapter), to establish a framework for the measurement of

processes.

Therefore, this allows us to have a series of processes of an external model

(usually ISO/IEC 12207 processes, that are specific to software), and a mechanism

for measuring, composed by 6 levels of capacity.

33

Level 5

Level 4

Level 3

Level 2

Level 1

Level 0

Process A Process B Process C Process D

Figure 9 – 2 dimensions model

On the other hand, to be able to measure the capacity of each process, it is

necessary to use what is known as process attributes.

These process attributes are grouped according to the different levels of capacity;

each attribute measures a particular aspect of the ability of each process.

In this way, we can see the following:

Capacity level ID of the process attribute

Process attribute

Level 0 Incomplete process

N/A N/A

Level 1 Performed process

PA 1.1 Realization of the process

34

Level 2 Managed process

PA 2.1 Management of the realization

PA 2.2 Work product management

Level 3 Established process

PA 3.1 Definition of the process

PA 3.2 Deployment of the process

Level 4 Predictable process

PA 4.1 Measurement of the process

PA 4.2 Control of the process

Level 5 Optimizing

PA 5.1 Innovation of the process

PA 5.2 Continual optimization

Table 1 – Capacity levels and process attributes

It is also appropriate to highlight that an external model can be used for the

definition of processes, but this model has to meet the following requirements:

It must contain documentation of the community that is interested in the

development of the model, and the actions performed to reach a

consensus within this community (this requirement is met by ISO/IEC

12207, given that it has been developed and maintained, by ISO.org).

The processes defined within the model, must have a description and a

unique process ID. In addition, the descriptions of the processes must

describe the general objectives that are trying to achieve with the

processes at a high level, as well as the results that can show that the

process is achieved (it is also complied by ISO/IEC 12207).

c.- ISO/IEC 15504-3:2004 Information technology. Process assessment. Part

3: Guidance on performing an assessment

It provides guidance for an evaluation using ISO/IEC 15504-2 as the reference,

providing an overview of the evaluation of processes.

In addition, this part of the standard also offers a guide to establish the framework

for the process capability measurement, oriented in the selection and use of

assessment tools. It helps to define competencies for evaluators, and also helps

with the verification of the conformity of the assessment.

35

On the other hand, this part of the standard also provides an example of

evaluation according to the ISO/IEC 15504-2 requirements, i.e., the part 2 of the

ISO/IEC 15504.

d.- ISO/IEC 15504-4:2004 Information technology. Process assessment. Part

4: Guidance on use for process improvement and process capability

determination

This part of the standard also provides guidance, but in this case the guide helps

to improve the processes, and also helps to determine the capability of a process.

If we make an analysis of a process based on their results (this is also known as

a result of the process or outcomes), and cross them with the business objectives

of a unit or organization area, we can identify the strengths, weaknesses and risks

that are associated to the processes (it can also help us to determine the ability of

the processes).

Therefore, this analysis can basically help us to determine the ability of the

processes, and the effectiveness of processes in achieving the business’

objectives, which can be used to identify and make improvements on the

processes.

e.- ISO/IEC 15504-5:2012 Information technology. Process assessment. Part

5: An exemplar software life cycle process assessment model

This part of the standard basically shows an example about how to perform an

assessment according to the requirements of part 2 of the ISO/IEC 15504, i.e. in

accordance with the ISO/IEC 15504-2.

The evaluation also relies on the model of the ISO/IEC 12207 processes, and it

establishes a series of indicators for process capability, which can be used to know

whether has achieved the established capacity.

36

f.- ISO/IEC 15504-6:2013 Information technology. Process assessment. Part

6: An exemplar system life cycle process assessment model

This part of the standard is similar to part 5, so this also gives an example about

how to perform an assessment according to the requirements of the ISO/IEC

15504-2; but in this case, it is only as a process reference model the standard

ISO/IEC 15288.

The main difference between ISO/IEC 12207 and ISO/IEC 15288 is that the first

one is focused on software life cycle processes, while the second one focuses on

the system life cycle processes.

Therefore, the main difference between ISO/IEC 15504-6 and ISO/IEC 15504-5,

is that the first one is focused on a model of evaluation of the system life cycle

processes, while the ISO/IEC 15504-5 focuses on a model of evaluation of the

software life cycle processes.

g.- ISO/IEC TR 15504-7:2008 Information technology. Process assessment.

Part 7: Assessment of organizational maturity

This is one of the most important parts of the standard (together with parts 1 and

2), and it mainly establishes part of the standard the minimum requirements for an

assessment of the maturity of an organization. This assessment of maturity relies

mainly on the evaluation of the ability of the processes, so it has an important link

with the ISO/IEC 15504-2.

In this way, an array like the one shown below can be used in order to determine

the level of maturity of an organization:

37

ML3

ML4

ML5

ML2

ML1

1A 1B 1C 2D 2E 2F 3D 3E 3F 4D 4E 4F 5D 5E 5F

Table 2 – Maturity levels

In the vertical line, you can see different levels of capacity (level 1, level 2, level 3,

level 4 and level 5), while in the horizontal line, you can see different categories of

the processes.

Representing each value as follows:

xA: minimum processes to the level of maturity n

xB: additional processes required for the level of maturity n

xC: optional processes for the maturity level of n

Where "n" indicates the level of corresponding maturity.

The crossing of the level of capacity with the corresponding process group,

determines the level of maturity of the organization.

In this way, for example, if the organization wants a level of maturity 3, the

organization should take all processes 1A, 1B, 1C, 2D, 2E, 2F, 3D, 3E and 3F to

reach the level of capacity 3.

Level 5

Level 4

Level 3

Level 2

Level 1

38

On the other hand, as you can see in the chart, to reach the level of maturity 4, it

is not necessary that all processes reach a level of ability 4 (for example, for the

level 4, are not necessary: 1B, 3E, 4D, and 4F).

With regards to the level of maturity 5, you can also see in the chart, that the

processes that are required to reach a level of capacity 5 are 1A, 2E, 3D and 4E,

although the latter is optional.

Therefore, the matrix above can be useful to determine the level of maturity of an

organization based on the levels of each process.

h.- ISO/IEC TS 15504-8:2012 Information technology. Process assessment.

Part 8: An exemplar process assessment model for IT service management

This part of the standard also provides an example of the evaluation of

processes; in this case, it is focused on the ISO/IEC 20000 processes (we will see

this standard in the next module, but it basically establishes the requirements to

define an IT service management system).

For the evaluation of the ISO/IEC 20000 processes, the standard is based on

ISO/IEC 15504-2.

i.- ISO/IEC TS 15504-9:2011 Information technology. Process assessment.

Part 9: Target process profiles

This part of the standard establishes a number of requirements to be able to

determine the ability to achieve the organization's specific needs, using process

profiles.

j.- ISO/IEC TS 15504-10:2011 Information technology. Process assessment.

Part 10: Safety extension

39

This part of the standard adds a security component as an extension to the

previous parts.

Figure 9 – Parts of ISO/IEC 15504

•Conceps and vocabularyISO/IEC 15504-1

•Performing an assessmentISO/IEC 15504-2

•Guidance on performing an assessmentISO/IEC 15504-3

•Guidance on the use for process improvement and process capability determinationISO/IEC 15504-4

•An exemplar software life cycle process assessment modelISO/IEC 15504-5

•An exemplar system life cycle process assessment modelISO/IEC 15504-6

•Assessment of organizational maturityISO/IEC 15504-7

•An exemplar process assessment model for IT service managementISO/IEC 15504-8

•Target process profilesISO/IEC 15504-9

• Safety extensionISO/IEC 15504-10

40

1.6.4. Software Life Cycle Processes – ISO 12207

So far, we have mainly focused on processes, but we need to know what

processes can be used together with ISO/IEC 15004 for the evaluation.

As you know, in this context, the ISO/IEC 12207 standard provides a model of the

software life cycle processes.

This standard consists of the following groups of processes:

Agreement processes

Organizational project-enabling processes

Project processes

Technical processes

Software implementation processes

Software support processes

Software reuse processes

Each of these groups, defines specific processes, which are described below.

Figure 10 – Agreement processes

Acquisition process

Supply process

Agreement processes

41

Acquisition process: Its main aim is to obtain the product or service in

accordance with the requirements established by the customer. So,

basically this process essentially identifies the needs of the client.

Supply process: Its main purpose is to provide the customer with a

product or service that is appropriate to their needs and requirements.

Figure 11 – Organizational project-enabling processes

Life cycle model management process: This process mainly provides a

series of procedures of the software life cycle, processes and policies, which are

consistent with the objectives of the organization, and that are defined, adapted,

improved and maintained to support the individual needs of a project, within the

context of the organization.

Life cycle model management process

Infrastructure management process

Project portfolio management process

Human resource management process

Quality management process

Organizational project-enabling processes

42

Infrastructure management process: This process helps to define,

provide and maintain the organization facilities, tools and resources of

communication and information technologies that are necessary for the activities

that plays the organization during the software life cycle.

Project portfolio management process: Its main objective is to define and

maintain a series of projects that may be necessary and sufficient for the strategic

objectives of the organization. Therefore, this process helps to make a proper

investment of resources.

Human resource management process: Its main aim is to provide the

human resources that are necessary for the organization. The process is also the

competence of human resources, which must be in accordance with the needs of

the business and helps to have qualified, specialized and experienced staff in the

software life cycle processes in order to help the organization reach its goals.

Quality management process: The main objective of this process is to

ensure the quality of products and services, which have to fulfil the objectives of

the customer as well as their needs.

Figure 12 – Project processes

Project planning process

Project assessment and control process

Decision management process

Risk management process

Configuration management process

Information management process

Measurement process

Project processes

43

Project planning process: The main objective of this process is to define

and communicate project plans. Therefore, the process describes the

scope of management and technical activities of the project, it also

identifies the outputs, tasks and deliverables of the project, and it also

establishes plans to perform the tasks in the project, including the scope,

and the resources needed to perform the tasks in the project.

Project assessment and control process: The aim of this process is to

monitor the status of the project, and ensure that it is performed according

to the established plans. During a review of the project, it can be

necessary to re-plan the project.

Decision management process: This process is intended to determine

the more commensurate decision with regards to the needs of the

organization, when there are different alternatives. Therefore, it intends to

analyze different alternatives, and the final decision is recorded along with

their reasons, for possible future decisions that are similar.

Risk management process: The main objective of this process is to

manage risks during the product, software, service, or system life cycle.

Configuration management process: This process’ main objective is to

establish and maintain the integrity of the outputs of a process or a project,

and make them available to the interested parties.

Information management process: The main objective of this process is

to manage the information. For doing so, it is necessary to provide

relevant, complete, valid, and confidential information in time, to the parties

concerned during the system life cycle (and where appropriate, also after

the life cycle). The information that can manage this process is the

information related to the project, including technical information of the

organization, agreements, and user information.

Measurement process: This process aims to obtain, analyze and report

data concerning products and processes implemented in the organization,

44

to support effective management of the processes, and objectively

demonstrate the quality of the products.

Figure 13 – Technical processes

Stakeholder requirements definition process: This process has as main

objective the definition of requirements for a system. The process also

transforms the needs of stakeholders in a set of requirements.

Stakeholder requirements definition process

System requirements analysis process

System architectural design process

Implementation process

System integration process

System qualification testing process

Software installation process

Software acceptance support process

Software opeation process

Software maintenance process

Software disposal process

Technical processes

45

System requirements analysis process: The aim of this process is to

transform the stakeholder requirements into a set of technical

requirements of the system.

System architectural design process: The main objective of this process

is to establish the system requirements that must be set for each element

of the system.

Implementation process: This process aims to implement an element of

the system.

System integration process: The main objective of this process is to

integrate the different elements of the system, thus producing a complete

system which satisfies the system design and requirements, as well as the

needs of the client. On the other hand, the elements of the system can be

software, hardware, manuals, and if necessary, other systems.

System qualification testing process: The aim of this process is to

ensure that each requirement of the system has been implemented and is

ready to be delivered.

Software installation process: The main objective of this process is to

install the software according to the requirements.

Software acceptance support process: The aim of this process is to get

that customer accepts that the software complies with the requirements.

Software operation process: The main objective of this process is to

operate the software product in the agreed environment, and also provide

help to customers on the operation of the software product they have

acquired.

Software maintenance process: The aim of this process is to provide

maintenance of the software, in a cost-effective way, to deliver the

software in appropriate conditions.

Software disposal process: The main objective of this process is to

remove or delete an element of a system. The removal must be done with

responsibility, and in accordance with applicable law, agreements,

46

restrictions that may exist in the organization and the stakeholders’

requirements.

Figure 14 – Software implementation processes

Software implementation process

Software requirements analysis process

Software architectural design process

Software detailed design process

Software construction process

Software integration process

Software qualification testing process

Software implementation

processes

47

Software implementation process: The main objective of this process is

to implement specific elements of the system. The result of this process is

a software element that meets the design requirements, and the

requirements of the stakeholders.

Software requirements analysis process: The main objective of this

process is to define the requirements of the elements.

Software architectural design process: The aim of this process is to

provide a design for the software, which will be implemented in accordance

with the established requirements and can be verified.

Software detailed design process: The main objective of this process is

to provide a design that implements the requirements. On the other hand,

the architecture of the software has to be sufficiently detailed in order to

allow the software codification and the corresponding tests.

Software construction process: The aim of this process is to build

software units that adequately reflect the design of the software,

Software integration process: The main objective of this process is to

integrate the units of software and the software components. This

integration is required to meet the functional and non-functional

requirements.

Software qualification testing process: The aim of this process is to

confirm that the integrated software product meets the requirements.

48

Figure 15 – Software support processes

Software documentation management process

Software configuration management process

Software quality assurance process

Software verification process

Software validation process

Software review process

Software audit process

Software problem resolution process

Software support

processes

49

Software documentation management process: The main objective of

this process is to develop and maintain the records produced by a process.

Software configuration management process: The main objective of

this process is to define and maintain the integrity of a process or project

software items, and keep them available to the interested parties.

Software quality assurance process: The main objective of this process

is to ensure that products and processes comply with the defined plans.

Software verification process: The main objective of this process is to

verify and confirm that each product, or each service of process or project,

adequately complies with the requirements.

Software validation process: The main objective of this process is to

validate and confirm the requirements for the specific use of a software

product.

Software review process: The main objective of this process is to keep

stakeholders informed on the progress of the objectives established, and

what can be done to ensure that the product meets the needs of the

stakeholders. So the software revisions are made during the life of the

project.

Software audit process: The aim of this process is to determine the

compliance of the requirements, plans and agreements. So, it is necessary

to perform audits.

Software problem resolution process: The main objective of this

process is to ensure that all of the problems detected, are identified,

analyzed, managed, controlled and solved.

50

Figure 16 – Software reuse processes

Domain engineering process: The main objective of this process is to

develop and maintain models, architectures, and resources for the domain.

Reuse asset management process: The aim of this process is to

manage the life of resources that are reusable.

Reuse program management process: The main objective of this

process is to plan, establish, manage, control, and review the reusable

programs of the organization.

1.6.5. SPICE Certification

Taking into account the information in the previous sections, if we wish to certify

an organization in SPICE, we need to work with the following standards:

Domain engineering process

Reuse asset management process

Reuse program management process

Software reuse

processes

51

ISO/IEC 15504-2: It defines the requirements for an evaluation of process

capability

ISO/IEC 15504-7: It defines the requirements for an assessment of the

maturity of an organization

ISO/IEC 12207: It defines a set of processes, structured by groups, for the

software life cycle.

ISO/IEC 17021: It defines the requirements for a certification body that

performs certification audits. So, this is standard needs to be implemented

only by the certification body.

Obviously, the other parts of the standard ISO/IEC 15504 can be also used, as a

support to implement and certify SPICE, but the essential parts are the ones which

have been summarized here.

Figure 17 – Standards for the SPICE certification

It is important to mention that there are different SPICE certification models,

which can have a different selection of processes for each maturity level.

SPICE certification

ISO 17021

ISO 12207

ISO 15504-2 + ISO

15504-7

52

In other words, to achieve each level of maturity, it is necessary to implement

certain processes of ISO/IEC 12207, with a certain level of ability. But what

processes should be taken into account?

For the AENOR certification body, the processes that must be implemented for

the maturity level of 1, 2 and 3 are the following:

Figure 18 – Processes of the AENOR model, maturity level 1

Maturity level 1

Supply process

Life cycle model management

process

Software configuration management

process

53


Maturity level 2

Stakeholder requirements

definition process

System requirements

analysis process

Project planning process

Project assessment and control process

Configuration management

process

Measurement process

Software quality assurance

process

54


Maturity level 3

Software requirements analysis process

Software architectural design process

System architectural design process

Infrastructure management process

Human resource management process

Risk management process

Decision management process

Software integration process

System integration process

Software veritication process

Software validation process

55

With this model, an organization can be certified SPICE from level 2 to 5, and to

obtain a level you it must have the previous ones, i.e., to achieve level 3, it is

necessary to implement all processes of the maturity level 3, in addition to those of

level 2, and level 1.

57

Capítulo II

Qu

alit

y o

f th

e P

rod

uct

an

d

Serv

ice

Quality of the Product

Quality of the Software product

Quality of the Service (ISO 20000)

Introduction to ISO/IEC 20000

Comunicaciones por Línea Eléctrica

Quality of the Product and Service

59

CAPÍTULO 2: QUALITY OF THE PRODUCT

AND SERVICE

2.1. Introducción

We have seen in the module 1 that the processes are key to obtain quality, but if

we also focus on the product and the service, considering some new parameters

related to the product, and new processes related to the service, we can obtain

better results.

2.2. Conceptual Framework

2.2.1. Introduction

The major interest of any company is to have satisfied customers, and for that, it

is fundamental to offer quality. So, if the company follows an adequate way

focusing on processes, the product, and the service, will be easier to obtain the

satisfaction of their clients.

Anyway, the more important here is the processes and the product, because can

be specific, although the service can be also interesting, but it is generic, and there

are various quality standards related to the service.

2.2.2. Quality of the Software Product

For the quality of the software product, we can follow various models, although

most of them have the same origin.

60

2.2.3. Main Standards About the Quality of the Software Product

The most important standards related to the quality of the software product are

ISO 9126 and ISO 25000, and they have a similar structure, based on

characteristics and attributes.

2.2.4. Standard ISO/IEC 25000 (SQuaRE)

One of the most important standards related to the quality of the software product

is ISO/IEC 25000, which is also known as SQuaRE, although really this standard is

composed of various groups of standards.

2.2.5. Tools for the Assessment of the Quality of the Software

Product

There are various tools that we can use for the assessment of the quality of the

software product, some of them are free.

2.2.6. Example of an Environment for the Evaluation of the

Software Product

It Is important to know how to evaluate the quality of a software product,

considering the characteristics and attributes defined by ISO/IEC 25000, so we will

see an easy example.

2.2.7. Introduction to ISO/IEC 20000

ISO/IEC is an international standard related to the quality of the IT service and

can help us to obtain the satisfaction of our clients. It is the best standard for

software companies because is specifically related to IT.

61

2.2.8. Structure of the ISO/IEC 20000

ISO/IEC 20000 is composed of various parts, defining each one specifics aspects

about the quality of the service.

2.2.9. Global management of ISO/IEC 20000

For the evaluation of the service, ISO/IEC 20000 uses processes related to the IT

service, and these processes are structured in various groups.

2.2.10. Model and key areas

For the management of the IT service, it is important to include the PDCA model,

to keep a continual improvement.

1.3. Examples (not to exceed 2 pages)

Como estandarizar la evaluación de la calidad del producto software - 1:

http://www.javiergarzas.com/2012/10/iso-9126-iso-25000-1.html

Cómo estandarizar la evaluación de la calidad del producto software – 2 :


2.2.11. Exercises

1.- The quality of a product is very subjective, so if you buy a software, how can

you evaluate if it has quality?

2.- We can use some standards related to the software product, for the evaluation

of the quality, but we can use an automatic way?



62

3.- If we think that we can also improve the service, what are the elements related

to the IT service that we need to consider?

2.2.12. Conclusions

The satisfaction of the client is very important, but all clients are different, but we

can establish a generic way, based on international standards, to give them quality,

from the point of view of the processes, the product, and the service.

2.3. Quality of the Product

2.3.1- Introduction

In software engineering is generally accepted that companies improve their

processes software, to improve the final product.

To do this, companies often use CMMI (one of the more widespread, popular and

used worldwide standard), or its equivalent, SPICE, which defines a model of

evaluation of maturity of processes.

As you recall, SPICE saw it in the previous module and based on a series of

levels of capacity and maturity, we could identify the maturity of an organization

(ISO/IEC 15504-7), and we could also identify the ability of each process (ISO/IEC

15504-2).

Now we will see a different approach, but it will also help us to develop software

with quality. Therefore, we will now focus on the quality of the product, which can

offer many guarantees to develop the software with quality, which will obviously

result in customer satisfaction.

63

Figure 1 – How to obtain the customer satisfaction

Therefore, the final idea is not to decide between use the model of processes or

focus only on the quality of the product, the best is to use both perspectives, to get

the development of a software with quality, and obtain the satisfaction of the

customers. So, now we will see the approach of the quality of the product.

2.3.2. Quality of the Software Product

It is clear that everyone prefers a product with quality that a product without

quality. Imagine that we buy a bicycle that is manufactured with quality controls,

and another in which quality controls have not been established.

Probably, the bicycle that has been produced with quality parameters will give us

fewer problems, because a company can give us a guarantee of quality. And if we

have problems, a company can give us a support.

But with the bicycle that has been produced without minimum quality parameters,

generally, nobody can ensure that the bicycle can run adequately.

Final client satisfied

Quality standards

Product

Processes

64

Products that have been developed with quality parameters, generally have a

higher cost, and it is logical, because ensure that quality takes time, and generally

additional resources are necessary.

We can also find a product that has been developed with quality parameters, and

which also fails or does not work well, but in principle the probability of failure is

low, and usually, we will have a support from the company.

In software occurs something similar, because we also have a product, but in this

case, the product is software.

But then, with a software product, how can we determine that it has a good

quality? Also, think that the software product never fails, but is very difficult to use

or understand.

Therefore, it is necessary to define some criteria or quality parameters and unify

them, so that they can be used to evaluate the quality of any software product.

In 1977 Mr. McCall created a list composed by several parameters that can affect

the quality of the software product, and which can be used to evaluate or measure

its quality.

This list was focused solely on the quality of the software product, i.e. not

addressed anything relating to processes, methodologies, etc.

These parameters of quality of the product addressed:

The operation of the software products

The revision of the software products

The transition from software products

65

Figure 2 –McCall model

Subsequently, specifically 1 year later, Boehm developed and published a model

similar to the of McCall, that also considered the "performance" and the "utility" of

the software product, to determine its quality.

Several years more afternoon, on 1991, he began to develop the first standards

for the assurance of the quality of the software product, and these standards were

used as a reference for the current standards.

2.3.3. Main Standards About the Quality of the Software Product

The first standard that was developed, related to the quality of the software

product, based on the models of McCall and Boehm, was the ISO 9126

With the publication of this first ISO standard, companies began to use these

models as a fundamental tool for the development of software product applying

quality parameters.

Operation

Revision

Transition

66

However, the ISO 9126 today really is not used, since it was updated by the ISO

25000, which actually consists of a set of standards, as we will see in the next

chapter.

Figure 3 – Standards related to the quality of the software product

These standards define a quality model mainly composed of 3 components:

Internal: Refers to the quality of the code of the software (development)

External: This refers to the quality of the execution of the software

(production)

Use: Refers to the use of the software (use from the point of view of the

user)

ISO 9126

ISO 25000

Standards related to the quality of the

software product

67

Figure 4 – Quality model of ISO 9126 and ISO 25000

On the other hand, this model also provides other 10 parameters, which are

known as "characteristics", that in turn, each feature, is also composed of multiple

attributes.

These 10 characteristics related to the quality of the software product are the

following:

Functional suitability: Capacity of the software product to provide

functions which meet stated needs, when the product is used under the

specified conditions.

Performance efficiency: Performance relative to a number of resources

that are used, under certain conditions.

Compatibility: Capacity of 1, or various systems or components, to

exchange information and/or perform its functions when they share the

same environment (software or hardware).

Usability: Ability that the software product can be understood, understood,

used, and that appeal is for the user when it is used under certain

conditions.

Internal

External

Use

68

Reliability: Ability of a system or a component, that can perform their

functions required, when it is used under certain conditions and for a

period of time.

Security: Ability to protect information, in such a way that persons, or also

the systems, cannot access to information which does not have access.

Maintainability: Capability of the software product to be modified in a way

to effectively and efficiently, by evolving needs, corrective or perfective.

Portability: Capacity of the software product, or a component, can be

transferred, in an effective and efficient way, from a hardware, software,

operational or use environment, to another.

Figure 5 – 10 characteristics about the quality of the software product

Each of these characteristics, in addition, has a number of attributes, we are also

going to see below in each case:

Functional suitability

Performance efficiency

Compatibility Usability

Reliability Security Maintainability Portability

69

Figure 6 – Attributes of the functional suitability

Functional completeness: Degree in which a set of capabilities covers all

the tasks and objectives of user specified.

Functional correctness: Capacity of the product, or a system, to provide

correct results, and the level of precision required.

Functional appropriateness: Capability of the software product, to

provide a set of functions appropriate for the tasks and objectives specified

by the user.

Fun

ctio

nal

su

itab

ility

Functional completeness

Functional correctness

Functional appropriateness

70

Figure 7 – Attributes of the performance efficiency

Time behavior: Response time, and processing, and ratios of throughput

of a system when it performs its functions under certain conditions, in

relation to an established benchmark.

Resource utilization: Quantities and types of resources used when the

software performs its function under certain conditions.

Capacity: Degree in which the limits of a parameter of a product or

system, comply with the requirements.

Perf

orm

ance

eff

icie

ncy

Time behaviour

Resource utilization

Capacity

71

Figure 8 – Attributes of the compatibility

Co-existence: Product capacity to coexist with other software, in a

common environment, sharing resources.

Interoperability: Ability of two or more systems or components to

exchange information and to use the information exchanged.

Co

mp

atib

ility

Co-existence

Interoperability

72

Figure 9 – Attributes of the usability

Appropriateness recognizability: Capacity of the product that allows the

user to understand whether the software is suitable for their particular

needs.

Learnability: Product capability that allows the user to learn its application.

Operability: Product capability that allows the user to use it easily.

User error protection: Capacity of the system to protect users to commit

errors.

User interface aesthetics: Capacity that the user interface can satisfy the

interaction of the user.

Accessibility: Product capability that allows that it can be used by users

with certain characteristics and disabilities.

Usa

bili

ty

Appropriateness recognizability

Learnability

Operability

User error protection

User interface aesthetics

Accessibility

73

Figure 10 – Attributes of the reliability

Maturity: Capacity of the system to meet the needs of reliability under

normal conditions.

Availability: Capacity of the system, or component, to be available and

accessible when required.

Fault tolerance: Ability of the system, or component, to operate according

to provisions against possible failures of hardware or software.

Recoverability: Capacity of the software product to recover any lost data,

and restore the system in case of interruption or failure.

Rel

iab

ility

Maturity

Availability

Fault tolerance

Recoverability

74

Figure 11 – Attributes of security

Confidentiality: Ability to protect information against unauthorized access.

Integrity: Capacity of the system, or component, to prevent unauthorized

modification.

Non-repudiation: Capacity to demonstrate that actions or events have

taken place, so that these actions or events may not subsequently be

repudiated.

Authenticity: Ability to trace clearly the actions of an entity.

Accountability: Ability to demonstrate the identity of a resource, or a

subject.

Secu

rity

Confidentiality

Integrity

Non-repudation

Authenticity

Accountability

75

Figure 12 – Attributes of the maintainability

Modularity: Ability of a system that allows a change in a component to

have a minimum impact on other components.

Reusability: Capacity of an asset that can permit that it can be used in

more than one software system, or in the construction of other assets.

Analyzability: Ease with which evaluate the impact that can have a

change, in the rest of the software, diagnosing deficiencies, or the causes

of the failure of the software, or by identifying the parts that must be

modified.

Modifiability: Product capability that allows that this be modified

effectively and efficiently, without introducing defects, or without degrading

performance.

Testability: Ease with which you can establish criteria of evidence for a

system or component, and ease with which tests may be done to

determine if these criteria are met.

Mai

nta

inab

ility

Modularity

Reusability

Analyzability

Modifiability

Testability

76

Figure 13 – Attributes of the portability

Adaptability: Capacity of the product that allows to be adapted effectively,

and efficiently, to different hardware environments, software, operational or

use.

Installability: Ease with which a product can install or uninstall without

problems in a given environment.

Replaceability: Capacity of a product so that this can be used instead of

other product software determined, with the same purpose, and in the

same environment.

Port

abili

ty Adaptability

Installability

Replaceability

77

2.3.4. Standard ISO/IEC 25000 (SQuaRE)

Arrived at this point, already know that the more important standard international,

with recognition world, related to the quality of the software product is the ISO / IEC

25000, that also is often know as SQuaRE, by its initial "Software Quality

Requirements and Evaluation". Therefore, we will see a little more detail the

structure of this standard.

ISO/IEC 25000 is actually composed of a series of standards, which can be

grouped into 5 groups:

Division of the quality management: Composed by the group of

standards ISO/IEC 2500n

Division for quality model: Composed by the group of standards ISO/IEC

2501n

Division for the quality measurement: Composed by the group of


Division for the requirements of the quality: Composed by the group of


Division for the quality evaluation: Composed by the group of standards

ISO/IEC 2504n

Figure 14 – Main groups of ISO/IEC 25000 standards

ISO/IEC 2501n ISO/IEC 2502n

ISO/IEC 2503n ISO/IEC 2504n

ISO/IEC 2500n

78

Then we will see the standards that make up each of these groups.

Figure 15 – Standards of ISO/IEC 2500n

The series of standards ISO/IEC 2500n basically defines models, terms and

common definitions that may be used or referenced by the standards of ISO/IEC

25000 series. On the other hand, ISO/IEC 2500n is composed of the following

standards:

ISO/IEC 25000 – Guide to SQuaRE: Defines the architecture of the

SQuaRE, the terminology of the family of standards of this series, including

a summary of each party, as well as reference models.

ISO/IEC 25001 – Planning and Management: Defines a set of

requirements and guidelines, to manage the assessment and specification

of the software product requirements.

ISO/IEC 2500n

ISO/IEC 25000

ISO/IEC 25001

79


The series of standards ISO/IEC 2501n includes detailed models of quality,

including features for the internal, external quality and use of the software product.

On the other hand, ISO/IEC 2501n is composed of the following standards:

ISO/IEC 25010 – System and software quality models: Establishes a

model of quality for the software product, and the quality in use. On the

other hand, it also defines a number of features and quality attributes to

evaluate the software product.

ISO/IEC 25012 – Data Quality model: Establishes a global model for the

quality of the data, which may be applicable to the data that is stored in a

structured way, and furthermore is a part of an information system.

ISO/IEC 2501n

ISO/IEC 25000

ISO/IEC 25001

80


The series of standards ISO/IEC 2502n includes a reference model for the

measurement of the quality of the product, including definitions of quality measures

(internal, external and in use), as well as practical guidelines for its implementation.

On the other hand, ISO/IEC 2502n is composed of the following standards:

ISO/IEC 25020 – Measurement reference model and guide: Establishes

a common model of elements for the measurement of the quality of the

software product, including also a guide to select or develop, and

implement the measures proposed by other ISO (for example, ISO/IEC

9126).

ISO/IEC 25021 – Quality measure elements: It is a set of metric base,

that can be used throughout the software life cycle.

ISO/IEC 25022 – Measurement of quality in use: Establishes specific

metrics to measure the quality of use of the product.

ISO/IEC 2502n

ISO/IEC 25020

ISO/IEC 25021

ISO/IEC 25022

ISO/IEC 25023

ISO/IEC 25024

81

ISO/IEC 25023 – Measurement of system and software product

quality: Establishes specific metrics for the measurement of the quality of

products and software systems.

ISO/IEC 25024 – Measurement of data quality: Establishes specific

metrics for the measurement of the quality of the data.


The series of standards ISO/IEC 2503n allows you to specify quality requirements

which may be used in the process of definition of the software product quality

requirements. On the other hand, ISO/IEC 2503n is composed by the following

standard:

ISO/IEC 25030 – Quality requirements: Provides a series of

recommendations for the establishment of quality requirements for the

software product.

ISO/IEC 2503n

ISO/IEC 25030

82


The series of standards ISO/IEC 2504n provides a set of requirements,

recommendations, and guidelines to perform the process of evaluation of the

software product. On the other hand, ISO/IEC 2504n is composed of the following

standards:

ISO/IEC 25040 – Evaluation reference model and guide: Defines a

generic reference model for evaluation.

ISO/IEC 25041 – Evaluation guide for developers, acquirers and

independent evaluators: Sets a number of requirements and

recommendations for the implementation of the evaluation of the software

product, from the point of view of the developers, from the point of view of

customers (entity that buys the software), and from the point of view of the

independent evaluators.

ISO/IEC 25042 – Evaluation modules: Sets the content of the

documentation that will be used to describe the evaluation module.

ISO/IEC 25045 – Evaluation module for recoverability: Provides a

module for the assessment of the recoverability.

ISO/IEC 2504n

ISO/IEC 25040

ISO/IEC 25041

ISO/IEC 25042

ISO/IEC 25045

83

Moreover, also there is a set of standards that are used as an extension, which

contains aspects more specific, and can be used as a complement to the

standards seen until now.

These complementary standards, are defined between the number ISO/IEC

25050 - ISO/IEC 25099, the most noteworthy being the following:

ISO/IEC 25051 – Requirements for quality of Commercial Off-The-

Shelf (COTS) software product and instructions for testing:

Establishes requirements for the product software COTS.

ISO/IEC 25062 – Common industry format for usability test reports:

Provides basic information on how to report the results of a test of usability

in a specific context of use.

2.3.5. Tools for the Assessment of the Quality of the Software

Product

Some tools that you can use to ensure the quality of the software product are the

following:

ChecKing QA: Allows to control elements of the software development

process, and also allows you to control the elements that can be analyzed,

as for example the source code, documentation, scripts, etc.

Kiuwan: Let's analyze code and measure the technical quality of the

software. It has a scorecard based on the ISO/IEC 9126, it runs in the

cloud, in SaaS mode.

PMD: Allows to analyze statically source code, identifying problems, such

as for example the repetition of code, or the use of nested ifs, etc. Mainly

be used for Java.

Check Style: Also allows analysis of static source code, checking if there

are style rules. Mainly it is also used for Java.

84

SONAR: Allows you to collect, analyze, and visualize source code metrics.

Mainly used for Java, although it provides support for other languages.

Google CodePro Analytix: Allows you to evaluate code, defining metrics,

analyze dependencies, generate unit testing, etc.

Simian: Allows detecting duplicated code.

2.3.6. Example of an Environment for the Evaluation of the

Software Product

Now that you know which are the parameters that can be used to measure the

quality of a software product, we can see a small example, very simple, about how

to assess a software product.

Imagine that we have developed a software, and we want to assess their quality

based on the characteristics and the attributes that we have seen in a previous

section.

If we want to evaluate each of the characteristics and attributes, we can use a table such as the following:

Characteristic Attribute Complies Not

complies Not

required


Functional completeness

Functional correctness

Functional appropriateness


Time behavior

Resource utilization

Capacity

Compatibility Co-existence

Interoperability

Usability

Appropriateness

Learnability

Operability

User error protection

User interface aesthetics

Accessibility

Reliability Maturity

Availability

85

Fault tolerance

Recoverability

Security

Confidentiality

Integrity

Non-repudiation

Authenticity

Accountability

Maintainability

Modularity

Reusability

Analyzability

Modifiability

Testability

Portability

Adaptability

Installability

Replaceability

Table 1 – Evaluation of characteristics and attributes

So, we will assess for each attribute of each characteristic, the software that you

want to evaluate.

So, after completing the table with data:

Characteristic Attribute Complies Not

complies Not

required


Functional completeness 1

Functional correctness 1

Functional appropriateness 1


Time behavior 1

Resource utilization 1

Capacity 1

Compatibility Co-existence 1

Interoperability 1

Usability

Appropriateness 1

Learnability 1

Operability 1

User error protection 1

User interface aesthetics 1

Accessibility 1

Reliability

Maturity 1

Availability 1

Fault tolerance 1

Recoverability 1

Security Confidentiality 1

86

Integrity 1

Non-repudiation 1

Authenticity 1

Accountability 1

Maintainability

Modularity 1

Reusability 1

Analyzability 1

Modifiability 1

Testability 1

Portability

Adaptability 1

Installability 1

Replaceability 1

Table 2 – Evaluation of characteristics and attributes with data

If we now make an analysis of the results obtained, classified by features, we have the following:

Characteristic Complies Not complies Not required Total


3 3


3 3

Compatibility 1 1 2

Usability 5 1 6

Reliability 2 1 1 4

Security 2 2 1 5

Maintainability 1 3 1 5

Portability 1 1 1 3

Total 18 9 4 31

Table 3 – Results of the evaluation

We see, therefore, that we have evaluated a total of 31 attributes, of which 18

comply with their duties (58% of the total), 9 do not meet (29% of the total), and 4

are not required (13% of the total).

With these results, and the definition of acceptance criteria:

Range Value Acceptance

0% - 30% Does not complies requirements

No

31% - 50% Minimally complies requirements

No

87

51% - 70% Complies properly Yes

71% - 100% Exceeds the requirements

Yes

Table 4 – Acceptance criteria

Given that the degree of compliance is 58%, and according to the defined

acceptance criteria table, is within the margin of 51% - 70%, we can conclude that

software adequately meets a minimum quality, so that we can accept it as well.

However, it should be noted that these tables are indicative, and can be developed

with other values.

2.3.7. Quality of the Service (ISO 20000)

We have talked about the improvement of processes, and improving the quality of

the software product, but we need also see the quality from the point of view of the

service.

I.e., if we consider that we need to provide a service to a client, which can be the

development and maintenance of a software product, we can also improve the

quality.

There are several international standards, but we are going to focus on ISO/IEC

20000 because it is one of the international standards with greater recognition,

related to the management of IT services.

2.3.8. Introduction to ISO/IEC 20000

The ISO/IEC 20000 is very similar to ITIL standard, although ITIL mainly defines

a set of best practices for IT service management and ISO 20000 in addition

clearly defines a model of continuous improvement, based on the PDCA cycle.

88

On the other hand, the companies can be certified in ISO/IEC 20000, as they can

be also certified ISO 27001, ISO 15504, or ISO 9001, etc. But they cannot be

certified in ITIL, because it is not an ISO standard (you can certify individuals in

ITIL, but not companies, because ITIL does not have a scheme of certification for

companies).

ISO 20000, like other management system, has many common points with other

ISOs and shares many elements with the ISO 27001 (in fact, one of the processes

included in ISO 20000, is the process of information security management).

This standard aims to improve or ensure the quality of the service so is intended

primarily for companies that provide technology services, such as for example

companies which develop software and offer a service of maintenance of this

software.

ISO 20000 was developed in 2005 and was subsequently revised in 2011, which

is now the current version of this standard. For the development of the first version,

was used as a basis the BS 15000, which was a British standard developed by BSI

(British Standards Institution).

2.3.9. Structure of the ISO/IEC 20000

This standard, similar to the ISO 15504 we saw in module 1, consists of several

parts, which are detailed below:

ISO 20000-1 – Service management system requirements: This is

actually the standard that is certifiable since it defines requirements for the

establishment of a service management system.

ISO 20000-2 – Guidance on the application of service management

systems: Offers a guide to the implementation of a service management

system, based on the requirements of ISO/IEC 20000-1.

89

ISO 20000-3 – Service providers: Provides a guide for the definition of

the scope and applicability of the standard.

ISO 20000-4 – Process assessment model: Offers a process reference

model.

ISO 20000-5 – Exemplar implementation plan for ISO/IEC 20000-1:

Provides an example of the implementation of the requirements of the

ISO/IEC 20000-1.

ISO 20000-9 – Guidance on the application of ISO/IEC 20000-1 to

cloud services: Offers a guide for the implementation of the ISO 20000-1

in service providers that offer cloud services.

ISO 20000-10 – Concepts and terminology: Offers a list of concepts and

terminology of the ISO/IEC 20000.

ISO 20000-11 – Guidance on the relationship between ISO/IEC 20000-

1:2011 and service management frameworks: ITIL@: Provides a guide

to the relationship between the ISO/IEC 20000-1 and ITIL.

90

Figure 20 – Parts of ISO/IEC 20000

• Service management system requirementsISO 20000-1

• Guidance on the application of service management systemsISO 20000-2

• Service providersISO 20000-3

• Process assessment modelISO 20000-4

• Exemplar implementation plan for ISO/IEC 20000-1ISO 20000-5

• Guidance on the application of ISO/IEC 20000-1 to cloud servicesISO 20000-9

• Concepts and terminologyISO 20000-10

• Guidance on the relationship between ISO/IEC 20000-1:2011 and service management frameworks: ITIL@

ISO 20000-11

91

2.3.10. Global Management of ISO/IEC 20000

The most important part of ISO/IEC 20000 is the part 1 (requirements definition),

and the part 2 (best practices for implementing the processes).

Therefore, to be able to manage an IT service, we need the requirements of

ISO/IEC 20000-1 (requirements for the establishment of a PDCA and processes),

and best practices of the ISO/IEC 20000-2 for the implementation of the

processes.

These processes that are based on the ISO/IEC 20000, are mainly grouped into 4

groups:

Service delivery processes

Control processes

Resolution processes

Relationship processes

Each of which contains a series of processes, which we can see below:

92

Figure 21 – Service delivery processes

Capacity management: This process tries to manage that the service is

provided with the agreed capacity. Therefore, in addition to constantly

monitor the service, you perform a trend analysis, i.e. measure the current

capacity, and estimate how it will evolve in the future.

Service continuity and availability management: This process allows

mechanisms so that, in the event that an unforeseen event occurs, the

service is not interrupted, or if it is interrupted, there is a plan to retrieve it.

With respect to availability, it is important that the service is available in the

Service delivery processes

Capacity management

Service continuity and availability management

Service level management

Service reporting

Information security management

Budgeting and accounting for services

93

conditions in which has been established by contract (usually in service

level agreements).

Service level management Allows you to define service levels, from

which the service will be provided. These levels must agree formally

between the customer and the company that provides the service.

Service reporting: The idea of this process is to keep informed, on a

regular basis, the customer about the status of the service by notifying

potential incidents, complaints, etc.

Information security management: The main objective of this process is

to manage all matters related to the information security, considering the

risk management, the information security policy, security information

incidents, etc.

Budgeting and accounting for services: The main objective of this

process is to bring control over budgets and basic accounting related to

the service. I.e. basically aims to take control of the expenses incurred by

the service, and the budgets of possible improvements that may be

needed (this process does not address the economic benefits of the

service).

94

Figure 22 – Control processes

Configuration management: This process sets out a number of elements

of the service configuration, which we can consider as configuration items

to provide the service. For the registration of these configuration items, it is

necessary to define a CMDB, which must ensure its integrity through

periodic revisions. Before launching a new service, it is usually set a base

line with the current configuration of the configuration items of the service.

Change management: This process manages any change that is

produced in the service. To do this, this process should consist of a stage

of request for change, another stage of review, another of approval, and

finally another stage of implementation of the change. During the change,

something goes wrong, we can have a procedure of roll-back.

Release and deployment management: The Mission of this process is to

manage everything concerning to the delivery of the service and their

Control processes

Configuration management

Change management

Release and deployment

management

95

subsequent deployment. For example, if we need to deliver a software

product, first we need to deliver an executable, and after we need to install

it in the production environment of the customer. In this case, can exist a

delivery of emergency, that the client requires with maximum priority.

Figure 23 – Resolution processes

Incident and service request management: With this process, we can

identify possible issues related to the service. In this case, it is also

important to have a knowledge base of the incidents registered and

treated, because if we have an incident that has been treated in the past,

the information registered can help us. It is also important to define

different levels of incidents, and you can assign to each level resolution

times (example: If the incidence is High, resolution time must be less than

24 hours, but if it is Low, the resolution time can be more than 48 hours,

etc.)

Resolution processes

Incident and service request management

Problem management

96

Problem management: This process is similar to the previous one, with

the difference that when an incidence is repeated and is unable to close

permanently, is considered a problem, and in that case, we need to search

the origin of the problem. Here is also important to have a knowledge

base.

Figure 24 – Relationship processes

Business relationship management: This process is related to the

relationship that exists between the company that and its clients. So, it is

necessary that the company establish a channel for possible claims, and

also need to measure the satisfaction of customers with respect to the

service provided.

Supplier management: With this process, we can evaluate suppliers that

we need to offer the service, i.e. if we provide a software maintenance

Relationship processes

Business relationship

management

Supplier management

97

service, we can need to outsource a service cloud, so, we will need to

establish quality requirements, and select a provider based on these

requirements.

2.3.11. The PDCA model and key areas

In addition to the processes you have seen until now, ISO 20000 is based on a

model of continual improvement, like any other ISO management system, for

example, ISO 9001, ISO 27001, ISO 22301, etc.

This model of continual improvement is based on 4 stages or key areas:

Figure 25 – Continual improvement model

This model is also commonly known as the Deming cycle, and in the case of the

management of an IT service, you can start with planning (Plan) of the service

(project plan, definition of activities/resources, development of documentation,

resources), subsequently you need to implement everything that has been planned

Plan Do

CheckAct

98

in the previous stage (Do), then checked that everything that has been

implemented is according to what we planned (Check), and finally if any

inconsistency is detected during the review, you need to perform corrective actions

for the treatment and resolution (Act).

101

GLOSARIO

- AENOR: Certification body with various headquarters in the world, including Latin

America.

- Audit: External review of the system.

- Benchmark: Bank of tests that generally is used to test software.

- BSI: British Standard Institution. It is an international entity that publishes

standards, and also gives certification services.

- Capacity: Ability of a process to meet some requirements.

- CMMI: Software model for quality evaluation.

- Developers: People that develops software.

- Environment: The place where an application is executed, for example an

internal network, or a production section.

- ISO: International organization that develops and publishes ISO standards.

- IT service: Service that is related to the Information Technologies.

- Knowledge base: Database with specific information about some topics.

- Maturity: Ability to offer quality in software developments.

- Outcomes: Results of processes.

- PDCA: Continual improvement model, composed by: Planning, Doing, Checking,

and Acting.

- Process: A set of activities that have several inputs and give several outputs.

102

- Software life cycle: The development of software is composed by different

steps, generally: Requirements analysis, Design of the software, Implementation,

Documentation and testing, and System Operation and maintenance.

- Software product: Result of the development of the software and the product

that can be sold to the customers.

- Source code: Lines of code the software is composed by. This source code can

be written in Java, C++, PHP, or any other programming language.

- SQuaRE: Software Quality Requirements and evaluation. It is a model composed

by various standards for the evaluation of the software product.

103

BIBLIOGRAFÍA

Satish Kini. 2016. So what the heck is ISO 15504 (SPICE) Capability assessment?

United Stated. Servicemanagers.org. Recovered from:

https://servicemanagers.org/iso-15504-spice-capability-assessment/

ISO/IEC 15504. Foundation. Recovered from:

https://en.wikipedia.org/wiki/ISO/IEC_15504

QUALITAT & INFORMATIK. 2016. What is ISO/IEC TR 15504?

Softwareresearch.net. Recovered from:

http://www.softwareresearch.net/fileadmin/src/docs/teaching/SS03/PM/1T3e

wa.pdf

Francisco J. Pino Correa, Mario Piattini Velthuls, Carlos Manuel Fernandez

Sánchez. 2015. Modelo de madurez de ingeniería del software. Editorial de

AENOR ediciones. España

Gonzalez (2014) Diseño de redes telemáticas. RA-MA. España

Xunta de Galicia.(2014) I Jornada sobre Calidad del Producto Software e ISO

25000. Santiago de Compostela: Colexio Profesional de Enxeñaría en

Informática de Galicia

Jarvier Garzas (2012) Cómo estandarizar la evaluación de la calidad del producto

software – 1. http://www.javiergarzas.com/2012/10/iso-9126-iso-25000-

1.html

Jarvier Garzas (2012) Cómo estandarizar la evaluación de la calidad del producto

software – 2. http://www.javiergarzas.com/2012/10/iso-9126-iso-25000-

2.html

https://servicemanagers.org/iso-15504-spice-capability-assessment/


http://www.softwareresearch.net/fileadmin/src/docs/teaching/SS03/PM/1T3ewa.pdf

http://www.softwareresearch.net/fileadmin/src/docs/teaching/SS03/PM/1T3ewa.pdf





104

ISO 25000 : http://iso25000.com/index.php/en/iso-25000-standards/iso-25010

ISO 20000 : https://www.iso.org/obp/ui/#iso:std:iso-iec:20000:-1:ed-2:v1:en

ISO 9126 : https://en.wikipedia.org/wiki/ISO/IEC_9126

León, M. Certificación en Pruebas Software:

http://mti.cucea.udg.mx/sites/default/files/Documento%20de%20titulacion%

20Miguel%20Angel%20Leon.pdf

http://iso25000.com/index.php/en/iso-25000-standards/iso-25010

https://www.iso.org/obp/ui/#iso:std:iso-iec:20000:-1:ed-2:v1:en


http://mti.cucea.udg.mx/sites/default/files/Documento%20de%20titulacion%20Miguel%20Angel%20Leon.pdf

http://mti.cucea.udg.mx/sites/default/files/Documento%20de%20titulacion%20Miguel%20Angel%20Leon.pdf

108

978-958-5467-21-7

editorial universidad manuela beltrán - pregrados - …...sistemas, certificado internacionalmente...

Documents