requirements engineering education: a systematic mapping study

ORIGINAL ARTICLE

Requirements engineering education: a systematic mapping study

Sofia Ouhbi • Ali Idri • Jose Luis Fernandez-Aleman •

Ambrosio Toval

Received: 1 February 2013 / Accepted: 7 November 2013

� Springer-Verlag London 2013

Abstract Requirements engineering (RE) has attracted a

great deal of attention from researchers and practitioners in

recent years. Requirements engineering education (REE) is

therefore an important undertaking if the field is to have

professionals who are capable of successfully accom-

plishing software projects. This increasing interest

demands that academia should provide software engineer-

ing students with a solid foundation in the subject matter.

This paper aims to identify and to present the current

research on REE that is available at present, and to select

useful approaches and needs for future research. A sys-

tematic mapping study was therefore performed to classify

the selected studies into five classification criteria: research

type, empirical type, contribution type, RE activity, and

curricula. A total of 79 papers were selected and classified

according to these criteria. The results of this systematic

mapping study are discussed, and a list of advice obtained

from the REE literature for instructors is provided.

Keywords Software requirements � Requirements

engineering � Education � Systematic mapping study

1 Introduction

Education research is becoming increasingly important not

only for educators but also for practitioners and researchers

in the domain. Top software engineering journals publish

papers on software engineering education. Important

Software and Requirements engineering (RE) Conferences,

such as the International Conference on Software Engi-

neering and the International Conference on RE, include

tracks or workshops on Software or requirements engi-

neering education (REE), respectively.

RE is concerned with the real-world goals for functions

(functional RE) and constraints (nonfunctional RE, e.g.,

constraints on quality and costs) on software systems. It is

also concerned with the relationships that these factors

have with precise specifications of software behavior and

with their evolution over time and across software families

[1]. RE is a multidisciplinary activity that deploys a variety

of techniques and tools at different stages of development

and for different kinds of application domains [2].

Research endeavors in software development have found

that the failures and deficiencies of software systems are

often rooted in the requirements activities undertaken [3–

6]. One of the main causes of this is the lack of appropriate

skills and knowledge of those engaged in RE activities [7].

The correct teaching of RE at university level has therefore

become an important mission for the profession. This

education should ideally be provided as an integrated part

of developing the requisite of RE and software engineering

technical skills, shortly before students become engineers

and enter the workforce [8]. There has thus been an

increasing emphasis on incorporating RE into the univer-

sity curricula of both undergraduate and postgraduate stu-

dents in the past decade [9–13], and a significant number of

studies have been carried out in the area of REE [14–17].

S. Ouhbi (&) � A. Idri

Software Project Management research team, ENSIAS,

University Mohammed V Souissi, Rabat, Morocco

e-mail: [email protected]

A. Idri


J. L. Fernandez-Aleman � A. Toval

Department Informatica y Sistemas, University of Murcia,

Murcia, Spain


A. Toval


123

Requirements Eng

DOI 10.1007/s00766-013-0192-5

In order to present academics and practitioners with

effective methodologies that can be adopted to provide

high quality and relevant RE courses, a preliminary survey

of REE was carried out beforehand [18]. The survey in

question identified the contribution types of 31 articles,

while the paper presented here provides a new and com-

plete systematic mapping study. In addition to the formal

protocol established by systematic mapping studies, the

scope has been broadened, now including 79 papers in

addition to new classification criteria, i.e., RE activities and

RE curricula are now considered. To the best of our

knowledge, no systematic mapping study has been pub-

lished in the field of REE to date. A systematic mapping

study [19] is a defined method with which a classification

scheme can be built and a field of interest structured. It

provides a structure of the type of research reports and

results that have been published by categorizing them. In

this paper, a systematic mapping study has been preformed,

which has allowed us to summarize the approaches pro-

posed for the development of the RE skills and to address

REE deficiencies. This mapping study has additionally

enabled us to discover which RE activities are most fre-

quently addressed in the REE literature and whether the

authors of the selected papers have based their solutions on

model curricula. The primary studies were identified by

using search strings in digital libraries.

The structure of this paper is as follows: Sect. 2 lists

main RE curricula. Section 3 presents the research method

of our study. Section 4 reports the results obtained from the

systematic mapping study. Section 5 discusses the main

findings of this mapping study, presents a list of hints for

RE instructors, and outlines threats to validity. Finally, our

conclusions and future work are shown in Sect. 6.

2 RE in model curricula, bodies of knowledge,

and standards

This section presents an analysis of the current situation in

REE, as regards the most important software engineering

model curricula, bodies of knowledge (BoKs), and stan-

dards. Various organizations have made great efforts to

identify RE knowledge [15]. Many curricula, BoKs, and

standards, including RE content, have been provided to

assist REE scholars to successfully undertake RE activities.

A BoK represents the complete set of concepts, terms, and

activities that a professional domain institutes, while a

model curriculum is a set of guidelines with which to

address education issues [15]. A standard is a technical

specification that has been approved by a recognized

standardization organization with the objective of achiev-

ing the optimum degree of order in a given context [20].

Some examples of the three categories are presented in the

following subsections in order to provide an overview of

the results presented in Sect. 4.8, although this is not an

exhaustive description:

2.1 Model curricula

• The Joint IEEE/ACM Computing Curriculum—Soft-

ware Engineering (CCSE) [10]. This is a basic under-

graduate curriculum in software engineering that

contains a bibliography of information that every soft-

ware engineer should know. One of the programs in this

curriculum aims to help students understanding the

process of determining client needs and translating

them to software requirements.

• The Joint IEEE/ACM Computing Curriculum—Com-

puter Science (CCCS) [11]. This curriculum contains

undergraduate programs in each of the major comput-

ing disciplines. It highlights their commonalities and

differences and describes the characteristics of gradu-

ates from each kind of undergraduate degree program.

Some of the units in this curriculum address software

requirements, specifications, and analysis.

• ACM/AIS Curriculum Guidelines for Undergraduate

Degree Programs in Information Systems (IS) [21].

One of the learning objectives of this curriculum is to

teach undergraduate students how to ‘‘apply informa-

tion requirements specification processes in the broader

systems analysis & design context’’ and how to ‘‘write

clear and concise business requirements documents and

convert them into technical specifications.’’

• Graduate Software Engineering (GSwE2009)—Curric-

ulum Guidelines for Graduate Degree Programs in

Software Engineering [22]. This curriculum is the first

product of Integrated Software & Systems Engineering

Curriculum (iSSEc) Project. It primarily addresses the

education of students for a professional masters degree

in software engineering. Among its guidelines is a

Knowledge Area (KA) that focuses on the comprehen-

sion of software requirements fundamentals, require-

ments elicitation, requirements analysis, requirements

specification, and requirements validation.

2.2 Body of knowledge

• The Software Engineering Body Of Knowledge

(SWEBOK) 12. is a comprehensive description of the

knowledge needed to practice software engineering. It

contains ten KAs such as requirements, design, testing,

construction, and configuration management. It breaks

down each KA into topic areas. SWEBOK refers to RE

Requirements Eng

123

as Software Requirements (SR), and it breaks the RE

KA into SR fundamentals, process, elicitation, analysis,

specification, validation, and practical considerations.

• Software Engineering Education Knowledge (SEEK) [9]

draws on the knowledge areas of the SWEBOK. It

defines SE education BoKs that are appropriate to guide

the development of undergraduate SE curricula. It

contains recommended course sequences, guidelines

for curriculum development, and recommendations for

alternative teaching environments. The KAs of SEEK are

as follows: foundations, requirements, design, construc-

tion, maintenance, process, quality, and management.

• The Requirements Engineering Body of Knowledge

(REBOK). Aoyama et al. [14] has developed the model

and architecture of the body of knowledge of REBOK

and its proof of concept.

2.3 Standards

• IEEE Std 1233-98 [23] for system requirements spec-

ification. This standard was replaced by ISO/IEC/IEEE

29148:2011, which provides additional guidance in the

application of RE and requirements management

processes.

• IEEE Std 1465-98 [24] is a guideline for quality

requirements in software packages.

• IEEE Std 830-98 [25] is a guideline for the production

and content of the software requirements specification.

• ISO/IEC 25030:2007 [26] is a guideline for software

product quality requirements and evaluation. It applies

the quality model defined in ISO/IEC 9126-1 [27], and

it complies with the requirement processes defined in

ISO/IEC 15288 [28], which is a system engineering

standard.

• ISO/IEC TR 24766:2009 [29] is a guideline or RE tool

capabilities.

3 Research methodology

According to Petersen et al. [19], the main goal of a sys-

tematic mapping study is to provide an overview of a

research area and identify the quantity and type of research

and results available within it. A researcher often wishes to

map the frequencies of publication over time to see trends.

A secondary goal may be to identify the fora in which

research in the area has been published.

Figure 1 shows the mapping process, which covers the

search for relevant publications, the definition of a classi-

fication scheme, and the mapping of publications. A

mapping study differs from a systematic literature review

[30], which aims to establish the state of evidence, focus on

identifying best practices, and shows where particular

evidence is missing or is insufficiently reported in existing

studies. This is not the goal of a systematic mapping study,

since the articles are not studied in sufficient detail. Its

main focus is rather on classification, conducting a the-

matic analysis and identifying publication fora [19].

3.1 Research questions

The overall objective of our study is to gain insight into the

solutions designed to address the problems in REE, such as the

problems identified by Memon et al. [31]. In order to obtain a

detailed view on this topic, the systematic mapping study

addresses seven research questions (RQ). Table 1 presents the

seven RQs with their corresponding motivations. These

questions will allow us to categorize the current research in

REE and to identify future areas of research in the field.

3.2 Search strategy

The articles were identified by consulting the following

sources: IEEE Digital Library, ACM Digital Library, and

Science Direct. Google Scholar was also used to seek gray

literature in the field such as white papers and technical

reports. The efficient use of Google Scholar to perform

bibliometric studies has been demonstrated [32]. The fol-

lowing search string was used in order to perform the

automatic search in the digital libraries selected:

‘‘Requirements’’ AND (‘‘Engineering’’ OR ‘‘Elicita-

tion’’ OR ‘‘Specification’’ OR ‘‘Analysis’’ OR ‘‘Valida-

tion’’ OR ‘‘Process’’) AND (‘‘Educat*’’ OR ‘‘Train*’’ OR

‘‘Teach*’’ OR ‘‘Course’’ OR ‘‘Learn*’’ OR ‘‘Curricul*’’

OR ‘‘Guide’’ OR ‘‘Coach*’’ OR ‘‘Explain*’’).

Fig. 1 Systematic mapping

process [19]

Requirements Eng

123

3.3 Study selection

The aim of the selection process was to identify the articles

that are the most relevant to the objective of this mapping

study. When the same paper appeared in more than one

source, it was considered only once according to our search

order. Each paper was retrieved by one author and evaluated

by two authors, in order to decide whether if it should be

included, by considering its title, abstract, and keywords.

Articles that were judged differently were discussed by the

two authors who carried out the evaluation of articles until

an agreement was found. The remaining authors reviewed

the final selection. The Cohen’s Kappa coefficient was used

to calculate the interrater agreement between the two

authors in their evaluation. The Kappa coefficient was 0.95

which, according to Landis and Koch [33], indicates an

almost perfect agreement between the two assessments.

The first step after the articles had been identified was to

eliminate duplicate titles, and titles clearly not related to

the review. The inclusion criteria were limited to the search

string, and the studies that met at least one of the following

exclusion criteria (EC) were excluded:

EC1 Papers that are not focused on education.

EC2 Papers presenting a general focus on software

education.

EC3 Papers about requirements for engineering

education.

Figure 2 shows the search process result. 79 studies

were selected of 1,359 identified studies.

3.4 Quality assessment

The quality assessment (QA) is usually carried out in

systematic literature reviews and less in systematic map-

ping studies. However, in order to enhance our study, a

questionnaire was designed to assess the quality of the

papers selected. The QA was carried out by the two authors

who retrieved the studies. The way in which this ques-

tionnaire was written was inspired by a previous mapping

study [34].

(a) The study provides a contribution toward how REE

can be conducted. The possible answers were ‘‘Yes

(?1)’’ and ‘‘No (?0).’’

(b) The study presents clear solutions to the problems in

REE. The possible answers were ‘‘Yes (?1),’’

‘‘Partially (?0.5),’’ and ‘‘No (?0).’’

(c) The study presents empirical results. The possible

answers were: ‘‘Yes (?1)’’ and ‘‘No (?0).’’

(d) The study has been published in a recognized and

stable publication source. This question was rated by

considering the computer science conference rankings

(CORE) [35] (A, B, and C), and the Journal Citation

Reports (JCR) lists. The possible answers to this

question were

• for conferences, workshops, and symposia:

• (?1.5) if it is ranked CORE A,

• (?1) if it is ranked CORE B,

• (?0.5) if it is ranked CORE C,

• (?0) If it is not in CORE ranking.

Table 1 Research questions

No. Research question Main motivation

RQ1 Which publication channels

are the main targets for

REE research?

To identify where REE

research can be found as

well as the good targets for

publication of future studies

RQ2 How has the frequency of

approaches related to REE

changed over time?

To identify the publication

trends over time of REE

research

RQ3 What are the research types

of REE studies?

To explore the different types

of research reported in the

literature concerning REE

RQ4 Are REE studies empirically

validated?

To discover whether research

on REE has been validated

through empirical studies

RQ5 What are the approaches that

were reported in REE

research that address REE

problems?

To discover the existing REE

approaches reported in the

existing REE literature

RQ6 What are the RE activities

that were addressed in REE

literature?

To identify in which RE

activities REE researchers

are interested in

RQ7 Were REE approaches

reported in literature based

on RE curricula?

To discover if researchers

take into consideration RE

curricula in REE

approaches design

Fig. 2 The primary studies selection process

Requirements Eng

123

• for journals:

• (?2) if it is ranked Q1,

• (?1.5) if it is ranked Q2,

• (?1) if it is ranked Q3 or Q4

• (?0) If it has no JCR ranking.

• for others; (?0).

The score for the quality criterion (d) refers to the fact

that journals are more advantageous than conferences,

workshops, and symposia because the authors believe that

the possibility of publishing work in Q1 or Q2 journal may

be more difficult than in other publication channels. The

sum of the four closed-question scores for each study

provided a final score (an integer between 0 and 5).

3.5 Data extraction strategy and synthesis method

The data extraction strategy was based on providing the set

of possible answers to research questions.

RQ1. To answer this question, publication source and

channel for each paper should be identified.

RQ2. To draw the publication trend, articles should be

classified per publication year.

RQ3. A research type can be classified into the

following categories [30]:

• Evaluation Research: An evaluation or an investiga-

tion of REE approaches is conducted. This also

includes the identification of problems in REE.

• Solution Proposal: A solution to REE problems is

proposed. This solution may be novel or a significant

extension of an existing approach. The potential

benefits and the applicability of the solution are

shown with a small example or a good

argumentation.

• Experience Papers: These papers must express the

author’s personal experience and explain what has

been done and how it was realized in practice.

• Other: e.g., Theoretical papers, opinion papers,

reviews.

RQ4. The empirical research type can be classified into

[36, 37]:

• Case study: An empirical inquiry that investigates a

phenomenon within its real-life context. This term

may refer to single or multiple case studies.

• Survey: A method for collecting quantitative infor-

mation concerning items in REE, e.g., a

questionnaire.

• Experiment: An empirical method applied under

controlled conditions and using students as subjects

in order to observe its effects on REE.

RQ5. An approach can be classified into the following

categories as proposed in [19, 38].

• Method: A means or manner of procedure and a

series of steps taken to acquire knowledge in REE.

• Tool: Anything used as a means to accomplish a task

or purpose in REE.

• Model: A representation of a system that allows for

investigation of the properties of REE to be

investigated.

• Guideline: An indication of policy or procedure by

which a course of action in REE can be determined.

• Framework: A real or conceptual structure intended

to serve as a support or guide for the building of

something that expands the structure into something

useful in REE.

RQ6. Upon considering the SWEBOK [12] RE KA

breakdown, RE activities can be classified into the

following:

• Requirements process. A paper is categorized into

this criterion if it discusses all RE activities in detail.

• Requirements elicitation. If the paper reports how

and where to collect RE.

• Requirements analysis. If the paper discusses

requirements classification, conceptual modeling,

architectural design, and requirements allocation.

• Requirements specification. If the paper shows the

procedure of the production of a document, or its

electronic equivalent, which can be systematically

reviewed, evaluated, and approved.

• Requirements validation. If the paper presents the

process used to examine the requirements documents

to ensure that the right system is being defined.

• Requirements engineering. If the paper does not

discuss any RE activity and if it mentions RE in

general.

RQ7. The word ‘‘curricula’’ in this question refers to

model curricula, bodies of knowledge, and standards

defined in Sect. 2 The answer to this question can be

classified into: SWEBOK, CCSE, CCCS, IS, GSwE,

REBOK, SEEK, IEEE 830, IEEE 1233, IEEE 1465,

ISO/IEC TR 24766, ISO/IEC 25030, or Other.

The synthesis method is based on counting the primary

studies that are classified in each of the RQ responses,

presenting a ranking of the primary studies based on their

QA, and finally, presenting charts for the classification

result. A variety of evaluation approaches have been used

in the analysis of the results and a narrative summary with

which to recount the principal findings of this systematic

mapping study and describe them is presented in the

discussion.

Requirements Eng

123

4 Results

This section describes the results related to the systematic

mapping questions presented in Table 1. Some studies

were chosen to illustrate examples of each RQ’s result. We

consider that they are relevant and make an important

contribution to REE.

4.1 Selection results

Of the 148 articles deeply investigated, 69 papers were

discarded and 79 were finally selected. The 79 papers

identified were analyzed in order to answer the RQs

explained above. Table 2 presents the list of the selected

papers with details of the overall classification results and

their QA.

4.2 RQ1. Which publication channels are the main

targets for REE research?

Table 3 lists all the sources, the different publication

channels, and the number of articles per publication source.

Four different publication channels were identified in

addition to one technical report and a working paper.

About 46 % of the selected papers appeared in workshops,

39 % were presented at conferences, 9 % were published

in journals, and 4 % were presented at symposia. Two

reports were also included in the systematic mapping study.

Note that 38 % of the selected papers were published in the

International Workshop on REE and training (REET).

4.3 RQ2. How has the frequency of approaches related

to REE changed over time?

Figure 3 presents the number of articles published per year

from 1995 to 2012. There are three apparent outliers in the

years 2005 and 2008. The reason for this result is that some

selected papers were published in workshops as it is shown

in Fig. 3. Workshops and especially REET workshop

impacted the trend of REE publications per year.

The REET workshop started in 2005. The next editions

of this workshop were held yearly after 2007, which

explains the drop in publications on 2006. The spike of

other sources on 2008 is explained by the fact that on this

year, 4 articles were presented in the IEEE International

Requirements Engineering Conference. The reader will also

notice that in 1996, 1998, and 2001, no articles were pub-

lished. That could be explained by the fact that REE gained

interest on the beginning of the last decades after the pub-

lication of software engineering BoK and RE standards. The

decrease in 2012 may be explained by the point in time at

which this mapping study was performed and probably does

not reflect the real number of articles in 2012.

4.4 RQ3. What are the research types of REE studies?

Four research types were identified in this systematic

mapping study. Evaluation research (17 articles), solution

proposal (31 papers), experience paper (30 papers), and one

review. Theoretical papers or opinion papers were not

found in the REE literature. Around 77 % of the selected

papers (solution proposal and experience paper) therefore

present a solution to an REE problem, while 22 % evaluate

REE approaches, and one paper reviews the methods

reported in the REE literature. Examples of research types

are presented in the following paragraph.

Regev et al. [8] described their experience in teaching

an RE course. The course used an active, affective, expe-

riential pedagogy giving students the opportunity to expe-

rience a simulated work environment which demonstrates

the social/design-problem complexities and richness of a

development organization in the throws of creating a new

product. Davis et al. [97] proposed an approach that was

designed to facilitate life-long learning and stimulate the

continuous improvement of RE practice. One of the most

significant planned contributions of their project was the

establishment of a test bed in order to integrate RE research

and education which has a direct impact on practice. Hai-

ney et al. [40] evaluated a game to teach requirements

collection and analysis in software engineering at tertiary

education level. The results indicate that the game did not

meet the expectations of Further Education (FE) learners

(college level) to the same degree as the Higher Education

(HE) learners (university level). The approach seems to be

more suited to HE learners in terms of knowledge acqui-

sition, aspects of the game, and perceptions as opposed to

FE learners.

4.5 RQ4. Are REE studies empirically validated?

The result for this question revealed that 34 % of the

studies are not empirical studies because they do not pro-

vide any type of validation and do not present any proof of

concept. About 52 % of the nonempirical studies identified

in this paper suggest solutions with which to enhance REE

without empirically validating the proposed approaches.

Another 44 % of the nonempirical studies report their

authors’ experiences in teaching RE. A review [112] was

also identified, which presents a comparison of practitio-

ners’ perspectives of the usefulness of the RE content

knowledge included in higher education programs. In

around 38 % of the selected papers, experiments were

conducted with students in order to evaluate the effec-

tiveness of the authors’ approach in REE. 6 % of the papers

reported case studies in industry, and 22 % reported the

results of surveys.

Requirements Eng

123

Table 2 Classification

References Classification Quality assessment

P. Channel P. Year Research type Empirical

type

Approach RE activity Curricula (a) (b) (c) (d) Score

[7] Workshop 2004 Evaluation research Survey Model Process CCCS 1 0.5 1 1 3.5

[8] Conference 2008 Experience paper Experiment Method Engineering No 1 1 1 1.5 4.5

[16] Conference 2008 Solution proposal Survey Tool Process Other 1 1 1 1.5 4.5

[17] Conference 2003 Experience paper Experiment Tool Process SWEBOK 1 1 1 1.5 4.5

[31] Conference 2010 Evaluation research Survey Guideline Engineering No 1 0.5 1 0.5 3

[39] Conference 2006 Experience paper Experiment Method Process No 1 1 1 1.5 4.5

[40] Journal 2011 Evaluation research Experiment Method Analysis No 1 0.5 1 2 4.5

[41] Journal 2007 Evaluation research Experiment Tool Engineering No 1 0.5 1 2 4.5

[42] Symposium 2011 Experience paper Survey Model Process No 1 1 1 1.5 4.5

[43] Workshop 2004 Solution proposal Experiment Framework Engineering No 1 1 1 1 4

[44] Conference 2004 Experience paper Survey Method Process No 1 1 1 1 4

[45] Conference 2008 Evaluation research Survey Tool Engineering No 1 0.5 1 1.5 4

[46] Conference 2008 Evaluation research Survey Method Process Other 1 0.5 1 1.5 4

[47] Conference 2000 Experience paper Survey Method Analysis No 1 1 1 1 4

[48] Conference 2005 Experience paper Case study Method Process No 1 1 1 1 4

[49] Conference 2007 Evaluation research Experiment Method Process No 1 0.5 1 1.5 4

[50] Conference 2005 Solution proposal Survey Framework Process No 1 1 1 1 4

[51] Symposium 2008 Evaluation research Survey Model Engineering No 1 0.5 1 1.5 4

[52] Conference 1995 Solution proposal Experiment Tool Elicitation No 1 1 1 1 4

[53] Conference 1999 Solution proposal Experiment Model Analysis No 1 1 1 1 4

[54] Conference 2009 Solution proposal Experiment Method Process SWEBOK 1 1 1 0.5 3.5

[55] Conference 2006 Experience paper Experiment Method Process No 1 1 1 0.5 3.5

[56] Journal 2009 Experience paper No Method Process No 1 1 0 1.5 3.5

[57] Conference 2009 Solution proposal No Model Engineering No 1 1 0 1.5 3.5

[58] Workshop 2008 Experience paper Experiment Framework Engineering No 1 1 1 0 3

[59] Workshop 2008 Experience paper Experiment Guideline Process No 1 1 1 0 3

[60] Workshop 2011 Experience paper Experiment Method Engineering No 1 1 1 0 3

[61] Conference 2009 Experience paper Survey Method Process No 1 1 1 0 3

[62] Workshop 2005 Experience paper Experiment Method Process No 1 1 1 0 3

[63] Conference 2010 Experience paper Experiment Method Engineering No 1 1 1 0 3

[64] Workshop 2010 Solution proposal Survey Method Process No 1 1 1 0 3

[65] Technical report 1997 Experience paper Experiment Tool Engineering No 1 1 1 0 3

[66] Workshop 2005 Experience paper Experiment Guideline Process SEEK 1 1 1 0 3

[67] Workshop 2007 Solution proposal Experiment Method Process Other 1 1 1 0 3

[68] Workshop 2009 Solution proposal Experiment Method Process SWEBOK 1 1 1 0 3

[69] Conference 2003 Experience paper No Method Elicitation IEEE830 1 1 0 1 3

[70] Workshop 2007 Experience paper Experiment Method Process No 1 1 1 0 3

[71] Workshop 2005 Solution proposal Experiment Method Process CCSE 1 1 1 0 3

[72] Working paper 2002 Solution proposal Experiment Framework Process No 1 1 1 0 3

[73] Conference 2011 Solution proposal Experiment Tool Process No 1 1 1 0 3

[74] Conference 2012 Solution proposal Case study Method Engineering No 1 1 1 0 3

[75] Conference 2008 Solution proposal No Tool Elicitation No 1 1 0 1 3

[76] Conference 2000 Experience paper No Method Process No 1 1 0 1 3

[77] Journal 2004 Solution proposal Experiment Tool Specification IEEE1233 1 1 1 0 3

[78] Conference 2005 Evaluation research Survey Method Process CCSE 1 0.5 1 0.5 3

[79] Workshop 2005 Solution proposal Experiment Method Process SWEBOK 1 1 1 0 3

[80] Journal 2010 Solution proposal Survey Tool Elicitation No 1 1 1 0 3

[81] Conference 2011 Evaluation research Experiment Method Analysis No 1 0.5 1 0 2.5

Requirements Eng

123

Note that if the paper selected indicates that it contains a

case study and the authors used students as subjects for

their case study, then according to our classification crite-

ria, it is classified as an experiment. All of these empirical

studies were carried out by using previously trained stu-

dents who had a high level of control as regards the study.

The students first received lectures about an RE topic (for

example, the tools and methodologies used to manage

requirements) and then applied the knowledge and skills

acquired to the design of a system or even the creation of a

physical product in a laboratory environment. The inves-

tigations identified could be repeated under identical con-

ditions. The studies therefore had execution control and

ease of replication, which are the research strategy factors

of an experiment, not a case study. According to Wohlin

et al. [113], ‘‘the difference between case studies and

experiments is that experiments sample over variables that

are being manipulated, while case studies sample from the

variables representing the typical situation.’’ In our opin-

ion, a case study that is carried out with students in the

environments described does not represent a typical situa-

tion that is suitable for the industrial evaluation of RE

methods and tools because the students are manipulated by

the researchers to act in a defined scenario.

Figure 4 shows that around the half of the solution

proposals and experience papers were empirically vali-

dated through experiments. It also shows that in order to

evaluate an existing approach, authors mainly use

Table 2 continued

References Classification Quality assessment

P. Channel P. Year Research type Empirical

type

Approach RE activity Curricula (a) (b) (c) (d) Score

[82] Workshop 2007 Experience paper Case study Method Process No 1 0.5 1 0 2.5

[83] Workshop 2005 Experience paper No Model Process No 1 0.5 1 0 2.5

[84] Workshop 2005 Evaluation research Experiment Method Engineering CCSE 1 0.5 1 0 2.5

[85] Workshop 2011 Evaluation research Experiment Method Engineering No 1 0.5 1 0 2.5

[86] Conference 2012 Evaluation research Survey Method Engineering No 1 0.5 1 0 2.5

[87] Workshop 2010 Evaluation research Experiment Method Process No 1 0.5 1 0 2.5

[88] Workshop 2005 Evaluation research Survey Model Process No 1 0.5 1 0 2.5

[89] Workshop 2007 Evaluation research Case study Model Analysis No 1 0.5 1 0 2.5

[90] Workshop 2007 Evaluation research Case study Method Process No 1 0.5 1 0 2.5

[91] Workshop 2008 Experience paper No Tool Elicitation No 1 1 0 0 2

[92] Workshop 2012 Solution proposal No Method Engineering No 1 1 0 0 2

[93] Workshop 2010 Solution proposal No Method Analysis No 1 1 0 0 2

[94] Workshop 2005 Solution proposal No Guideline Elicitation No 1 1 0 0 2

[95] Workshop 2008 Experience paper No Method Process No 1 1 0 0 2

[96] Workshop 2004 Experience paper No Method Engineering No 1 1 0 0 2

[97] Workshop 2005 Experience paper No Method Process Other 1 1 0 0 2

[98] Workshop 2005 Experience paper No Method Process CCSE 1 1 0 0 2

[99] Workshop 2008 Solution proposal No Method Engineering No 1 1 0 0 2

[100] Conference 2005 Experience paper No Method Elicitation No 1 1 0 0 2

[101] Workshop 2008 Solution proposal No Tool Engineering No 1 1 0 0 2

[102] Workshop 2008 Solution proposal No Model Process No 1 1 0 0 2

[103] Workshop 2010 Solution proposal Survey Method Process IEEE830 1 1 0 0 2

[104] Journal 2012 Solution proposal No Method Engineering No 1 1 0 0 2

[105] Journal 2011 Solution proposal No Tool Engineering No 1 1 0 0 2

[106] Symposium 2010 Solution proposal No Tool Process No 1 1 0 0 2



[109] Conference 2005 Experience paper No Guideline Process IEEE830 1 1 0 0 2

[110] Workshop 2011 Solution proposal No Method Analysis No 1 1 0 0 2

[111] Workshop 2009 Experience paper No Model Process SWEBOK 1 1 0 0 2

[112] Workshop 2005 Review No Method Process No 1 0.5 0 0 1.5

Requirements Eng

123

Table 3 Publication source

Publication source Channel References No. %

International Workshop on requirements engineering

education and training

Workshop [58–60, 62, 64, 66–68, 70, 71, 79, 82–85, 87–91,

93–95, 97, 98, 102, 103, 110–112]

30 37.97

IEEE international requirements engineering conference Conference [8, 16, 17, 45, 46, 57] 6 7.59

Conference on software engineering education & training Conference [31, 54, 55, 78] 4 5.06

ASEE/IEEE rrontiers in education conference Conference [52, 53, 69, 76] 4 5.06

IEEE international conference and workshop on the

engineering of computer-based systems

Conference [44, 48] 2 2.53

Australian workshop on requirements engineering Workshop [7, 43] 2 2.53

International workshop on multimedia and enjoyable

requirements engineering

Workshop [99, 101] 2 2.53

International workshop on advances and applications of

problem frames

Workshop [96] 1 1.27

International workshop on software engineering education

based on real-world experiences

Workshop [92] 1 1.27

International world scientific and engineering academy and

society conference

Conference [86] 1 1.27

Malaysian conference in software engineering Conference [81] 1 1.27

Mexican international conference on computer science Conference [108] 1 1.27

Norsk informatikkonferanse conference Conference [63] 1 1.27

ACM special interest group on computer science education

conference


World academy of science, engineering and technology Conference [109] 1 1.27

International technology, education and development

conference


International business conference Conference [50] 1 1.27

Australian software engineering conference Conference [47] 1 1.27

IEEE international conference on advanced learning

technologies


IEEE international conference on computer science and

automation engineering


International conference on computing in civil engineering Conference [100] 1 1.27

International conference on information technology: new

generations


International conference on intelligent virtual agents Conference [107] 1 1.27

International conference on software engineering Conference [49] 1 1.27

Symposium of the special commission of games and digital

entertainment

Symposium [106] 1 1.27

ACM-IEEE international symposium on empirical software

engineering and measurement

Symposium [51] 1 1.27

Symposium on computer science education Symposium [42] 1 1.27

Computers & education journal Journal [40] 1 1.27

Empirical software engineering Journal [41] 1 1.27

International journal of computer applications Journal [104] 1 1.27

International Journal of Computer Science Issues Journal [105] 1 1.27

International journal of engineering education Journal [77] 1 1.27

Requirements engineering journal Journal [56] 1 1.27

Revista Facultad de Ingeniera Universidad de Antioquia Journal [80] 1 1.27

Technical report Other [65] 1 1.27

Working paper Other [72] 1 1.27

Requirements Eng

123

questionnaires or conduct experiments. Examples of REE

empirical research are mentioned in the following

paragraph.

Beatty and Alexander [45] conducted an experience to

evaluate the existing research on the use of games in

training and their application to RE training. The authors’

experiences, the existing research, and theories from the

field of learning led them to conclude that games appeared

to be transferable and applicable to training in RE. Mead

et al. [68] presented a case study to validate a compre-

hensive teaching model for security Requirements Engi-

neering that centered on the employment of the SQUARE

method [114]. The results showed that when students learn

about security Requirements Engineering using SQUARE,

they have a better understanding of what is needed to

produce more secure software. Svahnberg et al. [51]

investigated students’ abilities to understand and assess

multiple perspective (basic skills, state of practice, and

state of the art) involvement in the requirements selection

process. The authors performed a survey in relation to

requirements selection as part of an Advanced Course on

Requirements Engineering at Blekinge Institute of Tech-

nology. The results indicated that students have a good

understanding of the way industry acts in the context of

requirements selection and that students may work well as

subjects in empirical studies in this area. Memon et al. [31]

attempted to assess the status of REE courses offered in

major public universities in Malaysia. The aim of their

paper was to discover out the problems that students con-

front on an RE course and compare them with those pre-

sented in the REE literature. The survey results confirmed

many of the problems presented in the literature but also

explored insights that may encourage researchers to be

aware of the limitations of the way in which RE courses are

commonly conducted.

4.6 RQ5. What are the approaches that were reported

in REE research that address REE problems?

The majority of all research type approaches describe

methods. About 20 % of REE approaches describe tools

and 16 % describe models. Guidelines (6 %) and frame-

works (5 %) are also proposed as solutions to REE prob-

lems. Figure 5 shows that the majority of authors report

Fig. 3 Number of articles published per year

Fig. 4 Research types and empirical type

Fig. 5 Approaches

Requirements Eng

123

their experience in delivering REE course methods. With

regard to proposing solutions, the authors’ tend to propose

both methods and tools. Models were almost equally

reported in solution proposals, in experience papers and in

evaluation research. The framework approach (in this

context ’framework’ is considered to be a structure that

forms a frame for REE) is mainly reported as an REE

proposed solution, while the guideline approach is reported

as the authors’ approach experience. Some of the REE

approaches reported in the selected papers are listed below.

Callele and Makaroff [39] presented a second-year

computer science course designed to simulate appreciation

for the need for RE and to provide students with an

opportunity to engage in RE activity. Damien et al. [62]

described a course that was taught in collaboration between

three universities in different locations, time zones, and

culture: Canada, Australia, and Italy. The students from the

three locations played the roles of a client and developer

and experienced the iterative development of requirements

specification in global projects. Smith and Gotel [16]

designed the RE-O-Poly game to explain and explore good

RE practices. Since requirements are often conflicting,

players have to learn to resolve conflicts and determine

priorities in the game. Zowghi and Paryani [17] used role

playing as a pedagogical tool to provide their students with

a fuller appreciation of the range of both technical and

nontechnical issues involved in RE practices. Knauss et al.

[101] presented a Web-based simulation game, which was

easy to understand and fun to play within just a few min-

utes in order to take RE more seriously. The Software

Quantum game helps students to understand one of the

principal challenges of RE which is to build the right

system within the available time. Barnes, Gause, and Way

[59] described sample techniques for the teaching of the

unknown (e.g., recommendations about requirements for

disaster recovery) and unknowable (e.g., requirements to

predict future trends) in RE for systems design and pre-

sented their experiences in teaching as part of requirements

analysis in an engineering systems design class. Armarego

[43] presented an approach based on problem-based

learning (PBL) which attempts to provide students with a

solid foundation in the subject matter while simultaneously

exposing them to real-world characteristics. It provides

students with a process to deal with problems within a

metacognitive-rich framework that makes complexity

apparent and allows students deal with it in an adaptive

manner.

4.7 RQ6. What are the RE activities that were

addressed in the REE literature?

About 52 % of the selected papers present approaches for

requirements processes, and 27 % of the selected papers

did not specify any RE activity and were therefore clas-

sified under Requirements Engineering. Requirements

elicitation was reported in 11 % of the selected papers,

9 % discussed requirements analysis, and only one paper

discussed requirements specification, while no require-

ments validation were identified. In Fig. 6, the answers

for RQ3, RQ4, and RQ5 were combined in order to

establish a mapping with the aim of providing an over-

view of RE activities. This mapping has allowed us to

obtain more information on how the results from each RQ

are related to the others. The diagram presents the

research facet related to RE activity, distributed over

approaches and research types. This figure allows us to

conclude that only one tool was proposed as a solution for

requirements specification that authors mainly report their

experience in teaching requirements processes and that

the majority of the solution proposals concern require-

ments processes. In addition, most of the REE tools are

designed for requirements elicitation activities. Further

information regarding the relationship between the three

facets is shown in Fig. 6.

4.8 RQ7. Were REE approaches reported

in the literature based on RE curricula?

Only 24 % of the selected papers take into consideration

RE model curricula, BoKs, or standards. SWEBOK, CCSE,

and IEEE std 830 are the main RE sources for authors.

Other curricula were identified: Business Analyst BOdy of

Knowledge (BABOK), IEEE Std 9003, and Requirements

Engineering Good Practice Guide (REGPG).

4.9 Quality assessment

Table 5 presents the quality assessment score for each

paper. About 59 % of the selected papers had an above

average score, and 13 % had an average score, while 28 %

were below average. This quality assessment may help

REE scholars to choose the relevant papers according to

the criteria defined in Sect. 3.4.

5 Discussion and implication

This section summarizes and discusses the results related to

the systematic mapping study. Advice for instructors is also

proposed in this section.

5.1 Principal findings

The goal of this systematic mapping study was to examine

the current knowledge in REE by selecting 79 papers from

a total of 148. They were then classified according to five

Requirements Eng

123

criteria: research type, empirical type, approach type, RE

activity, and RE model curricula. The principal findings of

our study are the following:

• The REE research area has gained increasing attention

since 2004, and 2005 marked the shift in the REE

publication trend as it was the year in which an

International Workshop on REET began. About 46 %

of the selected papers were published in workshops,

while only 9 % had reached the maturity of a journal

publication. However, upon taking into consideration,

the yearly Chaos Report series conducted by the

Standish Group [3] and the studies [5, 115, 116] that

demonstrate the criticality of RE with regard to the

success of SE projects, we believe that REE will

probably gain much more attention in the future.

• About 77 % of the selected papers reported solutions to

REE. This result shows that the REE field has not yet

attained sufficient maturity for evaluation and that the

main concern for REE researchers is to propose

approaches with which to enhance REE. This is also

shown by the fact that experience papers reporting the

authors’ teaching experience and solution proposals

were the most common research types found in the

literature. The objective of the solutions identified in

this study is principally to address students’ lack of

awareness of RE principles and practices in the

development of software projects [39, 50] and their

lack of interest in RE courses [16, 101].

• The most frequently reported approaches in the selected

papers were methods which were mainly reported as

RE courses. Some authors shared their teaching expe-

rience [8, 44, 70, 76] or their course designs [54, 74,

103] in order to present solutions to REE. Other authors

preferred to evaluate the efficiency of the existing

training and teaching programs [40, 46] and their

impact on REE improvement. Another approach cate-

gory that has received attention was that of tools.

Authors proposed games as teaching tools because

games are fun and engaging and provide friendly

competition [16, 17, 80, 106]. Many of the approaches

identified were suggested to enhance REE and to

replace traditional RE course teaching methods, such as

the use of simulated project examples that do not

provide the students with a realistic experience of

client–developer interaction [8, 42].

Fig. 6 RE activities

Table 4 Curricula

Curricula References Total

SWEBOK [17, 54, 68, 79, 111] 5

SEEK [66] 1

CCSE [71, 78, 84, 98] 4

CCCS [7] 1

IEEE Std 830 [69, 103, 109] 3

IEEE Std 1233 [77] 1

Other [16, 46, 67, 97] 4

Table 5 Quality assessment

References Score Total

[112] 1.5 1

[91–111] 2 21

[81–90] 2.5 10

[31, 58–80] 3 24

[7, 54–57] 3.5 5

[43–53] 4 11

[8, 16, 17, 39–42] 4.5 7

Requirements Eng

123

• In 38 % of the selected papers, the authors conducted

experiences, and in only 6 % of the selected articles did

they include industrial case studies in their research. In

RE, a case study is an observational study which

usually aims to track a specific attribute or identify

relationships between different attributes [113]. The

reduced number of studies identified in this category

can be justified by the amount of people involved and

the inherent difficulty of finding industrial projects.

What is more, in 22 % of the selected papers, the

authors used surveys to collect quantitative information

about REE approaches. About 73 % of the selected

papers were empirical studies and 58 % of these

empirical studies used students as subjects. A common

criticism of controlled experiments in which students

are used as subjects is their external validity, and it is

therefore difficult to generalize the results to industrial

settings [117]. However, the study by Svahnberg et al.

[51] shows that in certain circumstances, it may be

possible to use students as subjects for empirical studies

and to influence them to provide answers that are in line

with industrial practice.

• Around the half of the selected papers presented

approaches for requirements processes. About 11 %

discussed requirements elicitation, 9 % requirements

analysis, and only one paper discussed requirements

specification. No study for requirements validation was

reported. Researchers mainly propose approaches with

which to address the teaching of requirements process

that involves the general aspects of RE. More research

is needed to address the broad teaching of each RE

activity, and more attention should be paid to require-

ments validation.

• The main RE sources identified that inspired authors

were SWEBOK, CCSE, and IEEE std 830. However,

only a quarter of the selected studies took into

consideration the well-known RE model curricula,

BoKs, and standards in the design of their REE

approaches. This result shows that the remaining

approaches do not conform to curricula, BoKs, or

standards.

• About 24 % of the courses reported in the selected

papers were designed for undergraduate students, and

around 16 % were designed for professional engineers,

while only 9 % could be carried out with both

undergraduate and post graduate students. More

research is therefore needed to assess new teaching

methods and tools in postgraduate courses which

requires the acquisition of a more specialized knowl-

edge of the RE discipline.

• The main audience interested in this systematic map-

ping study could be instructors in the field of REE,

since the majority of the REE literature is written by

teachers and instructors. To support this statement, two

points were investigated: the affiliation of the authors

and the publication source. About 84 % of the papers

selected were written by instructors from universities,

while 10 % were written by professionals who are

mainly consultants, and 6 % were written jointly

between instructors and professionals. Only a few

practitioners therefore appear to be interested in REE.

Also, as it was mentioned in the RQ1 result, around

46 % of the selected papers appeared in REE-related

workshops and 39 % in conferences, which are mainly

held for instructors in order to give them the opportu-

nity to share their knowledge and experiences. Observe

that no paper was found in IEEE Transactions on

Education journal, which is one of the most important

journals related to education and computer science.

5.2 Implications and advice for instructors

The findings of our systematic mapping study have

implications for instructors who are working in REE, since

this study will allow them to discover the existing

approaches in the literature concerning REE and to exploit

combined approaches in REE. Moreover, the empirical

studies presented can provide an overview of the efficiency

of each approach. In order to improve the quality of RE

courses, instructors could take into consideration some

advice.

These pieces of advice are addressed toward instructors

since they are probably the main audience interested in this

systematic mapping study, as has been demonstrated in the

previous section. The studies cited in this section were

chosen by considering their content. First, those papers in

the mapping study that provided any kind of practical

guidance for REE were classified among those found in the

‘‘Study selection’’ phase, by reading their abstracts. Sec-

ond, one author carried out an in-depth review of previ-

ously selected papers in order to make recommendations

for the teaching of RE. No predefined templates or

guidelines were used in this screening process. During this

process, important problems in RE training and discussions

addressing key issues in RE were searched for. Recom-

mendations were extracted from the papers as provided by

the authors, and their original purpose and meaning were

maintained.

According to the literature reviewed [16, 31, 39–41, 45,

101], various issues have been identified to be considered

in this section. Other sources have also been identified

[118–128], which are not necessarily REE focused, but are

related to these issues; in that, they provide interesting

complementary information regarding the following

recommendations:

Requirements Eng

123

• Teach how to define scope of the problem and avoid

general and vague specifications [39]. In order to

address this issue, instructors can take into account the

different personalities of students when teaching an RE

course. Instructors can ask the students to answer a

questionnaire in order to discover their personality

traits. The results of this questionnaire will allow the

instructors to form teams which exhibit a better

performance. Previous studies [119] show a significant

positive correlation between the personality factor

extraversion and software product quality, including

satisfied requirements.

• Show how to select and use an RE tool. More than 100

RE tools can be found on the market [120]. Students

should be aware of the features that the current RE tools

provide and should be taught how to select the best tool

for a project, according to its needs: requirements

elicitation, requirements analysis, requirements speci-

fication, requirements verification and validation, and

requirements management.

• Promote activities in requirements analysis and model-

ing in addition to requirements management and

introduce the concept of prototyping in the course.

Prototyping is an important technique in the RE phase

[121]. Through prototyping, an executable model of the

software product can be achieved before the final

product has been created [122]. Moreover, prototypes

presented to stakeholders are very effective in ensuring

valid requirements [123]. A part of the course could be

devoted to prototyping the system described in the

Software Requirements Specification (SRS) and the

enterprise models. Students should rewrite the SRS and

submit an executive summary, including features such

as Return On Investment (ROI) in a nontechnical

language [41]. The difficulties involved in learning

these issues on RE courses have been identified in the

literature [31].

• Involve students in industrial projects in order to allow

them to acquire sufficient knowledge and practice,

especially on postgraduate courses. Instructors could

also invite industry practitioners to present real projects

and to describe their accumulated industrial experience.

The difficulties involved in how to apply RE knowl-

edge in the real world have been reported in the

literature [41, 118]. By providing the student with an

accurate view of reality and the adequate tools,

instructors can offer them the opportunity to attain a

critical mindset in order to deal with RE in practice.

• Have the ability, skills, and strategies needed to align

RE courses with contemporary global software devel-

opment (GSD) conditions. REE must adapt to meet the

changing demands in the global development environ-

ment. Instructors can refer to the experience by Damien

et al. [124] in designing and evaluating a framework to

teach GSD courses and to extract ideas that can be

implemented in RE courses.

• Familiarize students with approaches to problem solv-

ing, development methodologies, and development

tools [31]. Instructors can improve their courses by

using some of the solutions proposed in Sect. 4.6. RE

courses are often given in a traditional lecture/exercise

format [41]. It often occurs that students do not place

importance on RE activities and do not see their impact

on the success or failure of projects [101]. Instructors

are encouraged to include alternatives approaches in the

curricula. Some studies [16, 40, 45, 101, 125] have

shown that learning through play provides a successful

education experience. In general, game playing consists

of rules, goals, engagement, challenge, feedback, fun,

interactivity, outcome. and immediate reward.

• Use mobile devices as teaching tools. Mobile phones

are a particularly attractive avenue for delivering

courses and training in REE. M-learning (learning with

mobile devices) promises a continued extension toward

‘‘anywhere, anytime’’ learning [126]. Instructors can

deliver their courses in an interactive classroom: The

students can share a virtual whiteboard, electronic

textbook, and data over a networked environment to

actively participate in the course discussions [127].

These devices will encourage students to share their

knowledge, and students will be far more active than on

traditional courses. We propose that instructors refer to

the experience of Yau et al. [128] in order to adopt the

concept of a smart classroom which facilitates collab-

orative learning among students and can be adapted to

REE.

5.3 Threats to validity

Four kinds of threats to validity are discussed below.

• Construct validity: Construct threats to validity in a

mapping study are related to the identification of

primary studies [129, 130]. In order to ensure that as

many relevant primary studies as possible were being

included, two authors identified and proposed potential

search keywords in several iterations. Seven terms

related to RE and nine terms related to education were

used in the search string. However, the list might not

have been complete, and additional or alternative terms

might have altered the final list of papers found [131].

The search was performed by using IEEE Digital

Library, ACM Digital Library, Science Direct, and

Google Scholar. According to the statistics of literature

search engines [132], we believe that most of the

research on RE can be found in these electronic

Requirements Eng

123

libraries. To decrease the risk of missing related and

important publications, the authors also sought related

papers in major RE research venues (e.g., REJ, ER,

REFSQ, ICSE, TSE).

We believe that the inclusion of publication sources

that are not top journals and conferences in the review

might decrease the quality standards of the primary

studies, but it signifies that the representativeness of the

primary studies is increased. In particular, the REET

workshop is an essential venue when undertaking a

study on REE, in spite of its reducing the mean score

for primary studies in Table 2 (2.94 out of 5).

Moreover, certain papers may have been overlooked as

the result of the subscription limitations of our univer-

sity library, as was the case of two conference papers

found in the IEEE Digital Library. This problem was

overcome by asking the authors of the papers to provide

us with a copy of their published articles.

Another threat concerns the potential mishandling of

duplications, which might have slightly altered our

results. Two cases of possible duplication were

detected, which were examined exhaustively to dis-

cover whether or not they were the same study.

Although certain content elements were common to

different papers, these papers are based on innovative

ideas or new studies.

The final decision to select a study depended on the two

authors who conducted the search process. If a

disagreement arose between them, then a discussion

took place until an agreement was reached.

Finally, note that although the same papers were not

found in a replication of this secondary study (which is

only possible in a relatively narrow area with experts in

the area conducting the study), the same general

conclusions may be drawn [133].

• Internal validity: Internal validity deals with extraction

and data analysis [129, 130]. Two authors carried out

the data extraction and classification of the primary

studies, while the other two authors reviewed the final

results.

The decision as to which data to collect and how to

classify the papers therefore depended on the judge-

ment of the two authors conducting the systematic

mapping study. These authors who were from different

culture and research groups carried out two different

classifications for reliability purposes [134].

The Kappa coefficient was 0.95, reflecting a high level

of agreement between the authors, which indicates a

similar understanding of relevance, thus reducing this

threat significantly. The authors were in different time

zones and communicated with each other using video-

conferencing via Skype, and therefore needed to agree

on a timetable for their meetings to avoid them

becoming tired—a situation that is identified as a threat

in literature [135].

Data extraction from prose could also result in a

misclassification, but this problem was addressed by

developing a classification scheme on the basis of

widely accepted guidelines [12] and terminology pro-

posed for use in RE [136]. This would, therefore, only

have a minor influence on the general classification

derived in this study.

• Conclusion validity: The threat to conclusion validity is

concerned with the identification of incorrect relation-

ships which may lead to incorrect conclusions. In the

case of a mapping study, this threat refers to factors

such as missing studies and incorrect data extraction

[130].

The aim is to control these factors so that a systematic

mapping study can be performed by other researchers

[129, 131, 137] who will draw the same conclusions

[134].

Bias both as regards selecting and classifying primary

studies and analyzing data may therefore affect the

interpretation of the results. In order to mitigate this

threat, every step performed in the selection and data

extraction activity was clearly described as discussed in

the previous paragraphs.

The traceability between the data extracted and the

conclusions was strengthened through the direct gen-

eration of bubble plots and frequency plots from the

data by using a statistical package. In our opinion,

slight differences based on publication selection bias

and misclassification would not alter the main conclu-

sions drawn from the 79 articles identified in our

mapping study.

• External validity: External validity is concerned with

the generalization of this study [113, 137, 138]. The

systematic mapping results were considered with regard

to the RE domain, and the validity of the conclusions

drawn in this paper concerns only the REE context.

Since no time restriction was introduced in the search

for published studies, the representativeness of the

selected studies was not affected. This threat is not

therefore present in this context.

The search string and the classification scheme pre-

sented in this paper may serve as a starting point for RE

researchers, and practitioners can search for and cate-

gorize additional papers accordingly.

6 Conclusions and future work

This paper has presented a systematic mapping study that

summarizes the existing research in REE. Of 1359 studies,

148 papers were identified between 1995 and 2012, 79 of

Requirements Eng

123

which were selected and classified according to five crite-

ria: research type, empirical type, approach type, RE

activity, and RE model curricula. Publication source and

trend were also identified.

The results obtained showed that an increasing amount

of attention has been paid to REE since 2004. Around half

of the selected papers appeared in workshops, and only a

few papers had reached the maturity of a journal publica-

tion. The two main research types found were experience

papers and solution proposals. The majority of the selected

papers were empirical studies. Many papers proposed

solutions to address the education of RE, and the main

contribution types were methods. The RE process is the

most frequently addressed RE activity in the literature, and

few authors take into consideration model curricula, bodies

of knowledge or standards in the design of their REE

approaches.

This research could be a starting point to investigate

better ways in which to give RE courses in the future.

Furthermore, the REE approaches presented in this study

may help instructors to identify approaches that can be

adopted in order to improve the quality of their courses.

For future research on REE, greater presence in journals

should be considered and more attention should be paid to

the teaching of each RE activity, particularly requirements

specification and validation. More evaluation research

should be carried out in order to evaluate existing REE

approaches. Moreover, it is advised that future proposed

approaches should conform to the existing model curricula,

BoKs, or standards.

Ongoing research is based on performing a systematic

literature review to assess the research on REE by taking

into consideration the results found in this systematic

mapping study and performing a joint experiment

between the University of Murcia and the University of

Mohammed V Souissi in Rabat in order to study the RE

in GSD.

Acknowledgments This research is part of the project PEGASO-

PANGEA (TIN2009-13718-C02-02) financed by the Spanish Minis-

try of Science and Innovation (Spain), and also part of the GEODAS-

REQ project (TIN2012-37493-C03-02) financed by the Spanish

Ministry of Economy and Competitiveness. This research is also part

of the project Software Project Management using Data Mining

Techniques, (AP2010-2013), financed by Mohammed V Souissi

University (Morocco). The mobility grant of Sofia Ouhbi is financed

by the Mediterranean Office for Youth (MOY).

References

1. Zave P (1997) Classification of research efforts in requirements

engineering. ACM Comput Surv 29(4):315–321

2. Nuseibeh B, Easterbrook S (2000) Requirements engineering: a

roadmap. In: Proceedings of the conference on the future of

software engineering, ICSE ’00, ACM, New York, pp 35–46

3. Standish-Group, CHAOS summary (2009) [cited 2013]. http://

blog.standishgroup.com/pmresearch

4. Damian D, Chisan J (2006) An empirical study of the complex

relationships between requirements engineering processes and

other processes that lead to payoffs in productivity, quality, and

risk management. IEEE Trans Softw Eng 32(7):433–453

5. Smith MJ (2001) Troubled IT Projects: prevention and turn-

around, Institution of Electrical Engineers

6. Sommerville I, Ransom J (2005) An empirical study of indus-

trial requirements engineering process assessment and

improvement. ACM Trans Softw Eng Method 14(1):85–117

7. Minor O, Armarego J (2004) Requirements engineering: a close

look at industry needs and model curricula. In: Proceedings of

the 9th Australian workshop on requirements engineering,

AWRE’04, Australian Workshop on Requirements Engineering,

Adelaide, pp 9.1–9.10

8. Regev G, Gause DC, Wegmann A (2008) Requirements engi-

neering education in the 21st century, an experiential learning

approach. In: Proceedings of the 16th IEEE international

requirements engineering conference, RE ’08, IEEE Computer

Society, Washington, pp 85–94

9. The body of Software Engineering Education Knowledge

(SEEK) (2003) [cited 2013]. http://www.acm.org/education/

curricula.html

10. IEEE/ACM JTF-SEC, Computing Curricula—Software Engi-

neering (CCSE) (2004) [cited 2013]. http://sites.computer.org/

ccse/

11. IEEE/ACM (Ed.). The Joint Task Force on Computing Curricula

IEEE/ACM. Computing Curricula—Computer Science (CCCS)

[online] (2001) [cited 2013]

12. Abran A, Moore JW (2004) Guide to the software engineering

body of knowledge (SWEBOK), IEEE Computer Society

13. Kitchenham B, Budgen D, Brereton P, Woodall P (2005) An

investigation of software engineering curricula. J Syst Softw

74(3):325–335

14. Aoyama M, Nakatani T, Saito S, Suzuki M, Fujita K, Nakazaki

H, Suzuki R (2010) A model and architecture of REBOK

(Requirements Engineering Body of Knowledge) and its eval-

uation. In: Proceedings of the Asia Pacific Software Engineering

conference, APSEC ’10, IEEE Computer Society, Washington,

DC, USA, pp 50–59

15. Armarego J (2007) Educating requirements engineers in Aus-

tralia: effective learning for professional practice, PhD Infor-

mation Technology, University of South Australia

16. Smith R, Gotel O (2008) Gameplay to introduce and reinforce

requirements engineering practices. In: Proceedings of the 16th

IEEE International Requirements Engineering, IEEE Computer

Society, Los Alamitos, CA, USA, pp 95–104

17. Zowghi D, Paryani S (2003) Teaching requirements engineering

through role playing: Lessons learnt. In: Proceedings of the 11th

IEEE international requirements engineering conference, IEEE

Computer Society, Los Alamitos, CA, USA, p 233

18. Idri A, Ouhbi S, Fernandez-Aleman JL, Toval A (2012) A

survey of requirements engineering education. In: Proceedings

of the IEEE global engineering education conference, EDU-

CON’12, Marrakech, Morocco pp. 826–830

19. Petersen K, Feldt R, Mujtaba S, Mattsson M (2008) Systematic

mapping studies in software engineering. In: Proceedings of the

12th international conference on evaluation and assessment in

software engineering, EASE’08, Blekinge Institute of Technol-

ogy, Bari, Italy, pp 71–80

20. ISO/IEC Guide 2:1996 Standardization and related activities—

General vocabulary (1996)

21. ACM/AIS Curriculum Guidelines for undergraduate degree

programs in information systems (IS 2010) [cited 2013]. www.

acm.org/education/curricula-recommendations

Requirements Eng

123

http://blog.standishgroup.com/pmresearch

http://blog.standishgroup.com/pmresearch

http://www.acm.org/education/curricula.html

http://www.acm.org/education/curricula.html

http://sites.computer.org/ccse/

http://sites.computer.org/ccse/

http://www.acm.org/education/curricula-recommendations

http://www.acm.org/education/curricula-recommendations

22. Graduate Software Engineering (GSwE2009)–Curriculum

Guidelines for Graduate Degree Programs in Software Engi-

neering [cited 2013]. http://www.gswe2009.org/

23. IEEE Std 1233-1998, IEEE Guide for Developing System

Requirements Specifications (1998)

24. IEEE Std 1465-1998//ISO/IEC 12119:1994, IEEE Standard

Adoption of International Standard ISO/IEC 12119:1994(E),

Information Technology-Software Packages-Quality Require-

ments and Testing (1998)

25. IEEE Std 830-1998, IEEE Recommended Practice for Software

Requirements Specifications (1998)

26. ISO/IEC 25030:2007 Software engineering—Software product

Quality Requirements and Evaluation (SQuaRE)—Quality

requirements (2007)

27. ISO/IEC 9126-1 Software engineering—Product quality—Part

1: Quality model (2001)

28. ISO/IEC 15288 Systems and software engineering—System life

cycle processes (2008)

29. ISO/IEC TR 24766:2009 Information technology—Systems and

software engineering—Guide for requirements engineering tool

capabilities (2009)

30. Brereton P, Kitchenham BA, Budgen D, Turner M, Khalil M

(2007) Lessons from applying the systematic literature review

process within the software engineering domain. J Syst Softw

80(4):571–583

31. Memon RN, Ahmad R, Salim SS 2010 Problems in require-

ments engineering education: a survey. In: Preceedings of the

8th international conference on frontiers of information tech-

nology, FIT ’10, ACM, New York, NY, USA, pp 5:1–5:6

32. Rosenstreich D, Wooliscroft B (2009) Measuring the impact of

accounting journals using Google Scholar and the g-index. Br

Account Rev 41(4):227–239

33. Landis J, Koch G (1977) The measurement of observer agree-

ment for categorical data. Biometrics 33:159–174

34. Fernandez A, Insfran E, Abrahao S (2011) Usability evaluation

methods for the web: A systematic mapping study. Inform Softw

Technol 53:789–817

35. Computer science conference rankings CORE (2010) [cited

2013]. http://lamp.infosys.deakin.edu.au/era/

36. Condori-Fernandez N, Daneva M, Sikkel K, Wieringa R, Dieste

O, Pastor O (2009) A systematic mapping study on empirical

evaluation of software requirements specifications techniques.

In: Proceedings of the 3rd international symposium on empirical

software engineering and measurement, ESEM ’09, IEEE

Computer Society, Washington, DC, USA, pp 502–505

37. Tonella P, Torchiano M, Du Bois B, Systa T (2007) Empirical

studies in reverse engineering: state of the art and future trends.

Emp Softw Eng 12(5):551–571

38. Barmi ZA, Ebrahimi AH, Feldt R (2011) Alignment of

requirements specification and testing: A systematic mapping

study. In: Proceedings of the IEEE Fourth International Con-

ference on Software Testing, Verification and Validation

Workshops, ICSTW ’11, IEEE Computer Society, Washington,

DC, USA, pp 476–485

39. Callele D, Makaroff D (2006) Teaching requirements engineer-

ing to an unsuspecting audience. ACM SIGCSE Bull 38:433–437

40. Hainey T, Connolly TM, Stansfield M, Boyle EA (2011) Eval-

uation of a game to teach requirements collection and analysis in

software engineering at tertiary education level. Comput Educ

56:21–35

41. Karlsson L, Thelin T, Regnell B, Berander P, Wohlin C (2007)

Pair-wise comparisons versus planning game partitioning–

experiments on requirements prioritisation techniques. Emp

Softw Eng 12:3–33

42. Mohan S, Chenoweth S (2011) Teaching requirements engi-

neering to undergraduate students. In: Proceedings of the 42nd

ACM technical symposium on Computer science education,

SIGCSE ’11, ACM, New York, NY, USA, pp. 141–146

43. Armarego J (2004) Learning requirements engineering within an

engineering ethos. In: Proceedings of the 9th Australian work-

shop on requirements engineering, AWRE’04, Adelaide, Aus-

tralia, pp. 6–7

44. Al-Ani B, Yusop N (2004) Role-playing, group work and other

ambitious teaching methods in a large requirements engineering

course. In: Proceedings of the 11th IEEE international confer-

ence and workshop on the engineering of computer-based sys-

tems, IEEE Computer Society, Los Alamitos, CA, USA, p 299

45. Beatty J, Alexander M (2008) Games-based requirements

engineering training: An initial experience report. In: Proceed-

ings of the 16th IEEE international requirements engineering

conference, RE ’08, IEEE Computer Society, pp 211–216

46. Berenbach B, Rayment T (2008) The evaluation of a require-

ments engineering training program at Siemens. In: Proceedings

of the 16th IEEE international requirements engineering con-

ference, RE ’08, IEEE Computer Society, pp 205–210

47. Gibson JP (2000) Formal requirements engineering: Learning

from the students. In: Proceedings of the Australian software

engineering conference, ASWEC ’00, IEEE Computer Society,

Washington, DC, USA, pp 171–180

48. Jiang L, Eberlein A, Far BH (2005) Combining requirements

engineering techniques—Theory and case study. In: Proceedings

of the 12th IEEE international conference and workshops on

engineering of computer-based systems, ECBS ’05, IEEE

Computer Society, Washington, DC, USA, pp 105–112

49. Ludi S (2007) Introducing accessibility requirements through

external stakeholder utilization in an undergraduate require-

ments engineering course. In: Proceedings of the 29th interna-

tional conference on software engineering, ICSE ’07, IEEE

Computer Society, Washington, DC, USA, pp. 736–743

50. Nguyen L, Armarego J, Swatman P (2005) Understanding

requirements engineering process: a challenge for practice and

education. In: Proceedings of the 5th international business

information management conference, International Business

Information Management Association, Cairo, Egypt, pp 886–894

51. Svahnberg M, Aurum A, Wohlin C (2008) Using students as

subjects—an empirical evaluation. In: Proceedings of the Sec-

ond ACM-IEEE international symposium on empirical software

engineering and measurement, ESEM ’08, ACM, New York,

NY, USA, pp 288–290

52. Swigger KM, Brazile R, Shin D (1995) Teaching cooperation

and requirements elicitation via a computer-supported cooper-

ative problem solving environment. In: Proceedings of the

frontiers in education conference, vol 2 of FIE ’95, IEEE

Computer Society, Washington, DC, USA, pp 3c2–7

53. Tuya J, Garcia-Fanjul J (1999) Teaching requirements analysis

by means of student collaboration. In: Proceedings of the 29th

annual frontiers in education conference, vol 1, pp 11B4/11–15

54. Fernandes JM, Machado RJ, Seidman SB (2009) A requirements

engineering and management training course for softwaredevelopment professionals. In: Proceedings of the 22nd con-

ference on software engineering education and training, CSEET

’09, IEEE Computer Society, Washington, USA, pp 20–25

55. Mead NR, Hough ED (2006) Security requirements engineering

for software systems: case studies in support of software engi-

neering education. In: Proceedings of the 19th conference on

software engineering education & training, CSEET ’06, IEEE

Computer Society, Los Alamitos, CA, USA, pp 149–158

56. Regev G, Gause DC, Wegmann A (2009) Experiential learning

approach for requirements engineering education. Requir Eng

14(4):269–287

57. Zowghi D (2009) Requirements engineering education and

training: Key challenges and practical solutions. In: Proceedings

Requirements Eng

123

http://www.gswe2009.org/

http://lamp.infosys.deakin.edu.au/era/

of the 17th IEEE International Requirements Engineering

Conference, RE ’09, IEEE Computer Society, Los Alamitos,

CA, USA, p 358

58. Auriol G, Baron C, Fourniols JY (2008) Teaching requirements

skills within the context of a physical engineering project. In:

Proceedings of the 3rd international workshop on requirements

engineering education and training, REET ’08, IEEE Computer

Society, pp 6–11

59. Barnes Raymond J, Gause Donald C, Way Eileen C (2008)

Teaching the unknown and the unknowable in requirements

engineering education. In: Proceedings of the 3rd international

workshop on requirements engineering education and training,

REET ’08, IEEE Computer Society, Washington, DC, USA,

pp 30–37

60. Beus-Dukic L (2011) Final year project: A test case for

requirements engineering skills. In: Proceedings of the 6th

international workshop on requirements engineering education

and training, REET’11, IEEE Computer Society, Washington

DC, USA, pp 5–8.

61. Connor AM, Buchan J, Petrova K (2009) Bridging the research-

practice gap in requirements engineering through effective

teaching and peer learning. In: Proceedings of the 6th interna-

tional conference on information technology: new generations,

ITNG ’09, IEEE Computer Society, Washington, DC, USA,

pp 678–683

62. Damian D, Ban A, Cubranic D, Robles L (2005) Teaching

requirements engineering in global software development: a

report on a three-university collaboration. In: 1st International

Workshop on requirements engineering education and training,

REET’05, IEEE Computer Society, Washington DC, USA,

pp 121–127.

63. Danielsen A (2010) Teaching requirements engineering an

experimental approach. In: Proceedings of the Norsk informa-

tikkonferanse conference, NIK, Oslo, pp 77–86

64. Gabrysiak G, Giese H, Seibel A, Neumann S (2010) Teaching

requirements engineering with virtual stakeholders without

software engineering knowledge. In: Proceedings of the 5th

International Workshop on requirements engineering education


DC, USA, pp 36–45

65. Jones S, Britton C (1997) Using multimedia case study material

for teaching requirements engineering, Tech. rep., University of

Hertfordshire

66. Beus-Dukic L, Myers C (2005) Use and abuse cases. In: Pro-

ceedings of the 1st international workshop on requirements

engineering education and training, REET’05, IEEE Computer

Society, Washington DC, USA, pp 133–141

67. Martin M (2007) Improvisational theatre: an approach to soft

skills for requirements engineers. In: Proceedings of the 2nd



DC, USA, pp 56–60

68. Mead N, Shoemaker D, Ingalsbe J (2009) Teaching security

requirements engineering using SQUARE. In: Proceedings of

the 4th international workshop on requirements engineering

education and training, REET’09, IEEE Computer Society,

Washington DC, USA, pp 20–27

69. Garcıa F, Moreno M (2003) C-requirements specification

teaching. In: Proceedings of the 33rd annual frontiers in edu-

cation, vol 3 of FIE’03, pp 1–6

70. Takako N (2007) Improving the engineering mind in eliciting

requirements. In: Proceedings of the 2nd international workshop

on requirements engineering education and training, REET’07,

IEEE Computer Society, Washington DC, USA, pp 37–41

71. Madhavji NH, Miller J (2005) Investigation-based requirements

engineering education. In: Proceedings of the 1st international


REET’05, IEEE Computer Society, Washington DC, USA, pp 68–72

72. Nguyen L, Armarego J, Swatman P (2002) Understanding

requirements engineering: a challenge for practice and educa-

tion, Tech. rep., Deakin University, School of Information

Systems

73. Periyasamy K, Qin X, He D (2011) A requirements editor for

teaching requirements engineering. In: Proceedings of the 5th

international technology, education and development confer-

ence, INTED’11, IATED, Valencia, Spain, pp 4092–4100

74. Shubhamangala B, Rao L, Dakshinamurthy A, Singh C (2012)

Ability based domain specific training: a pragmatic solution to

poor requirement engineering in CMM level 5 companies. In:

Proceedings of the IEEE international conference on computer

science and automation engineering, vol 3 of CSAE’12,

Zhangjiajie, China, pp 459–464

75. Romero M, Vizcaıno A, Piattini M (2008) A simulator for

education and training in global requirements engineering: A

work in progress. In: Proceedings of the 8th IEEE international

conference on advanced learning technologies, ICALT’08, IEEE

Computer Society, Washington DC, USA, pp 123–125

76. Rosca D (2000) An active/collaborative approach in teaching

requirements engineering. In: Proceedings of the 30th annual

frontiers in education, vol 1 of FIE ’00, IEEE Computer Society,

Washington, DC, USA, pp T2C/9–T2C12

77. Salzer HT, Levin I (2004) Spreadsheet-based logic controller for

teaching fundamentals of requirements engineering. Int J Eng

Educ 20:939–948

78. Sindre G (2005) Teaching oral communication techniques in RE

by student-student role play: Initial experiences. In: Proceedings

of the 18th conference on software engineering education and

training, CSEET ’05, IEEE Computer Society, Washington, DC,

USA, pp 85–92

79. Svahnberg M, Gorschek T (2005) Multi-perspective require-

ments engineering education with focus on industry relevance.

In: Proceedings of the 1st international workshop on require-

ments engineering education and training, REET’05, IEEE

Computer Society, Washington DC, USA, pp 88–97

80. Mario ZJC (2010) Communication and traceability game: a way

to improve requirements elicitation process teaching, Revista

Facultad de Ingenierıa Universidad de Antioquia (56), 213–221

81. Albakry K, Kamalrudin M (2011) Pair analysis of requirements

in software engineering education. In: Proceedings of the 5th

Malaysian conference in software engineering, MySEC’11, Jo-

hor Bahru, Malaysia, pp 43–47

82. Beatty J, Agourida V (2007) Developing requirements engi-

neering skills: a case study in training practitioners. In: Pro-

ceedings of the 2nd international workshop on requirements



83. Huijs C, Sikkel K, Wieringa R (2005) Mission 2 solution:

requirements engineering education as central theme in the BIT

programme. In: Proceedings of the 1st international workshop



84. Ferrari R, Madhavji NH (2005) Requirements engineeringeducation for novice software architects. In: Proceedings of the

1st international workshop on requirements engineering educa-

tion and training, REET’05, IEEE Computer Society, Wash-

ington DC, USA, pp 106–110

85. Gabrysiak G (2011) Why should I help you to teach require-

ments engineering?. In: Proceedings of the 6th workshop on

requirements engineering education and training, REET’11,


86. Jamaludin NAA, Sahibuddin S (2011) Measurement of rasch

analysis towards requirement engineering education industry

Requirements Eng

123

perspective. In: Proceedings of the 6th international world sci-

entific and engineering academy and society conference, vol 9,

WSEAS’11, Stevens Point, Wisconsin, USA, pp 298–304

87. Penzenstadler B, Callele D (2010) Prototyping RE experiments

in the classroom—an experience report. In: Proceedings of the

5th international workshop on requirements engineering edu-

cation and training, REET’10, IEEE Computer Society, Wash-

ington DC, USA, pp 7–16

88. Simmons E (2007) Reflection on a successful corporate

requirements engineering training curriculum. In: Proceedings

of the 2nd international workshop on requirements engineering



89. Tsumaki T, Kaiya H, Tahara Y, Yoshioka N, Taguchi K,

Honiden S (2007) Errors and misconceptions in learning. In:

Proceedings of the 2nd international workshop on requirements



90. Yusop N, Mehboob Z, Zowghi D (2007) The role of conducting

stakeholder meetings in requirements engineering training. In:

Proceedings of the 2nd international workshop on requirements



91. Alexander M, Beatty J (2008) Effective design and use of

requirements engineering training games. In: Proceedings of the

3rd international workshop on requirements engineering edu-

cation and training, REET ’08, IEEE Computer Society,


92. Gabrysiak G, Guentert M, Hebig R, Giese H (2012) Teaching

requirements engineering with authentic stakeholders: Towards

a scalable course setting. In: Proceedings of the 1st International

Workshop on Software Engineering Education based on Real-

World Experiences, EduRex’12, IEEE, pp. 1–4

93. Berry Daniel M, Kaplan Craig S (2010) Planned programming

problem gotchas as lessons in requirements engineering. In:

Proceedings of the 5th international workshop on requirements

engineering education and Training, REET’10, IEEE Computer


94. Berenbach B (2005) A hole in the curriculum. In: Proceedings of

the 1st international workshop on requirements engineering



95. Beus Dukic L, Alexander I (2008) Learning how to discover

requirements. In: Proceedings of the 3rd international workshop

on requirements engineering education and training, REET ’08,


96. Bray IK (2004) Experiences of teaching problem frame based

requirements engineering to undergraduates. In: Proceedings of

the 26th international conference on software engineering—

W4S Workshop ‘‘1st international workshop on advances and

applications of problem frames (IWAAPF 2004)’’, pp 17–20

97. Davis AM, Hickey AM, Chamillard A (2005) Moving beyond

the classroom: Integrating requirements engineering research &

education to improve practice. In: Proceedings of the 1st inter-

national workshop on requirements engineering education and

training, REET’05, IEEE Computer Society, Washington DC,

USA, pp 78–87

98. Lami G (2005) Teaching requirements engineering in the small:

an under-graduate course experience. In: Proceedings of the 1st



DC, USA, pp 128–132

99. Hoffmann A (2008) Teaching soft facts in requirements engi-

neering using improvisation theatre techniques. In: Proceedings

of the 3rd international workshop on multimedia and enjoyable

requirements engineering—beyond mere descriptions and with

more fun and games, MERE ’08, IEEE Computer Society,

Washington, DC, USA, pp 1–3

100. Ozkaya I, Akin O, Tomayko JE (2005) Teaching to think in

software terms: An interdisciplinary graduate software require-

ment engineering course for AEC students. In: Proceedings of

the international conference on computing in civil engineering,

American Society of Civil Engineers

101. Knauss E, Schneider K, Stapel K (2008) A game for taking

requirements engineering more seriously. In: Proceedings of the

3rd international workshop on multimedia and enjoyable

requirements engineering, MERE ’08, IEEE Computer Society,


102. Liu L, Jin Z (2008) Balancing academic and industrial needs in

RE courses. In: Proceedings of the 3rd international workshop



103. Nakatani T, Tsumaki T, Tamai T (2010) Requirements engi-

neering education for senior engineers: Course design and its

evaluation. In: Proceedings of the 5th international workshop on

requirements engineering education and training, REET’10,


104. Jamaludin NAA, Sahibuddin S (2012) Challenges of project-

based learning towards requirement engineering. Int J Comput

Appl 50(3):1–5

105. Jamaludin NAA, Sahibuddin S (2011) Development strategy

using cognitive domain in e-requirement engineering learning

system. Int J Comput Sci Issues 8:318–322

106. Thiry RQ, Marcello G (2010) Development of a game to support

the teaching of requirements engineering: the requirements

island, In: Proceedings of SBGames, SBC, Florianopolis, Brazil,

pp 358–361

107. Romero M, Vizcaıno A, Piattini M (2008) Using virtual agents

for the teaching of requirements elicitation in GSD. In: Pro-

ceedings of the 8th international conference on intelligent virtual

Agents, IVA ’08, Springer, Berlin, pp 539–540

108. Romero M, Vizcaıno A, Piattini M (2009) Teaching require-

ments elicitation within the context of global software devel-

opment. In: Proceedings of the Mexican international

conference on computer science, ENC ’09, IEEE Computer

Society, Washington, DC, USA, pp 232–239

109. Sallim J (2005) Requirement engineering for enterprise appli-

cation development: seven challenges in higher education

environment. In: Proceedings of the 2nd World Enformatika

conference, WEC’05, pp 101–104

110. Sikkel K, Daneva M (2011) Getting the client into the loop in

information systems modelling courses. In: Proceedings of the 6th


REET’11, IEEE Computer Society, Washington DC, USA, pp 1–4

111. Zowghi D (2009) Teaching requirements engineering to the

Bahai; students in Iran who are denied of higher education. In:

Proceedings of the 4th international workshop on requirements

engineering education and training, REET ’09, IEEE Computer


112. Armarego J, Minor O (2005) Studio learning of requirements:

towards aligning teaching to practitioner needs. In: Proceedings

of the 1st international workshop on requirements engineering



113. Wohlin C, Runeson P, Host M, Ohlsson MC, Regnell B, Wes-

slen A (2000) Experimentation in software engineering: an

introduction. Kluwer, Norwell

114. Mead NR, Stehney T (2005) Security quality requirements

engineering (SQUARE) methodology. ACM SIGSOFT Softw

Eng Notes 30(4):1–7

115. Glass LR (1998) Software runaways: monumental disasters.

Prentice Hall, New Jersey

Requirements Eng

123

116. Glass LR (2002) Software engineering: facts and fallacies.

Addison-Wesley, Boston

117. Benestad HC, Arisholm E, Sjøberg DIK (2005) How to recruit

professionals as subjects in software engineering experiments.

In: Proceedings of the 28th information systems research con-

ference in Scandinavia, IRIS’05, Department of Information

Systems, Agder University College, Kristiansand, Norway

118. Mich L, Anesi C, Berry DM (2004) Requirements engineering

and creativity: An innovative approach based on a model of the

pragmatics of communication. In: Proceedings of requirements

engineering: foundation of software quality, REFSQ’04

119. Acuna ST, Gomez M, Juristo N (2009) How do personality,

team processes and task characteristics relate to job satisfaction

and software quality?. Inform Softw Technol 51(3):627–639

120. Carrillo de Gea JM, Nicolas J, Fernandez Aleman JL, Toval A,

Ebert C, Vizcaıno A (2011) Requirements engineering tools.

IEEE Softw 28(4):86–91

121. Ramesh B (1993) Process knowledge based rapid prototyping

for requirements engineering. In: Proceedings of IEEE interna-

tional symposium on requirements engineering, pp 248–255

122. Lichter H, Schneider-Hufschmidt M, Zullighoven H (1993)

Prototyping in industrial software projects—bridging the gap

between theory and practice. In: Proceedings of the 15th inter-

national conference on software engineering, ICSE ’93, IEEE

Computer Society Press, Los Alamitos, CA, USA, pp 221–229

123. Gabrysiak G, Giese H, Seibel A (2009) Interactive visualization

for elicitation and validationn of requirements with scenario-

based prototyping. In: Proceedings of the fourth international

workshop on requirements engineering visualization, REV ’09,

IEEE Computer Society, Washington, DC, USA, pp 41–45

124. Damian D, Hadwin A, Al-Ani B (2006) Instructional design and

assessment strategies for teaching global software development: a

framework. In: Proceedings of the 28th international conference

on software engineering, ICSE ’06, ACM, New York, USA,

pp 685–690

125. Myint Swe K (2011) Games in Education, Vol. 5, Contemporary

Approaches to Research in Learning Innovations. Sense Pub-

lishers, Rotterdam

126. Houser C, Thornton P, Kluge D (2002) Mobile learning: Cell

phones and PDAs for education. In: Proceedings of the Inter-

national Conference on Computers in Education, ICCE ’02,

IEEE Computer Society, Washington, DC, USA, p 1149

127. Abut H, Ozturk Y (1997) Interactive classroom for DSP/com-

munication courses. In: Proceedings of IEEE International

Conference on Acoustics, Speech, and Signal Processing, Vol. 1

of ICASSP ’97, IEEE Computer Society, Washington DC, USA,

pp 15–18

128. Yau Stephen S, Gupta Eep KS, Fariaz K, Sheikh I A, Yu W, Bin

W (2003) Smart classroom: Enhancing collaborative learning

using pervasive computing technology. In: Proceedings of the

6th WFEO world congress on engineering education & 2nd

American Society of Engineering Education, ASEE’03, Nash-

ville, Tennessee, USA

129. Elberzhager F, Munch J, Nha VTN A (2012) systematic map-

ping study on the combination of static and dynamic quality

assurance techniques. Inform Softw Technol 54(1):1–15

130. Ampatzoglou A, Charalampidou S, Stamelos I (2013) Research

state of the art on GoF design patterns: a mapping study. J Syst

Softw 86(7):1945–1964

131. Garousi V, Mesbah A, Betin-Can A, Mirshokraie S (2013) A

systematic mapping study of web application testing. Inform

Softw Technol 55(8):1374–1396

132. Zhang H, Babar MA, Tell P (2011) Identifying relevant studies

in software engineering. Inform Softw Technol 53(6):625–637

133. Wohlin C, Runeson P, da MotaSilveira Neto PA, Engstrom E,

doCarmo Machado I, de Almeida ES (2013) On the reliability of

mapping studies in software engineering. J Syst Softw

86(10):2594–2610

134. Portillo-Rodrıguez J, Vizcaıno A, Piattini M, Beecham S (2012)

Tools used in global software engineering: a systematic map-

ping review. Inform Softw Technol 54(7):663–685

135. Fernandez-Saez AM, Genero M, Chaudron M (2013) Empirical

studies concerning the maintenance of UML diagrams and their

use in the maintenance of code: a systematic mapping study.

Inform Softw Technol 55(7):1119–1142

136. Wieringa R, Maiden N, Mead N, Rolland C (2006) Require-

ments engineering paper classification and evaluation criteria: a

proposal and a discussion. Requir Eng 11(1):102–107

137. Easterbrook S, Singer J, Storey M-A, Damian D (2008)

Selecting empirical methods for software engineering research.

In: Guide to advanced empirical software engineering, Springer,

pp 285–311

138. Mateo PR, Usaola MP, Fernandez Aleman JL (2013) Validating

2nd-order mutation at system level. IEEE Trans Softw Eng

39(4):570–587

Requirements Eng

123

requirements engineering education: a systematic mapping study

Documents