What do Software Architects Expect from Requirements Specifications?

Results of Initial Explorative Studies

Anne Gross, Joerg Doerr

Fraunhofer IESE

Kaiserslautern, Germany

{Anne.Gross, Joerg.Doerr}@iese.fraunhofer.de

Abstract—Software requirements specifications (SRS) serve as

an important source of information for software architects

with regard to deriving suitable architectural design decisions.

However, from a software architect’s viewpoint, using these

documents efficiently and effectively is often difficult. One

could attribute these observations to the fact that SRS also

have to address and satisfy the information needs of other

document consumers involved in downstream activities like

interaction design, user interface design or testing – which is,

indeed, very challenging for requirements engineers. In this

position paper, we present goals and initial results of

explorative studies aimed at investigating information needs

that should be fulfilled in SRS from the viewpoint of software

architects. In these studies, we gained first insights into the

relevance of certain artifact types (like descriptions of

interactions or system functionalities) and their suitable

representation in terms of notations. Furthermore, the analysis

of these initial results revealed variances within the group of

software architects regarding information needs. This has

motivated the planning and conduction of further studies in

the near future that will investigate factors such as expertise or

individual motivation, which might influence the information

needs from software architects’ viewpoints.

Keywords-requirements specification; information needs;

empirical studies; software architects; view-based requirements

specifications

I. INTRODUCTION

Software requirements specifications (SRS) play an important role in software development projects as these documents contain all relevant and “official” information that is required to implement a software system [1]. In fact, SRS can probably be considered as one of the main sources of information for software architects. Based on the information documented in these documents, architectural decisions have to be made, including decisions about the system’s structure, i.e., its components, their relations, characteristics, and attributes. These decisions are very crucial, especially regarding the fulfillment of quality requirements (e.g., maintainability, reliability, security). Furthermore, these decisions, which are documented, for instance, in software architecture specifications, again form the baseline for subsequent development activities like component design and coding. This means that inappropriate architectural decisions have a tremendous effect on the final system.

However, not only software architects base their decisions on information documented in SRS; other engineers such as usability experts or testers also rely on the knowledge and information documented in these documents. This situation poses a challenge to a requirements engineer who is responsible for creating SRS: Different information needs and expectations have to be addressed in one specification even though they are strongly dependent on the particular role that the document stakeholders have within a project.

But even from the isolated viewpoint of a software architect as a consumer of an SRS, this situation leads to several problems [2]: Often they are provided with SRS that contain much more information than actually required to make architectural decisions. This includes information that is important to the architect being spread over different sections, or even the different documents that the SRS comprises. This again makes it hard for the software architect to find and analyze the information. Or the representation of the information in terms of the notations used or the level of detail is inappropriate. Finally, it can also be observed that important information is even missing completely in the SRS. All these observations have a negative impact on the efficient and effective usage of SRS from the viewpoint of their consumers [3]. In the worst case, this could result in document consumers neglecting or ignoring the SRS which finally ends in implementations of software systems that fail to meet the requirements actually documented in the SRS [4].

To tackle this problem, we are currently doing research on “view-based requirements specifications” [2] [3] [4]. The vision underlying this research is to generate views on SRS that address and fulfill role-specific information needs. Hence, the consumers of the SRS are only provided with information that they actually require for their particular role-specific tasks specified on the right level of detail and with suitable notations.

However, before such solutions can be realized, it is essential to first gain empirically valid knowledge about the particular information needs from the viewpoint of the various consumers of the SRS.

In this position paper, we share initial results that have been obtained when investigating such information needs from the viewpoint of software architects. In Section II, we first present some background information on best-practice artifact types documenting requirements in the context of

978-1-4673-4486-9/12/$31.00 c© 2012 IEEE TwinPeaks 2012, Chicago, Illinois, USA41

information systems. These artifact types have been the subject of studies investigating information needs from software architects’ viewpoints. The goals and results of these studies are presented in Section III. The paper concludes with a summary and outlook on future work in Section IV.

II. ARTIFACT TYPES (BACKGROUND)

According to [8] the term “information need” is characterized as “information seeking towards the satisfaction of needs”. Transferring this to the context of requirements engineering means that we have to investigate information that document consumers (like software architects) seek in an SRS to satisfy their needs. Typically, information is specified in an SRS by creating artifacts such as persona descriptions of supported stakeholders or UML activity diagrams visualizing a certain business process. These artifacts documenting the requirements can basically be categorized into artifact types, such as descriptions of stakeholders, descriptions of business processes, descriptions of goals, etc.

In the following, we briefly introduce several best-practice artifact types that have been the subject of the studies presented in this paper. These artifact types are based on applying the Task-Oriented Requirements Engineering

Framework (short: TORE) a well-proven approach when eliciting and specifying requirements in the context of information systems development [5]. Following this approach, system functionalities are derived systematically from the goals and tasks of the stakeholders who are to be supported by the system to be developed. To achieve this, requirements are refined by making and documenting decisions across different levels of abstraction. Figure 1 illustrates the various decision points and abstraction levels underlying this framework.

Figure 1: TORE Framework

A. Artifacts on Goal and Task Level

On this first level of abstraction, decisions are made regarding (1) stakeholders to be supported by the system, (2) goals that these stakeholders have and that should be fulfilled by the system, and (3) stakeholder tasks that operationalize these goals. Elicited information related to these decisions is specified in the form of artifacts by using appropriate notations. For instance, to document all relevant stakeholder information like role, work context, age, gender, language, educational background, etc., role descriptions and personas [6] can be used. The elicited goals and their relationships as well as possible conflicts can be documented in artifacts such as goal descriptions or goal graphs [7]. Finally, information

regarding stakeholder tasks is typically documented in the form of textual task descriptions (see Figure 2).

Figure 2: Task description (example)

B. Artifacts on Domain Level

The second abstraction level according to TORE is dedicated to further analyzing and refining the tasks identified before. This includes making decisions related to

activities currently being performed to execute the tasks (“as-is activities” / “as-is workflows”)

how the task execution will change with the introduction of the system (“to-be activities” / “to-be workflows”)

responsibilities of the system within the future task execution (i.e., identification of activities that are either automatically executed by the system or require interaction between the system and an actor)

domain data relevant for the execution of the tasks

To document all elicited information related to these decisions, various artifacts are created, comprising textual scenario descriptions, semi-formal / graphical process descriptions, e.g., in the form of UML activity diagrams [9], BPMN [10], or EPC models [11]. The specification of domain data often results in data models using UML class diagrams or entity-relationship diagrams. Finally, system responsibilities are typically visualized in the form of UML use case diagrams.

C. Artifacts on Interaction Level

On this last level of abstraction, all activities supported by the system (as identified in the decision point “System Responsibilities”) are further refined and specified. Decisions on the Interaction Level are related to

system functionalities that the system offers

how an actor (either human or external system) interacts with the system

data that is exchanged during interaction

logical grouping of system functionalities into workspaces

To document all information related to these decisions,

artifacts are created that comprise system function descriptions, (see Figure 3) interaction descriptions (e.g., in the form of use case descriptions), and descriptions of interaction data in the form of data models. UI structures are used to logically group common functionalities (i.e., system functions, use cases, and data) into workspaces.

42

All artifact types that have been introduced in this section have been the subject of studies that aimed to investigate the usefulness of these artifact types for architectural tasks.

Figure 3: System function description (example)

III. STUDIES

In this section we introduce goals, setting and results of three explorative studies. The overall research goal that has been addressed in these studies can be stated as:

This research goal has been refined into further sub-goals

following the GQM paradigm [12] (see Table 1).

Table 1: Refined research goals

A. Eye-tracking Study

In the following, we present setting and results of an eye-tracking study that has been planned and conducted with two experienced software architects to investigate the research goals stated above (G1_1 and G1_2).

1) Setting: The study object of this study was a given

SRS created within a real software project. The

specification comprised two documents (about 280 pages in

total). These documents were created by applying the TORE

framework and hence included a lot of artifacts types

specifying information related to the decision points

introduced in the background section of this paper. To

document these artifact types, several notations were used –

often also in combination (e.g., semi-formal descriptions

(UML activity diagrams) supplemented with unstructured

textual descriptions in case of “to-be workflows”). The task of the participants was to analyze the SRS (see

Figure 4). That is, while “thinking aloud”, the participants had to read the documents and judge the relevance of the documented artifact types and notations used to specify the artifact types. For this purpose, they had to imagine the situation of designing an architecture based on the information provided in the specification. In addition to the thinking aloud comments, the analysis of the data collected via an eye-tracking device allowed us to observe how the participants browsed through the documents and which sections got their attention.

Figure 4: Participant in eye-tracking study

Finally, in order to also collect quantifiable data for the purpose of statistical analysis, a questionnaire was prepared [13]. This questionnaire was filled out by the participants after the completion of the document analysis. In this questionnaire, they were asked to evaluate the usefulness of artifact types such as stakeholder descriptions, task descriptions, interaction descriptions, etc. on a scale from 1 = “very important” to 4 = “unimportant”. In addition, the questionnaire also captured information about whether the notations used to document the artifact types were considered to be useful for analyzing and understanding the information documented in the SRS – or whether they preferred other notations. The questionnaire also explicitly asked for any information that was missing in the SRS. The participants were not restricted by any time limitations.

2) Results: The analysis of the questionnaire regarding

RQ1 revealed that artifact types documenting information

about system responsibilities, data, system

functionalities, interactions, technical constraints as well

as stakeholder goals are very important for the

architecture experts who participated in this study (see

Table 2, column “Eye-Tracking”). Tasks and related

workflow descriptions as well as stakeholder descriptions

were also considered to be helpful but not strongly required. Regarding the notations (G1_2), the participants preferred

either structured text (e.g., in the form of bullet lists) where

Research Goal G1: Investigate information needs regarding SRS from the viewpoint of software architects

G1_1: Investigate best-practice artifacts types (see Section II) with respect to their usefulness from the viewpoint of software architects G1_2: Investigate representations of best-practice artifact types (i.e., notations) with respect to their usefulness from the viewpoint of software architects

43

important information is highlighted or semi-formal / graphical representations as these types of notations support the efficient analysis and mental processing of the information. Based on the “thinking aloud” comments, we understood that this is indeed very important for software architects as they have to detect functionalities that share common characteristics (such as data requirements or qualities).

Table 2: Relevance of artifact types (average ratings)

B. Studies in Practical Course

In the following, we present setting and results of two studies conducted in a practical software engineering course to investigate the research goal G1_1 (see Table 1).

1) Goals and Tasks: The participants of the first study

(Practical Course 1) were 13 students (9 computer science

students and 4 business administration students with

specialization in computer science) enrolled in a practical

software engineering course at the University of

Kaiserslautern. Prior to this practical course, the students

had been taught the principles of software engineering in a

separate semester course. In the subsequent practical course,

they basically had to develop an account management

system for a real customer. The practical course took about

three months, during which the students (divided into

groups of 4 -5 students) had to run through a complete SE

process, comprising activities like requirements engineering,

architecture design, UI design, implementation, and testing. After the students had completed the architecture design

phase, we captured the importance and usefulness of previously specified artifact types for architecture design by means of a questionnaire [13]. This questionnaire was basically designed in accordance with the questionnaire used in the eye-tracking study. That is, the importance of typical TORE-related artifact types (see Section II) for architecture design was rated on a scale from 1 (“very important”) to 4 (“unimportant”). In addition, we explicitly asked for any missing information and explanations about why certain artifact types are important. The SRS created by the students was also done by applying TORE, which had been taught in the previous software engineering course. In the practical course, the students also followed a set of specification guidelines, which included suitable templates supplemented with examples for specifying information related to the various artifact types. A first study run took place during the summer semester 2011. The study was recently repeated

within the scope of the same practical course currently being offered at the Computer Science department (Practical Course 2). All conditions are the same as in last year’s study, except for the development tasks. This year the students are asked to develop an information system to support the planning and organization of a social event. As of now, we have received the evaluation from five students.

2) Results: At first glance, the analysis of the data

collected with the questionnaire (see Table 2, columns

“Practical Course 1” and “Practical Course 2”) shows an

interesting observation: Comparing the average values

reveals only little consensus between the participants of

the eye-tracking study and the students of the two

practical courses. When we compare the results of two practical courses,

we can basically detect an agreement related to the relevance of the artifact types. With the only exception of the as-is activities, the average values of the collected data between the two groups can be considered quite homogenous.

A final and interesting observation we made regarding the data collected in the two groups of students were variances in the ratings given by the students. That is, in some cases it could be observed that some students rated the relevance of certain artifacts as being very important, whereas others rated these artifacts as rather unimportant. This is interesting, as all the participants had the same expertise, previous knowledge, and tasks within the project.

IV. CONCLUSION

In this paper, we presented the goals, settings, and results of three explorative studies. These studies were conducted to get an initial understanding of the information needs to be addressed in SRS from the viewpoint of software architects. For this purpose, we investigated research goals aimed at investigating the relevance (G1_1) and suitable representation (G1_2) of best-practices artifact types typically created in the context of requirements engineering activities for information systems. Our main findings related to these research goals can be summarizes as:

Descriptions of system functions and interactions have been identified as being very important artifact types in all three studies (refers to G1_1).

Artifacts types documenting information about system responsibilities, data, system functionalities, interactions, technical constraints, as well as stakeholder goals are very important for the software architecture experts who participated in the eye-tracking study (refers to G1_1)

There was only little consensus between the participants of the eye-tracking study (i.e., software architecture experts) and the participants of the studies in the two practical courses (i.e., students). A comparison of the results between the studies in the two practical courses (i.e., between students) revealed quite homogenous results (refers to G1_1)

44

The analysis of the data within the group of students revealed strong variances in some cases (refers to G1_1)

Regarding the notations, the software architects who participated in the eye-tracking study preferred either structured text (e.g., in the form of bullet lists) where important information is highlighted or graphical representations (refers to G1_2).

The results we have obtained so far basically prove that

the problem of different information needs really exists and that empirical investigations and derived solutions like “view-based requirements specification” might have the potential to contribute to this problem.

Especially the differences between the eye-tracking study and the studies conducted in the practical courses as well as the variances detected within the group of students motivated us to plan and conduct further empirical studies in the near future. These studies will especially be intended to investigate and control possible factors influencing information needs.

The results achieved in our initial explorative studies presented in this paper indicate that factors such as expertise, the complexity of the system to be developed, or even individual factors like personal motivation might influence the relevance of artifacts and hence the information needs.

That is, research goal (G) and related hypotheses (H) that we aim to investigate in the future include:

Table 3: Future research goal and hypotheses

Based on such knowledge, we will be able to interpret the information needs more reliably and thus enhance solutions like “view-based requirements specifications”.

Furthermore, we will collect further statistical data related to the research questions stated in Table 1 in order to draw empirically sound and valid conclusions about the information needs. Besides the viewpoint of software architects presented in this paper, we will also investigate information needs from the viewpoints of other consumers of SRS, like usability experts and testers. As of now, we have conducted initial studies with usability experts and the initial results have already revealed differences between the different roles [2].

REFERENCES

[1] I. Somerville, “Software Engineering”, 7th Edition, Pearson Educational Limited, 2004, pages 136-140

[2] A. Gross and J. Doerr: “What you need is what you get! – the Vision of View-based Requirements Specifications”, 20th IEEE International Requirements Engineering Conference, 2012, to appear

[3] Adam, S.; Riegel, N.; Gross, A.: “Focusing on the “Right” Requirements by Considering Information Needs, Priorities, and Constraints”, REEW’12, 2012

[4] Gross A.: “Perspective-based Specification of Efficiently and Effectively Usable Requirements Documents”. In: Proceedings of Doctoral Symposium RE’10, Sydney 2010

[5] S. Adam, J. Doerr, M. Eisenbarth, A. Gross, “Using Task-oriented Requirements Engineering in Different Domains – Experiences with Application in Research and Industry”, 17th IEEE International Requirements Engineering Conference, RE '09, 267-272, 2009

[6] A. Cooper, “The Inmates Are Running the Asylum: Why High-Tech Products Drive Us Crazy and How to Restore the Sanity”. Indianapolis, USA, 1999

[7] E. Letier and A. van Lamsweerde, “Deriving operational software specifications from system goals” SIGSOFT Softw. Eng. Notes 27, 6 119-128, 2002

[8] T.D. Wilson “On user studies and information needs”, Journal of Documentation, Vol. 37 Iss: 1, pp.3 – 15, 1981

[9] D. Marlon and A. ter Hofstede, “UML Activity Diagrams as a Workflow Specification Language” in UML» 2001 — The Unified Modeling Language. Modeling Languages, Concepts, and Tools”, pp. 76-90, 2001

[10] S.A. White and D. Miers, “BPMN Modeling and Reference Guide - Understanding and Using BPMN”, Future Strategies Inc., Lighthouse Pt, FL, ISBN-13: 978-0977752720, 2008

[11] G. Keller, M. Nüttgens, and A.W. Scheer, “Semantische Prozeßmodellierung auf der Grundlage Ereignisgesteuerter Prozeßketten (EPK)”, Universität des Saarlandes , 1992

[12] V.R. Basili, G. Caldiera, and H. D. Rombach, “Goal Question Metric Paradigm”, in: Encyclopedia of Software Engineering, Volume 1, pp. 528-532, John Wiley & Sons, 1994.

[13] Gross A.; Doerr J.: Investigating Information Needs - Elicitation Guidelines, Fraunhofer IESE Report 033.12/E, 2012

G2: Investigate factors with respect to their influence on information needs related to SRS from the viewpoint of different software architects. H2_1: The experience level related to architecture engineering has an influence on information needs. H2_2: The familiarity with the domain and scope of a system to be developed has an influence on information needs. H2_3: The individual modes of operation and personal motivation have an influence on information needs. H2_4: The software engineering process that is followed within a software engineering project has an influence on information needs.

45