S. DepartmentCommerce Computer ScienceUonal Bureau 30(1 TeChnOlOgV

K7n NBS Special Publication 500-74

W\ } Features of

Software Development Tools

(®)

/l\(in) (fn) (out)

// /l\W(l)(C) (T) (S)(D)(U)(M)

/I IWW1-4) (1-2) (1-6) (1-19) (1-9) (1-6) (1-6)

NATIONAL BUREAU OF STANDARDS

The National Bureau of Standards' was established by an act ot Congress on March 3, 1901.

The Bureau's overall goal is to strengthen and advance the Nation's science and technology

and facilitate their effective application for public benefit. To this end, the Bureau conducts

research and provides: (1) a basis for the Nation's physical measurement system, (2) scientific

and technological services for industry and government, (3) a technical basis for equity in

trade, and (4) technical services to promote public safety. The Bureau's technical work is per-

formed by the National Measurement Laboratory, the National Engineering Laboratory, and

the Institute for Computer Sciences and Technology.

THE NATIONAL MEASUREMENT LABORATORY provides the national system of

physical and chemical and materials measurement; coordinates the system with measurement

systems of other nations and furnishes essential services leading to accurate and uniform

physical and chemical measurement throughout the Nation's scientific community, industry,

and commerce; conducts materials research leading to improved methods of measurement,

standards, and data on the properties of materials needed by industry, commerce, educational

institutions, and Government; provides advisory and research services to other Government

agencies; develops, produces, and distributes Standard Reference Materials; and provides

calibration services. The Laboratory consists of the following centers:

Absolute Physical Quantities' — Radiation Research — Thermodynamics and

Molecular Science — Analytical Chemistry — Materials Science.

THE NATIONAL ENGINEERING LABORATORY provides technology and technical ser-

vices to the public and private sectors to address national needs and to solve national

problems; conducts research in engineering and applied science in support of these efforts;

builds and maintains competence in the necessary disciplines required to carry out this

research and technical service; develops engineering data and measurement capabilities;

provides engineering measurement traceability services; develops test methods and proposes

engineering standards and code changes; develops and proposes new engineering practices;

and develops and improves mechanisms to transfer results of its research to the ultimate user.

The Laboratory consists of the following centers:

Applied Mathematics — Electronics and Electrical Engineering' — Mechanical

Engineering and Process Technology' — Building Technology — Fire Research —Consumer Product Technology — Field Methods.

THE INSTITUTE FOR COMPUTER SCIENCES AND TECHNOLOGY conducts

research and provides scientific and technical services to aid Federal agencies in the selection,

acquisition, application, and use of computer technology to improve effectiveness and

economy in Government operations in accordance with Public Law 89-306 (40 U.S.C. 759),

relevant Executive Orders, and other directives; carries out this mission by managing the

Federal Information Processing Standards Program, developing Federal ADP standards

guidelines, and managing Federal participation in ADP voluntary standardization activities;

provides scientific and technological advisory services and assistance to Federal agencies; and

provides the technical foundation for computer-related policies of the Federal Government.

The Institute consists of the following centers:

Programming Science and Technology — Computer Systems Engineering.

'Headquarters and Laboratories at Gaithersburg, MD, unless otherwise noted;

mailing address Washington, DC 20234.

'Some divisions within the center are located at Boulder, CO 80303.

National Bureau of Standards

LibrarVf E-01 AdnUn. BIdg,

MAR 1 6 1981

not Ckc( (Urc

no. ^oo'id

m)c ^

I

1

Computer Scienceand Technology

NBS Special Publication 500-74

Features of

Software Development Tools

Raymond C. Houghton, Jr.

Center for Programming Science and Technologyinstitute for Computer Sciences and TechnologyNational Bureau of StandardsWashington, DC 20234

U.S. DEPARTMENT OF COMMERCEMalcolm Baldrlge, Secretary

National Bureau of Standards

Ernest Ambler, Director

Issued February 1 981

Reports on Computer Science and Technology

The National Bureau of Standards has a special responsibility within the Federal

Government for computer science and technology activities. The programs of the

NBS Institute for Computer Sciences and Technology are designed to provide ADPstandards, guidelines, and technical advisory services to improve the effectiveness

of computer utilization in the Federal sector, and to perform appropriate research

and development efforts as foundation for such activities and programs. This publica-

tion series will report these NBS efforts to the Federal computer community as well as

to interested specialists in the academic and private sectors. Those wishing to

receive notices of publications in this series should complete and return the form at

the end of this publication.

National Bureau of Standards Special Publication 500-74Nat. Bur. Stand. (U.S.), Spec. Publ. 500-74, 24 pages (Feb. 1981)

CODEN: XNBSAV

Library of Congress Catalog Card Nunnber: 80-600193

U.S. GOVERNMENT PRINTING OFFICE

WASHINGTON: 1981

For sale by the Superintendent of Documents, U.S. Government Printing Office, Wasliington, DC 20402

Price $1.75

(Add 25 percent for otfier tlian U.S. mailing).

TABLE OF CONTENTS

Pag e

1. Introduction 1

2. The Importance of Tool Features 2

3. Formalizing the Use of Tool Features 3

4. A Taxonomy of Tool Features A

5. Using the Taxonomy 16

6. Scope of the Taxonomy 18

7 . Concl us ion 19

8. References 19

iii

!

1'

Features of Software Development Tools

Raymond C. Houghton, Jr.

Abstract

Software tools are powerful productivity and qualityaids that in many cases are not being used effectively.This report discusses an effort to lessen this problem byproviding a formal way in which tools can be classifiedaccording to the features that they provide.

Key words: Dynamic analysis; programming aids; software de-velopment; software engineering; software tools; staticanalysis; taxonomy.

1. Introduction

Software development tools are computer programs thataid the specification, construction, testing, analysis,management, documentation, and maintenance of other computerprograms. Thus, software development tools include thetraditional tools of a programmer (e.g. compilers,editors), more recently developed tools (e.g. design aids,program analyzers, testing tools) , and tools currently inthe research stage (e.g. formal verifiers, programmingenvironments) . Tools are important because they can be usedto increase software productivity and quality. As a result,their use has evolved as an important part of softwaredevelopment. Most importantly, they represent a class ofsoftware that can be used and reused within many differentsoftware development environments. The use of softwaretools provides the opportunity to reduce cost and improvecompetitive advantage.

Although software tools are an important part ofquality software development, there does not currently exista clear body of techniques for making effective use of suchtools. This situation exists in part due to the confusion

NOTE: Certain commercial products are identified in thispaper for clarification of specific concepts. In no casedoes such identification imply recommendation or endorsementby the National Bureau of Standards, nor does it imply thatthe material identified is necessarily the best for thepurpose o

1

generated by the lack of standard terminology for thefunctions and characteristics of software tools. It isdifficult to compare and to evaluate alternative toolpackages for use in specific operating environments. As a

result, there is a lack of technology transfer by tooldevelopers to the user community.

This report discusses an approach aimed at correctingsome of these problems. It presents a framework for toolcomparison and a scheme for classifying tools by thefeatures they provide. Features of software tools arepresented in a hierarchical classification system called a

taxonomy. Following a period of comment and review, NBSplans to issue the taxonomy as a Federal InformationProcessing Standards Publication (FIPS-PUB) Guideline toprovide a common reference within the Federal Government forterms and definitions associated with software developmenttool s

.

2. The Importance of Tool Features

Software development tools have grown in complexityover recent years as have other software applications. Mostearly tools performed a single function and were very simpleto operate. These early, simpler tools have since given wayto more complex tools which in some cases have their ownbuilt-in command languages. This growth is leading to toolsystems where the programmer works totally within the systemto develop software.

This evolution of t®ol development has caused majorproblems in communicating exactly what a tool does. Forexample, a compiler once was a simple tool that performedthe single function of translating high level language tomachine language. Today, we have compilers that interpret,optimize, perform run-time checks, and generatecross-references, profiles, formatted output, and otherforms of documentation. Does the term compiler adequatelydescribe these multi-functional systems, or do people stillthink of a compiler as a simple tool that performs a singlefunction? There are other examples of tools whosecomplexity has outgrown their names. These includeverification systems, program analyzers, programpreprocessors, flow analyzers, software development systems,simulators, and programming environments. In each of theseexamples, one should press for further details about thetool to determine exactly what it does.

2

The features incorporated in a software tool are thedetails one should seek. Careful identification of featuresallows one to distinguish one tool from another and todetermine which is more appropriate for a given application.In order to do this, one must acquire detailed informationon a tool. This should include what a tool accepts as inputand how it accepts it, the way it manipulates and analyzesthat input, and what a tool produces as output for both thetool user and for further processing by other tools. With a

careful analysis of this information, one can begin tounderstand the real capabilities of a tool and can comparethese capabilities with those of other tools. The taxonomyof tool features presented in Section 4 provides a frameworkfor communicating and a basis for understanding thecapabilities of a tool.

3. Formalizing the Use of loci Features

For now let us assume that we have a way to carefullyand succinctly discover the individual features of a

software tool. The sections that follow will address howone could use the resulting information. Examples of eachof these uses will be shown in Section 5.

Compa r i son _& Eval ua t ion o f So f twa r e Tool s . Once all thefeatures of a tool are known, its features can be comparedto the features of other tools. The presence (or absence)of a specific feature in a given tool is apparent. Thus, a

potential tool user has a rational means for determiningwhether to include a tool for consideration. For example,if a tool user is interested in a tool that has the featuresof high level language input, program instrumentation,dynamic coverage analysis, and output report generation,then the user may find several tools that fall within thisclassification. The user can then focus on the additionalfeatures offered by each of the tools and perform a costbenefit analysis on these additional features. Once a setof candidate tools has been determined based on desiredfeatures, the user can focus on other considerations, suchas implementation languages, application language, hardwareand software requirements, availability, performance,portability, support and available documentation.

Basis fo r Info rmat ion Organ i za t ion . Since features providea vehicle for the comparison of software tools, one of theirapparent uses is as a basis for retrieval in a database.Just as library searches are based on keywords, toolsearches can be based on features. Relationships can beformed between the features of a tool and its name. Thus,when one wants to know all the tools available that have

3

certain features, then an additional relationship may bedefined that relates these features only with those toolsthat possess them. From here it is a simple matter toprovide any other information that may be desired about thetools in question.

The NBS Software Tools Database [Houg80] is a

relational database that possesses this capability. One ofthe purposes of the database is to initiate a means by whichpersons in the Federal Government can determine what toolsare available and what their capabilities are. An exampleof tool information retrieval from the database is shown inSection 5.

CI assi f ica t ion o f Software Tool

s

. Since features provide a

means by which we can describe software tools, then they canalso provide a means by which we can classify softwaretools. In order to do this, we must discover and describeall the features for every known software development tool,gather them all together, eliminate duplication, and orderthem. The resulting order is called a taxonomy. Thetaxonomy is used to classify tools by associating theordered features with the tool being classified. An exampleof this will be given in section 5 after first presentingthe taxonomy.

4. A Taxonomy of Tool Features

The taxonomy that will be discussed was developed tosatisfy the following goals:

a. The taxonomy shall allow NBS to classify all currentlyavailable software development tools.

b. Each class shall include terms that have the mostspecific meaning where possible,

c. The classes shall allow easy comparison and evaluationof tools.

d. The classes shall be related to the life cycle of anautomated data system.

All of the goals were met except for the last. Life cyclerelationships, although possible, were found to be outsidethe scope of the taxonomy that was developed.

The initial development of the taxonomy was performedunder contract [1] to the National Bureau of Standards and

[1] Contract NB79SBCA0273 to SoHar, Inc. and SoHaRSubcontract No. 102 to Software Management Consultants.

4

is reported in detail in [Reif80]. The present taxonomy isan update of the one presented in that report. Furtherupdates are anticipated as the taxonomy approaches adoptionas a FIPS. Therefore, comments on the usefulness,implementation, structure, and content are solicited by theautho r .

The taxonomy is a hierarchical order of software toolfeatures and is illustrated in figure 1. At the bottom orfeature level of the hierarchy are a total of 52 toolfeatures. Each of these features will be defined anddiscussed in the sections that follow. At the third levelare the classes of tool features. These are subject,control input, transformation, static analysis, dynamicanalysis, machine output, and user output. Each of the 52features have been placed into one of these classes. In

order to provide a way to abbreviate a tool classification,keys have been assigned to each of the elements in the thirdand fourth levels. Each of the keys will be identified inthe sections that follow and examples of their use will beshown in Section 5.

The second level of the taxonomy covers the basicprocesses of a tool. These are input, function, and output.The highest level, software development tool features,covers all the levels below. The taxonomy has been designedto be expandable. It is expected that future updates of thetaxonomy will include additional features and possiblyadditional levels.

5

+ +

ISoftware Development Tool

I

IFeatures

I

+ +

/ I \/ I \

/ I \

+ + + + + +

IInput

I IFunction | I

Output|

+ + + + + +

/ \ / I \ / \

ISub- [Control

ITrans- | Static

|Dynamic | Machine | User

I

Iject

IInput

Iformat ion

IAnal ys i s I Anal ys i s I

Output | Output

|

I I I I I I I I I I I I I I I I I I I I I I I I I i I I I

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+I I I I I I I I I I

Individual FeaturesI I I I I I I I I

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Figure 1. Taxonomy of Software Development Tool Features

6

Input . Input features are based on the forms of input whichcan be provided to a tool. These features fall into twoclasses, one which is based on how the tool should operate(control input) and the other based on what the tool shouldoperate on (subject) . Using a compiler as an example,. thesubject is the code that will be compiled and the controlinput is the set of commands which specify compiler options(optimize, debug, cross reference, etc.).

INPUTSubject Control Input

II; Text CI. Commands12. VHLL C2. Parameters13. Code14. Data

Table 1. Input

Sub j ec t (Key ; I) . The subject is usually the maininput to a tool. It is the input which is subjected tothe main functions performed by a tool. The four typesof subjects are data, code, VHLL (very high levellanguage) , and text.

11. Text - The input to the tool is presented in a

natural language form. Certain types of tools aredesigned to operate on text only (e.g., texteditors, document preparation systems) and requireno other input except directives or commands.

12. VHLL - The input to the tool is a programwritten in a very high level language that istypically not executed. Three recognized types ofVHLL's are requirements languages, designlanguages, and description languages.Requirements and design languages are both used todefine programs and to provide a means forgenerating automatic documentation. Examples ofrequirements languages include the ProblemStatement Language [Teic77] and the RequirementsStatement Language [Bell77]. An example of a

design language is Program Design Language[Cain75]. Description languages are used to

describe attributes of the input in high-level,non-procedural form. An example of a descriptionlanguage is Backus-Naur Form (BNF)

.

7

13. Code - The input to the tool is a programwritten in a language that is subject to a giventranslation process before it is executed. Codeis the language form in which most programmingsolutions are expressed and includes high levellanguages, assembly languages, objectrepresentations or parametric representations.High level languages are problem-oriented (e.g.,COBOL and FORTRAN) , while assembly languages aremachine oriented but symbolic in nature. Objectcode represents an input that is typicallyexecutable without further translation, whileparametric representation (e.g., NAMELIST inFORTRAN IV) inputs variables to be translated.

14. Data - The input to the tool is a string ofcharacters to which meaning is or might beassigned. The input (e.g. raw data) is not in aneasily interpreted, natural language form. Asimulator that accepts numeric data to initializeits program variables is an example of a tool thathas data as input.

Some tools, such as editors, operate on any if the fourof these input forms. In cases such as this, the inputform is chosen from the viewpoint of the tool. Sinceeditors view the input form as text, then text would bethe correct choice for this tool.

Control Input (Key ; C) . Control inputs specify thetype of operation and the detail associated with anoperation. They describe any separable commands thatare entered as part of the input stream.

CI. Command s - The control input to the toolconsists primarily of procedural operators, eachcapable of invoking a system function to beexecuted. A directive invoking a series ofdiagnostic commands (i.e., TRACE, DUMP, etc.) atselected breakpoints is an example. A tool thatperforms a single function will not have thisfeature but will most likely have the next.

C2. Parameters - The control input is a series ofparameters that are associated with the functionsperformed by the tool. Parameters are usuallyentered as a result of a prompt from a tool or maybe embedded in the tool input. An interactivetrace routine that prompts for breakpoints is anexample of a tool with parametric input.

8

Function . The features for this class are shown in Table 2.

They describe the processing functions performed by a tooland fall into three classes: transformation, staticanalysis, and dynamic analysis. Again using a compiler asan example, the transformation features would be translationand possibly optimization, the static analysis featureswould be error checking and possibly cross reference, and a

dynamic analysis feature may be tracing.

Transformation

Tl. Editing SI.

T2. Formatting S2.

T3. Instrumentation S3.

T4. Opt im i zat ion S4.

T5. Restr uctur ing S5.

T6. Translation S6.S7.S8.S9.S10SllS12S13S14S15S16S17818S19

FUNCTION

Static Analysis

Aud itingComparisonComplexity MeasurementCompleteness CheckingConsistency CheckingCost EstimationCross ReferenceData Flow AnalysisError Checking

. Interface Analysis

. Management

. Resource Estimation

. Scanning

. Scheduling

. Statistical Analysis

. Structure Checking

. Tracking

. Type Analysis

. Units Analysis

Dynamic Analysis

Dl. Assertion CheckingD2. Constraint EvaluationD3. Coverage AnalysisD4. Resource UtilizationD5. SimulationD6. Symbolic ExecutionD7. TimingD8. TracingD9 . Tuning

Table 2. Function

9

Transformation (Key ; T) . Transformation features describehow the subject is manipulated to accommodate the user'sneeds. They describe what transformations take place as theinput to the tool is processed. There are sixtransformation features. Each of these features is brieflydefined as follows:

Tl. Ed i ting - modifying the context of the input byinserting, deleting, or moving characters,numbers, or data.

T2. Formatting - arranging a program according topredefined or user defined conventions. A toolthat "cleans up" a program by making all statementnumbers sequential, alphabetizing variabledeclarations, indenting statements, and makingother standardizing changes has this feature.

T3. Instrumentation - adding sensors and counters toa program for the purpose of collectinginformation useful for dynamic analysis [Paig74].Most code analyzers instrument the source code atstrategic points in the program in order tocollect execution statistics required for coverageanalysis and tuning (see D2 and D9)

.

T4. Opt im i za t ion - modifying a program to improveperformance, e.g. to make it run faster or tomake it use fewer resources. Optimizing compilershave this feature.

T5. Restr uctur ing - reconstructing and arranging thesubject in a new form according to well-definedrules. A tool that converts unstructured codeinto structured code is an example of a tool withthis feature.

T6. Transl a t ion - to convert from one language formto another. Tools that have this feature includecompilers, structured language preprocessors, andconversion tools.

b. Static Anal ysis (Key : S) . Static analysisfeatures specify operations on the subject withoutregard to the executab il i ty of the subject [Howd78] .

They describe the manner in which the subject isanalyzed. There are nineteen static analysis features.Each is briefly described as follows:

SI. Aud i t ing - conducting an examination todetermine whether or not predefined rules havebeen followed. A tool that examines the source

10

code to determine whether or not coding standardsare complied with is an example of a tool withthis feature.

52. Compa r i son - Assessing similarities between twoor more items. A tool that compares programs ortest runs for maintaining version control has thisfeature

.

53. Compl ex i ty Measurement - Determining howcomplicated an entity (e.g., routine, program,system, etc.) is by evaluating some number ofassociated characteristics [Mcca76] [Hals77]. Forexample, the following characteristics can impactcomplexity: instruction mix, data references,structure/ control flow, number of interactions/interconnections, size, and number ofcomputations

.

54. Compl eteness Checki ng - assessing whether or notan entity has all its parts present and if thoseparts are fully developed [Boeh78]. A tool thatexamines the source code for missing parametervalues has this feature.

55. Consi stency Checki ng - determining whether ornot an entity is internally consistent in thesense that it contains uniform notation andterminology [Walt78], or is consistent with itsspecification [Robi77]. Tools that check forconsistent usage of variable names or tools thatcheck for consistency between designspecifications and code are examples of tools withthis feature.

56. Cost Est imat ion - assessing the behavior of thevariables which impact life cycle cost. A tool to

estimate project cost and investigate itssensitivity to parameter changes has this feature.

57. Cross Reference - referencing entities to otherentities by logical means. Tools that generatecall graphs or tools that identify all variablereferences in a subprogram have this feature.

58. Data Flow Anal ys i

s

- graphical analysis of thesequential patterns of definitions and referencesof data [Oste76] . Tools that identify undefinedvariables on certain paths in a program have thisfeature.

11

S9. Er ro r Checki ng - determining discrepancies,their importance, and/or their cause. Tools usedto identify possible program errors, such asmisspelled variable names, arrays out of bounds,and modifications of a loop index are examples oftools with this feature,

510. In te r face Anal ys i

s

- checking the interfacesbetween program elements for consistency andadherence to predefined rules and/or axioms. Atool that examines interfaces between modules todetermine if axiomatic rules for data exchangewere obeyed has this feature.

511. Manag ement - aiding the management or controlof software development. Tools that controlaccess, updates, and retrievals of software;tools that maintain and control data definitionand use; and tools that manage test data sets areexamples of tools with this feature.

512. Reso urce Estimation - estimating the resourcesattributed to an entity. Tools that estimatewhether or not memory limits, input/outputcapacity, or throughput constraints are beingexceeded have this feature.

513. Scanning - examining an entity sequentially toidentify key areas or structure. A tool thatexamines source code and extracts key informationfor generating documentation is an example of a

tool with this feature.

514. Sched ul ing - assessing the schedule attributedto an entity. A tool that examines the projectschedule to determine its critical path (shortesttime to complete) has this feature.

515. Statistical Anal ys i

s

- performing statisticaldata collection and analysis. A tool that usesstatistical test models to identify whereprogrammers should concentrate their testing isone example. A tool that tallies occurrences ofstatement types is another example of a tool withthis feature.

516. Structure Checki ng - detecting structural flawswithin a program (e.g. improper loop nestings,unreferenced labels, unreachable statements, andstatements with no successors)

.

12

517. Tracki ng - tracking the development of anentity through the software life cycle. Toolsused to trace requirements from theirspecification to their implementation in code havethis feature.

518. Type Anal ys i

s

- evaluating whether or not thedomain of values attributed to an entity areproperly and consistently defined. A tool thattype checks variables has this feature.

519. Uni ts Anal ys i

s

- determining whether or not theunits or physical dimensions attributed to anentity are properly defined and consistently used.A tool that can check a program to ensurevariables used in computations have proper units(e.g. hertz = cycl es/ seconds) is an example of a

tool with this feature.

Dynam ic Anal ys i s (Key ; D) . Dynamic analysisfeatures specify operations that are determined duringor after execution takes place [Howda78]. Dynamicanalysis features differ from those classified asstatic by virtue of the fact that they require someform of symbolic or machine execution. They describethe techniques used by the tool to derive meaningfulinformation about a program's execution behavior.There are nine dynamic analysis features. Each isbriefly described as follows:

Dl. Assertion Checki ng - checking of user-embeddedstatements that assert relationships betweenelements of a program. An assertion is a logicalexpression that specifies a condition or relationamong the program variables. Checking may beperformed with symbolic or run-time data. Toolsthat test the validity of assertions as theprogram is executing or tools that perform formalverification of assertions have this feature.

D2. Constra int Eval ua tion - generating and/orsolving path input constraints for determiningtest input [Clar76] . Tools that assist thegeneration of or automatically generate test datahave this feature.

D3. Cover ag e Anal ys i

s

- determining and assessingmeasures associated with the invocation of programstructural elements to determine the adequacy of a

test run [Fair78] . Coverage analysis is usefulwhen attempting to execute each statement, branch,path, or iterative structure (i.e., DO loops in

13

FORTRAN) in a program. Tools that capture thisdata and provide reports summarizing relevantinformation have this feature.

D4. Reso ur ce Ut i 1 i za t ion - analysis of resourceutilization associated with system hardware orsoftware. A tool that provides detailed run-timestatistics on core usage, disk usage, queuelengths, etc. is an example of a tool with thisfeature

.

D5. Simulation - representing certain features ofthe behavior of a physical or abstract system bymeans of operations performed by a computer. Atool that simulates the environment under whichoperational programs will run has this feature.

D6. Symbol ic Exec ut ion - reconstructing logic andcomputations along a program path by executing thepath with symbolic rather than actual values ofdata [Darr78]

.

D7. Tim ing - reporting actual CPU times associatedwith parts of a program.

D8. Trac ing - tracking the historical record ofexecution of a program. Tools that produce tracehistories or allow the setting of breakpoints fortracking down errors have this feature.

D9. Tun ing - determining what parts of a program arebeing executed the most. A tool that instrumentsa program to obtain execution frequencies ofstatements is a tool with this feature.

Output . Output features, which provide the link from thetool to the user, are illustrated in Table 3. They describewhat type of output the tool produces for both the humanuser and the target machine (where applicable) . Again usinga compiler as an example, the user output would bediagnostics and possibly listings and tables (crossreference) , and the machine output would be object code orpossibly intermediate code.

14

OUTPUT

User Output Machine Output

Ul.U2.U3.U4.U5.U6.

Computational Results Ml.Diagnostics M2,Graphics M3.Listings M4.Text M5.Tables M6.

DataIntermediate CodeObject CodePromptsSource CodeText

Table 3. Output

User Output (Key ; U) . User output features handlethe interface from the tool to the human user. Theydescribe the types of information that the toolprovides for the user and the form in which this outputis presented. There are six user output features.Each is briefly defined as follows:

Ul. Computational Resul ts - an output from the toolis simply the result of a computation. The outputis not in an easily interpreted natural languagef o rm (e.g. text).

U2. Diagnostics - an output from the tool simplyindicates that a software discrepancy hasoccurred. An error flag from a compiler is anexample

.

U3. Graphics - an output from the tool isgraphically presented with symbols indicatingoperations, flow, etc. A tool providing a

flowchart of a program is an example.

U4. Listings - an output from the tool is a listingof a source program or data and may be annotated.Many different forms of listings can be generated.Some may be user controlled through directives.

U5. Text - an output from the tool is in a naturallanguage form. The output may be a choice of manydifferent types of reports and the formats may beuser defined.

U6. Tables - an output from the tool is arranged inparallel columns to exhibit a set of facts orrelations in a definite, compact and comprehensiveform. A tool that produces a decision tableidentifying a program's logic (conditions,actions, and rules that are the basis of

15

decisions) is an example.

b. Machine Output (Key ; M) . Machine output featureshandle the interface from the tool to a target machine.They describe what the machine expects to see as outputfrom the tool. There are six machine output features.Each is briefly described as follows:

Ml. Data - the input to the machine isrepresentations of characters or numericquantities to which meaning has been assigned. Atool generating input to a plotter is an example.

M2. Intermed ia te Code - the input to the machine isbetween source code and machine code. A toolproducing P-code for direct machine interpretationis an example.

M3. Ob j ec t Code - the input to the machine is a

program expressed in machine language which isnormally an output of a given translation process.A tool producing relocatable load modules forsubsequent execution is an example.

M4. Prompts - the input to the machine is a seriesof procedural operators that are used tointeractively inform the system in which the tooloperates that it is ready for the next input.

M5. Source Code - the input to the machine is theprogram written in a procedural language that mustbe input to a translation process before executioncan take place.

M6. Text - the input to the machine is presented ina written form that can be read without machineinterpretation. A tool producing English textwhich is fed to the machine is an example.

5, Using the Taxonomy

Section 3 discussed in a very general level the use oftool features. Now that we have defined the features of a

tool and given each of them individual keys, we can discusssome very specific uses of the taxonomy.

Tool Classi f icat ion . To classify tools with the taxonomy,one must identify a tool's input, functional, and outputfeatures. As many features are identified as necessary tofully describe the capability of a tool. Identifying tool

16

features, in some cases, can be a major challenge becausemost tool descriptions fail to provide sufficientinformation. Considerable effort may be required to acquirethe information necessary to identify what the tool does andhow it interfaces with the external environment.

The result of classification is a features designatorcalled the taxonomy key. The key is formed by collectingthe individual feature keys according to the their positionin the taxonomy. For example, a tool that has the followingfeatures:

Code InputCommand ControlCode InstrumentationComplexity MeasurementCross Reference GenerationDynamic Coverage AnalysisListing OutputCross Reference and Coverage Reports OutputInstrumented Code Output

would have the following classification when individual keysare chosen:

I3.C1/T3.S3.S7.D3/U4. U5.M5

Note that the slash is used to separate the input,functional, and output features and the period is used toseparate individual keys. It can be seen from the examplethat the key clearly and succinctly communicates the resultsof classification and summarizes the tool attributes.

Since the identification of tool features can bedifficult for some types of tools, it is anticipated that a

user's guide to the taxonomy will be published. This guidewill provide a step-by-step procedure for identifying thefeatures of tools with many actual examples for a widevariety of software development tools.

Comparison o f Software Tool s and Information Retr ieval . Insection 3, we described a user who was interested in a toolthat has the features of high level language input, programinstrumentation, dynamic coverage analysis, and outputreport generation. Using individual keys from the taxonomy,a retrieval request could be specified as follows:

I3/T3&D3/U5

where 13, T3, D3, and U5 represent the features, the slashis used to separate input and output, and the & is a logical'and' of the features. In English this retrieval request

17

would be: "Give me all the tools which accept high levellanguage programs as input, perform the functions of programinstrumentation and dynamic coverage analysis, and providereports as output." A retrieval of this request from the NBSSoftware Tools Database would yield thetools:

following list of

NBS ANALYZERPETNODALITDEMCOTUNE IILOGICFAVS

JOVIAL TCARXVPPACETEST PREDICTORTATTLEPDSCAVS

EAVSDYNAPACE-CJIGSAWTPTJAVS

After examining and comparing the application languages andthe additional features offered by these tools, theretrieval request could be further specified as follows:

13 ' FORTRAN/T3&D3& (S1'^S8)/U5

where SI represents code auditing, S8 represents data flowanalysis, the ' delineates the actual language of the sourcecode, and " is a logical 'or'. This request in Englishwould be: "Give me all the tools which accept Fortranprograms as input, perform the functions of programinstrumentation and dynamic analysis and either codeauditing or data flow analysis, and provide reports asoutput." This retrieval request would yield the followingtool s

:

PET FAVS TPT

Note that the number of tools that a potentialmust consider has been greatly reduced.

tool user

6. Scope of the Taxonomy

The taxonomy is primarily a classification scheme forsoftware development tools. There are, however, many toolsthat are broader in function and application that can not beeasily classified by the taxonomy. These tools includeoperating systems, system utilities (linkage editors,loaders, tape handlers, file system, sorters), data basemanagement systems, and management information systems.These tools serve as an extended part of a system and areoutside the scope of the taxonomy. Since most of thefeatures in the taxonomy are specific to the softwaredevelopment process, only tools specific to softwaredevelopment should be classified.

18

7. Conclusion

Software tools are powerful productivity and qualityaids that are not being effectively used in many Federalprogramming environments. This report has discussed aneffort to lessen this problem by providing a formal way inwhich tools can be classified. The use of a taxonomy oftool features can be used to (1) categorize currentlyavailable tools, (2) standardize terminology associated withtools, and (3) ease the task of comparing and evaluatingtools. Use of the taxonomy should help users determineprecisely what a given tool can offer and consequently whatits benefits are.

8. References

[Bell77] T. E. Bell, D. C. Bixler and M. E. Dyer, "AnExtendable Approach to Computer-Aided SoftwareRequirements Engineering", IEEE Transactions onSoftware Engineering, Vol SE-3 , No 1, 1977.

[Boeh78] B. W. Boehm, J. R. Brown, H.

Lipow, G. J. MacLeod and M.

"Characteristics of SoftwareNorth-Holland Publishing Company, NY,

Kaspar, M.J . Me r r i tt

,

Qual i ty"

,

1978.

[Cain75] S. H. Caine and E. K. Gordon, "PDL: A Toolfor Software Design", Proceedings of the NationalComputer Conference, 1975.

[Clar76] L. A. Clarke, "A System to Generate Test Dataand Symbolically Execute Programs", IEEETransactions on Software Engineering, Vol SE-2,September 1976.

[Darr78] J. A. Darringer and J.of Symbolic ExecutionComputer, April 1978.

C. King, "Applicationsto Program Testing",

[Fair78] R. E. Fairley,Dynamic TestingApril 1978.

"Tutorial: Static Analysis andof Computer Software", Computer,

[Hals77] M. H. Halstead, "Elements of SoftwareElsevier - North Holland Pub. Co.,1977 .

Sc ience" ,

New York,

[Houg80] Raymond C. Houghton, Jr. and Karen A. Oakley,eds., "NBS Software Tools Database", NBSIR80-2159, October 1980.

19

[Howd78] W. E. Howden, "A Survey of Static AnalysisMethods", Tutorial: Software Testing andValidation Techniques, IEEE Cat. No. EH0138-8,1978.

[Howda78] W. E. Howden,"A Survey of Dynamic AnalysisMethods", Tutorial: Software Testing andValidation Techniques, IEEE Cat. No. EH0138-8,1978.

[Mcca76] T. J. McCabe, "A Complexity Measure", IEEETransactions on Software Engineering, Vol SE-2,December 1976.

[Oste76] L. J. Osterweil and L. D. Fosdick, "DAVE - A_ Validation Error Detection and Documentation- System for FORTRAN Programs", Sof tware~Pract ice

and Experience, October 1976.

[Paig74] M. R. Paige and J. P. Benson, "The Use ofSoftware Probes in Testing FORTRAN Programs",Computer, July 1974.

[Reif80] Donald J. Reifer and Harold A. Montgomery,"Final Report, Software Tool Taxonomy", SoftwareManagement Consultants Report No. SMC-TR-004,June 1980.

[Robi77] L. Robinson and K. N. Levitt, "Proof Techniquesfor Hierarchically Structured Programs",Communications of the ACM, April 1977.

[Teic77] D. Teichroew and E. Hershey III, "PSL/PSA: AComputer-Aided Technique for StructuredDocumentation of Information Processing Systems",IEEE Transactions on Software Engineering, VolSE-3, No 1, 1977.

[Walt78] G. Walters and J. McCall, "The Development ofMetrics for Software Reliability andMaintainability", Proceedings of the AnnualReliability and Maintainability Symposium, January1978.

20

NBS-n4A IREV. 2.8CI

U.S. DEPT. OF COMM.

BIBLIOGRAPHIC DATASHEET (See instructions)

1. PUBLICATION ORREPORT NO.

SP 500-74

2. Performing Organ. Report No. 3. Publication Date

February J.yoi

-f-ITl d Al^l^^ CMDTITI CTITLt AN U bU D 1 1 1 Lb

Features of Software Development Tools

5. AUTHOR(S)

Raymond C. Houghton, Jr.

6. PERFORMING ORGANIZATION (If joint or other than NBS. see instructions) 7. Contract/Grant No.

NATIONAL BUREAU OF STANDARDSDEPARTMENT OF COMMERCE 8. Type of Report & Period Covered

WASHINGTON, D.C. 20234

Final

j9. SPONSORING ORGANIZATION NAME AND COMPLETE ADDRESS (Street. City. State. ZIP)

I

I,

I

10. SUPPLEMENTARY NOTES

Library of Congress Catalog Card Number: 80-600193

(2J Document describes a computer program; SF-185, FIPS Software Summary, is attached.

1 11. ABSTRACT ( A 200-word or iess foctual summary of most significant information. If document includes a significantbibliography or literature survey, mention it here)

Software tools are powerful productivity and quality aids that in manycases are not being used effectively. This report discusses an effort tolessen this problem by providing a formal way in which tools can be classifiedaccording to the features that they provide.

i1 12. KEY WORDS (Six to twelve entries; alphabetical order; capitalize only proper names; and separate key words by semicolon s)

1

Dynamic analysis ; programming aids; software development; software engineering

;

software tools; static analysis ; taxonomy

13. AVAILABILITY

Unlimited

1 1For Official Distribution. Do Not Release to NTIS

DTI Order From Superintendent of Documents, U.S. Government Printing Office, Washington, D.C.20402.

14. NO. OFPRINTED PAGES

24

15. Price

Q Order From National Technical Information Service (NTIS), Springfield, VA. 22161

f

$1.75

1 it U.S. GOVERNMENT PRINTING OmCE : 1981 O— 340-997 (1526)

_

USCOMM-OC e043-P80

ANNOUNCEMENT OF NEW PUBLICATIONS ONCOMPUTER SCIENCE & TECHNOLOGY

Superintendent of Documents,

Government Printing Office,

Washington, D. C. 20402

Dear Sir:

Please add my name to the announcement list of new publications to be issued in

the series: National Bureau of Standards Special Publication 500-.

Name

Company

Address

City State Zip Code

(Notification key N-S03)

J

NBS TECHNICAL PUBLICATIONS

PERIODICALS

JOURNAL OF RESEARCH—The Journal of Research of the

National Bureau of Standards reports NBS research and develop-

ment in those disciplines of the physical and engineering sciences in

which the Bureau is active. These include physics, chemistry,

engineering, mathematics, and computer sciences. Papers cover a

broad range of subjects, with major emphasis on measurement

methodology and the basic technology underlying standardization.

Also included from lime to time are survey articles on topics

closely related to the Bureau's technical and scientific programs.

As a special service to subscribers each issue contains complete

citations to all recent Bureau publications in both NBS and non-

NBS media. Issued six times a year. Annual subscription: domestic

$13; foreign S16.25. Single copy, $3 domestic; $3.75 foreign.

NOTE: The Journal was formerly published in two sections: Sec-

tion A "Physics and Chemistry" and Section B "Mathematical

Sciences."

DIMENSIONS/NBS—This monthly magazine is published to in-

form scientists, engineers, business and industry leaders, teachers,

students, and consumers of the latest advances in science and

technology, with primary emphasis on work at NBS. The magazine

highlights and reviews such issues as energy research, fire protec-

tion, building technology, metric conversion, pollution abatement,

health and safety, and consumer product performance. In addi-

tion, it reports the results of Bureau programs in measurement

standards and techniques, properties of matter and materials,

engineering standards and services, instrumentation, and

automatic data processing. Annual subscription; domestic $11;

foreign $13.75.

NONPERIODICALS

Monographs—Major contributions to the technical literature onvarious subjects related to the Bureau's scientific and technical ac-

tivities.

Handbooks—Recommended codes of engineering and industrial

practice (including safety codes) developed in cooperation with in-

terested industries, professional organizations, and regulatory

bodies.

Special Publications—Include proceedings of conferences spon-

sored by NBS, NBS annual reports, and other special publications

appropriate to this grouping such as wall charts, pocket cards, andbibliographies.

Applied Mathematics Series—Mathematical tables, manuals, andstudies of special interest to physicists, engineers, chemists,

biologists, mathematicians, computer programmers, and others

engaged in scientific and technical work.

National Standard Reference Data Series—Provides quantitative

data on the physical and chemical properties of materials, com-piled from the world's literature and critically evaluated.

Developed under a worldwide program coordinated by NBS underthe authority of the National Standard Data Act (Public Law90-396).

NOTE: The principal publication outlet for the foregoing data is

the Journal of Physical and Chemical Reference Data (JPCRD)published quarterly for NBS by the American Chemical Society

(ACS) and the American Institute of Physics (AIP). Subscriptions,

reprints, and supplements available from ACS, 1 155 Sixteenth St.,

NW, Washington, DC 20056.

Building Science Series—Disseminates technical information

developed at the Bureau on building materials, components,

systems, and whole structures. The series presents research results,

test methods, and performance criteria related to the structural andenvironmental functions and the durability and safety charac-

teristics of building elements and systems.

Technical Notes—Studies or reports which are complete in them-

selves but restrictive in their treatment of a subject. Analogous to

monographs but not so comprehensive in scope or definitive in

treatment of the subject area. Often serve as a vehicle for final

reports of work performed at NBS under the sponsorship of other

government agencies.

Voluntary Product Standards—Developed under procedures

published by the Department of Commerce in Part 10, Title 15, of

the Code of Federal Regulations. The standards establish

nationally recognized requirements for products, and provide all

concerned interests with a basis for common understanding of the

characteristics of the products. NBS administers this program as a

supplement to the activities of the private sector standardizing

organizations.

Consumer Information Series— Practical information, based on

NBS research and experience, covering areas of interest to the con-

sumer. Easily understandable language and illustrations provide

useful background knowledge for shopping in today's tech-

nological marketplace.

Order the above NBS publications from: Superintendent of Docu-

ments, Government Printing Office, Washington, DC 20402.

Order the following NBS publications—FIPS and NBSlR's—fromthe National Technical Information Services, Springfield, VA 22161

.

Federal Information Processing Standards Publications (FIPS

PUB)—Publications in this series collectively constitute the

Federal Information Processing Standards Register. The Register

serves as the official source of information in the Federal Govern-

ment regarding standards issued by NBS pursuant to the Federal

Property and Administrative Services Act of 1949 as amended.

Public Law 89-306 (79 Stat. 1127), and as implemented by E\

ecutive Order 1 1717 (38 FR 12315, dated May 1 1, 1973) and Part o

of Title 15 CFR (Code of Federal Regulations).

NBS Interagency Reports (NBSIR)—A special series of interim or

final reports on work performed by NBS for outside sponsors

(both government and non-government). In general, initial dis-

tribution is handled by the sponsor; public distribution is by the

National Technical Information Services, Springfield, VA 22161,

in paper copy or microfiche form.

U.S. DEPARTMENT OF COMMERCENational Bureau of StandardsWashington, D C. 20234

OFFICIAL BUSINESS

Penalty for Private Use. $300

POSTAGE AND FEES PAIDU.S. DEPARTMENT OF COMMERCE

COM-21S

SPECIAL FOURTH-CLASS RATEBOOK