Cognitive Dimensions and Multiple Notations

Tim Wright and Andy Cockburn

September 1, 2005

Contents

1 Classification Scheme 11.1 Sample Programming Environments1

1.1.1 Logo . . . . . . . . . . . 11.1.2 StageCast. . . . . . . . . 31.1.3 OpenOffice.org. . . . . . 4

1.2 Three Fundamental Activities. . . 41.3 Three Cognitive Gulfs . . . . . . 51.4 Language Signatures. . . . . . . 6

2 Advanced Multiple Notation Environ-ments 72.1 Environments with multiple nota-

tions for reading. . . . . . . . . . 72.2 Environments with multiple nota-

tions for writing . . . . . . . . . . 72.3 Environments with Multiple Nota-

tions for Reading and Writing . . 92.4 Environments with an Arbitrary

Number of Notations. . . . . . . 9

3 Green and Petre’s Cognitive Dimen-sions 9

4 Summary 10

Abstract

Cognitive Dimensions is widely used frameworkto evaluate visual notations. Our research indi-cates that many programming environments letpeople program using multiple notations. Unfor-tunately, Cognitive Dimensions does not providemechanisms to assess the relationship between thenotations. In this paper we introduce our view ofhow notations are used in programming environ-ments and we perform a preliminary extention ofcognitive dimensions so it applies over multipledimensions.

Much of this work is taken from Tim Wright’sPhD thesis [21].

1 Classification Scheme

Central to our understanding of how notations areused in programming environments are two con-cepts: the activities people perform while pro-gramming and a method of specifying how the ac-tivities are supported by notations in a program-ming environment. We determined the fundamen-tal activities by performing an activity-based de-composition of programming, and identified threefundamental activities: reading programs, writingprograms, and watching programs run. We callour method of specifying how the activities aresupported in a programming environment a Lan-guage Signature. This section introduces our threeprogramming activities and Language Signatures.To aid the explanation of the activities and gulfs,we begin by introducing three programming envi-ronments that will be used as examples throughoutthis chapter: Logo, StageCast, and OpenOffice,and we examine how notations are used in theseenvironments.

1.1 Sample Programming Environ-ments

This section describes three sample programmingenvironments and examines how notations areused in these environments. From these environ-ments, we learn two things. First, that program-ming environments do use multiple notations andthat using different notations could cause prob-lems for users of the environments). Second, thesenotations are used for three separate programmingactivities: reading, writing, and watching. Thesethree activities are the foundation of the frame-work for understanding how notations are usedprogramming environments and are discussed fur-ther in the following section.

1.1.1 Logo

Logo, developed by Papert, is an environmentwhere users edit text to control how a turtle moves

1

sub transposerem ----------------------------------------------------------------------rem define variablesdim document as objectdim dispatcher as objectrem ----------------------------------------------------------------------rem get access to the documentdocument = ThisComponent.CurrentController.Framedispatcher = createUnoService("com.sun.star.frame.DispatchHelper")

rem ----------------------------------------------------------------------dim args1(1) as new com.sun.star.beans.PropertyValueargs1(0).Name = "Count"args1(0).Value = 1args1(1).Name = "Select"args1(1).Value = true

dispatcher.executeDispatch(document, ".uno:GoLeft", "", 0, args1())

rem ----------------------------------------------------------------------dispatcher.executeDispatch(document, ".uno:Cut", "", 0, Array())

rem ----------------------------------------------------------------------dim args3(1) as new com.sun.star.beans.PropertyValueargs3(0).Name = "Count"args3(0).Value = 1args3(1).Name = "Select"The largest number of end-user programming environments use two notations.args3(1).Value = false

dispatcher.executeDispatch(document, ".uno:GoRight", "", 0, args3())

rem ----------------------------------------------------------------------dispatcher.executeDispatch(document, ".uno:Paste", "", 0, Array())

end sub

Figure 1: Automatically generated OpenOffice.org macro to transpose two adjacent characters.

2

Figure 2: A mockup of the Logo ProgrammingEnvironment. Users read and write programs us-ing text, but see a turtle move around drawing lineswhen their program is executed.

around a screen and draws lines [12]. For exam-ple, a user might type:

REPEAT 4 [ Do the following 4 timesFD 50 MoveForwarD 50 unitsRT 90 TurnRighT 90 degrees

]

and see a picture of a square. A mock-up Logoenvironment is shown in Figure2.

The Logo environment contains two notations.The first is the textual description of what the turtlewill do when the program is executed and is usedwhen a user reads or writes a program. The secondnotation is the collection of lines that are drawn asthe turtle that moves around the screen as well asthe turtle itself drawing the lines. This notation isused for watching a program run

We postulate that the difference in notations be-tween the reading/writing and watching activitiescould cause problems for a user as they must rea-son about one notation using another notation. Forexample, a user might ask themselves “why did theturtle turn right instead of left” and have to rea-son about their program by asking “Is my mistaketypingLT instead ofRT or FD instead ofBW?”.Later in this chapter we examine this type of us-ability problem, caused when users must interactwith different notations for different programmingactivities.

1.1.2 StageCast

StageCast (formally called KidSim or Cocoa) is aprogramming environment for 2D visual simula-tions [17]. 2D simulations are programs that runin a two dimensional area of a screen where agentscan move around and interact with other. An ex-ample simulation is shown in Figure3. StageCastusers writebefore-afterrules to define behaviour

Figure 3: A 2D simulation. Many program-ming environments, including StageCast [17],AgentSheets [16], and Playground [4], use this do-main for programming. The environment pictured(PatternProgrammer) was developed by Wright[20].

Figure 4: A before/after rule similar to those usedin StageCast [17]. A user is writing a programto move a baboon past a bush. When StageCastexecutes, it will search the simulation for arrange-ments of agents matching the left-hand side of therule and replace the agents with the arrangementof agents on the right-hand side of the rule.

3

in the simulation. A before after rule has two parts:an arrangement of agents to search for in the sim-ulation, and an arrangement of agents with whichto replace the found agents. An example rule isshown in Figure4 (on page3).

StageCast avoids problems of different nota-tions by using the same notation for reading, writ-ing, and watching programs: users write before-after rules by manipulating visual representationsof agents and when users run their program theysee the same set of agents move around and inter-act with each other.

Unfortunately, StageCast is not perfect. Us-ability studies of StageCast found that childrenhave trouble predicting what StageCast will dowhen their program is executing [13]. The prob-lem was caused by a difference in StageCast’s rulescheduling algorithm and users’ expectations ofrule scheduling. This type of problem, where theway a program is executed is different from user’sexpectations of how it will be executed, is investi-gated in this chapter.

1.1.3 OpenOffice.org

OpenOffice.org is a free1 word processing pro-gram with capabilities similar to Microsoft Word.In particular, OpenOffice.org provides functional-ity so users can program macros to manipulatetheir documents. Users program using the sym-bols provided by the environment: for example, auser wishing to program a macro to transpose twoadjacent letters might turn on macro recording, se-lect and cut the letter to the right of the mouse cur-sor, then move the cursor left and paste the let-ter. Users can then execute the macro and watchletters in their document change places. Unfortu-nately, if users discover a problem with their pro-gram they must either re-demonstrate the programfrom scratch or edit their program using a complextextual language: OpenOffice Basic (an exampleof OpenOffice Basic can be found in Figure1 onpage2).

We postulate that this difference, between thenotation used for writing and watching programsand the notation used for reading and writing theprogram, could cause usability problems for a userof OpenOffice. Again, this chapter examines thistype of usability problem, caused by users inter-acting with different notations for different pro-gramming activities.

1OpenOffice.org is licensed under the GNU GPL.

1.2 Three Fundamental Activities

The previous section describes three programmingenvironments: Logo, StageCast, and OpenOffice,and describes several ways programming environ-ments can use notations. The three environmentsdescribed use notations in very different ways:Logo uses one notation for editing programs andanother notation for watching programs; Stage-Cast uses the same notation for editing and watch-ing programs; and OpenOffice uses one notationfor writing and watching programs and anothernotation for editing programs. By decompose theediting task into two sub-tasks, reading and writ-ing, we uncover three fundamental programmingactivities: reading, writing, and watching. Thesethree activities are both fundamental to program-ming and fundamental to our framework for un-derstanding how notations are used in program-ming environments. We now describe each activ-ity in depth.

Reading is the act of viewing a notation describ-ing program behaviour, writing is the process ofusing a notation to describe program behaviour,and watching is the act of viewing program be-haviour, either viewing an animation of the nota-tion or viewing the behaviour specified by the no-tation. The activities are shown in Figure7. Also,we use the term the “program representation” torefer to any notation used for reading and we usethe term the “program visualisation” to refer toany notation used for watching: program visual-isations can range from animations of the code toagents interacting in a 2D simulation.

To use an example (described in the previoussection and also shown in Figure7), consider auser of a word processing program who wantsto write a program to transpose two characters.First, using their word processor’s programmingby demonstration mechanism, they record them-selves transposing two characters. This record-ing of their behaviour is the writing activity, andin this example they are writing using the iconsand behaviour provided by their word processor.Next, they execute their macro a couple of times totranspose various characters that were out of order.This execution of their program is the watching ac-tivity. Again, in this example, they are watchingtheir program using the icons and symbols pro-vided by the word processor. After they have exe-cuted their macro several times, they discover thatthere is a bug in their macro: the macro transposesthe two characters to the left of the cursor insteadof the intended behaviour of the two characters

4

1. User has intentions and a model of how system works.

2. These intentions must be expressed using a set of allowable actions.

3. User must interpret state ofsystem to determine how well intentions have been met.

I want to make the fridge colder

I turn the left dial

Hmm. The fridge got colder, but the freezer got warmer.

User Updates Model of system

Gulf of Evaluation

Gulf ofExecution

Figure 5: Donald Norman’s gulfs of Executionand Evaluation. The gulf of execution is “the dif-ference between user intentions and allowable ac-tions” and the gulf of evaluation is “the amountof effort the the person must exert to interpret thephysical state of the system and to determine howwell the expectations and intentions have beenmet” [10]

surrounding the cursor. To fix their program, theuser opens their macro in the macro editor to firstread and understand it. This is the reading activ-ity. In many current word processors, they will seetheir code in a textual form close to a conventionalprogramming language. In this example, the pro-gram representation is the textual form they viewto edit their macro while the program visualisationrefers to the dynamic effects caused by the user ex-ecuting their program.

1.3 Three Cognitive Gulfs

In 1988, Norman examined the real world for us-ability problems [10]. He identified two cogni-tive gulfs: the gulfs of execution and evaluation.He describes the gulf of execution as “the dif-ference between user intentions and allowable ac-tions” and the gulf of evaluation as reflecting “theamount of effort the the person must exert to in-terpret the physical state of the system and to de-termine how well the expectations and intentionshave been met”. For example, imagine someonetrying to decrease the temperature of a fridge (Fig-ure5). First, they examine the dials on the fridgeand note two dials. Assuming that one dial isfor the fridge and the other dial for the freezer,they turn the left hand dial to the right. This dif-ference between the user’s intentions (make thefridge colder) and the allowable actions (two dialsthat can be turned left or right) is the gulf of exe-cution. After letting the fridge’s temperature sta-bilise, the user notices that while the fridge has be-

1. User has intentions and a model of how system works.

2. These intentions must be expressed using a set of allowable actions (language syntax).

3. User must interpret state ofsystem to determine how well intentions have been met.

I want to make a computer game

I type some stuff

What does 'compileerror' mean?

User Updates Model of system

Gulf ofExecution

Gulf of Evaluation

Figure 6: Donald Norman’s gulfs of Executionand Evaluation in a programming context [10].The gulf of execution is the difference betweena user’s model of desired program behaviour andhow they must express their program to the com-puter while the gulf of evaluation is how hard itis for the user to figure out if they have expressedtheir program correctly.

come colder, the freezer has also become warmer.The effort the user must expend interpreting thesystem and understanding why their actions hada different effect than they expected is Norman’sgulf of evaluation. Norman’s gulfs are shown inFigure5.

Figure 6contextualizes Norman’s gulfs in a pro-gramming context. While the gulf of executionis still applicable (users must translate their men-tal model into the symbols of a programming lan-guage), the gulf of evaluation has become morecomplex: a program has both a static representa-tion and a dynamic visualisation that users mustinterpret to determine how well their intentionshave been met. To distinguish the different gulfsof interpreting the representation of visualisation,we decompose Norman’s gulf of evaluation intotwo gulfs: the gulf of representation and the gulfof visualisation.

So, in a programming context, Norman’s twogulfs become three (and are shown in Figure7). Agulf of expression is created when the user’s men-tal model of desired program behaviour differs tohow the user must express the program for read-ing. A gulf of representation is created when theuser’s mental model of program behaviour and theprogram representation. A gulf of visualisation iscreated when the user’s mental model of programbehaviour differs from the program visualisation.

As an example, consider a user writing a pro-gram to transpose two letters. If the user is usinga word processor with a macro recorder, they canwrite the program using the icons and behaviour

5

Figure 7: The gulfs of expression, representation,and visualisation and their associated activities.In this example a user is writing a program us-ing programming by demonstration mechanisms.The user is reading an computer generated repre-sentation of their program. When they run theirprogram they see letters in their document beingtransposed.

of their word processor. As they are likely fa-miliar with the icons and behaviour of their wordprocessor, we argue they have a low gulf of ex-pression. However, imagine that their word pro-cessor had no macro recorder and the user had towrite their program using a conventional textuallanguage (like Visual Basic). Unless the user is ex-perienced with this conventional textual notation,they will be unfamiliar with the notation and weargue the notation creates a high gulf of expres-sion.

We define the gulf of visualisation as the cogni-tive difference between a user’s mental model ofprogram behaviour and the program visualisation(what the program actually does as it executes).To continue the example of a user writing a pro-gram to transpose two letters, consider the possi-bility that when the users executes their program,the program does not behave as expected. Al-though the program does not behave as expected,it uses the same symbols and behaviour that theuser wrote the program in (the icons and behaviourof their word processor), so we argue that there isa low gulf of visualisation (in this example we areusing the notation the user wrote their program inas a cognate for their mental model).

We define the gulf of representation as the cog-nitive difference between the users mental modelof their program and the program representation.As an example, consider our user who is writing a

program to transpose two letters. After they havewritten their program, they run their program anddiscover that it doesn’t work as expected. Whenthey view their program, they see conventionaltextual program code. Like the gulf of visualisa-tion, we can use the notation the user wrote theprogram in as a cognate for their mental model. Asthe notation consisting of the icons and behaviourof a user’s word processor is very different froma conventional textual programming language, weargue this example creates the risk of a high gulfof representation.

Smith, Cypher, and Tesler have also done workcontextualising Norman’s gulfs in a programmingcontext [18]. They argue that the appropriate wayto make programming easier is to reduce Nor-man’s gulfs by moving the system closer to theuser. Their assumption in this is that there is a con-sistent representation of the programming system:people write, read, and watch programs using thesame notation. Our structure of the three activi-ties (reading, writing, and watching) and the threegulfs (execution, representation, and visualisation)gives us flexibility to argue about systems that usedifferent notations for the three activities, or evendifferent notations for the same activity. We usean abstract syntax for how notations are used inprogramming environments called Language Sig-natures. Language Signatures are described in thefollowing section.

1.4 Language Signatures

By describing how programming environmentsuse notations for our three activities we can com-pare different programming environments and dis-cover ways of using notations that help userswhen programming and ways of using notationsthat hinder users when programming. Unfortu-nately, the description of how different notationsare used for different activities is long, increasingthe chance that someone will misread a notationdescription. They also make comparing how nota-tions are used in different programming environ-ments harder. For an example description, con-sider Logo. Logo is a programming environmentwhere programmers describe how a turtle movesaround a 2D surface and draws lines. An examplelogo program with output is shown in Figure2.The description of how notations are used in Logois: users read and write programs using textualcommands and watch a turtle draw lines.

To avoid parsing problems, and to help peoplecompare different how different programming en-

6

vironments use different notations, we use Lan-guage Signatures. A Language Signature suc-cinctly expresses how a programming environ-ment uses different notations for the three funda-mental programming activities. It is written insidesquare brackets with plus (+) symbols separatingdifferent notations. Each notation is described bystating the activities it supports, abbreviated to RE

(Reading), WR (Writing), and WA (Watching) andseparated by slash (/) symbols. For clarity, a tex-tual description of the notation’s symbols can besubscripted to the description of the activities sup-ported by that notation. To continue our exam-ple, Logo’s Language Signature is [RE/WRtext +WAturtle, lines].

An alternative to the Language Signature syn-tax we decided on is to have the Language Sig-nature describe first activities and second the no-tations used for that activity. The Logo signa-ture might then look like this: [Retext + Wrtext

+ Waturtle,lines]. This way of expressing Lan-guage Signatures suffers several flaws: it is hardto immediately determine the number of notationspresent in an environment; it is hard to determineif Logo uses the same notation for reading andwriting, and it is hard to see which activities a no-tation does not support.

2 Advanced Multiple NotationEnvironments

2.1 Environments with multiple nota-tions for reading

This section describes programming environmentsthat have multiple notations for reading and onenotation for writing. These programming environ-ments all risk creating a gulf of expression: withmultiple notations for reading and one for writ-ing they must have at least one read-only notation.Two environments in this category overcome thisgulf by providing an transient read-only notation:the program is read through the computer’s speak-ers. Unfortunately only informal studies have ex-amined the effects of a transient notation.

We found three programming environments inthis category: AgentSheets, Pecan, and Mondrian.

AgentSheetsis a programming environment forvisual simulations [16]. Users describe agentbehaviour with an iconic language. TheAgentSheets environment can also read aprogram through the computers speakers,

and has a [RE/WR/WAiconic + REspoken +WAagents] Language Signature. Anecdotalevidence about AgentSheets reveals that us-ing multiple program representations helpsusers write and understand programs2. Webelieve that AgentSheets’ spoken notationdoes not create a gulf of expression becausethe spoken representation is transient: usersintuitively know they cannot edit a spokenrepresentation. However, more research isneeded to confirm this belief, and examinethe effect of letting people write using aspoken notation: letting people dictate pro-grams to computers (which would changeAgentSheets’ signature to [RE/WR/WAiconic

+ RE/WRspoken + WAagents]).

Mondrian is a programming by example sys-tem for creating interactive drawings [8].Users can program new commands into Mon-drian by creating before-after rules. Whileusers are creating these rules, Mondrian readsthe users’ commands through the computer’sspeakers, and Mondrian can also convert theuser-defined rules into Lisp code.

Pecan produces a read-only Nassi-Shneidermandiagram from code [14]. Although we couldnot find any papers describing usability stud-ies of Pecan, we argue that Pecan risks creat-ing a gulf of expression because users mightbuild a mental model of program behaviourbased on the Nassi-Shneiderman diagram,and then want to edit the Nassi-Shneidermandiagram directly.

2.2 Environments with multiple nota-tions for writing

Environments with multiple notations to write pro-grams, but one way to read programs, risk creatinga gulf of representation: users must map from thenotation they used for writing to the notation theyuse for reading. We found one three-language en-vironment with multiple ways to write a program:Leogo [2]. Leogo levers the gulf of representationto teach users to program and is pictured in Fig-ure8.

Leogo is an extension of Logo where users canprogram using three different notations: textuallogo code, an iconic version of the code using but-tons for commands and sliders for amounts, and a

2Informal conversation with Alexander Reppenning

7

Type DescriptionText The notation is based on users typing or manipulating textual statements.

The statements might be like conventional code or more natural. Exam-ple environments that use a textual notation are Alice [3], Hands [11],and C [7].

Iconic Iconic notations are notations that use icons for programming statementsor have animated icons for visualisations. Flowcharts programming no-tations are an example of iconic programming notations. Environmentswith an iconic visualisations include StageCast and PatternProgrammer(see Figure3).

User Interface Some programming environments use the set of symbols present in astandard user interface for writing and watching programs. Typically,these environments are programming by demonstration environments(PBD) where users can demonstrate behaviour using a standard user in-terface and the PBD environment attempts to infer the user’s programand use the program at a later time.

Tangible Tangible notations use (for program representation) physical items inplace of program statements, or (for program visualisation) physicalitems that move and interact. An example environment that has a tan-gible representation is AlgoBlock: users join together physical cubeswhere each cube represents a single Logo statement [19]. An exam-ple environment that uses a tangible visualisation is Electronic Blocks:when a program is run, users see physical blocks make sounds, lights,move around, and interact with each other [23].

Table 1: Description of some common notation types used in Language Signatures.

Figure 8: The Leogo programming environment.Users can manipulate any of three notations (tex-tual, iconic, or directly-manipulate the turtle), andsee the results immediately in the other two nota-tions. For example, a user who wants to move theturtle forward 50 units can either drag the turtleforward 50 units and see the statement “FD 50”appear as well as the icon for forward movementdepress and a slider advance to 50, or they can type“FD 50” and watch the changes in the iconic rep-resentation and the output domain.

direct-manipulation notation where users can ma-nipulate the turtle immediately. Changes in onenotation in Leogo are immediately reflected in theother two notations (see Figure8). The rationalbehind using multiple notations was to help userslever their knowledge of how a turtle and iconicinterfaces work to help children learn Logo. Anevaluation of Leogo found that children could useLeogo, and that they tended to pick one nota-tion and stay with that. Leogo’s Language Sig-nature is [WR/WAturtle, lines + WR/WAiconic +RE/WR/WAtext], and is shown in Figure8.

One of Leogo’s motivations was to help aidknowledge transfer from a notation children wouldbe familiar with (the way the turtle moved) to anotation that provided more power (the textual no-tation). Both these environments provide multiplenotations for reading and writing—one close to thetask domain and one close to conventional code.The designers argue that this decision is useful asit can aid knowledge transfer from domain spe-cific knowledge to knowledge about conventionalcode. Unfortunately the designers of the two en-vironments did not perform an evaluation of thesuccesses of the knowledge transfer.

8

Figure 9: A screen snapshot of Mulspren: A MUl-tiple Language Simulation PRogramming ENvi-ronment.

2.3 Environments with Multiple No-tations for Reading and Writing

Mulspren is a three-language environment thatuses multiple languages to read and write pro-grams [22]. Mulspren avoids the risk of creatinggulfs of expression, representation, and visualisa-tion by using duel languages for writing and read-ing: users can move seamlessly between two tex-tual languages modifying either and viewing thechanges in both. Both languages are animatedwhen a program is executed. We plan to use Mul-spren to evaluate the effects of multiple languageson users without the hindrance of gulfs of repre-sentation and expression. A screen snapshot ofMulspren is inFigure 9.

2.4 Environments with an ArbitraryNumber of Notations

The only environment we reviewed that supportsan arbitrary number of notations is the Gardenprogramming environment [15] (Garden is a suc-cessor of the Pecan programming environment,which is described in the previous section). Usersof Garden write programs in one of any num-ber notations, and can add new notations to theGarden environment. Reiss describes how a usercan extend Garden so the user can program us-ing petri-nets. Despite this expressive power ofGarden, users must use the same notation to readtheir program as the one in which they wrotetheir program. This means that Garden pro-vides n notations for writing but only one for

reading—the notation that the user wrote their pro-gram in. We write Garden’s Language Signatureas [(WR+WR+. . . )/RE] rather than [RE/WR +RE/WR + . . . ], as the latter conveys that users canmove between the different notations after theyhad started writing code.

Unfortunately no user studies were performedon Garden, so we can neither analyse what the ef-fects of user-extensible environments are on usersnor examine what gulfs are created in a user-extensible programming environment.

3 Green and Petre’s CognitiveDimensions

Cognitive Dimensions is a framework developedto analyse visual programming notations and theprogramming environments used to create, mod-ify, and execute the notations [6]. The frame-work has thirteen dimensions, ranging from error-proneness (how easily can programmers make er-rors) to viscosity (how easily can programmerschange an existing program). The framework wasdeveloped to create a set of heuristics that an ex-pert can use to critique a programming notation.In this way, they are related to the heuristics usedfor ordinary user-interface evaluation [9], but aretightly focused on usability issues related to pro-gramming notations.

Cognitive dimensions and Language Signa-tures examine different and complimentary as-pects of notation use in programming environ-ments. Whereas Cognitive Dimensions providesheuristics to examine the usability of a particularnotation, Language Signatures provide heuristicsto examine how multiple notations in an environ-ment notations interact with each other and withthe user. This difference in scope of the two frame-works means that both are useful when designingand building programming environments.

While the Cognitive Dimensions frameworkwas created with the goal of analysing program-ming notations, some of the dimensions can beextended to analyse the the relationships betweenprogramming notations.

Abstraction Gradient. The abstraction gradientdimensions analyses the minimum and max-imum levels of abstraction, and looks at theability for a programmer to abstract frag-ments of a program. An extension of this di-mension to multiple notations would examine

9

the relationships between the abstraction gra-dient of each notation, and look what happensto one notation when a programmer abstractspart of the program in the other notation.

For example, consider a programmer who isworking in a multiple notation programmingenvironment and wants to create a new ab-straction by refactoring some common codeinto a method. Conventionally, the abstrac-tion gradient dimension would examine howmuch work the programmer must do to per-form the refactoring. However, in a multiplenotation programming environments, a envi-ronment designer must also consider the ef-fects on the other notations of the refactoring.

Closeness of Mapping.This dimension exam-ines the mapping between the problem worldand the syntax and semantics of the pro-gramming notation. The extension for mul-tiple notations also examines the closenessof mapping between the multiple notation:how much cognitive effort a user must ex-pend when switching notations.

Error-proneness. This dimension examines howeasy it is to make an error, and more impor-tantly how easy it is to recover from an error.An extension of this dimension into a mul-tiple notation system would examine the ef-fects of making an error in one notation onthe other notation.

For example, consider a user who makes anerror in one notation of a multiple notationprogramming environment. An analysis ofthe error-proneness dimension in a multiplenotation environment should consider ques-tions including examining the effects of theerror on the other notations, and the recov-erability from the error using the other nota-tions.

Hard Mental Questions. Multiple notation pro-gramming environments can create many ad-ditional cognitive tasks that users must over-come to be able to use the environment.These were enumerated by Ainsworthetal and include: understanding the relation-ships between representations and the do-main, translating between representations,and, if designing representations, selectingand constructing an appropriate representa-tion [1]. A cognitive dimensions analysis

of multiple notation programming environ-ments should include an analysis of how hardthese tasks are for a user to perform.

Role Expressiveness.This dimension refers tothe ease in which programs can be read (asopposed to Hard Mental Questions or Close-ness of Mapping which refer to the easein which a program can be written). In amultiple notation programming environment,users of this dimension to evaluation nota-tion usability should examine how easy it isfor people to understand the relationship be-tween the notations as well as read the indi-vidual notations.

Secondary Notation and Escape From Formalism.A secondary notation refers to extra infor-mation that is not part of the actual program:commenting and indentation. Green and Pe-tre argue that support for secondary notationis important for programming notations. In amultiple notation programming environment,a programmer using this dimension to anal-yse the notations should consider the effectsof modifying a notation on the commentsand layout of the other notations.

Viscosity. Viscosity refers to a notations resis-tance to local change, or to the ease in whicha programmer can make small changes to aprogram. In a multiple notation program-ming environment, small changes in one no-tation could lead to large changes in anothernotation. This high inter-notation viscos-ity is especially likely if the two notationsuse very different representations of the pro-gram. For example, consider a multiple nota-tion programming environment with two rep-resentations of a program: a control flowrepresentation and a data flow representa-tion. A simple change in the data flow rep-resentation could equate to a large changein the control flow representation. Program-mers analysing multiple notation program-ming environments should consider the ef-fects of small changes in one notation on theother notation.

4 Summary

This paper introduced our view of how notationsare used in programming environments and per-formed a preliminary analysis of how cognitive

10

dimensions could be extended to analyse the rela-tionship between notations in a programming en-vironment. We believe that extending the analysisof cognitive dimensions would make interestingreading in the special issue journal on cognitivedimensions.

References

[1] S.E. Ainsworth and N Van Labeke. Usinga multi-representational design framework todevelop and evaluate a dynamic simulationenvironment. InDynamic Information andVisualisation Workshop, Tuebingen, July2002.

[2] A. Cockburn and A. Bryant. Leogo: Anequal opportunity user interface for pro-gramming. Journalof Visual Languages&Computing, 8(5–6):601–619, 1997.

[3] Matthew Conway, Steve Audia, Tommy Bur-nette, Jim Durbin, Rich Gossweiler, ShuichiKoga, Chris Long, Beth Mallory, Steve Mi-ale, Kristen Monkaitis, James Patten, JoeShochet, David Staak, Richard Stoakley,John Viega, Jeff White, George Williams,Dennis Cogrove, Kevin Christiansen, RobDeline, Jeff Pierce, Brian Stearns, ChrisSturgill, and Randy Pausch. Alice: LessonsLearned from Building a 3D System forNovices. InHumanFactorsin ComputingSystems:CHI 2000ConferenceProceedings(USA), pages 486–493, April 2000.

[4] Jay Fenton and Kent Beck. Playground:An Object Oriented Simulation System withAgent Rules for Children of All Ages. InProc. OOPSLA ’89, pages 123–137, NewOrleans, Louisiana, United States, 1989.

[5] Ephraim P. Glinert, editor. VisualProgramming Environments, ParadigmsandSystems. IEEE Computer Society PressTutorial, 1990.

[6] T.R.G. Green and M. Perte. Usability Analy-sis of Visual Programming Environments: A‘Cognitive Dimensions’ Framework.Journalof Visual Languages& Computing, 7:131–174, 1996.

[7] Brian W. Kernighan and Dennis M. Ritchie.The C Programming Language. PrenticeHall, 1978.

[8] Henry Lieberman. Watch what I do:Programmingby Demonstration, chapter 16:Monderain: A Teachable Graphical Editor.MIT Press, 1993.

[9] Jakob Nielsen.Usability Engineering. Mor-gan Kauffman, 1993.

[10] DA Norman. The Psychologyof EverydayThings. London: Basic Books, 1988.

[11] John F. Pane, Chotirat ”Ann” Ratanama-hatana, and Brad A. Myers. Studying thelanguage and structure in non-programmers’solutions to programming problems.International Journal of Human-ComputerStudies, 54:237–264, 2001.

[12] S Papert. Mindstorms — Children,Computers,and Powerful Ideas. HarvesterPress, Brighton, 1980.

[13] Cyndi Rader, Cathy Brand, and ClaytonLewis. Degrees of Comprehension: Chil-dren’s Understanding of a Visual Program-ming Environment. InHuman FactorsinComputingSystems: CHI ’97 ConferenceProceedings(USA), pages 351–358, March1997.

[14] S.P. Reiss. Pecan: Program developmentsystems that support multiple views. In[5], pages 324–333. IEEE Computer SocietyPress Tutorial, 1990.

[15] S.P. Reiss. Working in the garden envi-ronment for conceptual programming. In[5], pages 334–345. IEEE Computer SocietyPress Tutorial, 1990.

[16] Alexander Repenning. Agentsheets: an Interactive Simulation Environ-ment with End-User ProgrammableAgents. In Proceedingsof the IFIPConferenceonHumanComputerInteraction(INTERACT ’2000, Tokyo, Japan), 2000.

[17] David Canfield Smith, Allen Cypher, andJim Spohrer. KidSim: ProgrammingAgents without a Programming Language.Communicationsof the ACM, 37(7):54–67,July 1994.

[18] David Canfield Smith, Allen Cypher, andLarry Tesler. Novice Programming Comesof Age. Communicationsof the ACM,43(3):75–81, March 2000.

11

[19] Hideyuki Suzuki and Hiroshi Kato.Interaction-Level Support for Collabo-rative Learning: AlgoBlock — An OpenProgramming Language. In John L.Schnase, editor,Proc.ComputerSupportedCollaborativeLearning’95, pages 349–355,Bloomington, Indiana, October 1995.

[20] Tim Wright. Pattern programmer. Unpub-lished programming environment.

[21] Tim Wright. Collaborative andMultiple-Notation ProgrammingEnvironments for Children. PhD the-sis, University of Canterbury, 2005.

[22] Tim Wright and Andy Cockburn. Mul-spren: a MUltiple Language SimulationPRogramming ENvironment. InIEEESymposiaonHuman-CentricLanguagesandEnvironments, pages 101–103, Arlington,Virginia, September 2002.

[23] Peta Wyeth and Helen C. Purchase. Pro-gramming Without a Computer: A New In-terface For Children Under Eight. Proc.Australian Computer ScienceConference,22(5):141–148, 2000.

12