A End-User Programming System For CreatingAdventure Games

CHRISTOFFER EKEROTH

Master’s Thesis at CSCSupervisor: Anders Hedman

Examiner: Cristian M Bogdan

TRITA xxx yyyy-nnn

Abstract

In today’s always online, always connected world we consume moredigital media than ever before. As part of this development, social mediahas blurred the line between producer and consumer, giving anyone withan Internet connection the opportunity to reach out to millions of users.

While platforms such as Twitter, Facebook and Tumblr allow any-one to be the editor of their own news channel, some forms of expressionare still out of reach for the everyday user. One such medium is com-puter games, which have always required—and still do—a high levelof programming skill to create. As such, games is a channel that hastraditionally been out of reach for the casual content creator.

The goal of this thesis is to address this issue by making the cre-ation of text-based adventure game accessible to non-programmers, aspart of an initiative from Paradox Interactive. More specifically, Para-dox Interactive had the goal of empowering their writing staff to createadventure games without requiring the assistance of programmers.

Through interviews and workshops with writers and other stake-holders at Paradox Interactive a set of requirements were elicited, whichwere complemented by analyzing the works of Joe Dever, author of theinfluential Lone Wolf series of gamebooks. From these requirements aseries of low-fidelity prototypes were developed, which were tested anditerated upon in collaboration with users of the system.

After several iterations on the prototype a final design specificationwas developed, which was turned into a high-fidelity software prototype.Building upon the high-fidelity prototype, a working adventure gamecreation software was developed.

Ett programmeringssystem för slutanvändareämnat för äventyrsspelI dagens ständigt uppkopplade värld konsumerar vi digitala medier somaldrig tidigare. Som en del av den här utvecklingen har sociala me-dier löst upp gränsen mellan producent och konsument, vilket gjort detmöjligt för vem som helst med en Internetuppkoppling att nå ut tillmiljontals andra användare.

Medan plattformar som Twitter, Facebook och Tumblr har gjortdet möjligt för vem som helst att bli redaktör för sin egen nyhetskanalfinns det vissa uttrycksformer som fortfarande håller vanliga användareutestängda. Ett exempel är datorspel, vilka ställer stora krav på förkun-skaper inom programmering för att man ska kunna skapa dem. Pågrund av detta har skapandet av datorspel traditionellt sätt varit utomräckhåll för vardagsanvändaren.

Målet med detta exjobb är att, på initiativ av företaget ParadoxInteractive, råda bot på detta genom att göra skapandet av textbaseradeäventyrspel tillgängligt för icke-programmerare. Mer specifikt har man

på Paradox Interactive velat skapa ett verktyg som skulle låta derasförfattare skapa äventyrsspel utan att behöva hjälp från programmerare.

En kravspecifikation för verktyget togs fram genom intervjuer ochworkshops med författare och andra inblandade på Paradox Interactive,vilken kompletterades med en analys av Joe Devers Ensamma Vargen-böcker. Från kravspecifikationen togs sedan ett antal prototyper fram,vilka testades och utvecklades i samarbete med systemets användare.

Efter ett flertal iterationer av prototoypen togs en slutgilitg speci-fikation fram, vilken låg till grund för en digital prototyp. Utifrån dendigitala prototypen utvecklades sedan en slutgiltig version av mjuk-varan.

Contents

Ett programmeringssystem för slutanvändare ämnat för äventyrsspel . . .

1 Introduction 11.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.1.1 Gamebooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.1.2 Lone Wolf . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

1.2 Problem definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.3 Goals and Delimitations . . . . . . . . . . . . . . . . . . . . . . . . . 3

1.3.1 Goals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.3.2 Delimitations . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

1.4 Structure of the Thesis . . . . . . . . . . . . . . . . . . . . . . . . . . 4

2 Previous work 52.1 Reducing language complexity . . . . . . . . . . . . . . . . . . . . . 52.2 Visual programming languages . . . . . . . . . . . . . . . . . . . . . 62.3 Programming by demonstration . . . . . . . . . . . . . . . . . . . . . 62.4 Domain-specific languages . . . . . . . . . . . . . . . . . . . . . . . . 72.5 Designing usable programming languages . . . . . . . . . . . . . . . 82.6 Human-Computer Interaction . . . . . . . . . . . . . . . . . . . . . . 8

3 Method 93.1 A user-centered approach . . . . . . . . . . . . . . . . . . . . . . . . 93.2 Finding an appropriate paradigm . . . . . . . . . . . . . . . . . . . . 103.3 Requirements gathering . . . . . . . . . . . . . . . . . . . . . . . . . 10

3.3.1 Workshops and interviews . . . . . . . . . . . . . . . . . . . . 113.4 Prototyping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

3.4.1 Paper Prototype . . . . . . . . . . . . . . . . . . . . . . . . . 113.4.2 High-fidelity prototype . . . . . . . . . . . . . . . . . . . . . . 133.4.3 Version One . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

4 Results 154.1 Finding an appropriate paradigm . . . . . . . . . . . . . . . . . . . . 154.2 Requirements analysis . . . . . . . . . . . . . . . . . . . . . . . . . . 16

4.2.1 Entries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

4.2.2 Branches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164.2.3 Side effects . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164.2.4 Preconditions . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

4.3 Workshops and Interviews . . . . . . . . . . . . . . . . . . . . . . . . 174.4 Flowcharting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184.5 Initial Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194.6 Low-fidelity prototype . . . . . . . . . . . . . . . . . . . . . . . . . . 19

4.6.1 Expressing side effects . . . . . . . . . . . . . . . . . . . . . . 194.6.2 Rolling dice . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204.6.3 Negative numbers . . . . . . . . . . . . . . . . . . . . . . . . 214.6.4 Marsh Vipers and random numbers . . . . . . . . . . . . . . 214.6.5 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

4.7 High-fidelity prototype . . . . . . . . . . . . . . . . . . . . . . . . . . 234.8 Version One . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

5 Conclusions 255.1 An Adventure Game Creation Tool . . . . . . . . . . . . . . . . . . . 255.2 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

6 Discussion 296.1 Methods used . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

6.1.1 A user-centered approach . . . . . . . . . . . . . . . . . . . . 296.1.2 Low-fidelity Prototyping . . . . . . . . . . . . . . . . . . . . . 296.1.3 Significance of Results . . . . . . . . . . . . . . . . . . . . . . 30

6.2 Low floor, wide walls . . . . . . . . . . . . . . . . . . . . . . . . . . . 306.3 Visual programming . . . . . . . . . . . . . . . . . . . . . . . . . . . 316.4 Less is Less? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 326.5 Providing A Template . . . . . . . . . . . . . . . . . . . . . . . . . . 336.6 Domain-specificity . . . . . . . . . . . . . . . . . . . . . . . . . . . . 346.7 Recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

Bibliography 37

Chapter 1

Introduction

1.1 Background

1.1.1 Gamebooks

Gamebooks (also colloquially known as “Choose Your Own Adventure Books”, fromthe series of the same name, or “Solo Adventures”) are works of interactive fictionwhere the reader is invited to make choices that shape the narrative of the book [4].This is accomplished by prompting the reader to turn to different pages in the bookdepending on what path she chooses for the protagonist.

Gamebooks bear many similarities to early text-based computer games, such asColossal Cave Adventure, which let players navigate through a branching story [11,p. 186]. However, the concept of books containing branching story lines dates evenfurther back: Jorge Luis Borges’ Examen de la obra de Herbert Quain[1] featuresa fictional author whose work branches out into several paths, leading to a total ofnine different endings.

Behavioral psychologist B.F. Skinner also experimented with branching-pathbooks for automated learning; at each branch the student would be presented witha question and a choice of answers, each leading to a different place in the book. Acorrect answer would direct the reader to a page with more information about thesubject and further questions. Answering incorrectly would yield a page explainingwhy the answer was wrong, and an instruction to turn back to the original page [4].

To better illustrate how a gamebook reads, here is an excerpt from the gamebookFlight from the Dark by Joe Dever [5]:

The path is wide and leads straight into thick undergrowth. The treesare tall here and unusually quiet. You walk for over a mile when suddenlyyou hear the beating of large wings directly above you. Looking up, youare shocked to see the sinister black outline of a Kraan diving to attackyou.

If you draw your weapon and prepare to fight, turn to 229.

1

CHAPTER 1. INTRODUCTION

If you evade the attack by running south, deeper into the forest, turn to99.

If the reader thinks she can dispatch the beast, she will turn to section 229 ofthe book and see how the story unfolds from there. On the other hand, the tall treeswould provide protection from the flying Kraan, so turning to section 99 might bethe safer option. . . if she can make it in time, that is.

The ambition of the stakeholders at Paradox Interactive AB, where the thesiswork was carried out, is to bring the gamebook experience into the digital realmby creating text-based adventure games for smartphones and tablet devices, suchas the Apple iPhone, the Apple iPad or devices powered by the Android operatingsystem.

Paradox Interactive employs a staff of several writers and game designers whomthey would like to task with creating these adventure games.

1.1.2 Lone Wolf

Lone Wolf is a series of fantasy gamebooks written by Joe Dever, in which theplayer takes on the role of a knight named Lone Wolf, who has sworn to defend theland of Sommerlund against the evil Darklords.

The Lone Wolf series has been an important source of inspiration for the stake-holders at Paradox Interactive, and has served as a benchmark for what theywish to accomplish with their adventure games. As such, the book Flight fromthe Dark [5]—the first in the Lone Wolf series of gamebooks—will be referencedthroughout the thesis.

If the reader wishes to find out more about Lone Wolf, a community effort isunderway to make the Lone Wolf series of books freely available online, sanctionedby Joe Dever. They can be found at http://www.projectaon.org.

1.2 Problem definition

The goal of Paradox Interactive is to allow for their writers, who may have little orno programming experience, to create adventure games. However, writing softwarefrom scratch for iPhone or Android devices is a very complex and time-consumingtask, even for people who are programmers by trade.

Is it possible then to simplify the task of creating software? David Canfield Smithnotes in KidSim: Programming Agents Without a Programming Language [29]:

This problem—the “end-user programming problem”—is an unsolvedone in computer science. In spite of many previous attempts to developlanguages for end users, today only a small percentage of people are ableto program.

2

1.3. GOALS AND DELIMITATIONS

As such, there is as of yet no one formula or approach that allows for end-usersto easily create software. However, there have been a number of successful attemptsto address smaller parts of the end-user programming problem:

Even though one may be hesitant to call Microsoft Excel a programming lan-guage, nonetheless it endows the end-user programmer with many powerful pro-gramming facilities. It also manages to do so in such a way that the user never hasto reflect over the fact that she is actually programming.

In A User-Centered Approach to Functions in Excel [9] Simon Peyton Jonesnotes:

It may seem odd to describe a spreadsheet as a programming lan-guage Indeed, one of the great merits of spreadsheets is that users neednot think of themselves as doing “programming”, let alone functionalprogramming—-rather, they simply “write formulae” or “build a model”.

1.3 Goals and Delimitations

1.3.1 GoalsFollowing requirements established together with stakeholders at Paradox Interac-tive, the following goals were set up:

• To design and develop a programming environment that supports the creationof text-based adventure games.

• Users of the programming environment should not need previous programmingexperience or experience with any other programming environment.

• It should be possible to recreate the adventure game book Flight from theDark (referenced in section 1.1.2) in its entirety using only the programmingenvironment.

• It should be possible to export finished adventures to a file. It should bepossible to play the adventure files in a “game player” client in the form of adesktop or mobile application.

• The programming environment should be flexible enough to allow the creationof adventure games set in a diverse variety of worlds and time periods, andshould not be limited to one specific setting (i.e. “Tolkienesque fantasy” or“Cyberpunk”).

1.3.2 DelimitationsTo contain the scope of the project, some delimitations were also selected:

• The programming environment should be domain-specific and need not sup-port general programming tasks.

3

CHAPTER 1. INTRODUCTION

• Certain elements of the adventure game experience, such as combat and in-ventory management, would not need to be handled explicitly in the editorbut could be seen as “black boxes” handled by the adventure game client.

1.4 Structure of the ThesisThis thesis consists of five primary sections, organized by topic:

• The Previous Work section details previous research in the field of usable pro-gramming languages and environments and related areas of usability research.

• The Method section contains descriptions of the methods used in the thesisand how the thesis research was carried out at Paradox Interactive.

• The Results section presents the results obtained during the thesis work, mostof them sourced from interviews and usability sessions.

• The Conclusions section presents a design specification for an adventure gamecreation tool based on the results in the Results section.

• The Discussion section ends the thesis with some concluding discussions ofthe results obtained and how they relate to previous research in the field.

While the thesis work also included the implementation and testing of the ad-venture game creation software, I chose to focus on results obtained during userresearch.

4

Chapter 2

Previous work

A lot of research effort has gone into making software creation tools more usable,for novices and experienced programmers alike. Kelleher, et al. have compileda taxonomy of existing programming tools designed to make programming moreaccessible, and the research behind them [10]. In the following section I will presentsome common approaches increasing the usability of programming languages.

2.1 Reducing language complexity

One of the early approaches to making programming easier was to make the lan-guages less complex. One of the first known realizations of this idea was the LOGOlanguage, one of the first attempts at bringing programming to a wider audience. In-vented by Daniel G. Bobrow, Wally Feurzeig, Seymour Papert and Cynthia Solomonin 1967, it emphasized simplicity and was intended for constructivist teaching ofprogramming. [23]

One of LOGOs central ideas was “turtle graphics”, the ability for the user todraw lines programmatically by manipulating an on-screen cursor (the “turtle”).This allowed users to experiment their way towards correct implementations ofgraphics drawing algorithms.

Below is some example LOGO code that makes the “turtle” draw a spiral, usingrecursion:

to spiral :sizeif :size > 30 [stop] ; an exit conditionfd :size rt 15 ; many lines of actionspiral :size *1.02 ; the tailend recursive call

end

The approach used in LOGO to reduce complexity was to keep the syntax simpleand to minimize the number of language primitives available to the programmer.Another method of simplifying programming had been attempted a few years earlier

5

CHAPTER 2. PREVIOUS WORK

when J.G. Kemeny and T. Kurtz developed the programming language BASIC [10,p. 6].

BASIC (Beginners All-purpose Symbolic Instruction Code) was developed atDartmouth College, as a less intimidating alternative to FORTRAN and ALGOL. Incontrast to the terser syntax used by LOGO, BASIC employed a more conversationalstyle of programming that was designed to be reminiscent of natural language [10,p. 7].

Below is a simple BASIC program that sums the numbers 1 to 10:

100 FOR I = 1 TO 10110 LET S = S + I120 NEXT I

2.2 Visual programming languagesEven if programming languages can be simplified, several studies suggest that theremay be usability problems intrinsic to textual programming that makes it difficultto comprehend and learn [18] [26] [12].

For example, even the simplest programming language syntax requires strictadherence, and so the user has to remember and understand it before she cancreate or manipulate programs.

An alternative approach to programming is to let the user manipulate graphicalobjects instead of text. These objects can then be made to only combine in mean-ingful ways, allowing the environment to help the user construct syntactically andsemantically correct programs. This eliminates a large body of user errors that canotherwise become major stumbling blocks for beginners [8].

One of the more recent visual programming languages that follows this paradigmis Scratch, developed by Resnick et al. [16] at the MIT Media Lab. Scratch rep-resents its language primitives as graphical blocks that can be “snapped” togetherlike puzzle pieces. Just like a puzzle, the Scratch blocks have connectors of differentshapes, which indicate which blocks fit together.

This paradigm has proven hugely successful; since its launch in 2007, users haveuploaded thousands of projects to the Scratch web site (http://scratch.mit.edu),with the core audience consisting of children between 8 and 16 years old. [24] Theprevalence of teens and pre-teens in the Scratch community attest to the success ofthe MIT team’s effort to create an accessible programming language.

The Scratch programming language has been a great inspiration throughoutthis thesis, both in the methodology used to develop the language and the findingspresented by Resnick et al.

2.3 Programming by demonstrationAnother way of making programming more accessible is to completely revise theapproach to writing programs. In the classical programming paradigm, the user

6

2.4. DOMAIN-SPECIFIC LANGUAGES

feeds the computer a program text, which gets translated to instructions that thecomputer can execute. In other words, the programmer has an idea for how shewishes the program to behave, then attempts to devise the instructions necessaryto achieve the desired behavior.

Programming by demonstration turns this approach on its head; given someprecondition, the user demonstrates the desired behavior to the program, and theprogram tries to infer the appropriate instructions to accomplish it. KidSim, de-veloped by Allen Cypher and David Canfield Smith, is an example of such an envi-ronment. [29] KidSim uses graphical rewrite rules, which lets the user demonstrateprogram behavior by specifying a “before” condition and a resulting “after” stateusing the graphical objects in the environment.

For example, the user might instruct a train to move along a track by puttinga train and some railroad in the “before” box, then moving the train further alongthe track in the “after” box. The program then tries to infer a general rule fromthe user’s demonstration, which will (hopefully) produce the desired result, even insituations slightly different from the one demonstrated.

2.4 Domain-specific languages

Studies have shown that the usability of a programming environment can further beimproved by making it domain-specific [7]. This means using language constructsthat are closer to the problem domain may make a programming language easier touse compared to general-purpose programming languages.

An example of such a language is SQL (initially known as SEQUEL) a domain-specific language for working with relational databases[2]. Its developers, DonaldChamberlin and Raymond Boyce, describes it as “a data base sublanguage for boththe professional programmer and the more infrequent data base user”. As such, it isstrictly concerned with operations on databases, which makes SQL easier to learn:

The simple block-structured English keyword syntax and simple op-erations on tables enable users to interact with the SEQUEL systemwith less training and sophistication than would be required in either acalculus-oriented system or a traditional procedural programming lan-guage.

Below is an example SQL query that retrieves the name and salary for everyemployee in the “TOY” department who is managed by “ANDERSON”:

SELECT NAME, SALFROM EMP

WHERE DEPT = ’TOY’ AND MGR = ’ANDERSON’

7

CHAPTER 2. PREVIOUS WORK

2.5 Designing usable programming languagesAs a result of the work that has gone into designing more usable programminglanguages and environments there has also evolved a body of work around bestpractices and frameworks for usable programming language design.

One paper that greatly influenced my work on this thesis was Some Reflectionson Designing Construction Kits for Kids [25], in which Mitchel Resnick and BrianSilverman reflect on their 20 years of experience designing technologies for kids. Ofparticular note was the concept of “Low floor and wide walls”, an analogy whichdescribes the property of being easy to learn (“low floor”) while at the same timeallowing for a wide range of different solutions to be built (“wide walls”).

Several compilations of best practices in programming language design havebeen published, such as Usability Issues in the Design of Novice Programming Sys-tems [21] which compiles a large body of research and organizes it according toJakob Nielsens heuristic usability principles. [19]

Another entry on best practices, grounded in personal expertise, is Seven DeadlySins of Introductory Programming Language Design by Damian Conway and LindaMcIver, wherein attributes of programming languages are evaluated for their rolesin aiding or hindering learnability [17].

The learnings presented in these “best practices” papers have been a great helpfor approaching the language design problem in an informed way.

2.6 Human-Computer InteractionThough creating a programming language environment has its unique set of chal-lenges it is still software intended to be used by people. As such, general principlesof human-computer interaction and software usability are still applicable.

In particular, Interaction Design: Beyond Human Computer Interaction hasbeen very useful to me in its comprehensible presentation of prototyping at variousstages in software development [27, p. 529]. The workflow used in the thesis work—requirements gathering, design, prototyping, evaluation—also follow the suggestedinteraction design process laid out by Preece et al. [27, p. 413]

8

Chapter 3

Method

3.1 A user-centered approachThe thesis work was carried out using a user-centered approach as described byPreece et al. in Interaction Design [27, p. 425]. The main tenets of this approachare:

1. Early focus on users and tasks

2. Empirical measurement

3. Iterative design

The details of these principles and how they were implemented in the thesiswork are described below.

Early focus on users and tasks

As the name suggests, a user-centered approach involves frequent interaction withthe users of the product. As such, I had been allowed access to both junior andsenior designers and writers at Paradox Interactive. I had also been assigned adesigner intern who would use the software in parallel with it being developed.

An early focus on users and tasks meant querying the designers who would usethe system about their current game creation process. How do they currently goabout creating adventure games? Do they already use computers in their workflow,or is their toolset completely analog? What are the biggest cognitive challenges intheir work and how do they cope with these?

In the user-centered approach the users are also actively involved in the designprocess. This allows for drawing upon their experience in the adventure game fieldwhen coming up with solutions to design problems.

The primary means of involving the users in the early stages of the project were aseries of interviews and workshop sessions (see section 3.3.1). To better understandthe domain of adventure games and gamebooks requirements were also gathered by

9

CHAPTER 3. METHOD

analyzing game books and conducting interviews. The results of these inquiries arepresented in section 4.3.

Empirical measurement

In the user-centered design philosophy, empirical measurement means measuringreactions and performance of intended users as they interact with various designartifacts. This also means setting up concrete goals for the user experience.

Together with the stakeholders at Paradox Interactive, a set of usability goalswere set up (see section 1.3.1) which design alternatives were continuously measuredagainst.

User reactions to designs were also recorded through usability testing at variousstages of development (see below).

Iterative design

Iterative design means to continuously test, evaluate and iterate the design duringthe development process. In the thesis work this was accomplished by developingprototypes at various levels of fidelity.

Usability tests (see above) were conducted with the prototypes with the followingartifacts:

1. Paper prototype (see section 3.4.1)

2. High-fidelity prototype (see section 3.4.2)

3. Working versions of the tool (see section 4.8)

The feedback from the test sessions was then integrated into subsequent itera-tions of each artifact.

3.2 Finding an appropriate paradigmAfter reviewing existing work (see section 2) on programming environments fornovices I reviewed some of the most common approaches and evaluated how wellthey would fit the task.

Having reviewed different approaches I concluded that a visual programmingenvironment similar to Scratch would be the most suitable. Since this finding in-fluenced the rest of my work on this project it is presented here briefly; for a moreelaborate review of the results obtained, see section 4.1.

3.3 Requirements gatheringBefore the design work began in earnest I gathered requirements to find out whattasks needed to be supported by an adventure game creation tool. The requirements

10

3.4. PROTOTYPING

gathering process followed the guidelines described by Preece et al. in InteractionDesign [27, p. 473].

Two methods were used in gathering requirements:

1. Analysis of the mechanics employed in the gamebook Flight from the Dark.

2. Interviews and workshop sessions with the designers involved in the project.

Another important source for both requirements and approaches to adventuregame creation was Joe Dever’s how-to guide Writing Solo Adventures [6], whichdocuments the process he used when writing the Lone Wolf series of gamebooks.

3.3.1 Workshops and interviews

Several interviews and workshop sessions were held with designers and other stake-holders at Paradox Interactive. The primary focus of the interviews were to findout how designers currently create adventure games. Areas of inquiry included:

• What activities are performed when designing adventure games.

• Tools designers use to support these activities.

• Problems with designers’ current tool sets and workflows.

• Common cognitive challenges encountered when designing adventure games.

Though none of the designers at Paradox had designed game books before, theydid have experience designing role-playing game systems and adventures, such asthe Coriolis series of role-playing games [15].

3.4 PrototypingThere are several problems in software design which have known, workable solutions.Photo manipulation, word processing and spreadsheet applications have existed fora long time and as such there are several common patterns and best practices thata designer can use.

Adventure game creation tools, on the other hand, are far less common and assuch it is less of a given how the user will interact with the tool. For this reason, Istarted out by developing a series of prototypes of increasing fidelity, which allowedfor quick exploration of design ideas [27, pp. 531-535].

3.4.1 Paper Prototype

For investigating how users might perform a set of tasks, Preece et al. recommendpaper prototyping [27, pp. 531-535]. A paper prototype is quick to develop, and is

11

CHAPTER 3. METHOD

easy to throw away if it doesn’t work. Also, because of its low fidelity, it keeps bothdesigners and subjects focused on the interaction part of the design.

The first version of the paper prototype was based on ideas and suggestionsgenerated in the workshops and Joe Dever’s Writing Solo Adventures [6] article. Iused an A3 sheet of paper to represent the user’s workspace, and Post-it notes torepresent the interactive elements of the interface. (See figure 3.1)

The prototype was tested in several session with subjects recruited both frominside and outside Paradox Interactive. Some of the subjects had previously partici-pated in workshops and interviews as role-playing design experts (see section 3.3.1),others had no prior experience with the project or adventure game design.

Each test session was modeled after the guidelines for usability tests in SteveKrug’s book Don’t Make Me think [13, pp. 130-159]:

1. Prior to starting the session, a video camera is set up to record the participantinteracting with the system. This allows the designer to review the materiallater and share the results with other stakeholders.

2. The session starts. The system under test is explained to the participant, andwell as why I need their help. I also explain that the session will be recorded,and ask for permission to show the recorded footage to other stakeholdersin the project. Krug recommends stressing that the objective is to test theusability of the system, and not the performance of the participant.

3. The participant is given a series of tasks to complete. In this case, the list oftasks was a set of scenarios compiled from the Flight From the Dark gamebook.The scenarios were chosen as to test as large a number of different interactionsas possible.

• Give the player two paths to choose from, one of which is only availableif the player has the Vordak Gem item.

• Give the player two paths to choose from, one of which is only availableif the player has ten or more gold pieces.

• Continuing from the previous assignment, subtract ten gold from theplayer’s inventory if she chose the path that requires gold.

• Reduce the player’s combat ability by two points if she does not have atorch in her inventory, then trigger combat with a Burrow Crawler.

• Have the player combat a Marsh Viper. If the player has fewer hit pointsafter fighting the viper than she had before combat, end the game.

• Give the player the choice to take a safe or a riskier path. The conse-quence of choosing the risky path should depend on the outcome of adice roll.

The subject is encouraged to think aloud as she completes each task. The de-signer should not prompt the subject, but simply tries to observe the subject,occasionally asking question about the subject’s thought process.

12

3.4. PROTOTYPING

4. After the subject has completed all the tasks the designer may ask some follow-up questions to further clarify the subject’s thought processes. After this, thedesigner thanks the subject for their time and takes note of any interestingfindings from the session.

Following Steve Krug’s advice, I tried to test each version in a small number ofsessions as opposed to testing one version many times over. The idea underlyingthis approach is that every test session yields diminishing returns in the number ofusability errors discovered; testing with five users is usually enough to find mostproblems, and testing with three users generally yields the best return on timeinvested [20].

Due to time constraints I could not always test each iteration of the prototypewith three users, but sometimes had to make do with two tests before creating thenext version. In some cases I also made exceptions in the case of glaring usabilityerrors, which I usually addressed after just one test. In total, four iterations of thepaper prototype were created.

3.4.2 High-fidelity prototypeThe next step in the design process was to create a high-fidelity prototype. Ahigh-fidelity prototype mimics the look and feel of the finished product, but severalessential features may be missing. The purpose of the high fidelity prototype wasto assert that the interactions designed for the paper prototype could be translatedto a computer environment.

I chose to develop the prototype using HTML and JavaScript, so that the pro-totype could be used in a web browser. The reasons for choosing HTML was thatit allowed for high portability (all popular operating systems ship with JavaScript-enabled web browsers) and high development speed. A screenshot of the prototypecan be seen in figure 3.2.

In order to quickly recreate the functionality of the paper prototype the open-source WireIt library was used. This allowed creating and connecting boxes to eachother in a way that closely resembled how users interacted with the paper prototype.

As with the paper prototype, the high-fidelity prototype was tested and iteratedupon several times. However, due to time constraints I only had time to conducttwo usability tests.

3.4.3 Version OneUsing the previous findings, I created an initial working version of the tool. Inparallel with developing the tool, I helped an intern at Paradox Interactive use thetool to create a “proof-of-concept” adventure game.

The feedback gathered from using the tool was continuously fed back into thetool’s development.

13

CHAPTER 3. METHOD

Figure 3.1. A work-in-progress prototype

Figure 3.2. The high-fidelity prototype

14

Chapter 4

Results

4.1 Finding an appropriate paradigmSeveral paradigms from previous work on novice programming environments wereevaluated in the initial phases of the project (see section 2). An early hypothesiswas that a domain-specific textual programming language would be a good fit forcreating adventure games. (See section 2.4)

Inspired by the work of Pane et. al [22] I asked users to describe events in LoneWolf in a formal manner, i.e. in a way that a computer might understand. Theidea was that the proposed solutions would then be used to inform the design of adomain-specific programming language, making it as natural to use as possible.

However, several difficulties were encountered early in the design process:

• Even though the syntax was designed to resemble real-world language, pro-grams were still required to conform to a strict syntax. This required the userto learn and memorize the syntax and semantics of the language in order touse it.

• When asked to give formal descriptions of scenarios in a game book, partici-pants gave very different answers. This indicated that while the semantics ofthe language could be made to sound natural to some users, this effect wouldnot be universal.

Due to these discouraging early results, I decided to abandon the textual pro-gramming idea and consider other paradigms.

Though programming by demonstration is an interesting concept I deemed itill-suited for working with adventure games because of the combinatorial complex-ity it would incur: Every game rule would have to be demonstrated by the user,and additional examples might be required for rules that interact in a non-obviousmanners. (See Making Programming Easier for Children by Cypher, Canfield andSmith [28][p. 8])

The most promising approach seemed to be designing a tool similar to Scratch,modified to better suit the adventure game domain. First and foremost, the building-

15

CHAPTER 4. RESULTS

block approach employed by Scratch would offer a good balance between expressivepower and ease of use. As put by the designers of Scratch:

The Scratch programming environment and language work together tocreate a system that is exceptionally quick to learn—users can be pro-gramming within fifteen minutes—yet with enough depth and variety tokeep users engaged for years.

[16]This would provide the “low floor and wide walls” necessary for making it easy

to get started while at the same time allowing for a wide variety of adventure gamesto be built, as stated in the requirements (see 1.3.1).

Taking these factors into account, I decided to use the visual programmingparadigm for the tool.

4.2 Requirements analysisAs mentioned in section 3.3, I had gathered requirements in order to inform thedesign of the tool. The first part of the requirements gathering process was toanalyze the gamebook Flight from the Dark to elucidate the rules used to drivethe game forward. Four basic building blocks were identified—entries, side effects,branches and preconditions—which will be presented below:

4.2.1 EntriesThe basic building block of a game book is the page, or “entry” as Joe Dever callsit [6]. Each page contains a textual description which advances the story, andusually concludes by presenting the player with a choice of what to do next.

4.2.2 BranchesThe alternatives presented to the player—which I will henceforth refer to as branches—consist of a short description of the choice and an instruction to turn to a specificpage. To select which branch to follow, the player simply turns to the appropriatepage and continues reading from there.

4.2.3 Side effectsAn important element of Flight from the Dark is that the choices the player makesmay have consequences that resonate through the entire adventure; as the story un-folds, the protagonist may collect items, learn new skills, make friends and enemies,and so on. In order to remember what has happened to the character, the playerrecords this information on a character sheet which is included in the gamebook.

The effects recorded on the character sheet are important since they affect thecourse of events in the gamebook. For example, if the character triggers a trap she

16

4.3. WORKSHOPS AND INTERVIEWS

may lose health points. If the number of health points is reduced to zero, the gameis over. To prevent the protagonist from dying, the player may consume a healthpotion that she has found earlier during the adventure. She would then remove onehealth potion from her inventory and restore the appropriate amount of health onher character sheet.

I will hereafter refer to instructions that affect the state of the game world asside effects.

4.2.4 Preconditions

The information on the character sheet can also influence the choices available tothe player. For example, if the player helps out a member of the mages guild shemight receive a special amulet. Later in the game, she may need to gain the favor ofanother mage; if the player has the amulet, she can choose to show it to the mage.

The way gamebooks implement this mechanic is to prefix certain branches witha precondition, e.g. “If you possess the Amulet of Xendor, you may present it tothe mage. Should you choose to do so, turn to page 123.”

4.3 Workshops and InterviewsSome of the most valuable findings in the thesis work were discovered during infor-mal brainstorming workshops.

One of my initial hypotheses were that letting the users create their own ab-stractions, just as in regular programming languages, would be a very useful feature.I presented the idea in an early brainstorming session, but the participants seemedconfused as to why this was necessary1.

Asking the group why they thought this way, one of the senior designers de-scribed the process he used to outline his adventures: He would first build a flowchartof the adventure, using Post-It notes to represent each node in the flowchart.

As he explained his process he drew a tree structures of boxes on the conferencewhiteboard (see figure 4.1). The boxes, which would be Post-It notes in his ordinaryworkflow, could be one of four different types:

• Story - a segment of exposition in which the player learns what is happening.

• Dilemma - a juncture where the player is asked to make a choice that willdirect the course of the adventure.

• Combat - an adversary which the player must defeat to proceed, using thegame’s combat rules to determine if the player succeeds.

• Puzzle - an obstacle which the player must use her wits to overcome.1Presenting my idea to the user group was a mistake—it’s usually a bad idea for the designer

to try to impose her ideas on users.

17

CHAPTER 4. RESULTS

He explained that these four basic building blocks were usually all he neededto construct adventures, which is why he didn’t see the need for creating furtherabstractions.

This finding, together with other insights detailed in section 4.4, greatly informedthe design of the adventure game creation tool.

In summary, letting the user create her own abstractions, encapsulating recur-ring or complex patterns, was not seen as an important (or even desirable) feature.Being an essential activity when writing general computer programs, I had takenfor granted that this would be a must-have for the adventure creation tool.

Similar results were found by Resnick et al. during the development of Scratch:

Early versions of Scratch had a mechanism for creating procedures. Inearly field tests, however, many users were confused by procedures sincethey seemed very similar to broadcasts—both involved associating aname with a collection of commands. [Note: A broadcast in Scratch isa set of commands to be executed when an actor receives a message.]In the interest of simplicity and minimalism, procedures were removedfrom the language before Scratch was officially released, and Scratch hasgotten along surprisingly well without them.

4.4 FlowchartingWhile researching tools of gamebook authors I found that Joe Dever, much like thedesigners at Paradox Interactive, had used flowcharts to outline his adventures. InWriting Solo Adventures[6] Dever writes:

Flowcharting is the guts of a solo gamebook. It enables you to visualizeand summarize exactly what will happen in each entry. It also helpsyou to balance the game-play (scoring, items found etc.) and make surethat technically the entries link together correctly.

In both Dever’s and the Paradox designers’ methods, each node in the flowchartrepresents one entry (see 4.2.1) with one or more branches (see 4.2.2) connecting itto subsequent entries. The designer might also keep notes at each branching pointabout any preconditions (see 4.2.4) that must be met for the player to be able toprogress along the given branch[6].

Using flowcharts in this manner helps the designer construct an overview ofthe adventure. Once this overview is complete, the designer then fleshes out theadventure with more detailed information about what happens to the player at eachentry.

Preece et al recommend using metaphors that users are familiar with in order tohelp them learn use a new system [27][p.543]. Since the designers at Paradox andJoe Dever had both chosen to use flowcharts when designing adventures, it seemedlike a flowchart would be a good interface metaphor to use for an adventure gamecreation tool.

18

4.5. INITIAL DESIGN

4.5 Initial DesignFrom my requirements gathering (see sections 4.2 and 4.4) I had identified four basicelements that could be used to construct adventure games: Entries, branches, sideeffects and preconditions. These were then translated into corresponding interfaceelements in the prototype:

• Entries (see 4.2.1) were represented by a post-it note on which the user couldwrite down a short summary of how the entry would advance the story.

• Branches (see 4.2.2) branches were represented by lines drawn in pen, con-necting entries with each other.

• Side effects (see 4.2.3) were represented as post-it notes attached to an entry.More than one side effect could be attached to the same entry. Although itwas never stated explicitly, the prototype used variables to store side effects.

• Preconditions (see 4.2.4) were represented as post-it notes attached to a branch.If several preconditions were attached to a branch it meant all of the precon-ditions had to be true for the branch to be available.

While entries and branches were very free-form in how they were implementedin the prototype, side effects and preconditions were more formalized. Based on theresults from the requirements analysis (see section 4.2) I created several templatePost-it notes for expressing side effects and preconditions where the user could fillin their own values.

I will not include the full initial list of operations here, but I will include someexamples for the purpose of illustration.

Some sample operations for use in preconditions:

• Comparisons (lesser than, equal to, etc.) which could be used to compare twovalues to each other.

• Player has ..., which tested if a player possessed a certain item or skill.

Some sample operations for use in side-effects:

• Set ...to ..., used to set a variable to a specific value.

• Roll dice, which would produce a number between one and six.

4.6 Low-fidelity prototype

4.6.1 Expressing side effectsThe first version of the prototype contained several template arithmetic operations,such as Add ...to ..., which could be supplied as input to the Set ...to ...

19

CHAPTER 4. RESULTS

operation. The intent was to give the users several options for how the value of avariable should be expressed: As a constant, as a reference to a different variableor as a function of one or more variables.

For example, to increase the player’s gold by ten the user was first supposedto construct the expression Add 10 to Gold and then use that as the right-handvalue of the Set ...to ... operation, resulting in the compound expression SetGold to (Add 10 to Gold). (By itself, Add 10 to Gold would not change anyvariables since it only produced a new value.)

However, participants in the initial test group found this highly unintuitive. Tothem, an expression like Add 10 to Gold looked like it would change the value ofthe Gold variable. When asked what operations he would use to increase the Goldvariable by ten one participant responded: “Intuitively, I would want to use Add...to .... But I guess it could just be an issue about how it’s phrased.”

When the intended use of Set ...to ... was explained to participants, theydid neither see the use for this construct, nor how it was different from just usingthe addition operation: “Is the idea that I should be able to put different thingsinto variables? What other things would you enter using number plus number?”

In the second iteration of the prototype, the generic arithmetic operations werechanged to better conform to users’ expectations. For example, the Add ...to... operation would now let the user increment a variable simply by specifying avariable name and a value. New participants who used this version of the prototypeconsistently chose the correct operations without any guidance.

4.6.2 Rolling dice

In the Lone Wolf gamebooks, the protagonist’s fate is sometimes left at the mercy offortune; in these situations the player is required to roll a die, and turn to differentpages depending on the result of the roll.

For these types of scenarios the prototype included a special Roll dice oper-ation, which produced a random number between one and six. The user was thenexpected to use this operation together with the Set variable to ... operationto save the result of the die roll, and then use this variable in a precondition (seesection 4.2.4) to decide what branches would be available to the player.

However, this design introduced many usability problems; like variable assign-ment, it had to be explained in every test session. Once again, reducing the gener-ality of the operation proved to be an effective solution: Instead of having the userstore the result of the die roll, a special variable—Dice result—was introduced.Whenever the Roll dice operation was used, the result was automatically storedin Dice result, making it instantly available to use in preconditions for branches.

As was the case with the assignment operations, specializing the Roll diceoperation yielded positive results; with the addition of the Dice result variableall of the participants used the Roll dice operation correctly without needing anyfurther instruction.

20

4.6. LOW-FIDELITY PROTOTYPE

4.6.3 Negative numbersHaving separate operations for addition and subtraction was another redundancy in-troduced to increase usability; technically, the same effect could have been attainedby providing either positive or negative numbers to the Add ...to ... operation.However, an earlier result indicated that this redundancy would probably be a goodthing:

In my early low-fidelity prototypes there was a Boost combat value by ...operation, which would increase the player’s combat proficiency with a certainamount.

One of the tasks in the prototyping sessions required the participants to reducethe combat value of the player. The “correct” way to do this was to provide anegative number to Boost combat value by ..., which would result in a decreaseof the player’s combat value. While this was consistent with the rules of arithmetic,the design of the operation did not suggest that it could be used in this manner.“It’s not intuitive that Boost combat value can also be used to decrease it. I wouldchange the name or add another operation for reducing the combat value” was onefeedback point I received.

While the case of Boost combat value could simply have been a result of un-fortunate naming, it would probably also qualify as an instance of what Conway etal. refer to as “Excessive cleverness” [17]:

Instances of “excessive” cleverness can be difficult to spot, precisely be-cause the “excess” exists only relative to the knowledge level of thenovice. Frequently the only way to detect excessive cleverness is to seea novice programmer’s complete misunderstanding of an “obvious” con-cept.

4.6.4 Marsh Vipers and random numbersThe most difficult assignments to solve proved to be the combat with the MarshViper (see section 3.4.1, 3) and having the outcome of a player’s choice be dependenton chance (see section 3.4.1, 3).

The difficulty of the Marsh Viper assignment arose from the user having torecord the player’s hit points before the combat by saving it to another variableand then comparing it to the player’s hit points after combat.

All participants had some difficulty solving the Mash Viper problem. Even ifthey realized that they should use variables in some way, it was not obvious how.When asked to think aloud, one participant said: “I suppose you could check ifyour hit points have decreased at all. But then you have to establish how manyhit points you have before combat has started. . . you would have to have somecalculation before combat that saves the value. . . ”

However, once they realized that they could store the player’s hit points beforecombat in a separate value, all participants managed to solve the assignment. Oneparticipant who had more experience with video game design remarked: “It takes

21

CHAPTER 4. RESULTS

a bit of thinking, but if you have previous experience with [game creation tools] Ithink you’ll get it. I’ve used Multimedia Fusion so I know how to use variables andso on.”

Having the outcome of a choice depend on chance presented other difficultieswhich I was never able to solve in a satisfactory manner. Random outcomes wereconstructed in the following way:

1. Create a Roll dice operation at the current entry.

2. From the current entry, create several branches with identical descriptions.

3. Make each of the branches lead to a different outcome, i.e. a different entry.

4. Attach preconditions depending on the dice result to each branch so that onlyone of them are ever shown given a certain dice result.

This means that the outcome of the choice has already been decided when theplayer reaches the entry; what the player sees is simply a branch that has alreadybeen singled out from several alternatives. However, since the descriptions of all ofthe branches are identical, there is no way for the player to tell the difference.

While this design has no impact on how the adventure is played, it was a grossviolation of the users’ mental models. In the test sessions it was made clear that theusers had the expectation that the dice roll should dictate where the player endedup as she made the choice, not before.

Having two branches which were identical in every fashion except that they ledto different sections was also unintuitive. One participant commented: “I don’tknow if I’ve missed some sort of presentation function. It is not obvious to me, asa designer, whether these two [branches] are visible or not.”

Introducing a special operation to cover the random outcome of choices couldhave been a solution to this problem, but I did not have time to try it out.

4.6.5 Evaluation

Apart from the difficulties mentioned above, the participants did not need anyadditional instruction to complete their assignments. Even though the participantsdid not participate in a complete reconstruction of Flight from the Dark, I believewith some certainty that they would be able to if the test sessions had continued.

Reviewing Flight from the Dark I managed to reconstruct all scenarios usingthe operations in the final version of the prototype, given that I would not need toreconstruct the combat rules of the game.

One exception, however, was the combat with the Marsh Viper: In the originaladventure the description says the player is to die if she is damaged by the viperduring combat, while in my version the player dies if she has less health points aftercombating the viper. These are not exactly equivalent since the combat rules allowsthe player to restore health points during combat, allowing her to take damage and

22

4.7. HIGH-FIDELITY PROTOTYPE

still have the same amount of health points (or more) as before. However, I judgedthat this was within the limits of what could be considered a faithful recreation ofthe game.

4.7 High-fidelity prototypeAs discussed in section 3.4.2, the high-fidelity prototype was basically a translationof the paper prototype into a computer environment.

While this proved useful for validating that the conclusions drawn from the low-fidelity prototype were still valid in a graphical user interface, it provided little inthe form of additional design insight. Once the participants had been introducedto the graphical environment, they completed their given assignments without anymajor difficulties.

4.8 Version OneOnce development of a working version had begun no major changes to the designwere made.

Given a completed tool, the intern at Paradox interactive managed to createa fully-functional adventure in a matter of two weeks, requiring only intermittentassistance.

A screenshot of the final version of the working tool can be seen in figure 4.2.

23

CHAPTER 4. RESULTS

Figure 4.1. A reproduction of the senior designer’s flowchart.

Figure 4.2. The final version of the tool

24

Chapter 5

Conclusions

5.1 An Adventure Game Creation ToolFrom my findings and results in the test sessions I created a design for a generaladventure game creation tool, the specification of which is presented here.

As described in section 4.2, the adventure game creation tool is built upon fourbasic concepts: Entries, branches, side effects and preconditions.

Each entry is associated with a descriptive text, and is linked to several otherentries via branches. The user creates branches by connecting entries together in agraphical user interface; see sections 4.2.1 and 4.2.2 for more details on entries andbranches.

To describe the state of the game world, the tool uses variables. Variables servethe same purpose as the character sheet in a gamebook (see section 4.2.3) andconsist of a name and value. All variables used in the adventure are visible in theuser interface.

There are several implications of using variables, such as typing, declaration, etc.However, the tool tries to keep this complexity removed from the user: Variablesare created automatically as they are used and are dynamically typed.

When a user creates a new adventure, some variables are already defined. Someof the predefined variables have a special meaning as pertains to the rule of thegame, while others mainly serve to provide a template for the user as she createsthe adventure. The predefined variables are described below:

• Health Points - represents the general well-being of the player. If the HealthPoints variable is decreased below one, the game is over.

• Combat Bonus - a temporary increase in the player’s combat prowess. Aftereach combat, this number is reset to zero.

• Combat Penalty - a temporary decrease in the player’s combat prowess (seeabove).

• Gold - the number of gold pieces the player has in her inventory.

25

CHAPTER 5. CONCLUSIONS

• Dice result - the result of the Roll Dice operation is stored here (see be-low).

For each entry, the user can add side effects, which affect the state of the ad-venture in some way. The operations that can be used as side effects are describedbelow:

• Add ...to ... - adds a value to a variable.

• Subtract ...from ... - same as above, but using subtraction instead ofaddition.

• Multiply ...by ... - multiplies a variable by a value.

• Divide ...by ... - same as above, but using division instead of multiplica-tion.

• Set ...to ... - sets a variable to a value.

• Roll dice - rolls a six-sided dice and saves the result to the Dice resultvariable.

Note that for any operation that requires a value, a variable can be used instead.For example, one might express the player purchasing a ferry trip with the expressionSubtract Ferry Price from Gold, given that there is a Ferry Price variable.

For each branch, the player can attach one or more preconditions which mustbe true in order for that branch to be available. The available preconditions are:

• ...is greater than ... - tests whether a value is larger than another value.

• ...is lesser than ... - tests whether a value is lesser than another value.

• ...equal to ... - tests whether one value is equal to another value.

• ...not equal to ... - tests whether one value is not equal to another value.

For reasons concerned with implementation details1, the player’s inventory andset of skills had to be implemented as lists, i.e. collections of values instead of singlevalues. The reason I mention these separately is that I do not believe they arestrictly necessary for building adventures, but for the sake of completeness they areincluded here anyway.

The following operations were used to handle the player’s inventory:

• Give ...to player - adds a value to the player’s inventory list.1A requirement of the game player client was that it should have an interface for managing

inventory items (see section 1.3.2). For this to work in the integration between player client andthe adventure creation tool items and skills had to be put into specially designated lists.

26

5.2. EVALUATION

• Take ...from player - removes a value from the player’s inventory list.

• ...in inventory - checks whether a given value is in the player’s inventorylist.

• ...not in inventory - checks whether a given value is not in the player’sinventory list.

Each of the operations above also had an equivalent for managing the protago-nist’s skills (see also section 4.2.3 for more details regarding items and skills in LoneWolf ).

5.2 EvaluationHaving arrived at a finished specification, did the adventure game creation tool fulfilthe stated requirements? Paraphrasing the goals section (1.3.1), the goals of thesystem were to:

• Provide a tool usable by non-programmers to create adventure games.

• Be flexible and powerful enough to allow for the recreation of the Lone Wolfadventure Flight from the Dark.

As indicated by the results in section 4.6, the system does not require previousprogramming experience to use and appears to be easy for beginners to learn anduse. As mentioned in section 4.6.5, it was possible to recreate Flight From the Darkusing the operations available in the final version of the prototype.

The design of the adventure game tool was also successfully translated to a fully-functioning computer version, as detailed in section 4.8. The computer version ofthe tool was used to construct an adventure that could later be used by ParadoxInteractive.

27

Chapter 6

Discussion

In this section I will discuss the methods used, the results achieved and the con-clusions that can be drawn from them. I will also attempt to relate my findings toother research in the field of programming language usability. Finally, I will presentsome recommendations for further work that could be done in the field of usablegame creation tools.

6.1 Methods used

6.1.1 A user-centered approach

First of all, I believe that using a user-centered approach from the onset of theproject was of great benefit to the development of the tool. Every test or workshopsession that I held with users yielded actionable insights, and allowed me to iterateand course-correct the design of the tool along the way.

I believe that if interactions with users had been less frequent or deferred tothe end of the project I might have taken the design in the wrong direction for toolong to bring it back on the right track. Also, there would have been the risk thatI, having invested a lot of time and effort into a bad design, would subconsciouslydismiss any indications that it did not work.

6.1.2 Low-fidelity Prototyping

Another method that I think greatly benefited my work was the use of low-fidelityprototypes. Even though they were the simplest prototypes imaginable, consistingof just paper and Post-It notes, they still generated good feedback.

The “low tech” nature of the prototype also seemed to foster a spirit of creativityand collaboration: I believe that the users felt empowered to make changes tothe prototype in a way they would not have with a high-fidelity version, probablybecause they could make the changes themselves, right there in the test session.

Using low-fidelity prototypes also allowed for an iteration speed that wouldnot have been possible otherwise. This quote from Some Reflections on Designing

29

CHAPTER 6. DISCUSSION

Construction Kits for Kids [25] rings especially true after my experience with low-fidelity prototyping:

Just as we want kids to iterate their designs, we apply the same principleto ourselves. In developing new technologies, we have found that wenever get things quite right on the first try. We are constantly critiquing,adjusting, modifying, revising. The ability to develop rapid prototypesis critically important in this process. We find that storyboards are notenough; we want functioning prototypes. Initial prototypes don’t needto work perfectly, just well enough for us (and our users) to play with,to experiment with, to talk about.

6.1.3 Significance of ResultsAll of the work presented in this thesis has to be viewed in light of the fact thatit was conducted with a relatively small group of people; all in all, about a dozenusers from inside and outside of Paradox Interactive were engaged in my researchand testing work. Thus, the results are by their nature very qualitative.

That is not to say that they are not significant however; as mentioned in sec-tion 3.4.1, testing with five users is usually enough to uncover most usability errors.However, it would have been interesting to test the prototype with people of morediverse backgrounds, for example fiction authors with no connection to the gameindustry.

6.2 Low floor, wide wallsSimplicity, composability and elegance are often lauded as ideals of programming,while coupling, specialization and redundancy are often considered anathema. Theideal programmer’s toolbox is small, but very powerful in its potential for creatingabstractions. From this toolbox the programmer is then free to compose and weavenew constructs, uniquely tailored to almost any domain or problem.

However, this “less is more” approach can have a detrimental effect on thelearnability of the programming environment. In Seven Deadly Sins of IntroductoryProgramming Languages[17], Linda McIver and Damian Conway write:

Learning to program is difficult. We believe that a substantial part ofthis difficulty arises from the structure, syntax and semantics of the pro-gramming languages which are commonly used to teach programming.Programming language designers are (of necessity) highly intelligent ex-perts in the field of programming, and are consequently far removedboth temporally and cognitively from the difficulties experienced by thenovice programmer. This gulf of experience and ability results in lan-guages which are either too restrictive or too powerful (or sometimes,paradoxically, both).

30

6.3. VISUAL PROGRAMMING

Further,

A “less is more” approach is usually justified in terms of paradigmaticpurity: strict adherence to a single functional, logical or object orientedparadigm. While this orthodoxy can make for a certain conceptual sim-plicity and elegance (which can be of considerable advantage in teachingconcepts such as scoping, recursion and encapsulation), in practice it canalso lead to extremely obscure and unreadable code.

In Some Reflections on Designing Construction Kits for Kids [25] Mitchel Resnickand Brian Silverman presents the “Low floor and wide walls” principle:

The Logo programming language is often described as having a lowfloor and high ceiling: it is easy for novices to get started (low floor)and possible for experts to work on increasingly sophisticated projects(high ceiling).The design challenge is to develop features that are specific enough sothat kids can quickly understand how to use them (low floor), but generalenough so that kids can continue to find new ways to use them (widewalls).

If the user is instead provided with a larger box of highly specialized tools itmay be easier to use, with the tradeoff that it might not be as powerful, genericor composable. However, as noted by Resnick et al., “What initially seems like aconstraint or limitation can, in fact, foster new forms of creativity.”

A recurring theme throughout the thesis work was that the major usabilityproblems I encountered could be solved by applying the “low floor, wide walls”philosophy. Coming from a software development background, I think that myselfand my peers may be predisposed to sacrificing ease of use for flexibility. As pro-grammers by trade, we know no two systems are the same, and so we fear paintingourselves into a corner if we opt for the “simpler” tool.

In short, instead of picking the Swiss Army Knife, programmers will choose avariety with a bigger blade, knowing it will be strong enough to carve completelynew tools if the need arises.

However, this is diametrically opposed to the needs of the novice or non-programmer;they neither want nor need the same amount of flexibility, since the domain of theirprogramming tasks are usually a lot more constrained. As such, there is no pointin sacrificing ease of use for the ability to solve any given programming problem.

6.3 Visual programmingFrom my findings in section 4.1 I think it safe to conclude that the visual program-ming paradigm was a good fit for this particular problem. I believe there wereseveral reasons a visual programming environment was appropriate:

31

CHAPTER 6. DISCUSSION

• The visual programming paradigm mapped well to the users’ existing men-tal model of viewing the adventure as a flowchart. Re-using the flowchartanalogy in the programming environment probably made it easier for users tounderstand how they should work with the system.

• A visual programming environment allows for supporting the user and elimi-nating user errors in entirely different ways than having the user type in text.As observed by Resnick et al., “When people play with LEGO®bricks, theydo not encounter error messages.” [16]

• A side effect of using visual programming was that it was easy to constructa low-fidelity prototype. This allowed me to find and solve design problemstogether with users at a very early stage.

As stated previously, a visual programming environment does not afford thesame level of control and flexibility as a text-based language. In this case, however,providing just enough flexibility proved to be just that—enough.

6.4 Less is Less?As touched upon previously in this section, a “less is more” approach can oftenimpact learnability in a negative way. In Seven Deadly Sins of Introductory Pro-gramming Languages[17] Damian Conway and Linda McIver write:

The “less is more” principle appears in many forms, almost all of whichseem to be ultimately detrimental to the learning process. Perhapsthe most obvious examples are the Scheme language and other LISPvariants. Scheme has effectively only one data type—the list—and oneoperation—evaluation of a list. While this abstraction is very simple toexplain, and not difficult for the beginner to grasp superficially, it doesresult in code that is difficult to read because of large numbers of nestedparentheses and the absence of other structuring punctuation.

The idea that less is not always more was something I found evidence of multi-ple times during the development and the evaluation of the tool. The best exampleis probably the results of changing the behavior of the Set ...to ... operation,breaking it up into different operations, with separate operations for addition, sub-traction, multiplication and division. (See section 4.6)

While this approach certainly afforded less flexibility and expressivity—the ar-gument to Set ...to ... could be an arbitrary mathematical expression, afterall—in proved to be easier to use, and not as limiting as I had imagined. This alsoreflects the experience of Resnick et al.; in a section of Some Reflections on Design-ing Construction Kits for Kids titled Make it as simple as possible—and possiblyeven simpler they write:

32

6.5. PROVIDING A TEMPLATE

In the mid-1990s, for example, we had developed a Programmable Brickthat was roughly the size of a child’s juice box. It could control fourmotors and receive inputs from six sensors. For a sponsor event atthe Media Lab, we wanted to create some interactive decorations for thetables. We didn’t need all of the capabilities of the Programmable Brick,so we quickly developed a smaller, scaled-down version, roughly the sizeof a matchbox car. [. . . ] We expected it to be a short-lived project;we “knew” that most users would want more motors and more sensors.But once we had developed the scaled-down version, which we called aCricket, people kept finding more and more creative applications for it,in spite of (or perhaps because of?) its apparent limitations. Over time,we shifted our research effort, making the Cricket the centerpiece of ournew construction kits. Even though the original Programmable Brickwas better suited for certain projects, the simplicity of the Cricket wonout.

Implicitly, I “knew” that the user would have to be able to define values us-ing complex mathematical expressions. However, the simpler way of working witharithmetic proved to be quite enough to solve the assignments given to the partici-pants

6.5 Providing A TemplateSeveral studies of novice programmers suggest that the largest problems that novicesface are not caused by misconceptions about language constructs; rather, the biggestdifficulties lie in learning how to plan and construct algorithms and programs [26][14] [12].

A tacit assumption that I harbored at the beginning of the thesis work was thatthe tool would have to support creating new abstractions, such as subprocedures ina regular programming language. However, as mentioned in section 4.3, I receivedstrong indications from the workshops that this was not necessary. Removing sub-processes from the design made it less powerful, but also preempted the compositionproblems that subprocesses may have introduced.

Another example of how composition problems were moved out of the usershands was the use of predefined variables. Having a “magic” variable called Diceresult removed the need for the user to compose her own way of storing and usingthe results of the Roll dice operation.

Some of the predefined variables, such as Gold, did not by themselves provideany additional functionality. However, they might have helped provide a templatethat could inform the user’s design of her adventure. Since I did not consider theabsence or presence of predefined variables as a factor in the user studies I cannottell for sure.

In summary, I found that removing incidental design decisions from the handsof the user made the system easier to use, which also ties in to the theme of making

33

CHAPTER 6. DISCUSSION

things “as simple as possible, and possibly even simpler”1. Furthermore, informingthe user’s own design by providing a template to follow (e.g. predefined variables)also improved the usability of the system.

6.6 Domain-specificityI believe one factor that greatly improved the usability of the system was the factthat it was built with the explicit goal of creating adventure games. There seemedto be a strong correlation between domain specificity and usability; steps taken toincrease one often led to an increase in the other. Some examples:

• The concept of flowcharting (see 4.4) was borrowed from the adventure gamedomain. I believe this was a great help in aiding the user’s understanding ofthe system, especially for those who had prior experience designing adventuresusing “analog” flowcharts.

• Specializing the functions of built-in operations had positive impacts on us-ability (see 4.6).

• Introducing a “template” of variables specific to adventure games seemed tohave a positive impact on usability. (see 6.5)

This also relieves the user of the task of having to come up with an appropriatemodel for the problem domain (see earlier section).

Previous work in usability research also indicates that making a tool specific toa particular domain makes it easier to use. In Novice Comprehension of ProgramsUsing Task-Specific and Non-Task-Specific Constructs[7] Engebretson and Wieden-beck note that:

[. . . ] there is a tension between task-specificity and generality. Task-specific languages allow programmers to easily create certain kinds ofprograms, but may not be so easily extensible to other tasks that aprogrammer wishes to do. General purpose programming languages arehard to learn but are powerful, flexible tools.

Engebretson and Wiedenbeck found that novice programmers’ comprehensionof programs was significantly improved by using task-specific constructs, and thatcorrectly modifying a program was much easier when task-specific constructs wereused [7].

In conclusion, unless an explicit goal of the system is to allow the user to easilytransition to a more advanced programming language, generality is of little intrinsicvalue in novice programming systems. Rather, the designer should embrace domain-specificity and use it as a tool to help users understand the system.

1See Some reflections on designing construction kits for kids by Resnick et al.[25]

34

6.7. RECOMMENDATIONS

As John Maloney notes: “[. . . ] many Scratch users may not go on to learn Javaor major in computer science, just as most of us who took music lessons did notgrow up to become professional musicians.” [30]

6.7 RecommendationsIt would be very interesting to see further developments in the area of usable gamecreation tools. In an era where social media has blurred the line between con-tent producers and consumers, there might also be room for creating and sharinginteractive stories.

Scratch has already made significant headway in enabling users to share theircreations; with a vibrant online community centered around the Scratch website ithas become a sort of “YouTube for interactive media” [24].

However, Scratch’s primary audience consists of kids and young adults—maybeit would be possible to develop a tool for creating interactive stories that wouldappear to a wider, or at least grown-up, audience?

Some conceivable applications for an interactive storytelling tool could be:

• Letting people create and share interactive novellas which could be played onsmartphones and tablet devices. A graphical programming environment couldalso allow users to create the story itself on the tablet, making the processeven more accessible.

• Revisiting B.F. Skinner’s original idea of using interactive textbooks for au-tomated learning, as mentioned in section 1.1.1. A functioning, self-guidedstudy system would allow students to learn at their own pace and allevi-ate teacher workload. Since smartphones are nearly ubiquitous, even amongyounger children, this would also allow for study outside the classrom. If thesystem is usable enough, a teacher could customize the textbook to suit herclass, maybe even create new interactive study material from scratch.

• Creating “expert systems” that can give users interactive support by simulat-ing the decision-making ability of a human expert. With the advent of the“AI winter” in the late eighties, work on expert systems ground to a halt;among the cited reasons for the collective “failure” of these systems were dif-ficulty of maintenance and high upkeep costs[3]. However, if the interface tothe knowledge base could be made easy enough for a domain expert to usedirectly, without involving programmers, these problems might go away.

Having evaluated the adventure game creation tool developed in this thesis Iwould recommend to use its design, or variants thereof, if one wishes to create atool for building Lone Wolf -style adventures.

As for developing end-user programming systems, I would summarize my rec-ommendations as follows:

35

CHAPTER 6. DISCUSSION

• Leverage low-fidelity prototyping! The low cost, quick iteration speed andhigh level of user participation makes low-fidelity prototyping a very efficientway of exploring design ideas.

• Don’t be afraid to make the system specific to a particular domain. Whendesigning systems for novices, it is often worth trading flexibility for easeof use. Don’t fall into the trap of trying to make your system solve everyconcievable problem!

• Provide templates and examples to guide the user as she learns about thesystem. Don’t be afraid of redundancy; providing several ways of solving aproblem can improve usability for novices.

• Test early and test often. Your inital assumptions about users, their behaviourand mental models are often wrong.

• Creating syntactically correct programs is hard and a big source of potentialuser errors. If possible, try to create tools where the users can disregard syntaxand focus on semantics instead.

36

Bibliography

[1] Jorge Luis Borges. Ficciones. Sur, 1941.

[2] Donald D. Chamberlin and Raymond F. Boyce. Sequel: A structured englishquery language. In Proceedings of the 1974 ACM SIGFIDET (Now SIGMOD)Workshop on Data Description, Access and Control, SIGFIDET ’74, pages249–264, New York, NY, USA, 1974. ACM.

[3] Wikipedia contributors. Ai winter article. http://www.en.wikipedia.org/wiki/AI_winter. Retrieced at 2015-05-07.

[4] Wikipedia contributors. Gamebook article. http://www.en.wikipedia.org/wiki/Gamebook. Retrieved at 2011-02-22.

[5] Joe Dever. Flight From The Dark. Beaver Books, United Kingdom, 1984.

[6] Joe Dever and Project Aon editors. Lone wolf miscellaneous 2: Correspondence,tie-ins, & articles. http://www.projectaon.org/en/pdf/misc/LW_Articles.pdf, 2009. Originally sent to members of the Lone Wolf club in 1990.

[7] Alec Engebretson and Susan Wiedenbeck. Novice comprehension of programsusing task-specific and non-task-specific constructs. In Proceedings of the IEEE2002 Symposia on Human Centric Computing Languages and Environments(HCC’02), pages 11–19, Washington, DC, USA, 2002. IEEE Computer Society.

[8] Christopher D. Hundhausen, Sean F. Farley, and Jonathan L. Brown. Candirect manipulation lower the barriers to computer programming and promotetransfer of training?: An experimental study. ACM Trans. Comput.-Hum.Interact., 16:13:1–13:40, September 2009.

[9] Simon Peyton Jones, Alan Blackwell, and Margaret Burnett. A user-centredapproach to functions in excel. SIGPLAN Not., 38(9):165–176, August 2003.

[10] Caitlin Kelleher and Randy Pausch. Lowering the barriers to programming:A taxonomy of programming environments and languages for novice program-mers. ACM Comput. Surv., 37:83–137, June 2005.

[11] Steven L. Kent. The Ultimate History of Video Games. Three Rivers Press,New York, NY, USA, 2001.

37

BIBLIOGRAPHY

[12] Andrew J. Ko, Brad A. Myers, and Htet Htet Aung. Six learning barriers inend-user programming systems. In Proceedings of the 2004 IEEE Symposiumon Visual Languages - Human Centric Computing, VLHCC ’04, pages 199–206,Washington, DC, USA, 2004. IEEE Computer Society.

[13] Steve Krug. Don’T Make Me Think: A Common Sense Approach to the Web(2Nd Edition). New Riders Publishing, Thousand Oaks, CA, USA, 2005.

[14] Essi Lahtinen, Kirsti Ala-Mutka, and Hannu-Matti Järvinen. A study ofthe difficulties of novice programmers. In Proceedings of the 10th annualSIGCSE conference on Innovation and technology in computer science edu-cation, ITiCSE ’05, pages 14–18, New York, NY, USA, 2005. ACM.

[15] Mattias Lilja and Joachim Burman. Coriolis. Jarnringen, 2008.

[16] John Maloney, Mitchel Resnick, Natalie Rusk, Brian Silverman, and EvelynEastmond. The scratch programming language and environment. Trans. Com-put. Educ., 10:16:1–16:15, November 2010.

[17] Linda McIver and Damian Conway. Seven deadly sins of introductory program-ming language design. In Proceedings of the 1996 International Conference onSoftware Engineering: Education and Practice (SE:EP ’96), SEEP ’96, pages309–, Washington, DC, USA, 1996. IEEE Computer Society.

[18] Raquel Navarro-Prieto and Jose J. Cañas. Are visual programming languagesbetter? the role of imagery in program comprehension. Int. J. Hum.-Comput.Stud., 54:799–829, June 2001.

[19] Jakob Nielsen. Heuristic evaluation. http://www.useit.com/papers/heuristic/. Retrieved at 2011-12-12.

[20] Jakob Nielsen. Usability engineering at a discount. In Proceedings of the ThirdInternational Conference on Human-computer Interaction on Designing andUsing Human-computer Interfaces and Knowledge Based Systems (2Nd Ed.),pages 394–401, New York, NY, USA, 1989. Elsevier Science Inc.

[21] J.F. Pane and B.A. Myers. Usability issues in the design of novice programmingsystems. Technical report, Carnegie Mellon University, School of ComputerScience, August 1996.

[22] John F. Pane, Brad A. Myers, and Chotirat Ann Ratanamahatana. Studyingthe language and structure in non-programmers’ solutions to programmingproblems. Int. J. Hum.-Comput. Stud., 54:237–264, February 2001.

[23] Seymour Papert. Mindstorms: Children, computers, and powerful ideas. BasicBooks, Inc., New York, NY, USA, 1980.

38

[24] Mitchel Resnick, John Maloney, Andrés Monroy-Hernández, Natalie Rusk, Eve-lyn Eastmond, Karen Brennan, Amon Millner, Eric Rosenbaum, Jay Silver,Brian Silverman, and Yasmin Kafai. Scratch: programming for all. Commun.ACM, 52:60–67, November 2009.

[25] Mitchel Resnick and Brian Silverman. Some reflections on designing construc-tion kits for kids. In Proceedings of the 2005 conference on Interaction designand children, IDC ’05, pages 117–122, New York, NY, USA, 2005. ACM.

[26] Anthony Robins, Janet Rountree, and Nathan Rountree. Learning and teachingprogramming: A review and discussion. Computer Science Education, 13:137–172, 2003.

[27] Helen Sharp, Yvonne Rogers, and Jenny Preece. Interaction Design: BeyondHuman Computer Interaction. John Wiley & Sons, Hoboken, NJ, USA, 2007.

[28] David Canfield Smith and Allen Cypher. Making programming easier for chil-dren, pages 201–221. Morgan Kaufmann Publishers Inc., San Francisco, CA,USA, 1998.

[29] David Canfield Smith, Allen Cypher, and Jim Spohrer. Kidsim: programmingagents without a programming language. Commun. ACM, 37:54–67, July 1994.

[30] Ian Utting, Stephen Cooper, Michael Kölling, John Maloney, and MitchelResnick. Alice, greenfoot, and scratch – a discussion. Trans. Comput. Educ.,10(4):17:1–17:11, November 2010.

39