jukka-pekka strandén technical framework in game ... · 2.1 development phases game development...
TRANSCRIPT
Lappeenrannan teknillinen yliopisto
Teknistaloudellinen tiedekunta. Tietotekniikan laitos
Tutkimusraportti 115
Lappeenranta University of Technology
Faculty of Technology Management. Department of Information Technology
Research Report 115
Jukka-Pekka Strandén
Technical framework in game development startups
Lappeenrannan teknillinen yliopisto
Teknistaloudellinen tiedekunta. Tietotekniikan laitos
PL 20
53851 Lappeenranta
ISBN 978-952-265-268-3
ISBN 978-952-265-269-0 (PDF)
ISSN 1799-3539
Lappeenranta 2012
Lappeenranta University of Technology
Faculty of Technology Management
Department of Information Technology
Master Thesis
Technical framework in game development startups
Supervisors: Professor Kari Smolander
Doctor Jussi Kasurinen
Lappeenranta 20.6.2012
Jukka-Pekka Strandén
Kaivosuonkatu 1A5
53850 Lappeenranta
Phone: 040-570 4216
ii
ABSTRACT
Lappeenranta University of Technology
Faculty of Technology Management
Jukka-Pekka Strandén
Technical framework in game development startups
Master's Thesis
2012
64 pages, 5 figures, 5 tables and 2 appendixes
Examiners: Professor Kari Smolander
Doctor Jussi Kasurinen
Keywords: Game development, game development tools, game development technical
framework
In this thesis, the objective is to study the technical framworks in game development
startups and compare them to medium size game development organizations with
established product lines. The thesis was done as qualitative research applying grounded
theory method in analysis of the data.
Based on the results, the game organizations, regardless of their size, are pleased with the
tools they have chosen. The main selection criterias for the tools are support for fast
iterations, -sharing and -prototyping regardless of the organization size. Game
development projects are adaptable and features can change rapidly. The technical
framework in game development has to support these features.
i
TABLE OF CONTENTS
1 INTRODUCTION 4
2 BACKGROUND AND LITERATURE REVIEW 6
2.1 Development phases 7
2.2 Design and preproduction 9
2.3 Production 12
3 RESEARCH PROCESS 17
3.1 Qualitative research 18
3.2 Quantitative research 19
3.3 Grounded theory method 20
3.4 Employed research methods 21
3.5 Studied organizations 23
4 OBSERVATIONS FROM THE ORGANIZATIONS 25
4.1 Organization size 25
4.2 Target platforms 25
4.3 Development process 27
4.4 Testing and quality 28
4.5 Outsourcing 30
4.6 Tools 31
4.7 Summary 35
5 DISCUSSION 37
5.1 Validation 37
5.2 Organization size impact 38
5.3 Target platforms 41
5.4 Development phases 42
5.5 Testing and quality 44
5.6 Summary 46
6 CONCLUSIONS 48
REFERENCES 49
1
ABBREVIATIONS AND TERMS
Crunch time Period of intensive working in an attempt to meet a deadline.
Usually situated near the end of the project. Common in game
development.
Feature creep Common term in game development for adding features into the
game during development phase.
GDD Game Design Document
SOCES Software Development in Creative Ecosystems
2
PREFACE
When I started my studies in 2003, I never imagined they would last this long. But along
the way I encountered few hardships and my studies kept prolonging. I even changed from
the one step master's program to two step master's program in 2010. But finally, there's
light at the end of the tunnel. Although, the tunnel was very interesting albeit a long one
too.
Likewise, when I started at 2003, I never dreamed that I would do my master's thesis on a
topic that's so close to my heart: video games. So, I'd like to thank supervisors prof. Kari
Smolander and dr. Jussi Kasurinen for getting me into this project and for their support,
guidance and patience with my endless questions during the writing process. I'd also like to
thank dr. Ossi Taipale for the interesting coffee table conversations and helpful hints for
writing the thesis. Special thanks to my family and friends for their support during my
entire studies.
Lappeenranta, June 19th 2012
Jukka-Pekka Strandén
3
1 INTRODUCTION
This master thesis work has been done as part of the SOCES (Software Development in
Creative Ecosystems) -project in Lappeenranta University of Technology. SOCES -project
has three goals; to help startup game companies, to help existing organizations to develop
their practices and improve the quality of their products and to develop a model for the
actual game development process.
The goal of this work is to study the tools used by young game organizations. Rather than
naming specific tools, this work discusses them in types; what kind of tools are used by the
organizations e.g. 3D modeling software, documents and game engine to name a few. This
work studies what are the common problems, the organizations seek to overcome with the
tools and if they are content with their choices, or are the tools missing some important
features that the organizations feel like would benefit them.
This thesis discusses game development in phases. In traditional software engineering, the
term “development” is associated with the phase where the application is implemented and
“production” is the phase where software is manufactured and shipped out [1]. However, in
literature which studies game development, development phase is called “production” and
the production phase is called post-production or shipping phase, so this work uses the
same terms [2–4] when talking about those phases. In this work the phases are called
preproduction, production and post-production. Figure 1 illustrates how the phases are
discussed in this work. Game development occurs in preproduction and production phases.
Work and project support run through the entire project and contains project management
and work assisting methods and tools.
This work is divided into five parts. Part 2 contains the background and literature review. It
discusses the motivations for studying games and game development as well as the
previous research on game development. The part is divided into three subsections where
4
the first section discusses the phases in general. The second subsection reviews literature
discussing the preproduction phase and the third discusses production phase. Part 3
discusses the employed research methods, SOCES -project in more detail and introduces
the organizations that participated in this study. Part 4 covers the results of the study and
part 5 discusses and analyzes the results regarding tools and the technical framework
surrounding game development.
5
2 BACKGROUND AND LITERATURE REVIEW
Games have become an increasingly popular research area in the 21st century. However, the
research has been mostly focusing on using games in education and the effect games have
on people. In the recent years, more articles regarding game development and game
development improvements have been published.
One reason for the increased interest towards games, is the increasing popularity of games.
Studies show that majority of people 10-72 years old play video games and the traditional
view, that gamers are mostly teenage boys who play in a dark cellar after school, is
outdated [5], [6]. The average age of a gamer now is above 30, people well into working
life. Report made by Entertainment Software Association [6] (ESA) concludes that 52% of
gamers are men and 48% are women. Games have become popular and are becoming part
of our culture, which is evident in the growth of the video game industry and its steady
growth since 2000. Suomen Pelinkehittäjät Ry (Finnish game development association)
[5], in their strategy plan for 2010-2015, states that in 2008, the worldwide game sales
were roughly 50 billion US dollars. Equivalent number for the movie industry is 84 billion
and 30 billion dollars for the music industry. Game sales have overtaken the music
business and are gaining the movie business with 1 billion per year. This trend is global
and shows also in Finland as game sales have been rising along with the number of people
working in Finnish game industry. The plan also predicts that Finnish game sales, as well
as Finnish game industry, will continue growing in the near future. [5], [6]
The plan predicts that digital markets will become increasingly popular distribution
method for game studios [5]. In digital markets, the user creates an account to a service and
can then purchase games either through the service's website or through a separate
application, which may be mandatory for the service to be accessed. Once the purchase is
made, the user can download and install the game, usually through the application and in
many cases, the application will keep the game up-to-date, installing patches and updates
automatically (e.g. Steam [7] and Origin [8]). This new marketing way has partly replaced
6
the traditional physical media, also eliminating the expenses of manufacturing game boxes
and the media. Applications, including games, for mobile platforms are sold through digital
marketplaces like Apple's iTunes [9] and Google Play [10]. Digital markets have also
become more popular in PC environment and are still growing [5]. In recent years, every
major console has gained a digital marketplace for buying games and downloadable
content for games. Examples of digital retailers are Steam [7], Origin [8] and GoG.com
(Good old Games) [11]. In traditional game production, the game studios present their
game idea or a concept demo to a publishing company (e.g. Electronic Arts [12], Paradox
[13]) which might agree to publish it. If so, the publisher pays the game organization for
the production of the game in an allocated time and in a way, it is akin to the game studio
working as a subcontractor. This also reflects on how the income, the game produces, is
divided. Roughly 10% of the income goes to the game studio and 90% to the publisher.
With digital release, the game studio gets 70% and the distributor gets 30%. With this
model however, the game studio often pays the expenses for the development and
marketing, although many digital marketplaces offer marketing packages. [5], [6]
2.1 Development phases
Game development can be divided into preproduction and production phases with work
support going alongside the phases for the entire project (figure 1). Furthermore, game
development is an iterative process where the game goes through several iterations before
its final release. Articles that focus on design phase seem to be more common while
articles that discuss the production phase are often more general in nature. Figure 2 shows
the game development process as it is described in literature. The boxes in the figure
illustrate phases and actions while the document symbols represent artifacts, documents in
this case.
7
Figure 1: Game development phases
Work and Project SupportPreproduction Production
The game is designed in preproduction phase. This involves creating the game design
document (GDD) and prototypes to polish the game design and mechanics. Once the
mechanics have been tuned, the game can move into production phase. During this
transition, the requirements document is created from the game design document. The
requirements documentation contains all the technical information required to implement
the game mechanics described in game design document into the actual game. In
production phase, the actual implementation of the final product starts. This is done in
several iterations and iterative developing methods such as Scrum are often used. The
game testing is already launched in production phase and based on the tests, revisions are
made to the code. Even changes to the game mechanics can be made at this point. When
changes are required, the game design- and requirements documents have to be updated to
match the change. After this the change is implemented into the next iteration. However,
large redesigning and prototyping is not made in production phase in this model. Instead
prototyping is used in preproduction to ensure that the design is good and does not require
much redesigning when writing the actual production code. After the game meets the
quality standards set for it, the game moves to postproduction phase. In this phase the
game is finished and handed out to the publisher for distribution. Postproduction in detail
8
Figure 2: Game development model in literature. Adopted from [3], [4], [16–
19]
Preproduction Production
Game DesignDocument
Requirements
UpdatesCreation
Iterations
Postproduction
TestingPrototyping
UpdatesCreation
is outside the scope of this thesis because the game is often handed out to the publisher for
printing and distribution, but it is mentioned in this process model for clarity. Sections 2.2
and 2.3 discuss the preproduction and production phases in more detail.
2.2 Design and preproduction
Many articles on game development focus on the design and the preproduction phase, the
methods employed in game development in this phase, identifying the problems and offer a
tool or method that could improve the situation. Preproduction phase is considered to be
very crucial to the success of the game project but is often overlooked by the management.
Generally, fixing a requirements defect at the end of the production phase is roughly at
least 50 times more expensive than fixing it in preproduction. [2–4]
Requirements for games are difficult to write, mainly because game development requires
very heterogeneous group to create the game. Artists, programmers, coders, dialogue and
plot writers to name a few that are required in major productions. All of them have their
own vocabulary and therefore communication between people with different vocations can
become a problem. Callele et al. [3] identified this as one problem when writing
requirements. Another problem that they mention is the existence of abstract concepts such
as fun, immersion and enjoyment which exist in game development but are very difficult to
describe in a technical document. [3] Petrillo et al. have noted the same problems in their
paper “What Went Wrong? A Survey of Problems in Game Development” [14]. Callele et
al. focus in documentation, mainly in game design document and requirements document.
The former is a document which defines the game idea, story, the characters in the game,
dialogue etc. It is written by game designers and while it should be thorough, it is not
formal. In fact, formality could be detrimental at this stage, because it can limit creativity.
The requirements document is a technical document, which has the GDD's information
turned into technical tasks. The two require very different styles, so it is unreasonable to
except one person to write both. Callele et al. [3] claim that many organizations do not
spend enough time writing the documentation properly and hurry on to production phase
9
too early. This can be detrimental in the long run. They state that problems stemming from
requirements can cost up to 100 times more to fix after delivery, than if caught at the start
of the project [3]. Bethke [2] mentions the same problem, although he also emphasizes that
it is important to understand the goals of the production. He reminds about the importance
of the common software engineering model, which is depicted in Figure 3. The project's
goal must be selected from a triangle with cost, schedule and quality as the triangle's sides.
For the project to be successful, only one point or one side can be chosen. If the goal is to
make the game in time and on budget, then time constraints prevent staying too long in the
preproduction phase. This of course is detrimental to the quality of the game but is often
necessary. [2]
However, documentation is overall thorough in the preproduction phase as documents are
generated and updated to help design the game and present the idea to the publisher.
Winget and Sampson [15] noted an occurrence where the game design document was left
inaccurate on purpose since the publisher might see it as a binding contract to deliver
everything mentioned in the document. Instead, the developers would present the ideas,
excluded from the game design document, in production phase if they had time to
implement them. This kind of approach is rare though and generally the documents are
detailed. In the production phase, however, they are updated less and less the longer the
project progresses. [15]
Lewis and Whitehead [16] mention communication to be a problem among game
10
Figure 3: Setting goals and the scope for the project in the early preproduction
is important.
CostQuality
Schedule
designers. They state, that game designers can apply broad genres at a high level to games
but the components, that make the game, remain undefined making it harder to reuse those
components. [16]
Prototypes are widely used in game development to test the game design before actual
implementation starts to see if the game design is feasible and working [15–17]. Both
paper- and software prototypes are employed by the industry. Ollila et al. [18] studied
prototype use in pervasive game development. They concluded that both software and
paper prototypes are useful in the early phase of development to test the ideas and game
concepts. Paper prototypes are useful for quickly testing simple ideas and concepts.
However, larger paper prototypes require significant effort to produce and the effort is not
worth the benefit. [18]
Software prototypes require the knowledge to use the prototyping tools but are useful to
test concepts that are dependent on multiple technical concepts e.g. different sensors. This
gives direct data if the whole game project is feasible at all. Abandoning or editing the
project in this phase can save the game studio a significant amount of money. Software
prototypes also give feedback on technical issues and usability problems. [18]
Simulation can be used as an extension to prototyping in early development to test game
design. Nummenmaa et al. in their article “Simulation as a Game Design Tool” [19] state
that simulation is good in early development to test how the system behaves. While many
consider prototyping and simulation to be technically the same task, Nummenmaa et al.
claim that since simulation requires less manual data input than prototypes, it can be more
effective as the designers get feedback on questions on how the system behaves and are not
burdened or distracted by the usability of the prototype. Simulation aims to find out how
the system behaves rather than what each individual component does. The latter is
answered by prototypes. Nummenmaa et al. also emphasize the importance of a formalized
model of the game. They believe it improves the programmer's understanding of the design
situation and is a starting point when making the simulation. Formalized model can
provide a common ground for the programmers and designers to discuss the game design
11
on. [19]
2.3 Production
Production phase follows the preproduction phase and in this phase, the game is
implemented. Most articles do not directly focus studying the production phase but are
more oriented to study game development in general. For example these articles discuss
the production phase without focusing on it [14], [16], [20–23].
One article, which is considered among the most important ones in the field, is Blow's
“Game Development: Harder Than You Think” [24]. It is written from the perspective of a
game developer and reveals many hardships and problems in game development. It also
justifies why game development can be considered the most difficult area of software
development. Games have grown in complexity and keep growing as the technology
advances. As such they the programmer must be well versatile in many different areas of
the game like the game engine, network code and artificial intelligence to name just a few.
And the tools to develop games and address the aforementioned problem areas are
insufficient and often awkward to use.[24]
Blow [24] criticizes development tools that are designed primarily for making other
software than games. He mentions an example of having a tool that would make it possible
to build continuous 3D meshes where multiple artists could work at same mesh at once or a
landscape generating program, that many people could work with to edit the same
landscape at the same time. He gave these as an example of features that would be more
practical and useful for game developers than technologies that focus on more trivial things
from a game developer's perspective such as certain aspects of animation. [24]
Developers have made plugins for the editors to facilitate game development. Blow [24]
claims that these plugins are often scarce in functionality and awkward to use. He also
12
mentions the difficulty of making good game engines and that there are only few available.
Another problem are in game development, according to Blow, is the workflow problems.
These problems do not stem from any specific software, but simply the compile-edit-debug
cycles are too long. Blow mentions that many games take half an hour or longer to compile
when starting from scratch or a major header file is changed [24]. Bethke [2] mentions the
same problem with compiling and emphasizes the importance of module based design, so
that changes to the code, require as little recompiling as possible. When compile times
become too long, the team will spend considerable amount of time into refactoring the
source code to make it build more quickly. However, at this point the links between the
different classes and files has become so intertwined that refactoring the game would mean
effectively starting the project from scratch. A good way to avoid this problem would be to
architect the game modules to have as little dependencies as possible. As a drawback, this
may cost runtime efficiency.
However, in recent years game engines such as Unity [25] and ShiVa [26] have been
released and they contain a game engine that handles physics, drawing etc., asset manager
and everything else that is required to build a game, excluding graphic and 3D modeling
software [24]. An example of the usefulness of a game engine such as Unity comes from a
paper by de Macedo's and Rodrigues. In their paper “Experiences with Rapid Mobile
Game Development Using Unity Engine”[27], they tested Unity as a game development
tool for developing a game for mobile platforms. They state that Unity increases efficiency
by having a visual editor and support for cross-platform development.
Kanode and Haddad [4] mention similar problem as Blow in using plugins [4], [24]. While
plugins, made for the development tools, allow content from third party programs, such as
modeling software or level editor, to be added into the game by non-programmers,
developing those plugins takes time. And the more complex the game, the more complex
the tools required to make it. This adds overhead to the project and while manufacturing
plugins may be cost effective in the long run, it is directly taken away from productive
work. On the other hand, game organizations prefer to make some of the tools they use
themselves rather than use third party software especially for innovation because they feel
13
it is necessary to truly innovate. Generating tools also adds overhead to the project as it
takes time. But the overall benefits can be worth it. This has been noticed in practice as
well. Arrowhead Studios in their postmortem [28] of their game Magicka [29], concluded
that one of the project's weaknesses was the lack of a level editor and animation tools.
They had to manually code new assets to XML and then play the level to the point where
the asset appeared in it. Then, if there was something wrong with it, either it was scaled
and positioned incorrectly or it did not behave the way the developer intended, they had to
change the values in the XML or source code, and then play the level again to see the
change. They conclude that for future games, they have to create good tools even though it
takes time from actual game development and that in the long run it will actually save
time. [21], [28]
As mentioned above, the tools have their own problems, but they are not the primary
reason why game projects fail, go over budget or the schedule fails. The reason is
management and that projects are not properly led. Kanode and Haddad [4] argue that
game development would benefit from software development practices as many of the
problems are same in game- and traditional software development [4]. Avoiding
standardized software practices in game development could be rooted to game industry as
it has a more heterogeneous group of people working than in other software development
areas. Therefore, finding a single method to follow might be difficult. Communication
itself can become difficult as people from different areas of expertise misunderstand each
other. Petrillo et al. [14], [20] studied game project post-mortems and found a trend that
even issues with poor quality were not because of the development- and design tools, but
rather the lack of good quality control from management. In many cases, the problems'
roots lay in the preproduction phase and crossed over to the production phase. In 75% of
the problem cases, the problem was overambitious features. This resulted in resources
being cut from quality control. Surprisingly, the stereotypical game industry problem of
“crunch time”, a time near the project's end when the people would have to work overtime
in order to meet the scheduled release date, was not conceived as a problem at all. [14],
[20]
14
The most common reason for failure to meet the schedule is “feature creep”; a term for
adding new features into the game in the middle of production phase. Implementing the
features takes time and resources. Kanode and Haddad [4] mention this as one of the main
reasons why the production schedule fails. While feature creep is more common in game
development, it is not exclusive to it. Project manager should reserve room in the schedule
for some feature creep to allow the addition of truly good ideas into the game. However,
extensive additions should be avoided as that would cause long delays and would require
updating the game design- and requirement documents. [4]
Game development has the same problems as traditional software development and largely
uses the same practices [14], [20]. A practice that was noted to be clearly different in game
development than in regular software production is the numerous iterations that are done of
the game within its lifetime. The main reason is that game development has terms attached
to it such as 'fun' and 'immersion'. It is difficult to define these in preproduction phase
when writing requirements. Polishing the game so that it is 'fun' and 'immersible' requires
many iterations in which to test the game and make adjustments. However, the changes to
the game in the production phase should be limited. If there is a fundamental flaw in the
game design, then it might be more cost-effective to abandon the project and start a new
one. Ensuring that the basic ideas and mechanics are valid before moving from
preproduction to production, is therefore essential. [4]
Kanode and Haddad [4] mention Scrum, Extreme Programming (XP) and other agile
methods to be effective in game development because of their iterative nature [4]. Kultima
and Alho [21] mention a similar trend with agile methods, and Scrum in particular,
becoming more popular in game design [21]. In Scrum, for instance, the development is
divided into sprints that range from one week to one month in length, typically two or three
weeks. In each sprint, a working version of the application is produced along with all the
necessary documentation and unit tests completed. At the end of every sprint, there is a
sprint review and sprint planning meeting where the results of the sprints are evaluated and
the next sprint is planned. In addition, there is a short “daily Scrum” meeting each
morning, which is typically 5-15 minutes long depending on the number of team members,
15
where the team members recount what they accomplished the previous day and their plans
for the current day. They also bring forth problems they are having. This way, team
members know what the others are doing in the project. [30]
There are other processes besides agile. Bethke [2] advocates the use of Unified Software
Development Process. The unified process is becoming industry standard and Bethke
names this as one of the main reasons to use the process in game development. As the
process is widely used, most developers ought to understand the diagrams and methods
behind it. This would make it easier to bring new people into the project. Bethke also
emphasizes the importance of use case diagrams in game development. They can help
understand the requirements of the game as well as provide a useful set of test cases for
testing later. The use case diagrams are also helpful in avoiding feature creep since they
help keep the focus on the few gameplay elements that are truly important for the game;
Bethke noted a trend that the most successful games only have few gameplay elements that
have been extremely well tuned. The unified process works also in iterations and after a
change is needed, the production enters another short preproduction phase where the new
requirements are gathered and evaluated before the production continues. [2]
16
3 RESEARCH PROCESS
This work studies the tools used by seven Finnish game organizations. Three of the
organizations are medium size and the others are small startups. The organizations are in
most cases young having been around less than two years. The goal is to find out what kind
of technical framework the organizations have in their game development, what kind of
tools they use in various development phases and do they see any need for improvement.
The technical frameworks of the startups are compared to the technical frameworks of the
medium size organizations with established product lines to see how they differ and if a
game organization has to undergo a change in their technical framework as well, when
growing from a small organization into a medium one. The technical framework in the
context of this work means documents, software and concepts (e.g. prototypes) and covers
areas like design, development, testing and project support. This work is part of SOCES
-project in Lappeenranta University of Technology that studies game organizations in
southeastern Finland. The goal of the project is to find methods and tools that could help
new startup game companies [31].
The study is done using grounded theory method, a qualitative research method, and the
data is collected with interviews. Qualitative methods study phenomena that cannot be
quantified and the sample size can be smaller for qualitative study than for quantitative
study. This does not mean that there is less collected data though. Commonly the data for a
qualitative study is collected via interviews, and with memos and transcriptions there is
considerable amount of data to analyze. Although quantitative methods are not employed
in this work, the basic idea behind quantitative research is explained briefly to better
illustrate the distinction between quantitative and qualitative research. Quantitative and
qualitative could be seen as mutually exclusive methods but that is not the case. Alasuutari
[32] states that quantitative methods give information which is accurate but very high level
and abstract, while qualitative methods give in-depth information but it is very specific. He
also states that combining the two methods to some degree is often useful in order to form
a hypothesis to the research questions. [32]
17
3.1 Qualitative research
The purpose of qualitative research is to study problems in which statistical methods
cannot be applied. In real life, different things are connected to each other with various
relationships. Data for qualitative research is often gathered with interviews and because
one interview can be several pages long in transcribed form, and the number of interviews
is low, the results cannot be analyzed statistically. However, these results are still useful for
qualitative analysis. According to Alasuutari [32], qualitative analysis consists of two
phases; data conceptualization and forming hypothesis. In data conceptualization, the
collected data is analyzed only from the point of the research question. Data
conceptualization turns even large amount of texts into manageable amount of concepts.
The concepts are then further analyzed in forming hypothesis -phase where a single
hypothesis is formed which explains the concepts and the data. If there is even a single
data entry that contradicts the hypothesis, it must be abandoned and a new one developed
which also includes the contradicting data concept. [32]
Main data collection methods are interviews, questionnaires, observation and documents.
Basic methods are common to different qualitative research methods, although some suit
some methods better than others. In qualitative research, data collection methods where the
research subjects' gestures and views can surface, are more popular. One such method is
interviewing. The strong side of interviewing is that the interviewer can interpret the
interviewee's behavior, gestures and speech. This makes it a flexible data gathering
method. As a drawback, the interviewed may feel threatened by the interview and will
withhold information or will even lie. In these cases the interviewer has to attempt to make
the interviewed person relax and spot the possible false information from the gestures.
Qualitative research has different approaches such as grounded theory practice (described
18
in 3.3), organizational storytelling, ethnography or shadowing. Organizational storytelling
focuses on analyzing stories that emerge from ongoing interactions [33]. Analyzing these
stories and how they are used in organizations, can help raise understanding on how people
act [33]. Ethnography is a qualitative research method where the researcher spends time in
the target organization either on the background as a passive observer or as an active
participant working in the organization but simultaneously collects data [33]. Shadowing is
a research method which involves a researcher closely following a person over a period of
time. The researcher goes to every meeting and event the shadowed person goes to. The
goal is to find out how the shadowed person fulfills his role in an organization. [34] All
these methods are used heavily in social sciences, especially ethnography. This work does
not describe the methods in more detail, with the exception of grounded theory method as
it is used in this study, but it is important to know that there are other viable qualitative
research methods besides grounded theory.
3.2 Quantitative research
Quantitative research aims to explain phenomena through numerical and statistical
analysis. According to Alasuutari [32], in quantitative research, the focus is on numbers
and their systematic statistical relationships. The basic idea is to put the results in tables
and assign variables to each research subject. Quantitative research is used extensively in
experimental research where repeatability and statistically valid sample sizes are
important. Quantitative research is also used in surveys where a statistically valid number
of people fill a questionnaire form. Before the questionnaire form for a survey is done, a
hypothesis is formed based on the research question. The results of the survey either prove
or disprove the hypothesis. [32] Creswell [35] emphasizes the point that quantitative
methods are more standardized in the sense that all the papers conform to same basic
layout and everything in the studies can be measured and quantified [35]. Because
quantitative studies need to be statistically valid, they require a large sample size. A sample
size of 30 is commonly seen as a small sample that is still statistically valid. However, the
sample size has to be adjusted according to the study. A sample size of 30 is too small
19
when studying the behavior of a nation for example while it might be acceptable when
studying small businesses in a limited area, e.g. a county, or a specific topic such as testing.
3.3 Grounded theory method
Grounded theory is a qualitative research method where collected data is categorized by
codes, relationships between the codes are modeled and finally a theory is formed from the
codes and their relationships to explain the collected data. The first reported use of the
grounded theory method in practice is from 1965 by two sociologists Barney Glaser and
Anselm Strauss [36]. In 1967 the two wrote a book “The Discovery of Grounded Theory”
in 1967 that described the method [37]. Later on, the method branched into two
distinguished methodologies. Glaser's method is oriented in observing the research target
over long period of time and forming theories as data emerges. Strauss and Corbin's theory
focuses on validation criteria and systematic approach; interviews, active data collection.
Both methods are recognized as valid grounded theory methods [38].
In Glaser's method there is a general research question in the start of the study but no
information about the area of the study is collected prior to the study as that could skew the
results. Data should also be collected iteratively and compared constantly to existing codes
and data properties. Adolph et al. [39] mention spending hours in target organizations
observing their software development methods. [39] This passive observation is common
for Glaserian method as intrusive approaches are considered harmful for the study. Specific
research question is also not present in the beginning, because it would bias the research
and the results. Memos and recording interviews are also discouraged as that would affect
the studied organization and skew the results.
Strauss and Corbin's method is more prescriptive approach. The data collection is more
planned and designed in the beginning and data is gathered with interviews rather than
observations. In Strauss and Corbin's method the systematic codification is in important
20
part. It proceeds in three steps; open, axial and selective. In open coding the general
concepts are marked from the data. This means dividing the data into general blocks which
have a common theme e.g. outsourcing. In axial coding the concepts are categorized into
more specific categories and the connections between other categories are formed.
However, Strauss and Corbin's “Basics of Qualitative Research” [40] mentions that
distinction between open and axial coding is purely artificial and axial coding is done
simultaneously with open coding. When doing open coding, the researcher often makes the
connections in his mind and categorizes the concepts. Thus separate axial coding step is
not necessary. In selective coding the main categories are formed as well as the cause and
effect relationships. Theory is formed in this step. Categories with conceptually similar
conclusions are categorized under the same conceptual label and each new code is
automatically put under this broader label. Each new code also raises new facts and
elaborates the main category. Supportive categories, that are not conceptually similar but
which support the main category, are marked to explain the data better. Once the main
category and its supportive categories explain the gathered data entirely, the hypothesis,
described by the main category, can be considered a valid one.
Glaserian and Strauss-Corbin methods can give different results. Glaserian methods can be
more accurate as the method focuses on unobtrusive observation and the theory is formed
from the observation. As a drawback the method requires experience from the researcher in
order to arrive at the right theory and arranging the long term observation can be difficult.
Strauss-Corbin method is more structured and while it is not necessarily as accurate as
Glaserian method, it does not require long term observation and the structured nature of the
method and the systematic codification in particular means, that even less experienced
researchers can develop a theory that drives the analysis towards a form of results.
3.4 Employed research methods
In this study we chose to use Strauss and Corbin's method because the study includes seven
game organizations which is quite a large sample size for a qualitative study but too small
21
for a quantitative study. Strauss and Corbin's method was chosen over Glaserian approach
because the more prescriptive Strauss-Corbin method gives better control. The method
allows gaining scientifically valid results with interviews and memos as the main sources
of information. The information was gathered with interviews held in four rounds, althou
this thesis focuses on the first two, more technology-oriented interview rounds. Table 1
shows a summary of the interview rounds and their themes.
Table 1: Interview rounds and themes
Round Theme IntervieweeRound 1 General information about projects Project manager or team leadRound 2 Implementation, testing and development tools Programmer or tester or
programmer with tesing
responsibilitiesRound 3 Innovation, marketing Upper management or
company ownerRound 4 Game design, creative aspects of the work Game designer
Because the target organizations are small (less than 10 people working) or medium (more
than 10 but less than 50 working) size, the interviewed person could be the same in
multiple interview rounds although the goal was to interview different people on each
round to get a more complete understanding of the target organizations. The first round
focused on interviewing project managers or team leaders. The purpose was to collect
general information about the organization, development platform, used processes and
their opinions of the game industry. The second round targets were programmers and
testers, or programmers with testing experience, who could provide more technical
information on the employed processes, tools, testing practices and the day-to-day
operations of the organizations. The third interview round was held by LUT Kouvola's
InnoVire project [41] and they focused on marketing and innovation. They interviewed the
upper management of the organizations. The fourth interview round's targets were the
game designers and people with responsibility with the game design. The goal is to get
information on the creative aspects of the work, namely how games are designed and how
22
their work differs from the more technical-oriented work. The second interview round was
considered the most important for this thesis, since it could provide the most information
on the technical framework the organizations have in their game development.
For the interviews a questionnaire was prepared for each round and mailed to the
organizations. During the interviews this questionnaire was followed loosely, using it as a
template to structure the interview on, rather than a strict guideline to follow. The
questionnaire for interview round 1 is in Appendix I and the questionnaire for interview
round 2 is in Appendix II.
3.5 Studied organizations
There were seven organizations that participated in the study. They are introduced here
briefly without identifying them. The organization size, production size, target- and future
platforms are mentioned as that is relevant information when analyzing the results.
Table 2 shows the size division used in this work for small, medium and large organization.
Table 3 shows a summary of the organizations participating in this study and their size,
target development platforms and possible future platforms.
Table 2: Size categories
Organization/production size category Number of employeesSmall 1-9Medium 10-50Large 50+
The organizations mentioned that they were developing games for the platforms listed
23
under “Development platforms” in table 3 and considered the platforms listed under
“Future platforms” as possible target development platforms sometime in the future. Below
the table, the organizations are described in more detail. It is also worth noting that
company size does not directly dictate production size. Production size indicates the
amount of people working in the organization's game production through outsourcing. The
development platforms are divided roughly to mobile, console, PC and web platforms.
Mobile platforms include Android and iOS devices including both mobile phones and
tablets. PC includes both Windows and Mac operating systems. Consoles include
XBox360 and Playstation 3 platforms. None of the target organizations developed for
Nintendo's platforms (Wii, DS).
Table 3: Organization chart
Organization Organization
Size
Production
size
Development
platforms
Future platforms
Organization A Small Medium Android and iOS WP7, PCOrganization B Small Small Android and iOSOrganization C Medium Large PC and consoles Everything depending
on the market situationOrganization D Medium Medium PC and consolesOrganization E Small Small Android Other mobile platformsOrganization F Medium Medium PC Mobile platformsOrganization G Small Small Web-browsers Mobile platforms
(embedded web
software)
24
4 OBSERVATIONS FROM THE ORGANIZATIONS
This section is divided into different sections based on the division of topics in the
interview forms. First are organization and target platform topics, then process and testing
topics and finally tools. Results on tools are reported in more detail than the other topics.
The interview forms are in appendixes I and II.
4.1 Organization size
Majority of the organizations were small with less than 10 people working in them.
Organizations C and D were the only two clearly medium size organizations and
organization F bordered between small and medium size organization but was considered a
medium size organization for this study as it reflects better their organizational culture and
production size. Most of the organizations were young still developing their first or second
game. Organizations C and D were the only organizations which have established product
lines.
The organization size was not clearly adjoined to the production size. With outsourcing, the
organizations could do one scale larger productions. Organizations' views on outsourcing
are reported in more detail in section 5.4.
4.2 Target platforms
All target platforms were represented among the target organizations. They develop games
for PC, Mac, console platforms (Playstation 3, Xbox 360) and mobile platforms (Android,
iOS). Some smaller market platforms such as Windows Phone 7 (WP7) was left out from
25
the initial release. However, many organizations expressed interest in the platform, because
there does not exist as much competition on the platform yet and the platform has potential
for future releases. Many stated also that if the game turns out to be successful in the
original platforms, they will consider porting it to WP7.
Most of the organizations target their games for multiple platforms limiting the release
only for a platform type. E.g. mobile phones and tablets or PC and consoles. The common
factor for all the organizations was, that they are releasing their games in digital markets
either through Steam on PC or the platform owner's stores in mobile- and console
platforms. The organizations which were currently developing PC games, saw mobile
platforms as a possible development platforms in the future. Generally the organizations
were also willing to expand or change to other platforms if necessary.
26
“Currently, we make games for mobile platforms. We focus on Android at the moment, but we
have been investigating the Windows Phone -area. That Phone 7...it looks, as a system,
interesting, but when one goes to check its application market, then it is obvious that there
might be room for new games...that is someone to publish new games to it, markets.”
-Organization E, Project Manager
“Our goal is platform independence. So, when we're making games for PC, it's Windows, Linux
and Mac. For mobile platforms, it's Android and iPhone as the top ones. And then we have
some Windows Phone 7 trials going on.” -Organization E, Programmer
“We are making games for mobile platforms at the moment, meaning iOS and Android. And
then of course PC, and consoles in theory.” -Organization A, Project Manager
“We have the capacity to develop games for every console platform, and then the most common
tablet devices and others. So we have chosen technologies, that we do not exclude anything. We
have some main platform and main goal that the project aims for but we make decisions so that
we do not exclude anything.” -Organization C, Project Manager
4.3 Development process
The development processes were mostly unorganized but the organizations felt that their
approaches worked well for them. The organizations do not follow any existing processes
strictly, but instead borrow parts from them that they consider suitable for their
organization. The most common methods were agile method, Scrum in particular.
Organizations C and D mentioned to follow the Scrum process and many of the other
organizations stated that they adapt approaches from Scrum such as development in
sprints, daily meetings and keeping the game compilable during the whole development
process. Goal orientation and iterations were common in all of the organizations and it was
also seen as one reason that kept the project from failing.
27
“We use our own version of agile and Scrum.” -Organization A, Project Manager
“We use partly Scrum.” -Organization E, Programmer
“So, if we make prototypes, we check that these are tasks. Then we see, that okay, these and
these need to be done by this date. Then we have milestones. We plan that these and these
features should be done into these milestones and now we'll do this.” -Organization A,
Programmer
“We divide the process into certain milestones and then we have agreed what internal
milestones we have.” -Organization D, Project Manager
“What describes our way of working, is that we set a goal to some date and then we determine
what we should have accomplished at that time. And then we make a rough plan on how we
need to act to accomplish that goal. And then we'll do it.” -Organization G, Project Manager
Roughly the development follows a model where developers set milestones (prerelease,
alpha, beta and commercial version) that they divide into task groups or sprints that have to
be made in a certain time slot, e.g. week or two weeks. The task groups are then divided
into daily tasks. The progress is measured with informal daily progress meetings or in
everyday conversation. Figure 4 illustrates the observed goal oriented process model. In
preproduction, a game design document is created first. The GDD was created with normal
word processors and in some cases Google docs. The latter was chosen because it made
collaboration easy. A concept idea was made based on the GDD. The concept demo type
varied. Playable demo, prototype illustrating the core mechanics of the game and fake
screenshots were reported as concept demos which were used to get funding for the project
or presentations in fairs. The production phase was divided into milestones often alpha,
beta and commercial ones. The alpha milestone goal is to get a version with the core
mechanics in it. The beta version is a version which has a large part of the content finished
and this kind of version was shown to outsiders for testing. Commercial version is the final
release. Each of the milestones is further divided into smaller task groups and they are
further broken down to daily tasks.
4.4 Testing and quality
Testing is a concurrent process with game development in the production phase. Most of
the organizations do not have a dedicated testing plan but they test their game intensively.
28
Figure 4: Goal oriented process model. Adapted from data collected through interviews
PreproductionAlpha Beta Commercial
Post-production/Release
Daily tasks
Production
Conceptdemo
Task groupsGDD
The testing is in most cases usability testing where the developer tests the feature he or she
programmed or the game is given to an outsider to test. Organization E was the only one
following test driven development and writing unit tests for every module. Organization C
reported that they tried test driven development but abandoned it as it made implementing
changes difficult and time consuming, increasing the time it took for the game to change
to the new specifications. Responses to testing goal were similar to the test plan in the
sense that the organizations had no clear definition of what the goal of testing was. There
was variation in the responses even between the individuals interviewed in the same
organization. Generally the opinions fall into two categories; One thinks that the goal of
testing is to test the game design and game balance while the other consider getting rid of
all the fatal bugs as the most important goal. Both categories are represented equally
among the interviewed people.
Quality was important for the organizations but like with testing, most had an approach
that was ad-hoc without any preset definitions of quality. Organizations C and D were the
29
“The idea of testing is to purely break things. But we try to avoid that. We want to test
everything.” -Organization E, Programmer
“The main goal for our game, is to balance it. That was it shortly. Longer explanation is, of
course the entertainment value of the game. But we saw that in the preproduction phase, that
these are the elements that we need, and that this is a good and fun game. But game balance is
the thing we want with testing.” -Organization A, Project Manager
“That the game does not crash. The not crashing is probably the most important fact. That it is
working and there are no critical bugs. That is, in my opinion, the most important thing in
testing. But then of course... is the game fun and things like that. That's a bit difficult to say
based on testing. That comes up from user feedback.” -Organization A, Programmer
“The most important thing to test is game mechanics. And then on close second place,
usability.” -Organization G, Project Manager
only exceptions. Quality was often measured in the companies by the features it has
implemented from the original design and soft values like playability, response, visual
appearance and how fun the game is. These unquantifiable values were gathered by
feedback from developers who tested the game and game test groups. However, not every
target organization used outside testing because of time constraints.
4.5 Outsourcing
Outsourcing is done in limited amounts focusing entirely on game assets rather than any
code. Sounds and music were the most outsourced resource. Organization E mentioned that
they used to outsource sounds and music in the past but it is currently done in-house.
Graphics and 3D objects were the second most common resource to outsource.
Third most common resource to outsource was game engine modules from the engine's
store. The modules were bought outside to save expenses since implementing the module
functionality in the organization could cost ten times as much as the module. Graphics and
3D object libraries were often outsourced for the same reason while in music the reason
was more in the lack of knowledge of that area and there were not enough work to employ
a musician full time in companies. The organizations would outsource more asset
production if they could afford it.
30
“We buy from two sources. We buy music from one source and starting animations from the
other.” -Organization B, Project Manager
“We have used outside help few times for audio things.” -Organization F, Project Manager
4.6 Tools
Tools use d by the organizations vary depending on the development target although there
are some tools that are used in every organization depending on the target development
platform. All organizations use graphics software and the ones working with 3D graphics
use a 3D-modeling software. The selected tools in these two categories were surprisingly
unanimous between all organizations. Graphics software Photoshop was used in all target
organizations for image creation and 3D Studio Max was used for 3D-modeling. Tools
used in preproduction and production phase do not differ greatly, though most of the
organizations do majority of the project documentation in preproduction. Documentation
tools are used in preproduction to create the game design document and share design ideas.
Generally the game design document for the organizations included game idea, game
mechanics, concept art, intended game price and sketches on game menus. Some of the
organizations document more technical information on the document like how modules are
going to be implemented but the main purpose was to list high level plans for the game.
The document was updated through preproduction but often neglected in production phase.
Organization E replied that they update the GDD through the whole project. Cloud services
such as Google Docs and Dropbox were used in sharing the documents. In general, cloud
services were used as project assisting services but they were not used in the actual
development in large scale other than in preproduction.
Concept art, on how the game might look when finished, and fake screenshots were
common in preproduction. Often they were produced to help discussion of the game and to
present the game idea to potential financiers. Organization A said they were using
traditional pen and paper sketching and graphics software when designing the user
interface. All of the organizations developing for mobile platforms were using a
31
“We have a short document where the main features are listed, how is the app purchased and
how much it costs. These things have been written down.” -Organization B, Project Manager
commercial game engine and they used the game engine for mocking the interface often
iterating the final interface from the prototype. Prototyping of the functional features and
game mechanics was also started in preproduction phase. The prototype was often a
concept demo to the financiers to present the game idea. In that sense, it served the same
purpose as the game design document in presentation material. Another purpose for the
prototype was to be the starting point of the development.
In production phase, the tools remained much the same as in preproduction although the
need for documentation tools decreased. Most organizations reported that they neglect
updating documents in production phase but because of the small team size, there is no real
need for it and knowledge sharing is done informally in conversations and emails. Daily
meetings ensure everyone knows what is happening in the project.
Version control systems, ticket systems and file sharing systems became more important in
production phase as project- and work support tools. In implementation, the organizations
using third party game engines, began working more with the game engine tools, while
integrated development environments (IDE) became more important for others, who are
developing their own game engines. Assets such as graphics, sounds and music were added
to the game by programmers and from the interviewees, no one had a dedicated tool for
artists to add their work to the game. However, organization A mentioned creating empty
templates which artists replace later with the proper assets.
32
“We discuss the issues in the morning meeting and then during the day for individual cases.
Which works perhaps because it's not a 50 person team that's making a massive game.”
-Organization A, Project Manager
“No, we have more like...we have this type of enemy now. We make templates for them and
then everything goes basically through design department. We do lots of cooperation with them.
So, what goes to where comes only from the designer.” -Organization A, Programmer
Table 3 summarizes the used tools in preproduction and production phases. The tools are
divided into three main activities; design and prototyping, implementation and project
support. Design and prototyping tools are tools, that are used to design the game and later
in prototyping features. Implementation tools are used to develop the game, including code
and game assets. Project support tools are not directly involved with the end product, but
assist the development. The more specific tool categories are in the cells in priority order.
In preproduction phase, documentation, graphics software and cloud services (e.g. Google
Docs., Dropbox) for sharing information are more important than development tools and
game engines. In production phase, the priority order is inverted, with development tools
becoming the most frequently used programs, followed by content creation software such
as graphics and audio software. Furthermore, audio is in most cases outsourced. In project
support the file sharing is an important aspect especially in preproduction phase. In
production phase the actual code is usually stored in some kind of version control system.
However, some file sharing software is still used in production phase to share individual
files and modules.
Table 4: Tool importance in game development phases
Activity Preproduction ProductionDesign & Prototyping Documentation (e.g. MS Office,
OpenOffice), cloud services (e.g.
Google Docs), graphics software
(e.g. Photoshop), development tools
(e.g. IDEs) & game engine (e.g.
Unity)
Development tools & game
engine, graphics software
Implementation Graphics software, development
tools & game engine, cloud services
Game engine & development
tools, graphics software,
audio software (e.g.
Audacity)Project Support Documentation, cloud services, file
sharing (e.g. Dropbox)
Version control, cloud
services, file sharing, ticket
systems (e.g. Jira)
33
Third party game engines were chosen over creating a game engine, because they make it
possible to focus more on creating the game content, and they reduce the development time
and expenses. The organizations developing for mobile platforms used Unity 3D game
engine. The organizations felt that the game engine made it possible to develop the game
for multiple platforms easily, requiring only little customization to be able to run on
different platforms.
The game engine had its own editor and asset manager, which the organizations found
useful. Another way the interviewees mentioned saving expenses with, was through the
shop that the game engine's developer had created to sell third party plugins for the game
engine. The plugins range from menus and scoreboards to physics engines. The
organizations found these plugins effective in reducing their expenses. Largest expenses in
acquiring software came from the graphics- and 3D modeling software, followed by the
license cost of the game engine. The IDEs, the organizations reported to be using, were in
most cases free.
The organizations were pleased with the tools they were using and did not report any clear
improvement areas. Organization E mentioned that they could use a tool for designing
levels and user interface. At the moment those are done with pen and paper and then the
programmers implement them in the game. In the other organizations, programmers add
the assets into the game as well, however they did not see it as a problem.
34
“We've had good experiences with Unity in our previous project. And it really is multiplatform,
what they promise, and it works pretty well. So, with relatively small changes, we've been able
to develop for iOS and Android.” -Organization B, Programmer
“We could develop within our own team, from our mobile versions, also a PC or console
version with Unity.” -Organization A, Project Manager
4.7 Summary
Most of the organizations are small, with the exceptions of organizations C, D and F. The
organizations develop games for all the major platforms, including mobile platforms, web-
browsers, PC and consoles. The organization developing for mobile platforms use a third
party game engine, with the exception of organization E which uses its own engine. The
reasons for choosing a third party game engine are savings in development times and
expenses and developing for multiple platforms with the same tools.
Outsourcing is used to some degree in most of the organizations. Sounds and music are
outsourced in nearly every target organization. Few of the organizations outsource the
production of graphical assets as well, but many do all the work, except sounds, in-house.
Only organization E reported that they do the entire production in-house and prefer it that
way, although in the future they might consider licensing a third party game engine. Other
organizations considered outsourcing a favorable practice and would outsource more work,
if they had the resources. The reason why sounds and music are not done in-house lies in
the difficulty of finding skilled audio people with sufficient programming skills. The small,
and even medium size, organizations cannot offer enough audio work to employ them full-
time.
Lack of resources was evident in most cases with the organizations. Testing was done in all
organizations, mostly as usability testing by the development teams and few organizations
used outside testing. The testing was started already in production phase and it was nested
into the development process, which was a goal oriented ad-hoc process itself in most
cases. Organizations C and D mentioned they were using Scrum, others said that they are
taking selected practices from Scrum. Despite the ad-hoc approach, the organizations felt
that their processes work for them and that they can test most of the functionality of the
game. Most interviewees said that they would spend even more time into usability testing,
if they had the resources.
35
Tools were largely the same through preproduction and production. Their importance was
simply inverted. In preproduction phase, documentation and cloud service tools were
important for writing the game design and sharing it. Mock screenshots and concept art of
the finished product were produced as well as a game design document which described
the game. In some cases the GDD was also used in presentations to possible financiers.
The documentation lessened dramatically in production phase where the focus was in
development tools. Many organizations said that they do not update the GDD in production
phase, but because of the small teams, there is no need for it. Everyone is aware of what
others are doing. The organizations were also unanimous in their software choices. Every
organization developing a 3D game was using 3D Studio Max and Photoshop was used by
every organization. There were variety in other tools. The organizations were pleased with
their choices and only organization E mentioned they could use a tool for level design.
36
5 DISCUSSION
This chapter focuses on discussion on the results. In the previous chapter, the results were
presented as neutrally as possible. In here they are analyzed. This chapter is divided so that
first is validation subsection, which presents arguments on behalf of the validity of this
work. The actual result analysis subsections start after that. First are organization size and
target platform analysis, then processes, phases and testing. All of these are analyzed by
their impact to tools and how they fit into the technical framework of game development.
5.1 Validation
There are threats that should be acknowledged when addressing the validity of qualitative
research [42]. Reliability and validity in qualitative research are not the same as in
quantitative research. Onwuegbuzie and Leech [42] emphasize that “a qualitative study
cannot be assessed for validity”. Onwuegbuzie and Leech list several threats in qualitative
that have to be addressed. According to them, the most common and severe threat is
personal biasing. The researcher's personal knowledge on the studied subject can affect his
data collection and -analysis. [42] In the study and in this thesis the aforementioned risks
have been taken into account when planning and implementing the study.
The target organizations were small and medium size game organizations, because the
focus of the study was game industry startups. The study was done using the grounded
theory research method, which was chosen, because the research team had used it
successfully in a previous large project which studied testing in organizations [43]. Seven
organizations participated in the study, which is a large enough sample size for qualitative
research method to gather data and certain trends to start to surface. The organizations
were all located in Southeastern Finland which validates the results because they are more
likely to give similar results. However, it is also a threat as some phenomena might have
37
been overseen because of the geographical location. Based on the results, the data began
saturating as same things were mentioned in several interviews and it is reasonable to
assume that no more differing data could be gathered with further interviews or enlarging
the number of participating organizations. Researcher biasing was avoided by having four
people plan the interview forms and at least two researchers were present in most
interviews. Two round 1 interviews were held by a single person. In addition round 3
interviews were held by LUT Kouvola's Innovire project [41]. The questions were
designed to be neutral, as not to steer the interviewee's answers. The interview
questionnaires for the first- and second round are in appendixes I and II respectively. The
data analysis was done by three researchers and the results presented in this thesis were
verified by a group of senior researchers.
5.2 Organization size impact
Target organizations were mostly small with three of them being medium size. The size
was a factor in the development process and documentation. The small organizations used
a more chaotic approach than the medium size ones. The organization size did not affect
target platforms though and all of the major platforms (PC, consoles, mobile platforms)
were represented.
The size of the organization did not reflect the size of the production. Graphics, game
engine, game engine modules and sounds were outsourced in many cases. Table 3 showed
the organization and production size of the organizations and two organizations were in the
middle of a larger scale production than their organization size would imply. This disparity
between organization and production sizes makes it possible to keep the core competence
in the organization and outsource rest of the production. Based on the interviews, many of
the organizations saw this as their strategy. Only one organization of the seven
participating in this study strove to do everything in the organization. But overall, the
attitude towards outsourcing was very positive.
38
The small company size was seen as both a positive and a negative fact. On the other hand,
the organizations were happy with their informal and, to some degree, unstructured
development process but saw the lack of resources as a significant problem, especially
when it came to testing. In most organizations testing was done without specific testing
plan or the testing plan was the same as game design document, with a goal of either
removing as many bugs as possible or balancing the game mechanics. Testing was done
largely in-house and in most of the organizations, not many outsiders were involved in the
testing process. Many organization would like to do more testing by having outsiders test
the game, but feel that they do not have the resources to do so at the moment.
The small organization size and unstructured process also affected the tool selection. The
programmers could, in smaller organizations, choose their own development tools, but
naturally the game's target platform affected the choice. The organizations using game
engine would develop using the game engine tools. The developers in other organizations
used IDEs they preferred.
Generally the tone was “Everyone can work with the tools they are comfortable with.” and
the only limitation from the organizations was that the tool had to enable development for
their target platform. The most similarities were in graphics and 3D-modeling, where every
organization used the same software. Both cases seem to be de-facto standards in Finnish
game industry as no alternatives were even discussed. One of the first round interviewees
mentioned that they tried to get their artists to use an alternative product but that the artists
refused.
The organizations felt that the methods and tools they used worked well for them, but if
they would grow then they would need to define stricter methods. Hiring an entire second
development team was also seen possible to enable working with the same unstructured
39
“Everyone can choose their own tools and it's totally fine as we do not compile the
environment.” -Organization G, Project Manager
process. However, if the two teams were to work within same project, some sort of formal
communication and documentation would have to be established in order for the method to
work. With separate projects the approach might work as is. Project support tools were
limited in small organizations. Development support tools such as version control and file
sharing services are used to support work. However, in many small organizations, project
management tools are not used or their use is discontinued after a short experiment. The
mindset is that everyone knows what other team members are doing so having an extra
management system just adds work that is not directly productive. And in small
organizations this is true as everyone works (in many cases) in the same room and can
discuss at will.
In medium size organizations the development processes were more organized. Tasks, bugs
and other issues were monitored with a ticket system. The used tools reflected the more
structured processes and the development tools were more unified within the organizations.
Developers in the same organization would use the same IDEs and other tools.
Organization C mentioned that the developers can choose their tools by category. When
they find a tool which enables them to do a certain task, they make a requisition for the
tool and then the organization gets it for every employee who needs it. This is not
surprising as with small size, the time spent on management can greatly take away from
productive work and everyone can use the tools they like as possible compatibility
problems can be solved in a reasonable time. In medium size organizations, the number of
developers has gone over the limit where unstructured method works. Common way of
working have to be set to manage the project and minimize compatibility problems. Also
the overhead from management and established process does not translate into same sort of
productive work loss as in small organizations. In medium size organizations, there is a
need for project management tools. The number of people to manage can easily be double
or triple the amount working in a small organization. Managing tasks without a tool suited
for it can become very difficult. More complexity comes from people working from home
and so project management tools are not optional for medium size organizations.
Projects can be maintained without any specific tools, but medium and large organizations
40
have developed processes and have project management tools. Perhaps the largest
difference comes from the development tools. Medium size game development
organizations may give more freedoms to their employees on tool selection than traditional
software organizations.
5.3 Target platforms
The chosen platform affects all organizations on how they work and what tools they are
going to use. Naturally, the platform itself presents some design limitations. Mobile
devices have small displays and are controlled by touch. PC and consoles can have full
high definition resolution so screen space is not an issue. PC and consoles also have
different games by their nature. Mobile devices do not have as powerful hardware and run
on battery. More than the hardware limitations, they are often played little at a time and so
they have to be able to be turned on and off quickly. This raises the importance of game
mechanics over story and graphics. This also showed in the interviews. The organizations
developing for mobile platforms stated that their test goal was to fine tune the game
mechanics.
The organizations developing for PC and consoles were leaning towards the more technical
solution of as bug-free software as possible. Table 5 illustrates this division. There were
seven organizations participating in this study, but one of them developed games for PC,
consoles and mobile platforms and so there are eight organizations in total in table 5. Also,
in mobile platforms there were two organizations whose interviewees had different views.
41
“There are many important things in testing. Of course, it is nice to find all fatal errors. But then
again, it is important, absolutely in my opinion, to ask people outside the development team
what they think about the game. Is it fun? Does it have something really annoying in it?”
-Organization B, Programmer
The project manager and game designer saw the game design as more important while the
second interviewee, a programmer, saw removing bugs as more important. This division is
not surprising given the different focus areas of the interviewees.
Table 5: Test goal relation to platform
Platform Number of organizations Test goal: game design Test goal: no bugsMobile 4 4 2PC, consoles 3 1 2Web 1 1 0
The target platform also affects the used tools however, not as much as game engine. If the
organization is developing their own game engine for the game, then the developers are
freer to use IDE they want although in medium size organizations at least, the tool is
jointly selected by the developers and everyone uses the same tool. If the organization is
using third party game engine for their game, then they are more likely to use the tools that
come with the game engine. Target platform was a central factor when deciding what game
engine and other tools to use. Most of the organizations developed with a third party game
engine mainly because it enabled easy cross-platform deployment.
5.4 Development phases
The observed game development was different from the development model in literature
(see figure 2) in the sense that many activities were done alongside the production phase,
such as testing and prototyping. Of course, the fast iterations cause real prototyping to
blend into normal production, but branches from the main project were also done in order
to test out a feature. Figure 5 shows the game development as observed in the
organizations. It differs from figure 2 in the sense that prototyping also occurs during
production step and testing is strongly tied to production. Test results guide the
development in production phase which in turn guides testing.
42
Game design document and requirements documents are created by all the interviewed
organizations but often they are the same document. Interviewed programmers saw the
game design document as an important document in their work when implementing
features. The design document did not contain much technical information and the
developers had a lot of freedom when implementing the features. In case the document did
not contain all the necessary information, the topic was discussed with a game designer or
other developers until a solution was found.
The interviews gave the impression that preproduction is short in many organizations and it
is used to create the game design document and start prototyping the main features. The
preproduction was also used to create a concept demo to show for possible financiers,
unless the game was self-financed. Tools used in preproduction are much the same as in
production phase. Development tools such as IDEs, other editors and game engine tools
were used for prototyping and creating the concept demo. However, development tools
only play a side role in preproduction. The main goal in preproduction is to plan the
production and test the feasibility of the game idea. Therefore, concept art is created along
with documentation of the game rules and possible high level technical specifications. In
this phase it is crucial to be able to easily share ideas with teammates. The organizations
43
Figure 5: Observed game development
Preproduction Production
Game DesignDocument
Requirements
Creation
Iterations
Postproduction
TestingPrototyping
Same document in manycases.
used cloud services such as Google Docs and Dropbox for sharing files and ideas. All the
organizations felt that the cloud services were really useful in sharing the data and having it
available for every project member. While the use of third party cloud services does raise
some security concerns as the services' end user agreements often give the intellectual
rights of the stored data to the cloud service's owner, these concerns are rather minor
compared to the benefits. The organizations were more wary about sharing source code in
cloud services.
Production phase is the longest and arguably most important phase in game development.
The reported project lengths ranged from three months to few years. It is difficult to see
preproduction take a large part of that although the bigger the project, the more planning it
is going to require. Naturally development tools raise in importance from side role to the
main tool used for work. Graphical work changes from concept art to game assets. And the
importance of cloud services drop. Most prefer to use their own version control for storing
the game assets and source code, although some assets are shared via third party cloud
services. As mentioned before the organizations were wary on putting source code to third
party services. The only exception was a game engine's team sharing option which was
semi-mandatory to use, as the game asset versions do not work properly in normal version
control. The interviewees did not clarify this further, but it could be a problem related to
binary files. Merging binary files of which the version control software knows nothing
about, is not feasible. In practice, the new version always overwrites the old one. In case of
conflicts, the user has to open both files and do the merge manually. If the binary files are a
part of game engine, then the game engine's own tools can do the merging automatically as
they understand the format.
5.5 Testing and quality
Every organization reported to do tests at least to some degree and saw testing as important
activity. In the small organizations the testing activities were more unstructured but
focused on the same aspects as in the medium size organizations. As table 4 shows, game
44
design- and usability testing was more important than bug free implementation. In general,
the usability and game design receives a great deal of attention in game development. This
observed unstructured process and user experience orientation differs from traditional
software development where the majority of the testing follows systematic methods and
the focus of testing is more in functional suitability, reliability and security [44], [45].
The amount of testing resources used for testing did not differ much between the target
organizations or what was observed in traditional (non-games) software development. If
100% testing resources was the perfect amount for testing the product without hurry or
lack of resources but also without having too much of either, the target organizations
reported having from 50% to 90% testing resources available. The median was 75%.
MASTO-project [43] studied testing practices in organizations and the equivalent median
the project got from traditional software development was 70% [44]. Therefore, there
seems to be no meaningful difference in the perceived testing resources between game
development and traditional software development. The most differences come from
testing processes which are more chaotic on game development side and from test goals
where usability is among the top criteria in game development and functionality in
traditional software development.
The used testing tools also reflect the more chaotic side of the game development. Most of
the organizations did not mention to use any specific testing tools. This is not surprising
when the testing goal was game design in many cases. Testing was done largely as
usability testing and few organizations collected metrics from the tests. Organization E was
the only one using test driven development and they were using tools for unit testing and
collecting the results. This differs from traditional software development where testing is
more structured and more tool-assisted although test automation is still low [44].
45
5.6 Summary
Development in small game organizations is unstructured. However, since the team size is
small, everyone can be aware of what others are doing and the project stays manageable.
The team size is not directly related to the production size though, as all the target
organizations use outsourcing to some degree and some would prefer to outsource most of
the game asset production. The outsourced work was most often game engine plugins and
sounds followed by models and graphics. Outsourcing makes it possible even for a small
organization to produce games in reasonable time and still include more man-hours worth
of work than would be possible for the organization alone.
The target platform affected the tool choices of the organizations which is not surprising as
many platforms have only single development environment they work in or can be plugged
in to limited amount of IDEs. The organizations developing for mobile platforms used a
game engine with multiplatform support with the exception or organization E who relied
on the normal SDKs. The ability to develop multiplatform product with one game engine
was one of the primary reasons why the tool was selected.
Production phase is more predominant than preproduction in game development, at least in
the target organizations. The role of preproduction is to develop the game idea, the major
mechanics and initial concept art. In some cases the organization would make a concept
demo of the game to show to publishers. Production phase is the main development phase
but none of the core features such as game mechanics and story are locked in this phase.
The development is very fast paced with large amount of iterations and it is agile, able to
respond to changes quickly.
Testing was seen as important by the organizations. However, the observed testing
practices were chaotic and the focus was in usability testing. This is in large contrast to
traditional software development where the test goal was functionality [45]. Especially the
organizations developing for mobile platforms saw game mechanics as the most important
46
test goal (see table 4). Testing also happens simultaneously with development in
production phase and problems that arise from tests are fixed in the iterations. This reduces
the need of documentation and very few of the target organizations actually document their
test results.
Perhaps the most surprising result was how satisfied the organizations were with their
tools. Literature gives a different view [4], [14], [24]. None of the organizations criticized
the tools they were using and the biggest improvement ideas were to implement ways to
get metrics from the game: Information such as how many times players have died in a
certain spot and how long levels take players to finish. While the organizations did not
comment the tools in detail, some conclusions can be made. Fast iterations and adaptability
to changes are defining elements in game development. Therefore, any tool that is made
for game development has to support these two facts and enable the developers to quickly
test out ideas.
From these observations, it is possible to form the following theory: Organizations choose
their tools based on how well they support fast development cycles and the ability to
quickly change the direction of the project. Fundamentally the selection principle for the
applied tool is based on the personal preferences as long as the selection supports the target
development platform as the organizational aspects, such as project manageability or
controlled maintainability, are not problems for the small game development organizations.
47
6 CONCLUSIONS
Game development in small organizations is unstructured but despite that, it is kept in
control. Conversations, meetings and small team size make it possible to keep track of the
of the project without clearly defined processes. As team size grows, formal processes have
to be adopted. Most of the organizations interviewed for this study recognized this fact.
And the rest of the organizations were already medium size and using controlled processes
such as Scrum in their development. The organization size is not directly related to the size
of the production and organizations can participate in bigger projects than the organization
size would imply and it is quite reasonable, and common, that a small organization is doing
a medium size production. Game organizations outsource game asset production to some
degree and many organizations said that they would like to outsource even more given the
resources.
Game development is fast- and, in small organizations, an unstructured process. The
organizations participating in this study were mostly small and have operated just a couple
of years or less. There were few medium size organizations in the study but they were the
exception. A common trend in the chosen tools and processes the organizations had chosen,
was that they had to support fast iterations and allow quickly changing the project's
direction. This was a fundamental selection principle even though in smaller organizations,
the developers were free to choose their own tools as long as it supported development for
the target platform. In medium size organizations the tool choices were done at
organizational level.
The results of this study can be hopefully used to help new start-up game organizations
develop their processes and find working tools. Further studies on how tool usage and
satisfaction with tools is comparable to traditional software engineering, especially in small
organizations, would be useful in providing insight on whether the results explained in this
work are unique to the game development or do they appear in traditional software
engineering as well.
48
REFERENCES
[1] S. Lawrence Pfleeger ja J. M. Atlee, Software Engineering: Theory and practice, 3. p.
Pearson Education Inc,, 2006.
[2] E. Bethke, Game development and production. Wordware Publishing, Inc.: , 2003.
[3] D. Callele, E. Neufeld, ja K. Schneider, “Requirements engineering and the creative
process in the video game industry”, in Requirements Engineering, 2005. Proceedings.
13th IEEE International Conference on, 2005, ss. 240 – 250.
[4] C. M. Kanode ja H. M. Haddad, “Software Engineering Challenges in Game
Development”, in Information Technology: New Generations, 2009. ITNG ’09. Sixth
International Conference on, 2009, ss. 260 –265.
[5] “Suomen pelitoimialan strategia 2010-2015”. Suomen Pelinkehittäjät ry, February-
2010.
[6] “Essential facts about the computer and video game industry”. Entertainment software
association, 2011.
[7] “Steam Store”. [Online]. Available: http://store.steampowered.com/. [Accessed: 30-
May-2012].
[8] “Origin”. [Online]. Available: http://store.origin.com. [Accessed: 30-May-2012].
[9] “Apple - iTunes”. [Online]. Available: http://www.apple.com/itunes/. [Accessed: 05-
June-2012].
[10] ”Android Apps on Google Play”. [Online]. Available: https://play.google.com/store.
[Accessed: 30-May-2012].
49
[11] “GOG.com”. [Online]. Available: http://www.gog.com/. [Accessed: 30-May-2012].
[12] “EA.com”. [Online]. Available: http://www.ea.com. [Accessed: 30-May-2012].
[13] “Paradox Interactive”. [Online]. Available: http://www.paradoxplaza.com/. [Accessed:
30-May-2012].
[14] F. Petrillo, M. Pimenta, F. Trindade, ja C. Dietrich, “What went wrong? A survey of
problems in game development”, Comput. Entertain., vol. 7, nro. 1, ss. 13:1–13:22,
February 2009.
[15] M. A. Winget ja W. W. Sampson, “Game development documentation and institutional
collection development policy”, in Proceedings of the 11th annual international
ACM/IEEE joint conference on Digital libraries, New York, NY, USA, 2011, ss. 29–
38.
[16] C. Lewis ja J. Whitehead, “The whats and the whys of games and software
engineering”, in Proceedings of the 1st International Workshop on Games and
Software Engineering, New York, NY, USA, 2011, ss. 1–4.
[17] M. J. Nelson ja M. Mateas, “A requirements analysis for videogame design support
tools”, in Proceedings of the 4th International Conference on Foundations of Digital
Games, New York, NY, USA, 2009, ss. 137–144.
[18] E. M. I. Ollila, R. Suomela, ja J. Holopainen, “Using prototypes in early pervasive
game development”, Comput. Entertain., vol. 6, nro. 2, ss. 17:1–17:17, July 2008.
[19] T. Nummenmaa, J. Kuittinen, ja J. Holopainen, “Simulation as a game design tool”, in
Proceedings of the International Conference on Advances in Computer Enterntainment
Technology, New York, NY, USA, 2009, ss. 232–239.
50
[20] F. Petrillo, M. Pimenta, F. Trindade, ja C. Dietrich, “Houston, we have a problem...: a
survey of actual problems in computer games development”, in Proceedings of the
2008 ACM symposium on Applied computing, New York, NY, USA, 2008, ss. 707–711.
[21] A. Kultima ja K. Alha, “‘Hopefully everything I’m doing has to do with innovation’
Games industry professionals on innovation in 2009”, in Games Innovations
Conference (ICE-GIC), 2010 International IEEE Consumer Electronics Society’s,
2010, ss. 1 –8.
[22] M. Peltoniemi, “Life-cycle of the games industry: the specificities of creative
industries”, in Proceedings of the 12th international conference on Entertainment and
media in the ubiquitous era, New York, NY, USA, 2008, ss. 54–58.
[23] R. Potanin, “Forces in play: the business and culture of videogame production”, in
Proceedings of the 3rd International Conference on Fun and Games, New York, NY,
USA, 2010, ss. 135–143.
[24] J. Blow, “Game Development: Harder Than You Think”, Queue, vol. 1, nro. 10, ss.
28–37, February 2004.
[25] “UNITY: Game Development Tool”. [Online]. Available: http://unity3d.com/.
[Accessed: 27-January-2012].
[26] “ShiVa 3D Game engine with development tools”. [Online]. Available:
http://www.stonetrip.com/. [Accessed: 27-January-2012].
[27] D. V. de Macedo ja M. A. Formico Rodrigues, “Experiences with rapid mobile game
development using unity engine”, Comput. Entertain., vol. 9, nro. 3, ss. 14:1–14:12,
March 2011.
[28] “Gamasutra - Features - Postmortem: Arrowhead Game Studios’ Magicka”. [Online].
Available:
51
http://www.gamasutra.com/view/feature/134840/postmortem_arrowhead_game_.php.
[Accessed: 09-March-2012].
[29] “Magicka”. [Online]. Available: http://www.magickagame.com/. [Accessed: 30-May-
2012].
[30] K. Schwaber ja J. Sutherland, “The Scrum Guide”. Scrum.org, October-2011.
[31] “SOCES-project at Lappeenranta Univ. of Tech.” [Online]. Available:
http://www2.it.lut.fi/project/SOCES/. [Accessed: 14-March-2012].
[32] P. Alasuutari, Laadullinen tutkimus 2.0, 4. p. Tampere: Vastapaino, 2011.
[33] T. W. Lee, T. R. Mitchell, ja C. J. Sablynski, “Qualitative Research in Organizational
and Vocational Psychology, 1979–1999”, Journal of Vocational Behavior, vol. 55, nro.
2, ss. 161 – 187, 1999.
[34] S. McDonald, “Studying actions in context: a qualitative shadowing method for
organizational research”, Qualitative Research, vol. 5, nro. 4, ss. 455–473, 2005.
[35] J. W. Creswell, Research Design: Qualitative, Quantitative and Mixed Methods
Approaches, 2. p. SAGE Publications, 2003.
[36] B. Glaser ja A. Strauss, Awareness of Dying. 1965.
[37] B. Glaser ja A. Strauss, The Discovery of Grounded Theory. 1967.
[38] J. C. van Niekerk ja J. D. Roode, “Glaserian and Straussian grounded theory: similar
or completely different?”, in Proceedings of the 2009 Annual Research Conference of
the South African Institute of Computer Scientists and Information Technologists, New
York, NY, USA, 2009, ss. 96–103.
52
[39] S. Adolph, W. Hall, ja P. Kruchten, “A methodological leg to stand on: lessons learned
using grounded theory to study software development”, in Proceedings of the 2008
conference of the center for advanced studies on collaborative research: meeting of
minds, New York, NY, USA, 2008, ss. 13:166–13:178.
[40] A. Strauss ja J. Corbin, Basics of Qualitative Research, 3. p. SAGE Publications,
2008.
[41] “InnoVire / LUT Kouvola”. [Online]. Available:
http://www.kouvola.lut.fi/en/research/innovire. [Accessed: 10-April-2012].
[42] A. Onwuegbuzie ja N. Leech, “Validity and Qualitative Research: An Oxymoron?”,
Quality & Quantity, vol. 41, nro. 2, ss. 233–249, 2007.
[43] “MASTO-project at Lappeenranta Univ. of Tech.” [Online]. Available:
http://www2.it.lut.fi/project/MASTO/. [Accessed: 22-May-2012].
[44] J. Kasurinen, O. Taipale, ja K. Smolander, “Software Test Automation in Practice:
Empirical Observations”, Advances in Software Engineering, Special Issue on
Software Test Automation, Hindawi Publishing Co., 2010.
[45] J. Kasurinen, O. Taipale, J. Vanhanen, ja K. Smolander, “Exploring Perceived Quality
in Software Organizations”, in Proceedings of the Fifth IEEE International Conference
on Research Challenges in Information Science, Guadeloupe, France, 2011.
53
APPENDIX IRound 1 Interviews: Team leaders / Project managers
Topic 1: Key figuresQ1: In your terms, explain what you do in your organization and how your work is related
to game development.
Q2: How large is the organization you are working in? How many people work on this
organization? How many people work in the company? How many people, including those
from outsourced or outside organizations, contribute to your product?
Q3: What kind of games do you develop? Large-scale games (i.e those requiring
specialized game consoles) or smaller-scale games (i.e. those that can be played through
the internet or with mobile systems) and what is your target market? PC market, Internet,
consoles, or other?
Topic 2: Development process and CreativityQ1: How do you decide on what product to make next? Who are involved in the creative
process? How do you design the game (for example, by prototyping, mockups etc)? Who
decides on the themes, rules or concepts included to the game?
Q2: How does your organization develop a new software product like full game or
expansion disc/DLC, What are the phases of this process? Do they differ between different
types of products? How long does it take to develop one major product in your
organization? Please describe your development process in a nutshell.
Q3: ISO/IEC 29110 defines the following activities as a part of the project management
(see Figure 1):
• Project planning; requirements and plans are created to guide the work
• Project plan execution; product is developed as planned
• Project assessment and control; the execution is assessed and controlled to steer the
work towards preferred results.
• Project closure; product is delievered, needs for changes assessed, asset and tool
reuse is decided upon.
How does your organization reflect to these activities? How much in your development is
planned beforehand, how much does your project plan change during the development
process? How do you address the change if it occurs?
Q4: ISO/IEC 29110 defines the following phases for software development (see Figure 1):
• Software Implementation Initiation: Resources are reserved
• Software Requirements Analysis: What needs to be done is decided
• Software Architectural and Detailed design: The product is designed
• Software Construction: The product is built.
• Software Integration and Tests: The product is tested and finalized
• Product delivery: The product is released to customers/publisher
How does your organization reflect to these phases? Does your development method
follow some certain model? In general, how formal is your development process?
Q5: Name two strengths and two possible weaknesses in your development. Why are these
things strengths or weaknesses?
Topic 3: Test processQ1: How is your software tested? What activities are done to test your software? Who
decides when the product is tested enough? Please describe the test process of a typical
software product in your organization.
Q2: What is the main objective of your testing work? (Find the most important bugs or
release a non-broken product or something else?)
Q3: How large percentage of your testing work is planned before actual testing work? Do
you do explorative testing? In your opinion, does this test plan work?
Topic 4: QualityQ1: How do you decide on the required quality of your software? How do you
communicate this quality requirement to other stakeholders? In your opinion, does this
approach work?
Q2: Does the required (or intended) quality change during the project? Why?
Topic 5: Process developmentQ1: What changes are you currently experiencing and what is causing them? ( e.g. growth
of your business, product changes, changes with the customer/target market, analysis of
project feedback data or any other such reasons) Has the change been sudden or anticipated
(i.e. managed/unmanaged?) In your opinion, is this change good, bad, neutral?
Q2: What are the current trends in the game industry that affect your work, and how? How
are you addressing them?
Q3: Do you do process development? Do you collect metrics on your process? What
metrics/why not?
Q4: Have you used some formal process development method (SPICE, TMMi, CMMi, TPI
etc…)? What would it require for your organization to do formal process development (or
certification program or comply with a standard)?
Topic 6: OutsourcingQ1: Do you do outsourcing? What kind of assets or resources do you acquire from outside?
How large percentage of your game is developed primarily “in-house”?
Topic 7: Knowledge management, Tools and ServicesQ1: What software tools do you use in your development and testing? In your opinion, are
they good enough or is there something that could be better? Is there any tool or service,
which you currently would like to have but are unable to acquire? How or why did you
choose this tool?
Q2: Name up to three most important abilities required for game development project.
Where can these abilities be learned or acquired?
Q3: Is cloud computing in any way applicable for game development in your organization?
If yes, how are you utilizing it? What are the advantages and disadvantages? If not, why
not?
Q3b: What aspects of your work (development and/or delivery of gaming
products/services) are/would be most affected by cloud computing? How?
Q4: Do the terms “gaming-as-a-service” and “gaming-on-demand” mean anything to you?
Please describe.
Topic 8: OtherQ1: In your opinion, is your work creative work which also has software development
components or software development work with creative components? Why?
Q2: If you had to choose, which in your opinion is more important, to make a financially
successful, or high quality game? Why?
Q3: Is there something you would like to add to this interview? Something you think is
relevant but was not asked, or something that needs to be emphasized?
Figure 1
ISO/IEC 29110 Software Engineering –Lifecycle profiles for Very Small Entities
Project management
Project planning
Project plan execution
Project assessment and control
Project closure
Software implementation
Implementation initiation
Requirement analysis
Component identification Construction Integration
and tests Delievery
APPENDIX IISOCES – SOFTWARE DEVELOPMENT IN CREATIVE ECOSYSTEMSRound 2 Interviews: Programmers and testers
Topic 1: Key figuresQ1: In your terms, explain what you do in your organization and how your work is related
to game development.
Q2: How large is the organization you are working in? How many people work in your
project? How many people directly affect your work, how many are affected by your
work?
Q3: What is the target platform of your products?
Topic 2: Development and ISO/IEC 29110Q1: How does your organization develop a new software product like full game or
expansion disc/DLC, and what are your activities in this process? Do they differ between
different projects? How long does it take to develop and test one major product?
Q2: How do you decide on what to do next? Who are involved in these decisions? In your
opinion, does this approach work?
Q3: Do you participate on the game design process, or have the ability to affect the
decisions made on the game design? Overall, how large effect the design documensts have
on your work?
Q4: ISO/IEC 29110 (see last page) defines the following activities as a part of the project
management:
• Project planning; requirements and plans are created to guide the work
• Project plan execution; product is developed as planned
• Project assessment and control; the execution is assessed and controlled to steer the
work towards preferred results.
• Project closure; product is delivered, needs for changes assessed, asset and tool
reuse is decided upon.
ISO/IEC 29110 defines the following phases for software development:
• Software Implementation Initiation
• Software Requirements Analysis
• Software Architectural and Detailed design
• Software Construction
• Software Integration and Tests
• Product delivery
In your opinion, how does your organization reflect to these phases? In general, how
formal is your development process? Would you change something in your current
development process?
Q5: Name up to three strengths and three possible weaknesses of your organization. Why
are these things strengths or weaknesses?
Topic 3: Outsourcing
Q1: Do you use outsourced resources? What kind of assets or resources do you work with?
In your opinion, do these assets affect your work?
Topic 4: Test processQ1: How is your game tested? What activities do you do to test your software? Who
decides on test objectives (i.e. when the product is tested enough)? In your opinion, is the
applied approach working? Why?
Q2: In your opinion, do you have enough resources (time/people/etc.) to develop and test
your product to a satisfactory degree? In your opinion, how large portion of the “optimal”
amount of resources do you have in development and in testing tasks?
Q3: How large percentage of testing work is based on the plans or documentation? Do you
do explorative testing? Does this test plan work/cover all the important parts that should be
tested?
Q4: What you consider to be the main objective of your testing work? (Find the most
important bugs vs. release a non-broken product)
Topic 5: QualityQ1: How do you decide on the required quality of your software? How is this requirement
communicated in your organization? In your opinion, does this approach work?
Q2: Does the required (or intended) quality change during the project? Why?
Q3: Are there tasks that are required to be done by some external stakeholder such as
publisher? If yes, what activities do these tasks include?
Q4: Name up to three changes in your development or test processes which would enable
better quality or otherwise improve the organization.
Topic 6: ToolsKnowledge Transfer
Q1: How do you share information? E.g. Do you use wikis? Email? Post-it notes?
Q2: What kind of information is shared? E.g. Technical specifications, Game Design
Document etc.? In your opinion, do you get enough data to work with on development or
testing tasks? What sort of data do you find most useful in your tasks?
Q3: How is communication between the team members done? Do you have instant
messaging tools for it e.g. Messenger, Google Talk, Skype etc.? How much information is
shared just by talking with people, compared to using some messaging tool?
Preproduction
Q1: Describe the tools you use in preproduction phase. For example, do you generate game
design document? Do you make prototypes and if so, what kind (paper/software)? How do
you consider these designs or prototypes affect the production most (quality-wise,
effiency-wise etc.)?
Q2: Do you feel that the tools you use support innovation or creativity? Is it easy to test
ideas with the tools you use?
Development
Q1: Describe the tools you use development phase to make the actual game. Do you reuse
material from preproduction (e.g. Software prototypes)?
Q2: Do you have any sort of asset manager and if so, what kind?
Q3: How is content added to the game? Are there utilities for writers and artists to add
content or is it solely up to the programmers?
Q4: Is cloud computing in any way applicable for game development in your organization?
If yes, how are you utilizing it? What are the advantages and disadvantages? What aspects
of your work are most affected by cloud computing?
Q5: Do the terms “gaming-as-a-service” and “gaming-on-demand” mean anything to you?
Please describe.
Work support and management
Q1: What kind of work support tools do you use? How do you keep track of tasks? If there
is a need for a change in the product, how is this documented and communicated?
Q2: Do you collect metrics on your process? What metrics/why not? Do you use any
benchmarking programs in your development? Do you get information on performance
spikes or overall data? How useful benchmarking programs are in optimizing the game
performance?
Q3: How do you track project milestones? Do you use any tool for that?
Closing questions about tools
Q1: Name two good features of the tools you're using. Is there any room for
improvements?
Topic 7: OtherQ1: Name up to three most important abilities required for game developer. Where can
these abilities be learned or acquired?
Q2: In your opinion, is your work creative work which also has software development
components or software development work with creative components? Why?
Q3: If you had to choose, which in your opinion is more important, to make a financially
successful, or high quality game? Why?
Q4: Is there something you would like to add to this interview? Something you think is
relevant but was not asked, or something that needs to be emphasized?
ISO/IEC 29110 Software Engineering –Lifecycle profiles for Very Small Entities
Project management
Project planning
Project plan execution
Project assessment and control
Project closure
Software implementation
Implementation initiation
Requirement analysis
Component identification Construction Integration
and tests Delievery