Bachelor Informatica

Virtual Reality in education

Tessa Klunder

June 17, 2015

Supervisors: Robert Belleman (UvA)

Signed: Robert Belleman (UvA)

ComputerScience—

UniversityofAmst

erdam

2

Abstract

Computer science and virtual reality are developing strongly, yet their current application ineducation is limited. The increasing influence of technology requires children to learn moreabout computers and programming. For this thesis a game has been developed for the OculusRift to research the effect of virtual reality on children. It also provides a way for schools toteach children the basics of programming in a three-dimensional environment. The results showthat children have a great interest in virtual reality and are eager to learn more about the virtualworlds. Difficulties such as motion sickness and eyes train deserve careful thought, but the usertests proved that these can be overcome. Virtual reality has many possibilities, but the step tointegrating it in the educational program has yet to be taken.

2

Contents

1 Introduction 5

2 Background and related work 72.1 Computer Science in the Netherlands . . . . . . . . . . . . . . . . . . . . . . . . . 72.2 Games in education . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82.3 Virtual reality in education . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

3 Implementation 113.1 The game . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

3.1.1 Programming the car . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113.1.2 Mini-map . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123.1.3 Levels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

3.2 Design choices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133.2.1 Three-dimensional experience . . . . . . . . . . . . . . . . . . . . . . . . . 133.2.2 Motion sickness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143.2.3 Reading of text . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143.2.4 User Interaction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

3.3 Technical implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153.3.1 Unreal Engine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153.3.2 Oculus Rift . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

4 Experiments 174.1 Beta test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174.2 Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

4.2.1 Test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

4.3.1 Controls . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194.3.2 Motion sickness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194.3.3 Difficulty level . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194.3.4 Times and number of attempts . . . . . . . . . . . . . . . . . . . . . . . . 194.3.5 Question form . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214.3.6 User reactions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

5 Conclusion 235.1 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235.2 Future Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

A Experiment form 27

B Experiment results 31B.1 Time results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31B.2 Attempts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32B.3 Form results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

3

4

CHAPTER 1

Introduction

Computer science has become an essential part of society, yet the educational program at highschools in the Netherlands has hardly changed since it was introduced in 1998[1, 11]. This isin stark contrast to the rapid innovations in the professional field. These innovations result ina high demand for well educated computer scientists, which currently cannot be satisfied1. Toovercome this shortage more children need to be educated in the field of computer science, sofar not many children choose this education[11]. This is partially due to the lack of availablecomputer science courses, but also because of the differences between the educational programand the interests of children[11]. Therefore it is important to find new ways to interest childrenin computer science.

An increasingly popular approach is the use of games to educate children. Many companieshave developed games to teach them about physics, math, languages and there are also plenty ofgames that teach the basics of programming. Examples are MIT’s Scratch2, Google’s Blockly3

and the Flash game Lightbot4. However these games are all two-dimensional and don’t takeadvantage of a vast growing field within computer science. Over the past few years virtualreality has revived5 and the large companies such as Facebook, Samsung, Sony and Google areeagerly investing in new technologies for virtual reality headsets.

In this study the possibilities of virtual reality games in education and the experience ofchildren inside virtual reality will be examined. It will not directly compare two-dimensionalgames with three-dimensional versions, but rather look at possible applications of these three-dimensional games in education. This is done by developing a game for the Oculus Rift, a virtualreality head-mounted display, which teaches children the basics of programming inside a virtualenvironment. The main research question is:

How can Virtual Reality contribute to computer science education in the Netherlands?

With this question in mind, the project has three main goals:

• Teaching children about programming

• Exploring the possibilities of virtual reality in education

• Providing a new way to introduce computer science to children

The next chapter will provide a brief summary of the current state of computer scienceeducation in the Netherlands and projects that have been started to improve Dutch education.Chapter 3 will describe the features of the game and its concept. Also touching on the possibleeducational application. In chapter 4 the user tests and the results of these tests will be discussed.

1http://www.ictmarktmonitor.nl/english-summary/#212https://scratch.mit.edu/3https://blockly-games.appspot.com/4http://lightbot.com/5http://eandt.theiet.org/magazine/2015/02/virtual-reality-revival.cfm

5

6

CHAPTER 2

Background and related work

2.1 Computer Science in the Netherlands

Computer science education is offered at only 55 percent of the high schools in the Netherlandsand only 12 percent of the students has chosen this course[11]. Fortunately there are a number ofparties who have started projects to change this and improve computer science in the Netherlands.

In 2012 the Royal Netherlands Academy of Arts and Sciences (Dutch: Koninklijke Neder-landse Akademie van Wetenschappen) published a report with the recommendation to completelyreform the course computer science at high schools[1]. The Dutch Ministry of Education, Cul-ture and Science took the advice and requested the national centre of expertise SLO to developa new plan for computer science education. Currently SLO has a committee developing a newcurriculum which will be offered for review in the fall of 2015.

Next to the changes in the national curriculum, there are projects to help startup companies.One of these companies is StartupDelta1, an initiative from the Dutch government to help star-tups grow quickly. Together with former European Commissioner Neelie Kroes they encouragestartups by bringing them in contact with larger businesses and providing the necessary fundsto help them on their way. One of their ambitions is to learn more children how to code. Shebelieves that programming is the reading and writing of the future[8]. She wants to achieve thiswith the national CodePact2. In this CodePact trade and industry join forces with the Dutchgovernment to teach 400.000 children how to program. The primary target audience are childrenfrom the last grade of primary school and the first grade of high school. The aim is to providechildren with the option to learn how to code and in the next summer a concrete plan will bedevised on how to achieve this.

But why is it important to integrate computer science in education? Computers are becomingessential in all professional fields and require the users to be able to use them. But in order totake this even further, users not only need to understand how to use them, but also how theywork. One of the approaches is by learning how to program. Not only does this provide a betterunderstanding of computers, it also teaches a number of skills. Studies have showed that childrenwho learn to program, develop an extra set of qualities[11, 2]:

• Logical thinking: thinking in steps and how to create a structured program.

• Problem solving: completely breaking a problem down and then finding a solution thatsolves all pieces.

• Creative thinking: freedom over what and how to create.

• Group work: working together on projects.

1http://www.startupdelta.org/2http://codepact.org/

7

2.2 Games in education

Often when speaking of video games its about the popular AAA games such as Grand TheftAuto, FIFA and Call of Duty. These games are purely focused on entertainment. But there isalso an entire gaming world dedicated to the so called serious games. These games are designedto train or educate its users and are used in areas such as scientific research, military defence, themedical world and education. Studies such as Chuang and Chen’s[4] show that games improvelearning capabilities of children. Chuang and Chen concluded from their research that “computer-based video game playing not only improves participants fact/recall processes, but also promotesproblem-solving skills by recognising multiple solutions for problems.”

Fortunately games that teach children to code is not a new idea in the world of game devel-opment. In 2003 the Lifelong Kindergarten Group at the MIT Media Lab developed the toolScratch3 (see Figure 2.1a). This program allows children to click and drag blocks together toform a program. A similar idea comes from Google with their game series Blockly Games4 (seeFigure 2.1b).

(a) Scratch. (b) Blockly.

Figure 2.1: Blockly and Scratch are both snap-together block languages.

There are also games who use grids and predefined moves to create a program. One of thesegames is Lighbot, which has become very popular on mobile devices. It teaches children toprogram a robot and give it a series of commands as can be seen in Figure 2.2. When satisfiedwith the created program, the children can execute it and the robot will perform all the assignedtasks.

Figure 2.2: Lightbot implements a grid and predefined commands for creating programs.

3https://scratch.mit.edu/4https://blockly-games.appspot.com/

8

2.3 Virtual reality in education

“Virtual reality is the computer-generated simulation of a three-dimensional image or environ-ment that can be interacted with in a seemingly real or physical way by a person using specialelectronic equipment, such as a helmet with a screen inside or gloves fitted with sensors.”5

Virtual reality has been around for a long time, yet only recently when Facebook acquiredOculus VR, has virtual reality revived. Every day new games and demos are being developedand the major companies have turned their interest towards virtual reality. The major playersbeing:

• Oculus VR with the Oculus Rift6.

• Google with Google Cardboard7.

• Sony with Project Morpheus8 for the PlayStation 4.

• Samsung with the Samsung Gear VR9 for mobile devices.

These are however mostly used for entertainment purposes and the development communityis still discovering the educational value that virtual reality can bring. Fortunately multipleresearchers have been investigating this and have published a number of papers regarding thesubject[3, 5, 6, 7, 9, 10]. For example a group of researchers have investigated the effect of three-dimensional games on the learning capability of children and compared this to two-dimensionalgames. At least three of them concluded that pupils learnt science topics better in 3D thanin 2D [3, 6, 7]. One of these studies developed a virtual reality game placed in ancient Egypt,called the Gates of Horus[6]. Children were able to explore and interact with an old Egyptiantemple. The goal was to reach the inner sanctuary and unlock the final mystery. Afterwards thechildren had to give a presentation about what they had learnt and seen. Expert evaluators thenestablished that the children who used the virtual environment developed a better understandingof the material than the group of children who had used the desktop to explore the temple.

5Source: https://goo.gl/BEMEHz6https://www.oculus.com/en-us/7https://www.google.com/get/cardboard/8https://www.playstation.com/en-gb/explore/ps4/features/project-morpheus/9http://www.samsung.com/global/microsite/gearvr/

9

10

CHAPTER 3

Implementation

As discussed in the previous chapter, games are an effective way for children to learn and under-stand complicated material. But how can virtual reality contribute to this? To learn more aboutvirtual reality in education, a game was developed for the Oculus Rift. In this game childrenlearn the basics of programming by writing programs that solve different mazes.

3.1 The game

The original idea for the game came from Google with their game series Blockly1. In this gamethe player has to create a program that leads the character out of a maze. However Blockly usesan aerial view of the maze, whereas this game places the player inside the maze. In the mazeare seated in a car that can be programmed. When the player is content, the program can berun and the car will execute all the commands in the program.

3.1.1 Programming the car

In order to program the car, there are a series of predefined commands that can be used. Thesemoves can be added to the program by selecting them on the “control screen” as can be seen inFigure 3.1. This screen is in front of the player when the car is not moving. It consists of a gridof square slots, that can contain commands. When clicking on one of the commands, they getinserted into the program from left to right. When the player executes the program, the screenwill disappear and the car will start executing the commands from left to right. The moves thatare available are:

• Move Forward: moves the player one step forward in the direction of the car’s forwardorientation.

• Rotate Left and Right: rotates the car 90 degrees in the specified direction.

• Loop: when encountered, the program will return to the start and continue from there.This way all commands before the Loop will be repeated until the finish has been reached orthe player stops the program. Consequently moves behind the loop will never be executed.

• If-statements: provide a way for the player to check possible paths. In this game thereare three predefined If-statements. The first one checks if there is a path in front of theplayer and the other two provide the options to check left and right of the car. With thesecommands a player can ultimately implement a wall follower algorithm2 that can guide theplayer out of the maze. An If-statement needs to be ended with a closing bracket. All themoves between the statement and the closing bracket will only be executed if the conditionis true.

1https://blockly-games.appspot.com/2https://en.wikipedia.org/wiki/Maze_solving_algorithm#Wall_follower

11

Figure 3.1: The controlscreen where the player can construct the program.

There were a lot of design choices that had to be made in order to create intuitive controls.Lightbot3 provided the biggest inspiration for the controlscreen. Its grid based approach providedthe solution for creating programs in a virtual world with a game controller. However it resultedin some problems for loops and if-statements. In this game loops are only effective at the endof the program. When encountering a loop, the car will go back to the beginning, ignoringall moves that might come after the loop. It is also not possible to break early from a loop.The if-statements were the most difficult problem to solve. How do you incorporate if-then-elseconstructions in a two-dimensional grid? And were do you store the condition rule? The solutionwas the creation of three fixed if-statements. The player could only choose these three: check ifa path is in front, left or right of the car. If that is the case, the moves between the if-block andthe closing bracket are executed. Otherwise the program will search for the closing bracket andcontinue from there.

3.1.2 Mini-map

To help the player a mini-map of the maze is provided. On this map the player can easily see theshape of the maze and the location of the finish. A blue grid shows the path to the finish andthe number of steps required. This was implemented to make it easier for the player to write aprogram, without first having to discover the entire shape of the maze. Next to the minimap isthe control mapping visible. This enables the user to check the functions of the different buttons.Both the minimap and the control mapping are always visible during the execution of programs.

Figure 3.2: The mini-map provides a top down view of the maze.

3http://lightbot.com/

12

3.1.3 Levels

The current prototype of the game consists of a series of five levels. These levels have anincreasing difficulty level and are made to show the possibilities of the game. The first level is tointroduce the player to the game and the controls. The path is a straight line and requires onlythree moves forward. The second level is a little tougher and requires the use of rotations to getaround the corner. In the following three levels, loops and if-statements are introduced. Themeshes and textures used in the levels are a part of the Stylized Rendering4 package providedby Unreal Engine 4.

(a) Level 1 introduces the controls. (b) Level 2 introduces rotations.

(c) Level 3 introduces the loop.(d) Level 4 requires a combination of the newlylearnt material.

Figure 3.3: The first four levels of the game with their minimap.

3.2 Design choices

Developing a game for a three-dimensional virtual environment brings a couple of new challengescompared to a two-dimensional game. Particularly the user interaction with the game and motionsickness are problems that were encountered. In this section various design choices are discussedand the motives behind them explained.

3.2.1 Three-dimensional experience

The original design for the game had the player floating behind a robot that could be pro-grammed. The robot would walk through the maze, while the player observed from a distance.This however would not provide an optimal three-dimensional experience, because the playerwould not fully become a part of the environment. To engage the player more, the decisionwas made to set the player inside a car which had to be programmed. Then after running thecreated program, the player would experience directly what the consequences of the programwere. On top of that the player could now move through the three-dimensional world and enjoythe scenery as can be seen in Figure 3.4.

4https://docs.unrealengine.com/latest/INT/Resources/Showcases/Stylized/index.html

13

Figure 3.4: Level 5 introduces If-statements.

3.2.2 Motion sickness

In early versions of the game, motion sickness was a serious problem. When moving throughthe maze with the car, players would quickly become nauseous and experience headaches. Todecrease this a couple of changes were made. The first improvement was to lower the speed withwhich the car would rotate and move through the maze. With this lowered speed players couldadjust more to the environment and become less nauseous. Second the moves that would getexecuted were displayed in front of the car. The user can see what move will be executed nextand prepare for the movement changes. A third improvement was changing the environmentitself. The large straight walls were replaced by bushes, which resulted in more open space anda reduced claustrophobic feeling. Also the asymmetric bushes provided a better sense of depth.

3.2.3 Reading of text

In the current version of the Oculus Rift (Development Kit 2), the ability to read text is notoptimal. Text has to be very large before it can be read and it quickly becomes blurry whenlooking around. This meant that the use of text had to be limited and that written instructionshad to be replaced by icons or animations. For this reason large icons and buttons were chosento represent the moves instead of smaller text blocks as used in Scratch or Blockly.

3.2.4 User Interaction

Letting the player interact with the game in virtual reality also proved to be a difficult obstacle.The standard interaction with computers is using a mouse and keyboard, yet in a virtual world,these input devices are impractical. The user has a display right before their eyes, which blocksthe visibility of the “real” world, thus leaving players blind and unable to perform tasks whichrequire movement of their hands. Besides the limitations in the physical world, a two-dimensionalmouse cursor is very ineffective in a virtual three-dimensional world. How do you display a twodimensional object appropriately? What is the right distance of the object to the camera? Andwhen does a cursor intersect with a button or field in a menu? To solve these problems a gamecontroller was used to allow player interaction. With a controller in hand the user does notrequire to move their hands and can easily access a large number of buttons and controls.

14

Figure 3.5: The functions of the buttons on an Xbox 360 controller5.

As shown in Figure 3.6 the controls were implemented to be as minimalistic as possible. Sochildren who do not have any experience with game controllers can also quickly start playing.Navigation through the move buttons is possible with the gamepad stick or the d-pad buttons.These allow the user to focus on a move and add it to the program. Subsequent to adding, amove can also be removed from the program.

3.3 Technical implementation

3.3.1 Unreal Engine

The game was developed in Unreal Engine 4.76. This is a very popular game engine, developedby Epic Games that was primarily used for first-person shooters. But as new features areconstantly being added, the engine increasingly lends itself for many more game genres. Alsothe ability to port games to a large set of platforms and the support for VR devices resultedin the choice for this engine. The game is currently compatible with Windows 7 and 8. UnrealEngine is written in C++ and thus provides the possibility to write the game directly in C++code. This is possible with an excellent integration in Visual Studio 2013. For the designersand non-programmers there is also the option to use Blueprints. This is the Visual Scriptingsystem that Epic Games designed to make the game engine accessible to people who do not haveexperience with programming. By using blocks and visual elements users are able to generatethe same code as they would have been able to do directly in Visual Studio. This project wasprimarily done with Blueprints. Most of the programming work went into the car itself. Thegame has been constructed such that the car holds all the intelligence. When placing the car inany environment, it can always execute the created programs. This allows for a lot of freedomin level creation, with the sole condition that all the walls of the maze have proper the rightsettings.

5http://www.xbox.com/xbox-3606https://www.unrealengine.com/

15

Figure 3.6: The event that handles player input in a Blueprint in Unreal Engine 4.7.

3.3.2 Oculus Rift

The key feature of the game is its compatibility with the Oculus Rift Development Kit 2 (DK2).This is a virtual reality head-mounted display developed by Oculus VR7. Oculus VR was recentlyacquired by Facebook which provides them with a larger budget and more publicity. It also senta signal to other players that virtual reality is a serious field that will continue to grow in thefuture. The DK2 is the most recent version of the Rift series which has a Full HD OLED displayand a camera that provides positional head tracking. Next to the Oculus Rift it is also possible toexport the game to Android. This greatly improves the applicability of the game. Schools can letthe children play the game on an Android device with a Google Cardboard. The interaction canthen be controlled with an OUYA controller8. This setup eliminates the need for an expensivegame computer and the purchase of an Oculus Rift. Of course there will be a difference inperformance, since a smart phone does not match the graphic capabilities of a high end gamingPC.

(a) Oculus Rift DK2. (b) Google Cardboard9.

Figure 3.7: The VR devices compatible with the game.

7https://www.oculus.com/8https://www.ouya.tv/9http://goo.gl/HxTL0B

16

CHAPTER 4

Experiments

To determine the possible application of the game in education, it was tested by a number ofchildren. The main priority of the test was to establish the user friendliness of the game. Howwell do the controls work and do the children experience motion sickness? And how do thechildren respond to the three-dimensional environment itself? Next to these practical matters,the goal is also to find out if children are interested in the game. Does this game spark theirinterest and trigger their curiosity? Or do they lose interest after five minutes of playing?

4.1 Beta test

Before the larger experiment, the game was beta tested by two children (9 and 11 years old).The main goals were to find possible bugs, determine the difficulty level and the overall gameexperience of the children. It was also used as a test run for the upcoming experiment, to findout how long it takes for them to complete the current game and where they required assistance.They played the game twice in order to see if they had fully understood the possible moves andto see how quickly they would finish the game after becoming familiar with the controls. In thefirst run they would not get any guidance. This was done to see how intuitive the game was byitself. The first reactions to the game were very positive. They enjoyed the three-dimensionalworld and the first couple of minutes were spent gazing around. After completing the game forthe first time, they immediately asked if they could play it again. Besides this positive reception,there were also a number of problems they encountered:

• Control difficulties: the main problem was understanding the functions of the variousbuttons. They would often forget which button to press, leading to unwanted additionor deletion of moves. Important to note is that these children did not have any priorexperience with these game controllers.

• If-statement: they did not understand the use of an if-statement and also got confusedby the icons that were on the buttons.

• Eyestrain: after playing for half an hour they indicated that their eyes had becomesomewhat tired.

It also showed that they needed a lot of guidance to help them understand the use of thebuttons and how to create a program that brings them to the exit. Primarily the control issueslimited the gameflow and they spent more time figuring out how to add or remove a move thanactually programming the car. Following their test, the game was modified to remove some ofthose difficulties. The controls were simplified and the colors in the controlscreen adjusted toincrease the visibility of the selected button. Also the icons of the If-statements were adjusted,such that they were less complicated and more intuitive.

17

4.2 Method

After the beta test the real experiment could start. This experiment consisted of two parts ondifferent locations. The first part was at the University day at the University of Amsterdam.At this event the children had various workshops in which they could participate. One of thembeing the possibility to play the game in the Oculus Rift. These children were between 7 and13 years old, most of them boys. The second half of the experiment was at the DigiVita event,organised by VHTO1. These were all girls between the ages of 8 and 14 years old.

4.2.1 Test

The test itself consists of three parts:

1. Playing the game.

2. Answering a number of questions regarding the experience (see Appendix A).

3. Solving two additional mazes on paper (see Appendix A).

For each child a number of statistics is recorded, such as sex, age, current school year andwhether the child wears glasses. The game itself acquires several gameplay statistics, such asrecording the time it takes for a player to complete each level and number of attempts per level.The latter meaning the number of programs that the player runs trying to reach the finish. Italso creates a unique file name and player ID so the acquired data can be matched to the rightchild.

A second method used is the recording of the entire experiment. This is done with a webcamdirected at the player and a recording of the game itself. This way all reactions to the game canbe observed and compared to the other children.

4.3 Results

During the experiments n total 22 children played the game. Each playthrough took around 5to 8 minutes, with an aditional 5 minutes for the questions and test mazes.

Figure 4.1: Kruse, M. (Photographer). 2015. Marc Kruse Fotografie Universiteitsdag 2015,Amsterdam. From: https://www.flickr.com/photos/marckrusephotography/18532023589/in/album-72157651999221294/

1http://www.vhto.nl/projecten/digivita/

18

4.3.1 Controls

The controls did not provide any immediate problems as they first did in the beta test. Afterexplaining the controls to the children once, they instantly picked them up and did not require anyfurther assistance. Although sometimes they did confuse the controls, resulting in the deletionof a move instead of adding it. But after taking a glimpse at the help screen in the game, theywere quickly able to fix their mistake.

4.3.2 Motion sickness

Two of the children indicated that they experienced mild dizziness after playing the game. Butafter answering the questions on the form, their dizziness had subsided. There were two otherchildren who indicated they experienced mild dizziness, but this was prior to playing the game.Their dizziness was caused by a roller coaster demo in the Oculus Rift at a different stand. Theirdizziness did not increase whilst playing the game and they were able to finish without the needto stop.

4.3.3 Difficulty level

They did show difficulties with finding the right rotation. Whether it was supposed to be a left orright rotation. This was more evident with the younger children, than the older ones. The oldergirls (13-14 years old) at the DigiVita event performed really well. They quickly understood thecontrols and were able to go through all the levels without much explanation. The If-statementdid provide some difficulties at first, but after a couple of hints they were able to produce theright program.

4.3.4 Times and number of attempts

The time results are shown in Figure 4.3, to give an overview of time spent per level for eachplayer. The graph shows that there are large differences between the performance of the children.Some understand what to do immediately, without any assistance while other need to be guidedthrough the game step by step. It also shows that the first and third level are instructive levels.They provide the player with new moves and give them the opportunity to apply them in a simplemaze. Almost all children quickly pick them up, because times for those levels are relatively lowand clustered together. The second and fourth level however, require the children to apply thenewly learnt material in a more difficult setting. These levels show larger differences betweenthe children and the points are much more spread out. The last level was intended to test howintuitive the If-statements are and not necessarily the ability for the children to understand whatit does.

19

Figure 4.2: The total amount of time spent per level. Each point represents the individual timea player spent in the accompanying level.

20

Figure 4.3: The number of attempts to reach the finish per level. Each point represents theresult of a single player for the accompanying level.

4.3.5 Question form

After completing the game, the children were asked to answer a number of questions (see Ap-pendix A). These were about the difficulty levels, how they had experienced the world and if theywanted to play more levels. The results (see Appendix B.4 and Table 4.1) show that childrenfound the game very attractive and that almost all of them wanted to play more levels. Theopinions about the difficulty level were however divided. Some indicated they found the gamevery easy, while others had struggled. However they all agreed that the last level was by farthe most difficult one. The results of the test don’t show any connection to the difficulty levelexperience by the children and their performance in the game.

Table 4.1: The results of the form (Appendix A) filled in by the children after playing the game.

Strongly Stronglydisagree Disagree Neutral Agree agree

The game was boring to play. 81% 14% 0% 5% 0%The game was difficult. 9% 23% 23% 27% 18%The environment was unattractive. 86% 14% 0% 0% 0%I do not want to play more levels. 86% 9% 5% 0% 0%The game was not instructive. 36% 23% 5% 9% 27%

Table 4.2: The percentages of children that had the correct answers for the two test mazes onpaper (Appendix A).

Maze 1 Maze 2Boys 50% 70%Girls 58% 67%Total 55% 68%

The results in table 4.2 show that there was not a significant difference between the perfor-mance of boys versus girls on the test mazes. The girls performed slightly better at the firstmaze, but the boys were somewhat better at the second.

21

4.3.6 User reactions

“I want to try it again! Until I have found the right solution!”“The beginning of the game was very easy, but helped figuring out how to play the game. Thelast level was a lot of fun!”“Look a bunny! The world is very pretty.”

22

CHAPTER 5

Conclusion

This thesis shows that virtual reality is suitable for children and the possible application ineducation. Children were challenged by the game that was developed and after the test theywanted to play more levels. The game is very accessible and children quickly understood the goalof the game and how to finish it. There were no significant differences between the performancesof boys and girls and both genders were eager in completing all the levels. The game not onlyincorporates virtual reality in education, but it also provides a way to introduce computer scienceto children. By programming their car, children learnt the basics of coding without even fullyrealising they were doing it. By solving various mazes, they learn skills such as logical thinkingand problem solving. There were some difficulties regarding the if-statements, but these werenot unexpected. With proper guidance and more levels, children will be able to understand theseconcepts and create more complicated programs.

The test mazes on paper showed that children understood the basic principles of the game,but that some had difficulties with the orientation of the car in 2D. They often forgot how manysteps to move forwards before rotating. Noteworthy was that these difficulties were less apparentinside the game itself.

Although there are some difficulties that have to be overcome, such as motion sickness anduser interaction, this game proves that with careful thought virtual reality can be a powerfultool to bring children more in contact with technology. The possible port to Android makes itmuch more accessible to schools and with a smart phone and Google Cardboard children canstart playing.

It is now up to developers and educational institutions to join forces and start developing newapplications for virtual reality. Not only for computer science, but also for courses such as biologyand physics and create an experience that two-dimensional textbooks cannot provide. Variouseducational parties have already shown interest in this game and the possibilities it provides.So schools are starting to discover the value of games as a teaching aid, but the step to virtualreality has yet to be made.

5.1 Discussion

Although the results showed that children quickly picked up the controls and found their waythrough the game, there are a number of things that would improve future tests. Due to thesetting of the experiment the tests were relatively short. Children had to learn a number ofcomplicated material in a short period of time. Also some children had already tried the Oculusright before the test on a different stand. This might have influenced some of the results, suchas motion sickness or eyestrain. It is however unlikely that it would have had a great impact onthe time results and the ability to understand how to write a program.

The tests at the DigiVita event were run on a laptop instead of the game computer at theuniversity. This meant a significant difference in performance. The graphics quality as well asthe frame rate were much lower and this might have resulted in a different experience. The goal

23

of the test was not about learning effectiveness, but more the usability of the game itself. Inorder to learn more about the educational effectiveness, a more thorough study will have to bedone.

5.2 Future Research

For future research it is good to find more ways to introduce virtual reality in education. Childrenwere very interested and wanted to learn more about these three-dimensional worlds. Withthe ability to port games to Android, the step to virtual reality becomes much lower and lessexpensive. A simple Google cardboard can provide schools with the ability to apply virtualreality into their classrooms.

Also this study is only the tip of the iceberg. The test group was relatively small and theexperiment time per child was very short. There are many more things to investigate. Forexample what happens when children will be able to play more levels? The results in graph 4.2show that the levels that introduced something new were relatively easy compared to the levelswhere children had to apply the newly learnt material. In future research it would be valuable tocreate more levels, so these differences become smaller and children have steadier learning curve.

As for the game itself, there are still many opportunities for expansions. This prototypeonly consists of five levels and these could be extended to many more. This way difficult steps,such as If-statements can be slowly explained and children can step-by-step learn the basics ofprogramming. An important feature that has to built in is the tutorial. In the current versionof the game, children need to explore all options for themselves. In future work it would beinvaluable to create a solid tutorial that takes over the role of a human tutor. This way the gamebecomes fully independent and can be used in classrooms to educate children. There are alsopossibilities to implement study material inside the world itself. For example showing animationsof teaching material while exploring the maze. This way the levels can get special themes andimprove the learning effectiveness even more.

24

Bibliography

[1] Digitale geletterdheid in het voortgezet onderwijs, Koninklijke Nederlandse Akademie vanWetenschappen (2012).

[2] Shut down or restart? The way forward for computing in UK schools, The Royal Society(2012).

[3] A. Bamford, The 3D in education white paper, THE journal (2011).

[4] T.Y. Chuang and W.F. Chen, Effect of computer-based video games on children: An exper-imental study, Digital Game and Intelligent Toy Enhanced Learning (2007), 114–118.

[5] C. Fowler, Virtual reality and learning: Where is the pedagogy?, British Journal of Educa-tional Technology (2014).

[6] J. Jacobsen, Digital dome versus desktop display in an educational game: Gates of horus,International Journal of Gaming and Computer-Mediated Simulations 3 (2011).

[7] H. Leung, H. Lee, K.P. Mark, and K.M. Lui, Unlocking the secret of 3D content for education,Teaching, Assessment and Learning for Engineering (2012), W1C–13–W1C–15.

[8] NOS.nl, Kroes: programmeren als schoolvak, http://nos.nl/artikel/

711096-kroes-programmeren-als-schoolvak.html, 17-10-2014.

[9] M. Polonen, T. Jarvenpaa, and B. Bilcu, Stereoscopic 3D entertainment and its effect onviewing comfort: Comparison of children and adults, Applied Ergonomics 44 (2013), 151–160.

[10] K. Squire, Video games in education, International Journal of Intelligent Games & Simula-tion 4 (2003), 49–62.

[11] J. Tolboom, J. Kruger, and N. Grgurina, Informatica in de bovenbouw havo/vwo, SLO -nationaal expertisecentrum leerplanontwikkeling (2014).

25

26

27

APPENDIX A

Experiment form

28

29

30

APPENDIX B

Experiment results

B.1 Time results

Table B.1: Time in seconds per level for each player, ‘-’ indicates the level was not completed.

Level 1 Level 2 Level 3 Level 4 Level 511901 69.4 46.1 17.5 96.6 137.613908 23.7 37.7 16.4 158.6 -15402 22.6 72.5 32.2 97.8 192.620905 19.9 88.3 20.9 67.3 202.022310 59.0 102.2 18.0 57.6 163.624153 28.5 32.9 17.3 48.7 -25133 18.5 54.1 15.3 97.5 178.632835 73.7 156.9 35.2 76.0 143.734307 44.7 37.8 32.2 119.6 109.635729 20.8 81.4 14.0 119.6 97.641255 27.8 149.9 92.1 135.1 129.543010 27.7 41.1 24.3 52.4 181.8103842 17.7 28.9 13.1 40.3 227.7113052 79.7 38.0 13.7 114.0 203.4124224 46.1 66.1 22.6 202.9 166.0125738 116.8 159.4 33.4 139.2 176.812133 51.7 96.0 17.0 53.5 184.725629 58.0 36.7 13.8 94.1 179.231002 40.0 144.1 38.7 151.6 208.032732 34.0 48.7 19.8 90.7 144.234308 27.2 68.0 23.7 64.9 278.235807 44.8 84.3 159.7 121.1 257.1

Table B.2: Time in seconds per level. The travel time is the time from starting the program toreaching the finish. This is the same for every player. The average time is the average time ofall players after a single play through of the game.

Level 1 Level 2 Level 3 Level 4 Level 5Shortest time 17.7 28.9 13.1 40.3 97.6Longest time 116.8 159.3 159.7 202.9 278.2Travel time 8.7 16.5 9.1 25.1 56.9Average time 43.3 75.9 31.4 100.0 178.1

31

B.2 Attempts

Table B.3: The number of attempts per level for every player.

Level 1 Level 2 Level 3 Level 4 Level 511901 1 1 1 1 113908 1 1 1 4 115402 1 2 2 2 120905 1 2 1 1 522310 3 5 1 1 224153 1 1 1 1 225133 1 3 1 2 132835 1 1 1 1 134307 1 1 2 3 135729 1 2 1 3 141255 1 2 2 1 143010 1 1 1 1 2103842 1 1 1 1 4113052 1 1 1 2 1124224 1 1 1 1 1125738 2 3 1 2 112133 1 1 1 1 125629 4 1 1 3 131002 1 5 2 2 132732 1 1 1 1 134308 1 2 1 1 135807 1 1 2 1 2

32

B.3 Form results

Table B.4: The experiment results from the form in Appendix A.

Sex Age Group Specs Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Q1011901 M 10 7 No Tablet No Yes - + - - - - + A D13908 F 13 1 Yes PC No Yes - - + - - - - + - D D

Console15402 M 8 5 No PC No Yes - + - - - - ++ D A20905 M 9 5 No PC Yes* Yes - - + - - - ++ D D

ConsoleTablet

22310 M 10 7 Yes PC Yes Yes - - + - - - - - A DTablet

24153 M 9 6 No PC No No - - ++ - - - - - B DConsole

25133 M 7 4 No Console No Yes - - - - - - - - ++ B DTablet

32835 M 7 3 No Console No No - - + + - - - - + B A34307 M 12 8 No Console No Yes - - - - - - - ++ B D

Tablet35729 F 10 7 No Tablet No No - - + - - - + - ++ A D41256 M 6 3 No PC Yes* No - - - - - - - - - A D

TabletCard

43010 M 8 6 No PC Card Yes - - + - - - - ++ B ATablet

103842 F 14 2 No PC Yes Yes + - - - - - - B D113052 F 13 1 No Tablet No Yes - - + - - - - - B D124224 F 9 5 No PC No Yes - - + - - - - - - - B D

Tablet125738 F 9 6 No Tablet No No - - ++ - - - - - - D A12133 F 9 5 No Tablet No No - - ++ - - - - - - B A25629 F 13 1 Yes Tablet No No - - - - - - - - - A A31002 F 10 5 No Tablet No No - - - - - - - - A B32732 F 10 6 Yes PC No No - - + - - - - - - B D

Tablet34308 F 9 7 No PC Yes Yes - + - - - - - - B D

Tablet35807 F 8 4 Yes PC No No - - - - - - - - - B D

Tablet

- - Strongly disagree Yes* Used the Oculus right before the test- Disagree Card Experience with Google Cardboard+ - Neutral Group School grade or year+ Agree Specs Wears glasses++ Strongly agree

33