IN DEGREE PROJECT COMPUTER SCIENCE AND ENGINEERING,SECOND CYCLE, 30 CREDITS

, STOCKHOLM SWEDEN 2020

A Performance Comparison for 3D Crowd Rendering using an Object-Oriented system and Unity DOTS with GPU Instancing on Mobile Devices.

MAX TURPEINEN

KTH ROYAL INSTITUTE OF TECHNOLOGYSCHOOL OF ELECTRICAL ENGINEERING AND COMPUTER SCIENCE

A Performance Comparison

for 3D Crowd Rendering using

an Object-Oriented system

and Unity DOTS with GPU

Instancing on Mobile Devices.

MAX TURPEINEN

Degree Programme in Computer Science and Engineering

Date: June 30, 2020

Supervisor: Björn Thuresson

Examiner: Tino Weinkauf

School of Electrical Engineering and Computer Science

Host company: Friend Factory AB

Sammanfattning

Denna rapport amnar sig att ta itu med vad som ar lamplig programmering-

paradigm for att rendera folkmassor i realtid ur ett prestandaperspektiv, med

smartphones som malplattform. Bland de mest framstaende och intuitiva pro-

grammeringsparadigmerna finns den objekt-orienterade (OO), men under de

senaste aren har data-orienterad design blivit allt mer vanlig. I den har rap-

porten jamfors Unity’s GameObject-strategi som bygger pa en objekt-orienterad

grund med deras DOTS-system med GPU-instansiering genom olika arranger-

ade testscenarier byggda med Xcode pa en iPhone 6S och en iPhone XR. Resul-

taten fran olika scenarier representeras genom flera diagram med fokus pa den

erhallna bildhastigheten, CPU-anvandningen och GPU-anvandningen. DOTS-

systemet visade sig overtra↵a det objekt-orienterade systemet i sex av atta sce-

narier med iPhone XR som den battre presterande smartphonen. Med DOTS for

narvarande under utveckling, saknas annu flera forbattringstekniker som annu

inte ska integreras, sasom culling eller LoD, som for narvarande kan anvandas

av dess motsvarighet. OO-systemet ar mer robust med variation medan DOTS-

systemet passar battre nar antalet karaktarer okar.

A Performance Comparison for 3D Crowd Renderingusing an Object-Oriented system and Unity DOTS

with GPU Instancing on Mobile Devices.Max Turpeinenmaxtu@kth.se

Royal Institute of TechnologyStockholm, Sweden

AbstractThis paper aims to address what is a suitable programmingparadigm for real-time crowd rendering from a performancestandpoint, with smartphones as the target platform. Amongthe most prominent and intuitive programming paradigms isthe object-oriented (OO) one, with data-oriented designs be-coming more common in recent years. In this paper, Unity’sGameObject approach built on an object-oriented foundationis compared with their DOTS system using GPU instancing,arranging di�erent test scenarios later built using Xcode onan iPhone 6S and an iPhone XR. The results from the di�erentscenarios and builds are represented throughmultiple graphsfocusing on the obtained frame rate, CPU usage and GPUusage. The DOTS system proved to outperform the object-oriented system in six out of eight scenarios with the iPhoneXR yielding better performance. With DOTS currently beingunder development, several acceleration and enhancementtechniques are yet to be integrated such as culling or LoD,which currently can be used by its counterpart. The OO sys-tem is more robust with variation whereas the DOTS systemis better suited when the number of characters increases.

1 IntroductionThe use of virtual crowds [33, 48] has practical uses as wellas an aesthetic contributing e�ect when trying to reenactreal-life scenarios. Examples of practical uses are evacuationscenarios or crowd psychology, where one tries to explaincrowd behaviour in contradistinction to individual behavior.Allowing for a crowd to be represented as a group has its ad-vantages in the uniform and consistent behavior in-betweencharacters, and is not as demanding in terms of computationswhereas disadvantages are to get unique characteristics andbehaviors for individuals within the group.In the entertainment industry like in games and movies,

the concept of crowds also comes into play in a more aes-thetic sense, trying to convey realism through populatingareas. Dealing with a group representation is prefered inreal-time strategy games such as Age of Empire (Ensem-ble Studios, 1997-2019), Total War (The Creative Assembly,2000-2019) and Starcraft (Blizzard Entertainment, 1998-2017),where you are mostly in control of armies rather than indi-viduals. In the sports game genre we are most likely familiar

with the titles of FIFA (Electronic Arts, 1993-2019), NHL(Electronic Arts, 1991-2019) or NBA 2K (Visual Concepts,1999-2019) where there is a tendency for an audience withspectators. These are also considered crowds but without theneed for individual character navigation or collision avoid-ance. Regardless of �eld of use, there is a constant strive forimproved realism which also puts a strain on performance.Based on the designated purpose of the crowd, some fac-

tors may be more important than others. The most frequentuse of crowds can be seen in the gaming industry. In strategygames, unilateral large-scale armies are of utmost impor-tance, meaning focus lies in the numbers rather than thevariation credibility. In sports games, we often �nd crowdsstaging as spectators in the stands where numbers and varia-tion are important, but visual representation is not since thecrowds are never the point of interest. Within the sandboxgenre e.g. GTA (Rockstar Games, 1997-2014) or Minecraft(Mojang, 2009), there is a greater need for interactive environ-ments in which crowd characters come into play. Dependingon the designated platform, sought performance and graphi-cal demand, the choice of rendering scheme is of the essence.Smartphones [55] have gone from being only used in a

business context to replacing almost every other type ofphone. Due to the rapid development pace of mobile devicesalong with their constant performance increase, we are cur-rently facing a wide range of devices at our disposal, eachwith its own speci�cation. Developing mobile applicationsoptimized for all current mobile generations is therefore aconsiderably cumbersome task. Looking at correspondingperformance benchmark tests of di�erent smartphone gen-erations [49], the speci�cations as well as the performancevaries immensely. This is one of the reasons why there isa need for either software adaptations or universal mobileapplications.To this date, a variety of methods have been developed

and investigated in how to simulate crowds based on how acharacter is represented and produced, with their associatedanimations. There are many crowd rendering techniqueswhere the most commonly used technique is Level of De-tail (LoD) rendering [15, 26]. The idea of LoD is having the

complexity of the model associated with a distance thresh-old from the view perspective with a handful of prede�nedmodel versions.The LoD approach is often associated with a polygonal

mesh representation where one renders a less complex 3Dmodel as the virtual camera traverses further from the mesh.The Image-Based Rendering (IBR) [2, 4, 6, 13, 21] approachrelies on having several (2D) images of the same object cap-tured from di�erent positions and angles to reconstruct theidea of an animated 3D object. Point Sample Rendering (PSR)[18, 23] uses the idea of having the view be represented by acollection of in�nitesimal view-independent surface pointswhich each contains the information of depth, normal, coloretc. which gets recalculated with the change in camera view.Besides using simply one LoD technique, several can be

used in parallel to construct a hybrid approach in the at-tempt to utilize the bene�ts of one approach and avoidingthe drawbacks of another. The idea behind a hybrid approachis to potentially neutralize the bottlenecks of each di�erentapproach and bring forth their individual bene�ts. Never-theless, hybrid approaches often add a layer of managementand run the risk of having to store multiple collections ofthe same object. Having to switch between representationsrequires communication in between hardware componentsand might stress their bandwidth.Despite all attempts to construct the optimal rendering

scheme, there is no single one that is strictly better thanthe others in every aspect. To justify the use of one speci�capproach is therefore based on the desired result and avail-able resources. Ideally, one could render endless amounts ofcharacters with arbitrary intelligence and visual satisfaction,but unfortunately as of today, that is not the case. When itcomes to rendering crowds we still struggle with trade-o�sbetween quantity, quality, memory usage and environmentaladaptation.

Apart from the di�erent standard approaches, further po-tential computation improvements can be made by lookingat e.g. frustum culling [9], memory usage [13], alternativedata representations [50], as well as visual improvementsthrough manipulation of lighting [19], shadowing [41] andcrowd variability (clothing) [28].

An alternative data representation currently under devel-opment is Unity’s Data-Oriented Technology Stack (DOTS)[46, 47], adding on top of their more commonly knownobject-oriented (OO) game engine, an additional data-oriented(DO) paradigm. The justi�cation for a data-oriented designand Unity DOTS is the CPU performance gain through theEntity Component System (ECS), the C# Job System andthe Burst Compiler. What is not considered in the generalOO approach nor the DOTS approach is the utilization ofthe GPU for improved performance. One suggestion thathas been made is to o�oad the CPU using GPU instancing(2.3.3).

In this paper, the aim is to address the question of Whichparadigm for 3D crowd rendering, based upon object-orientedprogramming and DOTS with GPU instancing respectively,gives the most e�cient performance on a smartphone deviceof today?

2 Related WorkThis section of the paper introduces some relevant conceptsof real-time crowd rendering together with the more promi-nent and controversial techniques. Firstly, a few di�erentcrowd animation approaches are introduced (2.1), movingover to di�erent character representations (2.2). Lastly (2.3),some of the more modern concepts are touched upon whichare later to be implemented in the Methodology and Imple-mentation section.

2.1 Character AnimationRendering refers to the way of presenting what is displayedon the screen at a given time, which is called a render. Asequence of renders is what adds up to an animation. Onepreferably aims to have a frame display frequency of at least30 frames per second, when animating. The result of the ani-mation is in many cases heavily dependent on the characterrepresentation.

There are a few approaches of going about crowd render-ing, such as �ow-based, entity-based and character-basedones. Along with the di�erent approaches there are a rangeof di�erent aspects to consider, depending on the accessibleresources and desired result.

When rendering crowds on an individual basis, a handfulof primitives has been covered throughout the years where inevery approach there is always a trade-o� between quantity,quality, e�ciency and versatility. To create highly accuratecharacters with �ne-grained appearance [1, 20, 29, 38] isimpractical when trying to run in real-timewith a demandingenvironment or larger crowds. Therefore attempts to makeimprovements through adaptations in e.g. animations havebeen made throughout, with varied results.

2.1.1 Skeletal animation. In recent years the most ex-plored way of doing animations is skeletal animations [27],where the base structure of the character consists of a bonehierarchy, with each bone linked to parts of the character’sskin or mesh. Animating using a skeletal basis is known asskinning and the process of constructing this setup is oftenreferred to as rigging. The rigging is usually done manually,but recent aiding tools have been created to speed up theprocess [3, 32, 35].

The movement of the bones is what decides how and howmuch the mesh moves by carefully mapping each bone toregions of the mesh. Multiple bones can a�ect the samemesh, which is why each individual bone has to be properlyweighted against themesh [11]. In between frames, the bonesare transformed from one keyframe to another and given

2

an arbitrary time of display, the pose can be interpolatedaccordingly.

2.1.2 Per-vertex animation. Another animation approachis the per-vertex approach, where unlike the skeletal ap-proach where we have bones, mesh deformations are basedon the vertices’ positions themselves. In each keyframe, thevertex positions are stored which later can be used to obtainnew deformations using interpolation [39, 54].With this approach the animator is given more freedom

at hand to alter movements not restricted to bones. Thisis especially useful if in need of facial expressions or handgestures, which can be hard to achieve with a limited set ofbones. Ulicny et al. [50] used the idea of switching betweenkeyframes containing pre-computed deformed meshes, cre-ating the perception of animation. The major drawback withthis approach was the need for large memory storing thepre-computed meshes.

2.1.3 Animation Individuality. By individuality in re-gard to crowds, we refer to diversity in appearance. Thisgoes for variation in animations, but also in clothing etc.When the number of characters increases in a scene, theneed for more distinct animations becomes inevitable tomaintain the perception of each character being a peculiarindividual. An attempt to handle this is by adding a randomo�set in the animation for each character, avoiding obvioussynchronization [31].

The animation bottleneck seldom lies in the memory foot-print, but rather in the CPU computations needed to blend allthe di�erent animations to persuade individuality amongstall the characters. What goes for most animation represen-tations, the di�erent poses are stored in memory as sets ofmatrices, which transforms the bones or vertices in the run-ning. Instead of sending the keyframematrices from the CPUto the GPU, which are costly on the CPU-GPU bandwidth,these can be preloaded directly to the GPU, consuming GPUmemory instead [37].

2.2 Level-of-detail representations for charactersA way of improving crowd rendering performance is to usethe level-of-detail (LoD) technique [9, 15], where the char-acter representation is based on its addition to the scene[16, 34]. The principle is that as a character gets further fromthe perceived scope, it gets less detailed in terms of represen-tation, allowing for a cheaper approach to be used. Recentadaptations have gone beyond using the concept of LoD onlyfor appearance, but also for sounds and animations [50].LoDs can either be used as continuous LoDs [14] or dis-

crete LoDs [8], where the latter is the most commonly used.Continuous LoDs e.g. progressive meshes, implies that themesh is changed as we move the virtual camera, using ascheme to re�ne the mesh as we move closer and simpli-fying it as we move further away. By moving the virtualcamera back and forth, the appearance might change over

time unless the transitions are rigid or stored in memory.The latter would of course be redundant, making the discreteapproach more perceivable and manageable.

As for the discrete LoDs, di�erent prede�ned representa-tions are set up to be used at a certain displacement thresholdfrom the virtual camera. Given the circumstances, one canchoose to utilize as many LoDs as possible or needed. Ulicnyet al. [50] were able to instantiate more than a thousand ofcharacters into his scene using di�erent discrete LoD repre-sentations.A common artifact with the LoD approach is the percep-

tible degradation, switching from one LoD representationto another, often called the popping artifact [4]. When mov-ing in a scene where LoDing is not done carefully with nofurther measures, one can clearly observe the switching inbetween models. To avoid this, one can add a delay in themodel transmission or letting both of the di�erent modelsbe displayed in tandem for a short period.

2.2.1 Polygon-based techniques. As mentioned before,continuous LoDs are not very suitable when it comes toanimations [24], therefore the use of static meshes are pre-ferred (Figure 1). Schmalstieg and Furhmann [36] chose todivide mesh into single- or multiple bone weighted regionsfor simpli�cation, later to reassemble them strategically. Forapproaches like these, the main issue is the need to prede�nesets of animations, limiting the dynamicity in variation.

Figure 1. A representation of how LoDs are used and repre-sented. As the camera gets further away from the characterthe LoD-level increases and the complexity decreases. Thepercentage implies how many percent of the original meshthe character consists of.

Landreneau and Schaefer [22] introduced an approach tocollapse mesh edges, decided by the given fault toleranceto positioning and weights of each edge. Through this, theywere able to construct new vertices and weights throughinterpolation, achieving new animation keyframes from theotherwise limited set.

2.2.2 Point-basedTechniques. Another primitive of rep-resenting static mesh is the point-based one, which was ini-tially suggested by Levoy and Witted [23]. It uses the idea of

3

having the view be represented by a collection of in�nitesi-mal view-independent surface points which each containsthe information of depth, normal, color etc. Each point alongwith its information then gets recalculated with the changeof camera and lighting. This approach works well in the fardistance, not having to render polygons [18] which cannotbe seen anyhow.

2.2.3 Image-based Techniques. One of the more com-monly used primitives is using images. This as well as thepoint-based approach, is most suited for characters furtheraway from the camera due to the lack of detail. A compari-son between these two approaches were made by Millan andRudomin [30]. The idea is having several (2D) images takenof the same object from di�erent angles to (when moving thecamera) convince the viewer that it is an actual 3D object.These images are often known as billboards, or imposterswhen talking about characters.

To generate the images one can either generate them dy-namically or statically. Dynamic generation means imagesof an object are generated at runtime and kept for as longvalid, decided by a given heuristic. Aubel et al. [2] were theone to come up with an initial version, re-using snapshotsfor more than one frame. Going by the static approach, onecaptures the images in advance, storing them only to usewhen needed. Tecchia et al. [40] were the �rst one to try this,to later also enhance the approach adding shadows givenfrom the silhouette of the character [38], which only workedconvincingly on planar surfaces. A major drawback besidesnot being considered detailed, is the memory usage neededto store all the di�erent angles of the model at each frame ofeach animation cycle.

2.2.4 Hybrid techniques. As we have come across di�er-ent approaches, each with their own bene�ts and drawbacks,several attempts to shift or eliminate the bottleneck of a par-ticular approach has been made through hybrids [4, 5, 10, 21].The most intuitive hybrid representation is the one usingstatic meshes close to the virtual camera, transitioning tostatic imposters further, which is what Dobbyn et al. [12] didnaming them Geopostors. The Geopostors shows an increasein performance in comparison to the sole polygon-based rep-resentation and an aesthetic improvement compared to theimposter one. Along with Geopostors a number of hybridapproaches have been introduced [4].

2.3 Current DevelopmentsAs development is constantly ongoing but rarely accessiblefor the general public, one usually has to wait until stableand tested versions are released. This is certainly more ac-curate for hardware than for software, even though newsmartphones are released every year. With the continuoushardware progression, the software needs to be saturatedthrough better performing paradigms like DOTS or task bal-ancing schemes like GPU instancing, in comparison to the

more commonly used paradigm of object-oriented program-ming.

2.3.1 Smartphones. When dealing with smartphones asa form of a computer [55], there are still a few practicaldi�erences. Due to the desired portability and size of smart-phones they cannot be provided the same consistency inelectricity nor the equivalent processing power. Hence, theyuse batteries which do not have the same need for a fanor a heat sink. When instead on the verge of overheating,smartphones tend to rely on thermal throttling, forcing appli-cations and processes to shut down, reducing the workloadand dispensing the heat over time. The size of a smartphoneis possible through the System on a Chip (SoC) architecture,having all the components packed into one single chip. Themain processing components in any computerized systemare usually the CPU and the GPU where in the smartphonecase, the CPU is heavily pressured having to be able to main-tain active apps [17] and being able to switch between them.The GPU is instead designed to handle more speci�c tasksrelated to graphics and due to compromise through the SoCand using a battery, mobile applications usually run in 30fps, rather than a higher frame rate.

2.3.2 Object-OrientedProgramming vsDataOrientedTech Stack. Recently, the transition towards a data-orienteddesign has had an upswing in the game industry due to itssituational performance bene�ts crucial to e.g. smartphones.Among these are Unity which are currently developing theirDOTS [46] system which aims to give high performance asa default. The system utilizes the C# Job System allowing formultithreading, Entity Component System (ECS) enforcingperformant code writing and further improvement throughthe burst compiler producing highly optimizedmachine code.Besides the automated parallelization and optimization ofmachine code, writing code according to ECS standards iswhat binds it all together. The traditional object-orientedwayof creating a game in Unity is by creating a GameObjects[44] and attaching MonoBehaviors [45], giving its function-ality like rendering, physics and movement. With ECS, thegame is instead split up into the three di�erent parts of En-tities, Components and Systems. Entities are used to grouptogether components, similar to the traditional GameObjects,but not as resource demanding. Components are contain-ers to store plain data and unlike GameObjects, lack logic.Systems are instead what operates on the entities given aspeci�c set of components, adding the logic. Just like withGameObjects, we can change entities’ component data usingsystems, e.g. a rendering system to change the renderingcomponent of an entity (Figure 2). Even though the outcomeseems to be the same, the computational strain is di�erentsince the same system can update several di�erent entitieswith the same components, whereas each GameObject has tohandle its own logic for each separate functionality. Further-more, having component data being labeled the same, they

4

can be grouped into the same chunk of memory, allowingfor easier and faster memory lookups by the correspondingsystem.

Figure 2. To the left is a representation of Unity’s traditionalway of creating games using GameObjects with attachedMonoBehaviors, each handling its own logic. To the right isthe ECS representation using Entities, where several Com-ponents can be handled by the same system.

2.3.3 GPU Instancing. Another welcome proposal in re-gards to smartphones is the one of balancing the burdenbetween the CPU and the GPU. One way in o�oading theprimarily exposed component, the CPU, is through GPU in-stancing [13, 25]. GPU instancing utilizes more of the GPUby letting the CPU preprocess data, converting it into a UVtexture which later can be handled by the GPU instead. AUV texture is essentially a matrix but instead consisting ofRGB-colors, where depending on the axes, one can formdi�erent metrics. As an example, one could express an char-acter’s position by converting its x-, y- and z-coordinates intoRGB-colors, having the x-axis of the UV texture representall vertices and the y-axis the corresponding positions.

3 Methodology and ImplementationIn this part of the paper we go through how the two di�er-ent systems of object-oriented GameObjects with MonoBe-haviour and DOTS with GPU-instancing were implementedand how they were tested. Onwards, the object-oriented im-plementation will be referred to as the OO system and theDOTS with GPU instancing implementation as the DOTSsystem.To answer the research question, research papers and

approaches relating to crowd rendering were looked at toget a general overview and understanding on the topic (2.Related Work). For starters, all non real-time rendering tech-niques could be excluded, like ray-tracing. One of the mostcontroversial approaches which i came across in regard tocrowd rendering was the one of DOTS (2.3.1) with perfor-mance in mind. Through using a data-oriented approachone can advantageously manage several components within

the same update call handled in parallel through workingthreads. This speaks a lot for the increasingly common ideaof diverging from hardware reliability and depending moreon the software archetype for performance [7]. Since this isan ongoing development and di�erent from the typical usedapproaches, this seemed like a good choice to implement andinvestigate further. With DOTS being under development,getting a reference from one of the more typical approachesis an intuitive way of going about a comparison study. AsUnity also has their traditional object-oriented way of cre-ating games and rendering crowd, this would give the twoapproaches the most justice being built on the same engine.Additionally, GPU instancing is also introduced to the DOTSimplementation, not interfering with the reference imple-mentation, yet making an attempt to balance the workloadfor improved results.Both implementations were developed in Unity Version

2019.3.0f3 Personal. The two phones that were used conduct-ing the tests were an iPhone 6S (32 GB) [51] and an iPhoneXR (64 GB) [53], both with the operating system iOS 13.4.1.The software (compatible with Unity) to build on the iPhoneswas Xcode [52] version 11.4.1. As the DOTS implementationwas produced during the development of DOTS some pack-ages were used in preview versions. The noteworthy packageversions used related to the DOTS system were Burst 1.2.1[42] and Entities 0.5.1 [43]. To compensate for the screenresolutions not being the same between the iPhone 6S (1334x 750) and iPhone XR (1792 x 828), the render scale of therender pipeline was reduced to 0.81 and 0.665 respectively.

The two implementations were developed in parallel withthe aim to be able to reconstruct as resembling scenes as pos-sible in every aspect (Figure 3), only di�er in the paradigmused. The reasoning being for the tests to be as systematicallyexecuted as possible, having the upcoming test be derivedfrom the previous, only having changed or added a singleelement in each iteration. Each change in the scenery wasmade with the assumptions that the OO system would bemore robust when adding variation and the DOTS system,when adding in numbers. Seeing how the OO system per-formed better than expected, this is what later led to the testswith the scaled re�nement in mesh.

The reason for using two phones of di�erent generationswith quite the speci�cation and performance gap, yet plau-sibly used today, was to get an idea of how current phonesperformwith the di�erent approaches. The idea is not to com-pare the performance between the two di�erent smartphonemodels, but rather get an insight in the performance rangeof relevant smartphones of today. How well-performing anapproach was on each phone could be seen when the appli-cation was built, being plugged into the computer runningwith Xcode. The performance values which were gatheredin Xcode were frame-rate (fps), CPU-usage and GPU-usage.All scenes were built from the same base with the basic

main camera, directional light and skybox that is included5

Figure 3. The OO system running to the left and the DOTSsystem running to the right trying to mimic the same scene.The coloring is di�erent in between the two approaches sinceECS does not support the same shaders for the moment.

by default, creating a new scene in Unity. A smaller planeforming the ground and a simple box acting as the focal pointfor the crowdwere also added into the scene. Lastly, an emptyGameObject was added in the corner of the plane, which thecorresponding spawner script was added depending on thesystem being tested. Beside all objects in the scene havingthe same positions, the di�erent spawn objects also had allthe same attributes attached to its script. Amongst otherthings, the prefab itself, the character count in x- and z-axis,the grid step in x and z, the focal point, as well as some noisein each axis.The implementations were tested against each other on

each phone separately, checking for when the frame ratedropped below 30 fps, indicating the upper limit. Even though30 fps might be considered low with most current applica-tions moving towards 60 fps or higher, 30 fps is a good bench-mark considering OO being an old concept [4] and phonesnot the optimal rendering platform. Every sample of eachbuild was run once and the measurements taken after �veminutes into the run to observe potential thermal throttling(2.3.1).

In the �rst test scenario, the same model and animationwere used for all characters. The number of characters wereincreased in each respective case until the frame rate droppedbelow 30 fps. Noteworthy is that each measurement wasreceived from a single discrete build, meaning the exact breakpoint was not produced but rather an estimate between twobuilds due to the time consumption of a build. For the secondtest scenario, nine di�erent crowd characters (Figure 4) alongwith nine di�erent animations were used, looking at howthe variation in appearance a�ects the performance of eachsystem. For the two last test scenarios (assumed to be bettersuited for the DOTS system), only one variation of crowdcharacter was used, but scaled in delicacy by reducing the

polygon count to 20%, respectively 10% of the original meshusing an external asset. This allowed for more characters tobe spawned, testing for less variety but larger scales whiletrying to maintain 30 fps. The di�erent test compositionscan be viewed in Figure 5.

Figure 4. The OO system running to the left and the DOTSSystem running to the right, having nine di�erent characterseach with its own animation.

4 ResultThe following graphs (Figure 6-13) show the received mea-surement data in frames per second (fps), processing time inmilliseconds for the CPU at each frame (CPU) and process-ing time in milliseconds for the GPU at each frame (GPU)for each speci�c test. The measurements are taken at a sin-gle frame after having the systems run for �ve minutes toobserve potential thermal throttling (2.3.1). In every graph,each sample is presented as bars in groups of three wherethe fps bar (�rst) is colored in blue, whereas the CPU bar(second) is colored in dark gray and the GPU bar (third) inlight gray. The coloring is done with the purpose of focusingon the fps primarily, since its performance is dictated by thecombination of performance amongst the CPU and GPU.

The targeted benchmark in all the tests are set to be 30 fps(2.3.1), which is also why whenever the system overperforms,all bars for that sample are capped at the 30 mark as anindication. Similarly, whenever the system underperforms,the bars are capped at the bottom mark of the graph withthe exception being whenever the system drops below 30fps for the �rst time. Note that this does not apply to the lasttest (Figure 13) where we can see how the frame rate of theOO system keeps dropping as the DOTS system maintainsits performance.

In all the graphs, two di�erent y-axis are used denoted as“frames per second (fps)” and "milliseconds (ms)" since thereare di�erent units between fps and CPU/GPU. Having bothunits visible simultaneously makes it easy to compare the

6

Figure 5. The colored squares indicate what system wastested for which settings, with the number of characters rep-resented on the y-axis and the smartphone and test scenariorepresented on the x-axis. “#” meaning the test scenario withthe same character and “All” including variation in appear-ance and animation.

performance between the two systems being next to eachother and also to distinguish patterns and correlations in thedi�erent measurements. Whenever the processing time ofboth the CPU or GPU is kept under 33 ms, so will the framerate, being capped at 30 fps. On the contrary, whenever thevalues no longer range between zero and 33 it indicates thatone of the processors is being overworked and a breakpointhas been reached. The x-axis contains at the very bottom,the number of characters and moving upwards, the render-ing system tested for that particular number of characters.Note that for all the graphs regarding iPhone 6S, the numericvalues (y-axis) ranges from 22-42 and for the iPhone XR, thenumeric values ranges from 18-52. The number of characters

and samples (x-axis) also varies across the graphs. The fol-lowing graphs are presented in pairs, where the same testsare executed only on di�erent devices with the iPhone 6Scoming �rst and the iPhone XR secondly.Throughout the graphs regardless of test or device, look-

ing at the far right gives a good indication of where the bestperforming system starts dropping. Paying close attention,one will also see how the CPU and GPU of the DOTS ap-proach is always in line with each other when reaching itslimit. When looking at the iPhone 6S test for number ofcharacters (Figure 6), the DOTS system sustains its framerate for several more characters added to the scene than theOO system. The OO system already starts dropping in framerate at 30 characters whereas the DOTS approach maintainsa solid frame rate of 30 fps up until 50 characters. At 45 char-acters on the other hand, the CPU hits 33 ms in processingtime and the GPU 32 ms, meaning it is closing in on its limitwhich also explains the considerable drop to 27,5 fps in thenext sample.

Figure 6. The fps-, CPU- and GPU performance on theiPhone 6S given the number of characters for each system.

For the same test but instead with the iPhone XR (Figure 7),unlike with the iPhone 6S, the DOTS system starts droppingin frames ahead of the OO system. At 120 characters thedi�erencemight seem substantial, but note that it only di�ersin three fps the systems apart.

Adding variety to the scene with the iPhone 6S (Figure 8)results in a much quicker fall in fps for both systems, withthe greater fall for the DOTS system. Comparing the valuesfor the OO system with the ones without variation (Figure6) the di�erence is much smaller than when comparing theDOTS system. This is expected, but bare in mind that theCPU- and GPU values are probably even higher for the OOsystem at 50 characters, compared to the DOTS system.

Yet again the OO system outperforms the DOTS system onthe iPhone XR with variation included (Figure 9). Strangelyenough, both systems show greater values in comparisonto the test without variation (Figure 7), when expected toperform worse due to the addition in complexity.

7

Figure 7. The fps-, CPU- and GPU performance on theiPhone XR given the number of characters for each system.

Figure 8. The fps-, CPU- and GPU performance on theiPhone 6S having nine di�erent characters with nine dif-ferent animations.

Figure 9. The fps-, CPU- and GPU performance on theiPhone XR having nine di�erent characters with nine di�er-ent animations.

Having the mesh of the crowd characters being reducedhas increased the amount of characters possible to render,

even for the iPhone 6S (Figure 10). By each character beingreduced to a �fth in size, one could make the bald assumptionthat the system now should be able to handle �ve times morecharacters. This proves to be the case, but still we cannottake this for granted. Both systems even happen to performbetter.

Figure 10. The fps-, CPU- and GPU performance on theiPhone 6S having multiple of the same character, reduces inmesh down to 20%.

Downscaling the complexity of the characters to 20% forthe iPhone XR (Figure 11) breaks the trend of having theOO system outperform the DOTS system on the iPhone XR.Instead, the DOTS system outperforms the OO system byquite a bit. At 600 characters we can see how the DOTSsystem reaches its exact break point of having 30 fps, yet33 ms for both the CPU and the GPU. Comparing this tothe test with the original mesh (Figure 7) the OO systemhandles about four times as many characters with the sameperformance, whereas, the DOTS system has surpassed theOO system in performance and pulls o� about six times asmany characters.

Figure 11. The fps-, CPU- and GPU performance on theiPhone XR having multiple of the same character, reduces inmesh down to 20%.

8

As the mesh goes down to 10% for the iPhone 6S (Figure12), the OO system can no longer keep up with the DOTSsystem, collapsing in the other end of the graph. Seeing howthe DOTS system almost produced 50% more characters inthe �rst test of the iPhone 6S (Figure 6), the DOTS systemnow produces twice as many characters, maintaining theframe rate of 30 fps.

Figure 12. The fps-, CPU- and GPU performance on theiPhone 6S having multiple of the same character, reduces inmesh down to 10%.

Having a last test on the iPhone XR with the charactersbeing 10% of the original mesh (Figure 13) we can now tracethe values for the OO system. The processing time for theCPU at each frame is steadily increasing as the frame rategoes down. Here one can clearly see how dependent theOO system is on the CPU, unlike at the end of all graphswhere one can see how the CPU and GPU is kept balancedwhenever the DOTS system reaches its limit.

Figure 13. The fps-, CPU- and GPU performance on theiPhone XR having multiple of the same character, reduces inmesh down to 10%.

5 DiscussionThis thesis has attempted to address the research question:Which paradigm for 3D crowd rendering, based upon object-oriented programming and DOTS with GPU instancing respec-tively, gives the most e�cient performance on a smartphonedevice of today?From the results one can deduce that the DOTS system

outperforms the OO system in almost every test case withthe exceptions of the ones executed on the iPhone XR withand without variation in characters and animation with theoriginal mesh. Therefore, with the initial statement of bothiPhone 6S and iPhone XR being considered smartphone de-vices of today, one cannot conclude that one system is strictlysuperior in all regards executed on today’s smartphones.

The initial expectation was for the OO system to performworse than theDOTS system overall, mainly due to it treatingeach crowd character as a unique object, whereas the DOTSsystem handles components of the same type simultaneouslyas groups. This of course speaks for a potential downsidefor the DOTS system as variation increases whereas the OOsystem should more or less perform the exact same.

Despite one system not coming out ahead in all the tests,the DOTS system could be considered more promising see-ing how it outperforms the OO system in six out of eighttests. Nonetheless, in all tests where the DOTS system is in-ferior, both systems drop below 30 fps at the same charactercount implying the performance di�erence being marginal.Additionally, whenever the DOTS system outruns the OOsystem the results are predominant in favor of the DOTSsystem. Judging from the tests where the mesh is reduced,the DOTS system almost produces twice the amount of char-acters before dropping below 30 fps in comparison to the OOsystem. Noteworthy is also how whenever the DOTS systemis pushed to its limit the CPU and GPU usage is perfectlybalanced, due to the CPU being able to o�oad the burdenon the GPU thanks to the GPU instancing. On the contrary,whenever the OO system gets pushed to its limit the CPUusage skyrockets whereas the GPU usage is comparably low.

Even though we denote iPhone 6S and iPhone XR as smart-phones of today, already as of this writing we have thevariations of the iPhone 11 with improved CPU and GPUlikely to perform better in the same tests. Besides the gen-eral improvement in hardware, with a plausible paradigmshift from object-oriented designs to data-oriented designs,hardware modi�cation in e.g. memory might give even fur-ther improvements in performance. The same can be saidabout interchangeable processing between the CPU and GPUachieving perfect balancing, optimizing processing time.

5.1 Methodology criticismThroughout the majority of the samples and test the mea-surements proved to be reasonably stable only �uctuatingaround the presented values. Even so, the measured data

9

could have been more accurate doing several tests, usingthe same scenery and presenting an overall average for eachsample. The reason for only running a single execution foreach sample was due to the provided resources and timeneeded to run the samples. Each sample had to be built for acertain smartphone and desired system at a time, meaningwith each modi�ed variable, another build had to be set up.As we can see in Table 1, more than 80 samples had to berun just collecting the necessary breakpoints. This does notinclude the test where four and nine respectively di�erentcharacters and animation were tested separately for each de-vice, adding up to another at least 40 samples. These sampleswere not included showing similar results to the samplesincluding both character and animation variation. By ques-tioning the accuracy of either sample all tests had to be rerunto be consistent, including the samples not displayed. Witha single sample execution taking approximately 20 minutes(not including the modi�cation time in between runs etc.),with over 100 samples to be tested multiple times, the timeneeded would have been counted in days.A related potential error is how the smartphones were

plugged into the computer one at a time, continuously charg-ing as the tests were built, generating additional heat, causinga potential drop in performance due to cooling measures.This was not considered during the process and might havelasted di�erent amounts of time, depending on if a phonewas charging as another scene was constructed etc. Thismight explain why the OO system performed better in the�rst tests on the iPhone XR, having the OO system built �rst.The majority of the samples were made for the �rst test toget a feeling for where the break point was and how diversedata to collect.

Another �aw to consider is how the numbers in the graphsmight be skewed going from only number of characters tohaving characters with nine di�erent animations and charac-ters due to the average character having a less re�ned meshthan the single one used for the number of characters tests.This has no e�ect on the �nal result when comparing be-tween the two systems, but rather results in an overestimatein performance for both the systems using nine charactersand animations. The same reasoning applies for the anima-tions.

Also, when randomizing the o�set of the characters in theOO system, the old math package related to MonoBehaviorswere used whereas for the DOTS system, the new mathpackage was used. This also meant that the randomness wasproduced in di�erent ways, where the o�set for the DOTSsystem was actually produced via noise instead of througha random function. Therefore nor was a random seed usedwhich could have determined the randomness, making thetests even more alike. To bear in mind is that the randomnessis only used initially placing the characters, only causing apotential delay on the �rst frame. Nor is this relevant forpotential characters ending up o�screen since we are not

using any culling, meaning regardless if a character can beseen or not, if placed in the scene this will a�ect performancein the same way.

5.2 Future workGiven the previous observation of the CPU usage skyrocket-ing for the OO system one could argue that for future workone should integrate GPU instancing into the OO system too�oad the CPU, which today is already possible in Unity.As previously mentioned (2.2), the concept of Level-of-detailis frequently used and one of the most prominent used en-hancement techniques when it comes to crowd rendering,but also in games in general. Furthermore, adding additionalperformance enhancers such as frustum culling [9] wouldalso give a performance increase if used correctly, whichhave been done previously in OO approaches i.a. in Unity.

6 ConclusionThrough comparing the results of the OO- and DOTS systemone can conclude that the DOTS system is performing betteras the numbers raise, but variation is kept low. The OOsystem performs well with having fewer numbers, but does abetter job maintaining its performance as variation increases.As for future work, both systems could bene�t from furtherenhancement techniques such as LoD or culling.

AcknowledgmentsForemost, with my sincerest gratitude and deepest apprecia-tion I would like to thank my supervisor Björn Thuressonwho has not only guided me in my research and throughthese severe times, but who has also mentored me throughmy studies at KTH.

My appreciation also extends to Friend Factory for havingme do my research work in their presence, accompanied bymy great friend Vincent E. Wong who I’ve had the greatestpleasure of sharing my studies with throughout our journeyat KTH.

References[1] Thalmann D. Aubel A. 2000. Realistic deformation of human body

shapes. Proc. Computer Animation and Simulation 2000 (2000).[2] Thalmann D. Aubel A., Boulic R. 1998. Animated impostors for real-

time display of numerous virtual humans. VW ’98 (1998).[3] PopoviC J. Baran I. (Ed.). 2007. Automatic rigging and animation of 3d

characters. ACM Transactions on Graphics, Vol. 26.[4] Pelechano N. Beacco, A. and C. Andújar. 2012. E�cient rendering of

animated characters through optimized per-joint impostors. Journalof Computer Animation and Virtual Worlds 23 (2012).

[5] Andujar C. Pelechano N. Beacco A., Spanlang B. 2011. A �exibleapproach for output-sensitive rendering of animated characters. Com-puter Graphics Forum 30 (2011).

[6] Pelechano N. Spanlang B. Beacco A., Andújar C. 2015. A Survey ofReal-Time Crowd Rendering. Computer Graphics Forum, 35, 8 (2015).

[7] Zhirnov V.V.. Cavin R., Lugli P. 2012. Science and Engineering BeyondMoore’s Law. Proc. IEEE (2012).

10

[8] Fahn C. Tsai J. Chen R. Lin M. Chen, H. 2006. Generating high-qualitydiscrete LODmeshes for 3D computer games in linear time.MultimediaSyst. 11 (2006).

[9] Silva C. T. Durand F. Cohen-Or D., Chrysanthou Y. L. 2003. A sur-vey of visibility for walkthrough applications. IEEE Transactions onVisualization and Computer Graphics 9, 3 (2003).

[10] Meyer A. Coic J., Loscos C. 2007. Three LOD for the Realistic andReal-Time Rendering of Crowds with Dynamic Lighting. ResearchReport RN/06/20, Université Claude Bernard (2007).

[11] Magnenat-Thalmann N. Cordier F. 2005. A data-driven approach forreal-time clothes simulation. Computer Graphics Forum 24 (2005).

[12] O’Conor K. O’Sullivan C. Dobbyn S., Hamill J. 2005. Geopostors: a real-time geometry/impostor crowd rendering system. I3D ’05: Proceedingsof the 2005 symposium on Interactive 3D graphics and games (2005).

[13] Peng C. Dong Y. 2019. Real-Time Large Crowd Rendering with E�cientCharacter and Instance Management on GPU. International Journal ofComputer Games Technology. (2019).

[14] Buchholz H. Döllner J. 2005. Continuous level-of-detail modeling ofbuildings in 3D city models. GIS ’05: Proceedings of the 13th annualACM international workshop on Geographic information systems (2005).

[15] Clark J. H. 1976. Hierarchical geometric models for visible surfacealgorithms. Communications of the ACM 19, 10 (1976).

[16] Dobbyn S. O’Sullivan C. Hamill J., McDonnell R. 2005. Perceptual eval-uation of impostor representations for virtual humans and buildings.Computer Graphics Forum 24, 3. (2005).

[17] Arafhin Mazumder T. Islam R., Islam R. 2010. Mobile Application andIts Global Impact. International Journal of Engineering & Technology,IJET-IJENS, Vol: 10, No:06. (2010).

[18] Bærentzen J. 2005. Hardware-accelerated point generation and ren-dering of point-based impostors. Journal of Graphics Tools 10 (2005).

[19] Sundstedt V. Bala K. Gutierrez D. O’Sullivan C. Jarabo A., Van Eyck T.2012. Crowd light: Evaluating the perceived �delity of illuminateddynamic scenes. Computer Graphics Forum (Proc. EUROGRAPHICS2012) 31, 2. (2012).

[20] Oat C. Gutierrez D. Jimenez J., Echevarria J. I. 2011. Practical andRealistic Facial Wrinkles Animation. GPU Pro 2 (2011).

[21] Collins S. Žára J. O’Sullivan C. Kavan L., Dobbyn S. 2008. Polypos-tors: 2d polygonal impostors for 3d crowds. I3D ’08: Proc. of the 2008symposium on Interactive 3D graphics and games (2008).

[22] Schaefer S. Landreneau E. 2009. Simpli�cation of articulated meshes.Computer Graphics Forum 28, 2 (2009).

[23] Whitted T. LevoyM. (Ed.). 1985. The Use of Points as a Display Primitive.[24] Behr J. Alexa M. Limper M., Jung Y. 2013. The pop bu�er: Rapid

progressive clustering by geometry quantization. Computer GraphicsForum 32, 7 (2013).

[25] Day A. M. Lister W., Laycock R. G. (Ed.). 2009. Geometric Diversity forCrowds on the GPU.

[26] Cohen J. Reddy M. Varshney A. Luebke D., Watson B. (Ed.). 2002. Levelof Detail for 3D Graphics. Elsevier Science Inc., New York, NY, USA.

[27] Thalmann D. Magnenat-Thalmann N., Laperrire R. 1998. Joint-dependent local deformations for hand animation and object grasp-ing. CHCCS/SCDHM, Edmonton, Alberta., Chapter In Proceedings onGraphics interface’ 88, 26–33.

[28] Collins S. O’Sullivan C. McDonnell R., Dobbyn S. 2006. Perceptualevaluation of lod clothing for virtual humans. Proc. of the 2006 ACMSIGGRAPH/Eurographics symposium on Computer animation, SCA ’06(2006).

[29] Coleman D. McLaughlin T., Cutler L. 2011. Character rigging, deforma-tions, and simulations in �lm and game production. ACM SIGGRAPH2011 Courses , SIGGRAPH ’11 (2011).

[30] Rudomin I. Millan E. (Ed.). 2006. A comparison between impostors andpoint-based models for interactive rendering of animated models.

[31] Curtis S. Narain R., Golas A. and Lin M. C. 2009. Aggregate dynam-ics for dense crowd simulation. ACM SIGGRAPH Asia 2009 Papers,

SIGGRAPH Asia ’09 (2009).[32] Sugimoto M. Pantuwong N. 2012. A novel template-based automatic

rigging algorithm for articulated-character animation. Computer Ani-mation and Virtual Worlds 23, 2 (2012).

[33] Badler N. Pelechano N., Allbeck J. (Ed.). 2008. Virtual Crowds: Methods,Simulation, and Control. Morgan & Claypool.

[34] Barham P. Barker R. Waldrop M. Ehlert J. Chrislip C. Pratt D., Pratt S.1997. Humans in large scale, networked virtual environments. Mas-sachusetts Institute of Technology, Chapter Presence, 547–564.

[35] Susin A. Ramirez J., Lligadas X. 2008. Automatic adjustment of rigs toextracted skeletons. Springer Berlin Heidelberg, Chapter In ArticulatedMotion and Deformable Objects, Perales F., Fisher R., (Eds.), vol. 5098of Lecture Notes in Computer Science., 409–418.

[36] Fuhrmann A. Schmalstieg D. (Ed.). 1999. Coarse View Dependent Levelsof Detail for Hierarchical and Deformable Models.

[37] Oat C. Tatarchuk N. Shopf J., Barczak J. 2008. March of the froblins:Simulation and rendering massive crowds of intelligent and detailedcreatures on gpu. ACM SIGGRAPH 2008 Games , SIGGRAPH ’08 (2008).

[38] Pai D. K. Sueda S., Kaufman A. 2008. Musculotendon simulation forhand animation. ACM Trans. Graph. (Proc. SIGGRAPH) 27, 3 (2008).

[39] Lorach T. (Ed.). 2007. Gpu blend shapes.[40] Chrysanthou Y. Tecchia F. 2000. Real-time rendering of densely pop-

ulated urban environments. Springer-Verlag, Chapter Proceedings ofthe Eurographics Workshop on Rendering Techniques 2000, 83–88.

[41] Chrysanthou Y. Tecchia F., Loscos C. 2008. Image-based crowd ren-dering. IEEE Computer Graphics and Applications 22, 2 (2008).

[42] Unity Technologies. 2020. Changelogs for the unity package “Burst”.Retrieved June 8, 2020 from h�ps://docs.unity3d.com/Packages/com.unity.burst@1.2/changelog/CHANGELOG.html

[43] Unity Technologies. 2020. Changelogs for the unity package “Entities”.Retrieved June 8, 2020 from h�ps://docs.unity3d.com/Packages/com.unity.entities@0.2/changelog/CHANGELOG.html

[44] Unity Technologies. 2020. Explanation of Unity’s GameObject. Re-trieved June 8, 2020 from h�ps://docs.unity3d.com/ScriptReference/GameObject.html

[45] Unity Technologies. 2020. Explanation of Unity’s MonoBehaviour. Re-trieved June 8, 2020 from h�ps://docs.unity3d.com/ScriptReference/MonoBehaviour.html

[46] Unity Technologies. 2020. Unity’s own ECS sample GitHub repositorywith description. Retrieved June 8, 2020 from h�ps://github.com/Unity-Technologies/EntityComponentSystemSamples

[47] Unity Technologies. 2020. Unity’s own summary of what DOTS is.Retrieved June 8, 2020 from h�ps://unity.com/dots?_ga=2.55008813.780278623.1590331289-1948476926.1586950390

[48] Musse S. Thalmann D. (Ed.). 2013. Crowd Simulation (2nd ed.). Springer,London.

[49] UL. 2020. Shows a wide range of smartphones with a numberindicator as a standardized measure for their performance. Re-trieved June 8, 2020 from h�ps://benchmarks.ul.com/compare/best-smartphones?amount=0&sortBy=PERFORMANCE&reverseOrder=true&osFilter=ANDROID,IOS,WINDOWS&test=SLING_SHOT_ES_30&deviceFilter=PHONE&displaySize=3.0,15.0

[50] Thalmann D. Ulicny B., Ciechomski P. d. H. 2004. Crowdbrush: Interac-tive authoring of real-time crowd scenes. Proceedings of the 2004 ACMSIGGRAPH/Eurographics symposium on Computer animation, SCA ’04(2004).

[51] Wikipedia. 2015. Thorough description on the speci�cations of the iPhone6S. Retrieved June 8, 2020 from h�ps://en.wikipedia.org/wiki/IPhone_6S

[52] Wikipedia. 2020. Thorough description on compatibility and releasesof Xcode. Retrieved June 8, 2020 from h�ps://en.wikipedia.org/wiki/Xcode

[53] Wikipedia. 2020. Thorough description on the speci�cations of the iPhoneXR. Retrieved June 8, 2020 from h�ps://en.wikipedia.org/wiki/IPhone_

11

XR[54] Alexa M. Hormann K. Winkler T., Drieseberg J. 2010. Multi-scale

geometry interpolation. Computer Graphics Forum 29, 2 (2010).

[55] Ni L. Zheng P. (Ed.). 2006. Smart Phone and Next Generation MobileComputing. Morgan Kaufmann.

12

TRITA -EECS-EX-2020:490

www.kth.se