Applicability of modern graphics libraries in

web developmentHow may current graphics APIs that allow GPU-rendered web content be

better inorporated for use in modern web application production?

Daniel Nordström

Computer Game Programming, bachelor's level

2019

Luleå University of Technology

Department of Computer Science, Electrical and Space Engineering

LULEÅ UNIVERSITY OF TECHNOLOGY

Applicability of modern graphics librariesin web development

How may current graphics APIs that allow GPU-rendered web content be betterinorporated for use in modern web application production?

Author:Daniel NORDSTRÖM

Supervisor:Patrik HOLMLUND

Bachelor of Science in Engineering TechnologyDepartment of Computer Science, Electrical and Space Engineering

June 12, 2019

iii

Luleå University of Technology

AbstractFaculty Name

Department of Computer Science, Electrical and Space Engineering

Bachelor of Science in Engineering Technology

Applicability of modern graphics libraries in web development

by Daniel NORDSTRÖM

This thesis presents an exploration into current web browser technologies for graph-ics development, and offers a framework-like solution to integrate WebGL basedgraphical features into any web application based on those findings. It is builtlargely of the 2017 investigative graduate work done at Explizit Solutions (an ITfirm based in Skellefteå, Sweden), where the goal was to discover how 3D graphicstechnology in web browsers could be incorporated into and improve the front-endof their booking system services. A refined version of the solution produced in thatwork is presented, discussed and evaluated in this dissertation along with the inves-tigative work done to produce it.

v

Acknowledgements

This project would not have been possible if not for the contribution of thesepeople. I hereby give thanks to:

Fredrik Bergqvist,for affording me the oppurtunity of this graduate work at Explizit.

Jens Vestermark,for providing me with Explizit’s tools and helping me set up a work environment.

Aksel Kornesjö,for providing advice in aspects of web development.

All others at Explizit,who contributed to my stay and made my graduate work there that much more

valuable.

Mom and Dad,for encouraging me to follow through with the paper.

Connie,for motivating me to actually follow through on it.

Patrik Holmlund,for his great patience and understanding in overseeing this thesis.

vii

Contents

Abstract iii

Acknowledgements v

1 Introduction 11.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.1.1 Web Browsers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.1.2 Explizit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.2 Goals and Purpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.2.1 Task . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.2.2 Thesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

1.3 Current State of Browser Rendering Technology . . . . . . . . . . . . . 31.3.1 Rendering the DOM . . . . . . . . . . . . . . . . . . . . . . . . . 31.3.2 Browser Usage of the GPU . . . . . . . . . . . . . . . . . . . . . 31.3.3 Advantages of Rendering with WebGL vs DOM . . . . . . . . . 41.3.4 WebGL Based Tools and Libraries . . . . . . . . . . . . . . . . . 4

1.4 Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51.4.1 Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51.4.2 Work Environment . . . . . . . . . . . . . . . . . . . . . . . . . . 6

1.5 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61.6 Social, Environmental and Ethical Concerns . . . . . . . . . . . . . . . . 7

2 Design and Implementation 92.1 Selecting Tools and Libraries . . . . . . . . . . . . . . . . . . . . . . . . . 9

2.1.1 HTML with CSS . . . . . . . . . . . . . . . . . . . . . . . . . . . 92.1.2 Selecting a Renderer . . . . . . . . . . . . . . . . . . . . . . . . . 10

2.2 Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112.2.1 Integration into Adoxa While Remaining Codebase Agnostic . 112.2.2 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

2.3 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122.3.1 Overall Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

View Side . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13Shading Side . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

3 Project API and Method Manual 173.1 Setting Up The ThreeView . . . . . . . . . . . . . . . . . . . . . . . . . . 173.2 Adding a Mesh To The View . . . . . . . . . . . . . . . . . . . . . . . . . 183.3 Adding a ScriptedMaterial to a Mesh . . . . . . . . . . . . . . . . . . . . 18

4 Results 214.1 Adoxa: Integration and Development . . . . . . . . . . . . . . . . . . . 21

4.1.1 Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214.1.2 Adoxa Front-End Enhancements . . . . . . . . . . . . . . . . . . 21

viii

4.1.3 Explizit DevTeam Opinion . . . . . . . . . . . . . . . . . . . . . 224.2 NodeJS: Integration and Further Prototyping . . . . . . . . . . . . . . . 23

4.2.1 Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234.2.2 Interactable 3D Object . . . . . . . . . . . . . . . . . . . . . . . . 234.2.3 Shading with Multiple Materials . . . . . . . . . . . . . . . . . . 24

5 Discussion 275.1 Technical Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

5.1.1 Project and Codebase Structure . . . . . . . . . . . . . . . . . . . 275.1.2 API Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285.1.3 Other Limitations and Future Improvements . . . . . . . . . . . 29

5.2 Usage and Application Review . . . . . . . . . . . . . . . . . . . . . . . 295.2.1 Solution Distribution . . . . . . . . . . . . . . . . . . . . . . . . . 295.2.2 Integration into a Web Application . . . . . . . . . . . . . . . . . 305.2.3 Use Within JS Framework . . . . . . . . . . . . . . . . . . . . . . 305.2.4 Useage in Front-End Development . . . . . . . . . . . . . . . . . 31

5.3 Potential of Graphics in Web Application Development . . . . . . . . . 31

6 Conclusion 33

A Animated DOM with CSS and JS 35

ix

List of Figures

1.1 The general flow of web browsers’ rendering pipeline. Image by TaliGarsiel. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

1.2 The thesis work process stages and transioning. . . . . . . . . . . . . . 5

2.1 The blocks displayed are DOM entities animated using CSS3 and na-tive JS [Appendix A]. They animate left to right, shifting in color asthey move. As is seen the screenshot also includes a test environment,with only one external library used for the FPS-meter[11]. . . . . . . . . 10

2.2 The MVC pattern that ASP.NET MVC adheres to visualised. TheController updates model and/or view, and the view displays themodel[16]. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

2.3 The ThreeView application cycle per frame. . . . . . . . . . . . . . . . . 132.4 Diagram in UML notation of the ’view’ side of the framework. . . . . . 142.5 Diagram in UML notation of the ’shading’ side of the resulting frame-

work. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

3.1 Code example of initialising the ThreeView. . . . . . . . . . . . . . . . . 173.2 Code example of loading a mesh from file and adding it to a scene. . . 183.3 Code example of loading a ScriptedMaterial, setting its parameters

and attaching it to a mesh. . . . . . . . . . . . . . . . . . . . . . . . . . . 19

4.1 The ASP.NET Razor code in the view that includes the project essen-tials. The " /ThreeJS" bundle includes ThreeJS core library as wellas extensions to it, wheresas the " /GraphicsWorkflow" includes ownmodules developed for this thesis as well as non-ThreeJS related li-braries eg TweenJS. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

4.2 Enhanced Adoxa login page. Green product logo in 3D courtesy ofClara Lundwall. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

4.3 Project setup in Node with ExpressJS. On the right is shown the codesetting up the NodeJS server and on the left the index HTML filewhich includes all the library scripts. Note that the solution root di-rectory is referred to as "midas". . . . . . . . . . . . . . . . . . . . . . . 23

4.4 Screenshots of the live demo described in this subsection. The toptwo demonstrate the behaviour of the sqaure DOM elements: the bluemoving side to side and the green expanding then shrinking. Thebottom two demonstrate the 3D Adoxa logo mesh mimicking theirbehaviour when synced. . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

4.5 Initialization of rendertargets used in this demo. The ThreeView mod-ule sets the rendertarget of a scene by the fourth string argument inthe scene constructor. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

4.6 Screenshot of the untouched textured plane above a screenshot of thesame but drawn on by the user. . . . . . . . . . . . . . . . . . . . . . . 25

xi

Terms and Abbreviations

AJAX Asynchronous Javacript And XMLAPI Advanced Programmer’s InterfaceASP.NET Microsoft web application frameworkASP.NET MVC Framework that extends ASP.NET and encourages application

development in a MVC patternNodeJS A JS web server APIASP.NET Razor Server-side markup language that allows embedding VB and C# into

web pagesBabylonJS, ThreeJS Graphics web APIs that provide a more accessible abstraction of WebGLC-API API that offers functions for code written in CExpressJS Minimalist web framework for Node.jsFramework Often conflated with library, a framework calls your code rather than a

library whose code you call. Frameworks can provide libraries to workwith but they are not libraries themselves. This fundamentaldifference tends to be quite subtle in practice, hence the tendencyfor conflation.

FX Effects, delivered to the users senses by audio, vibration or visualsGL Graphics LibraryGPU Graphics Processing UnitGUI Graphic User InterfacejQuery A DOM JS extension library that eases manipulation of DOM elementsJS JavascriptLibrary An API that presents the developer with a collection of functions or

entities to simplify certain programming tasksMDU Mobile Device UserMSDN MicroSoft Developer NetworkMVC Model View Controller, a design pattern commonly used in web

developmentNPM Node Package ManagerOO Object OrientedOOP Object Oriented ProgrammingPixi A HTML extension that provides developers with GPU accelerated

rendering of website contentTS TypeScriptUI User InterfaceUX User eXperienceWorkflow An intended methodology to apply when developing inside a frameworkXML eXtensible Markup Language

1

Chapter 1

Introduction

1.1 Background

1.1.1 Web Browsers

The internet is a worldwide network of computers that share content with each otherby a set of rules specified in the Internet protocol suite. Web browsers are softwareapplications used to locate and display digital content via the internet, and are to-day the most common way for individuals to access such content. The browserprimarily does so through the display of web pages: GUIs with internal function-ality that can navigate to other webpages. In spite of increases in internet trafficthrough mobile devices, and MDUs (Mobile Device Users) preferring native appsto the browser[21][30], browser usage has seen a steady linear growth by roughly200 million users every year since 2005[20]. As this was long before the rise of themobile app market, it suggests that browsers have merits of their own that nativeapps cannot achieve.

An important such merit is the fact that the content browsers provide does not haveto be downloaded and installed before access, requring time and storage which aremajor issues for MDUs[27]. The web’s content accessibility comes coupled with theconsistent improvements to browser-related technologies over the years[23]. Usinga browser today is a very different experience than it was in the past. Improve-ments to browser-related software technology have allowed for the development ofprogramming APIs and frameworks that enable websites to perform comparably todevice-specific applications. One such development was the introduction of WebGL,which opened up development advanced graphics features in the web on par withnative apps.

1.1.2 Explizit

Explizit is a concern in which the companies work in the development and distri-bution of administrative IT tools in the form of desktop and mobile applications,as well as web services accessible through a web browser[4]. They serve a vari-ety of Swedish and foreign customers with whom they maintain close relations asto provide technical support and to receive requests and feedback regarding theirproducts.

A subsidiary solutions department, Explizit Solutions, develops and maintains thesevarious products. This department consists of a core development team along with

2 Chapter 1. Introduction

management and auxiliary workers. The work that laid the foundations for this pa-per was done at this department, which provided me with developer access to theirweb products as to explore possible implementation of web graphics technology toimprove their products.

1.2 Goals and Purpose

1.2.1 Task

Explizit are always looking to developments in software technology and their po-tential use in the production of their own products. A relatively new developmentis the introduction of WebGL and the many libraries and frameworks built over itsince then, making the development of web content rendered in real-time 3D graph-ics much more accessible and logistically possible.

Explizit are interested in the potential benefits of incorporating 3D graphics intoAdoxa, a by Explizit developed and owned full-stack web tool for a wide range ofbooking and scheduling related tasks. Due to the uncertain profitability of such aventure it was offered up as possible internship job, and an oppurtunity for thisgraduate work. The work consisted firstly of an investigation into what program-ming tools and APIs are suitable to use within the Adoxa environment, as well ashow to actually integrate them and develop working features in the Adoxa front-end. In the end producing a demo version of Adoxa showcasing these features.Ultimately the goal was to discover any potential in developing 3D graphics contentfor Adoxa and other Explizit web-distributed products.

1.2.2 Thesis

This thesis draws upon the work described in the previous 1.2.1 subsection. Thework is an investigation with the goal of answering the key question of how 3Dgraphics may be implemented and used in Explizit’s software products. To discoverthis entails answering other questions. What libraries and/or frameworks exist toallow for 3D graphics development in the browser? How would these be integratedinto Explizit’s existing software? What potential features could be developed withthese?

While these questions have merit on their own they do not suffice as research ques-tions since they are very case-specific. A more broadly applicable yet focused re-search question would be how browser front-end APIs could be used to improveoverall UX in websites, which falls in line with the original goal of the task givenby Explizit. As the task was not so strictly defined however, the nature of the workbecame increasingly how to integrate the APIs in a way that would allow devel-opers without any experience in graphics programming to implement features thatutilise these. Hence the research question: How can current APIs that enable GPU-rendered web content be integrated into a workflow useable by any web developer,and how may this be applied to improve a website’s front-end?

The thesis thus hopes to accomplish an evaluation of the potential of web browsergraphic APIs to improve the front-end of web applications. Such an evaluation will

1.3. Current State of Browser Rendering Technology 3

be drawn from the work done at Explizit: an investigation into available web de-velopment tools, how to apply them to a modern web application and a review oftheir potential use in front-end web development. Then finally to provide a possiblesolution that integrates web graphics APIs into any web solution.

1.3 Current State of Browser Rendering Technology

In terms of rendering the internet browser was originally purposed to interpret anddisplay static documents. As browsers have evolved so to have their rendering capa-bilities. This section will provide exposition as to how browsers perform renderingand the implications this has for developers, essentials for understanding the con-tent of this thesis.

1.3.1 Rendering the DOM

The DOM is the de facto means of defining webpage content, complemented bystyle sheets that specify the visuals of said content. It describes webpages in a treestructure with dependent parent-child nodes. To display webpages browsers mustparse DOM and style sheet files to then interpret the combined data to form a render-able data structure [Fig. 1.1]. This structure is a data tree, referred to as the ’rendertree’ in the Safari renderer Webkit and the ’frame tree’ by Mozilla’s Gecko renderer.Whereas terminology and details in implementation vary all major browsers essen-tially follow the same flow for rendering webpages[5].

FIGURE 1.1: The general flow of web browsers’ rendering pipeline.Image by Tali Garsiel.

This is not an entirely linear process as it is desirable to start rendering web contentbefore the entire page has been parsed and the tree structures are completed. Hencefor a better UX the rendering process is not executed sequentially, but prioritizescompletion of individual tree nodes from parsing to painting in order to displayweb content as quickly as possible.

1.3.2 Browser Usage of the GPU

The most common and most widely supported means of sending data from thebrowser to the GPU is by WebGL. The only other real contender is Adobe’s Stage3Dwhich comes with the Flash11 browser plugin. However, WebGL is a JS-native API(no Flash plugin required) and it is supported on a wider range of devices andbrowsers[31]. Although Stage3D had the advantage of performance WebGL hasimproved to where it outperforms the former in a many cases[6]. As they com-pare today WebGL is thus the preferrable API to use when developing graphics forbrowser applications.

4 Chapter 1. Introduction

Chromium based browsers today such as Chrome and Opera actually utilise whatGoogle calls GPU accelerated compositing when rendering the DOM tree[33]. Itspurpose is to speed up rendering of DOM in certain use cases. However, as it is en-tirely implemented within the browser’s own rendering engines it is not accessibleto the web developer in any way.

Using WebGL a web developer can gain access to rendering with the GPU. To dothis a WebGL context is set up within the DOM which draws to a canvas element[8].WebGL is effectively an OpenGL ES port to the web browser, a graphics librarymaintained by Khronos for embedded devices such as smartphones[7], and henceprovides the same low-level graphics programming but in a browser-native JS API.

1.3.3 Advantages of Rendering with WebGL vs DOM

Browsers are primarily aimed at displaying static documents in what the user seesas a webpage. A task for which they utilise the CPU, except for Chromium basedbrowsers with GPU accelerated compositing enabled. Much of the processing poweris spent towards the document layout by interpreting the DOM tree describing awebpage. Browsers are thus optimised to deal with visually static applicationswhere the displayed elements maintain constant dimensions, style and, more impor-tantly, placement in the DOM tree. While convenient for this, the DOM is howeververy poor at coping with dynamic content such as visual effects or animated ele-ments. A single changed element in the DOM tree can cause a ripple of events thatforces the browser to recalculate properties of other elements connected to it in thetree[25]. This renders DOM useless for certain types of applications such as games.The performance issues of DOM become especially noticeable on less powerful de-vices such as phones and tablets.

The browser’s DOM rendering pipeline and WebGL were designed for very dif-ferent purposes. Whereas DOM is a convenient way of handling more or less staticdocuments WebGL opens up the features of OpenGL to the browser[14]. It expandsthe browser’s capabilities to include a slew of options for how the application mayinteract with the GPU. It enables the rendering of high quality real-time graphics inboth 2D and 3D at 60 FPS along with many new ways for the user to interact withthe browser. In short, WebGL (or Stage3D) allows browser application features thatwould be implausible with just the DOM rendering pipeline.

1.3.4 WebGL Based Tools and Libraries

The WebGL API is essentially an OpenGL interface that provides web developerswith graphics programming on the same low-level functionality as the C-API. Whilepowerful, it is relatively verbose and can be quite daunting for most web developers,and for most purposes it is too cumbersome to be useful. However, since its releasein March 2011[9], a number of powerful libraries have been built on top of WebGLto ease the development process.

PixiJS is WebGL-based 2D graphics engine which is renowned for being the high-est performing 2D renderer available for the browser[26]. It provides all the power

1.4. Method 5

of WebGL in a 2D-rendering format, but while it is much easier to learn than WebGLthere is still a considerable learning curve that would put off some web-developersand many web designers.

If the only concern is to render website content using the GPU then the open sourceHTML-GL framework might be the easiest to use for traditional web developers[24].It allows DOM elements to be rendered in WebGL by simply surrounding them with<html-gl> tags. It does not offer any interface for further graphics programming, butdoes expose the DOM element to other WebGL based frameworks such as PixiJS. Assuch however it does not stand well as a sole tool for developing WebGL as most de-veloper will choose a WebGL based tool for the purpose of developing more uniquevisual features.

Another very popular, this time 3D-capable WebGL-based library is the open sourcerenderer Three.js[17], a WebGL wrapper library that vastly eases the process of set-ting up a graphics rendering pipeline of a web-application. It is used in manybrowser graphics applications and tools [32][36] and has a very active developercommunity around it. It also has a slew of add-ons, like the CSS3DRenderer whichallows adding DOM elements into the WebGL scene as interactable 3D objects[3]. Itis highly versatile, being intended for use in all kinds of interactive 3D web applica-tions with an easy to use graphics API that still provides web developers with thesimilar low-level control that programming directly in WebGL does.

1.4 Method

1.4.1 Process

This thesis work can essentially be divided into five stages: research, design, im-plementation, integration and demonstration [Fig. 1.2]. The initial research stageconsists of looking into what web tools and APIs exist for 3D graphics and auxiliarytasks relevant to the goal set by Explizit. In the design phase these are then used toset up the design of a web developer’s tool, likely a library or a framework, and thenimplement this design to produce something that can be integrated into the productcodebase. After integration there comes the development of new front-end featuresto Explizit’s Adoxa applicaton, to showcase its successful useability and potential asa future tool for web development.

FIGURE 1.2: The thesis work process stages and transioning.

The programming tools and APIs used in the development of this project varied,but it was all done on a windows 10 PC DELL XPS Laptop and using the Chromebrowser for researching and JS debugging, as well as Firefox and IE11 for furthertesting. The programming tools used at first in the research and design stages were

6 Chapter 1. Introduction

mostly Atom, Notepad++ and the windows command line, as it mostly involved set-ting up simple JS servers or just interpreting an HTML file locally with the browser.As such the brunt of the work consisted of editing relatively small text files, andrunning simple cmd commands for installation and configuring projects such as wasdone for testing HTML-GL and Voodoo.JS.

In proceeding with implementation and the later stages it became necessary to useVS as the primary IDE, since it is the environment of choice by the devteam at Ex-plizit. For development of own examples outside of Adoxa a NodeJS server withNPM was used, with Atom as the editor of choice.

1.4.2 Work Environment

Due to the task being quite removed from the current activities of the Explizit de-vteam, the work was viewed more as R&D than actual production. Given this therewas no need for a structured workflow such as scrum or agile meant to streamlineworking in a team. It was thus more of an individual endeavour provided for byExplizit in which they had vested interest in.

The company provided quiet work location, necessary hardware, software licensefor VS as well as a forked version of Adoxa along with technical support. An ex-tremely flexible working schedule was enabled given 24-hour access to locale andphysical resources, and due to the independent nature of the work there were onlya few predetermined checkups and presentations.

1.5 Related Work

There has been numerous papers written on the topic of applying WebGL int theirwork across institutions the world over, particularly for purposes of representingreal-world 3D structures to aid in fields of natural sciences, medicine and engineer-ing. Similarly to the work this paper presents, a number involve developing toolsfor individuals of varying professions without experience in graphics programmingas to integrate 3D-visual graphics into their applications.

WebGL has seen particular use in the development of tools for molecular sciences,with the development of numerous APIs for rendering and interacting with molec-ular data in the browser. An interesting such is 3Dmol.js presented in a paper pub-lished in 2014 by Nicholas Rego and David Koes[29], essentially a rework of theGLmol API made available in 2012[35]. The newer 3Dmol.js differs from GLmol inthat it is written in WebGL instead of ThreeJS GLmol is based on. Since ThreeJS it-self is a WebGL abstraction, developers gain access to lower-level control of the GPUwith WebGL, but ThreeJS is of naturally easier to use. A later paper from 2017 byMaoxiang Shi, Juntao Gao, and Michael Q. Zhang present a more modern modernsolution for rendering molecular structures in the browser, opting as well to pro-gram directly in WebGL for gains in performance[15].

Other fields have also seen papers presenting web applications based on WebGL

1.6. Social, Environmental and Ethical Concerns 7

technology. One such published by authors from the Department of Computer Sci-ence and Engineering at the University of Mauritius describes a ThreeJS web appli-cation, which takes the user through a virtual tour of the university campus. It is aproof of concept for presenting users a life-like 3D view of any existing location, ofpotential interest to architects or workers in real estate[28].

Another paper of interest to this thesis work is one published in 2016 which presentsa 3D engine of similar function to ThreeJS. It describes the architecture and imple-mentation of such an engine and compares a prototype to ThreeJS and BabylonJS.The paper states the prototype’s "performance was very close to" the two establishedWebGL renderers[37]. This is interesting given that it shows it is still feasible to pro-gram graphics in WebGL, though an established engine such as ThreeJS or BabylonJSis likely to be just as or even better in performance.

1.6 Social, Environmental and Ethical Concerns

This project is solely focused on the visual and functional aspects of web front-endswhich already limits potential environemental, moral and ethical concerns . Firstlyit does not require any actual user data and hence concerns regarding user privacyand anonymity do not apply. The software provided by Explizit only uses test sam-ples where relevant. All the developmental work using their products is done onsite, to minimize the chance of sensitive company information being leaked.

The main environmental concern would be the heightened energy usage of UIs thatrely on 3D graphics pipeline. Since one of the aims of the project is to encourageincreased useage of such in building everyday web applications this project may bea contributing factor to heightening the global power useage of the world wide web.However, given the experimental nature of the project, this is a premature concernthat is not applicable unless the project solution actually reaches widespread useage.

9

Chapter 2

Design and Implementation

2.1 Selecting Tools and Libraries

A large portion of the design process was to look at existing tools and libraries thatmay be used to improve the Adoxa front-end. Subsequent design decisions dependentirely on how the chosen APIs are utilised and thus the choice needs to be quiteheavily justified. This section looks at the advantages and disadvantages of varioustools for rendering in the browser, and presents reason for selecting or disregardingsaid tools for later work stages.

2.1.1 HTML with CSS

Despite the features CSS today provides it is desirable to seek alternatives. Althoughanimations and shading effects can be achieved with CSS it is not so efficient due tothe DOM bloat that naturally occurs when developing larger websites. It quickly be-comes quite cumbersome upon implementing even slightly complicated effects, asdemonstrated by the following example which renders multiple visual effects usingonly CSS [Fig. 2.1].

In this example a DOM-element’s ’transform’ class-attribute are altered by two dif-ferent operations simultaneously: rotation and translation. The shortcomings of pro-gramming even slightly complex visuals using native web tools quickly becomesapparent here, as a new DOM child has to be added for each property we want toalter on the object. Even though position, scale, rotation and opacity can be resolvedquickly through hardware compositing other attributes are more costly to calculatein standard browser tools[13]. Further tests with this example involving more DOMelements showed massive deteriorations in performance, with values reaching 1-5FPS in when rendering anything more than 500 elements.

Hence it is much easier to render the application graphics in a separate WebGL con-text which does not trigger changes in the DOM, and thus suffer in performance.This approach ensures minimal modifications to existing webpage DOM and allowsclearer separation between that and code for GPU-rendered graphics. This also pro-vides a better chance of not bloating the website’s script and mark-up, which is vitalfor integration into larger web projects such as those of Explizit.

10 Chapter 2. Design and Implementation

FIGURE 2.1: The blocks displayed are DOM entities animated usingCSS3 and native JS [Appendix A]. They animate left to right, shiftingin color as they move. As is seen the screenshot also includes a test en-vironment, with only one external library used for the FPS-meter[11].

2.1.2 Selecting a Renderer

As stated in the introduction there exists a number of libraries to streamline the taskof rendering with WebGL. A number of these will be considered, but in the end it iseasiest to use just one for implementation. The limited project time also disalloweindeterminate time spent on research and testing of the many WebGL frameworksavailable, and thus this selection is made on somewhat loose grounds. The primaryconsiderations are thus the primary features it provides, how good of a documenta-tion it has and its ease of use.

Pixi is versatile engine that can very easily be integrated into any website[12]. Be-ing 2D allows it to fallback to a Canvas element, albeit at reduced performance cost,so compatibility with older browsers is easily programmable. Pixi also has its ownworkflow for shading, which it does through use of what it calls ’filters’. This sim-plifies shader usage somewhat but in reality functions more as to integrate shadinginto the Pixi framework, so that the graphics programming all falls under one API.

A an HTML extension library built using Pixi called HTML-GL provides websiteauthors with the ’html-gl’ tag[24]. Any elements of such tag get put through theHTML-GL pipeline which itself uses Pixi to render with WebGL. In effect this be-comes the single easiest way to render anything manually with WebGL in the browser.These elements are then also accessible via script to the Pixi API, allowing them tobe manipulated by filters and other features of the graphics API.

The most obvious disadvantage Pixi (and by extension HTML-GL) faces is that it isalmost entirely built around the concept of managing and rendering sprites. Trans-formations in 3D are not inherently supported and thus to achieve 3D visuals it be-comes very tricky if not impossible. Although this is a non-problem for the majorityof browser applications, Explizit explicitly wished to test the potential of 3D graph-ics, which means that at best these libraries could be used at best to complement toany solution.

2.2. Design 11

To render in 3D there exists a number of WebGL renderers available. However, themajority of these were designed to serve as renderers for game engines[10][34][18].There are a few engines that are able to satisfy more diverse needs such as Baby-lon.js, but one that stands out in particular is ThreeJS. Which is the chosen rendererto work with here.

The ThreeJS engine is by all definitions the most popular WebGL based 3D rendererwith the most diverse set of features[18]. It enjoys a large following on GitHub withnumerous comprehensive guides, examples, demos and complete documentationon every core aspect of the engine. Despite being highly streamlined on the sur-face it leaves a lot of room for developers to build entire new frameworks on it, e.g.WhitestormJS[1]. Overall ThreeJS is the choice due to its ease of use, its versatilityand wide selection of features.

2.2 Design

Here is presented the design process, and the decisions made that shaped the imple-mentation.

2.2.1 Integration into Adoxa While Remaining Codebase Agnostic

This is the primary concern of the project in terms of its system architecture. Adoxais built on the Microsoft ASP.NET MVC framework, which is a subset of the .NETframework used to develop web applications. ASP.NET MVC is in turn a frameworkbased on ASP.NET.

FIGURE 2.2: The MVC pattern that ASP.NET MVC adheres to visu-alised. The Controller updates model and/or view, and the view dis-

plays the model[16].

The fundamental idea behind the MVC pattern is to separate data from presenta-tion. This pattern is a common choice for dealing with data over a client-serversystem, and can be better understood by observation of its use in practice, for exam-ple by ASP.NET MVC. The framework takes a clear OO approach in implementingthe MVC pattern [Fig. 2.2], providing the developer with OO entities like classesto accomplish this. Classes inheriting the controller class contain methods as event

12 Chapter 2. Design and Implementation

handlers. It can alter the model before returning a view that is compiled server-side before being sent over as a webpage. Any additions or modifications to Adoxashould conform as far as possible to this design pattern. A server built strictly withASP.NET MVC would have all network requests handled by methods in a controllerclass with corresponding model class instances accessible to the view before compi-lation.

The best way to ensure integratability into such a design would be to localize theimplementation to the view as far as possible. Once it is part of any working webapplication it should ideally only be necessary for the developer to alter code in theview to build graphical features. This can be accomplished by providing a JS APIfor the client browser.

2.2.2 Summary

The project will hence be a front-end JS framework constructed with an OO ap-proach to ease integration with the Adoxa framework. It will attempt to separatethe graphics programming from the webpage mark-up as to provide the web-pageauthor with a more simplified workflow than just vanilla ThreeJS. The idea is toallow a webpage author to develop, modify or optimise their websites’ graphicalfeatures. The intended result is a small (as of now) library that allows web devel-opers of minimal experience in computer grahics to inject graphical enhancementssuch as animated 3D-models and shading into their website.

The OO design is achieved through use of the javascript module pattern[2], whichcan effectively emulate the concept of class. The pattern makes use of the language’sclosure feature to maintain closed scope ’hidden’ variables related to a specific ob-ject in addition to that object’s properties. This essentially allows for structuringcode using conceptual classes with private and public member fields and functionsin a manner that is inherent to OOP languages such as C#. This mimics the designphilosophy of the .NET framework into which integration is a primary goal of thisproject. It further renders the code more apprehensible to programmers familiarwith OOP, and allows for more efficient code structuring by separation of concepts.The UML diagrams here are not direct representations of the code as they may havebeen if the code was written in a strictly OOP language but rather show the conceptsbehind it.

2.3 Implementation

The implementation can be described as a prototype JS framework, which aims toseamlessly integrate hardware-accelerated graphics into a web front-end.

2.3.1 Overall Structure

The code can be split into two overall sections: the main ’view’ side and another’shading’ side. The ’view’ side provides the webpage author with streamlined man-agement of ThreeJS scenes, geometric entities in the scenes and rendering of these.The ’shading’ side deals with shaders and associated scripts for dynamic shading,

2.3. Implementation 13

and provides the webpage author access to shading parameters and the materialdata structure attachable by ThreeJS to a mesh for shading. These sides are com-pletely independent of each other, meaning one can be included and used in a projectwithout the other’s presence. Although possible, this is not recommended as theywere developed with conjunctive use in mind. Otherwise they both require ThreeJSas these are essentially wrappers to that library’s features.

View Side

The ’view’ side It provides the web developer with an API to seamlessly integrateThreeJS scenes and objects alongside the DOM. It accomplishes this by maintaininga separate application cycle for rendering in WebGL [Fig. 2.3].

FIGURE 2.3: The ThreeView application cycle per frame.

All the essentials for rendering with ThreeJS are provided for the developer throughthe ’ThreeView’ object [Fig. 2.4]. ’ThreeView’ is a global object that maintains scenes,entities rendered by these and other aspects directly associated with these such asscripting, animations, cameras and rendering.

Though largely independent ’ThreeView’ draws use of the external ’ThreeUtils’ mod-ule. This is a namespace that provides ’ThreeView’ and the webpage author withadditional functions. It also contains the geometryFileLoader: a module for loadingThreeJS geometry from a mesh file like .stl and caching it for later use.

The scene and sceneObject modules are object constructors defined in ’ThreeView’and are private in scope. A ’ThreeView’ ’scene’ instance wraps a ThreeJS scene in-stance as the ’content’ field and contains references to the camera and renderer in’ThreeView’ which ThreeJS uses to render the scene. The ’ThreeView’ ’sceneObject’serves as a reference to a particular entity within a scene, but it also contains entity-specific functionality such as event callbacks. The ThreeView uses JS-objects withnamed properties as associative arrays for storing scenes, sceneObjects, per-framefunctions, renderers and cameras.

14 Chapter 2. Design and Implementation

FIGURE 2.4: Diagram in UML notation of the ’view’ side of the frame-work.

Shading Side

What is here called the shading side of the project is a small pipeline for manag-ing ThreeJS materials and scripting associated with these [Fig. 2.5]. It is meant tostreamline the process of creating and attaching materials to objects in the scene,and to offer a preestablished way of separating shading code from scene and geom-etry code.

The ’scriptedMaterialFactory’ is a global object that serves the webpage author in-stances of a ’ScriptedMaterial’. The function ’AddScriptedMaterial’ attempts to fetcha ’ScriptedMaterial’ from the server, and calls the passed ’callback’ upon success.The material is then available to the webpage author by ’GetNewScriptedMateri-alInstance’ which returns a new instance of the fetched ’ScriptedMaterial’. Just like’ThreeView’ uses associative arrays to keep track of scenes and scene objects the’scriptedMaterialFactory’ uses an associative array to keep track of scripted materi-als.

2.3. Implementation 15

FIGURE 2.5: Diagram in UML notation of the ’shading’ side of theresulting framework.

A scripted material is a constructor function defined in a file at the path given by’materialsPath’ in ’scriptedMaterialFactory’, and must adhere to the structure givenby the ’ScriptedMaterial’ example in Figure 3.3. A ’ScriptedMaterial’ works as awrapper to a Three.Material with additional scripting capability. These scripts aredefined in or called by functions ’init’ and ’update’, which determine initial andruntime material behavior respectively by altering members of ’shaderVars’ whichThreeJS uses to determine value of shader uniforms.

17

Chapter 3

Project API and Method Manual

The JS API developed in this project was developed with an intended method of us-age in mind. This section serves as an introduction to said method and thus how todevelop graphical features with the API, by instruction and viewing code examples.

3.1 Setting Up The ThreeView

Initialising the ThreeView for the API user is an easy process [Fig. 3.1]. Firstly’ThreeView.init()’ detects if the browser supports WebGL and returns false if not.If true then two default renderers, a CSS3-renderer and a WebGL-renderer are cre-ated and made to fit the window. Default cameras are also instantiated along withother ThreeJS utilities such as the raycaster. As visible ’ThreeView.cameras’ is publicin scope and editable by the webpage author; ’ThreeView.renderers’ and ’.update-FunctionsHolder’ are also editable with runtime consequences. Unlike these, how-ever, editing ’clearColor’ has no effect after ’init’ is called. Global callbacks to which’ThreeView’ needs to respond are also registered here, and necessary update func-tions are defined.

FIGURE 3.1: Code example of initialising the ThreeView.

’ThreeView.addScene’ instantiates a ’scene’ object with the given parameters andinitialises its ’content’ member to an empty ’THREE.Scene’. The first parameter isthe name or label used to refer to the scene, the second and third are respectivelywhich renderer and camera to use when rendering the scene. The scene can later beaccessed by its ’name’ by calling ’thisView.getScene(name)’.

Once initialised the ’ThreeView’ application cycle can be started by calling ’update’,which calls ’requestAnimationFrame’ recursively to in order to actually loop. With-out calling this the application won’t update and remain static.

18 Chapter 3. Project API and Method Manual

3.2 Adding a Mesh To The View

Loading a mesh into the view can be done with very few steps [Fig. 3.2]. The ’Three-Utils.geometryFileLoader’ takes two parameters: the path to the file containing thegeometry, and a function to be used as callback upon successful loading of the ge-ometry. ThreeJS includes a suite of different geometry file parsers; the ’geometry-FileLoader’ checks the file extension of the given path and calls the appropriateparser. If no available parser matches the file extension a log is made and noth-ing else happens. If the parser successfully loads and parses the file the geometry iscached by the ’geometryFileLoader’ to later be accessed by the ’get’ member func-tion.

FIGURE 3.2: Code example of loading a mesh from file and adding itto a scene.

The callback function used above retrieves the cached geometry, creates a THREE.Meshobject with it and adds it to the scene with the label passed as argument. The’addSceneObject’ function adds the provided mesh to the content of named ’Three-View.scene’, and creates a ’ThreeView.sceneObject’ that references it. The ’geome-tryFileLoader’ cache is then cleared.

3.3 Adding a ScriptedMaterial to a Mesh

This section describes how to add a material with scriptable behaviour to the Three-View [Fig. 3.3]. The example provided here is attached to a plane geometry meshmeant as a dynamic window background.

Firstly the ’AddScriptedMaterial’ function makes a GET-request to the server usingan AJAX call to acquire the scripted material file. The first parameter is the name ofthe file, which the call will append to the private ’materialsPath’ variable in orderto get a full path to the file. If the script is loaded successfully it will then load theshaderprogram script in the same fashion, resolving the full path by appending the’scriptedMaterial.shaderprogram.name’ to the ’shadersPath’. Upon succeeding inthis the program will then proceed to execute the callback function.

By this time the scripted material is available to the web session and can be in-stantiated calling ’scriptedMaterialFactory.GetNewScriptedMaterialInstance’. This

3.3. Adding a ScriptedMaterial to a Mesh 19

FIGURE 3.3: Code example of loading a ScriptedMaterial, setting itsparameters and attaching it to a mesh.

instance is then configurable by the web developer through manipulating its shadervariables that ThreeJS use in the shading process. Before its ’material’ is used inconstructing a ’THREE.Mesh’ instance it must be initialised. The ’init’ function con-structs the ThreeJS material with which to render the object and it sets standard ma-terial configuration parameters and shader variables. The material’s ’update’ func-tion must also be added to the ’updateFunctionsHolder’ as shown for it to be calledin the application cycle. If the function needs to reference the scripted material itmust be attached as in figure 3.3.

21

Chapter 4

Results

4.1 Adoxa: Integration and Development

4.1.1 Integration

Including the framework in an ASP.NET application is a simple process [Fig. 4.1].As all of this project and its dependencies are just JS files, one can simply include itwith the <script> tag. Alternatively, as done for Adoxa, bundle it all using the .NETScriptBundle class and then include it with ’Scripts.Render’ as shown below, whichpastes it in a minified format onto the HTML document.

FIGURE 4.1: The ASP.NET Razor code in the view that includes theproject essentials. The " /ThreeJS" bundle includes ThreeJS core li-brary as well as extensions to it, wheresas the " /GraphicsWork-flow" includes own modules developed for this thesis as well as non-

ThreeJS related libraries eg TweenJS.

4.1.2 Adoxa Front-End Enhancements

Most major features of the framework presented in this paper are demonstrated bythe upgraded Adoxa front-end [Fig. 4.2]. These include loading 3D meshes fromfile that synchronize with the native DOM of the website, as well as the loading andimplementation of scripted shaders. The login page is shown in Figure 4.1 as it looksupon entering the website.

The dark-green logo top is a 3D mesh loaded from a ’.stl’ file from the fetched fromthe server. It has a registered ’onclick’ callback which triggers a crude jumping an-imation which is animated using TweenJS. The logo is also synced to the browser’swindow size causing it to automatically adjust position upon window resize. Itsposition is also set and reset relative to the screen using the associated ’ThreeUtils’functions.

22 Chapter 4. Results

FIGURE 4.2: Enhanced Adoxa login page. Green product logo in 3Dcourtesy of Clara Lundwall.

The background is a ’THREE.PlaneGeometry’ with a scripted material named ’Flash-ingGridBackground’ attached to it. It functions just as scripted materials were de-scribed in sections 2 and 3. The material is animated during runtime to create achanging background grid that softly fades in and out.

There is also a transition animation that is triggered upon successful login. Thewhite login form card in the center is rendered with the ThreeJS CSS3D rendererand animated just as the 3D logo to move offscreen during the transition.

4.1.3 Explizit DevTeam Opinion

The developers at Explizit had overall positive feelings towards the result. They arefavourable to the concept of themselves being able to implement 3D effects with aminimal learning curve. The subtle transition animations and background effectsare something they consider significant visual improvements to their platform, andwould like to see more of in other parts of Adoxa and potentially other products.The primary issue stated though was their inexperience with and in practice non-necessity to work with graphics. This meant further development of pre-built effectsalong with detailed manual and examples on how to use the project was necessary.

A more grounded response was met by management. The most primary concernwas that simple aesthetic enhancements were simply not worth the investment. Anessential software design mantra is that of consistency. It entails that visual or func-tional changes to any one part of the application should or even must be appliedto all similar parts[19], which multiplies development time and increases overallrisk. Although appreciative of the efforts and findings, it was decided not to includefront-end enhancements with the results of this project in their development.

4.2. NodeJS: Integration and Further Prototyping 23

4.2 NodeJS: Integration and Further Prototyping

For further development of features not directly applicable to the Adoxa platforma separate server was created. This was primarily to ensure integratability of theproject into a different application architecture than Adoxa, as well as to gain addi-tional insight into potential design limitations or benefits. As is visible in the demoscreenshots the flashing grid background is active throughout these as well.

4.2.1 Setup

The method used for the NodeJS examples was to include each project file and all thedependencies in the right order in ’index.html’. The script paths relative at serverlocation on disk were made available with ExpressJS [Fig. 4.3]. On the server allthe graphics utitilites including ThreeJS and the project modules lie in the "Graph-icsWorkflow" folder, whereas the demo scripts lie in the "examples" directory.

FIGURE 4.3: Project setup in Node with ExpressJS. On the right isshown the code setting up the NodeJS server and on the left the in-dex HTML file which includes all the library scripts. Note that the

solution root directory is referred to as "midas".

4.2.2 Interactable 3D Object

This first demo showcases features of the ThreeViewHTMLSync and how it is usedin the ThreeView. The Adoxa logo model used here is the same as for the Adoxademo in the previous section, and is also animated using TweenJS [Fig. 4.4]. Thesquares seen are animated DOM elements, also animated with TweenJS. The usercan drag and drop the model onto the squares whereupon the model’s transformwill be adjusted in real time according to the DOM elements’ bounding boxes. Toachieve this the ThreeView module methods un-/bindSceneObjectToElement arecalled at the appropriate event callbacks.

24 Chapter 4. Results

FIGURE 4.4: Screenshots of the live demo described in this subsection.The top two demonstrate the behaviour of the sqaure DOM elements:the blue moving side to side and the green expanding then shrinking.The bottom two demonstrate the 3D Adoxa logo mesh mimicking

their behaviour when synced.

4.2.3 Shading with Multiple Materials

This second and final demo demonstrates implementation of more advanced shaderfeatures. A textured plane is rendered upon which the user can paint [Fig. 4.6]. Thisexample makes use of much more shading features offered by ThreeJS, in particularRenderTargets [Fig. 4.5].

FIGURE 4.5: Initialization of rendertargets used in this demo. TheThreeView module sets the rendertarget of a scene by the fourth

string argument in the scene constructor.

4.2. NodeJS: Integration and Further Prototyping 25

FIGURE 4.6: Screenshot of the untouched textured plane above ascreenshot of the same but drawn on by the user.

27

Chapter 5

Discussion

5.1 Technical Review

In this section the solution codebase is reviewed, and hence goes into more tech-nical detail regarding code style, internal architectural design and other technicalelements at API level and below. Readers whom are more interested in utilising thisthesis solution in developing their own producs may consider skipping to the nextsection "Usage And Application Review".

5.1.1 Project and Codebase Structure

The solution consists of a number of independent modules, although there are onlythree modules which the developer is meant to call: ThreeView, ThreeUtils andScriptedMaterialFactory. Of these ThreeView is the most essential since it is themodule which presents to the developer functions that reflect the core concept ofthis solution: the ability to seamlessly integrate 3D graphics into an existing website.

Both of the other two modules, however, are useable on their own, although thisis not recommended. The ScriptedMaterialFactory for example does not make senseto use without the ThreeView module. This is due to the fact that the process ofwriting shaders is not simplified by the ScriptedMaterial, and any additional func-tionality can only run within the context of a ThreeView module. The ThreeUtilsprovide functions that might be useful in any ThreeJS application, such as the Geom-etryLoader submodule or those converting between screen and world coordinates.These are not difficult to implement oneself however. It is also necessary to note thatmuch functionality offered by the ThreeUtils module is written to be used within thesolution.

As such all the modules should have been grouped and exposed as a single library.This would also make it clearer in the application what is ThreeJS code and whatis provided by this thesis solution. This among other tasks could have been madeeasier by the use of task runners such as Gulp and module bundling systems likeRequire. Exposing the API through a single minified JS file would not only betterreflect design intent but also greatly ease solution application, as the user developerwould only have to include a single JS file to begin using it.

28 Chapter 5. Discussion

5.1.2 API Design

The API was designed with simplicity and ease of use in mind. It sets up the basicnecessities of adding graphics features to the website, and sets up an environmentthat satisfies many use cases that want to seamlessly mix GPU-rendered visuals withDOM rendered elements. The user interacts mostly with the ThreeView ( for reasonspresented in subsection 5.1.1) but calls ThreeUtils and ScriptedMaterialFactory us-ing their module namespaces.

In review though the API should have been presented as a single library with onenamespace from which all functions are called, as is the case for even far larger li-braries such as JQuery which just exposes all its API with a single ’$’. Internally itis still beneficial to separate the modules, but the problem really relates to what isdiscussed in the previous subsection 5.1.2, in where a solution may be obtained byuse of RequireJS and GULP.

Although simplifying the initial steps, there is not much added support for theimplementation of more advanced graphical features such as lighting or shadows,meaning the developer will have to heavily interact with ThreeJS directly when de-veloping such features with the thesis solution. However, ThreeJS does by itself pro-vide API that simplifies such tasks enormously and so it is arguable as to what needthere is to further abstract ThreeJS functions beyond what is already done. The focusfor the future should rather lie in expanding the preexisting selection of ScriptedMa-terials, which is likely of more interest to developers looking to apply the solution indeveloping their own front-ends.

Apart from what has already been discussed in this section regarding how the API isexposed to the developer, there are other aspects of the API that require review. Thenaming convenition used is one common to web development: camelCase wheremodule names start with a capital, whereas module members and nested moduledefinitions (eg sceneObject) begin with lower case letter. In review the suggestionto have nested module names begin with upper case anyway as to make the mod-ule vs module member more distinct emerges. All the main modules operate onnested modules on copied instances of it anyhow, with the exception of the geom-etryFileLoader where it is essentially a static member of ThreeUtils. Another re-view suggestion is that the prefix ’Three’ be excluded from the module names, sinceall modules depend on the ThreeJS library anyway. It becomes meaningless sinceScriptedMaterialFactory lacks it regardless.

A significant flaw with the solution is that it is designed and written to work en-tirely in JS. The strength of many popular JS frameworks today such as Angularor Vue is that they are designed to be used with HTML, modules and their com-ponents are defined in JS but they are selected in HTML for rendering. A similarapproach where scene object templates could be created from grouping ScriptedMa-terials, scripts and URLs to assets where the templates can simply be insterted in theHTML document to render. Instead of reinventing the wheel there is the option toprovide the solution in a package useable by Angular, although that would requiredifficult rewrites of the thesis solution modules since Angular builds its own HTMLcode around the defined modules, and the thesis solution modules operate on thewindow. Hence the choice of writing own HTML selector functionality might be lesscostly.

5.2. Usage and Application Review 29

5.1.3 Other Limitations and Future Improvements

One major limitation of the project as of now is that there is no internal supportfor caching ScriptedShaders and other resources, and thus would have to be donemanually by the user. A future improvement would be to support loading Script-edMaterial from the local filesystem, especially since one will want to cache largersized-textures, meshes etc. This will also enable running webpages with content de-pendent on this thesis solution locally without setting up a webserver, benefittingrapid prototyping which is often done in web design.

The modules of this project: ThreeView, ThreeUtils, MaterialFactory were all writtenusing native ECMAScript 5 JS as to allow support of older browsers. However, asthe Chrome debugger was used for development of the features IE11 support wasdropped for these as the demo shaders were written in template strings (“), a featurenot supported introduced in ECMAScript6 not supported by IE11. This error can beattributed to negligence since the error was not caught until the end of the projectwhen testing on IE11 appeared as an afterthought. However, all the core modulesstrictly use ECMAScript 5 features, and it is just the shaders that really should havebeen read from separate files instead. As such the first example of the NodeJS sam-ples runs on IE11, but not the second one which includes two ScriptedMaterials.

Another flawed design decision was the choice to use JQuery. Although it was usedextensively throughout the project for testing and development, the current solutiononly calls JQuery to get scripts from the web server. Although it aided with pro-duction within the project time, it is an unnecessary dependency as the script fetchcan be written with vanilla JS without breaking cross-browser functionality. In hind-sight, given the sheer size of the JQuery library, it becomes obviously desirable thatthe project not depend on it.

5.2 Usage and Application Review

This section reviews the technicalities faced by the developer in using the solutionpresented in this thesis. The discussion here brings up strengths and limitations ofthe project as a web development tool, and looks at potential modifications to im-prove upon it.

5.2.1 Solution Distribution

In short the thesis solution can be described as a ThreeJS library extension, given thatall the solution modules require ThreeJS to even load. This is reflected in choice ofnames for some of the main modules: ThreeView, ThreeUtils and ThreeViewHTML-Sync. As such, distribution of the solution as a ThreeJS plugin, done as such ofthe "CSSPlugin.js" module used in this project, would be more appropriate since itis probably impossible to develop interesting graphical features without calling theThreeJS API.

Regardless of this the solution as it exists in its current state is not suitable for open

30 Chapter 5. Discussion

or commercial distribution. The solution currently exists as a grouped collection ofJS files (as shown by fig. 4.3 in subsection 4.2.1) to be copied and pasted into theapplication server file system. This needs to be remedied by the changes discussedin subsection 5.1.2, which further impact the solution negatively in ways addressedin the following subsection.

5.2.2 Integration into a Web Application

Despite the structural disadvantages, an essential feature of the project is that it iseasy to use in any web application. All that is required to gain access to the solutionAPI and pipeline is to include the JS code, which additionally can be run on anymodern browser with WebGL.

Being pure JS it technically means the modules can be included in an HTML fileand run locally with a browser that supports WebGL. However, all assets are gotvia XmlHttpRequests, necessitating that the project is run in a hosted webserver.This is a drawback for web designers wanting to produce quick prototypes withoutthe need to setup a webserver, which could be counteracted by programming a fall-back procedure for fetching files from the localhost filesystem. However, given thewidespread availability of web server starter projects such as NodeJS, Apache andASP.NET this is of low priority.

The structural flaws of the solution discussed in subsection 5.1.2 were the most sig-nificant hindrances to integrating the solution. These were not much of a concern inAdoxa as ASP.NET has its own script bundling feature, but the NodeJS examples ex-perienced bloated HTML headers for the numerous scripts needed. As modern webapplications tend to consist of multiple webpages this quickly becomes untenableand as such the project modules must be provided by a single library file in futureuse cases. As demonstrated by the results this issue varies in severity depending onserver implementation. Any library or framework should be as codebase agnosticas possible, hence the suggestions made in subsection 5.1.2 should be implementedhere so that developers in desire to use the solution should not be obliged to imple-ment server-specific solutions to address these issues themselves.

5.2.3 Use Within JS Framework

A topic of interest that was not explored in this project was to test how well the li-brary would integrate into a JS framework, which are are commonly used in webdevelopment and tend to be favored by developers[22]. The reason for not doingso was simply that the Adoxa front-end used for this project is an older version notwritten in a framework but in ECMAScript5 vanilla JS. Given that the project waswritten with the intent to use directly in website script, it is likely that extensivework would be needed to allow the library to be callable within a framework. Firstoff it would have to be exposed as a single exportable module, so that it can be in-cluded in another module by for example calling "import ..." in TS.

This project would especially benefit from being used in a single-page web appli-cation framework such as Angular2, since this would mean the ThreeView mod-ule would only have to be initialised once. The framework is used by Explizit in

5.3. Potential of Graphics in Web Application Development 31

other products, and it is a common choice for enterprise web solutions. With Angu-lar2 where the application is built using self-contained modules and children com-ponents, usually with a single module bootstrapped in the index page. For usagewithin an Angular2 project the ThreeView should be initialised in the module andthen imported with TS as described previously.

5.2.4 Useage in Front-End Development

To apply this project one does need at least a basic ability with ThreeJS and com-puter graphics in general. Developing novel graphical features is not necessarilyeasier with this framework, as was especially noticeable in the second NodeJS ex-ample with the paintable canvas, where the graphics programming required wasmore advanced. What is provided however is a solution with which the developercan much more quickly implement graphical features to the website.

The project solution simplifies much of the work but does not exclude ThreeJS fea-tures from the developer at all. The ThreeView module sets up predeterminedscenes, cameras renderers and an eventhandling system for all of these, in. However,the developer is free to add their own customised ThreeJS entities into ThreeViewwhile maintaining access to solution features such as binding to DOM elements anduser input handling. The developer must at least write their own geometries andshading, but ThreeJS already supplies developers with API to create common graph-ics entities such as Point-/DirectionalLights and various mesh types.

5.3 Potential of Graphics in Web Application Development

The current environment of web application front-ends is one of two-dimensionalinterfaces. Although an appealing idea, there is strong reason as to why the majorityof web developers/designers, particularly in enterprise, exclude the use of graph-ics that are only achieveable by graphics APIs. Primarily it can be attributed to theadditional development costs. Graphics development requires programmers withdomain-specific knowledge as well as significant allocation of development time,which is one of the main issues the solution presented in this thesis attempts totackle.

This raises the question as to what graphical features are actually feasible to de-velop and apply to an existing web application. It is not only a technical questionin terms of code, but also a one of design as to what graphics features would alterUX to increase the product value more than the cost to develop it. The issue of actu-ally designing important visual and interractable aspects of a front-end such as UIto heavily require graphics APIs is a major concern as to the importance of any worksuch as that done in this thesis.

Although not highly conclusive, the results of the case of Adoxa clearly validatethis concern. It was difficult to imagine anything more than enhanced GUI effects,and the fact that it was a multi-page application meant any major GUI overhaulswould have had to be replicated across multiple pages. The solution could havebenefitted well from web designers familiar with Adoxa, whose ideas would likely

32 Chapter 5. Discussion

have provided better proofs of concepts in how it might have been used. The im-plementation of professional UX design concepts would indeed be vital to achieve iffurther development was to be done on this solution.

The chances for this solution to be used in industry web applications are in its cur-rent state limited. For example GUI features built completely using this solutioninstead of mere addon effects would have been more of a convincing showcase ofits capabilities, and perhaps served to discover less obvious improvements to makeit more scaleable. Unfortunately this was simply not within the timescope of thisgraduatework, and would have been more achievable if work was done towardsaccomplishing already thought up designs instead of just an exploration into possi-bilities.

33

Chapter 6

Conclusion

The goal of this paper is to present an abstracted and simplified approach to apply-ing graphical enhancements in a web front-end. This goal is acheived by offeringan integrateable framework for applying WebGL based graphics to any web appli-cation, as well as an intended methodology for developers to utilise the framework.The result is a viable alternative for developers with little to no experience in graph-ics programming to incorporate more advanced graphical features into their work,with a gentler learning curves when compared to all current graphics APIs.

As for the potential demand of the solution it is difficult to assert that another levelof abstraction for WebGL is desireable in web development today. Front-end devel-opers with graphics programming experience would likely prefer to continue work-ing with familiar APIs instead of an additional layer of abstraction that may feellimiting. It is thus unlikely that a solution as presented here would gain much pop-ularity in the web development community. As in the case for Explizit, not manyweb products actually require any graphical enhancements beyond what is alreadyaccomplishable by native web development tools in CSS/HTML and JS. The devel-opment of advanced graphics web application features is not likely worth the costfor most companies, but this solution does reduce the developer burden in doing so.

Given the ever increasing popularity of the web, demand for such a solution is agrowing possibility. Currently the vast majority of websites are designed and builtentirely in flat 2D design, but as explorations into UX and design continue this islikely to change. Future developments to this paper’s solution should have webdesigners with limited programming experience in mind. Configurable feature tem-plates of shapes and effects should be developed to lessen the developer’s need forproficiency with ThreeJS when using this framework. Integration into more moderncomponent-based frameworks such as React or Angular, potentially replacing theunderlying rendering mechanisms, is also to be desired.

Although more work is required to demonstrate the full potential of an integratablerendering solution such as the one proposed by this paper, the benefits are apparent.It lowers the barriers for developers and designers alike to utilize the more power-ful graphics pipeline of WebGL, while minimizing technical restrictions. In the endit is designers who shape the way we interact with the web, and they need morepowerful tools to improve the quality of life for all web users.

35

Appendix A

Animated DOM with CSS and JS

<style>h1 {

font-family: arial;color: #333399;

}h2 {

font-family: arial;color: #333399;

}.box {

width: 25px;height: 25px;display:inline-block;background-color: black;

}.translatingRight {

animation-name: translateRight;animation-duration: 2s;animation-iteration-count: infinite;animation-fill-mode: both;animation-timing-function: linear;

}.translatingLeft {

animation-name: translateLeft;animation-duration: 2s;animation-iteration-count: infinite;animation-fill-mode: both;animation-timing-function: linear;

}.rotating {

animation-name: rotate;animation-duration: 2s;animation-iteration-count: infinite;animation-timing-function: linear;

}.burning {

animation-name: burn;animation-duration: 1s;animation-iteration-count: infinite;

}.freezing {

animation-name: freeze;animation-duration: 1s;animation-iteration-count: infinite;

}@keyframes translateLeft {

0% {transform: translateX(50px);}25% {transform: translateX(100px);}75% {transform: translateX(0px);}100% {transform: translateX(50px);}

}@keyframes translateRight {

0% {transform: translateX(50px);}25% {transform: translateX(0px);}75% {transform: translateX(100px);}100% {transform: translateX(50px);}

}

36 Appendix A. Animated DOM with CSS and JS

@keyframes rotate {0% {transform: translateY(0px);}50% {transform: translateY(50px);}100% {transform: translateY(0px);}0% {transform: rotateZ(0deg);}50% {transform: rotateZ(180deg);}100% {transform: rotateZ(360deg);}

}@keyframes burn {

0% {background-color: red;}50% {background-color: yellow;}100% {background-color: red;}

}@keyframes freeze {

0% {background-color: blue;}50% {background-color: lightblue;}100% {background-color: blue;}

}</style>

<script src="../lib/fpsmeter.js"></script><script>

// page const elements, set in window.onloadvar mainElem;

// toggle testing elements visibilityvar visible = true;function toggleInvis() {

if (visible)mainElem.style.visibility = "hidden";

elsemainElem.style.visibility = "visible";

visible = !visible;}

// benchmarking variables, set prints in window.onloadvar initTime;var loadTime;var frameRate;var loadTimePrint;var frameRatePrint;

// runtime benchmarking at intervalfunction fpsUpdate(updateEvent) {

frameRate = updateEvent.fps;frameRatePrint.innerHTML = "FPS: " + frameRate;

}

function initRuntimeBenchmark() {if(!window.FPSMeter) {

alert("This test page doesn’t seem to include FPSMeter: aborting");return;

}document.addEventListener(’fps’, fpsUpdate, false);FPSMeter.run();

}

// testinitTime = (new Date()).getTime();window.onload = function() {

// set html elementsmainElem = document.getElementById("main");loadTimePrint = document.getElementById("loadTime");frameRatePrint = document.getElementById("frameRate");

// test content setupvar dx = 55;var dy = 55;var x = 0;var y = -dx;var alt = -1;var rowLen = 3;

Appendix A. Animated DOM with CSS and JS 37

var translater = document.createElement("div");translater.classList.add("translatingRight");for (var i = 0; i < 25; i++) {

x = (i % rowLen) * dx;if (x == 0) {

alt *= -1;translater = document.createElement("div");if (alt > 0)translater.classList.add("translatingRight");

elsetranslater.classList.add("translatingLeft");

mainElem.appendChild(translater);}

var rotater = document.createElement("div");rotater.className = "box";var colorShifter = document.createElement("div");colorShifter.className = "box";

if (i % 2 == 0)colorShifter.classList.add("burning");

elsecolorShifter.classList.add("freezing");

rotater.classList.add("rotating");rotater.appendChild(colorShifter);translater.appendChild(rotater);

loadTime = (new Date()).getTime() - initTime;loadTimePrint.innerHTML = "Loading Time: " + loadTime + " ms";

}initRuntimeBenchmark();

}</script><html>

<button id="toggleInvis" type="button" onclick="toggleInvis();">Toggle Visibility

</button><h1>

Test CSS3</h1><div id="loadTime"></div><div id="frameRate"></div><div id="main"></div>

</html>

39

Bibliography

[1] Alexander Buzin. WhitestormJS. 3D javascript engine. URL: https://medium.com/whitestormjs-framework.

[2] Ben Cherry. JavaScript Module Pattern: In-Depth. URL: http://www.adequatelygood.com/JavaScript-Module-Pattern-In-Depth.html.

[3] CSS3DRenderer.js. URL: https://github.com/mrdoob/three.js/tree/dev/examples/js/renderers/CSS3DRenderer.js.

[4] Explizit: Official. URL: https://explizit.se/sv/hem/.

[5] Tali Garsiel. How browsers work: Behind the scenes of modern web browsers. URL:http : / / taligarsiel . com / Projects / howbrowserswork1 . htm #Rendering_engines.

[6] Nikolaus Gebhardt. WebGL now faster than Flash? URL: http://www.irrlicht3d.org/pivot/entry.php?id=1298.

[7] Getting started and viewing WebGL. URL: http://learningwebgl.com/cookbook / index . php / WebGL : _Frequently _ Asked _ Questions #What_is_WebGL.3F.

[8] Getting started with WebGL. URL: https://developer.mozilla.org/en-US/docs/Web/API/WebGL_API/Tutorial/Getting_started_with_WebGL.

[9] Khronos Group. Khronos Releases Final WebGL 1.0 Specification. URL: https://www.khronos.org/news/press/khronos-releases-final-webgl-1.0-specification.

[10] HTML5 Game Engines: Which HTML5 Game Engine is right for you? URL: https://html5gameengine.com/tag/webgl.

[11] David Corvoysier aka kaizouman. fpsmeter GitHub. URL: https://github.com/kaizouman/fpsmeter.

[12] kittykatattack. A step-by-step introduction to making games and interactive mediawith the Pixi.js rendering engine. URL: https://github.com/kittykatattack/learningPixi.

[13] Paul Lewis and Paul Irish. High Performance Animations. URL: https://www.html5rocks.com/en/tutorials/speed/high-performance-animations/.

[14] Alex Mackey. Why you should learn WebGL. URL: https://www.pluralsight.com/blog/software-development/webgl-basics.

[15] Juntao Gao Maoxiang Shi and Michael Q. Zhang. “Web3DMol: interactive pro-tein structure visualization based on WebGL”. In: Nucleic Acids Research 45(W1 2017), W523–W527. URL: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5570197/.

[16] Model-View-Controller. URL: https://msdn.microsoft.com/en- us/library/ff649643.aspx.

40 BIBLIOGRAPHY

[17] Ricardo Cabello aka MrDoob. ThreeJS: GitHub. URL: https://github.com/mrdoob/three.js/.

[18] Ricardo Cabello aka MrDoob. WEBGL FRAMEWORK COMPARISON. URL:http://bnjm.github.io/WebGL-framework-comparison/.

[19] Anton Nikolov. Design principle: Consistency. URL: https://uxdesign.cc/design-principle-consistency-6b0cf7e7339f.

[20] Number of internet users worldwide from 2005 to 2017 (in millions). URL: https://www.statista.com/statistics/273018/number-of-internet-users-worldwide/.

[21] David Pakman. May I Have Your Attention, Please? The currency of the mediabusiness is attention. So where are we spending it? 2019. URL: https://pakman.com/may-i-have-your-attention-please-19ef6395b2c3.

[22] Jae Sung Park. The status of JavaScript libraries & frameworks: 2018 & beyond.URL: https : / / medium . com / @alberto . park / the - status - of -javascript-libraries-frameworks-2018-beyond-3a5a7cae7513.

[23] Daniel Pintilie. The History of Web Browsers. URL: http://www.instantshift.com/2010/10/15/the-history-of-web-browsers/.

[24] Denis Radin aka PixelsCommander. HTML-GL GitHub. URL: https://github.com/PixelsCommander/HTML-GL.

[25] Denis Radin aka PixelsCommander. HTML/CSS rendering via WebGL for highestperformance possible and unlimited animations abilities on the Web. URL: http://pixelscommander.com/en/web-applications-performance/render- html- css- in- webgl- to- get- highest- performance-possibl/.

[26] PixiJS: Official. URL: http://www.pixijs.com/.

[27] PYMNTS. Mobile Users Split On Apps Vs. Web Browsers. URL: https://www.pymnts.com/news/2016/mobile-user-split-on-apps-vs-web-browsers/.

[28] Mahendranath Ramodhin Raj Kishen Moloo Sameerchand Pudaruth and ReezwanahBibi Rozbully. “A 3D Virtual Tour of the University of Mauritius using We-bGL”. In: 2016 International Conference on Electrical, Electronics, and Optimiza-tion Techniques (ICEEOT) (2016). URL: https://ieeexplore.ieee.org/document/7755226/.

[29] Nicholas Rego and David Koes. “3Dmol.js: molecular visualization with We-bGL”. In: Bioinformatics 31 (8 2015), 1322–1324. URL: https://doi.org/10.1093/bioinformatics/btu829.

[30] Salman SH. Mobile App vs Mobile Web: One-third Users Prefer Both, But Bats ForOperability. URL: https://dazeinfo.com/2016/01/22/mobile-apps-vs-mobile-web-browser-usage-study/.

[31] Enuke Software. Graphics Technologies: Opengl Vs Stage3D. URL: http://www.enukesoftware.com/blog/tug-war-of-graphics-technologyies-opengl-vs-stage3d/.

[32] ThreeJS: Official. URL: https://threejs.org/.

[33] Vangelis Kokkevis & the Chrome Graphics team Tom Wiltzius. GPU Acceler-ated Compositing in Chrome. URL: https://www.chromium.org/developers/design-documents/gpu-accelerated-compositing-in-chrome.

BIBLIOGRAPHY 41

[34] Turbulenz Official: Developers. URL: http://biz.turbulenz.com/developers.

[35] Lukas Turcani. GLmol. URL: https://github.com/biochem-fan/GLmol.

[36] Whitestorm.js: GitHub. URL: https://github.com/WhitestormJS/whitestorm.js?.

[37] Xiaolong Wan Xinliang Wei Wei Sun. “Architecture and Implementation of 3DEngine Based on WebGL”. In: Applied Mathematics (AM) 7.7 (2016), pp. 701–708. URL: http://www.scirp.org/journal/PaperInformation.aspx?paperID=66063.