A Javascript Implementation of the Binary DIS Protocol

Don McGregorCurtis Blais

Don BrutzmanMOVES Institute, Naval Postgraduate School

700 Dyer Road Watkins Ext. 265

Monterey, CA 93943(831) 656-7605, (831) 656-3215, (831) 656-2149

mcgredo@nps.edu, clblais@nps.edu, brutzman@nps.edu

Keywords:Javascript, Distributed Interactive Simulation, DIS, Websocket, Web-based Simulation, WebLVC

ABSTRACT: Simulation applications running in a web browser offer the potential for simplified simulation management and distribution. The Websocket and WebRTC standards allow applications running in web pages to communicate with the server, or even directly with other web pages, but a format to use for message exchange in distributed simulations has not been widely agreed upon. Javascript Object Notation (JSON) text format has been used by some applications, but retaining interoperability creates a requirement for standardizing a JSON message format for simulation applications. We have implemented a Javascript library that instead uses the IEEE standard Distributed Interactive Simulation (DIS) binary format, and that runs in all recent major browsers.  The DIS binary Javascript library is able to process up to several thousand Protocol Data Units (PDUs) per second. The library simplifies the integration of existing simulation applications with web based simulation applications.

1. Background

For the last decade or more business applications have been migrating away from the paradigm of installing applications on desktop machines and towards web-based applications running in the web browser. This began with simple applications that were easy to migrate to HTML, and the advent of Asynchronous Javascript and XML (AJAX), fast browser Javascript language implementations, and HTML5 technologies have accelerated the process. The economics are compelling: applications can be deployed once on the server side, run in ubiquitous web browsers that are already installed, and can scale out on the cloud. The web applications are often cross platform across Windows, Unix, Apple OS X, and mobile devices. However, Distributed simulation applications, because of their demanding requirements for low latency, graphics, and computation, have generally not shared in this trend. This is likely to change as new technologies such as WebGL [1], Websockets, [2] and WebRTC [3] are deployed more widely. WebGL enables the creation of 3D virtual simulations, and websockets and WebRTC allow applications access to high bandwidth, low latency networking, relative to older polling technologies such as AJAX. Faster web browser Javascript engines and more sophisticated

Javascript libraries are also enabling more capable web applications. This can allow Live, Virtual, and Constructive (LVC) applications to be run entirely within a web page, using no other applications or plugins.

Distributed simulation applications must exchange state information and the Websocket and WebRTC Javascript APIs are a mechanism to allow this. The two APIs resemble, respectively, the TCP sockets and UDP sockets used in conventional desktop simulation applications. As with conventional TCP/IP socket programming, the API is silent about the format or content of the messages passed. It is up to the programmer to choose a format to pass simulation state information, and cooperating applications must agree to use a mutually intelligible message format. On the desktop a number of APIs and protocol wire formats have been adopted for simulation applications, including Distributed Interactive Simulation (DIS), High Level Architecture (HLA), and Test and Training Enabling Architecture (TENA). Web-based simulation applications are still in their infancy and formats for exchanging information have not yet been widely adopted or standardized. For example, HLA has no Javascript API and the lack of an interoperable over-the-wire network format means that various proprietary

binary encodings cannot be consistently sent or received.

In the absence of a standard method for exchanging state information most web-based simulation applications have used the Javascript Object Notation (JSON) format. JSON is human-readable, text-based, and lightweight format for describing the content of Javascript objects. A parser to convert data in the JSON format to Javascript objects is included in most Javascript implementations; this feature makes it easy to rapidly prototype and implement message exchange. An example representing an update to the state of an entity using JSON format is shown in figure 1.

{ MessageKind : 1, ObjectName : “F-16 Alpha”, ObjectType : “WebLVC:PhysicalEntity”, EntityIdentifier : [1,2,1], EntityType : [1,2,225,1,3,0,0], WorldLocation : [4437182.0232, -395338.0731, 873923.4663], VelocityVector : [57.04, 32.77, 89.263], Orientation : [-1.65, 2.234, -0.771], Marking : “F-16”, DamageState : 0

}Figure 1. DIS Object Update Expressed in JSON

Format

This example is taken from the WebLVC SISO study group [4], an effort to standardize message exchange formats using JSON that is closely patterned on the Real Time Platform Reference Federation Object Model (RPR-FOM). Fields in the example include the semantic information specified in the SISO Encoded Bit Values (EBV) document, such as the entity type, and other conventions such as the entity location, expressed in the earth-centered, Cartesian world location used by DIS and RPR-FOM.

JSON has surprisingly good performance, but also has some unavoidable drawbacks. Because its design goals include being human readable it uses text to describe the contents of objects. Because of the tag-value approach used by JSON, both the sender and receiver must agree on the field names used in the messages. Including the field name in the message takes more space, but does allow the application code to be more flexible when the format is still in flux. Using JSON effectively requires the standardization of a text format wire protocol of the sort discussed by WebLVC.

DIS takes a different approach: the simulation state values are saved in an array of packed binary values. The fields in the message are not tagged with a name, but instead use the position of fields in the message, as specified by the protocol external to the message. The messages sent in a binary protocol are typically smaller than the same information sent in JSON format, and in principle are more efficient to encode and decode, since the data does not need to be converted between text format and a binary representation. As a result the messages are more compact, and in theory fewer CPU instructions can be used to decode them. In the example shown in Figure 1, the update message takes about 300 bytes of text, while a DIS entity state Protocol Data Unit (PDU) (which carries similar though different data) takes about 144 bytes.

Javascript applications adopted the JSON format in part because Javascript, as a language, did not do the sort of binary manipulations required to encode and decode binary messages well. However new Javascript API features have made it more practical to encode and decode the packed binary messages used by DIS. The Javascript Typed Array API [5] feature allows programmers to encode and decode packed byte arrays. This gives programmers the tools to implement binary protocols such as DIS.

Javascript web programming is always an adventure in cross platform compatibility; Javascript language implementations are embedded in the web browser, browser vendors add new APIs at different times, and the browser version deployed at customer sites may lag from the most recent released version. Deployed browser support for the Typed Array API in various browsers is shown in figure 2 [6]. These browsers account for about 80% of the browsers in use on the internet in general today, though specific market segments such the Navy-Marine Corps Internet (NMCI) may have less support. For the most part web browsers released since 2011 or 2012 support the feature. In any event as browsers are upgraded support will steadily increase.

Internet Explorer

Firefox Chrome Safari

Supported Since Version

10.0 4.0 7.0 5.1

Figure 2. Web Browser Support for Typed Arrays

The JSON format is workable, but the DIS binary format has a significant practical advantage: the

standard has already been approved. If it can be implemented in Javascript then practitioners may be able to avoid creating a new wire protocol standard. Support for the DIS standard is also ubiquitous. Many existing simulation applications directly support DIS, and for those that don’t, there are usually protocol gateways that can convert application state information to the DIS format. This simplifies interoperability with existing applications.

The task we set ourselves was to implement the DIS protocol in Javascript. The potential outcome is web simulation applications that can directly interoperate with existing simulation applications in an efficient way, without the extra layer of a syntactic binary-to-JSON gateway and the requirement to create a new standard. Implementing DIS in Javascript also allows us to explore the current state of the art and tradeoffs in Javascript protocol implementations.

2. Implementation

2.1 Client Side

We implemented the Javascript version of the DIS library using the Open-DIS project [7]. Open-DIS contains an XML file that describes the fields, types, and layout of the DIS version 6 and 7 PDUs. The code to implement a PDU is mostly boilerplate. The Javascript object PDUs must be able to marshal and unmarshal themselves to the IEEE 1278.1 DIS standard binary format, and the data fields of the DIS PDUs must be accessible to other Javascript code. With the information contained in the XML file a relatively small Java program can generate the corresponding Javascript code that implements DIS PDUs and the code to marshal and unmarshal to DIS. The code, for both the generator and the generated Javascript code, is available at sourceforge.net and has a non-viral BSD open source license.

Example code for creating and decoding an Entity State PDU from a binary source in Javascript is shown in figure 3.

websocket.onmessage = function(msg) { var disMessage = new dis.EntityStatePdu();var is = new dis.InputStream(msg.data); disMessage.initFromBinaryDIS(is);

Figure 3. Creating an Entity State PDU from Binary Data

This shows the Websocket API’s onmessage() function, which passes in an argument containing data received from the websocket server. The Javascript object passed as an argument has a property called data, which contains the binary PDU data in IEEE 1278.1 DIS format. A new Entity State PDU is created, and then initialized from an InputStream object.

The InputStream (and its converse, OutputStream) are modeled on the Java standard library classes of the same name. They read data from the binary Typed Array and maintain a pointer to the current read position in the binary buffer. As each value is read the current read pointer advances by the size of the data type just read.

The initFromBinaryDIS() function is written by the Javascript code generator, as is the reverse function, encodeToBinaryDIS(). For example to encode an EntityStatePdu to a DIS PDU format the code in figure 4 would be used.

var dataBuffer = new ArrayBuffer(1000);var os = new dis.OutputStream(dataBuffer);espdu.encodeToBinaryDIS(os);var trimmedData = dataBuffer.slice(0, os.currentPosition);websocket.send(trimmedData);

Figure 4. Encoding and Sending an Entity State PDU Object

This example writes data to a typed data buffer that is large enough to allow it to hold any likely Entity State PDU used in this context. After it is written to a binary buffer and we know how much space it consumes we create an array that is trimmed to fit the actual size of the PDU, and then send it to the websocket server.

Different languages have different conventions for accessing object values. The most common Javascript programming convention is to not use getter or setter methods, and to instead access an attribute directly using dot notation, such as entityStatePdu.entityLocation. We adopted this convention in our Javascript DIS implementation in order to be aligned with industry practice. Since the Javascript code is generated, the code generator can be modified to adopt other programming conventions if so desired.

Reducing web application page load time and optimizing web applications can be a subject in and of

itself. The generated DIS protocol file is about 10,000 lines of Javascript code and has a size of about 400K. The Javascript DIS code must be downloaded to the web browser on page load. Application programmers in a production environment often run their Javascript files through a “minify” step that strips out code comments, eliminates whitespace, and takes other steps to reduce the size of the file. This will usually reduce the size of the Javascript file by half or more. The web browser and web server can also negotiate compression of the Javascript file before sending it, which will reduce the size further. A minified and compressed DIS protocol implementation is about 20K bytes. The server and browser can negotiate client-side caching techniques that prevent repeated downloads of the file from the web server; after the first download the Javascript DIS implementation file is cached by the web browser using local storage, and subsequent page loads by any web applications that access the Javascript resource do not have to retrieve the file from the server. The Javascript resources can also be stored on a Content Distribution Network (CDN) to reduce load times.

It should be noted that the Javascript language uses weak data typing, and that “classes” in Javascript do not exist in the same sense as they do in Java or C++. Javascript is a prototypical object language with weak (or non-existent) typing. Object instances are just collections of attribute-value pairs. Creating a new instance of a “class” in Javascript creates an object with a certain starting set of attributes, and attributes or functions can be added to or removed from that instance. For example the EntityStatePdu object created above may have completely new attributes added to it, such as a date attribute called “lastHeardFrom,” and it may have existing attribute values such as “entityLocation” removed. This means two objects that were instantiated as EntityStatePdus may not have identical data fields or functions after use in the application program. This is in contrast to Java, where all instances of a class must have the same attributes and all share the same methods.

Receiving and decoding DIS PDUs on the client side and being able to send IEEE 1278.1 compliant PDUs is only part of the battle. The client must also make sense of the entity position value decoded. DIS uses a Cartesian, geocentric coordinate system with its origin at the center of the earth to describe object locations. Many applications must position entities using other coordinate systems, such as latitude, longitude, and altitude. Virtual worlds typically use a local coordinate system set up on a plane tangent to a given location on the earth’s surface. In Java or C/C++ language environments we could make use the SEDRIS Spatial

Reference Model (SRM) software development kit (SDK) [8], but there is as of yet no Javascript implementation of this valuable tool. We were forced to implement a few classes that do basic transformations among earth-centered (geocentric) coordinate systems, local tangent plane cartesian coordinate systems, and latitude/longitude/altitude coordinate systems.

2.2 Server Side

For the server implementation we used Jetty, a configurable Java-based implementation of a web server and servlet container. It can also implement Java Server Pages (JSP) and act as a websocket server side implementation. The Jetty server listens on the server’s local network on a Java UDP socket for DIS traffic. When a PDU is received it sends it out on each connected websocket to a web page, as shown in figure 5.

Figure 5. Relay of DIS Messages

The Semi-Automated Forces (OneSAF) application shown in the diagram represents an existing, unmodified DIS application; PDUs will be read by the Jetty server via the native UDP socket and then forwarded to each recipient. Likewise, the web pages can send DIS PDUs, and the Jetty server will distribute them to other web pages and to the server’s native network. The Jetty server acts as a hub for all traffic to the web-connected clients. This can provide a central point for performing area of interest management or distributed data management, but the ability to do this is unexplored for now.

Websockets are in fact TCP sockets with a slightly different API, and have all the inherent advantages and disadvantages of any other TCP socket. This means that while they are reliable, they also have on average higher jitter, and may have “freeze” periods if state

update packets are dropped and the websocket goes through a timeout/resend cycle.

3. Results

3.1 Performance Benchmarks

It is important to benchmark performance on multiple browsers because each browser has its own implementation of a Javascript engine and Javascript standard libraries, and results can vary dramatically. Javascript engine performance has been the subject of intense R&D over the last several years and most current browsers implement some version of a hotspot compiler/interpreter. Most commercial browsers currently implement a Just-In-Time (JIT) compiler for Javascript, but implementing JITs is challenging and performance characteristics can vary. When and whether the JIT actually optimizes code can also depend on code characteristics for even micro benchmarks. This means making general claims of performance in all web browsers difficult. At best only general outlines can be described, and even that may be deceptive.

A series of test cases to compare the performance of JSON and binary DIS formats has been made available at a public site [9], and users can benchmark their own browser and hardware there. The results are shown below in figures 6 and 7.

Figure 6. Performance Benchmark Results

Figure 7. Performance Benchmark Results

Longer bars in figure 4 represent better performance. Chrome, Netscape Firefox, Internet Explorer 11

(Labeled as “Other” above), and Safari running on a OSX Laptop, Windows 8.1 laptop, Samsung Note 2 tablet, and on an iPhone 5 mobile device were tested. The blue and red bars are the results for decoding binary DIS. Blue bars show the results for decoding into a complex Javascript Entity State PDU (ESPDU) object that contains separate objects for fields such as the entity type, entity location, and other records, while red shows results for decodes into a Javascript object that is “flattened” with no contained objects, just uniquely named fields. Orange and green bars show the results for two techniques of decoding JSON-formatted DIS information. Orange represents the use of eval(jsonText), while green uses JSON.parse(jsonText). Eval() is an older technique for parsing JSON that is still fairly widely used, despite security concerns. JSON.parse() is the preferred solution.

It is readily apparent that the browser’s implementation of Javascript matters, both in absolute performance and in which approach works best within a given browser. Safari’s Javascript performance was quite good, probably a reflection on the engine currently used by that browser, and binary formats were faster than JSON formats by a considerable margin. Internet Explorer 11 and Chrome 35 web browsers apparently have an optimized native JSON parser that decodes a JSON representation of an ESPDU as quickly as it does a Javascript binary format message. (Or, alternatively, the binary typed array libraries aren’t as well optimized as those of Safari and Firefox.) Decoding the binary DIS message in Firefox 30.0 was faster than decoding a DIS PDU encoded in JSON—41,000 operations per second for JSON vs 70,000 operations per second for the binary DIS representation, though both were smaller on an absolute level than Safari using any format.

Other results are unsurprising. Mobile devices are slower than desktop devices. Still, modern mobile phones can decode a respectable number of PDUs in either format.

We also benchmarked decoding the WebLVC update message shown in figure 1. Again, the benchmarks are available for inspection at the jsperf.com site [10]. This message contains less information than a DIS entity state PDU, but is larger—roughly 300 bytes for the message displayed in figure 1, vs 144 bytes for a standard binary DIS PDU. Results for Chrome are shown in figure 8.

Figure 8. DIS vs WebLVC Update Message

This repeats the results of the earlier Chrome benchmark—JSON and binary were both decoded about as fast. The WebLVC JSON message was decoded much faster than either, probably a reflection of the smaller number of Javascript objects created and less information contained in the message.

The low level benchmarks used here can be misleading. In reality a simulation application will be doing other things, such as physics, artificial intelligence (AI), and graphics, in addition to receiving and parsing network messages. Some of the CPU cycles used by the simple benchmark code above will instead by used by these tasks.

3.2 Example Application

We implemented a simple Google Maps Javascript API application using the binary DIS implementation [11]. The web page loads the Google Maps Javascript API, displays a map, and then displays the positions of DIS entities visible to the server’s network using the architecture shown in figure 5. In addition the web page uses the browser’s geolocation API and sends entity state PDUs back to the server to demonstrate two-way communications. The web application works on Mozilla Firefox, Google Chrome, Apple Safari, and IOS (iPhone/iPad) mobile Safari. A screen shot is shown in figure 9

Figure 9. AIS Ship Locations Displayed in Google Maps.

This displays the results of an Automatic Identification System (AIS) feed. AIS is a real time location system that the Coast Guard and other agencies around the world require to be present on commercial ships. The AIS feed has been piped into Joint Simulation Bus (JBUS), a gateway that can translate between several protocols. JBUS converts the AIS messages into DIS and sends it on the local network on broadcast UDP port 3000. The websocket server application reads the DIS and forwards it, in binary format, to the web page, where the DIS entity state PDUs are decoded. Pins are added to the map that represent each ship reported by AIS. DIS’s geocentric position was converted to latitude and longitude to place the pins. Information about each ship can be displayed by clicking on the pin. The application is able to decode approximately 3000 DIS Entity State PDUs per second while displaying the map information when running on a laptop. The primary limitation seems to be map overhead and memory rather than the speed at which the application can decode DIS PDUs.

We have also implemented small 3D virtual worlds using WebGL and Three.js. Small virtual worlds of up to perhaps a few dozen entities in a simple scene can be implemented.

It is realistic to expect that web applications on modern desktop devices can process PDUs in the mid-four digits per second range, depending on what else the application is doing. 3D graphics applications will be much lower; those that only collect summary statistics will be at the high end of this range; and map applications will fall between these two extremes.

4. Conclusions

Either JSON or binary DIS messages can be used to display Live/Virtual/Constructive information in the web browser. JSON is fast enough for most applications, and some browsers decode JSON messages as fast as they do binary. Binary formats have the advantage of being more compact for messages with the same content.

Web application architects should be aware of the web browser being targeted. Javascript engines vary both in absolute performance and in what tasks they do best. Some decode JSON very well; others are better at decoding binary. The efficiency of the Javascript implementation can have a significant impact on the performance of the web application.

Unification of mobile and desktop web applications is possible, at least for some types of applications. Application architects need to be aware of what features are missing from mobile platforms. For example, WebGL is not generally supported on Apple’s IOS for mobile devices.

Binary DIS formats have one significant advantage: the standard has already been specified. JSON formats, in either WebLVC format or in JSON format DIS, do not have a widely recognized standard. Using binary format DIS reduces or eliminates the need for protocol translation gateways, and reduces the need for standards group efforts to define an acceptable JSON message format. The appeal of using the binary format for DIS in Javascript is straightforward: it’s already a standard, while a JSON format is not. Existing gateways can convert protocols like AIS to DIS, and a number of gateways exist that convert HLA RPR-FOM to DIS. Javascript simulation applications in the client browser can be coded using the well-known DIS specification without having to develop and maintain a new protocol.

5. Future Work

In a distributed simulation web application the program logic can span two locations: the client side and the server side. For example in a web application we could configure the server side to forward only new or changed entity state PDUs to the web page; this would eliminate the need for heartbeat PDUs to be forwarded across the bandwidth-constrained websocket link to the web page, while also remaining compliant with the DIS standard from the standpoint of external DIS applications. This illustrates the more general case of

Distributed Data Management (DDM) or Area of Interest Management (AOIM) on the server side.

Modern commercial web applications such as Google Mail or web-based games are cloud-based, and server-side web applications can dynamically scale as demand goes up. From an architectural standpoint, CPU capacity available on the server side is essentially unlimited. The implications of this insight have not been thoroughly explored. It may make sense to migrate more computation to the server side and offload mobile devices, or provide richer computation for applications by exploiting the resources available in the cloud.

The WebRTC API standard is just beginning to emerge from the standards process, and implementations exist on the most recent Chrome and Firefox browsers. WebRTC allows access to UDP sockets, the traditional mechanism for exchanging state information in distributed simulations. WebRTC also allows direct peer-to-peer communications between web browser pages. This capability is an obvious avenue for future research.

Support for the Extensible 3D (X3D) Graphics International Standard DIS nodes (EspduTransform and Signals PDUs) will next be demonstrated by integrating these capabilities into the open-source X3DOM javascript library. X3DOM provides plugin-free X3D rendering within HTML pages, allowing direct composition of X3D and HTML in a single web page (http://www.x3dom.org).

A web-based architecture can also be subject to other factors, such as the latency inherent in communications between the client and server. The tradeoffs have not been fully explored.

References [1] The Khronos Group, “OpenGL ES 2.0 for the

Web,” https://www.khronos.org/webgl. Retrieved June 10 2014.

[2] Internet Engineering Task Force, “RFC-6455: The Websocket Protocol”. http://tools.ietf.org/html/rfc6455. Retrieved June 10 2014.

[3] World Wide Web Consortium, “WebRTC 1.0: Real-time Communication Between Browsers,” W3C Editor’s Draft 10 April 2014. http://dev.w3.org/2011/webrtc/editor/webrtc.html. Retrieved June 10 1014.

[4] Simulation Interoperability Standards Group, “Web Live, Virtual, Constructive Study Group”, http://www.sisostds.org/StandardsActivities/Study

Groups/WebLVCSG.aspx, Retrieved June 10 2014.

[5] Khronos Group, “Typed Array Specification” www.khronos.org/registry/typedarray/specs/latest. Retrieved June 10 2014.

[6] Anonymous, “Can I Use Typed Arrays?”, http://caniuse.com/typedarrays. Retreived June 10 2014.

[7] McGregor, Don; Brutzman, Don: “Open-DIS: An Open Source Implementation of the DIS Protocol for C++ and Java”. SISO Interoperability Workshop 2007.

[8] SEDRIS: “Spatial Reference Model”. http://www.sedris.org/srm_desc.htm. Retrieved June 12 2014

[9] Don McGregor: “Javascript DIS Native vs JSON,” http://jsperf.com/javascript-dis-native-vs-json/2. Retrieved June 12 2014.

[10] Don McGregor: “DIS vs WebLVC,” http://jsperf.com/dis-vs-weblvc. Retrieved June 12 2014.

[11] Don McGregor: “Web-Based DIS Map,” https://movesinstitute.org/DIS/. Retrieved June 12 2014.

Author Biographies

DON MCGREGOR is a member of the research faculty in the Naval Postgraduate School Modeling, Virtual Environments, and Simulation (MOVES) Institute. His research interests include cloud computing, web programming, and network protocols.

CURTIS BLAIS is a member of the research faculty in the Naval Postgraduate School MOVES Institute. He has over 40 years of experience in management, specification, design, development, and application of M&S for training, analysis, and mission planning, and in M&S education. His research interests include application of web-based technologies for M&S interoperability.

DON BRUTZMAN is Technical Director for 3D Visual Simulation and Networked Virtual Environments in the MOVES Institute. As an Associate Professor at the Naval Postgraduate School in Monterey, California he is a member of two Academic Groups: Undersea Warfare and Modeling, Virtual Environments and Simulation. He is an investigator in the NPS Center for Autonomous Underwater Vehicle Research. His research interests include underwater robotics, real-time 3D computer graphics, artificial intelligence and high performance networking.