Honours Project Report

Scalable multi-threaded tool for the visualization of radio

astronomy data cubes

Robin Tyler rtyler@cs.uct.ac.za

Computer Science Department University of Cape Town

Supervised by:

Michelle Kuttel

mkuttel@cs.uct.ac.za Computer Science Department

University of Cape Town

Erwin de Blok edeblok@ast.uct.ac.za

Astronomy Department University of Cape Town

Category Min Max Chosen

1 Software Engineering/Systems Analysis 0 20 10

2 Theoretical Analysis 0 25 0

3 Experiment Design and Execution 0 20 10

4 System Development and Implementation 0 15 15

5 Results, Findings and Conclusion 10 20 15

6 Aim Formulation and Background Work 10 15 10

7 Quality of Report Writing and Presentation 10 10

8 Adherence to Project proposal and Quality of Deliverables 10 10

9 Overall General Project Evaluation 0 10 0

Total Marks 80 80

Department of Computer Science University of Cape Town

2011

Abstract Data cubes are the predominant format for storing radio astronomy data, the resolution and

coverage of which are growing rapidly as technology advances. Radio astronomical observations are typically three dimensional, so the raw volume of data captured is increasing equally quickly, if not more so. Despite this, applications commonly used for viewing data cubes have little support for 3D displays and do not implement a scalable approach to accessing files which exceed the capacity of

main memory.

The approach to visualising data cubes used by standard applications is one focused on the display of detailed, full resolution information. This often involves a large window displaying a static or non-user controlled view of a point in the file. Changing the position removes the old information and

replaces it with new information. With file sizes increasing as they are, the approach of working with such detailed sections of data is becoming increasingly ineffective, if not impossible. Furthermore,

the ratio of seen to unseen data is decreasing rapidly.

This report details the development of a prototype interface which uses a scalable 3D approach to visualising data cubes. The interface implements a technique of 3D volume rendering combined with

a user-driven process of iterative refinement to rapidly analyse vast amounts of data without exceeding machine resources. It seeks to shift the focus from analysis informed by detailed

information, to analysis informed mainly by overview information.

Intended to be the prototype of a single iteration in an iterative development approach, the interface is evaluated using a “client” base of three astronomers who perform expert reviews of the approach. The interface is found to be a highly feasible approach, prompting research into further

iterations and refinements in future works.

Acknowledgement Thank you to Michelle Kuttel who supervised our work and gave us encouragement throughout the

project. Thank you to the Astronomy Department, in connection with whom we worked on this project, especially our co supervisor Erwin de Blok, as well as Sarah Blyth and Kurt van der Heyden.

Table of Contents Abstract 1 Introduction 2 Background

2.1 Introduction 2.2 3D Visualisation

2.2.1 Data Abstraction 2.2.2 Visual Abstraction 2.2.3 Visual Interaction

2.3 Data 2.3.1 Observed 2.3.2 Simulated

2.4 3D Techniques 2.4.1 Isosurfaces 2.4.2 Direct Volume Rendering

2.5 Applications 2.6 Conclusion

3 Design 3.1 System Overview 3.2 User Interface Module: Aims 3.3 Interface Design 3.4 Evaluation 4 Implementation

4.1 Development Details 4.2 Main Interface Window 4.3 Context Control Object 4.4 Overview Display 4.4.1 Volume Renderer 4.4.2 Refine Area Tool 4.4.3 Overview Camera System 4.4.4 Data Visualisation 4.4.5 Data cube Interactions

4.5 Detail Display 4.5.1 Data Selection and Retrieval 4.5.2 Data Display 5 Evaluation and Results 5.1 Volume Renderer 5.1.1 Performance 5.1.2 Results 5.2 User Interface 5.2.1 Analysis Approach 5.2.2 Results 6 Conclusion 6.1 Future Work

List of Figures 2.1 Example of cut-away rendering 2.2 Visualisation using direct volume rendering 2.3 Isosurface rendering 4.1 Roll of Main Window in component communication 4.2 Rendered slices through a volume 4.3 Axis aligned and non-axis aligned slices 4.4 The relation between coverage and refined dimensions 4.5 Overview camera constraints 4.6 Narrow phase plane collision 4.7 Valid placement of selector frame 4.8 Determining closest plane 4.9 Relationship between Detail Display and original data 4.10 Detail Display parameters 4.11 The effect of data contiguity on read speeds 4.12 The relation between surface and texture size 5.1 Frames per second versus data cube volume 5.2 Primer image and its overview representation 5.3 Primer overview representation after refinement 5.4 The effect of thresholding on data visibility 5.5 Visible internal slices using adjusted clipping planes 5.6 Full resolution galaxy data 5.7 Cube refinement increases detail and visibility 5.8 A single object at full resolution

1. Introduction Current generation telescopes are capable of observing and recording incredibly large amounts of data, reaching far into the gigabytes. With initiatives such as the proposed Square Kilometre Array (SKA) and MeerKat, the amount of data being recorded is set to increase even further. Despite the field of astronomy being synonymous with astronomical amounts of data, there is currently little to no support for accessing data files which exceed a computers main memory. Without the capabilities to view the data once it has been recorded, astronomers often resort to indiscriminately segmenting it into small pieces which can be viewed separately. Analysing these numerous segments separately can be painstaking, and the divided data can lose the structures and connections which were visible in the original. The need for a system capable of accessing files of an incredibly large size is apparent, but opening the file is only half the challenge. Once the data is available the sheer scale can make viewing it and analysing it meaningfully a difficult task. Visualising the data effectively is as important a task as accessing the information. An interface which allows a user to see through unwanted information, and focus on more important data, is required. This report investigates a proposed approach for visualising data cubes of a very large size, typically exceeding the bounds of main memory. The experimental nature of the approach requires that the feasibility and applicability to the field of astronomy be evaluated. The project is intended to satisfy the following key success factors

Creating an interface combining approaches of overview and detail which aid in the analysis and navigation of very large astronomy data files.

Create a visualisation approach which is determined by experts to be feasible and possibly superior to existing methods for visualising very large astronomy data files.

2. Background Astronomy, a field often associated with large, multi-dimensional data sets, has long relied on visualisation to effectively convey the meaning of said data to experts and non-experts alike. This paper focuses on the importance of 3D visualisations as way to convey large amounts of information as intuitively as possible. General visualisation techniques are discussed, as well as their applications to 3D astronomy representations.

2.1 Introduction The effective analysis of large, multi-dimensional collections of data is a fundamental element of astronomy. To accomplish this, astronomy has always been closely linked with visualisation [1]. Astronomers have long used graphical representations of data for research and educational purposes alike. Despite the multi-dimensional nature of astronomy data, commonly used visualisation tools focus on 2D imagery, instead of taking full advantage of today’s 3D techniques and technologies *2+. While 2D representations of information are useful for establishing precise relationships between features, 3D views allow users to interpret data qualitatively [3]. Such views are also vital in effectively conveying the meaning of information to non-experts. We will look at key principles involved when visualising data, and how they apply to the field of astronomy. The source and format of the data itself is explored, as well as current techniques used to display it three dimensionally. Examples of real world applications of these techniques are examined. Lastly, the feasibility and usefulness of incorporating 3D visualisations into current astronomy visualisation tools is considered.

2.2. 3D Visualisation 3D graphs and images have, in the past, commonly been used in the final stages of a publication when garnering approval and increasing the popularity of findings. However, this is beginning to change, and 3D visualisations are increasingly being used in the data analysis stage of research projects. Astronomy has a multitude of visualisation techniques, including 2D channel maps and spectrum arrays. These techniques require data analysts to memorise certain aspects of the information, as dimensions beyond the second have been abstracted away [4]. This is why 3D visualisation is of particular interest when analysing large data sets like those found in astronomy. When visualising large amounts of data, it is necessary to perform various abstractions in order to effectively and intuitively convey information. Visual and data abstractions are commonly used when displaying a large amount of information [5]. Both these practices have specific applications in the field of astronomy. The ability to interactively combine these abstractions to change visualisation parameters in real time has proven invaluable in many research fields [1].

Figure 2.1: Demonstration of cut away rendering in a single frame from a supernova core collapse simulation. Visualization

from The Terascale Supernova Initiative at Scientific Discovery Through Advanced Computing.

2.2.1 Data Abstraction “Data abstraction refers to transformations applied to the data before being visually mapped including aggregation, filtering, sampling, or statistical summarization” *5+. A key application of such a process is the identification of features of interest and the occlusion of interfering data. This can be seen in “cut away” views such as in Figure 2.1. Here, a cross section of the internal structure present in a supernova core collapse simulation gives researchers an insight otherwise obstructed by unnecessary information.

2.2.2 Visual Abstraction In visualisations, visual abstractions change only the appearance of displayed information, without affecting the underlying data being displayed [5]. Such abstractions are used to gain a new perspective on existing information, allowing researches to form new conclusions. Common examples in 3D include translating, scaling, and rotating in order to improve understanding of data through different views. Visual abstractions can also aid in visually filtering irrelevant data through the use of colour and opacity values combined with direct volume rendering, discussed later.

2.2.3 Visual Interaction The ability to define which quantities from multivariate data to display and how to display them, then seeing real time changes in the visualisation, gives researchers a new and informative way to interpret large data sets. The time taken between the input of parameters and the reflection of this input in the visualisation is important when creating interactive displays [1]. Especially when analysing time dependant data, common to many astrophysics simulations.

2.3 Data Astronomy data can come from different sources and convey different aspects of information about objects. Due to the close link between astronomy and visualisation, the type of data can dictate which visualisations are viable and what information is available to be visualised. Astrophysical data is typically either categorised as observed or simulated [1].

2.3.1 Observed Observed data is collected from ground based or orbital telescopes and probes in the form of varying wavelengths of electromagnetic emissions. Due to the limited ability to gain views from multiple angles, spatial information gathered from such sources is typically restricted to two dimensions. A common technique for visualising such information in 3D is to use velocity as the third dimension, a value obtained from calculating the Doppler shift present in measured spectral data. Although such a model is not spatial, insight into many spatial structures and properties can still be gained [2].

2.3.2 Simulated Researchers frequently use analytical models intended to recreate astrophysical phenomena in order to test and develop hypotheses [1]. These simulations typically consist of millions and even billions of bodies interacting to output scalar values in 3D coordinates. This makes 3D visualisations ideal for displaying the spatial nature of the results, and allowing researchers to glimpse what such interactions may look like. Unlike observed data, the complete nature of the information allows interactions at every level to be graphically realised. However, multiple features of interest can lie along the same line of sight, and important information can be obscured, making the approach to visualisation and the techniques used an important factor.

2.4 3D Techniques The technique used to visualise astrophysical data in 3D is closely related to the data and how complete it is. Many of the techniques are inspired by imaging processes used in medical science, especially applications of tomography. The close link between the need for 3D imaging in both medicine and astronomy has been identified by research institutes such as the Harvard Astronomical Medicine Project [6]. Here, techniques and software typically used for creating 3D images from data gathered by MRI or PET scans are used to instead visualise astronomical data. Due to the vast space between observed features, and the limited viewpoints observable in astronomy, there are fundamental differences in visualising the data. However, various principles remain the same, and these similarities have been used in novel ways to gain new knowledge in the field. The technique used largely depends on the data available. Two predominant models of output exist, which differ mainly in depth of visualisation. Isosurfaces are typically used to visualise surface information, while direct volume rendering can display an object in complete 3D detail.

2.4.1 Isosurfaces An isosurface is a 3D surface representing points of equal value in a 3D data set [8]. Isosurfaces are commonly used in medical imaging to show areas of equal density within the human body [9]. Such a visualisation is ideal for observed astrophysical data, as information on internal structures is difficult to obtain. However, surface data is present in the 2D images of varying wavelengths of electromagnetic emissions received by telescopes and probes. A common practice is to calculate the Doppler shift of these wavelengths, allowing researchers to find the velocity of particular objects. Velocity is then used as the third dimension in visualisations, with spatial coordinates making up the other two (usually right ascension and declination). This technique has been used effectively to reduce the time taken to discover outflows of gas originating from the formation of young stars [2], discussed later.

A frequently used method for creating 3D isosurface displays is the Marching Cubes algorithm [7]. This algorithm creates a polygonal mesh consisting of multiple “cubes” shaped to fit the contour of the volume. This is achieved by combining 8 neighbour locations, forming an imaginary cube. It is then determined which of these 8 locations lie within the surface of the required volume, and which points lie outside. These values are used to determine the shape of the polygonal mesh relating to that specific cubes position. 256 possible arrangements exist for a single cube, all of which can be mapped to 14 variations through rotation and symmetry. This property is used to greatly increase rendering speed. A weakness of this algorithm is that it relies on fairly regular contours of values in order to render a smooth 3D surface. Such a property is not insured when dealing with astrophysical data, potentially creating sudden spikes in a model. However, this property has been used for identifying irregular features in certain cases [2].

2.4.2 Direct Volume Rendering Values in a 3D data set can be visualised in a number of ways, including a view of a 2D slice of the data, or approximating contour surfaces such as isosurfaces. Direct Volume Rendering involves creating a projected image directly from the 3D data. No intermediary functions or approximations are applied to the volume data. Direct volume rendering is best suited to visualising astrophysical simulation data, as the complete nature of the data set allows for inner structures to be visualised within the 3D volume. Additionally, no abstract geometric structure is imposed on the data, keeping the visualisation as intuitive as possible.

Figure 2.2: Direct volume rendering of a single time step of a core collapse supernova simulation. The scalar values

represented are of entropy and were mapped to an opacity and colour value.

An additional benefit of direct volume rendering involves a technique utilising ray casting. Many surface extraction techniques fail to account for materials which semi-transparent. Ray casting does not suffer from this limitation. In astronomy, the greatest advantage of this technique isn’t necessarily in rendering objects more realistically, but rather for allowing researchers to focus on important features. This is accomplished by mapping scalar values to opacity and colour, allowing for irrelevant information to be made more transparent and obscured objects to be revealed [1]. An example of this can be seen in figure 2.2. Direct volume rendering requires a value for every spatial coordinate. This prevents it from being used for the majority of astrophysical observational data.

2.5. Applications As described earlier, 3D visualisations, namely Right Ascension – Declination – Velocity graphs, have been used to great effect in quickly and accurately locating and identifying potential molecular outflows by Borkin et al. (2009) . Outflows are high velocity flows of gas which form near young stars. Due to this irregularly high velocity, these areas present as spikes when observing an isosurface representing velocity values, and allow for rapid identification [10]. A visualisation used in this study can be seen in figure 2.3. The original method for identifying such features involved viewing multiple 2D images or measurements of the same area, possibly taking months to complete accurately.

Figure 2.3: Isosurfaces representing the 12CO (purple) and 13CO (pink) emission in the B5 star-forming region of Perseus

with high velocity points (teal) and the B5 outflow (yellow) marked.

3D visualisation has also been used to identify properties occurring within arguably the most powerful and important features in the universe, supernovae. Due to the complex and seemingly chaotic nature of these events, a strong dependency on computational physics has formed. Advancements in our knowledge of supernovae have accompanied each advancement in the way we simulate and view these events. 1D hydrodynamic simulation was used to support the hypothesis that supernovae were powered by the gravitational energy from the core collapse of a star [11]. 2D Simulations were then used to discover a phenomenon known as spherical accretion shock instability (SASI). Finally, the system was then moved to a large scale 3D simulation and visualisation, and a non-axisymmetric mode of the SASI was discovered [12].

2.6. Conclusion Given the sheer volume of most data sets in astronomy, 3D visualisation is recommended if not required to fully comprehend the information being presented. Presenting data in 3D is not only vital when conveying as much information as possible, but also when attempting to occlude unwanted information in order to focus on features of greater importance. Given the refinements in 3D visualisation techniques, and the advancements in graphical hardware, rendering 3D displays is no longer in issue when ensuring timely responses to user interaction. In a field such as astronomy, which is so closely related to visual analysis, all visualisation tools should be supporting 3D displays of information.

3. Design The goal of this research project is to develop an application which allows access to, and analysis of, astronomical FITS data files. The files in question often exceed the size of the main memory available in a desktop computer, making fast efficient access to the data impossible with current applications. Additionally, difficulties arise when one must accommodate both the need for high spatial resolution as well as the limited field of view with which humans can sense their visual environments. We intend to overcome these difficulties by adopting an approach of Overview + Detail described by Schneiderman et al. as “the use of interaction to keep a view of the whole data available, while pursuing detailed analysis of a part of it” [1]. The interface will provide the user with detailed visual and analytical access to a small sub-region of the astronomical image data. This data will be of high resolution and the user will be able to alter the display, using tools such as panning and zooming to navigate through the data. A data-overview display will accompany this interface, providing the user with an overview of the entire set of available data, but in a lower resolution format. This Overview + Detail approach thus combines a high coverage, low resolution display (overview) with a low coverage, high resolution display (detail). This provides the user with a broader view of the data in addition to detailed access to specific parts of it. The end result is a system which “reduces search time, allows the detection of overall patterns, and aids the user in choosing the next move” *1+.

3.1 System Overview The proposed system consists of three parts: File Access, Image Processing, and the User Interface. Communication between components occurs primarily in that order, with the User Interface making the majority of its requests to Image Processing, and Image Processing to File Access. Exceptions to this pattern occur when conversion of data to a human readable form is unnecessary, such as opening files from disc or creating and saving entirely new files.

The File Access Component is concerned with all properties of the FITS file and the data within it. Its primary function is to provide the Image Processing and User Interface components with an object which simulates main memory access, without loading the entire file into memory at any one time. This component's performance is focussed on minimising memory requirements and servicing memory requests made by the other components as quickly as possible. In aid of this, the component implements a caching strategy to speed up data requests, as well as a FIITS file reader and writer, for opening existing files and creating new ones respectively. The Image Processing Component compiles data returned by file access requests into displayable images. The User Interface will typically request an image to be shown in either the overview or detail display, which the Image Processing component will request from File Access, and convert to a displayable format. The bulk of the work done by the Image Processing Component will be in constructing the overview representation. This will involve examining data values at interspersed locations throughout the data and compiling a condensed representation of the entire image, displayable in an area no larger than 300 to 400 cubic pixels. Although the file access stage will most likely be a bottle neck in terms of performance time, optimising this stage will ensure a responsive interface for users. Another concern is accuracy, which in this case translates to creating an overview display which is highly representative of the original data, and easy to use when navigating it. The User Interface Component provides the user with tools for analysing and navigating through image data stored in the FITS file format. Because the focus is on files of especially large sizes and resolutions, the interface seeks to primarily enhance file navigation, rather than provide in-depth scientific analysis. The primary goal of the User Interface is to facilitate informed file navigation, requiring as little main memory as possible, while still providing the user with all the information they may need. This report is concerned with this component and the remainder of this chapter will focus exclusively on its design.

3.2 User Interface Module: Aims The main aim of the User Interface module is to provide an intuitive and effective method for navigating through extremely large files. In this context, effective navigation involves allowing a user to make directed queries which are informed and deliberate. Given data sets of ordinarily unmanageable sizes, a user should be able to rapidly find and focus on the information most relevant to them, while being able to ignore irrelevant and noisy information. In the process of providing the user with an informed view of the data set, the process of identifying large scale structures and patterns will be simplified. Once patterns within data sets have been identified, a detailed comparison with other, potentially similar datasets becomes possible. Although the amount of disk memory accessed by the user can be enormous, responsiveness is also a key priority for the Interface subsystem. If a user’s request cannot be fully completed within a matter of seconds, it is important that a partially complete response be returned and iteratively improved upon, preventing the application from “hanging”. The time taken to complete a request will vary depending on the size of the data being accessed. However, in order to ensure scalability, the time taken to return partially complete iterations will be kept constant.

While the focus of the project is providing access to very large FITS files, it is also important that the user be able to examine the data once it is available. Although complex and highly informative methods of analysis exist, their implementation is outside the scope of this project. The primary form of analysis provided will be intensity thresholding and binning. This involves making intensity values within certain value ranges easily visible, or in the case of thresholding, making them partially or completely transparent. This visual abstraction will allow the user to quickly locate areas of particular interest.

3.3 Interface Design a) Detail Display

The Detail Display viewport will be centred over a specific point in the image data. The User Interface will request the data associated with this point from the Image Processing component. The data returned will correspond to the full resolution values extracted directly from the file being accessed. A low resolution image will be fetched directly from the already existing down-sampled overview data, and displayed until the full resolution data is available. This will serve to hide the delay between data requests and completions.

b) Overview Display

The Overview Display component will display an approximate representation of the image data as a whole. It will do this by mapping each pixel visible in the overview (roughly a 300x300x300 display) to a region of the image data being represented. By systematically retrieving values within the region, it will seek to approximate the correct value for this pixel. Scanning every value present in the data will be computationally expensive, so multiple scans will be performed, each extracting data not yet analysed. This process will allow the Overview Display to be iteratively improved over time, starting as a coarse, inaccurate representation, but rapidly becoming more accurate and representative of the image data. This will allow the application to be used almost immediately, rather than waiting long periods of time for information to become available. Once a FITS file is selected and loaded using the interface, information regarding the images height and width, and how it should be segmented, will be provided for the Image Processing component. A dedicated thread will run within the component, continuously requesting more information, and refining the overview image. The User Interface will periodically request the current iteration of the overview cube and display it to the user. The user will be able to use the Overview Display for rapid navigation by clicking on a location within the image. This will focus the Detail Display on the specific location, allowing the user to investigate at a higher resolution. The region of the image data being displayed in the Detailed Display will always be mirrored in the Overview Display, giving the user an intuitive reference point for further navigation.

3.4 Evaluation At the proposed system is intended for use in scientific analysis, general usability is not the primary

criteria we wish to evaluate. Instead, we will determine if the approach is considered feasible and

whether or not it could be considered an improvement over the currently available methods for

viewing large data cubes. A process of expert review will be undertaken to evaluate these criteria. As

the system is being developed for use in astronomy, qualified astronomers will be consulted.

4. Implementation This chapter discusses the details of the implementation of the GUI component. The development libraries and platforms used, and the reasons for these choices, are presented first. The separate components of the interface are analysed in terms of their purpose and implementation, and limitations and constraints are discussed as well.

4.1 Development Details Linux

Ubuntu Linux was chosen as the development platform for the project because it is Open Source, has several IDEs are included with the software, and the project team has familiarity with the operating system.

C++

To ensure compatibility between the various libraries used across all components involved, C++ was used to program the GUI.

Qt Framework

The Qt API was used in the creation of the GUI in order to avoid recreating commonly used interface components. This provided a platform compatible with other components within the application, as well as easy access to tools required for the GUI components, such as OpenGL.

OpenGL

The OpenGL API provides comprehensive 3D rendering capabilities, is Open Source, and is portable in terms of operating system compatibility. Its use is also well supported by Qt, the framework API used in creating the user interface.

4.2 Main Interface Window Function The interface’s Main Window contains all usable components of the GUI, and is responsible for their creation and control. The Main Window also performs functions which transfer signals between separate interface components, which are otherwise unconnected.

Implementation The Main Window class extends the QMainWindow class from the QT library, and is created at application start up. When created, the Main Window class instantiates all visible UI components and has a second Layout Widget class which assists with their sizing and positioning within the window. The bulk of the Main Window’s processing is performed by down-samplers, which produce displayable arrays of information extracted from FITS files. Multiple down-samplers can exist at a time, with every active session typically containing one overview-sampler and one detail-sampler. Tasks performed by a down-sampler object include: sorting of file reads, numerous disc accesses, and of course, down-sampling large collections of data into smaller sizes. These tasks are too demanding to be performed while in contention with other down-samplers, so the application is limited to one active down-sampler at any one time. The Main Window manages a down-sampling queue, which alternates between active down-samplers, allowing each to complete a single iteration of processing. The progress of a down-sampler is monitored by means of a polling timer, which periodically checks whether an iteration has been completed. The polling process begins when the queue’s length becomes non zero, and is stopped once the final object in the queue is removed.

All file operations are performed within the Main Window class. The application is able to access data cubes contained inside FITS files. Once a FITS file is opened, a Context for the session involving that file is created and placed in a Context list, containing all other active sessions. The Main Window manages these active contexts, and switches between them on demand, changing the interface’s current session. It is also possible to save and open custom files, known as overview data cube files, which have the extension .dc. Saving such a file while a session is active will write the entire contents of the currently displayed data cube array to file, as well as the cube’s dimensions, to be used during reconstruction. This functionality is implemented due to the potentially lengthy process involved when creating the overview data cube. At a later time, the user can open the file, which will instantly display the saved data cube, rather than having to recreate it each time.

Figure 4.1: An example of information being passed between the Overview and Detail components, via the Main Window.

When input is received in the Overview Display, a signal is sent to a slot function in the Main Window, passing on parameters to the Detail Display.

The Main Window communicates input which happens in one component, to another component which must reflect any changes that occur. This is done using signals and slots provided by the QT Framework. Public QAction objects are placed in components which need to communicate changes in state to other components. These QActions are connected to functions in the Main Window class, called slots. When a QAction is triggered by a UI component as a result of user input, the slot performs a task, informing a target component of the input, and potentially conveying relevant information.

4.3 Context Control Object Function Context Control objects store information and tools relating to specific sessions active within the application. Contexts allow a user to change between currently active sessions and provide a method for accessing frequently needed interface settings to any component which holds a reference to the current context. Implementation One Context Control object is created per session. A session refers to the environment in which a specific dataset is being examined and is typically created when a FITS file is opened, or an existing session’s dataset is refined to a smaller subset. The bulk of the information held by a Context Control object consists of two sets of data: the overview data cube and the detail display data. This information is held by a context’s overview-sampler and detail-sampler respectively. When a context is activated in the interface, it becomes the current context of both the Overview and Detail display components.

The Overview and Detail Components contain a pointer to a Context Control object, which references the currently active context of the application. This context contains information central to the purpose of the respective components, such as the overview data cube, and detail display data. The context also contains interface parameters which affect the display functions of both components, such as the current magnification setting, or the position of the clipping planes, which are discussed later. This means that changes made to the current context are instantly reflected in both displays, ensuring they correlate perfectly at all times.

4.4 Overview Display Function The purpose of the Overview Display component is to visualise the output produced by the down-sampling process. This component also provides a means of focusing the Detail Display on a specific point in the original data.

4.4.1 Volume Renderer Function The data cube must be displayed so as to provide the user with as much information about as much of the FITS file as possible. Implementation Due to the target data typically being in the form of a data cube, a 3D representation technique is used, with the capability to display 2D data. The technique implemented is 3D volume rendering using textured slices. The data cube, a one dimensional float array, is stored in the Image Processing component, along with the width, height, and depth dimension values, required to reconstruct the cube. The array consists of float values between 0.0 and 1.0, which represent adjusted intensity values, extracted from a FITS file. This array is converted to a 3D texture consisting of RGBA values using the glTexImage3D function. The RGBA value of each pixel in the 3D texture can be written as (1.0 , 1.0 , 1.0 , α), where α represents the adjusted intensity value for that position in the datacube.

Figure 4.2: A volume with multiple slices being rendered through it. Each slice is separated by an interval of Δd.

In order to ensure that the entire data cube is visually accessible when displayed, a volume rendering technique using textured slices has been implemented. This technique involves rendering quads which “slice” the 3D texture at numerous locations. This is accomplished by assigned texture coordinates to each vertex of the quad which places it “inside” the volume, displaying a 2D representation of the data which intersects the surface. By rendering multiple “slices” at regular intervals, and through the use of alpha blending, the 2D representations can be combined into a seemingly 3D display of the entire data cube.

A alternative approach we considered was to constrain the total number of rendered slices to a specific amount, and then vary the distance between each slice to suit the size of the volume being rendered. This approach provides a simple way of keeping performance constant and reliable, but has numerous draw backs. The main concern for an implementation involving scientific analysis is variations of accuracy and consistency in the display. For large volumes, where the distance between consecutive slices becomes relatively large, information which lies completely between two slices is not displayed at all. For this reason, it was instead chosen to fix the interval between slices and vary the number of slices depending on the dimensions of the volume. The distance between slices is roughly equal to the distance between data values in the 3D texture, ensuring a high degree of coverage. The orientation of these slices within the volume is also important in determining the appearance and coverage of the representation. Aligning the slices parallel to the axis of the volume requires very little computation and can be effective if the users’ view point can be constrained. However, as the view direction becomes perpendicular to the normal of the slices, the intervals between slices become visible, resulting in some screen pixels not being covered by any slices. In order to limit this effect at near-perpendicular view directions, the gap between successive slices must be very small, requiring many more quads to be drawn, resulting in slower rendering speeds. At completely perpendicular viewing directions, however, the volume becomes un-viewable.

Figure 4.3: The respective responses of axis aligned slices (left) and non-axis aligned slices (right) to the viewpoint at A.

To eliminate this problem, slices can be rendered perpendicular to the user’s view direction, preventing the intervals between quads from becoming visible. To correctly orientate the quads, the gluUnproject function is used to map window coordinates into object coordinates. Window coordinates consists of an x and y value, representing a point in the Overview Display window, and a third depth value, z. Object coordinates, while also consisting of an x, y and z value, exist in 3D space. By keeping the x and y coordinates in window space constant and varying the depth value z, a ray is constructed in object space which is parallel to the viewing direction. By projecting 4 separate rays, choosing a specific depth value will yield 4 coordinates in object space which can be used as vertices to create a plane perpendicular to the viewing direction. The 4 window coordinates chosen are the minimum and maximum x and y coordinates which the data cube occupies on screen. By applying the relative texture coordinate of each vertex, the area of the plane which lies inside the data cube will be textured with the values it intersects.

A portion of the projected plane’s surface will not lie within the data cube’s volume, and must be excluded from the display. To prevent this area from being rendered, 6 clipping planes, one for each face of the data cube, are used. The clipping planes identify geometry which lies outside the area described by the dimensions of the data cube, and remove it from the rendering pipeline.

Figure 4.4: Green represents Refined Dimensions, Red represents Coverage Dimensions. (left) An example where no clipping plane adjustments have taken place. Coverage Dimensions = Refined Dimensions. (right) The lower xz plane has been raised

by approximately half the height of the data cube. The Coverage Dimensions remain as before, but the y value of the Refined Dimensions has increased from 0 to height/2, while the height value has decreased from height to height/2.

The user is able to adjust the position of the clipping planes, removing layers from the data cube’s display. This allows unwanted information to be removed from the display, as well as making inner layers of the volume visible, exposing information which would otherwise be occluded by an opaque outer surface. This functionality is managed using two integer arrays, each of length 6, called the Coverage Dimensions and Refined Dimensions respectively.

Coverage Dimensions : x, y, z, Width, Height, Depth These values describe the dimensions of the FITS file data cube which are currently represented by the overview data cube. This means that a newly opened FITS file has 100% coverage, with the x, y and z values being equal to 0, and the width, height and depth values being equal to those of the original FITS data cube.

Refined Dimensions : x, y, z, Width, Height, Depth Similar to Coverage Dimensions, however the Current Coverage Dimensions include the adjusted position of the clipping planes. These values describe the portion of the FITS file data cube which is currently being visually displayed.

The process of shifting clipping planes is constrained by these two arrays. That is, a refined x, y or z value can never be less than a coverage x, y or z value, and a refined width, height or depth value can never exceed a coverage width height or depth value. The relation between these two arrays is illustrated in figure 4.4.

4.4.2 Refine Area Tool Function The user must be able to increase the level of detail displayed in the overview, but only with regards to a selected area, so as not to exceed memory limitations. Implementation The ability to adjust the position of clipping planes and reduce the size of the

Refined Dimensions gives rise to the Refine Area tool. This allows the user to effectively change the

value of the Coverage Dimensions so that the overview data cube only represents a subsection of

the entire FITS file. Once used, all subsequent data requests will ignore data which lies outside the

volume defined by the Coverage Dimensions, effectively creating an accessible FITS file of a smaller

size

The purpose of this refinement process is twofold. Firstly, reducing the size of the data being

accessed decreases the amount of time taken to create an overview data cube to display. When a

Refine Area request is made, the currently active down-sampler is killed and ceases to process. This

leaves all of its data intact, but resets its down-sampling cycle and removes it from the Main

Window’s sampler queue. This prevents the application from spending time processing areas of a

FITS file in which the user has no interest.

The second purpose involves the quality of the overview data cube. How well the down-sampled

output resembles the original data depends mainly on the down-sample ratio, that is, the size of the

overview data cube in relation to the FITS file data cube. The closer to 1 this ratio is, the higher the

quality of the display. However, the Overview Display has certain practical limitations, which cause

overview data cube dimensions of roughly 512 units or greater to begin to reduce usability,

performance, and memory efficiency. By reducing the FITS file coverage and keeping the overview

data cube’s dimensions roughly constant, the down-sample ratio is reduced, creating a higher

quality display.

4.4.3 Overview Camera System Function A virtual camera object is created which represents the point from which the user views the 3D display. The user should be able to easily manoeuvre the camera in 3D space, while being provided with comprehensive visual access to the data cube display. Implementation The camera object has rotational and magnification capabilities when viewing a data cube in 3D. The movement of the camera during these operations is described by a sphere surrounding the centre of the data cube. The radius of the sphere is defined by the current zoom factor of the camera, while the position on the sphere is determined by the circular and vertical rotation components respectively.

Figure 4.5: An illustration of the motion permitted by the camera system. The black dot represents the camera location,

while the black lines are its paths of motion, while always facing towards the centre of the square. The circular constraints of the system (left) are shown from a top down perspective. The vertical constraints (right) are shown from a side on view.

The circular component of the rotation represents horizontal camera movement around the sphere and occurs about the y axis. The viewing transformation needed for this rotation is created using the gluLookAt function which requires three parameters; the camera location, the location of the centre of the scene (which the camera faces), and an up vector. For this implementation, the centre location and up vector are kept constant at (0 ,0, 0) and (0, 1, 0) respectively. The camera location is determined by the circular rotation and the zoom factor value: (zoom × cos(circular) , zoom × sin(circular) , 0). The vertical component of the rotation represents movement up or down the sphere in the vertical direction, occurring about the x and z axis. This camera movement is simulated by instead rotating the data cube about its centre point, while keeping the camera position stationery. Because the rotation of the data cube must occur relative to the cameras current position, the resulting operation is determined by both the vertical and circular components. The user can adjust these two components using either mouse movement or on screen icons. While the circular rotation component can be adjusted from 0 to 360 degrees, the vertical component is restricted from -90 to 90 degrees of rotation. Further rotation would result in the data cube being viewed “upside down”, which could cause users to lose orientation.

4.4.4 Data Visualisation Function Due to the scale of the information being displayed, it is important that user be assisted in identifying certain properties of the data which makes up the 3D cube. Implementation In order to improve the users understanding of the data being displayed, values within the datacube are assigned a colour and opacity value. This assignment is determined by the individual data value’s magnitude, as well as user defined parameters. A red, green, or blue colour is applied to a datacube element by separating data values into high, medium, and low categories, based on magnitude. The values which determine the divisions of these categories are fully customizable by the user and changes made occur in the 3D display in real time. By adjusting the categories, a user can quickly identify the locations of specific values in the data cube.

Opacity values are assigned in a similar manner, divided into three categories, determined by two boundary values. Intensity values which fall below the minimum boundary are made completely transparent and do not appear in the display. Intensity values which exceed the maximum boundary value are made completely opaque in the datacube display. Lastly, values which fall between both boundaries are assigned an opacity value based on a linear gradient. This visualisation is useful for removing unwanted information from the display while still being able to view the overall structure present in the datacube. Changes made to the minimum and maximum boundaries are also immediately visible in the data cube display. These visualisations were implemented using a simple fragment shader which can be seen in appendix A.

4.4.5 Data cube Interactions Function As a central component in the GUI, the Overview Display must support several interactions which provide access to certain application functionality. Despite this interface being primarily 3 dimensional, the interactions should be easy to comprehend and use. Implementation The primary purpose of the Overview Display is to assist and direct the use of the Detail Display. The two display components are linked by a frame called the selector, which acts as a view finder. The selector lies on an exposed plane of the data cube, “selecting” the visual data which lies within its area. This represents what portion of the data cube is currently being displayed in full resolution in the Detail Display. Positioning the selector is accomplished via mouse input from the user, represented by 2D window coordinates. The process of positioning the selector in 3D space using 2D input is done in two phases; a broad phase which determines which of the 6 cube planes the user is selecting, and a narrow phase which determines the exact coordinates at which the mouse cursor intersects the chosen plane. To perform the broad phase, an openGL API function called selection mode is used, which alters the rendering process. This render mode allows the viewing volume to be redefined, creating only a small area around the cursor which is rendered. Data cube planes which intersect this viewing volume, and thus lie under the mouse cursor, are determined using a data structure called the name stack. Before drawing a plane, a name in the form of an integer is pushed onto the stack. If the surface being rendered intersects the viewing volume, the name currently at the top of the stack is recorded along with the depth of its geometry. Once all scene objects have been rendered, the hits are sorted according to depth, and the closest hit is returned. A result of 0 to 5 is returned for a positive plane collision, with a result of -1 being returned for no data cube collisions.

Figure 4.6: A ray can be cast originating at the mouse cursor and passing through a plane, allowing the exact point of

collision to be found.

Once it is determined which plane has been selected, the exact collision coordinate can be calculated. This is done using the equation of the selected plane and a vector constructed using gluUnproject with the 2D mouse coordinates and 2 differing depth values. For vector V with start point Pstart and plane with normal N passing through point Pplane the exact point of intersection Pi can be written as Pi = Pstart + λV Where lambda is the solution to the following equation λ = ( N ∙ ( Pplane - Pstart) ) ÷ ( N ∙ V ) Once found, Pi is used as the centre point for the selector frame. The location of the centre point is constrained so that the resulting area of the selector lies entirely upon the face of the data cube. This calculation is affected by 4 variables: the original centre point, the dimensions of the selected plane, the dimensions of the selector itself, and the adjusted position of the surrounding clipping planes.

Figure 4.7: The green surface represents the area with which it is possible to register a collision. The red area represents the

valid locations for the centre of the selector frame.

The selector is positioned by double clicking at a specific location on the desired face of the data cube. Alternatively, if the selector has previously been activated and is visible, its position can be changed by clicking and dragging while the cursor lies within its selection area. When the selector is activated or its position changes, the Detail Display adjusts its view accordingly. This occurs by triggering a QAction object in the Overview component, which conveys a signal to a function in the interface’s Main Window, informing it of the update. This function in turn informs the Detail Display of its new parameters.

Figure 4.8: The circular rotation divisions (left) and the vertical rotation divisions (right) for determining the closest face. A

vertical division that does not equal zero takes precedence over all circular divisions.

Another interaction supported by the 3D Overview Display is the ability to manually shift the clipping planes present in the volume renderer, altering the display dimensions of the data cube and providing visual access to internal slices. This is implemented by using a single face of the data cube as an interface to all 6 clipping planes and their positions. The cube face in question changes dynamically and represents whichever surface of the cube is currently closest to the user’s view point, known as the closest face. The closest face is determined using the following calculation, illustrated in the figure above (proper numbering once formatted): ( ( circular ÷ 45.0f ) + 1) ÷ 2 ) mod 4 This determines which of the 4 faces in the horizontal plane the user is currently facing. This finding can be superseded by the following calculation which determines whether the user is currently closest to the top or bottom face of the data cube: Maximum( -1 , Minimum( 1 , ( vertical ÷ 45 ) ) ) Interactions with the clipping planes are performed relative to this surface and consist of two different input methods. The first method involves the use of a 2D slider interface visible in the Overview Display. The interface has two marked locations, one on either end, the positions of which can be changed. These represent the relative positions of the two clipping planes which are parallel to the closest face, the closest of which is represented by the slider on the right, and the farthest by the slider on the left. A slider can be moved by clicking on and dragging it, which moves the corresponding clip plane. This allows the user to view consecutive slices in the volume without having to adjust the position of the camera. The second input method involves directly manipulating the data cube through interactions with the 4 edges which make up the closest face. This involves a process similar to that used in the narrow phase of the data cube face collision implementation. However, this check is active at all times, returning a positive hit whenever the mouse is within distance of one of the closest face’s edges. When a hit is detected, the edge in question changes appearance so as to indicate that it has been selected. At this point, clicking and dragging the edge will cause the clipping plane to be translated in the direction of movement. This allows all clipping planes perpendicular to the closest face to be manipulated.

4.5 Detail Display Function The Detail Display holds FITS file data in its original format. The data displayed in this component is extracted directly from the currently accessed FITS file without compressing or averaging any values. Users use the Detail Display to examine values as they were recorded, while being provided their pinpoint location within the FITS file.

4.5.1 Data Selection and Retrieval Function Various attributes of the Detail Display affect the amount of data that can be viewed in detail. This must be reflected in the data selection process so as to accurately inform the user of what data will be retrieved and displayed. Data requests originate from the Overview Display and exist in overview coordinates which are not compatible with direct file access. It must be insured that the correlation between data requested and data retrieved is maximised. Implementation The Detail Display component displays image data selected by the user in the Overview Display. This selection is represented by a 2D frame called the selector which lies on one face of the 3D data cube. The current state of the Detail View will determine the amount of data it is able to display. Because of this, the dimensions of the selector change as the state of the Detail View changes. The change in dimension of the selector accurately depicts what data will be displayed in the Detail Display, increasing correlation between it and the Overview Display.

Figure 4.9: The relationship between the full resolution data and the full resolution display, via the down-sampled data.

The primary factors which affect the selector’s dimensions are 1. Detail View Dimensions

The size of the Detail View is not necessarily static. The selector must represent the number of pixels which can currently be accommodated by the display. For example, the selector for a Detail View of size 400 by 300, placed on a data cube face representing data of resolution 800 by 600, will cover exactly one quarter of the surface.

2. Down-sample Ratio The size of the selector is determined by how many full resolution pixels are represented by a single overview pixel. For example, an 800 by 600 image down-sampled to a 100 by 75 overview has 64 full resolution pixels per overview pixel. The same image down-sampled to a 400 by 300 overview has only 4 full resolution pixels per overview pixel. Because of this, the selector’s dimensions will change depending on which face of the cube it is currently on.

3. Detail View Magnification The Detail View is able to zoom in and out on full resolution data, adjusting its magnification, with a default magnification of 1. Doubling the magnification has the effect of doubling the scale of each point of data in the display, but halving the overall amount of data. This in turn halves the size of the selector. Conversely, halving the magnification halves the scale and doubles the amount of information displayed, doubling the size of the selector.

Figure 4.10: The two parameters required for the extraction of full resolution data are the location of the selector in

overview coordinates, and the normal of the selected cube face.

A signal is sent to the Detail Display, via the Main Window, whenever a change to the selector’s position is made. This signal contains two parameters; the location of the selector frame in overview coordinates and the normal vector of the selected face. The selector’s location is used to determine where to begin extracting full resolution data from the FITS file. The amount of data to extract is determined by the Detail Display’s dimensions divided by the current magnification. How to extract the 2D slice of data from the 3D data cube is determined by the normal of the selected face. For example, if the normal is parallel to the Z axis, the width and height components of the Detail Display’s dimensions are applied to the X and Y coordinates of the selector’s location. This yields two points which define the area of data to be extracted from the FITS file. This process begins by converting the selector’s location from overview coordinates to file coordinates. Overview coordinates are affected by the down-sample ratio of each dimension and do not reflect the data’s position in the file. The coordinates are converted using the following formula:

Once the file coordinate of the first extraction point is determined, it is passed, along with the width and height of the 2D slice, to a down-sampler object called a full res sampler. This object performs the same role as the overview down-sampler, but performs no dimension reduction, returning unaltered, full resolution data from the FITS file. Before the request is made, any active processes which are accessing the hard disk are paused, typically an overview down-sampler, until the full resolution request is completed. This prevents more than one set of read requests to the hard disk from taking place simultaneously, a scenario in which read speeds would be excessively reduced, making the GUI unresponsive. Because a request for full resolution data represents active input from the user (a double click, or dragging the selector), it is prioritised above all other file access requests, and must be returned as soon as possible.

Figure 4.11: The determining factor in how quickly a full resolution data request completes is how contiguous the individual

requests are. White lines represent contiguous reads, while black lines represent breaks in contiguity. Reads occur left to right, top to bottom.

Time taken to complete a full resolution data request can vary largely, and depends primarily on the size and orientation (normal vector) of the request, as well as the dimensions of the file. The difference in request return times is due to how contiguous the data being read is, or is not. Data being read from the XY plane is row contiguous, with the distance between non-contiguous reads being determined by the width of the data cube. This situation is ideal, with return times being very fast. Data being read from the ZX plane is also row contiguous; however the distance between non-contiguous reads is greater, being determined by both the width and the height of the data cube. This situation is less ideal, though return times are still relatively quick. Requests from the YZ plane have no data contiguity at all, with each request being at least the width of the data cube away. This is the least ideal situation, with requests completing over a noticeably long period of time. To prevent the GUI from hanging or becoming unresponsive while the request returns, two approaches are used. The first approach consists of a sampler polling system similar to the one used for the overview down-sampler. The full resolution sampler is polled once every few milliseconds for returned data, and doesn’t cause the GUI to hang while waiting. The second approach involves displaying a low resolution image while the full resolution data is being retrieved. This takes advantage of the fact that a reduced quality version of the requested data already exists in memory, in the form of the down-sampled data cube. The data is retrieved from the overview cube using the selector’s location in overview coordinates, before they are translated into file coordinates. This process is blocking, and the GUI is not usable while it waits to complete. However, due to the overview cube’s dimensions typically being drastically reduced compared to the FITS cube’s dimensions, the time taken to complete this request is usually unnoticeable, and instantaneous from the user’s perspective.

4.5.2 Data Display Function Full resolution data extracted directly from FITS files must be displayed visually as well as providing access to precise data values and exact data cube coordinates. The ability to enhance specific areas for detailed analysis, as well as reducing focus to view a larger area, is required to make full use of the data. Implementation The Detail Display component uses OpenGL to display 2D data in a 3D environment. This is done using an orthogonal transformation, producing a parallel viewing projection. The parallel projection ensures that the detailed data being displayed is not distorted due to variances in depth.

The data is displayed by drawing a quad perpendicular to the view direction and applying texture coordinates to each vertex of the quad. The view point is positioned so as to always be directly above the centre of the quad. The maximum size of the quad is limited to the dimensions of the Detail Display window; though the minimum size is determined by the FITS file dimensions and the current magnification settings, and can be less than the dimensions of the window.

Figure 4.12: The relation between texture size and quad size with varying magnification.

The magnification system alters the scale of the displayed data, and is implemented using a combination of glScale and modified data extraction parameters. A magnification setting of 1 insures that the dimensions of the quad and full resolution texture are equal. This means that every square unit area of the quad is covered by exactly one pixel of full resolution data. A magnification setting of 2 represents a 2x zoom, and therefore halves the amount of data extracted from the FITS file. The quad is assigned the same dimensions as the extracted data, however a scaling transformation is created using glScale. The scale amount is equal to the magnification setting, in this case 2, and results in a quad whose dimensions are double that of the full resolution texture. This results in a single data value occupying double the visual space in the Detail Display, enlarging the selection. For magnification settings less than 1, the quad’s dimensions are smaller than that of the textures, accommodating more of the data in a smaller area. The data extraction parameters ensure that the size of the quad never exceeds the dimensions of the Detail Display. By moving the mouse over the Detail Display window, the user can determine the exact value at a point within the display, as well as where in the file that value is located. This is done by assigning a width and height offset to a mouse position within the display. The offset is (0,0) at the bottom left hand corner of the drawn quad, and is equal to the dimensions of the texture at the top right hand corner of the quad. These offset values are used to create an index into the data retrieved by the full resolution down sampler. The value stored at that location represents the value of the pixel which the cursor lies on. In order to find that pixel’s precise location within the FITS file, the offset value is added to the selector’s location in file coordinates.

5 Evaluation and Results Two aspects of the interface require evaluation, the first being the performance of the volume renderer under certain conditions. The second aspect is the interface approach as a whole, and how feasible it is as a framework for further development.

5.1 Volume Renderer The volume renderer is an integral part of the interface approach and the focus of the vast majority of interactions the user has with the application. Maintaining a certain level of performance and ensuring the implementation is successful is important in determining how feasible the technique is.

5.1.1 Performance Time between user input and visual response has been determined to be an important factor in the effectiveness of interactive data visualisations [1]. Because of this, the responsiveness of the 3D display when rendering data sets of varying size must be determined. The most important factor when determining this is how many frames can be displayed per second. The variable which most affects the number of displayed frames per second is the dimensions of the overview data cube being rendered. While increasing the data cube dimensions has the negative effect of increasing processing load, it has the positive effect of decreasing the down-sample ratio, which increases the level of detail present in the overview display. Determining the interaction between these two quantities; level of detail and frames per second, will assist in implementing an algorithm for determining the best output dimensions for a given set of input dimensions. The dimensions increase the processing required in three ways; the size of the geometry being drawn is larger, the number of quads or slices being drawn is more, and the number of fragments which must be processed by the fragment shader is increased. By increasing the total volume of the cube being displayed, the number of frames rendered over a period of time can be compared to overall volume of the cube. A frame counter was implemented which would determine the number of frames which had been completely rendered over a period of 60 seconds. Because the original draw loop was implemented with a timer system, the frame rate was limited to 60. The timer would also cause the draw function to be called whether the previous render had completed or not, skewing the results. This was replaced with an infinite loop system, where the draw loop would call itself as soon as it had completed. This system however was only used for testing, and was removed once the evaluation was complete. In order to determine the performance of the volume renderer only, the test was performed while no down-samplers were active. 50 frames per second was chosen as an acceptable minimum, as during practical use, the down-sampler would cause processing to become more erratic and unpredictable.

5.1.2 Results The outcome of the performance evaluation showed that the cube dimensions had far less an effect on frames per second than expected. The minimum limit of 50 frames was only met between volumes of 1024 and 1280 cubed. At these dimensions, holding the data cube in main memory becomes more of a concern, and is therefore not feasible in any case.

Figure 5.1: FPS vs data cube volume

The most likely reason for the unexpectedly low negative effect on performance is the level of detail system implemented in OpenGL. This system uses MIP maps to reduce the quality of textures which are a greater distance from the viewpoint. As the dimensions of the cube increase, the distance between the viewpoint and the furthest texture slice becomes greater, increasing the effect the level of detail system has.

5.2 User Interface As an application intended to be used by experts in analysing scientific data, evaluating usability was not the main objective. It was expected that a certain level of experience with and knowledge of similar applications would be required to meaningfully evaluate the interface. For this reason, it was chosen to evaluate using a smaller, more expert group, using an expert review approach. Three astronomers with experience in analysing data cubes were each asked to perform an hour long evaluation of the interface, consisting of a demonstration section, followed by a question and answer section.

5.2.1 Analysis Approach The application represents a prototype for the first iteration of an iterative development approach. Because of the experimental nature of the interface and its approach to analysis, the primary evaluation criterion is how feasible the interface is, and whether or not it is worth pursuing further. To effectively evaluate this, it was important that the expert users be adequately familiar with the interface, as well as the various techniques of analysis it provides. A demonstration lasting 20 to 30 minutes was conducted, during which the expert were given full control of the interface, while the evaluator would request that certain tasks be performed. The demonstration consisted of 3 stages involving analysis of 3 different FITS files, each with different characteristics which would influence the use of the interface.

Figure 5.2: The 3D display presented in stage 1 (left) and the image in its original form (right).

The first stage involved a primer file which did not contain astronomical data, but an artistic image. The image was chosen because it has distinct and recognisable features which can be used as reference points. As this was the expert’s first experience with the application, it was important that they understood how certain actions affected the display relative to certain image “landmarks”. The image consisted of a suitable range of intensity values to demonstrate the threshold and intensity binning functionality, and had sufficient dimensions that the application to displaying data cubes could be seen.

Figure 5.3: Once the user has limited the selection to contain only the crow at the centre of the image, the Refine Area Tool

is used to display a full resolution representation.

Stage 1 is conducted using the following procedure 1. At the start of the demonstration, the first file is loaded. 2. The various down-sampler iterations are explained and are visible to the user. 3. Once the file is loaded, the controls for the camera system are explained, and the user is

asked to rotate 360 degrees around the data cube. The user is then asked to focus on the XY plane using the HUD focus button.

4. The user is asked to display the front slice of the data cube in full resolution using the Detail Display, and to reduce magnification until the entire image is visible. This allows the user to compare the original data to the down-sampled representation shown in the Overview Display.

5. The intensity binning and threshold sliders are adjusted, and the effect on the Overview Display is compared to the data in the Detail Display.

6. The user is asked to adjust the clipping planes so that only a specific object in the image is visible.

7. The Refine Area Tool is then used to display the selection in full resolution. 8. This concludes the first stage of the evaluation.

Figure 5.4: The data cube used in stage 2 has opaque surface layers and noisy data when first viewed (left). By adjusting the

threshold and intensity binning values, the internal structures of a galaxy are made visible (right).

Stage 2 uses an observed astronomical data cube containing distinct hierarchical structures and dense intensity values. The file was chosen because it requires manipulation of threshold and intensity bins in order to identify the structures present in the data. The high density of data also requires that the clipping planes be adjusted to view obscured data.

Figure 5.5: Adjusting the position of the clipping planes exposes internal layers of the data cube.

Stage 2 is conducted using the following procedure

1. When the file is first loaded, most of the display is obscured by dense external noise data. Only the very outer surface is visible, which is nearly completely opaque.

2. The user is asked to adjust the clipping planes in order to view internal slices of the data cube, at which point new data becomes visible.

3. The user then adjusts the threshold settings until the noise data is no longer visible. At this point, the overall structure of a galaxy becomes visible.

4. The intensity bins are adjusted to reveal internal structures. The user can identify relatively low external intensities at the surface of the structure, with concentrated high intensity values at the centre in a spiral pattern.

5. The user is asked to increase the level of detail of the Overview Display by adjusting the clipping planes so that only the galaxy is visible, and then use the Refine Area Tool. The size of the down-sampled cube is kept relatively constant, but the volume of data being down-sampled is reduced, increasing the level of detail.

6. The Overview Display is then to be reduced to 10 slices from the centre of the galaxy using the Refine Area Tool. At this point, two separate branches of dense intensity values become visible. The user is asked to display one branch at full resolution. Various strata of intensity values become visible at this time.

Figure 5.6: Displaying a small section of the galaxy in full resolution.

In stage 3, the user is presented with a very large FITS data cube containing highly sparse collections of intensity values. Due to the relatively small size of individual objects in the FITS data cube, and the large spaces between them, minimising the down-sampling ratio is required to view the data meaningfully. This requires the user to iteratively select sub regions of the file which have a relatively high density of objects, and refine the area viewed to just that selection.

Figure 5.7: The entire FITS data cube used in stage 3 is displayed (left). An area roughly one quarter the size of the entire file

is selected and displayed, making objects larger and more distinct (right).

Stage 3 is conducted using the following procedure 1. When the file is first loaded, the display is almost entirely empty with only a few faint

objects visible. Decreasing the threshold brings several cylindrical objects into view, distributed throughout the volume.

2. The user is asked to reduce the size of the selected area to roughly a quarter of the original, and to choose an area with a relatively dense distribution of intensity values.

3. Once the area has been reduced, objects become more visible as the level of detail has increased. The user is asked to choose the single visible object which potentially has the highest intensity value present in the selection.

4. Once the user has chosen an object, they are asked to reduce the selected area so that it includes only that object. The Refine Area Tool is then used to display the object, with the selection now being small enough to be displayed in full resolution.

5. The user is then asked to use the Detail Display to find the value and position in the FITS file of the greatest intensity value present in the object.

Figure 5.8: A single object is shown at full resolution, clearly displaying the distribution of greatest intensity values.

Once all three stages had been completed, the user was asked to answer a few questions relating to the use of the interface. The questions focus on the feasibility of the interface, and how affective the approach is when analysing data cubes. This includes determining whether the interface is a potential improvement over existing applications, or whether a similar system is already offered. As experts and “clients”, it was also important to determine what issues they had with the approach, as well as the additions they would like to see. These suggestions would help to direct any future works in the correct direction. The full set of questions can be seen in appendix B.

5.2.2 Results The expert reviews were positive and reflected well on the feasibility of the approach. It was found that the users preferred a 3D display of FITS data cubes to the 2D display typically used in applications such as DS9. The users stated that the approach was preferable to applications such as DS9 in cases where 3D structures existed, but could become needlessly complicated for data without a third dimension. All users said that they were comfortable with the 3D interactions requires to use the interface, or that they could become comfortable after a short amount of use.

On the matter of similar existing systems, two examples were cited. The first was the now largely defunct Virtual Observatory. The second was a medical imaging application called Osirix. Because Osirix was not intended for astronomical data, its applications to astronomy are limited. It was also said that Osirix rendered static 3D views and was not as interactive as the demonstrated system. Users stated that the approach shifted focus from detail information to overview information, and that the interface could be feasible as a standard approach to viewing data cubes. The primary additions that were requested involved scientific analysis tools, which were not included in the scope of the project.

6 Conclusion A graphical user interface for displaying data cubes with a focus on radio astronomy data was designed and implemented. The approach used involved the use of iterative 3D displays. As it was not within the scope of the project, no scientific analysis tools were implemented. Expert reviews were conducted to verify the feasibility and applicability of the approach to data cube analysis. It was determined that the interface could serve as an improvement on current systems using a 2D approach, such as DS9. However, the approach was determined to be inferior when analysing 2D data. The main constraint of the volume renderer used was found to be the dimensions of the cube being displayed. This parameter had a negligible effect on frames per second due to a level of detail system included with OpenGL. However, the raw memory requirements for storing data cubes become exceedingly large as dimensions increase. An approach for automatically determining the size of an Overview Display data cube should include parameters for acceptable memory usage, rather than system performance capabilities. As a prototype, the application has proven to implement a feasible approach to visualising FITS data cubes containing astronomical data. With further additions and refinement, the application could prove a valuable tool to astronomers wishing to analyse data cubes of a very large size. One expert reviewer went so far as to send a personal recommendation for the application to members associated with the SKA (Square Kilometre Array), attached in appendix C.

6.1 Future Work As a prototype, the application would require many additions to become a fully usable piece of software. Among the most important, as reflected in the expert review, would be the implementation of various scientific analysis tools. Two such tools which were suggested by the expert reviewers were a spectrum graph, which would display the flux at a certain point in the data cube, and a more advanced threshold system, which would possible include logarithmic and square scaling, rather than just linear scaling as is implemented currently. Implementations which would improve performance and memory efficiency would also be required for the application to reach its full potential. The creation of a virtual memory system specifically for the storage of overview data cubes would reduce the memory impact of having multiple contexts active simultaneously. In such a system, instead of keeping a data cube in memory, it could be saved to the hard disk, and then discarded. This could reduce the memory requirements by only being required to accommodate one overview cube at a time. A caching system based on spatial locales would greatly increase the responsiveness of the Detail Display. By loading in data around a selected point, the user would be granted a degree of freedom to move around the file in a small area, without having to request full resolution data from the hard disk. A more comprehensive camera system would be required to improve the usability of the interface, and increase the cube coverage. Additional refinements to the data cube renderer itself could also result in the interface becoming a feasible 2D data viewer as well.

References [1] Johannessen, R. A., Interactive Visual Analysis and Exploration of Astrophysical Data, INF358 Seminar in Visualization (2008) [2] Borkin, M., Goodman, A., Alan, D., Kauffmann, J., & Halle, M. (2007). Application of Medical Imaging to the 3D Visualization of Astronomy Data. Proceedings of IEEE Visualization Conference. Sacramento, CA, USA *3+ M. Tory, “Mental Registration of 2D and 3D Visualizations (an Empirical Study),” Proc. IEEE Visualization Conf., pp. 371-378, 2003. [4] Kaufmann, J., ADASS Tutorial: 3D Visualization in Astronomy, London, 2007 September 23 [5] Stolte, C., D. Tang, and P. Hanrahan. Polaris: A System for Query, Analysis and Visualization of Multi-dimensional Relational Databases. IEEE Transactions on Visualization and Computer Graphics, 8(1), January 2002. [6] http://am.iic.harvard.edu/ *7+ Lorensen, William E. and Cline, Harvey E., ‘‘Marching Cubes: A High Resolution 3D Surface Construction Algorithm,’’ Computer Graphics, Vol. 21, No. 4, July, 1987, pp. 163-169. [8] Levoy, M. [1988]. Display of surfaces from volume data, IEEE Computer Graphics & Applications 8(3): 29–37. [9] http://www.sas.el.utwente.nl/open/research/medical_imaging [10] Borkin, M.A. Arce, H.G. Goodman, A.A., A COMPLETE Survey of Outflows in Perseus, BULLETIN- AMERICAN ASTRONOMICAL SOCIETY, 2007, VOL 39; NUMB 4, pages 154.01 [11] S. A. Colgate and R. H. White 1966, Astroph. J., 143, 626 *12+ Blondin, J.M., “Discovering new dynamics of core-collapse supernova shock waves”, 2005, Journal of Physics: Conference Series 16 370 [13] CARD, S. K., MACKINLAY, J. D., & SHNEIDERMAN, B. (1999). Readings in information visualization: using vision to think. San Francisco, Calif. [u.a.], Morgan Kaufmann

Appendices

Appendix A Simple Fragment Shader

#version 120

uniform sampler3D text;

uniform float magic;

uniform float thresholdOne;

uniform float thresholdTwo;

uniform float intensityOne;

uniform float intensityTwo;

void main()

{

float alpha= texture3D(text,vec3(gl_TexCoord[0].x,gl_TexCoord[0].y,gl_TexCoord[0].z)).x;

vec3 color;

if(alpha<intensityOne)

color=vec3(0.0,0.0,1.0);

else if(alpha<intensityTwo)

color=vec3(0.0,1.0,0.0);

else

color=vec3(1.0,0.0,0.0);

if(alpha<thresholdOne)

alpha=0.0;

else if(alpha>thresholdTwo)

alpha=1.0;

else

alpha=(0.1+((alpha-thresholdOne)/(thresholdTwo-thresholdOne)))/5.0;

gl_FragColor = vec4(color, alpha);

}

Appendix B Expert Review Questionnaire

1. Were you comfortable with using the 3D display, or do you think that you could become accustomed to it after more use?

2. Do you find the 3D representation of astronomical data cubes to be an intuitive visualisation?

3. What is the most challenging part of navigating large 3 dimensional files, and does this interface address those issues?

4. Which aspect of the interface do you find most useful? 5. Is this approach preferable to a 2D approach such as DS9? 6. Do you think this interface is feasible as a standard approach to viewing data cubes? 7. Is there another application you have used which offers the same services as this one? 8. Were there any problems with the interface that stood out, or any specific additions you

would like to see? 9. Would you use this application to view data cubes?

Appendix C Personal Recommendation

From: Sarah Blyth <sarblyth@ast.uct.ac.za> Date: 26 October 2011 12:11:22 PM SAST To: Jasper Horrell <jasper@ska.ac.za>, Simon Ratcliffe <sratcliffe@ska.ac.za>, Simon Ratcliffe <sratcliffe@gmail.com> Cc: Michelle Kuttel <mkuttel@cs.uct.ac.za>, Erwin de Blok <edeblok@ast.uct.ac.za>, Sarah Blyth <sarblyth@ast.uct.ac.za> Subject: Exciting UCT visualisation project Hi Jasper and Simon Today Erwin and I were asked to participate in user testing for a data cube visualisation tool that three Computer Science Honours students at UCT have been working on for the past few months. I must say that both Erwin and I were very impressed by the tool itself and by what they have managed to achieve in a very short time. They have produced something that has the potential to be very useful to astronomers (especially radio astronomers) right now and also in the era of much larger datasets as it can handle very large data cubes as well. I thought you both might be interested in their project and perhaps coming to see their Honours presentation on 17 or 18 November where they will present it. I will confirm with their supervisor (Michelle Kuttel, cc'd here) the final date closer to the time. Cheers! Sarah Dr Sarah Blyth ____________________________ Astrophysics, Cosmology and Gravity Centre (ACGC) Astronomy Department University of Cape Town Rondebosch South Africa tel: +27 21 650 5372