1

2

This report was produced by the Scientific Responsibility, Human Rights and Law Program (SRHRL) of the American Association for the Advancement of Science (AAAS). As a program of AAAS – the world's largest multidisciplinary scientific membership organization – SRHRL is committed to fostering and facilitating the responsible practice and application of science in the service of society. The Program promotes high standards for the practice of science and engineering; advances the human right to enjoy the benefits of scientific progress and its applications; engages scientists, engineers and their professional associations in human rights efforts; monitors and enhances assessment of emerging ethical, legal, and human rights issues related to science and technology; furthers the use of science and technology in support of human rights; and initiates activities to address the impact of developments at the intersection of science, technology, and law. This report was authored by: Jonathan Drake, Senior Program Associate, AAAS SRHRL Acknowledgement Primary support for this project was provided by the Oak Foundation through grant number ORIO-15-052. Disclaimer The opinions, findings, and conclusions or recommendations expressed in this publication are those of the author(s) and do not necessarily reflect the views of the AAAS Board of Directors, its Council, or membership, or the Oak Foundation. Contact AAAS welcomes comments and questions regarding its work. Please send information, suggestions, and any comments to SRHRL at srhrl@aaas.org. © Copyright 2018 American Association for the Advancement of Science Scientific Responsibility, Human Rights, and Law Program 1200 New York Avenue, NW Washington, DC 20005 USA

1

Introduction

Geospatial information is a useful tool for field researchers, and has been employed extensively in human rights research over the past decade. Along with traditional data products such as maps, photographs, and field sketches, remotely-sensed data in the form of satellite imagery has increasingly been a key component of this research, due to its ability to provide information about remote, dangerous, or otherwise inaccessible locations (AAAS 2013; Marx & Goward 2013; Wang et al. 2013). With a maximum ground sample distance of 30 centimeters per pixel, however, even the highest-resolution commercial satellite imagery is of limited utility when the phenomena of interest are on the scale of individual clandestine burials. In places like Colombia and Guatemala, the difficulties associated with this data product are further compounded by local factors such as frequent cloud cover, and/or the presence of continuous forest canopy. In situations like these, however, many of the difficulties associated with satellite imagery can be overcome by conducting the necessary mapping on-site using a process known as photogrammetry. Photogrammetry involves the construction of a three-dimensional model of a study area by taking multiple still images from different perspectives, identifying common features between the images, and using these features to generate three-dimensional point clouds via the principle of parallax. This process is facilitated by a suite of software tools such as Pix4DMapper and AgiSoft Photoscan, which can automate the process of identifying shared features. Photogrammetry can be performed using ground-based handheld photography, however the technique is significantly enhanced by the availability of airborne camera platforms such as unmanned aerial systems (UAS), which significantly increase the number of potential viewpoints from which data can be collected. The multiplicity of platforms that can be used to perform photogrammetry renders it a very resilient technique, allowing it to be adapted to multiple environments and terrain types. By flying beneath the clouds, for example, UAS can obtain optical images in areas that satellites are unable to observe. Similarly, ground-based photogrammetry can take place in terrain like dense jungle where even UAS cannot safely fly. The use of photogrammetry to enhance field research has already proven useful in a number of fields, including geology, civil engineering, and city planning (Shugar et al. 2017; Yeum et al. 2017; Zawieska et al. 2016). Additional applications of these tools have also emerged in the fields of archaeology, where they have been used to create virtual reconstructions of damaged cultural heritage sites, and law enforcement, where they have supplemented the traditional role of the crime-scene photographer in preserving evidence (McCarthy 2014; Al-Ruzouq et al. 2012; Agosto at al. 2008). While these latter applications are in many ways similar to the documentation of clandestine grave sites in a human rights context, multi-image photogrammetry has yet to be applied to this type of investigation. In the context of research on clandestine graves, photogrammetry has the potential to benefit investigations in three primary ways: by providing field researchers with enhanced situational awareness; by facilitating the identification of potential burial sites; and by enhancing the documentation of gravesites once they have been discovered. Traditionally, satellite imagery has been used in the first and second roles, however as mentioned above, its use is limited by weather, tree cover, and resolution, and therefore its usefulness in investigations is often limited to the very largest of gravesites, or to providing historical or contemporary context about the surrounding terrain. The outputs of photogrammetry may be able to transcend these limitations. For example, when the input images contain geotags in their EXIF data, which many GPS-enabled cameras generate automatically, the processing software can use this information to produce fully georeferenced orthomosaics and digital surface models (DSMs) with significantly higher resolution than comparable satellite data.

2

For the purposes of enhancing situational awareness, such an orthomosaic would be extremely useful for researchers that have recently arrived at the site of a suspected grave, allowing the team to quickly establish an operational plan for searching the site for potential burials, or to resume a search from a previous expedition. Although the cameras used for photogrammetry cannot see below the surface of the ground, themselves, in some cases the DSM may enable the identification of possible graves at a site by revealing small-scale topographic features such as mounds or depressions that may be related to the digging of graves. Depending on their size and relief, such features may be easy to miss with the naked eye, and could indicate potential sites for investigation using sub-surface measurement techniques such as electrical resistivity tomography. This capability is particularly relevant when the mobility of the field team is limited by inhospitable terrain or vegetation. Traditional methods of gravesite documentation in a human rights context have involved taking photographs or illustrating the site using pencil sketches. In the photographic case, after the grave has been opened and the remains cleaned to the greatest extent possible, the skeletons are numbered using markers placed next to them in the grave, a north-facing orientation marker is placed into the grave along with ruled measurement scales at multiple locations, and photographs of the grave are taken from a select number of vantage points (Figure 1a). These overview photographs are then often accompanied by zoomed-in vignettes of key areas of specific interest. This method has the benefit of creating an objective documentation of the site, however following removal of the remains, all subsequent examinations will necessarily be constrained by the perspectives chosen by the photographer on-site at the time. Furthermore, due to geometric and optical effects, the scale markers placed in the grave will only enable the most approximate measurements to be performed on the remains: foreshortening, perspective and lens distortion will combine with the three-dimensional geometry of most scenes in ways that limit the effectiveness of making measurements on a two-dimensional image. By constraining the field illustrator to a uniform scale, pencil sketches made on graph paper have the potential to resolve the geometric and optical effects inherent in photographic documentation (Figure 1b). Producing such a sketch, however, is a time-consuming process, and in the field, time is often a limited resource. The quality of such drawings can also vary significantly depending on the artistic talent of the illustrator, who may also have other primary responsibilities. Unlike photographs, the inherently subjective nature and degree of artistic interpretation associated with a pencil illustration may expose researchers to charges of bias or error if such a document is submitted as evidence in a court of law. Finally, both photographic and pencil documentation are limited by the fact that they are necessarily two-dimensional projections of a scene that is fundamentally three-dimensional in nature. The three-dimensional point cloud, digital surface model, and orthomosaic produced using photogrammetry do not suffer from these drawbacks. Like photographic documentation, they can be produced rapidly, however unlike photographs, the resulting model is fully georeferenced and can be used for measurements. Like pencil sketches, the orthomosaics is an accurate projection of the site on a two-dimensional plane, however unlike an illustration, the orthomosaic involves little to no artistic interpretation, and takes significantly less time to produce. For these reasons, we hypothesized that the application of photogrammetry to the study of clandestine burials in a human rights context was an idea worth investigating further.

3

A

B

Figure 1. Two traditional methods of gravesite documentation. (A): photographic documentation of a grave near Coban, Guatemala. Although of identical dimensions, the size on the image of rulers close to the camera will differ from those further away, due to perspective. (B): The beginnings of an illustration on graph paper of a mass grave elsewhere in Guatemala. This method produces geometrically accurate results, but is time-consuming and subject to artistic interpretation. Top image courtesy of the Guatemalan Forensic Anthropology Foundation (FAFG), bottom image by the author.

4

Data and Methods This investigation attempted to apply both air and ground-based photogrammetry to multiple sites in Latin America that were suspected of containing clandestine graves. The first goal of the investigation was to evaluate the technique’s usefulness as a tool for enhancing the situational awareness of field researchers conducting forensic investigations in rugged, unfamiliar terrain. The second goal, if any remains were found, was to evaluate the 3-D mapping capabilities of photogrammetry for the purposes of documenting the contextual relationships of the remains prior to their being exhumed. The first three sites were located in the vicinity of Chámeza, Colombia and were investigated in partnership with Equitas, an NGO based in Bogotá. Two of these sites, Teguita Alta #1, and Teguita Alta #2, consisted of open fields, which allowed the mapping to be conducted using UAS. At these sites, a stock DJI Phantom 4 was used for this purpose (Figure 1d). The UAS was controlled using a tablet computer attached to its remote control unit, which allowed for automated flight with the possibility of manual intervention at any time. Two different applications were used to upload flight plans to the aircraft, Pix4DCapture and DroneDeploy. Both of these rely on internet access in order to retrieve online maps to facilitate the process of defining the survey area. As no connectivity was available at the sites in question, the area was defined using a laser rangefinder and, where possible, by walking the perimeter of the site while holding the tablet, thereby establishing its boundaries using the tablet’s built-in GPS. The survey flights were conducted under computer control, without direct pilot input except in case of emergency. In order to ensure that the UAS would not be in danger of encountering any obstacles during these autonomous flights, a manual test flight was conducted prior to each survey flight, to establish the minimum altitude at which the aircraft could safely fly without encountering trees, terrain, or other obstructions. The final parameter of the flight involved defining the exact path followed by the UAS. The flight over the first site, conducted using DroneDeploy, was defined as a single-grid mission followed by a single circular orbit of the site. For the second site, controlled by Pix4DCapture, a double-grid mission was flown, in which the flight path consisted of overlapping grid paths, rotated at ninety degrees to one-another. At both sites, the amount of visual overlap between subsequent images was defined as eighty percent. Wherever possible, the area to be mapped was cleared of personnel prior to each flight to minimize the potential for error caused by motion in the scene. When this was not possible, everyone who was present in the area was requested to remain still. The third site under investigation, San José, was located under dense forest canopy, on the side of a hill sloping approximately north-south where dense vegetation precluded the automated flight of the UAS. Manual flight, while possible, was deemed too risky due to the potential for crashing the UAS. For this reason, the mapping at this site was performed on the ground using a Canon 6D Digital Single-Lens Reflex Camera with a Sigma 8mm circular fisheye lens; the Camera’s built-in GPS provided geolocation data. This lens provided a 180-degree field of view, which allowed for the maximum possible amount of overlap between successive images. The mapping proceeded on foot in an overlapping grid pattern, with the first pass covering the site through a series of traverses in an east-west direction at right angles to the slope, followed by a second series that covered the site via traverses in a north-south direction. Photographs were taken at approximately one-meter intervals, though the pattern often diverged from a perfect grid due to the practical considerations associated with navigating the uneven terrain on foot. The locations of the images acquired at this site are shown in Figure 1a.

5

A B

C D

Figure 1. A: Locations of images acquired on foot at San José; the pattern of overlapping grids is crude, but visible. B: Photograph showing the alignment markers used to facilitate 3-D reconstruction, as well as the challenges of overhanging vegetation. C: In Guatemala, the availability of a TotalStation to provide ground control points significantly improved the reliability of the model. D: The DJI Phantom UAS used for conducting aerial mapping in both Colombia and Guatemala. In Guatemala, a second Phantom belonging to FAFG, and a DJI Mavic Air were also used.

6

Due to the high level of visual complexity associated with the jungle terrain, and the similarity of many features with one-another (e.g., leaves, tree branches, etc.), it was considered highly likely that automated matching of the images with one-another would fail. In order to correct for this, large color-coded alphanumeric markers were placed on the sides of trees and logs at a variety of elevations throughout the site to provide unique, easily identifiable 3-D reference points during processing, as shown in Figure 1b. Although these markers were placed at high density throughout most of the study area, none were placed in the northeast quadrant of the study area, in order to evaluate whether it was possible to use naturally-occurring features as manual tie-points in this type of terrain. Weather was a challenge at both sites. During one of the mapping flights at Teguita Alta #2, a low bank of clouds rolled in over the mountain, requiring that the mapping mission be aborted and that the UAS be re-launched to re-acquire the data once conditions had improved. At San José, frequent rain showers meant that the team had to take shelter under tarps for significant portions of the day, and in one instance made it necessary to collect mapping data beneath an umbrella. Once collected, the data were downloaded and processed with Pix4D Desktop. For UAS flights, the default processing parameters for 3-D mapping produced good results from automatic image alignment, with a maximum of only 2.5 percent of images uncalibrated – primarily in areas outside the study area where the images included significant areas of trees. As expected, for the ground-based mapping, automatic image alignment was far less successful, with the vast majority of images failing to align with any other through automatic matching. At the same time, large numbers of images that the software identified as correctly calibrated had in fact keyed on erroneous tie-points leading to a highly incorrect geometry. As a result, with the exception of a small group of images that served as a baseline, all automatic calibration information was deleted from the ground-based photographs, and the images were manually calibrated with one-another using the alphanumeric markers described above. Following manual calibration, 221 out of 236 images were correctly calibrated. No TotalStation or survey-grade GPS was available at the three sites, so ground control points were not available to improve the geolocation data generated by the cameras’ onboard GPS. Instead, Pix4D’s “scale and orient” tool was used to adjust the size and rotation of the model by referencing stakes and measurement tapes that had been placed in the ground at the field site for that purpose. The fifteen images that failed to calibrate even after manual matching were located almost exclusively in the northeast quadrant where artificial references had been deliberately omitted. The calibrated images were then processed into a 3-D point cloud using Pix4D’s standard matching algorithm. The resulting point cloud contained a large number of points that corresponded to overhanging vegetation, and which interfered with interpretation of the scene. In order to simplify the resulting model, this point cloud was then manually edited to remove features that were not of interest to the investigation, including branches, leaves, and other elements that had the potential to obscure the surface of the terrain in the final mosaic. The lower ends of tree trunks, however, were preserved, as were logs already on the ground, because both of these are part of the landscape and can serve as useful reference points should the model be used to guide subsequent investigations. The edited point cloud was then processed into a 3-D model using a custom set of processing parameters that maximized the number of triangles in the completed mesh. Building on the experience gained collecting and processing data from the Colombian sites, AAAS subsequently partnered with the Guatemalan Forensic Anthropology Foundation (FAFG) to apply the same methods to sites in Guatemala. The first of these was located in a pasture in the northern highlands of the country known as Site Z (the exact name and location remains confidential). Unlike in Colombia, at this site a TotalStation was available, which enabled ground control points (GCPs) to be measured and used to constrain the model (Figure 1c). As no pre-existing survey points were available to be used as a datum, the TotalStation’s coordinate system was established using handheld Garmin GPS receivers and the “backsight via coordinate” method. To do this, the location of the TotalStation was first established by leveling the instrument and placing the GPS antenna directly beneath its downward-facing optical sight. The GPS unit was then powered on, and waypoint averaging was used to mark the instrument’s coordinates over approximately two thousand separate observations. The TotalStation’s position thus established, its orientation was defined by taking the

7

GPS receiver as far from the station as possible while remaining within its line of sight, and marking a waypoint via the same averaging method. Locating the point as far as possible from the TotalStation ensured that the linear error associated with the GPS measurement translated into the smallest possible angular error when the point was used to calibrate the instrument’s encoders. This point was then sighted in the TotalStation, and its location used to establish the TotalStation’s coordinate system. Because the GPS units used in this calibration were consumer handheld units, error for each the two points was estimated at one to two meters. As such, while the accuracy of the GCPs measured by the TotalStation was very high relative to one-another (on the order of one centimeter or less), the entire model likely was shifted from the true datum by up to four meters. While not ideal, this was the best available solution given the equipment on hand, and for the purposes of evaluating the contents of the grave, absolute accuracy with respect to global coordinate systems is less necessary than a high relative accuracy and internal consistency within the model. Following the calibration of the TotalStation, GCPs were acquired using the same alphanumeric markers used in Colombia. Because this site was an open field, in this case the markers were staked to the ground, and the GCPs measured at the upper-left corner of each marker. Over two dozen GCPs were measured, distributed roughly evenly throughout the site, with particular density in the area immediately surrounding the pit of the suspected grave. Particular care was taken with the positioning of these latter GCP markers, to ensure that they were positioned in such a way that they would be visible on multiple days of the excavation, and not be disturbed by subsequent digging, or covered with spoil from the excavation. On the first day of the excavation at site Z, one skeleton was unearthed, dressed in the remains of clothing, as shown in Figure 2. Following its discovery, scale rulers were placed in the grave and the entire site was mapped using the same DJI Phantom UAS that was used in Colombia. As before, the flight path was a double-grid mission programmed using Pix4DCapture, with the boundaries defined by walking the perimeter of the site prior to takeoff. Supplementing these airborne photos, ground-based photography was also conducted using the same Canon 6D digital SLR, although in this case a 20-35mm wide-angle zoom lens was used, rather than the 8mm fisheye. These images were acquired by walking around the lip of the exposed grave, with the lens focused on the skeleton at all times, taking a photograph at every step (approximately one meter in stride). Following documentation with the UAS and camera, FAFG’s procedures for exhumation were followed, in which the bones were carefully removed and catalogued for transport to their laboratory. The next day, a second skeleton was unearthed, this one wrapped in a pink sheet, as shown in Figure 2b. The same mapping procedures were followed for documentation before it, too, was removed. The third day, an additional skeleton was found, documented, and removed in the same manner. At no time at this site was more than one skeleton visible to the field team. The second site under investigation in Guatemala, Site X, was located in the northern lowlands of the country, on a densely-forested hillside somewhat similar to the terrain at San José, in Colombia. At the time the survey team arrived, a grave had been partially unearthed that appeared to contain the skeletal remains of at least ten individuals, interlaced with and superposed atop one-another in a rectangular trench approximately five meters long by one meter wide. Most skeletons were clothed, and many of the clothes showed burn marks around their edges, although no such charring was apparent on the bones themselves. For data collection, the lessons learned both in Colombia and at Site Z were applied. The TotalStation was set up and calibrated in the same manner, and ground control points were established both around and within the grave, as well as throughout the surrounding terrain. On a previous expedition to this region, during which no remains had been located, manual flight of the DJI Phantom under jungle canopy had been attempted and found to be feasible, if awkward. In light of this successful trial, an attempt was made to map the site using the Phantom from low altitude via the “Free Flight” mode of Pix4DCapture. While this flight took place without incident, it was determined that the one-meter minimum interval between photographs in that application was too large for the purposes of low-altitude gravesite photogrammetry under canopy. Furthermore, it was observed that the strength of the Phantom’s downdraft was sufficient to significantly disturb many elements present in the grave, such as rulers and identification markers which flew in different directions, along with large quantities of airborne dust kicked up by the rotors.

8

A B

C

Following this problematic trial, the team decided to evaluate whether the DJI Mavic Air, a small foldable UAS with a camera comparable to the Phantom, used by expedition members for aerial videography, might be better suited for low-altitude photogrammetry under dense jungle vegetation. Flying under full manual control, with photos being taken by manual presses of the shutter button, this smaller aircraft was found to be significantly better suited for the types of conditions present at Site X; despite flying very near the grave, little dust was kicked up, and all key elements of the grave remained stationary. Furthermore, the small size of the Mavic Air enabled far more precise maneuvering in the tight spaces around the grave mandated by the jungle environment. Unlike at Site Z, the skeletons at Site X were located one atop the other. This geometry of evidence required a change in mapping strategy. While at Site Z, mapping flights took place following the exposure of each new skeleton, here the priority after initial mapping was to remove all skeletons which significantly overlay others. The removal of this top layer of “primary” skeletons was chosen in a way that maximized the exposure of the underlying, or “secondary”, surface. The interlaced nature of the

Figure 2. Three skeletons unearthed at site Z, in the Guatemalan highlands, as they appeared when they were documented using air and ground-based photogrammetry. A: Skeleton exhumed on the first day of fieldwork. B: Skeleton exhumed on the second day at the site. C: Skeleton exhumed on the third day in the field. Only one skeleton was visible to the field team at any given time. Case details on the chalkboards have been blurred for legal reasons. The north-arrow in each grave is approximate, and was not used to calibrate the model.

9

skeletons, where (for example) the torso of one skeleton may form part of the top layer of bones, while its legs may lie several layers below, made this process difficult, however through careful deliberation over which sets of remains were exhumed first, it was possible to perform a series of data collection flights that ensured that as much surface as possible was exposed for each skeleton when the data were collected by the UAS. Data processing for the two Guatemala sites was performed using Pix4D and the default parameters for either 3-D Maps (for automated flights) or 3-D Models (for manual flights). In almost all cases, the image alignment and matching process was highly successful. Unlike in Colombia, even those images acquired under dense jungle canopy were calibrated into the model without major problems. This is likely due to several factors, including the presence of true three-dimensional ground control points in the Guatemalan example, a greater density of distinct artificial markers for the computer to identify around the grave, and more consistent photographic overlap between images, due to the UAS’s ability to follow a straight path, compared to ground-based methods which had to negotiate difficult terrain on foot. For both Site Z and Site X, 3-D models were created for each individual data collection run. The models of these collection runs were then combined with all other models associated with that site using the GCPs that were shared across each data collection flight. This process resulted in two merged “master” models, one for Site Z, and one for Site X. Within these models, which were based on images that had been acquired on multiple survey flights, the images associated with each flight were assigned to groups based on their flight, for ease of identification. The point cloud associated with each image group (i.e., each data collection flight) was then edited to remove artifacts that hindered the interpretability of the final model, such as soil present during earlier flights, that covered the remains of skeletons that were later exhumed.

Results In both Colombia and Guatemala, the aerial data collected using UAS produced high-quality mosaics and digital elevation models with a minimum of effort. Ground sample distance in the orthomosaics produced by autonomous flight was approximately two centimeters per pixel, over ten times better than the best available satellite imagery. After being generated by Pix4D, the 3-D analyst toolbox in ESRI ArcMap was used to convert the DSMs from the Colombia sites into slope maps, which were able to reveal several small depressions and areas of level ground that were not immediately obvious or accessible to the team in the field; these could be promising future targets for investigation using subsurface tomography. Examples of airborne-derived DSMs are shown in Figure 3.

A B

Figure 3. Digital Surface Models derived from UAS imagery at Teguita Alta sites #1 (A) and #2 (B).

10

Following point cloud editing, the 3-D model generated from the ground-based images acquired at San José successfully reproduced the essential features of the landscape, including fallen logs, tomographic equipment, and test pits dug during the course of the fieldwork. At the highest magnification, many areas of this model’s texture map were blurry, or contain multiple overlapping “ghosts” of the same feature. This is likely due to the large number of images that overlap a given point in this model, combined with small errors in the manual image calibration. Nonetheless, features as small as the heads of tomography electrodes are easily distinguishable as such in the model. Based on the dimensions of these features, the model’s texture map has an estimated spatial resolution of approximately 1cm per pixel. Despite the qualitative fidelity of the 3-D model, however, the orthomosaic generated from the ground-based images contained numerous image artifacts that could not be eliminated using the editing tools available in Pix4D. In order to produce a useful mosaic, the 3-D model, which was not subject to these errors, was viewed in orthographic projection at high magnification. By stitching together multiple vignettes and georeferencing the resulting mosaic according to the model’s geolocation data, a more faithful orthomosaic was produced. This output depicts the terrain located beneath the jungle canopy as it appeared during the collection of subsurface tomography data, and can inform the placement of subsequent tomography soundings. An overview of this mosaic is shown in Figure 4a. The lack of data in the northeast quadrant is due to failed image calibration resulting from a lack of alignment markers in that region, as described above. The artifacts associated with the orthomosaic did not extend to the Digital Surface Model that resulted from the ground-based photography. This output, like the 3-D model, reproduced the contours of the terrain and associated features, as shown in Figure 4b. Measurements conducted on objects of known length within this model show that the geometric corrections applied by the “scale and orient” tools did result in a model that faithfully reproduces their dimensions at a local scale. Due to the lack of ground control points in this case, however, whether that geometric consistency applies over greater distances could not be validated. Similarly, the elevations of the DSM, being based solely off of the camera’s built-in GPS, may vary significantly from the terrain in an absolute sense.

A B

Figure 4. Two views of San José. (A) Orthomosaic with overhanging vegetation digitally removed and (B) Digital Surface Model with elevation contours. Both results were derived from ground-based imagery.

11

In contrast to the mixed results achieved through ground-based photogrammetry at San José, the results from applying the technique to the exhumation of known gravesites in Guatemala were excellent. Because these models incorporated ground control points, their geometric accuracy could be established with confidence in a way that was not possible in Colombia; at Site Z, after processing, the software reported both the average ground sample distance and the estimated error between ground control points measured with the TotalStation to be 0.1 centimeters. The accuracy of the model was subsequently validated multiple times by making measurements within the model of the rulers that were placed in the grave; all measurements agreed to within one centimeter. The output obtained by merging models of the skeletons unearthed on the first, second, and third days of the expedition was likewise impressive. The resulting composite model clearly shows the three skeletons in their true orientations with respect to each other, as well as with the associated items present in the grave, such as clothing and other artifacts. This view, which was never observed in the field, reveals that the two skeletons unearthed on days two and three were buried significantly closer together than the skeleton recovered on day one. Similarly, the angles at which these two skeletons are buried is nearly identical, and subtly distinct from that of the first day’s skeleton. This may suggest that the skeletons of days two and three were buried at approximately the same time relative to the first set of remains, though this is far from certain. In addition to enabling measurements between multiple skeletons that were not possible in the field due to the sequence of exhumations, this model, by virtue of its being fully georeferenced, likewise allows investigators to measure any other parameter of the grave, such as the exact height of the individuals as they lay in place – information that was destroyed once the bones were removed from their original context. A depiction of this composite model is shown in Figure 5.

Figure 5. Georeferenced composite of all three skeletons at Site Z in the context of the others, all unearthed on different days (compare to Figure 2). From this view, it can be seen that the two skeletons at right, exhumed on days two and three, are more closely aligned and adjacent than the skeleton at left, exhumed on day one. This figure is a screenshot of a 3-D model that can be used by investigators to explore every aspect of the exhumation and conduct quantitative measurements, even long after the grave has been filled in. Estimated spatial resolution is one millimeter per pixel.

12

The airborne overviews of Site Z were also excellent, achieving a resolution of 2 centimeters per pixel, fifteen times higher than the best available satellite imagery and comparable to what was achieved using similar techniques in Colombia, but with the added benefit of consistent geometry due to the TotalStation’s GCPs. Combining the results of the air and ground-based datasets produced a series of maps that clearly show not only the details of the graves, but their broader context within the pasture, as shown in Figure 6. This information could be used to guide subsequent investigators to the site in the event that follow-up excavations are necessary, and enable them to avoid unnecessary effort by showing which areas have already been covered in previous expeditions. At Site X, the lessons learned from the data collection and processing at Site Z were applied successfully. Six models were created and merged together to create a comprehensive three-dimensional view of a grave that ultimately was found to contain the remains of eleven individuals. These results represent the culmination of the lessons learned over the past year and a half of research into the use of photogrammetry as a tool for gravesite documentation, however because this site is the subject of an active investigation and potential pending litigation, the results cannot yet be published for legal reasons. It is anticipated that as the case makes its way through the relevant judicial proceedings, that situation will change, and this report will be updated with a full presentation of these results when that occurs. In the meantime, it can be said with confidence that those results continue to bode very well for the use of photogrammetry in gravesite documentation.

Figure 6. Overview map showing the location of the gravesite at Site Z within a highland cow pasture in northern Guatemala. This view incorporates both ground-based and airborne photogrammetry along with ground control points measured using GPS and a TotalStation. Case information in the margins has been redacted at the request of FAFG.

13

Conclusions and Future Work This work demonstrates that photogrammetry is a powerful and effective tool to facilitate the search for and documentation of gravesites in a human rights context. In addition to enhancing the situational awareness of field teams investigating clandestine graves in remote environments, the technique was able to produce interactive visualizations that revealed information about multiple gravesites that was not immediately obvious on-site. These visualizations were not merely qualitative aids; the quantitative geospatial data associated with them allows them to be used to quantify parameters such as the distances between skeletons that were unable to be measured in the field, and which become impossible as the bones are removed from the context of the gravesite. Its ability to combine multiple datasets and render them in a spatially consistent manner, along with the speed with which data can be collected and the lack of perceived artistic interpretation, give photogrammetry significant advantages over methods of gravesite documentation such as traditional photography or illustrations on graph paper. Given that these results can be accomplished with inexpensive commercial off-the-shelf cameras and software that are far less expensive than laser scanners, this technique is likely to be particularly relevant to human rights NGOs operating with limited budgets. Where the terrain is sufficiently clear of obstacles to allow for automated flight operations, UAS-based photogrammetry can be a valuable tool for providing a broad overview of a site at resolution significantly higher than the best available satellite imagery. Under dense tree cover, ground-based imagery can also be used for this purpose, however the experience of this work shows that it is time-consuming in both the set-up phase (alignment markers are essential to the technique’s success, and must be carefully placed beforehand, low-lying vegetation must be cleared to allow for walking), and the processing phase (the resulting images must be manually aligned and calibrated). Due to the amount of effort necessary to create good results using ground-based photogrammetry in jungle terrain, it therefore cannot be recommended as a tool for surveying a wide area, however for up-close documentation of individual graves as they are exhumed, this technique produced excellent results. As an alternative to ground-based photogrammetry, this experience shows that smaller UAS such as the DJI Mavic Air can be flown effectively under forest canopy via full manual control (including the camera shutter), and can be used in this capacity for both the documentation of individual graves and smaller area surveys. The results of this technique were particularly effective at Site X, which could not be included in this report for legal reasons, but will be reported in an update. In all cases, confidence in the results of photogrammetric surveys was significantly enhanced through the use of ground control points to constrain the resulting model. These results would be improved yet further by using a real-time kinematic (RTK) GPS receiver to calibrate the TotalStation, rather than the handheld units available for this work. Currently, AAAS is exploring a number of applications of this technique that have the potential to increase its value to human rights advocates in the future. One such application involves correlating the orthomosaics generated from modern photogrammetric surveys with declassified US government satellite imagery acquired around the time that many of these graves were suspected to have been dug, which preliminary work suggests may improve the interpretability of contemporary data by providing relevant historical context. Another application involves using photogrammetry not just to document remains as they are exhumed in the field, but in the laboratory as well. Preliminary experiments at FAFG’s facilities in Guatemala City show that the technique can easily be adapted to creating appropriately-scaled 3-D models of skulls that bear evidence of trauma, for example (Figure 7). When the model resulting from these experiments was shared with a professional medical examiner, he indicated that the level of detail was sufficient to come to forensic conclusions about the remains. In the future, this technique could be an important way to preserve the record of trauma associated with remains, even after they have been returned to families for re-interment. Such models might even be 3-D printed, so that an accurate physical representation of the remains can be presented in court in cases where the remains themselves are no longer accessible.

14

A B

Figure 7. Proof-of-concept for the use of photogrammetry for the forensic documentation of human remains in a laboratory context. A: Photograph of a human skull suffering from trauma; one of a series taken from a stationary camera while the skull rotates on a turntable. B: View of the 3-D model resulting from photogrammetry, enabling measurements of the bullet wound (in this case, 1.7 centimeters), even after the remains are reinterred The potential use of this evidence in court raises another important issue associated with this technique. As an emerging technology that continues to evolve, developing consistent, workable standards and best practices for collecting and analyzing photogrammetric evidence in a legal context will be essential if its use in human rights investigations is to expand beyond the current proof-of-concept. To do this, conversations will need to take place among geospatial technologists, forensic scientists, human rights practitioners, lawyers, and members of the judiciary regarding the potential legal implications of the methods used to create these models. Training will necessarily be an important component of these conversations. Already, AAAS and FAFG have conducted multiple capacity-building exercises for both data collection and processing using anatomically correct plastic skeletons in simulated graves, and efforts are currently underway to expand these trainings to incorporate members of the Guatemalan legal community as well. Through these and similar efforts, the continued exploration of this and other emerging technologies will continue to improve the landscape of human rights practice, and form a core component of the Scientific Responsibility, Human Rights, and Law Program moving forward, consistent with its mandate to foster and facilitate the responsible practice and application of science in the service of society. References AAAS (2013). Human Rights Applications of Remote Sensing: Case Studies from the Geospatial Technologies and Human Rights Project. [Online] Available: https://www.aaas.org/report/human-rights-applications-remote-sensing Accessed 30 August 2017. Al-Ruzouq, R., Al-Rawashdeh, S., Tommalieh, O. and Ammar, M. (2012). Photogrammetry and GIS for three-dimensional modeling of the Dome of the Rock. Applied Geomatics, 4(4), pp.257-267. Agosto, E., Ajmar, A., Boccardo, P., Tonolo, F. and Lingua, A. (2008). Crime Scene Reconstruction Using a Fully Geomatic Approach. Sensors, 8(10), pp.6280-6302.

15

Marx, A. and Goward, S. (2013). Remote Sensing in Human Rights and International Humanitarian Law Monitoring: Concepts and Methods. Geographical Review, 103(1), pp. 100-111. McCarthy, J. (2014). Multi-image photogrammetry as a practical tool for cultural heritage survey and community engagement. Journal of Archaeological Science, 43, pp.175-185. Shugar, D. et al. (2017). River piracy and drainage basin reorganization led by climate-driven glacier retreat. Nature Geoscience, 10, pp. 370–375. Wang, B. et al. (2013). Problems from Hell, Solution in the Heavens?: Identifying Obstacles and Opportunities for Employing Geospatial Technologies to Document and Mitigate Mass Atrocities. Stability: International Journal of Security and Development. 2(3), p.Art. 53. DOI: http://doi.org/10.5334/sta.cn accessed 30 August 2018. Yeum, C. et al. (2017). Autonomous image localization for visual inspection of civil infrastructure. Smart Materials and Structures, 26(3), pp. 035051. Zawieska, D., et al. (2016). Multi-criteria analyses with the use of UAV’s for the needs of spatial planning. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, XLI-B1, pp. 1165-1171.