Touch And Go Camera System (TAGCAMS) Aboard the OSIRIS-REx Spacecraft

B.J. Bos1, M.A. Ravine2, M. Caplinger2, J.A. Schaffner2, J.V. Ladewig3, R.D. Olds3, C.D.

Norman3, D. Huish3, M. Hughes3, S.K. Anderson3, D.A. Lorenz1, A. May3, C.D. Jackman4, D.

Nelson4, M. Moreau1, D. Kubitschek5, K. Getzandanner1, K.E. Gordon1, A. Eberhardt1, D.S.

Lauretta6

1NASA Goddard Space Flight Center, Greenbelt, MD, USA (brent.j.bos@nasa.gov) 2Malin Space Science Systems, San Diego, CA, USA 3Lockheed Martin Space Systems, Littleton, CO, USA 4KinetX Inc., Simi Valley, CA, USA 5University of Colorado’s Laboratory for Atmospheric and Space Physics, Boulder, CO, USA 6Lunar and Planetary Laboratory, University of Arizona, Tucson, AZ, USA

Abstract NASA’s OSIRIS-REx asteroid sample return mission spacecraft includes the Touch And Go Camera

System (TAGCAMS) three camera-head instrument. The purpose of TAGCAMS is to provide imagery during the

mission to facilitate navigation to the target asteroid, confirm acquisition of the asteroid sample, and document

asteroid sample stowage. The cameras were designed and constructed by Malin Space Science Systems (MSSS)

based on requirements developed by Lockheed Martin and NASA. All three of the cameras are mounted to the

spacecraft nadir deck and provide images in the visible part of the spectrum, 400–700 nm. Two of the TAGCAMS

cameras, NavCam 1 and NavCam 2, serve as fully redundant navigation cameras to support optical navigation and

natural feature tracking. Their boresights are aligned in the nadir direction with small angular offsets for operational

convenience. The third TAGCAMS camera, StowCam, provides imagery to assist with and confirm proper stowage

of the asteroid sample. Its boresight is pointed at the OSIRIS-REx sample return capsule located on the spacecraft

deck. All three cameras have at their heart a 2592 × 1944 pixel complementary metal oxide semiconductor (CMOS)

detector array that provides up to 12-bit pixel depth. All cameras also share the same lens design and a camera field

of view of roughly 44° × 32° with a pixel scale of 0.28 mrad/pixel. The StowCam lens is focused to image features

on the spacecraft deck, while both NavCam lens focus positions are optimized for imaging at infinity. A brief

description of the TAGCAMS instrument and how it is used to support critical OSIRIS-REx operations is provided.

Introduction

The National Aeronautics and Space Administration (NASA) launched the OSIRIS-REx (Origins Spectral

Interpretation Resource Identification Security Regolith Explorer) asteroid sample return mission on September 8,

2016. The objective of the mission is to travel to the near-Earth asteroid (101955) Bennu, survey and map its

surface, obtain at least 60 g of surface material, and return the sample to Earth in 2023. An overview of the mission

is provided by Lauretta et al. (2017).

During the early development of the OSIRIS-REx flight architecture we determined that the selected science

instrument payload, when coupled to a composite spacecraft deck, would not provide sufficient pointing stability

over all expected thermal environments. Performance was found to be less than optimal from both an optical

navigation (OpNav) and natural feature tracking (NFT) standpoint, which are two of the key navigation techniques

for the mission. For instance, the pointing uncertainties when combined with the limited fields of view of the science

instrument payload were particularly concerning from a trajectory knowledge standpoint when operating close to

Bennu’s surface. In addition, lessons learned from other missions that executed proximity operations around other

low-gravity, Solar System objects highlighted the advantages of having an imaging system developed specifically

for asteroid navigation. After considering multiple architecture options we determined that having a dedicated and

fully redundant camera system was the best solution from a systems perspective. Such a system could be optimized

for acquiring the necessary spacecraft trajectory knowledge during maneuvers around a small, low-gravity body

such as Bennu. The Touch And Go Camera System (TAGCAMS) was added to the OSIRIS-REx spacecraft’s suite

of integrated sensors for this purpose. It provides situational awareness beyond what the science instrument suite

provides and primarily supports navigation to the asteroid, acquisition of the asteroid sample, and confirmation of

sample stowage.

TAGCAMS is a three camera-head system with two onboard digital video recorders (DVR). The primary navigation

camera is called NavCam 1 and interfaces to the first DVR. The backup navigation camera and primary natural

feature tracking (NFT) camera is called NavCam 2. It interfaces to the second DVR to provide system redundancy.

StowCam is the third camera head in the system and is used to verify asteroid sample stowage in the spacecraft’s

sample return capsule (SRC) and from the beginning had been planned to be a part of the spacecraft sensor

complement. Like NavCam 2, it interfaces to the second DVR. TAGCAMS supports critical OSIRIS-REx mission

operations so that the mission’s Level 1 science goals will be achieved, but the cameras were not designed to meet a

particular scientific imaging objective. However, we expect that the high-quality imagery and situational awareness

that the system provides will support supplementary scientific investigations during and after the primary mission.

The TAGCAMS system was designed and constructed by Malin Space Science Systems (MSSS) per requirements

developed by Lockheed Martin and NASA.

1.1 Primary Instrument Objectives and Requirements

The primary objective of NavCam 1 is to acquire images of the target asteroid with cataloged star fields in the

background no dimmer than 4th magnitude to support optical navigation of the OSIRIS-REx spacecraft while

approaching Bennu. Navigation images must be acquired without repointing the spacecraft while the asteroid fills up

to half of the camera field of view. This requires suppression of optical ghosts and stray light by the camera optical

system and the ability of the detector to accept an overexposure of up to 100 times without saturating the star field

portions of images. This is necessary because the typical NavCam scenes will have a high contrast ratio due to the

solar illumination of the asteroid and the low intensity of the target stars. Although the camera will not have to

provide images with both dim stars and the asteroid properly exposed in a single frame (separate images using

exposure times optimized for each object class will be acquired), the portions of the image that contain star fields

must remain reasonably clear of spurious light signatures to ensure reliable astrometric solutions. The optical

navigation algorithms require star images with signal-to-noise ratios (SNR) ≥4 while using exposure durations no

longer than 2 s.

NavCam 2 is a fully redundant navigation camera to NavCam 1 and has identical requirements, but is the primary

natural feature tracking (NFT) camera. The imaging requirements to support NFT are not nearly as difficult as those

that support optical navigation. NFT performance was an important factor considered during the development of the

40° × 30° field of view requirements to ensure that numerous features of interest on the asteroid are available for

image registration and image-to-image tracking. Similar considerations drove the requirement for the system to be

able to resolve features separated by ~0.5 mrad in object space with at least 10% contrast. The final key requirement

for NFT was a full-frame imaging rate of 0.1 Hz (one full frame every 10 s). This ensures that the NFT algorithms

will be able to calculate the position and orientation of the spacecraft with respect to the asteroid touch-and-go

landing site in real time, onboard the spacecraft.

StowCam is identical to the NavCams with two exceptions. StowCam includes a Bayer filter array on the camera’s

detector so that approximate true-color images can be acquired. The lens-to-detector distance is also different, to

optimize the focus for imaging the SRC located on the spacecraft deck. The primary purpose of the StowCam is to

provide imagery that confirms the proper seating of the asteroid sample head under the three latches located in the

SRC. Two of the three latches are within the StowCam’s field of view, while the third is obstructed by the sample

head itself. The orientation of the sample head in StowCam imagery will allow engineers to infer proper engagement

of the obstructed latch. This imaging task required the StowCam to have a field of view of least 40° × 30° and to

resolve objects separated by 1 mrad with a contrast of at least 10% over a depth of field from 0.7 m to 1.3 m.

2 System Description

2.1 Instrument Heritage and Background

MSSS has designed, built, and flown over a dozen imaging systems for deep-space science missions (e.g.,

Christensen et al. 2004; Malin et al. 2007; Robinson et al. 2010). TAGCAMS was developed from the MSSS

ECAM product line first released in 2013, a modular low-mass camera system intended to support in-flight

engineering diagnostics, deployment/actuator monitoring, space situational awareness, and public outreach, as well

as science applications.

Building on heritage from the MSSS Mars Science Laboratory cameras (Edgett et al. 2012), the ECAM cameras

utilize a digital video recorder (DVR) which contains an eight gigabyte (GB) flash memory buffer and a Xilinx

Virtex-4 radiation-tolerant field programmable gate array (FPGA), and can use any interface that operates with data

transmission standard RS-422, low voltage differential signaling (LVDS), or discrete voltage levels with up to four

signals in each direction. The DVR communicates with its attached camera heads using the SpaceWire (European

Cooperation for Space Standardization 2008) protocol running at a maximum rate of 100 MHz. It is powered from

the host spacecraft using unregulated 28 V power, and distributes switched 5 V power to the camera heads.

The first instance of an ECAM system to be delivered for flight is the TAGCAMS for OSIRIS-REx. An image of

the flight TAGCAMS is shown in Figure 1. In the flight system two DVRs are used for fault tolerance; NavCam 1 is

attached to its own DVR, with NavCam 2 and the StowCam connected to the second DVR. No cross-strapping

between DVRs is provided. See Figure 2 for a system block diagram.

Figure 1. Flight TAGCAMS camera heads and DVR, shown with the 84 mm long pocketknife for scale. The two DVRs are shown stacked on top of each other.

Figure 2. TAGCAMS system block diagram showing NavCam 1 electronically connected to the primary DVD and NavCam 2 and StowCam connected to the second DVR. Spacecraft communications and power flow between the camera interface adaptor and focal plane assembly.

2.2 Optics Overview

All three TAGCAMS cameras use the standard medium field of view (MFOV) lens offered by MSSS, a seven-

element all-refractive lens with a nominal focal length of 7.6 mm. When coupled with the TAGCAMS detector the

lens creates a field of view of about 44° × 32° and an on-axis pixel scale of 0.28 mrad/pixel. The nominal on-axis

focal ratio is 3.5. The front element is made of fused silica to provide radiation shielding for the remaining elements.

Distortion was not controlled in the design, and the lens exhibits 13.9% of barrel distortion at the corners of the field

of view. An IR-cutoff filter is included to restrict the lens bandpass to 400–700 nm. The relative illumination at the

edge of the field is about 1.1% less than at the center. Lens focus is adjusted by polishing an aluminum focus spacer

which was fixed in place after the lens was integrated with the electronics.

2.3 Electronics Summary

2.3.1 Camera Heads

The visible light camera heads are based on commercial complementary metal oxide semiconductor (CMOS) image

sensors. For TAGCAMS, the MSSS product line ECAMC50/M50 camera head is used; this camera uses the ON

Semiconductor (formerly Aptina) MT9P031 5-megapixel image sensor. This sensor has 2592 × 1944, 2.2-micron

pixels with an optional Bayer-pattern color filter array and on-chip 12-bit digitization. The vendor-reported 2.2-

micron pixel pitch was verified by microscopic inspection. StowCam uses the Bayer-pattern filter detector to

provide color images, while the NavCams use the monochrome version. A rolling electronic shutter controls

exposure times. Testing showed (Becker et al. 2008) that the sensor is tolerant to radiation of at least several krads

of total ionizing dose; the modeled total dose at the end of the mission is about 1 krad. The camera heads also

include an Actel FPGA for sensor control and communications, and voltage regulation to supply power to the

camera head components using the 5 V power provided by the DVR. A block diagram of the camera heads is shown

in Figure 3.

Figure 3. TAGCAMS camera head block diagram illustrating the command and power flow between the camera interface adaptor and focal plane assembly.

Two pixel bit-depth options are provided. The camera head electronics read out the sensor at a pixel rate of 5

megapixels/sec (12 bits) or 10 megapixels/sec (8 bits). The sensor always produces 12-bit pixel values with digital

numbers (DN) ranging from 0 to 4095; the readout rates are different to match the available data rate of the interface

to the DVR. The nominal system gain is 2.0 electrons/DN.

In 8-bit mode, 12-bit DNs are converted to 8-bit using a table-defined transfer function implemented in the camera

head. The transfer function can be set for square-root encoding to preserve the full 12-bit dynamic range. Any

power-of-two linear mapping can also be used (divide by 16, 8, 4, 2, or 1) to simplify data processing. In general,

linear divide-by-N mode can be thought of as discarding the least significant log2(N) and most significant 4-log2(N)

bits of each 12-bit pixel.

2.3.2 DVRs

The DVR’s Xilinx FPGA includes an embedded processor to run the instrument’s flight software, and a set of logic

peripherals to perform functions such as SpaceWire communication with the cameras, communication with the host

spacecraft, and data processing and compression of images. The latter includes selectable lossless and JPEG image

data compression, and for StowCam, Bayer-pattern interpolation and color-space management.

For OSIRIS-REx, the spacecraft interface with the DVR uses RS-422 electrical levels. Commanding and

engineering telemetry is sent using a bidirectional asynchronous serial interface running at 57.6 kbaud. Images are

sent using a unidirectional synchronous serial interface with three signals (clock, data, and frame sync) running at 4

MHz. A 5 V opto-isolated discrete line is used to send a one-pulse-per-second signal from the spacecraft for time

synchronization, and all images are time-tagged. A block diagram of the DVR is shown in Figure 4.

Figure 4. TAGCAMS DVR block diagram of a single DVR slice illustrating the potential to connect up to four camera heads to a single DVR. For the TAGCAMS primary DVR only the camera 0 interface is utilized for NavCam 1. For the second TAGCAMS DVR only the camera.

2.4 Flight Software

The flight software is a simple, single-threaded control program written in ANSI C. The internal SRAM memory

used for program and data storage is error corrected for fault tolerance, as is the flash memory buffer. A watchdog

timer is provided to protect against radiation-induced upsets, and the spacecraft can command a system reset that

completely reloads the FPGA logic and flight software to clear any latent errors.

The internal flash buffer can hold approximately 1,700 uncompressed 8-bit full-frame images or about half as many

12-bit images (stored in two bytes/pixel form). The maximum frame rate is a little over 1 frame per second in 8-bit

mode, and such video can be JPEG-compressed in real time. Subframes or down-sampled frames can be captured at

higher frame rates.

Images can be acquired in uncompressed form in either 12-bit or 8-bit mode and stored in flash for subsequent

playback. Images can be played back in their original form or, optionally, processed in a variety of ways: 12-bit

images can be packed to reduce their data volume from two bytes/pixel to 1.5 bytes/pixel; 8-bit images can be

compressed using a simple, lossless Huffman-encoded predictive algorithm, summed 2 × 2 in software to reduce

their data volume, or compressed using JPEG at a selected quality factor in either color or grayscale mode. In 8-bit

mode, the compression can be applied at acquisition time to save flash memory space; if this is done, further

processing is not possible.

By default, a version of software loaded into the DVR’s one-time-programmable EPROMS (erasable programmable

read-only memory) is run at power-on. A software update can be loaded into flash, and if present, is used instead of

the EPROM version. This allows software to be updated after delivery and in flight if necessary. A mechanism for

inhibiting the loading of the flash version is provided in the event an erroneous load is stored in flash.

The software provides a means of loading test images into flash for testing of optical navigation algorithms; this

capability was used extensively during OSIRIS-REx assembly, test, and launch operations.

A large number of operating parameters are stored in flash and can be updated persistently as needed. These

parameters include the rate of periodic telemetry production, timer values for camera power control, camera sensor

operating register values, and color interpolation and companding tables.

2.5 Ground Processing

The OSIRIS-REx mission-specific ground data system (GDS) for commanding and processing TAGCAMS imagery

consists of: the mission support area (MSA; Lockheed Martin, Denver, CO); the flight dynamics system (FDS;

Goddard Spaceflight Center, Greenbelt, MD, and KinetX Aerospace, Simi Valley, CA), and the science processing

and operations center (SPOC; University of Arizona, Tucson, AZ). The MSA conducts spacecraft operations

(command/telemetry). Spacecraft and instrument data for optical navigation are received at the MSA and channeled

to the SPOC for processing and then received by the FDS. The FDS team produces mission design and navigation

products for the mission based on this information. The SPOC processes science instrument data into high-level

combined data sets in standard image formats and archives the data to the Planetary Data System (PDS) for the

benefit of future users. The SPOC TAGCAMS image processing is not as extensive as it is for the science

instruments, modifying only the image headers while leaving the image portion of the data unchanged.

Upon image downlink from the OSIRIS-REx spacecraft, TAGCAMS image headers already contain 136 bytes of

ancillary camera and spacecraft information generated by onboard processing. The downlinked header information

includes spacecraft clock time, camera temperature, image sequence identification, spacecraft attitude, and

spacecraft attitude rate. After the images are received on the ground, the raw TAGCAMS files are processed by the

SPOC to create Flexible Image Transport System (FITS) format image files. SPOC processing appends the FITS

image file headers with spacecraft attitude information calculated using attitude and timing information downlinked

separately from the image files. The image portions of the FITS files are left unaltered by the SPOC processing, and

pixel values remain in digital number (DN) units for subsequent correction, calibration, or additional processing by

the end users of the images.

2.6 Spacecraft Camera Configurations

The TAGCAMS are mounted on the spacecraft’s nadir deck, in the XY plane of the spacecraft coordinate system.

This affords the NavCams a clear view to space for navigational purposes and unobstructed viewing of Bennu and

positions StowCam for optimal viewing of the sample as it is delivered to the SRC. NavCam 1 and NavCam 2 are

positioned close together on the –Y side of the spacecraft deck (the spacecraft coordinate system Z axis runs through

the center of the SRC) with NavCam 1 located slightly more inboard than NavCam 2. StowCam is located on the

+Y side of the spacecraft deck (see areas circled in Figure 5).

Figure 5. TAGCAMS camera locations on the OSIRIS-REx spacecraft deck. The coordinate system shown is the spacecraft coordinate system. The two NavCams are located on the –Y side of the spacecraft deck (+Z runs through the center of the sample return capsule (SRC)). The StowCam is located on the +Y side of the spacecraft deck and pointed toward the SRC.

The boresights and orientations of the cameras are optimized to meet mission objectives. StowCam is attached to a

canted, raised bracket for viewing of the SRC latches and sample stowage. NavCam 1, as the primary navigational

camera, is positioned with its boresight as close as practical to nadir while taking into account spacecraft and

operational constraints. Specifically, the boresight of NavCam 1 is canted +6° about the +Y axis and +20° about the

boresight. The camera’s as-installed boresight unit vector is [0.104044, 0.000706, 0.994572] in the spacecraft

coordinate system. NavCam 2, as the primary NFT camera, is optimized for asteroid viewing. Specifically, the

NavCam 2 boresight is canted –14° about the +Y axis and +17° about the boresight. The NavCam 2 as-installed

boresight unit vector is [–0.24324, 0.002187, 0.969963] in the spacecraft coordinate system. The cant angles are

achieved through the mechanical design of the camera brackets.

3 Performance

3.1 Instrument Calibration Summary

The flight cameras were calibrated at MSSS before delivery to the spacecraft. Tests there included sensor linearity,

read noise, full well, gain, dark frame, and dark current; absolute radiometry (conversion between DN and radiance);

flat fields; modulation transfer function (MTF); focal length, field of view and geometric distortion; instrument

alignment between boresight and alignment cube normals; and stray light performance. These tests were all done

under ambient conditions. A subset of calibration was repeated during instrument-level thermal vacuum testing. This

included dark current measurements and MTF to verify athermal optics performance.

Most of the tests were conventional. Linearity, read noise, gain, and radiometry tests made use of a quartz-tungsten-

halogen (QTH) light source and integrating sphere; the photon transfer technique (Janesick 2007) was employed to

determine the system scale factor. Instrument alignment was determined with a theodolite and autocollimator. Focal

length and distortion were measured by analysis of an ensemble of images of a dot target in multiple distances and

poses (using the same technique discussed further in Section 3.2). MTF was measured by analysis of a slanted edge

target.

Because of the challenging nature of the star/asteroid optical navigation application, we went to some effort to

simulate the performance of the system for such imaging. We used a spatial source with radiance similar to that of

an asteroid, surrounded by simulated 4th magnitude stars, to verify that the camera could detect the stars with

adequate signal-to-noise ratio (SNR) despite interference from asteroidal stray light. NavCam 1 detected the

simulated 4th magnitude stars with a SNR of 9.7 while NavCam 2 detected the same stars with a SNR of 6.1, well

above the 4.0 SNR requirement. While relatively crude, this test provided the initial verification that stray light had

been adequately controlled. The validity of this test was subsequently proven by night sky imaging of standard stars

using an engineering model camera identical in design to the flight cameras (see Section 3.2.3).

The cameras are required to meet focus and resolution requirements over a range of target distance: StowCam from

0.7 to 1.3 meters, and NavCam from 5 meters to infinity. The hyperfocal distance equation was used to derive the

single finite target distance that provides best depth of field over these ranges, which was verified by modeling of

MTF over the range. The cameras were focused using targets at those distances: 1.1 meters for StowCam and 8

meters for NavCam. Focus was initially set under ambient lab conditions and then adjusted for the shift in the plane

of best focus in vacuum. Figure 6 shows an image taken with the flight StowCam, focused at 1.1 m, under ambient

conditions and prior to the application of the vacuum focus offset. Figure 7 shows an image taken with the flight

Navcam1, focused at 8 m, again prior to the vacuum offset.

Figure 6. StowCam image of an optical target located 1.1 m away at Malin Space Science Systems, illustrating the StowCam image quality at the required object distance.

Figure 7. NavCam 1 image of an optical target located 8 m away at Malin Space Science Systems, illustrating the NavCam image quality at the required object distance.

While exposure times of up to 30 seconds can be achieved, typical exposures will be 2 seconds or less. The noise

performance over temperature for 2 s exposures was measured and is relatively constant up to a temperature of

about 40°C, when dark current begins to increase significantly. However, even at temperatures of 60°C useful

images can still be obtained.

The sensor has an adjustable on-chip gain amplifier that can be set from a gain factor of 1 to 8. For most

applications, we expect a gain factor of no more than 1.25 to be used to maximize the sensor’s dynamic range. A

summary table of the measured parameters of each camera head is shown in Table 1.

Table 1. Measured TAGCAMS imaging properties. Measured at ambient, note that Bayer color interpolation degrades the MTF somewhat from what would be expected for a monochrome system.

Parameter NavCam1 NavCam2 StowCam

focal length (mm) 7.79 7.65 7.78

Horizontal FOV (°) 43.55 44.32 43.58

Vertical FOV (°) 31.86 32.36 31.90

MTF (-40° C, %) 21.3 16.9 —

MTF (+55° C, %) 23.0 20.0 12.7*

scale (e-/DN, 1× gain) 1.79 1.76 1.76

noise (e-) 6.7 6.7 6.5

3.2 Engineering Model Tests

In addition to the three flight camera heads that MSSS provided, a flight-spare camera head and an engineering

model NavCam were delivered as well. Although the flight-spare head remained safely in bonded stores before the

launch of OSIRIS-REx, the engineering model (EM) camera allowed us to experiment with camera settings and

environments that would be imprudent or impractical to attempt with the flight system. The EM NavCam proved to

be invaluable in understanding camera idiosyncrasies, optimizing performance, and characterizing performance in

flight-like conditions.

3.2.1 EM Calibration Summary

Due to schedule constraints, the EM NavCam was not calibrated at MSSS. Instead, primary calibration of the EM

NavCam was performed at Lockheed Martin’s Space Operations Simulation Center (SOSC) in Littleton, Colorado.

The SOSC is a test facility that supports rendezvous and docking development tests and risk reduction activities. It

consists of several individual laboratories as well as a large high-bay, 60 m long by 15.2 m wide and 15.2 m tall with

dual six-degree-of-freedom motion simulators and a six-degree-of-freedom robot (Milenkovich and D’Souza 2012).

Two primary EM NavCam calibration tests were performed at the SOSC. The first measured the camera intrinsic

parameters of focal length, distortion, and the point of intersection between the optical axis and the focal plane. The

second test used photon transfer analysis (Janesick 2007) to measure the focal plane characteristics of sensitivity,

analog to digital converter (ADC) offset, read noise, full well onset, SNR, and fixed pattern noise.

Analysis of the optical EM NavCam measurements was aided by the Camera Calibration Toolbox for Matlab

(CCTM) (Bouguet 2014). Similar to the way the flight cameras were calibrated, the CCTM computes the camera

optical characteristics by imaging multiple views of a calibration target of known size but unknown position relative

to the camera. An iterative least-squares approach is used to solve for the camera intrinsics and extrinsics relative to

the target simultaneously. A total of 47 images were used in the solution, including the image shown in Figure 8.

Figure 8. On the left, raw EM NavCam example image of the checkerboard target used for calibrating camera intrinsic parameters using the Camera Calibration Toolbox for Matlab. On the right, EM NavCam image with the optical distortion shown on the left removed.

Focal length was measured to be 3465.35 ±2.48 pixels in the X (horizontal) direction and 3468.50 ±2.23 pixels in

the Y (vertical) direction. With a pixel pitch of 2.2 µm the focal lengths convert to 7.62 mm and 7.63 mm,

respectively.

The point of intersection between the optical axis and the EM NavCam detector was measured to be pixel (1382.26,

951.597). The NavCam detector image has a total of 2592 pixels in the horizontal, or X, direction, and 1944 pixels

in the vertical, or Y, direction, resulting in a mechanical center of (1295, 971) where the center of the first pixel in

the upper left corner is (0,0). Thus the optical axis is offset from the mechanical center of the focal plane by (87.3, –

19.4) pixels.

The CCTM analyzed both tangential and radial distortions. Total pixel shifts in the EM NavCam corners approach

160 pixels. The three sigma reprojection accuracy of the distortion map was measured to be 0.21 pixels in X and

0.18 pixels in Y over the entire set of test images.

Figure 9 shows the photon transfer test setup for the EM NavCam. The camera was placed such that the entire field

of view could see into an integrating sphere. The sphere was driven by a single superbright light-emitting diode

(LED). The LED was driven by a current regulating circuit such that it could be driven from off to full power at a

fine enough resolution to capture images across the entire dynamic range of the camera. The EM NavCam exposure

time was fixed at 0.5 seconds with a gain value of 1.25. LED current and voltage were recorded for every image.

Figure 9. EM NavCam viewing an integrating sphere for the photon transfer test completed at Lockheed Martin.

Two images were taken at each light setting and used in the photon transfer analysis.

Photon transfer image sets were captured for ten variations of camera settings. Each variation modified a particular

parameter in order to see the effects of that parameter on focal plane performance. Varied parameters included gain,

bit depth, and internal image processing functions. Investigation of these parameters led to more optimum pixel

transfer voltage and gain values than the MT9P031 sensor factory settings. Both the EM NavCam and flight

TAGCAMS use these improved sensor settings.

3.2.2 Natural Feature Tracking Tests

During the OSIRIS-REx touch and go (TAG) sample acquisition maneuver at the asteroid, NFT is one of the two

available technologies that can be used to guide the spacecraft’s descent to the desired sample acquisition (Olds et

al. 2015; Mario and Debrunner 2016). It is an important algorithm for the mission which operates autonomously on

the spacecraft computer, and high-fidelity testing of the entire NFT logic and supporting hardware was necessary to

validate performance. Light detection and ranging (LIDAR) to measure the limb crossing and single range value is

the second, independent technology provided by the spacecraft to support the TAG (Berry et al. 2013).

Of particular concern to NFT operations is the large amount of optical distortion present in NavCam images.

Because NFT must accurately predict where a landmark falls on the camera focal plane, NFT uses an OpenCV

distortion model (OpenCV 2016) to correct for optical distortion. EM NavCam optical distortion measurements

were used to verify the ability of the NFT software to properly model distortion effects. Figure 8 illustrates how the

optical distortion effects are corrected using these calibration measurements.

Another NavCam characteristic of interest to NFT operations is the camera’s rolling shutter. Due to the 5 MHz (in

12-bit mode) or 10 MHz (in 8-bit mode) readout of the TAGCAMS detectors, there is either a 0.6984 ms or 0.3492

ms exposure delay between each successive row in an image. For periods where the spacecraft has a trajectory that

causes terrain to move quickly through the camera field of view, this rolling shutter effect can cause a significant

error in the predicted pixel locations of objects of interest. This is especially true when the spacecraft is at low

altitudes and moving rapidly with respect to the surface.

To investigate the NFT impacts of the rolling shutter, the EM NavCam was moved at high lateral rates with respect

to a checkerboard target (see Figure 10). The lateral speed selected for the test was significantly higher than

expected in flight in order to produce an exaggerated rolling shutter effect of around 27 pixels (8 mrad). An

algorithm corner detector was used to measure the coordinates of the checkerboard intersections in one image. For a

subsequent image, the intersection coordinates were corrected with and without an NFT rolling shutter model to test

how well the algorithm could predict the new locations of the checkboard intersections in the second image. The

results are illustrated in the image in Figure 11. The blue circles represent the prediction without the rolling shutter

algorithm active, while the green circles show the predictions with the rolling shutter algorithm. The exaggerated

rolling shutter effect of 27 pixels (8 mrad) generated during the test was corrected to within a pixel (<0.28 mrad).

The test confirmed that the NFT rolling shutter model was able to accurately remove the NavCam rolling shutter

effects.

Following validation of the optical distortion and rolling shutter models used by NFT, a full end-to-end test was

performed at the SOSC by “flying” the EM NavCam along a TAG-like trajectory over a large model of a low-albedo

asteroid surface. The EM NavCam was mounted to a large robot that was used to simulate the trajectory over the

artificial landscape. Images of the simulated asteroid surface were captured and ingested into a NFT simulation to

see how well NFT could reconstruct the trajectory flown by the robot by tracking features on the asteroid model. A

depiction of this test is shown in Figure 12. The image shown in the figure is one example of an image captured by

NavCam. The figure illustrates the features on the wall being tracked in the image during the simulation. NFT

performance using the EM NavCam met requirements for all test cases.

Figure 10. EM NavCam checkerboard image from the rolling shutter test completed at Lockheed Martin.

Figure 11. Example of the NFT rolling shutter model compensation in an EM NavCam image. Blue circles show the predicted corner locations without compensation. Green circles show that the rolling shutter model accurately predicts where the checkerboard corners are located in a NavCam image.

Figure 12. NavCam image and NFT processing examples from the NFT asteroid wall test. NFT rendered tracking features are shown on the left. The NavCam image of the asteroid wall is shown in the upper right. Two plots illustrating the reduction of NFT position and velocity prediction errors over time are shown at the bottom of the figure. The colors in the two plots

correspond to the red, green and blue features shown in the larger image. Time is shown in units of seconds from the orbit departure maneuver.

3.2.3 Night Sky Imaging

Another test in the EM NavCam test campaign was a night sky imaging test to acquire images of stellar targets.

Sensitivity to relatively dim stars while bright, extended objects are in the NavCam field of view is a key camera

performance parameter and critical to the performance of our optical navigation algorithms. The test was conducted

outdoors, at the Lockheed Martin facility in Littleton, Colorado. The primary purpose of the test was to verify the

point source sensitivity of the NavCams by imaging several stellar targets with and without other bright sources in

the field of view. Stellar images were analyzed to determine if the NavCam design meets the requirement of imaging

4th magnitude stars with a signal-to-noise ratio ≥4 in the presence of significant stray light. The images acquired

verified the results of the prior point source and stray light sensitivity tests performed at MSSS on the flight units.

Figure 13 shows the large number of stars dimmer than 4th magnitude that were visible in the Cassiopeia

constellation in a two second–exposure EM NavCam image. Stars as faint as magnitude 5.06, over 2.5 times fainter

than required, were successfully detected. The atmospheric optical depth measured during the test was 0.29.

One observational scenario targeted 4th magnitude stars in the Pleiades constellation while a simulated asteroid was

in the field of view. The simulated asteroid was constructed from a white Styrofoam® disk positioned above the

camera and illuminated using a high-intensity stage light to produce object radiances of the same order of magnitude

expected from Bennu (Fig. 14). Images were obtained with the simulated asteroid at varying locations in the field of

view, varying illuminations, and varying exposure times. Two-second exposures of the asteroid under expected

flight illumination conditions produced images of several 4th magnitude stars with SNR >4, suitable for optical

navigation. Figure 15 shows a close-up of a two second–exposure, 4th magnitude star image from the test.

Figure 13. EM NavCam image subframe of the Cassiopeia constellation from a 2 s exposure. Multiple stars dimmer than 4th magnitude are indicated with yellow arrows along with their apparent stellar magnitude. Measurements taken during the night sky imaging test indicate the optical depth was 0.29 over the NavCam passband, consistent with the slightly hazy atmosphere with the peak zenith atmospheric transmittance of 0.75. The image subframe is displayed with the nonlinear DN stretch to enhance the visibility of dim stars.

Figure 14. EM NavCam night sky imaging test setup with simulated asteroid in the field of view. The EM NavCam is mounted to a motorized tripod with an integral star finder.

Figure 15. Close-up of a sample EM NavCam image of a 4th magnitude star with a SNR >4 and background light pollution removed. The grayscale colorbar on the right-hand side represents units of DN.

4 Operations

4.1 Optical Navigation

Optical navigation (OpNav) is one of the key OSIRIS-REx operations activities the TAGCAMS was designed to

support. OpNav, a subfunction of the Flight Dynamics System (FDS), uses information extracted from spacecraft

images to determine spacecraft trajectory. As described by Williams et al. (2017), there are two OpNav image

processing techniques used on the mission: star-based and landmark navigation. Use of one versus the other is

dependent on proximity to Bennu and the availability of digital terrain maps (DTMs) of the surface, which will be

developed over the course of proximity operations (Lauretta et al. 2017). Both techniques use NavCam images to

calculate the spacecraft ephemeris relative to the asteroid.

Star-based OpNav is utilized in early mission phases, before global imaging data and DTMs are available for

landmark navigation. The objective of star-based OpNav is to determine the position of the target body center

relative to starfields. To solve for a more accurate inertial camera pointing in the presence of pointing-knowledge

errors, the a priori pointing estimate is adjusted to minimize the differences between imaged background star

locations and the cataloged star positions. Bennu is too bright to allow for stars and a well-exposed asteroid to be

captured in a single NavCam image, and an imaging strategy has been developed to bracket a proper exposure of the

asteroid with longer exposures for stars. This is a three-step process. First, NavCam pointing errors are removed by

taking an image of background stars and using their known locations to solve for the true inertial camera pointing.

Next, the target body is imaged with a shorter exposure, and a solution is obtained for the target body center. This

provides accurate delta-quaternion data between NavCam exposures, allowing for accurate determination of the

asteroid position based on the inertial pointing solution from the previous stellar image. An additional third image,

exposed for stars, is acquired to verify the pointing-solution accuracy. The pointing solution degrades over time, so

the time between the three exposures is minimized.

Accurate inertial camera pointing allows for precise calculation of the predicted location of the Bennu center of

mass (CM). The observed location of the Bennu CM is derived using a variety of algorithms available in the KinetX

Star-Based Image Processing Suite, KXIMP (Jackman et al. 2016). The orbit determination (OD) estimation filter

minimizes the difference between observed and computed image-plane body centers, along with other radiometric

tracking data measurements. The OpNav technique is used from the beginning of the asteroid approach phase until

OSIRIS-REx enters a 1.5-km circular terminator orbit, when the transition to landmark navigation occurs. NavCam

1 becomes the primary OpNav instrument at the end of the asteroid approach phase, while NavCam 2 serves as the

backup. Prior to that, imagery provided by the OSIRIS-REx camera suite (OCAMS) supports OpNav (Rizk et al.

2017).

After Bennu global imagery is available to create DTMs of the surface (Gaskell et al. 2008) and the spacecraft is in

close proximity to the asteroid, we will use NavCam images for landmark navigation. Landmark navigation utilizes

DTMs to correlate landmark features in an image. These landmark measurements are used to significantly reduce

large pointing-knowledge errors once stars are no longer observable in a NavCam frame. The landmark center

positions and camera pointing are the key measurements extracted from the images and fed into the OD filter. The

transition from star-based to landmark OpNav is a critical phase of the mission and a key driver for NavCam

requirements. The transition from body-centric to feature-relative positioning affords better accuracy and is required

to meet challenging ephemeris-prediction accuracy requirements for the highest resolution science observations. The

large field of view of NavCam allows for imaging many landmarks on the surface, enabling the OpNav software to

break the degeneracy between position errors and large pointing-knowledge errors and compute accurate spacecraft

positions with respect to Bennu.

As previously discussed, the primary difference between NavCam 1 and its OpNav backup, NavCam 2, is mounting

orientation on the spacecraft. During proximity operations, the NavCam 1 orientation is preferential because it

simplifies the planning and execution of OpNav images. In orbit, this orientation tilts the boresight toward the sunlit

side of the asteroid and results in an image frame rotated such that the long diagonal is oriented with the down-track

direction of the trajectory, which is the direction of the largest uncertainty. NavCam 2 is oriented preferentially for

the NFT system and TAG attitude profile. Should an anomaly happen with NavCam 1, NavCam 2 will become the

prime camera for OpNav, and spacecraft operations and ground planning will incorporate off-nadir slews to capture

the sunlit portion of the asteroid.

4.2 Natural Feature Tracking

Natural feature tracking (NFT) uses NavCam 2 to autonomously track features on the surface of the asteroid during

the TAG attempt. A depiction of the TAG trajectory and operations concept is shown in Figure 16. After each image

is taken, NFT uses its onboard knowledge of the state to propagate the position and velocity to the current image

exposure time. Using an a priori state estimate, spacecraft attitude, and onboard camera models, NFT predicts which

features it expects to see in the camera field of view. The expected appearances of these predicted observations are

rendered onboard based on the estimated camera pose, intrinsic camera parameters, asteroid attitude, and the Sun

position. Each rendering is correlated against the image data retrieved from the NavCam memory. Only small search

regions in the image surrounding the expected position of the features are used in the correlation process for

efficiency. Correlation peaks represent where the actual feature was measured in the image. The residuals between

the predicted feature locations and the measured feature locations are used to update the onboard state knowledge.

This process is repeated for each NavCam image sent to the NFT software. Using this onboard knowledge of the

orbital state, NFT is also used to predict the state at future epochs.

Figure 16. Illustration of the OSIRIS-REx orbital departure, TAG maneuver, and sample collection. NFT processing of NavCam images starts 200 minutes before touchdown.

The spacecraft will assume the three spacecraft attitudes shown in Figure 19 during NavCam imaging: NFT attitude,

TAG look-ahead attitude, and TAG attitude. While the spacecraft holds these attitudes it does not actively track the

asteroid. The first NavCam image is collected by NFT at the TAG–200 minute point which is approximately 1 hour

after the orbit departure maneuver (ODM). At this point, NavCam begins imaging the surface once every 10

minutes. Each image is captured in 8-bit linear divide by 16 companding mode for NFT processing and is tagged

with the exposure time and spacecraft attitude.

4.3 Sample Delivery Verification

After asteroid sample acquisition is complete, the StowCam provides visual verification of the Touch And Go

Sample Acquisition Mechanism (TAGSAM) sampler head insertion into the SRC. Figure 17 shows the placement of

StowCam with respect to the SRC and the spacecraft deck hardware within its field of view.

Figure 17. Illustration of the StowCam location on the spacecraft deck and its corresponding field of view (magenta). StowCam documents the sample head stow procedure to confirm successful delivery of the sampler head to the SRC.

An example StowCam image from sampler head insertion during spacecraft-level testing is shown in Figure 18. The

image provides visual confirmation that the sampler head successfully latched into the SRC. Similar images

acquired prior to insertion will provide ground operators visual confirmation that the sampler head is properly

aligned with the SRC capture-ring latches. If unexpected alignment errors are encountered during flight operations,

the ground operators will use the StowCam images to determine how to precisely adjust the TAGSAM arm to reach

the required pre-stow positioning.

Figure 18. StowCam image taken during ground testing showing the TAGSAM sampler head latched into the SRC capture ring

5 Summary

NASA’s OSIRIS-REx asteroid sample return mission launched in September 2016 and will return a sample of

asteroid Bennu to Earth in 2023. The three-camera TAGCAMS instrument plays a critical role during navigation to

the asteroid, acquisition of the asteroid sample, and confirmation of sample stowage. Comprehensive ground-testing

and instrument calibration activities, at ambient as well as in-flight, operational temperatures, have verified that the

cameras will successfully support their mission-critical roles. TAGCAMS flight image data will be submitted to the

Planetary Data System (PDS) for use and study by the broader community. Although not specifically designed to

meet scientific imaging objectives, the high-quality imagery provided by TAGCAMS will provide important

capabilities in the investigation of Bennu’s surface.

Acknowledgements This material is based upon work supported by NASA under Contracts NNM10AA11C,

NNG12FD66C and NNG13FC02C issued through the New Frontiers Program. Copy editing provided by

Mamassian Editorial Services.

References

H.N. Becker, M.D. Dolphin, D.O. Thorbourn, J.W. Alexander, P.M. Salomon, Commercial sensor survey radiation testing progress report. Jet Propulsion Laboratory Publication 08-22, April (2008)

K. Berry, B. Sutter, A. May, K. Williams, B.W. Barbee, M. Beckman, B. Williams, OSIRIS-REx Touch-And-Go

(TAG) mission design and analysis. 36th Annual AAS Guidance and Control Conference, 13-095, 1–6 February (2013)

J.Y. Bouguet, Camera Calibration Toolbox for Matlab. (Computer Vision Research Group, Dept. of Electrical

Engineering, California Institute of Technology, 2014) http://www.vision.caltech.edu/bouguetj/calib_doc/. Accessed 30 December 2016

P.R. Christensen, B.M. Jakosky, H.H. Kieffer, M.C. Malin, H.Y. Mcsween Jr., K. Nealson, G.L. Mehall, S.H.

Silverman, S. Ferry, M. Caplinger, M. Ravine, The Thermal Emission Imaging System (THEMIS) for the Mars 2001 Odyssey mission. Space Sci. Rev. 110, 85–130 (2004)

K.S. Edgett, R.A. Yingst, M.A. Ravine, M.A. Caplinger, J.N. Maki, F.T. Ghaemi, J.A. Schaffner, J.F. Bell III, L.J.

Edwards, K.E. Herkenhoff, E. Heydari, L.C. KahMark, T. Lemmon, M.E. Minitti, T.S. Olson, et al., Curiosity’s Mars Hand Lens Imager (MAHLI) investigation. Space Sci. Rev. 170, 259–317 (2012)

European Cooperation for Space Standardization, SpaceWire—Links, nodes, routers and networks. Space

Engineering, ECSS-E-ST-50-12C, 31 July (2008) R.W. Gaskell, O.S. Barnouin-Jha, D.J. Scheeres, A.S. Konopliv, T. Mukai, S. Abe, J. Saito, M. Ishiguro, T. Kubota,

T. Hashimoto, J. Kawaguchi, M. Yoshikawa, K. Shirakawa, T. Kominato, N. Hirata, H. Demura, Characterizing and navigating small bodies with imaging data. Meteorit. Planet. Sci., 43(6), 1049–1061 (2008)

C.D. Jackman, D.S. Nelson, W.M. Owen Jr., M.W. Buie, S.A. Stern, H.A. Weaver, L.A. Young, K. Ennico, C.B.

Olkin, New Horizons Optical Navigation on Approach to Pluto. Advances in the Astronautical Sciences Guidance, Navigation and Control 157, 16-083 (2016)

J.R. Janesick, DN to λ, (SPIE Press, Bellingham, Wash., 2007), pp. 1–258 D.S. Lauretta, S.S. Balram-Knutson, E. Beshore, W.V. Boynton, C. Drouet d’Aubigny, D.N. DellaGiustina, H.L.

Enos, D.R. Gholish, C.W. Hergenrother, E.S. Howell, C.A. Johnson, E.T. Morton, M.C. Nolan, B. Rizk, H.L. Roper, A.E. Bartels, B.J. Bos, J.P. Dworkin, D.E. Highsmith, M.C. Moreau, D.A. Lorenz, L.F. Lim, R. Mink, J.A. Nuth, D.C. Reuter, A.A. Simon, E.B. Bierhaus, B.H. Bryan, R. Ballouz, O.S. Barnouin, R.P. Binzel, W.F. Bottke, V.E. Hamilton, K.J. Walsh, S.R. Chesley, P.R. Christensen, B.E. Clark, H.C. Connolly, M.K. Crombie, M.G. Daly, J.P. Emery, T.J. McCoy, J.W. McMahon, D.J. Scheeres, S. Messenger, K. Nakamura-Messenger, K. Righter, S.A. Sandford, OSIRIS-REx: Sample return from asteroid (101955) Bennu. Space Sci. Rev. 212:925 (2017).

M.C. Malin, J.F. Bell, B.A. Cantor, M.A. Caplinger, W.M. Calvin, R.T. Clancy, K.S. Edgett, L. Edwards, R.M.

Haberle, P.B. James, S.W. Lee, M.A. Ravine, P.C. Thomas, M.J. Wolff, Context Camera investigation on board the Mars Reconnaissance Orbiter. J. Geophys. Res. 112, E05S04 (2007)

C. Mario, C. Debrunner, Robustness and performance impacts of optical-based feature tracking to OSIRIS-REx

Asteroid Sample Collection Mission. 39th Annual AAS Guidance and Control Conference, 16-087, 5–10 February (2016)

Z. Milenkovich, C. D’Souza, The Space Operations Simulation Center (SOSC) and closed-loop hardware testing for

Orion rendezvous system design. AIAA Guidance, Navigation, and Control Conference, 1-16, 13–16 August (2012)

R. Olds, A. May, C. Mario, R. Hamilton, C. Debrunner, K. Anderson, The application of optical based feature tracking to OSIRIS-REx asteroid sample collection. 38th Annual AAS Guidance and Control Conference, 15-124, 30 January–4 February (2015)

OpenCV, Camera calibration and 3D construction. (Open Source Computer Vision, OpenCV.org, 2016)

http://docs.opencv.org/2.4/modules/calib3d/doc/camera_calibration_and_3d_reconstruction.html Accessed 30 December 2016

B. Rizk, et al., OCAMS: the OSIRIS-REx Camera Suite. Space Sci. Rev. (2017) M.S. Robinson, S.M. Brylow, M. Tschimmel, D. Humm, S.J. Lawrence, P.C. Thomas, B.W. Denevi, E. Bowman-

Cisneros, J. Zerr, M.A. Ravine, M.A. Caplinger, F.T. Ghaemi, J.A. Schaffner, M.C. Malin, P. Mahanti, A. Bartels, J. Anderson, T.N. Tran, E.M. Eliason, A.S. McEwen, E. Turtle, B.L. Jolliff, H. Hiesinger, Lunar Reconnaissance Orbiter Camera (LROC) instrument overview. Space Sci. Rev. 150, 81–124 (2010)

B. Williams, P. Antreasian, E. Carranza, C. Jackman, J. Leonard, D. Nelson, B. Page, D. Stanbridge, D. Wibben, K.

Williams, M. Moreau, K. Berry, K. Getzandanner, A. Liounis, A. Mashiku, D. Highsmith, B. Sutter, D.S. Lauretta, OSIRIS-REx flight dynamics and navigation design. Space Sci. Rev. (2017)