Full Terms & Conditions of access and use can be found athttp://www.tandfonline.com/action/journalInformation?journalCode=urnd20

Download by: [71.13.208.198] Date: 28 July 2017, At: 06:19

Research in Nondestructive Evaluation

ISSN: 0934-9847 (Print) 1432-2110 (Online) Journal homepage: http://www.tandfonline.com/loi/urnd20

Unmanned Aerial Vehicle (UAV)-Based Assessmentof Concrete Bridge Deck Delamination UsingThermal and Visible Camera Sensors: APreliminary Analysis

Rüdiger Escobar-Wolf, Thomas Oommen, Colin N. Brooks, Richard J. Dobson& Theresa M. Ahlborn

To cite this article: Rüdiger Escobar-Wolf, Thomas Oommen, Colin N. Brooks, Richard J. Dobson& Theresa M. Ahlborn (2017): Unmanned Aerial Vehicle (UAV)-Based Assessment of ConcreteBridge Deck Delamination Using Thermal and Visible Camera Sensors: A Preliminary Analysis,Research in Nondestructive Evaluation, DOI: 10.1080/09349847.2017.1304597

To link to this article: http://dx.doi.org/10.1080/09349847.2017.1304597

View supplementary material

Published online: 17 Apr 2017.

Submit your article to this journal

Article views: 35

View related articles

View Crossmark data

Unmanned Aerial Vehicle (UAV)-Based Assessment ofConcrete Bridge Deck Delamination Using Thermal andVisible Camera Sensors: A Preliminary AnalysisRüdiger Escobar-Wolfa, Thomas Oommena, Colin N. Brooksb, Richard J. Dobsonb,and Theresa M. Ahlborna

aDepartment of Geological and Mining Engineering and Sciences, Michigan Technological University,Houghton Michigan, USA; bMichigan Technological Research Institute, Michigan TechnologicalUniversity, Ann Arbor, Michigan, USA

ABSTRACTInfrared and visible cameras were mounted on an unmannedaerial vehicle (UAV) to image bridge deck surfaces and map likelyconcrete delaminations. The infrared sensor was first tested onlaboratory validation experiments, to assess how well it coulddetect and map delaminations under controlled conditions. Fieldtests on two bridge deck surfaces further extend the validationdataset to real-world conditions for heterogeneous concretesurfaces. Performance of the mapping instrument and algorithmswere evaluated through receiver operating characteristic (ROC)curves, giving acceptable results. To improve the performance ofthe mapping by reducing the rate of false positives, i.e., areaswrongly mapped as being affected by delamination, visibleimages were jointly analyzed with the infrared imagery. Thepotential for expanding the method to include other productsderived from the visible camera data, including high density 3Dpoint locations generated by photogrammetric methods, pro-mises to further improve the performance of the method, poten-tially making it a viable and more effective option compared toother platforms and systems for imaging bridge decks for map-ping delaminations.

KEYWORDSBridge-deck; delamination;infrared; visible; UAV

1. Introduction

Concrete surfaces need to be periodically inspected to detect deteriorationand defects. Concrete pavement surfaces, including those on bridge deckscan develop a particular type of defect, called delamination [1,2]. The pre-sence of delaminations will weaken the concrete, and eventually producespalling, leaving a hole in the concrete surface [2,3]. Common methods toidentify and map delamination areas involve both in situ and remote sensingtechniques. In situ methods, like hammer soundings and chain dragging

CONTACT Rüdiger Escobar-Wolf rpescoba@mtu.edu Department of Geological and Mining Engineeringand Sciences, Michigan Technological University, 630 Dow Environmental Sciences, 1400 Townsend Drive,Houghton, MI 49931, USA.Color versions of one or more of the figures in the article can be found online at www.tandfonline.com/urnd.

Supplemental data for this article can be accessed on the publisher’s website.

RESEARCH IN NONDESTRUCTIVE EVALUATIONhttp://dx.doi.org/10.1080/09349847.2017.1304597

© 2017 American Society for Nondestructive Testing

tests, tend to be labor intensive, inspector subjective, and expensive; alter-natively, delamination areas can be mapped using infrared remote sensing[4]. The physical principle behind this method is the difference in heatconduction between areas with delaminations and areas that do not havedelaminations [4,5]. As the concrete surface is exposed to a heat source, likethe natural sun-light, heat at the surface is transferred to the interior of theconcrete volume, whereas regions with delaminations will transfer heat lessefficiently causing the surface temperature to increase in those areas (seeFig. 1). By mapping the differences in concrete surface temperatures, andignoring other sources of variation, it is possible to infer the likely location ofdelaminations.

Temperature measurements of the concrete surface can be obtained remo-tely using an infrared thermal camera. The electromagnetic radiation emittedby a surface increases with the surface’s temperature, following the principlesof black body (or gray body, sensu stricto) radiance physics and Planck’s law[6]. For the purpose of mapping delaminations in many cases it is notnecessary to calculate actual temperatures with accuracy; having a map ofthe relative temperatures is sufficient, and this in turn can be obtained from amap of relative radiances. This approach has been applied successfully undera variety of conditions, and using different platforms to deploy thermalimaging instruments [4,7,8].

Recent studies have demonstrated several innovative applications ofUnmanned Aerial Vehicles (UAVs) to cope with disasters and conductinfrastructure and resource monitoring. For example, UAV applications forTsunami recovery [9], community-based forest monitoring [10], collectingtraffic information [11], hyperspectral terrain mapping [12], unpaved road

Figure 1. Illustration of the heat flow through the concrete body as it is heated by a sourceapplied on its surface. Areas above delaminations will heat more than adjacent areas that do nothave such defects.

2 R. ESCOBAR-WOLF ET AL.

assessment [13], and outdoor inspection of building facades [14]. Althoughthe potential applications of UAV abound, using UAVs have certain limita-tions due to the weight of the sensors and the potential flight times thathinder use in some fields [14,15].

In this study, the innovative application of deploying an infrared and opticalcamera sensors from an UAV platform to perform bridge deck inspection todetect likely delamination is presented. Compared with other platforms, UAVsoffer significant advantages for collecting infrared and optical images of con-crete surfaces on roads, bridge decks, etc. From the altitude and at the speeds atwhich UAVs can fly, a much larger area can be covered with high spatialresolution compared with the area that would be covered by the same cameramounted on a terrestrial vehicle, over the same amount of time. At the sametime, UAVs are much less expensive to operate than normal manned aircraft.New rules proposed by the U.S. Federal Aviation Administration (FAA) inFebruary 2015 for small UAVs (those under 25 kg), and a FAA commercialexemption process, make UAV deployments more feasible as an inspectiontool. To make this a viable application for bridge deck inspection, laboratoryand field testing are performed to demonstrate the capabilities of such coupledsensor-UAV systems.

The motivation to use this kind of platform is to reduce cost and time forthe data acquisition and analysis. UAVs could potentially be deployed with-out disrupting traffic, and due to the range of altitudes at which they couldfly, and depending on the sensor capabilities, bridge or road widths sectioncan be captured in a single frame, which would not be practically possible forterrestrial platforms.

2. Methods

2.1. Infrared imaging sensor

A Tau 2 uncooled core instrument manufactured by FLIR® was used as thethermal infrared imaging camera for the concrete surfaces. The Tau 2 issensitive to electromagnetic radiation in the ~8 to 15 µm spectral range, witha 336 x 256 sensor (pixels) array of uncooled VOx Microbolometers. Theexperiments were performed with a 13 mm (wide field of view) lens, result-ing in total field of view angles of 25º x 19º, with an instantaneous field ofview (per pixel) of 1.3 mrad (~ 0.074º). The Tau 2 small size (13 x 19 mm),low weight (< 70 g), and low power consumption (~1 W), is advantageouswhen used on UAV platforms. Full specifications and other details on theTau 2 thermal camera can be found on the manufacturer’s website [16].

A TeAx ThermalCapture® module was used to capture and store the Tau2 output as individual images, and stores them in a local USB memory forfurther processing and analysis. The ThermalCapture® module is specifically

RESEARCH IN NONDESTRUCTIVE EVALUATION 3

designed for the Tau 2 instruments, and was chosen because of its lightweight (45 g), small size (60 x 54 x 15 mm), and low power consumption(~2 W @ 5 V DC). The output images of the ThermalCapture module are14 bit binary files, at rates up to ~1.2 frames per second. More informationon the ThermalCapture module can be found on the manufacturer’s web-site [17].

To power the Tau 2 and ThermalCapture module, an EasyAcc® Xtra 12000mAh power bank battery was used. This battery provides 5 V DC current, atup to 2.1 A, through a USB interface. The battery is also compact (14.2 x 7.3x 2.3 cm) and adds only 320 g to the weight of the system.

The Tau 2 is not temperature calibrated, but calibrated temperatures arenot needed for mapping delamination. The sensor in the camera produces anoutput voltage that scales linearly with the radiance incident on the sensor.The voltage is digitized within the system, to produce digital count values foreach pixel in the sensor array, which are stored in the binary frame files.Post-processing of the data includes the conversion of the binary files tostandard 16 bit TIF format image files.

2.2. Optical imaging sensor

Additionally to the thermal infrared sensor, a readily available commercialoff-the-shelf (COTS) Nikon D800, digital single-lens reflex (DSLR) camerawas used to acquire high resolution imagery in the visible range of thespectrum. The camera has a full-size (FX) complementary metal oxidesemiconductor (CMOS) sensor with 36.3 megapixel (mp) resolution withframe rates of up to 4 frames per second (fps), capable of shooting TIF, JPEG,and raw image files at up to 1/8,000 second shutter speed. It weighs about 1kg, with a total of 1.5 kg with the selected Nikkor AF-S 50 mm f/1.4. The lenswas previously tested for an unpaved roads assessment study [13], as it is afast lens (one with a large aperture) capable of capturing more light thantypical zoom lenses and useful for photogrammetric data collection.

2.3. Unmanned aerial vehicle

The UAV platform used for this project was a Bergen hexacopter, a multi-rotor platform commercially available from Bergen R/C Helicopters [21]. Ithad previously been selected as a practical, relatively low cost (under US$6,000), easy-to-operate, and stable platform suitable for collecting imageryused to create 3D reconstructions from overlapping imagery with better than2.5 cm resolution [13]. It has up to 30 minutes of flight time, depending onconfiguration and weight load, and is more than capable of lifting the Nikonsensor or Tau 2 sensor, with an approximately 4 kg load limit. With twoyears of flight experience, its availability, and its known capability to lift

4 R. ESCOBAR-WOLF ET AL.

sensors with the weights needed for this project, the Bergen hexacopter wasselected as the study’s main UAV platform.

2.4. Laboratory testing and sensor validation

To assess the performance of the Tau 2 FLIR® and ThermalCapture system, aseries of controlled experiments were performed in the laboratory. The Tau 2measurements were compared to those obtained by a calibrated FLIR® SC-640 infrared camera [18], under simultaneous and identical measuring con-ditions. Delaminations were simulated by building concrete slabs withembedded pieces of Styrofoam™, at variable depths, under the assumptionthat the thermal conductivity of the Styrofoam™ is similar to that of air insidea real delamination [19]. Figure 2 shows the designs of the concrete slabswith the embedded Styrofoam™ pieces. The concrete slabs were cured for aminimum of 28 days before infrared imaging tests were performed.

Thermal imaging tests on the slab were performed by heating the concretesurface for 15 minute intervals with an infrared radiation lamp, then turningthe lamp off to allow the concrete surface to cool down while acquiring imagesover the latter time period. Thermal equilibration with the ambient temperaturewas reached about 45 minutes after turning off the heating lamp. To map thedelamination areas in the laboratory experiments, a classification of pixels intwo categories, delamination and non-delamination, was performed by thresh-old separation methods. Thresholds for the digital numbers representing thesurface radiance were chosen as percentile values from the cumulative

Figure 2. Concrete slab and simulated delaminations design [20].

RESEARCH IN NONDESTRUCTIVE EVALUATION 5

distribution. Datasets were evaluated by comparing moving pixel windows totheir local neighborhood. Classification performance levels were evaluatedthrough receiver operating characteristic (ROC) curves and parameters derivedfrom them. ROC curves describe the performance of the classification methodby graphically and quantitatively showing how the true positive rate (the ratioof correctly classified instance to the total number of classified instances) varieswith relationship to the false positive rate (the ratio of falsely classified instancesto the total number of classified instances), as the threshold value in theclassification algorithm is changed. For further details on ROC analyses, seeFawcett [20] and references therein. The Matlab® code used to do the classifica-tion is given in the Supplement.

2.5. Field tests for the UAV and sensor system

The system was tested in the field on Merriman and Stark Road overpassbridges, located on highway I-96 in Detroit, Michigan. The visible andinfrared sensors were mounted on the UAV, and flown separately over thebridge decks for data acquisition of the deck surfaces. Sensors were pointingvertically downward, and the UAV was flown at about 10 m above the bridgedeck surface. The instantaneous field of view (the pixel size at the bridge decksurface) was between 1.3 and 1.4 cm for the Tau 2 sensor, and about 0.25 cm(2.5 mm) for the Nikon D800 camera. Despite several UAV flight passesalong the length of the bridge decks in an attempt to acquire fully over-lapping thermal imagery of the bridges, the actual coverage was not completedue to a lack of real-time first-person-view of the infrared imagery acquisi-tion during the operation of the UAV. However, the overlapping thermalimagery did capture a majority of the deck surfaces at both field sites (seeFig. 3). Further improvements to the system would include such first-person-view control of the UAV, or alternatively, an automated UAV flight plan thatwould ensure complete coverage of the target surface [11]. The heat sourcefor the bridge decks was natural solar heating as it is usually the case for suchstudies [2–5,7,8].

To perform a validation test for the remote sensing data, the hammersounding technique was performed at seven locations on the Stark RoadBridge to identify and roughly delimit potential delamination areas to helpunderstand the usefulness of the nominated likely delaminations obtainedthrough the UAV-based thermal imagery data. The SC640 FLIR sensor wasalso used during the identification of delaminations to enhance the mapping.

Data processing was conducted using similar methods to those describedin Subsection 2.4 for the laboratory tests and for the infrared imagery.Additionally, the high resolution visible imagery was used to generate adigital elevation model (DEM), and corresponding orthorectified photo-graphs. The georeferenced orthophotography was used to georeference the

6 R. ESCOBAR-WOLF ET AL.

infrared images, using reflective marks and additional features that wereclearly visible in both the infrared and visible datasets, as tie-points. Thevisible imagery was used jointly with the infrared images to enhance theclassification as explained in the next section.

3. Results and discussion

3.1. Laboratory testing and sensor validation

Laboratory experiments on the concrete slabs showed that the infrared sensorcan detect the delaminations for a variety of conditions. Visual inspection ofthe infrared images revealed that the 10 x 10 cm delaminations were recog-nizable as elevated temperature regions in both the uncalibrated Tau 2 andthe temperature calibrated SC 640 FLIR sensors, but the smaller 2.5 x 2.5 cmdelaminations are not. The limits of the delamination areas on the laboratoryconcrete slabs appear sharper for shallower delamination depths, when thetemperature is higher, and become blurred when the delamination depthincreases and the temperature decreases (see Fig. 4); this is the expectedbehavior for concrete delaminations [4]. Figure 4 shows the results fordifferent delamination depths (2.5 and 5 cm) and cooling times (15 and 40minutes), for images acquired with both the SC 640 and the Tau 2 sensor.Although the SC640 sensor shows smaller noise levels compared to the Tau 2sensor, the sensitivity and overall performance of both sensors seemcomparable.

Figure 3. Infrared images (colorscale from black through red to yellow) overlaid on the highresolution orthophotos obtained via the visible camera. The Merriman Road overpass bridge isshown in (A), and the Stark Road overpass bridge is shown in (B).

RESEARCH IN NONDESTRUCTIVE EVALUATION 7

Mapping of delamination areas was performed by choosing thresholds forthe digital numbers (proxy for radiance) to classify pixels in two categories:delamination and non-delamination area. The choice of a threshold valuecan be based on the statistical properties of the pixel values dataset, and inthe case of the laboratory experiments, different percentiles of the distribu-tion of pixel values were used. After obtaining poor results by applying singlethresholds to entire image datasets, moving windows of different sizes wereapplied to the threshold values to more accurately capture the local variabilityof the data.

Because accurate dimensions of the delaminations were known from thelaboratory experiment design and construction, the results obtained from theinfrared remote sensing mapping can be compared with the actual locationand the extent of delaminations. ROC analysis was used to assess theperformance of the mapping algorithms for different test conditions (e.g.,moving window sizes, delamination depths, temperatures). To construct theROC curves, four parameters are needed, i.e., the numbers or true positives,false positives, true negatives, and false negatives. True positives are pixelsclassified as delamination by our method, that area real delaminations ascorroborated from the laboratory or field experiments. Similarly, false posi-tives are pixels classified as delaminations by the method, but which are notdelaminations in reality. True negatives correspond to pixels classified as notbeing delaminations, not being delamination piexles in reality, and falsenegatives are pixels that have been classified as not belonging to the delami-nation category, when in reality they are delaminations. Further details onhow to calculate the true positive rate, the false positive rate and their

Figure 4. Comparison between sensors. (A) and (C) are FLIR SC 640 images, and (B) and (D) areTau 2 images. The upper images (A and B) show delamination depths of 3.8 cm and coolingtimes of 40 minutes, the lower images show delamination depths of 2.5 cm and cooling times of15 minutes.

8 R. ESCOBAR-WOLF ET AL.

interpretation in the ROC diagram are given by Fawcett [20]. ROC curves fordifferent cases are shown in Fig. 5, and the results are summarized in Table 1for specific points along the curves. Under favorable conditions, e.g., shallow(2.5 cm) delaminations and short (< 30 minutes) cooling times, and using awide enough moving window (≥ 100 pixels), the algorithm performs rela-tively well, with Area Under the Curve (AUC) values > 0.8. The performancedegrades considerably when the delamination depth increases (e.g., 5 cm),the cooling times are more prolonged (> 30 minutes), and the movingwindow size decreases (e.g., 50 pixels); in such cases, the AUC values fallbelow 0.6. Table 1 and Fig. 5 also show the results for two specific thresholdvalues, a less conservative value of 50th percentile and a more conservativevalue of 90th percentile. The 50th percentile threshold results in higher falsepositive rates (FPR50), and true positive rates (TPR50), as compared with thecorresponding values for the 90th percentile threshold (FPR90 andTPR90),but has a lower accuracy (Acc50) as compared with the accuracy results ofusing the 90th percentile threshold (Acc90).

Figure 5. ROC curves for the laboratory concrete slab experiments. Delamination depths are2.5 cm for (A) and (B), and 5.1 cm for (C) and (D). Moving window sizes are 50 x 50 pixels for (A)and (C), and 150 x 150 pixels for (B) and (D). The different curve types correspond to the time inminutes for the acquisition of the thermal image, after powering off the heating lamp. Circlesand triangles in each plot mark the 50th and 90th percentile threshold points.

RESEARCH IN NONDESTRUCTIVE EVALUATION 9

The ROC analysis confirms the results from visual observation of theinfrared images. The area under the ROC curve (AUC) is a measure of thegeneral performance of the classification method, in this case whether or notthe pixels that were classified as delamination or non-delamination, were soin reality. AUC values can range from 0.5, corresponding to a completelyrandom classification without any informative power, to 1, in the case of aperfect (no false positives, nor false negatives) classification. As the depth ofdelamination becomes shallower and the experiment temperatures increase,the AUC values increase correspondingly. But the AUC also increases withthe moving window size of the classification algorithm, highlighting that the

Table 1. ROC curve parameters for the laboratory concrete slab experiments.D W T AUC FPR50 TPR50 Acc50 FPR90 TPR90 Acc902.5 50 18 0.73 0.53 0.87 0.49 0.06 0.17 0.892.5 50 28 0.70 0.54 0.82 0.49 0.07 0.18 0.882.5 50 34 0.67 0.54 0.77 0.48 0.06 0.16 0.892.5 50 41 0.64 0.53 0.72 0.48 0.07 0.15 0.882.5 100 18 0.90 0.59 0.98 0.44 0.06 0.54 0.922.5 100 28 0.88 0.60 0.97 0.43 0.06 0.52 0.912.5 100 34 0.84 0.62 0.94 0.42 0.05 0.41 0.922.5 100 41 0.78 0.61 0.89 0.42 0.06 0.33 0.912.5 150 18 0.94 0.69 0.99 0.36 0.05 0.70 0.942.5 150 28 0.93 0.68 0.99 0.36 0.07 0.74 0.922.5 150 34 0.89 0.71 0.97 0.33 0.05 0.57 0.932.5 150 41 0.80 0.71 0.95 0.33 0.06 0.37 0.913.8 50 19 0.68 0.53 0.83 0.50 0.08 0.12 0.853.8 50 25 0.64 0.53 0.76 0.49 0.08 0.12 0.853.8 50 40 0.59 0.54 0.67 0.48 0.08 0.12 0.843.8 100 19 0.82 0.58 0.91 0.46 0.07 0.47 0.883.8 100 25 0.77 0.59 0.87 0.45 0.07 0.40 0.883.8 100 40 0.68 0.60 0.78 0.43 0.09 0.31 0.863.8 150 19 0.86 0.67 0.93 0.39 0.07 0.66 0.903.8 150 25 0.81 0.67 0.90 0.38 0.07 0.55 0.903.8 150 40 0.70 0.68 0.82 0.37 0.09 0.40 0.875.1 50 7 0.62 0.53 0.71 0.49 0.08 0.14 0.855.1 50 9 0.60 0.54 0.67 0.48 0.07 0.13 0.855.1 50 15 0.59 0.54 0.65 0.48 0.08 0.13 0.855.1 50 20 0.57 0.54 0.63 0.48 0.07 0.12 0.855.1 50 23 0.56 0.54 0.62 0.48 0.08 0.13 0.855.1 100 7 0.75 0.59 0.87 0.45 0.09 0.33 0.865.1 100 9 0.72 0.60 0.84 0.44 0.07 0.29 0.875.1 100 15 0.71 0.60 0.83 0.44 0.07 0.28 0.875.1 100 20 0.69 0.61 0.80 0.43 0.06 0.24 0.885.1 100 23 0.66 0.61 0.78 0.43 0.07 0.23 0.875.1 150 7 0.76 0.66 0.91 0.39 0.10 0.39 0.865.1 150 9 0.75 0.69 0.89 0.36 0.07 0.35 0.885.1 150 15 0.75 0.69 0.88 0.36 0.07 0.36 0.885.1 150 20 0.73 0.71 0.87 0.34 0.05 0.29 0.895.1 150 23 0.69 0.70 0.84 0.35 0.07 0.26 0.87

Table 1 D is the delamination depth in cm. W is the analysis window width in pixels. T is the time in minutesof acquisition of the thermal image since the IR lamp was shut down (a proxy for cooling). AUC is the areaunder the ROC curve. FPR50, TPR50, and Accu50 are the false positive rate, true positive rate, and overallaccuracy at a threshold of 50th percentile. FPR90, TPR90, and Accu90 are the false positive rate, truepositive rate, and overall accuracy at a threshold of 90th percentile.

10 R. ESCOBAR-WOLF ET AL.

performance is not only dependent on the physical conditions of the test, butalso on the choice of the algorithm parameters during the analysis phase.

3.2. Field tests for the UAV and sensor system

The field test results from Merriman and Stark Roads further validate themethod, showing overall good agreement with independent determinationsof delamination areas. The thermographic analysis was performed on thearea where thermal images were available (see Fig. 3) and which covers mostof the bridge decks in both cases. Due to the rather limited extent and poorresolution (defining the exact boundary of delamination areas) of the ham-mer soundings, the validation of field experiments cannot be conducted asrigorously as it was with the delaminations in the laboratory concrete slabs;concrete coring would be needed for this, which was not available for thisproject due to an impending deck repair project. However, a comparisonbetween the infrared-based delamination mapping, the hammer sounding,and SC640-based mapping is still possible. The ROC curves and relatedparameters were obtained for six areas on the Stark Road Bridge deck, forwhich overlapping infrared imagery and hammer sounding data were avail-able, as shown in Fig. 6 and Table 2. These show a good degree of agreementbetween the field identified areas (via ground hammer sounding and SC649imaging), and the UAV infrared image–based mapping. The AUC values arevery high (> 0.95) for all tested cases, in part because the validation set is

Figure 6. ROC results for the field tests at the Merriman and Stark Road bridges. The six testcurves are tightly bundled and show a very good performance. Circles and triangles in each plotmark the 50th and 90th percentile threshold points.

RESEARCH IN NONDESTRUCTIVE EVALUATION 11

small and consists of very obvious delamination areas; this may indicate abias in the sampling, and therefore, the apparently good performance has tobe interpreted in that context.

The concrete bridge deck surfaces show a much more complex structure inthe infrared images than the concrete slabs tested in the laboratory. A patchyappearance is caused at least in part by the pattern of older vs. newer patchesof concrete from repairs performed over time (see Fig. 3). Although it isassumed that changes in the concrete surface radiance in an infrared imageare due to difference in the surface temperature, the differences can also bedue to changes in the emissivity of the material’s surface [4,8], and in thatcase the validity of the thermal imaging method for mapping delaminationsmay be compromised. This makes the classification more challenging.However, results are improved by combining the infrared and visible imagerythrough the methods described below.

3.3. Joint infrared and visible imagery analysis

One aim of the pixel classification method is to separate variations inemissivity from actual delaminations. To improve the classification method,visible imagery is included in the analysis. The Nikon D800 sensor detectsradiation in the visible part of the spectrum, specifically in three bands: red,green, and blue (RGB). The color of concrete is usually close to gray, whichresults in a high correlation of the RGB values, and hence a high informationredundancy in those bands; therefore, the three visible bands are joined intoa single averaged visible band.

The visible band is not sensitive to the delaminations, as is the case forthe infrared band, and can be used to detect potential false positives.However, changes in the emissivity due to changes in the concrete surfacematerial (e.g., patches of different concrete) are likely to show up in thevisible bands. Therefore, cases of anomalies detected in the thermal andvisible bands may be due to changes in the surface material properties andnot due to delaminations, while anomalies detected in the infrared band,but not on the visible bands, are much more likely to be caused by actualdelaminations.

Table 2. ROC parameter results for tests at the Merriman and Stark Road bridges.Point AUC FPR50 TPR50 Acc50 FPR90 TPR90 Acc901 0.97 0.39 0.99 0.62 0.03 0.79 0.972 0.96 0.40 1.00 0.61 0.04 0.68 0.953 0.98 0.40 1.00 0.60 0.05 0.91 0.955 0.98 0.45 1.00 0.55 0.06 0.97 0.946 0.97 0.45 1.00 0.55 0.04 0.79 0.957 0.96 0.43 0.99 0.58 0.05 0.82 0.95

Table 2 Nomenclature is the same as in Table 1.

12 R. ESCOBAR-WOLF ET AL.

Figure 7 illustrates the rationale behind the mapping algorithm thatincorporates the visible bands in the analysis. Figure 7a shows the visibleimage and Fig. 7b shows the infrared image of the same area. First, areas ofelevated temperature in the infrared image are flagged as potential delamina-tion regions (Fig. 7c). Second, areas of potentially different emissivity aremapped in the visible band, by applying a moving window method to high-light pixels with values 1.5 times the standard deviation above the mean value(Fig. 7d). Finally, the potential delamination areas obtained from the infraredband is compared with areas that may have different emissivity, and obtainedfrom the visible band. Only pixels for which there appears to be no emissivitychanges are classified as potential delamination areas (Fig. 7e).

The method just described was applied to the concrete bridge deck forwhich there was thermal image coverage (almost the entire bridge deck),followed by post-processing of the mapped areas, including a low-pass

Figure 7. Illustration of the algorithm for detecting delaminations combining infrared andthermal imager. See text for details.

RESEARCH IN NONDESTRUCTIVE EVALUATION 13

filtering, adjacency connectedness, and minimum area filtering of the classi-fied pixels to discard individual isolated pixels. The results are shown inFig. 8. The area covered by thermal and visible imagery is 968 m2, and 14 m2

were classified as delamination areas with this method, which is equivalent to1.5% of the total area imaged with the infrared and visible sensors on thatparticular concrete bridge deck. The Matlab® code used in the pixel classifi-cation is given in the supplemental file.

Further work will be necessary to develop the method in more depth.Controlled experiments using different types of surfaces and delaminationscould be used to test how well the joint thermal-optical method works. Theeffects of heating cycles due to daily insolation variations could also be explored,as this may be important for the application of the method. The method couldalso be compared to other techniques in the field (e.g., high speed deck vehiclescanners, GPR, etc.). The comparison to GPR could provide some insight into themethod, but the balance between the likely higher precision of the GPR versus thepotentially less expensive acquisition, operation, and analysis cost of the thermalUAV would have to be weighed. More sophisticated analysis algorithms, includ-ing machine learning and other data processing methods, could further improvethe delamination detection analysis.

4. Conclusions

The laboratory and field experiments using small infrared and DSLR visiblecameras mounted on a UAV show that the system can be effectively used tomap delamination areas on bridge decks. Laboratory tests were used tovalidate the detection of delaminations from the images collected with asmall infrared sensor that can easily be deployed on UAVs. The field testson two concrete bridge decks were further used as a validation set, under

Figure 8. Final results of the delamination mapping method. Delaminations are show by bluepolygons on the visible (a) and infrared (b) imagery.

14 R. ESCOBAR-WOLF ET AL.

more challenging but realistic conditions of heterogeneous concrete surfaces,at the larger scale of a real-world transportation infrastructure case. Theperformance of the infrared delamination mapping system can be improvedby including visible imagery to reduce the rate of potential false positives, i.e.,areas that are falsely mapped as delaminations but have a different surfaceemissivity.

The UAV platform offers advantages, including maneuverability andlarger field of view per image, compared to traditional data acquisitionplatforms for delamination mapping. However, several challenges must beovercome to make this method fully operational for delamination mappingpurposes. Further testing and validation is necessary to build more robustmapping algorithms. With the exception of the minimum area filteringcompleted in the post-processing of the images, all the applied analysismethods were pixel-based and have the disadvantage of taking intoaccount the spatial context in a marginal way, ignoring important char-acteristics, like the shape and size of the mapped delaminations. Adaptingthe method to include such spatially contextual information could improveeven more the performance. Although the visible bands were included inthe mapping algorithm, other products derived from the visible photogra-phy, including 3D photogrammetric products, like digital elevation modelsof the deck surface, would potentially improve the delamination mappingmethod.

Acknowledgments

Comments by two anonymous reviewers greatly contributed to enhance the quality of thispaper.

Funding

This work was possible through funding by the Michigan Department of Transportationunder contract number 2013-0067, project number Authorization No. 1, OR number OR13-008.

References

1. Guettala, A. and A. Abibsi. Materials and Structures 39(4):439–446 (2006).2. Masser, K. and W. Roddis. Journal of Transportation Engineering 116(5):583–601

(1990).3. Masliwec, T. Proceedings SPIE 0934, Thermosense X: Thermal Infrared Sensing for

Diagnostics and Control, 19. Orlando Technical Symposium. doi:10.1117/12.968482(1988).

4. Washer, G., R. Fenwick, and N. Bolleni. Development of hand-held thermographicinspection technologies. Missouri Department of Transportation. OrganizationalResults Research Report. (2009).

RESEARCH IN NONDESTRUCTIVE EVALUATION 15

5. Manning, D. and F. Holt. Concrete International 2(11):34–42 (1980).6. Jense, J. Remote Sensing of the Environment: An Earth Resource Perspective. 2nd

Edition. Prentice-Hall (2007). Upper Saddle River, NJ, USA.7. Vaghefi, K., R. Oats, D. Harris, T. Ahlborn, C. Brooks, K. Endsley, C. Roussi, R.

Shuchman, J. Burns, and R. Dobson. Journal of Bridge Engineering 17:886–895 (2011).8. Vaghefi, K., T. Ahlborn, D. Harris, and C. Brooks. Journal of Performance of

Constructed Facilities 29(4):04014102-1-8. doi: 10.1061/(ASCE)CF.1943-5509.0000465(2013).

9. Murphy, R., K. L. Dreger, S. Newsome, J. Rodocker, B. Slaughter, R. Smith, E. Steimle,T. Kimura, K. Makabe, K. Kon, H. Mizumoto, M. Hatayama, F. Matsuno, S. Tadokoro,and O. Kawase. Journal of Field Robotics 29(5):819–831 (2012).

10. Paneque-Gálvez, J., M. McCall, B. Napoletano, S. Wich, and L. Koh. Forests 5:1481–1507 (2014).

11. Liu, X. F., Z. R. Peng, L. Y. Zhang, and L. Li. J. Transport. Syst. Eng. Inform. Technol. 12(1):91–97 (2012).

12. Lucieer, A., S. Robinson, D. Turner, S. Harwin, and J. Kelcey. International Archives ofthe Photogrammetry, Remote Sensing and Spatial Information Sciences XXXIX-B1:429–433 (2012).

13. Dobson, R. J., T. Colling, C. Brooks, C. Roussi, M. K. Watkins, and D. Dean.Transportation Research Record: Journal of the Transportation Research Board 2433(1):108–115 (2014).

14. Roca, D., S. Lagüela, L. Díaz-Vilariño, J. Armesto, and P. Arias. Automation inConstruction 36:128–135 (2013).

15. Liu, P., A. Chen, Y. Huang, J. Han, J. Lai, S. Kang, T. Wu, M. Wen, and M. Tsai. SmartStructures and Systems 13(6):1065–1094 (2014).

16. FLIR. Tau 2 Technical Specifications. http://www.flir.com/cvs/cores/view/?id=54717(last accessed February 2015).

17. TeAx. ThermalCapture Technical Specifications http://www.teaxtec.de/index.php?id=23&L=1 (last accessed February 2015).

18. FLIR. SC 640 Technical Specifications. http://www.flir.com/uploadedFiles/Thermography_APAC/Products/Product_Literture/SC640_Datasheet%20APAC.pdf(last accessed February 2015).

19. Vaghefi, K. , PhD dissertation. Michigan Technological University, 156 pp (2013).20. Fawcett, T. Pattern Recognition Letters 27:861–874 (2006).21. Bergen R/C Helicopters. http://www.bergenrc.com/Multi.php (last accessed March

2015).

16 R. ESCOBAR-WOLF ET AL.