The Hough Transform for Vertical Object Recognition in 3D Images Generated from Airborne Lidar Data

Christopher Parrish

ECE533 Project December 2006

1

Introduction An important responsibility of NOAA’s National Geodetic Survey (NGS) is to conduct airport obstruction surveys in support of the Federal Aviation Administration (FAA). The primary objective in an airport obstruction survey is to accurately geolocate vertical objects, such as trees, buildings, towers, and poles, on and around airfields and in the approach paths. Any object that penetrates the FAA Obstruction Identification Surfaces (depicted in Figure 1) is, by definition, an airport obstruction that must be accurately surveyed and reported to the FAA.

Figure 1: The Obstruction Identification Surfaces (OIS ) are mathematically-defined 3D surfaces enveloping the airfield and approach paths. The left image, adapted with permission from U.S. Department of Transportation (1993) is a schematic diagram of the OIS, while the right image depicts a computer-generated model of the OIS (the blue-colored surfaces) for an actual airport overlaid on a digital orthophoto and digital elevation model (DEM). By definition, an obs truction is any object that penetrates (i.e., is of greater height than) the OIS, such as the group of trees near the center of the right image.

Traditional methods of conducting airport obstruction surveys have relied heavily on field work. Since the 1940s, NGS and its contractors have performed over 6,500 airport obstruction surveys using a combination of photogrammetric and conventional field survey techniques, which have been proven to yield very accurate, reliable data (Parrish et al., 2005; Tuell, 1987). However, these methods are also time consuming and expensive, which has led to increasing interest, particularly within the private sector, in using emerging remote sensing technologies to increase efficiency and reduce costs. One airborne remote sensing technology that appears particularly well suited to airport obstruction surveying is light detection and ranging (lidar). The basic principles of this technology are illustrated in Figure 2. The primary components of the airborne system include a laser, scanning mechanism, receiver, and integrated GPS and inertial measurement unit (IMU). Ranges are accurately computed from the round-trip travel time of laser pulses that are reflected from the earth’s surface (specifically, from elevated features, such tree canopy or buildings, and/or from the ground) and received back at the

2

sensor. The scanning mechanism is used to create a swath on the ground. By combining the laser ranges with scanner angle data and post-processed position and orientation data from the integrated GPS and IMU system, accurate X,Y,Z coordinates of terrain and elevated features in the mapping frame can be computed in a highly automated manner. Because the raw data consist of irregularly-spaced points in 3D space indicating the locations of various laser reflections, the term “point cloud” is often used to describe these data (Fowler, 2001).

Figure 2: Illustration of airborne lidar principles.

Previous studies have demonstrated that airborne lidar can be used to meet certain airport obstruction surveying standards (e.g., Parrish et al., 2005; Tuell, 2002). However, several problems remain unsolved. One current challenge involves automatic extraction of vertical features of interest, such as trees, towers, poles and stacks, from the lidar data. In this project, a method of using the Hough transform for vertical object recognition in 3D images generated from the airborne lidar data is investigated. Approach The approach investigated in this project is depicted in Figure 3. The first step is to grid (or “voxelize”) the lidar point cloud to create a 3D grayscale intensity image

3

(specifically, regularly-spaced 3D grids of intensity values, using 8 bits per voxel). Next, edge detection is performed on this 3D image using a 3D Sobel operator. Threshold segmentation is then performed to output a 3D binary edge image. Using the assumption that the vertical objects of interest can be reasonably well modeled as vertical cylinders, the next step is to perform a Hough transform to identify vertical cylinders in the 3D binary edge image. If the approach is successful, the major vertical cylinders (i.e., those receiving the most votes in the Hough transform algorithm) should correspond to the vertical objects of interest. The last three steps, beginning with computing the 3D grayscale edge image, are explained in greater detail below.

The gradient of a 3D image, f(x,y,z), is given by

[ ]T

Tzyx z

fyf

xf

GGG

∂∂

∂∂

∂∂

==∇f (1)

And the magnitude of the gradient is given by:

222zyx GGGf ++=∇=∇ f (2)

Figure 3: The major steps in the Hough transform-based approach investigated in this project to identify vertical cylinders in the lidar data.

4

One common form of the 3D Sobel operator (e.g., Spies and Barron, 2004) approximates the first derivatives Gx, Gy, and Gz using the 333 ×× filters given in Equation (3). Here, each filter is expressed as three 2D matrices, which, when stacked vertically, form the

333 ×× filter.

−−−

−−−

−−−

242000

242

,484000

484

,242000

242

:xG

−−

−

−−

−

−−

−

202404

202

,404808

404

,202404

202

:yG (3)

−−−−−−

−−−

242484

242

,000000

000

,242484

242

:zG

It should be noted that the coefficients in Equations (3) sum to zero, so that the output in sub-volumes of constant intensity is zero, as expected with a derivative operator. After applying the 3D Sobel operator and computing the magnitude of the gradient from Equation (2), the next step is to threshold the output. The thresholded (binary) image, g(x,y,z), is given by

>∇

=otherwise0 if 1

),,(Tf(x,y,z)

zyxg (4)

The particular 3D intensity images used in this project were fairly sparse and contained relatively little noise, so it was decided to set the threshold, T, to zero. In more complicated situations, the threshold segmentation could be formulated as a Bayesian hypothesis testing problem and the threshold determined using the maximum a posteriori probability (MAP) decision rule, as described in Kay (1998) and Gonzalez and Woods (2002). Alternately, an adaptive thresholding approach could be used. The next step is to detect cylinders in the binary edge image using a Hough transform. In general, five parameters are needed to describe a cylinder: one for the radius and four for the position and orientation of the axis (Beder and Förstner, 2006; Vosselman et al., 2004; Rabbani and van den Heuvel, 2005). However, the assumption that the cylinders are vertical (i.e., their axes are parallel to the Z-axis of the mapping frame) reduces the dimension of the parameter space to three. Specifically, a vertical cylinder can be represented as follows (Beder and Förstner, 2006):

5

222 )()( rtYsX =−+− (5) where the parameters s and t describe the (X,Y) location of the cylinder axis and r denotes the radius. The projection of a vertical cylinder onto a horizontal plane is a circle; thus, Equation (5) has the same form as the equation used to detect circles in a 2D image. The difference is that, in attempting to detect vertical cylinders in a 3D image, all coordinate triplets (Y,Y,Z) corresponding to edges (nonzero entries in the 3D binary edge image) are considered. The steps in my algorithm for finding vertical cylinders using the Hough transform are:

1. Quantize the 3D parameter space. 2. Initialize all accumulator cells in the 3D parameter space to zero. 3. For each nonzero voxel in the 3D binary edge image, step through all values of s

and t. At each location:

a. Solve Equation (5) for r b. Round r to its nearest accumulator cell value c. Increment the counter for that (s,t,r) accumulator cell.

4. Find the entry in the 3D accumulator array with the highest number of votes. If

the approach is successful, the parameters (s,t,r) of this entry should describe the cylinder corresponding to the vertical object of interest.

Work Performed The work carried out in this project entailed writing MATLAB code to perform each of the steps depicted in Figure 3 and then testing the approach on actual lidar data for two real objects. Following the adage that “one must walk before one can run,” the two objects selected for this project were specifically chosen based on the fact that they appear to be fairly well modeled as vertical cylinders, thereby increasing the probability of success. Figure 4 shows the two objects selected, which are both stacks of different heights. In examining Figure 4, two concerns are immediately apparent: 1) the objects of interest are not perfectly cylindrical; they are both slightly wider at the base than at the top; and 2) the lidar data are relatively sparse. As will be demonstrated in the Results section of this report, the Hough transform approach appears to work well with the sparse data and to be robust against slight discrepancies between the actual shapes of the objects and their parametric representations.

6

Figure 4: Vertical objects used in this project. The two leftmost images are 2D color images of the objects taken with an ordinary hand-held digital camera. The middle images are vertical slices through the 3D point clouds generated from the airborne lidar system. On the right are slices through the 3D grayscale images generated by voxelizing the lidar point clouds.

Following the procedure outlined in Figure 3, the first step entailed voxelizing the lidar point cloud to create 3D grayscale intensity images. The program voxelize_lidar.m (contained in the Appendix) performs this step. The voxelization procedure can be envisioned as draping a fine 3D grid mesh over the raw lidar point cloud (Figure 5). If a particular voxel in this 3D grid contains a point from the lidar point cloud, that point’s intensity value is assigned to the cell; otherwise, the cell is given the default value of zero. With a sufficiently fine 3D grid (i.e., greater than spatial resolution than the raw data), the probability of multiple input points within the same grid cell becomes negligibly small, so that this case need not be explicitly addressed. The two rightmost images in Figure 4 show horizontal and vertical slices through the 3D grayscale intensity images output from the voxelization program.

7

Figure 5: Conceptualization of the fine 3D grid mesh used in the voxelization procedure. Here the blue dots represent individual points in the lidar point cloud.

The next step is to generate the 3D binary edge images. The MATLAB program detect_edges (see Appendix) uses Equations (2) and (3) to compute the gradient of a 3D lidar intensity image. It then thresholds the output using Equation (4) to obtain the 3D binary edge image. Figure 6 shows the output.

Figure 6: 3D binary edge images output from the program detect_edges.m .

8

The final step is, of course, to perform the Hough transform to identify vertical cylinders in the 3D binary edge images. The program detect_cylinders.m (contained in the Appendix) implements the algorithm described in the Approach section above. Based on the sizes of the 3D images and the cylinders to be detected, the following ranges for each of the three parameters were used: [ ]45,1, ∈ts and [ ]4,1∈r . After some experimentation, the accumulator array cell size was set to 0.5 m in all three dimensions. Results Figures 7 and 8 show the vertical cylinders (in red) that received the most votes in the Hough transform program. (The images shown in these two figures are the same 3D binary edge images depicted in Figure 6; colors have been added simply to improve the visual display.) Qualitative analysis performed by visually inspecting the graphical output of the Hough transform cylinder detection program indicates that both the locations and radii of the detected cylinders appear correct. To quantify these results, the cylinder axes locations and radii were compared against data acquired using field survey and photogrammetric techniques (Table 1).

Figure 7: The cylinder (shown here in red) corresponding to vertical object “CB25” detected using the Hough transform.

9

Figure 8: Cylinder corresponding to vertical object “FID19” detected using Hough transform.

Table 1: Comparison of radii and axes location of HT detected cylinders with surveyed positions and radii.

Surveyed X coordinate (UTM Easting in m)

Surveyed Y coordinate (UTM Northing in m)

Surveyed radius (m)

HT detected cylinder X coordinate

HT detected cylinder Y coordinate

HT detected cylinder radius ∆X (m) ∆Y (m) ∆r (m)

FID19 304193.2 4771463.6 3.5 304194.0 4771464.0 3.0 0.8 0.4 0.5

CB25 302421.5 4771325.2 2.6 302422.0 4771325.0 2.0 0.5 0.2 0.6 The surveyed X and Y coordinates in Table 1 are UTM (NAD83) Zone 16N Eastings and Northings, respectively. The survey data were acquired using post-processed differential GPS. The radii of the stacks were measured from a digital orthophoto with 0.30 m spatial resolution. Given the spatial resolution of the orthophoto and the fact that, as noted previously, the stacks are both narrower at the top than at the base, the accuracy of these measured radii is almost certainly poorer than the accuracy of the surveyed X,Y axes

10

locations, but was considered sufficient for evaluating the performance of the Hough transform cylinder detection algorithm. The results in Table 1 show that the axes locations of the cylinders detected using the Hough transform algorithm were within 1 m of the GPS survey locations, which exceeds even the most stringent spatial accuracy requirements for airport obstruction data (see U.S. Department of Transportation, 1993). Additionally, the radii of the cylinders output from the program are very close to the radii measured from the orthophoto (within 0.6 m). Discussion The results obtained for the two stacks are quite promising. The Hough transform cylinder detection program detected both objects very close to their surveyed locations, despite the two challenges mentioned earlier: specifically, the fairly sparse lidar point data and the fact that the stacks are not perfectly cylindrical. However, there are additional considerations that must be taken into account in evaluating the utility of the algorithm for the intended application of lidar airport obstruction surveying. These considerations include computational costs and storage requirements and the ability to detect vertical objects that are not as well modeled as vertical cylinders as those shown in Figure 4. The computational costs of the algorithms developed in this project increase with the size of the input 3D lidar intensity image, which impacts both the number of edge voxels and the ranges of the translation parameters, s and t. It is also important to note that both the storage space requirements and computational complexity of the Hough transform algorithm are exponential in the number of parameters, three in this case. (The complexity of the Hough transform algorithm is discussed in, e.g., Trucco and Verri (1998).) In this project the computational costs were kept relatively low simply by using small input images created from small subsets of the original lidar data set and small ranges for all three parameters. However, increasing the number of parameters past three (e.g., using a more complicated parametric representation of the vertical objects of interest) or using very large input 3D images could quickly become very computationally expensive, possibly limiting the utility of this approach. The quantization of parameter space in the Hough transform algorithm affects both the computational complexity and the accuracy of the detected cylinders, so a tradeoff is involved. Too fine a quantization of parameter space leads to excessive computational costs. On the other hand, coarse quantization leads to poor result s. Figure 9 shows an example of a cylinder detected in an incorrect location, due to poor selection of the parameter ranges and quantization intervals.

11

Figure 9: Example of poor results obtained when the quantization intervals and parameter ranges in the Hough transform program are not chosen carefully. Here, the (X ,Y) location of the axis of the detected cylinder differs from the surveyed location by approximately 4 m and the radius differs from the measured value by 1.4 m.

The final consideration is how well the algorithm handles objects that are not well modeled as vertical cylinders. Figure 10 shows three examples of “non-cylindrical” vertical objects. Initial results indicate that the Hough transform-based cylinder detection program developed in this project does, in fact, have more difficulty detecting the tree shown in Figure 10 than the stacks shown in Figure 4; however, the situation is also complicated by the fact that there are other trees of similar height located within 13 m of this tree.

Figure 10: Examples of non-cylindrical vertical objects.

12

Two possible variations on the algorithms developed in this project that might improve the results for non-cylindrical objects are: 1) representing vertical objects of interest as cones, since the objects shown in Figure 10 are arguably better modeled as cones than cylinders; or 2) utilizing of a Generalized Hough Transform (GHT) approach. The GHT differs from the classical Hough transform in that it does not require a parametric representation of the objects to be detected, thereby allowing detection of arbitrary shapes. In lieu of parametric equations, prototype target objects are used to construct look-up tables in which the shape is described in terms of distances and angles from an arbitrary reference point (e.g.,Vernon, 1991). However, both the GHT approach and cone representation approach suffer the same drawback: high computational costs. The GHT is computationally expensive because it must examine all possible positions and orientations of each target object (of which there could be several) within the 3D image. The cone representation is computationally expensive because at least one additional parameter is needed (for example, even if the cone height is assumed known, the base elevation could still vary), thereby increasing the dimensionality of the Hough parameter space. Due to these challenges, future research is recommended to further investigate the extension of the approach to handle non-cylindrical vertical objects.

13

Appendix

MATLAB Code

14

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % subset_point_cloud % % % % For ECE533 class project. Subsets raw lidar point cloud (specifically % % 9-column Optech ALL files) based on a defined area of interest (AOI) % % specified in terms of center Easting and Northing and box size. % % % % C. Parrish % % Created: 11/21/2006 % %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% clear all; close all; clc; % Adjustable parameters %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% allFile{1,:} = 'H:\17506\campus_fullres_realm_out\Str_1.all'; allFile{2,:} = 'H:\17506\campus_fullres_realm_out\Str_10.all'; allFile{3,:} = 'H:\17506\campus_fullres_realm_out\Str_11.all'; allFile{4,:} = 'H:\17506\campus_fullres_realm_out\Str_12.all'; allFile{5,:} = 'H:\17506\campus_fullres_realm_out\Str_13.all'; allFile{6,:} = 'H:\17506\campus_fullres_realm_out\Str_14.all'; allFile{7,:} = 'H:\17506\campus_fullres_realm_out\Str_9.all'; outputFilename = strcat('D:\01Chris\University of Wisconsin\ECE_533\'... ,'Project\Lidar Subsets\FID19.txt'); centerEasting = 304191.96; % UTM Easting of center of AOI centerNorthing = 4771459.45; % UTM Northing of center of AOI boxSize = 44; % Size of box in meters %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % Compute bounding coordinates of box centerEasting = round(centerEasting); centerNorthing = round(centerNorthing); maxNorthing = centerNorthing + boxSize/2; minNorthing = centerNorthing - boxSize/2; maxEasting = centerEasting + boxSize/2; minEasting = centerEasting - boxSize/2; % Open the output file for appending fidOut = fopen(outputFilename,'a'); % Read through the input ALL files and write to the output file those % points within the AOI for i = 1:size(allFile,1) fidIn = fopen(allFile{i,:}, 'r'); if fidIn == -1 error(strcat(allFile{i,:},' could not be opened.')) end while ~feof(fidIn) line = fgetl(fidIn); B = sscanf(line, '%f %f %f %f %f %f %f %f %f'); if (B(2) > minEasting) && (B(2) < maxEasting) if (B(3) > minNorthing) && (B(3) < maxNorthing) fprintf(fidOut,'%s \n',line); end end end fclose(fidIn); end fclose(fidOut); disp('Done-- output file has been created!');

15

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % voxelize_lidar % % % % For ECE533 project. This program will voxelize an input lidar point % % cloud (in Optech ALL format) to create a 3D image of intensity values. % % It will output a MAT file containing: 1) the 3D image, 2) an array % % specifying the min easting, min northing, and min elevation; and 3) % % an array specifying the grid sizes. % % % % C. Parrish % % Created: 11/29/2006 % %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% close all; clear all; clc; % Adjustable parameters %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% inputFilename = strcat('D:\01Chris\University of Wisconsin\ECE_533\'... ,'Project\Lidar Subsets\FID19.txt'); outputFilename = strcat('D:\01Chris\University of Wisconsin\ECE_533\'... ,'Project\3D Images\FID19_1m_1m_1m.mat'); gridSizeX = 1.0; % Easting grid size (in meters) gridSizeY = 1.0; % Northing grid size (in meters) gridSizeZ = 1.0; % Elev. grid size (in meters) %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% fid = fopen(inputFilename, 'r'); if fid == -1 error('Lidar point cloud file could not be opened.') end % First read through the input file to determine the bounding coordinates % for the grid minEasting = inf; maxEasting = -inf; minNorthing = inf; maxNorthing = -inf; minElev = inf; maxElev = -inf; while ~feof(fid) line = fgetl(fid); B = sscanf(line, '%f %f %f %f %f %f %f %f %f'); if numel(B) < 9 % For some reason, there's no 2nd return B(9) = B(5); B(8) = B(5); B(7) = B(4); B(6) = B(3); B(5) = B(2); end if (max(B(2),B(5)) > maxEasting) maxEasting = max(B(2),B(5)); end if (min(B(2),B(5)) < minEasting) minEasting = min(B(2),B(5)); end if (max(B(3),B(6)) > maxNorthing) maxNorthing = max(B(3),B(6)); end if (min(B(3),B(6)) < minNorthing) minNorthing = min(B(3),B(6)); end if (max(B(4),B(7)) > maxElev) maxElev = max(B(4),B(7)); end

16

if (min(B(4),B(7)) < minElev) minElev = min(B(4),B(7)); end end minEasting = floor(minEasting); minNorthing = floor(minNorthing); minElev = floor(minElev); % Note: It is preferable for the number of cells in each direction to be a % multiple of 2. numCellsX = ceil((maxEasting - minEasting)/gridSizeX); if (mod(numCellsX,2) ~= 0) numCellsX = numCellsX + 1; end numCellsY = ceil((maxNorthing - minNorthing)/gridSizeY); if (mod(numCellsY,2) ~= 0) numCellsY = numCellsY + 1; end numCellsZ = ceil((maxElev - minElev)/gridSizeZ) + 1; if (mod(numCellsZ,2) ~= 0) numCellsZ = numCellsZ + 1; end maxEasting = minEasting + numCellsX*gridSizeX; maxNorthing = minNorthing + numCellsY*gridSizeY; maxElev = minElev + numCellsZ*gridSizeZ; % Now initialize a 3D array to hold the intensity values % intensity_grid = zeros(numCellsX,numCellsY,numCellsZ,'uint8'); lidar3dImage = zeros(numCellsX,numCellsY,numCellsZ); % Reset the file position indicator to the beginning of the file. Then % begin reading through the data file and assigning intensity values to % grid cells. status = fseek(fid,0,'bof'); while ~feof(fid) line = fgetl(fid); B = sscanf(line, '%f %f %f %f %f %f %f %f %f'); % First use the 1st return easting1 = B(2); northing1 = B(3); elev1 = B(4); if numel(B) == 5 intensity1 = B(5); else intensity1 = B(8); end xIndx = round(((easting1 + gridSizeX/2) - minEasting)/gridSizeX); yIndx = round(((northing1 + gridSizeY/2) - minNorthing)/gridSizeY); zIndx = round(((elev1 + gridSizeZ/2) - minElev)/gridSizeZ); % If the intensity grid cell corresponding to the current point is % empty, write the current intensity value to that cell. Otherwise, % average the current intensity and the existing intensity value at % that cell. May want to make this more sophisticated later. if (lidar3dImage(xIndx,yIndx,zIndx) == 0) lidar3dImage(xIndx,yIndx,zIndx) = intensity1; else lidar3dImage(xIndx,yIndx,zIndx) = mean(intensity1, ... lidar3dImage(xIndx,yIndx,zIndx));

17

end % Now use the 2nd return if numel(B) < 9 % For some reason, there's no 2nd return continue end easting2 = B(5); northing2 = B(6); elev2 = B(7); intensity2 = B(9); xIndx = round(((easting2 + gridSizeX/2) - minEasting)/gridSizeX); yIndx = round(((northing2 + gridSizeY/2) - minNorthing)/gridSizeY); zIndx = round(((elev2 + gridSizeZ/2) - minElev)/gridSizeZ); % If the intensity grid cell corresponding to the current point is % empty, write the current intensity value to that cell. Otherwise, % average the current intensity and the existing intensity value at % that cell. May want to make this more sophisticated later. if (lidar3dImage(xIndx,yIndx,zIndx) == 0) lidar3dImage(xIndx,yIndx,zIndx) = intensity2; else lidar3dImage(xIndx,yIndx,zIndx) = mean(intensity2, ... lidar3dImage(xIndx,yIndx,zIndx)); end end minCoords = [minEasting minNorthing minElev]; gridSize = [gridSizeX gridSizeY gridSizeZ]; % Now create the output MAT file save(outputFilename,'lidar3dImage','minCoords','gridSize') % Close the input file fclose(fid); disp('Done-- output file has been created!') %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % detect_edges % % % % For ECE533 project. The input is a 3D lidar intensity image (contained % % in the input MAT file). A 3D grayscale strength-of-edge image is % % computed using a 3D Sobel operator and then thresholded to produce a % % 3D binary edge image that can be used in the Hough transform program % % for finding vertical cylinders. The output MAT file will contain: % % 1) the original 3D grayscale intensity image; 2) the 3D binary edge % % image; 3) an array specifying the minimum coordinates (UTM Easting, % % Northing, and elevation) for the image. % % % % C. Parrish % % Created: 12/2/2006 % % Modified: % %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% close all; clear all; clc; % Adjustable parameters %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% inputFile = strcat('D:\01Chris\University of Wisconsin\ECE_533\', ... 'Project\3D Images\FID19_1m_1m_1m.mat'); outputFile = strcat('D:\01Chris\University of Wisconsin\ECE_533\', ... 'Project\3D Binary Edge Images\FID19_bin_edges.mat'); thresh = 0; % Strength-of-edge threshold

18

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% load(inputFile); % This will load the following variables: % lidar3dImage = 3D grayscale intensity image % gridSize = 3x1 vector specifiying grid sizes % minCoords = 3x1 vector specifying minimum % coordinates (Easting,Northing,elev) % First create the three 3D Sobel filters following Equation (3) of my % written report Gx(1,:,:) = [2 4 2 4 8 4 2 4 2]; Gx(2,:,:) = [0 0 0 0 0 0 0 0 0]; Gx(3,:,:) = [-2 -4 -2 -4 -8 -4 -2 -4 -2]; Gy(:,1,:) = [2 4 2 4 8 4 2 4 2]; Gy(:,2,:) = [0 0 0 0 0 0 0 0 0]; Gy(:,3,:) = [-2 -4 -2 -4 -8 -4 -2 -4 -2]; Gz(:,:,1) = [2 4 2 4 8 4 2 4 2]; Gz(:,:,2) = [0 0 0 0 0 0 0 0 0]; Gz(:,:,3) = [-2 -4 -2 -4 -8 -4 -2 -4 -2]; % compute the gradient x, y, and z components bx = imfilter(lidar3dImage,Gx,'replicate'); by = imfilter(lidar3dImage,Gy,'replicate'); bz = imfilter(lidar3dImage,Gz,'replicate'); % Compute the magnitude of the gradient to generate a grayscale % strength-of-edge image, b b = sqrt(bx.*bx + by.*by + bz.*bz); % % Plot the histogram of the grayscale edge image, b, to manually % % determine the threshold to use in generating the binary edge image. % v = reshape(b,size(b,1)*size(b,2)*size(b,3),1); % hist(v) % Threshold the grayscale strength-of-edge image, b, to produce a binary % edge image binEdgeImage = (b > thresh); % Now create the output MAT file save(outputFile,'binEdgeImage','lidar3dImage','minCoords') disp('Done-- output file has been created!')

19

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % detect_cylinders % % % % For ECE533 project. This program uses a Hough transform to detect % % cylinders in a 3D binary edge image. The assumption that the cylinders % % are vertical reduces the number of parameters to 3: one for the % % radius and two for the (X,Y) location of the axis. Input to the % % program consists of the binary 3D edge image read in from a MAT file. % % % % C. Parrish % % Created: 12/5/2006 % % Modified: 12/6/2006 Made parameter ranges and spacing adjustable. % %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% close all; clear all; clc; % Adjustable parameters %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% inputFile = strcat('D:\01Chris\University of Wisconsin\ECE_533\', ... 'Project\3D Binary Edge Images\FID19_bin_edges.mat'); rRange = [1 3.5]; % Min & max values of parameter r rStepSize = 0.5; % Cell size in r dimension of parameter space sRange = [1 45]; % Min & max values of parameter s sStepSize = 0.5; % Cell size in s dimension of parameter space tRange = [1 45]; % Min & max values of parameter t tStepSize = 0.5; % Cell size in t dimension of parameter space %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% tic; % Start the timer load(inputFile); % This will load the following variables: % lidar3dImage = 3D grayscale intensity image % binEdgeImage = 3D binary edge image % minCoords = 3x1 vector specifying minimum % coordinates (Easting,Northing,elev) % Initialize the counters of all accumulator cells in the 3D parameter % space to zero sDim = round((sRange(2) - sRange(1))/sStepSize) + 1; tDim = round((tRange(2) - tRange(1))/tStepSize) + 1; rDim = round((rRange(2) - rRange(1))/rStepSize) + 1; accumulatorArray = zeros(sDim,tDim,rDim); % Find the indices of all nonzero entries in the 3D binary edge image indx = 1; for i = 1:size(binEdgeImage,1) for j = 1:size(binEdgeImage,2) for k = 1:size(binEdgeImage,3) if binEdgeImage(i,j,k) == 1 edgeVoxels(indx,1) = i; edgeVoxels(indx,2) = j; edgeVoxels(indx,3) = k; indx = indx + 1; end end end end % For each nonzero voxel in the 3D binary edge array, step through all % values of s and t. At each location, solve the equation describing a % vertical cylinder for the radius, r. Round r to the nearest accumulator % cell value and increment the counter for that (s,t,r) cell. for i = 1:length(edgeVoxels) x = edgeVoxels(i,1);

20

y = edgeVoxels(i,2); for sIndx = 1:sDim s = sRange(1) - sStepSize + sIndx*sStepSize; for tIndx = 1:tDim t = tRange(1) - tStepSize + tIndx*tStepSize; r = sqrt((x-s)^2 + (y-t)^2); rIndx = round((r - rRange(1))/rStepSize) + 1; if (rIndx >= 1) && (rIndx <= rDim) accumulatorArray(sIndx,tIndx,rIndx) = ... accumulatorArray(sIndx,tIndx,rIndx) + 1; end end end end % Find the entry in the 3D accumulator array with the highest number of % votes maxVotes = max(max(max(accumulatorArray))); maxVoteIndx = find(accumulatorArray == maxVotes); [sMaxIndx tMaxIndx rMaxIndx] = ind2sub(size(accumulatorArray),maxVoteIndx); sHighestVote = sRange(1) - sStepSize + sMaxIndx*sStepSize tHighestVote = tRange(1) - tStepSize + tMaxIndx*tStepSize rHighestVote = rRange(1) - rStepSize + rMaxIndx*rStepSize disp(strcat('Done in ~',num2str(toc),' seconds')) % Plot the binary edge image sx = size(binEdgeImage,2)/2 + 5; sy = size(binEdgeImage,1)/2; sz = [4]; I = 10.*binEdgeImage; h = slice(I,sx,sy,sz); xlabel('X axis','fontSize',14) ylabel('Y axis','fontSize',14) zlabel('Z axis','fontSize',14) lims = volumebounds(binEdgeImage); lims(7) = -16; lims(8) = 45; axis(lims); axis equal; % Now generate and draw the cylinder that received the highest number of % votes in the same figure. Note that my UTM Easting corresponds to the % MATLAB cylinder's Y coord and my UTM Northing to the X coord, so I just % have to switch the X and Y to plot correctly. [Y,X,Z] = cylinder(rHighestVote); % Translate axis of cylinder to correct location X = X + sHighestVote; Y = Y + tHighestVote; hold on mesh(Y,X,Z.*size(binEdgeImage,3), ... 40.*ones(size(Z))); title('FID19: Cylinder Detected Using Hough Transform','fontSize',16); % Calculate the UTM Easting, Northing position of the detected cylinder % axis eastingCylinderAxis = minCoords(1) + sHighestVote northingCylinderAxis = minCoords(2) + tHighestVote

21

References Beder, C., and W. Förstner, 2006. Direct Solutions for Computing Cylinders from Minimal Sets of 3D Points. Proceedings of the European Conference on Computer Vision 2006, Graz, Austria, pp. 135-146. Fowler, R., 2001. Topographic Lidar, Digital Elevation Model Technologies and Applications: The DEM Users Manual, (D. Maune, editor), American Society for Photogrammetry and Remote Sensing, Bethesda, Maryland. Gonzalez, R.C., and R.E. Woods, 2002. Digital Image Processing, 2nd Ed. Prentice Hall, Inc., Upper Saddle River, New Jersey. Kay, S.M., 1998. Fundamentals of Statistical Signal Processing Vol. II: Detection Theory. Prentice Hall, Inc., Upper Saddle River, NJ. Olson, C.F., 2001. Locating Geometric Primitives by Pruning the Parameter Space, Pattern Recognition, Vol. 34, No. 6, pp. 1247-1256. Parrish, C.E., G.H. Tuell, W.E. Carter, and R.L. Shrestha, 2005. Configuring an Airborne Laser Scanner for Detecting Airport Obstructions. Photogrammetric Engineering and Remote Sensing, Vol. 71, No. 1. Parrish, C.E., J. Woolard, B. Kearse, and N. Case, 2004. Airborne LIDAR Technology for Airspace Obstruction Mapping. Earth Observation Magazine (EOM), Vol. 13, No. 4. Rabbani, T. and F. van den Heuvel, 2005. Efficient Hough Transform for Automatic Detection of Cylinders in Point Clouds. Proceedings ISPRS WG III/3, III/4, V/3 Workshop “Laser Scanning 2005”, Sept. 12-14, Enschede, the Netherlands. Simonse, M., T. Aschoff, H. Spiecker, and M. Thies, 2003. Automatic Determination of Forest Inventory Parameters Using Terrestrial Laserscanning. Proc. ScandLaser Scientific Workshop on Airborne Laser Scanning of Forests, Umeå, Sweden, pp. 251- 257. Sinha, P.K., F.-Y. Chen, and R.E.N. Home, 1993. Recognition and Location of Shapes in the Hough Parameter Space. IEEE Colloquium on Hough Transforms, pp. 11/1 – 11/4, 1993. Spies, H., and J.L. Barron, 2004. Evaluating Certainties in Image Intensity Differentiation for Optical Flow. IEEE, Canadian Conference on Computer and Robot Vision, May, pp. 408-416. Trucco, E., and A. Verri, 1998. Introductory Techniques for 3-D Computer Vision. Prentice Hall, Inc., Upper Saddle River, NJ.

22

Tuell, G., 1987. Technical Development Plan for the Modernization of the Airport Obstruction Charting Program. NOAA Charting Research and Development Laboratory, Office of Charting and Geodetic Services, Rockville, Maryland. Tuell, G., 2002. Airborne laser swath mapping (ALSM) for determining discrete airspace obstructions, Invited Presentation, Special Session on Airspace Obstruction Mapping, National Transportation Research Board (TRB), January, 2002, Washington, D.C. U.S. Department of Transportation, 1993. FAA Order 8260.19C, Flight Procedures and Airspace, Federal Aviation Administration, Washington, D.C. Vernon, D., 1991. Machine Vision: Automated Visual Inspection and Robot Vision. Prentice Hall, Inc., Upper Saddle River, NJ. Vosselman, G., B.G.H. Gorte, and T. Rabbani, 2004. Recognising Structure in Laser Scanner Point Clouds. IAPRS, Vol. 36, part 8/W2, pp. 22-38.

23

Task Assignment:

Project: The Hough Transform for Vertical Object Recognition in 3D Images Generated from Airborne Lidar Data Team: Christopher Parrish (single person team)

Task Christopher Parrish Literature review 100% Abstract and project proposal 100% Algorithm development 100% MATLAB implementation of algorithms for: voxelixing lidar point cloud data, applying 3D Sobel operator, performing threshold segmentation, and identifying vertical cylinders using the Hough transform

100%

Testing and debugging 100% Results assessment 100% Project report 100% Power point presentation 100% Overall Project Total 100% Signature ____________________________________________________________