adaptive morphology

8/9/2019 Adaptive Morphology

1/4

272 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 2, NO . 2, APRIL 1993

Adaptive Mathematical Morphology for Range ImageryJacques G. Verly and Richard L. Delanoy

Abstract-We explore the application of adaptive (i.e., data-dep enden t)mathematical morphology techniques to range imagery, i.e., the use ofstructuring elements that au tomatically adjust to the grayscale values ina range image in order to deal with (e.g., extract or eliminate) featuresof known physical sizes.

I. INTRODUCTIONOften, the output of a mathematical-morphology (MM) operation

[ l] either binary or grayscale) describes how well a shape, calleda structuring element (SE), either fits or d o e s not fit inside a localimage feature.Whereas the SEs used in most applications remain constant asthey probe an image, there are si tuations where SEs must changetheir size, orientation, and/or shape during probing. If these changesare determined a pr ior i , independently of the data, the SEs can besaid to be space var iant. If the change is made on the basis of thevalue(s) of one or more pixels around the pixel being probed, theSEs can be said to be data dependent or adapt ive . We are aware ofonly a few papers dealing with space-variant SEs, e.g., [ 2 ] (also in[3]) nd [4], and of none dealing with adaptive SEs.Since range imagery [5 ] ontains significant shape information andsince MM deals with shape in images, it is also surprising that so fe wpapers have been written on the application of MM to range imagery(examples are [6]- o]).Since angle-angle range images are subject to the usual perspec-tive distortions, the apparent length of a feature in such images isa function of the feature range. With range available at each pixel,one can simultaneously process (e.g., extract or eliminate) all thedifferently-scaled instances of the feature or object of interest byadapting the sizes of the SE(s) to the local range.

For simp licity, w e will assume that all SEs of interest become fullydetermined when their scalable z ize and y size are specified (s ndy, respectively, refer to the horizontal and vertical image axes). Inother words, w e assume that adapting an SE to range simply amountsto finding the appropriate z nd y sizes. Examples of useful SEs thathave such properties are the rectangular and ellipsoidal SEs (eitherbinary or grayscale). Note that, for this class of SEs, changing the san d y sizes can in som e cases cause a spatial compression/expansionof the pattern (either binary or grayscale). The goal of this paperis to develop a systematic, practical methodology for designing andapplying adaptive S Es for range imagery under the constraints justdescribed.

11. PRINCIPLESLe t us consider a range image R of a scene S. The grayscale value

r , associated with each pixel is related to the distance T , (say, inmeters) between the corresponding patch of S and the sensor by

r m - or t = [ 7 1Manuscript received January 8, 1991; revised September 15, 1992. This

work w as sponsored by the DARPA Tactical Technology O ffice. The associateeditor coordinating the review of this paper and approving it for publicationwas Dr. A. C. Bovik.The authors are with the Machine Intelligence Technology Group, MITLincoln Laboratory, Lexington, MA 02173-9108.IEEE Log Number 9206911.

\

1 2 3 4 5 6 ;Fig. 1. Range intervals associated with integer-length SEs when the SEs

musffit inside a feature.where ro is a range offset corresponding to the grayscale value of 0,b is the range quantization value, and [z] s the integer part of 2.The length nk in units of pixels of a horizontal feature F (inR) hat has an actual length of zm meters (in S ) and is located ata distance of rm meters from the imaging sensor is given by thesmall-angle approximation

where aZ. s the horizontal angular width (in radians) of a pixel.This expression can also be used to find the z dimension n , (inpixels) of the SE required to detect F in R (for convenience, we cantemporarily fo cus on the case of horizontal, rectangular, 1-pixel highSEs). Notice that we use primed symbols like n: to denote arbitrarylengths in units of pixels, and unprimed symbols like nl- or lengthsthat are an integer number of pixels.

From the functional dependency of r , upon n: for fixed cyz an d.rm (e.g., as in Fig. l) ,we observe the following.

First, consider SEs that m u s t f i t inside F in R. Such SEs will besaid to be of type MF . For example, a SE used in an opening withthe intent of p r e s e r v i n g a protrusion must be of type MF ; ditto forclosing and intrusion. A fixed-length SE designed to fit inside F ata given range will fit in the same F at a shorter range, but not ata longer range. Since the localization of F based on a fixed-lengthSE becom es less sharp as the range decreases, one should sw itch tothe next larger size SE as soon as its design range is reached. Insummary, in the case of MF-type SEs, the range interval where agiven S E can be used is from (and including) its associated range (i.e.,the range at which it fits exactly in F ) d o w n t o (but excluding) thedesign range of the next larger SE. For example, when all integer-length SEs are available, the semi-open range interval for the SEcorresponding to n, = 1 is obtained from the solid curve segmentlabeled 1 in Fig. 1( the interval is show n along the r,-axis). Pixelswith ranges corresponding to nl-= 0 should not be processed at all.

1057-7149/93$03.00 0 1993 IEEE


2/4

IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 2, NO. 2, APRIL 1993 273

"

1 2 3 4 5 6 " xFig. 2. Range intervals associated with integer-length SE's when the SE's

must not fi t inside feature.Second, consider SE's that must not fi t inside F in R . Such SE's

will be said to be of type M N F . For example, a SE used in anopening with the intent of eliminating a protrusion must be of typeM N F ; ditto for closing and intrusion. Here, the range interval wherea given SE can be used is from (but excluding) its associated range(i.e., the range at which it fits exactly in F ) up to and including thedesign range of the next smaller SE (Fig. 2).In practice, it may not be feasible to use all the sizes n , that arenecessary to cover the total range interval of R. The dashed lines inFigs. 1an d 2 show the range intervals and the corresponding SE sizesfor the hypothetical case where only the od d sizes n , are available.

111. IMPLEMENTATIONDenote the minimum and maximum values of R by T A , ~nd T B , ~ ,where i is a reminder that these numbers are integers. Using (l), one

can find the limits of the span Ira,,, rg .m]of actual range values(in meters) in R, i.e.,

TA ,m = To + A . % brB ,m = TO -I-rB.% - 1 ) b .

A . Selection ofNecessary SE'sUsing (2), one finds that the required interval of sizes n: is

[nL,min, n l , m a x ] , with

Assuming that the only SE sizes available are those in the orderedse t

N: = {ns , k k = 1,Zi: ;n z , i= 0: n r , k < n , , k + i )where li: is the number of available sizes, the corresponding intervalof integers n , is [nz,m,n,n,,max]here the limits are as follows.

1. Fo r SE's of type MF:nz,m,n= largest integer in .V:nz,max= largest integer in iV:

less than or equal to n:,mlnless than or equal to n: m a x .

2. Fo r SE's of type MNF :nz,min= smallest integer in N:nz,max= smallest integer in N:strictly gre ater than nL,minstrictly greater than nk,max.

Thus the set of SE's that can be called upon is the ordered subsetA V x = { 7 L z , k Ik = 1 , IC,;nz ,k E N,* :

1Lz,min I , ,k Iz . m ax : nr,k < n z , k + i >of i where li, 5 1;: is the maximum index k needed.B. Selected-SEs' Range Intervals in Metersin IV, by rZ3k,,,,, with

Denoting the actual range (in meters) associated with each n,,k

it can be shown that the lower and upper limits (in meters) of th esemi-open interval [ r , ,L , k , m , r,, H , , m ] are as follows.

1) Fo r SE's o f type M F :

2) For SE's of type MdVF:

C . Selected-SEs ' Range Intervals in Range IntegersExpressing interval limits in terms of range integers (instead of

meters) can speed up the repeated comparison of pixel grayscalelevels to the various intervals. IntroducingI T x , k . m - o

one can show that the upper and lower limits (in range integers) ofthe c losed interval [ r , , L , k , z ,T , , H , ~ , ~ ]ssociated with each size n , , kin ,Vz are as foIlows. '

br , k . z =

1) For SE's of Type M F :T ~ , H , ~ , * largest integer smaller than or equal to r; , k , *

2) For SE's of type M i V F :T , , L , ~ , ~ smallest integer strictly greater than r: ,k,z

D. General Implementation StrategySo far, we have focused on the x dimens ion of an SE and onthe particular case of a 1-pixel high horizontal SE . Of course, the

argument can be identically repeated for the y dimens ion and for a'The limits r , , L , k , z and T , , H , ~ , % are derived independently of theircounterparts r , ~ , l ; . ~nd r , , ~ , k , ~ .


3/4

274 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 2, NO. 2, APRIL 1993

Fig. 3 .B = R H 7 o - R H 3 0 ,Extraction of tank bodies from a synthetic range image: (A) range image R, (B ) closing R"7 0 (C) closing RH3 (D) output image

1-pixel wide vertical SE . In the case of an arbitrary SE , one mustdeal with both dimensions simultaneously.The general strategy for efficiently implementing adaptive MM

algorithms is to (a) determine the sets S, nd of applicable SEsizes, (b) find the interval I r . k ( I y . f ) f range values (either in metersor in range integers) each SE size n r , k E LVz(~~v,l-Vy)pplies to,(c) denote by I k , r the (possibly em pty) range interval that is commonto 1 r . k an d Iy , fnd associate with i t the available SE with sizesn,,k an d nY, l , nd (d) perform the desired operation on the image ofinterest by selecting the appropriate available SE at each pixel basedupon the range at that pixel. This last step may be implemented ina variety of ways, including in parallel, e.g., with one processor perapplicable SE . Once again , one should stress that the processing willdepend upon the type of each SE of interest, i.e., MF or M V F .

IV . APPLICATIONAdaptive MM is routinely used by the authors for the extractionof vehicles of known sizes from real low-resolution (2-6 meters)

tactical laser-radar imagery. It is one of the techniques used in theXTRS system described in [ l l] . Here, to better demonstrate the

potential of adaptive MM, we use a 2-m resolution synthetic imageof a scene consisting of a ground plane, a background vertical wallat 1,700 m, and 3 objects: a 3.6 m-wide tank and a 2.4 m-wide box,both at 700 m, and a replica of the first tank at 1,500 m (Fig. 3(A)).In images taken roughly at ground level, vehicles that have width UJand length 1 produce observable silhouettes whose length should be inthe approximate interval [w . d1 . For the tank in Fig. 3(A),the limits of this interval for the tank body are approximately 3 man d 7 m.

A n image B that highlights the tank bodies in the image R ofFig. 3(A) can be created by using the difference-of-closings

B = RH^ 0 - RH3where Hi 0 is a MF-type adaptive SE designed for a feature lengthz),~= 7.0 m and HY 0 is a M%-type adaptive SE designed for.rn, = 3.0 m (both SE's are I-pixel high). The result is shown inFig. 3(D). First, note that both tank bodies are correctly highlightedin spite of their different apparent widths.This results from the fact that the visible 3.6 m physical widths ofthe bodies ar e within the allowed 3-7 m physical-size range for thevisible widths of such bodies. Second, note that the box in front of th e


4/4

IEEE TRANSACnONS ON IMAGE PROCESSING, VOL. 2, NO. 2, APRIL 1991

farthest tank is not detected even though its apparent width is betweenthose of the tank bodies. Of course, the reason is that the physicalwidth of the box is outside the allowed range fo r physical body sizes.Clearly, the exact same technique would extract any number of tanks(e.g., in column), provided the tanks do not significantly overlap.

The success of our simple object extraction procedure is not onlydue to the well-known sieving properties of successive closings(or openings) with differently sized SEs [l],bu t also to the use ofrange-adaptive SEs properly designed with respect to the AlIF/A~f-\7Fdichotomy.

V . CONCLUSIONSIn this work, we have pointed out the role of adaptive MM in theprocessing of range imagery. We have emphasized the importance of

correctly distinguishing between, and implementing, adaptive (i.e.,data dependent) SEs that either must fit or must not fit withinthe feature(s) of interest. It should be clear that the choice amon g theelements of th e JIF/MAT dichotomy is based on the problem athand, and is not an intrinsic property of the MM operation used. Inother words, there is no such conc lus ion as saying that MM operationX (e.g., dilation, erosion, opening, closing, etc.) should always beused with an adaptive SE designed so that it fits (or, similarly, doesnot fit) inside the feature of interest.Although we have discussed adaptive MM in the specific case ofrange imagery, this technique is applicable to any type of image forwhich the distance to a scene element is available for each pixel. Sucha situation occurs in laser-radar imaging where pixel-registered rangeand intensity images are often available. In this particular application,adaptive MM can be applied to the intensity image by using thedistance information available from the range image.The techniques presented are limited to the case where the SEs ca nbe scaled through their .r an d y sizes. Clearly, the opportunity existfor developing a more general theory and methodology for designingand applying adaptive SEs to range imagery. Furthermore, the wholequestion of how to efficiently implement adaptive MM should beinvestigated in detail.

215

A CK N O WLED G M EN TTh e authors wish to acknowledge the help of Petros Maragos, fromHarvard University, for his help in locating some prior work in space-

variant mathematical morphology. They also thank one anonymousreviewer for pointing out [lo].

REF EREN CES[ 11 Jean Serra, Image Analysis and Mathematical Morphology. London,U.K.: Academic, 1982.121 S. Beucher, J.M. Blosseville, and F. Lenoir, Traffic spatial mea-surements using video image processing, in Intelligent Robots andComputer vision, Proc. P I E , vol. 848, Cambridge, MA : pp. 648-655,[3] Jean Serra, Examples of structuring functions and their uses, ImageAnalysis and Mathematical Morphology. Vol 2: Theoretical Advances.J. Serra, Ed., New York: Academic, 1988, chap. 4, pp. 71-99.[4 ] J. B. T. M. Roerdink and H. J. A. M. Heijmans, Mathematical morph-ology for structuring elements without translation symmetry, SignalProcessing, vol. 15, no. 3, pp. 271-277, 1988.15) Paul J. Besl, Active optical range imaging sensors, in Advances inMachine Wsion, J. L. C . Sanz. Ed. New York: Springer-Verlag, 1989,(61 Lawrence A. Ankeney, The Design and Performance Characteristics ofa C ellular Logic 3 -0 Image C lassification Processor. Ph.D. dissertation,Air Force Institute of Technology, Wright Patterson AFB, OH , 1981.(71 Thomas R. Esselman and Jacques G. Verly, Some applications ofmathematical morphology to range imagery, in Proc. Int. ConJ onAcoustics, Speech, and Signal Processing ICASSP, Dallas, TX, Apr.18) Thomas R. Esselman and Jacques G. Verly, Feature extraction fromrange imagery using mathematical morphology, in visual Communica-tions and Image Processing II , T. Russell Hsing, Ed. SPIE, vol. 845,27-29 Oct. 1987, pp. 233-240.[9] Thomas R. Esselman and Jacques G . Verly, Applications of mathe-matical morphology to range imagery, Tech. Rep. TR-797, M.I.T.,Lincoln Laboratory, Lexington, 02173-9108, Dec. 1987.

[lo] Robert M. Lougheed and Robert E. Sampson, 3-d imaging systemsand high-speed processing for robot control, Mach. vision and Appl.vol. I , no. 1, pp. 41-57 , 1988.1111 Jacques G . Verly, Richard L. Delanoy, and Dan E. Dudgeon, Amodel-based system for automatic target recognition, Automatic ObjectRecognition, Proc. SPIE, vol. 1471, pp. 186-196, Orlando , FL, Apr.3-5. 1991.

NOV.2-6, 1987.

Chap. 1, pp. 1-63.

1987, pp. 245-248.

adaptive morphology

Documents