DIGITAL RECTIFICATION AND GENERATION OF ORTHOIMAGESIN ARCHITECTURAL PHOTOGRAMMETRY

MATTHIAS HEMMLEB

Fokus GmbH, Färberstr. 13, D-04105 Leipzig, GermanyE-mail: matti@fpk.tu-berlin.de

ALBERT WIEDEMANN

Technical University of Berlin, Department for Photogrammetry and CartographySekr. EB9, Str. d. 17. Juni 135, D-10623 Berlin, Germany

E-mail: albert@fpk.tu-berlin.de

KEY WORDS: Rectification, Surface Development, Orthoimage, Architecture, Monument Preservation

ABSTRACT

This paper gives an overview over different photogrammetric single image techniques, like digital rectification, unwrap-ping of parametric surfaces and differential rectification methods. It shows the new possibilities for the generation oforthoimages and image mosaics using digital image processing methods even on low-cost personal computers. After ashort description of the implemented methods, there are presented some results in the field of architectural photogram-metry and monument preservation.

1. INTRODUCTION

Photogrammetric single image techniques, like the gene-ration of rectified images and orthoimages are well suitedfor use in architecture and monument preservation. Theycombine true scale geometric measurements with fullimage information under quite inexpensive productioncosts. Especially in the field of architectural orthoimagegeneration, the combination of image processing withphotogrammetric systems provides new solutions.

Digital image processing methods play a more and moreimportant role for the production of rectified images.Because of the increasing performance of personal com-puters as well as the availability of necessary peripheralhardware, like CD-writers and photorealistic printer units,rectification with analogue equipment is no longer neces-sary today. Especially the PhotoCD delivers good re-quirements for digitizing and storing digital image datafor photogrammetric purposes (HANKE, 1994).

There are different approaches for the photogrammetricrectification of images. The simplest method is the pro-jective rectification, used in the past with analogue recti-fiers. Digital orthoimage techniques, like differential recti-fication are well known in aerial photogrammetry, butthey are rarely used in architectural photogrammetry.With digital image processing methods some new rectifi-cation techniques became possible, like polynomial recti-fication or unwrapping methods (BÄHR, 1980), (KARRAS etal., 1996). Especially these unwrapping methods providenew interesting and non-expensive solutions in architec-tural photogrammetry.

The selection of an appropriate approach depends ondifferent facts. First of all it is necessary to describe thesurface, which should be rectificated. If it is nearly plane,a projective rectification should be chosen. In case ofsmall uneven areas within the surface, a polynomialrectification could be useful. If the object could bedescribed as a parametric volume, like a cylinder or a

cone, a rectification is possible with digital unwrappingtechniques. Complex surfaces have to be rectified withdifferential rectification methods.

In some cases complicated surfaces have to be rectifiedwith combinations of the described methods. To combinethese methods it is necessary to cut the images in diffe-rent parts and rectify the pieces part-by-part. This re-quires an enormous amount of interactive work to cutand merge the images and a lot of control information isnecessary. Small errors become distinct at the edges ofthe pieces. Under consideration of these problems thisapproach yields good results (MARTEN et al., 1994).

2. SELECTION OF THE SUITABLE APPROACH

2.1. Geometric Considerations

Figure 1: Geometric constraints usingrectification techniques

As already mentioned, it is necessary to decide about themost suitable technique to provide pictorial orthoprojec-tions first. By calculating the resulting folding of points infront or behind the main object surface objectivedecisions about the required techniques for the rectifica-tion of this images can be made.

The radial folding ∆r of the points P1 and P2 on the fa-cade depend on the focal length ck of the camera and onthe difference ∆y between the object distances y1 and y2:

∆ ∆rrc

yk

='2

If the maximum image radius in the corners of the ima-ges is r’max and a maximum radial folding ∆rmax is permit-ted, the maximum differences between the object distan-ces ∆ymax are:

∆ ∆yc

rrk

maxmax

max'=

2.2. Parametric versus Non-Parametric Approaches

There are two classes of rectification approaches: Theparametric and the non-parametric approaches. Whereasfor the parametric approach the knowledge of the interiorand exterior orientation parameters is required, non-para-metric approaches require more control-points.

The parametric approach uses a reverse ray tracing todetermine the grey-values for all pixel positions in therectified image. The resampling of the grey-values re-quires the position of the corresponding position in thedistorted image. This can be calculated by the collinearityequations using interior and exterior orientation para-meters and the position of the corresponding point in theobject coordinate system. The interior orientation may beknown from a calibration protocol of the image and ameasurement of fiducial marks or a reseau. The exteriororientation parameters can be calculated by a resectionin space using at least three control points or a bundleadjustment. Interior and exterior orientation parameterscan also be estimated together within a spatial resection.In this case the spatial distribution of the control pointsplays an important role in order to obtain correct results.Whereas at least five control points are necessary, moreof them are recommended to stabilise the calculation andto get statistical information about the accuracy.

In case of non-parametric approaches a planar coordi-nate transformation between the matrices of the distortedand the rectified image is used.

Transformation Type Number ofParameters

RequiredControlPoints

Similarity-Transformation 4 2Affine Transformation 6 3Projective Transformation 8 4Polynomial 2nd Grade 12 6Polynomial 3rd Grade 20 10

Table 2: Non-parametric approaches

The object coordinates of the control points have to bereduced to 2D-coordinates on the projection surface first.Table 2 shows the necessary control points for non-parametric approaches, depending on the number ofneeded coefficients.

3. DIGITAL RECTIFICATION METHODS

3.1. Projective Rectification

If the objects surface describes sufficient precise a plane,a projective rectification should be used. To perform aprojective rectification, a geometric transformation bet-ween the image plane and the projective plane is neces-sary. For the calculation of the eight unknown coefficientsof the projective transformation, at least four controlpoints in the object plane are required. The equations forprojective rectification are given as follows (ALBERTZ,KREILING, 1989):

xb x b y bb x b y

' =+ ++ +

11 21 31

13 23 1

yb x b y bb x b y

'=+ +

+ +12 22 32

13 23 1

The positions of the grey-values of the given image haveto be transformed with these equations into the rectifiedimage. For this task, called resampling, there are twopossibilities: In dependence on the direction of the trans-formation we distinguish between a direct and an indirectrectification. Digital rectification generally uses the indi-rect method, because the rectified image will be calcu-lated pixel by pixel. For that reason it is necessary to finda method for the interpolation of the grey-values, becau-se the calculated position of the needed grey-value in thegiven image is mostly between the image raster. Thereare different methods for grey-value interpolation: For in-stance Nearest Neighbour, Bilinear and Bicubic Inter-polation. The better the quality of the rectified imageshould be, the more calculation time is needed (table 3).But today the performance of personal computers provi-des good compromises: Even high resolved images canbe rectified with Bilinear Interpolation or better methods.

Interpolation Method InterpolationWindow

Number ofArithmeticalOperations

Nearest Neighbour 1x1 0Bilinear Interpolation 2x2 8Bicubic Interpolation 4x4 110Lagrange Interpolation 4x4 80

Table 3: Comparison of different interpolation methods

To examine the accuracy of the rectification, the controlpoints have to be superimposed with the rectified image.The control points only have to be transformed in theused image scale. Measurement errors and differencesbecause of uneven surfaces are easily to detect.

The digital projective rectification and other digital rectifi-cation methods provide some new advantages like the

production of rectified colour images. Even images witharbitrary angles - like in our example - are suited for theprocessing. Therefore, images of buildings with difficultmapping configurations, like small courtyards, may beeasily rectified. If necessary, a number of these rectifiedimages may be merged to an image mosaic. For thatreason it is necessary to use a uniform reference systemto calculate an offset for each image, that gives thegeometric position of the rectified single images in theimage mosaic. At least, a radiometric adjustment is ne-cessary. That will take place interactively, because auto-matic approaches are not yet usual for architecturalimages.

Figure 4: Historical metric photograph from the GreatHall in the Marmorpalais in Potsdam, New Garden

Figure 5 shows an example for a digital rectification. It isthe ceiling of the Great Hall of the Marmorpalais inPotsdam (Germany). The aim for this rectification wasthe production of a working basis for the reconstructionof the ceiling painting, which was destroyed in formertimes. We used three historical images, two of them fromthe Meydenbauer-Archiv, which is known for its highnumber of high-quality metric photographs (see figure 4).

3.2. Polynomial Rectification

It seems to be very dangerous to use higher grade poly-nomial transformations for the rectification of images.The required amount of control points and the risk of anoscillation is growing with the grade of the polynome.Their use may be suitable for curved elements of facadesif sufficient control information is available (MARTEN et al.,1994).

3.3. Unwrapping of Developable Surfaces

Another approach is the unwrapping of developable sur-faces. Using digital image processing methods it is pos-sible to unwrap parametric objects, for instance cylin-drical or conical objects. The shape of many towers andvaults may be approximated by such parametric volu-mes. As the result of the unwrapping process, we obtaina true scale image plot of the whole object.

To perform an unwrapping, we use a parametricapproach. That means we have to calculate a spatialresection for each image to get the parameters of theexterior and in most cases the interior orientation on thebasis of the collinearity equations (ALBERTZ, KREILING,1989):

x x ca x x a y y a z za x x a y y a z zh k'

( ) ( ) ( )( ) ( ) ( )

= −− + − + −− + − + −

11 0 21 0 31 0

13 0 23 0 33 0

y y ca x x a y y a z za x x a y y a z zh k'

( ) ( ) ( )( ) ( ) ( )

= −− + − + −− + − + −

12 0 22 0 32 0

13 0 23 0 33 0

On the other hand we need to describe the parameters ofthe object surface (cylinder or cone) with the help ofcontrol points. The height values h1 and h2 and theangles α1 and α2 of the developable area are given. Theorigin coordinates of the cylinder xm and ym and theradius r are calculated with a least-squares-adjustment.

Figure 5: Digital rectified image mosaic of the ceiling of the Great Hall in the Marmorpalais in Potsdam, New Garden

The equations for the least-squares-adjustment looks asfollows:

x x r y ym m= + − −2 2( )

y y r x xm m= + − −2 2( )

For a cone we additionally need two radii r1 and r2. Theequations are:

x x rr rh h

h z y ym m= + +−−

− − −( ( )) ( )21 2

2 12

2 2

y y rr r

h hh z x xm m= + +

−−

− − −( ( )) ( )21 2

2 12

2 2

The resampling process (like already explained in chap-ter 3.1.) uses the orientation parameters from the spatialresection and the calculated object parameters for theunwrapping on the basis of the collinearity equations. Forpractical work it is useful, to measure a number of controlpoints, which are well suited for both the spatial resectionand the estimation of the volume parameters. Because ofthe digitizing of the images the interior and the exteriororientation should be estimated together. Therewith it ispossible to consider offset, rotation and scaling problemswith the used scanner.

All calculations use least-squares techniques with sta-tistical methods like the detection of gross errors to getinformation about the accuracy of the rectified and un-wrapped image.

Our first example for the unwrapping of a developablesurface is a tower from the Moritzburg in Halle (Ger-many), which has nearly a cylindrical shape. The imagemosaic consists of seven single rectified images.

Figure 6: Metric photograph of the West-Towerof the Moritzburg, Halle

Figure 7: Digital unwrapped image mosaic of the West-Tower of the Moritzburg, Halle

Figure 8: Photogrammetric plot of the West-Tower of the Moritzburg, Halle

Our second example is the ceiling painting of the Iwein-cellar in the Hessenhof in Schmalkalden (Germany). Thispainting is about 750 years old. The vault is nearly acylinder, but in order to get precise plans of this paintingwe used a cone as the basis for the unwrapping, becausethis was the true shape of the vault. The image mosaicconsists of only three images, because the cellar is verysmall.

Figure 9: Iwein-cellar in the Hessenhof, Schmalkalden,one of three used metric photographs

3.4. Differential Rectification

If none of the already mentioned techniques is suitablefor the rectification of images a differential rectification isnecessary. Therefore a geometric description of the ob-ject surface, relative to the projection surface is neces-sary. This is usually provided by a Digital Surface Model(DSM), which is a raster data set storing the elevationabove the projection surface.

Beside a precise documentation of the elevation it isnecessary to store the exact location of discontinuities ofthe surface of the object. Resulting from the largeamount of discontinuities on building surfaces, e.g. win-dows or balconies, the geometric resolution of the DSMhas to be equal to the resolution of the orthoimage toproduce, whereas on continual surfaces an interpolationbetween the DSM meshes may be suitable. The mainproblem in the generation of digital orthoimages of faca-des is the delivery of a sufficient DSM (WIEDEMANN,1996).

The first approach was to generate a DSM by image mat-ching techniques. Most tests delivered insufficient results.Image based matching techniques failed at the edgesbecause of different backgrounds. The provision of initialvalues using image pyramids was disturbed by obstaclesin the foreground. Blunders resulted from the similarity offeatures.

Figure 10: Digital unwrapped image mosaic of the ceiling painting in the Iwein-cellar

A better approach is to derive the DSM from a CADmodel. The CAD model has to consist of surface ele-ments, a wire model is insufficient. The model is proces-sed face by face. For each raster position a point-in-poly-gon algorithm is used to decide, whether it lays below theface. In this case, the elevation of the face above the pro-jection surface is calculated and stored in the DSM. Ifthere is already an elevation the bigger of the two ele-vations is selected for storage in the DSM.

A common way to create a CAD model is a reduced pho-togrammetric restitution. This restitution has to take intoaccount only features, which have an influence on theDSM. Stereophotogrammetric and improved bundle-ad-justment techniques have been used for this purpose.Beside this technique other data sources for the DSMmay be used, e.g. Laser Scanners (WEHR, 1997) or al-ready existing CAD models.

After the generation of the DSM the differential rectifica-tion will deliver a digital orthoimage. During the rectifica-tion process each pixel is tested, whether the correspon-ding object point is covered by a different part of theobject. This will be done by counting the intersection ofthe image ray with the DSM. Whereas the derivation ofthe DSM is a time consuming task, the generation of thedigital orthoimage can be repeated with the same DSMbut another image within a few minutes. Different exteriororientations lead to different occluded areas. By merginga few images most occluded areas may be filled with in-formation. An example of a process for generating digitalorthoimages is shown in Figure 11.

4. CONCLUSIONS

The pictorial representations generated with the presen-ted approaches provide an enormous amount of informa-tion to the user, who can extract geometric informationwhile considering the image. Further usage of these pro-ducts can be anticipated in the field of architectural visu-alisation.

ACKNOWLEDGEMENTS

The authors like to thank Annette Scheer for her work onthe unwrapping of conical surfaces.

REFERENCES

ALBERTZ, J., KREILING, W.: Photogrammetrisches Ta-schenbuch, Herbert Wichmann Verlag Karlsruhe, 4. Auf-lage 1989.

BÄHR, H. P.: Einsatz digitaler Bildverarbeitung in der klas-sischen Photogrammetrie am Beispiel eines Archi-tekturobjekts. Bildmessung und Luftbildwesen (48) 1980,pp. 85-93.

HANKE, K.: The PhotoCD - A source and Digital Memoryfor Photogrammetric Images. International Archives ofPhotogrammetry and Remote Sensing, Vol. XXX, Part 5,Melbourne, Australia, 1994.

Figure 11: Data flow for the generation of digital orthoimages

KARRAS, G. E., PATIAS, P., PETSA, E.: Digital Monoplottingand Photo-Unwrapping of Developable Surfaces in Archi-tectural Photogrammetry. International Archives of Pho-togrammetry and Remote Sensing, Vol. XXXI, Part B5,Vienna, Austria, 1996, pp.209-294.

MARTEN, W., MAUELSHAGEN, L., PALLASKE, R.: Digital or-thoimage-system for architecture representation. ISPRSJournal of Photogrammetry and Remote Sensing, 49 (5),1994, pp.16-22.

WEHR, A.: Abbildende Laserscanner – Anwendungen inBauaufnahme und Denkmalpflege. In: Architekturphoto-grammetrie gestern - heute - morgen. Eds.: Albertz, J.and Wiedemann, A., Technical University of Berlin,pp.114-127.

WIEDEMANN, A.: Digital Orthoimages in Architectural Pho-togrammetry using Digital Surface Models. InternationalArchives of Photogrammetry and Remote Sensing, Vol.XXXI, Part B5, Vienna, Austria, 1996, pp.605-609.