optimizing the use of digital airborne images for 2.5d ......we would like to thank: fred hagman and...
TRANSCRIPT
![Page 1: Optimizing the use of digital airborne images for 2.5D ......We would like to thank: Fred Hagman and Willem Philipse (Aerodata International Surveys, Deurne, Belgium), Gunnar Bostro¨m](https://reader035.vdocument.in/reader035/viewer/2022062612/613673030ad5d206764808ba/html5/thumbnails/1.jpg)
Optimizing the use of digital airborne images
for 2.5D visualization
A.S. Sima & S.A.W. KayAgriculture and Fisheries Unit, Institute for Protection and Security of the Citizen,Joint Research Centre of the European Commission, Ispra (Va), Italy
Keywords: monoscopic rendering, visualizations, photogrammetry
ABSTRACT:Monoscopic virtual representations of 3Dgeometries are rapidly becomingimportant products of many databases and software applications. Many GIS tools – evenfreeware, such as Google Earth – permit the visualization of city planningmodels as wellas landscapes derived from 3D geometries (digital surface models draped with imagery,called 2.5D visualization).We propose that off-nadir parts of vertical imagery – typicallyignored after the orthorectification process – provide us systematically with much datathat can be used to optimize the 2.5D rendering process.We attempt to use nominally vertical imagery, collected under typical photogrammetric
conditions, to improve the rendering of vertical surfaces and facades. Results show imagequality at least equivalent to state of the art 2.5D rendering where standard “most nadir”vertical orthoimages are used.
1 INTRODUCTION
Virtual views of rural or urban landscapes presented in 2.5D viewing – that is,monoscopic rendering of imagery draped over a detailed surface model – came of age in2005.On-line services such asGoogle Earth have placed some level of 2.5D capability ondesktop computers, albeit still at a fairly generalized level of texture detail, with freesoftware and standard Earth observation imagery (satellite, airborne).However, all state of the art software and service providers of this nature use Earth
observation data that has been orthorectified, i.e. geometrically corrected to someprojection system and sampled so as to choose the most nadir viewing image available.Nevertheless, more and more imagery is acquired with multiple views – sometimes asmany as 6 or more – that are rarely used in the final orthoimage product or 2.5D renderedviewing. These data offer a potentially rich source for the optimization of textures used inrendering of any image products, either orthoimages or 2.5D rendered views.The products – 2D orthoimages with near-perfect geometry, and 2.5D visualizations
using the highly detailedDSMs – are impressive, and have quickly become economicallycompetitive data sources for large scale urban applications such as land registry updating.Fully orthorectified imagery flight plans, however, produce an enormous amount of
redundant data. Flightswith80%forwardand60%sidelapmayproduce asmanyas15pixelsfor a given point on the ground. Furthermore, the restriction of using only the most nadir
609
GeoInformation in Europe, M.A. Gomarsca (ed.)
�2007 Millpress, Netherlands, ISBN 9789059660618
![Page 2: Optimizing the use of digital airborne images for 2.5D ......We would like to thank: Fred Hagman and Willem Philipse (Aerodata International Surveys, Deurne, Belgium), Gunnar Bostro¨m](https://reader035.vdocument.in/reader035/viewer/2022062612/613673030ad5d206764808ba/html5/thumbnails/2.jpg)
viewing pixels causes 2.5D rendering to be “smeared” over vertical surfaces (perpendicularto the orthogonal projection plane), such as building facades. We propose instead to exploitthe remaining data, and instead of using only (near) nadir viewing imagery as a data input, tochoose those pixels with an optimized geometry for the rendering process. In our approach,the optimizedpixel is defined as the pixel corresponding to theviewanglemost orthogonal tothe DSM facet that models the true object surface. Figure 1 presents a simplified schema forchoosing the optimal pixel from a set of three images.The studywe present here is more aligned to a standard photogrammetric approach, by
contrast with other work in the computer vision field. We attempt to use nominallyvertical imagery, collected under typical photogrammetric conditions, to improve therendering of vertical surfaces and facades. Image orientation is achieved only through theuse of a precise camera model (including metric calibration information) and camerapositions are provided by the aerotriangulation of a photo block. TheDSM is furthermorealso created using a standard photogrammetric approach and is of a qualitycommensurate with that used for 2D orthoimage production.
2 STUDY DATASET
The images used in this studywere collected inMaussane lesAlpilles, France, on the 24thofMay 2005 byAerodata International Surveys (Deurne, Belgium).AVexcelUltraCamDdigital frame camera was used to collect 80 images in 5 strips, with 80% forward overlapand 60% side overlap.
Figure 1. Simplified schema demonstrating the process of optimizing pixel selection.
610 A.S. Sima & S.A.W. Kay
![Page 3: Optimizing the use of digital airborne images for 2.5D ......We would like to thank: Fred Hagman and Willem Philipse (Aerodata International Surveys, Deurne, Belgium), Gunnar Bostro¨m](https://reader035.vdocument.in/reader035/viewer/2022062612/613673030ad5d206764808ba/html5/thumbnails/3.jpg)
The flight plan was essentially of classical photogrammetric form (Graham and Koh2002), with nominally vertical images and a flying height of around 5000 m aboveground level. The data used for prototype development were four pansharpenedmultispectral image frames (henceforth termed candidate images), with a groundsampling distance of approximately 0.5m. Exterior orientation ancillary data derivedfrom a standard aerotriangulation process were also made available, comprising thecenters of projection (in UTM projection zone 31N, WGS-84 ellipsoid), camerarotation angles, as well as the calibration report of UltraCamD (Vexcel serial no.18)camera.The terrain model used in this exercise was a digital surface model prepared by
ISTAR S.A. (Sophia Antipolis, France) with a 2m raster cell size. It was extracted(using spatial autocorrelation techniques) from a Leica Geosystems ADS40 airbornesensor image set, acquired 14th of May 2003. The flight altitude was also approximately5000 m with a GSD of 0.5m. The vertical accuracy of this DSM was assessedindependently (Spruyt and Kay 2004) and as RMSEz, on well defined points, wascomputed to be 0.34m.
3 ALGORITHM OPTIMIZATION
The methodology can be divided into four steps:
1. Absolute orientation of candidate image arrays: The main goal of this part is tocalculate coordinates of each pixel of each candidate image frame in the project globalcoordinate system (UTM, zone 31N, ellipsoid WGS-84);
2. DSM processing: The DSM is reprojected back into the candidate image plane; eachtriangular facet of the DSM is represented as an irregular network in the candidateimage space.
3. The information concerning the DSM is used to determine which candidate imageframe is best suited for rendering into the 2.5D scene;
4. Finally, the image rendering is undertaken in a specialized visualization softwarepackage (Reconstructor) using the virtual image files.
3.1 Absolute orientation
Each pixel on the candidate image frame is assigned geodetic coordinates in the worldcoordinate system, using the classical colinearity equation, interior and exteriororientation parameters.In a typical exterior orientation model (built using at least two images), the scale
coefficient l describes the difference between the model and terrain scaling. In our(single) image exterior orientation, l is always equal to 1; there is no change ofscale, because the goal is to know the geodetic coordinates of each image pixel onthe image plane and then be able to rebuild the image geometry at the moment ofacquisition.
611Optimizing the use of digital airborne images for 2.5D visualization 611
![Page 4: Optimizing the use of digital airborne images for 2.5D ......We would like to thank: Fred Hagman and Willem Philipse (Aerodata International Surveys, Deurne, Belgium), Gunnar Bostro¨m](https://reader035.vdocument.in/reader035/viewer/2022062612/613673030ad5d206764808ba/html5/thumbnails/4.jpg)
3.2 DSM processing
In this step we find a correspondence between the DSM and the candidate images usingexternal orientation parameters. The DSM array (all pixels) is reprojected on the imagesplanes, centre of this projection is identical with the centre of projection of each ofcandidate images.The reprojection of the DSM array to the plane has some disadvantages. The
reprojection is calculated for all the DSM cells from array, even those which are notvisible from the center of projection (for example, on facets facing away from the cameraposition or occluded from view by tall objects). These obstructed pixels need to beidentified and eliminated. We implemented a simple algorithm that evaluates thereprojected position of each DSM cell on the image plane, and determines if adjacentpixels are obstructed.Finally, the now irregular grid of reprojected DSM data must be networked as
triangular facets. Only those pixels visible from the center of projection of each exposureare selected for this process.When for all vertices of triangles on the ground corresponding coordinates on the
image plane are found, texture extraction becomes possible.
3.3 Integrating DSM and candidate images
To extract the texture from candidate imagewe have to find position of all vertices in eachtriangle on the ground in the pixel coordinate system [column, row] of the image. This isdone by searching the position of all reprojected vertexes in the regular array of pixelscoordinates created in the Step 1.To determine which candidate image is the best choice for texturing any individual
DSM triangular facet, it is necessary to compare geometry of acquisition (view anglefrom the center of projection to the center of the triangular facet) of each candidate imagewith the normal to this triangle.The best texture is the candidate image pixel with a view angle most orthogonal to the
DSM facet, so the angle between the normal vector and the ‘view on triangle’ vectorshould be as close to 180� as possible (see Figure 1).
3.4 Image rendering
The algorithm was implemented in Interactive Data Language (RSI). Use was made of aLinuxoperating systemon aPCplatformdue to better possibilities ofmemorymanagement,a critical factor with the processing of the large datasets (thewhole imagematrix consists of11500� 7500 pixels, double-precision data). Processing times on a 2.4GHz machine werequite demanding, in the order of 6 h for image sets of only 800� 800 pixels.The resulting output files from our prototype algorithm are then processed using a
script written in MatLab programming language (G. Bostrom, pers. comm.), whichprepares files readable for the visualization software Reconstructor (Topotek 2005). Theexamples presented in the paper all use this software for rendering the 2.5D visualization.
612 A.S. Sima & S.A.W. Kay
![Page 5: Optimizing the use of digital airborne images for 2.5D ......We would like to thank: Fred Hagman and Willem Philipse (Aerodata International Surveys, Deurne, Belgium), Gunnar Bostro¨m](https://reader035.vdocument.in/reader035/viewer/2022062612/613673030ad5d206764808ba/html5/thumbnails/5.jpg)
4 RESULTS
The prototype algorithm was able to correctly select candidate images from the testdataset, and make virtual images available for rendering in the visualization softwareenvironment. Examples of the test results can be viewed in Figures 3 and 4 below.Figure 3 showsa typical rural landscape from theMausannedataset.The raisedobjects in
theDSMhave been rendered very accurately with the appropriate image data. The texturesappear to be very compatiblewith adjacent pixels derived from different candidate images,and no special artifacts are immediately visible in the 2.5D virtual image.Figure 4 shows a pseudo-colored image, with each of four colors representing one of
the input candidate image frames. It can be easily seen that raised objects (hedges, copses)have acquired texture from up to four images, which is rendered in accordance with theaspect and slope of the DSM fact being processed.
DSM Build an imagearray
Interiororientation
Externalorientation
Step 2
Step 1
Step 4
Image
Input data
Output data
LEGEND:
Image size: 12500 x 7500 pixPixel size: 9 µ
Focal length: 101,4 mmDistortion values
(X0, Y0, Z0, ω, ϕ, χ)External orientation parametersSelect the part
of DSM commonfor all candidate
images
Reproject the DSMonto image plane
(pinholeprojection)
Eliminate
Triangulate the data
search
column, row
Rendering,Reconstructor
Texture ID
Determinate theOptimum candidateimage to use as a
texture for the triangle
obstructed pixels
X, Y, Z X, Y, Z
Step 3
Figure 2. Simplified schema demonstrating the process flow of optimizing pixel selection.
Optimizing the use of digital airborne images for 2.5D visualization 613
![Page 6: Optimizing the use of digital airborne images for 2.5D ......We would like to thank: Fred Hagman and Willem Philipse (Aerodata International Surveys, Deurne, Belgium), Gunnar Bostro¨m](https://reader035.vdocument.in/reader035/viewer/2022062612/613673030ad5d206764808ba/html5/thumbnails/6.jpg)
Although these images show the successful execution of the optimization algorithm, thegeneralization due to the DSM cell size (2m) still causes a large degree of smoothing in therendered image. Furthermore, candidate images were not chosen optimally with respect tooverlap, resulting in a set of candidate images that in fact were near-vertical viewing.
Figure 3. The optimized 2.5D view sample: texturing with real images. The image and view point
is identical to Figure 5.
Figure 4. In this graphic, each color is a substitute of the texture from a one of the four candidate
images.
614 A.S. Sima & S.A.W. Kay
![Page 7: Optimizing the use of digital airborne images for 2.5D ......We would like to thank: Fred Hagman and Willem Philipse (Aerodata International Surveys, Deurne, Belgium), Gunnar Bostro¨m](https://reader035.vdocument.in/reader035/viewer/2022062612/613673030ad5d206764808ba/html5/thumbnails/7.jpg)
In order to try and overcome these circumstances – aswell as to test the implementationon an independent set of candidate images not used for the development – a second dataset(Brasschaat, Belgium), acquired by Aerodata International, was processed using theprototype algorithm implementation. The input data were similar to the original test site,acquired with the same digital camera; however, image scale was larger (20 cm groundsampling distance) and with a 1m DSM (autocorrelation on the original imagery).Moreover, the scene is composed of an urban environment, with numerous buildings aswell as landscape objects such as trees and bushes. The result of the 2.5D rendered imagecan be seen in Figure 5.Again, the algorithm performs satisfactorily with respect to the general selection and
positioning of image texture.However, theDSMis again rather smooth, causing the detailof buildings to be disrupted. Although some facades appear to be rendered with texturecolors differing from those used for building roofs, in fact there is still very little data onvertical surfaces available for the algorithm in such a flight configuration.
5 CONCLUSIONS
We have demonstrated here a proof of concept algorithm for the optimization of imageryfor 2.5D visualization.The algorithm has been implemented in a standard image processing tool and is able to
operate on frame camera imagery, such as the Vexcel UltraCamD. Results show imagequality at least equivalent to state of the art 2.5D rendering where standard “most nadir”vertical orthoimages are used.Limitations of the results so far are due to the rather vertical nature of the datasets
tested, rather than any specific issues with the algorithm itself. The visualization resultsare still inferior to datasets where either terrestrial imagery has been used for facades (forexample, Bostrom et al. 2004), or where specially acquired oblique airborne imagery isintegrated in the rendering process (Fruh et al. 2004).We believe that our results would be
Figure 5. Result of the supplementary trial with higher resolution dataset (Brasschaat, Belgium).
Optimizing the use of digital airborne images for 2.5D visualization 615
![Page 8: Optimizing the use of digital airborne images for 2.5D ......We would like to thank: Fred Hagman and Willem Philipse (Aerodata International Surveys, Deurne, Belgium), Gunnar Bostro¨m](https://reader035.vdocument.in/reader035/viewer/2022062612/613673030ad5d206764808ba/html5/thumbnails/8.jpg)
improved if imagery were acquired with a wider field of view and from a lower flyingheight; we conclude that the algorithm would nonetheless perform effectively with suchdata since there is no fundamental change in the geometry of image acquisition.Additionally, a more refined DSM would significantly improve visualization results.Many areas for optimization of the algorithm implementation remain. Data processing isslow due to lack of triangulation optimization, requiring around 6 h to prepare theoptimized 2.5D visualization of an area of 800 · 800 pixel image sample. We considerthat after further program optimization, processing time would be cut by 2/3rds.
ACKNOWLEDGEMENTS
We would like to thank: Fred Hagman and Willem Philipse (Aerodata InternationalSurveys, Deurne, Belgium), Gunnar Bostrom and Vitor Sequeira (Nuclear SafeguardsUnit, JRC Ispra, Italy), Giovanni Banchini (CGR S.p.A., Parma, Italy), Arthur Rohrbachand Udo Tempelmann (Leica Geosystems AG, St. Gallen, Switzerland).
REFERENCES
Bostrom, G., Fiocco, M., Puig, D., Rossini, A., Goncalves, J.G.M. & Sequeira, V. 2004. Acquisition,Modelling and Rendering of Very Large Urban Environments. Proc. of the Second InternationalSymposium on 3DData Processing, Visualization and Transmission, Thessaloniki, Greece, Sept6–9, 2004: 191–198.
Devos, W., Kay, S. 2005. Assessing Geometric Accuracy of Ortho-imagery: Towards ProductSpecification ofFullyOrthorectified Imagery forUrbanAreas,GIMInternational 19 (6): 14–17.
Fruh, C., Sammon, R. & Zakhor, A. 2004. Automated Texture Mapping of 3D City Models WithOblique Aerial Imagery. Proceedings of the Second International Symposium on 3D DataProcessing, Visualization and Transmission, Thessaloniki, Greece, Sept 6–9, 2004: 191–198.
Graham, R., Koh, A. 2002. Digital Aerial Survey: Theory and Practice, CRC Press, Boca Raton,Florida.
Spruyt, P., Kay, S. 2004. Quality assessment of digital airborne orthoimagery: A test study usingthe Leica Geosystems ADS40, GIM International 18 (6) 35–37.
Topotek, 2005. RECONSTRUCTOR: The laser data processing package, URL: http://www.topotek.it/modules.php?name=Content&pa=showpage&pid=14 Topotek, Consortium forTechnology Innovation, c/o Engineering–University of Brescia, 38, Branze, I–25123 Brescia,Italy (last date accessed: 22 Dec 2005).
616 A.S. Sima & S.A.W. Kay