Full Terms & Conditions of access and use can be found athttp://www.tandfonline.com/action/journalInformation?journalCode=tgis20

Download by: [Arizona State University Libraries] Date: 16 June 2016, At: 09:08

International Journal of Geographical InformationScience

ISSN: 1365-8816 (Print) 1362-3087 (Online) Journal homepage: http://www.tandfonline.com/loi/tgis20

Embedding user-generated content into obliqueairborne photogrammetry-based 3D city model

Jianming Liang, Shen Shen, Jianhua Gong, Jin Liu & Jinming Zhang

To cite this article: Jianming Liang, Shen Shen, Jianhua Gong, Jin Liu & Jinming Zhang(2016): Embedding user-generated content into oblique airborne photogrammetry-based 3D city model, International Journal of Geographical Information Science, DOI:10.1080/13658816.2016.1180389

To link to this article: http://dx.doi.org/10.1080/13658816.2016.1180389

Published online: 29 Apr 2016.

Submit your article to this journal

Article views: 56

View related articles

View Crossmark data

Embedding user-generated content into oblique airbornephotogrammetry-based 3D city modelJianming Liang a,b, Shen Shena,c, Jianhua Gonga,b, Jin Liua,c,d and Jinming Zhanga,e

aState Key Laboratory of Remote Sensing Science, Institute of Remote Sensing and Digital Earth, ChineseAcademy of Sciences, Beijing, China; bZhejiang-CAS Application Center for Geoinformatics, Jiashan, China;cUniversity of Chinese Academy of Sciences, Beijing, China; dNational Marine Data and Information Service,Tianjin, China; eInstitute of Geospatial Information, Information Engineering University, Zhengzhou, China

ABSTRACTOblique airborne photogrammetry-based three-dimensional (3D)city model (OAP3D) provides a spatially continuous representationof urban landscapes that encompasses buildings, road networks,trees, bushes, water bodies, and topographic features. OAP3D isusually present in the form of a group of unclassified triangularmeshes under a multi-resolution data structure. Modifying such anon-separable landscape constitutes a daunting task becausemanual mesh editing is normally required. In this paper, we pre-sent a systematic approach for easily embedding user-generatedcontent into OAP3D. We reduce the complexity of OAP3D mod-ification from a 3D mesh operation to a two-dimensional (2D)raster operation through the following workflow: (1) A region ofinterest (ROI) is selected to cover the area that is intended to bemodified for accommodating user-defined content. (2) Spatialinterpolation using a set of manually controlled elevation samplesis employed to generate a user-defined digital surface model(DSM), which is used to reform the ROI surface. (3) User-generatedobjects, for example, artistically painted road textures, procedu-rally generated water effects, and manually created 3D buildingmodels, are overlaid onto the reformed ROI.

ARTICLE HISTORYReceived 25 November 2015Accepted 14 April 2016

KEYWORDSOblique airbornephotogrammetry; 3D citymodel; 2D editing;user-generated content

1. Introduction

Modeling urban space in three dimensions is important for urban planning and urbanenvironmental analysis (Köninger and Bartel 1998). Data acquisition, data processing,and visualization technologies for 3D geospatial data have enhanced the process of 3Dcity modeling (Moser et al. 2010), which lays the foundation for a wide range of urbanapplications including urban climate simulation (Strzalka et al. 2011), urban planning(Wu et al. 2010), and visibility calculation (Yasumoto et al. 2011, 2012). An increasingnumber of applications and systems incorporate 3D city model as an integral compo-nent to serve urban planning and redevelopment, facility management, logistics, secur-ity, telecommunications, disaster management, location-based services, real estateportals as well as urban-related entertainment and education products (Döllner et al.

CONTACT Jianhua Gong gongjh@radi.ac.cn

INTERNATIONAL JOURNAL OF GEOGRAPHICAL INFORMATION SCIENCE, 2016http://dx.doi.org/10.1080/13658816.2016.1180389

© 2016 Informa UK Limited, trading as Taylor & Francis Group

Dow

nloa

ded

by [

Ari

zona

Sta

te U

nive

rsity

Lib

rari

es]

at 0

9:08

16

June

201

6

2006). An urban 3D GIS can potentially be extended to a virtual geographic environ-ment, which is a type of workspace for computer-aided geographic experiments andgeographic analyses with support for geo-visualization, geo-simulation, geo-collabora-tion, and human participation (Lin et al. 2013).

Traditionally, labor-intensive computer-aided design (CAD) modeling (Figure 1(b)) isemployed to manually create realistic 3D cities (Liang et al. 2003). Building footprints fromcadastral survey can also be extruded into geometrically simple polyhedra (Figure 1(a)),though with a lack of textural details. LiDAR has become a popular technology forobtaining accurate 3D geometric information (Rottensteiner and Briese 2002), yet thelack of textural information can be overcome only through aerial photography.

Oblique airborne photogrammetry is a recently developed solution for rapid andaccurate 3D city reconstruction at an acceptable cost. It has become an increasinglymature means for 3D city acquisition with the rapid progress in data acquisition anddata processing technologies. Affordable unmanned airborne vehicles and sensors cannow be easily integrated into a robust system to acquire multi-angle oblique imagery(Remondino et al. 2011). Compared to traditional image acquisition systems, unmannedairborne vehicles have demonstrated many advantages such as low cost, high flexibility,and fine image resolution. Photo-based automatic 3D city reconstruction, which isdriven primarily by computer vision research, has been standardized and streamlinedby leading industry solution providers. A commercial 3D reconstruction solution, such asAcute3D’s Smart3D, Airbus’s Street Factory, or Skyline’s Photomesh, can now

Figure 1. Different sources of 3D city model: (a) building footprint extrusions, (b) CAD modeling, and(c) OAP3D.

2 J. LIANG ET AL.

Dow

nloa

ded

by [

Ari

zona

Sta

te U

nive

rsity

Lib

rari

es]

at 0

9:08

16

June

201

6

automatically generate a highly realistic 3D urban landscape (Figure 1(c)) from anunorganized set of overlapping photos, without requiring any expert knowledge ofcomputer vision or GIS.

Several leading 3D reconstruction solution providers, such as Acute3D (acquired byBentley in January 2015), Airbus, and Skyline, have implicitly converged on a sharedmulti-resolution data model for efficient storage, streaming, and rendering of OAP3D.This data model is becoming a de facto industry standard widely accepted by relevantsolution providers and application developers. An OAP3D exported from any of thethree solutions can be imported directly into the open-source 3D rendering engineOpenSceneGraph for interactive navigation with built-in support for out-of-core datapaging. Similarly, Skyline offers seamless integration of OAP3D to its commercial virtualglobe platform Skyline Globe. With these sophisticated data acquisition, data processing,and data visualization solutions, a solid foundation has already been established forOAP3D to be exploited in 3D urban GIS applications in fields such as urban planning,traffic management, cadastral survey, and emergency management. These are keycomponents in developing next-generation smart cities (Tao 2013) to provide enhancedurban services.

While the multi-resolution data model shared by the leading OAP3D solution provi-ders has been highly optimized for data streaming and interactive visualization, it hasnot yet been sufficiently prepared for use in real-world 3D urban GIS applications. A 3Durban GIS application is usually built on the combination of multiple data sources, whichtypically consist of imagery, elevation, road networks, stream networks, points of inter-est, and other user-generated features. Integrating these data sources with an OAP3Dcan be challenging, since mesh re-triangulation (Vivoni et al. 2005) or data conflationissues (Kreveld and Silveira 2011) might be involved. To begin with, we introduce a fewcommon scenarios in which such data integration issues would be faced:

(1) Overlaying manually created 3D building models over an OAP3D (Figure 2). Urbanplanners sometimes might need to replace a city block within an OAP3D with aconceptual design exported from CAD. Urban residents who intend to build orrebuild a house might also want to merge an architectural design into an OAP3Dbackground to help aesthetic quality assessment.

(2) Removing trees from an OAP3D (Figure 3). As the structure of a tree is geome-trically very complex, it cannot normally be reconstructed using aerial photo-grammetry alone at the same level of quality as a building is done. As a result, itmight sometimes be necessary to remove the areas covered by trees to makeroom for artificially created objects of better visual quality.

(3) Merging user-generated roadways or structures into an OAP3D (Figure 3). Streetsmay be partially hidden by tree canopies, leading to bumpy surfaces. Roadwidening or new road construction work is not uncommon in cities that areexperiencing rapid urban development and population increase. Undergroundcivil engineering may also necessitate modification to OAP3D.

While robust industrial solutions for acquisition and distribution of OAP3D has beenestablished, directly utilizing such data in 3D urban GIS applications would still bechallenging. Hence, it is arguably necessary to develop a practical framework for easily

INTERNATIONAL JOURNAL OF GEOGRAPHICAL INFORMATION SCIENCE 3

Dow

nloa

ded

by [

Ari

zona

Sta

te U

nive

rsity

Lib

rari

es]

at 0

9:08

16

June

201

6

integrating OAP3D into 3D urban GIS applications with support for user-generatedcontent. This framework is intended to facilitate utilization of the increasingly largevolume of OAP3D data, which typically constitute the foundation of a 3D urban GISapplication.

This paper is organized as follows. Section 2 provides a detailed description of themulti-resolution data structure shared by the leading industrial solution providers.Section 3 presents the conceptual framework and its implementation for modifying

Figure 2. Potential need to replace individual building in OAP3D.

Figure 3. Potential need to remediate road surface that is covered by tree canopies in OAP3D.

4 J. LIANG ET AL.

Dow

nloa

ded

by [

Ari

zona

Sta

te U

nive

rsity

Lib

rari

es]

at 0

9:08

16

June

201

6

OAP3D and incorporating user-generated content. Section 4 presents some applicationexamples to demonstrate the applicability of this framework. Section 5 summarizes thecontributions of the presented method.

2. The shared data model

Manual 3D modeling in CAD remains a mainstream strategy for 3D city reconstruction. Amanually created 3D city is typically composed of a group of building models accom-panied with a variety of natural or artificial landscape features, for example, waterbodies, trees, and street lights. These 3D models are normally represented by an arrayof triangular meshes associated with material colors or textures. Since a 3D city canencompass thousands of buildings and a large number of accessories, a consumer-levelcomputer may not be able to accommodate the whole city for real-time rendering. Toovercome this bottleneck, the CityGML data model was introduced (Kolbe et al. 2005)and later approved by the Open Geospatial Consortium. In a CityGML-based 3D city,building objects can have multiple levels of details. With these multi-resolution buildingmodels and a reasonable view-dependent data streaming strategy, users can smoothlyexplore a large-scale 3D city on a computer of limited power. One of the majordisadvantages accompanying CityGML is that the multi-resolution data content of a3D city requires a substantial amount of labor to create.

Unlike traditional manual 3D modeling which is labor intensive, OAP3D can reducehuman intervention to an affordable level by means of unmanned airborne vehicle-based image acquisition and photo-based 3D reconstruction. A commercial 3D recon-struction solution can automatically transform a set of non-georeferenced obliqueimages into a spatially continuous triangular mesh that encompasses the ground,buildings, plants, and everything else that is visible from an unmanned airborne vehi-cle-borne camera.

Due to the huge volume of overlapping images captured from unmanned airbornevehicles for 3D reconstruction, a large urban area usually needs to be spatially parti-tioned into a grid of cells or sub-extents, each of which is utilized to extract a subset ofimages to generate a continuous triangular mesh bounded by it, and the sub-extentshould be limited in space so that the data volume of the subset of images inside maynot overburden the 3D reconstruction software or the hardware system on which it isrunning. As such a continuous 3D mesh are densely loaded with geometric and texturalinformation, a multi-resolution data model must be used in order to facilitate interactiveexploration. The CityGML data model, however, is not directly applicable to thesemeshes because they cannot automatically be segmented into individual buildings,which are fundamental elements comprising a CityGML city.

Across an OAP3D, a separate tree-based level-of-detail structure is used for each sub-extent, in which a large continuous mesh is partitioned into a given number of sub-tilesat each tree level, with each level hosting progressively downsampled geometric andtextural data from the bottom up (Figure 5). Structurally similar to an image pyramid,this type of level-of-detail mesh actually functions like a ‘mesh pyramid’. Normally, thetop tree level is split into only one or a few sub-tiles that are in the most simplified formacross the whole range of tree levels. From the top down, each parent tile is furtherdivided in one or more sub-tiles with higher resolution meshes and textures present

INTERNATIONAL JOURNAL OF GEOGRAPHICAL INFORMATION SCIENCE 5

Dow

nloa

ded

by [

Ari

zona

Sta

te U

nive

rsity

Lib

rari

es]

at 0

9:08

16

June

201

6

(Figure 5). Consequently, the bottom level has the largest number of sub-tiles, which areleaf nodes loaded with the most finely scaled meshes and textures commensurate withthe original data resolution.

This data model is highly optimized for real-time rendering and data streaming. A 3DGIS application can even dynamically manage terabytes of OAP3D data for interactivevisualization without causing a significant decrease in rendering performance.

To demonstrate how this data model performs in 3D urban applications with massivecity models, a test was performed with an OAP3D that covers a downtown area of45 km2 with a data volume of approximately 64.4 GB. This OAP3D was generated usingSkyline’ Photomesh with an image resolution of approximately 10–20 cm. The datasetwas loaded into a 3D interactive visualization system on a consumer-level desktopcomputer, which has a 2.90-GHz quad-core Intel Core i52310 CPU with 4 GB RAM anda NVIDIA GeForce GTS 450 graphics card with 1.0 GB RAM. The real-time renderingperformance and quality was evaluated at a progressively closer range of viewingdistance. As shown in Figure 4, the level-of-detail transition from far away to close upwas smooth and nearly seamless, and the rendering performance at any of the fourviewing positions was maintained between 90 and 140 frames per second with anaverage of 120, which is sufficient for most 3D GIS applications to accommodate otherGIS-related functionalities.

The standardized data model may have been sufficient for serving a visualization-centered 3D GIS application that utilizes OAP3D as a standalone data source, but it is notwell prepared for use in a 3D urban GIS application that requires integration of OAP3Dand other types of data sources.

Modifying OAP3D meshes in CAD is very complicated and thus time consuming.Because an OAP3D comprises a grid of sub-extents with independent mesh pyramids, abuilding may straddle multiple neighboring mesh pyramids across an OAP3D or multiple

Figure 4. Smooth level-of-detail transition of a massive OAP3D from far away to close up in real-time rendering at an average of 120 frames per second.

6 J. LIANG ET AL.

Dow

nloa

ded

by [

Ari

zona

Sta

te U

nive

rsity

Lib

rari

es]

at 0

9:08

16

June

201

6

neighboring mesh tiles within a single mesh pyramid (Figure 5). To edit a straddlingbuilding in CAD, these neighboring meshes must be separately processed in a way thatthey can be seamlessly stitched back together. To avoid this challenge that cannot beefficiently managed in processing large-area or geometrically complex features, such asroadways, streams, forested land, and buildings, we propose that a GIS-integrated 2Dediting approach be used.

3. Method

The shared data model of OAP3D is characteristic of both 2D DSMs and 3D triangulatedmeshes in that, for a top-down view it resembles an urban height field overlaid with adigital orthophoto map (DOM), while for a side view it becomes a 3D mesh model withmulti-angular textural information. Inspired by this understanding, we propose a newmethod that reduces the complexity of OAP3D modification from 3D mesh editing to 2Draster editing (Figure 6). The method is based on two assumptions: (1) an OAP3D meshcan be reformed as a 2D height field by updating the height value of each vertex, as inmost cases we do not have to view a target area as a static 3D mesh. For example, toreplace an existing 3D building from an OAP3D with a newly designed model, we needonly to lower the height field of the ROI to the ground level; (2) user-defined objects, forexample, 3D buildings, trees, or road segments can be generated externally and thenseamlessly merged into the reshaped mesh area at a later stage.

3.1. Orthoimagery generation

To enable surface elevation and texture editing in a 2D graphic environment, forexample, in ArcGIS or Photoshop, an OAP3D (Figure 7(a)) needs to be transformed

Figure 5. An OAP3D is partitioned into a grid of mesh pyramids to support level-of-detail interactivevisualization.

INTERNATIONAL JOURNAL OF GEOGRAPHICAL INFORMATION SCIENCE 7

Dow

nloa

ded

by [

Ari

zona

Sta

te U

nive

rsity

Lib

rari

es]

at 0

9:08

16

June

201

6

into a DOM (Figure 7(b)) and a DSM (Figure 7(c)). In computer graphics, there is atechnique known as ‘Render to Texture’. A CAD software application, for example, 3dsMax, may be able to utilize this technique to render 3D models onto a 2D image basedon a pre-configured virtual camera, and if the virtual camera is set to top-down view

Figure 6. Workflow for integrating user-generated content into OAP3D.

Figure 7. Generating DOM and DSM from OAP3D to facilitate 2D editing in a GIS.

8 J. LIANG ET AL.

Dow

nloa

ded

by [

Ari

zona

Sta

te U

nive

rsity

Lib

rari

es]

at 0

9:08

16

June

201

6

with an orthographic projection, the 2D image mosaic produced may serve as a DOM.There are several GIS-related issues, however, these are not considered in CAD software.For example, CAD software cannot normally accommodate the level-of-detail datastructure of an OAP3D and thus may not be able to handle the large data volumeassociated with such a 3D city. Moreover, CAD software such as 3ds Max is not able togenerate a DSM, because the ‘Render to Texture’ technique in CAD is intended only forproducing colored images. Inspired by the ‘Render to Texture’ technique used in CAD,OpenSceneGraph is used here as a 3D rendering engine for transforming OAP3D into 2Dorthoimages including DOM (Figure 7(b)) and DSM (Figure 7(c))

The OpenSceneGraph engine is widely used by application developers in fieldssuch as visual simulation, games, virtual reality, scientific visualization, and modeling.With an application developed under the framework of OpenSceneGraph, the geo-metric and textural data are dynamically loaded from disk files or web streams intosystem memory, and then transferred to video memory for rendering by graphicsprocessing units.

Similarly, a large urban scene also needs to be spatially partitioned into a grid of sub-extents before it is rendered onto images due to texture size and video memorylimitations, although this space partitioning may be independent of the internal gridstructure of the OAP3D being processed. An image of a given width and height isallocated for each sub-extent, and this image is defined as a sub-image, which covers asub-extent of the full scene area. For instance, given a scene area of 2 × 2 km2, a spatialresolution of 10 cm and a sub-image size of 1024 × 1024, approximately 200 × 200 sub-images would be required to fill the grid of the scene.

Normally, OpenSceneGraph outputs its rendering results as 3-byte or 4-byte colorsto the screen for immediate display. Therefore, the Render Target technique and theOpenGL shading language are employed to withhold the color and position datafrom the graphics processing unit buffers for user-defined output. A Render TargetTexture is allocated to store the 3-byte color data for the DOM and an additionalfloating point Render Target Texture is used for the DSM. The two Render TargetTextures are attached to an orthographic camera for storing the DOM and DSM. Inthe shading fragment program, the color data are written to the DOM render target(Figure 7(b)) and the floating point height data are written to the DSM render target(Figure 7(c)).

3.2. ROI delineation

An ROI is a polygon which serves to define the boundary of an area that is intended tobe modified (Figure 8). For example, if a lake needs to be removed from an OAP3D, itsboundary will be manually delineated to provide reference for the following surfaceelevation or texture editing.

3.3. DSM modification

OAP3D meshes can be reformed using a 2D approach, which can conveniently beperformed in a GIS environment. A DSM can be modified using two types of techniques,which include:

INTERNATIONAL JOURNAL OF GEOGRAPHICAL INFORMATION SCIENCE 9

Dow

nloa

ded

by [

Ari

zona

Sta

te U

nive

rsity

Lib

rari

es]

at 0

9:08

16

June

201

6

(1) Surface elevation sampling and spatial interpolation. To erase an above-groundobject from an OAP3D, surface elevations at the exposed ground immediatelysurrounding this object are sampled and then spatial interpolation is applied toreconstruct the ground surface hidden under this object. To flatten the surface of awater body, which was distorted due to lack of feature points in the process of 3Dreconstruction, elevation points that are considered within a reasonable range ofheight values may be sampled for spatial interpolation. The values of these elevationsamples may also be interactively adjusted in GIS based on prior knowledge.

(2) Flattening, which refers to applying a uniform elevation value to form a planarsurface. In some cases, the surface that is covered by an ROI can be assumed to beflat. For example, a lake is normally a planar surface. The flattening technique mayalso be utilized to place a building on an undulating surface.

3.4. DOM modification

An OAP3D is a complex landscape comprising bare earth, buildings, water bodies, roads,and vegetation, which are represented through a combination of geometric and texturalinformation. However, certain types of landscape elements can be sufficiently character-ized by texture alone, for example, water surfaces. Therefore, modifying the DOM alonethrough 2D texture editing may effectively change the way a landscape elementappears on an OAP3D. Image processing software such as Photoshop can be employedto edit a texture cropped from a DOM using ROI masking.

Figure 8. Delineating ROI in a GIS based on the DOM and DSM generated from the OAP3D.

10 J. LIANG ET AL.

Dow

nloa

ded

by [

Ari

zona

Sta

te U

nive

rsity

Lib

rari

es]

at 0

9:08

16

June

201

6

3.5. Overlaying user-generated 3D models on ROI

DSM modification can be conveniently performed in a GIS to reform an OAP3D meshfrom a 2D perspective. In real-world applications, however, a user may intend to overlayonto an OAP3D externally sourced 3D models, for example, building models that aremanually created in CAD or procedurally generated using certain rules according tourban designs.

3.6. Dynamic surface elevation and texture updating

The modified DOM and DSM will eventually need to be embodied in the OAP3D duringinteractive rendering. Since these types of DOM and DSM are usually too large to fit intovideo memory for real-time rendering, we propose that a DOM and DSM image pyramidstructurally identical to the mesh pyramid of each sub-extent be used. Upon loading amesh tile from a mesh pyramid, the corresponding DSM tile and DOM tile are retrievedfrom the image pyramid. The DSM is utilized to change the height values of the meshtile and the DOM tile serves as an extra texture layer to override the original pixelswhere modifications have been made.

Since an independent DOM and DSM image pyramid is used for dynamic updating,no change needs to be made to the original OAP3D. Therefore, we can maintainmultiple versions of image pyramids to keep track of changes as well as for inter-comparison, which may potentially benefit urban planners who are seeking optimizedsolutions.

4. Application examples

4.1. Road surface remediation

In this OAP3D, some roads and streets are partially covered by dense tree canopies(Figure 9). Tree removal, surface flattening, and texture painting were considerednecessary to create smoother and more realistic road surfaces, which are important forcertain purposes, for example, realistic simulation of vehicles or pedestrians navigatingthe streets.

Figure 9. Road surface covered by tree canopies in the original OAPD3D.

INTERNATIONAL JOURNAL OF GEOGRAPHICAL INFORMATION SCIENCE 11

Dow

nloa

ded

by [

Ari

zona

Sta

te U

nive

rsity

Lib

rari

es]

at 0

9:08

16

June

201

6

ROI polygons were delineated in a GIS to include the road pixels that werecovered by tree canopies. Surface elevation samples were also extracted from a GISfor the purpose of reconstructing the ROI surfaces. As the ground elevations beneathan above-ground object are unknown, we assume that the ground elevation of apoint at the object can be estimated by a group of known ground points from theexposed ground in the immediate vicinity. Elevation samples were evenly distributedaround the ROI polygons and restricted to flat road surfaces that were not coveredby trees. Inverse distance weighting interpolation was employed to produce a DSMusing these elevation samples. The DSM was processed using a moving average filterto make the road surfaces smoother (Figure 10). The DOM was then imported intoPhotoshop for texture editing. In Photoshop, the road pixels inside the ROI polygonswere erased and replaced with texture patterns that were visually consistent thesurrounding areas.

Figure 11 shows that the visual quality of the road surfaces have been considerablyimproved due to the flattening of the areas that were previously bulging outwardsbecause of the presence of trees, and the realistic texture patterns that were artisticallypainted over the tree pixels.

4.2. Replacement of above-ground objects for urban planning

When viewed from a distance, an OAP3D may appear highly realistic without noticeableartifacts. When viewed at a closer range, however, buildings may begin to show bulges,depressions, or other distortions. Moreover, buildings in OAP3D typically are representedas a meshed shell without any internal structure, which can be important for certainurban applications, for example, building information modeling, architectural design,indoor mapping, and navigation. In an OAP3D, tree branches and leaves are barely

Figure 10. Smooth road surface reconstructed using non-tree elevation samples in a GIS.

12 J. LIANG ET AL.

Dow

nloa

ded

by [

Ari

zona

Sta

te U

nive

rsity

Lib

rari

es]

at 0

9:08

16

June

201

6

separable and therefore a tree is usually rendered in the shape of a fully closed ellipsoidwith a bumpy surface.

Due to these technical imperfections and limitations facing OAP3D, users maysometimes intend to locally replace these areas with externally sourced 3D models.There is also a need from urban land redevelopment to reclaim old city blocks. Theworkflow for integrating 3D models into OAP3D (Figure 12) are similar to that usedin the road surface remediation, the only difference is that DOM reconstruction usingspatial interpolation may be spared since the ROI is to be covered by externallycreated 3D models.

Figure 11. Improved road surface with modified DSM and DOM from a GIS.

Figure 12. Workflow for replacing individual building: (a) original building, (b) ROI delineation andDSM editing in GIS, (c) building removed, and (d) new building placed at the same location.

INTERNATIONAL JOURNAL OF GEOGRAPHICAL INFORMATION SCIENCE 13

Dow

nloa

ded

by [

Ari

zona

Sta

te U

nive

rsity

Lib

rari

es]

at 0

9:08

16

June

201

6

4.3. Surface modification for land repurposing

In Figure 13(a), a land lot on the edge of the urban area was defined using a polygon ofyellow borders. Elevation samples were selected in a GIS to reform the topography ofthe polygonal area. Inside the polygon, elevation values at the sample points weremanually controlled to achieve a desired topography. Outside the polygon, elevationvalues at the sample points were derived directly from the DSM in order to obtain agradual sloping effect. Spatial interpolation was performed using a combination of theelevation samples from inside and outside the ROI polygon. In the first example, auniform height was assigned to the evenly distributed samples within the polygon toflatten the ground surface, which was decorated with grass texture patterns to showthat the land lot has been repurposed for greenery (Figure 13(a)). In the secondexample, several elevation samples were assigned significantly higher values to serveas mountain peaks for the artificially generated mountainous area (Figure 13(b)). In thethird example, the land lot was repurposed as a water body, which can serve as a lake ora reservoir. To approximate the topography of a water body, significantly lower eleva-tion values were applied inside the polygon (Figure 13(c)).

5. Discussion and conclusion

We have presented a method for embedding user-generated content into OAP3D inthree steps, that is, ROI delineation, DSM modification, and external content embed-ding. The technical framework described in Section 3 and the application examplespresented in Section 4 have shown that this method can: (1) support easy modifica-tion of OAP3D’s geometric content through 2D GIS operations; (2) support easymodification of OAP3D’s textural content through 2D image editing; and (3) dynami-cally apply the modifications during real-time rendering without changing the origi-nal OAP3D.

Figure 13. Land lot repurposing: (a) original land lot, (b) surface reconstruction in GIS, (c) repurpos-ing for greenery, (d) repurposing as mountains, and (e) repurposing as a water body.

14 J. LIANG ET AL.

Dow

nloa

ded

by [

Ari

zona

Sta

te U

nive

rsity

Lib

rari

es]

at 0

9:08

16

June

201

6

With the rapid development of unmanned airborne vehicle technology, OAP3D datawill increasingly be accumulated. Microsoft Bing Maps (http://www.bing.com/dev/en-us/maps-preview-app) and Google Earth (https://www.google.com/earth/) have acquiredand published hundreds of OAP3D-based virtual cities for augmenting the present 3Durban application, which traditionally relied mainly on high resolution imagery with few3D building models. This paradigm shift in 3D urban application was likely driven by theobvious advantages of OAP3D in terms of data accuracy, real-time rendering perfor-mance, and data acquisition efficiency. Many smaller businesses have also built up thecapacity to operate unmanned airborne vehicles and acquired OAP3Ds for commercialservices. The method presented in this paper can be very helpful in developing OAP3D-based 3D urban GIS applications, since it allows user’s ideas and needs to be embodiedin the static city models. With this method, the time and labor costs that wouldpotentially be required to bring user-generated content into an OAP3D can be signifi-cantly reduced, because the 2D-3D mapping approach is as straightforward as drawingon a canvas.

There are three implications of the presented method for application of OAP3D inGIS. (1) Data integration is one of the most challenging issues in fully exploiting thevalue of OAP3D in GIS. Our method offers a cost-effective solution to address this issueand has the potential to promote the use of OAP3D in a broader and deeper manner. (2)It is a systematically designed approach, which can potentially inspire researchers fromthe academia and product developers from the industry to develop a fully integratedsolution. (3) User-generated content is a very important data source for augmentingurban GIS applications. Although remotely sensed data can faithfully and accuratelycapture the present state of a city, the past and future can be created only through theknowledge and imagination of urban planners, artists, and decision-makers, who are inneed of an easy tool to help embed their ideas into OAP3D.

Acknowledgments

This research was supported and funded by the Foundation for Young Scientists of the StateKey Laboratory of Remote Sensing Science (15RC-08), the Key Knowledge Innovative Projectof the Chinese Academy of Sciences (KZCX2 EW 318), the National Key Technology R&DProgram of China (2014ZX10003002), and the National Natural Science Foundation of China(41371387).

Disclosure statement

No potential conflict of interest was reported by the authors.

Funding

This research was supported and funded by the Foundation for Young Scientists of the State KeyLaboratory of Remote Sensing Science: [grant number 15RC-08]; the Key Knowledge InnovativeProject of the Chinese Academy of Sciences: [grant number KZCX2 EW 318]; the National KeyTechnology R&D Program of China: [grant number 2014ZX10003002]; and the National NaturalScience Foundation of China: [grant number 41371387].

INTERNATIONAL JOURNAL OF GEOGRAPHICAL INFORMATION SCIENCE 15

Dow

nloa

ded

by [

Ari

zona

Sta

te U

nive

rsity

Lib

rari

es]

at 0

9:08

16

June

201

6

ORCID

Jianming Liang http://orcid.org/0000-0002-4043-6816

References

Döllner, J., Baumann, K., and Buchholz, H., 2006. Virtual 3D city models as foundation of complexurban information spaces. In: Proceedings of CORP 2006 & Geomultimedia06, February, Vienna,107–112. Available from: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.83.4098&rep=rep1&type=pdf [Accessed April 2016].

Kolbe, T.H., Gröger, G., and Plümer, L., 2005. CityGML: interoperable access to 3D city models. In: P.van Oosterom, S. Zlatanov, and E.M. Fendel, eds. Geo-information for disaster management.Berlin: Springer-Verlag, 883–899.

Köninger, A. and Bartel, S., 1998. 3D-GIS for urban purposes. Geoinformatica, 2 (1), 79–103.doi:10.1023/A:1009797106866

Kreveld, M.V. and Silveira, R.I., 2011. Embedding rivers in triangulated irregular networks withlinear programming. International Journal of Geographical Information Science, 25 (4), 615–631.doi:10.1080/13658816.2010.488240

Liang, J., et al., 2015. An open-source 3D solar radiation model integrated with a 3D GeographicInformation System. Environmental Modelling & Software, 64, 94–101. doi:10.1016/j.envsoft.2014.11.019

Lin, H., Chen, M., and Lu, G., 2013. Virtual geographic environment: a workspace for computer-aided geographic experiments. Annals of the Association of American Geographers, 103 (3), 465–482. doi:10.1080/00045608.2012.689234

Moser, J., Albrecht, F., and Kosar, B., 2010. Beyond visualisation–3D GIS analyses for virtual citymodels. International Archives of the Photogrammetry, Remote Sensing and Spatial InformationSciences, 38 (4), W15.

Remondino, F., et al., 2011. UAV photogrammetry for mapping and 3d modeling–current statusand future perspectives. International archives of the photogrammetry. Remote Sensing andSpatial Information Sciences, 38 (1), C22.

Rottensteiner, F. and Briese, C., 2002. A new method for building extraction in urban areas fromhigh-resolution LIDAR data. International Archives of Photogrammetry Remote Sensing andSpatial Information Sciences, 34 (3/A), 295–301.

Strzalka, A., et al., 2011. 3D city modeling for urban scale heating energy demand forecasting.HVAC&R Research, 17 (4), 526–539. doi:10.1080/10789669.2011.582920

Tao, W., 2013. Interdisciplinary urban GIS for smart cities: advancements and opportunities. Geo-spatial Information Science, 16 (1), 25–34. doi:10.1080/10095020.2013.774108

Vivoni, E.R., et al., 2005. Embedding landscape processes into triangulated terrain models.International Journal of Geographical Information Science, 19 (4), 429–457. doi:10.1080/13658810512331325111

Wu, H., He, Z., and Gong, J., 2010. A virtual globe-based 3D visualization and interactive frameworkfor public participation in urban planning processes. Computers, Environment and UrbanSystems, 34 (4), 291–298. doi:10.1016/j.compenvurbsys.2009.12.001

Yasumoto, S., et al., 2012. Virtual city models for assessing environmental equity of access tosunlight: a case study of Kyoto, Japan. International Journal of Geographical Information Science,26 (1), 1–13. doi:10.1080/13658816.2011.570268

Yasumoto, S., et al., 2011. The use of a virtual city model for assessing equity in access to views.Computers, Environment and Urban Systems, 35 (6), 464–473. doi:10.1016/j.compenvurbsys.2011.07.002

16 J. LIANG ET AL.

Dow

nloa

ded

by [

Ari

zona

Sta

te U

nive

rsity

Lib

rari

es]

at 0

9:08

16

June

201

6