probado3d - indexing and searching 3d cad databases

International Conference on Design & Decision Support Systems in Architecture and UrbanPlanning (2010)

PROBADO3D - Indexing and Searching 3D CAD Databases:Supporting Planning through Content-Based Indexing and

3D Shape Retrieval

Ina Blümel1, René Berndt2, Sebastian Ochmann3, Richard Vock3, Raoul Wessel3

1German National Library of Science and Technology, Hannover, Germanye-mail: [email protected]

2Institute of Computer Graphics and Knowledge Visualization, Graz University of Technology, Austriae-mail: [email protected]

3Institute of Computer Science II Computer Graphics, University of Bonn, Germanye-mail: {ochmann, vock, wesselr}@cs.uni-bonn.de

AbstractWhen modeling in 3D CAD, architects often search for already existing 3D content to complete their own design,e.g. environmental models, furniture models or detailed window profiles. 3D models are normally indexed andaccessed based on textual metadata. As this metadata is expensive to obtain, PROBADO3D aims to developworkflows and tools for semi-automatic indexing of 3D models. Another goal is the development of intuitive visualsearch and presentation interfaces that face the needs of architects looking for 3D content. PROBADO3D is apart of the PROBADO framework designed to support multimedia objects of different domains.

Keywords: Content-Based Indexing, 3D Shape Retrieval, Visual Search, Digital Library

1. ARCHITECTURAL PLANNING

Architects are “...working from abstract problem formu-lations to concrete solutions and splitting problems intosubproblems iterative and recursive processes...” [CR92].Within complex planning tasks a part of the decision-makingprocesses can be adopted by design methods to obtain ahigher effectiveness. [Sch93] is making decision possibili-ties explicit by characterizing different relevant approachesfor designing. He classifies several methods that can be in-serted into a general modelling environment and support dif-ferent aspects of planning, sketching and building. Thesemethods offer a solution to draft problems by the use ofsearch mechanisms. Within this context we will describe theusage of indexed 3D CAD model collections. ArchitecturalCAD models are becoming more heterogeneous and com-plex. Trades taking part in planning provide highly detailedindividual layers of the whole building model. Only few ofthem are developed completely from scratch. For efficiencyreasons architects and specialized planners use case-basedreasoning and search for already existing 3D models that A)

serve for inspiration or B) fit best into the given conditionswithin the actual building model. Case-based reasoning isa method long-known in architecture. It consists of findingand adapting a similar problem definition and the appropri-ate architectural solution for solving a new problem. The ad-justment process or the adaptation of existing architecture onnew problems is a complex procedure. The simplest stage isthe direct take over. In the next higher stage parts of archi-tecture solutions are transferred, others are adapted geomet-rically or regarding materials. In the most complicated formof the adaptation topological changes are made. Challengesare

1. the completeness of the architectural case database,2. methods for indexing the cases so that they can be found,3. methods for formulating the search so that all cases are

found which correspond to the given requirements withregard both to geometrical and topological aspects.

The first point is scribed in Section 2.1, however in this paperwe will especially go into 2 and 3, expounded in Section 3,and show how automatic indexing of 3D content can assist

I. Blümel, R. Berndt, S. Ochmann, R. Vock, R. Wessel / Supporting Planning through Content-Based Indexing and 3D Shape Retrieval

the architect in using 3D model databases for facilitating theplanning process.

2. 3D MODELS AND INDEXING

2.1. Content in PROBADO3D

The German National Library of Science and Technology,which references relevant scientific material for all areas ofengineering, is setting up a repository for 3D models, whichcurrently contains index files of approximately 7000 modelsof the architectural domain. Actually it works satisfying asbasis for testing query engine functionality, however, it can-not be considered to be a complete database for architecturaldesign cases yet.

One result of a survey among architects in the project con-text, see [BS09], is the gained information about usage ofarchitectural 3D models. First of all we classified five modeltypes, which are buildings, environmental models, compo-nents like A) construction units and B) furniture and 3Dmodel details. While building models are rather used as asource for inspiration due to the one-of-a-kind building pro-cess, the other model types are only slightly transformed ordirectly integrated into new drafts, and the demand for thismodel types is thus higher in architectural practices. Never-theless building models are first of all acting as precedentsin architectural education and design, and the potential of3D models over images or plans provided for instance in ar-chitectural magazines is high if access to the respectivelysearched content within the model can be assured. As a firststep, we concentrate on buildings and components, becauseof the enduring scientific interest in building configurationand the cumulative need for components to be directly takenover into the own building model.

3D model contributors in PROBADO3D are either stu-dents of architecture and component manufacturers or themodels are part of architectural CAD application libraries orpublic databases for architectural CAD models. The controlover the original files in most cases still resides with exter-nal servers, including access to the files (e.g. pay-per-view orIP-based access for certain groups). Only the index- and pre-view data are stored within the PROBADO3D system. Usersrequesting a model will be redirected to the server hostingthe original file. So PROBADO3D does not have to strugglewith legal problems when offering 3D models and can con-centrate on developing search engines for indexing modelsand making their content searchable.

The 3D models to be integrated in the PROBADO3D in-dex are of various file formats and bring along very littlemetadata, see [BWK08]. The developed pipeline for auto-matic processing of architectural 3D models and deductionof technical metadata is described in [BBW10]. We will nowhave a closer look at content-based indexing, which is a pre-requisite for different user queries on 3D model collections.

Figure 1: 3D model and underlying geometry.

2.2. Content-based Indexing

Content-based shape retrieval methods rely on an abstractmathematical characterization of the underlying 3D modelgeometry, which is usually given as a set of (unstructured)polygons, see e.g. Figure 1. In general, the mathemati-cal characterization can be derived automatically. By that,content-based shape retrieval does not rely on any user-generated textual annotations, allowing for fast automatic in-dexing of even large databases. Most approaches on content-based shape retrieval (for a detailed introduction see [TV08])concentrate on query-by-example, i.e. when presented a 3Dobject, they search an indexed database for geometricallysimilar shapes. This technique is suitable for scenarios inwhich the user can provide actual 3D content as a queryby either uploading an existing model to a search engine orby sketching a new model using a graphical interface thatis connected to the search engine. However, as another re-sult of the survey mentioned in Section 2.1, it became clearthat apart from visual-interactive-driven queries, users arehighly interested in text-based search relying on metadata.As this metadata is usually either not available or ambigu-ous, PROBADO3D aims at automatically extracting it fromthe underlying object geometry. For environmental, furni-ture, and buildings this requires automatic model classifi-cation according to certain shape taxonomy. For buildings,automatic extraction of information about the number ofstoreys, room areas, gross floor area, window areas per roomand per floor, number of rooms per floor are additionally ofgreat interest to the user. The currently supported content-based indexing modules are described in Section 3.

2.3. Related Work

Within this field there are related scientific initiatives for3D search engines. There are the Princeton Shape Retrievalgroup [SMKF04] with content-based search engines andAim@Shape [Fal04] with content-based and metadata basedsearch engines.


Figure 3: PROBADO3D accessed from a CAD program.

3. QUERY ENGINES AND INTERFACES

3.1. Interface Overview

Design and usability of the search interface are a crucial as-pect for the acceptance of a service like PROBOADO3D.Especially query-by-example requires an object [BHF09],which in the case of PROBADO3D is a 3D model. Up tonow there is no build-in browser-support for 3D content, soproviding these functionality can only be archived using anadditional plugin (The upcoming HTML 5 will include We-bGL (Web Graphics Library), which will provide support for3D without the need for a browser plugin).

The interface of PROBADO3D is built on Microsoft Sil-verlight, which is available for Windows, Apple OS X andLinux/Unix (using the open source Moonlight). Silverlightas other Rich Internet Applications (RIA) like JavaFX orAdobe Flex offer a rich user interface similar to desktop ap-plications including support for communication over the in-ternet (e.g. consuming web services). One main advantageof Silverlight is the use of the same declarative language(XAML) for describing the user interface as Windows Pre-sentation Foundation (WPF). This provides an easy and veryflexible way to guarantee the same rich user experience bothin web browsers and classic applications.

The current prototype supports for the query of eitherbuildings or components different interfaces for interactivesketching of 3D models and uploading query models (query-by-example, see Section 3.2), searching in the textual meta-data and browsing using different filters (category, contrib-utor, etc., see Figure 2 / Section 3.3), and furthermore forconstructing RCG query graphs (see Section 3.4). For de-tails regard [BBWS09].

In addition to the web based user interfaces, 3rd partymodeling tools like GoogleTMSketchUp can also be usedfor accessing the PROBADO3D search services (see Fig-ure 3). For this purpose a WPF desktop application was de-veloped, which sends a 3D model file to the PROBADO3Dweb-service and presents the result using the same look andfeel as the Silverlight interface. A SketchUp-Plugin is re-

Figure 4: Global shape descriptor as a vector. The coeffi-cients encode the 3D model geometry.

sponsible for exporting the 3D model to a file and to startthis WPF application. Using this mechanism, basically ev-ery CAD program, which provides a plugin API can be eas-ily extended to access Probado3D.

Different result representations, e.g. the 2D thumbnailcloud (see Figure 2), allow the user to interactively explorethe result sets, e.g. use one result as a new query object orview a 3D preview.

3.2. Query-by-example and Browsing forEnvironmental Elements

In this query-by-example scenario the user is looking for en-vironmental objects or furniture that is similar to a givenquery object. This query object is either uploaded or gen-erated using one of the PROBADO3D sketch interfaces (seeSection 3.1). When presented the 3D content, the query en-gine first generates a global shape descriptor that representsthe object. It consists of a vector of fixed size, the coeffi-cients encode the object geometry (see Figure 4). In a sec-ond step, this vector is compared to those associated to themodels contained in the PROBADO database. Finally, theobjects providing the largest similarity to the query objectare presented to the user.

Note that global shape descriptors are currently also usedfor browsing our 3D database. Once the user has selected amodel from the query-by-example result list, this object andthe according shape descriptor can again be used as a queryobject. Due to their global character, the shape descriptorsused for this task provide a rather coarse representation ofthe underlying object. However, they are very fast to com-pute which is crucial in a query-by-example scenario to keepresponse times short.

3.3. Textual Search for Objects and Browsing theDatabase

In order to make 3D collections of environmental objects andfurniture searchable using keywords, the contained models


Figure 2: Exploring results via browsing in PROBADO3D: categories or similar shapes.

Figure 5: Segmentation of the 3D object for applying localshape descriptors. The colors code which part of a primi-tive was detected: red - plane, green - cylinder, grey - torus,yellow - sphere, purple - cone.

must be automatically classified according to an architec-tural shape taxonomy. While usually low-level mathemat-ical descriptions are used to characterize the geometry of3D models, a shape taxonomy designed by experts includeshigh-level descriptions of object categories. Classificationschemes for example might contain classes including ob-jects that have a similar function (high-level) although theirshape (low-level) might strongly vary (e.g. dining chairs).This is one of the manifestations of the semantic gap, i.e.the gap between the abstract, high level user intention andlow level data representation and processing [NPWK05]. InPROBADO3D we aim at bridging this gap. Starting witha low-level mathematical description of the 3D content, weuse state-of-the art supervised learning schemes to incorpo-rate architectural expert knowledge into the classification de-cision.

We first generate a comprehensive characterization of theunderlying 3D geometry using local shape descriptors. Theobject is first segmented into somewhat meaningful ele-ments corresponding to parts of primitive geometric shapeslike planes, cylinders, cones, sphere, and tori (see Fig-ure fig:Segmentation). For each detected primitive, we thencompute a shape descriptor characterizing its geometry. Theresulting collection of local shape descriptors serves as inputfor object classification [WK10].

To predict object categories, we use a supervised learn-ing approach. Based on a number of manually classified ob-jects and their extracted local shape descriptors, we train analgorithm to learn which constellations of local shape fea-tures typically indicate a certain category affiliation. Whenpresented new unknown objects, the algorithm predicts theirprobable class membership according to the automaticallyextracted local features and stores this information in themetadata database. Note that this approach only requiresmanual object classification for the training step. Once thealgorithm has been trained, no further interaction for clas-sifying new objects is required. For a more detailed intro-duction to this approach we refer to our work presentedin [WBK09, WBK08a].

For the above described supervised learning framework,an object classification scheme tailored to architectural re-quirements is crucial. As architecture can be considered inseveral ways (historical, constructional, etc.) we initially hadto find suitable classification schemes for our type of contentand for the diverse views on the content within certain stagesduring the planning process, whenever architects search for


Figure 6: Retrieval results on form- (green) and function-oriented (blue) benchmark for 3D components.

3D models. First, there are two kinds of models to be clas-sified: components and buildings, see Section 2.1. Second,architects use both form- and functional approach duringtheir planning process, which is one result of another sur-vey among architects started in the project context. In de-sign planning, 57% of the participating architects are think-ing rather form-oriented, 14% function-oriented and 29%consider both concepts to be important. In execution plan-ning, 75% regard the function-oriented approach as muchmore important, 8% the form-oriented and 12% both con-cepts. Taking this into consideration, we provide classifi-cations that are both form- and function-oriented to allowintuitive searching within any stage of drafting. More de-tailed information to the classification schemes for compo-nent models, is presented in [WBK09].

Figure 6 diplays the results for the classification schemesregarding component models. In information retrieval con-texts, precision and recall are defined in terms of a set ofretrieved objects and a set of relevant objects. The preci-sion score depicts the relevant result retrieved by a search(but says nothing about whether all relevant objects were re-trieved) whereas the recall score shows the relevant objectsretrieved by the search (but says nothing about how manyirrelevant objects were also retrieved).

The precision recall performance according to form isslightly better than that of the function categories. The over-all performance is quite low due to the fact that not all cat-egories currently contain objects and have to be balanced ina better way (they contain e.g. very many chairs and tablescompared to other kind of furniture). Note that the perfor-mance is better on other benchmarks, for instance the prince-ton shape benchmark [SMKF04], regard [WBK09].

3.4. Retrieval and Metadata Generation with RoomConnectivity Graphs

Buildings and their designated use are particularly charac-terized by the topology of contained rooms and storeys thanby their overall shape. While the above described global andlocal shape descriptors represent efficient means to charac-terize environmental objects and furniture they can hardlydescribe this topology. Our tests have shown that buildingmodels are inferior to categorize only by shape descriptors.To alleviate this problem, we developed the concept of RoomConnectivity Graphs (RCGs), see [WBK08b]. RCGs are adata structure that consists of a graph in which rooms arerepresented as nodes and connections between rooms (e.g.doors or staircases) are represented as edges, see for exam-ple Figure 7.

When constructing the RCG of a building model, roomnodes and the connecting edges are additionally enriched bycertain attributes. For example, we extract room and windowareas or gross floor area.

For upgrading the construction of the RCGs and auto-matic enrichment by attributes, we established an admin in-terface whereby RCGs for a training set of building mod-els are generated manually (see Figure 8). Once this groundtruth data is existent, RCGs can be genereated more pre-cisely, eventually allowing training according to human’sperception of building topology.

Automatically extracted RCGs serve as a starting point forquery-by-example related searches. The PROBADO3D ser-vice provides a graphical interface, see [BBWS09], enablingthe user to easily draw room topologies he intends to searchfor.

Extracted RCGs provide a rich amount of metadata thatis important to architects and can be used for textual search.For example, we store the number of building stories, roomareas, gross floor area, window areas per room and per floor,number of rooms per floor etc. in the PROBADO3D meta-data database.

4. CONCLUSION AND FUTURE WORK

Most parts of the depicted query engines have already beenimplemented. It is planned that the search with RCGs canadditionally be refined by constraining the results to thosein which attributes of rooms or connections fullfill certainconstraints (e.g. only include results in which a room’s arealies within a certain range). Defining graph similarity withrespect to the depicted attributes is still a challenge. Discretefeatures (e.g. room type) can be inserted more easy into thequery engine’s matching algorithm than continuous features(e.g. square meters). The importance of different attributesstill remains an open question. It is not clear if square metersor other continuous attributes are more important to the cur-rent user. In addition similar features can make two buildings


Figure 7: Automatically generated Room Connectivity Graph: rooms are represented by nodes, edges depict connections likedoors or vertical connections like staircases.

Figure 8: Two examples from our interface for manual RCG assignation to achieve groundtruth data that is crucial for enhanc-ing automatic RCG generation.

similar though the topology incorporated by the pure RCGis quite different. In other words: When are graphs consid-ered to be similar and when not, regarding A) pure graphgeometry and B) the attributes? To alleviate we plan slideras part of the RCG query interface for adjusting the rankingof graph geometry and the different discrete and continuousfeatures.

Future work will further concentrate on improving the au-tomatic classification and processing of the 3D models. Sim-ilar to the categorization of components described in Sec-tion 3.2, we will additionally examine how building modelscan be automatically classified according to their RCG, re-garding the concepts of style covering and structural core,see [Blü05], plus the concepts of form and function. We

have precision recall performance results for the categoriesof component models so far, and it has to be proved if thefindings also apply to building models, and if, how the clas-sifications can be improved to enhance the values for func-tional categories. Users will be able to browse building mod-els by ground plan (core, form and function), form char-acteristic (cover and core, form), form typology/ buildingtype (cover, form) and building function (cover, function).For developing we again use approved classifications as astarting point, which are the Getty Art&Architecture The-saurus [Pet94], Dewey Decimal Classification [OCL03] andNeufert Bauentwurfslehre [Neu05].

One additional content-based indexer using Semantic en-richment methods based on procedural shape representations


[USF08] is currently implemented and integrated into thePROBADO3D system. By fitting a procedural description tothe target model the semantic information carried with thegenerative description can then also by applied to the targetmodel (e.g. number of columns, stairs, etc).

4.1. Acknowledgements

PROBADO is a joint research project supported by the Ger-man Research Foundation DFG under the LIS program.PROBADO started in February 2006 with a tentative dura-tion of five years. Partners are the University of Bonn, Tech-nische Universitaet Darmstadt, Graz University of Technol-ogy, the German National Library of Science and Technol-ogy in Hannover, and the Bavarian State Library in Mu-nich. The work presented in this paper was partially sup-ported under grants INST 9055/1-1, 1647/14-1, and 3299/1-1. For further information, please visit the project website athttp://www.probado.de/.

References[BBW10] BERNDT R., BLÜMEL I., WESSEL R.: Probado3d -

towards an automatic multimedia indexing workflow for archi-tectural 3d models. In Proceedings of 14th European Conferenceon Electronic Publishing (ELPUB) (2010), pp. 79–88. 2

[BBWS09] BERNDT R., BLÜMEL I., WESSEL R., SCHRECK T.:Demonstration of user interfaces for querying in 3d architecturalcontent in probado3d. In ECDL (Sept. 2009), vol. 5714 of Lec-ture Notes in Computer Science, Springer. 3, 5

[BHF09] BERNDT R., HAVEMANN S., FELLNER D.: 3D mod-eling in a web browser to formulate content-based 3D queries.In Web3D ’09: Proceedings of the 14th International Confer-ence on 3D Web Technology (New York, NY, USA, 2009), ACM,pp. 111–118. 3

[Blü05] BLÜMEL I.: Modeling techniques in architecture and in-formation technologies: Building cover and core. In Proccedingsof 3rd International Conference on Innovation in Architecture,Engineering an Construction (2005). 6

[BS09] BLÜMEL I., SENS I.: Das PROBADO-Projekt: Integra-tion von nichttextuellen Dokumenten am Beispiel von 3D Objek-ten in das Dienstleistungsangebot von Bibliotheken. Zeitschriftfür Bibliothekswesen und Bibliographie 2 (2009), 79–87. 2

[BWK08] BLÜMEL I., WESSEL R., KROTTMAIER H.: ThePROBADO Framework: A Repository for Architectural 3D-Models. In Proceedings of International Conference on On-line Repositories in Architecture (2008), Fraunhofer irb Verlag,pp. 250–259. 2

[CR92] CROSS N., ROOZENBURGH N.: Modelling the designprocess in engineering and in architecture. Journal of Engineer-ing Design 3, 4 (1992), 325–337. 1

[Fal04] FALCIDIENO B.: Aim@shape project presentation. InProceedings of Shape Modeling International (SMI) (Washing-ton, DC, USA, 2004), IEEE Computer Society, p. 329. 2

[Neu05] NEUFERT E.: Bauentwurfslehre, 38th ed. Vieweg,September 2005. 6

[NPWK05] NOVOTNI M., PARK G. J., WESSEL R., KLEIN R.:Evaluation of kernel based methods for relevance feedback in3d shape retrieval. In The Fourth International Workshop onContent-Based Multimedia Indexing (CBMI’05) (June 2005). 4

[OCL03] OCLC: Dewey decimal classification, 2003. OCLCOnline Computer Library Center, Inc. 6

[Pet94] PETERSEN T.: Art & Architecture Thesaurus. ACMPress, 1994. 6

[Sch93] SCHMITT G.: Architectura et Machina: Computer AidedArchitectural Design und Virtuelle Architektur. Vieweg+Teubner,1993. 1

[SMKF04] SHILANE P., MIN P., KAZHDAN M., FUNKHOUSERT.: The princeton shape benchmark. In Shape Modeling Interna-tional (June 2004). 2, 5

[TV08] TANGELDER J., VELTKMAMP R.: A survey of contentbased 3D shape retrieval methods. Multimedia Tools and Appli-cations 39, 3 (2008), 441–471. 2

[USF08] ULLRICH T., SETTGAST V., FELLNER D. W.: Seman-tic fitting and reconstruction. J. Comput. Cult. Herit. 1, 2 (2008),1–20. 7

[WBK08a] WESSEL R., BARANOWSKI R., KLEIN R.: Learningdistinctive local object characteristics for 3d shape retrieval. InVMV (2008), pp. 167–178. 4

[WBK08b] WESSEL R., BLÜMEL I., KLEIN R.: The room con-nectivity graph: Shape retrieval in the architectural domain. InThe 16-th International Conference in Central Europe on Com-puter Graphics, Visualization and Computer Vision’2008 (Feb.2008), Skala V., (Ed.), UNION Agency-Science Press. 5

[WBK09] WESSEL R., BLÜMEL I., KLEIN R.: A 3d shapebenchmark for retrieval and automatic classification of architec-tural data. In 3DOR (Mar. 2009), pp. 53–56. 4, 5

[WK10] WESSEL R., KLEIN R.: Learning the compositionalstructure of man-made objects for 3d shape retrieval. In EURO-GRAPHICS 2010 Workshop on 3D Object Retrieval (May 2010),pp. 39–46. 4

http://www.probado.de/

probado3d - indexing and searching 3d cad databases

Documents