buildings recognition and camera localization using image texture description
DESCRIPTION
3D GIS model/2D image registration called much attention in the recent years because of its vast range of potential applications in real and virtual navigation. However, automatic registration remains until now a challenge. This paper presents a methodology for enhancing and complementing a GIS database of buildings with a descriptor of their texture by using information extracted from a reference images. This descriptor is used to locate any other image by searching similar texture in the image. Then the absolute position and orientation of the camera of the new image can be computed if the camera parameters (like focal length) are known. The paper proposes a technique that can be used for achieving the identification of the facade in the photograph, calibrated camera geolocation and discusses the quality of the results.TRANSCRIPT
![Page 1: Buildings Recognition and Camera Localization Using Image Texture Description](https://reader034.vdocument.in/reader034/viewer/2022052400/559867bc1a28ab9a738b4587/html5/thumbnails/1.jpg)
1
Buildings Recognition and Camera Localization Using Image Texture
Description SULEIMAN Wassim1, JOLIVEAU Thierry1,
FAVIER Eric2
1ISTHME-ISIG CNRS/UMR EVS, Université Jean Monnet - Saint-Etienne. 2DIPI EA 3719 École Nationale d'Ingénieurs de Saint-Etienne
[email protected] [email protected]
25th International Cartographic Conference (Sageo) – 8 july 2011 – Palais de congrès Paris
![Page 2: Buildings Recognition and Camera Localization Using Image Texture Description](https://reader034.vdocument.in/reader034/viewer/2022052400/559867bc1a28ab9a738b4587/html5/thumbnails/2.jpg)
Objective
2
Find a building in an image
SIG 3D3D GIS
Locate the camera that took the image by using the location of the building
![Page 3: Buildings Recognition and Camera Localization Using Image Texture Description](https://reader034.vdocument.in/reader034/viewer/2022052400/559867bc1a28ab9a738b4587/html5/thumbnails/3.jpg)
Methodology
Enhancing the GIS database with information which can describe the building
unique information quantifiable information
■ The texture signature
3
Texture signature
![Page 4: Buildings Recognition and Camera Localization Using Image Texture Description](https://reader034.vdocument.in/reader034/viewer/2022052400/559867bc1a28ab9a738b4587/html5/thumbnails/4.jpg)
How to isolate the building facade in the image?
Manual method (long)
4
Sourimant 2009
Automatic method : (3D SIG model/2D image) registration (simple building)
![Page 5: Buildings Recognition and Camera Localization Using Image Texture Description](https://reader034.vdocument.in/reader034/viewer/2022052400/559867bc1a28ab9a738b4587/html5/thumbnails/5.jpg)
Work plan
1. GIS database enhancement with building texture information
2. Facade recognition3. Camera geolocation 4. Possible applications5. Limits
5
![Page 6: Buildings Recognition and Camera Localization Using Image Texture Description](https://reader034.vdocument.in/reader034/viewer/2022052400/559867bc1a28ab9a738b4587/html5/thumbnails/6.jpg)
Work plan
q Enhancing GIS databases with building texture information
q Facade recognitionq Camera Geolocation q Possible Applicationsq Limits
6
![Page 7: Buildings Recognition and Camera Localization Using Image Texture Description](https://reader034.vdocument.in/reader034/viewer/2022052400/559867bc1a28ab9a738b4587/html5/thumbnails/7.jpg)
Enhancing GIS databases with building texture information
7
Texture analyses(SIFT)
Finding the interest points with their local descriptor
![Page 8: Buildings Recognition and Camera Localization Using Image Texture Description](https://reader034.vdocument.in/reader034/viewer/2022052400/559867bc1a28ab9a738b4587/html5/thumbnails/8.jpg)
Enhancing GIS databases with building texture information
8
Finding the (x,y,z) of the interest points
Homography constraints
3D GIS model
![Page 9: Buildings Recognition and Camera Localization Using Image Texture Description](https://reader034.vdocument.in/reader034/viewer/2022052400/559867bc1a28ab9a738b4587/html5/thumbnails/9.jpg)
Enhancing GIS databases with building texture information
9
The texture descriptor :
list of interest points with their local descriptor and their 3D position
![Page 10: Buildings Recognition and Camera Localization Using Image Texture Description](https://reader034.vdocument.in/reader034/viewer/2022052400/559867bc1a28ab9a738b4587/html5/thumbnails/10.jpg)
Work plan
q Enhancing GIS databases with building texture information
q Facade recognitionq Camera Geolocation q Possible Applicationsq Limits
10
![Page 11: Buildings Recognition and Camera Localization Using Image Texture Description](https://reader034.vdocument.in/reader034/viewer/2022052400/559867bc1a28ab9a738b4587/html5/thumbnails/11.jpg)
Facade recognition
11
False matching because of the locality of the descriptor
![Page 12: Buildings Recognition and Camera Localization Using Image Texture Description](https://reader034.vdocument.in/reader034/viewer/2022052400/559867bc1a28ab9a738b4587/html5/thumbnails/12.jpg)
Facade recognition
12
Eliminate the false matching using the homography constraints
Select the best matching score between the current image and the stored descriptor in the databases
![Page 13: Buildings Recognition and Camera Localization Using Image Texture Description](https://reader034.vdocument.in/reader034/viewer/2022052400/559867bc1a28ab9a738b4587/html5/thumbnails/13.jpg)
Facade recognition
13
The facade in the 3D GIS
![Page 14: Buildings Recognition and Camera Localization Using Image Texture Description](https://reader034.vdocument.in/reader034/viewer/2022052400/559867bc1a28ab9a738b4587/html5/thumbnails/14.jpg)
Work plan
q Enhancing the GIS database with building texture information
q Facade recognition
q Camera geolocation q Possible applicationsq Limits
14
![Page 15: Buildings Recognition and Camera Localization Using Image Texture Description](https://reader034.vdocument.in/reader034/viewer/2022052400/559867bc1a28ab9a738b4587/html5/thumbnails/15.jpg)
Camera Geolocation
15
Association of the interest points with the 3D position of the matched points in the GIS databases
![Page 16: Buildings Recognition and Camera Localization Using Image Texture Description](https://reader034.vdocument.in/reader034/viewer/2022052400/559867bc1a28ab9a738b4587/html5/thumbnails/16.jpg)
Camera geolocation
16
4 points non-collinear (Yang & al. 2009)
Real position Measured position
![Page 17: Buildings Recognition and Camera Localization Using Image Texture Description](https://reader034.vdocument.in/reader034/viewer/2022052400/559867bc1a28ab9a738b4587/html5/thumbnails/17.jpg)
Camera Geolocation
Error for distance (20-100)m and angle (0-30°) camera direction and facade normal :■ Position : 1 - 3 m■ Orientation : 5 - 10°
17
![Page 18: Buildings Recognition and Camera Localization Using Image Texture Description](https://reader034.vdocument.in/reader034/viewer/2022052400/559867bc1a28ab9a738b4587/html5/thumbnails/18.jpg)
Error because the texture description is not an affine function 18
Real positionMeasured position
4 points non-coplanaires SOFTPOSIT (David et al. 2004)
Camera Geolocation
![Page 19: Buildings Recognition and Camera Localization Using Image Texture Description](https://reader034.vdocument.in/reader034/viewer/2022052400/559867bc1a28ab9a738b4587/html5/thumbnails/19.jpg)
Work plan
q Enhancing the GIS database with building texture information
q Facade recognitionq Camera geolocation
q Possible applicationsq Limits
19
![Page 20: Buildings Recognition and Camera Localization Using Image Texture Description](https://reader034.vdocument.in/reader034/viewer/2022052400/559867bc1a28ab9a738b4587/html5/thumbnails/20.jpg)
Possible Applications
Management of photos taken in urban areas
20
Link
GIS
![Page 21: Buildings Recognition and Camera Localization Using Image Texture Description](https://reader034.vdocument.in/reader034/viewer/2022052400/559867bc1a28ab9a738b4587/html5/thumbnails/21.jpg)
Possible Applications Navigation systems support in an urban
environment
21
Satellites visibility Multipath
![Page 22: Buildings Recognition and Camera Localization Using Image Texture Description](https://reader034.vdocument.in/reader034/viewer/2022052400/559867bc1a28ab9a738b4587/html5/thumbnails/22.jpg)
Possible applications
Initial phase in the (2D/3D) registration
22
Sourimant 2009
![Page 23: Buildings Recognition and Camera Localization Using Image Texture Description](https://reader034.vdocument.in/reader034/viewer/2022052400/559867bc1a28ab9a738b4587/html5/thumbnails/23.jpg)
Work plan
q Enhancing the GIS database with building texture information
q Facade recognitionq Camera geolocation q Possible applications
q Limits
23
![Page 24: Buildings Recognition and Camera Localization Using Image Texture Description](https://reader034.vdocument.in/reader034/viewer/2022052400/559867bc1a28ab9a738b4587/html5/thumbnails/24.jpg)
Limits Angle between camera direction and facade
normal has to be less than 30°
24
![Page 25: Buildings Recognition and Camera Localization Using Image Texture Description](https://reader034.vdocument.in/reader034/viewer/2022052400/559867bc1a28ab9a738b4587/html5/thumbnails/25.jpg)
Limits
Distance between camera and facade has to be less than 200 m
25
![Page 26: Buildings Recognition and Camera Localization Using Image Texture Description](https://reader034.vdocument.in/reader034/viewer/2022052400/559867bc1a28ab9a738b4587/html5/thumbnails/26.jpg)
Limits
Identical facades :
26
![Page 27: Buildings Recognition and Camera Localization Using Image Texture Description](https://reader034.vdocument.in/reader034/viewer/2022052400/559867bc1a28ab9a738b4587/html5/thumbnails/27.jpg)
Limits
Glass facade which reflects the sky and other buildings
27