REU WEEK IIIMalcolm Collins-SibleyMentor: Shervin Ardeshir

PROJECT• Cross-View Image Registration and

Semantic Segmentation• The goal is to use information from map

and satellite images, and project them on the screen which the user is observing, in a way that the user can see semantic segments overlaid on the scene.

PROJECT• Output Mockup

COMPLETED WORK• Readings:

• “Geometric Image Parsing in Man-Made Environments”• Olga Barinova et al

• “Recovering Surface Layout from an Image”• Derek Hoiem et al

• “Recovering Occlusion Boundaries from a Single Image”• Derek Hoeim et al

• “Entropy Rate Superpixel Segmentation”• MY Liu et al

COMPLETED WORK• Geometric Image Parsing Code

COMPLETED WORK• Geometric Image Parsing Code

COMPLETED WORK• Super-pixel Segmentation

With 8 super-pixels

COMPLETED WORK• Super-pixel Segmentation

With 20 super-pixels

COMPLETED WORK• Building Projection

COMPLETED WORK• Building Projection

CURRENT WORK• Within the Building Projection code:

• Building occlusion and self-occlusion• Works when a building occludes another,

but not when a building is occluding itself

CURRENT WORK• Occlusion Handling

Before After

CURRENT WORK• Occlusion HandlingBefore After

CURRENT WORK• Occlusion HandlingBefore After

CURRENT WORK• Occlusion HandlingBefore After

THE NEXT STEP• Understanding the occlusion handling code

• Making sure it is handling self-occlusions accurately

• Understanding the format of the output data in the line segments/horizon code

• Running the line segmentation code for all of the images in our dataset and saving all of the output variables in a structure

• Extracting the super pixels from images in the dataset and saving it in a structure

• Computing their pairwise similarities of the super pixels in terms of color and texture