Background Removal of Multiview Images by Lea

rning Shape Priors

Yu-Pao Tsai, Cheng-Hung Ko, Yi-Ping Hung, and Zen-Chung Shih

IEEE 2007

IntroductionIntroduction

Multiview images (MVIs) segmentation ─ In order to integrate image-based 3-D objects into a

chosen scene to efficiently and effectively remove the background from the foreground object.

MVI segmentation is image-based 3-D reconstruction using multiview images.

pan angle tilt angle

The notation of the MVI：

FlowchartFlowchart

MethodMethod

Automatic Initial SegmentationAutomatic Initial Segmentation Graph Cut Image SegmentationGraph Cut Image Segmentation Trimap Labeling Trimap Labeling

Segmentation With Shape PriorsSegmentation With Shape Priors Volumetric Graph CutsVolumetric Graph Cuts Distance Medial Axis ConstraintDistance Medial Axis Constraint

Graph Cut Image SegmentationGraph Cut Image Segmentation──using to “Boykov and Jolly” proposed methodusing to “Boykov and Jolly” proposed method

Trimap LabelingTrimap Labeling1) When an equi-tilt set of the MVI is captured, a larg

e proportion of the background scene is static.

2) Only one interesting object is presented in every image of the MVI.

3) The foreground and background color distributions are distinct in most cases.

Trimap Labeling:Trimap Labeling:

1)1) -Labeling-Labeling

2)2) -Labeling-Labeling

Volumetric Graph CutsVolumetric Graph Cuts

Discrete Medial Axis (DMA) ConstraintDiscrete Medial Axis (DMA) Constraint

ResultsResults

Initial Segmentation ResultsInitial Segmentation Results

Fig. 7. Results of the automatic initial segmentation corresponding to the image sequence shown in Fig. 3. The two images on the left show the segmentation results that should be selected for the 3-D reconstruction, while the other shows the segmentation result that should be excluded and refined in the next run. The red circles denote the noticeable segmentation errors in the image.

Fig. 8. Top row shows a portion of an equi-tilt set for the toy house MVI. The middle row shows the trimap labeling result for each image. Finally, the bottom row shows the results of the automatic initial segmentation. The red circles indicate the noticeable segmentation errors in each image, to be rectified in the next run.

Learning Shape PriorLearning Shape Prior

Rectification of Segmentation ErrorsRectification of Segmentation Errors

Fig. 13. First row shows three consecutive images in an equi-tilt set of the pottery cat MVI. The second row shows the result of trimap labeling. The third row shows the result of the automatic initial segmentation. In the fourth row, the projectionof the reconstructed 3-D model provides the information on regions that is quite difficult to obtain by the methods based on color and contrast alone. The last row shows the refinement of the segmentation result by using shape priors.

Fig. 14. First row shows three consecutive images in an equi-tilt set of the Armadillo MVI. Second row shows the result of trimap labeling. The third row shows the result of the automatic initial segmentation. In the fourth row, the projection of the reconstructed 3-D model provides the information on regions that is quite difficult to obtain by the methods based on color and contrast alone. Last row shows the refinement of the segmentation result by using shape priors, the comparison between the segmentation results produced by the proposed method and the ground truth. Red solid lines denote the contours of the ground truth,and the green dot lines denote the segmentation results produced by the proposed method.