computing epipolar geometry from dynamic silhouettes

Computing Epipolar Geometry From Dynamic Silhouettes Synchronization and Calibration of a Camera Network for 3D Event Reconstruction from Live Video Sudipta N. Sinha Marc Pollefeys Department of Computer Science, University of North Carolina at Chapel Hill RANSAC-based Algorithm If the epipoles are known, then 3 pair of matching lines through the epipoles are required to compute the epipolar homography. RANSAC is used to randomly explore the 4D space of epipole hypothesis as well as robustly deal with incorrect silhouettes. Results : Pair-wise Epipolar Geometry on datasets. Goal : To Calibrate a Network of Cameras from video streams. Metric Camera Calibration from Fundamental (F) Matrices. Motivation: No Calibration Object (LED, grid) required. Handles wide baselines, lack of sufficient scene overlap, lack of enough point features and arbitrary camera configuration. Frontier Points and constraints provided by Epipolar Tangents. Compute Convex Hull + its Dual Hypothesis Step Verification Step Incremental Projective Reconstruction Given, F 12 , F 23 , F 13 compute consistent projective cameras P 1 , P 2 , P 3 Work in progress: Silhouet te Extracti on Compute Pair wise Epipolar Geometry using Silhouettes Metric Calibration Of Camera Network From Fundamental Matrices F 12 . . F jk P 1 P 2 . . P n Multiple Video Streams Funding: NSF Career IIS 0237533 & DARPA/DOI NBCH 1030015 Recovering Synchronization (Assume fixed known frame-rate) Add a random guess for temporal offset to the hypothesis. First, use slow-moving silhouettes for coarse sequence alignment and then use fast moving ones for recovering finer synchronization + epipolar geometry. Method of Induction used to add new cameras to existing network. The re-projected Visual Hull in one of the views Temporal Interpolation of Silhouettes MIT Dataset (4 views), 4 minutes of 30 fps video Synthetic Data (MPI Saarbrucken) (25 views, 200 frames) Presented at “Mathematical Methods in Computer Vision Workshop, Banff, Canada, Oct 2006 Name Camera s Frames Good F’s Projective Bundle Adj. MIT 4 7000 5 / 6 1580 pts 0.26 pixels INRIA 8 200 20 / 28 3243 pts 0.25 pixels KUNGFU 25 200 268 / 300 19632 pts 0.11 pixels GATECH 9 200 25 / 36 2503 pts 0.55 pixels CMU 8 900 22 / 28 5764 pts 0.47 pixels MPI 8 370 26 / 28 4975 pts 0.28 pixels SriKuma r 5 1000 9 / 10 1686 pts 0.24 pixels Incremental Projective Reconstruction + Projective Bundle Adjustment Self- Calibrat ion Euclidean Bundle Adjustmen t F 12 . . F jk P 1 P 2 . . P n 1. Projector Camera Calibration 2. Calibrating Hybrid Camera Networks Color, Range scanner, Depth sensor and Infra-red cameras. 3. Modify algorithm to work with fuzzy silhouettes to deal with gross silhouette extraction errors. MIT INRI A GATECH MPI CMU KUNGFU

Upload: oni

Post on 15-Jan-2016

37 views

Category:

Documents

2 download

Report

Download

Tags:

Embed Size (px):

DESCRIPTION

P 1 P 2 . . P n. Metric Calibration Of Camera Network From Fundamental Matrices. F 12 . . F jk. Compute Pair wise Epipolar Geometry using Silhouettes. Silhouette Extraction. Multiple Video Streams. MIT Dataset (4 views), 4 minutes of 30 fps video. - PowerPoint PPT Presentation

TRANSCRIPT

Computing Epipolar Geometry From Dynamic Silhouettes

Synchronization and Calibration of a Camera Network for 3D Event Reconstruction from Live Video

Sudipta N. Sinha Marc Pollefeys Department of Computer Science, University of North Carolina at Chapel Hill

RANSAC-based Algorithm If the epipoles are known, then 3 pair of matching lines through the epipoles are required to compute the epipolar homography.

RANSAC is used to randomly explore the 4D space of epipole hypothesis as well as robustly deal with incorrect silhouettes.

Results : Pair-wise Epipolar Geometry on datasets.Goal : To Calibrate a Network of Cameras from video streams.

Metric Camera Calibration from Fundamental (F) Matrices.

Motivation: No Calibration Object (LED, grid) required. Handles wide baselines, lack of sufficient scene overlap, lack of enough point features and arbitrary camera configuration.

Frontier Points and constraints provided by Epipolar Tangents. Compute Convex Hull + its Dual

Hypothesis Step

Verification Step

Incremental Projective Reconstruction

Given, F12 , F23 , F13

compute consistent projective camerasP1 , P2 , P3 Work in progress:

Silhouette Extraction

Compute Pair wise Epipolar Geometry using Silhouettes

Metric CalibrationOf Camera

Network From Fundamental

Matrices

F12..Fjk

P1P2..PnMultiple Video Streams

Funding: NSF Career IIS 0237533 & DARPA/DOI NBCH 1030015

Recovering Synchronization (Assume fixed known frame-rate)

Add a random guess for temporal offset to the hypothesis. First, use slow-moving silhouettes for coarse sequence alignment and then use fast moving ones for recovering finer synchronization + epipolar geometry.

Method of Induction used to addnew cameras to existing network.

The re-projected Visual Hull in one of the views

Temporal Interpolation of Silhouettes

MIT Dataset (4 views), 4 minutes of 30 fps video

Synthetic Data (MPI Saarbrucken) (25 views, 200 frames)

Presented at “Mathematical Methods in Computer Vision Workshop, Banff, Canada, Oct 2006

Name Cameras Frames Good F’s Projective Bundle Adj.

MIT 4 7000 5 / 6 1580 pts 0.26 pixels

INRIA 8 200 20 / 28 3243 pts 0.25 pixels

KUNGFU 25 200 268 / 300 19632 pts 0.11 pixels

GATECH 9 200 25 / 36 2503 pts 0.55 pixels

CMU 8 900 22 / 28 5764 pts 0.47 pixels

MPI 8 370 26 / 28 4975 pts 0.28 pixels

SriKumar 5 1000 9 / 10 1686 pts 0.24 pixels

Incremental Projective Reconstruction

+ Projective Bundle

Adjustment

Self-Calibration

Euclidean Bundle

Adjustment

F12..Fjk

P1P2..Pn

1. Projector Camera Calibration2. Calibrating Hybrid Camera Networks Color, Range scanner, Depth sensor and Infra-red cameras.3. Modify algorithm to work with fuzzy silhouettes to deal with gross silhouette extraction errors.

MIT

INRIA

GATECH

MPI

CMU

KUNGFU

Previously Two view geometry: epipolar geometry Stereo vision: 3D reconstruction epipolar lines Baseline O O’ epipolar plane

Epipolar Geometry and the Fundamental Matrixqji/CV/epipolar_fundF.pdf9 Epipolar Geometry and the Fundamental Matrix The epipolar geometry is the intrinsic projective geometry between

EPIPOLAR GEOMETRY - Rutgers School of Engineeringsoe.rutgers.edu/~meer/UGRAD/cv13epipolar.pdfan epipolar plane intersects the left and right image planes in epipolar lines. Epipolar

Epipolar Geometry, cameras Feature Detection,facerec.ece.wisc.edu/cameranet/my-epipolar.pdf1 Epipolar Geometry, Feature Detection, and Feature Matching Multi-View Geometry Different

[PPT]Multiple View Geometry in Computer Visioncs.unc.edu/~marc/mvg/course11.ppt · Web viewThree questions: The epipolar geometry C,C’,x,x’ and X are coplanar The epipolar geometry

Epipolar Geometry and Stereo Vision - Course Website Directory