kari panoramas 02mar10 opt

113
Panoramas Kari Pulli Research Fellow Nokia Research Center Palo Alto 1

Upload: sriram-sampathkumar

Post on 11-Oct-2015

14 views

Category:

Documents


0 download

DESCRIPTION

Panoramas

TRANSCRIPT

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    1/113

    Panoramas

    Kari P ulli

    Research Fellow

    No kia Research Center Palo Alto

    1

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    2/113

    Are you getting the whole picture?

    Compact Camera FOV = 50 x 35

    Slide from Brown & Lowe2

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    3/113

    Are you getting the whole picture?

    Compact Camera FOV = 50 x 35

    Human FOV = 200 x 135

    Slide from Brown & Lowe3

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    4/113

    Are you getting the whole picture?

    Compact Camera FOV = 50 x 35

    Human FOV = 200 x 135

    Panoramic Mosaic = 360 x 180

    Slide from Brown & Lowe4

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    5/113

    PanoramaA wide-angle representation of the scene

    Panorama ofAlong the River During Qingming Festival, 18th century remake of a 12th century original by

    Chinese artist Zhang Zeduan.

    Image from Wikipedia

    5

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    6/113

    Panorama: Cinema for the early 19thcentury

    Burfords Panorama, Leicester Square, London, 1801

    Painting by Robert Mitchell6

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    7/113

    Creating panoramas with wide-angle optics

    http://www.0-360.com AF DX Fisheye-NIKKOR 10.5mm f/2.8G ED

    7

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    8/113

    Idea

    rotate camera or lens so that a vertical slit is

    exposed

    Swing lens rotate the lens and a vertical slit (or the sensor)

    typically can get 110-140 degree panoramas

    Widelux, Seitz,

    Full rotation

    whole camera rotates

    can get 360 degree panoramas

    Panoscan, Roundshot,

    8

    Rotation cameras

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    9/113

    Swing-lens panoramic images

    San Francisco in ruins, 1906

    101 Ranch, Oklahoma, circa 19209

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    10/113

    Lee Frost, Val DOrcia, Tuscany, Italy

    Flatback panoramic camera

    10

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    11/113

    Disposable panoramic camerawide-angle lens, limited vertical FOV

    11

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    12/113

    12

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    13/113

    13

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    14/113

    14

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    15/113

    15

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    16/113

    Stitching images together to make a mosaic

    16

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    17/113

    Stitching images together to make a mosaic

    Given a set of images that should stitch together by rotating the camera around its center of perspective

    Step 1: Find corresponding features in a pair of image

    Step 2: Compute transformation from 2

    nd

    to 1

    st

    imageStep 3: Warp 2ndimage so it overlays 1stimage

    Step 4: Blend images where they overlap one another

    repeat for 3rdimage and mosaic of first two, etc.17

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    18/113

    Aligning images: Translation?

    Translations are not enough to align the images

    left on top right on top

    18

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    19/113

    A pencil of rays contains all views

    real

    camerasynthetic

    camera

    Can generate any synthetic camera view

    as long as it has the same center of projection!

    and scene geometry does not matter 19

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    20/113

    Reprojecting an image onto

    a different picture plane

    the view on any picture plane can be projected onto

    any other plane in 3D without changing itsappearance as seen from the center of projection

    the sidewalk art of Julian Beever

    20

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    21/113

    mosaic PP

    Image reprojection

    The mosaic has a natural interpretation in 3D

    the images are reprojected onto a common plane

    the mosaic is formed on this plane

    mosaic is asynthetic wide-angle camera

    21

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    22/113

    Which transform to use?

    Translation

    2 unknowns

    Affine

    6 unknowns

    Perspective

    8 unknowns

    22

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    23/113

    Homography

    Projective mapping between any two PPs with thesame center of projection

    rectangle should map to arbitrary quadrilateral

    parallel lines arent

    but must preserve straight lines

    called Homography PP2

    PP1

    wx'

    wy'

    w

    "

    #

    $$$

    %

    &

    '''

    =

    h11 h12 h13

    h21

    h22

    h23

    h31

    h32

    h33

    "

    #

    $$$

    %

    &

    '''

    x

    y

    1

    "

    #

    $$$

    %

    &

    '''

    H pp

    To apply a homography H Compute p= Hp (regular matrix multiply)

    Convert pfrom homogeneous to imagecoordinates [x,y] (divide by w)

    23

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    24/113

    24

    Homography from mapping quads

    Fundamentals of Texture Mapping and Image Warping

    Paul Heckbert, M.Sc. thesis, U.C. Berkeley, June 1989, 86 pp.http://www.cs.cmu.edu/~ph/texfund/texfund.pdf

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    25/113

    Multiply out

    wx = h11x + h12y + h13

    wy = h21x + h22y + h23

    w = h31x + h32y + h33

    Get rid of w (h31x + h32y + h33)x (h11x + h12y + h13) = 0

    (h31x + h32y + h33)y (h21x + h22y + h23) = 0

    Create a new system Ah = 0

    Each point constraint gives two rows of A

    [-x -y -1 0 0 0 xx yx x]

    [ 0 0 0 -x -y -1 xy yy y]

    Solve with singular value decompositionof A = USVT

    solution is in the nullspace of A

    the last column of V (= last row of VT)

    25

    Homography from npoint correspondences

    wx'

    wy'

    w

    "

    #

    $$$

    %

    &

    '''

    =

    h11

    h12

    h13

    h21

    h22

    h23

    h31

    h32

    h33

    "

    #

    $$$

    %

    &

    '''

    x

    y

    1

    "

    #

    $$$

    %

    &

    '''

    H pp

    h11

    h12

    h13

    h21

    h = h22 h23

    h31

    h32

    h33

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    26/113

    from numpy import *

    # create original points

    X = ones([3,4])

    X[:2,:] = random.rand(2,4)

    x,y = X[0],X[1]

    # create projective matrix

    H = random.rand(3,3)

    # create the target points

    Y = dot(H,X)

    # homogeneous division

    YY = (Y / Y[2])[:2,:]

    u,v = YY[0],YY[1]

    A = zeros([8,9])

    for i in range(4):

    A[2*i ] = [-x[i], -y[i], -1, 0, 0, 0, x[i] * u[i], y[i] * u[i], u[i]]

    A[2*i+1] = [ 0, 0, 0, -x[i], -y[i], -1, x[i] * v[i], y[i] * v[i], v[i]]

    [u,s,vt] = linalg.svd(A)

    # reorder the last row of vt to 3x3 matrix

    HH = vt[-1,:].reshape([3,3])

    # test that the matrices are the same (within a multiplicative factor)

    print H - HH * (H[2,2] / HH[2,2])

    Python test code

    26

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    27/113

    Summary of perspective stitching

    Pick one image, typically the central view (red outline)

    Warp the others to its plane

    Blend27

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    28/113

    Example

    perspective reprojection

    common

    picture

    plane of

    mosaicimage

    28

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    29/113

    Using 4 shots instead of 3

    perspective reprojection29

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    30/113

    cylindrical reprojection

    Back to 3 shots

    surface ofcylinder

    30

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    31/113

    cylindrical reprojection

    Back to 3 shots

    surface ofcylinder

    31

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    32/113

    Back to 3 shots

    perspective reprojection32

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    33/113

    Cylindrical panoramas

    What if you want a 360 panorama?

    Project each image onto a cylinder

    A cylindrical image is a rectangular array

    mosaicimage

    x

    y

    33

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    34/113

    Cylindrical panoramas

    What if you want a 360 panorama?

    Project each image onto a cylinder

    A cylindrical image is a rectangular array

    To view without distortion

    reproject a portion of the cylinder onto a picture plane representing the

    display screen

    mosaicimage

    x

    y

    34

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    35/113

    2ndreprojection to a plane for display

    Imagine photographing the inside of a cylinder that iswallpapered with this panorama

    if your FOV is narrow, your photo wont be too distorted

    display

    FOV

    35

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    36/113

    http://graphics.stanford.edu/courses/cs178/applets/projection.html

    36

    Demo

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    37/113

    Changing camera center

    Does it still work? synthetic PP

    PP1

    PP2

    37

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    38/113

    Where to rotate? Nodal point?

    http://www.reallyrightstuff.com/pano/index.html

    38

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    39/113

    Many instructions say rotate around

    the nodal point wrong!

    http://toothwalker.org/optics/

    misconceptions.html#m6

    Correct: the entrance pupil

    the optical image of the physical aperture stop

    as 'seen' through the front of the lens system

    due to the magnifying effect of the front lens,

    the entrance pupil's location is nearer than

    that of the physical aperture

    39

    Rotate around the center of lens perspective

    The front and rear nodal points

    have the property that a rayaimed at one of them will be

    refracted by the lens such that itappears to have come from the

    other, and with the same angle

    with respect to the optical axis.

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    40/113

    40

    Test for

    parallax

    http://toothwalker.org/optics/cop.html#stitching

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    41/113

    41

    Wrong center of rotation -> parallax

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    42/113

    Map 3D point (X,Y,Z) onto cylinder

    Cylindrical projection

    X

    Y

    Z

    unit cylinder

    unwrapped cylinder

    Convert to cylindrical coordinates

    Convert to cylindrical image

    coordinates

    42

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    43/113

    Cylindrical projection

    Y

    X43

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    44/113

    Inverse cylindrical projection

    X

    Y

    Z

    (X,Y,Z)

    (sin!,h,cos!)

    44

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    45/113

    f = 180 (pixels)

    Focal length

    f = 380f = 280Image 384x300

    top-down view Focal length

    45

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    46/113

    Can get a rough estimate by measuring FOV:

    Can use the EXIF data tag

    might not give the right thing

    Can use several images together

    find f that makes them matchEtc.

    Focal length is (highly!) camera dependant

    46

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    47/113

    Assembling the panorama

    Stitch pairs together, blend, then crop

    47

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    48/113

    Problem: Drift

    Vertical Error accumulation small (vertical) errors accumulate over time

    apply correction so that sum = 0 (for 360 pan.)

    Horizontal Error accumulation

    can reuse first/last image to find the right panorama radius

    48

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    49/113

    Spherical projection

    Convert to spherical coordinatesX

    Y

    Z

    Map 3D point (X,Y,Z) onto sphere

    Convert to spherical image coordinates

    unwrapped sphere

    "

    49

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    50/113

    Spherical Projection

    Y

    X50

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    51/113

    Inverse Spherical projection

    X

    Y

    Z

    (x,y,z)

    (sin!cos",cos!cos",sin"

    )

    51

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    52/113

    Full-view Panorama

    52

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    53/113

    Building a Panorama

    M. Brown and D. G. Lowe. Recognising Panoramas. ICCV 200353

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    54/113

    We need to match (align) images

    54

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    55/113

    Detect feature points in both images

    55

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    56/113

    Find corresponding pairs

    56

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    57/113

    Use these pairs to align images

    57

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    58/113

    Matching with Features

    Problem 1:

    Detect thesamepoint independentlyin both images

    no chance to match!

    We need a repeatable detector

    58

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    59/113

    Matching with Features

    Problem 2:

    For each point correctly recognize the corresponding one

    ?

    We need a reliable and distinctivedescriptor

    59

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    60/113

    We should easily recognize the point by looking through

    a small window

    Shifting a window in any direction should give a large

    change in intensity

    Harris Corners: The Basic Idea

    60

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    61/113

    Harris Detector: Basic Idea

    flatregion:

    no change in all

    directions

    edge:

    no change along the

    edge direction

    corner:

    significant change in all

    directions

    61

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    62/113

    Harris Detector: Mathematics

    Window-averaged change of intensity for the shift [u,v]:

    IntensityShiftedintensity

    Windowfunction

    orWindow function w(x,y)=

    Gaussian1 in window, 0 outside62

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    63/113

    Harris Detector: Mathematics

    Expanding E(u,v)in a 2ndorder Taylor series expansion,

    we have, for small shifts [u,v], a bilinearapproximation:

    where Mis a 2$2 matrix computed from image derivatives:

    63

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    64/113

    Eigenvalues%1,%2of M at different locations

    %1and %2are large

    64

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    65/113

    Eigenvalues%1,%2of M at different locations

    large%1, small %265

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    66/113

    Eigenvalues%1,%2of M at different locations

    small%1, small %266

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    67/113

    Harris Detector: Mathematics

    %1

    %2

    Corner

    %1and %2are large,

    %1 ~ %2;Eincreases in all

    directions

    %1and %2are small;

    E is almost constantin all directions

    Edge%1>> %2

    Edge

    %2>> %1

    Flatregion

    Classification of

    image points using

    eigenvalues of M:

    67

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    68/113

    Harris Detector: Mathematics

    Measure of corner response:

    (k empirical constant, k = 0.04-0.06)

    68

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    69/113

    Harris Detector: Mathematics

    %1

    %2 Corner

    Edge

    Edge

    Flat

    Rdepends only on

    eigenvalues of M

    Ris large for a corner

    Ris negative with large

    magnitude for an edge

    |R| is small for a flat

    region

    R> 0

    R< 0

    R< 0|R|small

    69

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    70/113

    Ha rris Detector Wo rkflow

    70

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    71/113

    Ha rris Detector Wo rkflow

    Compute corner response R

    71

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    72/113

    Ha rris Detector Wo rkflow

    Find points with large corner response: R>threshold

    72

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    73/113

    Ha rris Detector Wo rkflow

    Take only the points of local maxima of R

    73

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    74/113

    Ha rris Detector Wo rkflow

    74

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    75/113

    Harris Detector: Summary

    Average intensity change in direction [u,v]can beexpressed as a bilinear form:

    Describe a point in terms of eigenvalues of M:measure of corner response

    A good (corner) point should have a large intensity change

    in all directions, i.e., Rshould be large positive75

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    76/113

    Harris Detector: Invariant to rotation

    Ellipse rotates

    but its shape (i.e., eigenvalues) remains the same

    Corner response Ris invariant to image rotation

    76

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    77/113

    Partial invariance

    Harris Detector: ~Invariant to intensite change

    !

    Only derivatives are used

    =>

    invariance to intensity shiftI&I + b

    !

    Intensity scale:I &a I

    R

    x(image coordinate)

    threshold

    R

    x(image coordinate)

    77

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    78/113

    Harris Detector:ot

    invariant to image scale!

    All points will be

    classified as edges

    Corner !

    78

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    79/113

    We know how to detect points

    Next question:

    How to match them?

    Point Descriptors

    ?

    Point descriptor should be:

    1.

    Invariant

    2. Distinctive79

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    80/113

    Descriptor overview:

    Determine scale (by maximizing DoG in scale and in space),

    local orientation as the dominant gradient direction.

    Use this scale and orientation to make all further computations invariant to

    scale and rotation.

    SIFT Scale Invariant Feature Transform

    D. Lowe. Distinctive Image Features from Scale-Invariant Keypoints IJCV 2004

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    81/113

    Descriptor overview:

    Determine scale (by maximizing DoG in scale and in space),

    local orientation as the dominant gradient direction.

    Use this scale and orientation to make all further computations invariant to

    scale and rotation.

    Compute gradient orientation histograms of several small windows (128

    values for each point)

    Normalize the descriptor to make it invariant to intensity change

    SIFT Scale Invariant Feature Transform

    D. Lowe. Distinctive Image Features from Scale-Invariant Keypoints IJCV 2004

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    82/113

    82

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    83/113

    Registration in practice: tracking

    Camera Module

    Real-Time

    Tracking

    !"#$% '()*$+

    time

    Current location

    83

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    84/113

    84

    Viewfinder alignment for tracking

    Andrew Adams, Natasha Gelfand, Kari Pulli

    Viewfinder Alignment

    Eurographics 2008 http://graphics.stanford.edu/papers/viewfinderalignment/

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    85/113

    85

    Project gradients along columns and rows

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    86/113

    86

    diagonal gradients along diagonals

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    87/113

    87

    and find corners

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    88/113

    88

    Overlap and match the gradient projections

    and determine translation

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    89/113

    89

    Apply the best translation to corners

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    90/113

    90

    Match corners, refine translation & rotation

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    91/113

    System Overview

    Camera Module

    Real-Time

    Tracking

    !"#$% '()*$+

    ,"-. /$+%012%3

    4*)-$+

    time

    Current location Panoramaexpansion

    91

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    92/113

    System Overview

    Camera Module

    Real-Time

    Tracking

    !"#$% '()*$+

    ,"-. /$+%012%3

    4*)-$+

    Image

    Warping

    Image

    Registration

    92

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    93/113

    Requirements

    Robust to illumination changes

    Robust to object motion in the scene

    Low computational complexity

    Image registration

    Registration example (including large illumination changes)93

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    94/113

    Hybrid multi-resolution approach

    I.B.

    F.B.

    F.B.

    F.B.

    Initial guess

    Registration parameters

    I.B. Image Based

    F.B. Feature Based

    94

    K. Pulli, M. Tico, Y. Xiong, X. Wang, C-K. Liang, Panoramic Imaging System for Camera Phones, ICCE 2010

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    95/113

    Progression of multi-resolution registration

    Actual

    size

    Applied

    to hi-res

    95

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    96/113

    Feature-based registration

    + +

    Convert to spherical

    coordinates

    Convert from spherical

    coordinates

    Apply the previous

    registration estimate

    + +

    Best block cross-

    correlation match

    Feature Detection

    (Harris corners)

    Previous estimate

    Feature Matching

    (spherical coordinates)RANSAC

    Validity

    check

    New estimate

    valid

    Update search range

    invalid

    96

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    97/113

    System overview

    97

    Camera Module

    Real-Time

    Tracking

    !"#$% '()*$+

    ,"-. /$+%012%3

    4*)-$+

    Image

    Warping

    Image

    Registration

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    98/113

    System overview

    Camera Module

    Real-Time

    Tracking

    !"#$% '()*$+

    ,"-. /$+%012%3

    4*)-$+

    Image

    Warping

    Image

    Registration

    Image

    Blending

    !"#$% '$#()$*$

    Photo by Marius Tico

    98

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    99/113

    Image blending

    Directly averaging the overlapped pixels results in ghosting artifacts

    Moving objects, errors in registration, parallax, etc.

    Photo by Chia-Kai Liang

    99

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    100/113

    Solution: Image labeling

    Assign one input image each output pixel

    Optimal assignment can be found by graph cut [Agarwala et al. 2004]

    100

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    101/113

    New artifacts

    Inconsistency between pixels from different input images

    Different exposure/white balance settings

    Photometric distortions (e.g., vignetting)

    101

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    102/113

    Solution: Poisson blending

    Copy the gradient field from the input imageReconstruct the final image by solving a Poisson equation

    Combined gradient field

    102

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    103/113

    Seam finding gets difficult w hen colors differ

    No color correction With color correction

    103

    Y. Xiong, K. Pulli, Fast Panorama Stitching on Mobile Devices, ICCE 2010

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    104/113

    Color correction in linearized RGB

    104

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    105/113

    105

    Dyn am ic programm ing finds go od cuts fast

    " Cumulative minimum error surface

    " Error surface

    Overlapping area in the current composite image

    Overlapping area in the current source image

    Overlapping areaSource images

    Error surf cumulative

    Optimal path

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    106/113

    106

    Fast Poisson blending

    SIGGRAPH 2009

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    107/113

    107

    Alpha blending

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    108/113

    p g

    After labeling

    Poisson blending

    108

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    109/113

    System Overview

    Camera Module

    Real-Time

    Tracking

    !"#$% '()*$+

    ,"-. /$+%012%3

    4*)-$+

    Image

    Warping

    Image

    Registration

    Image

    Blending

    5($6"$7 %3 5.%3$

    !"#$% '$#()$*$

    Photo by Marius Tico

    109

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    110/113

    Panorama Visualization

    Trivial method:

    Show the whole panorama on the screen

    Zooming and panning

    110

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    111/113

    No Projection Method is Optimal

    Zoom Spherical Projection Perspective Projection

    111

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    112/113

    Solution: Interpolate the Projection Coordinates

    Weights are determined

    by a sigmoid function ofzoom factor

    Zoom

    112

  • 5/20/2018 Kari Panoramas 02mar10 Opt

    113/113

    Slide credits

    Fredo Durand

    Alyosha Efros

    Bill Freeman

    Marc Levoy

    Chia-Kai Liang

    Steve Seitz

    Rick Szeliski

    Marius Tico

    Yingen Xiong