Discovering Panoramas in Web Videos

ACM Multimedia 2008Feng Liu1, Yuhen-Hu1,2 and Michael Gleicher1

Introduction Video analysis Discovering panoramas Panorama synthesis Experiments Conclusion

Outline

STEP 1 image alignment

STEP2 image stitching

Creating panoramic imagery

Not all video has appropriate sources◦ Not cover a wide field-of-view of a scene◦ Motion may be randomly◦ Image quality

Image sources in a video

Three parts◦ video analysis◦ panorama source selection◦ panorama synthesis

Purpose-Discover Panoramas in Video

Background-image homography

transformation

Feature matching – SIFT Compute homography parameters – RANSAC

algo◦ Run k times:

(1)draw n samples randomly (2) fit parameters Θ with these n samples (3) for each of other N-n points, calculate its

distance to the fitted model, count the number

of inlier points, c◦ Output Θ with the largest c

Background-compute homography

Example:line fittingn=2

c=3 c=15

…………………

Image homography◦ Points should match◦ Measure error distance and give penalty

Moving object detect◦ For activity synopsis◦ examining the discrepancy between its local

motion vector and the global motion

Video analysis (1)

Visual quality measures

Video analysis(2)

Method of [31]Tong et al 04

Method of [35]Wang et al 02

Average differencesacross block boundaries.

Good panoramas◦ Good homography between frames◦ Video have high image quality◦ Cover a wild field view

Collision◦ More frame more wild field of view◦ More frame more accumulate error to degrade

quality

Discovering panoramas(1)

, vistual quality, extent of the scene

Visual quality measure

Discovering panoramas(2)

Scene extent measure

Discovering panoramas(3)

Reference Reference

An Approximate Solution Steps◦ 1.Fetch a segment Sk from pool Sp◦ 2.Find the scene extent of Sk and corresponding

reference frame.◦ 3.Append the panorama set according to

equation(2). until .

◦ 4.If the scene meet , , add remainder to pool Sp.

◦ 5.If pool Sp != Ο , go to loop 1

Discovering panoramas(4)

Discovering panoramas(4)shot boundary

segments

video

divide segments that have too penalty

Repeat until done

Discard those extent with too little coverage <

Scene panorama synthesis◦ blending – feathering

◦ median-bilateral filtering

Panorama synthesis(1)

Panorama synthesis(2) Activity synopsis synthesis

Detect

Discard

Select and composite into scene

YouTube Travel and Events category – West Lake http://www.youtube.com/watch?v=6FKCHLfTns8&feature=player_embedded#!

◦ size 320 x 240

Experiments (1)

Query panorama from YouTube 6 query , top 10 videos 86.7% contain panoramas

Experiments (2)

Example

Example

ExampleNotre Dame, Paris

In this paper, we presented an automatic method to discover panorama sources from casual videos.

“Query panoramas from YouTube”supports our proposal of using web videos as panorama source.

More importantly, this method contribute to presenting or summarizing imagery databases using panoramic imageries by mining the possible sources to synthesize the representations.

Conclusion

[31] H. Tong, M. Li, H. Zhang, and C. Zhang. Blur detection for digital images using wavelet transform.In IEEE ICME, 2004.

[35] Z. Wang, G. Wu, H. Sheikh, E. Simoncelli, E.-H.Yang, and A. Bovik. Quality-aware images. IEEE Transactions on Image Processing, 15(6):1680 -1689,2006.

Original Videos: http://pages.cs.wisc.edu/~fliu/project/discover-pano.htm

Reference