1 25 years of computer graphics: back to the future? john patterson computer vision and graphics...
TRANSCRIPT
1
25 Years of Computer Graphics:Back to the Future?
John PATTERSON
Computer Vision and Graphics Group
Dept of Computing Science
University of GLASGOW
25 Years of Graphics at Bath
University of Bath
14-September-04
2
Back to the Future?
• ..maybe back to the 70s at 21st century clock rates
• e.g. graphics CPU/GPU relationship periodically gets reinvented
• What about relationship between Vector and Raster graphics?
• Advocate putting vector formats on same basis as raster formats
• Many pitfalls
• low-level approach because these issues need fixing first
3
Topics
• Fiddling at the margins
• Resizing and Resampling (and inter alia an outrageous proposal)
• What is a pixel anyway?
• Vectors Rule OK
4
Fiddling at the Margins (1)
Catmull-Rom patch with control points (x, y, h)
(x,y): pixel coordinates
h: pixel grey-value
Isochromic surface for original raster image:
What happens at the edges?
5
Edge PixelCurve segmentPP P Petc.12 3 4
Curve segmentP P Petc.2 3 4P0P1
Fiddling at the Margins (2)
curve is only defined between and
€
P2
€
P3
Need to ‘hallucinate’ additional point to accommodate edge pixel
€
P0otherwise get image erosion. Also applies to convolution filters, which may need more pixels ‘hallucinated’ to avoid erosion e.g. 2 for 5 x 5, 3 for 7 x 7 etc.
6
Fiddling at the Margins (3)
Set to zero
Set to
Extrapolate from (linear)
Extrapolate from (quadratic)
Extrapolate from (cubic)
..or?
€
P0
€
P0
€
P1
€
P2,P1
€
P3,P2,P1
€
P4,P3,P2,P1
The options are:
Not clear which, if any is ‘appropriate’.
7
Method: Crimesti, Perez & Toyama “Object removal by Exemplar-Based Inpainting “ CVVG 2003
This implementation favours reinforcement of linear features
Original Image Segment cut out Synthesised result
..or - use in-painting?- (1)
8
Hole regionImage region
Hole regionImage region
Hole regionImage region
..or - use in-painting (2)
•Find highest confidence point on fill front
•Track along isochromic contour for best match
•Replace point by matched region, confidence values = pixel match accuracy
Local isochromic contour
Patch size should be big enough to contain texture cell, here 9 x 9
9Texture synthesis example:
Synthesised result
Example shows repair of linear features. While a characteristic of the algorithm a different ‘data’ term was used to prioritise in-filling points here
Original Image
Segments cut out
10
Application to edge extension
High confidence point
In-painting involves reducing hole using high confidence points
Edge extension involves extending image with less support from local context
11
Resizing and Resampling (1)
€
1 −tan θ 2( )0 1
⎛
⎝ ⎜
⎞
⎠ ⎟
€
1 −tan θ 2( )0 1
⎛
⎝ ⎜
⎞
⎠ ⎟
€
1 0
sin θ( ) 1
⎛
⎝ ⎜
⎞
⎠ ⎟
Any whole-image transformation requires (a) resampling stage(s)
Example: rotation through using 3 skew operations:
Skew along x axis, resample in x
Skew along y axis, resample in y
Skew along x axis, resample in x
12
Resizing and Resampling (2)
This is what happens when you don’t resample in a skew
Fairly horrible!
€
1 −tan θ 2( )0 1
⎛
⎝ ⎜
⎞
⎠ ⎟
13
Resizing and Resampling (3)Even re-sizing requires resampling: carry out in 2 passes,
scale in x, resample in x; scale in y, resample in y
Ideal reconstruction filter:
€
sinc x( ) =sin x( )x
=1−x2
3!+x4
5!−x6
7!etc. is 1 when x=0
Gaussian is not very good approximation
Catmull-Rom basis is better
14
Resizing and Resampling (4)
A small entertainment:
Gaussian kernel usually modelled as right:
Usually applied in convolution using weighted samples, here weights modelled
as heights to match curve above
Alternative is to use stochastic sampling: treat Gaussian function as a probability
field and take spatially distributed samples which are averaged
Could do this for the sinc function also (and subtract samples in negative lobes)
Defeated by need for too many samples? What happens at edges?
15
Resizing and Resampling (5)So- how to resample?
• Catmull-Rom is popular, but is only first-order continuous
• Central convolution interpolation advocated- still first-order order continuous
•Resampling assumes continuity between samples. What about edges? Zero crossings (sign reversals in the second difference) should give these away (e.g. Canny)
• Also assumes you can trust your pixel values:
Invented by actuaries 100+ years ago
Use differences up to third degree and interpolate ‘up the middle’ using weights (convolution). Various formulae map out more sinc lobes
Point spread function
Noise
Blur (can blur edges away)
16
What is a pixel anyway?
32 x 32 pixel
image
An honest topography
17
The pretence
Image
ContinuousSurface (here Catmull-Rom)
Slicing the surface gives an isochromic contour (but need interpolation)
18
Girl’s face photoInput Image
Contour or ‘Vector’ form
Rendered from vectors
19
Direct Comparison at same sizes
Original image x4 bicubic Rendered image from contour map
20
Level Sets
Level Sets and Fast Marching methods originally developed to solve flame-front advancement calculations involving PDEs
Similar to reaction-diffusion equations originally worked on by Turing in the 1930s, so collectively known as ‘diffusion’
Outer contour
Inner contour
Outer boundary ‘advances’ towards inner boundary according to PDEs controlled by boundary conditions, e.g. curvature
Intermediate contour defines boundary position at set ‘time’ or parametric distance
Many image indexing problems may be cast in this form
21
Conclusion
• Images can be represented in terms of control points for isochromic contours
• Images shown do not take full advantage of smoothing options (data reduction) and do not yet use level set diffusion
• Whole image operations, including matting, matte-pulling, and texture synthesis can be expressed in vector form
• Computational topology (Morse theory, Homotopy, Critical point analysis etc.) applicable
• Level sets and Fast marching methods applicable both to spatial indexing between contours but also to contour evolution between images in a sequence (in-betweening) - originally developed as a numerical technique
• .. but in the end images are vectors. Isn’t this where we came in? (metafiles, SVG)?