geometrical methods for adaptive approximation of image ... · drwa2010 alba di canazei, 6-9...
TRANSCRIPT
-
Department of Mathematics University of Hamburg
Geometrical Methods for Adaptive Approximation
of Image and Video Data
Laurent Demaret and Armin Iske
based on joint work with Nira Dyn and Wahid Khachabi
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 1
-
Introduction
1 Introduction
Digital Image Compression: Basic Steps
(1) Data reduction from input image;
(2) Encoding of the reduced data at the sender;
(3) Transmission of the encoded data from the sender to the receiver;
(4) Decoding of the transmitted data at the receiver;
(5) Data reconstruction.
Original Image.
0101100011010110010 . . .
−→Reconstruction.
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 2
-
Introduction
Image Representation.
• A digital image I is a rectangular grid of pixels, X.
• Each pixel x ∈ X bears a greyscale luminance I(x).
• We regard the image as a function, I : [X] → [0, 1, . . . , 2r − 1],
where the convex hull [X] of X is the image domain.
image domain [X].
I−→
image I(X).
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 3
-
Introduction
Image Approximation.INPUT: The image I = {(x, I(x)) : x ∈ X} is given by discrete pixel values in X.
OUTPUT: Reconstructed image Ĩ = {(x, Ĩ(x)) : x ∈ X}.
≈
AIM. Increase Peak Signal to Noise Ratio (PSNR)
PSNR = 10 ∗ log10
(
2r × 2r
η̄2(I, Ĩ)
)
,
as much as possible, where
η̄2(I, Ĩ) =1
|X|
∑
x∈X
|I(x) − Ĩ(x)|2
denotes the mean square error (MSE).
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 4
-
Methods for Image Compression
2 Methods for Image Compression
Wavelets: The standard (EBCOT, JPEG2000)
Wavelet Image Approximation Scheme.
• The image is expanded in a fixed orthonormal basis of wavelets.
• The expansion coefficients below a given threshold are set to zero.
A mildly nonlinear approximation scheme.
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 5
-
Geometric Methods for Image Compression
Some recent highly nonlinear approximation schemes ...
... for capturing the image geometry.
• Bandelets: LePennec & Mallat (2005);
• Brushlets: Coifman & Meyer (1997);
• Curvelets: Candès & Donoho (2000, 2004/2005);
• Contourlets: Do & Vetterli (2005);
• Directionlets: Velisavljević, Beferull-Lozano, Vetterli & Dragotti (2006);
• Shearlets: Guo, Kutyniok, Labate, Lim (2006);
• Wedgelets: Donoho (1999); Romberg, Wakin & Baraniuk (2002);
• The Easy Path Wavelet Transform (EPWT): Plonka(2009),
Plonka, Tenorth & I.(2010), Plonka, Tenorth & Roşca (2009);
• Nonlinear edge-adapted multiscale decomposition: Cohen & Matei (2001);
• Adaptive approximation by anisotropic triangulations:
– Generic triangulations and simulated annealing: Lehner, Umlauf, Hamann (2007)
– Adaptive thinning algorithms: Demaret, Dyn & I. (2006), Demaret & I. (2006)
– Anisotropic geodesic triangulations: Bougleux, Peyré & L. Cohen (2009)
– Greedy triangle bisections: A. Cohen, Dyn, Hecht & Mirebeau (2010)
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 6
-
Linear Splines over Triangulations
3 Linear Splines over Triangulations
Definition. A triangulation of a planar point set Y = {y1, . . . , yN} is a
collection T (Y) = {T }T∈T (Y) of triangles in the plane, such that
(T1) the vertex set of T (Y) is Y;
(T2) any pair of two distinct triangles in T (Y) intersect at most
at one common vertex or along one common edge;
(T3) the convex hull [Y] of Y coincides with the area covered
by the union of the triangles in T (Y).
A triangulation of pixels.
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 7
-
Linear Splines over Triangulations
Linear Splines over Triangulations.
Triangulation of pixels.
0
5
10
15
0
5
10
15
0
50
100
Linear spline over triangulation.
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 8
-
Linear Splines over Triangulations
Approximation Spaces.
• Given any triangulation T (Y) of Y, we denote by
SY ={s : s ∈ C([Y]) and s
∣
∣
Tlinear for all T ∈ T (Y)
},
the spline space containing all continuous functions over [Y] whose
restriction to any triangle in T (Y) is linear.
• Any element in SY is referred to as a linear spline over T (Y).
• For given function values {I(y) :y ∈ Y}, there is a unique linear spline,
L(Y, I) ∈ SY , which interpolates I at the points of Y, i.e.,
L(Y, I)(y) = I(y), for all y ∈ Y.
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 9
-
Examples
Example 1: Geometrical Image PQuad.
Image PQuad
of size (512 × 512).
Adaptive Triangulation
with 800 vertices.
Reconstruction
at PSNR 42.85 db.
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 10
-
Examples
Example 2: Geometrical Image Game.
Image Game
of size (512 × 512).
Adaptive Triangulation
with 6000 vertices.
Reconstruction
at PSNR 36.54 db.
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 11
-
Examples
Example 3: Multiscale Image Aerial.
Image Aerial
of size (512 × 512).
Adaptive Triangulation
with 16000 vertices.
Reconstruction
at PSNR 30.33 db.
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 12
-
Examples
Example 4: Multiscale Image Boat.
Image Boat
of size (512 × 512).
Adaptive Triangulation
with 7000 vertices.
Reconstruction
at PSNR 31.83 db.
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 13
-
Approximation over Anisotropic Triangulations
4 Approximation over Anisotropic Triangulations
Goal: On input image I = {(x, I(x)) : x ∈ X},
• determine a good adaptive spline space SY , where Y ⊂ X;
• determine from SY the unique best approximation L∗(Y, I) ∈ SY satisfying
∑
x∈X
|L∗(Y, I)(x) − I(x)|2 = mins∈SY
∑
x∈X
|s(x) − I(x)|2.
• Encode the linear spline L∗ ∈ SY ;
• Decode L∗ ∈ SY , and so obtain the reconstructed image
Ĩ = {(x, L(Y, Ĩ)(x)) : x ∈ X}, where L(Y, Ĩ) ≈ L∗(Y, I).
OBS! Key Step: Construction of anisotropic triangulation T (Y) for Y ⊂ X.
• One possible approach is by adaptive thinning (AT).
• In AT, we take the Delaunay triangulation D(Y) of Y for SY ,
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 14
-
Basic Technique for Proving Error Estimates
The Bramble-Hilbert Lemma.
Recall classical error estimates from finite element methods (FEM).
Bramble-Hilbert: For any image f from Sobolev space W2,2(T), T ∈ T (Y), we
obtain the basic error estimate
‖f − ΠSY f‖L2(T) ≤ |f|W2,2(T), for f ∈ W2,2(T),
where ΠSY f is the orthogonal L2-projection of f onto SY .
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 15
-
Delaunay Triangulations
5 Delaunay Triangulations
Definition. The Delaunay triangulation D(X) of a discrete planar point set
X is a triangulation of X, such that the circumcircle for each of its triangles does
not contain any point from X in its interior.
Two triangulations of a convex quadrilateral, T (left) and T̃ (right).
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 16
-
Delaunay Triangulations
Properties of Delaunay Triangulations.• Uniqueness.
Delaunay triangulation D(X) is unique, if no four points in X are co-circular.
• Complexity.
For any point set X, its Delaunay triangulation D(X) can be computed in
O(N log N) steps, where N = |X|.
• Local Updating.
For any X and x ∈ X, the Delaunay triangulation D(X \ x) of the point set
X \ x can be computed from D(X) by retriangulating the cell C(x) of x.
y
Removal of the node y, and retriangulation of its cell C(y).
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 17
-
Adaptive Thinning
6 Adaptive Thinning
Popular Example: Test Image Fruits.
Original Image (512 × 512).
Delaunay Triangulation.
4044 significant pixels.
Image Reconstruction.
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 18
-
Adaptive Thinning
Adaptive Thinning Algorithm.
INPUT. I = {0, 1, . . . , 2r − 1}X, pixels and luminances, where
X set of pixels, r number of bits for representation of luminances.
(1) Let XN = X;
(2) FOR k = 1, . . . , N − n
(2a) Find a least significant pixel x ∈ XN−k+1;
(2b) Let XN−k = XN−k+1 \ x;
• OUTPUT: Data hierarchy
Xn ⊂ Xn+1 ⊂ · · · ⊂ XN−1 ⊂ XN = X
of nested subsets of X.
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 19
-
Adaptive Thinning
Controlling the Mean Square Error.
• For a given mean square error (MSE), η̄∗, the adaptive thinning algorithm
can be changed in order to terminate when for the first time, the MSE value
corresponding to the current linear spline L(Xp, I) is above η̄∗, for some Xp
in the data hierarchy, n = p a posteriori.
• We take as the final approximation to the image the linear spline
L∗(Xp+1, I), and so we let Y = Xp+1.
• Observe that L∗(Xp+1, I) satisfies
∑
x∈X
|L∗(Xp+1, I)(x) − I(x)|2/
|Xp+1| ≤ η̄∗,
as desired.
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 20
-
Pixel Significance Measures
7 Pixel Significance Measures
Quality Measure: Current ℓ2-Square Error.
η(Y; X) =∑
x∈X
|L(I, Y)(x) − I(x)|2, for Y ⊂ X.
Anticipated Error for the Greedy Removal of one Pixel.
e(y) = η(Y \ y; X), for y ∈ Y.
Definition. (Adaptive Thinning Algorithm AT).
For Y ⊂ X, a point y∗ ∈ Y is said to be least significant in Y, iff it satisfies
e(y∗) = miny∈Y
e(y).
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 21
-
Pixel Significance Measures
Aim: Compute anticipated error locally.
η(Y \ y; X) = η(Y \ y; X \ C(y)) + η(Y \ y; X ∩ C(y))
= η(Y; X \ C(y)) + η(Y \ y; X ∩ C(y))
= η(Y; X) + η(Y \ y; X ∩ C(y)) − η(Y; X ∩ C(y)).
where C(y) is the cell of y in the Delaunay triangulation D(Y) of Y.
Therefore, minimizing e(y) is equivalent to minimizing
eδ(y) = η(Y \ y; X ∩ C(y)) − η(Y; X ∩ C(y)), for y ∈ Y.
Proposition. For Y ⊂ X, a point y∗ ∈ Y is, according to the criterion AT,
least significant in Y, iff it satisfies
eδ(y∗) = min
y∈Yeδ(y).
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 22
-
Pixel Significance Measures
Greedy Two-Point-Removal.
Anticipated Error for the Removal of two Points.
e(y1, y2) = η(Y \ {y1, y2}; X) for y1, y2 ∈ Y,
can be rewritten as e(y1, y2) = η(Y; X) + eδ(y1, y2), where
eδ(y1, y2) = η(Y \ {y1, y2}; X ∩ (C(y1) ∪ C(y2))) − η(Y; X ∩ (C(y1) ∪ C(y2))),
which can for [y1, y2] /∈ D(Y) be simplified as
eδ(y1, y2) = eδ(y1) + eδ(y2).
Definition. (Adaptive Thinning Algorithm AT2).
For Y ⊂ X, a point pair y∗1, y∗2 ∈ Y is said to be least significant in Y, iff
eδ(y∗1, y
∗2) = min
y1,y2∈Yeδ(y1, y2).
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 23
-
Implementation of Adaptive Thinning
8 Implementation of Adaptive Thinning.
Efficient Implementation of Algorithm AT.
Initialization.
• Compute Delaunay triangulation D(X);
• Compute eδ(x) for all x ∈ X and store nodes of D(X) in a Heap.
Removal Step. For current Y ⊂ X
• Pop root y∗ ∈ Y from Heap, update Heap;
• Remove y∗ from D(Y) and compute DY\y∗ ;
• Reattach historical points in C(y∗) ∩ (X \ Y);
• Attach y∗ to new triangle in C(y∗);
• Update eδ(y) for neighbours of y∗ and update Heap.
Total Complexity. O(N log(N)) operations.
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 24
-
Implementation of Adaptive Thinning
Efficient Implementation of Algorithm AT2.
• Due to the representation
eδ(y1, y2) = eδ(y1) + eδ(y2), for [y1, y2] /∈ D(Y),
the maintenance of significances {eδ(y1, y2) : {y1, y2} ⊂ Y} can be reduced
to maintenance of {eδ(y1, y2) : [y1, y2] ∈ D(Y)} and {eδ(y) : y ∈ Y}.
• For efficient implementation of AT2 we use two different priority queues,
– HeapY for significances eδ(y) of pixels y ∈ Y;
– HeapE for significances eδ(y1; y2) of edges [y1; y2] ∈ D(Y).
• Each priority queue, HeapY and HeapE, contains a least significant element
at its head, and is updated after each pixel removal.
• The resulting algorithm has also complexity O(N log N).
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 25
-
Implementation of Adaptive Thinning
Further Computational Details.
• We do not remove corner points from X, so that the image domain [X] is
invariant during the performance of adaptive thinning.
Uniqueness of Delaunay triangulation.
• Recall that the Delaunay triangulation D(Y) of Y ⊂ X, is unique, provided
that no four points in Y are co-circular.
• Since neither X nor its subsets satisfy this condition, we apply an efficient
method, termed simulation of simplicity (Edelsbrunner & Mücke, 1990),
which assures uniqueness (by using lexicographical order of vertices).
• Unlike in previous perturbation methods, the simulation of simplicity
method allows us to work with integer arithmetic rather than with floating
point arithmetic.
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 26
-
Local Optimization by Exchange
9 Local Optimization by Exchange
Definition: For any Y ⊂ X, let Z = X \ Y. A point pair (y, z) ∈ Y × Z satisfying
η((Y ∪ z) \ y; X) < η(Y; X)
is said to be exchangeable. A subset Y ⊂ X is said to be locally optimal
in X, iff there is no exchangeable point pair (y, z) ∈ Y × Z.
Algorithm (Exchange)INPUT: Y ⊂ X;
(1) Let Z = X \ Y;
(2) WHILE (Y not locally optimal in X)
(2a) Locate an exchangeable pair (y, z) ∈ Y × Z;
(2b) Let Y = (Y \ y) ∪ z and Z = (Z \ z) ∪ y;
OUTPUT: Y ⊂ X, locally optimal in X.
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 27
-
Local Optimization by Exchange
Characterization of Exchangeable Point Pairs.Let Z = X \ Y, for any Y ⊂ X, and recall
η(Y \ y; X) = η(Y; X) + eδ(y; Y), for y ∈ Y,
where eδ(y; Y) = η(Y \ y; X ∩ C(y; Y)) − η(Y; X ∩ C(y; Y)).
Letting first Y = Y ∪ z, and then y = z, this implies
η((Y ∪ z) \ y; X) = η(Y ∪ z; X) + eδ(y; Y ∪ z)
η(Y; X) = η(Y ∪ z; X) + eδ(z, Y ∪ z).
Therefore, (y, z) ∈ Y × Z are exchangeable, iff
eδ(z; Y ∪ z) > eδ(y; Y ∪ z),
which simplifies to
eδ(z; Y ∪ z) > eδ(y; Y),
in case C(y; Y) = C(y; Y ∪ z), i.e., [y; z] /∈ D(Y ∪ z).
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 28
-
Implementation of Exchange
Efficient Implementation of Exchange.
• Due to the swapping criterion
eδ(z; Y ∪ z) > eδ(y; Y), for [y; z] /∈ D(Y ∪ z),
the localization of exchangeable point pairs can efficiently be accomplished
by maintenance of three different priority queues,
– HeapY for significances eδ(y; Y) of pixels y ∈ Y;
– HeapZ for significances eδ(z; Y ∪ z) of pixels z ∈ Z;
– HeapE for significances σ(y, z) = eδ(z; Y ∪ z) − eδ(y; Y ∪ z) of edges
[y; z] ∈ D(Y ∪ z).
• The priority queue HeapY contains a least significant element at its head;
the head of HeapZ and HeapE contains a most significant element.
• Each of the three priority queues is updated after each pixel exchange.
• The resulting complexity for one pixel exchange is O(log N).
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 29
-
Image Compression
10 Image Compression
• Our compression method replaces the image I by its linear spline
approximation L∗(Y, I), where Y ⊂ X are the significant pixels.
• In order to code L∗(Y, I), we code the information
{(y, I∗(y)) :y ∈ Y}.
Quantization.
• Apply uniform quantization to the optimal luminances I∗(y) = L∗(Y, I)(y),
• so obtain quantized symbols {Q(I∗(y)) :y ∈ Y},
• corresponding to quantized luminances {̃I(y)) :y ∈ Y}.
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 30
-
Image Compression
11 Theoretical Coding Costs
OBSERVE! Due to the uniqueness of the Delaunay triangulation,
no connectivity coding is required!
• We are only concerned with coding the elements of the set
{(y, Q(I∗(y))) :y ∈ Y} ∈ Isn,
where, with n = |Y|,
Isn ={{0, 1, . . . , 2s − 1}Z : Z ⊂ X and |Z| = n
}.
• The number of elements in Isn is(
|X|n
)
× 2s×n.
• If we assume that every element of Isn has the same probability of
occurrence, then the theoretical coding cost is
log2
((
|X|
n
))
+ s × n.
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 31
-
Scattered Data Coding
12 Scattered Data Coding
OBSERVE! We can reduce the theoretical coding costs by taking advantage
of the geometric structure of the image as follows.
The elements of {(i, j, Q(I∗(i, j))) : (i, j) ∈ Y} are coded by decomposing their
bounding cell
Ω = [0..2p − 1] × [0..2q − 1] × [0..2s − 1]
recursively, where [0..2s − 1] is the range for the quantized symbols.
Splitting of the cell Ω into eight subcells in three stages.
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 32
-
Scattered Data Coding
(1) Coding of Scattered Pixels.
• Coding of pixels in Y relies on a recursive splitting of the pixel domain
Ω = [X].
• For the sake of simplicity, let us assume that Ω is a square domain of the
form Ω = [0, 2q − 1] × [0, 2q − 1].
• In the splitting, a square subdomain ω ⊂ Ω (initially ω = Ω) is split
horizontally into two rectangular subdomains of equal size. A rectangular
subdomain is split vertically into two square subdomains of equal size.
• The splitting terminates at subdomains which are either empty, i.e., not
containing any pixel from Y, or atomic, i.e., of size 1 × 1.
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 33
-
Scattered Data Coding
(1) Coding of Scattered Pixels.
• This recursive splitting can be represented by a binary tree, whose nodes
correspond to the subdomains. The root of the tree corresponds to Ω, and
its leaves correspond to empty or atomic subdomains.
• In each node of the tree, with a corresponding subdomain ω, we store the
number |ω| of pixels from Y contained in ω, i.e., |ω| = |Y ∩ ω|.
• Note that for a parent node ω, and its two children nodes, ω1 and ω2, we
have the relation |ω| = |ω1| + |ω2|. This relation allows a non-redundant
representation of the binary tree.
• The bitstream, representing the tree, is constructed by a Huffman code.
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 34
-
Scattered Data Coding
(2) Coding of Quantized Symbols.
• To code the quantized symbols in QY , we first split the image domain Ω
into a small number of square subdomains of equal size.
• For each subdomain, the pixels from Y contained in it are ordered linearly,
such that close pixels in the image domain are close in this ordering.
• The quantized symbol of any pixel in this ordering is coded relative to the
quantized symbol of its predecessor, except for that of the first pixel.
• The coding is done by using a Huffman code.
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 35
-
Image Reconstruction
13 Image Reconstruction at the Decoder
Reconstruction of the compressed image from information
{(y, Q(I∗(y))) :y ∈ Y}
in four steps:
(1) Compute Delaunay triangulation D(Y) of Y;
(2) Construct unique linear spline L(Y, Ĩ) ∈ SY satisfying
L(Y, Ĩ)(y) = Ĩ(y), for all y ∈ Y,
from quantized luminance values {̃I(y) :y ∈ Y};
(3) Obtain reconstructed image by
Ĩ = {(x, L(Y, Ĩ)(x)) : x ∈ X}.
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 36
-
First Comparisons with JPEG2000
14 First Comparisons with JPEG2000
Preliminary Remarks.
• We compare the performance of our compression method AT2 with that of
EBCOT, which is the basic algorithm in JPEG2000.
• In each comparison, the compression rate, in bits per pixel (bpp), is fixed.
• The quality of the resulting reconstructions is measured
by their PSNR values, and by their visual quality.
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 37
-
First Comparisons with JPEG2000
Geometric Test Image Chessboard. AT versus AT2.
Original Image.
JPEG2000
AT
AT2
Delaunay triangulation.
Delaunay triangulation.
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 38
-
First Comparisons with JPEG2000
Geometric Test Image Chessboard.
Original Image
Chessboard
of size (128 × 128).
Reconstruction by
JPEG2000 at 0.23 bpp
PSNR 18.68 db.
Reconstruction by
AT2 at 0.23 bpp
PSNR 45.15 db.
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 39
-
First Comparisons with JPEG2000
Geometric Real Image Reflex.
Original Image
Reflex
of size (128 × 128).
Reconstruction by
JPEG2000 at 0.251 bpp
PSNR 28.74 db.
Reconstruction by
AT2 at 0.251 bpp
PSNR 42.86 db.
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 40
-
More Recent Comparisons with JPEG2000
15 More Recent Comparisons with JPEG2000
Current Version (AT2009):• L. Demaret, A. Iske, W. Khachabi (2009)
Contextual image compression from adaptive sparse data representations.
In: Signal Processing with Adaptive Sparse Structured Representations.
Workshop Proceedings, Saint-Malo (France), 6.-9. April 2009.
Previous Version (AT2006):• L. Demaret, A. Iske (2006)
Adaptive image approximation by linear splines over locally optimal Delaunay triangulations.
IEEE Signal Processing Letters 13(5), 281-284.
• L. Demaret, N. Dyn, A. Iske (2006)
Image compression by linear splines over adaptive triangulations.
Signal Processing 86(7), July 2006, 1604–1616.
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 41
-
Comparison between JPEG2000 and AT2009
Comparison between JPEG2000 and AT2009.
Original Image
Cameraman
of size (256 × 256).
Reconstruction by
JPEG2000 at 3.247 kB
PSNR 29.84 db.
Reconstruction by
AT2009 at 3.233 kB
PSNR 30.66 db.
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 42
-
Comparison between JPEG2000 and AT2009
Rate-Distortion Curves for JPEG2000 and AT.
0.2 0.25 0.3 0.35 0.4 0.45 0.5
32.5
33
33.5
34
34.5
35
35.5
36
36.5
37
bpp
PS
NR
(d
B)
AT2009
AT2006
JPEG2000
Fruits
0.22 0.24 0.26 0.28 0.3 0.32 0.34 0.36 0.38 0.4 0.42
28
28.5
29
29.5
30
30.5
bpp
PS
NR
(d
B)
AT2009
AT2006
JPEG2000
Cameraman
0.15 0.2 0.25 0.3 0.35 0.4 0.45
32.5
33
33.5
34
34.5
35
35.5
36
36.5
37
37.5
bpp
PS
NR
(d
B) AT2009
AT2006
JPEG2000
Lena
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 43
-
Asymptotic Behaviour of N-term Approximations
Asymptotic Behaviour of N-term Approximations.Theorem (Birman & Solomjak 1967): Let α ∈ (0, 2] and p ≥ 1 satisfy
α > 2/p − 1. Then, for any f ∈ Wα,p([0, 1]2) we have
EN(f) = O(N−α) for N → ∞
where
EN(f) = inf{‖f − f̂(QN)‖
2L2([0,1]2) :QN ∈ Q with |QN| = N
}.
Corollary (Demaret & I. 2010): Let α ∈ (0, 2] and p ≥ 1 satisfy
α > 2/p − 1. Then, for any f ∈ Wα,p([0, 1]2) we have
EN(f) = O(N−α) for N → ∞
where
EN(f) = inf{‖f − f̂(DN)‖
2L2([0,1]2) :DN ∈ D with |DN| = N
}.
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 44
-
Video Compression: Test Case Suzie
Video Compression: Test Case Suzie.
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 45
-
Video Compression
Video Compression: Preliminary Remarks.
• Natural videos are composed of a superposition of moving objects ...
• ... usually resulting from anisotropic motions;
• a video may be regarded as a sequence of consecutive natural still images ...
• ... or — a video may be regarded as a 3d scalar field;
• it is desirable to work with sparse representations of video data;
• ...
• Adaptive Thinning (AT) extracts significant video pixels ...
• ... to obtain a sparse representation of the video ...
• ... relying on linear splines over anisotropic tetrahedralizations.
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 46
-
Representation of Video Data
Representation of Video Data.
• A digital video V is a rectangular 3d grid of pixels, X.
• Each pixel x ∈ X bears a greyscale luminance V(x).
• We regard the video as a trivariate function,
V : [X] → {0, 1, . . . , 2r − 1}
where the convex hull [X] of X is the video domain.
INPUT: The video is given by its restriction to the pixels in X,
V∣
∣
X= {(x, V(x)) : x ∈ X}.
GOAL: Approximation of V from discrete data V∣
∣
X.
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 47
-
Linear Splines over Tetrahedralizations
Linear Splines over Tetrahedralizations.
• Given any tetrahedralizations T (Y) of Y, we denote by
SY ={s : s ∈ C([Y]) and s
∣
∣
Tlinear for all T ∈ T (Y)
},
the spline space containing all continuous functions over [Y] whose
restriction to any tetrahedron in T (Y) is linear.
• Any element in SY is referred to as a linear spline over T (Y).
• For given function values {V(y) :y ∈ Y}, there is a unique linear spline,
L(Y, V) ∈ SY , which interpolates V at the points of Y, i.e.,
L(Y, V)(y) = V(y), for all y ∈ Y.
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 48
-
Delaunay Tetrahedralizations
Basic Features of Delaunay Tetrahedralizations.• Uniqueness.
Delaunay tetrahedralization D(X) is unique,
if no five points in X are co-spherical.
• Complexity.
For any point set X, its Delaunay tetrahedralization D(X)
can be computed in O(N log N) steps, where N = |X|.
• Local Updating.
For any X and x ∈ X, the Delaunay tetrahedralization D(X \ x) of the point set
X \ x can be computed from D(X) by re-tetrahedralization of the cell C(x) of x.
Removal of the node x and re-tetrahedralization of its cell C(x).
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 49
-
Numerical Simulation
Numerical Simulation for Test Case Suzie.
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 50
-
Numerical Simulation
Test Case Suzie: Frame 0000.
Original Frame Suzie.
Delaunay tetrahedralization.
708 significant pixels.
Reconstruction by AT at 34.58 dB.
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 51
-
Numerical Simulation
Test Case Suzie: Frame 0001.
Original Frame Suzie.
Delaunay tetrahedralization.
118 significant pixels.
Reconstruction by AT at 35.15 dB.
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 52
-
Numerical Simulation
Test Case Suzie: Frame 0002.
Original Frame Suzie.
Delaunay tetrahedralization.
287 significant pixels.
Reconstruction by AT at 35.18 dB.
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 53
-
Numerical Simulation
Test Case Suzie: Frame 0003.
Original Frame Suzie.
Delaunay tetrahedralization.
338 significant pixels.
Reconstruction by AT at 34.91 dB.
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 54
-
Numerical Simulation
Test Case Suzie: Frame 0004.
Original Frame Suzie.
Delaunay tetrahedralization.
398 significant pixels.
Reconstruction by AT at 34.98 dB.
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 55
-
Numerical Simulation
Test Case Suzie: Frame 0005.
Original Frame Suzie.
Delaunay tetrahedralization.
448 significant pixels.
Reconstruction by AT at 34.99 dB.
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 56
-
Numerical Simulation
Test Case Suzie: Frame 0006.
Original Frame Suzie.
Delaunay tetrahedralization.
424 significant pixels.
Reconstruction by AT at 34.96 dB.
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 57
-
Numerical Simulation
Test Case Suzie: Frame 0007.
Original Frame Suzie.
Delaunay tetrahedralization.
460 significant pixels.
Reconstruction by AT at 34.92 dB.
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 58
-
Numerical Simulation
Test Case Suzie: Frame 0008.
Original Frame Suzie.
Delaunay tetrahedralization.
534 significant pixels.
Reconstruction by AT at 35.11 dB.
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 59
-
Numerical Simulation
Test Case Suzie: Frame 0009.
Original Frame Suzie.
Delaunay tetrahedralization.
523 significant pixels.
Reconstruction by AT at 34.82 dB.
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 60
-
Numerical Simulation
Test Case Suzie: Frame 0010.
Original Frame Suzie.
Delaunay tetrahedralization.
539 significant pixels.
Reconstruction by AT at 34.89 dB.
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 61
-
Numerical Simulation
Test Case Suzie: Frame 0011.
Original Frame Suzie.
Delaunay tetrahedralization.
534 significant pixels.
Reconstruction by AT at 34.95 dB.
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 62
-
Numerical Simulation
Test Case Suzie: Frame 0012.
Original Frame Suzie.
Delaunay tetrahedralization.
513 significant pixels.
Reconstruction by AT at 35.34 dB.
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 63
-
Numerical Simulation
Test Case Suzie: Frame 0013.
Original Frame Suzie.
Delaunay tetrahedralization.
432 significant pixels.
Reconstruction by AT at 35.30 dB.
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 64
-
Numerical Simulation
Test Case Suzie: Frame 0014.
Original Frame Suzie.
Delaunay tetrahedralization.
364 significant pixels.
Reconstruction by AT at 35.49 dB.
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 65
-
Numerical Simulation
Test Case Suzie: Frame 0015.
Original Frame Suzie.
Delaunay tetrahedralization.
311 significant pixels.
Reconstruction by AT at 35.68 dB.
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 66
-
Numerical Simulation
Test Case Suzie: Frame 0016.
Original Frame Suzie.
Delaunay tetrahedralization.
285 significant pixels.
Reconstruction by AT at 35.82 dB.
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 67
-
Numerical Simulation
Test Case Suzie: Frame 0017.
Original Frame Suzie.
Delaunay tetrahedralization.
293 significant pixels.
Reconstruction by AT at 36.32 dB.
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 68
-
Numerical Simulation
Test Case Suzie: Frame 0018.
Original Frame Suzie.
Delaunay tetrahedralization.
289 significant pixels.
Reconstruction by AT at 36.08 dB.
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 69
-
Numerical Simulation
Test Case Suzie: Frame 0019.
Original Frame Suzie.
Delaunay tetrahedralization.
307 significant pixels.
Reconstruction by AT at 36.25 dB.
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 70
-
Numerical Simulation
Test Case Suzie: Frame 0020.
Original Frame Suzie.
Delaunay tetrahedralization.
292 significant pixels.
Reconstruction by AT at 36.26 dB.
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 71
-
Numerical Simulation
Test Case Suzie: Frame 0021.
Original Frame Suzie.
Delaunay tetrahedralization.
293 significant pixels.
Reconstruction by AT at 36.02 dB.
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 72
-
Numerical Simulation
Test Case Suzie: Frame 0022.
Original Frame Suzie.
Delaunay tetrahedralization.
326 significant pixels.
Reconstruction by AT at 36.06 dB.
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 73
-
Numerical Simulation
Test Case Suzie: Frame 0023.
Original Frame Suzie.
Delaunay tetrahedralization.
341 significant pixels.
Reconstruction by AT at 36.08 dB.
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 74
-
Numerical Simulation
Test Case Suzie: Frame 0024.
Original Frame Suzie.
Delaunay tetrahedralization.
311 significant pixels.
Reconstruction by AT at 36.24 dB.
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 75
-
Numerical Simulation
Test Case Suzie: Frame 0025.
Original Frame Suzie.
Delaunay tetrahedralization.
321 significant pixels.
Reconstruction by AT at 36.16 dB.
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 76
-
Numerical Simulation
Test Case Suzie: Frame 0026.
Original Frame Suzie.
Delaunay tetrahedralization.
320 significant pixels.
Reconstruction by AT at 35.95 dB.
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 77
-
Numerical Simulation
Test Case Suzie: Frame 0027.
Original Frame Suzie.
Delaunay tetrahedralization.
273 significant pixels.
Reconstruction by AT at 35.60 dB.
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 78
-
Numerical Simulation
Test Case Suzie: Frame 0028.
Original Frame Suzie.
Delaunay tetrahedralization.
179 significant pixels.
Reconstruction by AT at 35.48 dB.
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 79
-
Numerical Simulation
Test Case Suzie: Frame 0029.
Original Frame Suzie.
Delaunay tetrahedralization.
669 significant pixels.
Reconstruction by AT at 35.00 dB.
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 80
-
Numerical Simulation
Performance Check: Data Size and Approximation.Number of significant pixels:
Total: 11,430; minimal: 118; maximal: 708; average: 381 pixels.
PSNR value:
Overall: 35.45 dB; minimal: 34.58 dB; maximal: 36.32 dB; average: 35.49 dB.
Number of significant pixels. PSNR values.
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 81
-
Literature
Relevant Literature.• L. Demaret and A. Iske (2010) Anisotropic triangulation methods in image
approximation. In: Approximation Algorithms for Complex Systems,
E.H. Georgoulis, A. Iske, and J. Levesley (eds.), Springer, Berlin, 47–68.
• L. Demaret, A. Iske, and W. Khachabi (2010) Sparse representation of video data by
adaptive tetrahedralisations. In: Locally Adaptive Filters in Signal and Image Processing,
L. Florack, R. Duits, G. Jongbloed, M.-C. van Lieshout, L. Davies (eds.), 197–220.
• L. Demaret, A. Iske, and W. Khachabi (2009) Contextual image compression from
adaptive sparse data representatons. Workshop Proceedings Signal Processing with
Adaptive Sparse Structured Representations, 6.-9. April 2009 - Saint-Malo (France).
• L. Demaret and A. Iske (2006) Adaptive image approximation by linear splines over
locally optimal Delaunay triangulations. IEEE Signal Processing Letters 13(5), 281–284.
• L. Demaret, N. Dyn, and A. Iske (2006) Image compression by linear splines over
adaptive triangulations. Signal Processing 86(7), July 2006, 281–284.
• L. Demaret, N. Dyn, M.S. Floater, and A. Iske (2005) Adaptive thinning for terrain
modelling and image compression. Advances in Multiresolution for Geometric Modelling,
N.A. Dodgson, M.S. Floater, and M.A. Sabin (eds.), Springer, 321–340.
DRWA2010 Alba di Canazei, 6-9 September 2010 Armin Iske 82