traditional image processing

www.ngi-central.org 129.04.2010

www.ngi-central.org 2

Traditional Image Processing

• Data Structures– Images, Palettes, Histograms, Profiles, etc.– For many datatypes (8bit, 16 bit, float, etc.)

• Algorithms– Dependent on Data Structures– Dependent on datatypes

• Combinatorial Explosion -> unmanageable– To implement one new algorithm, it must be

written many times – for each type

29.04.2010


Generic Image Processing• Generic Programming comes to the rescue

• Three key design techniques– Iterators decouple data structures and

algorithms– Functors allow to exchange parts of

computation– Generic algorithms in terms of

abstract iterators and functors

• Enabling technology of compilers is the template<> mechanism

29.04.2010

Algorithms

Iterators

Data


Standard Template Library

• STL is an implementation of generic programming concepts

• STL is available with all compilers since it is part of the C++ standard

• STL is around 10 years old and considered mature

• Educational material to learn generic programming in general and STL in particular is available

29.04.2010


Example: RGB to Gray

• Using STL style programming

struct RGBValue {unsigned char red, green, blue;

};

struct RGBToGray {unsigned char operator()(const RGBValue& rgb) const {

return 0.3*rgb.red + 0.59*rgb.green + 0.11*rgb.blue;}

};

vector<RGBValue> rgb;vector<unsigned char> gray;...transform(rgb.begin(), rgb.end(), gray.begin(), RGBToGray());

29.04.2010


New: works for all types!

Better example• Templated on the datatype<template class T>struct RGBValue {

T red, green, blue;};

<template class T>struct RGBToGray {

T operator()(const RGBValue<T>& rgb) const {return 0.3*rgb.red + 0.59*rgb.green + 0.11*rgb.blue;

}};

vector<RGBValue<float> > rgb;vector<float> gray;...transform(rgb.begin(), rgb.end(), gray.begin(), RGBToGray<float>());

29.04.2010


STL leaves things to desire

• STL is designed to work with 1D data• Images are inherently two(multi)dimensional• Some algorithms do not need the

dimensionality– Use STL in this case

• Some algorithms need the dimensionality– Cannot use STL, need something different

29.04.2010


Multidimensional Locators

• A locator is a multidimensional iterator

• Logical extension to an iterator

Move the iterator++it;--it;it+=100;it-=50;

Move the locator++it.x;--it.y;it.x+=100;it.y-=50;

29.04.2010


Algorithms change as well

// STL implementationtemplate<class IT, class F>IT transform(IT First, IT Last, IT Dest, F Func){

for (; First != Last; ++First, ++Dest)*Dest = Func(*First);

return (Dest);}

// 2D implementationtemplate<class IT, class F>IT transform(IT First, IT Last, IT Dest, F Func){

for (; First.y != Last.y; ++First.y, ++Dest.y)for (; First.x != Last.x; ++First.x, ++Dest.x)

*Dest = Func(*First);return (Dest);

}

29.04.2010


What do we gain?• An algorithm like transform replaces many

functions in traditional programming style

• It can do this, because it is templated on the function– We still need to program the functionality,

but decoupled from navigation– It saves us to rewrite the loops many times– Bonus: the function objects can be reused in

completely different algorithms as well

• It can do this, because it is templated on the datatype (by iterator indirection)– It saves us to rewrite the complete thing for each datatype

• The gain is tremendous (time, functionality, flexibility)

29.04.2010


NGI - Goals• Image Processing

– Image -> Image

• Image Analysis– Image -> Numbers

• Generic Library– C++– Templates– Source code– Independence of type

• High Performance– no penalty for generic code

• Portability– clean source code for easy portability

29.04.2010


Directory Structure• Code (include/)

• Third party code (toolkits/)

• Documentation (book/, reference/, presentations/)

• Sample code (samples/) and sample images (images/)

• Automated tests (tests/)29.04.2010


NGI - Tests

• Unit Tests– Features of a class/function

are tested in isolation

• Regression Tests– Outcome of a function is tested for regressions– Image/Text comparisons (now == previous)

• Benchmarks– Measured in clocks per pixel

29.04.2010


NGI - Documentation

• User Documentation– NGI Book

• Reference Documentation– Built with Doxygen from source code (HTML)– Always up-to-date

29.04.2010


Third Party Code

• Mandatory• boost (www.boost.org)

• Optional• FreeImage (freeimage.sourceforge.net)• Cairo (www.cairographics.org)

29.04.2010

http://www.boost.org/

http://freeimage.sourceforge.net/

http://cairographics.org/


NGI – Build System• CMake is used as build

engine

• modular– select compiler

(VS2005, VS2008, VS2010, Intel)– run tests– build documentation

• Currently setting up a continuous build machine with various virtual build environments

29.04.2010


Sample Code

• Many code samples • Console samples• MFC samples• .NET samples

• Tests are also samples• Using Boost.Test• Unit tests, benchmarks,

regression tests29.04.2010


NGI – Guidelines

• Header only library• Source code• Heavy use of templates• Concentration on core issues• using other Open Source libraries where

applicable– Boost, FreeImage, Cairo, Qt

29.04.2010


NGI – Concepts• Buffer

– stores data– handles allocation and deallocation– can use external buffers

• View– level of indirection– light, just a pointer and a few pitches– all accesses through views– can handle foreign data buffers– view manipulations

• sub-views (moving_view.exe)• rotated views (dimension_swap_view.exe)• reverse scanning directions (reverse_view.exe)• subsampling (subsampling_view.exe)

29.04.2010


NGI – Functionality Overview (1)

• Basic functions used everywhere else– Interpolation

• linear, spline– Geometry

• point, line_segment, ray, line, triangle, quadrilateral, rectangle, polygon, circle, ellipse

• widgets for all geometric elements– Other examples

• Base64, ZIP/GZIP Compression/Decompression, Fixed point, Matrix solver, solving quadratic and cubic equations, etc.

29.04.2010


NGI – Functionality Overview (2)

• STL like functions– copy, count, equal, find, ...

• Image Processing functions– Point, Filters, Morphology, ...

• Image Analysis functions– Blob, Pattern match, OCR/OCV, ...

• Display Engine– Images, Palettes, Curves, Widgets, Geometry, ...

• Framegrabber/Camera interface

29.04.2010


NGI – STL Like Functions• accumulate• copy• count• equal• fill• find• generate• inner_product• max_element• min_element• replace• transform

29.04.2010

1D2D 3D


NGI – Image Processing Functions (1)

• Point functions (point_operations.exe, profile.exe)• Filter and morphological functions (

neighborhood_operations.exe)– 1D, 2D, 3D– Separable– Fast box filters – independent of kernel size– Many predefined kernels, user can easily add– Framing (framing.exe)– Erode, Dilate, Open, Close, Gradient– Gray and binary behavior

29.04.2010



• Binning (binning.exe)– binning method selectable/user writable

• Geometric Transformation (resample.exe)– affine / polar– choice of resampling filter (nearest neighbor, box,

triangle, quadratic, cubic, bspline, sinc, kaiser, lanczos)

• Autofocus– based on variance, gradient

29.04.2010



• Color Processing– Color models (color_transform.exe)

• RGB, CMY(K), HLS, HSI, Lab, Lchab, Luv, Lchuv

– Color twist (color_twist.exe)– Bayer demosaic

• Statistic Functions– min, max, average,

variance, percentiles• Camera Calibration

29.04.2010


NGI – Image Analysis Functions (1)

• Blob analysis– Thresholding via Otsu‘s method– 2D, 3D Labelling– Fläche/Volumen– Chain Code (inner, outer),

perimeter– Moments• binary, gray• ordinary, central, Hu, Flusser

29.04.2010


NGI – Image Analysis Functions (2)

• Pattern matching and searching (search.exe)– gray or color– using pyramid for acceleration

• OCR/OCV– train a character set– read/verify text

29.04.2010


NGI - Acceleration• Written for Multi-Core

– Uses OpenMP– Most functions scale

nicely, can be seen with the benchmarks

• Working on SIMD acceleration– Vector Processing (128 byte chunks)– 16 Pixels at a time– First for point functions

• These two methods are orthogonal

29.04.2010


NGI – Display Engine (1)

• Display Engine– Direct2D, GDI Plus, OpenGL, Cairo– Alpha channel support (blit.exe)– Images (file_access.exe) – Palettes (palette.exe, false_color.exe)– Curves (histogram.exe)

29.04.2010


NGI – Display Engine (2)

– Widgets (scale.exe)

– Hierarchical widget composition(widget_box.exe)

– Callbacks for smart interaction (horizontal_cursor.exe)

29.04.2010


NGI – Framegrabber/Camera

• Grabbing into ringbuffer• asynchronous on different

thread• live/snapshot• pre-trigger/post-trigger• Gen<i>cam support planned

29.04.2010

01

2

3

4

567

8

9

10

11

Acquisition

Display


More Information

29.04.2010

www.ngi-central.org

[email protected]

http://www.ngi-central.org/

http://ngi-central.blogspot.com/

traditional image processing

Documents