Extended Introduction to Computer Science CS1001.py

Lecture 22.5: additional topics in image processing

Instructors: Benny Chor, Amir Rubinstein Teaching Assistants: Michal Kleinbort, Yael Baran

School of Computer Science Tel-Aviv University

Spring Semester, 2013-14 http://tau-cs1001-py.wikidot.com

Based on material prepared by Asaf Zaritsky

Lecture 22: Plan • Resolution and quantization (reminder)

• Noise reduction (using lecture 21 slides)

• Edge Detection

• Segmentation

2

3

Pixel Resolution

Image from wikipedia

Demo for pixel resolution effects: http://micro.magnet.fsu.edu/primer/java/digitalimaging/processing/spatialresolution/

• Pixel resolution == pixel count

Image Quantization

24 bit RGB 16 colors

• Number of bits per pixel

• Note that both images have the same pixel & spatial resolution

Image from: http://micro.magnet.fsu.edu/primer/digitalimaging/digitalimagebasics.html

Demo for quantization effects: http://micro.magnet.fsu.edu/primer/java/digitalimaging/processing/bitdepth/index.html

Image Quantization

• We will use the slides and code from the previous lecture

• Warm up exercise: • Recall the function join_h from last time, which joins 2

images horizontally (used for easy display and comparisons).

• Write a function join(), which appends a variable number of images (vertically or horizontally)

Noise reduction

Edge Detection

7

• Edge - sharp change in intensity between close pixels • Usually captures much of the meaningful information in the image

images extracted using Sobel filter from: http://micro.magnet.fsu.edu/primer/java/digitalimaging/russ/sobelfilter/index.html

Erosion and Dilation

8

• Erosion - the removal of pixels from the periphery of a (white) feature. • Dilation - the adding of pixels to that periphery.

• If we assume bright foreground and dark background: • Erosion shrinks foreground areas, and holes grow. • Dilation enlarges foreground areas, and holes shrink.

Erosion Dilation

Images from: http://micro.magnet.fsu.edu/primer/java/digitalimaging/russ/erosiondilation/index.html

Erosion and Dilation - examples

Erosion

Dilation

Dilation for Edge Detection

Original Dilation Edges

Diff

Erosion and Dilation - Code

def erosion(A, k=1): return local_operator(A, ??, k) def dilation(A, k=1): return local_operator(A, ??, k)

def local_operator(A, op, k=1): n,m = A.dim() res = copy(A) # brand new copy of A for i in range(k,n-k): for j in range(k,m-k): res[i,j] = op(items(A[i-k:i+k+1,j-k:j+k+1])) return res

diff - Code def diff(im1, im2): ''' creates the diff im1-im2 ''' assert im1.dim() == im2.dim() n,m = im1.dim() out = Matrix(n,m) for i in range(n): for j in range(m): out[i,j] = (im1[i,j] - im2[i,j]) return out

PIL's edge detection vs. our own

13

Original Diff PIL's filter

from PIL import ImageFilter im = Image.open("./coins.jpg") new = im.filter(ImageFilter.FIND_EDGES) new.show()

Segmentation

14

From Wikipedia: • In computer vision, image segmentation is the process of partitioning a digital

image into multiple segments (sets of pixels, also known as superpixels). The goal of segmentation is to simplify and/or change the representation of an

image into something that is more meaningful and easier to analyze. Image segmentation is typically used to locate objects and boundaries (lines,

curves, etc.) in images.

• Applications in medical imaging: - Locate tumors and other pathologies - Measure tissue volumes - Diagnosis, study of anatomical structure

Image from: http://www.sonycsl.co.jp/person/nielsen/applets.html

Segmentation by Thresholding

15

• Simplest segmentation method

• Apply a threshold to turn a gray-scale image into a binary image (BW) –

this is called Binary Segmentation

• Can apply more than one threshold, creating >2 segments

• The key is to select the appropriate threshold values

Human HT29 colon-cancer cells http://www.broadinstitute.org/bbbc/image_sets.html Binary segmentation,

threshold = 20

Binary Segmentation

Threshold = 20 Threshold = 40 Threshold = 60

• Which threshold is the best?

Original

Thresholding Demo

17

Demo for threshold effects on segmentation: http://micro.magnet.fsu.edu/primer/java/digitalimaging/processing/automaticthresholding/index.html

Variance

• Variance (σ2) of a set is a measure of how much the set is spread out. Defined as the mean distance2 from the set’s center.

A good threshold for segmentation: • Minimizes intra-segment variance • Maximizes inter-segment variance

Otsu method for threshold calculation • Assumes that the image contains two classes of pixels - foreground and

background.

• Finds an optimal threshold for segmentation: – A threshold that maximizes the variance between these classes. – Uses image histogram: grey level distribution in an image.

• x-axis – grey hues • y-axis – number of pixels with a particular hue

Gray-scale histogram

Otsu's formula For every threshold t denote:

Background: <= t (low gray levels) Foreground :> t (high gray levels) w_back – number of Background pixels w_fore – number of Foreground pixels mean_back – mean value of the Background pixels mean_fore – mean value of the Foreground pixels

The Otsu threshold is the threshold that maximizes the var_between over all possible thresholds t.

var_between(t) = w_back * w_fore * (mean_back - mean_fore)2

Otsu's Threshold = 38

Original

Otsu threshold - Run