ee565 advanced image processing copyright xin li 20091 ee565: advanced image processing xin li...

EE565 Advanced Image Processing Copyright Xin Li 2009 1

EE565: Advanced Image ProcessingXin LiLDCSEE, Fall 2009

dehaze


EE565 General Information

• Lectures and office hours

Meeting Time: TTh 9:30-10:45 in MRB 107 Office Hours: Mondays 2:00-3:00pm in ESB 939

Fragment Minutes: 15 minutes before and after each lecture

• Contact information

Instructor: [email protected]

For email submission of assignments, please use your MIX account


• Texts

• Prerequisites

EE465: Introduction to Digital Image Processing or equivalent

No textbook is required. The instructor will provide lecture notesat the course website

http://www.csee.wvu.edu/~xinl/courses/ee565/ee565.html

Additional material (e.g., classical papers, MATLAB demos,assignments and solutions) will also be posted there

• Follow-up (Spring 2010)

EE569: Digital Video Processing (more fun)


• Working load

- 8 assignments- One midterm and one final project

• Grading

Midterm project 30%(Technical report 5% included)

Assignments 40%

Auditing policy: you need to turn in all the assignments

Final project 30% (Oral Presentation 5% included)


Importance of Hand-on Experience Finishing all the assignments are necessary

preparation for working on larger-size projects

Midterm project will be development oriented (e.g., implementation of a published algorithm or some simple idea

of your own) Final project will be research oriented (e.g.,

improve upon a published algorithm or the idea you have tested in midterm)

Final project could lead to MS thesis or PhD qualifier exam problem


How to Do Scientific Research?

Inquiry (research)-based learning Difference between taking exams and doing

research Difference between textbook knowledge and

your own understanding Competition and collaboration

Your classmates are your competitors (grading will be ranking-based)

Your classmates are also your collaborators (cooperation is at the foundation of all engineering endeavors)


Tools for Effective Learning

http://masterxinli.wordpress.com/category/teaching/ee565/

In addition to classroom interaction, Blog offers a convenient platform for everyone to participate.

I might also try several other techniques: Think-Pair-Share, Minute Paper and Group Discussion

The most important lesson I have learned through years: Interest is the best instructor (everything I do is to try toget you hooked to learning this course – so pls. tell me whenyou feel bored)


Course Overview

Mathematical modeling of images Why do we care about images? Why do we take a mathematical approach?

Image restoration Improve image quality and usability

Image communication Move images from here to there and from now

to then Image analysis

Automatically extract information from images


Technological Importance of Images

Improve Human’s vision capabilities see far (e.g., watch Summer Olympics

in Beijing) see small (e.g., microscopic structures

such as neurons and cells) see through (e.g., ultrasound inspection

of pregnant women) see better (e.g., in the darkness or

adversary environmental conditions).


Scientific Reasons

Understanding how we see is the first step towards understanding human intelligence


D. Hubel’s “Eyes, Brain and Vision”

http://hubel.med.harvard.edu/bcontex.htm


Neural Network View

“The Next Generation of Neural Networks”http://www.youtube.com/watch?v=AyzOUbkUf3M


Why Mathematical Modeling?

What is mathematical modeling? A mathematical model uses

mathematical language to describe a system

Linear vs. nonlinear Deterministic vs. probabilistic Static vs. dynamic Homogeneous vs. heterogeneous

Philosophical considerations Causality vs. Synchronicity


Images of Favorite: Natural Images

What is natural images? No rigorous definition to the best of my

knowledge Loosely speaking, images of natural scenes

acquired by CCD cameras (Others call photographic images)

Why natural images? An important class of images with a variety of

applications (consumer electronics, biometrics, entertainment)

A good representative with high modeling complexity (arguably more challenging than other class such as medical images)

Cautious note about model complexity


To Understand Natural Images

Image Processing is also about Physics


Natural Scenes

How many different objects can appear in natural scenes?Countless – human faces, animals, buildings, mountains …

"Nature is not ecomonical of structures - only of principles" -Abdus Salam


Resolution Invariance


Scale Dependency

0.1m 1m

10m 100m


Self-Similarity: Fractals


Impact of Illumination

IndoorExample


Outdoor example


Story of “Lena” Image in USC Dataset


From USC to JPEG2K


The Space of Natural Images

(Courtesy of Prof. SC Zhu at UCLA)

contours

textures

smoothregions

By analogy, the space of natural images is very much like our universe


Challenge 1: Image Restoration

Practical limitation of image acquisition system Limited resolution (image size) Inevitable blurring and noise

Distortion introduced by image transmission Wireless channel: fading errors Internet: packet loss

If you work on communication, reliable communication of imagesthrough wired or wireless channel is a long-standing open problem


Image Denoising

Y=X+W

W: additive white Gaussian noise

denoisingalgorithm

X=f(Y)^


Our Tasks

Understand classical Wiener filtering Gaussian source, Gaussian noise Theoretically optimal

How does wavelet-based denoising work?

Why do statistical methods outperform others (e.g., PDE-based)?


Deblurring

Y=HX+W X=f(Y|H)^

H: linear blurring kernel

deblurringalgorithm

When H is unknown, itbecomes the notoriouslydifficult blind imageDeconvolution problem


Idea 1: Motion Deblurring

http://people.csail.mit.edu/fergus/research/deblur.html

“ Removing camera shake from a single image”Presented at SIGGRAPH 2006, Boston


Where is Blur?

Easy for human eyes but difficult for computers


Image Interpolation

interpolationalgorithm


Superresolution


Towards Gigapixel

http://www.tawbaware.com/maxlyons/gigapixel.htm

http://triton.tpd.tno.nl/gigazoom/Delft2.htm

Link 1

Link 2

3Mpel 1Gpel


Idea 2: Barcode Superresolution

How to extract the 1D barcode information from a 2D image?


Image Inpainting

Inpainting Algorithm


Application of Inpainting


Inpainting in Image Communication: Error Concealment


Deblocking

JPEG compressed imageat low bit rate

Restored image afterpost-processing


Deringing

JPEG2000 compressed imageat low bit rate

Restored image afterpost-processing


Challenge II: Robust Image Coding

5Mpel camera: 3bytes per pixel, 15MB per image512M memory: $40, $1 per image w/o compressionMemory will become less and less expensive (see next slide)


Holographic RecordingData

SLM Image Detector Image

RecoveredData

Channel

1 0 1 11 0 0 00 1 1 00 0 1 0

1 0 1 11 0 0 00 1 1 00 0 1 0

Dispersive channel

Courtesy of Kevin Curtis, InPhase Technologies


Bandwidth is STILL COSTY

Do you know how much Sprint charges for wireless data?


JPEG2000 vs. JPEG

http://www.aware.com/products/compression/j2kmaindemo.html

JPEG2000 JPEG

Compression ratio is the same: 217

Online comparison demo:


Our Tasks

Why is wavelet coding better? Properties of wavelet transforms Statistical modeling of natural images Importance of location uncertainty

How do we go beyond wavelet coding? Image quality assessment Rethink the role of bits (resolve location vs.

intensity uncertainty) Biologically-inspired approaches


Idea 3: Satellite Image Compression

Imagine you are in real-state business, don’t you wantto give your customers a virtual tour before a physical visit?


Challenge III: Image Analysis

Automatic target recognition

From low-level vision (image-in-image-out) to high-level vision (image-in-information-out)


Feature Point Matching at Low-level


Object Segmentation at Middle Level


Image Retrieval at High Level


Popular Demo: Face Detection

http://vasc.ri.cmu.edu/demos/faceindex/03282003/users/665.html


Challenges with Face Detection

http://vasc.ri.cmu.edu/demos/faceindex/12182002/users/2622.html


Idea 4: Face Image Indexing

How do we tell two people look alike?

FaceEye

nosemouth

Q: Can we automatically sort out face images based on their perceptual similarities?


Challenge IV: Image-related Security


Image Forensics

Courtesy of Dr. H. Farid at Dartmouth: http://www.cs.dartmouth.edu/farid/research/tampering.html

ee565 advanced image processing copyright xin li 20091 ee565: advanced image processing xin li...

Documents

digital image processing

image quality

image restoration

image analysis

images slide

usability image communication

assignments final project

digital video processing