image sensing & acquisition

Image Sensing& Aquisition

Presentation by Shashank Ram

12SS1A0446Dept. of ECE, JNTUH-

CES

ContentsWhich we will be covering

Digital Photo Sensing

Camera and it’s classification

Camera components and History

Analog photo sensing

Conclusion

Shashank RamJNTS

3

Camera ClassificationBased on the photo sensing element

Shashank RamJNTS

4

Sensor Sizes :Various image sensor sizes

5

Camera ObscuraDark Room in

Latin. A pin-hole camera

Silver NitrideReacts on

exposure to light

History of CameraA Timeline of the Camera Saga…..

6

Daguerreotype CameraA lens is used instead on a pin-hole camera

FilmIt reacts differently with different exposures of light.More durable.Roll film Camera

Compact and can accommodate more film .

Color film and DarkroomColor film is developed .

7

Advent of SLRProfessional cameras with optical viewfinders are introduced.SLR: Single Lens Reflex

Auto Focus RevolutionWith the addition of auto-focus many novice joined the SLR wagonDSLR

With the advent of transistors and digital logic this should be expected

CMOSCMOS sensors made the DSLRs more economical

8

Compact & MirrorlessWith the help of cmos small pocket cameras and cameras with no mirror for OVF developed using EVF

Machine Vision & VRComputer vison,AR & VR are the future applications

Shashank RamJNTS

9

Lens

Aperture

PhotoSensi

ng eleme

nt

Shutter

Buffer

ISP

Storage

Photon to BitImage Acquisition step by step

Shashank RamJNTS

10

Shashank RamJNTS

11

Microlenses

Microlens layer

Color filter layer

Metal opaque layer Photodetector Silicon substrate

•Main camera lens brings image to microlenses.

•Microlens funnels photons to active detector area.

Shashank RamJNTS

12

Bayer FilterA Color filter Array

(assumes UV and NIR filters)

• Each pixel gets covered by a colored filter– We use red, green, blue (RGB) CFA - best match for RGB

displays– Pixel colors arranged in “Bayer” pattern G R B G

Shashank RamJNTS

13

Photons to ElectronsThe essence of sensing an image

SiSi

Si Si

Si

Eh

e-e+

Covalently bonded silicon e-e+

P+

N

P

P+

h

Pinned photodiode

Shashank RamJNTS

14

• MOS-based charge-coupled devices (CCDs) shift charge one step at a time to a common output amplifier

CCD Image SensorYour great subtitle in this line

Shashank RamJNTS

15

Camera on a ChipActive pixel arrayAnalog signal chainAnalog-to-Digital Conv.VLSI Digital logic

I/O interface Timing and control Exposure control Color processing

CMOS SensorThe new age king!!

Shashank RamJNTS

16

CircuitryCMOS Sensor Circuitry

Read pixel signals out thru switches and wires

Timing And

Control Logic

Photodetector converts photons to electrons

Amplifier converts electrons to voltage after intrapixel complete charge transfer

Analog signal processor suppresses noise and further amplifies signal

Column multiplexer selects particular column(s) for routing to ADC(s)

Analog-to-digital converters (ADC) converts signal from volts to bits (usually 10 bits

Row select logic chooses which row is selected for readout.

Timing and control logic controls the timing of the whole sensor

Analog Signal Proc Column MuxADC ADC

resolution)

Shashank RamJNTS

17

Camera System-on-a-Chip integration is extensive•Color interpolation•Color correction, white balance•Dark signal correction, gamma and other normal corrections•Lens shading corrections•Format conversion and compression•Exposure control•Flicker detection and avoidance•Defect identification and correction•Auto focus support (focus score, actuator control)•Etc.

On-Chip ISPISP: Image Signal Processor

Shashank RamJNTS

18

Color InterpolationInterpolate = to estimate

Goal is to get best approximation for RGB at each pixel site but we start with just red, green or blue, not all 3.

Many ways to do color interpolation, for example:•Have blue, need green & red

• G = average of 4 neighboring greens• R = average of 4 neighboring reds

•Have green, need blue & red• B = average of 2 neighboring blues• R = average of 2 neighboring reds

•Have red, need green & blue• G = average of 4 neighboring greens• B = average of 4 neighboring blues

Shashank RamJNTS

19

Light OutEverything that glitters isn't GOLD

Image on screen should look like image seen, but better

Human eye response to color

LED LCD display color Spectrum and tuning

Computer processing

Silicon response andOn-chip processing

Camera processing

Shashank RamJNTS

20

• CMOS image sensors can incorporate other circuits on the same chip, eliminating the many separate chips required for a CCD.

• Not only does this make the camera smaller, lighter, and cheaper; it also requires less power so batteries last longer.

• Image stabilization and Image compression can be done on chip.• CMOS suffer in low light conditions due to less fill factor.

• The percentage of a pixel devoted to collecting light is called the pixel’s fill factor

• Less fill factor because each photosite is covered with circuitry that filters out noise and performs other functions.

• CMOS has more complex pixel and chip whereas CCD has a simple pixel and chip

• CCDs don’t suffer from rolling shutter effect as in CMOS• Hence CCDs are preferred in video.

But with BSI(back side illumination) and good image processing algorithms CMOS is winning the camera sensor battle!

CCD vs CMOSThe Battle begins….

Shashank RamJNTS

21

Resolution:The resolution of digital sensors is calculated by individual pixels , where as in film the resolution is estimated.

Resoltion of a large/medium format film is definitely very high (when scanned) compared to digital.

Noise/Grain:Digital sensors are more prone to noise where as film has the aesthetically pleasing grain.

Dynamic Range:Both compete head to head in this area.Film does well with highlights,where as digital with midtones..

Compactness and Ease of Development:In this regard Digital easily wins over Analog with more compactness and many post processing tools available at disposal.

Physical Darkroom vs Adobe Lightroom.

Analog(film) vs DigitalThe age old Debate….

Shashank RamJNTS

22

Image sensors have come along way since the 1st generation device – the CCD.

The 2nd generation device, the CMOS active pixel image sensor is going strong. “Billions and billions served”

• Image sensors are an emergent solution for practically every automation-focused machine-vision application.

• New electronic fabrication processes, software implementations, and new application fields will dictate the growth of image-sensor technology in the future.

ConclusionConcluding remarks……..

THANK YOUFor listening patiently