noise image sensors - university of waterlooece434/winter2008/noise.pdf · ... reset noise •...

114R.I. Hornsey, University of Waterloo

NoiseinImage Sensors


Introduction• We have seen how pixels are designed to

maximise the sensitivity to illumination

• However, this is only part of the story

• The overall performance of the sensor isultimately limited by the noise that is added bythe system to the signal

• In this sense, the noise figure of the detectorsystem is a measure of its “perfection”

• Noise comes from numerous sources and itsminimisation requires optimisation of manyindividual parts of the system

• Our discussion will not consider external noisesources, such as electrical pick-up

» the only “external” noise we will include is noise in theoptical signal itself

• The treatment of noise is a complex subject, andit is even harder to measure the individualcomponents accurately

» the theoretical treatment is important, however, as adesign tool for optimising the performance of specificstages in the system


Types of Noise

• “Noise” in image sensors is typically separatedinto two categories

» random noise

» pattern noise

• Random noise is what you might call “real”noise

» it is temporally random and is not constant from frameto frame in the image

» hence, it can be reduced by averaging successiveframes

» and is described by statistical distributions

• Pattern noise is effectively a spatial noise asseen by the observer of the image

» it does not change significantly from frame to frame

» and so cannot be reduced by frame averaging

• Pattern noise is divided into two components

» fixed pattern noise (FPN)

» photo-response non-uniformity (PRNU)


Pattern Noise

• FPN is the component of pattern noisemeasured in the absence of illumination

• It is mainly due to variations in

» detector dimensions

» doping concentrations

» contamination during fabrication

» characteristics of MOSFETs (VT, gain, W, L, etc.)

• PRNU is the component of pattern noise thatdepends on the illumination

• PRNU depends on

» detector dimensions

» doping concentrations

» thicknesses of overlayers

» wavelength of illumination (spectral response)

• Historically, pattern noise (FPN in particular) hasbeen the factor limiting the acceptability ofCMOS imagers

» PRNU is not often mentioned ...

» shortly, we will see how FPN can be reduced


Describing Noise

• Pattern noise is usually specified in terms of thevariation in the signals from individual pixelsunder uniform illumination

» usually as a percentage of the saturation output

• Random noise is expressed in terms ofparameters which describe the statisticaldistribution of voltage or current

• If there are n samples of the signal

» x1, x2, x3, . . . xn

• then the mean is x = (x1 + x2 + x3 + . . . xn) / n

• However, the mean for many noise sources iszero

» leaving the DC level of the signal unaffected

• So a more useful description of the noise iseither the variance (<x2>) or the standarddeviation (√<x2>, in rms units)

» which measures the scatter of the data points aboutthe mean

x2 = 1n

x j − x( )j=1

n∑

2


• To sum noise sources, we have to add thevariances

• or the standard deviation is given by

x = x12 + x2

2 + x32 + ... xn

2

x2 = x12 + x2

2 + x32 + ... xn

2


Importance of Noise

• We can illustrate the importance of the noise onthe overall sensor performance as follows

• Dynamic range = (saturation signal / rms noiselevel)

» saturation ≈ 200,000 e-, noise ≈ 40 e- rms

» typical value is 5,000 for a PD (~75dB)

» assuming dark current is not the limiting factor

102

103

104

105

106

# electrons

Light intensity(µW/cm2)

sensor + amplifier noise

saturation level

slope

= re

sponsi

vity

0.01 1010.1

saturation equivalentexposure

minimum resolvable signal


• Responsivity = (# electrons / light intensity)

» in linear portion of the curve (electrons.cm2/µW)

• Provided that the dark current is small, theminimum resolvable signal is determined by thenoise in the system

• Hence, a good responsivity is not enough toensure a good signal at low light levels

» a low “noise floor” is also required

• In a convenient model, the rms system noise is

» where the floor is determined by the amplifier noise,the reset noise, and the analog-to-digital converternoise

» the noise floor is often referred to as the read noise

• The other noise included above is called theshot noise, which arises because of thestatistical arrival of electrons

» due to the photo-generation of the electrons

» and the thermal generation of electrons

• We will now examine some of the noise sourcespresent in image sensors

nsys = nshot2 + nfloor

2 + npattern2


Thermal Noise

• Thermal noise is a white noise

» the noise power is constant over all frequencies

• For a resistor, the thermal noise root meansquare voltage is given by

» where R is the resistance, and B is the noiseequivalent bandwidth

• Since the thermal noise covers the entirefrequency range, the bandwidth determines theactual amount measured

• So the open circuit equivalent circuit is

• Alternatively

• However, an important factor is the noiseequivalent bandwidth for use in the calculation

vth = 4kTBR

ith =4kTB

R

RnoisyRnoiseless

<vth>=


Noise Equivalent Bandwidth• This is defined as the voltage-gain-squared

bandwidth of the circuit

• The ideal case is that the (gain)2 is constant at avalue of A0

2 up to the bandwidth (A0 = voltagegain)

• But, the behaviour of a real circuit is not abrupt

• The NEB is defined as the point at which the twoshaded areas equal

(gain)2

A02

Bfrequency

(gain)2

A02

Bfrequency

ideal response

real response


• Mathematically, this is given by

• So in the ideal case

• If we take the example of an RC low pass filter

• Calculating the transfer function

B = 1

A02 A f( )2 df

0

∞∫

bandwidth = 1

A02 A

0

∞∫

2

df =A0

2B

A02 = B

Vin

R

CVout

A ω( ) = vout

vin=

1jωC1

jωC+ R

= 11 + 2πfRC

since ω = 2πf

=f0

jf + f0 where f0 = 1

2πRC


• At f = 0, A(f) = A0 = 1 for this circuit

• Now we can calculate the noise equivalentbandwidth, using A0 = 1

• The reason for choosing this example is that itis directly applicable to the resetting ofphotodiodes and the output nodes of CCD andphotogate pixels

B =f0

f 2 + f02

0

∞∫

2

df

= f02 f0

2 + f 2( )0

∞∫

−1

df

= π2

f0


Reset Noise

• If we consider a diffusion (either a floatingdiffusion or a photodiode) being reset through aMOSFET

• Effectively, this is a capacitance being chargedthrough the resistance of the MOSFET channel

• So the ac-equivalent circuit is

Vreset

Vout

FETCVreset

R

R C Vout = 4kTBR


• From before, the bandwidth is

• So we find the rms noise voltage

• Usually, the noise voltages is expressed interms of electrons, in order to compare directlywith the electrons in the well

• In which case the reset noise on the capacitor iscalculated from Q = nq = Cvout, and the rmsnoise electrons is given by

• This noise is generally called “kTC noise” or, inthis case, reset noise

• Calculating this out at room temperature gives

• For a floating diffusion C ~ 20fF, so nkTC = 55 e-

• For a (10µm)2 photodiode, C ~ 60pF, so nkTC =100 e-

» currently, reset noise limits the read noise in PDs

B = π2

f0 = 14RC

vout = kTC

ne = Cq

kTC

= kTCq

nkTC,RT = 400 C pF( )


Shot Noise

• Shot noise is another white noise that arisesfrom the discrete nature of the electronsthemselves

» i.e. the random arrival of particles of charge

• This is the result of the random generation ofcarriers

» either by thermal generation within a depletion region(i.e. shot noise of the dark current)

» or by the random generation of photo-electrons,caused in turn by the random arrival of photons

• The rms signal is given by

• If the noise statistical distribution is describedby a Poisson distribution

» the variance is equal to the mean

» so <i2> = i

• So, if electrons are generated with a currentdensity, Jdark,in a sensor of area, A, over anintegration time, tint, the shot noise variance is

i = 2qIdcB

ndark2 = ndark =

JdarkAt intq


• Similarly, the photo-electron shot noise varianceis given by

» where I0 is the photon flux (photons/cm2s) and η is thequantum efficiency

• So the total rms shot noise contribution fromthe sensor is

• For example, with

» Jdark = 200nA/cm2

» A = (10µm)2

» tint = 30ms

» I0 = 1013 photons/cm2s

» and η = 0.5

• we find <nshot> = √(37,500dark + 150,000pe) = 430 e-

npe2 = npe = ηI0At int

nshot = ndark2 + npe

2 = ndark + npe

=JdarkAtint

q+ ηI0At int


Flicker (1/f) Noise

• At any junction, including metal-to-metal, metal-to-semiconductor, and semiconductor-to-semiconductor, conductivity fluctuations occur

» the causes of these are still not completely understood

• The rms 1/f noise current is given by

• 1/f noise arises mainly in amplifier circuitswhere there are numerous such contacts

• At low frequencies, 1/f noise can be thedominant component

» but, at higher frequencies, the 1/f noise drops belowthe thermal noise

» the frequency at which this happens depends on thesituation

i1 / f ∝ IdcBf

log fthermal

1/f

log <i>


Array Noise Componentsreset

sensor

in-pixelamp

columncircuit

on-chipo/p amps

A-D conversion

•reset (kTC) noise•FPN (from VT)

•photon shot•dark shot

•FPN•PRNU

•thermal•1/f

•FPN•(PRNU?)

•thermal•1/f

•FPN•kTC

•thermal•1/f

•quantisation noise

to noisy off-chip amps!

++

++

++

++

++

++

NO

T P

RE

SE

NT

IN

CC

D


“Referred” Noise Figures

• Conventionally, the noise figures are referredeither to the final output or to the output of theoptical detector

» i.e. to be compared directly with the number ofelectrons generated by the detector

» called input referred noise

• For input referred noise, the noise of laterstages must be divided by the gains of theintermediate stages

• Or vice versa for output referred noise

• Usually, authors in CMOS circles use the inputreferred figure

» but this is tough to obtain for intermediate stages inthe circuit owing to uncertainties in the gains of eachstage

» only the overall figure in electrons is practicallyfeasible because the appropriate inverse-conversionefficiency (e- per µV) is only known for the entireoutput circuit


Typical Noise Figures

• From Mendis, the calculated and measuredinput referred noises for a 128x128 elementphotogate array are

• Mendis also reported a photodiode read noise of~80 e- rms

• Typical read noises for CMOS sensors

• Remember that this does not include shot noiseor pattern noise

Noise source Calculatedrms

Measuredrms

kTC from reset of FD negligible negligibleIn-pixel amp. 1/f 111µVkTC from columnsample & hold

93µV

Column sourcefollower 1/f

46µV

Total column noise 86µV 120µVTotal noise 152µV 151µVTotal noise electrons 41 e - 41 e -

Technology RMS read noisePhotodiode APS 50 - 80 e -

Photogate APS 20 - 40 e -

Logarithmic APS 700 e -

Passive pixels 200 - 300 e -


Fixed Pattern Noise

• Fixed Pattern Noise is due to pixel-to-pixelvariations in the absence of illumination

• The main cause of FPN in CMOS imagers isvariations in VT

» between reset and buffer MOSFETs in the pixel

» and between MOSFETs in the column circuits

• FPN can also arise from repeating irregularitiesin the array clocking

» allowing small variations in integration time etc.

• In very large arrays, resistive drops in resetbuses may lead to a “droop” in the voltage towhich the pixels are reset,

» but this is not usually significant in CMOS imagers

• FPN is just as valid as a “noise” as the temporalvariety

» both affect the actual output voltage that the pixelproduces

» in a way that is not directly related to the illuminationto be measured


PRNU

• The issue of photo-response non-uniformity hasnot historically received much attention in theCMOS imager community

» although there is now some occasional mention of“gain nonuniformity”

• Like FPN, PRNU is essentially time-independent,but it is signal-dependent

• Both types of pattern noise can be specified interms of either an rms or a peak-to-peak value,referenced to an average value

» e.g. the full-well capacity

• A histogram of output signals is built up in thedark or light, as appropriate

» PNrms = rms of distribution / average value

» PNp-p = peak-to-peak variation / average value

• Since PRNU is signal dependent, it is oftenexpressed as a multiplier of the number ofphotons

» <nPRNU> = Unpe


Minimum Noise

• In principle, the noise floor and dark current canbe reduced so that the system is photon shotnoise limited

» this approximation is sometimes used to calculate thepixel sensitivity (µV/e-)

• But there will never be zero PRNU, so a moreachievable value would be

» The worst case when npe = nfull-well

• If we plot out this limiting noise as a function ofPRNU, it looks like

nsys = npe = npe

nsys = npe + Unpe( )2

100

1000

10000

0.01 0.1 1 10

photon shotnoise

PRNU

rms n

ois

e e

-

PRNU (%)

“knee” is optimum cost/performancevalue for PRNU, hereU ≈ 1/√npe npe = 50,000 e-

npe = 100,000 e-


Noise ReductionTechniques• Having seen some of the common sources of

noise in CMOS imaging systems, how might wego about reducing them?

• Essentially, there are three classes of noise

» those we can do nothing about, such as photon shotnoise

» those we can reduce by careful design of circuitcomponents, such as thermal noise

» those we can reduce by circuit design, such as FPN

• These techniques are inter-dependent

» we shall see that adding extra circuitry to reduce FPNalso introduces extra 1/f and kTC noise

» so the optimisation of noise is a system issue, not justa question of optimising each element individually

• We will look at the general techniques forreducing noise in electronic devices, as well ascircuit techniques for pattern noise etc

• The study of noise is a specialised topic, and wewill only look at the essentials


Shot Noise

• As we remarked earlier, photon shot noise isdependent on the illumination level, and there isnot much we can do about it

» except reduce the QE of the detector, which we don’twant to do!

• Shot noise also arises from the pixel darkcurrent

» which we can alter

• By changing doping levels, we can reduce thedark current

» but, in a regular photodiode, at the expense of QE

• And by removing the collection area away fromthe surface

» this is another advantage of the pinned photodiode

• The magnitude of the dark current is, of course,dependent on the pixel area

» so the shot noise will be smaller for smaller pixels

» although the perimeter component of the dark currentmeans that S/N still gets worse as pixel dimensionsare reduced

• But shot noise is not usually the limiting factor


Thermal Noise

• Thermal noise is important mainly in the inputstages of amplifiers

» because of the √(4kTBRchannel) from the MOSFETs

• In general, the power spectrum of the thermalnoise will be proportional to (W/L)-1

• But it is also dependent on the current throughthe devices

• A common way of expressing thermal noise isthe noise electron density (NED)

» where en(f) is the total equivalent noise voltage at theoutput stage (e.g. a floating diffusion)

» and Ct is the total capacitance present at the input,including diffusion capacitance, gate capacitance, andeverything else (to convert to electrons)

• en(f) represents the device noise, referred to theinput

» and so includes all factors such as the transistorgeometry, iDS, device area etc. that affect the gain

NED f( ) = en f( )Ctq

2


• NED, expressed in electrons2/Hz, is sketchedbelow for a bias current of 100µA

• Increasing W, for a fixed L, increases Ct becauseof the device area

• But increased W decreases en(f)

» because the current iDS, and the gain of the circuit, areincreased

» therefore reducing the input referred noise

• So the optimum condition is to keep W ≈ 15µmin these transistors

» too small a W reduces the gain

» and too large a W increases the C

• Also the NED decreases with L, which makessmaller devices advantageous

Transistor width, W (µm)

10 20 30 40 50

100

80

60

40

20

NE

D

(ele

ctr

on

2/M

Hz)

L = 3.2µm

L = 8.7µm


1/f Noise

• 1/f noise arises mainly from trapping de-trapping of electrons at the Si-SiO2 interface

• We can do some things to minimise 1/f noise

» reduce device W and increase device L

» use a buried channel device to separate the channelfrom the interface

• While in standard CMOS we cannot do muchabout the second option, we can change W, L

» but the gain of the amplifier is dependent on W/L

» so increasing both W and L is the best choice!

• In the pixel, this is a bad thing since we want tominimise the areas of the transistors toachieve a high fill factor

» and care is needed to ensure that the pixelsource follower can adequately drive the columncapacitance (i.e. enough W/L)

• Note also that we need the noise added at earlystages of the process to be minimised

» since this is amplified at all subsequent stages


Correlated Double Sampling

• Reset noise is difficult to design out of thesystem

» since the properties of the transistor cancel out

» although reducing the capacitance of the node isuseful for both kTC and conversion efficiency

• So the most common solution is to measure thevalue of the reset noise and then to subtract itfrom the signal

• A generic circuit for achieving this in a CCD orfloating gate APS would be

• During the sample and hold period, theappropriate switches are pulsed on & off

» to leave the voltages stored on the capacitors

+

–S&Hreset

S&Hsignal

FD

RST


• The sequence of events for a CCD or photogatewould be

• Here, the reset signal is given by

» Vreset = [VDD -(VT ± ∆VT)] ± (VkTC) ± (Vpart)

• Here the ∆VT is the component of FPN arisingfrom mismatches between the reset transistors

» and is approximately the same for each frame

• VkTC is the reset noise

» and is different from frame to frame

• Note that we are considering voltages (notelectrons) at this stage

» so the “kTC” noise is given by √(kT/C), and istherefore reduced for larger C

Subtract two stored valuesoutput = (signal + reset) - (reset) = signal

Sample & hold signal charge

Transfer signal charge(charge = signal + reset)

Sample & hold value of reset signal(charge = reset)

Reset pixel


• Vpart refers to what is called the partition noise

» when the reset FET turns off, the channel chargemoves either to the source (= FD) or to the drain

» difficult to determine how much goes to each

• This type of sample-and-hold technique isknown as correlated double sampling (CDS)

• The “correlated” part comes about because thenoise component of the two signals iscorrelated, and can therefore be subtracted out

• In a CCD, a single CDS circuit is neededbecause there is only one floating diffusionoutput node

• In CMOS APS, there is an output node per pixel

» but practically, we need only one CDS circuit percolumn of the array

» and the S&H is carried out for all columns in parallel

CDS

activerow

outputs


CDS for Photodiode APS

• This form of CDS works very well for pixels witha floating diffusion output node

» photogate, and pinned photodiode with transfer gate

• Indeed, the main advantage of using thephotogate structure is to facilitate the removalof reset noise

» since the improvement in conversion efficiency isoffset by the lower QE

• In photodiode designs, double sampling canonly remove the FPN that results frommismatches

» this is because the double sampling is not correlated

• In FD designs, the signal was added to theexisting (and stored) reset value

» so the subtraction was of exactly the same noisesignals

• In the photodiode, there is no separate outputnode, so the signal must be read out first

» and this signal includes the original reset voltage onthe photodiode

» which in turn includes FPN and kTC noise


• We can now reset the pixel again and subtractthis value

» the FPN will be much the same as that which wasincluded when we sampled the signal

» but the kTC noise will be different, i.e. not correlated

» remember kTC is the rms value of a distribution

• So now the sequence of events is

• This would be better termed pixel doublesampling

• Or, alternatively, a graphical representation is asfollows ...

Subtract two valuesoutput = (new signal + old reset) - new reset

Sample & hold reset valuecharge = new reset value

Reset pixel

Sample & hold signal valuecharge = new signal + old reset value


• For example, if we read out a single pixel overseveral integration periods

» assuming constant illumination (I.e. slopes parallel)

tint 2tint 3tint

idealsignal

measured signalVreset - Vsignal

VDD

VDD - VT

tint 2tint 3tint

range dueto kTC

S&H resetS&H signal

Vreset - Vsignal


• This is the reason why reset noise is now thelimiting noise source in photodiode circuits

• Note that even this noise reduction isunavailable to the logarithmic pixels, hence theirpoor FPN characteristics

• The conventional CDS circuit used in CMOSsensors is shown below

• Of course, the additional circuitry required forthe CDS implementation adds further noise tothe signal

» kTC from the sample-and-hold capacitors

» 1/f and thermal noise from the transistors

• But usually in CMOS sensors, the FPN is themore critical issue

VDD

load

load

columnload

ref. select

signalselect

SHR

SHS

VDD

VSS

VSS

VOUT

VREF

from column

1 pF

1 pF


Column FPN

• The other issue with using column-wise CDS isthat FPN is then added by the CDS circuitsthemselves

» appearing as vertical streaks in the image

• This can be removed by storing and subtractingcolumn reference signals off chip

• Alternatively a second stage of double samplingis performed

» where, after the readout of the differential signal, theS&H capacitors are shorted together

» this results in a differential output that is a measure ofthe mismatches between the two sets of output stages

» Mendis calls this a “crowbar” circuit and the processdelta difference sampling

+

–

S&Hreset

S&Hsignal

FD

RST

“crowbar”switch


Typical Figures

• Typical figures for FPN are hard to definebecause it depends so much on the preciseprocess used

• For photogates with a 2µm CMOS process,Mendis reported a p-p FPN of 1% – 2.5%saturation with the CDS circuit

» falling to ~0.1% sat. with the DDS as well

• A photodiode fabricated similarly showed a p-pFPN of ~0.5% sat. after CDS, and ~0.1% afterCDS + DDS

» typical raw data are about 2 - 3% p-p sat.

• For a 0.35µm process, the raw FPN for a PGarray was 6% sat.

» reducing to 0.4% after off-chip correction

• Mansoorian et al. give a final FPN of 0.6% sat. p-p for both PG and PD using a 0.55µm process

» using a similar DDS technique

• For logarithmic pixels, IMEC report a raw FPN of~100% of the useable signal range!


Patent Issues

• Other methods of reducing noise are possible,although probably not so good

• Hitachi have several patents which cover theidea of active pixel sensors and the use of CDSin these devices

» in their CMOS digital still camera, VLSI Vision use amechanical shutter in order to measure a true darkimage for subsequent subtraction

• One possibility is to smooth out large signalvariations between neighbouring pixels

» a smooth curve is fitted through points either side ofthe test point, and the test point moved to fit that curve

» the smoothing is improved if the number of neighboursis increased, but the “sharpness” of the image islowered

“corrected” value of test

real value of test

pixel count

signal

neighbouring test pixels


Feedthrough & Crosstalk

• We have considered here “natural” sources ofnoise such as 1/f, thermal, and shot noise

• And technological noise, such as FPN andPRNU

• In addition to these, there can be unwantedsignals in one part of the circuit due to theoperation of another part

» these can be addressed in the design of the array andcircuits

» although some sources are not easy to eliminated

• Feedthrough of digital signals from control linesinto the analog parts of the circuit can be aproblem

» analog and digital sections of the chip can beseparated to some extent

» but the array itself, and much of the analog signalprocessing is intrinsically both analog and digital

• The minimisation of these effects requirescareful layout

» and mixed signal design is currently a hot topic inmany areas, such as A-D & D-A conversion, DSP etc.


References – Part III

» H.W. Ott (1988), “Noise reduction techniques inelectronic systems”, Wiley

» G.C. Holst (1996), “CCD Arrays, cameras anddisplays”, SPIE Press

» T.E. Jenkins (1987), “Optical sensing techniques andsignal processing”, Prentice Hall

» S.K. Mendis (1995), “CMOS active pixel imagesensors with on-chip analog-to-digital conversion”,PhD Thesis, Columbia University, USA

» B. Mansoorian et al. (1997), “Megapixel CMOS APSwith analog and digital outputs”, IEEE CCD and AISWorkshop, Bruges, Belgium, June 5 - 7, 1997

» S.K. Mendis et al. (1997), “Active pixel image sensorsin 0.35µm CMOS technology”, IEEE CCD and AISWorkshop, Bruges, Belgium, June 5 - 7, 1997

» O. Yadid-Pecht et al. (1997), “Wide intrascenedynamic range CMOS APS using dual sampling”,IEEE CCD and AIS Workshop, Bruges, Belgium, June5 - 7, 1997

» B. Dierickx et al. (1997), “Offset-free offset correctionfor active pixel sensors”, IEEE CCD and AISWorkshop, Bruges, Belgium, June 5 - 7, 1997


CMOS Arrays

• The basic scanning of the array requires severalsets of inputs

» bias voltages

» clocking signals

» reset signals

• Bias voltages are required by the column sourcefollowers, and the S&H source followers

» and possibly by subsequent amplifier stages

• While bias voltages can be simply supplied by aresistive potential divider, this is not usuallyperformed on-chip because

» accurate resistors are hard to fabricate in digitalCMOS

» potential dividers dissipate power continuously,especially because large resistors are not feasible

» and we may well wish to adjust the biases, perhaps tochange the operating range of the sensor

• Usually, some externally adjustable voltages arerequired to program an internal active referencevoltage generator unit

» based on a bandgap reference source for stability


• An alternative approach is being pursued byJPL for low-power, digitally programmable stillcamera chip

• Here, four 5-bit digital-to-analog converters areused to program the DC reference voltages

» the digital input signal to each DAC can be latchedwith little power consumption

» and the DACs are powered down when their output isnot required to reduce power consumption

• This highlights an important area ofdevelopment, for battery-powered still camerasin particular

» power conservation by intelligent on-chip powercontrol

» e.g. standby modes (40µW for the JPL camera,~10mW during operation) by turning off analogcircuitry

» e.g. column-parallel ADCs only operated when columnis read out (plus maybe one warming up)


Timing• How the timing of the array readout is controlled

depends on the application

• In many cases, adherence to a standard videooutput format is not required

» and there is more flexibility in the control methodology

• If, however, the camera has to output signals instandard TV formats, for example, the frame rateand signal format are precisely defined

• The longest integration time allowable for thesensor is 1/(frame rate)

» so, 1/30 s for standard NTSC (National TelevisionSystem Committee) TV rates

• Video standards determine every aspect of thesignal output

» levels and timing of line, frame, blanking etc. pulses

» black levels and white levels

» compression of information – gamma correction

» frame rates

• There are two major standards, CCIR & EIA

» International Radio Consultative Committee (Europe)

» Electronic Industries Association (North America)


• And in each standard, there are two majorscanning modes

» progressive scan and interlaced scan

• The progressive scan achieves a high resolutionby displaying every line of the image

» e.g. computer monitor

• The interlaced scan achieves the same apparentresolution, but with a reduced bandwidth, bydisplaying every other line, and then going backto fill in the gaps

» e.g. all TV sets

» originally, the bandwidth of the radio transmission wasa major factor in adopting interlaced TV broadcasting

• Each scan is called a field, and each full pictureis a frame

» so 1 frame = 1 field for progressive

» and 1 frame = 2 fields for interlaced

1234567

1526374


• The frame rates are specified by the standardfield frequency

» CCIR: 50 Hz

» EIA: 60 Hz

• So an interlaced EIA system displays 30 frames(i.e. full pictures) per second

• In a CMOS imager, the easiest way to achieveinterlacing is simply to read out alternate rowsof the imager

» so there is effectively an independent array for eachfield

» [this does not work so well for CCDs, since the twofields cannot be reset separately; hence the fieldshave overlapping integration times, causing oddeffects for moving objects]

• After all the image processing, cameras forstandard video formats may include somedeliberately non-linear weighting

» known properly as “inverse-gamma correction”

» but often called simply “gamma” or “gammacorrection”


Gamma Correction

• The need for gamma correction arose becausethe inherent behaviour of CRT monitors is non-linear

» the brightness of the monitor is related to theaccelerating voltage of the electrons by

• So, if the camera were to have the inverserelationship, the properties of the scene wouldbe maintained

• Of course, this only works provided the correctmatch of camera and monitor is used

» although image processing systems, includingcomputers, usually offer a choice of gamma correctionparameters

• Because of the non-linearity, image processingshould be performed on the linearized image

» even if the signal is gamma corrected againafterwards

Ldisplay = K Vacc( )γ

Ldisplay = K Vacc( )γ = K Vscene

1γ

γ

= KVscene


• The precise value of gamma depends on theCRT manufacturer, but standard values arerecommended

» NTSC has γ = 2.2

» or an inverse γ of 0.45

• For low levels of light intensity, the effect ofgamma correction is very large, so there isusually some cut-off point below which gammais not applied

• In the camera, the gamma transformation isoften performed using a look-up table

» to convert the digital signals appropriately

» and in some cases, manufacturers may use their owncompression system to enhance contrast or signal-to-noise ratios


Exposure and Gain Control

• Exposure control refers to the amount of lightthat is being sensed by the camera

» primarily controlled by the integration time

• The simplest way of defining the integrationtime is to make it equal to the frame time

» i.e. the time between reading the pixel once, andreading it again on the next frame

• Alternatively, for non-TV applications, anelectronic shutter can be used to define theexposure time

• An integrated camera must be able to adjust itsexposure to the prevailing conditions byadapting its integration time

» but it can also adjust the gain of subsequentamplification stages to achieve the optimumcombination of detector sensitivity and output voltageswing


• In the VLSI Vision 5400 series monochromecamera chips, the gain and integration time arecontrolled together

» to maintain a relatively constant average output signal

• The number of pixels in a frame whose signal isabove some predefined threshold value iscounted

• If, for example, there are too few bright pixels

» first the integration time is increased

» if the integration time is near the maximum value, thegain is increased and the integration time reduced

» to allow room for further increase in the integrationtime

» the total range of exposure control (from min. tint &min. gain, to max. tint & max. gain) for the VVL 5400chips is about 100,000:1

• The same chips also allow for longer exposuresin part of the image in a “backlit mode”

» for dark objects on a bright background (snow, sand,sky etc)

» by applying a higher threshold in the central part of thearray

» this is a simple version of the methods used toincrease the dynamic range (see later)


Clocking and Speed

• The way in which the array is scanned verymuch depends on the application

» modifications must be made to accommodateinterlacing, variations in integration time, electronicshuttering etc.

• In the basic system of photodiode readout, wemust supply clocks to operate row and columnscanners, the two sample and holds, and thepixel reset

row advance

S&H signal

S&H reset

reset

column advance1 2 3 n

add delay here for longer tint


• Most camera chips generated these clocks on-chip from an externally regulated master clock

» usually a crystal oscillator like a computer to achievefrequency stability

» although on-chip oscillators are also used formaximum integration

• As with all circuits, the readout has a maximumfrequency of operation, determined by

» the fabrication technology

» the detailed design, both low-level and overall circuitfunctionality

• In particular, it is hard to drive large arrays fast,because of the capacitance of the address lines

» i = C dV/dt

» so larger C requires that the drivers of the addresslines must be able to source more current

» this requires larger transistors, and multiple stages(e.g. cascaded inverters), which also add delays

• It also makes the use of smaller pixelsadvantageous

» to reduce the array physical dimensions (hencereduce C) while maintaining resolution

» also reduces cost


• CMOS addressing is typically less capacitativethan for CCDs, but the on-chip drive circuitrycannot be too large

» because of the power dissipation

• For large arrays, such as the VVL 800K-pixelsensor, readout rates are 5 - 10 MPPS

» mega-pixels per second

» which is about typical

» or a few fps for megapixel arrays, and 30 - 100 fps forVGA (640 x 480) or less

• For specialised applications, such as industrialinspection, high frame rates may be required

» the Integrate Vision Products (IVP) MAPP2200 cancapture 4000 fps with a 256 x 256 array

» an impressive 260 MPPS

• But, in general, limiting the integration time tothe frame time results in tint being too long

» so how can we make a widely variable tint?

• This is also linked to the issue of reducingimage blur for rapidly moving objects

» e.g. scissor blades passing at 400 per minute

• The answer is an electronic shutter, analogousto that found in a mechanical camera


Electronic Shutter

• In a regular film camera, the shutter consists ofa slit made of two parts

» the first edge moves across rapidly, exposing the filmto the light

» the second edge follows the first, at a distancedetermined by the desired exposure time, cutting offthe illumination

» the exposure time is d/vslit, where d is the slit widthand vslit is the velocity

» this method, called a rolling shutter, is used to obtainimproved high shutter speeds

• The same method is also used to expose CMOScamera chips

» except that opening the “shutter” refers to selectiveactivation of rows of pixels

» rather than to a mechanical shutter

vslit

d


• Thus the readout is implemented as follows

• The integration time is thus variable from (1/m)to (1) times the frame time

» where there are m rows

» 70µs - 33ms for a VGA at 30fps

• A derivative of this technique has also beenused to increase the dynamic range of thecamera

» by allowing two integration times

scan

integratepointer

readpointer

integration period

inte

gra

te s

can

ner

read

/reset

scan

ner

de-activated pixelsare kept “reset” to

avoid blooming


WIDyR Sensor

• The Wide Intrascene DYnamic Range (WIDyR)sensor employs the same principle as the rollingshutter

» but has an additional column readout system

• The integration times can be controlled

» but not independently

» and the longest time depends on dark current

read &reset 1

read &reset 2

tshorttlong

output of integration

between reset 2 and read 1

output of integration

between reset 1 and read 2


Selective Spatial Scanning

• Many of the non-video-standard cameras offertwo modes of selective spatial scanning

» windowing

» sub-sampling

• Windowing refers to the selective scanning of arestricted area of the array

• Sub-sampling is the readout of every nth pixel

• The idea of both schemes is to get informationabout the scene without collecting all the data

» usually to enable faster scanning

window


• From the previous points, the scanning circuitshave to perform several functions

» sequentially scan out the columns and rows

» offer enough drive capability for the address lines

» be programmable for start, stop, and step

• There are two major candidates for the scanningcircuits

» shift register and counter/decoder

• Shift registers are simpler and more expandable

» every two clock periods, the signal on o/p1 istransferred to o/p2

» so whatever sequence we input at one end is passedalong to the array

Shift Registers

φ

φ

φ

φ

o/p 1 o/p 2


• So, to read out all the columns, we put a “1”followed by (n - 1) “0”s into the shift register,and this sequentially activates the columnreadouts

• The real advantage of the shift register is that itcan be made infinitely long without any changein architecture

» just by adding on extra cells

» the input circuitry etc all remains the same

• It can also handle two pulses for the shutter andthe WIDyR sensor, although it may well be betterto use two separate registers

• However, the shift register does not easily allowanything more complex than sequential readout

» random addressing is impossible; a “1” has topropagate all the way from the input before it readsout a column at the other end

» windowing & sub-sampling; how to determine startand stop? How to skip columns?

• So shift registers are usually used in caseswhere unconventional readout is not required


Decoder Scanning

• Decoder scanning has more in common with theaddressing of computer memory

» the decoder takes a binary input and selects theappropriate sequential output

• This allows any output to be selected at random

• Or, to scan the array, a binary counter canprovide the input

» this counter can have programmable counting limitsfor windowing

» and programmable counting steps for sub-sampling

• The main disadvantage is that the decodingcircuitry is more cumbersome and lessstraightforward to expand

» although modern layout tools minimise this problem

0 15...

0123binary input


Analog-to-DigitalConversion• The preceding sections have covered the main

building blocks required to operate anintegrated camera

• However, there is an increasing demand fordirect digital signal output, especially forcomputer interfacing such as

» digital still cameras

» machine vision & industrial inspection

» interne-based video telephony

• But there are many possible ways of includingADCs onto the chip

» one ADC per chip

» one ADC per column

» one ADC per pixel

• The choice depends on a number of factors,such as

» speed of conversion (1 - 50MPPS for video rates)

» space (area of layout, column pitch)

» number of bits (at least 8 for most applications)

» power consumption (<100mW to avoid local heating)


General Considerations

A/D

A/D A/D A/D A/D

A/D A/D A/D A/D

A/D A/D A/D A/D

A/D A/D A/D A/D

A/D A/D A/D A/D

A/D A/D A/D

A/D

In pixel

Per column

Per chip

Factor Inpixel

Percolumn

Perchip

Speed l £ ¡S/N l £ ¡FPN ¡ £ lAccuracy ¡ £ lFill factor /resolution

¡ l l

Powerconsumption†

¡ £ l

l good £ medium ¡ poor† Power consumption depends on how the system is operated.


Flash ADCs

• Flash ADCs, sometimes also called parallelADCs are the fastest converters, owing to theirparallel operation

• All comparators whose reference is below theinput voltage gives an output “1”

» and the priority encoder outputs the binary numbercorresponding to the highest active comparator

R

R

R

R

R

+

+

+

+

+

Vin Vref

digitaloutput

priorityencoder

smallestreferencevoltage

largestreferencevoltage


• The flash ADC is fast

» limited only by the delays in the comparators and theencoder

• But it requires 2N - 1 comparators (N = # bits)

» so the number ~ doubles for each extra bit

» and this requires a large area on the chip

• Moreover, the resistor chain imposes additionallimitations

• Firstly, there is a continuous current flowthrough the resistors

» Rs can be made large to reduce this, at the expenseof real estate

• Secondly, the accuracy of the ADC depends onaccuracy to which the resistors can befabricated

» limited to 8 - 10 bits without special “trimming”


Successive ApproximationADCs• These ADCs converge on the correct digital

word by comparing the input voltage with theoutput of a DAC corresponding to that word

» and adjusting the digital word to bring the DAC outputcloser to the analog input

• The converter first tests the most significant bitof the digital word

» and turns it on or off according to whether theresulting DAC-ed voltage is greater or less than theinput

» then each lesser significant bit is tested until the bestdigital equivalent is reached

• Thus the successive approximation ADCrequires N clock cycles to perform theconversion

DAC

Successiveapproximation

register

clock

output

Vinput


Single-slope ADC

• Also known as the Counting ADC, this is similarto, but simpler than, the successiveapproximation type

• Now, the binary counter is incremented (ordecremented) until the output from the DAC justexceeds (or falls below) the analog input

» when the comparator inhibits the counter

» and the output of the binary counter is the best digitalapproximation to the analog voltage

• The speed and accuracy of the conversion arenot good

» the speed depends on the voltage to be converted

» but the area and the power consumption are relativelylow

DAC

Binarycounter

clock

output

Vinput

stop


Dual-Slope ADC• The dual-slope, or ratiometric, ADC is another

counting converter, but is more sophisticatedand robust than the single slope method

• Here, the analog input it integrated for a fixedperiod of time

• And then the time required for a knownreference voltage to achieve the same finalvalue is measured by a binary counter

• This is achieved by using an integrator with ananalog switch to determine the input

-Vintint

tint

t

Vint int

Vref


• The accuracy of this technique can be very high,but at the expense of a long integration time

• Moreover, the effects of digital noise can beremoved by ensuring that the integration periodis an integral multiple of the interfering period

• But the power consumption and area are stilllow

–

+

Vref

(- ve)

Vin

–

+

binarycounter

clock

inhibit

analogswitch


Sigma-Delta ADC

• Sigma-Delta (∑-∆) converters offer an excellentimmunity to process variations

» whereas previous types rely on high accuracycomponents

• This is because it functions on an averagingbasis

» in its simplest form the ∑-∆ converter block diagram isas follows

• During each clock cycle, the integratoraccumulates multiples of the input voltage

» until the total voltage exceeds the reference voltage,Vmax, and the output, q, goes to “1”

» at which point, the feedback loop subtracts Vmax again

integrator(summer)

–

+Vin

(< Vmax)

Vmax

+

–counter

clock

qv


• The process repeats and, after another fewperiods, q → 1

• So, by counting the number of “1”s over 2N

clock periods (N = # bits), a binary counter willgive the analog output

• More complex versions of the ∑-∆ converter(second order etc) are possible

» they reduce the number of clock cycles for a givennumber of bits

» but at the expense of extra feedback loops, withhigher real estate costs

0 0 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 1 0

Vmax

v

Vin


Summary

• Following Mendis, the merits of the variousADCs can be summarized as follows

• Here, robustness refers to the immunity of thetechnique to the process variations inevitable inthe use of standard CMOS technology

• It is generally considered that flash ADCs aretoo large for other than one per chip

» but that the others would be suitable for one percolumn

» and one per pixel is not really viable

Type Resol-ution

Speed Power Area Robust-ness

Sigma-Delta

High Slow Low Low High

Succ.Approx.

Med. Fast High Med. Med.

Single/dualslope

High Slow Low Low Low

Flash Low Fast Veryhigh

High Med.


Colour Processing

• For most consumer applications, colour imageshave become mandatory

• While the technology to sense colour has beenavailable for a long time, the on-chip colourprocessing and video encoding is only nowbecoming feasible

• One method for obtaining very high qualitycolour images is to use three separate sensors

» and to split the light into primary colours (red, green,blue) by a prism

» or to use a single sensor and a colour filter wheel

• The principle of operation is that additive mixingcan reproduce the entire spectrum of colours

• An important factor when designing colourimaging systems is that the human observer ispart of the system

» and so human perception must be considered whendetermining the performance of the sensor

» the human colour perception has been standardizedsince 1931!


Colour Filter Arrays• For single-chip colour sensing, individual pixels

in the array are covered with coloured filters

» to form the colour filtered array (CFA)

• The spectral responses of conventional R, G, Bfilters are sketched below

• An alternative to the RGB system is to use thecomplementary colours, which have a higherpercentage transmission

» yellow (Ye) = R + G = W - B

» magenta (Mg) = R + B = W - G

» cyan (Cy) = G + B = W - R

» where W = white (transparent)

B G R

400 500 600 700Wavelength (nm)

1.0

0.8

0.6

0.4

0.2

Tra

nsm

issio

n


CFA Arrangement

• Coloured filters are typically arranged either instripes or in mosaics (checkerboards)

» the mosaic is often called a Bayer pattern after itsinventor (and 1976 patent holder)

• While the stripes have some advantages interms of simplicity of fabrication, the horizontalresolution suffers seriously

» and all CMOS imagers use the Bayer patterns

» recall that the VVL 800K-pixel colour sensor usesRGB and “teal” (blue/green) to achieve a propercolour balance with their process

R

R

R

R

G

G

G

G

B

B

B

B

R

R

R

R

G

G

G

G

B

B

B

B

G

G

G

G

Ye

Ye

Ye

Ye

G

G

G

G

Cy

Cy

Cy

Cy

G

G

G

G

Stripes

Mosaics

R

G

R

G

G

B

G

B

R

G

R

G

G

B

G

B

Mg

Ye

Ye Ye

G

G

Cy Cy

Cy Cy

G

G

Mg

Ye

Mg Mg

(Bayer pattern)

R

G

R

G

G

B

G

B

R

G

R

G

G

B

G

B(modified Bayer)


Matching Human Response

• The human eye derives most of its spatialinformation from the green part of the spectrum

• So the standard reponse calls for 59% of theluminance (brightness) signal to come fromgreen, 30% red, and 11% blue

• But, if only 11% of pixels detect blue light, thespatial resolution is too low, so the compromisesolution is the Bayer pattern with 50-25-25

» the modified Bayer pattern is sometimes used tofacilitate the interlaced scanning by repeating thepattern on adjacent rows

• The signals are then processed to achieve thecorrect colour balance

» in NTSC this is known as matrixing

• A selection of matrices may be available for useunder various illumination conditions (scenecolour temperatures)

» e.g. tungsten lighting, twilight, full sun

R2

G2

B2

=k11 k12 k13

k 21 k22 k23

k 31 k32 k33

R1

G1

B1


• The image processing required to NTSC colourprocessing is quite sophisticated, which is whya single chip version is only now available

» from VVL (ISSCC98)

• Another potential difficulty that arises from theuse of CFAs is the low resolution of eachcoloured “array”

» leading to the possibility of aliasing

» this can be corrected with off-chip filtering, at theexpense of further image processing

• Once the NTSC colour balance has taken place,the signals are then gamma corrected

• So the whole system may look as follows

array

S/H

S/H

S/H

R

B

G

colourcorrec-

tionmatrix

γ

γ

γ

NTSCencoder


Advanced focal-planeprocessing• One of the most fascinating aspects of image

sensor design is the constant battle to imitatenature

• It is a pretty good rule of thumb that if you wantto find any engineering solution, look first athow nature does the job and try to imitate it

» mechanics, structures, sensors

• Of course, there are things that artificialconstructions can do that nature cannot, andvice-versa

» and nature often has the edge by combining physics,chemistry and engineering at microscopic level

• Because of the importance of sight to humans,visual perception has always held a particularinterest

• So it is not surprising that many attempts havebeen made to implement biologically-inspiredvision systems in VLSI

» these run the range from attempts to verify models ofperception, through to engineers trying to satisfy anapplication


Human Eye vs. Silicon(B. Dierickx, IMEC) criterion eye CCD CMOS APS film

spectral

response

400-700 nm

peaked at 555

400-1000 nm

smooth

400-1000 nm 300-700 nm

peak quantum

efficiency

<20% >50% >50%

dynamic range 1e6 logarithmic

1e2 linear

1e4 linear 1e4 non-linear both

dark limit 0.001 lux

1E-6 W/m2

typ: 0.1 lux

<0.0001

possible

typ: 1 lux

0.001 possible

virtually zero

noise photons

(*)

10 10 100 100

integration time

(room

temperature)

0.3 s 40 ms typical

5 minutes

possible

40 ms typical

+10 seconds

possible

virtually

unlimited

max. frame rate ca. 15 Hz 10 kHz >> 10 kHz 1 shot only

(*) noise photons is defined as: noise photons = noise electrons / (quantumefficiency * fill factor)

resolution

criterion eye CCD CMOS APS film

#pixels 120M typical: 800Krecord: 60M

typical: 800Krecord: 4M

typical: 6e6grainsrecord: papersize

pixel pitch 2-3 µm 5-10 µm 5-10 µm typ 10 µm grainpattern

focal planesize

3 cm 1 mm ...11cm

1 mm ... 2cm

only limited byfilm size


operating conditions

eye CCD CMOS APS film

radiation hardness 1 mrad 10 krad 1 Mrad

operating Temp 36 C -200 C...+200 C 0 K ... +200 C 0K ... 100 C

power dissipation < 1mW 500 mW typ 50 mW typ nihil

on-board image processing


cosmetic quality perfect very good worse as good asperfect

color ideal poor (onlyRGB)

poor poor or printquality

absolutephotometry

impossible easy easy possible

focal planeprocessing

extensive none typically none none

access method data driven(focus ofattention)

serial only serial, randomaccess, ...

optical only

datapath 5M nerves 8-10 bits 8 bits none

logistics


price invaluable typ 10000 BEF typ 1000 BEF typ 10 BEF 2nd source none few many fewtechnologydevelopmentcycle

500 Myears 5 years 2 years 20 years

fabs 3E9 15 1000 10


Artificial Eye?

• Imaging chips made with standard techniquesmust be planar

» so detector and processor must be side-by-side

» to maintain resolution and sensitivity, we usuallydivide the chip into sensor area and processing areas

» which limits the speed and parallelism of the process

• The maximum area for our system is also limitedby the fabrication technology

» which will restrict the total amount of processingpower available to us

» if we could include a state-of-the-art microprocessorand memory on-chip, the camera would appear muchmore intelligent! The low-level technology is there, butnot the system level.

• So there are two extreme approaches, and asyet only a few combined approaches

» complex in- and inter-pixel processing at lowresolution, with much internal control

» higher resolution, but less autonomous, peripheralprocessor-type devices that we have seen before


• So, in this section, we will look at some of themore advanced “engineering” image sensors

» these typically follow the peripheral processorarchitecture, but with adaptive or programmablecapability

» but there are not too many of these

• We will look at some of the things that it wouldbe useful to do and how they are implementedcurrently

• Then we will look into the world of biologically-inspired image sensors

» many designs have been suggested to accomplishboth spatial and spatio-temporal image processingfollowing “retinal” lines

» more than 50 of these have been collected andreviewed by A. Moini from University of Adelaide,Australia (see references for web page)


Increased Dynamic Range

• We saw in the previous chapter that amodification of the electronic shutter idea couldgive a sensor with two integration times

» the WiDyR sensor from JPL

» the images could be recombined to offer a sensor witha wider dynamic range

» a similar concept has been used with CCDs (calledthe Hyper-D CCD) to give two exposures of differentlengths, which are later combined

• And one of the main reasons for tolerating thedisadvantages of logarithmic pixels was the 106

dynamic range that could be achieved

• One of the potential problems with approachessuch as the WiDyR is that of blooming

» in the long tint section, charge from bright pixels willspill over into neighbouring pixels

» making their signals indecipherable

• So apparently the only other viable approach toincreasing the dynamic range – by a circuitmethod – is

» to control the integration time of each pixel individually


Individual Pixel Reset

• Usually, photodiode pixels are reset by rowsafter the data has been read out

» hence the natural idea of the electronic shutter

• But the addition of another reset transistor canmake the reset signal X-Y addressable too

» this is achieved with a column reset (CRST) as well asa row reset (RRST)

• The circuit above is the most obvious method(e.g. Ginosar & Gnusin)

» but this introduces reset anomalies

» when CRST is pulsed, node N charges to VDD

» this charge is transferred to the diode when RRST ispulsed, even though CRST & RRST were never onsimultaneously

VDD

CRST

RRST

RS


• To remedy this reset problem, the circuit wasmodified as above

» for use as a star tracker by JPL (Yadid-Pecht)

• So now the idea is that we can control theintegration time of each pixel

» to allow poorly illuminated pixels more time

» and to read out and reset brightly lit pixels, therebyavoiding the blooming problem

» in this work, the integration time was controlledexternally

• This approach has several importantimplications for the sensor design

• Dark current becomes more of a limitation onthe dynamic range than saturation, determiningthe maximum tint

VDD

CRST

RRST

RS


• The more profound departure from normalpractice is that the time of the readout is nowimportant, as well as the signal level

» because two pixels may achieve the same signallevel, but after different integration times

» this makes reconstruction of the image more tricky

• While JPL’s approach was to control the tint

externally, several of the “biological” designsallow the pixel to control its own tint

» and maybe those of adjacent pixels as well, to achievea spatial image processing function rather than awider dynamic range per se

• Ginosar & Gnusin proposed a complicatedversion of the dual-tint approach

» where a latch in each pixel informed control circuitswhich tint applied to that pixel

» and this was adjusted for the next frame according tothe value of the present frame relative to a threshold

» but the fill factor of a 60 x 70µm pixel was only 13%

• Other attempts have led to similarly large pixelsizes

» e.g. > 100 x 100µm by Miyagawa & Kanade


Image Compression• The quantity of data contained in images is so

large that most transmission systems requirethe image to be compressed beforetransmission

» 1000 x 1000 pixels gives 1MByte if 256 grey levels

» times however many frames per second

» a 38.8k bits/s modem can transmit one uncompressed(100 pixel)2 8-bit image every ~ two seconds

• There are many algorithms for compression

• Some are standard formats

» JPEG (Joint Photographic Experts Group) for stillpictures

» MPEG (Motion Picture Experts Group) for movingpictures

» H.263 low bit-rate standard, e.g. for videotransmission over a telephone line

• Some are not-yet-standard mathematical tools

» wavelet compression, fractal-based compression

» e.g. Microsoft Encarta cd-rom encyclopaedia

• And others are “natural” methods, such as

» motion detection, foveation, data reduction


Mathematical Compression

• To date, there have been few reports of complexcompression techniques being performed on-chip

» JPEG encoding is possible in hardware, but has notyet been integrated with an image sensor

» MPEG encoding is hard to do on-chip, and theresulting chip would be large and have a high powerconsumption

• One of the most recent reports by Kawahito etal. in Dec. 1997 has been of an imager with two-dimensional discrete cosine transformcompression, in the analog domain

» a 128 x 128 passive pixel array was read out in blocksof 8 x 8 pixels

» the DCT operation is performed by two successiveweighted summations, each carried out in parallel, 8-bits at a time

» with intermediate results stored in an 8 x 8 memory

• While the analog processing gave theadvantages of speed and small silicon area

» about 1mm2 for the DCT processor in 0.35µm CMOS

» the precision of the method was degraded bymismatches between storage capacitors etc

noise image sensors - university of waterlooece434/winter2008/noise.pdf · ... reset noise •...

Documents