dip basics and rectification_2

1iirs

Digital Image Processing Basic Concepts

Mrs. Minakshi Kumar

[email protected] and Remote Sensing Division

Indian Institute of Remote Sensing

iirs

Digital Image Processing- Basic Concepts

Outline What is a digital image & multispectral image Data formats for storing digital image data Digital Image Preprocessing

DISCUSSION / INTERACTIVE SESSION

Digital Image Processing Minakshi,PRSD,IIRS 2

2iirs


A Picture is worth a thousand words

iirs


What is an Image ? An IMAGE is a Pictorial Representation of an

object or a scene. Forms of Images v Analog v Digital

Analog Imagesv Produced by photographic sensors on paper based

media or transparent mediav Variations in scene characteristics are represented as

variations in brightness ( gray shades) v Objects reflecting more energy appear brighter on the

image and objects reflecting less energy appear darker.

3iirs


Analog Images

iirs


What is a Digital Image ?

Produced byElectro optical Sensors Composed of tiny equal areas, or picture

elements abbreviated as pixels or pels arrangedin a rectangular array

With each pixel is associated a number knownas Digital Number ( DN) or Brightness value(BV) or gray level which is a record of variationin radiant energy in discrete form.

An object reflecting more energy records ahigher number for itself on the digital image andvice versa.

4iirs


Single Layer Digital Image-Panchromatic Imagery

256-2048 Brightness Levels 0 Darkest, 255/2047 - Brightest Higher spatial resolution

Hypothetical Digital Image (5 x 5 x 1)

iirs


Multi Spectral Remotely Sensed Image

Digital Images ofan area capturedin different spectralranges (bands) bysensors onboard aremote sensingsatellite.

A pixel is referredby its column, row,band number.

5iirs


What is a Multispectral Digital Image ?

72808492979485787581

72707478859397888179

71616770768290979387

727479777579798910396

4459778785848897110105

563951839195101100104104

7558435684106106938693

78797541406587867988

78828672453244698280

85858684775938457779

87918881858464415370

72808492979485787581

72707478859397888179

71616770768290979387

727479777579798910396

4459778785848897110105

563951839195101100104104

7558435684106106938693

78797541406587867988

78828672453244698280

85858684775938457779

87918881858464415370

72808492979485787581

72707478859397888179

71616770768290979387

727479777579798910396

4459778785848897110105

563951839195101100104104

7558435684106106938693

78797541406587867988

78828672453244698280

85858684775938457779

87918881858464415370

columns (x)

row

s (y)

72808492979485787581

72707478859397888179

71616770768290979387

727479777579798910396

4459778785848897110105

563951839195101100104104

7558435684106106938693

78797541406587867988

78828672453244698280

85858684775938457779

87918881858464415370

72808492979485787581

72707478859397888179

71616770768290979387

727479777579798910396

4459778785848897110105

563951839195101100104104

7558435684106106938693

78797541406587867988

78828672453244698280

85858684775938457779

87918881858464415370

72808492979485787581

72707478859397888179

71616770768290979387

727479777579798910396

4459778785848897110105

563951839195101100104104

7558435684106106938693

78797541406587867988

78828672453244698280

85858684775938457779

87918881858464415370

iirs


Multispectral Digital Image

Band 1 Band 2 Band 3 Band 40.45 - 0.52 m m 0.52 0.59 m m 0.62- 0.68 m m 0.77 0.86 m m

IRS-1B LISS II IMAGE OF PAONTA AREA

6iirs


Multispectral Imagery

Enhanced Features with Color Broad Area Coverage Tens of Color / IR Bands Allows Discrimination of difference in Brightness

iirs


Digital Image Data Formats

Data formats for storing digital images. Typical digital images occupy large memory. Example an IRS P6 LISS III image of 148 Km

by 148 km with a spatial resolution of 23.5 (~24m)and in 4 spectral regions No of scan lines = 6167 (148000/24) No of Pixels = 6167 per scan line Total no of pixels 6167 x 6167 x 4

= 152127556 pixels ~ 152 mega pixels~ 145 MB

Organisation / Storage of these pixels in memory ??

7iirs


Digital Image Data Formats

Commonly used formatsBand Sequential (BSQ)Band Interleaved by Line (BIL)Band Interleaved by Pixel (BIP)

Each of these formats is usually preceded on theby "header" and/or "trailer" information, whichconsists of ancillary data about the date, altitudeof the sensor, attitude, sun angle, and so on.

Such information is useful when geometrically orradiometrically correcting the data.

iirs


Band Sequential Format

Data for asingle band forentire scene iswritten as onefile

That is for eachband there isseparate file

8iirs


BAND INTERLEAVE BY LINE (BIL) Data for all the

bands are writtenas line by line onthe same file.

That isline1, band 1, line 1

band 2, line1 band3, line2, band 1,line 2 band 2, line2band 3 , line3,band 1, line 3 band2, line3 band 3 etc.

iirs


BAND INTERLEAVE BY PIXEL

data for the pixels in allbands are writtentogether.

That ispixel1 band1 pixel1 band2

pixel1 band3 pixel2 band1pixel2 band2 pixel 2 band3pixel3 band1 pixel3 band2pixel3 band3

9iirs


Advantages / Disadvantages

vBSQ if one wanted the area in the center of a scene in four

bands, it would be necessary to read into this locationin four separate files to extract the desiredinformation. Many researchers like this formatbecause it is not necessary to read "serially" pastunwanted information if certain bands are of no value

vBIL/BIP It is a useful format if all the bands are to be used in

the analysis. If some bands are not of interest, theformat is inefficient since it is necessary to readserially past all the unwanted data.

iirs


GeoTiff

"The GeoTIFF spec defines a set of TIFF(Tagged Image File Format) tagsprovided to describe all Cartographicinformation associated with TIFF imagerythat originates from satellite imagingsystems, scanned aerial photography,scanned maps, digital elevation models, oras a result of geographic analysis" (Ritterand Ruth 1999).

10

iirs


Different Data FormatsvIKONOS Data supplied in GeoTiff formatvNDC suppliesvIRS LISS-I,II,III,IV data in LGSOWG

(superstructure) / Fast formatv Stereo data of CARTOSAT 1 is in GEOTIFF

format

v Most of the Recent satellite data iisprovided in GeoTiff format

iirs


Digital image processing can be defined as thecomputer manipulation of digital values contained inan image for the purposes of image correction, imageenhancement and feature extraction.

Digital Image Processing

A digital image processing system consists of computerHardware and Image processing software necessary toanalyze digital image data.

Image Processingsystem

Hardware Software

11

iirs


DIGITAL IMAGE PROCESSINGSystem FunctionsSystem Functions

vData Acquisition/Restoration{Compensates for data errors, noise and geometric distortions introduced in the images during acquisitioning and recording} i.e Preprocessing (Radiometric and Geometric)vImage Enhancement

{Alters the visual impact of the image on the interpreter to improve the information content}

iirs


DIGITAL IMAGE PROCESSINGSystem FunctionsSystem Functions

vInformation Extraction{Utilizes the decision making capability of

computers to recognize and classify pixels on the basis of their signatures, Hyperspectral image analysis }vOthers{Photogrammetric Information Extraction , Metadata and Image/Map Lineage Documentation , Image and Map Cartographic Composition, Geographic Information Systems (GIS), Integrated Image Processing and GIS , Utilities}

12

iirs


Major Commercial Digital Image Processing Systems

ERDAS IMAGINE ENVI IDRISI ER Mapper PCI Geomatica eCognition MATLAB Intergraph

iirs

Open Source DIP Softwares

ILWIS (http://www.ilwis.org/index.htm) Opticks

http://opticks.org/confluence/display/opticks/Welcome+To+Opticks

GRASS (Geographic Resources Analysis Support System http://grass.osgeo.org/ )

OSSIM (Open Source Software Image Map www.ossim.org )

Multispec https://engineering.purdue.edu/~biehl/MultiSpec/index.html


13

iirs


Image Preprocessing Remote sensing systems do not function perfectly. Also, the Earths atmosphere, land, and water are

complex and do not lend themselves well to beingrecorded by remote sensing devices that haveconstraints such as spatial, spectral, temporal, andradiometric resolution.

Consequently, error creeps into the data acquisitionprocess and can degrade the quality of the remotesensor data collected.

The two most common types of error encounteredin remotely sensed data are radiometric andgeometric.

iirs


Image Preprocessingv Radiometric and geometric correction of remotely sensed data

are normally referred to as preprocessing operations becausethey are performed prior to information extraction.

v Image preprocessing hopefully produces a corrected imagethat is as close as possible, both radiometrically andgeometrically, to the true radiant energy and spatialcharacteristics of the study area at the time of data collection.

v Correct distorted or degraded image data to create a morefaithful representation of the original scene (usuallypreprocessing operations)

v Radiometric errors affect the Digital Number (DN) stored in animage

v Geometric errors change the position of a DN value

14

iirs


Radiometric errors- Causes

Radiometric errors are caused by detectorimbalance and atmospheric deficiencies.

Radiometric corrections are also called ascosmetic corrections and are done toimprove the visual appearance of theimage.

iirs


Radiometric errors

Several of the more common remotesensing systeminduced radiometricerrors are:

Periodic line or column drop-outs, line or column striping. random bad pixels (shot noise), partial line or column drop-outs

15

iirs


Line Dropouts l DN 0 or 255l not systematicl partial/entire line

Possible cause: failure of a detector

bad transmission storage defect processing defect

iirs

Dropped lines replacement No data available Correction is a cosmetic operation It is based on spatial auto-correlation of continuous physical

phenomena (neighbouring values tend to be similar)

l Replacement (line above, below)l Average line above and belowl Replacement based on correlation

between bands

16

iirs


Correction for Missing Lines /columns

It is first necessary to locate each bad line in thedataset.

A simple thresholding algorithm makes a passthrough the dataset and flags any scan linehaving a mean brightness value at or near zero.

Once identified, it is then possible to evaluatethe output for a pixel in the preceding line (BVi 1,j,k) and succeeding line (BVi+1,j,k) and assignthe output pixel (BVi,j,k) in the drop-out line

iirs

Dropped lines replacement Replacement : copy the contents from the line

above or below

Average line above and below

V = radiometric value (DN)

i,j = column, line indicator Replacing the line with other highly correlated

band

1-ji,ji, VV = V Vi, j i, j + 1=

V V Vi, j i, j 1 i, j 12

= ++ -( )

17

iirs


Line Striping (Banding)v Sensor induced

Radiometricerrors

v Systematic

DeDe--stripedstripedStripingStriping

iirs


Line Striping (Banding) the response of some of the detectors may

shift towards lower or higher end causingthe presence of a systematic horizontal /vertical banding banding pattern

Banding is an cosmetic defect and itinterferes with the visual appreciation ofthe patterns and features on the image

Variation in gain and offset (dark current)of each sensor (linear sensorcharacteristic) as the sensor deteriorates intime

18

iirs


Line striping correction Uses a linear expression to model the

relationship between input & output values. Assumes that mean and standard deviation

of data from each detector should be same. Linear sensor model :

y = a.x + ba = gainb = offsetx = inputy = output

iirs

Haze


Dark object subtraction method

Assumption: infrared bands are not affected by Haze Identify black bodies: clear water and shadow zones with zero

reflectance in the infrared bands Identify DN values at shorter wavelength bands of the same pixel

positions. These DN are entirely due to haze Subtract the minimum of the DN values related to black bodies of a

particular band from all the pixel values of that band

Scattered light reaching the sensor from the atmosphere Additive effect, reducing CONTRASTCorrection

Single band minimum (subtract minimum DN or minimum-1 or minimum-2)Dark object subtraction method

19

iirs

Atmospheric Haze


with hazewithout haze

iirs


Geometric Correction

The transformation of remotely sensedimages so that it has a scale andprojections of a map is called geometriccorrection.

A related technique, called registration, isthe fitting of the coordinate system of animage to that of second image of the samearea.

20

iirs


These errors can be divided into two classes

(a) those that can be corrected using data fromplatform ephemeris and knowledge of internalsensor distortion

(b) those that cant be corrected with acceptableaccuracy without a sufficient number of groundcontrol points (GCP).

Geometric Errors

iirs


Image Offset (skew) caused Image Offset (skew) caused by Earth Rotation Effects by Earth Rotation Effects

Earth-observing Sun-synchronous satellites arenormally in fixed orbits that collect a path (or swath)of imagery as the satellite makes its way from thenorth to the south in descending mode.

Meanwhile, the Earth below rotates on its axis fromwest to east making one complete revolution every24 hours. This interaction between the fixed orbitalpath of the remote sensing system and the Earthsrotation on its axis skews the geometry of theimagery collected

21

iirs


Earth Rotation EffectsEarth Rotation Effects

iirs


Dashed line indicate shape of distorted imageSolid line indicates restored image

22

iirs


Rectification is a process of geometrically correcting

an image so that it can be represented on a planar surface , conform to other images or conform to a map.

i.e it is the process by which geometry of an image is made planimetric.

It is necessary when accurate area ,distance and direction measurementsare required to be made from theimagery.

iirs


Rectification

It is achieved by transforming the data from one grid system into another grid system using a geometric transformation

In other words process of establishing mathematical relationship between the addresses of pixels in an image with corresponding coordinates of those pixels on another image or map or ground.

23

iirs


Difference in Geometry Shift

Scale

Rotation and

Skew

iirs


Image to Map Rectification Procedure (Steps)

Two basic operations must be performed to geometrically rectify a remotely sensed image to a map coordinate system:

Spatial Interpolation Intensity

Interpolation

24

iirs


Rectification Procedure (Steps)

1. Spatial Interpolation: The geometric relationship between input pixel location (row & column) and associated map co-ordinates of the same point (x,y) are identified.

This establishes the nature of the geometric co-ordinate transformation parameters that must be applied to rectify the original input image (x,y) to its proper position in the rectified output image (X,Y).

Involves selecting Ground Control Points(GCPS) and fitting polynomial equations using least squares technique

iirs


Ground Control Points (GCPs)

qA ground control point (GCP) is a location onthe surface of the Earth (e.g., a roadintersection) that can be identified on theimagery and located accurately on a map.qThere are two distinct sets of coordinates

associated with each GCP:qsource or image coordinates specified in i rows

and j columns, andqReference or map coordinates (e.g., x, y

measured in degrees of latitude and longitude,or meters in a Universal Transverse Mercatorprojection).

25

iirs


Ground Control Points (GCPs) cont..

qThe paired coordinates (i, j and x, y)from many GCPs can be modeled toderive geometric transformationcoefficients.qThese coefficients may be used to

geometrically rectify the remote sensordata to a standard datum and mapprojection

iirs


Ground Control Points (GCPs) Accurate GCPs are essential for accurate rectification Sufficiently large number of GCPs should be selected Well dispersed GCPs result in more reliable rectification GCPs for Large Scale Imagery

Road intersections, airport runways, towers buildings etc.

for small scale imagery larger features like Urban area or Geological features

can be used NOTE : landmarks that can vary (like lakes, other water

bodies, vegetation etc) should not be used. GCPs should be spread across image Requires a minimum number depending on the type of

transformation

26

iirs


Spatial Interpolation using coordinate transformation

Polynomial equations are used to convert the source file coordinates to rectified map coordinates.

Depending upon the distortions in the imagery, the number of GCPs used, their location relative to one other, complex polynomial equations are used.

The degree of complexity of the polynomial is expressed as ORDER of the polynomial.

The order is simply the highest exponent used in the polynomial

iirs


Mathematical TransformationsLinear Transformations/ Affine transformation/

first order transformationX = a0 + a1x + a2 yY = b0 + b1 x + b2 ywhere X , Y are the Rectified coordinates (output) x,y are the source coordinates (input) A first order transformation can change

Location in x and/or y Scale in x and/or y Skew in x and/or y Rotation

27

iirs


Polynomial transformation If the coefficients a0 ,a1,a2, b0, b1 and b2 are known

then, the above polynomial can be used to relate andpoint on map to its corresponding point on image andvice versa. Hence six coefficients are required for thistransformation (three for X and three for Y).

Requires Minimum THREE GCPs for solving theabove equation.

However before applying rectification to the entire setof the data, it is important to determine how well thesix coefficients derived from the least squareregression of the initial GCPs account for thegeometric distortion in the input image.

iirs


In this method, we check how good do selected points fit between the map and the Image?

To solve linear polynomials we first take three GCPs to compute the six coefficients. Its source coordinates in the original input image are say xi and yi. The position of the same points in reference map in degrees, feet or meters are say x,y

Accuracy of transformation

28

iirs


Accuracy of transformation

Now, if we input the map x,y values for the first GCP back into the linear polynomial equation with all the coefficients in the place, we would get the computed or retransformed x r and yrvalues , which are supposed to be location of this point in input image.

Ideally measured and computed values should be equal.

In reality this does not happen. There is discrepancy between the measured

and computed coordinates i.e we have to check the accuracy of transformation

iirs


Root Mean Square (RMS) error The method used involves computing Root

Mean Square Error (RMS error) for each of the ground control point

RMS error is the distance between the input (source or measured) location of a GCP and the retransformed (or computed) location for the same GCP.

RMS error is computed with a Euclidean Distance Equation.

29

iirs


RMS ERROR

Where xi and yi are the input source coordinates and xr and yr are the retransformed coordinates

iirs


RMS ERROR

RMS error is expressed as distance in the source coordinate system.

It is the distance in pixel widths An RMS error of 2 means that the

reference pixel is 2 pixels away from the retransformed pixel.

30

iirs


Acceptable RMS error or Tolerance of the error

Normally, a user specifies a certain amount (a threshold) of acceptable total RMS error.

The amount of RMS error that is tolerated can be thought of as a window around each source coordinate, inside which a retransformed coordinate is considered to be correct.

For example, if the threshold is 2, then the retransformed pixel can be 2 pixels away from the source pixel and still be considered accurate.

iirs


Acceptable RMS

31

iirs


Total RMS error

Where: Rx= X RMS error Ry= Y RMS error

T= total RMS error n= the number of GCPs i= GCP number XRi= the X residual for GCPi YRi= the Y residual for GCP i

From the residuals the Total RMS error, the X RMS error and the Y RMSerror can be computed as per the following formula

iirs


Acceptable RMS cont..

Acceptable RMS error depends upon the End use of the data The type of data being used, and The accuracy of the GCP and the ancillary

data.

Normally an RMS error of less than 1 per GCP and a total RMS error of less than half a pixel (0.5) is acceptable

32

iirs


Acceptable RMS cont..

After each computation of a transformation, the total RMS error reveals that a given set of control points exceeds the threshold then

Delete the GCP from the analysis that has greatest amount of individual error

Recompute the coefficients and the RMS error for all points

iirs


33

iirs


Grid Transformation At the end of this step we have established the

mathematical relationship between the coordinates of pixel and map coordinates or in other words we have fitted an empty grid over our reference map

iirs


Intensity Interpolation

Pixel brightness values must be determined.Unfortunately, there is no direct one-to-onerelationship between the movement of inputpixel values to output pixel locations. It will be shown that a pixel in the rectified output

image often requires a value from the input pixelgrid that does not fall neatly on a row-and-column coordinate.When this occurs, there must be some

mechanism for determining the brightness value(BV ) to be assigned to the output rectified pixel.This process is called intensity interpolation.

34

iirs


Intensity Interpolation or Image Resampling

Once an image is warped, how do you assign DNs to the new pixels?

Since the grid of pixels in the source image rarely matches the grid for the reference image, the pixels are resampled so that new data file values for the output file can be calculated.

This process involves the extraction of a brightness value from a location in the input image and its reallocation in the appropriate coordinate location in the rectified output image.

Types Input Driven Output Driven

iirs


Input Driven Resampling Movement of all pixels in the input image to

their new positions in the output image Disadvantages

Some output pixels are multiple assigned Some are not assigned creating Gaps in the output

image

35

iirs


Output Driven ResamplingvCreation of the new image by filling it pixelafter pixel and line after linevPixel indices in the output image aretransformed to position of corresponding pixelsin the input image

iirs


Resampling Techniques

Input Driven Output Driven

36

iirs


Resampling Techniques

vThis results in input line and columnsnumbers as real numbers ( and notintegers)vWhen this occurs, methods ofassigning Brightness values arevNearest NeighbourvBilinearvCubic

iirs


Nearest Neighbor The nearest neighbor approach uses the value of

the closest input pixel for the output pixel value. The pixel value occupying the closest image file

coordinate to the estimated coordinate will beused for the output pixel value in thegeoreferenced image.

37

iirs


iirs


Nearest NeighborADVANTAGES:Output values are the original input values. Other

methods of resampling tend to average surroundingvalues. This may be an important consideration whendiscriminating between vegetation types or locatingboundaries.Since original data are retained, this method is

recommended before classification.Easy to compute and therefore fastest to use.

DISADVANTAGES:Produces a choppy, "stair-stepped" effect. The image has

a rough appearance relative to the original unrectifieddata. Data values may be lost, while other values may be

duplicated.

38

iirs


Bilinear The bilinear interpolation approach uses the weighted

average of the nearest four pixels to the output pixel.

=

== 4

12

4

12

1

k k

k k

k

wt

D

DZ

BV

where Zk are the surrounding four data point values, and D2k are the distances squared from the point in question (x, y) to the these data points.

iirs


Bilinear

39

iirs


Bilinear

ADVANTAGES: Stair-step effect caused by the nearest

neighbor approach is reduced. Image looks smooth.

DISADVANTAGES: Alters original data and reduces contrast by

averaging neighboring values together. Is computationally more extensive than

nearest neighbor.

iirs


Cubic Convolution The cubic convolution approach uses the weighted

average of the nearest sixteen pixels to the output pixel. The output is similar to bilinear interpolation, but the smoothing effect caused by the averaging of surrounding input pixel values is more dramatic.

=

== 16

12

16

12

1

k k

k k

k

wt

D

DZ

BV

where Zk are the surrounding four data point values, and D2k are the distances squared from the point in question (x, y) to the these data points.

40

iirs


Cubic Convolution

iirs


Cubic Convolution

ADVANTAGES: Stair-step effect caused by the nearest

neighbor approach is reduced. Image looks smooth.

DISADVANTAGES: Alters original data and reduces contrast by

averaging neighboring values together. Is computationally more expensive than

nearest neighbor or bilinear interpolation.

41

iirs


Nearest Neighbour Bilinear Interpolation Cubic Convolution

iirs


Second Ground Control Point

42

iirs


iirs


43

iirs


iirs

Thank You

dip basics and rectification_2

Documents

multispectral digital

digital image andvice

digital analog imagesv

band number

energy records ahigher

forms of images v analog

brightness valuebv

brightness levels