csc 8610 & 5930 multimedia technology lecture 2 digital image representation

CSC 8610 & 5930Multimedia Technology

Lecture 2

Digital Image Representation

2Chapter 1

Today in class (1/30)

6:15 Recap, Reminders 6:25 Lecture – Digital Image Representation 7:30 Break 7:40 Workshop 1 discussion & presentations 8:20 Workshop 2 – Digital Image Exploration 9:00 Wrap

3

ANALOG VS. DIGITAL

4

Analog InformationExamples: time, weight temperature line length width and length of a sheet of paper sound loudness light brightness color saturation and hue Continuous information An infinite number of divisions exist between any two

measurements

5

What is the length of the pencil?

6

What is the length of the pencil?

7

What is the temperature?

8

Analog Thermometer vs. Digital Thermometer

digital thermometer

analog thermometer

9

Analog vs. Digital Analog information

– continuous– made up of infinite number of data points

Digital data– discrete

10

Discrete DataExamples: number of persons

There is no in-between one person and two persons.

choices in multiple-choice questionsThere is no in-between choice A and choice B.

11

Analog vs. DigitalTherometers and Scales What are the limitations of these analog and digital

devices?

What are the advantages of these analog and digital devices?

12

Analog vs. Digital Sight and sound we peceive in our natural world are

analog information--continuous and infinite number of points between any two points.

Computers handle discrete digital data. In addition, the amount of data has to be finite.

Sight and sound must be converted into finite discrete digital data in order for the computer to handle.

13

MONITORING A PUPPY'S WEIGHT IN HIS FIRST YEAR

14

Suppose you use an analog scale to weigh the puppy

15

Now, what is the weight you would note down for this puppy?

16

See the problem in picking a number to represent an analog measurement?

17

Number of Decimal Places In recording the weight, you must decide the number of

decimal places to use.

This determines the precision or exactness of the measurement.

How many will give an exact measurement? How many is enough? How many is too many?

18

Using More Decimal Places• Pros :

– increase the precision in general(But how many is meaningful?)

– Will allow finer distinction between values(will explain in the next slide)

• Cons:– Require more paper and paperwork.– Take longer to read through and interpret the numbers.

19

Distinction Between ValuesWith one decimal place:

– You can have 10 different values between say 2 and 3:2.1, 2.2, ...3.0

• You can distinct between 2.5 and 2.8.• But 2.5 and 2.8 would have been rounded to the same

value of 3 the values do not allow decimal places.

20

Distinction Between ValuesSuppose the allowable weight read outs are

these 10 levels:

0, 5, 10, 15, 20, 25, 30, 35, 40, 45

Then,

2 pounds: rounded to 0 pound

3 pounds: rounded to 5 pounds

The difference between 2 and 3 pounds is 1 pound.

But now, it become 5 pounds if we use these levels.

21

How many measurements to make?

A. once a yearB. once a monthC. every two weeksD. every weekE. every dayF. every hourG. every minuteH. every second

Now, how often would you weigh the puppy to produce a "good" monitoring of his weight over his first year?

22

What are your considerations in deciding how often to weigh the puppy?

23

Considerations• What happens if you weigh the puppy not often enough?

• What happens if you weigh the puppy too often?

• Is there one right answer?

• Will you use the same weighing schedule to monitor the weight of an adult dog?

24

DIGITIZATION:SAMPLING AND QUANTIZATION

Back to the Computer

25

Digitization To convert analog information into digital data that

computers can handle

2-step process:1. sampling

2. quantization

26

Sampling Analogous to weighing and recording the puppy's weight

During the sampling step, you need to set a sampling rate.

Sampling rate: how often you take a data

27

Suppose this is the true timeline of the puppy's first-year growth

28

Suppose you weigh the puppy once a month

29

You get these data points

30

You then interpolate the points

31

You would miss the changes that occur during the first month

32

But the rest matches with the true growth pretty well

33

What about weighing the puppy once a week?

34

You get these data points

35

The data is catching the changes occurring in the first month better

36

But is it exactly?

37

Now for the rest of the year, the data points seem too many

38

Sampling Rate

Weighing Puppy Scenario

Digitization

high(i.e. taking data often)

Pros: can catch more weight changes

Cons: produce more paperworkand thus take longer to read through all the data

Pros: can capture details (e.g. some changes of color within a small region in a picture or amplitude changes in sound within a short period of time)

Cons: produce larger file and thus take longer to process

low(i.e. taking data infrequently)

Pros: less paperwork and thus take shorter time to read through all the data

Cons: may miss weight changes

Pros: produce smaller file and thus take shorter time to process

Cons: may miss details (e.g. color changes in a picture or changes in sound)

39

Quantization• Analogous to rounding the weight to fix number of digits

in the weighing puppy scenario

• During the quantization step, you need to set bit depth.

• Bit depth refers to the number of allowable levels you map (or round) the values to.

40

Example: 10 levels of weightFor 10 discrete levels, you may have the 10 allowable

values as 2.0, 2.1, 2.2, 2.3, 2.4, 2.5, 2.6, 2.7, 2.8, and 2.9 0, 5, 10, 15, 20, 25, 30, 35, 40, 45 2, 4, 6, 8, 10, 12, 14, 16, 18, 20 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 ... and so forth

41

Suppose you choose 2.0, 2.1, ..., 2.9For 2.0, 2.1, 2.2, 2.3, 2.4, 2.5, 2.6, 2.7, 2.8, 2.9 Any weight data below 2.0 will be recorded as 2.0.

Any weight data higher than 2.9 will be capped at 2.9.

It works well if the puppy's weight falls in this range. But it does not seem to be the case.

42

Suppose you choose 0, 5,..., 45For 0, 5, 10, 15, 20, 25, 30, 35, 40, 45 A weight of 2 pounds would be rounded to 0 and a weight of

3 pounds to 5.

Cons: For example, the difference between 2 and 3 pounds is altered after they are mapped to the allowable value on this 10-level scale. The difference becomes 5 pounds not 1 pound.

Pros:Wider range.

Again, it works well if the puppy's weight falls in this range.

43

Well, what if we choose this:

2.0, 2.1, 2.2, ..., 44.8, 44.9, 45.0

44

Suppose you choose 2.0, 2.1,...,44.9, 45.0

You have increased the number of levels from 10 to 431.

Pros:– Increase precision compared to using

0, 5, 10, 15, 20, 25, 30, 35, 40, 45– Increase range compared to using

2.0, 2.1, 2.2, 2.3, 2.4, 2.5, 2.6, 2.7, 2.8, 2.9

45

Increase Number of Allowable Levels There does not seem to be any cons in the weighing

puppy scenario.

However, for digitization, increasing the number of allowable levels (i.e. increasing bit depth) will increase the file size.

46

Sampling and QuantizationDigitizing media involves sampling and quantization

regardless of the type of media:– images– video– audio

47

Overview of how sampling rate and bit depth affect digital media file quality

Sampling rate is related to:

Bit depth is related to:

digital images image resolution, or number of pixels

number of allowable colors in an image

digital video number of pixels in the video, frame rate

number of allowable colors

digital audio sampling rate of the audio (it limits how high the pitch of the audio can be captured)

number of allowable levels of amplitude

Details will be covered in chapters for each media type.

48

BITMAPS & DIGITIZATION

49

Pegboard Analogy

A 10 holes 10 holes pegboard

50

Pegboard Analogy

Suppose you want to copy this music note graphic on the pegboard.

51

Pegboard Analogy

Place one peg.

52

Pegboard Analogy

2 pegs.

53

Pegboard Analogy

3 pegs.

54

Pegboard Analogy

Suppose we only put peg in a hole if more than half of its area is covered by the musical note graphic.

55

Pegboard Analogy

Now remove the musical note overlay.

56

Pegboard Analogy

Details are lost because the gridis too coarse for this musical note.

57

How would you improve the details of the musical notes on the pegboard?

58

Using a pegboard with more holes

59

Using a pegboard with more holes

Now it looks closer to the original musical note.

60

Pixels Each peg hole on the pegboard is a sample point. The sample points are discrete.

In digital imaging, each of these discrete sample points is called a picture element, or pixel for short.

61

Pixel Dimensions Refer to an image’s width and height in pixels In the pegboard analogy, the dimension of this pegboard

would be 10 holes 10 holes.

62

Sampling Step

63

Let's look at the sampling step of digitizing a natural scene as if we are taking a digital photo of a natural scene.

64

A natural scene

Look up and let your eyes fall on the scene in front of you. Draw an

imaginary rectangle around what you see. This is your “viewfinder.”

Imagine that you are going to capture this view on a pegboard.

65

Sample into a grid of 25 20 discrete samplesSuppose you are going to sample the scene you see in the "viewfinder" into a pegboard with 25 20 holes.

66

One color for each peg hole.

Each peg hole takes only one peg. Suppose each peg has one solid color. Suppose the color of each of these discrete samples is determined by averaging the corresponding area.

67

This sampled image looks blocky. Details are lost because the grid is too coarse for this image.

68

For 25 20 sample points, it means you get a digitized image of 25 20 pixels.

69

Let's try a different grid size.

70

Sample into a grid of 100 80 discrete samples

Suppose you are going to sample the scene you see in the "viewfinder" into a pegboard with 100 80 holes.

71

Again, one color for each peg hole.

72

For 100 80 sample points, it means you get a digitized image of 100 80 pixels.

73

Sampling Rate Refers to how frequent you take a sample For an image, sampling frequency refers to how close

neighboring samples are in a 2-D image plane.

74

Sampling Rate For example, when we change the grid from 25 20 to

100 80, we say that we increase the sampling rate.– You are sampling more frequently within the same spatial distance.

75

Resolution In digital imaging, increasing the sampling rate is

equivalent to increasing the image resolution.

76

Consequences of Higher Resolution

With higher resolution, You have more sample points (pixels) to represent the

same scene, i.e., the pixel dimensions of the captured image are increased.

The file size of the digitized image is larger. You gain more detail from the original scene.

77

Resolution of Digital Photos Note that 25 20 and 100 80 pixels are by no means

realistic pixel dimensions in digital photography.

They are only for illustration purposes here. Most digital cameras can capture images in the range of thousand pixels in each dimension—for example, 3000 pixels 2000 pixels.

78

A Pixel is not a Square Block A pixel is a sample point. It does not really have a physical dimension associated

with it.

79

A Pixel is not a Square Block When you zoom in on a digital image in an image

editing program, you often see the pixels represented as little square blocks.

This is simply an on-screen representation of a sample point of an digitized image.

80

COLORS

81

Problems A natural image is colored in continuous tones, and thus

it theoretically has an infinite number of colors.

The discrete and finite language of the computer restricts the reproduction of an infinite number of colors and shades.

82

Quantization Step To encode an infinite number of colors and shades with

a finite list.

Quantizing the sampled image involves mapping the color of each pixel to a discrete and precise value.

83

Quantization Step First, you need to consider how many possible colors

you want to use in the image.

To illustrate this process, let’s return to the example of the 100 80 sampled image.

84

The sampled 100 80 image

85

Say, we want to map the color of each sample points into one of these four colors:

Mapping colors

86

Quantized with 4 Colors

87

Quantized with 8 Colors

88

Consequences of Quantization Reduce the number of allowed colors in the image.

When we reduce the colors, different colors from the original may bemapped to the same color on the palette. This causes the loss of the image fidelity and details.

The details that rely on the subtle color differences are lost during quantization.

89

The area outlined in red is made up of many different green colors.

The same area in the 4-color image now has only one color.

90

Bit Depth The number of colors used for quantization is related to

the color depth or bit depth of the digital image.

A bit depth of n allows 2n different colors. Examples:– A 2-bit digital image allows 22 (i.e., 4) colors in the image.– An 8-bit digital image allows 28 (i.e., 256) colors in the image.

The most common bit depth is 24. A 24-bit image allows 224 (i.e., 16,777,216) colors.

91

Increasing colors

It depends, and in most cases, can be yes.

The number of colors or the bit depth is not the only determining factor for image fidelity in quantizing an image.

The choice of colors for the quantization also plays an important role in the reproduction of an image.

Will increasing the number of colors in the palette improve the image fidelity?

92

Quantized with 8 Different Colors

93

Effect of Bit Depth on File Size Higher bit depth means more bits to represent a color.

Thus, an image with a higher bit depth has a larger file size than the same image with a lower bit depth.

94

VECTOR GRAPHICS

95

Vector Graphics Examples: graphics created in

– Adobe Flash– Adobe Illustrator

96

Vector Graphics Characteristics Generated mathematically,

i.e. instructionsnot pixel-based

Resolution independent

97

Vector Graphics Instruction Example A simple postscript example:

%!

newpath

200 200 moveto

300 200 lineto

stroke

showpage

98

Bitmap Images vs. Vector Graphics

An analogy:Driving Directions:

A visual map vs. a written instruction

Bitmap ~~> map

Vector Graphics ~~> a written instruction

99

Bitmap Images vs. Vector GraphicsAn analogy:

Driving Directions:A visual Triptik map vs. a written instruction

Which one takes up more storage space?Triptik map ~~> bitmap

Which one takes you more time to translate the direction into a mental image?written instruction ~~> vector graphics (take more computation to display on computer because it is mathematically generated)

100

Bitmap Images vs. Vector Graphics

An analogy:

Driving Directions:

A visual Triptik map vs. a written instruction

If you are going to draw out a map based on a written instruction, how big can you draw?

as big as you physically can ~~> the instruction is resolution independent

101

Bitmap Images vs. Vector Graphics

1111111111111111111111011111101111110111111011111101111111111111

%!newpath2 1 moveto6 5 linetostrokeshowpage

vector graphic bitmap image

102

Vector Graphics

%!newpath2 1 moveto6 5 linetostrokeshowpage

vector graphic

The unit is arbituary, i.e. when you print out an image, you may set one unit as an inch or a foot.

This means vector graphic is resolution independent.

103

Printing Bitmap Images

bitmap image

print bigger print smaller

have the same amount of data, i.e. same level of details

104

Printing Vector Graphics

vector graphics

print bigger print smaller

105

Bitmap Images vs. Vector Graphics Example

(a) Vector graphics:A line defined by two end points.

(b) Vector graphics: The same line is stroked with a certain width.

(c) & (d) The line is converted to a bitmap.

106

Curve Drawing in Vector Graphics Programs

defined by a set of points;we call them anchor points

the direction handles or tangent handles of a point controls the tangent at that point on the curve

107

Rasterizing Vector Graphics Raster means convert vector graphics into pixel-based

images.

Most vector graphics programs let you rasterize vector graphics.

Need to specify a resolution for rasterizing, that is, how coarse or how fine the sampling.

108

ALIASINGBlurriness, Blockiness, Moire Patterns, Jagged Edges

109

Aliasing

The rasterized image will appear jagged.

This jagged effect is a form of aliasing caused by undersampling (which means insufficient sampling rate.) Recall the musical note on a pegboard example.

Original vector graphics

Rastered vector graphics without anti-aliasing

110

Anti-aliasing Techniques

To soften the jaggedness by coloring the pixels with intermediary shades in the areas where the sharp color changes occur.

Original vector graphics

Rastered vector graphics without anti-aliasing

Rastered vector graphics with anti-aliasing

111

FREQUENCY IN DIGITAL IMAGES

112

Function over spatial domain Any image can be represented as a function Frequency – the rate at which color values change over

the space the image occupies

113

DISCRETE COSINE TRANSFORM

114

Discrete Cosine Transform Transforming the image data (function) from the spatial

domain to the frequency domain We won’t explore the mathematics Awareness of functions and transform is goal

115

COLOR MODELS

116

Color Models Used to describe colors numerically, usually in terms of

varying amounts of primary colors.

Common color models:– RGB– CMYK– HSB– CIE and their variants.

117

RGB Color Model Primary colors:

– red– green– blue

Additive Color System

118

Additive Color System

119

Additive Color System of RGB Full intensities of red + green + blue = white

Full intensities of red + green = yellow

Full intensities of green + blue = cyan

Full intensities of red + blue = magenta

Zero intensities of red, green, and blue = black

Same intensities of red, green, and blue = some kind of gray

120

Color Monitors

From a standard CRT monitor screen

121

Color Monitors

From a SONY Trinitron monitor screen

122

Color Monitors

From a LCD screen

123

RGB Color Model

depicted graphically as a cube defined by three axes in 3-D space

The maximum value on eachaxis is normalized to 1.

124

RGB Color Model

x-axis: red values y-axis: green values z-axis: blue values

125

RGB Color Model

The coordinates within the color cube represent the relative intensities of red, green, and blue colors.

126

RGB Color Model

origin (0,0,0): black

127

RGB Color Model

(1,0,0): full intensity of red

128

RGB Color Model

(0,1,0): full intensity of green

129

RGB Color Model

(0,0,1): full intensity of blue

130

RGB Color Model

(1,1,1): white

131

RGB Color Model

(1,1,0): full red + full green = full yellow

132

RGB Color Model

(1,0,1): full red + full blue = full magenta

133

RGB Color Model

(0,1,1): full green+ full blue = full cyan

134

Color Picker

So where is the color whose RGB values are (150, 200, 100) in the RGB cube?

135

Correlating RGB Color Cube with Color Picker

This is a 2-D slice containing all the colors with red = 150.

136

Correlating RGB Color Cube with Color Picker

This is a 2-D slice containing all the colors with green = 200.

137

Correlating RGB Color Cube with Color Picker

This is a 2-D slice containing all the colors with blue = 100.

138

Correlating RGB Color Cube with Color Picker

The color (150, 200, 100) is located in the 3-D space of the RGB color cube.It is at the intersection of the three 2-D slices.

139

CMYK Color Model Primary colors:

– cyan– magenta– yellow– black

Subtractive Color System

140

Subtractive Color System of CMY

141

Subtractive Color System of CMY Full intensities of cyan + magenta + yellow = black

(theoretically, but in practice with inks, it is not full black)

Full intensities of cyan + magenta = blue

Full intensities of cyan + yellow = green

Full intensities of magenta + yellow = red

Zero intensities of cyan, magenta, and yellow = white

142

HSB Color Model Hue:

– basic color– 0o to 360o : the location on a color wheel– in the order of colors in a rainbow

Saturation:– purity of the color– how far away from the neutral gray of the same brightness

Brightness

143

HSB Color Model

144

HSB Color Model

145

HSB Color Model

A slice of the color wheel from the HSB model

146

HSB Color Model

Matches well with the way humans intuitively think about colors

For example, how would you describe this color?

• Would you think of it in terms of how much red, green, and blue?

• Would you first think of it as a yellowish color and then figure out the lightness and how washed out the color is?

147

Problems with RGB and CMYK Color Space

Do not encompass all the colors human can see

148

CIE XYZ Color Model Primaries:

– X– Y– Z

Primaries are not physical colors

Its color space encompasses all the colors human can see.

Normally not used in digital image editing because monitors and printers cannot reproduce all the colors in the CIE XYZ color space anyway.

149

Color Gamuts

Refers to the range of colors of a specific system or device can produce or capture

150

Color Gamuts

a) Colors that human can see

b) RGB color gamut of typical CRT monitors

c) CMYK color gamut of typical inkjet printers

151

Color Gamuts

Monitors and inkjet printers cannot reproduce all the colors that human can see

Some of the colors that monitors can reproduce cannot be reproduced by inkjet printers. Most of these colors lie at the corners of the color gamut of the monitor, which means these are highly saturated colors.

152

Identifying Out-of-Gamut Colors in Images

Out-of-gamut colors will not be printed correctly.

In digital image editing programs such as Adobe Photoshop, you can tell whether a color is out of gamut based on your CMYK setting.

Click on the icon to use the closest color in gamut for printing.

153

Identifying Out-of-Gamut Colors in Images

Out-of-gamut colors will not be printed correctly.

In digital image editing programs such as Adobe Photoshop, you can tell whether a color is out of gamut based on your CMYK setting.

154

Identifying Out-of-Gamut Colors in Images

155

Identifying Out-of-Gamut Colors in Images

Click on the icon to replace the out-of-gamut color with the closest color in gamut for printing.

156

Identifying Out-of-Gamut Colors in Images

The out-of-gamut color is replaced with the closest color in gamut for printing.

157

Difficulties in Reproducing Colors in Digital Images

Digital devices cannot produce all of the colors visible to human

Difficulties exist in reproducing color across devices– different devices have different color gamuts– additive color system for screen display vs. subtractive color system for

printing

158

Indexed Color

159

Indexed Color