cs 414 – multimedia systems design lecture 5 – digital video representation

CS 414 - Spring 2009

CS 414 – Multimedia Systems Design Lecture 5 – Digital Video Representation

Klara Nahrstedt

Spring 2009


Administrative

MP1 is out (January 28) Deadline of MP1 is February 9 (Monday) Demonstration of MP1 will be Monday,

February 9 at 5-7pm in 216 Siebel Center Sign-up sheet will be available in class on

February 9 (during class)

Color and Visual System Color refers to how we

perceive a narrow band of electromagnetic energy source, object, observer

Visual system transforms light energy into sensory experience of sight

Human Visual System

Eyes, optic nerve, parts of the brain

Transforms electromagnetic energy

Human Visual System

Image Formation cornea, sclera, pupil,

iris, lens, retina, fovea Transduction

retina, rods, and cones Processing

optic nerve, brain

Retina and Fovea

Retina has photosensitive receptors at back of eye

Fovea is small, dense region of receptors only cones (no rods) gives visual acuity

Outside fovea fewer receptors overall larger proportion of rods

Fovea

Retina

Transduction (Retina)

Transform light to neural impulses

Receptors signal bipolar cells

Bipolar cells signal ganglion cells

Axons in the ganglion cells form optic nerve

RodsBipolar cellsConesGanglion

Optic nerve

Rods vs Cones

Contain photo-pigment Respond to low energy Enhance sensitivity Concentrated in retina,

but outside of fovea One type, sensitive to

grayscale changes

Contain photo-pigment Respond to high energy Enhance perception Concentrated in fovea,

exist sparsely in retina Three types, sensitive to

different wavelengths

Cones Rods


Tri-stimulus Theory 3 types of cones (6 to 7 million of them)

Red = L cones, Green = M cones, Blue = S cones Ratio differentiates for each person E.g., Red (64%), Green (32%), rest S cones E.g., L(75.8%), M(20%), rest S cones E.g., L(50.6%), M(44.2%), rest S cones Source of information:

See ‘cone cell’ in wikipedia www.colorbasics.com/tristimulus/index.php

Each type most responsive to a narrow band red and green absorb most energy, blue the least

Light stimulates each set of cones differently, and the ratios produce sensation of color

Color Perception (Color Theory) Hue

distinguishes named colors, e.g., RGB

dominant wavelength of the light

Saturation Perceived intensity of a specific

color how far color is from a gray of

equal intensity Brightness (lightness)

perceived intensity


Hue Scale

Saturation

Original

lightness

Source: Wikipedia

Visual Perception: Resolution and Brightness Spatial Resolution (depends

on: ) Image size Viewing distance

Brightness Perception of brightness is higher

than perception of color Different perception of primary

colors Relative brightness:

green:red:blue=59%:30%:11%

B/W vs. Color

CS 414 - Spring 2009Source: wikipedia

Visual Perception: Temporal Resolution


Effects caused by inertia of human eye Perception of 16 frames/second as continuous

sequence Special Effect: Flicker

Temporal Resolution

FlickerPerceived if frame rate or refresh rate of

screen too low (<50Hz)Especially in large bright areas

Higher refresh rate requires:Higher scanning frequencyHigher bandwidth


Visual Perception Influence

Viewing distance Display ratio (width/height – 4/3 for

conventional TV) Number of details still visible Intensity (luminance)


Television History

1927, Hoover made a speech in Washington while viewers in NY could see, hear him

AT&T Bell Labs had the first “television” 18 fps, 2 x 3 inch

screen, 2500 pixels

Television Concepts Production (capture)

2D array of light energy to electrical signals signals must adhere to known, structured formats

Representation and Transmission popular formats include NTSC, PAL, SECAM

Re-construction CRT technology and raster scanning display issues (refresh rates, temporal resolution) relies on principles of human visual system


Video Representations

CompositeNTSC - 6MHz (4.2MHz video), 29.97 fpsPAL - 6-8MHz (4.2-6MHz video), 25 fps

ComponentMaintain separate signals for color

Color spacesRGB, YUV, YCRCB, YIQ


Color Coding: YUV PAL video standard

Y is luminance UV are chrominance

YUV from RGB Y = .299R + .587G + .114B

U = 0.492 (B - Y)V = 0.877 (R - Y)

U-V plane at Y=0.5


YCrCb Subset of YUV that scales

and shifts the chrominance values into range 0..1

Y = 0.299R + 0.587G + .114BCr = ((B-Y)/2) + 0.5Cb = ((R-Y)/1.6) + 0.5


YIQ NTSC standard

YIQ from RGB Y = .299R + .587G + .114B

I = .74 (R - Y) - .27 (B - Y)Q = 0.48 (R - Y) + 0.41 (B - Y)


YIQ with Y=0.5

NTSC Video

525 scan lines per frame; 29.97 fps33.37 msec/frame (1 second / 29.97 frames)scan line lasts 63.6 usec (33.37 msec / 525)aspect ratio of 4/3, gives 700 horizontal pixels

20 lines reserved for control information at the beginning of each fieldso only 485 lines of visible data


NTSC Video

Interlaced scan lines divide each frame into 2 fields, each of which is 262.5 linesphosphors in early TVs did not maintain

luminance long enough (caused flicker)scanning also interlaced; can cause visual

artifacts for high motion scenes


HDTV Digital Television Broadcast (DTB) System Twice as many horizontal and vertical

columns and lines as traditional TV Resolutions:

1920x1080 (1080p) – Standard HDTV Frame rate: options 50 or 60 frames per

second


Pixel Aspect Ratio


• Computer Graphics parameter• Mathematical ratio describing horizontal length of a pixel to its vertical height• Used mainly in digital video editing software to properly scale and render videoInto a new format


Resolution (WxH)

Pixels Aspect RatioPixel aspect ratio

(Standard 16:9)

Video Format Description

1024×768 786,432 16:9 4:3 720p/XGAUsed on PDP HDTV

1280×720 921,600 16:9 1:1 (square) 720p/WXGAUsed on Digital television,

1366×768 1,049,088 16:9Approx. 1:1 (square)

720p/WXGA - HDTV standard format

Used on LCD/PDP HDTV displays

1024×1080 1,105,920 16:9 15:8 1080pUsed on PDP displays HDTV

1280×1080 1,382,400 16:9 3:2 1080pUsed on PDP HDTV displays

1920×1080 2,073,600 16:9 1:1 (square)1080p - HDTV standard format

Used on all types of HDTV technologies

3840x2160 8,294,400 16:9 1:1 (square) 2160p Quad HDTV,

HDTV

Interlaced and/or progressive formatsConventional TCs – use interlaced formats Computer displays (LCDs) – use progressive

scanning MPEG-2 compressed streams In Europe (Germany) – MPEG-4

compressed streams


Aspect Ratio and Refresh Rate Aspect ratio

Conventional TV is 4:3 (1.33) HDTV is 16:9 (2.11) Cinema uses 1.85:1 or 2.35:1

Frame Rate NTSC is 60Hz interlaced (actually 59.94Hz) PAL/SECAM is 50Hz interlaced Cinema is 24Hz non-interlaced


SMPTE Time Codes

Society of Motion Picture and Television Engineers defines time codes for video HH:MM:SS:FF 01:12:59:16 represents number of pictures corresponding to

1hour, 12 minutes, 59 seconds, 16 frames If we consider 30 fps, then 59 seconds represent 59*30

frames, 12 minutes represent 12*60*30 frames and 1 hour represents 1*60*60*30 frames.

For NTSC, SMPTE uses a 30 drop frame code increment as if using 30 fps, when really NTSC has only 29.97fps defines rules to remove the difference error


Take Home Exercise

Given a SMPTE time stamp, convert it back to the original frame numbere.g., 00:01:00:10


Summary

Digitization of Video SignalsComposite CodingComponent Coding

Digital Television (DTV)DVB (Digital Video Broadcast)

Satellite connections, CATV networks – best suited for DTV

DVB-S – for satellites (also DVB-S2) DVB-C – for CATV


cs 414 – multimedia systems design lecture 5 – digital video representation

Documents

visual systemcolor

temporal resolution

s cones ratio

set of cones

human visual systemeyes

rest s conese

brightnessspatial resolution

light energy