comp 294-9 :: fall 2003 video as a datatype ketan mayer-patel
Post on 21-Dec-2015
213 views
TRANSCRIPT
Comp 294-9 :: Fall 2003
The mothers of all video...• NTSC - American standard
• PAL - European standard
• SECAM - French standard (ugly stepchild)
• For better or worse, this is where it all starts.
Comp 294-9 :: Fall 2003
Color Spaces• One luminance component
• Two chrominance components– Original TV was black and white.– Adding color had to be done in a compatible way.
• NTSC: YIQ
• PAL: YUV
• In general: YUV and YCrCb used as terms.
Comp 294-9 :: Fall 2003
YUV vs. RGB• Simple linear transform from one cube to
the other.
• Specific transform coefficients vary for specific systems, but a common one:– Y = 0.299 R + 0.587 G + 0.114 B– Cr = -0.147 R - 0.289 G + 0.437 B + 0.5– Cb = 0.615 R - 0.515 G - 0.100 B + 0.5
Comp 294-9 :: Fall 2003
YUV Challenges• YUV is like taking RGB cube, standing it
on a corner with (0,0,0) on bottom and (1,1,1) at top, rotating slightly, and taking the bounding box.
• What problems might this incur?– Some RGB colors are illegal.– A lot of YUV colors are illegal.
Comp 294-9 :: Fall 2003
A Note On Color• Previous view of color is EXTREMELY
simplified.– Color is complicated
• Frequency dependent response for contrast, lightness, etc.
– Gamma correction.`
Comp 294-9 :: Fall 2003
Scanning• Analog video signal is continuous voltage
signal that gets scanned along the screen.
• The electron stream controlled by two orthogonal sets of magnets.– Horizontal: Beam is moved from left to right
and then quickly back.– Vertically: Beam is moved from top to bottom
and then quickly back.
Comp 294-9 :: Fall 2003
Structure of Video• Blinn’s article.
– Macrostructure• Frequency peaks at frame rate.
– Microstructure• Frequency peaks at line rate
• Adding color is a good first example of source-aware channel coding.– What was the problem?
– What was the solution?
– Why does it work?
– What are its drawbacks (i.e., when does it fail)?
Comp 294-9 :: Fall 2003
Interlacing• Progressive = every scan line done in turn.
• Interlaced = every other scan line done.– Creates two fields: odd field and even field.– NTSC: 262.5 lines per field at 60 fields per sec.– PAL: 312.5 lines per field at 50 fields per sec.
• Fields are separated in time.
Comp 294-9 :: Fall 2003
Analog Bandwidths• True meaning of the bandwidth.
• Y, U, and V signals are all continuous along a scan line.– A bit of a hybrid signal: discrete vertically,
continuous horizontally.
• NTSC:– Y is 4.2 MHz wide, I is 2 MHz, Q is 1 MHz
• PAL:– Y is 6 MHz wide, U is 3 MHz, V is 3 MHz
Comp 294-9 :: Fall 2003
Corresponding Data Rates• How much data can you put in 6 MHz band?
– Depends on S/N ratio.– Depends on modulation scheme.
• Typical: 27 - 36 Mbs
• How many cable channels do you get?– If 50, then 1.3 - 1.8 Gbs coming into your home.
• The real question for multimedia is:– Why haven’t we found a better use for 1.3 Gbs
than continuous broadcast of Real World.
Comp 294-9 :: Fall 2003
Digital Video Frames• Almost always progressive
• 3 planes of pixel values (Y, U, and V)
• Pixel depth
• Geometry of each plane: width x height– Chrominance is generally subsampled.
• How the planes relate to each other.
• Frame rate.
Comp 294-9 :: Fall 2003
CCIR-601• Standard established for digitizing NTSC
and PAL signals.
DigitalComponent
AnalogCounterpart
CCIR-601Recommendation
Height Lines NTSC: 480PAL: 576
Width Continuousvoltage changesalong scan line.
Sample at 13.5 MHzResults in 702 pixelsRecommended 720
Pixel depth Voltage range. Gamma-corrected8-bit sample.
Chrominancesubsampling
Bandwidthdifferences
4:2:2, 4:2:0, 4:1:1
Comp 294-9 :: Fall 2003
Why 8-bits for chrominance?• What’s another way to cut chrominance
bandwidth in half?– Use 4-bits per pixel.
• Why won’t that work?– Need the dynamic range for color.
• But what about when Y is either small or large?– Don’t need the range, but lots of YUV combinations
that won’t ever be used.
Comp 294-9 :: Fall 2003
4:2:2• For every 4 luminance samples, take 2
chrominance samples from odd lines and 2 from even lines.
• Chrom. planes just as tall, half as wide.
• JPEG does this.
Comp 294-9 :: Fall 2003
4:2:0• 2 chrominance samples for every 4
luminance samples, odd lines only.
• Chrominance halved in both directions.
• MPEG generally does this.
Comp 294-9 :: Fall 2003
4:1:1• What should this be?
– 1 chrominance for every 4 luminance for both odd and even lines.
– And that is what it is.
• But sometimes used to refer to this:
Comp 294-9 :: Fall 2003
De-interlacing• Since analog video fields are separated in
time by 1/2 the frame rate, at least half of a digital frame is missing no matter where you sample from.
• Normal solution: linear interpolation of even fields to produce matching samples for odd fields.
• Even better: linear interpolation of both fields to produce matching samples and thus digital frame rate will equal field rate.
Comp 294-9 :: Fall 2003
Film Frame Rates• Film is a different beast altogether.
– Continuous both vertically and horizontally– Projection is simultaneous for all parts of the
picture. – Expensive medium. – Combination of all of this motivates 24 fps.
• Film to digital is easier than video to digital.– No interlacing, sample where you want to.
• Film to video is harder.
Comp 294-9 :: Fall 2003
3:2 Pulldown• Converts 24 frames to 60 fields.
1 2 3 4 5
1e 1o 2e 2o 3e 2o 3e 3o 4e 4o 5e 5o
Comp 294-9 :: Fall 2003
Overall sampling lesson• Can’t recover what you don’t have.
• Conversion between representations requires estimation of missing samples.
• Interpolation causes errors:– Spatially: at the edges.– Temporally: when moving.
Comp 294-9 :: Fall 2003
Common Digital Video Sizes• CCIR-601 720x480 4:2:2, 4:2:0
• SIF 360x240 4:2:0
• CIF 360x288 4:2:0
• 4:3 HDTV 1440x1152 4:2:2, 4:2:0
• 9:16 HDTV 1920x1152 4:2:2, 4:2:0
• 4CIF, 16CIF, QCIF