digital multimedia, 2nd edition nigel chapman & jenny chapman chapter 7 this presentation ©...

Digital Multimedia, 2nd editionNigel Chapman & Jenny Chapman

Chapter 7

This presentation © 2004, MacAvon Media Productions

Video

© 2004, MacAvon Media Productions

7

• PAL uncompressed

• 768x576 pixels per frame

• x 3 bytes per pixel (24 bit colour)

• x 25 frames per second

• ≈ 31 MB per second

• ≈ 1.85 GB per minute

Data Size

191–192


7

• NTSC uncompressed

• 640x480 pixels per frame

• x 3 bytes per pixel (24 bit colour)

• x 30 frames per second (approx)

• ≈ 26 MB per second

• ≈ 1.6 GB per minute

Data Size

191–192


7

• In the camera – DV + Firewire

• In the computer – video capture card

• Digitization in camera (DV) means less noise

• Less noise allows better compression

Digitization

193–195


7

• Play back a video stream as it arrives over a network (like broadcast TV), instead of downloading an entire video clip and playing it from disk (like renting a DVD)

Streamed Video

197


7

• Start playing a downloaded clip as soon as enough of it has arrived

• Starts when the (estimated) time to download the rest is equal to the duration of the clip

HTTP Streaming

198


7 198


7

• Digital video devices must conform to standards

• Digital standards must maintain compatibility with older analogue standards for broadcast TV

Video Standards

199


7

• Required for TV, so encountered in captured footage

• Each frame is divided into two fields

• Field 1: odd lines; Field 2: even lines

• Fields are transmitted one after the other

• Frame is built out of the interlaced fields

Interlacing

200


7

• PAL (Phase Alternating Line)

• Western Europe, Australia & New Zealand, China,…

• NTSC (National Television Standards Committee)

• North America, Japan, Taiwan, parts of South America,…

Analogue Standards

199


7

• SECAM (Séquential Couleur avec Mémoire)

• France and former Soviet Union

• Standard only used for transmission

• Uses PAL cameras etc

Analogue Standards

199


7

• Frame has 625 lines, 576 are picture

• 25 frames (50 fields) per second

•NTSC• Frame has 525 lines, 480 are picture

• 29.97 frames (59.94 fields) per second

• (Often quoted as 30 frames per second)

PAL

202


7

• Digital video standard, properly called Rec. ITU-R BT.601

• 720 luminance samples (Y), 2x360 colour difference samples (B−Y and R−Y) per line

• PAL 720x576 pixels; NTSC 720x480 pixels

• Pixels are not square

CCIR 601

202–203


7

• Used in CCIR 601

• Twice as many Y samples as each of the colour difference samples

• Co-sited: same pixel is used for all three samples

• Reduces data rate to just over 20MB per second

4:2:2 sub-sampling

203


7

• Consumer format, also known as mini-DV

• DVCAM, DVPRO use different tape formats, but generate the same data stream

• 4:1:1 chrominance sub-sampling

• Data rate constant 25Mbits per second

• Compression ratio 5:1

DV

204


7

• PAL DV 4:2:0 chrominance sub-sampling

DV sub-sampling

210


7

• NTSC DV 4:1:1 chrominance sub-sampling

DV sub-sampling

210


7

• ISO/IEC Motion Picture Experts Group

• Series of standards including

• MPEG-1 intended for video CD

• MPEG-2 used in DVD and broadcast

• MPEG-4 for low bitrate multimedia

MPEG

204–206


7

• Profiles define subsets of the features of the data stream

• Levels define parameters such as frame size and data rate

• Each profile may be implemented at one or more levels

• Notation: profile@level, e.g. MP@ML

MPEG Profiles & Levels

204–205


7

• MPEG-2 Main Profile at Main Level (MP@ML) used for DVD video

• CCIR 601 scanning

• 4:2:0 chrominance sub-sampling

• 15 Mbits per second

• Most elaborate representation of MPEG-2 compressed data

MPEG-2 Profiles & Levels

205


7

• Designed to support a range of multimedia data at bit rates from 10kbps to >1.8Mbps

• Applications from mobile phones to HDTV

• Video codec becoming popular for Internet use, is incorporated in QuickTime, RealMedia and DivX

MPEG-4

205–206


7

• Visual Simple Profile (SP), suitable for low bandwidth streaming over Internet

• Visual Advanced Simple Profile (ASP) suitable for broadband streaming

• SP@L1 (Level 1 of Simple Profile), 64 kbps, 176x144 pixel frame

• ASP@L5, 8000 kbps, full CCIR 601 frame

MPEG-4 Profiles & Levels

205


7

• Spatial (intra-frame) compression

• Compress each frame in isolation, treating it as a bitmapped image

• Temporal (inter-frame) compression

• Compress sequences of frames by only storing differences between them

• Always some compression because of sub-sampling

Video Compression

206–208


7

• Image compression applied to each frame

• Can therefore be lossless or lossy, but lossless rarely produces sufficiently high compression ratios for volume of data

• Lossless compression implies a loss of quality if decompressed then recompressed

• Ideally, work with uncompressed video during post-production

Spatial Compression

207


7

• Key frames are spatially compressed only

• Key frames often regularly spaced (e.g. every 12 frames)

• Difference frames only store the differences between the frame and the preceding frame or most recent key frame

• Difference frames can be efficiently spatially compressed

Temporal Compression

207–208


7

• Purely spatial compression

• Apply JPEG to each frame

• Used by most analogue capture cards

• No standard, but MJPEG-A format widely supported

Motion JPEG

209–210


7

• Starts with chrominance sub-sampling of CCIR 601 frame

• Constant data rate 25Mbits per second

• Higher quality than MJPEG at same rate

• Apply DCT, quantization, run-length and Huffman coding on zig-zag sequence – like JPEG – to 8x8 blocks of pixels

DV Compression

210–211


7

• If little or no difference between fields (almost static frame), apply DCT to block containing alternate lines from odd and even fields

• If motion between fields, apply DCT to two 8x4 blocks (one from each field) separately, leading to more efficient compression of frames with motion

DV Compression

210–211


7

• Shuffling

• Construct video segments by taking 8x8 blocks from five different areas of the frame, to ‘average’ amount of detail

• Calculate coefficients for whole video segment, making more efficient use of available bytes

DV Compression

210–211


7

• Spatial compression based on quantization and coding of DCT coefficients

• Temporal compression based on motion compensation

• Record displacement of object plus changed pixels in area exposed by its movement

MPEG-1 Compression

211–212


7

• I-pictures purely intra-frame compressed

• P-pictures ‘predictive’

• Difference frames based on earlier I- or P-pictures

• B-pictures ‘bi-directionally predictive’

• Difference frames based on preceding and following I- or P-pictures

MPEG-1 Compression

212–213


7

• Group of Pictures (GOP)

• Repeating sequence of I-, P- and B-pictures

• Always begins with an I-picture

• Display order – frames in order they will be displayed

• Bitstream order – re-ordered so that every P- or B-picture comes after frames it depends on, allowing reconstruction of the complete frames

MPEG-1 Compression

213–214


7

• Source Input Format (SIF)

• 4:2:0 chroma sub-sampled

• 352x240 pixel frame

• MPEG-1 compressed SIF video at 30 frames per second has data rate of 1.86Mbits per second (CD video – 40mins of video at that rate)

• MPEG-1 can be scaled up to larger frames, but cannot handle interlacing

MPEG-1 Compression

214


7

• Standard defines an encoding for multimedia streams made up of different sorts of object – video, still images, animation, 3-D models…

• Higher profiles divide a scene into arbitrarily shaped video objects each one may be compressed and transmitted separately, scene is composed at the receiving end by combining them

• SP and ASP restricted to rectangular objects, usually complete frames

MPEG-4

215


7

• Refinement of MPEG-1 compression

• I-pictures compressed by quantizing and Huffman coding DCT coefficients

• Improved motion compensation leads to better quality than MPEG-1 at same bit rates

MPEG-4 Compression

215


7

• Simple Profile

• P-pictures only

• Efficient decompression, suitable for PDAs etc

• Advanced Simple Profile (ASP)

• B-pictures

• Global Motion Compensation

• Sub-pixel motion compensation

MPEG-4 Compression

215–216


7

• Cinepak – Longest established, high compression ratio, takes much longer to compress than to decompress

• Intel Indeo – Similar to Cinepak, but roughly 30% faster compression

• Sorenson – More recent, higher quality and better compression ratios than other two

• All three based on vector quantization

• Quality of all three inferior to MPEG-4

Older Codecs

216–219


7

• Divide each frame into small rectangular blocks (’vectors’)

• Code Book – collection of constant vectors representing typical patterns (edges, textures, flat colour,…)

• Compress by replacing each vector in image by index of vector from code book that most closely resembles it

Vector Quantization

216


7

• Making a constructed whole from a collection of parts

• Selection, trimming and organization of raw footage

• Apply transitions (e.g. dissolves) between shots

• Combination of picture with sound

• No changes made to the footage itself

Editing

223–230


7

• Changing or adding to the material

• Most changes are generalizations of image manipulation operations (e.g. colour correction, blurring and sharpening,…)

• Compositing – combining elements from different shots into a composite sequence

• Animating elements and combining animation with live action

Post-Production

230–236


7

• Compromises required to bring resource requirements of video within capabilities of delivery media (e.g. networks) and low-end machines

• Reduce frame size (e.g. downsample to quarter frame)

• Reduce frame rate (12fps is OK for smooth motion, flicker not a problem on computer)

• Reduce colour depth

Preparing for Delivery

236–237

digital multimedia, 2nd edition nigel chapman & jenny chapman chapter 7 this presentation ©...

Documents

chrominance subsampling210

video stream

camera dv firewirein

standardsdigital standards

entire video clip

picture25 frames

older analogue standards

digital video standard