1

VideoVideo

Video consists of image Video consists of image framesframes captured from real captured from real motion and shown in successionmotion and shown in succession

Animation is similar except that the frames are Animation is similar except that the frames are synthesizedsynthesized

The The frame rateframe rate is the number of frames shown per is the number of frames shown per second (fps)second (fps)

One minute of uncompressed video requires One minute of uncompressed video requires significant storage: 1.6 GB to 1.85 GBsignificant storage: 1.6 GB to 1.85 GB

Compression can be accomplished on the recording Compression can be accomplished on the recording device or using a video capture carddevice or using a video capture card

The IEEE 1394 (firewire) interface standard is often The IEEE 1394 (firewire) interface standard is often used for video captureused for video capture

2

Analog VideoAnalog Video

An image on a TV is drawn row by row from An image on a TV is drawn row by row from left to right, starting at the top left of the left to right, starting at the top left of the screenscreen

After the image has been drawn, it moves After the image has been drawn, it moves back up and begins againback up and begins again

The The refresh raterefresh rate of a TV is the number of of a TV is the number of times it draws a new image per secondtimes it draws a new image per second

The refresh rate is measured in Hertz The refresh rate is measured in Hertz (basically, per second)(basically, per second)

3

Analog VideoAnalog Video

There are two main TV formats:There are two main TV formats:• NTSC - National Television System CommitteeNTSC - National Television System Committee• PAL – Phase Alternating LinePAL – Phase Alternating Line

NTSC is used in America and Japan, while NTSC is used in America and Japan, while PAL is used in Europe and AustraliaPAL is used in Europe and Australia

An NTSC TV has a refresh rate of 60 Hz, An NTSC TV has a refresh rate of 60 Hz, while a PAL TV has a refresh rate of 50 Hzwhile a PAL TV has a refresh rate of 50 Hz

Standard movie frame rate: 24 fpsStandard movie frame rate: 24 fps

4

Video CodecsVideo Codecs

An algorithm that shrinks the size of a An algorithm that shrinks the size of a movie to allow the movie to be played on a movie to allow the movie to be played on a computer or from a networkcomputer or from a network

codeccodec is short for is short for cocompressor / mpressor / decdecompressorompressor

Use lossy compressionUse lossy compression

Some video codecs are “software only” Some video codecs are “software only” while others require special hardwarewhile others require special hardware

5

Video CodecsVideo Codecs

Most codecs are Most codecs are block-orientedblock-oriented

Divide the frame into blocksDivide the frame into blocks

Sometimes you can see the block of pixels Sometimes you can see the block of pixels when a video playswhen a video plays

Codecs differ in how the blocks are encodedCodecs differ in how the blocks are encoded

6

Video ArchitecturesVideo Architectures

Handles and synchronizes video filesHandles and synchronizes video files

An architecture creates file formats with its An architecture creates file formats with its codecscodecs

Examples of Architectures:Examples of Architectures:• QuickTimeQuickTime• RealSystem G3RealSystem G3• Microsoft Windows MediaMicrosoft Windows Media• MPEGMPEG

The architecture you choose determines The architecture you choose determines which codecs you can usewhich codecs you can use

Not all codecs are available for every Not all codecs are available for every architecturearchitecture

7

Compression TechniquesCompression Techniques

Codecs compress video using two techniques:Codecs compress video using two techniques:• Temporal CompressionTemporal Compression• Spatial CompressionSpatial Compression

Temporal compressionTemporal compression eliminates information eliminates information that is not necessary for visual continuity over that is not necessary for visual continuity over timetime

Spacial compressionSpacial compression eliminates information that eliminates information that is not necessary for continuity over areais not necessary for continuity over area

Other compression can be accomplished by Other compression can be accomplished by reducing colors, frame rate, and audio qualityreducing colors, frame rate, and audio quality

8

Temporal CompressionTemporal Compression

Temporal compression looks for differences Temporal compression looks for differences between consecutive framesbetween consecutive frames

A A keyframekeyframe is a frame that is used as the basis is a frame that is used as the basis for differencesfor differences

Keyframes contain the entire imageKeyframes contain the entire image

A A delta framedelta frame (or difference frame) contains (or difference frame) contains only the areas that are different from one only the areas that are different from one frame to the nextframe to the next

keyframe, delta, delta, …, keyframe, delta, keyframe, delta, delta, …, keyframe, delta, delta, ...delta, ...

Several keyframes are chosen throughout the Several keyframes are chosen throughout the movie whenever significant changes occurmovie whenever significant changes occur

9

Spacial CompressionSpacial Compression

Generally focuses on individual framesGenerally focuses on individual frames

If an area is all one color, the area is defined If an area is all one color, the area is defined using geometry and the color stored onceusing geometry and the color stored once

Combines bitmap and raster graphics Combines bitmap and raster graphics conceptsconcepts

Temporal and spacial compression work in Temporal and spacial compression work in concertconcert

10

Asymmetric vs. SymmetricAsymmetric vs. Symmetric

A codec can be A codec can be asymmetricasymmetric::• Encoding process is slower than the decoding Encoding process is slower than the decoding

processprocess• Better suited for movies that are prerecorded and Better suited for movies that are prerecorded and

played backplayed back

Or Or symmetricsymmetric::• Encoding process takes about the same time as Encoding process takes about the same time as

the decoding processthe decoding process• Better suited for live broadcasts or video Better suited for live broadcasts or video

teleconferencingteleconferencing

11

Video CodecsVideo Codecs

Some popular video codecs:Some popular video codecs:• Sorenson VideoSorenson Video• CinepakCinepak• MPEG:MPEG:

– MPEG-1MPEG-1– MPEG-2MPEG-2– MPEG-4MPEG-4

• Real Video:Real Video:– RealVideo G2RealVideo G2– RealVideo 8RealVideo 8

• H.261 / H.263H.261 / H.263

12

Sorenson Video CodecSorenson Video Codec

Developed by Sorenson Vision, Inc (www.s-Developed by Sorenson Vision, Inc (www.s-vision.com)vision.com)• High compressionHigh compression• High-quality/low data ratesHigh-quality/low data rates• High-end processors NOT neededHigh-end processors NOT needed

Recommended for:Recommended for:• Web video (fast viewing rates)Web video (fast viewing rates)• CD-ROM video (fit more movies on a CD-ROM)CD-ROM video (fit more movies on a CD-ROM)

It is asymmetricIt is asymmetric

A hardware specific version is also availableA hardware specific version is also available

13

Sorenson Video CodecSorenson Video Codec

Sorenson is supported by the QuickTime 3 Sorenson is supported by the QuickTime 3 and 4 architecturesand 4 architectures

It uses:It uses:• variable encodingvariable encoding• vector quantizationvector quantization• motion compensationmotion compensation

14

Variable EncodingVariable Encoding

Adjusts the bit depth according to how Adjusts the bit depth according to how much action the scene containsmuch action the scene contains

Two-pass encoding techniqueTwo-pass encoding technique

Movie is analyzed to determine “easy” Movie is analyzed to determine “easy” and “difficult” sectionsand “difficult” sections

Movie is compressed intelligently:Movie is compressed intelligently:• Attempts to give each frame the optimum Attempts to give each frame the optimum

number of bytesnumber of bytes

15

Vector QuantizationVector Quantization

Image divided into segments (vectors)Image divided into segments (vectors)

The mean value of each segment is The mean value of each segment is subtracted from the individual pixels in the subtracted from the individual pixels in the segment (residual vector pattern)segment (residual vector pattern)

Each residual segment pattern is compared Each residual segment pattern is compared to a set of previously determined patterns to a set of previously determined patterns (stored in a (stored in a codebookcodebook))

The codebook entry pattern closest to the The codebook entry pattern closest to the residual vector pattern is determinedresidual vector pattern is determined

The binary address of the codebook entry is The binary address of the codebook entry is sent to receiver to be decodedsent to receiver to be decoded

16

Vector QuantizationVector Quantization

The decoder only needs the address of the The decoder only needs the address of the residual pattern and the mean value of the residual pattern and the mean value of the individual segmentsindividual segments

If codebook pattern entries are determined If codebook pattern entries are determined ahead of time according to test imagery, the ahead of time according to test imagery, the quality of the movie will be goodquality of the movie will be good

The number of codebook entries can be The number of codebook entries can be increased substantially without too much increased substantially without too much effect on the bandwidth neededeffect on the bandwidth needed

17

Motion CompensationMotion Compensation

From frame-to-frame, there is usually some From frame-to-frame, there is usually some translation of blocks of pixels due to motiontranslation of blocks of pixels due to motion

Successive frames in a motion sequence are Successive frames in a motion sequence are searched to determine the searched to determine the motion motion compensation vectorscompensation vectors

Motion compensation vectors indicate the Motion compensation vectors indicate the direction and distance of translation of block direction and distance of translation of block of pixels that has movedof pixels that has moved

The whole frame does not have to be The whole frame does not have to be translated - the vectors contain the translated - the vectors contain the information on how to reconstruct the new information on how to reconstruct the new frame from the previous frameframe from the previous frame

18

Sorenson ToolsSorenson Tools

Allows for a Allows for a media key media key so that the video can so that the video can be locked with a passwordbe locked with a password

Tries to determine when new scenes begin Tries to determine when new scenes begin and marks them as key frames (smart and marks them as key frames (smart screen change detection)screen change detection)

Allows the user to insert a custom Allows the user to insert a custom watermark on all frameswatermark on all frames

Allows the user to specify intervals at which Allows the user to specify intervals at which frames should be dropped to increase frames should be dropped to increase playback performanceplayback performance

19

CinepakCinepak

Codec maintained by Compression Techniques, Inc.Codec maintained by Compression Techniques, Inc.

Medium-quality CD-ROM videoMedium-quality CD-ROM video

Does not require a lot of processing powerDoes not require a lot of processing power

Is asymmetricIs asymmetric

Uses vector quantizationUses vector quantization

Cinepak video codec is supported by the following Cinepak video codec is supported by the following architectures:architectures:• Video for WindowsVideo for Windows• QuickTimeQuickTime

20

MPEGMPEG

Developed by Motion Picture Expert’s groupDeveloped by Motion Picture Expert’s group

MPEG is both a file format and a codec (a file MPEG is both a file format and a codec (a file format that employs compression algorithms)format that employs compression algorithms)

Usually requires special hardware for Usually requires special hardware for encodingencoding

Usually requires substantial CPU Usually requires substantial CPU power/special hardware for playbackpower/special hardware for playback

Mainly used for high-end desktop videoMainly used for high-end desktop video

MPEG is an open standardMPEG is an open standard

21

MPEGMPEG

MPEG is a family of codecs containing:MPEG is a family of codecs containing:• MPEG-1MPEG-1• MPEG-2 MPEG-2 • MPEG-4 (approved in 1999)MPEG-4 (approved in 1999)

MPEG supported by the following MPEG supported by the following architectures:architectures:• QuickTime 3QuickTime 3• DirectShow 6DirectShow 6• Video CDVideo CD• DVD VideoDVD Video• Windows MediaWindows Media

22

RealVideoRealVideo

RealVideo a family of codecs containing:RealVideo a family of codecs containing:• RealVideo 8RealVideo 8• RealVideo G2RealVideo G2

Usually used for streaming videoUsually used for streaming video

Both RealVideo codecs are scalableBoth RealVideo codecs are scalable• compensates if the connection bit rate is slower than the compensates if the connection bit rate is slower than the

bit rate used to encode the videobit rate used to encode the video• Allows for smoother playback on a wide range of Allows for smoother playback on a wide range of

machinesmachines

AsymmetricAsymmetric

Requires a high-end computer for optimal Requires a high-end computer for optimal encoding/decodingencoding/decoding

RealPlayer 8 supports variable encodingRealPlayer 8 supports variable encoding