Multimedia – Digital Video

Dr. Lina A. NimriLebanese UniversityFaculty of Economic Sciences and Business Administration 1st branch

What is a video?• A video is a succession of images presented

at a certain rate. • The human eye is able to distinguish

approximately 20 images per second. ▫Thus, when more than 20 images are displayed

per second, it is possible to mislead the eye and create the illusion of an animated image.

• The fluidity of a video is characterized by the number of images per second (frame rate), expressed in FPS (Frames per second).

• In addition, multimedia video is usually accompanied by sound, i.e. audio data.

Digital and Analogue Video

•“Animated images” are classified into several families: ▫Cinema, which consists in storing the

succession of negative images on a film▫Analogue video, which represents

information as a continuous flow of analogue data: PAL , NTSC , SECAM

▫Digital video, which consists in coding the video in a succession of digital images.

PAL/SECAM

•The PAL/SECAM format (Phase Alternating Line/Sequential Colour with Memory),▫used in Europe for Hertzian television, ▫makes it possible to code videos on

625 rows (only 576 are displayed because 8% of the rows are used for synchronization), at a rate of 25 images per second with a 4:3 format (i.e. with a 4/3 width/height ratio).

PAL/SECAM• Problem: at 25 images per second, many people

perceive a beat in the image.▫ it was not possible to send more information due

to band-width limitations• Solution: interlace the images

▫ to send the even rows initially, then the odd rows▫“field” indicates the “half-image” formed either by

the even rows, or by the odd rows▫The whole transmission consisting of two fields is

called the interlaced screen, thus PAL/SECAM TV displays 50 fields per second

▫ If no interlacing the term progressive screen is used

PAL/SECAM

NTSC•The NTSC standard (National Television

Standards Committee), used in the United States and Japan,

•uses a system of 525 interlaced rows at 30 images/second (i.e. at a frequency of 60Hz).

•As in the case of the PAL/SECAM, 8% of the rows are used to synchronize the receiver.

•Thus, since the NTSC displays a 4:3 image format, the resolution actually displayed is of 640x480.

Digital video• Digital video consists in showing a succession

of digital images.▫Since these digital images are displayed at a

certain rate, it is possible to know the video display rate, i.e. the number of bytes displayed per sec.

• The necessary rate to display a video is equal to: ▫ size of the image x number of images per second.

(bytes/sec)

• Example:▫Consider a true color image (24 bits) with a

definition of 640X480 pixels.▫ In order to correctly display a video with this

definition, it is necessary to display at least 30 images per second: at a rate equal to: 900 KB * 30 = 27 MB/s

Subsampling

•Since the eye is not very sensitive to chrominance variations, the technique known as chroma subsampling (also called decimation), consists in removing chrominance information from a group of 4x4 pixels.

Video compression (codec)

•A non-compressed video image occupies approximately 1 MB.

•In order to obtain a fluid video, it is necessary to have a frequency of at least 25 or 30 images per second, ▫produces a data flow of approximately 30

MB/s, that is, more than 1.5 GB per minute. ▫storage spaces problem▫Use CoDec algorithms (for COmpression /

DECompression).

Video compression (codec)

•Need to compress the signal ▫Spatial redundancy (uniform fragments within the same screen)

▫Temporal redundancy (do not change in time)

▫Limitations of the eye

M-JPEG

•The principle of Motion JPEG (written MJPEG or M-JPEG, do not to confuse it with MPEG) consists in successively applying the JPEG compression algorithm to the various images of a video sequence.

•Since M-JPEG codes each image of the sequence separately it is possible to randomly access any part of a video. ▫Thus, its flow rate of 8 to 10 Mbps makes it

usable in digital assembly studios.

MPEG coding

• The MPEG (Moving Pictures Experts Group) Group was established in 1988 with the aim of developing international standards for compression, decompression, processing and coding of animated images and audio data.

• temporal redundancy– In most video sequences, most of the scenes are fixed or change very little▫When only the lips of the actor move, it is almost

only the pixels of the mouth that will be modified from one image to another; it is thus sufficient to just describe the change from one image to another (MPEG).

MPEG coding

•3 consecutive frames •Little change between successive images •Ideas

▫Differential encoding / previous image (ex. background)

▫Coding of movement of a block

Synchronization of audio and video flow in MPEG-1

MPEG coding

MPEG• There are several MPEG standards:

▫ MPEG-1: allows the storage of videos at a rate of 1.5 Mbps in a quality close to that of VHS cassettes on a CD (VCD)

▫ MPEG-2: offers a high quality at a rate that may go up to 40 Mbps (used for DVD videos)

▫ MPEG-4: allows multimedia coding of data in the form of digital objects, in order to achieve greater interactivity, which makes it suitable for the Web

▫ MPEG-7: provides a standard representation of audio and video to allow the search for information in such data flows. Also known as Multimedia Content Description Interface

▫ MPEG-21: under development, it is to provide a framework for all digital actors (producers, consumers,…) in order to standardize the management of these contents, as well as of the access rights, the copyrights,..

MPEG-1•Represents each image as a set of 16 X 16

blocks.•It makes it possible to obtain a resolution of:

▫352x240 at 30 images per second in NTSC ▫352x288 at 25 images per second in PAL/SECAM

•It makes it possible to achieve rates of 1.2 Mbps

•MPEG-1 allows videos to be encoded using several techniques: ▫Intra-coded frames (I frames) ▫Predictive coded frames (P frames) ▫Bi-directionally predictive coded frames (B

frames) ▫DC Coded frames (D frames)

MPEG-1: I framesIntra-coded frames•These images are coded only by using JPEG

coding, without worrying about images which surround them.

•Such images are necessary in a MPEG video because it is these that ensure image cohesion (since the others images are described relative to their surrounding images); ▫they are particularly useful for video streams

which can be tuned in at will at any time (television), and are essential in the event of any error in the reception.

▫There is one or two of these per second in a MPEG

MPEG-1: P framesPredictive coded frames • These images are defined by their difference

relative to the preceding image. • The encoder seeks the differences of the image

compared to the preceding one and defines blocks, called macroblocks (16x16 pixels) which will be superimposed on the preceding image.

• The algorithm compares both images block per block and starting from a certain difference threshold, it considers the preceding image area to be different from that of the image in progress and applies a JPEG compression to it.

• the search for the macroblocks determines the speed of the encoding: the more the algorithm seeks “good” blocks, the more time it wastes

• Compared to I frames, P frames require the preceding image to always be in memory.

MPEG-1: B frames

Bi-directionally predictive coded frames

• Like P frames, B frames work is based on differences relative to a reference image, except that in the case of B frames this difference can be carried out either on the preceding one (as in the case of P frames) or on the following one, ▫ this allows a better compression, ▫but induces a delay (since the following image

needs to be known) ▫and makes it necessary to keep three images in

the memory (the preceding one, the current one and the following one).

MPEG-1: D frames

DC Coded frames•These images offer a very low resolution

quality• but allow a very fast decompression,

▫which is particularly useful during fast forward viewing because “normal” decoding would require too many processor resources.

MPEG Optimization•In order to optimize MPEG coding, in

practice the image sequences are coded according to a succession of I, B, and P images (D being, as was mentioned above, reserved for fast forward viewing), ▫the order of these having been determined

experimentally.•The sequence known as GOP (Group Of

Pictures) is the following: ▫IBBPBBPBBPBBI▫An I image is thus enclosing all the 12

frames.

DivX format

•The DivX is a video compression-decompression format which allows videos to be compressed▫to very small volumes▫with a quite reasonable loss of quality.

•Thus, the DivX format makes it possible to store a complete film on a 650 or 700 MB CD-ROM.

MKV format•The MKV format (MatrosKa Video) is an

entirely free video format. •It is a container (Matroska refers to the

Russian dolls contained within another) making it possible to contain video (DivX, Xvid, RV9, etc), sound (MP3, MP2, AC3, Ogg, AAC, DTS, PCM), as well as subtitles (SRT, ASS, SSA, USF, etc) in the same file.

•The MKV format is based on a structure derived from XML, called EBML (Extensible Binary Meta Language).