outline multimedia applications outlineseas3.elte.hu/itcourse-2009/multimedia_printout.pdf ·...

Multimedia applications

László Kálmán1 Csaba Oravecz1 Péter Szigetvári2

1Research Institute for LinguisticsHungarian Academy of Sciences

2Department of English LinguisticsEötvös Loránd University

Lecture 9. / 18 Nov, 2009

Kálmán, Oravecz, Szigetvári Multimedia

Outline1 Introduction

Sample exam questionMultimedia

2 Multimedia devicesInternal devicesExternal devices

3 Digitizing media contentAD conversionFrom sound to computer dataFrom images to computer data

4 The world of colorColor modelsImage models

5 Recap6 Images

Digital image editingDigital photography

7 Multimedia on the InternetTransmitting contentSharing media content

8 Multimedia applicationsVideo playersAudio playersImage editorsSoftware galore


Introduction

abstract

this lecture tells you about

• some popular multimedia devices;

• basic properties of different multimedia files

• and the applications to manipulate them;

• serving and getting media content through the web.

• Sodipodi.


Introduction Sample exam question






5 Recap6 Images





Introduction Sample exam question

Sample exam question


✇ VoIP is . . .A a video file formatB a computer gameC an Internet goblinD an Internet protocol to carry voice signal ✔


Introduction Multimedia






5 Recap6 Images





Introduction Multimedia

Multimedia

• uses multiple forms of information content and informationprocessing (text, audio, graphics, animation, video,interactivity)

• also refers to the use of electronic media to store andexperience multimedia content


Multimedia devices Internal devices






5 Recap6 Images






Display adapter (a.k.a. video/graphics card)

Figure: Video card

• generates and outputs images to a display• GPU (graphics processing unit): a microprocessor dedicated to

manipulating and rendering (3D) graphics• video memory (128MB-2GB): separate onboard memory, (video

RAM (VRAM)) to store the display image, textures, otherelements. VRAM typically runs at higher speeds than desktopRAM.



Sound card

Figure: Sound Blaster X-Fi Platinum sound card

• in general motherboards have built-in sound functionality but forhigh quality audio separate card is needed

• music composition, editing video or audio, presentation,entertainment (games)

• full-duplex: can record and play digitized sound simultaneously



TV tuner card

Figure: Hauppauge WinTV

• allows television signals to bereceived by a computer

• can also function as video capturecards, allowing recording televisionprograms onto hard disk

• hybrids are capable of analog (thepresent) and digital (DVB – the future)reception

• designed for the radio frequencies and video formats used in eachcountry

• some have hardware MPEG encoders• (a tuner displaying an HDTV image on a computer monitor is

typically much cheaper than a high-definition television system. . . )



TV tuner card

Figure: Watching television on the desktop



Sample non-exam question

What is the difference between a video card with a TV outputand a TV tuner card . . .

The former allows you to make the TV a computer display, thelatter the computer display a televison.


Multimedia devices External devices






5 Recap6 Images






DVD drives

DVD-ROM• DVD±RW DL: (plus/dash!!!) rewritable, dual layer, compatible with

DVD+RW and DVD-RW format• capacity: up to 8.5GB• DVD+RW supports random write access (data can be added and

removed without erasing the whole disc and starting over)• Blu-ray Disc: ultra-high-density optical disc, uses shorter

wavelength (405 nm) blue-violet laser (DVDs use a650-nm-wavelength red laser); 25-50GB (single/dual)

DVD drives• speed: up to 16x writing (21MB/s), 40x reading

logo 6= format!



Regional lockout

Regional lockout

Programming practice, code, chip, or physical barrier used toprevent the playing of media designed for a device from thecountry where it is marketed on the version of the same devicemarketed in another country.

• (video games)• DVD: DVD drives come from the factory with RPC (Regional

Playback Control)• RPC-1 (older drives): player software has the responsibility of

enforcing the region control• RPC-2 (newer drives): control is enforced by the drive’s

firmware

• In practice, many DVD players allow playback of any disc, orcan be modified to do so.


Figure: DVD regions


Portable multimedia players

Figure: 5th generation iPod

• a hard disk or flash memory basedelectronic device

• capable of storing and playing files inone or more media formats

• dedicated proprietary software,non-transferable data

• can function as mass storage deviceswith some standard file system tostore data files

• Digital Rights Management (DRM):prohibits transfer of music from onetype of device to another


Digitizing media content AD conversion






5 Recap6 Images





Digitizing media content AD conversion

Analog to digital conversion

Aim

Convert analog real-world signals to discrete digital form.

Sampling

• discretization: signal space is split into chunks and eachchunk is replaced with its representative signal

• quantization: the representative signal values areapproximated by some value from a finite set

• the higher the sampling frequency, the higher the fidelity


Digitizing media content From sound to computer data






5 Recap6 Images






Sound processing

After sampling, audio signals are represented by one of a fixed numberof values → pulse-code modulation (PCM)

PCM

A digital representation of an analog signal in a series of symbols inbinary code.

Figure: 4-bit PCM of a signal



Sound processing

• sampling rate: from 8 (telephony) to 44.1k samples/s (CD)or 48k samples/s (professional audio)

• quantization: from 8 bits (telephony) to 16 bits (CD)• bitrate=sampling rate×quantization×num.o.channels

44100× 16× 2 = 1411.2 kbit/s

• dither: small amount of deliberately-added noise to thesignal before digitization to avoid noticeable ’jumps’ in lowlevel signals

The one major uncompressed audio format: PCM . . .

• is adequate for storing and archiving an original recordingbut

• needs large disk space



Important concepts I.

Audio codec (s/audio/video/gi)

A computer program/algorithm that compresses/decompressesdigital audio data according to a given audio file format orstreaming audio format. Most codecs are implemented aslibraries which interface to one or more multimedia players.

Container format

A computer file format that can contain various types of data,compressed by means of standardized codecs. The containerfile is used to identify and interleave the different data types(audio, video, subtitles, chapters, meta-data, synchronizationinfo).



Audio file formats

Uncompressed container formats for PCM encoded audio

• WAV(E) (.wav) Microsoft (from the Resource Interchange FileFormat (RIFF))

• Audio Interchange File Format (AIFF) (.aiff) Apple• AU (.au) Sun Microsystems

Lossless compressed formats/codecs

• (lossless) Windows Media Audio (WMA) (.wma): proprietaryMicrosoft; WMA lossless codec

• Free Lossless Audio Codec (.flac) compressionrate: 30-50%

• Apple Lossless: stored within an MP4 container (.m4a);compression rate: 40-60%



Audio file formats

Lossy compressed formats/codecs

• MP3: (MPEG-1 Audio Layer 3)• discards components less audible to human hearing (≈ jpeg

images)• quality is heavily dependent on the choice of encoder and encoding

parameters (eg. bit rate)• has several limitations, licensing and patent problems but too

popular to be replaced by better encoders . . .

• Ogg format with Vorbis codec• open source and patent-free• cross-platform

• AAC (Advanced Audio Coding, default in Apple iTunes); WMA



MIDI

Musical Instrument Digital Interface

• communications protocol that enables electronic musicalinstruments, computers and other equipment to communicate,control and synchronize with each other

• does not transmit audio but only digital data about:• the pitch and intensity of musical notes to play• control signals for parameters of volume, vibrato and panning• clock signals to set the tempo

• in computers it is sound cards that are MIDI-compatible andcapable of creating realistic instrument sounds (quality sound →quality card)

• file format: Standard MIDI File (.smf, .mid)• created using computer-based sequencing software• http://en.wikipedia.org/wiki/

➥ List_of_MIDI_editors_and_sequencers


Digitizing media content From images to computer data






5 Recap6 Images






Image processing

Digital image

A representation of a two-dimensional image as a finite set of digitalvalues, called picture elements or pixels.

• color• brightness information

Resolution• number of pixels in an image• single number (see eg. digital cameras) 6.2 megapixel• product (displays) 1024x768

Color depth

Number of bits used to represent the color of a single pixel (8–48 bpp).(cf. Highcolor→16bpp; Truecolor→24bpp)



Image processing

Digital video

Digital (binary) representation of the video signal.

• composed of still images; frame: a set of all pixels that correspondto a single point in time

• normally 25–30 frames per second, depending on standard

Video compression (lossy) — codecs

• MPEGa-1 Part-2: used for Video CDs, sometimes for online video;quality ≈ VHS; normally up to 1150 kbit/s and 352x288

• MPEG-2 Part-2: used on DVD, SVCD, in most digital videobroadcasting and cable distribution systems; 720x576 and ≈ 10Mbit/s for DVD

• MPEG-4 Part-2: used for internet, broadcast, storage media;improved quality

• MPEG-4 Part-10: (Advanced Video Coding (AVC)) state of the artstandardized compression technology

• DivX, Xvid, 3ivx: different implementations of MPEG-4 Part-2• Theora: open source and patent-free competitor to MPEG-4

aMoving Picture Experts Group



Feedback slide I.


✇ Which of the following is true?A With digital video, lossless compression is typical.B AVI, MOV, Ogg are codecs whereas Vorbis, WAV are

container formats.C AVI, MOV, Ogg are container formats whereas Vorbis,

Sorenson are codecs. ✔D A video card with a TV output allows you to watch television

on the computer.



Common image file formats

Lossless (compressed/uncompressed)

• Windows Bitmap (.bmp): a bitmapped graphics format usedinternally by the Microsoft Windows graphics subsystem;uncompressed (but compresses losslessly well) → large files

• Encapsulated Postscript (.eps): a PostScript file minimally with aBoundingBox comment, describing the rectangle containing theimage

• Graphics Interchange Format (.gif): 8-bit-per-pixel bitmapimage format using a palette of up to 256 distinct colors; losslessbut unsuitable for images with continuous color; multiple images inone file → animations

• Portable Network Graphics (.png): bitmap image format withlossless data compression (patent-free successor of GIF); noanimations, incompatible with early IE

• Tagged Image File Format (.tif(f)): widely supported losslessformat; can handle multiple images and data in a single file



Common image file formats

Figure: More and morelossy. . .

Lossy compressed — JPEG

• Joint Photographic ExpertsGroup

• .jpg,.jpeg

• compression method and(commonly) file format, too butother formats can carry jpeg (eg.TIFF)

• most used for storing andtransmitting photographs on theWeb

• JPEG2000: much improvedsuccessor of jpeg but not incommon use yet


world

The of

color

The world of color Color models






5 Recap6 Images






Color models

Color modelA model describing the way colors can be represented astuples of numbers (typically three or four values or colorcomponents). It is associated with a precise description of howthe components should be interpreted → color space.



RGB color model

Red, green, blue

• uses additive color mixing, describes what kind oflight needs to be emitted to produce a given color (out ofdarkness)

• representations (color vectors):• in the range 0.0 (minimum) to 1.0 (maximum); (full intensity red

is 1.0, 0.0, 0.0); used in color science• as percentages, from 0% (minimum) to 100% (maximum) (full

intensity red is 100%, 0%, 0%)• in the range 0 to 255 (full intensity red is 255,0,0)• same in hexadecimal (full intensity red is #ff,#00,#00);

contracted to #ff0000 if used as web colors



Truecolor — 24 bit representation

yellow(255,255,0)

green(0,255,0)

cyan(0,255,255)

red(255,0,0)

blue(0,0,255)

red(255,0,0)

magenta(255,0,255)



CMYK color model

Cyan, magenta, yellow, key

• uses subtractive colors in terms of light absorption.The colors that are seen are from the part of light that is notabsorbed.

• magenta + yellow = redmagenta + cyan = bluecyan + yellow = green

• the use of black: produce pure black (without mixing threecolors, which is not perfect and expensive)

• no complete equivalance between RGB and CMYK• representation: vector with components 0.0–1.0


The world of color Image models






5 Recap6 Images






Raster graphics image

Figure: RGB bitmap

Bitmap

• data file or structure representinga rectangular grid of pixels(points of color)

• color of each pixel is individuallydefined

• corresponds bit for bit with animage displayed on eg. screen

• characterized by the width andheight of the image in pixels andthe number of bits per pixel



Vector graphics

Figure: Vector scales well

• image is represented through theuse of geometrical primitives suchas points, lines, curves, andpolygons

• mainly used today in the context oftwo-dimensional computer graphicsto create an image on a rasterdisplay

• minimal amount of informationresults in small file size

• moving, scaling, rotating, filling etc.doesn’t degrade the quality of adrawing

• indefinite zoom in (cf. image b(vector) and image c (raster))


Recap

Recap

Whatever the source of various signals, for computers a binary(bitwise) representation is needed → digitization.

Sound, video• continuous (analog) signal → binary data

• sampling• quantization (resolution)

• compression (lossy/lossless) → codecs

• storage → container files/formats

Colors• binary representation of the color spectrum: RGB, CMYK


Recap

Recap

Images

• representation• raster• vector

• storage/file format

lossless lossycompressed ✔ ✔

uncompressed ✔ ✘


Images Digital image editing






5 Recap6 Images






Digital image editing

Editing

The process of altering digital images (for home use, typicallydigital photographs) by means of specialised softwareprograms (graphics editors).

Main features of editors• selection: selecting part(s) of an image, so applying a

change selectively without affecting the entire picture;complex means (eg. edge detection, color-based, lasso)

• layers: transparent “sheets” stacked on top of each other,each capable of being individually edited

• size alteration: scaling of images

• cropping: creating a new image by removing a desiredrectangular portion from the image being cropped



Digital image editing

Main features of editors• noise: adding or removing noise in an image• cloning: removing unwanted elements from the image• selective color change: changing the color of specific items in

an image• image gradient: gradual blend of color• orientation: rotation and mirror• merging: merging one or more individual images into a single

picture• special effects• contrast change and brightening: improving an underexposed

image• sharpening and softening images (eliminating red-eye effect)• color adjustments ✌?


Images Digital photography






5 Recap6 Images






Digital cameras

Connectivity

USB port, as USB mass storage device. FireWire, Bluetooth or Wifialso possible. ✌?

Storage

Memory cards: CompactFlash, Memory stick, SD card (up to 8GB)

Exif — Exchangeable image file format

• specification for the image file format used by digital cameras• TIFF for the highest quality format and JPEG as a space-saving

but lower quality format; (RAW: proprietary, minimally processedformat, not standardized, plugins needed for image editors tohandle these files)

• metadata tags in images: date and time; camera settings;thumbnail; descriptions and copyright information (normallyviewable in graphical file managers ✌?)



Exif data

Figure: Exif data in Nautilus file manager


Multimedia on the Internet Transmitting content






5 Recap6 Images






Internet telephony

Voice over Internet Protocol (VoIP)

• transmitting of voice conversations over the Internet (orany IP-based network)

• VoIP → VoIP free; VoIP → PSTNa not

• mobility: you can be reached regardless of where you areconnected to the network

• quality and reliability of phone connection is usuallydependent upon the quality, reliability, and speed of theinternet connection

• security: relatively easy to eavesdrop

apublic switched telephone network



Internet telephony

Requirements

• PC with microphone and speaker (or headset orUSBphone), or an IP telephone

• reliable Internet connection

• account with an Internet telephony service provider

• piece of software for making telephone calls over theInternet (softphone) (Skype, Gizmo, X-Lite,Coccinella, Ekiga etc.)



Media content on the Internet

Streaming media

• media content that is heard or viewed while it is being delivered• client-server• webcast: sending audio and/or video live over the Internet• e-Radio: continuous stream of audio with no user control

Podcasting

• multimedia computer files are transferred from a server to a client• client pulls down XML files containing the Internet addresses of

media files (mostly audio or video, but could be any file type)

Some concepts

• video podcast (vidcast/vodcast): online delivery of video ondemand

• vlog: blog that includes video• FLV (Flash Video): proprietary file format used to deliver video

over the Internet using Adobe Flash Player (cf. YouTube)


Multimedia on the Internet Sharing media content






5 Recap6 Images





Multimedia on the Internet Sharing media content

Sharing content

P2P (peer-to-peer) network

• no clients or servers, but only equal peer nodes simultaneouslyfunctioning as clients and servers to the other nodes on thenetwork

• all clients provide resources (bandwidth, storage space,computing power)

• security and legal concerns (poisoning, viruses, spamming, etc.)• most used for file sharing

File sharing

• making files available for other users to download• uses dedicated P2P network with appropriate client:

• Azureus: cross-platform; BitTorrent• Soulseek, Museek(+), soulseeX: MS, Linux, Mac; Soulseek• DC++, LinuxDC++, ShakesPeer: MS, Linux, Mac;

DirectConnect


Multimedia applications Video players






5 Recap6 Images





Multimedia applications Video players

Video players/editors

Players — free/non-free

• WMP: MS• QuickTime: Mac, MS• MPlayer: cross-platform• xine: Mac, Unix/Linux

Editors — free/non-free• Avidemux: cross-platform• Adobe Premiere Pro/Elements: MS• Final Cut: Mac• Blender: cross-platform• Cinelerra: Unix/Linux• Mencoder: cross-platform CLI• Windows Movie Maker: MS


Multimedia applications Audio players






5 Recap6 Images





Multimedia applications Audio players

Audio players/editors

Players — free/non-free

• Amarok: Unix/Linux, Mac

• BS.Player: MS; free with prop. license

• iTunes: Mac, MS; free with prop. license

• Winamp: MS; free with prop. license

• XMMS: Unix/Linux, Mac

Editors — free/non-free• Audacity: cross-platform

• Goldwave: MS

• Rosegarden: Unix/Linux

• Pro Tools: Mac, MS


Multimedia applications Image editors






5 Recap6 Images





Multimedia applications Image editors

Graphics editors

Raster — free/non-free• ImageMagick: cross-platform CLI

• GIMP: cross-platform

• Adobe Photoshop: Mac, MS

• Paint.Net: MS

Vector — free/non-free• Adobe Illustrator: Mac, MS

• CorelDRAW: MS

• Openoffice Draw: cross-platform

• Inkscape: cross-platform

• Sodipodi: Linux, MS


Multimedia applications Software galore






5 Recap6 Images






Multimedia software galore

Links• http://en.wikipedia.org/wiki/

➥ Comparison_of_file_sharing_applications➥ Comparison_of_media_players➥ List_of_video_editing_software➥ Comparison_of_video_editing_software➥ List_of_raster_graphics_editors➥ List_of_vector_graphics_editors➥ Comparison_of_raster_graphics_editors➥ Comparison_of_vector_graphics_editors



Homework

Home assigment

Do some background reading on P2P networks, in particular


outline multimedia applications outlineseas3.elte.hu/itcourse-2009/multimedia_printout.pdf ·...

Documents