Media – an overviewMedia – an overview

11

MediaMedia ImageImage VideoVideo FlashFlash SoundSound HTML Graphics (covered later)HTML Graphics (covered later) Tools to createTools to create

22

33

Media – all take storage spaceMedia – all take storage space

TermTerm DefinitionDefinition

BitBit Binary value 0 or 1Binary value 0 or 1

Kilobit (Kb)Kilobit (Kb) 1,000 bits (approx.)1,000 bits (approx.)

ByteByte 8 bits8 bits

Kilobyte (KB)Kilobyte (KB) 1,000 bytes1,000 bytes

Megabyte Megabyte (MB)(MB)

1,000,000 bytes1,000,000 bytes

Gigabyte Gigabyte (GB)(GB)

1,000,000,000 bytes1,000,000,000 bytes

KbpsKbps Kilobits per second (1,000 bits in a Kilobits per second (1,000 bits in a second)second)

KB/secKB/sec Kilobytes per second (1,000 bytes in a Kilobytes per second (1,000 bytes in a second)second)

ImagesImages

How they are createdHow they are created Some tips about photographySome tips about photography ResolutionResolution Image FormatsImage Formats

44

55

Image FormationImage Formation

How are images created.How are images created.

66

Image FormationImage Formation

Images are formed when a Images are formed when a SENSORSENSOR registers registers RADIATIONRADIATION that has that has interacted with interacted with PHYSICAL OBJECTSPHYSICAL OBJECTS

77

Types of ImagesTypes of Images Photography: reflected Photography: reflected

lightlight Range images: Range images:

distancedistance Tomography: tissue Tomography: tissue

densitydensity Infrared: heatInfrared: heat

We will concentrate on We will concentrate on the first typethe first type

(gray scale and color).(gray scale and color).

88

Digital ImagesDigital Images

are 2D arrays (matrices) of numbers:

99

Digital imagesDigital images

light intensities, or colorlight intensities, or color distances, or distances, or other physical quantities.other physical quantities.

Depending on the type of image, the numbers, pixel values, represent:

Color image and3 fields –red, green,blue

Blow up – shows pixels

Pixel values representDepth – white nearer

1010

Intensity ImagesIntensity Images

Light coming from the world hits the sensor.Light coming from the world hits the sensor.

1111

Distant objects appear smallerDistant objects appear smaller Even though B is larger than C it appears Even though B is larger than C it appears

the same size because it is more distant.the same size because it is more distant.

1212

Optical Parameters- about Optical Parameters- about photographyphotography

Angular aperturesAngular apertures

Larger aperture (larger diameter of opening)•Lets in more light, but, can have

Smaller aperture (smaller diameter of opening)

1313

Optical ParametersOptical Parameters Larger Aperture means:Larger Aperture means:

• More light comes in (can have faster shutter More light comes in (can have faster shutter speed …take quick picture and hence speed …take quick picture and hence capture faster moving objects).capture faster moving objects).

• BUT, also means incoming light is BUT, also means incoming light is uncollimated rays, resulting in a uncollimated rays, resulting in a sharp focus sharp focus only for rays with a certain focal lengthonly for rays with a certain focal length. This . This means that a greater aperture results in an means that a greater aperture results in an image that is image that is sharp around what the lens is sharp around what the lens is focusing on and blurred otherwise. focusing on and blurred otherwise. SMALLER DEPTH OF FIELD SMALLER DEPTH OF FIELD

For an apertureShutter speed too low

Object near flower are not in focusFor this larger aperture

1414

Optical ParametersOptical Parameters Smaller Aperture means:Smaller Aperture means:

• Less light comes in (need greater Less light comes in (need greater exposure time…not good for fast moving exposure time…not good for fast moving objects…get a blur).objects…get a blur).

• BUT, also means incoming light is highly BUT, also means incoming light is highly collimated rays, collimated rays, resulting in a sharp focus resulting in a sharp focus at the image plane for objects an a more at the image plane for objects an a more varied distance from the point focused on varied distance from the point focused on in the scene. GREATER DEPTH OF FIELDin the scene. GREATER DEPTH OF FIELD

Objects around flower are in focus withsmaller aperture

1515

Resolution, Size of ImageResolution, Size of Image Image size = (rows)*(colomns) Image size = (rows)*(colomns) Resolution = size of image = #pixels Resolution = size of image = #pixels low resolution ---> # of pixels is small low resolution ---> # of pixels is small high resolution ----> # of pixels is big high resolution ----> # of pixels is big HOW DO YOU SELECT A RESOLUTION?HOW DO YOU SELECT A RESOLUTION?

• if grid is too large you will get jagged edges if grid is too large you will get jagged edges • This is called Aliasing.This is called Aliasing.

No Aliasing Aliasing

1616

Loss in making a Digital Image?Loss in making a Digital Image?

Loss from sampling? Loss from sampling? • Not if choose the correct number of Not if choose the correct number of

samples.samples.• Over-Sampling Over-Sampling = when you have more = when you have more

samples than you need samples than you need • Under-Sampling Under-Sampling = not enough samples are = not enough samples are

used used Loss from quantization?Loss from quantization?

• AlwaysAlways unless your analog unless your analog image miraculously happens to image miraculously happens to only have values at the only have values at the quantization levels.quantization levels.

Ok samples Under sampled

256 levels 8 levels

Types of Types of GraphicsGraphics

Graphic types commonly used on Graphic types commonly used on Web pages:Web pages:• GIFGIF• JPGJPG• PNGPNG

1717

GIFGIF

Graphics Interchange FormatGraphics Interchange Format Best used for line art and logosBest used for line art and logos Maximum of 256 colorsMaximum of 256 colors One color can be configured as transparentOne color can be configured as transparent Can be animatedCan be animated Uses lossless compressionUses lossless compression Can be interlacedCan be interlaced

1818

Background color – no

transparency

Background color

configured to be

transparent

JPEGJPEG Joint Photographic Experts GroupJoint Photographic Experts Group Best used for photographsBest used for photographs Up to 16.7 million colorsUp to 16.7 million colors Use lossy compressionUse lossy compression Cannot be animatedCannot be animated Cannot be made Cannot be made

transparenttransparent Progressive JPEG – similar to Progressive JPEG – similar to

interlaced displayinterlaced display

1919

PNGPNG

Portable Network GraphicPortable Network Graphic Support millions of colorsSupport millions of colors Support multiple levels of transparencySupport multiple levels of transparency

(but browsers do not -- (but browsers do not -- so limit to one transparent color for Web display)so limit to one transparent color for Web display)

Support interlacingSupport interlacing Use lossless compressionUse lossless compression Combines the best of GIF & JPEGCombines the best of GIF & JPEG Browser support is growingBrowser support is growing

2020

VideoVideo

What is VideoWhat is Video Video FormatsVideo Formats Video EditingVideo Editing Video HostingVideo Hosting

2121

2222

Video basically a sequence of Video basically a sequence of images (frames)images (frames)

Video TermsVideo Terms

Frame rate =number of frames per Frame rate =number of frames per second – mostly 25-30 fps is what you will second – mostly 25-30 fps is what you will want.want.

Interlacing = displays odd then even rows Interlacing = displays odd then even rows in a frame (does not change all of them)in a frame (does not change all of them)

Resolution = number of rows and Resolution = number of rows and columns in a framecolumns in a frame

Aspect Ration= i.e. 16:9 width:height Aspect Ration= i.e. 16:9 width:height (this is high def)(this is high def)

2323

2424

Video lengthVideo length

File size is proportional to the movie File size is proportional to the movie length.length.

Videos longer than 1 or 2 minutes Videos longer than 1 or 2 minutes cause long download times. (moving cause long download times. (moving target)target)

If it is a long video, consider If it is a long video, consider streaming video or breaking it into streaming video or breaking it into smaller videos.smaller videos.

Video FormatsVideo Formats

mpg = MPEG = motion picture expert groupmpg = MPEG = motion picture expert group mp4 = MPEG -4 =version of MPEGmp4 = MPEG -4 =version of MPEG avi = audio video, microsoftavi = audio video, microsoft mov,qt = quick timemov,qt = quick time rv = real video by real networks = streamingrv = real video by real networks = streaming wmv = windows media video, newer microsoft wmv = windows media video, newer microsoft

standardstandard flv = flash video (requires flash player plugin)flv = flash video (requires flash player plugin)

2525

2626

Frame sizeFrame size

Larger “full-screen”Larger “full-screen” video is 640x480 pixels or more. video is 640x480 pixels or more.

A common frame size for web video is 160x120 A common frame size for web video is 160x120 pixels (smaller window inside a page).pixels (smaller window inside a page).

Not recommend to use a frame size larger than Not recommend to use a frame size larger than 320x240. (moving target)320x240. (moving target)

The size you should use depends on the CPU The size you should use depends on the CPU power and the Internet connection bandwidth of power and the Internet connection bandwidth of your clients.your clients.

Video Formats and Video Formats and CompressionCompression

GOAL: reduce the size to store video GOAL: reduce the size to store video by sometime removing not important by sometime removing not important informationinformation

CODEC: sw/algorithm to compress CODEC: sw/algorithm to compress (CO) and decompress (DEC) the video(CO) and decompress (DEC) the video

Usually formats specify compressions Usually formats specify compressions but, sometimes there are options.but, sometimes there are options.

Each format can have different versions Each format can have different versions – this is important.– this is important.

2727

Video Formats – which oneVideo Formats – which one

Many require plug-in to play Many require plug-in to play Choose a format based on what browsers Choose a format based on what browsers

will supportwill support Choose a format based on how good it Choose a format based on how good it

compresses and how fast it playscompresses and how fast it plays

A suggestion for getting fast playing high compression A suggestion for getting fast playing high compression = MPEG4 with code standard H.264 (best compression)= MPEG4 with code standard H.264 (best compression)

Alternatively for possibly larger size file use flash video Alternatively for possibly larger size file use flash video flvflv

2828

2929

MPEG ComparisonMPEG Comparison

MPEG-1MPEG-1 MPEG-2MPEG-2 MPEG-4MPEG-4

Standard sinceStandard since 19921992 19951995 1999+1999+

Default Video resolution Default Video resolution (NTSC)(NTSC)

352 x 288352 x 288 640 x 480640 x 480 640 x 480640 x 480

Max. Audio Frequency rangeMax. Audio Frequency range 48 KHz48 KHz 96 KHz96 KHz 96 KHz96 KHz

Max. audio ChannelMax. audio Channel 22 88 88

Regular data rateRegular data rate 1380 kbit/s1380 kbit/s 6500 kbit/s6500 kbit/s 880 kbit/s880 kbit/s

Frames per sec (NTSC)Frames per sec (NTSC) 3030 3030 3030

Video QualityVideo Quality SatisfactorySatisfactory Very goodVery good Very GoodVery Good

Hardware requirement for Hardware requirement for encoding/decodingencoding/decoding

LowLow MediumMedium HighHigh

Video Editing and Creation Video Editing and Creation ToolsTools

Adobe Premier Software, Adobe Media Encoder (conversion)Adobe Premier Software, Adobe Media Encoder (conversion)

Free tools – Free tools – • riva front end for front end for FFmpeg sw tools (conversion one sw tools (conversion one

format to another– harder to use), format to another– harder to use), • VLC (conversion one format to another and player) (conversion one format to another and player)• On windows machine – microsoft windows movie maker On windows machine – microsoft windows movie maker

= editing= editing• DivX ( (www.divx.com))• MPEG Stream Clip (search for it)MPEG Stream Clip (search for it)• xVId (xVId (xvid.org))

Cheap tools – quicktime (will do conversion into mpeg4)Cheap tools – quicktime (will do conversion into mpeg4)

3030

Video Editing / Conversion - Video Editing / Conversion - onlineonline

Search for free video editing / Search for free video editing / conversion online servicesconversion online services• www.zamzar.com

3131

YouTube hostingYouTube hosting

YouTube will take various input YouTube will take various input including (see youtube.com for including (see youtube.com for current list): .wmv, .avi, .mov,mp4current list): .wmv, .avi, .mov,mp4

Currently converts to .flv for most Currently converts to .flv for most playabilityplayability

They suggest uploading mp4 They suggest uploading mp4

3232

3333

QualityQuality Many video-editors allow you to set the overall Many video-editors allow you to set the overall

quality of the video.quality of the video.

The degree of compression controls the target The degree of compression controls the target quality.quality.

Frame rate and quality are usually tradeoff in Frame rate and quality are usually tradeoff in different applications.different applications.

Play with what quality setting is acceptable for Play with what quality setting is acceptable for you to reduce storage sizeyou to reduce storage size

AudioAudio

3434

Audio FormatsAudio Formats

mp3 = MPEG-3 audiomp3 = MPEG-3 audio mp4 = MPEG-4 audiomp4 = MPEG-4 audio ra/rm = streaming real audiora/rm = streaming real audio .wma, wav = windows media audio, .wma, wav = windows media audio,

waveform audio format by microsoftwaveform audio format by microsoft aif/aiff = audio interchange file aif/aiff = audio interchange file

format by Appleformat by Apple

3535

3636

Optimizing for the WebOptimizing for the Web

Length of the audio clipLength of the audio clip• Keep the audio clip as short as possible.Keep the audio clip as short as possible.

Number of channelsNumber of channels• A mono audio file is halved the space of A mono audio file is halved the space of

a stereo file.a stereo file.• Music – may want stereoMusic – may want stereo

3737

Optimizing for the WebOptimizing for the Web

Bit depthBit depth• Audio file on the Web are usually 8-bits.Audio file on the Web are usually 8-bits.• Half the size of a 16-bit file.Half the size of a 16-bit file.

Sampling rateSampling rate• Half the sampling rate will also halve the space Half the sampling rate will also halve the space

needed.needed.• Voice only audio file can be reduced to 8KHz.Voice only audio file can be reduced to 8KHz.• 22 KHz music clips are acceptable.22 KHz music clips are acceptable.

Putting all things together: Mono, 8-bit, Putting all things together: Mono, 8-bit, 22KHz, MP3 compression or better.22KHz, MP3 compression or better.

3838

Streaming AudioStreaming Audio

What is it?What is it?• Play almost immediately after the request, Play almost immediately after the request,

continues playing the continues playing the transferringtransferring data. data.

Advantages:Advantages:• Address the problem of long download time.Address the problem of long download time.• Control distribution and protect copyright, Control distribution and protect copyright,

because the user cannot get a copy of the file.because the user cannot get a copy of the file.

Disadvantages:Disadvantages:• Sound quality may be affected by low speed or Sound quality may be affected by low speed or

unstable Internet connection.unstable Internet connection.

3939

What is Streaming?What is Streaming?

4040

How does it work?How does it work?Streaming audioStreaming audio

Web browser

Web server

RealAudioServer

Web browser requesta RealAudio fromthe web server

Packet are sent to a buffer on the receivingComputer, the RealPlayer will play the soundFile when buffer full

4141

MIDI (.mid/.midi)MIDI (.mid/.midi)

MIDI stands for MIDI stands for ““MMusical usical IInstrument nstrument DDigital igital IInterfacenterface”” which is developed for electronic which is developed for electronic musical instruments.musical instruments.

MIDI files are very compact and very good MIDI files are very compact and very good for low-bandwidth delivery.for low-bandwidth delivery.

Instruments are Instruments are ““piano, drums, bass, piano, drums, bass, orchestral strings, orchestral strings, …”…”

Demo:Demo: www.findmidis.com

4242

Choosing an audio formatChoosing an audio format

Audio NeedsAudio Needs Suggested formatsSuggested formats

Short voice greetingsShort voice greetings MP3, WMA, WAV, AIFFMP3, WMA, WAV, AIFF

News broadcastsNews broadcasts Streaming solutions Streaming solutions (RealAudio, Windows Media)(RealAudio, Windows Media)

Background musicBackground music MP3, MIDI, WAV, WMAMP3, MIDI, WAV, WMA

Music samples for some Music samples for some audienceaudience

MP3 or QuickTimeMP3 or QuickTime

Radio-style or Live Radio-style or Live broadcastingbroadcasting

RealMedia SystemRealMedia System

Musical E-greeting cardMusical E-greeting card MIDI, WAVMIDI, WAV