introduction to multimedia1 ics 218 -multimedia systems and applications lecture 1 - introduction to...

Introduction to Multimedia 1

ICS 218 -Multimedia Systems and Applications

Lecture 1 - Introduction to MultimediaProf. Nalini [email protected]


Course logistics and details

Course Web page - http://www.ics.uci.edu/~ics218

Lectures - MW 2:00-3:20p.m, ELH 110Course Laboratories - machines on 3rd floor

CS labs ICS 218 Textbook:

Multimedia: Computing, Communications and Applications Ralf Steinmetz and Klara Nahrstedt, Prentice-Hall Inc.

Other reading materialTechnical papers and reports


Course logistics and details

Homeworks and Assignments 3 homeworks in the quarter of which 1 or 2 may

be programming assignments.

Tests Final Exam - as per UCI course catalog

Course Project Maybe done individually, in groups of 2 or 3(max) Potential projects on webpage

Data representation, multimedia systems and networks, multimedia applications


ICS 218 Grading Policy

Homeworks - 30% • (3 homeworks each worth 10% of the final grade).

Class Project - 40% of the final gradeFinal exam - 30% of the final gradeFinal assignment of grades will be based

on a curve.


Lecture Schedule

Weeks 1 and 2:Multimedia Representation

• Introduction to Multimedia• Audio/Image/Video Representation

Weeks 3 and 4: Multimedia Compression

• Encoding and Compression Techniques • Image Compression (JPEG) • Video Compression (MPEG/MPEG2/MPEG4)

Week 5 and 6: Multimedia Resource Management

• Multimedia Quality of Service and Server Design • Multimedia Operating Systems - Process Management,

Filesystems and buffer management


Course Schedule

Weeks 7 and 8: Multimedia Communication Systems

• MM Networking• MM Communication

Week 9: Multimedia Synchronization

• Multimedia Synchronization Concepts • Multimedia Synchronization Enforcement

Week 10: Multimedia Applications and Services

• Multimedia Video Conferencing• Multimedia Entertainment Applications


Introduction

Multimedia DescriptionWhy multimedia systems?Classification of MediaMultimedia SystemsData Stream Characteristics


Multimedia Description

Multimediais an integration of continuous media (e.g. audio,

video) and discrete media (e.g. text, graphics, images) through which digital information can be conveyed to the user in an appropriate way.

Multimany, much, multiple

MediumAn interleaving substance through which something

is transmitted or carried on


Why Multimedia Computing?

Application drivene.g. medicine, sports, entertainment, education

Information can often be better represented using audio/video/animation rather than using text, images and graphics alone.

Information is distributed using computer and telecommunication networks.

Integration of multiple media places demands oncomputation powerstorage requirementsnetworking requirements


Multimedia Information Systems

Technical challenges Sheer volume of data

Need to manage huge volumes of data

Timing requirementsamong components of data computation and

communication.Must work internally with given timing constraints - real-

time performance is required.

Integration requirementsneed to process traditional media (text, images) as well as

continuous media (audio/video).Media are not always independent of each other -

synchronization among the media may be required.


High Data Volume of Multimedia Information

Speech 8000 samples/s 8Kbytes/s

CD Audio 44,100 samples/s, 2 bytes/sample

176Kbytes/s

Satellite Imagery

180X180 km 2̂ 30m 2̂ resolution

600MB/image (60MB compressed)

NTSC Video 30fps, 640X480 pixels, 3bytes/pixel

30Mbytes/s (2-8 Mbits/s compressed)


Technology Incentive

Growth in computational capacityMM workstations with audio/video processing capabilityDramatic increase in CPU processing power Dedicated compression engines for audio, video etc.

Rise in storage capacityLarge capacity disks (several gigabytes)Increase in storage bandwidth,e.g. disk array

technology

Surge in available network bandwidthhigh speed fiber optic networks - gigabit networksfast packet switching technology


Application Areas

Residential Servicesvideo-on-demandvideo phone/conferencing systemsmultimedia home shopping (MM catalogs, product

demos and presentation)self-paced education

Business ServicesCorporate trainingDesktop MM conferencing, MM e-mail


Application Areas

EducationDistance education - MM repository of class videosAccess to digital MM libraries over high speed

networks

Science and Technologycomputational visualization and prototypingastronomy, environmental science

MedicineDiagnosis and treatment - e.g. MM databases that

provide support for queries on scanned images, X-rays, assessments, response etc.


Classification of Media

Perception MediumHow do humans perceive information in a computer?

• Through seeing - text, images, video • Through hearing - music, noise, speech

Representation MediumHow is the computer information encoded?

• Using formats for representing and information• ASCII(text), JPEG(image), MPEG(video)

Presentation MediumThrough which medium is information delivered by

the computer or introduced into the computer?• Via I/O tools and devices• paper, screen, speakers (output media)• keyboard, mouse, camera, microphone (input media)


Classification of Media (cont.)

Storage Medium• Where will the information be stored?• Storage media - floppy disk, hard disk, tape, CD-ROM

etc. Transmission Medium

• Over what medium will the information be transmitted?• Using information carriers that enable continuous data

transmission - networks• wire, coaxial cable, fiber optics

Information Exchange Medium• Which information carrier will be used for information

exchange between different places?• Direct transmission using computer networks• Combined use of storage and transmission media (e.g.

electronic mail).


Media Concepts

Each medium definesRepresentation values - determine the information

representation of different media• Continuous representation values (e.g. electro-

magnetic waves)• Discrete representation values(e.g. text characters in

digital form)Representation space determines the surrounding

where the media are presented.• Visual representation space (e.g. paper, screen)• Acoustic representation space (e.g. stereo)


Media Concepts (cont.)

Representation dimensions of a representation space are: Spatial dimensions:

two dimensional (2D graphics)three dimensional (holography)

Temporal dimensions:Time independent (document) - Discrete media

• Information consists of a sequence of individual elements without a time component.

Time dependent (movie) - Continuous media• Information is expressed not only by its individual value

but also by its time of occurrence.


Multimedia Systems

Qualitative and quantitative evaluation of multimedia systems Combination of media

continuous and discrete.

Levels of media-independencesome media types (audio/video) may be tightly

coupled, others may not.

Computer supported integrationtiming, spatial and semantic synchronization

Communication capability


Data Streams

Distributed multimedia communication systems

data of discrete and continuous media are broken into individual units (packets) and transmitted.

Data Streamsequence of individual packets that are transmitted in

a time-dependant fashion.Transmission of information carrying different media

leads to data streams with varying features• Asynchronous• Synchronous • Isochronous


Data Stream Characteristics

Asynchronous transmission mode • provides for communication with no time restriction• Packets reach receiver as quickly as possible, e.g.

protocols for email transmissionSynchronous transmission mode

• defines a maximum end-to-end delay for each packet of a data stream.

• May require intermediate storage• E.g. audio connection established over a network.

Isochronous transmission mode• defines a maximum and a minimum end-to-end delay

for each packet of a data stream. Delay jitter of individual packets is bounded.

• E.g. transmission of video over a network.• Intermediate storage requirements reduced.


Data Stream Characteristics

Data Stream characteristics for continuous media can be based on

Time intervals between complete transmission of consecutive packets

• Strongly periodic data streams - constant time interval• Weakly periodic data streams - periodic function with finite

period.• Aperiodic data streams

Data size - amount of consecutive packets• Strongly regular data streams - constant amount of data• Weakly regular data streams - varies periodically with time• Irregular data streams

Continuity• Continuous data streams• Discrete data streams


Classification based on time intervals

Strongly periodic data stream

Weakly periodic data stream

Aperiodic data stream

T

T

T1 T3T2

T1 T2

T


Classification based on packet size

TD1

D1

TD1D2D3D1D2D3

D1D2D3

Dn

Strongly regular data stream

Weakly regular data stream

Irregular data stream

t

t

t


Classification based on continuity

Continuous data stream

Discrete data stream

D

D1 D2 D3 D4

D

D1 D2 D3 D4


Logical Data Units

Continuous media consist of a time-dependent sequence of individual information units called Logical Data Units (LDU).

• a symphony consists of independent sentences• a sentence consists of notes• notes are sequences of samples

Granularity of LDUs• symphony, sentence, individual notes, grouped samples,

individual samples• film, clip, frame, raster, pixel

Duration of LDU:• open LDU - duration not known in advance • closed LDU - predefined duration


Granularity of Logical Data Units

Film

Clip

Frame

Blocks

Pixels

introduction to multimedia1 ics 218 -multimedia systems and applications lecture 1 - introduction to...

Documents