Multimedia Systems

What is a multimedia system?

A multimedia system supports the integrated storage, transmission and representation of the discrete media types text, graphics and image and the continuous media types audio and video on a digital computer.

Multimedia Systems

Early multimedia system, around 1985

Computers control the analog media streams, e.g., viacross-connect switches.

Multimedia Systems

Current Digital Multimedia System

The media streams are digital. They can be processed(e.g., compressed/decompressed, analyzed) in thecomputer.

Multimedia Systems Scope

Multimedia What is Multimedia?• When different people mention the term multimedia, they oftenhave quite different, or even opposing, viewpoints.

– A PC vendor: a PC that has sound capability, a DVD-ROM drive, and perhaps the superiority of multimedia-enabled microprocessors that understand additional multimedia instructions.

– A consumer entertainment vendor: interactive cable TV with hundreds of digital channels available, or a cable TV-like service delivered over a high-speed Internet connection.

– A Computer Science (CS) student: applications that use multiplemodalities, including text, images, drawings (graphics), animation,video, sound including speech, and interactivity.

• Multimedia and Computer Science:– Graphics, HCI, visualization, computer vision, data compression, graph theory, networking, database systems. Multimedia and Hypermedia

Multimedia

Components of Multimedia

• Multimedia involves multiple modalities of text, audio,images, drawings, animation, and video.

Examples of how these modalities are put to use:1. Video teleconferencing.2. Distributed lectures for higher education.3. Tele-medicine.4. Co-operative work environments.5. Searching in (very) large video and image databases fortarget visual objects.6. “Augmented” reality: placing real-appearing computergraphics and video objects into scenes.

Multimedia

Components of Multimedia (contd from last slide…)

7. Including audio cues for where video-conferenceparticipants are located.

8. Building searchable features into new video, andenabling very high- to very low-bit-rate use of new,scalable multimedia products.

9. Making multimedia components editable.

10. Building “inverse-Hollywood” applications that canrecreate the process by which a video was made.

11. Using voice-recognition to build an interactiveenvironment, say a kitchen-wall web browser.

Multimedia Current Multimedia Projects which are ongoing:• Many exciting research projects are currently underway. Here

are a few of them:1. Camera-based object tracking technology: tracking of the

control objects provides user control of the process.

2. 3D motion capture: used for multiple actor capture so that multiple real actors in a virtual studio can be used to automatically produce realistic animated models with natural movement.

3. Multiple views: allowing photo-realistic (video-quality) synthesis of virtual actors from several cameras or from a single camera under differing lighting.

4. 3D capture technology: allow synthesis of highly realistic facial animation from speech.

History Of Multimedia

• What can we say about the evolution of media that has taken place for thousands of years?

• Since the dawn of time, people have had the need to communicate with one another.

• This created what we called as communication media.

http://www.december.com/present/mediaev.html

History Of Multimedia

• Newspapers (perhaps) the first mass communication medium, which used mostly text, graphics, and images.

• 1895 Gugliemo Marconi sent hisfirst wireless radio transmission at Pontecchio, Italy. • 1901 he detected radio waves beamed across the Atlantic. • Initially invented for telegraph, radio is now a major medium for audio broadcasting.

History Of Multimedia

•Television new media for the 20th century. •It brought video (+audio) and has since changed the world of mass comm.

Some Important Events In Computer History

• 1945 - Vannevar Bush (1890-1974) wrote about Memex – a device in which an individual stores all his books,

records, and communications, and which it is mechanized so that it may be consulted with exceeding speed and flexibility

Vannevar Bush

Engelbart

Some Important Events In Computer History

• 1960s - Ted Nelson started Xanadu project (The Original Hypertext Project)

• 1968 - Douglas Engelbart demonstrated NLS (oN Line System) - The Debut of The Mouse

• 1969 - Nelson & Van Dam hypertext editor at Brown University.

Some Important Events In Computer History

• 1976 - Architecture Machine Group proposal to DARPA: Multiple Media

• 1985 - Negroponte, Wiesner: opened MIT Media Lab in Boston

• 1989 - Tim Berners-Lee proposed the World Wide Web to CERN (European Organization For Nuclear Research)

• 1990 – Kristina Hooper Woolsey headed Apple Multimedia Lab, 100 staff, for education

Some Important Events In Computer History

• 1993 - U. Illinois National Center for Supercomputing Applications: NCSA Mosaic• 1994 – Jim Clark and Marc Andreessen: Netscape• 1995 - JAVA language for platform-independent application development. • 1998 – XML 1.0 announced as a W3C recommendation• 1998 – Handheld MP3 devices (32MB) flash memory• 2000 – WWW size estimated over 1 billion pages.

Overlapping Technologies

• Different branches of multimedia grow together because of new, upcoming multimedia technology and applications.

• Two challenges lie ahead:– Timing requirements (synchronization etc.)

– Integration requirements (of different media types)

Multimedia

The notion of Multimedia

Consists of two words:

Multi (Latin)= many; much;

Medium (Latin) = An intervening substance through which something is transmitted or carried on.

Multimedia

What is Multimedia?• Multimedia means a (usually) interactive

combination of two or more media elements (multimedia building block), such as text, graphics, audio, video and animation integrated using a computer

• A multimedia system is a system that supports more than a single type of media.

Multimedia Building Block

Digital environment

USERUSER

Elements of Multimedia

Multimedia

• Multimedia has a number of distinct and unique features, including:– Based on Edgar Dale (Cone Of Learning), on

average, people remember:• 10% of what they read,• 20% of what they hear,• 30% of what they see, • 50% of what they hear and see, multimedia approach

– multimedia rich elements, multi-sensory delivery system can facilitate greater retention of new knowledge

Multimedia Modalities

• Modalities are the sensory systems through which a multimedia activity occurs

• This includes tactile (touch), gustatory (taste), visual (sight), auditory (hearing), olfactory (smell)

• Based on the multimedia elements we have today, only two modalities are regularly used.

Multimedia Channels

• Channels can be understood as existing within a modality.

• For example, with the auditory modality, we have different channels for noises, speech and music.

• With the visual modality, we have different channels for words, pictures and movies.

Multimedia Channels

• Bandwidth is a concept of how much information can be carried by a certain channel within a certain modality.

• For example, we can read at the rate of 150 words per minute which is the ‘printed text’ channel within the ‘visual’ modality.

• Much like your modem, you are unlikely to reach the theoretical maximum bandwidth of your channel within any modality, in practice.

Multimedia Channels

• The reason for this discrepancy in practice is because in theory, we assume a perfect encoder and decoder.

• For example, English text ‘encoded’ on a page and the English language ‘decoder’ in someone’s head is assumed to be perfectly compatible.

• In reality, however, it is highly dependent on the education level of the reader, the nature of the text information presented and many other factors.

Multimedia ‘Medium’

• A medium can be understood as a set of co-ordinated channels, spanning one or more modalities, which have come to be referred to as a unitary whole, and which possess a cross-channel language of interpretation.

• Examples include a television show, which typically uses the auditory and visual modalities; and picture, written text, speech and music channels.

Multimedia System Characteristics

• Multimedia systems must be computer controlled.

• All multimedia components are integrated.

• The interface to the final user may permit interactivity.

• Information must be represented digitally.