chapter 4 - building blocks edt-010311_033357
Post on 07-Apr-2018
224 Views
Preview:
TRANSCRIPT
-
8/6/2019 Chapter 4 - Building Blocks Edt-010311_033357
1/53
CMM5013 Multimedia AuthoringCMM5013 Multimedia Authoringand Programmingand Programming
Chapter 4:Chapter 4:
Multimedia BuildingMultimedia Building
BlocksBlocks
-
8/6/2019 Chapter 4 - Building Blocks Edt-010311_033357
2/53
Part 3 - Sound
What is sound?
Waveforms and attributes of sound
Capturing digital audio
Sampling
Soundcard technology
MIDI (Musical Instrument Digital Interface)
-
8/6/2019 Chapter 4 - Building Blocks Edt-010311_033357
3/53
Sound
Sound is a complex relationship involving avibrating object (sound source), a transmissionmedium (usually air), a receiver (ear) and aperceptor (brain)
As the sound vibrates it bumps into molecules ofthe surrounding medium causing pressurewaves to travel away from the source in alldirections
As pressure waves get further from the sourcethey become weaker as their energy dissipates
-
8/6/2019 Chapter 4 - Building Blocks Edt-010311_033357
4/53
Waveforms
Sound waves are manifest as waveforms
Periodic waveform=waveform that repeats itself atregular intervals
Noise=Waveforms that do not exhibit regularity Cycle=unit of regularity
Hertz (or Hz) after Heinrich Hertz (a pioneer inthe field of acoustics)
One cycle = 1 Hz kHz or kiloHertz
(1 kHz = 1000 Hz)
-
8/6/2019 Chapter 4 - Building Blocks Edt-010311_033357
5/53
Waveforms
-
8/6/2019 Chapter 4 - Building Blocks Edt-010311_033357
6/53
Example waveforms
PianoPiano
Pan flutePan flute
Snare drumSnare drum
-
8/6/2019 Chapter 4 - Building Blocks Edt-010311_033357
7/53
Capture and playbackof digital audio
Air pressureAir pressurevariationsvariations
Captured viaCaptured viamicrophonemicrophone
Air pressureAir pressurevariationsvariations
ADCADC
Signal isSignal isconverted intoconverted into
binarybinary01010011010101001101
01101011110110101111
AnalogueAnalogueto Digitalto DigitalConverterConverter
DACDAC
ConvertsConverts
back intoback intovoltagevoltage
Digital toDigital toAnalogueAnalogueConverterConverter
-
8/6/2019 Chapter 4 - Building Blocks Edt-010311_033357
8/53
The attributes of sound
Sound is describedin terms of severalcharacteristics: Pitch (frequency of
the waveform inHz)
Amplitude (orloudness)
Timbre (ortone quality)
In addition, allsounds have aduration and
successive
http://upload.wikimedia.org/wikipedia/commons/c/cc/9577_Guitarz1970_Clean_E9_Guitar_Chord_(Mike_Tribulas).ogghttp://upload.wikimedia.org/wikipedia/commons/c/cc/9577_Guitarz1970_Clean_E9_Guitar_Chord_(Mike_Tribulas).ogg -
8/6/2019 Chapter 4 - Building Blocks Edt-010311_033357
9/53
-
8/6/2019 Chapter 4 - Building Blocks Edt-010311_033357
10/53
Digital samplingSampling frequency
:D e fin itio n T h e re co rd in g o f va lu e a t d iscre te in te rv a ls in t:D e fin itio n T h e re co rd in g o f va lu e a t d iscre te in te rva ls in ti
-
8/6/2019 Chapter 4 - Building Blocks Edt-010311_033357
11/53
The Nyquist theorem
The Nyquist theorem states: In order to be able to reconstruct a signal,
the sampling frequency must be at leasttwice the frequency of the signal being
sampled Named after Harold Nyquist of Bell Telephone Labs
(1928)
The highest frequency that can be
produced in a given digital audio system(i.e. half the sampling rate) is called theNyquist frequency
-
8/6/2019 Chapter 4 - Building Blocks Edt-010311_033357
12/53
Digital samplingSampling frequency
-
8/6/2019 Chapter 4 - Building Blocks Edt-010311_033357
13/53
Sample resolution
The resolution of a sample is thenumber of bits it uses to store a givenamplitude value, e.g.
8 bits (256 different values) 16 bits (65536 different values)
A higher resolution will give higher qualitybut will require more memory (or disk
storage)
-
8/6/2019 Chapter 4 - Building Blocks Edt-010311_033357
14/53
Quantisation
Definition: The restriction of any continuouslyvarying signal to a finite set of discretevalues.
Samples are usually represented as integers.
If the input signal has a voltage correspondingto a value of between 53 and 54, the ADCmay round it off to 53
Due to this rounding that must occur, thevalue of a sample is generally slightlydifferent from the original signal This is known as quantisation error and is
unavoidable
Increasing the sample resolution can reduce
-
8/6/2019 Chapter 4 - Building Blocks Edt-010311_033357
15/53
Quantisation example
-
8/6/2019 Chapter 4 - Building Blocks Edt-010311_033357
16/53
Calculating the sizeof digital audio
The size is the raw uncompressed memorythat the digital audio occupies
The formula is as follows:
The answer will be in bytes Where:
sampling rate is in Hz duration is in seconds resolution is in bits number of channels = 1 for mono, 2 for
stereo, etc
8
channelsofnumberresolutiondurationrate
-
8/6/2019 Chapter 4 - Building Blocks Edt-010311_033357
17/53
Calculating the data rateof digital audio
Data rate = the rate at which the data mustbe supplied to the audio hardware in orderto be played back at the correct speed
The formula is as follows:
The answer will be in bytes per second Where:
sampling rate is in Hz resolution is in bits number of channels = 1 for mono, 2 for
stereo, etc.
8
channelsofnumberresolutionrate
-
8/6/2019 Chapter 4 - Building Blocks Edt-010311_033357
18/53
CD-DA (CD-Digital Audio)
Compact disks are recorded using 2channels (stereo) using 16 bits ofinformation at 44.1 KHz
It therefore requires just over 10MB to storea single minute of CD audio
Compact disks can store about 74 minutesof digital audio
but only around 650 MB of data!
A Sony MiniDisc has a data capacity ofabout 140MB
-
8/6/2019 Chapter 4 - Building Blocks Edt-010311_033357
19/53
Digital audio editingsoftware
One of the most powerful andprofessional PC-based packages is atool called Sound Forge
http://www.sonicfoundry.com/http://www.sonicfoundry.com/
-
8/6/2019 Chapter 4 - Building Blocks Edt-010311_033357
20/53
The purpose of a soundcard
The purpose of a soundcard is to provide some or all of the
following functions:
Play sounds
Provide a MIDI device interface
Synthesise musical sounds and sound effects
Provide a game port interface (joystick, game pad, etc...)
Record sounds from an input (microphone, CD player, etc...)
Mix sounds coming from many sources (microphone, CD
player, digital audio output) and send them to an output
(speakers, headphones)
Control a CD-ROM drive with a drive controller (mostly in
older soundcards)
-
8/6/2019 Chapter 4 - Building Blocks Edt-010311_033357
21/53
MIDI (Musical Instrument DigitalInterface)
MIDI is a standard for specifying a musicalperformance
Rather than send raw digital audio, it sendsinstructions to musical instruments telling them
what note to play, at what volume, using whatsound, etc.
The synthesiser that receives the MIDI events isresponsible for generating the actual sounds
The size of MIDI files is small Proportional to the number of MIDI events
Although they cannot contain any digital audio!
-
8/6/2019 Chapter 4 - Building Blocks Edt-010311_033357
22/53
MIDI sequencers
A MIDI sequencerallows musicians toedit and createmusical
compositions like aword processor Cut and paste
Insert / delete
-
8/6/2019 Chapter 4 - Building Blocks Edt-010311_033357
23/53
Summary
There are two main types of digital audio Sampled audio
Captured by sampling an analogue waveform at a setrate
MIDI data Instructions on how to perform some musical
composition
Sampled audio requires more storage space thanMIDI information
Modern soundcards can capture and playback bothsampled audio and MIDI information
-
8/6/2019 Chapter 4 - Building Blocks Edt-010311_033357
24/53
Part 4 - Video
Analogue video
What is digital video?
Calculating the size of digital video
Compression techniques Digital video formats
Video capture hardware
Digital video editing Consumer desktop video
-
8/6/2019 Chapter 4 - Building Blocks Edt-010311_033357
25/53
-
8/6/2019 Chapter 4 - Building Blocks Edt-010311_033357
26/53
PAL video
The PAL video image iscomposed of 625lines
The actual picture is
contained in 576lines The rest is taken up
by Teletextinformation
768 576 = 1.333 =4:3
25
( )
625
lines
625
lines
768 pixels768 pixels
576
lines
576
lines
-
8/6/2019 Chapter 4 - Building Blocks Edt-010311_033357
27/53
What is digital video?
Digital video is the digitisation of analoguevideo signals into numerical format
Conversion from analogue to digital format
requires the use on an ADC (Analogueto Digital Converter) A Digital to Analogue Converter (DAC) can be
used to output digital video on analogue
equipment Sound can be captured separately
Digital video can have zero or more channels ofaudio
NICAM stereo broadcasts have 2 channels
DVD has 6 channels of sound (called 5.1)
C l l ti th i f di it l
-
8/6/2019 Chapter 4 - Building Blocks Edt-010311_033357
28/53
Calculating the size of digitalvideo
Where:
width = image width in pixels
height = image height in pixels
colourdepth is measured in bits per pixel fps = number of frames per second
8
fpsdepthcolourheightwidth)(size
=bytes
-
8/6/2019 Chapter 4 - Building Blocks Edt-010311_033357
29/53
-
8/6/2019 Chapter 4 - Building Blocks Edt-010311_033357
30/53
Compression techniques
Since the size of raw digital video is soprohibitively large we need some meansto compress the information
Lossy compression techniques causesome information to be lost from theoriginal image
You can never recreate the source image from thecompressed version
Lossless compression techniques do notlose information
You can always recreate an exact replica of thesource information
-
8/6/2019 Chapter 4 - Building Blocks Edt-010311_033357
31/53
Compression techniques
The two main forms of compression are:
Compression of repeating information Take a newsreader as an example. Most of the
screen does not change (the background,desk, etc).
Only need to store parts of the image thatactually change
Removal of low-visibility artefacts Things the eye cannot easily identify can be
removed or highly compressed andsynthesised upon playback
-
8/6/2019 Chapter 4 - Building Blocks Edt-010311_033357
32/53
Digital video formats
Microsoft AVI
Files with a .avi extension
Apple QuickTime
Files with a .mov or .qt extension MPEG / MJPEG
Files with a .mpg extension
-
8/6/2019 Chapter 4 - Building Blocks Edt-010311_033357
33/53
Microsoft AVI
Audio Video Interleave format
Interleaving is a technique used to embed two ormore things into the same stream of information
In every chunk of information you will find some
video data and some audio data
001001010010010101010011110101...001001010010010101010011110101...
001001010010010101010011110101...001001010010010101010011110101...
......
001001010010010101010011110101...001001010010010101010011110101...
001001010010010101010011110101...001001010010010101010011110101...
VideoVideoinformationinformation
AudioAudioinformationinformation
8, 168, 16or 24or 24
bitsbits
-
8/6/2019 Chapter 4 - Building Blocks Edt-010311_033357
34/53
MPEG video
Named after the Moving Picture ExpertsGroup who devised the compression andfile formats
There are a number of MPEG formats: MPEG-2 is used for digital TV broadcasts and
DVDs
MPEG-1 is a format used for low quality video(generally displayed on computers)
MPEG-1 Layer 3 is the popular encodingmechanism for MP3 audio files
MPEG-4 is a new format for multimediapresentations
Can require separate hardware to decode
higher quality MPEG video data
-
8/6/2019 Chapter 4 - Building Blocks Edt-010311_033357
35/53
MPEG compression example
A simple scene showing a car movingA simple scene showing a car movingacross a desert landscapeacross a desert landscape
Only the differenceOnly the difference
between the currentbetween the currentand next frame needsand next frame needsto be storedto be stored
This is called intraframe codingThis is called intraframe coding
-
8/6/2019 Chapter 4 - Building Blocks Edt-010311_033357
36/53
QuickTime
Developed by Apple,Inc.
Primarily for playbackwithout any
hardware assistance Can achieve
compression ratiosof 25 to 200:1
The QuickTime formatcan also store audio,graphics, 3D andtext making it moremuch versatile formultimedia
-
8/6/2019 Chapter 4 - Building Blocks Edt-010311_033357
37/53
Video capture hardware
The hardware hasvarious input andoutput connectors Composite video
in/out S-Video in/out
Audio in/out
Special chips providethe processingpower tocompress/decompress the videoinformation
CODEC (Compressor /
-
8/6/2019 Chapter 4 - Building Blocks Edt-010311_033357
38/53
Digital video editing
Analogue tape editing is a linear process To find the section you want, you may have to
forward or rewind the video tape
To move a section to another place in the
sequence you have to either re-record thesection onto another tape or physically cutand splice the video tape
Digital video editing can be non-linear You can move sections around inside the
computer and play those sections back inany order
Non-destructive editing and Edit Decision Lists
(EDL)
-
8/6/2019 Chapter 4 - Building Blocks Edt-010311_033357
39/53
Consumer desktop video
-
8/6/2019 Chapter 4 - Building Blocks Edt-010311_033357
40/53
Summary
Today we have seen how analogue videoformats are composed and how digitalvideo can be used to store theseelectronically
Digital video demands huge file sizes even before sound is added on!
Compression techniques help to reduce the
file sizes to more manageable levels
-
8/6/2019 Chapter 4 - Building Blocks Edt-010311_033357
41/53
Part 5 - Animation
Depiction of objects as they vary overtime
Traditionally, based on individual drawing
or photographing the frames in asequence
Computer animation also results in asequence of images, but these are
created by software.
-
8/6/2019 Chapter 4 - Building Blocks Edt-010311_033357
42/53
Animation
Such animation is made from a series of stills and
relies on something calledpersistence of vision. Persistence of vision is the phenomenon were an
object on the eyes retina remains for a brieftime after viewing.
This means that a series of still images which varyslightly, if shown rapidly will give the illusion ofmovement.
If each of the eight pictures below were shown atthe same point in rapid succession, the result
would be a rotating arrow.
-
8/6/2019 Chapter 4 - Building Blocks Edt-010311_033357
43/53
Animation
The pictures shown below are stills of therotating Ford logo from the Ford MotorCompany site.
-
8/6/2019 Chapter 4 - Building Blocks Edt-010311_033357
44/53
Animation
TV gives the illusion of continuous movement byshowing the stills at the rate of 30 frames persecond.
Feature films are filmed at a rate of 24 frames per
second but shown at a rate of 48 frames persecond.
Today, most animation is performed by computer.Examples are Bugs, Toy Story, Jurasic Park and
the BBCs Walking with Dinosaurs. The computer will produce a wire frame of the
scene, then apply textures and light effectsbefore moving onto the production of the nextframe, each of which may take hours or evendays to produce.
-
8/6/2019 Chapter 4 - Building Blocks Edt-010311_033357
45/53
Cel Animation
Cel Animation is the technique used to producethe old Tom & Jerry cartoons and the newcomputer generated Disney cartoons.
It draws its name from the celluloid films used in
old hand drawn animations. The celluloid films permitted layering, where the
background to a sequence may be drawn on oneor more films and then the films containing the
animated characters place on top. Such animation starts with the production of
keyframes, which are thefirst and last frames ofan action scene.
-
8/6/2019 Chapter 4 - Building Blocks Edt-010311_033357
46/53
Cel Animation
The frames in between the keyframes arethen produced using a process known astweening.
Tweening is where the number of frameswhich must appear between keyframes iscalculated and the frames drawn.
Tweening may be performed by computer if
the frames are not too far apart.
-
8/6/2019 Chapter 4 - Building Blocks Edt-010311_033357
47/53
Computer Animation
Computer animation works in much thesame way as Cel Animation.
It even uses the same terms, such as
keyframes, layers and tweening. Although theoretically limited by the scan
rate of the monitor being used, the framerate of any animation is typically dictated
by the memory and processing power ofthe computer as well as the channel it isbeing shown across (if any).
-
8/6/2019 Chapter 4 - Building Blocks Edt-010311_033357
48/53
Computer Animation
To produce smooth animation, a minimum rate of15 frames per second must be sustained.
Any lower and the animation will appear jerky.
The frame rate depends mostly on the power of
the processor and the bandwidths between mainmemory (66/100MHz) and the processor and theprocessor and the graphics card (VL, PCI, AGPx1x2 x4).
It also depends of course on the performance ofthe graphics card and the speed of its onboardmemory and integrated acceleration techniques.
-
8/6/2019 Chapter 4 - Building Blocks Edt-010311_033357
49/53
Computer Animation
The frame rate does of course also depend on thecharacteristics of the animation frames.
The more frames, the greater the demands on thehardware.
The pixel depth of each frame also impacts greatlyon performance.
As does the resolution of each frame of theanimation.
If you do not have control of the playbackhardware, it may be necessary to work to andrecommend a minimum spec machine e.g. MPC2etc.
This will inevitably mean keeping frame rate and
pixel depths low and the size of the animation( ixels hi h and wide) relativel small.
-
8/6/2019 Chapter 4 - Building Blocks Edt-010311_033357
50/53
Morphing
Morphing is a special effect in motionpictures and animations that changes (ormorphs) one image into another through aseamless transition.
This is done by first creating or scanning thefirst and last images.
Then key points are specified, i.e. points on
the original image which should becomepoints on the final image.
The computer then produces the frames inbetween, with each successive frame
becoming progressively more morphed
-
8/6/2019 Chapter 4 - Building Blocks Edt-010311_033357
51/53
Morphing
The creator can usually specify the number ofintermediate frames to be produced as well thenumber of key points.
Both the number of the key points and number of
frames will impact on the time taken to producethe sequence of frames.
-
8/6/2019 Chapter 4 - Building Blocks Edt-010311_033357
52/53
Morphing
-
8/6/2019 Chapter 4 - Building Blocks Edt-010311_033357
53/53
Exercise
1)What is sound? What is the differentbetween sound and waveform? Pleaseelaborate more
2)What is the different between PAL andNTSC?
3)What is morphing?
4)
top related