clarke, r. j (2001) l213-08: 1 multimedia in organisations buss 213 lecture 8 modelling multimedia...
TRANSCRIPT
Clarke, R. J (2001) L213-08: 1
Multimedia in Organisations
BUSS 213
Lecture 8Modelling Multimedia Applications
Clarke, R. J (2001) L213-08: 2
NoticeAssignment 1 Results
Pick up your assignment 1 if you haven’t already done so:
check the marks sheet and make sure that marks have been included for all categories (where relevant) and that it is added up correctly
make sure that the total shown on the cover is the same as that shown on the results list (on my office door)
the final grade for the assignment will be rounded to an integer
Clarke, R. J (2001) L213-08: 3
Agenda (1)
in previous lectures we have considered the different types of media used in multimedia systems and the types of transformations performed on them
we have also considered multimedia systems from the point of view of the kind of design considerations used when building the interface
this lecture we consider what is behind the interface, the software architecture (abstractions) for multimedia implementation…
Clarke, R. J (2001) L213-08: 4
Agenda (2)
we consider the design of multimedia applications from an object-oriented approach (OOHDM)
then consider some of the types of building blocks or classes used in multimedia frameworks
these frameworks consist of media types and their properties and operations are also viewed using object-oriented programming
Clarke, R. J (2001) L213-08: 6
Object-orientation
In this lecture we consider the use of so-called ‘object-oriented’ development techniques as the basis for:Design of multimedia applicationsImplementation of Multimedia
Applications we describe both the design and
implementation aspects of multimedia as ‘modelling’ for our purposes
in this lecture we emphasis design
Clarke, R. J (2001) L213-08: 7
Object-orientation
thankfully however the implementation details of multimedia systems are hidden by the high-level tools we use during development (Director etc)
in this section we describe the general characteristics of object-orientation- these are broadly applicable in other development contexts as well
object-oriented approaches are a natural fit with multimedia systems- there are several reasons for this
Clarke, R. J (2001) L213-08: 8
Object-orientationMain Properties (after Borland 1991)
there are three main properties which characterise object-oriented development and languages which support object-orientation: Encapsulation: combining a record with the procedures
and functions that manipulate it to form a new data type- an object
Inheritance: defining an object and then using it to build a hierarchy of descendant objects, with each descendant inheriting access to all its ancestors’ code and data
Polymorphism: given an action one name that is shared up and down an object hierarchy, with each object in the hierarchy implementing the action in a way appropriate to itself
Clarke, R. J (2001) L213-08: 9
Object-orientationInheritance (after Borland 1991)…
as with other sciences and technologies a great deal of analysis and development is about classification- the creation of distinctions and there organisation into taxonomies
consider the category of ‘insect’, which has two divisions: Insects with visible wings, and Insects with hidden or no wings
Clarke, R. J (2001) L213-08: 10
Object-orientation… Inheritance (after Borland 1991)…
‘winged insects’ contain a large number of categories including moths, butterflies, flies etc. and each category has many subcategories
a scientist asks several questions when trying to classify a new animal or object How is it similar to the others of its general
class? How is it difficult?
Clarke, R. J (2001) L213-08: 11
Object-orientation… Inheritance (after Borland 1991) …
each different class has a set of behaviours and characteristics that define it
a scientist begins at the top of the tree and asks questions along the way eg. ‘wings’ or ‘no wings’ from the most general to the most specific
once a characteristic is defined all the categories beneath that definition include that characteristic…
Clarke, R. J (2001) L213-08: 12
Object-orientation… Inheritance (after Borland 1991)
once a new insect has been identified as a member of ‘flies’ you don’t need to identify your fly as having wings!
the new insect is said to inherit that characteristic from ‘flies’
insects
wingless winged
butterflies fliesmoths
new insect
Clarke, R. J (2001) L213-08: 13
Object-orientationPolymorphism (after Borland 1991) …
polymorphism is based on the Greek meaning ‘many shapes’- a way of giving an action one name that is shared up and down an object hierarchy with each object in the hierarchy implementing the action in a way appropriate to itself
the winged insects taxonomy is a good example of polymorphism in action- we would like each insect to be drawn on the screen for example- in fact this is a desirable property for all of our electronic insects…
Clarke, R. J (2001) L213-08: 14
Object-orientation… Polymorphism (after Borland 1991)
…but what is different for each object type is the way it must show itself on the screen for example bees have stripes on their body, but dragonflies don’t
part of the basic definition of insect would also implement so called virtual method called ‘Show’- which is resolved at program execution to display on the screen a bee rather than a dragonfly
Clarke, R. J (2001) L213-08: 16
Multimedia DesignObject Oriented Hypermedia Design Method
an Object Oriented Hypermedia Design Method (OOHDM) has evolved over recent years which applies a set of object-oriented design primitives to hypermedia design
Comprises four different activities: Conceptual Design Navigation Design Abstract Interface Design, and Implementation
Clarke, R. J (2001) L213-08: 17
Multimedia DesignOOHDM Activities …
each activity is performed as a mix of incremental, iterative and prototype-based development styles
during each activity a set of object-oriented models describing particular design concerns are built from previous iterations
Conceptual, navigation and interface design are separated in order to allow developers to concentrate on each one at a time
Clarke, R. J (2001) L213-08: 18
Multimedia Design… OOHDM Activities
the result of this separation is modular, reusable designs and produces a framework for reasoning about the design process while encapsulating design experience specific to each activity
design primitives can be mapped directly into non-object-oriented implementation languages or environments (eg. HTML) or into pure object-oriented environments
we describe each OOHDM’s activity in detail (we exclude implementation- considered latter)
Clarke, R. J (2001) L213-08: 19
Multimedia DesignOOHDM Conceptual Modelling (1)
Goal is to build a model of the application domain using object-oriented modelling principles and notations(like UML)
Output is a class schema where conceptual classes are built using aggregation and hierarchies like the taxonomy shown previously
Clarke, R. J (2001) L213-08: 20
Multimedia DesignOOHDM Conceptual Modelling (2)
Modelling conceptual entities as objects allows behaviour to be encapsulated inside them and therefore improves the design of complex applications
Conceptual models don’t just model the ‘domain’ of users and tasks but also can include complex algorithms and transactions applied to computational entities- permits a broad range of applications to be modelled
Clarke, R. J (2001) L213-08: 21
Multimedia DesignOOHDM Navigation Design
OOHDM does not prescribe any particular procedure to create navigation designs- but it does provide a method to iteratively develop them
Uses scenarios as a means to do this…
1. Determine user profiles (types of users), identify user tasks
2. Scenario collection3. Analysis of scenarios- create a simple
representation of the navigation path described by the scenario (the designer will have to fill in incomplete information, such as implied indexes, missing orderings, exception handling, etc..)
4. Synthesis of a partial context diagram, specifying navigation in contexts that support the task described in the scenario
5. Synthesis of final context diagrams through a process of union and amalgamation of partial schemas, produced in the analysis of each scenario (in Step 4)
Clarke, R. J (2001) L213-08: 22
Multimedia DesignOOHDM Abstract Interface Design
in the stage the interface objects that will be perceived by the user are defined
there is a distinction between navigation operations and interface operations- not everything in an interface is navigation oriented
it it useful to design interfaces in an abstract way- to achieve for example independence in the implementation environment
Clarke, R. J (2001) L213-08: 23
Multimedia DesignOOHDM Design Methods (Rossi et al 1999)
Conceptual ModelNavigation Views
Object Oriented Views of Conceptual Model
Interface Model(Presentation)
Clarke, R. J (2001) L213-08: 25
Multimedia Framework
multimedia represents a challenging type of programming with respect to detailed technical implementation because of: the variety of different types of media that have to be
handled and the fact that the characteristics of the hardware change all
the time (non-standardisation as well as technical development)
an advantage of OOP is that it can encapsulate information that is hide details of the particulars of a given media and the specifics of particular hardware
programmers only need to know what is referred to confusingly as the interface (object NOT user interface) rather than the details of the object implementation
Clarke, R. J (2001) L213-08: 26
Multimedia Framework Digital Content Workflows and Modularity
the workflows used in developing multimedia content are increasingly becoming digital (as we have seen in previous lectures for example when we discussed VR)
modern digital content workflows now include what used to be standalone and isolated devices- these devices are becoming virtually integrated together at the desk top or are now available as software rather than hardware
OOP can handle the complex interfacing needed to support interconnectivity and modularity between functions in digital content workflows- the interface between for example an audio effects processor (Sound Forge) and a digital sequencer (CakeWalk)
Clarke, R. J (2001) L213-08: 27
Multimedia Framework Extensibility & Portability
the requirements of multimedia systems are constantly evolving and relatively non-standard- this means that there will always be a need to extend applications in ways which are not always predictable
OOP supports this so called extensibility by providing standard ways in which programs can be enhanced
multimedia systems should be cross-platform as much as this is possible- in OOP the platform dependencies are made explicit in order for the code to be made portable - this means that the application need not be completely rewritten
as demonstrated in the following slides…
Clarke, R. J (2001) L213-08: 28
Object-Oriented MultimediaSpecialising a Framework
Abstract classes
Concrete classes
Platform 2
Abstract classes
Concrete classes
Platform 3
Abstract classes
Concrete classes
Platform 1
Abstract classes
Specialisation
Environments
Framework
Systems programming interfaces
Application programming interface (API)
Clarke, R. J (2001) L213-08: 29
Object-Oriented MultimediaSpecialising a Framework
Abstract classes
Concrete classes
Platform 2
Abstract classes
Concrete classes
Platform 3
Abstract classes
Concrete classes
Platform 1
Clarke, R. J (2001) L213-08: 30
Object-Oriented MultimediaSpecialising a Framework
Abstract classes
Concrete classes
Abstract classes
Concrete classes
Abstract classes
Concrete classes
Platform 2 Platform 3Platform 1
Clarke, R. J (2001) L213-08: 31
Object-Oriented MultimediaSpecialising a Framework
Abstract classes
Concrete classes
Abstract classes
Concrete classes
Abstract classes
Concrete classes
Platform 2 Platform 3Platform 1
Clarke, R. J (2001) L213-08: 32
Object-Oriented MultimediaSpecialising a Framework
Abstract classes
Concrete classes
Abstract classes
Concrete classes
Abstract classes
Concrete classes
Platform 2 Platform 3Platform 1
Clarke, R. J (2001) L213-08: 33
Object-Oriented MultimediaSpecialising a Framework
Abstract classes
Concrete classes
Abstract classes
Concrete classes
Abstract classes
Concrete classes
Platform 2 Platform 3Platform 1
Clarke, R. J (2001) L213-08: 34
Object-Oriented MultimediaSpecialising a Framework
Abstract classes
Concrete classes
Abstract classes
Concrete classes
Abstract classes
Concrete classes
Platform 2 Platform 3Platform 1
Clarke, R. J (2001) L213-08: 35
Object-Oriented MultimediaSpecialising a Framework
Abstract classes
Concrete classes
Abstract classes
Concrete classes
Abstract classes
Concrete classes
Platform 2 Platform 3Platform 1
Clarke, R. J (2001) L213-08: 36
Object-Oriented MultimediaSpecialising a Framework
Abstract classes
Concrete classes
Abstract classes
Concrete classes
Abstract classes
Concrete classes
Platform 2 Platform 3Platform 1
Clarke, R. J (2001) L213-08: 37
Object-Oriented MultimediaSpecialising a Framework
Classes ClassesClasses
Platform 2 Platform 3Platform 1
Clarke, R. J (2001) L213-08: 38
Object-Oriented MultimediaSpecialising a Framework
Platform 2 Platform 3Platform 1
Classes ClassesClasses
Clarke, R. J (2001) L213-08: 39
Object-Oriented MultimediaSpecialising a Framework
Platform 2 Platform 3Platform 1
Classes ClassesClasses
Clarke, R. J (2001) L213-08: 40
Object-Oriented MultimediaSpecialising a Framework
Platform 2 Platform 3Platform 1
Classes ClassesClasses
Clarke, R. J (2001) L213-08: 41
Object-Oriented MultimediaSpecialising a Framework
Classes ClassesClasses
Platform 2 Platform 3Platform 1
Clarke, R. J (2001) L213-08: 42
Object-Oriented MultimediaSpecialising a Framework
Classes ClassesClasses
Platform 2 Platform 3Platform 1
Clarke, R. J (2001) L213-08: 43
Application Application Application
Object-Oriented MultimediaSpecialising a Framework
Classes ClassesClasses
Platform 2 Platform 3Platform 1
Clarke, R. J (2001) L213-08: 45
Media Classes (1)
the low-level implementation of distinct types of media- that is the computer-based representation of media- can also described in object-oriented terms
in this section we consider how to model different media types and there properties and operations using object-oriented programming
the resulting building blocks are referred to as classes which can then be directly included into programs or further extended
Clarke, R. J (2001) L213-08: 46
Media Classes (2)
Gibbs and Tsichritzis (1995) in a now ‘classic work’ showed how media could be considered in detail using a multimedia framework which includes: Media classes Transform Classes Format Classes Component Classes
here we only briefly describe media classes in object oriented terms (in the following slides)
however the student with a technical background and interest is referred to Gibbs and Tsichritzis (1995) for further discussion
Clarke, R. J (2001) L213-08: 47
Media Classes (3)Objects, Instance Variables, Methods …
Objects are a programming constructs that encapsulate or contain its state- a collection of data items- and its behaviour- collection of operations
the object’s data items are called instance variables and its operations are called its methodsObject: myMp3PlayerInstance Variable: currentTrackMethod: PlayTrack(trackNumber)
Clarke, R. J (2001) L213-08: 48
Media Classes (4)… Objects, Instance Variables, Methods …
invoking or using a method can be thought of as sending a message to an object
this message contains both the method name and any arguments needed by the method
the syntax used to invoke the method method differs from language to language (and the stuff of vigorous debate) but the following form of object name followed by method- is commonly used:
myMp3Player.PlayTrack(4)
Clarke, R. J (2001) L213-08: 49
Media Classes (5)… Objects, Instance Variables, Methods …
the result of invoking a method often leads to a change of state of the object for example currentTrack instance variable is now 4
in fact in many OOP languages, the methods of objects are often the only interface by which objects are manipulated- you cannot modify an objects instance variables directly
Clarke, R. J (2001) L213-08: 50
Media Classes (6)Class Hierarchy Diagrams
following the work of Gibbs and Tsichritzis (1995) classes can be shown as class hierarchy diagrams ovals are classes lines depict superclass/subclass relationships the line points from superclass to the subclass classes (ovals) in dark blue are abstract classes
while those in pale pink are concrete classes
media are divided into types- each type is represented by a class…
Clarke, R. J (2001) L213-08: 51
Media Classes (7)…Class Hierarchy Diagrams
Media
Text
Graphic
ImageTemporalMedia
Audio
VideoMusic
Animation
Clarke, R. J (2001) L213-08: 52
Media Classes (8)Media Objects…
instances of media classes are called media objects
a media object consists of a descriptor and and a media value
descriptors bundle together attributes of media objects such as their values correspond to the actual data used to represent artifacts
Clarke, R. J (2001) L213-08: 53
Media Classes (9)… Media Objects…
010010010110010001
Media value(media data)
Mediaartifact
Mediadescriptor
represents
describes
MediaObject
Clarke, R. J (2001) L213-08: 54
Media Classes (10)General Categories of Methods
most general media classes have two general categories of methods:querying- allows media to determine the
media types that are present in an environment
editing- cut, copy and paste operations for any general media type
Clarke, R. J (2001) L213-08: 55
Media Classes (11)Video Classes & Sequence Elements
RawVideo
Video
CompressedVideo
MpegVideo
Frame
VideoElement
CompressedFrame
MpegFrame
Instances of Classes Corresponding Classes
Clarke, R. J (2001) L213-08: 56
Media Classes (12)Audio Classes & Sequence Elements
RcmAudio
Audio
CompressedAudio
AdpcmAudio
Sample
AudioElement
CompressedSample
AdpcmSample
Instances of Classes Corresponding Classes
Adpcm and AdpcmSample are abstract classes
Clarke, R. J (2001) L213-08: 57
Media Classes (13)Animation Classes & Sequence Elements
Instances of Classes Corresponding Classes
Adpcm and AdpcmSample are abstract classes
Animation
EventBasedAnimation SceneBasedAnimation
AnimationElement
AnimationEvent Scene
Clarke, R. J (2001) L213-08: 58
Media Classes (14)Music Classes & Sequence Elements
EventBasedMusic
MidiMusic
ScoreElement
MusicElement
MidiEvent
Instances of Classes Corresponding Classes
ScoreBasedMusic
Music
MusicEvent
Clarke, R. J (2001) L213-08: 59
References
Gibbs, S. J. and D. C.Tsichritzis (1995) Multimedia Programming: Objects, Environments and Frameworks Chapter 2: Media Types, ACM Press Books/Addison-Wesley Publishing Company, 15-78
Rossi, G.; Schwabe, D. and F. Lyardet (1999) “Integrating Patterns into the hypermedia development process” The New Review of Hypermedia and Multimedia 1999 (3), 59-80
Borland, Inc. (1991) Turbo Pascal for Windows: User Guide Borland International