the online resource selection instructional design script

7/28/2019 The Online Resource Selection Instructional Design Script

1/31

2010 Old City Publishing, Inc.

Published by license under the OCP Science imprint,

a member o the Old City Publishing Group

Tech., Inst., Cognition and Learning, Vol. 8, pp. 6796

Reprints available directly rom the publisher

Photocopying permitted by license only

The Olie Resore Seletio Istrtioal

Desig Sript (ORSIDS) ad Impliatios

for the Widespread Diffsio of

Learig Objets

Deborah elizabeth Cohen*

Sogang University

This article discusses a study that investigated the development o the OnlineResource Selection Instructional Design Script (ORSIDS) and its efcacy inassisting college-employed instructional designers in guiding aculty with select-

ing learning objects or their online courses. The study established the value oteaching instructional design methodology to aculty members to enable them todefne the curricular needs or learning objects or their online courses and toassist them in searching or and evaluating them. It additionally emerged romthe study that more is needed along the lines o searching inrastructure andstandardization to acilitate more reliable searches or learning objects.

Keywords: Learning objects; multimedia; metadata; instructional designers; online

learning; eLearning; instructional design; online faculty; professional development;

media selection; technology integration; metadata.

InTRODucTIOn

While eLearning is becoming prevalent in higher education, a review o the

literature suggested that aculty in higher education must learn new skills to tran-

sition into online teaching, both as course designers and instructors. Most college

aculty lack the knowledge and skills necessary to design quality online courses

(Davison-Shivers, 2002). I college aculty are expected to design online courses,

their lack o online course design skills must be remediated.


2/31

68 Deborah elizabeth Cohen

Among these skills is the ability to implement e-learning eatures including

the use o learning objects to deliver content. There are many online educational

resources, or learning objects, that can be incorporated into online classes. While

large amounts o money have been invested in these resources, educators have

been slow to adopt them. Gosper and collaborators (Gosper, Woo, Gibbs, Hand,

Kerr, & Rich, 2004) state that lack o adoption o learning objects is caused by

inadequate educator knowledge o how to use these resources.

Colleges are increasingly hiring proessional instructional designers to work

with aculty to design and develop online course materials and to serve as change

agents, diusing technological innovations into education and acilitating changes

in how academics think about teaching and learning. Instructional designers can

acilitate the adoption o learning objects, collaborating with aculty members toselect such materials based on sound instructional design principles.

Knowledge o the instructional design process makes skilled online course

design, including media selection, possible. Media selection techniques make it

possible to knowledgeably select learning objects. Prior to the existence o online

courses, Instructional Design and Technology theorists developed criteria or

media selection; these criteria can orm a oundation or making decisions about

which learning objects should be selected or inclusion in online courses. How-

ever, as the online environment oten results in the use o multimedia, two ormore media presented simultaneously, the media selection process must be

expanded to consider new issues that arise with the use o multimedia. Essentially

multimedia selection diers rom media selection because issues o cognitive

load must be considered. Considering the instructional attributes o multimedia

and how learners cognitively integrate inormation provides a oundation upon

which to assess whether to use and how to implement learning objects in online

courses.

ORSIDS (the Online Resource Selection Instructional Design Script) is a pro-

cess script based on media selection techniques whose purpose it to acilitate the

process o online searching or learning objects. Its efcacy and process o devel-

opment will be the topic o this article. TICL has presented a number o articles

related to learning objects, with several ocusing on issues also identifed in the

process o developing ORSIDS. They include the need or:

The development o search processes that integrate the perspective o instruc-

tional designers (Friesen, 2006).

A pre-analysis o the content domain (Scandura, 2006) and

Metadata specifcations extended to better represent knowledge representation


3/31

orSiDS anD impliCationSfortheWiDeSpreaDDiffuSionof learningobjeCtS 69

However, ORSIDS is unique in integrating the media selection techniques

developed by leaders in the instructional design eld into a methodology or

searching or learning objects.

Overview of Study

This study (Cohen, 2005) developed and tested a process script to assist

college-employed instructional designers in guiding aculty with selecting digital

multimedia learning objects or their online courses. This process refected the

context o the online environment and the cognitive constraints associated with

multimedia.

The purpose o the study was to see whether the proposed solution, the Online

Resource Selection Instructional Design Script (ORSIDS), could satisactorilyaddress three problems, one overarching problem and problems with two solu-

tions put in place to mend the rst:

The overarching problem: a lack o online pedagogical skills possessed by

online aculty.

The two sub-problems:

The less than eective deployment o the college-employed instructional

designers increasingly being hired by colleges and universities to workwith online aculty.

The lack o educator adoption o online resources, particularly the learning

objects and learning object repositories into which numerous organizations

world wide have poured substantial nancial resources.

The study investigated the ollowing research questions.

What steps are necessary to develop a process script that will help instruc-1.

tional designers guide online aculty in selecting online resources or their

online classes?

What criteria set must that script satisy in order to be considered successul?2.

What will be the best ormat and composition to enable the script to meet the3.

specied criteria?

How can a set o criteria candidates be established?4.

How can the criteria set be validated?5.

The script was based upon the oundation o the research ndings on mediaselection rom the eld o Instructional Design and Technology (Clark, 1999;


4/31


& Anderson, 1992; and Romiszowski, 1988). As multimedia selection makes it

necessary to consider issues related to cognitive load (Samaras, Giouvanakis,

Bousiou, & Tarabanis, 2004), the script was additionally based upon fndings by

Mayer and colleagues (Clark & Mayer, 2003) related to cognitive load.

This study (Cohen, 2005) was based on the ollowing premises:

When learning objects are comprised o multimedia, they can be thought o

and treated in a similar way

Multimedia share the attributes o media, but since multimedia consists o two or

more media, there is the need to consider constraints related to cognitive load

By leading online aculty members through a process based on instructional

design methodology, these aculty members can be prompted to identiy theircurricular needs or learning objects

A cognitive tool, or script, can be developed to assist instructional designers in

implementing this process

Abbreviated Review of Literature

Learning Objects

Many educational resources exist online. Online instructors can incorporatethem into their classes or use with a wide range o teaching activities. Such

online resources include digital materials displayed on websites such as the PBS

site accompanying the Ken Burns jazz series (http:www.pbs.org/jazz/lounge/) to

more structured environments and data stores including managed and virtual

learning environments and digital libraries as well as inormation gateways and

portals (Conole, 2002). OER Commons (http://www.oercommons.org/) is an

online open learning network where educators can share and access colleagues

course materials, evaluating them through Web 2.0 eatures including metatags,

reviews, and social networking. The Multimedia Educational Resources or Lear-

ning and On-Line Teaching (MERLOT) (http:www.merlot.org/help/FAQ.po) is a

national gateway to web-based peer-reviewed learning materials and a successul

demonstration project (Johnson, 2003). Learning object repositories are another

example o online searchable collections o digital educational resources that are

available online (Boyle, 2002).

Learning objects have several defnitions. The IEEE defnition developed dur-

ing the standardization process (IEEE, cited in Conole, 2002, p. 5) was a learn-

ing object is any entity, digital or non-digital, which can be used, re-used andreerenced during technology-supported teaching. The most common defnition


5/31


materials pictures, documents, simulations coupled with a clear or measurable

learning objective or designed to support a learning process.

While quality online resources can add a great deal to an online course, they

are expensive and time consuming to produce. Designing such materials is out-

side o the scope o what most individual instructors can reasonably be expected

to do as part o their routine workload.

To acilitate the development and sharing o such resources, governments

around the world have spent large sums o money on initiatives to develop learn-

ing objects, methods through which to fle and search them (metadata), and online

repositories in which to store them (Friesen, 2004). However, there has been a

general lack o adoption o learning objects and repositories or them.

Gosper et al. (2004) fnds that a major reason or the reluctance o educa-tors to share and reuse these objects is a lack o understanding o how such

objects can apply to the learning outcomes they plan to achieve. Additionally

methods through which to customize the objects and the applicable copyrights

are hard to understand. Friesen (2004) compares educator resistance to learning

objects with other technology-originated innovations introduced into educa-

tional settings. He writes that the problem with such innovations resulted rom

the act that such innovations are introduced bearing the stamp o their tech-

nical origins rather than in terms meaningul to educators. Yet when educatorsappear to resist such innovations, they are blamed or not readily adapting to

them. Friesen cites Rogers (1962) who fnds that the rate o adoption o innova-

tions by educators increased signifcantly when the innovations display sim-

plicity, compatibility with the practices o the environment, and apparent

advantages as compared with business as usual. Thus the provision o learn-

ing objects and related inrastructure alone will not lead to their adoption (Gos-

per et al., 2004). A bridge needs to be created enabling educators to understand

their use.

Johnson (2003) fnds that an enabler o educational learning object use

includes learning design, or the theories and techniques o teaching and learning

that ensure successul learning outcomes. The fnding is that while such a body o

knowledge exists and is sufcient, it is not widely understood. To make the cir-

cumstances surrounding the use o learning objects and other online educational

resources more avorable or adoption, such knowledge must be disseminated.

Instructional designers are perectly situated to disseminate this inormation.

Knowledgeable about learning theory and in the use o technology, skilled com-

municators who are amiliar with the needs o educators and the higher educationcontext, the instructional designer is well equipped to serve as a bridge between


6/31


Instructional Designers

Because the design and development o online courses and environments

requires considerable skill and knowledgeable planning, instructional designers

are increasingly being hired to work in colleges and universities (Pan, Deets,

Phillips & Cornell, 2003). The instructional designer unction in higher education

is two-old: instructional designers (1) design and develop online course materi-

als and (2) acilitate changes in how academics think about teaching and learning

(Torrisi-Steele & Davis, 2000). (While instructional designers work in many set-

tings, the discussion will be limited here, due to space limitations, to instructional

designers employed in higher education.)

The instructional designer translates the needs o the aculty member into a

plan to produce a product to meet that clients needs (Liu, Gibby, Quiros &Demps, 2002). The instructional designer guides the client through the design

process, eliciting needed inormation and providing the necessary inormation to

the client to enable him or her to make the correct design decisions. Instructional

designers are essentially problem solvers who invent solutions as a routine part o

their work (Schwier, Campbell & Kenny, 2004). Because the instructional

designer has knowledge o both technology and educational needs, his or her

communication skills make it possible to serve as a bridge between technological

innovations and the educator (Liu, Gibby, Quiros, Demps, 2002).The main goal o the instructional designer is to select, sequence, and synthe-

size content or instructional purposes and develop a desired product without, in

most cases, previous content expertise (Keppell, 2001). The instructional designer

role o higher education change agent is also an important one (Schwier, Camp-

bell & Kenny, 2004). Essentially, the instructional designer acilitates innovation

in colleges and universities through taking an active role in the proessional devel-

opment o teaching sta, enabling them to develop skills to produce online

courses and to teach online (Torrisi-Steele & Davis, 2000). This instructional

designer role can include introducing aculty to online learning objects.

Instructional Design Scripts

While scripts are rarely discussed in the instructional design literature, in

practice most instructional designers rely upon them to do their jobs. Keppel

(2004) fnds that since instructional designers almost always work in unamiliar

content areas, they rely upon experts and utilize a design model consisting o a set

o representations and generic strategies to accomplish their goals. These goals

include eliciting needed inormation rom the aculty subject matter expert andguiding the aculty member in the design process. The scripts inherent in such a


7/31


subject matter expert. Such scripts provide a shortcut or the instructional

designer in his or her work, making it possible to accomplish goals while using

less o the subject matter experts time.

The relationship between instructional designer and subject matter expert

aculty member is collaborative as they assist each other in what Keppell (2004)

defnes as an elicitation/conceptualization process that meets their mutual

goals. Keppell (2001) fnds that instructional design models lack the conceptual-

ization elicitation script needed by instructional designers to conceptualize the

unamiliar material presented by subject matter experts. As the development pro-

cess or higher education courses containing new media can be even more com-

plex than traditional courses (Liu, Gibby, Quiros, & Demps, 2002), the need or

such scripts becomes even greater.The purpose o the script developed in this study was to serve as a job aid to

support the consulting and conceptualization/elicitation process between the

instructional designer and online aculty member in the area o the selection o

learning objects. This study was based on the hypothesis that a script to guide

aculty members with the media selection process required to select learning

objects would greatly aid the instructional designer in his or her role o change

agent (Schwier, Campbell & Kenny, 2004) diusing the innovation o learning

objects and other online resources (Friesen, 2004).Select methods, media and materials is the third step o the ASSURE

model (Smaldino et al., 2005, p. 59). Included in this step is an appraisal checklist

o selection criteria or each media type that begins with the question: does it

match the curriculum? The goal o this study was to develop a script, a job aid,

to direct a consulting session to answer this question. A process - and script based

on the process - were created or the instructional designer to guide the online

aculty member in fnding learning objects and to then determine whether they

were appropriate or the curriculum. Determining whether or not a resource is

appropriate or the curriculum is based on the knowledge o instructional attri-

butes o various media and how they link to dierent learning outcome types.

This knowledge was derived rom the literature on media selection models, which

have had an important place in the feld o Instructional Design and Technology

(Richey & Nelson, 1996).

Media and Multimedia

Online educational resources are comprised o media and multimedia. A

medium is a channel o communication that carries inormation between a source,such as a computer, and a receiver, such as a learner (Smaldino, Russell, Heinich,


8/31


and graphics; video such as ull motion video, time lapse, and montage; and

audio, such as music, sound eects, and narration. Media are considered instruc-

tional media when they carry messages with an instructional purpose.

This article reers to both media and multimedia. The distinction is as ollows:

the term media is used when a single communication channel (such as a graphic) is

being reerred to; the term multimedia is used when a combination o two or more

such channels, such as a graphic combined with narration, is being discussed.

Media and multimedia can make a major curricular contribution. The ability

o media to represent real objects can be o great educational value. Media allow

designers to communicate about objects, representations, and ideas about reality

that would not otherwise be possible. The representational eatures o media are

useul when communicating about:

A procedure that is too risky to undergo in class

An object that is not easily accessible (such as a planet, a oreign artiact, a

rare song, or microscopic organism)

A process too lengthy to accomplish in a semesters time

A subject about which it is too hard to do the necessary experimentation to

obtain needed evidence (Djikstra, Jonassen, & Sembill, 2001).

With the use o media, the designer can create a learning environment

enabling the student to actively consult a constructed reality and potentially oper-

ate upon it (Djikstra, 2001).

A number o researchers emphasize the importance o dealing with media as

types, that is, or the attributes they provide and the learning strategies that they

can implement (Smith & Ragan, 1999). A media attribute is the potential o a

given medium to present inormation o a certain kind. It can acilitate learning

or specifc outcomes and certain kinds o learners. For example, to present a

song, a medium that provides the ability to play a song is necessary. Audio is the

logical medium to accomplish such a unction. Thinking about media in terms o

how their unctions contribute to teaching and learning, or their instructional

attributes, makes it possible to select learning objects that will result in maximum

learning eectiveness. Researchers have also investigated how the attributes o a

medium can support the individual cognitive processes needed or specifc learn-

ing tasks (Smith & Ragan, 1999).

There is more potential or the use o multimedia in the online environment

than in previous educational settings. Because the online environment o todayhas the capacity to deliver multimedia, a systematic approach to providing guid-


9/31

orSiDS anD impliCationSfortheWiDeSpreaD DiffuSionof learning objeCtS 75

Researchers who deal with cognitive load issues state that learning can actually

be depressed by the use o e-learnings technological capabilities to simultane-

ously deliver text, audio and video i designed unskillully (Clark & Mayer, 2003;

Samaras, Giouvanakis, Bousiou and Tarabanis, 2004). Poorly designed multime-

dia can place too much o a processing burden on the learner, interering with the

learning process (Sweller & Cooper, 1985).

Cognitive Learning Theory

Cognitive learning theory is learning theory infuenced by cognitive psychol-

ogy, which ocuses upon how people perceive, learn, remember and think about

inormation. An important eature o cognitive learning theory is the concept o

cognitive load(Samaras et al., 2004). Cognitive load is the amount o cognitiveprocessing that a particular task requires o a learner (Foshay, Silber & Stilnicki,

2003), or in other words, the amount o mental resource required by a task (Clark

& Mayer, 2003). Cognitive load theory assumes that human memory has two

channels or processing inormation visual and auditory and each channel has

a limited capacity or processing inormation (Mousavi, Low & Sweller, 1995).

Since multimedia provides input to both the visual and auditory channels, it is

important to limit the amounts o input to each channel so neither channel

becomes overloaded. For example, it is helpul to accompany visuals with audionarrative rather than text to describe visuals and present content. This splits the

input between the audio and visual channels. When learners must use the visual

channel to simultaneously process the graphics and the printed words that reer to

them, this concentrates the input in the visual channel and may strain cognitive

resources. Too much visual input can be particularly dicult or learners when

presented at the same time and at a rapid pace (Clark & Mayer, 2003). Cognitive

load is a primary issue to consider in the assessment o online multimedia learn-

ing objects (Cohen, 2004). In analyzing such a learning object, an assessment

should be made o whether cognitive load issues have been considered in its

design and whether its cognitive processing demands seem prohibitive.

Method Overview

Development methodology was used in the study to create the Online Resource

Selection Instructional Design Script, ORSIDS. The Pre-Design Phase consisted

o the review o literature and other inormation gathering. Establishing ormative

and summative review committees also occurred in this phase. TheDesign Phase

consisted o requirements development and validation to speciy the ideal compo-sition o ORSIDS. In this phase, members o an Expert Panel validated the candi-


10/31


review o literature. To do this, the Expert Panel participated in a series o meet-

ings using a customized Nominal Group Technique (NGT) process, the Proposal

Review process (Delbecq, Van de Ven. & Gustason, 1975). The Proposal Review

Process is recommended or planning situations and specifcally proposal review;

it is directed toward identiying the strengths and weaknesses o proposed solu-

tions and provides a vehicle to introduce modifcations to improve them.

Ater requirements development and validation, a detailed outline was cre-

ated. From the detailed outline, an alpha version o the process script was designed

in theDevelopment Phase.

In theDevelopment Phase, ORSIDS was developed and then refned through

iterative cycles o developmental and ormative evaluation. There were our tests

in all. Field testing was conducted by the researcher with two online instructors.Next, two instructional designers were trained to deliver the script in the pilot

study. A pilot study using the nonparticipant, naturalistic observation method (Gay

& Airasian, 2000) ollowed. The researcher observed and collected data as two

instructional designers delivered the script to one online aculty member each. The

researcher repeatedly improved ORSIDS based upon the session results ater

debriefng with the instructional designers. The script was revised fve times: once

ater each test, and once to incorporate eedback ater the initial Instructional

Designer Training. At the end o the testing, the researcher completed a fnal drato ORSIDS and prepared an Observation Report and questionnaire or summative

evaluation by the Expert Panel. Product evaluation occurred though summative

evaluation by the Expert Panel who validated the original requirements.

Study Results/Finding

This was a developmental study to create and test a process script (ORSIDS)

to orchestrate an instructional designer/aculty member consulting session, whose

goal was to search or and evaluate a useul learning object. The study yielded

many interesting results related not only to the research questions delineated at the

beginning o this article but to broader issues o the relatively new feld o learning

object search as well. The fndings will be presented here by study phase.

Pre-Design phase

Data rom the Pre-Design phase resulted rom a literature review and an

expert interview. The deliverables rom this phase included the requirements can-

didates or a process script or online resource selection and some o the content

needed or the product. Testing would ultimately fnd that the process discoveredin this phase based on the media selection literature was eective or the selection


11/31


based on media selection theory, proved less eective due to inadequate inra-

structure or educational learning object search.)

The literature review was quite extensive and provided much o value. The

literature review contributed to the study in a number o major ways. It provided

guidance or the development o the process and specically the instructional

design steps to include. The media selection literature (Briggs & Wager, 1981;

Kemp, 1980; Reiser & Gagne, 1983; Reynolds & Anderson, 1992; Romiszowski,

1988; and Smaldino, Russell, Heinich & Molenda, 2005) was an important infu-

ence. Media selection models are based on sound instructional design principles

and delineate simple, systematic processes or making media selection decisions.

The ORSIDS candidate requirements refected many elements o the media selec-

tion literature.The review o literature also provided an understanding o the most construc-

tive ways with which to think about the design and use o multimedia and the role

o instructional message design. The work o several authors (Djikstra, Jonassen,

& Sembill, 2001; Jonassen, 2001; Kozma, 2001b; and Rouet, Levonen &

Biardeau, 2001) made major contributions to the approach towards multimedia

used in this study and the understanding o multimedia uses underpinning several

candidate requirements. Mayer (1999) wrote o empirically valid scientic stud-

ies that proved the relationship between the design o instructional messages andhow people do or dont learn rom them. Mayers work on instructional message

design was an important philosophical underpinning o this study and underlay

several candidate requirements. Additionally an Expert Interview with Steven

Smith (personal communication, October 6, 2004) contributed an online search-

ing technique that ormed the oundation or one o the candidate requirements.

Design Phase

Requirements Validation. This developmental study was to result in two deliv-

erables: (1) a script, ORSIDS, based upon (2) a process or an ideal instructional

designer/aculty member consulting session. The requirements were divided into

Process Inputs, Process Steps, Process Outputs, Format (o the script), Reerence

Materials (in the script) and Eciency/Cost o Using ORSIDS. The Expert Panel

introduced several modications consisting o amplication and added specicity

to the requirements candidates related to the context in which the process would

actually be used. There was discussion related to instructional design methodol-

ogy and the order in which some o the process steps should take place. The panel

also recommended the inclusion o instructional message design guidelines romFleming and Levie (1993) in addition to those rom Mayer (1999). See Table 1 or


12/31


Development Phase

The development phase consisted o creating a topic outline or the script based

upon the above validated requirements and rom there developing a script drat based

upon the topic outline. The script was essentially a job aid or an instructional

designer. Its purpose was to delineate a consulting session between an instructional

designer and aculty member that would ideally result in an appropriate course learn-

ing object. The steps to take place in the consulting session (which correspond to

Requirements 4 21 o the Validated Requirements in Table 1) appear in Table 2.

TABLE 1

Validated Requirements or ORSIDS.

Inputs to Process

Requirements #1 3 Analysis o learners, a statement o each learning outcome,context

Steps in the Process Script

Learning Outcome Analysis Steps

Requirements #4 6 Decide which learning outcome(s) (or portions o a learningoutcome) should be ulflled with an online resource, taskanalysis, choose learning outcome category

Research and Gather OnlineResources Steps

Requirements #7 10 Choose an instructional method, Lecture/demonstration o multimedia ormats, choose a media ormat, survey onlinelearning resources/learning objects.

Learning Resource Evaluation Steps

Requirement #11 Can the learning object ulfll the learning outcome task analysis requirements?

Requirement #12 Appropriate instructional and physical attributes?

Requirement #13 Accepts student input in a manner that appropriately dem-

onstrates his or her knowledge?

Requirement #14 Appropriate or the learner?

Requirement #15 Credible?.

Requirement #16 Longevity as a link on the web?

Requirement #17 Acceptable amount o cognitive load?

Requirement #18 Compatible with Mayers as well as Fleming and Leviesprinciples or instructional message design?

Requirement #19 Compatible with fndings related to media and multimediaattributes?

Requirement #20 Aordable?Requirement #21 Has the need to accommodate various learning styles been

acknowledged with selection o the learning object that will


13/31


TABLE 2

ORSIDS Steps.

Step

1 Help the aculty member to determine which learning outcomes are good candidates or

ulfllment with online learning objects

2 Guide the aculty member in doing a task analysis or the specifed learning outcome.

3 Help the aculty member to determine the learning outcome category to which the specifed

learning outcome belongs.

4 Help the aculty member to determine which instructional methods may be appropriate or

the learning outcome.

5 Help the aculty member to ocus on the instructional and physical media attributes needed

by the online learning object to choose some potential media ormats.

6 Provide a small lecture/demonstration o various multimedia ormats and methods and their

uses.

7 Search together or online learning objects and provide search tips to the aculty memberenabling him or her to continue to search independently.

8 Evaluate one or possibly two learning objects or possible use

Inputs to Proess

Outputs of the Proess SriptRequirement #22 One or more appropriate online learning objects or each

selected learning outcome or utilization in the online course

Proess Sript Format

Requirement #23 The script will consist o text on paper and will include adecision tree. It will be supplemented by demonstrations onthe computer or the aculty member by the instructionaldesigner.

Requirement #24 It can be copied and used or each new instructional designer/ online aculty member online resource selection interaction.

Requirement #25 It will consist o directions in the second person or theinstructional designer, an actual script to be delivered verba-tim or improvised orm by the instructional designer, work-sheets to be flled in and some reerence materials.

Requirement #26 The process script will be clear, simple, intuitive, easilynavigatable, and usable

Referene Materials Inluded

with the Sript

Requirements #27 32 Searching Tips, A Cognitive Load and Instructional Mes-sage Design Job Aid, Multimedia Attribute Heuristic Guide,Learning Outcome Categories, Defnition o Instructional

Methods, Explanation o Task Analysis

TABLE 1. continued


14/31


Subsequent refned drats were developed based upon the results o testing.

Figure 1 shows the frst two pages o the fnal version o ORSIDS.

The purpose o the our testing sessions was twoold: (1) to test the usability

o the script and to allow or refnements based upon the test results and (2) to

test the efcacy o the underlying media selection process upon which the script

was based.

ScRIPT

Interaction keywords What is Said

Introdue the Meeting Agenda Instrutional Designer (ID): Thank you orcoming today. We have a lot o ground to cover

in our meeting in order to fnd and make

wise discriminations among online learning

resources. We will do the ollowing things.

Determine i learning outcome is a good

candidate

First well determine which o your learning

outcomes are good candidates or ulfllment

with online learning resources. Well choose

only one or two to work with and ocus on one

at a time.

Do task analysis Then well do a task analysis or the specifedlearning outcome(s).

Determine learning outcome categories Then Ill teach you about learning outcome cat-

egories and well decide which category the

learning outcome belongs to.

Assign instructional methods Next Ill introduce you to instructional methods

so we can determine which methods are appro-

priate or the learning outcome.

Analyze media attributes Next well learn about media ormats and the

instructional and physical attributes that they

provide. Then well think about the attributes

needed to ulfll your learning outcome so that

we can look or them in an online learning

resource

Multimedia demo Then Ill provide a small lecture/demonstration

on the computer o various multimedia ormats,

methods and their uses.

Search or resources Then we will search together or online resources

or one o your learning outcomes.

Evaluate resources Finally well choose one or two resources to


15/31


Researcher-Conducted Field Tests

The researcher conducted the rst two eld tests to iron out the major

kinks in the script ormat beore handing it o or use by instructional designer

test subjects. Many ormat problems in the script were discovered as a result o

these tests. However, the researcher was able to deliver eective consulting ses-

sions despite faws in the Script ormat because o an intimate knowledge o the

underlying media selection process. The rst eld test was conducted by theresearcher with a aculty member inexperienced with online teaching and

Step 1: Determine which learning outcomes

are good candidates or ulllment with online

learning resources.

Goal: Determine is learning outcome a good

candidate?

ID: Our rst step is to determine whether one or

more learning resources is needed, and i so

what kind. Are there one or more o your learn-

ing outcomes that you would like to ulllthrough use o an online learning resource

because you believe it is needed to get your edu-

cational point across?

1A) Faculty member (FM): Yes (decisively).

Faculty member (FM) gives good pedagogical

answer (IA). Go to Step 2.

ID: Which learning outcome(s) and why?

The aculty member gives an answer that makes

pedagogical sense. Go to Step 2.

OR

1B) FM: Yes (indecisively).

Otherwise (IB, C, or D) OR

1C) FM: Maybe (indecisively).

OR

1D) FM: No.

Ask questions to decide. Stop when you get an

answer that makes sense, and go to Step 2.

Do you need to teach about a topic that is impos-

sible to depict through mere text on a computer

screen?

ID: Is prociency in a skill or recall o many

acts needed?

Does time need to be compressed in the content

that you are presenting (or example, are you

trying to teach about the growth and develop-

ment o living things?)

FIGURE 1

ORSIDS Script Sample.


16/31


ered a technology-based course via video conerence. The session was success-

ul in that as a result o the process, the aculty member generated numerous new

ideas or the integration o learning objects into her course. Additionally the ses-

sion resulted in the identication o several good learning objects, ideas or strat-

egies with which to implement them, and additional URLs to search or more

resources. At the end o the meeting, the subject voiced that she now elt con-

dent about searching or and integrating learning objects where in the past she

had not used them because searching took so long and the results were disap-

pointing.

The process was successul in eliciting new insights rom the aculty mem-

ber about how technology could be used to enhance her class. These aculty

member insights can be classied as learner invention; they can be classied asthe synthesis category o human thinking skills in Blooms taxonomy (1956).

During the session, particularly during the discussion o methods, the aculty

member spontaneously came up with a number o ideas or learning objects that

would enhance her curriculum. By the time the methods step o the script had

been reached, the aculty member began to refect on what she does now in the

ace-to-ace situation and how technology-based resources could add to it. Some

o the learning objects she conceptualized and hoped to nd were actually dis-

covered during the search process. Additionally, it became apparent that one othe resources shown in the demo, the Virtual Piano http://www.pbs.org/jazz/

lounge/lab_virtual_piano.htm, was very eective at conveying potential cogni-

tive load problems. The subject viewed it and discussed experiences she had had

with technology-based resources in the past that were problematic in terms o

cognitive load.

The second eld test was conducted with a aculty member experienced with

online teaching as well as instructional design concepts including media selec-

tion, but unamiliar with media selection or the online environment. Despite the

act that this aculty member was experienced with instructional design and media

selection, this process caused him to dene his learning objectives much more

explicitly and in general to think more deeply about his curriculum.

Much o the search step was generated as a result o the researchers inde-

pendent judgment augmenting what was in the script. The searching process was

more dicult this time than it had been in the rst eld test. Appropriate repos-

itories or searching were not as readily apparent. The researcher and aculty

member did quite a bit o brainstorming to identiy potentially ertile search

terms, search engines, and repositories. Resources were discovered that the ac-ulty member could make use o in his online class. During the session, the ac-


17/31


ulty member came up with an idea or the ideal learning object to ulll his

learning objective, but this resource was not ound online through the searching

process.

The researcher pointed out that since the Faculty Members home institution

has development unds, such a resource could potentially be developed. This idea

was appealing to the Faculty Member.

In the process o using the script to work with the Faculty Member,

the researcher ound that many decisions needed to be tracked. This led to the

decision to develop a orm on which the various decisions arrived at in the

testing session could be recorded, the ORSIDS Decision Tracking Form (see

Figure 2).

The major ndings rom this session included the value o the instructionaldesign consultation even or aculty amiliar with instructional design methodol-

ogy. The individualistic and improvisatory nature o the searching that resulted

rom a lack o standard inrastructure (and a more loosely guided portion o the

script see Figure 3) was another major nding rom this test session.

Instructional-Designer Delivered Pilot Testing

Next two instructional designers were trained on the use o the script and

delivered it to one aculty member each. Both testing sessions resulted in positiveoutcomes in relation to the three problems within the scope o the study. Instruc-

tional designers were provided with a process that not only allowed them to

achieve some success in teaching instructional design methodology but also to

assist educators in adopting learning objects in a short period o time. Each test-

ing session resulted in one or more learning objects that were useul to the aculty

member.

Faculty increased their knowledge and condence, and in some cases shited

rom lukewarm attitudes to much more positive, in relation to online pedagogy

and their ability to adopt and evaluate online learning objects. But every testing

session also included a number o process execution errors on the part o the

instructional designer testing subjects that could be attributed to faws in the

design or ormatting o the ORSIDS process and script and/or lack o instruc-

tional designer content knowledge indicating that the instructional designer train-

ing was not entirely successul.

Following is a summary o the ndings rom each o the test consultations.

The researcher evaluated each session on the basis o elements o the Instruc-

tional Designer delivery o the script and the response o the learner. Priorto assessing the overall quality o the session and making recommendations or


18/31


FIGURE 2

ORSIDS Decision Tracking Form

Validated Learning Outome to be Fulflled with Online Resoures

Analyze Learning Outomes Fulfllment Requirements

Tasks or subtasks that need ulfllment with online learning resoures

Determine Learning Outome categories (chek all that Apply)

Motor skills Attitudes cognitive Strategies Verbal

Intelletual Skills

Proedures Priniples conepts (Abstrat or conrete)

Assign Instrutional Methods (chek all that Apply)

Drill and Pratie Eduational game Simulation

Problem Solving Disovery Learning

Presentation Demonstration Dialogi

Analyze Possible Media

Text Audio Animation Video Photos

Drawings Graphis Diagrams Audio with visualsText with visuals Audio with visuals and text

For eah task or subtask, answer the ollowing questions related to needed media attributes:

1. What instrutional attributes are needed? 2. Physial attributes? 3. How must student

input be aepted?

Task

Task

Task

Searh or Resoure candidates

Resoures that Passed Evaluation


19/31

orSiDS anD impliCationSfortheWiDeSpreaD DiffuSionof learning objeCtS 85

Interaction keywords What is Said

Step 7. Search together or online

resources.

Goal: To fnd one or two appropriate

online resources to partially or completely

ulfll a learning outcome.

Defne search questions and search terms.

ID: Now well search with the goal o fnding one or two

resources that will partially or completely ulfll one o your

learning outcomes (suggests some search terms and asks i

the FM agrees.)

(You may want to think urther about synonyms or the con-

cepts that you have identifed and how you might narrow or

broaden the topic. Use Rogets Thesaurus http://humani-

ties.uchicago.edu/orgs/ARTFL/orms_unrest/ROGET.html

i needed.)

Repository URLs are:

http://www.merlot.org/Home.po

http://www.academicinfo.net/digital.

html

http://www.learner.org/amerpass/

slideshow/archive_search.php

http://www.loc.gov/

ID: Here are some examples o some repositories that are

useul or fnding academic online resources. They include

Merlot, Academic Ino, American Passages, Library o

Congress, and Google Images.

Spend around an hour.

Keep track o the resources with book-

marks or by pasting them into an email.

Also record the URLs on the TrackingForm.

ID: (fgures out with FM which URLs or search engines to use

to try and fnd learning objects and online resources. Then

search until you fnd one or more promising resources in one

o the ollowing repositories.)

the previously discussed Merlot http://www.merlot.org/Home.po that contains many peer-reviewed learn-

ing resources that have been developed and contrib-

uted by other instructors.

the digital library collection at Academic Info

http://www.academicino.net/digital.html

American Passages: A Literary Survey (Annenberg/

CPB)

http://www.learner.org/amerpass/slideshow/archive_

search.php

or various images, sound clips, text, and primary sources

related to history and American literature or Library of Congress

http://www.loc.gov/

or visual, audio and textual artiacts rom the Library

o Congress Collection and links to digital collections

around the world.

GOOGLE.COM and specically the image search

(advanced option) in GOOGLE images (http://www.

google.com/)

(I absolutely no useul resources can be ound, a good case

can be made or requesting limited development dollars i

they are available.)

FIGURE 3


20/31


revisions to the script, the researcher evaluated each step o the ORSIDS process

as administered by the Instructional Designer:

Introdution: Both Instructional Designer Subjects executed this step skill-

ully. Both improvised rom the script and did not use it verbatim.

Step 1: Whih learning outomes are good andidates? Instructional

Designer Subject 1 was able to execute this step skillully. She delivered the

inormation clearly and was able to clariy misunderstandings. The Faculty

Member learner appeared to be stimulated by the interaction and spontane-

ously arrived at a new technology-based curriculum idea.

Instructional Designer Subject 2 executed this step less skillully. While she

asked the scripted questions or this step, she did not demonstrate the skillnecessary to connect the scripted questions with the Faculty Members candi-

date learning outcomes.

Step 2: Guide the task analysis The script did not enable either Instruc-

tional Designer Subject to execute this step skillully. Instructional Designer

Subject 1 skipped this step. In the Cognitive Walkthrough she revealed that she

thought that this step had already been done and had questions about the mean-

ing o the terms instructional attributes and physical attributes. In the ses-

sion acilitated by Instructional Designer Subject 2, the task analysis strayedrom the learning outcome under consideration. While this process helped the

Faculty Member working with Instructional Designer Subject 2 to generate

additional ideas, it did not ulfll the purpose o the step in terms o thoroughly

exploring the given learning outcome.

Step 3: choose learning outome ategory Both Instructional Designer

Subjects successully achieved the goal o this step to varying degrees. Instruc-

tional Designer Subject 1 executed the step as scripted. The Instructional

Designer was successul in guiding the Faculty Member through the process

o choosing the learning outcome category, and the Faculty Member did the

task correctly and appeared interested and engaged. Instructional Designer

Subject 2 departed rom the script and did not explain all o the learning out-

come types; however, the Faculty Member she worked with did successully

identiy the attitudes and verbal categories that applied to her learning

outcome. Some o the script terminology including cognitive strategy

and abstract vs. concrete concepts was conusing to both Instructional

Designers.

Step 4: choose instrutional methods Instructional Designer Subject 1executed this step skillully. She accurately delivered the script and addition-


21/31


In this session, the Faculty Member expressed some creative curriculum ideas.

Instructional Designer Subject 2 did not execute this step as skillully. She

omitted much o the script content or this step. In the cognitive walkthrough

she explained that the reason or the omissions was because she thought that

the Faculty Member Subject was already amiliar with the material.

Step 5: Select potential media formats Instructional Designer Subject 1

skillully contextualized and provided transitions in this part o the script. The

Faculty Member that she worked with had a number o creative curriculum

ideas. This step required the use o a Multimedia Heuristic provided in the

script, which proved difcult or both instructional designers to use. Instruc-

tional Designer Subject 1 attempted to cover everything in the Heuristic rather

than limiting the discussion to the portions relevant to the educators curricularneeds. Instructional Designer Subject 2 asked the Faculty Member whether

she thought selected media types might be useul or her purposes without

connecting the media in the Heuristic to the methods identifed in the previous

step. This discussion did prove eective in causing this Faculty Member to

discuss some potentially useul media assets.

Step 6: Provide a lecture/demonstration Instructional Designer Subject 1

skillully conducted the demo overall. The Instructional Designer and the Fac-

ulty Member seemed to really enjoy their interaction, and the Faculty Memberarticulated some good ideas or curriculum in response. Instructional Designer

Subject 1 appeared to fnd one section o the demo conusing. Instructional

Designer Subject 2 did not deliver the demo as scripted and omitted parts o

the demo that were necessary to illustrate critical concepts. In the cognitive

walkthrough, she stated that she omitted these sections because she thought

that the content might be too juvenile or the Faculty Member. The Faculty

Member that she was working with brought up some o her concerns including

issues related to aective learning and the importance o teaching students to

process visual material.

Discuss cognitive load and message design issues This discussion met

with resistance rom one o the educators when it was previously presented

during the Evaluation step. The researcher conjectured that i this discussion

were moved earlier in the session, perhaps it would be more easily accepted.

The discussion proved to be out o place there. Instructional Designer Subject

1 discussed it in relation to the wrong application, and Instructional Designer

Subject 2 skipped the step because there was a technical problem with show-

ing the application that exemplifed the concept.Step 7: Search Both Instructional Designer Subjects conducted their


22/31


lack o clear direction in the script. Neither ollowed all o the directives o the

script, but both were successul in helping the Faculty Member to nd one or

more useul online resources and to create plans or urther ruitul searching.

Instructional Designer Subject 2 bypassed many o the listed repositories and

omitted the use o the suggested keywords to look or specic media-related

resources. She also omitted a critical step, that o keeping a list o the discov-

ered online resources to give to the Faculty Member. She did, however, com-

pensate or the earlier fawed execution o the task analysis step with a

discussion that she guided here.

The searching o Instructional Designer Subject 1 refected the act that she

had not been suciently prepared in the training with inormation about the

use o online learning objects. This subject demonstrated a lack o knowledgeabout how to use learning objects (or example, that a source did not have to

standalone completely but could be scaolded with explanatory textual mate-

rial developed by a aculty member). This lack o knowledge caused her to

discount the value o useul resources that her team did nd. She also did not

integrate the Faculty Members earlier creative ideas into the search process

that she guided. This subject did, however, demonstrate the ability to meet

some o the goals o this step. In addition to nding several resources, this

dyad came up with a plan or urther searching.Intro to evaluation and evaluation steps Since Instructional Designer

Subject 1 was dissatised with the learning objects that she and the Faculty

Member did discover, she led the Faculty Member in a process to evaluate

imaginary online resources that would be discovered i their searching

plan was successul. Instructional Designer Subject 2 made several errors as

she led the aculty subject through the evaluation steps. Instructional Designer

Subject 2 did not initially lead the task analysis as delineated in the script but

compensated by doing the needed steps intuitively prior to embarking upon

the searching process. Both Instructional Designer Subjects had questions

about the meaning o the term instructional and physical attributes.

Post Pilot Test Learner Validation

Ater each pilot testing session, the researcher and Instructional Designer

debrieed the session. In the debriengs with the Instructional Designers that ol-

lowed the testing sessions, they expressed positive attitudes towards the process

but reported problems with the usability, ormat and content o the script, as well

as the lack o standardized inrastructure in which their searches took place. Bothagreed that the script made it ar easier to teach instructional design skills and to


23/31


own. Instructional Designer Subject 1 stated that the application is a great tool

with many applications that is hugely practical, provides a straightorward pro-

cess, and equalizes the playing eld between instructional designers with a lot o

knowledge and those without it. However, she elt the need or more at-a-glance

support in terms o the ormat o the script. She suggested eliminating text in the

script and replacing it with keywords where possible. Instructional Designer Sub-

ject 1 also elt that the Search step would be improved with the inclusion o more

repositories. She also stated that some o the language was too sophisticated and

that sentences were too long. She recommended that the Multimedia Heuristic

be retitled Learning Media Matrix. Both subjects recommended that the terms

instructional attributes and physical attributes be better explained and that

the ORSIDS Tracking Form be transormed into more o a job aid with embeddedprocess inormation. These changes were made, and Figures 1 and 2 refect the

incorporation o these suggestions.

To conclude, the pilot test ndings included the ollowing:

In both tests (and in the two preceding eld tests administered by the

researcher), the Instructional Designers were able to acilitate sessions result-

ing in successul searching or online resources, although in the case oInstructional Designer Subject 2 they were text based because o omission o

the provided search terms or media assets. Learning on the level o synthesis

(Bloom, 1956) related to technology-based course development was demon-

strated by the Faculty Member Subject who worked with Instructional

Designer Subject 1, the subject who delivered the script more accurately. This

type o learning was not demonstrated by the Faculty Member Subject who

worked with Instructional Designer Subject 2. There was insucient data to

identiy the Blooms Taxonomy level o learning or Faculty Member 2.

The ormat o the script and related training were not sucient to enable mas-

tery o the script delivery on the part o either Instructional Designer Subject.

As a result o these pilot tests, the beta script was revised, and the nal version

resulted.

Product Evaluation

Several items were prepared to communicate essential inormation about the

documents evolution and the testing results to the Expert Panel while minimizingthe amount o inormation that they had to absorb. Additionally a questionnaire


24/31


flled the validated requirements. The Expert Panelists rated the process script

highly overall while acknowledging the need or urther feld testing and possible

ormatting changes.

DIScuSSIOn

Three years ater the studys completion, the most striking study fndings

were:

The difculty caused by the lack o standardized searching inrastructure in

searches or educational learning objectsThat media selection theory (developed by instructional design theorists in the

mid to late 20th century) as distilled into ORSIDS proved to be an eective

method to guide the search or learning objects

The difculty experienced by the instructional designer subjects in grasping

media selection theory rom the ORSIDS script and training

The ease with which the educators were able to grasp instructional design and

media selection theory.

The ORSIDS process was limited in its success because o the lack o stan-

dardized inrastructure or the search o educational learning objects. Unlike the

rest o the script which could be delivered in a rather proscribed manner, because

o the lack o standardization, the ORSIDS search process required a good deal

o initiative and problem solving on the part o the instructional designers sub-

jects as well as knowledge o individual repositories and their idiosyncratic meth-

ods or Search Inrastructure.

What would standardized search inrastructure look like? It would consist o a

search experience and search terms that could predictably yield maximum results

across all websites and repositories. This was decidedly not the case.

What would be required or such an inrastructure to exist? When dealing with

one repository in which a metatagging system can be designed together with a

search and selection methodology, it is not difcult to create a standardized search

inrastructure. But when seeking to encompass disparate repositories throughout

the World Wide Web, the context or this study, what is required is eective meta-

data tagging schemes that are interoperable and the use o a uniorm search and

selection methodology.According to Understanding Metadata (2004, p. 1), Metadata is structured


25/31


resource. A metadata scheme is a set o metadata elements defned or a spe-

cifc purpose, such as describing a particular type o inormation resource. The

defnition or meaning o the elements themselves is known as the semantics o

the schema Metadata schemas generally speciy names o elements and their

semantics.

In the years since the 2005 study, a number o organizations have completed

initiatives to develop metadata schemas or educational learning objects includ-

ing IEEE Learning Objects Metadata (IEEE LOM), IMS Learning Resource

Metadata, and PBCORE Metadata. While the IMS Global Learning Consortium

in particular is working towards interoperability, and uses a schema similar in

many ways to the IEEE LOM, the many schemas are still not unctionally interop-

erable: the metadata schemas are not synchronized and electronic resources arenot organized in such a away that search results can all be accessed in a predict-

able and standard way that is transparent to the searcher.

According to Understanding Metadata (2004, p. 2): Interoperability is the

ability o multiple systems with dierent data structuresto exchange data with

minimal loss o content and unctionality. Using defned metadata schemes ...and

crosswalks between schemes, resources across the network can be searched more

seamlessly. The publication goes on to explain that there are two approaches to

interoperability.In cross-system search, implemeters do not share metadata but map their own

search capabilities to a common set o search attributes. A contrasting approach

taken by the Open Archives Initiative is or all data providers to translate their

native metadata to a common core set o elementsd and expose this or harvest-

ing. A search service provider then gathers the metadata into a consistent central

index to allow cross-repository searching regardless o the metadata ormats used

by participating repositories.

The IMS Global Learning Consortium (2008, p. 1) describes the situation at

the time it was written.

Currently, there are no agreed profles that address the needs o the learning

domain, and no established practices or combining existing specifcations

into complete solutions. Individual organizations are creating their own

solutions, and an opportunity to establish broader interoperability is being

missed.

For interoperability to exist, all o the organizations with their individual

metadata schemas would have to work together and allow their electronic

resources to be organized and made accessible in one o the ways decribedabove. But beyond the need or this cooperation, the question also arises


26/31


can be in supporting educator access to digital educational learning objects.

The media selection process proved eective in the study in assisting with

selecting learning objects to ulll curricular objectives. This suggests that

metadata elds based on the steps in the media selection process may be useul

and should be considered.

The process involved:

The identication o the learning outcome, or objective1.

Determining the applicable learning outcome category or categories2.

Assigning instructional methods3.

Analyzing possible media4.

While some o the major metadata schemes do include # 1 and #4, elds to

speciy learning outcome and media, none o the major educational metadata

schemes include #2 or #3, Learning Outcome Category and Instructional Method.

Their use warrants urther investigation.

Another question raised by the study is the appropriate user or a job aid

such as ORSIDS. The study was based on the assumption that instructional

designers would be appropriate consultants to transmit instructional design and

media selection theory to educators. The study results were surprising in thatthe instructional designers experienced diculties in learning the media selec-

tion methodology while the educators to whom the consultation was provided

were able to pick up instructional design theory relatively easily. There could

be many reasons or the lack o instructional designer mastery including faws

in the product itsel, insucient and unpaid time to learn the ORSIDS method,

and the corporate background o the instructional designers. The instructional

designers also lacked the content knowledge possessed by the educators with

whom they worked, which could have posed a stumbling block as well. Yet the

study showed that the prerequisite knowledge o media selection theory assumed

o the experienced Masters level instructional designers was not present, and

there is no reason to believe that it would be more so with college-employed

instructional designers with a broad range o experience, educational levels and

expertise. The educators on the other hand picked up many instructional design

concepts easily, raising the question o whether educators might be as well or

better equipped than college-employed instructional designers to make use o

scaolding assistance or the selection o learning objects, including the direct

rather than mediated use o a job aid such as ORSIDS that is based on the mediaselection process.


27/31


cOncLuSIOn

This study clearly proved the eectiveness o teaching instructional design,

and in particular media selection theory, to higher education aculty rom disci-

plines outside o instructional design. With ORSIDS, the ASSURE method and

elements o other media selection theories were modifed or use in the online

environment. This study demonstrated that instructional design media selection

theory is eective when transerred to the search or learning objects in the online

environment. The eectiveness o the consulting sessions supported with the

ORSIDS script document the efcacy o instructional design theory when trans-

erred to the online setting. In three out o the our tests, aculty who were taught

this theory responded by having original creative ideas connected to their curricu-lum and how they could be implemented through learning objects.

In the ourth case, where the aculty member did not respond by having origi-

nal creative ideas connected to the curriculum, the instructional designer did not

clearly transmit the instructional design theory. While the value o the ORSIDS

process was proven by the study, the Expert Panelists and the researcher agreed

that the most recent ORSIDS script and related instructional design training

should undergo usability testing and possible urther revisions.

Additionally it emerged rom the study that much more is needed along thelines o searching inrastructure and standardization to acilitate highly reliable

searches or learning objects. The purpose o each search session was to come up

with one good learning object or one learning outcome in a three our hour

session. This manageable goal was accomplished in all our testing sessions.

However, these positive results were hit or miss, resulting rom searches that that

were individualistic and idiosyncratic; by necessity they relied a good deal on the

instructional designers individual judgment and knowledge. The lack o stan-

dardized inrastructure or accessing learning objects, search engine repositories

and searching techniques is a problem that will have to be corrected beore the

wide spread diusion o learning objects can occur. Eorts in this direction have

occurred since the completion o the 2005 study, and it is hoped that eorts

towards standardization and interoperability continue in a manner that success-

ully makes multimedia learning objects accessible to all educators and students.

However, the study did make clear that the delivery o a process script such as

ORSIDS may result in an understanding o the value o learning objects on the

part o aculty. Comprehending how learning objects can apply to the learning

outcomes they plan may cause such aculty to embrace their use or their onlineclasses.


28/31


REFEREncES

Boyle, T. (2002). Design principles for authoring dynamic, reusable learning objects. Proceedings of

the 19th

Annual Conference of the Australasian Society for Computers in Learning in TertiaryEducation (ASCILITE). Accessed September 4, 2004, from http://www.unitec.ac.ne/ascilite/

proceedings/papers/028.pdf

Brewerton, P. & Millward, L. (2001). Organizational research methods. London: Sage.

Cohen, D. (2004).Development of a media selection tool for online faculty. Poster session presented

at the ED-MEDIA 2004 World Conference on Educational Multimedia, Hypermedia & Tele-

communications, Lugano, Switzerland.

Cohen, D. E. (2005). The development of ORSIDS: A process script for online resource selection.

Doctoral Dissertation, Nova Southeastern University, Florida. Available at: http://www.cogni-

tionignition.com/PublicationsandPresentations.htm

Conole, G. (2002). Systematizing learning and research information.Journal of Interactive Media in

Education, 2002 (7). Retrieved July 15, 2004, from http:///www.jime.open.ac.uk/2002/7/conole-027-paper.html

Clark, R. C. (1999).Developing technical training. Washington, DC: International Society for Perfor-

mance Improvement.

Clark, R. C. & Mayer, R. E. (2003).E-Learning and the science of instruction: Proven guidelines for

consumers and designers of multimedia learning. San Francisco: Jossey-Bass/Pfeiffer.

Davidson-Shivers, G. (2002). Instructional technology in higher education. In R. A. Reiser and J. V.

Dempsey (Eds.), Trends and issues in instructional design and technology (pp. 256268). Upper

Saddle River, NJ: Merrill Prentice Hall.

Delbecq, A. L., Van de Ven, A. H. & Gustafson, D. H. (1975). Grouptechniques for program plan-

ning: A guide to nominal group and delphi processes. Glenview, IL: Scott, Foresman and Com-

pany.

Dijkstra, S., Jonassen, D. & Sembill, D. (2001). The use of multimedia in education and training. In

S. Dijkstra, D. Jonassen, & D. Sembill (Eds.),Multimedia learning: Results and perspectives

(pp. 313). Frankfurt am Main: Peter Lang.

Foshay, W. R., Silber, K. H., Stelnicki, M. B. (2003). Writing training materials that work: How to

train anyone to do anything. San Francisco: Jossey Bass: Pfeiffer.

Friesen, N. (2004). Three objections to learning objects. In R. McGreal (Ed.). Online education using

learning objects. London: Routledge/Falmer. Retrieved July 15, 2004, from http://phenom.

educ.ualberta.ca/~nfriesen/

Friesen, N. (2006). Pedagogical Neutrality and Engagement. Technology, Instruction, Cognition and

Learning (TICL). Volume 3, Number 12.

Gay, L. R. & Airasian, P. (2000). Educational research: Competencies for analysis and application.

Upper Saddle River, NJ: Merrill.

Gosper, M, Woo, K., Gibbs, D., Hand, T., Kerr, S. & Rich, D. (2004). Learning objects: User perspec-

tives on the conditions surrounding their use. In L. Cantoni and C. McLoughlin (Eds.), Proceed-

ings of ED-MEDIA 2004 World Conference on Educational Multimedia, Hypermedia &

Telecommunications, Lugano, Switzerland.

The IMS Global Learning Consortium: Learning Object Discovery and Exchange Project Group

(2008). Retrieved February 5, 2008 from

http://www.imsglobal.org/lode.html

Johnson, L. F. (2003). Elusive vision: Challenges impeding the learning object economy. Retrieved

July 9, 2004 from http://download.macromedia.com/pub/solutions/downloads/elearning/elu-sivevision.pdf


29/31


Jonassen, D. (2001). Learning rom, in, and with multimedia: an ecological psychology perspective.

In S. Dijkstra, D. Jonassen, & D. Sembill (Eds.)Multimedia learning: Results and perspectives

(pp. 4167). Frankurt am Main: Peter Lang.

Keppell, M. (2001). Optimizing instructional designer-subject matter expert communication in the

design and development o multimedia projects. Journal of Interactive Learning Research 12

(2/3), 205223.

Keppell, M. (2004). Legitimate participation? Instructional designer subject matter expert interac-

tions in communities o practice. In L. Cantoni and C. McLoughlin (Eds.), Proceedings of ED-

MEDIA 2004 World Conference on Educational Multimedia, Hypermedia & Telecommunications,

Lugano, Switzerland.

Kozma, R. (2001) Robert Kozmas counterpoint theory o learning with media. In R. Clark (ed.)

Learning from media: Arguments, analysis and evidence (pp. 137178). Greenwich, Conn.:

Inormation Age Publishing.

Lee, S. H. (1999). Usability testing or developing eective interactive multimedia sotware: Con-

cepts, dimensions, and procedures.Educational Technology & Society 2(2). Retrieved Septem-ber 20, 2005 rom http://iets.ieee.org/periodical/vol_2_99/sung_heum_lee.html

Liu, M., Gibby, S. Quiros, O. & Demps, E (2002). Challenges o being an instructional designer or

new media development: A view rom the practitioners.Journal of Educational Multimedia and

Hypermedia 11 (3), 195225.

Mayer, R. E. (1999). Multimedia aids to problem-solving. International Journal of Educational

Research, 31(7), 611623.

Mousavi, S. Y., Low, R. & Sweller, J. (1995). Reducing cognitive load by mixing auditory and visual

presentation modes.Journal of Educational Psychology, 87(2), 319334.

Pan, C. S., Deets, J., Phillips, W. & Cornell, R. (2003). Pulling tigers teeth without getting bitten:

instructional designers and aculty. The Quarterly Review of Distance Education 4 (3),

289302.Paquette, G. & Marino, O. (2006). Learning Objects, Collaborative Learning Designs and Knowledge

Representation. Technology, Instruction, Cognition and Learning (TICL). Volume 3, Number

12.

Reiser, R. A. & Gagne, R. M. (1983). Selecting media for instruction. Englewood Clis, NJ: Educa-

tional Technology Publications.

Reynolds, A. & Anderson, R. H. (1992). Selecting and developing media for instruction. New York:

Van Nostrand Reinhold.

Richey, R. C. & Nelson, W. A. (1996). Developmental research. In D. H. Jonassen, Handbook of

research for educational communications and technology (pp. 12131245). New York: Mac-

millan LIBRARY Reerence USA.

Romiszowki, A. J. (1988). The selection and use of instructional media. London: Kogan Page.Rouet, J., Levonen, J. J. Biardeau, A. (2001). Introduction. In J. Rouet, J. Levonen, & A. Biardeau

(Eds.),Multimedia learning: Cognitive and instructional issues (pp. 18). Amsterdam: Perga-

mon.

Samaras, H., Giouvanakis, T., Bousiou, D. & Tarabanis, (2004). Towards a new generation o research

regarding the infuence o multimedia on learning. In L. Cantoni and C. McLoughlin (Eds.),

Proceedings of ED-MEDIA 2004 World Conference on Educational Multimedia, Hypermedia &

Telecommunications, Lugano, Switzerland.

Scandura, J. M., Abstract Syntax Tree (AST) Inrastructure in Problem Solving Research (2006).

Technology, Instruction, Cognition and Learning (TICL). Volume 3, Number 12.

Schweier, R. A., Campbell, K. & Kenny, R. (2004). Instructional designers observations about iden-

tity, communities o practice and change agency.Australasian Journal of Educational Technol-


30/31


ogy 20 (1), 69100. Retrieved July 15, 2004, from http://www.ascilite.org.au/ajet/ajet20/

schweier.html

Shneiderman, B. (1998).Designing the human interface. Reading, MA: Addison-Wesley.

Smaldino, S. E., Russell, J. D., Heinich, R., & Molenda, M. (2005).Instructional media and the new

technologies of instruction. Upper Saddle River, NJ: Pearson.

Smith, P. L. & Ragan, T. J. (1999).Instructional design. New York: John Wiley and Sons.

Sweller, J. & Cooper, G. (1985. The use of worked examples as a substitute for problem solving in

learning algebra. Cognition and Instruction, 2(1), 5989.

Torrisi-Steele, G. and Davis, G. (2000). A website for my subject. The experiences of some academ-

ics engagement with educational designers in a team based approach to developing online

learning materials.Australian Journal of Educational Technology, 16 (3), 283301. Retrieved

July 15, 2004, from http://www.ascilite.org.au/ajet/ajet16/torisi-steele.html

Understanding metadata (2004). National Information Standards Organization. Retrieved February 3,

2008 from www.uwm.edu/~mll/resource.html


31/31

Copyright of Technology, Instruction, Cognition & Learning is the property of Old City Publishing, Inc. and its

content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's

express written permission. However, users may print, download, or email articles for individual use.

the online resource selection instructional design script

Documents