1999-12-06ieee 1484.1, ltsa presentation, f. farance, ©1999 edutool.com 1 learning technology...

1999-12-06 IEEE 1484.1, LTSA Presentation, F. Farance, ©1999 Edutool.Com

1

Learning Technology Systems ArchitectureLTSA 5.0 Specification1999-12-06 Presentation

Frank Farance, +1 212 486 4700, [email protected]

Edutool.Com, a division of Farance Inc.Slides and Document: http://edutool.com/ltsa

Presentation to LTSC Architecture and Reference Model WG(IEEE 1484.1) http://ltsc.ieee.org/wg1


2

Overview

• Review of goals• Abstraction-Implementation Layers• Review of each layer• Demonstration of perspectives• What does conformance mean?• Open issues


3

Document Status

• Specification accepted as a base document for IEEE 1484.1

• Comments on draft 4.0:– Typos– Revising “human-centered features” to accommodate

several “development” orderings– Glossary revisions– Revise for IEEE style


4

Review of architecture goals

• Identify common components• Identify critical interfaces• Widespread applicability:

– platform-neutral– content-neutral– pedagogically-neutral


5

Architecture Definition And Purpose [1/3]

• LTSA 5.0 specification: http://edutool.com/ltsa• [Excerpt:] “In general, the purpose of developing

systems architectures is to discover high-level frameworks for understanding certain kinds of systems, their subsystems, and their interactions with related systems. ...” More than one architecture is possible


6


• [Excerpt cont’d:] “… An architecture isn't a blue print for designing a single system, but a framework for designing a range of systems over time, and for the analysis and comparison of these systems. …” Architecture used for analysis and communication


7


• [Excerpt cont’d:] “… By revealing the shared components of different systems at the right level of generality, an architecture promotes the design and implementation of components and subsystems that are reusable, cost-effective, and adaptable.” Critical interoperability interfaces/services identified


8

Abstraction-Implementation Layers

• The Groupings– abstractions vs. implementations

• The Priorities– ordering

• The Notations– text notation– system notation– bus notation

• The LTSA Layers– five layers


9

Abstraction-Implementation Layers:The Groupings

• Organize engineering details into several groupings from high-level (coarse granularity) to low-level (fine granularity)

• Groupings are called layers• Layers above are called abstractions• Layers below are called implementations


10

Information InclusionOrganize raw details as: good, borderline, and bad examples

Raw Details Bad

Good

Borderline


11

Information ExclusionEssential vs. Non-Essential Details: Depends on Context

Browser ServerWeb Content

#1: Minimal Details, Only Data Flow


Server Initiated(“Push”)

#3: (Again) More Details, Control Flow


User Request(“Pull”)

#2: More Details, Control Flow

Browser Server

Web Content

HTMLCGI

#4: More Details, Related Subsystems


12

Abstraction-Implementation Layers:The Priorities

• High-level design issues have most pervasive effect on design: first priority

• Low-level design issues are localized: consider these issues later

• Example: building a house– High-level (higher design priority): layout and number of

rooms– Low-level (leave till later): choosing wallpaper


13

Iterative AbstractionDesign Priorities: Colors (high), Shapes (medium), Raw Details

Raw Details

Abstraction of Raw Details:Shapes (Arrow, Box, Star)

Abstraction of Shapes:Colored Polygons (Red, Green, Blue)


14

Iterative ImplementationDesign Priorities: Colors (high), Shapes (medium), Raw Details

Implementation of Colored Shapes: Constraints: Arrows Connected to Polygons

Implementation of Shapes withConstraints: Red, Green, Blue

Implementation of Colored Polygons with Constraints:Eight or fewer sides


15

Abstraction-Implementation Layers:The Notations

• High-level: systems (Yourdon) diagrams– Process (oval), store (box), flow (arrow)– Static connections (flows) identify functional decomposition

• Low-level: bus diagrams– Bus (bar), connector (line), system (box)– Dynamic, point-to-point connections for on-demand,

arbitrary component communication

• Not Used:– UML (Unified Modeling Language): too, low level; OO-based– ERD (entity relationship diagrams): useful for applications,

but not for architecture and interoperability


16

Systems Notation:Processes, Stores, Flows

DataProcess

DataProcess

ControlProcess

One-Way Control Flow

One-WayData Flow

Two-Way Control Flow

Two-Way Data Flow

DataStore

ControlStore


17

Example of Systems Tiers/Hierarchies:A System and Its Implementation Details

System X

System X’sInputs/Outputs

Details Of System X

Distributed/Hidden DetailsSub-System

A

Sub-SystemB

Sub-SystemC

System X’sInputs/Outputs

Implementation Details


18

Bus Notation

• Common naming• Common control flow (initiating transactions)• Non-homogeneous data flow (different transactions)• Dynamic, on-demand connections


19

Bus Notation: Common Naming

P

Q

R

S

Each member of bus has unique name and shares namespace


20

Bus Notation: Common Control Flow

P

Q

R

S

Each member uses same protocol to request or respond to transactions


21

Bus Notation: Non-Homogeneous Data Flow

P

Q

R

S

Common (homogeneous) control protocol,but different (heterogeneous) data protocols

Examples:TCP/IP is common “control” protocol, but several “data” protocols may be used:

PQ uses FTP, but

PR uses HTTP

FTP overTCP/IP bus

HTTP overTCP/IP bus


22

Bus Notation: Dynamic, On-Demand Connections

P

Q

R

S

No permanent connections among members of bus

Each “transaction” is a connection

Examples:1. Q sends to P2. P broadcasts to all3. R sends to Q

12

3


23

Examples of Bus Tiers/Hierarchies:Internet Client-Server Bus, Client Perspective

Client connects to many servers

On-demand connections

Several servers may have same purpose, but varied resources.Like libraries: same interface, but varied collections of books

Mirror of server perspective

Server

Server

Server

Client


24

Examples of Bus Tiers/Hierarchies:Internet Client-Server Bus, Server Perspective

Server handles requestsfrom multiple clients

On-demand connections

Clients may appearhomogenous, but havedifferent identities

Mirror of client perspective

Client

Client

Client

Server


25

Examples of Bus Tiers/Hierarchies:Combining/Bridging Two Busses

User

WebClient

WebServer 1

WebServer 2

WebServer 3

GUIApp 1

GUIApp 2

GUIApp 3

Window ManagerApplication “Bus”

Examples:X Windows, Win95

World Wide Web“Bus” of Browsersand Web Servers

Web Browser: “sits” on two busses: both window manager application & web application


26

Examples of Bus Tiers/Hierarchies:Combining/Bridging Three Busses

User

WebClient

WebServer 1

WebServer 2

GUIApp 1

GUIApp 2

WindowManager

Application“Bus”

World Wide Web“Bus” of Browsersand Web Servers

Web Browser:GUI Web

ChatClient 1

ChatClient 2

ChatServer

Chat Session:GUI Chat

Chat “Bus”of Sessions


27

Combined Systems and Bus Notation:Processes, Stores, Flows, Busses

DataProcess

DataProcess

ControlProcess

One-Way Control Flow

One-WayData Flow

Two-Way Control Flow

Two-Way Data Flow

Data AndControl Bus X

Data-Only Bus Y


28

Systems vs. Bus Notation

• Systems notation useful for:– Small number of components– Dedicated functions for each component– Useful for LTSA layers 1, 3, 4, and 5

• Bus notation useful for:– Large number of components– Dynamic, on-demand connections between components– Useful for LTSA layer 5


29

Other Notations

• UML (Unified Modeling Language)– Not good for architecture definition– UML is useful for analysis and design notations– System and bus notation may be mapped to/from

UML for some UML applications

• IEEE 1471– LTSA stakeholder mappings satisfy IEEE 1471

recommended practices for multiple perspectives


30

Architecture Abstraction Hierarchy Reference Model (Belz, Suthers, Wheeler)

Application Model

Abstract System Model

- Interaction Model

- Conceptual Model

Abstract Implementation Model

Resource Model

LTSA Layer 1; IEEE 1484.1



LTSA Layer 5; IEEE 1484.1, 1484.2, 1484.4, 1484.5, 1484.6, 1484.7, 1484.8, 1484.9, 1484.10, 1484.11, 1484.12, 1484.13, 1484.14, 1484.15, 1484.16, 1484.17, 1484.18, 1484.20

No category:IEEE 1484.3, 1484.19



31

Abstraction-Implementation Layers:The LTSA Layers

• LTSA from high-level to low-level:– Layer 1: Learner and environment interactions– Layer 2: Human-centered features– Layer 3: System components– Layer 4: Stakeholder perspectives– Layer 5: Operational components and interoperability


32

Summary of Five LTSAAbstraction-Implementation Layers

120+ Stakeholder Perspectives/PrioritiesLayer 4

Human-Centered/Pervasive FeaturesLayer 2

Environment

Interactions

Layer 1Learner/

EnvironmentInteractions

L

LL

LearnerEntity

Layer 3LTSA

SystemComponents

LCI

LP

BM

PP PA

D

LE

E

CLR R

IC

Q

LC L

Layer 5APIs,

Codings,& Protocols

Requirements Functionality Conceptual Model Semantics APIs Codings Protocols Calling Data Comm. Conv. Formats Layers


33

Layer 1: Learner AndEnvironment Interactions

• Learner interacts with “environment” as part of a “learning experience”

Learner

Teacher

Other Learners*Books

Library

CoachMentor

Collaboration*

School

Other Employees*

Internet/Web

Television

Lab Parent

Computers

Employer

NewspaperMultimedia

* See later slide: collaboration is internal to Learner


34

Layer 1: Learner and Environment:How Environment Effects Learner

• Primary Focus:– The effect of environment on the learner– Not the effect of learner on the environment

• Learner can have important effects on environment:– Examples: research, teaching teachers, etc.– But “learner’s effect on environment” is not the primary

technical focus of learning technology


35

Layer 1: Learner and Environment:Collaboration: Internal to Learner

• LTSA “Learner Entity” (an abstraction) can represent one or more “Learners” (humans)

• Collaboration among Learners is internal to Learner Entity

• Identifying collaboration as internal is a significant simplifying feature for LTSA

• From an information technology perspective– All Learner Interaction is from the environment to the Learner

Entity — inbound information– Thus the “Learner Interactions” arrow points inward towards

the Learner Entity


36

Layer 1: Learner and EnvironmentLearning Interactions: Input to Learner

• Primary Focus:– Technology issues — the flow of information– No pedagogy implied

Learner

LearnerEntity

Collaboration

Learner

LearnerEnvironmentLearning

Interactions


37

Layer 2: Human-Centered/Pervasive FeaturesStrengths and Weaknesses [1/2]

• Humans are involved in the learning experience (human learning), not computers (machine learning)

• Human communication: fundamental strengths and limitations has pervasive effect on design

• Human-directed: learners can influence the process themselves

Environment LearningInteractions

LearnerEntity

- Learner is not a computer- Human learners’ strengths/ limitations — complicates receiving information from learning interactions- Humans can be self-directed


38

Layer 2: Human-Centered/Pervasive FeaturesStrengths and Weaknesses [2/2]

• Several “stories” motivate the need for human-centric and component-based learning technology systems– Illustration 1:

• Traditional• “Instructive approach”

– Illustration 2:• Learner-directed• “Constructive approach”

• Conclusion:– No single “right” story, but several applicable stories


39

Illustration 1:“Traditional” Approach

• Learner is directed (or instructed) by “coach”• Coach can influence and make decisions

about the learner’s learning experiences• Representative of traditional classroom and

training scenarios• Pervasive issue:

– humans as “information receivers”– both strength and weakness


40

Layer 2: Human-Centered Features Human communication affects design

• Humans aren’t reliable receivers of information:– Humans mishear or don’t hear at all– “Retransmission” is necessary

• Humans aren’t predictable receivers of information:– Humans hear different than desired– Different strategies are necessary


41

Layer 2: Human-Centered Features Wide variations affect design

• Humans learn differently (and over time):– Humans differently from each other– Same human may learn differently over time– Need intelligent customization, configuration– Need rich learning resources for diverse humans

• Negotiate learner preferences:– Learner alone or coach alone cannot determine best

learning style and/or preferences


42

Layer 2: Human-Centered Features Constructing LTSA System Components

• #1: Start with human sensory input: multimedia delivery to learner (similar to entertainment)

• #2: Add coaching/feedback loop for human “unreliability”• #3: Add learner records for varying teachers, infer

intelligently about learner capability• #4: Add rich learning resources, search, and lookup to

support diversity of humans• #5: Add negotiated learning preferences for direct

communication with learner


43

Step #1: Transferring Information

• Transfer information to Learner• Sensory input/interactions for humans• Delivery of Multimedia: same as entertainment

Delivery

MultimediaLearnerEntity


44

Step #2: Coaching/Feedback Loop [1/2]

• Feedback necessary– Humans are “unreliable” receivers– They “don’t hear what you tell them”

• Current “state” (i.e., the “sensor”) of the feedback loop:– Behavior– Evaluation– Assessment

Delivery Evaluation

Coach

Multimedia Behavior

A s s e s s m e n t

LearnerEntity

L o c a t o r


45

Step #2: Coaching/Feedback Loop [2/2]

• Coach:– Determines corrective/suggestive actions

• Locator:– Examples: next lesson (Learning Content) towards objective; web URL– Directs Delivery of Multimedia to Learner

• Loop recovers from errors, focuses on target

Delivery Evaluation

Coach

Multimedia Behavior

A s s e s s m e n t

LearnerEntity

L o c a t o r


46

Step #3: Learner Records

• Performance information– helps infer corrective action over long periods of time– nomadic learners: allows learner migration (handoff) to different

teachers (e.g, grade promotion) and different institutions • Learner Records: Learner’s history and objectives

Delivery Evaluation

Coach

(new)

Multimedia Behavior

LearnerRecords

Performance (current)

(history)

A s s e s s m e n t

LearnerEntity

Performance/Preferences

L o c a t o r


47

Step #4: Learning Resources

• Humans are “unpredictable” and diverse• Knowledge libraries necessary to match human diversity• Coach:

– directs different learning preferences and strategies– based on Learner’s history/objectives and Assessment– supported via queries to Learning Resources for appropriate Learning

Content

Delivery Evaluation

CoachLearningResources

Query (new)

Multimedia Behavior

LearnerRecords

Interaction Context


(history)

A s s e s s m e n t

LearnerEntity


L o c a t o rLearningContent

Catalog Info

Locator


48

LearningContent

Step #5: Advice from the Learner

• Humans are self-aware: can give advice on learner prefs• Learning Preferences:

– interaction between Learner and system to negotiate different learning preferences

– negotiation might actually be assertion: Learner may assert (e.g., experimentation), or coach may assert (e.g., primary school might dictate learning preference)

Delivery Evaluation


Query (new)

Multimedia Behavior

LearnerRecords


(history)

A s s e s s m e n t

LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


49

LearningContent

Review: System Generated SolelyBy Human-Centered Features

• #1: Transferring Information• #2: Feedback loop: unreliable Learners• #3: Learner Records: history supports intelligent inferences, history allows

Learner migration• #4: Learning Resources: varying/diverse learner capabilities• #5: Learning Preferences: negotiate between Learner/system

Delivery Evaluation


Query (new)

Multimedia Behavior

LearnerRecords


(history)

A s s e s s m e n t

LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


50

LearningContent



Learner migration• #4: Learning Resources: varying learner capabilities• #5: Learning Preferences: negotiate between Learner/system

Delivery Evaluation


Query (new)

Multimedia Behavior

LearnerRecords


(history)

A s s e s s m e n t

LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


51

LearningContent


• #1: Transferring Information• #2: Feedback loop: unreliable Learners• #3: Learner Records: history supports intelligent inferences, history

allows Learner migration• #4: Learning Resources: varying learner capabilities• #5: Learning Preferences: negotiate between Learner/system

Delivery Evaluation


Query (new)

Multimedia Behavior

LearnerRecords


(history)

A s s e s s m e n t

LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


52

LearningContent




Delivery Evaluation


Query (new)

Multimedia Behavior

LearnerRecords


(history)

A s s e s s m e n t

LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


53

LearningContent




Delivery Evaluation


Query (new)

Multimedia Behavior

LearnerRecords


(history)

A s s e s s m e n t

LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


54

Illustration 2:“Learner-Directed” Approach

• Learner is self-directed• Learner makes decisions about learner’s

direction and learning experiences• Representative of independent, self-taught

learning scenarios• Focus on humans as “directors of their

experience”• Compatible with “traditional approach” when

viewed via LTSA system components


55

Step #1: Choosing His/Her Direction

• “Individual Learner” has some goal• “Individual Learner”

– is the Learner Entity– is aware of his/her Learning Preferences– chooses his/her learning direction — “Individual Learner” also serves as

role of Coach

Coach

LearnerEntity

“Individual Learner”Has Some Learning Goal

LearningPreferences


56

Step #2: Locating Learning Resources

• “Individual Learner”– searches (Query) the Learning Resources– locates learning resources described by Catalog Info


LearnerEntity

“IndividualLearner”

“Individual Learner” searches (via Query)the Learning Resourcesto locate desirableLearning Resources(via Catalog Info)

Query

LearningPreferences

Catalog Info


57

Step #3: Using Learning Resources

• “Individual Learner”– uses Learning Recourses– “initiates” Delivery of Multimedia


LearnerEntity


“Individual Learner” usesthe Learning Resources

Query

LearningPreferences

Catalog Info

Delivery

Multimedia

L o c a t o rLocatorLearning

Content


58

Step #4: Learner Self-Evaluates/Directs

• “Individual Learner”– self-evaluation– learner chooses next/new/other learning experiences


LearnerEntity


Query

LearningPreferences

Catalog Info

Delivery

Multimedia


Content

Evaluation

Behavior

A s s e s s m e n t

Interaction Context


59

Step #5: Records Are Kept

• “Individual Learner”– may uses records to track progress– records may be used to improve self-directed “learner quality”


LearnerEntity


Query

LearningPreferences

Catalog Info

Delivery

Multimedia


Content

Evaluation

Behavior

A s s e s s m e n t

Interaction Context

LearnerRecords

(new)



(history)

“Records”


60

Layer 2: Conclusions

• Several possible “stories” motivate the need to address:– Learner-centric perspective– Traditional classroom/training scenarios– Self-directed learning– Pervasive design issues caused by humans

• Other stories are possible– Only two stories were illustrated


61

Layer 3: System Components[using Yourdon system notation]

• Processes: Learner Entity, Evaluation, Coach, Delivery• Stores: Learner Records, Learning Resources• Flows: Behavior, Assessment, Performance/Preferences (past, present, future),

Query, Catalog Info, Locator, Learning Content, Multimedia, Interaction Context, Learning Preferences

Delivery Evaluation


Multimedia Behavior

LearningPreferences

LearnerRecords

A s s e s s m e n tL o c a t o r

LearnerEntity

Query (new)



(history)Catalog Info

LocatorLearningContent

Interaction Context


62

LearningContent

(history)

A s s e s s m e n t

Learner Entity[a “Yourdon process”; “implemented” by humans]

• Abstraction of learner• Can represent more than one learner• Learner Entity:

– Receives Multimedia (one-way data flow)– Emits Behavior (one-way data flow)– Negotiates Learning Preferences (two-way data flow)

Delivery Evaluation


Query (new)

Multimedia Behavior

LearnerRecords


LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


63

LearningContent

Multimedia

(history)

A s s e s s m e n t

Learning Preferences[a “Yourdon 2-way data flow”]

• Learner Entity’s preferences (styles, language, cultural, etc.) as input to system• Coach can negotiate and/or restrict Learner Entity’s preferences• Surrogates, such as:

– parentlearner entity– teachercoach

• Other possible surrogates:– parentcoach– learnercoach– mentorlearner entity

Delivery Evaluation


Query (new)

Behavior

LearnerRecords


LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


64

LearningContent

(history)

A s s e s s m e n t

Behavior[a “Yourdon 1-way data flow”]

• Observable Behavior (data flow) of the Learner Entity• Coded, and sent to Evaluation (process)

Delivery Evaluation


Query (new)

Multimedia Behavior

LearnerRecords


LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


65

LearningContent

(history)

A s s e s s m e n t

Evaluation[a “Yourdon data process”]

• Evaluation:– Receives Behavior (input data flow)– Receives Interaction Context (input data flow), i.e., context of Learner Entity’s

Behavior– May retrieve Performance info (two-way data flow) to support Evaluation– Stores Performance info (two-way data flow) based on Evaluation of Behavior– Produces Assessment (output data flow) given to Coach

Delivery Evaluation


Query (new)

Multimedia Behavior

LearnerRecords


LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


66

LearningContent

Multimedia

(history)

A s s e s s m e n t

Performance Information Stored/Retrieved By Evaluation[a “Yourdon 2-way data flow”]

• Evaluation of Behavior: can emit Performance info• Evaluation may retrieve previous Performance info to assist Evaluation• Stored in Learner Records

Delivery Evaluation


Query (new)

Behavior

LearnerRecords


LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


67

LearningContent

(history)

A s s e s s m e n t

Learner Records[a “Yourdon data store”]

• Stores Performance information as Learner Entity “history”• Retrieves and stores Preference information for the Learner Entity• Retrieves and stores Learner Entity “history” records for Coach• Records aren’t restricted to historical information, may be “current lessons”

or “future objectives”

Delivery Evaluation


Query (new)

Multimedia Behavior

LearnerRecords


LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


68

LearningContent

Multimedia

(history)

A s s e s s m e n t

Performance/Preference Info Received By Coach[a “Yourdon 1-way data flow”]

• Coach gets Learner Entity history by retrieving Performance information from (Learner Entity’s) Learner Records

• Coach can retrieve Preference information, such as media preferences• Coach can select strategies based on historical, current, or future Learner

Records information

Delivery Evaluation


Query (new)

Behavior

LearnerRecords


LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


69

LearningContent

(history)

A s s e s s m e n t

Performance/Preference Info Stored By Coach[a “Yourdon 1-way data flow”]

• Coach can store Performance information:– historical information (e.g., results of lessons) — info about past– “suspends” (incomplete sessions) — info about present– certifications (milestones, completion of module)– objectives (outlines for future sessions) — info about future

• Coach can update Preference information

Delivery Evaluation


Query (new)

Multimedia Behavior

LearnerRecords


LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


70

LearningContent

Multimedia

(history)

A s s e s s m e n t

Assessment Information[a “Yourdon 1-way data flow”]

• Evaluation (process) can emit Assessments– “Where there learner is at”– information about the learner’s current state– used by the Coach to determine optimal learning experience

• Assessment information (data flow)– may be full and detail– have narrow scope determined by current context– but it is always related to the learner’s current activity

Delivery Evaluation


Query (new)

Behavior

LearnerRecords


LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


71

LearningContent

(history)

A s s e s s m e n t1

Coach[a “Yourdon data process”]

• Sends, receives, and “negotiates” Learning Preferences (two-way data flow) with Learner Entity

Delivery Evaluation


Query (new)

Multimedia Behavior

LearnerRecords


1

LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


72

LearningContent

(history)

A s s e s s m e n t

2

3


• Receives current Assessment information (input data flow), Preferences (input data flow), and Performance information (input data flow), e.g., history and objectives, for determining future learning experiences

Delivery Evaluation


Query (new)

Multimedia Behavior

LearnerRecords


2 3

LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


73

LearningContent

(history)

A s s e s s m e n t


• Queries to the Learning Resources:– Queries (output control flow) sent to search for appropriate material– Catalog Info (input data flow), a.k.a. learning object metadata, returned as

“found” Learning Content– Extracts (“peels”) the Locators (e.g., URLs) from the returned Catalog Info

(learning content metadata)

Delivery Evaluation


Query (new)

Multimedia Behavior

LearnerRecords


4

4

LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


74

LearningContent

Multimedia

(history)

A s s e s s m e n t5


• Locators (similar to web URLs) are sent to Delivery process to identify (but not transfer) Learning Content

• Note:– Delivery (process) is responsible for retrieving– Learning Resources (store) is responsible for transferring (emitting) Learning

Content

Delivery Evaluation


Query (new)

Behavior

LearnerRecords


5

LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


75

LearningContent

(history)

A s s e s s m e n t

Query[a “Yourdon 1-way control flow”]

• Query specifies “what to search for” in Learning Resources • Coach sends Queries (control flow) to search the Learning Resources

(store) for appropriate Learning Content, based on:– Coach (e.g., learner-directed)– Learning Preferences– Assessment info– Performance and Preference info from Learner Records– Possibly, other information outside of LTSA

Delivery Evaluation


Query (new)

Multimedia Behavior

LearnerRecords


LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


76

LearningContent

Multimedia

(history)

A s s e s s m e n t

Learning Resources[a “Yourdon data store”]

• Database searched via Query• Result of Query: Catalog Info of Learning Content matching the Query

(input control flow)• Catalog Info is also known as:

– learning content cataloging information– learning object metadata

• Emits Learning Content (output data flow) based on Locator (input control flow) from Delivery (process)

Delivery Evaluation


Query (new)

Behavior

LearnerRecords


LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


77

LearningContent

(history)

A s s e s s m e n t

Catalog Info (a.k.a. Learning Object Metadata)[a “Yourdon 1-way data flow”]

• Catalog Info is also known as:– learning content cataloging information– learning object metadata

• Search of Learning Resources (store) returns Catalog Info for Learning Content that matches Query

• Catalog Info (data flow) is similar to card catalog entries in a library• Locators (e.g., URLs) can be extracted (“peeled”) from Catalog Info

Delivery Evaluation


Query (new)

Multimedia Behavior

LearnerRecords


LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


78

LearningContent

(history)

A s s e s s m e n t

Locator Received By Delivery[a “Yourdon 1-way data flow”]

• Coach (process) extracts Locator from the Catalog Info for appropriate learning experience

• Locator (data flow) is similar to call number in a library, and similar to URL of the World Wide Web

Delivery Evaluation


Query (new)

Multimedia Behavior

LearnerRecords


LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


79

LearningContent

Multimedia

(history)

A s s e s s m e n t

Locator Sent By Delivery[a “Yourdon 1-way control flow”]

• Delivery (process) sends Locator (control flow) to the Learning Resources (store) for pointing to Learning Content

• Locator sent by Delivery is not necessarily same as Locator received by Delivery — Delivery might not request Learning Content is exactly the same order that was supplied by Coach

Delivery Evaluation


Query (new)

Behavior

LearnerRecords


LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


80

LearningContent

Multimedia

(history)

A s s e s s m e n t

Learning Content[a “Yourdon 1-way data flow”]

• Learning Resources (store) returns Learning Content (data flow) to Delivery (process) as specified by Locator (control flow)

Delivery Evaluation


Query (new)

Behavior

LearnerRecords


LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


81

LearningContent

(history)

A s s e s s m e n t

Delivery[a “Yourdon process”]

• Receives Locator (input data flow), e.g., URLs, that identify learning experiences desired by the Coach

• Uses Locator (output control flow) to request Learning Content from Learning Resources (store)

• Delivery (process) transforms Learning Content (input data flow) to interactive Multimedia (output data flow), which is sent to Learner Entity

• Sends Interaction Context (output data flow) to Evaluation (process) to give context of Learner Entity’s Behavior

Delivery Evaluation


Query (new)

Multimedia Behavior

LearnerRecords


LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


82

LearningContent

(history)

A s s e s s m e n t

Interaction Context[a “Yourdon 1-way data flow”]

• Delivery (process) sends Interaction Context (data flow) to Evaluation (process) to give context of Learner Entity’s Behavior

Delivery Evaluation


Query (new)

Multimedia Behavior

LearnerRecords


LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


83

LearningContent

(history)

A s s e s s m e n t

Multimedia[a “Yourdon 1-way data flow”]

• Delivery (process) generates interactive Multimedia (data flow) learning experience based on Learning Content

Delivery Evaluation


Query (new)

Multimedia Behavior

LearnerRecords


LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


84

Functional vs. Actual Learning Technology Systems

• LTSA components identify functional boundaries and critical interfaces

• Actual systems integrate several components into individual products

• Functionality exists in actual systems but may be– embedded within larger system (several components

combined), or– a single component is spread across several systems


85

LTSA Layer 3 Analogy:Stereo Component Architecture

• Well-defined components: tuner, CD-player, tape loop, pre-amp, amplifier, speakers

• Well-defined critical interfaces: media, phono jacks, unbalanced inputs, 4-16 ohm outputs

• Not all components need exist, e.g., stereo system without a CD-player

• Engineering/business rationale for highly integrated subsets (receivers, “boom box”), yet all have interoperability at critical interfaces, e.g., audio in (aux in), audio out (speakers), media (CDs)


86

Sample LTSA MappingsShows Relationship Types: 11, 1N, N1

• Web-based learning/training– Demonstrates tightly integrated components

• Flight simulator/flight instructor– Demonstrates shared component responsibility

• Home study course– Demonstrates student can have multiple roles

• Non-electronic, traditional classroom– Limiting case, no technology involved


87

Interaction Context

Query

Catalog Info

(new)

(history)


LearningPreferences

LearningContent

Example Mapping #1: Web-Based LearningDemonstrates Tightly Integrated Components

Delivery Evaluation


Multimedia Behavior

LearnerRecords


A s s e s s m e n t

Human

HumanInterface

e.g., X WindowsWin95

PresentationTool

(browser)Courseware

Database(web servers)

StudentRecords

(database)

LearnerEntity

L o c a t o rLocator

Human Learner Entity

AbstractionImplementation


88

Interaction Context

Query

Catalog Info

(new)

(history)


LearningPreferences

LearningContent


Delivery Evaluation


Multimedia Behavior

LearnerRecords


A s s e s s m e n t

Human

HumanInterface


PresentationTool

(browser)Courseware


StudentRecords

(database)

LearnerEntity


Windowing System LTSA Multimedia And

BehaviorAbstractionImplementation


89

Interaction Context

Query

Catalog Info

(new)

(history)


LearningPreferences

LearningContent


Delivery Evaluation


Multimedia Behavior

LearnerRecords


A s s e s s m e n t

Human

HumanInterface


PresentationTool

(browser)Courseware


StudentRecords

(database)

LearnerEntity


Web Browser,Presentation Tools Packaged AsCombination Of

LTSA Evaluation, Coach, Delivery



90

Query

Catalog Info

(new)

(history)


LearningPreferences

LearningContent


Delivery Evaluation


Multimedia Behavior

LearnerRecords


A s s e s s m e n t

Human

HumanInterface


PresentationTool

(browser)Courseware


StudentRecords

(database)

LearnerEntity


Web Servers,Courseware Databases LTSA Learning Resources

Interaction Context



91

Query

Catalog Info

(new)

(history)


LearningPreferences

LearningContent


Delivery Evaluation


Multimedia Behavior

LearnerRecords


A s s e s s m e n t

Human

HumanInterface


PresentationTool

(browser)Courseware


StudentRecords

(database)

LearnerEntity


Student Records Database LTSA Learner Records

Interaction Context



92

Interaction Context

Query

Catalog Info

LearningPreferences

LearningContent


Delivery Evaluation


(new)

Multimedia Behavior

LearnerRecords


(history)

A s s e s s m e n t

Human

HumanInterface


PresentationTool

(browser)Courseware


StudentRecords

(database)

LearnerEntity

* Note: Other mappings may co-exist.





93

Interaction Context

(new)

(history)


Query

Catalog Info

LearningPreferences

LearningContent

Example Mapping #2: Flight Simulator, Flight InstructorDemonstrates Shared Component Responsibility: Coach

Delivery Evaluation


Multimedia Behavior

LearnerRecords


A s s e s s m e n t

FlightInstructor

FlightSimulator

Pilot’sLogbook

Pilot

LearnerEntity





94

Interaction Context

(history)

(new)


Query

Catalog Info

LearningPreferences

LearningContent

Multimedia

A s s e s s m e n t

Example Mapping #3: Home Study CourseDemonstrates Student Can Have Multiple Roles/Responsibilities

Delivery Evaluation


Behavior

LearnerRecords


SchoolHomeStudy

CourseStudent

LearnerEntity





95

Interaction Context

(history)

(new)


Query

Catalog Info

LearningPreferences

LearningContent

Multimedia

A s s e s s m e n t

Example Mapping #4: Non-Electronic, Traditional ClassroomLimiting Case: No Technology Involved

Delivery Evaluation


Behavior

LearnerRecords


Teacher ReportCards,

Transcript

Student

SchoolLibrary

Student UsesLibrary Teacher Points Student To

Books (A Locator)Retrieves TeachingMaterials

Grades

LearnerEntity





96

Layer 4: Stakeholder Perspectives

• Each stakeholder has a view, but not complete picture

• Show each stakeholder perspective within LTSA• Stakeholder perspectives used to validate views and

subsets of LTSA components• “It’s in there!” (if not, let us know!)


97

LTSA Layer 4: Stakeholder PerspectivesGlobal Information Infrastructure (GII) Experience

• Common technology does not imply common architecture

• Stakeholder perspectives analyzed by:– Primary design issues: LTSA components that greatly

affect design and implementation for stake holder (shown as red/bold)

– Secondary design issues: LTSA components that affect design and implementation of stake-holder, but not critical (shown as blue/dash ---)

– Other LTSA components: insignificant design issue or not applicable (no highlight/green fill)


98

LTSA Layer 4: Stakeholder PerspectivesConsensus-Building

• Stakeholders, typically, have different and/or incompatible priorities

• Differing priorities: obstacles to consensus• Non-technical (political, business) issues just as

important as technical issues• Recognizing technical/business/political priorities

AND common technology can lead to common architecture

• Cataloging stakeholder perspectives is critical for building consensus


99

LearningContent

(history)

A s s e s s m e n t

Abstraction-Implementation:System Components Stakeholder/Design Perspectives

Delivery Evaluation


Query (new)

Multimedia Behavior

LearnerRecords



Various Stakeholder/Design PerspectivesExamples: Learner, Assessment, Institution, Metadata, Intelligent Tutor, Mentoring, Collaboration, Curriculum, Developer, Task

LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


100

Stakeholder Categories“Isolated” stakeholdersLearner-Centered, Institution-Centered,Assessment-Centered, Records,Certifications, Task Model, School-to-Work,Content-Centered, Content Developers,Metadata, Ontologies, Expert Systems,Digital Libraries, Learning Objects,Multimedia Search/Retrieval,Collaboration, Peripheral Devices,Asynchronous Learning, Multiple Role,Team Learning, Icon Conventions

“Parallel” stakeholdersParallel Sessions/Same LearnerStudent-TeacherMulti-Tier Process Improvement:

principal teacher student

“Overlapping” stakeholdersMentoring, Coaching,Electronic Performance SupportInteractive Environment, SimulationTool-to-Tool Communication,Sequencing, Pre-Requisites,Curriculum-Centered,Experimentation, Discovery,Intelligent Tutoring Tools,Distance/Distributed/Nomadic Learning

Related IndustriesHuman Factors/User Interfaces,Data Collection/Analysis,IT Decision-Support Applications,Expert Systems/Intelligent Systems,Entertainment/Multimedia Industry,Control/Feedback Systems


101

“Isolated” LTSA Stakeholders

• Isolated LTSA subsets -- little interaction with neighboring LTSA components

• Little overlap with other “isolated” stakeholders• Relatively “simple” integration• Can make use of isolated component standards• Typical stakeholder examples: “Records”,

“Metadata”, “Multimedia Search/Retrieval”


102

LearningContent

Multimedia

(history)

A s s e s s m e n t

Learner-Centered

• Primary design issues: learner keeps his/her records, learner has influence on negotiated learning preferences

• Secondary design issues: assessment info and performance info

Delivery Evaluation


Query (new)

Behavior

LearnerRecords


LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


103

LearningContent

(history)

A s s e s s m e n t

Assessment-Centered

• Primary design issues: educational standards, behavior, evaluation, assessment records, learner records reporting

• Secondary design issues: learning preferences, learner, process control (adaptive administration)

Delivery Evaluation


Query (new)

Multimedia Behavior

LearnerRecords


LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


104

LearningContent

(history)

A s s e s s m e n t

Records, Certifications

• Primary design issues: learner records, performance info, assessment info

• Secondary design issues: evaluation, coach

Delivery Evaluation


Query (new)

Multimedia Behavior

LearnerRecords


LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


105

LearningContent

(history)

A s s e s s m e n t

Learner Profiles

• Primary design issues: learner preferences, performance info, assessment info, learner records

• Secondary design issues: evaluation, coach

Delivery Evaluation


Query (new)

Multimedia Behavior

LearnerRecords


LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


106

LearningContent

(history)

A s s e s s m e n t

Student Administration Systems

• Primary design issues: learner records, student registration, institution back office

• Secondary design issues: performance info

Delivery Evaluation


Query (new)

Multimedia Behavior

LearnerRecords


LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


107

LearningContent

(history)

A s s e s s m e n t

Task Model, School-To-Work

• Primary design issues: learner records• Secondary design issues: performance info

Delivery Evaluation


Query (new)

Multimedia Behavior

LearnerRecords


LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


108

LearningContent

(history)

A s s e s s m e n t

Institution-Centered

• Primary design issues: system control, administration, records, reporting, negotiating (many times: asserting) learning preferences

• Secondary design issues: learner collaboration, teachers, evaluation (grading), delivery infrastructure (including multimedia and behavior observation correlated by learning content)

Delivery Evaluation


Query (new)

Multimedia Behavior

LearnerRecords


LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


109

LearningContent

(history)

A s s e s s m e n t

Learning Content Cataloging, Metadata

• Primary design issues: query, catalog info (metadata), locator (e.g., URLs), and associating locators with catalog info

• Secondary design issues: knowledge organization, learning content (presentation)

Delivery Evaluation


Query (new)

Multimedia Behavior

LearnerRecords


LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


110

LearningContent

Multimedia

(history)

A s s e s s m e n t

Ontologies, Expert Systems

• Primary design issues: knowledge design and knowledge representation in learning resources, learning content (presentation)

• Secondary design issues: query, catalog info (metadata), and locator (e.g., URLs)

Delivery Evaluation


Query (new)

Behavior

LearnerRecords


LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


111

LearningContent

(history)

A s s e s s m e n t

Digital Libraries

• Primary design issues: knowledge organization, query, catalog info (metadata), locator (e.g., URLs), and associating locators with catalog info, learning content (presentation)

• Secondary design issues: delivery, multimedia

Delivery Evaluation


Query (new)

Multimedia Behavior

LearnerRecords


LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


112

LearningContent

Multimedia

(history)

A s s e s s m e n t

Learning Objects

• Primary design issues: learning resources, query, catalog info (metadata), learning content (presentation)

• Secondary design issues: locator (e.g., URLs), delivery, coach, performance and preference info, assessment info

Delivery Evaluation


Query (new)

Behavior

LearnerRecords


LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


113

LearningContent

Multimedia

(history)

A s s e s s m e n t

Content Launch

• Primary design issues: coach, locator (e.g., URLs), delivery• Secondary design issues: evaluation, performance info,

assessment info, learning resources, catalog info (metadata), learning content (presentation)

Delivery Evaluation


Query (new)

Behavior

LearnerRecords


LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


114

LearningContent

(history)

A s s e s s m e n t

Content-Centered

• Primary design issues: delivery, multimedia, invocation via locator (e.g., URLs), learning content format

• Secondary design issues: assessment, content search (query, catalog info), learning resources, learner’s user interface

Delivery Evaluation


Query (new)

Multimedia Behavior

LearnerRecords


LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


115

LearningContent

Multimedia

(history)

A s s e s s m e n t

Content Objects

• Primary design issues: learning content, catalog info (metadata)• Secondary design issues: coach, locator (e.g., URLs), delivery,

learning resources

Delivery Evaluation


Query (new)

Behavior

LearnerRecords


LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


116

LearningContent

(history)

A s s e s s m e n t

Content Packaging

• Primary design issues: learning resources, catalog info (metadata), learning content (presentation)

• Secondary design issues: coach, locator (e.g., URLs), delivery

Delivery Evaluation


Query (new)

Multimedia Behavior

LearnerRecords


LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


117

LearningContent

(history)

A s s e s s m e n t

Content Developer

• Primary design issues: coach, content search (query, catalog info), delivery, invocation via locator (e.g., URLs), learning content format, delivery, multimedia,

• Secondary design issues: learner collaboration, learner preferences, assessment info, performance info, learning resources, interaction context

Delivery Evaluation


Query (new)

Multimedia Behavior

LearnerRecords


LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


118

LearningContent

Multimedia

(history)

A s s e s s m e n t

Digital Audio and Video

• Primary design issues: multimedia• Secondary design issues: delivery

Delivery Evaluation


Query (new)

Behavior

LearnerRecords


LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


119

LearningContent

Multimedia

(history)

A s s e s s m e n t

Multimedia Search/Retrieval

• Primary design issues: delivery, multimedia, query, catalog info (metadata), locator (e.g., URLs), hardware limitations

• Secondary design issues: coach, learning resources, learning content format, behavior

Delivery Evaluation


Query (new)

Behavior

LearnerRecords


LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


120

LearningContent

Multimedia

(history)

A s s e s s m e n t

Peripheral Devices

• Primary design issues: behavior, delivery, multimedia• Secondary design issues: learner

Delivery

\

Evaluation


Query (new)

Behavior

LearnerRecords


LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


121

LearningContent

(history)

A s s e s s m e n t

Collaboration, Asynchronous Learning

• Primary design issues: communication among learners with same roles (learner entity), communication modes (multimedia), delivery tools, and differing “time zones”

• Secondary design issues: system control by coordinator/moderator, learning preferences negotiated among participants and coordinator/moderator

Delivery Evaluation


Query (new)

Multimedia Behavior

LearnerRecords


LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


122

LearningContent

Multimedia

(history)

A s s e s s m e n t

LearnerEntity

Multiple Role, Team Learning

• Primary design issues: communication among learners, but different roles (learner entity), communication modes (multimedia), delivery tools, and differing “time zones”

• Secondary design issues: system control by coordinator/moderator, learning preferences negotiated among participants and coordinator/moderator

Delivery Evaluation


Query (new)

Behavior

LearnerRecords



LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


123

LearningContent

(history)

A s s e s s m e n t

Icon Conventions

• Primary design issues: learner, learning content• Secondary design issues: delivery, multimedia

Delivery Evaluation


Query (new)

Multimedia Behavior

LearnerRecords


LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


124

“Overlapping” LTSA Stakeholders

• Concerns many, most, or all LTSA components• Overlaps with other stakeholders• Differing, conflicting designs priorities• Relatively “complex” integration• Potential non-technical, political, or business

obstacles to consensus and interoperability• Typical stakeholder examples: “Experimentation”,

“Intelligent Tutoring Tools”, “Distance Learning”


125

LearningContent

Multimedia

(history)

A s s e s s m e n t

Mentoring, Coaching

• Primary design issues: learner and mentor/coach jointly perform evaluation and jointly choose direction

• Secondary design issues: tools and delivery support mentoring, coaching, experimentation, recordkeeping

Delivery Evaluation


Query (new)

Behavior

LearnerRecords


LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


126

LearningContent

Multimedia

(history)

A s s e s s m e n t

Electronic Performance Support Systems

• Primary design issues: coach chooses appropriate content based on learner’s environment, interaction context, behavior, evaluation, and assessment

• Secondary design issues: learner records, performance information, locator, delivery, multimedia

Delivery Evaluation


Query (new)

Behavior

LearnerRecords


LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


127

LearningContent

(history)

A s s e s s m e n t

Interactive Environment

• Primary design issues: learner specifies preferences, learner is observed, delivery is responsive to evaluation

• Secondary design issues: assessment info, coach, locator (e.g., URLs)

Delivery Evaluation


Query (new)

Multimedia Behavior

LearnerRecords


LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


128

LearningContent

Multimedia

(history)

A s s e s s m e n t

Simulation

• Primary design issues: delivery, multimedia, learning content to correlate multimedia and behavior, assessment info, coach, locator

• Secondary design issues: behavior, performance info, learner records

Delivery Evaluation


Query (new)

Behavior

LearnerRecords


LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


129

LearningContent

(history)

A s s e s s m e n t

Learning Tool-To-Tool Communication

• Primary design issues: coach, evaluation, assessment info, query, catalog info (metadata), locator (e.g., URLs), delivery, invocation

• Secondary design issues: multimedia, learning preferences, performance info, learner records, learning resources, learning content

Delivery Evaluation


Query (new)

Multimedia Behavior

LearnerRecords


LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


130

LearningContent


(history)

A s s e s s m e n t

Sequencing, Pre-Requisites, Co-Requisites

• Primary design issues: coach, evaluation, assessment info, query, catalog info (metadata), locator (e.g., URLs), delivery, invocation

• Secondary design issues: multimedia, learning preferences, performance info, learner records, learning resources, learning content

Delivery Evaluation


Query (new)

Multimedia Behavior

LearnerRecords

LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


131

LearningContent

(history)

A s s e s s m e n t

Curriculum-Centered

• Primary design issues: evaluation, learner records, catalog info, learning resources, learning content format, delivery

• Secondary design issues: learner collaboration, assessment info, performance info, content search (query), invocation via locator (e.g., URLs), multimedia

Delivery Evaluation


Query (new)

Multimedia Behavior

LearnerRecords


LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


132

LearningContent

Multimedia

(history)

A s s e s s m e n t

Content Management Systems,Entertainment Systems

• Primary design issues: learner, coach, locator (e.g., URLs), multimedia

• Secondary design issues: learning preferences, learner records, performance info,catalog info (metadata), learning resources, delivery

Delivery Evaluation


Query (new)

Behavior

LearnerRecords


LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


133

LearningContent

Multimedia

(history)

A s s e s s m e n t

Learning Management Systems

• Primary design issues: performance info, assessment info, query, catalog info (metadata), locator (e.g., URLs), invocation of learning content

• Secondary design issues: behavior, learner records, coach, learning resources, learning content, delivery

Delivery Evaluation


Query (new)

Behavior

LearnerRecords


LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


134

LearningContent

Multimedia

(history)

A s s e s s m e n t

Experimentation, Discovery

• Primary design issues: learner chooses direction, acquires knowledge during experimentation, learner, coach, query, catalog info (metadata)

• Secondary design issues: learner does self evaluation, tools and delivery support experimentation, behavior, assessment info, locator (e.g., URLs), delivery, multimedia

Delivery Evaluation


Query (new)

Behavior

LearnerRecords


LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


135

LearningContent

Multimedia

(history)

A s s e s s m e n t

Intelligent Tutoring Tools

• Primary design issues: learner/tutor choose direction, acquires knowledge during use of tutoring tool, learning resources may be implicit (not explicitly defined) in tutor

• Secondary design issues: tutor does evaluation, tools and delivery support experimentation

Delivery Evaluation


Query (new)

Behavior

LearnerRecords


LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


136

LearningContent

Multimedia

(history)

A s s e s s m e n t

Distance Learning, Distributed Learning,And Nomadic Learning

• Primary design issues: distributed and nomadic communication of all “flows” (behavior, performance, multimedia, etc.)

• Secondary design issues: processes and stores components• NOTE: Take note of primary design issues (red/bold)

– How does this slide differ from all others?– Answer: All flows are primary, everything else is secondary– A technical definition of distance/distributed learning: primary design

issues concern “flows”, all other LTSA components are secondary

Delivery Evaluation


Query (new)

Behavior

LearnerRecords


LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


137

“Parallel” LTSA Stakeholders

• Integration of multiple sessions, each with differing learning experiences and interactions

• Synchronization of multiple sessions: starting, stopping, coaching, learner interfacing

• Integration, collaboration, and synchronization of feedback and coaching strategies

• Typical stakeholder example: “Student Teacher”


138

Parallel Sessions for Same Learner

• Primary design issues: multiple, simultaneous learning experiences; multiple flows of behavior, learning preferences, multimedia

• Secondary design issues: multiple evaluation, coach, and delivery processes• Example: flight simulator -- simultaneous training of flying skills, navigation skills,

communication skills, cockpit resource management

Example: Aviate

Example: Communicate

Example: Navigate


139

Student

Teacher

Student Teacher

• Primary design issues: student teacher as a teacher• Secondary design issues: student teacher as a learner entity


140

Principal

Teacher

Student

Multi-Tiered Process Improvement

• Student is evaluated by teacher, teacher coaches student• Teacher is evaluated by principal, principal coaches teacher • Principal is evaluated by school board, school board coaches principal• Teacher is also evaluated by performance of his/her students• Principal is also evaluated by performance of his/her students


141

“Related Industries” LTSA Stakeholders

• Other industries overlap learning technology• LTSA components described in terms of other industries• Related industries represent significant portions of LTSA system

components• Related industries only identify common areas of technology,

not common purpose. Examples:– Learning isn’t just entertainment– Learning isn’t just control/feedback systems– And so on ...

• Related industries have architectures different from LTSA– LTSA is not a good architecture for describing generic IT decision support

systems, data collection/analysis systems, user interfaces, and so on ...


142

LearningContent

(history)

A s s e s s m e n t

Related Industries: Human Factors/User Interfaces“The LTSA Top Half”

• Primary design issues: human interfaces, human inputs (behavior), systems that interpret human inputs (evaluation), machine outputs (multimedia), systems that generate machine output (delivery)

• Secondary design issues: responsiveness (interaction context), adaptation (learning preferences)

Delivery Evaluation


Query (new)

Multimedia Behavior

LearnerRecords


LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


143

LearningContent

(history)

A s s e s s m e n t

Related Industries: Data Collection/Analysis“The LTSA Bottom-Right Quadrant”

• Primary design issues: repositories of information (learner records), information formats and their storage/retrieval (assessment info, performance info, preference info)

• Secondary design issues: application paradigms for accessing and processing information (coach, evaluation)

Delivery Evaluation


Query (new)

Multimedia Behavior

LearnerRecords


LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


144

LearningContent


(history)

A s s e s s m e n t

Related Industries: IT Decision-Support Applications“The LTSA Bottom Half”

• Primary design issues: processing of information (coach), supporting databases (learner records, learning resources), information formats and their storage/retrieval (learning preferences, assessment info, performance info, preference info, query, catalog info, locator)

• Secondary design issues: front-end, back-end, second-tier, and agent processing (evaluation, delivery, learning content)

Delivery Evaluation


Query (new)

Multimedia Behavior

LearnerRecords

LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


145

LearningContent

(history)

A s s e s s m e n t

Related Industries: Expert Systems/Intelligent Systems“The LTSA Bottom-Left Quadrant”

• Primary design issues: knowledge/expertise repositories (learning resources), access/usage methods (query, catalog info, locator, learning content)

• Secondary design issues: applications (coach, delivery)

Delivery Evaluation


Query (new)

Multimedia Behavior

LearnerRecords


LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


146

LearningContent

Multimedia

(history)

A s s e s s m e n t

Related Industries: Entertainment and Multimedia“The LTSA Left Half”

• Primary design issues: content (learning resources, learning content), delivery stream (delivery, multimedia)

• Secondary design issues: content selection (query, catalog info, locator), user/viewer/subscriber (learner entity)

Delivery Evaluation


Query (new)

Behavior

LearnerRecords


LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


147

LearningContent

Multimedia

(history)

A s s e s s m e n t

Related Industries: Control/Feedback Systems“The LTSA Center”

• Primary design issues: system (learner entity), sensor (behavior, evaluation, assessment info), controller (coach), actuator (locator, delivery, multimedia)

• Secondary design issues: second-order/alternative feedback (interaction context, learning preferences)

Delivery Evaluation


Query (new)

Behavior

LearnerRecords


LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


148

OLD: Related Standards/Specification Development Orgs In Education & Learning Technology

LTSC(P1484)

Specs submitted by Consortia/Fora

IEEE, ANSI, ISO: Accredited Standards

GEMPROMETEUS

GESTALT

Stds/Specs

http://ltsc.ieee.org/

http://grouper.ieee.org/groups/ltsc

http://ariadne.unil.ch/

http://www.adlnet.org/

http://www.ansi.org/

http://www.iso.ch/

http://www.ieee.org/

http://www.aicc.org/

http://www.imsproject.org/


149

NEW: Related Standards/Specification Development Orgs In Education & Learning Technology

LTSC(P1484)

LTSC & JTC1/SC36: Close Collaboration

ISO/IECJTC1/SC36

GEMPROMETEUS

GESTALT

Current, existingFuture possibilities

Sampling ofOrganizations



http://www.iso.ch/







150

Related Standards/Specification Development Orgs: Education & Learning Technology

LTSC(P1484)


ISO/IECJTC1/SC36

GEMPROMETEUS

GESTALT


Sampling ofOrganizations



http://www.iso.ch/







151

URLs For AccreditedStandards Development Organizations

• ISO: International Standards Organizationhttp://www.iso.ch

• ANSI: American National Standards Institutehttp://www.ansi.org

• IEEE: Institute for Electrical and Electronic Engineers http://www.ieee.org

• IEEE LTSC (P1484): Learning Technology Standards Committee http://ltsc.ieee.org

http://edutool.com/ltsa



http://www.iso.ch/

http://grouper.ieee.org/p1484/ltscdocs/index.html




http://edutool.com/papi




http://edutool.com/sid




152

IEEE LTSC (P1484) As A Stakeholder“The LTSC Roadmap”

• IEEE LTSC (P1484) working groups (WGs) and study groups (SGs) each involve certain areas of LTSA

• Some LTSA components should not be standardized, e.g., evaluation methods, delivery systems

• Some LTSA components should be standardized outside LTSC (P1484), e.g., multimedia, education standards, cultural adaptation

• Strategy: Standardize smallest, useful, doable specification that has technically feasibility, commercial viability, and widespread adoption


153

LearningContent

(history)

A s s e s s m e n t

Learning Technology That Should Not Be StandardizedExample: Evaluation

• Evaluation should not be standardized because:– Process/methods are not well-defined or agreed upon– Value-added feature of learning technology systems

• However ... inputs and outputs should be standardized:– Behavior coding methods– Learning content formats– Performance information– Assessment information

Delivery Evaluation


Query (new)

Multimedia Behavior

LearnerRecords


LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


154

LearningContent

(history)

A s s e s s m e n t

Learning Technology That Should Not Be StandardizedExample: Delivery

• Delivery should not be standardized because:– Value-added feature of learning technology systems– Commercial vendors provide “implementation quality” with delivery systems

• However ... inputs and outputs should be standardized:– Multimedia formats– Learning content format– Locators and launch methods– Standards profiles for browsers

Delivery Evaluation


Query (new)

Multimedia Behavior

LearnerRecords


LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


155

LearningContent

(history)

A s s e s s m e n t

Learning Technology Should Be Standardized Outside LTSCExample: Multimedia Formats

• Multimedia formats should be standardized outside LTSC because:– Not specific to learning technology– Other committees have more expertise

• However ... standards “profiles” (collections) should be identified:– Can point to collections of existing standards, e.g., JPEG, GIF, MPEG– Don’t lock into a single format to allow varying “implementation quality”

Delivery Evaluation


Query (new)

Multimedia Behavior

LearnerRecords


LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


156

LearningContent

Multimedia

(history)

A s s e s s m e n t

Learning Technology Should Be Standardized Outside LTSCExample: Education Standards

• Education standards should be standardized outside LTSC because:– Not technology standards– Highly political process– Local, national, and regional preferences ... unable to build consensus

• However ... education stds should build on learning technology using:– Common indexes (query, content, locator) for search/retrieval– Common formats for exchanging assessment/performance info– Common learning content structures for prerequisites and content aggregation

Delivery Evaluation


Query (new)

Behavior

LearnerRecords


LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


157

LearningContent

Multimedia

(history)

A s s e s s m e n t

Learning Technology Should Be Standardized Outside LTSCExample: Cultural Adaptation

• Cultural adaptation should be standardized outside LTSC because:– Not specific to learning technology, e.g., color blindness, language preference– Other committees have more expertise, e.g., ISO/IEC JTC1 SC35– Local, national, and regional preferences ... unable to build consensus

• However ... learning preferences can be identified independently, e.g.:– Learning styles– Mentoring/coaching/hinting preferences

• And ... LTSC should collaborate with cultural adaptation committees

Delivery Evaluation


Query (new)

Behavior

LearnerRecords


LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


158

LTSC Guidelines for Working Groups

• Choosing a new working group (WG): – smallest, useful, doable specification– technically feasibility– commercial viability– widespread adoption

• LTSA describes technology, not WG structure• No “grand framework” for new WGs (not ISO OSI)

• Large number of WGs, but many “small” standards• Quick schedule: WG formation draft standard


159

IEEE LTSC (P1484) Working/Study Groups

General Activities1484.1 Architecture and Reference Model1484.3 GlossaryLearner-Related Activities1484.2 Learner Model1484.4 Task Model1484.13 Student Identifiers1484.5 User Interfaces1484.19 Quality System for Life-Long

Learning1484.20 Competency DefinitionsContent-Related Activities1484.10 CBT Data Interchange1484.6 Course Sequencing1484.17 Content Packaging

Data and Metadata1484.12 Learning Objects Metadata1484.9 Localization1484.14 Semantics and Exchange

Bindings1484.15 Data Interchange Protocols1484.16 HTTP BindingsMgmt. Systems & Applications1484.11 Computer Managed

Instruction1484.18 Platform and Media Profiles1484.7 Tool/Agent Comm.1484.8 Enterprise Interfaces


160

IEEE LTSC (P1484)General Activities

• IEEE 1484.1 Architecture and Reference Model• IEEE 1484.3 Glossary


161

LearningContent

(history)

A s s e s s m e n t

IEEE 1484.1:Architecture and Reference Model WG

• Primary design issues: all components• Secondary design issues: none

Delivery Evaluation


Query (new)

Multimedia Behavior

LearnerRecords


LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


162

LearningContent

Multimedia

(history)

A s s e s s m e n t

IEEE 1484.3:Glossary WG

• Primary design issues: none• Secondary design issues: all components

Delivery Evaluation


Query (new)

Behavior

LearnerRecords


LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


163

IEEE LTSC (P1484)Learner-Related Activities

• IEEE 1484.2 Learner Model• IEEE 1484.4 Task Model• IEEE 1484.13 Student Identifiers• IEEE 1484.5 User Interface Icons• IEEE 1484.19 Quality System for Life-Long

Learning• IEEE 1484.20 Competency Definitions


164

LearningContent

(history)

A s s e s s m e n t

IEEE 1484.2:Learner Model WG

• Primary design issues: learner entity, learning preferences, evaluation, performance info, assessment info, coach

• Secondary design issues: behavior

Delivery Evaluation


Query (new)

Multimedia Behavior

LearnerRecords


LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


165

LearningContent

(history)

A s s e s s m e n t

IEEE 1484.4:Task Model WG

• Primary design issues: learner records, performance info (history)• Secondary design issues: performance info (current, new)

Delivery Evaluation


Query (new)

Multimedia Behavior

LearnerRecords


LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


166

LearningContent

(history)

A s s e s s m e n t

IEEE 1484.13:Student Identifiers (SID) WG

• Primary design issues: performance info, learner records• Secondary design issues: learner, coach

Delivery Evaluation


Query (new)

Multimedia Behavior

LearnerRecords


LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


167

LearningContent

(history)

A s s e s s m e n t

IEEE 1484.5:User Interfaces WG

• Primary design issues: multimedia, behavior• Secondary design issues: learner entity, learning preferences

Delivery Evaluation


Query (new)

Multimedia Behavior

LearnerRecords


LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


168

LearningContent

Multimedia

(history)

A s s e s s m e n t

IEEE 1484.19: Quality System ForLife-Long Learning WG

• Primary design issues: learner entity, evaluation, performance info, assessment info

• Secondary design issues: learning preferences, coach, learner records

Delivery Evaluation


Query (new)

Behavior

LearnerRecords


LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


169

LearningContent

(history)

A s s e s s m e n t

IEEE 1484.20:Competency Definitions WG

• Primary design issues: performance info, assessment info, learner records

• Secondary design issues: learner entity, learning preferences, evaluation, coach

Delivery Evaluation


Query (new)

Multimedia Behavior

LearnerRecords


LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


170

IEEE LTSC (P1484)Content-Related Activities

• IEEE 1484.10 CBT data interchange• IEEE 1484.6 Course sequencing• IEEE 1484.17 Content packaging


171

LearningContent

(history)

A s s e s s m e n t

IEEE 1484.10:CBT Data Interchange WG

• Primary design issues: coach, performance info, assessment info, query, catalog info (metadata), locator (e.g., URLs), invocation of learning content

• Secondary design issues: learning preferences, behavior, learner records, learning resources, learning content, delivery

Delivery Evaluation


Query (new)

Multimedia Behavior

LearnerRecords


LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


172

LearningContent

(history)

A s s e s s m e n t

IEEE 1484.6:Course Sequencing WG

• Primary design issues: coach, performance info, assessment info, query, catalog info (metadata), locator (e.g., URLs), invocation of learning content

• Secondary design issues: learning preferences, learner records, learning resources, delivery, learning content

Delivery Evaluation


Query (new)

Multimedia Behavior

LearnerRecords


LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


173

LearningContent

Multimedia

(history)

A s s e s s m e n t

IEEE 1484.17:Content Packaging WG

• Primary design issues: learning content, catalog info (metadata)• Secondary design issues: coach, locator (e.g., URLs), delivery,

learning resources

Delivery Evaluation


Query (new)

Behavior

LearnerRecords


LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


174

IEEE LTSC (P1484)Data and Metadata Activities

• IEEE 1484.12 Learning Objects Metadata• IEEE 1484.9 Localization• IEEE 1484.14 Semantics and Exchange Bindings• IEEE 1484.15 Data Interchange Protocols• IEEE 1484.16 HTTP Bindings


175

LearningContent

(history)

A s s e s s m e n t

IEEE 1484.12:Learning Objects Metadata (LOM) WG

• Primary design issues: catalog info (metadata), locator (e.g., URLs)• Secondary design issues: coach, query, learning resources,

locator, learning content, delivery

Delivery Evaluation


Query (new)

Multimedia Behavior

LearnerRecords


LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


176

LearningContent

(history)

A s s e s s m e n t

IEEE 1484.9:Localization WG

• Primary design issues: catalog info (metadata), learning resources, learning content

• Secondary design issues: coach, query, locator (e.g., URLs), delivery

Delivery Evaluation


Query (new)

Multimedia Behavior

LearnerRecords


LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


177

LearningContent

Multimedia

(history)

A s s e s s m e n t

IEEE 1484.14:Semantics and Exchange Bindings (SXB) WG

• Primary design issues: performance info, assessment info, catalog info, query

• Secondary design issues: evaluation, learner records, coach, learning resources

Delivery Evaluation


Query (new)

Behavior

LearnerRecords


LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


178

LearningContent

Multimedia

(history)

A s s e s s m e n t

IEEE 1484.15:Data Interchange Protocols WG


• Secondary design issues: evaluation, learner records, coach, learning resources

Delivery Evaluation


Query (new)

Behavior

LearnerRecords


LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


179

LearningContent

Multimedia

(history)

A s s e s s m e n t

IEEE 1484.16:HTTP Bindings WG


• Secondary design issues: evaluation, learner records, coach

Delivery Evaluation


Query (new)

Behavior

LearnerRecords


LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


180

IEEE LTSC (P1484)Mgmt. Systems and Applications Activities

• IEEE 1484.11 Computer Managed Instruction• IEEE 1484.18 Platform and Media Profiles• IEEE 1484.7 Tool/Agent Communication• IEEE 1484.8 Enterprise Interfaces


181

LearningContent

Multimedia

(history)

A s s e s s m e n t

IEEE 1484.11:Computer Managed Instruction (CMI) WG

• Primary design issues: performance info, assessment info, query, catalog info (metadata), locator (e.g., URLs), invocation of learning content

• Secondary design issues: behavior, learner records, coach, learning resources, learning content, delivery

Delivery Evaluation


Query (new)

Behavior

LearnerRecords


LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


182

LearningContent

Multimedia

(history)

A s s e s s m e n t

IEEE 1484.18:Platform Profiles WG

• Primary design issues: delivery, learning content, locator (e.g., URLs), multimedia, learner entity, behavior, learner entity

• Secondary design issues: learning preferences, evaluation, coach, query, catalog info (metadata), learning resources

Delivery Evaluation


Query (new)

Behavior

LearnerRecords


LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


183

LearningContent

Multimedia

(history)

A s s e s s m e n t

IEEE 1484.7:Tool/Agent Communication WG

• Primary design issues: coach, behavior, evaluation, assessment info, performance info, learner records, query, catalog info (metadata), locator (e.g., URLs)

• Secondary design issues: query, catalog info (metadata), learning resources, delivery

Delivery Evaluation


Query (new)

Behavior

LearnerRecords


LearnerEntity


LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


184

LearningContent

(history)

A s s e s s m e n t

IEEE 1484.8:Enterprise Interfaces WG

• Primary design issues: learning preferences, performance info, learner records

• Secondary design issues: learner entity, coach

Delivery Evaluation

LearningResources

Query (new)

Multimedia Behavior

LearnerRecords


LearnerEntity

Coach Performance/Preferences

LearningPreferences

L o c a t o r

Catalog Info

Locator

Interaction Context


185

Related Standards/Specification Development Orgs: Education & Learning Technology

LTSC(P1484)


ISO/IECJTC1/SC36

GEMPROMETEUS

GESTALT




http://www.iso.ch/







186

URLs For AccreditedStandards Development Organizations

• ISO: International Standards Organizationhttp://www.iso.ch

• ISO/IEC Joint Technology Committee 1, Information Technology, http://www.jtc1.org

• ANSI: American National Standards Institute,http://www.ansi.org

• IEEE: Institute for Electrical and Electronic Engineers, http://www.ieee.org

• IEEE LTSC (P1484): Learning Technology Standards Committee, http://ltsc.ieee.org

http://www.iso.ch/

http://www.iso.ch/

http://www.iso.ch/

http://www.iso.ch/

http://www.jtc1.org/

http://www.jtc1.org/



http://ltsc.ieee.org/ltsc






187

URLs For Related SDOs/Consortia/Fora/Orgs

• European Committee for Standardization (CEN), Information Society Standardization System (ISSS), Learning Technology (LT) Workshop http://www.cenorm.be/isss/workshop/lt

• AICC: Aviation Industry CBT Committee,http://www.aicc.org

• IMS Project, http://www.imsproject.org• ARIADNE: Alliance of Remote Instructional

Authoring and Distribution Networks of Europe http://ariadne.unil.ch

http://www.cenorm.be/isss/workshop/lt

















188

URLs For Related Consortia/Fora/Orgs

• PROmoting Multimedia access to Education and Training in EUropean Societyhttp://www.prometeus.org

• GESTALT: Getting Educational Systems Talking Across Leading-edge Technologieshttp://www.fdgroup.co.uk/gestalt

• GEM Project: Gateway to Education Materialshttp://www.geminfo.org

• ADL: DoD Advanced Distributed Learninghttp://www.adlnet.org

http://www.prometeus.org/

http://www.fdgroup.co.uk/gestalt





http://www.geminfo.org/







189

CEN/ISSS/LT:Learning Technology Workshop

• European Committee for Standardization (CEN), Information Society Standardization System (ISSS), Learning Technology (LT) Workshop

• Related activities: CEN/ISSS/MMI (multi-media metadata information)

• More info: http://www.cenorm.be/isss/workshop/lt








190

CEN/ISSS/LT:Learning Technology Workshop

• Topics:– Reusability and Interoperability, Metadata,

Taxonomies, Vocabularies, Bindings, Profiles, Internationalization, Structure of Learning Resources, Architectures, Rights Management, Data Protection and Privacy, Accreditation and Access


191

AICC: Aviation Industry CBT Committee

• Develops guidelines and recommendations for CBT in aviation applications

• Subject areas: Courseware Delivery Stations, Digital Audio, Digital Video, Operating Windowing System, CBT Peripheral Devices, Computer Managed Instruction (CMI), Courseware Interchange, Smart Graphics, Icon Standards

• More info: http://www.aicc.org



192

IMS Project

• Main areas of current/past development:– Metadata: learning technology metadata– Enterprise Interfaces: back office integration– Question/Test Interoperability: common testing– Profiles: learner records– Content objects: learning content, course structure– Content launch: starting/invoking content– Content packaging: bundling content– Management systems: collaboration, reporting– External services: interface to institution

• More info: http://www.imsproject.org



193

ARIADNE (European Union)

• Alliance of Remote Instructional Authoring and Distribution Networks for Europe

• Telematics for education and training• Tools and methodologies for producing, managing

and reusing computer-based pedagogical elements• Telematics-supported training curricula• Learning technology metadata• Working with IMS on metadata• More info: http://ariadne.unil.ch



194

PROMETEUS

• PROmoting Multimedia access to Education and Training in EUropean Society

• Topics:– Interchange, reusability, and portability of

multimedia education material– New way of corporate learning, using information

and communication technologies– Organizational and cooperative learning– Higher education best practices


195

PROMETEUS

• Topics (cont’d):– Web-based open and distance learning– Broadcasting-based learning– Primary and secondary school learning

environments based on information and communication technologies

– How to create self-sustainable new learning service and businesses

– Design of electronic learning environments– Interoperability of learning services


196

PROMETEUS:Further / Future Work Areas

• Multilinguality and multiculturablity• Intellectual property rights• Knowledge and skills assessment• Quality methods• Disabled, handicapped & elderly learning• ... any appropriate topic, through new SIG

creation procedure• More info: http://prometeus.org

http://prometeus.org/


197

GESTALT

• GESTALT: Getting Educational Systems Talking Across Leading-edge Technologies

• LOM and PAPI implementations• More info: http://www.fdgroup.co.uk/gestalt







198

GEM: Gateway to Education Materials

• Cataloging and Tools• More info: http://www.geminfo.org



199

ADL: DoD AdvancedDistributed Learning

• Research and development areas:– Scalable Content Object Reference Model (SCO-RM)– Promote widespread collaboration– Exploit Internet technologies– Develop next generation learning technologies– Create reusable content, and lower costs, with object-based

tools

• More info: http://www.adlnet.org



200

Layer 5: Operational Components

• Common components that build the system• Operational components may serve several functions• Several system components may be implemented via

the same operational components, e.g., performance info and learning content both coded as XML (an operational component)


201

Layer 5: Operational Components(cont’d)

• Operational components are identified by:– Systems, e.g., clients and servers– Connectors (interface protocols), e.g., HTTP, PPP– Busses (namespaces), e.g., URLs, telephone numbers

• Components (systems) on a bus can “talk” to each other via interface protocols


202

Layer 5: System Interoperability



Environment

Interactions

Layer 1Learner/


L

LL

LearnerEntity

Layer 3LTSA

SystemComponents

LCI

LP

BM

PP PA

D

LE

E

CLR R

IC

Q

LC L

Layer 5APIs,

Codings,& Protocols



203

Methodology: Work Flow And Progressive Deliverables

Requirements

Functionality

Conceptual Model

Semantics

Bindings: APIs Bindings: Codings Bindings: Protocols

Encodings:Data Formats

Encodings: CallingConventions

Encodings: VariousCommunication Layers

The Steps of Building SuccessfulInformation Technology Standards/Specifications

“The work flow/steps promote(1) consensus-building, and

(2) long-term stability, interpretation, maintenance of

the standard/specification.”

“Consensus-building is incremental.”

“Interpretation/maintenance is stabilized: each level is dependent on higher levels.”

“Interpretation Examples:- Ambiguities in bindings are resolved by interpreting the semantics;- Ambiguities in semantics are resolved by interpreting the conceptual model.”


204

Requirements: User/InstitutionNeeds and Desirables

• Informative, not part of standard => only conform to interoperability

• Examples:– Administration apps, e.g,

campus directory– Learning content access

to student info– Record grades

Requirements

Functionality

Conceptual Model

Semantics



Encodings:Calling Conventions

Encodings: Various Communication Layers


205

Requirements: Sources and Influences

Requirements

Functionality

Conceptual Model

Semantics





Requirements

Functionality

HL7, CORBAMED,TEPR, Z39.50

IMS, AICC, UK, CSU, UC, SIF, DoD/ATSC, NYS, UC, SPEEDE

Various VendorsAnd Products

Related Industries Industry-Specific Other Sources

“Requirements are gathered from industry-specific (application-related) organizations, related industries, and other sources.”

Requirements

“Requirements are only informative (improves reading the standard), not normative, i.e., not required by conforming implementations.”


206

Functionality: What It Does

• Stores, retrieves, searches learning content

• Partition into functional areas

• Examples:– Partition into personal,

preference, performance, portfolio info

– Get/put values/records

Requirements

Functionality

Conceptual Model

Semantics






207

Functionality: Sources and Influences

Requirements

Functionality

Conceptual Model

Semantics





Functionality

Conceptual Model

OMG, IETF, LDAP, VCard, DoD DMSO

AICC, IMS, ARIADNE,ISO-IEC/JTC1/SC35,

ISO-IEC/JTC1/SC22/WG11

Various Vendors, I18N, L10N, CAW

RequirementsRelated Industries

Industry-Specific Other Sources

“Functionality is based on requirements, with influences from industry-specific organizations, related industries, and other sources.”

Requirements Functionality


208

Conceptual Model: How A Virtual Implementation Works

• “Theory of Operation”• Examples:

– Application opens repository/mgmt system

– Authentication– Scan/search for data– Read/write data– Multiple open sessions– Close sessions

Requirements

Functionality

Conceptual Model

Semantics






209

Conceptual Model: Sources and Influences

Requirements

Functionality

Conceptual Model

Semantics





Conceptual Model

Semantics

Kerberos, FTP,SMTP, HTTP,

Sandia NL, X12

ISO-IEC/JTC1/SC2/WG2,ISO-IEC/JTC1/SC22/WG11,ISO-IEC/JTC1/SC22/WG14

ISO-IEC/JTC1/SC22/WG20,ISO-IEC/JTC1/SC22/WG21,ISO-IEC/JTC1/SC32/WG2

FunctionalityData Interchange Related Stds Related Stds

“The conceptual model of a standard based on its functionality, with influences from data interchange and service standards/specifications, and related standards/specifications.”

Functionality Conceptual Model


210

Semantics: Detailed Meaning Of Operations/Transactions

• Assertions: sentences using shall, should, or may

• Inquiries: range of values

• Negotiations: heuristics• Examples:

– Array: list of elements of same type in regular structure

– PutValue: replaces old object, in place

Requirements

Functionality

Conceptual Model

Semantics






211

Semantics: Sources and Influences

Requirements

Functionality

Conceptual Model

Semantics





Semantics

Bindings: APIs

C, C++, SQL, LISP,Java, Javascript,POSIX, Tcl, APL

FTP, HTTP, MIME,RFC 822, CORBA,

W3C P3P

ISO-IEC/JTC1/SC22/WG11,ISO-IEC/JTC1/SC32/WG2,NCITS L8, NCITS T2

Conceptual Model

Bindings: Codings

Bindings: Protocols

Language-Related Protocols Related Stds

“A standard’s semantics are based on its conceptual model, protocols, programming languages, related standards/specifications.”

Conceptual Model Semantics APIs, Codings, Protocols


212

APIs, Codings, Protocols —All Three Are Required [1/6]

• Semantics are “bound” to APIs, codings, and protocols• Designing for all three produces more coherent,

harmonized design• Different stakeholders have different foci:

– some want only APIs– some want only codings– some want only protocols– some want a combination

• APIs, codings, and protocols can “re-use”, “re-target”, or “re-apply” existing standards


213


• Motivation for not standardizing APIs:– Only interested in system-to-system interoperability,

i.e., focus is on codings or protocols• What happens if APIs are not standardized?

– Each application develops its own “libraries”– Code and application reusability is greatly decreased

because no common API or API framework– Much effort is spent “reinventing the wheel” by

building low-level subsystems that provide services normally supplied by APIs


214


• Motivation for not standardizing codings:– Programmers only use (standard) APIs– Codings are not necessary because databases are

proprietary– Too hard: vendor, user, institutional variants/extensions

• What happens if codings are not standardized?– Each application develops its own data model and

representation– Exchanging data among systems becomes impractical

because of data model incompatibilities– Data representation incompatibilities much data

“mediation” and conversion


215


• Motivation for not standardizing protocols:– Only (standard) APIs are necessary, protocols are

vendors’ “value-added” feature, thus proprietary– Only (standard) codings are necessary ... “just point me to

the data”, protocols are lower implementation details

• What happens if protocols are not standardized?– Each (proprietary) implementation develops its own

protocols to communicate across wide area networks– Proprietary solutions produce “islands” of interoperability– Wide area, multi-vendor solutions become impossible


216


Requirements

Functionality

Conceptual Model

Semantics





Semantics

Bindings: APIs

Bindings: Codings

Bindings: Protocols

- Std APIs may be implemented viastd or proprietary Protocols- Std Protocols may be accessedby std or proprietary APIs- Both std APIs/Protocols improvewide area interoperability

- Std APIs may use std orproprietary Codings- Std Codings may be usedby std or proprietary APIs- Both std APIs/Codingsimprove portable apps/data

- Std Protocols may use std orproprietary Codings- Std Codings may be exchangedvia std or proprietary Protocols- Both std Protocols/Codingsimprove system interoperability

Harmonized standard APIs, Codings,and Protocols promote:- Application portability- Data portability- Multi-vendor, “open” solutions- Wide area, end-to-end interoperability


217


• Common APIs:– Application portability– Separation of application and low-level implementation

• Common codings:– Representation of information– File/exchange formats, portable data

• Common protocols:– Necessary for multi-vendor and wide area networking– Internet relies on common protocols, not common APIs

• Common APIs, codings, protocols:– All are necessary


218

Codings vs. Encodings

• Codings: the structure and representation of information– Examples: XML, Tcl, tabbed, object-oriented, Java binding

• Encodings: the bit/byte representation and format– Examples: ASCII, ISO 10646, IEEE floating point, UNIX x86

calling conventions

• Both coding and encoding example:– “The name shall be coded in XML as

<NAME><FAMILY>…</FAMILY><GIVEN>…</GIVEN></NAME> and encoded in ASCII.”


219

Bindings: Mappings to OtherStandards Frameworks

• Bindings to APIs• Bindings to codings

(information organization)• Bindings to protocols• Examples:

– APIs: C/C++, VB, Perl, Java, Javascript

– Codings: XML, TCL– Protocols: HTTP, DCTP

Requirements

Functionality

Conceptual Model

Semantics






220

Encodings: Bit/Octet DataRepresentation and Format

• Not to be confused with “codings”

• Examples:– POSIX/Win32 subroutine

linkage– ASCII, ISO 8859-1, UTF8– IETF text format

Requirements

Functionality

Conceptual Model

Semantics






221

Bindings -> Encodings [1/6]

Requirements

Functionality

Conceptual Model

Semantics





Bindings: APIs


C/C++, VB, Perl,Java/Javascript,

LISP, Tcl

Sandia NL, Various Vendors and

Products

ISO-IEC/JTC1/SC22/WG11 LID, LIA, LIPC,OMG CORBA

Semantics

C/C++, JVM, RPC, RMI, RSH, RCMD, EDI

POSIX, ELF,Win32, INETD

Language-Related Other Related Stds

Language/Protocol-Related OS-Related

“Semantics, programming languages, related standards and other specifications influence APIs ... APIs are represented as calling conventions.”

Semantics APIs Calling Conventions


222

Bindings -> Encodings [2/6]APIs -> Calling Conventions

• APIs include ...– Programming language interfaces– Command line interfaces– Remote procedure calls

• Calling Conventions include ...– Subroutine/module linkage– Process environment– Name resolution, network transport


223


Requirements

Functionality

Conceptual Model

Semantics





Bindings: Codings


C/C++, LISP, Tcl,APL, tab-separated,

COM, CORBA

SGML, HTML, XML,XMI, ASN.1, CDIF, NCITS T2,T3,L8

ISO-IEC/JTC1/SC2, ISO-IEC/JTC1/SC32,ISO-IEC/JTC1/SC34

Semantics

ISO 8859-1, 10646,JIS X208, X209,Unicode, ASN.1

ISO-IEC/JTC1/SC2/WG2,ISO-IEC/JTC1/SC22/WG14,

ISO-IEC/JTC1/SC35,IETF

Related Stds/Specs Related Stds Stds Activity

Related Stds Stds Activity

“Semantics, information codings, and related standards/specifications influence codings ... codings are represented as data formats.”

Semantics Codings Data Formats


224

Bindings -> Encodings [4/6]Codings -> Data Formats

• Codings include ...– Record structures– File structures– Directory/folder structures

• Data Formats include ...– Character sets, character set extensions– Local, regional, remote pointers/linkage– Namespace, scoping, and syntax conventions


225


Requirements

Functionality

Conceptual Model

Semantics





Bindings: Protocols


CGI, AICC CMI,EDI, CORBAMED,DC, TELNET, FTP

HTTP, DCTP,CORBA, WebDAV,

IEEE 1596.5, CeBus

IETF, W3C, TIA, NCITS,ISO-IEC/JTC1/SC6,ISO-IEC/JTC1/SC25

Semantics

ISO OSI, TELNET,FTP, IEEE 1596.5

ASCII, ISO 8859-1, ISO 10646, UTF8

Related Stds/Specs Stds/Specs Stds Activity

Communication Features Octet Encodings

“Semantics, networking protocols, and related standards/specifications influence protocols ... protocols are represented as various communication layers.”

Semantics Protocols Communication Layers


226

Bindings -> Encodings [6/6]Protocols -> Communication Layers

• Protocols include ...– Application, presentation, session layers– Transaction sets– Network replication, publish/subscribe

• Communication Layers include ...– Presentation, session, transport, network layers– Distributed object/data/process frameworks– Connectivity/reliability frameworks


227

A Framework for Harmonized/Consistent ...Bindings: APIs, Codings, Protocols

Encodings: Calling Conventions, Data Formats, Communication Layers

Functionality

Conceptual Model

Semantics





Topic-SpecificInformative Wording

Topic-SpecificNormative Wording

Cross-TopicCodings: XML

Cross-Topic APIsInformative Wording

Cross-Topic APIs:Normative WordingJava, JavaScript,C/C++, Perl, Tcl, VB

Various Standards

Cross-Topic Protocolse.g.: Session Layers

Various Standards

Requirements


228

LTSC Cross-Working Group Bindings for IEEE 1484.X:APIs & Codings: IEEE 1484.14 Protocols: IEEE 1484.15, 1484.16

(IEEE 1484.X is: .2, .4, .6, .7, .8, .10., 11., 12., .17)

Functionality

Conceptual Model

Semantics





IEEE 1484.XInformative Wording

IEEE 1484.XNormative Wording

IEEE 1484.X,IEEE 1484.14 XML

IEEE 1484.14 SDAInformative Wording

IEEE 1484.X,IEEE 1484.14 SDANormative Wording

IEEE 1484.X, 1484.14,And Other Standards

IEEE 1484.15 DCTP, IEEE 1484.16 HTTP

Various Standards

Requirements


229

Summary of Five LTSAAbstraction-Implementation Layers



Environment

Interactions

Layer 1Learner/


L

LL

LearnerEntity

Layer 3LTSA

SystemComponents

LCI

LP

BM

PP PA

D

LE

E

CLR R

IC

Q

LC L

Layer 5APIs,

Codings,& Protocols



230

LTSA Layer Summary (cont’d)

• Layer 1:– Learning experience: learners interacting with the

environment– A learner entity (abstraction) may represent several learners

(humans)– From information technology perspective, flow is one-way:

environment ==> learner– Collaboration is within the learner abstraction


231


• Layer 2:– Because the learner abstraction is “implemented” with

“humans” (human learner), rather “implemented” with “computers” (machine learning), the “nature of humans” has pervasive effect on system design

– Fundamental limits of human communication


232


• Layer 2 (cont’d):– Humans aren’t reliable receivers of information:

retransmission– Humans aren’t predictable receivers of information: different

strategies– Humans learn differently (and over time): customizable,

configurable


233


• Layer 3:– System notation: process, store, flow– Human-centered feedback loop: learner, behavior,

evaluation, assessment, coach, delivery, multimedia– More “intelligence” from learning content: need learner

records and performance records– Learner has different “teachers” during his/her lifetime:

learner records facilitates “hand-off”


234


• Layer 3 (cont’d):– Humans each learn differently (and learn differently over

time): rich learning resources required; queries and catalog info (metadata) make best use of learning resources

– Learners and their surrogates (e.g, parent, mentor) negotiate learning preferences with coaches and their surrogates (e.g., institution, teacher, courseware developer)


235


• Layer 4:– Stakeholders each have own, incomplete perspectives– All stakeholders’ perspectives are incorporated in LTSA

system components diagram– IEEE 1484: working groups specifying critical interfaces;

some technical standards outside of 1484; “education standards” are necessary but outside scope of 1484 and “technical standards”


236


• Layer 5:– Bus notation: bus, connector, system– Operational components: actual implementation– One operational component (e.g., HTTP) might be used by

several system components (e.g., learner records, learning resources)

– One system component (e.g., delivery) might use several operational components (e.g., HTTP, HTML, MPEG)


237


• Layer 5 (cont’d):– Bus may be used for several purposes– Systems on bus share common namespace, e.g., URLs,

process IDs, object IDs– Systems may be connected, simultaneously, to several

busses


238


• Layer 5 (cont’d):– Systems dynamically connect, on-demand to other systems

(e.g., clients to servers)– System connectivity is not pre-arranged– Connections (protocols) may serve several purposes (e.g.,

XML used for performance records and learning content; MIDI used for behavior and multimedia)


239

LTSA Conformance

• LTSA conforming products/services– provide the critical interfaces (functionality) that conform to

one or more components of LTSA: Learner (Collaboration), Behavior, Evaluation, Performance, Learner Records, Coach, Query (search), Catalog Info (metadata), Locator (e.g., URLs), Learning Resources, Delivery, Multimedia

– identify the protocols that describe the bus interface– identify the interface point


240

LTSA Conformance

• LTSA conforming products/services• Measuring conformance: the system under test is

called the “implementation under test”• A system that conforms to the LTSA specification is

called a “conforming implementation”• “A conforming implementation shall conform to

all five layers.”


241

LTSA Layer 1 Conformance

• Validating Learner-Environment interactions• “The implementation under test shall identify the

boundaries of the Environment, the flows of Interactions, and the Learner Entity.”

• “The implementation under test shall identify the Learner Entity's relationship to human Learners.”

• Examples of Learner Entity/Learner relationship:– Is the Learner Entity represented by more than one

Learner?– Is there collaboration among the Learner?


242


• Validating human-centered/pervasive features based upon human strengths and weaknesses

• “The implementation under test shall identify the methods for addressing the human-centered/ pervasive features:”– self-directed learning– unreliable learners– unpredictable and diverse learners– nomadic learners– learner’s influence on learning preferences


243


• Validating decomposition into LTSA system components

• “The implementation under test shall identify the available LTSA system components: Learner Entity, Learning Preferences, Behavior, Evaluation, Performance and Preference Info, Learner Records, Assessment, Coach, Query, Catalog Info, Locator, Learning Resources, Learning Content, Delivery, Multimedia, Interaction Context.”


244


• Identifying which LTSA subset (stakeholder) is applicable — stakeholders can be any subset, not limited to the examples in the LTSA specification

• Mapping LTSA system components to actual implementation components

• “The implementation under test shall identify the subset of applicable LTSA system components.”

• “The implementation under test shall identify the boundaries and mapping of LTSA system components.”


245


• Identifying the information busses — bus protocols and topology not limited to examples in LTSA specification

• “The implementation under test shall identify the information busses in use, their namespace, control protocols, and mapping to LTSA system components.”


246

Open Issues

• Verifying/validating conformance clauses• Conversion to IEEE style guide• Ongoing verification/validation of existing/new

frameworks• Track comments/issues from related committees,

such as: ADL, AICC, ARIADNE, CEN/ISSS/LT, GEM, GESTALT, IMS, PROMETEUS

• Write Rationale annex


247

Summary [1/2]

• Yourdon (systems notation): tried and true• Bus notation used for larger number of component• Five abstraction-implementation layers:

– Learner and environment interactions– Human-centered/pervasive features– System components– Stakeholder perspectives– Operational components


248

Summary [2/2]

• Wide range of implementations supported• All identified stakeholders’ perspectives mapped to

LTSA• Reflects commercial systems

1999-12-06ieee 1484.1, ltsa presentation, f. farance, ©1999 edutool.com 1 learning technology...

Documents

ltsa layers

presentation frank farance

ieee style slide

layers layers

comltsa presentation

division of farance

blue slide

neutral slide