FREMA: e-Learning Framework Reference Model for Assessment

David Millard Yvonne Howard

Learning Technology GroupUniversity of Southampton, UK

Structure

• Overview– Introduction to FREMA– What is a Reference Model?– Views on the Model

• Do we have the right Personas – are you there?• Do our Concept Maps make sense to you?

• Workshop: Eliciting Common Usage Patterns– What are the most common tasks within assessment?– What is (and is not) being addressed by current software?– What do you consider to be the core elements of the

assessment domain?

Introduction• What is FREMA?

– JISC funded Project between Southampton, Strathclyde and Hull– Part of the e-Learning Framework (ELF) effort

• What is the ELF?– Service Oriented Architecture for e-learning systems– Layered Web Services (Domain services over Common Services)– Dynamic and evolving

• FREMA will develop a Reference Model for the Assessment domain– Supporting design-time activities– Supporting run-time activities– Virtual organisations and lifelong learning

• What is a Reference Model?– A description of how services behave within a particular domain– A community resource

Assessment and ELF

• Not enough to describe and define these services• Need a proper audit trail of decision making• Start by defining the domain • Work up through the services to a reference

implementation• This is an ELF Reference Model

Anatomy of a Reference Model• Domain Definition

– Overview of the domain, and how projects and standards fit within it

Assessment Domain Definition

Use CasesUse CasesUse Cases

Service Profiles

Gap Analysis

Reference Impl’

• Identifying Common Usage Patterns– Scoping the FREMA Project

• Gap Analysis– Mapping of Use Cases to the Services in ELF

• Service Profiles– Formal descriptions of those services

• Reference Implementation– Of key/core services– Examples– Validation– Resource

Common Usage Patterns

• Developing Use Cases– Formal descriptions of usage patterns

What does it look like?

• An evolving, cross-referenced, searchable web site

• Indexed resources and narrative descriptions of the domain• UML Use Cases and Scenario documents• Service descriptions, narrative and WSDL• Service implementations to download (Java/.NET)

• Different gateways into the model according to how you want to use it

Assessment Domain Definition

Use CasesUse CasesUse Cases

Service Profiles

Gap Analysis

Reference Impl’

Common Usage Patterns

How might you use it?

• Use the Reference Implementation– Build on some or all of the developed services

• Use the Service Profiles– To develop your own services that will fit into the framework

• Use the Use Cases– To help understand usage patterns within the domain– Develop new Service Profiles and thus Services

• Use the Domain Definition– To develop a context for your own work– Understand how existing work fits together– Identify standards– Locate experts

Assessment Domain Definition

Use CasesUse CasesUse Cases

Service Profiles

Gap Analysis

Reference Impl’

Common Usage Patterns

Use Case for a Service Oriented Reference Model (SORM)

A More Concrete Description

• An ontology for the resources of the domain

• A knowledge base of the resources

• Concept Maps that define the domain and aid searching – Processes (think of verbs)– Classes of Objects (things, think of nouns)– And many more gateways into resources

• Visualisations of – Concepts– Populations of resources over concepts

• Searchable, flexible, dynamic website

Process concept

map (verbs)

Classesconcept

map(nouns)

Another look at the SORM Use Case

How would these actors choose to view the domain?

Personifying the actors

• Will, Web services developer– Scenario:

‘I want to lookup use cases and scenarios to help me design my application. This will help me to define my footprint in the assessment domain. I see there are some web services I could download but some are missing. What standards can I use when writing my own web services to ensure that I can interoperate with the web services I’ve chosen?’

• Yvonne, Institutional Resource Manager– Scenario

‘I want an overview of what this domain is all about. I want to know what standards are applicable in the domain to ensure that we comply with quality assurance requirements. I want to examine use cases and scenarios to understand the available footprints. I also want to know who the key players are and what the key projects are.’

• Other Actors– Early Adopter– Toolkit developer– Course developer– And there are others . . .

Concept modelling

• What mental models might these and other actors use to orient their searches?

• How do they visualise the assessment domain?

Structure

• Overview– Introduction to FREMA– What is a Reference Model?– Discussion

• Do we have the right Personas?• Do our Concept Maps make sense?

• Workshop: Eliciting Common Usage Patterns– What are the most common tasks within assessment?– What is (and is not) being addressed by current software?– What do you consider to be the core elements of the

assessment domain?

Another concept map – based on a domain expert view

Items

Validation Final Grade

Structuring

Assessment

Marking

Grade book

Multiplicity Self assess

Mark

Learning

Peer Group

Student FeedbackEvidenceTracking

Artefacts

Course

Group

Member

Tra

nscrip

t

Schedule

Delivery

Plagiarism

Authorise

Authenticate

Group

Student Appeal

Help

Authoring

QA

Workflow

Failover