bridging the gap between practitioners and e-learning standards: a domain-specific modeling approach...

Bridging the Gap between Practitioners and E-learning Standards:

A Domain-specific Modeling Approach

Yongwu Miao, Tim Sodhi, Francis Brouns, Peter Sloep, and Rob Koper

Centre for Learning Sciences and TechnologiesOpen University of The Netherlands

ECTEL08, Maastricht, Sep. 16-19, 2008

Overview

• Problem and Goal• Domain-specific Modelling• Peer Assessment• Peer Assessment Modelling Language• Peer Assessment Modelling Tool• Summary and Future Work

Problem and Goal

• Developing a complicated Unit of Learning (UoL) using IMS LD constructors (e.g., roles, learning activities, properties, and conditions) is difficult and even impossible for ordinary practitioners, because the required level of pedagogical knowledge and technical knowledge is significant.

• Enabling ordinary practitioners to create pedagogically sound and technically executable Uols by providing pedagogy-specific modeling languages

Domain-specific Modeling

• What is Domain-Specific Modeling (DSM) or Domain-specific language (DSL) DSM is a kind of model-driven approach to develop software applications. It raises the level of abstraction beyond programming by specifying the solution in terms of concepts and associated rules culled from the very domain of the problem being solved. The final software products are generated from these high-level specifications.

• Why to Use DSMDSM are more expressive than and therefore tackle complexity better, making modeling easier and more convenient. More importantly, they allow automatic, full code generation, similar to the way today's compilers generate Assembler from a programming language like JAVA.

What are differences between DMS and other approaches• different users• different languages• different abstraction levels


LD & QTI modelMap to LD & QTI interpret


How to adopt DSM approach to develop software• Developing a meta-model• Develop executable components• Developing mapping functions to transform domain model to

components

Peer Assessment

Definitions:• Peer assessment is a process whereby groups rate their peers

(Falchikov 2001) • Peer assessment is an arrangement for peers to consider the level,

value, worth, quality or successfulness of the products or outcomes of learning of others of similar status (Topping, Smith, Swanson, & Elliot, 2000).

Peer assessment models:• Pure peer assessment (two learners do assignment and then assess the

outcome of his/her peers)• With additional stages (design assessment and react to feedback)• Group/collaborative assessment (more candidates and reviewers)• With involvement of the tutor• Different types of assignment tasks• Different distribution patterns• ……

design assignment

writing article1

reviewing article2

improving article1

writing article2

reviewing article1

improving article2

final assessment2final assessment1

learner1 learner2tutor

article2

feedback1

improved a2

paper1

article1

feedback2

improved a1

paper2

Peer Assessment

A peer assessment example

tutor’s feedback1 tutor’s feedback2

assessment form1 assessment form2

Peer Assessment

A typology of peer assessment (Topping 1998)

Peer Assessment Modelling Language

• Four kinds of stages • In each stage, people with a certain role perform activities• In each activity, there may be input and output documents

Peer Assessment Modeling Language

• Guide to make design decisions by showing guidance and consequence• Make decisions by selecting default or pre-defined options

Peer Assessment Authoring Tool

• A peer assessment authoring tool will be based on the peer assessment modeling language (meta-model).

• Users will be guided to specify a peer assessment model by making a series of design decisions.

• The consequences of the decision-making will be presented as a diagram.

• The tool is underdevelopment.• The design and use of the tool will be explained (in a

mock-up UI) by using an example

File Edit View Tool Template Help

docrole service lanestage

Peer assessment

Here shows the guidance to make design decisions

State bar

user’s choices (decision options)

Specify role structure, document, and service

Show and edit peer assessment model



Peer assessment

How many stages in the peer assessment process?

Please make a decision

user’s decisionsDefault (three stages)With design assessment stageWith additional feedback stageMore complicated setting



Peer assessment

How many roles will be involved?

Five stages have been defined

design assessment do assignment give feedback react to feedback give feedback 2

Default (two persons)With the involvement of tutorMore complicated setting


docrole service lanestagedesign assessment do assignment give feedback react to feedback

Peer assessment

give feedback 2

Would you like to define role structure?

reviewerR

designerR

candidateR

Default role setting and with the involvement of a tutor

learner1R

tutorR

learner2R

learner1R

learner2R

tutorR



Peer assessment

give feedback 2

Would you like to define a typical activity for each stage ?

reviewerR

designerR

candidateR


learner1R

tutorR

learner2R

learner1R

learner2R

tutorR

Default settingMore complicated setting



Peer assessment

give feedback 2

Would you like to define a typical activity for each stage ?

reviewerR

designerR

candidateR

Nine activities are defined and roles are assigned

learner1R

tutorR

learner2R

learner1R

learner2R

tutorR

learner1 learner1 learner1

learner 2 learner 2 learner 2

learner1

learner2

tutor



Peer assessment

give feedback 2

Which stages will the tutor be involved?

reviewerR

designerR

candidateR


learner1R

tutorR

learner2R

learner1R

learner2R

tutorR



learner1

learner2

Give feedback 1Give feedback 2More complicated setting

tutor



Peer assessment

give feedback 2

Which stages will the tutor be involved?

reviewerR

designerR

candidateR

The tutor is assigned to work in the stage of “give feedback 2”

learner1R

tutorR

learner2R

learner1R

learner2R

tutorR



tutor

tutor

tutor



Peer assessment

give feedback 2

Would you like to create documents ?

reviewerR

designerR

candidateR


learner1R

tutorR

learner2R

learner1R

learner2R

tutorR




tutor

tutor

tutor



Peer assessment

give feedback 2

Would you like to create typical documents ?

document are created



tutor

tutor

tutor

feedback1.1D

feedback1.2D

outcome1D

outcome2D

assignment1D

assignment2D

improved outcome1D

improved outcome2D

feedback2.1D

feedback2.2D



Peer assessment

give feedback 2

Could you please change the titles of documents and define documents?




tutor

tutor

tutor

feedback1.1D

feedback1.2D

outcome1D

outcome2D

assignment1D

assignment2D

improved outcome1D

improved outcome2D

feedback2.1D

feedback2.2D

finished



Peer assessment

give feedback 2

Could you define document?




tutor

tutor

tutor

feedback1D

feedback2D

article1D

article2D

paper1D

paper2D

improved article1D

improved article2D

tutor’s feedback1D


finished

learner 2 learner 2

Review Form

How do you think about this article?

accept

accept with minor/major modification?

reject

Comments:

assessment form

assessment editor

assessment formD



Peer assessment

give feedback 2

Could you define document?




tutor

tutor

tutor

feedback1D

feedback2D

article1D

article2D

paper1D

paper2D

improved article1D

improved article2D



finished

assessment formD



Peer assessment

give feedback 2

How will the documents be distributed?




tutor

tutor

tutor

feedback1D

feedback2D

article1D

article2D

paper1D

paper2D

improved article1D

improved article2D



assessment formD




Peer assessment

give feedback 2


documents are distributed by default



tutor

tutor

tutor

feedback1D

feedback2D

article1D

article2D

paper1D

paper2D

improved article1D

improved article2D



assessment formD



Peer assessment

give feedback 2


doc-flow are customized



tutor

tutor

tutor

feedback1D

feedback2D

article1D

article2D

paper1D

paper2D

improved article1D

improved article2D



assessment formD

Summary and Future Work

Summary:• Outline an approach to apply the domain-specific

modelling paradigm to the task of the development of peer assessment

• Develop a peer assessment modeling language• Propose a method to enable the end-user to model peer

assessment processes through making a series of design decisions

Future work:• Develop executable components• Design and implement mapping algorithm• Implement the tool and conduct evaluation

Thanks for your attention!

Questions?

bridging the gap between practitioners and e-learning standards: a domain-specific modeling approach...

Documents