an ontology-based architecture for knowledge base design ... filean ontology-based architecture for...

An ontology-based architecture for the design of knowledge bases in Intelligent Instructional Systems

Helmut Meisel Ernesto Compatangelo

Department of Computing Science, University of Aberdeen AB24 3UE Scotland, UK

{hmeisel, ecompata}@csd.abdn.ac.uk

Abstract This paper describes an architecture for the usage of Instructional Design (ID) knowledge in intelligent instructional systems. In contrast with other architectures, ontologies are used to represent ID knowledge about both what to teach and how to teach. Moreover, set-theoretic reasoning is used for the provision of inferential services. In particular, the paper shows how set-theoretic deductions can be applied (i) to support the modelling of ID knowledge bases, (ii) to retrieve suitable teaching methods from them, and (iii) to detect errors in a training design. The intelligent knowledge management environment CONCEPTOOL is used to demonstrate the benefits of the proposed architecture.

Keywords: Ontologies, Instructional Design, Automated Reasoning, Intelligent Tutoring Systems

1 Introduction

Instructional Design (ID) is part of Instructional Science, which encompasses theories, models, methodologies, and tools for instruction (Mizoguchi and Bourdeau, 2000). ID is “an engineering activity for which the artefact is some instructional product conceived to help a learner acquire some knowledge or skill” (Merrill, 2001). This activity applies strategies and techniques derived from behavioural, cognitive, and constructivist theories to the solution of instructional problems (Mizoguchi and Bourdeau, 2000).

Instructional Design theories are prescriptions for designing instructional products to optimise the learning outcome (Merrill, 2001; Mukhopadhyay and Madhu, 2001). These theories describe methods of instruction together with situations in which those methods should be used (Reigeluth, 1999). From a pragmatic viewpoint, the underlying questions about instructional design are (i) what to teach and (ii) how to teach (Dick etal, 2000). Research in ID has constantly focused on the dualism between teaching theory and practice in order to develop effective, efficient and appealing products for instruction.

Computing science has investigated ways of achieving this goal using software tools. Implemented computer-based tools range from intelligent tutoring systems to web based e-learning platforms (Gaudioso and Boticario, 2003). A common feature of all these systems is that they either implicitly or explicitly use knowledge about Instructional Design. However, incorporating knowledge in a computer program is a complex and labour-intensive task: firstly, ID knowledge must be acquired; secondly, procedures to use this knowledge must be implemented. Rather than building new

of 20

systems from scratch, the ID knowledge should be represented in such a way that it can be reused by different applications (Mizoguchi and Bourdeau, 2000).

This paper advocates the declarative knowledge bases to represent instructional design knowledge about both what to teach and how to teach. The authors propose an ontology-based architecture for the usage of these knowledge bases in educational technology systems. Ontologies are taxonomies that provide explicit specifications of conceptualizations (Gruber, 1993), promoting a shared understanding of a domain and encouraging collaborative development. Taxonomies or ontologies are not new to ID. Prominent examples are (i) Bloom’s taxonomy of knowledge (Bloom, 1956) and (ii) the taxonomy of learning outcomes by Gagne (Gagne, 1985). However, providing the conceptualisation of a domain is not the only benefit of ontologies. In fact, they should be also considered as the source of the intelligent behaviour in educational systems (Mizoguchi and Bourdeau, 2000). Unfortunately, an analysis of existing systems reported in (Bourdeau and Mizoguchi, 2002) concludes that only few intelligent educational systems had an explicit representation of knowledge about instructional theories and principles. None of the surveyed systems could either (i) “retrieve appropriate theories for selecting instructional methods” or (ii) “provide principles for structuring a learning environment” (Bourdeau and Mizoguchi, 2002).

In order to fill this gap, the authors show how set-theoretic deductions can be used (i) to support the modelling of ID ontologies, (ii) to retrieve suitable teaching methods from these ontologies, and (iii) to detect errors in a training design based on them. Together with the usage of ontologies, these specialised inferential services define an architectural framework for representing ID knowledge and for reasoning with it. The underlying principles of this architecture are based on the following two paradigms:

• Knowledge in an intelligent system should be represented by domain experts without the need to involve a knowledge engineer.

• The source of intelligent behaviour should be “standard” automated reasoning based on knowledge represented by domain experts. Here, “standard” reasoning means situation-independent or general-purpose reasoning.

Accordingly, ID ontologies should be represented at the conceptual level, which is the (only) one ID experts are generally familiar with. In this way, experts can directly maintain the system knowledge without the involvement of a knowledge engineer. The underlying ontology representation should be expressive enough to encode instructional design knowledge. Moreover, this representation should be based on a formal knowledge model that can be interpreted by a computer reasoner.

The rest of the paper is structured as follows. Section 2 explains the motivations for this research and reviews relevant work so far. Section 3 defines knowledge representation and reasoning issues in the proposed architecture. This section also shows how the intelligent knowledge management environment ConcepTool implements a core part of this architecture. Finally, section 4 summarises some major issues in the proposed architecture.

of 20

2 Background and Motivations While procedural knowledge is widely used in intelligent instructional systems, this is not true for declarative knowledge (such as the one contained in domain ontologies). Declarative knowledge can represent instructional design theories about courses, learning goals, and teaching methods. To let this knowledge become the driver of intelligent behaviour, an intelligent instructional system needs to include automated reasoning capabilities. To achieve maximum reuse, “standard” automated reasoning should be adopted, rather than implementing a special-purpose algorithms for one single use case only. This section reviews research work performed so far from the viewpoint of ID knowledge representation and reasoning. Existing approaches are compared with the framework that underpins the architecture introduced in this paper.

2.1 Declarative knowledge for instructional design In artificial intelligence, declarative knowledge (also called domain knowledge) is explicitly distinguished from procedural knowledge. While the latter is about how to perform some task, declarative knowledge describes existing (or relevant) entities in a Universe of Discourse. Declarative knowledge about instructional design includes:

• Different categories of concepts, such as entities, rules, processes.

• Concepts that describe objects and their properties in the ID domain, such as Teaching Method, Learning Goal, Learner, Instructor.

• Relations between concepts, such as hierarchical or partonomic relations.

Most research on the representation of instructional design knowledge has focused on the procedural aspects of this knowledge (Bourdeau and Mizoguchi, 2002). However, the additional usage of declarative knowledge bases can further enhance intelligent instructional systems. This type of knowledge can be identified in various areas of instructional design. In fact, while knowledge about the activities performed during the ID process (i.e. the knowledge about the sequence of steps to take) is of procedural nature, knowledge about what to teach and how to teach can be (at least partially) captured in declarative knowledge bases. The following paragraphs examine where declarative knowledge bases can be used in ID.

Determining what to teach means identifying the necessary knowledge components to acquire the desired knowledge or skills (Merrill, 2001). Knowledge about how to perform this activity is procedural knowledge. Conversely, the outcome of this process is declarative knowledge about concepts such as courses, their content and their learning goals. This outcome can be stored in a declarative knowledge base for subsequent (re)use. Automated reasoning can be then applied (i) to support the organisation of these knowledge bases, (ii) to enhance knowledge retrieval, and (iii) to decide what should be taught to a specific user.

Determining how to teach means (i) choosing the right instructional methods and (ii) applying them properly. Knowledge about how to apply a teaching method requires problem-solving expertise (Mizoguchi and Bourdeau, 2000); hence, it is procedural knowledge. Conversely, knowledge about which teaching method to choose can be represented as declarative knowledge. Teaching methods, together with their applicability conditions, can be stored in a knowledge base that relates instructional theories with concrete teaching methods. Automated reasoning can be used (i) to

of 20

support the organisation of the knowledge base, (ii) to retrieve teaching methods, and (iii) to validate the choice of these teaching methods.

In conclusion, the usage of declarative knowledge bases in instructional design yields the following benefits:

1. Declarative knowledge bases (e.g. a repository of Teaching Methods) provide a human-readable conceptualisation of the educational domain. Storing information such as teaching methods, instructional theories, and the content of finished instructional products, such conceptualisations also educate novices in ID.

2. Declarative knowledge bases provide the core of ID advisory systems. By way of automated reasoning, such systems can promote the practical application of ID theories, assisting in the design of a training. More concretely, they could:

• Assist a designer in the selection of appropriate teaching methods.

• Encourage the application of a wide range of available teaching methods.

• Instruct a designer about the application of particular teaching methods.

• Highlight errors in the design of a training or course.

3. Declarative knowledge bases provide the core of an adaptive delivery system. Using automated reasoning, such a system can choose instructional strategies or learning objects that are most suitable in a given situation.

However, the kind of benefits obtained using declarative knowledge bases explicitly depend on the reasoning capabilities of the system that uses these knowledge bases. This paper aims to define an architecture that supports specialised deductions based on set-theoretic reasoning about declarative ID knowledge bases.

2.2 Knowledge representation and reasoning in current ID systems

Early Intelligent Tutoring Systems (ITSs) focused on the representation of their taught content (what to teach), whereas more recently the explicit representation of how to teach has attracted more attention (Murray, 1996). Therefore, modern ITSs allow the authoring of instructional strategies (Murray, 1999). These strategies describe the behaviour of a system, specifying when to provide a hint, how many hints to provide, or what to do once a learner has completed a task. Examples of such systems are the ID Expert (Merrill and ID2 Research Group, 1998), EON (Murray, 1998), or the Generic Tutoring Environment (Marcke, 1998). The latter provides an instructional knowledge base that stores (i) instructional tasks, (ii) instructional methods, and (iii) instructional objects. Instructional tasks describe what needs to be done (e.g. Show hint, Teach topic, Clarify concept), while instructional methods describe how to do these tasks (e.g. Clarify concept by example, Clarify concept with definition).

Rather than with the declarative representation of procedural knowledge, the architectural framework introduced in this paper is concerned with the representation of ID knowledge at a different level of granularity, which captures:

• the existing teaching methods;

• the abstractions that can be used to classify these teaching methods;

• the circumstances in which a certain teaching method should be chosen.

A rule-based system approach for the partial automation of course design is based on the Instructional Material Description Language – IMDL – (Gaede and Stoyan, 2001).

of 20

Instructional elements (i.e. learners or learning objectives) are considered as rule pre-conditions, while didactic elements (i.e. software components for the display of courses) are considered as post-conditions. An inferential engine uses these rules to generate alternative courses. However, a rule-based knowledge representation is problematic to create or to maintain, as relationships between related knowledge parts (i.e. rules referring to the same entity) are not explicitly stated. More concretely, the representation of heuristic knowledge with rules can be a source of problems in educational systems. The relevant problems identified by (Mizoguchi and Bourdeau, 2000) include:

1. The “conceptual gap” between authoring systems and authors. In Artificial intelligence, the “knowledge level” is explicitly distinguished from the “symbol level” in which knowledge is encoded (Newell, 1982). The traditional approach to the development of rule-based systems involves a knowledge engineer, who encodes the knowledge elicited from a domain expert. Unfortunately, this results in a gap between the conceptualisation of the ID expert and the corresponding computer representation. Consequently, (i) the development, (ii) the verification & validation, and (iii) the maintenance of rule bases can become rather difficult.

2. The lack of theory awareness of systems. Heuristic rules cannot explicitly represent the theories they commit to.

3. The difficulty to integrate the latest research results. The lack of theory awareness prevents the adaptation of rule bases in order to accommodate subsequent results of ID research.

Ontological engineering has been used to solve the above problems (Bourdeau and Mizoguchi, 2002): In this project, ID theories have been described in an ontology that captures top-level decisions about instructional strategies. Additionally, a domain ontology has been created which consists of (abstract) “theories” and (concrete) “worlds” of learning, instruction, and instructional design (Bourdeau and Mizoguchi, 2002). However, this approach does not introduce any methods or tools for reasoning about these ontologies. Therefore, the architecture proposed in this paper is complementary to the above design-oriented approach.

A further approach to the design of educational systems is the Educational Modelling Language – EML – (Koper, 2001). This language provides a framework for the conceptual modelling of learning environments. It consists of four extendable top-level ontologies, which describe (i) theories about learning and instruction, (ii) units of study, (iii) domains, and (iv) how learners learn. As previously stated, the architecture proposed in this paper can be used to provide automated reasoning support for EML ontologies.

Ontologies are frequently used in e-learning platforms (i) to organise their content and (ii) to facilitate its retrieval. The Learning Object Metadata – LOM – standard (IEEE - LOM, 2003) has been introduced to enable sharing and reuse of learning objects across platforms. However, this standard only offers a limited number of attributes to describe pedagogical aspects of a learning object (such as the interactivity type, the kind of learner it is suitable for, e.t.c.). Therefore, the LOM standard is not suitable to fully capture pedagogical knowledge (Pawlowski, 2002; Recker and Wiley, 2001; Allert etal 2002b). However, learning objects described using the LOM standard can be integrated in more expressive ID ontologies by considering LOM Metadata classes as ontology concepts. This approach is pursued by the Open Learning Repository

of 20

project – OLR – (Allert etal 2002a). OLR separates a domain model from an instructional model in order to use different teaching strategies. However, OLR ontologies are represented in RDF/RDFS (W3C – RDF, 2003), which only provides basic modelling primitives to capture ontologies (Pan and Horrocks, 2001). Conversely, the architecture proposed in this paper focuses on capturing ID knowledge in a more expressive way together with set-theoretic reasoning.

The L3 project (Leidig, 2001) aims at developing a scalable platform for web-based distance learning. Using an ontology, a course author can specify (i) what kind of knowledge (s)he wants to incorporate, (ii) the addressed skills, (iii) how knowledge can be transferred, and (iv) the relations to other learning objects. However, while the L3 ontology organises the content of the L3 platform, it does not contain knowledge about instructional theories. Conversely, the architecture proposed in this paper aims to support knowledge representation and reasoning about both what to teach and how to teach.

The ontology in L3 is represented using conceptual graphs (Sowa, 1999), which are a syntactic variant of first order logic. Therefore, reasoning problems such as computing the validity of a conceptual graph or computing the subsumption between two conceptual graphs are undecidable. In other words, algorithms for the above two problems have been theoretically proved to enter infinite loops (Baader etal 2003). This means that a system built using conceptual graphs as its knowledge representation formalism cannot be equipped with the kind of automated reasoning services envisaged in the architecture proposed in the following section.

3 An architecture for the usage of instructional design knowledge

The authors’ architecture is based on ontologies for the representation of ID knowledge and on set-theoretic reasoning for the provision of inferential services. Instructional design experts should be able to maintain these ontologies without the support of a knowledge engineer. In order to let them become the source of intelligent behaviour, a reasoner should directly perform inferences on these ontologies. Such a reasoner requires a conceptual representation language for ID ontologies based on a formal semantics. In this way, the ontologies can be directly interpreted by a reasoner without the need to reformulate them in a computer-readable formalism. Automated reasoning can be used (i) to support the modelling of ID ontologies, (ii) to provide enhanced querying on these ontologies, and (iii) to detect errors in a training design. Accordingly, this section describes the knowledge representation and reasoning techniques that are at the core of the architecture proposed in this paper. It also shows how the intelligent knowledge management environment CONCEPTOOL (Compatangelo and Meisel, 2003) can be used in the context of ID knowledge representation and reasoning.

3.1 The representation of instructional design knowledge Ontologies, which are formal, explicit specifications of shared conceptualisations (Duineveld etal 2000), encourage collaborative development by different experts. Ontologies capture knowledge at the conceptual level, thus enabling ID experts to directly manipulate them without the involvement of a knowledge engineer. In its simplest form, an ontology is a taxonomy of terms (i.e. a “shared lexicon”) whereas more expressive approaches such as the Ontology Web Language OWL (W3C – OWL, 2003) encode knowledge in logical axioms. Ontologies support knowledge reuse by allowing more specific concepts to inherit the properties of those concepts

of 20

they specialise. This also allows the representation of knowledge at different abstraction levels. In this way, instructional theories at a high level of abstraction can be related to concrete teaching methods.

The ontology shown in Figure 1 represents the rule “Drill and Practice Teaching Methods are suitable for the memorisation of routine procedures or factual knowledge”. This rule can be represented in an ontology by defining an attribute Learning Goal of the Drill and Practice Teaching Method that is either of type Memorise Routine Procedure or of type Memorise Factual Knowledge. The ontology in Figure 1 also contains the TEACHING METHOD instance Welding of 20 Identical Vertical Cordons, whose associated Learning Goal is Learn to Weld. The latter is an instance of Memorise Routine Procedure. Considering both instantiation and superclass/subclass links, the teaching method instance Welding of 20 Identical Vertical Cordons can be shown to commit to the above rule, as it is applied to a Learning Goal that is of type Memorise Routine Procedure.

instance-of

Learning Goal

instance-ofinstance-of

Learning Goal

Learning Goal

Learning Goal<<concept>>Drill and Practice Teaching Method

<<concept>>Memorize Routine Procedure

<<concept>>Teaching Method

<<concept>>Learning Goal

<<concept>>Repetative Exercise

<<concept>>Memorize Factual Knowledge

<<concept>>Exercise

<<individual>>Welding of 20 identical vertical cordons

<<concept>>TM involving Individual Learning

<<individual>>Learn to weld

Figure 1: Teaching invariant tasks

Ontological knowledge is represented at two different levels: while the intensional level (called T-Box) describes Universe of Discourse (UoD) classes as sets of individuals (e.g. the set of all Drill and Practice Teaching Methods), the extensional level (called A-Box) describes UoD individuals (e.g. Welding of 20 Identical Vertical Cordons). The intensional level represents a conceptualisation of the UoD and imposes constraints on domain individuals. Hence, general rules and assertions about instructional design should be represented as concepts, while particular Teaching Methods (e.g. Welding of 20 Identical Vertical Cordons) should be represented as individuals.

Ontologies allow the definition of multiple views on concepts or on individuals. For instance, a Teaching Method can be classified either according to social dimensions (e.g. Teaching Methods for Collaborative Learning, Teaching Methods for Individual Learning) or according to the type of learning (e.g. Teaching Methods for Discovery Learning, Teaching Methods for Problem-based Learning).

of 20

As previously stated, two fundamental issues in the representation of instructional design are what to teach and how to teach. The rest of this section thus provides examples of how this kind of knowledge can be captured. Different approaches to instructional design trigger a series of issues, including which theory to use or whether to unify different theories (Bourdeau and Mizoguchi, 2002). Therefore, it seems unlikely that one single standard instructional design ontology can be suitable to express all possible scenarios. However, this is not a necessary requirement for the architectural framework proposed in this paper, as different ontologies can be introduced to achieve this goal. The ontologies presented in this paper should not be considered as a proposal for a shared ontology, but only as a way of demonstrating the potential of the proposed framework.

3.1.1 Representing “how to teach” In order to provide an answer to the issue about how to teach, an instructional design ontology needs to capture Teaching Methods together with the situations in which these can be applied (e.g. Learning Goals, characteristics of the Learners, course Domain). Moreover, an instructional design ontology should also provide information on how a particular Teaching Method can be applied in practice. These issues will be discussed using the exemplary ontology about Teaching Programming depicted in Figure 2, which is elicited from a case study of teaching methods in computing science (Nicholson and Fraser, 1997).

A Training in the Teaching Programming ontology consists of a Sequence of at least three different Teaching Methods. Moreover, there must be at least one Learning Goal for each Training. Additionally, each Learning Goal must be addressed by at least one Teaching Method. The selection of a Teaching Method in the Teaching Programming ontology depends (i) on the Domain the method is applied to (e.g. Programming), (ii) on its Learning Goal (e.g. Gain Fluency in Programming), and (iii) on the Learner. In this exemplary ontology, Teaching Programming is understood to be the set of all the teaching methods that are applicable to the Programming Domain.

The general rule which states that “fluency in a new programming language can only be attained by actually programming” (Nicholson and Fraser, 1997) is also included in the Teaching Programming ontology. This assertion is represented by relating the two concepts Programming Exercise and Gain Fluency in Programming using the attributes Learning Goal and Addressed by.

of 20

Figure 2: Exemplary Ontology

An ontology should also educate its viewers about the application of specific TEACHING METHODs. In the Teaching Programming ontology above, the attributes description, strength, and weakness of Teaching Method carry this information. Note, that such information can either be specified for an individual TEACHING METHOD or classes of Teaching Methods (e.g. general description of LECTURE). Subclasses of Teaching Method and their corresponding instances inherit this information. In the above exemplary ontology, Lecture about syntax and semantics is a subclass of the two Lecture and Teaching Programming classes. Hence, Lecture about syntax and semantics inherits the description, strength, and weakness information from both classes.

3.1.2 Representing “what to teach” The sample ontology depicted in Figure 3 shows how an ontology can be used to organise the result of the training design processes. A Course can be described together with information about (i) its Learning Goals, (ii) its Course Units, and (iii) the requirements about the prerequisite knowledge of the Learner. Note, that due to space restrictions, the ontology in Figure 3 does not include Learning Goals for individual Courses and Course Units. In this ontology, two different Java Courses that include Units about Java Syntax and Java Semantics are represented. The Beginners course in Java has no specific requirements for Learners and includes an additional unit about OO principles. The Java Conversion Course is only Suitable For a Learner with Knowledge about OO Principles and thus omits the Unit about OO Principles.

of 20

instance-of

Unit

Unit

instance-of

Learning Goal

instance-of

instance-of

Suitable Forinstance-of

instance-ofinstance-ofinstance-of

Unit

Knows

1..nLearning Goal

Suitable For

1..n

Learning Goal

1..n Unit <<concept>>Course

<<concept>>Course Unit

<<concept>>Learning Goal

<<concept>>Learner with Knowledge about OO Principles

<<individual>>Beginners course in Java

<<individual>>C++

<<individual>>C#

<<individual>>Java Conversion Course

<<individual>>Java Syntax

<<individual>>Java Semantics

<<individual>>OO-Principles

<<concept>>Java Course

<<individual>>Understand Java Semantics

<<concept>>OO-Language

<<concept>>Learner

Figure 3: Teaching Java

A Learner with knowledge about OO Principles is defined as a Learner who Knows an OO-Language. If it is stated in the ontology that an individual Learner knows C++ or C#, a set-theoretic reasoner can deduce that this Learner is in fact a Learner with knowledge about OO Principles. The following section describes set-theoretic reasoning and its application to instructional design in more detail.

3.2 Reasoning about ID knowledge

A knowledge representation formalism is a “fragmentary theory of intelligent reasoning” (Davis etal, 1993). Hence, an ontology representation should allow for deductions to be drawn from the statements that specify an ontology. These deductions can lead to the derivation of further knowledge that is not explicitly specified in the ontology. More specifically, reasoning should support the following use cases.

1. Modelling of the instructional design ontology.

Automated reasoning can provide support to ID experts during both the creation and the maintenance of instructional design ontologies. This favours a systematisation of ID knowledge (Mizoguchi and Bourdeau, 2000), where:

• Concepts found in the ID knowledge base are clearly defined.

• Concepts are organised in an is-a structure.

• Dependencies and relations among concepts are explicitly stated.

• The viewpoints used for structuring knowledge are made explicit.

• Knowledge consistency is maintained.

2. Enhanced querying

Automated reasoning offers enhanced querying possibilities that exceed the capabilities of conventional query languages like SQL. Enhanced queries can thus specify constraints to retrieve teaching methods or courses from the ontologies.

of 20

3. Verification & Validation of a training design

Verification is the act of checking whether the intentions of a training design have been met, while validation is the act of checking whether the designed training is fit for its purpose. A training can be checked for common errors in instructional design, or for violations of the rules specified in the ontology.

Reasoning based on the formal set-theoretic semantics of a represented ontology can be used in all the above use cases. Set-theoretic reasoning can derive implicit knowledge (Buchheit etal, 1993), such as:

• Unspecified set containment: the computation of whether a set A is included in another set B.

• Unspecified set elements: the computation of whether an individual I is an element of a set S.

• Consistency: the computation of whether either an individual I or a set S is inconsistent.

The complexity of algorithms for computing these inferences increases with the expressive power of the language used to specify a knowledge base (Tobies, 2001). Description Logic (DL) reasoners such as CLASSIC (Brachman etal, 1999), FaCT (Horrocks, 1999), or RACER (Haarslev and Moeller, 2001a) implement optimised algorithms for languages that are expressive enough to capture ID knowledge. Moreover, these algorithms were proven to be complete and sound. This means (i) that all the possible deductions are found, and (ii) that all the inferences are correct. Consequently, automated reasoning based on the above description logics is very powerful. However, there are some drawbacks associated with the usage of description logics.

• Firstly, the languages computed by DL engines are subsets of First-Order Logic. Hence, they require a background understanding of predicate logic. Unfortunately, this is not a desirable option in the modelling of ontologies by an instructional design expert. Moreover, DL engines analyse a knowledge base specified in textual form and only support a basic “ASK and TELL” interaction with the user (Baader etal, 2003). Although graphical DL editors such as OilEd (Bechhofer etal, 2001) provide a Graphical User Interface for a better interaction with a DL engine, this does not overcome the language barrier problem.

• Secondly, DL engines do not make the analyst aware of any existing differences between the knowledge as specified in the ontology (the so-called “told” knowledge) and the knowledge as interpreted by the DL engine at the end of its reasoning processes (the so-called “derived” knowledge). These differences typically correspond to rearrangements in the concept hierarchy, which the DL engine assumes to be true. For instance, an inconsistent concept is automatically treated as a sub-concept of all the other concepts in a knowledge base. While this is correct in terms of a purely set-theoretic interpretation, a knowledge modelling environment should rather report the inconsistency to the analyst and let her/him decide whether to accept it, reject it, or keep it on hold for a deferred decision.

of 20

• Thirdly, DL reasoning is based on the Open World Assumption (OWA). This means that anything not explicitly stated in a knowledge base is not necessarily false (Brachman etal, 1991). While the OWA is beneficial in reasoning with the TBox, this is not necessarily true for ABox reasoning. In practice, it has been reported that users experienced problems in reasoning within an OWA framework (Haarslev and Moeller, 2001b). For instance, the field of digital libraries is one of those domains where user expectations and actual Description Logic deductions may differ (Meghini and Straccia, 1996).

The CONCEPTOOL project (Compatangelo and Meisel, 2002) aims at incorporating the reasoning power of Description Logics in a user-friendly ontology modelling environment (see Figure 4 for a screenshot of the CONCEPTOOL Intelligent Knowledge Management Environment.).

Figure 4: The ConcepTool ontology editor

Reasoning in CONCEPTOOL is based on a “two-level approach” where the conceptual level is explicitly distinguished from the epistemological one (Compatangelo etal, 1999). The conceptual level contains the ontology as specified by the analyst, whereas the epistemological level describes the same ontology using a DL language. This explicit differentiation in two levels has the following advantages:

• Decoupling of deductions. Each deduction made by the DL reasoner is reported to the analyst, who can decide how to deal with it. For instance, in contrast with DL engines, inconsistent classes are not considered as a subclass of all other classes.

• Inclusion of conceptual modelling primitives. While Description Logics only use two generic modelling primitives, namely the concept and the role, CONCEPTOOL extends and specialises these primitives. More precisely, it explicitly distinguishes between classes and associations on one side and partonomic links, attributes, and class-association links on the other.

of 20

• Application of a localised Closed World Assumption. The rewriting process of the ontology in a Description Logic language adds assertions that emulate the Closed World Assumption. Therefore, the results of reasoning more closely match the expectations of the analyst.

• Explicit computation of a normalised concept. All inherited properties from superclasses are intersected with locally specified properties and presented in the user interface. Thus the analyst is aware of the knowledge contained in the ontology.

CONCEPTOOL uses a frame-based knowledge model that allows one to mix ABox and TBox assertions. For instance, the concept Learner with knowledge about OO Principles in the ontology in Figure 3 is used to denote any Learner with certain properties, rather than forcing the analyst to specify one particular value for the attribute Suitable For in the individual Java Conversion Course.

A detailed specification of the CONCEPTOOL knowledge model is out of the scope of this paper. However, the following paragraphs outline those features of the system that are relevant for the discussion of the proposed architectural framework. CONCEPTOOL has two types of concepts, namely CLASS and ASSOCIATION, together with two corresponding types of individuals. The class diagram depicted in Figure 5 shows the relationships between these four elements.

Association

KnowledgeBase

Class ClassIndividual AssociationIndividual

Individual

+attributes:AttributeLink[+parts:PartonomicLink[]+wholes:PartonomicLink+ownSlots:OwnSlot[]

Concept

+attributes:AttributeLink[+parts:PartonomicLink[]+wholes:PartonomicLink+ownSlots:OwnSlot[]

Figure 5:The knowledge model of ConcepTool

Concepts and individuals can be described using roles that belong to one of the following three specialised categories (where roles within each category can be arranged in a hierarchy):

• Attributes. An attribute link can be specified using a full range of cardinality restrictions and value restrictions. The latter allow one to define (i) the union or the intersection of classes, (ii) the enumeration of optional fillers, and (iii) the enumeration of mandatory fillers. These fillers must be ClassIndividuals.

• Partonomic links. Two concepts (i.e. either two classes or two associations) can be linked in such a way that one concept is understood to be part of a whole. This

of 20

partonomic relationship can be interpreted as two corresponding links. A part link pl of the whole points to the type part, which has an inverse whole link wl pointing to whole. The maximum cardinality of a whole link is restricted to 1 as a part cannot belong to two different wholes at the same time. All partonomic links are transitive, which means that if P2 is part of P1, which is in turn part of a whole, P2 is also part of this whole. Partonomic links between individuals can be introduced in a similar way.

• Association links: This bi-directional link relates a Class and an Association. The association link is also split into one link from the Class to the Association and an inverse link from the Association to the Class. The latter link is restricted to a maximum multiplicity of 1.

The following subsections describe how reasoning in CONCEPTOOL can be applied to the three use cases specified at the beginning of this section.

3.2.1 Use case 1 – ID ontology modelling Ontologies may be very large, complex in their structure, and possibly developed by a number of people during a long period of time. Therefore, the reuse of existing ontologies and the usage of tools to ‘’debug” them are vital for ontology development and maintenance (Fikes and Farquhar, 1999; McGuinness, 2000). During the creation of an ontology, automated reasoning can provide the following support:

• Detection of redundant knowledge. An entity in the real world could be represented more than once in the ontology. Consequently, two or more concepts could exist with equivalent interpretations. These concepts should be highlighted as synonyms.

• Highlighting of ambiguities. Two concepts supposed to denote different entities in the real world cannot be distinguished because they share the same interpretation. As in the previous case, these two concepts should be highlighted as synonyms.

• Derivation of hidden knowledge. Implicit knowledge such as normalised attributes and further links (e.g. super-subclass links, instantiation) is highlighted.

• Detection of inconsistencies and contradictions. Assertions about concepts in an ontology that contradict each other and result in an inconsistent concept or individual are highlighted. For instance, if it is stated in an ontology that a Teaching Method must mention its strengths and weaknesses, the system will not accept any Teaching Methods without this description.

These services are vital for the development and maintenance of a bug-free ontology.

3.2.2 Use case 2 – Enhanced querying Automated reasoning allows the enhanced querying of an ontology. An example of such a query could refer to “All Teaching Methods that are applicable to the Computing Domain and support the Learning Goal Gain fluency in Programming” in the ontology shown in Figure 2. This query must be phrased as a concept description so that the reasoner can retrieve all instances of this concept. In our exemplary ontology, a query that retrieves all the Teaching Methods for the Domain Computing Domain with the Learning Goal Gain fluency in Programming returns all elements of the classes Programming Exercise, Write Programs from Scratch, and Adapt Existing Code. As reasoning in Description Logics is sound, only those Teaching Methods are suggested which satisfy the constraints.

of 20

Suppose, the ontology shown in Figure 3 represents the knowledge base of an e-learning platform that automatically decides which courses are offered to a specific user. A Learner called John Doe wants to do a Java Course in this platform and his profile stores that he already Knows C++. The knowledge base could now be queried to find out all the instances of Java Course that are Suitable For John Doe, who already knows C++. Figure 6 shows a graphical representation of this query concept. In this case, the system returns the Java Conversion Course.

instance-of

Knows1Suitable For <<individual>>C++

<<concept>>Learner

<<concept>>Java Course

<<individual>>John Doe

Figure 6: A query concept to retrieve a suitable Java Course for John Doe

3.2.3 Use case 3 – Verification & validation of a training design Errors in the design of a training can be detected (i) by checking whether the design violates axioms stated in the ontology or (ii) by defining common error classes and by analysing whether the training is an instance of any of these error classes.

Using the ontology shown in Figure 2, a training to be validated must be defined as an instance of the concept Training. Therefore, either this individual commits to all the axioms specified for this concept, or it is inconsistent. Possible violations in this case include (i) the definition of two Teaching Methods only (where at least three ones are required), (ii) the omission of a Learning Goal, or (iii) the omission to address a Learning Goal with a Teaching Method.

4 Conclusions This paper demonstrates how ontologies can represent instructional design (ID) knowledge about what to teach and how to teach. The benefit of using ontologies is threefold. Firstly, they provide an easily understandable conceptualisation of the ID domain. Secondly, they allow knowledge to be represented at different levels of abstraction. Thirdly, they support different knowledge views, which are expressed using multiple classifications.

Automated reasoning is needed to make ontologies the source of intelligent behaviour in an educational system. This paper has described how set-theoretic deductions can be applied (i) to support the modelling of ID knowledge bases, (ii) to retrieve suitable teaching methods from them, and (iii) to detect errors in a training design.

The intelligent knowledge management environment CONCEPTOOL has been introduced as a mediator between a description logic reasoner and the ID analyst/designer. The two-level approach used in CONCEPTOOL – namely, the use of a

of 20

custom knowledge representation formalism and the introduction of rewrite rules to enable DL reasoning – can be also adopted in the development of systems that perform set-theoretic reasoning in the educational domain. The major advantage of a two-level system architecture is that it decouples the interpretation of a conceptual knowledge model as performed by an ID expert from the interpretation of the same knowledge model as performed by an automated reasoner. In this way, the ID expert makes the final decisions about the organisation of the knowledge base. A further benefit of separating the user-oriented conceptual level from the computer-oriented epistemological level is that a variety of inferential approaches can be used.

Automated reasoning as discussed this paper is not only restricted to set-theoretic deductions about formal definitions. Conversely, an ontology interpretation is also defined by the terms used to denote the elements of this ontology. Hence, a lexical analysis of relationships between ontology terms can lead to the discovery of further “potential relationships” between ontology elements. Such an analysis can be performed either by a simple string-substring matching algorithm (which infer, for instance, that Continuous Assessment might be a subclass of Assessment) or by a lexical database (like WordNet (Miller, 1995), which stores information relationships).

In general, the interpretation and thus the applicability of lexical analysis in ontology reasoning is limited by its own heuristic nature. For example, does a string-substring relationship between two terms denote a superclass relationship, a subclass relationship, or nothing at all? While heuristic analysis can be helpful during the knowledge modelling stage, reasoning soundness and completeness are necessary requirements both for knowledge retrieval and for verification & validation.

Certain aspects of instructional design knowledge, such as the ordering of elements, can neither be expressed in CONCEPTOOL nor in Description Logics. For instance, it is not possible to declare the order in which the units of a course are taught. However, this lack of expressivity can be overcome in two ways:

• Firstly, the conceptual knowledge representation can be extended, allowing the possibility of specifying lists and ignoring the ordering in DL reasoning.

• Secondly, using naming conventions (such as unit_1, unit_2, . . . , unit_n) and defining these units as sub-properties of unit, violations of a specified order (e.g. having the assessment before the content is taught) can thus be detected.

In conclusion, a formal knowledge representation always introduces intrinsic limitations in the expressive power, which prevent the complete specification of a domain. However, to date a formal representation is the only practical way to make a computer program “understand” the raw data contained in a knowledge base.

Acknowledgements This work is partially supported by the British Engineering & Physical Sciences Research Council (EPSRC) under grant GR/N15764 (IRC in AKT). We are grateful to Andreas Hörfurter for inspiring us to this research and for his input in an early version of this paper. We are also grateful to Frank Guerin for his valuable help during the revision of the final manuscript.

of 20

References

Allert, H. and Dhraief, H. and Kunze, T. and Nejdl, W. and Richter, C. (2002a), "Instructional Models and Scenarios for an Open Learning Repository - Instructional Design and Metadata", Proceedings of World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education (E-Learn 2002).

Allert, H. and Dhraief, H. and Nejdl, W. (2002b), "How are Learning Objects Used in Learning Processes? Instructional Roles of Learning Objects in LOM", Proceedings of the World Conference on Educational Multimedia, Hypermedia and Telecommunications (ED-MEDIA'02).

Baader, F. and Calvanese, D. and McGuinness, D. and Nardi, D. and Patel-Schneider, P. (2003), The Description Logic Handbook, Cambridge University Press, ISBN: 0521781760.

Bechhofer, S. and Horrocks, I. and Goble, C. and Stevens, R. (2001), "OilEd: a Reasonable Ontology Editor for the Semantic Web", Proceedings of the Joint German/Austrian Conference on Artificial Intelligence (KI'2001), Springer Verlag, pp. 396-408.

Bloom, B. (1956), Taxonomy of Educational Objectives, Handbook I: Cognitive Domain, Longmans & Green, ISBN: 0582280109.

Bourdeau, J. and Mizoguchi, R. (2002), "Collaborative Ontological Engineering of Instructional Design Knowledge for an ITS Authoring Environment", Proceedings of the 6th Intl. Conference on Intelligent Tutoring Systems (ITS 2002), Springer Verlag, pp. 399-409.

Brachman, R. and McGuinness, D. and Patel-Schneider, P. and Borgida, A. (1999), "Reducing CLASSIC to practice: knowledge representation meets reality", Artificial Intelligence, vol. 114, no. 1-2, pp. 203-237.

Brachman, R. and McGuinness, D. and Patel-Schneider, P. and Resnick, L. and Borgida, A. 1991, "Living with CLASSIC: when and how to use a KL-ONE-like language" in Principles of Semantic Networks, pp.401-456, Morgan Kaufmann.

Buchheit, M. and Donini, F.M. and Schaerf, M. (1993), "Decidable Reasoning in Terminological Knowledge Representation Systems", Journal of Artificial Intelligence Research, vol. 1, pp. 109-138.

Compatangelo, E. and Donini, F.M. and Rumolo, G. 1999, "Engineering of KR-Based Support Systems for Conceptual Modelling and Analysis" in Information Modelling and Knowledge Bases X, pp.115-131, IOS.

Compatangelo, E. and Meisel, H. (2003), "Conceptool: intelligent support to the management of expressive knowledge", Proceedings of the 7th Intl. Conference on Knowledge-Based Intelligent Information and Engineering Systems (KES'2003), Springer Verlag, pp. 81-88.

of 20

Compatangelo, E. and Meisel, H. (2002), "KShaRe: an architecture for sharing heterogeneous conceptualisations", Proceedings of the 6th Intl. Conference on Knowledge-Based Intelligent Information and Engineering Systems (KES'2002), IOS Press, pp. 1439-1443.

Davis, R. and Shrobe, H. and Szolovits, P. (1993), "What is a Knowledge Representation?", AI Magazine, vol. 14, no. 1, pp. 17-33.

Dick, W. and Carey, L. and Carey, J. (2000), The Systematic Design of Instruction (5th ed.), Pearson Allyn & Bacon, ISBN: 0321037804.

Duineveld, A. and Stoter, R. and Weiden, M. and Kenepa, B. and Benjamins, V. (2000), "Wondertools? A comparative study of ontological engineering tools", International Journal of Human-Computer Studies, vol. 52, no. 6, pp. 1111-1113.

Fikes, R. and Farquhar, A. (1999), "Distributed Repositories of Highly Expressive Reusable Ontologies", IEEE-Information Systems, vol. 14, no. 2, pp. 73-79.

Gaede, B. and Stoyan, H. (2001), "A Generator and a Meta Specification Language for Courseware", Proceedings of the World Conf. on Educational Multimedia, Hypermedia and Telecommunications 2001(1), pp. 533-540.

Gagne, R. (1985), The conditions of learning and theory of instruction, Wadsworth Publishing, ISBN: 0030636884.

Gaudioso, E. and Boticario, J. (2003), "Towards web-based adapting learning communities", Proceedings of the 11th Conference in Artificial Intelligence in Education, IOS Press, pp. 237-244.

Gruber, T. (1993), "A translation Approach to Portable Ontology Specification", Knowledge Acquisition, vol. 5, no. 2, pp. 199-221.

Haarslev, V. and Moeller, R. (2001a), "RACER System Description", Proceedings of the Intl. Joint Conference on Automated Reasoning (IJCAR'2001), pp. 701-706.

Haarslev, V. and Moeller, R. (2001b), RACER User's Guide and Reference Manual Version 1.6, University of Hamburg, Computer Science Department.

Horrocks, I. (1999), "FaCT and iFaCT", Proceedings of the Intl. Workshop on Description Logics (DL'99), pp. 133-135.

IEEE – LOM (2003), “Standard for Information Technology --Education and Training Systems -- Learning Objects and Metadata”, IEEE, available: http://ltsc.ieee.org/wg12/, last accessed 12.03.2004.

Koper, R. (2001) “Modeling Units of Study from a Pedagogical Perspective: the pedagogical meta-model behind EML”, available: http://eml.ou.nl/introduction/docs/ped-metamodel.pdf, last accessed 12.03.2004.

Leidig, T. (2001), "L3 - Towards an Open Learning Environment", ACM Journal of Educational Resources in Computing, vol. 1, no. 1, pp. 7.

of 20

Marcke, K. (1998), "GTE: An epistemological approach to instructional modelling", Instructional Science, vol. 26, no. 3--4, pp. 147-191.

McGuinness, D. (2000), "Conceptual Modeling for Distributed Ontology Environments", Proceedings of the 8th Intl. Conference on Conceptual Structures -- Logical, Linguistic, and Computational Issues.

Meghini, C. and Straccia, U. (1996), "Extending a Description Logic to Cope with the Completeness of Multimedia Documents", In Proceedings of the ECAI'96 Workshop on Knowledge Representation for Interactive Multimedia Systems, pp. 42-50.

Merrill, D. (2001), "Components of instruction toward a theoretical tool for instructional design", Instructional Science, vol. 29, no. 4, pp. 291-310.

Merrill, M. and ID2 Research Group (1998), "ID Expert: A second generation Instructional Development System", Instructional Science , vol. 26, no. 3--4, pp. 243-262.

Miller, G. (1995), " WordNet: a Lexical Database for English", Communications of the ACM, vol. 38, no. 11, pp 39-41

Mizoguchi, R. and Bourdeau, J. (2000), "Using ontological engineering to overcome common AI-ED problems", International Journal of Artificial Intelligence in Education, vol. 11, pp. 107-121.

Mukhopadhyay, M. and Madhu, P. (2001), "Instructional design in multi-channel learning system", British Journal of Educational Technology, vol. 32, no. 5, pp. 543-556.

Murray, T. (1999), "Authoring Intelligent Tutoring Systems: An analysis of the state of the art", International Journal of Artificial Intelligence in Education, vol. 10, pp. 98-129.

Murray, T. (1998), "Authoring Knowledge Based Tutors: Tools for Content, Instructional Strategy, Student Model, and Interface Design", Journal of the Learning Sciences, vol. 7., no. 1.

Murray, T. (1996), "Special Purpose Ontologies and the Representation of Pedagogical Knowledge", Proceedings of the Intl. Conference on the Learning Sciences.

Newell, A. (1982), "The knowledge level", Artificial Intelligence, vol. 18, no. 1, pp. 87-127.

Nicholson, A. and Fraser, K. (1997), "Methodologies for teaching new programming languages: a case study teaching LISP", Proceedings of the 2nd Australasian Conference on Computer science education.

Pan, J. and Horrocks, I. (2001), "Metamodeling Architecture of Web Ontology Languages", Proceedings of the 1st Semantic Web Working Symposium (SWWS'01), pp. 131-149.

of 20

Pawlowski, J. (2002), "Reusable Models of Pedagogical Concepts - a Framework for Pedagogical and Content Design", Proceedings of ED-MEDIA 2002, World Conference on Educational Multimedia, Hypermedia and Telecommunications.

Recker, M. and Wiley, D. (2001), "A non-authoritative Educational Metadata Ontology for Filtering and Recommending Learning Objects", Interactive Learning Environments, vol. 9, no. 3, pp. 255-271.

Reigeluth, C. (1999), Instructional design theories and models: A new paradigm of instructional theory, Lawrence Erlbaum Associates, ISBN: 0805828591.

Sowa, J. (1999), Knowledge Representation: Logical, Philosophical, and Computational Foundations, Brooks Cole Publishing Co, ISBN: 0-534-94965-7.

Tobies, S. (2001), Complexity results and practical algorithms for logics in knowledge representation. Dissertation at Technische Hochschule Aachen.

W3C - OWL (2003), “OWL Web Ontology Language Reference”, W3C, available: http://www.w3.org/TR/owl-ref, last accessed 12.03.2004.

W3C - RDF (2003), “Resource Description Framework (RDF)”, W3C, available: http://www.w3.org/RDF/, last accessed 12.03.200 .

of 20