A Simplified Agile Methodology for Ontology Development

Silvio Peroni

13th OWL: Experiences and Directions Workshop and 5th OWL Reasoner Evaluation Workshop

Bologna, Italy, November 20, 2016

Paper: https://w3id.org/people/essepuntato/papers/samod-owled2016.html

Inspiration and desiderata

• Desiderata (or long story short) – Methodology for developing relatively small-/medium-size ontologies quickly – Advantages of agile methodologies (from Software Engineering)

✦ adaptive planning ✦ evolutionary development ✦ early delivery ✦ continuous improvement ✦ rapid and flexible response to change

– No pair programming: from personal experience, the development of small ontologies usually involves only one ontology engineer

– Precise definition of different kinds of tests that the model must pass at each stage of the development, and that can be used for documenting the model as well

• Source of inspiration – Software Engineering: test-driven development – Ontology Engineering: other existing agile methodologies, in particular Extreme

Design (XD) – disclosure: I have been a user of this methodology since its introduction

SAMOD

• SAMOD, i.e., Simplified Agile Methodology for Ontology Development, a novel agile methodology for the development of ontologies

• Small steps of an iterative process that focuses on creating well-developed and documented models through using significative exemplars of data so as to produce ontologies that are always ready-to-be-used and easily-understandable by humans (i.e. the possible customers) without spending a lot of effort

• Why we should use data – Avoid inconsistencies: a common mistake when developing a model is to make a TBox

(i.e., only classes and properties) that is consistent if considered alone and that becomes inconsistent when we define an ABox (i.e., individuals and relations between them) for it

– Self-explanatory and easy-understandable models: using data as part of the ontology development, ontology engineers and developers are obliged to think about the possible ways users will understand and use the ontology they are developing

– Examples of usage: producing data within the development process means to have a bunch of exemplars that describe the usage of the model and allows users to apply a learn-by-example approach in understanding the model and during their initial skill acquisition phase

Basic terms

• Current model: a preliminary/final version of the ontology under-development/developed

• Modelet: a stand-alone model describing a particular domain – a modelet does not include entities from other models and it is not included in other models (it is a monolithic artefact)

• Test case: a bag containing – [done by ontology engineer + domain expert] motivating

scenario, informal competency questions, glossary of terms – [done by ontology engineer] a modelet/current model,

implementation of the examples in the motivating scenario, implementation of the informal competency questions

Material collected with domain experts

Motivating scenario

Informal competency

question

Glossary of terms

Kinds of tests

• Model test: it aims at checking the validity of the model against formal (via reasoners) and rhetorical (does it implement the motivating scenario? does it use appropriate language?) requirements

• Data test: it aims at checking the validity of the model with the implemented examples against formal (via reasoners) and rhetorical requirements (are the examples fully implemented?)

• Query test: it aims at checking the validity of teach SPARQL query against formal (does each SPARQL query run correctly on the examples implemented?) and rhetorical requirements (does the result returned by running the SPARQL query return the expected values?)

The processSAMOD is an agile methodology for the development of ontologies by means of small steps of an iterative process that focuses on creating well-developed and documented models through using significative exemplar of data

ymilestone

nGo to the previousmilestone

n

n

y

is there any further scenario?

did it pass model, data, query tests?

did it pass model, data, query tests?

did it pass model, data, query tests?

end of process

n

if we need to extendthe model in the future

Collect requirements and develop a modelet

Ontology Engineers (OEs) collect all the information about a specific domain, with the help of Domain Experts (DEs), build a modelet formalising the domain in consideration, and then create a new test case

milestone y

Merge the modelet with the current final model

OEs merge the modelet of the new test case with the current model produced by the end of the last process iteration (first iteration: modelet becomes current model)

Refactor the current final model

milestone y

OEs refactor (important: reuse existing knowledge) the current model, focussing on the last part added in the previous step

From modelets to the final model

First iteration: modelet

Second iteration: modeletFinal model after

two iterations

Experiments

• 9 Computer Science and Law people – 1 professor – 2 post-docs – 6 Ph.D. students

• Task: to use SAMOD (one iteration only) for modelling a particular motivating scenario provided as exercise

– SAMOD, as well as the main basics in Ontology Engineering, OWL, and Semantic Web technologies, were introduced to the subjects during four lectures of four hours each

• Three questionnaires – Previous experience in Ontology Engineering and OWL – System Usability Scale (SUS) – Experience in using SAMOD for completing the task

Results

• SUS score: 67.25 (in a 0 to 100 range) – Usability score: 65.6 – Learnability score: 73.75

• Other insights – SAMOD evaluated positively by 7 subjects (“useful”, “natural”, “effective”, “consistent”) – Criticisms by 5 subjects: needs of more knowledge about Semantic Web technologies and Ontology Engineering – Tests for assessing the ontology developed after a certain step: 3 positive comments vs. 1 negative one – Scenarios and examples: 3 positive comments – Implementation of competency questions in form of SPARQL queries: 2 positive comments

Conclusions

• SAMOD, an agile methodology for the development of small-size ontologies – Each iteration has three steps

• Experiments showed a good level of usability and learnability

• Future developments – Involve a larger set of users for testing it – Develop supporting tools for accompanying and

facilitating users in each step of the methodology

Silvio Peroni silvio.peroni@unibo.it - @essepuntato

Digital And Semantic Publishing Laboratory Department of Computer Science and Engineering

Alma Mater Studiorum – Università di Bologna Bologna, Italy

dasplab.cs.unibo.it – www.unibo.it

Thank you for your attention