© university of manchester creative commons attribution-noncommercial 3.0 unported 3.0 license...
DESCRIPTION
© University of Manchester Creative Commons Attribution-NonCommercial 3.0 unported 3.0 license 3 Aspects of Artifacts for Faithful Interoperability wrt terminologies / ontologies ►Competence / coverage - ►does it provide the information? can it answer the relevant questions? Identify value sets? Constrain what is legal? What can be said where? ►Correctness ►with regards to what gold standard? ►Usability - ►can it be used by authors? end users? reearchers? …? ►Reliability ►do users always enter / understand information the same way? the same user? different users? different locales? ►is it always bound to documents / EHRs in the same way? ►Robustness - what errors are introduced in the face of the unexpected / incorrect use? How likely are different types of incorrect usage? ►technical? ►human factors?TRANSCRIPT
QuickTime™ and a decompressor
are needed to see this picture.
© University of Manchester Creative Commons Attribution-NonCommercial 3.0 unported 3.0 license QuickTime™ and a decompressorare needed to see this picture.
Quality Assurance, Ontology Engineering, Quality Assurance, Ontology Engineering, and Semantic Interoperability wrt and Semantic Interoperability wrt
Terminologies & OntologiesTerminologies & Ontologies
Alan Rector
BioHealth Informatics GroupSchool of Computer Science &
Northwest Institute of BioHealth InformaticsUniversity of Manchester, Manchester M13 9PL
This project is funded by the European Union
© University of Manchester Creative Commons Attribution-NonCommercial 3.0 unported 3.0 licenseQuickTime™ and a
decompressorare needed to see this picture.
2
End-to-end preservation of meaningEnd-to-end preservation of meaning► Faithful transmission of information
►The same actions taken by human or machine on the output as would have been taken on the input
►Valid aggregation of data input / output in multiple sites►For EHRs / decision support, need more than markup
QuickTime™ and a decompressor
are needed to see this picture.
QuickTime™ and a decompressor
are needed to see this picture.
Decisions onInput /output
Decisions onoutput /
input
© University of Manchester Creative Commons Attribution-NonCommercial 3.0 unported 3.0 licenseQuickTime™ and a
decompressorare needed to see this picture.
3
Aspects of Artifacts for Faithful Aspects of Artifacts for Faithful Interoperability wrt terminologies / ontologiesInteroperability wrt terminologies / ontologies► Competence / coverage -
►does it provide the information? can it answer the relevant questions? Identify value sets? Constrain what is legal? What can be said where?
► Correctness►with regards to what gold standard?
► Usability -►can it be used by authors? end users? reearchers? …?
► Reliability►do users always enter / understand information the same way?
• the same user? different users? different locales?
► is it always bound to documents / EHRs in the same way?
► Robustness - what errors are introduced in the face of the unexpected / incorrect use? How likely are different types of incorrect usage?
►technical?►human factors?
© University of Manchester Creative Commons Attribution-NonCommercial 3.0 unported 3.0 licenseQuickTime™ and a
decompressorare needed to see this picture.
4
Faithfulness & QA at all levelsFaithfulness & QA at all levels
►Pragmatic‣ are the same decision taken by all interoperating systems / users?‣ does everyone use it the same way? (inter-rater reliability)?‣ do users use it as intended - rater-author consensus?
►Semantic / Ontological‣ are the intended meanings explicit? correct? attributed to authorities?‣ do users understand the implications of their entries? queries?
• Logical / Software- Are the inferences and answers to queries as intended?- Does the logical model match the intended semantics?- What extra-logical representation is needed? How can it be validated?- Is it logically self consistent ? Does extralogical operations conform to spec?
‣ Lexical / linguistic- Are ID and Naming conventions observed? synonyms adequate? - Do labels reflect clinical discourse intuitively? Are they misleading?- Are multilingual/multi-cultural issues addressed? Validated? Anomalies public?
- * Syntactic- Does the data model connect properly?- …
© University of Manchester Creative Commons Attribution-NonCommercial 3.0 unported 3.0 licenseQuickTime™ and a
decompressorare needed to see this picture.
5
QA methods for terminologies / ontologies QA methods for terminologies / ontologies Human Factors Ontological / Logical /
Software
Pragmatic End-to-end decision testinginter-rater reliability / repeatabilityCorpus studies for content coverage
Metadata for:provenance, editorial status, versioning & localisation, known issues, mappings, links to external resources, guidelines, …
Semantic / Ontological
Specification of questions to be asked / answeredAuthoritative consultationConsistency with external sources
Textual definitions & paraphrasesAnalysis of definitions & hierarchies
Logical /Software
Challenge testingUsability studiesLanguage generation for user friendly validation of logical structure
Logical definitions & descriptionsCorrectness of inferences, logical hierarchies, & query answering Logical consistencySchema / pattern / criteria conformance testing (unit testing; regression testing)Connectivity testing
Lexical / Linguistic
Corpus studies for language: synonyms anaphora, variants, …Comprehension studiesBack translation studies
Language generationText mining Lexicon structure