previous class - university of...

Previous class• Cognitive adequacy (Strube)

– strong (SCA) vs weak (WCA)

• WCA– dimensions

• Tasks (reading, writing, using)• Facets

– Learning (acuainted, competent, expert)– Productivity (rate of task completion over time, error-free?)– Error detection and correction– Satisfaction

– WCA and syntax (keywords vs symbols, more vs less)– WCA and expressive power– WCA and computation

1Tuesday, 25 March 2014

WCA & Computation• The more services the better

– if delivered in reasonable time– different services for one language

might have different performance characteristics

• Users often accept greater computation time– if they get perceived sufficient value– within sane bounds

• Toleration of computation time varies– with circumstances– greater toleration at development time

• getting right trumps getting it fast

• Toleration != contentment– Even if they tolerate it, users prefer speed


SCA & WCA• What is the relation, if any?• Possible relations

– helps vs. ensures– could be negative!

• SCA could preclude WCA

• Why might these hold?

3

→ SCA WCA

SCAYes (at least, in the same respect)

???

WCA??? Yes (at least,

in the same respect)

Tuesday, 25 March 2014

Strube’s argument• First, we separate out certain WCA issues

– Many ways to make systems unusable, don’t do them!

• Acknowledge issues of SCA– SCA systems are much harder to develop

• We may not even know how to do it!– A SCA rep might not work!

• Three big benefits of SCA:– “enhanced validity”– “added flexibility and stability”– “better security and ease of use”– (This mix up some bits.)


Strube’s Argument: Validity

• A broadness consideration– Knowledge is tangled up with other knowledge– Scoping is difficult– Modelling “how we think” induces naturally appropriate scoping

• Which can be used to understand what we drop out

• Big Issue:– Experts often don’t do this!

• Focus on the domain, rather than reflection• They use tools (books, programs, etc.)• We typically understand the domain better than people!

5

“validity refers to the content and representational format of domain-specific knowledge, to the strategies of reasoning employed, and to the conditions of use for both”


Strube’s Arg: Flexibility & Stability

• An emulation consideration– We use lots of reps and reasoning strategies– We’re pretty good

• Problems:– Flexibility might be a coping, rather than enabling strategy

• E.g., it’s the best way given human constraints!• But consider chess playing...programs don’t have our limitations

– so succeed with different strategies– Multiple representations can inhibit engineering

• Don’t Repeat Yourself (DRY)• How do we know that they are the same?

6

“The flexibility gained through selective application of different reasoning strategies, which in turn utilize different parts of knowledge [is] perhaps the most important single cause of our success in thinking and problem solving. It follows that the cognitive approach to knowledge engineering must strive for variety in reasoning, and must likewise try to embody domain-specific knowledge in multiple representation formats in the knowledge base.”


Strube’s Arg: Ease of Use

• A focused consideration– We need explanations (of certain kinds)– Hard to see how to get them without SCA

• Problems:– Do we need these explanations?

• A lot of modern day stuff is focused on debugging KRs• Software debuggers might be more relevant

– Do we need explanations?• Question answering systems may not!

–Or may be fine with just supplying facts– What about cost to the rest of the system?

7

A system that models the reasoning of real experts provides a sound base for giving explanations to its user, explanations that are both correct and understandable. By contrast, it is difficult to see how a system whose reasoning is utterly un-human might arrive at explanations that fulfil both requirements...A cognitive approach is necessary, at least to some degree, to make the explanation component of a system useful.


Strube’s Argument• It’s not clear that for many applications

– We need SCA– We need SCA to achieve validity, flexibility, etc.– We can achieve validity, flexibility, etc. via SCA

• Where might SCA still be useful?– Modelling people!


Definition Oriented Development(encore)

Sebastian Brandt [email protected]

(Slides by Bijan Parsia, [email protected])


mailto:[email protected]




Formali

ze

The Simple Development Cycle

has knowledge

Conceptualize

∃P.C ⊑ ∀P.DA⊓B ⊑ ∀P.D∃P(A⊔B)⊑¬D

has KnowledgeCommunicate

Inference


Recall: Definition Oriented Dev• “Reduce” (certain kinds of) effort

– Local focus on what terms mean

• Verification– There are consequences to what we say– We can spot wrong links

• Work on our part to detect problems• But inferred links are a subset of all links

– The reasoner can tell us about broken definitions• We still need to understand them!

• Improve interaction– The KR becomes “reactive”

• Comes at a computational cost!


Reduce which effort?• Target: Hierarchical controlled vocabularies

– Aka taxonomies (why do we want these?)

• Without (logically encoded) definitions– We must formulate the definitions– We must put terms “in their proper place”– We must assert every non-trivial “link”– We must check that these are the right links

• Thus we must determine what all the right links are• We must also verify that the links we include are right

• How much work?– 100 terms ≈ 10,000 (1002) possible subsumptions!

• But we don’t need to be naive about it– How many terms do we need?

• Current terminologies tend toward the hundreds of thousands• How about in principle?

Mediatedby humans!


Types of Bicycle injury•1972 ICD-9 (E826) 8 •1990 READ-2 (T30..) 81• late 1990s READ-3 87•1999 ICD-10 ……

http://www.cs.man.ac.uk/~rector/presentations/Reasoning-web-rector-GALEN-2006.pptTuesday, 25 March 2014

1999 ICD10: 587 codes

•V31.22 Occupant of three-wheeled motor vehicle injured in collision with pedal cycle, person on outside of vehicle, nontraffic accident, while working for income•W65.40 Drowning and submersion while in bath-tub, street and highway, while engaged in sports activity•X35.44 Victim of volcanic eruption, street and highway, while resting, sleeping, eating or engaging in other vital activities

http://www.cs.man.ac.uk/~rector/presentations/Reasoning-web-rector-GALEN-2006.pptTuesday, 25 March 2014

It’s getting worse!• “With the move to ICD-10, the [1] code for suturing an

artery will become 195 codes, designating every single artery, among other variables, according to OptumInsight, a unit of UnitedHealth Group Inc. A single code for a badly healed fracture could now translate to 2,595 different codes, the firm calculates. Each signals information including what bone was broken, as well as which side of the body it was on.”*

• A compositional problem– We are using opaque names– Thus every combination needs a new name– Thus must trade off

• Precision for number

15*http://online.wsj.com/article/SB10001424053111904103404576560742746021106.htm


http://online.wsj.com/article/SB10001424053111904103404576560742746021106.htm

http://online.wsj.com/article/SB10001424053111904103404576560742746021106.htm

Two Problems• We have a lot of terms

– lots and lots and lots– how do we manage them?

• We have pressures to add more terms– fundamental

• medical knowledge expands• medical record keeping needs expand

– artificial (or “technical”) • primarily, combinatorial pressure

16

Edwin Smith Papyrus

http://www.flickr.com/photos/78572993@N00/2226696853/




500 codes in pieces• 10 things to hit…

– Pedestrian / cycle / motorbike / car / HGV / train / unpowered vehicle / a tree / other

• 5 roles for the injured…– Driving / passenger / cyclist / getting in / other

• 5 activities when injured…– resting / at work / sporting / at leisure / other

• 2 contexts…– In traffic / not in traffic

17

V12.24 Pedal cyclist injured in collision with two- or three-wheeled motor vehicle, unspecified pedal cyclist, nontraffic accident, while resting, sleeping, eating or engaging in other vital activities

Slide from: http://www.cs.man.ac.uk/~rector/presentations/snomed-rector-history-and-future-of-terminology.ppt

10 * 5 * 5 * 2 = 500


http://www.cs.man.ac.uk/~rector/presentations/snomed-rector-history-and-future-of-terminology.ppt

http://www.cs.man.ac.uk/~rector/presentations/snomed-rector-history-and-future-of-terminology.ppt

Some Empirical Data• We took a number of medical vocabularies

– Parsed their terms– Divided the number of terms by the size of the basis


Terms as (class) expressions• The big move

– from names to expressions– from few names we induce many expressions

• a terse representation of the term space• humans work this way!

– position driven by definition

• Brings complexity– Terms have different grammatical roles

• classes, properties– We have to use logicy stuff

• reasoning!


Coordination• “...one key problem limiting re-use and application-

independence is that the formalisms used force application-specific choices...one of the key application-specific decisions which formalisms force on users is the organisation of a fixed taxonomy based on indivisible representations of concepts. Inevitably, indivisible representations of complex concepts conflate several ideas, and since any fixed organisation of such complex ideas is fundamentally arbitrary, choices can only be made on the basis of the current application...avoiding such decisions is an important technique for achieving reuse.

20

http://www.opengalen.org/download/aime-95.pdfTuesday, 25 March 2014

http://www.opengalen.org/download/aime-95.pdf

http://www.opengalen.org/download/aime-95.pdf

Forms of coordination• Pre-coordination

– No code without prior agreement• Only explicitly published codes• Maybe there’s an “other”• Integrating others is post-facto

– Code drift• Introducing new codes complicates old codings

– Though devtime definitions can help!

• Post-coordination– Core set of of terms

• We can fix these, even– A code is any valid expression

• We don’t need an expression– Many nonsensical codes!– Requires reasoning at “runtime”


Coordination Example• Imagine a simple taxonomy

– Nuts and nut allergy

• Notice the lack of specificity– No PecanAllergy, HazelnutAllergy, etc.– Why not?

• Consider a form using this– Very simple– We have a patient with an almond allergy

• Do we select– Allergy?






• Do we select– Allergy?– Nut Allergy?






• Do we select– Allergy?– Nut Allergy?– Other?


Coordination Example• All choices have problems

– The fact of (divergent) choice– The uncontrolled bits

• Can definitions help?


Coordination by definition• Note that now we have some expressive power


The pre-coordination cycle

has knowledge

Conceptualize

∃P.C

has

27

http://www.flickr.com/photos/seattlemunicipalarchives/4058808950/

No changes at runtime

Code set changes over time




The Post-coordination cycle

Formali

ze

has knowledge

Conceptualize

∃P.C ⊑ ∀P.DA⊓B ⊑ ∀P.D∃P(A⊔B)⊑¬D

has Communicate

Inference

28http://www.flickr.com/photos/seattlemunicipalarchives/4058808950/

Write expressions




Runtime?• UI might be tricky...


Some Pre-coordination Necessary• SNOMED is the canonical post-coordinated vocab

– Still has a medium coordination factor– The minimum basis might not be the sensible basis


Runtime vs. Development Time• We can see benefits at each stage• Each stage has different requirements

– E.g., Runtime has higher performance demands– E.g., Development time has higher correctness demands

• Beyond codes!– We can look at definition structure

• To structure other parts of forms!


previous class - university of...

Documents