invited talk 2007 international conference on case-based reasoning 13 august 2007
DESCRIPTION
Addressing Perceptions of Case-Based Reasoning. David W. Aha Head, Adaptive Systems Section Navy Center for Applied Research in AI Naval Research Laboratory, Code 5514 Washington, DC [email protected]. Invited Talk 2007 International Conference on Case-Based Reasoning 13 August 2007 - PowerPoint PPT PresentationTRANSCRIPT
Addressing Perceptions of Case-Based Reasoning Addressing Perceptions of Case-Based Reasoning David W. Aha David W. Aha ICCBR-07 ICCBR-07 13 August 2007 13 August 2007 Belfast, Northern IrelandBelfast, Northern Ireland 1
1. Perceptions 2. Objectives 3. Survey 4. Findings 5. Interpretation
Invited Talk2007 International Conference on Case-Based Reasoning
13 August 2007Belfast, Northern Ireland
David W. AhaHead, Adaptive Systems Section
Navy Center for Applied Research in AINaval Research Laboratory, Code 5514
Washington, [email protected]
Addressing Perceptions of Case-Based Reasoning
Addressing Perceptions of Case-Based Reasoning Addressing Perceptions of Case-Based Reasoning David W. Aha David W. Aha ICCBR-07 ICCBR-07 13 August 2007 13 August 2007 Belfast, Northern IrelandBelfast, Northern Ireland 2
1. Perceptions 2. Objectives 3. Survey 4. Findings 5. Interpretation
Goals of this presentation
1. Raise awareness on how to assess CBR R&D methods
2. Assess CBR R&D methods we’re publishing
3. Relate CBR’s R&D methods to those used in AI
4. Beg for your forgiveness?
Addressing Perceptions of Case-Based Reasoning Addressing Perceptions of Case-Based Reasoning David W. Aha David W. Aha ICCBR-07 ICCBR-07 13 August 2007 13 August 2007 Belfast, Northern IrelandBelfast, Northern Ireland 3
1. Perceptions 2. Objectives 3. Survey 4. Findings 5. Interpretation
Outline
1.Perceptions2.Objectives3.Survey4.Findings5.Interpretation
Addressing Perceptions of Case-Based Reasoning Addressing Perceptions of Case-Based Reasoning David W. Aha David W. Aha ICCBR-07 ICCBR-07 13 August 2007 13 August 2007 Belfast, Northern IrelandBelfast, Northern Ireland 4
1. Perceptions 2. Objectives 3. Survey 4. Findings 5. Interpretation
Outline
1.Perceptions• Story: Gnats, envy, & self-doubt• Quest
2.Objectives3.Survey4.Findings5.Interpretation
Addressing Perceptions of Case-Based Reasoning Addressing Perceptions of Case-Based Reasoning David W. Aha David W. Aha ICCBR-07 ICCBR-07 13 August 2007 13 August 2007 Belfast, Northern IrelandBelfast, Northern Ireland 5
1. Perceptions 2. Objectives 3. Survey 4. Findings 5. Interpretation
What perceptions of case-based reasoning (CBR) exist?
• Among active CBR researchers/practitioners• Among others
Addressing Perceptions of Case-Based Reasoning Addressing Perceptions of Case-Based Reasoning David W. Aha David W. Aha ICCBR-07 ICCBR-07 13 August 2007 13 August 2007 Belfast, Northern IrelandBelfast, Northern Ireland 6
1. Perceptions 2. Objectives 3. Survey 4. Findings 5. Interpretation
My perception
Case-Based Reasoning
Artificial Intelligence
……
Addressing Perceptions of Case-Based Reasoning Addressing Perceptions of Case-Based Reasoning David W. Aha David W. Aha ICCBR-07 ICCBR-07 13 August 2007 13 August 2007 Belfast, Northern IrelandBelfast, Northern Ireland 7
1. Perceptions 2. Objectives 3. Survey 4. Findings 5. Interpretation
Gnats, envy, & self-doubt
Gnat UK Gnat
Observation: CBR perceived differently by others
Addressing Perceptions of Case-Based Reasoning Addressing Perceptions of Case-Based Reasoning David W. Aha David W. Aha ICCBR-07 ICCBR-07 13 August 2007 13 August 2007 Belfast, Northern IrelandBelfast, Northern Ireland 8
1. Perceptions 2. Objectives 3. Survey 4. Findings 5. Interpretation
Gnats
In CBR (Pal & Shiu, 2004; Kolodner, 1993) expertise is embodied in a library of past cases… <long, accurate description of CBR> The major problem with CBR is that it lacks a sound theoretical framework for its application and has only achieved limited success. - Anonymous senior AI researcher/proposer, 2005
“Case based reasoning is often limited to surface features that may not be relevant to the operational military situation. (There is a need for deeper underlying reasoning, including analogical reasoning.)” - Anonymous ONR Program Manager, 2007
Addressing Perceptions of Case-Based Reasoning Addressing Perceptions of Case-Based Reasoning David W. Aha David W. Aha ICCBR-07 ICCBR-07 13 August 2007 13 August 2007 Belfast, Northern IrelandBelfast, Northern Ireland 9
1. Perceptions 2. Objectives 3. Survey 4. Findings 5. Interpretation
Gnats
ML: Prevailing view is CBR = Instance-based learning ML• No (e.g., 61% of papers in ECCBR-06 not related to ML)• Yet there is a relationship
–e.g., “CBR is a technique within the field of machine learning…” (Beltrán-Ferruz et al., ECCBR-06)
Artificial Intelligence: CBR not taught?• AIMA (Russell & Norvig, 2002-)
– 90% market share (1000+ universities, 91 countries)• CBR: Not discussed• IBL (3 pages) Statistical_Learning
Addressing Perceptions of Case-Based Reasoning Addressing Perceptions of Case-Based Reasoning David W. Aha David W. Aha ICCBR-07 ICCBR-07 13 August 2007 13 August 2007 Belfast, Northern IrelandBelfast, Northern Ireland 10
1. Perceptions 2. Objectives 3. Survey 4. Findings 5. Interpretation
Gnats
Computational analogy– “CBR systems…tend to use only minimal first-principles
reasoning…[and] rely on feature-based descriptions…[or] use domain-specific and task-specific similarity metrics. This can be fine for a specific application, but being able to exploit similarity computations that are more like what people do could make such systems…more understandable to their human partners.” (Forbus et al., IAAI-02)
Episodic memory– “Episodic memory can be thought of as the mother-of-all CBR
problems – how to store and retrieve cases about everything relevant in an entity’s existence. Most CBR research has avoided these issues.” (Nuxoll & Laird, ICCM-04)
Why isn’t some AI-related CBR research published at IC/ECCBR?
Addressing Perceptions of Case-Based Reasoning Addressing Perceptions of Case-Based Reasoning David W. Aha David W. Aha ICCBR-07 ICCBR-07 13 August 2007 13 August 2007 Belfast, Northern IrelandBelfast, Northern Ireland 11
1. Perceptions 2. Objectives 3. Survey 4. Findings 5. Interpretation
Gnats
Some Stereotypical perceptions of CBRUSA funding agencies Of questionable merit (?)
Cognitive Architectures Informal/incomplete models of episodic memory
Cognitive Psychology Cognitively implausible exemplar models
Artificial Intelligence A subfield, once dominated by speculative evaluation methodologies (Hall & Kibler, AIM 1985)
Machine Learning Case-based algorithms for supervised learning
Statistics A target?
Knowledge Management A panacea (still true?)
Business A mysterious technique whose name is rarely mentioned by its practitioners (still true?)
Us A discipline worthy of research & application
How could any misperceptions be addressed?How could any misperceptions be addressed?
Addressing Perceptions of Case-Based Reasoning Addressing Perceptions of Case-Based Reasoning David W. Aha David W. Aha ICCBR-07 ICCBR-07 13 August 2007 13 August 2007 Belfast, Northern IrelandBelfast, Northern Ireland 12
1. Perceptions 2. Objectives 3. Survey 4. Findings 5. Interpretation
Not interested in giving you yet another content survey
# Grouping Title (CBR = "Case-Based Reasoning") Authors1 CBR commentaries: Introduction Aha, Marling, & Watson2 CBR foundations Richter & Aamodt3 Representation in CBR Bergmann, Kolodner, & Plaza4 Retrieval, reuse, revision, and retention in CBR López de Mántaras et al. (13 authors)5 Integrations Marling, Rissland, & Aamodt6 Advances in conversational CBR Aha, McSherry, & Yang7 Textual CBR Weber, Ashley, & Brüninghaus8 Distributed CBR Plaza & McGinty9 Soft CBR Cheetham, Shiu, & Weber
10 Design, innovation, and CBR Goel & Craw11 CBR for diagnosis applications Goker, Howlett, & Price12 Case-based planning Cox, Muñoz-Avila, & Bergmann13 Medical applications in CBR Holt, Bichindaritz, Schmidt, & Perner14 CBR and law Rissland, Ashley, & Branting15 CBR-inspired approaches to education Kolodner, Cox, & González-Calero16 Knowledge management in CBR Althoff & Weber17 Image processing in CBR Perner, Holt, & Richter18 Case-based recommender systems Bridge, Goker, McGinty, & Smyth19 Fielded applications of CBR Cheetham & Watson20 Emergent CBR applications López de Mántaras, Perner, & Cunningham
Introduction
Techniques
Applications
Topic Areas
Task Areas
We We havehave existing surveys of CBR (e.g., existing surveys of CBR (e.g., KERKER 2005 special issue) 2005 special issue)
Addressing Perceptions of Case-Based Reasoning Addressing Perceptions of Case-Based Reasoning David W. Aha David W. Aha ICCBR-07 ICCBR-07 13 August 2007 13 August 2007 Belfast, Northern IrelandBelfast, Northern Ireland 13
1. Perceptions 2. Objectives 3. Survey 4. Findings 5. Interpretation
Envy
Am I (unnecessarily) wishing for something?Am I (unnecessarily) wishing for something?
Formal foundations envy?•e.g., Bayesian, first-order logic, decision theory, COLT, …•But we have this:
– e.g., Cover & Hart, 1967; Richter, FLAIRS-07; Richter & Aamodt, 2005 KER– And we’re the ultimate chameleons, even within AI
Methodological approach envy?•e.g., Experimental study of ML (Langley & Kibler, 1991), Crafting papers on ML (Langley, ICML-00), …
•Possibly: – We haven’t had received much proselytizing on this…yet– My awareness of these issues has increased; worth reviewing
Addressing Perceptions of Case-Based Reasoning Addressing Perceptions of Case-Based Reasoning David W. Aha David W. Aha ICCBR-07 ICCBR-07 13 August 2007 13 August 2007 Belfast, Northern IrelandBelfast, Northern Ireland 14
1. Perceptions 2. Objectives 3. Survey 4. Findings 5. Interpretation
EnvyCrafting papers (e.g., on ML) (Langley, ICML-00)•Content•Evaluation strategy•Communication
Paper content recommendations
•State the research goals and evaluation criteria•Specify the component (e.g., learning) & overall perf. task•Describe rep’n and organization of knowledge & data•Explain the system components (if any)•Evaluate the approach
–Empirical, theoretical, psychological, novel functionality•Describe related work
–Explain similarities/differences with your work•State the limitations
–Propose solutions
Addressing Perceptions of Case-Based Reasoning Addressing Perceptions of Case-Based Reasoning David W. Aha David W. Aha ICCBR-07 ICCBR-07 13 August 2007 13 August 2007 Belfast, Northern IrelandBelfast, Northern Ireland 15
1. Perceptions 2. Objectives 3. Survey 4. Findings 5. Interpretation
Self-Doubt
Summary questions
• How should (royal) we respond to possible misperceptions of CBR?– i.e., Other than to survey the field’s contents and its foundations
• Why are some folks ignoring CBR?• How can we attract them? • Does this concern our research methodologies (and/or their communication) rather than our research focus?
ProposalProposal: Examine our research methodologies: Examine our research methodologies
Realization: Realization: This requires a framework for investigationThis requires a framework for investigation
Addressing Perceptions of Case-Based Reasoning Addressing Perceptions of Case-Based Reasoning David W. Aha David W. Aha ICCBR-07 ICCBR-07 13 August 2007 13 August 2007 Belfast, Northern IrelandBelfast, Northern Ireland 16
1. Perceptions 2. Objectives 3. Survey 4. Findings 5. Interpretation
Quest: Identify, characterize, & compare CBR research methods
Don Quixote (Scott Gustafson)
Addressing Perceptions of Case-Based Reasoning Addressing Perceptions of Case-Based Reasoning David W. Aha David W. Aha ICCBR-07 ICCBR-07 13 August 2007 13 August 2007 Belfast, Northern IrelandBelfast, Northern Ireland 17
1. Perceptions 2. Objectives 3. Survey 4. Findings 5. Interpretation
Outline
1.Perceptions2.Objectives
• Questions• Conjectures/Hypotheses
3.Survey4.Findings5.Interpretation
Addressing Perceptions of Case-Based Reasoning Addressing Perceptions of Case-Based Reasoning David W. Aha David W. Aha ICCBR-07 ICCBR-07 13 August 2007 13 August 2007 Belfast, Northern IrelandBelfast, Northern Ireland 18
1. Perceptions 2. Objectives 3. Survey 4. Findings 5. Interpretation
Questions
1. How should we describe CBR to others?• i.e., in the context of AI
2. What R&D methodologies are we using? 3. Does CBR R&D differ from AI R&D?
Addressing Perceptions of Case-Based Reasoning Addressing Perceptions of Case-Based Reasoning David W. Aha David W. Aha ICCBR-07 ICCBR-07 13 August 2007 13 August 2007 Belfast, Northern IrelandBelfast, Northern Ireland 19
1. Perceptions 2. Objectives 3. Survey 4. Findings 5. Interpretation
Conjectures/Hypotheses
AI research is dominated by two methodologies (Cohen, 1991)• Model-centered (neat) (i.e., proving theorems on formal models)• System-centered (scruffy)
1. CBR research is not (currently) dominated by both• Dominated only by system-centered papers, which often lack models
for deriving claims, generating predictions, and explaining behavior
2. CBR research suffers from similar methodological problems• Model- and system-centered papers differ in whether they conduct
evaluations, assess performance, and describe expectations
3. The designation of CBR conference publications are distinguished by their research methodologies• Oral vs. poster presentations• Best paper nominees from others
Addressing Perceptions of Case-Based Reasoning Addressing Perceptions of Case-Based Reasoning David W. Aha David W. Aha ICCBR-07 ICCBR-07 13 August 2007 13 August 2007 Belfast, Northern IrelandBelfast, Northern Ireland 20
1. Perceptions 2. Objectives 3. Survey 4. Findings 5. Interpretation
Outline
1.Perceptions2.Objectives3.Survey
• Case base• Retrieval• Reuse• Revision
4.Findings5.Interpretation
Addressing Perceptions of Case-Based Reasoning Addressing Perceptions of Case-Based Reasoning David W. Aha David W. Aha ICCBR-07 ICCBR-07 13 August 2007 13 August 2007 Belfast, Northern IrelandBelfast, Northern Ireland 21
1. Perceptions 2. Objectives 3. Survey 4. Findings 5. Interpretation
Case Base
Case #1
Cohen, Paul R. (1991). A Survey of the Eighth National Conference on Artificial Intelligence: Pulling together or pulling apart? AI Magazine, 12(1), 16-41.
Paul R. Cohen (circa ~1991)
Frameworks for assessing AI R&D Methods
Summary of (Cohen, 1991)
• Conclusion: AI research follows two incomplete, complementary methodologies• Proposes: MAD (Modelling, Analysis, & Design) mixed methodology
RecommendationRecommendation: Make this required reading for AI researchers: Make this required reading for AI researchers
Read 150Papers!(can you imagine?)
Paul R. Cohen(circa ~2007)
Addressing Perceptions of Case-Based Reasoning Addressing Perceptions of Case-Based Reasoning David W. Aha David W. Aha ICCBR-07 ICCBR-07 13 August 2007 13 August 2007 Belfast, Northern IrelandBelfast, Northern Ireland 22
1. Perceptions 2. Objectives 3. Survey 4. Findings 5. Interpretation
(Cohen, 1991): 40 citations (Google Scholar, 8/1/07)
The importance of this link has been highlighted by several researchers, some even going so far as to state that AI will not advance as a science until the gap between those who construct models and those who build systems is closed. (Jennings, 1995 Artificial Intelligence).
There are two ways in which the fields proceed. One is through the development and synthesis of models of aspects of perception, intelligence, or action, and the other is through the construction of demonstration systems (Brooks, 1991 Science).
As Cohen (1991) demonstrated in his analysis of the papers presented at AAAI90, we are, as a discipline, just learning how to perform real, systematic experimentation. One hears a lot of talk about AI as an experimental science, but typically the “experiments" amount merely to writing a computer program that is supposed to validate some hypothesis by its very existence. (Pollock, 1992 Artificial Intelligence)
As Cohen (1991) points out, most research papers in AI, or at least at an AAAI conference, exploit benchmark problems; yet few of them relate the benchmarks to target tasks. (Howe & Dahlman, 2002 JAIR)
Cohen (1991) discovered that only 43% of the papers that described implemented systems report any kind of analysis of their contributions. Even of the papers that do describe evaluatory experiments, very few go beyond evaluating the programs to analyzing the scientific claims that the programs were written to demonstrate . (Ram & Jones, 1995 Philosophical Psychology)
Many of our suggestions are similar to the excellent points made by Cohen (1991) in his discussion of AI, but they seem worth instantiating for the field of machine learning (Langley & Kibler, 1991 “Experimental Study of ML”)
Methodological issues are by no means resolved (Cohen, 1991), but they are much discussed and a consensus is emerging on the importance of combining theoretical and empirical investigations. (Bundy, 1998 book)
Addressing Perceptions of Case-Based Reasoning Addressing Perceptions of Case-Based Reasoning David W. Aha David W. Aha ICCBR-07 ICCBR-07 13 August 2007 13 August 2007 Belfast, Northern IrelandBelfast, Northern Ireland 23
1. Perceptions 2. Objectives 3. Survey 4. Findings 5. Interpretation
Retrieval
My query: • Identify R&D methodologies being used in CBR•Compare results with general AI and other AI subfields
Case #1: (Cohen, 1991)•Develop and apply framework for analyzing AI R&D methodologies• Identify R&D methodologies being used•Propose novel R&D methodology (MAD)
Case #1
Cohen, Paul R. (1991). A Survey of the Eighth National Conference on Artificial Intelligence: Pulling together or pulling apart? AI Magazine, 12(1), 16-41.
Framework for assessing AI R&D Methods
1. Retrieve
Case #1
My Query
Addressing Perceptions of Case-Based Reasoning Addressing Perceptions of Case-Based Reasoning David W. Aha David W. Aha ICCBR-07 ICCBR-07 13 August 2007 13 August 2007 Belfast, Northern IrelandBelfast, Northern Ireland 24
1. Perceptions 2. Objectives 3. Survey 4. Findings 5. Interpretation
MAD Framework (Cohen, 1991)
Purpose
3. Models (Define, extend, generalize, provide semantics)
4. Theorems/Proofs for models
5. Present algorithm(s)
6. Analyze algorithm(s) Complexity Formal Informal
7. Present system
8. Analyze aspects of system Complexity Formal Informal
Argument
9. Example task Natural Synthetic Abstract
10. Task type Natural Synthetic Abstract
11. Task environment Embedded Not embedded
12. Assess performance
13. Assess coverage
14. Comparison
15. Predictions, hypotheses
16. Probe results
17. Unexpected results
18. Negative results
Note: It does not Note: It does not (completely) eliminate (completely) eliminate
subjective assessments!subjective assessments!
Addressing Perceptions of Case-Based Reasoning Addressing Perceptions of Case-Based Reasoning David W. Aha David W. Aha ICCBR-07 ICCBR-07 13 August 2007 13 August 2007 Belfast, Northern IrelandBelfast, Northern Ireland 25
1. Perceptions 2. Objectives 3. Survey 4. Findings 5. Interpretation
MAD Framework: Purpose fields
Field Description3. Models (Define, extend,
generalize, provide semantics)• Abstract, typically formal description of behavior and/or
environmental factors that affect behavior.• Purpose of building a model is to analyze its properties
4. Theorems/Proofs for models • e.g., Complexity, soundness, completeness, decidability
5. Present algorithm(s)
6. Analyze algorithm(s) • e.g., Complexity, soundness, completeness, decidability
7. Present system • Describes components, control flow
8. Analyze aspects of system
Addressing Perceptions of Case-Based Reasoning Addressing Perceptions of Case-Based Reasoning David W. Aha David W. Aha ICCBR-07 ICCBR-07 13 August 2007 13 August 2007 Belfast, Northern IrelandBelfast, Northern Ireland 26
1. Perceptions 2. Objectives 3. Survey 4. Findings 5. Interpretation
MAD Framework: Argument fields
Field Description9. Example task • Natural, synthetic, or abstract
10. Task type • Used iff multiple trials are described• Natural, synthetic, or abstract
11. Task environment • Used iff multiple trials are described• Embedded or not embedded (e.g., in other s/w, env’t)
12. Assess performance • Weak criterion: One perf. measure over many examples• e.g., a bakeoff is an assessment
13. Assess coverage • Solve instances of some problems in a defined problem space • Not a demo on superficially different problems w/o justification
14. Comparison • Goal: Study relative strengths/limitations of multiple techniques• e.g., a bakeoff is not a comparison
15. Predictions, hypotheses • Indicate reason to implement/test an idea • Not “My algorithm solves this problem”, or simple perf. demos• Many papers are vague as to why empirical work is described
16. Probe results • Go beyond central results (e.g.,follow-up expt’s, explanations)
17. Unexpected results • Infrequent
18. Negative results • Rare (if ever)
Addressing Perceptions of Case-Based Reasoning Addressing Perceptions of Case-Based Reasoning David W. Aha David W. Aha ICCBR-07 ICCBR-07 13 August 2007 13 August 2007 Belfast, Northern IrelandBelfast, Northern Ireland 27
1. Perceptions 2. Objectives 3. Survey 4. Findings 5. Interpretation
Reuse
1. Retrieve
MAD Framework
(Cohen, 1991)
Case Base
2. Reuse
MAD Framework
My Query
Analyze
Metrics
Results
Hypotheses
MAD Methodology
AAAI-90 Data
AAAI-90
ECCBR-06
ECCBR-06 Data
Analyze
AdaptedHypotheses
Results
ICCBR Audience(at lunch)
Addressing Perceptions of Case-Based Reasoning Addressing Perceptions of Case-Based Reasoning David W. Aha David W. Aha ICCBR-07 ICCBR-07 13 August 2007 13 August 2007 Belfast, Northern IrelandBelfast, Northern Ireland 28
1. Perceptions 2. Objectives 3. Survey 4. Findings 5. Interpretation
ECCBR-06: 36 Papers # First Author Title
1 Gabel Multi-agent CBR for cooperative RLs2 Ros Retrieving and reusing game plays for robot soccer3 Watson Self-organizing hierarchical retrieval in a case-agent system4 Karoui COBRAS; Cooperative CBR system for bilbliographic reference recommendation5 McDonnell A knowledge-light approach to regression using CBR6 Weber CBM for CCBR-based process evolution7 Gu Evaluating CBR sytems using different data sources: A case study8 Nicholson Decision diagrams: Fast and flexible support for case retrieval and recommendation9 Hervas CBR for knowledge-intensive template selection during text generation
10 Gupta Rough set feature selection algorithms for textual case-based classification11 Minor Experience management with case-based assistant systems12 McCarthy The needs of the many: A case-based group recommender system13 Kofod-Petersen Contextualised ambient intelligence through CBR14 Recio-Garcia Improving annotation in the semantic web and case authoring in textual CBR15 Herrera Unsupervised case memory organization: Analysing computational time and soft computing capabilities16 Freyne Further expeirments in case-based collaborative web search17 Bergmann Finding similar deductive consequences: A new search-based framework for unified reasoning from cases and general knowledge18 Baccigalupo Case-based sequential ordering of songs for playlist recommendation19 Perner A comparative study of catalogue-based classification20 Gomez-Gauchia Ontology-driven development of conversational CBR systems21 Massie Complexity profiling for informed case-base editing22 Wiratunga Unsupervised feature selection for text data23 Stahl Combining case-based and similarity-based product recommendation24 Coyle On the use of selective ensembles for relevance classification in case-based web search25 Bogaerts What evaluation criteria are right for CCBR? Considering rank quality26 Chakraborti Fast case retrieval nets for textual data27 Lamontagne Combining multiple similarity metrics using a multicriteria approach28 Althoff Case factory: Maintaining experience to learn29 Beltran-Ferruz Retrieval over conceptual structures30 Kuchibatla An analysis on transformational analogy: General framework and complexity31 Funk Discovering knowledge about key sequences for indexing time series cases in medical applications32 Montani CBR for autonomous service failure diagnosis and remediation in software systems33 Mendez Tracking concept drift at feature selection stage in SpamHunting: An anti-spam instance-based reasoning system34 Bergmann Case-based support for collaborative business35 Hefke A CBR-based approach for supporting consulting agencies in successfully accompanying a customer's introduction of KM36 Goker The PwC connection machine: An adaptive expertise provider
Addressing Perceptions of Case-Based Reasoning Addressing Perceptions of Case-Based Reasoning David W. Aha David W. Aha ICCBR-07 ICCBR-07 13 August 2007 13 August 2007 Belfast, Northern IrelandBelfast, Northern Ireland 29
1. Perceptions 2. Objectives 3. Survey 4. Findings 5. Interpretation
Outline
1.Perceptions2.Objectives3.Survey4.Findings
• Results• Analysis & Patterns• Followup
5.Interpretation
Addressing Perceptions of Case-Based Reasoning Addressing Perceptions of Case-Based Reasoning David W. Aha David W. Aha ICCBR-07 ICCBR-07 13 August 2007 13 August 2007 Belfast, Northern IrelandBelfast, Northern Ireland 30
1. Perceptions 2. Objectives 3. Survey 4. Findings 5. Interpretation
Findings: Results1 4 5 7
Complexity Formal Informal Complexity Formal Informal Natural Synthetic Abstract Natural
1 1 1 1 1 12 1 1 1 1 1 13 1 1 14 1 15 1 1 16 1 1 1 1 17 1 1 18 1 1 1 1 19 1 1 1
10 1 1 1 1 111 1 1 112 1 1 1 113 1 1 114 1 1 115 1 1 116 1 1 1 117 1 1 1 118 1 1 119 1 120 1 121 1 1 122 1 1 123 1 124 1 1 125 126 1 1 1 1 127 1 1 1 128 1 129 1 130 1 1 1 131 1 132 1 133 1 1 1 134 1 135 1 136 1 1
3
Present Algorithms
Present system
Example type Task Evaluation typeAnalyze Algorithms Analyze aspect(s) of system
#
Define, Extend, Generalize,
Differentiate, Semantics for Formal Models
Theorems and proofs re: Model
Informal Model to frame the research
1096 8
Addressing Perceptions of Case-Based Reasoning Addressing Perceptions of Case-Based Reasoning David W. Aha David W. Aha ICCBR-07 ICCBR-07 13 August 2007 13 August 2007 Belfast, Northern IrelandBelfast, Northern Ireland 31
1. Perceptions 2. Objectives 3. Survey 4. Findings 5. Interpretation
Findings: ExamplesField # Example
Pu
rpo
se
3. Models (Define, extend, etc.) 9 Similarity/deductive reasoning (Bergmann & Mougouie)
4. Theorems/Proofs for models 0
5. Present algorithm(s) 21 Retrieve k cases from a DD (Nicholson et al.)
6. Analyze algorithm(s) 7 Unsupervised algs. (Fornells Herera et al.)
7. Present system 21 PwC Connection Machine (Göker et al.)
8. Analyze aspects of system 7 Fast CRNs (Chakraborti et al.)
Arg
um
ent
9. Example task 22 Song playlists (Baccigalupo & Plaza)
10. Task type 26 European skiing holidays (McCarthy et al.)
11. Task environment 4 Game plays for robot soccer (Ros et al.)
12. Assess performance 27 Integrated CCBR evaluation (Gu & Aamodt)
13. Assess coverage 1 Complexity profiling (Massie et al.)
14. Comparison 15 SpamHunting (Méndez Reboredo et al.)
15. Predictions, hypotheses 19 Rough set feature selection (Gupta et al.)
16. Probe results 7 Ave. diversity of decision trees (Coyle & Smyth)
17. Unexpected results 6 Pima’s precision results (Bogaerts & Leake)
18. Negative results 0
Addressing Perceptions of Case-Based Reasoning Addressing Perceptions of Case-Based Reasoning David W. Aha David W. Aha ICCBR-07 ICCBR-07 13 August 2007 13 August 2007 Belfast, Northern IrelandBelfast, Northern Ireland 32
1. Perceptions 2. Objectives 3. Survey 4. Findings 5. Interpretation
Categorizing Papers: AAAI-90 (Cohen, 1991)
Papers
Models(3 4)
Algs(5 6)
Systems(7 8)
Field3. Models (Define, extend, etc.)
4. Theorems/Proofs for models
5. Present algorithm(s)
6. Analyze algorithm(s)
7. Present system
8. Analyze aspects of system
Models Algs
Systems
25 43 36
14
3
37
Categories
Model-Centered: (M A) S Hybrid: (M A) SSystem-Centered: (M A) S
AAAI-90
Addressing Perceptions of Case-Based Reasoning Addressing Perceptions of Case-Based Reasoning David W. Aha David W. Aha ICCBR-07 ICCBR-07 13 August 2007 13 August 2007 Belfast, Northern IrelandBelfast, Northern Ireland 33
1. Perceptions 2. Objectives 3. Survey 4. Findings 5. Interpretation
Categorizing Papers: ECCBR-06
Papers
Models(3 4)
Algs(5 6)
Systems(7 8)
Field3. Models (Define, extend, etc.)
4. Theorems/Proofs for models
5. Present algorithm(s)
6. Analyze algorithm(s)
7. Present system
8. Analyze aspects of system
Models Algs
Systems
1 5 7
03
6
13
Categories
Model-Centered: (M A) S Hybrid: (M A) SSystem-Centered: (M A) S
ECCBR-06
Addressing Perceptions of Case-Based Reasoning Addressing Perceptions of Case-Based Reasoning David W. Aha David W. Aha ICCBR-07 ICCBR-07 13 August 2007 13 August 2007 Belfast, Northern IrelandBelfast, Northern Ireland 34
1. Perceptions 2. Objectives 3. Survey 4. Findings 5. Interpretation
Comparing MAD Categorizations of Papers
Models Algs
Systems
1 5 7
03
6
13
ECCBR-06
Models Algs
Systems
25 43 36
14
3
37
AAAI-90
Models Algs
Systems
3% 14% 19%
0%8%
17%
36%
Models Algs
Systems
17% 29% 24%
1%3%
2%
25%
25% M50% M
25% Hybrid6% Hybrid
Addressing Perceptions of Case-Based Reasoning Addressing Perceptions of Case-Based Reasoning David W. Aha David W. Aha ICCBR-07 ICCBR-07 13 August 2007 13 August 2007 Belfast, Northern IrelandBelfast, Northern Ireland 35
1. Perceptions 2. Objectives 3. Survey 4. Findings 5. Interpretation
Results: ECCBR-06
System-CenteredM M+A A M+S M+S+A S+A S Totals
Distribution by fields 3-8 1 5 7 0 3 6 13 35A Natural 1 2 3 0 1 2 7 16A Synthetic 0 0 1 0 0 1 1 3A Abstract 0 2 0 0 0 1 0 3A None 0 1 3 0 2 2 5 13B Natural 0 4 6 0 2 4 6 22B Synthetic 0 0 1 0 0 1 0 2B Abstract 0 1 0 0 0 0 0 1B None 1 0 0 0 1 1 7 10C Embedded 0 0 1 0 2 1 0 4C Not embedded 0 5 6 0 0 3 6 20C None 1 0 0 0 1 2 7 11D Demo 1 5 7 0 2 5 6 26D No demo 0 0 0 0 1 1 7 9E Expectations 0 4 4 0 1 4 6 19E No expectations 1 1 3 0 2 2 7 16
Field 11: Embedded TaskFields 12-14: DemoFields 15-18: Followup
Model-Centered Hybrid
Field 9: Example
Field 10: Eval Task
LegendA=AlgorithmsM=ModelsS=Systems
Contingency table
Addressing Perceptions of Case-Based Reasoning Addressing Perceptions of Case-Based Reasoning David W. Aha David W. Aha ICCBR-07 ICCBR-07 13 August 2007 13 August 2007 Belfast, Northern IrelandBelfast, Northern Ireland 36
1. Perceptions 2. Objectives 3. Survey 4. Findings 5. Interpretation
Results: ECCBR-06
M-C H S-CDistribution by fields 3-8 13 9 13
A Natural 6 3 7A Synthetic 1 1 1A Abstract 2 1 0A None 4 4 5B Natural 10 6 6B Synthetic 1 1 0B Abstract 1 0 0B None 1 2 7C Embedded 1 3 0C Not embedded 11 3 6C None 1 3 7D Demo 13 7 6D No demo 0 2 7E Expectations 8 5 6E No expectations 5 4 7
Field 11: Embedded TaskFields 12-14: DemoFields 15-18: Followup
Field 9: Example
Field 10: Eval Task
Legend
M-C: Model-Centered
H: Hybrid
S-C: System-Centered
Addressing Perceptions of Case-Based Reasoning Addressing Perceptions of Case-Based Reasoning David W. Aha David W. Aha ICCBR-07 ICCBR-07 13 August 2007 13 August 2007 Belfast, Northern IrelandBelfast, Northern Ireland 37
1. Perceptions 2. Objectives 3. Survey 4. Findings 5. Interpretation
Analysis: Comparing ECCBR-06 with AAAI-90
Source Model-Centered Hybrid System-CenteredECCBR-06 13 9 13AAAI-90 104 8 37
LegendA=AlgorithmsM=ModelsS=Systems
χ2(6)=24.1, p<0.006
χ2(2)=19.0, p<0.0001
Could this distribution of M-C, Hybrid, and S-C methodologies have arisen by chance, or does it reflect a real difference between ECCBR and AAAI?
• The ECCBR/AAAI distinction is not independent of the research methodology class
System-CenteredSource M M+A A M+S M+S+A S+A S
ECCBR-06 1 5 7 0 3 6 13AAAI-90 25 43 36 1 4 3 37
Model-Centered Hybrid
Addressing Perceptions of Case-Based Reasoning Addressing Perceptions of Case-Based Reasoning David W. Aha David W. Aha ICCBR-07 ICCBR-07 13 August 2007 13 August 2007 Belfast, Northern IrelandBelfast, Northern Ireland 38
1. Perceptions 2. Objectives 3. Survey 4. Findings 5. Interpretation
Analysis: Examining ECCBR-06
M-C H S-CDistribution by fields 3-8 13 9 13Any Example 9 5 8No Example 4 4 5
M-C H S-CDistribution by fields 3-8 13 9 13
Any evaluation 12 7 6No evaluation 1 2 7
χ2(2)=0.4, p>0.8
Unlike AAAI-90, the methodological choice of an example is independent of the paper’s class.
χ2(2)=7.0, p<0.003
The methodological choice of whether an evaluation was conducted is not independent of the paper’s class.
• Model-centered and hybrid papers include evaluations significantly more frequently than do system-centered papers.
Addressing Perceptions of Case-Based Reasoning Addressing Perceptions of Case-Based Reasoning David W. Aha David W. Aha ICCBR-07 ICCBR-07 13 August 2007 13 August 2007 Belfast, Northern IrelandBelfast, Northern Ireland 39
1. Perceptions 2. Objectives 3. Survey 4. Findings 5. Interpretation
Analysis: Examining ECCBR-06 (cont.)
M-C H S-CDistribution by fields 3-8 13 9 13Demonstration 13 7 6No demonstration 0 2 7
M-C H S-CDistribution by fields 3-8 13 9 13Expectations 8 5 6No expectations 5 4 7
χ2(2)=9.9, p<0.007
Like AAAI-90, Model-centered and hybrid papers are more likely than system-centered papers to include (any type of) performance assessment.
χ2(2)=0.6, p>0.7
Surprisingly, and unlike AAAI-90, model-centered and hybrid papers do not provide (any types of) expectations more frequently than do system-centered papers.
• Perhaps this warrants a follow-up analysis • Perhaps system-centered researchers make predictions not
derived from models, which would be dangerous, or perhaps they are simply not stating the models, which is more likely.
Addressing Perceptions of Case-Based Reasoning Addressing Perceptions of Case-Based Reasoning David W. Aha David W. Aha ICCBR-07 ICCBR-07 13 August 2007 13 August 2007 Belfast, Northern IrelandBelfast, Northern Ireland 40
1. Perceptions 2. Objectives 3. Survey 4. Findings 5. Interpretation
Patterns: Comparing ECCBR-06 with AAAI-90
PatternObserved frequencies
AAAI-90 ECCBR-06
p(M) 0.49 0.25
p(S) 0.30 0.61
p(SM) 0.03 0.08
p(SM) 0.89 0.86
p(No or abstract examples | M-C) 0.76 0.46
p(Test implementations | M-C) 0.33 0.92
p(Prediction/hypothesis) <0.21 0.53
p(Evaluation) 0.30 0.72
p(prediction/hypothesis | evaluation) 0.69
p(Negative, surprising, or probe) 0.16 0.25
Generous?
Generous!
Näive
Addressing Perceptions of Case-Based Reasoning Addressing Perceptions of Case-Based Reasoning David W. Aha David W. Aha ICCBR-07 ICCBR-07 13 August 2007 13 August 2007 Belfast, Northern IrelandBelfast, Northern Ireland 41
1. Perceptions 2. Objectives 3. Survey 4. Findings 5. Interpretation
ECCBR-06: Distinguishing Papers from Posters
““Now we tread on hallowed groundNow we tread on hallowed ground” - Anon” - Anon
Hypothesis: Reviewers are human and subjective. While there’s probably a trend that oral presentations show more “maturity” than do posters, exceptions exist and this trend is probably not significant.
Results of analysis: I was wrong…• …assuming the presentation/use of models is
indicative of a paper’s level of maturity
Addressing Perceptions of Case-Based Reasoning Addressing Perceptions of Case-Based Reasoning David W. Aha David W. Aha ICCBR-07 ICCBR-07 13 August 2007 13 August 2007 Belfast, Northern IrelandBelfast, Northern Ireland 42
1. Perceptions 2. Objectives 3. Survey 4. Findings 5. Interpretation
ECCBR-06
Models Algs
Systems
0 0 2
01
5
5
Posters (13)
Models Algs
Systems
1 5 5
02
1
8
Oral Papers (22)
Models Algs
Systems
0% 0% 15%
0%8%
38%
38%
Models Algs
Systems
5% 23% 23%
09%
5%
36%
15% M50% M
46% Hybrid14% Hybrid
Addressing Perceptions of Case-Based Reasoning Addressing Perceptions of Case-Based Reasoning David W. Aha David W. Aha ICCBR-07 ICCBR-07 13 August 2007 13 August 2007 Belfast, Northern IrelandBelfast, Northern Ireland 43
1. Perceptions 2. Objectives 3. Survey 4. Findings 5. Interpretation
ECCBR-06: Distinguishing Papers from Posters
System-CenteredM M+A A M+S M+S+A S+A S
Oral 1 5 5 0 2 1 8Poster 0 0 2 0 1 5 5
Model-Centered Hybrid
2(5)=9.3, p<0.1
The poster/paper designation of an accepted paper at ECCBR-06 was not independent of the paper’s class.
• Tentative conclusion: If you want your accepted paper to be an oral presentation, then present your work in the context of a model.
Distribution by fields 3-8 M-C H S-COral 11 3 8Poster 2 6 5
2(2)=6.0, p<0.05
SoSo: Will you think about this, and want to learn more? : Will you think about this, and want to learn more?
But maybe you are unconvinced…But maybe you are unconvinced…
Addressing Perceptions of Case-Based Reasoning Addressing Perceptions of Case-Based Reasoning David W. Aha David W. Aha ICCBR-07 ICCBR-07 13 August 2007 13 August 2007 Belfast, Northern IrelandBelfast, Northern Ireland 44
1. Perceptions 2. Objectives 3. Survey 4. Findings 5. Interpretation
ECCBR-06: Distinguishing Papers from PostersOral Poster
Example 14 8None 8 5
Oral PosterEval 16 9None 6 4
Oral PosterDemo 17 9None 5 4
Oral PosterDemo 11 8None 11 5
Nothing else (so far) distinguishes papers from posters
2(1)=0.02, p>0.9
2(1)=0.05, p>0.8
2(1)=0.28, p>0.5
2(1)=0.44, p>0.5
Addressing Perceptions of Case-Based Reasoning Addressing Perceptions of Case-Based Reasoning David W. Aha David W. Aha ICCBR-07 ICCBR-07 13 August 2007 13 August 2007 Belfast, Northern IrelandBelfast, Northern Ireland 45
1. Perceptions 2. Objectives 3. Survey 4. Findings 5. Interpretation
ECCBR-06: Distinguishing Best Paper Nominees?
• But there were only 5• Future work: Analyze after adding 9 ICCBR-07 nominees
Addressing Perceptions of Case-Based Reasoning Addressing Perceptions of Case-Based Reasoning David W. Aha David W. Aha ICCBR-07 ICCBR-07 13 August 2007 13 August 2007 Belfast, Northern IrelandBelfast, Northern Ireland 46
1. Perceptions 2. Objectives 3. Survey 4. Findings 5. Interpretation
Hypotheses revisited
Summary
1. Unlike AAAI-90, CBR research is not dominated by both model-centered and system-centered methodologies Dominated only by system-centered papers
2. CBR research suffers from similar methodological problems• Model- and system-centered papers differ in whether they:
Conduct evaluations Assess performance Describe expectations
3. The class of a paper in the MAD framework distinguishes Oral vs. poster presentations Best paper nominees from others
Addressing Perceptions of Case-Based Reasoning Addressing Perceptions of Case-Based Reasoning David W. Aha David W. Aha ICCBR-07 ICCBR-07 13 August 2007 13 August 2007 Belfast, Northern IrelandBelfast, Northern Ireland 47
1. Perceptions 2. Objectives 3. Survey 4. Findings 5. Interpretation
Outline
1.Perceptions2.Objectives3.Survey4.Findings5.Interpretation
• A new case• Caveats• Next steps
Addressing Perceptions of Case-Based Reasoning Addressing Perceptions of Case-Based Reasoning David W. Aha David W. Aha ICCBR-07 ICCBR-07 13 August 2007 13 August 2007 Belfast, Northern IrelandBelfast, Northern Ireland 48
1. Perceptions 2. Objectives 3. Survey 4. Findings 5. Interpretation
A new case…hopefully
1. Retrieve
MAD Framework
(Cohen, 1991)
Case Base
2. Reuse
MAD Framework
How can we assess CBR R&D Methodologies?
AAAI-90
ECCBR-06
Frameworks for assessing AI R&D Methods
MAD mixed methodology
Today’s Results
3. Revise4. Retain
(Aha, 2007?)
Addressing Perceptions of Case-Based Reasoning Addressing Perceptions of Case-Based Reasoning David W. Aha David W. Aha ICCBR-07 ICCBR-07 13 August 2007 13 August 2007 Belfast, Northern IrelandBelfast, Northern Ireland 49
1. Perceptions 2. Objectives 3. Survey 4. Findings 5. Interpretation
Many of Cohen’s (1991) Points for AI Apply to CBR
• A goal of AI research is to develop science & technology to support the design and analysis of intelligent systems
• Model- and system-centered methods are complementary– Model-centered researchers typically develop algorithms for simpler
problems, but with deeper analysis, expectations, and demos– System-centered researchers typically build large systems to solve
realistic problems, but w/o explicit expectations, analyses or demos– See the MAD methodology (Cohen, 1991)
• Models are used to derive hypotheses & expectations• Few systems merit attention on the basis of existence alone• It is impossible to evaluate a system without predictions• Creating benchmarks will not fix AI’s methodological problems
Addressing Perceptions of Case-Based Reasoning Addressing Perceptions of Case-Based Reasoning David W. Aha David W. Aha ICCBR-07 ICCBR-07 13 August 2007 13 August 2007 Belfast, Northern IrelandBelfast, Northern Ireland 50
1. Perceptions 2. Objectives 3. Survey 4. Findings 5. Interpretation
Caveat #1: No cases for ECCBR-90, AAAI-06, etc.
AAAI-90 ECCBR-06
?!
Note: CBRW-91’s R&D methods differ greatly from ECCBR-06’s
Addressing Perceptions of Case-Based Reasoning Addressing Perceptions of Case-Based Reasoning David W. Aha David W. Aha ICCBR-07 ICCBR-07 13 August 2007 13 August 2007 Belfast, Northern IrelandBelfast, Northern Ireland 51
1. Perceptions 2. Objectives 3. Survey 4. Findings 5. Interpretation
Caveats (that could influence the results)
• The case base is small to compare CBR with AI– AAAI has changed since 1990! Compare to AAAI-07 (and IAAI-07!)– ECCBR’s differences may reflect a higher acceptance rate– No analysis of other subfields; how does CBR relate?
• No reliability data: Subjective classification of papers!– e.g., I gave up distinguishing “informal” and “no” system analysis– “Volunteers welcome!” ((Cohen, 1991), which I repeat)
• Not representative of CBR community’s work?– Is ECCBR-06 an aberration? (Wait until next year?)– Perhaps we publish model-centered work elsewhere (e.g., COLT)– ECCBR readers’ expectations match ECCBR-06’s class distribution?
• Lack of page-space places limitations on what is presented– e.g., hypotheses not made explicit
• Science/research works iteratively– Earlier exploratory research (e.g., involving surprising results) resulted in
changes to the model, algorithm, or system; we see only the end result
Addressing Perceptions of Case-Based Reasoning Addressing Perceptions of Case-Based Reasoning David W. Aha David W. Aha ICCBR-07 ICCBR-07 13 August 2007 13 August 2007 Belfast, Northern IrelandBelfast, Northern Ireland 52
1. Perceptions 2. Objectives 3. Survey 4. Findings 5. Interpretation
Caveat finale: I’ve ignored many issues!
• Relation of evaluation to:– Investigating the claims, if any– The predictions, if any
• Results of formal analysis – e.g., average- vs. worst-case
• Quality of the empirical evaluation (e.g., scale)• Significance of evaluation’s results• …
Addressing Perceptions of Case-Based Reasoning Addressing Perceptions of Case-Based Reasoning David W. Aha David W. Aha ICCBR-07 ICCBR-07 13 August 2007 13 August 2007 Belfast, Northern IrelandBelfast, Northern Ireland 53
1. Perceptions 2. Objectives 3. Survey 4. Findings 5. Interpretation
Other potential applications of the MAD framework
Next Steps
• A decision aid for predicting whether an accepted paper should be categorized as an oral presentation– Or to ensure diversity among the presentations
• Selection of best paper nominees• To assist reviewers with spotting novelty and/or
expected characteristics in a submission• Explaining/characterizing CBR methodologies to others
Addressing Perceptions of Case-Based Reasoning Addressing Perceptions of Case-Based Reasoning David W. Aha David W. Aha ICCBR-07 ICCBR-07 13 August 2007 13 August 2007 Belfast, Northern IrelandBelfast, Northern Ireland 54
1. Perceptions 2. Objectives 3. Survey 4. Findings 5. Interpretation
Next Steps
Conjecture: CBR has shifted from heavily model-centered to heavily system-centered research
– Needs analysis to provide evidence, but it’s obvious– AI needs both to achieve balance– We should consider this in our reviewing processes
Milestones: We are halfway to our 25th anniversary– A time for reflection– We should make it our goal that, by the 25th, we will
achieve a better balance
Roger Schank
David Leake
Agnar Aamodt(1st ICCBR Co-Chair)
Addressing Perceptions of Case-Based Reasoning Addressing Perceptions of Case-Based Reasoning David W. Aha David W. Aha ICCBR-07 ICCBR-07 13 August 2007 13 August 2007 Belfast, Northern IrelandBelfast, Northern Ireland 55
1. Perceptions 2. Objectives 3. Survey 4. Findings 5. Interpretation
Next Steps
• Refine our research methodologies– Ensure program PCs reflects them broadly – A CBR-related journal could assist (by providing feedback)
• Change perceptions by improving communication– WWW site (maybe AAAI, but probably not, as we have
different specifications in mind)– Co-locate our conferences with others
• e.g., IJCAI, ECAI, IR conference
• Reach out in new ways (e.g., Video)
Addressing Perceptions of Case-Based Reasoning Addressing Perceptions of Case-Based Reasoning David W. Aha David W. Aha ICCBR-07 ICCBR-07 13 August 2007 13 August 2007 Belfast, Northern IrelandBelfast, Northern Ireland 56
1. Perceptions 2. Objectives 3. Survey 4. Findings 5. Interpretation
Next Steps
AAAI-07 AI Video Competition (Co-Chairs: Thrun & Aha)• Goal: Encourage the interest of prospective students
• Results (see aivideo.org)– Quick funding– 30 Submissions (in a short time period)– Large turnout for awards ceremony– Invited, and will be held, for AAAI-08
• Two CBR videos1. k-nearest Neighbor Classification (Antal van den Bosch)2. Invisible Threats (Rosina Weber & André Testa)
Addressing Perceptions of Case-Based Reasoning Addressing Perceptions of Case-Based Reasoning David W. Aha David W. Aha ICCBR-07 ICCBR-07 13 August 2007 13 August 2007 Belfast, Northern IrelandBelfast, Northern Ireland 57
1. Perceptions 2. Objectives 3. Survey 4. Findings 5. Interpretation
Takeaway Message
And now: The 25 Re’s!!!• e.g., Reuse, Regurgitate, Repulse, …• Beats previous record of 13 by 12 (Bridge, ICCBR-05)
Just kidding – I’ll spare you
Bridge Gauntlet: 13 Re’s!
Derek Bridge
Addressing Perceptions of Case-Based Reasoning Addressing Perceptions of Case-Based Reasoning David W. Aha David W. Aha ICCBR-07 ICCBR-07 13 August 2007 13 August 2007 Belfast, Northern IrelandBelfast, Northern Ireland 58
1. Perceptions 2. Objectives 3. Survey 4. Findings 5. Interpretation
Concluding remarks
1. Goal: Raise awareness of CBR R&D methodologies
4. MAD framework has several potential uses
†This presentation is dedicated to my late colleague John Urban and the late great Donald Michie, for their support.
5. We have work to do to address perceptions
3. Paper/poster distinction relates to use of models
2. Current CBR methods are unlike traditional AI’s• CBR’s is not dominated by model-centered work• We must beware system-centered limitations• But there’s much more to learn
Thanks for listening