towards a conceptual model of computational...
TRANSCRIPT
![Page 1: Towards A Conceptual Model of Computational Sustainabilitycomputational-sustainability.cis.cornell.edu/PPT/Diett... · 2014-09-02 · Aral Sea Fisheries (Conrad, et al.) Bird Migration](https://reader034.vdocument.in/reader034/viewer/2022042416/5f316c3bb5594e600562b925/html5/thumbnails/1.jpg)
Towards A Conceptual
Model of Computational
Sustainability Tom Dietterich
Oregon State University
ICS Seminar
1
![Page 2: Towards A Conceptual Model of Computational Sustainabilitycomputational-sustainability.cis.cornell.edu/PPT/Diett... · 2014-09-02 · Aral Sea Fisheries (Conrad, et al.) Bird Migration](https://reader034.vdocument.in/reader034/viewer/2022042416/5f316c3bb5594e600562b925/html5/thumbnails/2.jpg)
Computational
Sustainability
ICS Seminar 2
The study of computational
methods that can contribute to
the sustainable management
of the earth’s ecosystems
biological
social
economic
Data Models Policies
Data
Integration
Data
Interpretation
Model Fitting
Policy
Optimization
Sensor
Placement
Policy
Execution
Policy
Explanation
Objective
Formulation
![Page 3: Towards A Conceptual Model of Computational Sustainabilitycomputational-sustainability.cis.cornell.edu/PPT/Diett... · 2014-09-02 · Aral Sea Fisheries (Conrad, et al.) Bird Migration](https://reader034.vdocument.in/reader034/viewer/2022042416/5f316c3bb5594e600562b925/html5/thumbnails/3.jpg)
Example Research Efforts
Objectives
detection probability
improving model accuracy
improving causal understanding
improving policy effectiveness
Active Learning for eBird (Damoulas & Dilkina)
Others?
ICS Seminar 3
Sensor
Placement
![Page 4: Towards A Conceptual Model of Computational Sustainabilitycomputational-sustainability.cis.cornell.edu/PPT/Diett... · 2014-09-02 · Aral Sea Fisheries (Conrad, et al.) Bird Migration](https://reader034.vdocument.in/reader034/viewer/2022042416/5f316c3bb5594e600562b925/html5/thumbnails/4.jpg)
Data Interpretation
Insect identification for population counting (Dietterich, Todorovic, Lin, et al.)
Freshwater macro-invertebrates
Rice pests
Raw data: images
Interpreted data: Count by species
Understanding tree swallow roosts from Doppler radar (Sheldon, et al)
Raw data: Doppler radar images
Interpreted data: Location and approx. size of swallow roosts over whole US
Estimating Bird Migration from Doppler Radar (BirdCast project)
Sensor Network Data Cleaning (Dereszynski & Dietterich)
ICS Seminar 4
Data
Interpretation
Sensor
Placement
image: Qing Yao
Florida
![Page 5: Towards A Conceptual Model of Computational Sustainabilitycomputational-sustainability.cis.cornell.edu/PPT/Diett... · 2014-09-02 · Aral Sea Fisheries (Conrad, et al.) Bird Migration](https://reader034.vdocument.in/reader034/viewer/2022042416/5f316c3bb5594e600562b925/html5/thumbnails/5.jpg)
Rice Pest Project
Working with Dr. Qing Yao from Zhejiang Sci-Tech
University
Challenge: Classifying overlapping specimens
ICS Seminar 5
Data
Interpretation
Sensor
Placement
Species Count
Nilaparvata
lugens( 12
Sogatella furcifera 8
Laodelphax
striatellus 0
Cnaphalocrocis
medinalis 0
Chilo suppressalis 45
Sesamia inferens 18
image: Qing Yao
![Page 6: Towards A Conceptual Model of Computational Sustainabilitycomputational-sustainability.cis.cornell.edu/PPT/Diett... · 2014-09-02 · Aral Sea Fisheries (Conrad, et al.) Bird Migration](https://reader034.vdocument.in/reader034/viewer/2022042416/5f316c3bb5594e600562b925/html5/thumbnails/6.jpg)
Data Integration eBird Reference Data Set + BirdCast
Landsat (30m; monthly)
land cover type
MODIS (500m; daily/weekly)
land cover type
“greening” index
Census (every 10 years)
human population density
housing density and occupation
Interpolated weather data
rain, snow, solar radiation, wind speed & direction, humidity
Integrated weather data (daily)
warming degree days
Digital elevation model (rarely changes)
elevation, slope, aspect
ICS Seminar 6
Data
Integration
Data
Interpretation
Sensor
Placement
Landsat NDVI:
http://ivm.cr.usgs.gov/viewer/
![Page 7: Towards A Conceptual Model of Computational Sustainabilitycomputational-sustainability.cis.cornell.edu/PPT/Diett... · 2014-09-02 · Aral Sea Fisheries (Conrad, et al.) Bird Migration](https://reader034.vdocument.in/reader034/viewer/2022042416/5f316c3bb5594e600562b925/html5/thumbnails/7.jpg)
Model Building and Model
Integration STEM: bird species distribution models (Fink, et al.)
OD-BRT: Occupancy Models parameterized via
boosted regression trees (Hutchinson, et al.)
ODE: Occupancy, Detection & Expertise (Wong, et
al.)
Discovering plant communities from field
observational data (Lettkeman & Dietterich)
Multiple-Species SDMs (Wong, Dietterich, et al.)
Moth Emergence Model (Sheldon, Dietterich, et al.)
Aral Sea Fisheries (Conrad, et al.)
Bird Migration Model: Collective Graphical Model
(Sheldon)
Oregon Centennial Fire Model (Montgomery, et al.)
ICS Seminar 7
Data
Integration
Data
Interpretation
Model Fitting
Sensor
Placement
![Page 8: Towards A Conceptual Model of Computational Sustainabilitycomputational-sustainability.cis.cornell.edu/PPT/Diett... · 2014-09-02 · Aral Sea Fisheries (Conrad, et al.) Bird Migration](https://reader034.vdocument.in/reader034/viewer/2022042416/5f316c3bb5594e600562b925/html5/thumbnails/8.jpg)
Example Fitted Model: STEM
Model of Bird Species Distribution
8 slide courtesy of Daniel Fink
Indigo Bunting
ICS Seminar
![Page 9: Towards A Conceptual Model of Computational Sustainabilitycomputational-sustainability.cis.cornell.edu/PPT/Diett... · 2014-09-02 · Aral Sea Fisheries (Conrad, et al.) Bird Migration](https://reader034.vdocument.in/reader034/viewer/2022042416/5f316c3bb5594e600562b925/html5/thumbnails/9.jpg)
ICS Seminar
Policy Optimization
9
Data
Integration
Data
Interpretation
Model Fitting
Policy
Optimization
Sensor
Placement Halibut Fisheries (Ermon, Conrad et al)
Wildfire Management :
LetBurn vs. Suppress (Montgomery, Houtman, et al.)
Spencer & Shmoys
Invasive Species Management
Tamarisk: (Albers, Hall, Taleghan, Dietterich)
Spencer & Shmoys
Red Cockaded Woodpecker (Sheldon, Finseth, et
al.)
Johne's Diease (Toese, et al.)
+++
![Page 10: Towards A Conceptual Model of Computational Sustainabilitycomputational-sustainability.cis.cornell.edu/PPT/Diett... · 2014-09-02 · Aral Sea Fisheries (Conrad, et al.) Bird Migration](https://reader034.vdocument.in/reader034/viewer/2022042416/5f316c3bb5594e600562b925/html5/thumbnails/10.jpg)
Data
Integration
Data
Interpretation
Model Fitting
Policy
Optimization
Sensor
Placement
Objective
Formulation
Objective
Formulation
10
Any?
ICS Seminar
![Page 11: Towards A Conceptual Model of Computational Sustainabilitycomputational-sustainability.cis.cornell.edu/PPT/Diett... · 2014-09-02 · Aral Sea Fisheries (Conrad, et al.) Bird Migration](https://reader034.vdocument.in/reader034/viewer/2022042416/5f316c3bb5594e600562b925/html5/thumbnails/11.jpg)
Policy
Explanation
Wildfire Management (McGregor)
Invasive Species (Taleghan)
Others?
ICS Seminar 11
Data
Integration
Data
Interpretation
Model Fitting
Policy
Optimization
Sensor
Placement
Policy
Explanation
Objective
Formulation
![Page 12: Towards A Conceptual Model of Computational Sustainabilitycomputational-sustainability.cis.cornell.edu/PPT/Diett... · 2014-09-02 · Aral Sea Fisheries (Conrad, et al.) Bird Migration](https://reader034.vdocument.in/reader034/viewer/2022042416/5f316c3bb5594e600562b925/html5/thumbnails/12.jpg)
Policy Execution
RCW?
Fisheries?
Invasives?
ICS Seminar 12
Data
Integration
Data
Interpretation
Model Fitting
Policy
Optimization
Sensor
Placement
Policy
Execution
Policy
Explanation
Objective
Formulation
![Page 13: Towards A Conceptual Model of Computational Sustainabilitycomputational-sustainability.cis.cornell.edu/PPT/Diett... · 2014-09-02 · Aral Sea Fisheries (Conrad, et al.) Bird Migration](https://reader034.vdocument.in/reader034/viewer/2022042416/5f316c3bb5594e600562b925/html5/thumbnails/13.jpg)
Learning Rules from
Incomplete Examples via a
Probabilistic Mention
Model
Mohammad Shahed Sorower, Janardhan Rao Doppa,
Thomas G. Dietterich
13 ICS Seminar
![Page 14: Towards A Conceptual Model of Computational Sustainabilitycomputational-sustainability.cis.cornell.edu/PPT/Diett... · 2014-09-02 · Aral Sea Fisheries (Conrad, et al.) Bird Migration](https://reader034.vdocument.in/reader034/viewer/2022042416/5f316c3bb5594e600562b925/html5/thumbnails/14.jpg)
Motivation and Goal
Text documents
Information
Extractor
Extracted facts
Rule learner
KB of rules
• Goal:
Induce general rules by reading about concrete facts o Ex: gameWinner(G,T1) :- teaminGame(G,T1), teaminGame(G,T2), gameLoser(G,T2)
ICS Seminar 14
![Page 15: Towards A Conceptual Model of Computational Sustainabilitycomputational-sustainability.cis.cornell.edu/PPT/Diett... · 2014-09-02 · Aral Sea Fisheries (Conrad, et al.) Bird Migration](https://reader034.vdocument.in/reader034/viewer/2022042416/5f316c3bb5594e600562b925/html5/thumbnails/15.jpg)
Challenges in Learning Rules from
Natural Text
Extracted ground facts are highly incomplete
Only a very small part of the “whole truth” is mentioned in a document
Even less is successfully extracted by NLP methods
Incompleteness is not “missing at random”
Speaker seeks to achieve communication goals concisely
Mention “newsworthy” or “surprising” facts
Let the reader fill in the rest by applying background knowledge
ICS Seminar 15
![Page 16: Towards A Conceptual Model of Computational Sustainabilitycomputational-sustainability.cis.cornell.edu/PPT/Diett... · 2014-09-02 · Aral Sea Fisheries (Conrad, et al.) Bird Migration](https://reader034.vdocument.in/reader034/viewer/2022042416/5f316c3bb5594e600562b925/html5/thumbnails/16.jpg)
Example “Given the commanding lead of Kansas city on the road, the Denver
Broncos’ 14-10 victory surprised many”
TeamInGame(g1,KansasCity)
TeamInGame(g1, DenverBroncos)
GameWinner(g1, DenverBroncos)
GameTeamScore(g1, DenverBroncos,14)
GameTeamScore(g1, KansasCity, 10)
AwayTeam(g1, KansasCity)
Does not mention
GameLoser(g1,KansasCity)
HomeTeam(g1,DenverBroncos)
Hard to learn rules such as
Winner not Loser
HomeTeam not AwayTeam
Winner is team that scores the most points
ICS Seminar 16
![Page 17: Towards A Conceptual Model of Computational Sustainabilitycomputational-sustainability.cis.cornell.edu/PPT/Diett... · 2014-09-02 · Aral Sea Fisheries (Conrad, et al.) Bird Migration](https://reader034.vdocument.in/reader034/viewer/2022042416/5f316c3bb5594e600562b925/html5/thumbnails/17.jpg)
Example 2:
“Ahmed Said Khadr, an Egyptian-born Canadian, was killed last October in
Pakistan”
BornIn(Khadr, Egypt)
CitizenOf(Khadr, Canada)
KillingEvent(e1)
Location(e1, Pakistan)
Victim(e1, Khadr)
How can we learn the rule
CitizenOf(P,C) :- BornIn(P,C) ???
Most articles only mention both CitizenShip and BirthPlace when they are
not equal
Pilot Study corpus: 23 BirthPlace mentions of which 14 violate the rule
ICS Seminar 17
![Page 18: Towards A Conceptual Model of Computational Sustainabilitycomputational-sustainability.cis.cornell.edu/PPT/Diett... · 2014-09-02 · Aral Sea Fisheries (Conrad, et al.) Bird Migration](https://reader034.vdocument.in/reader034/viewer/2022042416/5f316c3bb5594e600562b925/html5/thumbnails/18.jpg)
Occupancy-Detection Model
Yit Zi
i=1,…,M
t=1,…,T
Xi Wit
oi dit
Key Idea: Explicit model of the observation process
ICS Seminar 18
MacKenzie, et al, 2006
![Page 19: Towards A Conceptual Model of Computational Sustainabilitycomputational-sustainability.cis.cornell.edu/PPT/Diett... · 2014-09-02 · Aral Sea Fisheries (Conrad, et al.) Bird Migration](https://reader034.vdocument.in/reader034/viewer/2022042416/5f316c3bb5594e600562b925/html5/thumbnails/19.jpg)
Idea: Learn an explicit model of the
observation process = “Mention
Model”
ICS Seminar
Facts and Rules Believed by Writer
Mention Model
Generated Document Genera
tive P
rocess
19
![Page 20: Towards A Conceptual Model of Computational Sustainabilitycomputational-sustainability.cis.cornell.edu/PPT/Diett... · 2014-09-02 · Aral Sea Fisheries (Conrad, et al.) Bird Migration](https://reader034.vdocument.in/reader034/viewer/2022042416/5f316c3bb5594e600562b925/html5/thumbnails/20.jpg)
Learn Rules by Probabilistic
Inversion of the Mention Model
ICS Seminar
Facts and Rules Believed by Writer
Mention Model
Generated Document
Learn
ing P
rocess
20
![Page 21: Towards A Conceptual Model of Computational Sustainabilitycomputational-sustainability.cis.cornell.edu/PPT/Diett... · 2014-09-02 · Aral Sea Fisheries (Conrad, et al.) Bird Migration](https://reader034.vdocument.in/reader034/viewer/2022042416/5f316c3bb5594e600562b925/html5/thumbnails/21.jpg)
Mention Model:
Grice’s Maxims of Cooperative
Conversation
Be Truthful
Do not say things you believe are false
Do not omit things that would lead the hearer to believe
falsehoods [Added]
Quantity of Information
Say as much as is necessary
Do not say more than is necessary
Be Relevant
Be Clear
ICS Seminar 21
![Page 22: Towards A Conceptual Model of Computational Sustainabilitycomputational-sustainability.cis.cornell.edu/PPT/Diett... · 2014-09-02 · Aral Sea Fisheries (Conrad, et al.) Bird Migration](https://reader034.vdocument.in/reader034/viewer/2022042416/5f316c3bb5594e600562b925/html5/thumbnails/22.jpg)
Formalization
Reader believes K, is told F, and will infer G: 𝐾, 𝑀𝑒𝑛𝑡𝑖𝑜𝑛(𝐹) ⊢𝑟𝑒𝑎𝑑𝑒𝑟 𝐺
Mention true facts:
𝐹 ⇒ 𝑀𝑒𝑛𝑡𝑖𝑜𝑛(𝐹) [with some probability]
Don’t mention facts that can be inferred: 𝑀𝑒𝑛𝑡𝑖𝑜𝑛 𝐹 ∧ 𝐺 ∧ (𝐾, 𝑀𝑒𝑛𝑡𝑖𝑜𝑛(𝐹) ⊢𝑟𝑒𝑎𝑑𝑒𝑟 𝐺) ⇒ ¬𝑀𝑒𝑛𝑡𝑖𝑜𝑛(𝐺)
Mention facts needed to prevent incorrect inferences 𝑀𝑒𝑛𝑡𝑖𝑜𝑛 𝐹 ∧ ¬𝐺 ∧ 𝐻 ∧ 𝐾, 𝑀𝑒𝑛𝑡𝑖𝑜𝑛 𝐹 ⊢𝑟𝑒𝑎𝑑𝑒𝑟 𝐺 ∧ (𝐾, 𝑀𝑒𝑛𝑡𝑖𝑜𝑛 𝐹
∧ 𝑀𝑒𝑛𝑡𝑖𝑜𝑛 𝐻 ⊢𝑟𝑒𝑎𝑑𝑒𝑟 ¬𝐺) ⇒ 𝑀𝑒𝑛𝑡𝑖𝑜𝑛(𝐻)
ICS Seminar 22
![Page 23: Towards A Conceptual Model of Computational Sustainabilitycomputational-sustainability.cis.cornell.edu/PPT/Diett... · 2014-09-02 · Aral Sea Fisheries (Conrad, et al.) Bird Migration](https://reader034.vdocument.in/reader034/viewer/2022042416/5f316c3bb5594e600562b925/html5/thumbnails/23.jpg)
Implementation in Markov Logic
ICS Seminar
Facts and Rules believed by Writer:
𝑤0: 𝐹𝑎𝑐𝑡_𝐹 𝑥 ⇒ 𝐹𝑎𝑐𝑡_𝐺 𝑥
𝐹𝑎𝑐𝑡_𝐹 𝑎 , 𝐹𝑎𝑐𝑡_𝐺(𝑎)
𝐹𝑎𝑐𝑡_𝐹 𝑏 , 𝐹𝑎𝑐𝑡_𝑛𝑜𝑡𝐺(𝑏)
Gricean Axioms:
𝑤1: 𝐹𝑎𝑐𝑡_𝐹 𝑥 ⇒ 𝑀𝑒𝑛𝑡𝑖𝑜𝑛_𝐹 𝑥
𝑤2: 𝐹𝑎𝑐𝑡_𝐺 𝑥 ⇒ 𝑀𝑒𝑛𝑡𝑖𝑜𝑛_𝐺 𝑥
𝑤3: 𝑀𝑒𝑛𝑡𝑖𝑜𝑛_𝐹 𝑥 ∧ 𝐹𝑎𝑐𝑡_𝐺 𝑥 ⇒ ¬𝑀𝑒𝑛𝑡𝑖𝑜𝑛_𝐺 𝑥
𝑤4: 𝑀𝑒𝑛𝑡𝑖𝑜𝑛_𝐹 𝑥 ∧ 𝐹𝑎𝑐𝑡_𝑛𝑜𝑡𝐺 𝑥 ⇒ 𝑀𝑒𝑛𝑡𝑖𝑜𝑛_𝑛𝑜𝑡𝐺 𝑥
Generated Document 1: (cost 𝑤0 + 𝑤2)
𝑀𝑒𝑛𝑡𝑖𝑜𝑛_𝐹 𝑎 , 𝑀𝑒𝑛𝑡𝑖𝑜𝑛_𝐹 𝑏 , 𝑀𝑒𝑛𝑡𝑖𝑜𝑛_𝑛𝑜𝑡𝐺 𝑏 , ¬𝑀𝑒𝑛𝑡𝑖𝑜𝑛_𝐺 𝑎
Generated Document 2: (cost w0 + 𝑤3)
𝑀𝑒𝑛𝑡𝑖𝑜𝑛_𝐹 𝑎 , 𝑀𝑒𝑛𝑡𝑖𝑜𝑛_𝐺 𝑎 , 𝑀𝑒𝑛𝑡𝑖𝑜𝑛_𝐹 𝑏 , 𝑀𝑒𝑛𝑡𝑖𝑜𝑛_𝑛𝑜𝑡𝐺(𝑏)
23
![Page 24: Towards A Conceptual Model of Computational Sustainabilitycomputational-sustainability.cis.cornell.edu/PPT/Diett... · 2014-09-02 · Aral Sea Fisheries (Conrad, et al.) Bird Migration](https://reader034.vdocument.in/reader034/viewer/2022042416/5f316c3bb5594e600562b925/html5/thumbnails/24.jpg)
Inference During Reading
ICS Seminar
Facts and Rules believed by Writer (cost 𝑤0 + 𝑤2):
𝑤0: 𝐹𝑎𝑐𝑡_𝐹 𝑥 ⇒ 𝐹𝑎𝑐𝑡_𝐺 𝑥
Gricean Axioms:
𝑤1: 𝐹𝑎𝑐𝑡_𝐹 𝑥 ⇒ 𝑀𝑒𝑛𝑡𝑖𝑜𝑛_𝐹 𝑥
𝑤2: 𝐹𝑎𝑐𝑡_𝐺 𝑥 ⇒ 𝑀𝑒𝑛𝑡𝑖𝑜𝑛_𝐺 𝑥
𝑤3: 𝑀𝑒𝑛𝑡𝑖𝑜𝑛_𝐹 𝑥 ∧ 𝐹𝑎𝑐𝑡_𝐺 𝑥 ⇒ ¬𝑀𝑒𝑛𝑡𝑖𝑜𝑛_𝐺 𝑥
𝑤4: 𝑀𝑒𝑛𝑡𝑖𝑜𝑛_𝐹 𝑥 ∧ 𝐹𝑎𝑐𝑡_𝑛𝑜𝑡𝐺 𝑥 ⇒ 𝑀𝑒𝑛𝑡𝑖𝑜𝑛_𝑛𝑜𝑡𝐺 𝑥
Observed Document:
𝑀𝑒𝑛𝑡𝑖𝑜𝑛_𝐹 𝑎 , 𝑀𝑒𝑛𝑡𝑖𝑜𝑛_𝐹 𝑏 , 𝑀𝑒𝑛𝑡𝑖𝑜𝑛_𝑛𝑜𝑡𝐺 𝑏 , ¬𝑀𝑒𝑛𝑡𝑖𝑜𝑛_𝐺(𝑎)
𝐹𝑎𝑐𝑡_𝐹 𝑎 ,
𝐹𝑎𝑐𝑡_𝐹 𝑏 , 𝐹𝑎𝑐𝑡_𝑛𝑜𝑡𝐺(𝑏) 𝐹𝑎𝑐𝑡_𝐺(𝑎)
𝑀𝑒𝑛𝑡𝑖𝑜𝑛_𝐹 𝑥 ⇒ 𝐹𝑎𝑐𝑡_𝐹 𝑥 ; 𝑀𝑒𝑛𝑡𝑖𝑜𝑛_𝑛𝑜𝑡𝐹 𝑥 ⇒ 𝐹𝑎𝑐𝑡_𝑛𝑜𝑡𝐹(𝑥)
𝑀𝑒𝑛𝑡𝑖𝑜𝑛_𝐺 𝑥 ⇒ 𝐹𝑎𝑐𝑡_𝐺 𝑥 ; 𝑀𝑒𝑛𝑡𝑖𝑜𝑛_𝑛𝑜𝑡𝐺 𝑥 ⇒ 𝐹𝑎𝑐𝑡_𝑛𝑜𝑡𝐺(𝑥)
24
![Page 25: Towards A Conceptual Model of Computational Sustainabilitycomputational-sustainability.cis.cornell.edu/PPT/Diett... · 2014-09-02 · Aral Sea Fisheries (Conrad, et al.) Bird Migration](https://reader034.vdocument.in/reader034/viewer/2022042416/5f316c3bb5594e600562b925/html5/thumbnails/25.jpg)
Rule Learning Inputs:
Rule templates
Extracted mentions from documents
Outputs:
Weighted rules expressed as Markov Logic knowledge base
Algorithm:
Generate all possible rules from the templates
Compute # of supporting instances for each rule (on extracted mentions) and
keep the top 10 best scoring rules for each head predicate
Generate the Gricean rules from these candidate rules
Apply the EM algorithm to learn the weights on the fact rules and the Gricean
rules
ICS Seminar 25
![Page 26: Towards A Conceptual Model of Computational Sustainabilitycomputational-sustainability.cis.cornell.edu/PPT/Diett... · 2014-09-02 · Aral Sea Fisheries (Conrad, et al.) Bird Migration](https://reader034.vdocument.in/reader034/viewer/2022042416/5f316c3bb5594e600562b925/html5/thumbnails/26.jpg)
Experiment 1: Synthetic Data Synthetic data set
NFL games generated from true rules and ground truth
Two sets of correlated predicates:
GameWinner, GameLoser, GameTeamScore
GameHomeTeam, GameAwayTeam
Choose one literal from each set and mention it
Mention each of the other literals with probability 1 − 𝑞
Experiment
Train on data with 58% of literals missing (𝑞 = 0.97)
Test on data with varying amounts of literals missing
ICS Seminar
40%
50%
60%
70%
80%
90%
100%
0 10 20 30 40 50 60 70
% l
itera
ls c
orr
ectl
y p
red
icte
d
% test set literals missing
Inverse Gricean Inference
No Inference
26
![Page 27: Towards A Conceptual Model of Computational Sustainabilitycomputational-sustainability.cis.cornell.edu/PPT/Diett... · 2014-09-02 · Aral Sea Fisheries (Conrad, et al.) Bird Migration](https://reader034.vdocument.in/reader034/viewer/2022042416/5f316c3bb5594e600562b925/html5/thumbnails/27.jpg)
Experiment 2: Real Training;
Synthetic Test Data from BBN Extractions 12/16/10
D1: NFL BBN_training
D2: NFL BBN_robustness
Both data sets “repaired” using
learned integrity constraints
Delete literals in all possible ways to
satisfy the integrity constraints
Remove duplicates.
Data set sizes:
D1: 203 records
D2: 56 records
Test set: 100 examples manually
created from ground-truth NFL
database to cover all missingness
scenarios
ICS Seminar
Win/Lose
Dataset
Both
Missing
One
Missing
Both
Mentioned
D1 14.8% 49.2% 36.0%
D2 17.9% 57.1% 25.0%
Test 0.0% 100.0% 0.0%
Home/Away
Dataset
Both
Missing
One
Missing
Both
Mentioned
D1 85.7% 11.3% 3.0%
D2 17.9% 58.9% 23.2%
Test 20.0% 80.0% 0.0%
27
![Page 28: Towards A Conceptual Model of Computational Sustainabilitycomputational-sustainability.cis.cornell.edu/PPT/Diett... · 2014-09-02 · Aral Sea Fisheries (Conrad, et al.) Bird Migration](https://reader034.vdocument.in/reader034/viewer/2022042416/5f316c3bb5594e600562b925/html5/thumbnails/28.jpg)
Experiment 2 Results D1: The system was unable to
correctly learn the home/away
rule (not enough examples where
both Home and Away were
mentioned)
D2: The system is able to
correctly learn the rules and so it
matches the performance of the
true rules
An EM approach applied to D2
only achieves 50%.
ICS Seminar
10%
100%
0%
20%
40%
60%
80%
100%
D1 D2
% Whole Games Correctly Predicted Relative to True Rules
28
![Page 29: Towards A Conceptual Model of Computational Sustainabilitycomputational-sustainability.cis.cornell.edu/PPT/Diett... · 2014-09-02 · Aral Sea Fisheries (Conrad, et al.) Bird Migration](https://reader034.vdocument.in/reader034/viewer/2022042416/5f316c3bb5594e600562b925/html5/thumbnails/29.jpg)
Experiment 3
Birthplace and Citizenship.
Data: ACE08 Evaluation Corpus
Citizenship mentioned 583 times
Birthplace 25 times
Only 6 articles mention both; 2 of which violate the rule
𝑏𝑜𝑟𝑛𝐼𝑛 𝑋, 𝐶 ⇒ 𝑐𝑖𝑡𝑖𝑧𝑒𝑛𝑂𝑓(𝑋, 𝐶)
ICS Seminar
Probability Assigned to the
Correct Interpretation
Configuration Gricean Method EM Method
Citizenship missing 1.00 0.969
Birthplace missing 1.00 0.565
29
![Page 30: Towards A Conceptual Model of Computational Sustainabilitycomputational-sustainability.cis.cornell.edu/PPT/Diett... · 2014-09-02 · Aral Sea Fisheries (Conrad, et al.) Bird Migration](https://reader034.vdocument.in/reader034/viewer/2022042416/5f316c3bb5594e600562b925/html5/thumbnails/30.jpg)
Experiment 4
Somali Hijacking Incidents
41 news stories from coordination-maree-noire.eu
Manual extractions
25 stories mention only one fact (ownership or flag)
16 mention both, 14 of which violate the rule
𝑜𝑤𝑛𝑒𝑟𝑠ℎ𝑖𝑝𝐶𝑜𝑢𝑛𝑡𝑟𝑦 𝑆, 𝐶 ⇒ 𝑓𝑙𝑎𝑔𝐶𝑜𝑢𝑛𝑡𝑟𝑦(𝑆, 𝐶)
ICS Seminar
Probability Assigned to the
Correct Interpretation
Configuration Gricean Method EM Method
Ownership missing 1.00 0.459
Flag missing 1.00 0.519
30
![Page 31: Towards A Conceptual Model of Computational Sustainabilitycomputational-sustainability.cis.cornell.edu/PPT/Diett... · 2014-09-02 · Aral Sea Fisheries (Conrad, et al.) Bird Migration](https://reader034.vdocument.in/reader034/viewer/2022042416/5f316c3bb5594e600562b925/html5/thumbnails/31.jpg)
Discussion
Inverse Gricean Rule Learning is able to learn correct rules
from real extractions
Extractions should be tuned for high recall
Current algorithm relies on observing a decent number of cases where
both body and head are correctly extracted
Rules are “correct” within Markov Logic, but not necessarily identical
with the rules we would write by hand
ICS Seminar 31
![Page 32: Towards A Conceptual Model of Computational Sustainabilitycomputational-sustainability.cis.cornell.edu/PPT/Diett... · 2014-09-02 · Aral Sea Fisheries (Conrad, et al.) Bird Migration](https://reader034.vdocument.in/reader034/viewer/2022042416/5f316c3bb5594e600562b925/html5/thumbnails/32.jpg)
Concluding Remarks
We are making exciting contributions in Computational
Sustainability
Some of the ideas we are exploring have application in other
parts of computer science
ICS Seminar 32
![Page 33: Towards A Conceptual Model of Computational Sustainabilitycomputational-sustainability.cis.cornell.edu/PPT/Diett... · 2014-09-02 · Aral Sea Fisheries (Conrad, et al.) Bird Migration](https://reader034.vdocument.in/reader034/viewer/2022042416/5f316c3bb5594e600562b925/html5/thumbnails/33.jpg)
Thank-you
National Science Foundation Grants 0832804, 0905885, 1125228
DARPA Contract FA8750-09-C-0179 (BBN Technologies)
Any opinions, findings and conclusions or recommendations expressed in
this material are those of the author(s) and do not necessarily reflect the
views of the NSF, DARPA, the Air Force Research Laboratory (AFRL), or
the US government.
ICS Seminar 33