statistical relational learning: an introduction lise getoor university of maryland, college park...
TRANSCRIPT
![Page 1: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/1.jpg)
Statistical Relational Learning: An Introduction
Lise Getoor
University of Maryland, College Park
September 5, 2007Progic 2007
![Page 2: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/2.jpg)
Statistical Relational Learning: An Introduction
Lise Getoor
University of Maryland, College Park
September 5, 2007Progic 2007
biased
^X
![Page 3: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/3.jpg)
acknowledgements Statistical Relational Learning (SRL) is a synthesis of ideas of many
individuals who have participated in various SRL events, workshops and classes:
Hendrik Blockeel, Mark Craven, James Cussens, Bruce D’Ambrosio, Luc De Raedt, Tom Dietterich, Pedro Domingos, Saso Dzeroski, Peter Flach, Rob Holte, Manfred Jaeger, David Jensen, Kristian Kersting, Daphne Koller, Heikki Mannila, Andrew McCallum Tom Mitchell, Ray Mooney, Stephen Muggleton, Kevin Murphy, Jen Neville, David Page, Avi Pfeffer, Claudia Perlich, David Poole, Foster Provost, Dan Roth, Stuart Russell, Taisuke Sato, Jude Shavlik, Ben Taskar, Lyle Ungar and many others…
![Page 4: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/4.jpg)
Why SRL? Traditional statistical machine learning approaches assume:
A random sample of homogeneous objects from single relation
Traditional relational learning approaches assume: No noise or uncertainty in data
Real world data sets: Multi-relational and heterogeneous Noisy and uncertain
Statistical Relational Learning (SRL): newly emerging research area at the intersection of statistical
models and relational learning/inductive logic programming
Sample Domains: web data, social networks, biological data, communication data,
customer networks, sensor networks, natural language, vision, …
![Page 5: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/5.jpg)
SRL Theory Methods that combine expressive knowledge
representation formalisms such as relational and first-order logic with principled probabilistic and statistical approaches to inference and learning
Directed Approaches Semantics based on Bayesian Networks Frame-based Directed Models Rule-based Directed Models
Undirected Approaches Semantics based on Markov Networks Frame-based Undirected Models Rule-based Undirected Models
Process-based Approaches
![Page 6: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/6.jpg)
SRL Theory Methods that combine expressive knowledge
representation formalisms such as relational and first-order logic with principled probabilistic and statistical approaches to inference and learning
Directed Approaches Semantics based on Bayesian Networks Frame-based Directed Models Rule-based Directed Models
Undirected Approaches Semantics based on Markov Networks Frame-based Undirected Models Rule-based Undirected Models
Process-based Approaches
![Page 7: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/7.jpg)
Directed Frame-based Approaches Probabilistic Relational Models (PRMs)
Representation & Inference [Koller & Pfeffer 98, Pfeffer, Koller, Milch &Takusagawa 99, Pfeffer 00]
Learning [Friedman et al. 99, Getoor, Friedman, Koller & Taskar 01 & 02, Getoor 01]
Probabilistic Entity Relation Models (PERs) Representation [Heckerman, Meek & Koller 04]
Logical syntax for PRMs (PRL) [Getoor & Grant 06]
![Page 8: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/8.jpg)
Probabilistic Relational Models BN Tutorial PRMs w/ Attribute Uncertainty Inference in PRMs Learning in PRMs
PRMs w/ Structural Uncertainty
PRMs w/ Class Hierarchies
![Page 9: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/9.jpg)
Bayesian Networks
Good Writer Smart
Quality
AcceptedReview Length
nodes = domain variablesedges = direct causal influence
Network structure encodes conditional independencies: I(Review-Length , Good-Writer | Reviewer-Mood)
ReviewerMood
0.6 0.4
w
s
w
0.3 0.7
0.1 0.9
0.4 0.6
sw
s
s
w
SW P(Q| W, S)
conditional probability table (CPT)
![Page 10: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/10.jpg)
BN Semantics
Compact & natural representation: nodes k parents O(2k n) vs. O(2n) params natural parameters
conditionalindependenciesin BN structure
+ localCPTs
full jointdistribution
over domain=
W
M
S
Q
AL
),()|(),|()|()()(
),,,,,(
qmaPmlPswqPwmPsPwP
alqmswP
|
![Page 11: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/11.jpg)
Reasoning in BNs Full joint distribution answers any query
P(event | evidence)
Allows combination of different types of reasoning: Causal: P(Reviewer-Mood | Good-Writer) Evidential: P(Reviewer-Mood | not Accepted) Intercausal: P(Reviewer-Mood | not Accepted,
Quality)W
M
S
Q
AL
![Page 12: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/12.jpg)
To compute lqmsw
alqmswPaP,,,,
),,,,,()(
lqmsw
qmaPmlPswqPwmPsPwP,,,,
),|()|(),|()|()()(
factors
mood good writer
good
pissy
good
pissy
falsetrue
truefalse
0.70.3
0.10.9 A factor is a function from
values of variables to positive real numbers
Variable Elimination
![Page 13: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/13.jpg)
qmsw l
qmaPmlPswqPwmPsPwP,,,
),|()|(),|()|()()(
To compute
Variable Elimination
lqmsw
alqmswPaP,,,,
),,,,,()(
![Page 14: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/14.jpg)
lqmsw
mlPqmaPswqPwmPsPwP )|(),|(),|()|()()(,,,
To compute lqmsw
alqmswPaP,,,,
),,,,,()(
sum out l
Variable Elimination
![Page 15: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/15.jpg)
)(),|(),|()|()()(,,,
mfqmaPswqPwmPsPwPqmsw
1
To compute lqmsw
alqmswPaP,,,,
),,,,,()(
new factor
Variable Elimination
![Page 16: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/16.jpg)
),|()|()()(),|()(,,
swqPwmPwPmfqmaPsPwqms 1
To compute lqmsw
alqmswPaP,,,,
),,,,,()(
multiply factors togetherthen sum out w
Variable Elimination
![Page 17: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/17.jpg)
),,()(),|()(,,
sqmfmfqmaPsPqms
21
To compute lqmsw
alqmswPaP,,,,
),,,,,()(
new factor
Variable Elimination
![Page 18: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/18.jpg)
)(aP
To compute lqmsw
alqmswPaP,,,,
),,,,,()(
Variable Elimination
![Page 19: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/19.jpg)
Other Inference Algorithms Exact
Junction Tree [Lauritzen & Spiegelhalter 88] Cutset Conditioning [Pearl 87]
Approximate Loopy Belief Propagation [McEliece et al 98] Likelihood Weighting [Shwe & Cooper 91] Markov Chain Monte Carlo [eg MacKay 98]
• Gibbs Sampling [Geman & Geman 84]• Metropolis-Hastings [Metropolis et al 53, Hastings 70]
Variational Methods [Jordan et al 98]
![Page 20: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/20.jpg)
Learning BNs
Parameters only Structure and Parameters
Complete Data
Incomplete Data
See [Heckerman 98] for a general introduction
Easy: counting Structure search
EM [Dempster et al 77]
or gradient descent
[Russell et al 95]
Structural EM
[Friedman 97]
![Page 21: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/21.jpg)
BN Parameter Estimation Assume known dependency structure G Goal: estimate BN parameters
entries in local probability models,
is good if it’s likely to generate observed data.
MLE Principle: Choose so as to maximize l Alternative: incorporate a prior
),|(log),:( GDPGDl
)][Pa|(, uXxXPux
![Page 22: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/22.jpg)
Learning With Complete Data Fully observed data: data consists of set of
instances, each with a value for all BN variables
With fully observed data, we can compute = number of instances with , and
and similarly for other counts
We then estimate
swqN ,,
sw
swqswq N
NswqP
,
,,,, ),|(
q w s
![Page 23: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/23.jpg)
Dealing w/ missing values Can’t compute But can use Expectation Maximization (EM)
1. Given parameter values, can compute expected counts:
2. Given expected counts, estimate parameters:
Begin with arbitrary parameter values Iterate these two steps Converges to local maximum of likelihood
swqN ,,
i
iiiiswq swqPNE
instances,, )evidence|,,(][
][
][),|(
,
,,,,
sw
swqswq NE
NEswqP
this requires BN inference
![Page 24: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/24.jpg)
Structure search Begin with an empty network Consider all neighbors reached by a search operator
that are acyclic add an edge remove an edge reverse an edge
For each neighbor compute ML parameter values compute score(s) =
Choose the neighbor with the highest score Continue until reach a local maximum
)(log),|(log * sPsDP s
*s
![Page 25: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/25.jpg)
Mini-BN Tutorial Summary Representation – probability distribution factored
according to the BN DAG Inference – exact + approximate Learning – parameters + structure
![Page 26: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/26.jpg)
Probabilistic Relational Models BN Tutorial PRMs w/ Attribute Uncertainty Inference in PRMs Learning in PRMs
PRMs w/ Structural Uncertainty
PRMs w/ Class Hierarchies
![Page 27: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/27.jpg)
Relational Schema
AuthorGood Writer
Author ofHas Review
Describes the types of objects and relations in the world
Review
Paper
Quality
Accepted
Mood
LengthSmart
![Page 28: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/28.jpg)
Probabilistic Relational Model
Length
Mood
Author
Good Writer
Paper
Quality
Accepted
Review
Smart
![Page 29: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/29.jpg)
Probabilistic Relational Model
Length
Mood
Author
Good Writer
Paper
Quality
Accepted
Review
Smart
Paper.Review.Mood Paper.Quality,
Paper.Accepted |
P
![Page 30: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/30.jpg)
Probabilistic Relational Model
Length
Mood
Author
Good Writer
Paper
Quality
Accepted
Review
Smart
3.07.0
4.06.0
8.02.0
9.01.0
,
,
,
,,
tt
ft
tf
ff
P(A | Q, M) MQ
![Page 31: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/31.jpg)
Fixed relational skeleton : set of objects in each class relations between them
Author A1
Paper P1 Author: A1 Review: R1
Review R2
Review R1
Author A2
Relational Skeleton
Paper P2 Author: A1 Review: R2
Paper P3 Author: A2 Review: R2
Primary Keys
Foreign Keys
Review R2
![Page 32: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/32.jpg)
Author A1
Paper P1 Author: A1 Review: R1
Review R2
Review R1
Author A2
PRM defines distribution over instantiations of attributes
PRM w/ Attribute Uncertainty
Paper P2 Author: A1 Review: R2
Paper P3 Author: A2 Review: R2
Good Writer
Smart
Length
Mood
Quality
Accepted
Length
Mood
Review R3
Length
Mood
Quality
Accepted
Quality
Accepted
Good Writer
Smart
![Page 33: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/33.jpg)
P2.Accepted
P2.Quality r2.Mood
P3.Accepted
P3.Quality3.07.0
4.06.0
8.02.0
9.01.0
,
,
,
,,
tt
ft
tf
ff
P(A | Q, M) MQPissyLow
3.07.0
4.06.0
8.02.0
9.01.0
,
,
,
,,
tt
ft
tf
ff
P(A | Q, M) MQ
r3.Mood
A Portion of the BN
![Page 34: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/34.jpg)
P2.Accepted
P2.Quality r2.Mood
P3.Accepted
P3.Quality
PissyLow
r3.MoodHigh 3.07.0
4.06.0
8.02.0
9.01.0
,
,
,
,,
tt
ft
tf
ff
P(A | Q, M) MQ
Pissy
A Portion of the BN
![Page 35: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/35.jpg)
Length
Mood
Paper
Quality
Accepted
Review
Review R1
Length
Mood
Review R2
Length
Mood
Review R3
Length
Mood
Paper P1
Accepted
Quality
PRM: Aggregate Dependencies
![Page 36: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/36.jpg)
sum, min, max, avg, mode, count
Length
Mood
Paper
Quality
Accepted
Review
Review R1
Length
Mood
Review R2
Length
Mood
Review R3
Length
Mood
Paper P1
Accepted
Quality
mode
3.07.0
4.06.0
8.02.0
9.01.0
,
,
,
,,
tt
ft
tf
ff
P(A | Q, M) MQ
PRM: Aggregate Dependencies
![Page 37: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/37.jpg)
PRM with AU Semantics
)).(|.(),,|( ,.
AxparentsAxPP Sx Ax
SI
AttributesObjects
probability distribution over completions I:
PRM relational skeleton + =
Author
Paper
Review
Author A1
Paper P2
Paper P1
ReviewR3
ReviewR2
ReviewR1
Author A2
Paper P3
![Page 38: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/38.jpg)
Probabilistic Relational Models BN Tutorial PRMs w/ Attribute Uncertainty Inference in PRMs Learning in PRMs
PRMs w/ Structural Uncertainty
PRMs w/ Class Hierarchies
![Page 39: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/39.jpg)
PRM Inference Simple idea: enumerate all attributes of all objects Construct a Bayesian network over all the attributes
![Page 40: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/40.jpg)
Inference Example
ReviewR2
ReviewR1
AuthorA1
PaperP1
ReviewR4
ReviewR3
PaperP2
Skeleton
Query is P(A1.good-writer)Evidence is P1.accepted = T, P2.accepted = T
![Page 41: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/41.jpg)
A1.Smart
P1.Quality
P1.Accepted
R1.Mood
R1.Length
R2.Mood
R2.Length P2.Quality
P2.Accepted
R3.Mood
R3.Length
R4.Mood
R4.Length
A1.Good Writer
PRM Inference: Constructed BN
![Page 42: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/42.jpg)
PRM Inference Problems with this approach:
constructed BN may be very large doesn’t exploit object structure
Better approach: reason about objects themselves reason about whole classes of objects
In particular, exploit: reuse of inference encapsulation of objects
![Page 43: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/43.jpg)
PRM Inference: Interfaces
A1.Smart
P1.Quality
P1.Accepted
R2.Mood
R2.Length
R1.Mood
R1.Length
A1.Good Writer
Variables pertainingto R2: inputs and internal attributes
![Page 44: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/44.jpg)
PRM Inference: Interfaces
A1.Smart
P1.Quality
P1.Accepted
R2.Mood
R2.Length
R1.Mood
R1.Length
A1.Good Writer
Interface: imported and exportedattributes
![Page 45: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/45.jpg)
PRM Inference: Encapsulation
A1.Smart
P1.Quality
P1.Accepted
R1.Mood
R1.Length
R2.Mood
R2.Length P2.Quality
P2.Accepted
R3.Mood
R3.Length
R4.Mood
R4.Length
A1.Good Writer
R1 and R2 areencapsulatedinside P1
![Page 46: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/46.jpg)
PRM Inference: Reuse
A1.Smart
P1.Quality
P1.Accepted
R1.Mood
R1.Length
R2.Mood
R2.Length P2.Quality
P2.Accepted
R3.Mood
R3.Length
R4.Mood
R4.Length
A1.Good Writer
![Page 47: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/47.jpg)
A1.Smart
Structured Variable Elimination
Paper-1
Paper-2
Author 1
A1.Good Writer
![Page 48: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/48.jpg)
A1.Smart
Paper-1
Paper-2
Author 1
A1.Good Writer
Structured Variable Elimination
![Page 49: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/49.jpg)
A1.Smart
P1.Quality
P1.Accepted
Review-2Review-1
Paper 1
A1.Good Writer
Structured Variable Elimination
![Page 50: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/50.jpg)
A1.Smart
Review-2Review-1
Paper 1
A1.Good Writer
P1.Accepted
P1.Quality
Structured Variable Elimination
![Page 51: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/51.jpg)
R2.Mood
R2.Length
Review 2
A1.Good Writer
Structured Variable Elimination
![Page 52: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/52.jpg)
Review 2
A1.Good Writer
R2.Mood
Structured Variable Elimination
![Page 53: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/53.jpg)
A1.Smart
Review-2Review-1
Paper 1
A1.Good Writer
P1.Accepted
P1.Quality
Structured Variable Elimination
![Page 54: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/54.jpg)
A1.Smart
Review-1
Paper 1
R2.Mood
A1.Good Writer
P1.Accepted
P1.Quality
Structured Variable Elimination
![Page 55: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/55.jpg)
A1.Smart
Structured Variable Elimination
Review-1
Paper 1
R2.Mood
A1.Good Writer
P1.Accepted
P1.Quality
![Page 56: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/56.jpg)
A1.Smart
Paper 1
R2.MoodR1.Mood
A1.Good Writer
P1.Accepted
P1.Quality
Structured Variable Elimination
![Page 57: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/57.jpg)
A1.Smart
Paper 1
R2.MoodR1.Mood
A1.Good Writer
P1.Accepted
P1.Quality
True
Structured Variable Elimination
![Page 58: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/58.jpg)
A1.Smart
Paper 1
A1.Good Writer
Structured Variable Elimination
![Page 59: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/59.jpg)
A1.Smart
Paper-1
Paper-2
Author 1
A1.Good Writer
Structured Variable Elimination
![Page 60: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/60.jpg)
A1.Smart
Paper-2
Author 1
A1.Good Writer
Structured Variable Elimination
![Page 61: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/61.jpg)
A1.Smart
Paper-2
Author 1
A1.Good Writer
Structured Variable Elimination
![Page 62: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/62.jpg)
A1.Smart
Author 1
A1.Good Writer
Structured Variable Elimination
![Page 63: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/63.jpg)
Author 1
A1.Good Writer
Structured Variable Elimination
![Page 64: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/64.jpg)
Benefits of SVE
Structured inference leads to good elimination orderings for VE interfaces are separators
• finding good separators for large BNs is very hard
therefore cheaper BN inference
Reuses computation wherever possible
![Page 65: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/65.jpg)
Limitations of SVE Does not work when encapsulation breaks down
But when we don’t have specific information about the connections between objects, we can assume that encapsulation holds i.e., if we know P1 has two reviewers R1 and R2 but they are not
named instances, we assume R1 and R2 are encapsulated
Cannot reuse computation when different objects have different evidence
ReviewerR2
AuthorA1
PaperP1
ReviewerR4
ReviewerR3Paper
P2
R3 is notencapsulatedinside P2
![Page 66: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/66.jpg)
Probabilistic Relational Models BN Tutorial PRMs w/ Attribute Uncertainty Inference in PRMs Learning in PRMs
PRMs w/ Structural Uncertainty
PRMs w/ Class Hierarchies
![Page 67: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/67.jpg)
Learning PRMs w/ AU
Database Paper
Author
Review
Relational
Schema
PaperReview
Author
• Parameter estimation• Structure selection
![Page 68: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/68.jpg)
PaperQuality
Accepted
ReviewMood
Length
MRQP
APMRQP
N
N
.,.
.,.,.
APMRQPN
.,.,.where is the number of accepted,
low quality papers whose reviewer was in a poor mood
,
,
,
,,
tt
ft
tf
ff
P(A | Q, M) MQ?
?
?
?
?
?
?
?
ML Parameter Estimation
![Page 69: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/69.jpg)
PaperQuality
Accepted
ReviewMood
Length
MRQP
APMRQP
N
N
.,.
.,.,.,
,
,
,,
tt
ft
tf
ff
P(A | Q, M) MQ?
?
?
?
?
?
?
?
Count
Query for counts:
Reviewtable
Papertable
AcceptedP
MoodR
QualityP
.
.
.
ML Parameter Estimation
![Page 70: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/70.jpg)
Structure Selection
Idea: define scoring function do local search over legal structures
Key Components:legal models scoring modelssearching model space
![Page 71: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/71.jpg)
Structure Selection
Idea: define scoring function do local search over legal structures
Key Components:» legal modelsscoring modelssearching model space
![Page 72: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/72.jpg)
Legal Models
author-of
PRM defines a coherent probability model over a skeleton if the dependencies between object attributes is acyclic
How do we guarantee that a PRM is acyclic for every skeleton?
ResearcherProf. Gump
Reputationhigh
PaperP1
Accepted yes Paper
P2Accepted
yes
sum
![Page 73: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/73.jpg)
Attribute Stratification
PRMdependency structure S
dependencygraph
Paper.Accepted
Researcher.Reputation
if Researcher.Reputation depends directly on Paper.Accepted
dependency graph acyclic acyclic for any Attribute stratification:
Algorithm more flexible; allows certain cycles along guaranteed acyclic relations
![Page 74: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/74.jpg)
Structure Selection
Idea: define scoring function do local search over legal structures
Key Components:legal models» scoring models – same as BNsearching model space
![Page 75: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/75.jpg)
Structure Selection
Idea: define scoring function do local search over legal structures
Key Components:legal models scoring models» searching model space
![Page 76: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/76.jpg)
Searching Model Space
ReviewAuthor Paper
score
Delete R.MR.L ReviewAuthor Paper
scoreAdd A.SA.W
Author ReviewPaper
Phase 0: consider only dependencies within a class
)(.
.).(RattributesedescriptivBR
BRARParentsPotential
![Page 77: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/77.jpg)
ReviewAuthor Paper scoreAdd A.SP.A
score
Add P.AR.MReviewAuthor Paper
ReviewPaperAuthor
Phase 1: consider dependencies from “neighboring” classes, via schema relations
Phased Structure Search
)(.
.).(SRattributesedescriptivCS
CSARParentsPotential
![Page 78: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/78.jpg)
Phased Structure Search
scoreAdd A.SR.M
score
Add R.MA.W
Phase 2: consider dependencies from “further” classes, via relation chains
ReviewAuthor Paper
ReviewAuthor Paper
ReviewAuthor Paper
)(.
.).(TSRattributesedescriptivDT
DTARParentsPotential
![Page 79: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/79.jpg)
Probabilistic Relational Models BN Tutorial PRMs w/ Attribute Uncertainty Inference in PRMs Learning in PRMs
PRMs w/ Structural Uncertainty
PRMs w/ Class Hierarchies
![Page 80: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/80.jpg)
Reminder: PRM w/ AU Semantics
)).(|.(),,|( ,.
AxparentsAxPP Sx Ax
SI
AttributesObjects
probability distribution over completions I:
PRM relational skeleton + =
Author
Paper
Review
Author A1
Paper P2
Paper P1
ReviewR3
ReviewR2
ReviewR1
Author A2
Paper P3
![Page 81: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/81.jpg)
Kinds of structural uncertainty How many objects does an object relate to?
how many Authors does Paper1 have? Which object is an object related to?
does Paper1 cite Paper2 or Paper3? Which class does an object belong to?
is Paper1 a JournalArticle or a ConferencePaper? Does an object actually exist? Are two objects identical?
![Page 82: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/82.jpg)
Structural Uncertainty Motivation: PRM with AU only well-defined when the
skeleton structure is known May be uncertain about relational structure itself Construct probabilistic models of relational structure
that capture structural uncertainty Mechanisms:
Reference uncertainty Existence uncertainty Number uncertainty Type uncertainty Identity uncertainty
![Page 83: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/83.jpg)
Citation Relational Schema
Wrote
PaperTopic
Word1
WordN
…Word2
PaperTopic
Word1
WordN
…Word2Cites
Citing Paper
Cited Paper
AuthorInstitution
Research Area
![Page 84: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/84.jpg)
Attribute Uncertainty
Paper
Word1
Topic
WordN
Wrote
Author
...
Research Area
P( WordN | Topic)
P( Topic | Paper.Author.Research Area
InstitutionP( Institution |
Research Area)
![Page 85: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/85.jpg)
Reference Uncertainty
Bibliography
Scientific Paper
`1. -----2. -----3. -----
???
Document Collection
![Page 86: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/86.jpg)
PRM w/ Reference Uncertainty
CitesCitingCited
Dependency model for foreign keys
PaperTopicWords
PaperTopicWords
Naïve Approach: multinomial over primary key• noncompact• limits ability to generalize
![Page 87: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/87.jpg)
Reference Uncertainty Example
PaperP5
Topic AI
PaperP4
Topic AI
PaperP3
Topic AI
PaperM2
Topic AI
Paper P1Topic Theory
CitesCitingCited
Paper P5Topic AI
PaperP3
Topic AI
Paper P4Topic Theory
Paper P2Topic Theory
Paper P1Topic Theory
Paper.Topic = AIPaper.Topic = Theory
C1
C2
C1 C2
3.0 7.0
![Page 88: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/88.jpg)
PaperP5
Topic AI
PaperP4
Topic AI
PaperP3
Topic AI
PaperM2
Topic AI
Paper P1Topic Theory
CitesCitingCited
Paper P5Topic AI
PaperP3
Topic AI
Paper P4Topic Theory
Paper P2Topic Theory
Paper P6Topic Theory
Paper.Topic = AIPaper.Topic = Theory
C1
C2
PaperTopicWords
C1 C2
1.0 9.0
Topic
99.0 01.0
Theory AI
Reference Uncertainty Example
![Page 89: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/89.jpg)
Introduce Selector RVs
Cites1.Cited
Cites1.Selector
P1.Topic
P2.Topic
P3.Topic
P4.Topic
P5.Topic
P6.TopicCites2.Cited
Cites2.Selector
Introduce Selector RV, whose domain is {C1,C2}The distribution over Cited depends on all of the topics, and the selector
![Page 90: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/90.jpg)
PRMs w/ RU Semantics
PRM-RU + entity skeleton
probability distribution over full instantiations I
Cites
Cited
Citing
PaperTopic
Words
PaperTopic
Words
PRM RU
Paper P5Topic AI
Paper P4Topic Theory
Paper P2Topic Theory
Paper P3Topic AI
Paper P1Topic ???
Paper P5Topic AI
Paper P4Topic Theory
Paper P2Topic Theory
Paper P3Topic AI
Paper P1Topic ???
RegReg
RegRegCites
entity skeleton
![Page 91: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/91.jpg)
Learning Idea:
define scoring function do phased local search over legal structures
Key Components: legal models
scoring models
searching model space
PRMs w/ RU
model new dependencies
new operators
unchanged
![Page 92: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/92.jpg)
Legal Models
Cites
Citing
Cited
Mood
Paper
Important
Accepted
Review
Paper
Important
Accepted
![Page 93: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/93.jpg)
Legal Models
P1.Accepted
When a node’s parent is defined using an uncertain relation, the reference RV must be a parent of the node as well.
Cites1.Cited
Cites1.Selector
R1.Mood
P2.Important
P3.Important
P4.Important
![Page 94: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/94.jpg)
Structure Search
CitesCitingCited
PaperTopicWords
PaperTopicWords
Cited
Papers
1.0
Paper Paper
Paper Paper
Paper Paper
Paper Paper
Paper Paper
Paper
AuthorInstitution
![Page 95: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/95.jpg)
Structure Search: New Operators
CitesCitingCited
PaperTopicWords
PaperTopicWords
Cited
Paper Paper
Paper Paper
Paper Paper
Paper Paper
Paper Paper
Paper
Topic
ΔscoreRefine on Topic
Paper Paper
Paper Paper
Paper
Paper Paper
Paper Paper
Paper
AuthorInstitution
![Page 96: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/96.jpg)
Structure Search: New Operators
CitesCitingCited
PaperTopicWords
PaperTopicWords
Cited
Paper Paper
Paper Paper
Paper Paper
Paper Paper
Paper Paper
Paper
Topic
Refine on Topic
Paper Paper
Paper Paper
Paper
Paper Paper
Paper Paper
Paper
Paper Paper
Paper Paper
Paper
Paper
Paper Paper
Δscore
Refine on Author.Instition
AuthorInstitution
Institution
![Page 97: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/97.jpg)
PRMs w/ RU Summary Define semantics for uncertainty over which entities
are related to each other Search now includes operators Refine and
Abstract for constructing foreign-key dependency model
Provides one simple mechanism for link uncertainty
![Page 98: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/98.jpg)
Existence Uncertainty
Document CollectionDocument Collection
? ??
![Page 99: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/99.jpg)
Cites
Dependency model for existence of relationship
PaperTopicWords
PaperTopicWords
Exists
PRM w/ Exists Uncertainty
![Page 100: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/100.jpg)
Exists Uncertainty Example
Cites
PaperTopicWords
PaperTopicWords
Exists
Citer.Topic Cited.Topic
0.995 0005 Theory Theory
False True
AI Theory 0.999 0001
AI AI 0.993 0008
AI Theory 0.997 0003
![Page 101: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/101.jpg)
Paper#2 Topic Paper#3Topic
WordN
Paper#1
Word1
Topic
... ... ...
Author #1Area Inst
#1-#2
Author #2Area Inst
Exists
#2-#3
Exists
#2-#1
Exists
#3-#1
Exists
#1-#3
Exists
WordN
Word1WordN
Word1
Exists
#3-#2
Introduce Exists RVs
![Page 102: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/102.jpg)
Paper#2 Topic Paper#3Topic
WordN
Paper#1
Word1
Topic
... ... ...
Author #1Area Inst
#1-#2
Author #2Area Inst
Exists
#2-#3
Exists
#2-#1
Exists
#3-#1
Exists
#1-#3
Exists
WordN
Word1WordN
Word1
Exists
WordNWord1WordN
Word1WordN
Word1
ExistsExists Exists ExistsExists Exists
#3-#2
Introduce Exists RVs
![Page 103: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/103.jpg)
PRMs w/ EU Semantics
PRM-EU + object skeleton
probability distribution over full instantiations I
Paper P5Topic AI
Paper P4Topic Theory
Paper P2Topic Theory
Paper P3Topic AI
Paper P1Topic ???
Paper P5Topic AI
Paper P4Topic Theory
Paper P2Topic Theory
Paper P3Topic AI
Paper P1Topic ???
object skeleton
???
PRM EU
Cites
Exists
PaperTopic
Words
PaperTopic
Words
![Page 104: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/104.jpg)
Idea: define scoring function do phased local search over legal structures
Key Components: legal models
scoring models
searching model space
PRMs w/ EU
model new dependencies
unchanged
unchanged
Learning
![Page 105: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/105.jpg)
Probabilistic Relational Models BN Tutorial PRMs w/ Attribute Uncertainty Inference in PRMs Learning in PRMs
PRMs w/ Structural Uncertainty
PRMs w/ Class Hierarchies
![Page 106: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/106.jpg)
PRMs with classes Relations organized in a class hierarchy
Subclasses inherit their probability model from superclasses Instances are a special case of subclasses of size 1 As you descend through the class hierarchy, you can have
richer dependency models e.g. cannot say Accepted(P1) <- Accepted(P2) (cyclic) but can say Accepted(JournalP1) <- Accepted(ConfP2)
Venue
Journal Conference
![Page 107: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/107.jpg)
Type Uncertainty Is 1st-Venue a Journal or Conference ? Create 1st-Journal and 1st-Conference objects Introduce Type(1st-Venue) variable with possible
values Journal and Conference Make 1st-Venue equal to 1st-Journal or 1st-
Conference according to value of Type(1st-Venue)
![Page 108: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/108.jpg)
Learning PRM-CHs
Relational
Schema
Database:
TVProgram Person
Vote
Person
Vote
TVProgram
Instance I
• Class hierarchy provided
• Learn class hierarchy
![Page 109: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/109.jpg)
Learning Idea:
define scoring function do phased local search over legal structures
Key Components: legal models
scoring models
searching model space
PRMs w/ CH
model new dependencies
new operators
unchanged
![Page 110: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/110.jpg)
Journal
Topic
Quality
Accepted Conf-PaperTopic
Quality
Accepted
Journal.Accepted
Conf-Paper.Accepted
Paper
Topic
Quality
Accepted
Paper.AcceptedPaper.Class
Guaranteeing Acyclicity w/ Subclasses
![Page 111: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/111.jpg)
Learning PRM-CH Scenario 1: Class hierarchy is provided
New OperatorsSpecialize/Inherit
AcceptedPaper
AcceptedJournal AcceptedConference AcceptedWorkshopAcceptedWorksho
p
![Page 112: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/112.jpg)
Learning Class Hierarchy Issue: partially observable data set Construct decision tree for class defined over attributes
observed in training set
Paper.Venue
conferenceworkshop
class1 class3
journal
class2
class4
high
Paper.Author.Fame
class5
medium
class6
low
New operator Split on class attribute Related class attribute
![Page 113: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/113.jpg)
PRMs w/ Class HierarchiesAllow us to: Refine a “heterogenous” class into more
coherent subclasses Refine probabilistic model along class
hierarchyCan specialize/inherit CPDsConstruct new dependencies that were
originally “acyclic”
Provides bridge from class-based model to instance-based model
![Page 114: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/114.jpg)
Summary: PRMs
Focus on objects and relationships what types of objects are there, and how are they
related to each other? how does a property of an object depend on other
properties (of the same or other objects)?
Models support Attribute uncertainty Structural uncertainty Class Hierarchies
Efficient Inference and Learning Algorithms
![Page 115: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/115.jpg)
SRL Tasks Entity Resolution Link Prediction Collective Classification Information Diffusion Community Discovery/Group Detection
Other more complex integrated tasks such as information extraction, ontology alignment
Tasks common to any SRL approachl!Bayesian Logic Programs, Markov Bayesian NetworksLogic Networks,
Probabilistic Relational Models, Relational, Relational Dependency Networks, Relational Probability Trees, Stochastic Logic Programming to name a few
![Page 116: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/116.jpg)
SRL Challenges Collective Classification Collective Resolution Logical vs. Statistical dependencies Feature Construction – aggregation, selection Flexible and Decomposable Combining Rules Instances vs. Classes Effective Use of Labeled & Unlabeled Data Link Prediction Closed vs. Open World
Challenges common to any SRL approachl!Bayesian Logic Programs, Markov Logic Networks, Probabilistic Relational Models, Relational Models, Relational, Relational Dependency Networks, Relational Probability Trees, Stochastic Logic Programming to name a few
![Page 117: Statistical Relational Learning: An Introduction Lise Getoor University of Maryland, College Park September 5, 2007 Progic 2007](https://reader035.vdocument.in/reader035/viewer/2022062722/56649f275503460f94c3fcea/html5/thumbnails/117.jpg)
Conclusion Statistical Relational Learning
Supports multi-relational, heterogeneous domains Supports noisy, uncertain, non-IID data aka, real-world data!
Differences in approaches: rule-based vs. frame-based directed vs. undirected
Many common issues: Need for collective classification and consolidation Need for aggregation and combining rules Need to handle labeled and unlabeled data Need to handle structural uncertainty etc.
Great opportunity for combining rich logical representation and inference and learning with hierarchical statistical models!!