encoding generalized quantifiers in dependency-based compositional semantics

Encoding Generalized Quantifiers in

Dependency-based Compositional Semantics

Yubing Dong – University of Southern California

Ran Tian – Tohoku University

Yusuke Miyao – National Institute of Informatics, Japan

BackgroundGeneralized Quantifiers (GQ)

Generalized Quantifiers (GQ)

Most students like noodles.

Generalized Quantifier



Property-denoting noun phrase

Generalized Quantifier



Property-denoting noun phrase

PredicateGeneralized Quantifier


Most (Student) (LikeNoodles) ∈ {0,1}

DenotationsStudent ⊆ 𝑊

LikeNoodles ⊆ 𝑊Binary Relation over 𝑊


Most (Student) (LikeNoodles)

iff

𝐒𝐭𝐮𝐝𝐞𝐧𝐭 ∩ 𝐋𝐢𝐤𝐞𝐍𝐨𝐨𝐝𝐥𝐞𝐬

𝐒𝐭𝐮𝐝𝐞𝐧𝐭> 80%

The relation imposed by a GQ is usually based on the notion ⋅ of set cardinalities


Most (Student) (LikeNoodles)Many

ALotOf

Few

AFew

AtMost[n]

AtLeast[n]

BackgroundRecognizing Textual Entailment (RTE)

Recognizing Textual Entailment (RTE)

Example:• 𝑇1: Mary loves every dog.• 𝑇2: Tom has a dog.• 𝐻: Tom has an animal that Mary loves.• 𝑇1, 𝑇2 ⇒ 𝐻 i.e. 𝑇1 and 𝑇2 entails 𝐻

Definition: “𝑇 entails 𝐻" (𝑇 ⇒ 𝐻) if, typically, a human

reading 𝑇 would infer that 𝐻 is most likely true• Relatively loose, compared to logical entailment

GQ in RTE

At most 5 students like noodles.

At most 5 Japanese students like udon noodles.

GQ in RTE

At least 5 students like noodles.

At least 5 Japanese students like udon noodles.

GQ in RTE


Most Japanese students like udon noodles.

GQ in RTE

The FraCaS Corpus:• Built in mid-1990s• A set of hand-crafted entailment problems covering

wide range of semantic phenomena

Section 1 - Generalized Quantifiers:• 74 problems:

• 44 have single premise sentence• 30 have multiple premise sentence

GQ in RTE

SystemAccuracy

Single Multi Overall

NatLogMacCartney07 84.1%

N/AMacCartney08 97.7%

CCG-DistParser Syntax 70.5% 50.0% 62.2%

Gold Syntax 88.6% 80.0% 85.1%

Accuracies of previous systems on Section 1 of FraCaS corpus

GQ in RTE

SystemAccuracy





Gold Syntax 88.6% 80.0% 85.1%

TIFMO

Baseline 79.5% 86.7% 82.4%

Selection 90.9% 93.3% 91.9%

Relation 88.6% 93.3% 90.5%

Selection+Relation 93.2% 96.7% 94.6%


But I’m getting ahead of myself…

BackgroundProperties of GQs

Properties of GQsProblem with encoding the “perfect semantics”

Most (Student) (LikeNoodles)

iff

𝐒𝐭𝐮𝐝𝐞𝐧𝐭 ∩ 𝐋𝐢𝐤𝐞𝐍𝐨𝐨𝐝𝐥𝐞𝐬

𝐒𝐭𝐮𝐝𝐞𝐧𝐭> 80%

Challenge: set cardinalities are difficult to perfectly encode

Properties of GQs

Compromise: only encode major GQ properties• Interaction with universal and existential quantifications• Conservativity• Monotonicity

Properties of GQsInteraction with universal and existential quantifications

Case 1:𝐴 ⊆ 𝐵 ⇒ 𝐹 𝐴 𝐵 ⇒ 𝐴 ∩ 𝐵 ≠ ∅

Example: “most”

All students like noodles.


There are students who like noodles.


Case 2:𝐴 ⊆ 𝐵 ⇒ 𝐹 𝐴 𝐵 ⇒ 𝐴 ∩ 𝐵 ≠ ∅

Example: “a lot of”


A lot of students like noodles.



Case 3:𝐴 ⊆ 𝐵 ⇒ 𝐹 𝐴 𝐵 ⇒ 𝐴 ∩ 𝐵 ≠ ∅

Example: “at most n”




Properties of GQsConservativity

The “domain restraining” role of the noun argument• Eliminates objects that do not have the noun property• Only need to consider which of the rest has the predicate property

𝐹 𝐴 𝐵 ⟺ 𝐹(𝐴)(𝐴 ∩ 𝐵)

Example:• “Few apples are toxic.”⟺“Few apples are toxic apples.”• We don’t care non-apples toxicants, e.g. toxic oranges

Properties of GQsMonotonicity

A GQ 𝐹 ⋅ ⋅ is upward entailing in the noun argument if:𝐹 𝐴′ 𝐵 ⇒ 𝐹 𝐴 𝐵 ∀𝐴′ ⊆ 𝐴

Similarly, a GQ can also be• downward entailing in the noun argument, and • upward/downward entailing in the predicate argument



At most 5 Japanese students like udon noodles.

Example: “at most 𝑛” is downward entailing in each argument


Example: “at least 𝑛” is upward entailing in each argument

At least 5 students like noodles.

At least 5 Japanese students like udon noodles.


Example: “most” is neither upward nor downward entailing in the noun argument


Most Japanese students like noodles.


Example: but is upward entailing in the predicate argument


Most students like udon noodles.

BackgroundDependency-based Compositional Semantics (DCS) for RTE

• Proposed by Tian et al. (2014)

DCS for RTE

DCS tree for “All students like udon noodles”

DCS for RTE


Abstract Denotations:

𝐧𝐨𝐨𝐝𝐥𝐞 ⊆ 𝑊𝐮𝐝𝐨𝐧 ⊆ 𝑊𝐬𝐭𝐮𝐝𝐞𝐧𝐭 ⊆ 𝑊𝐥𝐢𝐤𝐞 ⊆ 𝑊 ×𝑊

DCS for RTE

𝐷1 = 𝐧𝐨𝐨𝐝𝐥𝐞 ∩ 𝐮𝐝𝐨𝐧


“udon noodles”

DCS for RTE


𝐷2 = 𝐥𝐢𝐤𝐞 ∩ 𝑊𝑆𝐵𝐽 × 𝐷1 𝑂𝐵𝐽


“like udon noodles”

DCS for RTE



𝐷3 = 𝜋𝑆𝐵𝐽 𝐷2


“subjects who like udon noodles”

DCS for RTE



𝐷3 = 𝜋𝑆𝐵𝐽 𝐷2𝐷4 = 𝑞⊆

𝑆𝐵𝐽 𝐷3, 𝐬𝐭𝐮𝐝𝐞𝐧𝐭


q⊆r R,C ≡ x ∅≠R∩ x ×Wr ⊆ x ×Cr

If 𝑅 and 𝐶 have the same dimension,• 𝑞⊆

𝑟 𝑅, 𝐶 = ∗ (0-dimension point set) when 𝐶 ⊆ 𝑅,• 𝑞⊆

𝑟 𝑅, 𝐶 = ∅ otherwise

wide reading of “⊆”

DCS for RTE





𝐷5 = 𝑞⊆𝑆𝐵𝐽

𝐷2, 𝐬𝐭𝐮𝐝𝐞𝐧𝐭


q⊆r R,C ≡ x ∅≠R∩ x ×Wr ⊆ x ×Cr

If 𝑅 and 𝐶 have the same dimension,• 𝑞⊆

𝑟 𝑅, 𝐶 = ∗ (0-dimension point set) when 𝐶 ⊆ 𝑅,• 𝑞⊆

𝑟 𝑅, 𝐶 = ∅ otherwise

narrow reading of “⊆”(“the set of udon noodles that all student like”)

DCS for RTE





𝐷5 = 𝑞⊆𝑆𝐵𝐽

𝐷2, 𝐬𝐭𝐮𝐝𝐞𝐧𝐭


Prove statement

• 𝐷4 ≠ ∅ (wide reading) or• 𝐷5 ≠ ∅ (narrow reading)

using forward chaining

DCS for RTE

Basic operators / functions:• × - Cartesian product of sets• ∩ - Set intersection• 𝜋𝑟 - Projection onto domain of semantic role 𝑟• 𝑙𝑟 - Relabeling• 𝑞⊆

𝑟 - Division

Basic types of statements:• Non-emptiness: 𝐴 ≠ ∅• Subsumption: 𝐴 ⊆ 𝐵

BackgroundDCS for RTE: the selection operator

• Also introduced in Tian et al. (2014)

DCS for RTE: the selection operator

• Introduced as an extension to represent the generalized selection operation in relational algebra

• Marked on a DCS tree node• Wrap the abstract denotation 𝐷 to form a new abstract

denotation 𝑠𝑓 𝐷

• The properties of 𝑠𝑓 𝐷 can be user defined

Example:

the set of highest mountains: 𝑠ℎ𝑖𝑔ℎ𝑒𝑠𝑡(𝐦𝐨𝐮𝐧𝐭𝐚𝐢𝐧)

Encoding Generalized Quantifiersas selections

Encoding GQs as Selections

We encode a GQ 𝐹 using selection 𝑠𝐹 as:

𝐹 𝐴 𝐵 ≡ 𝑠𝐹 𝐴 ⊆ 𝐵

Basic requirement:• 𝐹 should be upward-entailing in the predicate

argument 𝐵• A major limitation



• Entailment from universal quantification now written as:𝐴 ⊆ 𝐵 ⇒ 𝑠𝐹 𝐴 ⊆ 𝐵

• Conservativity as:𝑠𝐹 𝐴 ⊆ 𝐴 ∩ 𝐵 ⇔ 𝑠𝐹 𝐴 ⊆ 𝐵

• Both hold if we add axiom:𝑠𝐹 𝐴 ⊆ 𝐴



• Entailment to existence quantification now written as:𝑠𝐹 𝐴 ⊆ 𝐵 ⇒ 𝐴 ∩ 𝐵 ≠ ∅

• Holds if we add axiom:𝑠𝐹 𝐴 ∩ 𝐴 ≠ ∅



• Monotonicity in the noun argument 𝐴 (e.g. upward) now written as:

A ⊆ A′ ∧ 𝑠𝐹 𝐴 ⊆ 𝐵 ⇒ 𝑠𝐹 𝐴′ ⊆ 𝐵

• Holds if we add axiom:A ⊆ A′ ⇒ 𝑠𝐹 𝐴 ⊇ 𝑠𝐹 𝐴′

DCS tree for “At least 5 students like udon noodles.”where the GQ “at least 5” is encoded as selection 𝑠𝐴𝑡𝐿𝑒𝑎𝑠𝑡 5


Example: at least 𝑛

• Satisfied: upward-entailing in predicate argument

• Entails existential quantification:∀𝐴 𝑠𝐴𝑡𝐿𝑒𝑎𝑠𝑡 5 𝐴 ∩ 𝐴 ≠ ∅

• Upward-entailing in noun argument:∀𝐴, 𝐴′ 𝑠. t. A ⊆ A′

𝑠𝐴𝑡𝐿𝑒𝑎𝑠𝑡 5 𝐴 ⊇ 𝑠𝐴𝑡𝐿𝑒𝑎𝑠𝑡 5 𝐴′

Encoding GQs as SelectionsExample:

“At least 5 Japanese students like udon noodles.”

⇒ “ At least 5 students like noodles.”




𝐷3′ = 𝜋𝑆𝐵𝐽 𝐥𝐢𝐤𝐞 ∩ 𝑊𝑆𝐵𝐽 × 𝐧𝐨𝐨𝐝𝐥𝐞𝑂𝐵𝐽

Encoding Generalized Quantifiersas relations

Encoding GQs as RelationsIntro to Relations

• Review: GQ can be seen as binary relation over 2𝑊

• Therefore, we introduce a new extension: relation• A new type of statement• A relation 𝑟𝐹 𝐴, 𝐵 can represent arbitrary custom

relation between abstract denotations 𝐴 and 𝐵

Encoding GQs as RelationsIntro to Relations

Relation 𝑟𝐹 𝐴, 𝐵

• The inference engine keeps track of which term pairs are labeled with which relations• Does 𝐴 and 𝐵 have relation 𝑟𝐹?• What terms have relation 𝑟𝐹 to 𝐴?

• Supports custom axioms for a relation• What entails 𝑟𝐹 𝐴, 𝐵 ?• What does 𝑟𝐹 𝐴, 𝐵 entail?

Encoding GQs as Relations

We intuitively encode a GQ 𝐹 using relation 𝑟𝐹 as:

𝐹 𝐴 𝐵 ≡ r𝐹 𝐴, 𝐵




Statement:𝑟𝐴𝑡𝑀𝑜𝑠𝑡 5 𝐬𝐭𝐮𝐝𝐞𝐧𝐭, 𝐷3



• Entailment from universal quantification:𝐴 ⊆ 𝐵 ⇒ 𝑟𝐹 𝐴, 𝐵

• Entailment to existential quantification:𝑟𝐹 𝐴, 𝐵 ⇒ 𝐴 ∩ 𝐵 ≠ ∅

• Monotonicity (e.g. downward in both arguments):𝑟𝐹 𝐴, 𝐵 ∧ 𝐴 ⊇ 𝐴′ ∧ 𝐵 ⊇ 𝐵′ ⇒ 𝑟𝐹 𝐴′, 𝐵′



• Conservativity:𝑟𝐹 𝐴, 𝐵 ⇒ 𝑟𝐹 𝐴, 𝐴 ∩ 𝐵

• How about the other direction?𝑟𝐹 𝐴, 𝐴 ∩ 𝐵 ⇒ 𝑟𝐹 𝐴, 𝐵


𝑟𝐹 𝐴, 𝐴 ∩ 𝐵 ⇒ 𝑟𝐹 𝐴, 𝐵

Challenge:• The inference engine is based on forward chaining:

• Always try to deduce all possible implications from given premises• Efficient• Opens the possibility of adapting DCS for entailment

generation



Challenge:• The inference engine is based on forward chaining• Therefore it’s infeasible to enumerate all forms 𝑋 = 𝐴 ∩ 𝐵

when 𝑟𝐹 𝐴, 𝑋 is claimed• Number of possibilities explodes exponentially

• e.g. 𝑋 = 𝑋 ∩ 𝐶 ∀𝐶, 𝑋 = 𝐴 ∩ 𝐵 ∩ 𝐶 = 𝐴 ∩ 𝐵 ∩ 𝐶



Implementation: limit search using conditions 𝑋 ⊆ 𝐴 ∧ 𝑋 ⊆ 𝐵

If 𝑟𝐹 𝐴, 𝑋 and 𝑋 ⊆ 𝐴:• For each 𝐵 ⊇ 𝑋:

• Check if 𝑋 = 𝐴 ∩ 𝐵

We emphasize this detail because formal semantic researchers are often not aware of these difficulties.

Encoding GQs as RelationsLimitations


Limitation:

Relations in DCS trees are always explained as having the widest scope, hence cannot deal with multiple relations in a sentence.


Example:𝑃: At most 10 commissioners spend a lot of time at home.

We want to state𝑟𝐴𝑡𝑀𝑜𝑠𝑡 10 𝐜𝐨𝐦𝐢𝐬𝐬𝐢𝐨𝐧𝐞𝐫𝐬, 𝐷

where 𝐷 = “people who spend a lot of time at home”

But this is impossible if “a lot of” is also encoded as a relation


Example:𝑟𝐴𝑡𝑀𝑜𝑠𝑡 10 𝐜𝐨𝐦𝐢𝐬𝐬𝐢𝐨𝐧𝐞𝐫𝐬, 𝐷

𝐷 = "people who spend a lot of time at home"

Workaround:Since “a lot of” is upward-entailing in predicate argument, we can encode it using selection 𝑠𝐴𝐿𝑜𝑡𝑂𝑓, while still encode “at

most 10” using 𝑟𝐴𝑡𝑀𝑜𝑠𝑡 10


Example:𝑟𝐴𝑡𝑀𝑜𝑠𝑡 10 𝐜𝐨𝐦𝐢𝐬𝐬𝐢𝐨𝐧𝐞𝐫𝐬, 𝐷

𝐷 = 𝑞⊆𝑂𝐵𝐽

𝐷′, 𝑠𝐴𝐿𝑜𝑡𝑂𝑓 𝐭𝐢𝐦𝐞

where

𝐷′ = 𝐬𝐩𝐞𝐧𝐝 ∩ 𝑊𝑆𝐵𝐽 ×𝑊𝑂𝐵𝐽 × 𝐡𝐨𝐦𝐞𝑀𝑂𝐷

(“spend at home”)

Evaluation

EvaluationSet-up

The FraCaS Corpus:• Built in mid-1990s• A set of hand-crafted entailment problems covering

wide range of semantic phenomena

Section 1 - Generalized Quantifiers:• 74 problems:

• 44 have single premise sentence• 30 have multiple premise sentence

EvaluationSet-up

Settings:• Baseline• Selection• Relation• Selection+Relation

EvaluationSet-up

Settings:• Baseline

• Simply drop GQs• Same tree structure as follows

• Selection• Relation• Selection+Relation

EvaluationSet-up

Settings:• Baseline• Selection

• Implement all GQs as selections, even for those that are downward-entailing in predicate argument

• Relation• Selection+Relation

EvaluationSet-up

Settings:• Baseline• Selection• Relation

• Implement all GQs as relations• Selection+Relation

EvaluationSet-up

Settings:• Baseline• Selection• Relation• Selection+Relation

• Use relations to encode GQs that are downward-entailing in predicate argument

• Encode the rest with selections

Evaluation

SystemAccuracy





Gold Syntax 88.6% 80.0% 85.1%

TIFMO

Baseline 79.5% 86.7% 82.4%

Selection 90.9% 93.3% 91.9%

Relation 88.6% 93.3% 90.5%

Selection+Relation 93.2% 96.7% 94.6%


Conclusion

Conclusion

• Generalized Quantifiers are important (for RTE)

• We explored ways of encoding GQs in DCS for RTE• via selection extension• via relation extension (newly proposed)

• Significant improvement in performance, but not perfect• which suggests towards more powerful logical systems

encoding generalized quantifiers in dependency-based compositional semantics

Presentations & Public Speaking