distinguishing between cause and effect: estimation of...

Distinguishing between Cause and Effect:Estimation of Causal Graphs with two Variables

Jonas PetersETH Zurich

Tutorial

NIPS 2013 Workshop on Causality9th December 2013

F. H. Messerli: Chocolate Consumption, Cognitive Function, and Nobel Laureates, N Engl J Med 2012

Jonas Peters (ETH Zurich) Distinguishing between Cause and Effect 9th December 2013

Problem: Given P(X ,Y ), can we infer whether

X → Y or Y → X ?

Difficulty: So much symmetry:

P(X ) · P(Y |X ) = P(X ,Y ) = P(X |Y ) · P(Y )

We need assumptions!! (e.g. Markov and faithfulness do not suffice.)

Surprise (for some assumptions):

2 variables ⇒ p variables

J. Peters, J. Mooij, D. Janzing and B. Scholkopf: Causal Discovery with Continuous Additive Noise Models, arXiv:1309.6779

X → Y or Y → X ?

P(X ) · P(Y |X ) = P(X ,Y ) = P(X |Y ) · P(Y )

X → Y or Y → X ?

P(X ) · P(Y |X ) = P(X ,Y ) = P(X |Y ) · P(Y )

Idea No. 1: Linear Non-Gaussian Additive Models (LiNGAM)

Structural assumptions like additive non-Gaussian noise models break thesymmetry:

Y = βX + NY NY ⊥⊥ X ,

with NY non-Gaussian.

Asymmetry No. 1

Consider a distribution corresponding to

Y = βX + NY

• NY ⊥⊥ X• NY non-Gaussian

Then there is no

X = φY + NX

• NX ⊥⊥ Y• NX non-Gaussian

S. Shimizu, P.O. Hoyer, A. Hyvarinen and A.J. Kerminen: A linear non-Gaussian acyclic model for causal discovery, JMLR 2006

Asymmetry No. 1

Y = βX + NY

• NY ⊥⊥ X• NY non-Gaussian

Then there is no

X = φY + NX

• NX ⊥⊥ Y• NX non-Gaussian

S. Shimizu, P.O. Hoyer, A. Hyvarinen and A.J. Kerminen: A linear non-Gaussian acyclic model for causal discovery, JMLR 2006

Idea No. 2: Additive noise models

Nonlinear functions are also fine!

Y = f (X ) + NY NY ⊥⊥ X

P. Hoyer, D. Janzing, J. Mooij, J. Peters and B. Scholkopf: Nonlinear causal discovery with additive noise models, NIPS 2008

Asymmetry No. 2

Y = f (X ) + NY

with NY ⊥⊥ X

Then for “most combinations” (f ,P(X ),P(NY )) there is no

X = g(Y ) + MX

with MX ⊥⊥ Y

Asymmetry No. 2

Y = f (X ) + NY

with NY ⊥⊥ X

Then for “most combinations” (f ,P(X ),P(NY )) there is no

X = g(Y ) + MX

with MX ⊥⊥ Y

Y = f (X ) + NY , NY ⊥⊥ X

X = g(Y ) + NX , NX ⊥⊥ Y

Idea No. 3: Gaussian Process Inference (GPI)

We can always write∗

Y = f (X ,NY ), NY ⊥⊥ X

andX = g(Y ,NX ), NX ⊥⊥ Y

Which model is more “complex”? Use Bayesian model comparison.

J. M. Mooij, O. Stegle, D. Janzing, K. Zhang, B. Scholkopf:

Probabilistic latent variable models for distinguishing between cause and effect, NIPS 2010

∗E.g., J. Peters: Restricted Structural Equation Models for Causal Inference, PhD Thesis

Asymmetry No. 3

1 Fix the noise distribution to be N (0, 1).

2 Put prior p(θX ) on input distribution p(x | θX ) ( complexity of X ).

3 Put prior p(θf ) on the functions p(f | θf ) ( complexity of f ).

4 Approximate marginal likelihood for X → Y

p(x , y) = p(x) · p(y | x)

∫p(x | θX )p(θX ) dθX

·∫δ(y − f (x , e)

)p(e)p(f ) de df

5 Approximate marginal likelihood for Y → X .

6 Compare.

Asymmetry No. 3

1 Fix the noise distribution to be N (0, 1).

2 Put prior p(θX ) on input distribution p(x | θX ) ( complexity of X ).

3 Put prior p(θf ) on the functions p(f | θf ) ( complexity of f ).

4 Approximate marginal likelihood for X → Y

p(x , y) = p(x) · p(y | x)

∫p(x | θX )p(θX ) dθX

·∫δ(y − f (x , e)

)p(e)p(f ) de df

5 Approximate marginal likelihood for Y → X .

6 Compare.

Idea No. 4: Information Geometric Causal Inference (IGCI)

Assume a deterministic relationship

Y = f (X )

and that f and P(X ) are “independent”.

D. Janzing, J. M. Mooij, K. Zhang, J. Lemeire, J. Zscheischler, P. Daniusis, B. Steudel, B. Scholkopf:

Information-geometric approach to inferring causal directions, Artificial Intelligence 2012

Idea No. 4: Information Geometric Causal Inference (IGCI)

Assume a deterministic relationship

Y = f (X )

and that f and P(X ) are “independent”.

Asymmetry No. 4

Consider Y = f (X ) with id 6= f : [0, 1]→ [0, 1] invertible and X = g(Y ).If

“cov”(log f ′, pX ) =

∫log(f ′(x)) pX (x) dx −

∫log f ′(x) dx = 0

“cov”(log g ′, pY ) =

∫log(g ′(y)) pY (y) dy −

∫log g ′(y) dy > 0

Open Questions 1: Quantifying Identifiability

Proposition

Assume P(X ,Y ) is generated by

Y = f (X ) + NY

with independent X and NY .

Theninf

Q∈{Q:Y→X}KL(P ||Q) = ?

first steps to understand the geometry

gives us finite sample guarantees

Open Questions 2: Robustness

What happens if assumptions are violated? E.g., in case of confounding?

Can we still infer X → Y ? How useful is this?

Conclusions

In theory, we can brake asymmetry between cause and effect.

restricted structural equation models:- linear functions, additive non-Gaussian noise- nonlinear functions, additive noise

complexity measures on functions and distributions

“independence” between function and input distribution

... principles behind new methods from challenge?

Causal inference prob-

lem of climate change is

solved! Fight the cause!

Don’t fly! (Zurich-SFO 5.4t CO2)!

Compensate!

Conclusions

Compensate!

Conclusions

Compensate!

It turns out that if X → Y∫log |f ′(x)|p(x) dx <

∫log |g ′(y)|p(y) dy

Estimator:

CX→Y :=1

m∑j=1

∣∣∣∣yj+1 − yjxj+1 − xj

∣∣∣∣ ≈ ∫ log |f ′(x)|p(x) dx

Infer X → Y ifCX→Y < CY→X

Y = βX + NY , NY ⊥⊥ X , NY non-Gaussian

0.0 0.2 0.4 0.6 0.8 1.0

Y = βX + NY , NY ⊥⊥ X , NY non-Gaussian

0.0 0.2 0.4 0.6 0.8 1.0

X = φY + NX , NX ⊥⊥ Y , NX non-Gaussian

0.0 0.2 0.4 0.6 0.8 1.0

X = φY + NX , NX ⊥⊥ Y , NX non-Gaussian

0.0 0.2 0.4 0.6 0.8 1.0

Does X cause Y or vice versa?Real Data

Does X cause Y or vice versa?

No (not enough) data for chocolate

... but we have data for coffee!

No (not enough) data for chocolate

... but we have data for coffee!

0 2 4 6 8 10 12

coffee consumption per capita (kg)

Correlation: 0.698, p-value: < 2.2 · 10−16.

Nobel Prize→ Coffee: Dependent residuals (p-value of 0).Coffee→ Nobel Prize: Dependent residuals (p-value of 0).

⇒ Model class too small? Causally insufficient?

0 2 4 6 8 10 12

coffee consumption per capita (kg)

Correlation: 0.698, p-value: < 2.2 · 10−16.

Nobel Prize→ Coffee: Dependent residuals (p-value of 0).Coffee→ Nobel Prize: Dependent residuals (p-value of 0).

⇒ Model class too small? Causally insufficient?Jonas Peters (ETH Zurich) Distinguishing between Cause and Effect 9th December 2013

The linear Gaussian case

Y = βX + NY

with independent

X ∼ N (0, σ2X ) and

N ∼ N (0, σ2NY

Then there is a linear SEM with

X = αY + MX

How can we find α and MX ?

distinguishing between cause and effect: estimation of...

Documents

trig graphs. y = sin x y = cos x y = tan x y = sin x + 2

plot (x-values, y-values) >> x = linspace (-3, 3, 20); >> x...

town of minocquatrl voting wards · 2018. 1. 3. · xy xy...

linear and affine functions - slope-intercept form...-3-65)...

examples and sets. synthesize each of these choose((x,y)...

imag · #typedef unsigned if_sqn_set_type; #define...

x= {x 0, x 1,….,x j-1 } y= {y 0, y 1, ….,y k-1 } channel...

xy xy xy x x y xy xy x y x y xy x y xy x y x y xy x yx y x y...

wave...

algebra days 1-10 · a2 a x a y³ ÷ y 4 = y³ = y x y x y...

do now describe the translations in words. 1. 1. (x, y) ...

reflection 2 reflecteach shape in the mirror line. a) 1 a...

recap from last time: live variables x := 5 y := x + 2 x :=...

introduction - arxiv · m(x,x,y) = m(x,y,x) = m(y,x,x) =...

summer work packet for students entering algebra 1...

acrylated silicones suitable for sla 3d...

1 member(x,[y| _ ] ) :- x = y. member(x, [ _ | y]) :-...

reflection 2 reflecteach shape in the mirror line. a) 1 a)...

city of athens official zoning map august 21, 2014 map...

nmhc internship manual 2019-2020 final nmhc...