dependent random variables

Dependent Random Variables

Stanford CS 329P (2021 Fall) - https://c.d2l.ai/stanford-cs329p

Relative Change

• Influence of variables

• Influence is based on the change in value

• We need to determine the reference value to assess

whether a feature matters.

• Data type dependent, but means are a good way to start (images, text, tabular data, audio, etc.)

Δx

x0

i

f(x) = w⊤x + b =

n

∑i=1

wixi + bx0 Δx

Δx

x0

https://c.d2l.ai/stanford-cs329p


Distributions, joint, marginal, expectations

• Want to know the influence of on

• Direct influence via function (that’s what we’re after)

• Confounded by indirect influence on other features

(from on )

• Options

• Use conditional expectation

• Use marginal

xi f(x)

xi x−i

p(x−i |xi)

p(x−i)



Backdoor Problem

• are conditionally indep.

• Observed distribution

• Conditional is dependent

x1, x2

p(x1, x2) ≠ p(x1)p(x2)

x1 x2

zLatent

Observed

p(x1, x2) = ∫ dp(z)p(x1 |z)p(x2 |z)

trustworthy?



• Credit rating based on observed attributes

• Changing observed attributes will not change underlying condition.

Approve Credit?

x1 x2

zLatent

Observed

Fix thisDraw

independently

f (x)



Causal Connection

x1 x2

f (x)



More Context

• Pearl’s do operator

• Use samples from marginals rather than conditional distribution. This is easier and usually more correct.

E[y |do(X1 = x1)] = ∫ dp(x2, x3)E[y |x1, x2, x3]



The Parliament of Micronesia



The Parliament of Micronesia

The parliament of Micronesia is made up of four political parties, A, B, C, and D, which have 45, 25, 15, and 15 representatives, respectively. They are to vote on whether to pass a $1M spending bill and how much of this amount should be controlled by each of the parties. A majority vote, that is, a minimum of 51 votes, is required in order to pass any legislation, and if the bill does not pass then every party gets zero to spend.

Winning coalitions are (A,B), (A,C), (A, D), (B, C, D) and any

superset of these. Payoff function for set .v(S) S



Why do we care?

• Replace parties with features • Replace payoff with influence



Axioms

• Symmetry

For any payoff function not containing or with

we need the reward to satisfy .

• Dummy Player

For any without with we

need the reward to satisfy .

v(S) i j

ϕ(i) = ϕ( j)

S i v(S ∪ {i}) = v(S) + v({i})

ϕ(i) = v({i})

v(S ∪ {i}) = v(S ∪ {j}) for all S



Axioms

• Additivity

For any payoff function the rewards are also

additive, i.e.

• Shapley Value Theorem The payoff function satisfying symmetry, dummy and additivity that divides the payoff the grand coalition (everyone contributes) is uniquely defined by the Shapley value of the game.

v = v1 + v2

ϕ(i) = ϕ1(i) + ϕ2(i)



Shapley Value Function

• Definition

• Properties

• Average over all set sizes and over all ways to

choose . Normalized by cardinality.

• Add up all incremental contributions.

• Easy to check that it satisfies the axioms. Uniqueness is a bit more work.

|N |

S ∈ N \{i}

ϕ(i, N ) = ∑S∈N \{i}

1

|N | (|N | − 1

|S | )−1

[v(S ∪ {i}) − v(S)]



Back to Micronesia

• A (45), B (25), C (15), or D (15) have individual payoff 0.

• Any coalition including A succeeds (B, C, D are interchangeable)

• The only successful coalition without A is (B, C, D).

• Going over the definition yields payoffs

A =1

2 and B = C = D =

1

6

Back to Feature Scoring - use this for f(x)


dependent random variables

Documents