decision making as a model 3. heavy stuff: derivation of two important theorems
TRANSCRIPT
Recap: two types of measures of sensitivity (independent of criterion:)
2. Area under ROC-Curve: A
1. Distance between signal and noise distributions
cf. d'
Recap: four types of measures for criterion:
1. Likelihood ratio LRc = p(xc|S)/p(xc|N) = h/f (vgl β)
hf
2. Position on x-axis (c)
3. Position in ROC-plot (left down vs. right up)
4. Slope of tangent at point of ROC (S)
c
Interpreting A
Area theorem:
A is equivalent with proportion correct answers in 2AFC-experiment:
Given:1 noise stimulus1 signal (+noise) stimulus,Which is which?
Makes sense. Important
Produce a formula for proportion correct in a 2AFC-experiment (Pc)
Produce a formula for area under ROC-curve (A)
Show that the formula for A looks like the formula for Pc
Show that formulas are identical.
Approach:
PFA
PH
ffnn ffss
xx0 0 λλ
∞ PH = ∫ fs(x)dx
λ ∞
PFA = ∫ fn(x)dx λ
= H(λ)
= FA(λ)
λ = FA-1(PFA)
ROC-curve: PH = H(λ) = H[FA
-1(PFA)]
Recap: In general:
Specific model depends on fn and fs
ffnnffss
xx0 0 λλ
∞ PH = ∫ fs(x)dx
λ ∞ PFA = ∫ fn(x)dx
λ
= H(λ)
= FA(λ)
Reinterpretation for 2A FC experiment:
Two alternatives correspond with two points on the x-axis. Suppose λ is noise stimulus:
if xn = λ PC = p(xs>xn), p(xs>xn) = H(λ)
“summate” H(λ) for every λ ,weighted for density of λ [= fn(λ)]:
∞ PC = ∫ H(λ)fn(λ)dλ
-∞
PFA
PH
ffnn ffss
xx0 0 λλ
∞ PH = ∫ fs(x)dx
λ ∞
PFA = ∫ fn(x)dx λ
= H(= H(λλ))
= F= FAA((λλ))
ROC-curve: ROC-curve: PPHH [= H( [= H(λλ)] as a )] as a
function of Pfunction of PFAFA [ [= F FAA((λλ)])]
1 A = ∫ H(λ)dFA(λ) 0
Area under Roc-curve:
∞ PC = ∫ H(λ)fn(λ)dλ
-∞ A looks like PC ; A is PC ; can be proved
proof (optional):
dFA(λ) d(λ)------- = -fn(λ)
∞ PC = ∫ H(λ)fn(λ)dλ
-∞
dFA(λ) = -fn(λ)dλ
Still two small chores:Limits of integration and minus sign
ffnn(x)dx = 1 - f(x)dx = 1 - fnn(x)dx(x)dx∞
λ∫
-∞
λ
∫
1 A = ∫ H(λ)dFA(λ) 0 -f-fnn((λλ)d)dλλ
Limits: if FA(λ)=PFA= 0 then λ = ∞if FA(λ)=PFA= 1 then λ = -∞
reverse: -H(λ)fn(λ) H(λ)fn(λ)
∞ PC = ∫ H(λ)fn(λ)dλ
-∞
∫
∫ 0
1 -∞
∞
∞
∫
∫ -∞
-∞
∞
ffnnffss
xx0 0 λλ-∞ ∞ 1 A = ∫ H(λ)dFA(λ) 0 -f-fnn((λλ)d)dλλ
fromIntegration over FA
tointegration over λ ?
∫ -H(λ)fn(λ)dλ ?
PFA
PH
ROC-curve: PH as a function of PFA
Every point of ROC-curve gives criterion/bias at that sensitivity
Slope tangent at that point as measure for bias/criterium S = .49
dPHSlope S = ------ dPFA
Measures for criterion
ffnn ffss
xx0 0 λλ
∞ PH = ∫ fs(x)dx
λ ∞
PFA = ∫ fn(x)dx λ
dPH dPH dPFA ----- = ----- • ------.dx dPFA dx
d(1-PFA) dPFA dPFA dPH---------- = - ----- = fn , ----- = - fn , also: ------ = - fs …dx dx dx dx
(chain rule)
dPH dPH/dx S = ----- = -----------
dPFA dPFA/dx
- fs fs= ----- = ----- - fn fn
= LRc
Measures for criterion
dPH dPHfrom ----- to ------dx dPFA
Finite State models
High threshold: Yes 1 α detect signal
1-α η Yes uncertain
1-η No
1 η Yes noise uncertain
1-η No
HitHit
Hit Hit
MissMiss
FAFA
crcr
PH = α +η(1-α)PFA = η
Hits
Hits
False AlarmsFalse Alarms
αα
PH = α +η(1-α)PFA = η
ηη
Theoretical ROC curve
detect: Yes
uncertain: η Yes1-η No
α
“high threshold”
PH = α +η(1-α) PFA = η
PH = α + PFA(1-α)
PH = α + PFA - αPFA
α – αPFA = PH - PFA
α(1- PFA) = PH - PFA
PH – PFAα = -------------
1 - PFA
PH = α +η(1-α)
Pm = (1-η)(1-α)
Pm (1-α) = -------
(1-η)
Pm PH =α + η ------- (1-η)
η α = PH - ---- Pm (1-η)
Cf correction for guessing MC-questions:
N-AFC G Hits, F misses
η Sc = G - ------ F (1-η)
Analogously: a low threshold model :Signal leads always to uncertain state
noise leads with P = β to nondetect state (always NO) and else to uncertain state.
hits
False Alarms
1-β
β
Nondetect: No
Uncertain: η Yes1-η No