cs262 problem session - stanford university · 2016. 1. 12. · !"#$%&'(f*1,*:, (! %...
TRANSCRIPT
!"#$%&'()
! *+,(-./012(34&(56'(#7(8&9:4;(#7(#6;:#9<:(&=:&5('65;(56'(;#(#<&2(><?#'9<:(&=:&5@(56'(4+5(<#("&5;"9?;9#<52
! *$,(-./012(34&(&=:&5(*5AB;,(=&<#;&5(;4&(C"#$+$9%9;D(!(:9E&<("2
! *?,(-./012(!*FG(H,(A(!*H),!*F
)IH
),!*H
JIH
),222
! *=,(3KL12(M9;&"$9(=#&5(N.O(#7(<6'$&"5(849%&(-#"8+"=(=#&5(0LN(#7(<6'$&"52(09<?&(+%%(<6'$&"5(+"&(<#<P<&:+;9E&G(;4&(56'(95(:"&+;&"(;4+<(#"(&Q6+%(;#(;4&('+F2
! *&,(3KL12(34&D(+"&($#;4(R*STSTU,(84&"&(S(A(V(5;+;&5(+<=(U(A(5&Q6&<?&(%&<:;42
! *7,(-./012(>;(:9E&5(;4&(C"#$+$9%9;D(=95;"9$6;9#<(#E&"(5;+;&5(+;(&+?4(C#59;9#<2! *:,(-./012(M+<9%%+(WNN5(+"&(:##=(7#"(9<;"#<9?(+<=(<#;(5#(:##=(7#"(&F#<9?("&:9#<52(R<%D(;4&(9<;"#<9?("&:9#<5@(%&<:;4(5&&'5(;#(4+E&(+(=95;"9$6;9#<(;4+;(95(<&+"%D(:&#'&;"9?2
! *4,(-./012(X&(?+<(65&(49:4&"P#"=&"(WNN5(;#('#=&%(*?#<=9;9#<(#<,(;8#(#"('#"&(C"&E9#65(;9'&5;&C52
! *9,(-./012(><=&&=(1N(95(:6+"+<;&&=(;#(?#<E&":&(;#(+(5#%6;9#<G(;4#6:4(<#;(<&?&55+"9%D(;4&(:%#$+%%D(#C;9'+%(#<&2
!
Problem 2(a)
00
00101 11
iki
iki
klkl
i
lilk
i
lilklkkl
AAAa
xPxPibxeif
xPibxeaifA
Problem 2(b) • Baum‐Welch: Suppose Forward: Similar for Backward
bebeaaaa kkkllklkkl ,,
ifaifxe
aifxeif
kl
kllik
llklikk
1
1
Problem 2(b) • Baum‐Welch:
bEibifibifbE
AxP
ibxeaifxP
ibxeaifA
AxP
ibxeaifxP
ibxeaifA
kbxi
kkbxi
kkk
kli
kikkll
i
kiklkllk
lki
lillkk
i
lilklkkl
ii
11
11
11
11
Problem 2(b) • Baum‐Welch:
becE
bEcE
bEbe
aA
AA
Aa
aA
AA
Aa
k
ck
k
ck
kk
kl
iki
kl
iik
lklk
lk
iik
lk
iki
klkl
Problem 2(b) • Baum‐Welch: Given Inductive step: After training:
000000 ,, kkkllklkkl ebeaaaa
111111 ,,
,,
ik
ik
ikl
ilk
ilk
ikl
ik
ik
ikl
ilk
ilk
ikl
ebeaaaa
ebeaaaa
Nk
Nk
Nkl
Nlk
Nlk
Nkl ebeaaaa ,,
Problem 2(b) • Viterbi:
Viterbi parse may arbitrarily choose state k over state
k’ Akl Ak’l a’kl a’k’l
xkP
xexexeaaaaa
xexexeaaaaaxkP
i
Nikkk
Nikkki
NNNii
NNNii
, where
, where
10
10
1111211
1111211
π
π
Problem 2(c)
1P(x)=1P(y)=0
2P(x)=0P(y)=1
1/21 xxyx
1121
akl l=1 2
k=0 1 0
1 1/2 1/2
2 1 0
ek(b) b=x y
k=1 1 0
2 0 1
Akl l=1 2
k=0 1 0
1 1 1
2 1 0
Ek(b) b=x y
k=1 3 0
2 0 1
Problem 2(c)
x y x x
0
1 .9 .045 .3645 .1640
2 0 .405 0 0
Viterbi
xxyx1121
akl l=1 2
k=0 1
1 1/2 1/2
2 1
ek(b) b=x y
k=1 1 0
2 0 1
Problem 2(c) x y x x
0
1 .75 .1688 .1139 .0769
2 0 .0375 .0084 .0057
Viterbi xxyx1111
akl l=1 2
k=0 1
1 0.9 0.1
2 1
ek(b) b=x y
k=1 0.75 0.25
2 0.5 0.5
akl l=1 2
k=0 1
1 1 0
2 1
ek(b) b=x y
k=1 0.75 0.25
2 ? ?
Problem 3(a)
most likely sequence of states π, given the observed sequence x
sequence of states π for which the observed sequence x was most likely emitted
Problem 3(a)
Fair P(H) = 0.5 P(T) = 0.5
Loaded P(H) = 0 P(T) = 1
1 – 10‐googol 10‐googol
1 – 10‐googol 10‐googol
10‐googol
1 – 10‐googol
x = TTT
π = ?
Loaded Loaded Loaded? Fair Fair Fair?
Problem 3(a)
Fair P(H) = 0.5 P(T) = 0.5
Loaded P(H) = 0 P(T) = 1
1 – 10‐googol 10‐googol
1 – 10‐googol 10‐googol
10‐googol
1 – 10‐googol
Problem 3(a)
Fair P(H) = 0.5 P(T) = 0.5
Loaded P(H) = 0 P(T) = 1
1 – 10‐googol 10‐googol
1 – 10‐googol 10‐googol
10‐googol
1 – 10‐googol
Problem 3(b)
πi* πi Win(πi*, πi)
F F +CF F L ‐WL
L F ‐WF
L L +CL
Problem 3(b)
Problem 3(c)(i) 1 2 3 …
fF[0]
fL[0]
fF[1]
fL[1]
fF[2]
fL[2]
fF[599]
fL[599]
fF[600]
fL[600]
: :
Fair Loaded
“generate first i characters of x with exactly k loaded rolls, ending in state Fair.”
“generate first i characters of x with exactly k loaded rolls, ending in state Loaded.”
Problem 3(c)(i) … 998 999 1000
bF[0]
bL[0]
bF[1]
bL[1]
bF[2]
bL[2]
bF[599]
bL[599]
bF[600]
bL[600]
: :
Fair Loaded
“start state i in Fair, generate remaining characters of x with exactly k more loaded rolls.”
“start state i in Loaded, generate remaining characters of x with exactly k more loaded rolls.”
Problem 3(c)(i) “generate first i characters of x with exactly k loaded rolls, ending in state Fair.”
“generate first i characters of x with exactly k loaded rolls, ending in state Loaded.”
“start state i in Fair, generate remaining x with exactly k more loaded rolls.”
“start state i in Loaded, generate remaining x with exactly k more loaded rolls.”
Running time? Sequence length S # loaded rolls R
Problem 3(c)(ii)
Same as problem 3(b): Independently maximize expected payoff at each position.
Problem 4(a)
x \ y A C G T –
A pm ps ps ps pd
C ps pm ps ps pd
G ps ps pm ps pd
T ps ps ps pm pd
– pd pd pd pd 0
MIJ
1
pair‐HMM NW
pm m
ps ‐s
pd ‐d
Problem 4(b)
X prefix P(xi)
Y prefix P(yj)
X suffix P(xi)
Y suffix P(yj)
Overlap P(xi,yj)
start end τ
(1‐τ)/2
(1‐τ)/2
β
β β
β
α
τ
(1‐α‐τ)/2
(1‐α‐τ)/2 1‐β
1‐β
1‐β
1‐β
Problem 4(c)
M P(xi,yj)
I1 P(xi)
I2 P(xi)
IL P(xi) δ
ε ε
1‐ε 1‐ε 1
J1 P(yj)
J2 P(yj)
JL P(yj) ε ε
δ
1‐ε 1‐ε 1
1‐2δ
Problem 4(d)
k
γ(k)
d2
d3
e1
e2
e3
Problem 4(d)
M s(xi,yj)
I1 s(xi)
J1 s(yj)
1‐d1‐…‐ds‐d’1‐….‐d’s
e1
e’1
d1
I2 s(xi)
J2 s(yj)
e’2
d2
d’2
e2
Is s(xi)
es
Js s(yj)
e’s
ds
d’s
d’1
1‐e’1
1‐e’2 1‐e’s
1‐es 1‐e2
1‐e1