parameter estimation for hmm
DESCRIPTION
Parameter Estimation For HMM. Background Readings : Chapter 3.3 in the book, Biological Sequence Analysis , Durbin et al., 2001. M. M. M. M. S 1. S 2. S L-1. S L. T. T. T. T. x 1. x 2. X L-1. x L. Reminder: Hidden Markov Model. - PowerPoint PPT PresentationTRANSCRIPT
.
Parameter Estimation For HMM
Background Readings: Chapter 3.3 in the book, Biological Sequence Analysis, Durbin et al., 2001.
2
Reminder: Hidden Markov Model
Markov Chain transition probabilities: p(Si+1= t|Si = s) = ast
Emission probabilities: p(Xi = b| Si = s) = es(b)
S1 S2 SL-1 SL
x1 x2 XL-1 xL
M M M M
TTTT
11 11
( , ) ( , , ; ,..., ) ( )i i i
L
L L s s s ii
p p s s x x a e xs x
3
Reminder: Most Probable state pathS1 S2 SL-1 SL
x1 x2 XL-1 xL
M M M M
TTTT
Given an output sequence x = (x1,…,xL),
A most probable path s*= (s*1,…,s*
L) is one which maximizes p(s|x).
1( ,..., )
* *1 1 1* ( ,..., ) ( ,..., | ,..., )maxarg
Ls s
L L Ls s s p s s x x
4
Reminder: Viterbi’s algorithm for most probable paths1 s2 sL-1 sL
X1 X2 XL-1 XL
si
Xi
For i=1 to L do for each state l :
vl(i) = el(xi) MAXk {vk(i-1)akl }ptri(l)=argmaxk{vk(i-1)akl} [storing previous state for reconstructing the path]Termination: the probability of the most probable path
Initialization: v0(0) = 1 , vk(0) = 0 for k > 0
0
We add the special initial state 0.
p(s1*,…,sL
*;x1,…,xl) = { ( )}max kk
v L
5
A- C- T- T+
A C T T
G +
G
Predicting CpG islands via most probable path:
Output symbols: A, C, G, T (4 letters). Markov Chain states: 4 “-” states and 4 “+” states, two for each letter (8 states).The transitions probabilities ast and ek(b) will be discussed soon.The most probable path found by Viterbi’s algorithm predicts CpG islands.
Experiment (Durbin et al, p. 60-61) shows that the predicted islands are shorter than the assumed ones. In addition quite a few “false negatives” are found.
6
Reminder: finding most probable state
1. The forward algorithm finds {fk(si) = P(x1,…,xi,si=k): k = 1,...m}.2. The backward algorithm finds {bk(si) = P(xi+1,…,xL|si=k ): k = 1,...m}.3. Return {p(Si=k|x) = fk(si) bk(si) |k=1,...,m}.
To Compute for every i simply run the forward and backward algorithms once, and compute {fk(si) bk(si)} for every i, k.
s1 s2 sL-1 sL
X1 X2 XL-1 XL
si
Xi
fl(i) = p(x1,…,xi,si=l ), the probability of a path which emits (x1,..,xi) and in which state si=l. bl(i)= p(xi+1,…,xL,si=l), the probability of a path which emits (xi+1,..,xL) and in which state si=l.
7
Finding the probability that a letter is in a CpG island via the algorithm for most probable state:
The probability that an occurrence of G is in a CpG island (+ state) is:
∑s+ p(Si =s+ |x) = ∑s+ F(Si=s+ )B(Si=s+
)
Where the summation is formally over the 4 “+” states, but actually
only state G+ need to be considered (why?)
A- C- T- T+
A C T T
G +
G
8
Parameter Estimation for HMM
An HMM model is defined by the parameters: akl and ek(b), for all states k,l and all symbols b. Let θ denote the collection of these parameters.
s1 s2 sL-1 sL
X1 X2 XL-1 XL
si
Xi
lk
b
akl
ek(b)
9
Parameter Estimation for HMM
To determine the values of (the parameters in) θ, use a training set = {x1,...,xn}, where each xj is a sequence which is assumed to fit the model.Given the parameters θ, each sequence xj has an assigned probability p(xj|θ) (or p(xj| θ,HMM)).
s1 s2 sL-1 sL
X1 X2 XL-1 XL
si
Xi
10
ML Parameter Estimation for HMM
The elements of the training set {x1,...,xn}, are assumed to be independent, p(x1,..., xn|θ) = ∏j p (xj|θ).
ML parameter estimation looks for θ which maximizes the above.
The exact method for finding or approximating this θ depends on the nature of the training set used.
11
Data for HMM
Possible properties of (the sequences in) the training set:1.For each xj, what is our information on the states si (the
symbols x i are usually known).
2.The size (number of sequences) of the training set
S1 S2 SL-1 SL
x1 x2 XL-1 xL
M M M M
TTTT
12
Case 1: Sequences are fully known
We know the complete structure of each sequence in the training set {x1,...,xn}. We wish to estimate akl and ek(b) for all pairs of states k, l and symbols b.
By the ML method, we look for parameters θ* which maximize the probability of the sample set: p(x1,...,xn| θ*) =MAXθ p(x1,...,xn| θ).
s1 s2 sL-1 sL
X1 X2 XL-1 XL
si
Xi
13
Case 1: ML method
s1 s2 sL-1 sL
X1 X2 XL-1 XL
si
Xi
Let mkl= |{i: si-1=k,si=l}| (in xj). mk(b)=|{i:si=k,xi=b}| (in xj).
( )
( , ) ( , )
then: ( | ) [ ( )]kl km m bkkl
k l k b
jp a e bx
For each xj we have:
j
iii
L
i
jisss
j xeaxp1
1)()|(
14
Case 1 (cont)
s1 s2 sL-1 sL
X1 X2 XL-1 XL
si
Xi
By the independence of the xj’s, p(x1,...,xn| θ)=∏ip(xj|θ).
Thus, if Akl = #(transitions from k to l) in the training set, and Ek(b) = #(emissions of symbol b from state k) in the training set, we have:
( )
( , ) ( , )
1 ( | ) [ ( )],.., kl kA E bkkl
k l k b
np a e bx x
15
Case 1 (cont)
( )
( , ) ( , )
[ ( )]kl kA E bkkl
k l k b
a e b
So we need to find akl’s and ek(b)’s which maximize:
Subject to:
])(,
)(,
0 [
1 and 1 states allFor
bea
beak
kkl
l bkkl
16
Case 1 (cont)
( )
( , ) ( , )
( )
[ ( )]
[ ( )][ ] [ ]
kl k
kl k
A E bkkl
k l k b
A E bkkl
k l k b
F a e b
a e b
kb
Subject to: for all , 1, and e ( ) 1.kll
k a b
Rewriting, we need to maximize:
17
Case 1 (cont)
If we maximize for each : s.t. 1klAklkl
ll
k a a ( )and also [ ( )] s.t. ( ) 1kE b
k kbb
e b e b
Then we will maximize also F.Each of the above is a simpler ML problem, which is treated next.
18
A simpler case: ML parameters estimation for a die
Let X be a random variable with 6 values x1,…,x6
denoting the six outcomes of a die. Here the parameters areθ ={1,2,3,4,5, 6} , ∑θi=1Assume that the data is one sequence:
Data = (x6,x1,x1,x3,x2,x2,x3,x4,x5,x2,x6)So we have to maximize
2 3 2 21 2 3 4 5 6( | )P Data
Subject to: θ1+θ2+ θ3+ θ4+ θ5+ θ6=1 [and θi 0 ]
252 3 2
1 2 3 4 51
i.e., ( | ) 1 ii
P Data
19
Side comment: Sufficient Statistics
To compute the probability of data in the die example we only require to record the number of times Ni falling on side i (namely,N1, N2,…,N6).
We do not need to recall the entire sequence of outcomes
{Ni | i=1…6} is called sufficient statistics for
the multinomial sampling.
654321
5
154321 1)|(N
i iNNNNNDataP
20
Sufficient Statistics
A sufficient statistics is a function of the data that summarizes the relevant information for the likelihood
Formally, s(Data) is a sufficient statistics if for any two datasets D and D’
s(Data) = s(Data’ ) P(Data|) = P(Data’|)
Datasets
Statistics
Exercise:Define “sufficient statistics” for the HMM model.
21
Maximum Likelihood EstimateBy the ML approach, we look for parameters that maximizes the probability of data (i.e., the likelihood function ).Usually one maximizes the log-likelihood function which is easier to do and gives an identical answer:
63 51 2 4
551 2 3 4 1
log ( | ) log 1N
N NN N Nii
P Data
5
16
5
11loglog
i ii ii NN
A necessary condition for maximum is:
01
)|(log5
1
6
θ
i ij
j
j
NNDataP
22
Finding the Maximum
Rearranging terms:
ii
jj N
N Divide the jth equation by the ith
equation:
Sum from j=1 to 6:
ii
ii
j j
jj N
N
N
N
6
16
1
1
Hence the MLE is given by:
6,..,1 iN
N ii
6
65
1
6
1 NNN
i ij
j
23
Generalization for distribution with any number n of outcomes
Let X be a random variable with n values x1,…,xk denoting the k outcomes of an iid experiments, with parameters θ ={1,2,...,k} (θi is the probability of xi). Again, the data is one sequence of length n:
Data = (xi1,xi2
,...,xin)
Then we have to maximize
1 211 2( | ) , ( ... )knn n
kkP Data n n n
Subject to: θ1+θ2+ ....+ θk=1
11
1
1 11
i.e., ( | ) 1k
k
nknn
iki
P Data
24
Generalization for n outcomes (cont)
i k
i k
n n
By treatment identical to the die case, the maximum is obtained when for all i:
Hence the MLE is given by:
1,..,ii
ni k
n
25
Fractional Exponents
Some models allow ni’s which are not integers (eg, when we are uncertain of a die outcome, and consider it “6” with 20% confidence and “5” with 80%):We still can have
And the same analysis yields:
1,..,ii
ni k
n
1 211 2( | ) , ( ... )knn n
kkP Data n n n
26
Apply the ML method to HMM
Let Akl = #(transitions from k to l) in the training set.Ek(b) = #(emissions of symbol b from state k) in the training set. We need to:
s1 s2 sL-1 sL
X1 X2 XL-1 XL
si
Xi
( )
( , ) ( , )
Maximize [ ( )]kl kA E bkkl
k l k b
a e b
kb
Subject to: for all states , 1, and e ( ) 1.kll
k a b
27
Apply to HMM (cont.)
We apply the previous technique to get for each k the parameters {akl|l=1,..,m} and {ek(b)|bΣ}:
s1 s2 sL-1 sL
X1 X2 XL-1 XL
si
Xi
'' '
( ) , and ( )
( ')kl k
kl kkl kl b
A E ba e b
A E b
Which gives the optimal ML parameters
28
Adding pseudo counts in HMM
If the sample set is too small, we may get a biased result. In this case we modify the actual count by our prior knowledge/belief: rkl is our prior belief and transitions from k to l.rk(b) is our prior belief on emissions of b from state k.
s1 s2 sL-1 sL
X1 X2 XL-1 XL
si
Xi
' '' '
( ) ( )then , and ( )
( ) ( ( ') ( ))kl kl k k
kl kkl kl k kl b
A r E b r ba e b
A r E b r b
29
Summary of Case 1: Sequences are fully known
We know the complete structure of each sequence in the training set {x1,...,xn}. We wish to estimate akl and ek(b) for all pairs of states k, l and symbols b.
We just showed a method which finds the (unique) parameters θ* which maximizes p(x1,...,xn| θ*) =MAXθ p(x1,...,xn| θ).
s1 s2 sL-1 sL
X1 X2 XL-1 XL
si
Xi
30
Case 2: State paths are unknown:
In this case only the values of the xi’s of the input sequences are known. This is a ML problem with “missing data”. We wish to find θ* so that p(x|θ*)=MAXθ{p(x|θ)}.For each sequence x,
p(x|θ)=∑s p(x,s|θ), taken over all state paths s.
s1 s2 sL-1 sL
X1 X2 XL-1 XL
si
Xi
31
Case 2: State paths are unknown
So we need to maximize p(x|θ)=∑s p(x,s|θ), where the summation is over all the sequences S which produce the output sequence x. Finding θ* which maximizes ∑s p(x,s|θ) is hard. [Unlike finding θ* which maximizes p(x,s|θ) for a single sequence (x,s).]
s1 s2 sL-1 sL
X1 X2 XL-1 XL
si
Xi
32
ML Parameter Estimation for HMM
The general process for finding θ in this case is1. Start with an initial value of θ.2. Find θ’ so that p(x1,..., xn|θ’) > p(x1,..., xn|θ) 3. set θ = θ’.4. Repeat until some convergence criterion is met.
A general algorithm of this type is the Expectation Maximization algorithm, which we will meet later. For the specific case of HMM, it is the Baum-Welch training.
34
Baum Welch training
We start with some values of akl and ek(b), which define prior values of θ. Baum-Welch training is an iterative algorithm which attempts to replace θ by a θ* s.t.
p(x|θ*) > p(x|θ)Each iteration consists of few steps:
s1 s2 sL-1 sL
X1 X2 XL-1 XL
si
Xi
35
Baum Welch: step 1
Count expected number of state transitions: For each sequence xj, for each i and for each k,l, compute the posterior state transitions probabilities:
s1 SisL
X1 Xi XL
Si-1
Xi-1.. ..
P(si-1=k, si=l | xj,θ)
37
Baum Welch training
Claim:
1
( 1) ( ) ( )( = , = | , )
( | )
j j jk kl l i l
i i j
f i a e x b ijp s k s lp
x x
s1 SisL
X1 Xi XL
Si-1
Xi-1.. ..
where and are the forward and backward algorithms
for under .
j jk lf b
jx
38
Step 1: Computing P(si-1=k, si=l | xj,θ)
P(x1,…,xL,si-1=k,si=l) = P(x1,…,xi-1,si-1=k) aklek(xi ) P(xi+1,…,xL |si=l)
p(si-1=k,si=l | xj) = fk(i-1) aklel(xi ) bl(i)
fk’(i-1) ak’l’ek’(xi ) bl’(i)K’l’
= fk(i-1) aklek(xi ) bl(i)
Via the forward algorithm
Via the backward algorithm
s1 s2 sL-1 sL
X1 X2 XL-1 XL
Si-1
Xi-1
si
Xi
xj
39
Step 1 (end)
for each pair (k,l), compute the expected number of state transitions from k to l:
11 1
1 1
1( = , = , )
( )
1( 1) ( ) ( )
( )
|n L
kl i ijj i
n Lj j
kl l ik ljj i
jA p s k s lp x
f i a e x b ip x
x
40
Baum-Welch: Step 2
for each state k and each symbol b, compute the expected number of emissions of b from k:
1 :
1( ) ( ) ( )
( ) ji
nj j
k k kjj i x b
E b f i b ip x
41
Baum-Welch: step 3
'' '
( ) , and ( )
( ')kl k
kl kkl kl b
A E ba e b
A E b
Use the Akl’s, Ek(b)’s to compute the new values of akl and ek(b). These values define θ*.
It can be shown that: p(x1,..., xn|θ*) > p(x1,..., xn|θ)
i.e, θ* increases the probability of the data
This procedure is iterated, until some convergence criterion is met.
42
Case 2: State paths are unknown:Viterbi training
Also start from given values of akl and ek(b), which defines prior values of θ. Viterbi training attempts to maximize the probability of a most probable path; i.e., maximize
p((s(x1),..,s(xn)) |θ, x1,..,xn )Where s(xj) is the most probable (under θ) path for xj.
s1 s2 sL-1 sL
X1 X2 XL-1 XL
si
Xi
43
Case 2: State paths are unknown:Viterbi training
Each iteration:1. Find a set {s(xj)} of most probable paths, which
maximizep(s(x1),..,s(xn) |θ, x1,..,xn )
2. Find θ*, which maximizes p(s(x1),..,s(xn) | θ*, x1,..,xn )
Note: In 1. the maximizing arguments are the paths, in 2. it is θ*. 3. Set θ=θ* , and repeat. Stop when paths are not changed.
s1 s2 sL-1 sL
X1 X2 XL-1 XL
si
Xi
44
Case 2: State paths are unknown:Viterbi training
p(s(x1),..,s(xn) | θ*, x1,..,xn ) can be expressed in a closed form (since we are using a single path for each xj), so this time convergence is achieved when the optimal paths are not changed any more.
s1 s2 sL-1 sL
X1 X2 XL-1 XL
si
Xi