Hidden Markov Models in Keystroke Dynamics
Md Liakat Ali, John V. Monaco, and Charles C. Tappert
Seidenberg School of CSIS, Pace University, White Plains, New YorkMay 1, 2015
Contents
Objective o Generative Vs Discriminative
Classifier
Markov Modelo Markov chaino Hidden Markov Model
Elements of HMM Three problems for HMMs and
solutions.
Researches in Keystroke Dynamics Using HMMo Findings
Machine Learning Unsupervised learning is known
as clustering, and involves grouping data into categories based on some measure of inherent similarity or distance.
Supervised learning, i.e. learning where a training set of correctly identified observations is available. The computer is presented with example inputs and their desired outputs
Classification is an instance of supervised learning
Generative Vs Discriminative Classifier
The main difference –
o Generative model is a full probabilistic model of all variable .
For fewer training samples, the generative model performs better .
o Discriminative model is only for target variable(s) dependent on observed variable.
For large samples, discriminative classifiers generally outperform generative classifiers.
Faster classification of new data compared to generative model
Generative Vs Discriminative Classifier Popular generative models are:
• Gaussians, Mixture of Gaussians, • Naïve Bayes, Bayesian networks, • Hidden Markov Models, • Sigmoidal belief networks• Markov random fields.
Popular discriminative models are:
• Logistic regression, • Support Vector Machine, • Neural Networks, • Nearest neighbor, • Conditional Random Fields.
Andrey Andreyevich Markov was a Russian mathematician. He is best known for his work on stochastic processes. A primary subject of his research later became known as Markov chains and Markov processes. (Wikipedia)
Andrey Andreyevich Markov
A stochastic model which is used to model a system that randomly changes, and where the future state, given the past and the present, only depend on the present state not the past.
Types of Markov model:
MARKOV MODEL
System state is fully observable
System state is partially observable
System is autonomous
Markov chainHidden Markov model
System is controlled
Markov decision process
partially observable Markov decision process
The simplest Markov model which models a system that randomly changes through time.
A Markov chain is a discrete-time process where a Markov process is the continuous-time version of Markov chain.
Three-state Markov Process of Weathero Assume any given day t, the weather is observed as being one of the
following states:• State 1: rain• State 2: cloudy• State 3: sunny
And the transition probabilities between states is described by the transition matrix A is
Markov Chain
Fig. A Markov model with 3 states and state transition probabilities.
For example on the day (t) 1, the weather is sunny means in state 3 and we want to find the probability that the weather for next 7 days will be sunny-sunny-rain-rain-sunny-cloudy-sunny. Which means, we want to find the probability of the observation sequence, O = S3, S3, S3, S1, S1, S3, S2, S3
Three-state Markov Process of Weather
An extension of Markov chain where observation of the system is a probabilistic function of the state
Has an underlying stochastic process that is not observable (hidden), but can only be observed through another set of stochastic processes that produce a sequence of observation.
Hidden Markov Model
𝑠2
𝑞3 A A
Markov chain 𝑞1 𝑞2 𝑞3 𝑞𝑛 A A A
Hidden Markov Model 𝑞1 𝑞1 𝑞𝑛 A
𝑠1 𝑠3 𝑠𝑛
Hidden
Observable
B B\B
B B
Π0
Assume that we were kidnapped and kept in a locked room for several days. We cannot observe the weather directly-the only evidence we have one kidnapper brings food have brought an umbrella.
Let assume that the probability of rain seeing an
umbrella is 0.8, for cloudy 0.3 and for sunny 0.1. Before we were kidnapped, we could observe the
weather and the weather Markov process was:
Hidden Markov Model
But now as the actual weather is hidden and the probability of states of weather corresponds to seeing umbrella (u = True or False)
o For example the day we were kidnapped, it was sunny.
o The next day when the kidnapper brought food, he also carried umbrella into the room.
Hidden Markov Model
o Assuming the prior probability of carrying umbrella on any day is 0.5, we can find the probability of rain on second day by the following steps:
Hidden Markov ModelState 1: rain State 2: cloudy State 3: sunny
Previous example depicts how to construct a weather model via HMM
Some difficulties in modeling procedure such as Finding the number of states (model
size) of the model, How to choose model parameter (such as
transition probabilities) and The size of observation sequence.
Hidden Markov Model
Problem 1 (Evaluation problem): Given observation sequence, O={O1,O2,…OT} and a model λ=(A,B,π), how do we efficiently compute P(O|λ), the likelihood of the observation sequence given the model?
o The solution is given by the Forward and Backward procedures.
Problem 2 (Decoding Problem): Given observation sequence, O={O1,O2,…OT}and a model λ=(A,B,π), how do we choose a corresponding state sequence Q= {q1, q2, …qT} that is optimal such as best explains the data?
o The solution for this problem is provided by the Viterbi algorithm.
Problem 3 (Learning problem): How do we adjust the model parameters A,B, Π to maximize the likelihood P(O|λ).
o The solution is given by the Baum-Welch re-estimation procedure.
Three basic problems for HMMs
Keystroke dynamic studies using HMM
Study Participants
Samples Per User Features
Input Method EER (%)
Training Testing
Chen and Chang, 2004 [11]
Chang, 2005 [12]
20 20 200 DT, FT
User-fixed text-one word
Discrete HMM
NA
Rodgrigues et al., 2005 [13]
20 40 30 DT, FT, ASCII key code
User-fixed Number- 8
digit
Continuous HMM
3.6
Vuyyuru et al., 2006 [14]
43 9 20 DT Fixed-text: “master of science in computer science”
HMM 3.04
Jiang et al., 2007 [15] 315Training:
58Testing:
257
15 13 DT, FT, n-graph
User-fixed text
Minimum 9 characters
HMM , Gaussian
2.54
Zhang et al., 2010 [16] 12 20 40 DT, FT
User-fixed textMinimum 10
characters
HMM 2.00
Thank You
?