#markov model

In probability theory, a Markov model is a stochastic model used to model randomly changing systems.

Stochastic process

Let’s see what Stochastic process is first .

Stochastic is, having a random probability distribution or pattern that may be analysed statistically but may not be predicted precisely., or in simple terms refers to a randomly determined process . In probability theory and related…

When we are building markov model often we need to calculate state transition probabilities for 2 or 1 history steps. The following function can help you calculate these probabilities in R.

https://gist.github.com/ankitksharma/b6a63c5dfb9cbb5f27c5

The following code will explain how to find the probabilities for 2 history steps:

history <- 2 data <- as.factor(c(1,1,1,1,0,0,0,0,0,0,0,0,0,1,1,1,0,0,0,1,1,1)) levels(data)[1] <- "NoFault" levels(data)[2] <- "Fault" data [1] Fault Fault Fault Fault NoFault NoFault NoFault NoFault NoFault NoFault NoFault NoFault NoFault Fault Fault [16] Fault NoFault NoFault NoFault Fault Fault Fault

So, here we are trying to find the probability of getting a Fault state in next time step if for t-1 and t-2 steps the system had NoFault and NoFault states resp.

probabilities <- markovStateTransition(history, data) probabilities t2 t1 t probability 1 NoFault NoFault NoFault 0.8000000 2 Fault NoFault NoFault 1.0000000 3 NoFault Fault NoFault 0.0000000 4 Fault Fault NoFault 0.3333333 5 NoFault NoFault Fault 0.2000000 6 Fault NoFault Fault 0.0000000 7 NoFault Fault Fault 1.0000000 8 Fault Fault Fault 0.6666667

Therefore from the above table we can say the Probability of Fault at t given at t-1 & t-2 the state was NoFault and NoFault respectively is 0.2 (row 5)