Markov Chain Models

A class of stochastic processes in discrete or continuous time whose transition probabilities depend only on the current state (the Markov property); specified by a state space and a transition kernel or matrix, used to model time‑evolving system dynamics and derive stationary behaviour, hitting times and transient statistics.