Markov chain

A diagram representing a two-state Markov process. The numbers are the probability of changing from one state to another state.

A Markov chain or Markov process is a stochastic model describing a sequence of possible events in which the probability of each event depends only on the state attained in the previous event.[1][2][3] Informally, this may be thought of as, "What happens next depends only on the state of affairs now." A countably infinite sequence, in which the chain moves state at discrete time steps, gives a discrete-time Markov chain (DTMC). A continuous-time process is called a continuous-time Markov chain (CTMC). It is named after the Russian mathematician Andrey Markov.

Markov chains have many applications as statistical models of real-world processes,[1][4][5][6] such as studying cruise control systems in motor vehicles, queues or lines of customers arriving at an airport, currency exchange rates and animal population dynamics.[7]

Markov processes are the basis for general stochastic simulation methods known as Markov chain Monte Carlo, which are used for simulating sampling from complex probability distributions, and have found application in Bayesian statistics, thermodynamics, statistical mechanics, physics, chemistry, economics, finance, signal processing, information theory and speech processing.[7][8][9]

The adjectives Markovian and Markov are used to describe something that is related to a Markov process.[1][10][11]

  1. ^ a b c Gagniuc, Paul A. (2017). Markov Chains: From Theory to Implementation and Experimentation. USA, NJ: John Wiley & Sons. pp. 1–235. ISBN 978-1-119-38755-8.
  2. ^ "Markov chain | Definition of Markov chain in US English by Oxford Dictionaries". Oxford Dictionaries. Archived from the original on December 15, 2017. Retrieved 2017-12-14.
  3. ^ Definition at Brilliant.org "Brilliant Math and Science Wiki". Retrieved on 12 May 2019
  4. ^ Samuel Karlin; Howard E. Taylor (2 December 2012). A First Course in Stochastic Processes. Academic Press. p. 47. ISBN 978-0-08-057041-9. Archived from the original on 23 March 2017.
  5. ^ Bruce Hajek (12 March 2015). Random Processes for Engineers. Cambridge University Press. ISBN 978-1-316-24124-0. Archived from the original on 23 March 2017.
  6. ^ G. Latouche; V. Ramaswami (1 January 1999). Introduction to Matrix Analytic Methods in Stochastic Modeling. SIAM. pp. 4–. ISBN 978-0-89871-425-8. Archived from the original on 23 March 2017.
  7. ^ a b Sean Meyn; Richard L. Tweedie (2 April 2009). Markov Chains and Stochastic Stability. Cambridge University Press. p. 3. ISBN 978-0-521-73182-9. Archived from the original on 23 March 2017.
  8. ^ Reuven Y. Rubinstein; Dirk P. Kroese (20 September 2011). Simulation and the Monte Carlo Method. John Wiley & Sons. p. 225. ISBN 978-1-118-21052-9. Archived from the original on 23 March 2017.
  9. ^ Dani Gamerman; Hedibert F. Lopes (10 May 2006). Markov Chain Monte Carlo: Stochastic Simulation for Bayesian Inference, Second Edition. CRC Press. ISBN 978-1-58488-587-0. Archived from the original on 23 March 2017.
  10. ^ "Markovian". Oxford English Dictionary (Online ed.). Oxford University Press. (Subscription or participating institution membership required.)
  11. ^ Model-Based Signal Processing. John Wiley & Sons. 27 October 2005. ISBN 9780471732662.