Markov chain

A diagram representing a two-state Markov process, with the states labelled E and A. Each number represents the probability of the Markov process changing from one state to another state, with the direction indicated by the arrow. For example, if the Markov process is in state A, then the probability it changes to state E is 0.4, while the probability it remains in state A is 0.6.

A Markov chain is a stochastic model describing a sequence of possible events in which the probability of each event depends only on the state attained in the previous event.[1][2][3] In continuous-time, it is known as a Markov process. It is named after the Russian mathematician Andrey Markov.

Markov chains have many applications as statistical models of real-world processes,[1][4][5][6] such as studying cruise control systems in motor vehicles, queues or lines of customers arriving at an airport, currency exchange rates and animal population dynamics.[7]

Markov processes are the basis for general stochastic simulation methods known as Markov chain Monte Carlo, which are used for simulating sampling from complex probability distributions, and have found application in Bayesian statistics and artificial intelligence.[7][8][9]

The adjective Markovian is used to describe something that is related to a Markov process.[1][10]

  1. ^ a b c Gagniuc, Paul A. (2017). Markov Chains: From Theory to Implementation and Experimentation. USA, NJ: John Wiley & Sons. pp. 1–235. ISBN 978-1-119-38755-8.
  2. ^ "Markov chain | Definition of Markov chain in US English by Oxford Dictionaries". Oxford Dictionaries | English. Retrieved 2017-12-14.
  3. ^ Definition at "Brilliant Math and Science Wiki". Retrieved on 12 May 2019
  4. ^ Samuel Karlin; Howard E. Taylor (2 December 2012). A First Course in Stochastic Processes. Academic Press. p. 47. ISBN 978-0-08-057041-9. Archived from the original on 23 March 2017.
  5. ^ Bruce Hajek (12 March 2015). Random Processes for Engineers. Cambridge University Press. ISBN 978-1-316-24124-0. Archived from the original on 23 March 2017.
  6. ^ G. Latouche; V. Ramaswami (1 January 1999). Introduction to Matrix Analytic Methods in Stochastic Modeling. SIAM. pp. 4–. ISBN 978-0-89871-425-8. Archived from the original on 23 March 2017.
  7. ^ a b Sean Meyn; Richard L. Tweedie (2 April 2009). Markov Chains and Stochastic Stability. Cambridge University Press. p. 3. ISBN 978-0-521-73182-9. Archived from the original on 23 March 2017.
  8. ^ Reuven Y. Rubinstein; Dirk P. Kroese (20 September 2011). Simulation and the Monte Carlo Method. John Wiley & Sons. p. 225. ISBN 978-1-118-21052-9. Archived from the original on 23 March 2017.
  9. ^ Dani Gamerman; Hedibert F. Lopes (10 May 2006). Markov Chain Monte Carlo: Stochastic Simulation for Bayesian Inference, Second Edition. CRC Press. ISBN 978-1-58488-587-0. Archived from the original on 23 March 2017.
  10. ^ "Markovian". Oxford English Dictionary (3rd ed.). Oxford University Press. September 2005. (Subscription or UK public library membership required.)

From Wikipedia, the free encyclopedia · View on Wikipedia

Powered by easy search