Contents

Stochastic matrix

In mathematics, especially in probability theory and statistics, and also in linear algebra and computer science, a stochastic matrix is a square matrix whose columns are probability vectors which add up to one. It is the same thing as the matrix of transition probabilities of a finite Markov chain.

Here is an example of a stochastic matrix P:

<math>P = \begin{bmatrix}
0.5 & 0.2 & 0.3 \\ 0.3 & 0.8 & 0.3 \\ 0.2 & 0 & 0.4 \end{bmatrix}</math>

If G is a stochastic matrix, then a steady-state vector[?] or equilibrium vector[?] for G is a probability vector h such that:

<math> Gh = h </math>

An example:

<math>G = \begin{bmatrix}
0.95 & 0.03 \\ 0.05 & 0.97 \end{bmatrix}</math> and
<math>h = \begin{bmatrix}
0.375 \\ 0.625 \end{bmatrix}</math>

<math>Gh = \begin{bmatrix}
0.95 & 0.03 \\ 0.05 & 0.97 \end{bmatrix} 
 \begin{bmatrix}
0.375 \\ 0.625 \end{bmatrix} = \begin{bmatrix} 0.35625 + 0.01875 \\ 0.01875 + 0.60625 \end{bmatrix} = \begin{bmatrix} 0.375 \\ 0.625 \end{bmatrix}</math>

This case shows that Gh = 1h. For equations that show Gh = βh, for some real number β like Gh = 4h or Gh = -21h, see Eigenvectors.

A stochastic matrix is regular if some matrix power Pk contains only strictly positive entries.

Take P from above as a stochastic matrix:

<math>P^2 = \begin{bmatrix}
0.37 & 0.26 & 0.33 \\ 0.45 & 0.70 & 0.45 \\ 0.18 & 0.04 & 0.22 \end{bmatrix}</math>

Therefore, P is a regular stochastic matrix.

The Stochastic Matrix Theorem says if A is a regular stochastic matrix, then A has a steady-state vector t so that if xo is any initial state and xk+1 = Axk for k = 0,1,2,..... then the Markov chain {xk} converges to t as k -> infinity. That is:

<math>\lim_{k \to \infty} A^k \textbf{x}_0 = \textbf{t}</math>