Contents

Convergence of random variables

In probability theory, several different notions of convergence of random variables are investigated. These will be presented here. Throughout, we assume that (Xn) is a sequence of random variables, and X is a random variable, and all of them are defined on the same probability space (Ω, P).

Table of contents
1 Convergence in distribution
2 Convergence in probability
3 Almost sure convergence
4 Convergence in mean
5 Convergence in mean square

Convergence in distribution

We say that the sequence Xn converges towards X in distribution, if

<math>\lim_{n\rightarrow\infty}P(X_n\leq a)=P(X\leq a)</math>
for every real number a at which the cumulative distribution function of the limiting random variable X is continuous.

Essentially, this means that the probability that the value of X is in a given range is very similar to the probability that the value of Xn is in that range, if only n is large enough. This notion of convergence is used in the central limit theorems.

Convergence in distribution is also called convergence in law, since the word "law" is sometimes used as a synonym of "probability distribution." Another name is weak convergence.

Convergence in probability

We say that the sequence Xn converges towards X in probability if

<math>\lim_{n\rightarrow\infty}P\left(\left|X_n-X\right|\geq\varepsilon\right)=0</math>
for every ε > 0.

This means that if you pick a tolerance ε and choose n large enough, then the value of Xn will be almost guaranteed to be within that tolerance of the value of X. This notion of convergence is used in the weak law of large numbers.

Convergence in probability implies convergence in distribution.

Almost sure convergence

We say that the sequence Xn converges almost surely or almost everywhere or with probability 1 or strongly towards X if

<math>P\left(\lim_{n\rightarrow\infty}X_n=X\right)=1.</math>

This means that you are virtually guaranteed that the values of Xn approach the value of X. This notion of convergence is used in the strong law of large numbers.

Almost sure convergence implies convergence in probability.

Convergence in mean

We say that the sequence Xn converges towards X in mean or in the L1 norm if

<math>\lim_{n\rightarrow\infty}E\left(\left|X_n-X\right|\right)=0</math>
where E denotes the expected value.

This means that the expected difference between Xn and X gets as small as desired if n is chosen big enough. This convergence is considered in Lp spaces (where p = 1).

Convergence in the mean implies convergence in probability. There is no general relation between convergence in mean and almost sure convergence however.

Convergence in mean square

We say that the sequence Xn converges towards X in mean square or in the L2 norm if

<math>\lim_{n\rightarrow\infty}E\left(\left|X_n-X\right|^2\right)=0.</math>

This means that the expected squared difference between Xn and X gets as small as desired if n is chosen big enough. This convergence is considered in Lp spaces (where p = 2).

Convergence in mean square implies convergence in mean.