Contents

Law of large numbers

In probability theory, the weak law of large numbers states that if X1, X2, X3, ... is an infinite sequence of random variables, all of which have the same expected value μ and the same finite variance σ2, and they are uncorrelated (i.e., the correlation between any two of them is zero), then the sample average

<math>\overline{X}_n=(X_1+\cdots+X_n)/n</math>
converges in probability to μ. Somewhat less tersely: For any positive number ε, no matter how small, we have
<math>\lim_{n\rightarrow\infty}P\left(\left|\overline{X}_n-\mu\right|<\varepsilon\right)=1.</math>
Chebyshev's inequality is used to prove this result.

A consequence of the weak law of large numbers is the Asymptotic Equipartition Property.

The strong law of large numbers states that if X1, X2, X3, ... is an infinite sequence of random variables that are independent and identically distributed, and have a common expected value μ then

<math>P\left(\lim_{n\rightarrow\infty}\overline{X}_n=\mu\right)=1,</math>
i.e., the sample average converges almost surely to μ.

This law justifies the intuitive interpretation of the expected value of a random variable as the "long-term average when sampling repeatedly".