Shannon–McMillan–Breiman Theorem

For a discrete-time stationary ergodic stochastic process {X_i} over a finite alphabet with probability measure P, the normalized log-likelihood of length-n realizations converges almost surely to the entropy rate H: lim_{n→∞} -(1/n) log P(X1..Xn) = H, where H is the process entropy rate.