Kullback–Leibler Divergence

Natural & Formal Sciences Dictionary
A directed measure of discrepancy between two probability distributions defined as the expectation under one distribution of the log ratio of their densities; quantifies expected log‑loss when using one distribution to approximate another.