 ##  [Kullback–Leibler Divergence](/kullback-leibler-divergence) 

  ##  [Kullback–Leibler Divergence](https://natural.quantumdictionary.io/kullback-leibler-divergence-0) 

  
 [![Natural & Formal Sciences Dictionary](/sites/default/files/styles/large/public/2026-01/Natural%20%26%20Formal%20Sciences.png.webp?itok=2kCDRVQv)](/topic-specific-dictionaries/natural-formal-sciences)


**Natural &amp; Formal Sciences Dictionary**

 
Definition

A directed measure of discrepancy between two probability distributions defined as the expectation under one distribution of the log ratio of their densities; quantifies expected log‑loss when using one distribution to approximate another.