 ##  [Knowledge Distillation](/knowledge-distillation) 

  ##  [Knowledge Distillation](https://natural.quantumdictionary.io/knowledge-distillation-0) 

  
 [![Natural & Formal Sciences Dictionary](/sites/default/files/styles/large/public/2026-01/Natural%20%26%20Formal%20Sciences.png.webp?itok=2kCDRVQv)](/topic-specific-dictionaries/natural-formal-sciences)


**Natural &amp; Formal Sciences Dictionary**

 
Definition

A model compression technique in which a smaller 'student' model is trained to reproduce the behavior or softened outputs of a larger 'teacher' model so that the student inherits useful predictive structure while being more compact and efficient.