
Machine Learning
Analysis
Knowledge Distillation: Compressing Large Models Into Small Ones
How knowledge distillation trains small student models from large teachers, from soft targets and temperature to sequence-level and reasoning distillation, and where it falls short.
By Institute for Joint Cognition & AI · 6 min · 17 November 2026