Generative AI & LLMs
Model Distillation
Definition
Training a smaller model to reproduce useful behavior from a larger model using examples or other training signals.
Why it matters for your business
Can reduce serving cost and latency for a narrow task, but the smaller model needs its own evaluation.
In practice
A team trains a compact classifier on examples produced and checked with a larger model.