Implement Knowledge Distillation in PyTorch
Mediummodern-architecturesauto-graded
Train a smaller student model to match a larger teacher's soft predictions using temperature-scaled KL divergence loss.
Solve it
Check your answer
The grader verifies properties of your implementation, so a correct solution written differently from ours still passes.
pip install torchleet
from torchleet import check
check("knowledge-distillation", distillation_loss)