Implement DPO Loss from Scratch in PyTorch
Hardalignment-trainingauto-graded
Compute the Direct Preference Optimization loss that trains a policy directly from preference pairs without a reward model.
Solve it
Check your answer
The grader verifies properties of your implementation, so a correct solution written differently from ours still passes.
pip install torchleet
from torchleet import check
check("dpo", get_batch_logps, dpo_loss)