Implement ROPE Embeddings in PyTorch
Mediummodern-architecturesauto-graded
Build Rotary Position Embeddings that encode relative positions by rotating query and key vectors in complex space.
Solve it
Check your answer
The grader verifies properties of your implementation, so a correct solution written differently from ours still passes.
pip install torchleet
from torchleet import check
check("rotary-positional-embedding", rotate_half, apply_rotary_pos_emb)