Build a Complete LLM Inference Engine in PyTorch
Expertllm-inferenceauto-graded
Combine KV caching, continuous batching, and memory management into a production-grade inference server.
Solve it
Check your answer
The grader verifies properties of your implementation, so a correct solution written differently from ours still passes.
pip install torchleet
from torchleet import check
check("inference-engine", SimpleTokenizer, KVCache, MultiHeadAttention, FeedForward, TransformerBlock, MiniTransformer, top_p_sample, InferenceEngine)