Company
12 problems tagged for DeepMind.
Build 2D sinusoidal position encodings for vision transformers, encoding both row and column positions in patch grids.
Maintain and expand top-scoring partial sequences at each decoding step with length normalization and early stopping.
Build the InfoNCE contrastive loss and a CLIP-style training loop that aligns image and text embeddings.
Restrict attention to a fixed local window around each token, reducing memory from O(n²) to O(n·w) for long sequences.
Select the k highest-probability tokens, zero out the rest, renormalize, and sample for controlled text generation.
Sort logits, compute cumulative probabilities, mask tokens below the nucleus threshold, and sample from the filtered distribution.
Compute the Direct Preference Optimization loss that trains a policy directly from preference pairs without a reward model.
Build a gated MoE with top-k routing, load balancing loss, and expert capacity constraints for sparse computation.
Build Proximal Policy Optimization with clipped surrogate objective, value function baseline, and KL penalty for RLHF.
Build Mamba's selective scan mechanism with input-dependent parameters, achieving linear-time sequence modeling without attention.
Draft tokens with a fast model, verify in parallel with the target model, and accept/reject to guarantee identical output distribution.
Build Group Relative Policy Optimization that scores multiple completions per prompt and uses group-relative advantages.
