Company
17 problems tagged for Google.
Code Lloyd's algorithm with PyTorch tensors: random init, distance computation, centroid updates, and convergence check.
Build binary logistic regression from scratch with sigmoid, binary cross-entropy, and manual gradient descent updates.
Build numerically stable softmax using the log-sum-exp trick, handling overflow and underflow in raw tensor math.
Maintain and expand top-scoring partial sequences at each decoding step with length normalization and early stopping.
Train a smaller student model to match a larger teacher's soft predictions using temperature-scaled KL divergence loss.
Inject trainable low-rank decomposition matrices (A, B) into a frozen linear layer for parameter-efficient fine-tuning.
Restrict attention to a fixed local window around each token, reducing memory from O(n²) to O(n·w) for long sequences.
Build a Swin Transformer block from scratch: window partition, shifted-window attention masks, and a learned relative position bias.
Build the full denoising diffusion pipeline: forward noise schedule, U-Net denoiser, and reverse sampling to generate images.
Trade compute for memory by recomputing intermediate activations during backward instead of storing them all.
Build a gated MoE with top-k routing, load balancing loss, and expert capacity constraints for sparse computation.
Build Mamba's selective scan mechanism with input-dependent parameters, achieving linear-time sequence modeling without attention.
Draft tokens with a fast model, verify in parallel with the target model, and accept/reject to guarantee identical output distribution.
Build ViT with masked autoencoder pretraining — randomly mask patches, encode visible ones, decode to reconstruct.
Build Fully Sharded Data Parallel: shard parameters across GPUs, all-gather before forward, reduce-scatter gradients after backward.
Distribute attention computation across GPUs in a ring topology, enabling context lengths that exceed single-GPU memory.
Write a GPU kernel in Triton that fuses the softmax computation into a single pass, eliminating intermediate memory reads.