Company
20 problems tagged for Meta.
Build k-nearest neighbors classification using vectorized distance computation and top-k selection on tensors.
Build binary logistic regression from scratch with sigmoid, binary cross-entropy, and manual gradient descent updates.
Build numerically stable softmax using the log-sum-exp trick, handling overflow and underflow in raw tensor math.
Maintain and expand top-scoring partial sequences at each decoding step with length normalization and early stopping.
Train a smaller student model to match a larger teacher's soft predictions using temperature-scaled KL divergence loss.
Cache key and value tensors from previous timesteps so each new token only computes attention over one new position.
Inject trainable low-rank decomposition matrices (A, B) into a frozen linear layer for parameter-efficient fine-tuning.
Build a Swin Transformer block from scratch: window partition, shifted-window attention masks, and a learned relative position bias.
Implement the Adam optimizer from scratch on top of torch.optim.Optimizer: exponential moving averages of the gradient and its square, with bias correction.
Implement the AdamW optimizer from scratch on top of torch.optim.Optimizer: Adam's update with weight decay decoupled from the gradient, applied directly to the parameters.
Dynamically slot sequences in and out of a running batch as they finish, maximizing throughput without padding waste.
Compute the Direct Preference Optimization loss that trains a policy directly from preference pairs without a reward model.
Trade compute for memory by recomputing intermediate activations during backward instead of storing them all.
Implement the Muon optimizer from scratch on top of torch.optim.Optimizer: momentum orthogonalized by Newton–Schulz for matrix parameters, with an AdamW fallback for 1D parameters.
Build Proximal Policy Optimization with clipped surrogate objective, value function baseline, and KL penalty for RLHF.
Build ViT with masked autoencoder pretraining — randomly mask patches, encode visible ones, decode to reconstruct.
Write the tiled, fused attention kernel that computes exact attention in O(n) memory using online softmax in Triton.
Build Fully Sharded Data Parallel: shard parameters across GPUs, all-gather before forward, reduce-scatter gradients after backward.
Distribute attention computation across GPUs in a ring topology, enabling context lengths that exceed single-GPU memory.
Write a GPU kernel in Triton that fuses the softmax computation into a single pass, eliminating intermediate memory reads.
