Efficient MoE Pre-training at Scale on 1K AMD GPUs with TorchTitan

Training massive Mixture-of-Experts (MoE) models like DeepSeek-V3 and Llama 4-Scout efficiently is one of the challenges in modern AI. These models push GPUs, networks, and compilers to their limits. To…