Tech Apr 9, 2026 14 min MegaTrain Trains a 120B-Parameter LLM on a Single GPU at Full Precision MegaTrain flips the GPU-centric paradigm by treating CPU memory as primary storage and the GPU as a transient compute device, enabling full-precision training of 100B+ LLMs on a single GPU with up to 12.2x throughput over DeepSpeed ZeRO-3. LLM Machine Learning GPU DeepSpeed Memory Optimization
