ReplayOpt: Optimizer-State Replay to Resolve Critical-Path Bottlenecks in Offloaded Training

Seongtae Bang, Gyeongseo Park, Kyeonghyeon Ryu, and Daehoon Kim*. IEEE Computer Architecture Letters (CAL), 2026

Abstract

Keywords

Large Language Model, Optimizer, Training, Heterogeneous memory, Distributed training.