ReplayOpt: Optimizer-State Replay to Resolve Critical-Path Bottlenecks in Offloaded Training

Seongtae Bang, Gyeongseo Park, Kyeonghyeon Ryu, and Daehoon Kim*. IEEE Computer Architecture Letters (CAL), 2026
DOI
10.1109/LCA.2026.3676470

Abstract

Keywords

Large Language Model, Optimizer, Training, Heterogeneous memory, Distributed training.