ReFlow: Exposing Parallelism in CPU-Offloaded LLM Training via Register-Resident SIMD Fusion and Decoupled Update Scheduling

Seongtae Bang, Gyeongseo Park, Ki-Dong Kang, Hyunkyun Shin, Sungju Kim, and Daehoon Kim* ACM International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), 2027

Abstract

Keywords

LLM Training, CPU Offloading, SIMD, Optimizer Update Scheduling.