Resilience

大型語言模型預訓練常因硬體延遲或故障導致整體停擺。該論文提出 Decoupled DiLoCo，透過解耦 Learner 與異步整合機制，在模擬百萬級晶片環境中實現零停機時間，並將 Goodput 維持在 88%，為分散式訓練提供更具韌性的架構選擇。

Large language model pre-training can pause when hardware latency or node outages propagate through strict synchronization points. Decoupled DiLoCo uses independent Learners and asynchronous integration, allowing training to continue in simulated environments with millions of chips. Under the paper’s specified fault model, it reports zero global downtime and sustains 88% Goodput. The result is a more resilient architectural option for distributed training.