Kaplan 2020 論文：Scaling Laws 的演進與情境邊界
2020 年 OpenAI 提出模型大小與資料量的冪次律，建議『大模型、少資料』。兩年後 Chinchilla 修正 compute-optimal 配置，讓資料量的重要性重新被看見。拆解 Scaling Laws 演進，以及它如何影響你現在的訓練決策與成本結構。
Model Training
Kaplan 2020: Scaling Laws and Their Limits
In 2020, OpenAI proposed a power-law relationship between model size and data volume, which led many teams toward a simple idea: bigger models, less data. Two years later, Chinchilla revised that picture with a compute-optimal view, showing that data volume mattered far more than many people had assumed. This matters if you’re deciding whether the next gain should come from more GPUs, a larger parameter count, or a larger corpus. The history of scaling laws is not just academic. It changes how you think about training efficiency, model selection, and where your cost structure really sits.
弱監督強模型：探索弱到強泛化的效能邊界
用弱模型監督強模型，強模型真的能超越監督者嗎？OpenAI 實測發現，簡單微調只能恢復約一半的效能差距。透過置信度損失與引導策略，能將差距縮小至 20%，但仍有其邊界。本文深入拆解這份研究背後的機制與工程實踐。
Weak Supervision, Strong Models: Exploring the Performance Boundary of Weak-to-Strong Generalization
When a weak model supervises a strong model, can the strong model truly surpass its supervisor? OpenAI’s experiments found that simple fine-tuning recovers only about half of the performance gap. With confidence loss and guidance strategies, the gap can shrink to around 20%, but boundaries remain. This article breaks down the mechanisms and engineering practice behind the study.