sp26-cs639 cs639
Lecture 23: Scaling I Lecture 23: Scaling I
Content locked
The catalog is public. Reading this content requires approved access.
Sign in to readLecture 23: Scaling I
What should a larger training budget buy?
Power laws and regimes
Compute accounting
Revised compute-optimal scaling: Chinchilla
Beyond a fixed dense-model scaling curve
Data quality and pruning
Different capabilities have different bottlenecks
Mixture of experts
Connections
Source figure