Self-Play Pretraining with Zero Data

Two models start from random weights; one writes programs that teach the other, using no real data.

512

rounds

Round Fibonacci sequences first appeared

Random program sampling would need far longer to find them

How it works

Data is hand-curated
Pretraining relies on curated human data. Could a model instead generate its own useful training data, limited only by compute?
Generator teaches learner
A generator writes tiny Brainf*ck-like programs; their byte outputs train a learner. RL rewards programs at the learner's learning frontier.
Transfer scales with compute
Zero-shot loss on text, images, audio and code falls as a power law in compute. The learner also learns in context.

From the paper