01 / 14 · 00:00
Half the new web is machine-written. The next model learns from it.
GENERATION 03,172 words
what the toy model knew
GENERATION 100220 words
after learning only from itself
ILLUSTRATIVE · toy simulation · seed 2026 · not a real LLM
what the toy model knew
after learning only from itself
ILLUSTRATIVE · toy simulation · seed 2026 · not a real LLM
حدس بزن: بعد از ۱۰۰ نسل، چند درصد واژهها میماند؟
of new articles
and roughly flat since
human-written
Estimate from AI detectors on Common Crawl samples — not a count · Graphite, May 2026, three detectors averaged
“…tails of the original content distribution disappear.”Shumailov, Shumaylov, Zhao, Papernot, Anderson, Gal · AI models collapse when trained on recursively generated data · Nature 631 · July 2024
amber = the rare tails, |x| > 2 · 4.55% of a true bell curve · 200 runs × 1,000 generations
ILLUSTRATIVE · toy simulation · seed 2026 · not a real LLM · replace regime · medians of 200 runs
Rarest half of the vocabulary: 1,586 → 36 survive. Same arithmetic as genetic drift (Wright–Fisher): a word that hits zero never returns.
ILLUSTRATIVE · synthetic Zipf corpus, 20,000 tokens · 20 runs · seed 2026
width at gen 1,000 · 220 words at gen 100
width at gen 1,000 · 1,774 words
width at gen 1,000 · 3,172 words
ILLUSTRATIVE · toy simulation · seed 2026 · not a real LLM · idea: Gerstgrasser et al., 2024
Keep 10% real data, generation 100. Who is the tail of the web? Low-resource languages, dialects, small cultures. (hypothesis — not measured here)
ILLUSTRATIVE · toy simulation · seed 2026 · not a real LLM
physical qubits to break RSA-2048
much lower — next episode