u/BatPlack

Researchers Alec Radford (GPT, CLIP, Whisper), Nick Levine, and David Duvenaud just released talkie: a 13 billion parameter language model trained exclusively on text published before 1931. No internet. No Wikipedia. No World War II. Its worldview is frozen at December 31, 1930.

Why does this matter?

Every major LLM today (GPT, Claude, Gemini, Llama) ultimately shares a common ancestor: the modern web. That makes it nearly impossible to tell what these models genuinely reason versus what they simply memorized.

Talkie breaks that lineage entirely. From the team:

>"It's an important question how much LM capabilities arise from memorization vs generalization. Vintage LMs enable unique generalization tests."

Interestingly, Claude has a direct role in talkie's creation: Claude Sonnet 4.6 was used as the judge in talkie's reinforcement learning pipeline (online DPO), and Claude Opus 4.6 generated synthetic multi-turn conversations used in the final fine-tuning stage. The team even notes the irony: using a thoroughly modern LLM to help shape a model that's supposed to be frozen in 1930, and flagging it as a contamination risk they're actively working to eliminate in future versions.

The most striking example: talkie can learn to write Python code from just a few in-context examples... despite having zero modern code in its training data. It's reasoning from 19th-century mathematics texts, not retrieval.

What it's being used to study

• Long-range forecasting: how well can a model "predict" the future from its frozen vantage point?
• Invention: can it develop ideas that postdate its knowledge cutoff?
• LLM identity: what makes a model itself? Talkie's alien data distribution helps isolate what's architecture vs. what's just "vibes absorbed from the web"

Links

• Chat with talkie live
• Official blog post
• Original announcement on X
• Discussion on r/accelerate
• Discussion on r/singularity

Both models are Apache 2.0 licensed and open-weight on Hugging Face. The team is already planning a GPT-3-scale vintage model for later this year.

Researchers Alec Radford (GPT, CLIP, Whisper), Nick Levine, and David Duvenaud just released talkie: a 13 billion parameter language model trained exclusively on text published before 1931. No internet. No Wikipedia. No World War II. Its worldview is frozen at December 31, 1930.

Why does this matter?

Every major LLM today — GPT, Claude, Gemini, Llama — ultimately shares a common ancestor: the modern web. That makes it nearly impossible to tell what these models genuinely reason versus what they simply memorized.

Talkie breaks that lineage entirely. From the team:

>"It's an important question how much LM capabilities arise from memorization vs generalization. Vintage LMs enable unique generalization tests."

The most striking example: talkie can learn to write Python code from just a few in-context examples, despite having zero modern code in its training data. It's reasoning from 19th-century mathematics texts, not retrieval.

What it's being used to study

• Long-range forecasting: how well can a model "predict" the future from its frozen vantage point?
• Invention: can it develop ideas that postdate its knowledge cutoff?
• LLM identity: what makes a model itself? Talkie's alien data distribution helps isolate what's architecture vs. what's just "vibes absorbed from the web"

Links

• 💬 Chat with talkie live
• 📄 Official blog post
• 🐦 Original announcement on X
• 📌 Discussion on r/accelerate
• 📌 Discussion on r/singularity

Both models are Apache 2.0 licensed and open-weight on Hugging Face. The team is already planning a GPT-3-scale vintage model for later this year.

AI researchers (Nick Levine, David Duvenaud, Alec Radford) just released “talkie,” a 13B language model trained on 260B tokens of text from before 1931, so it basically talks like someone whose worldview is stuck around 1930. The point is to study how LLMs actually generalize vs just memorize, since this model wasn’t trained on the modern web. They trained it on old books, newspapers, scientific journals, patents, and other historical text, then test things like whether it can come up with ideas that were discovered later, forecast future events, or learn bits of Python from examples. Early results seem pretty interesting too, with the model doing surprisingly well on core language/numeracy tasks and showing early signs of learning simple Python despite not being pretrained on modern code.