What Is Recursive Self-Improvement (RSI), and Why Is Everyone Talking About It Now?

I went into Dario Amodei’s “We Must Pace the Frontier” expecting a general safety essay and came out stuck on one sentence: “This dynamic is called recursive self-improvement, and it is starting to happen across the industry, including at Anthropic, as we and others have described.”

Present tense. Not “may one day.” The essay then proposes a literal speed limit on it — “slowing the rate from ‘extremely fast’ to ‘only somewhat fast’ gives up relatively little strategic advantage, while potentially greatly improving safety.”

That’s a strange thing to propose about a hypothetical. So I went looking for what changed.

The idea is sixty years old

I. J. Good wrote it down in 1965. Build a machine that’s better than humans at designing machines, and the first thing it designs is a better version of itself. That one designs a better one. Good called the result an “intelligence explosion” and noted, dryly, that it would be the last invention humanity needed to make.

For sixty years this stayed philosophy, because nobody had the first machine. The loop needs an AI that can do AI research — not answer questions about it, actually do it: form a hypothesis, write the training code, run the experiment, read the results, decide what to try next. Every one of those steps was out of reach until very recently.

So RSI is not a new concept. What’s new is that people are measuring it.

The numbers the labs are pointing at

Anthropic’s public claims, as of mid-2026, are specific enough to argue with:

  • Claude writes more than 80% of the code merged into Anthropic’s own codebase. Jack Clark puts the per-person code volume at eight times what it was in 2024.
  • On a GPU kernel optimization task, Claude’s speedup went from 7x to 73x over the course of a year.
  • On picking which research direction to pursue, Claude’s choices scored better than human researchers’ 51% of the time in November 2025 and 64% of the time by April 2026.

OpenAI is blunter about the goal than about the evidence. Jakub Pachocki has said a fully automated AI researcher by March 2028 is “the most important goal for us,” and OpenAI says GPT-5.3 Codex had “a significant hand in its own development from start to finish.”

The independent number is METR’s. They measure how long a task a model can complete autonomously — not how well it scores on a quiz, but what length of human work it can finish end to end. Their original figure was a doubling every seven months. Their January 2026 update put the doubling rate since 2024 at about 89 days. Claude Opus 4.5 sits around 320 minutes, roughly five hours of human-equivalent work, though the confidence interval on that is enormous — 170 to 729 minutes.

Three months per doubling, on a metric that’s basically “how much of a researcher’s day can this thing take over.” That’s the number that moved RSI from the philosophy department onto policy roadmaps.

Then someone actually tried it

In July 2026, a Princeton-led group ran the experiment the claims imply. Peter Kirgis, Sayash Kapoor and colleagues took two unpublished NeurIPS 2026 submissions, gave Claude Opus 4.8 six days, $3,000 in API credits, and a GPU budget, and asked it to do the research.

Both resulting papers were rejected by the original authors. Not “needs revision.” Rejected.

What’s interesting is how it failed. The engineering was fine — the agents set up the whole pipeline with no human help. The failures were all judgment: running experiments on tiny synthetic datasets, committing early to an unpromising approach and never seriously reconsidering it, backtracking badly from dead ends, burning tokens and compute without noticing, and drifting off the instructions. Kapoor’s summary is harsh and worth quoting: “the agents were unambiguously bad at carrying out the research itself.”

The part I didn’t expect: Anthropic’s own cofounder agrees. Jack Clark called it a “bearish signal on short recursive self-improvement timelines,” pointing at “a certain absence of valuable, intuitive creativity in today’s AI systems.” The same person who gives RSI a 60% chance of happening by the end of 2028.

Both things can be true. Execution is close to solved; taste isn’t. The loop has three of its four pieces.

Why it’s a policy question and not just a research question

Because the missing piece might close quickly, and nobody knows what happens on the other side of it. Anthropic’s Responsible Scaling Policy — v3.4 as of July 2026 — has explicit AI R&D capability thresholds, and crossing AI R&D-4 obligates the company to make an affirmative case that it has mitigated misalignment risk before proceeding. That’s a commitment written specifically for a model good enough to meaningfully accelerate its successor.

Dario’s speed limit proposal is the same instinct at industry scale. And the safety people are already uncomfortable: Anthropic’s Dave Orr said “our margin for error is getting smaller over time… now we’re driving at 75 instead of 25.”

The obvious objection is that all of this is self-serving. Gary Marcus called the acceleration claims “just faster coding” from companies whose valuations depend on the story being true. He’s not wrong that the incentive exists. But the METR trend is measured by an outside org, and the Princeton result cuts against the labs and still got amplified by one of them.

Sources