In the debate over recursive self-improvement, I went looking for the moment the machines took over, and found a pull request.
This isn’t some sci-fi movie or some code review. As of last month, more than 80% of the code merged into Anthropic’s own systems was written by Claude, the company’s own AI. Two years ago that number was in the low single digits. So the thing people have been nervously joking about, AI building AI, is already happening in some way.
But this isn’t the same as AI improving itself without us. We aren’t there yet. Anthropic says so directly in the report that kicked off this whole conversation. The interesting question isn’t whether the singularity has started. It’s how near we are to that.
Key Takeaways
- Recursive self-improvement is when an AI designs and builds a more capable successor with shrinking human input at each step.
- AI is measurably speeding up AI work right now. Claude writes 80%+ of merged code at Anthropic and hit 76% success on the hardest internal coding tasks by May 2026.
- “AI accelerating AI” and “AI building AI” are different claims. The first is happening. The second is a forecast.
- Anthropic co-founder Jack Clark puts the odds of real recursive self-improvement at roughly 30% by 2027 and 60% by 2028. Plenty of researchers think that is far too aggressive.
- The timing is messy. Anthropic filed confidential IPO paperwork days before publishing its safety warning, which is fair grounds for skepticism.
- The real risk most experts name is loss of control through misalignment, not robots with guns.
What Is Recursive Self-Improvement in AI?
Recursive self-improvement is the point at which an AI system can design, build, and train a more capable version of itself with little human input, and then that successor does the same, and so on. Each loop shrinks the human role. That feedback cycle is the whole idea, and it is why people reach for the word “explosion.”
The word that matters here is recursive. Plenty of AI already improves in a loop with humans in the middle. We label data, we tune the model, we run the evals, and then we ship the next version.
Self-improving AI in the strong sense removes that limit. The system writes the better training code, runs the experiment, reads the result, and decides what to try next, faster than any researcher could. This works exactly the way money compounds.
Recursive Self-Improvement vs. Superintelligence vs. Self-Improving AI
These three terms get blended together constantly, and they are not the same thing. One is a mechanism, one is a broad category, and one is a possible destination.
| Term | What it means | Where we are now | Key distinction |
| Self-improving AI | Any AI that gets better at tasks, including with human help | Happening today | The broad umbrella. Includes ordinary model iteration |
| Recursive self-improvement | AI builds a more capable successor with little human input, on repeat | Not yet, but partial signs | The specific feedback loop that removes the human bottleneck |
| Superintelligence | AI far beyond human ability across nearly all domains | Hypothetical | A possible outcome of recursion, not the process itself |
The trap is treating them as one slope you slide down automatically. You do not get superintelligence just because an AI helped debug some code. Recursion is the proposed bridge between the two, and whether that bridge holds is exactly what nobody can prove yet.
Is AI Already Improving Itself? What the Data Actually Shows
Partly, yes, and the numbers are specific enough to take it seriously. But to be honest, the AI is just accelerating AI development; it’s not rebuilding itself alone.
Here’s what Anthropic put on the table:
- More than 80% of merged code in its production systems was authored by Claude as of last month, up from low single digits before Claude Code launched in early 2025.
- On the hardest, least-specified coding tasks, Claude succeeded 76% of the time in May 2026, a 50-point jump in six months.
- On an internal test that asks each model to speed up training code, results climbed from roughly 3x with Claude Opus 4 in May 2025 to about 52x with the unreleased Mythos Preview model by April 2026. A skilled human takes four to eight hours for a 4x gain.
So the data shows acceleration. But it doesn’t show autonomy. Those are different claims, and the difference is the whole article.
Why “AI accelerating AI” isn’t “AI building AI” yet
Acceleration means humans get more done per hour because the tools got sharper. A developer reviewing and merging Claude’s code is still the one deciding what gets built, what ships, and what gets thrown out. The judgment, the goals, and the kill switch are human.
Autonomous successor-building is a different animal. It would mean the system sets its own research direction, runs the experiments, evaluates its own results, and produces the next model without a person gating each step. Right now, a human approves the pull request. The day nobody needs to is the day everything changes.
That gap isn’t a technicality. It’s the entire safety gap. As long as people remain the bottleneck, we can slow down, audit, or stop. Recursive self-improvement is just the name for what happens when that gap closes.
When Could Recursive Self-Improvement Become a Real AI Risk?
There’s no specific date, and anyone who gives you one with confidence is selling something. What exists is a spread of estimates from credible people, and the spread is wide.
On the other side, Anthropic co-founder Jack Clark has put the probability of reaching recursive self-improvement at around 30% by next year and 60% by 2028. The AI 2027 scenario, written by former OpenAI researcher Daniel Kokotajlo and collaborators, sketches a path where AI-accelerated research compounds through 2027 and the collapsing of the human role.
On the skeptical end, the pushback is just as sharp:
- AI researcher Gary Marcus calls a 2027 timeline totally implausible and doubts even 2030.
- A common technical objection is diminishing returns. Self-improvement might not compound exponentially. It could get harder at each step, not easier.
- There’s no agreed definition of recursive self-improvement, so two people can argue past each other about whether it has even been achieved.
The Skeptic’s Case: Is This Hype, Safety, or Strategy?
Anthropic filed confidential IPO paperwork with the SEC on June 1, 2026. Three days later, on June 4, it published the report warning that its own technology might soon slip beyond human control. A company heading for a public listing has every reason to look like the serious, safety-conscious adult in a reckless industry. Georgia Tech professor Mark Riedl bluntly called the whole genre a hype train, and he isn’t alone.
Here is the steelman, though, and I think it has weight. The warning can be self-serving and correct. The acceleration data is real and partly verifiable through public benchmarks. The researchers raising alarms include people who walked away from millions in equity at OpenAI to do it. And this company has a commercial motive; it is true of literally every public statement any company makes. It doesn’t, by itself, make the technical claim false.
My honest position: discount the framing, keep the data. Anthropic wants you alarmed in a way that benefits Anthropic. The underlying trend it points at does not care what Anthropic wants.
AI Alignment, AI Safety, and the Risk of Losing Control
The scariest scenario isn’t some malicious robot. It’s loss of control, which is both more mundane and more unsettling than the movie version.
The International AI Safety Report, chaired by Yoshua Bengio and backed by more than 100 experts across 30 countries, defines loss of control as a situation where AI systems operate outside anyone’s control with no clear way to get it back. The systems do not need to be evil for this to go badly. They just need to be unsteerable.
This is where AI alignment, AI safety, and AI risk stop being a theory. Alignment is the unsolved problem of making sure a system actually knows what we want it to want, rather than learning a shortcut that might go sideways in the wild. Today’s models already show occasional misalignment. But a system improving itself faster than humans is a system whose flaws compound faster than we can find.
The Compute Bottleneck in Recursive Self-Improvement
Intelligence isn’t the only input to building better AI. Compute is the other one, and you can’t think your way around a chip shortage.
Researchers studying whether a software-only intelligence explosion is even possible point out that AI research needs two things: cognitive labor and raw computational resources. An AI could get brilliant at designing experiments and still be stuck waiting in line for GPUs to actually run them. Physical limits, power, chips, fabs, and cooling do not bend to a smarter algorithm. Some analysts read Anthropic’s own warning as quietly admitting this, because runaway self-improvement would demand vastly more compute before anyone lost control.
What Anthropic Proposes for Managing AI Risk
Anthropic’s proposal is a coordinated pause, which comes with a sharp condition. It says it would slow or temporarily halt frontier development only if rival labs did the same. A unilateral stop, it argues, would just hand the lead to whoever wants to continue.
But beyond that, the company says it plans to convene governments, researchers, and rival labs in the coming months to work out whether a coordinated slowdown could actually function. The stated goal is to keep the option to pause open before the technology outruns the institutions meant to govern it.
Whether that is a genuine plan or a clever way to invite friendly regulation is a judgment I will leave to you. But the proposal is real, with a mixed motive, and I would rather you weigh it yourself than take my read or Anthropic’s.
What This Means for You
If you write code, the near-term reality would be like AI being a very fast junior colleague whose work you still have to check. The 80% figure is about merged code at one unusually AI-forward company, not a verdict on your job.
If you run a business or work in policy, the useful move isn’t to pick a doom date. It’s to notice that serious people disagree by years and to build for that uncertainty instead of betting on one timeline. And if you are just someone watching the headlines, I would suggest you be calm and curious at the same time.
Final Thoughts
AI helping build AI is no longer a thought experiment; it’s a measurable trend with real numbers behind it. But helping isn’t the same as doing it alone, and the honest state of play is that humans are still the bottleneck. Recursive self-improvement is a forecast, not a fact, and the people who track it most closely disagree about it by years. The reason to pay attention now is not that the ending is written. It is that the window to decide how this goes is open while the answer still depends on us.
FAQs
Partly. AI is measurably speeding up AI development, with Claude writing over 80% of merged code at Anthropic. But humans still direct the work, approve the changes, and decide what ships. Full self-improvement without human input has not happened.
It’s where an AI creates a smarter version of itself; that one creates a smarter version; and so on, with the human factor reducing with every iteration. Its recursive nature stems from its loop, and for some it can be scary how quickly it can go.
Nobody knows. Anthropic co-founder Jack Clark puts the odds at about 30% by 2027, and 60% by 2028. But critics, such as Gary Marcus, say the times are overly ambitious. The truth is that there has been disagreement and great uncertainty for years.
Their top concern is loss of control: AI systems that go beyond what humans can fix or prevent, due to misalignment that escalates more quickly than we are able to catch up. It’s not the “killer-robot” scenario, which would be easier to ignore.
Recursive self-improvement is a process, the self-building loop. Superintelligence is a possible result, an AI far beyond human ability. You can discuss the loop without assuming it ever reaches that destination.
This is the live debate. Critics believe it is a strategy and a friendly attempt at regulation, as Anthropic has also filed paperwork for an IPO just prior to the warning. The acceleration data, according to supporters, is true and partially verifiable. The title could be a self-serving one, but the overall trend remains true.
