METR Report Finds Anthropic AI Development Doubling Speed

METR report Anthropic AI development speed doubling

A new investigation from METR, the AI safety research group, shows Anthropic’s models are now improving about twice as fast as they were a year ago-and the company’s own AI systems are doing the heavy lifting. The report dropped Thursday after an eleven-week deep dive inside the lab, and it’s some of the clearest proof yet that artificial intelligence is starting to accelerate its own creation. METR’s researchers are confident this isn’t just a case of throwing more compute at the problem, which has historically been the main driver. Instead, the bulk of the speedup as of October came from AI itself, and its contribution has only been growing.

The timing is awkward for Anthropic. The company is gearing up to go public even as CEO Dario Amodei has publicly called for the industry to pump the brakes on frontier development. METR’s work is separate from that program announced last month promising safety researchers “employee-like” access following the high-profile hacks over the summer. While Anthropic voluntarily handed over confidential research, performance data, and employee interviews-going beyond the original ask-the lab kept the right to redact the final report. And they used it heavily. METR posted on X that Anthropic blacked out a majority of the evidence the researchers found most informative, leaving the public to trust METR’s judgment rather than checking the raw material themselves.

Anthropic three unreleased models training complete

According to the report, Anthropic has finished training three models it hasn’t released. Codenamed Model 2, Model 3, and Model 4, they wrapped up in July, August-September, and September-October respectively. An extra unreleased model from 2022 was also flagged. A huge chunk of the productivity surge METR documented comes from these internal systems. In informal Slack polls and interviews, staff using the most capable unreleased model said they were roughly twelve times as productive as they’d be without AI help. That’s about double what people reported just a few months earlier working with a system called Mythos 5.

For now, the acceleration is mostly AI assisting human researchers rather than running the show autonomously. But METR warned there are early hints that fully autonomous contributions will start playing a bigger role soon. The researchers also saw no clear evidence of diminishing returns or bottlenecks that would stop even faster acceleration in the near term. That trajectory raises hard questions about how quickly the tech could advance toward a future where AI systems improve themselves with minimal human involvement.

AI transparency tension Anthropic safety research

Transparency remains the central tension. METR repeatedly stressed its information pipeline was relatively narrow and that a handful of data requests were denied. Because so much supporting evidence was redacted, the report admits many findings allow multiple plausible interpretations, making it tough for outsiders to assess the uncertainties or judge the evidence independently. Anthropic’s willingness to cooperate at all stands out in an industry where independent audits are rare, but the restrictions highlight just how much control frontier labs still hold over what the public learns about the pace and risks of their work.

AI research velocity feedback loop accelerating

For folks building with or using AI, the report signals the feedback loop between model capability and research velocity is tightening faster than many expected. If the trend holds, the gap between what leading labs can do internally and what’s available publicly will widen. Developers should keep an eye on Anthropic’s next public releases-especially any models inheriting capabilities from that unreleased series-and track METR’s future assessments for less filtered views. Safety researchers and policy teams, meanwhile, may need to recalibrate timelines for when autonomous AI research becomes a significant factor, and push for audit frameworks that limit a lab’s ability to redact the most consequential findings.

Frequently asked questions

How much faster are Anthropic's AI models improving compared to a year ago?

METR's investigation found Anthropic's models are improving about twice as fast as they were a year ago, with the company's own AI systems providing the bulk of the acceleration rather than just increased computing power.

What unreleased models has Anthropic completed training recently?

Anthropic has finished training three unreleased models codenamed Model 2 (July), Model 3 (August-September), and Model 4 (September-October), plus an additional unreleased model from 2022.

How much more productive are Anthropic researchers when using their most advanced unreleased AI model?

In informal Slack polls and interviews, staff using the most capable unreleased model reported being roughly twelve times as productive as without AI help, about double the productivity gain reported a few months earlier with a system called Mythos 5.

Why can't the public verify METR's findings independently?

Anthropic redacted a majority of the evidence METR researchers found most informative, and several data requests were denied, leaving the public to trust METR's judgment rather than examine raw materials directly.

Is Anthropic's AI now conducting research autonomously without human involvement?

Currently the acceleration comes mostly from AI assisting human researchers rather than running autonomously, though METR noted early hints that fully autonomous contributions will play a bigger role soon.

Source: The Verge