What's Anthropic's stance on open-weights models?

Dario Amodei, Anthropic's CEO, has weighed in on the debate surrounding open-weights models, especially those originating from China. It's worth noting that Amodei stresses Anthropic has never backed a ban on these models, seeing them as a public good that brings value to various stakeholders. The real concern, he argues, lies not with the models themselves, but with the potential risks that come with their development and use. Amodei points to two key worries: the risk of authoritarian governments leveraging more powerful AI models to gain military superiority or suppress their citizens, and the risk of these powerful models being misused for cyberattacks or biological attacks.

Amodei proposes a few measures to mitigate these risks. For starters, he suggests the US should limit the sale of powerful chips and chipmaking equipment to China, which would curb China's ability to build more powerful models. He also recommends cracking down on industrial-scale distillation operations, which currently allow China to build better models than its chip capacity would normally permit. Lastly, Amodei thinks all capable models - whether open or closed - should undergo mandatory safety testing to identify potential risks before they're released. This approach, he believes, would help address the concerns surrounding open-weights models without resorting to a blanket ban. As Amodei puts it, the goal is to find a balance that promotes innovation while keeping people safe.

The proposed measures are designed to tackle the root causes of the concerns, rather than just restricting the use of open-weights models. By taking a more nuanced approach, Anthropic hopes to promote a safer and more responsible development of AI models. It's crucial, as this discussion unfolds, to consider both the potential risks and benefits of open-weights models, and to work towards finding solutions that balance national security concerns with the need for progress and innovation in the field of AI.

Source: Anthropic