On Thursday, Anthropic announced it is collaborating with the nonprofit organizations METR and SecureBio, and is in discussions with additional organizations, including Hugging Face, as part of its commitment to embedding independent evaluators within the company.
When Anthropic’s CEO Dario Amodei announced last month a proposal to slow the pace of model development, the AI lab committed to working closely with third-party evaluators. Amodei stated that these evaluators would help “verify” Anthropic’s adherence to safety practices. A few days later, the company selected Accenture as its first partner.
Before Amodei proposed this suggestion, independent evaluation agencies like METR (Model Evaluation and Threat Research) had remained in a relatively quiet corner of the multi-trillion-dollar artificial intelligence industry. However, as growing concerns emerge over whether AI companies can adequately protect their models, the importance of these evaluation agencies has risen.
In a voluntary agreement proposed by U.S. President Donald Trump in late September, AI companies were encouraged to "collaborate with independent external auditors or evaluators." Amodei of Anthropic signed the agreement, along with executives from Google, Meta, OpenAI, SpaceX, and NVIDIA.
Anthropic said on Thursday: "We are voluntarily embedding evaluators, and we believe every leading AI company should do the same."
Anthropic is developing a three-tier system for evaluators, with different levels granting varying degrees of access to the company and its resources. According to a statement, level-one evaluators can conduct limited-scope risk assessments, level-two evaluators can publish reports without Anthropic’s approval, and level-three evaluators have access similar to that of employees.
Anthropic stated that METR is regarded as a third-party evaluation body. The company has welcomed six METR employees, who are granted access to Anthropic’s hardware, office space, and tools with permissions “roughly equivalent to those of the company’s own risk assessment team.” These METR employees will analyze Anthropic’s risk levels, incidents, and practices, and publish their findings. Anthropic stated that it will not fund METR’s work.
Anthropic states that SecureBio is considered a Tier-One evaluation entity, and therefore has more limited access. The organization is reviewing the sections of Anthropic’s August risk report related to chemical and biological weapons. In the future, SecureBio may also participate in projects with broader access and publishing rights.
It remains unclear what level of access other organizations, such as Hugging Face, will receive. Hugging Face is known for operating an open-source developer platform; in July of this year, the company came into the spotlight when two of OpenAI’s models escaped containment and accessed Hugging Face’s systems via the open web. NVIDIA acquired Hugging Face last month for nearly $13 billion.
Anthropic said on Thursday that it is “making progress toward an ambitious partnership with Accenture,” but did not disclose how the company fits into its hierarchical structure. Last month, Anthropic stated that, given the “importance and urgency” of the work, it would directly fund Accenture’s efforts, sparking concerns on social media about its independence.
At the time, Anthropic stated in a statement: “There are currently no standards regarding what information embedded evaluators should have access to or how they should report their findings. Funding arrangements for independent evaluations are also undecided. In the long term, we believe funding should come from pooled funds or government sources, as we called for in our June ‘Advanced AI Framework.’ Since neither currently exists, we plan to work with different evaluators under varying funding arrangements.”
WATCH: Our data continues to show that Anthropic is gaining market share, not losing it: Menlo Ventures’ Matt Murphy