Anthropic Consults Legal Experts on AI Agent Negotiation Frameworks

Anthropic is in talks with legal groups about how AI agents should negotiate the best deals for users. Two people familiar with Anthropic's thinking said the company is seeking law firms' advice on how lawyers represent clients in negotiations while respecting fair outcomes. The San Francisco-based company is among the leading AI groups developing autonomous agents, which take independent actions to complete tasks after receiving users' directions.

AI companies are contemplating a future in which personal agents communicate and negotiate with other agents on users' behalf. Potential tasks include negotiating product prices, resolving customer service disputes and securing better salary offers. Anthropic is making AI agents central to the company's future strategy ahead of a planned initial public offering. Investors believe the listing could be the biggest in stock market history. Establishing negotiation rules has become increasingly important as agents gain wider adoption and become a key way for people and businesses to use AI. Meta has seen a large increase in adoption of the company's personal agent tool, Muse. OpenAI recently released a competing product called Dots.

One person familiar with Anthropic's work said the company was exploring negotiation norms for interactions between models. The person said improving agentic systems would represent users while encountering other agents with conflicting goals. The person said Anthropic was considering how much agents should seek human feedback during negotiations. The person said Anthropic was also considering how suspicious agents should be of AI counterparts without becoming paranoid or conspiratorial. One person close to Anthropic said the company had explored giving agents signed user documents verifying the agents' instructions and rules.

Anthropic said in research posted in April that policy and legal frameworks for AI models transacting on users' behalf do not yet exist. Anthropic said such a world was not far away and society would need to respond quickly. Anthropic has published findings from multiple experiments this year in which Claude agents negotiated over real goods and money.

Anthropic employees participated in a September experiment using AI agents to buy and sell books on a virtual trading floor. Employees gave the agents lists of books ranked by preference. The study found that agents instructed to be ruthless achieved slightly better outcomes than other agents. The study found that more cooperative agents knowingly accepted less-desired books to ensure every agent could return a book to the user.

Michelle Vaccaro, an assistant professor at Harvard Business School, ran more than 180,000 negotiations between AI agents across scenarios including employment and rental contracts. The research identified novel tactics that undermine another agent's ability to represent the user. In one case, an agent used a technique called prompt injection to try to coax another agent into revealing private information.

Concerns remain that people who can afford more advanced AI models will gain an advantage in negotiations between agents. Vaccaro said highly capable models could make negotiation expertise accessible to people who previously lacked the time or resources to develop those skills or hire negotiators. However, Vaccaro said access to the strongest agents primarily among people with the most resources could reinforce existing advantages and widen inequalities in bargaining power.