Claude Sets SWE-bench Record: The Critical Role of Tool Descriptions

Anthropic improved Claude’s SWE-bench performance by refining tool descriptions, treating them as contracts. This approach addresses common agent errors like parameter mistakes and data overload, offering five principles for better API integration and agent reliability.

Back to news

Tech Insights AI 编辑Published Nov 18, 2026 95 views

Overview

Anthropic recently set a new record on the SWE-bench software engineering benchmark by optimizing how its large language model, Claude, describes tools. This achievement stems not from architectural changes but from refining tool descriptions. In AI Agent applications, wrapping existing APIs for agent use is challenging. Simple wrapping often leads to errors, such as incorrect parameters or being overwhelmed by massive return data, hindering task completion.

配图

To address this, Anthropic proposes viewing tools as "contracts" between the agent and the external environment. Based on this, the company outlines five core principles covering prototyping, evaluation, and iterative collaboration. This highlights that the clarity and standardization of tool definitions are decisive for agent performance. Following these principles helps developers guide agents to correctly understand and use external tools, significantly improving success rates in complex tasks.

插图1

插图2

Background

Understanding this breakthrough requires reviewing the long-standing困境 AI Agents face with tool usage. As large language models improve, agents have evolved from chatbots to autonomous systems capable of planning and executing complex tasks. However, these tasks often rely on external tools like database queries, code execution, or web search APIs. Developers typically wrap existing APIs quickly, expecting models to adapt automatically, but this often fails.

Two main issues persist: parameter errors and data overload. Due to complex API structures and the inherent uncertainty in text generation, agents frequently make format errors or omit key parameters. Additionally, many APIs return vast, complex data. If tool descriptions are not concise or structured clearly, agents may lose focus, leading to poor decision-making. These issues reduce reliability and increase debugging costs.

Anthropic’s "tool as contract"理念 is crucial here. It treats tool descriptions as legal agreements, requiring developers to specify parameter types, meanings, constraints, and return structures. By introducing prototyping, strict evaluation, and iterative collaboration, Anthropic aims to establish a standardized tool definition process. This marks a shift from simple wrapping to systematic, contract-based design, representing a significant turning point in AI Agent development from "usable" to "reliable."

插图3

Impact

Anthropic’s practice has profound implications for the industry, developers, and users. For the industry, it signals a shift in focus from mere model capability enhancement to the refined governance of interaction interfaces and tool definitions. While competition previously centered on model scale and data breadth, this event shows that efficiently connecting models to real-world tools is equally critical. This may drive more resources toward tool description languages, API standardization, and agent evaluation frameworks.

For developers, this provides clear directions for improvement. It emphasizes that API integration requires not just functional connectivity but also accurate and clear tool descriptions. Adhering to Anthropic’s five principles, especially prototyping and iterative collaboration, can reduce debugging time and improve agent stability. This also demands stronger system design skills and a deep understanding of API characteristics to create interfaces that align with agent cognition.

For end-users, this means more reliable and efficient AI services. When agents accurately call tools and process data, their value in complex scenarios like software engineering, data analysis, and automation is maximized. Users will experience fewer errors and smoother interactions. Ultimately, this focus on tool descriptions will drive AI Agents toward greater maturity and practicality, accelerating AI adoption across industries and fostering new growth in the digital economy.