Tool use and function calling

A language model can only emit tokens. "Tool use" is the set of conventions that let some of those tokens be read as a request to run code, search, or call an API, and that feed the result back as more context. It is the mechanism behind every agent, and the place where most agent security problems begin.

The mechanism: tools as structured text

A tool call involves three parties: the model, the harness (the program running the model), and the tool. The harness tells the model which tools exist, usually as a name, a natural-language description and a JSON Schema for the arguments:

{
  "name": "get_weather",
  "description": "Current weather for a city.",
  "input_schema": {
    "type": "object",
    "properties": { "city": { "type": "string" } },
    "required": ["city"]
  }
}

When the model decides a tool would help, it emits a structured block such as {"name": "get_weather", "input": {"city": "Oslo"}} and stops generating. The harness parses the block, runs the function, and appends the result to the conversation as a new message. The model then continues, conditioned on the result. Nothing in the network executes code. The model predicts that a call belongs here and which arguments it should have, and ordinary software does the rest.

This explains the characteristic failures. A model can call a tool that does not exist, fill in plausible but wrong arguments, or misread a result. Gorilla (2023) targeted exactly these: GPT-4 often generated inaccurate arguments and hallucinated API usage, and a fine-tuned model combined with retrieval over current API documentation did better.^[1] Reasoning models extended the pattern by calling tools in the middle of their chain of thought. OpenAI's o3 and o4-mini (April 2025) were trained to decide when to search, run Python or transform an image while still reasoning.^[2]

How models learned to call tools

Browsing as an environment. WebGPT (2021) fine-tuned GPT-3 to answer questions by issuing text commands to a browser and collecting references. Training combined imitation of human demonstrations with optimisation against a learned human-preference reward. Its answers were preferred to those of its human demonstrators 56% of the time.^[3]

Self-supervised tool use. Toolformer (2023) taught a 6.7B-parameter GPT-J model to call a calculator, a question-answering system, two search engines, a translator and a calendar, starting from only a handful of demonstrations per API.^[4] The model first proposes candidate calls inside ordinary training text. A call is kept only if inserting it and its result makes the following tokens easier to predict. Write $L_i^{-}$ for the loss on the next tokens without the call (taking the better of "no call" and "call with no result") and $L_i^{+}$ for the loss with call and result. The call is kept if

Worked example. In the sentence "The store sold 400 of its 1,400 items, which is 29% of its stock," a candidate call [Calculator(400/1400)] → 0.29 is inserted before "29%". Suppose the loss on the following tokens is 3.1 nats without the call and 1.6 nats with it. The reduction is 1.5 nats. With $\tau_f = 1.0$ the call is kept as training data. A call to a search engine at the same position that returns irrelevant text might reduce the loss by only 0.2 nats and is discarded. The model then trains on its own filtered calls. The usefulness criterion is the language-modelling loss itself, so no human has to label which calls are good.

Reasoning interleaved with acting. ReAct (2022) prompted models to alternate free-text "thoughts" with actions such as Wikipedia lookups. With one or two in-context examples it beat imitation- and RL-trained baselines by 34 and 10 absolute percentage points on the ALFWorld and WebShop benchmarks. On question answering, grounding in retrieved text reduced hallucination.^[5] The thought–action–observation loop is still the skeleton of most agents. The next lesson builds on it.

The interface is part of the model's capability. SWE-agent (2024) found that designing the commands an agent uses to view, navigate and edit a repository for the model changed results substantially. Its tailored "agent-computer interface" reached 12.5% on SWE-bench, well above the previous non-interactive state of the art.^[6] Tool descriptions, output formats and error messages should be treated as part of the prompt.

A shared protocol: MCP

Every application that offered tools used to define its own integration format. Anthropic introduced the Model Context Protocol (MCP) on 25 November 2024 as an open standard for connecting AI applications to data and tools.^[7] In the current specification (version 2025-11-25), hosts (LLM applications) run clients that keep stateful JSON-RPC 2.0 connections to servers. Servers can offer three kinds of feature: resources (data), prompts (templates) and tools (functions the model may execute). Clients can offer servers sampling (requesting a model completion), roots (filesystem boundaries) and elicitation (asking the user for input). The design takes its inspiration from the Language Server Protocol used by code editors.^[8]

The value is combinatorial. With 10 AI applications and 50 services, bespoke integrations need up to $10 \times 50 = 500$ connectors. A shared protocol needs $10 + 50 = 60$ implementations. On 9 December 2025 Anthropic donated MCP to the Agentic AI Foundation, a fund under the Linux Foundation that Anthropic, Block and OpenAI co-founded. At the time Anthropic reported more than 10,000 active public MCP servers and over 97 million monthly SDK downloads.^[9]

Tool output is untrusted input

Once a model reads tool results, anyone who can write into those results can write into its prompt. Greshake et al. (2023) called this indirect prompt injection. Instructions planted in a web page, email or document that an application retrieves can take over the model, because LLM-integrated applications blur the line between data and instructions. They demonstrated data theft and self-propagating attacks against real systems, including Bing's GPT-4-powered chat.^[10]

The risk grows with the agent's privileges. An assistant that reads email and can send it can be told by an incoming email to forward the inbox. AgentDojo (2024) turned this into a benchmark of 97 realistic tasks and 629 security test cases. It found that models failed many tasks even without attacks, and that existing attacks broke some security properties but not all.^[11] The MCP specification takes the same position. Tools are arbitrary code execution, a tool's self-description should be treated as untrusted unless the server is trusted, and hosts must obtain explicit user consent before invoking a tool.^[8]

The practical rules follow from the mechanism:

• Least privilege. Give an agent only the tools the task needs, and scope credentials to match.
• Separate reading from acting. Require confirmation before irreversible or outward-facing actions such as payments, sending messages or deleting data, especially after the agent has read untrusted content.
• Do not let the model grade its own safety. A check that runs in the same context as the injected text can be fooled by that text. Independent checks are the subject of this lab's correlated-critique note.

Key takeaways

• A tool call is structured text the model emits. The harness executes it and returns the result as context. The network itself never runs code.
• Toolformer kept a candidate call only if it lowered the loss on the following tokens. ReAct showed that interleaving thoughts and actions beats either alone.
• Interface design (tool descriptions, output formats, error messages) affects agent performance as much as the model does.
• MCP standardises tools, resources and prompts over JSON-RPC, turning $N\times M$ integrations into $N+M$. Since December 2025 it has been governed under the Linux Foundation.
• Every tool result is untrusted input. Indirect prompt injection is the central security problem of tool use, so limit privileges and gate irreversible actions.

Sources