← back to the archiveCover illustration for “Tool calling: how an LLM uses functions and APIs”
TERMday 172·3d ago·Published ·by Andy Padia

Tool calling: how an LLM uses functions and APIs

In short: Tool calling, also called function calling, lets a model request a function with structured arguments, so tool design decides answer quality.

Tool calling, also called function calling or tool use, is how a language model asks your code to run something. You describe available functions with names, descriptions and a JSON schema; the model replies with a structured request naming one function and its arguments; your code runs it and returns the result. The model never executes anything itself.

Tool calling in plain words

The model is a manager who cannot touch the keyboard. It writes a precise instruction slip, "run get_weather for Bengaluru", and someone else does the work and reports back.

How it works

  1. You send the conversation plus a list of tool definitions.
  2. The model decides whether a tool is needed and returns a tool-call block with arguments matching the schema.
  3. Your code validates the arguments, runs the function and sends back the output.
  4. The model uses that output to answer or call another tool.

OpenAI's function calling guide and Anthropic's tool use documentation describe the same round trip. The idea of a model deciding for itself when to call an external tool was shown in Toolformer in February 2023.

Example

{
  "name": "get_order_status",
  "description": "Look up an order by ID. Use when the user mentions an order number.",
  "input_schema": {
    "type": "object",
    "properties": { "order_id": { "type": "string" } },
    "required": ["order_id"]
  }
}

The description tells the model when to use the tool, which matters as much as the schema.

Where you'll meet it

  • Every major model API: OpenAI, Anthropic, Google and most open-weights servers
  • MCP servers, which package tools for reuse across clients
  • Agent frameworks and coding agents
  • Chat products that browse, run code or query databases

What it changes for you

Your API design is now also prompt design. Clear names, narrow inputs and helpful error messages raise accuracy more than prompt tweaks. For security teams, every tool is an action surface: validate arguments and scope credentials per tool.

Tool calling vs structured outputs

Structured outputs force the model's answer into a schema. Tool calling uses a schema to request an action. Both rely on a JSON schema, for different purposes.

Common mistake

My view: exposing thirty overlapping tools and hoping the model picks well. Fewer, sharper tools with "use when" descriptions beat a big menu every time.

Model Context Protocol · Structured outputs · Agent loop · Prompt injection · AI agent

Design tools the way you would design an API for a new teammate: few, narrow and clearly described.

#field-guide#agents#function-calling#apis

Sources

  1. developers.openai.com/api/docs/guides/function-calling
  2. platform.claude.com/docs/en/agents-and-tools/tool-use/overview
  3. arxiv.org/abs/2302.04761
← older drop
Token: the unit LLMs read, write and bill by
newer drop →
Embeddings: turning text into numbers that capture meaning

related drops

explore all 557 drops →
← back to the archiveday 175