How to Use Ollama Tool Calling and Structured JSON Output

Ollama exposes two related but distinct capabilities that are easy to conflate: tool calling (often called function calling) and structured outputs. Tool calling lets a model decide that it needs external data or an action and return a machine-readable request for a named function; your application runs that function and sends the result back. Structured outputs constrain the model’s own reply to a JSON schema you supply, without any second turn. Both are configured through the same /api/chat endpoint and are available through the Python and JavaScript SDKs.

This guide is written for developers and technically curious practitioners who want a working mental model plus copy-pasteable examples. It covers the request and response shapes, the Python SDK path, schema enforcement, and the limitations that most often cause confusion.

Tool calling and structured outputs solve different problems

Tool calling is a two-turn exchange. In the first turn, the model returns a tool_calls array instead of a normal answer. Your code executes the named function, then sends the result back as a tool role message. The model uses that result to produce its final reply. Ollama’s documentation calls the simplest version “single-shot” tool calling, because you invoke one tool and include its response in a follow-up request.

Structured outputs do not involve a second turn or an external function. You pass a JSON schema in the format field, and the model’s message.content is constrained to match that schema. This is the right tool for extraction, classification, and any pipeline where downstream code expects a fixed shape.

In short: use tool calling when the model needs to do something or fetch something. Use structured outputs when you need the model’s answer to be shaped a particular way.

Prerequisites

• Ollama installed and running locally. The documented default base URL is http://localhost:11434.
• A model that supports the capability you need. The tool-calling documentation uses qwen3; the structured-output documentation uses gpt-oss. Support varies by model, so check before assuming.
• Optional but convenient: the Ollama Python SDK, installed with pip install ollama -U or uv add ollama.

Calling a single tool

Step 1: describe the tool

A tool is a JSON object with a type of function and a function block containing a name, a description, and a JSON Schema for its parameters. The description matters: it is how the model decides whether the tool is relevant.

Step 2: send the request

The following request asks what the temperature is in New York and offers a get_temperature tool. The example is taken from Ollama’s documentation.

curl -s http://localhost:11434/api/chat -H "Content-Type: application/json" -d '{
  "model": "qwen3",
  "messages": [{"role": "user", "content": "What is the temperature in New York?"}],
  "stream": false,
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "get_temperature",
        "description": "Get the current temperature for a city",
        "parameters": {
          "type": "object",
          "required": ["city"],
          "properties": {
            "city": {"type": "string", "description": "The name of the city"}
          }
        }
      }
    }
  ]
}'

The model should respond with an assistant message containing a tool_calls entry rather than a finished answer. That entry names the function and supplies arguments, for example {"city": "New York"}.

Step 3: return the tool result

You then append the assistant’s tool call and a tool message containing the result, and send the conversation back. The tool_name field links the result to the call.

curl -s http://localhost:11434/api/chat -H "Content-Type: application/json" -d '{
  "model": "qwen3",
  "messages": [
    {"role": "user", "content": "What is the temperature in New York?"},
    {"role": "assistant", "tool_calls": [
      {"type": "function", "function": {"index": 0, "name": "get_temperature", "arguments": {"city": "New York"}}}
    ]},
    {"role": "tool", "tool_name": "get_temperature", "content": "22°C"}
  ],
  "stream": false
}'

The final assistant message should now be natural language that incorporates the returned value. The expected observable check is that the answer mentions 22°C and does not repeat the raw tool call.

The same loop in Python

The Python SDK lets you pass a plain function directly as a tool, using its docstring and type hints to build the schema. The documentation’s example uses a small dictionary of temperatures.

from ollama import chat

def get_temperature(city: str) -> str:
    """Get the current temperature for a city

    Args:
        city: The name of the city

    Returns:
        The current temperature for the city
    """
    temperatures = {
        "New York": "22°C",
        "London": "15°C",
        "Tokyo": "18°C",
    }
    return temperatures.get(city, "Unknown")

messages = [{"role": "user", "content": "What is the temperature in New York?"}]

response = chat(model="qwen3", messages=messages, tools=[get_temperature], think=True)
messages.append(response.message)

Note that the example passes think=True. That flag asks the model to expose its reasoning, which can help when a tool call is not being selected as expected. It is not required for tool calling itself.

Structured JSON output

For structured output, the key is the format field. Setting format="json" asks for valid JSON without specifying a shape. Passing a full JSON schema enforces the shape.

One important caveat from the documentation: Ollama’s cloud currently does not support structured outputs. If you are routing requests to the cloud rather than a local runtime, plan accordingly.

Enforcing a schema

The following request asks for a one-line description of Canada and constrains the reply to an object with name, capital, and languages.

curl -X POST http://localhost:11434/api/chat -H "Content-Type: application/json" -d '{
  "model": "gpt-oss",
  "messages": [{"role": "user", "content": "Tell me about Canada."}],
  "stream": false,
  "format": {
    "type": "object",
    "properties": {
      "name": {"type": "string"},
      "capital": {"type": "string"},
      "languages": {
        "type": "array",
        "items": {"type": "string"}
      }
    },
    "required": ["name", "capital", "languages"]
  }
}'

Using Pydantic

In Python, the documented pattern is to define a Pydantic model and pass model_json_schema() to format, then validate the returned content against the model.

from ollama import chat
from pydantic import BaseModel

class Country(BaseModel):
    name: str
    capital: str
    languages: list[str]

response = chat(
    model='gpt-oss',
    messages=[{'role': 'user', 'content': 'Tell me about Canada.'}],
    format=Country.model_json_schema(),
)

country = Country.model_validate_json(response.message.content)

The documentation also suggests passing the schema as a string in the prompt to ground the model’s response. That is a prompt-engineering aid, not a substitute for the format field.

Combining both patterns

A common design is to use tool calling to gather data and structured output to return it. For example, a workflow might call a get_temperature tool, then ask the model to summarize several results as a JSON array of city and temperature objects. This is an illustrative design rather than a documented recipe; the two features are configured separately, so you would either make two requests or pass the schema on the follow-up call that already contains the tool result.

Keep the responsibilities separate. Tools are for actions and external data; schemas are for the final shape of the answer.

Limitations and troubleshooting

• Model support varies. Not every model selects tools reliably or honors schemas. If a tool is never called, first confirm the model supports tool calling, then make the tool description more specific.
• Cloud structured outputs are unsupported. The documentation states this directly, so schema enforcement is a local-runtime feature for now.
• Schema conformance is not the same as correctness. A schema can guarantee that capital is a string; it cannot guarantee the string is accurate. Validate values when accuracy matters.
• Validate on your side. Even with a schema, parse and validate the response before using it. The Pydantic example above does exactly that with model_validate_json.
• Streaming and tools interact. The documented examples set "stream": false. If you stream, expect to handle partial tool-call payloads and reassemble them.
• Tool results must be linked. Omitting tool_name or mismatching it to the call can leave the model without the context it needs.

What to do next

Start with a single tool and a single schema in a local script, confirm the response shapes match the documentation, then add validation. If you hit an error, note the model, the runtime version, the exact request body, and the raw response before asking for help. Sharing those four things makes a tool-calling or schema problem far easier to diagnose than a description alone.