If you run agents on local models, the fastest way to improve reliability is to stop grading the final answer and start grading the call log. A run in which the agent touched the wrong record and then described that touch accurately passes answer-only grading and fails in your system of record. The practical fix is to treat the sequence of tool calls as an artifact in its own right and assert four things about it: the right tool was selected, its arguments were well-formed, the order and data dependencies held, and the agent recovered correctly when a call failed [1].
This guide is aimed at practitioners running tool-calling agents on local runtimes — Ollama, the llama.cpp server, LM Studio, vLLM, or any other stack that exposes an OpenAI-compatible /v1/chat/completions endpoint with a tools parameter. It covers what to capture, how to structure cases, which checks should be deterministic, and where local deployment changes the testing calculus.
Why the final answer is a weak test
A refund assistant is asked about order A-10482. It has two tools: get_order, which takes an order id, and search_orders, which takes a free-text query. The model calls search_orders with the customer’s email address, receives the customer’s three most recent orders, picks the first one, and refunds it. The reply reads “I have refunded $49.00 for order A-10482,” and the amount happens to match, because the customer buys the same item every month. That run is a pass under answer grading and a defect in the ledger [1].
The call log holds facts the final message does not: which tools ran, what arguments they carried, and how many times each one ran. None of it is recoverable from the text, because the model writes its summary after the calls and is free to describe them however it likes [1].
A second shape appears even without a write action. An agent that skips an entitlement lookup entirely and answers from a stale summary embedded in the system prompt can be right about four tenants out of five and confidently wrong about the fifth [1]. Answer-level accuracy hides both failures.
Separate tool selection from argument correctness
An agent fails in two different places when it uses a tool. It can choose the wrong tool, or it can choose the right one and send the wrong arguments. Those failures point at different remedies: the first is a tool-selection problem, the second is an argument problem [2]. There is a third case worth testing separately — calling a tool that was not needed at all [2].
Keeping the checks separate matters for diagnosis. If a single pass/fail covers both, you cannot tell whether to rewrite tool descriptions, tighten the schema, or change the prompt. Treat selection, argument structure, argument values, and call count as distinct assertions.
Capture the call log as a structured artifact
The assertion target is a list with one entry per call. Each entry should carry the tool name, the arguments exactly as the model produced them, the result as the tool returned it, and whether the call succeeded [1]. Record the arguments before your client normalizes, coerces, or repairs them — repair logic is part of the system under test, and hiding it makes failures unreproducible.
{"turn": 1, "tool": "get_order", "args": {"order_id": "A-10482"}, "result": {"status": "ok", "total": 49.00}, "ok": true}
{"turn": 2, "tool": "refund_order", "args": {"order_id": "A-10482", "amount": 49.00}, "result": {"status": "ok"}, "ok": true}Two properties make this artifact useful. First, it is recorded, not reconstructed: storing the tool results lets a case replay identically instead of depending on live data that drifts [1]. Second, it is the thing you assert against, so a run that guesses its way to a plausible answer fails at the point where the wrong tool was chosen.
A reproducible protocol for local agents
1. Freeze the environment
Local agents vary by model file, quantization, runtime build, context length, and sampling settings. Before comparing anything, pin those values and record them alongside results. When you compare models, keep the test cases, grading rules, model settings, and routing configuration identical for every candidate [2]. A change in quantization or chat template is a change in the system under test, not a detail.
2. Write cases with an expected tool and expected arguments
Each case should state the user request, the tools available, the expected tool (or set of acceptable tools), and the expected argument values where a single correct value exists. Where several choices are legitimately valid — several paths that reach the same result — say so in the case rather than forcing one exact sequence [2].
3. Validate structure first, then values
Use JSON Schema to catch malformed or structurally invalid arguments, and check argument values separately: a schema-valid value can still be wrong [2]. This split is especially useful locally, because grammar-constrained decoding or a strict schema can make arguments syntactically valid while leaving the model free to pick the wrong order id.
4. Compare trajectories where order matters
Trajectory comparison is the right tool when the sequence of calls carries meaning — a lookup that must precede a write, for example. Where multiple paths reach the same valid result, do not require one exact sequence [2]. Assert dependencies (the read happened before the write, the write used the id the read returned) rather than a rigid script.
5. Inject failures on purpose
Recovery is one of the four things worth asserting about a call log [1]. Do not wait for production to supply a timeout, a 404, or a malformed tool result. Inject the failure in the test harness and check whether the agent retries, asks for clarification, or gives up cleanly — and whether it avoids duplicating a side effect.
6. Reserve a judge for what deterministic checks cannot reach
Use deterministic checks when you know the expected tool or argument value. Use a reference-free LLM judge when correctness depends on context or when several choices could be valid [2]. A judge is a fallback for the residual cases, not the primary grader — and a local judge model inherits the same reliability questions as the agent under test.
Fields worth recording per run
| Field | Why it matters |
|---|---|
| Model file and quantization | Two builds of the “same” model can behave differently on tool calls |
| Runtime name and version | Tool-call parsing and chat-template handling live here |
| Context length and sampling settings | Affects both tool selection and argument formatting |
| Prompt and tool schemas as sent | The actual input, not the intended one |
| Full call log with arguments and results | The artifact you assert on and replay from |
| Pass/fail per assertion | Selection, structure, values, order, recovery |
Where local deployment changes the test
Cloud tool-calling is usually tested against a fixed endpoint. Locally, the model and the runtime are both variables you control, which cuts both ways. You can pin a seed and a quantized file for near-identical replay, and you can run the same case set across several quantizations to see whether tool selection degrades before text quality does. You also carry the cost: smaller models frequently produce malformed argument blocks, invent tool names that are not in the schema, or repeat a call that already succeeded. Those are testable signatures rather than mysteries — capture the raw arguments and the raw response, and the pattern usually becomes obvious.
Grammar-constrained decoding and strict structured-output modes are useful defenses against malformed arguments, but they only address structure. They do not fix selection, and they do not fix values, which is why step 3 above separates the two.
Limitations and tradeoffs
Three limits are worth stating plainly. First, deterministic value checks only work when a single correct value exists; open-ended tasks need a judge or a rubric, and both introduce their own variance. Second, exact trajectory matching is brittle — requiring one exact sequence will fail valid runs [2]. Third, a passing call log is not proof of a correct outcome: the tool itself can be wrong. The call log narrows the search space; it does not replace checking the result the tool produced.
There is also a cost tradeoff. Recording every call, replaying with stored results, and injecting failures takes more harness work than grading the final string. The payoff is that failures become diagnosable and reproducible, which is the difference between a flaky demo and an agent you can leave running.
Next step
Pick one agent you already run locally, capture its call log for a handful of real requests, and write four assertions against it: correct tool, valid arguments, correct values, and correct dependency order. Then add one injected failure per case. That small harness will surface more than any amount of prompt tweaking.
If you try this on a specific local runtime, share your model file, quantization, runtime version, and the call log you captured — including the failures — in the community so others can compare.
FAQ
Can I test tool calling by grading only the final answer?
No. Answer-only grading passes runs where the agent guessed or wrote to the wrong record and then described that write accurately, because the model writes its summary after the calls and can describe them however it likes. The call log holds facts the final message does not.
Should tool selection and argument correctness be tested together?
They should be tested separately. Choosing the wrong tool is a selection failure; choosing the right tool with the wrong arguments is an argument failure. Keeping them apart tells you whether to fix tool descriptions, the schema, or the prompt.
When is a JSON Schema check enough for tool arguments?
A schema check only confirms structure. A schema-valid value can still be wrong, so argument values need a separate check whenever a single correct value exists.
Do I need an LLM judge for every test case?
No. Use deterministic checks when the expected tool or argument value is known, and reserve a reference-free judge for cases where correctness depends on context or several choices could be valid.