How to Reproduce a Local Ollama Inference Run Exactly

Reproducing a local Ollama inference run exactly means pinning five things at once: the model artifact, the runtime version, the sampling parameters, the prompt and context, and the execution environment. A fixed seed combined with temperature: 0 is the usual starting point, but the Ollama issue tracker shows that this configuration alone has not always produced deterministic output — it was fixed once and reported broken again later. This guide explains what to pin, how to verify it, and where the limits of reproducibility actually lie.

What “exact reproduction” means

Before pinning anything, decide which level of sameness you need, because each level has a different cost.

• Same text output. The response string is byte-for-byte identical across repeated runs. This is what most people mean by reproducibility, and it is achievable in many local setups.
• Same token sequence. The model selects the same tokens in the same order. If the visible text matches but the stop sequence or truncation point differs, this is where the difference surfaces.
• Same timing and resource use. Identical latency and memory figures across runs. This is not a realistic target, because thread scheduling and batching vary between runs.

Cross-hardware reproduction — running the same request on a different GPU or CPU and expecting identical output — is a separate and considerably harder goal. The practical target for most users is: same machine, same configuration, repeated runs.

The five layers to pin

1. The model artifact, not just the tag

A tag such as llama3.1:8b is a movable pointer. The same tag can resolve to a different digest after an upstream update, and a re-quantized build will not reproduce the original output. Record the digest and quantization reported by:

ollama list
ollama show llama3.1:8b

Store that digest alongside your results. If the digest changes, your reproduction attempt is comparing two different artifacts.

2. The runtime version

Sampling code, GPU kernels, and request handling change between Ollama releases. The issue history for deterministic output is itself evidence of this: the problem was reported in September 2023, fixed in pull request #663, and then reported again in May 2024. A configuration that is reproducible today is reproducible for a specific build. Log the output of:

ollama --version

3. Sampling parameters — and where they go

This is the most common practical failure. In the Ollama HTTP API, sampling parameters must be nested inside an options object. A community comment on issue #586 documents the pattern explicitly: the parameters for deterministic, reproducible output need to be passed in with an "options" key in the JSON input. Parameters placed at the top level of the request body may simply be ignored.

A related issue, #5274, was opened about the API accepting non-functional parameters without returning an error message — which means a malformed request can look successful while silently running with defaults.

For greedy decoding, the configuration used by the public test-non-determinism toolkit is temperature: 0.0 with top_k=1. Beyond those two, pin everything else that can influence the token stream: seed, top_p, repeat_penalty, presence and frequency penalties, num_ctx, num_predict, and any stop sequences. The same comment thread notes that a fixed seed is described as not strictly needed when temperature is zero, but keeping it costs nothing and protects you if a parameter is dropped or overridden.

4. The prompt and the template

An invisible difference in whitespace, a trailing newline, or a changed system prompt changes the token stream. So does the endpoint: /api/generate and /api/chat apply different templating, and a chat template change in a newer model build will alter the prompt the model actually sees. Store the exact prompt text as a file rather than retyping it, and keep the same endpoint across the whole comparison.

5. The execution environment

The README of the test-non-determinism project names three sources of variance: sampling randomness, GPU thread scheduling, and batched requests across shared infrastructure. Its collection script therefore pins every relevant sampling parameter, sends a warmup request so the model is fully loaded before measurement begins, and issues requests one at a time. Reproduce that discipline: same machine, same backend, no concurrent requests, and a warmup call before the recorded runs.

A minimal reproducible request

The following request pins the model, the prompt, and the sampling options. It is an illustrative example of the request shape, not a recorded result:

curl -s http://localhost:11434/api/generate -d '{
  "model": "llama3.1:8b",
  "prompt": "List the first five prime numbers, one per line.",
  "stream": false,
  "options": {
    "seed": 42,
    "temperature": 0,
    "top_k": 1,
    "num_ctx": 2048,
    "num_predict": 64
  }
}'

Note that stream is set to false so the response arrives as a single JSON object rather than a stream of partial chunks. Streaming does not change the model’s output, but it makes automated comparison more work.

A verification protocol you can run

Reproducibility is a claim you should test, not assume. The test-non-determinism toolkit provides a concrete pattern: a collection script sends the same prompt to a locally running Ollama instance N times, saving each response to its own folder, with a warmup request first. Its documented example invocation uses llama3.1:8b, five runs, temperature 0.0, top_k=1, seed 42, and a 600-second timeout, with each run taking roughly 12 seconds in that environment.

A lighter-weight version of the same idea:

1. Send one warmup request and discard the response.
2. Send the identical request five times, saving each response body to a separate file.
3. Extract the response text and hash it: sha256sum run1.txt run2.txt ...
4. Compare the hashes. Identical hashes mean identical text; differing hashes mean you have a variable to eliminate.

The published README excerpt for that toolkit describes the collection method but does not include the analysis outcome, so treat the protocol above as a method to apply to your own environment rather than as a confirmed result. Run the comparison on your own hardware before relying on determinism for anything important.

When outputs still differ: ordered diagnosis

Work from the cheapest checks to the most invasive ones, and stop as soon as the hashes match.

1. Inspect the raw request body. Confirm every sampling parameter sits inside options. This is the single most common cause of “temperature 0 does nothing.”
2. Re-check the model digest. If ollama show reports a different digest than when you recorded the baseline, the artifact changed.
3. Eliminate concurrency. Ensure no other client is hitting the same Ollama instance during the run, and that your own requests are sequential.
4. Diff the prompts byte-for-byte. Compare files, not memory. Include the system prompt and the endpoint choice.
5. Confirm the runtime version and backend. A version bump or a switch between CPU and GPU execution invalidates the comparison.
6. Check context and truncation. A different num_ctx changes the KV cache and can change numerical results; a different num_predict or stop list changes where generation ends.
7. Accept residual variance. Floating-point arithmetic is not associative, and GPU kernels may reduce sums in a different order depending on launch configuration. If all of the above are fixed and output still drifts, that drift is a property of the stack, not of your request.

Limits: what you can and cannot promise

The history in issue #586 is the clearest caution available: determinism with a fixed seed and temperature zero was reported, fixed in pull request #663 and closed on 2023-10-02, then reported again on 2024-05-23. Determinism is therefore a property of a specific model digest, a specific Ollama build, and a specific configuration — not a permanent guarantee of the tool.

Two further limits are worth stating plainly. First, identical output on one machine does not imply identical output on another; different hardware may produce different numerical results even under greedy decoding. Second, reproducibility of text is not reproducibility of performance: latency and memory figures will vary run to run, so do not build benchmarks on the assumption that they will not.

Make it a habit: keep a run manifest

The most useful artifact you can produce is a short manifest stored next to your results. It should contain the model tag and digest, the Ollama version, the operating system and hardware, the full request body, the prompt file hash, and the output hashes. With that manifest, a failed reproduction becomes a diff between two known states instead of a guessing game.

If you run into a case where a fully pinned configuration still drifts, that is worth documenting in the community: share your model digest, Ollama version, hardware, the exact request body, and the differing outputs. Those details are what make a report actionable rather than anecdotal.