There is no single correct number of few-shot examples, and the evidence does not support “more is always better.” A large 2025 empirical study of LLM code translation found that functional correctness consistently peaked with 5–25 examples, while scaling to hundreds of examples often degraded it — a pattern the authors call the “many-shot paradox.” For most practitioners, the practical answer is to start small, measure on your own task, and treat the number of examples as a parameter to tune rather than a constant to memorize.
This article is for technically curious beginners and intermediate practitioners who already understand what a prompt is and want a defensible way to decide how many examples to include. It covers what few-shot prompting is, what current evidence says about scaling example counts, and how to run a small reproducible experiment to find your own number.
What few-shot prompting actually is
Few-shot prompting means placing a small number of worked examples in the prompt before the actual request. The model is not retrained or fine-tuned; it conditions on the examples inside its context window. This behavior is usually called in-context learning.
The terminology scales with the count:
- Zero-shot — instructions only, no examples.
- One-shot — a single example.
- Few-shot — typically two to a few dozen examples.
- Many-shot — dozens to hundreds of examples, made feasible by large context windows.
These boundaries are informal. What matters is that examples consume context, cost tokens, and change model behavior. They are not free.
Why more examples seem to help
Examples carry information that instructions alone often fail to convey: the exact output format, the label set, how to handle ambiguous inputs, and the style or level of detail expected. Adding examples can also shift the model toward the distribution of your task. Early demonstrations that large language models could perform new tasks from a handful of in-context examples helped popularize few-shot prompting as a default technique.
The intuition that follows is that more examples should keep helping until the model has “seen everything.” That intuition has a limit. At some point an additional example adds little new information, and past that point it can actively hurt.
The many-shot paradox
The strongest recent evidence against “more is better” comes from a large-scale study of LLM code translation. The authors evaluated over 90,000 translations, scaling in-context examples from zero-shot up to 625 examples, with prompts ranging from roughly 100,000 to 800,000 tokens.
Their results separate two kinds of quality:
- Static similarity metrics — measures of surface similarity to a reference — improved modestly as more examples were added.
- Functional correctness — whether the translated code actually works — consistently peaked with few-shot prompting of about 5–25 examples.
Beyond that range, providing substantially more examples often degraded functional performance. The authors describe this as a “many-shot paradox” and conclude that for code translation, “the quality of a few well-chosen examples outweighs sheer quantity,” underscoring the task-dependent nature of optimal prompting strategies (Oskooei et al., 2025).
This is a useful caution for anyone who assumes a larger context window automatically means better results. A model can attend to a very long prompt and still produce worse output.
What a controlled prompt experiment looks like
The Libretto prompt-engineering series approaches the same question empirically, using the Emoji Movie dataset from the Big Bench LLM benchmark — 100 questions that ask a model to identify a movie from a string of emojis. In earlier experiments the team held the number of few-shot examples fixed at three and varied which examples were used. The follow-up question is how performance changes as the number of examples changes (Libretto, “How many few shot examples should you use?”).
That structure — fix everything except the variable you care about — is the core of a useful experiment. You can replicate the logic on your own task without a benchmark suite.
A minimal protocol
- Build a held-out evaluation set. Collect inputs your prompt will actually see, with known correct outputs. Even 30–50 items gives a rough signal; more is better.
- Freeze everything else. Same model and version, same temperature and sampling settings, same instruction text, same output format. Change only the number of examples.
- Choose a task-relevant metric. For code, that means running the code. For classification, accuracy or F1. For extraction, exact-match or field-level accuracy. Avoid judging only by surface similarity, since the code-translation study shows similarity and correctness can diverge.
- Sweep the count. Test zero-shot, one-shot, and then several few-shot sizes — for example 2, 3, 5, 10, 25, and 50 — if your context and budget allow.
- Record cost and latency alongside quality. Note prompt tokens, response time, and any per-call price. A small accuracy gain from 50 examples may not justify the added token cost.
- Re-check with different example selections. Prompt performance is sensitive to which examples you pick, so a single set can mislead. Run at least two selection strategies at your best-performing count.
Report the result as a curve, not a single number. The useful output is the point where quality stops improving and cost keeps rising.
Practical guidance for choosing N
Until you have your own measurements, these starting points are reasonable:
- Start with 3–5 examples. This is a common default and matches the lower end of the range where code-translation correctness peaked.
- For simple formatting or classification tasks, fewer may be enough. One or two well-chosen examples often fix output structure.
- For complex reasoning, code, or multi-step transformations, test 5–25. This is the range the code-translation study found optimal for functional correctness.
- Treat hundreds of examples as an experiment, not a default. If you try many-shot prompting, verify with a functional or task-level metric rather than assuming improvement.
- Consider selecting examples dynamically. Retrieving the most similar examples per query can outperform a fixed set, though it adds infrastructure and must be measured like any other change.
Local AI, cost, and context considerations
For anyone running models locally or building automation workflows, the number of examples has direct operational consequences. Every example is prompt tokens that must be processed before the model generates anything. That increases prefill time and memory use, and it consumes part of the context window you may need for the actual input.
Context length is not the only constraint. On local hardware, larger prompts increase KV-cache memory and can slow generation, and the model’s usable context may be smaller than its advertised maximum. When estimating memory, account for the KV cache and runtime overhead rather than looking only at model weight size.
For WordPress and automation use cases — generating structured content, classifying support messages, or transforming data between systems — the tradeoff is usually straightforward: pick the smallest number of examples that reliably produces correct, well-formed output, then spend the remaining context on the actual task. A prompt that costs three times as many tokens for a marginal accuracy gain is rarely worth deploying at scale.
Limitations and open questions
Two caveats matter before you generalize from any of this.
First, the strongest quantitative finding here comes from a single task family — code translation. The authors themselves note that optimal prompting strategies are task-dependent. Classification, summarization, and extraction may have different curves.
Second, the Libretto work is framed as an investigation into how the number of examples changes performance; the setup is described, but this article cannot state specific results beyond what the source reports. Treat it as a template for your own testing rather than a source of a universal number.
There is also no guarantee that the best count for one model transfers to another. Example selection and model choice interact, and a configuration that works on a hosted frontier model may behave differently on a smaller local model with a shorter effective context.
What to do next
Pick one task you actually care about, build a small labeled evaluation set, and run a count sweep with everything else frozen. Measure a task-level metric, not just similarity. Record tokens and latency. The number you keep should be the smallest one that meets your quality bar — and you should be able to show the curve that justifies it.
If you run such an experiment, sharing your task, model, example counts, and results in the community is more useful than reporting a single “best” number, because the value of these findings comes from comparison across setups.
FAQ
Does adding more few-shot examples always improve results?
No. In a large 2025 code-translation study, functional correctness peaked at about 5–25 examples and often declined when hundreds of examples were used, even though static similarity metrics improved modestly.
What is a good number of few-shot examples to start with?
A practical default is 3–5 examples, then test larger counts on your own task. Simple formatting or classification tasks may need fewer; complex reasoning or code tasks may benefit from testing 5–25.
Why can many-shot prompting hurt performance?
The code-translation study found that surface similarity and functional correctness can diverge: more examples may make output look closer to a reference while making it less likely to work. Very long prompts can also dilute attention and add cost without adding useful information.