Prompt Engineering: A Practical Guide to Instructions, Examples, and Evaluation

Prompt engineering is the practice of writing, testing, and revising the instructions you give a language model so that its output reliably serves a defined purpose. It is not a collection of magic phrases. The version of the discipline that survives contact with real work has three parts: a clear instruction, well-chosen context or examples, and a measurement loop that tells you whether a change helped or hurt. Course material for the University of Maryland’s Fall 2026 CMS 389A frames the prompt as the program and the science half of the work as measurement [1].

This guide is written for readers who can already send a request to a hosted chat model or a local runtime and now want results they can depend on. By the end you should be able to structure a prompt into reusable parts, choose among the core techniques instead of stacking all of them, build a small evaluation set and run a comparison, recognise the failures that better phrasing cannot fix, and judge when the correct next step is retrieval, tooling, or a different model rather than a longer prompt.

Scope. This article covers instruction design for text-in, text-out generation with chat-style models, whether they run behind a vendor API or on your own hardware. It does not cover fine-tuning, training-data curation, or the internals of agent frameworks. Those are separate decisions that usually come after a prompt has been measured.

A prompt is a specification, not a wish

When people say a model “ignored” them, they usually mean one of three things failed. The model did not have the capability for the task. The model did not have the information in front of it. Or the instruction did not specify what a correct answer looks like. Separating those three possibilities is the single most useful habit in this field, because each has a different fix and only one of them is solved by editing text.

The prompt is the cheapest of the available levers. Changing weights — fine-tuning — requires data, compute, and a maintenance plan. Changing what the model can see — retrieval, or RAG (retrieval-augmented generation, where relevant documents are fetched and inserted into the context before generation) — requires an index and a retrieval pipeline. Changing the instruction costs a text edit. That is why prompt work comes first: you want to know how far a well-specified instruction gets you before you buy infrastructure.

The specification analogy is worth taking seriously. A good specification names the input, the required output, the acceptance criteria, and the behaviour when the input is ambiguous. A prompt can do all four:

• Input: what text or data will be supplied, and in what form.
• Output: the shape of the answer — prose, a list, a JSON object with named fields.
• Acceptance criteria: what makes an answer correct, complete, or acceptable.
• Ambiguity handling: what to do when the input does not contain enough information.

Most weak prompts specify only the first, and often only loosely.

The five slots of a reusable prompt

Almost every production prompt, regardless of vendor or runtime, can be decomposed into the same slots. Thinking in slots rather than in paragraphs makes prompts easier to test, because you can change one slot and hold the rest constant.

The University of Maryland course lists a comparable set of techniques in sequence — be clear and direct, assign a role, separate data from instructions, control the output format, let the model think step by step, give examples (few-shot prompting), and avoid hallucinations — and then combines them into a single complex prompt template [1]. The Quick Guide to Prompt Engineering follows a similar practical arc, moving from crafting a first prompt through testing, evaluating, and iterating on it [2].

The seven techniques that do most of the work

Each technique below buys something and costs something. The mistake beginners make is applying all seven to every task, which produces a long prompt whose parts contradict each other. Read each one as a tool with a trigger condition.

1. Be clear and direct

State the task as an instruction, not a topic. “Summarise this support ticket in one sentence for a queue dashboard” is a task. “Support ticket” is a topic. Directness also means removing politeness padding that carries no information, and resolving pronouns that could point at more than one thing.

The test is whether a competent colleague with no context could follow the instruction. If they would ask “for whom?” or “how long?”, the prompt is under-specified. This technique is free and should always be on.

2. Assign a role

A role sets vocabulary, assumed knowledge, and default priorities. “You are a database administrator reviewing a schema change” produces different framing from “You are a technical writer explaining the change to new developers.”

Roles are most useful when the audience or the standard of judgement is ambiguous. They are least useful as decoration: stacking three personas (“a world-class expert senior engineer with 20 years of experience”) adds tokens and rarely changes behaviour. If a role does not change what the model treats as important, delete it.

3. Separate data from instructions

When a prompt mixes your instructions with user-supplied text, the model has to infer where one ends and the other begins. Delimiters remove that ambiguity. Common choices are XML-style tags, triple backticks, or a labelled section:

Summarise the ticket below in one sentence.

<ticket>
{{ticket_text}}
</ticket>

Separation does two things. It improves reliability on long inputs, and it establishes a boundary you can defend against prompt injection — the case where text inside the data section contains something like “ignore the previous instructions”. Delimiters do not prevent injection on their own, but they make the boundary explicit and testable. The UMD course treats separating data from instructions as a technique in its own right [1], and it is the one most often skipped by beginners pasting text directly into a chat box.

4. Control the output format

If anything downstream reads the answer — a script, a spreadsheet, a WordPress field — the output needs a contract. Name the fields, the allowed values, and what to do when a value is unknown. For structured output, many modern APIs support a schema or a JSON mode; where they do not, you specify the shape in the prompt and validate the result yourself.

Two practical notes. First, always validate: a model that returns JSON-shaped text can still return a missing field or an unexpected enum value. Second, prefer a small number of fields. A twelve-field schema invites the model to invent values for fields the input does not support.

5. Let the model think step by step

For tasks with multiple steps — arithmetic, multi-constraint filtering, comparison against a rubric — asking the model to work through the reasoning before giving the final answer improves accuracy on many task types. The tradeoff is verbosity and cost: reasoning tokens are billed and occupy context.

The practical pattern is to separate the reasoning from the deliverable. Ask for reasoning, then ask for the answer in the required format, and discard the reasoning before it reaches a human or a parser. On some model families the reasoning is produced internally and only the final answer is returned; the instruction still helps, but you will not see the working.

6. Give examples (few-shot prompting)

Few-shot prompting means including a small number of worked input/output pairs in the prompt. It is the most reliable way to communicate a format or a judgement call that is hard to describe in words — for instance, how aggressive a classifier should be on borderline cases.

Examples have real costs. They consume context, they bias the model toward the patterns they contain, and badly chosen examples teach the wrong lesson. Two to five examples is a common starting range; more is not automatically better. Choose examples that cover the hard cases rather than the easy ones, and keep them consistent with the output contract. If your examples disagree with your instructions, the examples usually win.

7. Avoid hallucinations

Hallucination is fluent output that is not supported by the input or by reality. Prompt-level mitigations reduce it but do not eliminate it: ground the answer in supplied context, instruct the model to say when the context is insufficient, require citations to specific passages, and forbid answering from general knowledge when the task is document-bound.

The strongest mitigation is structural rather than verbal. If the model must answer from a document set, retrieve the documents and place them in the context; if the answer must be checkable, require the model to quote the supporting sentence. The UMD course lists avoiding hallucinations as its own technique [1], and the honest framing is that prompting lowers the rate while retrieval and verification lower it further.

Parameters that belong in your notes

Technique alone does not determine behaviour. Sampling temperature, top-p, maximum output length, and where the system instruction is placed all change results, and none of them are visible in the prompt text itself. Before comparing two prompts, fix these settings and write them down. A comparison between prompt A at one temperature and prompt B at another measures the settings, not the prompts.

How much can prompting actually fix?

Prompting is very effective at specification problems: wrong shape, wrong length, wrong audience, inconsistent handling of edge cases, missing abstention behaviour. It is moderately effective at grounding problems, especially when combined with retrieval. It is weak against genuine capability gaps — a model that cannot reliably do multi-step arithmetic will not become reliable because you asked it to be careful. Recognising which category your failure falls into saves weeks of rewording.

A worked example: triaging support tickets

The example below is illustrative, not a report of a tested deployment. It shows how the slots fit together and where a human check belongs.

Goal: classify incoming support tickets into a category, assign a priority, and produce a one-line summary for a queue dashboard.

Input: the raw ticket text, which may be short, angry, or missing information.

Output: a JSON object with three fields and fixed allowed values.

SYSTEM = '''You are a support triage assistant for a software product.
Return JSON only, matching this schema:
{"category": "billing" | "bug" | "how-to" | "other",
 "priority": "low" | "normal" | "high",
 "summary": "one sentence, under 25 words"}
Rules:
- Use only the ticket text. Do not infer account details.
- If the ticket lacks enough information to choose a category,
  use "other" and state what is missing in the summary.
- Priority is "high" only when the ticket describes data loss,
  a security concern, or a complete outage.'''

def build_messages(ticket_text):
    return [
        {'role': 'system', 'content': SYSTEM},
        {'role': 'user', 'content': '<ticket>n' + ticket_text + 'n</ticket>'},
    ]

Note that the placement of the system instruction depends on the API you use. Some accept a system role inside the message list; others take a separate top-level parameter. The important part is the separation of the fixed part (role, schema, rules) from the variable part (the ticket), which the UMD course highlights as a template practice [1].

Component responsibilities. The system frame carries the rules that must not vary between tickets. The user message carries only data. The schema is the contract that the downstream queue reads. The ambiguity rule handles the tickets that would otherwise produce confident nonsense.

Human review point. The dashboard is a routing aid, not an authority. A reasonable operating rule is that “high” priority tickets and any ticket categorised as “other” get a human glance before routing, because those are the two states where a wrong call has the largest cost.

Your first evaluated prompt, in one sitting

Prompt work without measurement is guesswork with extra steps. The following path takes an afternoon and produces something you can defend.

1. Pick one task and one model. Narrow enough that you can describe a correct answer. Record the model name and version, the runtime or API, and the parameter settings you use. A prompt that works on one model at one temperature is a data point, not a result.
2. Write twenty test inputs. Include easy cases, at least three genuinely ambiguous cases, and at least two that the model should refuse or flag. Twenty is enough to see large differences and small enough to grade by hand.
3. Write down your acceptance criteria before you look at outputs. “Correct category” and “valid JSON with no invented account details” are criteria. “Looks good” is not. Writing these first prevents you from rationalising whatever the model produces.
4. Write the first prompt using the five slots. Start with the instruction and the output contract. Add examples only if the format or the judgement is genuinely hard to describe.
5. Run every test case at least twice. Generation is stochastic. A prompt that passes twenty cases once may fail several of them on a second pass. If your runtime exposes a temperature or sampling control, set it low for evaluation runs and note the value.
6. Grade against the criteria and record failures verbatim. Keep the exact input, the exact output, and your verdict. Paraphrasing failures destroys the evidence you need for the next iteration.
7. Change one thing, then re-run the whole set. This is the step that turns tinkering into engineering. Changing one slot at a time is what lets you attribute an improvement to a cause.

Observable success check: you can state a pass rate for a named model, version, and settings, on a fixed set of inputs, and you can show which prompt change moved it. If you cannot state that sentence, you have not finished the loop.

Evaluation: the half of the job people skip

Evaluation is how you decide whether a prompt is good enough to ship and whether a revision made things better. The UMD course devotes a substantial block to it — human grading, graded evaluations by hand, model-graded evaluations, and tooling such as promptfoo [1] — which reflects how much of the real work sits here rather than in phrasing.

What to measure

Choose criteria that map to the cost of being wrong. For a classification task, accuracy on the fixed test set is the obvious one, but also track the specific error types that matter: false “high” priorities, invented account details, malformed output. For a writing task, criteria are usually a short rubric — covers the required points, stays within length, does not assert unsupported facts — graded on a small scale.

A useful rubric has three properties: each criterion is observable in the output, each is scored on a small number of levels, and a second person could apply it and reach roughly the same verdict. If a criterion requires reading the model’s mind, it is not a criterion.

Methods and their tradeoffs

Model-graded evaluation — often called LLM-as-judge — deserves a caveat. It is useful for triage and for catching regressions across hundreds of cases, but it is a measurement instrument with its own error rate. Calibrate it: grade a sample by hand, compare the judge’s verdicts with yours, and only trust the judge on the criteria where it agrees. Judges also tend to favour longer, more confident answers, so include an explicit rubric and, where possible, ask the judge to justify its verdict.

In practice, most teams end up combining methods: rule-based checks on every case because they are free, model grading on the full set for a coarse signal, and hand grading on a rotating sample to keep the judge honest.

Recording results so they mean something

A result is only useful if someone else could reproduce it. Record the prompt version, the model and version, the parameter settings, the test set, the grading criteria, the date, and the raw outputs. Cost estimates should follow the same discipline: token counts multiplied by cases multiplied by runs, using prices you have verified on the date you checked them. Do not carry a cost figure forward without re-checking it, because prompt length, model choice, and pricing all change.

The iteration lifecycle

Prompt work is maintenance, not a one-time build. A workable lifecycle has five stages.

1. Draft against the five slots, with the output contract written before the prose.
2. Test on the fixed set, running each case more than once.
3. Diagnose failures by category — format errors, missing information, wrong judgement, hallucination — because each points at a different slot.
4. Revise one slot and re-run the full set to catch regressions.
5. Version and record the prompt as an artefact with a changelog entry explaining why it changed.

Two disciplines keep this honest. First, never tune against the same cases you use to report success; hold back a few cases you only run at the end. Second, treat the test set as a living document. When a real failure occurs in production, add it as a case, fix the prompt, and confirm the fix does not break the earlier cases. That is how a prompt set becomes institutional knowledge rather than a folder of drafts.

Version control matters more than it sounds. Prompts embedded in application code are hard to review and easy to change accidentally. Keeping the prompt text in its own file, with a version identifier that appears in your logs, means a bad batch can be traced to the exact wording that produced it.

Prompt engineering on local models

Running models on your own hardware changes the practice in specific ways, and it is worth being concrete about them rather than assuming cloud habits transfer.

Smaller models need more explicit instructions. A model with fewer parameters has less capacity to infer unstated conventions. Where a large hosted model may correctly guess that “summarise” means “one sentence”, a smaller local model often needs the length, the audience, and the format spelled out. The five slots matter more, not less.

Few-shot examples can help more, but cost more. Examples are an efficient way to teach a format to a smaller model. On a local machine, however, every example occupies context, and context consumes memory. This is where the KV cache matters: the memory used to hold the conversation state grows with the length of the prompt and the generated output, on top of the memory used by the model weights themselves. A prompt that fits comfortably on a large GPU may push a smaller one into offloading and a sharp slowdown. Measure the actual prompt length rather than assuming it fits.

Quantisation can change instruction-following. Compressed weights trade fidelity for memory and speed. The same prompt may behave differently at different quantisation levels, which is a reason to record the quantisation setting alongside the model name when you report results. Treat it as part of the environment, not a detail.

Reproducibility is harder and more important. Local runtimes vary in how they handle sampling, seeds, and context truncation. Run each test case multiple times, keep the settings fixed during a comparison, and note the runtime and version. If your runtime exposes a seed and a temperature, record both.

Evaluation stays local. If data cannot leave your machine, model-graded evaluation must also run locally, which usually means a second model acting as judge. That is workable, but the judge’s reliability should be calibrated against hand grading on the same cases, exactly as with a hosted judge. A small local judge model may be adequate for format checks and unreliable for subtle quality judgements, so scope its role accordingly.

A practical local loop looks like this: keep the test inputs in a plain text or JSON file, keep the prompt in a separate file with a version string, write a short script that sends each case through the runtime and writes one output file per run, then grade the outputs with a checklist you can apply by hand. Nothing about this requires a cloud service, and the resulting record is portable if you later change runtimes.

Prompts inside WordPress and automation workflows

When a prompt is wired into a publishing or automation pipeline, the prompt stops being a text file and becomes a component with an interface. Four practices make the difference between a demo and something you can leave running.

• Define the input contract. Specify exactly what the workflow passes in — post body, title, taxonomy terms, retrieved context — and what happens when a field is empty. A prompt that assumes a title exists will produce odd output on drafts.
• Validate the output before it is used. Parse the response against the expected shape. On a validation failure, retry once with the error appended, then route to a human queue rather than publishing something malformed.
• Make the operation idempotent. If a job is retried after a timeout, you do not want two drafts or two published posts. Key each generation to a stable identifier and check whether output already exists for that key.
• Log prompts and outputs, with a retention policy. Logging is what lets you diagnose a bad batch later. It also means you are storing whatever was in the prompt, so decide how long that lives and who can read it before you turn logging on.

Keep a human review gate for anything that publishes under your name. A model can produce a fluent paragraph that is wrong in a way only a subject-matter reader will notice, and the cost of that error lands on your site, not on the model. A workable default is that the model drafts and a person approves, with automation handling formatting, taxonomy, and scheduling rather than editorial judgement.

Credentials deserve their own note. API keys for a hosted model should live in the application configuration, not inside prompt text, and should be rotatable without editing the prompt. If a prompt is stored in a database or a plugin settings field, treat it as configuration that may be readable by anyone with administrative access.

Security, privacy, and the limits of instruction

Prompt injection is the failure mode where untrusted text in the context contains instructions that the model follows. If your workflow summarises user-submitted content or fetched web pages, assume that content may contain adversarial instructions. Delimiters help, but the durable mitigations are architectural: do not give the model credentials or tools it does not need for the task, validate outputs before acting on them, and never let a model-triggered action bypass a human gate on anything consequential.

Data handling. Prompts are data. If you paste customer records, internal documents, or credentials into a hosted model, you are sending them to a third party under that provider’s terms. For local models, the data stays on your hardware, which is often the reason people choose them — but logging and backups can reintroduce the exposure. Never place secrets in a prompt; pass credentials through the application layer where they can be rotated and audited.

Over-trust. A prompt that produces good output most of the time is easy to stop checking. The Quick Guide to Prompt Engineering includes ethical considerations and bias mitigation as part of the discipline [2], and the practical version of that is simple: keep the evaluation set running after launch, sample production outputs periodically, and treat a rising failure rate as a signal to re-test rather than as noise.

Access and cost. Two operational constraints are easy to forget. Rate limits and quotas determine whether an evaluation sweep finishes before a deadline, and per-token pricing determines whether model-graded evaluation on thousands of cases is affordable. Estimate both before committing to a method, and note the date of any price you rely on.

Troubleshooting: symptom, cause, check, fix

Two of these deserve emphasis. “Works in chat, fails in the pipeline” is almost always a configuration difference rather than a prompt defect, and the fix is to log the exact prompt the application sends. “Quality drops on long inputs” is often a context problem that no rewording solves; the answer may be retrieval that selects less but better material.

Where to go next

The immediate next action is small and specific: choose one task you repeat weekly, write twenty test inputs with acceptance criteria, and run the first prompt against them twice. That single afternoon produces more useful knowledge than reading a dozen technique lists, because it tells you which failures are yours and which belong to the model.

From there, the progression is predictable. Once a prompt passes your set, move to retrieval if the failures are about missing information. Move to tools or structured APIs if the failures are about taking actions. Move to a larger or differently trained model if the failures are about reasoning quality that survives every prompt revision. Move to fine-tuning only when you have a large, stable, labelled dataset and a prompt that already performs well — because fine-tuning a badly specified task just makes the bad specification faster.

If you are working on your own hardware, the next specific question is usually how to run an evaluation loop without a cloud judge, and how much prompt length your memory budget can actually absorb. If you are working inside WordPress, it is usually how to validate model output before it reaches a published field, and how to keep a review gate without slowing the workflow to a crawl. Both are narrow enough to be answered by a focused tutorial rather than another pillar guide.

If you want to compare notes, bring your goal, your environment, the prompt version you are testing, and the results you recorded to the community discussion. Concrete setups and raw outputs are far more useful to other readers than a verdict about which model is best.