Hallucination is a prompt design problem before it is a model problem
When a language model writes "military-grade aluminium alloy" about a product you described only as "aluminium", it has not malfunctioned. It has done exactly what it was asked: produce text that resembles high-converting product copy. High-converting product copy in its training data contains phrases like "military-grade". The model has no way to distinguish the parts of that pattern that were grounded in fact from the parts that were decoration.
The fix is structural. You have to make accuracy the primary objective and persuasion the secondary one, in that order, explicitly.
Separate facts from instructions
The single highest-impact change is to put the product facts in their own clearly delimited block, and to state that the block is the complete set of permissible factual content.
VERIFIED FACTS (the complete set — do not add to it):
- 1.7 litre capacity
- 2200W concealed heating element
- 304 stainless steel body
- Boil-dry protection, auto shut-off
- 1.2m cord, 2-year manufacturer warrantyThen, separately:
RULES:
- Do not state any specification, material, certification, award,
rating, review count or performance number that is not in
VERIFIED FACTS.
- If a field would require information you do not have, write the
field using only what is available. Do not fill gaps with plausible
detail.That second rule matters more than it looks. Models are strongly biased toward completing a requested structure. If you ask for five bullets and supply four facts, you will get a fifth bullet, and it will be invented. Giving the model explicit permission to produce a shorter or more general answer removes the pressure that causes fabrication.
Ask for structured output, not prose
Free-form prose gives the model room to add connective tissue, and connective tissue is where invented claims hide. A JSON schema with named fields — title, bullets, description, keywords — narrows the space considerably. It also makes the output machine-checkable, which enables the next step.
Verify after generation, not just before
Prompting reduces fabrication. It does not eliminate it. Any serious pipeline needs a verification pass that runs on the output:
- Forbidden-term scan. A regex pass for the terms that are never acceptable in your category — FDA approved, cures, #1, guaranteed, competitor brand names, and any words on the brand's own banned list.
- Numeric grounding. Extract every number from the output and check that it appears in the input facts. This catches the single most damaging class of hallucination: a specification that is nearly right.
- Repair, don't discard. When the scan finds a violation, feed the violation list back to the model with an instruction to rewrite only the affected passages. A repair pass is cheaper than a regeneration and preserves the parts that were already good.
Temperature is not the lever people think it is
Lowering temperature makes output more predictable, not more truthful. A model at temperature 0 will invent the same specification every time. Grounding is a function of what is in the context window, not of sampling randomness. Set temperature for tone consistency and solve accuracy elsewhere.
Measure it
If you generate at volume, sample twenty outputs a week and check every factual assertion against the source data. Track the rate. Any prompt change that does not move that number is a change in taste, not in quality — which is fine, as long as you know which one you are making.
Written by the CopyForge AI team. Drafted with AI assistance, then reviewed and fact-checked by a human before publishing.
