The Art of Better Prompting: Reduce AI Hallucinations

When the Model Sounds Right and Is Wrong

A test engineer asks an AI assistant to generate a test plan for a payment service. The assistant produces a detailed, thorough document. It references a specific API endpoint, a rate limit, and an error code. All three are plausible. None of them match the actual system.

This is the hallucination problem as it shows up in practice: not gibberish, but confident specifics that do not exist. The output sounds authoritative because it was trained on authoritative text. The prompt did nothing to push back against that pattern.

That is the part you can change.

What Research Shows About Prompting and Accuracy

The connection between prompt structure and output accuracy is not anecdotal. Researchers at Google Brain published the foundational chain of thought study in 2022, demonstrating that asking a model to show intermediate reasoning steps before giving a final answer substantially improves performance on complex tasks. A 540B parameter model prompted with eight reasoning examples reached leading accuracy on GSM8K, a mathematical word problem benchmark, outperforming models trained directly on that specific task (Wei et al., arXiv:2201.11903).

A subsequent study from the same team introduced a consistency sampling technique that samples multiple reasoning paths from the same prompt and picks the answer that appears most frequently across them. On the GSM8K benchmark, this approach improved accuracy by 17.9 percentage points over standard chain of thought prompting. On AQuA, the improvement was 12.2 points. On SVAMP, 11.0 points (Wang et al., arXiv:2203.11171).

The gains come from the same mechanism in both cases: forcing the model to commit to intermediate steps before reaching a conclusion. A model that must show its work produces outputs that can be checked and that, in the process of generation, are constrained by their own logic.

More recent research confirms the direction while adding nuance. A January 2026 study tested a structured reasoning approach called KDCM on tasks requiring factual knowledge: WebQSP, CWQ, and GSM8K among them. The method embeds a code module into the chain of thought prompt, guiding knowledge graph exploration as part of the reasoning process. HIT@1 accuracy reached 98.4 percent on average across five benchmarks, compared to 90.2 percent for RAG and 91.3 percent for baseline verification methods (Hao et al., arXiv:2601.04086, 2026).

One finding runs against the intuition that more structure is always better. A 2024 study found that requiring a model to produce output in a strict format (JSON or XML) can reduce reasoning accuracy significantly, with stricter constraints causing larger drops. The distinction matters: structure in the reasoning steps helps. Structure imposed only on the output format can hurt (Tam et al., arXiv:2408.02442, October 2024). The design implication is to separate the two. Guide how the model thinks; constrain what it produces only where the format genuinely serves the downstream use.

Four Techniques That Actually Work

The research converges on a small set of techniques that consistently reduce hallucination rates in production contexts.

Give the model a role and a constraint, not only a task. A prompt that begins “You are a QA engineer reviewing an API contract. Your job is to identify inconsistencies between the spec and the implementation. Flag anything you cannot confirm from the provided text. Do not infer.” sets different generation conditions than “Review this API.” The role focuses the model. The explicit uncertainty instruction is the key addition. It gives the model a grammatically natural path to say it does not know, rather than generating plausible content to fill the gap.

Ask for reasoning before the answer. This is the core mechanism behind chain of thought accuracy gains. Adding a line like “Before answering, list the steps you are taking and the evidence you have for each” changes the token generation sequence. The reasoning tokens constrain the answer tokens. The model cannot easily contradict its own stated steps. For test case generation, this often surfaces gaps the model would otherwise paper over with invented detail.

Supply the facts explicitly. A model that hallucinates endpoint names or error codes is almost always doing so because those facts were not in the prompt. Grounding the prompt with the actual contract, schema, or specification changes the problem from one of recall to one of extraction. The model still makes mistakes, but the error modes are different. It misreads provided text rather than inventing absent detail, and those errors are easier to catch in review.

Use multiple samples for outputs where accuracy matters. Sampling the same prompt several times with temperature above zero and comparing the answers is a practical way to identify responses the model is uncertain about. Consistent answers across runs are more likely to be correct than answers that vary. This is the consistency sampling approach from the research applied without the complexity of the original implementation. It is straightforward to add to any test pipeline.

What This Means for a Testing Workflow

For QA teams integrating AI tools into their work, the practical implication is that prompt quality is a testable artifact. A prompt that consistently produces hallucinated API details should be treated the same way as a test case that gives false positives: it needs to be fixed, not worked around.

The fixes follow the techniques above. Constrain the model’s scope. Ground it in provided text. Ask for reasoning. Check consistency for outputs where accuracy matters.

None of this eliminates hallucinations. A 2025 study noted that chain of thought prompting reduces hallucination frequency but also makes the hallucinations that remain harder to detect, because the reasoning chain makes the wrong answer look more justified (Cheng et al., arXiv:2506.17088, 2025). Prompt structure reduces the error rate; it does not replace review.

The right mental model is defense in depth. Prompt design is one layer. Static analysis of AI output is another. Sandboxed execution is a third. Each layer catches what the previous one misses.

The ISTQB Generative AI (CT-GenAI) Certification Exam Prep covers the conceptual framework for evaluating generative AI outputs and the testing techniques that apply specifically to AI systems, including how to design test cases for hallucination detection.

For teams building test pipelines that use AI tools, the Generative AI and AI Agents for QA SDET Masterclass covers prompt design for test automation contexts, including how to structure prompts for test case generation, test data synthesis, and agentic pipelines where prompt reliability affects downstream execution.