Data Quality is Everything Why Fine-Tuning Fails

The Number That Changes Everything

Pick any two models trained on the same base. Give one 50,000 examples chosen at random. Give the other 1,000 examples chosen carefully. The one trained on 1,000 examples will often win.

That result keeps appearing in fine tuning research, and it keeps surprising teams who assume that more data is always better. The reason is that model weights encode relationships, not counts. A noisy example teaches the wrong relationship. A redundant example teaches nothing new. Neither contributes to what you actually want from the training run.

Getting data quality right before training starts is the most impactful work in the fine tuning pipeline. This post covers the failure modes that appear most consistently in the research and the fixes that address them.

Why Quantity Alone Fails

The LIMA study by Zhou et al. (arXiv:2305.11206, 2023) trained a 65B LLaMA model on exactly 1,000 curated examples with no reinforcement learning from human feedback. In human evaluations, the resulting model was equivalent to or preferred over GPT-4 in 43% of comparisons, preferred over Google Bard in 58% of comparisons, and preferred over DaVinci-003 in 65% of comparisons.

The paper’s core conclusion: “almost all knowledge in large language models is learned during pretraining.” The fine tuning phase is alignment, not knowledge acquisition. You are teaching the model a style, a format, and a set of response norms. Noisy training data corrupts the alignment signal without adding anything.

The AlpaGasus study by Chen et al. (arXiv:2307.08701, 2023) tested this directly. The researchers took the 52,000-example Alpaca dataset and filtered it down to 9,000 examples using an automated quality rating. The filtered model significantly outperformed the full dataset model on GPT-4 evaluation and human evaluation. The 13B variant reached over 90% of the performance of OpenAI’s davinci model trained with RLHF. Training time dropped from 80 minutes to 14 minutes for the 7B variant, a 5.7x speedup.

Duplication Corrupts the Distribution

The second failure mode is duplication. Lee et al. (arXiv:2107.06499, 2021) analyzed widely used pretraining datasets and found that more than 4% of validation set examples overlap with training data in standard benchmarks. One 61-word sentence appeared more than 60,000 times in the C4 dataset. When a model trains on the same text thousands of times, it memorizes it. That memorization shows up at inference as regurgitation of training examples rather than generalization.

Deduplication reduced memorized output by approximately 10x in that study. It also allowed models to reach the same accuracy with fewer training steps, because each step contained genuinely new signal rather than repetition of material the model had already processed.

For fine tuning datasets sourced from production logs, support tickets, or scraped documentation, deduplication is not optional. Semantically similar examples, where two records differ by a field name or minor formatting, cause the same problem as exact duplicates. Semantic deduplication using embedding similarity can catch these cases that exact string matching misses.

Format Inconsistency Teaches the Wrong Pattern

A model training on examples where the same task is answered with bullet points in one case, numbered steps in another, and prose in a third learns to be inconsistent. The training loss function interprets this as noise in the output space, and the model spreads probability across formats rather than committing to any one of them.

The fix is dataset auditing before training. If your goal is code generation, every example should end with syntactically valid code in the target language. If the goal is structured JSON output, every example should have valid JSON in the correct schema. One malformed example in a small dataset has a disproportionate effect on the final model, since each example represents a larger fraction of the training signal.

The Refine-n-Judge approach by Cayir et al. (arXiv:2508.01543, 2025) addressed this at scale by having a language model refine each response and judge whether the refined version improved on the original. Llama 3.1-8B and 3.3-70B models trained on this curated data were preferred by GPT-4 judges in over 74% of comparisons against models trained on the original unfiltered data, a gain of 5% on AlpacaEval and 19% on a multiturn benchmark suite. The method was tested across coding, math, and conversational tasks.

Train-Test Overlap Produces Misleading Benchmarks

If any of your evaluation examples appear in your training data, your benchmarks will overstate the model’s real capability. The model has seen the answers. Lee et al. found the scale of this problem in major public datasets: over 4% of standard validation sets contained examples from training.

For QA and testing teams building models from a codebase, a ticket system, or internal documentation, this risk is especially high when training data and evaluation data come from the same source. Hold out a representative evaluation set before any training data selection begins, and run explicit overlap checks between training and evaluation splits. That is the minimum process required to trust your metrics.

Dataset Reduction Without Accuracy Loss

Selecting a smaller, carefully chosen subset of a training corpus often matches or beats training on the full dataset. Lv et al. (arXiv:2504.12687, 2025) trained DeepSeek-Coder-Base-6.7B on 40% of the code instruction dataset and scored 66.9% average on code generation benchmarks, compared to 66.1% for the full dataset. Training time dropped from 47 to 34 minutes, and GPU memory dropped from 61.47 GB to 42.72 GB.

Harada et al. (arXiv:2506.14681, 2025) trained over 1,000 supervised fine tuning models and found that perplexity on the training data consistently predicts whether fine tuning will be effective for a given model and dataset pair. Very low perplexity means the model already knows the material and gains will be limited. The productive range sits in between.

The RAG Pipeline Warning

One finding that QA teams should take seriously: Barnett et al. (arXiv:2406.11201, 2024) found that fine tuning language models inside retrieval augmented generation pipelines can reduce overall system performance compared to the unmodified baseline. The fine tuned model’s behavior changed in ways that reduced its ability to use retrieved context. The authors urge rigorous validation before any RAG deployment, not only evaluation of the fine tuned model in isolation.

A Data Quality Checklist Before Training

Before any fine tuning run, the dataset should satisfy four conditions: every example should represent the actual task being optimized; examples should be deduplicated using both exact and semantic methods; outputs should conform to a consistent format; and the evaluation set should be held out before training data selection begins, with overlap explicitly checked.

The result is a better model, a faster training run, lower compute cost, and a benchmark you can trust.

For teams building QA automation with language models, the LangChain for QA: Build RAG, Agentic RAG and AI Chatbots course covers prompt design patterns that reduce the amount of model adaptation work required.

Teams who want to go deeper on model training, evaluation methodology, and how to measure whether a model change actually improved performance will find the evaluation sections in the Generative AI and AI Agents for QA SDET Masterclass directly applicable to these decisions.