- 1There is no universal number. The right example count depends on technique, task narrowness, and base-model quality; provider minimums (OpenAI's technical floor of 10 examples, Google's recommended starting point of 100, AWS's suggested 50–100 to fine-tune Claude 3 Haiku on Bedrock) are floors, not sufficiency thresholds, and change over time. {cite:openai}{cite:gemini}{cite:bedrock}
- 2LIMA showed that 1,000 carefully curated examples can align a 65B-parameter model to strong instruction-following quality, supporting the 'superficial alignment hypothesis' that most knowledge comes from pretraining and fine-tuning mostly teaches format. Stanford's Alpaca, by contrast, needed 52,000 self-instruct examples for broad instruction coverage. {cite:lima}{cite:alpaca}
- 3Parameter-efficient fine-tuning changes the math: LoRA cuts trainable parameters by roughly 10,000x versus full fine-tuning of GPT-3-scale models, which is why useful LoRA/QLoRA results often start in the hundreds to low thousands of examples rather than the tens of thousands full fine-tuning implies. {cite:lora}{cite:qlora}
- 4Preference tuning (RLHF or DPO) is measured in labeled comparison pairs, not documents, and typically needs thousands to tens of thousands of pairs, because the model is learning human judgment, not new facts; DPO simplifies the training algorithm but not the data volume it needs. {cite:dpo}
- 5Below a few hundred examples per class or intent, models tend to memorize surface patterns instead of generalizing, so the practical fix isn't a bigger fixed number, it's a decision framework: define task narrowness, pick the cheapest technique that fits, start at the provider floor, and run a doubling-curve test before committing to a large collection or licensing spend.
Ask 'how much data do I need to fine-tune an LLM' and every credible source gives a different number, because they're answering different questions. Full fine-tuning, parameter-efficient methods like LoRA and QLoRA, instruction-tuning, and preference tuning (RLHF/DPO) each need a different order of magnitude of examples, and the honest range runs from dozens to millions depending on which one you're doing and how narrow the task is. This guide replaces the fake single answer with the real decision framework: what the research (LIMA, Alpaca, LoRA, QLoRA, DPO) shows about quality versus quantity, what OpenAI, Google, and AWS/Anthropic publish as minimums and recommendations as of 2026, how domain adaptation differs from style adaptation, where diminishing returns and overfitting kick in, and a step-by-step process to estimate the number your specific project needs.
On this page ▾
- Why there's no single answer, and why that's the useful part
- How much data you need, by technique
- What the research shows about quality vs. quantity
- Domain adaptation vs. style adaptation need very different volumes
- What the major providers publish, and why it changes
- Diminishing returns and the overfitting risk of too little data
- A practical decision framework to estimate your own number
- Two worked examples: a ticket classifier vs. full domain adaptation
- Start small, measure, then scale to the number your task needs
Why there's no single answer, and why that's the useful part
Search for how much data you need to fine-tune an LLM and you'll find confident, contradictory answers: some pages say a few hundred rows, some say tens of thousands, some say millions of tokens. All of them are right, for a specific technique and a specific task, and all of them are misleading as a general answer, because they don't say which one they mean.
Fine-tuning is not one operation. Full fine-tuning updates every weight in the model and behaves like training a mini version of the original model from your data. Parameter-efficient fine-tuning (PEFT), most commonly LoRA or its quantized variant QLoRA, freezes the base model and trains a small set of added parameters, which changes the sample size math dramatically. Instruction-tuning teaches a model to follow a task format. Preference tuning, RLHF or its simpler successor DPO, teaches judgment between two candidate outputs rather than a single correct answer. And few-shot / in-context learning is the alternative to all of the above: no training run at all, just examples placed in the prompt. Each of these has a different, defensible answer to "how much data," and conflating them is the single biggest reason this question feels unanswerable.
How much data you need, by technique
The table below gives realistic ranges reported in research and provider documentation for each approach. Treat the low end as "enough to see a directional signal" and the high end as "enough to be confident in production," not as a guarantee; the actual number for your task still depends on how narrow it is and how good the base model already is at adjacent tasks.
| Approach | Typical example-count range | When to use it |
|---|---|---|
| Few-shot / in-context learning (no fine-tuning) | 2–50 examples in the prompt | Task changes often, volume is low, or you need to validate an approach before investing in data collection. GPT-3's own paper showed strong task performance from a handful of in-context examples with zero gradient updates. [2] |
| LoRA / QLoRA (parameter-efficient fine-tuning) | ~100–10,000 examples | Narrow, well-defined tasks (classification, format adaptation, a support-ticket router) where you want a persistent behavior change without full retraining cost. [2][4] |
| Instruction-tuning / supervised fine-tuning (SFT) | ~500–50,000+ examples | Teaching a model a task format or a narrow domain's conventions. LIMA reached strong results at 1,000 curated examples; Alpaca used 52,000 broader ones for general instruction-following. Task breadth is the swing factor. [2][4] |
| Preference tuning: RLHF / DPO | ~1,000–100,000+ comparison pairs | Aligning a model's judgment (tone, safety, which of two answers is better) after it can already produce competent candidate outputs. Measured in labeled pairs, not documents. [2] |
| Full fine-tuning / continued pretraining (domain adaptation) | Hundreds of thousands to millions of documents or tokens | The model needs new facts, vocabulary, or a genuinely different distribution than a simple format change, e.g. adapting a general model to a full legal or clinical corpus. |
What the research shows about quality vs. quantity
The most cited data point in this entire debate is Meta's LIMA study. Researchers fine-tuned a 65-billion-parameter LLaMA model on just 1,000 carefully curated prompt-and-response pairs, with no reinforcement learning and no large-scale human preference modeling, and found the result competitive with much more heavily trained systems. [6]
“Almost all knowledge in large language models is learned during pretraining, and only limited instruction tuning data is necessary to teach models to produce high quality output.”
The paper names this the superficial alignment hypothesis: a model's factual knowledge and reasoning ability come almost entirely from pretraining on huge amounts of text, and fine-tuning's real job is narrower than most teams assume. It mostly teaches the model which format and style to use when responding, not new capabilities. If that framing is right, a small, carefully selected set of examples can be enough to unlock a capability the base model already has latent, which is why curation quality can matter more than raw row count for this kind of task. [6]
That doesn't mean 1,000 examples is a universal number, and the counterexample sits right next to it in the literature. Stanford's Alpaca project fine-tuned a smaller 7-billion-parameter LLaMA model on 52,000 self-instruct-style examples generated by prompting OpenAI's text-davinci-003, because the goal was broad, general instruction-following across many task types, not narrow style alignment on a curated set. [6] The difference between 1,000 and 52,000 isn't a contradiction. It's the difference between teaching a narrow, well-scoped behavior and teaching broad coverage across many unrelated tasks.
Domain adaptation vs. style adaptation need very different volumes
The single biggest factor teams get wrong when estimating their own data need is conflating two very different jobs: teaching a model how to respond (style, format, tone) versus teaching it what is true in a domain it hasn't seen (new vocabulary, new facts, new patterns of reasoning specific to a field).
- Style and format adaptation (respond as JSON, use this tone, follow this template, classify into these five buckets) is close to what LIMA demonstrates: the model already knows the underlying content, it just needs to see the target shape a few hundred to a few thousand times. LoRA-scale example counts are usually sufficient. [4][6]
- Domain adaptation (a model that has never seen your industry's terminology, edge cases, or internal logic, like clinical coding, niche legal doctrine, or a company's proprietary product taxonomy) requires the model to acquire new information the base model never saw. That pushes the requirement toward continued pretraining or full fine-tuning on a much larger corpus, because it expands what the model knows rather than how it presents what it already knows.
- Most teams overestimate which one they're doing. A support-ticket router is usually style/format adaptation dressed up as a domain problem; a genuinely novel scientific or legal corpus is usually domain adaptation. Getting this distinction right before choosing a data budget saves both money and disappointment.
What the major providers publish, and why it changes
Provider documentation is useful as a floor, not as sufficiency guidance, and every one of these numbers can move without notice as models and platforms change. Treat the following as a snapshot dated to when it was checked, not a permanent spec.
| Provider / platform | Published guidance (verify current docs before relying on it) | As of |
|---|---|---|
| OpenAI fine-tuning | Technical minimum of 10 examples; sets under 100 examples need extra care with consistent prompting; expect roughly linear quality gains each time you double the dataset, up to a point. [2] | 2026 |
| Google Gemini supervised tuning | Recommends starting with 100 examples, scaling to thousands if needed; explicitly states dataset quality matters more than quantity. [2] | 2026 |
| AWS Bedrock, Claude 3 Haiku fine-tuning | Recommends 50-100 rows as a reasonable starting point; hard caps of 10,000 training records and 1,000 validation records. [2] | 2024 |
Diminishing returns and the overfitting risk of too little data
Two failure modes sit on opposite ends of the data-volume spectrum, and teams tend to worry about only one of them.
- Too little, too narrow data causes overfitting. Below roughly a few hundred examples per class or intent, a model tends to memorize surface patterns in the specific phrasing it saw rather than generalizing the underlying task. It performs well on training-set-like inputs and degrades sharply on anything slightly different in wording or edge-case structure.
- Past a certain point, more data yields shrinking returns. OpenAI's own guidance frames this directly: expect roughly linear improvement each time you double the training set, which also implies each doubling costs the same collection and compute effort for a smaller absolute gain. [4] Continuing to scale volume without also improving quality eventually stops being the highest-leverage lever.
- Diversity matters as much as count. A dataset with high volume but low variation (the same handful of phrasings repeated thousands of times) behaves like a much smaller dataset in practice, because it doesn't teach the model to generalize across the actual range of inputs it will see in production.
A practical decision framework to estimate your own number
Skip the search for a universal number and run this five-step process instead. It takes an afternoon and produces a defensible, task-specific estimate instead of a guess borrowed from someone else's use case.
- 1Classify the job: style, domain, or preference. Are you teaching a format/tone (style), new facts and vocabulary (domain), or a judgment between two outputs (preference)? This alone eliminates most of the wrong ranges in the table above.
- 2Pick the cheapest technique that plausibly fits. Few-shot first if volume and stakes are low. LoRA/QLoRA next for narrow, persistent behaviors. Full fine-tuning only when you have genuinely new domain knowledge to inject. RLHF/DPO only after the base model can already produce competent candidate outputs to compare. [4][6]
- 3Start at the relevant provider floor, not the mid-range. Run an initial pass at the low end of the range for your technique (50-100 for PEFT-style tasks, 1,000+ for broader instruction-tuning) and measure actual eval performance before collecting more. [4][6]
- 4Run the doubling test. Train on your current set and on half of it. A big performance gap means keep collecting; a small one means switch effort to curation quality, deduplication, and coverage of edge cases instead of raw volume. [4]
- 5Budget for the narrow-task floor, not the domain-adaptation ceiling, unless you've confirmed you need it. Most teams that think they need millions of documents need only a few thousand well-curated, well-labeled examples, because the task is narrower than it first appears once step 1 is done honestly.
Two worked examples: a ticket classifier vs. full domain adaptation
The same five-step process produces very different numbers depending on the task. Here's how it plays out for two teams with genuinely different jobs.
Team A: a support-ticket classifier
A startup wants a model that reads an inbound support ticket and routes it to one of 12 categories, then drafts a first response in the company's voice. Step 1: this is a style/format task, the base model already understands the underlying language and concepts; it just needs to learn the routing taxonomy and house tone. Step 2: LoRA is the right technique, since the behavior needs to persist across a high volume of tickets without the cost of full retraining. Step 3: start with roughly 300-500 labeled tickets per category (around 4,000-6,000 total), near the middle of the LoRA range. Step 4: the doubling test on an initial 2,000-example pass shows only a small accuracy gap versus the full set, signaling the team is close to diminishing returns already. Step 5: the team ships with roughly 4,000 well-labeled examples and spends the saved collection budget on cleaning mislabeled tickets instead of gathering more.
Team B: full domain adaptation for legal document analysis
A different startup, winding down a legal-tech product, holds a large corpus of contract clauses, redlines, and negotiation histories and wants a partner AI lab to adapt a general model into one that deeply understands a specific area of contract law it has never been trained on. Step 1: this is genuine domain adaptation, new legal doctrine and drafting conventions the base model doesn't reliably know, not a formatting change. Step 2: full fine-tuning or continued pretraining is the right technique, since LoRA-scale example counts can't inject that volume of new domain knowledge reliably. Step 3: the realistic floor is in the hundreds of thousands of clauses and documents, not thousands. Step 4: at that scale, the doubling test is run on training subsets during a pilot to confirm the curve hasn't flattened before committing the full corpus. Step 5: the buying AI lab licenses the full documented corpus, because in this scenario the volume genuinely is the value, unlike Team A's case.
| Team A: ticket classifier | Team B: legal domain adaptation | |
|---|---|---|
| Job type | Style / format | Domain adaptation |
| Technique | LoRA | Full fine-tuning / continued pretraining |
| Starting estimate | ~4,000-6,000 labeled tickets | Hundreds of thousands of documents |
| Where the doubling test lands | Near diminishing returns quickly | Still on the steep part of the curve at typical pilot scale |
| Where budget should go | Label quality and edge-case coverage | Corpus volume and provenance documentation |
Start small, measure, then scale to the number your task needs
The honest answer to how much data you need to fine-tune an LLM is that the question is underspecified until you name the technique and the task. Few-shot prompting can work from a handful of examples. [2] LoRA and QLoRA can produce useful, persistent behavior changes from hundreds to low thousands of well-chosen examples. [4][6] Instruction-tuning ranges from LIMA's 1,000 curated examples for a narrow behavior to Alpaca's 52,000 for broad coverage. [8][10] Preference tuning is measured in labeled comparison pairs, usually thousands to tens of thousands. [12] And genuine domain adaptation, teaching a model facts and vocabulary it has never encountered, is the one case where the honest floor is hundreds of thousands of documents or more.
Run the five-step framework before you commit a budget to data collection or licensing. Most teams discover their task is narrower than they assumed, and the number they actually need is smaller, cheaper, and more achievable than the six- or seven-figure corpus they braced for. The teams that do need the large end of the range, real domain adaptation on data no public corpus contains, are exactly the ones for whom a well-documented, provenance-clean proprietary dataset becomes a genuinely valuable asset rather than a nice-to-have.
Need the real dataset behind your fine-tune, not just more rows?
Dayda brokers vetted, provenance-documented proprietary datasets, support transcripts, domain corpora, preference pairs, to AI labs and enterprises doing genuine domain adaptation. Tell us what you're building and we'll help you scope the data you need.
See how buying works on Dayda→