RAG vs fine-tuning: which one your project needs

The decision bench below sends the same question through a RAG pipeline and a fine-tuned model simultaneously. Pick a scenario preset, slide the source date past the JAN 2026 training cutoff, degrade the corpus or retriever quality, and watch every bar on the center spine respond in real time.

New tab

Demo

RAG vs fine-tuning: one question, two pipelines

The same question runs through a RAG pipeline and a fine-tuned model side by side. Slide the source date past the JAN 2026 cutoff, degrade the corpus or retriever, and watch freshness, citations, and style fit trade off. All scores and latency are illustrative teaching rules, not benchmarks.

Built with Canvas 2D and vanilla JavaScript. Scores and latency are deterministic teaching rules, labeled simulated and illustrative in the UI; no model runs at runtime. Render and interaction by Pro Trailblazer.

What you're seeing

The canvas splits into two pipeline lanes. On the left, the RAG pipeline flows from query through a retriever, into a context packet fed to a base model, and out to a cited answer. On the right, the fine-tuned pipeline flows from query into a model whose weights were updated during training on data frozen at the JAN 2026 cutoff, producing an uncited answer in the target voice or format. Between them, a center spine compares the two lanes on seven dimensions: freshness, citability, style fit, retrieval hit, training age, latency (RAG path versus fine-tuned path, both labeled SIMULATED), and upkeep load.

The source date slider runs from OCT 2025 to JUL 2026 and crosses the training cutoff. Before the cutoff both lanes can know the fact; after it, only the RAG lane can retrieve it. The five scenario presets (TODAY'S POLICY, BRAND VOICE, PRIVATE MANUALS, NEW PRODUCT, OUTPUT FORMAT) each carry a distinct teaching beat so you can see how the trade-offs shift by use case. All spine bar scores and latency figures are deterministic teaching rules, labeled illustrative or SIMULATED throughout the UI. Tap any bar, tag, chip, doc, or training strip to open a plain-language explanation.

A few things to try:

Slide the source date past JAN 2026 on TODAY'S POLICY. Watch the RAG lane keep its freshness bar high while the fine-tuned lane's freshness drops and the answer chip shows the stale fact stated with full confidence.
Switch to BRAND VOICE with the date before the cutoff. Both lanes know the underlying fact, but the RAG lane can only quote the style guide it finds; the fine-tuned lane applies the tone by default. Style fit belongs in weights, not in a retrieval index.
Try PRIVATE MANUALS and drop corpus quality below 45%. The retriever fails to find the needed document. RAG responds with a confident wrong citation; the fine-tuned lane can recall the procedure but cannot cite the section it came from.
Switch to BOTH mode on OUTPUT FORMAT. The combined overlay runs a narrow third lane. The verdict reaches BEST TOGETHER, showing when a fine-tune supplies the output schema while RAG supplies current values. Then degrade retrieval: the combined lane inherits RAG's failure, tagged RETRIEVAL MISSED / WEIGHTS FALLBACK.
Read the caveat panel. It covers prompting, tool calls, structured outputs, and base-model cutoffs. In many real cases one of those three beats building either pipeline.

How RAG and fine-tuning actually work

RAG keeps the base model's weights entirely unchanged. At query time a retriever converts the query into a dense vector, searches an index of document for the nearest chunks, and pastes those chunks into the model's as grounding text. The model reads and cites the retrieved text on every call. If the index is kept up to date, the model can answer questions about documents published yesterday; the knowledge lives outside the weights in an updatable store.

Fine-tuning continues from a different starting point: an already-trained base model. The process exposes the model to a curated set of examples and updates the weights to shift the model's behavior toward the target voice, format, or task. Those updates are frozen at the end of training. A fact in the training data can be recalled at inference time without any retrieval step, but the model cannot reach past the training cutoff and cannot point to a specific source the way a retriever can.

The center spine separates the two on dimensions that reflect real engineering decisions. Freshness measures how current each side's facts can be. Citability measures whether the answer can point at its evidence. Style fit measures how reliably output matches a target voice or format. Upkeep measures the ongoing maintenance load: re-indexing versus retraining. These are the dimensions that actually determine which approach is cheaper to maintain as the world changes.

Why the cutoff matters more than it looks

A fine-tuned model states old facts with the same confidence it states current ones. The training cutoff is invisible in the output. When a policy changes or a product price updates, the fine-tuned model still returns the old answer, correctly spelled, fluently phrased, and indistinguishable from a fresh one. This is different from a retrieval failure, where a careful reader might notice the citation is missing. Staleness without any signal of staleness is the harder problem.

RAG moves the knowledge out of the weights and into an index so facts can be updated without retraining. But it shifts the failure mode: if the retriever returns the wrong document, a stale revision, or nothing at all, the model will cite that miss as confidently as a hit. The PRIVATE MANUALS scenario in the demo shows this directly: a retrieval miss with a citation attached looks more trustworthy than an honest "I don't know."

Combining both pipelines is a pairing decision, not a safety net. The OUTPUT FORMAT preset ends in a BEST TOGETHER verdict because the fine-tune provides a reliable output schema (behavior from weights) while RAG provides the current values (facts from the index). But when retrieval misses, the combined pipeline falls back to the weights, inheriting the fine-tune's cutoff. Combining does not eliminate either failure mode; it adds a new one: the failure is now harder to trace.

Before building either pipeline, it is worth asking whether a structured prompt with or a few worked examples would achieve the same result. The caveat panel in the demo covers this. For many business cases, retrieval and fine-tuning are both over-engineered answers to a problem that structured outputs or prompting already solves. See the RAG pipeline deep dive for the retrieval mechanics in detail, and the embeddings explainer for how the dense vectors that drive retrieval are actually built.

Frequently asked questions

When should I use RAG instead of fine-tuning?

Use RAG when the facts you need change frequently, when you need the model to cite its sources, or when your knowledge base is too large to fit in a context window reliably. Use fine-tuning when you need a consistent output format, a specific voice, or a behavior pattern that prompting alone cannot lock in. The two dimensions rarely overlap: freshness and citation push toward RAG; style and behavior push toward fine-tuning.

Can I use both RAG and fine-tuning together?

Yes, and it is sometimes the right call. The OUTPUT FORMAT scenario in the demo shows a case where a fine-tune provides the output schema while RAG provides the current values. The cost is that the combined system inherits both failure modes: a retrieval miss falls back to the fine-tune's training data, and a style failure is harder to diagnose when retrieval is also in the loop. Combine when the two serve genuinely different jobs; do not combine to make either one more robust.

Why does a fine-tuned model state stale facts so confidently?

The model learned that the old fact was true from its training data. At inference time it has no awareness of a training cutoff; it does not know that time has passed. The fact was true when the training examples were written, so the model returns it with the same confidence it returns any other fact it learned. No prompting instruction can make the model reliably flag its own cutoff for information it genuinely believes is current.

What are the main failure modes of a RAG pipeline?

Three are common. First, retrieval miss: the needed document is not in the index or is not returned by the retriever, and the model produces a fluent answer without valid grounding. Second, stale index: the document exists but a newer version superseded it and the index was not updated. Third, citation laundering: the model cites a retrieved passage that is plausible but not directly relevant, giving a confident wrong answer the appearance of sourced authority. The PRIVATE MANUALS scenario in the demo exercises all three.

How many tokens does RAG add to each request?

It depends on the retriever configuration: how many chunks are retrieved and how long each is. A typical RAG setup retrieves three to five chunks of 256 to 512 each, adding 800 to 2,500 tokens per request before the query. That cost is per call and adds to inference latency. The LATENCY stat in the demo reflects this as a simulated teaching value, not a measurement.

Key takeaways

RAG and fine-tuning change different things: retrieval changes what the model can read at answer time, while a fine-tune changes how the model behaves by default. Facts that move belong in an index; voice and format belong in weights.
Staleness never sounds unsure. The old fact was true when the model was trained, so a fine-tuned model states it confidently, and no answer-time setting can reach past the cutoff. Only retraining moves it.
A fine-tune is not a document database. It compresses patterns from training data, so it can recall a procedure but cannot cite the page it came from, and low-quality data blurs similar procedures into one confident wrong answer.
Most RAG failures are retrieval failures. A missing document or a superseded revision still produces a fluent answer with a citation attached, which makes a wrong answer look more trustworthy, not less.
Combining both is a pairing, not a safety net. When retrieval misses, the combined pipeline falls back to the weights, staleness included, and prompting or structured output modes often beat building either one.