Here is the number that surprises most buyers: fine-tuning a large language model, the actual training run, is cheap. A LoRA fine-tune costs $50 to $500 in compute, and a 70B QLoRA run on a rented H100 lands around $10 to $16. If someone quotes you six figures and points at “GPU costs,” they are either doing a full fine-tune you probably do not need, or the money is going somewhere they have not told you about.
It is going somewhere. Fine-tuning a model is a fraction of the project. The expensive part is getting a clean, labeled dataset in front of it, and proving the tuned model is actually better than the one you started with. Data preparation eats 35 to 50% of a typical fine-tuning budget, and it is the line item quotes love to bury.
We’re gmware, a custom software development firm in Austin, TX with engineering centers in Bangalore and Mohali, India, and fine-tuning work sits inside our AI delivery practice. This post is the transparent version of the quote: cost by approach, where the money actually goes, the honest answer on fine-tuning versus RAG, and how to scope a build so the surprise line does not show up in week six.
LLM fine-tuning cost by approach in 2026
There are three ways to fine-tune, and they sit at wildly different price points. Managed provider fine-tuning (OpenAI, Google, others) is the fastest to start and priced per token. LoRA and QLoRA on open-weight models are the cheap self-hosted middle. Full fine-tuning is the expensive ceiling almost nobody outside a research lab needs.
| Approach | What it is | Cost per run | When to use it |
|---|---|---|---|
| Managed provider fine-tune | Upload data, they train, you rent the result | $25 per 1M training tokens on GPT-4o | Fastest start, no infra to run |
| LoRA / QLoRA (PEFT) | Train a small adapter on an open-weight model | $50 to $500 | Most business use cases |
| Full fine-tune | Update every weight in the model | $2,000 to $50,000 | Deep behavior change only |
Cost per fine-tuning run, by approach
The gap between LoRA and full fine-tuning is the whole story. A full fine-tune of a 7B model needs roughly 100 to 120 GB of VRAM, about $50,000 of H100 hardware for one run, while the same model tunes on a single rented GPU with QLoRA for under $5. LoRA updates 0.1 to 1% of the parameters and, on the tasks businesses actually care about (style, format, domain tone), it lands close to full fine-tuning quality. Reach for full fine-tuning only when you need to genuinely rewire how the model reasons, and be ready for the bill.
LoRA and QLoRA: the cheap path most teams should take
Parameter-efficient fine-tuning (PEFT) is why fine-tuning stopped being a rich-lab activity. Instead of retraining the whole model, LoRA trains a small adapter that rides on top of frozen weights. QLoRA goes further, quantizing the base model so it fits on cheaper hardware. The compute numbers by model size are almost boringly small:
| Model size | Hardware | Training time | Compute cost |
|---|---|---|---|
| 7B | RTX 4090 | 2 to 4 hours | $1.10 to $2.20 |
| 13B | A100 40GB | 3 to 6 hours | $2.28 to $4.56 |
| 70B | H100 80GB | 8 to 12 hours | $10.64 to $15.96 |
H100 time rents for about $1.33 an hour on specialized GPU clouds, which is why a 70B adapter run costs less than lunch. Compare that to a full fine-tune of a 70B model, which needs 8x H100s for 24 to 48 hours at $255 to $510. Same model, 30x the cost, for a quality lift most tasks never feel.
The catch is not the run. It is that you can run LoRA ten times for the price of a sandwich, and every one of those runs is worthless if the training data behind it is dirty. Cheap compute makes the data the entire game.
Where the money actually goes
If the training run is $50 to $500, and a real project is $50K to $200K, the difference is not GPUs. It is three unglamorous stages: preparing the data, running the training experiments, and building the eval harness that tells you whether any of it worked.
Where a fine-tuning budget actually goes
Data preparation is the big one. Most firms spend $4 on data prep for every $1 spent on the model, and it is not abstract busywork. It is writing thousands of high-quality input-output pairs, labeling them, and reviewing them so a bad example does not teach the model the wrong lesson. A 10,000-example dataset of structured financial data runs $50K to $80K to label and clean; messy unstructured text like support tickets or legal documents can push past $120K for the same size.
Then comes evaluation, which is where fine-tuning projects quietly succeed or fail. Review cycles and QA run 20 to 40% of total fine-tuning cost. Without an eval set, “did the tune help?” is a vibe, not a number, and a model that regressed on half your cases ships to production because the demo looked good. The eval harness is not overhead. It is the thing that lets you say no to a bad checkpoint.
What a fine-tuned model costs to run each month
Training is a one-time hit. Serving is forever, and it splits by how you deploy.
If you fine-tune with a managed provider, you pay a per-token premium on top of the base rate. OpenAI’s fine-tuned GPT-4o inference runs about $3.75 per 1M input tokens and $15 per 1M output tokens, noticeably higher than the un-tuned model. That premium is fine at low volume and painful at high volume, which is the exact point where self-hosting starts to pay off.
If you self-host an open-weight model, you trade the per-token premium for fixed infrastructure. A 7B build runs about $953 a month self-hosted, and a 70B build over $3,200 a month in serving cost, before the MLOps time to keep it healthy. The upside on the open-weight path is inference economics: a LoRA-adapted Llama-3-8B serves at $0.20 to $0.40 per 1M tokens versus $10 to $15 for GPT-4o. At scale, that spread is the entire business case for fine-tuning a small model instead of prompting a large one.
Fine-tuning versus RAG: the honest answer
This is the question behind most fine-tuning requests, and the honest answer disappoints people who came in wanting a model of their own. Fine-tuning teaches a model how to respond: style, format, tone, domain reasoning. RAG controls what it knows by feeding it your documents at answer time.
Fine-tune when outputs must follow a strict house format, when the domain language is genuinely specialized, or when you want a small cheap model to punch above its weight. Use RAG when the facts change often and must be cited, because updating retrieval is an upload, not a training run. We wrote the full economics of the retrieval path in our RAG implementation cost guide, and the two stack cleanly: fine-tune for voice, retrieve for facts.
Stated plainly, from scoping these builds: most teams who arrive asking for fine-tuning need better prompting and retrieval first. Those are free and reversible. Fine-tuning is a commitment, and a fine-tuned model that memorized last quarter’s pricing is a liability the day pricing changes. Start with the cheap, reversible options. Graduate to fine-tuning when you can name the specific behavior prompting could not fix.
A realistic fine-tuning scoping plan
- Week 1, decide if you even should. Try prompting and RAG against your hardest cases first. If a well-built prompt clears the bar, you just saved six figures. This is the phase most vendors skip, because it might end the engagement.
- Weeks 2 to 4, build the dataset. Inventory and label your examples, write the input-output pairs, set up review. This is the 35 to 50% line, and it is where honest projects spend their time.
- Weeks 5 to 6, build the eval harness before you train anything. A few hundred real cases with known-good answers, scored automatically, before a single run. No eval, no verdict.
- Weeks 7 to 8, run and iterate. LoRA is cheap enough to run many times. Tune, score against the harness, keep the checkpoint that clears the gates, pilot it against real traffic.
The 8-week fine-tuning plan, phase by phase
How gmware scopes fine-tuning builds
We push back before we quote, because the cheapest fine-tuning project is the one you did not need. If prompting and retrieval clear your bar, we say so, even when it shrinks the engagement. When fine-tuning is genuinely the right call, we scope the dataset work as its own line rather than hiding it under “GPU costs,” and we write eval gates into the SOW so there is a number, not a feeling, deciding whether the model ships. Delivery runs from Austin with engineering in Bangalore and Mohali through our AI and machine learning practice, and when the answer is retrieval plus tuning together, our AI agents and LLM integration team handles the stack end to end.
Send us the behavior you want the model to have and a sample of the data behind it, and we’ll return a tiered estimate, prompting through fine-tuning, with the data-labeling line in plain sight, within 48 hours.