When a Custom Model Beats a Frontier API on Cost

A custom model beats a frontier API on cost once one task's monthly volume covers the model's fixed costs: the GPUs it keeps running, the evaluation, monitoring and retraining work, and the amortized build. The break-even volume is those fixed costs divided by the saving per API call, priced at the frontier rate you actually pay.

This guide is for the engineering lead or architect running that comparison. It gives the formula for each side, one worked example with every number shown, the costs teams forget, and a migration path that protects quality while traffic moves. Prices are list prices from OpenAI and Anthropic as of October 2026. Apply the configuration techniques in LLM cost optimization: 12 techniques ranked first, since they change the frontier price you are comparing against.

How do you calculate what a frontier API costs per month?

Multiply monthly API calls by the cost of one API call, and price that API call at the rate you would actually pay after caching, batching and moving to the cheapest tier that passes your eval. The list price of the model you prototyped on overstates the frontier side and makes every custom model look cheaper than it is.

frontier cost per month = API calls per month x cost per API call

cost per API call = uncached input tokens x input price
                  + cached input tokens   x cache price
                  + output tokens         x output price   (reasoning tokens count as output)

Adjust the price before you compare. Each line below moves the frontier side, and most of them move it down.

As of October 2026, Anthropic bills cache hits at a tenth of the base input price on most Claude models, OpenAI lists cached input for GPT-6 Astra and GPT-6 Luna at a tenth of standard input, and both providers price batch processing at 50% of standard rates. Retries push the other way: a task that retries 5% of its API calls pays 5% more per task.

Worked example: the frontier side

Take a claims-triage task with 2,000 input tokens and 200 output tokens per API call, where 1,500 of the input tokens are a stable instruction prefix that caches. It runs synchronously on Claude Sonnet 5.5, listed at $2 per million input tokens and $10 per million output tokens as of October 2026, with cache hits at $0.20 per million.

At steady traffic the prefix stays warm, so cache writes add little and are left out. Without the cache, the same API call costs $0.0060. Keep that gap in mind for the break-even below, because comparing against $0.0060 would put the break-even at about half the true volume.

What does it cost to run your own custom model?

A custom model costs the GPUs you keep running for it, busy or idle, plus the recurring work of keeping it good, plus the build spread over its useful life. Escalated requests that still go to a frontier model belong on this side of the ledger too.

custom cost per month = GPU hours x GPU price                (serving)
                      + engineering time x loaded cost        (evaluation, monitoring, retraining, on-call)
                      + retraining and evaluation compute
                      + build cost / months of useful life
                      + escalated API calls x frontier price

Serving. As of October 2026, on-demand list prices at RunPod, Lambda and CoreWeave put an H100 at about $2.90 to $6.20 per GPU-hour and an L40S at about $1.10 to $2.25. Committed capacity changes the rate and commits you to the hours, so price it from your provider's own quote.

Operating work. Running a model in production is ongoing work: refreshing the eval set as inputs change, scoring every candidate version, monitoring quality and latency, retraining as the task drifts, and owning the endpoint around the clock. Price it as a share of an engineer's time at your fully loaded cost. Leaving it out compares a managed service, the frontier API, against only the hardware half of the alternative.

Build. Data collection and cleaning, training runs, the first eval suite and the serving setup. Spread it over the months you expect the model to stay in service before a rebuild.

Why is GPU utilization the hidden variable?

A GPU bills by the hour whether it serves one request or a million, so the serving cost of a request is the hourly price divided by the requests that hour actually carried. Utilization, the share of capacity your traffic fills, decides that number more than the GPU price does.

Suppose a load test shows one GPU sustains 10 requests per second on this task at full load, or 36,000 an hour. At $4.00 per GPU-hour, a fully busy GPU serves a request for about $0.00011. The same GPU at 5% busy costs $0.0022 per request, two thirds of the cached frontier price.

GPUs sit idle for ordinary reasons. Traffic peaks during business hours and the fleet is sized for the peak. An endpoint that stays up through a failure or a deploy needs at least two replicas. Staging and evaluation environments hold GPUs of their own. Scale-to-zero serving trims idle hours at the price of a slow first response after a quiet spell. A model trained for one task usually runs on a much shorter prompt, since its instructions no longer ride along on every request, and that raises the requests each GPU can carry.

Worked example: the custom side

Here is the same claims-triage task on a custom model, with each assumption stated so you can replace it with your own.

Two GPUs at 10 requests per second each carry about 52.6 million requests a month at full load, so the fleet stays at two through every volume in this example. Serving is just over a quarter of the fixed total. The engineering time is the largest line.

What is the break-even volume for a custom model?

The break-even volume is the custom model's monthly fixed cost divided by what each API call saves after escalations. In the worked example it is about 7.35 million API calls a month, roughly 240,000 a day.

break-even API calls per month = F / ((1 - e) x p - m)

F = monthly fixed cost of the custom model (serving floor, operating work, compute, amortized build)
e = share of API calls still escalated to a frontier model
p = frontier cost per API call, at the price you actually pay
m = serving cost per API call above the fleet floor (zero while within capacity)

With F = $21,840, e = 10%, p = $0.0033 and m = 0, the break-even is 21,840 / (0.9 × 0.0033) = 21,840 / 0.00297 = 7,353,535 API calls a month.

Below that volume the frontier API is cheaper. Above it, each additional API call saves 0.9 × $0.0033 = $0.00297, so at 20 million API calls a month the custom model costs $28,440 against $66,000, a saving of 57%.

How sensitive is the break-even to each input?

The inputs are estimates, so run the formula across the range you believe. Each row changes one input from the worked example.

The GPU price moves the answer least. The frontier price you compare against moves it most, and each $1,000 of monthly fixed cost adds about 337,000 API calls a month to the break-even. The last row is the one to check first: as of October 2026, GPT-6 Luna lists at $0.10 per million input tokens and $0.50 per million output tokens, so this task would cost $0.0003 per API call on it before caching. If a tier that cheap passes your eval, cost alone rarely justifies a custom model, and the case rests on latency or where the data must stay.

Which costs do teams forget on each side?

Both sides carry costs that never show up in a first estimate. Add them before you trust the break-even.

OpenAI's reasoning guide states that reasoning tokens are billed as output tokens even though the API never returns them, which is how a reasoning model's real price per API call drifts above the estimate built from visible output. Measure both sides as cost per successful task, including every retry and escalation, as set out in our guide to cost per task and AI unit economics.

When should you replace a frontier model with a custom model?

Replace a frontier model on one task when its volume sits well above your break-even, the task is stable, and an eval can score it automatically. Latency and data location add reasons beyond cost. Keep the frontier model where volume is low, the task changes weekly, or nobody can define a good answer.

Leave margin above the break-even, because every input to the formula is an estimate until the model is serving real traffic. Our earlier comparison of a fine-tuned small model against the GPT-4 API covers accuracy and latency by task type, and when a SaaS should graduate from API calls to fine-tuning walks through the decision at a growing product company. This guide adds the operating work, the escalation share and the frontier price after caching and batching to that picture.

How do you migrate from a frontier API without risking quality?

Move traffic in stages, each gated by an eval. Run the custom model in the background on real requests, score it against your quality bar, give it a small live share with the frontier model as fallback, then ramp. At every stage, users only depend on a version that has already passed.

1. Baseline. Log cost per successful task and score the current frontier model on your eval, so the custom model has a bar to clear and a cost to beat.
2. Background scoring. Send a copy of real requests to the custom model while users still receive frontier answers. Score its outputs against the eval and watch latency and error rates under real load.
3. Eval gate. Promote only a version that meets the bar on the eval and on the background sample, with a named person signing off.
4. Small live share. Route a few percent of live traffic to the custom model, with automatic fallback to the frontier model on a validation failure, a timeout or low confidence.
5. Ramp. Raise the share in steps while the quality and cost reports hold.
6. Keep the fallback. Hard cases and new kinds of input keep going to a frontier model, and every retrained version goes back through the same gate.

Treat each retrained version as a new model. A retrain that improves one slice of traffic can regress another, and the gate catches it before users do. The routing patterns behind steps 4 and 6 are covered in LLM routing strategies to cut frontier API costs.

How does an adaptive model endpoint change the math?

An adaptive model endpoint takes the operating side of the formula off your team. It is one API endpoint for one of your tasks: frontier models answer at the start, a custom model trained on your requests and your team's feedback takes over once it passes your evals, and hard cases stay on frontier models.

The migration path above is built in. Each new version of the custom model runs in the background on real requests and is scored against your evals, and nothing switches without your sign-off. We monitor the model and retrain it as your task changes. It runs Ertas-managed, in your cloud or on-premise, and you own it. The endpoint is currently invitation only, and we are developing a more streamlined version for wider access. Your engineers change one model name, and frontier models serve every request at the start, so nothing waits on training.

The cost question then becomes one comparison: the endpoint against the frontier bill it replaces on that task, run on your own traffic and token counts.

The serving side can be very cheap for a model trained for one job. In our published QMSum study, a 1.2B model trained for query-focused meeting summarization had an estimated serving cost of $0.00053 per query on a rented cloud GPU, against $0.104 for Claude Opus 5 and $0.0130 for GPT-5.6 Luna at synchronous list prices. It answered in 0.39 seconds against 9.0 seconds for Claude Opus 5. The estimate excludes training cost, and quality is task-specific, which is why every switch waits on your evals.

Frequently Asked Questions

Is a fine-tuned model cheaper than a GPT or Claude API?

Per request, a model fine-tuned for one task usually costs far less to serve, because it can be much smaller and runs on a shorter prompt. Per month, it is cheaper only above the break-even volume, because GPUs, evaluation, monitoring and retraining cost money whether traffic arrives or not. Run the formula with the frontier price you actually pay.

What is the break-even point for self-hosting an LLM?

It is the monthly fixed cost of the self-hosted model divided by the saving per API call after escalations, F / ((1 - e) × p - m). In our worked example, with $21,840 a month of fixed cost and a cached frontier price of $0.0033 per API call, the break-even is about 7.35 million API calls a month. Your inputs will differ, so run it with your own.

How much does it cost to run your own LLM per month?

Serving, operating work and the amortized build. As of October 2026, an H100 lists at about $2.90 to $6.20 per GPU-hour on demand at RunPod, Lambda and CoreWeave, so two GPUs running all month cost about $4,200 to $9,000. In the worked example, the engineering time to evaluate, monitor and retrain the model is the larger line.

Should I compare a custom model against batch prices?

For work that can wait for results, yes. As of October 2026, OpenAI and Anthropic price batch processing at 50% of standard rates, and halving the frontier price doubles the break-even volume. Comparing an asynchronous workload against synchronous list prices flatters the custom model.

What happens to hard requests after switching to a custom model?

They keep going to a frontier model. Route on validation failures, timeouts and low confidence, and budget for that escalation share in the formula, since escalated API calls bill at frontier prices. With an adaptive model endpoint, hard cases stay on frontier models by design.

Who owns a custom model served through an adaptive model endpoint?

You do. The model is trained on your requests and your team's feedback for your task, and it runs Ertas-managed, in your cloud or on-premise. We monitor it and retrain it as your task changes.

The Short Version

A custom model wins on cost above one volume: its monthly fixed cost divided by what each API call saves. Price the frontier side after caching, batching and right-sizing, count the operating work and the escalation share on the custom side, and size the GPUs for the utilization you will really get. Then migrate behind an eval gate with a frontier fallback.

If one task is heading past its break-even, book a call and we will run the numbers on your traffic with you.

Sources: OpenAI API pricing, Anthropic pricing, Anthropic prompt caching documentation, OpenAI reasoning guide and on-demand GPU pricing from RunPod, Lambda and CoreWeave. Prices verified 6 October 2026.