LLM Cost Optimization: 12 Techniques Ranked

The LLM cost optimization techniques with the biggest effect are, in order: serving a high-volume task from a custom model, right-sizing the model per task, routing requests between models, batching work that can wait, and prompt caching. Instrument cost per task first, because every technique below is judged against that number.

This guide ranks twelve techniques by how much they save on the traffic they touch, with the effort and quality risk of each. Prices are list prices from OpenAI and Anthropic as of October 2026. For the management view of the same problem, see how to reduce AI token costs in your company.

What is LLM cost optimization?

LLM cost optimization is the practice of lowering what each completed task costs in model usage, by sending fewer tokens, sending them to cheaper models, or paying a lower rate for them, while holding quality at an agreed bar. The unit to manage is cost per task, because a cheap model that needs three retries can cost more than an expensive one that succeeds on the first attempt.

Every technique acts on one term of the cost of a task:

cost per task = sum over the task's API calls of
    uncached input tokens x input price
  + cached input tokens   x cache price
  + output tokens         x output price   (reasoning tokens count as output)

The table maps each term to the techniques that shrink it.

Because the techniques act on different terms, their savings multiply. Caching the prefix, capping the output and moving to a cheaper tier compound on the same API call.

How do you measure LLM cost per task?

Log token usage on every API call with a task ID and a task type, then aggregate per completed task. Both providers return usage on every response, so this is a few lines of code:

# USD per million tokens, list prices as of October 2026.
PRICES = {
    "claude-opus-5": {"input": 5.00, "cache_read": 0.50, "cache_write": 6.25, "output": 25.00},
    "gpt-6-luna": {"input": 0.10, "cache_read": 0.01, "cache_write": 0.125, "output": 0.50},
}

def api_call_cost(model, uncached_in, cache_read, cache_write, output):
    p = PRICES[model]
    return (
        uncached_in * p["input"]
        + cache_read * p["cache_read"]
        + cache_write * p["cache_write"]
        + output * p["output"]
    ) / 1_000_000

Watch how each provider counts cached tokens before you trust the totals. Anthropic's prompt caching documentation reports input_tokens as only the tokens after the last cache breakpoint, with cache_read_input_tokens and cache_creation_input_tokens beside it, so total input is the sum of all three. Check your other providers' usage fields for whether cached tokens sit inside the input count or beside it, or your dashboard will double count.

Report cost per successful task: the cost of every API call a task made, including retries and escalations, divided by tasks that met the quality bar. That number is what every technique below has to move.

Which LLM cost optimization techniques save the most?

The ranking below orders techniques by the saving on the traffic each one touches. Your total saving depends on how much of your traffic each one applies to.

Ranks 4 to 7 are configuration changes most teams can ship within a week. Ranks 1 to 3 change which model does the work, so they need an eval in place before any traffic moves.

1. Serve a high-volume task from a custom model

When one task, such as ticket triage, document extraction or call summarization, makes up a large share of your API calls, a model trained for that task can serve it at a fraction of frontier cost. A model trained for one job carries only the capability that job needs, so it costs less per token to serve and answers faster.

In our QMSum study, a 1.2B model trained for query-focused meeting summarization had an estimated serving cost of $0.00053 per query on a rented cloud GPU, against $0.104 for Claude Opus 5 and $0.0130 for GPT-5.6 Luna at synchronous list prices. It answered in 0.39 seconds on a single GPU against 9.0 seconds for Claude Opus 5 over the API. The estimate excludes training cost, and quality is task-specific, which is why the switch below is gated on your own evals.

This is the highest-effort technique on the list, because it needs a stable task, enough real traffic to learn from, an eval that defines the bar, and a safe way to move traffic. The section on when a task deserves its own model covers the decision.

2. Right-size the model per task

The price spread between tiers is the largest lever most teams have not pulled. Here are the October 2026 list prices per million tokens for the current lineups.

Within each lineup the top tier costs 10x to 100x the bottom tier per output token, so a task that passes on a lower tier is a large saving for a one-line change.

The method is an eval: a few hundred real inputs with known good outputs and a scoring rule. Run every candidate tier against it, pick the cheapest one that meets the bar, and rerun the eval whenever a provider ships a new model, because the cheapest passing tier moves with every release.

3. Route requests, or cascade them

Routing sends each request to a model chosen up front, by task label or by a classifier. A cascade tries the cheap model first and escalates to a stronger one when the answer fails validation or a confidence check.

The cascade arithmetic is simple. If the cheap model costs 5% of the strong one and escalates 30% of requests, the expected cost is 0.05 + 0.30 = 0.35 of sending everything to the strong model, a 65% saving. The saving disappears as the escalation rate climbs, so measure it per task type.

Route on explicit task labels first, such as a header or one endpoint per workflow step, and add a learned router only once you have labeled traffic to train and measure it on. A misrouted request costs quality, and difficulty prediction is a hard problem in its own right. Our earlier pieces on the SLM-first 80/20 routing pattern and choosing between fine-tuned, API and RAG paths go deeper on architecture. For a side-by-side comparison of routing strategies, and how to move traffic to a cheaper model safely, see LLM routing strategies to cut frontier API costs.

4. Batch everything that can wait

As of October 2026, OpenAI and Anthropic both price batch processing at 50% of the standard rate for input and output tokens, and OpenAI lists its Flex tier at the same rates as Batch for the GPT-6 lineup. Evaluation runs, backfills, nightly summaries, enrichment jobs and classification of backlogs all qualify.

The engineering cost is asynchronous plumbing: write requests as a file with stable custom IDs, submit, poll or take a webhook, then reconcile results and retry partial failures. Make each request idempotent so a retry never double-writes a result.

5. Cache the stable prefix

Prompt caching stores the processed prefix of a prompt so later API calls that share it are billed at the cache rate. As of October 2026, Anthropic bills cache hits at 0.1x the base input price on most Claude models, with cache writes at 1.25x for a 5-minute cache and 2x for a 1-hour cache. OpenAI lists cached input for GPT-6 Astra and GPT-6 Luna at a tenth of standard input, with cache writes at 1.25x.

At those multipliers caching pays from the first reuse: two uncached API calls cost 2.0 units of input, while one write plus one hit costs 1.25 + 0.1 = 1.35.

To get high hit rates:

• Put stable content first: system prompt, tool definitions, reference documents. Put the variable content last.
• Keep timestamps, request IDs and per-user data out of the prefix, since any change invalidates everything after it.
• Keep tool definitions in a fixed order.
• Check the minimum cacheable length. Anthropic's documentation lists 512 tokens for Claude Opus 5, so short prompts never cache.
• Track the cache hit rate per task type, since a refactor that reorders a prompt can silently drop it to zero.

Caching changes the price of input and leaves output untouched. When caching alone stops being enough, our piece on moving from prompt caching to fine-tuning covers the next step.

6. Cap output and use structured output

Output tokens cost five times as much as input tokens on both providers' current lineups: Claude Opus 5 at $5 and $25, GPT-6 Luna at $0.10 and $0.50 per million. Set a maximum output length on every API call. Ask for a JSON schema with short fields and enums when the consumer is code. Remove instructions such as "explain your reasoning" from production prompts when only the final label is used.

7. Control reasoning effort

OpenAI's reasoning guide states that reasoning tokens are billed as output tokens even though the API never returns them, and Anthropic reports thinking tokens as part of billed output. Run your eval at each reasoning effort or thinking budget and use the lowest setting that passes.

OpenAI's max_output_tokens caps reasoning and visible output together, and its guide warns that a response can stop before any visible output while still billing the input and reasoning tokens. Size the cap with headroom, then monitor incomplete responses.

8. Trim context and tune retrieval

Every token in the prompt is billed on every API call. For retrieval pipelines, retrieve broadly, rerank, then send only the top few chunks. Deduplicate overlapping chunks and strip boilerplate such as navigation text, headers and markup from source documents before indexing.

Long context windows make this easy to neglect. Anthropic bills its full 1M token context at standard per-token rates as of October 2026, so a 900,000-token request costs a hundred times a 9,000-token one. Longer context also degrades answer quality well before the window is full, as covered in the effective context length problem.

9. Compact conversation and agent history

Chat products and agents resend the conversation on every turn. If each turn adds a similar number of tokens, total input over a conversation grows with the square of its length, because turn n resends all n turns before it.

Keep a sliding window of recent turns, summarize older turns into a short state block, and drop tool outputs once the agent has used them. Caching lowers the price of the resent history while the token count keeps growing, so compaction is still worth doing on long sessions.

10. Limit agent steps and tool output

Agent runs vary widely in length, and the long tail drives the bill. Cap steps per run, set a token budget per run, and end a run that hits either limit with a handoff. Truncate tool outputs at the tool: return the matching rows, the relevant function or the top results, never the entire file or table. Log tokens per successful run alongside success rate, so a prompt change that makes runs longer is visible the day it ships. Our guide to agentic AI cost optimization applies these fixes to a full agent run, step by step.

11. Cache whole responses for repeated requests

When the same request arrives repeatedly, such as an FAQ answer or a classification of a known input, serve the stored response and skip the API call entirely. Key the cache on the normalized input, the model and the prompt version, and invalidate it whenever either version changes.

Semantic caching, which matches near-duplicate requests by embedding similarity, extends the hit rate at the cost of occasionally serving an answer to a different question. Use a strict similarity threshold and limit it to workloads where a near match is an acceptable answer.

12. Compress prompts

Shorter instructions and fewer few-shot examples reduce input tokens on every API call. The saving is real but small, and smaller still when the prefix is already cached at a tenth of the price. Do it after caching, and rerun the eval after every cut, since few-shot examples often carry more of the quality than their length suggests.

When should a high-volume task get its own model?

A task deserves its own model when it runs at high and steady volume, its definition of a good output is stable, and an eval can score it automatically. Compare against the frontier price you would actually pay, including batch discounts for asynchronous work, since that is the real alternative. The break-even arithmetic is worked through in when a custom model beats a frontier API on cost.

When most of these signals are present, the remaining question is the operating cost of the model over time. A custom model is a pipeline to run: collecting and curating training data, training, evaluating every candidate, serving it, watching for drift, and retraining as the task changes. The training itself is the smaller part of that work.

The adaptive model endpoint runs that pipeline for you. An adaptive model endpoint is one API endpoint for one of your tasks: frontier models answer at the start, a custom model trained on your requests and your team's feedback takes over once it passes your evals, and hard cases stay on frontier models.

In practice:

1. You point the task at the endpoint by changing one model name. Frontier models serve every request at the start, so nothing waits on training.
2. Each new version of the custom model runs in the background on real requests and is scored against your evals.
3. Nothing switches without your sign-off. Once a version passes, a growing share of requests moves to it, and hard cases stay on frontier models.
4. We monitor the model and retrain it as your tasks change, and every new version goes through the same gate.

The model runs Ertas-managed, in your cloud or on-premise, and you own it. The endpoint is currently invitation only, and we are developing a more streamlined version for wider access.

Frequently Asked Questions

What is the fastest way to reduce LLM costs?

Batch processing and prompt caching, because both are configuration changes with no effect on output quality. As of October 2026, batch processing costs 50% less at OpenAI and Anthropic, and cached input is billed at a tenth of the standard input price or less on most current models.

Does prompt caching reduce output costs?

Prompt caching lowers the price of repeated input tokens only. Output tokens, including reasoning tokens, are billed at the full output rate, so pair caching with output caps and a reasoning effort setting tested against your eval.

Is a cheaper model always cheaper per task?

A cheaper model is cheaper per token, but per task it depends on retries and escalations. A model that needs repeated attempts, or escalates most of its requests to a stronger model, can lose its advantage. Measure cost per successful task, including every retry and escalation.

How much does a custom model cost to serve compared with a frontier API?

In our QMSum study, a 1.2B model trained for meeting summarization had an estimated serving cost of $0.00053 per query, against $0.104 for Claude Opus 5 and $0.0130 for GPT-5.6 Luna at synchronous list prices. That estimate excludes training cost. Your numbers depend on model size, traffic volume and how fully the serving hardware is used.

Can these LLM cost optimization techniques be combined?

Most of them stack, because they act on different terms of the cost equation. Caching lowers the price of input, output caps cut output tokens, and right-sizing lowers the price of both, so the savings multiply on the same API call.

Do I need a machine learning team to run a custom model?

Running one yourself takes data curation, training, evaluation, serving and retraining, which is a real team commitment. With an adaptive model endpoint, Ertas runs that pipeline, your engineers change one model name, and your team approves each switch against its own evals.

The Short Version

Measure cost per successful task before you optimize anything. Take the configuration wins first: batch what can wait, cache stable prefixes, cap output and reasoning. Then move each task to the cheapest model that passes its eval, and give your highest-volume task a model trained for it.

If one task dominates your token bill, book a call and we will scope it with you.

Sources: OpenAI API pricing, Anthropic pricing, Anthropic prompt caching documentation and OpenAI reasoning guide. Prices verified 6 October 2026.