LLM Cost Control: 4 Tactics That Cut Our API Bill in Half
Published 2026-09-07 · Updated 2026-09-07 · 7 min read · AI 工作術 (FreeCo Co., Ltd.)
LLM cost control from a team that pays its own API bills: model tiering, prompt caching, prompt slimming, and batch processing — with real production numbers.
Most articles about LLM cost control are written by people who have never paid the bill. We have. We run an AI tools platform with more than a dozen production features, an AI video-clipping engine, and AI-powered ad tooling — every API call comes out of our own pocket. Four tactics do almost all of the work: tier your models, cache aggressively, put your prompts on a diet, and batch everything a user isn't actively waiting for. Applied together, they routinely cut an API bill by half or more without touching output quality.
One framing before the tactics: the goal isn't to be cheap. The goal is unit economics that can survive a subscription business. If you lose money every time a user touches a feature, building a better product just means bleeding faster. Cost control is what turns an AI feature from a demo into a business.
None of this requires exotic infrastructure. All four tactics are plain engineering work, and you can ship the first one this week.
Tier your models — stop shooting mosquitoes with a cannon
The single biggest source of waste is sending every task to a flagship model. In a real pipeline, roughly 80% of the steps don't need the smartest model available. Our video-clipping engine is a textbook case. Transcript segmentation, language detection, formatting cleanup — that's mechanical work, and the cheapest lightweight model handles it perfectly. The flagship only gets called for the one step that genuinely needs judgment: picking the most compelling 30 seconds out of an hour of footage. Same pipeline, before and after tiering, the cost difference approaches an order of magnitude.
Here's the method that actually works: build the entire pipeline on the flagship model first, then downgrade each step one at a time until accuracy drops — and step back up one tier. Always work top-down, never bottom-up. If you start with cheap models and a step fails, you can't tell whether the task is impossible or the model is just too weak. Start smart, then get frugal.
Within any provider's lineup, the price gap between the cheapest and most expensive tier is typically 10x or more per token — check the official pricing pages, because the exact numbers move every few months. If you're not sure which family of models fits which job, our ChatGPT vs Claude vs Gemini comparison covers how we think about it.
Cache aggressively — never pay twice for the same tokens
There are two caching layers, and they work at different levels:
| Layer | What it saves | Effort to implement |
|---|---|---|
| Prompt caching | Repeated prefixes: system prompts, knowledge-base context | A few extra API parameters |
| Result caching | Entire calls for repeated inputs | A hash key and a datastore |
Prompt caching is supported by every major provider now. The parts of your prompt that are identical on every call — the system prompt, your knowledge-base excerpts, your few-shot examples — get billed at a fraction of the normal input price once cached, often around a tenth (check your provider's pricing page for current rates). If your application carries a long system prompt, this one change alone can cut a meaningful chunk of your bill, and it's usually just a few parameters.
Result caching means never asking the same question twice. Our marketing tools see a lot of "many users query the same trending topic" traffic, so we hash the input, store the output, and serve the second request straight from cache. Hit rates vary wildly by product, but even a 20% hit rate is 20% of your bill picked up off the floor for free.
Put your prompts on a diet — tokens are billed by the piece
Most production prompts grow by accretion: every time a bug shows up, someone bolts on another paragraph of instructions. Six months later you have a 3,000-token monster where half the instructions repeat each other and some actively contradict each other. Our habit is a quarterly prompt cleanup: delete the duplicates, replace verbose examples with tight ones, and use structured output constraints instead of paragraphs of prose describing the format you want. The surprise every time: accuracy usually goes up after the diet. Models are like people — the cleaner the instructions, the better the execution. If you want the fundamentals of writing lean prompts, start with prompt engineering basics.
Your invoice is the best code review: every wasted token comes back to find you at the end of the month.
Don't forget the output side — it's usually the expensive side. Output tokens typically cost several times more than input tokens. Telling the model "answer directly, skip the reasoning narration" and replacing free-form prose with tight JSON fields saves tokens on exactly the end where they cost the most.
Batch anything a user isn't waiting for
Most providers offer a batch API at a steep discount — commonly around half price — with the trade-off that results can take hours. The whole game is sorting your traffic into two buckets: "a user is staring at a spinner" versus "tomorrow morning is fine."
In our stack, compliance scans of product copy, daily analytics summaries, and pre-generating content variants all run through batch. Only features where a user is actively waiting hit the real-time API. Moving the non-urgent traffic to batch is a permanent ~50% discount on that portion of your bill, and your users literally cannot tell the difference.
Measure first, then optimize
One piece of unglamorous groundwork makes all four tactics work: log every API call with the feature name, the model used, and the input/output token counts. Without that, you're optimizing blind. With it, your monthly report tells you exactly which feature and which pipeline step is eating the money.
That report once caught a feature consuming about 30% of our entire API spend — with almost no users touching it. We killed the feature. Best cost optimization we ever shipped, and no engineering required.
The same discipline applies to quality: every time you downgrade a model or shrink a prompt, you need a way to verify accuracy didn't quietly fall off a cliff. That's an evaluation problem, and we wrote up our whole approach in how we evaluate LLM output. Cost work without eval work is just gambling with better vibes.
Bottom line: LLM cost isn't dark magic. It's four pieces of ordinary engineering — tier, cache, diet, batch — plus measurement to aim them. If your AI feature's bill is starting to sting, you now have the checklist. Work it top to bottom; the first two tactics alone usually pay for the week you spend on them.
FAQ
Q: How much can I realistically cut my LLM API costs?
In our experience, 50–90% is a realistic range for a pipeline that has never been optimized, mostly from model tiering and caching. The exact number depends on how much of your traffic is repetitive (caching) and how much is non-urgent (batching). A pipeline that's already tiered will see smaller but still meaningful gains from prompt slimming.
Q: Which cost tactic should I implement first?
Start with measurement — log feature, model, and token counts per call, because you can't fix what you can't see. Then do model tiering, since it usually has the biggest payoff, followed by prompt caching, which is often just a few API parameters. Batching and prompt cleanup come after.
Q: Won't cheaper models hurt my output quality?
Not if you downgrade the right steps. Mechanical work like formatting, extraction, and classification rarely needs a flagship model. The key is to downgrade one step at a time with an evaluation set watching, and step back up the moment accuracy drops.
Q: Is prompt caching the same as caching responses?
No — they're different layers. Prompt caching discounts the repeated prefix of your prompt (system prompt, context) on every call, while response caching skips the API call entirely when the same input shows up again. Use both: they stack, and neither interferes with the other.