Set max_tokens Before It Sets Your Bill

You wake up to an API bill three times last month's figure. No new users. No deployment overnight — just a handful of longer-than-usual sessions from your pilot cohort. The culprit is a missing max_tokens cap: without it, the model writes back as many tokens as it wants on every call. This piece shows you exactly where to add that one parameter and how to size it correctly before your next pilot call.

Why did my API bill double without my product changing?

LLM APIs charge per token consumed — input tokens you send plus output tokens the model generates. A missing max_tokens cap means the model decides how long to write, and a single verbose session can cost fifty times more than a short one. User behaviour, not a pricing change, is usually the cause of sudden bill spikes in vibe-coded MVPs.

When you launched, your test sessions were short. Your pilot users are not testing — they are working. They paste longer documents, ask follow-up questions, and trigger the model to generate longer responses. Nothing changed in your code. Everything changed in how real people use it. The bill reflects that difference, amplified by every call that ran uncapped.

What is max_tokens and why is it the one setting that actually caps spend?

max_tokens is the hard ceiling on how many words the model can write back per request. The API enforces it before the response completes, so cost cannot exceed the ceiling — not as an approximation, but as a guarantee evaluated mid-generation.

Prompt engineering and model-switching are the two alternatives founders usually try first. Both are reasonable, but neither bounds spend the way max_tokens does. Rewriting your prompt to ask for shorter answers shifts the average response length down; it does not prevent a long answer. Switching from a premium model to a cheaper one reduces cost per token; it does not prevent a high token count. max_tokens is the only lever that converts unpredictable API spend into a number you can write into a budget and defend to an investor.

So what for you? Every other optimization is useful later. This one is the prerequisite.

Tokens are not words — here's the conversion that matters for your budget

A practical rule of thumb: 100 tokens is roughly 75 English words. That ratio lets you translate your product's expected response length into a token budget without an engineer in the room.

If your product should return a two-sentence confirmation, that is around 30 words — approximately 40 tokens. If it should return a structured summary of a meeting, that might run 400 words — approximately 530 tokens. You do not need to know how tokenizers work. You need to look at a typical good response from your product, count the words roughly, multiply by 1.33, and you have a starting token estimate. That number becomes your upper anchor when you calibrate the ceiling.

Setting max_tokens too low breaks the product — how do you find the safe floor?

A ceiling set below the minimum useful response length causes truncated outputs — sentences that cut off mid-thought, lists that end without a final item, answers that stop before the actual recommendation. Users experience this as a broken product, not a cost-control decision.

The calibration method is straightforward. Log ten to twenty real user interactions from your pilot. Look at the response lengths the model actually returned when it was doing useful work. Find the 90th-percentile token count — the length that nine out of ten good responses stayed under. Set max_tokens 20 percent above that figure. That buffer absorbs natural variation without giving the model room to run freely on an edge case. Review the number again after your next cohort adds meaningful usage volume.

How do I actually add max_tokens to my existing vibe-coded app?

Locate the API call in your codebase — in a vibe-coded MVP this is almost always a single function or file, often named something like callOpenAI, generateResponse, or fetchCompletion. Add max_tokens as a named parameter in the request object alongside the existing model and prompt (or messages) fields.

To confirm the cap is active, run a test call that is designed to push against the ceiling — a prompt that would normally produce a long response. Check the API response's usage object. If finish_reason reads as "length", the cap fired and is working. If it reads as "stop", the model finished naturally within your ceiling, which is also correct. What you are ruling out is a missing parameter or a typo that left the ceiling unset.

One practical note: vibe-coded apps often have the API call duplicated across features — a chat function here, a summary function there. Search the codebase for the API client import and confirm max_tokens appears in every call site, not just the first one you find.

What does the change look like in code — without the jargon?

Think of the API call as a request object — a set of named fields you hand to the API. Before the change, that object contains three fields: which model to use, the system instructions, and the user's message. The API receives those fields, generates a response, and stops when it decides the answer is complete.

After the change, the object contains four fields. The first three are unchanged. The fourth — max_tokens, set to your calibrated number — tells the API to stop generating at that ceiling regardless of whether the model considers the answer complete. The response comes back faster on truncated calls, and the usage object now shows both tokens used and the reason generation stopped.

You do not need to read code fluently to verify this was done correctly. Ask your freelancer or future engineer to show you the request object before and after — four fields rather than three, with max_tokens among them — and to run the finish_reason test described in the previous section. If both conditions are true, the cap is in place.

Should max_tokens be the same for every feature, or does each screen need its own cap?

Different product surfaces have different natural response lengths. A one-line search suggestion needs roughly 20 tokens. A structured report draft might need 600. A single global cap set to your highest-use case over-spends on every lightweight call — you pay for 600-token headroom on every autocomplete suggestion.

For a pre-seed product, the right balance is two or three token tiers, not a unique cap per feature. Group your features by response type: short outputs (suggestions, labels, classifications), medium outputs (summaries, step-by-step answers), long outputs (drafts, reports, detailed analyses). Assign one cap per tier. That structure gives you meaningful cost control without the maintenance overhead of managing a different number for every screen — which is complexity your codebase does not need before your Series A.

What else should I watch once max_tokens is in place?

Two levers close the remaining cost gaps. The first is prompt compression — removing redundant context from the input before the request is sent. Input tokens are billed before max_tokens even applies, so a bloated system prompt or a copy-pasted document you append to every call adds cost the output ceiling cannot touch. Trimming 30 percent of unnecessary input context often reduces total bill by a meaningful amount on input-heavy features.

The second is usage logging — recording tokens_used per call so you can see cost at the feature level, not just as a monthly aggregate. When a specific feature starts consuming a disproportionate share of your token budget, you want to know within days, not at the end of the billing cycle. A simple log that records feature name, input tokens, output tokens, and timestamp gives you that visibility without requiring a third-party tool.

One config line now, or a billing crisis when your pilot goes live

An unbounded API is a credibility risk in due diligence. It signals that cost is not controlled — that a spike in usage, a malformed user input, or a single long session can move your financial model in ways you cannot predict or explain. Investors and pilots read that as operational immaturity, not just a technical detail.

A bounded, logged API spend reads differently. It shows that you understand your unit economics at the call level, that you can forecast infrastructure cost as you scale DAU/WAU, and that the product has been hardened past the demo stage. That is the distinction between a vibe-coded prototype and a product someone can write a check for.

max_tokens is one parameter in one request object — a change a freelancer can make in an afternoon and you can verify by reading a single field in the API response. The audit trail it creates, and the cost predictability it establishes, do more for your next pilot call than any feature you could ship in the same time.

Set it today. Size it from real usage data. Review it after your next cohort. That is the sequence — and it starts before any engineer touches the rest of your codebase.