When people think about what a model call costs, they picture the answer it produces. But you pay for what you send as well as what you get back, and on a large and growing share of applications, what you send is the bigger number. Input tokens are the quiet driver behind many bills that look higher than they should.
Why input is easy to overlook
Output tokens are visible. You see the response in your product and can feel its length. Input tokens are invisible in the same way plumbing is invisible: the system prompt, the retrieved context, the conversation history, the examples, all of it is sent on every call and none of it appears in the user-facing output. Because you do not see it, you do not weight it, and estimates built on response length miss it entirely.
Where input tokens accumulate
Three patterns drive input cost. Conversation history is the first: every turn resends the accumulated exchange, so input grows across a session. Retrieval is the second: features that fetch documents or search results to ground a response send all of that context as input, often thousands of tokens per call. System prompts are the third: instructions, formatting rules, and examples that ride along on every single call, permanently.
The compounding effect
These patterns stack. A retrieval-augmented chat feature pays for a growing conversation history plus fetched documents plus a system prompt, all as input, on every turn. Individually each is modest. Together they can make input several times larger than output, which means a bill you thought was driven by generation is actually driven by everything you send before generation even starts.
Controlling input cost
Each source has a control. Cap conversation history with a token budget, and summarize older turns instead of resending them raw once the budget is hit. Tune retrieval to send the most relevant context rather than the most context, since more retrieved text is not always better and always costs more. Compress system prompts to what is genuinely required on every call, moving rare edge cases into application logic instead of prompt instructions.
Measuring the split
You cannot manage what you do not measure. Track the ratio of input tokens to output tokens per feature. A feature where input is many times output is a candidate for every control above. A feature where output dominates is fine as is. That single ratio tells you where input optimization will pay off and where it will not.
The lesson is simple and frequently missed: your bill is a function of total tokens, and input is often most of them. A cost model that only counts the answer will always run low, and a bill driven by input will always look mysterious until you start measuring the half of the equation you were not watching.