Ask most teams what an AI feature costs and they will quote the model API bill. That is the visible part and rarely the whole story. The true cost of an AI feature includes several components that never show up on the model invoice, and tracking all of them is what tells you whether the feature is actually worth running.
The components of true cost
An AI feature's real cost has more than one line. The model calls, including retries and discarded generations. The tool calls the feature triggers, like searches and lookups, which often do not register as model calls. Any retrieval or embedding cost if the feature uses context. And the development and eval cost to build and maintain it. Track only the first and you understate the rest.
Count the calls that produced nothing
Retries and rejected generations cost money and produce no user-facing output. A feature that generates, evaluates, and regenerates pays for every attempt. Track the ratio of billed calls to successful outcomes, and the gap is the true cost of the feature's inefficiency.
Include tool calls in the feature cost
Modern AI features rarely make a single clean call. They search, query databases, and run code, each with its own cost. Attribute those tool calls to the feature that triggered them, so the feature's cost reflects everything it does, not just its final model response.
Track it as cost per outcome
Total feature cost is only meaningful against what the feature delivers. Divide the true cost by the outcomes the feature produced to get cost per outcome. A feature that looks expensive in absolute terms can be cheap per valuable result, and one that looks cheap can be wasteful if most of its output goes unused.
Decide with the full number
The point of tracking true cost is the keep-or-cut decision. A feature justifies itself when its cost per outcome is low relative to the value of that outcome. Tracking only the model bill makes cheap features look expensive and hides the real cost of inefficient ones, which leads to the wrong decisions.
The model API bill is the tip of an AI feature's cost. Track the retries, the tool calls, the retrieval, and the build cost, express it as cost per outcome, and you will finally know which features are worth keeping.