Direct answer: The best single measure of whether your team's AI usage is good is how much of your spend is buying the same tokens twice. Healthy teams cache repeated context, succeed on the first try, and reuse each other's workflows, so their cost per unit of real work falls even as usage grows. Unhealthy teams pay again for every retry and every resent repository, so their bill grows faster than their output. To tell the two apart, measure the ratio of productive tokens to repeated ones, a number total spend hides inside a single line item.
Most engineering leaders I talk to judge their team's AI usage by one number, and it is close to useless for the job. The monthly bill goes up, they worry the team is wasteful. It goes down, they assume things are healthy. Both readings are usually wrong, and walking through why they are wrong tells you what to measure instead.
Here is a claim that sounds backwards until you sit with it: a rising AI bill is often the sign of a healthy team, and a flat one is often the sign of a stalled one. The teams doing the most ambitious work with agents spend more, because they are attempting more. What separates good usage from bad is the share of the bill that pays for work already purchased, the retries and the resent context you are buying a second and third time. So I judge team AI usage as a portfolio of workflows, where each workflow is a repeated way of getting a task done with a model, and I measure the portfolio on three things, none of which is total spend. The reason this matters is that the same dollar figure can describe a team pulling ahead and a team spinning in place, and the three signals below are what let you read which one you are looking at.
The first is first-try success rate. When a workflow calls a model, the task should complete without the engineer having to re-run it, re-prompt it, or throw the result away. A workflow that lands on the first try 90 percent of the time behaves very differently from one that lands 40 percent of the time, even when the second looks cheaper per call. The 40 percent workflow quietly pays for three attempts to get one usable answer, and those retries hide inside the same vendor line item where nobody counts them. A tool like Datadog will show you latency and error rates on the request. It will not show you whether the task the engineer wanted actually landed, because that is a semantic outcome and not an HTTP status.
The second is reuse. When one engineer builds a workflow that works, how many other people on the team adopt it. This is the single most underrated signal in AI usage, because it is the difference between a team where one person figured out how to work with Claude Code and a team where everyone did. Reuse is how a good pattern turns into team capability. If your best workflows are stuck on the machine of the person who built them, your usage is worse than it looks, no matter what the bill says.
The third is repeated context. Agents and coding tools resend the same material constantly. The same system prompt, the same repository, the same retrieved documents, request after request. When that context is cached, you pay for it once and reuse it. When it is not, you pay for it every single time, and the cost compounds silently because it never shows up as its own line. Measuring the ratio of cached to resent context tells you how much of your spend is buying the same tokens over and over.
These three share a property that makes them worth more than the bill: every one of them is behavioral, and behavior is what a leader can actually move. The price of a model token sits outside your control. How often a task succeeds, how widely a good pattern spreads, and how much context gets reused all sit inside it. Spend measures the thing you cannot change and buries the three things you can, which is why it makes such a poor dashboard for a decision you are trying to influence.
One trap deserves a warning before you go measure any of this. Once you can see per-workflow and per-person numbers, the pull toward ranking people is strong. Cheapest engineer wins, most expensive engineer gets a talking-to. Following that pull will cost you more than any waste it uncovers. The engineer running the most expensive workflows is often the one attempting the hardest problems in the company. The one running the cheapest has sometimes just stopped reaching for AI on anything difficult. Cost read without context misleads more reliably than no data at all, because it arrives looking precise while pointing in the wrong direction.
So the honest way to answer "is my team's AI usage good" is to watch first-try success climb, good workflows spread, and repeated context shrink across a few months. When those three move in the right direction, the usage is good and improving, and the bill drifts down behind them without being chased. Watch only the bill and you are reading the last page of the story, guessing at the plot that produced it.
We built Oberhahn to surface exactly these three signals from the telemetry your AI tools already produce, so a manager can see workflow health instead of inferring it from an invoice. You can walk a live demo with sample data before connecting anything.
Frequently Asked Questions
How do I know if my team's AI usage is good?
Track first-try success rate, how widely good workflows are reused, and how much repeated context is cached versus resent. Healthy usage means those three are improving over time, and cost per unit of real work falls even as total usage grows. Total spend is a poor measure on its own because it hides retries and repeated context inside vendor line items.
Is higher AI spend a sign my team is using it well?
It can be. Higher spend often means the team is attempting more ambitious work, and the strongest teams frequently spend more. What separates good usage from bad is how much of the spend is buying work you have already paid for, through retries and resent context. Per-workflow success and reuse are what let you interpret the number.
Should I rank engineers by their AI cost?
No. The most expensive engineer may be doing the most ambitious work, and the cheapest may have stopped using AI for hard problems. Cost without context misleads. Judge workflows, not people.
What is first-try success rate?
The share of times a workflow completes the task without a retry, re-prompt, or discarded result. It is a better efficiency signal than cost per call, because a cheap call that fails and reruns three times is not cheap.