What AI costs to serve one customer

Software got used to a comfortable fact: marginal cost was close to zero. One more customer on a SaaS product cost you some storage and a fraction of a server. Gross margins settled in the high eighties and unit economics discipline quietly atrophied, because there was not much to be disciplined about.

AI reverses that. Every customer action that touches a model costs real money, and the amount varies enormously between customers on the same contract.

Which means a question that has not mattered much for a decade suddenly matters a great deal: what does it cost you to serve one account?

Why this is a gross margin question

If AI serves your product to customers, it is a variable cost that scales with usage. That is the definition of cost of goods sold, and it belongs in gross margin.

Filed as a general technology cost in operating expense, it distorts two things at once. Gross margin looks better than it is, and operating expense looks worse. As the AI line grows, that distortion grows with it, and the reported margin drifts further from the real one.

There is a specific failure mode here worth naming. You can grow revenue into negative marginal contribution and not notice for several quarters. Revenue goes up, the AI line goes up, the two look correlated and reasonable. The fact that your five largest accounts are consuming eleven times the AI of an average account on the same pricing does not appear anywhere until gross margin moves, and by then it is a quarter's worth of a problem.

The distribution is the whole story

Averages are actively misleading here. Within any segment, AI consumption varies by an order of magnitude or more.

A handful of accounts routinely account for a disproportionate share, and there is usually a specific reason: one integration calling your API in a loop, one team using an AI feature as a batch processor in a way nobody anticipated, one customer whose data volume is simply much larger than the model your pricing assumed.

The average tells you none of that. The distribution tells you everything, including which accounts to look at.

This is also where the most common mistake happens. Companies compute an average cost per customer, find it comfortable against average revenue, and conclude the economics work. They work on average. They do not work at the top of the distribution, and the top of the distribution is where the growth usually is.

Self-serve is often the worst ratio

Enterprise accounts tend to look bad in absolute terms and fine as a ratio. Self-serve tends to be the reverse, and it is frequently the segment where the economics are genuinely uncomfortable.

The reason is that self-serve pricing was almost always set before anyone knew the marginal cost, using intuitions carried over from zero-marginal-cost software. Unlimited usage on a low monthly price is a reasonable offer when serving a customer costs nothing. It is a different proposition when a heavy user can consume several times their subscription in model calls.

This does not mean self-serve is wrong. It means the pricing needs to reflect a cost structure that did not exist when the pricing was designed, and you cannot do that without the number.

How to actually get the number

There are three levels of precision, and it is worth being honest about which one you are at.

Measured. Requests carry a customer or account identifier, so model spend attributes directly. This is the most accurate and it is usually a small engineering change: pass an identifier through to the provider call as metadata. If you are going to do one thing after reading this, do this one.

Apportioned. No customer identifier, but you know which workflows serve customers and you have a usage driver: seats, sessions, API calls, documents processed. Apportion the workflow's cost across accounts by that driver. Less precise, and the important discipline is to label it as apportioned so nobody treats it as measured. It is usually accurate enough to identify outliers, which is most of the value.

Segment level. Neither of the above, but you can separate customer-serving workflows from internal and non-production use, and you know roughly which segments use which features. Enough to show that self-serve carries a materially worse ratio than enterprise. Not enough to act on an individual account.

Start wherever you are. The segment view usually produces the first uncomfortable finding on its own, and that finding is generally enough to justify adding the identifier.

Keep platform costs out of it

One methodological point that causes a lot of confusion.

Some AI cost is genuinely marginal: it happens because a specific customer did something. Some is fixed platform overhead: an endpoint you keep warm, a model you host regardless of traffic, evaluation infrastructure.

Mixing them produces a per-customer number that is wrong in both directions depending on volume. At low volume, fixed costs spread over few customers make each look catastrophically expensive. At high volume they disappear and the number looks better than the marginal reality.

Report marginal cost per customer, and report platform cost separately as what it is. The two answer different questions and neither is improved by combining them.

What you do with it

Four things, roughly in order of how quickly they pay.

Find the outliers and understand them. Not to penalize anyone. Usually there is a specific integration or usage pattern behind an outlier, and sometimes the right answer is a product fix rather than a commercial one. A customer calling your API inefficiently is often a customer who would prefer not to.

Set rate limits that reflect cost. Most usage limits in AI products were set arbitrarily, because nobody had a cost basis when they were written. Limits informed by actual marginal cost are both more generous where they can afford to be and more protective where they need to be.

Price and package on evidence. Cost per feature tells you which capabilities belong in the base tier, which are premium, and which need metering. It also tells you what an unlimited-usage promise actually exposes you to at the tail, which is a number worth knowing before you make the promise.

Report it in gross margin. Once AI is where it belongs in the P&L, the business gets an honest margin figure and the trend becomes visible. That is the number that eventually determines what the company is worth, and reporting it correctly is not optional for long.

The uncomfortable version

If you sell an AI-backed product and you cannot currently say what AI costs to serve one customer, the honest position is that you do not know your gross margin.

That is a normal place to be right now. The tooling did not exist, the cost was not material until recently, and the discipline was not needed for a decade. None of that makes it a comfortable thing to say in a board meeting, and it becomes less comfortable every quarter the number grows.

The fix starts with one small change: pass a customer identifier through to your model calls. Everything else builds on that.