Tokens Are the Next Spice
Last quarter, my team built a spreadsheet to track token spend across five active projects. The plan was simple: log the model, monthly API cost, and projected automation savings. Payback timelines. ROI columns. It started as hygiene and turned into a reckoning.
Two projects were already underwater. One had burned through three months of projected savings in six weeks. Nobody had done anything wrong. The models were working. The prompts were reasonable. The problem was the assumption baked into every row: that the pricing we’d seen at kick-off was the pricing we’d be paying at scale. It wasn’t. It had never been meant to be.
That spreadsheet is what made the Token Price Index feel, when we found it, less like a report and more like confirmation.
Every commodity market needs a benchmark index to anchor sanity. Equities track the S&P 500. Crude oil prices reference Brent. Maritime shipping firms watch the Baltic Dirty Tanker Index to measure the raw friction of moving physical goods across the world. For years, the cloud compute market for text generation had no equivalent — no independent price anchor, no objective standard of measure. Teams spent freely, treating API usage as an operational footnote rather than a line item worth managing.
That changed when Harasha Muddy Patla at Monitor Deloitte introduced the Token Price Index. The benchmark arrived not as a curiosity but as a correction — a forcing function for engineering and finance teams that had been operating without a shared reality on what compute actually costs. The free lunch, to be direct, has expired.
A Market Built on Subsidy
The current state of cloud compute pricing did not emerge from normal market forces. It was engineered. Major cloud providers and model vendors spent billions deliberately pricing below cost to acquire developers, lock in workloads, and establish distribution dominance before market rates took hold. OpenAI reported roughly $5 billion in losses last year alone. Google, Anthropic, and others followed comparable paths at their own scale. The logic was venture-backed land grab: accept short-term losses, establish dependency, then normalize pricing once customers have no easy exits.
That normalization is underway. The Token Price Index makes it visible for the first time with something approaching objectivity. Teams that built cost models around current pricing are now discovering those models describe a market that no longer exists.
The Volume Multiplier
The launch of Gemini 3.5 Flash illustrates exactly how the fiscal math diverges from the marketing narrative. Google introduced the model as a high-performance tier — faster, more capable, with a stated price of $1.50 per million input tokens and $9.00 per million output tokens. On the surface, that is a three-times increase over the previous generation’s pricing. Manageable, teams assumed.
The reality measured by Artificial Analysis told a different story. Running their standard benchmark suite against Gemini 3.5 Flash costs 5.5 times more than running the same suite on Gemini 3 Flash — and 75% more expensive than the prior generation Pro model. The full picture across the model family:
| Model | Input / 1M tokens | Output / 1M tokens | Benchmark Run Cost |
|---|---|---|---|
| Gemini 3.1 Flash-Lite Preview | $0.25 | $1.50 | $93.60 |
| Gemini 3 Flash Preview | $0.50 | $3.00 | $278.26 |
| Gemini 3.1 Pro Preview | $2.00 | $12.00 | $892.28 |
| Gemini 3.5 Flash | $1.50 | $9.00 | $1,551.60 |
The table above shows benchmark execution cost — not theoretical per-token sticker prices. It reflects what a real workload actually costs to run.
The reason for the gap is structural, not incidental. Output token volume across these runs stays relatively flat at around 73 million tokens. The cost spike comes from the input side. More capable models require more back-and-forth exchange cycles on complex tasks, and each round trip compounds input cost. Teams watching sticker price per token miss this entirely. The real multiplier is loop depth and loop frequency.
This is where most engineering cost models break down. They price the token. They do not price the loop.
What Bad Architecture Costs
The financial exposure is not hypothetical. When teams cannot define what a bad result looks like — a flawed summary, a failed classification, a response that requires a follow-up — their software absorbs the cost of that ambiguity through repetition. Poorly scoped tasks generate more back-and-forth exchanges. Vague instructions produce outputs that require correction. Each correction is another round trip. Each round trip compounds the input volume that drives the actual bill.
Running these workloads through cloud endpoints introduces a second layer of compounding. Every exchange carries full conversation history — the entire accumulated context from the start of the session. The longer the session runs, the more history gets retransmitted on every exchange. This is not a quirk of one provider’s implementation. It is intrinsic to how stateless APIs handle context. The operational cost of a session scales faster than linearly with session length. Short, well-defined tasks are cheap. Long, ambiguous, multi-step workflows are expensive in ways that most teams only discover in their billing statements.
The Ownership Argument
The market logic eventually points in one direction: own the infrastructure or remain permanently exposed to vendor pricing decisions.
An API key is not infrastructure. It is a lease on someone else’s infrastructure, priced at whatever rate the vendor decides is sustainable for their business this quarter. That rate has been artificially low. The correction is structural and directional — prices will normalize toward actual cost, and teams whose architectures depend on cheap cloud compute will absorb that adjustment.
The alternative is local execution. High-volume, repeatable tasks — document processing, classification, structured extraction, code review — do not require cloud endpoints. They can run on commodity hardware or dedicated local silicon at fixed capital cost. The variable expense disappears. The vendor dependency disappears. The unpredictable billing cycle disappears.
The trade-off is real: upfront capital, operational overhead, and engineering time spent on infrastructure rather than product. But for teams running consistent, predictable workloads at scale, the math favors ownership. A server running local models is expensive once. An API running the same workload is expensive forever, at rates set by someone else.
The Token Price Index makes this calculation visible. It converts a vague intuition — “cloud compute feels expensive” — into a number teams can actually reason about. That number, compared against the amortized cost of owned hardware, tells you whether you have an infrastructure problem worth solving.
Most teams do. They just have not done the math yet.
References
- Monitor Deloitte — The CFO Guide to AI Token Economics
- Artificial Analysis — Gemini 3.5 Flash: Everything You Need to Know
- Dr. Earl Brandt — Prepare for the AI Token Rug Pull