Inference costs are dropping and that changes everything

A single API call to GPT-5.5 costs about four cents in output tokens. That number is about to get a lot smaller. This week three things happened that tell me the cost of running AI models is about to drop faster than most people expect. Not in theory. In dollars per token.

Why OpenAI built its own chip

On June 24, OpenAI unveiled Jalapeño, its first custom inference processor built with Broadcom. The chip went from concept to silicon in nine months with OpenAI’s own models helping design it. Early benchmarks running GPT-5.3-Codex-Spark show significantly better performance per watt than current alternatives.

The important detail is not the chip itself. It is that OpenAI now controls its own inference silicon, the same way Google controls TPUs and Amazon controls Trainium. Microsoft has its own chip work with Maia. The economics of running AI models are moving from renting Nvidia GPUs to running on purpose-built hardware that each lab owns. That shifts the cost structure from “whatever Nvidia charges” to “whatever it costs to fab this chip.”

Jalapeño targets inference specifically. Not training. That is where the volume is. Every ChatGPT query, every API call, every Codex session runs through inference silicon. Owning that layer lets OpenAI optimise for the actual workloads it runs instead of the general-purpose GPU workloads Nvidia optimises for.

How compute became a commodity market

The same week, SpaceX signed a $6.3 billion compute deal with Reflection AI, giving an open-weight AI lab access to Nvidia GB300 chips in the Colossus 2 data center near Memphis. Reflection pays $150 million a month through 2029. SpaceX already has similar deals with Anthropic ($1.25B/month) and Google ($920M/month).

The pattern is clear: the companies that own the compute infrastructure are renting it out like a utility. SpaceX built Colossus for xAI’s internal work, but when that stalled, it turned the GPUs into a revenue stream. Nvidia’s Blackwell systems are flooding the market with cheaper inference capacity. The marginal cost of a token is dropping because there is more compute available than any single lab can use.

OpenRouter launched Fusion the same week, pooling several models into one response and nearly matching Fable 5’s deep research quality at lower cost. The arbitrage opportunity is real: when you can route queries across models based on real-time pricing, the effective cost per good answer drops further.

Are token prices about to drop?

OpenAI is reportedly weighing significant cuts to token pricing, according to the Wall Street Journal. The move is widely seen as a preemptive strike against Anthropic’s pricing advantage ahead of both companies’ upcoming IPOs. Nvidia’s Blackwell systems have already started driving down the cost of inference, and the pricing crisis the WSJ describes is just the beginning.

When your marginal cost per token drops, you can afford to compete on price. When your competitor also has a custom chip and a compute deal, you have to compete on price. The result is a race to the bottom that benefits everyone building on top of these models. Developers get cheaper inference. Labs get more volume. The question is how low prices can go before the economics break for the labs that do not own their infrastructure.

What cheaper inference means for developers

If you build on AI models, the next 12 months look different from the last 12. Cheaper inference means you can afford to do more per query: longer context windows, more agentic loops, more retries without blowing your budget. It also means the moat is shifting from “we have a good model” to “we have the best infrastructure to run it.” The labs that own their chip, their data center, and their deployment stack will have a cost advantage that pure model quality cannot overcome.

This is the week the AI infrastructure race stopped being about model quality and started being about who can deliver the cheapest token.

FAQ

How much does AI inference cost right now?

GPT-5.5 costs about $15 per million output tokens as of June 2026. GPT-5.6 Sol, launched the same week, is priced at $30 per million output tokens for the flagship model, with Luna at $6 per million. Anthropic’s Claude Opus 4.8 and Sonnet 5 are priced similarly. These prices are widely expected to drop as custom inference chips enter production.

What is OpenAI’s Jalapeño chip and why does it matter?

Jalapeño is OpenAI’s first custom inference processor, built in nine months with Broadcom. It is designed specifically for running AI models rather than training them. Early tests show better performance per watt than current alternatives. The chip lets OpenAI control its own inference costs instead of paying Nvidia’s margins, which puts downward pressure on API pricing.

Will cheaper inference hurt AI model quality?

No. Inference cost and model quality are mostly independent. You can run the same model on cheaper hardware or more efficient silicon and get identical output quality. The cost improvements come from hardware efficiency, not from cutting corners on the model itself. Some price cuts may come from using smaller or faster model variants, but those tradeoffs are transparent in the API pricing.

How do SpaceX’s compute deals affect AI pricing?

SpaceX is leasing GPU capacity from its Colossus data centers to AI labs at scale. This adds significant supply to the compute market, which drives down the price of renting GPUs. More supply means lower costs for AI labs, which in turn means lower API prices for developers. The deals also show that owning compute infrastructure is becoming a defensible advantage in the AI market.

When will inference prices drop noticeably?

The first wave of price cuts is already expected in mid-to-late 2026. OpenAI’s token pricing review, Nvidia Blackwell deployments, and custom chips entering production all point to meaningful price reductions within the next two quarters. Builders planning for 2027 should model inference costs at 30-50% of current rates.