---
title: Inference costs are dropping and that changes everything
description: OpenAI's custom chip, SpaceX compute deals, and falling token prices are pushing inference costs down faster than most developers have noticed.
canonical: https://lukeocodes.dev/inference-costs-dropping-changes-everything
type: article
locale: en_GB
published: 2026-06-22T00:00:00.000Z
modified: 2026-06-22T00:00:00.000Z
authors:
  - Luke Oliff
section: AI
tags:
  - AI
  - Industry
  - News
feed: https://lukeocodes.dev/rss.xml
search: https://lukeocodes.dev/search.json?q={query}
schema:
  - "@context": https://schema.org
    "@graph":
      - "@type": Organization
        "@id": https://ably.com/#organization
        name: Ably
        url: https://ably.com
      - "@type": WebSite
        "@id": https://lukeocodes.dev/#website
        url: https://lukeocodes.dev
        name: Luke Oliff
        alternateName: lukeocodes
        inLanguage: en
        publisher:
          "@id": https://lukeocodes.dev/authors/luke#person
        description: Personal blog and writing by Luke Oliff — developer relations, voice AI, developer experience.
        potentialAction:
          - "@type": SearchAction
            target:
              "@type": EntryPoint
              urlTemplate: https://lukeocodes.dev/blog/search?q={search_term_string}
            query-input: required name=search_term_string
      - "@type": Person
        "@id": https://lukeocodes.dev/authors/luke#person
        name: Luke Oliff
        url: https://lukeocodes.dev/authors/luke
        description: Staff engineer and Staff Tech Lead for Developer Experience at Ably. Writing about developer relations, AI, and the voice industry, from the UK.
        mainEntityOfPage: https://lukeocodes.dev/authors/luke
        givenName: Luke
        familyName: Oliff
        image: https://lukeocodes.dev/authors/luke.jpg
        jobTitle: Staff Tech Lead, Developer Experience
        worksFor:
          "@id": https://ably.com/#organization
        sameAs:
          - https://www.linkedin.com/in/lukeocodes
          - https://github.com/lukeocodes
          - https://bsky.app/profile/lukeocodes.dev
          - https://x.com/lukeocodes
          - https://hashnode.com/@lukeocodes
          - https://dev.to/lukeocodes
        knowsAbout:
          - Developer Relations
          - Developer Experience
          - Voice AI
          - Text-to-Speech
          - Answer Engine Optimisation
          - Search Engine Optimisation
          - Open Source
          - Developer Tooling
          - API Design
          - Speech-to-Text
          - Voice Agents
          - Community Building
        knowsLanguage:
          - English
        nationality: British
        alumniOf:
          - "@type": Organization
            name: The Open University
  - "@context": https://schema.org
    "@graph":
      - "@type": Organization
        "@id": https://ably.com/#organization
        name: Ably
        url: https://ably.com
      - "@type": Person
        "@id": https://lukeocodes.dev/authors/luke#person
        name: Luke Oliff
        url: https://lukeocodes.dev/authors/luke
        description: Staff engineer and Staff Tech Lead for Developer Experience at Ably. Writing about developer relations, AI, and the voice industry, from the UK.
        mainEntityOfPage: https://lukeocodes.dev/authors/luke
        givenName: Luke
        familyName: Oliff
        image: https://lukeocodes.dev/authors/luke.jpg
        jobTitle: Staff Tech Lead, Developer Experience
        worksFor:
          "@id": https://ably.com/#organization
        sameAs:
          - https://www.linkedin.com/in/lukeocodes
          - https://github.com/lukeocodes
          - https://bsky.app/profile/lukeocodes.dev
          - https://x.com/lukeocodes
          - https://hashnode.com/@lukeocodes
          - https://dev.to/lukeocodes
        knowsAbout:
          - Developer Relations
          - Developer Experience
          - Voice AI
          - Text-to-Speech
          - Answer Engine Optimisation
          - Search Engine Optimisation
          - Open Source
          - Developer Tooling
          - API Design
          - Speech-to-Text
          - Voice Agents
          - Community Building
        knowsLanguage:
          - English
        nationality: British
        alumniOf:
          - "@type": Organization
            name: The Open University
      - "@type": BlogPosting
        "@id": https://lukeocodes.dev/inference-costs-dropping-changes-everything#article
        headline: Inference costs are dropping and that changes everything
        alternativeHeadline: Inference costs are dropping and that changes everything
        description: OpenAI's custom chip, SpaceX compute deals, and falling token prices are pushing inference costs down faster than most developers have noticed.
        datePublished: 2026-06-22T00:00:00.000Z
        dateModified: 2026-06-22T00:00:00.000Z
        url: https://lukeocodes.dev/inference-costs-dropping-changes-everything
        mainEntityOfPage: https://lukeocodes.dev/inference-costs-dropping-changes-everything
        inLanguage: en
        author:
          - "@id": https://lukeocodes.dev/authors/luke#person
        isPartOf:
          "@id": https://lukeocodes.dev/#website
        keywords: AI, Industry, News
        articleSection: AI
        abstract: A single API call to GPT-5.5 costs about four cents in output tokens. That number is about to get a lot smaller. This week three things happened that tell me the cost of running AI models is about to drop faster than most people expect. Not in theory. In dollars per token.
        speakable:
          "@type": SpeakableSpecification
          cssSelector:
            - .post-tldr
            - h1
        image: https://lukeocodes.dev/og.png?h=Inference+costs+are+dropping+and+that+changes+everything&d=OpenAI%27s+custom+chip%2C+SpaceX+compute+deals%2C+and+falling+token+prices+are+pushing+inference+costs+down+faster+than+most+developers+have+noticed.&e=Writing
        wordCount: 989
        timeRequired: PT5M
        publisher:
          "@id": https://ably.com/#organization
      - "@type": BreadcrumbList
        itemListElement:
          - "@type": ListItem
            position: 1
            name: Home
            item: https://lukeocodes.dev
          - "@type": ListItem
            position: 2
            name: Writing
            item: https://lukeocodes.dev/blog
          - "@type": ListItem
            position: 3
            name: Inference costs are dropping and that changes everything
            item: https://lukeocodes.dev/inference-costs-dropping-changes-everything
---

[Luke Oliff.](/)

- [Blog](/blog)
- [CV](/cv)
- [Uses](/uses)
- [User manual](/user-manual)
- [Living with ADHD](/living-and-working-with-adhd)
- [RSS](/rss.xml)

[Luke Oliff · Writing](/blog)

# Inference costs are dropping and that changes everything

Jun 2026 · [AI](/blog/category/ai) · 5 min read · [Luke Oliff](/authors/luke)

A single API call to GPT-5.5 costs about four cents in output tokens. That number is about to get a lot smaller. This week three things happened that tell me the cost of running AI models is about to drop faster than most people expect. Not in theory. In dollars per token.

### Why OpenAI built its own chip

On June 24, OpenAI unveiled Jalapeño, its first custom inference processor built with Broadcom. The chip went from concept to silicon in nine months with OpenAI's own models helping design it. Early benchmarks running GPT-5.3-Codex-Spark show significantly better performance per watt than current alternatives.

The important detail is not the chip itself. It is that OpenAI now controls its own inference silicon, the same way Google controls TPUs and Amazon controls Trainium. Microsoft has its own chip work with Maia. The economics of running AI models are moving from renting Nvidia GPUs to running on purpose-built hardware that each lab owns. That shifts the cost structure from "whatever Nvidia charges" to "whatever it costs to fab this chip."

Jalapeño targets inference specifically. Not training. That is where the volume is. Every ChatGPT query, every API call, every Codex session runs through inference silicon. Owning that layer lets OpenAI optimise for the actual workloads it runs instead of the general-purpose GPU workloads Nvidia optimises for.

### How compute became a commodity market

The same week, SpaceX signed a $6.3 billion compute deal with Reflection AI, giving an open-weight AI lab access to Nvidia GB300 chips in the Colossus 2 data center near Memphis. Reflection pays $150 million a month through 2029. SpaceX already has similar deals with Anthropic ($1.25B/month) and Google ($920M/month).

The pattern is clear: the companies that own the compute infrastructure are renting it out like a utility. SpaceX built Colossus for xAI's internal work, but when that stalled, it turned the GPUs into a revenue stream. Nvidia's Blackwell systems are flooding the market with cheaper inference capacity. The marginal cost of a token is dropping because there is more compute available than any single lab can use.

OpenRouter launched Fusion the same week, pooling several models into one response and nearly matching Fable 5's deep research quality at lower cost. The arbitrage opportunity is real: when you can route queries across models based on real-time pricing, the effective cost per good answer drops further.

### Are token prices about to drop?

OpenAI is reportedly weighing significant cuts to token pricing, according to the Wall Street Journal. The move is widely seen as a preemptive strike against Anthropic's pricing advantage ahead of both companies' upcoming IPOs. Nvidia's Blackwell systems have already started driving down the cost of inference, and the pricing crisis the WSJ describes is just the beginning.

When your marginal cost per token drops, you can afford to compete on price. When your competitor also has a custom chip and a compute deal, you have to compete on price. The result is a race to the bottom that benefits everyone building on top of these models. Developers get cheaper inference. Labs get more volume. The question is how low prices can go before the economics break for the labs that do not own their infrastructure.

### What cheaper inference means for developers

If you build on AI models, the next 12 months look different from the last 12. Cheaper inference means you can afford to do more per query: longer context windows, more agentic loops, more retries without blowing your budget. It also means the moat is shifting from "we have a good model" to "we have the best infrastructure to run it." The labs that own their chip, their data center, and their deployment stack will have a cost advantage that pure model quality cannot overcome.

This is the week the AI infrastructure race stopped being about model quality and started being about who can deliver the cheapest token.

## FAQ

#### How much does AI inference cost right now?

GPT-5.5 costs about $15 per million output tokens as of June 2026. GPT-5.6 Sol, launched the same week, is priced at $30 per million output tokens for the flagship model, with Luna at $6 per million. Anthropic's Claude Opus 4.8 and Sonnet 5 are priced similarly. These prices are widely expected to drop as custom inference chips enter production.

#### What is OpenAI's Jalapeño chip and why does it matter?

Jalapeño is OpenAI's first custom inference processor, built in nine months with Broadcom. It is designed specifically for running AI models rather than training them. Early tests show better performance per watt than current alternatives. The chip lets OpenAI control its own inference costs instead of paying Nvidia's margins, which puts downward pressure on API pricing.

#### Will cheaper inference hurt AI model quality?

No. Inference cost and model quality are mostly independent. You can run the same model on cheaper hardware or more efficient silicon and get identical output quality. The cost improvements come from hardware efficiency, not from cutting corners on the model itself. Some price cuts may come from using smaller or faster model variants, but those tradeoffs are transparent in the API pricing.

#### How do SpaceX's compute deals affect AI pricing?

SpaceX is leasing GPU capacity from its Colossus data centers to AI labs at scale. This adds significant supply to the compute market, which drives down the price of renting GPUs. More supply means lower costs for AI labs, which in turn means lower API prices for developers. The deals also show that owning compute infrastructure is becoming a defensible advantage in the AI market.

#### When will inference prices drop noticeably?

The first wave of price cuts is already expected in mid-to-late 2026. OpenAI's token pricing review, Nvidia Blackwell deployments, and custom chips entering production all point to meaningful price reductions within the next two quarters. Builders planning for 2027 should model inference costs at 30-50% of current rates.

[← Newer: TIL: Read speech marks from the Speechify API response](/til-speech-marks-speechify-api)

[All posts](/blog) →

## Search this site

`GET https://lukeocodes.dev/search.json?q=<words>` returns matching posts as JSON, each with its page URL and its markdown URL. Add `&limit=<1-50>` for more than 10. How it works: [/blog/search](https://lukeocodes.dev/blog/search.md).

```sh
curl "https://lukeocodes.dev/search.json?q=developer+relations"
```

---

© 2026 Luke Oliff. Posts live at the root of this site.

- [GitHub](https://github.com/lukeocodes)
- [LinkedIn](https://www.linkedin.com/in/lukeocodes)
- [X](https://x.com/lukeocodes)
- [Bluesky](https://bsky.app/profile/lukeocodes.dev)
- [Email](mailto:luke@lukeoliff.com)
- [RSS](/rss.xml)

## Cookies

This site sets one analytics cookie, from Google Analytics, and only in a browser that accepts the cookie banner. A client that does not run scripts sets no cookies, so there is nothing to accept or decline here.

[Cookie policy](/cookies)
