Time to First Call Now Measures an Agent, Not a Developer

Time to first call is the number of minutes between a developer finding your API (a way for their code to ask your service for something) and getting one successful response back. For years it has been a favourite onboarding number at API companies. In 2026 a coding agent (a program that reads docs, writes code and runs it for you) makes that first call more and more often, so the clock is mostly timing the agent.

GitHub’s yearly Octoverse report, out on 28 October 2026, says more than 40% of pull requests (the request to merge a change into a project) started with an agent’s first draft in September. In March it was 17%. When the agent writes the integration (the code that hooks your API into their app), the person may never open your quickstart (the short getting-started guide).

I’d still track the first call. Next to it I now want three newer numbers: time to a working result, failed requests on your docs, and an agent test suite run against your own product.

Where time to first call came from

The idea is simple. Count how long it takes a stranger to go from “I’ve heard of this API” to “it answered me”, and work to make that shorter. Some teams call it time to hello world.

The clearest public data I’ve found comes from Postman, the API testing tool. In April 2023, Joyce Lin wrote up what happened when API publishers gave developers a collection (a saved set of requests that are ready to run, keys and all). She asked publishers to pull their logs or run an experiment. For one API, called Company A in the post, the first successful call took 17 minutes without a collection and 10 minutes with one. The chart in the same post puts the gain at 1.7 to 56 times faster across the publishers in the sample.

Every API in the sample already had documentation, and Lin says the gains “cannot be attributed to the mere presence of documentation”. The win came from handing developers something they could run straight away. The post gives no sample size and names only one publisher, and the 56 times figure appears only in a chart caption.

Why does a fast first call mean less now?

The first call was always a sign of something else. Nobody signs up to an API to make one request. They sign up to build a feature, and the first response was the earliest point you could see they were on the way.

Marcos Placona wrote about this in September 2026 in a piece on how to show what developer relations (DevRel, the team that helps developers use a product) is worth to a business. He uses time to first value: “How long does it take a developer to go from signup to a real, working result?” He puts it next to activation (the share of sign-ups who become active users) and gives 15% to 35% as a typical range for activation inside 24 hours. His worked example moves activation from 20% to 30% by fixing onboarding friction, and he says to swap in your own data before you show it to anyone. Page views, follower counts, booth scans (badges scanned at a conference stand), GitHub stars and community member counts all go further down his list.

Agents changed the gap between the first call and a working result, because the first-call number now drops for every product, easy or hard. An agent can make a first call in seconds. It has a key saved in its settings, it has your docs, and it doesn’t get bored.

What the agent does after that first call is harder to see. Phil Leggetter, who runs developer experience at Hookdeck, put it like this in a post on publishing agent evals from 17 September 2026: “an agent that gets lost doesn’t file a support ticket. It works around the problem and reports success”.

What the data says about agents reading docs

Agents are already a big share of the readers on developer docs. Mintlify, which hosts docs for a lot of API companies, reported 257 million agent requests against 131 million human page loads in August 2026. I went through those numbers in does llms.txt work last month.

The same company ran 2,400 tests across 20 doc sites. Agents got the right answer 94 to 99% of the time, whatever format the docs were in. Where they went wrong was finding pages. On normal web pages (HTML) they made 2.23 requests per task for pages that did not exist. On Markdown (plain text with light formatting) that dropped to 1.42, and on Markdown that linked to an llms.txt file (a plain list of links to the docs pages) it was 0.11.

Postman found in 2023 that docs alone were not enough for people, and ready-to-run requests made the difference. Mintlify’s 2026 tests found failures dropped when pages were easy for agents to find, while the answers they got stayed just as accurate.

Agent work gets merged at about the same rate as the rest. In February 2026, 83% of pull requests with agent involvement merged within two weeks, against about 86% for pull requests without it, according to Octoverse.

Developers ship faster and still feel the friction

You might expect agents to make onboarding friction go away, but developers say it hasn’t.

Atlassian’s State of Developer Experience 2026, written up by Andrew Boyagi on 20 October, found 87% of developers say AI helps them ship faster. 72% say they feel the same or more friction than before AI. 65% now regularly spend time correcting AI output, and finding information is still the top cause of friction.

The 2025 Stack Overflow Developer Survey found more developers distrust the accuracy of AI tools (46%) than trust it (33%), while 84% use them or plan to.

In 2023 Abi Noda, Margaret-Anne Storey, Nicole Forsgren and Michaela Greiler published DevEx: What Actually Drives Productivity in ACM Queue, a computing magazine. Time to first call is a feedback loop number, which is the first of the three parts of developer experience they describe: feedback loops (how fast you find out if something worked), cognitive load (how much you have to hold in your head) and flow state (time spent focused without interruption).

How are teams measuring agents on their own products?

Some API companies now test agents against their product directly. The test is called an eval: a set of realistic tasks, each handed to an agent in a real project, with code that checks whether what it built works. You run it with and without your docs or skill files (instructions you publish for agents to read) and compare.

Leggetter’s survey found at least 23 platforms publishing results by 7 September 2026. Convex started in January 2025. Clerk followed that October, Supabase in April 2026, Auth0 in June and Rails on 4 August 2026. Nearly every published result is under a year old.

• Do the tasks state a goal, or list the steps? A list of steps tests whether the agent follows orders. A goal tests whether your product makes sense to it.
• Did the agent open the skill files at all? In Leggetter’s write-up, Vercel’s skill left its pass rate flat at 53%, and the transcripts (the record of what the agent did) showed the skill was not opened in 56% of runs. An instruction to explore the project and then use the skill took the pass rate to 79%.
• What was the run compared against? Leggetter says a score like “+25 points from our skills” can’t be judged until you know what the comparison run had to work with.

His post also points at Stripe’s write-up, where agents accepted invalid test data and read a 400 error (the code an API sends back when the request is wrong) as a success. Neither problem was the task. Both were problems with the test setup, and both were scored as the model failing.

His first rule is to write each task the way a colleague would file a ticket: say what you want to end up with and leave out how to get there. Hookdeck wrote that down for its own suite, so that an agent “can finish confidently and be wrong”. When the tasks list every step, the agent works down the list, every test passes, and the failures real users hit were never in the suite.

Activation has been on DevRel plans for a long time. Leggetter’s own AAARRRP framework, which he first presented at DevRelCon London in 2016, is a list of goals a DevRel team can pick from, and activation (a developer actually using the product) is one of them.

What to measure beside the first call

I would keep the first-call clock, stop treating it as the main number, and add these beside it.

Time to a working result

Placona’s time to first value. The clock stops when the thing the developer wanted works, not when the first successful response comes back. You can get this from your own product data: the first time a key does the thing your product is for, like sending a message or taking a payment.

Activation rate

The share of sign-ups who get to that working result. Placona’s typical range is 15% to 35% inside 24 hours.

Failed requests on your docs

Mintlify’s 2.23 versus 0.11 was counted per task in a controlled test. Your server logs (the record your web server keeps of every request) can’t tell you where one task ends, but they do show every request for a page that doesn’t exist. Split those by user agent (the name a browser or bot sends with each request) and track the agent share.

An agent eval with a clear comparison

Pick five tasks a real customer would file as a ticket and run them twice, once with a bare agent and once with your docs and skills. Record whether the agent opened your files, and keep the transcripts. That run also gives you failed requests per task.

Time to first call itself

Still worth having, because a broken sign-up or a key that takes a day to approve shows up here first.

What could go wrong with this

Agent evals are young, and most results are published by the company selling the product. Leggetter says so himself and uses his own company as the example of what not to do more than once. His checklist asks for the comparison run, the published tasks and a date on every result.

Time to a working result is harder to define than a first call. “Working” means something different for a payments API and a speech API, and teams will pick the definition that makes them look best. Write it down before you measure it, and don’t change it to fix a bad quarter.

The Postman figures are from 2023 and the method is thin, so I wouldn’t quote 56 times faster to anyone. The 17 to 10 minutes for Company A is the one result the post backs with numbers in the text.

Developers still check the code. Atlassian found 65% regularly correct AI output, and Stack Overflow found 46% distrust its accuracy.

Frequently asked questions

What is time to first call?

Time to first call is the minutes between a developer starting with an API and getting back their first successful response. It is a common onboarding number for API companies. Postman’s 2023 figures showed one API go from 17 minutes to 10 when the publisher gave developers a ready-made set of requests to run.

Is the first API call still worth timing?

Yes, but not on its own. It still shows how hard your sign-up, keys and first request are. It no longer shows who did the work, because a coding agent now often makes that request. Track it next to time to a working result and next to an agent test suite run against your own docs.

What is an agent eval for a developer product?

An agent eval is a set of realistic tasks handed to a coding agent inside a real project, with code that checks whether the result works. You run the tasks with and without your docs or skill files and compare the scores. Phil Leggetter counted at least 23 platforms publishing results by September 2026.

How do I measure developer onboarding when agents write the code?

Measure the outcome, not the first request. Time from sign-up to a working result, the share of sign-ups who get there, how often agents fail on your docs, and whether the agent opened the files you wrote for it. Read agent transcripts too, because an agent that gets stuck rarely files a support ticket.

In 2026, measure whether developers reach a working result, not how fast the first call comes back.