Engineers at a large ride-hailing company act on about two of every three comments from their AI code reviewer. That beats the rate for comments left by human reviewers. The figure is a clean example of agentic AI first-pass yield. It is the share of agent output that people accept and use without rework.
In my post on the KPIs that matter besides cost savings, I named five numbers worth watching. Then I took the first apart in a deep dive on cycle time. After that came the second, in a post on straight-through rate. This is the third in the series.
Yield answers the first question a risk officer asks, which is whether anyone can trust the output. It also decides whether the savings are real. A fast agent that needs a checker behind it often costs more than it saves. So the number matters on both the risk side and the money side.
What the number counts
The top of the fraction is simple. It counts outputs that went to their next use with no edit, no correction and no rerun. The bottom counts every output the agent delivered, not just the ones someone happened to review.
Rework needs a definition before the pilot starts. A loan officer who rewrites two lines of a credit memo has reworked it. Likewise, a reviewer who sends a draft back for a second try has reworked it. No human has to type a word. A rerun is still a rerun.
Straight-through rate and yield are easy to confuse, but they count different things. Straight-through rate asks whether a person touched the case at all. By contrast, agentic AI first-pass yield asks whether the output survived the touch unchanged. An agent can score high on one and low on the other.
Why it needs a partner
Agentic AI first-pass yield has a blind spot. It only counts the rework somebody notices. If a reviewer waves a flawed output through, the yield looks perfect and the error moves downstream.
That is why I pair it with the error-escape rate. This counts the mistakes found after acceptance, by a later step, an auditor or a customer. A high yield with a rising escape rate does not mean the agent improved. It usually means the reviewers got tired.
Anthropic’s engineers make a related point in their January guide, Demystifying evals for AI agents. They separate two scores. One is the chance an agent succeeds at least once in several tries. The other is the chance it succeeds on every try. As the number of tries grows, the first climbs toward certainty while the second can fall toward zero. A customer only gets one try, so yield belongs with the second score.
How to measure it without fooling yourself
First, write down what counts as rework before the pilot, not after the results come in. Next, log every edit and rerun against the output that caused it. Keep them in the same audit trail that records the agent’s actions. Finally, run a nightly eval suite so the trend shows up in days, not in a quarterly sample.
Split the numbers by product and by risk level, too. A blended yield hides the one category where the agent keeps getting it wrong. Three measures travel together.
| Metric | What it counts | Watch for |
|---|---|---|
| First-pass yield | Outputs used with no edit or rerun, out of all outputs delivered | Reviewers who stop reading closely |
| Rework time | Minutes people spend fixing the outputs that failed | Fixes done off the books, outside the log |
| Error-escape rate | Mistakes found after acceptance, by a later step or a customer | Escapes found only through complaints |
A code reviewer that earned its keep
The ride-hailing company built its reviewer because engineers were drowning in code changes, many of them written with AI help. The system now checks over 90 percent of the 65,000 or so changes its engineers land each week, the company reported.
The team measured acceptance two ways. Engineers can rate each comment, and they mark about 75 percent as useful. Meanwhile, an automated check reruns the reviewer five times on the final version of the code. If none of the reruns repeats a comment, the team counts it as addressed. More than 65 percent pass that test.
Human comments set the baseline. The company’s own audits found that authors treated just 51 percent of human comments as real bugs. Those were the ones fixed in the same change. In other words, the agent beat the people on yield.
It got there by saying less. The system scores every comment for confidence and drops the weak ones. It also suppresses whole categories of comments that engineers have ignored in the past. The team chose fewer comments with higher acceptance over more comments that people learn to skip.
What I count now
I never report agentic AI first-pass yield on its own anymore. I put it next to the error-escape rate and the rework minutes. Then I split all three by risk level. One clean number for a whole program tells me almost nothing.
Defining rework is a governance decision before it is a design decision. In regulated loan origination, I want that rule in the agent’s intent.md before a single file runs. What counts as an edit? Does a rerun count? Who decides? If those answers come after the pilot, someone will pick the version that looks best.
Yield also tells me when an agent has earned more room. It is one of my gates for moving an agent up an autonomy tier on the Five-Stage Roadmap. An agent with a high straight-through rate and a weak yield has not earned it. It has just moved the checking somewhere I cannot see.
Where yield sets the bar
The ride-hailing company’s 75 percent is a good bar for code review, where a bad comment costs an engineer a few seconds. I am not sure it travels to a credit memo, where one wrong number can mean a bad loan. My view is simple. A slow agent that is right beats a fast one that needs a second person behind it.
How do you define rework for your agents today, and does a rerun count against yield?
Subscribe to the Blog: I publish a new post every weekday on one agentic AI governance question. Enter your email and each post arrives in your inbox the moment it goes live.
My Books: Three books on agentic AI for regulated enterprises: Intent In, Outcomes Out, Earned Autonomy and The Governed Loop. Reading order, formats and buy links are on one page.

© Dr. Harish Kotadia, Ph.D., All Rights Reserved, 2026
Dr. Harish Kotadia, Ph.D., is an Enterprise AI Architect with 20+ years of IT consulting experience serving Fortune 100 clients, specializing in agentic AI systems built on Anthropic Claude, AWS Bedrock, and Google Vertex AI.
Disclaimer: This blog post is based on publicly available academic publications, vendor documentation, open standards, and news items from reputed media sources linked above. This post is intended for educational purposes, to help the enterprise agentic AI community build a shared vocabulary from public, authoritative sources.
Views and opinions expressed here are my own and do not represent those of any employer or client, past or present. The analysis presented is my independent interpretation of the published sources linked above and does not constitute legal, financial, or consulting advice of any kind.

