A 90% agent PR merge rate can still describe a weak engineering workflow
PR acceptance rate measures submitted work. Task selection, review effort and delivery outcomes reveal how much an agent workflow actually contributes.
Table of Contents
When a coding agent opens ten pull requests and reviewers merge eight, an 80% acceptance rate looks like a useful performance measure. It tells us that most submitted changes cleared review. It does not tell us whether the agent worked on the problems that mattered, reduced delivery time, or left the product in a better state.
The denominator contains only the work we gave the agent and allowed it to submit. Once teams begin optimising that ratio, they have an incentive to route small, well-specified changes to the agent and keep ambiguous, high-impact work elsewhere. The metric can improve while the agent contributes little to the team’s real constraint.
Acceptance measures the submitted portfolio
Pull-request acceptance rate is straightforward:
Accepted agent pull requests ÷ submitted agent pull requests
Both parts are shaped before review begins. Someone selects the task, decides whether the agent’s attempt is presentable, and chooses whether to open a pull request at all. A failed attempt abandoned in a local branch may never reach the denominator.
Task type changes the result too. A 2026 paper analysed 7,156 agent-generated pull requests from the AIDev dataset. Documentation changes had an 82.1% acceptance rate, while new features had 66.1%. Across all categories, the reported range ran from 84.0% for chores to 55.4% for performance work. The authors describe task type as a major factor and also note an alternative explanation: review standards may differ by category. This is observational evidence from public repositories, not proof that the same gap exists inside every company.
Another study examined 567 Claude Code pull requests across 157 open-source projects. It reported that developers tended to use the agent for refactoring, documentation and testing. Although 83.8% of the pull requests were merged, only 54.9% of those merged needed no further modification.
These studies do not establish the business value of any individual change. They show why an unsegmented acceptance rate is hard to interpret: the mix of work and the human correction behind the merge both matter.
A perfect score can come from a narrow assignment
Consider a hypothetical B2B SaaS backlog with three changes:
- Correct an API example in the documentation.
- Add validation for a missing optional field.
- Change subscription cancellation across billing, access control, stored files and outbound webhooks.
The first two tasks are bounded and easy to verify. The third crosses systems, contains unresolved product decisions and carries a larger rollback cost. Suppose an agent completes the first two, both pull requests are accepted, and the team handles the cancellation change manually.
The agent’s acceptance rate is 100%. That is accurate. It is also incomplete.
Sure, the agent removed two useful pieces of work, so we should not dismiss the contribution. But the problem appears when we use the score to answer a different question: how much valuable engineering work can this workflow take on? The metric has no representation of the unattempted third task, its relative impact, or the effort reviewers spent correcting the accepted changes for the 1st and 2nd scenarios.
This is a broader measurement problem. In an exploratory analysis of coding-agent transcripts from seven METR technical staff during January 2026, METR explicitly warned about task selection and task substitution. People used agents where they expected help and sometimes completed useful but lower-value tasks they might not otherwise have done. The author therefore treated observed task-level time savings as a soft upper bound on productivity improvement, not a productivity multiplier.
Review effort sits outside the numerator
A merged pull request can require a five-minute check or several rounds of correction. Acceptance counts both as one success.
For agent work, the hidden effort may include clarifying the ticket, restoring an assertion the agent weakened, checking tenant boundaries, rerunning a flaky test, or rewriting the change so it fits an existing design. The final merge tells us that a reviewer accepted the resulting code. It does not attribute how much of that result came from the agent.
Rejection has a similar ambiguity. A valuable attempt at a difficult migration may uncover an undocumented dependency even if its pull request is not merged. A trivial change can be accepted immediately. If we reward teams only for the ratio, the safer strategy is to submit more of the second kind.
The practical response is not to stop tracking acceptance. It is to retain the diagnostic signal and add the context needed to interpret it.
Measure the work before, during and after review
I would evaluate an agent workflow through five connected questions:
| Question | Evidence to collect |
|---|---|
| What work reached the agent? | Task category, expected impact, uncertainty, affected systems and a rough human effort estimate |
| Did it satisfy the task? | Reviewed acceptance criteria and end-to-end checks against the resulting behaviour |
| What did the human add? | Review time, number and type of corrections, and unresolved decisions returned to the author |
| Did it improve delivery? | Lead time from task start to production, including waiting and rework |
| Did the change remain healthy? | Rollbacks, escaped defects, operational incidents and the intended product or business result |
The categories do not need an elaborate scoring model. Start by separating documentation, tests, fixes, features, migrations and performance work. Compare acceptance only within similar groups and show the number of attempted, abandoned, submitted and merged tasks at each stage.
For task success, use the behaviour the change was meant to produce. OpenAI’s SWE-Lancer illustrates one way to make this concrete in an evaluation: it includes more than 1,400 real freelance software-engineering tasks with actual payouts, while independent tasks are graded using end-to-end tests reviewed three times by experienced engineers. It is a benchmark, not a template for measuring an internal team, but its design separates task outcome and economic context from whether a patch merely looks acceptable.
Delivery measures can then connect the agent’s contribution to the surrounding system. DORA’s current software-delivery measures include change lead time, deployment frequency, failed deployment recovery time, change fail rate and deployment rework rate. These are team-level measures and should not be attributed entirely to the agent. They help reveal whether more accepted pull requests coincide with safer, faster delivery.
Keep acceptance rate in its proper place
Pull-request acceptance rate is useful for finding friction. Segment it by task type, inspect rejection reasons, and track whether agent output routinely needs the same corrections. It can tell us where the workflow produces reviewable changes.
The larger decision requires a wider denominator. Include the tasks the agent attempted, the tasks we chose not to give it, and the human work needed to turn its output into production software. Then connect those results to delivery and product outcomes.
If I had to start over again, I would begin with one month of agent-assisted work. Label every task by type and uncertainty, record whether an attempt reached review, and sample the corrections behind accepted pull requests. We can then decide whether the agent is expanding the work the team can complete or becoming very efficient at the work that was already easiest to accept.