AI Coding Agents on a Five-Person Team: A Year of Actual Numbers

Somewhere in the last eighteen months, "AI coding agent" stopped meaning autocomplete and started meaning something that opens a pull request. We have five engineers and eight products — Wryte, OurTrail, envpilot.dev, LUMO, and four more across web and native — and at this point I'd guess a third of our merged lines started as agent output. Not a third of our commits. A third of our lines, after review, after rewrite, after the parts that didn't survive got deleted.

I want to write the version of this post that isn't a vendor testimonial and isn't a doom post either. Both genres are dishonest in the same direction: they assume the agent is either a junior engineer you're managing or a magic box that replaces one. It's neither. It's a tool with a sharply bounded competence, and the whole game is knowing the boundary.

Where the boundary actually is

The clean way to say it: agents are excellent at legible problems and unreliable at judgment problems. The less clean, more useful way to say it, from watching this on real diffs for a year:

They're genuinely great at boilerplate that has a correct answer. A new Convex CRUD module — schema, query, mutation, the React hook that wraps it — is maybe 80% mechanical once the shape is decided. Our agents write that 80% correctly on the first pass most of the time now, because the shape is decided by us, in the prompt, before a single line gets generated. Same for API route scaffolding, form validation with Zod, test fixtures, and — this one surprised me — migration scripts. Migrations are boilerplate with teeth: get the direction of a backfill wrong and you've corrupted data, but the pattern is so well-trodden that an agent that's seen ten thousand migrations writes a better first draft than an engineer who's written three.

They're good at review, specifically at the part humans skip. We started pointing an agent at every PR before a human looks at it — not as a gate, as a first pass that flags the boring stuff: an unused import, a missing await, a Convex query that isn't paginated, a type that got widened to any to make an error go away at 11pm. It's the equivalent of a very patient junior who never gets tired of saying "did you mean to do that." Nobody's ego is bruised by a bot pointing out a missing null check, which turns out to matter — humans are more willing to fix things a tool flagged than things a teammate flagged, for reasons that probably say more about us than about the tools.

They fail, reliably, at anything that requires holding the whole system in your head. Architecture decisions. "Should this live in a Convex action or should we push it to a queue." "Does this new feature belong in the existing variables module or does it need its own." "Is this abstraction premature." An agent will confidently produce an answer to all of these, and the answer will sound right, because it's trained on a huge corpus of code that mostly made these calls competently — but it's giving you the median answer, not the answer that fits your system, your constraints, your five-person team's actual maintenance capacity. We have exactly one incident where an agent-proposed abstraction — a generic "resource permission" layer meant to unify three different sharing models — cost us a real week to unwind, because it was locally reasonable and globally wrong, and the engineer who accepted it didn't have the full picture of the three models it was unifying.

Taste doesn't transfer. This is the boring, unglamorous finding, and it's the one I'd bet money still holds in another year. Whether a UI feels right — spacing that reads as calm instead of cramped, an error message that sounds like a person instead of a stack trace, a default that respects what the user probably wants — that's not a legible problem. Agents can execute a taste decision once a human has made it. They cannot make it.

The review-gate workflow

Here's the actual mechanics, because "we use AI carefully" is a sentence that means nothing without the gate underneath it.

Every AI-authored change — and we track this, it's a PR label, not a vibe — goes through the same pipeline as human-authored code, plus one extra step:

1. CI runs regardless of authorship. Typecheck, lint, the full check suite. An agent's code does not get a pass because an agent wrote it; if anything we're stricter, because agents are more likely to produce code that typechecks while being subtly wrong than code that fails to typecheck.
2. A human opens the diff and reads every line before merge. Not skims. Reads. This is the rule that actually costs us time, and it's the rule we've never once relaxed, because the one time we got sloppy about it — early on, on a small styling PR that "obviously" couldn't matter — the agent had silently dropped an aria-label fixing an unrelated lint warning it decided to also clean up. Small, harmless-looking, and exactly the kind of thing a skim misses and a read catches.
3. The reviewer's job is specifically to ask "would I have written this." Not "does this work" — CI answers that. The human question is about the shape of the solution: is this the right abstraction, does it match how the rest of the codebase does this, would the next engineer understand it in six months without the PR description.
4. Anything architectural gets scoped by a human before an agent touches it. We don't ask an agent "how should sharing permissions work across projects." We decide that in a doc or a conversation, then ask an agent to implement the decided shape. Direction is never delegated. Execution against a stated direction is delegated constantly.

The gate isn't there because we don't trust the tools. It's there because the tools are unevenly trustworthy, and unevenness is exactly the thing a blanket policy — "always review" or "never review" — can't handle. A rule that treats every agent PR the same as every human PR is the simplest rule that's actually correct, so that's the rule we run.

The honest cost math

Subscriptions for agentic coding tools across five engineers run us solidly into four figures a month at this point — more than our Convex bill, more than our email sending, more than most of our infra line items combined. That's a real number and I'm not going to pretend it isn't.

What we get back: our estimate, from watching sprint velocity before and after we made agent-assisted development the default rather than the exception, is somewhere in the 25–40% range on throughput for the categories of work agents are good at — which is not "40% faster at everything," it's 40% faster at boilerplate, migrations, and first-pass review, applied to maybe half of what we ship in a given week. Do the multiplication and it's real but it's not the tenfold productivity story you'll see claimed elsewhere. It's a genuinely good tool with a genuinely good ROI, not a headcount replacement.

The cost that doesn't show up on the subscription invoice is the review discipline. Reading every line of every AI PR is slower, per PR, than reading a smaller human PR — agents tend to produce more code per unit of intent than a human would, because "write me the CRUD module" generates the whole module instead of the two lines a human would've reused from somewhere else. We've had to actively coach the habit of "would three existing lines cover this" back into agent prompts, because the tools default toward writing new code over finding old code, and new code is a liability a codebase pays for indefinitely.

What I'd tell a team our size starting now

Use them for the legible 80%. Boilerplate, migrations, first-pass review, test scaffolding, anything with a well-worn shape. Don't use them for direction — architecture, taste, "should this exist" — because that's not a capability gap that's closing on any timeline we can plan around; it's a category difference between execution and judgment.

Keep the gate. Not as ceremony — as the actual mechanism that lets you say yes to the tool's speed without inheriting its blind spots. The teams I've seen get burned aren't the ones using agents heavily; they're the ones who stopped reading the diffs because the diffs kept passing CI.

We're a five-person team that ships eight products. That sentence was not true three years ago and the honest reason it's true now is a mix of the tools and the discipline around them — in roughly equal parts, and neither one works without the other.

---

Syntax Lab Technology is a five-engineer studio building Wryte, OurTrail, envpilot.dev, LUMO, and four other products from Islamabad.