3314112bb9
Add two new sources with summaries, new concepts (developer-as-agent-manager, review-is-the-new-bottleneck), new entities (SWEPR, Nikolai Sheiko), and a query on the Stanford source; update related concept pages, overview, index, and log.
4.4 KiB
4.4 KiB
Review Is the New Bottleneck
#concept
Summary
When agents write the code, the SDLC doesn't collapse completely — it collapses around the humans. nikolai-sheiko's two remaining "red squares" are the reviewer and the planner: tasks pile up in the review queue, reviewers burn out, quality drops, and headline output metrics (PRs, LoC) rise while real throughput barely moves. The organizational fix is twofold: review with the agent (not fully manual, not fully delegated) and measure completed tasks without rework rather than anything volume-based.
Current Understanding
- The mechanism. Generation got ~free, so the cost moved downstream to verification — and at team scale, downstream is a person with a queue. Sheiko's European-outsourcer case: more PRs than ever, net gain +1%, because rework consumed the difference. Manual-only review starts a spiral (queue → burnout → rubber-stamping → more rework); fully delegated review is the opposite error (async-by-default's "proof produced by the thing being checked").
- The middle path: review together with the agent. Treat the model as a smart student — direct it, pose hypotheses, locate problems jointly. This is the org-level sibling of Theo's reading economics (make-more-cheap-code: AI reviews before humans, per-file summaries, read only what's worth reading).
- The metric that resists gaming: a task counts as done only if it doesn't come back for rework; track task lifetime and rework time. LoC, commit count and PR count are all trivially hacked and all rise because of the bottleneck, not despite it.
- Externally measured: Stanford SWEPR coverage reports +91% PR review time and ~2.6× rework in AI-heavy workflows, and finds gross code volume up 30–40% while net gains are ~15–20% — the same rework-eats-half story Sheiko tells anecdotally (2026-07-30-stanford-swepr-widening-gap).
- Planning is the other red square. Time redistributes from coding to "planning on the left + verification on the right" — which is why Sheiko prescribes 20-minutes-minimum planning and why a Product engineer role emerges (developer-as-agent-manager).
Evidence
- Reviewer/planner as the remaining red squares; review-with-the-agent; the metrics table; the +1% outsourcer case — 2026-07-30-rakes-in-ai-sdlc-adoption.
- +91% PR review time, 2.6× rework, gross-vs-net gap — 2026-07-30-stanford-swepr-widening-gap (secondary-coverage numbers).
- Reading as the scarce human resource; AI-review-before-human-review; tiered reading discipline — 2026-07-24-youre-reading-way-too-much-code via make-more-cheap-code.
- Parallel agents multiplying diffs without multiplying review capacity — the attention pile-up already logged on async-by-default; this page names that open question as the bottleneck it becomes at team scale.
Related Pages
- Concepts: make-more-cheap-code (individual-level discipline for the same cost shift), async-by-default (the generation side that feeds the queue), developer-as-agent-manager (the role shift on the human side of the queue), seniority-and-the-junior-squeeze ("read what you approve" — why review can't just be dropped), maintenance-is-the-real-cost (kindred move: the visible activity was never the expensive part)
- Entities: nikolai-sheiko, swepr, theo-browne
Contradictions / Uncertainty
- How much review survives? Theo/Dax hold that agent diff-summaries surface anomalies; eugene holds that drift is precisely what summaries miss (leave-less-room-for-imagination); Sheiko's review-with-the-agent is a third position between them — asserted, not tested. Status: tentative.
- The completed-without-rework metric is better than LoC/PRs but still gameable (e.g. by inflating task granularity or quietly reclassifying rework as new tasks); the source doesn't address it. Status: tentative.
- Whether review-as-bottleneck is transitional (until verification is agentized) or structural (a human must always sign off — the seniority-and-the-junior-squeeze view) is open across the corpus.
Next Questions
- What does review-with-the-agent look like concretely — a checklist, a dialogue pattern, a skill? The corpus has the prescription but no transcript of it done well.
- Is there a non-engineer analog (the webinar audience reviews documents, not PRs)? The checker-skill design in 2026-07-24-non-engineer-throwaway-verification may be it.