3314112bb9
Add two new sources with summaries, new concepts (developer-as-agent-manager, review-is-the-new-bottleneck), new entities (SWEPR, Nikolai Sheiko), and a query on the Stanford source; update related concept pages, overview, index, and log.
39 lines
4.4 KiB
Markdown
39 lines
4.4 KiB
Markdown
# Review Is the New Bottleneck
|
||
|
||
#concept
|
||
|
||
## Summary
|
||
|
||
When agents write the code, the SDLC doesn't collapse completely — it collapses *around the humans*. [[nikolai-sheiko]]'s two remaining "red squares" are the **reviewer** and the **planner**: tasks pile up in the review queue, reviewers burn out, quality drops, and headline output metrics (PRs, LoC) rise while real throughput barely moves. The organizational fix is twofold: **review with the agent** (not fully manual, not fully delegated) and **measure completed tasks without rework** rather than anything volume-based.
|
||
|
||
## Current Understanding
|
||
|
||
- **The mechanism.** Generation got ~free, so the cost moved downstream to verification — and at team scale, downstream is a *person* with a queue. Sheiko's European-outsourcer case: more PRs than ever, net gain +1%, because rework consumed the difference. Manual-only review starts a spiral (queue → burnout → rubber-stamping → more rework); fully delegated review is the opposite error ([[async-by-default]]'s "proof produced by the thing being checked").
|
||
- **The middle path: review together with the agent.** Treat the model as a smart student — direct it, pose hypotheses, locate problems jointly. This is the org-level sibling of Theo's reading economics ([[make-more-cheap-code]]: AI reviews before humans, per-file summaries, read only what's worth reading).
|
||
- **The metric that resists gaming:** a task counts as done only if it **doesn't come back for rework**; track task lifetime and rework time. LoC, commit count and PR count are all trivially hacked and all rise *because* of the bottleneck, not despite it.
|
||
- **Externally measured:** Stanford SWEPR coverage reports **+91% PR review time** and ~2.6× rework in AI-heavy workflows, and finds gross code volume up 30–40% while net gains are ~15–20% — the same rework-eats-half story Sheiko tells anecdotally ([[2026-07-30-stanford-swepr-widening-gap]]).
|
||
- **Planning is the other red square.** Time redistributes from coding to "planning on the left + verification on the right" — which is why Sheiko prescribes 20-minutes-minimum planning and why a **Product engineer** role emerges ([[developer-as-agent-manager]]).
|
||
|
||
## Evidence
|
||
|
||
- Reviewer/planner as the remaining red squares; review-with-the-agent; the metrics table; the +1% outsourcer case — [[2026-07-30-rakes-in-ai-sdlc-adoption]].
|
||
- +91% PR review time, 2.6× rework, gross-vs-net gap — [[2026-07-30-stanford-swepr-widening-gap]] *(secondary-coverage numbers)*.
|
||
- Reading as the scarce human resource; AI-review-before-human-review; tiered reading discipline — [[2026-07-24-youre-reading-way-too-much-code]] via [[make-more-cheap-code]].
|
||
- Parallel agents multiplying diffs without multiplying review capacity — the attention pile-up already logged on [[async-by-default]]; this page names that open question as the bottleneck it becomes at team scale.
|
||
|
||
## Related Pages
|
||
|
||
- Concepts: [[make-more-cheap-code]] (individual-level discipline for the same cost shift), [[async-by-default]] (the generation side that feeds the queue), [[developer-as-agent-manager]] (the role shift on the human side of the queue), [[seniority-and-the-junior-squeeze]] ("read what you approve" — why review can't just be dropped), [[maintenance-is-the-real-cost]] (kindred move: the visible activity was never the expensive part)
|
||
- Entities: [[nikolai-sheiko]], [[swepr]], [[theo-browne]]
|
||
|
||
## Contradictions / Uncertainty
|
||
|
||
- **How much review survives?** Theo/Dax hold that agent diff-summaries surface anomalies; [[eugene]] holds that drift is precisely what summaries miss ([[leave-less-room-for-imagination]]); Sheiko's review-with-the-agent is a third position between them — asserted, not tested. Status: tentative.
|
||
- The completed-without-rework metric is better than LoC/PRs but still gameable (e.g. by inflating task granularity or quietly reclassifying rework as new tasks); the source doesn't address it. Status: tentative.
|
||
- Whether review-as-bottleneck is transitional (until verification is agentized) or structural (a human must always sign off — the [[seniority-and-the-junior-squeeze|accountability]] view) is open across the corpus.
|
||
|
||
## Next Questions
|
||
|
||
- What does review-with-the-agent look like concretely — a checklist, a dialogue pattern, a skill? The corpus has the prescription but no transcript of it done well.
|
||
- Is there a non-engineer analog (the webinar audience reviews documents, not PRs)? The checker-skill design in [[2026-07-24-non-engineer-throwaway-verification]] may be it.
|