WebinarNotes/wiki/sources/2026-07-30-rakes-in-ai-sdlc-adoption.md

Грабли во внедрении ИИ в SDLC — Rakes in AI Adoption in the SDLC (Nikolai Sheiko)

#source

Source Metadata

• Date: talk published 2026 (references events through Dec 2025); conclusions doc saved 2026-07-30
• Raw path: raw/sources/Грабли во внедрении ИИ в SDLC.md
• Source type: viewer's conclusions from a Russian-language YouTube talk (45:59) — https://www.youtube.com/watch?v=Nm3MsnngCJg — "Грабли во внедрении ИИ в SDLC — почему ИИ есть, а результата нет и как это лечить" ("why the AI is there but the results aren't, and how to treat it"). Not a transcript.
• Speaker: nikolai-sheiko — AI-adoption consultant/practitioner (works with client companies on SDLC adoption; background otherwise unknown)
• Ingestion date: 2026-07-30

Core Claims

• AI in development already delivers real gains, but people, companies and metrics throttle it by an order of magnitude. The real jump started Dec 2025 (Opus 4.5 / GPT-5.2 + Claude Code / Codex); the SDLC collapsed into days/hours — but not fully: two human "red squares" remain — the reviewer (tasks queue at review) and the planner/product person. Developer time redistributed from "coding in the middle" to "planning on the left + verification on the right." See review-is-the-new-bottleneck.
• Universal error #0: a developer is not a manager. A good developer is 3–5 hours of CPU-bound focus on one feature; a good AI-developer is an IO-bound manager of an agent-employee, running several tasks in parallel. "If you launched Claude Code and sit watching it work — you're a bad employee." Not everyone can make the psychological switch, and that's fine — don't force everyone. See developer-as-agent-manager.
• Measure completed tasks without rework — never LoC, commit count or PR count (all trivially hacked; the European-outsourcing case shipped more PRs for a +1% gain because rework ate everything). A task counts only if it doesn't come back for fixes; also track task lifetime + rework time.
• Review with the agent, not instead of it and not fully by hand. Manual-only review → queue → burnout → quality collapse; fully delegated review is the opposite error. Treat the model as a smart student: direct it, pose hypotheses, find problems together.
• Companies no longer need custom AI development. Key quote: "Come in, install Claude Code or Codex, configure everything, attach connectors, think about security — and it works better than any custom build." Corollary anti-pattern: hiring an external configurator who leaves behind a "magic artifact" nobody owns — teams must configure their own tools; what a company should buy is a teacher/curator, not a setup.
• Agentic Evolution — the key concept. Don't ask the expert to explain how they work (you get theory); instead take the new employee (the agent) by the hand through hard tasks, show it the rakes, then say: "remember all of this and write the manual for the next one." Verify a skill by launching a context-free subagent that must solve the same task from scratch using only the skill; the mentor agent watches what fails and fixes the skill. Without evolution you live on defaults; with it, vertical growth begins.
• Best practices matter more with agents, not less. The "compaction curse": on a huge codebase the agent gathers context → window overflows → compaction → re-gathers → compaction again, and the task barely completes. The cure is locality, isolated modules with interfaces — the codebase stores the context. Use AST search instead of grep on colossal projects. And embeddings/RAG over code do not work — don't use them unless you understand very well why.
• Role futures: a Product engineer emerges (answers why we build it this way, what to cut, what to ignore); users vs Agentic Operations (who tune the SDLC, feedback loops, prompts, skills) gradually split; AI eats Intelligence (action sequences requiring intellect) while Judgment (taste built over years, or domain expertise — oil & gas, medicine) stays human for now.
• Tokens get more expensive near-term, cheaper later. It's the wild west — experiment at full throttle while subscriptions are cheap; the goal is to land in the top half of the Stanford chart. Don't chase every new tool: what Claude Code / Codex doesn't absorb within a couple of months is probably useless.

Key Evidence / Details

• Timeline the talk builds on: METR study (Jul 2025, measured a slowdown, but methodologically contested) → Stanford study (Aug 2025, +20% speed but top-performers pull away — this is swepr's research, cited independently) → Karpathy's tweet (Nov–Dec 2025: "80% Claude Code, 20% by hand") → "SDLC is dead" article (stages collapse into days/hours).
• Case 1 (frontend migration): no feedback loop → give the agent a browser (Playwright / Chrome DevTools) to compare old vs new front; crunch → invest in planning (20 min minimum, hours are normal; every 10 min of planning saves hours; target one-shot implementation); Cursor with per-token billing → team economizes instead of experimenting (~30% dearer than subscriptions at the same level); training rollout: record sessions → expert reviews → writes feedback, team fixes their own agent instructions → focus on the top 2 performers (an hour with them is worth 10× more).
• Case 2 (European outsourcer): more PRs, +1% — rework was the cause; the metrics table (don't: LoC/commits/PRs; do: completed-tasks-without-rework).
• Case 3 (large codebase): the compaction curse; "agents mean we can drop best practices" is exactly wrong.
• Mini-cases: the middle dev who sped the team up by tens of %, was refused a raise, and left for much more ("if you're that middle — think; if you're the manager — think twice"); the startup doing spec-driven development without knowing what it wants — build the UI first (even with an in-memory browser DB), click around, then plan.
• Do-tomorrow list: close the feedback loop; write a skill that analyses your own sessions daily; automate it (Codex schedules / Anthropic routines); voice input (more context, and Russian gives more context than English); ignore tool churn.
• Q&A notes: GLM (good, no vision), Kimi (works, has vision), Xiaomi's agent (interesting memory implementation; ex-DeepSeek Head of AI) as the Chinese-model trend; hardware deficit (H100s unrentable); speaker's prediction that AI will be classed as a cyber-weapon with biotech-style licensing.
• One-shot recipe: feedback loop (must-have) + explicit real goal the agent self-checks against + skill verification via the context-free subagent.

Connections

• Entities: nikolai-sheiko, swepr (its Stanford study is the talk's central chart — "be in the top half"), claude-code
• New concepts: review-is-the-new-bottleneck, developer-as-agent-manager
• Corroborates: 2026-07-30-stanford-swepr-widening-gap (independent practitioner citation of the widening-gap result and of review-as-downstream-cost); solve-first-then-skillify (Agentic Evolution is its strongest formulation, plus the missing verification step); skills-as-memory (a second frontier-practitioner vote for the skills/evolution layer, against thorsten-ball's dissent — and a vote against embeddings/RAG over code, siding with Konstantin in the skills-vs-RAG contradiction); context-as-scarce-resource (compaction curse; codebase-stores-context converges with Thorsten from the opposite direction); async-by-default (IO-bound parallel management as the working mode); enterprise-ai-reality ("no custom AI development needed" names the same managed-harness market; per-token billing shapes behaviour — the metered-vs-subscription split already logged there); harness (install-and-configure beats custom builds); leave-less-room-for-imagination (explicit goal + planning discipline).
• Complicates: make-more-cheap-code (rework-free-completion as the metric is the org-level answer to "generation moved the cost downstream").

Open Questions

• The cases are anonymous client anecdotes with self-reported numbers (+1%, "tens of %"); none are verifiable. Status: tentative.
• "Embeddings/RAG over code don't work" is stated flatly with no mechanism given — strong claim, no evidence in the doc. Status: tentative.
• Does the context-free-subagent verification protocol actually measure skill quality, or only skill completeness for one task? Closely related to the corpus's proposed skills falsification test (skills-as-memory) — this is the first source to describe running one.
• The Dec-2025 "real jump" periodization is the speaker's own; the corpus's other timeline (ai-agent-evolution) slices eras differently.
• Predictions (AI as licensed cyber-weapon; token prices up then down) are speculation. Status: tentative.

Change Impact on Wiki

• Created nikolai-sheiko (entity), review-is-the-new-bottleneck and developer-as-agent-manager (concepts).
• Updated solve-first-then-skillify (Agentic Evolution + skill-verification protocol), skills-as-memory (second practitioner vote for skills; anti-RAG-for-code), context-as-scarce-resource (compaction curse, AST search, codebase-stores-context), async-by-default (IO-bound manager evidence), enterprise-ai-reality (no-custom-AI-dev quote; external-configurator anti-pattern; token-price prediction), make-more-cheap-code (related link), 2026-07-30-stanford-swepr-widening-gap (independent citation).
• Updated overview (12 → 13 sources), index.md, log.md.