WebinarNotes/wiki/sources/2026-07-30-rakes-in-ai-sdlc-adoption.md

# Грабли во внедрении ИИ в SDLC — Rakes in AI Adoption in the SDLC (Nikolai Sheiko)

#source

## Source Metadata

- **Date:** talk published 2026 (references events through Dec 2025); conclusions doc saved 2026-07-30
- **Raw path:** `raw/sources/Грабли во внедрении ИИ в SDLC.md`
- **Source type:** viewer's conclusions from a Russian-language YouTube talk (45:59) — https://www.youtube.com/watch?v=Nm3MsnngCJg — "Грабли во внедрении ИИ в SDLC — почему ИИ есть, а результата нет и как это лечить" ("why the AI is there but the results aren't, and how to treat it"). Not a transcript.
- **Speaker:** [[nikolai-sheiko]] — AI-adoption consultant/practitioner (works with client companies on SDLC adoption; background otherwise unknown)
- **Ingestion date:** 2026-07-30

## Core Claims

- **AI in development already delivers real gains, but people, companies and metrics throttle it by an order of magnitude.** The real jump started Dec 2025 (Opus 4.5 / GPT-5.2 + Claude Code / Codex); the SDLC collapsed into days/hours — but *not fully*: two human "red squares" remain — the **reviewer** (tasks queue at review) and the **planner/product person**. Developer time redistributed from "coding in the middle" to "planning on the left + verification on the right." See [[review-is-the-new-bottleneck]].
- **Universal error #0: a developer is not a manager.** A good developer is 3–5 hours of CPU-bound focus on one feature; a good AI-developer is an IO-bound **manager of an agent-employee**, running several tasks in parallel. "If you launched Claude Code and sit watching it work — you're a bad employee." Not everyone can make the psychological switch, *and that's fine* — don't force everyone. See [[developer-as-agent-manager]].
- **Measure completed tasks without rework** — never LoC, commit count or PR count (all trivially hacked; the European-outsourcing case shipped more PRs for a +1% gain because rework ate everything). A task counts only if it doesn't come back for fixes; also track task lifetime + rework time.
- **Review with the agent, not instead of it and not fully by hand.** Manual-only review → queue → burnout → quality collapse; fully delegated review is the opposite error. Treat the model as a smart student: direct it, pose hypotheses, find problems together.
- **Companies no longer need custom AI development.** Key quote: *"Come in, install Claude Code or Codex, configure everything, attach connectors, think about security — and it works better than any custom build."* Corollary anti-pattern: hiring an external configurator who leaves behind a "magic artifact" nobody owns — teams must configure their own tools; what a company should buy is a **teacher/curator**, not a setup.
- **Agentic Evolution** — the key concept. Don't ask the expert to explain how they work (you get theory); instead take the new employee (the agent) by the hand through hard tasks, show it the rakes, then say: *"remember all of this and write the manual for the next one."* Verify a skill by launching a **context-free subagent** that must solve the same task from scratch using only the skill; the mentor agent watches what fails and fixes the skill. Without evolution you live on defaults; with it, vertical growth begins.
- **Best practices matter more with agents, not less.** The "compaction curse": on a huge codebase the agent gathers context → window overflows → compaction → re-gathers → compaction again, and the task barely completes. The cure is locality, isolated modules with interfaces — **the codebase stores the context**. Use AST search instead of grep on colossal projects. And **embeddings/RAG over code do not work** — don't use them unless you understand *very* well why.
- **Role futures:** a **Product engineer** emerges (answers *why* we build it this way, what to cut, what to ignore); users vs **Agentic Operations** (who tune the SDLC, feedback loops, prompts, skills) gradually split; AI eats **Intelligence** (action sequences requiring intellect) while **Judgment** (taste built over years, or domain expertise — oil & gas, medicine) stays human for now.
- **Tokens get more expensive near-term, cheaper later. It's the wild west — experiment at full throttle while subscriptions are cheap**; the goal is to land in the top half of the Stanford chart. Don't chase every new tool: what Claude Code / Codex doesn't absorb within a couple of months is probably useless.

## Key Evidence / Details

- **Timeline the talk builds on:** METR study (Jul 2025, measured a *slowdown*, but methodologically contested) → Stanford study (Aug 2025, +20% speed but top-performers pull away — this is [[swepr]]'s research, cited independently) → Karpathy's tweet (Nov–Dec 2025: "80% Claude Code, 20% by hand") → "SDLC is dead" article (stages collapse into days/hours).
- **Case 1 (frontend migration):** no feedback loop → give the agent a browser (Playwright / Chrome DevTools) to compare old vs new front; crunch → invest in planning (20 min minimum, hours are normal; every 10 min of planning saves hours; target one-shot implementation); Cursor with per-token billing → team economizes instead of experimenting (~30% dearer than subscriptions at the same level); training rollout: record sessions → expert reviews → *writes feedback, team fixes their own agent instructions* → focus on the top 2 performers (an hour with them is worth 10× more).
- **Case 2 (European outsourcer):** more PRs, +1% — rework was the cause; the metrics table (don't: LoC/commits/PRs; do: completed-tasks-without-rework).
- **Case 3 (large codebase):** the compaction curse; "agents mean we can drop best practices" is exactly wrong.
- **Mini-cases:** the middle dev who sped the team up by tens of %, was refused a raise, and left for much more ("if you're that middle — think; if you're the manager — think twice"); the startup doing spec-driven development without knowing what it wants — build the UI first (even with an in-memory browser DB), click around, *then* plan.
- **Do-tomorrow list:** close the feedback loop; write a skill that analyses your own sessions daily; automate it (Codex schedules / Anthropic routines); voice input (more context, and Russian gives more context than English); ignore tool churn.
- **Q&A notes:** GLM (good, no vision), Kimi (works, has vision), Xiaomi's agent (interesting memory implementation; ex-DeepSeek Head of AI) as the Chinese-model trend; hardware deficit (H100s unrentable); speaker's prediction that AI will be classed as a cyber-weapon with biotech-style licensing.
- **One-shot recipe:** feedback loop (must-have) + explicit real *goal* the agent self-checks against + skill verification via the context-free subagent.

## Connections

- **Entities:** [[nikolai-sheiko]], [[swepr]] (its Stanford study is the talk's central chart — "be in the top half"), [[claude-code]]
- **New concepts:** [[review-is-the-new-bottleneck]], [[developer-as-agent-manager]]
- **Corroborates:** [[2026-07-30-stanford-swepr-widening-gap]] (independent practitioner citation of the widening-gap result *and* of review-as-downstream-cost); [[solve-first-then-skillify]] (Agentic Evolution is its strongest formulation, plus the missing verification step); [[skills-as-memory]] (a second frontier-practitioner vote *for* the skills/evolution layer, against [[thorsten-ball]]'s dissent — and a vote *against* embeddings/RAG over code, siding with Konstantin in the skills-vs-RAG contradiction); [[context-as-scarce-resource]] (compaction curse; codebase-stores-context converges with Thorsten from the opposite direction); [[async-by-default]] (IO-bound parallel management as the working mode); [[enterprise-ai-reality]] ("no custom AI development needed" names the same managed-harness market; per-token billing shapes behaviour — the metered-vs-subscription split already logged there); [[harness]] (install-and-configure beats custom builds); [[leave-less-room-for-imagination]] (explicit goal + planning discipline).
- **Complicates:** [[make-more-cheap-code]] (rework-free-completion as the metric is the org-level answer to "generation moved the cost downstream").

## Open Questions

- The cases are anonymous client anecdotes with self-reported numbers (+1%, "tens of %"); none are verifiable. Status: tentative.
- "Embeddings/RAG over code don't work" is stated flatly with no mechanism given — strong claim, no evidence in the doc. Status: tentative.
- Does the context-free-subagent verification protocol actually measure skill quality, or only skill *completeness* for one task? Closely related to the corpus's proposed skills falsification test ([[skills-as-memory]]) — this is the first source to describe running one.
- The Dec-2025 "real jump" periodization is the speaker's own; the corpus's other timeline ([[ai-agent-evolution]]) slices eras differently.
- Predictions (AI as licensed cyber-weapon; token prices up then down) are speculation. Status: tentative.

## Change Impact on Wiki

- Created [[nikolai-sheiko]] (entity), [[review-is-the-new-bottleneck]] and [[developer-as-agent-manager]] (concepts).
- Updated [[solve-first-then-skillify]] (Agentic Evolution + skill-verification protocol), [[skills-as-memory]] (second practitioner vote for skills; anti-RAG-for-code), [[context-as-scarce-resource]] (compaction curse, AST search, codebase-stores-context), [[async-by-default]] (IO-bound manager evidence), [[enterprise-ai-reality]] (no-custom-AI-dev quote; external-configurator anti-pattern; token-price prediction), [[make-more-cheap-code]] (related link), [[2026-07-30-stanford-swepr-widening-gap]] (independent citation).
- Updated [[overview]] (12 → 13 sources), `index.md`, `log.md`.