ingest: Stanford SWEPR widening-gap study and AI-in-SDLC adoption pitfalls
Add two new sources with summaries, new concepts (developer-as-agent-manager, review-is-the-new-bottleneck), new entities (SWEPR, Nikolai Sheiko), and a query on the Stanford source; update related concept pages, overview, index, and log.
This commit is contained in:
36
wiki/entities/nikolai-sheiko.md
Normal file
36
wiki/entities/nikolai-sheiko.md
Normal file
@@ -0,0 +1,36 @@
|
||||
# Nikolai Sheiko
|
||||
|
||||
#entity
|
||||
|
||||
## Summary
|
||||
|
||||
Speaker of the talk "Грабли во внедрении ИИ в SDLC" ([[2026-07-30-rakes-in-ai-sdlc-adoption]]). Russian-speaking AI-adoption practitioner/consultant who works with client companies (a frontend-migration team, a European outsourcer, a large-codebase company) on getting real results from AI in the software lifecycle. Background beyond the talk unknown.
|
||||
|
||||
## Current Understanding
|
||||
|
||||
His through-line: the models are already good enough — **people, companies and metrics are what throttle the gains**. Signature positions:
|
||||
|
||||
- The AI-developer is a **manager of an agent-employee**, IO-bound and parallel ([[developer-as-agent-manager]]).
|
||||
- **Review is the new bottleneck**; review *with* the agent; measure completed-tasks-without-rework, never LoC/PRs ([[review-is-the-new-bottleneck]]).
|
||||
- **Agentic Evolution**: walk the agent through hard tasks, then have it write the manual; verify skills with a context-free subagent ([[solve-first-then-skillify]]).
|
||||
- Companies should install and configure Claude Code / Codex rather than build custom AI tooling; buy a **teacher/curator**, not an external configurator ([[enterprise-ai-reality]]).
|
||||
- Best practices matter *more* with agents (compaction curse, AST search over grep); embeddings/RAG over code don't work ([[context-as-scarce-resource]]).
|
||||
- AI eats **Intelligence**; **Judgment** (taste or domain expertise) stays human for now.
|
||||
|
||||
## Evidence
|
||||
|
||||
- All positions — [[2026-07-30-rakes-in-ai-sdlc-adoption]].
|
||||
|
||||
## Related Pages
|
||||
|
||||
- Concepts: [[developer-as-agent-manager]], [[review-is-the-new-bottleneck]], [[solve-first-then-skillify]], [[enterprise-ai-reality]], [[context-as-scarce-resource]]
|
||||
- Entities: [[swepr]] (he cites their Stanford chart as the stakes — "be in the top half"), [[thorsten-ball]] (fellow frontier practitioner; they disagree on skills — Sheiko builds skill evolution, Thorsten uses none)
|
||||
|
||||
## Contradictions / Uncertainty
|
||||
|
||||
- His client cases are anonymous and self-reported; no numbers are verifiable. Status: tentative.
|
||||
- Identity/affiliation beyond the talk unknown (the raw doc names only the talk itself). Status: tentative.
|
||||
|
||||
## Next Questions
|
||||
|
||||
- Who is he professionally — independent consultant, agency, vendor? Affects how to weigh the "you don't need custom AI development" claim (it is also a consultant's pitch).
|
||||
35
wiki/entities/swepr.md
Normal file
35
wiki/entities/swepr.md
Normal file
@@ -0,0 +1,35 @@
|
||||
# SWEPR (Stanford Software Engineering Productivity Research)
|
||||
|
||||
#entity
|
||||
|
||||
## Summary
|
||||
|
||||
Research group at Stanford University measuring software-engineering productivity from private Git data — 600+ companies, ~100k–120k engineers since 2022 — using an ML model that replicates a panel of expert code reviewers. Public face: researcher **Yegor Denisov-Blanch**. Source of the corpus's only quantitative outside study of AI's productivity impact.
|
||||
|
||||
## Current Understanding
|
||||
|
||||
- Site: https://softwareengineeringproductivity.stanford.edu/ — offers an "AI Practices Benchmark" and "AI Impact" research to participating companies.
|
||||
- Known for three results: the **widening-gap DiD analysis** (46 vs 46 teams, gap 4.8% → 19%, 4×, Apr 2023–Jul 2025), the **~15–20% net average gain** figure (after rework; 30–40% gross), and the earlier **"ghost engineers"** finding (~9.5% of engineers show virtually no verifiable output).
|
||||
- Methodology is peer-reviewed (arXiv 2409.15152, 2502.20747); the headline AI-impact analyses are talk/deck-published (Sept 2025 AI Conference deck "Will AI Replace Software Engineers?").
|
||||
- Their measurement philosophy — functionality delivered, not commits or LOC — aligns with the corpus's own suspicion of volume metrics ([[make-more-cheap-code]]: generation volume is the *cheap* part).
|
||||
|
||||
## Evidence
|
||||
|
||||
- All claims and links — [[2026-07-30-stanford-swepr-widening-gap]].
|
||||
|
||||
## Related Pages
|
||||
|
||||
- Sources: [[2026-07-30-stanford-swepr-widening-gap]]
|
||||
- Concepts: [[levels-of-ai-usage]] (team-level twin of the mastery gap), [[context-as-scarce-resource]] (codebase-size finding), [[make-more-cheap-code]] (review-time shift)
|
||||
- Queries: [[2026-07-30-stanford-widening-gap-source]]
|
||||
- Entities: [[allie-miller]] (her prediction, their measurement)
|
||||
|
||||
## Contradictions / Uncertainty
|
||||
|
||||
- Data is proprietary and opt-in — no outside replication possible; participating companies may skew toward the measurement-friendly. Status: tentative.
|
||||
- The widening-gap analysis itself is not peer-reviewed as of 2026-07-30. Status: tentative.
|
||||
|
||||
## Next Questions
|
||||
|
||||
- Track whether the DiD analysis lands in a peer-reviewed venue.
|
||||
- What do they say *causes* the gap (tooling vs hygiene vs practices)? The talks assert "quality of usage" without decomposing it.
|
||||
Reference in New Issue
Block a user