3314112bb9
Add two new sources with summaries, new concepts (developer-as-agent-manager, review-is-the-new-bottleneck), new entities (SWEPR, Nikolai Sheiko), and a query on the Stanford source; update related concept pages, overview, index, and log.
52 lines
6.8 KiB
Markdown
52 lines
6.8 KiB
Markdown
# Context as the Scarce Resource
|
||
|
||
#concept
|
||
|
||
## Summary
|
||
|
||
Across sources, the binding constraint on agents is **context**, not model IQ. Managing it — what you load, when, and how long the agent stays sharp — is the core engineering discipline. Konstantin: the model has a **"smart zone"** (roughly the first third of context) where it's sharpest; beyond that it dulls, even with a million-token window.
|
||
|
||
## Current Understanding
|
||
|
||
Context pressure explains several otherwise-separate design choices:
|
||
|
||
- **Tool ceilings** — every injected tool costs context, so harnesses cap at ~30–40 tools and >100 confuses even Fable 5. See [[harness]].
|
||
- **Two-stage skill loading** — short descriptions always visible, long `SKILL.md` loaded on demand — is *the* trick that lets hundreds of [[skills-as-memory|skills]] coexist. See [[evolution-of-agent-tooling]].
|
||
- **Ralph loop** keeps the agent in the smart zone by restarting instead of letting context grow → summarize → decay. See [[agentic-loops]].
|
||
- **Skills-as-memory > RAG** — the harness decides what to load *when a skill activates*, rather than RAG pre-injecting facts before the model even starts.
|
||
- **Context engineering** (Allie) — "feeding the system who you are" via foundation docs — is the same idea from the user side. See [[personal-ai-operating-system]].
|
||
|
||
The human role has climbed prompt-engineer → **context-engineer** → harness-builder → loop-engineer, tracking exactly this concern.
|
||
|
||
**Information beats tuning** ([[2026-07-28-agentic-engineering-10x-developer]]). [[thorsten-ball]] states the strongest version: once you have a frontier model, **the dominant variable in output quality is the information you put in** — not which model, and not the effort level (medium vs high vs ultra). "If you're mad your model doesn't use camelCase, rethink your software engineering, not the model." He names the agent's only two information sources — **training data** (a senior engineer who's seen it all, but lossy and possibly stale) and **the context window** (your prompt, plus whatever the codebase and `AGENTS.md` supply) — and the operative asymmetry: a model cannot turn a thin prompt into a good one. Note what he does *not* conclude: the fix is a better-tended codebase and a longer prompt, not [[skills-as-memory|skills]] (contested there).
|
||
|
||
**Context now has a price — at fleet scale.** The same source names **token budget** as one of two variables separating winners from losers, alongside knowing how to use agents. Context has always been scarce per-request; this is the corpus's first claim that it is also scarce per-*wallet*. Scope, corrected 2026-07-28: the claim comes from **metered** usage (parallel remote sandboxes), and under a flat consumer subscription the corpus's own heavy users report no ceiling — so per-request scarcity remains the binding constraint for individuals, and per-wallet scarcity is a fleet and enterprise concern. See [[enterprise-ai-reality]] and [[explosion-of-internal-software]].
|
||
|
||
**The compaction curse — scarcity at codebase scale** ([[2026-07-30-rakes-in-ai-sdlc-adoption]]). On a huge codebase the agent gathers context → the window overflows → compaction → it re-gathers → compaction again, and the task barely completes. [[nikolai-sheiko]]'s cure inverts the "agents mean best practices don't matter" fallacy: **best practices exist for the agent now** — locality, isolated modules with interfaces, so that *the codebase stores the context*. This converges with [[thorsten-ball]]'s context-lives-in-the-codebase position from the opposite direction (Thorsten skips skills because his codebase carries context; Sheiko says make your codebase able to). Practical additions: **AST search instead of grep** on colossal projects (grep returns a wall of noise, AST returns the relevant slice), and a flat rejection of embeddings/RAG over code. The mechanism matches Stanford's measured gains-collapse toward 10M LOC (evidence below).
|
||
|
||
**The supply-side facet** ([[2026-07-22-ai-is-stupid]]): before context is *scarce* it is usually *absent*. "Intelligence without context loses to context without intelligence" — ten Nobel laureates asked about your sales month can only cite industry averages, while your rank-and-file employee answers better because they see your funnel, clients, and deals. The default "stupid AI" experience is a strong model given neither business context nor a [[harness]]; the fix is investing in context infrastructure (data, memory, integrations) before reaching for a bigger model.
|
||
|
||
## Evidence
|
||
|
||
- Smart zone, summarization decay, "context is the most valuable resource," tool/skill loading mechanics — [[2026-07-14-skills-based-on-git]].
|
||
- Context engineering vs prompt engineering; foundation docs as durable context — [[2026-07-14-gap-between-ai-users-irreversible]].
|
||
- "Intelligence without context loses"; Nobel-vs-employee analogy; invest in context before model upgrades — [[2026-07-22-ai-is-stupid]].
|
||
- Information > model choice > effort level; the two information sources; token budget as a winner/loser variable — [[2026-07-28-agentic-engineering-10x-developer]].
|
||
- Reading costs attention — the human-side analog of the same scarcity — [[make-more-cheap-code]], [[2026-07-24-youre-reading-way-too-much-code]].
|
||
- **First outside quantitative support:** Stanford SWEPR finds AI productivity gains collapse as codebases grow from ~10k to ~10M lines, attributing it to context-window limits and signal-to-noise degradation — the corpus's context-is-the-constraint claim, measured at scale — [[2026-07-30-stanford-swepr-widening-gap]].
|
||
- Compaction curse; best-practices-for-the-agent (locality, interfaces, codebase-stores-context); AST search over grep; embeddings/RAG over code rejected — [[2026-07-30-rakes-in-ai-sdlc-adoption]].
|
||
|
||
## Related Pages
|
||
|
||
- Concepts: [[harness]], [[skills-as-memory]], [[agentic-loops]], [[evolution-of-agent-tooling]], [[personal-ai-operating-system]], [[make-more-cheap-code]], [[explosion-of-internal-software]], [[enterprise-ai-reality]]
|
||
- Entities: [[konstantin]], [[allie-miller]], [[thorsten-ball]]
|
||
|
||
## Contradictions / Uncertainty
|
||
|
||
- "First third = smart zone" is a heuristic, not a measured boundary; likely model-dependent. Status: tentative.
|
||
- [[2026-07-22-ai-is-stupid]] names **RAG** and long-term assistant memory as the practical context mechanisms; [[2026-07-14-skills-based-on-git]] argues [[skills-as-memory|skills]] beat RAG (load-on-activation vs pre-injection). Possibly audience-driven (business data vs procedures) rather than a real disagreement. Status: tentative. *(2026-07-30: [[nikolai-sheiko]] adds a hard anti-RAG data point for the code domain specifically — "embeddings over code don't work" — which supports the audience-driven reading: the anti-RAG votes are both about code/procedures, the pro-RAG vote is about business data.)*
|
||
|
||
## Next Questions
|
||
|
||
- How do you measure where a given model's smart zone actually ends?
|