WebinarNotes/wiki/concepts/context-as-scarce-resource.md

Context as the Scarce Resource

#concept

Summary

Across sources, the binding constraint on agents is context, not model IQ. Managing it — what you load, when, and how long the agent stays sharp — is the core engineering discipline. Konstantin: the model has a "smart zone" (roughly the first third of context) where it's sharpest; beyond that it dulls, even with a million-token window.

Current Understanding

Context pressure explains several otherwise-separate design choices:

• Tool ceilings — every injected tool costs context, so harnesses cap at ~30–40 tools and >100 confuses even Fable 5. See harness.
• Two-stage skill loading — short descriptions always visible, long SKILL.md loaded on demand — is the trick that lets hundreds of skills-as-memory coexist. See evolution-of-agent-tooling.
• Ralph loop keeps the agent in the smart zone by restarting instead of letting context grow → summarize → decay. See agentic-loops.
• Skills-as-memory > RAG — the harness decides what to load when a skill activates, rather than RAG pre-injecting facts before the model even starts.
• Context engineering (Allie) — "feeding the system who you are" via foundation docs — is the same idea from the user side. See personal-ai-operating-system.

The human role has climbed prompt-engineer → context-engineer → harness-builder → loop-engineer, tracking exactly this concern.

Information beats tuning (2026-07-28-agentic-engineering-10x-developer). thorsten-ball states the strongest version: once you have a frontier model, the dominant variable in output quality is the information you put in — not which model, and not the effort level (medium vs high vs ultra). "If you're mad your model doesn't use camelCase, rethink your software engineering, not the model." He names the agent's only two information sources — training data (a senior engineer who's seen it all, but lossy and possibly stale) and the context window (your prompt, plus whatever the codebase and AGENTS.md supply) — and the operative asymmetry: a model cannot turn a thin prompt into a good one. Note what he does not conclude: the fix is a better-tended codebase and a longer prompt, not skills-as-memory (contested there).

Context now has a price — at fleet scale. The same source names token budget as one of two variables separating winners from losers, alongside knowing how to use agents. Context has always been scarce per-request; this is the corpus's first claim that it is also scarce per-wallet. Scope, corrected 2026-07-28: the claim comes from metered usage (parallel remote sandboxes), and under a flat consumer subscription the corpus's own heavy users report no ceiling — so per-request scarcity remains the binding constraint for individuals, and per-wallet scarcity is a fleet and enterprise concern. See enterprise-ai-reality and explosion-of-internal-software.

The compaction curse — scarcity at codebase scale (2026-07-30-rakes-in-ai-sdlc-adoption). On a huge codebase the agent gathers context → the window overflows → compaction → it re-gathers → compaction again, and the task barely completes. nikolai-sheiko's cure inverts the "agents mean best practices don't matter" fallacy: best practices exist for the agent now — locality, isolated modules with interfaces, so that the codebase stores the context. This converges with thorsten-ball's context-lives-in-the-codebase position from the opposite direction (Thorsten skips skills because his codebase carries context; Sheiko says make your codebase able to). Practical additions: AST search instead of grep on colossal projects (grep returns a wall of noise, AST returns the relevant slice), and a flat rejection of embeddings/RAG over code. The mechanism matches Stanford's measured gains-collapse toward 10M LOC (evidence below).

The supply-side facet (2026-07-22-ai-is-stupid): before context is scarce it is usually absent. "Intelligence without context loses to context without intelligence" — ten Nobel laureates asked about your sales month can only cite industry averages, while your rank-and-file employee answers better because they see your funnel, clients, and deals. The default "stupid AI" experience is a strong model given neither business context nor a harness; the fix is investing in context infrastructure (data, memory, integrations) before reaching for a bigger model.

Evidence

• Smart zone, summarization decay, "context is the most valuable resource," tool/skill loading mechanics — 2026-07-14-skills-based-on-git.
• Context engineering vs prompt engineering; foundation docs as durable context — 2026-07-14-gap-between-ai-users-irreversible.
• "Intelligence without context loses"; Nobel-vs-employee analogy; invest in context before model upgrades — 2026-07-22-ai-is-stupid.
• Information > model choice > effort level; the two information sources; token budget as a winner/loser variable — 2026-07-28-agentic-engineering-10x-developer.
• Reading costs attention — the human-side analog of the same scarcity — make-more-cheap-code, 2026-07-24-youre-reading-way-too-much-code.
• First outside quantitative support: Stanford SWEPR finds AI productivity gains collapse as codebases grow from ~10k to ~10M lines, attributing it to context-window limits and signal-to-noise degradation — the corpus's context-is-the-constraint claim, measured at scale — 2026-07-30-stanford-swepr-widening-gap.
• Compaction curse; best-practices-for-the-agent (locality, interfaces, codebase-stores-context); AST search over grep; embeddings/RAG over code rejected — 2026-07-30-rakes-in-ai-sdlc-adoption.

Related Pages

• Concepts: harness, skills-as-memory, agentic-loops, evolution-of-agent-tooling, personal-ai-operating-system, make-more-cheap-code, explosion-of-internal-software, enterprise-ai-reality
• Entities: konstantin, allie-miller, thorsten-ball

Contradictions / Uncertainty

• "First third = smart zone" is a heuristic, not a measured boundary; likely model-dependent. Status: tentative.
• 2026-07-22-ai-is-stupid names RAG and long-term assistant memory as the practical context mechanisms; 2026-07-14-skills-based-on-git argues skills-as-memory beat RAG (load-on-activation vs pre-injection). Possibly audience-driven (business data vs procedures) rather than a real disagreement. Status: tentative. (2026-07-30: nikolai-sheiko adds a hard anti-RAG data point for the code domain specifically — "embeddings over code don't work" — which supports the audience-driven reading: the anti-RAG votes are both about code/procedures, the pro-RAG vote is about business data.)

Next Questions

• How do you measure where a given model's smart zone actually ends?