Files
WebinarNotes/wiki/concepts/skills-as-memory.md
EugeneTes 3314112bb9 ingest: Stanford SWEPR widening-gap study and AI-in-SDLC adoption pitfalls
Add two new sources with summaries, new concepts (developer-as-agent-manager,
review-is-the-new-bottleneck), new entities (SWEPR, Nikolai Sheiko), and a
query on the Stanford source; update related concept pages, overview, index,
and log.
2026-07-31 08:33:56 +02:00

54 lines
9.3 KiB
Markdown

# Skills as Memory
#concept
## Summary
A **skill** is a directory + a gentleman's agreement: a `SKILL.md` describing *what to do*, plus optional tools (bash/python) and — Konstantin's key proposal — the **data and usage history** in the same git repo. Combined this way, skills become **the new memory of AI agents**: information about the user lives in skills, loaded exactly when a skill activates, rather than pre-fetched by RAG.
## Current Understanding
The unlock is **two-stage loading**: a short description always sits in the system prompt (the agent knows the skill exists); the long `SKILL.md` loads only when the agent enters the directory. This lets a harness hold *hundreds* of skills without blowing [[context-as-scarce-resource|context]] — impossible with raw tools, where a couple of calls exhaust context. Skills also **compose** (one skill calls another) and are **portable** (plain markdown folders migrate Claude → Perplexity → Gemini).
A classic skill is "**von Neumann without data**" (code, no data). Adding data + history makes the skill self-contained; storing it in **git** enables skill-first architecture (many agents on one skill), CI/CD integration, team sharing, and merge-conflict resolution by models. Skills can **auto-improve**: create a skill after >5 tool calls on a task; a curator prunes unused skills (30/90 days) and consolidates weekly. See [[hermes]].
The **method** for populating skills is [[solve-first-then-skillify]]: reach the final solution once, then freeze it (Eugene's variant of the heuristic: any correction loop longer than ~3 messages becomes a skill). The HR interviews add a social payoff: a packaged skill is a **handoff/de-risking asset** — a junior "with not even a third of your HR experience" can deliver a decent result, and the expert can take a vacation.
**The dissent: a frontier practitioner who skips skills entirely.** [[thorsten-ball]] — 99% of his company's code is AI-written — reports "**no custom slash commands, no skills, no MCP servers**" ([[2026-07-28-agentic-engineering-10x-developer]]). His substitute is not weaker context but *differently located* context: the codebase itself, a team-maintained `AGENTS.md`, and a long prompt written "like a Slack message to a senior engineer" (set the standard → state intent → riff on design → specify process → set sub-agent economics). He agrees with this page's premise — "where does the agent get its information from?" is the only question he thinks matters — and rejects its mechanism. Recorded as a live contradiction under *Contradictions* below rather than reconciled.
**The negative case: built-in memory as anti-feature.** The [[2026-07-21-larysa-interview|Larysa interview]] supplies the demand-side reason this architecture exists. Her core frustration is that the agent doesn't carry context between sessions — she re-explains, and re-pays in time and tokens. Eugene's answer is not "better memory" but *no* memory: "Memory is the worst thing agents have — it gives no benefit and confuses users to hell. Why even go there? … The memory exists, but the way it's implemented, it'd be better if it didn't." The claim is that an opaque, always-on memory that silently decides what to recall is worse than nothing, because the user can neither inspect nor correct it — whereas a skill is a file you can read, edit, version and delete. Skills are the memory you *author*.
## Evidence
- Skill definition, two-stage loading, data-in-git proposal, curator, auto-creation — [[2026-07-14-skills-based-on-git]].
- Skill = folder with one MD + resources; skills compose and transfer across tools/models; built-in skill-creator — [[2026-07-14-gap-between-ai-users-irreversible]].
- Markdown-file-as-service (G-brain tier) is the same "executable prose" instinct — [[2026-07-14-everything-we-knew-about-software-has-changed]].
- Skill as zip-and-hand-over onboarding asset; "create a skill for this" — [[2026-07-14-nina-interview]].
- ~3-message correction-loop heuristic; skills as the non-programmer ceiling (with CLAUDE.md) — [[2026-07-14-yulia-interview]].
- Cross-session memory loss as the #1 practitioner pain; "memory is the worst thing agents have"; skills committed as the webinar remedy — [[2026-07-21-larysa-interview]].
- Counter-evidence: no skills, no MCP, no slash commands at a 99%-AI-written company; `AGENTS.md` + codebase + rich prompt as the substitute — [[2026-07-28-agentic-engineering-10x-developer]].
- **A second practitioner vote *for* the skills layer** (2026-07-30): [[nikolai-sheiko]]'s "Agentic Evolution" makes skill-building-plus-verification the difference between living on defaults and "vertical growth," and he flatly rejects embeddings/RAG over code ("don't use them unless you understand *very* well why") — siding with the load-on-activation camp in the skills-vs-RAG contradiction below. His verification protocol (context-free subagent re-solves the task from the skill alone) is the closest thing yet to the falsification test this page asks for — [[2026-07-30-rakes-in-ai-sdlc-adoption]].
## Related Pages
- Concepts: [[evolution-of-agent-tooling]] (tools → MCP → skills), [[harness]], [[context-as-scarce-resource]], [[agentic-loops]], [[personal-ai-operating-system]], [[solve-first-then-skillify]], [[levels-of-ai-usage]], [[leave-less-room-for-imagination]]
- Tools: [[hermes]], [[claude-code]], [[amp]] (a harness with vendor-curated sub-agents and no user-authored skills layer)
- Entities: [[konstantin]], [[allie-miller]], [[eugene]], [[larysa]], [[thorsten-ball]]
## Contradictions / Uncertainty
- **Skills vs RAG** (recorded here 2026-07-28 by lint; previously logged only on [[context-as-scarce-resource]]). This page's Summary asserts load-on-activation beats RAG pre-injection as settled mechanism, but [[2026-07-22-ai-is-stupid]] names **RAG and long-term assistant memory** as *the* practical context mechanisms for a business audience. Possibly not a real disagreement — business *data* may want RAG where *procedures* want skills — but the corpus has never separated the two cases. Status: tentative.
- No standards yet for *what* data to put in a skill or its size limit (Konstantin: 200 GB in one, 100 KB in another, both fine — "ceiling not found"). Status: tentative.
- "Built-in memory is a net negative" is Eugene's strong position, not a corpus consensus — [[allie-miller]]'s [[personal-ai-operating-system]] happily uses persistent context docs and never condemns the memory feature. The two are reconcilable (both prefer *authored* context to *inferred* context), but the blanket "better if it didn't exist" is one voice. Status: tentative.
- **Skills may be unnecessary at the frontier** (added 2026-07-28). [[thorsten-ball]] ships 99%-AI-written code with no skills, no MCP and no slash commands. Three readings, none settled by the corpus:
1. **Situational.** He works daily in *one codebase he controls*, where context can live in the code and `AGENTS.md`. Skills earn their keep when work is spread across many ad-hoc tasks with no codebase to encode into — which is exactly the corpus's HR/BA audience ([[nina]], [[yulia]], [[larysa]]). Under this reading both are right and the disagreement is about who is speaking.
2. **The abstraction is premature.** Skills are scaffolding for models that needed it; a strong model plus a rich prompt plus a good repo may simply beat a skills library, making the whole layer a 2025 artifact. This is the uncomfortable reading for the webinar's central promise.
3. **He has skills under another name.** AMP's Oracle/Painter/Puck sub-agents and a maintained `AGENTS.md` *are* curated, reusable, two-stage context — just authored by the vendor and the team rather than the user. Under this reading the dispute is about who curates, not whether curation is needed.
Status: tentative. Note the evidential asymmetry — his is a first-hand report of daily practice at scale, where the pro-skills case rests on Konstantin's architecture argument plus self-reported individual workflows. *(Asymmetry softened 2026-07-30: [[nikolai-sheiko]] adds a second practitioner voice on the pro-skills side, from multi-company adoption work rather than one codebase — though his cases are anonymous anecdotes, so the readings above remain unsettled.)* **Presentation-safe restatement:** [[2026-07-28-webinar-theses]] reframes the claim as *context you author beats context that's inferred*, which holds under all three readings — Konstantin's skills, Allie's foundation docs, Eugene's anti-memory position and Thorsten's `AGENTS.md` are all authored context.
## Next Questions
- ~~What's a starter skill set for a non-engineer?~~ Answered in [[2026-07-14-best-first-skill-for-beginner]] (skill-creator as meta-skill; tone-of-voice + anti-AI-language as first content skill).
- Do skills actually solve *cross-project* context, or only per-procedure recall? Larysa's complaint may be the former, which skills don't obviously address.
- Is there a test that would separate reading 1 from reading 2 above? The cheapest one available: give a non-engineer the same task with and without a skill and compare drift — the corpus has never run it, and the webinar's promise rests on the answer. *(Adjacent evidence 2026-07-30: Sheiko's context-free-subagent protocol — [[solve-first-then-skillify]] — runs the with-skill half in practice, but never the without-skill control, so the question stands.)*