@ssheleg/agent-stack 0.17.0 → 0.18.0

package/CHANGELOG.md

@@ -1,5 +1,94 @@
+## v0.18.0 — how many runs before a difference is real, and the trajectory rule between its two measured edges
+
+`agent-evals` shipped 315 lines of doctrine about **what** to assert with **no
+`references/` directory at all**, and said nothing anywhere about how many times to run
+anything. A skill whose whole job is answering *did it get better* had no way to say
+whether a number was a result or noise.
+
+**`references/statistics.md`** is that layer, and every figure in it was recomputed rather
+than quoted:
+
+- `SE(p) = √(p(1−p)/n)` — at n=100, p=0.70 the 95% band is **±8.98 pp**, so a 73%-vs-70%
+  comparison on a hundred cases is a number inside its own noise. Error falls as `1/√n`,
+  which makes the remedy *more tasks*, not more argument.
+- **`pass@k` and `pass^k` differ by 91 points on the same agent** — at p=0.6, k=5 they are
+  99.0% and 7.8%. The first is a capability ceiling a human picks from; the second is what
+  a payment or a permission change needs. An operation with side effects may not "retry
+  until it works", so `pass@k` is not available to it as a metric at all.
+- **Trials are not independent, and the published data proves it.** τ-bench's airline
+  Pass^k for claude-3-5-sonnet runs 0.460 → 0.326 → 0.263 → 0.225, where independence from
+  Pass^1 would predict 0.460 → 0.212 → 0.097 → 0.045. Successes cluster by task, not by
+  trial. So `pass^k` cannot be computed from `pass^1`, and Anthropic's `0.75³ ≈ 42%` is the
+  right shape for an argument and the wrong number for a gate. **This one is ours** — it
+  came out of recomputing the table rather than restating it.
+- Pairing on the same tasks and the same 3–5 seeds with McNemar or a paired bootstrap,
+  because the task-difficulty variance the previous point measures is exactly what pairing
+  removes.
+- **The harness is a variable**: 6 pp between the most- and least-resourced setups on
+  Terminal-Bench 2.0 (p<0.01), moving within noise from 1× to 3× (p=0.40) and lifting ~4 pp
+  from 3× to uncapped — because generous headroom lets the agent attempt strategies a tight
+  cap forbids. Two caps measure two agents. The remedy is a floor *and* a ceiling.
+- A ladder for what a given piece of evidence authorises next, ending on the rule people
+  skip: **4/4 on a slice is not 100% system-wide.**
+
+**§5's trajectory rule moved between two measurements rather than being deleted.** It read
+*"Judge the trajectory, not just the answer. Right tools, right order, right arguments."*
+Anthropic calls exact tool-order assertions *"too rigid … agents regularly find valid
+approaches that eval designers didn't anticipate"*, with a worked case of an agent that
+solved a τ²-bench booking task through a policy loophole and failed the eval while serving
+the user better. But the opposite edge is measured too: a grader blind to the trajectory
+misses **44% of safety violations and 13% of robustness failures**. So the rule now reads
+*read the trajectory; do not match it* — assert what was produced and what changed, and use
+the trajectory as a **set and a forbidden list** for the claims an outcome cannot carry.
+§2's axis table lost its `→` sequence example for the same reason.
+
+The single-step example keeps its *"must call `find_meeting_times` first"*, with a sentence
+saying why: at that granularity the fixture **is** one decision, so ordering is the subject
+rather than a proxy for it. Across a trajectory it stops being one.
+
 # Changelog
 
+## v0.17.1 — the evals run for the first time, and the tails go to zero
+
+Wave-3 of the 2026-08-29 family audit (rows AST-05, AST-07, AST-08, AST-09, AST-10),
+plus the two board rows wave 2 filed (AST-A1, AST-B). All fixes and docs — no
+description or trigger changes, hence a patch.
+
+- **AST-05: the eval suite is executed for the first time.** `test/evals/RESULTS.md`
+  gains two dated rows — haiku and sonnet, 2026-08-31 — each a fresh blind subagent
+  per trigger query against the family's 28 skill descriptions, plus all three
+  scenarios scored line by line. Both models: 11/12 triggers. Scenario lines: 10/12
+  (haiku), 9/12 (sonnet). Each miss is named — haiku false-triggered `agent-harness`
+  on "rewrite this one system prompt so it sounds friendlier" (q09), sonnet answered
+  `none` on the Russian harness-audit query (q04) — and the Method section states the
+  protocol and its three limits rather than presenting the rates as clean-room.
+- **AST-B: the generator-evaluator citation lands where the doctrine lives.**
+  `agent-evals` §5a now cites Anthropic's *tuning a standalone evaluator to be
+  skeptical is more tractable than making a generator self-critical* (read
+  2026-08-30) beside the sentence making the same claim, marked as convergence
+  rather than invention — beside the first executed eval run, as the board row asked.
+- **AST-A1: the nine-subsystem coverage check is done and the row closes.** The ninth
+  subsystem the filing row could not name is **Automation**; all nine map to existing
+  doctrine with a `file:line` each, and the suspected identity/approval-policy gap
+  resolves as a split verdict — approval policy covered (`governance.md`),
+  authentication mechanics a named delegation (`layers.md:78-80`), not a hole. The
+  map lives in the board row (`docs/evidence/backlog.md`), not in a new reference.
+- **AST-07: the README stops counting its references.** "Twenty references" had
+  drifted to 24 actual files within a week of being written; the aggregate count is
+  dropped in favour of the per-skill counts that are still true, with the reason
+  stated in place.
+- **AST-10: the orchestrator's reference index lists all 11 references.** The three
+  memory rows — `memory-architecture.md`, `memory-lifecycle.md`,
+  `memory-landscape.md` — join the index table, each paraphrasing its own
+  "Load this when" line.
+- **AST-08: `license: MIT` in all four skill front matters** (was 1 of 4 —
+  `agent-evals` alone carried it). Every front matter re-checked with
+  `yaml.safe_load` after the edit.
+- **AST-09: `$schema` in both manifests** — `claude-code-plugin-manifest.json` for
+  `plugin.json`, `claude-code-marketplace.json` for `marketplace.json`, the two
+  schemastore addresses that resolve (the third candidate, `claude-code-plugin.json`,
+  is a 404 and two siblings currently point at it; referred, not copied).
+
 ## v0.17.0 — the descriptions stop colliding, and the claims get their dates
 
 Wave-2 of the 2026-08-29 family audit (rows AST-02, AST-03, AST-04, AST-06, AST-11), plus

package/README.md

@@ -33,8 +33,9 @@ Part of the [ssheleg skill family](https://github.com/ssheleg/sshlg-skills).
 
 Four skills — `agent-orchestrator` for wiring the loop, `agent-evals` for proving it
 behaves, `agent-interop` for everything it talks to outside its own process,
-`agent-harness` for what it is **told** — and twenty references they load on demand,
-plus one scanner.
+`agent-harness` for what it is **told** — and the references they load on demand,
+plus one scanner. The per-skill sections below carry their own counts; no total is
+stated here, because the last one drifted stale within a week of being written.
 
 **The orchestrator** (`SKILL.md`) — what the agent reads first:
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@ssheleg/agent-stack",
-  "version": "0.17.0",
+  "version": "0.18.0",
   "scripts": {
     "test": "python3 test/validate.py && python3 test/plant_guard_test.py && node test/installer_test.js"
   },

package/plugins/agent-stack/.claude-plugin/plugin.json

@@ -1,8 +1,9 @@
 {
+  "$schema": "https://json.schemastore.org/claude-code-plugin-manifest.json",
   "name": "agent-stack",
   "displayName": "Agent Stack",
   "description": "Four skills: agent-orchestrator — tool-calling loops, pipelines with checkpoints, provider routing with fallback, memory architecture, plus the wallet side of reselling LLM access; agent-evals — run/trace/thread evals, LLM judges, and fixtures grown from production; agent-interop — MCP servers and clients, A2A agent cards, the MCP Registry, and gateways; agent-harness — system prompts, tool shaping, workflow-vs-agent, and auditing an agent system.",
-  "version": "0.17.0",
+  "version": "0.18.0",
   "author": {
     "name": "ssheleg",
     "url": "https://x.com/sshlg93"

package/plugins/agent-stack/skills/agent-evals/SKILL.md

@@ -63,6 +63,10 @@ point — tool name, argument shape.
 > "Schedule a meeting with Harrison tomorrow morning", with `find_meeting_times`,
 > `schedule_meeting` and `send_email` available, must call `find_meeting_times` first.
 
+This is the one granularity where *first* is a legitimate assertion: the fixture is a
+single decision, so the ordering claim is the subject rather than a proxy for it. Across a
+whole trajectory it stops being one — see §5.
+
 Cheap, deterministic, CI-blocking. **Precondition: a stable agent architecture.** These
 break on a graph refactor, and a suite that fails on every refactor gets deleted.
 
@@ -72,7 +76,7 @@ Assert on three axes at once, with three different mechanisms:
 
 | Axis | Assert | With |
 |---|---|---|
-| Trajectory | tool-call sequence — `read_file` → `edit_file` → `run_tests` | set/subset/order matchers |
+| Trajectory | what the run **must not** do, and what it must have touched — never the order | set/subset matchers, forbidden-call lists |
 | Final response | quality, tone, policy compliance | rubric or judge |
 | **State change** | the memory row exists, the file was written, the artifact is there | direct inspection of the side effect |
 
@@ -147,8 +151,20 @@ That is what makes a judge reproducible and a disagreement resolvable.
 assertions, tool-call correctness — all deterministic, all faster and cheaper than a model
 call. Send to a judge only what cannot be decided by code.
 
-**Judge the trajectory, not just the answer.** Right tools, right order, right arguments.
-An agent that reaches a correct answer through three wrong tool calls is a latent outage.
+**Read the trajectory; do not match it.** An agent that reaches a correct answer through
+three wrong tool calls is a latent outage — and asserting the *sequence* to catch that is
+measurably the wrong instrument. Anthropic names the instinct and rejects its strict form:
+exact tool-order assertions are *"too rigid and results in overly brittle tests, as agents
+regularly find valid approaches that eval designers didn't anticipate"*, and the worked
+case is an agent that solved a τ²-bench booking task through a policy loophole, failing
+the eval as written while serving the user better. Grade **what was produced and what
+changed**, and let the path vary.
+
+The opposite edge is measured too, so this is not "grade the final answer": a grader blind
+to the trajectory misses **44% of safety violations and 13% of robustness failures**,
+because a policy breach on the way to a correct result leaves no trace in the outcome. Use
+the trajectory for the claims the outcome cannot carry — a forbidden call, a missing
+confirmation, a secret read — as a **set and a forbidden list**, never as an order.
 
 **Calibrate the judge before trusting it.** Collect human labels on the same traces,
 measure agreement, iterate the judge prompt until agreement is high — *then* let it score
@@ -168,6 +184,13 @@ the one most systems are missing: a **checker node** sitting between a parallel
 the node that consumes it. Its only job is *usable / not usable*, and the convergence
 depends on **it** rather than on the branches — otherwise the gate has a bypass.
 
+**That split is measured practice, not only this pack's position.** Anthropic's
+harness-design guidance (`anthropic.com/engineering/harness-design-long-running-apps`,
+read 2026-08-30) reports that *tuning a standalone evaluator to be skeptical is more
+tractable than making a generator self-critical* — the same reason the verdict belongs to
+a separate node rather than to the branches grading themselves. Dated and cited so a
+reader can tell doctrine that converged with the field from doctrine invented here.
+
 It matters here because it is the same machinery as §5, positioned differently:
 
 <!-- checker-contract: missing, empty, unevidenced, malformed, contradictory, off-topic | optional: under-confident -->
@@ -297,6 +320,16 @@ None of the above runs without these, and they are the part people skip:
 - [ ] Every production failure minimised into a permanent fixture
 - [ ] Annotation queue with filters, and the two reviewer roles kept separate
 - [ ] Simulated users trained on real transcripts, with adversarial personas
+- [ ] Every comparison that changes a decision states `n`, `k`, the reducer and the noise
+      band — `references/statistics.md`; a three-point gap on a hundred cases is noise
+
+---
+
+## References
+
+| Load | When |
+|---|---|
+| [`references/statistics.md`](references/statistics.md) | a number is about to change a decision — how many runs before a difference is real, `pass@k` vs `pass^k` and why trials are not independent, paired comparison, the harness as a variable, and what a given piece of evidence authorises next |
 
 ---
 

package/plugins/agent-stack/skills/agent-evals/references/statistics.md

@@ -0,0 +1,201 @@
+# Statistics for agent evals — how many runs before a difference is real
+
+**Load this when a number is about to change a decision:** picking between two models or
+two harness configurations, setting a release threshold, deciding whether a regression is
+real, or writing the sentence "X is better than Y" in a document somebody will act on.
+
+Everything here is arithmetic over binary outcomes. It needs no library and no
+statistician, and it is the layer most eval advice skips — including this skill's own
+first eight sections, which say what to assert and never say how many times.
+
+## Contents
+
+- [The band around a pass rate](#the-band-around-a-pass-rate)
+- [pass@k and pass^k are different questions](#passk-and-passk-are-different-questions)
+- [Trials are not independent, and the published data says so](#trials-are-not-independent-and-the-published-data-says-so)
+- [Pairing: same tasks, same seeds, per-task deltas](#pairing-same-tasks-same-seeds-per-task-deltas)
+- [The harness is a variable, so pin it](#the-harness-is-a-variable-so-pin-it)
+- [What a number authorises](#what-a-number-authorises)
+- [The report contract](#the-report-contract)
+
+---
+
+## The band around a pass rate
+
+A pass rate over `n` cases is an estimate, and its standard error is
+
+```
+SE(p) = sqrt( p * (1 - p) / n )
+```
+
+The 95% band is roughly `±1.96 · SE`. Computed, not quoted:
+
+| n | p | 95% band |
+|---|---|---|
+| 100 | 0.70 | **±8.98 pp** |
+| 400 | 0.70 | ±4.49 pp |
+| 1000 | 0.70 | ±2.84 pp |
+
+```python
+import math
+def band(p, n): return 1.96 * math.sqrt(p * (1 - p) / n) * 100   # percentage points
+```
+
+**So "the new one gets 73% where the old one got 70%, on a hundred cases" is not a
+result.** It is a number inside its own noise. The error shrinks as `1/√n`, which is the
+whole practical consequence: **the fix for a 2–3 pp expected gain is more tasks, not more
+argument.** Quadrupling the set halves the band.
+
+A corollary worth stating because leaderboards invite the opposite: **differences below
+about 3 pp deserve scepticism until both configurations are documented and matched.**
+
+> The formula assumes independent cases. A benchmark whose tasks share a fixture, an
+> environment or a generator violates that, and the true band is wider than this. Wider,
+> never narrower — so the table is a floor on your uncertainty, not a ceiling.
+
+## pass@k and pass^k are different questions
+
+Two metrics, one letter apart, measuring opposite things.
+
+| Metric | Formula | Asks | Who it is for |
+|---|---|---|---|
+| `pass@k` | `1 − (1 − p)^k` | did **at least one** of k attempts succeed | a **capability ceiling** — a human picks the best of k |
+| `pass^k` | `p^k` (if independent) | did **every** one of k succeed | a **reliability floor** — nobody is picking |
+
+At `p = 0.6`, `k = 5`:
+
+- `pass@5 = 1 − 0.4⁵ = 0.98976` → **99.0%**
+- `pass^5 = 0.6⁵ = 0.07776` → **7.8%**
+
+**A 91-point gap between two numbers describing the same agent.** The first makes a demo;
+the second is what a payment, a refund or a permission change actually needs.
+
+Anthropic states the same arithmetic for the everyday case: at a 75% per-trial rate, three
+trials all passing is `0.75³ ≈ 42%` — verified, `0.421875`.
+
+**The rule that follows: an operation with side effects may not "retry until it works."**
+If a failed attempt leaves a charge, a message or a mutated row behind, `pass@k` is not
+available to you as a metric — you cannot pick the best of five refunds. Sample in a
+sandbox or a rollback-capable environment, and count **every** failure.
+
+**A report that gives k without saying which k it means is unreadable.** *k independent
+samples of one task* and *k consecutive tasks on one live pipeline* are different claims.
+
+## Trials are not independent, and the published data says so
+
+`pass^k = p^k` assumes each trial is a fresh coin flip. Real benchmarks do not behave that
+way, and the τ-bench leaderboard is the cleanest demonstration — claude-3-5-sonnet on the
+airline domain, published Pass^k beside what independence would predict from Pass^1:
+
+| k | published Pass^k | `0.460^k` if independent |
+|---|---|---|
+| 1 | 0.460 | 0.460 |
+| 2 | **0.326** | 0.212 |
+| 3 | **0.263** | 0.097 |
+| 4 | **0.225** | 0.045 |
+
+The observed curve falls far slower than independence predicts. The reason is not
+mysterious: **some tasks are reliably easy and some reliably hard**, so successes cluster
+by task rather than scattering by trial. Positive correlation across trials of the same
+task.
+
+Two consequences, and they cut in opposite directions:
+
+- **You cannot compute `pass^k` from `pass^1`.** Exponentiating a headline rate gives a
+  number far below the truth. Measure `pass^k` directly, at the k you care about.
+- **Anthropic's `0.75³ ≈ 42%` is a worst case, not a forecast.** It is the right shape for
+  an argument — *consistency is a much harder bar* — and the wrong number to put in a
+  release gate.
+
+The other half of independence is the harness, not the task: Anthropic requires each trial
+start from a clean environment, because *"unnecessary shared state between runs (leftover
+files, cached data, resource exhaustion) can cause correlated failures."* Correlated
+failures break the arithmetic above — so isolation is not hygiene, it is what makes the
+metric mean anything.
+
+## Pairing: same tasks, same seeds, per-task deltas
+
+**Never subtract two independent averages.** Run both configurations over the *same* task
+list with the *same* fixed seeds, record a per-task win/loss/tie, and test the deltas.
+
+```
+for task in tasks:            # identical list
+    for seed in seeds:        # identical seeds, 3-5 of them
+        a = run(config_A, task, seed)
+        b = run(config_B, task, seed)
+        delta[task, seed] = a - b
+```
+
+- **3–5 seeds per configuration**, reporting mean and spread. A single run screens a
+  direction; it does not establish one.
+- Test the paired deltas with **McNemar** (binary outcomes) or a **paired bootstrap**.
+  Pairing removes the task-difficulty variance that dominates the unpaired comparison —
+  which is exactly the correlation the previous section measured.
+- Testing several hypotheses at once needs a **multiple-comparisons correction**, or an
+  independent re-run of whichever ones came out positive. Five hypotheses at p<0.05 gives
+  you roughly a one-in-four chance of a false positive somewhere.
+
+**Ship on three conditions, not one:** the difference exceeds the noise band, it survives
+the paired analysis, and it reproduces on a rerun.
+
+## The harness is a variable, so pin it
+
+The container spec is part of the measurement. On Terminal-Bench 2.0 the gap between the
+most- and least-resourced setups was **6 percentage points (p < 0.01)** — larger than most
+model differences anyone argues about.
+
+The shape of the effect matters more than the number:
+
+- Between **1× and 3×** the task's specified resources, scores move **within noise
+  (p = 0.40)**.
+- From 3× to uncapped, infrastructure errors drop a further **1.6 pp** and success jumps
+  nearly **4 pp** — because the extra headroom lets the agent attempt strategies that only
+  work with it: pulling large dependencies, spawning expensive subprocesses, running
+  memory-hungry suites.
+- Infrastructure errors alone: **5.8% of tasks at 1×**, cut to **2.1% at a 3× ceiling
+  (p < 0.001)**.
+
+So a tight cap and a generous cap **measure different agent strategies**, not the same
+agent more or less precisely. The remedy is a **floor and a ceiling**, calibrated so that
+scores at both fall within noise of each other — not a single pinned value.
+
+The effect is task-distribution dependent and does not transfer: SWE-bench moves only
+**1.54 pp from 1× to 5×**. Measure it for your own suite rather than importing a multiplier.
+
+## What a number authorises
+
+Evidence licenses the next action its scope supports, and nothing further. A worked
+three-round loop on a deliberately narrow slice — four tasks, one run each, model, seed,
+step limit and environment fixed, arm order alternated, **one variable changed per round**:
+
+| Round | The only change | Success | Tokens vs control | What it authorised |
+|---|---|---|---|---|
+| H1 | added navigation and final-check instructions | 25% → 25% | 0.47× | the prompt is not the bottleneck — stop tuning it |
+| H5 | accessibility feed → UIAutomator tree | 25% → **100%** | **2.498×** | right mechanism, too expensive — try to cheapen it |
+| H5C | prune invisible/textless/non-actionable nodes | 100% → 100% | **0.506×** | qualifies for a full rerun |
+
+Two rules fall out, and the second is the one people skip:
+
+- **Observation before prompt.** More detailed instructions cannot restore information the
+  agent never received. When a score will not move, ask what the agent could see before
+  asking how it was asked.
+- **4/4 on a slice is not 100% system-wide, and must not be reported as one.** With four
+  tasks per arm these numbers can decide whether a larger rerun is worth paying for. They
+  cannot estimate success across the benchmark. The ladder's output is *the next
+  experiment*, not a result.
+
+## The report contract
+
+A comparison that will change a decision states all of these, or it is a claim rather than
+a measurement:
+
+- [ ] `n` — how many tasks, and `k` — how many runs each
+- [ ] which reducer: `pass@k`, `pass^k`, `mean`, and **which k means what**
+- [ ] the noise band for that `n`, computed
+- [ ] paired or unpaired; if paired, the seed set
+- [ ] the harness configuration: resource floor and ceiling, isolation between trials
+- [ ] whether the difference reproduced on a rerun
+- [ ] what the scope of the evidence authorises next — not what it suggests
+
+**A green suite with none of these is a number, not a verdict.** That distinction is the
+reason this file exists.

package/plugins/agent-stack/skills/agent-harness/SKILL.md

@@ -13,6 +13,7 @@ description: >-
   "embed an agent", "agent SDK", "Pi harness", "системный промпт", "агент не вызывает тул",
   "аудит агента", "встроить агента". Not for the loop's plumbing, its evals, or its
   protocols — those are siblings.
+license: MIT
 ---
 
 # Agent harness — what the agent is told, and how to audit what someone else told theirs

package/plugins/agent-stack/skills/agent-interop/SKILL.md

@@ -12,6 +12,7 @@ description: >-
   "agentgateway", "tool federation", "MCP-сервер", "карточка агента", "интероперабельность
   агентов", "реестр MCP", "шлюз для агентов". Not for designing one server's tool set, nor for
   a skill's own construction — that is make-skill.
+license: MIT
 ---
 
 # Agent interop — the protocols an agent speaks outside its own process

package/plugins/agent-stack/skills/agent-orchestrator/SKILL.md

@@ -12,6 +12,7 @@ description: >-
   loop", "memory layer", "LLM billing", "token wallet", "агентная система", "оркестратор",
   "суб-агент", "роутер моделей", "человек в цикле", "слой памяти", "биллинг LLM", "граф задач".
   Not for a single LLM call in a script, or prompt wording.
+license: MIT
 ---
 
 # Agent Orchestrator — Production Best Practices
@@ -245,9 +246,7 @@ answer built from nothing, in no error log.
 **Design a memory layer from
 [`references/memory-architecture.md`](references/memory-architecture.md)**, not from this
 table — it also carries the context-budget trap, layer 0 carryover and workspace scale.
-The write path is [`references/memory-lifecycle.md`](references/memory-lifecycle.md);
-what to build on and measure with is
-[`references/memory-landscape.md`](references/memory-landscape.md).
+The write path and the build-or-adopt question have their own rows in the index below.
 ## 8. Self-Learning Feedback Loops
 
 Three cycles feed the memory layers, and they differ by what supplies the signal: a failed
@@ -373,15 +372,18 @@ heading — the ones that were learned by getting them wrong:
 ## References
 
 Each file opens with its own **Load this when** line — the authoritative trigger lives
-there, so this table stays an index and the two cannot drift apart.
+there; this table is only an index.
 
 | File | Read it when |
 |---|---|
-| [`references/graph-engineering.md`](references/graph-engineering.md) | you are deciding the **shape of the work** — the fake-edge test, the diamond, the checker node, static versus dynamic, and what the host actually runs |
+| [`references/graph-engineering.md`](references/graph-engineering.md) | you are deciding the **shape of the work** — fake edges, the diamond, the checker node, static versus dynamic |
 | [`references/pipeline.md`](references/pipeline.md) | one loop is **not the shape** — the planned path, its checkpoints, resume, and the interrupt that asks a person |
 | [`references/patterns.md`](references/patterns.md) | you need the **data models and algorithms** under the body |
+| [`references/memory-architecture.md`](references/memory-architecture.md) | an agent is **being given memory**, or memory is misbehaving |
+| [`references/memory-lifecycle.md`](references/memory-lifecycle.md) | the memory **write path** — what enters, and what leaves |
+| [`references/memory-landscape.md`](references/memory-landscape.md) | **build or adopt** a memory layer, and what is settled practice |
 | [`references/context-engineering.md`](references/context-engineering.md) | the loop is **running out of window** |
 | [`references/runtime.md`](references/runtime.md) | the agent must **survive a crash, a pause, a second message or a schedule** |
 | [`references/governance.md`](references/governance.md) | the question is **permission, not cost** — what it may do, and how you prove it |
 | [`references/llm-proxy-billing.md`](references/llm-proxy-billing.md) | the product **resells LLM access** |
-| [`references/provider-lifecycle.md`](references/provider-lifecycle.md) | the question is the **workforce, not the loop** — where providers come from, produced-once/bound-many, knowledge packs, canary trust, workspace lifecycle, fleet budgets |
+| [`references/provider-lifecycle.md`](references/provider-lifecycle.md) | the question is the **workforce, not the loop** — provider lifecycle, canary trust, fleet budgets |