@ssheleg/agent-stack 0.13.5 → 0.15.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +49 -0
- package/README.md +1 -0
- package/package.json +2 -2
- package/plugins/agent-stack/.claude-plugin/plugin.json +1 -1
- package/plugins/agent-stack/skills/agent-harness/references/audit.md +1 -0
- package/plugins/agent-stack/skills/agent-orchestrator/SKILL.md +14 -13
- package/plugins/agent-stack/skills/agent-orchestrator/references/memory-architecture.md +308 -0
- package/plugins/agent-stack/skills/agent-orchestrator/references/patterns.md +35 -0
- package/plugins/agent-stack/skills/agent-orchestrator/references/provider-lifecycle.md +151 -0
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,54 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## v0.15.0 — memory architecture, and the axes the layer table does not have
|
|
4
|
+
|
|
5
|
+
- **`agent-orchestrator/references/memory-architecture.md`** — form, function and dynamics
|
|
6
|
+
as the three axes of a memory decision, with the layer table demoted to what it is: one
|
|
7
|
+
property of the answer. Source pinned with a read date — *Memory in the Age of AI Agents:
|
|
8
|
+
A Survey*, arXiv:2512.13564v2, 13 Jan 2026, read 2026-08-27.
|
|
9
|
+
- **Retrieval was two mentions across 1,938 lines of references**, and it is four decisions:
|
|
10
|
+
whether to retrieve at all and from which store, what query to retrieve with, which
|
|
11
|
+
strategy runs the search, what reaches the prompt. The first is the one nobody
|
|
12
|
+
instruments — an agent that overestimates its own knowledge and skips retrieval answers
|
|
13
|
+
confidently from nothing, with **no error, no empty result and no latency spike**.
|
|
14
|
+
- **Facts are now separated from experience, and by entity.** Layers 3 and 4 are
|
|
15
|
+
experiential; nothing in them is a factual store. A stale fact about the USER makes the
|
|
16
|
+
agent rude, a stale fact about the ENVIRONMENT makes it wrong — one expiry rule for both
|
|
17
|
+
is wrong twice.
|
|
18
|
+
- **Forgetting gains the long-tail trap.** Frequency-based eviction is the easy policy and
|
|
19
|
+
the one that deletes the rarely-read entry preventing the rare expensive mistake. Where
|
|
20
|
+
storage is not the binding constraint: demote, not delete.
|
|
21
|
+
- Also carried: forms beyond token-level and what each costs (parametric memory cannot be
|
|
22
|
+
selectively deleted; latent memory cannot be inspected), why `agent-sync`'s leases are
|
|
23
|
+
coordination rather than shared memory, and abstention under low-confidence retrieval —
|
|
24
|
+
which needs a similarity floor, because semantic search always returns K results and an
|
|
25
|
+
empty store looks identical to an irrelevant one.
|
|
26
|
+
- **§7 was split rather than trimmed.** The house auditor refused the first attempt at
|
|
27
|
+
4981 tokens against a 4750 working limit and named the remedy itself. The context-budget
|
|
28
|
+
trap, layer 0 carryover and workspace scale moved into the new reference; the body is now
|
|
29
|
+
**4659 tokens — below the 4708 it sat at before any of this**.
|
|
30
|
+
- Discoverability, measured: against *"agent memory architecture: forms, functions,
|
|
31
|
+
dynamics, retrieval, forgetting"* this skill ranked **8th** among installed skills and now
|
|
32
|
+
ranks **1st**. The description is 965 chars, inside the 970-char working limit.
|
|
33
|
+
|
|
34
|
+
## v0.14.1 — the workforce axis: provider lifecycle and workspace-scale memory
|
|
35
|
+
|
|
36
|
+
(v0.14.0 was burned during release engineering: its tag landed on a commit a
|
|
37
|
+
protected branch could never reach, and the tag rules forbid deletion — so the
|
|
38
|
+
content ships as v0.14.1 and the dead tag stays as its own cautionary receipt.)
|
|
39
|
+
|
|
40
|
+
The orchestrator gains `references/provider-lifecycle.md` — where providers come
|
|
41
|
+
from and how one earns trust: produced-once/bound-many, the production pipeline
|
|
42
|
+
with its named-consumer gate, knowledge packs whose traps become planted
|
|
43
|
+
fixtures, the canary binding with recorded promotion, the two-extension-mechanisms
|
|
44
|
+
law, workspace lifecycle with the dependency projection, and fleet budgets with
|
|
45
|
+
the run scheduler. `patterns.md` gains the workspace-scale memory rules — the
|
|
46
|
+
journal spine, rebuildable projections with embedding-model versions, isolation
|
|
47
|
+
at the API, promotion with decay, memory-through-the-bundle. The harness audit's
|
|
48
|
+
tools track now asks what the agent was actually equipped with: required,
|
|
49
|
+
installed, loaded — three truths with two receipts. Distilled from the Passion
|
|
50
|
+
Code fabric design review of 2026-08-27.
|
|
51
|
+
|
|
3
52
|
## v0.13.5 — the shared seam is explicit
|
|
4
53
|
|
|
5
54
|
Both shared validators now state `diverges: none`, completing the umbrella
|
package/README.md
CHANGED
|
@@ -4,6 +4,7 @@
|
|
|
4
4
|
[](https://www.npmjs.com/package/@ssheleg/agent-stack)
|
|
5
5
|
[](LICENSE)
|
|
6
6
|
[](https://skills.sshlg.me/skills/agent-stack/)
|
|
7
|
+
[](https://skills.sh/ssheleg/agent-stack)
|
|
7
8
|
|
|
8
9
|
**Build agent loops, harnesses, evals and protocol boundaries that stay inspectable under production failure.**
|
|
9
10
|
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@ssheleg/agent-stack",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.15.0",
|
|
4
4
|
"scripts": {
|
|
5
5
|
"test": "python3 test/validate.py && python3 test/plant_guard_test.py"
|
|
6
6
|
},
|
|
@@ -20,7 +20,7 @@
|
|
|
20
20
|
"SECURITY.md"
|
|
21
21
|
],
|
|
22
22
|
"repository": "github:ssheleg/agent-stack",
|
|
23
|
-
"homepage": "https://
|
|
23
|
+
"homepage": "https://skills.sshlg.me/skills/agent-stack/",
|
|
24
24
|
"license": "MIT",
|
|
25
25
|
"author": {
|
|
26
26
|
"name": "ssheleg",
|
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
"name": "agent-stack",
|
|
3
3
|
"displayName": "Agent Stack",
|
|
4
4
|
"description": "Two skills: agent-orchestrator — tool-calling loops, multi-stage pipelines with checkpoints, provider routing with fallback, four-layer memory, context engineering, plus the wallet side of reselling LLM access; and agent-evals — run/trace/thread evals, judges, and fixtures grown from production.",
|
|
5
|
-
"version": "0.
|
|
5
|
+
"version": "0.15.0",
|
|
6
6
|
"author": {
|
|
7
7
|
"name": "ssheleg",
|
|
8
8
|
"url": "https://x.com/sshlg93"
|
|
@@ -56,6 +56,7 @@ Walk them in order. Later tracks assume earlier ones.
|
|
|
56
56
|
- Is there a default limit on response size, or only an optional one?
|
|
57
57
|
- Do errors **name the next action**?
|
|
58
58
|
- Are destructive tools guarded by shape (`confirm: true`, absolute paths, enums) rather than by instruction?
|
|
59
|
+
- What was the agent **actually equipped with**? Three different truths — *required* by the task, *installed* on the machine, *loaded* by the session — and the receipts are the compiled bundle's lockfile and the session-init capability list. An audit that reads only the config file has checked the first truth of three.
|
|
59
60
|
|
|
60
61
|
### 3 — Control flow
|
|
61
62
|
|
|
@@ -3,8 +3,8 @@ name: agent-orchestrator
|
|
|
3
3
|
description: >-
|
|
4
4
|
Use when building an agent system — an orchestrator, an LLM-powered tool, a chatbot with tool
|
|
5
5
|
use, an AI pipeline — or metering and billing the LLM access it burns. Covers tool-calling
|
|
6
|
-
loops, pipelines with human checkpoints, provider routing with fallback/retry,
|
|
7
|
-
|
|
6
|
+
loops, pipelines with human checkpoints, provider routing with fallback/retry, memory
|
|
7
|
+
architecture, retrieval and decay, context budgets, sub-agent coordination, error hierarchies; the
|
|
8
8
|
work as a graph — parallel layers, fake edges, a checker before convergence; for resale:
|
|
9
9
|
tiered wallets, one markup boundary, two-phase commit across database and provider API,
|
|
10
10
|
spend-delta polling, budget and loop guards, per-tenant keys. Triggers - "agent",
|
|
@@ -232,19 +232,19 @@ Four layers, each with a different lifetime and a different reason to exist:
|
|
|
232
232
|
| 3 Long-term learnings | per resource | months | what worked, with a confidence score |
|
|
233
233
|
| 4 Insights | per project | permanent | conclusions that outlived their resource |
|
|
234
234
|
|
|
235
|
-
|
|
236
|
-
|
|
237
|
-
|
|
238
|
-
mechanisms that decide what actually enters layers 3 and 4.
|
|
235
|
+
What enters layers 3 and 4 is decided by `references/patterns.md` — **Confidence
|
|
236
|
+
Management**, **Learning Extraction Heuristics**, **Fuzzy Deduplication**, **Conflict
|
|
237
|
+
Resolution**.
|
|
239
238
|
|
|
240
|
-
**
|
|
241
|
-
|
|
242
|
-
|
|
243
|
-
|
|
244
|
-
|
|
239
|
+
**These four are lifetimes, and lifetime is not the taxonomy.** Layers 3 and 4 are
|
|
240
|
+
*experiential*; **nothing here is a factual store**, and a stale fact about the USER makes
|
|
241
|
+
the agent rude while one about the ENVIRONMENT makes it wrong. Retrieval is absent here and
|
|
242
|
+
is four decisions, the first of which — whether to retrieve at all — fails as a confident
|
|
243
|
+
answer built from nothing, in no error log.
|
|
245
244
|
|
|
246
|
-
**
|
|
247
|
-
|
|
245
|
+
**Design a memory layer from
|
|
246
|
+
[`references/memory-architecture.md`](references/memory-architecture.md)**, not from this
|
|
247
|
+
table. It also carries the context-budget trap, layer 0 carryover and workspace scale.
|
|
248
248
|
## 8. Self-Learning Feedback Loops
|
|
249
249
|
|
|
250
250
|
Three cycles feed the memory layers, and they differ by what supplies the signal: a failed
|
|
@@ -381,3 +381,4 @@ there, so this table stays an index and the two cannot drift apart.
|
|
|
381
381
|
| [`references/runtime.md`](references/runtime.md) | the agent must **survive a crash, a pause, a second message or a schedule** |
|
|
382
382
|
| [`references/governance.md`](references/governance.md) | the question is **permission, not cost** — what it may do, and how you prove it |
|
|
383
383
|
| [`references/llm-proxy-billing.md`](references/llm-proxy-billing.md) | the product **resells LLM access** |
|
|
384
|
+
| [`references/provider-lifecycle.md`](references/provider-lifecycle.md) | the question is the **workforce, not the loop** — where providers come from, produced-once/bound-many, knowledge packs, canary trust, workspace lifecycle, fleet budgets |
|
|
@@ -0,0 +1,308 @@
|
|
|
1
|
+
# Memory architecture — deciding what the agent remembers before deciding where to put it
|
|
2
|
+
|
|
3
|
+
**Load this when** an agent is being given memory for the first time, when an existing
|
|
4
|
+
memory layer is being extended, or when memory is present and behaving badly — recall that
|
|
5
|
+
misses, a store that grows without bound, an agent confidently answering from nothing.
|
|
6
|
+
|
|
7
|
+
**Spec pinned:** *Memory in the Age of AI Agents: A Survey — Forms, Functions and
|
|
8
|
+
Dynamics*, Hu et al., `https://arxiv.org/abs/2512.13564` (arXiv:2512.13564v2, 13 Jan 2026;
|
|
9
|
+
NUS / RUC / Fudan / PKU / NTU and others) · read 2026-08-27. Paper list:
|
|
10
|
+
`https://github.com/Shichun-Liu/Agent-Memory-Paper-List`.
|
|
11
|
+
|
|
12
|
+
`SKILL.md` §7 owns the four **layers** and their lifetimes, and §8 the loops that feed
|
|
13
|
+
them. Both answer *how long does this live*. **This file is upstream of that**: it decides
|
|
14
|
+
what kind of memory is being built at all, and lifetime is one property of the answer
|
|
15
|
+
rather than the taxonomy. The survey's own position, and the reason this file exists:
|
|
16
|
+
*"traditional taxonomies such as long/short-term memory have proven insufficient to capture
|
|
17
|
+
the diversity and dynamics of contemporary agent memory systems."*
|
|
18
|
+
|
|
19
|
+
## Contents
|
|
20
|
+
|
|
21
|
+
- [The source, and what this file adds](#the-source-and-what-this-file-adds)
|
|
22
|
+
- [1. Three axes, and lifetime is not one of them](#1-three-axes-and-lifetime-is-not-one-of-them)
|
|
23
|
+
- [2. Form — what physically carries it](#2-form--what-physically-carries-it)
|
|
24
|
+
- [3. Function — what it is for](#3-function--what-it-is-for)
|
|
25
|
+
- [4. Dynamics — formation, evolution, retrieval](#4-dynamics--formation-evolution-retrieval)
|
|
26
|
+
- [5. Retrieval is four decisions, not one](#5-retrieval-is-four-decisions-not-one)
|
|
27
|
+
- [5.5 The budget, layer 0, and workspace scale](#55-the-budget-layer-0-and-workspace-scale)
|
|
28
|
+
- [6. Forgetting, and the long-tail trap](#6-forgetting-and-the-long-tail-trap)
|
|
29
|
+
- [7. Shared memory when there is more than one agent](#7-shared-memory-when-there-is-more-than-one-agent)
|
|
30
|
+
- [8. Trustworthy memory](#8-trustworthy-memory)
|
|
31
|
+
- [9. What this pack does NOT claim](#9-what-this-pack-does-not-claim)
|
|
32
|
+
- [10. The checklist](#10-the-checklist)
|
|
33
|
+
|
|
34
|
+
## The source, and what this file adds
|
|
35
|
+
|
|
36
|
+
The survey is a map of published work, not a build guide: it classifies roughly two
|
|
37
|
+
hundred systems and states open problems. What is taken from it here is the **taxonomy and
|
|
38
|
+
the named failure modes**, because those are what an architect needs before writing a
|
|
39
|
+
schema. What is added is the decision order — form, then function, then dynamics — and the
|
|
40
|
+
wiring to what this pack already owns: §7's layers, §8's learning loops, `patterns.md`'s
|
|
41
|
+
confidence arithmetic, `context-engineering.md`'s budget, and `agent-evals` for judging
|
|
42
|
+
whether any of it helped.
|
|
43
|
+
|
|
44
|
+
**Where the survey and this pack disagree, the pack says so rather than quietly adopting.**
|
|
45
|
+
§7's four layers are a *deployment* shape that has survived contact with real systems; the
|
|
46
|
+
survey's three functions are an *analytical* shape. They are not rivals — the mapping is in
|
|
47
|
+
§3 below — and a schema built from either alone is missing what the other sees.
|
|
48
|
+
|
|
49
|
+
## 1. Three axes, and lifetime is not one of them
|
|
50
|
+
|
|
51
|
+
Every memory decision is three independent questions. Answer them in this order; answering
|
|
52
|
+
the third first is how a vector store arrives before anyone has said what it holds.
|
|
53
|
+
|
|
54
|
+
| Axis | The question | Answered in |
|
|
55
|
+
|---|---|---|
|
|
56
|
+
| **Form** | what physically carries it | §2 |
|
|
57
|
+
| **Function** | what it is FOR | §3 |
|
|
58
|
+
| **Dynamics** | how it is formed, evolved and retrieved | §4, §5, §6 |
|
|
59
|
+
|
|
60
|
+
Long-term and short-term are **not** a fourth axis. They are a consequence: the survey's
|
|
61
|
+
formulation is that short- and long-term effects *"emerge not from discrete architectural
|
|
62
|
+
modules but from the temporal patterns with which formation, evolution and retrieval are
|
|
63
|
+
engaged."* A single store read once per task behaves as short-term; the same store read
|
|
64
|
+
across tasks behaves as long-term. Design the patterns, and the lifetimes follow.
|
|
65
|
+
|
|
66
|
+
## 2. Form — what physically carries it
|
|
67
|
+
|
|
68
|
+
Three realizations. Most agent work uses the first and never learns the other two exist,
|
|
69
|
+
which is fine until the first one's costs bite.
|
|
70
|
+
|
|
71
|
+
| Form | What it is | Reach for it when | What it costs |
|
|
72
|
+
|---|---|---|---|
|
|
73
|
+
| **Token-level** | text, key-value, documents, graphs — anything the model reads as tokens | almost always; it is legible, auditable and editable by hand | every read spends context, and the store competes with the task for the window |
|
|
74
|
+
| **Parametric** | the knowledge is in weights — fine-tuning, model editing, adapters | a behaviour must hold with no retrieval step and no context cost, and it changes rarely | not auditable, not selectively deletable, and **wrong entries are expensive to remove** — which collides with §8's right to be forgotten |
|
|
75
|
+
| **Latent** | compressed internal states — KV reuse, latent tokens | throughput or privacy dominates and the content need not be read by a person | opaque: nobody can inspect what it holds, so a defect in it is invisible until behaviour is wrong |
|
|
76
|
+
|
|
77
|
+
Token-level splits further by structure — flat (1D), planar/tabular (2D), hierarchical or
|
|
78
|
+
graph (3D). Structure is a **retrieval** decision, not a storage one: a graph is worth its
|
|
79
|
+
cost when queries are multi-hop, and is overhead when they are lookups.
|
|
80
|
+
|
|
81
|
+
**The default, stated so it is a choice and not an accident:** token-level, flat, until a
|
|
82
|
+
measured retrieval failure justifies structure. Structure added before that is a schema
|
|
83
|
+
maintained for a query nobody runs.
|
|
84
|
+
|
|
85
|
+
## 3. Function — what it is for
|
|
86
|
+
|
|
87
|
+
Three pillars. The name matters because **the update rule and the trust level differ per
|
|
88
|
+
pillar**, and a store that mixes them applies one rule to all three.
|
|
89
|
+
|
|
90
|
+
| Function | Answers | Subtypes | Update rule |
|
|
91
|
+
|---|---|---|---|
|
|
92
|
+
| **Factual** | *what does the agent know* | **user** facts (identity, stable preferences, task constraints, commitments) · **environment** facts (document state, resource availability, what other agents can do) | corrected on contradiction; the newest assertion usually wins |
|
|
93
|
+
| **Experiential** | *how does the agent improve* | case-based (whole solutions and trajectories) · strategy-based (insights, workflows, patterns) · skill-based (functions, code, tools it wrote) | earned from a **contrast** — see §8 of `SKILL.md`; never from a single success |
|
|
94
|
+
| **Working** | *what is it thinking about now* | single-turn (input condensation, observation abstraction) · multi-turn (state consolidation, hierarchical folding) | discarded at the task boundary unless promoted |
|
|
95
|
+
|
|
96
|
+
**The split this pack was missing, and why it matters.** §7's layers do not separate *user*
|
|
97
|
+
facts from *environment* facts, and the two have opposite failure modes. A user fact that
|
|
98
|
+
goes stale makes the agent **rude** — it addresses a person by a preference they abandoned.
|
|
99
|
+
An environment fact that goes stale makes the agent **wrong** — it acts on a file that
|
|
100
|
+
moved, a budget that was spent, a tool that was removed. So environment facts need a
|
|
101
|
+
freshness policy and a re-check on use; user facts need a correction path and a way for the
|
|
102
|
+
person to see and edit what is held about them. One TTL for both is wrong twice.
|
|
103
|
+
|
|
104
|
+
**Mapping to §7's layers**, so the two shapes can be held at once:
|
|
105
|
+
|
|
106
|
+
| §7 layer | Function it actually serves |
|
|
107
|
+
|---|---|
|
|
108
|
+
| 1 Chat history | working, multi-turn |
|
|
109
|
+
| 2 Working memory (per resource) | working promoted to factual-about-this-task |
|
|
110
|
+
| 3 Long-term learnings | experiential — strategy-based |
|
|
111
|
+
| 4 Insights | experiential — strategy-based, cross-resource |
|
|
112
|
+
| *(missing)* | **factual: user and environment** — add it as its own store, not as a learning |
|
|
113
|
+
|
|
114
|
+
## 4. Dynamics — formation, evolution, retrieval
|
|
115
|
+
|
|
116
|
+
Three operators, and a system is defined by which of them it runs and how often.
|
|
117
|
+
|
|
118
|
+
- **Formation** — what becomes a memory candidate at all. Selective, never the whole
|
|
119
|
+
transcript: *"extracting information with potential future utility rather than storing
|
|
120
|
+
the entire interaction history verbatim."* Owned here by §8 and
|
|
121
|
+
`patterns.md` → **Learning Extraction Heuristics**.
|
|
122
|
+
- **Evolution** — consolidation, updating, forgetting. Owned by `patterns.md` →
|
|
123
|
+
**Confidence Management**, **Fuzzy Deduplication**, **Conflict Resolution**. Forgetting is
|
|
124
|
+
under-specified there; §6 below closes it.
|
|
125
|
+
- **Retrieval** — this pack had almost nothing on it. §5 is the whole of it.
|
|
126
|
+
|
|
127
|
+
## 5. Retrieval is four decisions, not one
|
|
128
|
+
|
|
129
|
+
*"Memory retrieval is not a static search operation but a dynamic cognitive process."* Four
|
|
130
|
+
stages, in execution order. A system that implements only the third — the usual case — is
|
|
131
|
+
running one of four.
|
|
132
|
+
|
|
133
|
+
### 5.1 Timing and intent — whether to retrieve at all, and from which store
|
|
134
|
+
|
|
135
|
+
**The failure this stage exists to prevent has a name and it is silent.** When an agent
|
|
136
|
+
overestimates its own knowledge and does not retrieve, there is no error, no empty result
|
|
137
|
+
and no latency spike — there is a confident answer built from nothing. It is invisible to
|
|
138
|
+
every health check that watches for failures.
|
|
139
|
+
|
|
140
|
+
- **Always-on retrieval** is the safe default and it is not free: it spends context on
|
|
141
|
+
every turn and injects noise into questions that needed none.
|
|
142
|
+
- **Model-decided** retrieval is cheaper and introduces exactly the silent mode above.
|
|
143
|
+
- **Fast–slow** is the compromise worth the wiring: answer, self-assess, and retrieve
|
|
144
|
+
deeper only when the first answer is judged insufficient.
|
|
145
|
+
|
|
146
|
+
**Instrument it or do not ship it.** Log retrieval decisions including the negatives —
|
|
147
|
+
*asked, decided not to retrieve* — and give `agent-evals` a fixture where the answer is
|
|
148
|
+
only obtainable from memory. A memory that is never queried scores the same as a memory
|
|
149
|
+
that is empty, and only the log tells them apart.
|
|
150
|
+
|
|
151
|
+
### 5.2 Query construction — what to retrieve with
|
|
152
|
+
|
|
153
|
+
The user's words are not a good query against your index, and this stage is the one most
|
|
154
|
+
often skipped entirely. Two techniques, and they compose:
|
|
155
|
+
|
|
156
|
+
- **Decomposition** — break a compound question into sub-queries, retrieve per part.
|
|
157
|
+
Use when the question spans several facts that no single entry holds.
|
|
158
|
+
- **Rewriting** — restate the query in the index's own language, or generate a hypothetical
|
|
159
|
+
answer and search with *that* (HyDE). Use when user phrasing and stored phrasing diverge
|
|
160
|
+
— which is most of the time for a store written by the agent itself.
|
|
161
|
+
|
|
162
|
+
### 5.3 Strategy — how the search runs
|
|
163
|
+
|
|
164
|
+
| Strategy | Strong at | Weak at |
|
|
165
|
+
|---|---|---|
|
|
166
|
+
| **Lexical** (BM25, TF-IDF) | exact identifiers, tool names, error strings, precision | paraphrase, synonyms |
|
|
167
|
+
| **Semantic** (embeddings) | paraphrase, fuzzy match — the usual default | drift and forced top-K, which return *something* however irrelevant |
|
|
168
|
+
| **Graph** | multi-hop, relational, temporal constraints | cost, and a schema to maintain |
|
|
169
|
+
| **Hybrid** | lexical precision plus semantic reach | two systems to tune |
|
|
170
|
+
|
|
171
|
+
**Semantic-only retrieval always returns K results.** There is no "nothing matched" unless
|
|
172
|
+
a similarity floor is set, so an empty store and an irrelevant store look identical to the
|
|
173
|
+
model. Set the floor, and make "nothing relevant" a value the caller can act on.
|
|
174
|
+
|
|
175
|
+
### 5.4 Post-retrieval — what actually reaches the prompt
|
|
176
|
+
|
|
177
|
+
Raw hits are redundant, stale and mutually contradictory. Two operations:
|
|
178
|
+
|
|
179
|
+
- **Re-rank and filter** — drop low-relevance and expired items. Temporal validity is a
|
|
180
|
+
filter, not a tiebreak: a fact with a validity window that has closed is wrong, not
|
|
181
|
+
merely old.
|
|
182
|
+
- **Aggregate and compress** — merge duplicates and reconstruct one coherent context.
|
|
183
|
+
|
|
184
|
+
This is where `context-engineering.md`'s budget applies. §7's warning holds at every stage:
|
|
185
|
+
every layer competes for one window, so give working memory a floor or a large set of old
|
|
186
|
+
generalities will quietly evict what the user said a minute ago.
|
|
187
|
+
|
|
188
|
+
## 5.5 The budget, layer 0, and workspace scale
|
|
189
|
+
|
|
190
|
+
Moved here from `SKILL.md` §7 when that file reached its body budget: these are memory
|
|
191
|
+
*architecture*, and this is the file about it.
|
|
192
|
+
|
|
193
|
+
**The trap is the budget, not the storage.** Every layer competes for the same context
|
|
194
|
+
window, so allocation is decided per call rather than per layer. A session that trims chat
|
|
195
|
+
history to fit a large set of learnings has quietly chosen old generalities over what the
|
|
196
|
+
user said sixty seconds ago. **Give layer 1 a floor.** This is the same window
|
|
197
|
+
`context-engineering.md` governs, and §5.4's post-retrieval stage is where a retrieval
|
|
198
|
+
that ignores it does its damage.
|
|
199
|
+
|
|
200
|
+
**Layer 0 — carryover state.** Goal, artifacts, verified work and restrictive mode cross a
|
|
201
|
+
compaction boundary as copied typed blocks, not prose (`SKILL.md` §12). It is a memory
|
|
202
|
+
layer whose whole job is surviving one specific event.
|
|
203
|
+
|
|
204
|
+
**Workspace scale.** Managing persistent workspaces rather than sessions shifts the scopes
|
|
205
|
+
— run, workspace, global, doctrine — and adds the journal-spine rules:
|
|
206
|
+
`patterns.md` → **Workspace-scale memory**.
|
|
207
|
+
|
|
208
|
+
## 6. Forgetting, and the long-tail trap
|
|
209
|
+
|
|
210
|
+
Three policies, and they are orthogonal — a system usually needs more than one:
|
|
211
|
+
|
|
212
|
+
- **Time-based decay** — natural aging. `patterns.md` → Confidence Management.
|
|
213
|
+
- **Frequency-based** — LRU/LFU, evict what is not read.
|
|
214
|
+
- **Importance-driven** — score on temporal, frequency and semantic signals together, and
|
|
215
|
+
increasingly let a model judge salience rather than a counter.
|
|
216
|
+
|
|
217
|
+
**The trap, stated because frequency-based forgetting is the easy one to reach for:**
|
|
218
|
+
LRU-style eviction *"may eliminate long-tail knowledge, which is seldom accessed but
|
|
219
|
+
essential for correct decision-making."* The rarely-read entry is often the one that
|
|
220
|
+
prevents a rare and expensive mistake — the incident, the exception, the one customer whose
|
|
221
|
+
setup differs. **When storage is not the binding constraint, do not delete: demote.** Move
|
|
222
|
+
it out of the default retrieval path and keep it reachable by explicit query.
|
|
223
|
+
|
|
224
|
+
Deletion is also a **correctness** operation, not only a capacity one — see §8.
|
|
225
|
+
|
|
226
|
+
## 7. Shared memory when there is more than one agent
|
|
227
|
+
|
|
228
|
+
The progression, and both ends are wrong:
|
|
229
|
+
|
|
230
|
+
- **Isolated memories with message passing** — no interference, but redundancy, fragmented
|
|
231
|
+
context and communication overhead that grows with team size.
|
|
232
|
+
- **A naive global store** — every agent reads and writes one space. This buys joint
|
|
233
|
+
attention and costs **memory clutter, write contention, and no role- or permission-aware
|
|
234
|
+
access control**.
|
|
235
|
+
|
|
236
|
+
**What this pack already has, and what it is not.** `agent-sync` gives leases, race-free id
|
|
237
|
+
reservation and a run journal: it decides *who may write this file right now*. That is
|
|
238
|
+
coordination, and it is not shared memory — it says nothing about what an agent should be
|
|
239
|
+
allowed to *read*, or whose experiential memory is trustworthy enough to act on. An agent
|
|
240
|
+
system that needs both needs both.
|
|
241
|
+
|
|
242
|
+
**The design rule:** make shared writes **attributed and scoped**. An entry carries who
|
|
243
|
+
wrote it and under what role, and a reader may weigh it accordingly. Unattributed shared
|
|
244
|
+
memory means one agent's wrong conclusion becomes every agent's premise, with nothing in
|
|
245
|
+
the record to trace it back.
|
|
246
|
+
|
|
247
|
+
## 8. Trustworthy memory
|
|
248
|
+
|
|
249
|
+
Three pillars, and the survey's position is that these stop being features and become
|
|
250
|
+
requirements once an agent is deployed and persistent.
|
|
251
|
+
|
|
252
|
+
**Privacy.** Agent memory holds user-specific, persistent and potentially sensitive content
|
|
253
|
+
— a different risk class from a document index. Memory modules have been shown to **leak
|
|
254
|
+
private data through indirect prompt injection**: text the agent read becomes text the
|
|
255
|
+
agent stored becomes text the agent will repeat. Three controls: do not store what the task
|
|
256
|
+
does not need (the same rule `error-tracking` applies before events reach a third party),
|
|
257
|
+
scope reads, and make **verifiable forgetting** possible — which is exactly what parametric
|
|
258
|
+
memory (§2) cannot offer, and the strongest argument for keeping deletable knowledge in
|
|
259
|
+
token-level form.
|
|
260
|
+
|
|
261
|
+
**Explainability.** *"Users and developers still lack tools to trace which memory items
|
|
262
|
+
were retrieved, how they influenced generation, or whether they were misused."* Minimum
|
|
263
|
+
bar, and it is cheap if built in from the start: every retrieval is logged with the ids it
|
|
264
|
+
returned, and every answer that used memory can name the entries it used. Retrofitted, it
|
|
265
|
+
is a rewrite.
|
|
266
|
+
|
|
267
|
+
**Hallucination robustness.** The point of memory is fewer invented answers, and a memory
|
|
268
|
+
layer can add them: a confidently retrieved stale entry is worse than an empty store,
|
|
269
|
+
because it carries authority. **Abstention under low-confidence retrieval** — say *"I do not
|
|
270
|
+
have this"* rather than answer from the best of a bad set — is the single highest-value
|
|
271
|
+
behaviour here, and it needs the similarity floor from §5.3 to be expressible at all.
|
|
272
|
+
|
|
273
|
+
## 9. What this pack does NOT claim
|
|
274
|
+
|
|
275
|
+
Named so a reader does not take this file for more than it is:
|
|
276
|
+
|
|
277
|
+
- **No benchmark numbers are reproduced here.** The survey tabulates benchmarks (LoCoMo,
|
|
278
|
+
LongMemEval, MemBench, StreamBench and others, §6.1); which of them fits a given system
|
|
279
|
+
is an `agent-evals` question and none of them is quoted as a result.
|
|
280
|
+
- **The frontier sections are frontiers.** RL-trained memory management, latent/generative
|
|
281
|
+
memory, and offline consolidation are stated in the survey as open directions, not
|
|
282
|
+
settled practice. They are in §2 as forms with costs, and nothing here recommends
|
|
283
|
+
building on them.
|
|
284
|
+
- **This file was written from the survey, not from running these systems.** Where the pack
|
|
285
|
+
has its own measured experience — §7's layers, `patterns.md`'s confidence arithmetic —
|
|
286
|
+
that is marked as the pack's and is not attributed to the paper.
|
|
287
|
+
|
|
288
|
+
## 10. The checklist
|
|
289
|
+
|
|
290
|
+
Before an agent gets memory, answer these. An unanswered row is a decision that will be
|
|
291
|
+
made by accident:
|
|
292
|
+
|
|
293
|
+
1. **Which functions does it need?** Factual-user, factual-environment, experiential,
|
|
294
|
+
working — name each one you are building. Not all four are always needed; the ones you
|
|
295
|
+
skip should be skipped on purpose.
|
|
296
|
+
2. **What form carries each?** Default token-level and flat. Any other answer names the
|
|
297
|
+
cost it is paying for (§2).
|
|
298
|
+
3. **What forms a memory?** The contrast rule for experiential (§8 of `SKILL.md`); an
|
|
299
|
+
explicit write path for factual. Never "log the transcript".
|
|
300
|
+
4. **When is it retrieved, and is that decision logged — including the negatives?** (§5.1)
|
|
301
|
+
5. **What is the query?** Raw user text is the answer only if you have checked it works.
|
|
302
|
+
(§5.2)
|
|
303
|
+
6. **Is there a relevance floor, so "nothing relevant" is expressible?** (§5.3)
|
|
304
|
+
7. **What expires, what decays, what is demoted rather than deleted?** (§6)
|
|
305
|
+
8. **If more than one agent writes it: who wrote this entry, under what role?** (§7)
|
|
306
|
+
9. **Can a person see, correct and delete what is held about them?** (§8)
|
|
307
|
+
10. **What eval fails if memory is silently disabled?** If none, the memory layer is
|
|
308
|
+
unmeasured and its value is a belief. (`agent-evals`)
|
|
@@ -20,6 +20,7 @@ that costs no LLM call.
|
|
|
20
20
|
- [Conflict Resolution Pattern](#conflict-resolution-pattern)
|
|
21
21
|
- [Cross-Resource Learning Transfer](#cross-resource-learning-transfer)
|
|
22
22
|
- [Suggestion Engine (No LLM Cost)](#suggestion-engine-no-llm-cost)
|
|
23
|
+
- [Workspace-scale memory — the journal spine](#workspace-scale-memory--the-journal-spine)
|
|
23
24
|
|
|
24
25
|
|
|
25
26
|
## Data Models
|
|
@@ -454,3 +455,37 @@ constant to tune and a constant with two homes is one that will disagree with it
|
|
|
454
455
|
Both moved out of `SKILL.md` on 2026-08-16. The mechanisms they describe were already
|
|
455
456
|
in this file — the validation loop, the extractors, the confidence arithmetic — so the
|
|
456
457
|
body was holding a second copy of their surface. One home; the body keeps the decision.
|
|
458
|
+
|
|
459
|
+
## Workspace-scale memory — the journal spine
|
|
460
|
+
|
|
461
|
+
The four layers in the body's §7 are session-scale: chat, working, learnings, insights.
|
|
462
|
+
When the same machinery manages **persistent workspaces** — long-lived projects that own
|
|
463
|
+
agents, schedules and history and outlive every conversation — the scopes shift, and five
|
|
464
|
+
rules keep the store honest at that scale. *Distilled 2026-08-27 from the Passion Code
|
|
465
|
+
fabric design review; the workforce half of that review is
|
|
466
|
+
`references/provider-lifecycle.md`.*
|
|
467
|
+
|
|
468
|
+
| Scope | Holds | Lives |
|
|
469
|
+
|---|---|---|
|
|
470
|
+
| run-working | scratch, intermediates, the transcript | one run; artifacts survive by content hash |
|
|
471
|
+
| workspace | decisions, lessons, report context of one project | permanent, append-only, isolated |
|
|
472
|
+
| global | facts promoted above any one workspace | permanent, with decay |
|
|
473
|
+
| doctrine | intent and standards, versioned in git | the source everything else indexes |
|
|
474
|
+
|
|
475
|
+
1. **One append-only journal is the canonical ledger.** Memory writes are events; every
|
|
476
|
+
register anyone reads is a projection of them. Corrections supersede; erasure leaves a
|
|
477
|
+
tombstone. A store built table-first cannot adopt this later — history that predates
|
|
478
|
+
the journal is unrecoverable at any price.
|
|
479
|
+
2. **Every index is a rebuildable projection**, and an embedding row carries the
|
|
480
|
+
embedding model's name and version — otherwise the first model upgrade silently mixes
|
|
481
|
+
incomparable vectors and similarity search degrades without an error.
|
|
482
|
+
3. **Isolation is enforced at the memory API**, from the authenticated caller's scope —
|
|
483
|
+
never by asking the prompt to respect a boundary. One workspace never writes
|
|
484
|
+
another's memory; transfer happens only as an explicit, revisioned artifact
|
|
485
|
+
(`provider-lifecycle.md` names the vehicle).
|
|
486
|
+
4. **Promotion to global carries provenance, confidence, contradiction links and an
|
|
487
|
+
expiry.** Memory without decay accumulates confident lies, and the global scope is
|
|
488
|
+
where they do the most damage because nothing above it contradicts them.
|
|
489
|
+
5. **Memory reaches the model only through the compiled per-task bundle.** One entry
|
|
490
|
+
point means one supply-chain gate and one lockfile that pins what the agent knew —
|
|
491
|
+
which is the difference between debugging a bad answer and re-litigating it.
|
|
@@ -0,0 +1,151 @@
|
|
|
1
|
+
# Provider lifecycle — where agents come from, and how one earns trust
|
|
2
|
+
|
|
3
|
+
**Load this when** the question is the workforce rather than the loop: an agent is being
|
|
4
|
+
produced, adapted from an existing project, registered, replaced or retired, or a fleet
|
|
5
|
+
of workspaces needs governing. The loop that *runs* a provider is the body; what a call
|
|
6
|
+
costs is `llm-proxy-billing.md`; whether an action is permitted is `governance.md`. This
|
|
7
|
+
file owns the axis none of them hold: a provider's life from intake to retirement.
|
|
8
|
+
|
|
9
|
+
*Distilled 2026-08-27 from the Passion Code fabric design review (its ADR-0015 and
|
|
10
|
+
agent-production design), generalised for any estate of agent workspaces.*
|
|
11
|
+
|
|
12
|
+
## Contents
|
|
13
|
+
|
|
14
|
+
- Produced once, bound many times
|
|
15
|
+
- The production pipeline, stage by stage
|
|
16
|
+
- Knowledge packs — how expertise transfers between projects
|
|
17
|
+
- Trust is earned by watched runs: the canary binding
|
|
18
|
+
- Two extension mechanisms, and only two
|
|
19
|
+
- Workspace lifecycle, and the dependency projection under retirement
|
|
20
|
+
- Fleet governance: hierarchical budgets and the run scheduler
|
|
21
|
+
|
|
22
|
+
## Produced once, bound many times
|
|
23
|
+
|
|
24
|
+
The distinction the whole file stands on:
|
|
25
|
+
|
|
26
|
+
| | **Provider** | **Binding** |
|
|
27
|
+
|---|---|---|
|
|
28
|
+
| Is | the agent as artifact: repo, manifest, capability schemas, service or instruction pack | one workspace's versioned decision to use that provider for a capability |
|
|
29
|
+
| Created by | a production run — rare, expensive, gated | a registry write — cheap, reversible |
|
|
30
|
+
| Versioned as | provider revisions; v2 goes through the same pipeline as v1 | immutable binding revisions; a run pins one |
|
|
31
|
+
| Retired by | archiving its home project | unbinding — history and schedules survive it |
|
|
32
|
+
|
|
33
|
+
Conflate the axes and every hire becomes a project: nineteen role types across N
|
|
34
|
+
workspaces is nineteen providers and N× bindings, never 19×N projects. Rollout of a new
|
|
35
|
+
provider version is *rebinding*, never mutation of a binding a running task already
|
|
36
|
+
pinned.
|
|
37
|
+
|
|
38
|
+
## The production pipeline, stage by stage
|
|
39
|
+
|
|
40
|
+
Producing an agent is an ordinary project whose route is data — a versioned stage list,
|
|
41
|
+
not code. The stages that survived review:
|
|
42
|
+
|
|
43
|
+
| Stage | Gate that closes it |
|
|
44
|
+
|---|---|
|
|
45
|
+
| **intake** | capability named in the controlled vocabulary; **a consumer named** — the workspace or schedule that will actually call it; workflow-or-agent decided (`agent-harness`: if every step can be named now, it is a workflow behind a capability, not an autonomous agent); transport chosen by the interop rule; money- and publication-adjacent effects declared per agent |
|
|
46
|
+
| **knowledge** | sources named and distilled into a knowledge pack (below); every claim in it cites its origin |
|
|
47
|
+
| **scaffold** | manifest + capability schemas + one safe fixture validate against the pinned contract revision |
|
|
48
|
+
| **instructions** | the instruction pack is a revision, content-hashed, carrying the enumerated vocabulary — status values, capability names — generated from the schema, never retyped |
|
|
49
|
+
| **build** | the ordinary delivery pipeline of the estate, run inside the agent's own workspace |
|
|
50
|
+
| **evals** | golden fixtures pass AND planted defects are rejected, *watched* — on the two clocks `agent-evals` §6 defines: the **observable** for each requirement written at intake, before the build; the corpus grown from production, where the source project's recorded failures count as production |
|
|
51
|
+
| **admission** | shape conformance → protocol negotiation → side-effect-free semantic probes → an immutable admission record |
|
|
52
|
+
| **canary binding** | bound under mandatory checking and a budget cap; unsupervised operation is a later, recorded promotion |
|
|
53
|
+
|
|
54
|
+
The sharpest gate is the first: **no agent without a named consumer.** A role catalogue
|
|
55
|
+
is not a production queue, and the cheapest agent to operate is the one you did not
|
|
56
|
+
build because nothing would have called it.
|
|
57
|
+
|
|
58
|
+
Two entry doors, one pipeline: **build** (greenfield) and **adapt** — an existing
|
|
59
|
+
project with a stable surface gets inspected without execution, wrapped behind a
|
|
60
|
+
capability, and enters at scaffold with its own docs as the knowledge source and its own
|
|
61
|
+
recorded failures as the first fixtures.
|
|
62
|
+
|
|
63
|
+
## Knowledge packs — how expertise transfers between projects
|
|
64
|
+
|
|
65
|
+
The object that makes "reuse the knowledge, not the code" mechanical rather than
|
|
66
|
+
aspirational:
|
|
67
|
+
|
|
68
|
+
```
|
|
69
|
+
knowledge_pack(id, revision, content_hash,
|
|
70
|
+
sources[]: what was read — repos, docs, audits, retros, with refs
|
|
71
|
+
distilled:
|
|
72
|
+
patterns[] what works here, each citing file:line
|
|
73
|
+
traps[] the source's recorded failures and dead ends
|
|
74
|
+
fixtures[] ← traps, converted into planted-defect eval cases
|
|
75
|
+
glossary[] terms the new agent must use exactly as the source does
|
|
76
|
+
)
|
|
77
|
+
```
|
|
78
|
+
|
|
79
|
+
Three rules give it teeth:
|
|
80
|
+
|
|
81
|
+
- **A trap becomes a fixture.** The new agent is not admitted until it has been watched
|
|
82
|
+
rejecting the exact defects its predecessor was burned by. Knowledge transfers as a
|
|
83
|
+
check, not as prose an instruction pack hopes the model remembers.
|
|
84
|
+
- **A pack travels as an artifact, never as a memory write.** Workspace memory is
|
|
85
|
+
isolated (see `patterns.md` → *Workspace-scale memory*); the pack is the legal vehicle
|
|
86
|
+
between workspaces — explicit, attributable, revisioned.
|
|
87
|
+
- **A pack is an injection surface.** Text composed into a prompt from many sources is
|
|
88
|
+
supply chain; a pack produced by an agent passes the same eval gate as code.
|
|
89
|
+
|
|
90
|
+
## Trust is earned by watched runs: the canary binding
|
|
91
|
+
|
|
92
|
+
Authorship is not evidence. A freshly produced provider — your own included — enters
|
|
93
|
+
under a **canary binding**: its output gates through a checker (the contract lives in
|
|
94
|
+
`graph-engineering.md` §6) and its spend is capped, regardless of who wrote it. Removing
|
|
95
|
+
the supervision is a **promotion**: a recorded decision citing eval results and run
|
|
96
|
+
history, with an author. The record matters more than the ceremony — a checker quietly
|
|
97
|
+
dropped is indistinguishable from one that never existed, and the promotion row is the
|
|
98
|
+
only thing that says which.
|
|
99
|
+
|
|
100
|
+
Store, per provider revision, the **production provenance**: source repo, the run that
|
|
101
|
+
produced it, its eval set, its admission. "Where did this agent come from" must be a
|
|
102
|
+
query, not an archaeology project.
|
|
103
|
+
|
|
104
|
+
## Two extension mechanisms, and only two
|
|
105
|
+
|
|
106
|
+
Everything that extends an agent estate is one of:
|
|
107
|
+
|
|
108
|
+
1. **a versioned registry entry** — a capability name, a skill, a pipeline, a template,
|
|
109
|
+
an event kind;
|
|
110
|
+
2. **a provider behind a profile** — an agent, a connector, a checker.
|
|
111
|
+
|
|
112
|
+
The corollaries do real work: a *connector* is a deterministic provider of `collect.*`
|
|
113
|
+
capabilities (no separate plugin system to build); a *checker* is a provider of
|
|
114
|
+
`check.*` capabilities (so custom checkers ride the same production pipeline and
|
|
115
|
+
admission as any agent, and a checker may never be served by the same binding that
|
|
116
|
+
produced the work it checks). A feature that wants a third extension mechanism is a
|
|
117
|
+
design smell before it is a backlog item.
|
|
118
|
+
|
|
119
|
+
## Workspace lifecycle, and the dependency projection under retirement
|
|
120
|
+
|
|
121
|
+
A workspace moves `proposed → active → dormant → archived`, and two transitions carry
|
|
122
|
+
rules that prevent silent damage:
|
|
123
|
+
|
|
124
|
+
- **dormant pauses its schedules.** A sleeping workspace whose routines still tick burns
|
|
125
|
+
quota and money invisibly — dormancy that does not stop the clock is a label, not a
|
|
126
|
+
state.
|
|
127
|
+
- **archived requires the dependency projection to be empty for it**: no active binding
|
|
128
|
+
in another workspace may still point at this workspace's providers. That projection —
|
|
129
|
+
who consumes whose capabilities — is cheap to maintain and impossible to reconstruct
|
|
130
|
+
during an incident; without it, retiring a workspace is a surprise delivered to its
|
|
131
|
+
dependents at call time.
|
|
132
|
+
|
|
133
|
+
## Fleet governance: hierarchical budgets and the run scheduler
|
|
134
|
+
|
|
135
|
+
Per-call spend limits do not govern a fleet. Two objects do, and both are projections
|
|
136
|
+
over the run record rather than new subsystems:
|
|
137
|
+
|
|
138
|
+
- **A budget hierarchy** — estate → workspace → goal → task — where an exhausted level
|
|
139
|
+
refuses *admission of new runs* rather than killing running ones, and approaching a
|
|
140
|
+
cap is an attention signal. The money mechanics — wallets, reservations, reconciliation
|
|
141
|
+
— are `llm-proxy-billing.md`; the multi-level attribution argument is
|
|
142
|
+
`governance.md`. What this file adds: the cap must exist at every level, because
|
|
143
|
+
sixty workspaces individually under budget is still one bill nobody approved.
|
|
144
|
+
- **A run scheduler** — a ceiling on concurrent runs per host, priority classes
|
|
145
|
+
(incident > scheduled > backfill), and per-provider concurrency tied to the external
|
|
146
|
+
quota records the collectors keep. A fleet without one discovers its capacity limit
|
|
147
|
+
as a pile of half-finished runs on the busiest day of the year.
|
|
148
|
+
|
|
149
|
+
And one heartbeat rule: every scheduled worker writes an observation about itself; a
|
|
150
|
+
stale heartbeat is an attention row. A provider that is not watched is not operated —
|
|
151
|
+
the failure mode of every fleet is not the crash but the silence after it.
|