@ssheleg/agent-stack 0.21.0 → 0.22.0

package/CHANGELOG.md

@@ -1,3 +1,45 @@
+## v0.22.0 — the failures named from outside, and the control that is absence
+
+**`graph-engineering.md` was argued entirely from what breaks, with no citation behind it.**
+*Why Do Multi-Agent LLM Systems Fail?* (2025) measured the same ground: traces from seven
+mainstream frameworks — MetaGPT, ChatDev, AG2, Magentic-One — with human annotators
+independently analysing ~**150 traces** at **Cohen's kappa = 0.88**, yielding **14 failure
+modes in three groups**.
+
+The reason to carry it is that the three groups land on this file's own five-field node
+contract without being bent to fit: *system design flaws* on the interface and the `owner`,
+*inter-agent alignment failures* on the edge payload — §3's **Carries** column — and
+*missing task verification* on `check` and the checker node. An outside measurement arriving
+at the same joints is worth more than another argument from failure.
+
+**And its load-bearing result is the negative one.** Better prompts, more explicit roles and
+retries bought ChatDev **15.6%**, and the authors conclude the modes are **architectural
+rather than bugs** — the same claim §1 makes about the two fields nobody draws. The section
+is written as a review checklist, not a taxonomy to admire: per node, is the interface
+stated, is there exactly one owner, and does the check run on something other than the
+node's own claim of success.
+
+**`governance.md` put per-tool authorisation at the moment of invocation, and there is a
+strictly stronger control one layer earlier.** *The model cannot reason about capabilities
+it does not know exist.* A tool absent from the schema cannot be invoked, argued for, or
+probed for a bypass; a tool present and refused at call time is a negotiation, and
+negotiations are won sometimes. Sub-agent isolation needs both halves — schema filtering at
+construction **and** `message_history = None` at execution, because a sub-agent handed its
+parent's transcript has been told about every capability you removed from its schema.
+
+**Two numbers that point in opposite directions on purpose.** Eagerly loading every MCP tool
+schema at startup consumed **40% of the context budget before the first user message**; a
+metadata index with schemas fetched on selection takes it **under 5%**. Yet the same system
+builds *its own* prompt and tool schemas **eagerly**, in the constructor. The rule is not
+*lazy is better*: it is **eager for what you own and always need, lazy for what is foreign
+and might not be used.**
+
+**Approval fatigue is filed as a safety failure, not a UX complaint.** An approval system
+with no persistence makes users re-approve the same operations every session, which produces
+blanket auto-approval and defeats the safety system entirely — through the user rather than
+through a bug, so nothing in the logs looks wrong. **A control that is asked too often is a
+control on its way to being switched off.**
+
 ## v0.21.0 — the risk one tool cannot show you, and the money an iteration refund does not cover
 
 Two findings, both of them about a rule that is right on one axis and silently assumed to

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@ssheleg/agent-stack",
-  "version": "0.21.0",
+  "version": "0.22.0",
   "scripts": {
     "test": "python3 test/validate.py && python3 test/plant_guard_test.py && node test/installer_test.js"
   },

package/plugins/agent-stack/.claude-plugin/plugin.json

@@ -3,7 +3,7 @@
   "name": "agent-stack",
   "displayName": "Agent Stack",
   "description": "Four skills: agent-orchestrator — tool-calling loops, pipelines with checkpoints, provider routing with fallback, memory architecture, plus the wallet side of reselling LLM access; agent-evals — run/trace/thread evals, LLM judges, and fixtures grown from production; agent-interop — MCP servers and clients, A2A agent cards, the MCP Registry, and gateways; agent-harness — system prompts, tool shaping, workflow-vs-agent, and auditing an agent system.",
-  "version": "0.21.0",
+  "version": "0.22.0",
   "author": {
     "name": "ssheleg",
     "url": "https://x.com/sshlg93"

package/plugins/agent-stack/skills/agent-orchestrator/references/governance.md

@@ -36,6 +36,45 @@ guardrails" so often means only the first.
 The last one is the one most designs miss: a sub-agent that inherits its caller's
 authority silently widens every permission the caller had.
 
+## The cheapest control is absence
+
+The **Tool call** row above puts per-tool authorisation at the moment of invocation. There
+is a control one layer earlier and it is strictly stronger:
+
+> **The model cannot reason about capabilities it does not know exist.**
+
+A tool absent from the schema cannot be invoked, cannot be argued for, and cannot be probed
+for a bypass — there is nothing to jailbreak toward. A tool present in the schema and
+refused at call time is a negotiation, and negotiations are won sometimes.
+
+So **filter the schema at build time**, and treat runtime authorisation as the second line
+rather than the first. Sub-agent isolation comes from exactly two mechanisms used together:
+schema filtering when the agent is constructed, and no inherited conversation
+(`message_history = None`) when it runs. The second matters as much as the first — a
+sub-agent handed its parent's transcript has been told about every capability you carefully
+removed from its schema.
+
+**Two numbers, and they point in opposite directions on purpose.** Eagerly loading every
+MCP tool schema at startup consumed **40% of the context budget before the first user
+message**; a metadata index at startup with the full schema fetched on selection takes it
+**under 5%**. But the same system builds *its own* prompt and tool schemas **eagerly**, in
+the constructor. The rule underneath is not *lazy is better*: it is **eager for what you
+own and always need, lazy for what is foreign and might not be used** — the first removes
+latency and race conditions from the hot path, the second removes a cost you cannot predict.
+
+### Approval fatigue is a safety failure, not a UX complaint
+
+> Without persistence, users must re-approve the same operations every session, causing
+> approval fatigue that leads to **blanket auto-approval, defeating the safety system
+> entirely**.
+
+An approval system with no memory converts itself into no approval system, and it does so
+through the user rather than through a bug — so nothing in the logs looks wrong. The
+remedy is on the same axis as the section above: **decide once what does not need asking,
+remove it from the question, and spend the prompts on what genuinely changes.** A control
+that is asked too often is a control on its way to being switched off.
+
+
 ## Guardrails, and their honest limit
 
 The content-layer checks worth having, roughly in order of reliability:

package/plugins/agent-stack/skills/agent-orchestrator/references/graph-engineering.md

@@ -28,6 +28,7 @@ well-measured answer to the wrong question.
 - [9. What Claude Code actually executes](#9-what-claude-code-actually-executes)
 - [10. Barrier or no barrier](#10-barrier-or-no-barrier)
 - [11. Project defaults, written once](#11-project-defaults-written-once)
+- [11a. MAST — the failures, named from outside](#11a-mast--the-failures-named-from-outside)
 - [12. The source's four diagrams, and what each one is for](#12-the-sources-four-diagrams-and-what-each-one-is-for)
 - [Where this file disagrees with its source](#where-this-file-disagrees-with-its-source)
 
@@ -375,6 +376,32 @@ The last line is `context-engineering.md`'s *filesystem as context* stated as a
 rule: an edge that carries a path costs a few tokens, and an edge that carries a
 transcript costs the window.
 
+## 11a. MAST — the failures, named from outside
+
+Everything above is argued from what breaks. *Why Do Multi-Agent LLM Systems Fail?* (2025)
+measured it: execution traces from seven mainstream frameworks — MetaGPT, ChatDev, AG2 and
+Magentic-One among them — with human annotators independently analysing roughly **150
+traces** at **Cohen's kappa = 0.88**, producing **14 failure modes in three groups**.
+
+The three groups land on this file's own node contract, which is the reason to carry them:
+
+| MAST group | Modes include | The field it lands on |
+|---|---|---|
+| **System design flaws** | unclear interfaces between agents, overlapping roles, wrong tool configuration | `input` / `output` (the interface), and `owner` (overlapping roles) |
+| **Inter-agent alignment failures** | inconsistent understanding of the objective, downstream misinterpretation, logically contradictory operations | the **edge payload** — what actually crosses, §3's *Carries* column |
+| **Missing task verification** | an agent reports *completed* and the result does not meet the requirement | `check`, and §6's checker node |
+
+**The load-bearing result is the negative one.** Straightforward fixes — better prompts,
+more explicit role descriptions, retries — bought ChatDev only **15.6%**. The authors
+conclude the modes are **architectural rather than bugs**, which is the same claim §1 makes
+about the two fields nobody draws: an owner and a completion test are not documentation of
+a graph, they are the parts that make it a graph.
+
+Read the mapping as a review checklist rather than a taxonomy to admire: for each node, is
+the interface stated, is there exactly one owner, and does the *check* run on something
+other than the node's own claim of success.
+
+
 ## 12. The source's four diagrams, and what each one is for
 
 The article carries four hand-drawn figures. They are not decoration — each one is doing