@ssheleg/agent-stack 0.17.0 → 0.17.1

package/CHANGELOG.md

@@ -1,5 +1,46 @@
 # Changelog
 
+## v0.17.1 — the evals run for the first time, and the tails go to zero
+
+Wave-3 of the 2026-08-29 family audit (rows AST-05, AST-07, AST-08, AST-09, AST-10),
+plus the two board rows wave 2 filed (AST-A1, AST-B). All fixes and docs — no
+description or trigger changes, hence a patch.
+
+- **AST-05: the eval suite is executed for the first time.** `test/evals/RESULTS.md`
+  gains two dated rows — haiku and sonnet, 2026-08-31 — each a fresh blind subagent
+  per trigger query against the family's 28 skill descriptions, plus all three
+  scenarios scored line by line. Both models: 11/12 triggers. Scenario lines: 10/12
+  (haiku), 9/12 (sonnet). Each miss is named — haiku false-triggered `agent-harness`
+  on "rewrite this one system prompt so it sounds friendlier" (q09), sonnet answered
+  `none` on the Russian harness-audit query (q04) — and the Method section states the
+  protocol and its three limits rather than presenting the rates as clean-room.
+- **AST-B: the generator-evaluator citation lands where the doctrine lives.**
+  `agent-evals` §5a now cites Anthropic's *tuning a standalone evaluator to be
+  skeptical is more tractable than making a generator self-critical* (read
+  2026-08-30) beside the sentence making the same claim, marked as convergence
+  rather than invention — beside the first executed eval run, as the board row asked.
+- **AST-A1: the nine-subsystem coverage check is done and the row closes.** The ninth
+  subsystem the filing row could not name is **Automation**; all nine map to existing
+  doctrine with a `file:line` each, and the suspected identity/approval-policy gap
+  resolves as a split verdict — approval policy covered (`governance.md`),
+  authentication mechanics a named delegation (`layers.md:78-80`), not a hole. The
+  map lives in the board row (`docs/evidence/backlog.md`), not in a new reference.
+- **AST-07: the README stops counting its references.** "Twenty references" had
+  drifted to 24 actual files within a week of being written; the aggregate count is
+  dropped in favour of the per-skill counts that are still true, with the reason
+  stated in place.
+- **AST-10: the orchestrator's reference index lists all 11 references.** The three
+  memory rows — `memory-architecture.md`, `memory-lifecycle.md`,
+  `memory-landscape.md` — join the index table, each paraphrasing its own
+  "Load this when" line.
+- **AST-08: `license: MIT` in all four skill front matters** (was 1 of 4 —
+  `agent-evals` alone carried it). Every front matter re-checked with
+  `yaml.safe_load` after the edit.
+- **AST-09: `$schema` in both manifests** — `claude-code-plugin-manifest.json` for
+  `plugin.json`, `claude-code-marketplace.json` for `marketplace.json`, the two
+  schemastore addresses that resolve (the third candidate, `claude-code-plugin.json`,
+  is a 404 and two siblings currently point at it; referred, not copied).
+
 ## v0.17.0 — the descriptions stop colliding, and the claims get their dates
 
 Wave-2 of the 2026-08-29 family audit (rows AST-02, AST-03, AST-04, AST-06, AST-11), plus

package/README.md

@@ -33,8 +33,9 @@ Part of the [ssheleg skill family](https://github.com/ssheleg/sshlg-skills).
 
 Four skills — `agent-orchestrator` for wiring the loop, `agent-evals` for proving it
 behaves, `agent-interop` for everything it talks to outside its own process,
-`agent-harness` for what it is **told** — and twenty references they load on demand,
-plus one scanner.
+`agent-harness` for what it is **told** — and the references they load on demand,
+plus one scanner. The per-skill sections below carry their own counts; no total is
+stated here, because the last one drifted stale within a week of being written.
 
 **The orchestrator** (`SKILL.md`) — what the agent reads first:
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@ssheleg/agent-stack",
-  "version": "0.17.0",
+  "version": "0.17.1",
   "scripts": {
     "test": "python3 test/validate.py && python3 test/plant_guard_test.py && node test/installer_test.js"
   },

package/plugins/agent-stack/.claude-plugin/plugin.json

@@ -1,8 +1,9 @@
 {
+  "$schema": "https://json.schemastore.org/claude-code-plugin-manifest.json",
   "name": "agent-stack",
   "displayName": "Agent Stack",
   "description": "Four skills: agent-orchestrator — tool-calling loops, pipelines with checkpoints, provider routing with fallback, memory architecture, plus the wallet side of reselling LLM access; agent-evals — run/trace/thread evals, LLM judges, and fixtures grown from production; agent-interop — MCP servers and clients, A2A agent cards, the MCP Registry, and gateways; agent-harness — system prompts, tool shaping, workflow-vs-agent, and auditing an agent system.",
-  "version": "0.17.0",
+  "version": "0.17.1",
   "author": {
     "name": "ssheleg",
     "url": "https://x.com/sshlg93"

package/plugins/agent-stack/skills/agent-evals/SKILL.md

@@ -168,6 +168,13 @@ the one most systems are missing: a **checker node** sitting between a parallel
 the node that consumes it. Its only job is *usable / not usable*, and the convergence
 depends on **it** rather than on the branches — otherwise the gate has a bypass.
 
+**That split is measured practice, not only this pack's position.** Anthropic's
+harness-design guidance (`anthropic.com/engineering/harness-design-long-running-apps`,
+read 2026-08-30) reports that *tuning a standalone evaluator to be skeptical is more
+tractable than making a generator self-critical* — the same reason the verdict belongs to
+a separate node rather than to the branches grading themselves. Dated and cited so a
+reader can tell doctrine that converged with the field from doctrine invented here.
+
 It matters here because it is the same machinery as §5, positioned differently:
 
 <!-- checker-contract: missing, empty, unevidenced, malformed, contradictory, off-topic | optional: under-confident -->

package/plugins/agent-stack/skills/agent-harness/SKILL.md

@@ -13,6 +13,7 @@ description: >-
   "embed an agent", "agent SDK", "Pi harness", "системный промпт", "агент не вызывает тул",
   "аудит агента", "встроить агента". Not for the loop's plumbing, its evals, or its
   protocols — those are siblings.
+license: MIT
 ---
 
 # Agent harness — what the agent is told, and how to audit what someone else told theirs

package/plugins/agent-stack/skills/agent-interop/SKILL.md

@@ -12,6 +12,7 @@ description: >-
   "agentgateway", "tool federation", "MCP-сервер", "карточка агента", "интероперабельность
   агентов", "реестр MCP", "шлюз для агентов". Not for designing one server's tool set, nor for
   a skill's own construction — that is make-skill.
+license: MIT
 ---
 
 # Agent interop — the protocols an agent speaks outside its own process

package/plugins/agent-stack/skills/agent-orchestrator/SKILL.md

@@ -12,6 +12,7 @@ description: >-
   loop", "memory layer", "LLM billing", "token wallet", "агентная система", "оркестратор",
   "суб-агент", "роутер моделей", "человек в цикле", "слой памяти", "биллинг LLM", "граф задач".
   Not for a single LLM call in a script, or prompt wording.
+license: MIT
 ---
 
 # Agent Orchestrator — Production Best Practices
@@ -245,9 +246,7 @@ answer built from nothing, in no error log.
 **Design a memory layer from
 [`references/memory-architecture.md`](references/memory-architecture.md)**, not from this
 table — it also carries the context-budget trap, layer 0 carryover and workspace scale.
-The write path is [`references/memory-lifecycle.md`](references/memory-lifecycle.md);
-what to build on and measure with is
-[`references/memory-landscape.md`](references/memory-landscape.md).
+The write path and the build-or-adopt question have their own rows in the index below.
 ## 8. Self-Learning Feedback Loops
 
 Three cycles feed the memory layers, and they differ by what supplies the signal: a failed
@@ -373,15 +372,18 @@ heading — the ones that were learned by getting them wrong:
 ## References
 
 Each file opens with its own **Load this when** line — the authoritative trigger lives
-there, so this table stays an index and the two cannot drift apart.
+there; this table is only an index.
 
 | File | Read it when |
 |---|---|
-| [`references/graph-engineering.md`](references/graph-engineering.md) | you are deciding the **shape of the work** — the fake-edge test, the diamond, the checker node, static versus dynamic, and what the host actually runs |
+| [`references/graph-engineering.md`](references/graph-engineering.md) | you are deciding the **shape of the work** — fake edges, the diamond, the checker node, static versus dynamic |
 | [`references/pipeline.md`](references/pipeline.md) | one loop is **not the shape** — the planned path, its checkpoints, resume, and the interrupt that asks a person |
 | [`references/patterns.md`](references/patterns.md) | you need the **data models and algorithms** under the body |
+| [`references/memory-architecture.md`](references/memory-architecture.md) | an agent is **being given memory**, or memory is misbehaving |
+| [`references/memory-lifecycle.md`](references/memory-lifecycle.md) | the memory **write path** — what enters, and what leaves |
+| [`references/memory-landscape.md`](references/memory-landscape.md) | **build or adopt** a memory layer, and what is settled practice |
 | [`references/context-engineering.md`](references/context-engineering.md) | the loop is **running out of window** |
 | [`references/runtime.md`](references/runtime.md) | the agent must **survive a crash, a pause, a second message or a schedule** |
 | [`references/governance.md`](references/governance.md) | the question is **permission, not cost** — what it may do, and how you prove it |
 | [`references/llm-proxy-billing.md`](references/llm-proxy-billing.md) | the product **resells LLM access** |
-| [`references/provider-lifecycle.md`](references/provider-lifecycle.md) | the question is the **workforce, not the loop** — where providers come from, produced-once/bound-many, knowledge packs, canary trust, workspace lifecycle, fleet budgets |
+| [`references/provider-lifecycle.md`](references/provider-lifecycle.md) | the question is the **workforce, not the loop** — provider lifecycle, canary trust, fleet budgets |