@ssheleg/agent-stack 0.8.0 → 0.10.0

package/CHANGELOG.md

@@ -1,5 +1,137 @@
 # Changelog
 
+## [0.10.0] — 2026-08-15
+
+### Added
+
+- **`agent-orchestrator/references/graph-engineering.md` — deciding the shape of the work
+  before doing it.** The pack could wire a loop, prove it behaved, connect it and tell it
+  what to do, and had nothing to say about the question that comes before all four: *does
+  this actually have to happen in a line?* Node and edge, the fake-edge test, the diamond,
+  the two ways a diamond fails silently, the checker node, static versus dynamic, and the
+  cost table that says when a graph is not worth building.
+
+  Sourced from *Graph Engineering with Claude*
+  (`https://x.com/Mahaximus_/status/2082442856417956173`, published 2026-07-29), and the
+  link is kept so the original can be re-read rather than remembered through the summary.
+  Four sections are **this pack's** and are marked as such in the file: what the host
+  actually runs, the barrier distinction, what a checker costs, and auditability as a hard
+  rule rather than a preference.
+
+- **What Claude Code actually executes, with version evidence.** The source's one
+  operational claim aged out six weeks after publication: the `workflow` keyword it names
+  was renamed to `ultracode` in **v2.1.160**, and the YAML it shows is a way of describing
+  a graph in a prompt rather than a syntax the host parses — the execution contract is a
+  script whose primitives are `agent()`, `parallel()` (a barrier) and `pipeline()` (none).
+  Every claim in that section carries the `v2.1.x` changelog entry that establishes it,
+  which is why the correction reads as a dated fact rather than as an error by the author.
+
+- **`agent-harness` — *Static or dynamic, the second question*.** The workflow-versus-agent
+  table decided one thing and left the other open. Six rows, and one of them is hard: a run
+  that has to be auditable is static, because a graph that picks its own next nodes produces
+  a shape nobody drew, and then *"here is the design"* and *"here is what happened"* stop
+  being the same document.
+
+- **`agent-evals` §5a — the checker node as an evaluator that runs inside the graph.** Its
+  five catches split three code checks and two judge calls, so §5's *cheap checks first*
+  applies to a position in the graph rather than to a suite. And the part that makes it eval
+  work: **a checker that has never rejected anything is a finding, not a reassurance** — its
+  verdicts are scores with a source, and its rejection rate belongs on the same dashboard as
+  its pass rate.
+
+- **A sixth scanner detector, `unguarded-fanout`.** `asyncio.gather` with no
+  `return_exceptions=True`, or `Promise.all` with no `allSettled` and no per-branch
+  `.catch`: the first sibling to fail cancels the batch, the others' completed work is
+  discarded, and the node consuming the results cannot tell a failed branch from an empty
+  one. Conservative like the rest — the capturing form anywhere in the file silences it.
+
+### Changed
+
+- **The self-test proves silence as well as noise.** `PLANTS` and `CLEAN` replaced the
+  single dict, because one detector now reads two languages and needed a plant in each, and
+  because a detector that fires on the defect *and* on its fix has no discriminating power.
+  A fan-out that **does** capture its branches is now a fixture the pass must stay silent
+  on. `self-test: 6/6` → **`9/9`** (seven plants, two clean fixtures), counted by running it.
+
+- **`agent-orchestrator` §5 no longer contradicts its own data model.** `PlanStage` declared
+  `depends_on` and the executor beside it walked `plan.stages` in list order — a plan that
+  went to the trouble of saying it need not be serialised, serialised. It now executes in
+  dependency layers (Kahn), gates a layer of more than one on the checker before anything
+  consumes it, and `ExecutionPlan` grows the `layers()` that makes the declaration mean
+  something. A cycle fails the plan rather than deadlocking the run.
+
+### Removed
+
+- **The hardcoded context-window table.** `references/patterns.md` carried nine vendor model
+  ids with their windows and a `DEFAULT_CONTEXT_WINDOW` of 16 000. Every number was correct
+  when written and none survived: ids were renamed, long-context variants shipped under the
+  same family name, and a system reading that table would size its budget for a window an
+  order of magnitude smaller than the one it was given. Replaced by the resolution order —
+  configuration, then the provider, then a conservative floor **with a loud log line** — and
+  the check is mechanical: no vendor model id remains anywhere in the shipped skill text.
+  Two illustrative ones in `SKILL.md` and `llm-proxy-billing.md` went with it, because a
+  class fixed in one place and left in two others is not fixed.
+
+- **Two duplicated homes.** §8's learning tables and §10's prompt-assembly snippet restated
+  what `references/patterns.md` and `agent-harness/references/system-prompt.md` already own.
+  Both now state the decision and point at the home. `SKILL.md` is **502 lines / 5670
+  tokens** against a 4750 working budget — the number is counted, it is over, and the next
+  addition to this body should split rather than absorb.
+
+## [0.9.0] — 2026-08-15
+
+### Added
+
+- **`agent-harness/references/pi.md` and `pi-sdk.md` — the harness doctrine as a worked
+  implementation.** Every other reference in that skill states a rule; **Pi** is small enough
+  to read and complete enough to have made each of those decisions in public. So each section
+  says what Pi does and then **which rule it is an instance of** — the second half is the
+  point, and where Pi disagrees with this pack, that is said rather than smoothed over.
+
+  Read from `pi.dev/docs/latest` on 2026-08-15: sixteen doc pages plus the package source.
+  All sixteen verified reachable (`200`); the only non-resolving URLs in either file are the
+  two deliberate placeholders inside custom-provider examples.
+
+- **`pi.md`** — the four ways to run it; **sessions as a JSONL tree** (8-hex `id`, `parentId`,
+  version 3, `BranchSummaryEntry`) matched to *time travel and forking* in
+  `agent-orchestrator/references/runtime.md`; **compaction with the real numbers**
+  (`contextTokens > contextWindow - reserveTokens`, defaults 16,384 and 20,000, tool results
+  truncated to 2,000 chars while summarizing) matched to the compaction ladder, with what the
+  ladder adds that Pi leaves to you; settings precedence that **merges rather than replaces**;
+  skills, prompt templates and packages; and the trust model.
+
+  Its sharpest section is **the deliberate absence of a sandbox**, quoted: *"prompt injection
+  from repository files … is expected local-agent risk and cannot be reliably prevented by
+  pi."* That is `layers.md`'s delegation thesis stated by the project itself — and for an
+  audit it changes the finding, because "no permission model" here is a delegation, not a
+  defect. Three containerization patterns are compared by **where credentials end up**, which
+  is the question that actually decides between them.
+
+- **`pi-sdk.md`** — `createAgentSession()`, `ModelRuntime`, `SessionManager`, `defineTool()`,
+  `DefaultResourceLoader`; the RPC protocol with its command groups, its full event lifecycle
+  and the **`\n`-only JSONL framing warning**; JSON mode's delta-only records and why;
+  the `ExtensionAPI` surface; and **the eight seams** — `tool_call` (can block),
+  `tool_result` (a middleware chain), `context`, `before_agent_start`, the three provider
+  hooks, and the compaction pair — each matched to the doctrine it lets you implement.
+  `tool_call` blocking is called out as the single most important one for an audit: it is
+  where a per-tool, per-caller policy can actually live.
+
+- Noted with its evidence and its limit: **Pi discovers skills from `~/.agents/skills/`**,
+  which on this machine is the ssheleg hub — 72 entries, every family skill carrying the
+  `name` and `description` front matter Pi requires. Stated as a fact about the path and the
+  front matter, **not** as an observed load: Pi is not installed here. The reference also
+  names Pi's documented divergence from the Agent Skills standard (a skill name may differ
+  from its directory) and warns that `make-skill`'s validator enforces the strict rule.
+
+### Changed
+
+- `references/layers.md` points at the two new files as the worked example of the kernel
+  layer it describes abstractly.
+- `agent-harness`'s description gains the embedding triggers (`agent SDK`, `embed an agent`,
+  `Pi harness`, `встроить агента`) — 807/1024, inside the family's 970 working budget. The
+  repository's own front-matter gate caught a first draft at 1066 and refused the write.
+
+
 ## [0.8.0] — 2026-08-14
 
 ### Added

package/README.md

@@ -11,7 +11,7 @@ Part of the [ssheleg skill family](https://github.com/ssheleg/sshlg-skills).
 
 Four skills — `agent-orchestrator` for wiring the loop, `agent-evals` for proving it
 behaves, `agent-interop` for everything it talks to outside its own process,
-`agent-harness` for what it is **told** — and sixteen references they load on demand,
+`agent-harness` for what it is **told** — and nineteen references they load on demand,
 plus one scanner.
 
 **The orchestrator** (`SKILL.md`) — what the agent reads first:
@@ -31,6 +31,10 @@ plus one scanner.
   decay, plus conflict resolution when a new learning contradicts an old one
 - context budget allocation by priority
 - self-learning feedback loops
+- **the shape of the work, decided before the work**: an edge that carries no data is
+  no edge, a plan that declares `depends_on` is executed in dependency layers rather
+  than in list order, and a parallel layer gets a checker before the node that
+  consumes it
 
 **The evals skill** (`agent-evals/SKILL.md`) — how you know any of it works. An
 agent's behaviour is not in its source, so the artifact under test is the
@@ -61,19 +65,29 @@ ACP, AGNTCY, AP2, Agent Skills — so an agent stops guessing.
 model, and the one where most agent bugs actually live: *the biggest performance
 improvements often come from clearly explaining tool usage in the system prompt*, and *even
 small refinements to tool descriptions can yield dramatic improvements*. Before adding a
-retry or a sub-agent, it asks four questions about the text. Five references —
+retry or a sub-agent, it asks four questions about the text. Seven references —
 `system-prompt.md` (right altitude, enumerated vocabulary, and the three things reasoning
 models changed — starting with **do not add chain-of-thought**), `tools.md` (the
 agent–computer interface, with a worked before/after and poka-yoke), `techniques.md`
 (fifteen techniques, a verdict each **for production** rather than a benchmark),
 `layers.md` (which layer you are building at, and why permission boundaries are usually the
-environment's job), `audit.md` (seven tracks, evidence tiers, a plan instead of a score).
+environment's job), `audit.md` (seven tracks, evidence tiers, a plan instead of a score) —
+plus **`pi.md` and `pi-sdk.md`, the doctrine as a worked implementation**: Pi read end to
+end, each mechanism matched to the rule it instantiates, its divergences named, and the
+eight extension seams where a permission gate or a context rewrite can actually live.
 
 It runs in both directions: **building a harness and auditing somebody else's are one
 checklist read forwards and backwards.** `scripts/audit_agent.py` is the mechanical half —
-five conservative detectors, and it always prints what it *cannot* see plus a denominator,
+six conservative detectors, and it always prints what it *cannot* see plus a denominator,
 so its silence is never read as a pass.
 
+**`references/graph-engineering.md`** — deciding the shape of the work before
+doing it: node and edge, the fake-edge test, the diamond and the two ways it
+fails silently, the checker node and what it costs, static versus dynamic with
+auditability as the hard rule, when a graph is not worth building, and what a
+host actually executes when it fans out — with the version evidence, because the
+keyword the source named was renamed six weeks after it was published.
+
 **`references/context-engineering.md`** — what the loop gives up when the window
 runs out: the five-rung compaction ladder and why to re-measure between rungs,
 the tool-pair boundary invariant, typed carryover blocks copied across the

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@ssheleg/agent-stack",
-  "version": "0.8.0",
+  "version": "0.10.0",
   "description": "Production patterns for AI agent orchestrators \u2014 tool-calling loops, multi-stage pipelines with checkpoints, LLM provider routing with fallback, four-layer memory with confidence decay \u2014 plus the wallet side of reselling LLM access. This package is the installer CLI.",
   "bin": {
     "agent-stack": "bin/agent-stack.js"

package/plugins/agent-stack/.claude-plugin/plugin.json

@@ -2,7 +2,7 @@
   "name": "agent-stack",
   "displayName": "Agent Stack",
   "description": "Two skills: agent-orchestrator \u2014 tool-calling loops, multi-stage pipelines with checkpoints, provider routing with fallback, four-layer memory, context engineering, plus the wallet side of reselling LLM access; and agent-evals \u2014 run/trace/thread evals, judges, and fixtures grown from production.",
-  "version": "0.8.0",
+  "version": "0.10.0",
   "author": {
     "name": "ssheleg",
     "url": "https://x.com/sshlg93"

package/plugins/agent-stack/skills/agent-evals/SKILL.md

@@ -7,9 +7,10 @@ description: >-
   evals. Covers the three observability primitives (run, trace, thread) crossed with three
   eval granularities (single-step, full-turn, multi-turn), the offline/online/ad-hoc
   timing axis, pass-fail rubrics over scalar scores, cheap code checks before model
-  judges, simulated users with adversarial personas, annotation queues, and what to
-  instrument so any of it is possible. Triggers - "agent eval", "eval suite", "LLM judge",
-  "regression fixture", "trajectory eval", "is the agent getting better", "эвалы агента",
+  judges, the checker node as an evaluator running inside the graph, simulated users with
+  adversarial personas, annotation queues, and what to instrument so any of it is
+  possible. Triggers - "agent eval", "eval suite", "LLM judge", "regression fixture",
+  "trajectory eval", "checker node", "is the agent getting better", "эвалы агента",
   "оценка агента", "LLM-судья", "регрессионный набор", "как проверить агента". Not for
   unit tests of ordinary code, or for benchmarking a model.
 license: MIT
@@ -159,6 +160,44 @@ domain expert and accept that this tier stays human.
 
 ---
 
+## 5a. The checker node — an evaluator that runs inside the graph
+
+Everything above evaluates a run *afterwards*. One evaluator runs **during** it, and it is
+the one most systems are missing: a **checker node** sitting between a parallel layer and
+the node that consumes it. Its only job is *usable / not usable*, and the convergence
+depends on **it** rather than on the branches — otherwise the gate has a bypass.
+
+It matters here because it is the same machinery as §5, positioned differently:
+
+| It catches | Decided by |
+|---|---|
+| an empty or null output | a code check |
+| a confidence signal below the downstream bar | a code check |
+| a shape that will break the consumer's parsing | a code check |
+| two outputs that cannot both be true | a judge |
+| an output answering a different question than the one asked | a judge |
+
+Three of five are free. Run them first — §5's *cheap checks first*, applied to a position
+in the graph rather than to a suite.
+
+**A checker is a node, so it can be wrong, and its failure mode is silent approval.** A
+model checker that has never been shown a bad input passes everything, and a graph with a
+checker that always says yes is **worse** than one with none: the missing checkpoint has
+been replaced by a false one that everything downstream now trusts. Three consequences,
+and they are eval work rather than orchestration work:
+
+- **Watch it refuse a planted bad output** before trusting it, exactly as §5 requires of
+  any judge before it scores unattended.
+- **Record every verdict as a score bound to the run**, with `source: code_check` or
+  `llm_judge` (§7). A checker whose verdicts are not stored cannot be asked afterwards how
+  often it fired, which means it cannot be calibrated.
+- **A checker that has never rejected anything is a finding, not a reassurance.** Put the
+  rejection rate on the same dashboard as the pass rate; a rate of zero is either a
+  perfect upstream or a broken gate, and only the stored verdicts can tell you which.
+
+Where the checker sits in the shape, and why the convergence needs one at all:
+`agent-orchestrator/references/graph-engineering.md`.
+
 ## 6. The corpus grows from production
 
 Never author the suite up front. Every production failure and every thumbs-down becomes a
@@ -215,6 +254,8 @@ None of the above runs without these, and they are the part people skip:
 - [ ] Pass/fail rubrics with enumerated failure conditions, written with behaviour owners
 - [ ] Code checks before model judges
 - [ ] Judge calibrated against human labels before it is trusted
+- [ ] Every checker node watched refusing a planted output, its verdicts stored as scores,
+      and its rejection rate on the dashboard — a checker at zero rejections is a finding
 - [ ] Domain-expert review for output a general judge cannot grade
 - [ ] Every production failure minimised into a permanent fixture
 - [ ] Annotation queue with filters, and the two reviewer roles kept separate

package/plugins/agent-stack/skills/agent-harness/SKILL.md

@@ -2,15 +2,16 @@
 name: agent-harness
 description: >-
   Use when the question is what the agent is TOLD rather than how its loop is wired — writing
-  or fixing a system prompt, shaping tools so the model actually picks the right one,
-  deciding whether a job wants a workflow or an agent, or choosing between ReAct, reflection,
-  planning and voting. Also the audit direction: reviewing an agent system somebody else
-  built, with tracks, evidence tiers and a prioritized plan instead of a score, plus a
-  scanner for the defects that are mechanically visible. Triggers - "system prompt", "tool
-  description", "the agent picks the wrong tool", "agent loops forever", "prompt engineering",
-  "ReAct", "reflection", "workflow or agent", "audit this agent", "review our agent system",
-  "системный промпт", "агент не вызывает тул", "аудит агента", "воркфлоу или агент". Not for
-  the loop's plumbing, its evals, or the protocols it speaks — those are the sibling skills.
+  or fixing a system prompt, shaping tools so the model picks the right one, deciding whether
+  a job wants a workflow or an agent, or choosing between ReAct, reflection and voting. Also
+  auditing an agent system somebody else built: tracks, evidence tiers and a prioritized plan
+  instead of a score, plus a scanner. Carries Pi as a worked kernel implementation — SDK, RPC
+  and extension seams — for embedding or extending a harness. Triggers - "system prompt",
+  "tool description", "agent picks the wrong tool", "agent loops forever", "prompt
+  engineering", "ReAct", "workflow or agent", "static or dynamic", "audit this agent",
+  "embed an agent", "agent SDK", "Pi harness", "системный промпт", "агент не вызывает тул",
+  "аудит агента", "встроить агента". Not for the loop's plumbing, its evals, or its
+  protocols — those are siblings.
 ---
 
 # Agent harness — what the agent is told, and how to audit what someone else told theirs
@@ -81,6 +82,30 @@ that only where a fixed path genuinely cannot be written.
 from a known set. Orchestration invents the set per request. If you can enumerate the
 branches, you wanted routing and it is cheaper.
 
+### Static or dynamic — the second question, and it is not the same one
+
+Having chosen a workflow, one thing is still open: **is its shape known before it runs?**
+A **static** graph has every node and edge decided up front. A **dynamic** one grows — a
+node finishes, reads what it found, and decides what comes next.
+
+| Reach for | When |
+|---|---|
+| **static** | the task repeats and the structure is the same every time |
+| **static** | predictability and speed matter more than flexibility |
+| **static** | **always first** — go dynamic only after the static version hits a wall you can name |
+| dynamic | the scope of the work depends on what is discovered along the way |
+| dynamic | a node must choose its successors from its own output |
+| **never dynamic** | **the run has to be auditable** — see below |
+
+**The audit rule is hard, not a preference.** A dynamic graph's executed shape is not the
+shape anybody drew, so *"here is the design"* and *"here is what happened"* stop being the
+same document and every claim about the run becomes unfalsifiable from outside. Most
+workflows that feel like they need a dynamic graph need a better static one.
+
+The rest of the model — the fake-edge test, the diamond, the checker node before a
+convergence, and what a host actually executes when it fans out — is
+`agent-orchestrator/references/graph-engineering.md`.
+
 ---
 
 ## References
@@ -95,6 +120,8 @@ and `test/validate.py` fails the build on a reference that does not say when it
 | [`references/techniques.md`](references/techniques.md) | you are choosing between **ReAct, reflection, voting, planning** and the rest — every entry carries a verdict for production, not a benchmark score |
 | [`references/layers.md`](references/layers.md) | deciding **what your harness owns** — kernel, workbench and product layers, and why permission boundaries are usually somebody else's job |
 | [`references/audit.md`](references/audit.md) | reviewing **an agent system you did not build** — seven tracks, evidence tiers, and a prioritized plan |
+| [`references/pi.md`](references/pi.md) | you want the doctrine above as a **worked implementation** — Pi's sessions, compaction, config, skills, trust and containerization, each matched to the rule it implements, and the places it deliberately disagrees |
+| [`references/pi-sdk.md`](references/pi-sdk.md) | you are **embedding or extending** a harness — the SDK, the RPC protocol, JSON mode, and the eight extension seams where a permission gate, a context rewrite or a cost hook can actually live |
 
 **`scripts/audit_agent.py`** — the mechanical half of the audit. It finds what is visible
 without understanding intent (an unbounded loop, a tool with no description, a swallowed
@@ -149,6 +176,7 @@ prompt.
 ## Checklist — a harness worth shipping
 
 - [ ] Workflow-versus-agent decided deliberately, and the simpler option was actually tried
+- [ ] Static-versus-dynamic decided too, and a run that must be auditable is static
 - [ ] System prompt at the **right altitude** — heuristics, not hardcoded branches, not vague hope
 - [ ] Every status, category and enum the agent must produce is **enumerated in the prompt**
 - [ ] Today's date, and any other volatile context, injected rather than assumed

package/plugins/agent-stack/skills/agent-harness/references/layers.md

@@ -62,6 +62,10 @@ surrounds it. Read as a weakness, it looks like a missing feature. Read as archi
 is a clear statement: *a harness that also claims to be a sandbox is claiming a guarantee it
 cannot keep*, because it runs in the same process as the code it would be confining.
 
+**Worked out in full in `pi.md` and `pi-sdk.md`** — the kernel layer read end to end, with
+each mechanism matched to the rule it implements and its divergences named. Use it as the
+concrete example when this file's abstractions need a body.
+
 Two consequences worth stating plainly:
 
 - **A harness advertising "safe tool execution" without an OS-level or container boundary is