@imunitic/synapse 0.0.1-test.0

package/skills/synapse-node-authoring/SKILL.md

@@ -0,0 +1,188 @@
+---
+name: synapse-node-authoring
+description: How to write every node's prose in a build — sequentially by default, or fanned out to a configurable pool of concurrent subagents. Covers pool-size resolution from synapse.conf, the synapse brief data file each author reads, dispatch/verify/retry mechanics, and the standing contract every author works under. Use at /synapse-init's node-authoring step, or anywhere else a batch of nodes needs fresh prose (e.g. /synapse-rebuild-diff's re-orient class).
+---
+
+# Writing every node's prose: sequential by default, pooled on request
+
+Loaded wherever a *batch* of nodes needs prose written, not a single lazy regeneration (that
+is the `synapse-node` skill's job, one node, triggered by a stale read). Today's caller is
+`/synapse-init`'s node-authoring step. `/synapse-rebuild-full` inherits this for free — it
+delegates to `/synapse-init`'s procedure by reference rather than repeating it.
+`/synapse-rebuild-diff`'s *re-orient* class is a natural second caller (see the design's own
+Open Questions and the vault's `inbox/wire synapse-rebuild-diff's re-orient class onto the
+parallel-authoring skill` note) but is not wired to this skill yet — it still
+gathers its facts ad hoc rather than through `rank --lists`/`link-graph`, and needs that fixed
+first.
+
+This skill owns two things: **deciding how many authors run at once**, and **the standing
+contract every author works under**, whichever pool size is in play. Which nodes need
+authoring, and how each node's facts are computed (`build-lists`, `rank --lists`,
+`build-refs` + `link-graph`), stays the caller's job — those steps run before this skill is
+ever loaded.
+
+## 1. Resolve the pool size
+
+```sh
+pool="${SYNAPSE_AUTHOR_POOL:-}"
+if [ -z "$pool" ]; then
+  pool="$(grep -m1 '^SYNAPSE_AUTHOR_POOL=' ~/.claude/synapse.conf 2>/dev/null | cut -d= -f2- | tr -d '"')"
+fi
+pool="${pool:-0}"
+```
+
+Environment variable wins over the conf file, same precedence every other Synapse setting
+uses. Absent, empty, or malformed all fall through to `0`. **This is read by you, the
+orchestrating agent, directly — not by any Zig binary.** Nothing compiled dispatches
+subagents, so there is nothing for a `synapse` flag to feed. The conf file is the shared home
+for the setting; the reader is markdown, not code.
+
+`pool = 0` is not "a pool of zero workers" — go to §2. `pool >= 1` is a real worker pool — go
+to §3.
+
+## 2. Pool = 0: the original inline procedure
+
+Author every node yourself, one after another, in the same session — cross-node memory
+intact, no subagent, no brief file needed. For each node, in list order:
+
+```sh
+synapse rank --sources "$SYNAPSE_WORK_DIR/lists/NN.txt" --pool summary
+synapse rank --sources "$SYNAPSE_WORK_DIR/lists/NN.txt" --pool crux
+awk -F'\t' '$1 == "{Node Title}"' "$SYNAPSE_WORK_DIR/links.tsv"
+```
+
+Read the top few files each pool names, read this node's link rows for `## Links`
+candidates, then write `$SYNAPSE_WORK_DIR/b-NN.md` yourself, following the
+`synapse-node-format` contract. This is the whole procedure — no dispatch, no verification
+step beyond what you'd naturally do writing anything else.
+
+**This is the default, deliberately.** It's the safest choice, the one every existing
+build has already used, and the only mode that preserves the authorial consistency a
+single continuous session gives you across nodes — something no subagent-based mode, even
+at pool 1, can offer. Fan-out is opt-in, not assumed.
+
+## 3. Pool ≥ 1: fan out to a worker pool
+
+### 3a. Compute the briefs, once
+
+```sh
+synapse rank --lists "$SYNAPSE_WORK_DIR/lists"
+synapse brief --lists "$SYNAPSE_WORK_DIR/lists"
+```
+
+`build-refs` + `link-graph` should already have run by this point in `/synapse-init` (its own
+step 6) regardless of pool size; `rank --lists` has not, since §2 uses `rank --sources`
+instead and only this branch needs the batched form. `brief` reshapes both into one file per
+node — it computes nothing itself. Writes `$SYNAPSE_WORK_DIR/brief/NN.md` per node: the
+node's title, its source list's path and count, both ranked pools verbatim, its own rows from
+`links.tsv`, and every node's title in the namespace (for judging `part_of`). A brief is pure
+data — no instructions, no contract. That lives here and in the prompt you write per author, so
+the same brief means the same thing to a sequential fallback author and a concurrent one.
+
+### 3b. Dispatch: a refilling pool, not fixed batches
+
+Fixed batches (`dispatch 8, wait for all 8, dispatch the next 8`) are bounded by their
+slowest member — seven fast nodes sit idle waiting on one large straggler. A refilling pool
+never has that gap: dispatch up to `pool` nodes, and on every completion notification, verify
+that node (§3d) then dispatch the next queued node if any remain. Stop when the queue is
+empty and the last in-flight authors have reported back.
+
+This also matches how completions actually arrive in this harness — one notification at a
+time, in a later turn, no "wait for all N" primitive — so a pool needs no barrier-counting a
+fixed batch would.
+
+### 3c. Each author's brief
+
+One `Agent` call per node, background, general-purpose. The prompt is self-contained — a
+fresh subagent has no memory of this session:
+
+```
+Load the synapse-node-format skill first — it is the node contract you are writing against.
+
+Read your brief: {abs path to brief/NN.md}. It is data: the facts to work from, not
+instructions.
+
+Write this node's prose to: {abs path to b-NN.md}, following synapse-node-format exactly.
+Write nothing else, read nothing this brief doesn't point you at, and do not read any other
+node's brief or body.
+
+Decide `part_of` (containment) from the brief's "every node" list and its candidate links —
+it is a judgement call, never computed. `depends_on`/`uses` come from the candidate links
+section; pick whichever reads right per edge, and prune freely — the table is evidence for a
+candidate list, not something to copy verbatim.
+
+Never hard-wrap. Write each paragraph as one single unbroken line and let the editor soft-wrap
+it — a newline exists only where a real break is intended (between paragraphs, list items,
+headings). This is a vault-wide rule, not specific to this node, and it is not optional: a
+hard-wrapped paragraph renders as a ragged stack of short lines in Obsidian instead of flowing
+text.
+```
+
+Same model as the orchestrating session, no override — matches the constraint that "a
+parallel author produces exactly what a sequential one does," and keeps the quality
+comparison in the design's own checklist item to one variable (isolation/concurrency) rather
+than two (that, plus a cheaper model).
+
+**Why this is spelled out explicitly rather than assumed inherited:** a prior pooled run shipped
+hard-wrapped nodes despite the vault's no-hard-wrap rule living in global `CLAUDE.md`
+instructions — evidently not reliably carried into a fresh subagent's behavior on its own.
+Loading `synapse-node-format` does not cover it either, since that skill is about the node's
+structure, not the vault's prose-formatting convention. State it here, every time, rather than
+assuming it travels for free.
+
+### 3d. Verify on completion, retry once, then fall back
+
+When a completion notification arrives, read `b-NN.md` yourself — this is a check you read the
+file for, same as the section check next to it, not a shell one-liner (this codebase's own
+frontmatter reader deliberately avoids `grep`/`sed`/`awk` for exactly this field, per
+`core/query.zig`'s docstring — a hand-rolled pattern here would drift from it the same way the
+gap this section exists to close first happened). Confirm the file exists, opens with
+frontmatter carrying a non-empty `summary:` field (`synapse-node-format`'s own contract — see
+its "Each `b-NN.md` opens with its own one-line summary in frontmatter" line), and has
+`## Summary`, `## Crux`, and `## Links` sections. Pass → dispatch the next queued node (§3b) and
+move on.
+Fail (missing file, missing frontmatter summary, missing section, obviously truncated) → **one
+retry**, same brief, fresh subagent. A second failure → **write that one node yourself**,
+inline, the §2 procedure, rather than blocking the rest of the pool on it. Report which nodes
+needed a retry or a fallback in the final summary — a silent recovery hides a brief that might
+be systematically wrong for a whole class of node.
+
+Check the frontmatter here, at authoring time, rather than leaving it to `push-nodes` — a body
+that has all three sections but no `summary:` field passes every check above and then fails at
+push, node by node, after the whole pool has already finished and moved on.
+
+**Also check for hard-wrapping** — a prose paragraph (not a list) whose lines break before
+reaching a natural sentence boundary. A quick heuristic: within `## Summary`, a run of two or
+more consecutive non-blank lines that are *not* list items (don't start with `-`/`*`/a digit)
+is a hard-wrap, not a paragraph — a real paragraph is one line. Treat this as the same class of
+failure as a missing section: fix by rewriting the paragraph as one line yourself (this is a
+formatting fix, not a content one, so it doesn't need a fresh subagent) rather than letting it
+ship and pushing.
+
+### 3e. Pool = 1 is a real, useful degenerate case of this same pipeline
+
+Not a synonym for §2. It still bundles a brief, dispatches to an isolated subagent, verifies,
+and retries-then-falls-back — only the concurrency is 1. Useful on its own terms: it isolates
+whether a quality difference against the sequential baseline comes from *isolation* (no
+author sees another's output or memory — present even at pool 1) or from *concurrency*
+itself (only present at pool ≥ 2). Reach for it specifically when running the design's
+still-open "compare prose quality against a sequential build" checklist item, before
+committing to a higher pool size.
+
+## Guardrails
+
+- **No author writes shared state.** Not another node's `b-NN.md`, not the tags cache, not
+  the vault. Every fact an author needs is in its own brief, computed before dispatch and
+  read-only during it — this is what removes the concurrent-writer problem rather than merely
+  managing it.
+- **No author reads another node's brief or body.** Overlap in *source* reads (two nodes'
+  file lists sharing a path) is expected and accepted — see the design's Open Questions — but
+  an author's own inputs are its brief and nothing else.
+- **Never grow the pool mid-run.** The size is resolved once at the start (§1); do not read
+  `synapse.conf` again partway through a build.
+- **A retry gets the same brief, not a rewritten one.** If the brief itself was the problem,
+  fixing it is a §3a-level fix (regenerate every brief and restart), not something to
+  improvise per-node inside the retry.
+- **`part_of` is never computed, at any pool size.** It is containment, not reference, and
+  deriving it from directory nesting would turn a folder layout into a claim about concepts —
+  the same reasoning the design's own Alternatives section already rejected this on.

package/skills/synapse-node-format/SKILL.md

@@ -0,0 +1,205 @@
+---
+name: synapse-node-format
+description: The contract for a Synapse code-graph node — frontmatter fields, the crux pointer, `## Links`, `grounded_in`, `## Sources`, and what `synapse write-node` adds or refuses. Load before authoring or regenerating any node, whether from /synapse-init's first build, /synapse-rebuild's triage, or the synapse-node skill's lazy regeneration. Not for reading the graph (that is synapse-query) or for task notes in the vault (that is synapse-task).
+---
+
+# What a node is, and how to author one
+
+Every component that writes a node loads this: `/synapse-init` (first build), the `synapse-node`
+skill (Tier 2 lazy regeneration), and `/synapse-rebuild` (reseat, patch, re-orient).
+
+All three write the same artifact, so the format belongs in one place rather than being restated
+wherever it is used. What stays with each caller is what is genuinely specific to it — the skill's
+traps when *re*-authoring an existing node, rebuild's triage classes — not the contract itself.
+
+`synapse write-node` is the enforcement. Where this document states a rule the writer already
+refuses to break, that is deliberate and the script is the authority; this describes the artifact so
+a reader knows what to produce, not so anyone hand-builds one.
+
+Author the prose only — put each node's content in
+`$SYNAPSE_WORK_DIR/b-NN.md` (matching its `lists/NN.txt`), then run `synapse push-nodes`,
+which calls `synapse write-node` per node. The contract below is what that writer implements
+and what `synapse query stale` verifies; it is specified here because the two must agree
+exactly, not because you should hand-build the file. A node lands at
+`synapse/{repo}@{branch}/{Node Title}.md`:
+
+Each `b-NN.md` opens with its own one-line summary in frontmatter, so everything authored about a
+node is in one file:
+
+```markdown
+---
+summary: One line differentiating this node from its siblings.
+---
+
+## Summary
+...
+```
+
+The driver strips that frontmatter and the line becomes the node's `summary` field, which
+`synapse build-project-index` reads back to build the index bullet. Write it *for the index* — it has to distinguish this node
+from dozens of siblings, which is a different job from the node's opening sentence, whose job is
+to orient someone already inside. A node without one is an error, not a default.
+
+- **Filename/title:** short, senior-engineer-style description of the concept (e.g. "World —
+  entity/component/resource core"). Filesystem-illegal characters (`/ : * ? " < > |`) are
+  sanitized — but **reword the title instead of relying on that**, because Obsidian resolves a
+  wikilink by *filename*, so `[[World — entity/component/resource core]]` silently resolves to
+  nothing once the file becomes `...entity_component_resource core.md`. A broken wikilink is a
+  valid link to a not-yet-existing note, so it fails quietly. The writer warns when a title needs
+  sanitizing; treat that warning as "rename this node". Same trap when you *retitle* a node
+  mid-build: inbound links already written keep pointing at the old name.
+- **`sources`:** **every** file the node covers — repo-relative path plus that file's
+  `git hash-object <path>` output, run from the repo root at the moment of writing. Exhaustive,
+  not a sample: this is a **machine** field, and it is what makes Obsidian's search able to reach
+  a node from any file it covers (searching a class name that appears in no node's prose still
+  finds its node via this list). Do **not** trim it to a handful of "representative" files —
+  doing so silently destroys that lookup, leaves the node unable to answer "which files am I
+  about", and reduces hash verification to whatever survived the trim. Readability pressure
+  belongs on `## Sources` below, never here.
+- **`sources_digest`:** `sha256` over the sorted `path:hash` lines of `sources` (see "Computing
+  `sources_digest`" below). Lets a staleness check answer "has this node changed" by reading one
+  field instead of every hash.
+- **`built_at`:** machine local time (`date '+%Y-%m-%d %H:%M'`) — never inferred.
+- **`stale`:** `false` — freshly built.
+- **Body:** `summary` (plain-English, the explanation a senior engineer would give walking
+  someone through this subsystem), `crux` (the few lines that carry the actual logic — **authored
+  as line numbers, stored as text**: you point, the writer slices, so composing is impossible at
+  authoring time and nothing decays afterwards the way a stored line number would), `links` (typed
+  Obsidian wikilinks to other nodes in this same namespace: `depends_on`, `part_of`, `uses`, or
+  another type that fits better if one doesn't — for `depends_on`/`uses` specifically, `/synapse-init`
+  computes candidates before any node exists via `synapse link-graph`; read that node's rows from
+  `links.tsv` rather than guessing which siblings it relates to, `part_of` stays a judgement call
+  with nothing mechanical behind it), a `## Sources` section, and an empty `## Notes` section.
+- **Break a "does N things" enumeration into real bullets, not inline `(1)/(2)/(3)`.** A sentence
+  enumerating three or more parallel sub-points reads as a wall of text once each item carries its
+  own clause or parenthetical -- the node is read by a human skimming it in Obsidian as much as by
+  an agent (see the design note this format came from: hosting the graph as vault-readable markdown
+  was chosen specifically so it stays "just as readable by a human directly in Obsidian as it is by
+  Claude"), and a dense inline run-on defeats that. Use a markdown bullet list under the sentence
+  introducing them instead. This is narrow, not a general "prefer bullets" rule: an aside of one or
+  two items, or connected causal narrative ("X, because Y, which is why Z"), stays flowing prose --
+  over-bulleting ordinary narrative just trades one readability problem for another.
+- **Never write crux code. Point at it and let the writer cut it out.** In the body, emit a
+  directive instead of a code block:
+
+  ```
+  ## Crux
+  <!-- crux: crates/matcher/src/lib.rs 412-419 -->
+  ```
+
+  `synapse write-node` slices those lines out of the file, fences them with a language guessed
+  from the extension, appends a `path:start-end` provenance line, and records `crux_path` /
+  `crux_lines` in frontmatter. It refuses the write if the path is not one the node claims, if the
+  range runs past the end of the file, or if the span reaches 20 lines.
+
+  This exists because a typed crux can be a paraphrase that merely *looks* like a quote — the
+  invented `trait Matcher { /* no engine assumptions */ }` reads perfectly and is worth nothing. A
+  rule saying "quote, don't compose" depends on compliance; pointing makes composing impossible,
+  which is the same mechanics-belong-to-the-tooling move as the rest of the write path.
+
+- **Ground the summary in what the codebase asserts about itself.** Prefer, in this order: a test
+  (its name and assertions state behaviour that CI checks on every commit — the strongest evidence
+  short of running the code), a doc comment or module header (the author's own claim of intent), and
+  only then your own reading. Point at each piece of evidence, repeatably, anywhere in the body:
+
+  ```
+  <!-- grounded_in: src/test/java/PremiumTest.java 42-48 -->
+  <!-- grounded_in: src/main/java/Premium.java 10-14 -->
+  ```
+
+  The writer records each as path + lines + sha256 of the sliced text in the `grounded_in`
+  frontmatter list, then strips the directives — this is provenance, not display, so nothing is
+  rendered and the prose stays readable. `synapse query grounding` later re-slices each range and
+  compares, which turns "is this summary still true" from a judgement into a check.
+
+  Why it matters more than it looks: a claim traced to a test or a doc comment is one the codebase
+  made, so even a *wrong* doc comment beats an invented explanation — it is attributable and
+  findable. What this is guarding against is the confident causal story assembled from a stray
+  import ("output stays coherent because the printer is behind a mutex"), which reads exactly like
+  understanding and can be false from the moment it is written.
+
+  Not every sentence can be grounded, and that is expected. Architectural narrative and hard-won
+  debugging findings have no test asserting them. Ground what can be grounded; do not manufacture
+  evidence for the rest, and do not water down a true synthesis just to make it citable.
+
+- **`<!-- crux: none -->` is a real answer — use it.** A trivial data holder, a one-line
+  delegation, or logic spread evenly with no focal point genuinely has no crux, and a subsystem
+  node often has none either. A required field with no honest answer is exactly how a fabricated
+  one appears, so say `none` rather than picking a span to fill the slot. If something adjacent is
+  worth quoting instead, point at the module's own doc comment — that is an honest quote of
+  something nearby, not a fabricated quote of the thing itself.
+- **Prefer claims about structure over claims about mechanism.** "These three printers implement
+  the sink interface" is checkable and stays true; "the parallel path shares a printer behind a
+  mutex, which is why output stays coherent" is the kind of causal story that is easy to assemble
+  from a stray `use std::sync::Mutex` and wrong. Every later regeneration keeps the sentences the
+  diff does not contradict, so a mechanism invented here is permanent. State one only after
+  reading the code that implements it.
+- **`## Sources` is the human mirror of `sources`, aggregated rather than enumerated:** one line
+  per owning directory or module with a file count, `LC_ALL=C` sorted. A node covering 941 files
+  would otherwise put 75 KB of paths in front of a reader who wants to know which modules are
+  involved — and the frontmatter already carries every path for search, so the mirror doesn't
+  need to repeat them. Rewritten from `sources` on every write, never hand-edited. (Obsidian's
+  Properties panel flattens the raw `sources` field into a truncated one-line string, which is
+  why a mirror exists at all — but that is an argument for aggregating *the mirror*, not for
+  trimming the field.)
+- **`## Notes` is human-authored only.** Claude never writes into it — not at build time, not at
+  regeneration. It is created empty and preserved verbatim forever after.
+- **Fence the generated region.** Everything the generator owns sits between
+  `<!-- synapse:generated:start -->` and `<!-- synapse:generated:end -->`; everything outside is
+  re-emitted byte-for-byte. This is the mechanism behind the `## Notes` guarantee — without it,
+  "preserved verbatim" is a promise with nothing enforcing it.
+
+```yaml
+---
+title: "World — entity/component/resource core"
+node_type: synapse-node
+project: acme
+sources:
+  - path: acme_ecs/world.ml
+    hash: <git hash-object output>
+  - path: acme_ecs/world.mli
+    hash: <git hash-object output>
+  # ... every file the node covers, not a selection
+sources_digest: <sha256 over the sorted "path:hash" lines>
+stale: false
+built_at: "<now>"
+---
+
+# World — entity/component/resource core
+<!-- synapse:generated:start -->
+
+## Summary
+{plain-English explanation}
+
+## Crux
+<!-- crux: {path a source line below claims} {start}-{end} -->
+{or `<!-- crux: none -->` when no single span carries it. The writer replaces
+ this directive with the sliced code, so never write the code here yourself.}
+
+## Links
+- depends_on [[Other Node Title]]
+- part_of [[Another Node Title]]
+
+## Sources
+- `acme_ecs` (2)
+<!-- synapse:generated:end -->
+
+## Notes
+
+```
+
+### Computing `sources_digest`
+
+Pin this exactly — a writer and a verifier computing it differently is a silent
+false-positive generator, and the point of the field is to be trusted without reading
+`sources` at all. Adopted from Graft's `sources_digest` so the two remain comparable:
+
+```
+digest = sha256( "\n".join(sorted( f"{path}:{hash}" for each entry in sources )) )
+```
+
+Sort the joined `path:hash` lines themselves (not the paths, then the hashes), `LC_ALL=C`,
+newline-separated, no trailing newline.
+
+`project` is the repo half of the namespace key — `synapse namespace --repo-name`, not the task-prefix scheme that `/synapse-note` uses. The writer fills it in; it is described here so the field's meaning is documented once.
+