openwriter 0.15.0 → 0.17.0

package/skill/agents/openwriter-enrichment-minion.md

@@ -0,0 +1,177 @@
+---
+name: openwriter-enrichment-minion
+description: |
+  Enriches openwriter documents flagged stale by openwriter's save-time
+  drift/volume detector. Dispatch when ENRICHMENT_STATUS appears in MCP
+  init instructions OR when a `⚠ N docs need enrichment` footer fires on
+  list_documents / list_workspaces / get_workspace_structure. Reads each
+  dirty doc, generates frontmatter enrichment (logline, domain, concepts,
+  docRole, status), calls mark_enriched once with the whole batch.
+  Returns a one-line summary.
+model: haiku
+maxTurns: 500
+tools: mcp__openwriter__list_dirty_docs, mcp__openwriter__get_workspace_structure, mcp__openwriter__read_pad, mcp__openwriter__mark_enriched
+---
+
+# OpenWriter Enrichment Minion
+
+You are an isolated sub-agent. Your single job: take the workspace's dirty
+docs and stamp each one with concise, accurate frontmatter enrichment so the
+main agent can crawl the workspace at concept level without reading every
+body.
+
+Do the work. Return a one-line summary. Do not narrate process. Do not ask
+questions. The main agent dispatched you because the work needs doing.
+
+## What enrichment is
+
+Five frontmatter fields that capture each doc's identity in 50–200 tokens:
+
+- **logline** — précis (non-fiction) or logline (fiction) summarizing the
+  content. Under 250 chars. No scaffolding — describe the content itself,
+  not the kind of doc it is.
+- **domain** — single classification string. If the workspace declares a
+  `vocab` array, the value must come from that list (closed set). If no
+  vocab, pick a short durable label (1–3 words, title-case). Stay consistent
+  across docs in the same workspace.
+- **concepts** — named concepts the doc references. Specific terms
+  ("t-gate", "tournament male", "frame holding"), not topics ("biology",
+  "psychology"). Lowercase, hyphenated. 3–8 per doc. Skip (or `[]`) if
+  nothing distinct.
+- **docRole** — best fit from: `canonical` (master reference for its topic),
+  `vignette` (single illustrative example/story/worked instance),
+  `reference` (supporting info pulled in by other docs), `draft`
+  (work-in-progress, not yet authoritative), `chapter` (book-shaped
+  sequential content), `beat` (sub-chapter scene/argument), `scratch`
+  (brainstorm/dump/capture surface).
+- **status** — `draft` (default, work-in-progress), `canonical` (finished
+  authoritative version), or `stale` (superseded but not deleted). Use
+  `draft` when uncertain. Archive state lives in `archivedAt`, not here.
+
+## The exact procedure
+
+### Step 1. Find the work
+
+**If the dispatching prompt provided an explicit docId list**, use that list
+directly. Skip `list_dirty_docs`. Each docId in the prompt will have its
+`workspaceFile` attached or you can infer it from get_workspace_structure.
+
+**Otherwise**, call `mcp__openwriter__list_dirty_docs` with no arguments. It
+returns every workspace's dirty docs in one response. Each entry has
+`docId`, `filename`, `title`, `workspaceFile`, `reason` (`never_enriched` or
+`stale_flag`).
+
+If `total === 0`, return `"No enrichment work pending."` and stop.
+
+### Step 2. Pull workspace vocabularies
+
+Build a set of unique `workspaceFile` values from step 1. For each unique
+workspace file, call `mcp__openwriter__get_workspace_structure` with that
+filename. Read the response header for `vocab:`, `schema:`, `domain:`,
+`logline:`. Keep a map:
+
+```
+workspaceFile → { vocab: [...] | null, schema, domain, logline }
+```
+
+If a workspace has no vocab, that's fine — generate free-form domain labels
+for its docs (consistently within the same workspace).
+
+### Step 3. Enrich each doc
+
+For each dirty doc:
+
+1. `mcp__openwriter__read_pad` with `docId` to get the body.
+2. Synthesize the five fields. Use the workspace's vocab when present;
+   otherwise pick a durable label that fits the workspace's apparent
+   subject.
+3. Hold the result in memory. **Do not call mark_enriched per doc.**
+
+Specifics:
+
+- One-line / near-empty docs (`<50 chars` body): logline = title or a
+  one-phrase summary. `concepts: []`. `docRole: "scratch"` unless the
+  title clearly says otherwise.
+- Docs with `tweetContext` / `articleContext` / `blogContext` in metadata:
+  docRole maps roughly to `vignette` (tweet/quote/reply), `canonical`
+  (article/blog), `draft` (in-progress post).
+- Chapter-shaped docs (titles like "Ch 3 — Beats", "Chapter 5: ..."):
+  `docRole: "chapter"` for body-of-chapter content, `docRole: "beat"` for
+  beat-sheets / scene outlines.
+- Working surfaces ("Beat Sheet", "Decisions Log", "Open Questions"):
+  `reference` or `scratch` as fits.
+- Master reference docs (e.g. "Sexual Dimorphism — Master Reference"):
+  `docRole: "canonical"`, `status: "canonical"`.
+
+### Step 4. Single bulk write
+
+After processing every doc, call `mcp__openwriter__mark_enriched` ONCE with
+the full array:
+
+```
+mark_enriched({
+  docs: [
+    { docId, logline, domain, concepts, docRole, status },
+    ...
+  ]
+})
+```
+
+OpenWriter computes the at-enrichment baseline (sentence-hash snapshot,
+char count, timestamp) and clears each doc's `enrichmentStale` flag
+atomically. You do not compute or pass any of those — that is openwriter's
+bookkeeping.
+
+### Step 5. Report
+
+Return a one-paragraph summary in this shape:
+
+```
+Enriched N docs across M workspaces. Touched: ws-a (N₁), ws-b (N₂), ...
+Failures (if any): <docId> — <reason>.
+```
+
+Do not include the loglines or fields in your report. The main agent
+doesn't need to see them — they're on disk. Brevity matters.
+
+## Hard rules
+
+1. **Never modify a body.** Enrichment is frontmatter-only via
+   `mark_enriched`. The tools you have access to don't let you write to a
+   doc's body — that's by design.
+2. **Never invent vocab when the workspace declares one.** If the doc
+   doesn't fit any vocab term, pick the closest AND note the gap in your
+   summary report. Don't extend the vocab yourself.
+3. **One mark_enriched call.** Batch every doc into a single bulk write.
+   Per-doc calls are wasted round-trips.
+4. **No prose to the user.** Return only the summary. Don't explain your
+   methodology or apologize for skips. Done is done.
+5. **Loglines describe; they don't sell.** No "fascinating exploration
+   of...", no "deep dive into...". Just the structural fact: what's in the
+   doc.
+6. **Skip docs that fail to read.** If `read_pad` errors, omit the doc and
+   note it in your summary. Don't loop or retry.
+7. **Concepts are concrete.** Skip the field entirely (or use `[]`) before
+   listing vague topics. "biology" is not a concept; "t-gate" is.
+
+## Worked example
+
+Input: dirty doc titled "Sexual Dimorphism — Master Reference", body
+covering the T-gate mechanism, tournament-vs-pairbonding contrast, contest
+mosaic theory, dimorphic trait inventory. In the "territory" workspace
+with `vocab: ["Dimorphism", "Frame", "Territory", "Contest Mosaic"]`.
+
+Output:
+
+```json
+{
+  "docId": "b88ede9b",
+  "logline": "Master reference for human sexual dimorphism: T-gate mechanism, dimorphic traits, and contest-vs-pairbonding selection.",
+  "domain": "Dimorphism",
+  "concepts": ["t-gate", "contest-mosaic", "tournament-male", "pairbonding", "dimorphic-traits"],
+  "docRole": "canonical",
+  "status": "canonical"
+}
+```
+
+Run the procedure. Return the summary. Exit.

package/skill/docs/enrichment.md

@@ -0,0 +1,179 @@
+# Enrichment Dispatch — Detailed Procedure
+
+OpenWriter's frontmatter enrichment is dispatched via the
+`openwriter-enrichment-minion` custom subagent. SKILL.md firm rule 5
+covers the common case (single minion, small/medium batch). This doc
+handles the **large-corpus case** where one minion isn't enough — and
+the parallel-dispatch pattern that scales it.
+
+## When to chunk
+
+| Dirty docs (N) | Dispatch shape | Wall time |
+|---|---|---|
+| 1–30 | Single minion. Default prompt. | ~10–45 seconds |
+| 31+  | Chunked parallel minions. | ~30 seconds (regardless of N) |
+
+The minion's turn budget (`maxTurns: 500` in its frontmatter) can handle
+~50 docs serially, but at that size the wall-clock cost (3+ minutes)
+becomes visible to the user. Parallel dispatch keeps total wall time
+under ~30 seconds for any corpus size up to a few hundred docs.
+
+## Step-by-step (large corpus)
+
+### 1. Inventory the work
+
+```
+mcp__openwriter__list_dirty_docs()
+```
+
+Returns every dirty doc across all workspaces with `docId`, `title`,
+`workspaceFile`, `reason`. If `total ≤ 30`, stop — single minion path
+(firm rule 5) is correct. If `total > 30`, continue.
+
+### 2. Chunk by workspace
+
+Group the dirty docs by `workspaceFile`. Each chunk you build should
+hit only the workspaces in its docId list so the minion fetches each
+workspace's vocab exactly once.
+
+**Target: 8–15 docs per chunk.**
+
+- **Very large workspace (>15 dirty docs):** split that workspace into
+  multiple chunks of ~15 each.
+- **Many small workspaces (<5 dirty docs each):** combine 2–3 small
+  workspaces into one mixed chunk so you don't spawn an army of
+  minions for trivial work.
+
+You'll typically land on 4–10 chunks. Don't exceed ~10 parallel —
+Anthropic per-account rate limits kick in beyond that and you get
+serialized anyway.
+
+### 3. Dispatch all chunks in one message
+
+Send **every chunk in a single assistant message** with multiple `Agent`
+tool uses. This is the only way they actually run in parallel —
+sequential `Agent` calls block each other.
+
+Use `run_in_background: true` so you can keep talking to the user while
+the minions work. You'll receive a `<task-notification>` per chunk as
+each one finishes.
+
+### 4. Prompt format (explicit-list mode)
+
+The minion's agent file (`~/.claude/agents/openwriter-enrichment-minion.md`)
+supports an explicit-list mode — pass docIds in the prompt and the minion
+skips `list_dirty_docs` and uses your list directly.
+
+Example prompt for one chunk:
+
+```
+Enrich these specific openwriter docs:
+
+Workspace: territory-c20b4ab0.json
+- a1b2c3d4 — Frame Holding Master Reference
+- e5f6a7b8 — Tournament Male
+- 9z8y7x6w — Contest Mosaic Theory
+
+Workspace: book-3.0-d2f1.json
+- 1q2w3e4r — Ch 3 — Beats
+- 5t6y7u8i — Ch 4 — Draft
+
+Call get_workspace_structure once per workspace for vocab, then read_pad
++ enrich each doc, then bulk mark_enriched at the end.
+```
+
+Keep prompts short. The minion already knows the procedure from its
+agent file — you're just handing it the work list.
+
+### 5. Surface to the user (large-batch phrasing)
+
+Before dispatching, tell the user what's happening. Firm rule 5's
+"large batch" tier (N > 20) requires a heads-up. Example:
+
+> OpenWriter detected 73 docs that haven't been summarized yet —
+> first-time setup. Refreshing them in 6 parallel batches in the
+> background; this'll take ~30 seconds and a few cents of Haiku usage.
+
+Then dispatch. Stay silent as notifications come in unless one fails.
+When all are done, report once:
+
+> Enrichment complete: 73 docs across 8 workspaces. Cost: ~$0.15.
+
+### 6. Verify completion
+
+After every chunk has notified, call `list_dirty_docs` once more. If
+`total > 0`, some docs slipped — usually because a minion errored on a
+specific doc or a doc was modified mid-enrichment. Re-dispatch a single
+minion for the stragglers; don't redo the whole batch.
+
+## Why this shape
+
+**Why parallel, not serial single-minion at maxTurns: 500?**
+A single minion processing 100 docs takes 3+ minutes wall time. The
+user sits in silence. Six parallel minions of ~15 docs each finish in
+~30 seconds. Same total token cost — much better UX.
+
+**Why explicit docId list instead of letting each minion call `list_dirty_docs`?**
+Race conditions. If you spawn 6 minions and they all call
+`list_dirty_docs`, they all see the same 100 dirty docs and try to
+enrich the same docs in parallel. Most enrichments succeed (last write
+wins on the frontmatter), but it's wasteful and the per-doc baselines
+get computed multiple times. Explicit lists partition the work cleanly.
+
+**Why 8–15 docs per chunk and not 50?**
+Two reasons: (1) turn budget — each doc costs 1–2 turns (1 read_pad
+call, occasional workspace structure fetch); ~15 docs leaves headroom
+inside the 500-turn ceiling even with retries. (2) failure isolation —
+if one minion's batch errors, you lose 15 docs of work, not 50.
+
+**Why dispatch in one message, not sequential Agent calls?**
+Sequential `Agent` calls block each other. Only multiple `Agent` tool
+uses in the **same assistant message** run truly in parallel.
+
+## Cost ballpark
+
+Haiku token cost per doc: ~3K–6K (read_pad + enrichment synthesis +
+share of mark_enriched).
+
+| Corpus size | Approx cost |
+|---|---|
+| 30 docs   | ~$0.05 |
+| 100 docs  | ~$0.15 |
+| 500 docs  | ~$0.75 |
+
+Compare to ~$5.00 per doc if you used the general-purpose subagent with
+full MCP tool registry (~50K token overhead per spawn). The custom
+minion's tool allowlist (4 tools) is what makes the math work.
+
+## Failure modes
+
+- **Minion returns with no `mark_enriched` call** — almost always means
+  it hit the turn ceiling. Confirm its agent file has `maxTurns: 500`
+  in the frontmatter, then reduce chunk size to ~10 docs and re-dispatch
+  that chunk.
+- **Minion reports "No enrichment work pending"** — its assigned docs
+  got enriched by a sibling minion first (race condition from
+  `list_dirty_docs` mode, not explicit-list mode). Benign; the other
+  minion's work landed correctly.
+- **`<task-notification>` reports an error** — re-dispatch just that
+  one chunk. Don't restart the whole batch.
+- **Logline cap violations** — the minion's agent file enforces a
+  150-char hard cap. If you spot longer loglines on disk after the
+  fact, it's a minion regression — flag for agent-file revision rather
+  than re-enriching.
+
+## When NOT to chunk
+
+If `list_dirty_docs` returns ≤30 docs, dispatch a single minion with
+the default prompt:
+
+```
+Agent({
+  subagent_type: "openwriter-enrichment-minion",
+  description: "Enrich stale openwriter docs",
+  prompt: "Enrich all currently stale openwriter docs."
+})
+```
+
+The minion calls `list_dirty_docs` itself, processes everything in one
+pass, and reports back. Chunking ≤30 docs is overhead, not gain.

package/skill/docs/footnotes.md

@@ -0,0 +1,178 @@
+# Footnotes — Author Guide for Agents
+
+OpenWriter supports CommonMark / Pandoc footnote syntax for citation-heavy
+long-form writing. The editor renders inline references as superscript
+chips and corrals definitions into an end-of-doc "Footnotes" section.
+
+This doc explains how to write footnotes via MCP tools and what to expect
+on disk and in the editor.
+
+## The syntax (Pandoc / CommonMark)
+
+Two parts:
+
+- **Reference** (inline): `text[^N]` — appears in the prose
+- **Definition** (block): `[^N]: footnote text` — appears at end of doc
+
+Labels can be numeric or mnemonic:
+
+```markdown
+Humans run fixed action patterns[^1] too.
+
+Per Sapolsky[^sapolsky2017], stress responses follow a pattern.
+
+[^1]: Eibl-Eibesfeldt 1973, replicated and extended by Galati et al. 2003.
+
+[^sapolsky2017]: Sapolsky, R. (2017). *Behave*. Penguin Press.
+```
+
+The author label (`1` or `sapolsky2017`) is what pairs reference to
+definition. **Display numbering is automatic** — the editor's CSS counter
+shows sequential `[1] [2] [3]` regardless of label. Mnemonic labels stay
+on disk for human-readable file diffs.
+
+## How to write footnotes from MCP
+
+Just include the syntax in your markdown content — no special tool needed.
+
+### populate_document (initial draft)
+
+```ts
+populate_document({
+  docId: "abc12345",
+  content: `# Chapter 1
+
+Theory of mind develops late[^1] in non-human primates.
+
+[^1]: Premack & Woodruff (1978), Behavioral and Brain Sciences 1: 515–526.
+`
+})
+```
+
+The parser handles `[^1]` references and `[^1]: ...` definitions. The
+editor renders the reference as a superscript chip and the definition
+inside an end-of-doc "Footnotes" section.
+
+### write_to_pad (adding to existing doc)
+
+To add a new footnote to existing prose, you have two paths.
+
+**Append a new reference + definition together** (recommended):
+
+```ts
+// Step 1: rewrite the paragraph to add the reference
+write_to_pad({
+  docId: "abc12345",
+  changes: [
+    {
+      operation: "rewrite",
+      nodeId: "para_id",
+      content: "The same sentence now with a new claim[^2]."
+    }
+  ]
+})
+
+// Step 2: append the definition. If the doc already has a footnoteSection,
+// you can insert the definition inside it via afterNodeId pointing at the
+// last definition. If the doc has no footnotes yet, the parser auto-creates
+// the section when it sees `[^N]: ...` at the end of the markdown body.
+```
+
+**Simpler: just include both the reference and the definition in one
+write_to_pad call**:
+
+```ts
+write_to_pad({
+  docId: "abc12345",
+  changes: [
+    {
+      operation: "rewrite",
+      nodeId: "para_id",
+      content: "Sentence with new claim[^2]."
+    },
+    {
+      operation: "insert",
+      afterNodeId: "end",
+      content: "[^2]: Smith et al. (2020), Nature 580: 142–148."
+    }
+  ]
+})
+```
+
+The serializer normalizes definitions to the end-of-doc `footnoteSection`
+regardless of where they're inserted in the tree.
+
+## What you see in `read_pad`
+
+```
+title: My Chapter
+id: abc12345
+words: 423
+pending: 0
+---
+[h1:aa0001] Chapter 1
+[p:bb0002] Theory of mind develops late[^1] in non-human primates.
+[fnsec:cc0003]
+  [fndef:dd0004] [^1]: Premack & Woodruff (1978), Behavioral and Brain Sciences 1: 515–526.
+```
+
+The `[^N]` in the body is the inline reference. `[fnsec:...]` is the
+end-of-doc section. `[fndef:...]` is each definition.
+
+## Per-doc scope — important
+
+**Footnote labels are local to each doc.** Chapter 3's `[^1]` does not
+refer to Chapter 4's `[^1]`. Each chapter is its own `.md` file with its
+own numbering. Cross-chapter references are not supported at the editor
+level (a future book-export pipeline will handle global numbering at
+typeset time).
+
+If the author writes "see Ch 1 note 4" they're writing prose, not a
+cross-doc footnote link.
+
+## Multi-paragraph definitions
+
+Pandoc allows multi-paragraph footnotes via 4-space-indented continuation:
+
+```markdown
+[^1]: First paragraph of the definition.
+
+    Continuation paragraph, indented 4 spaces.
+
+    Another continuation.
+```
+
+The editor preserves the multi-paragraph structure inside the definition.
+Use this for footnotes that need substantial explanation (lengthy
+methodology notes, multi-source citations, etc.).
+
+## What's NOT supported (yet)
+
+- **Cross-doc footnote references.** Each doc has its own numbering.
+- **Bibliography auto-generation.** Authors manage citation text inline.
+  Zotero / Mendeley / BibTeX integration is a future enhancement.
+- **DOI auto-resolution.** Footnote text is plain — paste a DOI manually.
+- **Per-page footnotes (Phase 3).** The editor uses end-of-doc placement;
+  per-page placement is a print-layout concern handled at book-export
+  time, not in the editor.
+
+## When to use footnotes vs inline parentheticals
+
+Use footnotes when:
+- The citation count exceeds ~3 per ~500 words (inline parentheticals
+  start visibly disrupting the prose at that density)
+- The audience expects an academic register (popular nonfiction in the
+  Sapolsky / Wrangham / Pinker lineage)
+- The work targets book-class output (cumulative citation load at book
+  scale destroys readability under inline parentheticals)
+
+Use inline parentheticals when:
+- Citations are sparse (<1 per 500 words) and short
+- The author prefers a journalistic register
+- The work is short-form (tweet thread, blog post) where there's no
+  end-of-doc section to defer to
+
+## Reference docs (for the editor maintainers, not agents)
+
+- `docs/footnotes.md` (in the openwriter repo): full architecture
+- `adr/footnote-system.md`: load-bearing invariants + decision log