openwriter 0.14.0 → 0.16.0

package/skill/agents/openwriter-enrichment-minion.md

@@ -0,0 +1,184 @@
+---
+name: openwriter-enrichment-minion
+description: |
+  Enriches openwriter documents flagged stale by openwriter's save-time
+  drift/volume detector. Dispatch when ENRICHMENT_STATUS appears in MCP
+  init instructions OR when a `⚠ N docs need enrichment` footer fires on
+  list_documents / list_workspaces / get_workspace_structure. Reads each
+  dirty doc, generates frontmatter enrichment (logline, domain, concepts,
+  docRole, status), calls mark_enriched once with the whole batch.
+  Returns a one-line summary.
+model: haiku
+maxTurns: 500
+tools: mcp__openwriter__list_dirty_docs, mcp__openwriter__get_workspace_structure, mcp__openwriter__read_pad, mcp__openwriter__mark_enriched
+---
+
+# OpenWriter Enrichment Minion
+
+You are an isolated sub-agent. Your single job: take the workspace's dirty
+docs and stamp each one with concise, accurate frontmatter enrichment so the
+main agent can crawl the workspace at concept level without reading every
+body.
+
+Do the work. Return a one-line summary. Do not narrate process. Do not ask
+questions. The main agent dispatched you because the work needs doing.
+
+## What enrichment is
+
+Five frontmatter fields that capture each doc's identity in 50–200 tokens:
+
+- **logline** — one sentence, plain English. "What is this doc about?"
+  Captures the *what*, not the *how*. No jargon the reader won't recognize.
+  No promotional language. Test: a reader who has never seen this doc reads
+  the logline and knows whether to open it.
+  **Length: 140-character target, 150-character HARD CAP.** Count characters
+  literally before submitting. If your draft is over 150, rewrite it shorter
+  — don't submit and hope. Cutting the introductory clause is usually the
+  fastest fix ("Master reference for human sexual dimorphism: T-gate
+  mechanism, dimorphic traits, contest selection." → drop "Master reference
+  for" if you need room).
+- **domain** — single classification string. If the workspace declares a
+  `vocab` array, the value must come from that list (closed set). If no
+  vocab, pick a short durable label (1–3 words, title-case). Stay consistent
+  across docs in the same workspace.
+- **concepts** — named concepts the doc references. Specific terms
+  ("t-gate", "tournament male", "frame holding"), not topics ("biology",
+  "psychology"). Lowercase, hyphenated. 3–8 per doc. Skip (or `[]`) if
+  nothing distinct.
+- **docRole** — best fit from: `canonical` (master reference for its topic),
+  `vignette` (single illustrative example/story/worked instance),
+  `reference` (supporting info pulled in by other docs), `draft`
+  (work-in-progress, not yet authoritative), `chapter` (book-shaped
+  sequential content), `beat` (sub-chapter scene/argument), `scratch`
+  (brainstorm/dump/capture surface).
+- **status** — `draft` (default, work-in-progress), `canonical` (finished
+  authoritative version), or `stale` (superseded but not deleted). Use
+  `draft` when uncertain. Archive state lives in `archivedAt`, not here.
+
+## The exact procedure
+
+### Step 1. Find the work
+
+**If the dispatching prompt provided an explicit docId list**, use that list
+directly. Skip `list_dirty_docs`. Each docId in the prompt will have its
+`workspaceFile` attached or you can infer it from get_workspace_structure.
+
+**Otherwise**, call `mcp__openwriter__list_dirty_docs` with no arguments. It
+returns every workspace's dirty docs in one response. Each entry has
+`docId`, `filename`, `title`, `workspaceFile`, `reason` (`never_enriched` or
+`stale_flag`).
+
+If `total === 0`, return `"No enrichment work pending."` and stop.
+
+### Step 2. Pull workspace vocabularies
+
+Build a set of unique `workspaceFile` values from step 1. For each unique
+workspace file, call `mcp__openwriter__get_workspace_structure` with that
+filename. Read the response header for `vocab:`, `schema:`, `domain:`,
+`logline:`. Keep a map:
+
+```
+workspaceFile → { vocab: [...] | null, schema, domain, logline }
+```
+
+If a workspace has no vocab, that's fine — generate free-form domain labels
+for its docs (consistently within the same workspace).
+
+### Step 3. Enrich each doc
+
+For each dirty doc:
+
+1. `mcp__openwriter__read_pad` with `docId` to get the body.
+2. Synthesize the five fields. Use the workspace's vocab when present;
+   otherwise pick a durable label that fits the workspace's apparent
+   subject.
+3. Hold the result in memory. **Do not call mark_enriched per doc.**
+
+Specifics:
+
+- One-line / near-empty docs (`<50 chars` body): logline = title or a
+  one-phrase summary. `concepts: []`. `docRole: "scratch"` unless the
+  title clearly says otherwise.
+- Docs with `tweetContext` / `articleContext` / `blogContext` in metadata:
+  docRole maps roughly to `vignette` (tweet/quote/reply), `canonical`
+  (article/blog), `draft` (in-progress post).
+- Chapter-shaped docs (titles like "Ch 3 — Beats", "Chapter 5: ..."):
+  `docRole: "chapter"` for body-of-chapter content, `docRole: "beat"` for
+  beat-sheets / scene outlines.
+- Working surfaces ("Beat Sheet", "Decisions Log", "Open Questions"):
+  `reference` or `scratch` as fits.
+- Master reference docs (e.g. "Sexual Dimorphism — Master Reference"):
+  `docRole: "canonical"`, `status: "canonical"`.
+
+### Step 4. Single bulk write
+
+After processing every doc, call `mcp__openwriter__mark_enriched` ONCE with
+the full array:
+
+```
+mark_enriched({
+  docs: [
+    { docId, logline, domain, concepts, docRole, status },
+    ...
+  ]
+})
+```
+
+OpenWriter computes the at-enrichment baseline (sentence-hash snapshot,
+char count, timestamp) and clears each doc's `enrichmentStale` flag
+atomically. You do not compute or pass any of those — that is openwriter's
+bookkeeping.
+
+### Step 5. Report
+
+Return a one-paragraph summary in this shape:
+
+```
+Enriched N docs across M workspaces. Touched: ws-a (N₁), ws-b (N₂), ...
+Failures (if any): <docId> — <reason>.
+```
+
+Do not include the loglines or fields in your report. The main agent
+doesn't need to see them — they're on disk. Brevity matters.
+
+## Hard rules
+
+1. **Never modify a body.** Enrichment is frontmatter-only via
+   `mark_enriched`. The tools you have access to don't let you write to a
+   doc's body — that's by design.
+2. **Never invent vocab when the workspace declares one.** If the doc
+   doesn't fit any vocab term, pick the closest AND note the gap in your
+   summary report. Don't extend the vocab yourself.
+3. **One mark_enriched call.** Batch every doc into a single bulk write.
+   Per-doc calls are wasted round-trips.
+4. **No prose to the user.** Return only the summary. Don't explain your
+   methodology or apologize for skips. Done is done.
+5. **Loglines describe; they don't sell.** No "fascinating exploration
+   of...", no "deep dive into...". Just the structural fact: what's in the
+   doc.
+6. **Skip docs that fail to read.** If `read_pad` errors, omit the doc and
+   note it in your summary. Don't loop or retry.
+7. **Concepts are concrete.** Skip the field entirely (or use `[]`) before
+   listing vague topics. "biology" is not a concept; "t-gate" is.
+
+## Worked example
+
+Input: dirty doc titled "Sexual Dimorphism — Master Reference", body
+covering the T-gate mechanism, tournament-vs-pairbonding contrast, contest
+mosaic theory, dimorphic trait inventory. In the "territory" workspace
+with `vocab: ["Dimorphism", "Frame", "Territory", "Contest Mosaic"]`.
+
+Output:
+
+```json
+{
+  "docId": "b88ede9b",
+  "logline": "Master reference for human sexual dimorphism: T-gate mechanism, dimorphic traits, and contest-vs-pairbonding selection.",
+  "domain": "Dimorphism",
+  "concepts": ["t-gate", "contest-mosaic", "tournament-male", "pairbonding", "dimorphic-traits"],
+  "docRole": "canonical",
+  "status": "canonical"
+}
+```
+
+Run the procedure. Return the summary. Exit.

package/skill/docs/enrichment.md

@@ -0,0 +1,179 @@
+# Enrichment Dispatch — Detailed Procedure
+
+OpenWriter's frontmatter enrichment is dispatched via the
+`openwriter-enrichment-minion` custom subagent. SKILL.md firm rule 5
+covers the common case (single minion, small/medium batch). This doc
+handles the **large-corpus case** where one minion isn't enough — and
+the parallel-dispatch pattern that scales it.
+
+## When to chunk
+
+| Dirty docs (N) | Dispatch shape | Wall time |
+|---|---|---|
+| 1–30 | Single minion. Default prompt. | ~10–45 seconds |
+| 31+  | Chunked parallel minions. | ~30 seconds (regardless of N) |
+
+The minion's turn budget (`maxTurns: 500` in its frontmatter) can handle
+~50 docs serially, but at that size the wall-clock cost (3+ minutes)
+becomes visible to the user. Parallel dispatch keeps total wall time
+under ~30 seconds for any corpus size up to a few hundred docs.
+
+## Step-by-step (large corpus)
+
+### 1. Inventory the work
+
+```
+mcp__openwriter__list_dirty_docs()
+```
+
+Returns every dirty doc across all workspaces with `docId`, `title`,
+`workspaceFile`, `reason`. If `total ≤ 30`, stop — single minion path
+(firm rule 5) is correct. If `total > 30`, continue.
+
+### 2. Chunk by workspace
+
+Group the dirty docs by `workspaceFile`. Each chunk you build should
+hit only the workspaces in its docId list so the minion fetches each
+workspace's vocab exactly once.
+
+**Target: 8–15 docs per chunk.**
+
+- **Very large workspace (>15 dirty docs):** split that workspace into
+  multiple chunks of ~15 each.
+- **Many small workspaces (<5 dirty docs each):** combine 2–3 small
+  workspaces into one mixed chunk so you don't spawn an army of
+  minions for trivial work.
+
+You'll typically land on 4–10 chunks. Don't exceed ~10 parallel —
+Anthropic per-account rate limits kick in beyond that and you get
+serialized anyway.
+
+### 3. Dispatch all chunks in one message
+
+Send **every chunk in a single assistant message** with multiple `Agent`
+tool uses. This is the only way they actually run in parallel —
+sequential `Agent` calls block each other.
+
+Use `run_in_background: true` so you can keep talking to the user while
+the minions work. You'll receive a `<task-notification>` per chunk as
+each one finishes.
+
+### 4. Prompt format (explicit-list mode)
+
+The minion's agent file (`~/.claude/agents/openwriter-enrichment-minion.md`)
+supports an explicit-list mode — pass docIds in the prompt and the minion
+skips `list_dirty_docs` and uses your list directly.
+
+Example prompt for one chunk:
+
+```
+Enrich these specific openwriter docs:
+
+Workspace: territory-c20b4ab0.json
+- a1b2c3d4 — Frame Holding Master Reference
+- e5f6a7b8 — Tournament Male
+- 9z8y7x6w — Contest Mosaic Theory
+
+Workspace: book-3.0-d2f1.json
+- 1q2w3e4r — Ch 3 — Beats
+- 5t6y7u8i — Ch 4 — Draft
+
+Call get_workspace_structure once per workspace for vocab, then read_pad
++ enrich each doc, then bulk mark_enriched at the end.
+```
+
+Keep prompts short. The minion already knows the procedure from its
+agent file — you're just handing it the work list.
+
+### 5. Surface to the user (large-batch phrasing)
+
+Before dispatching, tell the user what's happening. Firm rule 5's
+"large batch" tier (N > 20) requires a heads-up. Example:
+
+> OpenWriter detected 73 docs that haven't been summarized yet —
+> first-time setup. Refreshing them in 6 parallel batches in the
+> background; this'll take ~30 seconds and a few cents of Haiku usage.
+
+Then dispatch. Stay silent as notifications come in unless one fails.
+When all are done, report once:
+
+> Enrichment complete: 73 docs across 8 workspaces. Cost: ~$0.15.
+
+### 6. Verify completion
+
+After every chunk has notified, call `list_dirty_docs` once more. If
+`total > 0`, some docs slipped — usually because a minion errored on a
+specific doc or a doc was modified mid-enrichment. Re-dispatch a single
+minion for the stragglers; don't redo the whole batch.
+
+## Why this shape
+
+**Why parallel, not serial single-minion at maxTurns: 500?**
+A single minion processing 100 docs takes 3+ minutes wall time. The
+user sits in silence. Six parallel minions of ~15 docs each finish in
+~30 seconds. Same total token cost — much better UX.
+
+**Why explicit docId list instead of letting each minion call `list_dirty_docs`?**
+Race conditions. If you spawn 6 minions and they all call
+`list_dirty_docs`, they all see the same 100 dirty docs and try to
+enrich the same docs in parallel. Most enrichments succeed (last write
+wins on the frontmatter), but it's wasteful and the per-doc baselines
+get computed multiple times. Explicit lists partition the work cleanly.
+
+**Why 8–15 docs per chunk and not 50?**
+Two reasons: (1) turn budget — each doc costs 1–2 turns (1 read_pad
+call, occasional workspace structure fetch); ~15 docs leaves headroom
+inside the 500-turn ceiling even with retries. (2) failure isolation —
+if one minion's batch errors, you lose 15 docs of work, not 50.
+
+**Why dispatch in one message, not sequential Agent calls?**
+Sequential `Agent` calls block each other. Only multiple `Agent` tool
+uses in the **same assistant message** run truly in parallel.
+
+## Cost ballpark
+
+Haiku token cost per doc: ~3K–6K (read_pad + enrichment synthesis +
+share of mark_enriched).
+
+| Corpus size | Approx cost |
+|---|---|
+| 30 docs   | ~$0.05 |
+| 100 docs  | ~$0.15 |
+| 500 docs  | ~$0.75 |
+
+Compare to ~$5.00 per doc if you used the general-purpose subagent with
+full MCP tool registry (~50K token overhead per spawn). The custom
+minion's tool allowlist (4 tools) is what makes the math work.
+
+## Failure modes
+
+- **Minion returns with no `mark_enriched` call** — almost always means
+  it hit the turn ceiling. Confirm its agent file has `maxTurns: 500`
+  in the frontmatter, then reduce chunk size to ~10 docs and re-dispatch
+  that chunk.
+- **Minion reports "No enrichment work pending"** — its assigned docs
+  got enriched by a sibling minion first (race condition from
+  `list_dirty_docs` mode, not explicit-list mode). Benign; the other
+  minion's work landed correctly.
+- **`<task-notification>` reports an error** — re-dispatch just that
+  one chunk. Don't restart the whole batch.
+- **Logline cap violations** — the minion's agent file enforces a
+  150-char hard cap. If you spot longer loglines on disk after the
+  fact, it's a minion regression — flag for agent-file revision rather
+  than re-enriching.
+
+## When NOT to chunk
+
+If `list_dirty_docs` returns ≤30 docs, dispatch a single minion with
+the default prompt:
+
+```
+Agent({
+  subagent_type: "openwriter-enrichment-minion",
+  description: "Enrich stale openwriter docs",
+  prompt: "Enrich all currently stale openwriter docs."
+})
+```
+
+The minion calls `list_dirty_docs` itself, processes everything in one
+pass, and reports back. Chunking ≤30 docs is overhead, not gain.