@hraness/wordcell 0.20.0-bootstrap.1

package/skills/wordcell/references/query.md

@@ -0,0 +1,252 @@
+# Query the knowledge base
+
+Use the cheapest precise view first, then broaden. Markdown files remain the
+authority; search scores, metadata rows, and graph results are derived views.
+
+## Locate the vault
+
+- Resolve `<vault>` to the directory containing its managed or authored `index.md`, then
+  set the shell-local `KB_ROOT` to that path (`KB_ROOT=kb` from a typical
+  repository root, or `KB_ROOT=.` from inside the vault).
+- Resolve `<repository>` to the repository root when the question concerns a
+  repository path (`KB_REPO=.` from that root).
+- Read the vault's applicable agent instructions and note conventions.
+- Pass the resolved path to every `--root`; do not scan a repository root merely
+  because that is where the agent session started.
+
+## Recover a stopped session
+
+When the user asks to resume earlier work, begin with the path being changed.
+Keep each retrieval signal separate so the agent can inspect why a record was
+returned:
+
+```sh
+wordcell context packages/parser/src/index.ts --root "$KB_ROOT" --repo "$KB_REPO"
+wordcell search "why parser retries stop" --root "$KB_ROOT" --mode exact \
+  --history --repo "$KB_REPO" --json
+wordcell backlinks notes/parser-contract --root "$KB_ROOT" --json
+wordcell history notes/parser-contract --root "$KB_ROOT" \
+  --repo "$KB_REPO" --json
+```
+
+Replace the example path, query, and note ID with values from the current task.
+Read the inherited guides and authoritative Markdown returned by these views.
+Use the backlink to inspect related plans and use Git history as provenance,
+not as proof that the note remains correct. This workflow recovers only context
+that was persisted in files or Git; it does not reconstruct private chat.
+
+## Choose the retrieval lane
+
+- Repository file or directory: run `wordcell context` first. Read its inherited
+  guides root to nearest, then inspect its maintained knowledge, active plans,
+  dated research, reports, and separate historical-plan group. Open only useful
+  context hubs or records.
+- Known frontmatter field or tag such as type, status, or area: use `wordcell list`.
+- Known note title, path, or alias: use `wordcell links` or `wordcell backlinks`, which
+  resolve note identities before returning authored relationships.
+- A whole-vault structural question or relationship audit: use `wordcell graph --json`,
+  then inspect the smallest relevant portion of its canonical output.
+- A phrase, identity, or concept expressed with different vocabulary: use `wordcell search`, whose default hybrid result preserves exact and QMD evidence separately.
+- Direct provenance for one note or repository path: use `wordcell history` or
+  `wordcell history search` without changing authored metadata or links.
+- Recent captures awaiting maintained disposition: use the advisory `wordcell inbox` view.
+- Broad orientation: read `index.md`, then follow the smallest useful link trail. Use `wordcell catalog` when an exhaustive disposable inventory is actually needed.
+- A question spanning registered vaults: use `wordcell portfolio search` with a reviewed registry and explicit authorization. Use `wordcell portfolio audit` to assess IDs, duplicate authority, links, attachments, and provenance without repairing content.
+
+```sh
+wordcell context src/parser.ts --root "$KB_ROOT" --repo "$KB_REPO"
+wordcell list --root "$KB_ROOT" --scope src/parser --where type=plan --json
+wordcell list --root "$KB_ROOT" --where type=plan --where status=in-progress --sort area --json
+wordcell list --root "$KB_ROOT" --tag retrieval --sort title --json
+wordcell backlinks "Plan title or path" --root "$KB_ROOT" --json
+wordcell links "Plan title or path" --root "$KB_ROOT" --direction both --depth 1 --limit 25 --json
+wordcell relation list "Plan title or path" --root "$KB_ROOT" --json
+wordcell graph --root "$KB_ROOT" --json
+wordcell search "why browser capture uses the current tab" --root "$KB_ROOT" --json
+wordcell search "accepted ingestion plans" --root "$KB_ROOT" --where type=plan --where status=accepted --tag ingestion --json
+wordcell search "notes/write-path" --root "$KB_ROOT" --mode exact --no-history --json
+wordcell history "notes/write-path" --root "$KB_ROOT" --repo "$KB_REPO" --json
+wordcell history search src/parser.ts --root "$KB_ROOT" --repo "$KB_REPO" --json
+wordcell inbox --root "$KB_ROOT" --limit 25 --json
+wordcell portfolio search "durable memory" --registry ./kb-portfolio.json --workspace .. --shared --json
+wordcell portfolio audit --registry ./kb-portfolio.json --workspace .. --vault hraness/wordcell --vault 0thernet/jungle --strict --json
+```
+
+Portfolio selection is an access decision. Prefer repeated `--vault owner/id`
+for deliberate private or personal scope. `--shared` selects only public and
+organization entries; `--all` exists only for audit. One operation is bounded
+to 32 vaults and never silently truncates a larger registry-derived selection.
+Open returned notes before concluding. Cross-vault scores are not compared;
+federation uses deterministic reciprocal local rank and stable logical IDs.
+
+When a reviewed consumer policy needs repeatable query shorthand, pass a strict
+v1 rules file. An alias is recognized only as the first token and can add a
+query, mode, metadata filters, tags, or repository scopes without discarding
+caller constraints. Priority ordering is a separate opt-in:
+
+```sh
+wordcell search "@active-plans parser" --root "$KB_ROOT" \
+  --rules ./search-rules.json --priority --json
+wordcell portfolio search "@active-plans parser" \
+  --registry ./kb-portfolio.json --workspace .. --shared \
+  --rules ./search-rules.json --priority --json
+```
+
+`--rules` alone enables aliases but retains relevance order. `--priority`
+requires a rules file and requests `priority-then-relevance`; exact identities
+still come first, and matching selected hits carry a rule trace. Keep this
+policy beside the registry or repository that owns it. Rules do not grant
+access, choose authority, author links, or mutate Markdown.
+
+`wordcell context` prints hub and record summaries, not their bodies. Each record
+states the exact `repository_scopes` declaration that matched, the match depth,
+and whether that declaration currently names a file, directory, or absent
+future or retired path. Current memory and terminal plans stay in separate
+groups. Guides remain
+the normative, always-loaded home for ownership, required commands,
+prohibitions, invariants, and edit gates. Scope hubs are optional pull-based
+rationale, history, examples, evidence, and links; they cannot override a guide
+or become the only home of a load-bearing rule. Use `--kind file` or
+`--kind directory` when `auto` cannot classify a missing target reliably.
+
+Repeated filters use AND semantics. Metadata paths may be dotted. String and
+tag comparisons are case-insensitive; array metadata matches by membership.
+Missing sort values come last, with path as the deterministic tie-breaker.
+`--where` addresses authored frontmatter only; it does not filter derived H1
+titles or file paths. Unquoted `true`, `false`, `null`, and numeric filter
+values are typed. Keep quotes inside the argument to match a string with the
+same spelling, for example `--where 'external_id="9007199254740993"'`.
+`--scope` is an exact, case-sensitive repository-scope filter rather than a
+substring or area match. Use it when the desired authored path declaration is
+known.
+
+## Use hybrid search as discovery
+
+`wordcell search` first scans current Markdown for identity, phrase, metadata, tag,
+and prose matches. By default it runs that exact lane alongside QMD's local
+full-text and vector rankings, then combines the ranked lists while retaining
+each lane's evidence. Exact title and alias identities stay ahead of broader
+matches. The QMD path avoids query expansion and reranking models by default.
+
+The first hybrid or semantic query downloads QMD's compact local embedding
+model; later queries reuse the local cache. Prewarm explicitly when useful:
+
+```sh
+wordcell index --root "$KB_ROOT"
+```
+
+Use `--mode exact` for live model-free search, `--mode keyword` for QMD
+full-text retrieval, or `--mode semantic` for its vector lane. Repeated
+`--where`, `--has`, and `--tag` constraints are checked against the live
+Markdown snapshot. QMD has no path-allowlist search in the pinned release, so a
+selective semantic query over-fetches a bounded global window and reports a
+degraded partial lane when that window cannot prove completeness. Treat every
+retrieval rank as a lead, not a fact. Open the returned Markdown, read enough
+surrounding context, and confirm claims against linked sources or capture
+manifests.
+
+Default search also returns bounded explicit graph context around the strongest
+results. Supply `--related <note>` to seed a known neighborhood or `--no-graph`
+when structure does not help. Search does no Git work unless provenance is
+requested. Use `--history --repo <repository>` for optional recent per-note
+provenance or `--require-history --repo <repository>` when the task cannot
+proceed with a partial Git lane. `--no-history` remains an explicit compatibility
+form. Graph neighbors and Git history remain separate from primary
+text rank. They explain and expand candidates without becoming authored facts,
+links, or recency boosts.
+
+`wordcell history <note>` returns the bounded commit history already associated with
+one resolved note. `wordcell history search <query-or-path>` searches the bounded Git
+projection directly and retains hashes, subjects, matched paths, co-change
+paths, and incomplete-detail diagnostics. Git co-change is historical evidence,
+not permission to write a scope or relationship.
+
+## Reuse one snapshot in code mode
+
+For several related queries, prefer one SDK session to repeated CLI process
+startup:
+
+```ts
+import { openKnowledgeBase, packUntrustedSearchContext } from "@hraness/wordcell/sdk";
+
+const kb = await openKnowledgeBase({ root: "kb", repository: "." });
+try {
+  const result = await kb.search({
+    query: "why browser capture uses the current tab",
+    graph: { depth: 1 },
+    history: "auto",
+  });
+  console.log(packUntrustedSearchContext(result).content);
+} finally {
+  await kb.close();
+}
+```
+
+`grep`, `list`, `read`, `links`, `backlinks`, `search`, `history`, and
+`searchHistory` share one confined read-only scan. QMD and Git initialize
+lazily. The session does not watch Markdown or repository changes. Close it
+before a write and open a new session after the final refresh and check.
+
+Pass only ordinary plain objects and arrays from Wordcell or parsed JSON into
+`packUntrustedSearchContext`. A same-realm JavaScript `Proxy` is executable
+code, not inert data; isolate or serialize it before projection.
+
+When independent queries can run concurrently, compose them with
+`defineWorkflow` and `runWorkflow` or import a packaged workflow. The runner
+validates a finite acyclic graph, caps global concurrency, serializes QMD nodes,
+and keeps Git concurrency bounded. Do not bypass those resource groups with
+unbounded `Promise.all` calls. Custom workflows use the staged
+`defineWorkflow<Input>("id").node(...).output(...)` builder so dependency
+results and the final output remain typed.
+
+## Use focused structural views
+
+`wordcell graph --json` returns the current resolved wikilinks, typed relationships,
+diagnostics, and note-level connection counts without creating a second graph
+store. Use it when a question spans the vault. Prefer `wordcell relation list`,
+`wordcell backlinks`, or `wordcell links` when a known note gives you a narrower starting
+point.
+
+`wordcell links` is cycle-safe and requires an explicit traversal depth and result
+limit. `wordcell relation list` separates authored outbound assertions from derived
+inbound relationships while retaining canonical note IDs and source
+provenance. Open the returned Markdown before treating an edge as correct: a
+typed relationship records an authored assertion, not proof.
+
+If a structural question is not covered by a named command, inspect the
+bounded JSON graph in the agent or a short task-local script. Do not create or
+commit a parallel graph database merely to answer one query. A recurring query
+is evidence for a focused, tested command with an explicit output contract.
+
+## Combine meaning with structure
+
+1. For a repository-path question, use `wordcell context` before broader retrieval.
+2. Use default hybrid search to discover candidate identities when exact
+   structure does not answer the question. Read its lane evidence and partial
+   diagnostics before relying on the order.
+3. Use `wordcell list` to narrow by authored metadata such as `type`, `status`,
+   `area`, or `tags`.
+4. Use `wordcell links` at depth 1 to inspect immediate explicit relationships and
+   `wordcell backlinks` for a focused inbound view. Increase depth only when the
+   first neighborhood is insufficient. Traversal defaults to 50 notes and
+   reports truncation; lower `--limit` for tighter agent context or raise it
+   deliberately when a high-degree hub is genuinely relevant.
+5. Use `wordcell graph --json` only when the question genuinely spans multiple
+   neighborhoods; keep one-off processing task-local.
+6. Read the authoritative notes and cited captures before synthesizing.
+
+A title match may identify a prerequisite, prior version, or supporting note
+rather than the artifact that owns the current outcome. Confirm status and
+ownership in the candidate Markdown before answering or editing it.
+
+Do not infer an edge from semantic similarity, or a conclusion from a tag. Do
+not write generated backlink sections into notes. If the query exposes stale
+metadata or a broken link, repair the authored Markdown and finish with
+`wordcell refresh --root "$KB_ROOT"` and `wordcell check --root "$KB_ROOT"`.
+Close any open SDK session before that repair and reopen it after validation.
+
+An authored `index.md` may declare `kb_catalog: authored`; refresh and check
+then leave it untouched. `wordcell catalog --root "$KB_ROOT"` renders the exhaustive
+inventory on demand. A managed vault keeps the original generated-catalog
+behavior. Neither mode changes scanning, graph analysis, semantic indexing, or
+attachment validation.

package/skills/wordcell/references/refresh.md

@@ -0,0 +1,133 @@
+# Refresh a knowledge base
+
+Use a refresh-review-check loop. Keep authored prose under deliberate editorial control. A managed vault gives the marked catalog region to the tool; an authored vault leaves its complete front door untouched.
+
+## 1. Locate the vault
+
+- Resolve `<vault>` to the directory containing its managed or authored `index.md`, then
+  set the shell-local `KB_ROOT` to that path (`KB_ROOT=kb` from a typical
+  repository root, or `KB_ROOT=.` from inside the vault).
+- When the change concerns `scopes/` or an `kb:context` marker, resolve the
+  repository root and set `KB_REPO` to that path (`KB_REPO=.` from the
+  repository root).
+- Read the vault's applicable agent instructions and note conventions before editing.
+- Preserve note voice, frontmatter, filenames, and link intent unless a reported finding justifies a specific change.
+
+## 2. Refresh derived state
+
+When several agents are still editing a managed vault, do not refresh its
+shared catalog in each lane. Validate the lane's Markdown and graph facts with:
+
+```sh
+wordcell check --root "$KB_ROOT" --no-catalog
+```
+
+The integrating agent performs the managed refresh once after the lanes join.
+An authored vault declares `kb_catalog: authored` in `index.md`; its refresh has
+no catalog write and is safe from that shared generated-file hotspot.
+
+Run:
+
+```sh
+wordcell refresh --root "$KB_ROOT"
+```
+
+In managed mode this command atomically updates only the marked catalog region
+in `index.md`. In authored mode it reports the index as authored and leaves the
+file unchanged. Use `wordcell catalog --root "$KB_ROOT"` for a disposable exhaustive
+inventory in either mode. Catalog links are navigation, so they do not count as
+contextual graph edges.
+
+## 3. Review the advisories
+
+Open every reported source line and the relevant target notes before deciding whether to edit.
+
+- Repair a broken wikilink only when its intended target is clear. Otherwise, report the uncertainty.
+- Repair a broken or ambiguous typed relationship only after confirming its exact canonical target and predicate from the source note.
+- Disambiguate a wikilink with a vault-root path only after confirming the author's intent.
+- Treat a contextual orphan as a prompt to inspect the note, not as a demand to add a link.
+- Treat an unlinked title or alias mention as a candidate, not proof that the sentence should link.
+- Add a contextual wikilink only when it improves the meaning or navigation of the sentence.
+- Repair a missing, escaping, ambiguous, case-mismatched, symlinked, or hard-linked local image, PDF, or tldraw target. External URLs remain outside this attachment gate.
+- When a repository-owned wrapper adds lifecycle findings, treat them as
+  migration advisories: active plans should have descriptions and exact
+  `repository_scopes`, in-progress plans should retain execution evidence,
+  terminal plans should record a result and durable-memory disposition, and
+  maintained notes should declare `type: note` or `type: concept`.
+- When a repository-owned scope audit reports an absent active or maintained
+  scope, inspect it as possible stale routing. Future paths may intentionally
+  be absent; terminal records may intentionally retain retired paths. The
+  portable `wordcell refresh` and `wordcell check` commands do not impose this lifecycle
+  policy by themselves.
+
+Backlinks are derived from explicit contextual wikilinks and typed
+relationships. Mention and percolation candidates are derived analysis. Never
+inject reciprocal, transitive, or similarity-derived relationships or generated
+backlink sections to improve graph counts. Never mutate authored prose
+automatically or apply suggestions mechanically in bulk.
+
+Run a bounded percolation review for each materially changed note:
+
+```sh
+wordcell percolate "<changed-note-id>" --root "$KB_ROOT" --limit 25 --json
+```
+
+Open the cited notes before deciding whether to create a reusable
+`type: concept` note or a source-owned typed relationship.
+
+Intentional orphans and unlinked mentions may remain. Record the reason instead of manufacturing a connection.
+
+Review recent captures without maintained disposition when useful:
+
+```sh
+wordcell inbox --root "$KB_ROOT" --limit 25 --json
+```
+
+The inbox ignores source-to-source and catalog links. It is advisory; an
+intentional leaf capture needs no manufactured backlink.
+
+## 4. Validate changed repository-context mappings
+
+If the change adds, removes, renames, or moves a scope hub, changes its
+`type` or `scope`, or edits an `kb:context` marker, run:
+
+```sh
+wordcell agents identity "<repository-scope>" --json
+wordcell agents check --root "$KB_ROOT" --repo "$KB_REPO"
+```
+
+Use the non-mutating identity command to derive the hub path and exact marker
+when creating or moving a mapping. The check command verifies canonical IDs,
+exact repository-relative
+directory scopes, collisions, repository confinement, real scope directories
+and guide files, guide shape, and reciprocal markers. A moved scope has a new
+identity, so update the hub filename and guide marker together. An unmapped
+`AGENTS.md` is valid.
+
+Use the audit when the change affects guide structure, inheritance, or repeated
+rules:
+
+```sh
+wordcell agents audit --root "$KB_ROOT" --repo "$KB_REPO"
+```
+
+The audit runs the correctness checks and adds deterministic per-guide,
+per-section, inherited-chain, long-bullet, and exact-duplicate advisories.
+Review each advisory in context. Length is not correctness: do not move a
+load-bearing ownership rule, prohibition, command, invariant, or gate out of
+`AGENTS.md` merely to satisfy a suggested budget. Guide discovery skips common
+generated and vendor directories and never follows symbolic-link directories.
+
+## 5. Re-refresh and check
+
+After any note or link edit, run the refresh command again so derived state and advisories reflect the final content. Then run the read-only gate:
+
+```sh
+wordcell check --root "$KB_ROOT"
+```
+
+Finish only when the graph check and any required agent-context check succeed,
+the configured catalog mode is satisfied, and broken or ambiguous links,
+relationships, and local attachments are resolved. Summarize deliberate
+concept, relationship, link, scope, and mapping edits plus advisories
+intentionally left in place.

package/skills/wordcell/references/save-pdf.md

@@ -0,0 +1,125 @@
+# Save a PDF to the knowledge base
+
+Use the installed `wordcell` CLI. Resolve `<vault>` to the directory containing its
+authored or managed `index.md` front door, then set the shell-local `KB_ROOT`
+to that path (`KB_ROOT=kb` from a typical repository root).
+
+Check the local conversion routes, then capture the PDF:
+
+```sh
+wordcell doctor
+wordcell pdf "/absolute/path/to/document.pdf" --output "$KB_ROOT/articles"
+wordcell pdf "https://example.com/document.pdf" --output "$KB_ROOT/articles"
+```
+
+Pass a stable slug or replace a prior tool-owned bundle only when needed:
+
+```sh
+wordcell pdf "/absolute/path/to/document.pdf" --slug ben-leaves-zo --output "$KB_ROOT/articles"
+wordcell pdf "/absolute/path/to/document.pdf" --output "$KB_ROOT/articles" --force
+```
+
+The command installs one atomic bundle:
+
+```text
+<slug>/
+  <slug>.md
+  capture.json
+  source.pdf
+  annotations.json  # present after a reviewed annotation pass
+  assets/
+```
+
+`source.pdf` is the byte-identical input. The manifest records its original
+name, hash, byte count, page count, bounded document metadata, processed-page
+and block counts, image geometry, OCR status, and warnings without retaining
+the original absolute path. A reviewed second pass also retains the exact
+normalized annotation array as `annotations.json`; the manifest records its
+path, count, byte count, and SHA-256 so the image interpretation remains
+reproducible.
+
+## Preserve text and visual evidence
+
+Treat native PDF text and image text as two independent extraction surfaces.
+Native text supplies layout, heading, link, and reading-order evidence. Local
+OCR supplies candidate text for scans and screenshots. Keep every extracted
+image as an asset even when its text is converted to Markdown.
+
+For recognizable conversations, review the source image and turn OCR into
+readable message blocks with available platform, author, channel, and timestamp
+metadata. Preserve uncertain words explicitly instead of silently repairing
+them. Read [the image-derived text review guide](pdf-review.md) before refining
+screenshot-heavy or mixed-media PDFs.
+
+The first capture supplies stable image IDs and hashes in `capture.json`.
+For screenshot-heavy documents, write reviewed interpretations to a JSON array
+and rerun the capture:
+
+```json
+[
+  {
+    "id": "page-5-image-1-0123456789ab",
+    "sha256": "0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef",
+    "kind": "mixed",
+    "method": "agent",
+    "markdown": "> bg @ Oct 15, 2024 at 4:26 PM\n> Message text",
+    "metadata": {
+      "platform": "Slack",
+      "contentType": "message screenshot",
+      "author": "bg",
+      "timestamp": "Oct 15, 2024 at 4:26 PM",
+      "participants": ["bg"]
+    }
+  }
+]
+```
+
+```sh
+wordcell pdf "/absolute/path/to/document.pdf" \
+  --output "$KB_ROOT/articles" \
+  --annotations /tmp/pdf-image-annotations.json \
+  --force
+```
+
+Use only IDs and SHA-256 values from the first manifest. The command rejects a
+stale interpretation if the extracted image changed. Omit metadata fields that
+are not visible, and use `kind: "visual"` with an optional `alt` instead of
+inventing a transcription for a non-text image. Put only the transcribed body
+in `markdown`; the renderer owns the image embed, “Text visible in…” heading,
+and visible metadata line.
+
+Embed primarily visual images in the Markdown. For text-bearing or mixed
+images, keep the source image embedded beside the transcription so diagrams,
+photos, UI state, and spatial meaning remain inspectable. The embed is the
+source-image reference; do not duplicate it inside an annotation.
+
+Heading inference follows native font, emphasis, spacing, and page geometry.
+Review semantic hierarchy separately: typography can identify a heading without
+proving whether it is a peer or a child of the preceding section. Report an
+ambiguous or incorrect level instead of silently treating the inference as
+source truth.
+
+## Report completeness literally
+
+Use `complete` only when every page was processed and every extracted image was
+classified. Preserve `partial` when a tool, page, image, byte, time, or OCR
+boundary was reached. A usable native-text extraction does not make
+unprocessed screenshot pages complete.
+
+Review:
+
+1. Compare the manifest page count with the PDF.
+2. Review inferred headings and report ambiguous or incorrect hierarchy.
+3. Sample native-text, screenshot, scanned, and visual-only pages.
+4. Confirm every retained asset resolves from the Markdown or manifest.
+5. Check that source-image links accompany OCR-derived text.
+6. Report the output path, status, page and image counts, OCR coverage, and
+   unresolved warnings.
+
+After adding or linking the capture, run the vault's normal refresh and check:
+
+```sh
+wordcell percolate "<maintained-note-id>" --root "$KB_ROOT" --limit 25 --json
+wordcell refresh --root "$KB_ROOT"
+wordcell check --root "$KB_ROOT"
+```