@inkeep/open-knowledge 0.46.0-beta.7 → 0.46.0-beta.8

package/dist/assets/skills/project/SKILL.md

@@ -3,7 +3,6 @@ name: open-knowledge
 description: "Authoritative agent-runtime contract for working inside an OpenKnowledge project — a markdown-CRDT knowledge base exposed over MCP. Use whenever reading, listing, searching, editing, or linting any `.md` or `.mdx` file in the project, and before any `mcp__open-knowledge__*` tool call (`exec`, `search`, `write`, `edit`, `lint`, and the rest). Installed by `ok init`, so its presence means this is an OpenKnowledge project and it governs every markdown file here. Covers the read/write tool surface, grounding and linking rules, folder/template conventions, the live browser preview, and the rule that OK's MCP tools — never native file tools — handle in-scope markdown."
 compatibility: "Claude Code, Claude Desktop, Claude Cowork, Claude.ai web. Requires OpenKnowledge MCP server + code execution."
 metadata:
-  version: "0.46.0-beta.7"
   author: "Inkeep"
   repository: "https://github.com/inkeep/open-knowledge"
 ---
@@ -19,17 +18,16 @@ OpenKnowledge (OK) is a markdown-CRDT collaboration platform exposed via MCP. Th
 
 1. **Reads:** `exec("cat …")` for one doc, `exec("ls -A …")` for a directory (folder defaults + template menu), `exec("grep …")` for literal, `search` for ranked retrieval. Native `Read` / `Grep` only on source code (`.ts` / `.py` / …), never on in-scope `.md` / `.mdx`.
 2. **Writes:** `write({ document: { path, content } })` for a new or full-replace doc; `edit({ document: { path, find, replace } })` for a body find/replace; `edit({ document: { path, frontmatter } })` for a frontmatter merge-patch (`null` deletes a key). `delete({ document })` removes, `move({ from, to })` moves/renames. Body find/replace is body-only. Pass a one-line `summary` (≤80 chars, user-facing outcome) on every content write.
-3. **Preview / open a doc — determine your ONE surface FIRST (once per session).** Before opening anything, pick where to focus, stop at first match: **`OK_DESKTOP_TERMINAL` set** (check your env — `echo "$OK_DESKTOP_TERMINAL"`) → OK Desktop's own terminal → `ok open <name>` (switches this window) · have an in-app browser (Claude Code Desktop's Browser pane, Cursor, Codex) → `preview_url` then open/navigate it to the doc (Claude Desktop: `preview_start({url})` to open, `navigate({url})` to move) · else plain CLI → `ok open <name>`. `ok open <name>` opens a **doc or folder** (auto-detected — no `--folder`); add `--skill <name>` for a skill. The `previewUrl` field is a route id, **not** your open mechanism — don't hand it to a browser just because you saw it. Don't `preview_screenshot` to confirm edits. Full Step-0 procedure + per-surface how-to: `references/preview.md`.
-4. **Workflow guides:** `workflow({ kind: 'ingest' | 'research' | 'consolidate' | 'discover' })` returns a procedural guide, not data. Use it when the work fits the layer.
+3. **Preview / open a doc — determine your ONE surface FIRST (once per session).** Stop at first match: **`OK_DESKTOP_TERMINAL` set** → OK Desktop's terminal → `ok open <name>` (switches this window) · in-app browser (Claude Code Desktop's Browser pane, Cursor, Codex) → `preview_url`, then open/navigate it to the doc · else plain CLI → `ok open <name>`. `ok open <name>` opens a doc or folder (auto-detected); `--skill <name>` for a skill. The `previewUrl` field is a route id, **not** your open mechanism. Don't `preview_screenshot` to confirm edits. Full Step-0 procedure + per-surface how-to: `references/preview.md`.
+4. **Knowledge layers:** capturing a source (ingest), synthesizing findings (research), promoting a decision (consolidate) — procedures, **not tool calls**; there is no `ingest` tool. Ingest ships here (`references/ingest-and-sources.md`); research + consolidate come with the `knowledge-base` pack. Layer model + packs: `references/starter-packs.md`.
 5. **Direct questions:** a plain business question ("which customers…", "what did we decide about…") routes to `search` / `exec` + a cited chat answer — no "research" keyword needed. Persist only when durable + multi-doc + not already covered, and *offer* first. See `references/corpus-qa.md`.
-6. **Authoring or improving a skill** ("write/make a skill", "improve this skill", "turn this into a skill"): STOP and invoke the **`open-knowledge-write-skill`** skill — it owns scope choice (project vs global), the SKILL.md contract, evaluation, and install; don't improvise from this skill. Author through the `skill` target (`write({ skill })`), never a raw `.ok/skills/…` document. **Never read, diff, or edit the installed projections under `.claude/skills/` · `.cursor/skills/` · `.codex/skills/` · `.opencode/skills/` · `.pi/skills/` · `.agents/skills/`** — `install` generates them and overwrites them on the next sync; they are NOT source of truth.
+6. **Authoring or improving a skill** ("write/make a skill", "improve this skill", "turn this into a skill"): STOP and invoke the **`open-knowledge-write-skill`** skill — it owns scope choice (project vs global), the SKILL.md contract, evaluation, and install; don't improvise here. Author through the `skill` target (`write({ skill })`), never a raw path document. Skills are REAL folders under the editor dirs' `skills/` (`.claude` · `.cursor` · `.codex` · `.opencode` · `.pi` · `.agents`): ONE is the source, the rest are managed copies/symlinks. **Read and edit via the tools (`skills`, `edit({ skill })`) — they route to the source.** Never hand-edit a non-source copy: unedited copies refresh from the source, and editing one forks it into its own skill that stops refreshing.
 
 ## Tool index — 21 tools (router; the MCP tool descriptions carry each tool's full contract)
 
-- **Reads** — `exec` (primary; read-only `cat`/`ls`/`grep`/`find`/`head`/`tail`/`wc`/`sort`/`uniq`/`cut` + frontmatter/backlink/history enrichment; one command or one pipe, not a shell), `search` (ranked BM25 + recency), `history` (versions for a doc), `links` (`kind: backlinks|forward|dead|orphans|hubs|suggest`, or an array for a one-call audit), `skills` (find + read skills: omit `name` to LIST every skill across Project + Global, pass `name` to READ one — addressed by `name`+`scope`, NEVER by path), `config` (resolved config), `palette` (markdown-native authoring forms + `html preview` starters + theme tokens; `palette({ components })` for JSX schemas), `preview_url` (browser preview URL on demand), `share_link` (GitHub-substrate share URL; read-only, errors when no GitHub remote), `lint` (markdown-lint violations: pass `document` for one doc, omit for the whole project; pass `fix: true` with `document` to auto-fix fixable rules in place — attributed, live in the preview; the rest need `edit`/`write`), `audit` (every lint violation + broken internal link in one read-only report, by source file with lines; `path` scopes; for link VALIDATION use this, not `links`).
-- **Writes** — four native CRUD verbs, polymorphic over `document` / `folder` / `template` / `skill` / `asset` (pass EXACTLY ONE target, nested under its address key): `write` (create/overwrite; `write({ skill: { name, description, body, scope? } })` authors a skill → `.ok/skills/<name>/SKILL.md`, a **Draft** until `install`), `edit` (body find/replace OR frontmatter merge-patch; no asset), `delete` (remove), `move` (move/rename, rewrites referrers; a skill also takes `scope`/`toScope` to move Project↔Global — history resets, re-`install`). Output mirrors the input target key; the preview envelope (`previewUrl`, `warning`) stays top-level. Plus `install` (the Draft → Installed step for a `skill` — projects it into your editors; run after `write({ skill })`), `checkpoint` (named version), and `restore_version` (roll back). A folder's frontmatter is open-shape and self-only (does NOT cascade); templates are what new docs start with. **Authoring/improving a skill? invoke `open-knowledge-write-skill` (TL;DR #6), don't improvise.**
+- **Reads** — `exec` (primary; read-only `cat`/`ls`/`grep`/… plus frontmatter/backlink/history enrichment; one command or one pipe, not a shell), `search` (ranked BM25 + recency), `history` (doc versions), `links` (`kind: backlinks|forward|dead|orphans|hubs|suggest`, or an array for one call), `skills` (search + read: `query` → skills.sh; omit `name` to LIST managed (Project + Global); `name` READs one — by `name`+`scope`, never path), `config` (resolved config), `palette` (authoring forms + `html preview` starters + theme tokens; `palette({ components })` for JSX schemas), `preview_url` (browser preview URL on demand), `share_link` (GitHub-substrate share URL; read-only, errors without a GitHub remote), `lint` (markdown-lint violations: `document` for one doc, omit for the project; `fix: true` with `document` auto-fixes fixable rules in place — attributed, live in the preview; the rest need `edit`/`write`), `audit` (every lint violation + broken internal link in one read-only report, by source file with lines; `path` scopes; for link VALIDATION use this, not `links`).
+- **Writes** — four native CRUD verbs, polymorphic over `document` / `folder` / `template` / `skill` / `asset` (pass EXACTLY ONE target, nested under its address key): `write` (create/overwrite; `write({ skill: {…} })` authors a skill as a REAL folder at the project's default skill home — live immediately for that folder's agent), `edit` (body find/replace/frontmatter merge-patch; no asset), `delete` (remove), `move` (move/rename, rewrites referrers; a skill also takes `scope`/`toScope` for Project↔Global — history resets, re-`install`). Output mirrors the input key; the preview envelope (`previewUrl`, `warning`) stays top-level. Plus `install` (WHERE a `skill` lives: `add`/`remove` locations additively — editor ids, `agents`, or custom roots; `mode` + `convert` re-form ONLY the locations named; `source` moves the real folder. The source folder IS the skill — no "uninstall everywhere"; a skill dies only via `delete`), `import` (acquire a skill-dir into `add`'s locations; scripts never run), `checkpoint` (named version), and `restore_version` (roll back). A folder's frontmatter is open-shape and self-only (does NOT cascade); templates are what new docs start with.
 - **Conflicts** — `conflicts` (`kind: list|content`), `resolve_conflict` (write a resolution + commit; destructive). See `references/conflict-resolution.md`.
-- **Workflow** — `workflow` (`kind: ingest|research|consolidate|discover`; procedural guides, not data). See `references/workflow-guides.md`.
 
 **Self-correcting on misuse:** constraints JSON Schema can't express ("exactly one target", "`find` needs a `replace`", body-XOR-frontmatter) return `isError: true` with a one-line corrective shape. Read it and retry with that shape; don't guess.
 
@@ -44,11 +42,9 @@ Tools NOT in OK MCP (your host's): `preview_start`, `preview_screenshot`, `WebFe
 - **Glob patterns that target markdown** (`**/*.md`, `specs/**`, `reports/**`, `docs/**`) — use `exec` with `find` or `exec("ls -A <dir>")`.
 - **Dispatching the Explore / general-purpose subagent for markdown-heavy exploration** — subagents use native tools internally and bypass OK. Do markdown exploration yourself via `exec` / `search`. Subagents remain appropriate for **source-code** exploration.
 - **Native `Read` / `Grep` on in-scope markdown inside `.ok/`** — `.ok/` is in-scope; treat its `.md` / `.mdx` like any other KB file.
-- **`ls` / `cat` / `find` on `.ok/skills/` to discover or read a skill** — `.ok/skills/` is opaque internal state; skills are addressed by `name`+`scope`, not by path. Use the `skills` tool (omit `name` to list across Project + Global, pass `name` to read one); never browse `.ok/skills/...` paths.
+- **`ls` / `cat` / `find` on skill folders to discover or read a skill** — skills are addressed by `name`+`scope`, not by path (a skill can live in any editor dir, the `.agents/skills/` hub, or a custom root, with copies elsewhere). Use the `skills` tool.
 
-**MCP tool visibility — not seeing `exec` is NOT the escape hatch.** MCP wiring varies by client (Claude Code, Cursor, Codex, Windsurf, VS Code). Server labels are user-defined; tools may not appear as top-level symbols named `exec`. If OpenKnowledge is registered as an MCP server here, route markdown reads through its `exec` / `search` via your client's documented MCP invocation (including any generic "call MCP tool" flow). Registration is the test, not top-level-symbol visibility.
-
-**Your initial tool list is NOT exhaustive — run tool discovery before concluding the MCP is missing.** Some clients (notably Codex) defer MCP tools behind a lazy `tool_search` / tool-discovery step, so `mcp__open-knowledge__*` is absent from the upfront tool set and only appears after you search for it. Absence from the visible list means "I have not discovered it yet," NOT "it is not registered." If you don't see the tools, your first move is to run your client's tool-discovery / `tool_search` for `open-knowledge` (or `exec` / `search` / `write`) — do not infer unavailability from the initial list.
+**Not seeing `exec` is NOT the escape hatch.** Wiring, labels, and tool visibility vary by client; some (notably Codex) defer MCP tools behind lazy discovery. Registration is the test, not top-level-symbol visibility — run tool discovery for `open-knowledge` first. Detail: `references/setup.md`.
 
 **Escape hatch.** Native `Read` / `Grep` / `Glob` on `.md` / `.mdx` is allowed **only** when, after running tool discovery (above), no OpenKnowledge MCP server is registered for this project, **or** immediately after you actually invoked an MCP call and it failed — then begin a user-visible sentence with `OpenKnowledge MCP unavailable:`. "Not registered" is a conclusion you may only reach after tool discovery turned it up empty — never from the initial tool list alone. Never use the hatch because you skipped your client's MCP path, didn't see `exec` as a top-level tool, didn't run tool discovery, or rationalized the skill wasn't necessary.
 
@@ -67,19 +63,20 @@ Call `write` / `edit` as soon as you have content (route through MCP per the STO
 
 **Persist incrementally — the knowledge base IS your checkpoint (MUST).** On any multi-step or long-running task — a research sweep, a multi-source synthesis, a batch of docs — write completed work to the KB as you finish each unit: per section, per source, per doc. Never hold finished findings only in your context waiting for one final write at the end. A rate limit, crash, or context compaction mid-task discards everything still unwritten; work already persisted survives, and you resume by reading the doc back. Create the target doc early (skeleton + frontmatter), then `edit` each section in as it firms up.
 
-**Pass a `summary` on every content write (SHOULD)** — a one-line (≤80 chars) user-facing change-note that becomes the timeline entry. **Reach for visual structure** (Callout, `mermaid`, table, `html preview`) where it carries the point better than prose; call `palette` as you draft. Advisory write-warnings, MDX authoring, delete/move mechanics, keeping `==`/`$`/`%%`/`<!--` literal, and visual authoring: `references/writing.md` + `references/components-and-visuals.md` + `references/media-and-assets.md`.
+**Pass a `summary` on every content write (SHOULD)** — a one-line (≤80 char) user-facing note; it becomes the timeline entry. **Reach for visual structure** (Callout, `mermaid`, table, `html preview`) where it carries the point better than prose; call `palette` as you draft. Advisory write-warnings, MDX authoring, delete/move mechanics, and visual authoring: `references/writing.md` + `references/components-and-visuals.md` + `references/media-and-assets.md`.
 
 ## Grounding — every factual claim needs a source (MUST)
 
-Knowledge-base docs are factual artifacts. Every claim must be traceable, and **the source has to live inside the knowledge base**, not float on the public web.
+KB docs are factual artifacts: every claim traceable, and **the source lives inside the knowledge base**, not on the public web.
 
-- **The knowledge base is source-of-truth — closed loop.** External sources don't get cited out to the live web; they get pulled in via `ingest`, then cited locally. A bare `[source](https://...)` URL inside a KB doc is **not** a finished citation — it's a TODO that says "this source still needs to be ingested." The chain only works if every leaf is a local doc.
-- **Every factual claim MUST cite its source at the point of claim.** No unsourced speculation.
-- **Web sources for KB docs** → fetch the page (host `WebFetch` / `WebSearch`), then `ingest` it as a local doc, then cite the local path: `[source name](./path/to/source.md)` (the local doc carries the URL in `source_url:`). Inline `[source](URL)` is a chat affordance, not a KB one.
-- **Self-fetched counts.** When YOU fetched a URL to ground a claim landing in the KB, that fetch triggers `ingest` exactly like a user share — don't downgrade to inline-URL citation because the fetch was agent-initiated.
-- **Internal cross-refs** → standard markdown link to the OK doc with the authoritative claim; that doc must cite its own sources (chains terminate in preserved local docs).
-- **If you don't have evidence:** run a web search and `ingest` it, OR mark inline `(TODO: needs source)`, OR don't write the claim. Do NOT fabricate. Unsourced speculation rots into untraceable tribal knowledge.
+**Ingest is a procedure, not a tool** — binary-vs-text classification, SSRF-safe fetch flags, size + executable gates, wrapper frontmatter — in [`references/ingest-and-sources.md`](references/ingest-and-sources.md). Read it before your first capture; a naive fetch-and-paste skips every gate.
 
+- **Closed loop.** External sources are pulled in by the ingest procedure, then cited locally. A bare `[source](https://...)` inside a KB doc is **not** a citation — it is a TODO meaning "still needs ingesting". The chain only works if every leaf is a local doc.
+- **Every factual claim MUST cite its source at the point of claim.** No unsourced speculation.
+- **Web sources** → fetch the page (host `WebFetch` / `WebSearch`), ingest it, then cite the path: `[source name](./path/to/source.md)` (the local doc carries `source_url:`). Inline `[source](URL)` is a chat affordance, not a KB one.
+- **Self-fetched counts.** A URL YOU fetched to ground a claim gets the same ingest — no inline-URL downgrade.
+- **Internal cross-refs** → link the OK doc holding the authoritative claim; that doc cites its own sources (chains terminate in preserved local docs).
+- **No evidence?** Search and ingest the result, OR mark `(TODO: needs source)`, OR don't write the claim. Do NOT fabricate — unsourced speculation rots into untraceable tribal lore.
 ## Linking — standard markdown links (MUST)
 
 Link every noun-phrase that names another document — `[text](./relative/path.md)` — and link liberally. **Every link must resolve to a doc that exists by the time you're done** (a same-pass forward-reference you create later in the pass is fine; for one that genuinely won't exist, leave the mention as plain prose + a tracked task). Never backtick a link (`` `[text](./foo.md)` `` is a bug) and never use HTML `<a>`. **Read `brokenLinks` on every `write`/`edit` response: `[]` means all links resolve; a populated list names each broken `href` + `reason` (`no-such-doc` / `no-such-file` / `unresolvable`) — fix them in a follow-up `edit`.** `audit` is the authoritative end-state link check (the editor's red-underline is slug-tolerant and lies, so trust the tool). External web sources are NOT inline body links (see Grounding). Full rule set + the `[[Page]]` legacy note: `references/linking.md`.
@@ -88,7 +85,7 @@ Link every noun-phrase that names another document — `[text](./relative/path.m
 
 Every `.md` / `.mdx` needs YAML frontmatter — `title` + `description` required, `tags` recommended. Two **opt-in, nested** folder mechanisms: folder frontmatter (`<folder>/.ok/frontmatter.yml` — the folder's own open-shape properties; self-only, does NOT cascade into child docs) and templates (`<folder>/.ok/templates/` — what new docs start with). Most folders have NO `.ok/`. A doc's frontmatter is exactly its own on-disk YAML. Structural model + the full pre-write checklist: `references/folder-model.md`. Template authoring + folder editing: `references/template-authoring.md`. Frontmatter-vs-body edit rules: `references/doc-editing.md`.
 
-- **Read the folder before writing (MUST).** Before creating/editing docs in a folder, call `exec("ls -A <folder>")` once per folder per session — it returns the folder's `title`/`description`/`tags` + `templates_available`. Skipping it lands docs that violate folder discipline. (If a folder has no frontmatter AND no templates AND the repo has substantial content elsewhere, it isn't onboarded — invoke `workflow({ kind: 'discover' })` first.)
+- **Read the folder before writing (MUST).** Before creating/editing docs in a folder, call `exec("ls -A <folder>")` once per folder per session — it returns the folder's `title`/`description`/`tags` + `templates_available`. Skipping it lands docs that violate folder discipline. (If a folder has no frontmatter AND no templates AND the repo has substantial content elsewhere, it isn't onboarded — run `references/onboard-existing-repo.md` first.)
 - **Use a template when one fits (MUST).** Instantiate via `write({ document: { path, template } })`; inherited templates count. Skip only when none match or the user asked for free-form (note why in chat). Create templates proactively when a shape recurs.
 - **When recurring per-doc properties emerge (MUST).** Writing the same frontmatter on multiple siblings → bake those starting values into a template (`write({ template })`). Folder frontmatter does not cascade values into docs.
 
@@ -104,26 +101,28 @@ Projects with GitHub sync may carry docs in merge-conflict state; mutating calls
 | Explore a markdown-heavy dir | `Agent(Explore)` (bypasses OK) | `exec`/`search` yourself |
 | Reference another doc | `` `[text](./p.md)` `` (backticked) or HTML `<a>` | `[text](./p.md)` |
 | Embed an image | `<img>`, a `localhost`/`preview_url` URL, hot-link | save locally + `![meaningful alt](./path)` |
-| Factual claim in a KB doc | prose with no citation, OR inline `[src](https://…)` | `ingest` the source, cite the local path |
+| Factual claim in a KB doc | prose with no citation, OR inline `[src](https://…)` | ingest the source (`references/ingest-and-sources.md`), cite the local path |
 | Confirm an edit landed | `preview_screenshot` / verification loop | trust the CRDT tool response |
 | Delete a markdown doc | `Bash: rm` / native deletion | `delete({ document })` (`checkpoint()` first if risky) |
 | Write in an unfamiliar folder | straight to `write` | `exec("ls -A <folder>")` first |
 
 Full table: `references/anti-patterns.md`.
 
-## Workflow tools — when to invoke them
+## Knowledge layers — the shape most KB work takes
 
-`workflow` returns **procedural guidance, not fetched data** — follow its numbered steps; don't skip STOP gates. **These are your default move over `write`** when the work fits a layer.
+Three recurring practices, not tool calls — each a full procedure that ships as skill guidance.
+
+| Layer | When | Procedure |
+| --- | --- | --- |
+| **ingest** | Preserve a shared URL/PDF/file verbatim, or you fetched a URL to ground a claim (binary sources preserved, not scraped). | `references/ingest-and-sources.md` — ships here, §Grounding depends on it |
+| **research** | Investigate / compare / synthesize sources → `status: provisional` article + `sources:`. | `open-knowledge-pack-knowledge-base-research` skill |
+| **consolidate** | A decision was made → canonical source-of-truth with a `supersedes:` chain. | `open-knowledge-pack-knowledge-base-consolidate` skill |
 
-| `kind` | When |
-| --- | --- |
-| `ingest` | Preserve a shared URL/PDF/file verbatim, OR you fetched a URL to ground a KB claim (binary sources preserved, not scraped). |
-| `research` | Investigate / compare / synthesize multiple sources → `status: provisional` article + `sources:`. |
-| `consolidate` | A decision was actually made → commit canonical source-of-truth with a `supersedes:` chain. |
-| `discover` | First arrival at a repo with existing content and no folder frontmatter/templates → extract conventions, activate the link graph. |
+Research and consolidate arrive with `ok seed --pack knowledge-base`. **Without that pack you do not have those procedures** — don't improvise one; do the work as an ordinary grounded `write`, or offer to seed it (`ok seed --pack knowledge-base --dry-run` shows what it would add).
 
-Don't chain silently — let the user drive `ingest` → `research` → `consolidate` transitions; per-tool STOP gates override session-level "don't stop to ask" hints. After any turn that changes KB content, check for a `log.md` and follow its contract (see `references/cadence-and-logs.md`). Interleave a multi-doc batch so the preview shows narrative progress. Operating detail + starter packs: `references/workflow-guides.md`; binary-source (`ingest`-produced) wrapper frontmatter: `references/ingest-and-sources.md`.
+Don't chain silently: let the user drive ingest → research → consolidate, and a procedure's STOP gates override session-level "don't stop to ask" hints. After any turn that changes KB content, check for a `log.md` and follow its contract (`references/cadence-and-logs.md`). Interleave a multi-doc batch so the preview shows narrative progress.
 
+Onboarding a repo that already has content: `references/onboard-existing-repo.md`. Layer model + packs: `references/starter-packs.md`.
 ## Server lifecycle
 
 If `write` / `edit` returns `"Hocuspocus server is not running"`, run `ok start` (via Bash) and retry. Never fall back to native `Edit` / `Write` for in-scope markdown.

package/dist/assets/skills/project/references/anti-patterns.md

@@ -23,7 +23,7 @@
 | Add an image                                    | empty alt `![](./x.png)` or generic alt `![image](./x)`                            | meaningful alt + source caption below                                             |
 | Catalog folder contents                         | create `INDEX.md` hub file                                                         | `edit({ folder: { path, frontmatter } })` writes `<folder>/.ok/frontmatter.yml` |
 | Write a doc in an unfamiliar folder             | go straight to `write` with hand-authored markdown                        | `exec("ls -A <folder>")` first — read the folder description + `templates_available` before writing |
-| Land in an existing repo without orienting      | go straight to `write` when no folder frontmatter / templates exist       | invoke `workflow({ kind: 'discover' })` once for the project — extracts conventions from siblings, sets folder frontmatter + templates, activates the link graph |
+| Land in an existing repo without orienting      | go straight to `write` when no folder frontmatter / templates exist       | run `onboard-existing-repo.md` once for the project — extracts conventions from siblings, sets folder frontmatter + templates, activates the link graph |
 | Author a doc when a matching template exists    | `write({ document: { path, content: "..." } })` from scratch                                 | `write({ document: { path, template } })` — templates carry the folder's frontmatter + body discipline |
 | Change a doc's title / tags                     | `edit({ document: { path, find, replace } })` to swap the YAML (rejected — HTTP 400 frontmatter-intersect) | `edit({ document: { path, frontmatter } })` for metadata; `write({ document: { path, content, frontmatter, position: "replace" } })` for full rewrites |
 | Repeat the same frontmatter on sibling docs     | hand-set identical `tags` / `title` prefix on every new file                       | `write({ template })` once — new docs start from the template |

package/dist/assets/skills/project/references/corpus-qa.md

@@ -1,6 +1,6 @@
 # Answering direct questions from the corpus
 
-A direct question you can answer from existing documents — "which customers have non-standard indemnity?", "can we use Alloy's logo?", "what did we decide about X?" — does **not** need the words "research" or "report" to route here. Retrieve with `search` / `exec`, read the relevant docs, and **answer in chat with inline citations to the source docs you used**. That is the complete, correct default — most questions end here. This is NOT `workflow({ kind: 'research' })`: research gathers and synthesizes *external* sources behind a scoping gate; a corpus question just reads what the knowledge base already holds. (Inside an active `workflow({ kind: 'research' })` session, research's own "file valuable Q&A back" step governs how answers are persisted — not this section.)
+A direct question you can answer from existing documents — "which customers have non-standard indemnity?", "can we use Alloy's logo?", "what did we decide about X?" — does **not** need the words "research" or "report" to route here. Retrieve with `search` / `exec`, read the relevant docs, and **answer in chat with inline citations to the source docs you used**. That is the complete, correct default — most questions end here. This is NOT the **research** layer: research gathers and synthesizes *external* sources behind a scoping gate; a corpus question just reads what the knowledge base already holds. (Inside an active research pass, research's own "file valuable Q&A back" step governs how answers are persisted — not this section.)
 
 **Offer to persist the answer only when it is durable knowledge the KB is currently missing** — when ALL of these hold:
 

package/dist/assets/skills/project/references/folder-model.md

@@ -33,7 +33,7 @@ A doc's frontmatter is exactly its own on-disk YAML — folder frontmatter never
 
 Before creating or editing docs in a folder, **always** call `exec("ls -A <folder>")` once. The response carries the folder's own `title`/`description`/`tags` + `templates_available` (the template menu for `write({ document: { template } })`). Skipping this is how agents land docs that violate folder discipline.
 
-0. **First-contact check.** If the folder has no frontmatter of its own AND `templates_available` is empty AND `exec("ls -A")` shows substantial content elsewhere, the project hasn't been onboarded — STOP and invoke `workflow({ kind: 'discover' })`. Skip on subsequent writes once confirmed.
+0. **First-contact check.** If the folder has no frontmatter of its own AND `templates_available` is empty AND `exec("ls -A")` shows substantial content elsewhere, the project hasn't been onboarded — STOP and run `onboard-existing-repo.md`. Skip on subsequent writes once confirmed.
 1. **Read the folder's description** — its `title`/`description`/`tags` tell you what the folder is for. (These describe the folder; they are NOT defaults the doc inherits.)
 2. **Read `templates_available`** — each entry has `name`, `title`, `description`, `scope` (`local` / `inherited`). If one matches, **prefer it** over free-form markdown (it's the folder's contract — templates carry frontmatter + body structure hand-authored docs routinely miss).
 3. **Read recent siblings** — new docs should match the shape of existing ones (filename, frontmatter, body structure).

package/dist/assets/skills/project/references/ingest-and-sources.md

@@ -1,24 +1,241 @@
-# Binary-source wrappers (`ingest`-produced)
+# Ingest — capture an external source as raw reference material
 
-Docs that wrap a co-located binary file under `external-sources/` carry extra frontmatter so the wrapper-binary pair is fully described:
+Capture an external source into the knowledge base as raw reference material. The KB is **closed-loop**: external sources are pulled IN so downstream docs cite local paths, never bare web URLs. This applies whether a user shared the source OR you fetched it yourself to ground a claim — agent-initiated fetches are not exempt (core §Grounding). **Raw preservation only** — no summary, no analysis, no interpretation.
+
+**Bias toward binary preservation.** When the source is a binary file (PDF, image, audio, video, Office doc, archive, dataset), preserve the raw bytes — do NOT settle for a text scrape. OK ships a complete asset-embed surface (`![[file.ext]]` wiki-embed, file-watcher pickup, sha256 dedup, blocked-extension enforcement); this procedure bridges the source to it via your shell tool.
+
+Sources land under `external-sources/` when the project has that folder (the `knowledge-base` starter pack's raw layer); otherwise pick the project's equivalent and stay consistent. Paths below are relative to the resolved `content.dir` (`config({ key: 'content.dir' })`).
+
+## Step 0: Is this source worth preserving?
+
+Before fetching anything, sanity-check:
+
+- **Is it in scope?** If the source is unrelated to what this knowledge base is accumulating, ingest is pollution. Check the existing layout: `exec("ls -A <content-dir>")`.
+- **Is it already ingested?** `exec("grep -rln <source-url-or-title-slug> <content-dir>")` — if the same source is already present with current content, stop and reuse. Re-ingest is appropriate when the source changed materially — see Step 5 for sha256-mismatch semantics.
+- **Is the intent actually preservation or analysis?** If the user wants findings synthesized rather than raw bytes archived, redirect to a research pass (which captures sources as needed). Don't pre-ingest speculatively.
+
+If all three checks pass, proceed.
+
+### Shell-capability self-detection (evaluate before any shell call)
+
+If your host does not expose a shell-like tool (no Bash / Terminal / shell-exec / equivalent), you cannot run `curl` for either the HEAD probe (Step 1a) or the binary download (Step 1b). Detect once at the start via a trivial probe (`echo ok` via the host's shell affordance, or a known-safe `which curl` invocation). On absence of a shell tool, route the source straight to **Step 1c (text fetch / shell-less fallback)** — skipping Step 1a's HEAD-tiebreaker classification and Step 1b's binary download entirely — and surface the degradation explicitly to the user.
+
+## Step 1a: Detect source kind (binary vs text)
+
+Classify the source before fetching. **`.svg` is intentionally absent from the binary-extension list — SVG can contain scripts and is treated as a scripted-document extension; see Step 1b's executable hard-block.**
+
+- **URL with a binary file extension** (`.pdf`, `.png`, `.jpg`, `.jpeg`, `.gif`, `.webp`, `.mp4`, `.webm`, `.mov`, `.m4v`, `.mp3`, `.wav`, `.ogg`, `.m4a`, `.flac`, `.docx`, `.xlsx`, `.pptx`, `.doc`, `.xls`, `.ppt`, `.zip`, `.7z`, `.tar`, `.gz`, `.rar`, `.csv`, `.tsv`, `.epub`, etc.) → treat as **binary**. Skip to Step 1b.
+- **URL with a clear HTML/text extension** (`.html`, `.htm`, none, `.txt`, `.md`) → treat as **text**. Skip to Step 1c. The executable hard-block does NOT apply here because Step 1c never writes the source extension to disk — it writes a `.md` wrapper containing the extracted text.
+- **Local file path** → use your native file read tool. If the local file is text (HTML, plain, markdown), skip to **Step 2b** (text wrapper). If it is a binary you want preserved verbatim, you already have the bytes on disk — skip Step 1b's download and go to **Step 2a** (binary wrapper), pointing `source_path:` at the existing local path (relative to the wrapper).
+- **Ambiguous URL** (no extension, query-string download URL, redirect-y link) → run `curl -IL --proto =http,=https --proto-redir =http,=https --max-redirs 5 -A 'Mozilla/5.0' <url>` to read response headers, then classify by `Content-Type`:
+  - `application/pdf`, `image/*` (except `image/svg+xml`), `video/*`, `audio/*`, `application/zip`, `application/vnd.openxmlformats-*`, `application/epub+zip` → **binary**
+  - `text/html`, `text/plain`, `application/json` (when the source is a doc/article, not data) → **text**
+  - `application/octet-stream` → ambiguous; treat as binary BUT note in chat that the server didn't declare a specific type, and the captured `sha256` + bytes record gives downstream tooling the signal if the bytes turn out to be HTML.
+
+### STOP gates evaluated during Step 1a (before any download)
+
+1. **Streaming-video / DRM-protected media.** If the URL hostname matches a known streaming pattern (examples — illustrative, not exhaustive: `youtube.com`, `youtu.be`, `vimeo.com`, `twitch.tv`, `tiktok.com`, `spotify.com`, `open.spotify.com`, `podcasts.apple.com`) **OR** the HEAD response shows `Content-Type: text/html` on a URL that looked like media **OR** the HEAD shows anti-bot characteristics (`Server: cloudflare` + `cf-mitigated` / `cf-chl-bypass` headers, an Akamai/Imperva challenge response, 403 with an HTML body) → **STOP**. Tell the user: "Streaming services and DRM-protected media require `yt-dlp` or vendor-specific tooling — out of scope. Paste the transcript, or run `yt-dlp` manually and re-share the resulting file as a local path."
+
+2. **URL scheme — HARD BLOCK.** If the URL scheme is not `http://` or `https://` (i.e., `file://`, `gopher://`, `ftp://`, etc.) → **HARD STOP**. Tell the user: "Ingest only fetches `http(s)` URLs. Other schemes (local-file paths, intranet protocols) bypass the size + redirect safeguards and risk pulling cloud-metadata or local-service responses into the KB. If you have a local file, pass its path directly as a local-file source instead of a URL."
+
+3. **Auth-walled / paywalled / 4xx.** If HEAD returns 401/402/403/407/429 → STOP. Ask the user to paste the content or share a local copy.
+
+## Step 1b: Binary fetch
+
+You've classified the source as binary and Step 1a's STOP gates passed. Step 1b adds **write-path-specific** STOP gates that only apply when we're about to land bytes on disk under the source extension:
+
+### Step 1b STOP gates (write-path only)
+
+1. **Executable / scripted-document extension — HARD BLOCK.** If the URL extension is one OK's upload blocklist covers — Windows executables, POSIX shells, scripted documents (including `.svg` and `.xml`), macOS installer classes, URL-files, cross-platform packages, and Windows shortcut classes — **HARD STOP**. Refuse to download. Tell the user: "Executable / scripted-document extensions are not auto-fetched — the runtime backstop is desktop-only, so plan-level enforcement is the only line of defense in CLI / web contexts. If you genuinely want to archive this file, `curl` it manually outside the agent and reference the local path." This gate fires HERE (write path) and NOT against Step 1c — Step 1c writes a `.md` wrapper with extracted text, not the source extension on disk.
+
+2. **Size pre-check.** Run `curl -IL --max-redirs 5 -A 'Mozilla/5.0' '<SOURCE_URL>'` (if not already done in Step 1a's ambiguous-URL branch) and read `Content-Length`:
+   - greater than 100 MB → **HARD STOP**. Tell the user the source exceeds GitHub's 100 MB file-size cap and offer the manual-`curl` + Git-LFS escape path.
+   - greater than 50 MB → STOP and ask explicitly: "Source is X MB (above the 50 MB GitHub warn threshold). Proceed?"
+   - absent → proceed; the `curl --max-filesize` flag is the enforced backstop. Note in chat that the size could not be pre-verified.
+   - 50 MB or under → proceed.
+
+### Slug derivation (strict shape)
+
+Pick a kebab-case slug from the source's filename or title (e.g., `karpathy-2024-llm-os`). The slug becomes both the binary basename and the wrapper basename. Don't put dates in the slug — dates go in frontmatter (and in dated-sibling slugs on re-ingest per Step 5).
+
+**The slug MUST match `^[a-z0-9][a-z0-9-]{0,99}$` — kebab-case, ASCII letters / digits / single hyphens only. NO dots, slashes, leading hyphens, or path segments. If you derive the slug from a server-controlled value (`Content-Disposition: filename=`, URL path basename), strip non-conforming characters before using it.** A server returning `Content-Disposition: attachment; filename="../../etc/passwd"` is adversarial; the slug constraint prevents path traversal from a malicious source landing bytes outside the sources folder. If the stripped result is empty, ask the user for a slug.
+
+### Download
+
+Use your shell tool. Create the destination first if it doesn't exist:
+
+```bash
+mkdir -p "<content-dir>/external-sources"
+curl -L --fail \
+  --proto =http,=https --proto-redir =http,=https \
+  --max-redirs 5 \
+  --max-time 60 \
+  --max-filesize 104857600 \
+  -o "<content-dir>/external-sources/<slug>.<ext>" \
+  -A 'Mozilla/5.0' \
+  '<SOURCE_URL>'
+```
+
+**Shell-escape any special characters in the URL (single quotes, backticks, dollar signs). Do NOT paste a URL containing a literal single quote directly into the single-quoted curl argument — break it into a shell-safe form first.**
+
+Flag rationale:
+- `-L` follows redirects (CDN-fronted sources need this).
+- `--proto =http,=https --proto-redir =http,=https` refuses any scheme other than http(s), and refuses redirects that would downgrade to another scheme. Prevents redirect-chain SSRF into `file://`, `gopher://`, or cloud-metadata endpoints reached via `--max-redirs`.
+- `--fail` exits non-zero on HTTP 4xx/5xx instead of writing an error body to disk.
+- `--max-redirs 5` caps redirect chains (exit 47 on excess).
+- `--max-time 60` caps total request time (exit 28).
+- `--max-filesize 104857600` enforces the 100 MB hard cap as defense-in-depth in case the server omits or lies about `Content-Length` (exit 63). The option has shipped since curl 7.10.8 (2003) — every modern install has it.
+
+If curl exits non-zero:
+- 47 (too many redirects) → STOP. Likely a tracking-link or auth-flow URL. Ask the user to paste.
+- 28 (timeout) → STOP. Slow source. Ask the user to retry or share locally.
+- 63 (`--max-filesize` triggered) → `rm -f` the truncated output. STOP and tell the user the server's `Content-Length` was missing or wrong.
+- 22 (HTTP error returned by `--fail`) → STOP. Auth wall, anti-scraping, or vanished URL. Ask the user to paste.
+- Other (e.g., 1 = unsupported protocol from `--proto`) → STOP and report the curl exit code to the user.
+
+After a successful download, record the byte size and sha256:
+
+```bash
+ls -l "<content-dir>/external-sources/<slug>.<ext>"      # bytes
+sha256sum "<content-dir>/external-sources/<slug>.<ext>"  # or 'shasum -a 256' on macOS
+```
+
+OK's file-watcher picks up the new file and emits `asset-create`. The sidebar refreshes; the file is now indexed.
+
+## Step 1c: Text fetch (HTML / article sources, or shell-less host fallback)
+
+Either the source is genuinely text (HTML article, plain text doc) OR your host is shell-less and binary preservation is unavailable.
+
+Use your available web fetch tool (`WebFetch` or your host's equivalent). If the fetcher returns an obvious *summary* of the page instead of the raw content (some LLM-backed fetch tools do this), note it and try a raw alternative (`curl -sL <url>` if your host has a shell, or ask the user to paste). The goal is verbatim bytes.
+
+If the fetch fails (login wall, 401/402/403/429, anti-scraping block), **stop and ask the user to paste the content directly**. Do not save a stub, an error page, or a login wall as "raw content" — that poisons the knowledge base.
+
+## Step 2a: Save the binary wrapper (only after Step 1b succeeded)
+
+Write a markdown wrapper at `external-sources/<slug>.md`. **Declare the full tag list explicitly — do NOT assume a folder-level cascade exists** (folder frontmatter is self-only; it does not flow into child docs — see `references/folder-model.md`).
 
 ```yaml
 ---
-title: ...
-description: ...
-source_url: https://example.com/file.pdf
-source_path: ./<slug>.<ext>      # relative to this wrapper
-media_type: application/pdf
-bytes: 1234567
-sha256: <64-char hex>            # of the embedded binary
+title: Original title of the source (from <title>, Content-Disposition filename, or your best read)
+description: One-line summary from the source (their words, not yours)
+source_url: https://example.com/path/to/file.pdf
+source_path: ./<slug>.<ext>    # relative to this wrapper
+media_type: application/pdf    # RFC 6838 type/subtype, from HEAD Content-Type
+bytes: 1234567                 # integer, from ls -l or stat
+sha256: <hex sha256 digest>    # full 64-char hex, of the embedded binary
 date_fetched: YYYY-MM-DD
-preservation: binary             # OR: text-only / text-extracted
-supersedes:                      # OPTIONAL — dated-sibling re-ingest
+author: Original author if known
+preservation: binary
+supersedes:                    # OPTIONAL — dated-sibling re-ingest (Step 5)
   - <prior-slug>.md
-tags: [source, immutable, layer-ingest, binary]
+tags:
+  - source
+  - immutable
+  - layer-ingest
+  - binary
 ---
 
 ![[<slug>.<ext>]]
 ```
 
-Body is just the wiki-embed. PDFs/opaque attachments render as a click-dispatching File row; `<Pdf src="./<slug>.pdf" />` is the opt-in inline viewer. See `ingest`'s tool body for full re-ingest / size / executable rules.
+The body is just the wiki-embed reference. For images, video, and audio, the embed renders inline. **For PDFs and other opaque file-attachment types (docx, xlsx, zip, etc.), the `![[file.ext]]` form renders as a Notion-style File row that click-dispatches to the appropriate viewer** — the pdfjs canvas viewer is opt-in via the explicit `<Pdf src="./<slug>.pdf" />` JSX form, NOT the wiki-embed default. If the user wants the inline canvas viewer for this specific PDF, they can post-edit the wrapper body to swap the embed for that form.
+
+Write via `write` (NOT native `Write` — the CRDT path is mandatory for in-scope markdown).
+
+## Step 2b: Save the text wrapper (only after Step 1c — text path or shell-less fallback)
+
+Write a markdown wrapper at `external-sources/<slug>.md` with the text content preserved verbatim in the body. Strip obvious boilerplate (nav menus, cookie banners, ads, footer links, "related articles" widgets) but **do not summarize, paraphrase, or interpret**.
+
+Frontmatter shape:
+
+```yaml
+---
+title: Original title of the source
+description: One-line summary from the source (their words, not yours)
+source_url: https://example.com/article
+media_type: text/html
+date_fetched: YYYY-MM-DD
+author: Original author if known
+preservation: text-extracted    # OR: text-only (a shell-less fallback for a binary source)
+# NOTE: no `source_path` — text wrappers ARE the content. `source_path` is meaningful only for
+# binary wrappers (Step 2a), where it points at the co-located binary sibling.
+tags:
+  - source
+  - immutable
+  - layer-ingest
+  - text
+---
+```
+
+**If this is a shell-less fallback for a source that should have been binary** (you detected binary in Step 1a but your host couldn't `curl`), set `preservation: text-only` AND prepend a top-of-body admonition so a future agent (or you, on a different host) can detect and upgrade:
+
+```markdown
+> Binary not preserved — this is an extracted-text snapshot of a binary source.
+> Re-run ingest from a shell-capable client to capture the original file.
+
+(...verbatim extracted text follows...)
+```
+
+Downstream tooling can grep `preservation: text-only` in frontmatter to find docs that need upgrading.
+
+## Step 3: Preserve the content faithfully
+
+For binary wrappers: the wrapper body is the wiki-embed reference; no further body content is needed. All metadata lives in frontmatter.
+
+For text wrappers:
+- **Keep** headings, lists, quotes, code blocks, images, citations, references.
+- **Strip** obvious boilerplate: nav menus, cookie banners, ads, footer links, "related articles" widgets.
+- **Do NOT** summarize, critique, paraphrase, or interpret.
+- **For very long sources**, consider splitting by major section with cross-references in frontmatter.
+
+## Step 4: Verify
+
+- File(s) exist under the sources folder (binary + wrapper, or wrapper alone for text).
+- Valid frontmatter (at minimum `title`, `description`, `source_url`, `preservation`, and the full `tags` list — plus `source_path` for binary wrappers).
+- For binary preservation: `sha256` and `bytes` recorded; `media_type` matches what the HEAD response returned.
+- `exec("ls -A <content-dir>/external-sources/")` lists the new file(s) with enrichment.
+
+## Step 5: Re-ingest semantics (sha256 mismatch on a previously-ingested source)
+
+If `external-sources/<slug>.<ext>` already exists when you reach Step 1b:
+
+1. Compute its sha256.
+2. If the new sha matches the existing sha → **STOP** (no-op). Tell the user: "Already at `external-sources/<slug>.<ext>` — sha256 matches, no change."
+3. If the new sha differs → save the new bytes as `external-sources/<slug>.YYYY-MM-DD.<ext>` (today's date in the slug) AND write a **new** wrapper `external-sources/<slug>.YYYY-MM-DD.md` whose `supersedes:` is a YAML list naming the predecessor:
+   ```yaml
+   supersedes:
+     - <slug>.md
+   ```
+   **Do not mutate the old wrapper or the old binary** — the sources layer is append-only by convention.
+
+The latest-dated wrapper is the "current view"; older wrappers remain valid as historical snapshots.
+
+## Step 6: Discuss takeaways with the user (no file write)
+
+After preservation, briefly surface back to the user what the source actually contains — in **chat**, not in the raw file. The raw file stays verbatim; the human collaborator gets a quick orientation.
+
+- 3-5 bullet points capturing the source's main claims, with no editorializing.
+- Note any **tensions** with existing knowledge-base docs you surfaced in Step 0 — agents that ingest in isolation miss the "wait, this contradicts `[prior article](./path/to/prior.md)`" signal.
+- Offer next steps: "Shall I research this topic now, or is preservation enough?" Don't silently chain into a research pass — the user may have just wanted the archive.
+- For binary preservation: include the one-line breadcrumb (`saved external-sources/<slug>.<ext> — <bytes> bytes, sha256 <abbrev>`).
+- For a shell-less text fallback of a binary source: explicitly tell the user the binary was not preserved and recommend re-running from a shell-capable client.
+
+## Step 7 (optional): Update neighbor docs to link the new source
+
+If the source is directly relevant to an existing article or research doc, update that doc to link the new raw source. A preserved source that no doc points at is an island. Limit this to 1-3 high-signal neighbors.
+
+- Follow the `write` / `edit` contract from the core skill.
+- For binary wrappers, prefer linking to the wrapper (the markdown doc), not the binary directly. The wrapper is the closed-loop citation target; the binary is the embedded asset within it.
+- Do NOT mass-update every neighbor. Focused cross-linking is rewarded; noisy neighbor-pings degrade the signal.
+
+## Non-goals
+
+- **No analysis** — don't interpret, compare, or critique the source. That is the research layer's job.
+- **No promotion to a canonical article** — that is the consolidate layer's job, later.
+- **No silent chaining into research** — ingest completes on its own; the user explicitly opts into a research pass.
+- **No synthesis inside the raw file** — takeaways live in chat or a separate summary doc, never mixed into the preserved source.
+- **No OCR / transcription** — binary preservation only.
+- **No streaming-video direct download** — `yt-dlp` is a separate workflow; STOP and route to it (Step 1a STOP gate 1).
+- **No executable / scripted-document auto-fetch** — hard-blocked at Step 1b STOP gate 1 (write-path only). The user runs `curl` manually outside the agent if they need to archive such a file.
+
+This procedure feeds a three-layer pipeline (ingest → research → consolidate). Ingest is this file, and ships with every `ok init`; research and consolidate are two member skills of the `knowledge-base` starter pack, available only once it is seeded. See `references/starter-packs.md`.