@aitne-sh/aitne 0.1.8 → 0.1.9

package/agent-assets/docs/features/wiki/commands.md

@@ -34,7 +34,7 @@ ask_examples:
   - How do I bridge two domains with `!connect`?
 locale: en-US
 created: 2026-05-12
-updated: 2026-05-12
+updated: 2026-05-28
 keywords:
   - !ingest
   - !compile
@@ -52,18 +52,24 @@ related:
   - guides/maintain-wiki-health
   - guides/explore-with-trace-and-connect
   - troubleshooting/wiki-ingest-full-blocked
+process_keys:
+  - wiki.ingest_url
+  - wiki.compile
+  - wiki.ask
+  - wiki.lint
+  - wiki.trace
+  - wiki.connect
 ui_anchors:
   - /wiki
   - /wiki/timeline
   - /settings/wiki
-  - /approvals
 ---
 
 # Wiki Commands
 
-Use these from a paired DM channel after enabling the wiki — open
-**Setup → Settings → Wiki** to enable, then browse from **My Life →
-Wiki**.
+Use these from a paired DM channel after enabling the wiki. Open
+**Settings → Wiki** (`/settings/wiki`) to enable a workspace, then
+browse the result from **My Life → Wiki** (`/wiki`).
 
 | Command | Effect |
 |---|---|
@@ -125,29 +131,32 @@ Settings → Wiki:
 
 - **Parallel** (default): all URLs fan out simultaneously up to the
   per-URL concurrency cap. Fastest; small risk of bursting rate
-  limits at the URL host.
+  limits at the URL host. Each URL replies on its own completion.
 - **Serial**: URLs are enqueued in submitted order; each agent
   session starts only after the previous one completes. Slower but
-  predictable budget and rate.
+  predictable budget and rate. You get a single consolidated reply
+  when the whole batch finishes.
 
-The acknowledgement DM tells you which mode ran
-(`in parallel` / `serially`).
+The acknowledgement DM names the mode that ran (`in parallel` /
+`serially`).
 
 ## `!compile full` — the Cost Gate
 
 Full rebuilds touch every wiki note and are the most expensive
 command in the wiki surface. The flow:
 
-1. The bang handler estimates the cost (raw count × assumed input
-   tokens × Sonnet unit cost, bracketed at 0.5× / 1× / 2×).
+1. The bang handler estimates the cost (raw note count × assumed
+   input tokens × Sonnet 4.6 input price, bracketed optimistic 0.5× /
+   expected 1× / pessimistic 2×).
 2. On an external git-tracked vault with **Auto-commit before
    `!compile full`** enabled and a clean working tree, Aitne runs
    `git add -A && git commit -m "aitne wiki: pre-compile snapshot <ts>"`
    before continuing. A dirty tree refuses the operation — commit or
    stash first.
-3. If the pessimistic estimate exceeds your per-workspace approval
-   threshold (default $2.00), the run is queued under **Approvals**
-   in the dashboard. Approve from `/approvals` to start the compile.
+3. If the pessimistic estimate (expected spend × 2) exceeds your
+   per-workspace approval threshold (default $2.00), the run is queued
+   for approval. Open **Settings → Wiki** (`/settings/wiki`) and
+   confirm it in the Approvals queue there to start the compile.
 4. Otherwise, the run starts autonomously and you receive a
    completion DM with actual spend.
 
@@ -225,9 +234,10 @@ finding is itself useful.
 
 When no active wiki workspace exists, every `wiki.*`-routed bang
 command (`!ingest`, `!compile`, `!ask`, `!lint`, `!trace`, `!connect`)
-replies with:
+replies with a message like:
 
-> Wiki is not enabled. Open `/settings/wiki` to enable.
+> Wiki is not enabled. Open `/settings/wiki` to enable the internal
+> workspace.
 
 `!wiki help` is exempt — it returns the command list regardless of
 enablement so you can discover the surface before opting in.

package/agent-assets/docs/features/wiki/cost-and-approval.md

@@ -0,0 +1,240 @@
+---
+schema_version: 1
+slug: features/wiki/cost-and-approval
+title: Wiki Cost Estimation and Approval Flow
+id: wiki-cost-and-approval
+aliases:
+  - wiki cost
+  - wiki approval
+  - !compile full approval
+  - wiki cost gate
+  - wiki cost estimate
+  - wiki budget
+  - wiki precompile snapshot
+  - git pre-compile
+category: features
+summary: |
+  How Aitne predicts the cost of a `!compile full` run, when the cost
+  gate escalates to the dashboard approval queue, and how the
+  pre-compile git snapshot interacts with both. Also covers the
+  `--preview` dry-run and how the same estimator powers the
+  per-workspace cost banner in `/settings/wiki`.
+section: wiki
+tags:
+  - wiki
+  - cost
+  - approval
+  - safety
+  - core
+status: stable
+ask_examples:
+  - How does !compile full estimate cost?
+  - When does !compile full need approval?
+  - How do I change the wiki approval threshold?
+  - What happens if my Obsidian vault is dirty before !compile full?
+  - Does !compile or !ingest need approval?
+  - How accurate is the wiki cost estimate?
+  - What is the pre-compile git snapshot?
+  - Why is my !compile full sitting in /approvals?
+  - Can I see the cost before running !compile full?
+locale: en-US
+created: 2026-05-21
+updated: 2026-05-28
+keywords:
+  - cost estimate
+  - cost bracket
+  - 0.5x
+  - 2x
+  - bracketed cost
+  - approval threshold
+  - !compile full
+  - approval queue
+  - /approvals
+  - dashboard approval
+  - git pre-compile
+  - pre-compile snapshot
+  - dirty working tree
+  - cost banner
+  - compile preview
+  - --preview
+  - --dry-run
+  - char to token
+  - token approximation
+  - CJK token ratio
+related:
+  - features/wiki/overview
+  - features/wiki/commands
+  - features/wiki/workspaces
+  - features/operations/approvals
+  - concepts/costs-and-quotas
+  - guides/budget-and-cost-for-wiki
+  - troubleshooting/wiki-ingest-full-blocked
+ui_anchors:
+  - /settings/wiki
+  - /
+api_endpoints:
+  - /api/wiki/:workspace/estimate
+  - /api/wiki/:workspace/compile/preview
+  - /api/wiki/:workspace/git/status
+  - /api/approvals
+process_keys:
+  - wiki.compile
+config_keys:
+  - full_compile_approval_threshold_usd
+---
+
+# Wiki Cost Estimation and Approval Flow
+
+`!compile full` is the most expensive command in the wiki surface
+because it touches every raw note in the workspace and rewrites the
+canonical wiki pages from scratch. Aitne predicts the cost before
+the run starts and gates the actual execution behind an approval
+threshold you control per workspace.
+
+The same estimator backs four surfaces so the numbers cannot drift:
+
+- `GET /api/wiki/:workspace/estimate` — the dashboard banner in
+  `/settings/wiki` reads from here so you can see what the next
+  `!compile full` will cost before running it.
+- `GET /api/wiki/:workspace/compile/preview` — the `--preview`
+  dry-run.
+- The `!compile` / `!compile full` bang handler — uses the same
+  estimate to decide whether the run starts autonomously or
+  escalates to approvals.
+- The DM acknowledgement message — quotes the bracketed estimate
+  back to the user.
+
+## How the estimator works
+
+Pure JS, no agent session, no SDK call. The estimator
+(`packages/daemon/src/core/wiki/cost-estimate.ts`) opens each raw
+note under `10_raw/`, approximates its token count from on-disk
+content, then multiplies the total by the tier unit cost and
+brackets with 0.5×/2× multipliers.
+
+### Character → token approximation
+
+- **Latin / English content** — ~4 chars per token. The well-known
+  OpenAI rule-of-thumb; matches Anthropic's tokenizer within ±15%
+  for prose; confirmed against the gpt-tokenizer dist.
+- **CJK content** — ~1.5 chars per token. BPE merges short CJK runs
+  but not as aggressively as Latin word fragments.
+
+The classifier counts Unicode code points whose script is one of
+Han, Hiragana, Katakana, Hangul, Bopomofo. If the document is
+majority-CJK we apply the CJK divisor to the entire file; otherwise
+Latin. A per-script split per file would be marginally more accurate
+but adds 30% code for a sub-percent gain on typical mixed-script
+files.
+
+### The 0.5×/2× bracket
+
+The estimator never returns a single number. It always returns three:
+
+| Field | Meaning |
+|---|---|
+| `expectedUsd` | The point estimate: total approximated tokens × tier input cost. |
+| `optimisticUsd` | 0.5× `expectedUsd` — the "everything compresses well, the LLM exits early" scenario. |
+| `pessimisticUsd` | 2× `expectedUsd` — the "the LLM rewrites every page from scratch" scenario. **The approval gate compares this against your threshold.** |
+
+The bracket is wide on purpose. `!compile` is an LLM and may merge
+or skip pages inside the agent loop in ways the estimator cannot
+predict from on-disk content alone — but the pessimistic bound will
+not undercount the actual touch set.
+
+(History: P2 originally shipped a flat `rawCount × 1500` heuristic.
+P4.C upgraded to the per-file char→token approximation above because
+the flat heuristic under-counted on long ingested articles and
+over-counted on one-line stubs.)
+
+## The cost gate
+
+The `!compile full` flow:
+
+1. **Estimate** — bang handler calls the estimator.
+2. **Pre-compile snapshot** — on an external git-tracked vault with
+   `git_pre_compile_enabled = 1` (the default) and a clean working
+   tree, Aitne runs `git add -A && git commit -m "aitne wiki:
+   pre-compile snapshot <ts>"` before continuing. A dirty tree
+   refuses the operation — commit or stash first, then re-run.
+3. **Threshold check** — if `pessimisticUsd` ≤ the workspace's
+   `full_compile_approval_threshold_usd` (default $2.00), the run
+   starts autonomously.
+4. **Approval** — if `pessimisticUsd` > the threshold, the run is
+   queued under the **Approvals** panel on the dashboard home (`/`).
+   The acknowledgement DM links there; approve from the dashboard to
+   start the compile.
+5. **Completion** — a completion DM lands with the actual spend (not
+   the estimate). Significant overshoots are flagged in the audit
+   row.
+
+`!compile` (incremental) does not go through the cost gate — it's
+only invoked when raw notes have changed since the last compile, and
+its budget envelope is bounded by the per-process `maxBudgetUsd`
+(default $5.00). The same applies to `!ingest`, `!ask`, `!lint`,
+`!trace`, `!connect`.
+
+## Changing the threshold
+
+Per workspace, in `/settings/wiki` — the **Approval threshold (USD)**
+input. The value lives on `wiki_workspaces.full_compile_approval_threshold_usd`
+with a CHECK constraint `> 0`. Setting it lower escalates more runs
+to manual approval; setting it higher trusts the estimator more.
+
+A common pattern: start at $2.00, watch the spend on the next
+`!compile full`, then dial up or down based on how close the
+estimate landed to the actual cost.
+
+## The pre-compile git snapshot
+
+Only meaningful on an **external + git-tracked** vault. The gate
+decides:
+
+| Workspace state | Outcome |
+|---|---|
+| Internal mode | Skip (`NotApplicable`). The `md_file_snapshots` mechanism is the recovery surface. |
+| External, not a git repo | Skip (`NoBackup`). The approval-gate DM tells you no git backup was taken. |
+| External + git, `git_pre_compile_enabled = 0` | Skip (`Disabled`). |
+| External + git, **dirty** working tree | **Refused.** The bang handler aborts with a DM telling you to commit or stash first; no agent session is spawned. |
+| External + git, clean working tree | Run `git add -A` + `git commit -m "aitne wiki: pre-compile snapshot <ts>"`. Operator git hooks fire as normal (no `--no-verify`). |
+
+The snapshot is your rollback target if `!compile full` produces a
+surprise — `git reset --hard <snapshot-commit>` puts the vault back.
+
+The pre-compile commit uses no special author identity; it lands
+under whatever git is configured to use. If you co-author with
+Aitne, you'll want to set that up at the git config level.
+
+## The `--preview` dry-run
+
+`!compile --preview` (alias: `--dry-run`) calls
+`GET /api/wiki/:workspace/compile/preview` to show what `!compile`
+would do without spending tokens:
+
+- **added** — wiki pages that would be created (raws with no
+  existing wiki match).
+- **modified** — wiki pages that would be rewritten (raws whose
+  slug matches an existing wiki page).
+- **unchanged** — wiki pages the compile is expected to skip.
+- **est. cost** — Optimistic / pessimistic bracket plus expected
+  spend.
+- **est. duration** — Rough wall-clock estimate (intentionally
+  pessimistic).
+
+The preview is an upper bound for the same reason the cost bracket is
+(the compile is an LLM and may merge or skip pages mid-loop), but it
+will not undercount the touch set. Reply `!compile` (or `!compile
+full`) to actually run.
+
+The preview is free — no agent session runs, no tokens spent.
+
+## See also
+
+- [features/wiki/commands](commands.md) for the user-facing reference
+  of every wiki bang command.
+- [features/wiki/workspaces](workspaces.md) for where the per-workspace
+  threshold and git toggle live in `/settings/wiki`.
+- [features/operations/approvals](../operations/approvals.md) for the
+  general dashboard approval flow.
+- [guides/budget-and-cost-for-wiki](../../guides/budget-and-cost-for-wiki.md)
+  for the operator-level "tune the budget" walkthrough.

package/agent-assets/docs/features/wiki/dashboard.md

@@ -0,0 +1,255 @@
+---
+schema_version: 1
+slug: features/wiki/dashboard
+title: Wiki Dashboard Surfaces
+id: wiki-dashboard
+aliases:
+  - wiki dashboard
+  - wiki ui
+  - wiki page
+  - wiki timeline page
+  - wiki settings page
+  - /wiki
+  - /wiki/timeline
+  - /settings/wiki
+category: features
+summary: |
+  Reference for the three dashboard surfaces the wiki feature renders:
+  `/wiki` (workspace summary + index + recent activity), `/wiki/timeline`
+  (full chronological log + latest health report), and `/settings/wiki`
+  (configuration). Explains what each card shows, where the data comes
+  from on disk, and what to do when a card is empty.
+section: wiki
+tags:
+  - wiki
+  - dashboard
+  - reference
+  - core
+status: stable
+ask_examples:
+  - What does the /wiki page show?
+  - Where do I find the wiki health report?
+  - Why is the Recent activity card empty?
+  - How do I see every !ingest run for a workspace?
+  - Where is the wiki configuration page?
+  - What is _index.md and where does the wiki page render it?
+  - How is /wiki different from /wiki/timeline?
+  - What does the Enable Wiki button do?
+locale: en-US
+created: 2026-05-21
+updated: 2026-05-28
+keywords:
+  - /wiki
+  - /wiki/timeline
+  - /settings/wiki
+  - wiki dashboard
+  - workspace summary card
+  - _index.md
+  - log.md
+  - health report
+  - 90_meta/health
+  - recent activity
+  - timeline filter
+  - Enable Wiki CTA
+  - vault path picker
+related:
+  - features/wiki/overview
+  - features/wiki/commands
+  - features/wiki/workspaces
+  - features/wiki/cost-and-approval
+  - guides/build-your-wiki
+  - guides/maintain-wiki-health
+  - guides/use-an-existing-obsidian-vault
+ui_anchors:
+  - /wiki
+  - /wiki/timeline
+  - /settings/wiki
+api_endpoints:
+  - /api/wiki/workspaces
+  - /api/wiki/:workspace/index
+  - /api/wiki/:workspace/files/log.md
+  - /api/wiki/:workspace/health
+  - /api/wiki/:workspace/estimate
+  - /api/wiki/:workspace/compile/preview
+process_keys:
+  - wiki.ask
+  - wiki.compile
+  - wiki.ingest_url
+  - wiki.lint
+  - wiki.trace
+  - wiki.connect
+---
+
+# Wiki Dashboard Surfaces
+
+The wiki has three distinct pages in the dashboard. The split mirrors
+Aitne's broader IA: content browsing lives under **My Life**,
+configuration lives under **Setup → Settings**. The pages share a
+single read path through the daemon's `/api/wiki/*` routes — every
+request carries an `x-process-key: wiki.ask` header so the safety
+layer can attribute reads correctly.
+
+## `/wiki` — workspace home
+
+The page you open most often. Three cards stacked top-down:
+
+### 1. Workspace summary
+
+A single card with the workspace name, root path, kind badge
+(`Internal` / `External`), language badge, and a stats table:
+
+| Stat | Source |
+|---|---|
+| Raw notes | Count of files under `10_raw/` |
+| Wiki pages | Count of files under `20_wiki/` |
+| Outputs | Count of files under `30_outputs/` |
+| Last ingest | `wiki_workspaces.last_ingest_at` |
+| Last compile | `wiki_workspaces.last_compile_at` |
+
+Two action buttons at the bottom of the card: **Timeline & health**
+(jumps to `/wiki/timeline`) and **Configuration** (jumps to
+`/settings/wiki`).
+
+### 2. Index
+
+Renders the latest `20_wiki/_index.md`, the LLM-maintained catalogue
+of wiki pages. The agent rewrites this file at the end of every
+`!compile` run.
+
+States:
+- **Empty** — no `_index.md` yet; the CTA tells you to run `!compile`
+  from a DM.
+- **Loaded** — the file is rendered verbatim as a monospaced code
+  block (the index uses wikilink syntax, so rendering as markdown
+  would lose information).
+
+### 3. Recent activity
+
+The last 8 entries from `log.md`, the wiki's append-only operational
+log. Each entry shows the wiki process key (`wiki.ingest_url`,
+`wiki.compile`, `wiki.ask`, `wiki.lint`, `wiki.trace`, `wiki.connect`),
+the operation (`write`, `delete`, …), the affected path, and the
+timestamp. A **View full timeline** button at the top of the card
+opens `/wiki/timeline`.
+
+A freshly-enabled wiki has no `log.md` yet — the route returns 404
+and the card shows the "No activity yet" hint instead of a scary
+error toast.
+
+### Disabled state
+
+When no `active=1` workspace exists in `wiki_workspaces`, the page
+shows an **Enable Wiki** CTA that jumps to `/settings/wiki`. The
+sidebar entry is always visible (gated only on the workspace
+catalogue being reachable), so this disabled state is reachable via
+the sidebar, a deep link, or browser history.
+
+## `/wiki/timeline` — full chronological history + health
+
+Two surfaces stacked on one page:
+
+### 1. Latest health report
+
+Renders the newest `90_meta/health/<YYYY-MM-DD>.md` produced by
+`!lint`. The card shows:
+
+- A purple date badge and the source file path.
+- A `## Summary` block (bulleted list).
+- An `## Action items` list (the punch-list of orphan notes, broken
+  wikilinks, schema drift, taxonomy candidates, and stale-note
+  warnings).
+- A collapsible **View full report** button that reveals the raw
+  Markdown body of the report.
+
+Empty state: "No health reports yet — send `!lint` from a DM to
+generate the first one."
+
+### 2. Activity timeline
+
+A reverse-chronological view of `log.md` with a process-key filter.
+The filter dropdown lists every distinct `wiki.*` key found in the
+log plus an **All commands** default. Entries render the same way
+they do in the `/wiki` recent-activity card — process-key badge,
+operation, path, timestamp.
+
+Both surfaces read live from the wiki API; the timeline page is a
+pure rendering pass over files the wiki skills already produce — no
+extra daemon-side schema backs them.
+
+## `/settings/wiki` — configuration
+
+The configuration surface. Two-state:
+
+### Disabled (no active workspace)
+
+A two-card chooser:
+
+- **Internal** (recommended) — managed by Aitne in its data directory
+  (default `~/.personal-agent/wiki`), schema seeded automatically.
+  The **Enable internal wiki** button turns it on with nothing else
+  to configure.
+- **Existing Obsidian vault** (external) — point Aitne at a folder you
+  already own via the path picker, then confirm with **Use this
+  folder**.
+
+The path picker opens your OS-native folder dialog (Finder on macOS,
+File Explorer on Windows, the system folder dialog on Linux) and shows
+an inline validation banner once you pick a path:
+
+- Path-collision rules — the external root must not overlap
+  `dataDir`, your primary Obsidian vault, or another wiki workspace.
+- Existing-Obsidian-vault detection — when the picker finds an
+  Obsidian vault at the target, the wiki layout is detected and
+  migrated on demand so you can import the vault's content (see
+  [guides/use-an-existing-obsidian-vault](../../guides/use-an-existing-obsidian-vault.md)
+  and [features/wiki/workspaces](workspaces.md)).
+
+### Enabled
+
+The full configuration:
+
+- **Workspace** — name, kind (internal / external), root path, language.
+- **Dispatch mode** for `!ingest` (Parallel / Serial) and the
+  concurrency cap.
+- **Write strategy** (`fs` / `cli` / `auto`) — internal workspaces
+  always use `fs`; external workspaces start in `auto` and probe on
+  first write (see [features/wiki/workspaces](workspaces.md)).
+- **Git auto-commit before `!compile full`** toggle (only meaningful
+  on a git-tracked external vault).
+- **Approval threshold (USD)** for `!compile full`.
+- **Commands & models** — per-command selectors for the backend,
+  model, turn limit, and per-run budget on each `wiki.*` process key
+  (`wiki.ingest_url`, `wiki.compile`, `wiki.ask`, `wiki.lint`,
+  `wiki.trace`, `wiki.connect`). All six default to the medium tier
+  (Claude Sonnet 4.6) with a sensible `maxTurns` / `maxBudgetUsd`; you
+  can override per key.
+- **Archive / delete** — archive keeps the row but flips `active=0`;
+  delete drops the row (data on disk is untouched on external mode).
+
+The top of the enabled page carries a **Browse wiki** link that jumps
+back to `/wiki`.
+
+## Where each surface reads from
+
+| Surface | API route | On-disk source |
+|---|---|---|
+| Workspace summary | `GET /api/wiki/workspaces` | `wiki_workspaces` table |
+| Index card | `GET /api/wiki/:ws/index` | `20_wiki/_index.md` |
+| Recent activity | `GET /api/wiki/:ws/files/log.md` | `log.md` |
+| Health report | `GET /api/wiki/:ws/index` + `/files/...` | `90_meta/health/<date>.md` |
+| Activity timeline | `GET /api/wiki/:ws/files/log.md` | `log.md` |
+| Settings | `GET /api/wiki/workspaces`, `PATCH /api/wiki/workspaces/:ws` | `wiki_workspaces` table |
+
+Every wiki API request is gated by the `x-process-key` header. The
+dashboard uses `wiki.ask` as the closest read-only intent; the
+auth layer accepts any `wiki.*` key for GETs (see
+`authorizeWikiRequest` in `packages/daemon/src/api/routes/wiki.ts`).
+
+## Contextual help
+
+Every wiki page in the dashboard exposes a `?` Help button in the
+top-right action strip. Clicking it opens the relevant wiki doc in a
+slide-over for in-context reading (mirrors the global help pattern
+used everywhere else in the dashboard). `/wiki/timeline` opens this
+doc; `/wiki` opens [features/wiki/overview](overview.md) and
+`/settings/wiki` opens [features/wiki/workspaces](workspaces.md).