pi-vault-mind 0.7.5 → 0.8.0

package/README.md

@@ -87,7 +87,7 @@ obsidian plugin:install id=actions-uri enable
 obsidian plugin:install id=shellcommands enable
 ```
 
-Full walkthrough: see [`docs/OBSIDIAN_SETUP.md`](docs/OBSIDIAN_SETUP.md).
+Full walkthrough: see [`docs/integrations/OBSIDIAN_SETUP.md`](docs/integrations/OBSIDIAN_SETUP.md).
 
 ### Recommended: Obsidian Community Plugins
 
@@ -182,7 +182,7 @@ The Broadcaster agent can generate podcasts, study guides, and slide decks from
 - **Embedding Provider** — choose one:
   - `@xenova/transformers` — built-in, no external deps (uses all-MiniLM-L6-v2, offline-capable)
   - `ollama` — requires Ollama running locally with `embeddinggemma` (higher quality)
-  - `modal` — optional, **bring-your-own deploy**: offload embedding + bulk re-index to a cloud GPU service and sync vectors down for offline search (see [docs/MODAL_EMBEDDING.md](docs/MODAL_EMBEDDING.md)). Default behavior is unchanged until you opt in.
+  - `modal` — optional, **bring-your-own deploy**: offload embedding + bulk re-index to a cloud GPU service and sync vectors down for offline search (see [docs/integrations/MODAL_EMBEDDING.md](docs/integrations/MODAL_EMBEDDING.md)). Default behavior is unchanged until you opt in.
 
 > **On the Modal provider:** the local providers above work for everyone with
 > zero infrastructure — that's the default. `modal` is an **optional, self-hosted
@@ -371,21 +371,21 @@ Edit `pi-vault-mind.config.json` to match your domain:
 
 | Doc | Description |
 |---|---|
-| [docs/QUICKSTART.md](docs/QUICKSTART.md) | **Fastest path** — 60-second install, setup, first append |
-| [docs/NEW_VAULT_WALKTHROUGH.md](docs/NEW_VAULT_WALKTHROUGH.md) | **Fresh Mac → new vault → Modal → sync → verified**, one linear path (incl. multi-vault storage isolation) |
-| [docs/INSTALL.md](docs/INSTALL.md) | **Canonical install playbook** — all 5 layers (pi ext, skills, Obsidian, config, external CLIs) |
-| [docs/GETTING_STARTED.md](docs/GETTING_STARTED.md) | End-to-end workflow + daily "drop & forget" usage |
-| [docs/WALKTHROUGH_PROMPT.md](docs/WALKTHROUGH_PROMPT.md) | Paste-into-pi guided setup with checkpoints between phases |
+| [docs/getting-started/QUICKSTART.md](docs/getting-started/QUICKSTART.md) | **Fastest path** — 60-second install, setup, first append |
+| [docs/getting-started/NEW_VAULT_WALKTHROUGH.md](docs/getting-started/NEW_VAULT_WALKTHROUGH.md) | **Fresh Mac → new vault → Modal → sync → verified**, one linear path (incl. multi-vault storage isolation) |
+| [docs/getting-started/INSTALL.md](docs/getting-started/INSTALL.md) | **Canonical install playbook** — all 5 layers (pi ext, skills, Obsidian, config, external CLIs) |
+| [docs/getting-started/GETTING_STARTED.md](docs/getting-started/GETTING_STARTED.md) | End-to-end workflow + daily "drop & forget" usage |
+| [docs/getting-started/WALKTHROUGH_PROMPT.md](docs/getting-started/WALKTHROUGH_PROMPT.md) | Paste-into-pi guided setup with checkpoints between phases |
 
 ### Architecture & design
 
 | Doc | Description |
 |---|---|
-| [docs/AGENTS.md](docs/AGENTS.md) | Agent Roster and Multi-Agent Architecture ("Fork & Review" model) |
-| [docs/EXTENSION_WIRING.md](docs/EXTENSION_WIRING.md) | Extension dependencies, runtime wiring, auto-install patterns |
-| [docs/DISPATCHER_SPEC.md](docs/DISPATCHER_SPEC.md) | Technical spec for the passive file-watcher and subagent routing — incl. the "Fork & Dispatch" rationale and thread resume |
-| [docs/AGENTS.md](docs/AGENTS.md) | Agent roster + the Miner's "Drop & Forget" ingestion pipeline |
-| [docs/OBSIDIAN_SETUP.md](docs/OBSIDIAN_SETUP.md) | Recommended Obsidian vault structure, plugins, and CLI |
+| [docs/architecture/AGENTS.md](docs/architecture/AGENTS.md) | Agent Roster and Multi-Agent Architecture ("Fork & Review" model) |
+| [docs/architecture/EXTENSION_WIRING.md](docs/architecture/EXTENSION_WIRING.md) | Extension dependencies, runtime wiring, auto-install patterns |
+| [docs/architecture/DISPATCHER_SPEC.md](docs/architecture/DISPATCHER_SPEC.md) | Technical spec for the passive file-watcher and subagent routing — incl. the "Fork & Dispatch" rationale and thread resume |
+| [docs/architecture/AGENTS.md](docs/architecture/AGENTS.md) | Agent roster + the Miner's "Drop & Forget" ingestion pipeline |
+| [docs/integrations/OBSIDIAN_SETUP.md](docs/integrations/OBSIDIAN_SETUP.md) | Recommended Obsidian vault structure, plugins, and CLI |
 
 ### Modal embedding service (local integration done)
 
@@ -395,12 +395,12 @@ fallback that degrades to FTS, never crashes), a sync-down path that pulls
 server-side vectors into the local LanceDB with a monotonic `seq` watermark,
 remote bulk re-index (`/wiki reindex --all --reembed --remote`), and debounced +
 batched append embedding via the coalescer. Existing non-modal users see no
-behavior change. See [docs/MODAL_EMBEDDING.md](docs/MODAL_EMBEDDING.md) for the
+behavior change. See [docs/integrations/MODAL_EMBEDDING.md](docs/integrations/MODAL_EMBEDDING.md) for the
 full design.
 
 | Doc | Description |
 |---|---|
-| [docs/MODAL_EMBEDDING.md](docs/MODAL_EMBEDDING.md) | Design of record: ADRs, HTTP contract, sync protocol, roadmap for the cloud embedding service |
+| [docs/integrations/MODAL_EMBEDDING.md](docs/integrations/MODAL_EMBEDDING.md) | Design of record: ADRs, HTTP contract, sync protocol, roadmap for the cloud embedding service |
 | [docs/plans/modal-embedding-plan.md](docs/plans/modal-embedding-plan.md) | **Consolidated plan & next steps** — phases, owners, status, decision gate (in repo, not on the docs site) |
 | [docs/plans/embedding-open-questions.md](docs/plans/embedding-open-questions.md) | Embedding-strategy decision log (decided + open questions) |
 | [modal/](modal/) | The deployable Modal app: embedding service + bulk worker + sync + dataset generator |
@@ -421,22 +421,22 @@ full design.
 
 | Doc | Description |
 |---|---|
-| [docs/TESTING.md](docs/TESTING.md) | Test plan for agent / human / HITL personas, regression suite |
-| [docs/E2E_MANUAL_TEST.md](docs/E2E_MANUAL_TEST.md) | Manual end-to-end verification procedure (watcher → dispatch → vault) |
+| [docs/testing/TESTING.md](docs/testing/TESTING.md) | Test plan for agent / human / HITL personas, regression suite |
+| [docs/testing/E2E_MANUAL_TEST.md](docs/testing/E2E_MANUAL_TEST.md) | Manual end-to-end verification procedure (watcher → dispatch → vault) |
 
 ### Development
 
 | Doc | Description |
 |---|---|
-| [docs/CONTRIBUTING.md](docs/CONTRIBUTING.md) | Dev setup, testing, and commit conventions |
-| [docs/dev/PUBLISHING.md](docs/dev/PUBLISHING.md) | How to publish this extension to npm |
-| [docs/dev/FUTURE_WORK.md](docs/dev/FUTURE_WORK.md) | Roadmap — codegraph integration, pagination, TUI rendering, etc. |
+| [docs/development/CONTRIBUTING.md](docs/development/CONTRIBUTING.md) | Dev setup, testing, and commit conventions |
+| [docs/development/PUBLISHING.md](docs/development/PUBLISHING.md) | How to publish this extension to npm |
+| [docs/development/FUTURE_WORK.md](docs/development/FUTURE_WORK.md) | Roadmap — codegraph integration, pagination, TUI rendering, etc. |
 
 ### Research
 
 | Doc | Description |
 |---|---|
-| [docs/COMPETITOR_COMPARISON.md](docs/COMPETITOR_COMPARISON.md) | Tier 1/2/3 comparison vs. other Obsidian-LLM tools (22 competitors) |
+| [docs/research/COMPETITOR_COMPARISON.md](docs/research/COMPETITOR_COMPARISON.md) | Tier 1/2/3 comparison vs. other Obsidian-LLM tools (22 competitors) |
 | [docs/research/naming-decisions.md](docs/research/naming-decisions.md) | Historical record of the 2026-06-06 decision to name the project `pi-vault-mind` |
 | [docs/research/obsidian-links-reviewed.csv](docs/research/obsidian-links-reviewed.csv) | Curated subset of starred Obsidian repos with adoption verdicts |
 
@@ -449,7 +449,7 @@ full design.
 
 ## Contributing
 
-See [docs/CONTRIBUTING.md](docs/CONTRIBUTING.md) for dev setup, testing, and commit conventions.
+See [docs/development/CONTRIBUTING.md](docs/development/CONTRIBUTING.md) for dev setup, testing, and commit conventions.
 
 ## License
 

package/dist/src/agent-queue.d.ts

@@ -0,0 +1,108 @@
+/**
+ * Agent dispatch queue.
+ *
+ * Persists every watcher dispatch as a job record so users can inspect what
+ * is pending, running, done, or failed, and retry failures. Jobs are stored
+ * append-only in a JSONL file inside the vault (`<vault>/.vault-mind/agent-queue.jsonl`).
+ *
+ * Status lifecycle:
+ *   pending → running → done
+ *                      → failed → (retry) → pending → ...
+ *             → cancelled (from pending only)
+ */
+export type AgentJobStatus = "pending" | "running" | "done" | "failed" | "cancelled";
+/**
+ * Event emitted by the queue whenever a job is created, updated, or
+ * reaches a terminal state. Subscribers receive a fresh, fully
+ * populated `AgentJob` (not a delta) so the WebSocket layer can
+ * ship a snapshot.
+ */
+export type QueueEvent = {
+    type: "job-created";
+    job: AgentJob;
+} | {
+    type: "job-updated";
+    job: AgentJob;
+} | {
+    type: "job-completed";
+    job: AgentJob;
+};
+/** Subscribe to queue events. Returns an unsubscribe function. */
+export declare const subscribeQueue: (cb: (e: QueueEvent) => void) => (() => void);
+export interface AgentJob {
+    /** Unique job ID (same as the dispatchId used for writeback). */
+    id: string;
+    /** Vault root path this job belongs to. */
+    vaultPath: string;
+    /** File that contained the @agent marker(s). */
+    filePath: string;
+    /** Role extracted from the marker (e.g. miner, broadcaster). */
+    role: string;
+    /** Resolved agent name (e.g. vault-mind-miner). */
+    agentName: string;
+    /** Combined instruction sent to the subagent. */
+    instruction: string;
+    /** Number of markers bundled into this job. */
+    markerCount: number;
+    /** If part of a batch, the batch id (shared with other jobs in the same dispatch). */
+    batchId?: string;
+    /** Current status in the queue. */
+    status: AgentJobStatus;
+    /** Number of dispatch attempts (including retries). */
+    attempts: number;
+    /** ISO timestamp when the job was created. */
+    createdAt: string;
+    /** ISO timestamp when the job was last dispatched (running). */
+    dispatchedAt?: string;
+    /** ISO timestamp when the job reached a terminal state. */
+    completedAt?: string;
+    /** Last error message if status is failed. */
+    lastError?: string;
+}
+/** List jobs, optionally filtered by status. */
+export declare const listJobs: (vaultPath: string, status?: AgentJobStatus) => AgentJob[];
+/** Get a single job by id. */
+export declare const getJob: (vaultPath: string, id: string) => AgentJob | undefined;
+/** Create a new pending job. */
+export declare const createJob: (vaultPath: string, job: Omit<AgentJob, "status" | "attempts" | "createdAt">) => AgentJob;
+/** Update a job's status and optional fields. Returns the updated job. */
+export declare const updateJob: (vaultPath: string, id: string, status: AgentJobStatus, updates?: Partial<AgentJob>) => AgentJob | undefined;
+/** Mark a running job as completed because its dispatch marker was replaced. */
+export declare const markJobDone: (vaultPath: string, id: string) => AgentJob | undefined;
+/** Mark a running job as failed with an error message. */
+export declare const markJobFailed: (vaultPath: string, id: string, error: string) => AgentJob | undefined;
+/** Mark a job as dispatched (running). Increments attempts and sets dispatchedAt. */
+export declare const markJobRunning: (vaultPath: string, id: string) => AgentJob | undefined;
+/** Mark a pending job as cancelled. */
+export declare const cancelJob: (vaultPath: string, id: string) => AgentJob | undefined;
+/** True if the file still contains the dispatch marker for this job. */
+export declare const markerStillPresent: (filePath: string, dispatchId: string) => boolean;
+/** Scan a vault's queue and mark any running job whose marker is gone as done. */
+export declare const sweepCompletedJobs: (vaultPath: string) => AgentJob[];
+/** Compact the queue JSONL by rewriting only the latest record per job. */
+export declare const compactQueue: (vaultPath: string) => void;
+/** Group pending jobs in the same vault+role. Excludes cancelled/done/failed. */
+export declare const listPendingByRole: (vaultPath: string) => Record<string, AgentJob[]>;
+/** Mark a list of jobs as running, sharing the same batchId. */
+export declare const markBatchRunning: (vaultPath: string, ids: string[], batchId: string) => void;
+/**
+ * Shared retry helper: validates the job is in a retryable state
+ * (failed or cancelled), marks it running, and returns the job so
+ * the caller can re-send the dispatch message. Returns null with a
+ * reason string when the job cannot be retried.
+ */
+export declare const retryJob: (vaultPath: string, id: string) => {
+    job: AgentJob;
+} | {
+    error: string;
+};
+/**
+ * Shared cancel helper: validates the job is pending, cancels it,
+ * and returns the updated job. Returns null with a reason when the
+ * job cannot be cancelled.
+ */
+export declare const cancelJobHelper: (vaultPath: string, id: string) => {
+    job: AgentJob;
+} | {
+    error: string;
+};

package/dist/src/agent-queue.js

@@ -0,0 +1,225 @@
+/**
+ * Agent dispatch queue.
+ *
+ * Persists every watcher dispatch as a job record so users can inspect what
+ * is pending, running, done, or failed, and retry failures. Jobs are stored
+ * append-only in a JSONL file inside the vault (`<vault>/.vault-mind/agent-queue.jsonl`).
+ *
+ * Status lifecycle:
+ *   pending → running → done
+ *                      → failed → (retry) → pending → ...
+ *             → cancelled (from pending only)
+ */
+import * as fs from "node:fs";
+import * as path from "node:path";
+const queueSubscribers = new Set();
+/** Subscribe to queue events. Returns an unsubscribe function. */
+export const subscribeQueue = (cb) => {
+    queueSubscribers.add(cb);
+    return () => queueSubscribers.delete(cb);
+};
+const emitQueueEvent = (e) => {
+    for (const cb of queueSubscribers) {
+        try {
+            cb(e);
+        }
+        catch {
+            // never let a bad listener crash the writer
+        }
+    }
+};
+const eventTypeFor = (status, prev) => {
+    if (!prev)
+        return "job-created";
+    if (status === "done" || status === "failed" || status === "cancelled")
+        return "job-completed";
+    return "job-updated";
+};
+const queueFileName = ".vault-mind/agent-queue.jsonl";
+const queuePath = (vaultPath) => path.join(vaultPath, queueFileName);
+const ensureQueueDir = (vaultPath) => {
+    const dir = path.dirname(queuePath(vaultPath));
+    if (!fs.existsSync(dir)) {
+        fs.mkdirSync(dir, { recursive: true });
+    }
+};
+/** Read all job records and keep the latest entry per job id. */
+const loadRecords = (vaultPath) => {
+    const p = queuePath(vaultPath);
+    if (!fs.existsSync(p))
+        return {};
+    const out = {};
+    for (const line of fs.readFileSync(p, "utf-8").split(/\r?\n/)) {
+        if (!line.trim())
+            continue;
+        try {
+            const rec = JSON.parse(line);
+            if (rec.id)
+                out[rec.id] = rec;
+        }
+        catch {
+            // skip corrupt line
+        }
+    }
+    return out;
+};
+const appendRecord = (vaultPath, job) => {
+    ensureQueueDir(vaultPath);
+    fs.appendFileSync(queuePath(vaultPath), `${JSON.stringify(job)}\n`, "utf-8");
+};
+/** List jobs, optionally filtered by status. */
+export const listJobs = (vaultPath, status) => {
+    const records = Object.values(loadRecords(vaultPath));
+    if (!status)
+        return records;
+    return records.filter((j) => j.status === status);
+};
+/** Get a single job by id. */
+export const getJob = (vaultPath, id) => loadRecords(vaultPath)[id];
+/** Create a new pending job. */
+export const createJob = (vaultPath, job) => {
+    const full = {
+        ...job,
+        status: "pending",
+        attempts: 0,
+        createdAt: new Date().toISOString(),
+    };
+    appendRecord(vaultPath, full);
+    emitQueueEvent({ type: "job-created", job: full });
+    return full;
+};
+/** Update a job's status and optional fields. Returns the updated job. */
+export const updateJob = (vaultPath, id, status, updates = {}) => {
+    const current = getJob(vaultPath, id);
+    if (!current)
+        return undefined;
+    const next = {
+        ...current,
+        ...updates,
+        id,
+        status,
+    };
+    if (status === "done" || status === "failed" || status === "cancelled") {
+        next.completedAt = new Date().toISOString();
+    }
+    appendRecord(vaultPath, next);
+    emitQueueEvent({ type: eventTypeFor(status, current.status), job: next });
+    return next;
+};
+/** Mark a running job as completed because its dispatch marker was replaced. */
+export const markJobDone = (vaultPath, id) => updateJob(vaultPath, id, "done");
+/** Mark a running job as failed with an error message. */
+export const markJobFailed = (vaultPath, id, error) => updateJob(vaultPath, id, "failed", { lastError: error });
+/** Mark a job as dispatched (running). Increments attempts and sets dispatchedAt. */
+export const markJobRunning = (vaultPath, id) => {
+    const current = getJob(vaultPath, id);
+    if (!current)
+        return undefined;
+    return updateJob(vaultPath, id, "running", {
+        attempts: current.attempts + 1,
+        dispatchedAt: new Date().toISOString(),
+        completedAt: undefined,
+        lastError: undefined,
+    });
+};
+/** Mark a pending job as cancelled. */
+export const cancelJob = (vaultPath, id) => {
+    const current = getJob(vaultPath, id);
+    if (!current || current.status !== "pending")
+        return undefined;
+    return updateJob(vaultPath, id, "cancelled");
+};
+/** True if the file still contains the dispatch marker for this job. */
+export const markerStillPresent = (filePath, dispatchId) => {
+    try {
+        const content = fs.readFileSync(filePath, "utf-8");
+        return content.includes(`<!-- pi-dispatch:${dispatchId} -->`);
+    }
+    catch {
+        return false;
+    }
+};
+/** Scan a vault's queue and mark any running job whose marker is gone as done. */
+export const sweepCompletedJobs = (vaultPath) => {
+    const completed = [];
+    for (const job of listJobs(vaultPath, "running")) {
+        if (!markerStillPresent(job.filePath, job.id)) {
+            const updated = markJobDone(vaultPath, job.id);
+            if (updated)
+                completed.push(updated);
+        }
+    }
+    return completed;
+};
+/** Compact the queue JSONL by rewriting only the latest record per job. */
+export const compactQueue = (vaultPath) => {
+    const p = queuePath(vaultPath);
+    const records = Object.values(loadRecords(vaultPath));
+    if (records.length === 0) {
+        if (fs.existsSync(p))
+            fs.rmSync(p, { force: true });
+        return;
+    }
+    fs.writeFileSync(p, `${records.map((r) => JSON.stringify(r)).join("\n")}\n`, "utf-8");
+};
+/** Group pending jobs in the same vault+role. Excludes cancelled/done/failed. */
+export const listPendingByRole = (vaultPath) => {
+    const out = {};
+    for (const job of listJobs(vaultPath, "pending")) {
+        const key = `${vaultPath}\u0000${job.role}\u0000${job.agentName}`;
+        if (!out[key])
+            out[key] = [];
+        out[key].push(job);
+    }
+    return out;
+};
+/** Mark a list of jobs as running, sharing the same batchId. */
+export const markBatchRunning = (vaultPath, ids, batchId) => {
+    for (const id of ids) {
+        const current = getJob(vaultPath, id);
+        if (!current)
+            continue;
+        updateJob(vaultPath, id, "running", {
+            attempts: current.attempts + 1,
+            dispatchedAt: new Date().toISOString(),
+            completedAt: undefined,
+            lastError: undefined,
+            batchId,
+        });
+    }
+};
+/**
+ * Shared retry helper: validates the job is in a retryable state
+ * (failed or cancelled), marks it running, and returns the job so
+ * the caller can re-send the dispatch message. Returns null with a
+ * reason string when the job cannot be retried.
+ */
+export const retryJob = (vaultPath, id) => {
+    const job = getJob(vaultPath, id);
+    if (!job)
+        return { error: `Job ${id} not found.` };
+    if (job.status !== "failed" && job.status !== "cancelled") {
+        return { error: `Job ${id} is ${job.status}. Only failed or cancelled jobs can be retried.` };
+    }
+    const updated = markJobRunning(vaultPath, id);
+    if (!updated)
+        return { error: `Failed to mark job ${id} as running.` };
+    return { job: updated };
+};
+/**
+ * Shared cancel helper: validates the job is pending, cancels it,
+ * and returns the updated job. Returns null with a reason when the
+ * job cannot be cancelled.
+ */
+export const cancelJobHelper = (vaultPath, id) => {
+    const job = getJob(vaultPath, id);
+    if (!job)
+        return { error: `Job ${id} not found.` };
+    if (job.status !== "pending") {
+        return { error: `Job ${id} is ${job.status}. Only pending jobs can be cancelled.` };
+    }
+    const cancelled = cancelJob(vaultPath, id);
+    if (!cancelled)
+        return { error: `Failed to cancel job ${id}.` };
+    return { job: cancelled };
+};

package/dist/src/auth.d.ts

@@ -0,0 +1,30 @@
+/**
+ * Bearer-token auth for the extension's HTTP/WS server.
+ *
+ * Token resolution order:
+ *   1. `process.env.PVM_API_TOKEN`
+ *   2. `<homedir>/.pi/agent/vault-mind.env` (key=value, # comments)
+ *   3. `wiki.embedding.modal.apiToken` in the loaded config
+ *
+ * Returns `null` if no token is configured (auth is off until set).
+ *
+ * Cross-platform: `os.homedir()` (never `~/`); `chmod 600` on the
+ * dotenv file is the caller's responsibility and is a no-op on
+ * win32 (the file system doesn't honor POSIX modes there).
+ */
+import type { UniversalConfig } from "./types.js";
+/** Resolve the bearer token. `cfg` is optional; pass the loaded config to fall back to it. */
+export declare const resolveAuthToken: (cfg?: UniversalConfig) => string | null;
+export interface IncomingLike {
+    headers: {
+        authorization?: string | string[] | undefined;
+    };
+}
+export interface AuthResult {
+    ok: boolean;
+    reason?: "missing" | "malformed" | "mismatch";
+}
+/** Authorize a request against the resolved token. Returns `ok: true` if auth is off (no token configured). */
+export declare const authoriseRequest: (req: IncomingLike, cfg?: UniversalConfig) => AuthResult;
+/** Test-only: forget the cached token so tests get a fresh resolve on each call. */
+export declare const _resetAuthCache: () => void;

package/dist/src/auth.js

@@ -0,0 +1,93 @@
+/**
+ * Bearer-token auth for the extension's HTTP/WS server.
+ *
+ * Token resolution order:
+ *   1. `process.env.PVM_API_TOKEN`
+ *   2. `<homedir>/.pi/agent/vault-mind.env` (key=value, # comments)
+ *   3. `wiki.embedding.modal.apiToken` in the loaded config
+ *
+ * Returns `null` if no token is configured (auth is off until set).
+ *
+ * Cross-platform: `os.homedir()` (never `~/`); `chmod 600` on the
+ * dotenv file is the caller's responsibility and is a no-op on
+ * win32 (the file system doesn't honor POSIX modes there).
+ */
+import * as fs from "node:fs";
+import * as os from "node:os";
+import * as path from "node:path";
+const ENV_VAR = "PVM_API_TOKEN";
+const DOTENV_REL = path.join(".pi", "agent", "vault-mind.env");
+/** Cached token resolution. The cache is invalidated by `setToken` / `clearToken`. */
+let cached = null;
+const readDotenvToken = () => {
+    const home = os.homedir();
+    if (!home)
+        return null;
+    const p = path.join(home, DOTENV_REL);
+    try {
+        const text = fs.readFileSync(p, "utf-8");
+        for (const line of text.split(/\r?\n/)) {
+            const m = line.match(/^\s*([A-Z_][A-Z0-9_]*)\s*=\s*("([^"]*)"|'([^']*)'|[^\s#]+)/);
+            if (!m)
+                continue;
+            if (m[1] !== ENV_VAR)
+                continue;
+            return m[3] ?? m[4] ?? m[2] ?? null;
+        }
+    }
+    catch {
+        // file missing or unreadable
+    }
+    return null;
+};
+const readConfigToken = (cfg) => {
+    return cfg?.wiki?.embedding?.modal?.apiToken;
+};
+/** Resolve the bearer token. `cfg` is optional; pass the loaded config to fall back to it. */
+export const resolveAuthToken = (cfg) => {
+    if (process.env[ENV_VAR])
+        return process.env[ENV_VAR] ?? null;
+    if (cached && cached.value !== null)
+        return cached.value;
+    const fromFile = readDotenvToken();
+    if (fromFile) {
+        cached = { value: fromFile };
+        return fromFile;
+    }
+    const fromCfg = readConfigToken(cfg);
+    if (fromCfg) {
+        cached = { value: fromCfg };
+        return fromCfg;
+    }
+    return null;
+};
+/** Constant-time compare to avoid timing attacks on the bearer token. */
+const safeEqual = (a, b) => {
+    if (a.length !== b.length)
+        return false;
+    let mismatch = 0;
+    for (let i = 0; i < a.length; i++) {
+        mismatch |= a.charCodeAt(i) ^ b.charCodeAt(i);
+    }
+    return mismatch === 0;
+};
+/** Authorize a request against the resolved token. Returns `ok: true` if auth is off (no token configured). */
+export const authoriseRequest = (req, cfg) => {
+    const expected = resolveAuthToken(cfg);
+    if (!expected)
+        return { ok: true };
+    const raw = req.headers.authorization;
+    const header = Array.isArray(raw) ? raw[0] : raw;
+    if (!header)
+        return { ok: false, reason: "missing" };
+    const m = header.match(/^Bearer\s+(.+)$/i);
+    if (!m)
+        return { ok: false, reason: "malformed" };
+    if (!safeEqual(m[1], expected))
+        return { ok: false, reason: "mismatch" };
+    return { ok: true };
+};
+/** Test-only: forget the cached token so tests get a fresh resolve on each call. */
+export const _resetAuthCache = () => {
+    cached = null;
+};

package/dist/src/auto-index.d.ts

@@ -0,0 +1,47 @@
+/**
+ * Background auto-indexing for collection directories.
+ *
+ * Watches the directories that contain configured collection JSONL files.
+ * When a collection file is created, modified, or deleted, a debounced
+ * reindex is triggered for that collection.
+ *
+ * Controlled by `wiki.autoIndex` (default false). Started alongside the HTTP
+ * server and stopped on session shutdown.
+ */
+import * as fs from "node:fs";
+import type { CollectionDef, UniversalConfig, WikiConfig } from "./types.js";
+export declare const DEFAULT_DEBOUNCE_MS = 2000;
+export interface AutoIndexState {
+    /** Whether the auto-index watcher is active. */
+    running: boolean;
+    /** Active fs.watch handles keyed by watched directory. */
+    watchers: Map<string, fs.FSWatcher>;
+    /** Pending per-collection debounce timers. */
+    debounceTimers: Map<string, ReturnType<typeof setTimeout>>;
+}
+export interface StartAutoIndexOptions {
+    /** Optional pre-created state object (useful for tests). */
+    state?: AutoIndexState;
+    /** Override the default 2s debounce. */
+    debounceMs?: number;
+    /** Override the default reindex implementation. */
+    reindex?: (dataDir: string, collection: string, def: CollectionDef, cfg: WikiConfig) => Promise<void>;
+}
+export declare function createAutoIndexState(): AutoIndexState;
+/**
+ * Start watching configured collection directories.
+ * Returns a stop function that closes watchers and clears timers.
+ */
+export declare function startAutoIndex(cfg: UniversalConfig, options?: StartAutoIndexOptions): {
+    stop: () => void;
+    state: AutoIndexState;
+};
+/**
+ * Stop all watchers and clear pending debounces.
+ */
+export declare function stopAutoIndex(state: AutoIndexState): void;
+/**
+ * Re-index a single collection by reading its JSONL source file and upserting
+ * each valid entry into LanceDB. Malformed lines are skipped.
+ */
+export declare const reindexCollection: (dataDir: string, collection: string, def: CollectionDef, cfg: WikiConfig) => Promise<void>;